There are many different methods of usability evaluation. One of the most popular methods is expert evaluation. The reasons for its frequent use are varied: Firstly, expert evaluation does not require a research infrastructure. Additionally, expert evaluation delivers very fast results and is independent of test subjects. Another advantage is the relatively low cost of conducting usability evaluation. Conducting an expert evaluation does not require tools, needs little preparation, and is very simple to execute.
So, what exactly is an expert evaluation? The answer is quite simple: An independent expert puts themselves in the user’s position as best as they can and evaluates the system under review. In the UX industry, two methods of expert evaluation have become established: heuristic evaluation and the cognitive walkthrough. I will now take a closer look at each of these.
The Cognitive Walkthrough: Put Yourself in Your User’s Shoes
The Cognitive Walkthrough is a task-based approach. The expert puts themselves in the user’s position to play through the most common and relevant use cases from the user’s perspective. A prerequisite for this method is that the target group is known and that the expert can empathize with it. To achieve this goal, tools such as personas, empathy maps, and the questions posed by Spencer (2000) can help:
- Will the user know what to do at this step?
- If the user has completed this step correctly, will they know if they have made the right move and are on the right path to achieving their goal?
The analysis of usability problems is based on the expert’s knowledge of usability issues in general, as well as the ability to empathize with the customer and knowledge about the customer themselves. If the following questions can be answered with a ‘yes’, a Cognitive Walkthrough can deliver good results.
- Does the expert have sufficient expertise in usability and can identify problems as such?
- Is the target group clearly and distinctly defined?
- Have personas, empathy maps, etc., been developed?
- Does the expert have the ability to empathize with the target group?
- Is the expert independent?
The last point is very important. If the expert is biased because they have to justify changes to their boss or because they have to do extra work, they may be prone to serious thinking errors that negatively affect the quality of the test results. This problem exists in almost all usability testing.
The Heuristic Evaluation: The Standardized Usability Test
Unlike the Cognitive Walkthrough just described, heuristic evaluation is a guideline-based approach and is based on criteria catalogs of guidelines by Nielsen and Molich (1994) or DIN EN ISO 9241-10 (1996). Nielsen’s 10 heuristics are as follows:
- Visibility of system status:
The system should always inform users about what is happening, through appropriate feedback within a reasonable time frame. - Match between the system and the real world:
The system should speak the user’s language, not with system-oriented terminology (Error 303xxx23Fea), but with words and phrases that the user understands. The logical order should be maintained. - User control and freedom:
Users often choose functions by mistake and need an “emergency exit” to leave the unwanted state. This is served by the system’s or browser’s “Back” function. - Consistency and standards:
Users should not have to wonder whether different terms or actions mean the same thing everywhere. Therefore, conventions and standards should be adhered to. - Error prevention:
Even better than good error messages is a careful design that prevents errors. - Recognition rather than recall:
Objects, options, and actions should be visible. Users should not have to remember information from an earlier part of the dialogue with the system. Instructions should be visible or easy to find. In other words: The system should be intuitively usable. - Flexibility and efficiency of use:
Frequently used actions should be customizable by the user to allow advanced users to operate more quickly. Shortcuts are very useful for a high level of usability. - Aesthetics and minimalist design:
Dialogues should not contain irrelevant information that distracts the user and causes the essentials to be lost. - Help users recognize, diagnose, and recover from errors:
Error messages should be expressed in natural language (no error codes), precisely describe the problem, and constructively suggest a solution. - Help and documentation:
Any information in the help or documentation should be easy to find, tailored to the task, and list the specific steps for the solution. It should also not be too long.
As mentioned earlier, both methods are referred to as expert evaluation. If used correctly, the method can uncover many problems and provide a good starting point to uncover the biggest weaknesses of a website or application. But this is precisely where the greatest difficulty of expert evaluation lies: its proper use.
Expert evaluation is often seen as a substitute for actual user tests, rather than as a complementary tool. Expert evaluation can uncover many problems, but it has the disadvantage that some problems may not be found, or problems are uncovered that are actually not present.
How is Expert Evaluation Correctly Used?
Expert evaluation is excellent for formulating hypotheses about usability and uncovering both major and minor problems. However, expert evaluation is NOT(!) a substitute for usability tests with real users, as usability experts cannot sufficiently represent the target group. With this limitation in mind, we can now consider the following points for the correct methodology:
- Use both the Cognitive Walkthrough and the Heuristic Evaluation. Both methods have their individual advantages and disadvantages. By using both, you can offset these to some extent.
- Usability experts should, if possible, not have previously come into contact with the system. Under no circumstances should a usability expert evaluate a system in which they were involved. Even if the expert works to the best of their knowledge and belief, they will subconsciously overlook errors.
- Use several experts to evaluate the system. This can also be 3 to 5 experts. However, these experts must perform the test independently of the other testers to prevent groupthink and its disadvantages (fear of social rejection/desire for social recognition). Only in the subsequent discussion