Page 279 -

268 CHAPTER 10 Usability testing

An expert-based test involves interface experts in using a number of different
structured methods for finding interface flaws. An automated test is a software pro-
gram that applies a series of guidelines (developed by the experts) to an interface
and determines where the interface doesn’t meet the guidelines. A user-based test
involves representative users performing representative tasks (at various stages in
the development process). While user-based tests are the majority focus of usability
evaluation, expert-based tests and automated tests are sometimes used in human-
computer interaction (HCI) practice.
As multimethod research approaches gain strength, we expect to see a greater ap-
pearance of expert and automated usability testing. Note that expert and automated
usability tests are sometimes known as usability inspections, and usability testing is
reserved for user-based testing. Whole books have been written about each type of
usability testing, so this chapter provides only a summary of each type. Since the
primary interest in HCI research is users and collecting data from users, this chapter
primarily focuses on user-based testing. First, we briefly discuss expert-based testing
and automated testing.

10.4.1 EXPERT-BASED TESTING
Expert-based tests are essentially structured inspections by interface experts. The
people who developed the prototype interface being evaluated should not be in-
volved with the expert review, as that may bias the results. People who are unfamiliar
with the interface should carry out the expert reviews. Expert-based tests are often
used in conjunction with user-based tests, but the expert-based tests always come
first. Interface experts are experts in interfaces but they are typically not experts in
the tasks to be performed within a certain interface. Conversely, representative users
are typically experts in performing the tasks but are not experts in interface design.
Often a certain portion of interface functionality can be understood and improved
without a deep understanding of the tasks, but other portions of the interface can only
be examined with a deep understanding of the tasks involved.
Interface experts first use a structured inspection to attempt to uncover some of
the more obvious interface flaws, such as confusing wording, inconsistent or mis-
leading layouts, and color inconsistency. If possible, suggested improvements to the
interface from the expert review should be made before user-based usability testing
occurs. This timeline allows the experts to find the obvious interface flaws and get
them fixed; the users can then find the deeper, more granular, and task-related inter-
face flaws which may not be obvious to the interface experts (Lazar, 2006). If there
are many interface flaws and no expert has reviewed the interface, the users may be
distracted by the major interface flaws and may be unable to help the developers by
identifying the more granular, task-based flaws.
There are a number of different types of expert review, also known as expert in-
spections or usability inspections. The most common expert reviews are the heuristic
review, the consistency inspection, and the cognitive walkthrough. In a heuristic re-
view, an expert takes a set of heuristics (rules of thumb) and compares the heuristics