Page 277 -

266 CHAPTER 10 Usability testing

engineering than traditional research (Wixon, 2003). Wixon’s assertion is that us-
ability testing, like engineering, is involved in building a successful product, in the
shortest amount of time, using the fewest resources, with the fewest risks, while op-
timizing trade-offs. Often in industry, schedule and resource issues, rather than theo-
retical discussions of methodology, drive the development process (Wixon, 2003).
One practice that is somewhat accepted in usability testing is to modify the interface
after every user test, when major flaws are discovered, to help immediately elimi-
nate the flaws and improve the interface. Making immediate changes to the interface
allows for those changes to be evaluated during the next user test, which can help
ensure that no new interface problems have been introduced in making the changes
(Wixon, 2003). While this may not happen due to time constraints, it is an acceptable
practice. Clearly, this practice would be considered unacceptable in experimental
design, where the goal would be to ensure that all users in a group have the same
treatment. And since usability testing is an industrial, practical approach, it is also
important to note that not all interface flaws discovered during usability testing are
fixed. Very often, the list of interface flaws discovered is prioritized and only the
most serious problems are fixed.
By now, it should be clear that the goal of usability testing is to be practical and
have a major impact. Since the goal is often to improve interfaces and have an impor-
tant impact on the financial bottom line of a company, many companies don’t publish
their usability test findings, as they consider it confidential and a part of their com-
petitive advantage. There are, however, a number of documented cases of usability
testing that we have included in this chapter.
There are some similarities and some differences between usability testing, and the
ethnography and participatory design methods discussed in chapter 9. Ethnography
is more focused on understanding people, their groups, their processes, and their be-
liefs. Often, ethnographic methods are used as part of a systems development method
called participatory design (again, discussed in detail in Chapter 9). The end goal of
ethnography is simply understanding a group, an organization, or a problem, whereas
the end goal of participatory design is building a computer system. Usability testing
follows a similar pattern, with an end goal of improved interface design in a specific
system. In fact, participatory design includes the stages of both ethnographic obser-
vation (in the user’s situational context) and usability testing. Development meth-
ods or lifecycles, such as participatory design, the systems development lifecycle,
the web development lifecycle, or community-centered design, can be thought of as
recipes, with the individual activities, such as ethnographic observation and usability
testing, as the ingredients in those recipes. The methods used in usability testing
borrow most closely from experimental design and ethnography. Table 10.1 provides
a comparison of classical research methods (such as experimental design and eth-
nography) and usability testing. Again, it is important to note that while many of the
same approaches from classical research can be utilized in usability testing, they are
often implemented differently, with different end goals.
To make things a bit more confusing, there is also research about usability
testing! That is, research exists on evaluating which usability testing methods are