Page 287 -

276 CHAPTER 10 Usability testing

By now, “five participants in usability evaluation” is part of the HCI lore, in the
same way that “7 ± 2 menu items” is part of the HCI lore. We are told that we should
organize our menu items and menu bars into chunks of five to nine items, based on
classic psychological research literature (Miller, 1956). However, this is misleading:
the 7 ± 2 limitation in short-term memory applies to recall, not recognition, and most
interface design (including menus) is recognition, where we see or hear an icon or
item and think, “oh yes, that’s what I wanted” (Preece et al., 2002 explained this well,
although their explanation hasn’t appeared in later editions of their book). However,
“five participants” and “7 ± 2 menu items” remain part of the HCI folklore, even
when there is real debate about their validity.
The reality is that most usability testing will never uncover all, or even most, of
the usability flaws. And even if all of the flaws were uncovered, most of them will
never be fixed. Instead, the goal should be to find the major flaws, the flaws that will
cause most problems, and get them fixed. From an industry point of view, the exer-
cise of finding flaws, without the consideration of whether they can be fixed, is not
of value (Wixon, 2003). It simply would not make sense to expend all of the available
“usability time” in a development lifecycle on finding flaws, rather than balancing
time between finding flaws and fixing flaws. It may be useful to examine the ef-
fectiveness of various usability testing methods. But in industry, usability testing
logistics are often driven not by what should or needs to be done, but instead, on how
much time is left in the development process, how much money has been set aside
by management for usability testing, and how many users are available and will-
ing to participate. For instance, in usability testing on the website of the American
Speech-Language-Hearing Association, the usability engineer identified 16 different
user populations for the website. But after the prototype of the new website was built,
the budget only allowed for usability testing with school-based, speech-language
pathologists, the largest group of users for the website (Lazar, 2006). So instead of
saying, “how many users must you have?,” maybe the correct question is “how many
users can we afford?,” “how many users can we get?” or “how many users do we
have time for?”

10.5.4 LOCATIONS FOR USABILITY TESTING
Usability testing can take place anywhere. It can take place in a fixed laboratory,
a workplace, a user’s home, over the phone, or over the web. The location may be
determined by what locations are available or where participants are, as well as what
type of data you want to collect. None of the types of location are superior to any
others. You should use whatever works for your specific usability testing project.
The most traditional setting for usability testing is a two-room setup. The user sits
in one room and works on the tasks on a computer. Microphones and cameras record
what the user is doing and output from the user’s computer screen is also recorded.
In the other room, the test moderators, and other stakeholders, sit and watch what
the user is doing during the test. The moderators’ room generally has a number of
computer screens and monitors and the recording equipment, so all appropriate data