Page 285 -

274 CHAPTER 10 Usability testing

and strict controls on user recruitment, usability testing may, in fact, become ex-
perimental design. The only difference would be that experimental design is looking
for statistically significant differences between groups to learn some research truth,
whereas usability testing is looking for ways to identify usability flaws and improve
specific interfaces.

10.5.2 STAGES OF USABILITY TESTING

Usability testing is not something that just happens. It requires a lot of advance plan-
ning. Different authors on the topic describe different steps, but the reality is that
there are a lot of advance planning steps involved. See Table 10.3 for examples of the
stages of usability testing from two different authors.

Table 10.3 Stages of Usability Testing From Different Authors
Stages of Usability Testing
(Rubin and Chisnell, 2008) (Lazar, 2006)
Develop the test plan Select representative users
Setup the test environment Select the setting
Find and select participants Decide what tasks users should perform
Prepare test materials Decide what type of data to collect
Conduct the test sessions Before the test session (informed consent, etc.)
Debrief the participants During the test session
Analyze data and observations Debriefing after the session
Report findings and recommendations Summarize results and suggest improvements

There are a number of stages of usability testing that seem very similar to experi-
mental design (see Chapter 3). Often, a usability expert, taking the role of the usability
moderator, manages the process. For more detailed information about moderator roles,
we suggest that you consult Dumas and Loring (2008). The moderator should deter-
mine which users would be appropriate, representative participants to take part in the
usability testing. If the typical users of the new interface system are nurses at a hospital,
it is inappropriate (and probably unethical) to use undergraduate students in business to
perform the usability testing (although nursing students might be appropriate, depend-
ing on the level of domain knowledge and job experience required). If appropriate user-
centered design methods have been utilized, there should be existing user personas
and task scenarios that can help guide you in this process. Some of the most common
criteria for determining representativeness of users, include age, gender, education, job
responsibility and or/domain expertise, technical experience (in general), and experi-
ence with specific software or hardware devices (Tullis and Albert, 2008).
Once you have figured out who the representative, appropriate users are, the next
goal is to try and recruit them. Again, this is very similar to experimental design.
For instance, users expect to be paid for their participation in usability testing, just
as they expect to be paid for their participation in an experimental study. However,