Page 304 -

10.6 Other variations on usability testing 293

problems. An experienced usability moderator may use their expertise to determine
which flaws should be prioritized, which flaws are not as problematic, and how to
make improvements which do not introduce new problems.
Rubin and Chisnell (2008) suggest splitting the report into three sections:
• why you did usability testing and how you prepared;
• what happened during the testing; and
• the implications and recommendations.
While typical research publications need to be thorough and detailed, if usability
testing reports are going to management, they should be short and to the point. If
certain aspects of the interface worked well, it might be useful to note that in the
report as well. When interface flaws are fixed and changes are made, new flaws can
be introduced into the interface. So it can be helpful to note the interface components
that worked well and should not be changed.
It is important to note that you should never include names or identifying infor-
mation for the participants who took part in the usability testing (Dumas and Loring,
2008). If all participants are from within a specific organization, even giving a com-
bination of age, gender, and job title could be the equivalent of identifying someone.
When in doubt, provide only the average age of participants, the number of each
gender who took part, and basic job titles. You never want to identify who took part
in the usability testing, so it’s a good idea to refer to the participants as Participant
#1, Participant #2, and so on. You never know to whom and where your usability re-
porting results will be sent to, so make sure you would be comfortable with that fact.

10.6 OTHER VARIATIONS ON USABILITY TESTING
This chapter has presented traditional ways of doing usability testing. But, since us-
ability testing is all about being practical and about changing methods to fit the needs
that you have in a project, of course, there are new and different approaches to us-
ability testing. If you read the proceedings of any well-established HCI conference,
you can find new approaches, new hybrids, combining multiple methods, that could
potentially be used in certain types of usability engineering activities. Two of the
more well-known approaches are “technology probes” and “Wizard-of-Oz testing.”
Technology probes wouldn’t technically be considered usability testing, but they
are certainly closer to usability testing than traditional research. A technology probe
is similar to a cultural probe (described in Chapter 8). However, a cultural probe
has the goal of generally learning more about people, their groups, and their life-
styles. Technology probes involve putting a technology into a real-world setting.
Technology probes combine the social science goal of collecting information about
people in a real-world setting, the engineering goal of evaluating a new technology,
and the design goal of creating new ideas for potential technologies (Hutchinson
et al., 2003). A technology is installed in a real-world setting to see how it is used and
then reflection on these experiences gives feedback on who the users are and what