Page 303 -

292 CHAPTER 10 Usability testing

now and then to describe what they are doing, if the user talks continuously for 10
minutes, clearly, the task performance time is of questionable use.
Since usability testing is a practical approach to solving problems, hybrid ap-
proaches are often used. In a reflection or interpretation session, users, immediately
after completing a series of tasks, review the raw video with the usability modera-
tors, and help interpret the interface problems (Frokjaer and Hornbæk, 2005). Even
without a formal interpretation session, users often make comments about the inter-
face during the debriefing which follows the usability testing session. Without being
prompted, users often make comments out loud during the usability testing session.
All feedback from users is important data!
In addition, qualitative data, in terms of observation by moderators, is very im-
portant. Moderators can often tell a lot about how participants are managing an in-
terface even when the participant is not saying anything. Participants may sigh or
grunt and their facial expressions may tell a story of frustration. It is possible to see
frustration or anger in the facial expressions of participants. In fact, certain muscle
movements in the face are clear signs of stress (Hazlett, 2006). Even without com-
plex interpretation, it is very probable that, if a user keeps moving towards the screen
or squinting, the icons or fonts on the screen may be a bit too small.

10.5.8 MAKING SENSE OF THE DATA
Analyzing data from usability testing is similar to analyzing data from any other
type of research. However, the goal of the analysis is different. Since usability test-
ing often uses fewer participants, inferential statistics often are not possible; but
simple descriptive statistics are possible. With traditional research, the goal is to
write up the results in a paper, publish it in a journal, conference proceedings, or
book, and help influence future research and design. With usability testing, the goal
is often to write up the results and help influence the design of the specific interface
that was tested. Sometimes, a presentation about the results is made to a group of
developers or managers who have the power to ensure that the interface is changed.
The usability testing report (or presentation) should be oriented towards the goal
of improving the specific interface and to those who will read it: interface design-
ers, software engineers, project managers, and other managers involved in software
development.
The usability test may have uncovered many different interface flaws that should
be addressed. However, due to time concerns, not all of these flaws will be improved
upon. So while the report should identify all flaws discovered during usability test-
ing, the report should also prioritize which ones are most important to fix. For each
flaw identified, the report should describe the problem, present the data from the
usability test, identify the priority of the flaw, suggest a fix, and also estimate the
time for the fix. Sometimes, data from usability testing can point to which flaws
caused users to lose the most time or be unable to complete their tasks and which
flaws were easily overcome by users. It is not always clear how to improve ev-
ery single flaw. Sometimes, you may improve upon one flaw but introduce other