Page 301 -

290 CHAPTER 10 Usability testing

10.5.7 THE USABILITY TESTING SESSION
Before the testing session is scheduled, it is important to contact the participants,
remind them about the upcoming session, and confirm the location, regardless of
where the usability testing session will take place. Make sure to leave extra time in
your schedule, since the participants may show up late, or take longer than expected.
Immediately before the session starts, confirm that all computers, recording devices,
and other technologies are working properly.
Remember that while the goals of usability testing may be different from classical
research like experimental design or ethnography, the protection of human subjects are
exactly the same. Just as in any type of research, participants must be given notice of
their rights, agree if they are to be video- or audio-recorded, and be allowed to leave at
any time. At no point can participants be held against their will, or punished in any way.
Unless the participants have specifically given permission to do so, their participation
must remain anonymous—at no point can their participation be identified to the outside
world. Their data must be protected as in any other type of research method.
It is important to let the participants know if there are any time constraints, either
on the session as a whole, or for completing specific tasks. For more information
about human subjects protections and IRB forms, see Chapter 15. In usability testing,
when new interfaces are being tested, these interfaces might be confidential company
information. So participants may also be asked to sign some type of confidentiality
agreement, in which they agree not to release or discuss any details of this new in-
terface product (Dumas and Loring, 2008). Finally, it should be clarified before the
testing session begins whether participants will receive payment at the end of the ses-
sion or if a check (or a gift card or something similar) will be mailed to their home.
It should also be made clear to the participants that even if they cannot complete the
session or feel the need to end the session early, as is common practice, they will still
be paid for their participation.
As noted previously, usability testing is about finding flaws that can be fixed, not
about having a perfect methodology. One practice that is common in usability test-
ing is to modify the interface after every user test, to help immediately improve the
interface flaws discovered; those changes are then evaluated during the next user test
(Wixon, 2003). If changes aren’t made immediately after each user, changes may be
made to the interface after a few users, and then a second round of usability testing
is held, using the same tasks, to see if the performance improves and if the changes
improved the interface. See the “Usability Testing at Fidelity Investments” sidebar
for an example of this practice. Making changes after each user, while an interface
is still under development, is commonplace in usability testing. A newer approach to
usability testing is A/B testing, where minor tweaks are made in interfaces that are
already in daily use. So, for websites that are visited by thousands of users a day, us-
ers may receive versions that have slight differences in color, layout, terminology, or
other changes that might not be noticeable to the user, with data collected about pat-
terns of usage. After data is collected over perhaps a few weeks, the interface changes
that are deemed to be successful, increasing traffic, increasing sales, and reducing
costs are permanently rolled out.