Page 291 -

280 CHAPTER 10 Usability testing

representative user population is not within easy traveling distance of the us-
ability evaluators or moderators. Or there could be other logistical factors that
limit the ability to do face-to-face usability testing. For instance, it could be
appropriate to do remote usability testing with individuals with disabilities for
whom transportation might be a problem (see Chapter 16). When an interface
will be deployed in multiple countries, it is necessary to have users from all
of the countries test the interface but it may not be possible for the evaluators
to visit all the countries (Dray and Siegel, 2004). For doing usability testing
involving children, the familiarity with the testing location is only one of the
potential logistical concerns (also including topics such as how to get consent
for children to participate). For those who are doing usability testing involving
children, it is suggested to consult a thorough guide to children and HCI, such
as Fails et al. (2012) or Hourcade (2007).
Remote usability testing is typically where users are separated from the evalu-
ators by space, time, or both (Andreasen et al., 2007). Video, audio, and network
connections allow evaluators to monitor users, including streaming output from the
user’s screen and clickstream data. While this used to be done using videoconfer-
encing and private networks (Dray and Siegel, 2004), now, it is just as likely that a
web-based remote usability testing tool (such as UserZoom) is utilized for remote
testing. One of the challenges with remote testing is the difficulty (with synchro-
nous videoconferencing) or impossibility (with many web-based usability testing
tools which offer limited video and audio) of picking up nonverbal and interper-
sonal cues. Also with asynchronous remote testing, it is hard (or impossible) to pro-
vide instructions when things “go wrong,” you can’t ask any probing questions, and
you often miss the context of what was happening. To offset these drawbacks, there
are many benefits of remote usability testing, such as easy access to more poten-
tial participants (since you are not geographically limited), and easy collection and
analysis of clickstream data (which can easily be turned into graphs and heatmaps).
Remote usability testing, on the whole, works better for summative testing, when
you are more interested in quantitative metrics, than for formative testing, where you
tend to be more interested in the qualitative observations (Dray and Siegel, 2004).
In addition, synchronous remote usability testing, where the evaluators are observ-
ing the users in different locations at the same time using videostreaming, may be
more effective than asynchronous testing, where the evaluator observes at a later
time (Andreasen et al., 2007). While remote usability testing can be technically very
challenging and small problems can delay testing (since moderators aren’t there to
address any technical problems), it can be a very useful technique in the toolbox of
the usability evaluator. Table 10.4 displays benefits and drawbacks of remote us-
ability testing.
Finding an appropriate place to do usability testing, and recruiting a suffi-
cient number of representative users, is always an ongoing challenge. The next
two sidebars describe how two of the leading technical companies, Google and
Yahoo, use innovative approaches for recruitment of participants for usability
testing.