Page 423 -

414 CHAPTER 14 Online and ubiquitous HCI research

“test-drive” before your main session. If you only have a limited amount of time
for each participant, you do not want to spend too much of it working on software
configuration.
Even if all of the technology works perfectly, you should be prepared for a differ-
ent experience when conducting online observational studies. The lack of physical
presence will make it harder to gauge participants' reactions, and even to know how
intently they are focusing on the task (Dray and Siegel, 2004). More bluntly, you
might have no way of knowing when users are surfing the web as opposed to attend-
ing to your questions. You might also find that contextual inquiries are somewhat
limited: webcams might do a great job of showing faces, but they will not help you
see anything that does not involve computer work, such as filing papers or reviewing
printed material. Finally, do not forget that any plans to record sessions should be
approved by your institutional review board or equivalent (see Chapter 15).
Online studies are not limited simply to web-conferencing systems. HCI research-
ers have experimented with other techniques designed to address shortcomings of
both in-person and online studies. One effort found that 3D virtual world simulation
of a usability lab provided some advantages over a web-conferencing-based usability
study (Madathil and Greenstein, 2011). Webcam-based eye trackers (see Chapter 13)
have also been used to remotely collect low-level interaction data as needed for us-
ability studies (Chynał and Szymański, 2011). Alternatively, you might consider sim-
pler, more low-tech approaches. Online reporting of critical usability incidents and
posting of usability problems to online forums have been shown to be effective for
identifying usability problems, although at a lower rate than in-person usability tests
(Andreasen et al., 2007; Bruun et al., 2009). Such studies also have the advantage
of being potentially asynchronous—you might ask participants to complete tasks at
their convenience, reporting usability problems as appropriate. As is often the case
with usability studies and expert reviews, providing specific predefined tasks may
help participants identify more usability problems (Bruun and Stage, 2012).
Remote online usability studies can be useful for recruiting and including hard-
to-find participants, such as individuals with disabilities who might have some dif-
ficulty in making the trip to a usability lab. Although this approach has been shown
to have some potential utility, technologies should be chosen carefully to fit the needs
of target populations (Petrie et al., 2006) (see Chapter 16 for further discussion of
HCI research involving people with disabilities). Remote studies also provide for
the possibility of software enhancements for people with specific disabilities, such
as sign-language facilities for use by deaf participants (Schnepp and Shiver, 2011).

14.2.2 ONLINE DATA COLLECTION
Online HCI techniques are not limited to usability and think-aloud studies. Online
surveys have become very familiar—see Chapter 5 for a discussion of the ins and
outs of conducting online surveys. Chapter 12 discusses two useful online research
techniques: the use of instrumented software to collect user interaction data and the
use of web logs to study how web sites are used.