Page 433 -

424 CHAPTER 14 Online and ubiquitous HCI research

approaches involving both in-person and online research may provide additional data
and avoid some of the downsides associated with each approach.
Online studies involving surveys, self-selected visits to web sites, crowdsourcing,
or other approaches that do not require synchronous interactions with researchers might
be subject to frequent dropouts, as users decide to start a task and then stop half-way
through. Study designs should anticipate such dropouts and consider how they might
be reported. If you are looking at task completion success rate, it is probably appropri-
ate to include all participants who started the task. If you are looking at task comple-
tion times, you might want to focus only on those who completed the tasks. Providing
numbers for those who started tasks, those who completed tasks, and indicating which
groups were considered for which analyses is probably most appropriate.
14.2.4.4 Ethical concerns
Although the usual guidelines regarding protection of participants apply to online
research, numerous confounding factors can create some interesting and challenging
dilemmas.
Studies of online communities must consider questions of privacy and online
consent. What is the expectation of privacy when participants in an online forum
post messages publicly? Are such messages fair game for researchers? Is informed
consent required before messages can be used? What if the site is only accessible to
users who register and login? These questions have generated debate, discussions,
and some guidelines (Bruckman, 2002; Frankel and Siang, 1999), but specific issues
vary from case to case. Researchers are urged to be particularly careful when ex-
ploring communities describing sensitive topics such as health. The trust needed for
participants to share stories of challenging personal times such as illnesses may lead
some users to forget that they are effectively participating in a public forum where
materials may be read by many individuals. Lurking in such communities or posing
as a member may not be seen as appropriate behavior. Before doing so, you might
consider talking to the organization or individuals responsible for the site and intro-
ducing your study to the group. Creating communities specifically for research pur-
poses can be a successful—if not always practical—alternative (Bruckman, 2002).
Informed consent and debriefing for online studies can also be tricky. Providing
important information for either of these tasks via online text may not be sufficient.
In-person studies provide the possibility of direct feedback: experimenters know if
participants have any questions or if there is any postexperiment distress. These fac-
tors are much harder to gauge online (Azar, 2000). Although one study indicated that
comprehension of informed consent forms online may be comparable to comprehen-
sion of forms on paper, poor recall in both cases illustrates the general challenge of
constructing effective consent forms (Varnhagen et al., 2005). These issues may be
even thornier for studies conducted retrospectively, through API access to posted
data or other methods allowed under web site terms of service. Although such studies
are not inappropriate, and may not require consent, it is still best to tread carefully.
When possible, provide clear and easily understandable descriptions of research
goals and implications. In any case, these studies should not be undertaken without