Page 494 -

486 CHAPTER 15 Working with human subjects

Human subjects research in HCI can be an unpredictable and often unsettling
process. Unforeseen problems, including misinterpreted tasks and goals, systems
failures, and missed appointments, are routine: it's rare that a study (of any sort)
goes off completely without a hitch. These matters can complicate data collection
and interpretation: if a user chooses an interpretation of a written task that differs
from your intent and then completes the task correctly, how do you interpret the re-
sult—is it correct or not? What should be done with results from a user who decides
to withdraw from a study after completing only a portion of the tasks? As hard and
fast rules for handling situations like these are few and far between, you may have to
handle each issue on a case-by-case basis. The specific decisions that you make may
be less important than how they are enforced: consistent application of policies and
procedures will ensure your ability to make meaningful comparisons.
All participants in HCI research studies should be well treated and approached
with an open mind. Participating in HCI studies should be fun and engaging when-
ever possible: by making our studies positive experiences, we encourage people both
to participate and to provide useful feedback. As researchers, we should “expect the
unexpected”: software will crash, devices won't work, and (perhaps most distress-
ingly) users will hate our beloved inventions. High-quality HCI research takes these
setbacks in its stride, all the while striving to observe carefully while maintaining
respect for the people who give a bit of their time to help our studies along. By watch-
ing and listening carefully, we can learn from what users do and how they do it. That,
after all, is the point of conducting user studies.

DISCUSSION QUESTIONS

1. University researchers occasionally ask students in a class to participate in
research studies. However, this practice may involve elements of coercion, as
students may be concerned that refusal to participate may negatively impact
their grade. Is voluntary informed consent possible in such a situation? What
steps might be taken to reconcile the researcher's need for subjects with the
students' right to decline to participate?
2. The virtual reprise of Milgram's experiment (see Section 15.2.1) asked participants
to inflict harm upon a computer-generated avatar. This approach eliminates some
of the potential ethical concerns associated with the original experiment, but may
raise additional questions. As user behavior was similar to what was observed in
the original experiments, it is possible that participants in the “virtual” versions
would experience similar patterns of nervousness and distress. Do you consider
this sort of research to be appropriate? What might be done to protect participants
in this sort of experiment?
3. As part of a larger study of how various aspects of interaction in online
worlds impact the offline lives of participants, you are interested in observing
participants both online and offline. As you know, participants in online games