Page 470 -

462 CHAPTER 15 Working with human subjects

long known that participants may examine “demand characteristics “—trying to pro-
vide responses that they think will please the researcher (Orne, 1962). Although care-
ful researchers will always be on the lookout for opportunities to reduce potential
sources of bias, when working with incentives it might be particularly important to
stress to participants that they will be rewarded regardless of the answers they give.

MENU TASK PERFORMANCE STUDIES WITH BLIND USERS

Task performance with hierarchical menus has been the subject of many
studies over the years, leading to a general consensus that menus with many
choices at each of a few levels (broad, shallow trees) lead to faster task
completion than menus with a few choices at each of many levels (narrow,
deep structures), see Chapter 1. As these studies have generally been
conducted with sighted users, who could rely upon a visual scan to quickly
identify items in a long list, we were interested if these results would hold for
blind users who rely upon the serial presentation of items by screen readers.
To address this question, we designed a study based on an early experiment
that looked at breadth versus depth in web-based choices from an encyclopedia
(Larson and Czerwinski, 1998).
Experimental studies involving blind people can be particularly challenging
to run. As blind people often face challenges in transportation, asking users to
come to our university offices would be inconvenient. We also knew that we
wanted a particular population: experienced users of a particular screen-reader
package, who did not have any residual vision.
For this research project, we collaborated with the National Federation of
the Blind (NFB), who helped identify potential participants and provided us
with access to space in their offices, where we were able to run the study. NFB
was paid as a consultant on the project and study participants were compensated
as well. Due to the specific nature of the participants, compensation was
significantly higher than is customary for similar studies. With NFB’s help, we
were able to recruit a sufficient number of qualified participants, and we found
that, like sighted individuals, our blind participants fared better with broad,
shallow menu structures (Hochheiser and Lazar, 2010).

Compensation should be commensurate with the amount of time requested and
the type of participants involved. Busy professionals may command a higher fee
than students or children. For longer ethnographic or case studies, particularly with
domain experts, direct payment for study participation is unlikely to account for the
value of their time. In these cases, finding ways to pay experts as consultants may be
the best approach. For formative studies aimed at capturing requirements for systems
to be used by domain experts, the ability to use the software being developed in their
daily work might be a powerful enticement.