Page 465 -

15.2 Identifying potential participants 457

be used by a broad range of users in nonvoluntary circumstances, such as mandatory
workplace timesheet reports. Unmotivated users can also be useful for studies aimed
at understanding the factors that might influence reluctance to adopt new technology.
Expertise is always an important consideration: study participants should have
expertise that is comparable to that of the expected users. We usually define exper-
tise in terms of two largely separable dimensions: computer expertise and domain
expertise—knowledge of the problems, systems, goals, and tools used in a specific
line of work. If you are testing a tool that is built for highly trained professionals
who rarely use complex computer applications, you'll be looking for users who may
be computer novices, even though they have significant domain expertise. In other
cases, you might be looking for sophisticated computer users who are using a new
type of software for an unfamiliar task: computer experts but domain novices. Any
differences in expertise between your target population and the participants in your
study may lead to results that are hard to interpret.
Interfaces that are intended for use by a broad audience present relatively little
difficulty in terms of user characteristics. General-purpose desktop computing tools
and interfaces on widely used communications devices are likely to be used by many
motivated users, so study participants do not need to meet many specific criteria and
can often (but not always) be similar to each other.
The need for appropriate participants becomes more apparent with tools that are
designed for specific populations. Children and adults have vastly different cognitive
and physical abilities, which directly influence their ability to act as useful study
participants. Similarly, cultural differences between users may play a significant role
in task performance for communication systems. Whenever possible, studies of tools
designed for specific ages, genders, social backgrounds, and physical or cognitive
abilities should involve participants who fit the appropriate category. Asking college
students to evaluate a tool designed for elderly users would almost certainly be inap-
propriate. Ethnographic studies (Chapter 9) of specific users and situations are also
sensitive to the appropriateness of the participants. If study participants are not the
intended users of a system, you can only make limited claims about the utility of the
system for the intended population.
Systems designed for domain experts can be particularly challenging in this re-
gard. As the construction of tools for highly specialized tasks requires a detailed
understanding of domain-specific work practices, there is a natural tendency to use
techniques such as participatory design to involve users in system design. This in-
clusion may lead to valuable insights, but domain experts who were involved in the
design of a tool may have biases in favor of the resulting design, making them inap-
propriate candidates for subsequent usability tests or other summative evaluations.
Differences between users can also be an important part of study design.
Investigations of potential gender differences in organizing certain forms of infor-
mation would require both male and female participants. Similarly, an experiment
exploring the role of user motivation in understanding the effectiveness of a given
interface design may need participants who are highly motivated, as well as those
who are not at all motivated.