Page 69 -

54 CHAPTER 3 Experimental design

across conditions. In other words, we need to make sure that the groups are as
similar as possible, except for the personal characteristics that are experimental
variables under investigation.

3.3.2.2 Within-group design
Within-group design is more appropriate when the experiment investigates tasks
with large individual differences, tasks that are less susceptible to the learning ef-
fect, or when the target participant pool is very small. As discussed previously,
complicated tasks that involve substantial human cognitive and perceptual capa-
bilities generally encounter much larger individual differences than simple tasks.
Therefore, when an experiment investigates complicated tasks such as reading,
comprehension, information retrieval, and problem solving, a within-group design
might be more appropriate since it effectively isolates individual differences from
the main effects.
Most of the tasks that examine complicated or learned skills or knowledge—such
as typing, reading, composition, and problem solving—are less susceptible to learn-
ing effects. For example, if an experiment investigates the impact of two fonts (i.e.,
Times New Roman and Arial) on participants' reading speed, the learning effect be-
tween the two conditions would be very limited. Reading one text document of sev-
eral hundred words is unlikely to improve an individual's reading speed. Therefore, a
within-group design would be appropriate as long as the text materials presented to
the participant under the two conditions are different in content but similar in levels
of difficulty.
Difficulty in finding and recruiting qualified participants is a problem frequently
faced by many HCI researchers. One typical example is the field of universal us-
ability, which focuses on developing applications usable by diverse user populations.
Numerous studies in this field examine how individuals with disabilities interact with
computers or computer-related devices. Although the total number of people fall-
ing into a specific disability or disease category is quite large, the number of such
individuals living in a particular area is very limited. Therefore, the sample sizes
are normally smaller than that in studies examining users without disabilities (e.g.,
Taylor et al., 2016).
Recruiting participants with specific disabilities is always a challenging task. For
more detailed discussion on working with participants with disabilities, please refer
to Chapter 16. The same problem also occurs when the target population is well
trained, highly experienced, professionals, such as business executives or experi-
enced project managers, simply because they are too busy to be bothered. Under
those circumstances, it is almost impossible to recruit the number of participants
needed for a between-group design, forcing the experimenters to adopt a within-
group design.
Having decided to adopt a within-group design, you need to consider how to con-
trol the negative impact of learning effects, fatigue, and other potential problems as-
sociated with a within-group design. As discussed previously, a general approach to