Page 66 -

3.3 Investigating a single independent variable 51

In order to effectively exclude the impact of noise and make significant findings,
a comparatively larger number of participants are needed under each condition. This
leads to the second major disadvantage of the between-group design: large sample
size. Since the number of participants (m) in each condition should be comparatively
larger than that in a within group design and approximately the same number of par-
ticipants are needed for each condition (let n be the number of conditions), the total
number of participants needed for the experiment (m × n) is usually quite large. For
example, if an experiment has 4 conditions and 16 participants are needed under each
condition, the total number of participants needed is 64. Recruiting the number of
participants needed for a between-group experiment can be a very challenging task.

3.3.1.2 Advantages and disadvantages of within-group design
Within-group design, in contrast, requires a much smaller sample size. When analyz-
ing the data coming from within-group experiments, we are comparing the perfor-
mances of the same participants under different conditions. Therefore, the impact
of individual differences is effectively isolated and the expected difference can be
observed with a relatively smaller sample size. If we change the design of the ex-
periment with 4 conditions and 16 participants from a between-group design into a
within-group design, the total number of participants needed would be 16, rather than
64. The benefit of a reduced sample size is an important factor for many studies in
the HCI field when qualified participants may be quite difficult to recruit. It may also
help reduce the cost of the experiments when financial compensation is provided.
Within-group designs are not free of limitations. The biggest problem with a
within-group design is the possible impact of learning effects. Since the participants
complete the same types of task under multiple conditions, they are very likely to
learn from the experience and may get better in completing the tasks. For instance,
suppose we are conducting a within-group experiment that evaluates two types of
ATM: one with a button interface and one with a touch-screen interface. The task
is to withdraw money from an existing account. If the participant first completes
the task using the ATM with the button interface, the participant gains some ex-
perience with the ATM interface and its functions. Therefore, the participant may
perform better when subsequently completing the same tasks using the ATM with
the touch-screen interface. If we do not isolate the learning effect, we might draw
a conclusion that the touch-screen interface is better than the button interface when
the observed difference is actually due to the learning effect. Normally, the potential
bias of the learning effect is the biggest concern of experimenters when considering
adopting a within-group design. A Latin Square Design is commonly adopted to
control the impact of the learning effect.
Another potential problem with within-group designs is fatigue. Since there are
multiple conditions in the experiment, and the participants need to complete one or
more tasks under each condition, the time it takes to complete the experiment may be
quite long and participants may get tired or bored during the process. Contrary to the
learning effect, which favors conditions completed toward the end of the experiment,
fatigue negatively impacts on the performance of conditions completed toward the