Page 68 -

3.3 Investigating a single independent variable 53

The size of the individual differences is very hard to estimate. However, it
is empirically confirmed that individual differences are smaller when the tasks
are simple and involve limited cognitive process (Dillon, 1996; Egan, 1988).
In contrast, individual differences are larger when the task is complicated or
involves significant cognitive functions. For example, when the task mainly in-
volves basic motor skills, such as selecting a target on the screen, the individual
2
differences among participants might be comparatively small. But when the
task involves more complicated cognitive or perceptual functions, such as read-
ing, comprehension, information retrieval, and problem solving, the individual
differences have a much larger impact. So when the task is simple, the im-
pact of individual differences is limited and a between-group design would be
appropriate.
Depending on the types of task, some experiments are more vulnerable to
the learning effect than others. For example, in an experiment that compares the
navigation effectiveness of two types of menu within a website, a participant
who completes the navigation tasks under one condition would have gained a
significant amount of knowledge of the website architecture. The knowledge
would make a great impact on the participant's performance when completing
the tasks under the other condition. Therefore, within-group design is highly
inappropriate for this type of task and between-group design would have to be
adopted.
There are many circumstances when it is totally impossible to adopt a within-
group design. Taking hypotheses H2 and H3, previously stated, as examples:
• H2: There is no difference in the time required to locate an item in an online
store between novice users and experienced users.
• H3: There is no difference in the perceived trust toward an online agent among
customers who are from the United States, Russia, China, and Nigeria.
You can see that there is no way to compare the performances of novice users
and experienced users through a within-group design because an individual cannot
be both a novice user and an experienced user of the online store at the same time.
For the same reason, a within-group design is not appropriate for H3 since any par-
ticipant can only represent one of the four cultures. Under those circumstances, a
between-group design is obviously the only option we have.
After choosing a between-group design for an experiment, we need to take
special caution to control potential confounding factors. Participants should be
3
randomly assigned to different conditions whenever possible. When assign-
ing participants, we need to try our best to counterbalance potential confound-
ing factors, such as gender, age, computing experience, and internet experience,

2
Note that the individual differences in these types of tasks can be quite substantial when the partici-
pants come from different age groups or when individuals with motor disabilities are involved.
3
We cannot randomly assign participants to different conditions in the cases of H2 and H3, obviously.