Page 47 -

32 CHAPTER 2 Experimental research

Variables in the fifth category describe the cognitive and physical demand that an
application or a task exerts on an individual or how long an individual can interact
with an application without significant fatigue. This category of measures is less
studied but they play an important role in technology adoption.

2.3 BASICS OF EXPERIMENTAL RESEARCH

In order to understand why experimental research can allow causal inference while de-
scriptive and relational investigations do not, we need to discuss the characteristics of
experimental research. In a true experimental design, the investigator can fully control
or manipulate the experimental conditions so that a direct comparison can be made
between two or more conditions while other factors are, ideally, kept the same. One
aspect of the full control of factors is complete randomization, which means that the
investigator can randomly assign participants to different conditions. The capability to
effectively control for variables not of interest, therefore limiting the effects to the vari-
ables being studied, is the feature that most differentiates experimental research from
quasi-experimental research, descriptive investigations, and relational investigations.

2.3.1 COMPONENTS OF AN EXPERIMENT

After a research hypothesis is identified, the design of an experiment consists of three
components: treatments, units, and assignment method (Oehlert, 2000). Treatments,
or conditions, refer to the different techniques, devices, or procedures that we want
to compare. Units are the objects to which we apply the experiment treatments. In
the field of HCI research, the units are normally human subjects with specific char-
acteristics, such as gender, age, or computing experience. Assignment method refers
to the way in which the experimental units are assigned different treatments.
We can further explain these three terms through an example. Suppose a researcher
is running an experiment to compare typing speed using a traditional QWERTY
3
keyboard and a DVORAK keyboard. The treatment of this experiment is the type of
keyboard: QWERTY or DVORAK. The experiment units are the participants recruited
to join the study. To achieve the goal of fair comparison, the researchers would have
to require that the participants have no previous experience using either keyboard. If
most participants can touch type using the QWERTY keyboard but have never used a
DVORAK keyboard before, it is obvious that the results will be highly biased towards
the QWERTY keyboard. The researcher can employ different methods to randomly
assign the participants into each of the two conditions. One well-known traditional
method is to toss a coin. If a head is tossed, the participant is assigned to the QWERTY
condition. If a tail is tossed, the participant is assigned to the DVORAK condition.
Obviously, researchers are not busy tossing coins in their lab; more convenient random-
ization methods are used today. We discuss those methods in Section 2.3.2.

3
Dvorak keyboard is an ergonomic alternative to the commonly used “QWERTY keyboard.” The
design of the Dvorak keyboard emphasizes typist comfort, high productivity, and ease of learning.