Page 48 -

2.3 Basics of experimental research 33

4
The keyboard comparison case illustrates a simple between-subject design with
two conditions. There are much more complicated designs involving multiple treat-
5
ments and both between-subject and within-subject comparisons. No matter how
complicated the design is, all experiments consist of these three major components:
treatments, units, and assignment methods.

2.3.2 RANDOMIZATION
The power of experimental research lies in its ability to uncover causal relations. The
major reason why experimental research can achieve this goal is because of complete
randomization. Randomization refers to the random assignment of treatments to the
experimental units or participants (Oehlert, 2000).
In a totally randomized experiment, no one, including the investigators themselves, is
able to predict the condition to which a participant is going to be assigned. For example,
in the QWERTY vs. DVORAK experiment, when a participant comes in, the research-
ers do not know whether the participant will be using the QWERTY keyboard or the
DVORAK keyboard until they toss a coin and find out whether it settles as heads or tails.
Since the outcome of tossing the coin is totally random and out of the control of the re-
searchers, the researchers have no influence, whether intentionally or subconsciously, on
the assignment of the treatment to the participant. This effectively controls the influence
of hidden factors and allows a clean comparison between the experiment conditions.
Traditional randomization methods include tossing a coin, throwing dice, spin-
ning a roulette wheel, or drawing capsules out of an urn. However, these types of
randomization are rarely used in behavioral research and HCI studies nowadays. One
method to randomize the selection of experimental conditions or other factors is the
use of a random digit table. Table 2.2 is an abbreviated random digit table taken from
the large random digit table generated by RAND (1955). The original table consisted
of a million random digits.

Table 2.2 An Abbreviated Random Digit Table
Line Random Digits
000 10097 32533 76520 13586 34673
001 37542 04805 64894 74296 24805
002 08422 68953 19645 09303 23209
003 99019 02529 09376 70715 38311
004 12807 99970 80157 36147 64032
005 66065 74717 34072 76850 36697

There are several ways to use this table. Suppose we are running a study that
compares three types of navigation schemes for a website: topical, audience split,

4
A between-subject design means each participant only experiences one task condition. The details of
between-subject design are discussed in Chapter 3.
5
A within-subject design means each participant experiences multiple task conditions. The details of
within-subject design is discussed in Chapter 3.