Page 49 -

34 CHAPTER 2 Experimental research

and organizational. We recruit 45 participants and need to assign each of them to
one of the three conditions. We can start anywhere in the random digit table and
count in either direction. For example, if we start from the third number on the first
row and count to the right for three numbers, we get 76520, 13586, and 34673. We
can assign the first three participants to the conditions according to the order of the
three random numbers. In this case, 76520 is the largest, corresponding to condition
3; 13586 is the smallest, corresponding to condition 1; and 34673 corresponds to
condition 2. This means that the first participant is assigned to the design with the or-
ganizational navigation scheme, the second participant to the topical scheme, and the
third participant to the audience split scheme. We can continue counting the numbers
and repeating the process until all 45 participants are assigned to specific conditions.
Nowadays, software-driven randomization is also commonly used among re-
searchers and practitioners. A large number of randomization software resources are
available online, some of them free of charge, such as the services offered at http://
www.randomization.com. Randomization functions are also available in most of the
commercial statistical software packages, such as SAS, SPSS, and SYSTAT.
In a well-designed experiment, you will frequently find that you not only need to
randomize the assignment of experiment conditions, but other factors as well. In a
6
longitudinal study reported by Sears et al. (2001, 2003), the researchers investigated
the use of recognition software to generate text documents. Each of the 15 partici-
pants completed a total of nine tasks on different days. During each task, the partici-
pant composed a text document of approximately 500 words in response to one of
nine predefined scenarios. The researchers found it necessary to randomize the order
of the scenarios being used in the nine tasks. If the order of the scenarios were not
randomized, it is likely that the characteristics of the scenarios would become a fac-
tor that influences the results. Randomizing the order of the scenarios cancels out the
potential errors introduced by differences in scenarios. 7
Counter balancing is commonly used in experiments to address the problem of
systematic differences between successive conditions. In this case, researchers usu-
ally rotate the sequences of treatments or conditions through a “Latin Square Design”
illustrated in Table 2.3 (Rosenthal and Rosnow, 2008). In this table, letters A, B, C,

Table 2.3 Latin Square Design
Order of Administration
1 2 3 4
Sequence 1 A B C D
Sequence 2 B C D A
Sequence 3 C D A B
Sequence 4 D A B C

6
A study in which data is gathered for the same participants repeatedly over a period of time.
7
Special attention was paid during the development of the scenarios so that they are similar to each other
in the degree of difficulty in responding, which was confirmed by the reported results. However, it is good
practice to randomize the order of the scenarios in case there are unanticipated differences between them.