Page 62 -

3.2 Determining the basic design structure 47

three conditions under this independent variable as well. Since we need to test each
combination of values of the two independent variables, combining the two indepen-
dent variables results in a total of nine (3 × 3 = 9) conditions in the experiment.
The identification of dependent variables will allow us to further consider the ap-
propriate metric for measuring the dependent variables. In many cases, multiple ap-
proaches can be used to measure the dependent variables. For example, typing speed
can be measured by the number of words typed per minute, which is equal to the
total number of words typed divided by the number of minutes used to generate those
words. It may also be measured by number of correct words typed per minute, which
is equal to the total number of correct words typed divided by the number of minutes
used to generate those words. We need to consider the objective of the experiment to
determine which measure is more appropriate.
Another issue to consider when designing experiments is how to control the inde-
pendent variables to create multiple experimental conditions (Kirk, 1982). In some
experiments, control of the independent variable is quite easy and straightforward.
For instance, when testing the previously stated hypothesis, we can control the type
of pointing device by presenting participants with a mouse, a joystick, or a trackball.
In many other cases, the control of the independent variable can be challenging. For
instance, if we are developing a speech-based application and need to investigate
how recognition errors impact users' interaction behavior, we may want to compare
two conditions. Under the control condition, the speech recognizer would be error
free and recognize every word that the user says correctly. Under the comparison
condition, the speech recognizer would make errors and recognize a percentage of
the words incorrectly. This sounds straightforward, theoretically. But in practice, all
speech recognizers make errors. There is no way to find a recognizer that would
satisfy the requirements of the controlled condition. A possible solution to meet the
needs of this experiment is the Wizard-of-Oz approach (Feng and Sears, 2009). That
is, we can have a human acting as a speech recognizer, listening to what the user says
and entering the user's dictation into the system. The truth would normally not be
revealed to the participants until the end of the experiment. Therefore, all participants
would believe that they are interacting with the speech recognizer when completing
the task. The Wizard-of-Oz approach allows us to test ideal applications that do not
exist in the real world. This approach is not without its limitations. Humans also
make errors. It is very likely that the human “wizard” would make errors when listen-
ing to the dictation or when typing the words. Therefore, it is very difficult to control
the independent variable to achieve the desired condition (Feng and Sears, 2009; Li
et al., 2006). One approach that addresses this problem is the development of techni-
cal tools to assist the human wizard (Li et al., 2006).

3.2 DETERMINING THE BASIC DESIGN STRUCTURE
At the first stage of experimental design, we need to construct the experiment based
on the research hypotheses that have been developed. This enables us to draw a big
picture of the general scope of the experiment and, accordingly, come up with a