Page 75 -

60 CHAPTER 3 Experimental design

The general relationship between the actual value we are looking for and the
observed values can be expressed as follows:
Observed values = Actual valueRandom error+

Random errors are also called “chance errors” or “noise.” They occur by chance
and are not correlated with the actual value. Random errors push the observed val-
ues to move up or down around the exact value. There is no way to eliminate or
control random errors but we can reduce the impact of random errors by enlarging
the observed sample size. When a sample size is small, the random errors may have
significant impact on the observed mean and the observed mean may be far from the
actual value. When a sample size is large enough, the random errors should offset
each other and the observed mean should be very close to the actual value. For ex-
ample, in the typing task earlier, if we observe only Session 1, the mean would be
46, which is 4 words from the true value of 50 words per minute. If we increase the
number of observed sessions to 5, the mean of the observed values is 49.8, very close
to the actual value. In reality, we can never claim that we are 100% confident that the
observed value is the actual value. But we can be 100% confident that the larger our
sample size is, the closer the observed value is to the actual value.

3.5.2 SYSTEMATIC ERRORS
Systematic errors, also called “biases,” are completely different in nature from ran-
dom errors. While random errors cause variations in observed values in both direc-
tions around the actual value, systematic errors always push the observed values in the
same direction. As a result, systematic errors never offset each other in the way that
random errors do and they cause the observed mean to be either too high or too low.
Using the typing task example, the participant might consistently underperform
during all five observation sessions, because of tiredness or nervousness, and we may
collect the following data:
Session 1: 47 words per minute
Session 2: 44 words per minute
Session 3: 45 words per minute
Session 4: 42 words per minute
Session 5: 46 words per minute
In this case, the mean of the observed values is 44.8, 5 words lower than the actual
value. Figure 3.7 shows the performance of the participant in each case. Under the un-
biased conditions, the observed values fluctuate due to random errors, but the fluctua-
tions occur in both directions around the actual value and offset each other. However,
under the biased condition, the systematic error consistently pushes all values down,
causing the mean of the observed values to be significantly below the actual value.
Systematic errors can greatly reduce the reliability of experimental results; there-
fore, they are the true enemy of experimental research. We can counter systematic er-
rors in two stages: we should try to eliminate or control biases during the experiment