Page 88 - Statistics for Environmental Engineers

L1592_Frame_C09 Page 81 Tuesday, December 18, 2001 1:45 PM

Multiple Sources of Variation (or Reproducibility ≠ Repeatability)
The measure of whether errors are becoming smaller as analysts are trained and as techniques are refined
is the standard deviation of replicate measurements on identical specimens. This standard deviation must
include all sources of variation that affect the measurement process.
Reproducibility and repeatability are often used as synonyms for precision. They are not the same.
Suppose that identical specimens were analyzed on five different occasions using different reagents and
under different laboratory conditions, and perhaps by different analysts. Measurement variation will
reflect differences in analyst, laboratory, reagent, glassware, and other uncontrolled factors. This variation
measures the reproducibility of the measurement process.
Compare this with the results of a single analyst who made five replicate measurements in rapid
succession using the same set of reagents and glassware throughout, while temperature, humidity, and
other laboratory conditions remained nearly constant. This variation measures repeatability.
Expect that reproducibility variation will be greater than repeatability variation. Repeatability gives
a false sense of security about the precision of the data. The quantity of practical importance is
reproducibility.

Example 9.4

Two analysts (or two laboratories) are each given five identical specimens to test. One analyst
made five replicate measurements in rapid succession and obtained these results: 38.2, 39.7, 37.1,
39.0, and 38.6. The average is 38.5 and the variance is 0.97. This measures repeatability.
The other analyst made five measurements on five different days and got these results: 37.0, 38.5,
37.9, 41.3, and 39.9. The average is 38.9 and the variance is 2.88. This measures reproducibility.

The reproducibility variance is what should be expected as the laboratory generates data over a period
of time. It consists of the repeatability variance plus additional variance due to other factors. Beware
of making the standard deviation artificially small by replicating only part of the process.

Interlaboratory Comparisons
A consulting firm or industry that sends test specimens to several different laboratories needs to know
that the performance of the laboratories is consistent. This can be checked by doing do an interlaboratory
comparison or collaborative trial. A number of test materials, covering the range of typical values, are
sent to a number of laboratories, each of which submits the values it measures on these materials.
Sometimes, several properties are studied simultaneously. Sometimes, two or more alternate measuring
techniques are covered. Mandel (1964) gives an extended example. One method of comparison is
Youden’s Rank Test (Youden and Steiner, 1975). The U.S. EPA used these methods to conduct interlab-
oratory studies for the 600 series analytical methods. See Woodside and Kocurek (1997) for an application
of the Rank Test to compare 20 laboratories.
The simplest method is the Youden pairs test (Youden, 1972; Kateman and Buydens, 1993). Youden
proposed having different laboratories each analyze two similar test specimens, one having a low con-
centration and one having a high concentration. The two measurements from each laboratory are a
Youden pair.
The data in Table 9.1 are eight Youden pairs from eight laboratories and the differences between the
pairs for each lab. Figure 9.1 is the Youden plot of these eight pairs. Each point represents one laboratory.
Vertical and horizontal lines are drawn through sample averages of the laboratories, which are 2.11 µg/L
for the low concentration and 6.66 µg/L for the high concentration. If the measurement errors are unbiased
and random, the plotted pairs will scatter randomly about the intersection of the two lines, with about
the same number of points in each quadrant.

© 2002 By CRC Press LLC