Page 157 - Statistics for Environmental Engineers

L1592_frame_C17 Page 154 Tuesday, December 18, 2001 1:51 PM

To further emphasize this, Hooke (1963) identified these inadequacies of significance tests.

1. The test is qualitative rather than quantitative. In dealing with quantitative variables, it is
often wasteful to point an entire experiment toward determining the existence of an effect
when the effect could also be measured at no extra cost. A confidence statement, when it can
be made, contains all the information that a significance statement does, and more.
2. The word “significance” often creates misunderstandings, owing to the common habit of
omitting the modifier “statistical.” Statistical significance merely indicates that evidence of
an effect is present, but provides no evidence in deciding whether the effect is large enough
to be important. In a given experiment, statistical significance is neither necessary nor sufficient
for scientific or practical importance. (emphasis added)
3. Since statistical significance means only that an effect can be seen in spite of the experimental
error (a signal is heard above the noise), it is clear that the outcome of an experiment depends
very strongly on the sample size. Large samples tend to produce significant results, while
small samples fail to do so.
Now, having declared that we prefer not to state results as being significant or nonsignificant, we pass
on two tips from Chatfield (1983) that are well worth remembering:

1. A nonsignificant difference is not necessarily the same thing as no difference.
2. A significant difference is not necessarily the same thing as an interesting difference.

References
Chatfield, C. (1983). Statistics for Technology, 3rd ed., London, Chapman & Hall.
Hooke, R. (1963). Introduction to Scientific Inference, San Francisco, CA, Holden-Day.
Simpson, R. D. and A. Dudaitis (1981). “Changes in the Density of Zooplankton Passing Through the Cooling
System of a Power-Generating Plant,” Water Res., 15, 133–138.
Sokal, R. R. and F. J. Rohlf (1969). Biometry: The Principles and Practice of Statistics in Biological Research,
New York, W. H. Freeman and Co.
Tukey, J. W. (1991). “The Philosophy of Multiple Comparisons,” Stat. Sci., 6(6), 100–116.
Wilcock, R. J., C. D. Stevenson, and C. A. Roberts (1981). “An Interlaboratory Study of Dissolved Oxygen
in Water,” Water Res., 15, 321–325.

Exercises

17.1 Antimony. Antimony in fish was measured in three paired samples by an official standard
method and a new method. Do the two methods differ significantly?

Sample No. 1 2 3
New method 2.964 3.030 2.994
Standard method 2.913 3.000 3.024

17.2 Nitrite Measurement. The following data were obtained from paired measurements of nitrite
in water and wastewater by direct ion-selective electrode (ISE) and a colorimetric method.
Are the two methods giving consistent results?

© 2002 By CRC Press LLC