Page 241 - Statistics for Dummies

Chapter 14: Claims, Tests, and Conclusions
Assessing the Chance of a Wrong Decision
After you make a decision to either reject H or fail to reject H , the next step is
o
o
living with the consequences, in terms of how people respond to your decision.
✓ If you conclude that a claim isn’t true but it actually is, will that result in
a lawsuit, a fine, unnecessary changes in the product, or consumer boy-
cotts that shouldn’t have happened? It’s possible.
✓ If you can’t disprove a claim that’s wrong, what happens then? Will
products continue to be made in the same way as they are now? Will no
new law be made, no new action taken, because you showed that noth-
ing was wrong? Missed opportunities to blow the whistle have been
known to occur.

Whatever decision you make with a hypothesis test, you know there is a
chance of being wrong; that’s life in the statistics world. Knowing the kinds of
errors that can happen and finding out how to curb the chance of them occur- 225
ring are key.
Making a false alarm: Type-1 errors
Suppose a company claims that its average package delivery time is 2 days,
and a consumer group tests this hypothesis, gets a p-value of 0.04, and
concludes that the claim is false: They believe that the average delivery time
is actually more than 2 days. This is a big deal. If the group can stand by its
statistics, it has done well to inform the public about the false advertising
issue. But what if the group is wrong?
Even if the group bases their study on a good design, collects good data, and
makes the right analysis, it can still be wrong. Why? Because its conclusions
were based on a sample of packages, not on the entire population. And as
Chapter 11 tells you, sample results vary from sample to sample.
Just because the results from a sample are unusual doesn’t mean they’re
impossible. A p-value of 0.04 means that the chance of getting your particu-
lar test statistic, even if the claim is true, is 4% (less than 5%). You reject H
o
in this case because that chance is small. But even a small chance is still a
chance!
Perhaps your sample, though collected randomly, just happens to be one of
those atypical samples whose result ended up far from what was expected.
So, H could be true, but your results lead you to a different conclusion. How
o
often does that happen? Five percent of the time (or whatever your given
cutoff probability is for rejecting H ).
o

3/25/11 8:14 PM
21_9780470911082-ch14.indd 225
21_9780470911082-ch14.indd 225 3/25/11 8:14 PM