Page 57 - Statistics for Dummies

Chapter 3: Taking Control: So Many Numbers, So Little Time
✓ Should the line-item veto be available to the president to eliminate waste
(yes/no/no opinon)?
✓ Does the line-item veto give the president too much individual power
(yes/no/no opinion)?

✓ What is your opinion on the presidential line-item veto? Choose 1–5,
with 1 = strongly opposed and 5 = strongly support.
The first two questions are misleading and will lead to biased results in oppo-
site directions. The third version will draw results that are more accurate in
terms of what people really think. However, not all surveys are written with the
purpose of finding the truth; many are written to support a certain viewpoint.
Research shows that even small changes in wording affect survey outcomes,
leading to results that conflict when different surveys are compared. If you
can tell from the wording of the question how they want you to respond to it,
you know you’re looking at a leading question; and leading questions lead to
biased results.(See Chapter 16 for more on spotting problems with surveys.) 41
Looking for lies in all the right places
Every once in a while, you hear about someone who faked his data, or “fudged
the numbers.” Probably the most commonly committed lie involving statistics
and data is when people throw out data that don’t fit their hypothesis, don’t
fit the pattern, or appear to be outliers. In cases when someone has clearly
made an error (for example, someone’s age is recorded as 200), removing that
erroneous data point or trying to correct the error makes sense. Eliminating
data for any other reason is ethically wrong; yet it happens.
Regarding missing data from experiments, a commonly used phrase is
“Among those who completed the study. . . .” What about those who didn’t
complete the study, especially a medical one? Did they get tired of the side
effects of the experimental drug and quit? If so, the loss of this person will
create results that are biased toward positive outcomes.
Before believing the results of a study, check out how many people were
chosen to participate, how many finished the study, and what happened to all
the participants, not just the ones who experienced a positive result.
Surveys are not immune to problems from missing data, either. For example,
it’s known by statisticians that the opinions of people who respond to a
survey can be very different from the opinions of those who don’t. In general,
the lower the percentage of people who respond to a survey (the response
rate), the less credible the results will be. For more about surveys and
missing data, see Chapter 16.

3/25/11 8:17 PM
07_9780470911082-ch03.indd 41 3/25/11 8:17 PM
07_9780470911082-ch03.indd 41