Page 274 - Statistics for Dummies

258
Part V: Statistical Studies and the Hunt for a Meaningful Relationship
the respondents would be those who strongly agree or disagree with the
proposed rules. Suppose only 200 people respond ed — 100 against and 100
for the issue. That would mean that 800 opinions weren’t counted. Suppose
none of those 800 people really cared about the issue either way. If you could
count their opinions, the results would be 800 ÷ 1,000 = 80% “no opinion,”
100 ÷ 1,000 = 10% in favor of the new rules, and 100 ÷ 1,000 = 10% against the
new rules. But without the votes of the 800 non-respondents, the researchers
would report, “Of the people who responded, 50% were in favor of the new
rules and 50% were against them.” This gives the impression of a very differ-
ent (and a very biased) result from the one you would’ve gotten if all 1,000
people had responded.
The response rate of a survey is a ratio found by taking the number of respon-
dents divided by the number of people who were originally asked to partici-
pate. You of course want to have the highest response rate you can get with
your survey; but how high is high enough to be minimizing bias? The purest
of the pure statisticians feel that a good response rate is anything over 70%,
but I think we need to be a little more realistic. Today’s fast-paced society is
saturated with surveys; many if not most response rates fall far short of 70%.
In fact, response rates for today’s surveys are more likely to be in the 20% to
30% range, unless the survey is conducted by a professional polling organiza-
tion such as Gallup or you are being offered a new car just for filling one out.
Look for the response rate when examining survey results. If the response rate
is too low (much less than 50%) the results are likely to be biased and should
be taken with a grain of salt, or even ignored.
Don’t be fooled by a survey that claims to have a large number of respondents
but actually has a low response rate; in this case, many people may have
responded, but many more were asked and didn’t respond.
Note that statistical formulas at this level (including the formulas in this book)
assume that your sample size is equal to the number of respondents, so statis-
ticians want you to know how important it is to follow up with people and not
end up with biased data due to non-response. However, in reality, statisticians
know that you can’t always get everyone to respond, no matter how hard you
try; indeed, even the U.S. Census doesn’t have a 100% response rate. One way
statisticians combat the non-response problem after the data have been col-
lected is to break down the data to see how well it matches the target popula-
tion. If it’s a fairly good match, they can rest easier on the bias issue.
So which number do you put in for n in all those statistical formulas you
use so often (such as the sample mean in Chapter 5)? You can’t use the
intended sample size (the number of people contacted). You have to use the
final sample size (the number of people who responded). In the media you
most often see only the number of respondents reported, but you also need
the response rate (or the total number of respondents) to be able to critically
evaluate the results.

3/25/11 8:13 PM
24_9780470911082-ch16.indd 258 3/25/11 8:13 PM
24_9780470911082-ch16.indd 258