Page 283 - Statistics for Dummies

Chapter 17: Experiments: Medical Breakthroughs or Misleading Results?
No treatment
“No treatment” means the researcher can’t help but tell which group the sub-
ject is in, due to the nature of the experiment. The subjects in this case aren’t
receiving any type of intervention in terms of their behavior, but they still
serve as a control, establishing a baseline of data to compare their results
with those in the treatment group(s). For example, if you want to determine
whether speed walking around the block ten times a day lowers a person’s
resting heart rate after six months, the subjects in your control group know
they aren’t going to be speed walking — obviously you can’t do fake speed
walking (although faking exercising and still reaping the benefits would be
great, wouldn’t it?).
In situations where the control group receives no treatment, you still make

sure the groups of subjects (speed walkers versus non–speed walkers) are sim-
ilar in as many ways as possible, and that the other criteria for a good experi-
ment are being met. (See “Designing a Good Experiment” for the list of criteria.)
Selecting the sample size 267
The size of a (good) sample greatly affects the accuracy of the results. The
larger the sample size, the more accurate the results, and the more powerful
the statistical tests (in terms of being able to detect real results when they
exist). In this section, I hit the highlights; Chapter 14 has the details.
The word sample is often attributed to surveys where a random sample is
selected from the target population (see Chapter 16). However, in the setting
of experiments, a sample means the group of subjects who have volunteered
to participate.
Limiting small samples to small conclusions
You may be surprised at the number of research headlines that have been
made regarding large populations that were based on very small samples.
Such headlines can be of concern to statisticians, who know that detecting
true statistically significant results in a large population using a small sample
is difficult because small data sets have more variability from sample to
sample (see Chapter 12). When sample sizes are small and big conclusions
have been made by the researcher, either the researchers didn’t use the right
hypothesis test to analyze their data (for example, using the Z-distribution
rather than the t-distribution; see Chapter 10) or the difference was so large
that it would be very difficult to miss. The latter isn’t always the case, however.
Be wary of research conclusions that find significant results based on small
sample sizes (especially for experiments involving many treatments but only a
few subjects assigned to each treatment). Statisticians want to see at least five
subjects per treatment, but (much) more is (much) better. You do need to be

3/25/11 8:13 PM
25_9780470911082-ch17.indd 267
25_9780470911082-ch17.indd 267 3/25/11 8:13 PM