Page 289 - Statistics for Dummies

Chapter 17: Experiments: Medical Breakthroughs or Misleading Results?
of ethics concerns (or because of expense or other reasons), a large body of
observational studies examining many different factors and coming up with
similar conclusions is the next best thing. (See Chapter 18 for more about
cause-and-effect relationships.)
Collecting good data
What constitutes “good” data? Statisticians use three criteria for evaluating
data quality; each of the criteria really relates most strongly to the quality of
the measurement instrument that’s used in the process of collecting the data.
To decide whether you’re looking at good data from a study, look for these
characteristics:
✓ The data are reliable — you can get repeatable results with subse-
quent measurements. Many bathroom scales give unreliable data. You
get on the scale, and it gives you one number. You don’t believe the
number, so you get off, get back on, and get a different number. (If the 273
second number is lower, you’ll most likely quit at this point; if not, you
may continue getting on and off until you see a number you like.) Or you
can do what some researchers do: Take three measurements, find the
average, and use that; at least this will improve the reliability a bit.
Unreliable data come from unreliable measurement instruments or unre-
liable data collection methods. Errors can go beyond the actual scales to
more intangible measurement instruments, like survey questions, which
can give unreliable results if they’re written in an ambiguous way (see
Chapter 16).
Find out how the data were collected when examining the results of a
study. If the measurements are unreliable, the data could be inaccurate.
✓ The data are valid — they measure what they’re supposed to measure.
Checking the validity of data requires you to step back and look at the
big picture. You have to ask the question: Do these data measure what
they should be measuring? Or should the researchers have been collect-
ing altogether different data? The appropriateness of the measurement
instrument used is important. For example, many educators say that
a student’s transcript is not a valid measure of their ability to perform
well in college. Alternatives include a more holistic approach, taking into
account not only grades, but adding weight to elements such as service,
creativity, social involvement, extracurricular activities, and the like.
Before accepting the results of an experiment, find out what data were
measured and how they were measured. Be sure the researchers are
collecting valid data that are appropriate for the goals of the study.

3/25/11 8:13 PM
25_9780470911082-ch17.indd 273
25_9780470911082-ch17.indd 273 3/25/11 8:13 PM