Page 27 - The Importance of Common Metrics for Advacing Social Science Theory and Research

The Importance of Common Metrics for Advancing Social Science Theory and Research: A Workshop Summary
http://www.nap.edu/catalog/13034.html

MEASUREMENT IN THE SOCIAL SCIENCES 15

1. Repeated use gives meaning to a metric; overuse may reify it.
2. Meet a real scientific and/or policy need. If no one else will use a
measure, it is not worth the effort. Widespread use is rewarding. A
check of citation indexes attests to the fact that the biggest citation
counts go to people who develop useful measures, not those who
analyze data.
3. Seek simplicity in content and construction. To the extent that
an indicator is hard to ascertain, is complicated to construct, and
admits multiple interpretations, it will be less useful.
4. Avoid relative measurement: above all, avoid percentile ranks,
standard deviations, and shares of variance.
5. Avoid descriptive terms for arbitrarily or subjectively deter-
mined ranges of a quantitative indicator. Such terms invite
misinterpretation.
6. Study the operational and analytical behavior of a measure to as-
sess its validity, not merely the details of its construction.
7. Weigh the balance between internal and external validity. Informa-
tion loss may vary positively with comparability, and sometimes
loss is gain.

His closing remark was that nothing is more important and scientifi-
cally rewarding than the development of standard metrics that are useful
in theory and in practice.

DISCUSSION

In her discussant remarks, Christine Bachrach (Duke University and
University of Maryland) posed three broad questions to further extend
the range of issues based on her reading of the workshop papers and
presentations.
First, how healthy is measurement science in the social sciences? Un-
derstanding common metrics to advance social science theory as the focus
of the workshop, Bachrach probed whether theory is actually advancing
metrics, common or not, in an adequate fashion in the social sciences. It
is important to carefully define the constructs one wants to measure, she
cautioned.
In addition, Bachrach noted that the seriousness with which measure-
ment is approached and the degree to which it is grounded in scientific
principles and scientific methods actually vary tremendously across the
behavioral and social sciences. She observed that there are structural fac-
tors that contribute to placing measurement on the sidelines, chief among
them the balkanization of disciplines, with some placing greater emphasis
on measurement issues.

Copyright © National Academy of Sciences. All rights reserved.