Page 22 - The Importance of Common Metrics for Advacing Social Science Theory and Research

The Importance of Common Metrics for Advancing Social Science Theory and Research: A Workshop Summary
http://www.nap.edu/catalog/13034.html

10 THE IMPORTANCE OF COMMON METRICS

ment intersects with policy. At the same time, however, standardization can
entail the loss of information, and too much standardization may make ex-
tensive evidence uninformative and misleading. A delicate balance must be
negotiated, he said, between standardization of measurement and validity
of social scientific constructs. This can be complicated, because measure-
ment can overlap with representation (who or what is being measured),
analysis (how data will be described and used), theory, and policy.
Hauser then illustrated his point with a number of public metrics, in
declining order of success, based on his judgment of the validity and usage
of the measures:

• The unemployment rate is a social scientific invention based on
a detailed behavioral report of job searching during a reference
week by members of the labor force. It is defective in the sense that
the officially unemployed do not include “discouraged workers,”
persons who have given up on their search for employment, or the
underemployed. This defect is exacerbated when unemployment
is high, as the measure underestimates the extent of economic
distress.
• The official poverty line is a more recent scientific invention fre-
quently used in policy applications despite major weaknesses that
greatly limited its validity and usefulness from the outset. It is an
absolute standard in real dollars, updated only to reflect changes
in the consumer price index. Because of this and the fact that living
standards and the share of food in family budgets have changed,
the standard has become increasingly obsolete. In Hauser’s estima-
tion, the official poverty line has been overused in thousands of
research papers and books, and perceptions about poverty and the
poor would differ if a standard measure of greater validity were
widely accepted.
• Academic achievement levels offer a more recent example of a
nominally social scientific, standardized measure that has become
visible and influential in public discourse and policy. Although
drawn on questionable and subjective methods, academic achieve-
ment levels have nevertheless become ubiquitous in reports on
diverse subjects at state and national levels. Public and political
demands for understandable metrics of academic accountability
have trumped their negative evaluations, he said. In this case,
Hauser pointed out, the creation of a supposedly scientific set of
standards led to their reification in law, to the creation of compet-
ing standards, and to comparisons of populations in differing but
nominally identical metrics.

Copyright © National Academy of Sciences. All rights reserved.