Page 45 - The Importance of Common Metrics for Advacing Social Science Theory and Research

The Importance of Common Metrics for Advancing Social Science Theory and Research: A Workshop Summary
http://www.nap.edu/catalog/13034.html

INDICATORS 33

ing periods of intense scrutiny. In general, measures that are more
about outcomes than outputs are less vulnerable to distortion.
• Diminishing utility: For example, as soon as any measure of money
supply becomes an official policy target for a government, it im-
mediately becomes less useful because of market behavior antici-
pating movements in the indicator. Another example is the use of
standardized tests and international benchmarking. Not only have
these been powerful tools to drive up standards in mathematics and
science literacy, but they also have diverted attention away from
equally important but less measurable aspects of learning, such as
noncognitive skills, social skills, resilience, motivation, and other
key predictors of lifetime earnings, social mobility, and life success.
• Obsolescence: Some standardized measures reflect society or the
economy at a particular point and become less useful over time.
The utility for policy makers of evolving indicators may outweigh
the utility of consistency.
• Limited relevance: While standard measurements may reflect the
views of officials and professionals, they may be very different
from those used by the public. For example, quality in health care
services may be measured by official statistics in terms of waiting
times or mortality, but the public may describe such factors as
service style as most important.

Categories of Standardized Measurement:
Underlying Causes and Relationship

Mulgan commented that, in most areas of public policy, there is little
agreement about the fundamentals of causation and theory. Grade retention
in school in the United States, for example, can be explained by economists
as an issue of economic incentives of the labor market. Sociologists will
insist that peer pressure is a key factor. Educators will claim that perfor-
mance at age 11 affects a student at age 14, and psychologists may focus
on personality structure. Consequently, he said, policy makers may not
agree on which causal mode is correct, and there is no single approach to
resolving disagreements.
In addition, he continued, there are also fields in which new indicators
are needed, for example, the use of the Internet for public services. Related
to this topic, Mulgan reported on a review that he recently conducted on
the state of knowledge about behavior change and its relevance to health
policy. He found an uneven evidence base on the efficacy of either financial
incentives or “nudge-type” methods of environmental shaping of behavior.

Copyright © National Academy of Sciences. All rights reserved.