Page 44 - The Importance of Common Metrics for Advacing Social Science Theory and Research

The Importance of Common Metrics for Advancing Social Science Theory and Research: A Workshop Summary
http://www.nap.edu/catalog/13034.html

32 THE IMPORTANCE OF COMMON METRICS

approach has evolved into one of viewing standards as a tool for account-
ability and democracy. In addition, he observed, some of the long-run
trends involve measuring, not things, but rather less tangible concepts and
intangibles, as well as moving from single measures to indices and from
activities to outputs and outcomes. Also, there is a movement from objec-
tive facts to subjective measures of experience, for example, fear of crime
as well as crime volume, patient satisfaction, and other relational measures
of trust and feedback as well as classic health outcomes. He observed that
a broader shift to complement output and outcome measures with rela-
tionship measures is moving quickly around the world, although with less
speed in the United States. In addition, he told the audience, measurement
has moved from being primarily an issue for policy makers and the state
to becoming a source enabling the public and media to assess the progress
made by government. The latter includes measuring performance at the
local level, with indicators set at the level of very small neighborhoods as
well as the town or city.
Mulgan asserted that these new uses of indicators regarding place raise
two major issues related to experiential relational data and the balancing of
present performance and future prospects. Specifically, what is the appro-
priate benchmark? And how can these measures of assessment of current
performance be combined with some dynamic indicators to determine the
future success of that area, for example, in terms of individual and business
resilience?

Weaknesses and Risks in Standardization

Mulgan listed several classic weaknesses inherent in more widespread
use of metrics in policy:

• Excess simplicity: There is a risk of using excessively simple re-
sponses to complex problems, such as unemployment rates, that
can distort reality or encourage excessive focus (e.g., targeting
measures of household burglary may divert resources from other
equally important crimes). In contrast, discussions under way in
the United Kingdom on reducing cancer mortality focus on in-
creasing the quality of clinical services, as well as addressing the
environment, stress, and a host of other presumably causal factors.
• Distortions to behavior: There are many ways in which bureaucra-
cies and professions respond to standardized targets, particularly
when monetary or other incentives are involved. Examples include
suppressing performance for fear that improvements will be used as
baselines for impossible targets or bringing in extra resources dur-

Copyright © National Academy of Sciences. All rights reserved.