Page 86 - The Importance of Common Metrics for Advacing Social Science Theory and Research

The Importance of Common Metrics for Advancing Social Science Theory and Research: A Workshop Summary
http://www.nap.edu/catalog/13034.html

74 THE IMPORTANCE OF COMMON METRICS

the private sector. Finally, a bipartisan public-private effort referred to as
SUSA (State of the USA) publicizes on a regular basis a set of indicators
across all sectors of economics and society and the environment, in an ef-
fort to inform the democratic process. So there is attention to making some
indicators easily available to the broad public.
Geoff Mulgan said that it is helpful to consider the three sets of inter-
est groups: the scientific community, the government, and the public. In
his view, the scientific community has an obligation to itself, to science,
and to a degree to the public but not to the state. The closer any indicator
gets to being used for actual administrative decisions, as with the poverty
indicator, the less appropriate it is for the scientific community to lend its
legitimacy to it because of the risks of distortion. However, to treat any
indicator as essentially a feedback system, there are different interests in
place as to what counts as good feedback. For the scientific community,
there is a lengthy time scale, cumulative knowledge, etc. For the public,
one of the criteria could be whether to hold the state to account. Mulgan
observed that different indicators will respond to these three interests in
different ways at different times.
Arthur Kendall (U.S. General Accounting Office, retired) shared his
perspective as a social psychologist and mathematical statistician. He ad-
vised that when dealing with a concept in a particular construct, it is impor-
tant to look across disciplines to see what connotations and denotations the
terms have in other disciplines. Ontology could be semantics. It is important
to pay attention to how other people are using the concepts. He believes
that an important role of the scientific community is to facilitate commu-
nications among the disciplines and between the disciplines and the policy,
intelligence, government, and current administration and congressional
groups. He added that if something is incomplete, that does not mean it is
wrong. He also pointed to the importance of level of analysis—for example,
a change in the number of children counted as proficient is not the same as
a change in the number of those whose proficiency has changed.
Robert Hauser underscored the importance of persistence in getting a
measure accepted. Measurement breakthroughs can take a long time. The
fundamental measurement work that showed how old the universe is and
that it is expanding was based on measurements that began in 1974. An-
other example is Measuring Poverty, the 1995 National Research Council
report. It has persisted and perhaps may yet have the kind of effect that
was originally intended. There was recently action in Congress to move it
forward, championed by Mayor Michael Bloomberg in New York. A third
example is the addition of occupational mobility questions to the Survey
of Income and Program Participation (SIPP). Despite initial sentiments that
there was no national interest in measuring social mobility, Hauser and his

Copyright © National Academy of Sciences. All rights reserved.