Page 15 - The Importance of Common Metrics for Advacing Social Science Theory and Research

The Importance of Common Metrics for Advancing Social Science Theory and Research: A Workshop Summary
http://www.nap.edu/catalog/13034.html

INTRODUCTION 3

it is impossible to measure the variables demanded by the theory. A consis-
tent theme at the workshop was the paramount need for theory as well as
for a public policy purpose in motivating standardization of measurement
for a particular construct.
Good theory and good measurement are often prerequisites for a stan-
dardized measure. Sometimes a measure is introduced and becomes popular
and thus is accepted as the standard. Sometimes the need for or utility of a
measure drives the momentum toward a standardized measure. Sometimes
a concept is based on a theory that is widely accepted in the scientific com-
munity and that prescribes how the concept is to be measured. The ability
to develop a standardized measure thus depends in part on the state of
theory in different fields.
Although theory guides measurement for scientific purposes, political
judgments often influence the development of standardized measures. The
more consequential a measure is for policy, the more likely that politics
will override science in establishing a standardized measure. And of course
how a social concept, such as poverty or disability, is measured has serious
policy implications. The standardization of measures is a social and po-
litical process involving negotiation. In some situations, what is measured
may be less important than how it is perceived and classified. An example
is the challenge of assessing change that involves not only aging but also
the perception of the change with age. Skepticism often accompanies met-
rics that are generated from a process that is too obviously political. The
integrity of statistical agencies is more easily maintained if the construction
of measures is guided by accepted theories and is as resistant as possible to
political and other pressures.
The social and political context of the academic community is another
consideration. Even when there is benefit to standardization, the incentives
to develop common metrics may be inadequate, especially in fields that tend
to reward the development of novel methods, concepts, and constructs or
new measures for the same construct.
Workshop participants had diverse ideas responding to the question of
what the research community can do to foster common metrics when they
are warranted. If the process of adopting an official standardized measure
for policy purposes is transparent, that may create an opportunity for the
scientific community to weigh in on its scientific suitability. Because com-
mon concepts and constructs are measured differently by different disci-
plines, it is important to learn how each one uses terms and interprets their
connotations and denotations. Improvements in theory may come from
greater interactions among the social sciences, as well as between these
disciplines and others, with a movement toward greater interdisciplinary
research. Agreeing on the type of data to collect could be another way of
promoting common metrics. The use of common metrics also can be ef-

Copyright © National Academy of Sciences. All rights reserved.