Page 14 - The Importance of Common Metrics for Advacing Social Science Theory and Research

The Importance of Common Metrics for Advancing Social Science Theory and Research: A Workshop Summary
http://www.nap.edu/catalog/13034.html

2 THE IMPORTANCE OF COMMON METRICS

The planning committee considered a large range of issues in designing
the workshop and selecting the presentations and participants. For exam-
ple, it might seem as if the benefits of common metrics are obvious. Just as a
common language facilitates learning and communication of knowledge for
many purposes, so do common metrics facilitate cumulative and compara-
tive research and its dissemination for policy, practice, and common un-
derstanding. However, the importance attached to common metrics varies
tremendously across the behavioral and social sciences. In economics, there
is a history of reliance on theory to define measures, although that is not
always the case, and the development of standardized economic measures
has accompanied the development of the idea of data in the public service.
In the health field, a diverse set of morbidity-based indicators suggests that
less arbitrary ways of summarization are needed, with the Patient-Reported
Outcomes Measurement Information System offering a roadmap for the
future. And in psychology, in the cases where psychological processes lack
an overall theory, a reward structure has developed that tends to place a
premium on inventing new measures for the same construct rather than
encouraging the use of common metrics.
The benefits of standardized measures depend ultimately on their ac-
ceptance by the research and policy communities. Use drives measures in
the first place and therefore whether they are standardized. Measurement
must begin with the end in mind, and, if common metrics are the goal, then
their purposes must be considered. That said, one size does not fit all. In this
regard, it may be that a common metric per se, is not the ideal, but rather
a few metrics widely used.
Another issue considered by the planning committee is that different
metrics serve different purposes. When no measure is a candidate for wide-
spread application, the use of multiple measures can help to triangulate a
construct and to test the robustness of effects across different operational
definitions. Thus, harmonization of measures might be possible when stan-
dardization is not. Scientists tend to favor harmonization because it reflects
the competition of ideas, and persistent use is evidence of a measure’s util-
ity. Harmonization is seen as a form of standardization established among
scientists, not imposed on them.
Although the original intent of the workshop was to focus on the im-
portance of common metrics for advancing social science theory, in fact the
discussion centered predominantly on how theory can inform measurement
and on how common metrics can inform policy. Because common metrics
require common concepts and construct validity, agreeing on an underlying
theory is important. Sometimes theory is necessary but not sufficient for
metric development in the social sciences. Often the lack of strong theories
is reflected in the dearth of well-accepted common metrics. At other times,

Copyright © National Academy of Sciences. All rights reserved.