Page 30 - The Importance of Common Metrics for Advacing Social Science Theory and Research

The Importance of Common Metrics for Advancing Social Science Theory and Research: A Workshop Summary
http://www.nap.edu/catalog/13034.html

18 THE IMPORTANCE OF COMMON METRICS

He recognized that these are empirical questions and not ones that can be
settled easily.
Nancy Cartwright (University of California, San Diego, and London
School of Economics and Political Science) emphasized the need to consider
sociology and politics not only outside the academic community but also
within it. She observed that there can be pressure in the academic commu-
nity to use the measures of one’s supervisor or to pursue the kind of results
that are likely to bring professional rewards.
Harris Cooper (Duke University) turned the discussion to meta-
analyses, contending that a more modern view of meta-analyses sees them
as not providing definitive answers but perhaps setting the stage for where
one should look to define the next experiment or investigation. He ac-
knowledged that meta-analyses can only be as good as the studies that are
included in them. He sees his colleagues in medicine as leading the way with
regard to use of what they refer to as individual patient data meta-analyses.
Hauser responded that he sees the challenge as going from effect sizes in
different studies to metrics that have more meaning. He is convinced of the
need for overlapping metrics in different studies in order to get to a real
metric in the course of analysis.

WHAT CAN BE LEARNED FROM THE ECONOMIC SCIENCES?

Robert Willis (University of Michigan) provided some history on stan-
dardization, touching on the politics associated with standardizing mea-
sures in economics before turning attention to the U.S. national accounts.
National accounts represent a standardization of method and approach
that has been quite powerful yet incomplete in a fundamental way. There
are ways to make them more complete by essentially using extensions of
standard methodology, such as gathering better data and developing better
theory. Willis discussed another approach, which is to complement so-
called objective measures with more subjective ones. He also argued that
established statistical agencies have had to apply economic theory in order
to produce economic data that are useful and credible for science and for
policy.

Historical and Political Considerations
Willis began by observing that because economics is so directly relevant
to policy and politics in a democratic society, the development of standard-
ized economic data has gone hand in hand with the development of the idea
of data in public service. He recounted the history of the formation of the
National Bureau of Economic Research (NBER) to illustrate the tradition
in the field of connecting facts (data) and policy. Founded in 1920 as a

Copyright © National Academy of Sciences. All rights reserved.