Page 26 - The Importance of Common Metrics for Advacing Social Science Theory and Research

The Importance of Common Metrics for Advancing Social Science Theory and Research: A Workshop Summary
http://www.nap.edu/catalog/13034.html

14 THE IMPORTANCE OF COMMON METRICS

which women are a very important component of the labor force. In ad-
dition, it turns out that education alone generates a better scale than com-
posite indexes, such as those that include both income and education. The
story of the Duncan SEI is a case history of the rise and fall of a standard
sociological measure that became obsolete over time. There is now an in-
ternational socioeconomic index developed by Treiman and colleagues that
is well suited for comparative work. 6

Normalization of Metrics

Multiplicative scales and log transformations are analytic schemes for
normalizing metrics to achieve comparability in levels or effects. Hauser
discussed how such transformations can range from truly useful to utterly
misleading.
One of the simplest and most powerful transformations, under appro-
priate circumstances, is the log transformation. Because log transformations
reduce positive skew and increase negative skew, it is often desirable to add
a constant (start value) before transforming the original variable.
Both location and metric affect comparisons. Hauser pointed out that
interaction effects may be an artifact of differences in location on the same
scale (when effects are not linear). As an example, he pointed to compari-
sons of returns to education among blacks and whites in the United States
(Hauser et al., 2000). Vignette measurement circumvents some of these
problems by trying to calibrate individual scales, rather than trying to as-
sume that there is a common scale for everyone in ordering objects (see
King et al., 2004).
Hauser turned next to meta-analysis, which typically involves statisti-
cal analyses of the combined results from different analytic studies. In his
view, meta-analysis is vastly inferior to pooled analyses of primary data.
In particular, the dominant use of “effect size” in standard deviation units
does not create common understanding, since these units are not necessar-
ily in the same metric and are not real units. As data sharing increases and
as people’s capabilities to use multiple sources of data increase, his hope is
that meta-analysis will become less important.
Hauser’s selective review of past efforts provides a cautionary account
of the prospects for useful and valid common metrics in the social sciences.
He ended his presentation with seven lessons for the creation of sound,
standard, and comparable social, economic, and behavioral measures:

6 See Ganzeboom, De Graaf, and Treiman (1992) for discussion about the International
Socioeconomic Index of Occupational Status.

Copyright © National Academy of Sciences. All rights reserved.