Page 77 - The Importance of Common Metrics for Advacing Social Science Theory and Research

The Importance of Common Metrics for Advancing Social Science Theory and Research: A Workshop Summary
http://www.nap.edu/catalog/13034.html

SOCIAL SCIENCE CONSTRUCTS 65

• Although attempts at standardization would be premature, there
are advantages to working toward standardization and a common
metric while allowing metric diversity to continue.

DISCUSSION

Rebecca Maynard (University of Pennsylvania) said that the presenta-
tions in this session collectively have done a good job of modeling what
is often desirable, and sometimes not, about common metrics in the social
sciences. They also illustrated for her the limitations of moving too quickly
to common metrics. She then made a number of observations.
She first observed that even when the science and technology for devel-
oping common metrics exist, there is a time and place for common metrics.
Cartwright and Bradburn (2010) laid out a three-step process of defining
what is to be measured, selecting the metric for measuring it, and applying
the metric. These same steps are also the gatekeepers demarking readiness
for common metrics. The current poverty index came about because there
was a readiness—a need in the war on poverty, a ready metric, and an
ability to apply that metric. There has been little progress to change this
measure—despite very good work by NRC and other researchers demon-
strating all the pitfalls of the current measures and other ways to measure
poverty better—not only in large part because of inertia, but also because
there has been no compelling reason to adopt an alternative.
One of the areas in which Maynard hopes common metrics will be
developed is what she termed 21st-century skills, which are skills needed to
improve the labor market readiness of those at the bottom of the skills dis-
tribution and national productivity. Such vocational skills include aspects
of self-regulation (or social competence), the ability to take direction, and
reading. It seemed to her that the research literature may provide a strong
foundation for understanding what to measure as well as the psychometric
capacity to develop such a metric. Although none of the papers explicitly
cautioned against creating common measures “before their time,” she be-
lieves that the papers by Grusky and Hoyle came close.
Maynard next observed that there is a temptation to clump concepts—
the things to be measured—under neat labels and to want common mea-
sures for them. In some cases, she surmised, consensus and utility might
be much quicker to achieve for narrower concepts. In each of the three
domains considered in this session—poverty, social mobility, and self-
regulation—the concepts to be measured could well be context specific. She
noted Michael’s point about why different definitions of poverty might be
needed or different measures advantageous if the intent is to apply the mea-
sure cross-nationally. The concept of poverty also might differ if the focus is
on children, prime age adults, or the elderly. Similarly, she noted Grusky’s

Copyright © National Academy of Sciences. All rights reserved.