Page 80 - The Importance of Common Metrics for Advacing Social Science Theory and Research

The Importance of Common Metrics for Advancing Social Science Theory and Research: A Workshop Summary
http://www.nap.edu/catalog/13034.html

68 THE IMPORTANCE OF COMMON METRICS

rival of easily accessible data from IQ tests created a huge market for the
use of IQ as the “ability” in economic models of education and educational
and economic success. This resulted in a dominant line of interest involv-
ing the consequences of cognitive ability. There is now a research program
centered in Scotland looking at the correlation between IQ and mortality
(which appears all over the world), but there is nothing in the literature that
explains why the correlation occurs. Hauser argued that it is exceptionally
important to have a few widely accepted measures of self-regulation. In
the Wisconsin Longitudinal Study, Hauser has looked at the IQ-mortality
relationship over a span of 52 years from ages 18 to 68 and found the
expected relationship, which he attributes to a simple explanation: the ef-
fect of IQ is completely mediated by rank in high school class, which he
believes is closely tied to self-regulation, conscientiousness, dependability,
and other regularities in behavior. He further argued that there is a com-
pelling public interest to get the story straight. To accomplish this, widely
accepted metrics are required. He noted that this was also true years ago of
social standing and occupational standing. Rather than novelty, he believes
that something socially useful, which helps to nail down narrowly defined
cognitive measures, will make a difference in people’s lives. Hoyle agreed
with Hauser but was not clear how to move to a widely accepted measure
of self-regulation.
Rick Moser (National Cancer Institute) was intrigued by the idea of
creating incentives for the use of standardized measures. A psychologist
by training, he understands the rewards for innovation in his field but ex-
pressed concern that psychology specifically has suffered as a result in the
building of cumulative knowledge. The National Cancer Institute is creating
a tool to facilitate standardization and has questioned how to create incen-
tives for the use of standardized measures, especially in light of the com-
peting rewards acting against this. He recognizes that some constructs and
associated measures are not ready for standardization, but he questioned
at what point refinement needs to stop and use begin.
Maynard sought to discover ways to encourage people to start with the
best, most relevant measure, improve on it using new data, and ultimately
create cross-walks between studies. She also encouraged making data sets
publicly available after publication. Funding agencies can help by requiring
that contractors and grantees draw on what exists or justify why they need
to deviate. Widespread adoption of measures is more likely if the measures
are publicly or readily available. Maynard said she is aware of a major
ongoing initiative of the Department of Education for a compendium of
measures; other federal agencies also support similar efforts.
George Bohrnstedt also thinks that federal agencies can be influential
in pushing for cooperative agreements and use of common measures. Hoyle
observed that the problem can be one of framing, not just incentives. Once

Copyright © National Academy of Sciences. All rights reserved.