Page 67 - The Importance of Common Metrics for Advacing Social Science Theory and Research

The Importance of Common Metrics for Advancing Social Science Theory and Research: A Workshop Summary
http://www.nap.edu/catalog/13034.html

SOCIAL SCIENCE CONSTRUCTS 55

poses, and they help to ensure that different procedures measure the same
thing.
Cartwright and Bradburn (2010) proposed a number of general rules,
including that the procedures need to be consistent with definitions of the
concept and the particular representation of them. Empirical regularities are
central to this. Cartwright added that procedures are a way of zeroing in
on the concept to be defined. Most procedures are situation specific; many
procedures zero in on the concept in different ways. In a new context, the
linkage between concept and procedures may not hold.
One of the problems with Ballungen concepts is that the measurement
procedures may violate the commonsense understanding of the concept.
Bradburn considered unemployment to be a good example of this, because
the way in which it is actually measured seems to violate the commonsense
understanding of unemployment (in that it removes discouraged workers
from the denominator). He emphasized that the subjective component can
be very important. The meaning of “looking for work” is somewhat ambig-
uous, especially for youth. In the Current Population Survey, the report on
youth behavior often comes from the parent, and the parent’s view about
whether a child is looking for a job could differ from that of the child.
Another often used measurement procedure is combining different
variables and questions. Bradburn cautioned that it is important to assess
whether the underlying relationship of those variables to other factors is the
same. As an example, he has found the concepts of happiness and satisfac-
tion to have different relationships with age. Yet in the literature to date,
happiness and satisfaction are treated as if they are the same. In fact, they
are related in different ways to underlying concepts.
Bradburn continued that the concepts with different procedures can
suit different purposes. Measures of quality of life, even the ones from the
Patient-Reported Outcomes Measurement Information System, are dif-
ferent for different purposes. Particularly with respect to policy-related
indicators, the explicit values become an important part of the measures.
These indicators, if adopted for a considerable time, become very difficult
to change, because some groups have been advantaged by one set of pro-
cedures, without necessarily having a scientific basis for the choice. Values
and value implications are hard to eliminate.
The kind of distinctions made in Cartwright and Bradburn (2010) have
three major implications. First, common metrics are possible and desired
if the definitions, representations, and procedures are all well specified and
appropriate. Second, when concepts are used for different purposes, so that
the definitions, representations, or procedures are different—or all of the
above—then there will be difficulty getting to common measures. Third,
many policy-related social science concepts lack a firm scientific or theoreti-
cal basis for their definition, and often their definitions depend on values.

Copyright © National Academy of Sciences. All rights reserved.