Page 21 - The Importance of Common Metrics for Advacing Social Science Theory and Research

The Importance of Common Metrics for Advancing Social Science Theory and Research: A Workshop Summary
http://www.nap.edu/catalog/13034.html

MEASUREMENT IN THE SOCIAL SCIENCES 9

Patient-Reported Outcomes Measurement Information System (PROMIS)
at the National Institutes of Health (NIH).
Bohrnstedt ended with a set of ideas for constructing good measures in
which the items reflect constructs:

• Define the concept as carefully as possible, specifying the domain
of meaning.
• Use factor analysis to explore the dimensionality of the concept.
• After determining dimensionality, do a confirmatory factor analysis
to verify.
• Estimate the internal consistency reliability of the measures con-
structed on the basis of the analysis.
• Fit the items for each dimension to a Rasch model.
• If the items will not fit a one-parameter or Rasch model, then fit
them to a two-parameter model.
• Ensure that parameter estimates are invariant for various
subpopulations.
• Develop new items to bolster sparse areas on the latent dimensions.

With respect to index construction, Bohrnstedt observed that in sociol-
ogy, economics, and policy research, in some cases the assumption is that
the indicators define the construct rather than the other way around. This is
sometimes called a “formative” as opposed to a “reflective” model of index
construction. Examples include an index of socioeconomic status, consist-
ing of education, income, and occupation, and the consumer price index,
which is based on a market basket of goods and services. The construct
is in fact determined by or defined by the indicators that go into it. Typi-
cally, the indicators are simply unit-weighted, but in some cases they are
weighted on the basis of theory, differential utilities, or other preferences
(e.g., relative importance based on a community survey). One can estimate
the weights of the indicators if there are multiple indicators and multiple
causes (the MIMIC model).

COMPARABLE METRICS: SOME EXAMPLES
Robert Hauser (Division of Behavioral and Social Sciences and Edu-
cation, National Research Council, Washington, DC, and Vilas Research
Professor, Emeritus, University of Wisconsin, Madison) reflected on the
tradeoff inherent in standardization. In the social, behavioral, and eco-
nomic sciences, standardization of measures can help the accumulation of
evidence because it permits valid comparisons across time, place, or units
of observations (e.g., persons, families, settings, localities, organizations).
Standardization also can create common understandings, when measure-

Copyright © National Academy of Sciences. All rights reserved.