Page 61 - The Importance of Common Metrics for Advacing Social Science Theory and Research

The Importance of Common Metrics for Advancing Social Science Theory and Research: A Workshop Summary
http://www.nap.edu/catalog/13034.html

INDICATORS 49

OPEN DISCUSSION
Jack Triplett commented on a point raised by both Prewitt and Snipp
about controlling the denominator because it makes the data difficult for
many purposes if ratios are based on different classification systems. As an
example, Snipp pointed out that the U.S. Department of Education is not
using the same categories as the Census Bureau, so the denominator comes
from a different set of categories than the numerator.
Harris Cooper praised the quality of the papers presented, which he
felt were especially valuable in relation to one another. Based on his under-
standing of the day’s presentations, he did not consider it a problem that
common social and health metrics and indices are not possible. It is not
that they are impossible, responded Pollak, but rather it depends on the
definition. For example, if the marriage category is defined only as being
legally married and living together, then that definition can be used in any
data set as an independent and dependent variable. He contended that it is
better to have the raw data in order to see what independent variables are
correlated with a given definition. While it is possible to define some notion
and insist that it is used by everybody, this approach may not be advisable,
he continued. Hauser said that aggregation, rather than data collection or
measurement, is the key issue; the American Community Survey asks for
national origin, and it is a completely open-ended question.
Prewitt and Snipp both expressed concern about the use of genetic
markers in conjunction with racial and environmental characteristics,
thinking that some lines of research should be avoided. Pollak raised a
different topic concerning the benefits and limitations of self-reported race
on the decennial census. On one hand, he said, it raises an interesting be-
havioral theory of what people report, but on the other it is also a topic for
people interested in discrimination. He emphasized that there are different
purposes in a social science context, and it is important to keep them in
mind when considering various research questions. For this reason he is
less concerned than Prewitt and Snipp about incorporating genomic issues
related to medicine.
Taking issue with Prewitt’s preference for administrative data that
comes with associated costs, Grusky was interested in Prewitt’s reaction to
the view that they can have some leverage, since the data are intended for
research purposes. Grusky continued by raising a point regarding Pollak’s
main concern that, in the absence of theory, standardized measurements
would be vulnerable to political manipulation. He suggested that there may
be other ways to protect against manipulation aside from theory, since the
goal is to have consensus, which can be secured in other ways. He offered
the examples of unemployment and official poverty measures as ones that
are not defined by theory but are prevalent in usage. Setting the question of

Copyright © National Academy of Sciences. All rights reserved.