Page 41 - The Importance of Common Metrics for Advacing Social Science Theory and Research

The Importance of Common Metrics for Advancing Social Science Theory and Research: A Workshop Summary
http://www.nap.edu/catalog/13034.html

MEASUREMENT IN THE SOCIAL SCIENCES 29

expression of preference for death and (2) suicide suggests that there are
states worse than dead.
The challenge is how to incorporate these opinions into a data set and
model them mathematically in a nonarbitrary way. The analytic methods
are not yet developed, and not all index developers agree that states worse
than dead should be allowed. In fact, there are some who refuse to believe
that such states exist. According to Fryback, the only tool available right
now is to average all the different points of view. In comparing the different
indexes, Fryback speculated that it is more a matter of preference in scaling
rather than a substantive issue. He wondered if it would make sense to try
to reach different preference subgroups with different scales.
Hauser is not convinced that it is necessary to obtain different evalua-
tions for different population subgroups. It struck him that a good quality
of life metric would need to demonstrate some invariant properties for the
ratings across different populations and different segments of the same
geographically defined population. Otherwise, it would be difficult to make
sense of those as utilities in an aggregate analysis. Fryback pointed out
that there is nothing in the theory to suggest that everyone has the same
underlying set of preferences that would lead to such invariance. As there
is no way to assign people to one preference or another, he saw no way
around needing to ask respondents their preferences. However, he pointed
out that many different HRQoL systems order the states and scale them in
approximately the same way.
Paul Courtney (National Cancer Institute) expressed concern about
the trade-off between an overly reductionist approach and fidelity of mea-
surement. Fryback agreed that the tension between essentially descriptive
detail and the ability to summarize aggregated higher levels with standard
measures is very real for health measures. His interest has mostly been in
the measures that aggregate rather than disaggregate for deep understand-
ing of pathways to outcomes. But he pointed to the WHO aggregate mea-
sure, which includes an extensive list of environmental factors (e.g., curb
cuts) that can greatly affect the quality of life for someone with restricted
mobility. Fryback further believes that the social environment must be in-
cluded more than it has been. This would include consideration of whether
a person can interact with friends, perform a job role, engage in outside
social activities, have intimacy. Fryback saw the potential for PROMIS in
reinforcing the idea of standardized patient-reported outcome measures
across the NIH and across the broad front of medicine.
Robert Michael (University of Chicago) directed attention to the dis-
tinction between standardization and harmonization. Harmonization has
more to do with coordination, and it is often encouraged as a way to
facilitate joint analyses and thus preferred to rote standardization. Willis
described how harmonization has been a major issue for the Health and

Copyright © National Academy of Sciences. All rights reserved.