Page 33 - The Importance of Common Metrics for Advacing Social Science Theory and Research

The Importance of Common Metrics for Advancing Social Science Theory and Research: A Workshop Summary
http://www.nap.edu/catalog/13034.html

MEASUREMENT IN THE SOCIAL SCIENCES 21

and market prices omit much of what goes into people’s preferences. For
example, public goods provided by the government enter into the national
accounts at cost, since there is no way of valuing them. The environment is
a nonmarketed shared resource, nothing of which appears in the national
accounts. Individual consumption in families and households, as well as
future (or lifetime) consumption based on a set of expectations under states
of uncertainty, also are not directly measured. When markets are absent,
there is little alternative but to try direct measurement of “output.”
Willis related a frustrating tale told by Angus Deaton at Princeton Uni-
versity about a largely failed attempt to determine trends in the number or
proportion of people living on less than $1 per day. He used the purchasing
power parity (PPP) approach to develop a measure of the amount of local
currency needed to buy $1 worth of goods in countries around the world.
Extreme poverty measured in this way is how many people live on $1 a day
or less. Deaton could not get sensible results until he incorporated measures
of self-rated well-being from a Gallup poll. He traced the problem with PPP
to a failure to have data on prices on comparable items, such as the qual-
ity of shirts in Kenya, New York, and London. Willis interpreted Deaton’s
experience to reflect not so much the inadequacy of mainstream theory as
the difficulty of measuring the variables demanded by the theory.
He turned next to recent development of measurements that fall outside
the conventional accounting framework used in economics. He observed
that economists are increasingly willing to consider supplementing their
market-based measures with subjective ones. Health is a good example
for which objective measures are hard to come by, and it is not clear if
self-reported measures of health are valid and interpersonally comparable.
Anchoring vignettes can be a way to try to disentangle the rating scale from
“true” value (see Hauser, 2010).
Willis ended with his belief that economics has developed a powerful
method for using market data prices and quantities to create standardized
measures of income and related variables that can be compared across
people, countries, and time. They can be aggregated and disaggregated
by the economic framework, but the framework fails to account well for
goods and services that are produced and consumed outside markets. One
approach to deal with this is to develop new measures of choice sets and
behavior in implicit markets. Another is to relax the economist’s preference
for objective data and revealed preference in favor of subjective measures.
He sees very few measures based on implicit markets or subjective measure-
ment ready for standardization in the sense of official statistics. There is
great value, he said, in having comparable measures available for research
that will allow improvement of new measures. Meanwhile, one needs to
recognize the dangers of using imperfect or incomplete standardized mea-
sures as guides to policy.

Copyright © National Academy of Sciences. All rights reserved.