Page 23 - The Importance of Common Metrics for Advacing Social Science Theory and Research

The Importance of Common Metrics for Advancing Social Science Theory and Research: A Workshop Summary
http://www.nap.edu/catalog/13034.html

MEASUREMENT IN THE SOCIAL SCIENCES 11

• The 1992 National Adult Literacy Study reported five levels of
literacy, based on four cutoff points set at equal intervals, with-
out specific descriptors, that presumably indicate discrete breaks
in competence. From this score distribution, it is not possible to
determine the number of people who are considered illiterate in
the United States. The National Center for Education Statistics,
when it was about to undertake the successor National Assess-
ment of Adult Literacy (NAAL) in 2003, asked the National Re-
search Council (NRC) to recommend standards for adult literacy
that could be used in the NAAL and applied retroactively to the
National Adult Literacy Study in order to compare literacy lev-
els across the decade among all adults and specific population
groups. The NRC report Measuring Literacy: Performance Levels
for Adults (National Research Council, 2005) developed five cat-
egories with explicit descriptions corresponding roughly to readi-
ness for successive levels of formal education. The NRC report
concludes from experimental work that the whole enterprise of line
drawing is on very shaky ground.
• The Voluntary National Tests were a 1997 proposal of the Clinton
administration for tests of reading at grade 4 and mathematics
at grade 8 that became a dramatic and failed effort to create a
common metric for the assessment of academic achievement and
changes in it. The proposal was to give the same assessment to all
students nationwide, and individual reports would be shared with
students, parents, teachers, and school administrators. Advocates
believed that this diagnostic information would increase motiva-
tion to improve academic achievement. Hauser said that the project
ultimately died due to strong opposition from Republicans who be-
lieved it would destroy the traditional prerogatives of local school
systems and from minority groups afraid it would stigmatize them.
He mentioned two proposals by Congress for NRC studies to ad-
dress measurement issues in ways that would permit this project to
go forward without giving everyone the same test. The first one, to
equate the scales of existing tests, was considered not feasible. The
second proposal, to insert modest numbers of existing items from
national assessments into existing tests on state assessments, also
was rejected because of substantial differences in context or admin-
istration between the state and national testing programs. Hauser
was struck by the fact that Congress directly addressed technical
issues of comparability in measurement, at least attempting to
establish national comparability in the measurement of individual
academic performance in its proposals to the NRC.

Copyright © National Academy of Sciences. All rights reserved.