Page 53 - The Importance of Common Metrics for Advacing Social Science Theory and Research

The Importance of Common Metrics for Advancing Social Science Theory and Research: A Workshop Summary
http://www.nap.edu/catalog/13034.html

INDICATORS 41

Number of Successes (or Failures)
__________________________________
Number at Risk of Success (or Failure)

While it is easier to quantify success or failure in the numerator, Warren
identified a number of scenarios that complicate measuring the denomina-
tor. For example, how should the denominator of a measure account for
migration into or out of a particular geographic area? How should students
who are expelled or otherwise pushed out of high school be counted in the
denominator? When students transfer from one school to another, should
they be counted in the first school’s denominator, the second school’s de-
nominator, neither, or both?
In his overview of status rates, Warren explained that the fraction of the
population that falls into a population subcategory is measured at a given
point in time. For the purpose of describing amounts of human capital in
a population or a geographic area, he addressed how status completion
or dropout rates are imperfect. For example, in the previous presentation,
Pollak described a method to treat all high school credentials as essentially
equivalent; however, this is not necessarily the best approach, because
economists have long questioned the relative labor market value of GEDs,
and little is known about alternative credentials.
To measure a school’s holding power, dropout and completion rates
need to directly and accurately reflect a specific location. This involves the
use of cohort rates, which measure the fraction of individuals who transi-
tion into a particular status among those who share a common status at
the outset. Cohort rates are based on longitudinal administrative data that
school districts and states keep about students. School districts are increas-
ingly using longitudinal tracking systems to follow students over time;
however, there are still problems with the way states and districts define
numerators and denominators in order to lower their dropout rates. Warren
argued that the most effective data would represent each graduating or
incoming student cohort and be made available annually.
He discussed how few trend analyses have been completed, because
measures change over time and cross-state or cross-district comparisons
have been difficult to carry out. In this regard, the movement toward using
standards that were initially proposed in 2008 by the National Governors
Association and the U.S. Department of Education is a step forward. These
standards include restricting the numerator to regular diploma recipients
who obtain diplomas within four years and the denominator to people
who are at risk of getting those diplomas and appropriately accounting for
things like migration. If states consistently implement the standards laid out
by the Department of Education, eventually cohort rates can be compared
over time and across states.

Copyright © National Academy of Sciences. All rights reserved.