Page 128 - Composition in Convergence The Impact of the New Media on Writing Assessment

VALIDITY AMD RELIABILITY 95

numbers of students to satisfy an explanation of what is happening
in the writing classroom and the qualifier to prefer smaller num-
bers of students to provide a general pattern of activities in a com-
position class. What is significant about this point is that writing
departments are now filled with readers from both camps and each
side reads student papers. Although assessment indeed drives in-
struction, recognizing that assessment can be defined through two
opposite theoretical approaches illustrates the fissures that can oc-
cur in departments regarding written evaluation as well as the rifts
that emerge in pedagogy.
The schism that seems to exist in writing assessment practices
from program to program, perhaps even from instructor to in-
structor, mirrors Composition's mistrust of assessment. Few
practitioners and scholars understand tests and measurements, as
Brian Huot (2002) rightly noted. Even fewer recognize that Com-
position is not bound to quantitative definitions of concepts like
validity and reliability to describe what occurs in the classroom. It
is possible to reconfigure these terms to accommodate the flexibil-
ity necessary to discuss the writing process in networked writing
classes and to do so without causing great conflict with social
constructivist pedagogical models. The way to realign validity
and reliability in writing assessment in these computer-enhanced
contexts is to think even more qualitatively about evaluation.
University of Michigan education professor Pamela A. Moss, in
her research on accountable assessment with portfolios (1992),
indicated that frequently a student writer's growth is made evi-
dent by examining the qualitative aspects of the writing. For in-
stance, having instructors look at the increasing levels of
complexity in student problem solving often reveals that there is a
loss of control in mechanics or organization as student writers de-
velop richer interpretations of a text. According to Moss (1992),
there are other subtle indicators of growth like quality of voice
and elaboration. These characteristics tend to be understated in
psychometric approaches and subsumed under broader criteria,
which causes faculty to miss or to misinterpret critical moments
of a writer's development.
In thinking qualitatively about writing assessment in networked
environments, instructors are asked to consider depth instead of
comprehensiveness in evaluation. This means that instead of col-
lecting webfolios or electronically generated assignments from ev-