Page 132 - Composition in Convergence The Impact of the New Media on Writing Assessment

VALIDITY AND RELIABILITY 99

be concise enough to appease administrators and other officials
who typically understand quantitative data. Although this idea
sounds Utopian, it is not. Instead of examining the dilemma of eval-
uation from its traditional tests and measurement roots, writing
instructors can look to a more congruent area for communication
convergence—the media. Compositionists can find innovative ave-
nues of critique and commentary in media research that can trans-
form older notions of writing assessment without sacrificing
validity and reliability.
One new way to think about writing assessment in networked en-
vironments is what I call "deep assessment." Deep assessment arises
from the work of two different compositionists writing almost a de-
cade apart, Margaret Himley (1991) and Ann Watts Pailliotet (1999).
The concept underlying deep assessment emerges from Himley's
"deep viewing" techniques (1991) that were applied to children's
writing. In 1999, Pailliotet adapted deep viewing to accommodate
critique of visual and electronic texts. In the deep assessment ap-
proach, I modify Pailliotet's (1999) and Himley's (1991) ideas to ini-
tiate a postmodern turn in the evaluation of writing.
As with deep viewing, deep assessment reflects a three-tiered
approach. Together, teachers and students, as participant observ-
ers, amass multiple data sources and artifacts that lead to describ-
ing elements of the texts. These descriptions form the basis for
responses and interpretations of what is found in the texts.
Whether in teams that divide the responsibilities of deep evalua-
tion or as single evaluators, the instructors write comments, no-
tations, or sketches about the material in front of them as talk
begins about each selection. This talk unfolds into interconnected
discussion, and the written comments emerge as the artifacts that
concretize the evaluators' exchanges.

TWO STRATEGIES FOR IMPLEMENTING DEEP ASSESSMENT

Using qualitative research to invert the traditional meanings of va-
lidity and reliability in assessment is important for documenting the
evolution of writing and writing instruction that takes place when
extensive computer use is introduced to the composition classroom
experience. Because computer-enhanced writing instruction is fre-
quently a fluid series of exchanges among writers and because the
products that arise from networked classes are frequently seamless