Page 129 - Composition in Convergence The Impact of the New Media on Writing Assessment

96 CHAPTER 4

eryone in the class, teachers work with smaller representative
samples of each assignment to examine what happens in the course
over time. In the process, a theory of instruction that is situated to
the course and to the institution develops. Not only is approaching
writing assessment from this perspective more in line with Compo-
sition's interest in cooperative writing, ethnography, protocols,
and discourse analysis, but the inductive method of inquiry com-
mon to qualitative research also parallels more closely a writer's
composing activities. From a practical standpoint when working in
internetworked spaces, qualitative assessment helps faculty con-
struct categories of incidents or events that happen in students' on-
line writing processes that then can be refined and examined for
patterns or themes to describe various relations among the multi-
ple literacies and activities in the class. Once the preliminary con-
nections are made, instructors can integrate the information into a
coherent explanation of what transpires in the computer- en-
hanced classroom using a theoretical framework that corresponds
with their teaching philosophies.
Even in the more quantitative instructional design and develop-
ment circles, qualitative methods are now thought of as having
some validity and reliability compared with a decade or so ago. This
is particularly so when qualitative approaches are used to triangu-
late student test scores or to discuss specific student populations.
This suggests that there is enough of a precedent set in research de-
sign for qualitative assessment to be valid in all writing classes. To
have qualitative writing assessment data stand alone and maintain
validity and reliability without statistical support, however, a level
of confidence must be created for quantitative folks to respond
favorably to the data put before them.
For those who teach in computer-enhanced classrooms, this point
is especially important. I don't mean to scare writing instructors,
but new electronic essay-reading software programs that imitate
variations of the ETS holistic scoring process or the predicate analy-
sis method found in the Intelligent Essay Assessor are gaining pub-
licity. These programs can easily become the type of representation
quantitative researchers recognize as acceptable writing assessment
mechanisms for networked writing classes. The administrative ap-
peal for these systems is understandable; low-cost, high-volume
production that speaks in statistically accurate language. To counter
such a regressive approach to writing evaluation, though,