Page 140 - Composition in Convergence The Impact of the New Media on Writing Assessment

VALIDITY AND RELIABILITY 107

classroom. Furthermore, TOPIC/ICON constructs a prototype for
what larger writing programs can do to be more efficient in
delivering course content in an age of technological convergence. At
the K-12 level, OLR demonstrates a similar effectiveness in working
with younger students' writing. What the OLR and TOPIC/ICON
systems show compositionists is that deep assessment of networked
writing can occur in very different forms to meet various institu-
tional needs. Composition does not have to be dependent on older
understandings of writing assessment to offer legitimate evaluation
methods for electronic texts. It is possible for writing specialists to
construct new assessment models that draw on the two technologies
and still acknowledge validity and reliability, albeit in ways that
break away from Composition's past.
Bob Broad proposed a fresh idea with regard to writing assess-
ment that shows potential for working with electronic texts. In his
book What We Really Value (2003), Broad detailed the Dynamic Cri-
teria Map (DCM). The DCM is a series of circular regions, some
linked, others not, that address varying textual qualities. Two re-
gions, Change in Student/Author and Rhetorical, are linked
through Broad's "epistemic spectrum" (2003, p. 40). Changes in
Student/Author are marked by growth in learning and in revision,
whereas the Rhetorical region is defined by audience awareness and
persuasive abilities.
Broad ranked the epistemic spectrum as the "most substantial cri-
terion" in the model, because it positions affective and moral think-
ing, epistemic knowledge, and intellectual analysis along a
continuum that bisects the Change in Student/Author and Rhetori-
cal constellations (2003, p. 40). An offshoot constellation, Aesthet-
ics, that reflects criteria dependent upon the writer's craft (texture,
creativity, humor, etc.), links to both the Change cluster and the
epistemic knowledge range of the continuum.
The DCM also includes assessment criteria clusters regarding
Agency/Power (author as writer) that intersects with Ethos (author
as person) and a discrete area defined as "part to whole," which
houses the structural elements of writing such as focus, pace, rele-
vance, clarity, flow, and so on (Broad, 2003, p. 40). Two smaller
compartments, "mystery criterion" and "general writing ability,"
are set apart from the larger domains.
The appeal of the DCM for e-texts is in how the model addresses a
full complement of writing needs. Regardless of whether the elec-