Page 118 - Composition in Convergence The Impact of New Media On

Chapter 4

Rethinking Validity and Reliability
in the Age of Convergence

The commonsense notion in Composition Studies is to create as-
sessment strategies that correspond to our pedagogical practices.
When writing teachers use the traditional quantitative under-
standings of validity and reliability to evaluate their students'
e-texts, instructors are doing the opposite of what past wisdom
suggests. New methods and processes in technology, similar to
new methods and processes in the teaching of writing, require us-
ers to reexamine older practices that measure how far, how fast,
or how accurate is the recent change. Often this reweighing of ear-
lier ways comes from inaccuracies discovered during an evalua-
tion, from a shift in perspective that allows a different view to
emerge, or from more information learned over time. For writing
assessment in an age of convergence, all three possibilities con-
tribute to the importance of rethinking the old standby concepts
of validity and reliability to address students' written competence
because computers have provided a (r)evolutionary movement in
the teaching of writing.
Validity and reliability are the two epistemiological cornerstones
of assessment, this much writing teachers know. Because reliability
is easier to define, let me start there. Simply put, reliability refers to
the ability to consistently give the same answer at different points in
time. Reliability depends on three factors: the stability of a result to
withstand time, the internal consistency of performance along a

87