Page 56 - Literacy in the New Media Age

LITERACY AND MULTIMODALITY 45

is of course also ‘interested’; the receiver of the sign treats the sign as a prompt
for interpretation, not for decoding even though that is the popularly accepted
assumption. The receiver sees, hears or feels only the form, the signifiers, and
from their ‘shape’ and on the basis of her or his knowledge of the social
place where the sign/message has come from, and on the basis of her or his
interest, will produce a signified and hence a sign as her or his meaning from it.
This is Peirce’s interpretant. There is a chain of semiosis in which the sign leads
to an interpretant, which itself becomes an object/referent for a new sign in
communication, which is the basis for the forming of yet another interpretant.
Through conventions of many kinds, society keeps this process relatively firmly
controlled, sometimes and in some domains very much so, sometimes and in
some domains hardly at all, not least by conventions around what may be
thought and communicated. If that were not the case communicability would be
lessened or threatened.
In the era of the new technologies of information and communication, mode
and choice of mode is a significant issue. Mode is the name for a culturally and
socially fashioned resource for representation and communication. Mode has
material aspects, and it bears everywhere the stamp of past cultural work, among
other things the stamp of regularities of organisation. These regularities are what
has traditionally been referred to as a grammar and syntax. In the high era of the
book, of writing and of print, choice of mode was not an issue, or seemed far less
so: books were covered in print, though of course, images of various kinds could
also appear. Walls of churches were covered with images, and there were spaces
specially made for statuary. The relation of mode and medium – writing and
book, painting and wall – was then nearly invisible, through the naturalising
effects of long-standing convention. When we can choose mode easily, as we
now can through the facilities of the new media, questions about the
characteristics of mode arise, in ways that they had not really done before: what
can a specific mode do? What are its limitations and potentials? What are the
affordances of a mode? The materiality of mode, for instance the material of
sound in speech or in music, of graphic matter and light in image, or of the
motion of parts of the body in gesture, holds specific potentials for representation,
and at the same time brings certain limitations. Cultures work with these material
affordances in ways which arise from and reflect their concerns, values and
meanings.
One fundamental distinction brought with the materiality of mode is that of
space and of time. Time-based modes – speech, dance, gesture, action, music –
have potentials for representation which differ from space-based modes – image,
sculpture and other 3D forms such as layout, architectural arrangement,
streetscape. The fundamental logics of the two types of mode differ. The logic of
space leads to the spatial distribution of simultaneously present significant
elements; and both the elements and the relations of the elements are resources
for meaning. The logic of time leads to temporal succession of elements, and the
elements and their place in a sequence constitute a resource for meaning. Each of