Page 47 - Literacy in the New Media Age

36 LITERACY IN THE NEW MEDIA AGE

done with them has differed also. The theoretical change is from linguistics to
semiotics – from a theory that accounted for language alone to a theory that can
account equally well for gesture, speech, image, writing, 3D objects, colour, music
and no doubt others. Within that theory, the language-modes – speech and
writing – will also have to be dealt with semiotically; they are now a part of the
whole landscape of the many modes available for representation – though of
course special still in that they have a highly valued status in society and, in the
case of speech, certainly still carry the major load of communication.
The terms in that new theory that I will consider – even if only briefly – are
inevitably a selection of those that one ought to look at. I have selected those
which most prominently, obviously, tellingly, show a necessary new way of
thinking about literacy. There is, first and foremost, the concept of meaning, and
associated concepts such as learning and creativity. Something needs to be said
about semiotics and semiosis. Clearly, the concept of mode is crucial in a
multimodal theory of literacy, and so are associated concepts such as inherent
and culturally made affordance, modal specialisation, functional load and
materiality. In a semiotic theory the foundational concept is that of the sign;
associated with that are concepts such as interest, analogy, and metaphor. In my
conception of sign it is both motivated and conventional, which is certainly not
the position taken in the mainstream of thinking. In a semiotic theory, concepts
such as representation and communication, and interpretation and articulation,
are clearly right in the centre of attention.
A new theory of meaning cannot do without the concept of transformation; it
explains how the modal resources provide users of the resource with the ability
to reshape the (form of the) resources at all times in relation to the needs of the
interests of the sign-maker. Transformation needs to be complemented by the
concept of transduction. While transformation operates on the forms and
structures within a mode, transduction accounts for the shift of ‘semiotic
material’ – for want of a better word – across modes. This relates entirely to the
(psychological) processes of synaesthesia, which clearly have a semiotic
analogue. It is in the realm of synaesthesia, seen semiotically as transduction and
transformation, that much of what we regard as ‘creativity’ happens. A theory
that deals with multimodality comes up against the need for a usable definition
of text, given that our present sense of text comes from the era of the dominance
of the mode of writing, and the dominance of the medium of the book. We need
to become clear how we wish to use the term text and the units internal to it; we
need at the same time to be clear about the principles of organisation and shaping
of text, such as coherence and cohesion. There are then the other principles of
organisation which shape text, above all genre and discourse. And in a way that
was not obvious before the era of the new media of information and
communication, it is absolutely essential now to consider the sites and media of
the appearance of text, above all the page and the screen.
So something needs to be there in the theory about the (new and the old) media
of information and communication, and their facilities. The world of