Page 52 - Literacy in the New Media Age

LITERACY AND MULTIMODALITY 41

concern with form in one mode – if we see language-as-such as a mode, or two
modes – if we see speech and writing as distinct modes (as I do), to a concern
with form-and-meaning in many modes. The move is away from a theory in
which form is dealt with separately from meaning. It is also a move from the
assumption – implicitly or explicitly held – that linguistic theory can provide a
satisfactory and generally applicable account of representation and
communication, to the realisation that we need a theory which is not specific to,
or derived from, one mode but which applies to all modes. Mainstream
linguistics has largely focused on form – in semiotic terms, the signifier;
meaning had been exported to peripheral disciplines – semantics, pragmatics,
socio-linguistics, stylistics. For most of the twentieth century, linguistics has
been the science of the signifier. Semiotics by contrast is the science of the sign,
a fusion of form/signifier and meaning/signified. Semiotics promises to provide
categories which apply to representation and communication in all modes
equally. At the same time, that semiotic theory will tell us that when we deal
with a mode at a more specific level we need to use terms and descriptions which
pertain to that specific mode. But the terms that deal with a specific mode – let
us say writing – will still be semiotic terms, not the terms of linguistic theories.
There is no switching of theories as we move from one level – of multimodal
description – to another – of specific mode description.
Semiotics has been the domain of two large schools of thought; one deriving
from the work of the Swiss linguist Ferdinand de Saussure, the other deriving
from the work of the American philosopher Charles Sanders Peirce. The
semiotics of Saussure bears recognisable traces of its origins in the historical
linguistics of the nineteenth century. In it, the sign is taken to be an arbitrary
combination of form and meaning, of signifier and signified, a combination
which is sustained by the force of social convention. In the example usually
quoted, Saussure said that even though the object in the world referred to by the
word tree in English or arbre in French is the same object, the sound-forms
which represent this same object in the two languages are very different, proving
that the relation of form and meaning was an arbitrary one. This embodies a
fundamental error, a confusion which has gone unrecognised by and large, and
endlessly repeated. It is a mistake about levels and forms: the level of the
signified tree – the meaning – is matched by the level at which the signifier is
lexical form – the word: not phonetic or phonological form as Saussure is said to
have stated. In Saussure’s formulation the level of meaning is mismatched with
the level of sound; meaning is thought to be realised in sound. But the matching
of signified with signifier is always like with like, and realisation is like with like.
The signifier/form for the signified/meaning tree is the lexical form/signifier
‘tree’.
In Peirce’s semiotics the focus is less on the internal constitution of the sign
than on the uses of the sign by readers/users, and on the relation of the sign to
that which it represents. Peirce focused on what the sign represented, on the
object/referent in the world, on how it was interpreted, assuming that there was