Page 89 - Foundations of Cognitive Psychology : Core Readings

88 Jay L. McClelland, David E.Rumelhart, and Geoffrey E.Hinton

to insist upon the idea of distributed representation.Lashley may have been
too radical and too vague, and his doctrine of equipotentiality of broad regions
of cortex clearly overstated the case.Yet many of his insights into the diffi-
culties of storing the ‘‘engram’’ locally in the brain are telling, and he seemed
to capture quite precisely the essence of distributed representation in insist-
ing that ‘‘there are no special cells reserved for special memories’’ (Lashley,
1950, p.500).
In the 1950s, there were two major figures whose ideas have contributed to
the development of our approach.One was Rosenblatt (1959, 1962) and the
other was Selfridge (1955).In his Principles of Neurodynamics (1962), Rosenblatt
articulated clearly the promise of a neurally inspired approach to computation,
and he developed the perceptron convergence procedure, an important advance
over the Hebb rule for changing synaptic connections.Rosenblatt’s work was
very controversial at the time, and the specific models he proposed were not
up to all the hopes he had for them.But his vision of the human information
processing system as a dynamic, interactive, self-organizing system lies at the
core of the PDP approach.Selfridge’s contribution was his insistence on the
importance of interactive processing, and the development of Pandemonium,an
explicitly computational example of a dynamic, interactive mechanism applied
to computational problems in perception.
In the late 60s and early 70s, serial processing and the von Neumann com-
puter dominated both psychology and artificial intelligence, but there were a
number of researchers who proposed neural mechanisms which capture much
of the flavor of PDP models.Among these figures, the most influential in our
work have been J. A.Anderson, Grossberg, and Longuet-Higgins.Grossberg’s
mathematical analysis of the properties of neural networks led him to many
insights we have only come to appreciate through extensive experience with
computer simulation, and he deserves credit for seeing the relevance of neu-
rally inspired mechanisms in many areas of perception and memory well be-
fore the field was ready for these kinds of ideas (Grossberg, 1978).Grossberg
(1976) was also one of the first to analyze certain properties of the competi-
tive learning mechanism.Anderson’s work differs from Grossberg’s in insist-
ing upon distributed representation, and in showing the relevance of neurally
inspired models for theories of concept learning (Anderson, 1973, 1977); work
on distributed memory and amnesia owes a great deal to Anderson’s inspira-
tion.Anderson’s work also played a crucial role in the formulation of the cas-
cade model (McClelland, 1979), a step away from serial processing down the
road to PDP.Longuet-Higgins and his group at Edinburgh were also pursuing
distributed memory models during the same period, and David Willshaw, a
member of the Edinburgh group, provided some very elegant mathematical
analyses of the properties of various distributed representation schemes (Will-
shaw,1981).Hisinsightsprovide oneofthe sourcesofthe idea of coarse cod-
ing.Many of the contributions of Anderson, Willshaw, and others distributed
modelers may be found in Hinton and Anderson (1981).Others who have
made important contributions to learning in PDP models include Amari (1977),
Bienenstock, Cooper, and Munro (1982), Fukushima (1975), Kohonen (1977,
1984), and von der Malsburg (1973).