Page 81 - Foundations of Cognitive Psychology : Core Readings

80 Jay L. McClelland, David E.Rumelhart, and Geoffrey E.Hinton

working memory.In PDP models, though, this is not the case.In these models,
the patterns themselves are not stored.Rather, what is stored is the connection
strengths between units that allow these patterns to be re-created.In the Jets
and Sharks model, there is an instance unit assigned to each individual, but
that unit does not contain a copy of the representation of that individual.In-
stead, it is simply the case that the connections between it and the other units in
the system are such that activation of the unit will cause the pattern for the in-
dividual to be reinstated on the property units.
This difference between PDP models and conventional models has enormous
implications, both for processing and for learning.We have already seen some
of the implications for processing.The representation of the knowledge is set
up in such a way that the knowledge necessarily influences the course of pro-
cessing.Using knowledge in processing is no longer a matter of finding the
relevant information in memory and bringing it to bear; it is part and parcel of
the processing itself.
For learning, the implications are equally profound.For if the knowledge is
the strengths of the connections, learning must be a matter of finding the right
connection strengths so that the right patterns of activation will be produced
under the right circumstances.This is an extremely important property of this
class of models, for it opens up the possibility that an information processing
mechanism could learn, as a result of tuning its connections, to capture the
interdependencies between activations that it is exposed to in the course of
processing.
In recent years, there has been quite a lot of interest in learning in cognitive
science.Computational approaches to learning fall predominantly into what
might be called the ‘‘explicit rule formulation’’ tradition, as represented by the
work of Winston (1975), the suggestions of Chomsky, and the ACT* model of
J.R.Anderson (1983).All of this work shares the assumption that the goal of
learning is to formulate explicit rules (propositions, productions, etc.) which
capture powerful generalizations in a succinct way.Fairly powerful mecha-
nisms, usually with considerable innate knowledge about a domain, and/or
some starting set of primitive propositional representations, then formulate
hypothetical general rules, e.g., by comparing particular cases and formulating
explicit generalizations.
The approach that we take in developing PDP models is completely different.
First, we do not assume that the goal of learning is the formulation of explicit
rules.Rather, we assume it is the acquisition of connection strengths which al-
low a network of simple units to act as though it knew the rules.Second, we do
not attribute powerful computational capabilities to the learning mechanism.
Rather, we assume very simple connection strength modulation mechanisms
which adjust the strength of connections between units based on information
locally available at the connection.Our purpose is to give a simple, illustrative
example of the connection strength modulation process, and how it can pro-
duce networks which exhibit some interesting behavior.

Local versus Distributed Representation Before we turn to an explicit consider-
ation of this issue, we raise a basic question about representation.Once we
have achieved the insight that the knowledge is stored in the strengths of the