Page 90 - Foundations of Cognitive Psychology : Core Readings

The Appeal of Parallel Distributed Processing 89

Toward the middle of the 1970s, the idea of parallel processing began to have
something of a renaissance in computational circles.We have already men-
tioned the Marr and Poggio (1976) model of stereoscopic depth perception.
Another model from this period, the HEARSAY model of speech understand-
ing, played a prominent role in the development of our thinking.Unfortu-
nately, HEARSAY’s computational architecture was too demanding for the
available computational resources, and so the model was not a computational
success.But its basically parallel, interactive character inspired the interactive
model of reading (Rumelhart, 1977), and the interactive activation model of
word recognition (McClelland & Rumelhart, 1981; Rumelhart & McClelland,
1982).
The ideas represented in the interactive activation model had other pre-
cursorsaswell.Morton’s logogen model (Morton, 1969) was one of the first
models to capture concretely the principle of interaction of different sources
of information, and Marslen-Wilson (e.g., Marslen-Wilson & Welsh, 1978) pro-
vided important empirical demonstrations of interaction between different
levels of language processing.Levin’s (1976) Proteus model demonstrated the
virtues of activation-competition mechanisms, and Glushko (1979) helped us
see how conspiracies of partial activations could account for certain aspects of
apparently rule-guided behavior.
Our work also owes a great deal to a number of colleagues who have been
working on related ideas in recent years.Feldman and Ballard (1982) laid out
many of the computational principles of the PDP approach (under the name of
connectionism), and stressed the biological implausibility of most of the pre-
vailing computational models in artificial intelligence.Hofstadter (1979, 1985)
deserves credit for stressing the existence of a subcognitive—what we call
microstructural—level, and pointing out how important it can be to delve
into the microstructure to gain insight.A sand dune, he has said, is not a
grain of sand.Others have contributed crucial technical insights.Sutton and
Barto (1981) provided an insightful analysis of the connection modification
scheme we call the delta rule and illustrated the power of the rule to account for
some of the subtler properties of classical conditioning.And Hopfield’s (1982)
contribution of the idea that network models can be seen as seeking minima
in energy landscapes played a prominent role in the development of the
Boltzmann machine and in the crystallization of ideas on harmony theory and
schemata.
The power of parallel distributed processing is becoming more and more
apparent, and many others have recently joined in the exploration of the capa-
bilities of these mechanisms.We hope this chapter represents the nature of the
enterprise we are all involved in, and that it does justice to the potential of the
PDP approach.

Acknowledgments
This research was supported by Contract N00014-79-C-0323, NR667-437 with the Personnel and
Training Research Programs of the Office of Naval Research, by grants from the System Develop-
ment Foundation, and by a NIMH Career Development Award (MH00385) to the first author.