Page 85 - Foundations of Cognitive Psychology : Core Readings

84 Jay L. McClelland, David E.Rumelhart, and Geoffrey E.Hinton

A natural extension of this rule to cover the positive and negative activation
values allowed in our example is:
Adjust the strength of the connection between units A and B in
proportion to the product of their simultaneous activation.
In this formulation, if the product is positive, the change makes the connection
more excitatory, and if the product is negative, the change makes the connec-
tion more inhibitory.For simplicity of reference, we will call this the Hebb rule,
although it is not exactly Hebb’s original formulation.
With this simple learning rule, we could train a ‘‘blank copy’’ of the pattern
associator shown in figure 4.12 to produce the B pattern for rose when the A
pattern is shown, simply by presenting the A and B patterns together and
modulating the connection strengths according to the Hebb rule.The size of the
change made on every trial would, of course, be a parameter.We generally as-
sume that the changes made on each instance are rather small, and that con-
nection strengths build up gradually.The values shown in figure 4.13, then,
would be acquired as a result of a number of experiences with the A and B
pattern pair.
It is very important to note that the information needed to use the Hebb rule
to determine the value each connection should have is locally available at the
connection.All a given connection needs to consider is the activation of the
units on both sides of it.Thus, it would be possible to actually implement such
a connection modulation scheme locally, in each connection, without requiring
any programmer to reach into each connection and set it to just the right value.
It turns out that the Hebb rule as stated here has some serious limitations,
and, to our knowledge, no theorists continue to use it in this simple form.More
sophisticated connection modulation schemes have been proposed by other
workers; most important among these are the delta rule; the competitive learn-
ing rule; and the rules for learning in stochastic parallel models.All of these
learning rules have the property that they adjust the strengths of connections
between units on the basis of information that can be assumed to be locally
available to the unit.Learning, then, in all of these cases, amounts to a very
simple process that can be implemented locally at each connection without the
need for any overall supervision.Thus, models which incorporate these learn-
ing rules train themselves to have the right interconnections in the course of
processing the members of an ensemble of patterns.

Learning Multiple Patterns in the Same Set of Interconnections Up to now, we
have considered how we might teach our pattern associator to associate the
visual pattern for one object with a pattern for the aroma of the same object.
Obviously, different patterns of interconnections between the A and B units are
appropriate for causing the visual pattern for a different object to give rise to
thepattern forits aroma.Thesameprinciplesapply, however, and ifwepre-
sented our pattern associator with the A and B patterns for steak, it would
learn the right set of interconnections for that case instead (these are shown in
figure 4.13). In fact, it turns out that we can actually teach the same pattern
associator a number of different associations.The matrix representing the set of
interconnections that would be learned if we taught the same pattern associator