Page 122 - Biomimetics : Biologically Inspired Technologies

Bar-Cohen : Biomimetics: Biologically Inspired Technologies DK3163_c003 Final Proof page 108 21.9.2005 11:41pm

108 Biomimetics: Biologically Inspired Technologies

near-largest input intensity. Localized mutual inhibition between cortical neurons (which is known
to exist, but is not included in the above simplified model) then sees to it that there are no additional
winners; even if the control input keeps rising. Note also that the rate of rise of the control signal
can control the width of the band of input excitations (below maximum) for which neurons are
allowed to win the competition: a fast rate allows more neurons (with slightly less input intensity
than the first winners) to become active before inhibition has time to kick in. A slow rate of rise
restricts the winners to just one symbol. Finally, the operation control input to the network can be
limited to be less than some deliberately chosen maximum value: which will leave no symbols
active if the sum of the all neuron’s input excitation, plus the control signal, are below the fixed
‘‘threshold’’ level. Thus, an attractor network confabulation can yield a null conclusion when there
are no sufficiently strong answers. Section 3.1 of the main chapter discusses some of these
information processing effects; which can be achieved by judicious control of a lexicon’s operation
command input signal.
An important difference between the behavior of this simple attractor network model and that of
thalamocortical modules is that, by involving inhibition (and some other design improvements such
as unifying the two neural fields into one), the biological attractor network can successfully deal
with situations where even hundreds of stable x field vector fragments (as opposed to only a few in
the simple attractor network) can be suppressed to yield a fully expressed dominant fragment x k .
This remains an interesting area of research.
The development process of feature attractors is hypothesized by the theory to take place in
steps (which are usually completed in childhood; although under some conditions adults can
develop new feature attractor modules).
Each feature attractor module’s set of symbols is used to describe one attribute of objects in the
mental universe. Symbol development starts as soon as meaningful (i.e., not random) inputs to
the feature attractor start arriving. For ‘‘lower-level’’ attributes, this self-organization process
sometimes starts before birth. For ‘‘higher-level’’ attributes (modules), the necessary inputs do
not arrive (and lexicon organization does not start) until after the requisite lower-level modules
have organized and started producing assumed fact outputs.
The hypothesized process by which a feature attractor module is developed is now sketched. At
the beginning of development, a sizable subset of the neurons of cortical layers II, III, and IV of the
module happen by chance to preferentially receive extra-modular inputs and are stimulated
repeatedly by these inputs. These neurons develop, through various mutually competitive and
cooperative interactions, responses which collectively cover the range of signal ensembles the
region’s input channels are providing. In effect, each such feature detector neuron is simultaneously
driven to respond strongly to one of the input signal ensembles it happens to repeatedly receive;
while at the same time, through competition between feature detector neurons within the module, it
is discouraged from becoming tuned to the same ensemble of inputs as other feature detector
neurons of that module. This is the classic insight that arose originally in connection with the
mathematical concepts of vector quantization (VQ) and k-means. These competitive and coopera-
tive VQ feature set development ideas have been extensively studied in various forms by many
researchers from the 1960s through today (e.g., see Carpenter and Grossberg, 1991; Grossberg,
1976; Kohonen, 1984, 1995; Nilsson, 1965, 1998; Tsypkin, 1973; Zador, 1963). The net result of
this first stage of feature attractor circuit development is a large set of feature detector neurons
(which, after this brief initial plastic period, become largely frozen in their responses — unless
severe trauma later in life causes recapitulation of this early development phase) that have
responses with moderate local redundancy and high input range coverage (i.e., low information
loss). These might be called the simple feature detector neurons.
Once the simple feature detector neurons of a module have been formed and frozen, additional
secondary (or ‘‘complex’’) feature detector neurons within the region then organize. These are
neurons which just happen (the wiring of cortex is locally random and is essentially formed first,
during early organization and learning, and then is soon frozen for life) to receive most of their