Page 54 - Neural Network Modeling and Identification of Dynamical Systems

42 2. DYNAMIC NEURAL NETWORKS: STRUCTURES AND TRAINING METHODS

FIGURE 2.8 An example of a structural organization for a layered neural network with layer-level parallelism. (A) Feed-
forward ANN. (B) ANN with feedbacks.

ment of time can operate independently from without branches and cycles. In structures with
each other in an arbitrary order or parallel if parallelism at the layer level for networks, as
there is such a technical capability. showninFig. 2.8, both forward “jumps” and
Suppose we have an ANN organized accord- feedbacks can be present. Such structures bring
ing to the “stack of layers” scheme. The logic of nonlinearity to the cause-and-effect chains; in
neuron activation (i.e., the sequence and condi- particular, they provide tree structures and cy-
tions of neuron operation) in this ANN ensures cles.
the absence of conflicts between them. If we in- The cause-and-effect chain should show
troduce a parallelism at the layer level in the which neurons transmit signals to some ana-
ANN, we need to add some additional synchro- lyzed neuron. In other words, it is required to
nization rules to provide such conflict-free net- show which neural predecessors should work
work operation. so that a given neuron receives a complete set of
Namely, a neuron can work as soon as it is input values. As noted above, this is a necessary
ready to operate, and it will be ready as soon condition for the readiness to operate a given
as it receives the values for all its inputs. Once neuron. This condition is the causal part of the
the neuron is ready for functioning, we should chain. Also, the chain indicates which neurons
start it immediately, as soon as it becomes possi- will get the output of this “current neuron.” This
ble. This is significant because the outputs of this indication will be the “effect” part of the cause-
neuron are required to ensure the operational and-effect chain.
readiness for other neurons that follow. In all the considered variants of the ANN
For the particular ANN, it is possible to spec- structural organization, only forward and back-
ify (to generate) a set of cause-and-effect rela- ward links were contained, i.e., connections be-
tions (chains) that provide the ability to monitor tween pairs of neurons in which the neurons
the operational conditions for different neurons from this pair belong to different layers.
to prevent conflicts between them. The third kind of connections that are possi-
For layered feedforward networks with the ble between neurons in the ANN is lateral con-
structures shown in Fig. 2.7, the cause-and- nections, in which the two neurons, between
effect chains will have a strictly linear structure, which the connection is established, belong to