Page 95 - Machine Learning for Subsurface Characterization

Shallow neural networks and classification methods Chapter 3 79

testing dataset, which can be as high as 50-50 split between training and testing
datasets. Due to the limited size of data available for building the two ANN-
based models, 85% of the dataset is randomly selected to create the training
dataset, and the remaining 15% forms the testing dataset. Notably, it is of crit-
ical importance that each feature and target in the original dataset, testing data-
set, and training dataset should have similar statistical properties and
distributions.
We intend to build two ANN models to synthesize the 64 discrete ampli-
tudes corresponding to the 64 T 2 bins. The first model simultaneously predicts
all the 64 values of the T 2 bins, whereas the second model predicts 6 parameters
that define a bimodal Gaussian distribution, which approximates the 64 discrete
values of the T 2 bins. Each ANN model is built with two hidden layers, which is
enough for most of the function-approximation problems of similar complexity
as our study. Hidden layers lie between the output layer and the input layer. In
this study, the input layer takes in 22 conventional and inversion-derived logs
and 5 categorical flags. The output layer for first ANN model contains 64 neu-
rons, and that of the second ANN model contains 6 neurons. All neurons of out-
put and hidden layers are connected to all the neurons of the preceding layer;
such a connection of neurons is also referred as a dense connection.
Neurons are the computational units of an ANN model. In a densely con-
nected network, each neuron in a hidden layer is connected to all the neurons
in the immediately previous layer. A neuron is a mathematical transformation
that first sums all the inputs multiplied by weights of the corresponding connec-
tions and then applies a nonlinear filter or activation function (generally having
a sigmoid shape) on the summation to generate an output that is a value ranging
between 0 and 1 or 1 and 1. When a feature vector is fed to the ANN model,
the input values are propagated forward through the network, neuron by neuron
and layer by layer, until the sequence of mathematical transformations applied
on the feature vector reach the output layer.
During the training, the output of the network (i.e., final values obtained at
the output layer) is compared with the desired/expected/known output, using a
loss function. The error values for each neuron in the output layer are then prop-
agated from the output layer back through the network, until each neuron has an
associated error value that reflects its contribution to the original output. Back-
propagation algorithm uses these error values to calculate the gradient of the
loss function. In the second phase, this gradient is fed to the optimization
method, which in turn uses it to update the weights/bias of each neuron, in
an attempt to minimize the loss function. ANN training leads to learning the
weights and bias for each neuron. ANN training involves feedforward of data
signals to generate the output and then the backpropagation of errors for gradi-
ent descent optimization. During the process of training an ANN model, first,
the weights/biases for the neurons in the output layer are updated; then, the
weights/biases of the neurons of each preceding layer is iteratively updated, till
the weights/biases of neurons of the first hidden layer are updated.