Page 230 - Machine Learning for Subsurface Characterization

200 Machine learning for subsurface characterization

then shifts the random sample by the mean vector in the latent layer and scales it
by the variance vector in the latent layer. The first 3-dimensional layer of the
decoder is followed by 16-dimensional hidden layer and the 64-dimensional
output layer that upscale the latent vector to develop a reconstruction of the
original input. For any sampling of the latent distributions, the decoder
model learns to accurately reconstruct the input, such that the samples that
are nearby in the latent space result in very similar reconstructions. Similar
to the VAE, the loss function of the VAEc model has two components:
reconstruction loss and the Kullback-Leibler (KL) divergence loss. The
reconstruction loss emphasizes the difference between the original and
synthesized NMR T2 distribution, and the KL divergence loss evaluates the
difference between the distribution of the latent vector and a unit Gaussian
distribution.
In the second stage of training the VAEc-NN (Fig. 7.5), a four-layered NN
model is connected to the frozen pretrained three-layered decoder. The
decoder trained in the first stage of VAEc-NN training is frozen to preserve
the generalization of the T2 distributions. Only the four-layered neural
network undergoes weight updates in the second stage of training. The
4-layered neural network comprises one 10-dimensional input layer, one
30-dimensional hidden layer, one 10-dimensional hidden layer, and one
final 3-dimensional hidden layer attached to the frozen decoder network. In
doing so, the 4-layered neural network processes the 7 mineral composition
and 3 fluid saturation logs to produce a 3-dimensional vector, which can be
decoded by the trained decoder to generate the 64-dimensional T2
distribution. After the training process is complete, for purposes of testing
and deployment, the trained four-layered neural network followed by the
frozen three-layered decoder synthesizes the T2 distributions of subsurface
formations by processing the seven formation mineral content logs and
three fluid saturation logs (similar to Fig 7.3).

4.5 LSTM architecture, training, and testing
Long short-term memory (LSTM) network is a type of recurrent neural network
(RNN) that operates over sequences of input/output vectors. Unlike other neural
networks that are stateless, LSTM and RNN use the internal state at one
timestep as one of the inputs for calculating the internal state and output at
the next timestep. The recurrence of internal state serves as a form of
memory that allows contextual information to flow through the network so
that relevant outputs or hidden states from previous timesteps can be used by
the network at the current timestep. In our implementation the LSTM
network processes the sequence of 10 inversion-derived logs to encode an
intermediate vector that can be used to generate the sequence of 64 discrete
values that constitute the 64-dimensional NMR T2 distribution (Fig. 7.6).