Page 97 - Machine Learning for Subsurface Characterization

Shallow neural networks and classification methods Chapter 3 81

mitigate overfitting [16]. Regularization introduces a penalty term in the target
function that penalizes additional model complexity. Penalty parameter is set at
0.7 in the first model and 0.5 in the second model based on extensive numerical
experiments to find the optimal value of the penalty term [16, 17]. In place of
regularization, cross validation and dropout can be performed to avoid overfit-
ting of ANN models. However, cross validation generally requires larger
dataset.

3 ANN model training, testing, and deployment
3.1 ANN models
Twenty-two conventional and inversion-derived logs and five categorical flags
are used as features for the first ANN model to predict the 64 discretized ampli-
tudes of T 2 distributions and for the second ANN model to predict the six param-
eters of the bimodal Gaussian distribution that fits the T 2 distribution. Logs from
416 different depths are randomly split into 354 (85%) depths to be used as the
training dataset and 62 (15%) depths as the testing dataset. Normalized root
mean square error (NRMSE) for each T 2 bin across the entire dataset is used
as a score to evaluate the accuracy of T 2 distributions synthesized by the
ANN models. The formulation of normalized root mean square error (NRMSE)
is described in Eqs. (3.7) and (3.8).

3.2 Training the first ANN model
Fig. 3.5 presents the prediction performance of the first ANN model for 25 ran-
domly selected depths from the training dataset. Prediction performance on the
training dataset is also referred as the memorization performance. For a learning
model, the memorization performance cannot be used in isolation as evaluation
metric. Memorization performance in conjunction with generalization perfor-
mance can indicate the level of over-fitting in the learning process. Bimodal
T 2 distributions with small dispersions are difficult to predict. Noise in the con-
ventional logs and in the T 2 distributions negatively affects the performance of
the trained model. Predictions are in good agreement with true T 2 distributions
for unimodal and bimodal distributions with high dispersivity around the dom-
inant T 2 . The median NRMSE for the training dataset is 0.1201, which indicates
a good prediction performance. Histograms of NRMSE for the 354 depths in the
training datasets are plotted in Fig. 3.6. Most NRMSE values are lower than 0.2,
implying a good prediction performance of the first ANN model.
Four examples of prediction performance of the first ANN model are illus-
trated in Fig. 3.7. Each example shows the original and predicted T 2 distribu-
tions. Fig. 3.7 aids the qualitative understanding of the training performance.
2
The first subplot at the top left with R ¼ 0.99 is the case with best performance.
During the training, all depths with single-peak T 2 distribution are trained at a