Page 44 - Machine Learning for Subsurface Characterization

Unsupervised outlier detection techniques Chapter 1 29

on FS3 is comparable with that on FS1 and much better than that on FS2.
This mandates the use of shallow-sensing logs as features for outlier detection.
Visual representation of the performances in terms of balanced accuracy score
is shown in Fig. 1.7B.
Outlier detection performance on Dataset #2 clearly shows that when fea-
tures that are not strongly affected by hole size (e.g., deep resistivity, RT)
are used, the model performance drops, as observed in FS2. On the contrary,
when shallow-sensing DTC and RXO are used as features, the model perfor-
mance improves. We conclude that feature selection plays an important role
in determining the performance of ODTs, especially in identifying “contextual
outliers.” IF model is best in detecting contextual outliers, like the group of log
responses affected by bad holes. F1 scores are low because the fraction of actual
outliers in the dataset is a small fraction (0.022) of the entire dataset, and we do
not set contamination level a priori. Being an unsupervised approach, in the
absence of constraints such as contamination level, the model is detecting many
original inliers as outliers. Therefore, balanced accuracy score and ROC-AUC
score are important evaluation metrics (Table 1.2).

5.3 Performance on Dataset #3 containing shaly layers and bad holes
with noisy measurements

Performance on Dataset #3 indicates how well a model detects depths where log
responses are affected by either noise or bad hole in a heterogenous formation
with thin layers of sparsely occurring rock type (i.e., shale). The objective of this
evaluation is to test if the models can detect the noise and bad-hole influenced
depths (samples) without picking the rare occurrence of shales as outliers. Out-
lier methods are designed to pick rare occurrences as outliers; however, a good
shale zone even if it occurs rarely should not be labeled as outlier by the unsu-
pervised methods.
Comparative study on Dataset #3 involved experiments with four distinct
feature subsets sampled from the available features GR, RHOB, DTC, RT,
and NPHI logs, namely, FS1, FS2, FS3, and FS4. FS1 contains GR, RHOB,
and DTC; FS2 contains GR, RHOB, and RT; FS3 contains GR, RHOB,
DTC, and RT; and FS4 contains GR, RHOB, DTC, and NPHI. In all feature
sets, 70 points are known outliers, and 704 are known inliers, comprising sand-
stone, limestone, dolostone, and shales. Isolation forest (IF) model performs
better than the rest for all feature sets. Interestingly, with respect to F1 score,
IF underperforms on FS2 compared with the rest, due to lower precision and
imbalance in dataset. This also suggests that DTC is important for detecting
the bad-hole depths, because FS2 does not contain DTC, unlike the rest
(Table 1.3). Visual representation of the performances in terms of balanced
accuracy score is shown in Fig. 1.7C.