Page 162 - Computational Retinal Image Analysis

CHAPTER

9
Validation

b
a
a
c
Emanuele Trucco , Andrew McNeil , Sarah McGrory , Lucia Ballerini , Muthu
a
a
d
Rama Krishnan Mookiah , Stephen Hogg , Alexander Doney , Tom MacGillivray c
a VAMPIRE Project, Computing (SSEN), University of Dundee, Dundee, United Kingdom
b Centre for Cognitive Ageing and Cognitive Epidemiology, University of Edinburgh,
Edinburgh, United Kingdom
c VAMPIRE Project, Centre for Clinical Brain Sciences, University of Edinburgh,
Edinburgh, United Kingdom
d Division of Population Health and Genomics, University of Dundee, Ninewells Hospital and
Medical School, Dundee, United Kingdom
1 Introduction: Why is validation difficult?

Our observation of nature must be diligent, our reflection profound, and our
experiments exact
Denis Diderot

The truth is rarely pure and never simple
Oscar Wilde, The Importance of Being Earnest
We start from the conventional definition of validation adopted in nearly all MIA
papers [1–5]: the experimental procedure aimed to show that the output of an al-
gorithm is sufficiently close to a reference standard given a specific image analysis
task. Examples of “specific tasks” are detecting image regions of interest, e.g., mi-
croaneurysms or blood vessels in fundus camera images, or macular holes in OCT
scans; or grading fundus camera images for diabetic retinopathy. “Sufficiently close
to a reference standard” means that quantitative test criteria, e.g., sensitivity and
specificity, achieve or exceed some desired levels. The reference standard is provided
in the form of annotations, i.e., any information added to an image or video by ex-
perts. Examples include labels (e.g., a binary flag for the presence or not of a specific
lesion, or the name of a lesion from a dictionary of possibilities), grades (e.g., dia-
betic retinopathy for a whole fundus image, or the tortuosity level of a vessel), and
contours traced manually on images (e.g., the contour of the optic disc in a fundus
image, or of a layer in an OCT image).

Computational Retinal Image Analysis. https://doi.org/10.1016/B978-0-08-102816-2.00009-5 157
© 2019 Elsevier Ltd. All rights reserved.