Page 144 -

Chapter 8
The Importance of Ontological Structure:
Why Validation by ‘Fit-to-Data’ Is Insufficient

Gary Polhill and Doug Salt

Abstract This chapter will briefly describe some common methods by which
people make quantitative estimates of how well they expect empirical models to
make predictions. However, the chapter’s main argument is that fit-to-data, the
traditional yardstick for establishing confidence in models, is not quite the solid
ground on which to build such belief some people think it is, especially for the
kind of system agent-based modelling is usually applied to. Further, the chapter
will show that the amount of data required to establish confidence in an arbitrary
model by fit-to-data is often infeasible, unless there is some appropriate ‘big data’
available. This arbitrariness can be reduced by constraining the choice of model.
In agent-based models, these constraints are introduced by their descriptiveness
rather than by removing variables from consideration or making assumptions for the
sake of simplicity. By comparing with neural networks, we show that agent-based
models have a richer ontological structure. For agent-based models, in particular,
this richness means that the ontological structure has a greater significance and yet
is all too commonly taken for granted or assumed to be ‘common sense’. The chapter
therefore also discusses some approaches to validating ontologies.

Why Read This Chapter?
When you have built an agent-based model, you need some way of assessing how
‘good’ it is. We will tell you how this is done traditionally in empirical contexts,
through measures of fit-to-data. You will learn why fitting to data is not enough
in the kind of situation where agent-based models are useful and why you also
need to assess the model’s ontological structure. The chapter will tell you what the
ontological structure is, how to assess it and whether and if so how it can be traded
off against fit-to-data.

G. Polhill ( )•D. Salt
The James Hutton Institute, Craigiebuckler, Aberdeen, AB15 8QH, UK
e-mail: gary.polhill@hutton.ac.uk; doug.salt@hutton.ac.uk

© Springer International Publishing AG 2017 141
B. Edmonds, R. Meyer (eds.), Simulating Social Complexity,
Understanding Complex Systems, https://doi.org/10.1007/978-3-319-66948-9_8