Page 53 -

4 Different Modelling Purposes 47

4.3.2 Risks

Clearly, there are several risks in the project of establishing a complex explanation
using a simulation—what counts as a good explanation is not as clear-cut as what is
a good prediction.
Firstly, the fit to the target data to be explained might be a very special case. For
example, if many other parameters need to have very special values for the fit to
occur, then the explanation is, at best, brittle and, at worst, an accident.
Secondly, the process that is unfolded in the simulation might be poorly
understood so that the outcomes might depend upon some hidden assumption
encapsulated in the code. In this case, the explanation is dependent upon this
assumption holding, which is problematic if this assumption is very strong or
unlikely.
Thirdly, there may be more than one explanation that fits the target data. So
although the simulation establishes one explanation, it does not guarantee that it
is the only candidate for this.

4.3.3 Mitigating Measures

To improve the quality and reliability of the explanation being established:
• Ensure that the mechanisms built into the simulation are plausible or at least
relate to what is known about the target phenomena in a clear manner.
• Be clear about which aspects of the outcomes are considered significant in terms
of comparison to the target data—i.e. exactly which aspects of that target data
are being explained.
• Probe the simulation to find out the conditions for the explanation holding
using sensitivity analysis, addition of noise, multiple runs, changing processes
not essential to the explanation to see if the results still hold and documenting
assumptions.
• Do experiments in the classic way, to check that the explanation does, in fact,
hold for your simulation code—i.e. check your code and try to refute the
explanation using carefully designed experiments with the model.

4.4 Theoretical Exposition

4.4.1 Motivation

If one has a mathematical model, one can do analysis upon its mathematics to
understand its general properties. This kind of analysis is both easier and harder
with a simulation model—to find out the properties of simulation code, one just