Page 190 -

9 Verifying and Validating Simulations 187

9.3.3.6 Solution Space Exploration

The techniques discussed in Sect. 9.3.3.5 are useful for basic output validation under
specific parametrisations. However, they do not provide a general understanding
of how input parameters influence model behaviour, nor they consider the broader
picture of overall model assumptions, which encompass not only input parameters,
but also internal model structure, employed submodels and model elements, as well
as their inter-relations. In solution space exploration, model assumptions are varied
in order to reach a better understanding of how the assumptions of interest affect the
model.
The exploration of the solution space can be as simple as testing “what
if” scenarios for observing model behaviour under different inputs—similar to
what was discussed in the previous subsection—or follow a more systematic
approach based on carefully designed experiments (Montgomery 2012). The latter
approach aims to get the maximum amount of information from the model with
the minimum number of simulation runs (Pereda et al. 2015), and is generally
more efficient than hand-guided runs where alternative model configurations are
experimented with (Law 2015). Nonetheless basic hand-guided experiments are also
valuable for model validation, namely when trying different conceptual- or system-
level assumptions. Conceptual-level assumptions include internal mechanisms or
submodels that constitute the larger model (e.g. the decision processes of the
agents, their learning mechanisms or their interaction topology), while system-level
assumptions involve low-level elements of the model (e.g. agent activation regimes).
If changing elements at the system-level determines different behaviours of the
model that cannot be adequately interpreted, then the validity of the model can be
compromised. The case of changing elements at conceptual levels is more subtle
and the validity of the results must be assessed by the researcher with reference to
the validity of the composing elements of the model. This is basically a kind of
cross-model or cross-element validation, as described in Sect. 9.4.
The exploration of the solution space is often undertaken with one or more
targeted objectives in mind, especially in the case of formally designed experiments.
Typical objectives include optimisation, calibration, uncertainty analysis and sensi-
tivity analysis (Lee et al. 2015). While these objectives may overlap, brief definitions
and their potential roles in model validation can be given. In model optimisation,the
researcher is interested in finding parameters or assumptions that minimise some
cost or elicit specific model events or behaviour, which can be directly related with
event validity, as discussed in Sect. 9.3.3.4. In turn, calibration is concerned with
finding the assumptions that maximise the agreement of the model behaviour with
the target system behaviour, thus making it a crucial aspect in model validation
and in the model development process. Uncertainty analysis provides measures
related to the reliability of results and how do input uncertainties propagate through
to the collected outputs. These measures affect simulation output validity and
directly influence the interpretation of data obtained through sensitivity analysis.
The latter is arguably the most common objective when exploring the solution
space of a model. In essence, small perturbations are applied to model assumptions