Page 202 -

9 Verifying and Validating Simulations 199

machinery), the kind of interaction topology or environment, and the passive
entities with which the agents interact. The model used should be as general
as possible for the context in consideration as well as flexible for testing how
parameters vary in particular circumstances. Empirical data—if available—
should be used to help configure the parameters. Both the descriptions of the
model and the parameters used should be validated for the specific context of
the model. For example, suppose empirical data are available for specifying
the consumer demand of products. If the demand varies from sector to sector,
one may use data to inform the distribution upon which the parameter could be
based for each specific sector.
(b) Specifying expected behaviours of the computational model: Micro and macro
characteristics that the model is designed to reproduce are established from the
outset based on theoretical and/or empirical knowledge. Any property, from
quantitative to qualitative measures, such as emergent key facts the model
should reproduce (stylised facts), the statistical characteristic or shape of time-
data series (statistical signatures) and individual agents’ behaviour along the
simulation (individual trajectories), can be assessed. This may be carried out
in innumerable ways, according to different levels of description or grain, and
be more or less general depending on the context of the model and the kind of
empirical knowledge available. For instance, in some systems it may be enough
to predict just a “weak” or “positive” measure on some particular output, such
as a positive and weak autocorrelation. Or we might look for the emergence
of unpredictable events, such as stable regimes interleaved with periods of
strong volatility, and check their statistical properties for various levels of
granularity. Or the emergence of different structures or patterns associated with
particular kinds of agents, such as groups of political agents with “extremist” or
“moderate” individuals.
(c) Testing the computational model and building and validating post-compu-
tational models: The computational model is executed. Both individual and
aggregate characteristics are computed and tested for sensitivity analysis. These
are micro-validated and macro-validated against the expected characteristics
of the model established in step B according to a variety of validation
techniques, as described in the previous sections. A whole process of building
post-computational models takes place, possibly leading to the discovery of
unexpected characteristics in the behaviour of the computational model which
should be assessed with further theoretical or empirical knowledge about the
problem domain.

Further Reading

Good introductions to validation and verification of simulation models in general
are Sargent (2013) and Troitzsch (2004), the latter with a focus on social simulation.
Validation of ABMs in particular is addressed by Amblard et al. (2007).