Page 184 -

9 Verifying and Validating Simulations 181

9.3.1 The Goal of Validation: Goodness of Description

If one is using a predictive model, then the purpose of the model is to predict either
past or future states of the target system. On the other hand, one may strive for a
model that is able to describe the target system with satisfactory accuracy in order
to become more knowledgeable about the functioning of the system, to exercise
future and past scenarios, and to explore alternative designs or inform policies.
The objective in this section is to define the purpose of validation in terms of
the purpose of simulating social complexity, which we will define as being of good
description. This position entails that there is no single method or technique for
validating a simulation. A diversity of methods for validating models is generally
applied.
In the rest of this chapter we adopt the agent-based paradigm for modelling. A
conceptual understanding of validation, similar but more general than Moss and
Edmonds (2005), will be used:

The purpose of validation is to assess whether the design of micro-level mechanisms, put
forward as theories of social complexity validated to arbitrary levels, can be demonstrated
to represent aspects of social behaviour and interaction that are able to produce macro-level
effects either (i) broadly consistent with the subjacent theories; and/or (ii) qualitatively or
quantitatively similar to real data.

By broad consistency we mean the plausibility of both micro specification and
macro effects accounted as general representations of the target social reality. In its
most extreme expression, plausibility may be evaluated on a metaphorical basis.
By qualitative similarity to real data we mean a comparison with the model in
terms of categorical outcomes, accounted as qualitative features, such as the shape
of the outcomes, general stylised facts, or dynamical regimes. As for quantitative
similarity we mean the very unlikely case in which the identification of formal
numerical relationships between aggregate variables in the model and in the target—
such as enquiring as to whether both series may draw from the same statistical
distribution—proves to be possible.
Notice that this definition is general enough to consider both the micro-
level mechanisms and macro-level effects assessed on a participatory basis. It is
also general enough to consider two methodological practices for building social
simulation models, namely the extent to which models should be based on formal
theories or on the intuition of the model builders and stakeholders—an issue that
we will come back to later. These are omnipresent methodological questions in
the social simulation literature and are by no means irrelevant to the purpose of
simulation models.
Suppose that on the basis of a very abstract model, such as the Schelling model,
you were to evaluate the similarity of its outputs with empirical data. Then you will
probably not take issue with the fact that the goal of predicting future states of the
target would be out of the scope of simulation research for that kind of modelling.
However, despite the belief that other sorts of validation are needed, this does not
imply excluding the role of prediction, but simply emphasises the importance of