Page 124 -

7 Checking Simulations: Detecting and Avoiding Errors and Artefacts 121

Moreover, the challenge of understanding a computer simulation does not end
when one is confident that the code is free from errors; the complex issue of
identifying what parts of the code are generating a particular set of outputs remains
to be solved. Stated differently, this is the challenge of discovering what assumptions
in the model are causing the results we consider significant. Thus, a substantial
part of this non-trivial task consists in detecting and avoiding artefacts: significant
phenomena caused by accessory assumptions in the model that are (mistakenly)
deemed irrelevant. We explain this in detail in subsequent sections.
The aim of this chapter is to provide the reader with a set of concepts and a range
of suggested activities that will enhance his ability to understand simulations. As
mentioned before, simulation models can be seen as functions operating on their
inputs to produce the outputs. These functions are created by putting together a
range of different assumptions of very diverse nature. Some assumptions are made
because they are considered to be an essential feature of the system to be modelled;
others are included in a somewhat arbitrary fashion to achieve completeness—i.e.
to make the computer model run—and they may not have a clear referent in the
target system. There are also assumptions—e.g. the selection of the compiler and the
particular pseudorandom number generator to be employed—that are often made,
consciously or not, without fully understanding in detail how they work, but trusting
that they operate in the way we think they do. Finally, there may also be some
assumptions in a computer model that not even its own developer is aware of, e.g.
the use of floating-point arithmetic, rather than real arithmetic.
Thus, in broad terms, understanding simulations requires identifying what
assumptions are being made and assessing their impact on the results. To achieve
this, we believe that it is useful to characterise the process by which assumptions
accumulate to end up forming a complete model. We do this in a structured way
by presenting a general framework that summarises the process of creating and
using agent-based models through various stages; then, within this framework, we
characterise the different types of assumptions that are made in each of the stages
of the modelling process, and we identify the sort of errors and artefacts that may
occur; we also propose activities that can be conducted to avoid each type of error
or artefact.
The chapter is structured as follows: the following section is devoted to explain-
ing what we understand by modelling, and to argue that computer simulation is
a useful tool to explore formal models, rather than a distinctively new symbolic
system or a uniquely different reasoning process, as it has been suggested in the
literature. In Sect. 7.3 we explain what the essence of agent-based modelling is in
our view, and we present the general framework that summarises the process of
designing, implementing, and using agent-based models. In Sect. 7.4 we define
the concepts of error and artefact, and we discuss their relevance for validation
and verification. The framework presented in Sect. 7.3 is then used to identify the
various stages of the modelling process where different types of assumptions are
made and, consequently, where different types of errors and artefacts may appear.
We then propose various activities aimed at avoiding the types of errors and artefacts
previously described, and we conclude with a brief summary of the chapter.