Page 233 -

11 How Many Times Should One Run a Computational Simulation? 231

in this way may be subject to empirical testing in a future laboratory experiment or
through real data. An example of this type of ABM can be the simulation of a team
of doctors and nurses working in the emergency room of a hospital, to isolate those
socio-cognitive attitudes that may lead to increased performance (Thomsen 2016).
Another example comes from political science (de Marchi and Page 2014) and it
concerns the model of incumbent advantage in elections proposed by Kollman et al.
(1992), that was first studied by simulation and then successfully tested empirically.
The techniques presented in this chapter are not necessarily pertinent to these first
two model types described above, because they call for an exploratory approach
in which the configurations of parameters and the number of runs are not rigidly
chosen in advance, but they may be modified by trial and error while the researcher
explores the potential outcomes of the model. 1
A third objective of ABM is measurement, namely providing a numerical value
for a quantity of interest. Since most agent-based models in the social sciences are
too simplified a representation of reality to provide accurate estimates of real-world
quantities, the models that pursue this objective are generally constrained to specific
disciplines in which the rules of behavior of the agents are simple or particularly
well known (for an example in biology see Sect. 1.1.1 in Railsback and Grimm
(2011); for examples in transportation research, see Maggi and Vallino 2016). In this
case, even if statistical tests can still be of interest, the researcher may direct his/her
statistical analysis towards different tools. On the one hand, data from an ABM
may be compared, through a distance (e.g., Lamperti 2015), with real time series to
assess whether the two are similar enough. On the other hand, the researcher may
settle on a sample size that guarantees a certain precision in the value computed for
the quantity of interest rather than a certain level of power (see Sect. 11.5.1 below).
Finally, a fourth objective of ABMs is to test hypotheses in a controlled
environment often emulating, with simplified rules, a real-world situation. The
advantage of ABMs in this respect is that they allow the researcher to analyze a
realistic situation by removing all the confounding factors arising in the observation
of the real-world phenomenon. In this case, agent-based models can be considered
the computational equivalent of laboratory experiments (Gilbert and Terna 2000).
In the following pages, we explain how the parallel can be established. Note,
however, that this is not the only possible setting. It is customary that an ABM
has several parameters entering its formulation. The aim of a model can be that
of exploring whether these parameters bear any impact on a quantity of interest,
obtained as an outcome of the simulations of the model. The usual way is to identify
some configurations of parameters that would correspond to different alternative
treatments in an experiment, and to run several simulations of the model under each
configuration. Each run of the model corresponds to an observation (e.g., a subject)
in an experiment: the measured outcome can either be the terminal value of the
series or a value computed on (a part of) the trajectory. The presumed independence

1
Note, moreover, that the researcher should not test a hypothesis on the data that have been used to
generate it.