Page 231 -

Chapter 11
How Many Times Should One Run
a Computational Simulation?

Raffaello Seri and Davide Secchi

Abstract This chapter is an attempt to answer the question “how many runs of
a computational simulation should one do,” and it gives an answer by means of
statistical analysis. After defining the nature of the problem and which types of
simulation are mostly affected by it, the chapter introduces statistical power analysis
as a way to determine the appropriate number of runs. Two examples are then
produced using results from an agent-based model. The reader is then guided
through the application of this statistical technique and exposed to its limits and
potentials.

Why Read This Chapter?

To understand and reflect on the importance of determining an appropriate number
of runs for a simulation of a complex social system, especially agent-based simu-
lation models. Also the chapter guides readers through (a) the issues surrounding
this determination, (b) the use of statistical power analysis to identify the number of
runs, and (c) two examples to practice the computation.

11.1 Introduction

This chapter explores the issue of how many times a simulation should run. This is
an often neglected issue (Ritter et al. 2011) that, sooner or later, all modelers dealing
with simulations of complex systems encounter. The literature takes an agnostic
stance on how many runs—per configuration of parameters or, as economists put
it, ceteris paribus—a simulation is to be run. In fact, the focus has mostly been on

R. Seri
University of Insubria, Varese, Italy
D. Secchi ( )
University of Southern Denmark, Slagelse, Denmark
e-mail: secchi@sdu.dk

© Springer International Publishing AG 2017 229
B. Edmonds, R. Meyer (eds.), Simulating Social Complexity,
Understanding Complex Systems, https://doi.org/10.1007/978-3-319-66948-9_11