Page 194 -

9 Verifying and Validating Simulations 191

Computational
Conceptual model A
model A compare
results
Computational
Conceptual model B’
model B’
Computational
Conceptual model B
model B
Fig. 9.4 Model alignment, also referred to as docking

two models A and B requires modifying certain aspects of model B—for instance
turning off a specific feature—in order to become equivalent to model A. This is
represented in Fig. 9.4.
The work of Axtell et al. (1996) is arguably the most-cited attempt to align
two distinct but similar models. Rather than re-implementing Axelrod’s culture
dissemination model, Axtell and colleagues focused on the general case of aligning
two models that reflected slightly distinctive mechanisms. For this purpose, Epstein
and Axtell’s Sugarscape model (1996) was progressively simplified in order to
align with the results obtained by Axelrod’s culture dissemination model (1997b).
They concluded that comparing models developed by different researchers and
with different tools (i.e. programming languages and/or modelling environments),
can lead to exposing bugs, misinterpretations in model specification, and implicit
assumptions in toolkit implementations.
Model alignment has been further investigated in a series of meetings called
model-to-model (M2M) workshops (Rouchier et al. 2008). The M2M workshops
attract researchers interested in understanding and promoting the transferability of
knowledge between model users.
Submodel comparison, often referred to as cross-element validation, rather than
comparing whole models, compares the results of a model whose architecture of
the agents differs only in a few elements. The goal is to assess the extent to
which changing elements of the model architecture produces results compatible
with the expected results of the (larger) model. It is essentially an exercise in
composing different submodels within a larger model, and is related to solution
space exploration since submodels are varied, and the consequences of that variation
are analysed and compared. In this process, the overall validity of the larger
model with reference to the validity of each one of the submodels can also be
assessed. For instance, one may study the effects of using a model with agents in a
bargaining game employing either evolutionary learning or reinforcement learning
strategies, and assess which one of the strategies produces results compatible with
theoretical analysis in game theory (Takadama et al. 2003). Submodel comparison
is represented in Fig. 9.5.
Submodel comparison can also be used as a model replication or alignment
aid. For example, Radax and Rengs (2009) proposed a method for replicating
insufficiently described ABMs, consisting in systematically varying ambiguous