Page 313 -

312 L.R Izquierdo et al.

behaviour is characterised by the distribution of the state of the system X n for a
fixed time-step n 0. The asymptotic behaviour (see Sects. 13.7.3 and 13.7.4)is
characterised by the limit of the distribution of X n as n goes to infinity, when this
limit exists.
This section explains how to calculate the transient distribution of a certain
THMC, i.e. the distribution of X n for a fixed n 0. In simple words, we are after
a vector a (n) containing the probability of finding the process in each possible state
(n)
(n)
in time-step n. Formally, a (n) D [a 1 , ::: , a M ], where a i (n) D P(X n D i) denotes
the distribution of X n for a THMC with M possible states. In particular, a (0) denotes
(0)
the initial distribution over the state space, i.e. a i D P(X 0 D i). Note that there is
no problem in having uncertain initial conditions, i.e. probability functions over the
space of possible inputs to the model.
It can be shown that one can easily calculate the transient distribution in time-
step n, simply by multiplying the initial conditions by the n-th power of the transition
matrix P.
n
Proposition 1 a (n) D a (0) P n Thus, the elements p (n) i,j of P represent the proba-
bility that the system is in state j after n time-steps having started in state i, i.e.
p (n) i,j D P(X n D jjX 0 D i). A straightforward corollary of Proposition 1 is that
m
a (nCm) D a (n) P .
As an example, let us consider the one-dimensional random walk again. Imagine
that the random walker starts at an initial random location, i.e. a (0) D [1/17, ::: ,
1/17]. The exact distribution of the walker’s position in time-step 100 would then
be a (100) D a (0) ::: P 100 . This distribution is represented in Fig. 13.12, together with
an empirical distribution obtained by running the model 50,000 times.
Having obtained the probability function over the states of the system for any
fixed n, namely, the probability mass function of X n , it is then straightforward to
calculate the distribution of any statistic that can be extracted from the model. As
argued in the previous sections, the state of the system fully characterises it, so any
statistic that we obtain about the computer model in time-step n must be, ultimately,
a function of fX 0 , X 1 , ::: , X n g.
Admittedly, the transition matrix of most computer models cannot be easily
derived, or it is unfeasible to operate with it. Nonetheless, this apparent drawback
is not as important as one might expect. As we shall see below, it is often possible
to infer many properties of a THMC even without knowing the exact values of its
transition matrix, and these properties can yield useful insights about the dynamics
of the associated process. Knowing the exact values of the transition matrix allows
us to calculate the exact transient distributions using Proposition 1; this is desirable
but not critical, since we can always approximate these distributions by conducting
many simulation runs, as explained in Sect. 13.6.