Page 175 - A First Course In Stochastic Models

168 CONTINUOUS-TIME MARKOV CHAINS

Then Kolmogoroff’s forward differential equations can be written as P (t) = P(t)Q
′
for any t > 0. It is left to the reader to verify that the solution of this system of
differential equations is given by
n
∞
t
n
P(t) = e tQ = Q , t ≥ 0. (4.5.4)
n!
n=0
The matrix P = (p ), i, j ∈ I, can be written as P = Q/ν + I, where I is the
ij
identity matrix. Thus
∞ n
tQ νt(P−I) νtP −νtI −νt νtP −νt (νt) n
P(t) = e = e = e e = e e = e P .
n!
n=0
On the other hand, by conditioning on the number of Poisson events up to time t
in the {X(t)} process, we have

∞ n
−νt (νt) (n)
P {X(t) = j | X(0) = i} = e p ij ,
n!
n=0
(n)
where p is the n-step transition probability of the discrete-time Markov chain
ij
{X n }. Together the latter two equations yield the desired result.
Corollary 4.5.3 The probabilities p ij (t) are given by

∞ n
−νt (νt) (n)
p ij (t) = e p ij , i, j ∈ I and t > 0, (4.5.5)
n!
n=0
(n)
where the probabilities p can be recursively computed from
ij
(n) (n−1)
p = p p , n = 1, 2, . . . (4.5.6)
ij ik kj
k∈I
(0) (0)
starting with p = 1 and p = 0 for j = i.
ii ij
This probabilistic result is extremely useful for computational purposes. The
series in (4.5.5) converges much faster than the series expansion (4.5.4). The com-
putations required by (4.5.5) are simple and transparent. For fixed t > 0 the infinite
series can be truncated beforehand, since
∞ n ∞ n
−νt (νt) (n) −νt (νt)
e p ij ≤ e .
n! n!
n=M n=M
For a prespecified accuracy number ε > 0, we choose M such that the right-hand
side of this inequality is smaller than ε. By the normal approximation to the Poisson