Page 171 - A First Course In Stochastic Models

164 CONTINUOUS-TIME MARKOV CHAINS

since a finite sum of o( t) terms is again o( t). Hence
p ij (t + t) − p ij (t) o( t)
= q kj p ik (t) − ν j p ij (t) + .
t t
k =j
Letting t → 0 yields the desired result.

The linear differential equations (4.5.1) can be explicitly solved only in very
special cases.

Example 4.5.1 An on-off source

A source submitting messages is alternately on and off. The on-times are indepen-
dent random variables having a common exponential distribution with mean 1/α
and the off-times are independent random variables having a common exponential
distribution with mean 1/β. Also the on-times and the off-times are independent
of each other. The source is on at time 0. What is the probability that the source
will be off at time t?
This system can be modelled as a continuous-time Markov chain with two states.
Let the random variable X(t) be equal to 0 when the source is off at time t and
equal to 1 otherwise. Then {X(t)} is a continuous-time Markov chain with state
space I = {0, 1}. The transient probabilities p 10 (t) and p 11 (t) satisfy

′
p (t) = −βp 10 (t) + αp 11 (t), t ≥ 0,
10
′
p (t) = βp 10 (t) − αp 11 (t), t ≥ 0.
11
A standard result from the theory of linear differential equations states that the
general solution of this system is given by
λ 1 t λ 2 t
(p 10 (t), p 11 (t)) = c 1 e x 1 + c 2 e x 2 , t ≥ 0
provided that the coefficient matrix

−β α
A =
β −α
has distinct eigenvalues λ 1 and λ 2 . The vectors x 1 and x 2 are the corresponding
eigenvectors. The constants c 1 and c 2 are determined by the boundary conditions
p 10 (0) = 0 and p 11 (0) = 1. The eigenvalues of the matrix A are λ 1 = 0 and
λ 2 = −(α+β) with corresponding eigenvectors x 1 = (β −1 , α −1 ) and x 2 = (1, −1).
Next it follows from c 1 /β + c 2 = 0 and c 1 /α − c 2 = 1 that c 1 = αβ/(α + β) and
c 2 = −α/(α + β). This yields

α α −(α+β)t
p 10 (t) = − e , t ≥ 0.
α + β α + β