Page 162 - A First Course In Stochastic Models

ERGODIC THEOREMS 155

Definition 4.3.1 A probability distribution {p j , j ∈ I} is said to be an equilibrium
distribution for the continuous-time Markov chain {X(t)} if

ν j p j = p k q kj , j ∈ I.
k =j

Just as in the discrete-time case, the explanation of the term ‘equilibrium dis-
tribution’ is as follows. If P {X(0) = j} = p j for all j ∈ I, then for any t > 0,
P {X(t) = j} = p j for all j ∈ I. The proof is non-trivial and will not be given.
Next we prove Theorem 4.2.1 in a somewhat more general setting.

Theorem 4.3.1 Suppose that the continuous-time Markov chain {X(t)} satisfies
Assumptions 4.1.2 and 4.2.1. Then:

(a) The continuous-time Markov chain {X(t)} has a unique equilibrium distribution
{p j , j ∈ I}. Moreover
π j /ν j
, j ∈ I, (4.3.2)
p j =
π k /ν k
k∈I
where {π j } is the equilibrium distribution of the embedded Markov chain {X n }.

(b) Let {x j } be any solution to ν j x j = k =j x k q kj , j ∈ I, with j |x j | < ∞. Then,
for some constant c, x j = cp j for all j ∈ I.

Proof We first verify that there is a one-to-one correspondence between the solu-
tions of the two systems of linear equations

ν j x j = x k q kj , j ∈ I
k =j

and

u j = u k p kj , j ∈ I.
k∈I
If {u j } is a solution to the second system with |u j | < ∞, then {x j = u j /ν j } is a

solution to the first system with |x j | < ∞, and conversely. This is an immediate
consequence of the definition (4.3.1) of the p ij . The one-to-one correspondence
and Theorem 3.5.9 imply the results of Theorem 4.3.1 provided we verify

π j
< ∞. (4.3.3)
ν j
j∈I
The proof that this condition holds is as follows. By Assumption 4.2.1, the process
{X(t)} regenerates itself each time the process makes a transition into state r. Let a