Page 107 - A First Course In Stochastic Models

THE EQUILIBRIUM PROBABILITIES 99

An explanation of the term equilibrium distribution is as follows. Suppose that the
initial state of the process {X n } is chosen according to

P {X 0 = j} = π j , j ∈ I.

Then, for each n = 1, 2, . . . ,

P {X n = j} = π j , j ∈ I.

In other words, starting the process according to the equilibrium distribution leads
to a process that operates in an equilibrium mode. The proof is simple and is based
on induction. Suppose that P {X m = j} = π j , j ∈ I for some m ≥ 0. Then

P {X m+1 = j} = P {X m+1 = j | X m = k}P {X m = k}
k∈I

= p kj π k = π j , j ∈ I.
k∈I

An important question is: does the Markov chain have an equilibrium distribution,
and if it has, is this equilibrium distribution unique? The answer to this question
is positive when Assumption 3.3.1 is satisfied.

Theorem 3.3.2 Suppose that the Markov chain {X n } satisfies Assumption 3.3.1.
Then the Markov chain {X n } has a unique equilibrium distribution {π j , j ∈ I}. For
each state j,
n
1 (k)
lim p = π j (3.3.6)
n→∞ n ij
k=1

independently of the initial state i. Moreover, let {x j , j ∈ I} with j∈I
x j < ∞
be any solution to the equilibrium equations

x j = x k p kj , j ∈ I. (3.3.7)
k∈I
Then, for some constant c, x j = cπ j for all j ∈ I.

The proof of this important ergodic theorem is given in Section 3.5. It follows
from Theorem 3.3.2 that the equilibrium probabilities π j are the unique solution
to the equilibrium equations (3.3.5) in conjunction with the normalizing equation

π j = 1. (3.3.8)
j∈I