Page 135 - A First Course In Stochastic Models

THEORETICAL CONSIDERATIONS 127

(all states in R are positive recurrent). These results follow directly from Theo-
rem 3.3.1 by noting that π j = 0 when j is transient and f ij = 1 for all i ∈ I when
j is recurrent. We are now able to prove a main result.

Theorem 3.5.9 Suppose that the Markov chain {X n } satisfies Assumption 3.3.1.
Then the probabilities π j , j ∈ I defined by (3.5.3) constitute the unique equilibrium

distribution of the Markov chain. Moreover, letting {x j , j ∈ I} with j |x j | < ∞
be any solution to the equilibrium equations

x j = x k p kj , j ∈ I, (3.5.6)
k∈I
it holds that, for some constant c, x j = cπ j for all j ∈ I.

Proof We first show that the π j satisfy (3.5.6) and

π j = 1. (3.5.7)
j∈I

To do so, we use the relation (3.2.1) for the n-step transition probabilities. Averaging
this relation over n, we obtain for any m ≥ 1

m m
1 (n+1) 1 (n)
p ij = p p kj
m m ik
n=1 n=1 k∈I
m
1
(n)
= p ik p kj , j ∈ I, (3.5.8)
m
k∈I n=1
where the interchange of the order of summation is justified by the non-negativity
of the terms. Next let m → ∞ in (3.5.8). On the right-hand side of (3.5.8) it is not
allowed to interchange limit and summation (except when I is finite). However,
we can apply Fatou’s lemma from Appendix A. Using (3.5.4), we find

π j ≥ π k p kj , j ∈ I.
k∈I

Next we conclude that the equality sign must hold in this relation for each j ∈ I,
otherwise we would obtain the contradiction

π j > π k p kj = π k p kj = π k .
j∈I j∈I k∈I k∈I j∈I k∈I
We have now verified that the π j satisfy the equilibrium equations (3.5.6). The

(n)
equation (3.5.7) cannot be directly concluded from j∈I p ij = 1 for all n ≥ 1.