Page 136 - A First Course In Stochastic Models

128 DISCRETE-TIME MARKOV CHAINS

However, by letting m → ∞ in
 
m m
1 (n) 1 (n)
1 =  p ij  = p ij
m m
n=1 j∈I j∈I n=1
and using Fatou’s lemma from Appendix A, we can conclude that

π j ≤ 1. (3.5.9)
j∈I
Since the set R of recurrent states is not empty, we have by (3.5.5) that

π j > 0. (3.5.10)
j∈I
Next we prove that the solution to the equilibrium equations (3.5.6) is uniquely
determined up to a multiplicative constant. As a by-product of this proof we will

find that π j must be equal to 1. Let {x j } with |x j | < ∞ be any solution
j∈I
to the equation (3.5.6). Substituting this equation into itself, we find

x j = x ℓ p ℓk p kj = x ℓ p ℓk p kj
k∈I ℓ∈I ℓ∈I k∈I

(2)
= x ℓ p , j ∈ I,
ℓj
ℓ∈I
where the interchange of the order of summation in the second equality is jus-
tified by Theorem A.1 in Appendix A. By repeated substitution we find x j =

(n)
x ℓ p , j ∈ I for all n ≥ 1. Averaging this equation over n, we find
ℓ∈I ℓj
after an interchange of the order of summation (again justified by Theorem A.1 in
Appendix A) that
m
1
(n)
x j = x ℓ p ℓj , j ∈ I and m ≥ 1.
m
ℓ∈I n=1
Letting m → ∞ and using (3.5.4) together with the bounded convergence theorem
from Appendix A, it follows that

x j = π j x ℓ , j ∈ I.
ℓ∈I
This proves that any solution to (3.5.6) is uniquely determined up to a multiplicative
constant. Summing both sides of the latter equation over j, we find
 

x j =  π j  x ℓ .
j∈I j∈I ℓ∈I