Page 140 - A First Course In Stochastic Models

132 DISCRETE-TIME MARKOV CHAINS

i, j ∈ I,
(n) n
|p − π j | ≤ αβ , n = 1, 2, . . . .
ij
In particular,
(n)
lim p ij = π j , i, j ∈ I.
n→∞
Proof Let s be any recurrent state of the Markov chain. Since the Markov chain
is aperiodic, we have by part (c) of Theorem 3.5.7 that there exists an integer
ν ≥ 1 and a number ρ > 0 such that
(ν)
p ≥ ρ i ∈ I.
is
(n) (n)
For any j ∈ I, define the sequences {M , n ≥ 0} and {m , n ≥ 0} by
j j
(n) (n) (n) (n)
M = max p and m = min p .
j ij j ij
i∈I i∈I
(0) (0)
Note that M = 1 and m = 0. Applying relation (3.2.2), we find
j j
(n+1) (n) (n) (n)
M = max p ik p ≤ max p ik M = M max p ik ,
j kj j j
i∈I i∈I i∈I
k∈I k∈I k∈I
and so, for any j ∈ I,
(n+1) (n)
M ≤ M , n = 0, 1, . . . .
j j
Similarly, we find for any j ∈ I that
(n+1) (n)
m ≥ m , n = 0, 1, . . . .
j j
(n) (n)
Since the sequences {M } and {m } are bounded and monotone, they have finite
j j
limits. Next we establish the inequality
(n) (n) (n−ν) (n−ν)
0 ≤ M − m ≤ (1 − ρ)[M − m ], n ≥ ν (3.5.15)
j j j j
for any j ∈ I. Suppose for the moment that we have proved this inequality. A
repeated application of the inequality shows that
(n) (n) [n/ν] (0) (0)
0 ≤ M − m ≤ (1 − ρ) (M − m ), n = 0, 1, . . . , (3.5.16)
j j j j
where [x] denotes the largest integer contained in x. Here we used the fact that
(n) (n)
M −m is decreasing in n. By (3.5.16), we have that the limits of the monotone
j j
(n) (n)
sequences {M } and {m } coincide. Denote the common limit by π j . Hence
j j
(n) (n)
lim M j = lim m j = π j .
n→∞ n→∞