Page 141 - A First Course In Stochastic Models

THEORETICAL CONSIDERATIONS 133

(n) (n) (n) (n) (n)
Using the inequalities m ≤ p ≤ M and m ≤ π j ≤ M , we find
j ij j j j
(n) (n) (n)
|p − π j | ≤ M − m , n = 0, 1, . . . (3.5.17)
ij j j
for any i, j ∈ I. Together the inequalities (3.5.16) and (3.5.17) yield the assertion
of the theorem except that we have still to verify that {π j } represents a probability
(n)

distribution. Obviously, the π j are non-negative. Since p = 1 for all n and
j∈I ij
(n)
p → π j as n → ∞, we obtain from the finiteness of I that the π j sum to 1.
ij
It remains to verify (3.5.15). To do so, fix j ∈ I and n ≥ ν. Let x and y be the
(n) (n) (n) (n)
states for which M = p and m = p . Then
j xj j yj
(n) (n) (n) (n) (ν) (n−ν) (ν) (n−ν)
0 ≤ M − m = p − p = p p − p p
j j xj yj xk kj yk kj
k∈I k∈I
(ν) (ν) (n−ν)
= {p − p }p
xk yk kj
k∈I
(ν) (ν) + (n−ν) (ν) (ν) − (n−ν)
= {p − p } p − {p − p } p ,
xk yk kj xk yk kj
k∈I k∈I
where a = max(a, 0) and a = − min(a, 0). Hence, by a , a ≥ 0,
−
+
+
−
(n) (n) (ν) (ν) + (n−ν) (ν) (ν) − (n−ν)
0 ≤ M − m ≤ {p − p } M − {p − p } m
j j xk yk j xk yk j
k∈I k∈I
(ν) (n−ν) (n−ν)
} [M
= {p − p (ν) + − m ],
xk yk j j
k∈I
where the last equality uses the fact that
a + =
a − if
k k = 0. Using
a
k k k k
the relation (a − b) = a − min(a, b), we next find
+

(n) (n) (ν) (ν) (n−ν) (n−ν)
0 ≤ M − m ≤ 1 − min(p , p ) M − m .
j j xk yk j j
k∈I
(ν)
Since p ≥ ρ for all i, we find
is
(ν) (ν) (ν) (ν)
1 − min(p , p ) ≤ 1 − min(p , p ) ≤ 1 − ρ,
xk yk xs ys
k∈I
which implies the inequality (3.5.15). This completes the proof.
Exponential convergence of the n-step transition probabilities does not hold in
general for an infinite-state Markov chain. Strong recurrence conditions should be
imposed to establish exponential convergence in infinite-state Markov chains.