Page 131 - A First Course In Stochastic Models

THEORETICAL CONSIDERATIONS 123

(c) If the recurrent states are aperiodic, then there is an integer ν ≥ 1 such that
(ν)
p > 0 for all i ∈ I and j ∈ R.
ij
Proof Since the Markov chain has no two disjoint closed sets, the closed set R of
recurrent states is irreducible by Theorem 3.5.4. Hence, by Lemma 3.5.2, any two
states in R communicate with each other. This implies that for any i, j ∈ R there
(n)
is an integer n ≥ 1 such that p > 0. Next we prove that for any i ∈ I and j ∈ R
ij
(n)
there is an integer n ≥ 1 such that p > 0. To verify this, assume to the contrary
ij
that there is a transient state i ∈ I such that no state j ∈ R is accessible from i.
Then there is a closed set that contains i and is disjoint from R. This contradicts
the assumption that the Markov chain has no two disjoint closed sets. Hence for
any transient state i ∈ R there is a state j ∈ R that is accessible from i. Thus any
state j ∈ R is accessible from any i ∈ I, since any two states in R communicate
with each other.
To verify parts (b) and (c), define under the condition X 0 = i the random variable
N ij by
N ij = min{n ≥ 1 | X n = j}.

Fix now j ∈ R. For each i ∈ I, let r i be the smallest positive integer n for which
(n)
p > 0. Define
ij
(r i )
and ρ = min p .
r = max r i ij
i∈I i∈I
Since I is finite, we have r < ∞ and ρ > 0. Next observe that
(r i )
P {N ij > r} ≤ P {N ij > r i } = 1 − p ≤ 1 − ρ, i ∈ I.
ij
Thus, for any i ∈ I,
k
P {N ij > kr} ≤ (1 − ρ) , k = 0, 1, . . . .
Since the probability P {N ij > n} is decreasing in n and converges to 0 as n → ∞,
it follows from 1 − f ij = lim n→∞ P {N ij > n} that f ij = 1. Since P {N ij > n} is
decreasing in n, we also obtain

∞ ∞ rk

µ ij = P {N ij > n} = 1 + P {N ij > ℓ}
n=0 k=1 ℓ=r(k−1)+1
∞
k
≤ 1 + r(1 − ρ) ,
k=1
showing that µ ij < ∞. This completes the proof of part (b).
It remains to prove (c). Fix i ∈ I and j ∈ R. As shown above, there is an integer
(v)
v ≥ 1 such that p > 0. By part (b) of Lemma 3.5.5 there is an integer n 0 ≥ 1
ij
(n) (v+n) (v) (n)
such that p > 0 for all n ≥ n 0 . Hence, by p ≥ p p , it follows that
jj ij ij jj