Page 197 - Probability, Random Variables and Random Processes

CHAP. 51 RANDOM PROCESSES 189

Fig. 5-10 State transition diagram.

(a) The state transition diagram of the Markov chain with P of part (a) is shown in Fig. 5-lqa). From Fig.
5-10(a), it is seen that the Markov chain is irreducible and aperiodic. For instance, one can get back to
state 0 in two steps by going from 0 to 1 to 0. However, one can also get back to state 0 in three steps
by going from 0 to 1 to 2 to 0. Hence 0 is aperiodic. Similarly, we can see that states 1 and 2 are also
aperiodic.
(b) The state transition diagram of the Markov chain with P of part (b) is shown in Fig. 5-10(b). From Fig.
5-10(b), it is seen that the Markov chain is irreducible and periodic with period 3.
(c) The state transition diagram of the Markov chain with P of part (c) is shown in Fig. 5-10(c). From Fig.
5-10(c), it is seen that the Markov chain is not irreducible, since states 0 and 4 do not communicate,
and state 1 is absorbing.

5.36. Consider a Markov chain with state space (0, 1) and transition probability matrix

(a) Show that state 0 is recurrent.
(b) Show that state 1 is transient.
(a) By Eqs. (5.41) and (5.42), we have

Then, by Eqs. (5.43),

Thus, by definition (5.44), state 0 is recurrent.
(b) Similarly, we have

00
and fll=P(Tl <mIXo=l)= fl1(")=i+O+O+-.-=~<1
Thus, by definition (5.48), state 1 is transient.