Page 194 - Probability, Random Variables and Random Processes

186 RANDOM PROCESSES [CHAP 5

Again, by the law of total probability,

In terms of vectors and matrices, Eq. (5.1 14) can be expressed as

which indicates that Eq. (5.39) is true for k + 1. Hence, we conclude that Eq. (5.39) is true for all n 2 1.

5.30. Consider a two-state Markov chain with the transition probability matrix

(a) Show that the n-step transition probability matrix Pn is given by

(b) Find Pn when n -, a.
(a) From matrix analysis, the characteristic equation of P is

Thus, the eigenvalues of P are 1, = 1 and A, = 1 - a - b. Then, using the spectral decomposition
method, Pn can be expressed as
Pn = AlnE, + A2"E2 (5.1 18)
where El and E, are constituent matrices of P, given by
1 1
El =- [p - 1211 E, =- [f' - 1111 (5.1 19)
11 - 12 12 - A1
Substituting 1, = 1 and 1, = 1 - a - b in the above expressions, we obtain

Thus, by Eq. (5.1 18), we obtain

(b) IfO<a<l,O<b<l,thenO< 1-a< 1andI1-a-bI< l.Solimn,,(l-a-b)"=Oand

Note that a limiting matrix exists and has the same rows (see Prob. 5.47).

5.31. An example of a two-state Markov chain is provided by a communication network consisting of
the sequence (or cascade) of stages of binary communication channels shown in Fig. 5-9. Here X,