Page 193 - Probability, Random Variables and Random Processes

CHAP. 51 RANDOM PROCESSES

By Eq. (5.107),

1 1
Thus Var(&) = - [n(l) + 2(n - l)(3) + 03 = - (2n - 1)
n2 n

DISCRETE-PARAMETER MARKOV CHAINS
5.28. Show that if P is a Markov matrix, then Pn is also a Markov matrix for any positive integer n.

Let

Then by the property of a Markov matrix [Eq. (5.391, we can write

where aT=[l 1 11

Premultiplying both sides of Eq. (5.1 11) by P, we obtain
P2a = Pa = a
which indicates that P2 is also a Markov matrix. Repeated premultiplication by P yields

which shows that P" is also a Markov matrix.

5.29. Verify Eq. (5.39); that is,

We verify Eq. (5.39) by induction. If the state of X, is i, state XI will be j only if a transition is made
from i to j. The events {X, = i, i = 1, 2, . . .} are mutually exclusive, and one of them must occur. Hence, by
the law of total probability [Eq. (1.44)],

In terms of vectors and matrices, Eq. (5.1 12) can be expressed as
~(1) = P(0)P
Thus, Eq. (5.39) is true for n = 1. Assume now that Eq. (5.39) is true for n = k; that is,
PW = pWk