Page 201 - Probability, Random Variables and Random Processes

CHAP. 51 RANDOM PROCESSES

and by Eq. (5.49b), the matrices Q and R are given by

Then

and

By Eq. (5.50),

Thus, the probabilities of absorption into state 0 from states 1 and 2 are given, respectively, by
UIO = - u2,=- q2
and
1 - P9 1-P9
and the probabilities of absorption into state 3 from states 1 and 2 are given, respectively, by

p2
P
UI3 = - u,, = -
and
1-P9 1 - P9
Note that

which confirm the proposition of Prob. 5.39.

5.42. Consider the simple random walk X(n) with absorbing barriers at 0 and 3 (Prob. 5.41). Find the
expected time (or steps) to absorption when X, = 1 and when X, = 2.

The fundamental matrix @ of X(n) is [Eq. (5.1 27)]

Let be the time to absorption when X, = i. Then by Eq. (5.51), we get

5.43. Consider the gambler's game described in Prob. 5.38. What is the probability of A's losing all his
money?

Let P(k), k = 0, 1, 2, . . . , N, denote the probability that A loses all his money when his initial capital is
k dollars. Equivalently, P(k) is the probability of absorption at state 0 when X, = k in the simple random