Page 200 - Probability, Random Variables and Random Processes

192 RANDOM PROCESSES [CHAP 5

Now since lirn,, ,(l - p), = 0, we have
lim P{Xn E B) = 0 or lim P{Xn E B = A) = 1
n+ co n+ a,
which shows that absorption of X(n) in one or another of the absorption states is certain.

5.40. Verify Eq. (5.50).
Let X(n) = {X, , n 2 0) be a homogeneous Markov chain with a finite state space E = (0, 1, . . . , N}, of
which A = (0, 1, . . . , m), m 2 1, is a set of absorbing states and B = {m + 1, . . . , N) is a set of nonabsorbing
states. Let state k E B at the first step go to i E E with probability p,, . Then

ukj = P{Xn = j(~ A) I X, = k(~ B))

Now

Then Eq. (5.1 24) becomes

But pkj, k = m + 1, ..., N; j = 1, ..., m, are the elements of R, whereaspki, k = m + 1, ..., N; i = rn + 1, ...,
N are the elements of Q [see Eq. (5.49a)I. Hence, in matrix notation, Eq. (5.125) can be expressed as
U=R+QU or (I-Q)U=R (5.126)
Premultiplying both sides of the second equation of Eq. (5.126) with (I - Q)-', we obtain
u=(I-Q)-'R=@R

5.41. Consider a simple random walk X(n) with absorbing barriers at state 0 and state N = 3 (see
Prob. 5.38).
(a) Find the transition probability matrix P.
(b) Find the probabilities of absorption into states 0 and 3.
(a) The transition probability matrix P is [Eq. (5.1 23)]

(b) Rearranging the transition probability matrix P as [Eq. (5.49a)],