Page 175 - Schaum's Outlines - Probability, Random Variables And Random Processes

168 RANDOM PROCESSES [CHAP 5

That is, starting from j, the probability of eventual return to j is one. A recurrent state j is said to be
positive recurrent if
E(qIXo =j) < co (5.45)
and state j is said to be null recurrent if
E(qT;.Xo=j)= co

Note that

3. Transient States:
State j is said to be transient (or nonrecurrent) if
fjj=P(q< coIXo=j)< 1
In this case there is positive probability of never returning t~ state j.
4. Periodic and Aperiodic States :
We define the period of state j to be

where gcd stands for greatest common divisor.
If d(j) > 1, then state j is called periodic with period d(j). If d(j) = 1, then state j is called aperiodic.
Note that whenever pjj > 0, j is aperiodic.
5. Absorbing States:
State j is said to be an absorbing state if pjj = 1 ; that is, once state j is reached, it is never left.

E. Absorption Probabilities:
Consider a Markov chain X(n) = {X,, n 2 0) with finite state space E = (1, 2, . . . , N) and tran-
sition probability matrix P. Let A = (1, . . . , m) be the set of absorbing states and B = {m + 1, . . . , N)
be a set of nonabsorbing states. Then the transition probability matrix P can be expressed as

where I is an m x m identity matrix, 0 is an m x (N - m) zero matrix, and

Note that the elements of R are the one-step transition probabilities from nonabsorbing to absorbing
states, and the elements of Q are the one-step transition probabilities among the nonabsorbing states.