Page 301 - A Course in Linear Algebra with Applications

8.2: Systems of Linear Recurrences 285

The interesting problem for a Markov process is to deter-
mine the ultimate behavior of the system over a long period of
fc
time, that is to say, limfc_ +00(P ). For the (i,j) entry of this
matrix is the probability that the system will go from state Si
to state Sj in the long run.
The first question to be addressed is whether this limit
always exists. In general the answer is negative, as a very
simple example shows: if P = ( 1, then P k equals either

1 or I 1, according to whether k is even or odd;

so the limit does not exist in this case. Nevertheless it turns
out that under some mild assumptions about the matrix the
limit does exist. Let us call a transition matrix P regular
if some positive power of P has all its entries positive. For

example, the matrix I 1 is regular; indeed all powers
after the first have positive entries. But, as we have seen, the

matrix I I is not regular. A Markov system is said to
be regular if its transition matrix is regular.
The fundamental theorem about Markov processes can
now be stated. A proof may be found in [15], for example.

Theorem 8.2.1
Let P be the transition matrix of a regular Markov system.
A:
Then limfc_^ 00(P ) exists and has the form (XX ... X)
where X is the unique eigenvector of P associated with the
eigenvalue 1 which has entry sum equal to 1.

Our second example of a Markov process is the library
book problem from Chapter One (see Exercise 1.2.12).
Example 8.2.5
A certain library owns 10,000 books. Each month 20% of the
books in the library are lent out and 80% of the books lent out