Page 157 - A First Course In Stochastic Models

150 CONTINUOUS-TIME MARKOV CHAINS

Interpretation of the equilibrium equations
A physical explanation of the equilibrium equations can be given by using the
obvious principle that over the long run
the average number of transitions out of state j per time unit

= the average number of transitions into state j per time unit. (4.2.4)
Since p j is the long-run fraction of time the process is in state j and the leaving
rate out of state j is ν j , it is intuitively obvious that

the long-run average number of transitions out of state j
per time unit = ν j p j . (4.2.5)

Also, the following result will be intuitively obvious:

the long-run average number of transitions from state k to state j
per time unit = q kj p k . (4.2.6)

For a better understanding of (4.2.6), it is helpful to point out that q kj can be
interpreted as the long-run average number of transitions per time unit to state j
when averaging over the time the process is in state k. A rigorous proof of the
result (4.2.6) is given in Section 4.3. By (4.2.6),

the long-run average number of transitions into state j

per time unit = q kj p k . (4.2.7)
k =j
Together (4.2.4), (4.2.5) and (4.2.7) give the equilibrium equations (4.2.2). These
equations may be abbreviated as

rate out of state j = rate into state j. (4.2.8)

This principle is the flow rate equation method. To formulate the equilibrium
equations in specific applications, it is convenient to use the transition rate diagram
that was introduced in the previous section. Putting the transition rate diagram in a
physical context, one might think that particles with a total mass of 1 are distributed
over the nodes according to the equilibrium distribution {p j }. Particles move from
one node to another node according to the transition rates q ij . In the equilibrium
situation the rate at which particles leave any node must be equal to the rate at
which particles enter that node. The ‘rate in = rate out’ principle (4.2.8) allows for
a very useful generalization. More generally, for any set A of states with A = I,
rate out of the set A = rate into the set A. (4.2.9)