Page 109 - A First Course In Stochastic Models

THE EQUILIBRIUM PROBABILITIES 101

the heuristic reasoning

π j = P {X ∞ = j} = P {X ∞ = j | X ∞−1 = k}P {X ∞−1 = k}
k∈I

= p kj π k , j ∈ I. (3.3.11)
k∈I

Example 3.2.1 (continued) The weather as Markov chain
In this example the three-state Markov chain {X n } has no two disjoint closed sets
and thus has a unique equilibrium distribution. The equilibrium probabilities π 1 ,
π 2 and π 3 can be interpreted as the fractions of time the weather is sunny, cloudy
or rainy over a very long period of time. The probabilities π 1 , π 2 and π 3 are the
unique solution to the equilibrium equations

π 1 = 0.70π 1 + 0.50π 2 + 0.40π 3

π 2 = 0.10π 1 + 0.25π 2 + 0.30π 3
π 3 = 0.20π 1 + 0.25π 3 + 0.30π 3
together with the normalizing equation π 1 + π 2 + π 3 = 1. To get a square system
of linear equations, it is permitted to delete one of the equilibrium equations. The
solution is
π 1 = 0.5960, π 2 = 0.1722, π 3 = 0.2318

in accordance with earlier calculations in Section 3.2.

Example 3.1.2 (continued) A stock-control problem
In this example the Markov chain {X n } describing the stock on hand just prior to
review has a finite state space and has no two disjoint closed sets (e.g. state 0 can be
reached from each other state). Hence the Markov chain has a unique equilibrium
distribution. The equilibrium probability π j denotes the long-run fraction of weeks
for which the stock on hand at the end of the week equals j for j = 0, 1, . . . , S.
Thus
s−1

the long-run average frequency of ordering = π j
j=0
S

the long-run average stock on hand at the end of the week = jπ j
j=0