Page 159 - A First Course In Stochastic Models

152 CONTINUOUS-TIME MARKOV CHAINS

A few words of explanation are in order. Intuitively, (4.2.11) may be obvious by
noting that p j gives the long-run fraction of time the stock on hand is j. The long-
t
run average stock on hand is defined as lim t→∞ (1/t) X(u) du. This long-run
0
average can be seen as a long-run average cost per time unit by imagining that a
cost at rate j is incurred when the stock on hand is j. Using this interpretation,
the result (4.2.11) can be seen as a consequence of Theorem 4.2.2, which will be
discussed below. The result (4.2.12) uses the PASTA property: in the long run the
fraction of customers who find the system out of stock upon arrival equals the
fraction of time the system is out of stock. Further, we have
the long-run average number of stock replenishments per time unit = µp 0 .

This result follows from (4.2.6) by noting that the average replenishment frequency
equals the average number of transitions from state 0 to state Q per time unit.

Example 4.1.2 (continued) Unloading ships with an unreliable unloader
In this example we need a regularity condition to ensure that Assumption 4.2.1 is
satisfied (Assumption 4.1.2 trivially holds). Let γ denote the expected amount of
time needed to complete the unloading of a ship. It is not difficult to verify that
γ = µ −1 (1 + δ/β); see (A.5) in Appendix A. In order to satisfy Assumption 4.2.1
it should be required that the arrival rate of ships is less than the reciprocal of the
expected completion time γ . That is, the assumption
βµ
λ <
β + δ
should be made. The proof is omitted that under this condition the expected cycle
length in Assumption 4.2.1 is finite (take state (0, 1) for the regeneration state
r). Denote the equilibrium probabilities by p(j, 0) and p(j, 1). The probability
p(j, 1) gives the long-run fraction of time that j ships are present and the unloader
is available and the probability p(j, 0) gives the long-run fraction of time that j
ships are present and the unloader is in repair. Using the transition rate diagram
in Figure 4.1.2 and applying the ‘rate in = rate out’ principle, we obtain the
equilibrium equations:

λp(0, 1) = µp(1, 1),
(λ + µ + δ)p(i, 1) = λp(i − 1, 1) + µp(i + 1, 1) + βp(i, 0), i = 1, 2, . . . ,
(λ + β)p(1, 0) = δp(1, 1),
(λ + β)p(i, 0) = λp(i − 1, 0) + δp(i, 1), i = 2, 3, . . . .

This infinite system of linear equations together with the normalizing equation
∞ ∞

p(i, 0) + p(i, 1) = 1
i=0 i=1