Page 192 - A First Course In Stochastic Models

REFERENCES 185

4.23 Repeat the analysis in Exercise 4.22 when the repair time is H 2 distributed with param-
eters (p 1 , ν 1 , p 2 , ν 2 ) rather than Erlang (R, λ) distributed. Verify that the results remain the
same when we take R = 2 and replace the matrix Q by
 
−δ δp 1 δp 2
Q =  ν 1 −ν 1 0 
ν 2 0 −ν 2
4.24 At a facility for train maintenance, work is done on a number of separate parallel tracks.
On each of these tracks there is room for two trains on a front part and a back part. Trains
can leave the tracks only on the same side they enter the tracks. That is, upon completion of
its maintenance a train may be locked in by another train that arrived later on the same track
but has not yet completed its maintenance. For each of the tracks there are two maintenance
crews, one for the train at the front part of the track and one for the train at the back. Trains
requesting maintenance arrive at the maintenance facility according to a Poisson process
with rate λ. A train immediately receives maintenance when it finds a free place at one of
the tracks upon arrival; otherwise, the train waits until a maintenance place becomes free.
A newly arriving train is directed to a front part if both a front part and a back part are free.
The amount of time needed to serve a train has an exponential distribution with mean 1/µ.
3
It is assumed that λ < cµ.
2
(a) Formulate a continuous-time Markov time chain for the performance evaluation of
the maintenance track.
(b) Argue that the geometric tail approach can be used to reduce the infinite system of
equilibrium equations to a finite system of linear equations. This problem is based on Adan
et al. (1999).

BIBLIOGRAPHIC NOTES

The theory of continuous-time Markov chains is more delicate than the theory
of discrete-time Markov chains. Basic references are Anderson (1991) and Chung
(1967). The continuous-time Markov chain model is the most versatile model in
applied probability. The powerful technique of equating the flow out of a state
to the flow into that state has a long history and goes back to the pioneering
work of Erlang on stochastic processes in the early 1900s; see also Kosten (1973).
The uniformization technique for the transient analysis of continuous-time Markov
chains goes back to Jensen (1953) and is quite useful for both analytical and
computational purposes. The extension of the uniformization method to compute the
transient probability distribution of the sojourn time in a given set of states is due to
De Soua e Silva and Gail (1986). The material in Section 4.6.2 for the computation
of the transient reward distribution is based on Goyal and Tantawi (1988) and Tijms
and Veldman (2000); see also Sericola (2000) for an alternative method. The Hubble
telescope problem from Example 4.5.3 is taken from Hermanns (2001).

REFERENCES
Adan, I.J.B.F. and Resing, J.A.C. (1999) A class of Markov processes on a semi-infinite
strip. In Proc. 3rd International Meeting on the Numerical Solution of Markov Chains,
edited by B. Plateau, W.J. Stewart and M. Silva, pp. 41–57. Zaragoza University Press.