Page 147 - A First Course In Stochastic Models

REFERENCES 139

BIBLIOGRAPHIC NOTES

Many good textbooks on stochastic processes are available and most of them treat
the topic of Markov chains. My favourite books include Cox and Miller (1965),
Karlin and Taylor (1975) and Ross (1996), each offering an excellent introduction
to Markov chain theory. A very fundamental treatment of denumerable Markov
chains can be found in the book of Chung (1967). An excellent book on Markov
chains with a general state space is Meyn and Tweedie (1993). The concept of the
embedded Markov chain and its application in Example 3.1.3 are due to Kendall
(1953). The idea of using the geometric tail behaviour of state probabilities goes
back to Feller (1950) and was successfully used in the papers of Everett (1954)
and Takahashi and Takami (1976).

REFERENCES

Chung, K.L. (1967) Markov Chains with Stationary Transition Probabilities, 2nd edn.
Springer-Verlag, Berlin.
Cox, D.R. and Miller, H.D. (1965) The Theory of Stochastic Processes. Chapman and Hall,
London.
Everett, J. (1954) State probabilities in congestion problems characterized by constant hold-
ing times. Operat. Res., 1, 279–285.
Feller, W. (1950) An Introduction to Probability Models and its Applications, Vol. I, John
Wiley & Sons, Inc., New York.
Fox, B. and Landi, D.M. (1968) An algorithm for identifying the ergodic subchains and
transient states of a stochastic matrix. Commun. ACM , 11, 619–621.
Karlin, S. and Taylor, H.M. (1975) A First Course in Stochastic Processes, 2nd edn. Aca-
demic Press, New York.
Kendall, D.G. (1953) Stochastic processes occurring in the theory of queues and their anal-
ysis by the method of the embedded Markov chain. Ann. Math. Statist., 24, 338–354.
Markov, A.A. (1906) Extension of the law of large numbers to dependent events (in Russian).
Bull. Soc. Phys. Math. Kazan, 15, 255–261.
Meyn, S.P. and Tweedie, R. (1993) Markov Chains and Stochastic Stability. Springer-Verlag,
Berlin.
Ross, S.M. (1996) Stochastic Processes, 2nd edn., John Wiley & Sons, Inc., New York.
Stewart, W.J. (1994) Introduction to the Numerical Solution of Markov Chains. Princeton
University Press, Princeton NJ.
Takahashi, Y. and Takami, Y. (1976) A numerical method for the steady-state probabilities of
a GI/G/c queueing system in a general class. J. Operat. Res. Soc. Japan, 19, 147–157.