Page 150 - A First Course In Stochastic Models

THE MODEL 143

The convention p ii = 0 for all states i is convenient and natural. This conven-
tion ensures that the sojourn time in a state is unambiguously defined. If there
are no absorbing states, it is no restriction to make this convention (the sum of a
geometrically distributed number of independent lifetimes with a common expo-
nential distribution is again exponentially distributed). Throughout this chapter the
following assumption is made.

Assumption 4.1.1 In any finite time interval the number of jumps is finite with
probability 1.

Define now the continuous-time stochastic process {X(t), t ≥ 0} by
X(t) = the state of the system at time t.

The process is taken to be right-continuous; that is, at the transition epochs the
state of the system is taken as the state just after the transition. The process {X(t)}
can be shown to be a continuous-time Markov chain. It will be intuitively clear
that the process has the Markov property by the assumption of exponentially dis-
tributed sojourn times in the states. Assumption 4.1.1 is needed to exclude patho-
logical cases. For example, suppose the unbounded state space I = {1, 2, . . . }, take
2
p i,i+1 = 1 and ν i = i for all i. Then transitions occur faster and faster so that the
process will ultimately face an explosion of jumps. With a finite state space the
Assumption 4.1.1 is always satisfied.

Example 4.1.1 Inventory control for an inflammable product
An inflammable product is stored in a special tank at a filling station. Customers
asking for the product arrive according to a Poisson process with rate λ. Each
customer asks for one unit of the product. Any demand that occurs when the tank is
out of stock is lost. Opportunities to replenish the stock in the tank occur according
to a Poisson process with rate µ. The two Poisson processes are assumed to be
independent of each other. For reasons of security it is only allowed to replenish the
stock when the tank is out of stock. At those opportunities the stock is replenished
with Q units for a given value of Q.
To work out the long-run average stock in the tank and the long-run fraction of
demand that is lost, we need to study the inventory process. For any t ≥ 0, define
X(t) = the amount of stock in the tank at time t.
The stochastic process {X(t), t ≥ 0} is a continuous-time Markov chain with
state space I = {0, 1, . . . , Q}. The sojourn time in each state is exponentially
distributed, since both the times between the demand epochs and the times between
the replenishment opportunities are exponentially distributed. Thus the sojourn time
in state i has an exponential distribution with parameter

λ, i = 1, . . . , Q,
ν i =
µ, i = 0.