Page 186 - A First Course In Stochastic Models

EXERCISES 179

Then the cumulative reward variable R(t) represents the number of repair missions
that will be prepared up to time t. Note that in this particular case the stochas-
tic variable R(t) has a discrete distribution rather than a continuous distribution.
However, the discretization algorithm also applies to the case of a reward variable
R(t) with a non-continuous distribution. For the numerical example with λ = 0.1,
µ = 100 and η = 5 we found that P {R(t) > k} has the respective values 0.6099,
0.0636 and 0.0012 for k = 0, 1 and 2 (accurate to four decimal places with
= 1/256).

EXERCISES

4.1 A familiar sight in Middle East street scenes are the so-called sheroots. A sheroot is a
seven-seat cab that drives from a fixed stand in a town to another town. A sheroot leaves
as soon as all seven seats are occupied by passengers. Consider a sheroot stand which has
room for only one sheroot. Potential passengers arrive at the stand according to a Poisson
process at rate λ. If upon arrival a potential customer finds no sheroot present and seven
other customers already waiting, the customer goes elsewhere for transport; otherwise, the
customer waits until a sheroot departs. After a sheroot leaves the stand, it takes an exponential
time with mean 1/µ until a new sheroot becomes available.
Formulate a continuous-time Markov chain model for the situation at the sheroot stand.
Specify the state variable(s) and the transition rate diagram.
4.2 In a certain city there are two emergency units, 1 and 2, that cooperate in responding
to accident alarms. The alarms come into a central dispatcher who sends one emergency
unit to each alarm. The city is divided in two districts, 1 and 2. The emergency unit i
is the first-due unit for response area i for i = 1, 2. An alarm coming in when only
one of the emergency units is available is handled by the idle unit. If both units are not
available, the alarm is settled by some unit from outside the city. Alarms from the districts
1 and 2 arrive at the central dispatcher according to independent Poisson processes with
respective rates λ 1 and λ 2 . The amount of time needed to serve an alarm from district
j by unit i has an exponential distribution with mean 1/µ ij . The service times include
travel times.
Formulate a continuous-time Markov chain model to analyse the availability of the emer-
gency units. Specify the state variable(s) and the transition rate diagram.
4.3 An assembly line for a certain product has two stations in series. Each station has only
room for a single unit of the product. If the assembly of a unit is completed at station 1, it
is forwarded immediately to station 2 provided station 2 is idle; otherwise the unit remains
in station 1 until station 2 becomes free. Units for assembly arrive at station 1 according to
a Poisson process with rate λ, but a newly arriving unit is only accepted by station 1 when
no other unit is present in station 1. Each unit rejected is handled elsewhere. The assembly
times at stations 1 and 2 are exponentially distributed with respective means 1/µ 1 and 1/µ 2 .
Formulate a continuous-time Markov chain to analyse the situation at both stations. Spec-
ify the state variable(s) and the transition rate diagram.
4.4 Cars arrive at a gasoline station according to a Poisson process with an average of
10 customers per hour. A car enters the station only if less than four other cars are present.
The gasoline station has only one pump. The amount of time required to serve a car has an
exponential distribution with a mean of four minutes.
(a) Formulate a continuous-time Markov chain to analyse the situation of the gasoline
station. Specify the state diagram.
(b) Solve the equilibrium equations.