Page 280 - A First Course In Stochastic Models

EXERCISES 273

of $5 for each cubic metre of chemical waste that is removed. Compute an average cost
optimal policy by policy iteration or linear programming.
6.5 A stamping machine produces six-cornered plates of the illustrated form.
a
c b

b c
a
The machine has three pairs of adjustable knives. In the diagram these pairs are denoted by
a, b and c. Each pair of knives can fall from the correct position during the stamping of a
plate. The following five situations can occur: (1) all three pairs have the correct position,
(2) only pairs b and c have the correct position, (3) only pair b has the correct position, (4)
only pair c has the correct position and (5) no pair has the correct position. The probabilities
q ij that during a stamping a change from situation i to situation j occurs are given by
3 1 0 0 0
 
4 4
 1 1 1 
 0
2 4 4 0 
 

(q ij ) =  0 0 3 0 1  .

4
 4 
 0 0 0 1 1 

2 2 
0 0 0 0 1
After each stamping it is possible to adjust the machine such that all pairs of knives have
the correct position again. The following costs are involved. The cost of bringing all pairs
of knives into the correct position is 10. Each plate produced when j pairs of knives have
the wrong position involves an adjustment cost of 4j. Compute a maintenance rule that
minimizes the average cost per stamping by policy iteration or linear programming.
6.6 An electricity plant has two generators j = 1 and 2 for generating electricity. The
required amount of electricity fluctuates during the day. The 24 hours in a day are divided
into six consecutive periods of 4 hours each. The amount of electricity required in period
k is d k kWh for k = 1, . . . , 6. Also the generator j has a capacity of generating c j kWh
of electricity per period of 4 hours for j = 1, 2. An excess of electricity produced during
one period cannot be used for the next period. At the beginning of each period k it has to
be decided which generators to use for that period. The following costs are involved. An
operating cost of r j is incurred for each period in which generator j is used. Also, a set-up
cost of S j is incurred each time generator j is turned on after having been idle for some
time. Develop a policy-iteration algorithm that exploits the fact that the state transitions are
deterministic. Solve for the numerical data d 1 = 20, d 2 = 40, d 3 = 60, d 4 = 90, d 5 = 70,
d 6 = 30, c 1 = 40, c 2 = 60, r 1 = 1000, r 2 = 1100, S 1 = 500 and S 2 = 300.
6.7 Every week a repairman travels to customers in five towns on the successive working
days of the week. The repairman visits Amsterdam (town 1) on Monday, Rotterdam (town
2) on Tuesday, Brussels (town 3) on Wednesday, Aachen (town 4) on Thursday and Arnhem
(town 5) on Friday. In the various towns it may be necessary to replace a certain crucial
element in a piece of electronic equipment rented by customers. The probability distribution
of the number of replacements required at a visit to town j is given by {p j (k), k ≥ 0} for
j = 1, . . . , 5. The numbers of required replacements on the successive days are independent
of each other. The repairman is able to carry M spare parts. If the number of spare parts
the repairman carries is not enough to satisfy the demand in a town, another repairman has
to be sent the next day to that town to complete the remaining replacements. The cost of
such a special mission to town j is K j . At the end of each day the repairman may decide to
send for a replenishment of the spare parts to the town where the repairman is. The cost of
sending such a replenishment to town j is a j . Develop a value-iteration algorithm for the