Page 52 - A First Course In Stochastic Models

RENEWAL-REWARD PROCESSES 43

Example 2.2.1 Alternating up- and downtimes
Suppose a machine is alternately up and down. Denote by U 1 , U 2 , . . . the lengths
of the successive up-periods and by D 1 , D 2 , . . . the lengths of the successive
down-periods. It is assumed that both {U n } and {D n } are sequences of independent
and identically distributed random variables with finite positive expectations. The
sequences {U n } and {D n } are not required to be independent of each other. Assume
that an up-period starts at epoch 0. What is the long-run fraction of time the machine
is down? The answer is

the long-run fraction of time the machine is down
E(D 1 )
= with probability 1. (2.2.1)
E(U 1 ) + E(D 1 )
To verify this, define the continuous-time stochastic process {X(t), t ≥ 0} by

1 if the machine is up at time t,
X(t) =
0 otherwise.
The process {X(t)} is a regenerative process. The epochs at which an up-period
starts can be taken as regeneration epochs. The long-run fraction of time the
machine is down can be interpreted as a long-run average cost per time unit
by assuming that a cost at rate 1 is incurred while the machine is down and
a cost at rate 0 otherwise. A regeneration cycle consists of an up-period and a
down-period. Hence
E(length of one cycle) = E(U 1 + D 1 )

and

E(cost incurred during one cycle) = E(D 1 ).
By applying the renewal-reward theorem, it follows that the long-run average cost
per time unit equals E(D 1 )/[E(U 1 ) + E(D 1 )], proving the result (2.2.1).
The intermediate step of interpreting the long-run fraction of time that the process
is in a certain state as a long-run average cost (reward) per time unit is very helpful
in many situations.

Limit theorems for regenerative processes
An important application of the renewal-reward theorem is the characterization
of the long-run fraction of time a regenerative process {X(t), t ∈ T } spends in
some given set B of states. For the set B of states, define for any t ∈ T the
indicator variable

1 if X(t) ∈ B,
I B (t) =
0 if X(t) /∈ B.