Page 65 - A First Course In Stochastic Models

56 RENEWAL-REWARD PROCESSES

specific problem let the continuous-time stochastic process {X(t), t ≥ 0} describe
the evolution of the state of a system and let {N(t), t ≥ 0} be a renewal process
describing arrivals to that system. As examples:

(a) X(t) is the number of customers present at time t in a queueing system.
(b) X(t) describes jointly the inventory level and the prevailing production rate at
time t in a production/inventory problem with a variable production rate.

It is assumed that the arrival process {N(t), t ≥ 0} can be seen as an exogenous
factor to the system and is not affected by the system itself. More precisely, the
following assumption is made.

Lack of anticipation assumption For each u ≥ 0 the future arrivals occurring
after time u are independent of the history of the process {X(t)} up to time u.

It is not necessary to specify how the arrival process {N(t)} precisely interacts
with the state process {X(t)}. Denoting by τ n the nth arrival epoch, let the random
−
variable X n be defined by X(τ ). In other words,
n
X n = the state of the system just prior to the nth arrival epoch.
Let B be any set of states for the {X(t)} process. For each t ≥ 0, define the
indicator variable

1 if X(t) ∈ B,
I B (t) =
0 otherwise.
Also, for each n = 1, 2, . . . , define the indicator variable I n (B) by

1 if X n ∈ B,
I n (B) =
0 otherwise.
The technical assumption is made that the sample paths of the continuous-time
process {I B (t), t ≥ 0} are right-continuous and have left-hand limits. In practical
situations this assumption is always satisfied.

Theorem 2.4.1 (Poisson arrivals see time averages) Suppose that the arrival
process {N(t)} is a Poisson process with rate λ. Then:

(a) For any t > 0,

E[number of arrivals in (0, t) finding the system in the set B]
t

= λE I B (u) du .
0