Page 178 - A First Course In Stochastic Models

TRANSIENT STATE PROBABILITIES 171

absorbing state is to take its leaving rate equal to zero. In the modified continuous-
time Markov chain the set C is replaced by a single absorbing state to be denoted
by a. The state space I and the leaving rates ν in the modified continuous-time
∗
∗
i
Markov chain are taken as

ν i , i ∈ I\C,
∗ ∗
I = (I\C) ∪ {a} and ν =
i 0, i = a.
The infinitesimal transition rates q are taken as
∗
ij

q ij , i, j ∈ I\C with j = i,

∗
q = q ik , i ∈ I\C, j = a,
ij k∈C
0, i = a, j ∈ I\C.

Denoting by p (t) the transient state probabilities in the modified continuous-time
∗
ij
Markov chain, it is readily seen that
∗
Q iC (t) = 1 − p (t), i /∈ C and t ≥ 0.
ia
The p (t) can be computed by using the uniformization algorithm in the previous
∗
ij
subsection (note that p ∗ = 1 in the uniformization algorithm).
aa
Example 4.5.3 The Hubble space telescope
The Hubble space telescope is an astronomical observatory in space. It carries a
variety of instruments, including six gyroscopes to ensure stability of the telescope.
The six gyroscopes are arranged in such a way that any three gyroscopes can keep
the telescope operating with full accuracy. The operating times of the gyroscopes
are independent of each other and have an exponential distribution with failure
rate λ. Upon a fourth gyroscope failure, the telescope goes into sleep mode. In
sleep mode, further observations by the telescope are suspended. It requires an
exponential time with mean 1/µ to put the telescope into sleep mode. Once the
telescope is in sleep mode, the base station on earth receives a sleep signal. A shuttle
mission to the telescope is next prepared. It takes an exponential time with mean
1/η before the repair crew arrives at the telescope and has repaired the stabilizing
unit with the gyroscopes. In the meantime the other two gyroscopes may fail. If
the last gyroscope fails, a crash destroying the telescope will be inevitable. What
is the probability that the telescope will crash in the next T years?
This problem can be analysed by a continuous-time Markov chain with an
absorbing state. The transition diagram is given in Figure 4.5.1. The state labelled
as the crash state is the absorbing state. As explained above, this convention enables
us to apply the uniformization method for the state probabilities to compute the
first passage time probability

Q(T ) = P {no crash will occur in the next T years
when currently all six gyroscopes are working}.