Page 243 - Elements of Distribution Theory

P1: JZX
052184472Xc07 CUNY148/Severini May 24, 2005 3:59

7.7 Monte Carlo Methods 229

ˆ
When this approach was used with N = 1,000, the estimate P N = 0.244 was obtained; when
ˆ
N = 10,000 was used, the estimate P N = 0.2464 was obtained. Recall that the distribution
of Y is uniform on the interval (0, 1); thus, the exact probability Pr(Y ≤ 1/4) is 1/4.

Of course, use of the Monte Carlo method requires that we be able to generate random
variables with a specified distribution. Standard statistical packages generally have proce-
dures for generating data from commonly used distributions. In many cases if the density
or distribution function of X is available, then it is possible to construct a method for gen-
erating data with the same distribution as X; this is particularly true if X is real-valued or
is a random vector with independent components. These methods are generally based on
a procedure to generate variables that are uniformly distributed on (0, 1); such procedures
are well-studied and widely available. However, depending on the exact distribution of X,
the actual method required to generate the data may be quite sophisticated. The following
two examples illustrate some simple methods that are often applicable.

Example 7.30 (Generation of standard exponential random variables). Consider genera-
tion of observations with a standard exponential distribution, required in Example 7.29. Let
U denote a random variable that is uniformly distributed on (0, 1) and let X =− log(1 − U).
Then

Pr(X ≤ x) = Pr(U ≤ 1 − exp(−x)) = 1 − exp(−x), x > 0.

Hence, a sequence of independent uniform random variables may be easily transformed to
a sequence of independent standard exponential random variables.
This approach, sometimes called the inversion method, can be used whenever X has
distribution function F on the real line and the quantile function corresponding to F is
available; see the proof of Theorem 1.3.

Example 7.31 (Hierarchical models). Suppose that the random variable X follows a hier-
archical model, as discussed in Section 5.4. Specifically, suppose that the distribution of X
can be described in two stages: the conditional distribution of X given a random variable
λ and the marginal distribution of λ.If algorithms for generating data from the conditional
distribution of X given λ and from the marginal distribution of λ are both available, then
random variables from the distribution of X may be generated using a two-stage process.
For each j = 1,..., N, suppose that λ j is drawn from the marginal distribution of λ; then
we can draw X j from the conditional distribution of X given λ = λ j .
This method can also be used in cases in which the distribution of X is not originally
described in terms of a hierarchical model, but it is possible to describe the distribution of
X in terms of a hierarchical model.

The primary advantage of the Monte Carlo method is that it may be used in (nearly)
every problem; it is particularly useful in cases, such as the one in the following example,
in which an exact analysis is very difficult.

Example 7.32 (An implicitly defined statistic). Let X = (Z 1 ,..., Z n ) where the Z j are
independent and identically distributed standard exponential random variables and consider