Page 224 - Computational Statistics Handbook with MATLAB

Chapter 6: Monte Carlo Methods for Inferential Statistics 211

6. The probability of making a Type I error is

M
ˆ 1
α = ----- ∑ I i . (6.9)
M
i = 1
Note that in step 6, this is the same as calculating the proportion of times the
null hypothesis is falsely rejected out of M trials. This provides an estimate of
the significance level of the test for a given critical value.
The procedure is similar for estimating the Type II error of a hypothesis
test. However, this error is determined by sampling from the distribution
when the null hypothesis is false. There are many possibilities for the Type II
error, and the analyst should investigate the Type II error for those alternative
hypotheses that are of interest.

PROCEDURE - MONTE CARLO ASSESSMENT OF TYPE II ERROR

1. Determine a pseudo-population of interest where the null hypoth-
esis is false.
2. Generate a random sample of size n from this pseudo-population.
3. Perform the hypothesis test using the significance level α and
corresponding critical value.
4. Note whether a Type II error has been committed; i.e., was the null
hypothesis not rejected? Record the result for this trial as,

 1; Type II error is made
I i = 
 0; Type II error is not made.

5. Repeat steps 2 through 4 for M trials.
6. The probability of making a Type II error is

M
ˆ 1
β = ----- ∑ I i . (6.10)
M
i = 1
The Type II error rate is estimated using the proportion of times the null
hypothesis is not rejected (when it should be) out of M trials.

Example 6.8
For the hypothesis test in Example 6.6, we had a critical value (from theory)
of -1.645. We can estimate the significance level of the test using the following
steps:

© 2002 by Chapman & Hall/CRC