Page 223 - Computational Statistics Handbook with MATLAB

210 Computational Statistics Handbook with MATLAB

We find the estimated p-value by counting the number of observations in Tm
that are below the value of the observed value of the test statistic and divid-
ing by M.
% Get the p-value. This is a lower tail test.
% Find all of the values from the simulation that are
% below the observed value of the test statistic.
ind = find(Tm <= Tobs);
pvalhat = length(ind)/M;
We have an estimated p-value given by 0.007. If the significance level of our
test is α = 0.05, then we would reject the null hypothesis.

s
MonteC Ca ar rl oloA AA sssesses smme ennt t of ofof of HypotheHypothe ssii issT Tees st tinging
HypotheHypothe
ssessess
ssmm eenntt
inging
TTeesstt
Monte
ss
si
MonteMonte
As
CC aarr lolo
Monte Carlo simulation can be used to evaluate the performance of an infer-
ence model or hypothesis test in terms of the Type I error and the Type II
error. For some statistics, such as the sample mean, these errors can be deter-
mined analytically. However, what if we have an inference test where the
assumptions of the standard methods might be violated or the analytical
methods cannot be applied? For instance, suppose we choose the critical
value by using a normal approximation (when our test statistic is not nor-
mally distributed), and we need to assess the results of doing that? In these
situations, we can use Monte Carlo simulation to estimate the Type I and the
Type II error.
We first outline the procedure for estimating the Type I error. Because the
Type I error occurs when we reject the null hypothesis test when it is true, we
.
must sample from the pseudo-population that represents H 0
PROCEDURE - MONTE CARLO ASSESSMENT OF TYPE I ERROR
1. Determine the pseudo-population when the null hypothesis is true.
2. Generate a random sample of size n from this pseudo-population.
3. Perform the hypothesis test using the critical value.
4. Determine whether a Type I error has been committed. In other
words, was the null hypothesis rejected? We know that it should
not be rejected because we are sampling from the distribution
according to the null hypothesis. Record the result for this trial as,

 1; Type I error is made
I i = 
 0; Type I error is not made.

5. Repeat steps 2 through 4 for M trials.

© 2002 by Chapman & Hall/CRC