Page 236 -

234 R. Seri and D. Secchi

where ˛ 2 Œ0; 1 is called Type-I error rate, i.e. the probability of rejecting the null
hypothesis when it is true, or significance level. 4
be the
Suppose now that the alternative hypothesis is verified and let P H 1
probability distribution under the alternative hypothesis. If the null hypothesis
is associated with the absence of an effect (of a variable on the outcome), the
alternative hypothesis implies generally that there is an effect. It is generally the
5
case that this effect can be measured through a quantity d called effect size, that
. If we suppose that the alternative hypothesis is true,
enters the formulation of P H 1
the probability that T belongs to A or to R is:
fT 2 Ag D ˇ .d/
P H 1
fT 2 Rg D 1 ˇ .d/
P H 1
where ˇ, belonging to Œ0; 1 ,isthe Type-II error rate, i.e. the probability of accepting
the null hypothesis when it is false. Note that ˇ depends on the effect size d.The
quantity 1 ˇ, especially when seen as a function of the effect size d, is called
(statistical) power of the test and measures the probability that the test correctly
identifies the presence of an effect when there is one.
In a hypothetical simulation, for example, one may want to study how decision
making makes employee motivation more effective under conditions of more or less
organized corporate structures (Herath et al. 2017). The null hypothesis may be that
the average motivation does not vary under alternative corporate structures, and this
can be tested using, say, ANOVA. The Type-I error rate or significance level ˛,
that is usually set at 5%, can be used to obtain an acceptance region A. If the null
hypothesis is false, the effect size d—i.e. the “strength” of the effect—measures
the impact that the different conditions exercise, on average, on the outcome. The
probability that the statistic takes a value inside A under the alternative hypothesis is
ˇ and depends on d. If the effect as measured by d is small, the alternative hypothesis
is near to the null hypothesis, and the probability ˇ that the statistic T falls inside A
under H 1 is near to the probability 1 ˛ under H 0 .If d increases, ˇ decreases.
It is clear that there are several degrees of freedom in the choice of the probability
˛, of the statistic T for testing H 0 , and of the acceptance region A. Now, while the
test statistic is often suggested by the problem under scrutiny, the probability ˛ is
chosen routinely from a set of possibilities that have been determined by tradition
more than by reflection. As to the choice of A and R, it is often the case that R

4 We note that Neyman (1950, p. 259) used the term “accept” where most modern treatments
propose to use “fail to reject” or “do not reject.” The original choice of the author is in line with
his idea of testing as leading to decision, while the modern use appears to be incorrectly borrowed
from Fisher’s approach (Fisher 1955, p. 73). However Pearson was more cautious (Pearson 1955,p.
206) and this even suggested to some authors the idea that he had rejected the approach pioneered
with Neyman (Mayo 1992).
5
More generally, the effect size d measures the distance of the true distribution from the distribution
under the null hypothesis, and is generally a function of the parameters.