Math 373 Lecture 19
Type I, II Errors, β and Power
There are two types of error:
Type I error: Falsely rejecting H0. (H0 is true but rejected.)
Type II error: Falsely accepting H0. (H0 false but accepted.)
THEOREM. The probability of falsely rejecting H0 is the
level of significance α.
Proof. Suppose H0 is falsely rejected. Then (a) H0 is true
and (b) the statistic is in the rejection region. The
probability of this is α.
DEFINITION. Let β = P(falsely accepting H0) = P(accepting
H0 when it is false). The power of the test is P(correctly
rejecting H0) = P(accepting Ha when it is true).
THEOREM. The power of the test is 1− β.
Proof. The power = P(correctly rejecting H0) =
P(rejecting H0 when it is false) =
1− P(accepting H0 when it is false) = 1− β.
Since α and β are the probabilities of type I and type II
errors, we want both to be as low as possible. We can
choose the level of significance α to be as low as we
wish, but reducing the probability α of making a type I
error, increases the probability β of making a type II
error. The only way to decrease both types of error is to
increase the sample size.
Suppose the null hypothesis is µ= µ0 where µ0 is the
null value. The alternate hypothesis is Ha: µ= µ0. This
says what µ is not but doesn’t give us enough information
to calculate β or the power. But suppose we believe µ to
be a specific alternate value µa. Then we can replace the
alternate hypothesis µ= µ0 with the more specific
alternative hypothesis Ha: µ= µa.
Given a specific alternate value µa, we can calculate β.
Suppose the acceptance region is µ0+zα/2SE = [a1, a2]. By
definition, β is the probability of accepting H0 when it is
false. Accepting H0 when it is false means:
v Ha is true. (Hence the heavy curve is the true distribution.)
v The statistic µ is in the acceptance region (grey area).
Thus β is the area under the solid heavy bell curve
between a1and a2 (Cross hatched area.)
µ
a
µ
0
a1
a2
In this picture, µa is in (-5, a1]. It could also be in [a1, a2] or [a2, 5).
If the null hypothesis had been true (dotted line), then the
probability of the acceptance region would have been the
shaded area with probability 1− α. But under the alternate
hypothesis µ= µa, the probability of the acceptance region
is the area pictured with diagonal rulings. Under the
alternate hypothesis, the normal curve (heavy solid curve) till
has std. dev. = SE but the mean µ is µa instead of µ0
Suppose the null value is µ0 = 50 and the alternate value
is µa = 52. Suppose SE = 1 and the significance level is α =
5%. Find β and the power of the test.
Find or calculate the following:
v zα/2, and SE, SE is given here but not in the homework.
v acceptance region [a1, a2], use the null hypothesis µ= µ0= 50.
v z-scores of a1 and a2, use alternate hypothesis that µ= µa= 52.
v the probability β= P( [a1, a2]).
zα/2 = 1.96, SE = 1. In your homework, you calculate SE from s.
[a1, a2] = [µ0+ zα/2SE] = [50 + (1.96)(1)] = [48.04, 51.96].
z-score of 48.04* = (µ− µa)/SE = (48.04−52)/1 = -3.96 .
z-score of 51.96 = (µ− µa)/SE = (51.96−52)/1 = -.04 .
Since the tail left of -3.96 has approximately 0 area, we
can replace it with −5.
β = P( zι[-3.96, -.04] ) ≈ P( zι(-5, -.04] ) = .484 .
Thus β = .48 .
The power of the test = 1− β = .52 .
*The acceptance region [a1, a2] is calculated under the
assumption of the null hypothesis and uses the null mean
µ0. But the probability of [a1, a2] is calculated with the
alternate hypothesis. Thus the z-scores of a1, a2 are
computed using the alternate value µa as the mean.
Type II errors are considered less serious than type I
errors but still, β= 48% is an unacceptably high error rate.
β is high since the difference between µ0 and µa is too
close to the margin of error (1.96ξ1). To adequately
distinguish between µ0 and µa, we must increase the
sample size and reduce the margin of error so that µ0 and
µa can be distinguished with a reasonably small (say 10%)
probability of type II error.
In general, we don’t know the alternate value µa. But if
we can estimate how far it might be from µ0, then we can
estimate β as above.
Type II errors (falsely accepting H0 ) are less serious since
H0 is the status quo. No change is needed if H0 is
accepted. A type I error (falsely accepting Ha) is more
serious. It means abandoning the status quo for a false
assumption. There may be a considerable investment
behind the status quo and change may be expensive. In
our legal system, a type II error (freeing a guilty man) is less
serious than a type I error (convicting an innocent man).