Math 373 Lecture 19 Type I, II Errors, β and Power There are two types of error: Type I error: Falsely rejecting H0. (H0 is true but rejected.) Type II error: Falsely accepting H0. (H0 false but accepted.) THEOREM. The probability of falsely rejecting H0 is the level of significance α. Proof. Suppose H0 is falsely rejected. Then (a) H0 is true and (b) the statistic is in the rejection region. The probability of this is α. DEFINITION. Let β = P(falsely accepting H0) = P(accepting H0 when it is false). The power of the test is P(correctly rejecting H0) = P(accepting Ha when it is true). THEOREM. The power of the test is 1− β. Proof. The power = P(correctly rejecting H0) = P(rejecting H0 when it is false) = 1− P(accepting H0 when it is false) = 1− β. Since α and β are the probabilities of type I and type II errors, we want both to be as low as possible. We can choose the level of significance α to be as low as we wish, but reducing the probability α of making a type I error, increases the probability β of making a type II error. The only way to decrease both types of error is to increase the sample size. Suppose the null hypothesis is µ= µ0 where µ0 is the null value. The alternate hypothesis is Ha: µ= µ0. This says what µ is not but doesn’t give us enough information to calculate β or the power. But suppose we believe µ to be a specific alternate value µa. Then we can replace the alternate hypothesis µ= µ0 with the more specific alternative hypothesis Ha: µ= µa. Given a specific alternate value µa, we can calculate β. Suppose the acceptance region is µ0+zα/2SE = [a1, a2]. By definition, β is the probability of accepting H0 when it is false. Accepting H0 when it is false means: v Ha is true. (Hence the heavy curve is the true distribution.) v The statistic µ is in the acceptance region (grey area). Thus β is the area under the solid heavy bell curve between a1and a2 (Cross hatched area.) µ a µ 0 a1 a2 In this picture, µa is in (-5, a1]. It could also be in [a1, a2] or [a2, 5). If the null hypothesis had been true (dotted line), then the probability of the acceptance region would have been the shaded area with probability 1− α. But under the alternate hypothesis µ= µa, the probability of the acceptance region is the area pictured with diagonal rulings. Under the alternate hypothesis, the normal curve (heavy solid curve) till has std. dev. = SE but the mean µ is µa instead of µ0 Suppose the null value is µ0 = 50 and the alternate value is µa = 52. Suppose SE = 1 and the significance level is α = 5%. Find β and the power of the test. Find or calculate the following: v zα/2, and SE, SE is given here but not in the homework. v acceptance region [a1, a2], use the null hypothesis µ= µ0= 50. v z-scores of a1 and a2, use alternate hypothesis that µ= µa= 52. v the probability β= P( [a1, a2]). zα/2 = 1.96, SE = 1. In your homework, you calculate SE from s. [a1, a2] = [µ0+ zα/2SE] = [50 + (1.96)(1)] = [48.04, 51.96]. z-score of 48.04* = (µ− µa)/SE = (48.04−52)/1 = -3.96 . z-score of 51.96 = (µ− µa)/SE = (51.96−52)/1 = -.04 . Since the tail left of -3.96 has approximately 0 area, we can replace it with −5. β = P( zι[-3.96, -.04] ) ≈ P( zι(-5, -.04] ) = .484 . Thus β = .48 . The power of the test = 1− β = .52 . *The acceptance region [a1, a2] is calculated under the assumption of the null hypothesis and uses the null mean µ0. But the probability of [a1, a2] is calculated with the alternate hypothesis. Thus the z-scores of a1, a2 are computed using the alternate value µa as the mean. Type II errors are considered less serious than type I errors but still, β= 48% is an unacceptably high error rate. β is high since the difference between µ0 and µa is too close to the margin of error (1.96ξ1). To adequately distinguish between µ0 and µa, we must increase the sample size and reduce the margin of error so that µ0 and µa can be distinguished with a reasonably small (say 10%) probability of type II error. In general, we don’t know the alternate value µa. But if we can estimate how far it might be from µ0, then we can estimate β as above. Type II errors (falsely accepting H0 ) are less serious since H0 is the status quo. No change is needed if H0 is accepted. A type I error (falsely accepting Ha) is more serious. It means abandoning the status quo for a false assumption. There may be a considerable investment behind the status quo and change may be expensive. In our legal system, a type II error (freeing a guilty man) is less serious than a type I error (convicting an innocent man).