Inference in Practice
BPS chapter 16
© 2006 W.H. Freeman and Company
Significance level
A certain manufacturer of paints uses an additive to get the drying time
for a specific paint to be 75 minutes. If there’s too much additive,
the drying time could be longer than specified but too little additive
will decrease the drying time. In testing the amount of additive, they
use these hypotheses: H0:  = 7 ml vs. Ha:   7 ml. Which of the
following would be an implication of having a small ?
a)
b)
Concluding that the mean amount of additive is different from 7 ml
more often.
Concluding that the mean amount of additive is not different from 7
ml more often.
Significance level (answer)
A certain manufacturer of paints uses an additive to get the drying time
for a specific paint to be 75 minutes. If there’s too much additive,
the drying time could be longer than specified but too little additive
will decrease the drying time. In testing the amount of additive, they
use these hypotheses: H0:  = 7 ml vs. Ha:   7 ml. Which of the
following would be an implication of having a small ?
a)
b)
Concluding that the mean amount of additive is different from 7 ml
more often.
Concluding that the mean amount of additive is not different
from 7 ml more often.
Statistical significance
A group of researchers wanted to know if there was a difference in
average yearly income taxes paid between residents of two very
large cities in the midwestern United States. The average for the
first city was $6,505 and for the second city, it was $6,511. The
difference provided a P-value of 0.0007. Were these results
statistically significant?
a)
b)
c)
d)
No, because a $6 difference is probably too small to really matter.
No, because the P-value is small.
Yes, because the P-value is small.
Yes, because the difference of $6 is bigger than 0.
Statistical significance (answer)
A group of researchers wanted to know if there was a difference in
average yearly income taxes paid between residents of two very
large cities in the midwestern United States. The average for the
first city was $6,505 and for the second city, it was $6,511. The
difference provided a P-value of 0.0007. Were these results
statistically significant?
a)
b)
c)
d)
No, because a $6 difference is probably too small to really matter.
No, because the P-value is small.
Yes, because the P-value is small.
Yes, because the difference of $6 is bigger than 0.
Practical significance
A group of researchers wanted to know if there was a difference in
average yearly income taxes paid between residents of two very
large cities in the midwestern United States. The average for the
first city was $6,505 and for the second city, it was $6,511. The
difference provided a P-value of 0.0007. Were these results
practically significant?
a)
b)
c)
d)
No, because a $6 difference is probably too small to really matter.
No, because the P-value is small.
Yes, because the P-value is small.
Yes, because the difference of $6 is bigger than 0.
Practical significance (answer)
A group of researchers wanted to know if there was a difference in
average yearly income taxes paid between residents of two very
large cities in the midwestern United States. The average for the
first city was $6,505 and for the second city, it was $6,511. The
difference provided a P-value of 0.0007. Were these results
practically significant?
a)
b)
c)
d)
No, because a $6 difference is probably too small to really
matter.
No, because the P-value is small.
Yes, because the P-value is small.
Yes, because the difference of $6 is bigger than 0.
Type I error
Which of the following defines Type I error?
a)
b)
c)
d)
Reject H0 when H0 is true.
Reject H0 when H0 is false.
Do not reject H0 when H0 is true.
Do not reject H0 when H0 is false.
Type I error (answer)
Which of the following defines Type I error?
a)
b)
c)
d)
Reject H0 when H0 is true.
Reject H0 when H0 is false.
Do not reject H0 when H0 is true.
Do not reject H0 when H0 is false.
Type II error
Which of the following defines Type II error?
a)
b)
c)
d)
Reject H0 when H0 is true.
Reject H0 when H0 is false.
Do not reject H0 when H0 is true.
Do not reject H0 when H0 is false.
Type II error (answer)
Which of the following defines Type II error?
a)
b)
c)
d)
Reject H0 when H0 is true.
Reject H0 when H0 is false.
Do not reject H0 when H0 is true.
Do not reject H0 when H0 is false.
Type I and Type II error
If we fail to reject the null hypothesis, we may have made
a)
b)
c)
A Type I error.
A Type II error.
Either a Type I or Type II error.
Type I and Type II error (answer)
If we fail to reject the null hypothesis, we may have made
a)
b)
c)
A Type I error.
A Type II error.
Either a Type I or Type II error.
Type I error
Suppose that a regulatory agency will propose that Congress cut
federal funding to a metropolitan area if its mean level of NOx is
unsafe—that is, if it exceeds 5.0 ppt. The agency gathers sample
NOx concentrations on 60 different days and calculates a test of
significance to assess whether the mean level of NOx is greater
than 5.0 ppt.
What is a Type I error in context?
a)
b)
c)
d)
Believing the mean level of NOx exceeds 5.0 ppt when it really
does.
Believing the mean level of NOx exceeds 5.0 ppt when it really does
not.
Believing the mean level of NOx is 5.0 ppt or less when it really is.
Believing the mean level of NOx is 5.0 ppt or less when it really is
not.
Type I error (answer)
Suppose that a regulatory agency will propose that Congress cut
federal funding to a metropolitan area if its mean level of NOx is
unsafe—that is, if it exceeds 5.0 ppt. The agency gathers sample
NOx concentrations on 60 different days and calculates a test of
significance to assess whether the mean level of NOx is greater
than 5.0 ppt.
What is a Type I error in context?
a)
b)
c)
d)
Believing the mean level of NOx exceeds 5.0 ppt when it really
does.
Believing the mean level of NOx exceeds 5.0 ppt when it really
does not.
Believing the mean level of NOx is 5.0 ppt or less when it really is.
Believing the mean level of NOx is 5.0 ppt or less when it really is
not.
Type II error
Suppose that a regulatory agency will propose that Congress cut
federal funding to a metropolitan area if its mean level of NOx is
unsafe—that is, if it exceeds 5.0 ppt. The agency gathers sample
NOx concentrations on 60 different days and calculates a test of
significance to assess whether the mean level of NOx is greater
than 5.0 ppt.
Which of the following best describes the implications of a Type II
error?
a)
b)
c)
d)
Cutting federal funding when in fact the level of NOx is greater than
5.0 ppt.
Cutting federal funding when in fact the level of NOx is equal to 5.0
ppt or less.
Providing federal funding when in fact the level of NOx is greater
than 5.0 ppt.
Providing federal funding when in fact the level of NOx is equal to
5.0 ppt or less.
Type II error (answer)
Suppose that a regulatory agency will propose that Congress cut
federal funding to a metropolitan area if its mean level of NOx is
unsafe—that is, if it exceeds 5.0 ppt. The agency gathers sample
NOx concentrations on 60 different days and calculates a test of
significance to assess whether the mean level of NOx is greater
than 5.0 ppt.
Which of the following best describes the implications of a Type II
error?
a)
b)
c)
d)
Cutting federal funding when in fact the level of NOx is greater than
5.0 ppt.
Cutting federal funding when in fact the level of NOx is equal to 5.0
ppt or less.
Providing federal funding when in fact the level of NOx is
greater than 5.0 ppt.
Providing federal funding when in fact the level of NOx is equal to
5.0 ppt or less.
Type I and Type II error
Suppose that a regulatory agency will propose that Congress cut
federal funding to a metropolitan area if its mean level of NOx is
unsafe—that is, if it exceeds 5.0 ppt. The agency gathers sample
NOx concentrations on 60 different days and calculates a test of
significance to assess whether the mean level of NOx is greater
than 5.0 ppt.
Suppose the agency concludes that the mean level of NOx exceeds 5.0
ppt. Which of the following is true?
a)
b)
c)
d)
Neither a Type I error nor a Type II error could have been
committed.
We definitely did not make a Type I error, but a Type II error may
have been committed.
We definitely did not make a Type II error, but a Type I error may
have been committed.
We may have made both a Type I error and a Type II error.
Type I and Type II error (answer)
Suppose that a regulatory agency will propose that Congress cut
federal funding to a metropolitan area if its mean level of NOx is
unsafe—that is, if it exceeds 5.0 ppt. The agency gathers sample
NOx concentrations on 60 different days and calculates a test of
significance to assess whether the mean level of NOx is greater
than 5.0 ppt.
Suppose the agency concludes that the mean level of NOx exceeds 5.0
ppt. Which of the following is true?
a)
b)
c)
d)
Neither a Type I error nor a Type II error could have been
committed.
We definitely did not make a Type I error, but a Type II error may
have been committed.
We definitely did not make a Type II error, but a Type I error
may have been committed.
We may have made both a Type I error and a Type II error.
Significance level
The significance level  is
a)
b)
c)
d)
e)
A Type I error.
A Type II error.
The probability of a Type I error.
The probability of a Type II error.
The power of a test.
Significance level (answer)
The significance level  is
a)
b)
c)
d)
e)
A Type I error.
A Type II error.
The probability of a Type I error.
The probability of a Type II error.
The power of a test.

 is
a)
b)
c)
d)
e)
A Type I error.
A Type II error.
The probability of a Type I error.
The probability of a Type II error.
The power of a test.
 (answer)
 is
a)
b)
c)
d)
e)
A Type I error.
A Type II error.
The probability of a Type I error.
The probability of a Type II error.
The power of a test.
1-
1 -  is
a)
b)
c)
d)
e)
A Type I error.
A Type II error.
The probability of a Type I error.
The probability of a Type II error.
The power of a test.
1 -  (answer)
1 -  is





A Type I error.
A Type II error.
The probability of a Type I error.
The probability of a Type II error.
The power of a test.
Type II error
Suppose we set our significance level to be  = 0.01. To decrease the
probability of committing a Type II error, we can
a)
b)
Increase our sample size.
Decrease our sample size.
Type II error (answer)
Suppose we set our significance level to be  = 0.01. To decrease the
probability of committing a Type II error, we can
a)
b)
Increase our sample size.
Decrease our sample size.
Significance level and power
True or False: As the significance level for a test is decreased, the
power is increased.
a)
b)
True
False
Significance level and power (answer)
True or False: As the significance level for a test is decreased, the
power is increased.
a)
b)
True
False
Significance level
True or False: As the significance level for a test is decreased, the
probability of making a Type I error is increased.
a)
b)
True
False
Significance level (answer)
True or False: As the significance level for a test is decreased, the
probability of making a Type I error is increased.
a)
b)
True
False
Significance level
True or False: As the significance level for a test is decreased, the
probability of making a Type II error is increased.
a)
b)
True
False
Significance level (answer)
True or False: As the significance level for a test is decreased, the
probability of making a Type II error is increased.
a)
b)
True
False
Significance level and power
True or False: In a significance test with  = 0.05, if n is increased,
then the power of the test increases.
a)
b)
True
False
Significance level and power (answer)
True or False: In a significance test with  = 0.05, if n is increased,
then the power of the test increases.
a)
b)
True
False
Significance level and power
True or False: For small n,  is approximately equal to 1 - .
a)
b)
True
False
Significance level and power (answer)
True or False: For small n,  is approximately equal to 1 - .
a)
b)
True
False
Significance level and power
True or False: For large n (i.e., n > 30),  is approximately equal to 1 .
a)
b)
True
False
Significance level and power (answer)
True or False: For large n (i.e., n > 30),  is approximately equal to 1 .
a)
b)
True
False
Significance level
True or False: In a significance test with  = 0.05, if n is increased,
then  increases.
a)
b)
True
False
Significance level (answer)
True or False: In a significance test with  = 0.05, if n is increased,
then  increases.
a)
b)
True
False