Quiz on Error Rates, Power, and Significance versus Importance
I was considering doing a bungee jump. I asked the operator if it is safe. He said the
following.
There is no evidence that it is unsafe. I tested it 3 times with a 500 pound weight, and it
worked perfectly each time.
1.
2.
3.
The operator’s null hypothesis is
A.
It is safe
B.
It is unsafe
The operator’s decision is to
A.
reject the null hypothesis
B.
not reject the null hypothesis
In the bungee-jump example, a Type 1
error would be
A.
Deciding it is unsafe, when, in
fact, it is unsafe.
B.
Deciding it is unsafe, when, in
fact, it is safe.
4.
C.
Deciding it is safe, when, in fact, it is safe.
D.
Deciding it is safe, when, in fact, it is unsafe.
I decide NOT to jump, because I am worried about
A.
a Type 1 error
B.
a Type 2 error
C.
a Type 3 error
D.
a Standard error
E.
A result that is statistically significant but not practically important
5.
In the bungee-jump example, the reason that I am worried about an error is
because
A.
the weight is too small (i.e., the effect is too small)
B.
the weight is too large (i.e., the effect is too large)
C.
the samples size is too small
D.
the sample size is too large
E.
None of the above
children, there is highly
Toothpaste
Cost per 4 oz
tube
Sample Size
Mean
Cavities per
Year per
Child
significant statistical
Holtzman’s
$19.99
6,000
0.48
evidence that the mean
Competitor
$1.99
6,000
0.61
Based on a study of
12,000 public school
annual number of
cavities is less for those using Holtzman’s new improved toothpaste (P < 0.0001).
6.
A caring parent who is a wise consumer would see Holtzman’s toothpaste as not
worth the cost because the study seems to have a
7.
A.
Type 1 error
B.
Type 2 error
C.
result that is statistically significant but not practically important
If a result is statistically significant then
A.
the data confirm that the result is biologically important
B.
the effect of the treatment must have been very large
C.
we are sure that the true value of the population parameter is different
from the hypothetical (null) value
D.
All of the above
E.
None of the above
8.
If a result is not statistically significant then
A.
the data confirm that the result is not biologically important
B.
the effect of the treatment must have been negligible biologically
C.
we are not sure that the true value of the population parameter is actually
different from the hypothetical (null) value. The difference observed
between the estimate of the parameter and the hypothetical value of the
parameter might just be due to natural, random variation
9.
D.
All of the above
E.
None of the above
If the sample size is very large, and the P-value is very small, e.g., P < 0.001, and
the underlying standard deviation is small, then which of the following is not
true.
A.
We have the conditions under which statistical significance does imply
practical, e.g., biological, e.g., clinical, importance.
B.
We have the conditions under which statistical significance may very well
not imply practical, e.g., biological, e.g., clinical, importance.
C.
a very small, biologically unimportant effect could very easily be
statistically significant.
10.
In hypothesis testing, if the decision is to not reject the null hypothesis, then
which of the following consequences is not possible?
A.
A Type 1 error
B.
A Type 2 error
C.
A correct decision
D.
A result that is statistically significant but not practically important
E.
Both choice A and choice D are not possible.