Power: Example Questions and Outline Model Answers
Question 1
Two groups of 20 schoolchildren (20 subjects per group) were tested in a study of
anagram solving. The mean time to solve the anagrams for group 1 (who had not
received special literacy training) was 10.5 seconds (standard deviation= 6.7 seconds).
The corresponding figures for group 2 (who had received the literacy training) were
13.9 and 6.9 seconds.
(a)
Explain the elements which appear in the approximate formula for calculating
the statistical power of a two-tailed test, given below. (20% of question marks)
(b)
Use the formula to calculate the observed power of this study, for tests at the 5%
significance level. Remember to adjust the formula because 1-tailed tests would
be carried out in this study. Based on the information given below you will be
able to express your answer as a pair of values within which the observed power
lies. Show your working. (35% of question marks)
(c)
If there were a real group difference as large as the one observed in the data,
then what is your estimate of how often the null hypothesis would fail to be
rejected? What is the name for this quantity? (10% of question marks)
(d)
By using the approximate formula below, estimate the sample size that would be
required in each group in order that the above experiment would have 80%
power to detect a group difference equivalent to that measured in the
experiment. Base your calculations on one-tailed tests at the 5% significance
level. The formula is for a two-tailed test and needs adjusting appropriately.
Show your working. (35% of question marks)
Approximate Power Formula
power = z-1{d*sqrt(n/2) – z(1-/2)}
where sqrt(x) is the square-root of x
Approximate Sample Size Formula
required sample size per group = 2*{z(power) +z(1-/2)}2/d2
Data That May Be Needed in The Calculations

.050
.100
.200
.300
.400
.500
.600
.700
.800
.900
z(1-/2)
1.960
1.645
1.282
1.036
.842
.674
.524
.385
.253
.126
x
-1.95
-1.65
-1.30
-1.00
-0.85
-0.65
-0.50
-0.25
-0.05
0.00
z-1{x}
0.026
0.049
0.097
0.159
0.198
0.258
0.309
0.401
0.480
0.500
Note: the z-1{x} function is symmetrical about x=0; so z-1{1.95}=1-z-1{-1.95} etc.
Question 1: MODEL ANSWER
Note that the perfect answer can be pretty short:(a)
Elements to explain: z and z-1; d; n; α. z( ) is the inverse cumulative distribution
function of the normal distribution -- the function takes a probability and
converts it into a z-score. Thus, a probability of 0.975 is converted to a z-score
of +1.96 because the z-score below which 97.5% of the normal distribution lies
is +1.96. The function z-1( ) is the inverse of z and is the cumulative distribution
function of the normal distribution -- the function takes a z-score and converts it
to a probability. Thus, a z-score of +1.96 is converted to a probability of 0.975,
i.e. 97.5% of the normal distribution lies below this z-score. (An explanation of
z and z-1 without the exact jargon, but which is accurate, will also get full marks
here.) d is a measure of the between-groups effect size: it is the mean difference
between the groups divided by their pooled standard deviation. n is the number
of subjects in each of the 2 groups (and the group size must be equal for this
formula). α is the Type I error rate, conventionally 5%.
(b)
In the example n=20 and α=0.05. To calculate observed power we need the
observed effect size. To calculate the observed value of d: the mean difference
between the groups is 3.4 secs and the groups have s.d. of 6.7 and 6.9. When we
pool the s.d. of equal-sized groups we can take the average s.d. i.e. 6.8. Thus
d=3.4/6.8=0.5. The adjustment to the formula that is required for a one-tailed
test is to replace z(1-α/2) with z(1-α). To see this, consider doing a z-test on a
mean which is supposed to be drawn from a standard normal distribution
(mean=0; s.d.=1). To test (two-tailed, with α=0.05) if the observed value of the
mean is not significantly different from zero then the critical z-values are
z(0.975)=+1.96 and z(0.025)=-1.96. In other words, the value must lie between
z(α/2) and z(1-α/2). To test (one-tailed, with α=0.05) if the observed value is not
significantly above zero the critical z-value is z(0.95), i.e. the value must be
below z(1-α).
the term in the {} brackets must be worked out first:{0.5*sqrt(10) - z(0.95)}
from the table given if α=0.1 then z(1- α/2)=z(0.95)=1.645
so the {} term works out to be -0.06
thus, from the table, z-1{-0.06} lies between 0.480 and 0.401, but much closer
to 0.480. Thus, the observed power for this experiment is between 40 and
48%.
(c)
Between 52% and 60% of the time (but closer to 52%). This is known as the
Type II error rate.
(d)
To calculate the required sample size for 80% power by a 1-tailed t-test we must
use an adjusted formula once again. Again, replace z(1-α/2) with z(1-α). [Check:
a 1-tailed test should increase power and so reduced required number of
subjects. z(1-α/2) would be replaced with a smaller value z(1-α) in the formula
for required sample size and so this is correct].
the {} term is [z(0.8) + z(0.95)]2
to look up z(0.8) we need to look up z(1-0.4/2) thus we need to use the table for
α=0.4. Thus, the {} term =[0.842 + 1.645]2 = 6.185
and so the required sample size in each group = 2*6.185/0.52= 49.5 = 50.