STAT 405 - BIOSTATISTICS
Handout 4 – Power for Tests of a Single Binomial Proportion
Up to this point, we have reviewed the use of the binomial distribution for testing
hypotheses concerning a proportion and for constructing confidence intervals. In this
handout, we will consider the issues of power and sample size. Your text discusses
power calculations based on the normal distribution in Section 7.10 of your text (we will
also use the binomial distribution in this handout).
EXAMPLE: Recall the Cancer Study from the previous handout. We noted that 13
deaths occurred among 55-64 year-old male workers in a nuclear-power plant and that
in 5 of them the cause of death was cancer. Based on vital-statistics reports,
approximately 20% of all deaths were attributed to some form of cancer. Our hypothesis
test did NOT provide evidence that the proportion of deaths from cancer in nuclearpower plant workers was greater than the proportion of deaths from cancer in men of
comparable age in the general population (p-value = .0991).
Questions:
1. Is it possible that the cancer mortality rate is in fact greater for the study
population than the general population, and our test simply failed to detect this?
Explain.
2. Do you think the sample size of 13 is large enough to uncover a difference if one
does in fact exist? Why or why not?
This is a question of power, which is the focus of this handout. First, let’s review some
basics of hypothesis testing.
When testing for a difference,
there are two possible outcomes:
Ideally, these outcomes would
occur:
1. We find evidence that a
difference exists.
1. When a difference does in fact
exist.
2. We fail to find evidence that a
difference exists.
2. When there is not a difference of
practical significance.
29
However, we know that hypothesis tests are not error proof! The following table
summarizes all possible scenarios in carrying out a hypothesis test (the probability of
each is listed in parentheses):
Null is True Alternative is True
Reject the Null
Fail to Reject the Null
Note that two types of errors exist:

Type I Error: This occurs when we falsely reject the null hypothesis. That is, we
reject the null when the null is true. The probability of committing this error is
given by α.
Note that we control the Type I error rate by our selection of α prior to
conducting an experiment.

Type II Error: This occurs when we fail to reject the null when a particular
alternative scenario is true. The probability of committing this error is given by β.
Our control of the Type II error rate depends on the sample size, and to some
degree on the test used (most powerful test – STAT 460).
The table shown above also describes scenarios in which we do not commit errors. For
example, note that one possible scenario is to reject the null when the alternative is true.
This is a good outcome, and it leads into the concept of power.
Power: By definition, the power of a test is the probability of correctly rejecting the null
hypothesis if a particular alternative scenario is true. In other words, the
power is the probability of rejecting the null hypothesis given that a specific
difference does exist. This is calculated as 1- β.
Back to the Cancer Example:
Recall that we used a sample size of 13 in the study. How likely is the hypothesis test to
detect that the cancer mortality rate is greater in the study population than in the general
population, if this is in fact really the case? To answer this question, we must calculate
the power of this test. First, consider our hypotheses:
H0: p = .20
Ha: p > .20
30
To find the power, we need to find the probability of rejecting the null given that the
alternative is true. However, note that our alternative hypothesis is very general—there
are infinitely many scenarios in which the true proportion could be greater than 20%! It
turns out that in order to calculate the power, we need to define a PARTICULAR
alternative scenario. That is, we must specify the size of the difference that we want the
test to detect.
Calculating the Power for the Binomial Test
The following list summarizes what is needed for a power calculation for a binomial test:
1. A specific alternative (i.e., the size of the difference we want to detect).
2. The sample size (i.e., the number of replications for each treatment).
3. The Type I error rate (α).
The power calculations for the Cancer example are shown below.
Power = P(reject the null hypothesis | a particular alternative scenario is true)
Step 1:
Pick a specific alternative. We will find the chance of finding a significant
result (using a sample size of 13) if the true rate of cancer death for the 55-64
year old male nuclear-power plant workers is actually twice as high as that of
the typical 55-64 year old male (40%).
Step 2:
Find the values of the binomial random variable for which the null hypothesis
will be rejected.
31
You can use SAS to find this “critical value”:
data Rejection_Region;
do x = 0 to 13;
cutoff = 1-cdf('binomial', x-1, .2, 13); output;
end;
proc print noobs; run;
We can also use the Binomial Table Generator I created in JMP.
Step 3:
Note that we will reject the null hypothesis if the number of cancer deaths is 6
or more. Now, find the probability of observing 6 OR MORE cancer deaths
given that the true proportion of cancer deaths in the 55-64 male nuclearpower plant workers is actually 40%. Why?
Power = P(reject Ho | a particular alternative scenario is true)
The following graphic shows a binomial distribution with n=13 and p = .40.
32
We can use SAS to calculate the probability of 6 or more cancer deaths:
data CalcPower;
power=1-cdf('binomial', 5, .4, 13);
run;
proc print; run;
Interpretation: There is a 42.6% chance of finding a significant result, based
on a sample size of 13, if the true rate of cancer deaths among 55-64 year old
male nuclear-power plant workers is actually twice as high as that of typical
55-64 year old males.
Making the Power Calculations Simpler!
SAS PROC POWER can be used for the calculations. For the power of tests for a
binomial proportion, you can use the following code:
proc power;
onesamplefreq test=exact
sides = 1
nullproportion = 0.20
proportion = 0.40
ntotal = 13
power = .;
run;
In JMP select DOE > Sample Size and
Power > One Sample Proportion
In R use the following command:
> 1 - pbinom(5,size=13,prob=.4)
[1] 0.4256036
33
Compare this result to that based on normal-theory approximations using SAS:
proc power;
onesamplefreq test=z method=normal
sides = 1
nullproportion = 0.20
proportion = 0.40
ntotal = 13
power = .;
run;
Calculating the Power for the Binomial Test Using Normal-Theory Methods
Again, we will consider the power calculations for the Cancer Study example. The power
based on normal-theory methods was already calculated on the previous page using
PROC POWER; here, we will take a closer look into the calculations.
Step 1:
Pick a specific alternative. Once again, we will find the chance of finding a
significant result (using a sample size of 13) if the true rate of cancer death for
the 55-64 year old male nuclear-power plant workers is actually twice as high
as that of the typical 55-64 year old male (40%).
Step 2:
Using the normal approximation to the binomial distribution, find the values
for which the null hypothesis will be rejected.
Recall the following for a binomial random variable, X:

Mean =

Standard Deviation =
34
We can derive the following for the sample proportion, X/n:

Mean =

Standard Deviation =

Z-value =
p
Step 3:
Power = P(reject Ho | a particular alternative scenario is true)
Next, we find the probability of our observed Z-value falling in the rejection
region given that the true proportion of cancer deaths in the 55-64 male
nuclear-power plant workers is actually 40%.
That is, we must find P(𝑝̂ > .3825) using a normal distribution with

Mean =

Standard Deviation =
35
This probability is illustrated in the following graphic:
To find this normal approximation probability in R, you can use the following
command:
> 1 - pnorm(.3825,mean=.4,sd=.1358)
[1] 0.5512682
Note that this matches the power calculated using PROC POWER.
Finding the necessary sample size
In many instances, researchers desire to find the necessary sample size to achieve a
certain power BEFORE the study is carried out. PROC POWER makes this process very
easy if you are willing to use normal-theory methods. For example, once again consider
the Cancer Study. Suppose we want to achieve 80% power:
proc power;
onesamplefreq test=z method=normal
sides = 1
nullproportion = 0.20
proportion = 0.40
ntotal = .
power = .80;
run;
36
Next, calculate the power assuming this sample size using exact binomial probabilities in
SAS and in JMP:
proc power;
onesamplefreq test=exact
sides = 1
nullproportion = 0.20
proportion = 0.40
ntotal = 29
power = .;
run;
37
Power for a Two-Sided Test
Suppose we wanted to show that the proportion of deaths from cancer in nuclear-power
plant workers was DIFFERENT FROM the proportion of deaths from cancer in men of
comparable age in the general population. The power calculations for this two-sided test
are shown below.
Power = P(reject the null hypothesis | a particular alternative scenario is true)
Step 1:
Pick a specific alternative. We will find the chance of finding a significant
difference (using a sample size of 13) if the true rate of cancer death for the 5564 year old male nuclear-power plant workers is actually twice as high as that
of the typical 55-64 year old male (40%).
Step 2:
Find the values of the binomial random variable for which the null hypothesis
will be rejected.
Find the “critical value”:
data Rejection_Region;
do x = 0 to 13;
cutoff = 1-cdf('binomial', x-1, .2, 13); output;
end;
proc print noobs; run;
We can also use the Binomial Table Generator I created in JMP.
38
Step 3:
Note that we will reject the null hypothesis if the number of cancer deaths
exceeds 7. Now, find the probability of observing 7 OR MORE cancer deaths
given that the true proportion of cancer deaths in the 55-64 male nuclearpower plant workers is actually 40%.
The following graphic shows a binomial distribution with n=13 and p = .40.
We can use SAS to calculate the probability of 7 or more cancer deaths:
data CalcPower;
power=1-cdf('binomial', 6, .4, 13);
run;
proc print noobs; run;
In R:
> 1 - pbinom(6,size=13,prob=.4)
[1] 0.228844
In SAS PROC POWER:
proc power;
onesamplefreq test=exact
nullproportion = 0.20
proportion = 0.40
sides = 2
ntotal = 13
power = .;
run;
In JMP select DOE > Sample Size and
Power > One Sample Proportion
39
The details for determining power using normal theory methods:
40
41