Hypothesis Testing
and
Sample Size Calculation
Po Chyou, Ph. D.
Director, BBC
Hypothesis Testing
on
• Coefficients based on
regression model
Population proportion(s) • Odds ratio
• Relative risk
• Population variance(s)
• Population correlation(s) • Trend analysis
Association based on • Survival distribution(s) /
curve(s)
contingency table(s)
• Goodness of fit
• Population mean(s)
• Population median(s)
Hypothesis Testing
1. Definition of a Hypothesis
An assumption made for the sake of argument
2. Establishing Hypothesis
Null hypothesis - H0
Alternative hypothesis - Ha
3. Testing Hypotheses
Is H0 true or not?
Hypothesis Testing
4.Type I and Type II Errors
Type I error: we reject H0 but H0 is true
α = Pr(reject H0 / H0 is true) = Pr(Type I error)
= Level of significance in hypothesis testing
Type II error: we accept H0 but H0 is false
 = Pr(accept H0 / H0 is false) = Pr(Type II error)
Hypothesis Testing
5. Steps of Hypothesis Testing
- Step 1
- Step 2
- Step 3
- Step 4
- Step 5
Formulate the null hypothesis H0 in
statistical terms
Formulate the alternative hypothesis Ha in
statistical terms
Set the level of significance α and the
sample size n
Select the appropriate statistic and the
rejection region R
Collect the data and calculate the statistic
Hypothesis Testing
5. Steps of Hypothesis Testing (continued)
- Step 6
If the calculated statistic falls in the
rejection region R, reject H0 in favor of Ha;
if the calculated statistic falls outside R, do
not reject H0
Hypothesis Testing
6. An Example
A random sample of 400 persons included 240 smokers and 160 nonsmokers. Of the smokers, 192 had CHD, while only 32 non-smokers
had CHD.
Could a health insurance company claim the proportion of smokers
having CHD differs from the proportion of non-smokers having
CHD?
CHD
S m okers
N on-S m okers
x1
x2
CHD
S m okers
N on-S m okers
1 92
32
No CHD
n1 - x1
n2 - x2
n1
n2
n = n1 + n2
No CHD
48
1 28
2 40
1 60
4 00
Hypothesis Testing
Example (continued)
Let P1 = the true proportion of smokers having CHD
P2= the true proportion of non-smokers having CHD
- Step 1
H0 : P1 = P2
- Step 2
Ha : P1  P2
- Step 3
α = .05, n = 400
Hypothesis Testing
Example (continued)
- Step 4
statistic =  =
P1 - P2
P(1-P) (1/n1 + 1/n2)
where P1 = x1 , P2 = x2 and P = x1 + x2
n1
n2
n 1 + n2
Hypothesis Testing
Example (continued)
- Step 5
CHD
S m okers
No CHD
x1
x2
N on-S m okers
CHD
S m okers
N on-S m okers
1 92
32
n1 - x1
n2 - x2
n1
n2
n = n1 + n2
P1 = x1 = 192 = .80
No CHD
n1 240
4 8 2 40
P2 = x2 = 32 = .20
1 28 1 60
n2 160
4 00
P = x1 + x2 = 192 + 32 = 224 = 0.56
n1 + n2 240 + 160 400
=
P1 - P2
P(1-P) (1/n1 + 1/n2)
=
.80 - .20
= .60 = 11.84 > 1.96
(.56) (1-.56) (1/240 + 1/160) .05066
Hypothesis Testing
Example (continued)
- Step 6
Reject H0 and conclude
that smokers had
significantly higher
proportion of CHD than
that of non-smokers.
[P-value < .0000001]
Hypothesis Testing
7. Contingency Table Analysis
Density
The Chi-square distribution (2)
α=.05
2
2
0
.05, 1
Do not reject H0
=3.841
Reject H0
Hypothesis Testing
Equation for chi-square for a contingency table
2

=  (Oij - Eij
i, j
Eij
2
)
For i = 1, 2 and j =1, 2
2= (O11 - E11)2 + (O12 - E12)2 + (O21 - E21)2 + (O22 - E22)2
E11
E12
E21
E22
Hypothesis Testing
Equation for chi-square for a contingency table (cont.)
E11 = n1m1
n
E21 = n2m1
n
E12 = n1 - n1m1 = n1m2
n
n
E22 = n2 - n2m1 = n2m2
n
n
E 11
E 21
m1
E 12
E 22
m2
O 11
O 21
m1
O 12
O 22
m2
n1
n2
n1
n2
n = n1 + n2 = m 1 + m 2
Hypothesis Testing
Example : Same as before
- Step 1
H0 : there is no association between smoker
status and CHD
- Step 2 Ha : there is an association between smoker
status and CHD
- Step 3  = .05, n = 400
- Step 4 statistic =
2= (O11 - E11)2 + (O12 - E12)2 + (O21 - E21)2 + (O22 - E22)2
E11
E12
E21
E22
Hypothesis Testing
Density
Example (continued) : Same as before
α=.05
2
2
0
.05, 1
Do not reject H0
=3.841
Reject H0
Hypothesis Testing
Example (continued) : Same as before
- Step 5
CHD
S m ok ers
N on -S m ok ers
O 11
O 21
m1
CHD
S m ok ers
N on -S m ok ers
1 92
32
2 24
CHD
No CHD
O 12
O 22
m2
n1
n2
n
No CHD
48
1 28
1 76
No CHD
S m ok ers
E 11
E 12
N on -S m ok ers
E 21
E 22
2 40
1 60
4 00
Hypothesis Testing
Example (continued) : Same as before
- Step 5 (continued)
E11 = n1m1 = 240 * 224 = 134.4
n
400
E12 = n1 - n1m1 = 240 - 134.4 = 105.6
n
E21 = n2m1 = 160 * 224 = 89.6
n
400
E22 = n2 - n2m1 = 160 - 89.6 = 70.4
n
S m o k ers
N o n -S m o k ers
Expectation
Counts
CHD
1 34 .4
8 9 .6
No CHD
1 05 .6
7 0 .4
Hypothesis Testing
Example (continued) : Same as before
- Step 5 (continued)
2= (O11 - E11)2 + (O12 - E12)2 + (O21 - E21)2 + (O22 - E22)2
E11
E12
E21
E22
= (192 - 134.4)2 + (48 - 105.6)2 + (32 - 89.6)2 + (128 - 70.4)2
134.4
105.6
89.6
70.4
=
24.68
+
31.42
+ 37.03 +
47.13
= 140.26 > 3.841
Hypothesis Testing
Example (continued) : Same as before
- Step 6
Reject H0 and conclude
that there is an
association between
smoker status and CHD.
[P-value < .0000001]
Sample Size Estimation and
Statistical Power Calculation
Definition of Power
Recall :
 = Pr (accept H0 / H0 is false) = Pr (Type II error)
Power = 1 -  = Pr(reject H0 / H0 is false)
Sample Size Estimation
for Intervention on Tick Bites Among
Campers
Assumptions
1. Given that the proportion (PCON) of tick bites among
campers in the control group is constant.
2. Given that the proportion (PINT) of tick bites among
campers in the intervention group is reduced by 50%
compared to that of the control group after
intervention has been implemented.
3. Given that a one- or two- tailed test is of interest with
80% power and a type-I error of 5%.
Sample Size Estimation
for Intervention on Tick Bites Among
Campers
Summary Table 1
R e q u ire d N fo r ea c h g ro u p
T w o -ta ile d
O n e -ta ile d
4500
3600
P CON
.0 1
P IN T
.0 0 5
(5 0 % red u ctio n )
.0 5
.0 2 5
(5 0 % red u ctio n )
1170
922
.1 0
.0 5 0
(5 0 % red u ctio n )
475
374
.1 5
.0 7 5
(5 0 % red u ctio n )
305
240
Statistical Power Calculation
for Intervention on Obesity of Women in
MESA
Assumptions
1. Given that the proportion (PCON) of women who are obese at
baseline (i.e., the control group) is constant. There are a total of
840 women in the control group. Based on our preliminary data
analysis results, approximately 50% of these 840 women at
baseline are obese (BMI >= 27.3).
2. Given that the proportion (PINT) of women who are obese in the
intervention group is reduced by 5% or more compared to that of
the control group after intervention has been implemented. There
are a total of 680 women who had been newly recruited. Based on
our preliminary data analysis results, 50% of these 680 newly
recruited women are obese. Assume that 60% of these women will
agree to participate, we will have 200 women to be targeted for
intervention.
Statistical Power Calculation
for Intervention on Obesity of Women in
MESA (continued)
Assumptions
3. Given that a one-tailed test is of interest with a type-I error of 5%, then
the estimated statistical powers are shown in Table 1 for detecting a
difference of 5% or more in the proportion of obesity between the
control group and the intervention group.
Table 1
P C O N (n = 8 40 )
.5 0
.5 0
.5 0
.5 0
.5 0
.5 0
P IN T (n = 6 80 )
.4 5
.4 4
.4 3
.4 2
.4 1
.4 0
D ifference
.0 5
.0 6
.0 7
.0 8
.0 9
.1 0
P ow er
61%
75%
85%
92%
96%
98%
Reference
“Statistical Power Analysis
for the Behavioral Sciences”
Jacob Cohen
Academic Press, 1977
Take Home Message:
• You’ve got questions : Data ? STATISTICS?...
• Contact Biostatistics and consult with an
experienced biostatistician
– Po Chyou, Director, Senior Biostatistician (ext. 9-4776)
– Dixie Schroeder, Secretary (ext. 1-7266)
OR
• Do it at your own risk
Free Handout