Sample size
Hein Stigum
Presentation, data and programs at:
http://folk.uio.no/heins/
courses
May-16
H.S.
1
Sample size problem
• Needle in a haystack
• Effect size
– Large: easy
– Small: need many
– Large: easy
– Small: need many
5 ingredients:
1. Effect size
May-16
H.S.
2
Type I and type II errors
Does the medicine work?
True value
Our
test
Yes
No
Yes
OK
Type I error
a=5%
No
Type II error
b=20%
OK
Two next
ingredients
May-16
a = sigificance level
1-b = power
H.S.
=5%
=80%
3
Two expressions of random error
Last two ingredients:
Population
Random sample
Precision:
Variance
N
Estimate
True value
Estimate with confidence interval
(
Population
|
)
Confidence
interval
Random Sample
Estimate 1
Estimate 2
True value group 1
True value group 2
|
|
May-16
H.S.
group 1
group 2
p-value
4
Planned and simple analyses
Planned analysis
• Continuous outcome
Simple analysis
• Continuous outcome
– Intervention
– Linear regression
• Categorical outcome
– One-sample T-test
– Two-sample T-test
• Categorical outcome
– Logistic regression
– 2 by 2 table, chi-square test
• OR with confidence interval
Sample size calculation
May-16
H.S.
5
Calculators and Equations
• Calculators
–
–
–
–
Russ Lenth: power (uses Java)
PS: Power and Sample size
Rothman: episheet
Stata: Power and Sample size
– Hein Stigum homepage: Utilities > Excel > Sample size
• Equations
– Laake et al. s 275
– Veierød, Lydersen, Laake 2012
May-16
H.S.
6
Russ Lenth Power and Sample size
• Continuous
– 1 group
– 2 groups
Pull sliders
or click to write
Lenth, R. V. (2006). Java Applets for Power and Sample Size [Computer software]. Retrieved month
day, year, from http://www.stat.uiowa.edu/~rlenth/Power.
May-16
H.S.
7
Continuous outcome: Hemoglobin level
Test: T-tests (p-values)
a) Intervention: one sample
b) Compare men and women: two sample
May-16
H.S.
8
Iron supplement intervention
• One sample T-test (paired T-test)
• Before treatment
–
Women: mean hemoglobin=11
– sd=2 (sigma)
• Desired treatment effect
– Increase by 1 unit (mu-mu0)
• Options
– Power: 80%, alpha: 5%
– Sample size? (n)
Conclusion: Need 34 women
May-16
H.S.
9
Compare men and women
– Two sample T-test
• Compare 2 groups
– sd=2 (sigma, Equal)
– Allocation: Equal
– True diff of means=1
• Options
– Power: 80%, alpha: 5%
– Sample size? (n)
Conclusion: Need 64 women and 64 men
May-16
H.S.
10
Compare men and women:
Find small effect
– Two sample T-test
• Compare 2 groups
– sd=2 (sigma, Equal)
– Allocation: Equal
– True diff of
means=0.5
• Options
– Power: 80%, alpha: 5%
– Sample size? (n)
Conclusion: Need 255 women and 255 men
May-16
H.S.
11
T-test Equations
2
 sd 
nk  c

n  sample size
k  1 or 2 sample T - test
sd  standard deviation (sigma)
Δ  difference in means (mu - mu0)
c  significance and power combination
2
 2 
2
 7.9
 0.5 
 253
(in each group)
Significance and power combinations, c in equations
1-b
0.5
0.8
0.9
0.95
0.10
2.7
6.2
8.6
10.8
a 0.05
3.8
7.9
10.5
13.0
0.01
6.6
11.7
14.9
17.8
May-16
H.S.
Laake et al. s 275
12
Power and Sample size in Stata
Two sample T-test
power twomeans 11 12, sd(1)
power twomeans 11 12, sd1(1) sd2(2)
power twomeans 11 12, sd(1) nratio(0.5)
means=11 and 12, sd=1
sd=1 and 2
ratio N1/N2=0.5
Two proportions (chi-square test)
power twoprop 0.1 0.5
power twoprop 0.1, rrisk(5)
power twoprop 0.1, orato(9)
p=0.1 and 0.5
p=0.1, RR=5
p=0.1, OR=9
Estimated total sample size for a two-sample means test
Total sample size (N)
t test assuming 1 =
H0: 2 = 1 versus Ha:
power twomeans 11 12,
sd(0.5(0.1)2) graph
2
1
120
100
80
60
40
20
.5
Parameters:
May-16
=
2
H.S
1
1.5
Common standard deviation ( )
= .05, 1- = .8,
= 1,
1
= 11,
2
2
= 12
13
Odds ratios
2 by 2 tables
Using confidence intervals
http://folk.uio.no/heins/Utilities/Utilities.html
May-16
H.S.
14
Sample size using confidence interval
• Proportion, incidence rate
– No overlap with given number
(
5%
• RR, OR
|
)
– Significant effect = No overlap with 1
(
1.0
May-16
H.S.
|
)
15
The 2 by 2 table
Exposure
+
-
Disease
+
100 a b 100
10 c d 100
OR=
1 1 1 1
se(ln( OR )) 
  
a b c d
.01+.01+.1 +.01 =.13
10.0
The lowest number sets the precision
To increase power:
Cohort:
balance exposure
Case-Control: balance disease
May-16
(north-south)
(east-west)
H.S.
16
Examples
• Cohort
– Hair coloring and congenital malformations
• Case-control
– Hair coloring and congenital malformations
May-16
H.S.
17
Hair coloring and
congenital malformations
• Cohort
–
–
–
–
–
–
May-16
Pregnant women
Follow up time: pregnancy
Exposure:
color hair:
10%
Disease:
malformations:
0.5%
Expected RR:
2.0
N? (large enough so CI around RR does not include 1)
H.S.
18
Hair coloring and
congenital malformations
Cohort:
Odds ratio and relative risk
Values
25 000
0.5 %
10.0 %
2.00
95 %
80 %
Number of subjects, N=
Proportion with disease=
Proportion exposed=
Expected OR=
Confidence level:
Power:
Names
malformation
color hair
malformation
color hair
+
-
Proportion exposed=
Decimals=
2
2
May-16
+
-
23
102
2 477
22 398
125
18.1 %
24875
10.0 %
Proportion with disease
2 500
0.9 %
22 500
0.5 %
25000
0.5 %
10.0 %
OR=2, 95% CI=(1.04 , 3.84)
RR=1.99, 95% CI=(1.04 , 3.81)
H.S.
19
Hair coloring and
congenital malformations
Case Control:
May-16
H.S.
20
Special cases
• Sample size is based on “guestimates”, use
simple analyses
• Exceptions
– Continuous exposure:
– Interaction:
– Mixed models:
linear regression
QUANTO
ignore dependencies, or
𝑁𝑒𝑓𝑓 = 𝑁/(1 + (𝑛𝑐𝑙𝑢𝑠 − 1)𝜌)
– RCTs:
May-16
same as cohort study
H.S.
21
Summing up
• Sample size
–
–
–
–
needed size to “overcome” random error
“needle in the haystack” = significant effect
small effects  large N
Balance matters
– Based on “guestimates”, use simple analyses
• Exception: interaction QUANTO
• Mixed models with large intraclass correlation and many
clusters
May-16
H.S.
22