Sample Size
Annie Herbert
Medical Statistician
Research & Development Support Unit
Salford Royal Hospitals NHS Foundation Trust
annie.herbert@manchester.ac.uk
0161 206 4567
Timetable
Time
60 mins
Task
Presentation
Discussion
15 mins
Break
60 mins
Demonstration of S.S. calculation
Practical tasks
Outline
When are sample size calculations necessary?
Single proportions
Two proportions
Two means
Other situations
Practical implications
Useful references
Why is it important to consider
sample size?
To have a high chance of detecting a clinically
important treatment effect if it exists.
To ensure appropriate precision of estimates.
To avoid wasting resources and the time of
participants.
To avoid misleading conclusions.
When is a sample size calculation
not necessary?
Truly qualitative research.
Pilot studies that will be used to inform
larger studies (and not make conclusions).
Example 1:
Population studies, single proportion (1)
What is the prevalence of dysfunctional
breathing amongst asthma patients in
general practice?
(Thomas et al, BMJ 2001)
Results: Sample proportion of those
suffering from dysfunctional breathing and
a confidence interval for this proportion.
Population studies, single proportion (2)
Primary outcome variable: Binary
Required information:
1) Estimate of what proportion will be,
(if rate totally unknown pick 50% as most
conservative estimate).
2) Size of population if population small,
e.g., < 20,000.
3) Acceptable deviation from this population estimate,
(half width of confidence interval).
Population studies, single proportion (3)
Statement for Protocol (1)
Include all figures that you’ve inputted into the calculation.
Possibly add statement about response rate/drop out.
Name the person who did the sample size calculation and
any software.
E.g., ‘A sample of 324 patients will be required to obtain a
95% confidence interval of +/- 5% around a prevalence of
approximately 30%. This was calculated by the PI using
StatsDirect. We expect that 60% of those who we approach
will agree to take part and as this is a questionnaire-based
study that there’ll be next to no drop-outs, so we intend to
approach 540 patients.’
How sample size varies with precision:
Expected
Prevalence
Acceptable
Deviation
Sample Size
30%
10
81
30%
5
313
30%
1
4466
Example 2:
Comparing two proportions (1)
Study: RCT comparing the effectiveness of
colony-stimulating factors (CSFs) in reducing
sepsis in premature babies.
Results: Rate of sepsis at 2 weeks in CSF
group and Placebo group, difference
between these two proportions, confidence
interval for this difference.
Comparing two proportions (2)
Primary outcome variable: Binary.
Required information:
1) Estimate of proportion in each group
(difference is clinically important).
2) Power.
3) Significance level.
4) Treatment:Control ratio (often 1:1).
Definitions (1)
‘Effect Size’
– What do you expect to see?
– What has been seen previously?
– What is a clinically important difference?
‘Power’
– Probability of detecting a clinically
important effect, if it exists.
– Typically 80%, 90%.
Definitions (2)
‘Significance Level’
– Cut-off level at which you would say a p-value
is significant/non-significant.
– Probability of concluding that there is a
statistically significant difference in the sample
when there is in fact no true difference in the
population.
– Typically 5%.
– Should be set lower if multiple statistical tests
have been planned.
Comparing two proportions (3)
Statement for Protocol (2)
Comparing two proportions:
E.g., ‘A sample size of at least 149 patients
per group is required to be able to detect
an absolute difference of 16% (50% vs.
34%) in the rate of sepsis between groups
with 80% power, at 5% significance level’.
Example 3:
Comparing two means (1)
A RCT to evaluate a brief psychological intervention in
comparison to usual treatment in the reduction of
suicidal ideation.
(Guthrie et al, BMJ 2001)
‘Suicidal ideation will be measured on the Beck scale; the
standard deviation of this scale in a previous study was
7.7, and a difference of 5 points is considered to be of
clinical importance.’
Results: Mean reduction in Beck score in the Intervention
group and Usual Treatment group, difference between
these two means, confidence interval for this difference.
Comparing two means (2)
Primary outcome variable: Numerical
Required information:
1) Estimate of standard deviation of primary
outcome variable.
2) Effect size (difference in means).
3) Power.
4) Significance level.
5) Treatment:Control ratio.
Where to find an estimate of the
standard deviation:
Pilot study.
– Though note standard deviations on very small numbers
may be imprecise.
Previous studies.
In-house data.
Rough estimate:
Quarter of the range of ‘usual’ values.
Comparing two means (3)
Statement for Protocol (3)
Comparing two means:
E.g., ‘A sample size of at least 39 patients
per group is required to be able to detect a
difference in mean Beck score of 5 points
or more with 80% power at 5%
significance level. This is assuming a
standard deviation of 7.7 for the Beck
scale.
Things to Note
Power is linked to effect size.
– All trials have an infinite number of powers!
Post-hoc power calculations are pointless.
– Power conveyed by confidence interval.
If secondary outcomes are important separate
sample size calculations should be done for
these too.
– The largest size resulting from these calculations
should be used so powerful enough for all analyses.
Other Situations
More than 2 groups.
Non-randomised studies, e.g. Case-Control.
Equivalence trials.
Paired data, e.g., crossover, before/after trial.
Time-to-event data.
Cluster-randomised studies.
Diagnosis studies.
Sample size calculations in practice:
Often look at a range of assumptions.
– Best case/worst case scenario.
Balance between ideal statistical power,
resources and time.
– Bear in mind only ¼ may consent (or lower).
Interim/sub-group analyses.
– Seek advice and adjust p-values, etc. accordingly.
Issues of power should not overshadow issues
of quality.
Useful References
Sample size calculations in randomised trials:
mandatory and mystical.
Schulz & Grimes, The Lancet 2005; 365
An Introduction to Medical Statistics.
Bland, M, OUP 2000
Sample size calculations for clinical studies.
Machin, Campbel, Fayers & Pinol, Blackwells