Statistics for Health Research
A Taxonomy of
Research Design
Peter T. Donnan
Professor of Epidemiology and Biostatistics
Objectives of session
• Realise importance of design
• Understand difference between
•
•
•
experimental and observational design
Choose appropriate design
Understand main forms of experimental
design
Understand main forms of observational
design
Why is design important?
• Poor design may not answer question
• Choice of design determines the
type of analysis
• Incorrect design leads to waste of
resources
• Poor design is unethical if patient
exposed to danger for little return
A Taxonomy of Research Design
• Experimental
Randomised Controlled Trials
Factorial design
Crossover
Cluster randomisation
n.b. All are prospective since following
experimental units forward in time
A Taxonomy of Research Design
• Non-randomised - Observational
1) Cohort (Prospective and
Retrospective)
2) Case-Control
3) Cross-sectional
4) Time series
RANDOMISED CONTROLLED
TRIAL (RCT)
•Gold standard method to assess
Efficacy of treatment or
demonstrate causal relationship
•First proper clinical trials from
1940s in tuberculosis
•Ethical requirements set out in
Declaration of Helsinki, 1960
RANDOMISED CONTROLLED
TRIAL (RCT)
Random allocation to
intervention or control so
likely balance of all factors
affecting outcome
Hence any difference in outcome
‘caused’ by the intervention
Randomised Controlled Trial
Eligible subjects
RANDOMISED
Intervention
Control
RANDOMISED CONTROLLED
TRIAL (RCT)
•RCT necessary to
demonstrate efficacy
•Cost-effectiveness (ICER)
•NICE and SMC assessment
for recommendation to NHS
Examples of RCT
•Trials of tamoxifen for treatment of
early breast cancer1
•Reduction in recurrence of 29%
•Reduction in mortality by 14% in women
with oestrogen receptor positive cancers
•Trial published recently in Lancet on
herceptin (trastuzumab)
1. Lancet 1998; 351: 1451-67.
RANDOMISED FACTORIAL
DESIGN
Patients are randomised more
than once
All possible combinations of
interventions are studied
Simple 2-way factorial has 4
combinations
Randomised Factorial Trial
Eligible subjects
RANDOMISED
Intervention 1
RANDOMISED
Intervention 2
Control 2
Control 1
RANDOMISED
Intervention 2
Control 2
Example factorial design
• Scottish Trial of steroids or
acyclovir in Bell’s Palsy
• Recruited 500 patients
1. Steroids / Placebo
2. Steroids / Acyclovir
3. Acyclovir / Placebo
4. Placebo / Placebo
Strengths and Weaknesses of
factorial design
• Efficient – two trials for the price of one
• Relies heavily on assumption of no
interaction between treatments i.e.
effects of treatments are additive
• Results – Sullivan, Swan, Donnan et al,
Early treatment with prednisolone or
acyclovir and recovery in Bell’s palsy.
NEJM 2007; 357: 1598-607
Crossover designs
•In this design each patient receives
ALL treatments
•Order of receipt is randomised
•Requires a wash-out period between
treatments
•Only applicable to transient effects in
chronic conditions e.g. asthma, pain
Crossover Trial
Eligible subjects
RANDOMISED
Intervention
Wash-out
Control
Wash-out
Control
Intervention
Example of Crossover Trial
•2-period 2-treatment trial
of acarbose vs. placebo in
type 2 diabetes
•Mean difference in HbA1c
was 0.3% with SD of 0.5%
•No significant effect
Strengths and Weaknesses of
crossover design
• Within patient characteristics remain
same as matched analysis
• Smaller sample size needed compared with
parallel design
• Not suitable for intervention that ‘cures’,
used in chronic pain, asthma, diabetes.
• Carry-over effect needs wash-out period
Senn S. (1993) Crossover trials in clinical research. Chichester, John Wiley
CLUSTER RANDOMISATION
Man is a unit of the greater
beasts, the phalanx. The
individual relates to the large
unit or phalanx
John Steinbeck
WHY USE CLUSTER
RANDOMISATION?
•Intervention at the group level e.g.new
appointment system in general practice
•Sometimes only practical design
•Can reduce contamination
•Easier to implement intervention (e.g
counselling to reduce smoking)
Intervention at practice
level & effects on patients
Practice
level
IMPLICATIONS OF USING
RANDOMISATION BY PRACTICE?
Sample size needs to be
inflated:
Subjects within practice more alike
than subjects in different practices
so independence assumption of
statistical tests is incorrect
IMPLICATIONS OF USING
RANDOMISATION BY PRACTICE?
Sample size obtained with no
clustering is inflated by factor:
IF = 1 + (m - 1) ρ
Where  is the intra-cluster correlation
and m is the mean cluster (practice) size
Inflation or DESIGN EFFECT depends on
size of intra-cluster correlation
Assume 40% reach lipid target on new
statin and 30% on old statin
•Total(ind) 
Inflation
1
Total(Cl)
• 944
0.000
944
•944
•944
0.001
0.01
1.029
1.29
1218
•944
0.05
2.45
2312
971
How do you know degree of
intra-cluster correlation?
•Obtain from pilot work
•Previous published values
•HSRU, Aberdeen
Outcomes  0.05
Process  0.05 – 0.15
IMPLICATIONS OF USING
RANDOMISATION BY PRACTICE?
Ethical issues:
Cluster-cluster – guardian decides
on behalf of all patients
Cluster-individual – both patient
and GP decide on consent
IMPLICATIONS OF USING
RANDOMISATION BY PRACTICE?
Analysis:
Need to take clustering into
account e.g. Multi-level
modelling
Study Design - Observational
•Cohort
•Case-Control
•Cross-sectional survey
•Time series
Walker & Stampfer
...”should not denigrate the
observational nature of the data.
Most of what we learn, and will
continue to learn, about adverse
drug effects are from
observational studies”
Lancet 1996;348:489
Cohort Design
A cohort consists of a group of
individuals from a well-defined
population (exposure and
characteristics known) followed up
over time to observe what
happens to which groups and when
n.b.sociologists call these panels
How many smokers and nonsmokers died?
Non-smokers
Smokers
Events
Simple Cohort study
Smokers vs. non in Tayside population
Non-smoker
1/1/2006
TIME
Framingham cohort study
Followed up 5,573 people white, initially free of CVD,
but including people with
hypertension and diabetes
from 1968 onwards
Developed predictive algorithms
for CHD
Summary Statistics in Cohort
Study
•Drug safety study of NSAIDS
and hospitalisation for GI bleed
•Rate of hospitalisations for GI
bleed in NSAID and non-NSAID
users over 1 year
•Relative risk (RR) is ratio of rates
Relative Risk
Hosp GI bleed
Yes
No
Exposed
NSAIDS
a
b
Unexposed
c
d
RR = a/(a+b)
/ c/(c+d)
Example Relative Risk
Hosp GI bleed
Yes
No
Exposed
NSAIDS
136
4,390
Unexposed
2,104
126,060
RR = a/(a+b)
/ c/(c+d) =
1.83
Example Relative Risk
Risk in exposed = 136/(136+4390)
= 0.0300 or 30 per 1000 people
Risk in unexposed =
2104/(2104+126060) = 0.0164 or
16 per 1000 people
Hence RR = 0.0300/0.0164 = 1.83
Interpretation
RR = 1.83 and 95% CI (1.19, 2.82)
which is highly statistically
significant (p < 0.001)
83% higher risk of hospitalisation for
GI bleed in those exposed to NSAIDS
compared with those not exposed
Limitations
In reality subjects have different
length of follow-up – so need event
rates per person-years follow-up
e.g. 30 events per 1000 person years
Raw unadjusted rates and Relative
Risk – Need to take account of
confounding through regression
Retrospective Cohort
Design
•In this design group of subjects or
cohort is identified in the past
•Follow-up is then to the present
•Advantage that most data already
collected and events have occurred
•Cheaper to perform as do not have to
wait long period before analysis
Log-rank test
generalised Wilcoxon
21 = 10.6, p = 0.001
21 = 7.5, p = 0.006
Donnan et al Prognosis following first acute myocardial infarction in type 2 diabetes: A
comparative population study. Diabetic Med 2002; 19: 448-455.
Case-Control Design
•In this design group of subjects with
disease or condition are identified
(Cases)
•Suitable Control group identified
without the condition
•Frequency of Exposure or risk factor
compared in cases and controls
Case-Control Design
Past
Unexposed
Present
CASES
Exposed
Unexposed
Exposed
CONTROLS
Example of Case-Control Study
•Early studies of leukaemia
around nuclear power stations
•Consumption of red meat and
risk of colorectal cancer
•Use of mobile phones and
RTAs
Case-Control Design
•May be matched or unmatched
•Selection of cases and controls prone to bias
•Sometimes have both hospital and population
controls
•Ascertainment of exposure prone to bias
(e.g. recall bias)
•Relatively cheap to carry out but difficulties
with confounding
Cross-sectional survey
Present
Events
Exposed
Unexposed
No Events
Exposed
Unexposed
Cross-sectional Design
•Both exposure and events measured at same
time
•Often questionnaire–based surveys
•Prone to volunteer bias and poor response
•Relatively cheap to perform
•Main difficulty in ascertaining temporal
direction
•Examples are postal surveys such as census
Time series
•Can be considered a set of regular crosssectional surveys over time
•Examples are rainfall on daily basis,
performance of stocks and shares, no. of
hospitalisations on daily basis
•Requires sophisticated analyses which
account for autocorrelation – ARIMA models
•Interrupted time series can be powerful
method of assessing change following policy or
organisational changes
Example Time series
9.00
Mean HbA1c (t2)
8.00
7.00
6.00
5.00
89
87
85
83
81
79
77
75
73
71
69
67
65
63
61
59
57
55
53
51
49
47
45
43
41
39
37
35
33
31
29
27
25
23
21
19
17
15
13
11
9
7
5
3
1
Month (1=Jan 1998)
Change in mean HbA1c following introduction
(vertical line) of web-based Managed Clinical
Network for diabetes in Tayside
Study Design
What design could be utilised to answer
following questions?
•Mobile phone use and brain cancer
•Effect of Herceptin on later stage breast cancer
•Methadone and deaths in drug users
•Intensive management of high risk patients to
prevent emergency hospitalisation
•Effect of MMR vaccination and development of
autism
Study Design
•Design is critical to success
•Design determines type of analysis
•Remember
‘Chance favours the prepared mind’