Complier Average Causal
Effect analysis
Jon Nicholl
ScHARR, Sheffield
• Pragmatic evaluations allow patients and clinicians to comply
with treatment regimes as they would in practice
• This often means not completing (adhering to) or following
(complying with) the allocated treatment, eg X-overs in
surgical trials
• ITT analysis of everyone gives a true estimate of the real
world population treatment effect, but not of the effect in an
individual who is treated (because ITT is diluted by noncompliers)
Dealing with non-compliance in TAU or TAU+placebo
controlled trials:
Complier Average Causal Effect analysis
Complied with A
Treatment
group
N
Intervention 200
A
Control
B (TAU)
Non-compliers with A
All
events
ER
N
E
ER
N
E
ER
40
0.2
100
30
0.3
300
70
0.233
300
80
0.266
ITT analysis: all A vs all B = 0.233 / 0.266 = 0.875. This estimate is diluted
(biased towards 1.0) by the non-compliers
• Sometimes the average effect in individuals who comply is
estimated by a per protocol analysis
Dealing with non-compliance in TAU or TAU+placebo
controlled trials:
Complier Average Causal Effect analysis
Complied with A
Treatment
group
N
Intervention 200
A
Control
B (TAU)
Non-compliers with A
All
events
ER
N
E
ER
N
E
ER
40
0.2
100
30
0.3
300
70
0.233
300
80
0.266
ITT analysis: all A vs all B = 0.233 / 0.266 = 0.875. This estimate is diluted
(biased towards 1.0) by the non-compliers
Per protocol analysis: Complied A vs all B = 0.2 / 0.266 = 0.75
Per protocol analysis
• Pointless. It certainly doesn’t work in
pragmatic, open trials
• And may not in double blind placebo
controlled trials
Five-year mortality in CHD patients given clofibrate,
according to cumulative adherence to protocol prescription.
Adherence
No of
patients
Clofibrate
Placebo
% died
No of
patients
< 80%
357
24.6 ± 2.3
882
28.2 ± 1.5
> 80%
708
15.0 ± 1.3
1813
15.1 ± 0.8
NEJM 1980; 303: 1038-
% died
Five-year mortality in CHD patients given clofibrate,
according to cumulative adherence to protocol prescription.
Adherence
No of
patients
Clofibrate
Placebo
% died
No of
patients
< 80%
357
24.6 ± 2.3
882
28.2 ± 1.5
> 80%
708
15.0 ± 1.3
1813
15.1 ± 0.8
% died
2)
Selection by patient preference:
Five-year mortality in CHD patients given clofibrate,
according to cumulative adherence to protocol prescription.
Adherence
No of
patients
Clofibrate
Placebo
% died
No of
patients
< 80%
357
24.6 ± 2.3
882
28.2 ± 1.5
> 80%
708
15.0 ± 1.3
1813
15.1 ± 0.8
% died
The problem is different types of patients have different
adherence/compliance rates
Can we estimate the control treatment effect in patients who
would have complied with A?
Dealing with non-compliance in TAU or TAU+placebo
controlled trials:
Complier Average Causal Effect analysis
Complied with A
Treatment
group
N
Intervention 200
A
Control
B (TAU)
Non-compliers with A
All
events
ER
N
E
ER
N
E
ER
40
0.2
100
30
0.3
300
70
0.233
300
80
0.266
?
CACE analysis: Complied A vs ER in controls who would have complied with A
= 0.2 / ?
• Why is this important for cmRCT ?
• How does it work?
Why is it important?
• cmRCT designs randomly select some eligible patients from
the cohort to be offered a treatment
• For an unbiased analysis all eligible patients offered the
treatment are compared with all eligible patients not selected
• If the take up of the offer is low, cmRCT designs may seriously
underestimate the treatment effect
• CACE is a method for adjusting for this
How does CACE analysis work?
Complied with A
Treatment
group
N
Intervention 200
A
Control
B (TAU)
Non-compliers with A
All
events
ER
N
E
ER
N
E
ER
40
0.2
100
30
0.3
300
70
0.233
300
80
0.266
?
CACE analysis: Complied A vs ER in controls who would have complied with A
= 0.2 / ?
How does CACE analysis work?
Treatment
group
Compliers with A
Non-compliers with A
All
N
Intervention 200
A
Control
B (TAU)
200
events
ER
N
E
ER
N
E
ER
40
0.2
100
30
0.3
300
70
0.233
?
100
300
80
0.266
First, we imagine that there are two types of patient – ‘compliers with A’
and ‘non-compliers with A’.
since the trial is randomised, we know (estimate) what % of controls are
‘non-compliers with A’ = 100/300
How does CACE analysis work?
Treatment
group
Compliers with A
Non-compliers with A
All
N
Intervention 200
A
Control
B (TAU)
200
events
ER
N
E
ER
N
E
ER
40
0.2
100
30
0.3
300
70
0.233
?
100
0.3
300
80
0.266
Next, we assume that controls had the same treatment as intervention
group A patients who didn’t comply. So the event rate in controls who are
‘non-compliers with A’ would be the same as non-compliers with A given
treatment A = 0.3
Is this assumption reasonable?
Is this assumption reasonable?
• Design: TAU + A vs TAU
• Assumption is right
• Design: A vs TAU.
– Then the question is whether non-compliers with A have
TAU or nothing
• When non-compliance = X-over the assumption is right
• When A = offer of A, and TAU = no offer, then the assumption is
also likely to be right unless the offer of A affects outcome
• Design: TAU + A vs TAU + placebo.
• Then the question is whether there is a placebo effect
How does CACE analysis work?
Treatment
group
Compliers with A
Non-compliers with A
All
N
Intervention 200
A
Control
B (TAU)
200
events
ER
N
E
ER
N
E
ER
40
0.2
100
30
0.3
300
70
0.233
?
100
30
0.3
300
80
0.266
Given the event rate, we can now estimate the number of events in the
control group who are ‘non-compliers with A’ = 100 x 0.3 = 30
How does CACE analysis work?
Treatment
group
Compliers with A
Non-compliers with A
All
N
events
ER
N
E
ER
N
E
ER
Intervention 200
A
40
0.2
100
30
0.3
300
70
0.233
Control
B (TAU)
80 -30
= 50
?
100
30
0.3
300
80
0.266
200
And, by subtraction, the number of events in the control group who are
‘compliers with A’
How does CACE analysis work?
Treatment
group
Compliers with A
Non-compliers with A
All
N
events
ER
N
E
ER
N
E
ER
Intervention 200
A
40
0.2
100
30
0.3
300
70
0.233
Control
B (TAU)
50
?=
100
50/200
= 0.25
30
0.3
300
80
0.266
200
And hence we can estimate the event rate in the control group who are
‘compliers with A’
How does CACE analysis work?
Treatment
group
Compliers with A
Non-compliers with A
All
N
events
ER
N
E
ER
N
E
ER
Intervention 200
A
40
0.2
100
30
0.3
300
70
0.233
Control
B (TAU)
50
?=
0.25
100
30
0.3
300
80
0.266
200
CACE analysis: Complied A vs ER in controls who would have complied with A
= 0.2 / 0.25 = 0.8
How does CACE analysis work?
Treatment
group
Compliers with A
Non-compliers with A
All
N
events
ER
N
E
ER
N
E
ER
Intervention 200
A
40
0.2
100
30
0.3
300
70
0.233
Control
B (TAU)
50
?=
0.25
100
30
0.3
300
80
0.266
200
ITT analysis: All A vs all B = 0.233 / 0.266 = 0.875
Per protocol analysis: Complied A vs all B = 0.2 / 0.266 = 0.75
CACE analysis: Complied A vs ER in controls who would have complied with A
= 0.2 / 0.25 = 0.8
The analysis works in the same way for continuous
outcomes
Compliers with A
Non-compliers with A
All
N
Mean
outcome
N
Mean
outcome
N
Mean
outcome
Intervention 200
A
60.0
100
50.0
300
56.7
Control
B (TAU)
53.0
100
50.0
300
52.0
Treatment
group
200
(200 x ?) + (100 x 50.0) = 52.0
ITT analysis: All A vs all B = 56.7 – 52.0 = 4.7
Per protocol analysis: Complied A vs all B = 60.0 – 52.0 = 8.0
CACE analysis: Complied A vs ER in controls who would have complied with A
= 60.0 – 53.0 = 7.0
CACE was made for cmRCT trials
• CACE analysis can be used for A vs TAU trials as in cmRCT
designs
• It is very helpful for dealing with compliance rather than
adherence (ie when patients randomised to treatment don’t
take it up at all). This is the case in trials of the offer of
treatment - as in cmRCT designs
• Some sort of CACE analysis (as opposed to ITT) is essential
when compliance is low – as may be the case in cmRCT
studies because there is no pre-selection of patients
Thank you
• Compliance/adherence and CACE analysis:
• Hewitt CE, et al. Can Med Assoc J 2006;175:347 (for a simple
explanation).
• Dunn G, et al. Statistical methods in medical research 2005; 14:
369-395 (for a statistical exploration)