Methods of explanatory analysis
for psychological treatment
trials workshop
SESSION 2
Mediation and moderation of treatment effects
Andrew Pickles
Funded by:
MRC Methodology Grant G0600555
MHRN Methodology Research Group
Methodology Research Group
Moderators & Mediators
•
Moderator is a variable that modifies
the form or strength of the relation between
an independent and a dependent variable.
•
Mediator is a variable that is
intermediate in the causal sequence relating
an independent variable to a dependent
variable.
2
Moderators in RCTs
•
Moderators are baseline characteristics that
influence the effect of treatment, or the effect
of treatment allocation (on intermediate or
final outcomes).
•
They are pre-randomisation effectmodifiers.
•
Examples: sex, age, previous history of
mental illness, insight, treatment centre,
therapist characteristics, genes etc.
3
Typical local example
Figure 2. SF36 scores by abuse categories at baseline and follow-up (treated
patients only) Creed et al., Psychosomatic Medicine 67:490–499 (2005)
4
Testing for Moderation
• A moderator variable is typically a baseline
variable (e.g. not-abused, abused)
• Makes treatment effect greater in one group
than another (moderator may or may not have
an additional direct effect on outcome). It is a
source of treatment effect heterogeneity
• A classic error is to claim moderation when
treatment effect is significant effect in one group
and not significant in another. Is simply a recipe
for increasing Type I (false positive) error rate
5
Interaction & Synergy
• Need to show significant interaction with treatment on
outcome
• But on what scale?
– Can find that interaction significant on one scale but is not
significant if outcome variable is transformed. Choice of scale
requires both statistical and clinical considerations.
– If outcome binary then usual test is for interaction on the logodds scale
• Some argue that main effects on log-odds scale already suggests
synergy
– e.g. if the base outcome rate is low and the treatment and moderator
each increase outcome by 100% then the two together increase the
outcome rate not by 200% but by 300% even without an interaction
6
The SoCRATES Trial
• SoCRATES was a multi-centre RCT designed to
evaluate the effects of cognitive behaviour therapy
(CBT) and supportive counselling (SC) on the
outcomes of an early episode of schizophrenia.
• Participants were allocated to one of three conditions:
Analysed as two conditions
Control condition: Treatment as Usual (TAU),
Treatment condition: TAU plus psychological, either
CBT + TAU or SC + TAU.
7
SoCRATES (contd.)
• 3 treatment centres: Liverpool, Manchester and Nottinghamshire.
Other baseline covariates include logarithm of untreated psychosis
and years of education.
• Outcome (a psychotic symptoms score) was obtained using the
Positive and Negative Syndromes Schedule (PANSS).
• From an ITT analyses of 18 month follow-up data, both
psychological treatment groups had a superior outcome in terms of
symptoms (as measured using the PANSS) compared to the control
group.
8
SoCRATES (contd.)
• Post-randomization variables that have a potential explanatory role
in exploring the therapeutic effects include the total number of
sessions of therapy actually attended and the quality or strength
of the therapeutic alliance.
• Therapeutic alliance was measured at the 4th session of therapy,
early in the time-course of the intervention, but not too early to
assess the development of the relationship between therapist and
patient. We use a patient rating of alliance based on the CALPAS
(California Therapeutic Alliance Scale).
.
• Total CALPAS scores (ranging from 0, indicating low alliance, to 7,
indicating high alliance) were used in some of the analyses reported
below, but we also use a binary alliance variable (1 if CALPAS score
≥5, otherwise 0).
9
SoCRATES - Summary Statistics
Centre 1 - Liv
Centre 2 - Man
Centre 3 - Nott
Mean (SD)
Control
N=39
Treated
N=29
Control
N=35
Treated
N=49
Control
N=26
Treated
N=23
Baseline PANSS
80.0 (12.36)
77.7 (13.93)
97.9 (16.6)
100.5 (16.3)
84.9 (14.91)
83.4 (10.84)
18 month PANSS
69.5 (13.55)
50.2 (13.48)
73.2 (22.4)
74.4 (20.00)
54.5 (10.07)
49.1 (7.25)
CALPAS
-
5.73 (0.81)
-
5.07 (0.88)
-
5.15 (1.47)
Sessions
0
18.14 (3.60)
0
16.16 (4.58)
0
13.87 (4.95)
High Alliance: N(%)
-
23 (79.3)
-
30 (61.2)
-
13 (56.5)
# of observed 18m
PANSS
23
23
25
39
21
22
Lewis et al, BJP (2002); Tarrier et al, BJP (2004); Dunn & Bentall, Stats in Medicine (2007).
10
Socrates positive symptoms:
basic analysis
xi: regress enpstot psubtota rgrp
Source |
SS
df
MS
-------------+-----------------------------Model | 792.779676
2 396.389838
Residual | 5945.22032
222 26.7802717
-------------+-----------------------------Total |
6738
224 30.0803571
Number of obs
F( 2,
222)
Prob > F
R-squared
Adj R-squared
Root MSE
=
=
=
=
=
=
225
14.80
0.0000
0.1177
0.1097
5.175
-----------------------------------------------------------------------------enpstot |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------psubtota |
.34999
.0785569
4.46
0.000
.1951774
.5048026
rgrp | -2.240193
.7425587
-3.02
0.003
-3.703559
-.7768275
_cons |
6.986856
1.954491
3.57
0.000
3.135127
10.83859
------------------------------------------------------------------------------
11
Socrates positive symptoms:
including main effects of centre
xi: regress enpstot psubtota i.centre rgrp
-----------------------------------------------------------------------------enpstot |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------psubtota |
.1710413
.0847491
2.02
0.045
.0040172
.3380653
_Icentre_2 |
1.679312
.8588526
1.96
0.052
-.0133193
3.371944
_Icentre_3 | -2.857869
.823287
-3.47
0.001
-4.480408
-1.235331
rgrp | -2.158757
.7039854
-3.07
0.002
-3.546176
-.7713389
_cons |
11.42025
2.038804
5.60
0.000
7.402161
15.43833
------------------------------------------------------------------------------
testparm _Icen*
( 1)
( 2)
_Icentre_2 = 0
_Icentre_3 = 0
F(
2,
220) =
Prob > F =
13.56
0.0000
12
Socrates positive symptoms:
treatment effect by centre
xi:xi: bysort centre : regress enpstot psubtota rgrp
-> centre = 1
-----------------------------------------------------------------------------enpstot |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------psubtota |
.1686252
.1880974
0.90
0.373
-.2066189
.5438693
rgrp | -3.439661
1.348812
-2.55
0.013
-6.130467
-.7488547
_cons |
12.34583
4.501713
2.74
0.008
3.365161
21.3265
------------------------------------------------------------------------------> centre = 2
-----------------------------------------------------------------------------enpstot |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------psubtota |
.0768268
.1548842
0.50
0.621
-.2314624
.385116
rgrp | -1.785964
1.448293
-1.23
0.221
-4.668719
1.096791
_cons |
15.31862
4.007697
3.82
0.000
7.341504
23.29575
------------------------------------------------------------------------------> centre = 3
-----------------------------------------------------------------------------enpstot |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------psubtota |
.3170135
.086804
3.65
0.001
.1437987
.4902283
rgrp | -.7935641
.7000109
-1.13
0.261
-2.190414
.6032859
13
_cons |
4.4636
2.042415
2.19
0.032
.3880247
8.539176
------------------------------------------------------------------------------
Socrates positive symptoms:
moderation/heterogeneity?
xi: regress enpstot psubtota i.centre*rgrp
-----------------------------------------------------------------------------enpstot |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------psubtota |
.1685492
.0866722
1.94
0.053
-.0022736
.339372
_Icentre_2 |
.6983508
1.398679
0.50
0.618
-2.058313
3.455015
_Icentre_3 | -4.532945
1.481842
-3.06
0.002
-7.453517
-1.612374
rgrp | -3.439764
1.245653
-2.76
0.006
-5.894829
-.9846987
_IcenXrgrp_2 |
1.458311
1.720016
0.85
0.397
-1.931679
4.848301
_IcenXrgrp_3 |
2.418837
1.779205
1.36
0.175
-1.087808
5.925483
_cons |
12.3476
2.257408
5.47
0.000
7.898459
16.79674
-----------------------------------------------------------------------------testparm _IcenX*
( 1)
( 2)
_IcenXrgrp_2 = 0
_IcenXrgrp_3 = 0
F(
2,
218) =
Prob > F =
0.94
0.3912
14
Mediators in Randomised
Clinical Trials (RCTs)
• Mediators are intermediate outcomes on the causal
pathway between allocation to or receipt of treatment
and final outcome.
• By definition, in an RCT, they are measured after
randomisation.
• Treatment effect may be fully or partially explained by a
given mediator. Possible for a given mediator to serve
the role of surrogate outcome.
• Possibility of multiple mediators (multiple pathways)
and interactions between mediators.
15
Post-randomisation effect
modifiers
• Intermediate outcomes that influence either (a) the
effects of treatment/treatment allocation on other
intermediate outcomes (mediators) or (b) the effects of
the other intermediate outcomes on the final outcome.
• Candidates: amount of treatment (sessions attended),
treatment fidelity, therapeutic alliance.
• Distinction between these variables and mediators not
obvious.
16
Examples
Compliance with allocated treatment
Does the participant turn up for any therapy?
How many sessions does she attend?
Fidelity of therapy
How close is the therapy to that described in the
treatment manual? Is it a cognitive-behavioural
intervention, for example, or merely emotional support?
Quality of the therapeutic relationship
What is the strength of the therapeutic alliance?
17
Examples (cont.)
What is the concomitant medication?
Does psychotherapy improve compliance with
medication which, in turn, leads to better outcome? What
is the direct effect of psychotherapy?
What is the concomitant substance abuse?
Does psychotherapy reduce cannabis use, which in turn
leads to improvements in psychotic symptoms?
What are the participant’s beliefs?
Does psychotherapy change attributions (beliefs), which,
in turn, lead to better outcome? How much of the
treatment effect is explained by changes in attributions?
18
The Mediation Industry
Baron RM & Kenny DA (1986). The moderator-mediator
variable distinction in social psychological research:
conceptual, strategic, and statistical considerations.
Journal of Personality and Social Psychology 51, 11731182.
As of 16th September 2009: 12,292 citations!
Assumptions are very rarely stated, let alone their validity
discussed.
One suspects that the majority of investigators are
oblivious of the implications.
19
A Naïve Look
at mediation:
the B&K
framework
Randomised to
Psych treatment
Independent X
c’
a
e3
Number of
sessions
Mediator M
b
Psychotic
Symptoms
Dependent
Y
e2
Regression eqns used to assess mediation
Y=d1+cX+e1
Y=d2+c’X+bM+e2
M=d3+aX+e3
total effect=c
mediated effect= ab or (c-c’) (in simple linear models these should be equal if
estimated on same sample)
20
Testing for Mediation
^
^
ab
Estimate of mediated effect =
Confidence interval
^ab^ +/- 1.96*se
ab
Estimate of seab = sqrt( ^a2 seb2
+
b^ 2 sea2)
Bootstrap resampling better (allows for asymmetry)
Test of mediation
(1) if 0 within CI
^ab^
(2) z-test for
/seab
21
Baron & Kenny Steps:
naïve mediation
•
•
•
•
Effect of X on Y (c) must be significant
Effect of X on M (a) must be significant
Effect of M on X (b) must be significant
When controlling for M, the direct effect of
X on Y (c’) must be non-significant
22
naïve mediation
xi:regress nosess rgrp psubtota i.centre
-----------------------------------------------------------------------------nosess |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------rgrp |
13.82383
.5893788
23.45
0.000
12.66366
14.98401
psubtota |
.1047549
.0649339
1.61
0.108
-.0230656
.2325754
_Icentre_2 | -1.387014
.7189374
-1.93
0.055
-2.802223
.0281941
_Icentre_3 |
-2.87773
.7188629
-4.00
0.000
-4.292792
-1.462668
_cons | -1.210907
1.551379
-0.78
0.436
-4.264754
1.84294
------------------------------------------------------------------------------
23
naïve mediation
xi:regress enpstot nosess rgrp psubtota i.centre
-----------------------------------------------------------------------------enpstot |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
-------------+---------------------------------------------------------------nosess |
.0417879
.0795974
0.52
0.600
-.1151377
.1987135
rgrp |
-2.81782
1.345742
-2.09
0.037
-5.470936
-.1647028
psubtota |
.1606631
.0871823
1.84
0.067
-.011216
.3325421
_Icentre_2 |
1.926335
.9083031
2.12
0.035
.1356243
3.717046
_Icentre_3 |
-2.54384
.9285473
-2.74
0.007
-4.374462
-.7132184
_cons |
11.47856
2.103109
5.46
0.000
7.332299
15.62482
------------------------------------------------------------------------------
A=13.80 (0.59) , B=0.042 (0.08):
A times B =0.58 (1.10)
Sobel estimate of standard error sqrt(13.82*0.082+0.0422*0.592)=1.10
24
Stata code for naïve mediation:
bootstrap 1
global model1 “nosess rgrp psubtota i.centre"
global model2 “enpstota nosess rgrp psubtota i.centre"
program mediate, rclass
version 8
xi:regress $model1
matrix a=e(b)
xi:regress $model2
matrix b=e(b)
return scalar mediate=a[1,1]*b[1,1]
end
bootstrap mediate product=r(mediate), reps(100) dots
25
Stata code for naïve mediation:
bootstrap 2
bootstrap mediate product=r(mediate), reps(100) dots
command:
mediate
statistic:
product
= r(mediate)
................................................................................
....................
Bootstrap statistics
100
Number of obs
=
Replications
213
=
-----------------------------------------------------------------------------Variable
| Reps Observed
Bias Std. Err. [95% Conf. Interval]
-------------+---------------------------------------------------------------product |
100 .5776688 -.1432935 1.057901 -1.521436
2.676773
(N)
|
-1.682636
2.333766
(P)
|
-1.682636
2.333766 (BC)
-----------------------------------------------------------------------------Note: N
= normal
P
= percentile
BC = bias-corrected
26
Mediation and measurement
error: mediation or direct and
indirect effects in SEM (Mplus)
• Testing for and estimating mediation can
be susceptible to measurement error bias
27
Direct and Indirect: Longitudinal and
sleeper effects
y1
a
y2
b
y3
c
y4
• y1 directly influences y2 through path a
• y1 only indirectly influences y3 through y2 on paths a
and b
• In a longitudinal study if y1 influences y3 directly (i.e. not
through y2) this is a ‘sleeper effect’
• This structure of restricting effects to those from the
previous occasion is known as first order autorgression
(AR1)
28
Longitudinal Ability Data
% correct at ages 6,7, 9 and 11
STANDARD DEVIATIONS
6.374 7.319 7.796 10.386
CORRELATION MATRIX
1
0.809 1
0.806 0.850 1
0.765 0.831 0.867 1
29
AR1 Model: ability1.inp
TITLE:
DATA:
MODEL:
OUTPUT:
Ability autoregressive model
FILE IS D:\courses\mplus\ability.dat;
TYPE IS CORRELATION STDEVIATIONS;
NOBSERVATIONS=204;
VARIABLE: NAMES ARE y1-y4;
USEVARIABLES ARE y1-y4;
y2 on y1;
y3 on y2;
y4 on y3;
SAMPSTAT STANDARDIZED RESIDUAL;
30
Indirect effects: Ability1b.inp
TITLE:
DATA:
Ability latent autoregressive model
FILE IS D:\courses\mplus\ability.dat;
TYPE IS CORRELATION STDEVIATIONS;
NOBSERVATIONS=204;
VARIABLE: NAMES ARE y1-y4;
USEVARIABLES ARE y1-y4;
MODEL:
y2 on y1;
y3 on y2;
y4 on y3;
MODEL INDIRECT:
y4 IND y1;
y3 IND y1;
OUTPUT:
STANDARDIZED;
CINTERVAL;
31
AR1 Model Output-1
Effects from Y1 to Y4
Total
Total indirect
0.971
0.971
0.076
0.076
12.814
12.814
0.971
0.971
0.596
0.596
Specific indirect
Y4
Y3
Y2
Y1
0.971
0.076
12.814
0.971
0.596
Effects from Y1 to Y3
Total
Total indirect
0.841
0.841
0.056
0.056
14.956
14.956
0.841
0.841
0.688
0.688
Specific indirect
Y3
Y2
Y1
0.841
0.056
14.956
0.841
0.688
32
Autoregressive Output-1
Chi-Square Test of Model Fit
Value
Degrees of Freedom
P-Value
62.124
3
0.0000
! This fits
! Very badly
ESTIMATED MODEL AND RESIDUALS (OBSERVED - ESTIMATED)
Model Estimated Covariances/Correlations/Residual Correlations
Y2
Y3
Y4
Y1
________
________
________
________
Y2
53.305
Y3
48.263
60.481
Y4
55.745
69.857
107.341
Y1
37.556
34.003
39.275
40.429
Y2
Y3
Y4
Y1
Residuals for Covariances/Correlations/Residual Correlations
Y2
Y3
Y4
Y1
________
________
________
________
0.000
-0.001
-0.001
7.114
-0.001
-0.001
33
0.000
5.852
11.120
0.000
Autoregressive Output-2
TOTAL, TOTAL INDIRECT, SPECIFIC
Estimates
Effects from Y1 to Y4
Total
0.971
Total indirect
0.971
Specific indirect
Y4
Y3
Y2
Y1
0.971
Effects from Y1 to Y3
Total
0.841
Total indirect
0.841
Specific indirect
Y3
Y2
Y1
0.841
INDIRECT, AND DIRECT EFFECTS
S.E. Est./S.E.
Std
StdYX
0.076
0.076
12.814
12.814
0.971
0.971
0.596
0.596
0.076
12.814
0.971
0.596
0.056
0.056
14.956
14.956
0.841
0.841
0.688
0.688
0.056
14.956
0.841
0.688
34
Simplex Model
V’s measured with error
Autoregressive F’s
Age 6
Age 7
Age 9
Age 11
y1
y2
y3
y4
f1
f2
f3
f4
Curiously, middle part of model is identified without restrictions,
but the whole model is not identified without some restrictive
assumptions e.g. measurement error and reliability constant with age
35
Simplex Model: ability2.inp
TITLE:
DATA:
Ability latent autoregressive model
FILE IS D:\courses\mplus\ability.dat;
TYPE IS STDEVIATIONS CORRELATION;
NOBSERVATIONS=204;
VARIABLE: NAMES ARE y1-y4;
USEVARIABLES ARE y1-y4;
MODEL:
f1 by y1 (1);
f2 by y2 (1);
f3 by y3 (1);
f4 by y4 (1);
y1 y2 y3 y4 (2);
f2 on f1;
f3 on f2;
f4 on f3;
MODEL INDIRECT: f3 IND f1;
f4 IND f1;
OUTPUT:
STANDARDIZED;
36
Simplex Model: ability2.out
TESTS OF MODEL FIT
Chi-Square Test of Model Fit
Value
Degrees of Freedom
P-Value
1.440
2
0.4835
TOTAL, TOTAL INDIRECT, SPECIFIC INDIRECT, AND DIRECT EFFECTS
Estimates
Effects from F1 to F3
Total
1.170
Total indirect
1.170
Specific indirect
F3
F2
F1
1.170
Effects from F1 to F4
Total
1.516
Total indirect
1.516
S.E.
Est./S.E.
Std
StdYX
0.074
0.074
15.901
15.901
0.925
0.925
0.925
0.925
0.074
15.901
0.925
0.925
0.105
0.105
14.396
14.396
0.877
0.877
0.877
0.877
37
Simplex Model: conclusion
• Conclusion.
In the presence of measurement error in the mediator
the mediated effect is underestimated (attenuated) and
the residual “direct” effect over-estimated.
With multiple predictors (mediators) measurement error
can result in decreased, increased and quite spurious
effects being estimated.
• But still ignores possible confounding – to be
addressed this afternoon
38