BABCP Annual Conference, Exeter, July 17th, 2009
THE EVIDENCE-BASE FOR CBT SUPERVISION:
SYSTEMATIC REVIEWS OF THE EMPIRICAL LITERATURE
DEREK MILNE, ALIA SHEIKH; SUE PATTISON; ANDREW WILKINSON
Background
Clinical supervision training regarded as essential.
Yet training not been given systematic attention
(“Something does not compute”: Watkins, 1997) :
concerns that evidence-base generally weak, etc.
One way forward: selective (BES) systematic review of
controlled studies.
13/04/2015
2
Best evidence synthesis (BES): a novel, affirming
approach to the systematic literature review:
 BES “combines the meta-analytic approach of
extracting quantitative information in a common
standard format…with a systematic approach to
the assessment of study quality and study
relevance” (Petticrew &Roberts, 2006, p.181);
 Often used in education, to answer questions
about what works (pragmatic focus);
 Careful selection of studies - Milne et. al. 2006:
9 inclusion criteria, inc: ‘CS manipulated;
demonstrated effectiveness; in scientific journal’.
Objectives
 To assess the degree of empirical support for
supervisor training
 To define training methods that are supported
in the selected literature
 To specify recommendations for the training of
supervisors
13/04/2015
4
Methodology
 Conducted systematic literature search of three
databases (MedLine, PsycInfo, ASSIA)
 Key words: “clinical supervision,” and “clinical
supervisor”
 Narrowed search using term “empirical”
 Studies selected according to specific inclusion
criteria, based on NICE(R) Manual for
Systematic Reviews of the Clinical Supervision
Literature (Version 11; Milne & James, 2000)
 Final sample: 11 studies (N= 145 supervisors;
251 supervisees; 147 clients)
13/04/2015
5
Methodology
 Studies also evaluated according to NICE(R)
(2001) coding manual
 Coding system allows analysis of the
methodological quality of studies (e.g., use of
measures, threats to validity, etc) & of the
methods (Independent variables)
 2 studies independently coded by researchers,
1 at start (inter-rater agreement=81%) & 1 at
end (82%)
13/04/2015
6
Results: 2 Questions
Addressing the two questions posed by
Whitman, Ryan & Rubinstein (2001):
What empirical support is there for
supervisor training?
Which elements of such training are
supported by evidence?
13/04/2015
7
Q1: What empirical support is there
for supervisor training?
 11 studies provided empirical support for supervisor
training (e.g. they had sound methodological validity and
an average 67% impact on the 145 supervisors’
learning)
 15 training methods- primarily:
corrective feedback (all 11 studies);
educational role-play &
observational learning (both in 8);
teaching (in 5 studies: NB: next slide arranged
by frequency of methods use in training)
13/04/2015
8
Q2: Which elements of such training are supported by evidence?
 Feedback
 Educational role-play
 Modeling (live/video
demonstration)
 Teaching (verbal
instruction)
 Written assignments
 Behavioural rehearsal
 Providing a rationale
 Guided reading
13/04/2015
 Discussion
 Educational needs
assessment
 Direct observation
 Questions and answers
 Agenda-setting
 Homework
assignments
 Quiz
9
These methods were blended, to be:
Multiple (mean 5.1 per study);
Structured (address training cycle);
Complementary (experiential learning cycle)
Structured approach recommended
Loganbill and Hardy (1983):
3 necessary components of training: theoretical content,
simulated experience, and in vivo practise with supervisees;
Russell and Petrie (1994): combine didactic and
experiential (“critical element in an effective supervisory
training programme” ; p. 39). More recent reviews agree:
Steinert et al.,2006 (systematic review; faculty);
Falender et al., 2004; Kaslow et al., 2004 (consensus).
Example: Bambling, et.al. (2006),
Psychotherapy Research, 16, pp. 317-331.
N=40 supervisors
(RCT: CBT or Dynamic;
N= 127 therapists & N= 127 patients)
Trained separately (1 day workshops:
teaching about instruments & role-plays)
Evaluated similarly: learning (self-
rated adherence) & impact (BDI; Working
Alliance Inventory; client satisfaction)
Discussion:
Some problems with BES:
Ignores negative findings?
True, in order to address pragmatic questions; but mixed
results, yet similar conclusions
Problem of induction follows?
True, but accidently coded weak/negative papers were also
similar (failed to falsify);
Also, BES review = falsifiable statement
Generalize to non-LD/brief supervision?
True, there very likely are significant differences:
awaits more studies of ‘complex’ supervision
Conclusions & recommendations
 BES systematic review = pragmatic method for
addressing inconsistency between policy and practice
 Results offer a ‘foundational’ evidence-base for CBT
supervisor training, consistent with educational theory,
best practise consensus and policies (add ‘complex’
supervision evidence as it emerges)
 Mental health professionals should be prepared for the
recognized specialization of supervision following the
structured training format
13/04/2015
14
Acknowledgements
 This study was funded by the Higher Education
Academy, Psychology Network
 We are grateful to Helen Aylott, Nasim
Choudhri, Christopher Dunkerley, Helen
Fitzpatrick, and Sarah Wharton for their initial
work on this review
 Thanks also to Kathryn Mark, Barbara Mellors,
and Thomas Cliffe for help in preparing the ppt
13/04/2015
15
Authors
DEREK MILNE, School of Psychology, Newcastle
University, Newcastle upon Tyne, NE1 7RU
d.l.milne@ncl.ac.uk
ALIA SHEIKH, School of Psychology, Newcastle University,
Newcastle upon Tyne, NE1 7RU
a.i.sheikh@ncl.ac.uk
SUE PATTISON, School of Education, Communication and
Language Sciences, Newcastle University, Newcastle upon
Tyne, NE1 7RU
susan.pattison@ncl.ac.uk
ANDREW WILKINSON, North Tyneside Primary Care
Trust. Andrew.Wilkinson@northtyneside-pct-nhs.uk
13/04/2015
16