CBT-PCT-PDT - 1
Effectiveness of cognitive-behavioural, person-centred and psychodynamic therapies as practised in UK National Health
Service settings.
Stiles, William B.; Barkham, Michael; Twigg, Elspeth; Mellor-Clark, John; Cooper, Mick
Psychological Medicine, Vol 36(4), Apr 2006, 555-566
[pre-publication version]
17 August 2005
Effectiveness of Cognitive-Behavioural, Person-Centred, and Psychodynamic
Therapies as Practiced in National Health Service Settings
William B. Stiles, Miami University
Michael Barkham and Elspeth Twigg, University of Leeds
John Mellor-Clark, CORE Information Management Systems
Mick Cooper, University of Strathclyde
Text: 4,549 words
Address for editorial correspondence
William B. Stiles
Department of Psychology
Miami University
Oxford, OH 45056, USA
Tel. +1 513 529 2405
Fax +1 513 529 2420
E-mail stileswb@muohio.edu
CBT-PCT-PDT - 2
Effectiveness of Cognitive-Behavioural, Person-Centred, and Psychodynamic
Therapies as Practiced in National Health Service Settings
ABSTRACT
Background. Psychotherapy's equivalence paradox is that treatments have equivalently
positive outcomes despite non-equivalent theories and techniques. We compared the
effectiveness of contrasting approaches practiced in routine care.
Method. Patients (n = 1,309) who received cognitive-behavioural therapy (CBT), personcentred therapy (PCT) and psychodynamic therapy (PDT) at one of 58 NHS primary and
secondary care sites during a three-year period completed the Clinical Outcomes in Routine
Evaluation-Outcome Measure (CORE-OM) at the beginning and end of their treatment.
Therapists indicated which treatment approaches were used on an End of Therapy form. We
compared outcomes of six groups: three treated with CBT, PCT, or PDT only, and three
treated with one of these plus one additional approach (e.g., integrative, supportive, art),
designated CBT+1, PCT+1, or PDT+1, respectively
Results. All six groups averaged marked improvement (pre-post effect size = 1.36).
Treatment approach and degree of dilution ("only" vs "+1" ) each accounted for statistically
significant but comparatively tiny proportions of the variance in CORE-OM scores
(respectively, 1% and .5% as much as pre-post change). Distributions of change scores were
largely overlapping.
Conclusions. Results were consistent with the Dodo verdict, "Everybody has won, and all
must have prizes". Caution is warranted because of limited treatment specification, nonrandom assignment, lack of a control group, missing data, and other issues.
CBT-PCT-PDT - 3
Effectiveness of Cognitive-Behavioural, Person-Centred, and Psychodynamic
Therapies as Practiced in National Health Service Settings
INTRODUCTION
Three widely influential approaches to counselling and psychotherapy are cognitivebehavioural therapy (CBT), person-centred therapy (PCT) and psychodynamic or
psychoanalytic therapy (PDT). Each encompasses a range of techniques and should be
considered them a family of treatments rather than a specific treatment protocol.
Nevertheless, they are distinct from each other in terms of their usual repertoires of
interventions and their assumptions about the nature and sources of psychopathology, and
each is considered by its practitioners to be widely applicable to the problems presented for
psychotherapeutic treatment (Feltham & Horton, 2006; Gabbard, Beck, & Holmes, 2005).
We studied the effectiveness of treatments employing varied versions of these three
approaches as delivered to 1309 patients in 58 National Health Service (NHS) mental health
service settings during a three-year period from January 1999 to November 2001.
There is strong evidence for the efficacy and effectiveness of CBT for a wide variety
of disorders (e.g., Dobson, 1989; Hollon, & Beck, 2004; Hollon, Thase,& Markowitz, 2002;
Westbrook & Kirk, 2005). Fewer studies have systematically examined outcomes of the other
two approaches, but available evidence similarly supports the efficacy and effectiveness of at
least some varieties of PCT (Elliott, Greenberg, & Lietaer, 2004; Greenberg & Watson, 1998;
Goldman, Greenberg, & Angus, in press; Ward et al., 2000) and PDT (Leichsenring, 2001;
Leichsenring & Leibing, 2003; Leichsenring Rabung,& Leibing, 2004). Clinical trials
comparing alternative approaches (e.g., Barkham et al., 1996; Elkin et al., 1989; Shapiro et
al., 1994) and broadly-based reviews (e.g., Roth & Fonagy, 2004; Wampold, 2001) have
concluded that bone fide therapies that have been actively researched tend to be similarly
effective. This is the equivalence paradox: many psychotherapies appear to have equivalently
CBT-PCT-PDT - 4
positive outcomes despite manifestly non-equivalent theories and techniques. The paradox is
expressed by the Dodo verdict, "Everybody has won, and all must have prizes" (Carroll,
1865/1946, p. 28; italics in original), which has been quoted by psychotherapy researchers for
seven decades (e.g., Beutler, 1991; Luborsky, Singer, & Luborsky, 1975; Norcross, 1995;
Rosenzweig, 1936; Seligman, 1995; Stiles et al., 1986).
Despite these indications of equivalent effectiveness across many treatments, the
overwhelming quantity of published research on CBT (Roth & Fonagy, 2004; Wampold,
2001) has given CBT a greater credibility than the other approaches. For example, the great
majority of approaches on the list of empirically supported treatments (formerly, empirically
validated treatments) produced by the American Psychological Association's Division 12
Task force (Chambless et al., 1998) were in the CBT family, having been included because of
the quantity and quality of research about them (Chambless & Hollon, 1998) rather than their
demonstrated superiority over alternative bona fide treatments.
Holmes (2002) argued that CBT has been oversold within the British NHS. Although
CBT is clearly effective, its dominance is more a function of differential research attention
than evidence of differential effectiveness (but see also Tarrier, 2002). Thus, the question
arises, are there differences in effectiveness between alternative approaches as they are
currently being practiced in NHS settings?
Ward et al. (2000) randomly assigned depressed clients to PCT (which they called
non-directive counselling) or CBT, each delivered in doses of up to 12 sessions, at 24 NHS
practices in London and Manchester. Results showed both approaches were similarly
efficacious. Both psychotherapies outperformed general practitioner care over a four-month
interval, though at a 12-month follow-up, the GP care group had caught up, and patients in all
three conditions showed similar and substantial improvement. Barkham et al. (1996)
randomly assigned clients to PDT (which they called psychodynamic-interpersonal therapy)
CBT-PCT-PDT - 5
with CBT for depression, each delivered in 8- or 16-session doses at 3 NHS sites. Again,
results showed both approaches were similarly efficacious, though there was no untreated
comparison group. Our study complements these efficacy studies by addressing the
effectiveness of different treatments as they are applied in routine practice. The logic of
effectiveness research is that the risks of selection biases associated with lack of
randomisation and the lack of assurance that the treatments were delivered in a standard way
are balanced by the greater realism (cf. Seligman, 1995). Results address the effects of
treatments as routinely delivered, using practitioners' versions of the treatments and the
patients who typically receive them.
Our study answers calls for clinically representative research (e.g., Shadish, Navarro,
Matt, & Phillips, 2000; Stirman, DeRubeis, Crits-Chrisoph, & Brody, 2003; Street,
Niederehe, & Lebowitz, 2000), drawing on routine treatments and using data collected
primarily for clinical and administrative use, rather than for research. Our null hypothesis was
that there would be no difference in clinical effectiveness between CBT, PCT, and PDT as
delivered in routine practice. We assessed effectiveness by comparing pre-treatment to posttreatment scores on the Clinical Outcomes in Routine Evaluation-Outcome Measure (COREOM; Barkham et al., 1998, 2001, 2005; Cahill et al., submitted; Connell et al., submitted;
Evans et al., 2000, 2002).
METHOD
Participants
We studied data from 1309 patients who received CBT, PCT, or PDT and completed
the CORE-OM at the beginning and end of their treatment. The data were collected during a
three-year period (January, 1999 – November, 2001) at 58 National Health Service (NHS)
sites delivering counselling and psychotherapy services where the CORE-OM was routinely
administered. The 58 NHS sites each contributed from 1 to 111 of the patients (mdn=16.5; 24
CBT-PCT-PDT - 6
of the sites each contributed 20 or more of the patients). Most of the patients (n=844; 64.5%)
were seen in primary care counselling services; the others were seen in sites described as
Secondary Care/Psychology Service (n=87; 6.6%), Tertiary Care/Specialist (n=43; 3.3%),
Primary Care Psychology/Secondary Care in Primary Setting (n=128; 9.8%) or Psychology &
Counselling Service (n=207; 15.8%). The data were anonymised at the sites and forwarded to
the University of Leeds. Data collection complied with data protection procedures for the use
of routinely collected clinical data. Therapist characteristics were not forwarded; however,
based on the 878 patients for whom therapist ID numbers were recorded, 251 therapists each
saw from 1 to 29 of the patients (mdn = 2; 15 of the therapists each saw 10 or more of the
patients). These patients were a subsample of the patients described by Barkham et al. (2005);
selection procedures are described later.
Of the 1309 patients, 29.3% (n=383) were male; 2.8% (n=36) were aged under 20
years, 19.6% (n=257) were aged 20-29 years, 29.7% (n=389) were aged 30-39 years, 24.7%
(n=323) were aged 40-49 years, 15.2% (n=199) were aged 50-59 years, and 8% (n=105) were
aged over 60 years. Patients presented a variety of psychological problems, as described later.
Nearly half of the patients (n=638; 48.7%) were taking prescribed medication at the start of
therapy, and data on medication was missing for 44 patients (3.4%). Of those who were
taking psychotropic medication, 42.7% (n=555) were prescribed only one type of medication;
5.3% (n=69) were prescribed 2 types, and .8% (n=10) were prescribed 3 types. Most of those
prescribed medications were taking anti-depressants (88.7%), while 14.7% were taking
anxiolytics/hypnotics, 4.7% were taking anti-psychotics, and 2.7% were taking other
psychotropic medications.
Measures
Self-report outcome measure. The CORE-OM comprises 34 items addressing
domains of subjective well-being, symptoms (anxiety, depression, physical problems,
CBT-PCT-PDT - 7
trauma), functioning (general functioning, close relationships, social relationships) and risk
(risk to self, risk to others). Half the items focus on low intensity problems (e.g. ‘I feel
anxious/nervous’) and half focus on high intensity problems (e.g. ‘I feel panic/terror’). Eight
items are positively keyed. Items are scored on a 5-point scale, scored from 0 to 4 (anchored
Not at all, Only occasionally, Sometimes, Often, and All or most of the time). CORE clinical
scores are computed as the mean of all completed items, which is then multiplied by 10, so
that clinically meaningful differences are represented by whole numbers. Thus, CORE
clinical scores can range from 0 to 40. Forms are considered valid if up to three items are
omitted (Evans et al., 2002). Internal consistency reliability for the 34-item scale in clinical (n
= 713) and non-clinical (n = 1009) samples were .94 and .94, respectively (Barkham et al.,
2001). The one-week test-retest correlation was Spearman's rho = .90 (n = 43) in a student
sample (Evans et al., 2002).
Therapist assessments. The CORE Assessment (Mellor-Clark et al., 1999; MellorClark & Barkham, 2000) comprises two practitioner-completed sections, the Therapist
Assessment form, completed at intake, and the End of Therapy form. On the Therapist
Assessment form, therapists gave referral information, patient demographics, and data on the
nature, severity, and duration of presenting problems using the following 14 categories:
depression, anxiety, psychosis, personality problems, cognitive/learning difficulties, eating
disorder, physical problems, addictions, trauma/abuse, bereavement, self-esteem,
interpersonal problems, living/welfare and work/academic.
On the End of Therapy form, therapists indicate which type(s) of therapy was (were)
undertaken with the patient; categories were psychodynamic, psychoanalytic, cognitive,
behavioural, cognitive/behavioural, structured/brief, person-centred, integrative, systemic,
supportive, art, and other. Therapists were asked to indicate as many as were appropriate.
They also reported the number of sessions attended and other aspects of the treatments.
CBT-PCT-PDT - 8
Procedure
Data collection. All patients attending for psychological assessment or therapy at
participating services were asked to complete a CORE-OM before treatment began. Forms
were completed during screening or assessment by 77.1% of the patients and immediately
before the first therapy session by the remaining 22.9%. Patients were allocated to treatments
and therapists following normal procedures at these sites. Sites were instructed to give the
post-treatment CORE-OM at the last session; the timing and specific procedures were
determined by what worked best for each service administratively and were not recorded.
Therapists completed the Therapist Assessment form after the intake session and the End of
Therapy form when the patient was discharged or stopped attending for therapy. Patients
completed a consent form at the start of therapy stating that they agreed to their data being
processed. Completed measures were sent to the University of Leeds for processing with no
patient identifiers. Each patient was allocated a unique code number by the site.
Selection of patients. At least one CORE-OM or CORE Assessment form was
returned regarding 10,351 patients seen at these sites during the three-year data collection
period (see Barkham et al., 2005, for further details of this sample). Of these, 1345 were
excluded because they did not return a pre-treatment CORE-OM form, and 138 were
excluded because their pre-treatment form had more than three items missing (most of these
apparently failed to turn over the page on the form). A further 5444 failed to return a posttreatment CORE-OM form, including those who did not attend any sessions and those who
had not ended their treatment by the closing date; 14 more returned a post-treatment COREOM that was incomplete; and the therapists of 359 more failed to complete an End of
Therapy form. This left 3051 patients who completed reliable pre- and post-treatment COREOM forms and whose therapists completed End of Therapy forms.
From these 3051, we selected six treatment groups based on therapists' reports on the
CBT-PCT-PDT - 9
End of Therapy form regarding the type(s) of therapy undertaken. Most therapists indicated
more than one of the 12 categories provided (M = 2.06; range = 1-10). For the purposes of
this study, we classified the three targeted approaches as follows:
CBT = cognitive, behavioural, and/or cognitive/behavioural
PCT = person-centred
PDT = psychodynamic and/or psychoanalytic.
Using these targeted approaches, we defined six groups of patients. Three groups were those
whose therapists specified therapies belonging to one and only one of the targeted
approaches--CBT, PCT, or PDT. The other three groups were those whose therapists
specified one of the targeted approaches plus one treatment not included in the targeted
approaches (i.e., one of the following: structured/brief, integrative, systemic, supportive, art,
or other), abbreviated CBT+1, PCT+1, and PDT+1, respectively. We reasoned that the latter
three groups offered comparisons among the targeted approaches that were parallel to, but
more diluted than, the former three groups.
Of the 3,051 patients who completed pre- and post-treatment CORE-OM forms, 1309
met specifications for one of the six groups. Numbers in each group are shown in Table 1.
Patients who received either none of the targeted approaches (n = 685), or more than one of
the targeted approaches (n = 305), or who received more than one treatment in addition to
one of the targeted approaches (n = 752) were not considered further in this study.
RESULTS
Effectiveness of treatment in NHS Settings
Patients in these treatments showed very substantial gains, with patients improving,
on average, from 17.41 (sd = 6.52; internal consistency alpha=.93) to 8.50 (sd = 6.27) on the
CORE-OM, a difference of 8.9 (sd = 6.81). The overall treatment effect size, calculated as the
mean pre-post difference divided by the pre-therapy sd was 1.36.
CBT-PCT-PDT - 10
Table 1 shows the mean pre-treatment and post-treatment CORE-OM clinical scores
for each of the six groups, mean differences across treatment, and effect sizes. A one-way
analysis of variance (ANOVA) comparing the pre-therapy means across the six groups was
not significant, F(5, 1303) = 0.66, p=.654, partial η2 = .003, indicating that all groups began
treatment with equivalent levels of disturbance.
To assess treatment effectiveness, we conducted a repeated-measures (pre-treatment
vs post-treatment) ANOVA, with treatment approach (CBT vs PCT vs PDT) and degree of
dilution (pure vs "+1") as fixed factors. Results showed a very large overall within-patients
main effect of treatment, F(1, 1303) =1905.70, p < .001, partial η2 = .594, indicating that
improvement across treatment accounted for a large proportion of the variation in the
obtained CORE-OM scores. In this analysis, a differential treatment effect appears as a
treatment by occasion of assessment (pre-post) interaction The results showed that this effect
was significant but very small, F(2, 1303) = 3.94, p = .020, partial η2 = .006; none of the posthoc comparisons among group means were significant. The comparative effectiveness of the
pure vs diluted forms of treatments (the dilution by occasion of assessment interaction) was
also significant but very small F(1, 1303) = 4.02, p = .045, partial η2 = .003. The three-way
treatment by dilution by occasion interaction, which would have indicated that purity versus
dilution was differentially important for the therapies, was not significant, F(2, 1303) = 1.40,
p = .248, partial η2 < .001.
Figure 1 depicts the distributions of the pre-post differences in CORE-OM clinical
scores for each group in the form of notched box plots, which indicate the median, middle
50%, and range. Although these change scores ranged widely (from -21 to 32 out of the
possible CORE-OM range of -40 to 40), the medians were similar, and the distributions of all
six groups were largely overlapping.
Following Jacobson and Truax (1991), we distinguished patients who had achieved
CBT-PCT-PDT - 11
reliable and clinically significant improvement (RCSI) as those who met two criteria: (a)
reliable improvement, defined as a pre-post difference that, when divided by the standard
error of the difference, is equal to 1.96, which was a decrease of 4.8 points on the COREOM, based on this sample, and (b) clinically significant improvement--entering treatment in a
dysfunctional state and leaving treatment in a normal state--defined as moving from above to
below the recommended CORE-OM clinical cut-off score of 10 (Connell et al., submitted).
Table 2 shows the number and percentage of patients in the each treatment group who
(a) achieved RSCI, (b) achieved reliable improvement only, (c) showed no reliable change,
and (d) showed reliable deterioration, defined as an increase of 4.8 or more points on the
CORE-OM. We restricted this analysis to the 1129 patients whose pre-therapy CORE-OM
scores were 10 or higher, insofar who patients whose initial scores were below the clinical
cut-off could not, by definition, achieve clinically significant improvement. The results
showed substantial rates of improvement in all groups. A 2 x 3 chi-square test comparing the
rates of RCSI versus non-RCSI (combining the other three cells in each row of Table 2)
across the three pure treatment groups (CBT, PCT, PDT) was nearly significant, χ2(2) = 5.89
(p = .053, n = 641), reflecting the 13.9% spread in RCSI rates across groups (CBT highest,
PDT lowest). The parallel test across the three diluted groups (CBT+1, PCT+1, PDT+1) was
not significant, χ2(2) =1.92 (p = .382, n = 488). The 2 x 2 chi-square test comparing RCSI
rates across the combined pure versus diluted groups was significant, χ2(1) = 8.38 (p = .004, n
= 1129), reflecting a somewhat higher RCSI rate in the groups given diluted versions of the
therapies than in the groups given pure versions (66.2% vs 57.7%).
Characteristics of patients allocated to treatment groups
Table 3 shows distribution of treatment groups across types of NHS settings in this
sample. Most notably, CBT was delivered at a variety of settings, whereas the other
treatments were delivered mainly in primary care counselling services.
CBT-PCT-PDT - 12
Table 4 shows the distribution of presenting problems across treatment groups, as
reported by the therapists just following their first contact with the patients. Note that
therapists could indicate multiple problems, so the columns sum to more than the number
patients in each group. Broadly similar distributions of problems were treated within each
group, but there appeared to be a few differences. For example, patients presenting with
interpersonal problems seemed differentially likely to be seen in PCT.
Table 5 shows the gender, age, mean number of problems indicated, and mean
number of sessions attended for patients in each treatment group. Gender was somewhat
unevenly distributed across groups, χ2(5) = 29.07, p < .001, n = 1309, with a relatively higher
percentage of male patients in the CBT group. Mean age was similar in the three groups; a
one-way ANOVA was not significant, F(5, 1303) =0.72, p = .607, partial η2 = .003. The
numbers of presenting problems across treatment groups varied significantly across the six
groups, F(5, 1270) =13.76, p<.001, partial η2 = .051, reflecting somewhat larger numbers of
problems being attributed to patients in the PDT and PDT+1 groups. There was also a modest
but significant variation in numbers of sessions attended, F(5, 1266) =8.90, p<.001, partial η2
= .034, as patients in PDT averaged a somewhat higher number of sessions than patients in
other groups.
DISCUSSION
These therapies were effective, on average, with effect sizes on the CORE-OM
comparable to effect sizes observed on similar measures in previous efficacy and
effectiveness trials (Barkham et al., 1996; Dobson, 1989; Lipsey & Wilson, 1993; Roth &
Fonagy, 2004; Shapiro et al., 1994; Wampold, 2001; Ward et al., 2000; Westbrook & Kirk,
2005). The mean differences among the three targeted approaches, CBT, PCT, and PDT,
though statistically significant, were small in relation to this substantial change across
treatment. Mean improvement across treatment accounted for approximately 100 times as
CBT-PCT-PDT - 13
much of the variance in CORE-OM scores as did the differential effects of the three
approaches. As Figure 1 shows, the distributions of change scores were largely overlapping,
with little difference among the groups' median scores. All of the treatment groups had
substantial RCSI rates, though there was a nearly-significant trend toward a lower rate in the
PDT group (Table 2). In summary, our results tended to support the Dodo verdict for these
three treatment approaches as practiced routinely across a range of NHS settings.
The diluted treatment groups (CBT+1, PCT+1, and PDT+1) averaged slightly greater
effectiveness than did the undiluted treatment groups in this sample (see Tables 1 and 2),
though the size of this effect was also very small. It is worth remarking, nevertheless, that, at
least across the range of purity represented by the comparisons in this study, greater purity of
CBT, PCT, or PDT did not appear to yield greater effectiveness. Advocates of therapy
integration might describe the "+1" groups as enhanced rather than diluted. There was a great
deal of variation in outcomes within each group (see Figure 1), contrasting with the minor
differences between groups.
Our results extend Ward et al's. (2000) and Barkham et al's. (1996) findings by
showing that the equivalence of these treatments within the NHS may be observed in routine
practice as well as in randomized trials. Our results may be of particular interest to
practitioners of PCT and PDT, insofar as these approaches' comparable effectiveness to CBT
in routine practice may have been unappreciated (cf. Holmes, 2002).
Limitations
Our design's limitations, though common in research on routine treatments (Shadish et
al., 2000; Westbrook & Kirk, 2005), place restrictions on how results can be interpreted.
Limited specification of treatments
We had no independent check on whether or how the therapists delivered the
treatments they indicated and, indeed, no precise definitions of what these treatments
CBT-PCT-PDT - 14
comprised or details of the therapists' qualifications. The observed approximately equivalent
effectiveness could, in principle, reflect a lack of differences in how the treatments were
conducted. Or, systematic failure to implement a treatment correctly could account for its
having failed to prove superior.
On the other hand, we know of no reason why the therapists would have
misrepresented the theoretical approach from which they worked. The most plausible
assumption, we suggest, is that they sincerely sought to implement the approach they
specified. To put it another way, for better or worse, these treatments represented the way
CBT, PCT, and PDT are currently practiced in these settings.
Each of these approaches is based on distinct literature, lore, and experience shared
among its practitioners (e.g., though training, supervision, workshops, and conferences). Each
claims conceptual and technical resources to respond to a wide variety of problems and
circumstances encountered in routine practice. Even in a randomized controlled trial, each
approach must be flexible; even protocol-driven psychotherapies are far from standardised.
More than most medical treatments, psychotherapies must be adapted to the emerging needs
of patients in ways that are not specified in a protocol but depend on the skill and
interpersonal responsiveness of the therapist (Hardy, Stiles, Barkham, & Startup, 1998; Stiles,
Honos-Webb, & Surko, 1998).
Non-random assignment of patients to treatment groups
We did not randomly assign patients to treatments, so we cannot rule out differential
selection as an explanation for our results. In principle, the observed approximate equivalence
could reflect non-random assignment that placed the most difficult patients in the most
effective therapies and vice-versa, compensating for effectiveness differences that would
have otherwise emerged. For example, a relatively larger number of CBT patients were
treated in secondary care settings (Table 3). If secondary care treatment were systematically
CBT-PCT-PDT - 15
more effective than primary care treatment, then this selective assignment could have masked
a greater CBT effectiveness.
On the other hand, there was no statistically significant difference among groups on
the pre-treatment CORE-OM, indicating that the groups began with similar overall levels of
disturbance (Table 1). There were significant differences in the patient mix across groups,
including a relatively larger proportion of males in the CBT group (Table 5), a relatively
larger number of interpersonal problems in the PCT group (Table 4) and a relatively larger
total number of problems in the PDT group (Table 5). If patients with more problems tended
to be assigned to PDT, this might help explain their trend toward a lower RCSI rate and their
tendency to receive more sessions than patients in the other groups (Table 5). Of course, data
on the number and nature of problems were drawn from therapists' reports and thus could
reflect differences in case conceptualization across approaches.
Absence of a control group
Although the pre-post change scores indicated substantial improvement (see Table 1
and Figure 1), there was no comparison group of untreated patients. Consequently, we cannot
attribute the improvement to the treatments with certainty (cf. Corney & Simpson, 2005).
Control groups are often not feasible in routine settings. Ethically, it would be hard to
justify withholding treatment from comparable patients seeking care within the NHS, and
restricting consideration to patients who agreed to be randomised would undermine the
clinical representativeness. In this respect, it is reassuring that the pre-post effect sizes in
these therapies were comparable to those observed in efficacy studies where superiority to
comparison groups was observed (e.g., Ward et al., 2000). Our results may be considered as
complementary to such studies.
Missing data
As is typical in routine practice settings (e.g., Gilbert, Barkham, Richards, &
CBT-PCT-PDT - 16
Cameron, submitted; Greasley, & Small, 2005; Stiles et al., 2003), many patients who
completed a pre-therapy CORE-OM subsequently failed to appear for treatment or failed to
complete a form after treatment. There is evidence that, even among patients who do receive
treatment, those who complete post-treatment measures are (a) more likely to have agreed
with their therapist about when treatment should end (Barkham et al., in press) and (b) likely
to have improved more during treatment (Stiles et al., 2003) than are patients who fail to
complete them. Our results may not generalize to patients who leave treatment without
completing post-treatment measures.
Restriction to a single self-report measure
As a self-report instrument, the CORE-OM does not provide a clinician-rated or
external assessment. In principle, self-report instruments are vulnerable to distortions.
Patients may have exaggerated their distress before treatment or exaggerated their
improvement following treatment. The CORE-OM is a broad spectrum measure and does not
focus on the specific problems. Qualitatively different results of the different treatments
might have shown up on more targeted measures.
Arguably, however, subjective symptoms and distress define the need for
psychotherapeutic treatment in most cases; in this sense, self-report could be considered as a
criterion. We note that the CORE-OM is highly correlated with other self-report and clinician
rated measures that have been widely used to assess psychotherapy outcome (Leach et al., in
press-a, in press-b; Cahill et al., submitted; Evans et al., 2002). It would, of course, be useful
and informative to additionally examine external measures of functioning in future research.
Investigator allegiance
In a review of psychotherapy outcome studies, Luborsky et al. (1999) reported that
the investigator's allegiance, assessed by ratings of previous publications, ratings by
colleagues, and self ratings, was strongly correlated with the outcomes of the treatments in
CBT-PCT-PDT - 17
published reports (r = .85, p<001, n = 29 studies). The present study's first author has
published papers dealing with the equivalence paradox, so in a sense this report fits the
pattern. He, along with the fifth author, has a particular interest in the PCT approach. The
second author is an accredited CBT therapist and has delivered manualised versions of both
CBT and PDT in randomized efficacy studies. The third and fourth authors are not qualified
therapists and profess no differential allegiance to any one of these three approaches
Responsiveness and the Dodo verdict
Equivalent outcomes are often ascribed to common factors in the relationship, such as
the alliance, empathy, and collaborative involvement (Norcross, 2002). There may also be
self-regulatory processes that cut across treatment approaches, as therapists and patients
respond to optimize gains, given limited time and resources, within each approach's
framework (Stiles et al., 1998). For example, participants appear to regulate the dose of
therapy to achieve a good enough level of outcome--in terms of number of sessions, if that
choice is available (Barkham et al., in press), or in terms of intensity of process, if the number
of sessions is fixed (Reynolds et al., 1996). This logic of appropriate responsiveness does not
argue against the theoretical accounts. Instead, it suggests that all provide concepts and
techniques that the participants use to achieve what they seek or what they will settle for,
given the context, resources, and constraints.
CBT-PCT-PDT - 18
REFERENCES
Barkham, M., Connell, J., Stiles, W. B., Miles, J. N. V., Margison, F., Evans, C., &
Mellor-Clark, J. (in press). Dose-effect relations and responsive regulation of
treatment duration: The good enough level. Journal of Consulting and Clinical
Psychology.
Barkham, M., Evans, C., Margison, F., McGrath, G., Mellor-Clark, J., Milne, D. &
Connell, J. (1998). The rationale for developing and implementing core outcome
batteries for routine use in service settings and psychotherapy outcome research.
Journal of Mental Health, 7, 35-47.
Barkham, M., Margison, F., Leach, C., Lucock, M., Mellor-Clark, J., Evans, C., Benson,
L., Connell, J., Audin, K. & McGrath, G. (2001). Service profiling and outcomes
benchmarking using the CORE-OM: Towards practice-based evidence in the
psychological therapies. Journal of Consulting and Clinical Psychology, 69, 184-196.
Barkham M, Gilbert, N., Connell, J., Marshall, C. & Twigg, E. (2005). Suitability and
utility of the CORE-OM and CORE-A for assessing severity of presenting problems
in psychological therapy services based in primary and secondary care settings.
British Journal of Psychiatry, 186, 239-246.
Barkham, M., Rees, A., Shapiro, D. A., Stiles, W. B., Agnew, R. M., Halstead, J.,
Culverwell, A., & Harrington, V. M. G. (1996). Outcomes of time-limited
psychotherapy in applied settings: Replicating the Second Sheffield Psychotherapy
Project. Journal of Consulting and Clinical Psychology, 64, 1079-1085.
Beutler, L. E. (1991). Have all won and must all have prizes? Revisiting Luborsky et al.'s
verdict. Journal of Consulting and Clinical Psychology, 59, 226-232.
Cahill, J., Barkham, M., Stiles, W. B., Twigg, E., Hardy, G. E., Rees, A., & Evans, C.
(submitted). Convergent validity of the CORE Measures with measures of depression
CBT-PCT-PDT - 19
for clients in cognitive therapy for depression. Manuscript submitted for publication.
Chambless, D. L., Baker, M. J., Baucom, D. H., Beutler, L. E., Calhoun, K. S., CritsChristoph, P., Daiuto, A., DeRubeis, R., Detweiler, J., Haaga, D. A. F., Johnson,
S. B., McCurry, S., Mueser, K. T., Pope, K. S., Sanderson, W. C., Shoham, V.,
Stickle, T., Williams, D. A., & Woody, S. R. (1998). Update on empirically
validated therapies, II. The Clinical Psychologist, 51, 3–16.
Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported therapies.
Journal of Consulting and Clinical Psychology, 66, 7-18.
Connell, J., Barkham, M., Stiles, W. B., Twigg, E., Singleton, N., & Evans, O., Miles, J.
N. V. (submitted). Distribution of CORE-OM scores in a general population, clinical
cut-off points, and comparison with the CIS-R. Manuscript submitted for publication.
Corney, R., & Simpson, S. (2005). Thirty-six month outcome data from a trial of
counselling with chronically depressed patients in a general practice setting.
Psychology and Psychotherapy: Theory, Research and Practice, 78, 127-138.
Dobson K. (1989). A meta-analysis of the efficacy of cognitive therapy for depression.
Journal of Consulting and Clinical Psychology, 57, 414-420.
Elliott, R., Greenberg, L. S., & Lietaer, G. (2004). Research on experiential
psychotherapies. In M. J. Lambert, (Ed), Bergin and Garfield's handbook of
psychotherapy and behavior change (5th ed.), New York: Wiley.
Elliott, R., Stiles, W. B., & Shapiro, D. A. (1993). "Are some psychotherapies more
equivalent than others?" In T. R. Giles (Ed.), Handbook of effective psychotherapy
(pp. 455-479). New York: Plenum Press.
Evans, C., Connell, J., Barkham, M., Margison, F., Mellor-Clark, J., McGrath, G. &
Audin, K. (2002). Towards a standardised brief outcome measure: Psychometric
properties and utility of the CORE-OM. British Journal of Psychiatry, 180, 51-60.
CBT-PCT-PDT - 20
Evans, C., Mellor-Clark, J., Margison, F., Barkham, M., Audin, K., Connell, J. &
McGrath, G. (2000). CORE: Clinical outcomes in routine evaluation. Journal of
Mental Health, 9, 247-255.
Feltham, C., & Horton, I. E. (Eds.) (2006). Handbook of counselling and psychotherapy,
second edition. London: Sage.
Gabbard, G. O., Beck, J. S., & Holmes J. (Eds.) (2005). Oxford textbook of psychotherapy.
Oxford, UK: Oxford University Press.
Gilbert, N., Barkham, M., Richards, A., & Cameron, I. (submitted). The effectiveness of a
primary care mental health service delivering brief psychological interventions: A
benchmarking study using the CORE System. Manuscript submitted for publication.
Goldman, R. N., Greenberg, L. S., & Angus, L. (in press). The effects of adding emotionfocused interventions to the therapeutic relationship in the treatment of depression.
Psychotherapy Research.
Greasley, P., & Small, N. (2005). Evaluating a primary care counseling service: Outcomes
and issues. Primary Care Research and Development, 6, 124-136.
Greenberg, L. S. & Watson, J. C. (1998). Experiential therapy of depression: Differential
effects of client-centered relationship conditions and process experiential
interventions. Psychotherapy-Research, 8(2), 210-224.
Hardy, G. E., Stiles, W. B., Barkham, M., & Startup, M. (1998). Therapist responsiveness
to client interpersonal styles during time-limited treatments for depression. Journal of
Consulting and Clinical Psychology, 66, 304-312.
Hollon, S. D., Thase, M. E., & Markowitz, J. C. (2002). Treatment and prevention of
depression. Psychological Science in the Public Interest, 3, 39-77
Hollon, S. D., & Beck, A. T. (2004). Cognitive and cognitive behavioral therapies. In M. J.
Lambert (Ed.), Garfield and Bergin's Handbook of psychotherapy and behavior
CBT-PCT-PDT - 21
change (5th ed., pp 447-492). New York: Wiley.
Holmes, J. (2002). All you need is cognitive-behaviour therapy? BMJ, 324, 288-290.
Leach, C., Lucock, M., Barkham, M., Noble, R., Clarke, L. & Iveson, S. (in press-a).
Assessing risk and emotional disturbance using the CORE-OM & HoNOS outcome
measures. Psychiatric Bulletin.
Leach, C., Lucock, M., Barkham, M., Stiles, W. B., Noble, R, & Iveson, S. (in press-b).
Transforming between Beck Depression Inventory and CORE-OM scores in routine
clinical practice. British Journal of Clinical Psychology.
Leichsenring, F. (2001). Comparative effects of short-term psychodynamic psychotherapy
and cognitive-behavioral therapy in depression: a meta-analytic approach. Clinical
Psychology Review, 21, 401-419.
Leichsenring, F., & Leibing, E. (2003). The effectiveness of psychodynamic therapy and
cognitive behavior therapy in the treatment of personality disorders: a meta-analysis.
American Journal of Psychiatry, 160, 1223-1232.
Leichsenring F., Rabung, S., & Leibing E. (2004). The efficacy of short-term
psychodynamic psychotherapy in specific psychiatric disorders: a meta-analysis.
Archives of General Psychiatry, 6, 1208-16.
Lipsey, M. W., & Wilson, D. B. (1993). The efficacy of psychological, educational, and
behavioral treatment: Confirmation from meta-analysis. American Psychologist, 48,
1181-1209.
Luborsky, L., Diguer, L., Seligman, D. A., Rosenthal, R., Krause, E. D., Johnson, S.,
Halperin, G., Bishop, M., Berman, J. S., & Schweizer, E. (1999). The researcher's
own therapy allegiances: A ''wild card'' in comparisons of treatment efficacy. Clinical
Psychology: Science and Practice, 6, 95-106.
Luborsky, L., Singer, B., & Luborsky, L. (1975). Comparative studies of psychotherapies:
CBT-PCT-PDT - 22
Is it true that "Everyone has won and all must have prizes"? Archives of General
Psychiatry, 32, 995-1008.
Mellor-Clark, J. & Barkham, M. (2000) Quality evaluation: methods, measures and
meaning. In C. Feltham & I. Horton (eds) Handbook of counselling and
psychotherapy (pp. 225-270). London: Sage Publications.
Mellor-Clark, J., Barkham, M., Connell, J. and Evans, C. (1999). Practice-based evidence
and standardized evaluation: informing the design of the CORE system. European
Journal of Psychotherapy, Counselling and Health, 2, 357-374
Norcross, J. C. (1995). Dispelling the dodo bird verdict and the exclusivity myth in
psychotherapy. Psychotherapy, 32, 500-504.
Norcross, J. C. (Ed.) (2002). Psychotherapy relationships that work: Therapist contributions
and responsiveness to patient needs. New York: Oxford University Press.
Reynolds, S., Stiles, W. B., Barkham, M., Shapiro, D. A., Hardy, G. E., & Rees, A.
(1996). Acceleration of changes in session impact during contrasting time-limited
psychotherapies. Journal of Consulting and Clinical Psychology, 64, 577-586.
Roth, A., & Fonagy, P. (2004). What works for whom? A critical review of psychotherapy
research (second edition). New York: Guilford Publications, 2004.
Rosenzweig, S. (1936). Some implicit common factors in diverse methods of psychotherapy.
American Journal of Orthopsychiatry, 6, 412-415.
Seligman, M. (1995). The effectiveness of psychotherapy. American Psychologist, 50, 965974.
Shadish, W. R., Navarro, A. M., Matt, G. E., & Phillips, G. (2000). The effects of
psychological therapies under clinically representative conditions: a meta-analysis.
Psychological Bulletin, 126, 512-529.
Stiles, W. B., Honos-Webb, L., & Surko, M. (1998). Responsiveness in psychotherapy.
CBT-PCT-PDT - 23
Clinical Psychology: Science and Practice, 5, 439-458.
Stiles, W. B., Leach, C., Barkham, M., Lucock, M., Iveson, S., Shapiro, D. A., Iveson,
M., & Hardy, G. (2003). Early sudden gains in psychotherapy under routine clinic
conditions: Practice-based evidence. Journal of Consulting and Clinical Psychology,
71, 14-21.
Stiles, W. B., Shapiro, D. A., & Elliott, R. (1986). "Are all psychotherapies equivalent?"
American Psychologist, 41, 165 180.
Stirman, S. W., DeRubeis, R. J., Crits-Christoph, P., & Brody, P. E. (2003). Are samples
in randomized controlled trials of psychotherapy representative of community
outpatients? A new methodology and initial findings. Journal of Consulting and
Clinical Psychology, 71, 963-972.
Street, L., Niederehe, G., & Lebowitz, B. (2000). Toward greater public health relevance
for psychotherapeutic intervention research: An NIMH workshop report. Clinical
Psychology: Science and Practice, 7, 127-137.
Tarrier, N. (2002). Yes, cognitive behavioural therapy may well be all you need. BMJ, 324,
291-292.
Wampold, B. E. (2001). The great psychotherapy debate: Models, methods, and findings.
Mahwah, NJ: Lawrence Erlbaum Associates.
Ward, E., King, M., Lloyd, M., Bower, P., Sibbald, B., Farrelly, S., Gabbay, M., Tarrier,
N., & Addington-Hall, J. (2000). Randomised controlled trial of non-directive
counselling, cognitive-behaviour therapy, and usual general practitioner care for
patients with depression. I: Clinical effectiveness. BMJ, 321, 1383 - 1388.
Westbrook, D., & Kirk, J. (2005). The clinical effectiveness of cognitive behaviour therapy:
Outcome for a large sample of adults treated in routine practice. Behaviour Research
and Therapy, 43, 1243-1261.
CBT-PCT-PDT - 24
AUTHOR NOTES
Authors affiliated to the Psychological Therapies Research Centre, University of Leeds, were
supported by the Priorities and Needs Research and Development Levy from Leeds
Community Mental Health & Teaching Trust.
CBT-PCT-PDT - 25
Table 1. CORE-OM clinical scores for treatment groups: Pre- and post-therapy means,
differences and confidence intervals for the differences
Treatment
n
Pre-therapy
Post-therapy
Pre-Post Difference
Mean
sd
Mean
sd
Mean
sd
95% CI Effect size
group
CBT
298
16.9
7.0
8.1
6.4
8.9
7.1
8.1-9.7
1.27
PCT
332
17.6
6.6
8.9
6.1
8.7
6.6
8.0-9.4
1.32
PDT
122
17.6
6.3
9.9
6.8
7.7
6.4
6.5-8.8
1.23
CBT+1
181
17.9
6.3
7.9
5.6
10.0
6.5
9.1-11.0
1.59
PCT+1
249
17.3
6.4
7.8
6.3
9.5
6.9
8.6-10.3
1.48
PDT+1
127
17.3
5.9
9.1
6.3
8.2
6.9
7.0-9.4
1.38
Note: CBT = cognitive, behavioural, or cognitive/behavioural therapy; PCT = person-centred
therapy; PDT = psychodynamic/psychoanalytic therapy; CBT+1 = CBT combined with one
other therapy; PCT+1 = PCT combined with one other therapy; PDT+1 = PDT combined
with one other therapy; CI = confidence interval. Effect size calculated as the mean
difference divided by the pre-therapy standard deviation.
CBT-PCT-PDT - 26
Table 2. Reliable and clinically significant improvement in treatment groups
Treatment
RCSI
group
Reliable
No reliable change
Deterioration
improvement
Count
Row %
Count
Row %
Count
Row %
Count
Row %
CBT
152
62.0
46
18.8
44
18.0
3
1.2
PCT
167
57.6
63
21.7
55
19.0
5
1.7
PDT
51
48.1
23
21.7
31
29.2
1
0.9
CBT+1
116
70.3
28
17.0
18
10.9
3
1.8
PCT+1
136
64.5
35
16.6
39
18.5
1
0.5
PDT+1
71
63.4
17
15.2
23
20.5
1
0.9
Note: n = 1129 patients whose pre-therapy CORE-OM scores were at or above the
recommended cutoff of 10 (Connell et al., submitted). RCSI = reliable and clinically
significant improvement. CBT = cognitive, behavioural, or cognitive/behavioural therapy;
PCT = person-centred therapy; PDT = psychodynamic/psychoanalytic therapy; CBT+1 =
CBT combined with one other therapy; PCT+1 = PCT combined with one other therapy;
PDT+1 = PDT combined with one other therapy.
CBT-PCT-PDT - 27
Table 3. Distribution of treatments across types of services
Type of service
Treatment group
CBT
PCT
PDT
CBT+1
PCT+1
PDT+1
Primary Care Counselling
83
298
91
120
149
103
Secondary Care/Psychology
53
0
13
17
0
4
Tertiary Care/Specialist
11
0
13
17
1
1
Primary Care Psychology/
90
21
2
3
12
0
61
13
3
24
87
19
Secondary Care in Primary Setting
Psychology & Counselling
Note. CBT = cognitive, behavioural, or cognitive/behavioural therapy; PCT = person-centred
therapy; PDT = psychodynamic/psychoanalytic therapy; CBT+1 = CBT combined with one
other therapy; PCT+1 = PCT combined with one other therapy; PDT+1 = PDT combined
with one other therapy.
CBT-PCT-PDT - 28
Table 4. Allocated therapy by presenting problem
Presenting problem
Treatment group
CBT
PCT
PDT
CBT+1
PCT+1
PDT+1
Anxiety
252
236
95
144
194
109
Depression
200
220
91
126
190
107
Interpersonal Problems
99
169
75
90
147
86
Self Esteem
120
132
54
89
129
74
Bereavement/Loss
34
119
50
49
81
51
Work/Academic
51
66
30
45
54
31
Trauma/Abuse
42
51
33
33
45
25
Physical Problems
32
65
15
35
47
24
Living/Welfare
27
40
10
32
29
14
Addictions
12
11
10
6
15
2
Personality Problems
10
2
5
8
6
2
Eating Disorder
7
9
5
3
7
2
Cognitive/Learning
2
8
2
2
4
1
Psychosis
4
3
3
0
1
1
Other
9
10
1
6
4
8
Note: CBT = cognitive, behavioural, or cognitive/behavioural therapy; PCT = person-centred
therapy; PDT = psychodynamic/psychoanalytic therapy; CBT+1 = CBT combined with one
other therapy; PCT+1 = PCT combined with one other therapy; PDT+1 = PDT combined
with one other therapy. Columns add to more than the number of patients in each treatment
because therapists indicated multiple problems for some patients..
CBT-PCT-PDT - 29
Table 5. Demographics and mean numbers of presenting problems and sessions attended by
treatment groups
Treatment
Percent
group
male
CBT
40.9
PCT
Mean age
Mean of number of
Mean number of sessions
presenting problems
attended
41.1
3.13
6.52
22.9
40.4
3.52
6.11
PDT
25.4
40.9
4.20
8.53
CBT+1
28.4
39.1
3.80
6.03
PCT+1
29.3
39.7
3.86
5.62
PDT+1
23.6
40.6
4.23
6.11
Note. CBT = cognitive, behavioural, or cognitive/behavioural therapy; PCT = person-centred
therapy; PDT = psychodynamic/psychoanalytic therapy; CBT+1 = CBT combined with one
other therapy; PCT+1 = PCT combined with one other therapy; PDT+1 = PDT combined
with one other therapy.
CBT-PCT-PDT - 30
Figure Caption
Figure 1. Notched box plots showing pre-post differences in CORE-OM clinical scores. The
notch shows the 95% confidence interval around the median. The boxes show the middle
50% of the distribution. The whiskers show the range, except that observations falling 1.5
times the interquartile range or more away from the top or bottom of the box are considered
outliers and are shown separately. CBT = cognitive, behavioural, or cognitive/behavioural
therapy; PCT = person-centred therapy; PDT = psychodynamic/psychoanalytic therapy;
CBT+1 = CBT combined with one other therapy; PCT+1 = PCT combined with one other
therapy; PDT+1 = PDT combined with one other therapy.
CBT-PCT-PDT - 31
Pre-Post difference in CORE-OM Clinical Score
40
20
0
-20
-40
CBT
PCT
PDT
CBT + 1
Treatment group
PCT+1
PDT+1