The Philosophy and Practice of
Feedback: Using CORE OM as
a Clinical Tool
Dr James Macdonald
Director of Clinical Training
CORE IMS
The long and winding road
Using Session by
Session Measures
How have you
been this week?
Um. I think a bit
better. I haven’t
been sleeping
very well
Can you tell me a
bit more about
your sleep?
Well, I’ve had
some horrible
dreams recently
Creativity and Measurement
• Creativity has been defined as the development of original
ideas that have value (Sir Ken Robinson, 2011)
• In small groups (e.g. 4/5 people) discuss creative ways in which
you might use formal measurement (e.g. outcome measures)
to benefit your clients (10 minutes)
Measurement and Creativity in Medicine – an example
•
•
•
•
In the 1950s one in thirty newborns died at birth. Dr Virginia Apgar [d. 1974] was an anaesthetist who was
appalled by the care that many newborns received. Babies who were born malformed or too small or just blue
and not breathing well were listed as stillborn, placed out of sight and left to die. She developed a scale, the
Apgar score, as it became universally known, which allowed nurses to rate the condition of babies at birth on a
scale from zero to ten. An infant got two points if it was pink all over, two for crying, two for taking good
vigorous breaths, two for moving all four limbs, and two if its heart rate was over a hundred. Ten points meant a
child born in perfect condition. Four points or less meant a blue, limp baby.
Published in 1953 to revolutionary effect, the score turned an intangible and impressionistic clinical concept –
the condition of new babies – into numbers that people could collect and compare. Using it required more
careful observation and documentation of the true condition of every baby. Moreover, even if only because
doctors are competitive, it drove them to want to produce better scores – and therefore better outcomes – for
the newborns they delivered.
Around the world, virtually every child born in a hospital came to have an Apgar score recorded at one minute
after birth and at five minutes after birth. It quickly became clear that a baby with a terrible Apgar score at one
minute could often be resuscitated – with measures like oxygen and warming – to an excellent score in five
minutes. Neonatal intensive care units sprang into existence. The score also began to alter how childbirth itself
was managed. Spinal and then epidural anesthesia were found to produce babies with better scores than
general anesthesia. Prenatal ultrasound came into use to detect problems for deliveries in advance. Fetal heart
monitors became standard. Over the years, hundreds of adjustments and innovations were made, resulting in
what’s sometimes called “the obstetrics package”. And that package has produced dramatic results. In the
United States today, a full-term baby dies in just one childbirth out of five hundred, and a mother dies in less
than one in ten thousand. If the statistics of the 1930s had persisted, 27,000 mothers would have died last year
(instead of fewer than five hundred) – and 160,000 newborns (instead of one-eighth that number).
From: Atul Gwande (2008). Better: A Surgeon’s Notes on Performance. London: Profile Books. P169ff, mainly p 186-188. (One page).
Discussion
• What do questionnaires do well?
• What do people do well?
• What thoughts did you have in your groups about the creative
use of measures in therapy?
Changing paradigm…?
Models
Techniques
Therapist Objectivity
Client response
Client individuality
Client agency
Therapy relationship
Therapist responsiveness
‘Meta-communication’
Process-Outcome Research
• Orlinsky, Rønnestad & Willutzki (2004); (see also
Bohart & Wade, 2013; Norcross et al, 2011)
major review of process-outcome research:
‘quality of patients’ participation [emerges] as
the most important determinant of outcome
(p324)’ [more than therapist attitudes,
behaviours or techniques]
– Characterised as ‘active shoppers’ vs ‘passive
recipients’
– Collaborative clients do better
– Resistance to therapeutic processes predict poor
outcomes (openness predicts good)
– Zuroff et al (2007) autonomous motivation
Bohart, A. & Wade, A. (2013). The Client in Psychotherapy. In M. Lambert
[ed.] Bergin and Garfield’s Handbook of Psychotherapy and Behavior
Change (Sixth Edition), p 219-257. Hoboken, NJ: John Wiley & Sons.
‘Clients are not passive recipients of treatment like patients in
surgery. Rather, they actively intersect with what therapists have
to offer (even if that ‘activity’ sometimes consists of adopting a
passive or resistant stance). How they learn involves their degree
of involvement, their resonance with therapists and methods,
how much effort they put in, their own creativity, and how they
interpret and implement the input they receive.’ (p220)
A
Safety
Net
Therapist Blindsidedness
Hannan et al (2005); Chapman et al (2012):
Prediction of deterioration
Hatfield et al (2010): Noticing deterioration
Walfish et al (2009); Dew & Reimer (2003): Selfassessment bias
The Cognitive Science of Mistakes
What
Base rate fallacy
You
See
Attentional bias
Is All
There
Anchoring bias
Is
Confirmation bias
Availability heuristic
"all the women are strong, all the men
are good looking, and all the children are
above average"
Should clients and services rely on clinician self-assessment?
My
Outcomes Are
Great!
Trust me –
I’m a
psychotherapist
Why is assessment of our own performance so
difficult in psychotherapy?
• Highly complex ‘low validity’ Environments
(Kahnemann, 2011; Kahneman & Klein,
2009)
• Self-assessment bias
• Client deference and opaque mental states
(e.g. Regan & Hill, 1992; Fonagy et al 2003)
Therapist Blindsidedness: Small Groups Discussion
• Discuss the findings about the limitations of therapists’
knowledge about how therapies progress.
• Reflecting back to your creative ideas for the use of measures
discuss how you think questionnaires could be used to mirror
information to therapists that may be lacking due to the
blindspots we have discussed.
• 10 minutes
Using CORE Net to Improve Outcomes
• Introduction to the system
• Introducing clients to feedback informed therapy
• Problem solving with off track clients
– Ruptures
– Motivation
– Life events
CORE System Tools for Feedback-led Therapy
Copyright: CORE IMS Ltd
Highlighting more problematic items…
A tracking graph…
‘Flags’
Introducing feedback…
• What do you think might be the key elements in introducing
session by session monitoring to a couple at the beginning of
therapy?
• Themes: What is involved….Why you are suggesting it ….
How do they feel about it?
Introducing Feedback
The message to your client:
‘Your input is crucial; your participation
matters. We invite you to be a partner in
your care. We respect what you have to say,
so much so that we will modify the
treatment to see that you get what you
want.’ (Miller, Hubble, Duncan & Wampold,
2010)
In the beginning: Introducing
session by session monitoring
• Role Play: In pairs role play
introducing session by session
monitoring to a new client.
• (10 mins)
Principles for Working with Off-Track Cases
Systematic Use of Questionnaires to Spot
Problems
Awareness of Solutions to Common Problems
Problem Solving Dialogue
Using
CORE Net
with
Off-Track
Cases
Working with Off-Track Cases…
Key Processes
Linked to
Outcomes?
The Therapy
Relationship
(Bond, Goal & Task)
Client
Motivation/
Understanding
of Therapy
Adverse Life Events….
Lack of Social Support
Perfectionism/
Harsh selfcriticism
Something missed in the formulation
Eg. Substance misuse.
Mike Lambert: Clinical Support Tools
Using a process measure
ARM 5
HAT (Helpful
Aspects of
Therapy) Form
Scenario 1: No Change – Problem in the Therapeutic
Relationship
Score on the ARM 5 (relationship measure)
ANSWER
QUESTION
7. Strongly agree
1. My therapist is supportive
6. Moderately agree
2. My therapist and I agree about how to
work together
3. Slightly agree
3. My therapist and I have difficulty
working as a partnership
7. Strongly agree
4. I have confidence in my therapist and
his/her techniques
4. Neutral
5. My therapist is confident in him/herself
and his/her techniques
ARM score 5.4 (Max score is 7)
Therapeutic orientation to ruptures
• Genuine and gentle Enquiry into how the client has experienced the
relationship with you in the session (leading to exploration of the client’s
‘construal’ of the therapeutic relationship)
• Empathy with client’s experience in the session (‘no wonder you felt angry
if you thought I was judging you like that’)
• ‘Meta-communication’ (e.g. ‘it seems to me at the moment as though
we’re in a dance where I’m the cat and you’re the mouse and each time I
comment on how I am understanding your experience, you say ‘no, that’s
not it’ and move away, as if you fear being ‘caught’. Is that anything like
how you are experiencing being with me at the moment?’ ‘Wrecking ball’)
• Taking responsibility for one’s own part in any rupture (e.g. ‘yes, thinking
back on it, I think I may have been pursuing you and trying to pin you down
somehow. I think I might have felt some frustration that we could not reach
a shared understanding of what was happening for you, but I understand
now how I came across in a way which was frightening and made you pull
back more. I’m really sorry to have put you more in that position. Thank you
for discussing this with me’)
Summary of common rupture repair
interventions (Safran et al, 2011)
1.
2.
3.
4.
5.
6.
Repeating the therapeutic rationale
Changing Task or Goals
Clarifying misunderstandings
Exploring relational themes associated with the rupture
Linking the rupture to common patterns in client’s life
New Relational Experience
Role Play
• Take it in turns to role play a recent client of yours whose
failure to improve appears to be connected to problems in
the therapeutic relationship. Imagine like Lil he/she has not
improved on CORE 10 and that his/her ARM score shows
that he/she is having some difficulties collaborating or
agreeing with you in therapy
• As therapist, aim first to empathically understand the
client’s experience of the rupture with you. You can then
take the conversation in any of the directions suggested by
Safran et al for resolving ruptures.
• As observer, note strategies used by the therapist and
impact on the client
• Discussion: what did you learn from this? What was
difficult? What seemed to work?
Scenario 2: No Change – Problem with client
motivation
Stages of Change
• Precontemplation
– 4 “Rs” Reluctance; Rebellion; Resignation; Rationalisation
• Contemplation
– Ambivalence
• Preparation
• Action
• Maintenance
(Prochaska & Norcross)
Assessing motivational problems
• “Honestly, I really don’t understand what I can get from
therapy”
• “I am not really sure what to work on in therapy”
• “I had thoughts about quitting therapy; it’s just not for me”
• “Although I am currently unhappy with life, there is nothing I
can do about it”
• “I am in therapy because someone is requiring it of me”
From Lambert et al ‘Clinical Support Tool’
Motivational Interviewing
• Express Empathy
– Understanding client’s frame of reference
– Accepting ambivalence about change as normal
– Therapist reflection ‘a guess as to what client means’
• Develop Discrepancy
– Make explicit values and discrepancy with current behaviour
• Roll with Resistance
– Normalise/accept resistance
– Explore and deepen awareness of pros and cons from client perspective
• Support Self-Efficacy
Miller & Rollnick 2002
Role Play
• Take it in turns to role play a recent client of yours whose failure to
improve appears to be connected to ambivalence about working in
psychotherapy. Imagine like Lil he/she has not improved on CORE 10.
• Take it in turns to role play as therapist, using open-ended questions and
empathic summaries/reflections, etc, to help the client articulate and
deepen their understanding of their ambivalence and discrepancies
between their values and their current behaviour
• Observer: note strategies used by therapist and impact on client
• Discussion: what did you learn from this? What was difficult? What
seemed to work?
Other scenarios associated with lack of change 1:
Social Support
• Client Lacks Social Support (Lambert CST)
– Assess client’s social network
– Refer for group therapy
– Role play to develop social skills
– Bring significant others to session
– Specific anxiety reduction practices for social anxiety
– Encourage participation in positive social activities
Other scenarios associated with a lack of change 2:
Life events; Diagnosis
1. Life Events
Divorce, illness, stress at work, housing problems, relationship conflict
Interact with other client variables
Explore nature of stressor and client coping style
2. Diagnosis
e.g. substance misuse, unrecognised bipolar disorder
Consider medication as an adjunct to psychotherapy
Scenario 5: Client deterioration
‘Learning to Fail Successfully’ (Duncan, 2010)
1) Discuss possible reasons (follow procedures above)
2) Review client’s experience of therapeutic relationship (ARM)
and experience of the sessions
3) Supervision
4) Consider changing approach
5) Consider changing therapist
6) Duncan (2010) ‘Checkpoint Conversations’ (3rd or 4th session)
and ‘Last Chance Discussions’ (6th or 7th session).
Experience of the Workshop and Next Steps
• What do you think you will be able to take from this afternoon’s
workshop?
• What was most surprising to you?
• What still seems unclear or difficult to you?
• How do you see yourselves taking the next steps with CORE Net?
Further reading… [james@coreims.co.uk]
http://www.headingtonpsychotherapy.co.uk/downloads/FeedbackasPsychoanalyticTechniqueMacdonald2013.pdf
Basics of using feedback
• Reinforcing ‘on track’ work
• Exploring ‘lack of change’. Considering
– Motivation
– Therapeutic relationship
– Unacknowledged substance abuse
– Lack of social support
– Stressful life events
• Acting in the case of deterioration
– ‘Failing successfully’
How do you know
how you do?
How do you know how you do?
How easy is it to know how you are
doing as a therapist?
What gets in the way of knowing
how you are doing?
In the Fog of Complexity, Mirages Abound
• Expert prediction and intuition are notoriously
problematic in highly complex situations
(Kahnemann, 2011; Kahnemann & Klein, 2009)
• In such complex environments, where
straightforward feedback is lacking, assessment
of one’s own performance may be more subject
to self-assessment biases
Paul Meehl and the Fog of Therapy
Clinical psychologist,
Academic and
Psychoanalyst
Grove & Meehl (1996)
• meta-analysis of studies
comparing actuarial versus
clinical predictions. ‘Of the 136
studies, 64 favored the actuary
…, 64 showed approximately
equivalent accuracy, and 8
favoured the clinicians’ (p298).
Ægisdóttir et al. (2006)
• The Meta-Analysis of Clinical Judgement Project: Fifty Six Years of
Accumulated Research on Clinical vs Statistical Prediction
• Clinical decisions are accurate 47% of the time, whereas statistical
decisions are accurate 53% of the time p 359.
• 13% increase in accuracy with statistical predictions
• More information to clinicians resulted in less accuracy
‘High Validity environments’
• Expert intuition can be relied on in ‘high validity’
environments
– in which there are environmental regularities
which provide cues to the situation,
– and where the expert has had adequate
opportunity to learn these regularities.
– Examples of high validity environments include:
chess, fire fighting and aspects of nursing
‘Low validity’ environments
• Expert intuition is suspect in ‘low validity’ environments
– These are highly complex and irregular environments, which
don’t provide reliable predictive cues.
– Examples of low validity environments include: medium to
long-term political forecasting, stock-market predictions,
personnel selection and longer-term clinical judgements,
– In these situations, statistical algorithms often outperform
professional judgements (Kahneman & Klein, 2009; Grove &
Meehl, 1996).