Session by Session Monitoring of
Outcome and Alliance
David C. Low
Family & Systemic Psychotherapist
Feedback Informed Treatment (ORS & SRS) Trainer
david.low1964@gmail.com
Tel. 01473-280416
Aim of the Presentation
• Provide introduction to the research and
clinical rational for Session by Session
monitoring of outcome and alliance.
• Consider how session by session
measuring fits with systemic thinking and
practice.
• Introduce the Outcome Rating Scale (ORS)
and Session Rating Scale (SRS) and how
they can be used with families.
• Introduce how the measures can be used to
bring the clients’ voice into clinical
supervision.
“So What?”
5 Minute Exercise
• Get in small groups of 3.
• Introduce yourselves.
• Choose someone to feedback themes to the rest
of us.
• Very briefly discuss the relevance of session by
session measuring has to each of you, and why
any part of it would be important to you.
The Turn to “Outcome”
• “Outcome,” “Evidence,”
“Effectiveness”
&“Accountability” are the
watchwords of the day.
• Part of a world wide
discourse and trend and not
specific to mental health.
What Works in Therapy: the Tale of
Two Paradigms
• The Medical Model Evidence Based
Practice Paradigm:
– Diagnosis + RCT Tx = Good Outcome?
– NICE and PBR “What Works for Whom.”
To
• The Contextual Model Practice Based
Evidence Paradigm:
– Is this relationship between this client and this
therapist working for this young person/family at
this time and place?
– i.e. Session by Session Measuring
Session by Session Monitoring
the Evidence
• Currently, 13 RCT’s involving 12,374 clinically,
culturally, and economically diverse clients
provides evidence that routine monitoring and
feedback
– As much as doubles treatment “effect size”;
– Decreases drop-out rates by as much as
half;
– Decreases deterioration by 33%;
– Reduces hospitalizations and shortened
length of stay by 66%;
– Significantly reduced cost of care (nonfeedback groups increased).
Miller, S.D. (2010). Psychometrics of the ORS and SRS. Results from RCT’s and meta-analyses of routine outcome
monitoring and feedback: The available evidence. http://www.scottdmiller.com/?q=blog/1&page=2.
How Does Session by Session Measures Fit
with Systemic Thinking?
• Cybernetics from Greek kubernētēs steersman,
from kubernan to steer, control
• Cybernetics concerned with self-regulating systems
and how their stability and functioning were
maintained through mechanisms of information,
control and feedback.
• A key notion of cybernetic is the principle of
feedback– how information about the results or
consequences of a previous action return or loop
back into the system in order regulate subsequent
sessions.
• Session by session, ultra brief, reliable and valid
measures provide feedback into the therapeutic
system to inform when we are on track and when we
are off track and need to adjust or do something
different in order to have a positive outcome.
Why do I need Formal Measures
as I Already ask for Feedback?
• Routine tracking of client progress using
standardized measures is especially important
given clinicians’ tendencies to:
– be overly optimistic about the meaning of
clients’ lack of progress
– their over optimism of eventual
improvement in the face of slow progress,
– and their failure to judge when clients are
headed toward a negative outcome.
Lambert, M.J. (2010) Prevention of treatment failure: The use of measuring, monitoring,
and feedback in clinical practice. Washington, DC: American Psychological Association.
Self-Assessment Bias:
Clinicians Have an Overly Positive Assessment of
Personal Performance
• 25% of mental health professionals viewed their skill to be at
the 90th percentile when compared to their peers
•
Most thought they were above average.
• None viewed themselves as below average.
• Clinicians tended to overestimate their rates of client
improvement, and underestimate their rates of client
deterioration.
• Using session by session measures keeps us grounded,
and improving our performance and outcomes.
Walfish, S, McAlister, B., O'Donnell, P., & Lambert, M.J. (2012)An Investigation of Self-assessment Bias in Mental Health Providers.
Psychological Reports: Volume 110, Issue, pp639-644.
Benefits of Session by Session
Measuring
• Puts the client’s voice at
center stage
• Allows services to be client
and family driven
• Brings consumers into the
inner circle of decisions
• Partners in monitoring the
benefit and fit of services
• Collaboration, the Next
Generation: Going boldly
where we have not gone
before.
Ok, But Every Session?
• The evidence demonstrates that frequent
measurement, leads to improved
outcomes, decreases deterioration rates,
and reduces no shows.
• Would you advise a parent that it is ok to
be consistent some of the time with
parenting strategy and still expect their
preferred outcome.
• Can you imagine a GP who is treating you
for high blood pressure saying, “Do I
need to measure the blood pressure
EVERY TIME?”
Lambert, M.J. (2010) Prevention of treatment failure: The use of measuring, monitoring,
and feedback in clinical practice. Washington, DC: American Psychological Association.
The purpose of the ORS & SRS:
• To give therapists ongoing, formalized
feedback about the progress and
process of therapy, allowing them to
maximize the effect of services.
• The ultra brief design and formats for
different age groups including small
children, allows all members of the
family an important feedback voice
into the therapeutic system, and
enhances everyone’s engagement
and participation in family work.
• The ORS and SRS have good validity
and reliability, and high feasibility.
Reluctant Using the ORS & SRS as It Might
Infringe on Therapy Time
• Using the ORS and SRS to get
feedback about the benefit & fit
of treatment need not be
cumbersome or intrusive on
therapy time.
• Clients appreciate your
dedication to getting it right,
• It is painless and can fit your
natural way of working.
Using the ORS and SRS to Help
Level Power Imbalances
• Invites clients into inner circle of
decision making
• Available in multiple languages
• Gives a voice to diversity and the
disenfranchised
• Local culture and context over
privileged knowledge
ORS & CORS
Integrating Outcome Into Care
• The ORS, a brief measure of global
subjective distress, is administered at
the beginning of each session to get real
time feedback from young people and
carers regarding their perspective of
treatment progress.
• There is a version for those 13 to adult,
and a younger version with smiley faces
called CORS for 6-12 year olds.
Postmodernism and ORS/CORS
• Other measures focus on symptoms and
diagnostic constructs and tends to
measure dysfunction (or rather peoples
perception on constructed notions of
dysfunction), and increase possibilities of
inviting a person to adopt a 'mental
illness' identity.
• In contrast, the ORS is a simple generic
measure that takes at face value what
people construct as meaningful for
themselves.
• It is easy and feasible to integrate into a
variety of practice modalities including
post modern models that help clients restory and develop new narratives.
Introducing the ORS/CORS to Young
Person and Carer
“The ORS is an outcome measure that allows
us to track where you’re at, how you’re doing,
how things are changing or if they are not. It
allows us to determine whether the therapy is
being helpful so we can do something different
if it’s not helping. It also is way to make sure
that your perspective stays central here, that
we are addressing what you think is most
important. It only takes a minute to fill out and
most clients find it to be very helpful. Would
you like to give it a try? Are you ok with that?”
“Ok, so let me go over the instructions with
you”.
Introducing the ORS/SRS to other
Members of the Family
• At most first interviews you won’t know where the distress is,
so you can ask all family members present to complete the ORS
on themselves. This allows you to "see" who is distressed.
• To carer and other family members present: “I would also be
grateful if the rest of you would also fill the form out on
yourselves to help me understand how things are going for you,
and to make sure what we are doing here is benefitting you too.
Even if things are going ok for you, I would be grateful if you
could do this anyway, so we can monitor and ensure that
whatever we do together doesn't impact you negatively.”
Outcome Rating Scale (ORS)
(Ages 13 to Adult)
Individually
(Personal well-being)
I----------------------------------------------------------------------I
Interpersonally
(Family, close relationships)
I----------------------------------------------------------------------I
Socially
(Work, school, friendships)
I----------------------------------------------------------------------I
Overall
(General sense of well-being)
I----------------------------------------------------------------------I
Child Outcome Rating Scale (CORS)
Ages 6-12
Me
(How am I doing?)
I------------------------------------------------------------------------------------I
Family
(How are things in my family?)
I------------------------------------------------------------------------------------I
School
(How am I doing at school?)
I------------------------------------------------------------------------------------I
Everything
(How is everything going?)
I------------------------------------------------------------------------------------I
The ORS: The Bare Bones
• The ORS has no specific content
other than the 4 domains—a
skeleton to which clients add the
flesh & blood of their
experiences.
• At the moment clients connect
the marks with what they find
distressing, the ORS is a
meaningful measure of progress
& potent clinical tool. Get a good
rating
First Session: Connect ORS to the
Client’s Described Experience
Principles:
• Where is the distress?
• Constructing meaning of the
scores
i.e. Connect the client’s
described experience of their
lives to the marks on the
scales
• Systemic questioning
Graphing: A Helpful Visual
• Adds a visual
component — another
way to focus tx
• At a glance tells the
story of the client’s
progress.
• Indispensable with
couples & families,
helps discuss multiple
viewpoints
ORS/CORS Graph
40
35
CORS Cutoff
(age 6-12)
30
25
20
child = 32
carer = 28
15
10
5
ORS Cutoff
Age 13-17= 28
0
Age 18 + = 25
Date
Making the Measures Work with Families
• Invite all members of the
family to Complete the
ORS or CORS.
• Invite all members of the
family to interpret.
• Summarize progress as
entry point into the
meeting.
15 Year Old ORS
7.6
5.7
• What stands out from
this teens scores in
addition to being
above the clinical cut
off which is 28?
• Suppose the mother’s
8.0
collateral scoring of the
teen is much lower
7.5
than the teens self
report scoring, how
could we use this
28.8
information to begin or
focus the session?
Linking Outcome to Treatment
• See clients more frequently
when the slope of change is
steep.
• Begin to space the visits as
the rate of change lessens.
• See clients as long as there is
meaningful change & they
desire to continue.
ORS/CORS
Reliable Change Index
• Reliable Change Index = 5
Reliable
Deterioration
5 point or more
worsening from
pre-treatment
score
Reliable
Change
5 point or more
improvement
from pretreatment score
Clinically Significant Change!
5 point or more improvement from pretreatment score and crossed the cutoff
score for age group
Alliance comprised of four empirically established
components which the SRS Measures:
1. agreement on goals, meaning or
purpose of treatment;
2. agreement on the means and methods
used;
3. agreement on the therapist’s role
(including being perceived as warm,
empathic and genuine; and
4. accommodating the client’s
preferences.
Norcross, J.C. (2010). The therapeutic relationship. In B.L. Duncan, S.D. Miller, B.E. Wampold, & M.A. Hubble (Eds.), The heart and soul
of change 2nd ed.: Delivering what works in therapy (pp.113-141).Washington, D.C.: American Psychological Association
Why is Monitoring the Alliance So Crucial?
• Researchers have repeatedly found that the
therapeutic alliance as rated by the client,
is one of the best predictors of outcome
across different types of therapy including
psychopharmocology.
• The feedback from the SRS helps
clinicians make the necessary adjustments
to align simultaneously with all family
members in the pursuit of common
agreement on goals and tasks in therapy.
Martin, D.J., Garske, J.P., & Davis, M.K. (2000). Relation of the therapeutic alliance with outcome and other variables: A meta-analytic review.
Journal of Consulting and Clinical Psychology, 68, 438-450.
Wampold, B.E. (2001). The great psychotherapy debate: Models, methods, and findings. Mahwah, NJ: Erlbaum.;
Norcross, J.C. (2010). The therapeutic relationship. In B.L. Duncan, S.D. Miller, B.E. Wampold, & M.A. Hubble (Eds.), The heart and soul of
change 2nd ed.: Delivering what works in therapy (pp.113-141).Washington, D.C.: American Psychological Association.
But I Already Ask My Clients for Feedback,
Why do I Need use an Alliance Measure?
• Research has shown that clinicians are
poor at gauging their client’s experience
of the alliance and they need to request
more formal real time alliance feedback.
• The key here is that people THINK they
ask more often than they do.
• Plus, and more importantly, since the data
says it’s the client's point of view that is
most predictive of outcome, you don't
want to leave it up to the therapist to
decide WHEN to ask.
Norcross, J.C. (2010). The therapeutic relationship. In B.L. Duncan, S.D. Miller, B.E. Wampold, & M.A. Hubble (Eds.), The heart and soul of change 2nd ed.:
Delivering what works in therapy (pp.113-141).Washington, D.C.: American Psychological Association.
Anker, M.G., Duncan, B.L., Owen, J., & Sparks, J.A. (2010). The alliance in couple therapy: Partner influence, early change, ad alliance patterns in a
naturalistic sample. Journal Consulting and Clinical Psychology, 78(5), 635-645.
Lambert, M.J. (2010) Prevention of treatment failure: The use of measuring, monitoring, and feedback in clinical practice. Washington, DC: American
Psychological Association.
Quickest Way to Prevent Drop Out
• Clients drop out for 2 reasons:
therapy is not helping
(monitor outcome) & alliance
problems—not engaged or
turned on.
• Direct way to improve
effectiveness is to keep
people engaged in therapy.
• Gotta measure the alliance to
improve your successes!
SRS & CSRS
Using the Client’s Experience of the Alliance to Guide
Practice
• The SRS is administered at the end
of each session to get feedback on
the alliance and therapy process from
young people and carers perspective.
• Like the ORS there is a version for
those 13 on up, and a younger
version called CORS for 6-12 year
olds.
• There is also a Group Version called
GSRS.
Introducing the SRS/CSRS
“Ok, we need to end in a couple minutes, but before we do I would be
grateful if you would take a minute to fill out this other form which asks
your opinion about our work together today?
I am not hoping for a perfect score: life is not perfect and I know I am
not either. I rely on this feedback to keep me on track, and let me know
when I am off track and need to make some changes to ensure that what
we are doing is benefitting you.
The ultimate purpose of using these forms is to make every possible
effort to make our work together beneficial. There might be some small
things that you think you might be able to live with or are not important,
but your feedback on even the small things are critical to me as it can be
the small things that make the difference. If anything feels missing, or I
said something that did not sit right or left you feeling blamed, you
would be doing me the best favor if you let me know. Can you help me
out?
Ok. Here is one for each of you. Remember please feel free to give your
honest opinion when filling this out. Ok?”
Session Rating Scale (SRS)
(Ages 13 to Adult)
Relationship
I did not feel
heard,
understood, and
respected.
We did not work on
or talk about what I
wanted to work on
and talk about.
I-----------------------------------------------------------------------I
Goals and Topics
I------------------------------------------------------------------------I
Approach or Method
The therapist’s
approach is not a
good fit for me.
There was
something missing in
the session today.
I-----------------------------------------------------------------------I
Overall
I------------------------------------------------------------------------I
I did feel heard
understood,
and respected.
We did work on or
talk about what I
wanted to work on
and talk about.
The therapist’s
approach is a good fit
for me.
Overall, today’s
session was right
for me.
Child Session Rating Scale (CSRS)
Ages 6-12
did not
always
listen
to me.
What we did and
talked about was
not really that
important to me.
Listening
I-----------------------------------------------I
How Important
I-----------------------------------------------I
listened to
me.
What we did and
talked about
were
important to
me.
What We Did
I did not like
what we did
today.
I wish we
could do
something
different.
I-----------------------------------------------I
Overall
I-----------------------------------------------I
I liked what we
did today.
I hope we do the
same kind of
things next
time.
SRS/CSRS Graph
40
35
30
25
20
15
10
5
0
Date
SRS
Cutoff =36
Discuss
SRS Scores Above the Cutoff Score: Keep
Convey Negative Feedback is a Gift
•
Keep encouraging client to let you
know…
•
No bad news on the SRS! Not a
measure of competence or anything
negative about you or the client.
•
Negative Feedback is a gift from the
client that helps you to be better.
•
Unless you really want it, you are
unlikely to get it.
•
You won’t get it from everyone.
What About Below 36 or 9cm
• Scores < 36 or 9cms should be
discussed.
• Don’t expect specifics or
revelations—any feedback is a
godsend
• “Is there anything else I could have
done, something I should have
done more of or less of, some
question or topic I should have
asked, was there anything I said
that did not sit right with you?”
Low SRS Scores Can be OK
Caution:
• Don’t assume a low SRS score means
that there are alliance issues.
• The most effective therapists tend to
have lower SRS scores and seem to be
more skilled at generating honest
feedback and using negative feedback
to improve performance and outcomes
• Key is to consider if the ORS/CORS
scores are improving. If not, there may
be a rupture or problem with the
alliance.
Does Not Seem to be Improving
“Okay, so things haven’t changed since the
last time we talked. How do you make
sense of that?
Should we be doing something different here,
or should we continue on course steady as
we go?
If we are going to stay on the same track, how
long should we go before getting worried?
When will we know when to say ‘when’ and
do something different?”
Using Session by Session
Measures in Supervision
• The measures and the graphs bring
the feedback and voice of the young
person and carer directly into the
supervisory session.
• Supervisors and supervisees can
utilize the measures and graphs from
a variety of cases to reflect on
potential areas for professional
growth and development.
How to Get the Measures Free
• The ORS/CORS and SRS/CSRS measures are licensed for
member groups of CORC and CYP IAPT by Scott D. Miller.
• Individual clinicians may also obtain a licence and the
measures at no cost in English, and languages other than
English, at www.scottdmiller.com.
• Note: When down loading the measures for the first time, you
may need to adjust the analogue scales to make sure they are
exactly 10cm. Beyond that adaptation the license agreement
involves no alterations to the ORS/CORS and SRS/CSRS.
Further Help and Training in using the
ORS/CORS and SRS/CSRS
• Information on Training in how to integrate
real time outcome and alliance feedback using
the ORS & SRS to improve clinical effectiveness
with young people and families contact:
– David C. Low, Family and Systemic Psychotherapist,
david.low1964@gmail.com, 01473-280416
– Scott D. Miller, Ph.D., Director of the International
Center for Clinical Excellence -info@scottdmiller.com
“So What, Now?”
5 Minute Exercise
• Get back into your small groups of 3.
• Choose someone to feedback themes to
the rest of us.
• Very briefly discuss what you each might
try doing as a result of the material
covered this morning.