Making Emotional Health Services more responsive to service
user’s Needs: the role of evidence and outcome measures
Evidence Based Outcome Measure and Informed Practice Conference, Leeds,14th March 2016
What are outcome based services?
(and why service users like them)
Dr Miranda Wolpert
Reader in Evidence Based Practice and Research UCL
Director of Evidence Based Practice Unit
Director of Child Outcomes Research Consortium
14th March 2016
What outcomes?
• Symptoms
• Wellbeing
• Educational attainment or
attendance
• Achieving goals
• Reslience in face of
adversity
• Greater understanding
• Self management
skills
• Improved relationships
• Safety
• Coping skills
• Contributing to society
• Good experience of
help?
Whose outcome?
•
•
•
•
•
Child
Parent
Teacher
Practitioner
Social worker
•
•
•
•
Policy makers
Local authority
Service managers
Public health
specialists
What outcomes are relevant for
emotional health service users
© Evidence
Based
Practice Unit
(EBPU)
Why are outcomes based
services important
• Accountable
• Can adapt in light of new information
• Ensure service user voice at heart of
services
• Helps clarify what is being worked towards
(prevents COWS)
• Helps know when succeeded and when not
Why do service users like
them?
“Outcomes tools can be really useful, not only for
the therapists but for the young people. They allow
you to evaluate where things are going well and
where they could be improved. Feedback is
important for us to learn and grow. However, I do
think you must use outcomes tools with the correct
intention and in the right way.”
Young person; (Law & Wolpert, 2014, p. 28).
What are outcomes based
services?
• Service users experience and outcomes at
heart of all activity
• Information used at all levels
• With individuals
• To review classes
• To consider services as an integrated whole
TARGET
UNDERSTANDING YOUR MODEL
Who is the
intervention for?
MODERATORS
Will it affect
everyone the same?
INTERVENTIO
WhatNis the
intervention?
CHANGE MECHANISMS
Why does the
intervention
work?
OUTCOME
S
What difference
will it make?
Case study: effective commissioning of
child emotional wellbeing services
Domestic
violence
support
NEET
support
CAMHS
NHS
provider
(CCG)
Social care
LAC (LA)
CAMHS
voluntary
provider
(CCG)
School
counsellor
(school
cluster)
Youth
Offending
Service
Social care
family
therapy (LA)
Improved child
and young person
emotional health
and mental
wellbeing
Public health
voluntary
provider (LA)
1. Where is resource best allocated in the system?
2. Is there a shared way to understand impact?
3. Is there a shared vision for best practice around person-centred/ evidence
based services?
4. What are the implications for the child’s experience - moving along a
pathway, communicating with different professionals, understanding how we
Facilitated discussions
What aren't outcome based
services
• Using measures without thought
• Not considering what makes sense
• Not reading measures
Plan for Talk
• CORC approach
• Whole system outcomes
• Individual level outcomes
CORC
Making use of messy real
world data to improve the
quality of emotional health
services
Central and Regional Team
Board and Committee
Miranda Wolpert, Ashley Wyatt, Alan Ovenden, Mick
Matt Barnard, Jenna Jacob, Benjamin Ritchie,
Atkinson, Julie Elliott, Kate Martin, Duncan Law and
Kate Dalzell, Victoria Zamporeni, Elisa
Napoleone, Andy Whale, Amy McDougall,, Craig Ann York
Hamilton, Alison Ford, Sally Wilson and Deborah
Sheppard
corc@annafreud.org
www.corc.uk.net
14
15
System level
16
6 steps for selecting cross-sector outcome
measures and indicators
Strategic decisions
1. What outcome is the whole system trying to achieve?
Why?
What is the desired direction of change?
Has the direction of change been sense checked?
Only proceed if partner organisations agree on the above
2. Which potential measure of the
outcome fares the best when assessed
against key criteria?
Technical detail
Return to step
2, 3 or 4 as
appropriate
3. What are the ‘ingredients’ for an
indicator of the outcome?
What information will be useful for
interpreting the data?
4. How will the indicator of the
outcome be operationalised?
Steps 2, 3 and 4 may need to be addressed in an iterative
rather than sequential manner
Sign-off
No
5. Do partner organisations agree with the proposed
measure, indicator definition and info to aid interpretation?
Yes
6. Ensure an outcome indicator and data linkage
proposal document has been completed
17
Measure of well-being: WEMWBS
•
•
•
•
•
•
•
•
•
•
•
•
•
•
I’ve been feeling optimistic about the future
I’ve been feeling useful
I‘ve been feeling relaxed
I’ve been feeling interested in other people
I’ve had energy to spare
Response options: None of
I’ve been dealing with problems well the time, Rarely, Some of the
time, Often, All of the time (5)
I’ve been thinking clearly
I’ve been feeling good about myself
I’ve been feeling close to other people
I’ve been feeling confident
I’ve been able to make up my own mind about things
I’ve been feeling loved
I’ve been interested in new things
I’ve been feeling cheerful
Measures of mechanisms/protective
factors: Student Resilience Scale
At home, there is an adult who…
1. Is interested in my school work
2. Believes that I will be a success
3. Wants me to do my best
4. Listens to me when I have something to say
At school, there is an adult who…
5. Really cares about me
6. Tells me when I do a good job
7. Listens to me when I have something to say
8. Believes that I will be a success
Away from school, there is an adult who…
9. Really cares about me
10. Tells me when I do a good job
11. Believes that I will be a success
12. I trust
Away from school…
13. I am a member of a club, sports team, church group, or other group
14. I take lessons in music, art, sports, or have a hobby
Response options: 1 (Never) – 5 (Always)
Are there students at your school who would…
15. Choose you on their team at school
16. Tell you you’re good at doing things
17. Explain the rules of a game if you didn’t understand them
18. Invite you to their home
19. Share things with you
20. Help you if you hurt yourself
21. Miss you if you weren’t at school
22. Make you feel better if something is bothering you
23. Pick you for a partner
24. Help you if other students are being mean to you
25. Tell you you’re their friend
26. Ask you to jo in in when you are all alone
27. Tell you secrets
28. I do things at home that make a difference (i.e., make things better)
29. I help my family make decisions
30. At school, I help decide things like class activities or rules
31. I can work out my problems
32. I can do most thin gs if I try
33. There are many things that I do well
34. I feel bad when someone gets their feelings hurt
35. I try to understand what other people feel
36. When I need help, I find someone to talk to
37. I know where to go for help when I have a problem
38. I try to work out problems by talking about them
39. I have goals and plans for the future
40. I think I will be successful when I grow up
41. I do things at my school that make a difference (i.e. make things better)
Measure of mechanisms/protective
factors (self-control): SSIS
• I stay calm when teased
• I stay calm when people point out my
mistakes
• I can find a good way to end a disagreement
• I stay calm when dealing with problems
• I stay calm when others bother me
• I stay calm when I disagree with others
Response options: Never, Sometimes, Often, Always (4)
MINDFUL Approach
•
•
•
•
•
•
•
Multiple perspectives
Interpretation level
Negative differences
Directed discussions (75% - 25% rule)
Funnel plots
Uncertainty; need to triangulate
Learning collaborations
Wolpert M, Deighton J, De Francesco D, Martin P, Fonagy P, Ford T. (2014) From ‘reckless’ to ‘mindful’ in the use of
outcome data to inform service-level performance management: perspectives from child mental health. BMJ Quality &
Safety.
Individual level
Measures for Practice;
Guidance
Outcomes focus for the individual
Assessment/Choice
Partnership/ongoing
work
Review & Close
• What’s the problem?” (assessment) This is understanding the issue the young person
or family have come for help with
• “What do you want to change?” (goals or aims of therapy) – this is understanding
the specific goals the young person or family have - the things they want to work on
in coming to a service
• How are we getting on together?” (engagement or alliance) It is important to get
things right from the start
• “How are things going?” (Symptom/goal tracking) – this is tracking to see if things
are progressing during and intervention
• Have we done as much as we can/need to?” (collaborative decision to close or refer
on) – re-review of question 1 e.g. Time2 SDQ (if not used as tracker in long-term case)
• “How has this experience been generally?” (experience of service overall).
Ensuring voice of young people
• Young service users and carers want their views considered in
intervention decisions and therapy.
• Young people can experience difficulties in getting their views
across in a way that feels safe and respected.
• Forms can help ensure that practitioners ask about the key aspects
of the problem for young people and families and realise when
someone is about to disengage from therapy.
• Forms should support the voice of young people and families who
use services as part of collaborative practice and shared decisionmaking.
• Forms can become a barrier to conversations if not introduced and
used in sensitive ways (see the dos and don’ts)
•
Law, Jones and Wolpert 2015
Measuring outcomes in practice: young
peoples’ views
PROMs help make
the balance of power
more equal.
Gives us a shared understanding of
…where we’re starting from.
…where we’re heading to.
…how we’re going to get there.
Enables us to get an in-depth
understanding of what we’re
feeling, why we’re feeling it
and what we can do about it.
It means if we go
off track or get a
bit lost along the
way, we can both
figure out how to
find the way
back again.
Makes us feel like it’s a shared experience
between us and the clinician... like we’re in
this together.
It is important to
monitor outcomes to
make sure the person
feels better not worse
It makes us feel like there is a
point to our therapy
Quotes from young people from YoungMinds consultation in Devon. Reference : Talking About Talking Therapies/Devon CAMHS
Views of members of VIK Young Minds
27
Anxiety
Oocamhs.com
Experience of service
Dos and Donts
CORC accreditation framework
Leadership &
Management
• Organisational vision
• Organisational
commitment to
collection and
collation
• Organisational
commitment to
interpretation and
use
• Organisational
culture supportive of
use and learning
Staff Development
• Understanding of use
of different data
sources (including
measures)
• Use of particular
data sources
(including measures)
• Training and
Continued
Professional
Development (CPD)
• Review of measures
and feedback in
supervision
Technology and
Information
Management
• Enabling data use in
direct practice with
clients
• Enabling use of data
at practitioner level
• Enabling use of data
at team level
• Enabling use of data
at service level
Experience of Service
• CYPPC understanding
of measures
• Communication with
CYPPCs about
measures
• Collaborative setting
of goals and choice
of measures
• CYPPC feedback on
support
Contact
• Miranda.Wolpert@ucl.ac.uk
• EBPU@annafreud.org
• CORC@annafreud.org