Developing a case-mix classification for CAMHS:
Results from the CAMHS Payment System Project
EBPU Seminar: Anna Freud Centre, London, 13 May 2015.
Peter Martin
Lead Statistician, EBPU
Thanks to:
Amy Macdougall, Andy Whale, Benjamin Ritchie, Miranda Wolpert, Panos Vostanis, Roger Davies
& the Payment Systems Project Group
1
Why am I here?
To present results from the CAMHS Payment System Project:
 Groupings and how they were developed
 Can data teach us something about CAMHS?
Context:
 Underfunding of CAMHS
 Lack of comparable data on outcomes, treatment activity, and problem
description across services
2
CAMHS Payment System Project
 2012-2015. Commissioned by the Department of Health, but transferred to
NHS England in 2014
 Main aim: to develop “clusters” for CAMHS. A classification should satisfy the
following quality criteria:
o clinical meaningfulness
o ability to identify instances or periods of care (or advice/help) of similar
resource use, reflecting service user need
o reliability of identification.
 A system for grouping health service users is called a “case-mix classification”.
3
Approach and Sources of Information
4
Data Collection and Description
 Data collected from 20 CAMHS services: Sep 2012 through June 2014
 Focus on working with services to achieve high data quality
 Analysis Sample: 4,573 closed or dormant “periods of contact” from 11
services (for analyses predicting resource use)
 Full Sample: 11,353 open, closed or dormant “periods of contact” from the
same 11 services
 Community CAMHS only
5
Current View Tool: Problem Description
6
Current View Tool: Complexity Factors, Contextual Problems, and EET Issues
7
Note: N = 4573. 40 periods of contact were recorded to have attended more than 30 appointments. These are not
shown in this graph, but are included in the analysis.
8
Insights from Data Analysis: Appointments
 Around a quarter of children and young people (CYP) presenting at CAMHS
attend a single appointment before the case is closed.
 Around half of CYP attend three sessions or fewer.
 Around 5 % of CYP attend thirty appointments or more. These 5 % account for
about a third of all appointments that happen in CAMHS.
Source: CORC data submitted 2012/13.
9
Note: N = 4573.
10
Note: N = 4573. For the purpose of this graph, multiple Anxieties were counted as if they constituted a single problem.
11
Note: N = 4573. For the purpose of this graph, multiple Anxieties were counted as if they constituted a single
problem.
12
Note: N = 4573.
13
Number of Complexity Factors
Note: N = 4573.
14
Note: N = 4573.
15
Number of Contextual and EET Issues rated moderate or severe
Note: N = 4573.
16
Insights from data analysis (1)
Presenting Problems
 Around a quarter of CYP appear to present with mild problems only (from
Current View Ratings)
 About half of all CYP present with “multiple difficulties”: i.e. have more than
one problem rated at least ‘moderate’
 Family Relationship Difficulties, Low Mood/Depression, and Peer Relationship
Difficulties are the three difficulties most frequently rated as present.
Complexity Factors
 About 45 % of CYP were rated to have one or more complexity factors at case
opening
 About 20 % of CYP were rated to have two or more complexity factors
17
Insights from data analysis (2)
Contextual Problems, EET Issues
 Around half of CYP had one or more contextual problem or EET issue rated as
moderate or severe
 Around a quarter of CYP had two or more such issues rated as moderate or
severe
18
What does “clustering” (“grouping”) mean?
Note: These images represent simulated illustrations only. No CAMHS data were used to make them.
19
Development of a case-mix classification
Approach 1: Supervised Cluster Analysis (Andy Whale & PM)
 Method: Regression Trees
 This resulted in models that were unreliable (different random subsamples of the data
yielded different models)
Approach 2: Unsupervised Cluster Analysis (Amy Macdougall & PM)
 Method: k-medoid cluster analysis
 This resulted in an ill-fitting classification, and a poor prediction of resource use
Approach 3: Conceptually guided clustering using NICE guidance categories
 This resulted in the best prediction of resource use overall
 Different classifications were compared (3 clusters, 5 clusters, 16 clusters, 18 clusters)
 Model Comparison using statistical criteria (AIC / BIC)
20
Group Development (Approach 3 from previous slide)
The development of groups was informed by two principles:
 The THRIVE model for CAMHS
 NICE guidance
Philosophy:
Assignment of a CYP to a group should be made by the clinician after a
collaborative decision on treatment or advice has been made.
21
THRIVE
The THRIVE Model inspired a broad distinction between three categories of service
users:
 “Getting Advice”: CYP thought to benefit from signposting, advice on selfmanagement, or assessment
 “Getting Help”: CYP who are thought to benefit from goals-focused, evidenceinformed, outcomes-oriented intervention
 “Getting More Help”: CYP who are thought to benefit from extensive treatment
22
Classification of CYP into NICE Guidance Categories
We identified 14 types of presenting problems for which NICE clinical guidelines were
available:
ADHD
Autism Assessment
Autism Management
Bipolar Disorder
Conduct Disorder
Depression
Eating Disorder
Emerging Borderline Personality Disorder
Generalized Anxiety Disorder or Panic Disorder
Obsessive-Compulsive Disorder
Psychosis
PTSD
Self Harm
Social Anxiety
23
Algorithm for Group Allocation
Information from CAMHS practitioners’ Current View ratings at first contact was used to
check, for each case, whether presenting problems appeared to ‘fit’ a NICE clinical
guideline. To ‘fit’ a NICE guideline, a CYP had to fulfil the following criteria:
 Have the “index problem” defined by the NICE guideline, rated ‘moderate’ or ‘severe’
 Not have a significant “comorbidity” that would mean that NICE guideline may not be
applicable in a straightforward way
Example:
To be classified into the category “OCD”, a CYP had to:
 Have “Compelled to do or think things” rated moderate or severe (this is the “index
problem”)
 Not have any of 23 specific other problems (e.g. “Low Mood”, “Delusional Beliefs or
Hallucinations”, etc.) rated at equal or higher severity compared to the index problem
24
Algorithm for Group Allocation (2)
If, after applying the rules explained on the previous slide, the CYP could not be allocated to
a group, they were assigned in one of five additional groups (on which see below), according
to clear rules.
In practice, the algorithm is intended to be an aid in grouping only. The clinician should
always be able to override the algorithm’s suggested group allocation.
We did not develop an algorithm for “Emerging Borderline Personality Disorder”, because
we judged that this is rarely identifiable at first contact.
25
26
Groupings: Overview
We propose to group children seen in CAMHS into 19 groups.
 14 groups are defined with reference to a NICE guideline; their names
reference diagnostic categories, but a formal diagnosis is not required for a
CYP to belong to one of these groups
 2 groups are defined by the presence of specific types of co-occurring
difficulties:
o Getting Help with Co-occurring Behavioural and Emotional Difficulties
o Getting Help with Co-occurring Emotional Difficulties
 3 groups are not symptom specific, but are distinguished by the type of agreed
treatment:
o Getting Advice: Signposting and Self-management Advice
o Getting Help with [other] Difficulty or Difficulties
o Getting More Help with Co-occurring Difficulties of Severe Impact
27
Estimated percentages of grouping membership
Short Estimated
Label
percent
ADV
27.70 %
NEU
3.47 %
ADH
6.96 %
AUT
2.16 %
BIP
1.03 %
BEH
5.18 %
DEP
5.76 %
GAP
4.22 %
OCD
1.11 %
PTS
1.74 %
SHA
5.68 %
SOC
1.59 %
BEM
1.69 %
EMO
7.65 %
COD
16.08 %
EAT
1.76 %
PSY
1.24 %
CSI
8.43 %
Group
Getting Advice: Signposting / Self-management
Getting Advice: Neurodevelopmental Assessment
ADHD
Autism Spectrum
Bipolar Disorder
Antisocial Behaviour, Conduct Disorders
Depression
GAD and/or Panic Disorder
OCD
PTSD
Self-harm
Social Anxiety Disorder
Co-occurring Behavioural And Emotional Difficulties
Co-occurring Emotional Difficulties
Other Difficulty/ Co-occurring Difficulties
Eating Disorders
Psychosis
Co-occurring Difficulties of Severe Impact
28
Notes: n = 11,353. The group
‘Neurodevelopmental Assessment’
is not mutually exclusive with the
remaining seventeen groups. Thus
percentages sum to 100 %, not
counting ‘NEU’. The grouping
‘Symptoms/Presentation
Suggestive of High Risk of
Emerging Borderline Personality
Disorder or Potential BPD’ is not
represented, since there is
currently no allocation algorithm for
this group.
Percentage of Group Membership by Age Band
Note: Total n = 10,172. There were 1180 children in the Full Sample who had no information on age and are excluded from this table. Error bars
show 95 % confidence intervals. See also notes to previous slide.
29
Number of Appointments by Group
Notes: N = 4573. For notes on groups not shown see slide 23.
30
What about Complexity?
Estimated membership proportions:
 Behavioural and Emotional Difficulties:
~ 2%
 Co-occurring emotional difficulties:
~ 8%
 Other (co-occurring difficulties):
~16 %
 Co-occurring difficulties with severe impact: ~ 8 %
We estimate that around a third of children present with significant co-occurring
problems, which may mean that no single NICE guidance is in itself sufficient to
inform treatment.
But what about complexity factors, contextual problems, and EET issues?
31
Testing the relevance of complexity factors (etc.) for predicting resource use
32
Notes to previous slide: The plot on the previous slide is based on a model predicting the “number of appointments”
using 18 groupings and 19 complexity, context and EET factors as predictors. Coloured bars show estimates of the
effect of having the associated risk factor, compared to the risk factor being absent. A bar reaching ‘up’ indicates that
the associated risk factor is predicted to increase the number of appointments; a bar reaching ‘down' indicates that
the associated risk factor is predicted to decrease the number of appointments. Error bars around the coloured bars
show 95 % confidence intervals. If error bars span the value “0”, then there is no strong evidence for the influence of
the associated risk factor. See below for a legend to labels, and for the model specification. The estimated effects of
the 18 clusters are shown alongside the effects for complexity, context and EET factors. Factors are distinguished by
colour: beige bars show complexity, contextual, or EET effects; blue bars show groups belonging to “Getting Help”,
purple bars show groups belonging to “Getting More Help”. The influence of each cluster or risk factor is shown
compared to a child in the “Getting Advice: Signposting and Self-management” group without any risk factors. It can
be seen that Group Membership is a more important predictor of “number of appointments” than any of the
associated risk factors.
The model used is called a mixed effects negative binomial regression; it includes a random effect for the service the
CYP attended (effect not shown). A full model specification is given at the end of these slides.
33
Legend to Slide 32
Complexity Factors
ABU: Experience of Abuse or Neglect
CIN: Child in Need
FIN: Living in financial difficulty
JUS: Contact with Youth Justice System
LAC: Looked after Child
LD: Learning Disability
NEI: Neurological Issues
PAR: Parental Health Issues
PHY: Physical Health Problems
PRO: Current Protection Plan
REF: Refugee or asylum seeker
WAR: Experience of War, Torture or Trafficking
YC: Young Carer
Contextual Problems
ENG: Service Engagement
COM: Community Issues
HOM: Home
SCL: School, Work or Training
Education/Employment/Training
ATA: Attainment Difficulties
ATE: Attendance Difficulties
Groupings: Getting Advice
ADV: Getting Advice: signposting/self-management
NEU: Neurodevelopmental Assessment
Groupings: Getting Help
ADH: ADHD
AUT: Autism
BIP: Bipolar Disorder (moderate)
BEH: Antisocial Behaviour / Conduct Problems
DEP: Depression
GAP: Generalized Anxiety or Panic Disorder
OCD: Obsessive Compulsive Disorder
PAN: Panics
PTS: PTSD
SHA: Self Harm
SOC: Social Anxiety
BEM: Behavioural and Emotional Difficulties
EMO: Co-occuring Emotional Difficulties
COD: Other (co-occurring) Difficulties
Groupings: Getting More Help
EAT: Eating Disorder
PSY: Psychosis
CSI: Co-occurring difficulties of severe impact
34
Summary
The classification of CAMHS cases according to our designed groupings provides a
better and more reliable prediction of resource use than “a-theoretical” models
found by statistical methods (cluster analysis, regression trees).
Once group membership was taken into account, there was no strong evidence of
an additional contribution by context and complexity factors to the prediction of
resource use.
35
Summary: Estimated Grouping Proportions
Proportions by “Super Grouping” give an impression of the frequency with which
different types of need are encountered in CAMHS:
 Getting Advice: 28 %
 Getting Help: 61 %
 Getting More Help: 11 %
Proportions shown by “NICE-relevance” indicate an aspect of the diversity and
complexity of CYP seen in CAMHS:
 Groups defined by NICE guidance: 39 %
 Groups defined by specific “Comorbidities”: 9 %
 “Other” Groups: 52 %
36
Quality criteria
 Clinical meaningfulness: The classification relates to NICE best practice
guidelines, and takes into account frequent “comorbidities” and distinguishes
groups of CYP that don’t appear to neatly fit into a category. We recommend
that allocation to a group is based on a shared decision about treatment aims.
 Relation to resource use: Modest relationship to resource use (but better than
purely ‘data’ driven models).
 Reliability of identification: Unclear. Reliability will depend on reliability of
Current View Ratings, as well as reliability of practitioners’ group allocation
decisions (when overriding the algorithm’s suggestion).
37
Clinical Meaningfulness? Hypothetical case studies of group allocation
based on shared treatment decision
Hypothetical Case 1:
“Algorithm suggests ‘Getting Help: With Co-occurring Emotional Difficulties’ (H22) on the basis of
‘Compelled to do or think things (OCD)’ and ‘Depression/low mood (Depression)’ rated as ‘moderate’
on the Current View tool. Young person and clinician believe that the depression arose after the OCD
had seriously limited ability to get on with life. They think a care package focussed on helping with the
compulsion to do things would be more appropriate than a care package aimed at helping with both
compulsion to do things and depression, as it is felt that treatment for the former would help to lift
depression. Thus ‘Getting Help: Guided by NICE Guideline 31 (OCD)’ (H4) is chosen.”
Hypothetical Case 2:
“Algorithm suggests ‘Getting Help: Guided by NICE Guideline 26 (PTSD)’ (H2) on the basis of ‘Disturbed
by traumatic event (PTSD)’ rated as ‘moderate’ on the Current View tool. Discussion with young person
regarding treatment options and recommendation that seen for weekly sessions to process trauma.
Young person chooses not to access treatment as now understands symptoms, is able to manage
these and does not wish to face difficult trauma memories at present. Symptom management
strategies agreed between clinician, young person and their family. Bibliotherapy with one-off follow up
is offered. Thus ‘Getting Advice: Signposting and Self-management Advice’ (A1) is chosen.”
38
Conclusions
 Application: Our idea is that grouping allocation should be made by the
clinician based on a shared decision between the clinician and a child or
young person (and their family) regarding the aim of advice or treatment
 Algorithm: The algorithm which ‘predicts’ membership in a specific group is
intended as an aid to decision making; the algorithm may always be overruled
by the clinician
 What’s next: We recommend further investigations to establish (and, if
necessary, improve) the reliability and validity of the groupings, and to gauge
training needs for CAMHS staff involved in using the groupings
39
Appendix: Model Specification for Slide 32
A mixed-effects negative binomial regression approach was used to explore the effect of contextual
problems and complexity factors. The model includes a random effect for “CAMH service” in order to
take into account the nested data structure.
𝑝
log(𝜇𝑖𝑗 ) = 𝛽0 + ∑𝑘=1 𝛽𝑘 𝑥𝑖𝑗𝑘 + 𝑢𝑖 ,





where:
Yij ~ NB(μij, α) is the number of appointments after the first session (so that the minimum number is
zero) for the jth child treated by the ith service, with mean μij and dispersion parameter α;
β0 is an intercept term;
βk, k = 1, … , p, is a vector of slope coefficients corresponding to the p predictor variables x1, …, xp;
ui ~ N(0, σu 2 ) is the random intercept term for the ith service, i = 1, …, 11;
The variance function is defined as: V(α) = μ + αμ2 . (This is called the “NB2 parameterization”.)
40