Routinely collected CGAS ratings –
are they of any use to
Child and Adolescent
Mental Health Services?
Anna Lundh, M.D., Ph.D.
Department of Clinical Neuroscience
Karolinska Institutet
Department of Child and Adolescent Psychiatry
Stockholm County, Sweden
WHY DO WE USE
RATING SCALES AND
DIAGNOSTIC INSTRUMENTS?
2
Objectify and
standardise clinical
data with valid tools
ª Symptoms/ Diagnostic
category
ª Global functioning
3
WHAT IS GLOBAL
FUNCTIONING?
4
AT H
OME
5
S
D
N
E
I
R
F
H
T
I
W
6
7
Unidimensional scales
ª Global Assessment scale, GAS, 1973 (Endicott 1976)
ª Children’s Global Assessment Scale, CGAS, 1983 (Shaffer 1983)
ª Global Assessment of Functioning, GAF, 1987
(DSM-III-R)
8
CGAS
ª Clinician rated
ª All patients in Stockholm
§  Intake and end-of-treatment
ª 1 July 2006
ª Lowest level of functioning
ª Time period - last month
ª Between 1 and 100
9
Cut-offs
70 71-­‐100 indicates normal func4oning 61-­‐70 probable clinical case 60 1-­‐60 definite clinical case (Shaffer 1983, Bird 1987)
10
A NOTE ABOUT SCALES
11
CGAS is an ordinal scale….
12
…that needs psychiatric training to use and
to interpret the result….
13
…but appears to be a ratio scale!
14
Inter-rater reliability, CGAS
ª Intra-class correlation coefficient (ICC)
ª From 0 to 1
ª 19 case vignettes, 5 raters (medical residents)
§  ICC 0.84 (Shaffer 1983)
ª 10 case vignettes, 78 raters, <1 hour introduction/
training
§  ICC 0.61 (Hanssen-Bauer 2007)
ICC*=Intraclass correlation coefficient
15
What is a “true” CGAS rating?
ª No Gold standard for global functioning
ª Common: test CGAS ratings against the group mean
rating
ª An alternative: use expert ratings
16
Training
ª No instructions in original work if training was
required
ª Recommended (Schorre 2004, Winters 2005, Rush 2007)
ª No evaluation of training effect
17
”Let’s start using CGAS
for all inpatients in
Stockholm!”
Eva-Britt Hallquist and Peter Engelsöy,
former and present head of Inpatient unit.
2003
2005
2007
2009
2011
18
Is CGAS possible to use among
hundreds of clinicians?
How can we train large groups?
19
Case vignettes
ª 80 anonymous charts from
first visits at outpatient clinic
and the ER
Nybe
ånge sök. Sök
e
s
mod t och oro r pga
ern.
tills m
Soc
Bor t
ills m
2 sys
e fld
kon
och
Skola
G
å
r
klass
i åk 4
extra som kan . Liten
hjälp
g
skola
med e K
svår rbetet. A
ta
llt
sake tt lära sig id haft
r
nya
Emelie 13
Nadja 16
Johanna 5
Emil 10
Erik 14
ª 50 cases were constructed
20
Expert rating, the best estimate for gold standard
ª 5 experienced clinicians
§  3 child psychiatrists and 2 psychologists
ª Rated 50 cases individually
ª Discussion and consensus rating
§  EXPERT RATING!
21
Development of training programs
ª  Two methods – same content
§  Live seminar
§  Computer-based
ª  Theoretical part
ª  Skills training
22
Study I
23
24
Moderate inter-rater reliability before and
after training
ª Baseline ICC values
ª Endpoint ICC values
§  0.92 (expert raters)
§  0.71 (seminar group)
§  0.78 (seminar group)
§  0.76 (CD group)
§  0.78 (CD group)
§  0.67 (comparison group)
§  0.79 (comparison group)
ª ICC = 0.60-0.80
Moderate
ª ICC = 0.80-1.0
Substantial
25
High trainee satisfaction
ª Scale 1-6
§  Seminar group
5.4
§  CD group
4.9
26
Conclusions, Paper I
ª  The inter-rater reliability is moderate
ª  CGAS is a useful instrument
§  Caution when comparing CGAS ratings of the same patient from
different practitioners
ª  Differences in ratings between professional categories
§  How and by whom should CGAS ratings be performed?
§  Team ratings?
27
Conclusions, Paper II
ª  No difference between the two forms of training
ª  Small improvement, unlikely to be clinically relevant
ª  Similarly positive effect in the non-randomised comparison
group that received no training
ª  Future education trials should include regular, randomly
assigned control (WILL TAKE PLACE IN MARCH 2013)
ª  Personalised training?
28
A NOTE ABOUT EFFICACY
AND EFFECTIVENESS
29
Efficacy studies
Effectiveness studies
ª Randomised
controlled studies
ª Real world setting
ª Clear-cut diagnosis
ª No comorbidity
ª Unselected
clinicians
ª Exclusion criteria
ª No exclusion criteria
ª Patients and
clinicians behave
differently
ª No control group
ª Unselected patients
30
Efficacy studies
Effectiveness studies
ª Randomised
controlled studies
ª Real world setting
ª Clear-cut diagnosis
ª No comorbidity
ª Unselected
clinicians
ª Exclusion criteria
ª No exclusion criteria
ª Patients and
clinicians behave
differently
ª No control group
ª Unselected patients
31
Gap between efficacy and effectiveness
studies
ª  Clinical guidelines are based on EFFICACY STUDIES Randomised Controlled Trials
ª  The results from efficacy studies not generalisable
ª  Low number of EFFECTIVENESS STUDIES
§  Routinely collected data - CGAS
32
33
Aims
ª  To investigate outcomes of child psychiatric outpatient treatment
as measured by change in CGAS ratings – ΔCGAS - before and
after treatment in a large naturalistic cohort
ª  To identify predictive clinical factors for CGAS change in two
diagnostic groups:
§  mood disorder
§  attention deficit hyperactivity disorder (ADHD)
34
Methods I
ª  12,613 outpatients, 1 July 2006 - 31 January 2010
ª  Age 12.0 (3.9)
ª  Girls 52% and Boys 48%
ª  Number of visits 13.1 (14.4)
ª  Treatment period more than one month
35
Methods II
ª  Clinical database Pastill
§  Diagnosis
•  DSM-IV and/or ICD10
§  CGAS rating
§  Treatment
•  Counselling and psychotherapy
à  Time frames
à  Settings (individual, group, parents, family)
•  Medication
à  Central stimulant
à  Other than central stimulant
§  Psychosocial stressor
•  physical health problem, abuse, death etcetera
36
Selected, excluded and included patients
44 261 Registered patients
11 344
7 796
Treatment period <one month
Continuing treatment
25 121 Selected patients
12 508
4 494
5 526
4
281
278
1 704
221
12 613 Included
outpatients
31 648
Excluded patients
CGAS=0* at baseline
CGAS missing at baseline
CGAS=100 at baseline
Age <4 or >20 years old
Inpatients
CGAS=0 at case closure
CGAS missing at case closure
Not selected/excluded patients
*Zero does not represent a CGAS score, but indicates that the professional has not
performed the assessment, which is not the same as missing data
Diagnostic category
n=
Baseline
Mean (SD)
ΔCGAS
Mean (SD)
Suicide attempt
302
43.5 (9.9)
16.1 (14.5)
Mood disorders
2,213
50.3 (9.8)
13.4 (12.2)
Anxiety disorder
2,446
51.3 (10.2)
13.2 (11.9)
ADHD
1,169
50.5 (9.5)
6.5 (8.9)
41.2 (14.0)
3.9 (9.6)
Mental retardation
224
38
J. AM. ACAD. CHILD. ADOLESC. PSYCHIATRY, 45:12, DECEMBER 2006
39
TABLE 1. CGAS, scores by treatment group
Vitiello et al
Treatment
arm
Combination
FLX + CBT
CGAS total score
Baseline
Week 6
Week 12
Mean +/-SD
50,0 +/-7,52
62,4 +/-11,2
66,6 +/-11,91
Range
35,0
40,0
Mean +/-SD
49,5 +/-7,26
59,9 +/-10,58
62,1 +/-11,91
Range
35,0
65,0
40,0
90,0
35,0
90,0
109
99
98
Mean +/-SD
50,0 +/-7,58
56,7 +/-9,66
60,0 +/-11,47
Range
34,0
35,0
Mean +/-SD
Range
No
All treatments
95,0
95
No
Placebo
40,0
98
No
CBT
90,0
107
No
FLX
80,0
Mean +/-SD
Range
No
68,0
80,0
40,0
90,0
111
97
90
49,1 +/-7,59
57,0 +/-9,22
59,3 +/-12,72
32,0
67,0
40,0
75,0
35,0
85,0
112
95
96
49,6 +/-7,47
59,0 +/-10,43
62,0 +/-12,3
32,0
80,0
439
35,0
90,0
389
35,0
95,0
379
40
Clinical factors associated with CGAS
change
ª  Mood disorders
ª  ADHD
41
Mood disorder (n=2,213)
Beta P value
Teenage psychotherapy
0.10
<0.001
Short term psychotherapy
0.11
<0.001
Family therapy/counselling
0.10
<0.001
Other treatment
0.06
0.010
Family crisis intervention
0.06
0.004
Number of visits
0.06
0.013
Counselling with family and social network
-0.09
<0.001
Managed by others than psychologist, social
worker, medical doctor
-0.08
<0.001
Medication
42
n.s.
Conclusions, Mood disorders
ª  Medication in mood disorder treatment were not significantly
associated with improvement of CGAS ratings
§  Differs from clinical trials
ª  Several of the different psychotherapeutic interventions were
positively correlated with baseline CGAS and ∆CGAS in mood
disorders
§  Similar to clinical trials
43
ADHD (n=1,169)
Beta P value
Intervention variables
Number of visits
0.10
0.003
Managed by medical doctor
0.10
0.001
Parent counselling, group
0.08
0.007
Therapeutic summer camp
0.06
0.033
Central stimulants
n.s.
44
Selected, excluded and included patients
44 261 Registered patients
11 344
7 796
Treatment period <one month
Continuing treatment
25 121 Selected patients
12 508
4 494
5 526
4
281
278
1 704
221
12 613 Included
outpatients
31 648
Excluded patients
CGAS=0* at baseline
CGAS missing at baseline
CGAS=100 at baseline
Age <4 or >20 years old
Inpatients
CGAS=0 at case closure
CGAS missing at case closure
Not selected/excluded patients
*Zero does not represent a CGAS score, but indicates that the professional has not
performed the assessment, which is not the same as missing data
Selected, excluded and included patients
44 261 Registered patients
7 796
Continuing treatment
25 121 Selected patients
12 508
4 494
5 526
4
281
278
1 704
221
12 613 Included
outpatients
31 648
Excluded patients
CGAS=0* at baseline
CGAS missing at baseline
CGAS=100 at baseline
Age <4 or >20 years old
Inpatients
CGAS=0 at case closure
CGAS missing at case closure
Not selected/excluded patients
*Zero does not represent a CGAS score, but indicates that the professional has not
performed the assessment, which is not the same as missing data
Frequency of central stimulants
Continued treatment
28.3 %
Ended treatment
12.7 %
47
ADHD, ended treatment. With or without CS
CS
N=132
No CS
N=910
Unpaired
t-test
Mean (SD)
Mean (SD)
P Value
Age
13.3 (3.5)
11.0 (3.8)
<0.001
Number of diagnoses
1.9 (1.1)
1.9 (1.1)
0.842
Number of
psychosocial stressors
1.8 (1.3)
2.2 (1.7)
0.008
CGAS at baseline
52.2 (9.5)
50.3 (9.6)
0.038
Number of visits
14.8 (13.8)
13.9 (13.8)
0.503
ΔCGAS
7.1 (10.0)
6.4 (8.9)
0.385
48
CS, N (%)
No CS, N (%)
Chi-2
Boys
95 (72.0)
658 (72.3)
0.935
Girls
TREATMENT INTERVENTIONS
37 (28.0)
252 (27.7)
0.935
Guidance to parents
82 (62.1)
707 (77.7)
<0.001
Guidance to teenagers
47 (35.6)
363 (39.9)
0.346
Family therapy/counselling
19 (14.4)
299 (32.9)
<0.001
Teenage psychotherapy
5 (3.8)
32 (3.5)
0.875
Social network counselling
9 (6.8)
102 (11.2)
0.127
Cooperation counselling
0
22 (2.2)
0.086
Group treatment
2 (1.5)
29 (3.2)
0.291
Child psychotherapy
1 (0.8)
4 (0.4)
0.621
Short term psychotherapy
4 (3.0)
18 (2.0)
0.432
Interaction treatment
1 (0.8)
12 (1.3)
0.587
Medication, excluding central
19 (14.4)
32 (3.5)
<0.001
stimulants
49
Conclusions, ADHD
ª  CGAS at baseline was lower compared to clinical trials
ª  The size of improvement (CGAS) was lower compared to
clinical trials
ª  Patients that received central stimulants received less
psychotherapy to family and parents and less parent training
§  Differs from clinical guidelines
ª  Routinely collected CGAS must include regular assessments
during on going treatment, every 3 or 6 months?
50
Paper IV
The Children’s Global Assessment Scale (CGAS)
predicts negative outcomes in early adulthood
Lundh A, Forsman M, Serlachius E, Långström N, Lichtenstein P, Landén M.
Submitted
51
Aim
ª  To investigate whether CGAS ratings can predict future
outcomes
§  criminal conviction
§  mental health disorders
§  suicide attempt
§  substance misuse
§  accidents
52
Methods
ª  4,876 patients, 1 July 2006 - 31 December 2009
ª  Treatment period more than one month
ª  Clinical database Pastill
§  CGAS at intake and end-of-treatment
ª  Follow-up 1½ years
ª  National registers
§  Patient Register
§  Crime Register
53
CGAS end-of-treatment ≤60 (n=2,260) vs
CGAS end-of-treatment >60 (n=2,616)
Outcomes
Unadjusted HR
Adjusted HR
Criminality
2.1 ***
2.4 **
Suicide attempt
3.8 **
1.1
Depression
2.6 ***
1.5
Anxiety disorder
2.6 ***
1.3
Eating disorder
3.4 *
8.1
Schizophrenia
N/A
N/A
Bipolar disorder
6.3 ***
4.7 *
BPD
11.7 ***
15.7 **
Substance misuse
1.9 **
1.2
Accidents
1.1
0.8
54
Conclusions, Paper IV
ª  CGAS ratings at end-of-treatment - but not at intake - provide
specific information about the outcome of child psychiatric
patients
ª  Intensified follow-up might be needed for adolescents with an
end-of-treatment CGAS score of 60 or less
55
Summary
ª  CGAS is a reliable and valid measure
ª  Suitable as routinely collected variable in large clinical settings
ª  Useful rating tool
§  Evaluate effectiveness of child psychiatric treatment
§  Predict negative outcomes
ª  Training effect
§  Future evaluation studies need randomised control groups to
control for unspecific learning effect, just like placebo
56
57
Thank you!
Mikael Landén
Carl Johan Sundberg
ª Mats Forsman
ª Paul Lichtenstein
ª Clara Gumpert
ª Niklas Långström
ª Jan Kowalski
ª Eva Serlachius
Namn Efternamn
januari 30, 2013
58
58