TF-CBT Program Performance
and Outcome Evaluation
Cricket Mitchell, PhD
Senior Associate, CIMH
TF-CBT Evaluation:
What You Should Have
• Guide to Evaluation and All Measures (PDF File)
– TFCBT Outcome Evaluation Guide and Forms-3
– “Quick-Start” instructions for evaluation
– Measures of symptom-specific and global youth
mental health functioning (PTSD-RI and YOQ)
• Database (Excel File)
– TFCBT Data Entry Shell2
– Holds all data for all children served
– Submit to CIMH twice a year; keep entering new data
into same database (ongoing record)
2
Overview of Training
• Standardized Measures Used in the TF-CBT
Evaluation
– UCLA Post Traumatic Stress Reaction Index©
(PTSD-RI)
• Description, Administration, Scoring, Clinical Utility
– Youth Outcome Questionnaires©
(YOQ & YOQ-SR)
• Description, Administration, Scoring, Clinical Utility
• Instructions for TF-CBT Data Entry & Data
Submissions
3
Overview of Training
• Standardized Measures Used in the
TF-CBT Evaluation
– UCLA Post Traumatic Stress Reaction Index©
(PTSD-RI)
• Description, Administration, Scoring, Clinical Utility
– Youth Outcome Questionnaires©
(YOQ & YOQ-SR)
• Description, Administration, Scoring, Clinical Utility
• Instructions for TF-CBT Data Submissions
4
Importance of Program Performance
and Outcome Evaluation
• Assessment is the beginning of developing
a relationship with the child and parents
– Demonstrates a desire to know what the child
is experiencing
– By incorporating standardized measures of
functioning, the efficiency and thoroughness of
assessment is enhanced
5
Importance of Program Performance
and Outcome Evaluation
• Using standardized measures of
functioning…
– Assists in initial clinical impressions
– Provides valuable information to guide
treatment/interventions
– Assesses sufficiency of treatment delivered
– Demonstrates treatment-related improvements
in child functioning
6
UCLA Post-Traumatic Stress
Reaction Index© (PTSD-RI)
7
CIMH PTSD-RI Training
• Content for today’s training courtesy of:
– National Center for Child Traumatic Stress at
UCLA
• 2004 article in Current Psychiatry Reports by Alan
Steinberg, Melissa Brymer, Kelly Decker and Robert
Pynoos (included in LA PEI MAP Evaluation Guide and
Forms PDF file)
– National Child Traumatic Stress Network
• Video-taped training on the administration and scoring
of the PTSD-RI (Alan Steinberg, William Saltzman and
Melissa Brymer)
– Personal communications with Laura Murray
8
PTSD-RI Description
• Symptom-specific measure of functioning
• Assesses the frequency of occurrence of
children’s post-traumatic stress reactions
– Parent/caregiver report for children 3 and older
– Self-report for children 7 and older
• Readability is age 12
• Preferred method of administration is to read aloud to
informant
• Sensitive to clinical change over time
• Valid and reliable
9
PTSD-RI Description
• Part I: 14 items
– Assesses lifetime history
of exposure to trauma
– Yes or No
– If more than one trauma,
focus on event most
currently bothersome
• Part II: 13 items
– Assesses objective and
subjective features of the
trauma exposure
– Maps on to DSM-IV
Criteria A1 & A2
• Part III: 20 items
– Assesses the frequency of
PTS symptoms during the
past month
– Maps on to DSM-IV
Criteria B, C & D
– 5-point Likert scale
response options
•
•
•
•
•
0 = None (of the time)
1 = Little (of the time)
2 = Some (of the time)
3 = Much (of the time)
4 = Most (of the time)
10
Let’s take a look at the
PTSD-RI...
(pgs 6-10 in the TF-CBT
Outcome Evaluation Guide and
Forms pdf)
11
PTSD-RI Administration
• Readability is age 12
– Preferred method of administration is to read
measure aloud to informant
• Ask parents and youth to respond to the
questionnaires as honestly as possible
– Informants can easily be influenced by the attitude of
the person administering the scale
– Let them know that this questionnaire will help you, as
a clinician, better understand how the child is doing
overall
• Ask parents and children to complete all items
– “Don’t Know” responses are not scored
12
PTSD-RI Administration
• Part I & II: Lifetime history of exposure
• Part III: “How much of the time during the past
month?”
– For Part III, guide the respondent through the
Frequency Rating Scale prior to administration
• Tear off Page 5 so they can see it while they respond
• Explain each response option
• Ensure understanding of each response option
– “Suppose I ask you how often in the past month you…
» … had a headache?”
» … did your homework?”
» … had green hair?”
13
Let’s take a look at the
PTSD-RI Scoring Worksheet...
(pg 27 in the TF-CBT Outcome
Evaluation Guide and Forms pdf)
14
PTSD-RI Scoring
• Each DSM-IV Criterion is established separately
• DSM-IV Criterion A
– Exposure to trauma (Part 1)
• At least 1 “Yes” on Q#s 1-13
– Criterion A1 (Part II)
• To be met, >1 “Yes” on Q#s 15-21
– Criterion A2 (Part II)
• To be met, >1 “Yes” on Q#s 22-26
– Criterion A
• To be met, exposure to trauma and A1 and A2 met
15
PTSD-RI Scoring
• DSM-IV Criteria B, C, and D
– Based on Part III: Questions 1-20
– Transfer each item’s response score onto the scoring
sheet next to the appropriate Question #
– PTSD-RI items map directly on to DSM-IV criteria
• Except Q14 & Q20 which assess associated features
• Severity score for each Criterion is the sum of the
items that map on to that Criterion
• Each Criterion is met (to assist in your diagnostic
impressions) if a minimum number of symptoms are
“present” (see slides 17-19)
– Symptom Cutoff Score >3
– A score of 3 or 4 (much or most of the time) indicates
that a symptom is “present”
16
PTSD-RI Scoring
DSM-IV Criterion B (re-experiencing)
Met if >1 of the 5 symptoms present* (*score 3 or 4)
DSM-IV Criteria
PTSD-RI Items
• B1) recurrent and intrusive thoughts
• Q#3*
• B2) recurrent distressing dreams
• Q#5
• B3) acting or feeling as if event recurring
• Q#6
• B4) intense psychological distress at cues • Q#2
• B5) physiological reactivity to cues
• Q#18
17
PTSD-RI Scoring
DSM-IV Criterion C (avoidance)
Met if >3 of the 7 symptoms present* (*score 3 or 4)
•
•
•
•
•
•
•
DSM-IV Criteria
C1) avoids thoughts, feelings or talks
C2) avoids activities, places or people
C3) inability to recall important aspect
C4) decreased interest in activities
C5) feelings of detachment
C6) restricted range of affect
C7) sense of foreshortened future
PTSD-RI Items
• Q#9
• Q#17
• Q#15
• Q#7
• Q#8
• Q#10 or Q#11
• Q#19
18
PTSD-RI Scoring
DSM-IV Criterion D (increased arousal)
Met if >2 of the 5 symptoms present* (*score 3 or 4)
•
•
•
•
•
DSM-IV Criteria
D1) difficulty falling or staying asleep
D2) irritability or outbursts of anger
D3) difficulty concentrating
D4) hypervigilance
D5) exaggerated startle response
PTSD-RI Items
• Q#13
• Q#4
• Q#16
• Q#1
• Q#12
19
PTSD-RI Scoring
PTSD Severity Overall/Total Score
• Based on Part III: Questions 1-20
• Transfer each item’s response score onto the scoring
sheet next to the appropriate Question #
– For one item score, transfer only the higher number of two
Question responses
• Q#s 10 or 11
• Note that the Parent Scoring Worksheet also states to select higher
score between Q#s 3 and 21.
– Items 14 and 20 are omitted (Associated Features)
• A total of 17 items are summed to determine the
PTSD Severity Overall/Total Score
• Post-TF-CBT administration only includes Part III – the
frequency of symptom items
20
Clinical Utility of the
PTSD-RI
• PTSD Severity Overall/Total Score
– Possible scores range from 0 to 68
– Clinical cutpoint is 38 or higher
– Scores in the high 20’s and 30’s indicate subclinical, yet significant levels of PTS reactions
that are appropriate for intervention
21
Clinical Utility of the
PTSD-RI
• Asks about a broad range of traumatic
events
– Primary reason for referral may not be only
history of trauma exposure
– Structured questions can help elicit additional
information about exposure to traumatic
events
– Often, children have not been asked directly
about traumatic events they’ve experienced
22
Clinical Utility of the
PTSD-RI
• Although not a diagnostic tool, the PTSD-RI
can inform clinical impressions
– Items map directly onto DSM-IV Criteria for
PTSD, 309.81
– Each item/question in Part III is labeled with
subscript indicating the specific DSM-IV 309.81
criterion (letter and number) that it assesses
• e.g., 1D4 … 3B1 … 7C4
• AF = Associated Feature (i.e., guilt, avoidance)
23
Clinical Utility of the
PTSD-RI
• Informs clinician about specific posttraumatic stress reactions that are most
bothersome to this particular child
– Helps prioritize symptoms for intervention
– Guides specific techniques that will be used
– Guides psycho-education
• Not all symptoms need to be normalized for each
child presenting with PTSD or PTS reactions
24
Clinical Utility of the
PTSD-RI
• Research has shown that certain types of
treatment approaches are better for certain
aspects of PTS symptoms
– e.g.,
• avoidance responds best to in vivo types of exposure
• sleep disturbances would suggest the use of
behavioral regimens and/or relaxation techniques
• significant rumination and self-blame would indicate
the need for cognitive interventions
25
Clinical Utility of the
PTSD-RI
• Comparisons of pre/post scores reveal
areas of clinical improvement
– e.g.,
• Does the child’s Overall/Total PTSD Severity
Score decrease substantially?
• Does the child’s symptomotology improve in all
domains of post-traumatic stress reactions?
26
Youth Outcome Questionnaires©
(YOQ & YOQ-SR)
27
CIMH YOQ & YOQ-SR Training
• Information on the administration, scoring,
and clinical utility of the YOQ & YOQ-SR
was obtained from each measure’s
respective Administration and Scoring
Manual published by OQ Measures, LLC
28
YOQ & YOQ-SR Description
• General measure of functioning
• Assesses the global mental health
functioning of children
– Parent/caregiver report for children ages 4-18
– Self-report for adolescents 12-18
• Sensitive to clinical change in short
periods of time
• Valid and reliable
29
YOQ & YOQ-SR Description
• Parallel versions of
the same measure
• 64 items
• 5-point Likert scale
response options*
–
–
–
–
–
• Six Scale Scores
–
–
–
–
–
–
Intrapersonal Distress (ID)
Somatic (S)
Interpersonal Relations (IR)
Social Problems (SP)
Behavioral Dysfunction (BD)
Critical Items (CI)
Never or Almost Never
Rarely
Sometimes
• Total Score
Frequently
Almost Always or
Always
*response values vary by item
30
Let’s take a look at the
YOQ-SR...
(pgs 37-38 in the TF-CBT
Outcome Evaluation Guide and
Forms pdf)
31
YOQ & YOQ-SR Administration
• “… during the past 7 days.”
• Ask parents and youth to fill out the
questionnaires as honestly as possible
– Informants can easily be influenced by the
attitude of the person administering the scale
– Let them know that this questionnaire will help
you, as a clinician, better understand how the
child is doing overall
• Ask parents and youth to complete all items
32
YOQ & YOQ-SR Scoring
• For each item, transfer the value corresponding to the
selected response into the box at the right-hand side of
the page titled, ‘For Office Use Only’
– Each item loads onto one of the six scales (e.g., ID, SP, IR)
– Note that some items have negative response option values
• Sum the items in each scale on Side1
– Add the numbers in all boxes under the heading ID and enter
that subtotal into the ID box at the bottom of the page
– Repeat for each scale
•
•
•
•
Sum the items in each scale on Side 2
Transfer the subtotals from Side 1 to Side 2
Sum the subtotals to determine Scale Scores
Sum the Scale Scores to determine Total Score
– Note that it is possible to have negative values for scores
33
YOQ & YOQ-SR Scoring
• Missing Data (items that are left blank)
– If 5 or more items are missing, consider the
questionnaire invalid
– Substitute a mean item response for the
missing item
• Determine in which scale the missing item belongs
• Add up the other items in that scale, and determine
their average
• Substitute the average score for the missing
response
34
Clinical Utility of the
YOQ & YOQ-SR
• Total Score
– Possible scores range from -16 to 240
– Clinical cutpoints
• 46 or higher on the YOQ
• 47 or higher on the YOQ-SR
– Lower scores indicate more normative, nonclinical, aspects of general mental health
functioning
• Elevations on certain scales indicate areas of
specific distress for the child
35
Clinical Utility of the
YOQ & YOQ-SR
• Intrapersonal Distress (ID) Scale
– Possible scores range from -4 to 68
– Clinical cutpoints
• 16 or higher on the YOQ
• 17 or higher on the YOQ-SR
– Assesses the amount of emotional distress in
the child, including anxiety, depression,
fearfulness, hopelessness, and self-harm
– High scores indicate a considerable degree of
intrapersonal distress in the child
36
Clinical Utility of the
YOQ & YOQ-SR
• Somatic (S) Scale
– Possible scores range from 0 to 32
– Clinical cutpoints
• 5 or higher on the YOQ
• 6 or higher on the YOQ-SR
– Indicates change in somatic distress or
physical complaints
– High scores indicate the parent/caregiver is
aware of, or the youth is experiencing, a high
number of somatic symptoms; while low scores
indicate either absence or unawareness of them
37
Clinical Utility of the
YOQ & YOQ-SR
• Interpersonal Relations (IR) Scale
– Possible scores range from -6 to 34
– Clinical cutpoints
• 4 or higher on the YOQ
• 3 or higher on the YOQ-SR
– Assesses issues relevant to the child’s
relationship with parents, other adults, and
peers
– High scores indicate significant interpersonal
difficulty; while low scores reflect a cooperative,
pleasant interpersonal demeanor
38
Clinical Utility of the
YOQ & YOQ-SR
• Social Problems (SP) Scale
– Possible scores range from -2 to 68
– Clinical cutpoints
• 3 or higher on the YOQ & YOQ-SR
– Assesses problems that are socially related
including aggression and delinquency
– A feature of these items is that they are slow to
change; whereas, content tapped by many of the
other scales often changes over a period of time
as a result of treatment intervention
39
Clinical Utility of the
YOQ & YOQ-SR
• Behavioral Dysfunction (BD) Scale
– Possible scores range from -4 to 40
– Clinical cutpoints
• 12 or higher on the YOQ
• 11 or higher on the YOQ-SR
– Assesses inattention, hyperactivity,
impulsivity, concentration, ability to
organize tasks, and ability to handle
frustration
40
Clinical Utility of the
YOQ & YOQ-SR
• Critical Items (CI) Scale
– Possible scores range from 0 to 36
– Clinical cutpoints
• 5 or higher on the YOQ
• 6 or higher on the YOQ-SR
– Assesses areas such as paranoia,
obsessive-compulsive behaviors,
hallucination, delusions, suicide, mania,
and eating disorders
41
Clinical Utility of the
YOQ & YOQ-SR
• Assesses a variety of specific areas of
difficulty in youth mental health functioning
• Assists in initial clinical impressions
• Provides valuable information to guide
treatment/interventions
• Comparisons of pre/post scores reveal
areas of clinical improvement as well as
areas of potential unmet need
42
Summary of YOQ/YOQ-SR Score
Ranges and Clinical Cutpoints
YOQ/YOQ-SR Scale
Range of
possible
scores
Clinical
Clinical
Cutpoint for Cutpoint for
YOQ
YOQ-SR
Intrapersonal Distress (ID)
-4 to 68
16
17
Somatic (S)
0 to 32
5
6
Interpersonal Relations (IR)
-6 to 34
4
3
Social Problems (SP)
-2 to 30
3
3
Behavioral Dysfunction (BD)
-4 to 40
12
11
0 to 36
5
6
-16 to 240
46
47
Critical Items (CI)
Total Score
43
TF-CBT Data Entry &
Data Submissions
44
TF-CBT Data Entry Shell2 –
‘Demos, etc’ Spreadsheet
Client Information:
Client
ID
Early Therapy Information:
(DSM-IV
code)
Date of Date of
Primary Therapist
First
DOB Gender Ethnicity Axis I
ID
Referral Session
End of Therapy Information:
Date of Total # Completed
Last
of
Session Sessions TF-CBT?
The last three fields are
for end-of-therapy
information.
Data should only be
entered when a child is no
longer in TF-CBT.
45
TF-CBT Data Entry Tips –
‘Demos, etc’ Spreadsheet (slide 1 of 2)
• Client ID automatically populates into the other
spreadsheets (pre and post outcome data on the
PTSD-RI and YOQ/YOQ-SR measures)
– Use a unique identifier for each child (e.g., chart #)
• Pull-down menus are provided for all categorical
data (gender, ethnicity, and Completed TF-CBT?)
• DSM-IV Axis I diagnosis should be entered as the
numerical code
– Enter only one DSM-IV numerical code
– If more than one Axis I diagnosis applies, enter the
Primary Axis I
46
TF-CBT Data Entry Tips –
‘Demos, etc’ Spreadsheet (slide 2 of 2)
• The last three fields (Date of Last Session, Total #
of Sessions, and Completed TF-CBT?) are end-oftherapy fields only
– They should be blank/empty for children currently
being seen
– They should be filled in for children no longer in TFCBT
• Completed TF-CBT?
– Select ‘yes’ if the child completed the intervention
– Select ‘no’ if the child dropped out of TF-CBT prior to
completing the intervention (e.g., moved away,
stopped coming)
47
TF-CBT Data Entry Excel File –
Pre & Post PTSD-RI Spreadsheets
Pre-TF-CBT
PARENT REPORT: CHILD REPORT:
PTSD-RI
Total
Total
Client ID
Severity Score
Severity Score
0
0
Post-TF-CBT
PARENT REPORT: CHILD REPORT:
PTSD-RI
Total
Total
Client ID
Severity Score
Severity Score
0
0
48
TF-CBT Data Entry Tips – Pre
and Post PTSD-RI Spreadsheets
• There is a separate spreadsheet for the
Pre- PTSD-RI data and the Post- PTSD-RI
data
– Enter the PTSD Overall/Severity Score for
the Parent and Child informants into their
respective fields
• Do not enter text into these fields
• If data are missing, leave the fields blank
49
TF-CBT Data Entry Excel File –
Pre & Post YOQ Spreadsheets
PrePARENT/CAREGIVER REPORT:
TF-CBT
YOQ (Scale Scores and Total Score)
Client
ID
ID
S
IR
SP
BD
YOUTH SELF-REPORT:
YOQ-SR (Scale Scores and Total Score)
CI Total ID
S
IR
SP
BD
CI Total
0
0
PostTF-CBT
PARENT/CAREGIVER REPORT:
YOUTH SELF-REPORT:
YOQ (Scale Scores and Total Score)
YOQ-SR (Scale Scores and Total Score)
Client ID ID
0
0
S
IR
SP
BD
CI Total ID
S
IR
SP
BD
CI Total
50
TF-CBT Data Entry Tips – Pre
and Post YOQ Spreadsheets
• There is a separate spreadsheet for the
Pre- YOQ & YOQ-SR data and the PostYOQ & YOQ-SR data
– Enter the Scale Scores and the Total Score
for the Parent and Adolescent informants
into their respective fields
• Do not enter text into these fields
• If data are missing, leave the fields blank
51
TF-CBT Data Submissions
• Data submissions occur twice a year
– The end of February (reflecting children served thru January)
– The end of August (reflecting children served through July)
– Note that this is the anticipated schedule; actual data
submission dates may vary slightly
• An email notice is sent approximately one month in
advance of each data submission deadline
• After data are submitted, sites continue to enter new data
into the same database (always reflects an ongoing,
historical record of children served)
• Every effort is made to distribute reports within two
months of data submission
– Aggregate and site-specific reports
52
Questions
53
The End
Contact Information
•Cricket Mitchell, PhD
•Email: cmitchell@cimh.org
•Cell phone: 858-220-6355
54