CiMH Palette of Measures
Evaluation Training: UCLA Post
Traumatic Stress Disorder Reaction
Index (PTSD-RI)
Cricket Mitchell, PhD
CIMH Evaluation Consultant
Palette of Measures Evaluation:
What You Will Need (slide 1 of 2)
• Palette of Measures Data Entry Shell v2 (Excel file)
– Developed by CiMH and customized for each
participating agency
– Holds all data for clients served thru the Palette of
Measures project
– Demographics, service delivery information, pre- and
post- outcome measure data
• Palette of Measures Data Dictionary v2 (Word
document)
– A guide for using the associated data entry shell
– Defines each column in the excel file
2
Palette of Measures Evaluation:
What You Will Need (slide 2 of 2)
• Outcome measures from the two-pronged
approach
– General Outcome Measure
– Target-Specific Outcome Measure(s)
– For example…
• UCLA Post Traumatic Stress Disorder
Reaction Index© (PTSD-RI)
– Available at no-cost to interested Palette of
Measures participating agencies under
permissions granted to CiMH by the developers of
the measure
3
Overview of Training
• Brief Overview of Palette of Measures evaluation
protocol
• UCLA Post Traumatic Stress Disorder Reaction
Index©: Target-specific measure for trauma
– Administration
– Scoring
– Clinical Utility
• Instructions for Palette of Measures data entry and
data submissions
– Data entry: PTSD-RI
– Data entry: Demographics & Services
– Data submissions to CiMH
4
Brief Overview of Palette of
Measures Evaluation Protocol
5
Outcome Assessment
• Palette of Measures providers will track outcomes
using data from pre- and post- administrations of
standardized measures of functioning
• Pre- and Post- a “dose” of treatment / an
intervention interval
– General measure of youth mental health functioning
(e.g., YOQ/YOQ-SR, CANS, Ohio Scales)
– Target-specific measure linked to focus of
treatment/intervention (e.g., AQ, ECBI, PHQ-9,
PTSD-RI, RCADS)
• Providers may choose to administer mid-course
assessments as well
– e.g., at 3-month intervals in usual care
6
A note about the use of standardized
assessment measures… (slide 1 of 2)
• Assessment is the beginning of developing
a relationship with the child and family
– Demonstrates a desire to know what the child
and family are experiencing
– By incorporating standardized assessment
measures of functioning, the efficiency and
thoroughness of assessment is enhanced
7
A note about the use of standardized
assessment measures… (slide 2 of 2)
• Using standardized assessment measures
of functioning…
– Assists in initial clinical impressions
– Provides valuable information to guide
treatment/interventions
– Assesses sufficiency of treatment delivered
– Demonstrates treatment-related improvements
in child functioning
8
UCLA Post-Traumatic Stress
Reaction Index© (PTSD-RI)
9
CiMH PTSD-RI Training
• Content for today’s training courtesy of:
– National Center for Child Traumatic Stress at
UCLA
• 2004 article in Current Psychiatry Reports by Alan
Steinberg, Melissa Brymer, Kelly Decker and Robert
Pynoos
– National Child Traumatic Stress Network
• Video-taped training on the administration and scoring
of the PTSD-RI (Alan Steinberg, William Saltzman and
Melissa Brymer)
– Personal communications with Laura Murray,
TF-CBT trainer and expert clinical consultant
10
PTSD-RI Description
• Target-specific measure for clients with
exposure to trauma
• Assesses the frequency of occurrence of
children’s post-traumatic stress reactions
– Parent/caregiver report for children age 3 and
older
– Self-report for children age 7 and older
• Available in English and Spanish
11
PTSD-RI Description
• Administration of the full measure takes 2030 minutes
– After initial assessment, only the symptom
severity portion need be administered again
• Part III ~ 10 minutes to complete
• Sensitive to clinical change over time
• Valid and reliable
• Available for use by CiMH partnering
agencies under permissions of the developer
12
PTSD-RI Description
• Part I: 14 items
– Assesses lifetime history
of exposure to trauma
– Yes or No
– If more than one trauma,
focus on event most
currently bothersome
• Part II: 13 items
– Assesses objective and
subjective features of the
trauma exposure
– Maps on to DSM-IV
Criteria A1 & A2
• Part III: 20 items
– Assesses the frequency of
PTS symptoms during the
past month
– Maps on to DSM-IV
Criteria B, C & D
– 5-point Likert scale
response options
•
•
•
•
•
0 = None (of the time)
1 = Little (of the time)
2 = Some (of the time)
3 = Much (of the time)
4 = Most (of the time)
13
Let’s take a look at the
PTSD-RI...
14
PTSD-RI Administration
• Readability is age 12
– Preferred method of administration is that the
measure be read aloud
– Can be completed independently
• All items should be completed
– “Don’t Know” responses are not scored
• Parents/caregivers may not know about their
child’s exposure to trauma and/or their child’s
symptom experience
15
PTSD-RI Administration
• Part I & II: Lifetime history of exposure
• Part III: “How much of the time during the past
month?”
– For Part III, guide the respondent through the
Frequency Rating Scale prior to administration
• Tear off Page 5 so they can see it while they respond
• Explain each response option
• Ensure understanding of each response option
– “Suppose I ask you how often in the past month you…
» … had a headache?”
» … did your homework?”
» … had green hair?”
16
PTSD-RI Administration
• Part I & II
– Administer pre- (only) a dose of treatment, or an
intervention interval, focused on trauma
• No need to re-administer after initial assessment
• Part III
– Administer pre- and post- a dose of treatment, or an
intervention interval, focused on trauma
• PTSD-RI child version completed by client
• PTSD-RI parent version completed by mother/mother
figure or father/father figure (when available)
– Both can be completed for clinical purposes; however, only
one should be entered for outcome tracking purposes
• Select parent/caregiver who spends most time with client
17
Let’s take a look at the
PTSD-RI Scoring Worksheet...
18
PTSD-RI Scoring
• There are two ways that item responses can
be scored
1. To assist with initial clinical impressions and
diagnostic formulation
•
Items map on to each of the four criteria in DSM-IV
for Post Traumatic Stress Disorder, 309.81
2. To assess the overall severity of current posttraumatic stress symptomotology
•
Sum of all symptom frequency items in Part III
19
PTSD-RI Scoring
• Each DSM-IV Criterion is established separately
• DSM-IV Criterion A
– Exposure to trauma (Part 1)
• At least 1 “Yes” on Q#s 1-13
– Criterion A1 (Part II)
• To be met, >1 “Yes” on Q#s 15-21
– Criterion A2 (Part II)
• To be met, >1 “Yes” on Q#s 22-26
– Criterion A
• To be met, exposure to trauma and A1 and A2 met
20
PTSD-RI Scoring
• DSM-IV Criteria B, C, and D
– Based on Part III: Questions 1-20
– Transfer each item’s response score onto the scoring
sheet next to the appropriate Question #
– PTSD-RI items map directly on to DSM-IV criteria
• Except Q14 & Q20 which assess associated features
• Severity score for each Criterion is the sum of the
items that map on to that Criterion
• Each Criterion is met (to assist in your diagnostic
impressions) if a minimum number of symptoms are
“present” (see slides 22-24)
– Symptom Cutoff Score >3
– A score of 3 or 4 (much or most of the time) indicates
that a symptom is “present”
21
PTSD-RI Scoring
DSM-IV Criterion B (re-experiencing)
Met if >1 of the 5 symptoms present* (*score 3 or 4)
DSM-IV Criteria
PTSD-RI Items
• B1) recurrent and intrusive thoughts
• Q#3*
• B2) recurrent distressing dreams
• Q#5
• B3) acting or feeling as if event recurring
• Q#6
• B4) intense psychological distress at cues • Q#2
• B5) physiological reactivity to cues
• Q#18
22
PTSD-RI Scoring
DSM-IV Criterion C (avoidance)
Met if >3 of the 7 symptoms present* (*score 3 or 4)
•
•
•
•
•
•
•
DSM-IV Criteria
C1) avoids thoughts, feelings or talks
C2) avoids activities, places or people
C3) inability to recall important aspect
C4) decreased interest in activities
C5) feelings of detachment
C6) restricted range of affect
C7) sense of foreshortened future
PTSD-RI Items
• Q#9
• Q#17
• Q#15
• Q#7
• Q#8
• Q#10 or Q#11
• Q#19
23
PTSD-RI Scoring
DSM-IV Criterion D (increased arousal)
Met if >2 of the 5 symptoms present* (*score 3 or 4)
•
•
•
•
•
DSM-IV Criteria
D1) difficulty falling or staying asleep
D2) irritability or outbursts of anger
D3) difficulty concentrating
D4) hypervigilance
D5) exaggerated startle response
PTSD-RI Items
• Q#13
• Q#4
• Q#16
• Q#1
• Q#12
24
PTSD-RI Scoring
PTSD Severity Overall/Total Score
• Based on Part III: Questions 1-20
• Transfer each item’s response score onto the
scoring sheet next to the appropriate Question #
– For one item score, transfer only the higher number of
two Question responses
• Q#s 10 or 11
• Note that the Parent Scoring Worksheet also states to select
higher score between Q#s 3 and 21.
– Items 14 and 20 are omitted (Associated Features)
• A total of 17 items are summed to determine the
PTSD Severity Overall/Total Score
25
Clinical Utility of the
PTSD-RI
• PTSD Severity Overall/Total Score
– Possible scores range from 0 to 68
– Clinical cutpoint is 38 or higher
– Scores in the high 20’s and 30’s indicate subclinical, yet significant levels of PTS reactions
that are appropriate for intervention
26
Clinical Utility of the
PTSD-RI
• Asks about a broad range of traumatic
events
– Primary reason for referral may not be only
history of trauma exposure
– Structured questions can help elicit additional
information about exposure to traumatic
events
– Often, children have not been asked directly
about traumatic events they’ve experienced
27
Clinical Utility of the
PTSD-RI
• Although not a diagnostic tool, the PTSD-RI
can inform clinical impressions
– Items map directly onto DSM-IV Criteria for
PTSD, 309.81
– Each item/question in Part III is labeled with
subscript indicating the specific DSM-IV 309.81
criterion (letter and number) that it assesses
• e.g., 1D4 … 3B1 … 7C4
• AF = Associated Feature (i.e., guilt, avoidance)
28
Clinical Utility of the
PTSD-RI
• Informs clinician about specific posttraumatic stress reactions that are most
bothersome to this particular child
– Helps prioritize symptoms for intervention
– Guides specific techniques that will be used
– Guides psycho-education
• Not all symptoms need to be normalized for each
child presenting with PTSD or PTS reactions
29
Clinical Utility of the
PTSD-RI
• Research has shown that certain types of
treatment approaches are better for certain
aspects of PTS symptoms
– e.g.,
• avoidance responds best to in vivo types of exposure
• sleep disturbances would suggest the use of
behavioral regimens and/or relaxation techniques
• significant rumination and self-blame would indicate
the need for cognitive interventions
30
Clinical Utility of the
PTSD-RI
• Comparisons of pre/post scores reveal
areas of clinical improvement
– e.g.,
• Does the child’s Overall/Total PTSD Severity
Score decrease substantially?
• Does the child’s symptomotology improve in all
domains of post-traumatic stress reactions?
31
Palette of Measures Data Entry
and Data Submissions
32
Palette of Measures Data Entry
Shell
• There is a separate spreadsheet in the excel
workbook (aka database) for each type of
data:
–
–
–
–
–
Demographics & Services
Pre- General outcome measure
Post- General outcome measure
Pre- Target-specific measure(s)
Post- Target-specific outcome measure(s)
• Specific outcome measure spreadsheets included in
each agency’s database varies across Palette of
Measures project participants
33
Palette of Measures Data Entry
Shell
• In addition to the spreadsheets that hold
data...
– There is an Instructions spreadsheet
• Basic data entry instructions
• Contact information for T.A. (Cricket Mitchell)
– There is a Data Lists spreadsheet at the end
of the workbook that you will not use
• Data Lists populate the pull-down menus in other
spreadsheets
34
Palette of Measures Data Entry:
PTSD-RI
Pre-PTSD-RI
Date of
Client ID#
Assessment
Parent Report:
(ages 3-18)
P-PTSD1
Self-Report:
(ages 7-18)
C-PTSD1
•There is a separate spreadsheet for Pre-PTSD-RI data,
Post-PTSD-RI data, and Mid-PTSD-RI data
•In each spreadsheet, there is a separate field for the
Total/Overall Severity Score for each informant
•In the event of missing data, leave the fields blank/empty.
Do not enter text into any of the fields.
35
Palette of Measures Data Entry:
PTSD-RI
Mid-PTSD-RI
Date of
Client ID#
Assessment
Parent Report:
Assessment
(ages 3-18)
Interval
P-PTSDM
Self-Report:
(ages 7-18)
C-PTSDM
•For agencies who will conduct mid-treatment assessments,
indicate the Assessment Interval in the Mid-PTSD-RI
spreadsheet by selecting from the available pull-down menu
•e.g., 1st mid-treatment assessment, 2nd mid-treatment
assessment
36
But, before you enter any
outcome data, you’ll enter
Demographics & Service
Delivery Information...
37
Palette of Measures Data Entry:
Demographics & Services (1 of 2)
Client Information:
Primary (DSM-IV code) (DSM-IV code)
Client ID#
DOB
Gender Ethnicity Language Primary Axis I Secondary Axis I Therapist ID
•Use a unique identifier for Client ID#
•Categorical variables will have pull-down menus from which
you’ll select an option (e.g., gender, ethnicity, language)
•Dates should be entered as xx/xx/xxxx
•Axis I diagnoses s/b the numeric DSM-IV code
•Therapist ID is optional
38
Palette of Measures Data Entry:
Demographics & Services (2 of 2)
Focus 1 of Treatment
Focus1
Date of
First
Session
Date of Total # Completed
Last
of
Session Sessions Focus 1?
(if Focus 1 not
completed)
Completed
(if Services not
completed)
Reason
Services?
Reason
•Select Focus from pull-down menu (e.g., anxiety, depression)
•The shell will hold data for up to 4 foci, or treatment targets
•Enter Date of First Session
•The remaining fields are to be completed at the end of treatment
targeting this particular focus (e.g., Date of Last, # Sessions)
• 2 levels of “Completed?”
• Treatment targeting this particular focus
• Overall service delivery
39
Palette of Measures Data
Submissions (slide 1 of 3)
• Data submissions to CiMH will occur twice a year
throughout the duration of the project
– The end of each May (reflecting all clients served from the
initiation of the project through the end of that April)
– The end of each December (reflecting all clients served
from the initiation of the project through the end of that
November)
– Note that this is the anticipated schedule; actual data
submission dates may vary slightly
• An email notice will be sent to Palette of Measures
site leads approximately one month in advance of
each data submission deadline
40
Data Submissions (slide 2 of 3)
• Providers may choose from among the following
methods for submitting their Palette of Measures
Excel databases to CiMH:
– Use YouSendIt, or another secure web-based transfer
site, to submit data electronically
• YouSendIt (www.YouSendIt.com) is a vendor that supports the
secure transfer of electronic data (encrypted and passwordprotected)
– Mail a password-protected CD to CiMH and submit the
password separately (via email or phone)
– Email an encrypted, password-protected file(s) to CiMH
and submit the password separately (via email or phone)
41
Data Submissions (slide 3 of 3)
• After data are submitted, sites continue to
enter new data into the same database
– Always reflects an ongoing, historical record of
clients served through the Palette of Measures
project
• Every effort is made to distribute reports
within two months of each data submission
– Aggregate and site/agency-specific reports
42
Questions
43
The End
Contact Information
•Cricket Mitchell, PhD
•Email: cmitchell@cimh.org
•Cell phone: 858-220-6355
44