CiMH Palette of Measures
Evaluation Training: Youth Outcome
Questionnaires (YOQs)
Cricket Mitchell, PhD
CIMH Evaluation Consultant
Palette of Measures Evaluation:
What You Will Need (slide 1 of 2)
• Palette of Measures Data Entry Shell v2 (Excel file)
– Developed by CiMH and customized for each
participating agency
– Holds all data for clients served thru the Palette of
Measures project
– Demographics, service delivery information, pre- and
post- outcome measure data
• Palette of Measures Data Dictionary v2 (Word
document)
– A guide for using the associated data entry shell
– Defines each column in the excel file
2
Palette of Measures Evaluation:
What You Will Need (slide 2 of 2)
• Outcome measures from the two-pronged
approach
– General Outcome Measure
– Target-Specific Outcome Measure(s)
– For example…
• Youth Outcome Questionnaire © (YOQ) and
Youth Outcome Questionnaire – Self Report ©
(YOQ-SR)
– Provided to agencies participating in the Palette of
Measures project under CiMH’s statewide license
3
Overview of Training
• Brief Overview of Palette of Measures evaluation
protocol
• Youth Outcome Questionnaires© (YOQ & YOQ-SR):
General outcome measures of youth mental health
functioning
– Administration
– Scoring
– Clinical Utility
• Instructions for Palette of Measures data entry and
data submissions
– Data entry: YOQ/YOQ-SR
– Data entry: Demographics & Services
– Data submissions to CiMH
4
Brief Overview of Palette of
Measures Evaluation Protocol
5
Outcome Assessment
• Palette of Measures providers will track outcomes
using data from pre- and post- administrations of
standardized measures of functioning
• Pre- and Post- a “dose” of treatment / an
intervention interval
– General measure of youth mental health functioning
(e.g., YOQ/YOQ-SR, CANS, Ohio Scales)
– Target-specific measure linked to focus of
treatment/intervention (e.g., AQ, ECBI, PHQ-9,
PTSD-RI, RCADS)
• Providers may choose to administer mid-course
assessments as well
– e.g., at 3-month intervals in usual care
6
A note about the use of standardized
assessment measures… (slide 1 of 2)
• Assessment is the beginning of developing
a relationship with the child and family
– Demonstrates a desire to know what the child
and family are experiencing
– By incorporating standardized assessment
measures of functioning, the efficiency and
thoroughness of assessment is enhanced
7
A note about the use of standardized
assessment measures… (slide 2 of 2)
• Using standardized assessment measures
of functioning…
– Assists in initial clinical impressions
– Provides valuable information to guide
treatment/interventions
– Assesses sufficiency of treatment delivered
– Demonstrates treatment-related improvements
in child functioning
8
Youth Outcome Questionnaires©
(YOQ & YOQ-SR)
9
CiMH YOQ & YOQ-SR Training
• Information on the administration, scoring,
and clinical utility of the YOQ & YOQ-SR
was obtained from each measure’s
respective Administration and Scoring
Manual published by OQ Measures, LLC
10
YOQ & YOQ-SR Description
• General measure of functioning
• Assesses the global mental health
functioning of children
– Parent/caregiver report for children ages 4-18
– Self-report for adolescents 12-18
– Adult self-report is also available for 19+
• Available in English and Spanish
11
YOQ & YOQ-SR Description
• 5-15 minutes to complete
• Sensitive to clinical change in short periods of
time
– Regardless of treatment modality, diagnosis, or
discipline of the treating professional
• Valid and reliable
• Provided to partner agencies under CiMH’s
statewide license
– Partnering agencies sign a User’s Agreement and
return it to CIMH
12
YOQ & YOQ-SR Description
• Parallel versions of
the same measure
• 64 items
• 5-point Likert scale
response options*
–
–
–
–
–
• Six Scale Scores
–
–
–
–
–
–
Intrapersonal Distress (ID)
Somatic (S)
Interpersonal Relations (IR)
Social Problems (SP)
Behavioral Dysfunction (BD)
Critical Items (CI)
Never or Almost Never
Rarely
Sometimes
• Total Score
Frequently
Almost Always or
Always
*response values vary by item
13
YOQ & YOQ-SR Administration
• Administer pre- and post- a dose of
treatment, or an intervention interval
– YOQ-SR completed by adolescent client
– YOQ completed by mother/mother figure or
father/father figure (when available)
• Both can be completed for clinical purposes;
however, only one should be entered for outcome
tracking purposes
– Select parent/caregiver who spends most time with client
14
Let’s take a look at the
YOQ-SR...
15
YOQ & YOQ-SR Administration
• “… during the past 7 days.”
• Ask parents and youth to fill out the
questionnaires as honestly as possible
– Informants can easily be influenced by the
attitude of the person administering the scale
– Let them know that this questionnaire will help
you, as a clinician, better understand how the
child is doing overall
• Ask parents and youth to complete all items
16
YOQ & YOQ-SR Scoring
• For each item, transfer the value corresponding to the
selected response into the box at the right-hand side of
the page titled, ‘For Office Use Only’
– Each item loads onto one of the six scales (e.g., ID, SP, IR)
– Note that some items have negative response option values
• Sum the items in each scale on Side1
– Add the numbers in all boxes under the heading ID and enter
that subtotal into the ID box at the bottom of the page
– Repeat for each scale
•
•
•
•
Sum the items in each scale on Side 2
Transfer the subtotals from Side 1 to Side 2
Sum the subtotals to determine Scale Scores
Sum the Scale Scores to determine Total Score
– Note that it is possible to have negative values for scores
17
YOQ & YOQ-SR Scoring
• Missing Data (items that are left blank)
– If 5 or more items are missing, consider the
questionnaire invalid
– Substitute a mean item response for the
missing item
• Determine in which scale the missing item belongs
• Add up the other items in that scale, and determine
their average
• Substitute the average score for the missing
response
18
Clinical Utility of the
YOQ & YOQ-SR
• Total Score
– Possible scores range from -16 to 240
– Clinical cutpoints
• 46 or higher on the YOQ
• 47 or higher on the YOQ-SR
– Lower scores indicate more normative, nonclinical, aspects of general mental health
functioning
• Elevations on certain scales indicate areas of
specific distress for the child/youth
19
Clinical Utility of the
YOQ & YOQ-SR
• Intrapersonal Distress (ID) Scale
– Possible scores range from -4 to 68
– Clinical cutpoints
• 16 or higher on the YOQ
• 17 or higher on the YOQ-SR
– Assesses the amount of emotional distress in
the child/youth, including anxiety, depression,
fearfulness, hopelessness, and self-harm
– High scores indicate a considerable degree of
intrapersonal distress in the child/youth
20
Clinical Utility of the
YOQ & YOQ-SR
• Somatic (S) Scale
– Possible scores range from 0 to 32
– Clinical cutpoints
• 5 or higher on the YOQ
• 6 or higher on the YOQ-SR
– Indicates change in somatic distress or
physical complaints
– High scores indicate the parent/caregiver is
aware of, or the youth is experiencing, a high
number of somatic symptoms; while low scores
indicate either absence or unawareness of them
21
Clinical Utility of the
YOQ & YOQ-SR
• Interpersonal Relations (IR) Scale
– Possible scores range from -6 to 34
– Clinical cutpoints
• 4 or higher on the YOQ
• 3 or higher on the YOQ-SR
– Assesses issues relevant to the child/youth’s
relationship with parents, other adults, and
peers
– High scores indicate significant interpersonal
difficulty; while low scores reflect a cooperative,
pleasant interpersonal demeanor
22
Clinical Utility of the
YOQ & YOQ-SR
• Social Problems (SP) Scale
– Possible scores range from -2 to 68
– Clinical cutpoints
• 3 or higher on the YOQ & YOQ-SR
– Assesses problems that are socially related
including aggression and delinquency
– A feature of these items is that they are slow to
change; whereas, content tapped by many of the
other scales often changes over a period of time
as a result of treatment intervention
23
Clinical Utility of the
YOQ & YOQ-SR
• Behavioral Dysfunction (BD) Scale
– Possible scores range from -4 to 40
– Clinical cutpoints
• 12 or higher on the YOQ
• 11 or higher on the YOQ-SR
– Assesses inattention, hyperactivity,
impulsivity, concentration, ability to
organize tasks, and ability to handle
frustration
24
Clinical Utility of the
YOQ & YOQ-SR
• Critical Items (CI) Scale
– Possible scores range from 0 to 36
– Clinical cutpoints
• 5 or higher on the YOQ
• 6 or higher on the YOQ-SR
– Assesses areas such as paranoia,
obsessive-compulsive behaviors,
hallucination, delusions, suicide, mania,
and eating disorders
25
Clinical Utility of the
YOQ & YOQ-SR
• Assesses a variety of specific areas of
difficulty in child/youth mental health
functioning
• Assists in initial clinical impressions
• Provides valuable information to guide
treatment/interventions
• Comparisons of pre/post scores reveal areas
of clinical improvement as well as areas of
potential unmet need
26
Summary of YOQ/YOQ-SR Score
Ranges and Clinical Cutpoints
YOQ/YOQ-SR Scale
Range of
possible
scores
Clinical
Clinical
Cutpoint for Cutpoint for
YOQ
YOQ-SR
Intrapersonal Distress (ID)
-4 to 68
16
17
Somatic (S)
0 to 32
5
6
Interpersonal Relations (IR)
-6 to 34
4
3
Social Problems (SP)
-2 to 30
3
3
Behavioral Dysfunction (BD)
-4 to 40
12
11
0 to 36
5
6
-16 to 240
46
47
Critical Items (CI)
Total Score
27
Palette of Measures Data Entry
and Data Submissions
28
Palette of Measures Data Entry
Shell
• There is a separate spreadsheet in the excel
workbook (aka database) for each type of
data:
–
–
–
–
–
Demographics & Services
Pre- General outcome measure
Post- General outcome measure
Pre- Target-specific measure(s)
Post- Target-specific outcome measure(s)
• Specific outcome measure spreadsheets included in
each agency’s database varies across Palette of
Measures project participants
29
Palette of Measures Data Entry
Shell
• In addition to the spreadsheets that hold
data...
– There is an Instructions spreadsheet
• Basic data entry instructions
• Contact information for T.A. (Cricket Mitchell)
– There is a Data Lists spreadsheet at the end
of the workbook that you will not use
• Data Lists populate the pull-down menus in other
spreadsheets
30
Palette of Measures Data Entry:
YOQ/YOQ-SR
Pre-YOQs
PARENT/CAREGIVER REPORT: (ages 4-18)
Date of
Client ID#
Assessment
YOQ (Scale Scores and Total Score)
YOQ1-ID YOQ1-S YOQ1-IR YOQ1-SP YOQ1-BD YOQ1-CI YOQ1-Total
…
•There is a separate spreadsheet for Pre-YOQ data, Post-YOQ
data, and Mid-YOQ data
•In each spreadsheet, there is a separate field for each of the
YOQ and YOQ-SR Scale Scores and Total Scores
•In the event of missing data, leave the fields blank/empty. Do
not enter text into any of the fields.
31
Palette of Measures Data Entry:
YOQ/YOQ-SR
Mid-YOQs
Date of
Client ID#
Assessment YOQ (Scale Scores and Total Score)
YOQM- YOQM- YOQM- YOQM- YOQM- YOQM- YOQMAssessment Interval
ID
S
IR
SP
BD
CI
Total
•For agencies who will conduct mid-treatment assessments,
indicate the Assessment Interval in the Mid-YOQs
spreadsheet by selecting from the available pull-down menu
•e.g., 1st mid-treatment assessment, 2nd mid-treatment
assessment
32
But, before you enter any
outcome data, you’ll enter
Demographics & Service
Delivery Information...
33
Palette of Measures Data Entry:
Demographics & Services (1 of 2)
Client Information:
Primary (DSM-IV code) (DSM-IV code)
Client ID#
DOB
Gender Ethnicity Language Primary Axis I Secondary Axis I Therapist ID
•Use a unique identifier for Client ID#
•Categorical variables will have pull-down menus from which
you’ll select an option (e.g., gender, ethnicity, language)
•Dates should be entered as xx/xx/xxxx
•Axis I diagnoses s/b the numeric DSM-IV code
•Therapist ID is optional
34
Palette of Measures Data Entry:
Demographics & Services (2 of 2)
Focus 1 of Treatment
Focus1
Date of
First
Session
Date of Total # Completed
Last
of
Session Sessions Focus 1?
(if Focus 1 not
completed)
Completed
(if Services not
completed)
Reason
Services?
Reason
•Select Focus from pull-down menu (e.g., anxiety, depression)
•The shell will hold data for up to 4 foci, or treatment targets
•Enter Date of First Session
•The remaining fields are to be completed at the end of treatment
targeting this particular focus (e.g., Date of Last, # Sessions)
• 2 levels of “Completed?”
• Treatment targeting this particular focus
• Overall service delivery
35
Palette of Measures Data
Submissions (slide 1 of 3)
• Data submissions to CiMH will occur twice a year
throughout the duration of the project
– The end of each May (reflecting all clients served from the
initiation of the project through the end of that April)
– The end of each December (reflecting all clients served
from the initiation of the project through the end of that
November)
– Note that this is the anticipated schedule; actual data
submission dates may vary slightly
• An email notice will be sent to Palette of Measures
site leads approximately one month in advance of
each data submission deadline
36
Data Submissions (slide 2 of 3)
• Providers may choose from among the following
methods for submitting their Palette of Measures
Excel databases to CiMH:
– Use YouSendIt, or another secure web-based transfer
site, to submit data electronically
• YouSendIt (www.YouSendIt.com) is a vendor that supports the
secure transfer of electronic data (encrypted and passwordprotected)
– Mail a password-protected CD to CiMH and submit the
password separately (via email or phone)
– Email an encrypted, password-protected file(s) to CiMH
and submit the password separately (via email or phone)
37
Data Submissions (slide 3 of 3)
• After data are submitted, sites continue to
enter new data into the same database
– Always reflects an ongoing, historical record of
clients served through the Palette of Measures
project
• Every effort is made to distribute reports
within two months of each data submission
– Aggregate and site/agency-specific reports
38
Questions
39
The End
Contact Information
•Cricket Mitchell, PhD
•Email: cmitchell@cimh.org
•Cell phone: 858-220-6355
40