CiMH Palette of Measures
Evaluation Training: How I Think
Questionnaire (HIT)
Cricket Mitchell, PhD
CIMH Evaluation Consultant
Palette of Measures Evaluation:
What You Will Need (slide 1 of 2)
• Palette of Measures Data Entry Shell v2 (Excel file)
– Developed by CiMH and customized for each
participating agency
– Holds all data for clients served thru the Palette of
Measures project
– Demographics, service delivery information, pre- and
post- outcome measure data
• Palette of Measures Data Dictionary v2 (Word
document)
– A guide for using the associated data entry shell
– Defines each column in the excel file
2
Palette of Measures Evaluation:
What You Will Need (slide 2 of 2)
• Outcome measures from the two-pronged
approach
– General Outcome Measure
– Target-Specific Outcome Measure(s)
– For example…
• How I Think Questionnaire © (HIT)
– Copyrighted and published by Research Press
• Must be purchased by interested Palette of
Measures project participating agencies
3
Overview of Training
• Brief Overview of Palette of Measures evaluation
protocol
• How I Think Questionnaire©: Target-specific
measure for disruptive behavior
– Administration
– Scoring
– Clinical Utility
• Instructions for Palette of Measures data entry and
data submissions
– Data entry: HIT
– Data entry: Demographics & Services
– Data submissions to CiMH
4
Brief Overview of Palette of
Measures Evaluation Protocol
5
Outcome Assessment
• Palette of Measures providers will track outcomes
using data from pre- and post- administrations of
standardized measures of functioning
• Pre- and Post- a “dose” of treatment / an
intervention interval
– General measure of youth mental health functioning
(e.g., YOQ/YOQ-SR, CANS, Ohio Scales)
– Target-specific measure linked to focus of
treatment/intervention (e.g., AQ, ECBI, PHQ-9,
PTSD-RI, RCADS)
• Providers may choose to administer mid-course
assessments as well
– e.g., at 3-month intervals in usual care
6
A note about the use of standardized
assessment measures… (slide 1 of 2)
• Assessment is the beginning of developing
a relationship with the child and family
– Demonstrates a desire to know what the child
and family are experiencing
– By incorporating standardized assessment
measures of functioning, the efficiency and
thoroughness of assessment is enhanced
7
A note about the use of standardized
assessment measures… (slide 2 of 2)
• Using standardized assessment measures
of functioning…
– Assists in initial clinical impressions
– Provides valuable information to guide
treatment/interventions
– Assesses sufficiency of treatment delivered
– Demonstrates treatment-related improvements
in child functioning
8
How I Think Questionnaire© (HIT)
9
CiMH HIT Training
• Information on the administration, scoring,
and clinical utility of the HIT was obtained
from the measure’s Manual, written by
Alvaro Q. Barriga, John C. Gibbs,
Granville Bud Potter and Albert K. Liau,
and published by Research Press
10
©
Questionnaire
How I Think
(HIT) Description
• Target-specific measure for disruptive
behavior
• Measures youth tendencies to engage in
self-serving cognitive distortions, or
thinking errors
– Youth self-report for ages 12-19
• 5-15 minutes to complete
11
©
Questionnaire
How I Think
(HIT) Description
• Written at a 4th grade reading level
• Valid and reliable
• Copyrighted and published by Research
Press
– After purchase of the manual (~$2) the cost per
use ~$1.20
• **20% discount if purchased through CIMH
– http://www.researchpress.com/product/item/5136/
12
HIT© Description
• 54 items
• 6-point Likert scale
response options
–
–
–
–
–
–
Disagree strongly (1)
Disagree (2)
Disagree slightly (3)
Agree slightly (4)
Agree (5)
Agree strongly (6)
• Eight Subscale Scores
–
–
–
–
–
–
–
–
Self-Centered (SC)
Blaming Others (BO)
Minimizing/Mislabeling (MM)
Assuming the Worst (AW)
Opposition-Defiance (OD)
Physical Aggression (PA)
Lying (L)
Stealing (S)
• Two Summary Scale Scores
– Overt
– Covert
• HIT Total Score
13
Example: Items from the HIT©
• People should try to work on their
problems.
• If I see something I like, I take it.
• When I get mad, I don’t care who gets
hurt.
• Everybody lies, it’s no big deal.
• You should get what you need, even if it
means someone has to get hurt.
14
HIT Administration
• Administer pre- and post- a dose of
treatment, or an intervention interval,
focused on disruptive behavior
– HIT completed by client
– Some agencies may choose to administer
mid-treatment assessments as well
15
Let’s take a look at the
HIT...
(not distributed due to copyright laws)
16
How I Think Questionnaire© Scoring
• Use the HIT Questionnaire Computational Form
– Only one is provided with each HIT manual, this form will
need to be copied to score each HIT
• For each item, transfer the value corresponding to
the circled response into box under the appropriate
scale (item # in parentheses)
– Values for each item response are on slide 13
– Each item loads onto one of the nine scales
• Determine each scale’s Mean score
– Sum all responses and divide by the number of items in
that scale
• Determine the Mean Summary Scores (e.g. Overt
Scale, Covert Scale, Overall HIT Score)
17
HIT Scoring
• Missing Data (items that are left blank)
– If 5 or more items are missing, consider the
questionnaire invalid
– If an item is left blank, it is not coded and will not
contribute to the scale score in which it falls
• Note that this will change the denominator (divisor) in
determining the scale score, which is the mean, or
average of all scale items
• Two Responses to One Question
– If a youth has circled two responses to an item, take
the higher value of the two
18
HIT Scoring
• Use the HIT Questionnaire Profile Form to
plot a youth’s scores
– As with the HIT Questionnaire Computational
Form, only one is provided with each HIT
manual, this form will need to be copied to score
each HIT
• The Profile Form shows youth scores that
are above the clinical cutpoint, are
borderline-clinical, or are in the nonclinical
range
19
Clinical Utility of the
HIT
• HIT Total Score/Overall HIT Score
– Possible scores range from 1.0 to 6.0
– Clinical cutpoint
• 2.98 or higher
– Total Scores are reflective of self-serving cognitive
distortions, or thinking errors, reported by youth
– High scores reflect more cognitive distortions that are
likely related to antisocial behaviors; whereas, low
scores reflect more normative, non-clinical thought
processes
• It is important to examine the profile of elevations
across all summary scales and subscales
20
Clinical Utility of the
HIT
• HIT Subscale Scores
– Possible scores range from 1.0 to 6.0
– Clinical cutpoints vary (see slide 26)
– Each subscale is named for its item content
• e.g., the Blaming Others subscale contains items that
assess youth tendencies to blame others for their
actions, the Lying subscale contains items that assess
youth condoning and engaging in lying
– Higher scores reflect a higher endorsement of
that thought process or behavior
21
Clinical Utility of the
HIT
• HIT Subscale Scores
– Examining the profile of cognitive distortion
scales should provide a method for directly
assessing the particular cognitive style
displayed by a youth
– Permits clinicians to individualize treatment
approaches more effectively
• e.g., youth with elevated Self-Centered scores are
likely to require cognitive interventions that
address a strong egocentric bias
22
Clinical Utility of the
HIT
• Overt Summary Scale
• Opposition-Defiance (OD) and
• Physical Aggression (PA)
– The Overt Scale reflects a predilection for antisocial
behavior that typically involves confrontation of a
victim
• Covert Summary Scale
• Lying (L) and
• Stealing (S)
– The Covert Scale reflects a preference for antisocial
behavior that is primarily nonconfrontational
23
Clinical Utility of the
HIT
• Treatment planning will likely vary for
youth with high Overt Scale scores vs.
youth with high Covert Scale scores
– Numerous studies have found differences in
the etiological factors, symptom presentation,
and long-term prognosis between youth who
display overt aggression and youth who
engage in delinquency that is more covert
24
Clinical Utility of the
How I Think Questionnaire©
• Assesses a variety of specific aspects of selfserving youth thinking errors that are likely to
contribute to aggression and delinquent behavior
– e.g., blaming others, assuming the worst, minimizing
and mislabeling
• Informs which aspects of thinking errors and
associated antisocial behavior youth have the
greatest need for intervention
• Comparisons of pre/post scores demonstrate
treatment-related outcomes in maladaptive
thought processes
25
Summary of HIT Score Ranges
and Clinical Cutpoints
HIT Score
(range 1-6)
Clinical
Cutpoint
HIT Score
(range 1-6)
Clinical
Cutpoint
Self-Centered (SC)
3.16
Lying (L)
3.39
Blaming Others (BO)
3.08
Stealing (S)
2.57
Minimizing/Mislabeling (MM)
2.96
Assuming the Worst (AW)
2.92
Overt Summary Score
3.06
Opposition-Defiance (OD)
3.20
Covert Summary Score
2.96
Physical Aggression (PA)
3.01
HIT Total Score
2.98
26
Palette of Measures Data Entry
and Data Submissions
27
Palette of Measures Data Entry
Shell
• There is a separate spreadsheet in the excel
workbook (aka database) for each type of
data:
–
–
–
–
–
Demographics & Services
Pre- General outcome measure
Post- General outcome measure
Pre- Target-specific measure(s)
Post- Target-specific outcome measure(s)
• Specific outcome measure spreadsheets included in
each agency’s database varies across Palette of
Measures project participants
28
Palette of Measures Data Entry
Shell
• In addition to the spreadsheets that hold
data...
– There is an Instructions spreadsheet
• Basic data entry instructions
• Contact information for T.A. (Cricket Mitchell)
– There is a Data Lists spreadsheet at the end
of the workbook that you will not use
• Data Lists populate the pull-down menus in other
spreadsheets
29
Palette of Measures Data Entry:
HIT
Assessm Self-Report (ages 12-19)
Mid-HIT Date of
ent How I Think Questionnaire Mean Total, Scale, and Subscale Scores
HIT- HIT- HIT- HIT- HITHIT- HIT- HIT- HITAssessm
HIT HIT- COV SC BO MM AW HIT- PA LY
ST AR
Client ID# ent
Interval TotM OVM M M M M M ODM M M M M
•There is a separate spreadsheet for Pre-HIT data, Post-HIT
data, and Mid-HIT data
•In each spreadsheet, there is a separate field for each of the
HIT Scale Scores, Subscale Scores, and Total Scores
•In the event of missing data, leave the fields blank/empty.
Do not enter text into any of the fields.
30
Palette of Measures Data Entry:
HIT
Assess- Self-Report (ages 12-19)
Date of ment How I Think Questionnaire Mean Total, Scale, and Subscale Scores
HIT HIT- HIT- HIT- HIT- HIT- HIT- HIT- HIT- HIT- HIT- HITAssess- Interval Tot OV COV SC BO MM AW OD PA LY
ST AR
Client ID# ment
M
M
M
M
M
M
M
M
M
M
M
M
MidHIT
•For agencies who will conduct mid-treatment assessments,
indicate the Assessment Interval in the Mid-HIT spreadsheet
by selecting from the available pull-down menu
•e.g., 1st mid-treatment assessment, 2nd mid-treatment
assessment
31
But, before you enter any
outcome data, you’ll enter
Demographics & Service
Delivery Information...
32
Palette of Measures Data Entry:
Demographics & Services (1 of 2)
Client Information:
Primary (DSM-IV code) (DSM-IV code)
Client ID#
DOB
Gender Ethnicity Language Primary Axis I Secondary Axis I Therapist ID
•Use a unique identifier for Client ID#
•Categorical variables will have pull-down menus from which
you’ll select an option (e.g., gender, ethnicity, language)
•Dates should be entered as xx/xx/xxxx
•Axis I diagnoses s/b the numeric DSM-IV code
•Therapist ID is optional
33
Palette of Measures Data Entry:
Demographics & Services (2 of 2)
Focus 1 of Treatment
Focus1
Date of
First
Session
Date of Total # Completed
Last
of
Session Sessions Focus 1?
(if Focus 1 not
completed)
Completed
(if Services not
completed)
Reason
Services?
Reason
•Select Focus from pull-down menu (e.g., anxiety, depression)
•The shell will hold data for up to 4 foci, or treatment targets
•Enter Date of First Session
•The remaining fields are to be completed at the end of treatment
targeting this particular focus (e.g., Date of Last, # Sessions)
• 2 levels of “Completed?”
• Treatment targeting this particular focus
• Overall service delivery
34
Palette of Measures Data
Submissions (slide 1 of 3)
• Data submissions to CiMH will occur twice a year
throughout the duration of the project
– The end of each May (reflecting all clients served from the
initiation of the project through the end of that April)
– The end of each December (reflecting all clients served
from the initiation of the project through the end of that
November)
– Note that this is the anticipated schedule; actual data
submission dates may vary slightly
• An email notice will be sent to Palette of Measures
site leads approximately one month in advance of
each data submission deadline
35
Data Submissions (slide 2 of 3)
• Providers may choose from among the following
methods for submitting their Palette of Measures
Excel databases to CiMH:
– Use YouSendIt, or another secure web-based transfer
site, to submit data electronically
• YouSendIt (www.YouSendIt.com) is a vendor that supports the
secure transfer of electronic data (encrypted and passwordprotected)
– Mail a password-protected CD to CiMH and submit the
password separately (via email or phone)
– Email an encrypted, password-protected file(s) to CiMH
and submit the password separately (via email or phone)
36
Data Submissions (slide 3 of 3)
• After data are submitted, sites continue to
enter new data into the same database
– Always reflects an ongoing, historical record of
clients served through the Palette of Measures
project
• Every effort is made to distribute reports
within two months of each data submission
– Aggregate and site/agency-specific reports
37
Questions
38
The End
Contact Information
•Cricket Mitchell, PhD
•Email: cmitchell@cimh.org
•Cell phone: 858-220-6355
39