CiMH Palette of Measures
Evaluation Training: Eyberg Child
Behavior Inventory™ (ECBI)
Cricket Mitchell, PhD
CIMH Evaluation Consultant
Palette of Measures Evaluation:
What You Will Need (slide 1 of 2)
• Palette of Measures Data Entry Shell v2 (Excel file)
– Developed by CiMH and customized for each
participating agency
– Holds all data for clients served thru the Palette of
Measures project
– Demographics, service delivery information, pre- and
post- outcome measure data
• Palette of Measures Data Dictionary v2 (Word
document)
– A guide for using the associated data entry shell
– Defines each column in the excel file
2
Palette of Measures Evaluation:
What You Will Need (slide 2 of 2)
• Outcome measures from the two-pronged
approach
– General Outcome Measure
– Target-Specific Outcome Measure(s)
– For example…
• Eyberg Child Behavior Inventory™ (ECBI)
– Trade-marked and published by Psychological
Assessment Resources, Inc. (PAR)
• Must be purchased by interested Palette of Measures
project participating agencies
3
Overview of Training
• Brief Overview of Palette of Measures evaluation protocol
• Eyberg Child Behavior Inventory (ECBI): Target-specific
measure when the focus of treatment is disruptive
behaviors
+ Sutter-Eyberg Student Behavior Inventory (SESBI)
– Administration
– Scoring
– Clinical Utility
• Instructions for Palette of Measures data entry and data
submissions
– Data entry: ECBI & SESBI
– Data entry: Demographics & Services
– Data submissions to CiMH
4
Brief Overview of Palette of
Measures Evaluation Protocol
5
Outcome Assessment
• Palette of Measures providers will track outcomes
using data from pre- and post- administrations of
standardized measures of functioning
• Pre- and Post- a “dose” of treatment / an
intervention interval
– General measure of youth mental health functioning
(e.g., YOQ/YOQ-SR, CANS, Ohio Scales)
– Target-specific measure linked to focus of
treatment/intervention (e.g., AQ, ECBI, PHQ-9,
PTSD-RI, RCADS)
• Providers may choose to administer mid-course
assessments as well
– e.g., at 3-month intervals in usual care
6
A note about the use of standardized
assessment measures… (slide 1 of 2)
• Assessment is the beginning of developing
a relationship with the child and family
– Demonstrates a desire to know what the child
and family are experiencing
– By incorporating standardized assessment
measures of functioning, the efficiency and
thoroughness of assessment is enhanced
7
A note about the use of standardized
assessment measures… (slide 2 of 2)
• Using standardized assessment measures
of functioning…
– Assists in initial clinical impressions
– Provides valuable information to guide
treatment/interventions
– Assesses sufficiency of treatment delivered
– Demonstrates treatment-related improvements
in child functioning
8
Eyberg Child Behavior
InventoryTM (ECBI™)
and
Sutter-Eyberg Student Behavior InventoryTM
(SESBI-RTM)
9
CiMH ECBI Training
• Information on the administration, scoring,
and clinical utility of the ECBI & SESBI
was obtained from the measures’
Professional Manual, written by Sheila
Eyberg and Donna Pincus, published by
Psychological Assessment Resources,
Inc. (PAR)
10
ECBI™ and SESBI-R™ Description
• Target-specific measure when the
treatment focus is disruptive behavior
• Measures the current frequency and
severity of child disruptive behavior
problems
– Parent/caregiver report for children ages 2-16
– Teacher report for children ages 2-16
• Available in English and Spanish
11
ECBI™ and SESBI-R™ Description
•
•
•
•
Written at a 6th grade reading level
5-10 minutes to complete
Valid and reliable
Trade-marked and published by Psychological
Assessment Resources, Inc. (PAR)
– Must be purchased by interested Palette of Measures
project participating agencies
– After purchase of the manual ($52), the cost per use
~$1.60
– http://www4.parinc.com/Products/Product.aspx?Prod
uctID=ECBI
12
ECBI™ and SESBI-R™ Description
• ECBI: 36 items
• SESBI-R: 38 items
• 7-point Likert scale
response options
–
–
–
–
–
Never (1)
Seldom (2-3)
Sometimes (4)
Often (5-6)
Always (7)
• Two Scales
– Intensity
• Frequency with which the
child exhibits the
behaviors
– Problem
• Extent to which the parent
or teacher considers the
behaviors to be a problem
for him/herself
13
Example: Items from the ECBI™
•
•
•
•
•
•
•
Refuses to do chores when asked
Argues with parents about rules
Has temper tantrums
Hits parents
Lies
Physically fights with friends own age
Is overactive or restless
14
Example: Items from the SESBI-R™
•
•
•
•
•
•
•
Teases or provokes other students
Does not obey school rules on his/her own
Demands teacher attention
Acts bossy with other students
Sasses teacher
Physically fights with other students
Has difficulty staying seated
15
ECBI™ and SESBI-R™ Administration
• Administer pre- and post- a dose of
treatment, or an intervention interval
– ECBI completed by mother/mother figure or
father/father figure (when available)
• Both can be completed for clinical purposes;
however, only one should be entered for outcome
tracking purposes
– Select parent/caregiver who spends most time with client
– SESBI can be completed by a teacher or child
care worker
• Residential treatment settings can have staff
complete this measure
16
Let’s take a look at the
ECBI...
(not distributed due to copyright laws)
17
ECBI™ and SESBI-R™ Administration
• Approximately 5 minutes to complete
• Parents and/or teachers complete independently
– Written at a 6th grade reading level
– Can be read aloud if necessary/desired
• There are no “right” or “wrong” answers;
encourage honest responses
– Informants can easily be influenced by the attitude of
the person administering the scale
– Let them know that this questionnaire will help you, as
a clinician, better understand how the child is doing
overall
• Ask parents (and teachers) to complete all items
18
ECBI™ and SESBI-R™ Scoring
• Calculate the Intensity scale Page 1 subtotal by summing
the marked scores and recording this sum in the space
provided
• Tally the number of Problem responses for which a “Yes”
rating was provided, and record this total in the Page 1
subtotal for the Problem scale
• Transfer Page 1 subtotals to appropriate row on Page 2
• Calculate Page 2 subtotals in the same manner as Page 1
• Each scale total raw score is obtained by summing Page 1
and Page 2 subtotals
19
ECBI™ and SESBI-R™ Scoring:
Missing Data
• If more than four items missing from Intensity
scale, it is invalid and cannot be scored
– With three or fewer items missing, circle 1 (Never)
and calculate raw score
• If more than four items missing from Problem
scale, it is invalid and cannot be scored
– With three or fewer items missing, circle the “No”
response and calculate raw score
20
ECBI™ and SESBI-R™ Scoring
• ECBI Raw Scores (36 items):
– Intensity Scale Raw Scores Range from 36-252
– Problem Scale Raw Scores Range from 0-36
• SESBI-R Raw Scores (38 items):
– Intensity Scale Raw Scores Range from 38-266
– Problem Scale Raw Scores Range from 0-38
21
ECBI™ and SESBI-R™ Scoring
• Convert Raw scores to T scores
– T scores are standardized scores with a mean of 50
and a standard deviation of 10
– Intensity scale T score conversions are in Appendix C
(ECBI) and Appendix E (SESBI-R)
• Pg 53 and pg 57 of the ECBI/SESBI-R Professional Manual
– Problem scale T score conversions are in Appendix D
(ECBI) and Appendix F (SEBI-R)
• Pg 55 and pg 59 of the ECBI/SESBI-R Professional Manual
22
Clinical Utility of the ECBI™ and
SESBI-R™
• Higher scores indicate a greater level, or
greater frequency, of conduct-disordered
behavior (Intensity) and a greater impact
on the parent or teacher (Problem)
23
Clinical Utility of the ECBI™ and
SESBI-R™
• T scores > 60 are clinically significant
– ECBI Intensity Raw Score > 131
– ECBI Problem Raw Score > 15
– SESBI-R Intensity Raw Score > 151
– SESBI-R Problem Raw Score > 19
• T scores < 60 are within the normal range
24
Clinical Utility of the ECBI™
• Parenting education workshops and
parent training interventions can be helpful
even to those who do not score in the
clinical range
– Behavior problems of “normal” children are
often annoying to parents and others
– ECBI is sensitive to change even when initial
administration is not in clinical range
25
Clinical Utility of the ECBI™
• Discrepancies between scores on the
ECBI Intensity and Problem scales can
occur; however, it happens relatively
infrequently…
26
Clinical Utility of the ECBI™
• …When the Intensity score is low and
the Problem score is high, interpret
cautiously
– Parent could have a low tolerance for normal
misbehaviors of child
– Parent may have unrealistically high expectations
for child conduct and an authoritarian parenting
style
– Parent may have limited understanding of child
behavior and the kinds of behavior problems that
are developmentally appropriate
– Parent may be attempting to cope with chaotic
and difficult circumstances and be overwhelmed
by the stressors of child rearing
27
Clinical Utility of the ECBI™
• …When the Intensity score is high and
the Problem score is low
– Parent may have a high tolerance for misbehavior
– Parent may hold well-intentioned beliefs about
permissive parenting that are mismatched to the
temperament of his/her child
– Parent may be detached from the child either
because of the child’s problems or because of
characteristics in the parent’s own personality
28
Clinical Utility of the SESBI-R™
• The SESBI-R is a newer measure with
much less empirical study than the
ECBI
– While cutoff scores are provided to estimate the
severity of child behavior problems at school,
there is a wide variability both within and between
geographic locations
– Use caution in interpretation, and consider
prevailing local customs and norms as needed
29
Clinical Utility of the ECBI™ and
SESBI-R™
• Comparisons of pre/post scores reveal
specific areas of treatment-related
improvement in child disruptive behavior
– e.g.,
• Does the frequency of disruptive child behavior
decrease substantially?
• Does the extent to which disruptive child
behaviors are problematic to parents and/or
teachers improve?
30
Palette of Measures Data Entry
and Data Submissions
31
Palette of Measures Data Entry
Shell
• There is a separate spreadsheet in the excel
workbook (aka database) for each type of
data:
–
–
–
–
–
Demographics & Services
Pre- General outcome measure
Post- General outcome measure
Pre- Target-specific measure(s)
Post- Target-specific outcome measure(s)
• Specific outcome measure spreadsheets included in
each agency’s database varies across Palette of
Measures project participants
32
Palette of Measures Data Entry
Shell
• In addition to the spreadsheets that hold
data...
– There is an Instructions spreadsheet
• Basic data entry instructions
• Contact information for T.A. (Cricket Mitchell)
– There is a Data Lists spreadsheet at the end
of the workbook that you will not use
• Data Lists populate the pull-down menus in other
spreadsheets
33
Palette of Measures Data Entry:
ECBI
TEACHER REPORT (SESBI): IF
PARENT REPORT (ECBI): AGES 2-16 AVAILABLE
Pre-ECBI
Date of
Intensity Scale
Problem Scale
Intensity Scale
Problem Scale
EEEESSSSClient ID# Assessment IntRaw1 IntT1 ProbRaw1 ProbT1 IntRaw1 IntT1 ProbRaw1 ProbT1
•There is a separate spreadsheet for Pre-ECBI&SESBI data,
Post-ECBI&SESBI data, and Mid-ECBI&SESBI data
•In each spreadsheet, there is a separate field for Intensity
and Problem Scales, Raw and T Scores, for each measure
•In the event of missing data, leave the fields blank/empty.
Do not enter text into any of the fields.
34
Palette of Measures Data Entry:
ECBI
PARENT REPORT (ECBI): AGES 2- TEACHER REPORT (SESBI): IF
16
AVAILABLE
Mid-ECBI
Date of
Client ID# Assessment
Assessment Intensity Scale Problem Scale Intensity Scale Problem Scale
Interval
EEEESIntRaw IntT ProbRaw ProbT IntRaw
M
M
M
M
M
SSSIntT ProbRaw ProbT
M
M
M
•For agencies who will conduct mid-treatment assessments,
indicate the Assessment Interval in the Mid-ECBI&SESBI
spreadsheet by selecting from the available pull-down menu
•e.g., 1st mid-treatment assessment, 2nd mid-treatment
assessment
35
But, before you enter any
outcome data, you’ll enter
Demographics & Service
Delivery Information...
36
Palette of Measures Data Entry:
Demographics & Services (1 of 2)
Client Information:
Primary (DSM-IV code) (DSM-IV code)
Client ID#
DOB
Gender Ethnicity Language Primary Axis I Secondary Axis I Therapist ID
•Use a unique identifier for Client ID#
•Categorical variables will have pull-down menus from which
you’ll select an option (e.g., gender, ethnicity, language)
•Dates should be entered as xx/xx/xxxx
•Axis I diagnoses s/b the numeric DSM-IV code
•Therapist ID is optional
37
Palette of Measures Data Entry:
Demographics & Services (2 of 2)
Focus 1 of Treatment
Focus1
Date of
First
Session
Date of Total # Completed
Last
of
Session Sessions Focus 1?
(if Focus 1 not
completed)
Completed
(if Services not
completed)
Reason
Services?
Reason
•Select Focus from pull-down menu (e.g., anxiety, depression)
•The shell will hold data for up to 4 foci, or treatment targets
•Enter Date of First Session
•The remaining fields are to be completed at the end of treatment
targeting this particular focus (e.g., Date of Last, # Sessions)
• 2 levels of “Completed?”
• Treatment targeting this particular focus
• Overall service delivery
38
Palette of Measures Data
Submissions (slide 1 of 3)
• Data submissions to CiMH will occur twice a year
throughout the duration of the project
– The end of each May (reflecting all clients served from the
initiation of the project through the end of that April)
– The end of each December (reflecting all clients served
from the initiation of the project through the end of that
November)
– Note that this is the anticipated schedule; actual data
submission dates may vary slightly
• An email notice will be sent to Palette of Measures
site leads approximately one month in advance of
each data submission deadline
39
Data Submissions (slide 2 of 3)
• Providers may choose from among the following
methods for submitting their Palette of Measures
Excel databases to CiMH:
– Use YouSendIt, or another secure web-based transfer
site, to submit data electronically
• YouSendIt (www.YouSendIt.com) is a vendor that supports the
secure transfer of electronic data (encrypted and passwordprotected)
– Mail a password-protected CD to CiMH and submit the
password separately (via email or phone)
– Email an encrypted, password-protected file(s) to CiMH
and submit the password separately (via email or phone)
40
Data Submissions (slide 3 of 3)
• After data are submitted, sites continue to
enter new data into the same database
– Always reflects an ongoing, historical record of
clients served through the Palette of Measures
project
• Every effort is made to distribute reports
within two months of each data submission
– Aggregate and site/agency-specific reports
41
Questions
42
The End
Contact Information
•Cricket Mitchell, PhD
•Email: cmitchell@cimh.org
•Cell phone: 858-220-6355
43