Using Information for Decision
Making
Identifying Interventions that support
Targeted and Intensive Students
Susan Barrett
www.pbis.org
www.pbismaryland.org
Monitor Objectives
& Outcomes
• Big Idea:
– The staff determine what they want to
answer, what data will answer the question,
the simplest way to get that data, and then
write an objective for where they want to be
in the future.
Questions
How does the team use to data to:
• Get 80% staff buy-in (staff survey, TIC)
• Keep 80% buy-in (BIG 5, communication)
• for problem solving across all
teams/committees
• Utililize the three-tiered model of
prevention (integrate student services,
academic support, teacher support)
Positive Behavior Support
Supporting
Staff Behavior
INFORMATION
Supporting
Decision
Making
School wide
Non classroom
Classroom
office discipline referrals
PBS self assessment survey
all staff (parents/students?) input
academic progress
attendance
direct observation
Individual students
school improvement goal progress
SYSTEMS
PRACTICES
Supporting
Student Behavior
define (behav. expect. & routines)
teach
acknowledge
correct
follow up & feedback
consensus & collaboration
Why Collect Discipline
Information?
• Decision making
• Professional Accountability
• Decisions made with data (information) are
more likely to be (a) implemented, and (b)
effective
Key features of data systems that
work.
• The data are accurate
• The data are very easy to collect (1% of staff
time)
• Data are used for decision-making
– The data must be available when decisions need to
be made (weekly?)
– Difference between data needs at a school building
versus data needs for a district
– The people who collect the data must see the
information used for decision-making.
What data to collect for decisionmaking?
• USE WHAT YOU HAVE
– Office Discipline Referrals/Detentions
• Measure of overall environment. Referrals are affected
by (a) student behavior, (b) staff behavior, (c)
administrative context
• An under-estimate of what is really happening
• Office Referrals per Day per Month
– Attendance
– Suspensions/Expulsions
– Vandalism
Office Discipline Referral
Processes/Form
• Coherent system in place to collect office
discipline referral data
– Faculty and staff agree on categories
– Faculty and staff agree on process
– Office Discipline Referral Form includes needed
information
•
•
•
•
Name, date, time
Staff
Problem Behavior, maintaining function
Location
Observe Problem Behavior
Warning/Conference with Student
No
Use Classroom
Consequence
Complete Minor
Incident Report
Does student
have 3 MIR slips
for the same
behavior in the
same quarter
Write the
student a
REFERRAL to
the main office
Is behavior
office
managed?
Yes
Classroom
Managed
Office
Managed
•Preparedness
•Calling Out
•Classroom Disruption
•Refusal to Follow a
Reasonable Request
(Insubordination)
•Failure to Serve a
Detention
•Put Downs
•Refusing to Work
•Inappropriate
Tone/Attitude
•Electronic Devices
•Inappropriate
Comments
•Food or Drink
•Weapons
•Fighting or Aggressive
Physical Contact
•Chronic Minor
Infractions
•Aggressive Language
•Threats
•Harassment of Student
or Teacher
•Truancy/Cut Class
•Smoking
•Vandalism
•Alcohol
•Drugs
•Gambling
•Dress Code
•Cheating
•Not w/ Class During
Emergency
•Leaving School
Grounds
•Foul Language at
Student/Staff
Write referral to
office
Administrator
determines
consequence
Administrator
follows through
on consequence
Administrator
provides teacher
feedback
SIDE BAR on Minor Incident Reports
•Issue slip when student does not respond to pre-correction, re-direction, or verbal warning
•Once written, file a copy with administrator
•Take concrete action to correct behavior (i.e. assign detention, complete behavior reflection
writing, seat change)
Minor Incident Reports
• Overall Design
– Smaller than referral
– In triplicate
– Replaced Detention Forms
• Pre-Referral Documented Step
• Flexible Consequences
– Ex: Detention, reflection, parent signature, etc.
• Administrative Intervention Before Referral
When Should Data be Collected?
• Continuously
• Data collection should be an embedded part of
the school cycle not something “extra”
• Data should be summarized prior to meetings
of decision-makers (e.g. weekly)
• Data will be inaccurate and irrelevant unless
the people who collect and summarize it see
the data used for decision-making.
Organizing Data for “Information”
• Counts are good, but not always useful
• To compare across months use “average office
discipline referrals per day per month”
Total Ref versus Ref/Day/Mo
NV High School
70
R e fe rra ls
60
50
40
30
20
10
0
Aug Sept
Oct
Nov Dec
Jan
Feb Mar
School Month
Apr
May Jun
M e a n R e fe rr a ls p e r D a y
Total Ref versus Ref/Day/Mo
5
4
3
2
1
0
Aug
Sept
Oct
Nov Dec
Jan
Feb
School Month
Mar
Apr
May
Jun
Using Data for On-Going Problem
Solving
• Start with the decisions not the data
• Use data in “decision layers”
– Is there a problem? (overall rate of ODR)
– Localize the problem
– (location, problem behavior, students, time of day)
– Get specific
• Don’t drown in the data
• It’s “OK” to be doing well
• Be efficient
Is there a problem?
• Office Referrals per Day per Month
• Attendance
• Faculty Reports
Interpreting Office Referral Data:
Is there a problem?
• Absolute level (depending on size of school)
– Middle Schools (>5 per day)
– Elementary Schools (>1.5-2 per day)
• Trends
– Peaks before breaks?
– Gradual increasing trend across year?
• Compare levels to last year
– Improvement?
A v e R e fe rra ls p e r D a y
Office Referrals per Day per Month
Last year
2
1.5
1
0.5
0
Sept
Oct
Nov
Dec
Jan
Feb
Mar
School Months
Apr
May
Jun
A v e R e fe rra ls p e r D a y
Office Referrals per Day per Month
Last year
20
15
10
5
0
Sept
Oct
Nov
Dec
Jan
Feb
Mar
School Months
Apr
May
Jun
A v e R e fe rra ls p e r D a y
Office Referrals per Day per Month
Last year
20
15
10
5
0
Sept
Oct
Nov
Dec
Jan
Feb
Mar
School Months
Apr
May
Jun
A v e R e fe rra ls p e r D a y
Office Referrals per Day per Month
This Year
20
15
10
5
0
Sept
Oct
Nov
Dec
Jan
Feb
Mar
School Months
Apr
May
Jun
A v e R e fe rra ls p e r D a y
Office Referrals per Day per Month
Last Year and This Year
20
15
10
5
0
Sept
Oct
Nov
Dec
Jan
Feb
Mar
School Months
Apr
May
Jun
A v e R e fe rra ls p e r D a y
Office Referrals per Day per Month
Last Year and This Year
20
15
10
5
0
Sept
Oct
Nov
Dec
Jan
Feb
Mar
School Months
Apr
May
Jun
Is There a Problem? #1
Maintain - Modify - Terminate
A v e R e fe rra ls p e r D a y
Office Referrals per Day per Month
This Year
20
15
10
5
0
Sept
Oct
Nov
Dec
Jan
Feb
School Months
Mar
Apr
May
Is There a Problem? #4
Maintain - Modify - Terminate
A v e R e fe rra ls p e r D a y
Office Referrals per Day per Month
Last Year and This Year
20
15
10
5
0
Sept
Oct
Nov
Dec
Jan
Feb
Mar
School Months
Apr
May
Jun
What systems are problematic?
• Referrals by problem behavior?
– What problem behaviors are most common?
• Referrals by location?
– Are there specific problem locations?
• Referrals by student?
– Are there many students receiving referrals or only
a small number of students with many referrals?
• Referrals by time of day?
– Are there specific times when problems occur?
Referrals by Problem Behavior
Referrals per Prob Behavior
N u m b e r o f R e fe rra ls
50
40
30
20
10
0
Lang
Achol
Arson
Bomb Combs Defian Disrupt Dress Agg/fgt
Theft
Harass Prop D
Skip
Tardy
Types of Problem Behavior
Tobac
Vand
Weap
Referrals by Problem Behavior
Referrals per Prob Behavior
N u m b e r o f R e fe rra ls
50
40
30
20
10
0
Lang
Achol
Arson
Bomb Combs Defian Disrupt Dress Agg/fgt
Theft
Harass Prop D
Skip
Tardy
Types of Problem Behavior
Tobac
Vand
Weap
Referrals per Location
N u m b e r o f O ffic e R e fe rra ls
Referrals by Location
50
40
30
20
10
0
Bath R
Bus A
Bus
Caf
Clas s
Comm
Gym
Hall
Libr
School Locations
Play G
Spec
Other
N u m b e r o f O ffic e R e fe rr a ls
Referrals by Location
50
40
30
20
10
0
Bath R
Bus A
Bus
Caf
Class
Comm
Gym
Hall
Libr
School Locations
Play G
Spec
Other
N u m b e r o f R e fe rra ls p e r S tu d e n t
Referrals per Student
20
10
0
Students
N u m b e r o f R e fe rra ls p e r S tu d e n t
Referrals per Student
Students per Number of Referrals
20
10
0
Students
Referrals by Time of Day
Referrals by Time of Day
N u m b e r o f R e fe rra ls
30
25
20
15
10
5
0
7:00
7:30
8:00
8:30
9:00
9:30
10:00
10:30
11:00
11:30
12:00
12:30
Time of Day
1:00
1:30
2:00
2:30
3:00
3:30
Referrals by Time of Day
Referrals by Time of Day
N u m b e r o f R e fe rra ls
30
25
20
15
10
5
0
7:00
7:30
8:00
8:30
9:00
9:30
10:00
10:30
11:00
11:30
12:00
12:30
Time of Day
1:00
1:30
2:00
2:30
3:00
3:30
Decision-making
•
•
•
•
Is there a problem?
What areas/systems are involved?
Are there many students or a few involved?
What kinds of problem behaviors are
occurring?
• When are these behaviors most likely?
• What is the most effective use of our
resources to address this problem?
Designing Solutions
• If many students are making the same mistake
it typically is the system that needs to change
not the students.
• Teach, monitor and reward before relying on
punishment.
• An example (Kartub et al, JPBI, 2000)
IF...






More than 40% of students receive one or more office
referrals
More than 2.5 office referrals per student
More than 35% of office referrals come from nonclassroom settings
More than 15% of students referred from nonclassroom settings
More than 60% of office referrals come from the
classroom
50% or more of office referrals come from less than
10% of classrooms

More than 10-15 students receive 5 or more office
referrals

Less than 10 students with 10 or more office referrals
Less than 10 students continue rate of referrals after
receiving targeted group settings
Small number of students destabilizing overall
functioning of school


FOCUS ON...
School Wide System
Non-Classroom System
Classroom Systems
Targeted Group Interventions /
Classroom Systems
Individual Student Systems
Targeted Group
Supports
• For those students who exhibit difficulties
despite proactive school-wide prevention
efforts
• Likely to be student with both academic &
behavioral challenges
• Approximately 10% of school population
Remember…
Without school-wide prevention,
we cannot reliably identify
targeted-level students.
Halls Ferry Elementary School
YEAR 1
YEAR 2
YEAR 3
YEAR 4
YEAR 5
High Five Approach - school wide social skill lessons
Central Data System
Produced school-wide expectations video
Cafeteria routine and lessons
Playground routine and lessons
Produced bathroom expectations & routines video
Produced indoor recess expectation & routines video
Newcomer's Club
Study Skills - Homework Support
Social Skills Club
FBA Training
Coaches Training
Present to Board Mentoring
Champs Theater
Universal School-wide Systems
Secondary / Targeted Group
District wide web based data system
District level leadership team
District level collaborative team
District wide networking system
Tertiary / Individual Student
Classroom
Bus expectations
District Level Systems
Function-Based
Student Support
Team
Using Data to Select Interventions
Who are the students who need additional support?
(referrals by student, targeted:2-5 referrals, intensive: 6 or more
referrals)
What are the interventions? (link to SW)
What is the referral process? (teacher support-quick, easy, )
What is the time frame students get access to the
intervention? (72 hours)
What is the process to determine function?
• FACTS, student questionnaire, role of academics, hypothesis
statement, team process?
Does the SST have access to the data for ongoing
monitoring of intervention?
Tertiary Prevention:
Baltimore County
2004-2005
32 Schools:
14 ES, 12 MS, 6 HS
~5%
Interagency Partnerships
Community Partnerships
Court/ Institutional Liaisons
Spot Light On Schools/ P.O.s
Maryland’s Tomorrow Program
Homeless Program
NW/SW IEP Team
Individual Student IEP/ 504 Plans
Student Therapeutic Support
Student Threat Assessments
Abuse and Neglect Reporting Program (CPS Liaisons)
Traumatic Loss Teams
~15%
Secondary Prevention:
Primary Prevention:
School Improvement Planning
School Emergency Safety Plans
Positive Behavior Plans/ Codes
Character Education Programs
Parent/ Family Resources
Student Support Teams
Health Screenings/ Protocols
School Nurse/ Assistant Services
Essential Guidance Curriculum
School Counselor Services
Pupil Personnel Services
Residency/ Attendance Officer Services
School Social Work Services
School Psychologist Services
~80% of Students
Project Attend/ FACE
School Resource Officer Program
D.A.R.E./ S.A.D.D. Programs
Student Assistance Programs
Wellness Centers
Health Action Plans/ Appraisals
Peer Helper/ Mediation Programs
Pupil Personnel Home Visits/ Social Histories
Instructional Support Teams (IST)
Functional Behavior Assessments
Behavior Intervention Plans
Educational/ Psychosocial Assessments
Psychological Assessments
Student Case Management
Social Skills, Anger Management, Conflict Resolution
Parent Presentations/ Trainings
Alternative School/Program Placements
Targeted and Intensive /Programs/Initiatives
Name of school:
County:
School Year 2005-06
Programs
Purpose
Expected
Outcome
Target Group
Outcome Data (Is it working?)
Recommendations:
1. What programs groups can we eliminate?
2. What programs groups can we combine?
3. What programs groups need to be supported for improved outcomes and sustained functioning?
Targeted and Intensive /Programs/Initiatives
Name of school:
County:
School Year 2005-06
Programs
Purpose
Expected
Outcome
Target Group
Outcome Data (Is it working?)
Homework
Club
Recommendations:
1. What programs groups can we eliminate?
2. What programs groups can we combine?
3. What programs groups need to be supported for improved outcomes and sustained functioning?
Conduct Brief Functional Assessment
Is the behavior
maintained by escape
from social
interaction?
Escape Motivated BEP
Reduce adult
interaction
Use escape as a
reinforcer
Is the behavior
related to lack of
academic skills?
Is the behavior
maintained by peer
attention?
BEP + Academic
Support
Peer Motivated BEP
Increase academic
support
Allow student to
earn reinforcers to
share with peers
Horner, Hawken, Marsh
DATA
Graphic
demonstration
SWIS or Central
Data System
Multiple team
members trained
Multiple reports
(daily average)
Defined process
Daily
Systematic
Data Entry
Office Discipline
Referrals
Time-out,
Buddy Rooms, etc.
Comprehensive
ISS
Checklist
Matches Data Entry
Detention
Efficient Referral
Form
Comprehensive
Multiple Data
Sources to Identify
Secondary/Tertiary
Teacher referral
Screening, etc.
L. Newcomer
Summary of PBIS “BIG IDEAS”
Systems (How things are done)
 Team based problem solving
 Data-based decision making
 Long term sustainability
Data (How decisions are made)
 On going data collection & use
 ODR’s (# per day per month, location, behavior, student)
 Suspension/expulsion, attendance, tardies
Practices (How staff interact with students)
 Direct teaching of behavioral expectations
 On-going reinforcement of expected behaviors
 Functional behavioral assessment
Using Data
• School-Wide
– What’s happening?
– Are we meeting our goals?
– Are we doing what we said we would?
• Individual Students
– What’s happening?
– What do we do next?
– Are we doing what we said we would?
School-Wide Analysis:
What’s Happening
•
•
•
•
Can we predict our problems?
Why are they occurring?
What can we do to prevent?
Is it working?
School-Wide Data Analysis:
What are our Goals?
•
•
•
•
What do we want?
Is it happening?
If Yes - what next?
If No - what next?
School-Wide Analysis:
Are we Doing it?
• Are we doing what we said we would?
• What are our barriers?
• What do we have to do to make it work?