Building a Tier 1 & 2
Evaluation Plan
Presented by
Lori Cameron & Peg Mazeika
WI PBIS Technical Assistance Coordinators
The Wisconsin RtI Center/Wisconsin PBIS Network (CFDA #84.027) acknowledges the support of the Wisconsin Department of Public Instruction in the development of this
presentation and for the continued support of this federally-funded grant program. There are no copyright restrictions on this document; however, please credit the
Wisconsin DPI and support of federal funds when copying all or part of this material.
Today’s Agenda
• Overview of Available Surveys/Tools on the
PBIS Assessment Website
• Developing an Evaluation Action Plan
– Decisions on which tools to choose
– Decisions on timelines
• Yearly Action Plan
• 3-5 Year Plan
• District Examples
• Questions/Comments
Acknowledgements
PBIS Assessment District Evaluation Planning Webinar
• Celeste Rossetto Dickey, May 92013
Evaluation Blueprint for School-Wide Positive Behavior
Interventions & Support
•
Bob Algozzine, Robert H. Horner, George Sugai, et.al
The Role of Fidelity Measures in Implementing Evidencebased Practices: Lessons Learned from School-wide PBIS
• Tary J. Tobin , Claudia G. Vincent, et.al
PBIS Evaluation Blueprint
“Evaluation is the process of collecting and
using information for decision-making. A
hallmark of School-wide Positive Behavior
Support (SWPBS) is a commitment to
formal evaluation.”
Evaluation Questions
• Fidelity
– To what extent was SWPBS implemented as
designed?
– To what extent was SWPBS implemented
with fidelity?
“Are we doing what we said we would
do?”
Current Tools In PBIS Assessment
Universal
System
Secondary
&
Tertiary
Systems
Research
Tool
Annual Assessment
Tool
Progress
Monitoring
Tool
System-wide
Evaluation Tool
(SET)
Benchmarks of Quality
(BoQ)
Early Childhood BoQ (EC
BoQ)
PBIS Self Assessment
Survey (SAS)
Team
Implementation
Checklist (TIC)
Benchmarks of Advanced
Tiers (BAT)
PBIS Self Assessment
Survey (SAS)
Monitoring
Advanced Tiers
Tool (MATT)
Outcome Measures:
School Safety Survey
www.pbisapps.org
Evaluation & Action Planning
• Decisions about Evaluation Tools
– Which tools answer your district/schools’ evaluation
questions?
• Decisions about Evaluation Timeline
– When/how often will you administer the tools?
• Decisions about Evaluation Reporting
– How will you report outcomes to schools and to the
District?
– How often will you report outcomes to schools and to the
District
– How do you plan to report to Dist. Leadership Team and
action plan district training and support based on results?
Evaluation & Action Planning
Decisions about Evaluation Tools
– Does your District/School need a Research tool?
• (SET/ISSET)
– Which Annual Assessment Tool(s) will you use?
• BoQ, SAS, Safety Survey, BAT, EC BoQ
– Which Progressing Monitoring Tool(s) will you use?
• TIC, MATT
– Does your District/School need to evaluate
Tier I, Tier II - III Implementation or all?
Examples of
PBIS Evaluation Plans
Example
District/School A Evaluation Questions
Newly implementing; Grant needing specific research tools as
well as Fidelity of Implementation
• What is our baseline, and what is our progress through
the year?
• Are we improving on our implementation of PBIS?
• How do our individual staff members rate our
implementation?
• Are our self-assessments consistent with data from
outside evaluators?
District A: Evaluation Plan
Measure
Fall
SET
X
TIC
X
BoQ
SAS
BluePrint
Self
Assessment
Dst.Leader
ship Team
X
Year One
Year Two
Winter Spring Fall
Winter Spring
X
X
X
Year Three
Fall
Winter Spring
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Example
District/School B Evaluation Questions
Implementing PBIS for three years: We have been using the
TIC/BoQ & SET and have been consistently over 80%
• What specific areas in PBIS are we strong in, and
in which areas do we need training?
• Are we improving on our implementation of
PBIS?
• How do our staff members actually rate our
PBIS implementation?
• How is our implementation of Tiers 2 (3)?
District B: Evaluation Plan
Measure
Fall
Year One
Year Two
Winter Spring Fall
Winter Spring
BoQ
X
SAS
X
Matt
X
Year Three
Winter Spring
X
X
X
Fall
X
X
X
X
X
X
BATT
X
X
X
Self Assess
Dist.
Leadership
Team
X
X
X
Example
District D Evaluation Questions
The District wants to bring RtI Behavior and Academics together
and create one evaluation plan.
• In what specific PBIS areas are we strong, and in which
areas do we need training?
• In what academic areas are we strong, and in which
areas do we need more improvement?
• How can we tailor our training/evaluation to meet the
needs of various schools?
• How can we schedule our evaluation so that we are not
overwhelming our schools?
Assessment
Instrument
Fall
(August/Sept/Oct)
Team
Implementation
Checklist (TIC)
X
Self-Assessment
Survey (SAS)
BoQ
Winter
(Dec/Jan)
Spring
(April/May)
X
Purpose
X
Fidelity of
Implementation Tier 1
Fidelity of
implementation
Staff perception T1,2,3
X
Fidelity of
Behavior & Academic Assessment
Timeline
X
Implementation Tier 1
MATT
X
Fidelity of
implementation Tier2/3
X
BATT
X
Fidelity of
Implementation Tier2/3
X
Universal Screener
Early ID
Internalizer/Externalizer
Screening
Progress Monitor
SSBD
X
X
X
X
AIMsWeb
SAT10
Summative Evaluation
X
DIBELS
X
Blue Print Self
Assessment Dist
Leadership Team
Adapted from Nieves Flores, 2012
X
X
X
Screening
Progress monitoring
Training and
implementation action
planning and support.
Task: Developing your Plan
Questions to Consider:
• Which surveys are your schools/districts currently using?
• Do they answer your evaluation questions:
– Are we implementing SWPBIS with fidelity? (SAS, BoQ)
– Is there more information that we need? (TIC, SSS)
• Do your surveys assess the Tiers you are implementing?
(TIC, ECBoQ, BoQ, MATT, BATT)
• Do you need a research tool? (SET)
• Are you using a Universal Screener for Behavior/Social
Skills?
• Are you Action Planning from your Assessments?
Task: Developing your Plan
• Are you “topping out” of the surveys you are currently
taking?
• Do you need to customize your evaluation plan for the
schools, or will one evaluation plan fit for all schools?
• Do you want to assess all tiers of PBIS implementation?
• Is your Evaluation Plan manageable for schools?
• Do you have a District Leadership Team that plans for
PBIS Implementation and support?
• Does your District Leadership Team Self Assess
annually?
Task:
Measure
Complete this Plan for your District/School
Year One
Fall
Winter
Year Two
Spring
Fall
Year Three
Winter
Spring
Fall
Winter
Spring
Action Plan
Use the Action Plan templates on
PBISassessment.org for TIC, SAS & BoQ etc.
Use Action Plan available on
Wisconsinpbisnetwork.org
TIPS format meeting minutes action plan
State specifically when you will review your
goals in the plan!
School- Wide PBIS Action Plan
School name_________________________________ Date______________
Action plan based on data from (TIC, SAS, BoQ, MATT, BATT etc)__________________________
Item/Section
Evidence/Data
Roadblocks
Strategies and
resources
Action:
Next Steps
Who?
When?
Goal:
Universal Screeners
For info on Selection of Universal Screeners
Go to Wisconsinpbisnetwork.org
enter Universal Screeners in Search box
Merrill Elementary School
School District of Beloit
Katie Smullen, Principal
Terri Ross - Kindergarten Teacher, Tier 1 and
Tier 2 Team Member
- We completed the assessments which we were scheduled to
complete by our internal coach
- The power was in our PBIS team’s response to what we learned from
the assessments
- Began with a staff committed to making the change
TIER 1: SAS in the Fall
BOQ in the Spring
TIC – Twice a year
TIER 2: MATT in the FALL
BATT in the Spring
SET in the Spring
- Two schools combined in 2011-12
- When the schools combined, we opened up PBIS Tier 1 Committee to
everyone who wanted to be involved (which resulted in a large team,
which was great for us)
- Staff commitment to PBIS from the beginning
- 253 Students
- Grades Kindergarten – 3rd (adding 4K this coming school year)
- 96.8% Economically Disadvantaged
- 37.5% - Black 39.9% - Hispanic, 15% - White, 6.7% - Two or More
- 17.4% - Students with Disabilities
- 26.1% - English Language Learners
* From 1/31/12 (Neither school had taken it before)
Strengths:
- School-wide expectations
- Supervising and interacting with students in non-classroom
settings
- Classroom routines and expected behavior for those routines
- Teaching the expected behaviors (not leaving anything to
chance)
- PBIS Tier 1 Team organization
- Administrator support
Examples to highlight these strengths:
https://sites.google.com/a/sdb.k12.wi.us/sdbpbis/home
- Posters
- Cool Tools
- Matrix – classroom matrixes
Weaknesses:
- Problem behaviors receive consistent consequences
- Consequences are clearly defined
- Procedures for expected and problem behaviors are consistent
- Office Managed versus Classroom Managed
- Booster trainings happen routinely, based on data
- Formal opportunities for parent training
How we addressed these weaknesses:
- Reviewed Office versus Classroom
Managed at Staff & PBIS Meetings
- Created a PBIS Binder
- Offered a 7 Habits of Highly Effective
Parenting Class
• Taken April 2012 (99%)and April 2013 (90%)
Weaknesses:
- Data Analyzed at least monthly
- Data shared with team and faculty monthly
- Major/Minor behaviors are clearly differentiated
- Procedures exist for tracking classroom behavior problems
- Students are surveyed about PBIS
How we addressed these weaknesses:
- Made review of SWIS data a priority and part of staff meetings
- Reviewed Major and Minor behaviors (specifically around physical
aggression)
- Created a classroom behavior tracking database (still working on
the implementation of this)
- Gave a student a survey during their guidance class.
Weaknesses:
- Did not have a para or parent on our team
- Did not have a procedure to collect minor classroom managed
behaviors
- Did not have our personnel established for our CICO (Year 1)
How we addressed these weaknesses:
- Invited a para and a parent (then two parents the 2nd year)
- Created a classroom form to report minor classroom managed
behaviors
- Built time into our Math Interventionist’s schedule to check
students out daily
MATT in the FALL
- Realized we needed to develop a formal time for our Tier 2 team to
meet
- Communication letter with families needed to be established for
our students who had qualified for Tier 2 (SAIG Group)
- Developed a way to review the behavior goals assessed weekl
BAT in the Spring
- Solidified that we were off to a good start with our CICO (trying
CICO without these assessments in place was a difficult road)
SET in the Spring
- Most helpful to hear the feedback gathered from our external
coach after interviewing the students and teachers throughout our
school. To have her in our building, talking to our students who
knew our expectations and were following them, made us all very
proud.
Date Completed
Total Score
January 23, 2013
March 11, 2013
Tier II Total Score Tier III
69%
92%
33%
45%
85%
78%
0%
- The leadership our district provided and the Wisconsin PBIS
Network provided was invaluable to us.
- We could not have done this without a very caring and committed
staff
- The increased positive behavior in our building seems to be correlated
with student academic success. In our 2nd year of PBIS, we had
the highest test scores in 5 years (in both Reading and Math).
Thank you for your kind attention!