:
A collaborative project between the Florida Department of Education and the University of South Florida
FloridaRtI.usf.edu
Who are we?
Regional Coordinators
Beth Hardcastle
Kelly Justice
Brian Gaunt
Graduate Assistant
Coordinators
Decia Dixon
Amanda March
Florida Department
Of Education
Project Evaluator
Jose Castillo
Project Leader
Clark Dorman
Co-Directors
Mike Curtis
George Batsche
Available at:
www.florida-rti.org
Florida Department of Education
2008
State Infrastructure
• State Management Group
Todd Clark, Bureau Chief, Bureau of Instruction and Innovation
Shan Goff, Executive Director, Office of Early Learning
Evan Lefsky, Executive Director, Just Read, Florida!
Bambi Lockman, Bureau Chief, Bureau of Exceptional
Education & Student Services
Jay Pfeiffer, Deputy Commissioner, Accountability Research and
Measurement
Hue Reynolds, Director of Communications and Public Affairs,
Office of Communications and Public Affairs
Mary Jane Tappen, Deputy Chancellor for Curriculum, Instruction,
and Student Services, Office of the Chancellor
Iris Wilson, Deputy Chancellor for Student Achievement, Office of
the Chancellor
Representatives from State Transformation Team
State Infrastructure
• State Transformation Team
Ginger Alberto, Office of Achievement through Language
Acquisition, FLDOE
George Batsche, Mike Curtis, Clark Dorman – Problem
Solving/Response to Intervention Project, USF
Liz Crawford, Florida Center for Reading Research, FSU
Heather Diamond, Bureau of Exceptional Education & Student
Services, FLDOE
Sandy Dilger, Bureau of School Improvement, FLDOE
Don Kincaid, Heather George, Karen Childs – PBS Project, USF
Mary Little, Response to Intervention’s Teaching Learning
Connections, UCF
Martha Murray, Bureau of Exceptional Education and Student
Services, FLDOE
Rob Schoen, Office of Mathematics and Science, FLDOE
Melinda Webster, Just Read, Florida!, FLDOE
State Infrastructure
•
State Advisory Group - representatives from:
Regional Implementation Teams (district contacts, coaches, etc.)
Florida Center for Reading Research (FCRR)
Florida Center for Research – Science, Technology, Engineering, Math
(FCR-STEM)
Early Childhood Association of Florida (ECA)
Florida Association of District School Superintendents (FADSS)
Florida Association of School Administrators (FASA)
Florida Educators Association (FEA)
Council of Administrators of Special Education (CASE)
Family Network for Students with Disabilities (FND)
Florida Association of Student Services Administrators (FASSA)
Purpose is to Provide on-going stakeholder input
Regional Meetings
Top Level District Administrators
8 sites across the state
3 eight hour sessions per site
Focused on creation of district PS/RtI plans
Consensus
Infrastructure
Implementation
Florida PS/RtI Project
Two purposes of PS/RtI Project
 Evaluate the impact of PS/RtI on educator,
student, and systemic outcomes in pilot sites
implementing the model
 Statewide training in PS/RtI
Selected Pilot Sites
• 7 Demonstration School Districts
 34 pilot PS/RtI schools
 27 matched comparison schools
• Districts and schools vary in terms of
 Geographic location
 Student demographics
 District size: 6,200 – 105,000 students
Demonstration Districts
Statewide Training
Overview
• 3 year training curriculum
 Problem Solving Process
 3-Tiered RtI Model
 Systems Change
• Limited technical assistance and support
• Limited data collection
Statewide Training Sites
Consensus
Florida’s Change Model
Consensus
Infrastructure
Implementation
In the beginning
None of
Beliefs, Skills,
Knowledge
All of Beliefs,
Skills,
Knowledge
necessary to participate in a Problem Solving/
Response to Intervention Model
Over Time
None of
Beliefs, Skills,
Knowledge
All of Beliefs,
Skills,
Knowledge
necessary to participate in a Problem Solving/
Response to Intervention Model
Goal
None of
Beliefs, Skills,
Knowledge
All of Beliefs,
Skills,
Knowledge
necessary to participate in a Problem Solving/
Response to Intervention Model
Why have past initiatives
failed?
•
•
•
•
•
•
•
Purpose unclear
Lack of ongoing communication
Unrealistic expectations of initial success
Failure to measure and analyze progress
Participants not involved in planning…
School culture is ignored
Failure to achieve CONSENSUS
Consensus Building
Educators will embrace new ideas when two
conditions exist:
• They understand the NEED for the idea
• They perceive that they either have the
SKILLS to implement the idea OR they
have the SUPPORT to develop the skills
1
Survey Item
19 - Disaggregate data as necessary
16 - Modify interventions based on student
RtI
15 - Interpret graphed PM data for
decision-making
13a - Select data to use for progress
monitoring
12a - Determine implementation integrity
11a - Provide necessary implementation
support
10a - Ensure intervention is supported by
collected data
8a - Access resources to develop EBIs
6a - Identify appropriate data to determine
hypotheses
5a - Develop hypotheses
4f - Gap data to determine focus of
instructional adjustments
4e - Calculate gap between student and
benchmark
4d - Determine peer performance levels
4c - Data to determine benchmarks
4b - Data to define current target behavior
performance level
4a - Define referral concern related to
replacement behavior
3a - Use data to make decisions
2a - Access Data
Skill Level
5
Total School Staff Perception of RtI Skills Comparison
(Academic)
Example Elementary
4
3
2
BOY
EOY
Guiding Questions
• Did your beliefs change?
• How consistent are the overall beliefs of
your building with your building’s
perceptions of the practices occurring?
• To what extent do you believe that your
building possesses the skills to use schoolbased data to evaluate Tier 1 and Tier 2
instruction?
What is Consensus
Building?
A process that:
• Shows people precisely what is being proposed
and why
• Gets people on the same page
• Gives people time to explore and ask questions
• Is open, honest, accurate
• If done well, results in commitment and buy-in
Tilly, 2007
Strategies for building
consensus
• Explain “the why” behind RtI
- What we’ve been doing hasn’t worked
- New practices are available
- Accountability
• Facilitate a shift in thinking
• Provide a clear vision
• Explain the scope and sequence
Strategies (cont.)
•
•
•
•
•
Provide a voice for all stakeholders
Find success stories
Look at school data
Provide professional development
Anticipate resistance & get it out in the
open
Example Elementary SBLT Fall 2007 to Winter 2008 Comparison
Self-Assessment of Problem Solving Implementation (SAPSI)
Consensus
3
Status
2
1
3=
2=
1=
0=
Maintaining
Achieved
In Progress
Not Started
0
District Commitment
SBLT support
Faculty involvement
SBLT present
Data to assess
commitment
1
2
3
4
5
Item
Reactions to Guided
Questions
• A lot of what we do isn’t consistent with our
beliefs.
• What information do we need to share with
staff to start changing practices?
• What skills do our staff members need?
Qualitative Insight: Consensus
Coaches
Facilitators
Barriers
Future
Directions
•Focus on Key
Stakeholders’
•Consensus before Staff
Trainings
•Meaningful
Discussions
Regional Coordinators
•Strong, Skilled Coaches
•District Level Commitment
•“Family Feeling”
•Healthy Staff Relationships
•Moving from
• Coaches’ having difficulty
Theory/Rationale to
Application and
Implementation
applying systems change
principles
•Focus on Rapport &
•Focus on Purposeful &
Personal Relationships
with Teachers
Planned Consensus Building
Strategy
• Explain the Why
Role of School
Psychologist
• Facilitate overviews,
trainings, serve on
teams, meet w/ grps
• Shifts in Thinking
• Child-centered vs.
alterable variables,
Gen Ed vs. SPED,
eligibility for SPED
Strategy
Role of School
Psychologist
• Clear Vision/Scope and
• Planned/systematic,
• Input from Stakeholders
• Roundtables, team
• Success Stories
• Team effort, internal and
Sequence
involvement on team, do
your homework, schoolspecific
structure, exchange of
ideas, communication
external, action plan
Role of School
Psychologist
Strategy
• Use of school data
• Professional
• Expertise, guide
•
Development
• Anticipate Resistance
•
others, desensitize,
inservice
Study groups, data
meetings, Showers’
work
Problem Solving
model, address
barriers
Systems Change Model
Building Infrastructure
Infrastructure Building
“…where sites examine their implementation
against the critical components of RtI, find
aspects that are being implemented well and
gaps that need to be addressed.
Infrastructure building centers around
closing these practice gaps.”
NASDSE (2008)
Conceptual Framework
• School Building as unit of change
• Change guided by use of key questions
• NASDSE Blueprints
• School Assessment of Problem-Solving
Implementation (SAPSI)
• Surveys on staff beliefs, practice, skills.
• Build upon & coordination with consensus
activities
• Focus on team building, training & coaching,
and alignment of resources
Team Building
• School-based Leadership Team (SBLT)
includes key stakeholders and roles:





Administrator/Resource agent
Facilitator
Data mentor(s)
Content Specialist(s)
Staff Liaison
PS/RtI Training
• 13 6-hour Sessions over 3 years.
• Days of training staggered for TA and
Coaching activities between.
• School-based full-time coaches
• Technical assistance as needed by
Regional Coordinators
15 - Interpret graphed PM data for decisionmaking
13a - Select data to use for progress
monitoring
12a - Determine implementation integrity
11a - Provide necessary implementation
support
10a - Ensure intervention is supported by
collected data
19 - Disaggregate data as necessary
16 - Modify interventions based on student RtI
Survey Item
8a - Access resources to develop EBIs
6a - Identify appropriate data to determine
hypotheses
5a - Develop hypotheses
4f - Gap data to determine focus of
instructional adjustments
4e - Calculate gap between student and
benchmark
4d - Determine peer performance levels
4c - Data to determine benchmarks
4b - Data to define current target behavior
performance level
4a - Define referral concern related to
replacement behavior
3a - Use data to make decisions
2a - Access Data
Skill Level
What Skills Are Staff
Reporting They Possess?
Total School Staff Perception of RtI Skills Comparison
(Academic)
Elementary
5
4
3
2
1
BOY
EOY
What Do the Data Mean?
• Reflect what skills staff perceive they
possess
• Issues to consider
• Self report tends to be upwardly biased
• Responses can vary as a function of their
understanding of what is being asked
• Is there another way to assess skill
development?
What Skills Are Educators
Demonstrating?
County School District: Direct Skills Assessment - Day 5
2
2
1.75
1.71
Score
1.58
0.96
1
Possible points
0.75
0
1. Appropriate
Goal
2. RtI Quality
3. Modifications
Implemented
4. What
modifications
Questions
5. Target Next
6. Problem ID
Data Systems
• Foundation to all efforts towards effective
•
•
•
•
PS/RtI
Required to guide school change
Focus on data already in use
Guide School-Based Leadership Team
(SBLT) to align services based on data.
Use of PS as framework for decisionmaking.
Questions guiding
infrastructure
• How will efforts towards implementing PS/RtI be monitored? How are
•
•
•
•
•
•
•
•
staff involved?
What assessments are in use and what is their role for use at your
school?
What resources are available for use? How aligned with student
needs at each tier?
How sufficient is your core services/programs/instruction? And for
what student populations?
Where needs exist with core, how will they be addressed?
How will effectiveness of interventions at each tier be monitored?
What professional development needs exist? And how addressed?
What supplemental and intensive services/programs are available for
use? How delivered?
How will you determine which students require more intensive
planning and supports?
Data used to
make decisions
Data presented
to staff
Data used to
evaluate core
acad programs
Data used to
evaluate core
beh programs
CBM data used
to ID students
needing
interventions
ODR data used
to ID students
needing beh
interventions
Data used to
evaluate Tier 2
interventions
Data used to
determine Tier
3 RtI
0
Data is
collected
Status
Example Elementary Fall 2007 to Winter 2008 Comparison
Self-Assessment of Problem Solving Implementation (SAPSI)
Infrastructure Development- Data Use
3
2
1
6
7
8
9
10
11
12
13
14
Item
Sp Ed eligibility
uses RtI for
EBD
Sp Ed eligibility
uses RtI for
SLD
Evidence-based
practices for
Tier 1
Evidence-based
practices for
Tier 2
Evidence-based
practices for
Tier 3
SBLT has
regular meeting
schedule
SBLT evaluates
target students'
RtI
SBLT involves
parents
SBLT regularly
evaluates Tier 1
and 2 data
Status
Example Elementary Fall 2007 to Winter 2008 Comparison
Self-Assessment of Problem Solving Implementation (SAPSI)
Infrastructure Development- Team Structure
3
2
1
0
15a
15b
16a
16b
16c
17
18
19
20
Item
Additional Strategies
• Development of protocols
• Developing consensus and communication
channels between district and schools
• Resource Mapping
• Identifying staff professional development
needs through referrals and staff input
• Infusing use of PS/RtI within existing
school plans.
RC & Coaches Perspective on
Infrastructure in Pilot Schools
Coaches
Facilitators
•Policies/Procedures:
-More Time for PS
-More Frequent Meetings
•Focus of Service Delivery
on Tier 1
•Innovative Resource
Allocation
Barriers
Regional Coordinators
•Data Management System
•Preparing to build up
infrastructure
•Strong, Skilled Coaches
• Lack of Data Collection
•Organizational Bureaucracy
• Tools for Certain Domains •Conflicting District Policies &
Procedures
• Coaches difficulty applying
change model
Future
Directions
•Focus on Resource Mapping
& Identifying Tier I,II, & III
Supports
Implementation
Implementation
• Professional Development / Technical Assistance /
Evaluation
•
•
•
•
District Leadership Teams
School Leadership Teams
Coaches
Staff
• Problem Solving Process
• Tiered Systems of Instructional Service Delivery
• Data Collection & Analysis / Data Based Decision Making
• Fidelity, Fidelity, Fidelity
(or if you prefer, Integrity, Integrity, Integrity)
Implementation Questions
• How will you launch and use the
infrastructure you built?
•
•
•
•
How will PD be sequenced/scheduled/delivered?
How will students access the tiers?
How will data make it to/from the database?
How will decision makers access database?
Implementation Questions
• Does your infrastructure do what you want it
to do?
• Did recipients of PD learn the concepts and skills
taught?
• Are teams engaging in a consistent problem
solving process?
• Does the database yield instructionally relevant
information that is being utilized?
• Are the tiers effective?
Are we implementing with
integrity?
3 Ways to Assess Implementation Integrity
• Self-Report
 Most efficient
 Least reliable
• Permanent Product Reviews
 Moderately efficient
 Moderately reliable
• Observations
 Least efficient
 Most reliable
Clearly defined
Tier 1 acad
instruction
Clearly defined
Tier 1 Beh
Instruction
Clearly defined
Tier 2 acad supp
instruction
Clearly defined
Tier 2 Beh Supp
Instruction
Evidence-based
Tier 3 Acad
Strategies
Evidence-based
Tier 3 Beh
Strategies
Status
Example Elementary Fall 2007 to Winter 2008 Comparison
Self-Assessment of Problem Solving Implementation (SAPSI)
Implementation- Three-Tiered Intervention System
3
2
1
0
21a
21b
21c
21d
21e
21f
Item
Evidence-based strategies for interventions
Support identified for interventions
Intervention integrity is documented
RtI evlauated through data collection
Intervention changed based on student RtI
Parents involved in interventions
Monitoring plan exists
SBLT meets at least 2x per year
SBLT meets with District Team 2x per year
22b
22c
22d
22e
22f
22g
22h
22i
23
24
25
Feedback of PS/RtI project provided to school
SBLT conducts problem analysis with data
22a
Plan changed based on data
SBLT defines replacement beh
0
Define problem as data-based discrepancy
3
Example Elementary Fall 2007 to Winter 2008 Comparison
Self-Assessment of Problem Solving Implementation (SAPSI)
Implementation- Problem-Solving Process
2
1
Fall 07
Spr 08
Win 09
26
27
Data to determine
effectiveness of
Decisions made to
modify core or
Universal screening
or other data used
Team hypotheses to
identify reasons for
Data used to
determine
Modifications made
to core instruction Modifications made
to core instruction Modifications made
to core instruction Supp. instruction
developed or
Supp. instruction
developed or
Supp. instruction
developed or
Criteria for positive
RtI were defined
Progress monitoring
data
Decision regarding
student RtI was
Plan for contin,
modifying, or
0.00
Data to determine
effectiveness of
Score
What Evidence of PS/RtI Exists in
Permanent Products?
2.00
1.00
1a
1b
2
3
4
5
6a
6b
6c
7a
7b
7c
8
9
10
11
Item
What Steps of PS/RtI Were Evident
During Data Meetings?
Tiers I & II Observation Checklist
Example Elementary
100%
90%
80%
Percentage Present
70%
60%
50%
Percent
Present
40%
30%
20%
10%
0%
Roles Represented
Problem Identification
Problem Analysis
Domain
Intervention
Program
Development/Support Evaluation/Resonse to
Intervention
What PS/RtI Steps Were Evident in
Individual Student Focused Meetings?
Initial Problem-Solving Team Checklist
100%
Example A Elementary
Date 10/20/08
90%
80%
Percentage Present
70%
60%
50%
Percent
Present
40%
30%
20%
10%
0%
Roles Represented
Problem Identification
Problem Analysis
Domain
Intervention
Development/Support
What About Follow-Up Meetings?
Follow -Up Problem-Solving Team Checklist
Example A Elementary
100%
90%
80%
Percentage Present
70%
60%
50%
Percent
Present
40%
30%
20%
10%
0%
Roles Represented
Program Evaluation/Response to
Intervention
Domain
Qualitative Insight: Tier 1
Implementation
Coaches
Facilitators
•Planning for
Intervention Support
•Utilizing Progress
Monitoring Data
•Thorough Problem
Analysis
Regional Coordinators
•Understanding Core
Curriculum
•Data Collection &
Management System
•Strong, Skilled Coaches
•Focusing on Tiers II & III
Barriers
Before Tier I
•Too Many Excuses
•Unhealthy Staff Relationships
Future
Directions
•Building Capacity
• “Sharing the
Wealth”
•Focus on Purposeful and
Planned Tier I Activities
Q&A