WHY YOUR RTI MODEL IS
FAILING
Dr. Lisa Kilanowski
Associate Professor of School
Psychology and Program Director
Niagara University
About Me
• Associate Professor of School Psychology at
Niagara University
• Extensive consultation work establishing RtI
policy, RtI models, and professional
development
• Substantial background in literacy, core and
supplemental reading instruction, and reading
disabilities
An Important Note
• Systems Issue
• Info today most relevant to those working K
through 6
Things I Know
• Resources are limited (time, money, staffing)
• Schedule changes are necessary (block
scheduling)
• Must think flexibly about who provides and
what materials are used
• Great variability in teacher knowledge of best
practice intervention
Lack of Uniform Policy and Procedures
• Regardless of the assessment tools used and
interventions provided, the absence of
uniform and consistent policy regarding the
implementation of RtI serves as a significant
barrier to successful implementation
• Uniformity in procedures as part of the RtI
process across schools within a district is
essential
Lack of Uniform Policy and Procedures
• Stems from district-wide planning and
consensus, not school-based protocol
• Results in the development of a cogent,
district-wide RtI model with clearly delineated
procedures across buildings
Lack of Uniform Policy and Procedures
• Must attend to:
• Uniformity in assessment practices (assessment tools used
and when they are administered for both benchmarking
and progress monitoring)
• Benchmarking decision making (data analysis training)
• Progress monitoring decision making (data analysis
training)
• Diagnostic assessment tools
• Intervention materials and corresponding training for
teachers
• Frequency and duration of intervention
• Data collection and data maintenance procedures
Poor Core Program
• No amount of intervention will “fix” a universally weak
core program
• NYS RtI Guidance Document and best practices indicate
REGULAR USE of school-wide assessment data to
inform practice regarding core program. Skewed
distribution of students at Tier 2 and 3 and less than
80% of students at Tier 1 indicative of core program
issues
• Core analysis should feature analysis of program
components and how it is delivered (fidelity)
• Districts can be slow to evaluate core programs and
implementation
Poor Core Program
• Today’s worst offenders:
• Balanced literacy programs that lack explicit
instruction in the Big 5
• Workshop programs that lack explicit
instruction in the Big 5
• Use of CCSS modules without providing
explicit instruction in the Big 5
What Do You Do About This?
• Scope of influence
• http://www.centeroninstruction.org/reviewin
g-a-reading-program-professionaldevelopment-module
Use of Inappropriate Assessment Tools
• Despite copious sources of evidence-based information
regarding assessment practices in reading, math, and
RtI in general, districts continue to use or misuse
evaluation tools within their RtI models
• WE ARE STILL PRONE TO FAD DECISION MAKING, NOT
EVIDENCE-BASED DECISION MAKING
• Select non-evidence based tools for benchmarking
• Select non-evidence based tools for progress
monitoring
• Select non-evidence based tools or procedures for
diagnostic assessment
Use of Inappropriate Tools
• Benchmarking occurs three times per year
• Not all screening measures are created equalmust be valid and reliable for the purpose of
screening
• Only a select number of measures possess
technical adequacy for the purposes of screening
and also progress monitoring
• Typically use same measure for benchmarking
and progress monitoring and use other more
expansive measures for diagnostic assessment
within RtI to further drive intervention
Assessment Tools
• Questions you should ask:
• Is there a substantial evidence base supporting
use of this tool for benchmarking and progress
monitoring?
• Are the reviews independent?
• Is this tool sensitive to small increments of
growth or only large?
• Will this tool yield normative comparisons of
established benchmarks and rates of
improvement or other uniform metrics for
evaluating GROWTH?
Assessment Tools
• www.studentprogress.org
• http://www.intensiveintervention.org/chart/p
rogress-monitoring
• http://www.rti4success.org/resources/toolscharts/screening-tools-chart
Failure to Systematically Use
Benchmark Assessment Data
• NYS RtI Guidance Document and best practices indicate
that benchmark data be used three times per year to
identify at-risk students and a) progress monitor them
in Tier 1 for five weeks to further monitor need or b)
immediately begin intervention provision and progress
monitoring
• The intent is to ensure that all students in need of
intervention are immediately addressed and provided
for
• We should not rely solely on a referral model-must be
data driven
• What am I seeing? Not systematic use of assessment
data
Weak Intervention Protocols
• Great variability in understanding of what
intervention should consist of, and just how
much intervention it actually takes to close an
achievement gap
• Lack of intensity in intervention
• Not adhering to the NYS recommendations
regarding intervention length
• Lack of a clear understanding of the various
ways in which intervention can be provided
Intervention Duration
NYS RtI Guidance: Tier 2:
Varies, but no less than three times per week for a
minimum of 20-30 minutes per session
Tier 3: Varies, but more frequently than Tier 2 for a
minimum of 30 minutes per session
Duration of intervention: Tier 2: 9-30 weeks
Tier 3: A minimum of 15-20 weeks
*All decision making contingent on student
progress
What Is Intervention?
•
•
•
•
•
•
Preferential seating
Prompting
Extended time
Reduced assignment length
…are all accommodations, not interventions!
Intervention in reading (or math) involves a
change in instructional approaches used
• NYS (and best practices) indicate that changes in
instructional approaches form the substance of
RtI
Intervention of Increasing Intensity
• NYS mandates intervention that increases in
intensity over time contingent upon student need
• This provision designed to meet the needs of
students who do not respond to core instruction
or initial efforts at providing supplemental
instruction
• Providing more of the same intervention for
longer (if it is not working) does not fit into this
model
• Frequency and intensity of intervention is
dictated by the intervention you choose
What Is Meant By Intensity?
•
•
•
•
Intensity can include a few things:
Duration (amount of time spent on an intervention)
Frequency (how often intervention is provided)
Actual design of the intervention: some interventions
are inherently less intensive (those involving use of
inferential learning opportunities or whole language
approaches), moderately intensive (feature targeted,
direct instruction and opportunity for feedback), or
substantially intensive (feature principles of Direct
Instruction or Multisensory Structured Language)
Implicit vs. Explicit Instruction
Implicit
Whole
Language
Approaches
Explicit
Embedded
Approaches
Systematic
Approaches
Moderate
MacKenzie, 2000
Systematic
Approaches
with Direct
Instruction
Multisensory
Structured
Language
Over Reliance on Stand Alone
Interventions
• There is a tendency for schools to implement one
intervention at a time in reading (e.g.,
segmentation activities, fluency activities)
• This may help students on the “cusp” in terms of
performance but is not very helpful for students
with multiple and significant concerns
• Need more rigorous protocols
• Need to consider standard protocol materials to
support this
• Need time to provide adequate support
Common Characteristics of
Intervention
• Supplemental to core instruction at Tiers 2 and 3
• Matched to student need (as identified via
assessment) at all Tiers
• Features explicit and direct skill instruction,
including providing models of proficient problem
solving and verbalization of thought processes
• Provide opportunity for frequent practice of skill
• Provide opportunity for frequent corrective
feedback from teacher
Research Based or Evidence Based
Intervention
• There is a difference!
• Research Based: There is a body of research
demonstrating that a component of the
intervention has been found to be critical in
the instruction of a particular skill. For ex., a
program includes instruction in phonemic
awareness…and research has proven that
phonemic awareness is a critical component
of reading and reading instruction
Research Based or Evidence Based
Intervention
• Evidence Based: A program or approach has been
EVALUATED to determine its’ efficacy. For ex.,
PATR has been found to increase the phonemic
awareness of students in first grade; Direct
Instruction in reading has been found to be the
most effective remedial intervention approach.
• We want to use evidence based practices when
possible, followed by research based practices
RESEARCH
BASED!
Use Caution
BUY ME!
• Program publishers and others in the reading
community, including those involved in designing
policy, use the term “research based” indiscriminately
• As a result, interventions used within RtI models may
vary from those found to be “important” in acquiring
reading skill to those that have actually been found to
improve reading skill
• Best practices dictate that interventions with some
degree of demonstrated efficacy be used
• Incumbent upon those designing intervention policy
and programming to maintain awareness of evidence
based practices
Structure of Intervention: Problem
Solving V. Standard Protocol
• Two major approaches to determining what kind of reading
intervention should be provided to students (systems level
decision)
• Problem solving involves brainstorming and designing a
specific intervention for a particular student based on their
particular needs (i.e., the team decides that Johnny will
engage in repeated readings 5x weekly for 15 minutes to
remediate his fluency difficulties)
• Decisions regarding intervention type are made on a case
by case basis- ICT team makes recommendations for each
individual child based on strengths/needs (see work of
Rosenfield and Gravois (1999; 2008) for more information
on problem solving and IC Teams)
Standard Protocol
• Standard protocol intervention approaches
involve using a set protocol/series of
interventions for all students with certain
types of presenting difficulties (i.e., all
students with reading fluency difficulties will
engage in Great Leaps 5x weekly for 15
minutes)
Standard Protocol
 Typically involves a PROGRAM that is EVIDENCE BASED
 Can use interventions that are not part of a program,
but use of evidence based programs is most common
 The program may vary in intensity
 All students with a particular deficit utilize the program
(i.e., all students with fluency and PA difficulties use X
program) for a prescribed amount of time
 Can be conducted by teachers in the classroom (least
practical and contingent on the program), push in
support, and pull out support
 More intensive standard protocols require pull-out
support
 Occur during supplemental instruction time
Fuchs, Mocks, Morgan & Young, 2003
Problem Solving Only
IC Team meets to develop
intervention at each tier for each
student (case by case basis)
Team decides that reading
intervention for student x will
consist of interventionist designed
activities such as:
-instruction in sound/symbol
awareness
-sound blending activities
-phoneme segmentation activities
-phoneme deletion activities
Standard Protocol of Stand Alone
Interventions (Interventions not
part of a program)
IC Team/administration develops
uniform protocols of stand alone
interventions for various types of
difficulties
All students evidencing phonemic
awareness and phonics difficulties
will engage in sound
blending/phoneme
segmentation/deletion activities;
all students evidencing fluency
difficulties will engage in repeated
readings/choral reading/paired
reading…
Standard Protocol Using Shorter
Duration Intervention Programs
IC Team/administration develops
uniform protocols of program
oriented interventions for various
types of difficulties. A combination
of shorter (15 minutes)
intervention programs are used to
meet student needs
Phonemic awareness difficulties
will be addressed by activities from
“Road to the Code” or PATR;
Fluency difficulties will be
addressed by using PALS, and so
on..
Standard Protocol Using Intensive
Intervention Programs
IC Team/administration develops
standard protocol of intensive
program based interventions at
Tiers 2 and 3
Reading Mastery will be used with
all Tier 3 students
Wilson Reading will be used with
all Tier 2 students, and so on…
Kilanowski, 2010
Poor Data Analysis and Usage
• As per NYS RtI Guidance Document:
• Decision rules or criteria for decision making need to be
created prior to implementation of the intervention to
determine when:
• students are not responding adequately to instruction and
need supplemental intervention;
• students are responding adequately to instruction and no
longer need supplemental intervention;
• an intervention may need to be changed; and/or
• a student may need a referral for special education services
to determine if a student’s learning difficulty is the result of
a disability.
Underdeveloped Student Data
Interpretation Procedures
• Contemporary research indicates that co-existing
deficits in rate of improvement and benchmark scores
(overall proficiency relative to grade-level expectations)
are the greatest indicators of failure to respond to
intervention
• “Double Deficit” or “Dual Discrepancy”
• Chronic double deficits predictive of need for
continuous support
• Must evaluate progress monitoring data for Rate of
Improvement information: The amount of progress
your student evidences on a weekly basis relative to
same-age peers
Data Teaming
• Districts should have a consistent mechanism
in place for reviewing student progress
monitoring data
• Must feature analysis of Rate of Improvement
relative to benchmarking scores
• Can be part of or exclusive of problem solving
model
• Typically use of data teams the most common
and research-supported modality
Data Teaming
• Data teams meet after benchmark periods to identify
students at-risk
• Set appropriate goals founded in best-practice approaches
• Meet every 6 to 8 weeks to review student progress based
on DATA
• Analyze Rate of Improvement and performance in
relationship to benchmark
• Recommend alterations to intervention intensity and
movement through the tiers
• Project where students will function in the future based on
their current rate of improvement
• It is progress that dictates movement through the tiers, not
a fixed amount of time in each tier
Lack of Diagnostic Assessment
Procedures
• Benchmarking and progress monitoring insufficient in
isolation
• Must have procedures for additional diagnostic
assessment if students fail to respond to intervention
at Tier 2 and Tier 3
• These procedures can be used to further inform
intervention
• Examples of additional assessment procedures include
diagnostic reading and math measures (e.g., CTOPP-II,
GORT-V), additional skill and content based measures
(e.g., STAR, Discovery products), and qualitative
analysis of reading performance
Lack of Intervention Fidelity
• Intervention implementation
recommendations not followed, publisher
recommendations not followed
• Need procedures (at some point) to monitor
intervention fidelity
• http://www.iup.edu/page.aspx?id=134567
Lack of Training
• Lack of training as to the semantics of the model,
use of intervention programs, procedures for
progress monitoring and data based decision
making
• Over reliance on use of AIS specialists to design
reading intervention- they may not have training
in designing and sequencing reading intervention
(special education degree)
No Set Plan for Using RtI Data In SLD
Determination
• Using available protocol and assessment, must
determine criteria for SLD identification
• If you do not have access to assessment
results that feature Rate of Improvement data,
growth reports, or any kind of projection of
future performance based on historical
growth, you will not have sufficient data to
classify as SLD with integrity (Kovaleski et al.,
2014)
Failure to RtI
• Chronic failure to RtI is determined by
considering the following:
• Individual Rate of Improvement relative to grade
level Rate of Improvement
• Future projections on Rate of Improvement
barring any substantial change in performance
(use current ROI data)
• Student benchmark relative to normative
expectations
• Duration of intervention provision in reference to
the above
SLD Determination
• RtI alone: use of the previously cited data in
reference to evidence of consistent sub-average
academic achievement in the classroom
• RtI combined with comprehensive assessment
data: use of the previously cited data in
conjunction with cognitive assessment data
indicating a pattern of processing strengths and
weaknesses that match areas of academic need
• All of those convoluted cognitive discrepancy
analyses you learned in grad school but never
used: You need to use them
NYS RtI Guidance Document
• http://www.p12.nysed.gov/biling/docs/RTIGui
dance-Final11-10.pdf
Steps to Identify and Remediate RtI
Model Issues
• Conduct a review of existing RtI Policy (should be on file)
• Determine the degree to which your model matches the
NYS RtI Guidance Document and best practices
• Take inventory of available resources for the provision of
intervention, including available intervention programs and
materials, providers, and assessment tools. Evaluate the
integrity of available resources using evaluation resources
provided by NYS RtI TAC and National Response to
Intervention resources (see dropbox).
• Where possible, work collaboratively to remedy areas of
incongruence. This requires the involvement of multiple
layers of administration
• If district-wide involvement is not possible, work
collaboratively at the building level to do the same
Quality of General Education and
Special Education Intervention
• Need to regularly review district and building
wide data for trends- this informs practice at
all Tiers
• Rigorous in gen ed, more rigorous in special ed
Thank You!