RTI Implementer Webinar Series:
Using Screening Data for
Decision Making
National Center on Response to
Intervention
National Center on
Response to Intervention
RTI Implementer Series Overview
Introduction
Screening
Progress
Monitoring
Multi-level
Prevention
System
Defining the
Essential
Components
What Is RTI?
What Is
Screening?
What Is Progress
Monitoring?
What Is a Multilevel Prevention
System?
Assessment and
Data-based
Decision Making
Understanding
Types of
Assessment within
an RTI Framework
Using
Screening Data
for Decision
Making
Using Progress
Monitoring Data
for Decision
Making
IDEA and Multilevel Prevention
System
Establishing
Processes
Implementing RTI
Establishing a
Screening
Process
National Center on
Response to Intervention
Selecting
Evidence-based
Practices
2
Upon Completion Participants Will
Be Able To:
 Understand different types of decisions that can be
made with screening data
 Learn how the placement of a cut score can
influence who is identified as at risk
 Analyze different types of screening data
National Center on
Response to Intervention
3
Vocabulary Handout
Term
Prediction
Final Meaning
Primary
prevention
level
The bottom of
the pyramid that
represents
instruction given
to students
without learning
problems
Instruction delivered to
all students using
research-based curricula
and differentiation in the
general education
classroom. Incorporates
universal screening,
continuous progress
monitoring, and
outcome measures or
summative assessments.
National Center on
Response to Intervention
Picture/Sketch/Example
Primary
prevention
4
District Educational Decisions:
Screening
 Program improvement and curriculum decisions
 Innovation and sustainability decisions
• General effectiveness of implementation and general
effectiveness of RTI model
 Ensuring equitable services and supports across
schools
• Access to supplemental supports, access to effective
instruction, and SLD identification
 Allocation of resources and professional development
National Center on
Response to Intervention
5
School Educational Decisions: Screening
 General school-and grade-level trends or issues
 Effectiveness of school-wide curriculum and
instructional delivery
 Areas of need and guidance on how to set
measurable school-wide goals
National Center on
Response to Intervention
6
Grade-Level Educational Decisions:
Screening
 Grade-level trends or issues
 Effectiveness of grade-level curriculum and
instruction
 Areas of need and guidance on how to set
measurable grade-level goals
 Students who may need additional instruction or
assessment
National Center on
Response to Intervention
7
Cut Score
 A cut score is a score on a screening test that divides
students who are considered potentially at risk from
those who are considered not at risk.
National Center on
Response to Intervention
8
Identifying Students as At Risk
 RTI success depends on accurate identification of the
students identified as at risk.
 Perfect screening would result in 100% accurate
identification of “True Positives” (those who need
additional support) and “True Negatives” (those who
do not need additional support).
 Cut scores for screening tools are often set to
overidentify students as at risk.
National Center on
Response to Intervention
9
Number of students
Categorical Vs. Continuous
45
40
35
30
25
20
15
10
5
Arbitrary cut
score
True
Negatives
& False
Negatives
True
Positives
& False
Positives
0
Sickle Cell Anemia
No Sickle Cell Anemia
Scores on a measure of reading/math
National Center on
Response to Intervention
10
Clinical Decision-Making Model
At risk
Not at risk
Screen
Outcome
At risk
Not at risk
True
Positive
False
Positive
False
Negative
National Center on
Response to Intervention
True
Negative
True Positive – students
correctly identified at risk
False Positive – students
incorrectly identified at risk
False Negative – students
incorrectly identified not at
risk
True Negative – students
correctly identified not at risk
11
Comparison Based on Changing the Cut
Score
Overlapping distributions
N=100 students
Overlapping distributions
N=100 students
TP
40
FP
10
TP
33
FP
2
FN
10
TN
40
FN
17
TN
48
Poor
Readers
80%
80%
Good
Readers
Poor
Readers
65%
20%
20%
Number of items correct on
screening instrument
National Center on
Response to Intervention
5%
95%
Good
Readers
35%
Number of items correct on
screening instrument
12
Setting Realistic Cut Scores
Poor Readers
Good Readers
Number of items correct on screening instrument
National Center on
Response to Intervention
13
Over Vs. Under Identification
Public Health
Education


Overidentification
•
•

Expense of additional
testing
•
Expense of additional testing
•
Expense of early intervention
services
Unnecessary worry
Underidentification
•
Overidentification
Miss serious health
problem
National Center on
Response to Intervention

Underidentification
•
Miss opportunity for
prevention/early intervention
14
Screening: Establishing Cut Scores
 Logical practices to establish cut scores indicating
skill proficiency
• National cut scores (e.g., AIMSweb, DIBELS)
• Local norms
• Cut scores based on likelihood of demonstrating
mastery on core testing
 Typically based on statistical analysis
National Center on
Response to Intervention
15
Benefits of District-Established
Over School-Established Cut Scores
 More effective and efficient allocation of resources
 Increased buy-in and use of data by schools/teachers
 Common message and focused activities
 Increased equity in access to supplemental supports
National Center on
Response to Intervention
16
Problems with Schools Independently
Establishing Cut Scores
School
School 1
School 2
School 3
National Center on
Response to Intervention
Percent At or Above
School Cut Score
50%
63%
48%
17
Problems with Schools Independently
Establishing Cut Scores
63%
50%
National Center on
Response to Intervention
48%
18
Importance of District Cut Scores
20%
4%
44%
National Center on
Response to Intervention
19
Establishing Routines and Procedures
for Data-Based Decision Making
Data teams should establish:
 Routines and procedures for conducting data
reviews
 Decision-making processes
 Explicit decision rules for assessing student progress
National Center on
Response to Intervention
20
Conducting Data Reviews
 Conduct data reviews at logical, predetermined
intervals
 Schedule data reviews prior to the beginning of
instruction
 Use established meeting structures
 Involve relevant team members
National Center on
Response to Intervention
21
Data-Based Decision-Making Routines
and Procedures
 Articulate routines and procedures in writing
 Implement established routines and procedures with
integrity
 Ensure routines and procedures are culturally and
linguistically responsive
National Center on
Response to Intervention
22
Establishing Routines and Procedures
Consider clarifying the following in writing:
 What are you looking for?
 How will you look for it?
 How will you know if you found it?
National Center on
Response to Intervention
23
Examples of Explicit Decision Rules
Consider articulating, in writing, what happens
when:
 More than 80% of students are above the cut score
 Less than 80% have reached the cut score
 Lack of progress is evident
 Student progress varies by target group (e.g., Title I,
special education, low SES)
National Center on
Response to Intervention
24
Data Analysis
National Center on
Response to Intervention
25
Data Analysis
 Data analysis should occur at the:
• District Level
• School Level
• Grade/Class Level
• Student Level
National Center on
Response to Intervention
26
Purpose of Data Analysis
 Identify students who need additional assessment
and instruction
 Evaluate effectiveness of core curriculum and
instruction
 Allocate resources
 Evaluate effectiveness of instruction programs for
target groups (e.g., ELL, Title I)
National Center on
Response to Intervention
27
Commonly Confused Terms
 Cut Score – score on a screening test that divides
students who are considered potentially at risk from
those who are considered not at risk.
 Target or Benchmark – predetermined level of
performance on a screening test that is considered
representative of proficiency or mastery of a certain
set of skills.
 Criterion Scores – scores on a screening test that
separate students into performance levels (e.g.,
established, emerging, deficient)
National Center on
Response to Intervention
28
Interpreting Screening Data
 Norm Referenced
 Criterion Referenced
 Target Score
National Center on
Response to Intervention
29
Norm Referenced
 Students are measured against others, NOT a
defined criterion.
 Permits a fixed proportion of students to pass and
fail.
• This means that standards may vary from year to year,
depending on the quality of the cohort;
 Effective way of comparing students.
National Center on
Response to Intervention
30
Norm Referenced: Bell Curve
Above Average Range
Above 90th
percentile
90th percentile
Average Range
(middle 50%)
75th percentile
Median (50th percentile)
25th percentile
Below Average Range
10th percentile
National Center on
Response to Intervention
31
Norm Referenced: Box and Whisker
Plots
90
th
%tile
75th
%tile
50th
%tile
25th
%tile
National Center on
Response to Intervention
10th
%tile
32
Norm Referenced: Box and Whisker
Plots Benchmark Scores for Grade 2 Screening Measure
70
Score
60
Above
Average
50
40
Average
30
Below
Average
20
Student
10
Fall
National Center on
Response to Intervention
Winter
Spring
33
Norm Referenced: Box and Whisker
Plots
90th%ile
75th%ile
Words Read Correctly
200
50th%ile
175
25th%ile
150
10th %ile
100
Cut score
75
School A
50
Composite
25
Grade 1
National Center on
Response to Intervention
Grade 2
Grade 3
34
Words Read Correctly
Norm Referenced: Box and
Whisker Plots Handout
160
150
140
130
120
110
100
90
80
70
60
50
40
30
20
10
90th%ile
75th%ile
50th%ile
25th%ile
10th %ile
Cut score
School A
Composite
Grade 2
National Center on
Response to Intervention
35
Criterion Referenced
 Students are measured against defined (and
objective) criteria.
 Criterion-referenced assessment is often, but not
always, used to establish a person’s competence
(whether s/he can do something).
 Criteria typically do not vary from year to year
(unless the criteria change).
National Center on
Response to Intervention
36
Criterion Referenced
There are multiple ways to determine the criterion.
One example is percentile ranks:
• Below 10 percentile
=
deficient
• 10 percentile – 25 percentile =
emerging
• Above 25 percentile
established
National Center on
Response to Intervention
=
37
National Center on
Response to Intervention
38
Norm Referenced Vs. Criterion Referenced
ID
Name
Corrects
Errors
Accuracy
Performance Summary
Cut score=77
Potential Instructional Action
1256
Jim
73
Well Above Average
Continue Primary Prevention
2343
Jenny
70
Well Above Average
Continue Primary Prevention
16705
Jackie
69
Well Above Average
Well Above Average =68 (90th percentile)
Continue Primary Prevention
ID
Name
Corrects
Errors
Accuracy
Performance Summary
Emerging> 75
1256
Jim
73
Emerging
2343
Jenny
70
Emerging
16705
Jackie
69
Emerging
2341
Jill
67
Above Average
Continue Primary Prevention
2341
Jill
67
Emerging
23602
Jerry
67
Above Average
Continue Primary Prevention
23602
Jerry
67
Emerging
14507
Jack
67
Above Average
Continue Primary Prevention
14507
Jack
67
Emerging
6235
Jerome
67
Above Average
Continue Primary Prevention
6235
Jerome
67
Emerging
1267
Joann
67
1267
Joann
67
Emerging
20002
Jared
60
Emerging
Jason
60
Emerging
Above Average
Above Average =66 (75th Percentile)
Continue Primary Prevention
20002
Jared
60
Average
Continue Primary Prevention
12
2345
Jessica
58
Average
Continue Primary Prevention
12325
Jeff
60
Emerging
1384
Jen
58
Average
Continue Primary Prevention
2345
Jessica
58
Emerging
4312
Jim
56
Average
Continue Primary Prevention
1384
Jen
58
Emerging
4312
Jim
56
Emerging
8752
Jeremy
50
Average
Continue Primary Prevention
8752
Jeremy
50
Emerging
14562
Jackson
47
14562
9873
5631
Jackson
Jessie
Jillian
47
41
41
Average
Average = 43 (25th percentile)
Below Average
Below Average
2344
Juanita
40
Below Average
12074
Jaclyn
38
Below Average
13551
Janet
1834
Jade
23515
James
22145
Jed
37
35
Below Average
Below Average = 36 (10th percentile)
Well Below Average
Assess and Consider Tertiary
Prevention
Well Below Average
Assess and Consider Tertiary
Prevention
Assess and Consider Tertiary
Prevention
National Center on
Response
18 to Intervention
Well Below Average
9
Assess and Consider Secondary
Prevention
Assess and Consider Secondary
Prevention
Assess and Consider Secondary
Prevention
Assess and Consider Secondary
Prevention
Assess and Consider Secondary
Prevention
Assess and Consider Secondary
Prevention
Assess and Consider Secondary
Prevention
Assess and Consider Secondary
Prevention
Assess and Consider Secondary
Prevention
Assess and Consider Secondary
Prevention
Assess and Consider Secondary
Prevention
Assess and Consider Secondary
Prevention
Assess and Consider Secondary
Prevention
Assess and Consider Secondary
Prevention
Emerging
Assess and Consider Secondary
Prevention
Assess and Consider Secondary
Prevention
Continue Primary Prevention
Assess and Consider Secondary
Prevention
Assess and Consider Secondary
Prevention
Assess and Consider Secondary
Prevention
Assess and Consider Secondary
Prevention
Assess and Consider Secondary
Prevention
Potential Instructional Action
Cut score=77
Deficient > 46
9873
Jessie
41
Deficient
Assess and Consider Need for Tertiary
Prevention
5631
Jillian
41
Deficient
Assess and Consider Need for Tertiary
Prevention
2344
Juanita
40
Deficient
Assess and Consider Need for Tertiary
Prevention
12074
Jaclyn
38
Deficient
Assess and Consider Need for Tertiary
Prevention
13551
Janet
37
Deficient
Assess and Consider Need for Tertiary
Prevention
1834
Jade
35
Deficient
Assess and Consider Need for Tertiary
Prevention
23515
James
18
Deficient
Assess and Consider Need for Tertiary
Prevention
Deficient
Assess and Consider Need for Tertiary
Prevention
22145
Jed
9
39
Target Score
 Typically based on statistical analysis
 Can be correlated with high-stakes testing
• Example: students who reach the target score have an
80% likelihood of scoring proficient on the state test
National Center on
Response to Intervention
40
Target
Performance
Summary
Established
Established
Potential Instructional Action
Continue Primary Prevention
Continue Primary Prevention
105
Established
Continue Primary Prevention
Jill
Jerry
103
101
Established
Established
Continue Primary Prevention
Continue Primary Prevention
14507
Jack
101
Established
Continue Primary Prevention
06235
Jerome
90
Established
Continue Primary Prevention
01267
20002
Joann
Jared
88
86
Established
Established
Continue Primary Prevention
Continue Primary Prevention
ID
01256
02343
Name
Jim
Jenny
Corrects
107
107
16705
Jackie
02341
23602
Errors
Accuracy
------------Cut score = 82----------Established
00012
Jason
80
Continue Primary Prevention
12325
Jeff
77
Established
Continue Primary Prevention
02345
Jessica
77
Established
Continue Primary Prevention
01384
Jen
74
Established
Continue Primary Prevention
04312
08752
Jim
Jeremy
72
71
Established
Established
Continue Primary Prevention
Continue Primary Prevention
Emerging > 70
14562
Jackson
69
Emerging
Assess and Consider Secondary Prevention
09873
Jessie
69
Emerging
Assess and Consider Secondary Prevention
05631
Jillian
60
Emerging
Assess and Consider Secondary Prevention
02344
Juanita
57
Emerging
Assess and Consider Secondary Prevention
12074
Jaclyn
55
Emerging
Assess and Consider Secondary Prevention
13551
Janet
53
01834
Jade
National Center on23515 James
Response to Intervention
22145
Jed
43
39
31
Emerging
Assess and Consider Secondary Prevention
Deficient > 46
Deficient
Assess and Consider Need for Tertiary Prevention
Deficient
Assess and Consider Need for Tertiary Prevention
Deficient
Assess and Consider Need for Tertiary Prevention
41
DISTRICT DATA ANALYSIS
National Center on
Response to Intervention
42
District Level: Box and Whisker Plots
Handout
90th%ile
75th%ile
200
50th%ile
175
25th%ile
150
10th %ile
100
Target
School A
75
School B
50
School C
School D
25
School E
Grade 1
National Center on
Response to Intervention
Grade 2
Grade 3
Composite
43
Comparison of Benchmark Scores for
Grade 2 Across The Year
Percent
100
90
80
18
70
60
50
40
27
30
13
17
16
80
70
5555
20
10
Fall
National Center on
Response to Intervention
Winter
Spring
44
Analyzing Growth of Subgroups
Across the District
Words Read Correctly
140
Other
Target Score
120
Title I Status
100
Special
Education
80
60
40
20
Fall
National Center on
Response to Intervention
Winter
Spring
45
Analyzing Growth of Ethnic Groups
Across the District
Words Read Correctly
Caucasian
140
Unidentified
120
Target Scores
Hispanic/Latino
100
African American
80
Asian
60
40
20
Fall
National Center on
Response to Intervention
Winter
Spring
46
Words Read Correctly
Analyzing Growth of English Language
Learners Across the District
140
Non-ELLs
120
Target
Scores
100
ELLs
80
60
40
20
Fall
National Center on
Response to Intervention
Winter
Spring
47
SCHOOL-LEVEL DATA ANALYSIS
National Center on
Response to Intervention
48
Norm Referenced:
Box and Whisker Plots
90th%ile
75th%ile
200
50th%ile
Score
175
25th%ile
150
10th %ile
Target
Score
100
75
School A
50
Composite
25
Grade 1
Grade 2
National Center on
Response to Intervention
Grade 3
Grade 4
Grade 5
49
Performance of Average Student
Benchmark Scores for Grade 1-5 Screening Measure
Fall
70
Winter
60
Spring
Score
50
40
30
20
10
Grade 1
Grade 2
National Center on
Response to Intervention
Grade 3
Grade 4
Grade 5
50
Analyzing Growth of Ethnic Groups at
the School Level Handout
Caucasian
Words Read Correctly
140
Unidentified
Target Scores
120
Hispanic/Latino
100
African American
80
Asian
60
40
20
Fall
National Center on
Response to Intervention
Winter
Spring
51
GRADE- AND CLASSROOM-LEVEL
ANALYSIS
National Center on
Response to Intervention
52
Grade Level—Analyzing Effects of
Changes to Instruction Handout
100
Percent
90
80
70
60
50
40
30
12
15
10
14
15
32
5573
53
76
20
10
Fall
National Center on
Response to Intervention
Winter
Spring
53
Analyzing Growth of Special
Education Students by Grade or Class
Words Read Correctly
140
Other
120
Target
Scores
Special
Education
100
80
60
40
20
Fall
National Center on
Response to Intervention
Winter
Spring
54
Classroom Comparison
Fall
70
Winter
Score
60
Spring
50
40
30
20
10
Class 1
National Center on
Response to Intervention
Class 2
Class 3
Class 4
55
ID
01256
02343
Name
Jim
Jenny
Corrects
107
107
16705
02341
Jackie
Jill
105
103
23602
14507
06235
Jerry
Jack
Jerome
101
101
90
01267
20002
00012
Joann
Jared
Jason
88
86
80
Established
Established
Established
Continue Primary Prevention
Continue Primary Prevention
Continue Primary Prevention
12325
Jeff
77
Established
Continue Primary Prevention
02345
01384
04312
08752
Jessica
Jen
Jim
Jeremy
77
74
72
71
Established
Established
Established
Established
Continue Primary Prevention
Continue Primary Prevention
Continue Primary Prevention
Continue Primary Prevention
Emerging > 70
Emerging
14562
Jackson
69
09873
Jessie
69
05631
02344
12074
13551
Jillian
Juanita
Jaclyn
Janet
60
57
55
53
01834
Jade
43
23515 Center
James
39
National
on
Response
22145 to Intervention
Jed
31
Errors
Accuracy
Performance
Summary
Established
Established
Potential Instructional Action
Continue Primary Prevention
Continue Primary Prevention
Established
Established
-------------Cut score = 102----------Established
Established
Established
Emerging
Continue Primary Prevention
Continue Primary Prevention
Continue Primary Prevention
Continue Primary Prevention
Continue Primary Prevention
Assess and Consider Secondary Prevention
Assess and Consider Secondary Prevention
Emerging
Assess and Consider Secondary Prevention
Emerging
Assess and Consider Secondary Prevention
Emerging
Assess and Consider Secondary Prevention
Emerging
Assess and Consider Secondary Prevention
Deficient > 46
Deficient
Assess and Consider Need for Tertiary Prevention
Deficient
Assess and Consider Need for Tertiary Prevention
Deficient
Assess and Consider Need for Tertiary Prevention
56
IDENTIFYING STUDENTS IN NEED OF
ADDITIONAL SUPPORT
National Center on
Response to Intervention
57
Student Comparison Lower than Norm
90th%ile
75th%ile
200
50th%ile
Score
175
25th%ile
150
10th %ile
100
Target
Student
75
50
25
Fall
National Center on
Response to Intervention
Winter
Spring
58
Student Comparison Higher than Norm
90th%ile
75th%ile
200
50th%ile
Score
175
25th%ile
150
10th %ile
100
Cut score
Student
75
50
25
Fall
National Center on
Response to Intervention
Winter
Spring
59
Identifying Students in Need of
Additional Support
 May vary based on needs and resources of school
• Target or criterion scores
• Lowest percentage of students whose needs can be
met by resources (e.g., 20%)
 If more than 20%, focus should be on improving core
instruction/curriculum
National Center on
Response to Intervention
60
Secondary Level or Tertiary Level Support
Access to
supplemental
supports may be
based on school
resources
ID
Name
1256
Jim
107
Established Continue Primary Prevention
2341
Jill
103
Established Continue Primary Prevention
Cut Score=100
6235
Jerome
90
Established Continue Primary Prevention
2345
Jessica
77
Established Continue Primary Prevention
Emerging > 75
1384
4312
National Center on
Response to Intervention
Jen
Jim
Corrects Errors
74
72
Accuracy
Performance
Summary Potential Instructional Action
Emerging
Assess and Consider
Secondary Prevention
Emerging
Assess and Consider
Secondary Prevention
Assess and Consider
Secondary Prevention
13551
Janet
53
Emerging
Deficient> 46
1834
Jade
43
Deficient
Assess and Consider Need
for Tertiary Prevention
22145
Jed
31
Deficient
Assess and Consider Need
for Tertiary Prevention
61
Problems with Cut Scores to
Determine Supplemental Support
80%
56%
National Center on
Response to Intervention
96% in
need
62
Target Identification Rate
 Target identification rate is the proportion of students
to be identified as at risk.
• May depend on program objectives and resources.
 Unique target identification rates may be specified for
different skill areas.
 Schools and districts will need to think about
reallocating resources or securing additional funds to
support all students in need.
National Center on
Response to Intervention
63
Determining Target Identification Rate
Tertiary
3%
5%
Secondary
15%
12%
Primary
80%
85%
School 1:
School 2:
Resources available for 20%
Resources available for 15%
National Center on
Response to Intervention
64
Things to Remember
 Good data IN… Good data OUT
• Know where your data came from and the validity of
that data
 Focus on the big picture for ALL students
• Are most students making progress?
 ALL instructional and curriculum decisions should be
based on DATA.
 Keep it SIMPLE and EFFICIENT!
National Center on
Response to Intervention
65
Need More Information?
National Center on Response to Intervention
www.rti4success.org
RTI Action Network
www.rtinetwork.org
IDEA Partnership
www.ideapartnership.org
National Center on
Response to Intervention
66
National Center on
Response to Intervention
This document was produced under U.S. Department of
Education, Office of Special Education Programs Grant No.
H326E07000.4 Grace Zamora Durán and Tina Diamond served
as the OSEP project officers. The views expressed herein do
not necessarily represent the positions or policies of the
Department of Education. No official endorsement by the U.S.
Department of Education of any product, commodity, service
or enterprise mentioned in this publication is intended or
should be inferred. This product is public domain.
Authorization to reproduce it in whole or in part is granted.
While permission to reprint this publication is not necessary,
the citation should be: www.rti4success.org.
National Center on
Response to Intervention
67