Y3D1 SBLT
Tier 3 Problem Identification &
Problem Analysis
School Implementation Blueprints
A collaborative project between the Florida Department of Education and the University of South Florida
R
Advance Organizer
•
•
•
•
Year 3 Training Plan & Rationale
Importance of Established PS Process
Tier 3 Problem Identification - Worksheet
Tier 3 Problem Analysis – Worksheet
Training Outline Influences
• Skills to Reinforce
–
–
–
–
–
–
• New Skills
Problem Solving Process
– Tier 3 Data Sources
Intervention Dev/Support
– Characteristics of Tier 3
Interventions
Intervention Integrity
– Tier 3 Scheduling/Resource
Integrating the Tiers
Let’s look at the dataMapping
which
Decision Rules
drove a few of these–decisions
Integrating the Tiers
Scheduling
– Eligibility
• Other Indications from Data Review
–
–
–
–
Staff Involvement
Parent Involvement
School Implementation Plan
Using Data to Inform Implementation
–
–
–
–
Scaffolding
District-School Communication
Facilitation
Graphing/Technology
Problem Solving
Process
Staff
Involvement
Scaffolding
Year 1 SBLT:
Scaffolding Heavy
100%
80%
93
85
88
76
60%
Average…
40%
20%
0%
1. Problem ID
2. Problem
Analysis
3. Intervention 4. Progress
Development Evaluation/RtI
Year 1 SBLT:
Scaffolding Lite
Scaffolding
100%
80%
60%
80
Average Score
40%
35
20%
0%
1. Identification of Skill Area Problem
1. Problem ID: Review
Situation
2. Application of Problem ID to
Novel Situation
2. Problem ID: New
Situation
Scaffolding
Year 2 SBLT: Scaffolding
Heavy
Direct Skills Assessments: All Project Schools SBLTs
100
Percentage of Possible Points
Attained
93
90
82
82
Problem ID & Intervention Decisionmaking
Intervention Plan Evaluation
80
70
60
50
40
30
20
10
0
Problem Identification
Area Assessed by Direct Skills Exercise (Team Exercises)
Direct Skills Assessments
All Project Schools Targeted Grade-Level Staff
100
Percentage of Possible Points Attained
Scaffolding
90
80
74
70
60
56
56
B.O.Y. Data Use: Scaffolding Medium
B.O.Y. Data Use: Scaffolding Medium
50
40
30
20
10
0
Year 1
E.O.Y. Data Use: Scaffolding Heavy
Year 2
Area Assessed by Direct Skills Exercise
School
Implementation
Plan
District-School
Communication
Implications for
Training Plan
Facilitated Through:
• Problem Solving Process
• Scaffolding
• New Skills
• School Implementation Plan
• District-School Communication
• Staff Involvement
Skills Training
School Level
Blueprint for
Implementation
Review of Y3 Training
Plan
• Instructional Template
– Case Study Format
– Four Steps of Problem Solving at Tier III
– Worksheets
– Skills Exercises
Problem Solving
Process
• Goal is Student Achievement
– At Tiers I, II, and III
• Accomplished through ongoing
instructional decision making in
response to student performance
data
Response to
Instruction/
Intervention
• after the delivery of instruction
Intervention
Design
• based upon verified hypotheses of
why
• an identified problem is occurring.
Problem
Analysis
Problem
Identification
Problem Solving
Process
Identify
the Problem
Analyze
the Problem
Design
Intervention
Implement
Intervention
Monitor
Progress
Evaluate
Intervention
Effectiveness
J
L
Identifying Students in Need of Individualized, Intensive Instruction
TIER 3 PROBLEM IDENTIFICATION
Three Tiered Model of School Supports
Academic Systems
Behavioral Systems
Tier III: Comprehensive and
Intensive Interventions
Tier III: Intensive Interventions
( Few Students)
Provided to students who need Individual
Intervention
Provided to students who need
Individualized Interventions
( Few Students)
Tier II: Strategic Interventions
Tier II: Targeted Group Interventions
(Some Students)
(Some Students)
Provided to students who need
more support in addition to the
core curriculum
Provided to students who need more
support in addition to school-wide
positive behavior program
Tier I: Core Curriculum
Provided to All students
Tier I: Universal Interventions
Provided to all students; all
settings
16
From the Florida Problem Solving/Response to Intervention Project
Tier 3: Intensive,
Individualized Instruction
• Services provided to students who are
intensely behind and/or not
sufficiently responsive to core and
strategic (Tier 2) instruction.
• Required by small percentage of
students (approximately 5%)
Identifying A Tier 3
Problem
Typically students come to our attention
through one of the following:
• Periodic review of universal screening
data
• Review of progress monitoring data for
students receiving strategic instruction
• Teacher or parent referral to PS team
Digits correct per minute
Math Computation
peer group
benchmark
target student
90
Tier 2 Bar Graph with Trend by Student (5)
80
70
60
AIM LINE
50
S1
S2
S3
40
S4
S5
30
target student
20
10
0
B W1 W3 W5 W7 W9
B W1 W3 W5 W7 W9
B W1 W3 W5 W7 W9
B W1 W3 W5 W7 W9
B W1 W3 W5 W7 W9
Tier 3: Never Assume
When a student is identified for intensive
instruction, data should indicate:
• Core and strategic instruction has been
provided with fidelity
• Access to effective core instruction (80%
of students successful)
• Access to effective strategic instruction
(70-80% successful)
Problem Identification
Data Required:
• Replacement behavior or target skill
• Current/Observed level of performance
• Expected level of performance
• Peer level of performance
• GAP analysis
Review Replacement
Behavior or Target Skill
Replacement behaviors/target skills should:
• be specific, measurable and observable
• indicate what you WANT the student to
be able to do
• attract reinforcement
Standards for Expected
Level of Performance
Sources include, but are not limited to:
• School, district, state or national norms
• Benchmark standards
• District standards and/or benchmarks
• Teacher expectations
• Direct peer comparison
• Criteria for next environment
(Kurns & Upah, 2007, p. 44)
“Which one do I use?”
Choose the standard specific to the target
behavior.
When available, it may be appropriate to use
more than one.
For example, if both benchmark and local
norms are used we know how level of
current performance compares to research
standards and peer performance.
(Kurns & Upah, 2007, p. 44)
Performance
Discrepancy
The “gap” or magnitude of the
discrepancy is the size of the
difference between expectation and
current/observed performance
Reviewing Gap
Analysis
Gap =
Expected Level
Current/Observed Level
• Rule of thumb: 2.0 = significant gap
Reviewing Gap Analysis
•
Target Student’s Observed/Current Level of Performance:
– 40 WCPM
•
Expected Level of Performance
– 92 WCPM
•
Peer Level of Performance
– 98 WCPM
•
GAP Analysis:
–
–
•
Expected Level/Target Student
Expected Level/Peer
92/40= 2+X difference SIGNIFICANT GAP
92/98= <1 X difference NO SIGNIFICANT GAP
Is this an individual student problem?
Individual Student
Problem
120
100
80
60
40
20
0
0
1
2
3
4
5
6
7
8
9
10
11 12
Considering Tier 3
Is Tier 2 Effective?
• At least 70% - 80% of students should
benefit from Tier 2
• Decision Making Options:
– Integrity + Rate of Response
– Unit of analysis is the group of students
• % benefiting from Tier 2 supports
Rose
Struggling in target skill area.
Tier One data indicate that approximately 15%
of the student’s in Rose’s class demonstrate
this discrepancy. Core effectiveness: 85%
Rose is receiving 20 min. of strategic instruction
(Tier 2) five times per week along with two
other classmates.
Is Tier 2 instruction
effective for 70-80%?
benchmark
Are Tier 3 services
appropriate?
Yes
The Tier 2 Rose has been receiving is
effective for approximately 70% of the
intervention group.
Intervention integrity is very good.
Jarrett
Not proficient in target skill area.
Tier One data indicate that approximately 20%
of the student’s in Jarrett’s class
demonstrate this discrepancy. Core
effectiveness: 80%
Jarrett is receiving 20 min. of strategic
instruction five times per week specific to the
target skill area.
Is Tier 2 instruction
effective for 70-80%?
benchmark
Are Tier 3 services
appropriate?
No
Intervention integrity is good.
Yet, progress monitoring data indicate
that the Tier 2 intervention Jarrett is
being provided is not effective for 7080% of the intervention group.
Stephanie
Below benchmark in target skill area.
Tier One data indicate that approximately 20%
of Stephanie’s class demonstrate this
discrepancy. Core effectiveness: 80%
Stephanie is receiving 30 min. of strategic
instruction 3 times per week along with a
small group of classmates.
Is Tier 2 instruction
effective for 70-80%?
benchmark
Are Tier 3 services
appropriate?
No
The Tier 2 Stephanie has been receiving
is effective for approximately 70% of
the intervention group.
However, Tier 2 intervention integrity
has been jeopardized due to
Stephanie’s excessive absences.
Randy
Review case study
Randy
• What is the replacement behavior or target skill? (measurable,
observable, reportable)
• What is the student’s current level of performance? (Be sure to
include data that directly assesses the target skill you want the
student to perform).
• What is the benchmark/expected level of performance?
• What is the peer level of performance?
• Gap Analysis
Benchmark & Student
Benchmark & Peer
Peer & Student
• What percentage of students in the classroom demonstrate this
discrepancy?
• At what tier will this problem be addressed (circle one)?
Tier I, Tier II, Tier III
Randy
• Do we have enough information to complete
Problem Identification?
• If yes, go to Problem Analysis
• If no, what information is still needed?
• When will we meet again?
Your Student
• As a team, complete worksheet for
Problem Identification
Identifying Students in Need of Individualized, Intensive Instruction
TIER 3 PROBLEM ANALYSIS
How Does it Fit Together?
Step 2
Step 1
All Students at
a grade level
Intensive
Supplemental
Behavior
Academics
ODRs
Monthly
Bx
Screening
BenchMark
Assessment
Step 3
Addl.
Diagnostic
Assessment
Instruction
Individual
Diagnostic
Individualized
Intensive
Results
Monitoring
1-5%
5-10%
Group
Diagnostic
Small
Group
Differentiated
By Skill
As necessary
Approximately
monthly
Core
Annual
Testing
Step 4
80-90%
None
Continue
With
Core
Instruction
Grades
Classroom
Assessments
Yearly Assessments
Problem Analysis:
Problem Analysis is the process of
gathering information in the domains of
instruction, curriculum, environment,
and learner (ICEL) through the use of
reviews, interviews, observations, and
tests (RIOT) in order to evaluate
underlying causes of a problem and to
validate hypotheses.
Tier 3 Problem Analysis
Diagnostic process…
• Student individually assessed
• Determine most probable cause of
problem
• Match intervention to student need
Content area experts are necessary!
Role of assessment in
problem analysis
Question-driven assessment process:
• What do we know/need to know about
problem?
• What are some possible causes?
• What are some predictions about
solutions?
Role of assessment
(con’t)
• What data are needed to support/refute
hypotheses?
• What intervention best matches student
need?
Assessment:
• Ongoing process—not an “event”
• Functional—relevant and directly
related
• Purposeful—data not collected until
assessment questions developed
• Formative—will inform instruction
Why is the problem
occurring?
PROBLEM ANALYSIS
Step 1: Gather assessment information
Step 2: Develop hypotheses
Step 3: Validate hypotheses
Step 4: Link to intervention
Step 1: Gather information
To answer these questions:
1) WHY the difference btw.
expected/observed
2) WHAT do we need to teach (curriculum)
and HOW do we need to teach
(instruction)?
3) WHICH intervention will have the highest
probability of being successful?
Determining What Data to
Collect
Educationally
Relevant & Alterable
Known
Information
Gather this Existing
Information
(Classroom screening data, ODRs)
Unknown
Information
Conduct Assessments to
Gather this Information
(Behavior observations, specific skill
assessments)
These are assessment questions
Less Educationally
Relevant & Unalterable
Disregarded or Low
Priority
(Height, eye color)
Don’t Go Here!
(Cognitive processing?)
Gather information that…
• is educationally relevant
• is alterable
• will directly impact student gains in the
classroom environment
Assessment domains
Gather information from:
• Instruction
• Curriculum
• Environment
• Learner
Is there something we could change
about I, C, E that would enable student
to learn?
Assessment methods
Gather information through:
• Review of existing records
• Interview teacher, parent, student, etc.
• Observe in classroom or other
appropriate setting
• Test learner on particular skill/concept
Step 2: Develop hypotheses
Developing assumed causes…
After initial assessments, focus of PA
becomes hypothesis generation—
process of making informed statements
about why a problem occurs
Develop hypotheses
Hypotheses…
• Are developed to determine reasons for why the
replacement behavior is not occurring
• Should be based on research relevant to the target
skills
• Focus on alterable variables
• Should be specific, observable, and measurable
• Should lead to intervention
Develop hypotheses
Hypotheses…
• Must consider both SKILL and PERFORMANCE
deficits:
– Skill Deficit
• Student does not have the skills to perform the task
– Student lacks fluency skill for grade level
– Student lacks private speech for self control
– Performance Deficit
• Student does perform existing skill or performs at lower level
– Student reads slowly because of fear of ridicule by peers for
mistakes
– Peers reinforce bad choices more than teacher reinforces good
choices
General format:
“______________________(the problem)
Is occurring because
_____________________(hypothesis)”
Format for behavior:
“The problem is occurring because when
this occurs____________(trigger), the
student_____________(behavior) in
order to/because_______________
(hypothesized function/reason).
(Heartland AEA 11)
Prediction statement:
• Inference from hypothesis
• What would be expected if some other
action took place
• Comprised of specific actions that are
reasonable and feasible
• If/then wording
• Used to develop assessment questions
to confirm/disconfirm hypotheses
Prediction statement:
“If_____________(specific action) would
occur, then______________(problem)
would be reduced.
Hypotheses Validation
Why do Problem Solving Teams need to
Validate a Hypothesis?
If the hypothesis is inaccurate and the wrong
intervention is implemented valuable time
could be wasted on an intervention that was
not an appropriate instructional match for the
student.
Step 3: Validate hypotheses
• Formulate assessment questions to
gather information or data that will
support/refute hypothesis
• Collect additional data as needed
• Validate before intervention is
implemented
Domains for Assessment
DOMAINS
I
Instruction
C
Curriculum
E
Environment
L
Learner
R
Review
I
Interview
O
Observe
T
Test
Format for Hypothesis Validation
Hypothesis
Prediction
Mary is noncompliant because
she does not have the skills to
complete the work successfully.
Assessment Question(s):
If we reduce the academic
demand or improve her skills,
Mary will become more
compliant.
Is task difficulty appropriate for Mary’s skill level?
Where are the answers?:
Review Learner records for evidence of skills; Review
Curriculum to understand expectation.
Answers:
Review of records and review of curriculum indicates that
Mary has the skills to complete the requested tasks.
Validated?:
No
Format for Hypothesis Validation
Hypothesis
Prediction
a. Mary is not being positively reinforced
for compliant behavior.
b. Mary is being reinforced for
noncompliant behavior
If Mary is positively reinforced for
compliant behavior / not reinforced for
noncompliant behavior, her compliance
will increase.
Assessment Question(s):
Is Mary being positively reinforced for compliant behavior?
Is Mary being reinforced for noncompliant behavior?
Where are the answers?:
Observe the Environment in the situations where Mary
displays noncompliance and compliance.
Answers:
Observations indicate that Mary is not being consistently reinforced for
compliance in large group settings outside of the homeroom, but is being
consistently reinforced within the homeroom where she displays compliant
behavior. She is also avoiding assignments through noncompliance.
Validated?:
Yes
Step 4: Link to intervention
From validated hypotheses, team selects
hypothesis which seems most likely to
lead to effective intervention
Keep in mind…
• skills of interventionist
• feasibility of implementation
Problem Analysis
Practice
Joe has difficulty keeping his hands to
himself in the lunchroom. Examples include
when he touches other students, grabs
students’ plates and food, and pushes trays
of food onto the floor.
“Joe” example from Heartland AEA 11, “Polishing our Practice”
Teamwork
• Identify the specific assessment questions
you would ask regarding Joe’s lunch room
problem
– Consider each of the ICEL domains
• How would you collect the info (RIOT x
ICEL)?
Problem Analysis: Joe’s
Assessment Questions
• What are lunch room rules?
• Have they been taught?
• Is he having problems with
specific students?
• What has been tried before
and how did he respond?
• Does it happen with certain
lunchroom attendants?
• What does he gain or avoid?
• What is the adult:kid ratio?
• Has Joe had scientificallybased instruction in reading?
• Does Joe have limited
English proficiency?
• Does Joe have health or
medical concerns?
77
Assumed Causes
Joe touches other people’s food and
pushes trays of food to the floor when he
does not get to sit by two specific people in
order to gain the preferred seating
arrangement. He has trouble using verbal
communication to let people know where he
wants to sit.
Prediction statements
Predictions:
• If there was a seating arrangement for Joe to
sit by his preferred lunch partners, then Joe’s
inappropriate use of hands would decrease.
• If Joe had a way to communicate his wants
for lunch partners, his inappropriate use of
hands would decrease.
Validate Hypothesis
Information from interview of lunchroom
supervisors indicates that Joe has fewer
incidences of inappropriate use of hands on
days when he is sitting by preferred lunch
partners. Information from teacher indicates
that when Joe can communicate his wants
and needs, he displays fewer inappropriate
physical behaviors.
Linking Problem Analysis to
Intervention Design
Recommendations for intervention design based
on problem analysis:
• Provide a seating arrangement for lunch
where Joe is seated next to at least one of his
preferred lunch partners
• Continue to work on teaching Joe to verbally
communicate his wants and needs.
Randy
Review case study
Randy
• Hypothesis 1: The problem is occurring because: the curriculum
being delivered to him does not address reading fluency and
accuracy.
• Prediction Statement 1: If ____ would occur, the problem would be
reduced
• Relevant Data
• Validated Yes/No
Randy
• Hypothesis 2: The problem is occurring because: Randy does not
have adequate decoding skills to read accurately and fluently.
• Prediction Statement 2: If ____ would occur, the problem would be
reduced
• Relevant Data
• Validated Yes/No
Randy
• Hypothesis 3: The problem is occurring because: Randy does not
self-monitor while reading.
• Prediction Statement 3: If ____ would occur, the problem would be
reduced
• Relevant Data
• Validated Yes/No
Randy
• Hypothesis 4: The problem is occurring because: Randy does not
have adequate grade level sight words.
• Prediction Statement 4: If ____ would occur, the problem would be
reduced
• Relevant Data
• Validated Yes/No
Randy
• Do we have enough information to complete Problem Analysis?
• If yes, go to Intervention Development
• If no, what information is still needed?
• When will we meet again?
Your Student
• Complete worksheet for Problem Analysis