Monitoring Student
Progress to Develop
Standards-Based IEPs
OSEP GSEG Project Manager’s Meeting
Gerald Tindal, Ph. D
University of Oregon
Martin Ikeda, Ph. D.
Iowa Department of Education
1
Overview

Intent is to provide an overview of some
ideas we have been thinking about


Jerry has researched CBM for 25 years +
Marty has supported implementation for 12
years +
Conversational presentation-take
comments and questions as we go
 Representative of our best-thinking todate

2
“Big Ideas”
CBM is a viable tool for decision making
about participation in the alternate
assessment against modified academic
achievement standards
 CBM is a viable tool for decision making
about progress against grade level
standards

3
Issues around 2%:

IEP Participation Decision:



Objective evidence demonstrating that the student’s
disability has precluded the student from achieving
grade-level proficiency in the content area assessed.
The IEP team is reasonably certain that, even if
significant growth occurs, the student is not likely to
achieve grade level proficiency within the year covered
by the IEP
IEP Development Issues


IEP goals based on State grade level academic content
standards
Means for an annual determination of progress
4
Our interpretation

Need a way to operationalize proficient
performance on grade level content
standards


Need a way to predict if the child can
realistically achieve grade level proficiency
within one year


State Test: 1X/year depiction
“1-2 years behind?”
Need a way to monitor performance
toward the operationalization
5
What we want:
Align decisions about participation,
present levels of academic achievement
and functional performance and IEP goals
referencing grade-level proficiency
 Assess progress more frequently than
annually, so that instructional effects can
be assessed and changes made to
programs if needed

6
Consider Golf

Many Components of a Good Golf Game





Grip
Choosing the Correct Club
Backswing & Follow Through
Putting Skill
General Outcome Measure for Golf

Number of Strokes
7
Curriculum-Based Measures: A potential
solution

CBM is a validated
technique for a variety of
decisions, but particularly
for monitoring
performance over time






General Outcomes
Brief
Repeatable
Sensitive to Changes in
Performance over Time
Operationalize content
standards at grade level
(ambitious)
Support instructional
decision-making
8
Jacob: Grade 5

Grade level proficiency standard:



75 wpm local norm
100 wpm published performance level
Jacob: 25 wpm in Grade level material

Problem?
9
Illustration: Jacob


Examination of
performance against
other 5th graders in the
district (local norm)
Data generated during
Spring for fifth graders
on Grade 5 material

In the Fall, Jacob
would be given probes
from Grade 5 from
year’s end material
150
140
130
120
110
100
90
80
70
60
50
40
30
20
5
10
Oral
Reading
Fluency
Jacob's
Performance
Compared to Peers
90th percentile
75th percentile
50th percentile
25th percentile
10th percentile
Jacob Score
10
What are realistic growth rates in
reading?







Grade
Grade
Grade
Grade
Grade
Grade
1:
2:
3:
4:
5:
6:
2 words correct/week
1.5 words correct/week
1 word correct/week
.85 words correct/week
.50 words correct/week
.30 words correct/week
It may be difficult for Jacob to “catch up” by
year’s end. The IEP team might decide he is a
candidate for the Alternate Assessment against
Modified Academic Achievement Standards.
11
Illustration: Juarez

In the Fall, on year end material,
Juarez is reading 80 wpm.


Students reading at this rate are
getting meaning from text
They may not be reading fluently
enough to earn a proficient score
on the AYP test

However, Juarez is performing
within grade level & is likely to
“catch up” by year’s end

It would be defensible for the IEP
team to conclude that Juarez is
not a candidate for the alternate
assessment against modified
academic achievement standards
and instead participate in the
general assessment with
accommodations
150
140
130
120
110
100
90
80
70
60
50
40
30
20
5
10
Oral
Reading
Fluency
Juarez’
Performance
Compared to Peers
And Performance
Standard
90th percentile
75th percentile
50th percentile
25th percentile
10th percentile
Juarez Score
12
CBM and Participation Decisions

Potentially useful framework



Grade Level Proficiency
Projected Growth
Establishing alignment between the CBM
metric, Grade Level Content Standards,
and Grade Level Proficiency
13
Alternate Assessments based on
Modified Academic Achievement Standards
Connecting CBM in Reading
with Grade Level Standards
Gerald Tindal
University of Oregon
14
Alternate Forms


Progress monitoring requires alternate forms to allow meaningful
interpretation of student data across time. Without such cross-form
equivalence, changes in scores from one testing session to the next are
difficult to attribute to changes in student skill or knowledge.
As student reading skills progresses through the different skill areas in the
broad construct of reading, it is necessary to use different reading measures
to be able to continue to track the progress students are making as
developing readers
15
Technical Reports



Alonzo, J. & Tindal, G. (2007). The Development of Early
Literacy Measures for use in a Progress Monitoring
Assessment System: Letter Names, Letter Sounds, and
Phoneme Segmenting. Technical Report # 39. University of
Oregon, Eugene: Behavioral Research and Teaching.
Alonzo, J. & Tindal, G. (2007). The Development of Word and
Passage Reading Fluency Measures for use in a Progress
Monitoring Assessment System. (Technical Report # 40).
University of Oregon, Eugene: Behavioral Research and
Teaching.
Alonzo, J., Liu, K., & Tindal, G. (2007). Examining the
Technical Adequacy of Reading Comprehension Measures in a
Progress Monitoring Assessment System. (Technical Report
#41). University of Oregon, Eugene: Behavioral Research and
Teaching.
16
Design of Alternate Measures
Defined universe of items in a pilot
 Used common items and nonequivalent
groups design
 Scored tests at the item level
 Reassembled items for equivalent forms

17
Distribution of the Measures
Across the Grades
Grade Ltr
Names
Ltr
Sounds
Phon.
Seg
Word
Fluency
K
X
X
X
X
1
X
X
X
X
Passage
Fluency
MC
Comp
X
2
X
3
X
X
4
X
X
18
Data Analyses
One-parameter Rasch model
 Estimates the difficulty of individual test
items and the ability level of each
individual test taker
 Standard error of measure
 Mean square outfit to evaluate goodness
of fit (values in the range of 0.50 to 1.50)

19
Letter Names, Sounds, Segmenting



16 letter names exceeded mean sq outfit of 1.5 but
were included given low SEM-3 letters found to not fit
(g, H, and Y)
16 letter sounds exceeded mean sq outfit of 1.5 but
were included given low SEM-6 letter sounds found to
not fit (B, C, d, j, p, and Qu)
A total of 181 words used in segmenting remained in
the item bank
20
Word Reading Fluency



Tests students’ ability to read both sight-words and
words following regular patterns of letter/sound
correspondence in the English language
Students are shown a series of words organized in a
chart on one side of a single sheet of paper and given a
set amount of time (30-60 seconds)
The words we used during the pilot study came from a
variety of sources: Dolch word lists, online grade-level
word lists, and a list of ‘the first 1000 words’ found in
Frye’s Book of lists (1998).
21
Word List Design




Between 144 and 2654 students provided pilot test data on
each word
We kept each of the pilot forms short (68 words in
Kindergarten, 80 in grades 1-3)
We administered 5 different forms of the Word Reading
Fluency test to students in Kindergarten, 4 forms to
students in first grade, and 3 forms to students in third and
fourth grade.
Each form contained 5 words that served as anchor items,
common across all 15 forms of the test (and appearing in
the same location)
22
Passage Reading Fluency
Tests students’ ability to read
connected narrative text accurately. In
this individually-administered measure,
students are shown a short narrative
passage (approximately 250 words)
 Omissions, hesitations, and
misidentifications were counted as
errors

23
Passage Fluency Design




Measures were all written specifically for use in this
progress monitoring assessment system.
All 80 passages were written by graduate students enrolled
in College of Education courses in the winter of 2006
Passage writers followed written test specifications and
were systematically reviewed by Lead Coordinator and then
teachers in field
Each passage was divided into three paragraphs of
approximately even length and checked the readability of
each paragraph using the Flesch-Kinkaid readability index
(1.5, 2.5, 3.5, 4.5)
24
Analysis
On word list, we used Rasch analysis to
scale words on difficulty and ability
 For passages, we analyzed correlations
and mean differences between the
different forms of the measures using a
repeated measures analysis
 Variations in passage outcomes were
reduced by rewriting passages

25
Results of Word List


Initial analyses revealed 283 words outside the
acceptable Mean Square Outfit range of 0.50 –
1.50. These items were dropped from the item
bank, resulting in 465 remaining words
List created with the easiest words appearing first
in the list and subsequent words increasing in
difficulty
26
Word List – Easiest 10
Word
Count
Measure
Mean Square Outfit
I
238
-7.33
1.36
is
195
-6.31
1.29
1960
-6.21
1.10
it
195
-6.01
1.21
ten
243
-5.65
1.00
top
195
-5.37
0.93
and
2654
-5.20
1.15
an
195
-4.90
0.95
sun
195
-4.84
0.71
man
245
-4.32
1.37
the
27
Word List – Most Difficult 10
Word
Count
Measure
Mean Square Outfit
produce
208
4.09
1.11
cultivate
243
4.11
1.30
period
193
4.24
0.69
irrigate
243
4.41
1.00
divided
254
4.65
0.66
deception
210
4.70
1.14
thousands
254
4.76
0.76
commercial
243
4.78
1.31
though
254
5.33
1.37
compromise
210
5.36
1.19
28
Grade 3 Passages
Passage
Title
n
M
SD
Gr3PR_1_C
Susan’s New School
239
128.79
39.00
Gr3PR_2_C
Sara’s Fun Visit
240
131.38
44.29
Gr3PR_3_C
Horses at the Fair
241
125.15
39.18
Gr3PR_4_C
Ben’s Truck
240
127.55
43.61
Gr3PR_5_C
Surprise Sandwiches
242
128.55
36.85
Gr3PR_6_C
Swiming
243
131.77
40.99
Gr3PR_7_C
Boring Weekends
243
121.67
43.07
Gr3PR_8_C
Birthday Wishes
243
124.92
40.79
Gr3PR_9_C
A Special Bike
239
126.45
39.93
Gr3PR_10_C
The New Puppy
240
118.22
38.49
Gr3PR_11_C
Childhood Dreams
239
103.28
41.57
Gr3PR_12_C
The Perfect Instrument
240
121.64
40.47
Gr3PR_13_C
The Breaking Story
237
118.58
39.99
Gr3PR_14_C
The Dream House
237
124.06
41.83
Gr3PR_15_C
American Sports
237
110.19
37.67
Gr3PR_16_C
The Backpacking Trip
236
119.29
40.80
Gr3PR_17_C
The Garden
231
116.26
37.23
Gr3PR_18_C
Abby’s Birthday
230
126.10
39.16
Gr3PR_19_C
Sammy the Shark
231
143.02
45.36
Gr3PR_20_C
Mike’s Red Sneakers
231
119.28
44.62
29
Grade 4 Passages
Passage
Title
n
M
SD
Gr4PR_1_C
Birthday Surprise
207
134.82
35.00
Gr4PR_2_C
Amusement Park
208
139.96
37.74
Gr4PR_3_C
Farm Dog Goes to Town
208
135.29
36.77
Gr4PR_4_C
A Day of Celebration
208
137.56
38.45
Gr4PR_5_C
Billy’s Garden with Grandpa
204
143.63
38.65
Gr4PR_6_C
Maria’s Secret Friend
204
130.35
34.83
Gr4PR_7_C
Lisa Gets to Drive
204
139.11
42.22
Gr4PR_8_C
Toni the Shark
203
132.88
39.62
Gr4PR_9_C
Marta’s New Sweater
203
139.84
41.27
Gr4PR_10_C
Back to School
203
132.83
38.68
Gr4PR_11_C
The Perfect Present
200
131.39
36.65
Gr4PR_12_C
The Perfect Assignment
200
136.51
40.32
Gr4PR_13_C
President David
198
141.40
38.44
Gr4PR_14_C
Above the Clouds
199
138.70
37.68
Gr4PR_15_C
Super Powers
198
131.42
38.79
Gr4PR_16_C
A Friend for Jared
199
131.19
42.27
Gr4PR_17_C
Fieldtrip to the Zoo
196
139.05
42.69
Gr4PR_18_C
Hurt Feelings
195
136.56
39.41
Gr4PR_19_C
Billy and Spike
195
135.96
44.92
Gr4PR_20_C
The Rainy Day Jar
195
136.76
43.55
30
MC Reading Comprehension



We developed the MC Comprehension Tests in a twostep process.
 First, we wrote the stories that were used as the
basis for each test
 Then, we wrote the test items associated with each
story
 We embedded quality control and content review
processes in both these steps throughout instrument
development
Stories were narrative fiction of approximately 1500
words with three types of items written from them:
literal, inferential, and evaluative
20 items per story were developed with 6-7 items of
each type noted above; 3-options were provided
31
Authors of MC Test

The lead author, who oversaw the creation and revision of the stories and test items earned her Bachelor of Arts degree in
Literature from Carleton College in 1990, worked for twelve years as an English teacher in California public schools, was
awarded National Board for Professional Teaching Standards certification in Adolescent and Young Adulthood English
Language Arts in 2002, and was a Ph.D. candidate in the area of Learning Assessments / System Performance at the
University of Oregon at the time the measures were created.

The item writer earned his Ph.D. in education psychology, measurement and methodology from the University of Arizona.
He has worked in education at the elementary and middle school levels, as well as in higher education and at the state level.
He held a position as associate professor in the distance learning program for Northern Arizona University and served as
director of assessment for a large metropolitan school district in Phoenix, Arizona. In addition, he served as state Director of
Assessment and Deputy Associate Superintendent for Standards and Assessment at the Arizona Department of Education. He
was a test development manager for Harcourt Assessment and has broad experience in assessment and test development
32
Design of MC Test



We used a common-person / common item
piloting design
The 20 different forms of each grade level
measure were clustered into 5 groups, with 5
forms in each group
Each test grouping contained two overlapping
forms, enabling concurrent analysis of all
measures across the different student samples
33
Sample Analysis
Item Number
Raw Score
Count
Measure
Standard Error
Outfit Mean Squares
1
88
95
-1.78
0.42
.37
2
86
95
-1.47
0.37
.50
3
90
95
-2.18
0.48
.41
4
62
95
0.52
0.24
1.12
5
71
95
-0.05
0.26
1.03
6
25
95
2.53
0.25
2.32
7
72
95
-0.13
0.27
.97
8
75
95
-0.35
0.28
.94
9
74
95
-0.27
0.28
.61
10
48
95
1.29
0.23
1.20
11
64
95
0.4
0.25
1.06
12
58
95
0.75
0.24
1.04
13
74
95
-0.27
0.28
.84
14
77
95
-0.51
0.29
.80
15
66
95
0.28
0.25
.92
16
42
95
1.6
0.23
1.19
17
80
95
-0.78
0.31
.75
18
67
95
0.21
0.25
.91
19
76
95
-0.43
0.28
.99
20
60
95
0.64
0.24
.99
34
Distractor Analysis
Entry #
1
2
Data Code
Score
Value
Count
%
Average
Measure
S.E. Mean
A
0
2
2
-0.77
.27
C
0
5
5
-0.37
.26
B
1
88
93
1.50
.13
Missing
**
C
0
4
4
-0.39
.32
B
0
4
4
-0.27
.51
A
1
86
91
1.53
.13
Missing
**
1
1
0.24
35
Getting Started
Menu
PM or BYOA
36
More on Getting Started
Grade Group to CBM
–
Difficulty
–
Measure
37
Fluency and Comprehension
38
Administering a Measure
39
Reporting Outcomes
40
Reporting Outcomes - Diagnostic
41
Instructional Records
42
Reporting Outcomes - Formative
43
Alternate Assessments based on
Modified Academic Achievement Standards
Standards-Based IEPs
Gerald Tindal
University of Oregon
44
Menu and Options
45
Overview
46
Flow Chart
IEP
Goals
Assessment
Participation
Materials
Perception
Skills – CBM
IEP
Components
Goals
Data and
Reporting
Glossary
47
Participation Options
48
Perceptions
49
Curriculum-Based Measures
50
Curriculum-Based Measures
51
IEP Goal Wizard
52
IEP Goal Wizard
53
IEP Goal
In Math, the content of the IEP goal in Data Analysis and
Probability is to propose and justify conclusions and
predictions that are based on data and design studies to
further investigate the conclusions or predictions.
Student will classify numbers, shapes, or objects, 10
tasks, with the competency level of 90%. The goal should be
fulfilled by 2008-05-23.
The following contingencies apply: Read aloud problems
54
IEP Goal Analysis
55
CBM Sophistication







National Center on Student Progress Monitoring
(studentprogress.org)
Intervention Central (interventioncentral.com)
Research Institute on Progress Monitoring
(progressmonitoring.net)
National Research Center on Learning Disablities
(nrcld.org)
National Center on Response to Intervention
(rti4success.org)
Microsoft Excel
Graph paper and pencil
56
57
58
59
60
61
Data Collection and Charting
Student Improvement is Job #1 Goal Area
Reading
East Elementary
District South Iowa
Jacob
Year 07
School
Teacher
Name
By
June
2008,
given
passages
from
the
fifth
grade
curriculum
Jacob
will
read
70
words
correct
in
one
minute.
Goal
Expected Level of Performance25 Words Correct per Minute #1
#2
#3
#4
Service Providers
Parent Participation Parent will provide extra oral reading time at home. They would like graph sent home biweekly.
Baseline
100
Ruso
1
90
80
70
60
Goal
50
40
30
20
10
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
10
8 4
13
12
11
11
12
13
62
Data Collection and Charting
Student Improvement is Job #1 Goal Area
Reading
East Elementary
District South Iowa
Jacob
Year 07
School
Teacher
Name
By
January,
given
passages
from
the
fifth
grade
curriculum
Jacob
will
read
70
words
correct
in
one
minute.
Goal
Expected Level of Performance 25 Words Correct per Minute#1
#2
#3
#4
Ruso
Service Providers
Parent Participation Parent will provide extra oral reading time at home. They would like graph sent home biweekly.
Baseline
100
1
2
90
80
70
60
Goal
50
40
30
20
10
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
10
8 4
13
12
11
15 19
11
12
13
21 25
20 23
19
63
Data Collection and Charting
Student Improvement is Job #1 Goal Area
Reading
District
Jacob
Year 07
School
Teacher Ruso
Name
Goal By January, given passages from the fifth grade curriculum Jacob will read 70 words correct in one minute.
Expected Level of Performance25 Words Correct per Minute #1
#2
#3
#4
Service Providers
Parent Participation Parent will provide extra oral reading time at home. They would like graph sent home biweekly.
Baseline
1
100
3
2
90
80
70
60
Goal
50
40
30
20
10
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
10
8 4
13
12
11
15 19
11
12
13
21 25
20 23
35
16
19
18
35
44
38
64
Instructional Decision Making
Instructional
Intervention
Plan
Decision Making Plan:
Jacob
Student
Intervention Designer
Phase
1
2
3
Instructional Procedure
Vocabulary Preteaching
Instruction provided by general and
sp ed teacher. Phonemic awareness
training. Begin rereadings of
passages.
Same instructional procedures
Add oral reading time each day
Data will be collected at least once per week and charted. If three consecutive
data points fall below the goal line the problem solving team will reconvene and
an instructional change will be made.
Reading
Goal Area
Advisor
Tammy Tyler
Materials
Arrangements
Grade level vocabularyDuring small group reading in
the classroom. Time added to
Jacob’s group each day for this
instruction
M.Ikeda/G. Tindal
Time
15 minutes
Daily
Special Ed teacher will coteach
Word Walk curriculum
30 minutes
Small groups will rotate between
5.0 reading passages
Daily
teachers increasing teacher
contact time.
Same
Tradebooks
At the end of each day, Jacob
will read orally to resource
teacher
15 minutes
Daily
Motivational Strategies
Verbal Praise
Verbal Praise
Classroom motivators
Verbal Praise
Classroom motivators
65
IEP Goals: example 2
Juarez attends a 5-6-7 building. He has a natural
transition between Grades 7 and 8.
Juarez is currently reading 80 words per minute in
Grade 5 material. In order to increase the likelihood
that he scores proficient on the State test, Juarez
needs to increase his proficiency to 120 words per
minute in Grade level material. By June 2010, given
passages from 7th grade reading curriculum
material, Juarez will read 120 words correct in one
minute with five or fewer errors.
66
Data Collection and Charting
Student Improvement is Job #1 Goal Area
Reading
District
Juarez
Year 07
School
Teacher Ruso
Name
Goal By June 2010, given passages from the seventh grade curriculum Juarez will read 120 words correct in one minute.
Expected Level of Performance70 Words Correct per Minute#1
#2
#3
#4
Service Providers
Parent Participation Parent will provide extra oral reading time at home. They would like graph sent home biweekly.
Year 2
Year 1
Year 3
140
130
120
110
100
90
80
Goal
70
60
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
67
Data Collection and Charting
Student Improvement is Job #1 Goal Area
Reading
District
Juarez
Year 07
School
Teacher Ruso
Name
Goal By June 2010, given passages from the seventh grade curriculum Juarez will read 120 words correct in one minute.
Expected Level of Performance 50 Words Correct per Minute
#1
#2
#3
#4
Service Providers
Parent Participation Parent will provide extra oral reading time at home. They would like graph sent home biweekly.
Year 1
Year 2
Year 3
140
130
120
110
100
90
80
Goal
70
60
MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
68
CBM in a Standards Based IEP
Grade level reference
 Content Standard Aligned
 Performance benchmarks
 Multiple measures
 Sensitive to growth
 Established research base
 High expectations
 Focus on Instruction

69
CBM: Cautions

Requires thoughtful decisions


Present Level
Projected Level
A Metric: may not be diagnostic
 May not align with all relevant standards
 Balanced Instruction-not a “teach to the
test”

70
Iowa’s Accountability System for Students with
Disabilities
Core Content Standards
and Benchmarks
Tests Aligned to
Content Standards
Iowa Tests of Basic
Skills/Educational
Development
Without Accommodations
Fair measurement of
student knowledge
Alternate
Assessment
Alternate Achievement Standards
With Accommodations
71