Making Numbers Work…
To Improve Instruction
NHSAA: Living with the NCLB Act
Ann Remus
September 21, 2004
Where to Begin?…
one analysis at a time
Examples:
I. Program evaluation
II. Authentic assessment that fosters good
teaching
III. Diagnosis to allow differentiated
instruction/value-added assessment
I. Program Evaluation



Growth by Grade on a Norm-referenced test:
Gates-MacGinite Reading Tests
We used to disaggregate by hand for Title I
and SPED – led to program improvements
Difficulty keeping consistency of early skills
program when grades were split among three
schools in 99-00.
Growth By Grade, Gates-MacGinite
Gates MacGinitie Total Battery Extended Scale Score
By Grade Over Time
1999-2003
535
540
3rd Grade
+4
520
2nd Grade
+9
Total Battery ESS
500
474 477 475
480
533 536 536
505 505
481 484
1st Grade
+16
460
440
420
501 500 501
5th Grade
+1
538
432 432
416
419
424
400
1999 2000 2001 2002 2003 1999 2000 2001 2002 2003 1999 2000 2001 2002 2003 1999 2000 2001 2002 2003
Grade 1
Grade 2
Grade 3
Grade 5
Grade Level Over Time
Note: Black lines indicate national median Total Battery ESS for each grade level.
1st Grade: 391, 2nd Grade: 446, 3rd, Grade: 472, 5th Grade: 507.
Growth By Grade: Observations


District focus/professional development on
early skills may account for growth in grades
1 and 2
The data:


identifies the need to investigate upper level
elementary grades’ potential for increased
growth
Identifies the need to disaggregate data


By cohort
By NCLB categories
Disaggregation:
Stanine Ranges of Cohorts
Gates-MacGinite Average Total Battery ESS Grade 2
Growth for Higher Stanine Range: 10 ESS
540
Growth for Middle Stanine Range: 17 ESS
Total Battery ESS
520
498
500
480
468
464
460
508
507
502
472
455
Middle
Higher
National Median
ESS: 446
440
420
400
Going into 6th
Grade
Going into 5th
Grade
Going into 4th
Grade
Cohort
Going into 3rd
Grade
Growth by Cohort, Disaggregated
by Stanine Ranges : Conclusions

Disaggregating cohorts helps us see which groups
are growing more or less quickly



The middle stanine range improved more than did the
higher stanine range
Progress of high and middle ranges against national
median score has improved across time
Instructional implication:


To “catch up,” the lower and middle groups must continue
to grow more quickly
Need focused attention on best practices for the upper
group as well
Introduction of Open Court
Reading Program



Prior series no longer available; wanted to
make use of the research from the National
Reading Panel in our programming
2002-2003: pilot, with one teacher in each
grade, grades 1-4, in all three schools
2003-2004: first year of full district-wide
implementation, grade 1
Average Total Score
Gates MacGinitie Reading Test
(1999-2004)
+18
460
Extended Scale Score
450
+16
450
440
430
432
2002
2003
424
420
410
432
416
419
400
390
1999
2000
2001
School Year
2004
Disaggregating grade 1 by NCLB
Comparison of Disaggregated Groups
categories
after our first year of
Grade 1 Gates Reading Tests Results
Open Court reading
97
Percentile Equivalent of Mean Score
100
87
90
80
79
78
70
62
59
60
60
52
50 50
2002-2003
2003-2004
50
39
40
29
30
20
10
0
Bedford
National Norm
Average Grade for Grade 1
1
Special
Education
Groups
ESL
Reading
Support
Math Apps
II. Authentic Assessment to
Encourage Good Teaching



In the mid-1990’s, the district looked closely
at its curriculum, deciding on essential
questions and grade level expectations.
That work was continued with detailed
curriculum maps.
One of the outcomes expected of grade 2
science, is understanding mass and volume.
Grade 2 Authentic Assessment
Percent
measuring liquid volume and mass:
following procedures and making
predictions
100
90
80
70
60
50
84
74
Average Score
1999
2004
Year
Source: Market Research
Another Example of
Authentic Assessment
Impact of Process Writing
on Writing Assessment
Leadership Seminar
Presented by
Barbara Gondek
May 4, 2004
Hypothesis

Students assigned to strong writing
instructors in grades 7 and 8 will be shown to
perform better on the Bedford Formal
Writing Assessment than those who had
average or weak writing instructors.
Parameters




Grades Seven and Eight
Two years of scores (2002 and 2003)
Sorted by Language Arts teachers
Teachers were classified on the basis of the
extent to which they used all the steps of
the Writing Process in their classrooms
during 2002 and 2003.
Writing Process Steps
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
Find an idea or topic.
Formulate a thesis - narrow the topic.
Generate details - facts, examples, and support.
Outline if it seems helpful.
Write your first draft.
Let things rest for a day or two.
Revise - look at the paper from the reader's point of view;
reorganize and add explanations where necessary.
Let things rest again.
Revise again and edit for conventions.
Type or write the final draft.
Proofread at least three times - once aloud, once for any
errors you habitually make, and once backwards.
Classification of Teachers
Strong
•Used Process
Writing steps
regularly
•Required
students to
write every day
Average
• Used Process
Writing Steps
occasionally
•Required
students to write
2-3 times per
week
Weak
• Seldom used
Process Writing
•Most writing
was “quick
writes”
Extent of Process Writing vs.
Student Writing Skills
100
80
Percent
60
40
20
0
67
(263)
67
(251)
73
(254)
Strong (392)
Average (373)
Weak (350)
Use of Writing Process
High Basic and Above
Basic and Below
Extent of Process Writing vs. Special
Education Student Writing Skills
100
90
80
70
60
Percent 50
40
30
20
10
0
56
(33)
59
(30)
72
(44)
Strong (59)
Average (51)
Weak (61)
Use of Writing Process
High Basic and Above
Basic and Below
Current Writing Assessment



Writing prompt is given at the
beginning of the class.
Students are given a limited time to
organize ideas, brainstorm, and draft
an essay.
Final copy is completed by the end of
the testing session.
Writing Assessment Team


Representatives from Grades 1 – 8
worked together over the summer
Developed an assessment format and
rubric that lines up with process
writing.
Writing Process Steps
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
Find an idea or topic.
Formulate a thesis - narrow the topic.
Generate details - facts, examples, and support.
Outline if it seems helpful.
Write your first draft.
Let things rest for a day or two.
Revise - look at the paper from the reader's point of view;
reorganize and add explanations where necessary.
Let things rest again.
Revise again and edit for conventions.
Type or write the final draft.
Proofread at least three times - once aloud, once for any
errors you habitually make, and once backwards.
Assessment Plan for Fall 2004




Pre-assessment in September
Post assessment in May-June
Analytical scoring as opposed to
holistic
Assessment given in stages similar to
process writing, over a period of days
Assessment is not the end of
the writing process.
It is the bridge to revision.
III. Diagnostic/Value-Added
Assessment



NWEA - given in grades 3 through 8 each fall
and spring: piloted in 02-03, full
implementation in 03-04.
Fall administration gives information
necessary for immediate differentiation of
instruction
Yearly growth by RIT ranges shows
effectiveness of differentiation
Grade 7 Math
NWEA Fall to Spring Growth
7th Grade NWEA Fall to Spring RIT Growth
Fall to Spring RIT Growth
18.0
16.0
2003-2004
14.0
12.0
10.0
8.0
6.0
4.0
2.0
n= 5
4
23
67
111
67
20
0.0
190-199 200-210 210-219 220-229 230-239 240-249 250-259
RIT Ranges
McKelvie Average Fall to Spring Growth
NWEA Growth Norms
Grade 7 Language Usage
NWEA Fall to Spring Growth
2003-2004
Fall to Spring RIT growth
7
McKelvie
average fall to
spring growth
6
5
NWEA national
norms
4
3
2
1
16
80
143
200-209
210-219
220-229
RIT Ranges
60
0
230-239
Gates-MacGinite total battery ESS
vs. NWEA reading RIT total
Gates v NWEA
280
y = 0.32x + 46.04
2
R = 0.79
260
NWEA (RIT)
240
220
200
180
160
140
120
100
200
300
400
500
Gates (ESS)
600
700
800
What to do?




Collect data, backwards and forwards
Zero in on groups, smaller groups, and
then individuals
Talk about what you see in the data:
different people may see different
things
Keep at it…