Assessing Student Growth to
Foster School Excellence
Coalition for Psychology in Schools and
Education
Education Leadership Conference
American Psychological Association
September 18, 2006
Overview
• Goal: To understand current uses of assessment in
NCLB and to clarify consensus on how APA might
influence NCLB reauthorization.
• Presentations
– Status, improvement, and growth models of school
accountability (Jeff Braden)
– Educational evaluation of English Learners: Issues and
answers for measuring progress fairly (Sam Ortiz)
– What is school excellence? Is it the same for everyone? (Gary
Stoner)
• Discussion: How should we work to maintain/change
NCLB? (Steve Rollin)
Status, Improvement, and
Growth Models of School
Accountability
Jeffery P. Braden, PhD
Committee on Psychological Tests &
Assessments
North Carolina State University
Overview
• Identify adequate yearly progress (AYP)
definitions under No Child Left Behind
• AYP models
– Status
– Improvement (aka “Safe Harbor”)
– Growth
• Promises and Problems with Growth Models
No Child Left Behind:
Adequate Yearly Progress
• AYP currently defined by three types of targets
– Participation in annual state tests
– Performance on state tests (% proficient)
– One non-test indicator (attendance, graduation rate)
• NCLB mandates testing for…
– Reading & Mathematics (and in 2007, Science)
– Grades 3-8, once in HS (but different for Science)
– Specific populations w/in schools
AYP Targets Defined by…
• Test outcomes = 4 dimensions
– Participation (> 95%) & Performance (% proficient)
– For two subject areas (Reading & Math)
– 2 subject areas X 2 types of goals = 4 dimensions
• Student groups = 9
– All students
– Ethnicity (American Indian/Alaska Native, Asian/PI,
black, Hispanic, white)
– Economically disadvantaged
– Limited English Proficient
– Students with Disabilities
AYP Targets For A School
• Pool data from all eligible grades
• Determine if minimum sample size is reached
for the 9 target groups
– If yes, target applies
– If no, target does not apply and data “roll up” to
district, state
• Compare test data to participation &
performance goals for all eligible groups
• Failure to miss any target for any group = failure
to meet AYP for school
AYP is determined by making
it over all 18 hurdles (9 hurdles
for reading and 9 for math) by
disaggregation of data.
Reading
Composite American
Indian
Asian
Black
White
Hispanic Students with Low
Disabilities Income
LEP
Composite American
Indian
Asian
Black
White
Hispanic Students with Low
Disabilities Income
LEP
Math
AYP = Status Model
Goldschmidt et al. (2005). Policymaker’s guide to growth models for school accountability: How do
accountability models differ? Washington, DC: Council of Chief State School Officers.
AYP Safe Harbor: Improvement
Model
Goldschmidt et al. (2005). Policymaker’s guide to growth models for school accountability: How do
accountability models differ? Washington, DC: Council of Chief State School Officers.
Comparison of Models
• Promise of status model
– Easy to understand
– Same goals for all (equity)
– Leads to 100% proficiency by 2014
• Problems of status model
– Fails to accommodate differences in student
populations between schools
– Fails to identify/reward progress toward proficiency
– Increasingly difficult targets yield high rates of
school failure
Growth Model
Goldschmidt et al. (2005). Policymaker’s guide to growth models for school accountability: How do
accountability models differ? Washington, DC: Council of Chief State School Officers.
Comparing Models:
A Hypothetical Example
• North Carolina AYP Performance Targets
– 76.7% annual target for Reading
– 95% participation goals
• Failing AYP
– Not meeting AYP in three consecutive years
– No meaningful improvement; fails to meet safe
harbor provisions
– Yet strong growth for students entering in 3rd grade
over three consecutive years
100
90
80
70
60
50
40
30
20
10
0
State Proficiency Level for 5th Grade = 247
67
66
M = 251
66
M = 216
2004
2005
AYP
Actual
2006
Additional Issues for Growth Models
• Value added
– Implies causality by attempting to factor out other
influences (e.g., demographics, typical school gains)
• What constitutes appropriate growth?
– Progress toward proficiency standard
– Improvement generally
• Can growth be measured longitudinally?
– Qualitative vs. quantitative change across academic
domains, scaling issues
Using Growth Models for AYP
• Promises
– Accommodates differences between schools in
student populations
– Could reward progress
– Potential linkage to progress monitoring
• Problems
–
–
–
–
Capacity to measure & predict change over time
Mobility
Equity
Practicality
• Educational evaluation of English Learners:
Issues and answers for measuring progress fairly
(Sam Ortiz)
• What is school excellence? Is it the same for
everyone? (Gary Stoner)
Discussion
Proposed Changes to NCLB
• Drop 100% proficiency goal by 2014
– Replace with “goals that are ambitious, scientifically
achievable, and increase regularly…”
• Drop safe harbor (improvement) provisions
– Replace with “growth concurrent with achieving
proficiency on State annual assessments or on
scientifically based measures for student progress
monitoring (as defined by the National Center on
Student Progress Monitoring)”
Proposed Changes to NCLB
• Require states to align proficiency definitions to
NAEP (prevents low/varying standards)
• Increase test inclusion requirement for students in
US schools from three to six years.
• Alternate assessments
– Change “significant [cognitive] disability”
– Increase cap counted toward proficiency from 1% to
2%
• Modified achievement standards cap from 2% to
3%