PAY ATTENTION!
There is going to be a test.
Standardized Tests
and Education Reform
Keith Clay, GRCC
Our love of tests, part I
“Are we going to be tested on this?”
An inservice teacher at a biotechnology
workshop at UC San Francisco
Our love of tests: Part II
Check the web.
The WASL is:
“stupid”
“evil”
“AWFL”
“child abuse”
“terrorism”
Tests and Education Reform
Why Standardized Tests?
The Art and Science of Testing
The state of Washington
Do tests help with education reform?
Do education reforms help on the tests?
Why standardized tests?
Some critics argue that “High-Stakes
Testing” is just a bad idea…
…but it’s not…
…It’s the law.
Why standardized tests?
Assessment of curricula

Core Knowledge? FOSS? PBI?
Assessment of systems

Seattle School District? Mrs. Nelson’s class?
Assessment of individuals

Formative? Summative? Instructive?
Assessment of Curricula
Identify
goals
Design
curriculum
Analyze
results
Design
assessment
Administer
assessment
Assessment of systems
Identify substandard schools
Identify outstanding schools
Then what?


Allow students to leave weak schools?
Financially reward low performance?
Assessment of individuals
“Formative” or “Diagnostic”


Identify strengths and weaknesses
Construct strategies for improvement
“Summative” or “Evaluative”


Grades
Graduation Requirement
 10th Grade WASL in 2008 (5th graders now)
Summative Eval Question:
“Should a PhD historian be able to pass a state
physics test without ever taking a physics
class?”
Bernie Khoury, AAPT
Should a “state physics test” be physics
content free?
Should a PhD historian be allowed to
graduate from high school?
Assessment as Teaching Tool
“Instructive” or “reflective” assessment
Students study the results of their own
assessment throughout learning
White and Frederiksen, 2000


“Inquiry + reflective assessment” produced
better results than “Inquiry + discussion”
Differences were greatest for weakest
students and hardest material
Design of assessment tools:
Curriculum assessment probes for
“student averaged” content weaknesses
Systems assessment probes “student
and content averaged” weaknesses
Individual summative tests probe for
“content averaged” weakness
Individual formative tests probe for
specific content and student weakness
High-Stakes Assessment:
NCLB mandated assessments often try to
do all four things at once.
Nat’l Assessment of Educ. Progress
(NAEP) is “high-bandwidth, low-fidelity.”
“Measurement at the level of individual
students is poor.”
NRC, Knowing What Students Know, 2001
The art and science of testing
THE ASSESSMENT TRIANGLE (KWSK, 2001)
Observation
Interpretation
Cognition
model
The art and science of testing
Psychometrics:
For statistical analysis, correct responses
(cr) must outnumber random chance (rp)
Usual requirement: cr > 2 rp



A,B,C,D choice, cr > 50%
A,B,C,D,E choice, cr > 40%
Open ended: Average > 40% to 50%
The art and science of testing
Compromises: Can we get 40% correct?
Q: Throw a ball straight up into the air and it
comes back down to your hand. Define “up” to
be the “positive direction.” When the ball is at
the maximum height, is has an acceleration of:
A) 9.8 m/s2
B) -9.8 m/s2
C) 4.9 m/s2
D) -4.9 m/s2
E) zero
Most students choose E, so psychometricians
suggest removing it from the list of options.
Can we get 40% correct, if…
We frame the same question but ask…
1) …at maximum height, is has a velocity of:
A) 9.8 m/s
B) -9.8 m/s
C) 4.9 m/s
D) -4.9 m/s
E) zero
2) …at max. height, is has an acceleration of:
A) 9.8 m/s2
B) -9.8 m/s2
C) 4.9 m/s2
D) -4.9 m/s2
E) zero
40% of students get #2 right if asked #1 first.
The state of Washington:
EALR, WASL, SCIF, SALT, and OIAs
EALR: Ess. Acad. Learning Requirement
WASL: Wa. Assess’t of Student Learning
SCIF: Science Curr. Instr. Frameworks
(they write the EALRs)
SALT: Sci. Assess’t Leadership Team
(They write the WASL)
OIAs: Other Important Acronyms
The state of Washington:
SCIF and SALT talk to each other
(this is unusual)
Aligned on content and level (Bloom)
From SCIF: EALR and Nat’l Standards
alignment document in progress
From SALT: content specialists may
override psychometricians
The state of Washington:
K8 Science according to SCIF and SALT:
INQUIRY
SYSTEMS:
Ecosystems,
Circuits,
Rock cycle,
Nerves…
30%
Rockets,
Solar system,
Spiders…
40%
30%
DESIGN
Do tests help education reform?
“Whereas teaching directly to the items
on a test is not desirable, teaching to
the theory of cognition and learning
that underlies a test can provide a
positive direction for instruction.”
NRC, Knowing What Students Know, 2001
Do tests help education reform?
“Large-scale standardized assessments
can communicate across time and
place, but by so constraining the
content and timeliness of the message
that they often have limited utility in the
classroom.”
NRC, Knowing What Students Know, 2001
Do tests help education reform?
The Frame and the Tapestry
Thompson and Zeuli, 1999
Michigan Educ. Assessm’t Prog. (MEAP)
MEAP has both goals and assessments
Most local school districts aligned their
curricula with MEAP and NSES
Do tests help education reform?
“…considering how deep-seated most
teachers’ ideas are about subject matter,
teaching, and learning, one would not
expect these… substantially aligned
documents to produce conceptually
transformative teacher learning on a
broad scale. Our classroom-level
research confirms that they have not.”
Thompson and Zeuli, 1999
Does reform help the tests?
Education reform for teachers is a
necessary precondition for success
“It is now widely accepted that in order to
realize recently proposed reforms in what is
taught and how it is taught in math and
science, teachers will have to unlearn much of
what they believe, know, and know how to do,
while also forming new beliefs, developing
new knowledge, and mastering new skills.”
Thompson and Zeuli, 1999
Teaching reform requires…
For teachers:





Cognitive dissonance
Time for reflection
Connection with their classrooms
A repertoire of techniques
Continuing support
For the system:

A scalable, sustainable process
Thompson and Zeuli, 1999
How long will this take?
“This is a generation-long process.”
Pinky Nelson, WWU & Project 2061