Formative assessment:
definitions and relationships
Division H Invited Session: Formative Assessment:
International Perspective and Applications
Annual meeting of the American Educational
Research Association, April 2011: New Orleans, LA
Dylan Wiliam
www.dylanwiliam.net
Origins and antecedents
 Feedback (Wiener, 1948)
• Developing range-finders for anti-aircraft guns
• Effective action requires a closed system within which
o Actions taken within the system are evaluated
o Evaluation of the actions leads to modification of future actions
• Two kinds of loops
o Positive (bad: leads to collapse or explosive growth)
o Negative (good: leads to stability)
• “Feedback is information about the gap between the actual
level and the reference level of a system parameter which is
used to alter the gap in some way” (Ramaprasad, 1983 p. 4)
 Feedback and instructional correctives (Bloom)
What’s wrong with the feedback metaphor?
In education
 Feedback is any information
given to the student about
their current performance
 … or at best, information
that compares current
performance with desired
performance
 Much rarer is information
that can be used by learners
to improve
In engineering
 That’s just data
 That’s just a thermostat
 That’s a feedback system
A blossoming of research reviews…








Fuchs & Fuchs (1986)
Natriello (1987)
Crooks (1988)
Bangert-Drowns, et al. (1991)
Dempster (1991, 1992)
Elshout-Mohr (1994)
Kluger & DeNisi (1996)
Black & Wiliam (1998)








Nyquist (2003)
Brookhart (2004)
Allal & Lopez (2005)
Köller (2005)
Brookhart (2007)
Wiliam (2007)
Hattie & Timperley (2007)
Shute (2008)
Effects of feedback
 Kluger & DeNisi (1996) review of3000 research reports
 Excluding those:
•
•
•
•
•
without adequate controls
with poor design
with fewer than 10 participants
where performance was not measured
without details of effect sizes
 left 131 reports, 607 effect sizes, involving 23,363
observations of 12,652 individuals
 On average, feedback increases achievement (d =0.41)
• Effect sizes highly variable (sd>1)
• 38% (50 out of 131) of effect sizes were negative
Effects of formative assessment
Standardized effect size: differences in means, measured
in population standard deviations
Source
Kluger & DeNisi (1996)
Effect size
0.41
Black &Wiliam (1998)
Wiliam et al., (2004)
Hattie & Timperley (2007)
0.4 to 0.7
0.32
0.96
Shute (2008)
0.4 to 0.8
Problems with effect sizes
 Restriction of range
 Sensitivity to instruction
 Ambiguous comparisons
Definitions of formative assessment
We use the general term assessment to refer to all those activities
undertaken by teachers—and by their students in assessing
themselves—that provide information to be used as feedback to modify
teaching and learning activities. Such assessment becomes formative
assessment when the evidence is actually used to adapt the teaching to
meet student needs” (Black & Wiliam, 1998 p. 140)
“the process used by teachers and students to recognise and respond to
student learning in order to enhance that learning, during the learning”
(Cowie & Bell, 1999 p. 32)
“assessment carried out during the instructional process for the
purpose of improving teaching or learning” (Shepard et al., 2005 p. 275)
“Formative assessment refers to frequent, interactive
assessments of students’ progress and understanding to identify
learning needs and adjust teaching appropriately” (Looney,
2005, p. 21)
“A formative assessment is a tool that teachers use to measure
student grasp of specific topics and skills they are teaching. It’s a
‘midstream’ tool to identify specific student misconceptions and
mistakes while the material is being taught” (Kahl, 2005 p. 11)
“Assessment for Learning is the process of seeking and interpreting
evidence for use by learners and their teachers to decide where the
learners are in their learning, where they need to go and how best to
get there” (Broadfoot et al., 2002 pp. 2-3)
Assessment for learning is any assessment for which the first priority in
its design and practice is to serve the purpose of promoting students’
learning. It thus differs from assessment designed primarily to serve the
purposes of accountability, or of ranking, or of certifying competence.
An assessment activity can help learning if it provides information that
teachers and their students can use as feedback in assessing themselves
and one another and in modifying the teaching and learning activities in
which they are engaged. Such assessment becomes “formative
assessment” when the evidence is actually used to adapt the teaching
work to meet learning needs. (Black et al., 2004 p. 10)
Which of these is formative?
A. A science adviser uses test results to plan
professional development workshops for teachers
B. Teachers doing item-by-item analysis of 4th grade
math tests to review their curriculum
C. A school tests students every 10 weeks to predict
which students are “on course” to pass a big test
D. “Three fourths” of the way through a unit test
E. Exit pass question: “What is the difference between
mass and weight?”
F. “Sketch the graph of y equals one over one plus x
squared on your mini-dry-erase boards.”
What does formative assessment form?
Cycle length
Long
Medium
Short
✔
✔
Student involved assessment
✔
✔
Student engagement
✔
✔
Teacher cognition about learning
✔
✔
Curriculum alignment
✔
Monitoring progress
✔
Responsive classroom practice
✔
Formative assessment: a new definition
“An assessment functions formatively to the extent that
evidence about student achievement elicited by the
assessment is interpreted and used to make decisions
about the next steps in instruction that are likely to be
better, or better founded, than the decisions that would
have been taken in the absence of that evidence.”
(Wiliam, 2009)
Formative assessment involves the creation of, and
capitalization upon, moments of contingency in the
regulation of learning processes.
Unpacking formative assessment
 Key processes
• Establishing where the learners are in their learning
• Establishing where they are going
• Working out how to get there
 Participants
• Teachers
• Peers
• Learners
Aspects of formative assessment
Teacher
Peer
Learner
Where the learner
is going
Where the learner is
How to get there
Clarify and share
learning intentions
Engineering effective
discussions, tasks and
activities that elicit
evidence of learning
Providing feedback
that moves learners
forward
Understand and
share learning
intentions
Activating students as learning
resources for one another
Understand
learning intentions
Activating students as owners
of their own learning
Five “key strategies”…
 Clarifying, understanding, and sharing learning intentions
• curriculum philosophy
 Engineering effective classroom discussions, tasks and
activities that elicit evidence of learning
• classroom discourse, interactive whole-class teaching
 Providing feedback that moves learners forward
• feedback
 Activating students as learning resources for one another
• collaborative learning, reciprocal teaching, peer-assessment
 Activating students as owners of their own learning
• metacognition, motivation, interest, attribution, self-assessment
(Wiliam & Thompson, 2007)
Dual-pathway model (Boekaerts, 1993)
“It is assumed that students who are invited to participate in a
learning activity use three sources of information to form a
mental representation of the task-in-context and to appraise it:
(1) current perceptions of the task and the physical, social, and
instructional context within which it is embedded; (2) activated
domain-specific knowledge and (meta)cognitive strategies
related to the task; and (3) motivational beliefs, including
domain-specific capacity, interest and effort beliefs.” (Boekaerts,
2006, p. 349)
Growth and well-being
 Share learning goals with students so that they are able to
monitor their own progress toward them.
 Promote the belief that ability is incremental rather than fixed;
when students think they can’t get smarter, they are likely to
devote their energy to avoiding failure.
 Make it more difficult for students to compare themselves
with others in terms of achievement.
 Provide feedback that contains a recipe for future action
rather than a review of past failures.
 Use every opportunity to transfer executive control of the
learning from the teacher to the students to support their
development as autonomous learners.