NIACE Annual Disability
Conference
London: 29 September 2008
3rd Tomlinson Memorial Lecture:
Assessment for learning: why it
matters for all students
Dylan Wiliam
Institute of Education, University of London
www.dylanwiliam.net
Raising achievement matters…
For individuals
Improved control over one’s life
Increased lifetime salary
Improved health
Longer life
For society
Improved ‘pro-social’ behaviour (e.g., participation in democracy)
Lower criminal justice costs
Lower health-care costs
Increased economic growth
Which of the following categories of skill is disappearing from the workplace most rapidly?
1. Routine manual
2. Non-routine manual
3. Routine cognitive
4. Complex communication
5. Expert thinking/problem-solving
…but what is learned matters too…
Autor, Levy & Murnane, 2003
…more now than ever
$35.00
$30.00
$25.00
Dropout
$20.00
HS Diploma
Some College
BA/BSc
$15.00
Prof Degree
$10.00
$5.00
05
20
03
20
01
20
99
19
97
19
95
19
93
19
91
19
89
19
87
19
85
19
83
19
81
19
79
19
77
19
75
19
19
73
$0.00
There is only one 21st century skill
So the model that says learn while you’re at school, while you’re young,
the skills that you will apply during your lifetime is no longer tenable. The
skills that you can learn when you’re at school will not be applicable. They
will be obsolete by the time you get into the workplace and need them,
except for one skill. The one really competitive skill is the skill of being
able to learn. It is the skill of being able not to give the right answer to
questions about what you were taught in school, but to make the right
response to situations that are outside the scope of what you were taught
in school. We need to produce people who know how to act when they’re
faced with situations for which they were not specifically prepared.
(Papert, 1998)
Preparation for future learning (PFL)
Cannot be taught in isolation from other learning
Students still need the basic skills of literacy, numeracy, concepts and
facts
Learning power is developed primarily through pedagogy, not
curriculum
We have to change the way teachers teach, not what they teach
Teachers make a difference
QuickTime™ and a
decompressor
are needed to see this picture.
Students taught by the
best teacher in a group of
50 learn in 6 months what
students taught by the
average teacher take a
year to learn
For students taught by the
least effective teacher in a
group of 50, the same
learning will take two
years
…but more for some than others
Impact of teacher quality on student outcomes (Hamre & Pianta, 2005))
Achievement gaps
Disadvantaged background
(mother’s education)
Poor behavior
Teacher’s provision
of instructional
support
High
No (good)
Average No (good)
Low
Yes (bad)
High
Teacher’s provision
of emotional support
High
Yes (bad)
Average Yes (bad)
Low
Yes (bad)
High
Yes (bad)
Average Yes (bad)
Low
Yes (bad)
No (good)
Average Yes (bad)
Low
Yes (bad)
The ‘dark matter’ of teacher quality
Teachers make a difference
But what makes the difference in teachers?
Advanced content matter knowledge
Pedagogical content knowledge
Further professional qualifications (MA, NBPTS)
Total “explained” difference
<5%
10-15%
<5%
20-25%
Learning power environments
Key concept:
Teachers do not create learning
Learners create learning
Teaching is the engineering of effective learning environments
Key features of learning power environments:
Create student engagement (pedagogies of engagement)
Well-regulated (pedagogies of contingency)
Why pedagogies of engagement?
Intelligence is partly inherited
So what?
Intelligence is partly environmental
Environment creates intelligence
Intelligence creates environment
Learning environments
High cognitive demand
Inclusive
Obligatory
Motivation: cause or effect?
high
arousal
Flow
anxiety
challenge
control
worry
relaxation
apathy
boredom
low
low
competence
high
(Csikszentmihalyi, 1990)
How do students make sense of this?
Attribution (Dweck, 2000)
Personalization (internal v external)
Permanence (stable v unstable)
Essential that students attribute both failures and success to internal,
unstable causes. (It’s down to you, and you can do something about it.)
Views of ‘ability’
Fixed (IQ)
Incremental (untapped potential)
Essential that teachers inculcate in their students a view that ‘ability’ is
incremental rather than fixed
(by working, you’re getting smarter)
Prediction is hard, especially about the future…
Source: Autumn package (2001), DfES
Why pedagogies of contingency?
Several major reviews of the research…
Natriello (1987)
Crooks (1988)
Kluger & DeNisi (1996)
Black & Wiliam (1998)
Nyquist (2003)
… all find consistent, substantial effects
The AfL hi-jack continues…
Long-cycle
Span: across units, terms
Length: four weeks to one year
Medium-cycle
Span: within and between teaching units
Length: one to four weeks
Short-cycle
Span: within and between lessons
Length:
 day-by-day: 24 to 48 hours
 minute-by-minute: 5 seconds to 2 hours
Unpacking formative assessment
Key processes
Establishing where the learners are in their learning
Establishing where they are going
Working out how to get there
Participants
Teachers
Peers
Learners
Aspects of formative assessment
Where the learner
is going
Teacher
Peer
Learner
Where the learner is
Engineering effective
Clarify and share discussions, tasks and
activities that elicit
learning intentions
evidence of learning
How to get there
Providing feedback
that moves learners
forward
Understand and
share learning
intentions
Activating students as learning
resources for one another
Understand
learning intentions
Activating students as owners
of their own learning
Five “key strategies”…
Clarifying, understanding, and sharing learning intentions
curriculum philosophy
Engineering effective classroom discussions, tasks and activities that
elicit evidence of learning
classroom discourse, interactive whole-class teaching
Providing feedback that moves learners forward
 feedback
Activating students as learning resources for one another
 collaborative learning, reciprocal teaching, peer-assessment
Activating students as owners of their own learning
metacognition, motivation, interest, attribution, self-assessment
(Wiliam & Thompson, 2007)
…and one big idea
Use evidence about learning to adapt teaching and learning to meet
student needs
Practical techniques: eliciting
evidence
Key idea: questioning should
cause thinking
provide data that informs teaching
Getting away from I-R-E
basketball rather than serial table-tennis
‘No hands up’ (except to ask a question)
class polls to review current attitudes towards an issue
‘Hot Seat’ questioning
All-student response systems
ABCD cards, Mini white-boards, Exit passes
Practical techniques: feedback
Key idea: feedback should
 cause thinking
 provide guidance on how to improve
Comment-only marking
Focused marking
Explicit reference to scoring guides and mark schemes
Suggestions on how to improve
 Not giving complete solutions
Re-timing assessment
 (eg three-quarters-of-the-way-through-a-unit test)
Practical techniques: sharing learning
intentions
Explaining learning intentions at start of lesson/unit
Learning intentions
Success criteria
Intentions/criteria in students’ language
Posters of key words to talk about learning
e.g., describe, explain, evaluate
Planning/writing frames
Annotated examples of different standards to ‘flesh out’ mark schemes
(e.g. lab reports)
Opportunities for students to design their own mark-schemes and tests
Practical techniques: activating
students
Students assessing their own/peers’ work
with scoring guides
with exemplars
“two stars and a wish”
Training students to pose questions/identifying group weaknesses
Self-assessment of understanding
Traffic lights
Red/green discs
End-of-lesson students’ review
Putting it into practice
Implementing AfL requires changing
teacher habits
Teachers “know” most of this already
So the problem is not a lack of knowledge
It’s a lack of understanding what it means to do AfL
That’s why telling teachers what to do doesn’t work
Experience alone is not enough—if it were, then the most experienced
teachers would be the best teachers—we know that’s not true
(Hanushek, 2005; Day, 2006)
People need to reflect on their experiences in systematic ways that build
their accessible knowledge base, learn from mistakes, etc. (Bransford,
Brown & Cocking, 1999)
Teacher learning takes time
To put new knowledge to work, to make it meaningful and accessible
when you need it, requires practice.
A teacher doesn’t come at this as a blank slate.
Not only do teachers have their current habits and ways of teaching—
they’ve lived inside the old culture of classrooms all their lives: every
teacher started out as a student!
New knowledge doesn’t just have to get learned and practiced, it has to go
up against long-established, familiar, comfortable ways of doing things that
may not be as effective, but fit within everyone’s expectations of how a
classroom should work.
It takes time and practice to undo old habits and become graceful at
new ones. Thus…
 Professional development must be sustained over time
A model for teacher learning
Content, then process
Content (what we want teachers to change)
Evidence
Ideas (strategies and techniques)
Process (how to go about change)
Choice
Flexibility
Small steps
Accountability
Support
Design and intervention
Our design process
cognitive/affective
insights
synergy/
comprehensiveness
set of
components
Teachers’ implementation process
set of
components
synergy/
comprehensiveness
cognitive/affective
insights
Summary
Learning power is developed more by how—than by what—we teach
Teaching is the engineering of effective learning environments
Effective learning environments involve
Pedagogies of engagement
Pedagogies of contingency
Personalisation
Mass customization (rather than mass production or individualisation)
Diversity
A valuable teaching resource (rather than a challenge to be minimized)
Assessment is the bridge between teaching and learning, and thus the
central process of teaching (as opposed to lecturing).