Research on formative assessment
practices
2013 AERA Classroom Assessment SIG Symposium:
Advancing Research in Classroom Assessment
Dylan Wiliam
Section 3: Formative assessment
2

Four chapters
 Formative
and summative aspects of assessment:
Theoretical and research foundations in the context of
pedagogy (Black)
 Gathering evidence of student understanding
(Heritage)
 Feedback and instructional correctives (Wiliam)
 Examining formative feedback in the classroom
context: New research perspectives (Ruiz-Primo & Li)
Formative and summative aspects
of assessment: Theoretical and
research foundations in the context
of pedagogy
Paul Black
Formative and summative assessment
4

A simple model of
instruction

aims
 Planning activities
 Interaction
 Review of the learning
 Summing up
Regulation of learning
processes
 Clear
 Proactive
 Interactive
 Retroactive
Tensions arise between summative and
formative functions, especially in terms
of instruments, methods, and agents
The role of teachers
5


Teachers’ exclusion from summative functions of
assessment weakens the kinds of inferences that
can be made
But, inclusion of teachers brings its own issues
 Poor
construct definition makes assessment design
more of a discretionary process than it should be
 Teachers model their assessments on tests
 “Summative drives out formative”
 Alignment of curriculum, assessment and instruction
 Case
studies (Australia, England, Scotland, Sweden)
Gathering evidence of student
understanding
Margaret Heritage
Eliciting evidence
7

Two fundamental purposes of assessment
 provide
information on current achievement
 inform future instruction
Sources and quality of evidence
8

Sources of evidence
 What
learners say, write, make, or do (Griffin 2007)
 Interactions (questions, discussions)
 Tasks
 Directed
(model-eliciting activities)
 Partially directed (activities)
 Undirected (observations)

Quality of evidence
 Construct
underrepresentation
 Construct-irrelevant variance
 systematic
 random
(i.e., unreliability)
Learning progressions
9


“What gets better when someone gets better”
Two key aspects
1. progressions lay out in successive steps, increasingly
more sophisticated understandings of core concepts
and principles in a domain, and
2. progressions describe typical development over an
extended period of time.

Two approaches to their development
 Top-down
(theoretical)
 Bottom-up (empirical)
Feedback and instructional
correctives
Dylan Wiliam
Reviews of research on feedback
11

Fuchs & Fuchs (1986)

Nyquist (2003)

Natriello (1987)

Brookhart (2004)

Crooks (1988)

Allal & Lopez (2005)

Bangert-Drowns et al. (1991)

Köller (2005)

Dempster (1991, 1992)

Brookhart (2007)

Elshout-Mohr (1994)

Wiliam (2007)

Kluger & DeNisi (1996)

Hattie & Timperley (2007)

Black & Wiliam (1998)

Shute (2008)
Feedback: an evolving concept (Brookhart, 2007)
12
Conceptualization




Information about the
learning process…
… that teachers can use
for instructional
decisions…
…and students can use to
improve performance…
…which motivates
students
Source(s)

Scriven (1967)

Bloom, Hastings and
Madaus (1971)

Sadler (1983; 1989)

Natriello (1987); Crooks
(1988); Black and Wiliam
(1998); Brookhart (1997)
Back to the future
13
“These considerations of utility and alternative interventions
suggest that even [a feedback intervention (FI)] with
demonstrated positive effects on performance should not be
administered whenever possible. Rather, additional
development of [feedback intervention theory] is needed to
establish the circumstance under which positive FI effects on
performance are also lasting and efficient and when these
effects are transient and have questionable utility. This research
must focus on the processes induced by FIs and not on the
general question of whether FIs improve performance—look at
how little progress 90 years of attempts to answer the latter
question have yielded.” (Kluger & DeNisi, 1996 p. 278)
Dual pathway theory (Boekaerts)
14
“It is assumed that students who are invited to participate in a
learning activity use three sources of information to form a
mental representation of the task-in-context and to appraise it:
(1) current perceptions of the task and the physical, social, and
instructional context within which it is embedded;
(2) activated domain-specific knowledge and (meta)cognitive
strategies related to the task; and
(3) motivational beliefs, including domain-specific capacity,
interest and effort beliefs.” (Boekaerts, 2006 p. 349)
15

As a result of the appraisal, the student activates
energy and attention along one of two pathways
the growth pathway (increasing competence)
 the well- being pathway (prevent harm, threat or loss)


Integration of other theories
 Mindset
(Dweck, 2000)
 Mastery and performance goals (Dweck, 2000)
 Interest (Hidi & Harackiewicz, 2000)
 Self-regulated learning (Deci & Ryan, 1994)
Examining formative feedback
in the classroom context:
New research perspectives
Maria Araceli Ruiz-Primo and Min Li
Formative assessment in the classroom
17

Updated review of research on feedback in
classroom learning
 Vast
 No
majority of studies are poorly designed
control groups
 Feedback effects evaluated in the instructional session
 No evidence of pedagogical orientation (e.g., orientations
to the purpose of feedback, or how it could be used)
A new, expanded, notion of feedback
18

Feedback should:
Be seen as a process guided by the learning goals towards
which the teacher and students work
 Actively involve students in the process
 Be considered as an instructional scaffold that goes beyond
written or oral comments.
 Be specifically intended to improve learning outcomes
 Ensure its usefulness by making feedback accessible and
practical.
 Consider different sources of information
 Demonstrate, over time, alignment with a learning
trajectory

Future priorities
19




Defining feedback
Understanding variability in feedback practices
Understanding feedback in use
Reviewing feedback research from new theoretical
understandings
Major unresolved questions
20



Can teachers be involved in the summative
assessment of their students without
compromising the potential of formative
assessment to improve achievement?
How can learning progressions be developed?
How can we better theorize the impact of
feedback interventions?