What is current research telling
us about assessment and
feedback in higher education?
TEAN
‘there is more leverage to improve
teaching through changing
assessment than there is in changing
anything else’ (Gibbs & Simpson, 2004:22)
Criteria
(e.g. LOs, prof.
standards, grade
descriptors
Holistic v
Academic
engagement
transparency
& assessment
discourse
marking
Tension
with
standards
Need for
dialogue
analytical
moderation
Assessment of
professional
competency
Studenttutor power
relationship
Innovative
assessment
methods
Skills of
judgement
formative
assessment
Changing
views of
feedback
Learning
–oriented
assessme
nt
Peer & self
assessment
Computer
assisted
assessment
Professional judgement comes from:
‘repeated engagement in the appropriate activities in the company of those
with expertise’ (Arnal & Burwood (date): 388)
This is a ‘community of practice’ approach
to learning academic standards.
We learn informally, through participation;
through being partners in the assessment
processes, not through being instructed.
(O’Donovan & Rust, 2008).
Approaches to developing student understanding of assessment criteria
Active student engagement
Active methods used to
communicate tacit
knowledge, e.g. marking
exemplars using criteria
2. Explicit model
Standards explicitly
articulated and
passively presented
to students
4. The ‘cultivated’ community
of practice approach
Tacit standards
communicated through
participation in informal
knowledge exchange
‘seeded’ by activities
1. The traditional
model
Tacit standards
absorbed informally
and serendipitously
Adapted from O’Donovan et al
(2008)
Passive Student engagement
informal activities and inputs
Formal activities and inputs
3. The Social Constructivist
Approach
Criteria
(e.g. LOs, prof.
standards, grade
descriptors
Holistic v
Academic
engagement
transparency
& assessment
discourse
marking
Tension
with
standards
Need for
dialogue
analytical
moderation
Assessment of
professional
competency
Studenttutor power
relationship
Innovative
assessment
methods
Skills of
judgement
formative
assessment
Changing
views of
feedback
Learning
–oriented
assessme
nt
Peer & self
assessment
Computer
assisted
assessment
Transparent standards?
• The drive for transparency is based on the
theoretical idea that if students know what
standards they are being judged against,
they are more able to fill the gap between
their present performance and those
standards (Sadler 1989), and marking will
be fairer and more reliable.
Programme assessment
environment
•
•
•
•
•
•
•
•
•
Factors
How many exams
Variety of ass. methods
How much summative
assessment
How much formative
assessment
How much oral feedback
How much written feedback
Timeliness of feedback
Explicit criteria & standards
Alignment between outcomes
& assessment
•
•
•
•
•
•
•
From Gibbs & Dunbat-Goddet (2007)
Can influence
Student effort
How much of the syllabus they
cover
Usefulness of feedback
Use of feedback by students
Whether students know what is
expected of them
Whether they focus on deep or
surface approaches to learning
Whether exams encourage
learning
Feature of assessment
environment
Scales from AEQ (v3.3) with
significantly lower scores
High variety of assessment
methods
Quantity and quality of feedback;
Use of feedback;
Appropriate assessment;
Clear goals and standards;
Deep approach to learning;
Learning from the examination
Satisfaction
High degree of explicitness of
goals and standards
Coverage of the syllabus;
Quantity and quality of feedback;
Use of feedback;
Appropriate assessment;
Deep approach to learning;
Learning from the examination
High degree of alignment of
goals and standards
Coverage of the syllabus;
Use of feedback;
Appropriate assessment;
Clear goals and standards;
Deep approach to learning;
Learning from the examination
Feature of assessment
environment
Scales from AEQ (v3.3)
with significantly
higher scores
High volume of formative Coverage of syllabus;
quantity
and
quality
only assessment
of
feedback;
use of feedback;
learning from examination;
appropriate assessment;
clear goals and standards;
deep approach to learning;
satisfaction.
Learning what is required
• Repeated cycles of
formative assessment
allow students to
gradually become part
of a subject community
of practice.
This cannot be short
circuited by trying to write
down the criteria; it is tacit
knowledge.
“Students…narrowed their attention and
their effort to those things they were told
would be assessed, put in less effort,
covered less of the syllabus, adopted less
of a deep approach and gained less
satisfaction from their studies.’
From Gibbs & Dunbar-Goddet (2007) p 24
Characteristics that appear to
support success and retention
• Academic engagement
• Social engagement
• Active learning
Crosling, Thomas & Heagney
(2007)
Improving Student Retention in
Higher Education: The Role of
Teaching and Learning
Spoon feeding
• Too much guidance prevents students
engaging properly with the task, with
‘doing’ the subject
• Instead they seek to give us exactly what
we have set out
• Miller (2007) talks about ‘painting by
numbers’ but not ever really getting the
whole picture of what assignments are
about.
Dialogue
• Research is increasingly stressing the
importance of student/ tutor dialogue in
order to help them understand both
feedback and guidance.
Challenges in marking research
• Disjunction between espoused and actual
marking (Orrell) - criteria featured in neither
much (teacher educators may be an exception);
• Tutors use internalised, holistic judgement
• Staff work backwards from holistic judgement to
the published criteria (Grainger et al, Tan &
Prosser)
• Many studies show variations between tutors in
marking and interpretation of criteria in practice
• Impact of power relations and negotiation in
marking (Orr)
Issues in Assessing Professional
Competence (Tang 2008)
• consistency between different assessors,
• the dual (conflicting?) role of facilitating student learning
and judging whether the standards have been achieved
• judgements made less by use of the published standards
but more holistically drawing on the tacit knowledge and
expertise of the assessor.
• concern about how much assessment takes into account
different contexts.
• Tang argues that a mechanistic approach to standards
may lead to a focus on discrete teaching behaviours
rather than a broader assessment of good teaching.
Discussion
• To what extent do the findings explored so
far match your experience of assessment
in teacher education and higher education
in general?
Criteria
(e.g. LOs, prof.
standards, grade
descriptors
Holistic v
Academic
engagement
transparency
& assessment
discourse
marking
Tension
with
standards
Need for
dialogue
analytical
moderation
Assessment of
professional
competency
Studenttutor power
relationship
Innovative
assessment
methods
Skills of
judgement
formative
assessment
Changing
views of
feedback
Learning
–oriented
assessme
nt
Peer & self
assessment
Computer
assisted
assessment
Don’t take a deficit approach
• It is easy to conflate the idea of
‘ability’….with [a]..lack of awareness and
experience of the values, assumptions and
practices of higher education.’
• ‘problems in decoding and responding to
expectations appear to be particularly
acute in relation to assessment criteria
(Haggis 526 & 528)
Criteria
(e.g. LOs, prof.
standards, grade
descriptors
Holistic v
Academic
engagement
transparency
& assessment
discourse
marking
Tension
with
standards
Need for
dialogue
analytical
moderation
Assessment of
professional
competency
Studenttutor power
relationship
Innovative
assessment
methods
Skills of
judgement
formative
assessment
Changing
views of
feedback
Learning
–oriented
assessme
nt
Peer & self
assessment
Computer
assisted
assessment
• Assessment of learning
• Assessment for learning
• Assessment as learning
Characteristics of assessment which
promote learning and employability
•
•
•
•
•
•
•
•
•
•
Formative, involving dialogue
Demands higher order learning
Learning and assessment are integrated
Students are involved in assessment
It promotes thinking about the learning process;
Assessment expectations should be made clear;
Involves active engagement of students, developing
independent learning;
Tasks should be authentic and involve choice;
Tasks align with important learning outcomes
Assessment should be used to evaluate teaching.
Formative assessment
Consistent evidence
shows formative
assessment has
particular benefits for
low attainers and
disadvantaged
learners
Black & Wiliam 1998
Feedback Agenda for Change
• It needs to be acknowledged that high level and
complex learning is best developed when
feedback is seen as a relational process that
takes place over time, is dialogic, and is integral
to learning and teaching. Feedback must be
seen as a process not a product.
• There needs to be recognition that valuable and
effective feedback can come from varied
sources, but if students do not learn to evaluate
their own work they will remain completely
dependent upon others. The abilities to self and
peer-review are essential graduate attributes. 27
Studies suggest that
useful feedback to
students
• comes at the draft
stage;
• Is part of the teaching
& learning;
• is forward looking;
• Involves dialogue;
• focuses on skills
rather than content.
• Is relational
But written feedback
often
• Arrives after the work
is finished;
• Is not integrated into
the teaching;
• Doesn’t involve any
dialogue;
• Focuses on the work
completed;
• Is anonymous
What does timely feedback really mean?
The problem with written feedback is that
‘monologue’ is trying to do the work of ‘dialogue’
Writing more and better feedback on assignments
won’t improve matters because it cannot provide
the interaction necessary to help students clarify
their understanding.
(Nicol 2008)
Feedback from different sources
collaborative tasks
team assessment
peer assessment
work-based mentors
studio feedback, etc
These allow students to check out their
understanding in relative safety and see
how others go about things.
30
In-class activity
Out of class activity
Example of module-level approaches:
the use of exemplars annotated with feedback to
encourage dialogue about assessment criteria
Students
write and
submit
individual
assignments
Tutor leads
discussion of
exemplars
previously
marked and
annotated
with
feedback
Module timeline
From Oxford Brookes
FDTL project on
feedback
Tutor
assesses
assignments and
prepares
feedback
Tutor hands
back
assignments
and leads
discussion on
feedback
Submission point
31
From Oxford Brookes
FDTL project on
feedback
1.
Students
draft &
submit
individual
assignments
In-class activity
Out of class activity
Example of module-level approaches:
Generic (non-personalised) feedback on drafts
plus reflective commentary
2. Tutor marks
sample of
assignments
and prepares
generic
feedback
4. Students rewrite
and submit
assignments with
reflective
commentary on
how they have
incorporated the
generic feedback
5. Tutor
assesses
assignments
6. Tutor
hands back
assignments
with minimal
formative
feedback
3. In-class
discussion of
generic
feedback
Module timeline
Submission point
32
Conclusions
• How we help students to become part of our academic
communities, really understanding what is expected of
them, and thus able to be in control of their achievement;
• How we integrate feedback more thoroughly into our
teaching and make it part of a dialogue with students;
• How we broaden our diet of assessment methods in
order to validly assess the broad range of learning
outcomes that characterise modern higher education
without confusing students about expectations;
• How we tackle the reliability issues of assessing
students holistically using tacit knowledge
• How we help students understand assessment and
become partners in the process
• Any questions?