Assessment Standards Knowledge exchange
Engaging students with assessment
feedback
Prof. Margaret Price,
Director ASKe Centre for Excellence
FDTL Engaging Students with assessment feedback
https://mw.brookes.ac.uk/display/eswaf/Home
ASKe Assessment Standards Knowledge exchange
Centre for Excellence in Teaching and Learning
http://www.brookes.ac.uk/aske/!
aske@brookes.ac.uk
Purpose of Workshop
• Problems and responses
• Engagement with feedback
• Where to start
• Resources and effectiveness
We have a problem!
•
•
Surveys and audits
Research literature
Feedback problems
•
•
•
•
•
Unhelpful feedback (Maclellan, 2001)
Too vague (Higgins, 2002)
Subject to interpretation (Ridsdale, 2003)
Not understood (e.g. Lea and Street, 1998)
Don’t read it (Hounsell, 1987. Gibbs & Simpson
2002)
• Damage self-efficacy (Wotjas, 1998)
• Has no effect (Fritz et al, 2000)
• Seen to be too subjective (Holmes & Smith,
2003)
Some responses to the feedback ‘crisis’:
•
•
•
•
Provide more of the same
Simplistic rules about timing
Standardisation
Label feedback
• Setting expectations
• Introducing new methods
• a complex problem so no simple solution
Exploring feedback (activity)
• What is its purpose?
• What counts as feedback?
• What can it achieve?
• How do you know it is working?
Student engagement with feedback
Student outcome
influences later
engagement with
feedback
Confidence
Resources; skills
Opportunities for action
Self-efficacy
Specific, immediate
action, such as applying
feedback insights to next
assignment
Collection
… or does
not collect
Immediate
attention
Cognitive
response
.. or cannot read,
so ignores
Immediate or
latent action
... or cannot
understand, so
does not proceed
... or, after reflection,
rejects, or
‘mis’understands, etc.
Price et al (submitted)
OUTCOME
Assessment
feedback
Developmental changes
which generate the latent
potential for future action
Activity
In 3’s, discuss:
• How do you currently prepare students to
understand and engage with feedback?
Where to start
 Preparation and setting expectations early in the
programme
 Identifying ‘feedback moments’ and application
opportunities within the programme
 Emphasize the relational dimension of feedback
 Building in space for dialogue
What can we do? (1)
• Aligning expectations (of staff & students, & between teams of
markers)
• Identify what is feasible in a given assessment context written feedback can often do little more than ‘diagnose’
development issues and then direct students to other
resources for help and support
• Identifying all feedback available
• Ensure it is timely - ‘quick and dirty’ generic feedback,
feedback on a draft, MCQs & quizzes, etc. (using technology
may help)
• Model and encourage the application of feedback
What can we do? (2)
• Require and provide feedback on self-assessment
• Improve the linkage of assessment strategies across
programmes and between modules/units
• Consider the role of marks - they obscure feedback
• Reduce over-emphasis on written feedback - oral can be
more effective (McCune, 2004). Face to face feedback
with 140 students (FDTL Case study:
https://mw.brookes.ac.uk/display/eswaf/Home.
• Review resource allocations
What can we do (3)
• Support the relational dimension of feedback
Students say that relationships in which staff are supportive
and approachable help them to engage
Avoid anonymous marking
Ensure associate (and permanent) staff have sufficient time
and/or space
Provide some continuity of staff contact (personal tutors)
Provide opportunity for dialogue (e.g. discuss feedback in
class, peer review, peer assisted learning)
Peer marking using model answers
(Forbes & Spence, 1991)
Scenario:
• Engineering students had weekly maths problem sheets
marked and problem classes
• Increased student numbers meant marking impossible
and problem classes big enough to hide in
• Students stopped doing problems
• Exam marks declined (Average 55%>45%)
Solution:
• Course requirement to complete 50 problem sheets
• Peer assessed at six lecture sessions but marks do not
count
• Exams and teaching unchanged
Outcome: Exam marks increased (Av. 45%>80%)
Peer feedback - Geography (Rust, 2001)
Scenario
• Geography students did two essays but no apparent improvement
from one to the other despite lots of tutor time writing feedback
• Increased student numbers made tutor workload impossible
Solution:
• Only one essay but first draft required part way through course
• Students read and give each other feedback on their draft essays
• Students rewrite the essay in the light of the feedback
• In addition to the final draft, students also submit a summary of how
the 2nd draft has been altered from the1st in the light of the feedback
Outcome: Much better essays
Peer feedback - Computing (Zeller, 2000*)
The Praktomat system allows students to read, review, and assess each
other’s programs in order to improve quality and style. After a successful
submission, the student can retrieve and review a program of some fellow
student selected by Praktomat. After the review is complete, the student
may obtain reviews and re-submit improved versions of his program. The
reviewing process is independent of grading; the risk of plagiarism is
narrowed by personalized assignments and automatic testing of submitted
programs.
In a survey, more than two thirds of the students affirmed that reading each
other’s programs improved their program quality; this is also confirmed by
statistical data. An evaluation shows that program readability improved
significantly for students that had written or received reviews.
[*Available at:
http://www.infosun.fim.unipassau.de/st/papers/iticse2000/iticse2000.pdf]
Figure 1: Peer-review as a method of encouraging
students to discuss and compare their
understanding of assessment criteria
IN-CLASS
ACTIVITY
OUT OF CLASS
ACTIVITY
Figure 1
2. Students
bring draft
individual
assignments
for peer
review
1. Tutor leads
discussion on
assessment
criteria and
process of peer
review
4. Students
rewrite and
submit
individual
assignments
3. In-class
discussions
between student
groups as they
review each other’s
work, monitored by
tutor.
5. Tutor
marks
assignments
and prepares
feedback
6. Tutor
hands back
assignments
and leads
discussion on
feedback
MODULE TIMELINE
Week1
Week12
Assignment
point
Figure 2: the use of 'exemplars' as a
mechanism for encouraging dialogue
about assessment criteria
IN-CLASS
ACTIVITY
OUT OF CLASS
ACTIVITY
Figure 2
2. Students
write and
submit
individual
assignments
1. Tutor leads
discussion of
previouslymarked
exemplars
annotated with
feedback
3. Tutor
marks
assignments
and prepares
feedback
4. Tutor
hands back
assignments
and leads
discussion on
feedback
MODULE TIMELINE
Week1
Week12
Assignment
point
Figure 3: Generic feedback
and self critique
IN-CLASS
ACTIVITY
OUT OF CLASS
ACTIVITY
Figure 3
1. Students
draft and
submit
individual
assignments
2. Tutor marks
sample of
assignments
and prepares
generic
feedback
4. Students rewrite and
submit individual
assignments with
reflective commentary
on how they have
incorporated the
generic feedback
3. In-class
discussion of
generic cohort
feedback
based on
coursework
sample
5. Tutor
grades
assignments
6. Tutor
hands back
assignments
with grade
only
MODULE TIMELINE
Week1
Week12
Assignment
point
Activity
Individually:
Choose one or more specific ideas to improve
feedback that you think you could use. In as much
detail as possible, identify how you would put the
idea/s into practice.
In pairs:
Take it in turns to explain your plans to your partner.
The job for the listener is to be a friendly and
constructive critic
Feedback moments
• Where there is a clear opportunity to apply feedback
• Pre assessment
• Reflection points
Identify them within each programme
PROGRAMME LEVEL
Regular review meetings
with personal tutor to
discuss feedback
PROGRAMME DURATION
Student preparation – HE
orientation
MODULAR LEVEL
Sem 1: Introduction to selfassessment and peer review.
Discussion of criteria and use of
exemplars
Sem 2: Introduction to group work,
continued development of peer
and self-assessment. Continued
use of exemplars
End Yr1
Student involvement in
peer-assisted learning
(voluntary)
Year 2: Support in the transition to
Stage 2 modules, with more
formative feedback
End Yr2
Year 3: Expectation that students
will engage in more selfassessment, and will demonstrate
ability to critique own work
Student involvement in
mentoring others (voluntary)
End Yr3
Figure 4:Taking
an overview
Refs
•
•
•
•
•
•
•
•
•
•
•
•
Forbes, D., & Spence, J. (1991). An experiment in assessment for a large class. In R.Smith (Ed.),
Innovations in engineering education. London: Ellis Horwood.
Fritz, C.O., Morris, P.E., Bjork, R.A., Gelman, R. & Wickens, T.D. (2000) When further learning fails:
Stability and change following repeated presentation of text, British Journal of Psychology, 91, pp.
493-511
Gibbs, G. & Simpson, C. (2002) Does your assessment support your students’ learning available at:
http://www.brookes.ac.uk/services/ocsd/1_ocsld/lunchtime_gibbs.html (accessed November 2002)
Higgins, R., Hartley, P. & Skelton, A. (2002) The conscientious consumer: reconsidering the role of
assessment feedback in student learning. Studies in Higher Education, 27 (1) pp. 53-64
Hounsell, D. 1987. Essay writing and the quality of feedback. In J.T.E. Richardson, M.W. Eysenck
& D. Warren-Piper, eds. Student Learning: Research in Education and Cognitive Psychology, 42,
no.2: 239-54.
Holmes, L. E., & Smith, L. J. (2003). Student evaluations of faculty grading methods. Journal of
Education for Business, Vol. 78 No. 6, 318.
Lea, M. & Street, B. (1998) Student Writing in Higher Education: an academic literacies approach.
Studies in Higher Education, 23 (2), pp. 157-172
McCune, V., (2004) Development of first –year students’ conceptions of essay writing. Higher
Education, 47, pp. 257-282.
Maclellan, E. 2001. Assessment for learning, the different perceptions of tutors and students.
Assessment and Evaluation in Higher Education. 26, no.4: 307-318
Ridsdale, M.L.“I’ve read his comments but I don’t know how to do”:International postgraduate
student perceptions of written supervisor feedback. In ‘Sources of confusion: refereed proceedings
of the national language and academic skills conference held at La Trobe University, November 2728,2000’ edited by \k \charnock, pp272-282.
Rust, C. (2001) A briefing on assessment of large groups, LTSN Generic Centre Assessment
Series, No. 12, York, LTSN
Wotjas, O. 1998. Feedback? No, just give us the answers. Times Higher Education Supplement.
September 25