Technology-enhanced assessment
and transformational change
David Nicol,
Project Director, REAP www.reap.ac.uk
Deputy Director
Centre for Academic Practice and Learning
Enhancement (CAPLE)
University of Strathclyde
Nottingham Trent University, 3rd November 2006
The REAP Project







3 HEIs (Strathclyde, Glasgow Caledonian Business
School, Glasgow University)
Focus is on large 1st year classes
Pedagogy: assessment for learner self-regulation
Range of technologies: online tests, simulations,
discussion boards, e-voting, e-portfolios, peer/feedack
software, admin systems, VLEs, offline-online
Goals: Learning quality and teaching efficiencies
Outputs: case studies of redesign: advice to support
strategic change in institutions (transformation).
Dissemination across HE/FE sector
My Plan
Briefly discuss issues relating to first year
experience and assessment
 Review literature on formative assessment and 1st
year experience (synthesis)
 Provide case study of first year course redesign
(drawn from REAP project)
 Analysis of case study in relation to assessment
principles from literature
 Transformational and strategic change

First Year: The academic experience
Why students leave academic programmes early:
1. Flawed decision-making in initial choices
2. Events that impact on students’ lives outside the
classroom
3. Students’ experiences of the programme and the
institution
4. Failure to cope with the academic demands of
programmes
Yorke and Logden (2004) Retention and student
success in higher education.
First Year: The academic experience
What is important?
 Coping with transition
 Understanding what is required
 Engagement with academic programmes
 Support and feedback
 Experiences of success
 Agents in own learning
 Belief in self (ability) and motivation
 Social dynamics of learning (belonging)
Yorke and Tinto’s research
Gibbs and Simpson (2004)
Gibbs, G. & Simpson, C (2004) Conditions
under which assessment supports students
learning, Learning and Teaching in Higher
Education, 1, 3-31.
See: Formative Assessment in Science Teaching
(FAST) project at:
http://www.open.ac.uk/science/fdtl/
Gibbs and Simpson (2004)
Assessment tasks [Conditions 1-4]
1. Capture enough study time (in and out of class)
2. Are spread out evenly across timeline of study
3. Lead to productive activity (deep vs surface)
4. Communicate clear and high expectations
Gibbs and Simpson (2004)
Feedback [Conditions 5-11]
5.
Is sufficient (in frequency; detail)
6.
Is provided quickly enough to be useful
7.
Focuses on learning rather than on marks
8.
Is linked to assessment criteria/expected outcomes
9.
Makes sense to students
10. Is received by students and attended to
11. Is acted upon, to improve work and/or learning
Nicol and Macfarlane-Dick (2006)

Literature review:
Nicol, D. & Macfarlane-Dick, D. (2006). Formative
assessment and self-regulated learning: A model and
seven principles of good feedback practice. Studies in
Higher Education, 34 (1), 199-218
See:
 Student Enhanced Learning through Effective
Feedback [SENLEF] project funded by HE Academy
 REAP project: www.reap.ac.uk
Research on Assessment in HE
Teaching/learning paradigm
Constructivist
Transmission
[student-centred]
Assessment paradigm
Transmission
[teacher-centred]
Some key research findings




Students are always engaged in self-assessment/selfregulation of their own learning (Winne, 2005; Black
& Wiliam, 2005). Logically entailed by constructivist
thinking.
The act of using teacher feedback implies that selfassessment must be present (Sadler, 1983, Nicol and
Macfarlane-Dick, 2006)
Feedback in HE is being significantly reduced so how
are students still learning?
The question is: how can we scaffold students
learning so they become better at self-regulation of
learning? (Lajoie, 2005)
Scaffolding self regulation: 7 principles of good feedback
1.
2.
3.
4.
5.
6.
7.
Clarify what good performance is (goals, criteria,
standards).
Facilitate reflection and self-assessment in learning
Deliver high quality feedback to students: feedback
that enables students to monitor and self-correct
Encourage peer and tutor dialogue around learning
Encourage positive motivational beliefs & self esteem
through assessment
Provide opportunities to close the feedback loop
Use feedback information to shape teaching
Source: Nicol and Macfarlane-Dick (2006) Formative assessment and self-regulated learning: A model and
seven principles of good feedback practice, Studies in Higher Education
Re-engineering Assessment Practices [REAP]
Psychology Case Study
Context:
 560 first year students
 Mixture of psychology majors (130) and those
taking psychology only for one year (430)
 6 topic areas, 48 lectures, 4 tutorials, 12 practicals
 Assessment; 2 x MCQs (25%), tutorial attendance
(4%), taking part in experiment (5%), essay exam
(66%)
Psychology Case Study
Problems identified with the course:
 Students got no practice in writing skills but
required in the exam
 No feedback except on MCQs (percent correct)
 Didn’t want to increase staff workload
 Wanted to improve overall exam marks
 And standard of entrant to second year
Psychology re-design
Stage 1: Question 1: moderate difficulty (50 words)
Individual response – post it –discuss answer in groups (of 6)
Timed release: model answer to self-evaluate their response
Stage 2: Question 2: difficult (100 words)
Group response – discuss (online) – agree – post response
Model answer released for stage 2
Stage 3: Question 3: complex (300 word essay)
Group response – discuss (online) – agree – post
Model answer released for stage 3
Psychology



Task 1: Define and describe structural encoding,
phonological encoding and semantic encoding.
Provide an example of each construct. (50 words,
individual)
Task 2: Describe the serial position effect and its
two separate components. Discuss the specific
structural components of memory that are
responsible. (100 word, group response)
Task 3: Summarise the ‘stage theory’ of memory.
To what extent does it provide an adequate theory
of memory? (300 word, group response)
Psychology re-design: key features







Writing tasks related to current lecture delivery (on
human memory).
78 volunteers –induction task
Question of increasing difficulty and length of
answer (50 words, 100 words and 300 words)
Moving from individual to group responses.
‘Scaffolding’ of learning through peer discussion
(and cognitive growth thro conflict resolution)
Comparing work against model answers
Use of assignment/ discussion tools in (WebCT)
Planned developments





Replace half the lectures with discussion tasks
Tasks will become progressively more demanding
within and across topics as the year progresses.
Participation will be compulsory and monitored
Individual responses will precede online
discussion and response for all tasks
Once a final group response has been agreed each
student will submit a copy to VLE (WebCT)
Purpose is to deal with free rider-effects

Source: Jim Baxter, Psychology, Strathclyde University
The seven feedback principles






Standard format and model answers provide progressive
clarification of expectations (clear goals, principle 1)
Students encouraged to self-assess against model
answer (self-assessment, principle 2)
Online peer discussion aimed at reaching consensus
about response (dialogue, principle 4)
Staged complexity and focus on learning rather marks
(motivation, principle 5)
Repeated cycle of topics and tasks (closing gap,
principle 6)
Tutors can monitor progress and adapt (shaping
teaching, principle 7)
Gibbs & Simpson’s four assessment conditions




Assessment tasks require significant study out of class
(capture sufficient study time, condition 1)
They are distributed across topics and weeks (are spread
out evenly, condition 2)
They move students progressively to deeper levels of
understanding (productive/ deep learning, condition 3)
There are explicit goals and progressive increase in
challenge (communicates clear and high expectations,
condition 4)
What can we learn from this case study?






Use of a simple technology (discussion board)
Considerable thought gone into the pedagogical
design [which is transferable].
Supported by robust assessment/learning
principles
Coordinated approach within the department
Evaluation showed a lot of supportive
socialisation during induction task
Intention to compare year on year examination
performance.
Assessment and the the first year


Two super principles that need to be balanced
within the first year and across years
Engagement: Organised and structured academic
activities (so students know they are making
progress).
Empowerment: Opportunities for students to
internalise academic standards and supervise
themselves as they study in relation to these
standards (self and peer assessment processes)
First Year: The academic experience
What is important?
 Coping with transition
 Understanding what is required
 Engagement with academic programmes
 Support and feedback
 Experiences of success
 Sense of agency in own learning
 Belief in self (ability) and motivation
 Social dynamics of learning (belonging)
Yorke and Tinto’s research
Questions and discussion
Transformational change
SFC definitions (pace/scale, cost-quality, levels)
Enablers
 Motivation
 Powerful pedagogic designs
 Readiness factors (benchmarking)
 Links to other strategy developments and external
drivers
 Scenario planning
 Dissemination

Case Study 2



Large class taught through lectures, tutorials
Exam – series of multiple choice questions
to test breadth of understanding
Sometimes essay or short answer questions
in exam.
Redesign






Students introduced to MCQs – how they work
After lectures but before tutorial/problem solving
class students in pairs prepare MCQ including
feedback for right and wrong answers.
In tutorial pairs swap MCQs and get feedback –
revise - post in VLE.
Refinement: students identify which level of
Bloom’s taxonomy they questions are testing
Final exam teacher selects some students’ questions
but has them provide reasons for answers.
Producing questions compulsory
Benefits of Redesign





Students develop questioning skills
Creating feedback develops writing skills and
critical thinking – giving reasons for correct and
wrong answers - deep learning
Sharing in class encourages peer feedback
Identifying Blooms level leads to further reflection
Use in final exam encourages class to share work
Relation to feedback principles







Students create MCQs (actively create criteria, principle 1)
Evaluating content against criteria & Bloom categories
(self-assessment/reflection, principle 2)
Tutor monitoring and general feedback (principle 3)
Peer feedback during creation and in tutorials (dialogue,
principle 4)
MCQs used in exam and the creation process is engaging
(motivation, principle 5)
Cyclical development of MCQs (closing gap, principle 6)
Teaching could be shaped by results (principle 7)
Gibbs & Simpson’s four assessment conditions




Writing MCQs as preparation for tutorials (capture
sufficient study time, condition 1)
Task could be a regular requirement and built into
tutorials (are spread out evenly, condition 2)
MCQs could move to deeper levels matching Blooms
taxonomy (productive/ deep learning, condition 3)
The goals are clear and there is progressive increase in
challenge (communicates clear and high expectations,
condition 4)
Task


Do you have a learning design that you could share with
the group or a specific method?
Using the seven principles/questions and/or four
conditions can you identify ways of improving some
aspect of your own class teaching?
Other relevant papers
Nicol, D (1996), Increasing success in first year courses:
assessment redesign, self-regulation and learning technologies,
Paper being presented at ASCILITE Conference, Sydney, Dec 36, 2006.
Nicol, D (in press), Laying the foundation for lifelong learning:
cases studies of technology supported assessment processes in
large first year classes, British Journal of Educational
Technology (to be published 2007).
Nicol, D. & Milligan, C. (2006), Rethinking technologysupported assessment in relation to the seven principles of good
feedback practice. In C. Bryan and K. Clegg, Innovations in
Assessment, Routledge.