Advances in Health Sciences Education (2007) 12:169–186
DOI 10.1007/s10459-005-5046-6
Springer 2006
Peer Assessment in Problem-Based Learning:
A Qualitative Study
TRACEY PAPINCZAK*, LOUISE YOUNG and MICHELE GROVES
Mayne Medical School, School of Medicine, University of Queensland, Herston, 4006, Brisbane,
Queensland, Australia (*author for correspondence, E-mail: traceypapinczak@optusnet.com.au)
Received 26 June 2005; accepted 10 November 2005
Abstract. Peer assessment provides a powerful avenue for students to receive feedback on their
learning. Although student perceptions of peer assessment have been studied extensively in
higher education, little qualitative research has been undertaken with medical students in
problem-based learning (PBL) curricula. A qualitative study of students’ attitudes to, and perceptions of, peer assessment was undertaken within the framework of a larger study of metacognition with first-year medical students at the University of Queensland. A highly structured
format for provision of feedback was utilised in the study design. Many recommendations from
the higher education literature on optimal implementation of peer-assessment procedures were
put into practice. Results indicated the existence of six main themes: (1) increased responsibility
for others, (2) improved learning, (3) lack of relevancy, (4) challenges, (5) discomfort, and (6)
effects on the PBL process. Five of these themes have previously been described in the literature.
However, the final theme represents a unique, although not unexpected, finding. Students expressed serious concerns about the negative impact of peer assessment on the cooperative, nonjudgmental atmosphere of PBL tutorial groups. The practical implications of these findings are
considered.
Key words: assessment, peer assessment, peer evaluation, problem-based learning, qualitative
study
Introduction
In recent years, the quality of medical education has attracted considerable
attention, especially in light of the requirement to make education more
relevant to the needs of modern society. Medical education must prepare
students to deal with problems in the future, equipping them with skills
necessary to become active, self-directed learners, rather than passive recipients of information (Dolmans and Schmidt, 1996). Recognition of this need
was responsible, in part, for the development of problem-based learning
(PBL) (Barrows and Tamblyn, 1980).
Within the field of medical education, PBL is a curriculum innovation that
involves students in authentic learning activities using ill-defined medical
170
TRACEY PAPINCZAK ET AL.
problems as the stimulus and focus for learning (Norman and Schmidt,
1992). The pedagogical appeal of PBL is its perceived capacity to foster,
through these learning processes, enhanced clinical reasoning skills, and the
development of both an adaptable knowledge base to use in professional
settings and the skills in self-directed learning necessary to become lifelong
learners in that profession (Kelson and Distlehorst, 2000). Four critical
conditions for a deep approach to learning are encompassed within the PBL
approach: a well-structured knowledge base, active learning, collaborative
learner interaction, and a context designed to promote internal motivation
through the provision of pragmatic goals (Margetson, 1994). Evaluation of
student progress in such a student-centred curriculum, however, has remained a challenge (Eva, 2001).
Assessment protocols within PBL curricula have sometimes sought to
include participative (or peer) assessment, in which students contribute to the
evaluation of each others’ work, in order to match evaluation procedures
with the curricular philosophy. Peer assessment helps to develop the acquisition of self-directed learning skills (a key objective of PBL) as students
participate in the assessment experience (Ballantyne et al., 2002). As PBL
emphasizes the development of proficiency in the resolution of clinical
problems, the assessment of student skills, processes and attitudes would take
place most appropriately within the tutorial setting (Eva, 2001). There are
several advantages to employing tutorial-based peer assessment, including:
prolonged interaction between peers for provision of constructive feedback
based on multiple observations of performance; and opportunity to assess
areas of proficiency (such as communication skills, self-directed learning, and
respect for others) not readily evaluated by more traditional forms of
assessment (Eva, 2001). Higher education literature suggests that peer
assessment has many additional benefits, including enhanced metacognitive
skills (Ballantyne et al., 2002), and improved understanding of subject matter
(McDowell, 1995). Peer assessment encourages students to reflect on their
own approaches to assessment tasks (Dochy et al., 1999), to develop critical
reasoning skills (Hanrahan and Isaacs, 2001; Searby and Ewers, 1997) and
skills of reflection (Searby and Ewers, 1997; Somervell, 1993). Although
strong support for peer assessment is evident in the literature, difficulties and
limitations have repeatedly been reported.
Students’ perceptions that peer assessment can be unreliable and unfair
has been raised in several studies in higher education (McDowell, 1995;
Norton and Brunas-Wagstaff, 2000). Sluijmans et al. (2001) confirmed the
existence of bias in peer marking due to interpersonal relationships between
students. Students frequently report feeling uncomfortable carrying out peer
assessment, often because they feel unqualified to make these judgements
(Orsmond and Merry, 1996), lack faith in their own objectivity (Hanrahan
PEER ASSESSMENT IN PROBLEM-BASED LEARNING
171
and Isaacs, 2001), or feel the task is too challenging (Mowl and Pain, 1995;
Topping et al., 2000). The time-consuming nature of peer assessment is also
regarded negatively by students (McDowell, 1995; Topping et al., 2000).
While studies in medical education of students’ attitudes to peer assessment
in PBL have, to date, proven elusive, the results of quantitative studies
suggest that peer ratings are not a reliable means of tutorial-based assessment
in PBL (Reiter et al., 2002; Sullivan et al., 1999). Qualitative studies of peer
assessment within PBL have the capacity to provide insights into the nature
and extent of benefits to learning.
Research into medical education has largely emphasized quantitative rather than qualitative evaluation, particularly in the area of peer assessment.
Van Rosendaal and Jennett (1992) reported that peer evaluation was viewed
as an intrusion into the relationship among medical colleagues. Duffield and
Spencer (2002) surveyed medical students’ attitudes to assessment procedures
and found meaningful feedback about their progress in the course was
considered very important. Peer assessment provides one avenue for such
feedback to be delivered within the setting of the PBL tutorial.
Given the lack of research into student attitudes to peer assessment within
the PBL tutorial setting, this study has the potential to provide important
insights and to expand upon the findings from studies in non-PBL curricula
reported above. There have also been calls for greater qualitative exploration
of the social and educational aspects of PBL, in particular, ethnographic
accounts of PBL which endeavour to explore the influence of the culture of
the group (Leung, 2002). Such research has the potential to enhance our
awareness of all aspects of the PBL process.
The aim of this study was to explore student attitudes to, and perceptions
of, peer assessment concerning student fulfilment of roles and responsibilities
within their PBL tutorials. The PBL environment, with its emphasis on selfdirected and collaborative learning, provides a unique context in which to
study students’ attitudes to, and perceptions of, their self-directed learning
experiences. Qualitative methods allow the researcher to study these dynamic
processes.
Description of the Study
Data were gathered as part of a larger study of metacognitive processes
undertaken with first-year students enrolled in the Bachelor of Medicine and
Bachelor of Surgery (MBBS) Program at The University of Queensland.
Only the qualitative results of the peer assessment portion of the study will be
reported here.
Methods associated with participatory action research (Kemmis and
Wilkinson, 1998) and ethnography (Kamil et al., 1985) were adopted to gain
insight into student attitudes to, and perceptions of, peer assessment. Collins
172
TRACEY PAPINCZAK ET AL.
(1991) recommends the following epistemological grounding for engaging in
research among adult learners:
Instead of deploying intrusive research designs based on independent
learning project protocols, (we) might invite willing individuals to engage in exploratory conversations, as part of a pedagogical and hermeneutic process, to discover with what meanings adults endow their
own learning experiences. (Collins, 1991, p. 114)
Ethnographic inquiry endeavours to understand social behaviour from the
participant’s frame of reference (Kamil et al., 1985). In this context, the
experiences, behaviours, and attitudes of students learning medicine in their
PBL tutorials are observed and documented. The role of this researcher (a
PBL tutor) was based on participant–observer inquiry (Bishop, 1999). PBL
tutors are in a fortunate position to be able to observe the ‘culture’ of the
PBL tutorial and to participate as much or as little as desired in the ensuing
discussion and debate. The context and the participants were used to guide
research questions (Bishop, 1999) – the ‘classroom’ realities, developing
knowledge, and developing perceptions of the group allowed the researcher
to adapt the study design. The resulting design aimed to more fully reflect the
reality of the experience of students in their PBL tutorials.
Action research, involving overlapping cycles of planning, acting and
observing, and reflecting (Kemmis and Wilkinson, 1998), was utilised to
maintain a responsive study design. Data collection and analysis were combined and used to shape ongoing data gathering in order to develop a participative-assessment procedure responsive to student priorities and concerns.
The study design drew on literature in the area of peer assessment and
action research, incorporating numerous elements deemed important
including:
1. assessment of process rather than learning outcomes (see Boud et al.,
1999);
2. use of assessment criteria targeting five significant areas of performance
in PBL (refer Das et al., 1998; Willis et al., 2002), including assessment of
participation (Dancer and Kamvounias, 2005);
3. use of multiple specific criteria on the assessment instrument, targeting
discrete areas of student performance (as endorsed by Miller, 2003) rather than a few global indicators;
4. student feedback helping to guide final criteria selection, and explicit
details of implementation of the peer-assessment process (refer Ballantyne et al., 2002);
5. practice in peer assessment procedures to improve student familiarity (in
keeping with the recommendations of Ballantyne (2002), Sluijmans et al.
(2001), and Sullivan et al. (1999);
PEER ASSESSMENT IN PROBLEM-BASED LEARNING
173
6. provision of examplars of ‘good‘ and/or ‘poor’ work to students in initial
weeks of orientation (in accordance with the recommendations of Ballantyne et al., 2002; Hanrahan and Isaacs, 2001);
7. constant monitoring of student perceptions in order to guide the action
research process (Kemmis and Wilkinson, 1998).
Setting
The MBBS Program introduced a 4-year, graduate-entry PBL curriculum in
1997. First-year students, in small groups of 9 or 10, undertake 5 h of PBL
tutorial time each week for 33 weeks of the year. Working in collaboration
with their small group of peers, students analyse a problem of practice,
formulate hypotheses, and undertake self-directed learning to try to explain
the patient’s ‘disease’ process.
Subjects
The study was conducted with 165 first-year medical students and 20 tutors
in two phases over 2 years (refer Table I). In the first year, small student
focus groups were invited to participate in refinement of the study design and
peer-assessment instrument. In the following year (2004), 215 self-selected
subjects, representing 69% of the student group, were recruited for the main
study through an information session in the first week of the academic year.
Statistical analysis showed that study subjects were representative of the
entire student cohort on measures of age, gender, and primary degree. As
part of a larger study of metacognition, students were allocated to either
control or intervention groups. Allocation to the intervention cohort, based
on the criterion of total participation of group members within each PBL
Table I. Involvement of two cohorts of first-year medical students in the two phases of the
research
Cohort
Timing of
qualitative research
Number of
student participants
Research activities
Cohort 1
(2003)
Phase 1: April & June
Phase 2: July to
September
20 first-year
medical students 10
10 first-year
medical students
10 first-year
medical students
Phase 3: February to
September
125 first-year
medical students
Focus group discussions
to identify areas of need
Negotiation of criteria
for peer assessment
Trial of peer assessment
instrument and procedure
and re-negotiation
Implementation of
peer assessment procedure
Student feedback
Cohort 2
(2004)
174
TRACEY PAPINCZAK ET AL.
tutorial, resulted in 13 intervention (125 students) and 16 control tutorial
groups (90 students). All tutors assigned to intervention tutorial groups
agreed to participate. Intervention students took part in educational activities
within their PBL tutorials, including peer assessment, for a period of five
months (from which both qualitative and quantitative data were generated).
PROCEDURE
Ethical approval was obtained from the University of Queensland’s
Behavioural and Social Sciences Ethical Review Committee.
Initial exploration of peer-assessment and development of the instrument
Several focus groups from the first cohort of students (2003) were engaged in
an initial exploration of peer assessment options, some negotiation of the
criteria to be included in the assessment instrument (based on the relevant
literature in this field), and a short trial of the peer assessment procedure.
Qualitative data were collected from direct observation and focus group
interviews.
Initially, two different groups of 10 students were consulted within the
PBL tutorial setting in order to ‘test the water’ regarding student acceptability of participative assessment. An unstructured interview arrangement,
with the explicit use of group interaction to produce greater insight (Punch,
1998), was adopted. Remarks such as: ‘Marks allocated to PBL would recognize the amount of work done during this time and help motivate people to
work each week’ indicated student support for the overall concept. Subsequently, another group of ten students were invited to have input into
selection of appropriate marking criteria for participative assessment.
Numerous authors support the involvement of students in discussion and
negotiation of criteria to be used in assessment as it encourages them to
clarify their objectives and think critically about their work. Students were
distributed with a list of criteria proposed by relevant literature – such as
criteria for self-evaluation in medical education described by Das et al.
(1998), and student perceptions of motivational influences and assessment
within PBL (Willis et al., 2002). With discussion and negotiation, students
agreed that all characteristics listed were valid and important in describing
students who are ‘good’ participants, with some reservations concerning selfawareness items.
The negotiated Peer Assessment Instrument (with 19 items) was then
trialled with another group of 10 medical students and rated as easy to use
and understand by all participants. Student dissatisfaction with two items
resulted in their removal from the final version of the instrument. The
resulting scale measures several features of successful adult education, such
PEER ASSESSMENT IN PROBLEM-BASED LEARNING
175
as participation, punctuality, respect for others, effective communication,
and critical analysis. However, the inclusion of items specifically targeting
self-directed learning and self-awareness, core features of PBL, allows it to be
differentiated from others which may be appropriate for open-ended, but less
student-centred, approaches - such as case-based instruction (Hay and Katsikitis, 2001). The instrument consists of 17 items scored on a Likert scale of
one to five across five sub-scores: responsibility and respect, information
processing, communication, critical analysis, and self-awareness. These criteria were designed to address some facets of the important aspects of
learning in PBL, including self-directed learning, effective skills in collaboration and group-work, higher level thinking skills, and the ability to identify
one’s weaknesses and strengths.
In order to gain some measure of face validity, three experienced PBL
facilitators were asked to indicate whether each of the 17 items on the
instrument was relevant to PBL performance and able to be adequately assessed using the item in question. Unanimous face validity was obtained for
all items in the four sub-scores: responsibility and respect, information
processing, communication, and critical analysis. Some dissent about the
validity of the self-awareness sub-score was evident. Construct validity describes the degree to which the items used in the instrument define the constructs (Pressley and McCormick, 1995). The five constructs or domains of
performance were reported extensively in the medical and nursing education
literature. Each of the three PBL tutors and ten PBL students were asked to
categorize the 17 items into the five specified domains. In all cases, the items
were distributed in accordance with the domains as defined on the instrument. Values for Cronbach’s alpha ranged from 0.76 to 0.84, indicating good
internal consistency among the five sub-scores. Acceptable reliability was
found, with Pearson correlation coefficients for peer- and tutor-assessment
ranging from 0.40 to 0.60. Notably, self-awareness items were problematic
with a significant number of students consistently entering ‘not applicable’
for those two items.
Unfortunately, time constraints prevented further renegotiation of the
Peer Assessment Instrument or the details of the peer-assessment procedure
with the second cohort prior to the commencement of the main study.
Main study
The main study was undertaken with the following cohort of medical students, using the instrument and the refined design of the previous cohort.
Written student feedback was collected during, and at the conclusion of, the
main study. This feedback was applied to modify aspects of the larger study
framework, only some of which included the peer-assessment process itself.
176
TRACEY PAPINCZAK ET AL.
The intervention phase of the study involved participation of intervention
groups, for 20 weeks, in an activity designed to promote metacognition and
enhance learning. As part of this larger framework, students were given
practice in peer assessment for approximately 20 min each week as an integral part of the wrap-up phase of PBL. It was anticipated that repeated
exposure to a system of participative assessment would create a culture of
enthusiastic acceptance, with greater student support for the process, leading
perhaps to greater satisfaction with the PBL process operating within tutorial
groups. Control groups were not involved in this process.
The intervention itself comprised two key components, both of which were
readily integrated into the existing tutorial format with minimal additional
time required from tutors or students:
1. Reflection on learning: Each week one student from each intervention
tutorial group was asked to compose a summary of the week’s problem
incorporating the clinical reasoning and collaborative learning occurring
in their PBL tutorial group. One student was encouraged to present the
summary to the group as a concept map or in mechanistic case-diagramming format (see Guerrero, 2001) to give a visual representation of
both the content and the clinical reasoning (a ‘knowledge object’ (Entwistle and Marton, 1994)).
2. Participative assessment: The student presentation, in association with
fulfilment of PBL roles and responsibilities for that week, was assessed
using the Peer Assessment Instrument (refined through student negotiation with the previous cohort). Both peer- and tutor-assessment took
place concurrently. Scores from these worksheets were compared to look
for correlation between pairs of scores as part of a quantitative study of
peer assessment.
Constant monitoring of student perceptions of, and attitudes to, the
intervention helped to monitor the effects of the dual activities to maximize
student learning. This is in keeping with action research.
In the first week of the main study (week four of the academic year),
students in the intervention cohort were distributed two documents: a copy of
the Peer Assessment Instrument to enable students to become familiar with
the evaluation criteria, and an exemplar outlining ‘good’ and ‘poor’ outcomes for the criteria. Tutors assigned to each intervention group then led
their group in a practice session, with a tutor-led presentation of a summary
of the previous week’s medical case, in order to establish familiarity with the
instrument. The summary itself was written by the researcher and presented,
with explanation, to each tutor in the week prior to the trial.
In the ensuing weeks (5–26 of the academic year), intervention tutorial
groups implemented the summarization and peer-assessment activities at the
start of each week as part of the ‘wrap-up’ of the previous week’s medical
PEER ASSESSMENT IN PROBLEM-BASED LEARNING
177
case. Tutors were encouraged to give concise feedback (based on written peer
comments on the reverse of the assessment sheet) to students as soon as
possible after the complete on of the peer-assessment procedure. Scores obtained from the Peer Assessment Instrument were collated and analysed.
These quantitative results are to be reported elsewhere.
Student feedback about the peer-assessment exercise was regularly invited
as part of the action research process. Many groups chose not to reveal their
opinions, therefore more in-depth feedback from selected intervention tutorial groups was obtained using one-on-one interviews. One major criticism of
the peer-assessment process was identified – student preference for more
prompt, anonymous, and private feedback from the tutor (based on peer
scores and written responses). This feedback was used to modify the main
study design and tutors were advised to provide this form of feedback to
students as much as possible.
The attitudes of a randomly chosen control tutorial group to peer
assessment were also sought at this time. Tutor feedback was also collected at
10-week intervals. During the course of the implementation, three tutorial
groups withdrew from the study. Their justification for withdrawal and
perceptions of their experiences of peer assessment will be reported. The
remaining ten groups in the intervention cohort continued until week 28 of
the academic year. Qualitative data were sought from these groups.
DATA ANALYSIS
Responses to all open questions about peer-assessment and concise observations of focus groups were collated into a single document. Raw data
themes, in the form of direct quotations, functioned as the essential unit of
analysis. In keeping with the context of qualitative research, inductive data
analysis strategies were used with themes and categories emerging from the
data rather than the data being grouped into pre-determined categories
(Patton, 1990).
Results
The inductive data analysis procedure resulted in the identification of six main
themes. The first two are positive perceptions of peer assessment – increased
responsibility for others, and improved learning. The final four themes reflect
negative student perceptions – lack of relevancy, challenges, discomfort, and
effects on the PBL process. While the literature supports five of these themes,
the final theme which encompasses the potentially negative consequences of
implementing peer assessment on the PBL process has not previously been
described. However, Eva (2001) hypothesized that this effect may become
evident in PBL tutorial groups when peer assessment is implemented.
178
TRACEY PAPINCZAK ET AL.
PERCEIVED BENEFITS OF PEER ASSESSMENT
Increased responsibility for others
Concern for the potential effects of their feedback on the learning of their
peers led students to make comments such as: ‘Peer review is helpful (because) it keeps me mindful of the fact that all my input is ... important to the
learning of the whole group’. Another student’s comment: ‘It keeps you on
your toes and challenges you to come up with (feedback) that helps others’ –
shows how strongly some students felt about producing quality feedback for
peer benefit in the collaborative learning environment of PBL.
Improved learning
In this research, assessment of participation and leadership, in which the
contribution of students towards the process of learning was evaluated and
rewarded, was undertaken as a meaningful form of assessment of process (as
supported by Armstrong and Boud, 1983). Two main types of advantages were
described: immediate benefits to learning and long-term, sustainable benefits.
Immediate benefits. The positive consequences of constructive criticism
were reported on several occasions by participants. Students felt their peers
were in an appropriate position to judge their performance as functioning
members of the PBL group: ‘(peer-assessment) gives you a good idea of....
how you could improve and gives lots of ... ideas’ and ‘it is good to get
feedback from peers so that you know how you are going and where your
strengths and weaknesses are’. Through feedback from peers, opportunities
for students to engage in self-evaluation were provided. This assisted students
to identify deficiencies in their understanding and skills that may otherwise
not become readily apparent, thereby enhancing self directed learning.
Long-term benefits. Comments on future benefits to learning from the
practice of peer assessment were made by a small number of participants.
Mindful of a future as a medical professional, one student explained the
benefits of her peer-assessment experience: ‘... good to learn how to do this
appropriately, as I think we will need to be able to assess our peers’ performance, as well as our own, throughout our careers’. Students made references to ‘reflecting on strengths and weaknesses’ as a means of improving
future performance.
CRITICISMS OF PEER ASSESSMENT
Lack of relevancy
Despite the process of negotiation of assessment criteria with a previous
cohort of students followed by repeated practice at peer assessment,
PEER ASSESSMENT IN PROBLEM-BASED LEARNING
179
perceptions that the exercise lacked relevancy persisted. Two groups of responses were able to be identified in this category: irrelevant criteria, and
sceptical attitudes.
Irrelevant criteria. Some students failed to see the relationship between
the criteria on the feedback sheet and the aspects of PBL performance they
were expected to assess. One student did not view the activities of PBL as
sufficiently important to their learning as to warrant assessment: ‘Most of the
questions were irrelevant to the learning process’. Another respondent was
dissatisfied with the criteria: ‘(I) don’t feel that the assessment sheets enable
us to give relevant feedback’.
Sceptical attitudes. Frequent comments such as: ‘Not taken too seriously’
and ‘Not too much thought goes into the marking’ reflect a casual attitude to
the peer-assessment process. Some students were overt in their scepticism,
with 23% of the intervention cohort (three groups) withdrawing from the
peer-assessment activity before its’ completion partly for that reason.
Challenges
The difficult nature of peer assessment was frequently described by students.
Qualitative data analysis of students’ comments allowed two lower order
categories to be identified: ‘newness’ of peer assessment and lack of motivation.
‘Newness’ of peer assessment. A common complaint from student assessors was difficulty with the scoring system of the feedback sheet. These
problems were lack of familiarity and difficulties with the Likert scale. In
addition, a small number of comments were made concerning the alien
nature of peer assessment. One student noted: ‘Unless you have a couple of
weeks to get used to it, students don’t really know how to’. Nevertheless,
students were given 20 weeks to become accustomed to the practice and this
comment was received in the final week.
Lack of motivation. A minority of students expressed a dislike for peerassessment attributable to apathy. One individual comment highlighted the
nature of this perception:
I found myself consistently giving five’s for most areas…It was easier
than reading through every criteria and grading the person…I think I’m
not alone in doing this.
Discomfort
Two lower order categories encompassing discomfort were identified: problems with objectivity and need for greater anonymity.
180
TRACEY PAPINCZAK ET AL.
Problems with objectivity. A strong reaction to peer assessment was the
widespread perception that this process could be corrupted by bias due to
friendship marking or lack of honesty. Analysis of data from students who
left prematurely from the study indicated this was a very important influence
on the decision to withdraw. The following set of comments sums up the
attitude among many study participants:
I find it difficult to downgrade my peers.
(It is) hard to criticize friends.
Relationships between students can colour opinions.
No one wants to criticise others in PBL.
Most people are too afraid to honestly mark their peers.
An often reported criticism of peer assessment among students in this
study was lack of confidence in the evaluation skills of peers, for example,
‘Peers are not qualified to mark....They can be biased or picky or too lenient’.
One student expressed greater confidence in his own assessment abilities: ‘I
think my judgement is better than that of my peers’.
Anonymity. A minority of students expressed concern at the potential for
others to discern the source of individual scores or comments. Two significant remarks include: ‘I think (peer assessment) is great but (it) needs to be
done more anonymously’ and ‘It is very difficult to assess peers within the
confines of the PBL room, especially when we hand the forms in to the tutor
through many sets of hands’.
Some students may be more sensitive than others to the negative consequences of anonymity being compromised (an unlikely scenario given the
study design which preserved anonymity to a high degree). It was notable
that one student praised the highly confidential nature of her peer assessment
experience: ‘It’s good to be able to do this anonymously – a bit too confrontational face-to-face’.
Effects on the PBL process
Serious reservations about the negative impact of peer assessment on the
integrity and interactive functioning of the PBL tutorial group were demonstrated in this research. Both students and tutors revealed their concerns
that the ‘family atmosphere’ may be compromised by peer evaluation. One
student commented:
It promotes judgmental attitudes (and) tension, and destroys harmonious learning. By not having to rate my peers, (we could) learn in a
friendly, non-judgemental environment. We (could)… ask questions
without fear of embarrassment or humiliation.
PEER ASSESSMENT IN PROBLEM-BASED LEARNING
181
Within their PBL tutorials, students need to feel free to hypothesize, to ask
questions and request clarification of points raised by others in order to
create a dynamic learning environment (Woods, 1994). Some students recognized that the collaborative learning within their small group tutorials, so
dependent on affable working relationships, could be disrupted by ill-feeling
brought about by negative peer evaluations. This susceptibility was also
recognized by an experienced PBL tutor:
I would not like to see a competitiveness develop in a PBL group.
… Everybody should feel comfortable to express their opinions, ask
questions, and generally just have a go without a fear of being assessed.
These potentially negative consequences were not only identified by students and tutors experiencing peer assessment, but were anticipated by students in a randomly selected control group. One student expressed her
expectation:
The lack of a threatening environment is what allows us to discuss ideas
and concepts without fear of being rated or judged. In an environment
like this, feeling safe and respected in your group is paramount... this
would be screwed up by a scheme that asked you to rate your friends.
Discussion
Much of the data is congruent with findings in all disciplines of higher
education. Students have expressed appreciation for the manner in which
peer assessment enhances their learning, both short-term and in anticipation
of future learning. Peer assessment was considered to strengthen the sense of
responsibility that group members have for one another. These positive aspects of peer assessment can be regarded as endorsing the PBL approach,
with its emphasis on collaborative and meaningful learning. Some students
were enthusiastic and committed to assessment of their peers, endeavouring
to give valid and helpful feedback to support others’ learning and to graciously receive feedback from other PBL group members. Initial quantitative
data analysis suggests that these individuals were more accurate peer
assessors.
Several studies undertaken with students in non-medical courses have also
revealed strong positive perceptions of peer assessment – encouraging students to take a more active and constructive role in cooperative learning
(Hanrahan et al., 2001; Orsmond and Merry, 1996), and to develop attitudes
of responsibility for the learning of other group members (Burnett and
Cavaye, 1980; Topping et al., 2000). Furthermore, studies show peer
assessment enhances learning by allowing students to observe how others
182
TRACEY PAPINCZAK ET AL.
study and write (Ballantyne et al., 2002; Hanrahan and Isaacs, 2001), and by
training students to differentiate high from poor quality work (Race, 1998;
Searby and Ewers, 1997).
Negative perceptions were also strongly apparent in this research. A significant number of respondents believed that peer assessment was irrelevant
to the important aspects of learning in PBL. Scepticism and feelings of discomfort were very commonly reported. Of all five sub-scores, self-awareness
was the most problematic – during negotiation of items for the instrument
and in terms of student and tutor acceptability. Upon reflection, omission of
this sub-score may have improved student confidence in the peer-assessment
process (despite the principal role of self-awareness in PBL). While the PBL
tutorial group offers the students an opportunity to embrace unfamiliar
learning activities, such as collaborative small group learning with its
emphasis on peer-to-peer interaction and communication, peer assessment
may be the final straw for those students struggling to adapt to this learning
environment. It may be that learning environments that are highly dependent
on effective collaboration are not well suited to peer assessment. Alternatively, the nature of peer assessment undertaken in highly collaborative
learning environments may dictate a requirement for different criteria.
Negative perceptions of peer assessment were anticipated in this research
based on the findings of other studies in higher education.
Studies in non-medical courses have found widespread scepticism among
students concerning the validity of peer assessment aligned with beliefs that
assessment should be the sole responsibility of staff (Brindley and Scoffield,
1998; Orsmond and Merry, 1996; Searby and Ewers, 1997). Apprehension
about the arbitrary nature of marking are a major drawback for students
experiencing peer assessment (Ballantyne et al., 2002; Cheng and Warren,
1999). Pond and Rehan (1997) discuss the need to consider the problems of
friendship marking (leading to over-marking) and collusive marking (resulting
in lack of differentiation within groups) when considering the introduction of
peer assessment. In his influential article on student attitudes towards assessment, Williams (1992) reported that students viewed peer assessment as criticism of friends. The recommendations made to address this perception, which
included anonymous feedback from groups of peers with staff involvement to
reduce bias, were adopted in this study with little apparent impact.
It is a common finding that students struggle with the ‘newness’ of peer
assessment as a formal assessment tool (Cheng and Warren, 1999; Sluijmans
et al., 2001). They often have little prior experience with the procedure.
Criteria for peer assessment can create additional problems as some students
have different understandings of individual criteria compared to their peers
and tutor, particularly criteria requiring higher order thinking (Orsmond
et al., 1996,2000). Lack of familiarity may also lead to feelings of
PEER ASSESSMENT IN PROBLEM-BASED LEARNING
183
awkwardness. Students in many studies, including studies of cooperative
learning in postgraduate courses (Divaharan and Atputhasamy, 2002) and
PBL courses in education (Segers and Dochy, 2001; Sluijmans et al., 2001)
describe the discomfort of peer assessment.
A significant concern unique to peer assessment in PBL settings (and
apparently fairly unique to this study) was revealed: students perceived that
peer assessment had the potential to negatively impact on PBL. The intimate
quality of relationships within PBL tutorials, which has often been embraced
for its ability to improve self-evaluation and communication, also makes
students reluctant to rate their peers for the purposes of evaluation (Van
Rosendaal and Jennett, 1992). As Eva (2001) comments in his critique of
tutorial-based assessment:
There is a real concern that conducting evaluation in the tutorial setting
may negatively impact on the process itself. So, paradoxically, while the
tutorial is the best place to observe these behaviours, the act of observation and evaluation may inhibit their demonstration. (p. 244)
There are few studies exploring the effects of peer assessment on the process
of problem-based or case-based learning. Sluijmans et al. (2001) explored
several aspects of peer assessment in a PBL course, collecting mostly quantitative data, but also noting that students were aware of the serious consequences of negative critiques within the small group environment. Divaharan
and Atputhasamy (2002) described the unease experienced by students in the
face-to-face assessment of peers within the confines of cooperative small
groups, which had the potential to interfere with peer relationships. Boud
et al. (1999), in their discussion of the limitations and contradictions of peer
assessment, report that formal assessment of others within small group
tutorials can inhibit cooperation, pitting one student against another. However, peers are a source of valuable and valid feedback which can assist
learners to make judgements about themselves and their learning in PBL.
Conclusion
The results from this study support and extend research in higher education on the positive and negative perceptions of students about peer
assessment. However, the qualitative data, gathered through ethnographic
enquiry, presents a more in-depth representation of student attitudes to
the introduction of peer assessment into a PBL course within a medical
school.
The study design adopted in this research followed many guidelines
promoted in the literature. This included, but was not limited to, the use of
student-negotiated criteria for assessment, provision of exemplars of
184
TRACEY PAPINCZAK ET AL.
performance, and student practice in peer assessment to gain familiarity.
Despite implementation of these recommendations, student perceptions of
their peer-assessment experience remained quite negative. It may be that
students need years of practice in peer assessment in order to become comfortable with the process. Even though students were aware that peerassessment criteria were developed through negotiation with individuals in
the previous cohort, time constraints early in the academic year prevented
participating students from renegotiating criteria. This minimised ownership
of the assessment process, an important consideration in enhancing student
acceptance of participative assessment (Boud, 1995; Strahan and Wilcox,
1996). Student ownership of criteria for peer evaluation may be crucial to its’
acceptance as a meaningful activity.
The learning advantages offered by peer assessment are quite significant
and warrant further research. Productive benefits may be gained through
both receipt of feedback and provision of meaningful feedback to others.
These types of activities can improve confidence and enhance learning (Black
and Wiliam, 1998; Woods, 1994). Peer-assessment needs to be further explored with students in PBL and other cooperative small group settings,
including evaluation of fulfilment of the roles and responsibilities of group
members (perhaps utilising criteria reflecting what students value most from
their PBL experience). While the highly structured format of peer assessment
was viewed negatively by many students, an unprompted, anonymous and
open-ended design may be more acceptable. Such an approach may enable
students to give feedback which is relevant to their perceptions of valuable
learning and meaningful evaluation in PBL. Black and Wiliam (1998) found
that an approach to providing student feedback that involved writing comments on work, but not assigning scores, resulted in greater learning.
An integral part of the PBL tutorial process is the willingness and ability
of students to embrace their duties as active members of a group of learners.
Principles such as critical analysis, self-awareness, and information processing play an important role in the development of self-direction as a learner.
One of the gauges of quality in tertiary education is the extent to which selflearning ability is nurtured (Oldfield et al., 1995). A key aspect of this is
developing the ability to judge objectively the quality of their own and others’
work.
References
Armstrong, M.T. & Boud, D. (1983). Assessing class participation: An exploration of the issues. Studies in
Higher Education 8(1): 33–44.
Ballantyne, R., Hughes, K. & Mylonas, A. (2002). Developing procedures for implementing peer
assessment in large classes using an action research process. Assessment & Evaluation in Higher Education 27(5): 427–441.
PEER ASSESSMENT IN PROBLEM-BASED LEARNING
185
Barrows, H.S. & Tamblyn, R.M. (1980). Problem-Based Learning: An Approach to Medical Education,
New York: Springer.
Bishop, W. (1999). Ethnographic Writing Research, Portsmouth, NH: Boynton/Cook Publishers.
Black, P. & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education 5(1): 7–74.
Boud, D. (1995). Enhancing Learning Through Self-Assessment, London: Kogan Page.
Boud, D., Cohen, R. & Sampson, J. (1999). Peer learning and assessment. Assessment & Evaluation in
Higher Education 24(4): 413–427.
Brindley, C. & Scoffield, S. (1998). Peer assessment in undergraduate programmes. Teaching in Higher
Education 3(1): 79–89.
Burnett, W. & Cavaye, G. (1980). Peer assessment by fifth year students of surgery. Assessment in Higher
Education 5(3): 273–278.
Cheng, W. & Warren, M. (1999). Peer and teacher assessment of the oral and written tasks of a group
project. Assessment & Evaluation in Higher Education 24(3): 301–314.
Collins, M. (1991). On contemporary practice and research: Self-directed learning to critical theory. In: R.
Edwards, A. Hanson & P. Raggatt (eds.), Boundaries of Adult Learning, London: Routledge/Open
University Press.
Dancer, D. & Kamvounias, P. (2005). Student involvement in assessment: A project designed to assess
class participation fairly and reliably. Assessment & Evaluation in Higher Education 30(4): 445–454.
Das, M., Mpofu, D., Dunn, E. & Lanphear, J.H. (1998). Self and tutor evaluations in problem-based
learning tutorials: is there a relationship? Medical Education 32(4): 411–418.
Divaharan, S. & Atputhasamy, L. (2002). An attempt to enhance the quality of cooperative learning
through peer assessment. Journal of Educational Inquiry 3(2): 72–83.
Dochy, F., Segers, M. & Sluijmans, D. (1999). The use of self-, peer-, and co-assessment in higher
education. Studies in Higher Education 24(3): 331–350.
Dolmans, D.H. & Schmidt, H.G. (1996). The advantages of problem-based curricula. Postgraduate
Medical Journal 72(851): 535–538.
Drew, S. (2001). Student perceptions of what helps them learn and develop in higher education. Teaching
in Higher Education 6(3): 309–331.
Duffield, K.E. & Spencer, J.A. (2002). A survey of medical students’ views about the purposes and fairness
of assessment. Medical Education 36(9): 879–886.
Entwistle, N.J. & Marton, F. (1994). Knowledge objects: Understandings constituted through intensive
academic study. British Journal of Educational Psychology 64(1): 161–178.
Eva, K.W. (2001). Assessing tutorial-based assessment. Advances in Health Sciences Education 6(3): 243–
257.
Gale, K., Martin, K. & McQueen, G. (2002). Triadic assessment. Assessment & Evaluation in Higher
Education 27(6): 557–567.
Guerrero, A.P.S. (2001). Mechanistic case diagramming: A tool for problem-based learning. Academic
Medicine 76(4): 385–389.
Hanrahan, S.J. & Isaacs, G. (2001). Assessing self- and peer-assessment: The students’ view. Higher
Education Research and Development 20(1): 53–70.
Hay, P.J. & Katsikitis, M. (2001). The ‘expert’ in problem-based and case-based learning: Necessary or
not?. Medical Education 35(1): 22–28.
Kamil, M., Langer, J.A. & Shanahan, T. (1985). Understanding Reading and Writing Research, Boston:
Allyn & Bacon.
Kelson, A.C.M. & Distlehorst, L.H. (2000). Groups in problem-based learning (PBL): Essential elements
in theory and practice. In: D.H. Evenson & C.E. Hmelo (eds.), Problem-Based Learning: A Research
Perspective on Learning Interactions, Mahwah, NJ: Lawrence Erlbaum Associates.
Kemmis, S. & Wilkinson, M. (1998). Participatory action research and the study of practice. In: B. Atweh,
S. Kemmis & P. Weeks (eds.), Action Research in Practice: Partnerships for Social Justice in Education,
pp. 21–36. London: Routledge.
Leung, W. (2002). Why is evidence from ethnographic and discourse research needed in medical education:
the case of problem-based learning. Medical Teacher 24(2): 169–172.
McDowell, L. (1995). The impact of innovative assessment on student learning. Innovation in Education
and Training International 32(4): 302–313.
186
TRACEY PAPINCZAK ET AL.
Margetson, D. (1994). Current educational reform and the significance of problem-based learning. Studies
in Higher Education 19(1): 5–19.
Miller, P.J. (2003). The effect of scoring criteria specificity on peer and self assessment. Assessment &
Evaluation in Higher Education 28(4): 383–394.
Mowl, G. & Pain, R. (1995). Using self and peer assessment to improve students’ essay writing: A case
study from geography. Innovation in Education and Training International 32(4): 324–335.
Norman, G. & Schmidt, H.G. (1992). The psychological basis of problem-based learning: A review of the
evidence. Academic Medicine 67(9): 557–565.
Norton, L., Brunas-Wagstaff, J. (2000). Students’ perceptions of the fairness of assessment. Paper presented at the Institute for Learning and Teaching in Higher Education Annual Conference, College of
Ripon and York, St John.
Oldfield, K.A., MacAlpine, J. & Mark, K. (1995). Peer and self-assessment at tertiary level: An experiential report. Assessment & Evaluation in Higher Education 20(1): 125–132.
Orsmond, P. & Merry, S. (1996). The importance of marking criteria in the use of peer assessment.
Assessment & Evaluation in Higher Education 21(3): 239–250.
Orsmond, P., Merry, S. & Reiling, K. (2000). The use of student derived marking criteria in peer- and selfassessment. Assessment & Evaluation in Higher Education 25(1): 23–38.
Patton, M.Q. (1990). Qualitative Evaluation and Research Methods, Newbury Park, CA: Sage Publications.
Pond, K. & Rehan, U.H. (1997). Learning to assess students using peer review. Studies in Educational
Evaluation 23(4): 331–348.
Pressley, M. & McCormick, C.B. (1995). Advanced Educational Psychology for Educators, Researchers and
Policymakers, New York: Harper Collins College Publishers.
Punch, K.F. (1998). Introduction to Social Research, London: Sage.
Race, P. (1998). Practical pointers on peer-assessment. In: S. Brown (ed.), Peer Assessment In Practice,
SEDA Paper 102, Birmingham: SEDA.
Reiter, H.I., Eva, K.W., Hatala, R.M. & Norman, G.R. (2002). Self and peer assessment in tutorials:
Application of a relative-ranking model. Academic Medicine 77(11): 1134–1139.
Sambell, K. & McDowell, L. (1998). The value of self and peer assessment to the developing of lifelong
learning. In: C. Rust (ed.), Improving Student Learning: Improving Students as Learners, Oxford: Oxford
Centre for Staff and Learning Development.
Searby, M. & Ewers, T. (1997). An evaluation of the use of peer assessment in higher education: A case
study in the school of music. Assessment & Evaluation in Higher Education 22(4): 371–383.
Segers, M. & Dochy, F. (2001). New assessment forms in problem-based learning: The value-added of the
students’ perspective. Studies in Higher Education 26(3): 327–343.
Sluijmans, D.M.A., Moerkerke, G., Merrienboer, J.J.G.van & Dochy, F.J.R.C. (2001). Peer assessment in
problem-based learning. Studies in Educational Evaluation 27(2): 153–173.
Somervell, H. (1993). Issues in assessment, enterprise and higher education: The case for self-, peer- and
collaborative assessment. Assessment & Evaluation in Higher Education 18(3): 221–233.
Strahan, I.B. & Wilcox, S. (1996). Peer and self-assessment of group work: Developing an effective
response to increased enrolment in a third-year course in microclimatology. Journal of Geography in
Higher Education 20(3): 343–353.
Sullivan, M.E., Hitchcock, M.A. & Dunnington, G.L. (1999). Peer and self assessment during problembased tutorials. The American Journal of Surgery 177(3): 266–269.
Topping, K.J., Smith, E.F., Swanson, I. & Elliot, A. (2000). Formative peer assessment of academic
writing between postgraduate students. Assessment & Evaluation in Higher Education 25(2): 146–169.
Van Rosendaal, G.M.A. & Jennett, P.A. (1992). Resistance to peer evaluation in an internal medicine
residency. Academic Medicine 67(1): 63.
Williams, E. (1992). Student attitudes towards approaches to learning and assessment. Assessment &
Evaluation in Higher Education 17(1): 45–58.
Willis, S.C., Jones, A., Bundy, C., Burdett, K., Whitehouse, C.R. & O’Neill, P.A. (2002). Small-group
work and assessment in a PBL curriculum: A qualitative and quantitative evaluation of student perceptions of the process of working in small groups and its assessment. Medical Teacher 24(5): 495–501.
Woods, D.R. (1994). Problem-based Learning: How to Gain the Most from PBL, Waterdown, ON: Donald
R. Woods.