Helping psychology undergraduates
improve examination writing
Kathy Harrington, Mercedes Freedman, Savita Bakhshi, Peter O’Neill
London Metropolitan University
Promoting Psychology Student Learning
through Assessment
27 March 2009
London Metropolitan University
HEA Psychology Network
Write Now Centre for Excellence in Teaching & Learning
Overview of presentation
Aims and context of study
 Pedagogical rationale
 Description of the intervention
 Method
 Findings
 Conclusions and implications

2
Study aims
Improve students’ examination writing
and performance
 In addition, we wanted to…




Build on previous research on use of
assessment criteria to improve student
learning
Develop and evaluate an intervention based on
collaboration between psychology academics
and writing specialists (Writing in the
Disciplines model)
Create re-useable teaching & learning
resources
3
Context of study



2nd-year cognitive psychology module
Assessment 100% by examination
Traditionally lower than average pass rate




Identified by university as a “killer module”
Plan of response required
Write Now CETL works with discipline-based
academics to improve student learning through
curriculum and teaching development
Write Now CETL runs university Writing Centre
staffed by trained student peer mentors in
academic writing


Collaborative, non-directive, supportive
Enable students to take responsibility for own work
4
Pedagogical rationale

Students and tutors often interpret meanings of assessment
criteria differently (Harrington et al, 2006; Lea & Street, 1998;
Merry et al, 1998)

Providing clear and explicit criteria is a first step in helping
students understand what tutors are looking for in written work

However, research has also shown that facilitating students' active
engagement with the criteria is necessary if learning and
performance are to be demonstrably enhanced (Price et al, 2001)

Structured interventions focussed on understanding and
demonstrating assessment criteria have been shown to lead to
improvements in student learning and performance (Norton et al,
2005; Rust et al, 2003)

Other research has found that students value being able to talk to
peer tutors about their writing, and that psychology students
prefer peer tutors from their own discipline (Bakhshi et al, in
press)
5
The intervention

4 hour-long compulsory workshops embedded as part of
module teaching across the autumn semester 2008-09

Immediately following two-hour lectures

Delivered in alternate weeks, with workshops run by
lecturers in between (focussed more explicitly on lecture
content)

Designed by team of academic writing specialists,
psychology lecturer, psychology PhD student


with experience of pedagogical research in area of student
writing and assessment
Delivered by academic writing specialists and peer writing
mentors studying psychology (3rd-year and PhD students)
6
The intervention (continued)

Exam answers posted in VLE prior to workshops for
students to read and give a grade

Set of accompanying materials developed using extracts
from authentic examination answers annotated with
comments in relation to assessment criteria


Specific focus on cognitive psychology
Use of departmental assessment criteria

In workshops, materials used to facilitate discussion about
demonstrating assessment criteria at different levels of
performance

Students guided in small groups to adopt role of examiner
and apply criteria to whole past examination answers

Final class discussion drew out students’ insights and
summarised main points
7
Data collection and analysis


Attendance registers taken at workshops 2, 3 and
4 (not at first workshop)
Questionnaire distributed at last workshop
(n=63)

Likert scale: students’ perceptions of helpfulness of
workshops in relation to





Examination writing
Meeting assessment criteria
Understanding subject matter of cognitive psychology
Examination grades
Analysis using SPSS to produce descriptive and
inferential statistics
8
Study sample


N=205 students enrolled on the module who also
took the examination
40 students enrolled on module did not take the
examination
Study population
Total students enrolled on
module
Conversion Diploma
64 (31.2%)
72 (29.4%)
Single Honours
93 (45.4%)
113 (46.1%)
Other/unidentified
5 (2.4%)
11 (4.5%)
Joint Honours
43 (21%)
49 (20%)
Total
205 (100%)
245 (100%)
9
Findings: pass rate and mean grade
Number of
students
Pass rate
Fail rate
Mean final
grade
2007-08
191
71%
29%
47%
2008-09
205
62%
38%
43.6%
10
Pass rates by degree course
Test
Value
df
Asymp.
Sig.
(2-sided)
Pearson
ChiSquare
47.754
3
.000**
Final Grade category (pass or fail)
Pass
Fail
Total
Conversion
Diploma
Count
57
7
64
%
89.1%
10.9%
100.0%
Single
Honours
Count
54
39
93
%
58.1%
41.9%
100.0%
Count
5
0
5
%
100.0%
.0%
100.0%
Count
11
32
43
%
25.6%
74.4%
100.0%
Count
127
78
205
Other
Joint Honours
Total
11
Attendance at workshops
40
Actual student attendance at workshops for weeks 2, 3 and 4
(n=205)
35
30
25
% 20
15
10
5
0
Week 2
Week 3
Week 4
• Attendance not recorded for first workshop
• Attendance highest at second workshop (37.1%)
12
Percentage who attended workshops
Frequency
Percent
Valid Percent
Cumulative
Percent
0
100
48.8
48.8
48.8
1
50
24.4
24.4
73.2
2
29
14.1
14.1
87.3
3
26
12.7
12.7
100.0
205
100.0
100.0
Valid
Total
• 100 (48.8%) students did not attend any workshops
• 105 (51.2%) students attended at least one workshop
13
Attendance and pass/fail in examination
Pass or fail
Pass
Fail
Total
Count
51
49
100
%
51.0%
49.0%
100.0%
Count
76
29
105
Test
Value
df
Asymp.
Sig.
(2-sided)
Continuity
Correction
9.047
1
.003**
No
Attendance at
at least
one
workshop
Yes
%
72.4%
27.6%
100.0%
Count
127
78
205
%
62.0%
38.0%
100.0%
Total
14
Attendance and final grade category
Figure 1: Percentage of students who attended workshops by grade category
60
50
1st
40
2.1
2.2
% 30
3rd
20
Fail
10
0
0
1
2
Number of workshops attended
3
15
Attendance and course studied
Figure 2: Percentage of students who attended workshops by course studied
60
50
40
0
1
% 30
2
3
20
10
0
Convers i on Di pl oma
Si ngl e Honours
Joi nt Honours
Cours e s tudi ed
16
Workshop attendance and grades: 1
Correlation between total number of workshops attended and final grade
Pearson Correlation
Final grade
Total number of
workshops
Final grade
Total number of
workshops
1.000
.314*
Sig. (2-tailed)
.012
N
63
63
Pearson Correlation
.314*
1.000
Sig. (2-tailed)
.012
N
63
63
*. Correlation is significant at the 0.05 level (2-tailed).
17
Workshop attendance and grades: 2
Figure 3: Correlation between number of workshops attended and final grade
• significant positive relationship (r=.314, n= 63, p<0.05)
• the more workshops attended, the higher the grade
18
Workshop attendance and grades: 3
Figure 4: Total number of workshops attended and mean final grade
Mean final grade overall: 43.61%
Conversion Diploma: 60.34%
Single Honours: 38.8%
Other: 58.2%
Joint Honours: 27.44%
19
Similar findings in other research

Lusher (2007)



Small-group workshops focused on assessment
criteria with 3rd-year health psychology
students
Significant correlation between attendance and
mean examination scores (r=0.254, N=111,
p<0.01)
Multiple regression showed that performance
did not independently predict attendance, so
not just a matter of more able students
attending workshops
20
Students’ perceptions of workshops
7-point Likert scale: strongly disagree to
strongly agree
 Mean scores for all items were positive






Understanding what assessment criteria are
Understanding subject matter of cognitive
psychology
Achieving a better grade
Producing better writing
Understanding how to demonstrate
assessment criteria
21
Students’ perceptions (continued)
Item
Questions
N
Minimum
Maximum
Mean
Std.
Deviation
5.1
The workshops helped me understand what assessment
criteria are
63
3
7
6.16
.884
5.2
The workshops helped me understand what makes a good
examination essay in Cognitive Psychology
63
2
7
5.92
.955
5.3
The workshops helped me understand the subject-matter of
Cognitive Psychology
63
1
7
4.56
1.457
5.4
The workshops will help me achieve a better grade in
Cognitive Psychology
63
1
7
5.43
1.132
5.5
The workshops will help me achieve a better grade in my other
psychology modules
63
2
7
5.43
1.043
5.6
The workshops will help me produce better academic writing
in general
63
2
7
5.57
1.027
5.7
The workshops helped me understand how to critically
evaluate in psychology examination writing
63
2
7
5.51
1.014
5.8
The workshops helped me understand how to use theories
and evidence in psychology examination writing
63
1
7
5.30
1.186
22
Students’ perceptions (continued)
Item
Questions
N
Minimum
Maximum
Mean
Std.
Deviation
5.9
The workshops helped me understand how to develop an
argument in psychology examination writing
62
1
7
4.71
1.323
5.10
The workshops helped me understand how to structure a
psychology examination essay
63
1
7
5.41
1.278
5.11
The workshops helped me understand how to address the
question in a psychology examination essay
62
1
7
5.23
1.298
5.12
The workshops helped me understand how to demonstrate my
understanding of the subject in psychology examination
writing
63
1
7
5.30
1.159
5.13
The workshops helped me understand how to use appropriate
language in psychology examination writing
63
1
7
4.65
1.393
5.14
The workshops helped me understand how to identify
strengths and weaknesses in my own psychology writing
63
1
7
5.02
1.442
5.15
The workshops helped me feel more confident about
examination writing in psychology
63
1
7
5.13
1.251
5.16
The workshops motivated me to spend more time studying
Cognitive Psychology
62
1
7
4.52
1.457
23
Students’ perceptions (continued)

Highest mean scores for



“The workshops helped me understand what assessment
criteria are” = 6.16
“The workshops helped me understand what makes a
good examination essay in Cognitive Psychology” = 5.92
Lowest mean scores for


“The workshops motivated me to spend more time
studying Cognitive Psychology” = 4.52
“The workshops helped me understand the subjectmatter of Cognitive Psychology” = 4.56
24
Conclusions

Module pass rate lower this year at 62%, compared to 71% in
2007-08

However, a number of minor changes were made to content and
delivery, so comparison across years problematic

In 2008-09, attendance at the workshops was significantly
correlated with higher examination grades

Confounding factor is that more able students are more likely to
be attending in first place


Students’ who attended perceived the workshops to be helpful for






More tests needed, cf. Lusher (2007)
Understanding what assessment criteria are
Understanding how to demonstrate the criteria in their own writing
Understanding the subject matter of cognitive psychology
Achieving a better grade
Producing better writing
Difficulty of addressing needs of weaker students, even with
“embedded” teaching
25
Implications




Students benefit from explicit focus on writing within
disciplinary teaching
Students benefit from talking about their own academic
writing with trained peers in same discipline
Importance of embedding teaching of writing and “writing
to learn” activities within modules, rather than viewing
writing as an add-on skill separate from learning subject
matter
Planned changes to module for next year



Less distinction between lectures and workshops by identifying
3-hour “teaching blocks” instead, with varied mix of lecture
and workshop activities
Time to practice writing in teaching sessions
Peer review of own writing, facilitated by peer mentors
26
References
Bakhshi, S., Harrington, K., and O'Neill, P. (in press). Psychology students’ experiences of
academic peer mentoring at the London Metropolitan University Writing Centre,
Psychology Teaching and Learning, Spring 2009.
Harrington, K., Elander, J., Norton, L., Reddy, P., Aiyegbayo, O. & Pitt, E. (2006). A
qualitative analysis of staff-student differences in understandings of assessment
criteria, in C. Rust (Ed.), Improving Student Learning through Assessment. Oxford:
Oxford Centre for Staff and Learning Development.
Lea, M. R. & Street, B. (1998). Student writing in higher education: an academic
literacies approach, Studies in Higher Education, 23, pp. 157-72.
Lusher, J. (2007). How study groups can help examination performance, Psychology
Update, 16, 1 & 2.
Norton, L., Harrington, K., Elander, J., Sinfield, S., Lusher, J., Reddy, P., Aiyegbayo, O. &
Pitt, E. (2005). Supporting students to improve their essay writing through
assessment criteria focused workshops, in C. Rust (Ed.), Improving Student
Learning: Inclusivity and Diversity. Oxford: Oxford Centre for Staff and Learning
Development.
Merry, S., Orsmond, P. & Reiling, K. (1998). Biology students’ and tutors’ understanding
of a ‘good essay’, in C. Rust (Ed.), Improving Student Learning: Improving Students
as Learners. Oxford: Oxford Centre for Staff and Learning Development.
Price, M. and O'Donovan, B. and Rust, C. (2001). Strategies to develop students'
understanding of assessment criteria and processes, in C. Rust (Ed.), Improving
Student Learning - 8: Improving Student Learning Strategically. Oxford: Oxford
Centre for Staff and Learning Development.
Rust, C., Price, M. & O’Donovan, B. (2003). Improving students’ learning developing their
understanding of assessment criteria and processes, Assessment and Evaluation in
Higher Education, 35, pp. 453-472.
27
http://www.writenow.ac.uk
28