An Action Research Approach to Assessment and Learning on a 2nd-Year Classics Module
=== CASE STUDY ===
Centre for Integrative Learning, University of Nottingham
Date of Submission: July 2009
Project Leader: Lynn Fotheringham
School/Department: Humanities/Classics
Introduction
Module Q82105 Independent Second Year Project (ISYP), offered annually in the Department of
Classics since Spring 1992, has been convened by various members of the Department and
contributed to by all. It is an example of on-going panDepartmental Action Research: each year, colleagues’
experience, students’ work, and External Examiners’
reports, are available as data to be reflected on.
Generalisations are made from this data, and changes
designed to improve the module are tested on the next
cycle, providing further experience to be reflected on in
turn (image taken from Cowan 1998).
ISYP belongs to a sequence of Classics modules which
introduce students to increasing amounts of independent
study, culminating in the 3rd-year dissertation. The dissertation, in which students define their
own topic of research and work alone with a supervisor rather than in a classroom, has
traditionally been viewed as the crowning achievement of the undergraduate Arts degree. In
contemporary discussions of HE, the ability to work independently is viewed as a valuable
graduate attribute contributing to employability. Traditional and contemporary attitudes
therefore both place a high value on independent work. ISYP gives students a first opportunity to
define their own area of research. It is compulsory for all Single Honours students in the
Department not taking 20 credits of language in the relevant semester (the acquisition or
improvement of language-skills being viewed as of foundational importance to Classics). As a 20credit 2nd-year module, it develops students’ ability to work independently for relatively low
assessment stakes.
Over the last ten years, increased student-numbers and increasingly rapid staff-turnover have
had their effect on ISYP. In 1998, when twenty-five students took the module, consistency in
assessment was ensure by having all projects double-marked by the same two members of staff;
by 2004, when enrollment topped fifty, this was no longer reasonable in relation to staff
workloads. Until 1999, those convening, assessing and supervising on the module had been
involved in it since its inception; from 2000 there have been new staff-members every year
needing induction into the innovative aspects of ISYP (for which see below under ‘Integrative
Learning rationale’), to serve as supervisors and, from 2004, as assessors. The results have been
a) the development of material which attempts to make explicit aspects of the module which
were implicit before, and b) an emerging debate among colleages over various assessmentissues.
Process a) is a familiar one, reflecting recent attempts to move away from a ‘hidden curriculum’
towards explicit statements of Aims & Objectives, Learning Outcomes, Assessment Criteria, Grade
Descriptors, and so forth. Process b), itself part of the on-going action-research cycle, set the
questions to be addressed in this study. ISYP was one of the last assessment-methods in the
Department to be provided with written Assessment Criteria (2003), following essays,
translations and comment-questions – all of which occur in more than one module, and so
required explanation more urgently. By 2007, these Assessment Criteria did not capture all the
Learning Outcomes of the module, or all the evidence of learning which the students produced. A
first attempt at change was made in 2007, the year before the CIL Case Study began.
For the participants in the module-activities over the three years in question, see Table 1.
Integrative Learning rationale
ISYP provides students with a variety of Integrative Learning opportunities, which individual
students configure to their own personal tastes. The requirements of ISYP are broadly defined.
Students choose the format as well as the subject-matter of their work, and design their own
projects deserving 20 credits. Projects must be approved by an individually assigned supervisor
and the module convener; this process ensures the students are undertaking work of academic
value in an appropriate subject-area. Formats chosen in the past include: website; creative or
journalistic writing; teaching plan; museum exhibit; theatrical performance; children’s book;
artwork; database; board-game; reconstruction of ancient objects/activities; guidebook.
A variety of possibilities for Integrative Learning are provided by ISYP.

The module is integrated into a sequence which gradually increase the students’
experience of independent study.

Integration across module boundaries: students are encouraged to be explicit about how
they integrate the knowledge acquired in various previous modules with newly acquired
knowledge.

Integration of University study with the world of work: students may choose a projectformat in order to gain experience useful in a future career; obvious examples include
teaching-related projects and journalistic writing, while less obvious examples include a
adverts for Roman beauty projects by a student interested in an advertising career.

Integration of University study with the wider community: many formats imply an
audience other than the academics who usually see and assess student work. Teachingprojects and childrens’ books are the most obvious examples here, but students designing
a website, a board-game or a museum exhibit must also take some thought for the
intended audience.

Integration with peers: team-projects are an option on the module. In addition, many
students can be observed collaborating at all stages of the module, from discussing how to
select a project, to comparing difficulties, to helping one another out by serving as testaudiences, volunteer re-enactors, photographic models, and so on.
The diversity of project-formats means that Assessment Criteria must be general enough to
encompass different types of knowledge-acquisition and communication. The concept of
Integrative Learning was a key tool in constructing a discourse about ISYP which made explicit to
students and staff what is expected of the module. The career- and communication-issues listed
above were utilised to help teachers describe the difference between ISYP and other modules in
ways which help the students do well in both traditional and innovative assessment. The work is
therefore an example of the principles of Integrative Learning being recognised and made explicit
in existing teaching practice, with resulting improvement of the situation of both students and
teachers.
Methodology
Action Research methodology (McNiff, Lomax & Whitehead 1996) emphasises the gathering of
data and opinion both before and after implementation of ideas. Discussion with the Spring 2007
convener and examination of the External Examiners’ reports was followed by extensive
consultation of stakeholders (students, colleagues, externals). After the module had been taught
in Spring 2008, further consultation was undertaken. Throughout, the process was informed by
advice from education specialists in the CIL, as well as by an increasing knowledge of the existing
bibliography on assessment.
Since most assessment in Humanities subjects involves interpretation, the approach taken to
assessment in this Project is interpretivist (Johnston 2004). Examples provided to staff assessors
and postgraduate assistants aimed to build the interpretative community of ISYP-‘connoisseurs’,
in the renowned model of Eisner (e.g. 1985). Students also appreciated the opportunity to view
examples of high-achieving projects from the past, becoming ‘connoisseurs’ as well as producers
– as they do with academic writing in traditional modules. An emphasis on discussion, both
among staff and between staff and students, reflects the emphasis on consultation prominent in
the classic work on interpretivist evaluation, Guba & Lincoln 1989, and chimed with the Action
Research methodology. This also provided a model for advice to students on investigating
options and evaluating results, and was thus tested within the module itself.
The goal of both model-provision and discussion, to make the assessment criteria as explicit as
possible, was grounded in literature such as Bryan & Clegg 1996, Boud & Falchikov 2007, and
Dunn et al. 2004. The need for clarity is further emphasised in works as diverse as Black et al.
1994 (on portfolios) and Knight & Yorke 2003 (on employability). Further help on the
assessment of creative work was provided by several articles in Art, Design and Communication in
Higher Education, including Cowdroy & Williams 2006 and Gordon 2004.
Implementation
Preliminary work indicated that the most significant area for consideration was how the moduleassessment should take into account the different items submitted by the student: the project
itself, a reflective statement, and a portfolio documenting the working process. Should these
different elements be awarded different marks? What should the assessment criteria for each be?
The following were now consulted: the previous year’s students (questionnaire); members of the
Department (e-mail consultation and group discussion); External Examiners (e-mail consultation
on draft documents). ISYP work submitted 2007 was also recorded in order to provide a bank of
material to work with.
The assessment decisions taken in Autumn 2007 were as follows:

to maintain the innovation, introduced in Spring 2007, of awarding 75% of the module
mark to the project, and 25% to the reflective statement;

to discard the previous year’s practice of subdividing the 75% and 25% into smaller
arithmetic divisions attached to each of the assessment criteria (15% for Knowledge of
the Ancient World, 10% for Aims and Objectives, etc.), in favour of a holistic approach to
assessment comparable to that practiced in the Department with regard to essays, etc.;

to maintain the practice of not awarding a separate mark for the documentation of
process, but to develop an explicit explanation for the students of how this documentation
relates to both the project itself and the reflective statement;

to revise the Assessment Criteria (see Table 2).
The CIL funding also gave the Project Leader time and materials available for providing the
following support to students and staff:



STUDENTS

revision of module booklet/hand-outs to take into account the new criteria and the
Integrative Learning rationale;

increased WebCT support: detailed discussion of Assessment Criteria; advice on
individual project-formats; more examples of previous projects; examples of
previous process-documentation; examples of good and mediocre reflective
statements (invented, in order to avoid making poor work from previous students
public); FAQs page;

supply of books/dvds in the Departmental library, including ‘how to’ books on
various skills (from teaching to re-enactment), and ‘models’ for various projecttypes (children’s books, documentary films);

the presence of postgraduate teaching-assistants in the introductory meetings,
large portions of which are given over to student buzz-groups, providing more
students with individual attention at the project-selection stage;
STAFF

provision of sample sets of previous projects, illustrating how a project-format can
be done to various levels of quality, with explanatory commentary;

lunch-meeting to discuss changes, and problems arising in supervisions;

provision of hard copy of all advice given to students, to ensure that all colleagues
involved in assessment were aware of what the students were expected to do and
what the students were expecting to be rewarded/penalised for;
BOTH

detailed grade descriptors written and distributed.
One the basis of staff- and student-questionnaires, feedback was provided by the postgraduate
assistants, External Examiners’ reports and the convener’s observation of the 2008 assessment
process, further alterations were then suggested for the following year. In Autumn 2008 support
for these alterations was sought and received from staff and External Examiners via e-mail; the
module booklet was further revised to incorporate material which had been developed later in
the Spring 2008 semester, and the WebCT site updated to take account of changes.
The assessment decisions taken in Autumn 2008 were as follows:

to return to the pre-2007 practice of giving a single mark to each student’s work; but

to revise the Assessment Criteria to ensure that the reflective statement and processdocumentation can be taken account of in the mark, and be commented on in the feedback
(see Table 2 and Appendix 1).
Although the 2007 experiment of splitting the mark for the module between 75% for the project
and 25% for the statement has been abandoned for the present, after two attempts to make it
work, this is not simply a return to the status quo ante, since the Assessment Criteria have been
entirely revised, and the process has generated a considerable amount of support-material for
both students and assessors, which can now be disseminated.
Evaluation
Evaluation is based on the comments of stakeholders: students, staff, and External Examiners.
24 students from the 2007 module and 29 students from the 2008 module filled in a
questionnaire. Both results indicated a high level of satisfaction with the module, confirming the
Department’s belief that most students value ISYP. 2008 generated generally higher results for all
aspects of the module, with none receiving a ‘very unsatisfactory’ from any student (previously
given to ‘Explanation of how to document the process’). There was in 2007 a general demand for
more examples to help them understand what is needed, which was supplied via WebCT. 2008
students were asked to state how often they had used the WebCT site, and the results indicated
that the site had proved both attractive and useful to students.
Student-response to open-ended questions were also encouraging. Again, the level of satisfaction
was and continued high, with especial approval for the freedom of choice element of the module:

Could choose any subject, which meant everyone worked on something they were
interested in. (2007)

I liked the way it gave us an opportunity to combine Classics and the career you would like
to pursue. (2008)

I liked the chance to do something interesting and different, using different skills than just
writing essays. (2008)
Most notably, the number of complaints about vagueness and lack of clarity reduced
substantially, from seventeen in 2007 to two in 2008. This suggests that the positive effect of new
explanatory and other support material.
Staff comments in 2008 included both endorsement of various elements and suggestions for
further variations. The application of a mark for the reflective statement but not for the processdocumentation was still considered a problem, but advice and reading indicated that the
development of detailed assessment material for these portfolio-like structures would be
extremely complicated; the solution attempted in 2009, as noted, was to give a single mark but to
ensure that aspects of the statement and documentation were included in that mark. The module
convener’s observed that assessors (including the externals) discussed the marks in terms of
final marks for the whole work, not one mark for the project and one for the statement; the
decision received widespread in the Autumn 2008 consultation. Positive comments were made
about the grading descriptors, the limits set on the length of the process-documentation, and the
quality of the reflective statements produced by the students.
The externals were uniformly positive. Comments include:

ISYP is the most exciting, and successful, experiment in post-traditional learning methods
ever conducted in its field, and the length and depth of Departmental expertise in
nurturing, managing, and assessing creative and skills-based learning is a resource
without peer anywhere in the subject.

The revision of Q82105 ISYP included also clearer guidelines for the students on how to
go about their independent project. This is reflected in a higher proportion of very good
projects in comparison to last year.
The positive responses from all groups of stakeholders indicate that the work done so far has
been valuable, but Action Research is by definition on-going. The results of the 2009 module and
consultation remain to be analysed; colleagues may in the future wish to make further changes.
Just as it is hoped students learn about their own approach to more traditional modules by doing
something different, one result of this work has been to focus staff-attention on the need to
explain what is expected of the students in traditional assessment-methods as well as to
innovative ones. Furthermore, it may be questioned whether the provision of Assessment
Criteria and Grade Descriptors, unbolstered by the support of explanations and support
(currently provided on WebCT), is an adequate response to the need for explicitness.
Dissemination and Sustainability
The ISYP module is taught every year to over fifty undergraduates; over a hundred students have
already benefitted from this Case Study, which will continue to benefit others. The Department is
committed to the on-going teaching of this module, and the work of the past two years has
provided a corpus of material to be used as as resource by both teachers and students.
The project has been showcased at a number of CIL events:



03/06/08 Practitioner-led Project Showcase
17/09/08 CIL advisory board
10/12/08 “Embedding Innovation in Teaching and Learning” (joint showcase held by UoN
CETLs)
A poster advertising the module and the work being done on it was also presented at the
Classical Association AGM in Glasgow, 03-06/04/09.
Funding was obtained from the HEA Subject Centre in History, Classics and Archaeology to build
on the work already done by disseminate the ISYP methodology to colleagues from Classics
Departments across the country. This will lead to an inter-institutional meeting and the
production of a ‘teacher’s guide’ describing how to implement ISYP-style assessment.
Further support from the CIL has enabled an inter-departmental meeting of Nottingham
colleagues interested in non-traditional assessment. Colleagues from English Studies, Philosophy,
Archaeology and Music were able to attend the meeting; further colleagues from History, Cultural
Studies, Biosciences and Veterinary Medecine have expressed interest in being updated on future
activities.
Future plans include a publishable ‘student’s guide’ and an article in an educational research
journal.
References
Black, Daiker, Sommers & Stygall (eds.), New Directions in Portfolio Assessment, Portsmouth, NH
1994.
Boud & Falchikov (eds.) Rethinking Assessment in Higher Education: Learning for the Longer Term,
London 2007.
Bryan & Clegg (eds.), Innovative Assessment in Higher Education, Abingdon 1996.
Cowan, J. On Becoming an Innovative University Teacher. Reflection in Action, Maidenhead 1998.
Cowdroy & Williams, “Assessing creativity in the creative arts”, Art, Design and Communication in
Higher Education 5 (2006) 97-117.
Dunn, Morgan, O’Reilly & Parry, The Student Assessment Handbook. New directions in traditional
and online assessment, London/New York 2004.
Eisner, The Art of Educational Evaluation, London/ Philadelphia 1985.
Gordon, “The ‘wow’ factors: the assessment of practical media and creative arts subjects’, Art,
Design and Communication in Higher Education 3 (2004) 61-72.
Guba & Lincoln, Fourth Generation Evaluation, Newbury Park 1989.
Johnston, “Summative assessment of portfolios: an examination of different approaches to
agreement over outcomes”, Studies in Higher Education 29 (2004) 395-412.
Knight & Yorke, Assessment, Learning and Employability, Maidenhead 2003.
McNiff, Lomax & Whitehead, You and Your Action Research Project, London 1996.
Table 1: Participants in ISYP, 2007-2009
Year
Spring 2007
Spring 2008
Spring 2009
Convener
Katharina Lorenz
Lynn Fotheringham
Lynn Fotheringham
# students taking module
68
66
591
# students answering
questionnaire
24
29
?
# staff supervising/
marking
?
9
9
# staff new to module
2
2
0
# staff also involved in
previous year
not relevant2
5
4
# staff consulted
n/a
173
124
# external examiners
scrutinising marks
3
3
35
There were more JH students, for whom the module is not compulsory, in the 2009 2nd-year
cohort.
2 In 2006, the assessment process had been the same in previous years, so all staff with prior
experience of the module in 2007 had the same experience. This was not the case in the following
years, as the changes described in the Case Study were being made.
3 All current and some retired colleagues were consulted in Autumn 2007; 8 current and 1
retired colleague attended a lunch-meeting in November.
4 All current colleagues were briefly consulted by e-mail in Autumn 2008; 12 replied.
5 The same three individuals acted as External Examiners throughout the three-year period.
1
Table 2: Developing Assessment Criteria for ISYP, 2004-2009
from 2004
2007
Project
 Project Design
 Project Execution
 Skills Development
 Knowledge & Understanding of
the Ancient World
 Project Design
 Project Execution
 Skills Development
 Knowledge & Understanding of the
Ancient World
Statement
(25% when
assessed
separately)
n/a
 Aims & Objectives
 Methods & Difficulties
 Assessment
2008
2009
Project
 Knowledge & Understanding of
the Classical World
 Communication of Knowledge &
Understanding
 Project Planning & Execution
 Skills Development &
Deployment
 Knowledge & Understanding of the
Classical World
 Communication of Knowledge &
Understanding
 Project Planning & Execution
 Skills Development & Deployment
 Documentation & Evidence
 Reflective Statement
Statement
(25% when
assessed
separately)
 Structure & Argument
 Critical Thinking
 Use of Evidence
n/a
Appendix 1: Sample Coursework Feedback Form, showing ‘holistic’ assessment of different
elements
School of Humanities
Department of Classics
Spring Semester 2008/09
INDEPENDENT
PROJECT
FEEDBACK
Name of student
A.N.Other
Project type/title
Journalistic writing: reviews of recent films
Report by
I. Teacher
FORM
a = excellent b = good c = adequate d = poor e = unacceptable
These gradings are a guide to performance but do not reflect specific marks. You should refer to
Undergraduate Handbook 6.1-2 for further information on marking standards and assessment criteria.
1. Knowledge and
understanding of
the classical world
b
[Positive comments, identification of notable errors.]
2. Communication
of knowledge and
understanding
a
[Praise with specific reference to particular points, commenting on both writing
and layout.]
3. Project planning
and execution
b
[Comments on improvements, perhaps with reference to how they could be
implemented in the disseration next year.]
4. Skills
development and
deployment
a
[Praise, and suggestions for future development.]
5. Documentation
and evidence
b
[Positive comments on what was there (e.g. intelligent analysis of model
articles), regretful comments on what was missing (e.g. evidence of up-to-date
reading on academic topic).]
6. Self-assessment
statement
b
[Suggestions and advice for future writing.]
Other/general
comments
Unpenalized mark, if applicable:
MARK:
69