UNIVERSITY OF EXETER
EXTERNAL EXAMINERS’ REPORTS:
CHUMS UNDERGRADUATE PROGRAMMES OVERVIEW 2013/14
1
Introduction
1.1
This paper forms a general overview; highlighting good practice, common themes and key recommendations highlighted by the 37 External
Examiners’ Reports for CHUMS undergraduate programmes. The standard online report form for undergraduate programmes was used by all
External Examiners.
1.2
It should be noted that External Examiners reports are effectively monitored through the University’s Quality Review Framework section on
External Examining.
2
Good Practice
2.1
All 36 External Examiners highlighted aspects of good practice as follows:
(i)
(ii)
(iii)
(iv)
(v)
(vi)
(vii)
(viii)
The wide and fascinating range of course modules that are offered, which challenge and stretch students;
The strong and active relationship between the teachers' research/professional practice and their teaching, which underpins the whole
provision;
The wide range of assessment that ensures that candidates are assessed across a range of their abilities, and the assessment
procedures which encourage the development of presentational skills and group work;
The rigorous moderation process is viewed seriously by staff, and clear explanations are provided where action has been taken to resolve
discrepancies between marks awarded by first marker and moderator;
The practice of comprehensive double grading assessments done on both new courses and courses taught by new members of staff,
which helps to ensure calibration to the University's assessment criteria and provides valuable mentoring as regards the provision of
feedback;
The use of marks over 80% to properly reward exceptional work, and robust marking of work that is very good but lacks the incisiveness
and originality to merit a mark of 70+;
The feedback to students on work is extensive, explanatory and constructive, and personal;
The efficient use of Dropbox to deliver exam papers;
(ix)
(x)
The inclusion on the coursework header sheet of the opportunity for the student to comment on their own evaluation of success, effort,
difficulties etc.;
The dedication and hard work of academic staff and the administrative team who have worked efficiently throughout the year.
3
Common Themes/Key Recommendations:
3.1
Those recommendations categorised as Essential, Advisable and Desirable are outlined below:
Theme
Essential
4
Externals
made
recommendations:
Teaching and learning
Assessment
Advisable
Essential 16
Externals
made
recommendations:
Desirable
Advisable 23
Externals
made
Desirable
recommendations:
 Modern Languages/FLC - To set up
a new Chinese programme staff
development is important. The
teaching members should be
sponsored to attend the annual
meeting for Chinese language
teachers of British universities and
research staff to join British
Association of Chinese Studies.
They should be given sufficient time
to develop the new modules and
teaching materials to assure the
quality of tuition.
 English – The External suggests
that it would be good, as a longer
term aspiration, to see a wider
variety of more specialised modules
on offer. Poetry, life writing, writing
for young people and writing for
theatre are examples where the
provision could be enhanced and
expanded.
 FLC - The design of some of the in-  FLC - For the Advanced 3 Reading
class tests needs to be revised. The
and Writing In-class test, the text
listening and writing test needs to be
students
have
to
read
is
more challenging. The writing
unnecessarily long (3 pages), and
element in this test is, in the
the piece they are required to write
External’s opinion, unnecessary,
(250-300 words) too short for this
since writing is already assessed in
level. The External would like to
other elements of the coursework
see the balance reversed so that
and exam. Trying computer marked
students can engage more fully
testing of listening would be an
with the productive side of the task.
excellent development, since it can
For a two-hour paper the External
be prepared ahead, and adequate
would expect students at this level
resources should be made available
to write at least 400 words.
to staff who are interested in  Drama – The department should
developing and piloting this type of
reconsider the titles of DRA 1004
assessment.
and DRA 1012, ensuring that they
 Drama - The department should
reflect the module content and
continue its revisions of the number
assessments and consider the
of assessment points, including its
parity of weighting of different
review of the use of continuous
elements of assessment.
assessment. Some units would  Drama – The department should
benefit from fewer, more substantial
ensure that there is an expectation
assessment assignments; and some
that all student writing (including
clarification of particularly complex
reflective
essays)
features
scholarly tasks, for instance, the
appropriate scholarly grounding
combining of consideration of
(evidenced
by
bibliographic
semiotic theories with a performance
references).
analysis in a relatively short piece of  English - The Department exercises
writing.
an extreme degree of care both in
 History – While the new comparative
the assessment of work, and in
courses are an excellent idea they
constructive feedback to students.
need further development.
The
Maintaining so high a level of staff
moderation
is
sometimes
investment must be a considerable
perfunctory
and
the
External
challenge as student numbers rise.
suggests doing more formative
Further streamlining of assessment
assessment instead.
procedures will be to the benefit of
both academic staff and students.
 FLC – The External suggests that
example questions should be
added for all questions where
appropriate.
 Modern Languages - Internal
Marking and moderation
 FLC - Further revisions are required
to the descriptor for the 40-49 band
which currently uses terms such as
‘poor’,
‘inadequate’,
‘many
grammatical inaccuracies’, ‘impedes
understanding’. These suggest that
the performance of the student does
not meet the requirements for a
pass. The wording needs to be
revised using terms such as
'adequate', 'limited', 'sufficient', etc.
 Modern Languages – The External
remains unhappy with joint courses
with Arabic, Turkish and Indonesian,
where marks are regularly above
80% or even 90%, thereby inflating
a student's grade artificially. This
was made point last year, and the
matter was referred to the relevant
Department, which affirmed the
veracity of their marking system.
The External encourages the
moderation has been enhanced,
and there is an opportunity for more
inventive use of External Examiners
e.g. asking them to look at
programmes more broadly rather
than just individual modules, and
ensuring
comparability
of
assessment procedures between
modules on different programmes.
 Modern Languages – The External
suggests that the provision of
translations
into
English
for
commentary questions on some
examination papers is unnecessary
and potentially confusing for the
students.
 Drama – The department should  Drama – To put in place the same
continue its development of robust
triple marking processes on core
benchmarking
procedures
modules for very high grades (i.e.
particularly in large team-taught
over 75%) as currently exist for
modules
where
experienced
lower grades.
teachers are working alongside  History - The range of modules
inexperienced graduate teaching
available to students is a great
assistants. The department should
strength. Many of these include
consider more effective means of
innovative assessment patterns.
encouraging a use of wider ranges
The modules are taught by a wide
of marks, particularly in first and 2.2
range of individuals.
This ranges, as there is a tendency to the
combined with the innovative
mean in many units overall results.
assessment patterns - means that
 Drama – Consideration should be
there is a need to ensure
given to standards of assessment for
consistency
of
assessment
the performance element of year
between tutors. There was a wide
one coursework and whether it is
range of marks this year, especially
appropriate, for instance, for ALL
on 'Uses of History', and the
students (over 100) taking the core
External would suggest that in
module DRA 1012 to receive grades
future there is a statistical review of
of 2:1 or above.
the marks awarded by each tutor
which can then (if necessary)
 Drama – Consideration should be
University to establish a Facultywide grading system, so that all
marks within a degree programme
are equivalent, and therefore a
student's final grade is 'fair', not just
to that student, but other students
within the Faculty.
 History
The
treatment
of
moderation appears to be uneven
and bunching of marks a major
concern. This requires urgent
rectification.
 History - Absence of External
access to annotated copies of
course work means no real
assessment/moderation of marking
is possible.




given to under what circumstances a
trigger a review of whether scaling
'threshold' pass mark (40 - 42%) is
needs to take place to ensure
appropriate, ensuring that there are
consistency. It would be useful to
clear shared understandings of the
have in place a mechanism to allow
differences between this work and
review. The External Examiners
work that should fail. This includes
were provided quite late with
essays that include plagiarised
statistical data showing variation
material.
between modules. This suggested
that, on the whole, differences in
English - The Unit should address,
mean marks between similar
the issues of consistency of marking:
modules
were
justified
(i.e.
a) between exams and coursework,
students' marks were typically in
especially at the 2.2 level, where
line with their marks on other
there is a reluctance to mark poorer
modules). It would be helpful if
coursework at this level which is not
Externals could see this data
so apparent in exam marking;
earlier, and the Department should
b) in coursework and dissertations,
make use of this valuable data,
at the 2.1/1st border, where due
when reviewing its own provision.
weight should be given to second
markers' judgements to be given in  History – The External believes that
discussions between markers.
the moderation of marks tends to
encourage
rather
too
much
English - Since online submission
'agreement' over marks and
via Turnitin is now practised for
suggests that it might be better to
coursework, the Unit should move to
blind-double
mark
a
certain
using Turnitin's anonymised marking
percentage
of
assignments
facility to ensure robust coursework
(perhaps 20%).
marking.
FLC – Two Externals advise the  FLC – The External feels that the
high weighting for grammar in
writers of the marking criteria to
written exams (50%) tends to
review them both in terms of levels
distort
overall
exam
results.
and in terms of categories (e.g.
Although this doesn't seem to
fluency
separate
from
impact significantly on overall
pronunciation).
module marks, it produces slightly
Modern Languages – The External
misleading impressions for the
was concerned by the student
exam itself. While the level of
achievement profile for the module
difficulty for these exams is
SML2244, which generated some
appropriate, a rebalancing of the
52% of scripts in the first-class
weight of the grammar section
category and a comparatively high
would perhaps produce more
mean mark of 67. The External
reiterates the suggestion made in
last year's report that the department
explore ways in which basic
statistical information (i.e. mean and
median marks, standard deviation
and numbers and percentages of
scripts falling into the different
classes) could be compiled as
modules are being marked (e.g.
through the creation of Excel
spreadsheets for each module)
which could then be made available
to External Examiners along with
scripts for moderation. Such a
practice would greatly facilitate both
moduleand,
especially,
programme-level moderation by
flagging up anomalous achievement
profiles and enabling them to be
scrutinised by External Examiners.
 English - Regarding the element of
peer feedback in the module Finding
a Voice, where students gave written
comments on presentation, the
External believes it may be
necessary to look at this element of
the assessment again.
The
feedback was highly variable in
quality and it wasn't clear how it was
being used, if at all, in arriving at a
mark for the presentations.





realistic
picture
of
students'
achievements and abilities.
FLC - The introduction of enhanced
moderation/second
marking
is
welcome, but the process could still
be
made
more
transparent.
Moderators should briefly explain
their approach and in particular
provide justification for any mark
changes.
FLC – The External would like to
encourage module convenors to
review the high distinction rate for
several modules, consider possible
reasons and eventuality of possible
corrections, and fail without timidity
when required according to marking
criteria.
English - It would be helpful for the
Externals to see all coversheets to
enable
easy
comparison
of
feedback from different markers.
English - In some cases, the
process of moderation is still not
transparently recorded (e.g. where
marks have been changed to
respond
to
problems
or
unevenness between markers).
There are instances of very good
practice, but not everyone is filling
in the forms clearly. Sometimes,
borderline cases like a 69 are not
moderated. It might be good
practice to moderate all borderlines,
not as benchmarks of a grade band
but precisely as special cases that
could benefit from being checked.
English – The External would




Feedback
 History – While there is excellent 
paperwork in place for assessment
the 2 Externals suggest that:
a) The space for feedback on the
BART tick-box sheet could be
made slightly larger to allow for
more comments. Occasionally
comments to students were a 
suggest a greater reluctance to
mark at 59 and 69. These marks
can cause doubt for students, and if
the marker is sure the work is a 2:1,
then perhaps a 68 would clarify this
status.
Classics and Ancient History – The
External felt that some of the
marking seemed a little tough,
especially at the top end of the
range, where markers could
justifiably be less cautious about
awarding marks in the high 70s or
80s.
Classics and Ancient History – It
was
noticeable
that
some
Exchange students had performed
very
poorly,
this
warrants
consideration.
Modern Languages - There were
some instances in which the
marking criteria had been changed
but documentation had not been
updated.
Modern Languages - The External
noted that the notation and
recording of instances in which
word length penalties have been
applied could be more consistent
and clear.
Drama - The department might
reflect on whether its written
feedback need be so extensive in all
cases; and whether the time taken
to produce a lengthy summary
comment could be better spent in
more marginal feedback.
Drama – To continue working to
Dissertations
little slight.
ensure that quality and quantity of
b) It would be useful for Externals
feedback is comparable for all
to receive photocopies of work
students taking the same module
with tutor's comments on, in
(especially for core modules with
order to assess accurately
large cohorts marked by different
levels of feedback.
tutors).
 Archaeology - Assessment feedback  Archaeology
–
The
External
forms can be variable. While the
suggests some re-design of the
quality of feedback was always
BART feedback forms so that
good, in a number of cases it was
elements
like
grammar
and
unclear who the first marker was,
referencing become graded tick
and in a number it was unclear
boxes leaving more space for
whether or not the piece had been
formative comment on academic
second marked/moderated.
content.
 FLC – The External advises the  Archaeology – The handwriting on
teaching team to continue to
some assessment feedback sheets
improve in the area of student
could be improved.
feedback
for
coursework
by  Modern Languages – The External
providing consistent feedback both
indicates that markers could make
in terms of quality and quantity.
fuller use of the 'in order to
improve...' section on the feedback
sheets.
 Modern Languages – The External  English - Turnitin submission of
was concerned about the very high
dissertations
would
help
the
number of final year dissertations
External gain sufficient time, by
which had been awarded a mark
seeing scripts earlier than at
ending in -9, suggesting that marks
present, to look over their marking.
ending in -9 (which are not  Archaeology – Sufficient time
composite marks) should be avoided
should be allowed for External
in a situation where a weighted
Examiners
to
moderate
module has such an impact on a
dissertations.
The timescale for
degree classification. It is especially
doing so this year was very tight.
important not to revert to marks  English - In the case of
ending in -9 as a compromise
dissertations, which are double
between two markers rather than a
marked, the transparency of the
return to the marking criteria
marking process could be enhanced
by markers consistently providing
brief comments on why a particular

Processes




Staffing and Resources
 Modern Languages - For the new
Chinese, the College has got only
four MLM modules of the Chinese
language and studies. This is not
enough for the following years. To
mark has been agreed for those
dissertations where a class band
separates the marks awarded by
the two assessors.
Drama - The College should put in
FLC – The External suggests that
place a comprehensive training
'justifications' as well as answers for
programme for all users (staff and
the Reading Comprehensions are
students) of the new online
sent to the External in advance in
submission and marking system.
order to enhance the reliability of
the test.
FLC – The External advises the FLC
to better enable External Examiners  History – The External recommends
to provide their moderation by
that the length of dissertations be
reconsidering
such
a
short
extended so that it matches those of
turnaround (7 days) or at least for
comparable institutions.
dates to be given in advance as  Modern Languages – The practice
when the samples will be ready for
of anonymity was not observed at
examination.
the Board of Examiners which is out
of line with other institutions.
Modern Languages - In future any
potentially
contentious
or
problematic cases of oral exams
(e.g. where candidates have had to
start again, or required support in
some way) should be sent to the
Externals as a matter of course.
Modern Languages – 2 Externals
suggest that practice should be
further
scrutinised
regarding
permission given to final year
students to undertake Level 1
modules in their final year.
Archaeology - The online system for
reviewing work could be made more
user-friendly for the externals.
 Modern Languages - Given the
staffing levels and expertise in
Portuguese, there are very strong
foundations for expanding the
Portuguese provision.
keep the programme running, more
modules
are
needed
with
contributions
from
more
staff
members. The College should also
appoint staff members in Chinese
studies whose native language is
English. Without such teaching staff,
some aspects of Chinese studies
may not be well covered from British
or Western viewpoints and some
Chinese
language
learning
approaches may not be well focused
on English-speaking learners of
Chinese as a foreign language.
4
Partnership with the SWMTC
4.1
SWMTC run Theology programmes through a partnership with CHUMS. The External Examiner completed the same online form as that used by
all undergraduate External Examiners. Dr Hickman commended the team on the variety of written tasks across the programme, which were
wide-ranging and stimulating, enabling some impressive imaginative work to be produced and helped to encourage a successful balance of
academic rigour and spiritual reflection. The design of the programme produced some excellent applied theology from the students. Feedback,
particularly for those essays in which the students dealt with challenging ministerial practical situations, was noteworthy for offering both critique
and a high level of support. One Desirable recommendation was made suggesting that the University develop the criteria for the lower range of
the marking criteria in more detail (i.e. the 40% range and fail range), so it becomes clearer what it would take for an essay to fail. While
Feedback is helpful and encouraging the External would encourage the team to have consistency in the volume of feedback, particularly for
borderline cases and for first class marks.
Quality Review
January 2015
MAK/
N:\UFO & PGADD\QUALITY REVIEW\External Examiners\Overview\2013_14\UG\By_College\CHUM\CHUM_UG_Overview_13_14.doc