[Secretary to complete: PAPER NUMBER]
[Secretary to complete: OPEN / CONFIDENTIAL]
Academic Board
QUALITY STANDARDS AND REGULATIONS COMMITTEE
8th March 2010
Annual Report on External Examining 2008/09
REPORT SUMMARY
This report provides statistical analysis of the annual reports submitted by external
examiners and advisers during the 2008-09 academic year, including details of the receipt of
Examiners’ reports and Faculties’ responses taken from the Quality, Standards, Review and
Enhancement team’s (QSRE) database records as at 25th January 2010.
The report identifies issues raised by externals (both positive and negative), particularly
where trends can be see across the University or within individual faculties.
The 5 external examiners who made negative comments to the report summary statements
in their reports is extremely low as a proportion of the total numbers of externals (484) (see
Appendix B, Table 3).
Recommendation
The committee is invited to receive the report and approve the recommendations contained
therein.
Appendices
Appendix A: Provides an overview of the appointment of External Examiners/Advisers and
Mentors
Appendix B: Provides information on the breakdown of External Examiner, Adviser and
Mentor appointments and the number of examiners’ and advisors’ reports and faculty
responses received during the 2008-09 academic year up to 25th January 2010.
Appendix C: Provides information from feedback received from External Examiners’
Development sessions.
AUTHOR
Name
Carol Blackall
Job title
Senior Officer, Quality, Standards, Review and Enhancement
Email address
c.blackall@leedsmet.ac.uk
‘Phone number
0113 8123919
533574400
APPROVAL OF REPORT
Approved by
Julie Hudson, Director of Planning and Registry Services
Approval date
15 February 2010
VC’s Group date
n/a
Annual Report on External Examining 2008/09
Introduction
1
The report provides analysis of the annual reports submitted by external examiners,
advisers and mentors during the 2008-09 academic year. An overview of a range of
activities undertaken at University level in relation to the appointment of external
examiners covering the same period is also included at Appendix A.
2
The report identifies issues raised by externals (both positive and negative).
3
Human and financial resources are required across the University, within Faculties
and the RSO to support the appointment of and activities related to the
development of external examiners and the performance of their duties.
4
External examiners are an integral and significant part of all UK universities’
frameworks for the assurance and assessment of academic standards and external
examining provides one of the principal means for maintaining nationally
comparable standards within autonomous higher education institutions. The main
purpose of this report is to provide evidence to the University of the effectiveness of
its external examining system and its ability to provide an independent guarantee of
the standard of the awards of the University.
Overview of reports submitted
5
The external examiner report form contains a summary section for internal use
which provides an immediate overview and immediately identifies any serious
matters of concern.
6
In the summary section, externals are asked to respond with yes/no answers to
questions on standards set, student performance and the conduct of processes.
Externals are also asked to identify areas for commendation in this section.
7
The main report requires examiners to provide a qualitative written response under
nine headings, including the following:
a) the operation and conduct of the Board of Examiners (and any mitigation Panel or
Examination Committee attended);
b) the action, if any was required, taken in response to comments in the report of
last year (not relevant for those examining for the first time);
c) the overall performance of the students in relation to that of comparable levels of
work in other institutions;
d) the curriculum, teaching or resourcing of the programme of study as indicated by
the performance of students in the assessment.
8
Externals in the main submitted their reports electronically with only a couple of
exceptions. Examiners were sent an electronic receipt for their reports which stated
the date by which they should expect to receive a response from the faculty within
60 working days from receipt of the report by the QSRE team. This response date
was also given to the faculty when each examiner’s report was forwarded to them
by the QSRE team for circulation. These dates are monitored by the QSRE team on
the external examiner database.
9
Externals associated with the University’s Edexcel provision were asked to complete
an additional section of the proforma, which asked the following:
a) any major issues with regard to the BTEC programme(s) with which you are
associated;
b) areas of good practice you have identified specifically relating to BTEC
programme(s) with which you are associated;
c) any other comments that you wish to make that are related specifically to the
BTEC programme(s) with which you are associated.
10
These questions enable the University to capture the data requested by Edexcel for
its Annual review of Edexcel licensed centre BTEC programmes. The University’s
Annual Review Report for 2008-09 is due to be submitted to the Edexcel University
Chief Examiner by the 31st March 2010. Prior to that the report will be presented to
the External Examiner Sub-Committee to be held on 4th March 2010.
11
External advisers were required to complete a different report form, which included
the same summary section as for external examiners. However, the main report
covered a narrower range of issues relevant to the adviser’s role of assisting
external examiners in a particular subject area. The majority of the university’s
external advisers work on the Languages for International Communication Scheme
within the Leslie Silver International Faculty.
12
Mentors are required to complete a report describing the support they have
provided for the mentee and how the mentoring arrangement has worked.
13
As of 25th January 2010, a total of 390 external examiner reports have been received
and considered in compiling this report. A total of 40 reports remain outstanding
for 2008/09. In some cases, there are legitimate reasons for the later submission of
these reports, for example Boards of Examiners which meet outside of the standard
undergraduate assessment timetable. However a number of externals who should
have submitted their reports have failed to do so despite the sanction of
withholding their fees and expenses. The QSRE team sends a reminder letter to all
externals who have not submitted their reports.
14
As in previous years, there is variety in the length, focus and quality of external
examiner reports received. However, the majority of externals completed their
reports in detail and answered all questions fully. The QSRE team is considering the
provision of an exemplar report to assist examiners in their reporting.
15
Each annual report received will be considered within schools/faculties and has
been scrutinised by the QSRE team. A small number of notable issues were raised by
examiners across three faculties, as follows:
Faculty of Health
16
One examiner commented that in some cases a lack of understanding of
confidentially requirements within students’ work was an issue. The faculty has
responded to this issue and reported that a clear and consistent message will be
given to students about its high importance.
17
One examiner commented that there had been spelling errors and grammatical
errors in the proposed assessments to be set and that were sent to the examiner to
review and suggested that in future they be spell-checked and proof read prior to
being submitted to the examiner.
18
One examiner gave a no response to student performance at Level 3 and
commented that students at Level 3 should be aware of how to reference and
several in the sample sent to the examiner did not. Texts that were referenced
were very basic and not appropriate to the level. Some students were unable to
write an essay so what was presented was more like a set of notes. The examiner
commented that the overall standard at Level 3 was below what she would expect
from them in terms of critical analysis. The Dean will give an update on action taken
regarding this matter.
Faculty of Business and Law
19
Some external examiners within the Faculty of Business and Law expressed concerns
about administrative and communication issues connected with the assessment
process, for example the lateness of the submission to them of work to moderate.
The Dean will give an update on action taken regarding this matter.
20
Three externals commented on the high numbers of students seeking and gaining
mitigation and their being able to choose the highest mark achieved following
having been given mitigation and that this process encourages “insurance
migitation” by students, particularly in examinations. The Faculty have noted these
comments and staff within the faculty share these concerns and the issue will be
discussed at Faculty Board.
21
One examiner commented on the short period of time between students being
notified of being unsuccessful and their ability to re-sit creating pressure on both
staff and students and the Faculty is aware of this and is reviewing the process for
the next academic year.
22
One external examiner commented that the faculty’s PACE module is beginning to
have an impact and is improving student performance.
Innovation North Faculty of Information and Technology (RUN provision)
23
One examiner responded no to student performance and stated that he thought
that students lack some of the skills that many of the students who get into the
major drama schools have and stated that it was evident in both performance and
their understanding of the process. The course team have responded to these
comments by stating that the examiner appears to be measuring the FD with
national acting conservatoires. The course team will clarify these issues further with
the external examiner.
24
One examiner expressed no in all of the summary statements stating that he was
unable to confirm the standards on the programme due to a lack of student work or
information. The standards on the professional development and practice modules
appear to be inconsistent with a lack of justification from the assessor as to the
rationale for different marks being awarded. He also stated that there were two
different modules on the programme that have identical module handbooks and
identical forms of assessment. The examiner also stated that student performance
was extremely low on the programme, particularly at Level 2. In one instance all of
the students failed a module. On a different module only one student passed the
module with 40%. The examiner also commented that there was no evidence of the
development of scientific skills of data collection, analysis and interpretation. The
examiner also commented that there was a lack of information upon which to base
his judgements on the conduct of processes as he had not been provided with any
student work (Psychology module), no examination question papers (Exercise
Physiology) and unclear information in the module handbook regarding assessment
(PDP). He also commented that there was no moderation or second marking on the
programme apart from one Level 4 nutrition module.
25
Following receipt of the report the College concerned had arranged a meeting with
the external examiner to address the issues directly with him and to propose
solutions to the ongoing issues. The Dean will give a report on action taken
regarding this matter.
26
One examiner had placed no in all of the summary statements stating that the work
sampled was generally over-marked, and adjustments had to be made during the
moderation process. He also commented that with only one student completing in
the academic year it was difficult to draw general conclusions, but the work seen
was of a lower standard than the work of other students nationally in the final year
of a BA degree. He also commented that the student sampled had not met the
stated requirements of the modules, failing to hand in a significant component in
several cases. This had been overlooked during the assessment process and had to
be adjusted through moderation.
27
All of the issues raised by the examiner have been addressed by the course team in
their response to the examiner’s report. The Dean will give a summary review of
the outcome of this matter.
Faculty responses to external examiner reports
27
As outlined in the University’s Academic Principles and Regulations, it is the
responsibility of a Senior Academic Manager (nominated by the Dean of Faculty) to
provide external examiners with a written response to their comments and
recommendations within 60 working days of receipt of their report. To this end,
Faculty External Examiner administrators have been asked to provide the QSRE team
with copies of all responses sent to external examiners. The receipt of Faculty
responses is monitored by the QSRE team as part of the continuous audit process.
The number of Faculty responses recorded as received on the QSRE team’s database
as at 25th January 2010 is included in Appendix B, Table 2 and is comparative to
previous years at this point in the academic year.
28
Of the Faculty responses to reports that have been submitted the issues raised by
external examiners are addressed in the main by faculties with the identification of
proposed solutions being made. When examiners send in their reports next
academic year it will be seen whether those issues were in fact resolved.
29
Of those external examiner reports so far submitted all but two have stated that
their issues from their previous year’s report were addressed and in the two where
they were not addressed the faculties have given the reasons to the examiners why
this was the case and the examiners were content with those reasons.
30
Deans have been asked to report on reasons for responses to external examiner
reports not being sent within the deadlines.
Analysis of external examiner/adviser reports
Overall
31
As in previous years, the overall tone of most external examiners’ reports was
positive. External examiners are asked to comment on areas of commendation and
a number of areas of good practice emerge from this (see paragraph 55)
32
The number of external examiners who made negative comments in their report
summaries was only five and this number is extremely low as a proportion of the
total number of externals (440). In two of these cases the reasons for the negative
comments given were:

One external examiner had retired during his tenure and therefore did not
feel able to comment on the student performance being comparable with
similar programmes or subjects in other UK institutions with which he was
familiar.

Another examiner was unable to agree that the student performance during
the academic year 2008/09 was comparable to other UK institutions due to a
number of issues he had raised with the course team. However he did say
that for the academic year 2009/10 the faculty have reviewed the programme
and put in place many improvements which would address the issues raised
and lead to a more appropriate level of student performance.

Please also see paragraphs 23, 24 and 25 above.
Conduct of Process
33
Examiners commented on the excellent administration of processes by
administration teams and the good liaison with examiners by course teams and the
university generally. Two examiners commented on the process of the reading out
of all levels of students’ results where students were being given their final awards.
The previous level marks had already been through a Board of Examiners’ process
and the examiners commented that this re-reading of results appeared to prolong
the board unnecessarily.
34
Reliable, conscientious, thoughtful and constructive marking and moderation and
the provision of relevant information to examiners in a timely manner was also
highlighted by examiners. Comments were made by 5 examiners on the late
notification of the dates of boards of examiners and late changes to the dates,
which then precluded them from attending.
35
A number of examiners across all faculties and the RUN stated that they had been
informed of the date of the board of examiners rather late and had been sent work
for moderation very late in the cycle. Faculties are asked to ensure that examiners
are contacted as early as possible to inform them of the dates of the boards of
examiners and to keep examiners informed of any changes to dates and venues.
Faculties are also asked to ensure that samples for moderation are sent in a timely
fashion and are not bunched towards the end of the academic year if possible to
allow examiners to give timely consideration to samples.
Standards set
36
All but two examiners commented that the standards set were appropriate. In both
cases the Faculty had addressed the issues raised by the examiners. Please also see
paragraphs 24, 25, 26 and 27 above.
37
A number of examiners stated that they thought that the Unfair Practice Board
process had taken away the decision making process from the Boards of Examiners
and did not feel that this was helpful to students. It was also commented that
External Examiners had not been consulted about this change in process and that
guidelines should be given to Faculties about the process and the level of discussion
expected at Boards of Examiners.
38
Another examiner commented that the plagiarism penalties appeared overgenerous and that it was somehow inappropriate for a student found guilty to
resubmit without capping of the mark, particularly as students who fail at the first
attempt are not given the same opportunity. This comment has been highlighted
for discussion during the forthcoming review of the regulations.
39
Two examiners commented that they deemed the 40% pass mark for Masters’
inadequate and that it was detrimental to the academic system.
40
One examiner also commended the commitment to maintaining robust quality
systems at both faculty and institutional level.
Data systems issues
41
At one RUN college issues with the data systems led to the incorrect calculation of
marks presented to the Board of Examiners. The college has since addressed these
issues in their response to the two examiners who raised the issue.
42
At one university board of examiners there appeared to be confusion as to the
presentation of the data, where the University’s guidance was apparently at odds
with the actual presentation of data on the weighted averages. The examiner did
comment however that the Board took time and considerable care to fully
understand this and ensured that no students were disadvantaged in their final
classification. The QSRE team will discuss this issue more fully with the Faculty and
the Banner team to identify the issues.
Performance of students
43
All but two externals felt that the work of students was comparable to that of other
institutions with which they are familiar and in most cases student performance was
considered to be of a very high calibre. In both cases the Faculty had addressed the
issues raised by the examiners. Please also see paragraphs 24, 25, 26 and 27 above.
44
As in previous years, external examiners reported on a wide range of strengths and
weaknesses in students’ work.
45
A number of externals were concerned about students in terms of their spoken and
written English language proficiency, i.e. spelling, grammar, punctuation, the use of
the first person and the structure of their written assessments. It was also
commented by many examiners that some students had a poor understanding of
referencing and the production of bibliographies. Some examiners commented on
students’ over-reliance on web-based materials and articles rather than books and
journals. In all instances where these remarks were made the examiners stated that
this was a problem nationally and not particularly related to Leeds Met students.
46
Comments were also made that weaker students have difficulties in relating theory
to practice, as well as having poorly developed analytical skills. It is notable that
these issues are raised by externals annually and faculties are asked to consider
these issues in more detail.
47
Many examiners commented on the high level of employability skills demonstrated
by students on programmes across all faculties.
48
Many examiners commented very favourably on the use of the Virtual Learning
Environment for both assessment and feedback but did make comments on their
inability to access the VLE. The QSRE team via the University External Examiner subcommittee is seeking to address the issue of externals’ access to the VLE.
49
The areas for improvement most regularly raised are the use of referencing by
students and the integration of theory and practice in vocationally-orientated
courses. It is recommended that scheme/course teams consider these issues in
more detail.
50
Some examiners commented that the introduction of the student representation
system has improved communication with cohorts of students.
51
It was further noted that a large number of students were given mitigation long
after the deadline for the submission of mitigation had passed. It was suggested
that the university should seek to eliminate this practice by being firmer on the
deadlines for submission.
Action taken on previous comments
52
External examiners are asked in their reports to comment on the action that was
taken (if any was required) in response to their report of the previous year. Those
external examiners appointed during the 2008-09 academic year were unable to
comment on this issue in their first year of appointment. Whilst not all external
examiners’ recommendations can necessarily be implemented either in full, or
immediately, faculties should give consideration to these issues during the annual
review cycle.
53
Most examiners reported positively that their previous suggestions for
improvements had been properly considered and appropriate action had been
taken.
54
Recommendations for action to be taken in response to external examiner reports
are identified as part of the University’s Annual Review process. It is recommended
to faculties that if it is not possible to address all of the issues raised by examiners
that a reason why this is the case be included when responding to the examiner
rather than no response at all being given.
Assessment
55
In line with previous years, many external examiners reported positively on the
range and variety of assessment methods employed and the innovative approaches
taken by many course teams. The quality of feedback provided and support offered
to students was also commended by virtually all externals. There were instances
where inconsistency in the format of feedback to students required addressing and
it is recommended that faculties consider solutions to this quality enhancement
issue during the annual review cycle.
Good practice
56
The majority of external examiners commented on areas of good practice they had
identified from their involvement with the assessment process. Common themes
that emerged included:
a) Well run and rigorous Boards of Examiner processes.
b) Varied and innovative assessment methods which allow students to demonstrate
a range of skills and knowledge. These include the use of group work and nontraditional assessment methods;
c) High quality, detailed and transparent feedback provided to students, including
formative feedback which informs how students approach subsequent work;
d) The commitment, enthusiasm and professionalism of academic and support staff;
e) Rigorous and reliable moderation and marking, including clear evidence of second
marking;
f)
Good academic and pastoral support provided for students. Tutors having a good
knowledge of their students and their individual progress;
g) An emphasis on reflective and creative thinking in particular areas of study;
h) The establishment of external links and external guidance and support that staff
have provided for students on a range of courses;
i)
Marking criteria are on the whole clear and easy to follow.
j)
Engagement with local employers is particularly commended.
Mentor reports
57
Mentors commented on the mentor process working well and highlighted it as
excellent practice.
58
One mentor suggested that the university should provide a more prescribed
indication of roles and tasks that are likely to need the support of a mentor.
Recommendations
59
Committee members are asked to endorse the following recommendations:
a) that faculties ensure when it is not possible to address all of the issues raised by
examiners in their reports, they provide a reason why this is the case when
responding to the examiner;
b) that QSRE closely monitors the receipt of examiners’ reports and writes to
examiners to ask for their reports in a more timely fashion where the deadline has
been exceeded;
c) that QSRE closely monitors the receipt of faculty responses to reports to ensure
compliance with the 60 day response deadline by providing a monthly report to
Deans and Chairs of Faculty Quality and Standards Committees;
d) that QSRE provides a report to the Quality Enhancement Committee on any
quality enhancement issues raised by external examiners.
e) that our university extends its institutional oversight of the arrangements for and
management of the external examining process. It is proposed therefore to
broaden the remit and terms of reference of the University External Examiners’
Sub-Committee to enable that group to undertake this task, formally reporting its
findings to Academic Board.
Conclusion
60
The Committee is invited to receive the report and approve the recommendations.