The external examiner
system at Cardiff University
Andy Lloyd, Clive Brown
Registry and Academic Services
21st April 2014
Welcome from Prof. Patricia Price
Pro Vice-Chancellor (Student Experience and Academic
Standards)
The external examiner system at Cardiff University
•
Presentation and discussion (11.00 – 12.30)
– Introduction to the external examiner system;
– The information you will receive to support
your role;
– The University’s Regulations;
– The evidence you should use to make
judgements;
– The reports you are requested to complete;
– Fees and payments.
•
Lunch (12.30 – 2.00)
– With an opportunity to meet colleagues from
academic schools;
•
Afternoon (2.00 onwards)
– As arranged with academic schools.
Points you wish this session to cover
• Understanding regs and process (Exam
Boards)
• Expectations of XX
• Guidance on referencing
• Timescales
• Moderation
• Nature and balance of the role
• Role of the Exam Board
External examining in UK Higher Education
• “The fundamental importance of
external examining to maintaining
academic standards and assuring and
enhancing quality”
• The indicator – “Higher education
providers ensure independent and
external participation in the
management of threshold academic
standards.”
• QAA (2011) UK Quality Code for
Higher Education, Part B: Assuring
and enhancing academic quality, B7:
External examining
[www.qaa.ac.uk/Publications/InformationAndGuidance/Documents/B7.pdf]
External examining at Cardiff University
• External examiners shall ensure that the Assessment
process is valid, reliable and explicit and provide informative
comment and recommendations upon whether or not:
– the University is maintaining the threshold academic standards set for its
awards in accordance with the frameworks for higher education
qualifications and applicable subject benchmark statements;
– the assessment process measures student achievement rigorously and fairly
against the intended outcomes of the Programmes(s) and is conducted in
line with Senate Regulations and related policies and guidance;
– the academic standards and the achievements of students are comparable
with those in other UK higher education institutions of which the external
examiners have experience.
The evidence base on which judgements can be made
Assessment
tasks / Exam
papers
Meetings
with
students
Programme
Information
Cardiff
Assessment
Regulations
External
Examiners
Samples of
student work
External
reference
points (e.g.
QAA / PSRBs)
Assessment
outcomes /
reports
The typical calendar of assessment
• September to March – Draft assessments / exam papers sent
to external examiners for approval;
• April to June – Samples of marked scripts / assessments given
to external examiners for moderation;
• June – Exam Board meetings;
• June to July – External examiners to submit annual report
form (within 1 month of exam board meeting);
• July to November – Institutional response issued where
required (and if needed, to be informed by the academic
school).
Information you will receive
• General information from the University
(through the External Examiners handbook)
– Background information about the University;
– Guidance on the role of external examiners in the assessment process;
– Assessment Regulations that cover marking, extenuating circumstances and
exam boards;
– Information on taught master's dissertations that will be made available to
external examiners;
– The reporting procedures (including guidance on the annual report form);
– Equality and Diversity matters (including the University’s Equality Plan,
information about specific provision regulations, and rules on the assessment
of students through the medium of Welsh, religious observance days etc.);
– Information on rates of fees and expenses payable to external examiners.
Information you will receive
•
Programme specific information from academic schools, including:
– A copy of the school brochure for the Programme(s);
– Information about the aims and learning outcomes of the programme, the methods of
assessment, and syllabus details of each module or unit of study;
– A copy of the marking scheme and details of school procedures that ensure the
reliability and consistency of marking;
– The final report of any retiring external examiner for the programme
(where available at the time of appointment), and;
– Details of opportunities to meet with students in the absence of staff
(either in person or by electronic means).
•
and in addition to the above, as they become available:
–
–
–
–
The date of the examining board(s);
Draft examination papers contributing to the final award for approval;
Arrangements for the approval of supplementary assessments, and;
Any proposed changes to the forms of assessment of any module or unit of
study contributing to the final award for comment.
Cardiff assessment Regulations
• “Where institutional audit reports have noted that institutions are
operating multiple sets of assessment and classification
arrangements, they have generally tended to recommend measures
to reduce the number of such arrangements, with a clear
preference that institutions adopt frameworks, or a single
framework, within which classification decisions can be taken more
consistently and fairly.” (QAA, 2006)
• Comments from external examiners that raised concerns over the
different rules being used within different schools;
• A desire to improve the student experience of assessment.
Consistency
Transparency
Equity
Cardiff assessment Regulations
• Changes are being made
through a University-wide
project (2010-14) that was
designed to:
– Enhance feedback to
students on assessment;
and
– Improve consistency in
assessment processes
and practices.
“the rationale for
differing approaches
to regulations has
become obscure”
Burgess Report
Assessment Changes at Cardiff
• A limited number of rule sets to manage taught modular degrees across
the University;
• The removal of ‘discretion’ from exam boards;
• Consistent rules to manage extenuating circumstances;
• Consistent rules and practice for:
–
–
–
–
late submission of coursework
Penalties for exceeding word limits
anonymous marking of all assessments (where possible)
second marking and moderation
• Reduction of the volume of assessment, and better management and
co-ordination of assessment within programmes of study, and;
• Development of a common, shared understanding of feedback on
assessments between staff and students.
External Examiners are not responsible for …
• Resolving disputes between internal
examiners or markers;
• Making recommendations on, or
changing the marks of, individual
students;
• The assessment of individual students
(except in the case of clinical and/or
practical-based assessments);
• Judgements related to extenuating
circumstances or unfair practice;
• Communication with individual
students
External Examiners and programme information
An information source that can be used to help shape
judgements on:
• the coherency and currency of the programme and its component
parts;
• the extent to which the programme reflects any additional
Professional, Statutory and Regulatory Body requirements.
• whether the programme is maintaining the threshold academic
standards set for its award in accordance with the frameworks for
higher education qualifications and applicable benchmark
statements;
• whether the academic standards and the achievements of students
are comparable with those in other UK higher education institutions
of which the External Examiner has experience.
External Examiners and setting assessments
• External examiners are required to consider, comment on, and approve
all assessments that contribute 50% or more to the mark for any
module or unit of study that contributes to the final Award. It is good
practice for schools to provide a response to external examiners on any
suggested amendments to assessments.
• When approving assessments external examiners should seek to ensure
that these:
–
–
–
–
are appropriate to the level of study;
will enable learners to demonstrate achievement of the learning outcomes;
cover the subject content appropriately; and
are accessible and fair, so that all learners have equal opportunity to
demonstrate achievement of the learning outcomes.
• Schools shall consult with external examiners on major changes to the
methods of assessment for any module or unit of study contributing to
the final award.
External examiners and marking and moderation
•
External examiners should be provided with a sample
of examination scripts, class test scripts, outcomes of
practical assessments and coursework to determine
that internal marking and classifications are of an
appropriate standard.
•
At least 10% of dissertations for a taught Master's
programme, or a minimum of 10 (whichever is the
higher figure)
•
You will be invited to consider and comment upon
marking and moderation procedures, to ensure the
reliability and consistency of marking and feedback.
•
External examiners may ask the board to review the
marks of whole cohorts or of individual assessments
External examiners and the confirmation of results
As well as reports on student achievements there will be a report to help you look at data
over the past 5 years at programme level.
External examiners and extenuating circumstances
•
A sub-committee will consider student requests to consider extenuating circumstances in advance of the
board and will determine if a student’s performance has been impacted by their circumstances or
otherwise. The sub-committee will present its outcomes to the board;
•
Exam boards allow students shall be given the opportunity to re-take failed assessments where they have
had extenuating circumstances (as a 1st attempt);
•
Where a student has passed modules impacted by extenuating circumstances, the exam board shall
remove the marks of these modules from a student’s final mark (up to 1/6 of the contributing credit)
where it benefits the student’s final award – ‘Discounting’;
•
Exam boards shall not:
– receive or discuss the details of individual extenuating circumstances cases,
– consider possible extenuating circumstances where these have not been reported through
appropriate channels,
– seek to review or overturn decisions made by extenuating circumstances groups.
•
To discuss personal circumstances at an exam board would risk compromising confidentiality and lead to
judgements being made on the basis of partial and/or incomplete information.
External examiners and meetings with students
•
External examiners and students are encouraged to meet (either in person or by
electronic means) to discuss the programme of study and its assessment. Such
meetings may lead to the identification of possible programme enhancements and
provide the external examiner with a rounded overview of the quality and
standard of the student experience and enhance the external examiner’s
contribution to the examining board. The annual reports of external examiners
often usefully include headline issues raised in such meetings.
•
Schools are expected to confirm to external examiners, at the start of the
academic year, the opportunities available to them to meet with students in the
absence of members of staff.
•
Such meetings should normally include the opportunity to meet with student
representatives in the later stages of the programme. Such students should be
well placed to provide informed views of the overall student experience.
The evidence base on which judgements can be made
Cardiff
Assessment
Regulations
External
reference
points (e.g.
QAA / PSRBs)
Assessment
outcomes /
reports
Programme
Information
External
Examiners
Samples of
student work
Assessment
tasks / Exam
papers
Meetings
with
students
External Examiners and any serious concerns
• External examiners have a right to raise any matter of serious concern
with the Vice-Chancellor, if necessary by means of a separate confidential
written report. The University will provide a considered and timely
response to any confidential report received, including details of any
action that will be taken as a result.
• Where an external examiner has a serious concern relating to systemic
failings with the academic standards and has exhausted all published
internal procedures, including the submission of a confidential report to
the Vice-Chancellor, he/she may invoke QAA’s concerns scheme or inform
any relevant professional, statutory or regulatory body.
External examiners and quality and standards
External examiner
reports
Academic Quality and
Standards Committee
Programme review
conducted by the
School
Annual Review and
Enhancement
Process
Student feedback
(including NSS
outcomes and
comments)
Action plans for
enhancement
Annual reports on taught programmes
• To be submitted within
one month of the
Examining Board;
• Consistent and transparent
approach regardless of the
size or level of the
programme.
The ways in which reports are considered and
used
• The University reviews the reports to identify:
– Enhancement issues that require a response from the University
(often informed by dialogue with the Schools);
– Particularly positive aspects of the reports.
• University Annual Report on Assessment
• Reports and institutional responses are published online
• Please avoid mentioning staff or students by name
2012/13
Matters for
Positive
further
comments
attention
Validity
Reliability
Explicitness
1.1 Style of Assessment
1.2 Word Limits
1.3 Feedback Opportunities and Enhancement
1.4 Over-assessment
1.5 Variety and appropriateness of assessments
2.1 Double Marking/Internal Moderation
2.2 Use of the Full Mark Range
2.3 Marks/Information Presented to Examining Boards
2.4 Analysis of Mark Trends/Comparative Statistics
2.5 Plagiarism / Citations
2.6 Handling of Extenuating Circumstances
2.7 Resolution of Marking Disputes
2.8 Time available to External Examiners
2.9 Opportunities for External to meet students
2.10 Consistency of marking/Mark Schemes
2.11 Rounding of Marks
2.12 Anonymity
2.13 Joint Hons. / Examining Board Discretion to Raise Classification
2.14 Alignment of Marking Scheme with Learning Outcomes
2.15 Group-work Assessment
2.16 Error on Papers
3.1 Sufficiency / Timing of Explicit Feedback
3.2 Information / Induction / Audit Trail Needs of External Examiners
3.3 Sampling Issues
3.4 Role of External Examiner (including moderation of exam papers)
33
8
1
4
17
32
21
16
9
7
6
4
5
12
6
1
2
4
9
1
3
29
28
13
6
12
0
1
0
20
8
6
3
0
0
1
0
1
3
4
0
0
5
1
2
0
60
29
0
5
Example (1)
Example (2)
Masters’ dissertations
• There is a separate, and briefer, Master’s
degree report form that is issued to external
examiners along with dissertations for
moderation.
• These are managed in the same way by the
University.
Fees and expenses
• Fees will be cleared for payment upon receipt of the
Annual Report Form
• Claims for expenses are processed upon receipt of the
Claim Form
• Payments are made by BACS processing (monthly) – the
arrangements are set up at the time of appointment
• Payment runs occur on the last working day of each month
Further information
• Online resources
http://learning.cf.ac.uk/quality/review/external-examiners/
• Clive Brown, Registry Officer
BrownC@Cardiff.ac.uk
Tel: +44 (0)29 2087 5249
• General enquiries
ExternalExaminers@Cardiff.ac.uk
Tel: +44 (0)29 2087 9222