External Examiner Comments & Course Organiser Responses, 2013/14
Course
Professional & Clinical Skills 1-4 & GEP
External Comments
No major changes to the examination are
suggested as overall the quality and standard of
the assessment has been very good. The
following minor comments are put forward for
consideration by the course team:

The careful consideration by the Boards
of examiners of decisions made relating
to incomplete Portfolios is to be
commended. The decisions made on
actions (e.g. email for minor corrections,
Portfolio fails) are consistent and
transparent. As discussed at the Board
of examiners meeting, it will be valuable
to document these decisions so that
they can be referred to and applied in
future years to ensure equivalent
standards are maintained.

Many of the discussion points across all
years related to use of correct templates
and the inclusion of complete and
correct documentation. Clear efforts
have been made to communicate these
requirements to students and the selfcertification check list for each year is an
example of good practice. It is essential
to continue to emphasise to students
the critical importance of these
elements and the likely consequences of
failure
to
meet
the
Portfolio
requirements.
The dose calculation
Course Organiser Response
Overall the external examiner is happy. She
supports the plan to continue to improve
student guidance regarding portfolio
submissions.
We have introduced questions relating to
feedback in the post course questionnaires and
have had some frustrating poor responses.
Almost all of the feedback on this course is
immediate, but the students still rated us poorly
on the question about whether the feedback has
been prompt. For next year we will change the
wording of these questions so that it’s not just
related to the in course assessment and we will
include some information about this in the
introductory lectures for all year groups.
1
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
printouts were an issue in each year,
again making students aware that there
is scope for mistakes to be made here
may help minimise the number of
students making these mistakes in
future years.

Professional Development
The Portfolio review period prior to the
Board of examiners meetings for PCS 1-3
& GEP could be extended by adjusting
the timings of this meeting. This would
facilitate the review of a broader range
of 1st-3rd year Portfolios by the external
examiner in future years.
No major changes to the examination are All the comments are fair.
suggested as overall the quality and standard of
the assessment has been very good. The
following minor comments are put forward for
consideration by the course team:
1. There was discussion of the challenges
of assessing course attendance when an
online or blended approach to learning
is adopted. In particular, managing the
inevitable technical problems which
occur from time to time. A clear
protocol for managing these situations
(including student actions) would help to
clarify
the
situation
regarding
attendance and facilitate confident
application of the course attendance
2
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
requirements.
Course Organiser Response
2. The consequences of failing the course
for students who have elected to join
the course (rather than compulsory
attendance) need to be clarified – the
course team is aware of this potential
issue which did not affect any students
in this cohort.
AHWFS 1
Animal Body 1
Review of the written papers demonstrated a
generally reasonable level of understanding by
students, although there were some clear gaps
in knowledge for some candidates in certain
parts of the curriculum. In light of my comments
above, I was pleased to hear at the Examination
Board that any MQCs receiving a high
percentage of correct answers will be reviewed
for future use in the exam.
I share the external examiner’s concerns
regarding the relatively high proportion of
‘straight forward’ MCQ questions and this will be
addressed in future assessments.
Otherwise pleased with the very positive
external examiner’s comments.
External Examiner 1
CO response not returned.
I was satisfied with the assessment in terms of
its
content
and
reliability.
My
comments/suggestions are as follows:
• Model answers were not always provided
with questions when papers were sent to
externals for evaluation prior to the written
examinations. This makes it difficult to provide
constructive comments on the strengths /
weaknesses of the question. The amount of time
provided for scrutinising questions was
3
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
sometimes rather short. Feedback would be
welcomed in terms of whether suggested
changes have been incorporated (or not) in the
final version of the papers. In the Interpretation
paper a formatting error was picked up but not
corrected and in the MCQ, a distractor that was
considered correct was not modified.
• Some logistical issues were brought to the
Examinations Board’s attention in terms of the
written assessment (cold room, incorrect
information provided to students) that were
outside the control of the vet school. This
potentially impacted on the performance of a
number of students, which needed to be taken
into consideration. It should be stressed to the
University that deficiencies in facilities and
invigilator performance can compromise a highstakes examination such as AB1.
• Annotation on scripts. I would like to
encourage internal examiners to write
constructive comments on scripts as this helps
external examiners to appreciate how that
individual has arrived at their mark and also
serves as formative feedback to students who
choose to review their scripts / performance in
the examination (particularly those failing
students).
• The external examiners all agreed that the
viva voce element of the in-course assessment
was very useful. In particular, this format
allowed assessment of communication skills,
which is very important for this particular
4
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
course. The performance of some of the
students was exemplary.
• The new system provides external examiners
with an opportunity for oversight of quality and
process of the assessment, with scrutiny of a
range of scripts, rather than focusing on
borderline and failing students. We were happy
that the latter role could be undertaken
internally (e.g. by Chair of Examination Board
and Course Director).
• The provision of extensive statistical analysis
of the performance of the assessment and of
individual questions is to be highly commended.
This information could be sent to externals in
advance, so that more time could be spent
scrutinising scripts on the morning preceding the
Examination Board.
• Statistical analysis of the Short Answer paper
identified one question (10) that was an outlier.
This was found to be a relatively easy question
that was not particularly discriminatory. Analysis
of the Interpretation paper indicated this to be
reliable, although one question (3) was
particularly challenging, and could possibly have
been simplified. The Cronbachs Alpha indicated
that the Spot paper was likely to be
oversampled, although there are logistical
reasons why this should not be reduced below
15 questions. In contrast, the MCQ has a
relatively low KR20, which is likely due to
undersampling. More MCQs should be included
in the paper and staff should be encouraged to
5
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
submit more questions. We did look carefully at
those questions that did not seem to perform
well, based on their Speedwell scores (2, 8, 9, 11,
20) but there were no obvious reasons to
exclude these. Faculty could use this information
to identify areas of deficiency/misconception
that might need to be emphasised in the course.
• There was a considerably amount of
discussion regarding the use of standard setting
(Hofstee) to set the pass mark. My personal
opinion is that this method is more objective and
defensible, compared to using an arbitrary value
of 50%. It is clear that there are still some issues
surrounding implementation of this method and
the potential for anomalies when applied to a
spreadsheet, but the School should be
commended for “grasping the nettle” and being
pro-active in addressing this issue. Whether
standard setting should be applied to
Pass/Credit and Credit/Distinction boundaries
also remains to be resolved.
External Examiner 2
Very impressive to see the adoption of
Speedwell and Hofstee to such good effect.
External Examiner 3
I am fully supportive of the new role of the
external examiner in your assessment process.
Though I fully buy in to standard setting, I was
initially dubious about the method of standard
setting (mainly through unfamiliarity with
6
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
Hofstee), but having had the discussions at the
exam board, and having read more literature
about it, I am now convinced it is a good way of
doing things.
Generally speaking I really like the Edinburgh
way of doing things. I would recommend that
the recent change to the role of the external
examiner is kept.
External Examiner 4
Consideration should be given to the amount of
material covered by this exam given its timing so
early in the first year – or consider holding the
exam later in the academic year.
Animal Body 2
External Examiner 1
I was present at the AB1 and AB3 ‘Board of
Examiners’ and for those modules a pilot
standard setting system using the Hofstee
method was described. I assume that this will
apply to all modules i.e. AB2 and AB4 as well. I
was very impressed with this approach to
standard setting and fully support it, and hope
that the pilot ensures it will soon be applied
officially.
The external examiners were supportive of the
course and assessment in general but made
some useful suggestions including more careful
analysis of short and interpretation questions to
ensure they have the appropriate level of
discrimination. The adding up of final marks etc
should be done well in advance of the exam
board meeting so that the appropriate
‘borderline’ students can be identified and
scripts re-read if necessary.
External Examiner 2
In general, I was satisfied with the assessment in
terms of its content and implementation. My
suggestions are as follows:
 There was adequate time provided for
7
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
us to review the papers, prior to the
written examination. Where comments
had been made, changes were generally
implemented in the final version, but we
were not necessarily told that this had
been done. I would recommend that the
papers be circulated to the external
examiners once these have been
finalised. A decision not to implement
changes should be justified.
 It was reported that there was an issue
with print copy for a number of students
during the exam and that subsequently
some students had black and white
images rather than colour. It is
important that papers are adequately
proof-read and checked in advance, as
this situation impacts on the student
perception of the credibility of the
assessment.
 There was a great deal of variability in
the comments made and marks
indicated on the scripts from different
examiners. There should be consistency
in this approach and I would encourage
all examiners to write on scripts to
justify the marks allocated and to
provide formative feedback to students
who might wish to view their scripts
(particularly failing students) at a later
date.
 The marking strategy seems to have
8
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
been to allocate a proportion of scripts
to each internal examiner, who marked
the full set of questions for those
individuals. This is likely to introduce
individual variation and bias into the
assessment and reduce reliability.
Although we did not identify major
inconsistencies in the marking across the
piece, it would be better practice for one
internal examiner to mark each question
for the whole cohort, or to allocate two
examiners who liaise with each other for
consistency of marking that particular
question. There were some minor
inconsistencies where marks were
allocated by one individual but not
another, but these did not impact
substantially on the final grade.
 There was some overlap between
internal and external processes on the
morning preceding the Exam Board. This
led to problems, when an anomaly was
identified in the spreadsheet of marks,
which moved a significant number of
students
down
across
the
distinction/credit/pass/fail boundaries.
Therefore, the sample that we had
identified to evaluate in these various
categories was no longer valid, but there
was insufficient time available to
address this issue. I would recommend
that all internal processes and
9
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
procedures be completed the day before
the Exam Board at the very latest, so
that the external examiner process is not
compromised.
Application of the Hofstee method for setting
the pass mark was recommended and approved
by the Board on this occasion, which was to the
benefit of a significant number of students
achieving <50%, who would have otherwise
failed the examination. A decision needs to be
made as to whether this standard setting
method is to be implemented across the course,
and what parameters are to be used, so that
there is more consistency between Exam Boards.
External Examiner 3
The students are fortunate to have exposure to
the expertise of so many eminent researchfocussed staff. My impression from outside is
that this course could be improved with a little
more input from a specialist teacher, to ensure
that content is fully appropriate and delivered in
the best way and is completely aligned to the
school’s educational mission.
External Examiner 4
While the exam is, in overall terms,
comprehensive and robust, there are areas of
concern, including the combining in-course
assessment marks (which tend to be more
generous) with marks gained under exam
10
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
conditions. There are also aspects of the
Interpretation Paper questions that would
warrant consideration (see above). It would be
more helpful if internal checking of results
spreadsheets could be completed before the
external examiners attend.
Further, the current procedures mean that
although external examiners are given the
opportunity to comment on draft exam papers,
the impact of those comments on the final paper
is not given. For good feedback to external
examiners, it would be helpful if responses
could be given to the external examiners, and
they were sent the final paper, before they
arrive to review exam scripts.
A further comment, reflecting on the balances of
AB1-4 exams overall is that the level of detail
required for the AB1 and AB2 exams differs from
that for the AB3 and AB4 exams – AB1 and AB2
appear to focus more on detail while AB3 and
AB4 focus more on concepts (but with some
detail and depth). It may be appropriate for
Chairs of the four exam boards to discuss this
difference and whether the extent of the
difference is appropriate.
11
External Examiner Comments & Course Organiser Responses, 2013/14
Course
Animal Body 3
External Comments
External Examiner 1
Some thought should be given to the weighting
of the in-course assessment. Use of ‘qualifiers’
should be considered, for example ‘students
must gain an overall pass mark for the written
part of the examination’. Although Hofstee
standard setting was not applied on this
occasion, this would have failed two borderline
passing students in Module 2. The feeling of
external examiners was that these two students
were weak, with significant gaps in their
knowledge and understanding. These individuals
might have benefited in the longer term from
revisiting the material in preparation for re-sit
examination, before advancing to the next stage
of the course. My personal opinion is that, on
this occasion, applying the Hofstee method
would have been defensible. Having said that,
we were happy with the overall standard and
performance of the assessment.
Course Organiser Response
Overall external examiners were impressed
with the diversity of methods used for
assessment as well as with the performance
of the students. Three external examiners
commented on whether the use of Hofstee
analyses was appropriate for all cases and on
the effect of in-course assessment marks on
overall course marks.
External Examiner 2
Exam conducted fairly. Discussion of Hofstee
required, since it seems to generate some
concern.
External Examiner 3
The role of the external examiner has recently
changed. Instead of external examiners making
judgements on individual students, their role
now involves focusing on the overall statistics of
the assessment. A huge amount of information
12
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
regarding the students’ performance in all
question types was provided and these support
the fact that the assessment was fair and
appropriate. A random selection of scripts were
analysed to ensure marking was appropriate and
fair. In particular the method for standard
setting chosen is the Hofstee method which is a
very accurate and fair way of setting a pass mark
for this kind of examination. At the moment this
is being piloted, but all the evidence so far has
confirmed that it is suitable.
I have no suggestions for change and would like
to commend all staff involved in the teaching of
this part of the programme, from delivery and
student support to conduct of assessment and
exam boards. In particular the involvement of
the externals in this process is highly rewarding.
External Examiner 4
•
Please comment on the conduct of the
Board of Examiners meeting
Excellent. The Exam Board then reviewed each
aspect/component of the exam in a thorough
manner.
For module 1, the KR20 value was low for the
MCQ component although it was acknowledged
that there were only 27 questions in that paper
so the statistics may not be that reliable. Many
of the questions showed good discriminatory
power. However, there were several questions
13
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
were one of the distracters was not selected by
any student, again raising the issue of the need
for four distracters rather than three good ones
(see comments on External Examiner’s Report
for AB1). Other questions (Q7, 8, 15, 19) were
essentially non-discriminatory (too easy) and rewording these questions should be considered.
Course Organiser Response
Very few of the interpretation questions in that
exam paper were truly interpretation questions.
These are difficult questions to set – Q2 on that
paper is a good “model” to follow in this respect.
There may be a case for further staff training in
setting good interpretation questions.
It was noted that the mean mark for the short
answer questions in module 2 was 3.9/10, ie:
below 50% whereas all other components of the
exam had a mean class mark of over 50%. These
questions may need to be amended for future
use. In contrast, the interpretation paper had
high mean marks for all questions. The MCQ
statistics for module 2 showed impressive
profiles, all having a p value of >0.4 and many
with good discriminatory power. Nevertheless it
was noted that if the MCQ pass mark stays at
50/100 then the MCQ component of the
combined exam is likely to push the Hofstee
mark upwards.
Applying Hofstee to the raw marks raised the
pass mark of both module 1 (from 50 to 54.1)
14
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
and module 2 (from 50 to 53.6). For module 1,
there was one student who failed the exam
bothe pre- and post- application of Hofstee. In
the case of module 2, this meant that 2 students
(who gained over 50/100) would have failed the
exam. The papers of those two students had
been scrutinised by the external examiners
before the Exam Board meeting; there was a
clear difference between those two students
and the students with the next lowest module
marks – suggesting that applying Hofstee would
be relevant.
Course Organiser Response
It was also noted that the overall exam mean
mark was above the mark at which a Distinction
is awarded. Marks were, in most cases, “raised
up” by the in-course assessment and MCQ
marks; the standard of responses in the short
answer and interpretation papers did NOT
reflect this very high overall class mean mark. If
the in-course assessment had been removed
from the exam marks, then 4 students would
have failed the December exam and 7 would
have failed the March exam. Thus there is an
overall concern for this year’s marks, especially
as 66 of the 113 candidates achieved a
Distinction. This is in sharp contrast to the
number of distinctions awarded in other Animal
Biology exams – there is a case for applying
Hofstee principles to the Distinction boundary?
15
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
•
Please summarise your suggestions for
change and your general comments
Course Organiser Response
See above. While the exam is, in overall terms,
comprehensive and robust, there are areas of
concern, including the combining in-course
assessment marks with marks gained under
exam conditions. There are also aspects of the
Interpretation Paper questions that would
warrant consideration. Further consideration of
the introduction of Hofstee and a clear
explanation of the procedure to students is
required (eg: they gain an overall mark of
greater than 50/100 but fail the exam once
Hofstee is applied). Consideration should also be
given to applying Hofstee to other grade
boundaries, not just the pass/fail mark.
Animal Body 4
External Examiner 1
I was present at the AB1 and AB3 ‘Board of
Examiners’ and for those modules a pilot
standard setting system using the Hofstee
method was described. I assume that this will
apply to all modules i.e. AB2 and AB4 as well. I
was very impressed with this approach to
standard setting and fully support it, and hope
that the pilot ensures it will soon be applied
officially.
External Examiner 2
In general, I was very happy with the assessment
in terms of its content, implementation and
outcome. My comments/suggestions are as
External examiners’ comments were generally
very positive, though did reflect several of the
concerns of the teaching staff such as an
overreliance on external examiners for the viva
and a large number of distinction marks. The
viva and interpretation papers were felt to be
valuable although there were some helpful
suggestions as to how the latter may be
improved further to test the students’
interpretative and problem-solving skills rather
than their recall of specific facts from the
problem sets. An open-book format stretching
over 2 days was suggested – however, this may
present logistical challenges particularly given
16
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
follows:
 The external examiners all agreed that
the viva voce element of the
examination was very relevant to this
module. In particular, this allowed
assessment of students’ ability to
integrate different concepts and
principles both from an anatomical and
physiological perspective, as well as
evaluating communication skills. There
was a suggestion last year that the viva
voce examination would be extended
from 15 to 20 minutes, which was
endorsed by the external examiners.
However, this was not possible this year
due to a deficiency of internal examiners
willing/able to participate. There is some
concern regarding the logistics of
running this part of the assessment, with
heavy reliance on participation from
external examiners. There should be full
engagement of staff who teach on this
course in the assessment process.
 This year, there was a marked
improvement in terms of internal
examiners
writing
constructive
comments on scripts, allowing external
examiners to appreciate how that
individual arrived at their mark. This was
particularly noticeable for the essay
questions,
where
very
detailed
comments were provided. This is also
Course Organiser Response
that we are hoping to reduce, not increase, the
total number of assessment hours for the
course.
17
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
good practice as it serves as formative
feedback to students who choose to
review their scripts / performance in the
examination (particularly those failing
students).
 There was some discussion as to the
merits of having a relatively small
number of essay questions of a very
broad nature as part of this assessment.
Given the difficulties in ensuring
reliability of marking of this type of
question, with increasing student
numbers and reliance on two members
of staff, it is reasonable to reflect on
their value as an assessment tool. The
discussion focused on consideration of
replacing this part of the assessment
with one that might better align with the
nature of the AB4 course. In this respect,
consideration could be given to
introducing an ‘assessed problem set’
that is either carried out under open
book conditions, or whereby the
students receive the assessment in two
parts, with time in between to source
further information on the topic, in
preparation for the second part of the
assessment. This would make it explicit
to the students that problem-solving
skills are valued and not just
regurgitation of information obtained
from the problem sets undertaken
18
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
during the course.
 In terms of in-course assessment, I feel
that this is valuable, providing the
opportunity for formative feedback and
for students to gauge their performance
during the course. However, the marks
were relatively high, providing an
advantage to the weaker students who
did not perform particularly well in the
final examination. The weighting of the
ICA is important in this respect, making
it of sufficient value that students will
engage, but not so much that it creates
problems in terms of allowing
progression of students who are clearly
deficient in their knowledge and
understanding.
 The external examiners noted a high
proportion
of
students
gaining
Distinction and Credit, with relatively
few Pass grades. Although it is
appropriate to reward students whose
performance deserves recognition, there
is a risk of “devaluing” the higher levels,
if most of the class are considered
“above average”. This issue probably
needs
some
further
discussion
internally, particularly in light of how the
Hofstee method is going to be applied
and whether this will involve not only
the Pass/Fail boundaries, but also
boundaries for Credit and Distinction.
19
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
External Examiner 3
The course is very appropriate for the end of
year 2 of the BVMS course. It can be further
improved by some change of assessment, to
maintain sufficient challenge and also to test
appropriate skills
Course Organiser Response
External Examiner 4
While the exam is, in overall terms,
comprehensive and robust, there are areas of
concern, including setting interpretation
questions that genuinely require students to
interpret data or images given, and not factual
recall on, for example, topographical anatomy or
biochemical pathways. In relation to integrative
questions, oral and essay formats lend
themselves to exploring the students’ abilities in
this regard, and such formats can be structured
so that the assessment is reliable. An alternative
approach might be to inform the students of the
general/broad subject area that is to be
examined a few days before the exam – and
then set an exam that requires students to apply
(not just factual recall but application of
knowledge) their knowledge to, for example, a
case scenario, where several facets of the course
material could be integrated into a single case.
Indeed, a two-part open book exam format
might be suitable, with the first part of the
question based on interpretation of data
provided in the question and the second part
requiring integration/application of knowledge
20
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
across organ systems/subject disciplines and a
longer answer – to allow the better students the
opportunity to demonstrate their ability to
integrate knowledge from diverse parts of the
course.
Course Organiser Response
Animal Body GEP
EE reports never received from either external
The AB4 exam will be altered to reflect the
concerns of staff and externals re over
examining of a small 10 credit module. . The
essay component will be removed.
AHWFS 2
External Examiner 1
For the written exams I feel there does need to
be a standardisation of marking technique:
namely that for all short answer questions marks
should be indicated on the script itself and that
there should be an agreement amongst all
markers on rounding half marks up or down.
With regard to the practical exams, I would
recommend that consideration be given to the
use of an alternative venue for the dog & cat
practical exam. While the Dog & Cat Home does
have the major benefit of providing a ready
source of dogs for handling, in other respects it
is not fit for purpose. The area in which the
This year due to the higher number of
students failing practical exams at 1st
attempt, there were considerable more
students attending for a 2nd attempt in the
presence of an external examiner, which
made it a much more onerous task than in
previous years. Although one of the external
examiners felt that they would only need to
observe a few 2nd attempt practical exams
across the range of species, rather than all of
them, the internal examiners all agreed that
having a second examiner at the 2nd attempt
is vital to ensure that any fails can be fully
justified as failure at 2nd attempt then
requires the student to attend for a resit
21
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
exam is conducted is very small, very hot and
practical in August. In future, if there are
very noisy. Examinations can be disrupted by
large numbers of students sitting at 2nd
the entrance of staff during the exam itself. In
attempt, we may consider using a second
addition the only assessment of candidate’s
internal examiner to reduce the time
ability with cats (that I witnessed) was to identify
commitment of the externals.
breeds from photographs and at no time was the
candidate’s competence in handling or
The external examiner concerns regarding
restraining a cat examined.
the apparent ‘luck of the draw’ aspect of the
cattle handling exam were fully discussed at
With regard to the cattle practical exam, I was
the post course review meeting and also
disappointed that no calves were available for
with Professor Rhind. All staff concerned
the exam since questions around management
were satisfied that the exam was fair and
and handling of neonatal calves is an area which
that the students were given clear
allows the more able student to demonstrate
information and sight of the exam mark
knowledge gained during EMS.
sheet prior to the exam to fully prepare
them. It was indicated that most failing
students actually fail on the easier aspects of
this exam (such as applying a halter) rather
External Examiner 2
than the tasks that are perceived to me
Practical exams generally
more challenging (such as placing a mouth
I really enjoy doing the practical exams but do
gag).
wonder, as the main job of the external
examiners is to assess the exam process, if
The external examiner concerns regarding
spending so much time examining is really the
the cat and dog practical exam were also fully
best use of our time. Maybe the externals in the
discussed at the post course review meeting.
future could have some ‘mini sessions’ (4-6
The suitability of the Edinburgh Cat and Dog
students) on each practical section with another
home as a venue for the Cat and Dog
observer then taking. This would be especially
handling exams was discussed and although
useful where there are lots of fails. For example I
there was some agreement with their
don’t think that Graham has never observed
concerns, currently no suitable alternative
‘exotics’ but by now will have seen more than
venue could be identified. The internal
enough students on dogs and cats.
examiners still consider the application of a
22
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
tape muzzle to be a very important skill but
Exotics practical exam
will discuss this further before the next round
of exams.
 Exam very well set up and prepared in
advance by the lovely lady who runs the
It was decided that half marks in the written
centre whose name I have sadly
exam would continue but they must not be
forgotten.
rounded up.
 Good species range available and
education of students obviously very
multi-species, which is good and
probably a unique selling point for
Edinburgh.
 Students generally well prepared and
seemed to have made use of the
revision sessions on offer.
 The practical tasks and questions
covered the performance criteria
examined very well.
 Nice examining (Kevin Eatwell) with a
calm
and
supportive
approach,
encouragement of students and good
use of questioning.
 Good care taken not to ‘over use’
individual animals for the exam.
Cattle practical exam
 A well organised and well prepared
exam.
 Students seemed very well prepared
with obvious access to revision sessions.
 I did wonder about the variation in
difficulty of the tasks (anti-kick bar/
23
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
ageing/casting ropes) and consequently
the ‘luck of the draw’ influencing
outcome. In reality however, the
students that failed did so based on lack
of knowledge and most managed the
practical aspect of the exam very well.
 Many of the students seemed unfamiliar
with the practical application of what
they were being asked to do. Whilst the
quality of teaching appears to be more
than adequate, I am concerned about
how much farm and large animal
practice the students complete outside
of vet school classes. If the intention is
to maintain a truly omni-competent
degree then the students must be
familiar with and expected to be able to
work at a high standard with all species.
This is not a criticism of the staff or
teaching, but it is important not to
‘dumb down’ on large animal skills just
because the students are less familiar
with them or see themselves as never
having to use them.
 Excellent examining (Wilson Lee). Some
of the students appeared to forget they
were in an exam the atmosphere was so
relaxed, whilst at the same time the
rigor of the exam was fully ensured.
Dog and cat practical exam
 Nicely examined by Caroline on the
24
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
whole. A little slow to start (a lot to do in
10min especially when with less able
students) but once moving Ok got
through the students only a little over
time. I sometimes thought Caroline
could have guided the students a little
more than she chose to do (e.g. ‘what
age is this dog?’ could have been
followed up by ‘is it a puppy, an adult, an
old dog?’ rather than just waiting for an
answer).
 This isn’t the ideal environment (dog and
cat home) and seems to get worse each
year. The room is very small, no facilities
for cleaning the table between animals,
constant barking dogs etc. One student
asked for gloves and we had to go and
find some as they were not available.
The dogs themselves are also not ideal
for an exam as they are too nervous or
excited. Although I didn’t mind (it’s the
work I do), it is perhaps unreasonable to
ask the observer to hold the dogs to the
extent that is necessary and it must
certainly be challenging to do so if this is
not your day job.
 Some new dog pictures are badly
needed. In Scotland shouldn’t the dogs
all have tails!
 Need some more on cats. Better pictures
and a winder range of questions. Ideally
would have some cats or cat models to
25
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
handle.
 I worry that some students are lifting
badly – they starting kneeling on the
floor next to the dog. Not so much bad
dog handling so much as poor training
lifting weights.
 The muzzle! I’m not sure why the
students have to do this. I appreciate
that knowing how to do it is important,
but in reality the students would not be
safe placing a tape muzzle on an
aggressive dog. As an employer, my view
is that for a ‘day one competence’ the
students need to make a dog safe in a
consult room. I’d prefer to see them
correctly place a ‘Baskerville’ or ‘Mikki’
muzzle on a live dog rather than apply a
tape muzzle to a mannequin. Using the
fluffy dog it’s hard to know what is really
competence – is the student too close,
should they be in front of the dog, are
their fingers going to get bitten? This is
almost a theoretical test rather than a
real practical one. Please consider
changing this.
 The examiners need to know the age
(check dentition), weight and sex of the
dogs beforehand to ensure they know
when the student has answered
correctly, I wasn’t sure this was always
the case. I’m sure this was examined
adequately, in that nothing seemed
26
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
obviously incorrect, but would be have
been easier if all values were pre
determined.
Written papers
Multiple choice
I assume that some use if being made of the
answers in order to develop better questions in
the future? Overall seems to be good use of this
format with a good span of marks. MCQs do
seem to allow students that perform relatively
poorly on the short answers to pull above the
pass mark, which I assume is partly their
intended function?
Short answers
General comments
 I like the changes so that 50% needs to
be gained on the written papers for a
pass, meaning the practical exams
cannot pull up very poor students.
 There needs to be agreement on how
the staff mark the scripts. Some staff
show where each mark has been gained,
whilst others give a single overall mark
(e.g. Q1 and Q8). The former does more
easily allow checking of marking and
ensures standardisation. Having both
options is confusing for external
examiners and no doubt for the students
if they request to see their papers.
27
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
 There needs to be agreement on the
rounding up (or not) of half marks. Q5
had consistently been rounded up,
whilst this does not seem to have
happened for other questions.
 Q1 Really hard to know where marks
have come from on this question. The
examiner has given some explanation for
why so few students gained good marks,
but maybe this means the question
should have been worded more
carefully.
 Q3 and Q4 Really nice to see this subject
being well answered and good questions
set that made sense and that the
students could answer.
 Q5 See comments above re half marks.
 Q6 Nice range of marks across a well
answered question.
 Q7 Students seem to do poorly on this
subject, unsure why.
Comments on staff
The exams were a pleasure to be involved in,
mostly due to the professionalism and kindness
of the staff.
28
External Examiner Comments & Course Organiser Responses, 2013/14
Course
AHWFS GEP
External Comments
External Examiner 1
With regard to the poorly answered AHFWS GEP
short answer question (Q9) as a general policy it
might be better to pose questions that might be
answered from the basic course notes or to
clearly advise students that further reading was
required to prepare oneself for the exam. My
personal preference given the existing breadth
of AHFWS course, that is would be better for all
“examinable content” to be covered within the
basic course notes.
For the written exams I feel there does need to
be a standardisation of marking technique:
namely that for all short answer questions marks
should be indicated on the script itself and that
there should be an agreement amongst all
markers on rounding half marks up or down.
Course Organiser Response
This year due to the higher number of students
failing practical exams at 1st attempt, there
were considerable more students attending for a
2nd attempt in the presence of an external
examiner, which made it a much more onerous
task than in previous years. Although one of the
external examiners felt that they would only
need to observe a few 2nd attempt practical
exams across the range of species, rather than
all of them, the internal examiners all agreed
that having a second examiner at the 2nd
attempt is vital to ensure that any fails can be
fully justified as failure at 2nd attempt then
requires the student to attend for a resit
practical in August. In future, if there are large
numbers of students sitting at 2nd attempt, we
may consider using a second internal examiner
to reduce the time commitment of the externals.
With regard to the practical exams, I would
recommend that consideration be given to the
use of an alternative venue for the dog & cat
practical exam. While the Dog & Cat Home does
have the major benefit of providing a ready
source of dogs for handling, in other respects it
is not fit for purpose. The area in which the
exam is conducted is very small, very hot and
very noisy. Examinations can be disrupted by
the entrance of staff during the exam itself. In
addition the only assessment of candidate’s
ability with cats (that I witnessed) was to identify
breeds from photographs and at no time was the
The external examiner concerns regarding the
apparent ‘luck of the draw’ aspect of the cattle
handling exam were fully discussed at the post
course review meeting and also with Professor
Rhind. All staff concerned were satisfied that
the exam was fair and that the students were
given clear information and sight of the exam
mark sheet prior to the exam to fully prepare
them. It was indicated that most failing students
actually fail on the easier aspects of this exam
(such as applying a halter) rather than the tasks
that are perceived to me more challenging (such
as placing a mouth gag).
29
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
candidate’s competence in handling or
restraining a cat examined.
With regard to the cattle practical exam, I was
disappointed that no calves were available for
the exam since questions around management
and handling of neonatal calves is an area which
allows the more able student to demonstrate
knowledge gained during EMS.
External Examiner 2
Practical exams generally
I really enjoy doing the practical exams but do
wonder, as the main job of the external
examiners is to assess the exam process, if
spending so much time examining is really the
best use of our time. Maybe the externals in the
future could have some ‘mini sessions’ (4-6
students) on each practical section with another
observer then taking. This would be especially
useful where there are lots of fails. For example I
don’t think that Graham has never observed
‘exotics’ but by now will have seen more than
enough students on dogs and cats.
Course Organiser Response
The external examiner concerns regarding the
cat and dog practical exam were also fully
discussed at the post course review meeting.
The suitability of the Edinburgh Cat and Dog
home as a venue for the Cat and Dog handling
exams was discussed and although there was
some agreement with their concerns, currently
no suitable alternative venue could be identified.
The internal examiners still consider the
application of a tape muzzle to be a very
important skill but will discuss this further before
the next round of exams.
It was decided that half marks in the written
exam would continue but they must not be
rounded up.
Exotics practical exam
 Exam very well set up and prepared in
advance by the lovely lady who runs the
centre whose name I have sadly
forgotten.
 Good species range available and
education of students obviously very
30
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
multi-species, which is good and
probably a unique selling point for
Edinburgh.
 Students generally well prepared and
seemed to have made use of the
revision sessions on offer.
 The practical tasks and questions
covered the performance criteria
examined very well.
 Nice examining (Kevin Eatwell) with a
calm
and
supportive
approach,
encouragement of students and good
use of questioning.
 Good care taken not to ‘over use’
individual animals for the exam.
Cattle practical exam
 A well organised and well prepared
exam.
 Students seemed very well prepared
with obvious access to revision sessions.
 I did wonder about the variation in
difficulty of the tasks (anti-kick bar/
ageing/casting ropes) and consequently
the ‘luck of the draw’ influencing
outcome. In reality however, the
students that failed did so based on lack
of knowledge and most managed the
practical aspect of the exam very well.
 Many of the students seemed unfamiliar
with the practical application of what
they were being asked to do. Whilst the
31
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
quality of teaching appears to be more
than adequate, I am concerned about
how much farm and large animal
practice the students complete outside
of vet school classes. If the intention is
to maintain a truly omni-competent
degree then the students must be
familiar with and expected to be able to
work at a high standard with all species.
This is not a criticism of the staff or
teaching, but it is important not to
‘dumb down’ on large animal skills just
because the students are less familiar
with them or see themselves as never
having to use them.
 Excellent examining (Wilson Lee). Some
of the students appeared to forget they
were in an exam the atmosphere was so
relaxed, whilst at the same time the
rigor of the exam was fully ensured.
Dog and cat practical exam
 Nicely examined by Caroline on the
whole. A little slow to start (a lot to do in
10min especially when with less able
students) but once moving Ok got
through the students only a little over
time. I sometimes thought Caroline
could have guided the students a little
more than she chose to do (e.g. ‘what
age is this dog?’ could have been
followed up by ‘is it a puppy, an adult, an
32
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
old dog?’ rather than just waiting for an
answer).
 This isn’t the ideal environment (dog and
cat home) and seems to get worse each
year. The room is very small, no facilities
for cleaning the table between animals,
constant barking dogs etc. One student
asked for gloves and we had to go and
find some as they were not available.
The dogs themselves are also not ideal
for an exam as they are too nervous or
excited. Although I didn’t mind (it’s the
work I do), it is perhaps unreasonable to
ask the observer to hold the dogs to the
extent that is necessary and it must
certainly be challenging to do so if this is
not your day job.
 Some new dog pictures are badly
needed. In Scotland shouldn’t the dogs
all have tails!
 Need some more on cats. Better pictures
and a winder range of questions. Ideally
would have some cats or cat models to
handle.
 I worry that some students are lifting
badly – they starting kneeling on the
floor next to the dog. Not so much bad
dog handling so much as poor training
lifting weights.
 The muzzle! I’m not sure why the
students have to do this. I appreciate
that knowing how to do it is important,
33
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
but in reality the students would not be
safe placing a tape muzzle on an
aggressive dog. As an employer, my view
is that for a ‘day one competence’ the
students need to make a dog safe in a
consult room. I’d prefer to see them
correctly place a ‘Baskerville’ or ‘Mikki’
muzzle on a live dog rather than apply a
tape muzzle to a mannequin. Using the
fluffy dog it’s hard to know what is really
competence – is the student too close,
should they be in front of the dog, are
their fingers going to get bitten? This is
almost a theoretical test rather than a
real practical one. Please consider
changing this.
 The examiners need to know the age
(check dentition), weight and sex of the
dogs beforehand to ensure they know
when the student has answered
correctly, I wasn’t sure this was always
the case. I’m sure this was examined
adequately, in that nothing seemed
obviously incorrect, but would be have
been easier if all values were pre
determined.
Written papers
Multiple choice
I assume that some use if being made of the
answers in order to develop better questions in
34
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
the future? Overall seems to be good use of this
format with a good span of marks. MCQs do
seem to allow students that perform relatively
poorly on the short answers to pull above the
pass mark, which I assume is partly their
intended function?
Course Organiser Response
Short answers
General comments
 I like the changes so that 50% needs to
be gained on the written papers for a
pass, meaning the practical exams
cannot pull up very poor students.
 There needs to be agreement on how
the staff mark the scripts. Some staff
show where each mark has been gained,
whilst others give a single overall mark
(e.g. Q1). The former does more easily
allow checking of marking and ensures
standardisation. Having both options is
confusing for external examiners and no
doubt for the students if they request to
see their papers.
 There needs to be agreement on the
rounding up (or not) of half marks. Q5
had consistently been rounded up,
whilst this does not seem to have
happened for other questions.
 Q1 Really hard to know where marks
have come from on this question. The
examiner has given some explanation for
why so few students gained good marks,
35
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
but maybe this means the question
should have been worded more
carefully.
 Q3 and Q4 Really nice to see this subject
being well answered and good questions
set that made sense and that the
students could answer.
 Q5 See comments above re half marks.
 Q6 Students seem to do poorly on this
subject, unsure why.
 Q9 Poorly answered. Natalie has
provided info on this that shows the
subject has been well taught, was the
question itself a problem or did the
students not attend lectures?
Course Organiser Response
Comments on staff
The exams were a pleasure to be involved in,
mostly due to the professionalism and kindness
of the staff.
SSC1
No report received
Revise course handbook to explain the rationale
regarding stipulating non-BVM&S topics for
SSC1, and highlight this more in the introductory
lecture. (It is unlikely this will completely
forestall student criticisms of this type as some
student are intensley focussed on veterinary
content and may not grasp the opportunity Prezi
allows to develop generic skills).
Explain the principles and philosophy of peer
assessment to the students in the introductory
36
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
lecture.
Allow topics from veterinary history to be
included to satisfy some students’ wishes to
tackle veterinary topics, but in a non-BVM&S
context.
Give examples of suitable non-veterinary topics
in course handbook.
Revise assessment criteria into a new standard
SSC1 mark sheet, with a mark out of 20 drawn
from 10 criteria statements that can be marked
0, 1 or 2. Six criteria relate to the Prezi; four
relate to the presentation and discussion session
for each group.
Expand the numbers of internal examiners to
allow a more compressed period of assessment
for SSC1, still maintaining two examiners for
every Prezi and discussion.
37
External Examiner Comments & Course Organiser Responses, 2013/14
Course
Vet Path
External Comments
Course Organiser Response
The students’ performance in the examination
Standard setting introduced.
was exceptional – a very high proportion of the
cohort passed the examination with an
aggregate mark greater than 60%. I have no
doubt that this outcome reflects the hard work
of the students and the dedication of the
teaching team. However I am concerned that
the examination could be made more
challenging in order to stretch the students more
and achieve a more even distribution of marks in
the 50-59% range by comparison to those marks
in the 60-69% range and the 70% or over range.
This might be done, for example, by always
including a question on pathogenesis or disease
mechanisms in the spot test; including some
more microscopic images (for example from
smears) in the spot and MCQ test; and trying to
design more of the MCQs to test problemsolving rather than factual recall. The teaching
team were very receptive to these suggestions
at the Examination Board and I have no doubt
that these improvements can be made in coming
years.
I have indicated how I believe the MCQ and spot
examination could be made more challenging in
coming years and will be interested to learn how
this progresses. I am otherwise wholly satisfied
with the conduct of the examination and am
pleased that this year’s cohort of students has
performed so well.
38
External Examiner Comments & Course Organiser Responses, 2013/14
Course
Clinical Foundation Course
Dog & Cat
External Comments
Overall this is a very good course sited at an
appropriate stage of the BVM&S degree, with
generally an appropriate level of knowledge
expected from the students. The only area of
obvious discussion during the BoE was the
spherocyte part of one question, which
subsequently was removed from the
examination – one suggestion for this would be
to review question topics (and specimen
answers) in light of what an adequate student
would be expected to know, and also to check
on how the necessary information for that
question has been delivered in the course.
However some of these more difficult/less
mainstream questions probably should be kept
to act as discriminators.
No report received
Course Organiser Response
This year we welcomed a new external examiner
to the CFC. We are very pleased with the
comments from the external examiner. He has
highlighted concern with one question on the
examination paper, which in retrospect was not
appropriate for students at this stage in the
course. Student results for the whole year were
moderated to take this into account but it will
make us a little more fastidious in relation to
review of examination papers at an earlier stage
and not being quite so reliant on individual
question setters.
It is planned to try to introduce more small
group teaching covering ‘first opinion’ areas of
vaccination and parasite control. This was a
result of identifying this as an area of weakness
in final year students, and gives a practical
application to the more theoretical knowledge
taught about infectious diseases. Ideally would
also like to introduce an ophthalmology
practical, based on feedback from students, but
this will be dependent on staffing.
39
External Examiner Comments & Course Organiser Responses, 2013/14
Course
Farm Animal
External Comments
No reports received
Course Organiser Response
No comments available to respond to.
Equine
One question in paper 1 (Q2) clearly caused
some difficulties, with a low mean and high SD
of marks gained. Individual performance in this
question however did not affect outcomes in
terms of pass/fail. The question overall was in
my opinion fair but demanding of the candidate
group. However it was structured such a way
that failure to correctly answer part 1 could not
subsequently be recovered in parts 2-4. The
answer to part 1 was however obvious, and
should have been evident to all individuals in the
candidate group. Nevertheless, in future
questions where marks cannot be recovered
after early failure to correctly answer should be
considered carefully before their inclusion in the
paper.
Changes for 14/15
1) Lecturers should concentrate on less
complicated procedures, particularly those
needed for day one skills
2) Lecturers should read over their current notes
and edit them as appropriate and should then go
through their PowerPoint presentations, editing
and adding/deleting new slides as appropriate
3) Most importantly, in view of numerous
requests from the students to ensure that
equine lecture notes and power points are
synchronised (they do not have to be identical)lecturers should ensure they cover similar areas
chronologically
4) Lecturers should consider whether
appendices are necessary and if they are, they
should be as brief as possible. PMD/RR to clarify
40
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
to all lecturers that material from appendices is
not examinable
5) If possible, lecturers should insert a brief
summary slide at the end of each lecture section
to highlight the important points on that topic
e.g. a few summary slides per lecture
6) In Jan 2015 - we will give a short introduction
to the equine course just before the first lecture
to outline the nature of the course,
acknowledging the fact that it is compact course
and especially aims to teach day one knowledge
& skills. We will more clearly explain the two
different assessments that will take place, i.e.
MCQ s only in the first examination (3% of the
total examination marks – on subject matter
that cover the first 2 weeks of equine course) ,
and MCQ’s and short questions in the main
examination (97% of total marks) and also
clarify about appendices to students
7) The interim examination results will be given
back to the student’s within 24 hours
8) Following the degree examination, the course
organiser or the chair of the examination
committee will review the questions with all
students, giving he correct answers for all MCQs
and model answers for short answer questions emphasising the questions that were not
answered well and also can have a general
discussion with the whole class. Unlike in 2014,
41
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
time must be made available for this feedback
session
Exotics
No comments or suggestions made.
No concerns raised by external examiner.
Positive comments on quality and content being
extremely high and setting a benchmark for
other Universities
AHWFS 4 (VPH)
I support the introduction of the new
Assessment Standards allowing for more
objective and defensible standard setting of pass
marks.
I recommend that there is an extensive review of
the reasons for the population-level decline in
student performance in this examination this
year, with development of defined corrective
actions. While I was willing to support such a
significant Hofstee adjustment, on this occasion,
in the context of a major change in the over-all
Not yet received
42
External Examiner Comments & Course Organiser Responses, 2013/14
Course
Final Year Rotations
External Comments
structure of the teaching year and assessment
year – thereby making the examination ‘harder’
this year, I don’t consider such significant
adjustments are acceptable in an on-going basis.
This is because of the impact on ability to
demonstrate achievement of RCVS Day One
Skills. Importantly, such adjustments are
potentially damaging to the morale and
motivation of staff involved in teaching and
assessment.
I made a recommendation at the Exam Board
meeting in May 2013 that consideration be given
to setting a minimum required mark in each of
the practical and written components of this
exam, such that students cannot progress to
pass if they achieve less than this mark in one or
the other of these elements (for example,
students must achieve a minimum of 45% in
each element, with an over all combined mark of
50% in order to pass). I would be interested in a
response to this recommendation.
External Examiner 1
This is the last year for this format of practical
exams, so the validity of any comments made
areof limited value other than general feedback.
I have often wondered if the current roll of the
external examiner has been explained to the
students prior to sitting their exams. After some
discussion with various people I think it would
be useful to explain to all students that we are
not examining them but the process of the
exam,as many are still under the mistaken
Course Organiser Response
The external examiners were in general happy
with the conduct of the exams/exam board etc.
Sarah Bailie was a little concerned about the
quality of the MCQ questions chosen for this
year. In future years all staff writing MCQs will
be required to attend training. All questions will
be vetted by a panel of staff and standard set
ahead of the exam to improve quality in future.
External examiners would have like to have been
given more details relating to the clinical stations
ahead of time, but this is impractical in some
43
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
impression that we are examining them directly.
The addedpressure placed on the candidates
may be reduced if they have a better
understanding.
The construction of the scenarios observed was
ly of a high standard, but my impression
was that the practical hands on skills were not as
evident in some areas as in previous years.
Last year we requested copies of scenarios prior
to (or on the morning of) the exams, this only
happened for about 50% of stations and makes
our job more diffrcult as there is only limit time
between candidates to gain that information and
examiners have tasks to perform within that
period. Training and information for new
external examiners remains virtually
nonexistent, and
thought should be given to this
The examiners have generally continued a high
level of consistency and the new examiners
should be commended for their efforts and
performance. Continued training of examiners as
formats change for consistency of approach is
likely to be a very important feature in the
future.
The general practitioner stations were of good
quality and this part of the exam was an asset,
and
I think it would be useful to find away to include
their skills in the revised system.
The use of video cameras in many of the stations
is a great innovation especially if it is possible
Course Organiser Response
situations where the exam uses the hospital’s
current clinical case load. In general the
performance of our assessors was strongly
praised, but there were still some incidences of
improper small talk and teaching rather than
assessment within the stations which is
disappointing with the required examiner
training last year. Concern about the students’
knowledge gaps particularly related to use of
calving equipment, hoof testers etc is worrying
and efforts will be made next year to modify
assessment to ensure all students are prepared
for D1 practical skills. The external examiners
have given us guidance on the skills they view
are D1 for practice.
44
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
to use the video to improve the technique of
examiners and possibly to help students
overcome
specific problems. I am uncertain how these will
be used under the new regime, but will be
interested to see the outcome.
A couple of points on the locations and circuits
used for the exams, the rain on the Monday
afternoon was so heavy that it made hearing
difficult in the Large Animal Hospital pens. This
would be very difficult to do anything about but
is worthy of note. Also the flow of the
Farm/Equine circuits crossed by the cattle crush,
meaning a student could be exiting through that
area while an exam was in progress.
The expanded Multiple Choice Questions (MCQs)
are an improvement on the previous shorter
version. Although it is important to ensure that
the questions are of a high quality, this is
extremely important when asking examinee's to
chose the best answer so that opinions do not
creep in that may be questioned later.
I am hopeful that the new exam system will
allow for broader knowledge across the species
and
promote comprehension and thought rather
than regurgitation.
Finally I would like to commend all examiners,
sections teams, general practitioners, observer
volunteers and role-players for the massive
effort.
External Examiner 2
45
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
Course Organiser Response
This is my first year as external and my overall
impression of the way the exams were
conducted was very good. Suggestions for
change would be largely irrelevant as the viva
aspect of the finals is to be abandoned. From
the aspect of giving the students a realistic day 1
scenario I am sorry to see it disappear.
More emphasis (and I know the students should
not need to be spoon-fed and the lecturers will
already be aware of this) needs to be given to
'joining up' some teaching. As I commented
above one cannot advise on parasite problems if
the parasite cannot be identified by the
candidate. Similarly, one cannot comment on a
radiograph or MRI of the equine foot if the
student has not remembered salient details of
the anatomy of the foot from Second Year. I
could give many such examples of, 'I knew this in
first year but didn't realise it was that
important'.
As I commented above there perhaps could to
be more rigor applied to the quality and quantity
of EMS by the Directors of Studies. As stated
above, this was reflected in the exams by a
disquieting unfamiliarity with some students of
basic items such as hoof testers. If a student
does not have a good 'home' practice or they are
a student from overseas it must be very difficult
for them. If this was improved I feel overall
performance at the finals would improve and
the Dick would send students with more
46
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
confidence into the world. I have made some
suggestions, as requested, about testing day 1
skills in the OSCE's for equine and farm animal to
Rachel Whittington and I am happy to help in
any way.
External Examiner 3
Course Organiser Response
SSC2
These were recorded in the Board of Examiners’
meeting minutes, along with key points from the
ensuing discussions, and responses. Actions are
recorded in the subsequent minutes. While
mindful of duplication, I briefly review my
comments at the first meeting here; some have
since been dealt with:
(1 & 2) Mark descriptors - The marking sheet is
being revamped completely for 2014->
(3) This year the top prize was chosen from the 5
best performing projects – it is proposed to
repeat this for 2014->
(4) The marking sheet has been modified to
incorporate the abstract in the final mark and an
Abstract is in the requirements for all projects
(9) It is hoped that the new system of SSC2
project approval / management described below
will help the Course Organiser and Administrator
with the course
1.
Mark descriptors could be improved to
promote greater standardisation of marks across
projects and project types. Percentage
increments could be banded by grade to make
this more intuitive. At the moment, well-worded
check-box descriptors for continuous
assessment do not link explicitly to a mark or
grade, which seems a missed opportunity, while
resolution of scores is coarse – for example,
continuous assessment point 1 ranges from
excellent to satisfactory. The organisers might
consider codifying a finer scale with appropriate
descriptors, which ought to help markers
•
Planned changes in teaching, learning
and assessment (indicate how these originate ie
are they responses to students, external
examiners or teaching staff)
The system of SSC2 project approval /
management (including checking for Risk
47
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
achieve fair and objective outcomes. In
particular, this would make it more difficult for
easily pleased supervisors to give unrealistically
high marks for continuous assessment without
any further justification.
2.
Mark sheets varied enormously between
supervisors in terms of the depth of comment
added, with some adding little or no insight into
student performance. It might be worth adding
an instruction to the mark sheet stressing that
these comments are particularly important for
students given very high (for consideration of
prizes) or low (pass/fail) marks.
3.
No set procedure for awarding prizes
was in place and this needs to be determined.
4.
The plan to reduce continuous
assessment to a less than 50% share of marks
across project types seems sensible.
5.
Should the format of all projects include
an abstract? This was often missing, especially
form literature reviews, and would seem to be
an essential part of critical scientific writing.
Course Organiser Response
Assessments, Ethical approval (animal and
clients), Student survey overload) is moving to
an electronic system (Pebble Pad / Atlas) from
October 2014. This should make the managing
for 3rd, 4th and final year students and their
SSC2 much less manual and time consuming for
the Course Organiser and Administrator
Clinical project performance marks will only be
worth 30% of the overall mark. For laboratory
and clinical projects performance marks will be
from a grade of A to H and converted to a
percentage, to reduce variation between
supervisors. All marking sheets being revamped
in consultation with Exam Board Convenor and
Clinical Course Organiser
Teaching time was allocated at the end of fourth
year to support SSC2 by those students wanting
to complete / work on their project before they
started the new final year
Submission of SSC2 has been moved from end of
August to Easter of final year to reflect the
changes to final year for 2014-2015 onwards.
6.
In future it would be useful to the
external examiner to see the mark-sheet with
metrics, i.e. averages and measures of variance
across project types.
48
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
7.
The requirements for re-takes (of failed
projects) were somewhat unclear and discussed
during the meetings. It is noted that repeating or
extending data collection is usually not possible.
The consensus view, which I support, was that
these should take a form similar to ‘major
corrections’ of rejected papers, with re-analysis,
re-writing, and addition of supplementary data
(if already collected or available in the literature)
all being viable options. This was trialled for fails
this year and seemed to work well.
Course Organiser Response
8.
Feedback given to students who failed
the component was excellent, with clear
instructions on what was needed to reach the
required standard, and as a result all re-takes
were successful.
9.
I commend all involved for a very well
run, serious component, which adds a great deal
to the course in terms of research and
transferable skills, and evidence based
veterinary medicine. I believe that the
importance and special nature of this unit
warrants its status as a separate subject, with
Examination Board, dedicated external
examiner, and administrative support. I am
somewhat concerned that a great deal hinges on
the provision of adequate administrative
support, and also on the commitment and
dedication of a single member of academic staff.
The consequent teaching load is heavy and SSC2
49
External Examiner Comments & Course Organiser Responses, 2013/14
Course
External Comments
is vulnerable to the sudden unavailability of this
member of staff, especially at crucial periods. At
the same time, I acknowledge that it would be
difficult to standardise moderation if more staff
were involved. Nevertheless, thought ought to
be given to how the staff resources
underpinning the course should be protected
and rotated, and to contingency plans in case of
disruption.
Course Organiser Response
50