BVM&S Assessment Standards 2013-14
The first part of this document relates to MCQ based assessments; the second part is more general
guidance for examination boards when considering overall mark profiles for cohorts of students.
A further document will follow giving practical step by step guidance on how to interpret the
Speedwell statistics and carry out standard setting using the Hofstee method.
MCQ Question Performance
1. In advance of a meeting of the Board of Examiners the Chair and Course Organiser are
recommended to review the item analysis within the Speedwell statistics and identify
questions whose profile may suggest a question has been misinterpreted or contained
inappropriate distractors.
2. The point biserial should be between 0 and 1. Where it is below 0.19 the question should be
reviewed. Ideally a question will have a point biserial above 0.3.
Review of Overall Mark Profiles
In years 1-4, the Hofstee method will be used to inform examination board decisions.
This is a recognised ‘compromise’ standard setting method which incorporates:
1. Academic judgement as to the difficulty of the exam and therefore the ‘tolerance’ for a
score which allows a student to pass
2. Tolerance for fail rates
The Assessment Standards Working Group supported this approach as a trial for session 13-14. In
addition, we will trial a similar method around the distinction boundary to address the concern from
some exam boards and externals that the numbers of distinction candidates are now too high.
In practice what this means is that in advance of each examination board, a meeting will be
scheduled to present the exam board chair with the standard setting data and discuss the
implications for setting of pass/fail and distinction boundaries.
The suggested ‘trial’ values for the Hofstee approach are:
Pass: Fail Boundary
Lowest acceptable percent-correct score that allows a borderline student to pass
Highest acceptable percent-correct score that allows a borderline student to pass
Lowest acceptable percentage of students to fail
Highest acceptable percentage of students to fail
45
55
0
10
Distinction boundary
Lowest acceptable percent-correct score as distinction boundary
Highest acceptable percent-correct score as distinction boundary
Lowest acceptable percentage of students to gain distinction
Highest acceptable percentage of students to gain distinction
70
80
5
15
Implications for Resit Assessments
This type of standards setting approach is not appropriate for small numbers of students. It is
therefore important that previously tested questions are used in resits.
The pass mark will then take into account previous borderline students’ performance on these
questions.