Support for External Examiners not Working in
Higher Education
A Report for the Higher Education Academy
External Examiner Project
Authors: Colleen Connor & Dr Bill Davies
CONTENT
1. Introduction
2. Background Information
3. Methodology
4. Analysis & Discussion
5. Conclusion
6. References
Appendices
(i)
(ii)
(iii)
Letters to University Registries
Covering letters to External Examiners
Questionnaire
1. Introduction
1.1 The purpose of the following report is to provide a description of a study of
external examiners working in non-academic settings (i.e. not employed in a
higher education institution) and their experience of induction & support for the
external examiner role.
1.2 The study was conducted following a workshop led by Professor Howard Colley
at the University of Wales Institute, Cardiff (UWIC) in February 2005. One issue
arising at this workshop concerned the appointment of external examiners from
outside the higher education context and whether they had any specific needs in
terms of induction.
1.3 The study thus sought to establish the number and nature of external examiners
drawn from non higher education contexts within a small group of Welsh
universities and to investigate their experience of induction.
1.4 The following section explains the rationale for undertaking the study together
with a short review of background literature that informed the investigation. The
methodology of the study is then outlined (Section 3), followed by an analysis and
discussion of the results (Section 4). Finally some conclusions are drawn together
with suggestions for further investigation (Section 5).
2. Rationale and Background
2.1 The role of external examiners has been extensively articulated and discussed
(Ashworth and Harvey, 1994; Ashworth, 2004 and Jackson 2004) and embodied
in a code of practice (QAA, 2004).
2.2 While there is some variation in the tenor of this literature and some different
interpretations of the role (Smith & Oliver, 2004) there seems to be broad
agreement that the fundamental purpose of an external examiner rests on three
functions, namely:
 to act as an arbiter of standards and to confirm that the awards made to students
are comparable in standards with awards in higher education generally.
 to monitor, assist and advise on the enhancement of the quality of programmes;
 to provide advice particularly on matters of student assessment, learning
outcomes, content, and the inter-relationship of these elements
2.3 Reflecting on the fundamental contribution external examiners make to the
quality assurance of courses a number of writers have argued the case for a
coherent approach to the development of external examining as a professional
progression within Higher Education. Cowan (2004) suggests that
‘The enhancement of professional development for external
examining should be integrated with the strategic development of
infrastructures to support the wider professional development of HE
teachers’ p.2-3
2.4 Similarly Jackson (2004) suggests that external examining be seen as one of a
raft of peer review, consultancy and evaluation activities that constitute advanced
professional practice in the field of teaching and learning.
2.5 The notion of locating external examining expertise within an academic or higher
education context is echoed / replicated by analysis of the skills and attributes
required to be effective as external examiners. Jackson (2004) reviews the
discussions and researches of the Higher Education Academy external examiner
working group and suggests that the observations and judgements that external
examiners make, to be effective, have to be informed by contextual knowledge of
being a higher education teacher for example
“to understand the technical cultural and procedural language of
the discipline, the institution and the national frameworks for
describing and evaluating standards” p2
2.6 In a later paper (working paper 7) Jackson reiterates the expressed view from
within the sector that academic experience and understanding are central to the
examiner role.
“locating external examining within the career profile of a teacher/
scholar is an important step in understanding the nature of the
experiential learning and more formal professional development that
underlies external examining” p2
2.7 It can be seen therefore that a consistent theme is the central relevance and
inescapable importance of a facility and familiarity with the standards and
processes that operate with academic institutions and that these attributes can only
be gained through lengthy immersion in such institutions.
2.8 Whilst this is intuitively and empirically valid, the rise of diverse vocationally
oriented programmes and the continued involvement of professional bodies in
approving qualifying programmes have meant that institutions have to assure
congruence of programmes with the practitioner and marketplace context and
needs. This may become particularly important if modes of delivery become even
more workplace located and oriented.
2.9 Ashworth (2004) offers a range of added value functions offered by external
examiners who are practitioners rather than academics. These include the
provision of a focused business perspective to vocational academic areas; an input
of current technical and professional knowledge into the scrutiny of academic
endeavor; a view on appropriate transferable skills and graduateness which
complements the academic view; and the furnishing of an additional dimension to
the programme enhancement. These inputs and perspectives are likely to be more
highly emphasized where professional accreditation is a programme outcome.
2.10 Ashworth indicates that the professional bodies covering the built environment
sector tend to have a high level of interest in the contribution made by external
examiners to their approved programmes for example stipulating that at least one
examiner per programme should be a member of the relevant professional body or
requesting a combination of academic and practitioner examiners. Other
professional disciplines such as social work and speech therapy have similar
requirements and expectations
2.11 The QAA Code of Practice section on External Examining (August, 2004) also
notes the need for those with practice experience to take the external examiner
role.
The increasing diversity of programmes and modes of study within higher
education suggests that institutions will, where appropriate, want to appoint
external examiners with professional or practice-based expertise, in addition to
those with more typical academic backgrounds.
2.12 The literature repeatedly emphases the need for academic pedigree and depth of
experience of external examiners to enable them to discharge their responsibilities
of maintaining standards across institutions and ensure quality within institutional
frames of working. Therefore it is not surprising that there are significant
concerns about the ability of examiners located in non academic contexts to fully
fulfill the examiner role.
2.13 Smith and Oliver (2004) from an investigation of external examiners perceptions
of the role indicate the centrality of the academic grounding required by citing
one respondent
“all external examiners are primarily engaged in teaching. Neither
managers nor quality officers are suitable” p4
2.14 It was however acknowledged by respondents in the survey that the situation
may be slightly different for those working in a vocational discipline, where
elements such as cutting edge knowledge of professional practice, and access to a
network of external contacts and understanding of employment opportunities are
of critical importance. Even so, there was a clear view that having experience of
higher education was essential and that ‘non academic external examiners would
operate in clearly circumscribed circumstances.’
2.15 Jackson’s (2004) investigation into the institutional perspective on supporting
external examiners in general found a similar view to be held among academic
staff. The workshop discussions with the SE England group raised questions
about the issue of non academic externals, noting that the role of the external
examiner requires a lot of knowledge about assessment, generic, subject specific
and institutional. None of these areas are ones where a practitioner from a non
academic context can be assumed to have the requisite expertise
2.16 It is not just academics and the institutions that recognize potential problems in
the use of non academic examiners. Ashworth (2004) presents the difficulties
perceived by some practitioners in fulfilling an external examiner role. In addition
to those relating to assessment and understanding institutional systems, criteria
and procedures for marking and grading practitioners feel that the limited
experience they may have across institutions makes comparison of standards
difficult. They also profess to have problems in resisting stereotyping of
academics as lacking real world experience; they have difficulties in
understanding the pressures felt by academics within their own community; they
may perceive academics as too lenient with failing students , or alternatively be
unable to accept the reluctance of academics to use the full range of marks or
grades. Finally there may be incongruence between to concepts of graduate
competence and professional competence that are not easy to reconcile.
2.17 So, a number of forces can be identified; the imperative to maintain the
effective monitoring and development of quality and standards; the need to embed
external examining within a career structure that ensures individual professional
development based on experience within the academic context; and the increasing
need to draw upon the services of external examiners from non academic
backgrounds to permit continued validation and appropriate alignment of
vocational courses with employment needs and the various professional worlds of
students emerging from vocational courses.
2.18 It is evident through a number of consultations and discussions that have taken
place and been widely reported (Jackson, 2004; Colley, 2005) that while the
establishment of support and induction systems for all examiners is under way
there is still considerable room for the development of an agreed and consistent
framework. This is not surprising given the diversity of needs, contexts and
environments in which external examiners are required to operate.
2.19 But if the institutional responsibility highlighted by Cooke (2003) and neatly
paraphrased by Jackson in LTSN working paper 2 that…
“HEI’s… have the responsibility to prepare their external examiners
for their work in their institution. External examiners have to master
the peculiarities of the institutional environment in which they are
examining in order to fulfill their role”
is to be accepted and operationalised there is one specific area demanding of
attention.
‘An emergent issue is how such frameworks might also support non-academic
external examiners drawn from business, industry and the professional world.’
Cowan (2004, p.3)
2.20 The support, integration into the institutional context and efficient utilization of
examiners from non academic backgrounds remains an area for investigation and
development. Given the potential for dissonance and the greater potential for an
effective contribution to the development of programmes aligned with academic
and professional demands it is clear that a greater understanding of the training
and support needs of non academic examiners must be achieved.
3. Methodology
3.1 A multi- stage research design was followed consisting of an exploratory stage to
establish viable sample parameters and a questionnaire survey stage to obtain
preliminary responses from the population of relevant examiners. The intention
was to complete a follow up stage comprising telephone interviews with a
selection of survey respondents to further explore issues raised in the survey. This
final stage has not yet been completed due to external factors that could not be
foreseen.
3.2 Stage – (i) Identification of population and sample parameters.
In order to obtain a good spread whilst also maintaining manageability it was
planned to draw the sample of external examiners from Welsh universities that
attended the HE Academy network meeting in UWIC on 03/02/05. These were:
North East Wales Institute; Cardiff University; Aberystwyth University; Swansea
University; University of Wales, Newport and University of Wales Institute,
Cardiff
3.3 The advantage of this were seen to be
 Both pre and post 92 universities are represented
 A wide range of disciplines are covered
 The participants at the network meeting would
provide a contact point and they identified the issue
as important.
 It represent a discrete geographic area
 It would raise the profile of the HE Academy in
Wales
3.4 It was difficult to predict the number of ‘non-academic’ external examiners
appointed to these universities. A snapshot of UWIC external examiners showed a
total of 23 from non-academic settings. This was made up as follows:
School of Applied Sciences
School of Art & Design
Business School
School of Education
School of Health & Social Sciences School of Lifelong Learning
School of Product & Engineering Design
School of Sport
School of Hospitality, Tourism & Leisure Management -
4
2
1
3
4
1
1
1
3
3.5 Stage (ii) Institutional survey
This suggested that the numbers that could be drawn from other institutions
would provide a sustainable and realistic sample for worthwhile conclusions to be
drawn. However it was felt important to actually ascertain the number of externals
from non-academic settings and the programme areas they serve as appointed by
the different universities in order to be confident in the appropriateness of the
sample spread to be used and to permit a representative sampling grid.
3.6 It was also necessary to work closely with the Registries of the collaborating
institutions as the release of examiner data was problematic under the application
of the Data Protection Act
3.7 The six potential collaborating institutions were contacted and asked to return a
brief proforma containing information on current examiners. This included an
indication of those “not working in a higher education context” or working in
“non academic settings”.
3.8 The following table provides the distribution of “Academic” and “Non
Academic” external examiners received from Registries of participating
universities. (NB responses are provided according to whether the university is
pre 92 or post 92 in order to retain anonymity)
Table 1 Distribution of “Academic” and “Non Academic” external examiners
Pre 92 (A)
Pre 92 (B)
Pre 92 (C)
Post 92 (D)
Post 92 (E)
Post 92
TOTAL
Externals from
Externals from non
academic settings
academic settings
Undergrad
Post
Undergrad
Post grad
grad
107
127
5
13
60
41
13
4
135
85
5
3
90
62
1
2
86
18
17
2
No response to request
278
333
41
24
3.9 Thus responses were received from two Post 92 universities and three Pre 92
universities. The total number of external examiners from non academic settings
in the three Pre 92 universities numbered 43 (23 serving undergraduate courses
and 20 serving postgraduate courses). The total number of external examiners
from non academic settings in the two Post 92 universities numbered 21 (18
serving undergraduate courses and 4 serving postgraduate courses).
3.10 Development and pilot of questionnaire
A questionnaire was devised based on the key issues identified in the literature
(Jackson 2005, Colley 2005) covering the themes of:
Occupational and academic background;
Examining experience;
Current examining role;
Experience of induction and support;
Perceived needs in relation to induction and support.
Respondents were also asked to indicate their willingness to participate in a
follow up telephone interview.
3.11 Administration of survey
All institutions agreed to distribute questionnaires and covering letters on behalf
of the researchers. Questionnaires were distributed in January 2006 and a follow
up prompt was sent out via collaborative institutions in March 2006.
Table 2
Distribution of, & responses to, questionnaires to “Non Academic” external
examiners
Externals from non academic settings
Pre 92 (A)
Pre 92 (B)
Pre 92 (C)
Post 92 (D)
Post 92 (E)
TOTAL
No. sent
18
17
8
3
19
65
Responses
7
5
3
2
13
30
3.12 An initial response rate of 46% was achieved. The follow up prompt failed to
elicit any further responses. (The reason for this is unknown however it can be
surmised that industrial action and its effects on university registries may have had an
impact). One return questionnaire was not valid and thus a total of 29 questionnaires
were analysed (i.e. valid responses as a % of total = 45%).
4. Analysis and Discussion
4.1 Number of external examiners not working in an academic environment
The following proportion of external examiners from non-academic settings was
revealed in the information received from Registries regarding the number of
external examiners serving their university.
Table 3
On undergraduate programmes
University
Total Number of
external examiners
Pre 92 (A)
Pre 92 (B)
Pre 92 (C)
Post 92 (D)
Post 92 (E)
TOTAL
112
73
140
91
103
319
% of external
examiners from nonacademic settings
4.5
17.8
3.6
1.1
16.5
12.8
Table 4
On post-graduate programmes
University
Pre 92 (A)
Pre 92 (B)
Pre 92 (C)
Post 92 (D)
Post 92 (E)
TOTAL
Total Number of
external examiners
140
45
88
63
20
356
% of external
examiners from nonacademic settings
9.3
8.9
3.4
1.6
10
6.5
4.1.1 Whilst external examiners working in non-academic settings appear to be in
the minority -whether the programme is undergraduate or post-graduate- there
were nearly twice as many working on undergraduate programmes in terms of
total numbers in the study. However this pattern is not reflected in the distribution
between the programmes of individual universities. Rather there is considerable
variation between individual universities even in this small sample. Furthermore
the study did not show any distinction between pre and post 92 universities in
their appointment of external examiners from non-academic settings. Although, as
stated, the study confirms that this is a minority group within the external
examiner population a total of 65 external examiners from non-academic settings
in a sample of only 5 universities is not inconsequential and supports the necessity
to recognize the needs of this group.
4.1.2 It is commonly accepted that external examiners from the professions and
industry are most likely to be sought by professional/vocational programmes. The
University Registries in the study initially provided a breakdown of the
School/Faculty external examiners were attached to. The distribution of non
academic external examiners between different disciplinary areas according to
this breakdown is shown in Table 5.
Table 5 Disciplinary areas
University
Pre 92 (A)
Pre 92 (B)
Pre 92 (C)
Post 92 (D)
Post 92 (E)
School/Faculty
3 Architecture
3 City Planning
2 History &
Archaeology
4 Media & Cultural
Studies
4 Medicine &
Dental
1 Nursing
1 Pharmacy
10 Arts
7 Social Sciences
1 Human Sciences
5 Health Sciences
2 Engineering
2 Health & Soc
Science
1 Business
4 Applied Science
2 Art & Design
3 Education
7 Health & Social
Sciences
1 Lifelong Learning
2 Sport & Leisure
4.1.3 It is difficult to draw any firm conclusions from this information owing to
the different categorization of Schools/Faculties across institutions. Also one
Registrar highlighted the difficulty of identifying external examiners “not
working in a higher education academic context” particularly in the fields of
medicine and other health-related disciplines where “the lines between higher
education and the professions are often intertwined”.
4.1.4 Nevertheless the distribution does show that whilst a significant number are
working in health professions there is quite a wide spread of such ‘non-academic’
external across a range of disciplinary fields.
4.2 Experience of external examiners not working in a higher education academic
context
A number of the concerns regarding support for external examiners not working
in a higher education context derive from the contention that all external
examiners require contextual knowledge gained from experience of higher
education. The survey thus sought information regarding prior experience.
4.2.1 In terms of prior qualification 52% of respondents had a first degree; 38%
had a Masters degree; 3.4% had a Master of philosophy and 3.4% had a PhD.
69% had a professional qualification. It may be expected that a high proportion
of this group held a professional qualification, although it may be of some
concern that only around half the group had experienced taking a first degree.
4.2.2 Only 9 (31%) of the group had experience of working in higher education
(apart from the external examiner role), 20 (69%) had not. Only two of the 9
had experience of working in higher education on a full-time basis. 5 of the 9
reported having experience of course design and curriculum development and 4
of the 9 reported having involvement in examination boards.
4.2.3 It therefore appears that even though a great deal of literature states the
necessity of contextual experience for example, curriculum design and the work
of examining boards, very few of the external examiners surveyed had such
experience other than through external examining itself. The only significant
prior experience mentioned was previous external examining. 21 of the 29
reported having such prior experience and in all cases this was on
professional/vocational courses.
4.3 Current duties of examiners
There is also a strong argument in the literature for external examiners not
working in the higher education context to have a circumscribed remit
specifically concerned with professional issues. The survey therefore requested
information regarding the expectations of their current role.
4.3.1 It was reassuring to find that 22 (75.9%) reported that they worked as part
of an examining team and only 6 (20.7) were not part of a team. (One did not
reply to this question).
4.3.2 Examiners surveyed were also asked to indicate which area(s) of work
they took responsibility for. 24 (82.8%) indicated responsibility for
professional issues in course content; 18 (62.1%) reported taking responsibility
for academic course content and 11 (37.9%) said they took responsibility for
professional practice e.g. placements.
4.3.3 The following table shows whether the examiners took sole or joint
responsibility for the above areas of work.
Table 6 Responsibility for areas of work
Professional issues
Academic course content
Professional practice
Sole
8 (27.6%)
7 (24.1%)
1 (3.4%)
Joint
16 (55%)
11 (38%)
10 (34%)
4.3.4 Although the high percentage of joint working is reassuring there are still
seven people working on their own in an area where they may not have current
contextual understanding and therefore the support given is critical.
4.4 Induction and support needs
Examiners were asked to rate those support processes they had received.
Information was gathered regarding the support received and regarding their
evaluation of these processes. All processes were rated on a five-point scale.
4.4.1 A large majority (twenty eight) reported having received written
information and relevant documentation at the commencement of their
appointment. Of these thirteen found it very useful (rated 5) and ten found it
useful (rated 4). Five respondents rated it 3 and no respondents rated it below 3
(although one did not offer a rating which according to the question wording
would suggest they had not received such information). It therefore appears that
written information and documentation is experienced as very helpful.
4.4.2 There was a much wider spread of responses regarding experiences of
institutional events for new examiners. Only nine indicated that they had
experienced such an event (although this does not necessarily imply they were
not offered such an opportunity). Three rated this as 5; two rated it as 4; three
rated it as 3; and one rated it as 2. While difficult to interpret this response
without additional investigation it is worth noting the low number of examiners
receiving such support and that this support is not universally viewed as helpful
(as, for example, documentation is).
4.4.3 An institutional event for all examiners (i.e. not just new examiners) was
only rated by two respondents and it is therefore difficult to draw any
conclusions other than the obvious one that few of the survey respondents had
experienced such an event. However the two that did rate this experience found
it very helpful (rated 4 and 5).
4.4.4 More encouragingly twenty two respondents had experienced a visit to the
department and discussion with members of the course team. Ten found it very
useful (rated 5) and a further ten rated it 4. The remaining two rated it as 3.
This is obviously a well-established practice and comments made in the
questionnaire suggest that it was seen as a valuable opportunity not only to
develop working relationships with colleagues but also to gain a grounding in
the working practices of the institution.
4.4.5 Four respondents indicated they had experienced shadowing an outgoing
examiner. Two rated this as 5 and two rated it as 4 suggesting that while this
experience was comparatively rare it was viewed as a very helpful process.
4.4.6 In a similar vein mentoring by an experienced examiner was reported as
received by four respondents. Three rated this as 5 and one rated it as 3.
4.5
Required aspects of support and induction
In addition examiners were asked to comment on what they found to be the
most useful aspects of induction and support and what other information or
support they believed would be helpful.
4.5.1 Nine respondents indicated that meeting the course team was the most
useful aspect of their induction. Three specifically mentioned shadowing in one
form or another, for example, shadowing the outgoing external or observing
examination boards. Another specifically mentioned the usefulness of
networking with other externals within the professional area. Two mentioned
the importance of clear documentation and two others the importance of having
an opportunity to discuss the role. Finally one external mentioned the
usefulness of receiving specific support from the department in response to
questions asked.
4.5.2 In terms of the other support that could be helpful a concerning response
from two externals was that some form of formal induction was necessary, with
one adding “regarding what is required e.g. the content of examiner reports”.
4.5.3 Five felt they needed more information about their responsibilities and
duties with one stating that this needed to be more than just copies of
regulations and student handbooks and needed to be tailored to the external
examiner role. Six suggested contact with the previous and/or other examiners
would be helpful. Two specifically mentioned wanting more information on the
course content and context of the programme. One of these wanted to know
about “course details -- learning outcomes, assessment, marking grids” and the
other wanted to have an explanation of “the systems”.
4.5.4 All the above comments suggest some lack of clarity in role expectation
and confirm the need for additional support arising from unfamiliarity with the
higher education context. One respondent explicitly commented on feeling
unqualified in the area, adding “I’m not within a university so am not aware of
recent developments”. Another acknowledged “… without previous experience
as a course leader I would have really struggled based on information
provided”.
4.5.5 The responses to this question indicate a range of experiences with
universities providing varying levels of support.
4.6 Difficulties in performing the role
Given the presumed problems of external examiners working in non-academic
settings, the survey requested information regarding difficulties in performing
the external examiner role.
4.6.1 Twelve stated there were no difficulties experienced, two were equivocal
in stating “not yet” and “too soon to tell” and a further three did not respond to
this question.
4.6.2 Seven expressed concern about the need “to turn work around quickly”.
This concern was either related to receiving work later than agreed or to the
time expectation of universities being shorter than was felt appropriate. A
related theme was that one examiner indicated they were given insufficient
notice of dates and times of exam boards which meant they were unable to
attend.
4.6.3 A further two examiners had concerns about there being a lack of response
to queries. One indicated that this was related to the wording of examination
questions, and the other, interestingly, mentioned difficulties in receiving a
response to administrative related queries. While academic staff were helpful in
this instance they were unable to offer guidance on particular policies and
procedures. (Although this is only one instance it is worth noting in passing that
the concern about a lack of procedural understanding may not only apply to
those external to higher education).
4.6.4 Finally one examiner again complained of insufficient clarity regarding
the role.
5. Conclusion
5.1 The standard offering of most universities in terms of induction for external
examiners appears to be provision of documentation and a visit to the course team.
Both activities, when well executed, undoubtedly make a significant contribution to
supporting new external examiners. Those external examiners that come from
outside the academic context indicate that they require more than this in order to
obtain the information that is sometimes assumed by those working in higher
education.
5.2 A high level of joint working is undoubtedly a strength of current practice
however even in this small survey it is evident that a number of external examiners
were given sole responsibility for academic content issues. While, in itself, this
may not be detrimental it is not an indication of good practice in terms of support.
5.3 None of the examiners surveyed indicated that they had any problems with taking
responsibility for professional practice issues. Overwhelmingly their concerns were
in the area of what could be called ‘academic practice and procedures’.
5.4 Although some professional groups have established systems to support external
examiners (e.g. social work) these systems tend to focus on practice issues and further
support is likely to be required from the university if there are additional expectations
of the role. Furthermore some externals working outside academic institutions are not
part of such examiner networks and, perhaps for this group in particular, there is a
need for firmer guidance on what should be provided by universities.
References
Ashworth A (2004)External Examining:A guide for those working in the Built
Environment CEBE Working Paper 3
Ashworth A & Harvey RC Assessing Quality in Further and Higher Education London
Jessica Kingsley Publications
Colley H (2005) Report on the Workshops for Staff Supporting the External Examiner
System June
Cowan, J (2004) Guide for Busy Academics: Enhancing Support for External
Examining
http://www.heacademy.ac.uk/resources.asp?process=full_record&section=generic&id=3
81
Jackson, N (2004) Enhancing Support for External examining. Report and action Plan
September
Jackson, N (2004) Working Paper 7 Professional Standards for External Examining
and their Relationship to the Proposed Professional Teaching Standards Framework
Jackson, N (2005) Guide to the Process of Academic External Examining January
Smith, H. and Oliver, M. (2004) A Grounded Theory of External Examining
http://www.ltsn.ac.uk/genericcentre/index.asp?id=20336