An Coiste Feabhais Riarachán agus Seirbhísí
The Committee on Administration and Services Quality Improvement
The Administration and Services Quality Assurance Programme
2005 – 06
REVIEW OF
THE EXAMINATIONS OFFICE
FINAL REPORT
23rd March 2006
2
Examinations Office: Review Report 2005–06
This report arises from a visit by a review group to the Examinations Office on the
22nd – 24th November 2005. The Examinations Office had already prepared and
submitted a 'Self Assessment Report' that, with other documentation, was made
available to the review team in advance of the visit.
The Review Group consisted of: Mr Derek Ord, Head of Student Administrative
Services, University of Hull (Chair); Ms Mary McDonald, Student Records &
Examinations Office, University College Cork; Dr John Newell, Department of
Mathematics, NUI Galway; and Ms Nuala McGuinn, Adult & Continuing Education
Office, NUI Galway acting as Rapporteur.
The report is structured to cover the following main topics:
1. Aims and Objectives
2. Organisation and management of the Office
Recommendations for Service Enhancement
3. Communication
Areas of Good Practice
Recommendations for Improved Communication
3.2.1
Academic Community
3.2.2
Student Body
3.2.3
Administrative Services
4. The Wider Context
Academic Community
Student Body
Data Issues
4.3.1
Recommendations for Improvement
5. Summary and Concluding Remarks
25/11/2008
3
Examinations Office: Review Report 2005–06
1.
Aims and Objectives
The Quality Review of the Examinations Office spanned over two days and consisted
of meetings with staff of the Examinations Office itself, members of the University
Management Team, Administrative Officers and Heads of Student Support Areas in
addition to undergraduate and postgraduate students of NUI Galway.
The review group identified three main objectives for its review visit outlined as
follows:
2.

To ascertain the level of satisfaction and the effectiveness of the support
service provided by the Examinations Office in the organisation of written
examinations, the processing of examination results (including appeals and
rechecks) and the processing of postgraduate theses.

To assess the functions of the examinations office and identify if the current
practices are in line with best practice, identifying areas of good practice and
areas for improvement.

To explore the extent to which University structures and processes within the
administrative and academic community impact on the service as provided by
the Examinations Office.
Organisation and Management of the Office
During the course of meetings with University academics and student support staff it
was widely recognised that the service had improved dramatically over recent years.
This was achieved through the following:
1.
The appointment of a new Examinations Officer, who has initiated and managed
change within the office and the improved use of the new IT System.
2.
The setting up of team structures in the Office to manage core functions has
contributed in a very positive way. It devolved power and created a shared
responsibility. This enabled ownership of projects and provided essential cover.
It contributed to clearly defined job roles.
3.
The reduction in overtime has lead to improved working conditions. This has
been achieved by the increased compliance to deadlines for the submission of
papers and marks, the electronic return of examination marks and the increase in
staff support in the Examinations Office.
4.
The Office has started the practice of having internal review meetings after each
examination session. This is an important function for updating processes. These
meeting are set to continue which is a positive development as they also provide a
forum for putting forward new ideas which will further improve the service.
5.
The Web is the main medium for communicating information to staff and
students. Currently, students can access examination timetables and results
(specific courses only) on the Web and there is staff access to examination
25/11/2008
4
Examinations Office: Review Report 2005–06
timetables. The Examinations Office Web Page carries a lot of relevant
information for both academic staff and students.
2.1 Recommendations for Service Enhancement
Further improvement of services can be achieved by:
1.
Establishing a management responsibility role for Customer Services by
assigning a Senior Staff Member to manage all facets of customer services.
2.
In view of the projected growth area within postgraduate programmes,
establishing clearly defined documentation for students and their supervisors on
the submission of theses and creating a more defined customer presence for this
function within the Examinations Office. While much of this documentation is
already available on the web, there is still a misconception among the student
cohort about submission requirements which may indicate the need to review how
this material is presented to students in the first instance.
2.
Establishing a system to ensure that telephone, e-mail and all communications to
the office are replied to within a stated period of time. This recommendation is to
address the perception of students that it was difficult to communicate with the
Office.
3.
The current configuration of office space is not conducive to optimal use. To
address this issue the layout of the office should be architecturally and
ergonomically evaluated. An open-plan office would be very beneficial and make
for better use of the available space.
4.
A previous review has proposed the amalgamation of the front of house offices
of Admission, Fees and Examinations. Pending implementation of this proposal,
as an interim measure, a notice should be placed in the open waiting area, facing
the stairs, listing the services provided by each office.
5.
Further enhancing the use of the Examinations Office Newsletter to keep
customers informed about new developments within the Office and also
informing them of what types of information are available on its Website.
3.
Communication
3.1 Areas of Good Practice
Many areas of the University, including students and academic departments have
acknowledged the recent improvements in the Examinations Office web site. The
content particularly has improved and is felt that it provides more useful information.
An associated improvement is the fact that the web site has now been advertised to
students, and they are now positively encouraged to use electronic means to
communicate with the University.
A recent improvement in the service to students is the provision of their assessment
results electronically. All of the staff involved in this project, both within the
25/11/2008
5
Examinations Office: Review Report 2005–06
Examinations Office and in other areas should be commended for this very positive
and student-focused work.
Surveys that the Examinations Office have performed recently of staff and students
are a very good feature. Such surveys should be performed on an occasional basis in
the future, to ascertain whether any changes have made an effect. They may also
prove beneficial in seeing what changes might usefully be made in the future. The
continued contact with their “customers” is to be encouraged as much as possible.
Opening hours have been extended, in order to give a better service to students, which
is a good example of customer focus.
The Examinations Office has delivered some training for academic departments
recently, and have plans for more. This direct communication with academic
departments is a very positive move and should be encouraged. Heads of Academic
Departments and Deans of Faculty have commented that communication with the
Examinations Office has improved, and as a result, their perception of the whole
assessment system has also improved. Another area of training performed recently
has been the health and safety training of invigilators. This is obviously a very
important aspect of the running of exams and it is pleasing to see the proactive
attitude taken by the Examinations Office in this respect.
3.2 Recommendations for Improved Communication
There are several possible areas of improvement with regards to communications,
none of which are serious and none of which should be taken as major criticisms.
These recommendations fall into three broad customer categories, the Academic
Community, the Student Group and Administrative Services.
3.2.1
Academic Community
Academic departments in particular are very keen to have more guidance on the
assessment and results process. They recognise that communication has improved
with the availability of training courses on how the examination systems operates, but
would also welcome this communication and dialog continuing, so that they may feel
better supported and in turn better support the Examinations Office. The departments
indicated that they sometimes learn systems by “trial and error”, and subsequently
make mistakes. The Examinations Office could usefully be involved in this process,
assisting departments with their problems. There is also the issue of learning the MIS
system, which could well be a joint challenge for the Examinations Office, the
Records Office and MIS to work on. While much of this work cannot be undertaken
by the Examinations Office in isolation, but rather in partnership with other key
offices, the Examinations Office themselves have recognised that they perhaps need to
explain better to academic departments the consequences of their actions; particularly
the non-compliance with deadlines, but also in other areas.
3.2.2
1.
Student Body
Postgraduate students in particular found the thesis submission process to be
unclear, and would welcome some guidance in this area. While information in
this regard is already available on the Examinations Office website, it is apparent
25/11/2008
Examinations Office: Review Report 2005–06
6
that students are not receiving the message in the manner it is intended. This
issue may be one of how the information is presented as opposed to the
information not being available. This transfer of information could be facilitated
by a simple sheet, or web page which outlines the basic process of thesis
submission and the subsequent stages in approval, together with some guidance
timescales for each stage. This could be best brought about in consultation with
the students themselves, both those who have yet to submit, and those who have
already submitted. This should not be a large task, but would have a large
beneficial effect.
2.
There is a perception that the registration statement is poorly laid out and may
be confusing for some students. This is a natural consequence of the desire to
give the student as much information as possible as they start the year. It would
be beneficial if this registration statement was redesigned with the student in
mind, to make it clearer and to highlight the important information. There may
also be an issue with the timing of the information given. Students tend to suffer
from “information overload” at the start of an academic year, and there is merit in
being selective in the information given at that time of year, with a view to giving
other information as it is needed, later on in the year. While it is acknowledged
that the design of the registration statement is not the direct remit of the
Examinations Office, it has an impact on the workings of this office. In this
regard, it is considered beneficial if the Admissions Office and Examinations
Office worked in collaboration on designing a more customized statement for
students.
3.
The student groups did raise some issues related to communicating with the
Examinations Office, particularly at busy times. There were cases of students
unhappy with their face-to-face service, with a lack of a timely response (or any
response) to emails, phone calls unanswered and messages not replied to. It is
difficult to ascertain the extent of these problems, and it may well be that these
are isolated cases. There is certainly little evidence of a major and widespread
problem. However, the Examinations Office, as for any office with contact with
customers, must ensure that its direct communication is as best as it can possibly
be. It is suggested that the customer response times be examined to see if there is
any room for improvement, or if there is a better method of capturing the
problems before they escalate.
4.
The Examinations Office reported that students often came to the wrong service
window for attention. Similarly, students themselves mentioned that they did not
always know where to go. This problem could be easily reduced by posting
more information beside the service areas, in addition to the office titles.
5.
Students raised the issue of being unaware at the start of their course of the
assessment requirements, of what the basic structure of their course was (e.g.
definition of credits, etc), what pre-requisites were needed and the contribution of
each assessment to their overall mark. It may be the case that students are
supplied with this information, but it may not be in the most appropriate format,
and at the most appropriate time. It would be worth reviewing, with other offices
(as this is not simply an Examinations Office issue) the information provided to
students, perhaps by including students in the review. Other institutions have
found it very worthwhile including students in the design process of information
25/11/2008
7
Examinations Office: Review Report 2005–06
that is aimed at students and this is an approach that could usefully be employed
here.
6.
Students also requested more possibilities of contact via the web, particularly in
requesting services and making payments. The University would be well advised
to consider going down this route, not only to provide a better service to students,
but to improve their own efficiency. Students also expressed uncertainty about
the appeal and related processes, and this should be examined to see if this
process could be better explained and made more visible to students.
3.2.3
Administrative Services
1.
The communications link with MIS is an area where greater cooperation might
bring benefits. Although it is recognised that the two areas are communicating
regularly, the review team did get the impression that the two areas were not
always thinking the same thing with regards to priorities and responsibilities.
These issues in particular would benefit from closer communication.
2.
There is a technical problem in that it is very difficult at the moment to email
groups of students in a meaningful and efficient way. The review group was
informed that a technical solution is expected, and we would encourage work in
this area, so that email can be used more as a direct communication tool to
students.
3.
The use of some kind of contact management system would help the
Examinations Office in this regard. Ideally, such a system would be used by all
offices in the same area, so that a full history of student contact with the area
could be kept. This will also allow the various staff within the offices to know
who has had contact with students in the past, on what issue, and what the
response was. It would also assist in monitoring the questions and queries posed
by students, so the office could then best focus its services. For example, if a
large proportion of queries are on a specific subject, then this system would
highlight that, and the Examinations Office might then be better placed to provide
that information in a better way.
4.
Other administrative areas have welcomed the chance to take part in this review,
and as a result, have requested a closer working relationship with the
Examinations Office in the future. They would welcome occasional, but regular
meetings, to discuss areas of common concern.
5.
It would be desirable to introduce a separate “user group” forum, including staff
from all service areas to complement the existing Exams Review Group forum
which already exists and includes officers such as the Deans, Registrar, Academic
Secretary, Admissions and Record Officers, Faculty Officers and Student Union
Representatives. The purpose of this forum group would be to provide a
discussion platform for issues and problems to be raised in an informal manner.
Issues to be addressed include “sick bay” arrangements, dealing with last minute
deferral issues, management of examination venues etc. This kind of group need
only meet once or twice a year, but would be a positive and easily implementable
step.
25/11/2008
8
Examinations Office: Review Report 2005–06
4.
The Wider Context
4.1 Academic Community
There was no evidence of any negative opinion of the Examinations Office personnel
and function within the academic community, as represented by the Deans and
Administrative heads. The general consensus was that the personnel within the
Examinations office are co-operative, very efficient and very helpful. In particular
the level and continued improvement of the service provided by the Examinations
Office was recognised with particular reference to electronic submission of exam
marks and electronic broadsheet preparation, the streamlining of sick bay, the new
seat numbering system and the provision of online timetables and marks.
Difficulties that arose from the academic community representatives related to the
examinations system itself and not the actual Examinations Office. There was
considerable frustration evident with the consequence of bonding, where timetables
are scheduled by course instance and not by module. This results in a cumbersome
timetable which is becoming a serious issue given the continual increase in the
number and choice of new modules and courses offered to students. It was
recognised that this system adds considerable constraints when seeking an optimal
timetable and typically involves many additional manual interventions to the software.
The lack of a central policy on marks and standards, and lack of consistency relating
to exam deferrals and sick bay policy was raised and the requirement of a central
policy recognised. The amount and nature of assessments was considered too high
and dissatisfaction expressed with the system of assigning credit weightings.
Concerns were raised regarding the student records system which is not functioning
for the academics in terms of access to view marks and the ability to create custom
reports.
While the Review Group widely acknowledges that many of the above issues fall
outside the direct remit of the Examinations Office, they do have an impact on the
workings of the office and how it is perceived among the student and academic
community. The Review Group recommends a wider sharing of expertise with
regards to assessment - an area that the Examinations Office may usefully take part
in, perhaps by organising exams or assessment forums, giving a chance for the
departments and the Examinations Office to meet on a more informal basis, to share
ideas and experiences.
4.2
Student Body
The student body were represented by elected student and class representatives.
Recognition of the continued improvement in the Examinations Office was evident
with particular praise directed towards the invigilators. There was considerable
dissatisfaction regarding the number and location of the exam venues and the desks
and heating therein. It was clear that a proportion of the student body tends to remain
uninformed despite the relevant information in place on the Examinations Office
website. Frustration with the use of Áras na Macleinn as an exam venue, thus ruling
out the use of the hall in its primary purpose as a sports hall was clear. There was a
lack of understanding of the Examinations Office role amongst the student body
representatives as there was a tendency to mistakenly associate most of the complaints
25/11/2008
9
Examinations Office: Review Report 2005–06
(e.g. timetables issues) directly towards the Examinations Office rather than the
examinations system.
The Review Team highlight a numbers of areas where the Examinations Office could
have a positive and immediate impact on student welfare, these include:
1.
2.
3.
4.
The provision of more suitable exam desks;
A justification as to the price of Transcripts;
Clearer guidelines as to the sick bay regulations;
Clearer information regarding which form of identification students
should use on exam answer books (i.e. Pin number or student ID number).
The procedure for making past papers available to students is very inefficient as
highlighted by both the Students Union Office and the Library. At present all papers
are scanned in the Examinations Office as only a small number are being sent in
electronic format by the various academic departments. The Review Team
recommends that a mechanism for electronic uploading and categorisation of all
examination papers is required. This could form the basis of a joint project with
MIS driven by the Examinations Office.
Areas identified by the students which are not in the direct control of the
Examinations Office and consequently more difficult to resolve included transport to
and from venues and the inconsistency in the amount of assessment relative to actual
credit weighting.
4.3
Data Issues
One of the most important issues for an Examinations Office is information. The data
that they use, and produce, must be 100% accurate, and be produced in a useful and
timely manner. Individual student’s assessments are dependent on this data being
correct, and the burden on the Examinations Office and other areas is significant in
this respect. The new seat numbering system has improved things considerably in the
exam venues. This is a result of the improvements brought about by the exam
scheduling software, which is more flexible than the manual process exercised in the
past.
Mark submission and broadsheets have both improved immeasurably, which in turn
has led to a great improvement in the efficiency of board meetings. Electronic
submission of marks, on-line results and timetables are all examples of where data
have been processed to a high standard, to improve the service to students and other
areas of the institution. All staff that were involved in these projects should be
commended for their hard work in bringing these projects to a successful conclusion.
4.3.1
Recommendations for Improvement
The student records system is not functioning to its best for the academic areas.
Their access to information is limited, and the academic departments struggle to have
a full view of their own student information. They sometimes feel constrained by the
system, rather than empowered. There is a fundamental question as to who owns the
data? Certainly, the system may be owned by those entrusted with its smooth
operation, but the data themselves should be clearly owned by defined groups, and the
power over that data put in those areas’ hands. The result of the problems in academic
25/11/2008
Examinations Office: Review Report 2005–06
10
areas is that academic departments produce their own duplicate data systems to run
their programmes. This must be seen as bad practice, as it may lead to inaccuracies or
omissions. The Review Group acknowledge that these problems are not the fault of
the Examinations Office, but they often have to fix the problems caused, and any
errors in information usually fall on them to resolve. In this regard, it is recommended
that the Examinations Office liaise with the Records Office on these issues as they
influence the day to day operations of this office and how it is perceived within the
wider University context.
The examination timetable is produced by course instance and not by module. There
are multiple venues (perhaps too numerous?) and an excessive number of exams. All
groups that the review group met indicated that students are over-assessed – this
is an urgent and increasingly burdensome problem. There are also very complicated
ways of bringing courses together (i.e. bonding is just one very obvious problem that
was mentioned by all groups that we met). This leads to a very complicated set of
rules being used to schedule the exams. Add to that, the constraints imposed on the
system, such as the desire to spread exams out as much as possible, and to use as few
venues as possible and it is hardly surprising that there are a large number of manual
fixes that must be made to the timetable. Even after all that work, the timetable is not
ideal, but it must be mentioned that an excellent job is done in producing a working
timetable in these circumstances.
None of the above constraints are imposed by the Examinations Office, but rather are
imposed upon them, and it is therefore imperative that all areas of the University are
made aware of the later impact of their decisions earlier on when they are deciding
how to put their courses together, and how to assess. It is not the intention that the
exams system should dictate how assessment is done – that must remain an academic
decision – but these other factors should be borne in mind when making that decision.
It is also imperative that Examinations Office and MIS continue to work closely
together to assist each other and the rest of the University in ensuring that IT is used to
best effect in improving the service to students and staff. The relationship between
the Examinations Office and MIS would benefit from greater clarity as to their
respective roles in specifying improvements to systems and priorities in those
improvements, taking into account the overall demands on MIS staff.
5.
Summary and Concluding Remarks
The Examinations Office is responsible for the central administration of the
University examinations. It involves the organisation of written examinations, the
processing of examination results and postgraduate theses. It provides an elaborate
support service to a complex centralised examination system for all Faculties and
programmes at NUI Galway. The scale of service currently offered by the
Examinations Office has increased significantly over the past number of years due to
the extraordinary growth in programmes, student numbers, the extension of
semesterisation and increased modularity of programmes. All of which provide an
expansive range of modules for students to choose from. The Examinations Office has
responded well to such changes, particularly at a time of significant staff changes and
the introduction of a new student records system.
25/11/2008
Examinations Office: Review Report 2005–06
11
The Examinations Office has transformed itself by introducing a new team structure in
terms of its own internal organisation, carries out review meetings with key
stakeholders for continuous improvement and has enhanced its use of information
technology in how it carries out its core functions (i.e. electronic submission of a
certain percentage of exam papers, electronic return of marks by academics, relay of
student marks via the web, and a concentrated focus on the web as a central
information point for students).
In meeting with key customer groups, the Review Team were impressed with the level
of respect that the Examinations Office holds within the University. There was a
widespread feeling that the examinations system itself and not the Examinations
Office was in need of a significant overhaul. Recommendations in this regard related
to the requirement for a University wide policy on deferrals, guidelines on sick bay,
standardisation of marks and standards, introduction of an assessment tariff for
examinations, a review of the practice of bonding examinations across Faculties and a
review of the student information service as provided to academics by the current
student records systems. The Examinations Office can play an active role in advising
the academic community on best practice in a number of these areas. In particular, it
has a role in advising on programme assessment and the impact that specific
requirements may have on the examinations system and assisting in the development
of University policies in relation to deferral, sick bay etc. There is, however an added
requirement for “buy-in” by all departments and University management if such broad
policies are to be adopted widely.
Contact with the student body highlighted a significant area of improvement for the
Examinations Office in terms of enhancing communication with this group. While
much has been achieved in the area with increased use of the web as a central point of
information, a number of misconceptions were found among the student group in
relation to what information was available to them and where. These issues may be a
matter of how material is presented, marketing strategies in use etc , but all highlight
the need to review how the Examinations Office can best capture the attention of its
target market. In this regard, an increased emphasis on customer service is
recommended with the establishment of management responsibility for this function
within the Examinations Office. The requirement for clearer guidelines for students
on examination procedures (both undergraduate and postgraduate), clear
communication of sick bay and deferral procedures, earlier posting of examination
timetables (where feasible and possible), correct use of PIN and student ID numbers in
relation to examination issues etc are required. Enhanced use of the web in
communicating this information may be effective, however alternative marketing and
PR activities must be explored in order to “reach” the student group.
Significant improvements were noted in the use of information technology in the
management of the examination function. However, a number of data related issues
still exist which have an impact on day to day operations. Examination timetabling
has been noted as problematic for the office, even with new software. An electronic
system of categorisation, storage and provision of past examinations papers to
students is also required. Through the course of the review meetings issues with
Quercus were referred to by the academic community which highlight that this system
is not fully servicing their data requirements. A contact management system is also
recommended for the Examinations Office to effectively manage their customer
service function. The communications link with MIS was noted as an area where
25/11/2008
Examinations Office: Review Report 2005–06
12
greater cooperation might bring benefits. It was acknowledged that the prioritisation
of key service functions and responsibilities between both groups would benefit closer
communication between these areas.
It must be recognised that many of the improvements that could usefully be made by
the Examinations Office are quick and of a low cost. Other changes may be more
complicated and carry a higher initial cost, with savings coming over the longer term.
It is therefore essential that the University continues to ensure that the Examinations
Office, as for all administrative areas, are funded appropriately.
Finally, the Examinations Office in their own self-assessment report have described
themselves as too accommodating. Perhaps inhere lies an opportunity. The
Examinations Office have gained immense respect from its customer groups over the
past number of years. There is an opportunity for them to interact more proactively
with these groups in highlighting their requirements and the constraints which
programme structures impose on the examination systems. As much of the inputs to
the examinations system in terms of examinations papers and results come from
outside their remit, it is essential that the Examinations Office begin to articulate itself
within the wider academic community and engage proactively with this group to
achieve widespread change. The Review Group wish them every success in this
venture!
Mr Derek Ord, (Chair)
Ms Mary McDonald,
Dr John Newell
Ms Nuala McGuinn,
25/11/2008