LONDON’S GLOBAL UNIVERSITY
RESPONSE by UCL to CL14/05: QAA CONSULTATION ON OPERATIONAL
DESCRIPTION FOR THE REVISED AUDIT PROCESS 2006-11
Introductory Comments:
UCL welcomes the opportunity to comment on the Agency’s operational
description for the revised audit process 2006-11. However, UCL firmly
believes that the Agency’s review processes should be designed according to
assessed risk at each institution, not applied uniformly and we consider that
the document has failed to make clear the QAA’s understanding of the need
to make intervention proportionate to risk. The proposed duration of an audit
visit seems to be unaffected by questions of prior experience or quality, or by
the nature of the information received from the institution. There are a
number of other comments we wish to make concerning the following
proposed areas of change:
Emphasis on Quality Enhancement
(1)
(2)
The emphasis in the Operational Description on quality enhancement
[Introduction Page 2, bullet point 2 and elsewhere], while not in itself objectionable,
seems to be an admission that the QAA methodology to date has not been
effective in promoting QE in the way it has long claimed to aspire to do.
Change in Judgement (from ‘Broad Confidence’ to ‘Confidence’)
(3)
UCL welcomes the proposed alteration in the most positive judgment from
'broad confidence' to 'confidence’ [Introduction Page 2, bullet point 4 and
elsewhere], - but it should be noted that UCL (and others) requested this in
comments on the Operational Description for the transitional period. Though
it is nowhere acknowledged in the document, we are pleased that the QAA
has changed its previous position (which seemed to us as unhelpful as it was
unjustifiable); however, we would like an assurance that the QAA will ensure
that it is made clear to external 'stakeholders' that 'broad confidence' in the
period 2002-05 equates to 'confidence' in future audits. (Otherwise, HEIs will
run the risk of stakeholders assuming that they did not command the full
confidence of QAA auditors in their audit conducted in the 2002-05 period).
Reduced Burden
(4)
The reduced demands placed on HEIs in preparing for Institutional Audit
visits [Introduction Page 2, bullet point 6 and elsewhere] are welcome in principle.
However, UCL would wish to make the point that its benefit in practice will
depend considerably on how consistently the 'light touch' is applied by QAA
officers. In UCL’s experience, some QAA officers are significantly more
willing than others to apply a light touch (which, in the case of our 2005
Institutional Audit, resulted in some mixed messages from the Agency’s
officers - and, as a result, additional workload for UCL).
(5)
UCL welcomes the fact that the institutional briefing paper will bring with it a
reduced expectation in terms of 'self-criticism'. Our own experience
suggests that one of the most useful purposes a paper of this kind can serve
is to describe the structures and processes of the HEI - not least because
audit teams are not always able to read the full range of background
information which the HEI provides to supplement its 'self-evaluation
1
document' or equivalent. This proved to be the case in the 2005 Institutional
Audit of UCL - as the auditors themselves admitted - and resulted in:
(i)
the audit team's needing to ask questions to which they would have
known the answers had they read more of the material supplied to
them;
(ii)
a limited understanding of UCL’s processes, which persisted in the
draft report.
(6)
UCL would wish to question the references to 'reducing' the size of the team
[Impact Analysis, Page 12, box 3 and elsewhere]. In our own experience, there
have never been more than three institutional auditors on the team in any of
the three audits we have undergone since 1993. If the reduction takes into
account the fact that there will no longer be discipline auditors, then this
should be made explicit in the Operational Description.
(7)
UCL welcomes the decision to discontinue Discipline Audit Trails (with the
important proviso that the coverage of thematic trails in future is light touch).
This is welcome confirmation that the Agency is moving away from ‘quasiSubject-Review’.
(8)
Finally, in the context of reduction of burden, UCL wishes to comment on the
lack of clarity concerning the role of the mid-cycle review. The absence of a
clear sense of the purpose, scale and outcome of this proposed element has
the potential to create additional work and offset the promised reductions in
the burden of assessment.
Emphasis on Examining Arrangements for Postgraduate Programmes of Study
(9)
UCL notes the proposed greater scrutiny of institutional arrangements for
maintaining appropriate academic standards of postgraduate research
programmes [‘Outline of the Process’, Page 3, Paragraph 3 and elsewhere]. We
have some concern about the potential for unnecessary duplication arising
from the new emphasis on research programmes and wish to emphasise the
need (not noted anywhere in the Operational Description) for the QAA to
collaborate with and draw on the expertise of the organisations already
involved in monitoring provision in this area (e.g.; Research Councils).
Provision of Information
(10)
UCL wishes to express reservations concerning the amplified requirements
for provision of information as set out in the document [‘Information’, Page 5,
paragraph 14 and elsewhere]. This is premature in view of the fact that the
stated aim of the second phase of the review of the National Quality
Assurance Framework is the collection of institutional views on TQI and the
NSS. Similarly, the proposed emphasis on the use of Programme
Specifications [paragraph 45] is premature in view of the fact that the QAA
have not yet confirmed guidelines on Programme Specifications which clarify
their purpose – guidelines which the Agency has agreed to be needed in the
face of evidence of a lack of shared understanding across the higher
education sector as to whether Programme Specifications should be
regarded either as:
(i)
a source of information to students/prospective students;
(ii)
an internal planning tool for institutions.
13.01.06
2