INSTITUTIONAL AUDIT – MID-CYCLE FOLLOW-UP BRIEFING PAPER Preparation of this briefing paper 1. This briefing paper was prepared by the officers of UCL’s Quality Management and Enhancement Committee (QMEC) and endorsed by members of QMEC (who include the UCL Union’s Education and Campaigns Officer and a student nominated by the UCL Union). The final version of the paper was approved by the Vice-Provost (Education) on behalf of UCL. Preparation of UCL’s internal Institutional Audit action plan 2. In advance of the 2009 Institutional Audit (IA), UCL took the deliberate decision that it did not wish to treat the IA as simply an external compliance-testing exercise. Instead, it wished to use the IA, and in particular the process of preparing for the IA, as an opportunity to reflect internally on issues relating to academic standards and quality assurance and to maximise the IA’s potential as an aid to the enhancement of the student learning experience. Consequently, as part of the preparations for the IA, which were overseen by a steering group of QMEC (the Institutional Audit Steering Group or IASG for short), committee/administrative officers with responsibility for various aspects of UCL’s operations were asked to participate in an internal selfevaluation exercise. The areas covered by this process were as follows: External examiners; Approval, monitoring and review of award standards; Assessment policies and regulations; Management information (including progression and completion statistics); Approval of new degree programmes; Monitoring and review of degree programmes; Student representation and feedback and approach to the NSS; The role of students in quality assurance and enhancement; Link between research or scholarly activity and learning opportunities; Learning resources; Admissions policies; Student support; Staff support, development and reward; Dissemination of good practice; Support for postgraduate-research students; Accuracy of published information; Teaching Quality Information; Review of internal quality assurance review processes. 1 3. A designated officer (usually the relevant administrative or committee officer) was identified for each of the above areas. The designated officers were requested to review, in consultation with other UCL staff as necessary, the area in question and then produce a brief self-evaluation report addressing the following key questions: What are the main documents which explain UCL’s structures/processes/policies/procedures/guidelines etc in relation to this area of activity? Are the above documents available internally, externally, or both? How are UCL’s structures and processes in this area of activity reviewed? Who or what is responsible for undertaking those reviews? What evidence is used to assess the effectiveness of the structures and/or processes in this area of activity? What use is made of external reference points or benchmarks? What are the main strengths of UCL’s current structures and/or processes in this area? Are there any ways in which these structures/processes might be improved? If so, how? Are there any examples of good practice which have been identified in this area of activity? If so, what are the mechanisms for disseminating that good practice more broadly within UCL? 4. The self-evaluation reports were scrutinised by IASG and were an important source of information in preparing UCL’s Institutional Briefing Paper (IBP) for the IA. Importantly, in the context of this briefing paper, the reports also identified scope for improvement across a range of areas. It was intended that UCL would return to those areas and explore in more detail ways of strengthening them following the IA. 5. Following the IA visit and once the final IA report had been received from the QAA, IASG was re-convened by UCL’s Academic Committee (AC), with a slightly reconfigured membership, as the Post-Institutional Audit Steering Group (or PIASG for short). Its purpose was to oversee UCL’s response to the specific recommendations in the QAA’s IA report and to address a range of other issues which had emerged as a result of the IA process - including the areas for improvement identified in the self-evaluation reports prepared by the designated officers. 6. PIASG prepared a comprehensive action plan which identified action points arising from the following sources: the specific actions recommended by the QAA auditors in their report; issues which had been identified by the UCL Union in its Student Written Submission; issues which had been identified by the QMEC officers in preparing the IBP and in preparing for the IA specific areas for improvement which had been identified by the designated officers in their self-evaluation reports. 7. The action plan detailed 80 separate points of action: 2 of them arising from the QAA IA report with the rest having been identified internally. A copy of the final (July 2011) version of the PIASG action plan is at Annex A. (The action plan is also publicly available on the UCL website at http://www.ucl.ac.uk/ras/acs/qme/qmea-z/ia.) 2 Implementation and monitoring of the PIASG action plan 8. Once the action plan had been approved by AC, QMEC was kept informed as to progress in implementing the various action points. QMEC, in turn, submitted an annual report to AC, focusing latterly on any remaining areas where action was outstanding. A note detailing QMEC’s and AC’s consideration of the PIASG action plan is at Annex B. 9. Although the precise detail of the original action plan has, to some extent, been overtaken by subsequent events, projects and initiatives, all of the actions identified in the action plan have, with one exception (see Para 10 below), been either completed or are of a nature such that the requisite follow-up action is ongoing. 10. The one exception where significant progress has yet to be made relates to an issue that was not identified by the QAA auditors but emerged from UCL’s internal preparations for the IA, namely the creation of single section of the UCL web site for storing key strategy, policy and procedure documents, or ‘policy zone’. The creation of such a zone, which would act as a ‘one stop shop’, is seen as something which would improve access to key institutional information for both staff and students. This initiative is being taken forward by the relevant officers, including the AC officers and colleagues in UCL’s Information Services Division (ISD), within the broader context of the development of a new institutional web strategy and UCL’s general approach to communications. This issue remains on AC’s agenda and a further report will be submitted to the May 2012 meeting of AC. 11. With hindsight, it could be argued that UCL created unnecessary work for itself by producing such a comprehensive action plan which includes issues derived from a number of sources, instead of simply responding to the two recommendations identified by the QAA auditors in their report. Also, there is a risk that such a wideranging document can quickly become out-of-date or irrelevant as it is overtaken by subsequent policy developments, structural and organisational changes, and new initiatives. However, as mentioned above, our view remains that, on balance, the process of preparing for the IA and of responding to the QAA team’s report has provided an additional opportunity for genuine institutional self-reflection across a range of areas as part of UCL’s ongoing quality management and enhancement (QME) strategy. UCL’s response to the specific recommendations in the IA report 12. In their report, the QAA auditors made two recommendations for advisable further action, which were as follows In the light of previous progress on harmonisation, and in order to consolidate further the equivalence of the student learning experience, UCL should maintain its momentum towards achieving the institutional coherence on regulatory and academic processes identified by its own committees. and Where an institutional position has been reached on the harmonisation and simplification of regulatory and academic processes, UCL should seek to achieve full and timely departmental engagement and alignment. 13. UCL accepted the above recommendations as they reflected some of the general themes which were explored with the QAA auditors during the IA. However, as they 3 are couched in general terms, PIASG studied the detail of the IA report to try to find any specific issues which the team had noted which may have contributed to them making those recommendations, in order to ensure that any such points were incorporated in UCL’s follow-up action plan. 14. In relation to the first recommendation, PIASG identified the following: the need for UCL to harmonise the upgrading process for MPhil to PhD [paras 99 & 105]; the need for UCL to achieve the full engagement of students and staff with the Research Student Log [para 105]; students were aware of certain variations in the way different departments dealt with certain operational aspects of their assessment experience including the potentially variable quality and timeliness of the feedback they received [para 36]. 15. In relation to the second recommendation, the IA report identified the following: that the adherence to institutional policy on peer observation of teaching by departments was not universal and identified the need for UCL to continue to work to improve the take-up and reporting mechanisms [paras 72 & 80]; the need for UCL to ensure the training of research supervisors before they undertake supervision [para 104]; UCL’s admission of residual deficiencies in the student record system (Portico) regarding Research Student records [para 104]. 16. All of the above issues were included in the PIASG Action Plan and have been, or continue to be, addressed. A summary of recent action in these areas, in addition to those set out in the July 2011 update of the PIASG action plan, is as follows: Research Student Log - improvements to the Research Student Log continue to be made, eg the skills self-assessment element of the Log was re-designed for session 2011-12 to incorporate the new Vitae Researcher Development Framework. Variable quality and timeliness of feedback to students – UCL-wide service standards for the provision of feedback to students were agreed by AC http://www.ucl.ac.uk/silva/academic-manual/part-3/feedback-to-students-on-assessedwork. Implementation of those service standards is monitored by Education Committee. Training of research supervisors – the training programme for research student supervisors (http://www.ucl.ac.uk/silva/calt/supportfor/phd-supervision) continues to be enhanced, as does the policy regarding monitoring the supervision of research students (http://www.ucl.ac.uk/academic-manual/part-5/pgr-supervision). UCL has also implemented a process for the central recording of approval for academic staff to act as research student supervisors. Research Student Records - a more comprehensive and transparent UCL research student record was created, containing detailed data on attendance, interruptions, funding, upgrade, submission, viva examination, etc. Peer observation of teaching - faculties are required to confirm to QMEC (and thence to AC) as part of the Annual Monitoring process that this has taken place in all of their constituent departments. QMEC and AC remain alert to the need to ensure that peer observation of teaching is implemented in all areas and continues to develop new ways in which good practice can be shared in this area. 4 17. More broadly, the process of achieving greater harmonisation is an ongoing one, especially in a large, complex and multi-disciplinary institution such as UCL. So too is the process of ensuring full departmental engagement with institutional policies and processes. As such, AC and its key subordinate committees1 and senior officers remain alert to the continuing need to achieve, where appropriate, greater harmonisation of, and engagement with, institutional policies and procedures which have been agreed via UCL’s collective deliberative structures. 18. There have been two ways in which AC, with the support of its key subordinate committees, seeks to pursue the twin goals of achieving an optimal level of harmonisation and full engagement, whilst respecting genuine disciplinary and pedagogical differences: First, through the receipt of standard annual reports, AC is able to (i) identify areas where greater alignment between faculties and departments is required or desirable and (ii) evaluate whether faculties and departments are operating within UCL’s agreed policy or procedural framework. These reports include: - Second, through the development of particular strategies (accompanied by implementation plans which are monitored by the relevant committees) and policies and the completion of specific initiatives, UCL has sought, where appropriate, to harmonise processes within faculties and departments, and to ensure an equivalence of student experience across the university. Examples in the period since the 2009 IA include the following: - - - 1 Annual Report on the Proceedings of Faculty Teaching Committees (Annex C); Annual Report on Annual Monitoring and Augmented Annual Monitoring (Annex D); Annual Student Feedback Data Overview Report (Annex E); Annual Reports from Lead Officers with responsibility for specified areas of operation (Annexes F-I); Annual Reports from its key subordinate committees (Annexes J-M). a revised UCL Learning and Teaching Strategy 2010-2015, supplemented by revised faculty and departmental learning and teaching strategies; a new Personal Tutoring Strategy; a new Library Strategy 2011-2014, comprising the following performance areas: student experience; research support; support for healthcare; space management; widening participation and public engagement; a project to construct a new Student Hub on the UCL main site; introduction of a Common Timetable system; review of the UCL Harmonised Scheme of Award; review of the range of assessment types used across departments; review of the Internal Quality Review (IQR) process; Education Committee, Research Degrees Committee, Quality Management and Enhancement Committee, Joint Staff Student Committee – information on UCL’s committee system is at http://www.ucl.ac.uk/ras/acs/governance/committees 5 - - - - changes to the Annual Monitoring process, including the introduction of an accelerated timetable to improve timeliness of reporting and thereby departmental engagement; minimum requirements for departments on the use of Moodle; the Smart IT programme led by ISD which aims to deliver improved IT enablement for research and for teaching and learning, underpinned by common, standard shared services; service standards on the provision of feedback to students; standard penalties for late submission of work and excessive word length; regulations for the award of degrees under special provisions and aegrotat provisions for Masters students; revised admissions service standards; revised deadlines for admission and enrolment; revised policy and procedure for the conduct of interviews in admissions; introduction of a standard checklist for Faculty Teaching Committees (FTC), setting out the key business that FTCs should be dealing with during the course of the academic year, supplemented by formal briefing sessions for secretaries to FTCs (such a checklist already existed for Departmental Teaching Committees); reform of AC and its committee substructure. 19. A note detailing the IQRs (UCL’s periodic review process) which have been undertaken since the 2009 IA is at Annex N. 6