ANGLIA POLYTECHNIC UNIVERSITY Academic & Quality Systems Office – Quality Assurance Division Summary Report to the Senate of Annual Monitoring of Delivery in 2003/04 1. Introduction 1.1 The purpose of this paper is to provide a summary to the Senate on the annual monitoring process for delivery of pathways in 2003/04, conducted between September 2004 and February 2005. 1.2 The following reports are attached as appendices and were the main sources of information for the Summary Report: Five Regional Faculty Board (RFB) Overview Reports (Business & Technology; Creative Arts; Education, Negotiated & Interdisciplinary Studies; Health & Sciences and Humanities, Law & Social Sciences). All Overview Reports were approved by the relevant RFBs at their meetings in January/February 2005; Synoptic Report for all foundation degrees (this report complements the individual monitoring of foundation degrees undertaken as part of the standard process and provides an overview of the foundation degree award). 1.3 The Senate is invited to consider this report, agree appropriate action where necessary and conclude the annual monitoring process for delivery in 2003/04. 2. Process 2.1 A number of amendments were made to the annual monitoring process employed during 2004/05 following feedback from Deans of Schools, RFBs, Directors of Studies and Field Committees and in response to issues raised in the QAA Institutional Audit. The key changes included: A revised template for the individual annual monitoring report (AMR); A revised template for the Reader’s Report; The discontinuation of the Director of Studies’ Synoptic Report; The establishment of standing committees of the RFBs with responsibility for overseeing the annual monitoring process within each Regional Faculty including the consideration of all Readers’ Reports and S.M.A.R.T. Action Plans; The introduction of RFB Overview Reports; The presentation of a Summary Report to the Senate. 3. Conclusions 3.1 All five RFBs report that the annual monitoring process of the delivery of pathways in 2003/04 was conducted effectively and undertaken in a sufficiently critically reflective manner, with a clear improvement on the rigour and usefulness of the process from the previous year. There was also significant evidence, via the Readers’ Reports, that there had been a good level of staff engagement with the process. Summary Report to the Senate 1 Annual Monitoring of Delivery in 2003/04 3.2 The RFBs report that, in general, the action plans attached to the individual annual monitoring reports were well constructed and, in the vast majority of cases, followed the S.M.A.R.T. format. However, the Heath & Sciences RFB reports some variance in the construction of the plans. Consequently, it is recommended to the Senate that staff development on writing AMRs and S.M.A.R.T. Action Plans, coordinated by the Quality Assurance Division in Autumn 2005, is provided for report authors. 3.3 In general, RFBs report that External Assessors’ reports were positive and complimentary. They were effectively used as part of the annual monitoring process with S.M.A.R.T. action plans containing appropriate references to issues requiring attention. In the few cases where this had not occurred, the S.M.A.R.T. Action Plans were returned to the authors for amending. The Business & Technology RFB noted that, whilst issues raised by External Assessors had been addressed, the actions undertaken were not always contained in the action plans as the work had been completed by the time the action plan was compiled. It is recommended to the Senate that AMRs should detail all responses to External Assessors in the S.M.A.R.T. Action Plan, even where action has been completed, in order to provide a single point of reference to evidence that appropriate action has been taken. 3.4 Recurring themes from External Assessors’ reports included: The adequacy of time given to consider proposed forms of assessment prior to use; The inconsistency and varied standard of feedback to students on assessed work and the need for feedback to relate to learning outcomes and assessment criteria. 3.5 All the RFBs report that, in general, University Assessors’ reports (on delivery in regional partner institutions) were positive and constructive. They were effectively used as part of the annual monitoring process (where relevant) with S.M.A.R.T. Action Plans containing appropriate references to issues requiring attention. There were a minority of cases where target dates for completion of actions needed to be more specific. In the few cases where issues were not adequately addressed, revised S.M.A.R.T. Action Plans have been requested. 3.6 There were no recurring themes from University Assessors reported. 3.7 All the RFBs report that there was significant evidence of issues identified in the previous year’s S.M.A.R.T. Action Plans having been completed. Where actions had not been completed, these have been brought forward for inclusion in the S.M.A.R.T. Action Plans for 2004/05. 3.8 There were no major issues of concern raised by any of the Overview Reports with regard to professional and statutory bodies. 3.9 The RFBs report that, with regard to the essential requirements of the annual monitoring process, annual monitoring reports have been received for all fields/courses with the relevant documents attached. Where some aspects are missing, these continue to be followed up by the Quality Assurance Division (see individual RFB Overview Reports for detailed information for each RFB). It is recommended to the Senate that all outstanding actions are required to be completed by AMR authors and readers by 31st May 2005, to be reported to the final 2004/05 meeting of the Senate in June 2005. Summary Report to the Senate 2 Annual Monitoring of Delivery in 2003/04 4. Issues of Institutional Significance 4.1 The provision of accurate and timely statistical data to support the annual monitoring process is highlighted as an issue of institutional significance which requires action by all the RFBs. Some of the difficulties encountered include: Poor or non-existent accessibility to SITS: Vision and associated reporting systems, particularly for collaborative partners, impeding the consideration of statistical data in the process; Confusing format of statistical data (eg: insufficient detailed breakdown by pathways/cohorts) preventing any opportunity for useful analysis; Lack of data with regard to age, gender, ethnicity and geographical origins which would be particularly useful within the context of foundation degrees and the widening participation agenda which is a defining characteristic of foundation degrees. 4.2 The absence of “forgiveness” (compensation for, or condonation of, failure) in the Curriculum Regulations (Business & Technology, Health & Sciences). This issue is already being addressed by the Steering Group on the 15/30 Credit Curriculum. 4.3 The impending changes to the car-parking policy at APU, particularly at the Essex Campus. It is felt that the changes will have a serious impact on the recruitment and retention of students (Business & Technology, Health & Sciences). 4.4 The physical environment, including heating and uncomfortable and inappropriate furniture in teaching rooms used by the Mathematics & Technology Field, lighting and blinds in the Ashcroft Building and opening hours of the Essex Refectory for twilight study (Business & Technology). 4.5 The lower than expected level of recruitment and retention to a number of pathways delivered at regional partner colleges. This highlighted concerns about the standard and adequacy of the marketing used for such pathways (Business & Technology, Creative Arts). 4.6 The lack of statistical data provided to Awards Boards which details trends in progression rates. The need for clarification of what exactly is meant by ‘progression’ in the context of APU’s regulatory structure also needs to be addressed (Business & Technology). 4.7 The lack of consistency with regard to module information, between the J:drive, E-Vision and the Module Definition Forms held by the teaching staff, and the resulting impact on the assessment process (Health & Sciences). 4.8 The classing of students as debtors who have agreed a payment plan, via instalments, with the University, penalising such students unfairly (Health & Sciences). 4.9 An inconsistency in administrative processes, especially those related to assessment, between APU and various collaborative partners (Creative Arts). Summary Report to the Senate 3 Annual Monitoring of Delivery in 2003/04 5. Examples of Good and Innovative Practice for Wider Dissemination 5.1 A significant number of examples of good and innovative practice were highlighted by the RFBs. These are listed in Appendix 1. The Senate is invited to endorse these examples which shall be disseminated by the Quality Assurance Division to the wider University community with details of colleagues who can be contacted for further information. 5.2 The Business & Technology and Humanities, Law & Social Sciences RFBs raised the issue of the provision of advice on what constitutes good practice as many of the items listed by AMR authors were not considered to be good practice by the Standing Committee’s definition but standard practice. It is recommended to the Senate that the following definition of good practice is adopted for the annual monitoring process: Good practice is defined as "a method, strategy, system, procedure or process, which has, over an appropriate period of time, resulted in improved academic standards, an enhanced quality of education and/or an improved level of service to stakeholders (eg: students, staff, external examiners, collaborative partners, employers etc.) and which can, when appropriately adapted, be implemented in other areas of the institution." Such good practice can be evidenced in a variety of ways. Examples include student performance, statistical information, feedback from stakeholders (ie: via questionnaires, Field Committee meetings, Employer Liaison Panel meetings etc.). 6. Further Enhancements and Amendments to the Future Process 6.1 The RFB Overview Reports provide clear evidence that the amendments made to the annual monitoring process, including the use of Standing Committees to scrutinise Readers’ Reports and S.M.A.R.T. Action Plans, has resulted in a considerably more thorough approach, allowing a proper oversight of the monitoring process by the RFBs and, subsequently, the Senate. The changes also provide evidence that the University has started to address the concerns about multi-layering that were highlighted as a concern in the report of the QAA Institutional Audit of APU (the Education, Negotiated & Interdisciplinary Studies RFB notes that this aim has not yet been fully achieved). It is therefore recommended to the Senate that the changes implemented in 2004/05 be continued in 2005/06 (for the monitoring of the delivery of pathways in 2004/05). 6.2 The changes in the curriculum management structures mean that the organisational unit of the Field will no longer exist in 2005/06. The Field has been the focus of the annual monitoring process in previous years. The new University structures will see all pathways assigned to departments within the new faculties (the identity of the various departments is unknown at the time of writing). It is therefore necessary to determine on which structure AMRs will be focussed in 2005/06. There are two alternatives: the former Field structure: as the process will be evaluating the delivery of pathways in 2004/05 which operated under the Field organisational unit; Summary Report to the Senate 4 Annual Monitoring of Delivery in 2003/04 the new Department structure: in order to move towards and reflect the University’s new structures in one of its key quality assurance processes (this option will require special arrangements to be made for the small number of pathways where operational management has been split from Pathways Teams as a result of the new structures). It is recommended to the Senate that the focus of the annual monitoring process for 2005/06 be determined by the Senate from one of the above listed alternatives. 6.3 In addition to the Readers’ Reports and the S.M.A.R.T. Action Plans, the RFB Standing Committees considered the relevant synoptic reports from the four regional collaborative partners who have delegated authority for aspects of the annual monitoring process (City College Norwich, Colchester Institute, Homerton School of Health Studies and Norwich School of Art & Design). The purpose of the Synoptic Report is to provide the relevant RFB with an overview of the outcome of the annual monitoring process conducted locally at the regional collaborative partner. The partner is strongly advised to adopt the line of enquiry contained in APU’s template for AMRs. However, in recognition of the fact that the partner may wish to use the Synoptic Report for internal purposes without substantial redrafting, broad guidelines on content are provided giving appropriate flexibility for both internal and APU purposes. The Business & Technology, Creative Arts and Education, Negotiated & Interdisciplinary Studies RFBs, whilst commending the thoroughness of the Synoptic Reports, report that the purpose of the Synoptic Report and/or the role of the RFB in considering the Synoptic Report requires further clarification. It is therefore recommend to the Senate that the purpose of the Synoptic Reports provided by the four collaborative partners, and the role of the Faculty Board in considering the reports, needs to be clarified in time for the process to be conducted in 2005/06. 6.4 In addition to the problems encountered in the provision of statistical data (see para. 4.1), all of the RFBs report that, where such information was made available, there was little evidence of engagement by report authors. With many authors, the data was merely attached as an appendix to the main report with no reference to the data in the body of the report or use of the information to inform the action plan. It is recommended to the Senate that the training and staff development on the writing of AMRs (see para. 3.2) covers the use and interpretation of statistical data for the purposes of annual monitoring with more explicit guidelines provided for report authors. Furthermore, the Synoptic Report for foundation degrees highlights the defining characteristics of (i) widening participation and (ii) the linkages to employment and draws attention to the weaknesses in the provision of statistical data to AMR authors to allow evaluation of APU’s foundation degrees in relation to these key markers. It is therefore recommended to the Senate that future statistical data includes information regarding first destinations and entrance characteristics. 6.5 One of the enhancement features of the annual monitoring process is the provision of examples of good and innovative practice and their dissemination throughout the wider University community. In an effort to encourage the effective consideration and use of the annually circulated examples of good and innovative practice, the Business & Technology RFB recommends to the Senate that, the annual monitoring report template should be revised to ask authors to comment on whether any of the examples of good practice highlighted in the previous year have been considered and/or adapted for use. Summary Report to the Senate 5 Annual Monitoring of Delivery in 2003/04 6.6 The Business & Technology and Education, Negotiated & Interdisciplinary Studies RFBs noted that there was a demonstrable improvement in the standard of the Readers’ Report on previous years. Efforts had been made to reduce the burden on readers by reducing the number of individual reports each reader should consider. However, it is still possible for one reader to be asked to consider a high number of reports. In order to address this concern, and a similar concern raised during the QAA Institutional Audit, it is recommended to the Senate that the number of annual monitoring reports and S.M.A.R.T. Action Plans that be considered by any one reader is no more than ten. Furthermore, it is recognised that the use of a higher number of readers will lead to a number of different approaches to the reader’s role and, consequently, in order to ensure consistency, it is recommended to the Senate that staff development (see para 3.2) should include provision for readers in future years. 6.7 The Business & Technology RFB believes that it would be good practice to be able to demonstrate to External Examiners that their comments have been taken into account through a formal process. It is therefore recommended to the Senate that a copy of the complete annual monitoring report and S.M.A.R.T. Action Plan should be sent by the Field Leader (the Head of Department from 2005/06), for information, to all relevant External Examiners. This will also help to enhance the External Examiners’ understanding of the modules/pathways/ courses they moderate. 6.8 The Humanities, Law & Social Sciences RFB determined that the membership of their Standing Committee would consist of the readers of the individual annual monitoring reports in order to increase the committee’s level of knowledge of the detail of the individual AMRs. The Senate is invited to consider whether this stipulation should be made for the constitution of all future standing committees. 6.9 The Humanities, Law & Social Sciences RFB believes that, in order to help monitor achievement of the previous year’s action plan, it would be helpful for the readers of AMRs to see the previous year’s S.M.A.R.T. Action Plan. Furthermore, the RFB believes that a response to the previous year’s Reader’s Report would also provide good evidence of enhancement. It is therefore recommended to the Senate that the AMR template is amended to require authors to include the previous year’s S.M.A.R.T. Action Plan and the associated Reader’s Report, together with a formal response to the Reader’s Report from the AMR author. 6.10 The Synoptic Report for foundation degrees highlights the need for AMR authors to improve the level of evaluation of work based learning and employer involvement in relation to foundation degrees (which are key characteristics of the award). It is therefore recommended to the Senate that the AMR template is revised to capture such information. The Synoptic Report also comments on the possible advantages of requiring foundation degrees to be presented separately from, or distinctly within, AMR Field reports, especially where a pathway is delivered across multiple centres. However, it is noted that one important feature of the University’s new curriculum management structures is the enhanced level of responsibility for all pathways by Heads of Department, wherever they are delivered. Consequently, it is recommended to the Senate that the annual monitoring of foundation degrees should continue to be embedded in the Field Report (or agreed equivalent report – see para 6.2 above) but with a separate distinct section on foundation degrees. The Senate is further invited to consider whether a Synoptic Summary Report to the Senate 6 Annual Monitoring of Delivery in 2003/04 Report for foundation degrees should continue to be produced in future years, following the transfer of responsibilities for foundation degrees to the faculties. PAUL BAXTER Head of Quality Assurance April 2005 Summary Report to the Senate 7 Annual Monitoring of Delivery in 2003/04 Appendix 1 Business & Technology Successful implementation of a student adviser system in the Business School throughout 2003/04; The introduction of small lecture-based assessment methods, designed to help improve retention by early identification of students experiencing difficulties, in the Computer Science Field; The use of a dedicated Work Placement Officer for the FdA E-Commerce pathway at City College Norwich; Creative Arts The success of a very student-centred approach to learning in the Graphic Arts Field; The support provided to students in preparing their dissertation, organised around a programme of presentations on set tasks, designed to ensure systematic progress of dissertations during the first semester of level 3, in the revalidated Approaching Research module in the Study of Art Field; The use of postgraduate research bursaries for study abroad in the MA Music pathway; The production and use of training and clinically-based videos in the MA Music Therapy pathway; The use of local context, in particular, the local teacher-training college, providing career opportunities for graduates (graduation exhibition and visit to book fair) in the MA Children’s Book Illustration pathway; The excellent use of primary sources, exhibition resits and work/technical/ museum log books with archival statements and annotated comments in the HNC/D Fashion with Textiles pathway at Cambridge Regional College; The success of the final year show in a high profile business venue and a student winning D&AD award in the HNC/D Graphic Design and HND/Dip. HE Graphic Art pathways at Cambridge Regional College; A strong and well run HNC 3D Spatial Design pathway at Easton College (teaching staff delivering similar courses would undoubtedly benefit from a visit to the college to meet the course team and see the students’ work); An initiative to set up a business to support self-employed students in the HND Photography pathway at Great Yarmouth College; The practice of conducting group assessment involving all staff and students in the HNC/D Textiles with Fashion pathway at Great Yarmouth College; Education, Negotiated & Interdisciplinary Studies The use of formative assessment and feedback across a number of programmes, noted as enhancing achievement and increasing students confidence as learners; Summary Report to the Senate 8 Annual Monitoring of Delivery in 2003/04 The development of a guide for mentors involved in the delivery of foundation degrees; The visiting of all students on work placements; The cross institutional collaboration on moderation and assessment; Visits to partner colleges by APU course teams to support consistent approaches to standards of work and assessment; The strong emphasis on the close linking of theory to practice on a number of pathways; A consistency, equity and fairness in resolving student issues; Health & Sciences The use of module specific evaluation forms (separate from the generic form) to address module-specific issues in the Environmental Biology and General Science Field; A subject-specific careers advisor in the Environmental Biology and General Science Field to provide students with more detailed and biologically relevant advice; The approach to the delivery of a ‘core curriculum’ to support a variety of pathways in the Life Sciences in the Microbial and Molecular Biology Field; The operation of revision sessions for students who have resit examinations (usually a week prior to the resit exam) in the Optometry and Ophthalmic Dispensing Field and the offer of a trial “revision day” for all students, to see what effect this has on student performance within the examination; Coursework feedback systems in the Psychology Field; Procedures developed for the successful delivery of distance learning pathways in the Radiography Field; An inquiry based learning approach which fosters deep learning (a particular example being the “management simulation exercise”), including the concept map poster presentation for assessment, in Adult Nursing and the Common Foundation Programme in Nursing; The inclusion of negotiated elements in Childhood Studies Field which is challenging and encourages innovative work from students; The continued success of the Communication Skills for Study in HE module in the HNC Childhood and Youth Studies pathway delivered at Great Yarmouth College; The style of feedback, providing constructive comments to students in feedback in the Child Observation module, commended by the University Assessor, in the HNC/HND Childhood and Youth Studies pathway at West Suffolk College; Summary Report to the Senate 9 Annual Monitoring of Delivery in 2003/04 The increased use of blended approaches to learning - Web CT as a communication tool for students – in the Continuing Health Care Education Field; Collaborative work with the NHS in curriculum design and delivery in the Health and Community Studies Field; The electronic submission of assignments in the Health and Community Studies Field; Distance learning study support for RCSA students in the Health and Community Studies Field; The management, by individual members of staff, of their own peer review strategy, with evidence of the reviews documented and contributing towards individual annual appraisals in the Learning Disabilities Field; The demonstration of the dynamic and effective range of learning and teaching strategies via the peer review process in the Learning Disabilities Field; The continuing involvement of mental health service users in many aspects, including curriculum design and review, teaching, quality assurance and annual monitoring by professional bodies and research in the Mental Health Studies Field; The ethos of partnership with education commissioners through the substantial use of staff secondments, thereby increasing credibility and enhancing the student experience, in the Mental Health Studies Field; The use of clients/service users to enhance module delivery in the Midwifery and Human Sexuality Field; The virtual learning environment at the South-East Essex VI Form College which allows students to work from home in the HND/ Dip. HE Sport Health and Exercise pathway; The peer teaching of IT skills in the Social Policy Field; The proactive use of personal tutorials in the retention strategy in the Social Policy Field; Good liaison with partner agencies in the Social Work Field; Excellent internal moderating, rigorous marking and good feedback to students in the Social Work Field; Guidance and planning for students with difficulties in the Social Work Field; The systematic and sensitive involvement of service users in the qualifying programme in the Social Work Field; Interprofessional working and computer facilitated learning in the Social Work Field; Summary Report to the Senate 10 Annual Monitoring of Delivery in 2003/04 The production of a Field Bulletin to ensure feedback to students in the Social Work Field; The weekly feedback on the portfolio, which monitors student progress and development, in the Counselling Field; The philosophy of self managed learning, encouraging a high degree of personal reflectivity and maturity, in the Counselling Field; Humanities, Law & Social Sciences The use of practitioners in teaching (e.g. Communication Studies, MA Communication, Culture & Production, English & Drama [Royal Literary Fund Writer-in-Residence]); Opportunities for international student mobility and exchanges (e.g. Communication Studies, Languages, MA European Intercultural Communications); Team-teaching (e.g. Communication Studies); Opportunities for work experience (e.g. FdA Public Service); Innovative learning and teaching methods (e.g. group construction of website as assessment instrument in the Theories in Practice module, use of portfolio to develop confidence and improve attainment in Languages); Support for research students (e.g. Reading Group in MA English Studies, enhanced student support at dissertation stage in MA European Language & Intercultural Studies, Work in Progress seminars in MA Women’s Studies); Use of Independent Study Module on BA (Hons) Humanities at West Suffolk College; Development of formative assessment in MA Pastoral Theology; Development of distance learning using an international pool of expert tutors in MA Jewish Christian Relations; The Summer School integrating members of public as well as course members (also MA Jewish Christian Relations). Summary Report to the Senate 11 Annual Monitoring of Delivery in 2003/04 Appendix 2 Summary of Recommendations to the Senate In summary it is recommended to the Senate that: staff development on writing AMRs and S.M.A.R.T. Action Plans, coordinated by the Quality Assurance in Autumn 2005, is provided for report authors. AMRs should detail all responses to External Assessors in the S.M.A.R.T. Action Plan, even where action has been completed, in order to provide a single point of reference to evidence that appropriate action has been taken. all outstanding actions identified by the RFBs and their Annual Monitoring Standing Committees are required to be completed by AMR authors and readers by 31st May 2005, to be reported to the final 2004/05 meeting of the Senate in June 2005. the following definition of good practice is adopted for the annual monitoring process: Good practice is defined as "a method, strategy, system, procedure or process, which has, over an appropriate period of time, resulted in improved academic standards, an enhanced quality of education and/or an improved level of service to stakeholders (eg: students, staff, external examiners, collaborative partners, employers etc.) and which can, when appropriately adapted, be implemented in other areas of the institution." Such good practice can be evidenced in a variety of ways. Examples include student performance, statistical information, feedback from stakeholders (ie: via questionnaires, Field Committee meetings, Employer Liaison Panel meetings etc.). the changes implemented to the annual monitoring process in 2004/05 be continued in 2005/06 (for the monitoring of the delivery of pathways in 2004/05). the focus of the annual monitoring process for 2005/06 be determined by the Senate from one of the listed alternatives: the former Field structure: as the process will be evaluating the delivery of pathways in 2004/05 which operated under the Field organisational unit; the new Department structure: in order to move towards and reflect the University’s new structures in one of its key quality assurance processes (this option will require special arrangements to be made for the small number of pathways where operational management has been split from Pathways Teams as a result of the new structures). the purpose of the Synoptic Reports provided by the four collaborative partners, and the role of the Faculty Board in considering the reports, needs to be clarified in time for the process to be conducted in 2005/06. the training and staff development on the writing of AMRs covers the use and interpretation of statistical data for the purposes of annual monitoring with more explicit guidelines provided for report authors. Summary Report to the Senate 12 Annual Monitoring of Delivery in 2003/04 future statistical data includes information regarding first destinations and entrance characteristics. the annual monitoring report template should be revised to ask authors to comment on whether any of the examples of good practice highlighted in the previous year have been considered and/or adapted for use. the number of annual monitoring reports and S.M.A.R.T. Action Plans that be considered by any one reader is no more than ten. staff development for the annual monitoring process should include provision for readers in future years. a copy of the complete annual monitoring report and S.M.A.R.T. Action Plan should be sent by the Field Leader (the Head of Department from 2005/06), for information, to all relevant External Examiners. the stipulation that the membership of the Annual Monitoring Standing Committees comprises the readers of the individual annual monitoring reports in order to increase the committees’ level of knowledge of the detail of the individual AMRs should be made for the constitution of all future standing committees. the AMR template is amended to require authors to include the previous year’s S.M.A.R.T. Action Plan and the associated Reader’s Report, together with a formal response to the Reader’s Report from the AMR author. the AMR template is revised to capture information on the evaluation of work based learning and employer involvement in relation to foundation degrees (which are key characteristics of the award). the annual monitoring of foundation degrees should continue to be embedded in the Field Report (or agreed equivalent report) but with a separate distinct section on foundation degrees. The Senate is further invited to consider whether a Synoptic Report for foundation degrees should continue to be produced in future years, following the transfer of responsibilities for foundation degrees to the faculties. Summary Report to the Senate 13 Annual Monitoring of Delivery in 2003/04