Cause for concern: How can we identify and remediate academic, clinical or professional underperformance in Final Year Veterinary Undergraduates? Project leader: Jennifer Hammond, University Teacher, School of Veterinary Medicine Contact details: Jennifer.Hammond@Glasgow.ac.uk Other contributors: Jim Anderson, Associate Head of School for Learning & Teaching, School of Veterinary Medicine Summary Development of a new final year course for the Glasgow BVMS Curriculum (starting in May 2013) has provided opportunities to review the way that Veterinary students are assessed on clinical placements. One important decision has been to introduce a system of “Cause for concern” notifications, to provide timely identification and remediation of students underperforming on clinical placements. The system is designed to allow clinical supervisors to provide feedback and identify remedial opportunities relating to underperformance across a range of areas, including those such as professional behaviours and attitudes which can be difficult to address with traditional assessment methods. Students receiving a cause for concern notification are required to provide evidence in their summative portfolio demonstrating that the issue identified has been effectively addressed. The aim of this project is to develop and evaluate a pilot scheme to run in the academic session 2013-13, to ensure that the “cause for concern” scheme (CFCS) develops to incorporate the views and requirements of stakeholders (including staff, students, and accrediting bodies) and to identify elements which have potential for wider dissemination or transfer to different contexts. This is an innovative system of assessment designed to capture a broad range of potential causes for concern, and critically, to provide targeted constructive feedback and remediation. As such this project addresses the priority area of assessment and feedback identified in the University’s Learning and Teaching strategy. As the CFCS is embedded in the development of the new BVMS curriculum and designed to interface with all other assessment modalities (portfolio, workplace based and knowledge assessments) it also links to the QAA enhancement theme developing and supporting the curriculum, working towards educating high quality Veterinary graduates while improving the efficiency of assessment processes by targeting feedback and remediation to individual students and performance elements which are considered critical to professional success. Aims and Outcomes The aim of this project is to develop the evidence base on which to build an effective “Cause for concern” system, namely: An understanding of the range and nature of incidents or episodes which are identified by staff and students as potential triggers for a “cause for concern” note. Identification of appropriate processes for remediation in these cases. Description of the types of evidence which would be considered as part of a claim to have adequately addressed a cause for concern and how these might be assessed. The initial phase (2012/13) is described as a pilot phase, with further development and evaluation of the system continuing into the implementation phase in 2013-14 which could potentially form the basis of a second LTDF bid. The outcome of the pilot phase will be a clearly defined and documented CFCS which is acceptable to stakeholders and which can then be implemented to facilitate the identification of episodes of underperformance on clinical placements and provide effective remedial opportunities for underperforming students. By designing the system as an integral part of the Final year course assessment as well as planning on-going evaluation and development, the benefits of this project will be sustained beyond the pilot phase in 2012-13. Initially Final year Veterinary students at the University of Glasgow will benefit from the project, while students earlier in the programme and in other Professional programmes could also benefit if the system is extended further. 1 Previous Work In the recent past and across a range of Professional disciplines, there has been an increasing recognition that a good student and a good professional requires skills and attributes over and above those required for traditional academic success. Whilst excellent subject knowledge still forms the core of most Professional qualifications, there is an increasing emphasis among accreditors, professional bodies and educators on the importance of developing clinical and professional skills and attributes. In the context of Veterinary education this is described as part of the Royal College of Veterinary Surgeons (RCVS) Day One Competencies (RCVS 2006) In other professional areas the phrase “Failure to fail” has been coined to describe a range of barriers identified to addressing underperformance in undergraduate students (Cleland, 2008). Anecdotal reports would suggest that similar issues may apply in the context of Veterinary education and that minor instances of underperformance or lapses in professional behaviour may not be considered significant enough to trigger either feedback or remediation. A further issue is that of what constitutes effective remediation and how evidence can be produced to support a claim for adequate performance. Again, medical educators have described limited long term effectiveness of remediation for students underperforming on clinical assessments (Pell, 2012) and there is a paucity of evidence in the Veterinary educational context regarding what constitutes effective remediation for clinical or professional underperformance (Rush, 2005). Student Engagement Student consultation has formed an integral part of the development of the new BVMS Final Year Curriculum and student views on assessment have been incorporated into the final proposals, including the cause for concern system. Students will continue to inform the development of the system through their involvement in focus group meetings and participation in and evaluation of the pilot phase. The potentially sensitive and confidential nature of the assessment process limits direct involvement of current undergraduates in preliminary data analysis, but a summer student scholarship will be designed to participate in analysis of coded data and evaluation of the pilot phase. Methodology The initial phase of the project will involve research into Final year undergraduate and staff experiences of underperformance and remediation during the Final year of the Glasgow BVMS Programme. Preliminary focus groups with staff and students will develop a preliminary definition for “cause for concern” and explore potential events and their classification in order to create a pilot system of documentation and procedures to run for the second term of Final year 2012-13. A short staff training session will be developed to describe use of the pilot scheme and appropriate procedures for reporting, and staff will be asked to report episodes which could (in future) trigger “Cause for concern” notification, as well as recording any actual or potential feedback given or remedial action taken. The data collected at the end of Term 2 will be analysed and used to inform staff and student focus group discussions with the aim of identification of areas of underperformance reported, what constitutes appropriate remediation and what evidence could be considered to support successful remediation. The Cause for concern system policy will then be drafted and reviewed for inclusion in the Course information document for 2013-14 with the definition or classification modified as necessary. Approval will be sought from the School Ethics Committee prior to commencement of the project. Potential Applicability / Transferability The outcomes of the Pilot phase of the project will also be potentially transferable to other parts of the BVMS Programme as well as Veterinary & Health professions education or other areas of Higher education; dissemination could include: Report to the Learning and Teaching Centre Learning and Teaching Seminar or Community for educational scholarship workshop Presentation to School Learning and Teaching Committee Wider dissemination through workshops, conference presentations and journal articles Evaluation Staff and student evaluation of the draft cause for concern policy will be sought at the end of the initial pilot phase as part of student project using a short presentation followed by a survey (this may require a small incentive to be 2 provided). Questions will focus on whether the system appears to be acceptable, what are the likely impacts of its implementation and whether it has the potential to deliver improvements in: Identification of underperformance Providing feedback to students on their underperformance Successful remediation of underperformance Providing evidence for the successful remediation of underperformance The major evaluation of the system will take place following the first year of implementation (2013-14) and will focus on actual performance of the system as well as broader indicators of learning outcomes. It is likely that this phase & subsequent evaluation will form a second bid to the LTDF. Timetable Aug - Sep 12 Oct – Nov 12 Dec 12 Jan – Mar 13 Focus groups with staff and students Outline pilot CFCS Materials and systems developed for pilot CFCS Staff training sessions for Pilot phase Pilot phase CFCSruns for 1 term (40-50 students) Data collection on system use Follow-up focus groups with students and staff conducted & transcribed Mar – Apr 13 Apr – May 13 May-June 13 Development of draft materials and system Refinement and finalisation of CFCS for 2013-14 cohort Proposed start date for phase 2 begins: implementation of CFCS for new cohort July-Aug 13 Summer student scholarship: Evaluation survey of staff and student opinions & focus group data analysis Dissemination of pilot phase findings Budget The project will be co-ordinated by Mrs Hammond & Professor Anderson as part of their normal duties. This will include development of relevant materials, staff training and preliminary focus group data analysis. A student summer project will be designed to complete the detailed evaluation of the CFCS, including analysis of survey and focus group data. The total funding sought is £2,580 (£620 in 2011-2 and £1960 in 2012-13). Non salary costs 2011-12 2012-13 Equipment: Voice recorder with microphone £120 £0 Consumables: Survey printing, form design and printing, batteries etc £200 £300 £300 £660 Other (please provide details) Transcription and administration for qualitative research Student summer project costs (4 weeks) £800 Small incentives for student participation development and evaluation of pilot £200 Approval Should this application be successful and be granted funding, I give consent for this application to be published on the Learning and Teaching Centre website. Signature of Project Leader: Mrs Jennifer Hammond 3