AQSC 13/09-10 INSPECTION OF THE WARWICK DIPLOMA IN ORTHODONTIC THERAPY UNIVERSITY OF WARWICK 29 MAY and 17 JULY 2009 REPORT OF THE INSPECTORS MR P BUTLER MS M HARRIS (CHAIR) MRS R KHAN DR A SHEARER Overview We identified many strengths for this programme. These include the excellent facilities available to students at the Leamington Spa Orthodontic Centre (LSOC) which are of a particularly high standard. There are strong links with the University of Warwick in terms of access to pastoral support for students. All students are highly motivated and of a high quality. Also impressive is the level to which the practice teams are engaging with the trainee therapists as well the scope for trainers to remain involved with the programme after the course is completed. Good quality control mechanisms ensure that issues and problems are being addressed swiftly. There are a small number of areas which we felt could be improved. Students would benefit from additional tutoring on preparation and planning for case presentations to include advice on selecting suitable patients. The use of the Virtual Learning Environment (VLE) could be boosted, particularly as these students do a good deal of their training in the workplace. This might include making all lectures available on the VLE. We also felt that the logbook system could be improved. It was not working well for all students who reported it was difficult to complete. Purpose & GDC process 1. As part of its duty to protect patients and promote high standards, the General Dental Council (GDC) monitors the education of dental students and dental care professionals (DCPs) at institutions whose qualifications are approved by the GDC. The aim is to ensure that these institutions provide high-quality learning opportunities and experiences and that students who obtain a dental qualification are safe to practise. 2. The purpose of the inspection was to assess whether the diploma in orthodontic therapy conforms to the GDC’s requirements for the training of orthodontic therapists and whether, on qualification, students with the diploma would be suitable for registration with the GDC. Introduction 3. This report sets out the findings of a one-day inspection of the programme (29 May 2009) together with a one-day inspection of the final examination (17 July 2009), using the assessment principles and guidelines set out in Developing the Dental team – Curricula Frameworks for Registrable Qualifications for Professionals Complementary to Dentistry (DDT) as a benchmark. The report highlights many areas of good practice, but also draws attention to areas where issues of improvement and development need to be addressed. The report is based on the findings of the inspections and on a consideration of supporting documents prepared prior to the inspections and made available during the inspections. 4. We were welcomed by Dr Richard Cure (Clinical Director of Orthodontics), Dr Liz Hopkins (Director of Orthodontic Therapy Course) and Professor Bob Ireland (Course Development and Teaching Support). During the programme inspection, we met with workplace trainers and staff involved in the delivery and management of the programme. We also met with all students on the orthodontic therapy programme. 2 We would like to thank all concerned for their help, courtesy and co-operation during the inspection. Core course and study days 5. Similarly to other Orthodontic Therapy courses, the Warwick programme begins with a core course. Students attend the Leamington Spa Orthodontic Centre for a week and then return to their workplace for a week. This cycle continues for the first 8 weeks of the course and comprises 20 study days. We felt this was an innovative approach since it allowed students to take what they had learned during a particular week and consolidate it in their individual workplace setting. This set-up also avoids unnecessarily overloading students with a large volume of information which might occur with a core course comprising a single 4-week block. 6. Students return to the LSOC after the core course for a further 12 study days which are spread out over the following 30 weeks at approximately one per fortnight. Students found this tiring and demanding, especially those travelling long distances. However, students commented that most study days had been arranged at the end of the week meaning they were not back at work the next day. 7. The course has a modular format which enables students to be assessed from an early point in the course. The format also enables students quickly to grasp the level of work and commitment required of them. Blank space is built into the timetable and this is set aside to go over aspects of the course which the students have found problematic or which they feel needs additional coverage. 8. It is intended that study skills will form an element of the core course for future intakes. This will encompass the Warwick University online study skills course as well as visits to the University library for training in IT and essay writing skills. We would welcome this addition to the core course. Workplace training 9. Students spend the majority of the programme in workplace training. The workplace training is normally based in the practice where they worked prior to commencing the programme. 10. We were very pleased to learn that the course has a very strong approach to team working. In our meetings with those involved in the delivery of the course and particularly with the workplace trainers it was very apparent that the whole practice takes an active role in the work of the trainee orthodontic therapist. 11. We felt it will be necessary to monitor the potential conflict between the trainee and their employer who also acts as the workplace trainer. In some cases, the workplace trainer is also a course tutor on the programme as well as being an employer. We recognise that this situation might be uncomfortable for some students, especially when raising issues or concerns. We were pleased to learn that regular feedback taken from students at the end of each module is anonymised to help negate potential difficulties. 12. Ways of creating stronger links between the various workplace trainers are being investigated and developed. This might be through a video link which would enable 3 the trainers to have easy and regular contact with each other. We recommend this work continues. Quality assurance systems 13. Firm processes are in place to ensure the suitability of potential trainee therapists. The potential trainer and trainee are interviewed individually and together during the initial interview day at LSOC. Subsequently, a workplace visit is undertaken during which the clinical and training facilities are assessed. 14. Induction for the workplace trainer also involves the practice manager. We felt this was good practice in terms of ensuring an understanding of the overall educational experience of the trainee therapist within the practice setting. Induction and training days need to be set and advertised well in advance to ensure these dates can be cleared more easily by those needing to attend. 15. The Staff Student Liaison Committee (SSLC) appears to work very well. Students viewed this as a useful tool to provide feedback and to raise issues and highlight aspects of the course which may need attention. We felt that comments made were taken on board and dealt with efficiently and swiftly. 16. A ‘buddy’ system is planned which will make use of the network of previous trainers and students to provide advice, support and information to new student cohorts. We strongly support this plan and feel that the aspiration to maintain links with past students and workplace trainers is commendable. Student issues 17. We met all students on the course during the inspection and were impressed by their dedication and commitment to the course. They are an inspiring group. They told us that the course had involved a very steep learning curve which was extremely intense. Despite this, they had found the course to be very enjoyable and fulfilling. When problems or issues are raised with staff they feel they are listened to and that their points are taken on board and dealt with in a timely fashion. This supports our views of the SSLC outlined above. They also reported a good level of support within their individual practice settings. 18. We were pleased to note that students have a good understanding of the grading scales involved in assessments and therefore appreciate what is expected of them. They did tell us, however, that they found the selection of appropriate patients for case presentations a difficulty and we endorse their desire for additional support and guidance in this aspect of the course. 19. The students’ main concern was with the logbooks they are required to complete. They found them difficult to complete and struggled with the relevance of some of the elements. It was clear to us that the logbooks are not working well for all students. They cover the core competencies of the course, provide a record of completed treatments and act as a personal development tool. We feel, however, that the logbooks need to be reviewed to make them more user-friendly. 20. We were told there is currently limited use made of the VLE for uploading lectures and for general communication and information. We would like to see this increase. Students’ have access to a general e-forum. Work is under way to establish a forum, 4 via the students union, which can be accessed by the students only. The students we met were eager for this to be in place. Assessment 21. Summative assessments take place at the conclusion of each module within the programme. These module assessments take a variety of forms including Multiple Choice Questions (MCQs), Short Answer Questions (SAQs), Objective Structured Clinical Examinations (OSCEs) and logbook assessments. In-course module assessments do not contribute to final grades although all modules must be passed in order to progress to the final examination. 22. We attended the final examination day on 17 July 2009. The final examination consists of four elements: An OSCE; 15 minute structured viva; two 15 minute case presentations and a 2 hour SAQ written paper. The written paper was taken on 10 July 2009 and was not attended by the GDC. We were given the opportunity to look at completed scripts and felt that the questions were of an appropriate level. We did, however, feel that some questions required far lengthier answers than would normally be expected for SAQs. Marks for the written paper appeared to have been arrived at by calculating the average of two grades given by independent markers. We would recommend that, in future, the two markers agree together on a final mark after grading rather than simply awarding the average of the two independent marks. Marks for each question were given out of 100. Normally, we would expect to see each question marked out of 10 or 20. This allows the marker to identify key points in each answer more easily. 23. Overall, we felt that the examination day was extremely well run. It was clear that much attention had been paid to the organisation of the examination. With several elements of the examination running concurrently, staff worked hard to ensure that students were kept separate from each other to avoid the potential for the sharing of information. 24. The two case presentations and general viva were held consecutively and we considered that this was quite stressful for the students. 45 minutes is a long time to have to speak without a break. We do recognise, however, that this period of time provides a good opportunity for examiners to assess candidates’ ability. Candidates are questioned by two internal examiners. The external examiner had the opportunity to observe the case presentations/vivas but was not directly involved in examining. We would recommend that while one examiner is questioning, the other should take notes. This would be of particular importance should a candidate decide to challenge a result. 25. Students’ study models were available to examiners for use during the case presentations. However, these were seldom used and, for future sittings, consideration should be given to the value of their inclusion. 26. The general viva which takes place after the two case presentations only really tests knowledge of subject areas. It covers topics such as basic sciences, CPD requirements and the remit of an orthodontic therapist. It does not test candidates’ understanding and these topics might, therefore, be covered elsewhere just as easily. For future sittings, consideration could be given to covering these topics in written papers. The general viva would not then be required. This would reduce the 5 length of the oral component of the examination and ease the perceived pressure on candidates. 27. The OSCE comprised 20 stations each with a set of clinical photographs and series of SAQs relating to them. It was our view that an OSCE normally tests what a written paper can not test. Most OSCEs have a practical element. The OSCE is described in assessment related material as a practical assessment. We would recommend that the OSCE is developed so that it does have a practical make-up. 28. The external examiner was happy with the examination process. He had been given the opportunity to view examination materials in advance and had made minor modifications to some questions. At the exam board meeting, which we also attended, there was an opportunity for the external examiner to comment on proceedings. Some helpful suggestions were offered which we would support. These were in relation to noise levels in exam rooms, the need for a short examiners meeting at the beginning of the session to ensure calibration of mark scheme descriptors and methods of putting candidates at their ease during the examination. 29. The external examiner also felt that examination questions should be mapped against the curriculum to ensure full coverage. Again, we would support this and feel that carrying out a blueprinting of the examination against the curriculum would be a useful exercise. Requirements Clearer guidance and support needs to be provided for students in relation to selection of patients for case presentations (18). A review of student logbooks needs to be undertaken to ensure they work effectively (19). A ‘blueprinting’ of the final examination against the curriculum should be carried out (29). Recommendations Efforts to develop stronger links between workplace trainers should continue (12). Marks should be agreed by the two markers rather than calculating the average score (22) Each examiner should take notes during oral examinations while the other examiner is questioning the candidate (24). Consideration should be given to the value of including study models during case presentations (25). Consideration should be given to the general viva and whether the questions it covers could be covered via the written papers (26). 6 Consideration should be given to including practical tests as part of the OSCE examination (27). Comments made by the External Examiner should be considered and addressed as necessary (28). (Numbers in brackets refer to individual paragraphs within the main body of the report.) 7