UCL QUALITY REVIEW FRAMEWORK - ANNEX 6.1.3: ASER EVALUATIVE REPORT AND DEVELOPMENT & ENHANCEMENT PLAN Academic Session: Department: Faculty: 2014-15 Mechanical Engineering Engineering EVALUATIVE REPORT Overview Academic year (AY) 2014-15 was a double first for the Department of Mechanical Engineering. The first year of the IEP and also the first year for the “with business finance” programmes (wBF) being supported by Management Science and Innovation Department (MS&I) in UCL. The old programme was in conjunction with the London School of Economics. The Department had its Internal Quality Review in March 2015 as part of this exercise a new teaching and learning strategy document was created for the Department. The IQR panel raised no serious items of concern nor points that the Department was not already aware of. The Department was last accredited in June 2012 and this accreditation is valid up to and including the intake for AY 16-17. To ensure continuous accreditation an EAB visit is planned for Oct/Nov 2016. Year 1 Thanks to significant hard work by many members of the Department the first year of the IEP was a success. There were the inevitable teething problems but nothing major. The first year is being refined for its second iteration. There is still plenty of room for improvement but the priory activity has been preparation of the second year of IEP and Minors. Our new Teaching Fellows have been well received by the students with good feedback. Congratulations go to Dr Tim Baker for winning the Provost teaching award this year. We have as made extensive use of PGTA – all of whom have received training. One concern is that we may not have enough PGTA within the Dept of necessary standing and willingness to cover all our requirements across all years of IEP. Creating the role of PGTA Coordinator was suggested (separating out this aspect of the role of Graduate Tutor) to help with this. The switch to the new “with business finance” programmes has been very smooth. Two points are of note. Because of the structure of the outgoing programme we had a one-off extra teaching load. Year 2 business finance students take a pair of first year modules. We therefore had to teach the full IEP first year range of modules and two of the old first year modules. There was also some dissatisfaction amongst some of the new intake who were under the impression that they would be taking lectures at LSE. We had been very careful to remove all reference to LSE from our literature for Sept 2014 intake. For the decommissioned modules the Moodle pages still had to be maintained and exam papers still had to be set and marked for those students who had failed and opted to re-sit the exam. While this is only a handful of students in any module, it is a not insignificant extra load. 1 Year 2 Academic year 2014-15 was the last year of running for most second-year modules as next AY (2015-16) they will be replaced by a new Integrated Engineering programme (IEP) with one module (maths) shared across the Engineering Faculty and one optional module in T2 forming the first part of a “minor”. This has provided an excellent opportunity to review and refresh the material during 2014-15. Thus each module coordinator was able to review their module and adapt as necessary to fit into the revised 2nd year programme paying particular attention to design of coursework (CW) and how their module integrated with the 4 one week scenarios. The organisation, timing and manner in which laboratory classes were conducted was a major cause for concern as the space available was totally inadequate and due to class sizes it could take as much of 8 weeks to cycle a whole cohort through on lab. Two consequences of this were that the lecture and labs were not tightly connected and timely feedback was difficult to provide in a way that is fair to all students. Next year with the IEP the concept of a lab week will be introduced. In lab week students have no lectures only lab classes. A student would have 3 or 4 labs in lab week. Lab week will replace reading week which has never been recognised in this Department. The roll out of the IEP will be an issue for students on the “old” programmes who fail to progress to the third year. The use of referrals will be helpful here. As with 1st year students who were in a similar situation this year it will be treated on a case by case basis. An acceptable solution was found for all students who failed 1st year. Year 3 Standard modules were run effectively with similar issues mentioned elsewhere. In particular, the efficiency of feedback to students has been targeted, through streamlining assessments and adjusting the timetables of deadlines. 3rd year individual projects involve a large effort from our teaching staff, but the organisation has been efficient this year. AY 15-16 is the last time that the third year modules will run and they will all be replaced by new modules as the IEP rolls out. The individual project is the exception to this and apart from re-branding will have no major change. Year 4 As the majority of the 4th year modules are common with the MSc students it makes more sense to deal with these in the PGT report. However two points of note are worthy of mention here. Students are increasingly becoming involved in competitions and our Formula Student team is improving each year. How we as a Dept . mange students and student competitions needs to be reviewed and formalised as it is rather ad hoc at present. See also external examiners’ comments. As part of the Departmental teaching and leaning strategy we want to grow this and build them into our Minors and generally integrate a few key competitions into the curriculum across earlier years as well as 4th year. 2 The 4th year modules are shared with MSc students which sometimes results in large class sizes (120-150) and diverse backgrounds, which is a challenge for teaching. There is a mismatch in a priori knowledge between our 4th year students and the incoming MSc students. The result is that the first few lectures in the 4th year are levelling material aimed at the MSc intake which our 4th year students find frustrating. We have attempted to address this by producing a set of revision notes (a starter pack) for our incoming MSc students. The group projects have been successful but we are developing the methods of assessment to ensure they remain fair and transparent across a diverse set of project remits, and we are looking into industrial input in the project specifications. External examiners The external examiners would prefer an exam board which was not anonymous, but they understand the requirement. The External examiners have made very positive comments regarding the content and structure of the course, and the appropriateness of the exams. To quote one, “…the examiners and administrators have carried out an excellent job of maintaining quality and consistency. I believe that the UG programme in UCL is competitive and very high in quality.” Both examiners noted that the exam papers have generally been of a very high standard and appropriate for the level being examined, but with worked solutions to all papers and that we implement a more rigorous internal scrutiny of examination papers before they are sent to the external examiners. They suggested that the coordinator of group projects to obtain industrial supervision from willing engineers within industries keeping academic staff within the department as the main supervisors of the group projects. Last year the external examiners suggested an all-staff preliminary exam board to identify any issues spanning across programmes, this was implemented successfully this year. Learning & resources The Department has one of the highest student-to-staff ratios in UCL, as a consequence we tend to focus on the core activity of programme delivery to the detriment of activities such as programme development, QA etc. Preparation for IEP and IQR put an extra load on staff in AY 2014/15. The Department has recruited new staff including three Teaching Fellows (part time) to help address the student-to-staff ratio. We are designing the new modules for the IEP to reduce staff load while improving the student experience. The two areas of concern for resources are lecture rooms (size and quality) and lack of space for laboratory classes. Lecture quality is variable, some of it is high quality whilst some less than good. Where poor teaching has been identified this has been dealt with and is generally associated with new lecturers usually PDRAs who may be intellectually able but lack maturity and knowledge compared to previous years this problem is declining due to compulsory training for all PDRA. Feedback is good for experienced staff. Unfortunately there are also a few isolated instances of senior staff appearing to be disinterested in teaching and undergraduates. This is being addressed by HoD. 3 Feedback In the NSS survey, our lowest marks are for timeliness & effectiveness of feedback and over the last 3-4 years we have been tackling this issue with highest priority. We are making slow but steady progress. A designated coursework coordinator (academic staff) has undertaken a review of all assessments and made changes to rationalise the timing of deadlines. We have publicised the UCL policy on return of feedback within 2 weeks (or 4 weeks for very large classes) and agreed to that as a Department. An administrative assistant has updated and maintains the record of deadlines for all modules on their Moodle page and uses that timetable to send reminder emails to staff when feedback is due. Where necessary the HoD will chase overdue items. We still hear some instances of feedback being late or absent, e.g. in our Staff-Student Committee and the HoD has clearly asked students to bring examples of this to him so that he can take action. The majority of staff engage in this process but there are still a few who are effective “black holes”. A complete review of our marking and feedback processes are planned for AY15-16. The use of lecture cast is another repeated request from UG students. There is significant resistance to its use by many members of the Department and supportive, gentler nudge approach need to be used. Two members of staff agreed to trial lecture cast in AY 14-15 but unfortunately one of them was allocated a lecture theatre that did not have the facilities. The other will present his experiences at the next Dept meeting and act as a lecture cast champion with the aim of encouraging other members of staff to take it up. A third common feedback theme is the desire for more tutorial sessions. With the current student-to staff ratio – the least favourable in UCL - this is a challenge, and we are making efforts to make more effective use of Postgraduate Teaching Assistants in this regard as a recognized contribution to the teaching team. The role and (limits of) responsibility have been formally defined and publicised and we introduced a full day training session which is now compulsory for PGTAs to ensure standards of awareness of policy and practice. Estates The students’ most frequent comments are regarding the estates & facilities, including locations and suitability of lecture theatres. Sometimes these are easily fixed (e.g. poor audio facilities in a lecture theatre), but there is a significant issue of space which is becoming more pressing as student numbers rise. As many of our classes are over 120 (intake in 2014 was 147 +10 affiliates) there are very few lecture theatres in UCL that are large enough. We have procured new laboratory equipment which is portable and can be temporarily installed and removed to allow the space to be used for several purposes through the academic year. The facilities in Birkbeck come in for the most criticism from students and staff. Staff We use some external teaching staff in parts of modules and often this is very effective but this does raise issues of the staff not being available and occasional differences in style and awareness of regulations. We are seeking to reduce our use of external lecturers by recruiting more permanent 4 academic staff and teaching fellows. The professional services office are now in single site bringing the team together physically for more efficient operation and a clearer hub for staff and students to approach. This includes a new reception desk which is a focus for students, staff and visitors. This has been a significant impact as student now have a clear end effective first point of contact. Peer observation of teaching In AY 13-14 we had a very poor take up and a major effort was made to improve this for AY14-15. Two actions to improve this were for support staff to monitor and remind staff about peer observation. The other was to create a list of experienced staff to act as observers – this proved successful and was identified as an item of best practice in our IQR visit. However while 25 peer observations were recorded (up from 3 the year before) as completed 18 staff involved in teaching did not have a peer observation of teaching. Degree classification. We are very satisfied with the distribution of final awards for the UG programmes, at first sight the BEng results might look poor but this is because the majority of the stronger students are on the MEng programmes, this pattern is seen nationally for pre 1992 universities offering UK Mechanical Engineering at BEng. A significant proportion of our better BEng students apply for our MSc programmes. Overall we have a higher percentage of 1st class awards than the UCL average and a lower percentage of 2A class awards than the UCL average . Looking at just the MEng programmes our award distribution is slightly higher than the national average (2012 data) for Mechanical Engineering degrees. Table 1. Distribution of degree classifications Route Name 1st 2i 2ii 3rd/Pass Total 1st 2i 2ii 3rd/Pass BEng Mech Eng 12 8 8 2 30 40% 27% 27% 7% BEng EBF 7 6 8 21 33% 29% 38% 0% MEng Mech Eng 12 11 23 52% 48% 0% 0% MEng EBF 7 9 16 44% 56% 0% 0% 38 34 16 2 90 42% 38% 18% 2% UCL in total 1345 1944 361 42 3692 36% 53% 10% 1% UK MEng* 341 409 34 3 787 43% 52% 4% <1% TOTAL *2012 Data for all pre 1992 universities in UK Mechanical Engineering MEng(1) Table 2 shows that for all designations of student we have a higher percentage of 1st class awards than the UCL average and a lower percentage of 2A 5 class awards than the UCL average. However as with UCL data our BME students are performing less well than our White students. Note that for some designations of student our sample size is very low so % values should be treated with caution. A similar picture i s shown in Table 2. Note that the females outperform the males. This is often the case in engineering due regrettably to the tougher road they have taken to enter the programme. Again due to low numbers the data should be treated with caution. Table 2. Dept UG award ethnicity and domicile comparison Ethnicity UK domicile BME 1 2A Mechanical Engineering 35% 43% Mechanical Engineering 8 UCL UCL Overseas White 2B/3/P BME White 1 2A 2B/3/P 1 2A 2B/3/P 1 2A 2B/3/P 22% 45% 45% 9% 43% 39% 18% 75% 0% 25% 10 5 5 5 1 21 19 9 3 30% 60% 10% 37% 58% 5% 37% 50% 13% 45% 46% 9% 14 625 103 560 875 79 324 437 118 164 170 33 1 Table 3. Dept UG award ethnicity and gender comparison Gender Ethnicity Male Female BME White 1 2A 2B/3/P 1 2A 2B/3/P 1 2A 2B/3/P 1 2A 2B/3/P Mechanical Engineering 42% 40% 18% 47% 33% 20% 40% 40% 19% 53% 33% 13% Mechanical Engineering 30 29 13 7 5 3 29 29 14 8 5 2 UCL 37% 53% 10% 35% 58% 7% 33% 55% 12% 38% 56% 6% UCL 678 977 187 712 1191 154 638 1062 221 724 1045 112 Admissions and recruitment As is typical for the discipline in the UK we have a high male to female ratio, but is similar to many other Mechanical Engineering Depart in the UK. We would like to see it reduce. 6 Our admission numbers are strong with a significant number of overseas students. There is also concern that he overseas is dominated by Chinese students. We are keeping an eye on long term trends to see if we can identify any impact of the IEP or the switch from LSE to MS&I but with so many variable at paly it is difficult to identify cause and effect. Table 4 shows a summary of the recruitment pattern for our students, more detail is shown in in Figures 1 and 2, Table 5 shows a breakdown by fees status. Nearly half our intake is now non EU much higher than the UCL average, conversely our UK intake is much lower than the UCL average. The proportion of non EU students has been steady increasing and 50% is probably our limit. In the AY 14-15 from our total intake of 147 students 48(33%) were from state schools and 40 from independent schools (27%). We are pleased with our gender split with 117 (80%) male and 30 (20%) female, this may appear low for females but nationally Mechanical Engineering only attracts 8% females the lowest of any engineering discipline(1). Table 4. Recruitment summary Route Name Applications Offers Acceptances Intake Affiliate Mechanical Engineering 15 11 10 10 BEng Engineering (Mechanical) 596 257 117 59 MEng Engineering (Mechanical) 593 393 107 55 BEng Engineering (Mechanical w Business Finance) 181 59 35 22 MEng Engineering (Mechanical with Business Finance) 155 69 27 11 1525 778 286 147 Total (ex affiliates) Table 5. Breakdown by fees status Mech Eng* UCL UK Other EU Non EU 37% (55) 15% (22) 48% (77) 50% 15% 35% *Excluding affiliates Over the past 5 year the number of applications for the Mechanical Engineering programmes has more than doubled from 561 to 1210. In comparison the business finance applications have only risen 30% from 265 to 349. Hence the proportion of business finance entries is dropping from over a third 5 years ago to les then a quarter now. The intake for the Mechanical Engineering programmes has increased by just 50% from 73 to 114 while the intake of the Business Finance programmes has seen a slight decrease from 42 to 33. The current Dept policy is 7 to have about 40 on this programme. We will try to raise the number back to the 40 mark for AY 16-17 intake. Figure 1. Mechanical Engineering recruitment data Figure 2. Business Finance recruitment data Reference 1. “Pathways to success in engineering degrees and careers” Pub RAEng July 2015 ISBN: 978-1-909327-12-2 http://www.raeng.org.uk/publications/reports/pathways-to-success 8 Academic Partnership Activity: Programme Name: BEng Engineering With Business Finance Programme Name: MEng Engineering With Business Finance Priorities for Action: Historically some modules for Y1 and Y2 of these programmes have provide by the London School of Economics. These two programs are being phased out and replaced by the IEP which makes use of modules provided by MS&I. AY14-15 was the last time that our Y2 students took modules from the LSE, no problems or issues were reported. Our Y1 students are on the IEP. A few students will be taking resits of LSE modules next year. So far the winding up of our academic partnership with the LSE has gone smoothly. 9 Source Issue Action DEVELOPMENT AND ENHANCEMENT PLAN Source Issue Action Person Responsible Progress Implementation Evidence Date Comments/ Issues *NSS – National Student Survey, EE – External Examiner Reports, DS – Data Set, PTES – Postgraduate Taught Experience Survey UGT – UG tutor, DoS – Director of Studies, DM – Dept manager Person Implementa Comments/ Progress Evidence Responsible tion Date Issues Organisation and Management [NSS, EE, etc*] SSCC and staff [Identify key issue] Laboratory timetabling was a Cinderella exercise leading to confusion and poor arrangements. [Identify agreed action for resolution] A member of T&L staff has been dedicated to this task but it is complicated by the roll out of the IEP [Identify responsibility] T&L PS team have taken this on board. [identify start date, or mark complete] On going Submit initial documentation DoS Accreditation Review of syllabus and benchmarking against QAA descriptor Preparation started Sept 2015 DoS Accuracy of exam papers Create subject review panels to review exam papers for accuracy and consistency across subject Academic responsible for exam papers with support from DoS, UGT tutor and T&L PS team Teaching 10 Approved at DTC 20/10/15 [Identify evidence that issue has been resolved i.e. NSS etc*] Aug 2015 Initial documentation to be submitted by Nov 2015 Full submission June 2016 Prepare for visit in Oct/Nov 2016 EE [Identify date of implementation] From Nov 2015 EAB visit to cover UG and MSc programmes. An added benefit will be development of subject themes through the years for the programme. Source Faculty Faculty Faculty Issue IEP IEP IEP Person Responsible Progress Development of new modules for Y3 of IEP DoS with nominated IEP strategy team On going Development of Maritime minor for IEP Dr Bradbeer (Y2 T2 module) and Prof Thomas (Y3 T2) module Action Module for Y3 T1 of Regenerative Minor Module Coordinator Implementation Evidence Date Comments/ Issues Ongoing, Oct 2016 UPC due Dec 2015 Preparation started Oct 2015 Significantly delayed and at risk Jan 2017 Ongoing, UPC due Dec 2015 June 2015 UPC twice rejected by DTC. The Module Coordinator still has not provided outline syllabus. Mod Coord is failing to engage with process. Consider an alternative or abandon? Assessment and Feedback NSS IQR (# 3) UCL NSS EE UCL NSS Clarity and Accessibility of Marking Criteria Quality and Promptness of Feedback NSS Assessment and Feedback UCL Mandated Action: You are asked for explicit confirmation that all modules and programmes have clear assessment and marking criteria and that these are available and clearly publicized to students. DM, Comms, Marketing & Events Manager, Senior T & L Administrator UCL Mandated Action: You are asked to develop and approve minimum standards for feedback within the department for implementation during this academic year. UGT and DoS for communicating responsibility of Module Co-ords in process to academics. You are asked to confirm explicitly that effective mechanisms exist to ensure that feedback on assessment is returned to students within four weeks in line with UCL’s service standards document. T&L PS team for sending reminders and chasing emails UCL Mandated Action: you are asked to benchmark your practice Required for accreditation Ongoing complete 21/9/15 Approved DTC 20/10/15 HoD if issues are escalated. ISLE and 11 Action point 1 from the Improving the Student Learning Experience (ISLE) meeting 21/9/15 Nov 2015 Source UCL Issue Benchmarking Staf SSCC Project guidelines DoS students HoD IQR (#3) Review of CW Loading Student assessment Action against the NUS Assessment and Feedback Benchmarking Tool and take immediate action where needed, paying particular attention to this year's finalists in the first instance. Create clearer guidelines for staff and students on what is expected for 3rd year individual projects and 4th year group projects in terms of outcomes. Both for students to assist with writing final reports and for staff when marking. Rolling review of what the labs entail in order to refine them with a view to maintaining learning outcomes but eliminate non useful output and streamline marking. ISLE to report back to DTC on this Transforming the Experience of Students Through Assessment (TESTA) audit by CALT Person Responsible Progress Implementation Evidence Date Comments/ Issues DTC 3rd yr indivudal project module coordinator 4th yr group project module coordinator UGT to lead Not started. Ideally in place by Feb 2016 Started 21/9/15 Audit requested, CALT have recommended that we wait until next term. DoS UGT DM Spring 2016 Learning Resources Everyone Many lecture rooms are Keep complaining! We provide ? 12 Getting ? ? I know everyone Source Students SSCC IQR (#2) SSCC IQR (# 8) Issue Action substandard and do not have the facilities we require especially some of those in Birkbeck. Overcrowding is also an issue especially in the first few weeks of term. advance warning of class size growth but this is ignored. Lecture cast: Person Responsible Progress Implementation Evidence Date worse Comments/ Issues is fully aware of this bit must be repeated. Limit student numbers? Double teach classes but that would add a significant teaching load and increase demand on rooms. Need to win over staff and encourage them to use Lecture Cast Concerns raised by Year 4 students registered for the MEng programme who at times have had to repeat elements of the curriculum already covered in modules undertaken earlier in their programme. An induction pack has been designed and the use of this will be introduced for the 2015 intake. Its effectiveness will be monitored from analysis of student performance and feedback. The induction pack will be updated as a consequence. Promote greater and more consistent use of Moodle across subjects to all module convenors within the Department. Student representatives noted differences between modules The Department introduced a Moodle baseline for all modules and all pages were redesigned. Many staff are avid Moodle users, but some are less so. The department is now addressing the issue of wanting to develop its use of Moodle within the context of DoS, UGT, Dr XXX Dr XXX tried lecture cast in AY14-15. He is to present at next Dept meeting about his experience. Students are keen on this but there is significant staff reluctance UGT, DoS Graduate Tutor (Taught) HoD, DM, UGT, Senior T & L Administrator Completed Phase 1 to be completed by March 2016 Moodle content will be continually 13 Sept 2015 The effectiveness of the induction pack will be kept under review. Source Issue Action whereby some failed to provide relevant information, including assessment criteria, samples of previous assessments and model answers. different levels of engagement and different needs of individual teaching staff. No one wants to hold back those making good use of Moodle just so that their courses are uniform with those who use it less, but we do understand students are finding the very different feels difficult. Moreover, we feel strongly that it is important to allow academics to decide how much they feel engaging with Moodle is appropriate for the content of their modules and their own teaching styles. So whilst a consistent approach to using Moodle will not be achieved, or even strived towards in a rigid manner, the department is instigating the following steps to improve the current situation and is fully committed to increasing the use of Moodle within the department. The HoD has instigated a review of teaching materials used across the department to allow for them to be pulled into a uniform template, so that the documents uploaded to Moodle will have a branded and uniform appearance, are error-free and are typeset at very high professional standards, giving a feeling on consistency for students. Person Responsible Progress monitored and kept up to date 14 Implementation Evidence Date Comments/ Issues Source Issue Action Person Responsible Progress Implementation Evidence Date Comments/ Issues These will remain working documents so that the academics will be able to update them when necessary. A publishing professional is already hired and in place (on a temporary basis) in order to facilitate hands-on this transition. The use of Moodle will be championed by academic staff with senior teaching positions and good examples, such as the MSc Naval Architecture pages, will be shared as examples to encourage development. The Professional Services staff will retain an overview of Moodle content and ensure that it remains up to date and relevant and report anything out of date to module coordinators initially. Personal Development ISLE IQR (# 4) Personal tutoring, and ways of encouraging both sides to engage more, and to make it as useful as possible. Plan the discussion points and relevant times of year – create a schedule for these. Communicate this with students and staff. UGT, Comms, Marketing & Events Manager, Senior T & L Administrator, Year coordinators BME Progression and Achievement 15 Ready to go AY 15-16 Y3 students now have their project supervisor as their P tutor. Source Issue Action Person Responsible Progress Implementation Evidence Date Comments/ Issues No issues identified Widening Participation IQR (#9) Enhance communication and engagement with prospective students who have been offered a place on a programme Review methods of communication and support provided for affiliate students by considering what IQR (# 10) improvements could be made to enable greater interaction between the Dept. and the students Comms, Marketing & Events manager & Senior T & L Administrator have designed a keep warm series of events and communications to new students to help them stay connected to the department and to instigate the feeling of belonging before they commence their studies. DM, Comms, Marketing & Events Manager, Senior T & L Administrator This has been identified as a problem caused by a lack of vigilance when building the mailing lists for the programmes. Professional Services staff have been sensitised to this issue and have included tracking of affiliate students taking modules as a process for maintaining departmental student mailing lists. DoS, Affiliate Tutor, UGT, DM, Senior T & L Administrator Creation of new post : Enterprise & Projects Industrial Liaison Administrator New appointee Completed July 2015 Completed Academic Support SSCC, EE Lack of support and focus for student – industry links including internships DATE OF SUBMISSION TO FACULTY TEACHING COMMITTEE: 25 Oct 2015 16 appointed 1/10/15 Students will be canvassed for feedback on both points with a view to analysing success and identifying improvements for future intakes. Measures in place, but under continual review as part of maintaining mailing lists The keep-warm series will be reviewed and renewed by the department on an ongoing basis.