Report of External Evaluation and Review Quantum Education Group Limited Confident in educational performance Confident in capability in self-assessment Date of report: 15 August 2014 Contents Purpose of this Report................................................................... 3 Introduction ................................................................................... 3 1. TEO in context.......................................................................................... 3 2. Scope of external evaluation and review .................................................. 5 3. Conduct of external evaluation and review ............................................... 6 Summary of Results ...................................................................... 7 Findings ........................................................................................ 9 Recommendations ...................................................................... 19 Appendix ..................................................................................... 20 MoE Number: 8441 NZQA Reference: C14386 Dates of EER: 31 March‒9 April 2014 Final Report 2 Purpose of this Report The purpose of this external evaluation and review report is to provide a public statement about the Tertiary Education Organisation’s (TEO) educational performance and capability in self-assessment. It forms part of the accountability process required by Government to inform investors, the public, students, prospective students, communities, employers, and other interested parties. It is also intended to be used by the TEO itself for quality improvement purposes. Introduction 1. TEO in context Name of TEO: This external evaluation and review (EER) included all three entities owned by the Quantum Education Group: Quantum Education Group Limited (Ministry of Education number 8441), Quantum Education QT (MoE 8740) and Quantum Education ES (MoE 7425) Type: Private training establishments (PTEs) Location: Head Office Service Centre, 7A William Pickering Drive, Albany, Auckland National Call Centre/Training Centre and Student Enrolment and Support Centre, Level 1 Durham Street, Tauranga Academic Support Centre, 67 Hayton Road, Sockburn, Christchurch Delivery sites: (8441) Quantum Education Group Limited: Multiple drop-in sites nationally: New Lynn, Manurewa, Manukau, Whangarei, Hamilton, Tauranga, Tokoroa, Rotorua, Wanganui, Hastings, Napier, and Riccarton in Christchurch; and the international campus in Auckland CBD (8740) Quantum Education Group QT Limited: International student site, Levels 10-12, 3 Wakefield St, Auckland Culinary Institute of New Zealand, 243 Waipaai Road, Kerikeri Final Report 3 (7425) Quantum Education Group ES Limited: 129 Commerce Street, Kaitaia First registered: 18 August 1997 Courses currently delivered: (8441) Quantum Education Group Limited: • Diploma of Professional Counselling (Level 6) • Certificate in Computing (Level 4) (7425) Quantum Education Group ES Limited: • Certificate in Business Information Systems (Level 3) (8740) Quantum Education Group QT Limited: • Diploma in Hospitality Management (Confederation of Tourism and Hospitality (CTH)) (Level 5) • Diploma in Tourism Management (CTH) (Level 5) • Diploma in Hospitality Management (CTH) (Level 6) • Diploma in Tourism Management (CTH) (Level 6) • Diploma of Professional Counselling (Level 6) • New Zealand Diploma in Business (Level 6) • Certificate in Travel, Tourism Māori and IATA (International Air Transport Association) Fares and Ticketing (Level 3) • General English/IETS (International English Language Testing System) Preparation • Academic English/IELTS Preparation Code of Practice signatory: Yes; 18 years of age and over Number of students: Domestic: 1,425 students, including those in extension (938 EFTS (equivalent full-time students) in 2014) • 67 per cent Māori • 17 per cent Pasifika • 16 per cent other International: 6 per cent of the total student population are international students (majority from India with Final Report 4 others from the Pacific, Nepal, Thailand, Fiji and Brazil). Number of staff: 107 full-time equivalents Scope of active accreditation: In addition to the courses listed above, the organisation has approval for a range of courses in Cookery and Food and Beverage Services, and a range of further local certificates in business, computing and tourism. See: http://www.nzqa.govt.nz/providers/nqfaccreditations.do?providerId=844120001 (8441) http://www.nzqa.govt.nz/providers/nqfaccreditations.do?providerId=874012001 (8740) http://www.nzqa.govt.nz/providers/nqfaccreditations.do?providerId=742535001 (7425) Distinctive characteristics: The majority of learners on the computing and tourism courses complete their learning via distance. The organisation uses a blended approach whereby students also attend drop-in centres. Recent significant changes: Quantum Education QT limited ceased delivery of aviation training in 2012. Previous quality assurance history: The previous EER visit conducted in 2010 determined that NZQA was Confident in Quantum Education Group Limited’s educational performance and capability in selfassessment (also included Quantum ES and Quantum QT Limited). An action plan to meet NZQA national external moderation requirements for the New Zealand Diploma in Business was accepted in 2013, and implementation of the plan was in progress at the time of this EER. Other: The organisation is due to merge into one entity if the sale of the businesses to the Intueri Group goes ahead as planned in June 2014. 2. Scope of external evaluation and review The EER focused on reviewing a range of courses at different levels across a sample of two of the entities. Quantum Education ES was not visited as only a small number of students are enrolled in courses at the Kaitaia site. The courses selected were: • Diploma of Professional Counselling (Level 6) • Diploma in Hospitality and Tourism Management (Level 5) Final Report 5 • Certificate in Computing (Level 4) • Certificate in Travel, Tourism and IATA Fares and Ticketing (Level 3) The EER also included the mandatory focus areas: • Governance, management and strategy • International students. 3. Conduct of external evaluation and review All external evaluation and reviews are conducted in accordance with NZQA’s published policies and procedures. The methodology used is described fully in the web document Policy and Guidelines for the Conduct of External Evaluation and Review available at: http://www.nzqa.govt.nz/providers-partners/registration-andaccreditation/external-evaluation-and-review/policy-and-guidelines-eer/introduction. The TEO has an opportunity to comment on the accuracy of this report, and any submissions received are fully considered by NZQA before finalising the report. All three entities were evaluated at the same time because the organisation has the same governance and management team and support services (including student enrolments, assessment and moderation) across all entities. The EER team consisted of one lead evaluator and three team evaluators, who carried out the evaluation at six sites over four days and a large amount of document submissions were reviewed subsequent to the site visits. The sites included the head office in Albany, the international student centre in Auckland CBD, a student support centre in New Lynn, the national call centre/training centre in Tauranga, the student drop-in centre in Riccarton and the national academic centre in Sockburn, Christchurch. The team interviewed the Quantum Education Group board members, members of the Intueri Group board, the chief executive officer, the national academic manager, national assessment and moderation staff, and tutors and students from each focus area. In addition, the lead evaluator held a phone conversation with the chairperson of Intueri Group. Evidence from the interviews was supported by the documents reviewed on site and subsequent to the EER visit. The documents reviewed included internal evaluation reports; organisation business plans; stakeholder meeting minutes; data on international, Māori, Pasifika and overall student outcomes; Tertiary Education Commission (TEC) educational performance indicator data; course monitoring; tutor reporting and monitoring; and internal and external moderation reports. Final Report 6 Summary of Results Statement of confidence on educational performance NZQA is Confident in the educational performance of Quantum Education Group Limited. The reasons for this confidence rating can be summarised as follows: • Over the past two years, Quantum Education Group has been achieving above the median against the educational performance indicators developed by the TEC. • Overall qualification achievement percentage rates are high for both Māori and Pasifika students. However, there is no breakdown of this data to show how these students are succeeding in comparison with the overall student population. • The organisation is guided by an experienced, long-serving governance/owner team who provide clear and comprehensive advice on education and direction. Any changes or planning are communicated to staff openly, and over the past three years the organisation has focused on providing staff training and upskilling programme staff. As a result, staff are confident in their roles and have the ability to carry them out effectively. • Centralising of student services for all three entities has been effective in managing student enrolment. Students receive timely advice and are directed to the appropriate programme. • Distance learning strategies, including one-to-one tutor sessions, regional workshops and regular fortnightly contact have improved student engagement and contributed to increased qualification achievement. • Quality, student-focused learning support is provided through an increased number of drop-in centres and online communication throughout the learning. • International student support is tailored to individual needs, with additional time provided for academic support, as well as personalised support from the centre manager and support staff. • Programmes are matched to the students’ needs and needs of the standardsetting bodies, providing practical career pathways for students. Areas for improvement: • Better formal analysis and evaluation about individual programmes is needed to assure NZQA that there is similar achievement between centres for the same programmes. Final Report 7 Statement of confidence on capability in self-assessment NZQA is Confident in the capability in self-assessment of Quantum Education Group Limited. Reasons for the judgement in the TEO’s self-assessment capability include: • The board and senior management monitor all centre achievement towards business objectives, including key performance measures to meet TEC targets. • Systematic, ongoing reporting of tutor and centre educational performance is used to monitor progress towards organisational goals and to identify where student achievement needs improvement. This reporting has been used to strengthen student support at specific centres; for example, the Manurewa site has introduced a student support person to help with attendance. It is not known whether this successful strategy will be adopted across other centres. • Organisation-wide internal evaluation and reviews were set up in 2011 to check that tutors, and all other staff including management, have the capability to carry out their roles, and to identify any training needs. • The implementation of an internal evaluation and review system has contributed to the ongoing review of staff performance and areas for development. Areas for improvement: • Detailed analysis of student outcomes on the same courses across centres would help to identify trends in achievement and whether there are any differences that could indicate areas for improvement. • Further analysis is required of Māori and Pasifika achievement, as well as international students in relation to overall outcomes, would help to identify trends, differences or areas where further support is needed. • Better understanding of destinational outcomes is in progress. Again, using the data to provide useful analysis of the destinations of particular groups of students, including international students, following the completion of courses, and identification of any trends, could help identify areas for improvement to programmes and teaching. • While most of the external moderation reports ‒ as well as internal moderation outcomes ‒ are meeting requirements as a result of a focus on improvement, the organisation does not include internal moderation of local components for the level 6 professional counselling programme to confirm that assessments are consistent and valid. • Quantum Education Group has still to generate a benefit from stakeholder advisory groups in programme development to ensure that programme content reflects employer needs in training across the board. Final Report 8 Findings1 1.1 How well do learners achieve? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. Over the past two years, Quantum Education Group has been achieving above the median against the educational performance indicators developed by the TEC. Table 1. Achievement against Tertiary Education Commission indicators, 2011-2013 Course completion (%) Qualifications achieved (%) Higher study (%) Retention (%) Entity 2011 2012 2013 2011 2012 2013 2011 2012 2013 2011 2012 2013 TEC median 86 85 Not avail 80 82 Not avail 23 23 Not avail 75 77 Not avail 8441 82 89 95 82 95 102 27 12 Not avail 65 65 78 8740 92 93 97 86 100 118 23 17 Not avail 70 78 74 7425 89 95 94 71 100 97 33 27 Not avail 63 71 61 The organisation conducted a review of outcomes across all of its sites as a result of lower performance in 2011. This led to the identification of several areas where it could improve, specifically around learner entry, enrolment and support. In response, more drop-in sites and home visits were arranged ‒ enabling blended delivery options ‒ and the entry and enrolment services were centralised for consistency. The organisation’s monitoring of course completion shows a 30 per cent decrease in withdrawals since these actions were taken, leading to improved outcomes against the performance measures set by the TEC. Already for 2014, course completions are around 80-90 per cent, which indicates that people are completing their courses and able to gain a qualification. The 2013 results for Māori and Pasifika students are strong, with 96 and 97 per cent course completion respectively in 2013, and 107 per cent qualification achievement for both, according to the TEC educational performance indicators. Quantum Education Group achieves very strong Māori and Pasifika participation and achievement. Ongoing intake opportunities contribute to the sustainability of 1 The findings in this report are derived using a standard process and are based on a targeted sample of the organisation’s activities. Final Report 9 this success. Although the participation of Māori and Pasifika students over 2013 was 80 per cent, a report presented by Quantum suggested that this had dropped to only 55 per cent at the time of the 2014 EER visit. Quantum Education Group was subsequently able to identify that this was due to a mistake in data reporting, and that the participation had risen to 84 per cent (66 per cent Māori and 18 per cent Pasifika). Quantum Education Group would benefit from a more detailed understanding of Māori and Pasifika participation and achievement trends that contribute to current high levels of success, and the sustainability of that success at higher levels of study. In addition, there was no separate analysis of international student qualification achievement data. The international student data provided following the EER visit was not comprehensive or sufficiently coherent to show how well these students performed in relation to domestic students or to identify where improvements could be made. Better recording and analysis of this data would help to improve understanding of international students’ educational achievement, especially in comparison to domestic students. This is of particular importance as the students interviewed had aspirations to work in New Zealand and become New Zealand residents. All individual course outcomes are monitored throughout the year to identify whether student completion rates will meet TEC targets. This is sufficient to identify where students may have difficulty in achieving, as evidenced by the response to non-completion in computing courses. The course completion rate has consistently been around 80-90 per cent, with near 100 per cent qualification achievement. This rate of achievement is similar for the travel and tourism students. The professional counselling programme has a high completion rate (88 per cent in 2013), but it is difficult to determine whether the 74 per cent qualification achievement is notable or valid without further evidence of internal and external moderation of the outcomes (which are not unit standards-based and make up the majority of the programme outcomes). Nor is there evidence to demonstrate that students are doing well by referencing results to similar courses externally. An analysis of reasons for the lower completion figures for this course in relation to the other courses was also not available. The organisation could better use the data to understand how well international students are achieving on this course, rather than just using it for monitoring completion. Overall, the organisation is monitoring progress against targets set by the TEC for each entity, across courses and for each site. This helps to identify where a site may have issues with achieving these targets. However, there is insufficient analysis to show whether the sites delivering the same course are achieving similar results, or whether there are specific student groups that are not performing as well as other groups. The amount of data recorded would enable useful internal benchmarking analyses to use in conjunction with current weekly centre outcome reports. Final Report 10 1.2 What is the value of the outcomes for key stakeholders, including learners? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. Quantum Education Group enrols students who are mostly over 25 years of age. Many who have not previously gained a qualification, or they are looking for tertiary qualifications to assist them into further study and/or employment. The TEC educational performance indicators show that around one-third of students go on to further study, of whom three-quarters enrol in courses within the organisation. This is the general desire of students enrolled in the lower-level computing and tourism and travel courses. There is no formal analysis of employment outcomes matched with career intentions to demonstrate the value of the qualifications to the students. This analysis could be carried out using student entry profiles against destinational data collected to show which courses are of value to graduates seeking employment. The Diploma in Hospitality and Tourism Management students interviewed by the EER team had clear intentions to work in the sector or to enrol in higher study. Although there was anecdotal evidence to support that students did this, as well as supporting data the organisation has begun collecting over the past year, there was a lack of analysis to determine whether goals were being met. Around 60 per cent of students enrol based on word-of-mouth recommendations from family and friends, indicating the popularity of courses. The potential purchase by Intueri Group is an opportunity for the organisation to offer students education at a higher level, or offer them different career pathways with other training partners within the group. The EER team heard from students and tutors about the value of specific courses, such as the professional counselling programme. Students increased their communication skills and experienced personal growth, which helped to improve their personal relationships and practicum experience. On completion of the course, graduates can become a member of the New Zealand Association of Counsellors (NZAC). The lack of formal analysis of student outcomes ‒ such as the number who gained employment ‒ meant that the value of the training for employment outcomes was not known. Anecdotal information about the remaining courses that were a focus at this evaluation confirmed that students were enrolling in other courses with the organisation and finding employment. Overall, graduate and employer surveys, along with better analysis of destinational data, would help to demonstrate the value of the qualifications to students and employers, or help identify any issues with the training. In addition, there is a lack of evidence of comparative analysis of Final Report 11 outcomes by course, sites and different ethnic groups to demonstrate the value of qualifications for different groups for achieving their goals. 1.3 How well do programmes and activities match the needs of learners and other stakeholders? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. The computing and travel and tourism courses offered across the majority of the sites use a blended learning approach with self-paced learning workbooks and tutor-led workshops at drop-in centres nationally. This suits the students’ life situations where many have family and work commitments to manage alongside their studies and are located outside of main centres. The workshops also provide opportunities for individual and peer support. Most sites provide night classes to enable students to attend for extra study after work. The national provision of replicated workshops has contributed to the higher attendance and lower drop-out rate of students because they are able to pick up the course if they move towns/cities. The tutors regularly review the content and activities for courses at each site to ensure they meet students’ learning styles and to update any activities. The overall course reviews are conducted by the academic team and tutors annually using tutor reports, internal moderation reports and student course evaluations to check that the courses are current and align to the job market. The course workbooks are mainly developed by external developers contracted by Quantum Education Group, and the developers complete any required amendments as a result of academic reviews. Student course evaluations and feedback to the EER team was positive about the quality of the learning activities and confirmed that they assist learning. The organisation has recently updated all of its computers across all sites as a result of feedback from centre reports. The professional counselling programme uses the resources and texts developed by the previous owners of the programme, which was bought by (8740) Quantum Education Group QT in 2006. The programme uses workbooks and online resources to cover the basic therapies used in counselling. Students then attend the required number of seminars and complete sufficient theory workbooks to complete the qualification. Feedback on the seminars is mostly positive, as they are seen as a critical component of the course to enhance learning and provide value through practical activities and the opportunity for peer review and feedback from tutors. The seminars also contribute to the face-to-face counselling practice hours required for professional membership of NZAC. Final Report 12 Reviews of this course are carried out through tutors’ engagement with stakeholders and practicum supervisors, and through student course evaluations. Recently, Quantum QT altered the practicum hours to meet NZAC requirements for 200 practice hours. Quantum Education Group needs to ensure that there is sufficient moderation to ensure that assessment activities were clear and understandable for all components of the programme. In addition, students at the international site were unclear about how they could access the counselling network to find practicum placements to complete their qualification. International students enrol in the tourism and hospitality courses because they wish to gain New Zealand qualifications relevant for employment or to enrol in higher-education programmes. The hospitality and tourism tutors use their professional networks to help students find work placements and assist international students to prepare curriculum vitae to find employment in New Zealand and overseas. There is some anecdotal evidence that students are employed by large hotel chains, but further analysis would help to identify whether the course is matching hospitality employer needs for international workers, or whether students are entering into further training here or overseas. The organisation has to revise the Diploma in Hospitality and Tourism Management course assessments because the course owner, the UK-based Confederation of Tourism and Hospitality, has changed the assessment requirements to projectbased rather than examinations. This has caused some confusion for current students as to course requirements. The organisation has responded by providing more tutorial sessions and is looking to align work placements with the level 6 (diploma) curriculum to help with understanding. Currently, all international students can complete low-level hospitality training to help find part -time work while studying. The organisation would benefit from more formal feedback from graduates and employers of graduates to understand whether the programmes are meeting their most important needs. The external advisory groups mainly consist of tutors, with little opportunity for graduates and employers not involved in the delivery to provide feedback. 1.4 How effective is the teaching? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. Many of the systems put in place following the organisation’s review of student achievement have helped to improve attendance, and course completion rates have subsequently risen. Changes include modifying the activities used for distance learning courses to suit the students’ learning styles better, which has helped to create a more conducive learning environment. For example, the Final Report 13 development of more drop-in centres for face-to-face learning support ‒ and organising workshops for people completing similar workbooks to access one-toone tuition and peer support ‒ are engaging the learners, as evident from the organisation’s own review and student course evaluations. Quantum Education Group employs tutors with an industry background, and many continue to work part-time alongside their tutoring roles. This helps to develop teaching activities using current industry scenarios as learning examples. Tutors across the organisation’s three entities receive professional development training in adult education and adult literacy education. This has been a focus for the governance team over previous years, and all tutors currently hold these qualifications, with the exception of new tutors who are still completing the training towards the qualifications. This was identified as a need to ensure that students were receiving appropriate teaching in accordance with their learning needs. In addition, the tutors meet together on a regular basis at each site and across the whole of the organisation annually to review their teaching effectiveness. These strategies contribute to the consistent approach and development of shared teaching resources. All assessments are sent to markers with specific sector knowledge, based in Christchurch. This system was introduced following the organisation’s review of moderation systems three years ago and was made to enhance consistency and ensure assessments were at the national level in response to NZQA’s adverse national external moderation report. In some cases, the students felt disconnected because they did not have the same relationship with the marker as with the tutor. However, tutors said the new system enabled more objective marking practices, and they could still provide personalised feedback to the students about their assessments to identify areas for improvement. The new arrangement also facilitates better moderation because moderators can comprehensively cover the unit standards over the duration of the moderation plan to ensure assessments materials and marking judgements are accurate. The professional counselling tutors are required to operate within the ethical guidelines of the counselling profession, and the teaching environment reflects these guidelines. The sector’s industry training organisation (Careerforce) has not requested the three unit standards involved in the programme for moderation since its merger with Te Kaiawhina Ahumahi (the industry training organisation previously responsible for social services training). These have been internally moderated with good results by the academic team moderators. However, this does not provide a robust validation of the course outcomes as there was no evidence that the local components, which comprise a significant majority of the course, had been moderated. However, the organisation has developed a moderation plan that includes a comprehensive internal moderation of the local components of the programme. Final Report 14 International students are able to attend additional English language classes to help them with their studies, and there is good one-to-one support available from their tutor for academic writing as well as employment skills. The academic team and management monitors tutor performance weekly using individual student progress reports and overall centre results. Tutors also meet with a centre manager each Friday to review student achievement. This helps to identify any gaps and ensure that students are making progress. Tutor performance is monitored using the centre outcome reports, and annual 360degree feedback is conducted to help identify areas for development or to confirm good performance. 1.5 How well are learners guided and supported? The rating for performance in relation to this key evaluation question is Excellent. The rating for capability in self-assessment for this key evaluation question is Good. The centralisation of student enrolment processes is working well to encourage and support student achievement. Staff members at the enrolment centre in Tauranga check each application to ensure that the relevant information is received, including IELTS levels. A course advisor previews each application to determine whether the course is suitable for the students or whether they should be referred to a precourse taster if required. The students then receive their course materials and are contacted by their tutor within two weeks of the course start date to introduce themselves and answer any queries the students may have about the course. Regular contact via email, social media, telephone and face to face creates an ongoing relationship where the students feel comfortable to contact the tutor if they require assistance. Tutors maintain a contact log to monitor students’ activity and will contact them if it appears they are falling behind with assignments. Home visits are also part of the strategy, where students with low attendance are identified through the weekly reports and contact log, and the tutor will arrange one-to-one support. These processes are helping with the students’ engagement, as demonstrated by the increased participation and completion of courses, and lower withdrawal rates (from 40 per cent to 10-20 per cent) in the past few years, noticeably in Manurewa. The tutors provide a safe learning environment for the professional counselling seminars, where students are required to sign a confidentiality agreement because of the nature and content of the peer and tutor sessions. Tutors are trained in handling the sensitive topics that are covered during the seminar, and regularly keep in contact to provide appropriate levels of support. International student support is managed at the central city site in Auckland, and staff members on site help students with advice about course requirements and living in New Zealand. International students are required by their visa conditions Final Report 15 to maintain a high attendance rate, and this requirement is closely monitored by the centre manager. Homestays are arranged for these students, but many choose to live independently. International students are provided with access to English language classes or are referred to external courses if they need help with learning and maintaining compliance with student visa requirements. The organisation has implemented numeracy testing as part of the initial enrolment process, but there is insufficient data to conduct an analysis to see whether there are significant gains in numeracy as a result of the teaching, or whether the literacy testing might be a useful tool to identify any gaps. The organisation could also work to identify organisation-wide approaches to providing culturally appropriate support systems for the high number of Māori and Pasifika students. 1.6 How effective are governance and management in supporting educational achievement? The rating for performance in relation to this key evaluation question is Good. The rating for capability in self-assessment for this key evaluation question is Good. The organisation’s board provides clear guidance to the management team and regularly monitors progress towards educational and business goals. The ability of the organisation to implement changes successfully demonstrates how well the board and senior management team communicate changes to staff. There is evidence that staff members are consulted about changes, as shown by the successful implementation of the centralisation of services as a result of the organisation-wide review, and with the potential purchase of the organisation by Intueri Group. The organisation also provides staff with training to upskill them to adapt to new systems or business requirements. The organisation has established a system of ongoing internal evaluation and review, which it uses to identify how well tutors and all other staff, including management, are performing against the NZQA key evaluation questions. This has identified areas for staff professional development, which is one of the mechanisms for improving student engagement and review of the business plan to check that strategies are working. The organisation complies with the Code of Practice for the Pastoral Care of International Students, and recent reviews have led to the employment of an additional staff member in the support role to assist with the orientation and ongoing care of students. Agent services are reviewed to ensure they are providing correct advice to students coming to New Zealand. Further follow-up of student destinational outcomes would help the organisation to understand whether students are achieving their educational and career goals. The business strategy lacks a specific Māori strategy. It would be prudent to develop such a strategy considering that 67 per cent of the current student roll is Final Report 16 Māori, which is above the 2014 enrolment target of 65 per cent Māori. The organisation has also not analysed Māori educational and destinational outcomes in comparison with other students on similar courses to ensure that the programmes and support match their needs for qualifications and employment. A Māori advisory committee has been set up to provide advice, which could help with establishing relevant indicators and goals for this student demographic. Ongoing review of performance against the TEC measures ‒ using weekly tutor reports and centre reports ‒ is used to identify any gaps. This is enabled by the student support system and centralisation of student records. The management team reviews centre course results to identify where additional support is needed, which has led to improved completions at sites. However, this information could be better analysed by using qualification outcomes for similar courses across centres for internal benchmarking and comparison over time. Final Report 17 Focus Areas This section reports significant findings in each focus area, not already covered in Part 1. 2.1 Focus area: Governance, management and strategy The rating in this focus area for educational performance is Good. The rating for capability in self-assessment for this focus area is Good. 2.2 Focus area: International students The rating in this focus area for educational performance is Good. The rating for capability in self-assessment for this focus area is Adequate. 2.3 Focus area: Diploma of Professional Counselling (Level 6) The rating in this focus area for educational performance is Good. The rating for capability in self-assessment for this focus area is Good. 2.4 Focus area: Diploma in Hospitality and Tourism Management (Level 5) The rating in this focus area for educational performance is Adequate. The rating for capability in self-assessment for this focus area is Adequate. 2.5 Focus area: Certificate in Computing (Level 4) The rating in this focus area for educational performance is Good. The rating for capability in self-assessment for this focus area is Good. 2.6 Focus area: Certificate in Travel, Tourism and IATA Fares and Ticketing (Level 3) The rating in this focus area for educational performance is Good. The rating for capability in self-assessment for this focus area is Good. Final Report 18 Recommendations NZQA recommends that Quantum Education Group: • Collect and review course outcomes across sites to enable better comparative analysis. • Enhance systems to review domestic and international data across campuses, courses and differing levels for educational and destinational analysis. • Incorporate analysis of activities identified in internal evaluation review to show whether they have been effective in improving outcomes for students. • Review the composition of the external advisory groups to include input from graduates and employers. Final Report 19 Appendix Regulatory basis for external evaluation and review External evaluation and review is conducted according to the External Evaluation and Review (EER) Rules 2013, which are made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. Self-assessment and participation in external evaluation and review are requirements for maintaining accreditation to provide an approved programme for all TEOs other than universities. The requirements are set through the NZQF Programme Approval and Accreditation Rules 2013, which are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. In addition, the Private Training Establishment Registration Rules 2013 require registered private training establishments to undertake self-assessment and participate in external evaluation and review, in accordance with the External Evaluation and Review Rules (EER) 2013, as a condition of maintaining registration. The Private Training Establishment Registration Rules 2013 are also made by NZQA under section 253 of the Education Act 1989 and approved by the NZQA Board and the Minister for Tertiary Education, Skills and Employment. NZQA is responsible for ensuring non-university TEOs continue to comply with the rules after the initial granting of approval and accreditation of programmes and/or registration. The New Zealand Vice-Chancellors’ Committee (NZVCC) has statutory responsibility for compliance by universities. This report reflects the findings and conclusions of the external evaluation and review process, conducted according to the External Evaluation and Review (EER) Rules 2013. The report identifies strengths and areas for improvement in terms of the organisation’s educational performance and capability in self-assessment. External evaluation and review reports are one contributing piece of information in determining future funding decisions where the organisation is a funded TEO subject to an investment plan agreed with the Tertiary Education Commission. External evaluation and review reports are public information and are available from the NZQA website (www.nzqa.govt.nz). The External Evaluation and Review (EER) Rules 2013 are available at http://www.nzqa.govt.nz/assets/About-us/Our-role/Rules/EER-Rules.pdf, while information about the conduct and methodology for external evaluation and review can be found at http://www.nzqa.govt.nz/providers-partners/external-evaluation-andreview/policy-and-guidelines-eer/introduction/. NZQA Ph 0800 697 296 E qaadmin@nzqa.govt.nz www.nzqa.govt.nz Final Report 20