Report of External
Evaluation and Review
The College of Future Learning New
Zealand (FutureCOL)
Highly Confident in educational performance
Confident in capability in self-assessment
Date of report: 14 August 2013
Contents
Purpose of this Report................................................................... 3
Introduction ................................................................................... 3
1. TEO in context.......................................................................................... 3
2. Scope of external evaluation and review .................................................. 5
3. Conduct of external evaluation and review ............................................... 5
Summary of Results ...................................................................... 7
Findings ........................................................................................ 9
Recommendations ...................................................................... 17
Appendix ..................................................................................... 18
MoE Number:
8698
NZQA Reference:
C10292
Dates of EER visit:
4 and 5 March 2013
2
Purpose of this Report
The purpose of this external evaluation and review report is to provide a public
statement about the Tertiary Education Organisation’s (TEO) educational
performance and capability in self-assessment. It forms part of the accountability
process required by Government to inform investors, the public, students,
prospective students, communities, employers, and other interested parties. It is
also intended to be used by the TEO itself for quality improvement purposes.
Introduction
1. TEO in context
Name of TEO:
The College of Future Learning New Zealand
(FutureCOL)
Type:
Private training establishment (PTE)
Location (head office):
304 Ellison Road, Hastings
Delivery sites:
Hastings and Wairoa
First registered:
23 July 1992
Courses currently
delivered:
•
National Certificate in Computing (Level 3)
•
National Certificate in Hospitality (Cookery)
(Level 4)
•
National Diploma in Computing (Level 5)
•
Certificate in Cookery, Bar and Restaurant
Services (Level 3)
Code of Practice signatory
for International Students
Yes
Number of students:
Domestic tertiary:
•
Youth Guarantee ‒ nine students
•
Future-Focused Training Opportunities
(FFTO) ‒ eight students
•
Student Achievement Component (SAC) ‒
24 students
International: up to 25 students at present
Number of staff:
14 full-time equivalents
3
Scope of active
accreditation:
Distinctive characteristics:
Unit standards required for:
•
National Certificate in Business (First Line
Management) (Level 4)
•
National Certificate in Business (Small
Business Management) (Level 4)
•
National Certificate in Business
Administration (Level 4)
•
National Certificate in Computing (Level 3)
•
National Certificate in Design (Level 3)
•
National Certificate in Hospitality (Cookery)
(Level 4)
•
National Diploma in Computing (Level 5)
•
Certificate in Cookery, Bar and Restaurant
Services (Level 3)
•
Certificate in Small Business and
Supervisory Management (Level 4)
FutureCOL provides education from the senior
secondary-school level (via Alternative Education
and other programmes) to level 4 tertiary
education.
FutureCOL is a self-identified Māori provider and
has a significant proportion of Māori learners
(around 50 per cent).
Recent significant changes:
Over the period 2011-2012, 51 per cent of shares
in FutureCOL were sold to National Tertiary
Education Consortium (Ntec). This is a grouping
of four PTEs, including FutureCOL. FutureCOL
continues to operate as a stand-alone entity,
although some marketing and other shared
services are provided to FutureCOL via Ntec. The
integration within the Ntec group has seen a
greater focus on international learners emerging at
FutureCOL.
The remaining 49 per cent of shares are held by
the managing director and facilities manager.1
1
These two people are married and shares are owned jointly.
4
Previous quality assurance
history:
FutureCOL was quality assured by NZQA in 2008
and met requirements at that time.
2. Scope of external evaluation and review
The agreed scope of the external evaluation and review was threefold:
•
Governance, management and strategy
•
All active programmes of delivery
•
International students.
At the time of this report, governance, management and strategy was a mandatory
focus area. International learners and domestic learners are integrated into the
same classroom cohorts; however, it was determined that a special focus on the
experiences of international learners was warranted, given the growth in those
learner numbers. All programmes were included in the evaluation, given that
FutureCOL was uncertain which programmes would be fully operational when the
scope of the evaluation was originally undertaken (in December 2012).
3. Conduct of external evaluation and review
All external evaluation and reviews are conducted in accordance with NZQA’s
published policies and procedures. The methodology used is described fully in the
web document Policy and Guidelines for the Conduct of External Evaluation and
Review available at: http://www.nzqa.govt.nz/providers-partners/registration-andaccreditation/external-evaluation-and-review/policy-and-guidelines-eer/introduction.
The TEO has an opportunity to comment on the accuracy of this report, and any
submissions received are fully considered by NZQA before finalising the report.
The evaluation comprised two evaluators. The team visited the FutureCOL site in
Hastings on 4 and 5 March 2013. The final document requested was submitted
and reviewed on 6 March. During the visit the evaluation team met with:
•
Ntec directors/FutureCOL shareholders and governors
•
FutureCOL managing director (with above and separately)
•
Academic director/head of school, and head of school cookery
•
Tutors in cookery, computing system engineering, Youth Guarantee
programmes
•
Student support staff
•
Learners in various cohort groupings.
5
The evaluation team also contacted via telephone employers and other
stakeholders, including a local polytechnic and former graduates of FutureCOL.
Attempts were also made to contact a relevant industry training organisation but
were unsuccessful at the time of the evaluation.
The evaluators also undertook visual observations of all key learning areas,
including classrooms, a computer laboratory, a kitchen area and restaurant
facilities.
During the course of the evaluation, the team also received and undertook a
succinct analysis of key documentation of FutureCOL. This included quality
management documents, strategic planning documentation, course delivery
information (including 2013 planning), student records, student and staff
evaluations, student assessments and other learning materials. These materials
were in addition to the summary of self-assessment (and related documents)
provided prior to the site visit. These documents contributed to the findings of this
evaluation.
6
Summary of Results
Statement of confidence on educational performance
NZQA is Highly Confident in the educational performance of The College of
Future Learning New Zealand (FutureCOL)
Accumulated evidence from learners, educators, managers and external
stakeholders, as well as empirical data, confirms that FutureCOL is meeting the
most important needs of its learners and stakeholders. This finding is consistent
throughout the various streams of educational delivery, demonstrating sound
educational performance for learners at FutureCOL.
Learners presenting in courses orientated towards second-chance education and
employment will – much more often than not – achieve the qualifications they
require and employment they seek. This is an equally true for second-chance
learners returning to education, domestic learners engaged in higher-level
qualifications such as cookery and computer systems engineering, and for
international learners with their first educational experience in New Zealand.
Qualification success rates are high in all areas (often above 90 per cent). In
addition, learners in the cookery area are also likely to receive international
accreditation, to ensure the knowledge and skills gained are fully transportable,
which is the outcome these learners desire. The evaluation has found this occurs
because FutureCOL has put in place highly effective teaching processes and
strong management processes to underpin educational performance.
Strong learner outcomes are a result of a focus on educational quality throughout
FutureCOL. This evaluation and review is satisfied that the range of data and
information presented, when considered cumulatively, demonstrates consistently
high performance that caters to the needs of learners and stakeholders. Evidential
information includes learner and stakeholder feedback identified through
FutureCOL’s own self-assessment approaches, information from learners and
stakeholders gathered directly through this evaluation, formal educational
performance data, and employment data for FTTO programmes with a vocational
orientation. This evaluation and review has found both the quality of data, and
processes used by FutureCOL to be of high quality. This results in programmes
having the necessary resources required; and importantly, there is a focused and
committed team of educators guiding delivery across all areas of provision.
Ultimately this situation means most students enjoy participating in learning at
FutureCOL, including learners who have been unsuccessful in other learning
settings. Learners are consistently offered a positive opportunity to extend
themselves. At the time of the evaluation and review there were no significant gaps
of weaknesses in educational delivery at FutureCOL.
7
Statement of confidence on capability in self-assessment
NZQA is Confident in the capability in self-assessment of The College of Future
Learning New Zealand (FutureCOL)
FutureCOL has an effective self-assessment approach which ensures that
managers and senior staff are conscious of, and can reflect on, key educational
matters of relevance. Good systems are in place to support this, including various
scheduled reporting and staff meeting mechanisms to review quality. Evidential
notes are kept so that changes and improvements can be tracked. This results in a
process of on-going quality monitoring and improvement. Importantly, selfreflection activities include genuine means to gather and understand the views of
stakeholders, such as employers and learners. This results in improvements in the
match of programmes to the needs of learners and industry.
Overall, these approaches make self-assessment activities purposeful and
effective. It is possible, however, for FutureCOL to extend self-assessment
activities further in other areas of learner achievement, to ensure a greater level of
effectiveness. In particular this would likely involve more robust collation of
achievement data, greater gathering of evidence from community stakeholders to
inform self-assessment, and a greater focus on self-assessment reporting.
8
Findings2
1.1 How well do learners achieve?
The rating for performance in relation to this key evaluation question is Excellent.
The rating for capability in self-assessment for this key evaluation question is
Good.
The majority of learners at FutureCOL successfully complete formal qualifications
and acquire industry-useful information and knowledge. (Note: results information
on domestic learners seeking formal tertiary qualifications is included in the table
below.) Alongside this, learners gain confidence, self-esteem and work-readiness
competencies. Evidence supporting these findings was demonstrated in reporting
on learning outcomes, analysis of student feedback and evaluations, student
interviews, graduate feedback and stakeholder and employer comments.
This is consistent throughout recent years and throughout the various cohorts of
learners FutureCOL works with, which can be summarised as youth, secondchance tertiary students, domestic tertiary learners and international learners.
In the Future-Focused Training Opportunities (FFTO) programmes, data on
FutureCOL’s performance indicates positive contractual outcomes against all three
outcome measures; (completions, unit standards gained, and employment).
Ultimately these programmes are orientated towards finding employment for
learners, and FutureCOL is of the view that employment outcomes are high within
their community context. In Hastings, over 50 per cent of 2012 students had been
employed at the time of this evaluation (March 2013).3 FutureCOL considers this to
be a positive outcome that aligns with its strategic goals in this area. Evidential
information from learner and stakeholder interviews also indicates satisfaction with
this provision. Given the outcomes data, positive endorsement and employment
outcomes, this evaluation is satisfied that the approach, and the results, of
FutureCOL in this area meets educational performance expectations and leads to
positive learner outcomes.
For domestic students in Student-Achievement Component funded programmes,
successful qualification completions across courses range from 70 per cent to 100
per cent for 2012. FutureCOL is of the view that this is a satisfactory and
appropriate outcome. In addition, FutureCOL has been tracking employment and
further education outcomes of domestic learners in these programmes, which also
show the majority of learners will gain direct value for their studies (as shown in the
table below). This evaluation has found, based on the data presented and further
2
The findings in this report are derived using a standard process and are based on a targeted
sample of the organisation’s activities.
3
Wairoa data is not included as there is a rollover of students into a 2013 programme.
9
student and stakeholder comments, that learner achievements are realistic stretchtargets and demonstrate sound educational performance.
In 2012, FutureCOL began enrolling greater numbers of international students.
The qualification completion rate for this group in 2012-2013 is 100 per cent.4 This
is above the 90 per cent target FutureCOL has established for itself. Based on the
positive comments from International students on FutureCOL provision, the
evaluation team is also satisfied that strong learner achievement is being promoted
at FutureCOL for this cohort of learners. FutureCOL, however, while having high
cognisance of learner achievement outcomes and having comprehensive data on
these, could improve its systematic representation of results ‒ turning data into
management information – in order to reflect more fully on learner outcomes.
2012 SAC Provision
Qualification
gained
Employed
Further training
Level 3 Cookery 1
88.89%
44.44%
55.56%
Level 3 Cookery 2
100%
0%
100%
Level 3 Cookery 3
88.89%
11.11%
77.78%
Food and beverage
70%
40%
10%
Systems engineering
100%
25%
25%
1.2 What is the value of the outcomes for key stakeholders, including
learners?
The rating for performance in relation to this key evaluation question is Good.
The rating for capability in self-assessment for this key evaluation question is
Good.
Learning outcomes are highly valued by learners and other external stakeholders,
in that both groupings agreed that the skills and knowledge gained from
FutureCOL’s educational provision met their needs, and were highly applicable to
industry.
Evidence supporting this finding was found in student interviews, stakeholder
interviews and in information FutureCOL has directly gathered from stakeholders
and presented within its self-assessment materials.
In the computing area (i.e. the system engineering qualification), two employers
indicated that they were involved in the design of the programme, and that it met
4
This is based on the programmes offered at the time of the evaluation. Future programmes
in 2013 may have differing outcomes.
10
their needs in that graduates presented with sufficient technical skills for
employment. In addition, a representative of a local polytechnic indicated that
efforts had been made by FutureCOL to ensure recognition of prior learning in a
higher-level degree offered at that institution. The situation was (correctly)
described as beneficial for learners and both institutions.
In the cookery area, FutureCOL visits its key stakeholders to deliver and collect
survey data. The collated information demonstrates high satisfaction from
employers with the learning outcomes and skills demonstrated by FutureCOL
students. FutureCOL also integrates an international qualification within its
programme (City & Guilds accreditations), as this is sought by some stakeholders,
in particular international learners at FutureCOL.
The deliberate way in which FutureCOL has developed stakeholder relationships is
positive. There is, however, scope to extend self-reflection in this area to a greater
range of stakeholders to reflect more fully the learners that participate in
FutureCOL programmes. Examples include strengthening iwi and Māori
community relationships given the focus on Māori learners; increasing stakeholder
engagement in the area of system engineering; and further participation in industry
initiatives in the food and beverage area. Increasing this level of stakeholder
engagement could provide FutureCOL with better information to maintain
programme relevance, greater access to further external support services, and
potentially further linkages to industry and employment for learners.
1.3 How well do programmes and activities match the needs of
learners and other stakeholders?
The rating for performance in relation to this key evaluation question is Excellent.
The rating for capability in self-assessment for this key evaluation question is
Excellent.
FutureCOL matches its educational offer well with the needs of its learners. This
includes regular programme design reviews, needs analysis (of learners and
industry), maintaining relevant and updated curricula and ensuring the appropriate
resourcing of programmes.
Evidence supporting this finding is threefold. First, in relation to learners, pre-entry
skill and competency assessments are undertaken to ensure a match with course
provision. This includes literacy and numeracy testing, interviewing and trial
learning periods. Approaches are customised to fit best with the cohort of learners
(e.g. IELTS5 requirements for English-second language learners). This is good
practice.
5
International English Language Testing System
11
Second, programme content is regularly reviewed to ensure continued relevance.
As examples, the pace of change in the computing sector and the emergence of
new technologies (e.g. tablets) were noted as development areas that FutureCOL
was focused on; the food and beverage course was also being redesigned to fit
new NZQA requirements; and evidence was provided to demonstrate that
conscious educational choices were made to redesign FFTO provision to fit with
contractual changes in 2011 (to ensure a stronger focus on employment).
Third, FutureCOL’s resourcing for its programmes and its facilities are sound.
Teaching staff report good access to all necessary resources, which is also the
view of learners. Both practical dimensions (such as a kitchen area and computer
laboratory) and theoretical learning requirements (classrooms, course books) are fit
for purpose. FutureCOL also runs (on its premises) a restaurant area, allowing for
simulated work experience for cookery and hospitality learners.
Decisions on ensuring the matching of programmes to learners are made
consciously and reflected upon by tutors and heads of departments at regular
fortnightly meetings. There is good evidence that information on student and
stakeholder needs shapes the design of programmes and that self-assessment
practices are strong.
1.4 How effective is the teaching?
The rating for performance in relation to this key evaluation question is Good.
The rating for capability in self-assessment for this key evaluation question is
Good.
FutureCOL’s teaching is delivered with high effectiveness which maximises
learning opportunities for the majority of learners and ensures an inclusive learning
environment.
All programmes delivered by FutureCOL have a clear delivery plan, which sets out
progressive learning stages and key learning outcomes. Learners are aware from
the outset of this plan, and course modules and assessment dates and tasks are
provided in learner course documentation and displayed in classrooms/learning
environments. There is an appropriate balance of classroom learning to selfdirected time, and considered ratios to balance theory and practice in all areas.6
Peer review of teaching was introduced (formally) in 2012, and all junior staff are
also peered with more experienced educators, to support the quality of their
delivery.
Educators all have the necessary unit standard (4098 Use standards to assess
candidate performance) to ensure valid assessments, and key staff have at least a
6
These ratios vary according to programmes of delivery, but are based on the notion of 10
hours of learning per credit, according to a staff submission.
12
Diploma in Adult Education or other teaching qualification. All staff interviewed had
the necessary trade and sector knowledge in their specialised areas. Senior staff
have also been able to visit other institutions to compare delivery approaches.
Students are of the view that teaching staff have strong subject knowledge and are
easy to relate to and are approachable. This makes for a positive and safe learning
environment.
Assessments are a mix of theory and practical exercises, and a reassessment
policy is in place. Samples of student assessment materials were provided, and
based on the evidence presented, assessment processes are considered fair, valid
and transparent.
FutureCOL has an internal moderation system (25 per cent of assessments
moderated per annum). This is good practice.
Since 2010, the majority of unit standards submitted for external moderation with
NZQA have fully met the requirements of the national standard. Minor
modifications were required to the assessment materials for five unit standards
(which have been addressed), and there were repeat issues arising in relation to
one further unit standard (which resulted in an Action Plan in 2011). FutureCOL
advises that there are no external moderation matters arising with the industry
training organisations it works with, and that it also has successfully completed
external moderation processes for City and Guilds.
1.5 How well are learners guided and supported?
The rating for performance in relation to this key evaluation question is Excellent.
The rating for capability in self-assessment for this key evaluation question is
Good.
FutureCOL learners are provided with comprehensive study information and
ongoing assistance to support their learning needs.
For all students, the classroom tutor is the first point of student support; in addition,
FutureCOL has staff focused on both domestic and international student support
services.
Support for international learners includes initial airport collection, provision of
temporary accommodation in Hastings, and a social services orientation (i.e. visits
to banks, libraries, health providers etc). Support and guidance for youth includes a
vehicle to collect and return students, and career planning and social service
components are integrated into programmes. For adults in employment-focused
classes, FutureCOL circulates lists of vacancies, and staff proactively approach
employers to seek out both work placement and employment opportunities. Some
effort is also made to bridge cultural differences, through shared meals, pōwhiri and
other social events (e.g. cricket games). Because of these types of initiatives,
learners, both domestic and international, indicated they felt well supported and
clearly guided in their course of study. All learners were aware of what was required
13
to complete their studies successfully, and what staff members were available to
provide support.
Self-assessment approaches in this area are effective, although as the cohorts of
learners who present at FutureCOL change (for example, through increasing
numbers of international learners), it is reasonable to expect that greater effort will be
required to maintain appropriate types of support. As yet, self-assessment
documentation has not progressed to delineate between the needs of different
clusters of international learners, and/or any issues arising due to the change in the
overall learner demographic at FutureCOL.
1.6 How effective are governance and management in supporting
educational achievement?
The rating for performance in relation to this key evaluation question is Good.
The rating for capability in self-assessment for this key evaluation question is
Good.
FutureCOL management provides effective leadership which creates a sound basis
for high-quality educational delivery and positive learner outcomes.
The leadership team (comprising three senior staff) ensures resources are
appropriately allocated, that a focus on learners is maintained throughout the
provision, and that there are quality management systems in place to guide staff
and students. Overall, quality management is active and is reflected upon regularly
(fortnightly within teams, monthly at a management level, annually at a governance
level). The new directors of FutureCOL have made a conscious decision not to
change or become involved in the hands-on management of FutureCOL, and
advise that they are more than satisfied with existing management arrangements.
Given existing management practices are contributing towards sound learner
outcomes this governance approach is an appropriate at present.
There is good evidence that FutureCOL responds to change, and it is presently
commencing a process of sharing resources and marketing via Ntec. From an
educational perspective, the Ntec relationship allows FutureCOL to access a
greater amount of shared learning resources, and that is likely to be of increased
benefit going forward.
The managing director reports that there are no legal or ethical issues arising, and
no staff or student complaints (although a formal complaints process is in place).
Staff at FutureCOL report that they feel valued. Job descriptions and roles are
clear, and an appraisal system is in place, although it is being redeveloped to
ensure staff can self-reflect further on their own development and educational
contribution.
Given the overall high educational performance of FutureCOL, an opportunity
exists for the organisation to increase its focus on internal quality assurance
14
matters. This includes greater clarity of learner achievement data, and greater
levels of stakeholder engagement, which will be of use to further develop reflective
practices and educational provision at FutureCOL.
15
Focus Areas
This section reports significant findings in each focus area, not already covered in
Part 1.
2.1 Focus area: Governance, management and strategy
The rating in this focus area for educational performance is Good.
The rating for capability in self-assessment for this focus area is Good.
2.2 Focus area: All programmes
The rating in this focus area for educational performance is Good.
The rating for capability in self-assessment for this focus area is Good.
2.3 Focus area: International students
The rating in this focus area for educational performance is Good.
The rating for capability in self-assessment for this focus area is Good.
16
Recommendations
The following recommendations are made for FutureCOL:
1. FutureCOL should work to improve its systematic representation of learner
results in order to reflect more fully on learner outcomes. This should
include explicitly delineating between the needs of different clusters of
international learners and/or any issues arising due to the change in the
overall learner demographic at FutureCOL.
2. FutureCOL should improve its external moderation processes and
responsiveness in instances where external moderation identifies issues for
remedial action.
FutureCOL should extend self-reflection activities to a greater range of
stakeholders to reflect more fully the learners that participate in FutureCOL
programmes. Examples include strengthening iwi and Māori community
relationships given the focus on Māori learners; increasing stakeholder
engagement in the area of system engineering; and further participation in industry
initiatives in the food and beverage area.
17
Appendix
Regulatory basis for external evaluation and review
Self-assessment and external evaluation and review are requirements of programme
approval and accreditation (under sections 249 and 250 of the Education Act 1989) for all
TEOs that are entitled to apply. The requirements are set through the Criteria for Approval
and Accreditation of Programmes established by NZQA under section 253(1)(d) and (e) of
the Act and published in the Gazette of 28 July 2011 at page 3207. These policies and
criteria are deemed, by section 44 of the Education Amendment Act 2011, to be rules made
under the new section 253.
In addition, for registered private training establishments, the criteria and policies for their
registration require self-assessment and external evaluation and review at an organisational
level in addition to the individual programmes they own or provide. These criteria and
policies are also deemed, by section 44 of the Education Amendment Act 2011, to be rules
made under section 253. Section 233B(1) of the Act requires registered PTEs to comply
with these rules.
NZQA is responsible for ensuring non-university TEOs continue to comply with the rules
after the initial granting of approval and accreditation of programmes and/or registration.
The New Zealand Vice-Chancellors’ Committee (NZVCC) has statutory responsibility for
compliance by universities.
This report reflects the findings and conclusions of the external evaluation and review
process, conducted according to the EER process approved by the NZQA Board.
The report identifies strengths and areas for improvement in terms of the organisation’s
educational performance and capability in self-assessment.
External evaluation and review reports are one contributing piece of information in
determining future funding decisions where the organisation is a funded TEO subject to an
investment plan agreed with the Tertiary Education Commission.
External evaluation and review reports are public information and are available from the
NZQA website (www.nzqa.govt.nz).
Information relevant to the external evaluation and review process, including the publication
Policy and Guidelines for the Conduct of External Evaluation and Review, is available at:
http://www.nzqa.govt.nz/providers-partners/registration-and-accreditation/externalevaluation-and-review/policy-and-guidelines-eer/introduction/
NZQA
Ph 0800 697 296
E qaadmin@nzqa.govt.nz
www.nzqa.govt.nz
18