Report of External
Evaluation and Review
Social Services Industry Training
Organisation – Te Kaiawhina Ahumahi
Inc
Confident in ITO performance
Confident in capability in self-assessment
Date of report: 5 October 2010
Contents
Purpose of this Report................................................................... 3
Introduction ................................................................................... 3
1. ITO in context ...........................................................................................3
2. Scope of external evaluation and review..................................................5
3. Conduct of external evaluation and review...............................................5
Summary of Results ...................................................................... 6
Findings ........................................................................................ 9
Recommendations ...................................................................... 18
Further actions ............................................................................ 18
MoE Number:
8153
NZQA Reference:
C02336
Date of EER visit:
20-23, 29 July 2010
2
Purpose of this Report
The purpose of this external evaluation and review report is to provide a public statement
about the industry training organisation’s (ITO) performance and capability in selfassessment. It forms part of the accountability process required by Government to inform
investors, the public, trainees, communities, employers, and other interested parties. It is
also intended to be used by the ITO itself for quality improvement purposes.
Introduction
1. ITO in context
Location:
Wellington en.
Type:
Industry Training Organisation
First recognised:
2005
Number of trainees:
Domestic: 900 (including 700 standard trainee measures
known as STMs)
International: Nil
Number of staff:
Scope of active accreditation:
20 full-time equivalents
•
National Certificate in Employment Support
(Level 4
•
National Diploma in Employment Support
(Level 6)
•
National Certificate in Iwi/Māori Social
Services (Level 4)
•
National Diploma in Iwi/Māori Social Services
(Level 6)
•
National Certificate in Social Services (Level 4)
•
National Certificate in Social Service Work
with Abuse, Neglect and Violence (Level 4)
•
National Diploma in Social Service Work with
Abuse, Neglect and Violence (Level 6)
•
National Diploma in Social Work (Level 6)
•
National Certificate in Tamariki Ora-Well Child
Services with strands in Mainstream, Māori and
3
Pacific (Level 4)
•
National Certificate in Whānau/Family and
Foster Care (Level 4)
•
National Diploma in Whānau/Family and Foster
Care (Level 6)
•
National Certificate in Youth Work (Youth
Leadership) (Level 3)
•
National Certificate in Youth Work (Level 4)
•
National Diploma in Youth Work (Level 6)
•
National Diploma in Community Work (Level
6)
•
National Diploma in Counselling (Level 6)
Sites:
No additional sites
Distinctive characteristics:
Social Services Industry Training Organisation – Te
Kaiawhina Ahumahi Inc (SSITO) has undergone major
changes in governance, management, and strategy since
2007. At the time of the EER it was focused on
developing industry-led national degrees in social work
and youth work.
Recent significant changes:
In 2008, the social services industry completed a review
of the governance of the industry training organisation
(ITO) by voting in a new constitution focused on
outcomes for all learners. The new board structure will
be implemented in 2011. In the interim, a two-person
board and chair govern the ITO. In 2009, a second
internal restructure since 2007 took place to ensure that
the ITO has the people with the knowledge and skills it
needs to do its work successfully.
Previous quality assurance
history:
At the last external quality assurance visit by NZQA, an
audit in 2005, SSITO did not meet five of 13
requirements, three related to Workplace Assessors and
two to National External Moderation. A follow-up visit
in September 2006 verified that all requirements had
subsequently been met.
National external moderation results show that SSITO
is currently meeting its NZQA external moderation
requirements.
Other:
Since 2007 the SSITO has consulted its stakeholders
widely and restored confidence in the ITO by
4
addressing past issues, improving delivery of business
as usual, and planning for the next decade to 2020.
2. Scope of external evaluation and review
The scope of the external evaluation and review of Social Services Industry Training
Organisation - Te Kaiawhina Ahumahi Inc included the following areas.
•
Governance, management, and strategy.
This focus area was chosen to capture the significant changes, both structural and
methodological, that have been made over the past three years.
•
Social work training including the National Diploma in Social Service Work (Level
6)
This training takes place mainly on the job. This area has the most trainees.
•
Youth work training levels 3-6 including the National Diploma in Youth Work
(Level 6)
This training takes place mainly off the job. This area has the second most trainees.
3. Conduct of external evaluation and review
All external evaluation and reviews are conducted in accordance with NZQA’s published
policies and procedures. The methodology used is described fully in the document Policy
and Guidelines for the Conduct of External Evaluation and Review available at:
http://www.nzqa.govt.nz/for-providers/keydocs/index.html) (Refer to ITO supplement)
The external evaluation and review team comprised one lead evaluator, a second lead
evaluator, and an external evaluator. The team spent three days and a one-hour meeting on
site in Wellington with a one-hour meeting after the visit. The team spoke to the chief
executive, collaboration and engagement manager, finance manager, literacy and numeracy
project manager, national moderator, two regional workforce development coordinators,
assessors, employers, trainees, and the social workers registration board.
Social Services Industry Training Organisation - Te Kaiawhina Ahumahi Inc has had an
opportunity to comment on the accuracy of this report, and submissions received have been
fully considered by NZQA before finalising the report. The NZQA evaluation team knew
that SSITO had recently undergone an independent audit commissioned by TEC but it did
not know the outcomes at the time of the external evaluation.
5
Summary of Results
Statement of confidence on ITO performance
NZQA is Confident in the performance of Social Services Industry Training
Organisation – Te Kaiawhina Ahumahi Inc. Key reasons for this include the following.
SSITO has a very clear purpose and direction supported by its strategic and business plans
which relate to meeting the identified needs of its industry and trainees. SSITO bases what
it does on clear principles and values. SSITO has an interim board and a new governance
model which are expected to support the ITO in continuing to improve outcomes for all
learners and communities. The ITO has made significant progress in gaining the
confidence of its sector through engagement and partnership that support the development
of the capability of social services organisations.
SSITO has restructured its staff so that it has the knowledge, skills, and experience to
deliver initiatives which include improving qualification completion rates, engaging
employers, developing literacy and numeracy, reviewing the capacity of the assessor pool,
meeting the needs of all sector workers from all cultures including Māori and Pasifika, and
developing the workforce. Additional new appointments were pending at the time of the
external evaluation.
SSITO is establishing a professional development framework focused on ensuring that
national standards and qualifications are relevant to industry, are widely used by social
services staff and employers, and provide educational and employment pathways, from
support worker roles to advanced specialist practice and management roles. Based on
widespread consultation, SSITO has identified the need to meet the growing
professionalism of its industry. It is developing industry-led workplace-based national
degrees in social work and youth training work and has partnered with an institute of
technology (ITP) to help in this. SSITO has developed the unit standards for the degrees
and submitted them to NZQA for registration.
SSITO has improved its qualification completions from 9 per cent in 2007 to 23 per cent as
at July 2010. This rate exceeds the Tertiary Education Commission (TEC) key performance
indicator for SSITO’s 2007-2010 investment plan. This significant progress is encouraging
but comes off a low baseline, and SSITO governance, management, and staff are aware that
there is a need to further increase the number and proportion of completions.
SSITO has developed strategies and begun implementing practices which are enhancing the
quality and effectiveness of the training. These include working more closely with
employers by involving them with their employees in training, supporting and recognising
in-house training, and organising individual trainees and organisations into regional groups
where they learn together. The recent development of more robust identification of
industry, employer, and trainee needs prior to signing on any learner helps ensure that those
needs are met.
Employers, assessors, and trainees spoken to by the evaluation team confirmed that SSITO
understands the current and future needs of its sector. Its work in this regard includes the
6
development of a 2020 vision for the social services workforce in consultation with the
sector.
New assessors and the national moderator independently confirmed that assessment and
moderation support are provided to new assessors who are registered when they are
sufficiently experienced to take part in the collegial peer moderation process which is both
paper-based and face to face. Assessors reported that the literacy and numeracy (LN)
strategies, including resources and assessments with literacy and numeracy embedded in
them, effectively address LN issues and improve skills.
Statement of confidence on capability in self-assessment
NZQA is Confident in the capability in self-assessment of Social Services Industry
Training Organisation – Te Kaiawhina Ahumahi Inc. Key reasons for this include the
following.
SSITO has consulted widely and this has contributed to its self-assessment and enabled it to
identify issues, develop strategies and practices which address them, and make
improvements which have helped regain the support of the sector. The chair of the SSITO
board, the chief executive, and management have been able to articulate the ITO’s
leadership role. SSITO has recruited staff with the appropriate knowledge, skills, and
experience to carry out its statutory responsibilities and deliver a quality service consistent
with its leadership role.
SSITO has recognised the complex processes associated with identifying and meeting the
training needs of a diverse and strategically important industry and aligning them with the
needs of the different stakeholders including the government’s Tertiary Education Strategy
with its emphasis on the achievement of national qualifications. SSITO has simplified its
processes and systems to more robustly identify the needs of all its key stakeholders,
including trainees. It has moved from traditional workplace learning based on individual
trainees, to a relationship which meets the employer’s needs and gains the support of the
employer, uses learner groups effectively, and is supervised by workforce development
coordinators. The increase in completions is considered to be the result of the improved
relationship between the employer, employee and the ITO and the support provided by
regional workforce development coordinators and assessors; it validates the ITO’s policy to
focus on supporting whole enterprises rather than individual trainees in isolation from their
work environment. Examples of the increased numbers of completions and improved
quality and effectiveness of the training are expected to become increasingly evident.
SSITO’s assessment and moderation practices have been researched and improved so that
they are effective in ensuring assessments for SSITO standards and qualifications are fit for
purpose, valid, reliable, fair, transparent, consistent, and meet the national standard. The
national assessment and moderation system also supports SSITO’s in-house recognition of
qualifications in conjunction with employers. SSITO is establishing a professional
development framework for the sector which is based on opportunities for trainees to
progress from qualifications at one level to a higher level. It has integrated pathways and
maintained or postponed review of existing standards and qualifications while prioritising
7
such development and in order to meet a need for industry-led national degrees as perceived
by the sector and the ITO.
SSITO is not only focused on meeting current needs. It has consulted with the social
services sector to develop a workforce development strategy for the next decade. SSITO
has drafted a strategy, Skills for Wellbeing, 2020. Realising that a coordinated approach
requires ongoing broad sector support, SSITO had distributed the draft for wide sector
consultation at the time of the external evaluation. The recent completion and handover of
a project that has comprehensively identified all SSITO’s stakeholders in a very diverse
sector should enable the ITO to further identify, understand and meet the needs of its sector
on an ongoing basis.
SSITO reports the total number of qualification completions related to the number of
trainees and the number of credits achieved, which have improved from 2007 to 2010 and
have met or exceeded TEC’s requirements for SSITO. SSITO and the evaluation team
reviewed quantitative and qualitative information, which showed the basis on which SSITO
is making successful improvements.
While SSITO records and reports total qualification and credit completions, it also breaks
down the information so that it knows, for example, how the completions and length of time
taken for completions in social work compare with youth work. This has enabled it to
identify that the shorter, lower-level youth work qualifications are completed close to the
expected duration, whereas the social worker, higher-level qualifications are not. SSITO
has considered the reason for this difference and is making further specific as well as
generic improvements referred to above.
Self-assessment has led SSITO to identify the need for better recording and monitoring of
trainee data and better analysis. The upgrading of SSITO’s database and reporting system
will enable it to capture and record information more effectively. Both formal and informal
anecdotal feedback is important, valuable, and valid. The new database will capture
quantitative and qualitative information from employers and trainees on the value of the
outcomes of SSITO training, how well trainees are achieving, and how effective the
training is. This will enable SSITO to make further ongoing improvements.
ITO response
Social Services Industry Training Organisation – Te Kaiawhina Ahumahi has confirmed the
factual accuracy of this report.
8
Findings1
1.1 How well does the ITO understand and meet the needs of industry,
learners and government?
The rating for performance in relation to this key evaluation question is Good.
The rating for capability in self-assessment for this key evaluation question is Good.
SSITO understands its sector very well. Since 2007 it has consulted the sector widely and
regained and increased its support. As part of the process, in 2008 it carried out a survey,
which attracted a limited number of responses but some were very detailed and drew
together the conclusions of a number of peak bodies.
SSITO has improved its working relationships and moved from relative isolation to
collaboration on the basis that partnerships and relationships work best. SSITO has forged
sector partnerships which include WelTec, the New Zealand Family and Foster Care
Federation (NZFFCF), Association for Supported Employment in New Zealand (ASENZ),
National Youth Workers Network Aotearoa (NYWNA), Te Kahui Atawhai O Te Motu –
National Collective of Iwi Māori Social Services; and relationships with Aotearoa New
Zealand Association of Social Workers (ANZASW), Social Workers Registration Board
(SWRB), and Aotearoa New Zealand Association of Social Work Educators (ANZASWE).
While SSITO has increased its industry support, it is still developing its knowledge and
understanding of the sector and is not working with all the sector yet. This is an intentional
strategy related to its capacity given its still relatively low number of trainees, standard
trainee measures (STMs), and completions, and in order to ensure its investment in
relationships with stakeholders is carefully targeted and worthwhile. A recently completed
project which has robustly identified all SSITO’s stakeholders will help facilitate SSITO’s
increasing involvement with the whole sector it represents.
From its consultation, SSITO has identified current and future industry needs which it has
begun to address. It is establishing a professional development framework which the
SSITO and the whole sector can use to progress from completion of one qualification to
another at a higher level, and contribute to further developments. SSITO is also focused on
identifying and meeting future needs. It has consulted with the social services sector, peak
bodies, non-governmental organisations (NGOs), and government agencies to help
determine a workforce development strategy for the next decade. From this consultation
SSITO has drafted a strategy, Skills for Wellbeing, 2020. Consistent with its realisation that
a coordinated approach requires broad sector support, the draft had been sent out for wide
sector consultation at the time of the evaluation visit.
SSITO understands the value of staircasing qualifications and is motivated by wanting to
provide an incentive to those who have currently reached diploma level so that they can go
1
The findings in this report are derived using a standard process and are based on a targeted sample of
the organisation’s activities.
9
on and achieve an industry-led workplace-based national degree. It sees this as an
important need to be met if the growing professionalism of the sector is to reach its
potential and benefit individuals, family and whānau, and the communities of New Zealand.
There are currently 13,500 social workers in New Zealand and 1500 of them have a degree.
SSITO is well placed to develop unit standards and qualifications from levels 3-6.
However, it has realised that it needs to have broad support and to work with higher-level
tertiary organisations if it is to succeed in applying for the registration of an industry-led
national degree in social work and another in youth work. It has formed a partnership with
an institute of technology (ITP) and is consulting the Social Workers Registration Board
and universities. The ITP and SSITO have developed and submitted high-level unit
standards for social work and youth work which will be the components of the national
degrees. The unit standards and degrees are subject to approval by NZQA. Their
acceptance for registration purposes are subject to approval by the SWRB.
The evaluation team is satisfied that SSITO is aware of the risks related to focusing on the
degree development when a growing number of trainees are starting at the lower levels,
especially in youth work. SSITO intends to build its progression from one qualification to
another from the top down rather than from the bottom up in its bid to provide a pathway to
a national degree that has fewer steps or barriers for trainees.
SSITO has enhanced the delivery of the lower-level qualifications through its regional
workforce development coordinators working more closely with employers and organising
individual learners into groups. There are already examples of how these kinds of practices
are improving completions in youth work and the quality and effectiveness of the training.
In addition, youth worker training is acknowledging and developing the knowledge, skills,
and experience of workers who previously received limited training and qualifications.
SSITO intends that youth workers will have the same kind of learning pathways and
qualifications as social services workers progressing to a national degree.
SSITO reports the total number of completions related to the number of trainees and the
number of credits achieved, which meet TEC’s requirements for SSITO and shows
improvement from 2007 to 2010. SSITO and the external evaluation team have gathered
rich stories and information about how SSITO is making successful improvements. For
example, the ITO identified in contracted training a mismatch between the training and
assessment, stopped the programme, ensured both were in alignment, and then resumed.
This led the ITO to plan a meeting of all those involved in similar training, especially
trainers and assessors, to prevent any recurrence of such a situation.
Both formal and informal and anecdotal feedback and data are important, valuable, and
valid. SSITO records and reports total qualification and credit completions but also breaks
down the information so that it knows, for example, how the completions and duration for
completions in social work compares with youth work. This has enabled it to identify that
the shorter, lower-level youth qualifications are completed within a timeframe very close to
what is nominally expected, whereas the social services higher-level qualifications are not.
SSITO has considered the reasons why and is making further specific improvements as well
as the generic improvements referred to above.
10
SSITO has information about individual trainees’ current progress, but it is not able to
readily access such information over all, other than manually, beyond the immediate
programme or qualification the trainee is undertaking, which limits its ability to track and
monitor individual trainee progress easily. It has identified that upgrading its database will
enable it to capture, record, and process information more effectively, including formal
evaluation data from employers and trainees on the value of the SSITO training to
employers and trainees, how well trainees are achieving, and how effective their training is.
This will enable SSITO to make further ongoing improvements and evaluate its own current
performance more easily and further contribute to how well it understands and is meeting
the current as well as future needs of its industry, trainees, and government.
1.2 What is the value of the outcomes for employers and their
trainees?
The rating for performance in relation to this key evaluation question is Good.
The rating for capability in self-assessment for this key evaluation question is Adequate.
The value of the outcomes for employers and their trainees is that the trainees develop
knowledge, understanding, and skills which are directly relevant to their employment. The
training not only benefits the individual trainees but also benefits their employers, the
quality of the services they deliver, and their clients.
SSITO has identified that its sector and responsibilities continue to include a significant
number of Māori trainees. It has also identified a need to better serve Pasifika employees,
who may be gaining unit standards or qualifications for the first time. Their training and
achievements benefit not only their employment but also their community and culture,
encouraging others to pursue education and training. TEC-funded embedded literacy and
numeracy initiatives include resources, strategies, and support that benefit the individual
trainees’ ability to learn and achieve assessment success, as well as the organisation the
trainee works for.
The setting up of learning groups by SSITO workforce development coordinators in regions
such as South Auckland and Taranaki appears to be improving the achievement of
outcomes significantly. Organisations and trainees learning in groups of five to ten in a
small region helps to provide a consistent ethical voice and theory underpinning the benefits
of their employment and their new learning. A workforce coordinator shared with the
evaluation team how one trainee moved from hostility to the training, to seeing and
understanding it, to actively developing study groups.
Available data shows that social workers are further along their learning pathway and
qualifications staircase than youth workers. This is partly historical, and youth workers are
now gaining recognition of their existing knowledge, skills, and experience, most
commonly at levels 3 and 4. Both sector groups have the opportunity to progress their
learning and qualifications up to level 6 and in the near future towards the possibility of
industry-led national degrees referred to in section 1.1 of this report. The provision of a
pathway and staircase is important to the individual trainees and the industry because it not
11
only improves individual and organisational performance but also supports a career
structure which can encourage and motivate people in their work.
The information that SSITO has about the value of specific training is currently positive,
according to feedback from employers, assessors, and trainees. The evaluation team
confirmed this through interviews. SSITO’s last comprehensive survey to capture such
information was carried out in 2008. The robust identification of stakeholders by a
contractor (whose work has just completed) and the development of a stakeholder database
should help SSITO capture both generic and specific, informal and formal feedback, as part
of the upgrading of the database system scheduled for the end of 2010. This should enable
SSITO to better capture quality information, process and analyse it, reflect on what it means,
and make any changes it identifies to improve the value of the outcomes. Currently SSITO
has no destinational information about where training graduates go within their own
organisation or to other organisations but it intends to track and monitor such information in
the future.
1.3 How well do trainees achieve?
The rating for performance in relation to this key evaluation question is Good.
The rating for capability in self-assessment for this key evaluation question is Good.
SSITO has improved its qualification completions from 9 per cent in 2007 to 23 per cent as
at July 2010. This exceeds the Tertiary Education Commission (TEC) key performance
indicator for SSITO’s 2007-2010 investment plan. This significant progress is encouraging
and lifts SSITO’s ranking compared with other ITOs but comes off a low baseline, and
SSITO governance, management, and staff are aware that there is need to further increase
the number of completions. SSITO has developed strategies, recruited appropriate staff
such as workforce development coordinators, and begun implementing practices that are
already achieving improved completions and enhancement of the effectiveness of the
training. These practices include working more closely with employers, supporting and
recognising in-house training, and organising individual learners into groups, as referred to
earlier in this report.
SSITO reports internally monthly and quarterly, at the same high level that it reports to
TEC in terms of qualification and credit completions. SSITO quarterly meetings with
contracted training providers focus on active and inactive trainees and completions. How
long trainees take to complete qualifications is reported as required by TEC. However,
SSITO has broken down the information so that it knows, for example, how the
completions and the timeframe for completions in social work compare with youth work
training. This has enabled the ITO to identify the different completion timeframes for the
shorter, lower-level youth qualifications and the social services higher-level qualifications,
as referred to above. SSITO is making specific improvements as well as generic ones
referred to above.
Workforce development coordinators and assessors out in the field monitor progress, while
the collaboration and engagement manager has regular meetings with contracted providers
of arranged training. Feedback from SSITO and employers and assessors interviewed by
12
the evaluation team included impressive examples of high trainee retentions, high
qualification completions, and high credit completions, for example 33 out of 36 foster care
trainees graduating. Such individual cases show that there are clearly pockets of high
achievement in the SSITO training.
The upgrading of SSITO’s database will enable it to more readily and effectively capture
and record information which it can use to determine actual trainee achievement, including
the progress or value-added component, as business as usual. This will include information
from large agencies, employers, and contracted providers and reflect the value of SSITO
training outcomes to employers and trainees, how well trainees are achieving, and how
effective their training is, as referred to in section 1.1. This will in turn enable SSITO to
make further ongoing improvements.
1.4 How effective is the training arranged by the ITO?
The rating for performance in relation to this key evaluation question is Good.
The rating for capability in self-assessment for this key evaluation question is Good.
SSITO has rolled out a new initiative, a training model for mainly on-job training in social
services work which is based on involvement of the employer from the start. This helps
ensure the relevance of the training of the social services worker to the particular service
provider organisation they work for and helps gain the ongoing support of the employer.
The needs of the employer are matched with those of the trainee, from the initial needs
assessment to starting the programme. This process may take some time, a matter of a few
weeks or even months, to ensure the suitability of the training for the employing
organisation and the individual trainee. The two workforce development coordinators have
a very important role in this process and three more such positions are expected to be filled
shortly.
Once the trainee is accepted for training, their progress is monitored by their employer and
SSITO noting their dates of completions, through information provided by the assessor, and
addressing any specific or generic issues that arise. The collaboration and engagement
manager or workforce development coordinator talks to the employer and/or trainee and/or
assessor on a regular basis throughout the process until the trainee completes their unit
standards and gains the credits for their qualification. The trainee is then advised of the
opportunities available to go on to further study, standards, and qualifications. The
employer is consulted on the outcomes achieved and their value to the employing
organisation. SSITO intends to monitor and track employees’ career progress within their
own organisation or any other employer they move to referred to in section 1.2 of this
report.
Workforce development coordinators arrange training on behalf of the ITO and set up
learning groups as referred to in more detail in section 1.2 of this report. This initiative is
ensuring effective training as well as improved completions through shared learning,
mutual encouragement and support, mentoring and motivation.
13
SSITO contracts organisations that have proven performance to deliver off-job training
mainly but not exclusively for youth workers. The contracted organisation typically subcontracts a training provider to actually deliver the training. SSITO meets regularly with
the contracted organisation in order to assure the quality of the arrangements and the
training and the achievement of the outcomes. As mentioned in section 1.1, SSITO recently
identified a serious mismatch between training and assessment and took action to remedy
the defect. This led the ITO to make changes in contractual arrangements including a
meeting of all those involved in of the training, especially trainers and assessors, before
training starts to prevent any recurrence of such a situation.
SSITO’s feedback from employers, contracted organisations, assessors, and workforce
development coordinators confirms that the training arranged by the ITO is effective.
Increased completions also confirm this. SSITO intends to develop its evaluations of
trainee satisfaction with the training and its effectiveness and quality, including the
arrangements related to it in the near future when it upgrades its data system. This will
enable SSITO to capture, process and analyse, reflect, and make any ongoing generic and
specific improvements it identifies based on the systematic electronic capture of all data,
including oral, which is currently held on a manual system. This will supplement the
information from its various sources which also include moderation.
1.5 What is the quality of the assessment being undertaken towards
industry standards and qualifications?
The rating for performance in relation to this key evaluation question is Good.
The rating for capability in self-assessment for this key evaluation question is Good.
SSITO reviewed its assessment and moderation system in 2008 and now runs an effective
system of moderation for accredited providers and SSITO’s own registered assessors based
on quality, effectiveness, and good practice. Moderation is based on meetings and postal
moderation. A six-monthly moderation meeting is held for accredited providers.
The quality of the assessment towards industry standards and qualifications is based on
assessors completing assessor training and standards, good assessment practices, ongoing
mentoring and support, assessor feedback, relevant workbooks and assessment materials
which contain embedded literacy and numeracy, and robust moderation which in turn
contributes to ongoing improvement of all of these aspects.
SSITO had 60 contracted assessors at the time of the review. Not all assessors are used and
the ITO intends to review this situation. Some assessors the evaluation team spoke to said
they would appreciate more communication from the ITO about ongoing organisational
changes.
A new assessor and the national moderator independently confirmed that assessment and
moderation support was provided to new assessors who are registered only when they are
trained, qualified, sufficiently successfully experienced, and ready to engage in the national
assessment and moderation process, which is both face to face and paper-based. To reach
this point of approved status, the new assessors are mentored by two experienced assessors,
14
one of whom has subject-specific expertise. This helps ensure consistency in the
assessment and moderation practices that apply to unit standards and qualifications. The
new assessor also confirmed that the literacy and numeracy advice they were given by the
literacy and numeracy project manager on strategies worked very successfully the first time
they were used.
SSITO’s feedback and the evaluation team’s interviews with employers, assessors, the
national moderator, and the collaboration and engagement manager confirmed the quality of
assessment for SSITO standards and qualifications. SSITO’s assessment and moderation
practices have been researched and improved and are effective in ensuring assessments for
SSITO standards and qualifications are fit for purpose, valid, reliable, fair, transparent,
consistent, and meet the national standard.
SSITO has maintained or postponed review of some existing standards and qualifications
and prioritised the development and submission of new unit standards for a new industryled work based degree in conjunction with an ITP which will meet a perceived need by the
sector and the ITO. The evaluation team is confident that if and when these standards and
qualifications are approved, SSITO will apply its current and effective assessment and
moderation practices and systems to the new standards and national degrees. In the
meantime, robust direct feedback on assessments by trainees in particular would
complement the information SSITO receives directly and indirectly from its assessors,
those who are responsible for and deliver the training, and the workforce development
coordinators.
1.6 How well does governance and management support trainee
achievement?
The rating for performance in relation to this key evaluation question is Good.
The rating for capability in self-assessment for this key evaluation question is Good.
Three values and principles guide what SSITO does and how. These are Kōtahitanga
(connecting people), Manākitanga (honouring others, honouring ourselves), and Kaitianga
(upholding responsibilities). Kōtahitanga is about unity, acting in a unifying way and
building and promoting unity; Manākitanga is about sustainability, acting in a way that
ensures the sustainability of the organisation and the professions, industries, and
communities SSITO works with; and Kaitianga is about responsibility, taking responsibility
for actions and acting in a way that promotes responsible actions by the team, the
organisation, the industry, and the communities they work with.
SSITO has a very clear purpose and direction supported by its strategic plan and business
plans which relate to meeting the identified needs of the sector and trainees. First, SSITO
has made significant progress in gaining the confidence of its sector through engagement
and partnership, a consultative and collaborative approach to support the capability
development of social services organisations, the involvement of employers with employees
in industry training, and the development of a 2020 vision for the social services workforce.
Secondly, it is continuing to establish a professional development framework focused on
ensuring that national standards and qualifications are relevant to industry, are widely used
15
by social services staff and employers, include educational and employment pathways, and
lead from support worker roles through professional roles to advanced specialist practice
and management roles. Thirdly, it has begun to increase completion rates through
developing completion strategies which include embedded literacy and numeracy. Fourthly,
it has made progress in integrating biculturalism into its services to ensure both Māori and
non-Māori communities are supported. Fifthly, it has ensured the quality of its organisation
through basing what it does on its principles and values and developing a forward looking
strategic plan for the next ten years related to its 2020 vision.
The board is clear about the difference between governance and management. The chair is
very supportive of the chief executive and they meet regularly between monthly reports,
board meetings, and quarterly reports. The board is focused on increasing completions
more quickly than currently and the responsibility and accountability the chief executive,
management, and staff have for this. The board endorses and supports the changes SSITO
has made and has confidence in the changes. It supports the development of the industryled national degrees and the top-down development of the related qualifications pathways
they would offer without compromising the integrity of the lower-level qualifications and
their quality and the trainees who gain them.
While SSITO has recently successfully identified all its stakeholders, it is very aware that it
does not want to overstretch its current capacity. SSITO has restructured its staff twice
since 2007 in order to get the staff it needs to deliver its initiatives including improving
qualification completion rates, engaging employers, developing literacy and numeracy,
fostering workforce development, reviewing the capacity of the assessor pool, and meeting
the needs of all sector workers from all cultures including Māori and Pasifika.
SSITO has rich data which it uses to meet TEC and other agency reporting requirements.
SSITO is committed to developing the way it captures the valuable qualitative information
as well as quantitative data it already obtains from interactions with contracted
organisations, employers, workforce development coordinators, assessors, moderators, and
trainees. SSITO has made a significant investment in upgrading its electronic database by
the end of 2010 which will make this information capture much easier.
16
Focus Areas
This section reports significant findings in each focus area, not already covered in Part 1.
2.1 Focus area: Governance, management, and strategy
The rating in this focus area for ITO performance is Good.
The rating for capability in self-assessment for this focus area is Good.
2.2 Focus area: Social work training including the National Diploma in
Social Work (Level 6)
The rating in this focus area for ITO performance is Good.
The rating for capability in self-assessment for this focus area is Good.
2.3 Focus area: Youth work training levels 3-6 including the National
Diploma in Youth Work (Level 6)
The rating in this focus area for ITO performance is Good.
The rating for capability in self-assessment for this focus area is Good.
17
Recommendations
There are no recommendations arising from the external evaluation and review other than
those implied within the report.
Further actions
The next external evaluation and review will take place in accordance with the Tertiary
Education Commission’s re-recognition cycle, which is likely to occur within five years of
the date of this report.
18
Appendix
Regulatory basis for external evaluation and review
In 2009 NZQA introduced the evaluative approach to quality assurance in the tertiary
education sector, consisting of self-assessment and external evaluation and review. This is
applied in the quality assurance of course approvals and accreditations under sections 258
and 259 of the Education Act 1989, as well as for PTE registration under Part 18 of that
Act.
The NZQA Board also published policies and guidelines for the conduct of external
evaluation and review of ITOs on 27 May 2010. NZQA relies on ITOs to build the selfassessment and external evaluation and review model into their infrastructure. ITOs will
be evaluated on their infrastructure and use of the model in the following ways:
•
Advising Ministerial recognition of an ITO under the Industry Training Act, 1992
•
Accepting arrangements for monitoring and assessing of industry training under
section 10 of the Industry Training Act, 1992
•
Registering standards on the Directory of Assessment Standards
•
Awarding consents for ITOs (or their assessors) to assess against standards on the
Directory of Assessment Standards under section 253(1)(c) of the Education Act,
1989
•
Applying course approval criteria where ITOs are course owners.
External evaluation and review is also used by NZQA as a monitoring and evaluation tool,
with the outcomes of these processes informing Tertiary Education Commission decisions
relating to re-recognition of an ITO. In addition, external evaluation and review reports
are one contributing piece of information in determining future funding decisions in
relation to an investment plan agreed between an ITO and the Tertiary Education
Commission. The understandings and expectations for the implementation of ITO quality
assurance are set out in a protocol to the Memorandum of Understanding between NZQA
and the Tertiary Education Commission.
This report reflects the findings and conclusions of the external evaluation and review
process, conducted according to the policies and criteria approved by the NZQA Board.
The report identifies strengths and areas for improvement in terms of the ITO’s
performance and capability in self-assessment.
External evaluation and review reports are public information and are available from the
NZQA website (www.nzqa.govt.nz).
Information relevant to the external evaluation and review process is summarised in the
publication Policy and Guidelines for the Conduct of External Evaluation and Review. Two
other documents explain how the process is applied to ITOs: EER Policy and Guidelines –
ITO Supplement and Evaluation Indicators for Industry Training Organisations. These
19
documents are available at: http://www.nzqa.govt.nz/providers-partners/registration-andaccreditation/external-evaluation-and-review/policy-and-guidelines-for-eerito/introduction/
NZQA
Ph 0800 697 296
E eeradmin@nzqa.govt.nz
www.nzqa.govt.nz
20