GUIDELINES
Monitoring of programmes
1
leading
NZQA Monitoring of programmes
to degrees
leading to degrees andand
related qualifications: Guidelines
related qualifications
December 2014
2
NZQA Monitoring of programmes leading to degrees and related qualifications: Guidelines
December 2014
Guidelines for the monitoring of
programmes leading to degrees and
related qualifications
for Tertiary Institutions other than Universities
Version 1.0
December 2014
NZQA
125 The Terrace
PO Box 160
Wellington
3
NZQA Monitoring of programmes leading to degrees and related qualifications: Guidelines
December 2014
Contents
Contents
4
Introduction 5
Section 1
The monitoring process .................................................................................... 7
Section 2
The monitoring visit ........................................................................................... 9
Section 3
Self-monitoring................................................................................................ 13
Appendix – Monitoring report template................................................................................... 15
4
NZQA Monitoring of programmes leading to degrees and related qualifications: Guidelines
December 2014
Introduction
This document provides guidance to tertiary institutions, other than universities, and monitors on
the requirements for the monitoring of programmes of study leading to degrees and related
qualifications. Institutions include:

institutes of technology or polytechnics (ITPs)

Wānanga

private training establishments (PTEs).
NZQA will monitor programmes of study that lead to Graduate Diplomas and Graduate
Certificates, Bachelor’s Degrees, Post-graduate Diplomas and Post-graduate Certificates,
Bachelor Honours Degrees, Master’s Degrees and Doctoral Degrees qualifications.
Statutory basis
NZQA’s rules set out what institutions have to do to meet NZQA’s quality standards and comply
with NZQA processes. They indicate what actions and information are required from institutions.
All NZQA Rules are listed on the NZQA website at http://www.nzqa.govt.nz/about-us/ourrole/legislation/nzqa-rules/.
The NZQF Programme Approval and Accreditation Rules 20131 (the Rules) require institutions to
participate in monitoring as one of a number of requirements to maintain their accreditation to
provide an approved programme2.
Rule 12.2 of the NZQF Programme Approval and Accreditation Rules 2013
12.2 To continue to maintain accreditation to provide an approved programme or
part of an approved programme leading to a degree or post-graduate qualification at
levels 7 to 10, institutions (other than universities) must:
e.
participate in monitoring, which will consist of either:
(i) annual visits to the institution by the NZQA appointed monitor, reports by
the monitor on the implementation of the programme, and reviews of the
first graduating years of programmes by the institution; or
(ii) where NZQA permits, self-monitoring in accordance with any conditions
imposed by NZQA.
1
http://www.nzqa.govt.nz/assets/About-us/Our-role/Rules/Prog-App-Accred-Rules.pdf.
See Degrees and Related Qualifications – Guidelines for Programme Approval and Accreditation to Provide Programmes for
more information.
2
5
NZQA Monitoring of programmes leading to degrees and related qualifications: Guidelines
December 2014
The purpose of monitoring
The purpose of monitoring programmes of study that lead to degrees and related qualifications is
to provide assurance to NZQA and stakeholders that a programme and its delivery continue to
meet the criteria for ongoing approval. Monitoring reports and, when applicable, self-monitoring
reports must provide evidence that:

The programme is being managed, planned and implemented as it was approved.

Consideration has been given to any recommendations made by the evaluation panel.

Any minor modifications and enhancements made by the institution are consistent with the
intent of the approved programme and the ongoing development of a quality programme,
and in line with type 1 change.

There is independent, external academic input during reviews and consideration of
significant programme enhancements (i.e. type 2 changes).

NZQA is made aware of issues affecting the satisfactory provision of the programme.

The quantity and quality of staff research outputs are consistent with the development and
maintenance of an ongoing research culture in support of the programme.
Monitoring by NZQA is not intended to replace the actions taken by institutions to monitor, review
and regularly improve the quality of the programmes for which they are responsible.
6
NZQA Monitoring of programmes leading to degrees and related qualifications: Guidelines
December 2014
Section 1 The monitoring process
Under Rule 12.2 of the NZQF Programme Approval and Accreditation Rules 2013, an institution
is required to participate in monitoring to continue to maintain accreditation to provide an
approved programme or part of an approved programme at levels 7 to 10.
NZQA will appoint one monitor who will produce an annual monitoring report following an on-site
visit to the institution3.
Appointment of monitors
NZQA will appoint monitors with expert knowledge of the discipline area of the programme and
who are experienced in academic processes. Monitors will:

have an independent and neutral perspective on the programme and the institution

be appointed by NZQA in consultation with the programme evaluation panel and the NZQA
advisor, and with the agreement of the institution

be appointed by NZQA through a contract for service which includes a conflict of interest
and a confidentiality4 clause.
Regular rotation of monitors will ensure the effectiveness and impartiality of the monitoring
process.
Role of NZQA advisor
An NZQA advisor may accompany the monitor for the first monitoring visit.
The role of the NZQA advisor, if accompanying the monitor is to:

inform the monitor of NZQA’s expectations and processes

provide neutral and experienced management of the monitoring visit

when necessary, provide a link between the approval and/or accreditation of the
programme.
The involvement of the advisor on future visits will be determined by NZQA following discussion
with the advisor, the monitor and the institution. An institution may request the ongoing
participation of the NZQA advisor.
Monitoring in conjunction with regulatory bodies
Where professional registration is a prerequisite for practising in a particular field or occupation,
professional registration bodies will be involved in the content and quality of education being
delivered.
Written formal agreements between registration bodies and NZQA detail the relevant processes
and requirements, including shared responsibilities for the monitoring of programmes.
3
Multiple monitors may be appointed depending upon the complexity and breadth of the programme(s) being delivered.
Monitors are required to declare any potential conflicts of interest to NZQA, and to preserve confidentiality in relation to
information shared by the institution during the monitoring process.
4
7
NZQA Monitoring of programmes leading to degrees and related qualifications: Guidelines
December 2014
Monitoring of programmes delivered offshore
Programmes delivered offshore that lead to the award of New Zealand qualifications are subject
to the same monitoring requirements as programmes delivered within New Zealand.
The monitoring process may be extended to meet the requirements of any relevant overseas
accreditation body.
In addition, where programmes are delivered in conjunction with an overseas institution, the
monitoring process must include a formal evaluation of the effectiveness of the collaborative
arrangement.
Collaborative delivery
If a programme is being delivered collaboratively, the monitor will need to examine this
arrangement and may need to visit other associated sites.
8
NZQA Monitoring of programmes leading to degrees and related qualifications: Guidelines
December 2014
Section 2 The on-site monitoring process
Under Rule 12.2 e (i) of the NZQF Programme Approval and Accreditation Rules 2013,
monitoring consists of ‘annual visits to the institution by the NZQA appointed monitor, reports by
the monitor on the implementation of the programme, and reviews of the first graduating years
of programmes by the institution’.
Organising the visit
Monitoring should take place at an appropriate time during the delivery of the programme(s),
taking into account the length of the programme, the start and end dates of the academic year,
and the availability of staff and learners.
On-site monitoring visits will typically be made at the beginning of the second year of the
delivery of the programme.
The monitor, NZQA advisor where relevant, and the institution will determine a mutually
agreeable date for the monitoring visit to occur. Dates should be determined at least two months
prior to the visit to allow sufficient time for arrangements to be made.
If an advisor is not involved in the monitoring process on-site, NZQA should still be kept
informed of the agreed date.
At least one month before the visit, the institution is expected to provide the monitor with
information, including but not limited to:

enrolment information

learner retention and achievement

graduate destination information

internal and external moderation activities

feedback from learners, teaching staff and external stakeholders

consultation with external stakeholders

how the institution has addressed any recommendations from:
o
the initial evaluation report and/or
o
the most recent monitor's report

changes to the programme and/or its delivery since the last report

staffing changes since the last report

current resources to maintain delivery of the programme

staff professional development activities

research activities of staff

significant issues and challenges

collaborative and/or sub-contracting arrangements.
9
NZQA Monitoring of programmes leading to degrees and related qualifications: Guidelines
December 2014
Conducting the on-site monitoring process
Monitoring will generally take place over one day and will include meetings with:

senior management

programme manager(s)

lecturers/teaching staff

resource managers (e.g. librarians, learner support)

learners.
The monitor may also ask to speak with:

external advisory committee members

external moderators

other stakeholders.
Fees
The total costs of the monitoring process, including fees, travel, accommodation, and NZQA
costs, are the responsibility of the institution.
Responsibilities of the monitor, the institution and NZQA
On their appointment by NZQA, monitors are provided with a contract, the Guidelines and the
monitoring report template.
The monitor is responsible for:

coordinating the timing of the visit with the institution at least two months before the
proposed date

conducting the monitoring process with the institution

completing a draft monitoring report, no later than ten working days after the visit, in
accordance with the Guidelines

seeking confirmation of the factual accuracy of the draft monitoring report from the
institution

providing the draft monitoring report (in an editable electronic format) to NZQA within ten
days of receipt of the confirmed factual accuracy from the institution

raising any issues with NZQA which may have an impact on the monitoring process or
outcome.
The institution is responsible for:

coordinating, at least two months before, the date of the visit with the monitor

coordinating any requirements relating to the monitoring visit with the monitor

providing appropriate documentation to the monitor at least one month before the visit

confirming or correcting the factual accuracy of the draft monitoring report with the
monitor, within ten working days of receipt of the report
10
NZQA Monitoring of programmes leading to degrees and related qualifications: Guidelines
December 2014

raising any issues with NZQA which may have an impact on the monitoring process or
outcome.
NZQA is responsible for:

appointing monitor(s)

providing the institution with the final version of the monitoring report within ten working
days of receipt of the draft report from the monitor

addressing any issues brought to its attention by the monitor, institution or other relevant
party which may have an impact on the monitoring process, and

granting the institution the ability to self-monitor.
Following the on-site visit
The main outcome of any monitoring visit is a monitoring report. This is produced by the
monitor(s) and considered by NZQA, and details the monitor’s professional opinion on the
quality of the institution’s delivery of the programme.
The monitoring report
The monitoring report will inform NZQA of the outcomes of the monitor’s visit. The report will
contain evidence and analysis confirming that the programme continues to meet the programme
approval and accreditation criteria.
NZQA will provide the final monitoring report5 to the institution. If there are any major concerns
raised by the report, NZQA may request that the institution takes appropriate steps to address
the concerns.
If NZQA continues to have serious or ongoing concerns about the quality and/or stability of a
programme or its delivery, it may initiate the procedure to withdraw an institution’s accreditation
to deliver the programme.
Timelines
The monitor is expected to send the draft monitoring report to the institution within ten working
days of the monitoring visit.
The institution is expected to confirm factual accuracy of the report within ten working days of
receiving the draft monitoring report from the monitor.
The monitor is expected to send the monitoring report to NZQA within ten working days of it
being finalised.
NZQA is expected to send the final version of the monitoring report to the institution within ten
working days of receipt of the report from the monitor.
Graduating year review and special reviews
Graduating year review
Under Rule 12.2 (e) (I), the review of the first graduating year of a programme needs to occur as
a part of the institution’s annual programme evaluation, and the findings reported. These
findings should be included in the information given to the monitor.
5
A template for the monitoring report is provided in the Appendix.
11
NZQA Monitoring of programmes leading to degrees and related qualifications: Guidelines
December 2014
The review must include the following:

a programme statement with a brief overview of the programme, the extent to which its
original aims have been achieved and any changes that have occurred

a description of the review process

the outcomes of the review process, including the ongoing acceptability of the programme
to stakeholders, assessment and learner performance, enrolment and completion data

the outcomes of any external evaluations, for example by professional registration bodies
or by NZQA external evaluation and review (EER)

a statement of intention to continue or discontinue the programme and a rationale for that
decision.
Special reviews
Under Rule 19 of the NZQF Programme Approval and Accreditation Rules 2013, NZQA retains
the right to undertake a special review of an approved programme and/or its delivery by an
accredited institution.
NZQA may initiate a special review if it becomes aware of serious or ongoing concerns about
the quality and/or stability of a programme or its delivery.
If NZQA has serious concerns about the delivery of a programme following a special review, it
may initiate the procedure to withdraw an institution’s accreditation to deliver the programme.
12
NZQA Monitoring of programmes leading to degrees and related qualifications: Guidelines
December 2014
Section 3 Self-monitoring
Under Rule 12.2 e (ii) of the NZQF Programme Approval and Accreditation Rules 2013, selfmonitoring in accordance with any conditions imposed by NZQA can be considered.
When is self-monitoring considered?
When the programme is well established, after the first cohort has graduated, the monitor may
recommend that an institution requests to discontinue monitoring and replace it with selfmonitoring. This occurs when the monitor considers that a programme and its delivery are
stable, and that all conditions for changing the institution’s monitoring status are met.
It is expected that an institution will continue to use an external monitor as part of the selfmonitoring process. Where NZQA approves self-monitoring, other registration bodies may
continue to monitor the programme.
Self-monitoring involves the provision of an Annual Programme Evaluation Report (APER) to
NZQA. This arrangement must be approved by NZQA.
If serious concerns regarding the programme(s) are identified, NZQA may revoke the
institution’s approval to self-monitor.
Conditions for requesting self-monitoring
Conditions for the transfer to self-monitoring include confirmation from all parties that the
institution is managing the programme appropriately and in particular that:
1.
The programme is being implemented as planned and presented at the time of approval,
subject to modifications and enhancements broadly consistent with the intent of the
programme and the natural evolution of a quality programme.
2.
Recommendations made by the evaluation panel and the monitor have been appropriately
addressed.
3.
Mechanisms are in place at an institutional level to ensure independent, external
academic input during reviews and consideration of proposed programme enhancements.
4.
That NZQA will gain sufficient awareness of any issues with the programme or its delivery
from the APER.
Consultation and approval
NZQA will consult with the monitor before reaching a decision. Approval for transferring the
responsibility for monitoring to the institution will be granted by the Deputy Chief Executive,
Quality Assurance Division. If the request is granted, the involvement of the NZQA monitor will
be discontinued.
Request for self-monitoring
Institutions meeting the above conditions must request a transfer to self-monitoring status by
formally writing to NZQA.
13
NZQA Monitoring of programmes leading to degrees and related qualifications: Guidelines
December 2014
Monitoring by Annual Programme Evaluation Report
NZQA does not prescribe how an institution must conduct its self-monitoring. However, NZQA
would expect to find the following information in an APER:

enrolment information

learner retention and achievement

graduate destination information

internal and external moderation activities

feedback from learners, teaching staff and external stakeholders

consultation with external stakeholders

how the institution has addressed any recommendations from:
o
the initial evaluation report and/or
o
the most recent monitor's report

changes to the programme and/or its delivery since the last report

staffing changes since the last report

current resources to maintain delivery of the programme

staff professional development activities

research activities of staff

significant issues and challenges and proposed actions.
Report format
When a request for monitoring a programme by APER has been approved, the institution is
responsible for carrying out a review process each year and sending a copy of the APER to
NZQA by 30 June. The review will be carried out according to the institution’s quality
management system. No specific report format is required.
The purpose of the report is to provide evidence that:

The programme continues to be implemented as approved.

There has been ongoing consultation with stakeholders including monitoring by an
independent academic.

Minor changes and developments have been approved by the institution’s board.
It is important to clearly identify any issues or problems that may have arisen during the year,
the way these matters have been or will be addressed, and the possible need to make an
application for major change (type 2 change) to a programme.
14
NZQA Monitoring of programmes leading to degrees and related qualifications: Guidelines
December 2014
Appendix – Monitoring report template
Monitoring Report
[Programme name]
[Name of institution]
Name of Monitor(s): [Name]
On-site Date:
[Date]
[Delete any comments in square brackets (including this one) before providing the
report to the institute for confirmation of factual accuracy]
15
NZQA Monitoring of programmes leading to degrees and related qualifications: Guidelines
December 2014
Introduction
[Provide a short paragraph providing background to the report: e.g. the programme is in the
fifth year of delivery/this is the first monitoring visit since the programme was approved/this is
the second monitoring visit this year etc].
Current Operation
[Please comment as indicated under the following headings].
Previous recommendations
[Provide a brief account of the issues that were identified at the previous monitoring visit (or
evaluation panel visit).
How quickly and effectively has the institution addressed or mitigated these issues?]
Summary of the visit
[Provide a brief summary of the monitoring visit e.g. individuals met, sites visited, the
particular ‘focus’ of the visit.]
Programme structure
[Comment on any changes the institution has made to the programme structure in the last 12
months e.g. new courses developed, changes to electives.
How effectively do the changes meet the needs of learners and other stakeholders?]
Learner achievement
[Comment on the effectiveness of programme delivery in terms of learner achievement. How
well are learners guided and supported?
What value do the learning outcomes have for learners and other stakeholders?
How effectively is information on learner retention, withdrawals and achievement measured,
recorded and used?]
Teaching effectiveness
[Comment on the effectiveness of the teaching/learning strategies.
Have there been enhancements to the delivery methods (e.g. a move to online delivery, move
to work-based learning), programme structure, or learning and assessment strategies?
How effective have the changes been for learners?]
Assessment
[Comment on the effectiveness and appropriateness of the assessment tools/methods. Have
any issues arisen from internal or external moderations, and if so, how have these issues
been addressed?
How effective are the internal moderation activities?]
16
NZQA Monitoring of programmes leading to degrees and related qualifications: Guidelines
December 2014
Programme regulations
[Outline any changes that have been made to the programme regulations, the rationale for the
changes, and the impact the changes will have on learners and other stakeholders.
What was the goal of any changes to programme regulations?
How were these changes communicated to the learners and stakeholders?
How well have the changes met the goals?]
Resources
[Comment on the current level of teaching, staffing (academic, support), facilities and physical
resources available for the programme’s delivery. It would be helpful if an indication of total
staff (FTE) that contribute to the teaching of the programme could be identified.
How effectively are physical resources and facilities used to deliver the programme?
How appropriate are the available resources and facilities?
How appropriate are the staff and staffing levels for delivering the programme?]
[In the case of programmes delivered at more than one site (e.g. at a remote campus, or in
collaboration with another institution), please comment on the effectiveness of the institution’s
systems and resources to manage and quality assure the multi-site delivery.
How effective are the collaborative arrangements?
How well are the partner institutions’ and learners’ needs being met by the arrangements?]
Stakeholder feedback
[Comment on the effectiveness of the institution’s process for seeking feedback on the
programme from learners and stakeholders and how this feedback is used.
How effectively is feedback from learners, teaching staff and external stakeholders captured
and analysed?
How is this feedback used to inform improvements to the programme and/or its delivery?]
Research
[Comment on the current level of research activity.
What type and depth of research is the teaching staff involved in?]
Issues and challenges
[Comment on any significant risks or challenges to the programme that are likely to be
encountered in the next 12 months (e.g. falling enrolments, need for major equipment or
accommodation upgrades, changes to specific professional requirements etc.). Comment on
the processes the institution has put in place to deal with these challenges.
How will significant risks and challenges be identified?
How relevant are the processes to deal with these challenges to the upcoming challenges?
What other processes need to be considered?]
17
NZQA Monitoring of programmes leading to degrees and related qualifications: Guidelines
December 2014
Commendations
[Outline specific areas of good practice that deserve special mention.]
Recommendations
To the institution
[List the recommendations (if any) you wish to provide to the institution to enhance the
delivery of the programme. These may have already been identified in the preceding sections
but a summary list is helpful. This section can also be used when monitors wish to
recommend that the institution apply for self-monitoring.
Which aspects of the programme and/or delivery need further development or improvement?
What specific development or improvement is needed?]
To NZQA
[Use this section when there are serious concerns with particular aspects of the degree, which
NZQA will need to follow up urgently with the institution (e.g. special review).
What aspect(s) of the programme and/or delivery show a serious non-compliance with NZQA
requirements or a lack of quality that is likely to seriously disadvantage learners?
What specific action(s) does NZQA need to take to address the situation?]
This report was confirmed as accurate by [Name of person at institution], [position at institute]
on [date].
Monitor’s Signature
Date
Please submit this report to DegreeMonitoring@nzqa.govt.nz
18
NZQA Monitoring of programmes leading to degrees and related qualifications: Guidelines
December 2014