Risk indicators - Australian Skills Quality Authority

advertisement
Risk Assessment Framework
This Risk Assessment Framework is made under section 190 of the National
Vocational Education and Training Regulator Act 2011 (the Act).
This framework sets out a suggested risk assessment process for the National VET
Regulator in making decisions about:

assessing and responding to the risk of non-compliance by applicants against the
VET Quality Framework (as defined in the Act), and

assessing and responding to the risk of non-compliance by RTOs against the VET
Quality Framework, and

monitoring the compliance of RTOs against the VET Quality Framework.
This risk assessment framework outlines processes that can be used to determine
arrangements for assessing registration applications, including applications for initial
registration, applications for continuing registration and applications for extensions
to scope of registration.

For initial registration, factors considered in determining the level of risk and
appropriate responses include financial management and governance
arrangements.

For continuing registration or change of scope, factors considered in determining
the level of risk and appropriate responses include financial management,
governance arrangements and the RTO’s past performance.
The specific operating context may also affect risk management. For example,
qualifications that lead to licensed outcomes may be assessed as high risk, resulting
in higher levels of intervention from the VET Regulator, irrespective of the RTO's
performance outcomes.
This framework supports an approach where applicants assessed as having a lower
risk of non-compliance, and RTOs that are delivering high-quality training and
assessment services, will receive less monitoring by the VET Regulator. Applicants or
RTOs assessed as higher risk in terms of the likelihood of negative impacts on quality
outcomes for clients, and potential impact on the vocational education and training
(VET) system more broadly, will receive more regular monitoring and attention
through audit, with the aim of improving their performance outcomes.
Prescriptive instructions on how to calculate risk ratings (including definition of
impacts and weighting of risk indicators), or tools to assess compliance are not
included in this framework, although some suggestions are provided. Rather, this
provides a general framework for the VET Regulator to apply in undertaking risk
management in the context of the VET Quality Framework.
It should be noted that while assessment of financial management is included in this
framework, the full suite of financial viability risk assessment requirements are
established by a legislative instrument called the Financial Viability Risk Assessment
Requirements, determined under section 158 of the Act, which forms part of the VET
Quality Framework.
Risk Assessment Framework
Operating principles
The following principles underpin the risk management processes
applied by the National VET Regulator.
Principle 1 – Consistency
The National VET Regulator works with other VET Regulators to facilitate
consistent interpretation and implementation of this framework
nationally.
Principle 2 – Effectiveness
 The VET Regulator implements the risk assessment framework
and ensures that applicants/RTOs are appropriately risk managed.
 Risk management is part of the VET Regulator’s decision-making
process.
 Risk management processes are systematic, timely and structured.
Principle 3 – Proportionality
Regulatory responses are in proportion to the risk assessment of the
applicant’s / RTO’s operations.
Principle 4 – Responsiveness
 Risk management is based on the best available information.
 VET Regulators will use relevant and current data to develop and
review risk ratings and actions.
Principle 5 – Transparency
 Risk management is transparent.
 Risk management processes are documented and published. Each
RTO will know its risk rating.
 VET Regulators will respond to requests from applicants / RTOs for
information about their risk rating or about the process for
appealing decisions about a rating.
2
Risk Assessment Framework
Protocols
1. Risk assessment will inform VET Regulator monitoring of an RTO
The VET Regulator uses a risk assessment process to assess each RTO and all registration
applications, including applications for initial registration, applications for continuing
registration and applications for extensions to scope of registration, using these risk
assessment processes. The risk assessment assists the VET Regulator determine how it will
assess an application and informs the extent to which it will monitor an RTO to ensure its
operations meet the requirements of the VET Quality Framework.
A risk assessment is undertaken when an application is received, and is reviewed in the light
of current evidence of performance or any other information about an RTO.
2. Audits
Audits against the VET Quality Framework are undertaken at initial registration and for
applications of renewal of registration in accordance with the risk assessment framework,
and at any other time that the VET Regulator deems necessary, based on a risk assessment.
3. Risk management is incorporated into the VET Regulator’s decision-making
Risk management processes are systematic, timely and structured.
Having assessed the risks and determined a risk rating, the VET Regulator will determine the
scheduling and scope of audits.
4. Current data will be maintained
The VET Regulator collects data regularly from RTOs, especially higher-risk RTOs, to ensure
that information held is current.
The VET Regulator updates data when an RTO applies for renewal of registration or change
to / extension of scope, and at the time of audits.
At any time, the VET Regulator may ask an RTO to provide additional information relevant to
its risk assessment.
5. Risk management will be continuously improved
In line with the continuous improvement approach embedded in the VET Quality
Framework, ongoing analysis of risk data and information on the quality of the training and
assessment outcomes from RTOs is used to refine and improve risk-assessment and riskmanagement processes, and to identify risk issues associated with particular RTO types.
The Standards Council will monitor and review the operation of the VET Quality Framework
and the effective use of risk indicators, and recommend any necessary amendments to the
risk assessment framework.
6. Audits will be integrated where possible
The VET Regulator minimises the impact of audit activity on RTOs by integrating audits
against the VET Quality Framework, wherever possible, with audits required by other
authorities.
3
Risk Assessment Framework
Risk assessment process
The risk assessment process can be triggered by any of the following circumstances:

application for initial registration

application for renewal of registration

application for extension to scope of registration

the receipt of information on an RTO’s operations or performance that may change the
outcome of a risk assessment process and assist the VET Regulator determine a schedule
for monitoring audits.
The risk assessment process outlined in this framework is based on the Australian Standard
for Risk Management (AS/NZS ISO 31000:2009), which defines risk as ‘the effect of
uncertainty on objectives’. For the VET Quality Framework, risk relates to the potential
impact on the delivery of quality training and assessment services.
Effective management of risk in the VET Quality Framework context involves four key steps:
1
Identification of indicators of risk to be used in the risk assessment process
2
Risk assessment, which involves consideration of the potential impact if quality training
and assessment services and outcomes are not delivered and the likelihood that this will
occur, to determine a risk rating for the applicant or RTO
3
Response: assessment of applications, audits and monitoring. Assessment of
applications, audits and monitoring of RTOs are sources of risk information. The risk
rating is used to determine the scheduling and scope of audits and other monitoring
mechanisms
4
Ongoing review: the assessment of applications, audits and monitoring of RTOs
performance are sources of risk information. The VET Regulator will review any
information that may change the outcome of a risk assessment.
For initial registration, factors considered in assessing risks include the applicant’s financial
and governance arrangements, and supplementary risk indicators associated with the
proposed scope of operations. Where relevant, an applicant’s past performance as an RTO
will also be used to inform a risk rating.
For renewal of registration or change of scope, factors considered in assessing risks include
the RTO’s financial and governance arrangements, and past performance including quality
indicators and complaints history. Supplementary risk indicators associated with specified
operating contexts will also apply.
4
Risk Assessment Framework
Risk indicators
Based on the requirements of the VET Quality Framework, this risk assessment framework
suggests four groups of risk indicators for consideration in the risk assessment process:
performance, financial management, governance and supplementary risk indicators.
Performance risk indicators
Performance risk indicators contribute to the risk rating for an RTO at continuing
registration.
Performance risk indicators focus on the performance of each RTO in delivering quality skills
outcomes. The performance risk indicators are:

history of audit compliance

data from quality indicators

history of complaints.
Performance risk indicators are indicators of the likelihood that quality skills outcomes will
not be achieved.
Suggested considerations for assessing performance risks that meets the requirements of
the VET Quality Framework are at Attachment A to this framework.
Financial risk indicators
Financial risk indicators contribute to the risk rating for an applicant / RTO at initial
registration and renewal of registration.
Financial risk indicators focus on the financial viability of an applicant / RTO and the
potential impact on the delivery of quality training and assessment services and outcomes.
Financial risk indicators are indicators of the likelihood that quality skills outcomes will not
be achieved.
When assessing financial risk the VET Regulator will have regard to the Financial Viability
Risk Assessment Requirements as established under section 158 of the Act.
Governance risk indicators
Governance risk indicators contribute to the risk rating for an applicant / RTO at initial and
throughout the registration period.
Governance risk indicators focus on ensuring that an applicant / RTO has sufficient
governance structures in place to deliver quality training and assessment services and
outcomes.
The governance risk indicators are:

quality of business planning

transparency of ownership and management structure

skills and experience of senior officers and directors
Governance risk indicators are indicators of the likelihood that quality skills outcomes will
not be achieved.
5
Risk Assessment Framework
Suggested considerations for assessing governance risks that meets the requirements of the
VET Quality Framework are at Attachment B to this framework.
Supplementary risk indicators
Supplementary risk indicators apply to the operating context of each RTO, and particular
regulatory contexts.
Supplementary risk indicators contribute to the risk rating for an applicant / RTO at initial
and throughout the registration period. They may also influence the scope of an audit or
monitoring activity.
The supplementary risk indicators include:

the scope of the registration application

the delivery of training that leads to a licensed or regulated outcome

the RTO delivering training to overseas students studying in Australia

the RTO having multiple sites

the delivery of training offshore

partnering or subcontracting arrangements

the RTO accepting fees in advance from students

the RTO delivering training to students under the age of 18

the mode of delivery

compliance with and value of government training contracts.
Supplementary risk indicators are indicators of the potential impact if quality skills outcomes
are not delivered.
6
Risk Assessment Framework
Summary of risk indicators and points at which they apply
Indicator
Initial
Apply at
Continuing
Contribute to
Risk Rating
Performance
History of audit compliance


Likelihood
Data from quality indicators


Likelihood
History of Complaints


Likelihood
Financial projections


Likelihood
Financial viability risk assessments


Likelihood
Fit and proper person test


Likelihood
Quality of business planning


Likelihood
Transparency of ownership/management structure


Likelihood
Scope of registration application


Impact
Delivery of training leading to licensed or regulated
outcome


Impact
RTO delivering training to overseas students in
Australia


Impact
RTO having multiple sites


Impact
Delivery of training offshore


Impact
Partnership or subcontracting arrangements


Impact
Accepting fees in advance from students


Impact
Delivering training of students under 18


Impact
Mode of delivery


Impact
Financial
Governance
Supplementary
Risk assessment
An applicant or RTO’s risk rating is determined by evaluating information on risk indicators
and:
1
assessing the potential impact if quality skill outcomes are not achieved
2
assessing the likelihood that quality skill outcomes will not be achieved
3
determining the overall risk rating of the applicant or RTO.
7
Risk Assessment Framework
Impact
Information and data on the supplementary indicators are relevant to determine the
potential impact of a failure to deliver quality skill outcomes by the applicant or RTO. The
assessment of impact should consider the potential consequence to students, industry and
the reputation of the VET sector.
The following table may be used to rate potential impact:
Severe
Extreme disruption / inconvenience for a large number of students or damage
to the reputation of the VET sector
Major
Significant disruption / inconvenience for large number of students or damage
to the reputation of the VET sector
Moderate
Some disruption / inconvenience for students or damage to reputation of the
VET sector
Minor
Minor or negligible disruption / inconvenience for students or damage to
reputation of the VET sector.
Likelihood
Information and data on the performance, financial and governance indicators are relevant
to determining the potential likelihood of a failure of the applicant or RTO to deliver quality
training and assessment services and outcomes. That is, likelihood considers the range of
systems, controls and mitigating strategies that an applicant / RTO has or plans to
implement, and that are apparent from the performance, financial and governance
indicators.
The following table may be used to rate potential likelihood:
Almost Certain
Failure to deliver quality skills outcomes is expected
Likely
Failure to deliver quality skills outcomes will probably occur
Possible
Failure to deliver quality skills outcomes might occur
Unlikely
Failure to deliver quality skills outcomes is not expected to occur
Rare
Failure to deliver quality skills outcomes may occur only in exceptional
circumstances
8
Risk Assessment Framework
Overall risk rating
Consideration of the risk indicators and an assessment of potential impact and likelihood of
the applicant or RTO failing to deliver quality training and assessment services and outcomes
will inform an overall risk rating.
This framework suggests four categories of risk rating: low, medium, high and extreme.
The table below can be used to determine an overall risk rating based on the impact and
likelihood ratings of an applicant or RTO.
Likelihood
Impact
Minor
Moderate
Major
Severe
Almost Certain
High
Extreme
Extreme
Extreme
Likely
Medium
High
High
Extreme
Possible
Low
Medium
High
High
Unlikely
Low
Low
Medium
Medium
Rare
Low
Low
Low
Medium
The overall risk rating for the applicant / RTO will inform the scope and scheduling of audit
activity, and other monitoring mechanisms.
The following table sets out possible responses by the VET Regulator, based on the overall
risk rating. Specific actions are discussed under ‘Response’ below.
Rating
Possible responses
Extreme
Immediate action required by the VET Regulator of a type determined
appropriate by the VET Regulator
High
Intolerable risk. Applicant / RTO to be audited and monitored. Other mitigation
strategies may also be applied (conditions placed on the RTO’s registration, etc)
as deemed appropriate by the VET Regulator.
Medium
Risk mitigation through a program of audit and/or monitoring activity.
Low
Tolerable risk. No specific audit activity required and may include a no audit
option.
Specific actions are discussed under ‘Response’ below.
9
Risk Assessment Framework
Response: Assessment of applications and audits
The nature, frequency and scope of audits should be determined by the overall risk rating,
derived from the assessment of performance, financial, governance and supplementary
risks.
Nature of audit
The response and approach to audits should vary depending on the overall risk rating and
the category of risk indicator that has driven the overall risk rating. For example, if the
likelihood of financial collapse is more probable than the likelihood of poor performance,
then the focus of audits may be on financial results and financial position, rather than the
operations of the RTO.
Audits by the VET Regulator may be:

Site audits, conducted at the premises or proposed premises of an RTO or applicant
and/or at locations where it delivers or proposes to deliver training and assessment

Compliance assessments, where an RTO or applicant submits documents or information
for assessment by the VET Regulator as evidence of compliance with the VET Quality
Framework.
Initial registration
At initial registration, the overall risk rating of an applicant cannot be determined until the
information submitted to the VET Regulator in support of the application is assessed. As
such, on initial registration:
 a comprehensive compliance assessment will be performed, focussing on:
o Governance risk indicators
o Financial risk indicators

a site audit may be performed to examine an applicant’s preparedness to commence
delivery.
10
Risk Assessment Framework
Renewal of registration and ongoing monitoring
On renewal of registration or change to scope of registration, or for the purpose of
monitoring RTO performance, the nature of the audit is based on the overall risk rating. For
example:
Extreme
High
Medium
Low
Renewal of registration
Renewal of registration
Renewal of registration
Renewal of registration
Extensive VET Regulator
intervention
Comprehensive site audit
Site audit or compliance
assessment focussed on
high risk indicators
Site audit or compliance
assessment
Change to scope
Change to scope
Change to scope
Change to scope
Extensive VET Regulator
intervention
Site audit or compliance
assessment focussed on
significant changes
Compliance assessment
focussing on significant
changes or self assessment
Compliance assessment
focussing on significant
changes or self assessment
Ongoing monitoring
Ongoing monitoring
Ongoing monitoring
Ongoing monitoring
De-register or show cause
Site audit or compliance
assessment focussed high
risk indicators
Site audit or compliance
assessment focussed on
high risk indicators
Self assessment of
compliance
Frequency and scheduling of audits
The frequency and scheduling of audits should vary depending on the overall risk rating and
the period since the RTO was last audited. At a minimum, an audit of some nature should be
conducted:

on initial registration

post registration – within one year of initial registration or commencement of training
delivery

on renewal of registration or extension to scope of registration

at intervals determined by the VET Regulator and based on the RTO’s risk rating.
RTOs rated high or medium risk may receive additional audits or monitoring:

For high risk RTOs such additional audits should be scheduled and may focus on
monitoring those indicators which present a higher risk

For medium risk RTOs, additional audits may be scheduled to monitor higher risk
indicators
Other circumstances in which audits may be conducted include:

National or jurisdictional strategic industry audits – The need for and scope of a national
strategic industry audit is defined in consultation with all jurisdictions and
representatives of the relevant industry. The need for and scope of a jurisdictional
11
Risk Assessment Framework
strategic industry audit is defined by the jurisdiction requiring the audit, in consultation
with representatives of the industry.

Complaints against an RTO – The VET Regulator will refer and manage a complaint about
a RTO appropriately. Complaints about a RTO will also inform regulatory approach and
this may include an audit.

Change to the management or operation of an RTO – Advice by an RTO regarding a
significant change in the ownership, management or operation of an RTO may trigger an
audit. The VET Regulator decides how the change is to be considered and this may
include an audit.

Training that leads to a licensed or regulatory outcome - Where an agreement has been
established with an industry regulator, an application for registration must be audited in
accordance with the agreement, and monitoring audits are conducted in accordance
with the agreement.
Audit scope
The overall risk rating, the higher risk indicators and the supplementary risk indicators which
apply to the operating context should determine the scope of audits.
Initial registration
Applications for initial registration are generally the first source of risk information on
applicants. On initial registration the audits should be comprehensive and focus on
assessing compliance with all elements of the VET Quality Framework.
Information gathered during the initial registration audit will determine an RTO’s initial risk
rating.
Renewal of registration and ongoing monitoring
On renewal of registration or extension of scope of registration, or for the purpose of
monitoring RTO performance, the scope of the audit, where required, should be based on:

elements of the registration that are changing

higher risk indicators – the assessment of performance, financial and governance risk
indicators may impact the scope of any audit activity. Where such indicators are
assessed at a likelihood level of ‘likely’ or above, they should be reflected in the scope of
an audit.

supplementary risk indicators
12
Risk Assessment Framework
The impact of supplementary risk indicators on the scope of an audit
The supplementary risk indicators may influence the scope of an audit for RTOs with an
overall risk rating of high, medium or low, as follows:
High
Medium
Low
Scope of registration
application
The scope of an audit should directly reflect the scope of registration. An
assessment of an applicant or RTO’s suitability should consider the range of
training and qualifications that it is or is proposing to deliver.
Training that leads to a
licensed or regulated
outcome
Where an agreement has been established with an industry regulator, an
application for registration must be audited in accordance with the
agreement.
Delivery to overseas students
studying in Australia
AQF qualifications offered to overseas students will be included in the audit
sample.
RTOs with multiple training
sites
At each audit a sample of
training sites is audited in
addition to the head
office.
RTOs operating off-shore
Ensure evidence of compliance is assessed at least once during the
registration period.
Partnerships/subcontracting
arrangements
Audit a sample of
partnership/
subcontracting
arrangements at each
audit
Fees taken in advance from
students
Compliance may be checked at audit against the VET Quality Framework,
particularly if the RTO accepts the majority of fees from individual students
in advance of training delivery.
Delivery of services to
students under the age of 18
Compliance with relevant legislative requirements for the protection of
children is checked at each audit.
Mode of delivery
An audit takes into account the modes of delivery used and focuses on the
risks of a learner not achieving competency to industry standards and the
management of those risks.
The VET Regulator audits a sample of training
sites at least once during the registration
period.
Audit a sample of partnership / subcontracting
arrangements at least once during the
registration period.
RTOs rated extreme risk will receive extensive intervention of a nature deemed
appropriate by the VET Regulator.
13
Risk Assessment Framework
Attachment A: Considerations for assessing performance risks
Performance risk indicators focus on the performance of each RTO in delivering quality skills
outcomes. The performance risk indicators are:



history of audit compliance
data from quality indicators
history of complaints
Performance risk indicators are particularly relevant to the RTOs risk rating, as at initial
registration there will commonly not be any performance information or data available to
inform a risk assessment services and outcomes.
Performance risks should be reviewed when new information on the performance of an RTO
becomes available (e.g. through complaints or audits).
When assessing performance risk indicators to determine the likelihood of a failure to
deliver quality training and assessment services and outcomes, the VET Regulator may
consider the following:
Almost certain
Likely
Possible
Unlikely
Very unlikely
Audit
compliance:
Audit
compliance:
Audit
compliance:
Audit
compliance:
Audit
compliance:
Recent history
of critical noncompliance
with the VET
Quality
Framework
Recent history
of significant
non-compliance
with the VET
Quality
Framework
History of
significant noncompliance with
the VET Quality
Framework
Recent history
of minor noncompliance with
the VET Quality
Framework
History of minor
non-compliance
with the VET
Quality
Framework
or
or
or
or
Quality
indicators data
reflects very
poor
performance
Quality
indicators data
reflects very
poor
performance
Quality
indicators data
reflects
marginal
performance
Quality
indicators data
reflects high
standard of
performance
or
or
or
or
History of
verified
complaints
demonstrating
critical noncompliance
History of
verified
complaints
demonstrating
significant noncompliance
History of
verified
complaints
demonstrating
minor noncompliance
No history of
verified
complaints
14
or
Quality
indicators data
reflects high
standard of
performance
and
No history of
verified
complaints
Risk Assessment Framework
Attachment B: Considerations for assessing governance risks
Governance risk indicators focus on ensuring that an applicant / RTO has sufficient
governance arrangements in place to deliver quality training and assessment services and
outcomes.
The governance risk indicators are:
• Quality of business planning
• Transparency of ownership and management structure
• Skills and experience of senior officers and directors.
Quality of business planning
In assessing the quality of business planning undertaken, the VET Regulator may consider
information including but not limited to:
• Evidence of market research and budget projections, including details of assumptions
underlying projections (revenue, student enrolments etc.)
• Clarity and realism of objectives and strategies for achieving the objectives
• Demonstrated understanding of the VET Quality Framework
• Appropriate consideration of financial resources, staffing and assets required to deliver the
plan
• A comprehensive risk plan and appropriate mitigation strategies
• Appropriate governance structure with clear and transparent roles and accountabilities.
Transparency of ownership and management structures
In assessing the quality of transparency of ownership and management structures, the VET
Regulator may consider information including but not limited to:
• Complexity of ownership structures – i.e. can the ultimate business owners be easily
identified and held accountable (this may include whether or not they are based in Australia)
• Transparency of management responsibilities/reporting lines
• Adequacy of management controls in place
• Whether the organisation has any existing domestic operations
Skills and experience of senior officers and directors
In assessing the skills and experience of senior officers and directors who are in a position to
influence the management of the organisation, the VET Regulator may consider information
including but not limited to:
• Level of educational expertise and background of senior officers
• The extent of prior experience of senior officers in managing an RTO
• The extent of prior experience of directors in the business of an RTO
• The level and quality of independent educational expertise available to senior officers and
directors
• Extent to which control of the RTO/applicant is based in the jurisdiction of registration
• Evidence of involvement of senior officers and directors in business planning
15
Download