Guide to Planning and Implementing Independent Validation of

advertisement
Guide to Planning and
Implementing Independent
Validation and Moderation of
Assessment
December 2013
NSW Public Sector ITAB
Guide to Planning and Implementing Independent Validation and Moderation of Assessment
Acknowledgements
This Guide has been produced by the NSW Public Sector ITAB in conjunction with GE Consulting Pty
Ltd, as part of a pilot project on Independent Validation of Assessment that was funded by the NSW
Department of Education and Communities. The Guide draws on previous work in this area and
acknowledges sources of documents and templates, where known.
The NSW Public Sector ITAB would like to thank the project’s participants from the Water Industry
for their contribution in shaping the contents of this Guide that can be utilised by RTOs from any
industry.
Guide to Planning and Implementing Independent Validation and Moderation of Assessment
Table of Contents
Section
Purpose of Guide
How to use the Guide
1
Page
1
1
Introduction to Independent Validation and Moderation of
Assessment
Integrating industry consultation with validation and moderation
2
2
Definitions
4
3
Benefits of Independent Validation and Moderation of Assessment
5
4
Timing of Validation Activities
6
5
Steps in the Validation/Moderation Process
7
6
Templates and Forms
1 – Validation and Moderation Schedule
2 – Validation and Moderation Plan
3 – Steps for Planning the Validation/Moderation Process
4 – Steps for Conducting/Participating in Validation and Moderation
Activities
5 – Steps for Reviewing Validation/Moderation activity
6 – Validation of Assessment Tools
7 – Moderation of Assessment Evidence
8 – Validation of Assessment Process
9 – Checklist of what to organise when planning the activities
10 – Confidentiality Agreement
11 – Questions for Independent Parties
12 – Sample Student Questions
13 – Roles and responsibilities of Internal and External Parties
14 – Questions for evaluation of validation activity
13
14
15
16
18
Code of Professional Practice for RTOs and Independent Parties
Participating in Validation and Moderation
39
7
3
19
20
24
27
30
32
34
36
37
38
Guide to Planning and Implementing Independent Validation and Moderation of Assessment
Purpose of the Guide
This Guide has been designed for Registered Training Organisations (RTOs) to use and customise
when undertaking independent validation and moderation of assessments.
The purpose of the Guide it to:
 encourage you to recognise the benefits of adopting independent validation and moderation
 describe the process for the inclusion of independent (or external) parties in assessment
validation and moderation
 enable you to reflect on your current practice
 provide you with suggested templates to use to plan and record the outcomes from
independent validation and moderation of assessment
How to use the Guide
The Guide has a number of sections and depending on your needs you may not need to read the
whole document. The sections include
1. An introduction to independent validation and moderation of assessments
2. Definitions
3. Benefits of independent validation of assessment and moderation
4. Timing of validation and moderation activities
5. Steps in the process
6. Templates and forms
7. Code of Practice for independent validation and moderation of assessment
December 2013
Page 1
Guide to Planning and Implementing Independent Validation and Moderation of Assessment
Section 1:
Introduction to Independent Validation and Moderation of Assessment
Recent research has revealed that unlike other comparable countries, Australia’s vocational
education and training (VET) sector does not have well-established internal and external processes
for the validation and moderation of assessment. Given assessment practices have long been a
concern in the VET sector, the Commonwealth Government is now moving towards developing a
national model or a set of principles that can be adopted by the VET sector to ensure independent
validation of assessment is undertaken by RTOs.
Validation and moderation are essential components of a quality review process, with the outcomes
informing the development and improvement of assessment practice. The input from independent
parties is invaluable in improving the confidence of employers and students in vocational education
and training.
In simple terms, independent validation and moderation is really about:



Engaging with stakeholders involved in the assessment process
Ensuring that RTOs and/or assessors are being inclusive in the development and
implementation of assessment
Ensuring that what you are doing is meeting the client, learners and other stakeholders’
needs
RTOs need to consider that validation is a planned, systematic and ongoing approach which could
include multiple activities that captures the information you require. It may involve informal
activities such as asking questions of supervisors or students at work placement or traineeship visits.
It may also involve formal meetings with a group or panel of suitably experienced industry members
or assessors outside of the RTO. However, all validation activities and outcomes must be
documented, using the RTO’s agreed processes.
Validation and moderation activities do not necessarily need to be undertaken in a face to face
environment. This may not be possible for many independent parties. The use of technology in
validation/moderation such as Skype, teleconferences and video links can be useful and provide a
valid enabler for the activity.
December 2013
Page 2
Guide to Planning and Implementing Independent Validation and Moderation of Assessment
Integrating industry consultation with validation and moderation
It is not difficult to incorporate independent validation and moderation with your regular interaction
with clients. You should be lateral with your thinking and questioning of managers, supervisors and
employees. The templates in this Guide may assist you with your questioning.
The flow diagram below illustrates how validation/moderation can be integrated with your industry
consultation.
December 2013
Page 3
Guide to Planning and Implementing Independent Validation and Moderation of Assessment
Section 2:
Definitions
Independent parties are those external to the RTO, therefore not employed or contracted by the
RTO. External parties could include industry associations, small or large business, assessors from
other RTOs or any unbiased or impartial third-party. Students can contribute to the validation and
moderation process by providing feedback on the assessment tasks, assessment process and
decision making (refer to Sample Templates). The importance is that independent representatives
have the knowledge, skills and relevant experience in the area themselves. This experience can
either be as an industry expert or a qualified VET assessor.
Validation
“… is a quality review process. It involves checking that the assessment tool produced valid, reliable,
sufficient, current and authentic evidence to enable reasonable judgements to be made as to
whether the requirements of the relevant aspects of the Training Package or accredited course had
been met. It includes reviewing and making recommendations for future improvements to the
assessment tool, process and/or outcomes”.
Ref: Code of Professional Practice for Validation and Moderation, p 7
Moderation
“……is the process of bringing assessment judgements and standards into alignment. It is a process
that ensures the same standards are applied to all assessment results within the same Unit(s) of
Competency. It is an active process in the sense that adjustments to assessor judgements are made
to overcome differences in the difficulty of the tool and/or the severity of judgements.”
Ref: Code of Professional Practice for Validation and Moderation, p 8
Ongoing, systematic approach
Collecting evidence of validation can take many forms and include a number of activities. The key is
to use a number of methods over a period of time, showing how they are used to improve the
assessment practice within the RTO. This could include asking some key questions of supervisors or
students in the workplace and documenting the information as part of the review process. Anyone
within the RTO can collect this information using some of the suggested sample questions in the
templates.
December 2013
Page 4
Guide to Planning and Implementing Independent Validation and Moderation of Assessment
Section 3:
Benefits of Independent Validation and Moderation of Assessment
The key benefit of any validation and moderation process is that it is likely to strengthen the validity
and consistency of assessment.
For RTOs and assessors, it:









Encourages consistent understanding of the competency standards and assessment
practices and processes
Allows the evaluation of the quality of the assessments
Promotes discussion of issues of concern about the assessment process
Provides for suggested improvements to the assessment system or processes
Fosters the development of informal assessor networks and the exchange of information
Provides for the development of benchmark performances (examples of what competent
performance looks like)
Increases confidence amongst assessors and RTOs in decision making
Enables assessors to keep up-to-date about what is happening in their industry
Enables RTOs and assessors to review their own practices and professional development
needs
For industry, it:






Promotes consultation with RTOs
Facilitates ongoing relationships with RTOS
Facilitates partnerships with RTOs
Engages industry in the assessment process
Allows employers to have their needs addressed
Promotes confidence amongst industry
December 2013
Page 5
Guide to Planning and Implementing Independent Validation and Moderation of Assessment
Section 4:
Timing of Validation Activities
Validation may occur before, during and after assessment.
Validation before assessment
Involves assessors and independent parties collaborating together, concentrating on:
 Interpreting the unit/s of competency
 Coming to a common understanding of the standard to be achieved so agreeing on what a
competent person would ‘look like’
 The overall design of the assessment process; includes what evidence would be collected,
where it would be collected and the number of assessments. This then informs;
 the development of the assessment tools which includes all student assessment tasks and
assessors documents including evidence guides
 briefing relevant workplace personnel
 gathering of relevant workplace and/or industry documentation
 ask questions of independent parties before and during design (refer to Templates)
You can involve one or a number of independent parties at this developmental stage, when
gathering information about the industry.
Validation during assessment
May involve assessors and independent parties working in teams or panels or with a ‘lead’ assessor,
concentrating on:
 The assessment process and the role of the assessor
 The actual performance of the student
Validation after assessment
Occurs when reviewing or comparing assessment tools, evidence and moderating assessment
decisions, concentrating on:







the effectiveness of the student assessment tasks and process
the effectiveness of the assessors’ documents
questions for independent parties (refer to Sample Templates)
the validity of the evidence collected (moderation)
the standard of performance achieved (moderation)
the consistency and accuracy of decision making (moderation)
the selection of a random sample of student assessments where a decision has been made
(moderation)
 reporting and recording
December 2013
Page 6
Guide to Planning and Implementing Independent Validation and Moderation of Assessment
Section 5:
Steps in the validation/moderation process
The size and staffing arrangements within an RTO will guide how the independent validation and
moderation process is managed and implemented.
Your RTO may be large enough that there is a person responsible for the managing and planning of
the schedule and who oversees the process. This person would undertake the tasks in Table 1 which
assumes a mix of planning and management responsibilities.
If your RTO is small, the staff member overseeing validation and moderation may undertake a
combination of the tasks in Tables 1, 2 and 3.
What do you need to do?
Tables 1, 2 and 3 on the following pages describe the steps required to plan, conduct and review the
validation and moderation activities.
Section 6 of this document provides these tables as checklists (Templates 3, 4 and 5 respectively)
which may assist you through the process.
December 2013
Page 7
Guide to Planning and Implementing Independent Validation and Moderation of Assessment
Table 1: Steps for Planning the Validation/Moderation Process
Steps
Responsibility
Comments
Sample
Template
(will depend on
structure of the
RTO)
1
Establish assessment
validation/moderation
schedule for the year
Training Manager,
Training
Coordinator
The schedule should be
developed for a 12 month
period and based on a ‘risk
assessment’ approach.
Ensure the schedule includes
the random selection of
students assessments across
the units/qualifications
1, 3
2
Identify independent
parties to be involved in the
process
Management,
Assessors,
Training Manager,
Training
coordinator
Selection would depend on
what units/qualification are
being validated/moderated.
Internal and independent
parties must be involved and
listed in the schedule.
This may involve students,
trainers/assessors, industry
representatives, professional
association representatives,
assessor networks, small and
large businesses or other
relevant independent parties.
1, 3
3
Plan the activities using
schedule in step 1
Training Manager.
Training
Coordinator
Ensure it includes random
selection of evidence for
moderation
2, 4
4
Inform relevant
independent parties of
validation/moderation
requirements
Training Manager,
Training
Coordinator
This would involve invitations
being extended to all relevant
independent parties. Include
an explanation about their
roles and responsibilities.
2, 4, 13
5
Review information already
on file that has been
gathered by others over
time.
Training Manager,
Training
Coordinator
This is the information
provided by independent
parties that have answered
the questions in an
ongoing/systematic way.
11, 12
6
Ensure activities are
undertaken and
documented
Training Manager
Training
Coordinator
Documentation must be
completed, signed and dated
by all in attendance using the
2, 6-8
(relevant
ones)
December 2013
Page 8
Guide to Planning and Implementing Independent Validation and Moderation of Assessment
Steps
Responsibility
Comments
(will depend on
structure of the
RTO)
Sample
Template
Lead person (chair) organisational templates
supplied
7
Review process and
outcomes
Training Manager
Training
Coordinator
This could be undertaken with
lead person (chair or
independent participants)
5, 14
8
Ensure recommendations
are reviewed and
implemented where
appropriate
Training Manager
Training
Coordinator
Completed templates have a
follow up section which must
be completed and checked to
ensure there is a responsible
person
All
relevant
templates
9
Review schedule
Training Manager,
Training
Coordinator
Schedule should be reviewed
at least yearly taking into
account validation and
moderation activities and
results
1
December 2013
Page 9
Guide to Planning and Implementing Independent Validation and Moderation of Assessment
Table 2:
Steps for conducting/participating in validation and moderation activities
Steps
Responsibility
Comments
Sample
Template
(will depend on
the structure of
the RTO)
1
Review plan developed
from schedule and
confirm appropriateness
of planned activity
Internal and
independent
participants
This should be completed before
the validation activity is conducted
to ensure relevance.
2
2
Book venue and/or
electronic equipment,
arrange resources.
Identify lead (chair)
person for the validation
activity
Training
Manager,
Training
Coordinator
This could involve some
participants meeting face-to-face,
whilst others use technology for
the same activity. Resources must
include training package, AQF
levels, relevant
legislation/regulations that may
impact, all assessment
documentation, all assessment
evidence.
9
3
Review assessment
documentation
forwarded by the RTO
Internal and
independent
participants
This ideally should be completed
before activity to save time
3, 4, 9
4
Ensure independent
parties are briefed
about their role
Lead person
(chair)
The lead person’s role is to ensure
the activity remains on track and
everyone has input into the
discussion. Review roles and
responsibilities of all participants.
Ensure Code of Practice and/or
Confidently agreement is in place
9, 10, 13
5
Ensure all
documentation is
available
Lead person
(chair)
This would include training
package, AQF levels, relevant
legislation/regulations, workplace
documents, all assessment tasks
and tools (validation), all student
evidence used to make the
assessment decision (moderation).
This also should include
information gathered in answer to
questions(ongoing/systematic)
3, 4, 9
6
Participate in activity
Internal and
independent
participants
This includes group agreement on
how to work together (see
attachment for further assistance)
and clarifying roles of participants
5-8
December 2013
Page 10
Guide to Planning and Implementing Independent Validation and Moderation of Assessment
Steps
Responsibility
Comments
Sample
Template
(will depend on
the structure of
the RTO)
7
Complete
documentation as
required
Lead person
(chair)
Documentation must be
completed, signed and dated by all
in attendance using the
organisational templates supplied.
2, 6-8
(relevant
ones)
8
Undertake modifications
as directed
Nominated
person
Follow the RTO’s continuous
improvement procedure
6, 7, 8
(whichever
is relevant)
December 2013
Page 11
Guide to Planning and Implementing Independent Validation and Moderation of Assessment
Table 3: Steps for reviewing validation/moderation activity
1
Steps
Who is
responsible
Comments
Sample
Template
Collect feedback from
participants regarding
activities
Lead person/chair
Open discussion on how
participants felt about the
activity
5 , 14
What was learnt from the
activity
2
Discuss how the next
validation may be
streamlined or improved
Internal and
independent
participants
This should be documented
as part of the RTOs
processes
5 , 14
3
Review results
Internal and
independent
participants
Open discussion
5 , 14
4
Provide feedback to RTO
Lead person/chair
Using RTOs documentation
and processes
RTOs
processes
December 2013
Page 12
Section 6:
Templates and Forms
December 2013
Page 13
Template 1 - Validation and Moderation Schedule
Period from:
To:
This schedule should be completed by the RTO on a yearly basis and must include independent parties involved in any activity
Training Package Title
and Code
Unit/s within this
training package to be
reviewed
Activity
(meeting,
teleconference, video
conference etc)
Proposed dates
Authorised Person’s Name:
Role:
Signature:
Date:
Review date:
Person to undertake review:
December 2013
Resources required
Parties required
(industry, other RTOs,
internal assessors etc)
Page 14
Template 2- Validation and Moderation Plan
This plan is to be completed by the RTO for each validation and moderation activity.
RTO Name and National ID:
Date:
Training Package Title and Code:
Unit/s within this
training package to be
reviewed
Focus of the activity
(assessment process,
tools evidence,
decision making)
Methods/approaches
to be used (meeting,
teleconference, video
conference etc)
Parties involved
(industry, other RTOs,
internal assessors etc)
Name of Person Responsible for Plan:
Role:
Signature:
Date:
Resources required
Documents to be used
to record the activity
Name of RTO Manager/coordinator:
Signature:
December 2013
Page 15
Template 3 - Steps for Planning the Validation/Moderation Process
This checklist can be used by the RTO to assist when planning the validation and moderation
process. It can be used as a guide and as evidence the process the process has been managed.
Refer to Table 1 in the Guide to Planning and Implementing Independent Validation and Moderation
of Assessment.
Steps
Has the following been completed?
1
Establish assessment
validation/moderation schedule for
the year
2
Identify independent parties to be
involved in the process
3
Plan the activities using schedule in
step 1
4
Inform relevant independent parties of
validation/moderation requirements
5
Review information already on file that
has been gathered by others over time
6
Ensure activities are undertaken and
documented
7
Review process and outcomes
8
Ensure recommendations are
reviewed and implemented where
appropriate
9
Review schedule
December 2013
Yes
No
Comments/actions to be taken
Page 16
Steps
Has the following been completed?
Yes
No
Comments/actions to be taken
Name and signature of person completing the checklist:
Date:
December 2013
Page 17
Template 4 - Steps for Conducting/Participating in Validation and Moderation activities
This checklist can be used to assist when conducting and participating in the validation and
moderation process. It can be used as a guide and as evidence of undertaking the process.
Refer to Table 2 in the Guide to Planning and Implementing Independent Assessment Validation and
Moderation.
Steps
Has the following been completed?
1
Review plan developed from schedule
and confirm appropriateness of
activity
2
Book venue and/or electronic
equipment, arrange resources.
Identify Lead Person(chair) for the
validation activity
3
Review assessment documentation
forwarded by the RTO
4
Ensure independent parties are
briefed about their role
5
Ensure all documentation is available
6
Participate in activity
7
Complete documentation as required
8
Undertake modifications as directed
Yes
No
Comments/actions to be taken
Name and signature of person completing the checklist:
Date:
December 2013
Page 18
Template 5 - Steps for reviewing validation/moderation activity
This checklist can be used to assist when reviewing in the validation and moderation activity. It can
be used as a guide and as evidence of undertaking the process. Steps 1-4 are to be undertaken as a
discussion and led by the ‘chair’ or the person ‘facilitating the activity’. It is not a ‘feedback form’, it
is to be used to generate the open feedback. Ensure the discussion is documented and reviewed.
Steps 5 and 6 relate to actions taken by the RTO after the validation activity to ensure
recommendations are implemented.
Refer to Table 3 in the Guide to Planning and Implementing Independent Assessment Validation and
Moderation.
Steps
Has the following been completed?
1
Collect feedback from participants
regarding activities
2
Discuss how the next validation may be
streamlined or improved
3
Review results
4
Provide feedback to RTO
5
Ensure recommendations are reviewed
and implemented where appropriate
6
Review schedule
Yes
No
Discussion/feedback
Name and signature of person completing the checklist:
Date:
December 2013
Page 19
Template 6 - Validation of Assessment Tools
Comments about this
template
Names of validation activity
participants and their role eg
external RTO and/or
assessor, industry and VET
person
Use this document when validating internally within your RTO
and/or externally with other RTOs and industry representatives
who have technical expertise in the areas covered by the unit/s. All
participants may be issued with this form but one person must be
responsible for completing the form for the RTO.
1.______________________________________________________
2. _____________________________________________________
3. _____________________________________________________
Date of activity
Unit/s of competency
List all assessment
documents reviewed in the
activity. Note all assessment
tasks, tools; assessor’s
documents must be
available.
List all support documents
referred to today
Recommendations
Explanation of recommendations:
Modification required
 YES  NO
Assessment Tools ready for
use
 YES  NO
Modifications:
Name of person who is to
complete modifications
Date modifications to be
completed by
December 2013
Page 20
Signatures:
1:
2:
3:
Modifications completed
Date /Signature of
authorising officer
December 2013
Page 21
Clearly describe in the action column, the adjustments that are required to the tools.
Principles of assessment
Criteria met?
Validity
Yes
Comments/ Actions
No
Tools assess ALL components of the unit
including knowledge, skills and critical
aspects?
Tools address the level of performance
required by unit and the AQF?
Tools address task skills, task
management skills, contingency
management skills and role environment
skills
Assessment tasks relate to real work
place contexts and situations?
Assessment tasks have been designed to
allow holistic and integrated assessment
of knowledge and skills?
The tools demonstrate a variety of
evidence over time and contexts?
Tools reflect the literacy and numeracy
requirements of the unit?
Reliability
Tools consistently measure against the
unit AQF level, and workplace context?
Instructions ensure consistent application
of the tools by assessor, workplace
supervisors (where applicable) and
candidates?
Tools achieve consistent results in
different situations and with different
candidates?
Assessment exemplars, model responses,
checklists etc have been prepared for use
by assessors?
There is sufficient space for the assessor
to write comments on the assessment
tools for future reference in decision
making
Flexibility
The tools allow for flexibility in the
assessment to meet the needs of the
candidate and workplaces?
December 2013
Page 22
Fairness
Tools meet the needs of the candidate in
various contexts?
Tools are free of bias?
Written information is worded
appropriately to the requirements of the
units of competency and the context for
assessment, including candidate needs?
The level of difficulty of the activity is
appropriate to the competency or
competencies and qualification being
assessed?
Assessment conditions are clearly
defined?
Rules of Evidence
Validity –
Tools cover all components of assessment
and consider the workplace context, and
candidate?
See also criteria above
SufficiencyTools allow sufficient evidence of skills
and knowledge to be demonstrated over
a period of time?
Authenticity –
Tools ensure that the evidence gathered
is the candidate’s own work?
Currency Tools allow candidate to demonstrate
currency industry / work based
competency?
Additional Questions
Can these tools be easily used by another
industry assessor?
Will these tools allow for consistent
decisions to be made?
Source: Adapted from: Assessment validation and continuous improvement workshop guide, DEC NSW, 2012
December 2013
Page 23
Template 7- Moderation of Assessment Evidence
When to use this template
Use this document when moderating internally within your RTO
and/or externally with other RTOs and industry representatives
with technical expertise in the areas covered by the unit/s. All
participants may be issued with this form but one person must be
responsible for completing the form for the RTO.
Moderation activity
participant names:
1.______________________________________________________
2. _____________________________________________________
3. _____________________________________________________
Date
Unit/s of competency
List evidence reviewed. All
candidates evidence must be
available during the activity
List all support documents
referred to today
Recommendations:
Explanation of recommendations:
Signatures:
1:.
2:
3:
December 2013
Page 24
Follow up
Date /Signature of
authorising officer
Is the evidence

Directly related to the units
of competence being
assessed

Based on the critical
aspects of evidence

Appropriate for the AQF
level

Sufficient to make a
decision about
competence

Current

Authentic

Based on realistic
workplace activities and
contexts

Sufficient to cover the full
range of knowledge and
skills needed to
demonstrate competence

Taken from more than one
source over a period of
time

Collected in a timely and
cost effective way
December 2013
Yes
No
Comments /Actions
Page 25
Is the evidence

Yes
No
Comments /Actions
Demonstrating task skills,
task management skills,
contingency management
skills and role environment
skills
Source: Adapted from Assessment validation and continuous improvement workshop guide, DEC NSW, 2012
December 2013
Page 26
Template 8 - Validation Assessment Process
When to use this template
Use this document when validating internally within your RTO and/or
externally with other RTOs and industry representatives with technical
expertise in the areas covered by the unit/s. All participants may be issued
with this form but one person must be responsible for completing the
form for the RTO.
Validation participant
names:
1.______________________________________________________
2. _____________________________________________________
3. _____________________________________________________
4. _____________________________________________________
5. _____________________________________________________
Date
Assessment processes
being validated
Choose an assessment process to work on, for example:
Policies, procedures and
guidelines that currently
govern this process:
List them
Recommendations:
Explanation of recommendations:
Signatures:
1.__________________________________________




Candidate briefing
Giving feedback to candidates
Collecting feedback from candidates
Conducting question and answer assessments with groups
2. __________________________________________
3. __________________________________________
4. __________________________________________
5. __________________________________________
December 2013
Page 27
Follow up:
Date /Signature of
authorising officer
Does the process
Yes
No
Comments /Actions
Clearly identify the steps involved
in the assessment process for
example in a flow chart
Provide plain English instructions
to assessor
Provide guidance to the assessor
on preparing the candidate
Provide guidance to the assessor
on collecting evidence
Provide guidance to the assessor
on making decisions
Provide guidance to the assessor
on reviewing the process
Provide advice on reasonable
adjustment in assessment
Provide guidance to the assessor
on providing feedback to the
candidate
Provide guidance to the assessor in
case of appeal
Allow for feedback to be provided
by candidates and /or clients
Allow for feedback to be provided
by assessors
Provide plain English instructions
to candidates and/or clients
December 2013
Page 28
Does the process
Yes
No
Comments /Actions
Allow for flexibility to be adapted
to suit specific candidates and/or
workplace needs
Allow for documentation of
agreement between assessor and
candidate and/or client
Allow for support mechanisms
Clearly document and disseminate
appeals processes
Provide assessors with
documentation to complete for
record keeping
Document confidentiality and
privacy processes
December 2013
Page 29
Template 9- Checklist of what to organise when planning the activities
The following template can be used to assist in preparing for the validation or moderation activity.
The checklist would be part of your Quality Processes.
Arrangements to make
Availability of participants- external and
internal
Yes No
Comments
Ensuring participants have the
knowledge/skills required
Venue booking
Catering if required
Any technology requirements
How recording/reporting of outcomes will
occur
Dissemination of validation and moderation
plan
Dissemination of documents to participants
before activity
Documents to organise
Assessment plans /process
Assessment documents ie student tasks,
assessors tools
Completed assessment evidence
Units of competency
Training package assessment guidelines
Qualification packaging rules
AQF descriptors
December 2013
Page 30
Source documents/materials such as candidate
feedback , supervisor feedback
Relevant regulation/legislation
Code of practice and/or confidentiality
agreement
Name of person completing this checklist:
Signature:
December 2013
Date:
Page 31
Template 10 - Confidentiality Agreement
This agreement can be used when conducting independent validation or as part of your validation
process.
All participants who are taking part in validation are requested to sign this agreement.
I agree to observe the principles of confidentiality and mutual trust and respect with
regard to the activity I am participating in.
This includes the assessment plan, process documentation, assessment tools and
evidence shared in this validation meeting. I will not use the information for any reason
not acceptable to the validation group and code of conduct for assessors. I will also
respect others who participate in this meeting and use open and clear communication.
RTO Name:
RTO Representative:
Qualification/units of competence:
Meeting date:
Location:
Name:
Organisation:
Email:
Phone:
Signature:
Name:
Organisation:
Email:
Phone:
Signature:
Name:
Organisation:
December 2013
Page 32
Email:
Phone:
Signature:
December 2013
Page 33
Template 11 – Questions for Independent Parties
The following questions can be asked by anyone within the RTO of independent stakeholders to
assist in gathering information in an ongoing manner to be used as part of the validation and
moderation process. Document the responses which can then be used for consultation and
validation and moderation.
Name of RTO Representative:
Date:
Name of Individual Interviewed:
Role:
Organisation:
Questions at various stages in the process
Before assessment
1.
2.
3.
4.
5.
6.
7.
8.
Is the assessment process consistent with your expectations?
Will the assessment practice fit in with your organisational activities and workplace culture?
Do the selected assessment methods and tools reflect current industry standards and practices?
Are there any areas you consider to be a risk in your work environment we need to be aware
of?
How do you consider a worker could show their competence?
Are the tasks relevant to your work environment?
Will the workplace supervisors able to complete the documentation easily?
Will the assessment process having a positive impact on your organisation?
During assessment
1. Is the assessment process meeting your expectations?
2. Is the assessment fitting in with your workplace activities and culture?
3. Is the workplace supervisor able to easily complete the documentation?
After assessment
1.
2.
3.
4.
5.
6.
7.
8.
9.
Were the tasks relevant to your work environment?
Does the evidence meet the standards you would expect in the industry?
Has the assessment process had a positive impact on your organisation?
Did the assessment practice fit in with your organisational activities and workplace culture?
Did the selected assessment methods and tools reflect current industry standards and
practices?
Were the workplace supervisors able to complete the documentation as required?
Would you employ these students?
What key areas of improvement have you identified?
What recommendations would you make?
December 2013
Page 34
Template 12- Sample Student Questions
The following questions can be asked to gain feedback from the student regarding the assessment.
Document the responses which can then be used for consultation and validation and moderation.
The questions can be asked at various stages and be contained in a number of documents already in
place within the RTO. The table below provides examples.
Question
Possible RTO document
where question can be
asked and have student sign
off
1. Were you informed of the assessment process?
Assessment cover sheet,
enrolment form, assessment
guidelines
2. Were you given clear written instructions about what to do in
the assessment?
Assessment cover sheet,
assessment task instructions
3. Were you given sufficient information about what was going to
be assessed and how and when you were going to be assessed?
Cover sheet
Student assessment guide
4. Were you informed of who the assessor was?
Cover sheet
Student assessment guide
5. Were you informed of any available support during the process?
Assessment guidelines
6. Do you think that the assessment process was fair and
equitable?
Feedback form
7. Was the assessment linked to the workplace?
Feedback form
8. Do you think that the person observing your performance was
skilled, fair and unbiased?
Feedback form
9. Do you think that the overall judgement about your
performance was accurate and fair?
Feedback form
10. Were you given adequate feedback about your performance
after the assessment?
Feedback form
11. Were you told about your right to an assessment appeal?
12. Were you given useful advice about further learning after the
assessment?
December 2013
Assessment guidelines
Enrolment form
Feedback form
Page 35
Template 13 - Roles and responsibilities of Internal and External Participants
The inclusion of independent parties during validation and moderation means that all should be
aware of their role and responsibilities. The table below assists in identifying this for internal and
external parties.
Internal participants
External participants
Could include program developers, assessors, RTO
relevant staff
Could include industry representatives, small and large
business, assessors from other RTOs, specific client or
any unbiased or impartial third-party














Plan and organise activities on an ongoing
basis
Interpret the unit/s of competency
Come to a common understanding of the
standard to be achieved so agreeing on
what a competent person would ‘look like’
Contribute to design of the assessment
process
Contribute to the development of the
assessment tools
Ask questions of external parties before
and during design
Share their knowledge of training and
assessment
Share their industry knowledge
Provide feedback to external parties during
the process
Contribute to feedback and discussion on
the effectiveness of the student
assessment tasks and process; the
assessors’ documents
Encourage participation of others
Contribute to discussion on the validity of
the evidence collected; the standard of
performance achieved; the consistency
and accuracy of decision making
Reporting and recording
December 2013











Come to a common understanding of the
standard to be achieved
Contribute to the development of the
assessment tools
Share their industry knowledge
Provide information of what would work in the
work environment
Discuss the standard required in the workplace
Describe workplace culture
Discuss implications for the workplace of the
assessment process
Provide feedback to external parties during the
process
Contribute to feedback and discussion on the
effectiveness of the student assessment tasks
and assessment process
Encourage participation of others
Contribute to discussion on the validity of the
evidence collected; the standard of
performance achieved; decision making
Reporting and recording
Page 36
Template 14- Questions for evaluation of validation activity
To ensure continuous improvement all validation and moderation activities should be reviewed and
the feedback used to guide further activity. This process should be ‘chairperson’ or facilitator led
with answers documented. All participants should be part of this review process.
1. Was participating in this activity what you thought it would be?
2. What are 2 key things you learnt from participating in this activity?
3. Would you have preferred to be more involved in the activity? If so how?
4. Can you provide two ways that the activity could be streamlined or improved?
5. Were the results of the activity what you anticipated? If not why not?
6. Would you agree to participate in further activities? If not why not?
December 2013
Page 37
Section 7
Code of Practice for Validation and Moderation
December 2013
Page 38
Code of Professional Practice for RTOs
and Independent Parties Participating in
Validation and Moderation
TABLE OF CONTENTS
About this Code .................................................................................................................................................... 41
1
Purpose ........................................................................................................................................................ 41
2.
Agreement .................................................................................................................................................... 41
2.1 Intention ................................................................................................................................................. 41
2.2 Overarching agreement ......................................................................................................................... 41
2.3 Principles of working together ............................................................................................................... 42
2.4 Approach .................................................................................................................................................. 4
2.5 Outcomes ................................................................................................................................................. 4
2.6 Definitions .............................................................................................................................................. 44
Validation................................................................................................................................................ 44
Moderation ............................................................................................................................................. 44
Independent parties ............................................................................................................................... 44
Continuous Improvement....................................................................................................................... 44
Assessment ............................................................................................................................................. 45
Assessment Requirements ..................................................................................................................... 45
Principles of Assessment ........................................................................................................................ 45
Rules of evidence .................................................................................................................................... 46
3.
References and Further reading ................................................................................................................... 46
Code of Professional Practice for Validation
About this Code
1. Purpose
The purpose of this professional code of practice for validation is to:




Set out principles and processes for RTOs and Independent Parties when participating in
validation and moderation activities.
Document an agreement between participating VET practitioners and independent
parties.
Ensure practitioners meet VET obligations as outlined in the VET Quality Framework.
Build a culture of continuous professional improvement.
Practitioners and Registered Training Organisations working with independent parties will be
required to agree to and fulfill the requirements of this code.
2. Agreement
This agreement is between participating RTOs, VET practitioners and independent parties
involved in the Independent Validation of Assessment Project.
2.1 Intention
Practitioners will be required to participate with the following intentions:



To improve the quality of their own professional practice.
To work to improve the standards of the VET sector as a whole by engaging in a process
of continuous improvement in assessment practices.
to improve the confidence of stakeholders in VET qualifications
2.2 Overarching agreement
Participating practitioners will collaborate with colleagues to:







Agree and commit to completing the planned actions; and also that if planned actions
are consistently not completed the practitioner may request, and/or be excluded.
Share assessment resources with others while adhering to confidentiality and privacy
principles.
Ensure they attend all meetings or, should that not be possible, to send an alternate
delegate from their organisation of similar professional stature and commitment. If this
is the case that person must be briefed fully so they can actively participate in the
process.
Adhere to principles of privacy and confidentiality
Be open, transparent and fair in their discussions with others
Strive to reach agreement or consensus with others
Listen to others point of view
December 2013
Page 41
Code of Professional Practice for Validation
2.3 Principles of working together
To make consensus possible all group members must:

agree this is a ‘safe environment’ where all parties have the opportunity to contribute to
the conversation and shares their thoughts

start by reaching agreement on what ‘competency’ means for each unit – ie. what
outcomes should the candidate demonstrate and to what level.

clarify roles - chair/ timekeeper, record keeper, reporter to whole group

listen actively – be sure you have heard each others’ comments and are not simply
impatiently waiting to speak yourself

summarise as you go - be able to paraphrase each others’ comments briefly and
straightforwardly before making another point or disagreeing with them

be open to others ideas - avoid taking a position, digging in and defending it – no matter
how ridiculous it becomes

be inclusive – take note of quiet members of the group, encourage them to have a say,
then show you have understood what they said, e.g. paraphrase it

do not assume silence = agreement, check everyone agrees before moving on

work with others to reduce or eliminate conflict
2.4 Approach
Assessor Partnerships
The validation and moderation process to be used is referred to as Consensus Approach.
The Consensus Approach involves working with a group of assessors (even just two) with a focus
on quality review and involves:

Utilising a collaborative approach to foster collegiality, mutual assistance and
confirmation. This is achieved by:
- Sharing and discussing each other’s assessment tools, processes and outcomes
- Helping each other through support and assistance to work through and resolve any
issues
- Comparing assessment tools and decisions made against the criteria outlined in
units of competency and other aspects of the Qualifications Framework.
December 2013
Page 42
Code of Professional Practice for Validation
2.5 Outcomes
Practitioners conducting validation and moderation will be required to contribute to continuous
review processes, where recommendations for improvements are made (including a justification for
the recommendation).
Resources produced from this validation project will be made publicly available. This material may
be freely used by Australian Training Organisations for educational purposes. You may not make it
available for hire or re-sale with financial gain.
Practitioner agreement:
I agree to adhere to the principles and processes of validation and moderation as outlined in this
Code of Practice. I also understand that if I consistently ignore my commitments to this agreement I
may be forfeiting my participation in the project.
Name:
Organisation:
Position:
Signature:
Date:
December 2013
Page 43
Code of Professional Practice for Validation
2.6 Definitions
Validation
“… a quality review process. It involves checking that the assessment tool produced valid, reliable,
sufficient, current and authentic evidence to enable reasonable judgements to be made as to
whether the requirements of the relevant aspects of the Training Package or accredited course had
been met. It includes reviewing and making recommendations for future improvements to the
assessment tool, process and/or outcomes”.
Ref: Adapted from NQC A Code of Practice for Validation and Moderation, 2009.
moderation
“……process of bringing assessment judgements and standards into alignment. It is a process that
ensures the same standards are applied to all assessment results within the same Unit(s) of
Competency. It is an active process in the sense that adjustments to assessor judgements are made
to overcome differences in the difficulty of the tool and/or the severity of judgements.”
Ref: Adapted from NQC A Code of Practice for Validation and Moderation, 2009.
Independent parties
... are those independent to the RTO therefore not employed or contracted by the RTO. The
independent parties could include industry representatives, small and large business, assessors from
other RTOs or any unbiased or impartial third-party. Feedback from students can contribute to the
validation and moderation process by providing feedback on the assessment tasks, assessment
process and decision making (see attachment). The importance is ensuring the independent
representatives have the knowledge, skills and experience in the area themselves.
Continuous Improvement
“Continuous improvement means a planned and ongoing process that enables an RTO to
systematically review and improve its policies, procedures, products and services in order to
generate better outcomes for clients and to meet changing needs. It allows an RTO to constantly
review its performance against the VET Quality Framework and to plan ongoing improvements to its
performance. Continuous improvement involves collecting, analysing and acting on relevant
information collected from clients and other interested parties, including the RTO’s staff” (Standards
for NVR Registered Training Organisations, 2011, SNR 3).
December 2013
Page 44
Code of Professional Practice for Validation
Assessment
“Means the process of collecting evidence and making judgements on whether competency has
been achieved, to confirm that an individual can perform to the standard expected in the workplace,
as expressed by the relevant endorsed industry/enterprise competency standards of a Training
Package or by the learning outcomes of a VET accredited course” (Standards for NVR Registered
Training Organisations, 2011, SNR 3).
Assessment Requirements
“Assessment requirements are the endorsed component of a Training Package. Assessment
requirements set out the industry’s approach to valid, reliable, flexible and fair assessment”
(Standards for NVR Registered Training Organisations, 2011, SNR 3).
Principles of Assessment
“Principles of assessment are required to ensure quality outcomes. Assessments should be fair,
flexible, valid and reliable as follows:
a) Fairness: Fairness requires consideration of the individual candidate’s needs and characteristics,
and any reasonable adjustments that need to be applied to take account of them. It requires
clear communication between the assessor and the candidate to ensure that the candidate is
fully informed about, understands, and is able to participate in, the assessment process, and
agrees that the process is appropriate. It also includes an opportunity for the person being
assessed to challenge the result of the assessment and to be reassessed if necessary.
b) Flexible: To be flexible, assessment should reflect the candidate’s needs; provide for recognition
of competencies no matter how, where or when they have been acquired; draw on a range of
methods appropriate to the context, competency and the candidate; and, support continuous
competency development.
c) Validity: or relevance-In simple terms, the assessment must assess what it proposes to assess.
it requires assessment against specified criteria; a sufficient range of the performance of the
person being assessed; assessors having demonstrated competence in the area they are
assessing; using multiple approaches to assessment
d) Reliability: or consistency-In general, reliability is an estimate of how accurate or precise the
task is as a measurement instrument. Reliability is concerned with how much error is included
in the evidence” (adapted from Standards for NVR Registered Training Organisations, 2011, SNR
3).
December 2013
Page 45
Code of Professional Practice for Validation
Rules of evidence
Rules of evidence are closely related to the principles of assessment and provide guidance on the
collection of evidence to ensure that it is valid, sufficient, authentic and current as follows:
a) Validity: see Principles of assessment. is the evidence sufficiently relevant to the competencies
being assessed.
b) Sufficiency: Sufficiency relates to the quality and quantity of evidence assessed. It requires
collection of enough appropriate evidence to ensure that all aspects of competency have been
satisfied and that competency can be demonstrated repeatedly. Supplementary sources of
evidence may be necessary. The specific evidence requirements of each unit of competency
provide advice on sufficiency.
c) Authenticity: To accept evidence as authentic, an assessor must be assured that the evidence
presented for assessment is the candidate’s own work.
d) Currency: Currency relates to the age of the evidence presented by candidates to demonstrate
that they are still competent. Competency requires demonstration of current performance, so
the evidence must be from either the present or the very recent past.
3. References and Further reading
1.
2.
3.
NQC, A Code of Practice for Validation and Moderation, 2009.
Standards for NVR Registered Training Organisations, 2011.
http://www.comlaw.gov.au/Details/F2011L01356/ExplanatoryStatement/Text
Australian Qualifications Framework (AQF) http://www.aqf.edu.au/
December 2013
Page 46
Download