Consistency of Assessment
(Validation)
Webinar – Part 1
Renae Guthridge
WA Training Institute (WATI)
What is your understanding of
validation?
Describe an assessment validation
process in which you have been
involved.
– What was the purpose?
– Who was involved?
– What was validated?
– Were any problems identified?
– How were the findings reported and
followed-up?
What is assessment validation all
about?
• What are the purposes of assessment
validation?
Contribute to RTO
quality assurance
Evaluate the
quality of
assessment
tools
Quality review of the
assessment process
To ensure our assessment
judgements are
consistent
Determine whether
different assessors
interpret the same
evidence similarly
Demonstrate
compliance against
Standards for RTOs
Increase assessor
confidence
PURPOSES
Improve assessment
practices
Determine whether
assessment
decisions reflect the
principles of
assessment and
rules of evidence.
Address any identified
area of risk in assessment
practice and quality
Provide professional
development
Ensure assessments
meet the evidence
requirements of the
competency
standards or other
assessment
benchmarks
Determine whether
different assessors
using the same tools
collect the same
types and levels of
evidence
What is assessment validation all
about?
• What kinds, types and methods of
assessment documentation do we need
to validate?
Learning based
assessment
Direct
RPL
Assessment Plans
Indirect
Mapping Documents
Marking Guide
KINDS, TYPES &
METHODS OF
ASSESSMENT
DOCS
Instructions for
Assessor
Observation; Q&A;
Case Studies;
Portfolios; Projects…
Assessment Strategies
Supplementary
Instruments
Instructions for
Candidate
What is assessment validation all
about?
• Who wants assurance of quality?
Industry /
Employers
Technical / Subject
Matter Experts
Registering / Course
Accreditation Bodies
WHO
WANTS
QUALITY?
Assessors /
Trainers
Learners / Candidates
What is assessment validation all
about?
• What are our benchmarks of quality?
Standards
for RTOs
Training Packages /
Accredited Courses
Rules of Evidence
Principles of
Assessment
BENCHMARKS
OF QUALITY
AQF
TAE10 Training and
Education Training
Package…assessment
units.
What is assessment validation all
about?
• What aspects of the assessment process
need to be validated?
Planning assessment
processes (industry
consultation, links
with delivery, TP/AC
requirements,
recognition of client
needs)
Evidence gathering
processes and tools (onthe-job, workplace,
simulation)
Management of assessors /
evidence gatherers (selection,
supervision, professional
development)
Processes for the review
of assessment services
(validation and
continuous
improvement)
ASPECTS TO
VALIDATE
Judgement making
procedures
Delivery of
assessment services
(modes, plans,
resources)
Support for learners /
candidates
(Information,
feedback, flexibility,
reasonable
adjustment, appeals)
What is assessment validation all
about?
• How do we assess quality and make
improvements?
Assessors working in
collaboration to
review, compare and
evaluate assessment
processes and
outcomes
Can occur before, during
and after assessment
Foxes and Vixen
Validating methods, tools and
interpretation of evidence
Verifying consistency of
evidence and
judgements made
against benchmark
ASSESSING
QUALITY
It’s systematically
planned and
implemented
Foxes and a Vixen
Four
Very
Rampant
Foxes
Principles of Assessment
A
Satisfied
Cunning
Vixen
Rules of Evidence
Foxes and a Vixen
Flexibility
Authenticity
Validity
Sufficiency
Reliability
Currency
Validity
Fairness
Principles of Assessment
Rules of Evidence
Foxes and a Vixen
Flexibility
Foxes and a Vixen
Validity
Foxes and a Vixen
Reliability
Foxes and a Vixen
Fairness
Foxes and a Vixen
Authenticity
Foxes and a Vixen
Sufficiency
Foxes and a Vixen
Currency
Foxes and a Vixen
Validity
Standards for RTOs 2015
What do you know?
What is different to AQTF / VET Quality Framework?
Standards for RTOs 2015
Validation is the quality review of the assessment process.
Validation involves checking that the assessment tool/s produce/s valid,
reliable, sufficient, current and authentic evidence to enable reasonable
judgements to be made as to whether the requirements of the training
package or VET accredited courses are met.
It includes reviewing a statistically valid sample of the assessments and
making recommendations for future improvements to the assessment tool,
process and/or outcomes and acting upon such recommendations.
Standards for RTOs 2015
Standard 1 – The RTO’s training and assessment strategies and practices
are responsive to industry and learner needs and meet the requirements
of training packages and accredited courses
• comprehensive plan of systematic validation
• risk-based approach to developing the plan
• Trainers and assessors involved in training and assessment may also
participate in the validation process
1.9. The RTO implements a plan for ongoing systematic validation…for
each training product on the RTO’s scope of registration:
a)
when assessment validation will occur;
b)
which training products will be the focus of the validation;
c)
who will lead and participate in validation activities; and
d)
how the outcomes…will be documented and acted upon.
Standards for RTOs 2015
1.10. …each training product is validated at least once every five years,
with at least 50% of products validated within the first three years of each
five year cycle, taking into account… risks…on the RTO’s scope…including
those risks identified by the VET Regulator.
1.11. …systematic validation…is undertaken by one or more persons who
are not directly involved in the particular instance of delivery and
assessment of the training product being validated, and who collectively
have:
a)
vocational competencies and current industry skills relevant to
the assessment being validated;
b)
current knowledge and skills in vocational teaching and learning; and
c)
the training and assessment qualification or assessor skill set referred
to in Item 1 or 3 of Schedule 1.
Industry experts may be involved in validation to ensure there is the
combination of expertise set out in (a) to (c) above.
Standards for RTOs 2015
Independent validation of training and assessment qualifications
1.25. From 1 January 2016, to deliver any AQF qualification or assessor skill
set from the Training and Education Training Package (or its successor), the
RTO must have undergone an independent validation…
Independent validation means the validation is carried out by a validator/s who:
a) are not employed or subcontracted by the RTO to provide training and
assessment; and
b) have no other involvement or interest in the operations of the RTO.
Standard 2. The operations of the RTO are quality assured.
2.2. The RTO:
b) systematically evaluates and uses the outcomes of the evaluations to
continually improve the RTO’s training and assessment strategies and
practices. Evaluation information includes but is not limited… validation
outcomes...
Standards for RTOs 2015
What do you think?
What impact is this going to have on consistency of assessment?
What will need to change in your RTO?
RQI - 321
Write down three things you recall about
today’s webinar.
Write down two questions you have about
today’s webinar.
Write down one interesting point about today’s
webinar.
Tomorrow…Part 2…
Tips and techniques for:
• Meeting principles of assessment
• Ensuring rules of evidence are met
• Developing criteria for reviewing, comparing
and evaluating assessment processes
All pictures in these slides were taken from
http://search.creativecommons.org/