Islamic Republic of Pakistan
Manual No. 5:
Developing NVQF
Moderation System
National Vocational Quali cations Framework
NVQF
To deliver a skilled and quali ed workforce in Pakistan
Version 1
January 2016
National Vocational and Technical Training Commission (NAVTTC)
IO
N
NAVTTC
TR
COMMISS
NAL VOCAT
IO
TIO
N
NA
TECHNICAL
NG
NI
AI
&
AL
Manual
On
Developing NVQF Moderation System
Version 1
January 2016
Table of Contents
Abbreviations used
Introduction
1
Principles of Moderation
2
Process and Guidelines for Moderation
3
Processes and Guidelines for Moderation Meetings
4
Post-Assessment Moderation Judgement
5
Appendix-1: Checklist for Moderation of Assessments
Appendix-2: Moderation Activity Summary Report
Appendix-3: Assessor Self Assessment Checklist
6
8
9
Figures
Figure 1: Quality Assurance Process in NVQF
1
Abbreviations used
AT
BTEs
CB
CBT
CBET
CCIs
CQ
CS
CUs
DACUM
EQF
IAGs
IBCC
ISCED
LDs
M&E
NAVTTC
NCRC
NOSS
NSS
NSIS
NTB
NVC
PNQF
PNVQF
PVTC
QCC
QR
QSC
QSSC
RCC
RPL
SAARC
SDA
SDCs
Apprenticeship Training
Boards of Technical Education
Competency-Based
Competency-Based Training
Competency-Based Education and Training
Chambers of Commerce and Industries
Certi cation of Quali cation
Competency Standard
Competency Units
Developing A Curriculum
European Quali cation Framework
Industry Advisory Groups
Inter Board Committee of Chairmen
International Standard Classi cation of Education
Level Descriptors
Monitoring and Evaluation
National Vocational & Technical Training Commission
National Curriculum Review Committee
National Occupational Skill Standards
National Skill Strategy
National Skill Information System
National Training Bureau
National Vocational Certi cate
Pakistan National Quali cation Framework
Pakistan National Vocational Quali cation Framework
Punjab Vocational Training Council
Quali cation Curriculum Committee
Quali cation Register
Quali cation Standard Committee
Quali cation Standard Sub-Committee
Recognition of Current Competencies
Recognition of Prior Learning
South Asian Association for Regional Cooperation
Skill Development Analysis
Skill Development Councils
1
Introduction
This manual describes the processes and provides guidelines for moderation of assessments of NVQF quali cations. Moderation
is the quality assurance process carried out to ensure assessment is fair, valid and at the national standard, and that the
assessors are making consistent judgements about learner performance. Moderation also helps to improve assessment
practices.
Moderation is a quality management process that is designed to ensure there is uniform interpretation and application of
assessment criteria to ensure assessment judgments are consistent. The function of moderation activities will be to:
Ÿ
Verify that assessments are fair, valid and consistent
Ÿ
Identify where there is any need for redesigning assessment activities, or for reassessing applicants
Ÿ
Adjust interpretations of the criteria for the future
Ÿ
Provide a mechanism for handling appealed assessment
Ÿ
Provide feedback on the quality of the criteria
Ÿ
Provide feedback to assessors on the quality of their judgements
The National Vocational Quali cation Framework (NVQF) recognises assessment as the key to achieve NVQF quali cations
through whatever learning pathway an individual may come from. Hence the Moderation System will ensure the assessment
tasks and judgements made for any competency are the same as or equivalent to any others that are being administered
anywhere in the national system for the same competencies.
Awarding Bodies have the responsibility to organise, manage and report on moderation activities to NAVTTC.
Figure 1: Quality Assurance Process in NVQF
Developing NVQF Moderation System
1
2
Principles of Moderation
The principles of moderation are sufficiency, validity, reliability, fairness and consistency:
Sufficient evidence
a) Assessors must collect enough evidence to be con dent that the candidate has met all the requirements of the
standard. The assessor needs sufficient evidence to ensure that the candidate can repeat the required performance
consistently.
Valid evidence
a) Evidence must match the requirements of the standard being assessed. If the candidate is required to know certain
information, it is not valid to consider their ability to write well or speak con dently as part of the assessment. It is the
knowledge of the principles that is being assessed.
b) The method of assessment must also match what is being assessed, that is, assessing knowledge with a knowledge
test; assessing performance with a skill test and action methods.
Reliable evidence
a) Evidence must consistently measure what it is supposed to measure. Highly reliable assessment methods should be
designed so as to eliminate errors and can be used repeatedly in an unchanging situation to produce constant results.
Fair evidence
a) Candidates may be assessed against the same criteria in a variety of ways, but each must be treated fairly. They should
not be hindered or assisted by factors irrelevant to the performance in question.
b) The time provided for the assessment should re ect the time the activity could take in a normal working environment.
c) Performance should not be constrained by unfamiliar language or conditions, or poorly expressed instructions.
d) Knowledge or understanding should not be masked by the demands of extended writing. For example, if a candidate
is required to demonstrate a task, it would be inappropriate to require them to write an essay about the task.
e) Understanding of processes should not be hindered by lack of resources. For example, it can be appropriate for notes,
texts or manuals to be made available, as these are normally available in the workplace.
f) Legitimate responses that differ from the assessor's expectation but match the evidence required should be accepted.
g) Special needs should be accommodated where appropriate; however, disabilities that prevent candidates from
performing as speci ed in standards should not be compensated for.
h) Assessment should be free from bias arising from gender, cultural and ethnic factors. The assessor's own behaviour
can be critical here.
Consistent evidence
a) Assessors must make consistent decisions about candidates' competence. Another assessor considering the same
evidence should make the same assessment decision based on similar evidence at different times, on different
occasions.
2
Developing NVQF Moderation System
3
Processes and Guidelines for
Moderation
a) Moderation in the PNVQF will be post-assessment moderation, meaning it is undertaken after assessment using
nationally validated and noti ed evidence guides has been concluded. Post-assessment moderation ensures that the
assessment that took place and the evidence collected on which the judgement was made met the principles of
moderation, the requirements of the standard and the expectations of the industry.
b) An Awarding Body will develop an annual or quarterly Moderation Plan based on reports from the National Skills
Information System (NSIS) to moderate all competency standard assessments for each NVQF quali cation, conducted
by registered assessors, over a period of at least three years.
c) The reports from NSIS will include the following data:
n
Competency standards assessed
n
Name and registration number of assessor associated with those competency standards
n
Dates assessed
n
Assessment result – competent, not-yet-competent.
d) Based on the Moderation Plan, the Awarding Body will establish a schedule of moderation meetings at local, regional
and provincial level to meet the overall requirement in 2. B) above.
e) At planned intervals, the Awarding Body will request assessors of competency standards identi ed in the Moderation
Plan to attend a moderation meeting. The request will identify the competency standards for moderation and the
assessment evidence to be brought to the meeting, the date, venue and meeting timeframes. Random inspections of
assessment processes conducted by Registered Assessors by QABs, TEVTAs, NAVTTC.
f) The Awarding Body will appoint a suitably quali ed person to manage moderation activities as Moderator. They will
act as Moderation Meeting Chairperson and:
n
Chairs national moderation meeting
n
Provides professional guidance and support to participants in moderation activities
n
Ensures NSIS is updated with a record of competency standards and assessors moderated
n
Completes administrative functions relating to the national aspects of the moderation system
n
Ensures moderation activities are completed professionally
n
Completes documentation relating to moderation panel activities, and reports on moderation issues, including:
¡
Outcomes of moderation activities
¡
Issues arising from moderation activities
¡
New developments, requirements or recommendations
n
Distributes the moderation report (see Appendix-2 Moderation Activity Summary Report)
Developing NVQF Moderation System
3
4
Processes and Guidelines for
Moderation Meetings
a) The Assessor will be requested to bring to the moderation meeting:
n
All evidence and documentation for one example of assessment for which competency was granted at the rst
assessment opportunity.
n
All evidence and documentation for one example of assessment for which competency was granted at the rst
re-assessment.
n
All evidence and documentation for at least one example of an assessment that resulted in a “not yet
competent” decision being made.
b) The assessors present under the guidance of the Moderation Meeting Chairperson will consider, discuss and agree on
the standard of evidence required for each competency standard under moderation.
c) Assessors will use Checklist for Moderation (see Appendix-1) to record their discussions during the moderation
meeting.
d) The assessors present will carry out the moderation of assessments in accordance with the moderation guidelines and
consider presented assessment evidence and supporting documentation to ensure that assessment was:
n
Fair – the assessment did not unfairly advantage or disadvantage anyone
n
Constructive – included clear and constructive feedback to the candidate and comments about their
performance
n
Recorded – records of the evidence and judgement are accurate and complete
n
Sufficient – there was enough evidence to make a judgement
n
Consistent – another assessor would make the same judgement on the same or similar evidence
n
Current – the evidence is up-to-date
n
Authentic – it was the candidate’s own work
e) They will reach a consensus on the standard of evidence required in the future assessments undertaken for that
competency standard based on the principles listed above.
f) Every assessor will participate in the discussions and standard-setting activities.
g) Every assessor present will complete a Checklist for Moderation for each competency standard considered.
h) The Moderation Meeting Chairperson will complete their report of the meeting's agreements within 5 working days
of the completion of the meeting date.
i) The completed Moderation Meeting Report will be presented to the relevant Awarding Body.
j) The Awarding Body will ensure all registered assessors receive the Moderation Meeting report within 5 working days
of the time of presentation.
4
Developing NVQF Moderation System
5
Post-Assessment Moderation
Judgement
Post-assessment moderation judgements to be determined by those present in the moderation meeting consist of two
characteristics: the judgement and the associated recommendations or statements.
The post-assessment moderation judgements are one of:
a) Meets the standard, with the possible recommendations being:
n
Approved with no recommendations:
¡
Assessment process was fair, constructive and correctly recorded
¡
Evidence was relevant, sufficient, current, accurate and authentic
¡
Assessor's judgement re ects the evidence and meets the competency standard requirements
n
Approved with recommendations or feedback for future Assessor practice:
¡
Can holistically see that the candidate has achieved the standard required but evidence and/or records are
unclear or incomplete
¡
Given professional expertise, the assessor can make assumptions sufficient to make judgement
¡
Candidate may have clari ed understanding in discussion with Assessor. Note for future use: to be identi ed
as such in evidence guide
b) Does not meet the standard, with the following statements:
n
Supplied evidence does not meet the evidence requirements of the competency standard
n
Evidence and/or records are unclear, inconsistent, not relevant to industry, or do not appear to be authentic
(select those that apply)
Developing NVQF Moderation System
5
Appendix-1
Checklist for Moderation of Assessments
Competency Standard Code and Title:
_______________________________________________________________________
6
Developing NVQF Moderation System
Developing NVQF Moderation System
7
Appendix-2
Moderation Activity Summary Report
8
Developing NVQF Moderation System
Appendix-2
Assessor Self Assessment Checklist
Developing NVQF Moderation System
9
National Vocational & Technical Training Commission (NAVTTC)
5th Floor, Evacuee Trust Complex, Sector F-5/1, Islamabad
Tel: +92 51 904404, Fax: +92 51 904404, Email: info@navttc.org
Website: www.navttc.org