Welcome
To The HN & Vocational
Senior External Verifier Event
August 29th 2013
Agenda
Welcome & Overview
Matthew
Selection process for this session
Alastair
New Approach Forms and Guidance feedback
David/Veronica
BREAK
11am
New online reporting system
Juliette
Report system familiarisation session
Workshops
LUNCH
12.30pm
Updated guidance - summary of changes
David
Qualification Verifier Update
Juliette
Appointee Update
Mark Hood
Questions and close
9.30am
3.00pm
QA Past, Present and Future….
And then……………
Your Role
‘A Senior External Verifier provides leadership and
support to a team of external verifiers through the
processes in accordance with SQA policy and
instruction.’
Alastair Morton
Operations Planning Manager
(HN and Vocational Qualifications)
Topics
Review of Verification Activity 2012 – 13
College Review
Future of Reporting
Comments:
75 Holds reported (1st Aug 2012 – 31st July 2013)
54% of Allocated Events were Successful
38% of Events were updated as Not Running vs 36% Session 2011 - 12
Includes visits conducted under new approach
Profile excludes China and International Centres
How SQA plans to tackle this
Colleges
 Provided Verification Plans to all other Scotland’s Colleges
 Process worked well with 10 partner colleges over last 2 sessions
 Colleges highlighting Not Running – internally select for
accreditation purposes – SQA will set to Not Running
 First time this process has been considered
ETPs




Reviewed entry patterns for all approved SVQs over last 2 years
Allowed SQA to identify areas of candidate activity
Also allowed SQA to target individual Work Assessed units
Will select -> de-select as per College process
Longer Term
 IQA project – significant expenditure in designing complete system
Profile of Verification
Activity
Comment:
150 more Successful events vs session 2011-12 (2112 vs 1962)
Includes visits conducted under new approach
Profile excludes China and International Centres.
Comparing 2012-13 vs 2011-12:
Type
2012-13
2011-12
Diff
SVQ
963
1049
-86
HN G/U
378
376
2
HN
358
357
1
Impact of Verification Profile
• Resource impact on Centres
• Resource impact on Appointees
• Missing assessment windows – (e.g. Semester 1)
•
•
•
•
•
Availability of candidate evidence at Centres may drive part of this profile
Moving away from single allocation period -> quarterly allocations
React to changing developments at centre / across subject area / industry
Allocations will be issued throughout the session -> ongoing verification
Initial concentration on ETPs – assessment often throughout year
Scotland’s College Update
Scotland’s Colleges Update
Verification Planning
•
•
•
•
•
All Scotland’s Colleges – met / discussed finalised Verification plan
Real focus on Entry information
Discussed verification areas which have over / not verified
Plans are agreed in partnership
Swing from HN qualification to NCs
College Mergers
• Number of Colleges in process of merging
• Sensitive to the transitional period, particularly around availability of
candidate evidence across sites
• Majority of merging centres have yet to finalise single SEED number
and will be operating under original numbers for next 6 – 12 months
Challenges of Changing Quality
Assurance Approaches in Merging
Colleges
Scenario 2
Scenario 1
New
New
New
New
New
Old
New
Old
Scenario 3
New
New
Old
Scenario 4
Old
New
New
Old
College Feedback
Some Feedback
sessions could be
more
comprehensive
Verifiers using a
College email
address and not a
private one….
More Comments
need to be provided
against Criteria
Verifiers are not
liaising with the
SQA Coordinator
Visit Plans are not
being provided
ahead of visit.
Visit plans are
issued less than 2
weeks before visit
Sampling needs
to be more
widespread
Some Verifiers are not
using standard email
templates for
contacting / planning
HN and VQ Approach to Quality Assurance Walkthrough
Preparing Planning Verification
Prepare
EV+
Send Visit
Plan Docs
Request
Information
Accept
Carrying Out
Select
Sample
Contact
Prepare
for Visit
Agree
Agenda
General
Points
Completing
Report
& Follow-Up
Follow-up
Flowchart
Communicating
Decision
Plan
Feedback
Complete
Report
Reporting
Flowchart
Explain
Decision
Conduct
Sampling
Judge
sufficiency
Planning
Accepting Centre Allocation on EV+
Verification allocations will only be provided via EV+ system
Do’s
• Accept allocation within
14 days
Accept
You will receive
notification of
allocation from EV+ via
email.
Decline
•
•
•
•
If you decline the allocated visit, please provide a reason
If you do not accept your allocation within 14 days, an
automatic reminder email will be sent by EV+
SQA may re-allocate to another Verifier if no response is
received within this timescale
Please highlight any conflict of interest to QA Logistics Team
• If you decline a visit,
please provide a reason
• Once you have
registered, please
remember keep your
password safe. SQA will
not provide you with a
password.
Don’ts
Communicating
Decision
Plan to deliver balanced feedback
 Communicate verification decisions on criteria
compliance and overall risk rating.
 Highlight positive aspects of visit, including
good practice.
 Think clearly – what really is Good Practice ?
 No specific guidance on length of time
required for feedback.
Do’s
• Consider confidentiality
and location of feedback
session
• Give balanced feedback
• Recognise centre’s
efforts where sufficient
evidence provided.
• Take time out between
making the decision and
giving feedback
Don’ts
• Focus solely on Action
Points
Criteria Reporting –
The Way ahead
Evolution of Verification Outcome Reporting
Pre 2010
Activity
SEV Report
EV8A Reports
2010-11
Activity
2012-Now
Activity
EV8A Reports
New Verification
Reports
Count of Holds
Senior Review
SEV Report
Count of Holds
Senior Review
SEV Report
Detailed Analysis
Detailed Analysis
The Way Forward…
Activity
Database (QAMS)
ONLINE WEB
FORMATTED
REPORT
Benefits from this session:
•
•
•
•
•
•
Real time information available – easy to extract and disseminate
Understand “pockets” of non-compliance quickly
Criteria compliance reporting supported by narrative
Provide information on more frequent basis
Share with Centres / SQA teams and Appointees to highlight, understand and fix
Explore use of Social Media
Example
Activity
Database
Centres
Information
via Profiles
Appointees
SQA teams
Contains profile of
criteria (R / A / G)
and Good Practice
Awareness and
analysis through
number of views
/ reports
Highlight and
understand
Non Compliance
pattern
Implement and
Communicate
Fix
David Pirnie
Lead Verifier – HN & VQ
Results of Survey Questionnaire
June 2013
Centres
Qualification Verifiers
 Website is easy to follow - 93%
Clear flowcharts and procedures - 100%
 Guidance informs what to expect Understand the type of evidence
from a Qualification Verification
required – 74%
 Visit – 75%
Guidance is effective - 81%
 Qualification Verification forms
and templates are clear – 64%
Qualification Verification forms
and templates are clear – 10%
Veronica Kirk
Quality Enhancement Manager
QEM feedback
 Anecdotal feedback from centres on the new
systems and criteria.
Tea Break
Juliette McGinley
Quality Assurance Logistics Manager
Overview
 Introduction and team update
 EV+ functionality
 Quality Assurance Management System
Quality Assurance Logistics Team
EV+ documents and templates
 Review of documents
 Template index and restructure of
documents
 Quality Framework documents archived
Allocations via EV+
Process Allocation
Accept or
Decline
Visit Date
Visit Report
Location
Reminder
Facility
EV+
EV+
Online reporting
EV+ then
QAMS
EV+
Contact QA
Logistics Team
EV+
EV+
Report upload
to EV+
Documentation
from EV+
EV+
Style
New
Approach to
QA
Visiting
SVQ/HN
Visiting NQ
EV+
EV+
EV+
EV+
EV+
Report upload
to EV+
EV+ Summary
 Continue to accept/decline allocations
 Log estimate and actual visit dates
 Availability of visit plan PDF
 Some reminder facilities will remain
Quality Assurance Management
System (QAMS)
Benefits of the Quality Assurance
Management System (QAMS)
 Online Reporting for Systems and
Qualification Verification
 Workflow for verification activity
 Information Management System
 Intelligence to inform future selections for
Systems and Qualification Verification
 Reporting output to centres
Benefits of the Quality Assurance
Management System (QAMS)
SQA
Information Management System
Reduction in manual monitoring, logging and tracking
Intelligence building for each centre
Centre
Improved report layout with non-compliances at glance
Improved Service Level Agreements
Evidence review only contains non-compliant criteria
Qualification and System Verifiers
Ease of use, web based system
Built in Guidance and on screen errors
Only need to view active, outstanding workload
Evidence review only contains non-compliant
criteria
Quality Assurance Management
System (QAMS)
.
Online Reporting – do not use OLD excel
report forms
Web link to QAMS included in guidance
and EV+
Automated reminders email addresses
- EV+ noreply@sqa.org.uk
- QAMS asv@sqa.org.uk
Welcome Back
David Pirnie
Lead Verifier – HN & VQ
Supporting Qualification Verification
 Update of SQA’s Quality Assurance Guidance
 New system for dealing with Qualification Verification
queries
 Introducing the Standardisation Record
Quality Assurance Guidance has
been updated
The purpose of SQA’s Quality Assurance
Guidance is to:
 support Qualification Verifiers
 promote consistency and transparency in Qualification
Verification
 provide a source of reference for verifiers and centres
Visit Planning
1. Accept allocations on EV+
2. Make initial contact with each centre and ascertain optimum time to
visit (if visit is feasible)
3. *Request information from centre
4. Select sample, formulate and send Visit Plan
*Where centres have agreed to provide evidence for all
selected units, they should still let you know who the
assessors/verifiers are and list the
assessment sites being used
Criteria wording changes
2.8
There must be evidence of initial and on-going reviews of assessment
environment(s), assessment procedures, equipment, learning and
assessment materials
4.1
The centre’s assessment and verification procedures must be
documented and implemented to meet qualification and SQA
requirements (the “review” requirement removed – now in 2.8)
3.2
Candidate’s development needs and prior achievements (where
appropriate) must be matched against the requirements of the award
(“These must be related to agreed personal action/training plans”
has been removed and placed in examples of evidence
QCF Assessor/Verifier Requirements
QCF and Customised Awards Assessors/Internal Verifiers do not need
to show evidence of having achieved nationally recognised assessor/verifier
qualifications e.g. A1/V1/9DI as per SVQs.
General requirements apply:
All Assessors/Internal Verifiers
In general terms, SQA requires all Assessors/Internal Verifiers to be
competent in the subject/occupational area to a level appropriate to the
qualification and competent in assessment/internal verification of the type
involved in the qualification.
Context section has been added
Criterion 2.8: There must be evidence of initial and on-going reviews of
assessment environment(s), assessment procedures, equipment,
learning and assessment materials.
Rationale for criterion inclusion
It is each centre’s responsibility to ensure that the assessment environment,
assessment procedures, equipment, learning and assessment materials are
up to date and provided in sufficient quantity to meet ………………………..
Context
“Evidence of planned or/and past reviews can be
Considered in judging whether sufficient evidence
exists” .
Context section has been added
Context - other examples:
3. 2
“Collective and/or individual candidate development needs can be
considered when reviewing evidence”
4.1
“Comparing expected practice (documented procedures) with actual
practice (the implementation of those procedures) can help provide
appropriate evidence”
4.2
“Assessment methods/instruments and decisions should be sampled “
Other changes to Quality Assurance
Guidance
Interviewing candidates is important to the verification process (all
qualifications) and should take place whenever possible
Where the qualification(s) sampled include one or more SVQ Units, the
relevant Sector’s Assessment Strategy applies to those units
New Guide to Assessment about to be published
Processing Qualification Verifier queries
All queries to be submitted via the
QA Appointee Website
The Quality Assurance Panel
(QAP) will agree response
wording and advise whether it
should be included in Frequently
Asked Questions (FAQs) or
Answering Your Queries (AYQ)
within 5 days.
New items are added to either the
FAQs and/or AYQ .
The FAQs and AYQ will be
reviewed and the results used to
update Qualification Verification
procedures and guidance.
Draft responses, dependent
on what they are, may be
sent to a nominated SQA
Senior Manager/Department
for their comment.
FAQs accessible to all SQA
centres and all SQA staff and
appointees on SQA’s Q A
Website.
AYQ accessible to SQA staff
and appointees only on SQA’s
QA Appointee website.
Introducing the Qualification Verification
Standardisation Decision Record
A Standardisation Decision record is:
A single continuous and up to date record used by a Verifier Group to
record key decisions relating to the group’s subject area
Benefits





All key decisions relating to externally verifying qualification(s) in one place
New (and more timely) decisions can be added without face to face meetings
Focuses on group agreement, promoting consistency
Relevant SQA Staff can more efficiently access decisions made by each group
New group members can get up to speed more quickly
Questions?
Juliette McGinley
Quality Assurance Logistics Manager
Standardisation Meetings and EV
Updates
 Standardisation meetings are booked for 4 Saturdays in
October and 1 in November.
 Singleton Update in November
 Invites will be sent out to Seniors by 3rd September.
 QAMS training sessions will run each Saturday
 Disseminate information relating to Quality Assurance at
standardisation meetings
Service Level Agreements
Qualification Approval
 Accept or decline allocated visit within 3 working days
 Advise of visit date within 5 working days of acceptance
 Visit plan sent to centre copying in asv@sqa.org.uk
 Visit Report submitted to SQA within 5 working days of visit
Qualification Verification
 Accept or decline allocated visit within 2 weeks
 Contact centre to arrange visit as soon as possible
 Re contact centre within 6 weeks of visit to request information
 Visit plan sent to centre and SQA within at least 2 weeks before the
visit
 Visit Report submitted to SQA within 5 working days of visit
Housekeeping
 College regionalisation – please ensure SQA has a current list of
exception centres.
 EV contact details – please ensure SQA holds your most current
details.
 Please submit IAR ASAP, deadline 30th August
 New EVs – contact QA Logistics team to create an EV+ account.
 Quality Assurance Appointee website – migration from SQA Academy.
 Visit Plans must be sent to SQA.
Moving Forward
 Quality Assurance Appointee Website will be updated with
documentation discussed today:
– Standardisation documentation
– Processing qualification verification queries
– EV+ guidance
– SQA’s Quality Assurance presentation/videos
– Qualification verification guidance
 Regular e-zines to update Verifiers as documentation is published on
Appointee website
 All centres offering HN & VQ will move to SQA’s Quality Assurance for
Higher National and Vocational Qualifications by August 2014
Further Information
Quality Assurance Appointee Website:
http://www.sqa.org.uk/qaappointee
QAMS online reporting:
http://theia.sqameet.net/qa/qat.nsf
Questions?
Senior External Verifier
Event
29 August 2013
Appointee Management
Mark Hood
Training and Development Manager
Tel: 0845-213-5365
Email: mark.hood@sqa.org.uk
Senior External Verifier Event
HN/SVQ External Verifier Qualifications
Key Performance Measures (KPM)
Data Protection Act (1998)/
Protecting Information
SQA Academy Courses
HN External Verifier Award –
2012/2013 Academic Session
Externally Verify the Assessment Process in
Non Workplace Contexts – (FX33 36)
2012/2013 Academic Session
 75 HN External Verifiers enrolled on the award
 39 Started in 2012/2013 academic session
 36 Started in the 2011/2012 academic session
HN External Verifier Award –
2012/2013 Academic Session
Externally Verify the Assessment Process in
Non Workplace Contexts – (FX33 36)
2012/2013 Academic Session
 10 Candidates completed the award in the
2012/2013 academic session – started
2011/2012 academic session
 11 Candidates from 2011/2012 academic
session are scheduled to complete.
HN External Verifier Award –
2012/2013 Academic Session
Externally Verify the Assessment Process in
Non Workplace Contexts – (FX33 36)
2012/2013 Academic Session
 15 Candidates from the 2011/2012 academic
session require centres to be allocated in the
2013/2014 academic session in order to
complete award.
 16 Candidates from 2012/2013 academic
session are scheduled to complete prior
to January 2014.
HN External Verifier Award –
2012/2013 Academic Session
Externally Verify the Assessment Process in
Non Workplace Contexts – (FX33 36)
2012/2013 Academic Session
 23 Candidates from the 2012/2013 academic
session require centres to be allocated in the
2013/2014 academic session in order to
complete award.
SVQ External Verifier Award –
2012/2013 Academic Session
Externally Monitor and Maintain the Quality of
Workplace Assessment – (FD44 04)
2012/2013 Academic Session
 29 SVQ External Verifiers enrolled on the award
 13 Started in 2012/2013 academic session
 16 Started in the 2011/2012 academic session
SVQ External Verifier Award –
2012/2013 Academic Session
Externally Monitor and Maintain the Quality of
Workplace Assessment – (FD44 04)
2012/2013 Academic Session
 7 Candidates completed the award in the
2012/2013 academic session – started
2011/2012 academic session
 1 Candidate from 2011/2012 academic session
is scheduled to complete.
SVQ External Verifier Award –
2012/2013 Academic Session
 Externally Monitor and Maintain the Quality
of Workplace Assessment – (FD44 04)
2012/2013 Academic Session
 8 Candidates from the 2011/2012 academic
session require centres to be allocated in the
2013/2014 academic session in order to
complete award.
 6 Candidates from 2012/2013 academic session
are scheduled to complete prior to
December 2013.
SVQ External Verifier Award –
2012/2013 Academic Session
Externally Monitor and Maintain the Quality of
Workplace Assessment – (FD44 04)
2012/2013 Academic Session
 6 Candidates from the 2012/2013 academic
session require centres to be allocated in the
2013/2014 academic session in order to
complete award.
HN/SVQ External Verifier Awards
2013/2014 Academic Session
HN External Verifier Award
 Scheduled to commence from November 2013
SVQ External Verifier Award
 Scheduled to commence from December 2013
Key Performance Measures (KPMs)
KPM – No. 6
 submit CPD record with agreed timescales
where appropriate
Continuing Professional
Development Record (CPDR)
Submission Date:
30 June in any given year
 Total Number of CPDRs expected
442
 Total Number of CPDRs submitted
 Total Number of CPDRs not submitted
334
98
Continuing Professional
Development Record (CPDR)
Submission Date:
30 June in any given year
 Total Number of CPDRs submitted
 Total Number of CPDRs Compliant
 Total Number of CPDRs Non-Compliant
 Total Number of CPDRs Compliant – Late
334
285
9
32
CPDR – Standardisation
 Total Number of CPDRs Non Compliant
on first submission
 Total Number of CPDRs Compliant on
second submission
 Total Number of CPDRs Non-Compliant
submitted after – 30 June
 Total Number of CPDRs Non-Compliant
11
5
5
9
Result of CPDR Non-Submission
The following may not be re-appointed for the
2013/2014 academic session as CPDR was not
submitted for 2 years in a row:
15 EVs across HN and SVQ Qualifications Blocks
Key Performance Measures (KPMs)
Principal External Verifier (1 of 3)
 Train, develop and support members of the EV
team to meet SQA requirements <SQA>
 Monitor performance of members of the EV
team to meet SQA requirements <SQA>
 Ensure consistency of decisions made by
members of the EV team <SQA>
 Ensure reports generated by members of the EV
team meet quality standard <SQA>
Key Performance Measures (KPMs)
Principal External Verifier (2 of 3)
 Positive engagement with all members of the
team through leadership and management of
people <SQA>
 Positive engagement with all members of SQA
staff<SQA>
 Complete Principal External Verifier annual
report within conditions and timescales<SQA>
Key Performance Measures (KPMs)
Principal External Verifier (3 of 3)
 Satisfactorily fulfil the KPMs of the External
Verifier role <SQA>
Key Performance Measures (KPMs)
External Verifier (1 of 2)
 Complete External Verifier training/updates
<SQA/PEV>
 Complete scheduled centre allocation of visits
within conditions and timescales for SQA
<SQA/PEV>
 Complete scheduled central events within
conditions and timescales for SQA where
appropriate <SQA/PEV>
Key Performance Measures (KPMs)
External Verifier (2 of 2)
 Complete External Verifier reports as per SQA
quality standard and within agreed timescale
<PEV>
 Positive engagement with all members of the
team, SQA centre’s personnel and with all
members of SQA staff <SQA/PEV>
 Submit CPD record within agreed timescales
where appropriate <SQA>
Data Protection Act (1998)/
Protecting Information
SQA Data Protection policy has recently been
reviewed. The updated policy aims to:
 Clearly explain the implications of the Data
Protection Act for SQA’s work
 Set out expectations of employees, appointees,
nominees, agency workers and secondees in
carrying out their duties.
 Explain the consequences for SQA for any
policy breach
Data Protection Act (1988)/
Protecting Information
2013/2014 Academic Session
Protecting Information Course





Protecting and Storing Information
In the workplace
On the move
Staying safe online
Assessment
SQA Academy Programmes
In October 2013, I shall email all SEVs/EVs and
advise that the following 2 courses have been
updated for this current academic session:
 CPD for HN/SVQ External Verifiers
 Explaining the External Verifier Awards
CPD for
HN/SVQ External Verifiers
Changes for 2013/2014 Academic Session
One
 If you are a Principal External Verifier or External
Verifier with a Vocational Qualification
appointment will be asked to confirm that you
meet the occupational requirements of the
Assessment Strategy for your area of expertise
in any one academic session.
CPD for
HN/SVQ External Verifiers
Two
 If the awarding body is audited by a regulatory
body (for example: SQA Accreditation or Ofqual)
we may ask you to submit evidence to this effect
for audit purposes.
Three
 The Continuing Professional Development
Record (CPDR) is now in a PDF format.
CPDR
SQA Accreditation Unit Audit
SQA Accreditation Unit Audit – July 2013
 17 Verification Groups were selected for audit
CPDR
SQA Accreditation Unit Audit

















Carpentry and Joinery (165)
Construction and Civil Engineering Services (352)
Construction Conservation (432)
Construction Engineering (352)
Construction Management (316)
Construction (Plant and Ops) (186)
Construction Technician (161)
Emergency First Aid at Work (488)
Licensed Trade Qualifications (425)
Painting and Decorating (169)
SVQ Sport and Recreation – Facilities Management/Development (202)
SVQ Sport and Recreation – Exercise and Fitness (200)
SVQ Sport and Recreation – Outdoor Education (204)
SVQ Sport and Recreation – Playwork (201)
SVQ Sport and Recreation – Spectator (204)
SVQ Sport and Recreation – Sports Coaching (199)
Wood Machining (170)