Sharing of Experience from Hong
Kong, China on the Implementation of
AMP Competency Assessment System
Queenie CC Lam
Hong Kong Observatory
8 December 2011
WMO VCP Workshop on Implementing Competency Assessment for
Aeronautical Meteorological Personnel
Hong Kong, China, 5-9 Deceomber 2011
Outline

Implementation plan of Competency
Assessment System (CAS) for
Aeronautical Meteorological Observers
(AMOB) and Aeronautical Meteorological
Forecasters (AMF)

Implementation steps and tools used

Feedback from assessors and assessees
HKO’s Implementation plan of CAS

Oct 2010
Dec 2010 – Jan 2011
Feb 2011

Mar 2011

Apr-May 2011

Jun 2011

Aug-Oct 2011

Nov 2011

2012


Gap Analysis
System and Tools Design
Refresher Training Courses &
Briefing on CAS for regular Aviation Forecasters &
Observers
CAS Launch
Assessment for regular Aviation Forecasters &
Observers
CAS assessment summary and follow-up action ;
System review and refinement
Briefing on CAS for non-regular Aviation Forecasters &
Observers
Assessment for non-regular Aviation Forecasters &
Observers
CAS assessment summary, system review and
refinement
Integration of CAS into QMS
Implementation Steps
Initial
Review
System
Design
Preparation of
Documents
System
Implementation
System
Review
Feedback
Implementation steps and tools used
– Initial review

Scope and Target Assessees

In the CAS-AMOB, the target assessees for assessment
include :



All Weather Observers
Scientific Assistants who are trained Weather Observers
and may take up weather observation duties at the Airport
Meteorological Office occasionally
In the CAS-AMF, the target assessees for assessment
include :


All regular and occasional Aviation Forecasters
Experimental Officers who are trained Aviation Forecasters
and may take up weather forecasting duties at the Airport
Meteorological Office occasionally

Responsibilities




Top Management
Officer-in-Charge
Lead-Assessor of CAS-AMOB
Lead-Assessor of CAS-AMF
Initial Review – Gap Analysis
Gap analysis using cross-reference table method

Knowledge and
skills required
Source of knowledge &
acquisition of skills
Lectures
1
2
3
.
.
.
Read-up
materials
Trainin gap
Remarks
Initial Review – Gap Analysis (2)

HKO’s gap analysis cross-reference table for AMOB
Initial Review – Gap Analysis (3)

HKO’s gap analysis summary for AMOB
Initial Review - Refresher Training

Organise refresher training courses to address the
knowledge gap
System Design
HKO’s Competency Assessment System (CAS) Overview
System Design – Assessment Tools

Direct observation

Oral/Experiential questions

Written assessment

Case study/simulation
System Design –
Competency Assessment Matrix for AMOB
System Design –
Competency Assessment Matrix for AMF (1)
System Design –
Competency Assessment Matrix for AMF (2)
System Design – Selection of Assessors

Assessors

Staff responsible for the supervision of AMP and selected
trainer(s) of the Weather Observation Course and
Meteorology Course for Aviation Forecasters as
appropriate

satisfy the qualification criteria for a trainer on the subject
as specified in Quality System Procedure on training

Preferably possess knowledge of internal audit

For assessors of AMF – qualified as Meteorologist following
WMO requirements
System Design – Officers

Lead-Assessor




Officer-in-Charge, Division Head of Aviation Weather Forecast and
Warning Services



prepare and conduct assessments in coordination with other assessors
recommend overall competency rating of AMP
Maintain individual AMP portfolio
oversee the effective operation of the CAS
endorse the competency rating (either “Competent” or “Not Competent”)
Top Management, Assistant Director of the Aviation Weather
Services Branch


approve the structure and implementation plan of the CAS
ensure the integrity of CAS as part of the QMS
System Design – Schedule & Validity

Assess every two years (similar to the frequency of
revalidation test for ATC controllers in Hong Kong)

Guidelines for determining the schedule



Allow flexibility to cater for different types of weather
situations in different seasons
Last around 1 day for each assessee
Agreed between assessor and assessee

Competence notification will carry a validity period of 24
months (in consistent with the assessment frequency)

revalidation of competence notification for another 24
months before expiry
System Design – Staff Engagement

Comments and
engagement from
assessors and
assessees
Preparation of Documents - Checklists
Example : a sample of HKO’s checklists for Aeronautical Meteorological Observers
Preparation of Documents –
Assessment Papers

Around 30 multiple choice questions

Around 10 short questions

Time allowed 2 hours

Passing mark 70
Preparation of Documents –
Case Study/Simulation
Preparation of Documents
– Assessment Procedures

Prior to assessment – prepare for assessment

During assessment – conduct and record
assessment

Post assessment – feedback, document, evaluate,
review and follow-up

Reassessment procedures
System Implementation

set implementation date

Conduct briefings to assessors
and assessees

arrange assessment dates

Carry out assessment process
using planned tools based on
competency assessment matrix

Collect evidence and produce
records

Keep records of training,
qualification, years of forecasting
experience, competency
assessments in the Competency
Portfolio of individual AMP
System Review (1)

Implementation plan on schedule

Feedback/Suggestions from assessors and assessees and
observations

Observers see the assessment as a good training and learning
mechanism

facilitate continuous improvement

suggest regular refresher courses

Suggest to allow more time for preparation by assessees in
between refresher training courses and assessments

Suggest to record answers in softcopy form when taking written
assessments especially the short questions to save time
System Review (2)

Minimal oral questions during direct observation
assessment to minimise interference to operational work
of the assesses

Combined testing of competency performance criteria as
far as practicable

Make use of routine weather briefing (such as weather
conference/teleconference, handover briefing, briefing to
clients) to evaluate the assessee by direct observation

Refine CAS based on feedback/suggestions and observations

Implement suggestions made in the review
Sharing of Implementation Plans

Webpage on CAT developed
by CAeM TT-CAT
on the CAeM training website
(http://www.caem.wmo.int/moodle)
“CAT” under “Regulatory and
Reference Material”
Login as a guest
Q&A