Design, Development and Delivery of
Safety Management System Training to
Transport Canada: Six Years and Counting
David Wilson, Stantec (formerly Jacques Whitford)
Roger Barker, ISR
Outline
• The Mandate
• The Plan
• Course Design and Development
– Transport Canada’s Start Point
– Inclusion of International and Audit Content
• Course Delivery
– Institutional Challenges
– Regional Challenges
• Internal vs. External Reaction
2
The Mandate
“The broad objectives of the SMS course are:
• For Delegated Officers (DOs) and
Management to develop a detailed
understanding of SMS, thereby instilling
confidence and obtaining buy-in to TC’s SMS
program
• For DOs to develop the skills necessary to
execute their mandates in the assessment of
SMS”
3
Aviation mission and vision
To develop and administer policies and
regulations for the safest civil aviation system for
Canada and Canadians using a systems
approach to managing risks
– Continuous improvement
– High level of public confidence
Flight 2005: A Civil Aviation Framework for Canada
4
Evolution of Safety Management
INCIDENT RATE
3.0
2.5
2.0
1.5
1.0
Safe Environment
Supervision
Driven
Safety Awareness
Team Driven
0.5
0.0
Command & Control
Safe Person in Safe
Environment
Behavior
Driven
Programs
5
Management Systems
Towards Zero...
…is the condition to which risks are managed to
acceptable levels.
6
TC Requirements
• “Aviation organizations should establish and
maintain a safety management system that
conforms to all of the requirements of
applicable Canadian Aviation Regulations
(CARs) and Standards”
7
TC Requirements
There are 9 parts to the CARs:
1. General Provisions
2. Aircraft Identification and Registration and Operation of a
Leased Aircraft by a NON-registered owner
3. Aerodromes and Airports
4. Personnel Licensing and Training
5. Airworthiness
– Aircraft Certification (AC)
– Maintenance & Manufacturing (M&M)
6.
7.
8.
9.
General Operating and Flight Rules
Commercial Air Services
Air Navigation Services
Repeals and Coming into Force
8
The Plan
• Phase 1: Analysis
– Deliverable 1: Training Needs Analysis, Learning
Strategy, Final Work Plan and Background Update
• Phase 2: Evaluation and Design for CBA and AMM
– Deliverable 2: Course Syllabus, Course Materials
• Phase 3: Pilot Course
– Deliverable 3: Pilot Course, Course Evaluation
• Phase 4: Course Delivery to CBA and AMM
– Deliverable 4: CBA Course, AMM Course
9
The Plan cont’d
• Phase 5: Validation of Course Delivery to CBA and AMM
– Deliverable 5: Confirmed Assessment Protocol, Course
Validation
• Optional Phase 6: Adaptation for Other Functional Branches
– Deliverable 6: Adapted Course(s)
• Optional Phase 7: Course Delivery to Other Functional
Branches
– Deliverable 7: Other Functional Branch Course(s)
• Optional Phase 8: Validation of Course Delivery to Other
Functional Branches
– Deliverable 8: Course Validation(s)
10
Course Design
• Definition of the target population
• Identification of the SMS knowledge level of
target audience
• Review of CARs and notices of proposed
amendments (NPAs) to CARs relating to SMS
• Review all applicable national and
international (e.g., UK, Australia and GAIN)
aviation SMS policies and guidelines
• Assessment of the existing safety culture
within the target audience
11
Safety Metrics Assessment
• Survey-based tool from sub-consultant
• Distributed internally to approx. 1000 DOs:
150 replies
• Intent was to identify:
– Internal issues/barriers within TC
– Issues/barriers external to TC (customer,
industry, etc.)
12
Training Needs Assessment
• Front line staff were fully aware of their needs
and challenges
– Different measures are required to measure safety and
SMS;
– The interdependencies of attitude, behaviour, and
motivation cannot be ignored
– Inspectors need practical demonstration of a functioning
SMS from the ground up,
– If support, guidance, and structural leadership from
management are not forthcoming (as happened with Risk
Management training), then SMS training will fail.
13
Global Review of TC SMS
• Global references:
–
–
–
–
OHSAS18001
GAIN
CASA Australia - Guide to Aviation SMS
SMS for Commercial Air Transport Operations - Guide to
Implementation (UK)
– OGP Checklist for an audit of safety management
• Resulting additions to the TC model:
–
–
–
–
Continuous Improvement
Feedback
Annual Management Review of the SMS
Contractors must meet safety standards which do not impinge
on the SMS of the organisation
– Inter and Intra-company Safety Performance Comparison
14
The TC
SMS
Model
1. Safety Management Plan
• Safety Policy
• Non-punitive Safety Reporting Policy
• Roles, Responsibilities & Employee Involvement
• Communication
• Safety Planning, Objectives & Goals
• Performance Measurement
• Management Review
2. Document Management
• Identification & Maintenance of Applicable Regulations
• SMS Documentation
• Records Management
3. Safety Oversight
• Reactive Processes
• Proactive Processes
• Investigation and Analysis
• Risk Management
4. Training
5. Quality Assurance
15
6. Emergency Preparedness
Course Development
• Addressing Gaps:
– Audit Approach
– Approach to Assessing SMS: Assessors vs.
Inspectors
– The TC SMS Model vs. Plan-Do-Check-Act
16
Course Delivery
• Initial 4-day course (‘03-’04)
– 19 deliveries
• V2: 4-day course (’04-’05)
– 18 deliveries
• V3: 3-day course (’05-’09)
– 15 deliveries (eLearning course starts in fall ‘09)
• 2-day awareness course
• 2-day multi-model course
• ½ day refresher course
17
3-day Course Content
• History of SMS
• Introduction to TC SMS
• SMS Elements
– Safety Management Plan
– Documentation
– Safety Oversight
– QA, Training, EPR
• Safety Performance Measurement
• Safety Culture
• Implementation of SMS
• SMS Evaluation Guide
• Process to Assess Industry SMS
• Completing the Assessment
18
The Assessment should get to the
“Real” Issues !!
D
19
Addition of Broader Audit Approach
• DOs used to inspections and checklist-driven
audits
• ‘Process’ vs. ‘procedure’ view not well
differentiated: had to get up out of the trees
• No methodology for producing defensible,
quantified assessments of SMS: development
of the SMS Assessment Process
20
SMS Assessment Process
• PreAssessment
• On-Site
Observations
• Post
Assessment
and Follow Up
21
Assessment Criteria
22
Plugging the Holes with PDCA
• Gaps in management system elements
identified in course design were not addressed
until V2/V3 of the course:
– Management review
– Proactive process (risk management)
• Some backlash from attendees as a result,
however intent was to arm DOs as best
possible to allow them to engage industry
23
Institutional Challenges
• SMS needs a Paradigm shift within aviation from :
– DOs are responsible for aviation safety
• responsible for operator’s actions and take
accountability for them
– Operators must get all actions approved by
TC
– Audits of operators are only done by DOs and
the operators cannot be trusted to do carry
out internal ones themselves
– Conflict of interest in the "old way of doing
things"
24
Regional Challenges
• Different degrees of involvement and
engagement by Regional Directors
• Regional cultural differences: adversarial vs.
partner approaches from POIs/PMIs
25
Internal vs. External Reaction
• Within TC:
– Sounds good in theory
– Are we prepared to back this up?
– How does this system scale from large to
small?
– Will TC really enforce the SMS CARS?
26
Internal vs. External Reaction cont’d
• Within industry:
– Larger operators/AMOs/airports: we do this
now (but may lack some rigour…)
– Hazard analysis and identifying risk significant
activities often a weakness
– Smaller operators: what is it exactly that you
want me to do (and when exactly do I get to
make a living…)?
27
In Conclusion
• Training of TC Civil Aviation a major
undertaking (time, personnel, cost)
• Paradigm shift just now starting to show
broadly in TC
• Communicate, communicate, communicate:
can it ever be enough?
• We see senior managers taking more of a
lead, and see that aviation experts are
needing to become more proficient in
management
28
In Conclusion cont’d
• SMS is about risk control - the loop of hazardrisk-objectives-measures-audit-management
review drives a big shift in the way
management perceive their roles and forces
communication into the organizations
• Tough part awaits: small operators
• Grateful to have been able to play a role
29
Questions