An academic view of
Human Factors in
Maintenance
Simon Place
Danny Jayakody
Hamad Rashid
RAeS conference
8 October 2008
A limited selection of
recent research
1. Maintenance error – Proposal for Error Management in
Numerical Airworthiness Terms [Simmons, 2002]
2. Maintenance error prediction modelling [Leach, 2005]
3. A critical analysis of the links between the taxonomies
used by MEDA & HFACS-ME [Eshati, 2006]
4. Aviation Maintenance Monitoring Process (AMMP)
[Rashid]
5. Risk Assessment in Continuing Airworthiness of Air
Transport [Jayakody]
Error Management in
Numerical Airworthiness
Terms
ECI
[Simmons, 2002]
Catastrophic
Error Criticality
Indices (ECI)
Major
Minor
1.00
0.00
10E-3
10E-6
10E-9
Probability
• Estimate the criticality of a maintenance task, based on:
–
–
–
–
Effect if left uncorrected at Release To Service (RTS)
The severity of that effect
Where in the maintenance sequence it occurs
Whether an adverse outcome has been anticipated in design
Error Management in
Numerical Airworthiness
Terms
[Simmons, 2002]
Assessment of
Data from Human
reliability analysis
maintenance
tasks
Modify with
Performance
shaping factors
Evaluate ECI
Difference
calculation
No action
Introduce new
controls / re-design
Consider
cost savings
• Compare the criticality against the probability of error
derived from existing Human reliability tools.
• Identify any required changes needed to meet an
acceptable risk
Risk Factor Analysis
[Leach, 2005]
Combination of Failure modes and
effects analysis (FMEA) and MSG-3
• Expert System of Engineers (ES) analyse maintenance
tasks:
• Likelihood of Occurrence – range from 1 (almost
impossible) to 16 (almost certain)
• Event severity – range from 2 (not noticeable) to 24
(probable aircraft loss)
• Detection – range from 1 (easy to detect) to 16 (difficult)
Risk factor = likelihood x severity x detection
Risk Factor
Analysis [Leach, 2005]
WARNING: Flight safety
possibly endangered, additional
safety nets are required
Insufficient control measures currently applied
1740
MODERATE: Flight safety
is not endangered, but financial
penalties often exist
Model proved to be
effective at
identifying key
events - risk factor
between 2 and 6144
550
STANDARD: existing control
measures are probably sufficient
Links between MEDA
and HFACS-ME
[Eshati, 2006]
• To apply HFACS-ME and MEDA human factor
taxonomies to the same incident and accident reports to
identify the causal contributing factors of the errors.
• To identify the applicability, effectiveness and limitations
of both tools (MEDA and HFACS-ME).
HFACS - Maintenance
Extension
Management Conditions
Organizational
Supervisory
Medical
Maintainer Conditions
Crew Coordination
Working Conditions
Environment
Equipment
ource: Naval Safety Center chool Of Aviation Safety
Readiness
Workspace
Maintainer Acts
Error
Violation
ACCIDENT
Application of modelling options
(Source: Rashid / Jayakody)
CAW Study
(Bayesian Logic)
AMMP
(Fuzzy Logic)
Work Place /
Tasks / Aircraft
Supervision /
Immediate
Management
Higher Management /
Decision Makers
Regulator /
Policy Designer
Continuing Airworthiness / Strategic Issues
Task arena/ Detail issues
Intermeshing errors at manufacturer (aircraft /
• This research manages maintenance
task) and immediate workplace levels.
• Rotary-wing aircraft are taken as case study.
Aviation Maintenance
Monitoring Process
(AMMP)
• To identify Root Causes of human errors in helicopter
maintenance for both individual and organizational levels.
• To conduct retrospective and prospective analysis, under
the HERMES [Cacciabue, 2004] methodology
• Retrospective HFACS-ME applied to study 58 reports of
incidents and accidents related to maintenance
• To verify integrity of the developed process within
helicopter maintenance industry.
Fuzzy
Analytical
Network
Processes
Model
ANP
Goal
Preliminary Steps
Error Potentiality Weighting of (XI) Aircraft
Maintenance Job No. (x.y.z)
ANP
Model
Main Maintenance Tasks
Parts Removal
Paint Stripping
 Calculating potentiality
of maintenance error for
a given task (x).
Parts Cleaning
Sub Maintenance Tasks
Adjust levels / pressures / flows
Checks/ Readings/
Measurements
Adjust cable tensions / rod connections
Adjust pipe connections / Seal leakages
Troubleshooting/ Analysis/
Decision Making
Repair electrical connections / generators /
c. breakers / batteries / bulbs / warnings
Adjust voltages / currents / resistances
Topping/ Charging/
Installation/ Fixing/
Sealing/ Adjustment
Autopilot / computers / navigators / Radio
Perform structural / parts repairs / Balance
Install serviceable parts
G
Inspection
Functional Test
Rotary parts balancing / Vibration control
Engines / Hydraulics / Pneumatics / Air
Final Steps
Mechanical fits / joints / fixation / torques
Re-set the ‘‘on-the-way’’ systems / parts
Fuzzy ANP
[Chang 1992, 1996,
adapted by Rashid]
µA(x)
1.0
EI
WMI
SMI
VSMI
AMI
1
3/2
2
5/2
3
0.5
0.0
1/2
7/2
x
• Prospective: ANP weighs task errors potentialities from
both designing and immediate workplace variables
concerning that task.
• Direct product: Weighting error potentiality of maintenance
tasks and subtasks.
• Error risk = f (Potentiality, Criticality).
Fuzzy Comparison
Matrices
Pair-wise comparison of importance for tasks
within a given maintenance job
Prep
Preparation 1,1,1
Removal
Checks
Analysis
Installation
Function
Test
x, y, z
Removal
Checks Analysis Installation
Function
Test
1/x, 1/y, 1/z
1,1,1
1.1.1
1.1.1
1.1.1
Overall output: Full detailed risk ranking of
maintenance errors for a given task (changing
the main rotor of a given helicopter). 1,1,1
Risk Assessment in
Continuing
Airworthiness
[Jayakody]
1.
2.
3.
4.
Aims of research 2007-2010
To design and develop a generic model for risk
assessment in continuing airworthiness process
To determine a method to optimise regulatory oversight
programme on the basis of risk
To determine if parts of regulatory oversight inspections
could be devolved
To determine how expert opinion could be represented
as a measurable parameter in a decision analysis tool
Risk Assessment in
Continuing
Airworthiness
2007-2010
• Risk-based oversight (RBO) from Hampton Report
recommendation (leading to “Better Regulation”).
• ICAO mandate on Safety Management Systems
• Follow-on from Regulatory Oversight Weighting Index (C/SI)
• Analytical Hierarchical Process (AHP) – adopted by Dutch
CAA for assessing risk within MROs:
– Assess risks from Quality (Part reqts and Quality system) and
Organisation (Culture, features and process)
– Make an implicit risk analysis more explicit
– Informs the decision on intensity of supervision by regulator
Bayesian Belief
Networks
Information B
Prior
P(A)
Information C
Posterior
P(A l B)
New Posterior
P(A l C, B)
• Means to quantify probability of accident scenario using
“Conditional probability theory”
• Estimate the conditional probability of one causal factor
given the presence of other factors.
• Conducted using fusion of data and “beliefs” of Subject
Matter Experts (SME)
Influence Diagram
Source: Luxhoj, 2002
Consequence
Task/Environment
Individual
Mn facility
Task load
Organisation
Mn training
Corporate
deficiency
Maint HR
Regulator
Knowledge/
Experience
Maint Mgt
Mn org
struct
Operator on
oversight
Work system
design
Repair
Repair
Inspection
Failure
Current status
CAW Study
(Bayesian Logic)
AMMP
(Fuzzy Logic)
Work Place /
Tasks / Aircraft
Supervision /
Immediate
Management
Higher Management /
Decision Makers
Regulator /
Policy Designer
Continuing Airworthiness / Strategic Issues
Task arena/ Detail issues
Intermeshing
• AMMP – data gathering to start this year, with helicopter
designers/maintainers
• Risk in CAW – interest gained from military and civil operators
and maintainers. Now undertaking design of models
Bayes’ concept
x
in action
Sectors failed due to document errors
Sectors failed
Total sectors flown
for period
Sectors
not failed
Document errors present but sector not failed
Example of Bayes
• Flight documentation is inspected regularly. 20% of late flights had experienced
document errors; 30% of on-time flights had document errors.
• 10% of flights are late.
• What is the probability of document errors, given that flight is on-time?
Solution
• Let A = event that sector is late; p(A) = 0.1 and B = event that documents have
errors.
• 10% that were late (A) had document errors (B), so p(B|A) = 0.2
• 90% that were successful (Ā) had document errors (B), so p(B|Ā) = 0.3
• The probability of a sector being late (A), given that it has document errors (B)
is written p(A|B)
Using Bayes
p(B | A).p(A)
p(B | A).p(A)

p(B)
p(B | A).p(A)  p(B | A ).p( A )
(0.2)( 0.1)
0.02
p( A | B) 

 0.07
(0.2)( 0.1)  (0.3)( 0.9) 0.29
p( A | B) 