Simple Structured Risk Assessment
(Learning from experience – the
importance of Near Miss and
Incident Reporting/ Investigation)
“BOW TIES and BARRIERS”
David Slater - Cardiff University
October 19th 2011
19/10/2011
Bow Ties and Incidents
1
Risk Assessment is a Simple, Natural
Process
19/10/2011
Bow Ties and Incidents
2
How to Manage my Risk?
save time
incomplete risk
picture
save ££££
be safe
I like to do a good job
I want to be safe
I’ve done this often before
Is it different from usual?
I am judged on….
Does doing this feel right?
I want the business to succeed
Will my boss/shareholders support me if ..…?
19/10/2011
Bow Ties and Incidents
3
and if I get it wrong……
19/10/2011
Bow Ties and Incidents
4
or……
$
19/10/2011
Bow Ties and Incidents
5
and quite possibly……
19/10/2011
Bow Ties and Incidents
6
What are the hazards how bad could it be?
$
19/10/2011
Bow Ties and Incidents
7
Failure thro’ Imperfection – “Human”
Error?
• In the Swiss Cheese model, individual
weaknesses are modelled as holes in slices of
Swiss cheese, such as this Emmental. They
represent the imperfections in individual
safeguards or defences, which in the real
world rarely approach the ideal of being
completely proof against failure.
19/10/2011
Bow Ties and Incidents
8
Bow Ties – an overview
•
•
•
•
•
•
•
•
•
Bow ties evolved out of Reason’s “Swiss cheese” mould. They added a crucial insight
There is a point between “Cause” and “Consequences” where you lose control. This is the
“Knot”.
Up to the “knot” any "barriers" are there to stop you losing control - they are a measure
of the Vulnerability of the system.
After the knot the outcome is often pure chance (slipping on ice, falling off a ladder!). Any
barriers here are to avoid/reduce the consequences (seat belts, air bags!). Their
effectiveness is then a measure of the Resilience of the system.
The big advantage of the method is that it is an overview of the incident structure(and
underlines (justifies?) the importance of recording near misses - the one's that don't get
past the knot!)
It gives especially non technical people a feel for where their performance or otherwise
affects a particular barrier and the purpose of resilience barriers which are not necessarily
redundant (BP gulf of Mexico).
The advantages of adopting this way of analyzing accidents, is that you quickly find that
the majority fit a reasonably small set of Bow tie templates.
By recording incident and near miss data on to these templates you start to build up a real
life indication of barrier effectiveness (Swiss Cheese permeability) or not?
Finally you can use this recorded data to calculate a value of system integrity using LOPA.
19/10/2011
Bow Ties and Incidents
9
What prevents the hazards being
realised?
cause
cause
Loss of
control
cause
cause
cause
cause
19/10/2011
“barriers” or
“safeguards”
Bow Ties and Incidents
10
What mitigates the consequences?
consequence
$
consequence
Loss of
control
consequence
consequence
consequence
“barriers” or
“safeguards”
19/10/2011
Bow Ties and Incidents
11
A shared picture of how the hazard is
managed
$
cause
consequence
cause
consequence
cause
Loss of
control
consequence
cause
consequence
cause
consequence
cause
19/10/2011
“barriers” or
“safeguards”
Bow Ties and Incidents
12
The “Bowtie” Methodology
Everything leading up to the
accident
Everything following the
Accident
Causes
Damage
Accident
Fault-tree
19/10/2011
Event-tree
Bow Ties and Incidents
13
After the Knot! (La Deluge?)
19/10/2011
Bow Ties and Incidents
14
FOR A “SIMPLE” MODEL THERE ARE SOME
FUNDAMENTAL INSIGHTS FROM THE “BOW TIE”
PARADIGM
• The Knot is highly significant, it is the point where we lose
control
• A logical (and useful )definition of “Vulnerability” then follows
as - “The Propensity to loss of control”
i.e. The Left Hand (LHS)
•
And similarly “Resilience” is “The Effectiveness and depth of Defences, once control is lost”
i.e. The Right Hand Side (RHS)
• Aren’t these more rational and rigorous definitions ?
19/10/2011
Bow Ties and Incidents
15
MANAGEMENT IMPLICATIONS OF THE
“BOW TIE” PARADIGM
• LHS – Reduce VULNERABILITY (avoid the accident!)
Design out branches, ideally ensure inherent safety, limits
and boundaries ;Design in checks and balances. (ABS)
“RISK” is then what you can’t control or guarantee to
stop!
• RHS – (Its going to happen), ensure RESILIENCE!
Barrier effectiveness/ performance checks, availability/
(maintainability), permeability, and degradation
(complacency, relevance/ credibility/(short) Cuts)
- Panic Button, Fail to Safety, ESD, Dump and Recover,
Dead man’s handle, Response, Redundancy. (Airbags)
• If the consequences are really serious – Plan for Worst
case survivability (or if in doubt, don’t do it!)
19/10/2011
Bow Ties and Incidents
16
LHS -Take away the causes,
Reduce Vulnerability
Causes
Damage
Accident
Fault-tree
19/10/2011
Event-tree
Bow Ties and Incidents
17
LHS - can’t remove cause orfor all RHS pathways – Put up barriers
Causes
Damage
Accident
Fault-tree
Event-tree
• But how effective are the Barriers really?
19/10/2011
Bow Ties and Incidents
18
Barrier effectiveness
• What are the “barriers”?
• How do I know what makes a barrier effective?
• How do I know when it won’t work?
• With multiple barriers could one failure go
undetected?
• How might my actions impair the effectiveness of
a barrier?
• How can I improve the effectiveness of barriers?
19/10/2011
Bow Ties and Incidents
19
Barriers are the same as in the
Swiss Cheese model
Lines of defence
Defects
19/10/2011
Bow Ties and Incidents
20
- and fail as in the Swiss Cheese
model!
Lines of Defence
19/10/2011
Bow Ties and Incidents
21
This means we can still see a path
thro’ - An Accident Sequence ……
Causes
Damage
Accident
Fault-tree
19/10/2011
Event-tree
Bow Ties and Incidents
22
Bow Ties and Barriers
Lack of visibility due
to poor design
CCTV in driver's
cabin
Banksman controls
crane operations
C Crane
operations
C.01c Accident in
laydown area
No more than two
people in laydown
area (PTW)
Injuries and fatalities
Accident
Used extensively now in Hazardous Industries
Equally useful in all Operational Phases
• Before – Design,
• During – “Tool Box Talks”, and
• After – Incident investigation
This is a case where an offshore crane operator dropped a
load of drill pipe in a lay down area which was supposed
to be “off limits” and controlled by CCTV and “Safety
Rules”!.
19/10/2011
Bow Ties and Incidents
23
BARRIERS– Defence in Depth or Leaky
False Comforters?
Loss of
Control
Accident
Safeguard 1
Safeguard 5
Safeguard 3
Safeguard 2
19/10/2011
Safeguard 4
Bow Ties and Incidents
24
19/10/2011
Bow Ties and Incidents
25
A Common Format
•
Threat
/Cause
.
.
Protection
Barriers
Loss of
Control
Mitigation
Barriers
Consequences
This is consistent with the standard Bow tie analysis output, HAZOP output and
Incident/ occupational databases (such as Story builder) and accident
investigation (cause and effect) Templates.
• F1 – Cause/ Threat frequency – From a choice of sources (incident
databases, Delphi sessions, Markow/Montecarlo distribution functions
(from e.g. Modelrisk, @Risk, Goldsim, etc))
• P1,P2, P3,- The Probabilities of Failure on Demand of the Protective Barriers
• P4, P5, P6 - The Probabilities of Failure on Demand of the Mitigation Barriers
• N1,N2, N3 – The consequences of the unmitigated and mitigated outcomes
So What? – (Stage 4)
From the sequence and Data above, the analyst has now the ability to print out
and record a range of essential outputs (displayed in real time if he wishes).
• F2 – the Expected Frequency of the Loss of Control (Top event) – the
VULNERABILITY to that threat. (F1xP1xP2xP3-• F3 – the Expected Frequency of the Consequences identified – the
RESIDUAL RISK
(F2 x P4xP5 xP6 ---• The system RESILIENCE is then F2 / F3 etc and PIG outputs are logF and logN
19/10/2011
Bow Ties and Incidents
26
A Virus Attack
• How would this work for a Virus attacking our Computer System?.
Threat
/Cause
Protection
Barriers
Loss of
Control
Mitigation
Barriers
Consequences
.
• This is still consistent with the standard Bow tie analysis output,
• F1 – Cause/ Threat frequency – How likely is it my system will be
attacked? Probably - Very likely = at least once per year and
increasing? (from records?)
• P1,P2, P3,- The Probabilities of Failure on Demand of the Protective
Barriers which are?-Standard Firewalls, Training/standards
compliance, access restriction?
• P4, P5, P6 - The Probabilities of Failure on Demand of the Mitigation
Barriers – Virus removal patch- unless its a new virus, isolation,
quarantine, Hard Disk firewall
• For each of these we can use incident records, or intelligence
estimates, plus the option of cloud sources, real time monitoring,
Dependency analysis and/or a combination of all of the above..
19/10/2011
Bow Ties and Incidents
27
Barriers can be Hard and Soft
• And helpful visually
19/10/2011
Bow Ties and Incidents
28
• We can utilise
actual incident
databases –
• Company ones
are best(such as
TRACTION in BP)
• This
Storybuilder
database is
available in the
Netherlands
19/10/2011
PFD’s from Actual
Incident Records
Bow Ties and Incidents
29
Lets work thro’ an Example
The Macondo Well Incident Outcomes:
• - Safety: 11 fatalities / 115 rescued / Rig
sinks
• - Environment: Largest oil spill in US history
• - Multiple inquiries
• - Regulatory agency reorganizations
• - Many new technical and permitting
requirements
19/10/2011
Bow Ties and Incidents
30
1.
What Went
Wrong?
19/10/2011
“Wrong” kind of cement in well
casing. (Hard, Extant, Unrevealed)
2. Drill pipe NRV failed. (Hard, Design)
3. Staff “misread” key pressure reading
(Soft, Human, Procedures, Training)
4. Rig crew did not recognise the (oil & gas)
influx (Soft, Human, Training)
5. At the surface – flow diverter failed to
dump oil and gas overboard(Hard,
Design, Management of Change)
6. Oil and gas vented directly on to the rig
(drilling floor)( Hard, Design)
7. Fire Detection/Prevention system failed
– “allowing flammable gas into the
engine rooms” (Hard, Design)
8. The “failsafe” blowout preventer (BOP
Stack) failed. Fire prevented remote
shutdown, but the BOP had flat
batteries and a faulty solenoid anyway.
Bow Ties and Incidents
31
(Jackpot!)
Lets take BP’s Barriers!
(The following is taken from the BP report on the Macondo Well incident and is used as an
illustrative application, treating the information as a given and not necessarily accurate!)
19/10/2011
Bow Ties and Incidents
32
BP project had LHS Barriers, (to reduce
their vulnerability, of Three kinds)
1. It was designed to ensure well containment and “shutinability”!(Barriers 1&2)
2. There was a range of instrumentation, procedures,
training and designated management responsibilities
to monitor, check and assure “normal”
behaviour(Barriers 3&4)
3. There was a dedicated “Blow out Preventer” function,
cabin, full time operator, instrumentation and
emergency valves to protect against “Loss of Control”
(Barriers 5&6)
But ---Design, construction, systems and procedures all
failed --- it was (de facto?) very VULNERABLE?
19/10/2011
Bow Ties and Incidents
33
The project had RHS – (Resilience) Barriers
again three kinds
1. Fire and Gas Detection, and Ignition
prevention/suppression to avoid Fire, (Barrier 7
shown the wrong side of knot in BP’s diagram?)
2. Emergency Procedures/drills to Isolate/ disperse
potential casualties ( note it worked for support
vessels) (Barrier ? unclaimed)
3. Sub-sea wellhead BOP valves to seal in the well.
Again all failed , so they were highly Vulnerable and as it
turned out also had no effective defences(B8)
= Zero Resilience?
How can that be?
What mouse was eating their CHEESE?
19/10/2011
Bow Ties and Incidents
34
Lets put some numbers in?
• The knot (Top Event) in the Bow Tie should be the
uncontrolled release of oil and gas, the fire and
environmental effects are the consequences.
• F1 in this case is the expected frequency of occurrence
of hitting a high pressure gas pocket.
• (If they had used the HAZOP input spreadsheet this
would have been a cause of the deviation MORE
PRESSURE)
• This is a function of specific geology, but generic data
say from OREDA suggest it is to be expected at least
once per hole? (say 10
per year conservatively)
19/10/2011
Bow Ties and Incidents
35
Probabilities of Failure on Demand –
PFD’s
• First Protective Barrier – Cemented casing – probability of failure on
demand can be estimated from direct experience with this contractor in
his track record and an estimate of the quality of the crew.(10 -1?)
• Second Protective Barrier – Non Return Valve – If we took this from
engineering plant commissioning and operational data, some companies
would have assessed the probability of its failure on demand as
50%!(0.5)
• Third Protective Barrier – Crew/operator training and procedures. Most
people would look at the latest audit data for this region/ crew? Are they
in compliance (also check the auditor –PFD – 10 -1?)
• We would also be able to interface to the BSI/Infogov online compliance
and audit checking package – Proteus; which could return (via XML) an
indication of the status of compliance with procedures and ISO
Standards. Which status %, is a measure of (the inverse of) the
probability of failure number we require.
19/10/2011
Bow Ties and Incidents
36
Real Time Status?
• Fourth (and final) Protective Barrier –The Blow out Preventer –
• This is such a crucial piece of equipment that, although we could
take historical data for its failure probability, the more useful way
would be to monitor the real time status of the equipment. If there
is a control cabin with all the relevant feeds available, our NIMBi
module could relay its status data in real time. In the Gulf of Mexico
incident there were queries as to its availability due to flat
batteries, this would have given a probability of failure on demand
of 1?
• Once you’re past the BOP, you’re out of control (past the knot on
the Bow Tie!). Hence the other four BP barriers come after, as
mitigation, not protection.
• (The expected frequency of a blow out was then their expected
state of vulnerability– from 1 in 10 to 1 in a 100 per year?)
19/10/2011
Bow Ties and Incidents
37
Display it in Real Time?
• Wouldn't that have been very useful?
19/10/2011
Bow Ties and Incidents
38
Expected Mitigation Effectiveness?
• Now let’s deal with the barriers which were
designed to mitigate the consequences of an
uncontrolled release of oil and gas. (It is
planned to include access to a consequence
modelling package (perhaps based on PHAST
or similar), to give quantitative estimates of
impacts- probably in categories)
• We can then assess the RESILIENCE of the
system.
19/10/2011
Bow Ties and Incidents
39
Barrier PFD’s
• First Mitigation Barrier – Flow Diverter – This was no
longer used as a diverter and so is a missing barrier – (P = 1)
• Second Mitigation Barrier – Fire and Gas Detection
system/Alarms – Again seems not to have been adequate
for the scale of incident? (P = 1?)
• Third Mitigation Barrier – Fire /Explosion suppression –
inadequate for scale of release? (P = 1)
• Fourth Barrier – Evacuation survival procedures – Support
Vessel Response was prompt, as required, but in total not
effective in containing consequences (P=1?)
• Hence the residual risk was still 1 in 10 to 1 in 100 – not the
kind of (lack of) resilience that the operators could have
ignored, had they been aware.
19/10/2011
Bow Ties and Incidents
40
So What is the Relevance to the BP
Incident?
• The company needs to look again at the effect of “influencing factors”
affecting human performance in that environment.
• Economic and (lack of?) Regulatory pressures need also to be identified
and their mode of influence recognised.
• It was almost certainly not the result of the sudden, simultaneous and
statistically improbable failure of 8 completely independent
Barriers!(extant fail and unrevealed)
• But we do need to recognise their inherent complexity as more than
simple “Reason-able?”“cheese” Barriers!
• We need to use “state of the art” tools to manage “state of the art”
projects’ and move on from nice pictures and analogies.
• We need to learn from incidents – yes, but we would prefer to predict
and avoid them ---if all else fails---?
19/10/2011
Bow Ties and Incidents
41
Monitoring Barriers
• Knowledge of the status of Barriers is key:
• Formal focused in-depth reviews –
excellent,
Barrier Status – a to f
• but infrequent
• - TTS (e.g. Statoil) − 5 yearly
• - Audits − 3 yearly
• - Planned Inspections − 1 year
• Lessons learned from Incident
investigations −
• excellent AND high frequency
• - BSCAT approach − every incident / near
miss
• means some barriers failed / degraded
• - For many facilities this is 100+ events /
year
• - Only current answer - collect statistics
and root causes
• Can we afford to wait?
19/10/2011
Bow Ties and Incidents
42
Top-Ten Missed Opportunities from
Accident Investigation (Kletz, 2003)
•
•
•
•
•
•
•
•
•
•
Accident investigations often find only a single cause
Accident investigations are often superficial
Accident investigations list human error as a cause
Accident reports look for people to blame
Accident reports list causes that are difficult to remove
We change procedures rather than designs
We may go too far!
We do not let others learn from our experience
We read or receive only overviews
We forget the lessons learned,
19/10/2011
Bow Ties
and Incidents
43
and allow the
accident
to happen again
Societal
Market forces
Govt/ regulatory system
Corporate
Poor change mgt
Organisational
Poor
engineering design
Operating
in alarm
mode
Incorrect
operation
of manual
bypass
valve
Poor shift
handover
Condensate
overflow
Physical accident sequence
19/10/2011
Govt failure to provide
alternative supply
Inadeq regulatory system
Esso cost cutting
Absence of engineers
Poor
supervision
THE ESSO LONGFORD EXPLOSION
Exxon control failure
Focus on LTIs
Maintenance
backlog
Poor
maintenance
priorities
Failure of
incident
reporting
system
Inadequate
procedures
& training
Plant interconnections
Warm
oil
restart
Warm
oil
pump
trip
Embrittlement of heat
exchanger
Bow Ties and Incidents
Failure to ID
interconnection
hazard
Failure to
HAZOP GP1
Poor
auditing
Loss of
supply
Explosion
2 wk site
closure
44
So What?
• We need to accept reality and the lessons of recent history.
• Modern Infrastructure Systems have become (“stiff”); too
complicated for simplistic risk management approaches!
• Complex Systems can have “Normal Accidents” (Perrow).
• Management requires a more "Holistic” overview of how
incidents occur (Hopkins).
• We need to adopt a much more thoughtful and structured
approach to Risk Management and Incident Investigation
• And we need to ensure we have a system for recording,
analysing and monitoring/ warning us of our actual incident
and near miss records to really “learn the lessons!”
• “Bow Ties” is a “Cheese" development which fits the bill!
• We can now focus on designing in “Adequate RESILIENCE”!• rather than “Acceptable Risk”!
•19/10/2011
If it can --, It will ! –
Bow Ties and Incidents
45
Risk Management really is a matter of life
or Death!
19/10/2011
Bow Ties and Incidents
46