Identifying Risk via
Confidential Reporting
Systems
RAe Soc HF Group EMSG
Conference, 10th October 2007
Dr Mike Rejman
Director, CIRAS
Recent Career 1988 - present
 Head of Human Factors Unit, UK Army Air Corps
(accident investigator, set up their confidential
reporting system)
 Principal Consultant, DERA, QinetiQ,
 Assistant Director of Patient Safety at National
Patient Safety Agency (accident investigation
training, reporting system, risk assessment)
 Director of CIRAS, the confidential reporting
system for the UK railways
Understanding the Problem
• ~ 80% of accidents are attributable to human
factors issues, at the individual level, the
organisational level, or more commonly both
• This is probably a conservative figure, and is
irrespective of domain
• To manage this we need to identify and understand
the risks
• Without this understanding we can’t put
appropriate remedial action in place
Reason’s Swiss Cheese Model
Organisation and processes
- Deficiencies
“Latent failures” - precursors, preexisting enabling conditions
“Active failures” (skill, rule
and knowledge errors)
Multiple Defences
Accident or
incident
The Accident Iceberg
(Heinrich, Bird, and others)
accidents
1
?
serious incidents
incidents
?
near misses
?
& concerns
often
unreported
Prior Indicators of Risk
• Herald of Free Enterprise, 1987
– Numerous accounts of previous sailings with the
bow doors open
– Economic pressures to spend
less turnaround time in port
Prior Indicators of Risk
• Challenger Space Shuttle
– Ample previous evidence of seals shrinking
in cold temperatures
– Voiced by some engineers
– Political pressure to launch
Prior Indicators of Risk
• Kings Cross Fire, 1987
– Numerous fires, bundles of rubbish and
wooden escalators with greased tracks, some
records but nothing done
– Complacency and no system to evaluate risk
Prior Indicators of Risk
• Hillsborough, 1989
– Two Police Officers on horseback prevented
similar incident the previous year by blocking
access to the same terrace area
– No organisational memory
Investigating Accidents with the Army Air
Corps
• Experience while conducting investigations
indicated that the workforce held lots of
important information that could have been
used to prevent the accident
• The culture didn’t encourage reporting and
there was no ‘safe’ route for this information
to be passed on and assessed
AAC views on confidential reporting
• (Some) Senior Command
– Blame & punishment
– Would subvert the chain of command
• Others
– System required
– RAF-run Condor system inappropriate
AAC maintenance issues
• Gazelle maintenance team change
– habit, slip
• Lynx tail rotor gear box problem and
cannibalisation
– culture, knowledge error, perceptual problem,
assumptions (some of these issues could have
emerged via a confidential reporting system)
The situation in the NHS
Prof Sir Liam Donaldson, CMO England
•
‘Organisation with a Memory’ (2000)
•
National Patient Safety Agency formed (2001)
•
NHS Staff views – e.g. Primary Care
– Pharmacists
– GPs
– Dentists
Patient safety risk
18
16
14
Australia 16.6%
England 10.8%
Denmark 9%
New Zealand 12.9%
Canada 7.5%
Japan 11%
France 14.5%
12
10
8
6
4
2
0
% of acute admissions
The situation in the NHS
• The NHS is a labour-intensive industry
– leading to a large number of human interactions
– increasing the risks of decision-making errors
– and increasing the risk of communication
errors
The national reporting and learning system
• Positive:- the system now has over
well over one million reports
• Negative:- however the quality of
much of the data is not high
• The system was originally set up to
be anonymous, rather than
confidential
NHS example
• ~ I30 types of infusion pump available
• Study revealed that 47 different types
were present in one Trust
• And 6 different types were found on
one ward
• Leading to high level of performance
errors
NHS examples
• Removal of the wrong kidney
• Chemotherapy drug vincristine
delivered intrathecally instead of
intravenously
Rail Confidential Incident and Analysis
System
•
•
•
•
Pilot study in Scotland 1996
Rolled out nationally after
Ladbroke Grove accident and
Cullen Inquiry 2000
3 regional offices run by
contractors, now one in-house
team in London
In 10 years of operation there
has never been a breach of the
confidentiality guarantee for
reporters
UK Rail
• Is one of the safest
forms of transport
both for travellers
and staff
• But it could be even
safer as the
accident in Cumbria
in February showed
CIRAS reports on non-compliance
with rules
Forms of non-compliance across all sectors (2004 - 2006)
17.6%
Intentional rule violation
51.0%
Best practice ignored
Rule violation
31.4%
Non-compliance by sector
Forms of non-compliance by sector (2004 - 2006)
10
9
8
No. of reports
7
6
Intentional rule violation
5
Rule violation
4
Best practice ignored
3
2
1
0
NR
Subcontractor
Sector
TOC
LUL
Causal factors for each sector
Sector
Cause 1
Cause 2
Cause 3
Network Rail
Performance
before safety
46.7%
Poor planning
20.0%
Poor
management
13.3%
Subcontract
- or to NR
Inadequate
training
33.3%
Staff
shortages
22.2%
Performance
before safety
22.2%
TOC
Performance
before safety
28.0%
Other
22.2%
Inadequate
training
16.7%
LUL
Performance
before safety
55.6%
Staff
shortages
22.2%
Poor
management
22.2%
Quotes from reporters
“ We’ve refused to
do the job on the
grounds of safety …
…. and we’ve been
threatened with
disciplinary action...”
“I’ve been told
that if I didn’t
want to do the
job [on safety
grounds] then
I shouldn’t
bother coming
to work
tomorrow”
27
How can we best identify risks?
•
Reactive/retrospective
–
–
–
Accidents and incident investigation
Root Cause Analysis (what, who, how, WHY)
Reporting systems: incidents
•
But we need to move people from ‘fire-fighting’
the last error to trying to prevent the next one
•
Proactive/prospective
–
–
Confidential Reporting systems: near misses, and safety
concerns
Hazard identification, and prospective risk assessments
Barriers to Reporting
•
•
•
•
Fear (including concerns re confidentiality)
– blame culture, job loss, commercial issues
Practicalities
– time to report
– complex forms
Ignorance
– what to report (definitions e.g. ‘near miss’)
Apathy
– lack of perceived benefit to individual vs
potential cost
– lack of faith in the system to change things
– lack of any feedback (‘black hole’ syndrome)
Reporting rates and triangulating risk
•
For these reasons, incidents, near-misses &
safety concerns will always be under-reported
whatever the system
•
But there are ways to increase reporting, by
targeting specific issues, and seeking other
ways to triangulate risk including surveys
Reporting rates and triangulating risk
•
More reports = good news, not bad news. With
more information to analyse we can act as an
Early Warning System
•
The industry must therefore encourage and
reward reporting, not penalise it
•
CIRAS has recently moved beyond the model of
the ‘passive post-box’ and is seeking to be more
‘proactive’ in engaging industry groups e.g. with
workshops and surveys
Confidential Reporting Systems
In an ideal world we wouldn’t need
confidential reporting systems. Staff
would be happy to volunteer information
about errors they made, or safety
concerns they had, without fear of blame
or victimisation, and management would
willingly address all the issues raised by
their staff…………
Confidential Reporting Systems
But we don’t live in an ideal world !
Consequently in virtually all safety-critical
industries, here and abroad, it has been
necessary to incorporate confidential
reporting systems as part of the suite of
reporting systems available and operating
within safety management systems
Identifying Risk by
Donald Rumsfeld
As we know,
there are known knowns.
There are things we know we know.
We also know there are known unknowns.
That is to say
We know there are some things we do not know.
But there are also unknown unknowns,
The ones we don’t know we don’t know.
Confidential reporting systems
• Are there to access all these
categories
• Are an indispensable part of the
process of identifying risk in any
safety-critical industry
• Should be an integral part of any
safety management system
mike.rejman@ciras.org.uk