Accident Investigation
and
Aircraft Hazard Areas
in the Post-Columbia World
Paul D. Wilde, Ph.D., P.E.
FAA/AST-4
Columbia Accident Investigator
Introduction
• I was an investigator for the Columbia
Accident Investigation Board (CAIB).
– At the CAIB, I investigated the technical
cause and the public safety issues.
– The implications listed are derived from my
CAIB and other experience.
• Some things have changed since the
CAIB, but some thing have not.
– Aircraft Hazard Area (AHA) implementation
has evolved substantially (Murray AIAA 2010-1349)
– Aircraft and space safety and investigation
paradigms remain vastly different.
Overview of CAIB Findings and
Implications for Space Safety
Finding
• Space launches are
risky
Implication
• Be prepared for
accidents
• Past success does not • Understand
provide future success
anomalies
• Standards and formal
structure can help
• Don’t short cut
formal processes
• Independent technical
authorities are
valuable
• Safety vigilance is
challenging
Space Vehicles Are Dangerous
Evidence
Implication
• “Building rockets is hard. Part • Accidents should be
of the problem is that space
expected; prepare
travel is in its infancy.” CAIB
plans for emergency
Vol. 1 page 19
response
• “Building and launching
rockets is a very dangerous
business and will continue to
be so for the foreseeable
future” CAIB Vol. 1 page 19
• Rockets fail catastrophically
10-100 thousand times more
often than commercial
transport aircraft (per flight).
• Prepare investigation
and RTF plans,
including interface to
media and other orgs
• No presumption of
safety: accidents
usually stop all flights
until cause is found
Independent Technical
Authorities Are Valuable
Evidence
Implication
• A compliance verification
• Independent
organization independent of
compliance verification
operational program cited as
enhances the safety of
key to success for Navy subs
complex technical
and nuclear reactors, and in
systems
Air Force launch verification.
• “Organizations that deal with • Checks and balances
high risk operations must
promote communication
always have a healthy fear of
(in-flow of new info,
failure - operations must be
addressing minority
proved safe rather than the
opinions)
other way around.” CAIB Vol.
1 page 190
• Safety takes real effort
Public Safety
• Columbia break-up during re-entry clearly
could have caused public casualties
• Lack of public casualties due to Columbia
break-up was the expected outcome given
the sparse population
– P>1 serious injury was <50% (~10-30%)
– Same accident over a major city expected to
produce a few public casualties
• Hypersonic ops late at night lowers risk
– Roofs protect effectively from most debris
• Relatively high probability of failure makes
“safe” for public difficult to verify
Risk to Aircraft Flying Near
Columbia Break-up
• At the time of Columbia break-up, FAA
was unaware of any hazard to aircraft.
– TFR issued ~ 45 minutes afterward based on
radar detection of debris, media rept., etc.
• Post CAIB analysis by FAA showed
aircraft PI ~ 0.001 to 0.01
• Post CAIB simulation illustrates the issue
– Actual aircraft flight locations/trajectories
– Blue dots are recovered debris locations
– Statistical distribution of debris during fall
– The view is from the southeast
• Green lines show County boundaries
8
Safety of Aircraft Flying Near
Space Launch or Re-entry
• To provide safety and efficiency in US
NAS, both pre-defined and real-time
AHA are used.
– AHA for planned debris (jettisoned stages)
– Break-up generally spreads debris over a
large area; aircraft PI often exceeds 1E-6
– During exo-atmospheric flight, several
minutes between break-up and debris
reaching aircraft altitudes.
– Vulnerability of aircraft to such debris strikes
is highly uncertain and under investigation.
BACK-UP
Sub-models for AHA Development
PROBABILITY OF
FAILURE (POF)
Probability of debris events
(failure) allocated to each
time in flight and vehicle
response mode (VRM)
VULNERABILITY
Probability of a
consequence (e.g.
casualty) for a given
aircraft impact
TRAJECTORY
Break-Up State Vectors
(BUSV) for each time in
flight and VRM
IMPACT PROBABILITY
Probability of an impact
on a given aircraft (size
and trajectory) for each
category of debris
DEBRIS LIST
A list of debris for each
BUSV: debris groups
of similar fragments
DEBRIS DISPERSION
Probability distributions
for the dispersion of each
category of debris given
each BUSV
The last two (vulnerability and impact probability), plus the risk criteria for
aircraft, have aspects that are necessarily unique to aircraft hazard area
analysis; all other sub-models are common with the debris risk analysis
Aircraft Grid & Trajectory Approaches
to PI Estimate
• Grid approach
– Assumes aircraft
continuously
present in each
grid cell
– Produces
conservative
results
• Specified
trajectory
– Accounts for
aircraft azimuth
and limited dwell
time in each cell
– More realistic PI
is 2x to 7x lower
Airbus A300: Struck by a missile at 8,000 ft but landed safely
22 Nov 2003
Aircraft Vulnerability
Modeling
See Wilde & Draper
AIAA paper 2010-1542
Current Efforts Toward Higher Fidelity
Aircraft Vulnerability Models (AVMs)
• FAA sponsored higher fidelity analysis using previously
developed tools (e.g. military) and input data
• FAA impact testing to improve skin penetration eq., evaluate
• Influence of obliquity, fragment density, distance from support, etc.
• Available results show
– Current penetration equation is conservative
– 321-10 AVMs are excessively conservative, esp. for
“catastrophe”
V_Terminal_Velocity_Fragment
V_Relative_Velocity_Fragment
Elevation Angle
V_Aircraft
Public Safety Findings
• NASA should
– Implement public risk acceptability policy
– Mitigate public risk from STS flight
– Study debris to improve risk estimates
• Collective public risk from space flight is small
compared to civil aircraft operations.
– Principle reason is huge number of aircraft
operations relative to launches.
• One in a million risk to individuals is a
recognized benchmark for both and others
• Complete report at www.caib.us Vol.II D-16
Understand Anomalies
Evidence
Implication
• O-ring blow-by and foam • Anomalies are often
early warnings
impacts were previously
detected as anomalies
• Successes do not
prove problem solved
• The cause, effect, and
or not dangerous
limits of these anomalies
were not understood
• Examine all data on
anomalies separately
• “Engineers understood
and as a set
what was happening,
but they never
• Provide technical rigor
understood why.”
in all requirements,
CAIB Vol. 1 page 196
rationales, validations
Formal Structure Can Help
Implication
Implication
• Formal standards help
define what is an
anomaly
• Formal documentation
traces what was done
to verify requirements
were satisfied
• More uncertainty,
justifies more attention
and more caution
• Formal structure can
ensure that the burden
of proof is on those
• Formal documents and
saying it’s safe
peer reviews promote
better decisions and
• Formal structure
help inform future
identifies the
generations
responsible party
Informal Processes Are
Not Effective
Evidence
Implication
• Several informal attempts to • Clearly defined roles
obtain on-orbit imagery failed
and rules improve
• Lack of ground rules
effectiveness
hampered engineering
• Design structure to
teams that evaluated the
promote communication
issues CAIB Vol. 1 page 200
• Management teams violated
their own rules
• Minority opinions
should be addressed
• “When …analyses are
• Communication needs
condensed to fit on
to flow both up and
a…overhead slide,
down the chain of
information is inevitably lost.”
CAIB Vol. 1 page 191
command