17.3 Accident Analysis Data (by Captain Donald W. Pitts)
Since the mid 1960’s, the commercial aviation industry has achieved an
extraordinarily high level of safety. For the past three decades the annual hull loss
rate for U.S. operators of large transport aircraft has averaged 1 aircraft per million
departures. In comparison, the worldwide rate of aircraft destroyed has averaged 1 to
3 accidents per million flights. Clearly these statistics highlight a marked
improvement over the rates recorded in 1959 when U.S. commercial operators
suffered 26 major accidents per million flights. Formulation of the Federal Aviation
Administration (FAA) in 1958 and its safety charter clearly illustrate that a high level
of safety oversight is assumed and expected by the American public.
The uniqueness of aviation accidents coupled with the large numbers of
deaths associated with a single aircraft loss spotlight these events with a high level of
public awareness -- even though quite infrequent, they gather tremendous attention as
events leading up to the catastrophic loss of life are recounted by the media. The
traveling public and the concerns they raise to lawmakers ultimately demand
demonstration of due diligence by responsible agencies in the aftermath of an aviation
accident. In the U.S., this was most apparent in 1996 with the formation of the White
House Commission on Aviation Safety and Security. This commission, chartered by
President Clinton, responded to the public outcry following the crash of a ValueJet
DC-9 into the Florida Everglades, followed closely by TWA 800, the complete
destruction of a Boeing 747 departing JFK International Airport. In these case
studies, a general sense of inadequate safety standards and/or increased threat from
terrorist activity lowered the public’s confidence in aviation’s system wide safety.
Even though the risk of exposure to a life-threatening event is significantly higher in
other endeavors, public acceptance of that same level of risk is not tolerated in
aviation--a higher standard of prevention is expected.
Many attribute the significant improvements in aviation safety over the past
40 years to technological advances resulting from reliability of power plants, i.e., the
departure from piston powered to turbine powered aircraft. Additionally, the better
understanding of accident causal factors made available by flight parameters captured
on data recorders has made significant improvement in aviation safety. With the
assistance of recorded data, post accident analysis provided safety recommendations
which, when acted upon, add layers of protection to the system as a whole and
generally raise the “safety bar,” lowering the risk. The compelling force for change
comes in the form of safety recommendations from various agencies and regulations
written into law by the FAA as Federal Aviation Regulations (FARs). This reactive
process, of incident/accident, investigation, recommendation and regulation has
clearly improved the safety record of aviation compared to that of the record of the
1950’s. As successful as this record is, it still falls short of the moral obligation
safety professionals face daily as they pursue the idealistic goal of zero losses while
quietly acknowledging there are no risk free endeavors. This is the challenge
aviation experts (and those who manage them) face as they strive to meet the public’s
expectation of no human tragedy, demonstrated by a continuous effort to improve
safety records.
As a result of a 1958 FAA mandate, the first accident-protected analog flight
data recorders were placed into service with four flight parameters captured on a foil
recording medium. Today, the latest generation of Digital Flight Data Recorders
(DFDRs) provides hundreds of flight parameters and record not only dynamic flight
values but also the status of various systems and sub-components of the aircraft.
These recorders capture factual data which can be used to better understand causal
factors in aviation accidents. As a result of these data, better information has been
obtained over the years, identifying the largest contributing factor to the accident
chain of events to be the human element--found causal in 70-80 percent of accidents.
Investigations today strive to go beyond simply placing the blame at the feet of the
individual(s) holding the last link in the chain of events leading up to an accident. In
the past, those reviewing incidents with a compliance mindset along the lines of strict
regulatory enforcement of FARs, have focused criticism of wrongdoing on these
lesser human performances, failing to allow credence to the honest mistakes that
professionals will make. This has proven ineffective in prevention of future
occurrences of similar events and discourages the reporting of anecdotal information
necessary to reveal hidden systemic deficiencies. In this scenario, punishment is
rendered, the books are closed on the event, while the larger community of aviators
does not gain benefit, and a hazard typically lies dormant awaiting another prey.
Over the years, identifying what has been irreverently referred to in investigation
circles as the “Guilty Bastard” and administering an appropriate corrective action
(punishment), has done little or nothing to improve human performance and their
susceptibility to errors caused by lack of education, poor aircraft design, faulty
maintenance, natural phenomena, inadequate procedures, poorly developed missions
or inadequate standards and controls. Proper corrective action and follow-on control
measures cannot be developed if the precursors that lead to an accident chain of
events are never brought to the attention of those with the ability to understand the
need for or the authority to implement change. Post accident review from
investigative agencies such as the National Safety Transportation Board (NTSB) with
their subsequent safety recommendations have clearly been a positive influence on
the overall system safety; however only in recent times has there been significant
progress made in addressing the man-machine interface and the systemic hazards that
may have triggered an accident.
Zero Accidents: The Quest for Better Prevention
The primary goal of any safety program must be the prevention of incidents
and accidents. Identifying situations that require corrective action before problems
occurs not only supports the zero accident goal but can also reduce operational costs
and significantly enhance training effectiveness. This translates into operating
efficiencies and cost savings. To achieve the highest levels of accident prevention,
the hazards faced by those at risk must first be identified accurately enough to allow
development of control measures to mitigate that risk. This is no easy task.
<<Insert Figure 17.1 Here>>
Figure 17.1, an equilateral triangle divided into three sections by horizontal
lines, depicts Heinrich’s Pyramid, a safety model. The top region of the triangle
represents accidents. The middle section represents incidents, while the base contains
unreported occurrences. In the case of one major U.S. airline these everyday
unreported occurrences could stem from as many as 2500 daily flight operations.
Clearly, accidents will draw scrutiny, while incidents, depending upon the severity,
will also be reviewed by investigators. However, the largest body of information lies
in the unreported occurrences which, for the most part, go unnoticed except by those
who personally experience the event. This model represents root cause(s) of a
catastrophic event--the accident resulting in loss of life. The point of this
characterization is that most of the information highlighting root causes is not noticed
during the everyday normal conduct of business. The primary reason for overlooking
this information is failure to identify/recognize the hazard or simply not enough
resources to spend reviewing otherwise innocuous events. Absent this precursor
knowledge, safety professionals are condemned to providing after-the-fact accident
analysis in a post mortem effort to understand what happened and why it occurred.
Understanding the “what-why” relationship is critical to every safety program, be it
reactive or proactive. Realizing that rarely is a deliberate or intentional violation
found to be involved in an incident, some means is needed to induce those involved to
freely come forward and provide details of the incident. The concept of selfdisclosure has been considered to be indicative of a constructive attitude worthy of
protection in the pursuit of safer operations.
Airline executives have long realized that technology and operational knowhow reduced the accident rate of aviation to today’s extremely low levels. However,
it is also well understood that as consumer demand for commercial aviation travel
increases, the total number of accidents will also increase given the mathematical
relationship of increasing the numerator applied to the current (tolerated) accident
rate. Without a reduction in the rate, this rise in demand is forecast to generate more
total accidents than considered acceptable by the public. Furthermore, three decades
of statistically significant data suggest that dramatic improvements in aviation safety
are unlikely if the traditional regulatory compliance/enforcement reactive approaches
are taken toward future safety programs. The common belief among aviation safety
experts is there will not be a significant reduction in the actual accident rate until
front-line operators (managers, supervisors and employees) are in the mindset to
provide experts the precursor information they need to prevent the next accident. To
encourage this information flow, an improved safety culture that shares values,
procedures, and skills must be present. This allows stakeholders to jointly identify
and respond to hazards without fear of reprisal, prosecution or disciplinary action by
any authority reviewing the information. To move beyond the current plateau, and to
truly become proactive, safety processes will need a more powerful tool--one which
will allow analysis of previously identified and potential failures with the focus on
how to best apply limited resources to mitigate future risks.
One hurdle proactive safety departments face is that public laws hold the FAA
responsible for the proper investigation and disposition of all suspected cases of noncompliance with the FARs. The FAA Administrator has a statutory responsibility to
enforce safety regulations, and in previous times that have traditionally been punitive
in nature, the FAA drew its approach for corrective measures from civil and criminal
law. In these cases the underlying assumption was one of commission of error
through negligence or recklessness. Fortunately, today, it is recognized that highly
skilled professionals can and will make mistakes, regardless of regulatory
prohibitions. What must be better understood are which factors or influences
contributed to the human error.
Hazard Identification: The Pursuit of Information
US military services led the way in aviation accident investigation following
their first aircraft accident on September 17, 1908. In that accident an aircraft piloted
by Orville Wright with observer Lt. Thomas Selfridge crashed while demonstrating
the machine to U.S. Army officials at Fort Myer, VA. Lt. Selfridge was killed.
Wright survived with broken bones and was able to provide key information to the
investigators. Post accident review indicated that elongated propeller blades of a
never-tested design led to excessive vibration which caused the propeller to strike a
guy wire on the aircraft, tearing the wire from its fastening in the rudder, and
breaking the propeller two feet from its tip.
Obviously, critical information to the investigation resided with the first-hand
knowledge of the surviving pilot who provided insight necessary to take corrective
action and thus prevent reoccurrence. Without this valuable input a similar
catastrophe could have afflicted another unwary operator. By the 1940s, both the
military and industry recognized the value of voluntary incident reporting in an effort
to prevent accidents. The lessons-learned approach in military aviation influenced
their civilian counterparts. A need for a U.S. Incident Data System was raised during
the FAA enactment hearings in 1958. Following a United Airlines incident in
October 1974, which foreshadowed a TWA accident in December 1974, a study of
the National Air Transportation System was conducted by a task force focusing on
the FAA’s safety mission. In May 1975, Advisory Circular 00-46 was issued. With
that circular, the FAA first implemented the Aviation Safety Reporting Program. In
1976 it was later modified and implemented by both NASA and the FAA as the
Aviation Safety Reporting System (ASRS). This program receives, processes and
analyzes voluntarily submitted reports from pilots, air traffic controllers and other
aviation industry stakeholders. Since the mid-1970s, NASA has collected hundreds of
thousands of reports outlining human performance errors in the National Aviation
System (NAS). The purpose of this program is to identify deficiencies and
discrepancies in the NAS with particular concern paid to the quality of human
performance. Once a deficiency or discrepancy is identified the objective is to
improve the current aviation system and to provide data for future planning and
improvements, system wide. This program identifies potential safety problems while
providing limited protection to airmen reporting under provisions of FAR Part 91. A
paradigm shift from regulatory compliance and punitive-based incentives to
assurances of non-disciplinary action against self-disclosed operational errors was
evolving.
The value of ASRS as an aviation safety research database is recognized
worldwide; however, even though the program processes large quantities of data, its
ability to correct identifiable aviation hazards is severely limited. Data from ASRS
implied that numerous significant events were occurring in the NAS that were
unrecognized by the airlines or the regulators. Yet typically, ASRS could not report
details of specific events back to the airlines or the FAA due to requirements of
confidentiality and jurisdiction. Furthermore, the airmen involved were unlikely to
report this precursor information to anyone other than NASA without greater
regulatory assurances that the report would not be used to precipitate a follow-up
review and potential action taken against them. Therefore, detailed analysis of
incidents that could potentially lead to accidents was not always available to those
who could implement corrective measures. Be that as it may, progress was being
made moving from regulatory compliance and punitive-based incentives to
assurances of non-disciplinary action against self-disclosed operational errors – thus
providing a better analysis tool of the root causes during hazard identification.
Taking the philosophy of non-punitive safety processes one step further was
the Allied Pilots Association, which represents the professional interests of the pilots
serving American Airlines. In 1994, then chairman of the National Safety
Committee, Captain K. Scott Griffith, envisioned a program to provide a program
whereby pilots would actively identify and report safety issues to a tripartite review
comprised of the pilot’s union, management and the FAA. A key element of this
critical information source was confidence that neither the airline nor the FAA would
use those same reports to take enforcement action against an errant pilot. A trial
program was allowed for 18-months based on pilot self-reporting and a cooperative
effort to improve safety through a partnership among the three signatories to the
memorandum of understanding. The program was called ASAP for Aviation Safety
Action Program. ASAP extended the regulatory assurances, so critical to ASRS
success, and further provided for the collection, analysis and retention of safety data
that would otherwise be unavailable to either the FAA or the airline’s management.
ASAP reports did not preclude continued data collection by the human performance
experts at NASA but rather multiplied the effort by providing an additional stream of
information to the ASRS database as all ASAP reports automatically generated an
ASRS report. The Event Review Team comprised of the tripartite would then further
analyze the ASAP report.
Since the only way of achieving a goal of zero accidents was to apply
corrective action before an accident occurred, the objective of ASAP was to have an
unfiltered view of operational hazards with aggressive and timely corrective action.
In order to ensure the benefit of self-reporting, the FAA offered certain non-punitive
enforcement-related incentives to encourage individual employees and the certificateholding airline to report incidents of inadvertent non-compliance with the FARs even
when the infraction went unnoticed. ASAP is based on the principles of identification
and corrective action rather than immunity. The only immunity associated with
ASAP is the relationship existing from participation in the NASA ASRS program,
which is independent of ASAP.
The ASAP program was granted several extensions
over the next five years and proved to be so successful that other airlines have now
adopted this model. ASAP was ultimately incorporated into FAA Advisory Circular
120 – 66A and forms the foundation of self reporting hazard identification programs
which allow safety experts to formulate preventative recommendations based on
quantifiable data gathered in a non-punitive environment.
The Need for Data Driven Processes
Insight from operational flight data recorders and information derived from
crash survivable accident data has proven to be invaluable in the identification of
accident causes and contributing factors. Developed over the past decade, a strong
consensus exists among aviation safety professionals, stating that making safety
improvements based on reactive accident data and to a more limited extent some
incident data, can sustain us at our current level of safety, however it may be
insufficient to adequately anticipate future problems. A need exists for more robust
data, recorded automatically without dependency on a dedicated act or effort of the
observer. This data would then be compiled into aggregate data and plotted in a
manner that highlighted events falling outside some pre-determined normal
distribution, thus pointing to precursors of incidents or accident that could then be
reviewed for systemic concerns. This identification of root causes provides high
leverage potential where future accident prevention plans could be evaluated based on
the numbers of safety events that would be addressed, the severity of those events,
and expected effectiveness of the plan. Using this approach accident prevention
would move beyond tactical to more strategic operation. This far-reaching “macro”
view builds the foundation of a comprehensive safety risk management program.
Flight Operations Quality Assurance: The Pursuit of Knowledge
In January 1995, the Department of Transportation’s Aviation Safety
Conference recommended that the FAA encourage and facilitate the voluntary
implementation of a program using airborne digital recording equipment to record
flight data for routine monitoring of operations. This Flight Operations Quality
Assurance (FOQA) data, when combined with other data and operational experience
would then be used to develop objective information that enhances flight, safety and
maintenance decisions.
FOQA-like programs developed by European air carriers over the past three
decades indicate that FOQA data is a valuable source of information that significantly
enhances aviation safety. Those airlines currently using FOQA agree that the insights
derived from established programs have prevented serious incidents and accidents.
Based on this past operational experience and the recommendations of the Flight
Safety Foundation, the FAA began to consider FOQA programs as a tool for
continuously monitoring and evaluating operational practices and procedures.
Consistent with the lessons learned in ASAP, which offered an alternative to
traditional FAA legal enforcement in proactive safety programs, the FOQA approach
was chosen in 1998 with the announcement of the FAA’s Safer Skies initiative. This
program, based on a comprehensive review of the causes of aviation accidents,
resulted in adoption of a focused priority safety agenda designed to bring about a
five-fold reduction in fatal accidents -- a truly was aggressive goal. With Safer Skies
the FAA agreed to concentrate its resources on the most prevalent causes of aircraft
accidents and vowed to use the latest technology to help analyze U.S. and global data
to find the root causes of accidents.
Safer Skies vowed to use partnerships between the FAA and the aviation
industry. These programs would include ongoing analytical programs to determine
causes of accidents. Once understood, intervention strategies would be developed
and evaluated to determine which mitigating factors provided the largest “bang for
the buck” impact on safety. Once implemented, effectiveness was to be reviewed
during internal evaluations in a feedback loop. Thus the initiative will use data in
new ways that allow operators, manufacturers and the FAA to focus on breaking
causal sequences and taking action before an identified chain of events leads to an
accident.
To support the data requirements of such an effort, a program which used
empirical data such as that recorded on flight data recorders was needed. Unlike post
accident investigations, the program sought to provide decision makers better tools to
manage operational risks before an incident or accident. Through the use of computer
software programs processing massive quantities of data, safety experts would review
distributions of operational activities, thus moving from reacting to mishap
investigation information to acting on precursor data. This results in a proactive
approach focusing on systemic concerns – allowing managers to anticipate problems
avoiding costly surprises. As one Air Line Pilots Association representative put it, “If
you can’t measure it…you don’t know about it. If you don’t know about it...you
can’t fix it.”
A FOQA program will identify latent conditions in flight operations and help
quantify active trends. This trending and their associated change program activities
provide feedback to operational practices and training. Previously unknown
relationships are discovered using atypical event data review empowering “drill
down” analysis. Additionally FOQA programs provide feedback for development of
standard deviations in future risk management of like events -- What we don’t know
can hurt us. FOQA is the Rosetta stone of a proactive safety program, providing the
key to understanding how to optimize system performance.
Conclusion
The aviation industry as a whole must be aware and constantly vigilant of the
pressures exerted on those corporate executives fiscally responsible for conducting
business in the airline industry. Revised capitalization plans, innovative management
practices, maintenance and training savings, are tactics used in an effort to achieve a
competitive business advantage in a marketplace offering a slim two to five percent
profit margin. These executive decisions most certainly are preceded with empirical
data from “bean counters” who fully appreciate the value of a dollar, but rarely
foresee the unintended consequence of reducing or failing to implement a safety layer
of protection.
The benefits of proactive data-driven processes are clear. The National Civil
Aviation Safety Review Commission stated that “whenever possible, FOQA should
become part of safety risk management programs.” Their recommendations went on
to state that FOQA systems have the potential to become the basis for aviation safety
decisions at three levels: the company, the aircrew and the air transportation system
as a whole. They further went on to state that FOQA information on a national level
could identify faults in systems, procedures, airport operations, air space structures,
aircraft certification and human-automation interface. Accomplishing the goal of a
dramatic reduction in the accident rate will require this kind of strategic plan, one that
places emphasis on a cooperative interaction of information sharing and collaborative
development of solutions to safety issues. As an aircraft designer, how will you
ensure your design fits into the FOQA program?