Postgraduate Research Students

advertisement
Intelligent Techniques for Improving the
Aviation Operations
Professor Lakhmi C. Jain
PhD, ME, BE(Hons), Fellow (Engineers Aust),
KES Founder
http://www.kesinternational.org/organisation.php
Courtesy: Universal press
Adelaide City
4
UNIVERSITY OF SOUTH AUSTRALIA
Education Adelaide
Where are we?
Australia
6
UNIVERSITY OF SOUTH AUSTRALIA
Education Adelaide
KES International
www.kesinternational.org
 KES regularly provides networking opportunities
for professional community through one of the
largest conferences of its kind in the area of
KES.
KES Conferences
1997-99
2000
2001
2002
2003
2004
2005
2006
2007
2008
Adelaide, Australia
Brighton, UK
Osaka, Japan
Crema, Italy
Oxford, UK
New Zealand
Australia
UK
Italy
Croatia
2009
2010
2011
2012
2013
2014
2015
2016
Chile
UK
Germany
Spain
Japan
Poland
Singapore
UK
2015 KES Conferences
 Innovations through Knowledge Transfer
2015
 Agents and Multi-Agent Systems 2015
 Sustainability in Energy and Buildings
2015
 Intelligent decision Technologies 2015
 Intelligent Interactive Multimedia Systems
and Services 2015
2015 KES Conferences
 Innovations in Medicine and Healthcare
2015
 Knowledge-Based and Intelligent
Information and Information Systems 2015
 Archiving for the Future 2015
 Innovations in Music 2015
KES Journals
 International Journal of Knowledge-Based
Intelligent Engineering Systems, IOS Press,
The Netherlands
 http://www.iospress.nl/html/13272314.php

 Intelligent Decision Technologies, IOS Press,
The Netherlands, 2007-present.
 http://www.iospress.nl/loadtop/load.php?isbn=187249
81
BOOK SERIES
KES members are involved in book series
published by leading publishers.
 Smart Innovation, Systems and
Technologies:
www.springer.com/series/8767
 Intelligent Systems Reference Library:
www.springer.com/series/8578
 Advanced Information and Knowledge
Processing: www.springer.com/series/4738
University of South Australia
Intelligent Flight Data Monitoring System for
Improving the Safety of Aviation Operations
.
University of South Australia
AIMS and Objectives
 Develop an intelligent Flight Data Monitoring (FDM) system
incorporating supervised learning
 Identify adverse trends and potential unsafe deviations from the
accepted norms of flight operations
 Provide early warning signals to flight crew, and/or air traffic control
staff
 Explanation to flight crew as to why the current flight operation is
considered safe/unsafe
Specific Objectives
 To acquire flight data and other relevant information for flight operations
and safety
 Develop CI-based algorithms with incremental learning capabilities to
categorise flight data and perform risk prediction
 To extract rules to explain the prediction from the CI-based algorithms
 To evaluate the effectiveness of the FDM system for aviation
operations and safety.
University of South Australia
Background
 Air travel in modern turbine passenger aircraft has become
extremely safe.
 This has largely been attributed to increased mechanical reliability,
increased reliability of on-board automated systems and the wide
spread development and implementation of flight crew training in
Crew Resource Management (CRM) .
University of South Australia
Background
 However, even with all these advances in aviation safety there remains
a stubborn remnant of air crashes which are seemingly not eradicable.
 Of these accidents, worldwide, Helmreich and Foushee have
suggested that 70% are due to flight crew actions or in some cases
inactions. HUMAN ERROR.
 This is despite the fact that pilots are extremely technically competent
and well trained in CRM.
 There is no question that flight crews are highly trained to operate in
the technical and human environments of the cockpit.
University of South Australia
Background
 Why do such accidents happen and, perhaps more
disturbingly, why do they continue to happen?
 Research has shown that most are due to a
momentary loss of concentration or awareness
during which the flight crew did not consciously notice
that a necessary event did not occur, or that an
adverse event did occur.
University of South Australia
Background
 In order to function pilots build up a mental model of
what is happening around them.
 New information they receive will be perceived and
restructured in terms of this model until an event
happens which forces an unsettling recognition that
the model is actually false. This is termed loss of
situation awareness (SA).
 If this happens too late on in a critical process, the
result can be an adverse event.
University of South Australia
Background
 One solution to the problem is increased automation.
 However despite the high reliability and accurate flight path
control automation can actually decrease a flight crew’s
“awareness of parameters critical to flight path control through
out-of-the-loop performance decrements, over-reliance on
automation, and poor human monitoring capabilities.
 Weiner describes reports of pilots unintentionally creating flight
paths to wrong locations which went undetected and resulted in
collision with a mountain.
 This type of accident is referred to as a controlled-flight-into-
terrain accident or CFIT.
University of South Australia
The Statistics
CFIT and approach-and-landing accidents (ALAs)
accounted for 80% of the fatalities in commercial transportaircraft accidents (Flight Safety Foundation, Recent Study).
The FSF Approach-and-landing Accident Reduction Task
Force Report concluded that the two primary causal
factors for such accidents are:
“omission of action/inappropriate action” and
“loss of positional awareness in the air”.
University of South Australia
Current Technology
 In 1974 the Federal Aviation Administration (FAA) mandated that all heavy
airliners be fitted with a GPWS. This has lead to a decrease in CFIT
accidents however there continues to be a relatively large number of
fatalities attributed to ALA or CFIT accidents.
 The GPWS uses information from the radar altimeter and air data computer
to determine the aircraft’s vertical distance from the terrain below. The
system is limited because it only perceives vertical separation between the
aircraft and the ground directly below the aircraft in real time.
 Since 2003 the GPWS has been replaced by Enhanced GPWS (EGPWS)
on all turbine aircraft with 10 or more passenger seats. The EGPWS has a
predictive terrain hazard warning function.
University of South Australia
Current Technology
How does it work?
 The EGPWS compares the aircraft’s position and altitude derived from the
Flight Management and Air Data computers with a 20MB terrain database. In
the terrain database the majority of the Earth’s surface is reduced to a grid of
9x9 km squares. Each square is given a height index. In the vicinity of
airports the grid resolution is increased to squares of 400m x 400m. The
height index and the aircraft’s predicted 3 dimensional position 20 to 60
seconds into the future are compared to see if any conflict exists.
 If it does the EGPWS displays an alert or warning to the flight crew. Other than
to initially alert the pilots of “TERRAIN” up to 40-60 s before impact or warn the
pilots to “PULL UP” up to 20-30 s before impact it does not offer any other
solution to the potential problem.
Proposed Solution
 Develop a Flight Data Monitoring System.
 It involves collecting and analysing data recorded during flight
operations
 Identifying and rectifying adverse trends and deviations from accepted
norms of flight operations and safety
 Understanding flight operations by tracking trends and detecting flaws
before they lead to major incidents
 Developing preventive and/or corrective actions such as increased
training.
Flight Data Monitoring Systems
 A number of commercial products are
available in the market.
 Battele, USA
 Teledyne, USA
 CEFA, France
 Sagem, France
 Flightscape, Canada
 (These products do not incorporate online learning capabilities)
Flight Data Monitoring Systems
 Most successful work reported thus far is by Battele and NASA. It is a
software tool that aggregates large volumes of flight data and then uses
statistical cluster-based techniques to find the unexpected or the
abnormal events.
 NASA Ames research Centre also reported a clustering-based
techniques to find the expected or the abnormal events.
Proposed Solution
 Computational Intelligence (CI)-based to process and analyse flight
data
 Ability to incrementally learn and absorb knowledge of aviation
instructors/trainers
 A variant in the family of Adaptive Resonance Theory (ART) called
Fuzzy ARTMAP (FAM) is under investigation
Fuzzy ARTMAP (FAM)
 FAM is a supervised neural network
 It is able to perform on-line or off-line learning
 FAM is an integration of Neural net and Fuzzy system. This integration
brings the learning capability of neural net and the reasoning capability
of fuzzy system.
 These are the reasons of selecting FAM as a core engine of the FDM.
FAM Architecture
Input pattern
Target Class
Methodology
 FAM network monitors the flight data and identify normal as well as a






typical flight operations.
Data is related to flight dynamics, aircraft status, weather and
environmental variables.
The FAM network examines these data as a combined flight pattern
and yields a predicted risk pertaining to whether the flight pattern is
within or outside the normal operating conditions.
In FAM, each recognition category, which is associated with a target
output, corresponds to an If-Then rule. The features stored in each
recognition category can be expressed directly as rule antecedents.
There is an increase in rules and recognition categories with time.
It is proposed to use a pruning strategy to remove insignificant and
noisy recognition categories.
It is planned to tag each rule with a confidence factor to reflect its
significance.
The Operation Phase of FDM
 Flight crew performs some flight operations. Flight data as well as
relevant environmental and weather information are captured and these
form the input to the FDM.
 If the input excites a recognised category (due to previous learning) in
the FDM knowledge base, then a prediction (safe or unsafe flight
pattern) with the corresponding If-Then explanation is retrieved and
displayed to the user.
 However, if the input pattern does not excite any existing recognition
categories in the FDM knowledge base, then the user is so informed of
the unknown flight pattern.
The Supervised Learning Phase of FDM
 Flight patterns recorded from the flight crew during the operation phase




are used for learning with the FAM model.
The flight patterns, together with the associated predictions, are first
evaluated, one-by-one, by domain experts (qualified flight
instructors/check-and-training pilots).
Update of the FDM knowledge base can only be initiated if and only if a
prediction pertaining to a particular flight pattern is confirmed by domain
experts.
The FDM system is able to continue learning and absorbing new flight
patterns into its knowledge base, even after its development.
The capability of supervised learning initiated by domain experts is
made possible by the characteristics of the FAM model in overcoming
the stability-plasticity dilemma, i.e. absorbing knowledge continually
without corrupting its previously acquired knowledge.
Characteristics
 In order for the system to be reliable it must be have a large database
from which to learn and make classification decisions of safe/unsafe,
and this database must be expandable.
 In human terms, the system must be able to know the difference
between a safe and unsafe flight situation, and have the ability to learn
whether a new and previously unseen operation is safe.
 Both of these deployment requirements are met through the use of
supervised learning by domain experts.
University of South Australia
Pilots Fear Automation!
An In-flight agent to monitor
pilot situation awareness
Air crash examples
Case 1:
Eastern airlines flight 401, 29 December 19721,
 Departed from John F Kennedy International Airport New York at
9:20p.m.
 Started the Approach to Miami Airport at approx. 11:30p.m.
 Captain noticed nose gear indicator light problem
 Started cycling around the airport at 2000ft
 First officer switched on the Auto pilot
 Captain , Flight engineer and First officer started resolving the problem.
 Altitude hold mode was accidentally switched off
 Started slow descent
 Crash
Reason for the crash: The crew were concentrating on the nose gear
indicator light, failed to realize the drop in altitude.
1. NTSB report ,1973,http://www.airdisaster.com/reports/ntsb/AAR73-14.pdf
Air crash examples contnd..
Air crash examples contnd..
Air crash examples
Case 2:
Kenyan airways Flight KQA507 , 5 May 20072
 Took off from Douala to Nairobi at 23:06 on 5th of May
 After reaching 1000 ft, started losing height, went unnoticed by the
pilots
 Pilots assumed that the auto pilot was engaged
 Because of the bad weather & dark night there was no visual reference,
still no instrument scanning was done.
 Erratic inputs from the pilot in last few seconds resulted in rapid loss of
height and crash. There were no survivors in the plane.
 Reason for the crash : loss of control by the crew as a result of spatial
disorientation, failing to perform instrument scanning in the absence of
external visual references.
2. http://www.ccaa.aero/surete-et-securite-aerienne-141/aviation/actualite/384,technical-investigation-.html
Air crash examples contnd..
Case 3:
Turkish Airlines Flight, 25 February 20093

Started Approach to Schilpol Airport at approx. 10:14a.m.

Left hand radio altimeter showed wrong altitude reading (-8ft)

Auto pilot was on and responding to the change Auto throttle reduced
the engine power

The flight crew failed to the control column manually

The pitch altitude increased but the airspeed was decreasing

Crash

Reason for the crash: The Crew failed to recognize the reduction in
airspeed & decrease in thrust setting. When the speed decreased at a
height of 750 feet, pilots should have noticed the speed & height if they
had scanned the airspeed indicator and the artificial horizon indicator
respectively on the primary flight displays.
3.http://aviation-safety.net/database/record.php?id=20090225-0
Air crash examples
Case 4:
Air France Flight 447, 1 June 20094
 The aircraft disappeared over the mid-Atlantic without providing any
clue about the cause of accident.
 The investigation which took over two years, revealed that the captain
was confused when the auto pilot was disengaged.
 The airspeed was very low , instead of pushing the control forward the
pilot pulled it back resulting into stall.
 The copilot tried to take over, he was not successful in comprehending
the situation.
 The pilots lost control of the aircraft and it crashed in to the midAtlantic.
 Reason for the crash: Pilots failed to cross- check the instruments even
after receiving the auto pilot disengaged warning. Though there was no
major mechanical failure and accident would have been avoided,
repeated mistakes of pilots resulted in the fatal air crash which claimed
228 lives.
4. http://www.popularmechanics.com/technology/aviation/crashes/what-really-happened-aboard-air-france-447-6611877
Major Cause of All
Four Crashes
LOSS OF SITUATION
AWARENESS
What is situation Awareness (SA)?
 Situation awareness is the ability to perceive
information from the environment, understanding the
meaning and predicting future events based on these
factors.(Endsley,1995)
Limitations
 A number of these systems try to detect
conflicts but do not resolve them.
 Existing inflight systems overload pilot with
information on top of already existing
complex system.
What has not been done
Some of the key questions that need to be
addressed in SA research should include:
 How to monitor a pilot’s actions?
 What parameters are needed?
 How to translate an action into cognition?
Monitoring cognition , attention
Cognition and attention can be monitored by
observing:
- Brain waves
- Heart rate
- Body temperature or
- Eye movement
- Facial expression
Eye movement tracking and facial expression
monitoring can be done non-intrusively, but all
other methods require devices to be attached to
the human body.
Experimental Setup
X-plane
Flight
simulator
setup
faceLAB5
Eye tracker
Sample data collection
System Diagram
Flight Instrument Scan
 It is observed from accident databases that inappropriate




instrument scan have led to fatal flight accidents.
Instrument scanning skills are very important for instrument pilots.
Cross -checking the readings of all the instruments together to
make a collective decision is an important aspect of flight
instrument scan in instrument flying.
As a result any single instrument is not fixated for longer than a
defined threshold.
Federal Aviation Administration(FAA) has classified instruments
as primary and supporting instruments for each flight maneuver.
Flight Maneuver vs Instrument to Scan
Instrument Role
Flight Maneuver
Primary
Supporting
(Secondary)
Straight-And-Level
ALT/DG
AI/VSI/TC
Constant Airspeed
Climb/Descent
ASI/DG
AI/VSI/TC
Standard Rate Turn
ALT/TC
AI/VSI
AI - Attitude Indicator, DG - Directional Gyro, ALT – Altimeter, VSI - Vertical Speed
Indicator, ASI - Airspeed Indicator, TC - Turn Coordinator
Example Instrument Scan
Radial instrument scan (From FAA Instrument flying hand book)
Parameters
The main parameters to be observed in scan pattern of
a pilot are:
 Average fixation on instruments, which are required
to be scanned during a specific scenario.
 Scanning frequency on instruments.
 Gaze transitions between instruments.
 Rate of change during transitions.
Approach
A prototype has been developed which derives the following
information from the metadata
 Which instruments were scanned.?
 Fixation duration on each instrument.
 When was the instrument scanned last time.?
 How long an instrument was not observed after last scan.?
 If all of the expected instruments were scanned?
 If any of the expected instruments were not scanned.?
 Gaze transition from instrument
 Number of occurrence of each transition
The information derived was used in an inference engine which
used rule base systems to compare an expert pi
lot’s gaze pattern with several novices gaze pattern.
Application
 The current project can be used inflight to
alert the pilot when there is a possibility of
loss of SA.
 The system can be used as an alternative SA
evaluation technique, since the system is
completely automated it does not require a
human supervisor.
 The pilot behavior can be recorded during
flight or flight training and can be used for
analysis purposes.
Conclusion
Though the aviation is considered one of the safest mode of
transportation, the aviation accident examples from various
transport safety boards explain how the human errors have caused
loss and fatalities.
This research study shows that the pilot behavior can be classified
into safe or unsafe based on the instrument scan pattern and the
potential loss of SA can be detected before the mishaps.
The focus is to detect the error by monitoring the instrument crosscheck pattern. The other causes of loss of SA are not considered.
Conclusion
 Ladies and Gentlemen. Let me conclude with
one final verse, this time from Sanskrit.
Which can be interpreted to mean
 Do your duty, without expecting a reward
 Your reward will come, in due course
Download