Software Enabled Control Experiments with

AIAA 2005-6956
[email protected]
26 - 29 September 2005, Arlington, Virginia
Software Enabled Control Experiments with
University-Operated Unmanned Aircraft
Eric N. Johnson*, Daniel P. Schrage†, and George Vachtsevanos‡
Georgia Institute of Technology
{Eric.Johnson, Daniel.Schrage}, [email protected]
On August 25, 2004, a series of flight experiments and demonstrations were flown at the
McKenna urban operations complex at Ft. Benning, GA. These experiments represented the
culmination of the rotary wing segment of the DARPA Software Enabled Control program. To
support these efforts, an open system Unmanned Aerial Vehicle testbed architecture was developed
for the GTMax and GTSpy university-operated research aircraft. This paper includes a description
of these systems, and then discusses results from the final experiments. This includes: fault-tolerant
flight control, adaptive flight control, fault detection and accommodation, reconfigurable control,
trajectory generation, envelope protection, vision-aided inertial navigation, vision-based obstacle
avoidance, and the first air-launching of a hovering aircraft.
On August 25, 2004, a series of final experiments
were flown at the McKenna urban operations complex
at Ft. Benning, GA. These experiments represented the
culmination of the rotary wing segment of the DARPA
Software Enabled Control (SEC) program.
objectives of the SEC program include control systems
for Unmanned Aerial Vehicles (UAV), automation for
extreme maneuvers, improved disturbance rejection and
fault tolerance, and to provide reusable middleware for
coordinated, embedded software control on multiple
aircraft types. The rotary wing segment included team
members from the Georgia Institute of Technology,
Draper Laboratory, Scientific Systems Company, Inc.
(SSCI), Vanderbilt University, Honeywell, the Oregon
Graduate Institute, and Boeing. The technologies
included in these final experiments and discussed here
include neural network adaptive control, adaptive mode
transitioning control, fault tolerant control, aggressive
maneuver guidance logic, envelope protection, visionaided inertial navigation, verification & validation
Lockheed Martin Assistant Professor of Avionics
Integration, Daniel Guggenheim School of Aerospace
Professor, Daniel Guggenheim School of Aerospace
Professor, School of Electrical and Computer
Copyright © 2005 by Eric N. Johnson, Daniel P.
Schrage, and George J. Vachtsevanos. Published by the
American Institute of Aeronautics and Astronautics,
Inc., with permission.
approaches, and the Open Control Platform (OCP)
reusable middleware.
This paper begins with a description of the two
university-operated research unmanned rotorcraft that
were utilized for the SEC rotary wing final
experiments: the GTMax, a small helicopter, and the
GTSpy, a micro ducted-fan rotorcraft. Flight test
results from the SEC final experiments follow.
Figure 1 – GTMax Research UAV, primary SEC
rotary-wing final experiment platform, 157 pounds,
10.2 ft diameter rotor
GTMax Research Helicopter
The GTMax, shown in Figure 1, is a helicopterbased UAV platform (based on the Yamaha RMAX
with a 10.2 ft diameter rotor).
The hardware
components that make up the basic flight avionics
include general purpose processing capabilities and
sensing, and add approximately 35 lbs to the basic
airframe for a total weight of 157 pounds as configured.
Copyright © 2005 by Eric N. Johnson, Daniel P. Schrage, and George J. Vachtsevanos. Published by the American Institute of Aeronautics and Astronautics, Inc., with permission.
The interface to the helicopter is via a modified
Yamaha Attitude Control System (YACS) that allows
raw servo commands to be given without modification.
The configuration also included:
266MHz Embedded PC, 500 Mb Flash Drive
(Primary flight computer)
850 MHz Embedded PC, 500 Mb Flash Drive with
frame grabber (Secondary flight computer)
ISIS Inertial Measurement Unit (IMU)
(3 accelerometers, 3 rate gyros)
NovAtel RT-2 Differential GPS (DGPS)
3-Axis magnetometer
Sonar altimeter
Vehicle telemetry
(RPM, Voltage, Remote pilot inputs)
Actuator control interface to YACS
11 Mbps Ethernet data link and an Ethernet switch
RS-232 serial data link
CCD camera
Axis pat/tilt/zoom camera
GPS Module
exchangeable modules: 2 computer modules, the GPS
module, the data link module (wireless Ethernet,
wireless serial, Ethernet switch), and the IMU module.
NovAtel RT-2 GPS Receiver
These modules are placed in a vibration-isolated rack
below the main body of the helicopter, which can be
seen in Figure 1. There is a sonar/magnetometer
assembly at the tail, a power distribution system
including circuit breakers near the module rack, and
mounting points for camera systems and other
components under the nose. The power distribution
system utilizes the onboard generator, which outputs
12V DC. It includes a hot-swappable connection to use
external power. Each component has a dedicated
individual circuit breaker.
Wiring external to the modules consists of RS-232
Serial, Ethernet, and 12V DC only. The wiring diagram
is shown in Figure 2, including a typical configuration
of RS-232, Ethernet, and power wiring. Note the
compartmentalization in modules, dedicated power
regulation within each module, and the interface to the
YACS via multiple serial lines.
A payload deployment mechanism was added to the
back of the helicopter to enable automatic deployment
of a small payload, visible behind the avionics rack in
Figure 1. This same system was also later used for the
air-launching of the GTSpy small ducted fan aircraft.
Sonar Altimeter
Data Link
Flight Computer
Power Distribution
Serial Extension Board
Battery 12V Generator
Primary Flight Computer
Guidance, Navigation, & Control for
Helicopter and Camera
Yamaha Attitude
Control System
RC Receiver YACS IMU
Second Computer
Image Processing and Tracking
RS-232 Serial
DC Power
Figure 2: GTMax wiring diagram, including separation into modules, each with their own power regulation, aircooling, and EMI shielding – The GTSpy system is a simplified version with nearly identical guidance, navigation,
and communication software
GTSpy Small Ducted Fan
The GTSpy, Figure 3, is based on a micro/organic
air vehicle platform (the MASS HeliSpy, 11 inch duct
diameter). The GTSpy avionics architecture is similar
to the GTMax, however greatly miniaturized1, with
custom made processor and primary sensor components
shown in Figure 4.
It does not include the
magnetometer, sonar, radar, second computer, wireless
network, generator, pan/tilt camera, or the Yamaha
electronics. A video transmitter is however added to
enable off-board video processing. Smaller and lighter
GPS and IMU hardware is utilized.
with 7 neural network outputs for each of the 7 degrees
of freedom3,4. The 7 degrees of freedom include the
usual 6 rigid-body degrees of freedom plus a degree of
freedom for rotor RPM.
Simulator tools were developed that run on personal
computers or laptops to support operation as research
aircraft. They include an aircraft model, the aircraft
interface model, and sensor models. The aircraft model
has six rigid-body degrees of freedom plus engine, fuel,
and rotor dynamics. The scene generator is a real-time
3-D graphics window, Figure 5, showing the aircraft
and the terrain, and has additional functionality to aid in
data visualization or use in the UAV Ground Control
Station (GCS)5.
Figure 3 – GTSpy Research UAV, 6 pounds, 11 inch
diameter duct utilizes the MASS Helispy airframe
Figure 5 – Simulator and Ground Control Station
(GCS) interface
Operating system specific software is minimized,
and so most software is common for both the GTMax
and the GTSpy, including GCS, navigation, guidance,
controller, and communications.
To support the
research needs of both of these vehicles, operatingsystem-specific software has been developed to enable
it to operate under the µC operating system as well as
QNX, Windows, and Linux.
Figure 4 – Small autopilot hardware used for small
unmanned rotorcraft such as the GTSpy
Common Software Infrastructure
The baseline navigation system (used for both
vehicles) running on the flight computer is a 17 state
Extended Kalman Filter2. The flight controller is an
adaptive neural network trajectory following controller
The overall GTMax software flight configuration
for the SEC final experiments is illustrated in Figure 6.
The Open Control Platform (OCP) managed software
components operating on the secondary computer,
while the primary flight computer conducted nominal
guidance, navigation, and flight control tasks. For
some specific tests, flight control functionality is
transferred to the secondary computer.
Raw Data
Raw Data
Datalinks to
Ground Control
Station (GCS)
Secondary Computer
Figure 6 – SEC Final Experiments top-level
software architecture for the GTMax; control
functions were transferred to the secondary computer
for several of the tests
Approximately 290 research flight tests have been
conducted on the GTMax. The majority of these were
for SEC program testing. Approximately 15 untethered
flights of the GTSpy have been conducted, including air
deployment from another aircraft. Both aircraft were
utilized in final demonstrations for that program in
August 2004. This section summarizes results from
these activities, including some SEC rotary wing final
experiments results that occurred after August 2004.
Video of many of these tests can be found at:
Most of these results were obtained in four flights at
the McKenna Soldier Battle Lab in Ft. Benning
Georgia. Mission (flight) #1 was an unmanned supply
sustainment mission that included the automated
delivery of supplies to an urban area. Mission #2 was a
Reconnaissance mission.
During both of these
missions, vehicle damage was simulated that resulted in
a change in the course of the mission. The technologies
demonstrated were adaptive control, trajectory
generation, adaptive mode transitioning control, fault
tolerant control, extreme maneuver guidance, visionaided inertial navigation, envelope protection, and
envelope reshaping for fault accommodation. The final
two flights at the McKenna facility were used to
demonstrate the GTMax and the GTSpy research
aircraft individually.
Neural Network Adaptive Control (Georgia Tech)
The adaptive flight control system that tracks
desired trajectories has been tested extensively on the
GTMax3,4 and the GTSpy6. For these tests, a simple
linear model corresponding to the hover flight condition
is the only model information provided a priori to the
system. Online training of the neural network is relied
upon to correct for the resulting model error over the
entire speed flight envelope of the helicopter.
For the GTMax, the neural network adaptive flight
control system was utilized for all takeoffs and
landings, and for any flight conditions in Mission #1
and Mission #2 that required flight control. Figure 1
shows the helicopter departing for one of these missions
utilizing this system. The system had previously been
tested for the complete speed envelope of the aircraft3.
For automatic takeoffs and landings, the navigation
system determines if the helicopter is on the ground by
comparing the altitude above ground level (AGL) with
a pre-selected value. Landings end with a slow vertical
descent command until ground contact is detected and
rotor RPM and collective pitch are reduced to an idle
A key part of mission #1 was an unmanned supply
sustainment element, where the GTMax utilizing the
neural network adaptive flight controller was
commanded to drop off its supply container at a
prescribed location in the urban section of the McKenna
facility at a prescribed time – demonstrating a requiredtime-of-arrival capability (note that chronologically this
event took place after the Active Mode Transition
Controller test described below).
A number of tethered and approximately 15 untethered flights using the small autopilot on the GTSpy
were conducted, all in 2004. One of these flights was at
the August 2004 SEC final experiments. These flights
included operations in some wind gusts (up to about 15
knots airspeed), automatic takeoff and landing.
State Dependent Riccati Equation Control
(Oregon Graduate Institute)
A State Dependent Riccati Equation (SDRE) control
design was tested, providing trajectory-tracking lowlevel control for the helicopter.
This involved
formulation of a system of equations into pseudo-linear
form and on-line iterative solution of a Riccati
An additional nonlinear compensator
replaces traditional trim control for faster dynamic
response and compensation for model error. The
ground track for this segment of the flight is illustrated
in Figure 7.
Adaptive Mode Transition Control (Georgia Tech)
Active Mode Transition Control (AMTC) is
designed to accommodate internal and external
disturbances by enabling smooth transition between
multiple flight control system modes8. An external
disturbance was simulated by modifying state
information provided to the controller in such a way to
emulate a wind shear severe enough that the system
switched smoothly to an alternate control mode. This
flight path segment is also illustrated in Figure 7.
Neural Network
Figure 7 – Ground track for first segments of Mission
#1: Neural network adaptive controller for takeoff,
SDRE controller for large pattern, and AMTC
controller for smaller pattern. Image represents an
area about one third of a mile across.
Aggressive Maneuver Guidance Logic (Draper)
In Aggressive Maneuver Guidance Logic (AMGL),
discrete dynamics-based motion primitives (maneuvers)
capture the capability of the aircraft. Flight logic then
stacks these elemental maneuvers to synthesize a
desired trajectory9. This was demonstrated by flight
over the urban section of the McKenna facility,
including simulated introduction of a sniper fire
position that necessitated aggressive maneuvering to
maintain terrain/obstacle masking to protect the vehicle.
The flight path around the village is illustrated in Figure
Validation and Verification (Honeywell)
A computational verification of the AMGL test was
conducted, where an automated set of simulation runs
were utilized to estimate the limits on wind and
trajectory tracking performance required to achieve a
given confidence level for mission success (as a
probability). This was a demonstration of the capability
to achieve such a test at the mission level, and proved
directly useful by providing a basis for wind limitations
for the AMGL experiment to be conducted10.
Figure 8 – Active Mode Guidance Logic (AMGL)
utilized to patch together maneuver primitives to
guide flight around buildings, including the use of
buildings to protect the vehicle from a sniper location
provided in real-time
Intelligent Reconfigurable Control (SSCI)
An adaptive reconfigurable controller was
developed, which provides on-line failure detection,
identification, and reconfiguration11. This system was
tested by simulating reduced tail rotor effectiveness and
reduced collective-pitch effectiveness. In all cases, the
controller was not given knowledge of the failures. The
failures had to be identified by the subsequent effect on
the dynamics of the aircraft.
A related path planning capability was also tested,
which calculates a set of optimal waypoints as required
to avoid known obstacles or pop-up threats under basic
kinematic/dynamic vehicle-environment constraints
utilizing a quasi-discrete differential games approach.
Fault Tolerant Control (Georgia Tech)
The scenario of a stuck collective pitch actuator was
simulated in flight by limiting the deflection of swash
plate actuators in such a way to prevent changes in
collective pitch. A fault tolerant control module was
developed that generated a rotor RPM command that
allows the existing flight controller to continue to
function, albeit at reduced capability12. A typical flight
test result is illustrated in Figures 9 and 10, where up
and down step responses of 10 feet were performed in
between hover segments. Altitude hold performance is
significantly worse without collective pitch, but still
collective pitch when rolling left or right, which
necessitates corresponding changes in rotor RPM to
maintain vertical tracking. In all cases performance is
degraded, but control is maintained, making a return to
base or revised mission feasible (note that
chronologically this later test was at the end of the
second SEC final experiment mission).
Altitude (ft)
Stuck Right Swashplate Actuator - Level box pattern
-z (feet)
3440 3460 3480
Time (sec)
Figure 9 – Utilizing rotor RPM to accommodate a
simulated stuck collective actuator, step responses;
altitude and altitude command
Rotor Angular Rate (RPM)
x (feet)
y (feet)
Figure 11 – Flying a small box pattern with a
simulated stuck right swash plate actuator;
performance is worse than nominal, but may be
sufficiently good to allow the aircraft to return for
Vision-Aided Inertial Navigation (Georgia Tech)
3440 3460 3480
Time (sec)
Figure 10 – Utilizing rotor RPM to accommodate a
simulated stuck collective actuator, step responses;
measured rotor RPM; note transient in RPM during
climb and descent
For the SEC final experiments, this RPM-control
reconfiguration was utilized to “save” the aircraft after
simulated damage, bringing the aircraft back to the
landing area at low speed and limited to gentle
maneuvers. This ended the first mission of the SEC
final experiments.
Subsequent tests also included simulated failure of
an individual swash-plate actuator, where only
combinations of cyclic and collective pitch were
possible – and rotor RPM utilized to maintain vertical
control throughout. A stuck actuator on the right side
swash plate result is shown in Figure 11. This
particular failure makes it necessary to change
In these tests, a 2-D vision sensor looking at a
selected image target aided an Inertial Navigation
System (INS) on the GTMax13. Target location in the
image and size of the target were used to provide
updates to the INS to bound drift in the position and
velocity estimates.
This was accomplished by
augmenting the existing GTMax navigation system
with the vision input, and simply disregarding GPS
information during tests. This navigation solution was
utilized by the autopilot, resulting in automated flight of
an unmanned rotorcraft without GPS; significant for the
case of GPS failure, denial, or poor signal such as in an
urban scenario. A block diagram of this architecture is
shown in Figure 12.
Several different optical targets have been used,
including an artificial one, and windows on actual
buildings. Typical results are shown for one position
axis (North/South) is in Figure 13, showing
commanded position, estimated, and raw GPS data for
comparison. Pictures from the close approach to a
building at the beginning of the second SEC final
experiment mission are shown in Figure 14. The
approach to the building was utilized to locate a sniper,
who subsequently attempted to flee the area. The
human operator then commanded adjustments to
camera angle and position of the GTMax to follow the
Image Processing+
Computer Vision
Figure 12 – Vision-aided inertial navigation
architecture, where inertial + vision navigation
solution is utilized by the autopilot
Figure 14 – GTMax utilizes vision-aided inertial
navigation to approach a building (top); window
near center of onboard image on the second floor of
the building (bottom) provides position updates; GPS
is not used
Figure 13 – Commanded, estimated, and raw GPS
position for a single axis (GPS data used for
comparison). Vehicle is about 100 ft from the image
Automatic Flight Envelope Protection
(Georgia Tech)
An automatic flight envelope protection system that
utilizes an online trained neural network to predict and
then avoid flight envelope limits was then tested during
simulated evasion of a simulated threat from the
ground. It should be noted that this work was done
utilizing the neural network adaptive flight controller
described previously, demonstrating stability of
simultaneous adaptive limit detection and flight control.
Flight tests conducted to date include avoidance of a
normal load factor limit and a rotor stall prediction
parameter (Erits factor, in units of speed)14. In the SEC
final experiments, the system was utilized to perform a
rapid turn around at high speed while not exceeding
these two limits.
Fault-Adaptive Control (Vanderbilt)
For the previously mentioned stuck cyclic actuator
test, the fault detection function was accomplished
utilizing bond graph models for the actuators and a
diagnosis engine. Results indicated correct detection of
fault in 2-3 seconds. No false positive detections were
experienced in the flight testing phase.
launched from another aircraft. Subsequently, both the
GTMax and GTSpy were maneuvered independently.
Automatic Obstacle Avoidance Using Stereo Vision
A stereo pair of cameras was utilized to
automatically identify an obstacle and reroute the
GTMax around that obstacle during several tests in
September and November 2004. A picture from one
such test is shown in Figure 15.
Figure 16 – GTSpy mounted on the tail of the GTMax
in preparation for air launch
Figure 15 – GTMax with stereo cameras re-plans
route to avoid collision with actual obstacle
Airborne Deployment of a Rotorcraft
(Georgia Tech)
In preparation for an airborne launch of the GTSpy
from the GTMax, the GTSpy was installed on the tail of
the GTMax is shown in Figure 16. An additional
wireless modem was also added to the GTMax to allow
it to relay digital communications between the ground
and the GTSpy.
On November 20, 2004, the GTSpy was dropped
from the GTMax while the GTMax was in an automatic
hover at approximately 400 ft above ground level. The
GTSpy fell tail first until it was able to arrest the
descent and smoothly went into a hover 80 ft below the
GTMax. A picture of both aircraft immediately after
the separation is shown in Figure 17. It is believed that
this is first time a hovering aircraft was ever air
Figure 17 – GTSpy is dropped from hovering GTMax
(sequence of four images); GTSpy subsequently
hovered 80 feet below the GTMax; This is believed to
be the first time a hovering aircraft has ever been air
This paper described flight test results obtained
from two university-operated research unmanned
rotorcraft: the GTMax, a small helicopter, and the
GTSpy, a micro ducted-fan rotorcraft, in final
experiments of the DARPA Software Enabled Control
program. An open system UAV testbed architecture
was described for the two aircraft. The testbed
architecture enabled a wide array of different
approaches to be tested safely and effectively. Flight
test results included: fault-tolerant adaptive flight
control, fault detection and accommodation,
reconfigurable control, envelope protection, visionaided inertial navigation, vision-based obstacle
avoidance, and the first air-launching of a hovering
aircraft. What these results have in common is that
they have the potential to enable future unmanned
systems to be more reliable through improved vehicle
This work was supported by the DARPA Program
under contracts #33615-98-C-1341 and #33615-99-C1500, with John Bay as Program Manager. SEC final
experiments included technologies developed under
separate Software Enabled Control (SEC) program
contracts at: Boeing, Oregon Graduate Institute, The
Charles Stark Draper Laboratory, Scientific Systems
Company, Honeywell, and Vanderbilt University. The
authors would also like to acknowledge some of the
many other important contributors to this work at
Georgia Tech: Henrik Christophersen, Scott Clements,
J. Eric Corban, Graham Drozeski, Luis Gutierrez,
Murat Guler, Bonnie Heck, Jeong Hur, Phillip Jones,
Suresh Kannan, Adrian Koller, Eric Martens, Alex
Moodie, Wayne Pickell, J.V.R. Prasad, Alison Proctor,
Nimrod Rooz, Bhaskar Saha, Michael Turbe, Suraj
Unnikrishnan, Linda Wills, Allen Wu, and Ilkay
Christophersen, H., Pickell, W., Koller, A.,
Kannan, S., and Johnson, E., “Small Adaptive Flight
Control Systems for UAVs using FPGA/DSP
Technology,” Proceedings of the AIAA “Unmanned
Unlimited” Technical Conference, Workshop, and
Exhibit, 2004.
Dittrich, J., and Johnson, E., “Multi-Sensor
Navigation System for an Autonomous Helicopter,”
Digital Avionics Systems Conference, 2002.
Johnson, E. and Kannan, S., “Adaptive Flight
Control for an Autonomous Unmanned Helicopter,”
Proceedings of the AIAA Guidance, Navigation, and
Control Conference, 2002.
Kannan, S. and Johnson, E., “Adaptive Trajectorybased Flight Control for Autonomous Helicopters,”
Proceedings of the 21st Digital Avionics Systems
Conference, 2002.
Johnson, E. and Schrage, D., “System Integration
and Operation of a Research Unmanned Aerial
Vehicle,” AIAA Journal of Aerospace Computing,
Information, and Communication, Vol. 1, No. 1,
January 2004.
Johnson, E. and Turbe, M., “Modeling, Control,
and Flight Testing of a Small Ducted Fan Vehicle,”
Proceedings of the AIAA Guidance, Navigation, and
Control Conference, 2005.
Bogdanov, A., Wan, E., and Harvey, G., “SDRE
Flight Control for X-Cell and R-Max Autonomous
Helicopters,” Proceedings of the IEEE Conference on
Decision and Control, 2004.
Gutierrez, L., Vachtsevanos, G., and Heck, B., “An
Approach to Adaptive Mode Transition Control of
Unmanned Aerial Vehicles,” Proceedings of the
American Control Conference, 2003.
Singh, L., Plump, J., McConley, M., and Appleby,
B., “Software Enabled Control: Autonomous Agile
Guidance and Control for a UAV in Partially Unknown
Urban Terrain,” Proceedings of the AIAA Guidance,
Navigation, and Control Conference and Exhibit, 2003.
Elgersma, M., Ganguli, S., Ha., V., and Samad, T.,
Autonomous Rotorcraft,” Proceedings of the IEEE
International Symposium on Intelligent Control, 2004.
Garagić, D., Bošković, J. and Mehra, R., “Robust
Nonlinear Fault-Tolerant Control of a Small-Scale
Helicopter”, Proceedings of the American Helicopter
Society 60th Annual Forum, 2004.
Drozeski, G., Saha, B., and Vachtsevanos, G., “A
Fault-Tolerant Architecture for Unmanned Rotorcraft,”
Proceedings of the AHS International Specialists’
Meeting on Unmanned Rotorcraft, Chandler AZ,
January 18-20, 2005.
Wu, A., Johnson, E., and Proctor, A., “VisionAided Inertial Navigation,” Proceedings of the AIAA
Guidance, Navigation, and Control Conference, 2005.
Yavrucuk, I., Prasad, J.V.R., Calise, A., and
Unnikrishnan, S., ”Adaptive Limit Control Margin
Prediction and Avoidance,“ 58th AHS Annual Forum,