AIAA 2005-6956 Infotech@Aerospace 26 - 29 September 2005, Arlington, Virginia Software Enabled Control Experiments with University-Operated Unmanned Aircraft Eric N. Johnson*, Daniel P. Schrage†, and George Vachtsevanos‡ Georgia Institute of Technology {Eric.Johnson, Daniel.Schrage}@ae.gatech.edu, George.Vachtsevanos@ece.gatech.edu On August 25, 2004, a series of flight experiments and demonstrations were flown at the McKenna urban operations complex at Ft. Benning, GA. These experiments represented the culmination of the rotary wing segment of the DARPA Software Enabled Control program. To support these efforts, an open system Unmanned Aerial Vehicle testbed architecture was developed for the GTMax and GTSpy university-operated research aircraft. This paper includes a description of these systems, and then discusses results from the final experiments. This includes: fault-tolerant flight control, adaptive flight control, fault detection and accommodation, reconfigurable control, trajectory generation, envelope protection, vision-aided inertial navigation, vision-based obstacle avoidance, and the first air-launching of a hovering aircraft. ∗†. INTRODUCTION On August 25, 2004, a series of final experiments were flown at the McKenna urban operations complex at Ft. Benning, GA. These experiments represented the culmination of the rotary wing segment of the DARPA Software Enabled Control (SEC) program. The objectives of the SEC program include control systems for Unmanned Aerial Vehicles (UAV), automation for extreme maneuvers, improved disturbance rejection and fault tolerance, and to provide reusable middleware for coordinated, embedded software control on multiple aircraft types. The rotary wing segment included team members from the Georgia Institute of Technology, Draper Laboratory, Scientific Systems Company, Inc. (SSCI), Vanderbilt University, Honeywell, the Oregon Graduate Institute, and Boeing. The technologies included in these final experiments and discussed here include neural network adaptive control, adaptive mode transitioning control, fault tolerant control, aggressive maneuver guidance logic, envelope protection, visionaided inertial navigation, verification & validation ∗ Lockheed Martin Assistant Professor of Avionics Integration, Daniel Guggenheim School of Aerospace Engineering † Professor, Daniel Guggenheim School of Aerospace Engineering ‡ Professor, School of Electrical and Computer Engineering Copyright © 2005 by Eric N. Johnson, Daniel P. Schrage, and George J. Vachtsevanos. Published by the American Institute of Aeronautics and Astronautics, Inc., with permission. approaches, and the Open Control Platform (OCP) reusable middleware. This paper begins with a description of the two university-operated research unmanned rotorcraft that were utilized for the SEC rotary wing final experiments: the GTMax, a small helicopter, and the GTSpy, a micro ducted-fan rotorcraft. Flight test results from the SEC final experiments follow. Figure 1 – GTMax Research UAV, primary SEC rotary-wing final experiment platform, 157 pounds, 10.2 ft diameter rotor GTMax Research Helicopter The GTMax, shown in Figure 1, is a helicopterbased UAV platform (based on the Yamaha RMAX with a 10.2 ft diameter rotor). The hardware components that make up the basic flight avionics include general purpose processing capabilities and sensing, and add approximately 35 lbs to the basic airframe for a total weight of 157 pounds as configured. Copyright © 2005 by Eric N. Johnson, Daniel P. Schrage, and George J. Vachtsevanos. Published by the American Institute of Aeronautics and Astronautics, Inc., with permission. The interface to the helicopter is via a modified Yamaha Attitude Control System (YACS) that allows raw servo commands to be given without modification. The configuration also included: • • • • • • • • • • • • 266MHz Embedded PC, 500 Mb Flash Drive (Primary flight computer) 850 MHz Embedded PC, 500 Mb Flash Drive with frame grabber (Secondary flight computer) ISIS Inertial Measurement Unit (IMU) (3 accelerometers, 3 rate gyros) NovAtel RT-2 Differential GPS (DGPS) 3-Axis magnetometer Sonar altimeter Vehicle telemetry (RPM, Voltage, Remote pilot inputs) Actuator control interface to YACS 11 Mbps Ethernet data link and an Ethernet switch RS-232 serial data link CCD camera Axis pat/tilt/zoom camera IMU Module GPS Module These components were packaged into exchangeable modules: 2 computer modules, the GPS module, the data link module (wireless Ethernet, wireless serial, Ethernet switch), and the IMU module. NovAtel RT-2 GPS Receiver These modules are placed in a vibration-isolated rack below the main body of the helicopter, which can be seen in Figure 1. There is a sonar/magnetometer assembly at the tail, a power distribution system including circuit breakers near the module rack, and mounting points for camera systems and other components under the nose. The power distribution system utilizes the onboard generator, which outputs 12V DC. It includes a hot-swappable connection to use external power. Each component has a dedicated individual circuit breaker. Wiring external to the modules consists of RS-232 Serial, Ethernet, and 12V DC only. The wiring diagram is shown in Figure 2, including a typical configuration of RS-232, Ethernet, and power wiring. Note the compartmentalization in modules, dedicated power regulation within each module, and the interface to the YACS via multiple serial lines. A payload deployment mechanism was added to the back of the helicopter to enable automatic deployment of a small payload, visible behind the avionics rack in Figure 1. This same system was also later used for the air-launching of the GTSpy small ducted fan aircraft. HMR-2300 Magnetometer 5V ISIS-IMU 5V DC/DC 12V DC/DC Sonar Altimeter Auxiliary Module Data Link Module Flight Computer Module Power Distribution Module Serial Extension Board Battery 12V Generator Primary Flight Computer Guidance, Navigation, & Control for Helicopter and Camera Aironet MC4800 5V DC/DC 12V Yamaha Attitude Control System Ethernet Hub Freewave DGR-115 RC Receiver YACS IMU Second Computer Image Processing and Tracking 5V DC/DC 12V Pan/Tilt Camera actuators RS-232 Serial Ethernet DC Power Figure 2: GTMax wiring diagram, including separation into modules, each with their own power regulation, aircooling, and EMI shielding – The GTSpy system is a simplified version with nearly identical guidance, navigation, and communication software GTSpy Small Ducted Fan The GTSpy, Figure 3, is based on a micro/organic air vehicle platform (the MASS HeliSpy, 11 inch duct diameter). The GTSpy avionics architecture is similar to the GTMax, however greatly miniaturized1, with custom made processor and primary sensor components shown in Figure 4. It does not include the magnetometer, sonar, radar, second computer, wireless network, generator, pan/tilt camera, or the Yamaha electronics. A video transmitter is however added to enable off-board video processing. Smaller and lighter GPS and IMU hardware is utilized. with 7 neural network outputs for each of the 7 degrees of freedom3,4. The 7 degrees of freedom include the usual 6 rigid-body degrees of freedom plus a degree of freedom for rotor RPM. Simulator tools were developed that run on personal computers or laptops to support operation as research aircraft. They include an aircraft model, the aircraft interface model, and sensor models. The aircraft model has six rigid-body degrees of freedom plus engine, fuel, and rotor dynamics. The scene generator is a real-time 3-D graphics window, Figure 5, showing the aircraft and the terrain, and has additional functionality to aid in data visualization or use in the UAV Ground Control Station (GCS)5. Figure 3 – GTSpy Research UAV, 6 pounds, 11 inch diameter duct utilizes the MASS Helispy airframe Figure 5 – Simulator and Ground Control Station (GCS) interface Operating system specific software is minimized, and so most software is common for both the GTMax and the GTSpy, including GCS, navigation, guidance, controller, and communications. To support the research needs of both of these vehicles, operatingsystem-specific software has been developed to enable it to operate under the µC operating system as well as QNX, Windows, and Linux. Figure 4 – Small autopilot hardware used for small unmanned rotorcraft such as the GTSpy Common Software Infrastructure The baseline navigation system (used for both vehicles) running on the flight computer is a 17 state Extended Kalman Filter2. The flight controller is an adaptive neural network trajectory following controller The overall GTMax software flight configuration for the SEC final experiments is illustrated in Figure 6. The Open Control Platform (OCP) managed software components operating on the secondary computer, while the primary flight computer conducted nominal guidance, navigation, and flight control tasks. For some specific tests, flight control functionality is transferred to the secondary computer. Actuator Raw Data Sensor Raw Data Sensor Interfaces OCP & Control Components Sensor Data Navigation Filter State Estimate Flight Controller Control Actuator Interface Datalinks to Ground Control Station (GCS) Draper Guidance Flight AMTC Envelope Controller Protect Secondary Computer Figure 6 – SEC Final Experiments top-level software architecture for the GTMax; control functions were transferred to the secondary computer for several of the tests RESULTS Approximately 290 research flight tests have been conducted on the GTMax. The majority of these were for SEC program testing. Approximately 15 untethered flights of the GTSpy have been conducted, including air deployment from another aircraft. Both aircraft were utilized in final demonstrations for that program in August 2004. This section summarizes results from these activities, including some SEC rotary wing final experiments results that occurred after August 2004. Video of many of these tests can be found at: http://uav.ae.gatech.edu. Most of these results were obtained in four flights at the McKenna Soldier Battle Lab in Ft. Benning Georgia. Mission (flight) #1 was an unmanned supply sustainment mission that included the automated delivery of supplies to an urban area. Mission #2 was a Reconnaissance mission. During both of these missions, vehicle damage was simulated that resulted in a change in the course of the mission. The technologies demonstrated were adaptive control, trajectory generation, adaptive mode transitioning control, fault tolerant control, extreme maneuver guidance, visionaided inertial navigation, envelope protection, and envelope reshaping for fault accommodation. The final two flights at the McKenna facility were used to demonstrate the GTMax and the GTSpy research aircraft individually. Neural Network Adaptive Control (Georgia Tech) The adaptive flight control system that tracks desired trajectories has been tested extensively on the GTMax3,4 and the GTSpy6. For these tests, a simple linear model corresponding to the hover flight condition is the only model information provided a priori to the system. Online training of the neural network is relied upon to correct for the resulting model error over the entire speed flight envelope of the helicopter. For the GTMax, the neural network adaptive flight control system was utilized for all takeoffs and landings, and for any flight conditions in Mission #1 and Mission #2 that required flight control. Figure 1 shows the helicopter departing for one of these missions utilizing this system. The system had previously been tested for the complete speed envelope of the aircraft3. For automatic takeoffs and landings, the navigation system determines if the helicopter is on the ground by comparing the altitude above ground level (AGL) with a pre-selected value. Landings end with a slow vertical descent command until ground contact is detected and rotor RPM and collective pitch are reduced to an idle state. A key part of mission #1 was an unmanned supply sustainment element, where the GTMax utilizing the neural network adaptive flight controller was commanded to drop off its supply container at a prescribed location in the urban section of the McKenna facility at a prescribed time – demonstrating a requiredtime-of-arrival capability (note that chronologically this event took place after the Active Mode Transition Controller test described below). A number of tethered and approximately 15 untethered flights using the small autopilot on the GTSpy were conducted, all in 2004. One of these flights was at the August 2004 SEC final experiments. These flights included operations in some wind gusts (up to about 15 knots airspeed), automatic takeoff and landing. State Dependent Riccati Equation Control (Oregon Graduate Institute) A State Dependent Riccati Equation (SDRE) control design was tested, providing trajectory-tracking lowlevel control for the helicopter. This involved formulation of a system of equations into pseudo-linear form and on-line iterative solution of a Riccati Equation7. An additional nonlinear compensator replaces traditional trim control for faster dynamic response and compensation for model error. The ground track for this segment of the flight is illustrated in Figure 7. Adaptive Mode Transition Control (Georgia Tech) Active Mode Transition Control (AMTC) is designed to accommodate internal and external disturbances by enabling smooth transition between multiple flight control system modes8. An external disturbance was simulated by modifying state information provided to the controller in such a way to emulate a wind shear severe enough that the system switched smoothly to an alternate control mode. This flight path segment is also illustrated in Figure 7. AMGL 5 SDRE 2 AMTC 3 1 Neural Network Figure 7 – Ground track for first segments of Mission #1: Neural network adaptive controller for takeoff, SDRE controller for large pattern, and AMTC controller for smaller pattern. Image represents an area about one third of a mile across. Aggressive Maneuver Guidance Logic (Draper) In Aggressive Maneuver Guidance Logic (AMGL), discrete dynamics-based motion primitives (maneuvers) capture the capability of the aircraft. Flight logic then stacks these elemental maneuvers to synthesize a desired trajectory9. This was demonstrated by flight over the urban section of the McKenna facility, including simulated introduction of a sniper fire position that necessitated aggressive maneuvering to maintain terrain/obstacle masking to protect the vehicle. The flight path around the village is illustrated in Figure 8. Validation and Verification (Honeywell) A computational verification of the AMGL test was conducted, where an automated set of simulation runs were utilized to estimate the limits on wind and trajectory tracking performance required to achieve a given confidence level for mission success (as a probability). This was a demonstration of the capability to achieve such a test at the mission level, and proved directly useful by providing a basis for wind limitations for the AMGL experiment to be conducted10. Figure 8 – Active Mode Guidance Logic (AMGL) utilized to patch together maneuver primitives to guide flight around buildings, including the use of buildings to protect the vehicle from a sniper location provided in real-time Intelligent Reconfigurable Control (SSCI) An adaptive reconfigurable controller was developed, which provides on-line failure detection, identification, and reconfiguration11. This system was tested by simulating reduced tail rotor effectiveness and reduced collective-pitch effectiveness. In all cases, the controller was not given knowledge of the failures. The failures had to be identified by the subsequent effect on the dynamics of the aircraft. A related path planning capability was also tested, which calculates a set of optimal waypoints as required to avoid known obstacles or pop-up threats under basic kinematic/dynamic vehicle-environment constraints utilizing a quasi-discrete differential games approach. Fault Tolerant Control (Georgia Tech) The scenario of a stuck collective pitch actuator was simulated in flight by limiting the deflection of swash plate actuators in such a way to prevent changes in collective pitch. A fault tolerant control module was developed that generated a rotor RPM command that allows the existing flight controller to continue to function, albeit at reduced capability12. A typical flight test result is illustrated in Figures 9 and 10, where up and down step responses of 10 feet were performed in between hover segments. Altitude hold performance is significantly worse without collective pitch, but still effective. collective pitch when rolling left or right, which necessitates corresponding changes in rotor RPM to maintain vertical tracking. In all cases performance is degraded, but control is maintained, making a return to base or revised mission feasible (note that chronologically this later test was at the end of the second SEC final experiment mission). 280 278 Estimated Command 276 Altitude (ft) 274 272 Stuck Right Swashplate Actuator - Level box pattern Actual Command 270 268 230 266 220 264 -z (feet) 262 3400 210 3420 3440 3460 3480 Time (sec) 3500 3520 200 190 180 Figure 9 – Utilizing rotor RPM to accommodate a simulated stuck collective actuator, step responses; altitude and altitude command 170 50 40 50 30 40 30 20 20 10 Rotor Angular Rate (RPM) 950 x (feet) 0 10 0 y (feet) 900 Figure 11 – Flying a small box pattern with a simulated stuck right swash plate actuator; performance is worse than nominal, but may be sufficiently good to allow the aircraft to return for repair 850 Vision-Aided Inertial Navigation (Georgia Tech) 800 3400 3420 3440 3460 3480 Time (sec) 3500 3520 Figure 10 – Utilizing rotor RPM to accommodate a simulated stuck collective actuator, step responses; measured rotor RPM; note transient in RPM during climb and descent For the SEC final experiments, this RPM-control reconfiguration was utilized to “save” the aircraft after simulated damage, bringing the aircraft back to the landing area at low speed and limited to gentle maneuvers. This ended the first mission of the SEC final experiments. Subsequent tests also included simulated failure of an individual swash-plate actuator, where only combinations of cyclic and collective pitch were possible – and rotor RPM utilized to maintain vertical control throughout. A stuck actuator on the right side swash plate result is shown in Figure 11. This particular failure makes it necessary to change In these tests, a 2-D vision sensor looking at a selected image target aided an Inertial Navigation System (INS) on the GTMax13. Target location in the image and size of the target were used to provide updates to the INS to bound drift in the position and velocity estimates. This was accomplished by augmenting the existing GTMax navigation system with the vision input, and simply disregarding GPS information during tests. This navigation solution was utilized by the autopilot, resulting in automated flight of an unmanned rotorcraft without GPS; significant for the case of GPS failure, denial, or poor signal such as in an urban scenario. A block diagram of this architecture is shown in Figure 12. Several different optical targets have been used, including an artificial one, and windows on actual buildings. Typical results are shown for one position axis (North/South) is in Figure 13, showing commanded position, estimated, and raw GPS data for comparison. Pictures from the close approach to a building at the beginning of the second SEC final experiment mission are shown in Figure 14. The approach to the building was utilized to locate a sniper, who subsequently attempted to flee the area. The human operator then commanded adjustments to camera angle and position of the GTMax to follow the sniper. Autopilot UAV IMU + Navigation Image Processing+ Computer Vision Camera Figure 12 – Vision-aided inertial navigation architecture, where inertial + vision navigation solution is utilized by the autopilot Figure 14 – GTMax utilizes vision-aided inertial navigation to approach a building (top); window near center of onboard image on the second floor of the building (bottom) provides position updates; GPS is not used Figure 13 – Commanded, estimated, and raw GPS position for a single axis (GPS data used for comparison). Vehicle is about 100 ft from the image target. Automatic Flight Envelope Protection (Georgia Tech) An automatic flight envelope protection system that utilizes an online trained neural network to predict and then avoid flight envelope limits was then tested during simulated evasion of a simulated threat from the ground. It should be noted that this work was done utilizing the neural network adaptive flight controller described previously, demonstrating stability of simultaneous adaptive limit detection and flight control. Flight tests conducted to date include avoidance of a normal load factor limit and a rotor stall prediction parameter (Erits factor, in units of speed)14. In the SEC final experiments, the system was utilized to perform a rapid turn around at high speed while not exceeding these two limits. Fault-Adaptive Control (Vanderbilt) For the previously mentioned stuck cyclic actuator test, the fault detection function was accomplished utilizing bond graph models for the actuators and a diagnosis engine. Results indicated correct detection of fault in 2-3 seconds. No false positive detections were experienced in the flight testing phase. launched from another aircraft. Subsequently, both the GTMax and GTSpy were maneuvered independently. Automatic Obstacle Avoidance Using Stereo Vision (SSCI) A stereo pair of cameras was utilized to automatically identify an obstacle and reroute the GTMax around that obstacle during several tests in September and November 2004. A picture from one such test is shown in Figure 15. Figure 16 – GTSpy mounted on the tail of the GTMax in preparation for air launch Figure 15 – GTMax with stereo cameras re-plans route to avoid collision with actual obstacle Airborne Deployment of a Rotorcraft (Georgia Tech) In preparation for an airborne launch of the GTSpy from the GTMax, the GTSpy was installed on the tail of the GTMax is shown in Figure 16. An additional wireless modem was also added to the GTMax to allow it to relay digital communications between the ground and the GTSpy. On November 20, 2004, the GTSpy was dropped from the GTMax while the GTMax was in an automatic hover at approximately 400 ft above ground level. The GTSpy fell tail first until it was able to arrest the descent and smoothly went into a hover 80 ft below the GTMax. A picture of both aircraft immediately after the separation is shown in Figure 17. It is believed that this is first time a hovering aircraft was ever air Figure 17 – GTSpy is dropped from hovering GTMax (sequence of four images); GTSpy subsequently hovered 80 feet below the GTMax; This is believed to be the first time a hovering aircraft has ever been air launched 3 CONCLUSIONS This paper described flight test results obtained from two university-operated research unmanned rotorcraft: the GTMax, a small helicopter, and the GTSpy, a micro ducted-fan rotorcraft, in final experiments of the DARPA Software Enabled Control program. An open system UAV testbed architecture was described for the two aircraft. The testbed architecture enabled a wide array of different approaches to be tested safely and effectively. Flight test results included: fault-tolerant adaptive flight control, fault detection and accommodation, reconfigurable control, envelope protection, visionaided inertial navigation, vision-based obstacle avoidance, and the first air-launching of a hovering aircraft. What these results have in common is that they have the potential to enable future unmanned systems to be more reliable through improved vehicle maneuverability, fault tolerance, and fault accommodation. ACKNOWLEDGEMENTS This work was supported by the DARPA Program under contracts #33615-98-C-1341 and #33615-99-C1500, with John Bay as Program Manager. SEC final experiments included technologies developed under separate Software Enabled Control (SEC) program contracts at: Boeing, Oregon Graduate Institute, The Charles Stark Draper Laboratory, Scientific Systems Company, Honeywell, and Vanderbilt University. The authors would also like to acknowledge some of the many other important contributors to this work at Georgia Tech: Henrik Christophersen, Scott Clements, J. Eric Corban, Graham Drozeski, Luis Gutierrez, Murat Guler, Bonnie Heck, Jeong Hur, Phillip Jones, Suresh Kannan, Adrian Koller, Eric Martens, Alex Moodie, Wayne Pickell, J.V.R. Prasad, Alison Proctor, Nimrod Rooz, Bhaskar Saha, Michael Turbe, Suraj Unnikrishnan, Linda Wills, Allen Wu, and Ilkay Yavrucuk. REFERENCES 1 Christophersen, H., Pickell, W., Koller, A., Kannan, S., and Johnson, E., “Small Adaptive Flight Control Systems for UAVs using FPGA/DSP Technology,” Proceedings of the AIAA “Unmanned Unlimited” Technical Conference, Workshop, and Exhibit, 2004. 2 Dittrich, J., and Johnson, E., “Multi-Sensor Navigation System for an Autonomous Helicopter,” Digital Avionics Systems Conference, 2002. Johnson, E. and Kannan, S., “Adaptive Flight Control for an Autonomous Unmanned Helicopter,” Proceedings of the AIAA Guidance, Navigation, and Control Conference, 2002. 4 Kannan, S. and Johnson, E., “Adaptive Trajectorybased Flight Control for Autonomous Helicopters,” Proceedings of the 21st Digital Avionics Systems Conference, 2002. 5 Johnson, E. and Schrage, D., “System Integration and Operation of a Research Unmanned Aerial Vehicle,” AIAA Journal of Aerospace Computing, Information, and Communication, Vol. 1, No. 1, January 2004. 6 Johnson, E. and Turbe, M., “Modeling, Control, and Flight Testing of a Small Ducted Fan Vehicle,” Proceedings of the AIAA Guidance, Navigation, and Control Conference, 2005. 7 Bogdanov, A., Wan, E., and Harvey, G., “SDRE Flight Control for X-Cell and R-Max Autonomous Helicopters,” Proceedings of the IEEE Conference on Decision and Control, 2004. 8 Gutierrez, L., Vachtsevanos, G., and Heck, B., “An Approach to Adaptive Mode Transition Control of Unmanned Aerial Vehicles,” Proceedings of the American Control Conference, 2003. 9 Singh, L., Plump, J., McConley, M., and Appleby, B., “Software Enabled Control: Autonomous Agile Guidance and Control for a UAV in Partially Unknown Urban Terrain,” Proceedings of the AIAA Guidance, Navigation, and Control Conference and Exhibit, 2003. 10 Elgersma, M., Ganguli, S., Ha., V., and Samad, T., “Statistical Performance Verification for an Autonomous Rotorcraft,” Proceedings of the IEEE International Symposium on Intelligent Control, 2004. 11 Garagić, D., Bošković, J. and Mehra, R., “Robust Nonlinear Fault-Tolerant Control of a Small-Scale Helicopter”, Proceedings of the American Helicopter Society 60th Annual Forum, 2004. 12 Drozeski, G., Saha, B., and Vachtsevanos, G., “A Fault-Tolerant Architecture for Unmanned Rotorcraft,” Proceedings of the AHS International Specialists’ Meeting on Unmanned Rotorcraft, Chandler AZ, January 18-20, 2005. 13 Wu, A., Johnson, E., and Proctor, A., “VisionAided Inertial Navigation,” Proceedings of the AIAA Guidance, Navigation, and Control Conference, 2005. 14 Yavrucuk, I., Prasad, J.V.R., Calise, A., and Unnikrishnan, S., ”Adaptive Limit Control Margin Prediction and Avoidance,“ 58th AHS Annual Forum, 2002.