MIT
ICAT
Human and Automation Integration
Considerations for UAV Systems
Prof. R. John Hansman
Roland Weibel
MIT International Center for Air Transportation
Department of Aeronautics & Astronautics
MIT
ICAT
y
Remote Sensing
†
†
†
†
†
†
y
Meteorology
Scientific Research
Aerial Photography/ Mapping
Pipeline Spotting
Disaster Monitoring
Agriculture
Surveillance
†
†
†
†
y
Possible Commercial UAV
Applications - Motivation
Border Patrol
Homeland Security/ Law Enforcement
Traffic Monitoring
Search and Rescue
Data Delivery
† Communications Relay
† Multimedia Broadcast
y
Cargo Transport
MIT
ICAT
Possible Military UAV
Missions - Motivation
y Intelligence
†
†
†
†
†
†
Reconnaissance
Target Monitoring
Forward Air Control
Electronic Warfare
Search and Rescue
Battle Damage Assessment (BDA)
y Offensive Operation
† Suppression of Enemy Air Defenses (SEAD)
† Close Air Support
† Deep Strike
y Cargo Transport
MIT
ICAT
Aerovironment Black
Widow – 2.12 oz.
Current Unmanned Aerial Vehicles
Sig Kadet II RC
Trainer – 5 lb
BAE Systems
Microstar – 3.0 oz.
Boeing/ Insitu Scaneagle – 33 lb
IAI Scout – 351 lb
NOAA
Weather
Balloon
2-6 lb
Boeing X-45A UCAV – 12,195 lb (est)
Aerovironment
Pointer – 9.6 lb
Allied Aero.
LADF – 3.8 lb
Gen. Atomics – Predator B – 7,000 lb
Northrop-Grumman
Global Hawk 25,600 lb
Bell Eagle Eye – 2,250 lb
0
1
Micro
**Mass Range**
Large range of UAV types as users of NAS
-propulsion, configuration, capabilities, etc
10
Mini
100
UAV Weight (lb)
Short Range
1,000
Tactical
10,000
High Alt / UCAV
100,000
20000
10000
70000
Legend
60000
50000
+
*
0
1
Electric
Piston
Turbocharged
Turboprop
Turboshaft
Turbofan
40000
30000
Pointer
Sender
Azimut
Biodrone
Seascan
Survey-Copter 1
Sheddon
Luna
Shed Mk3
Mini-V
Camcopter
Fox Shadow 200
RMAX
Spectre
Phoenix
Scout
Solar Bird
ProwlerII
Searcher
Helios
Gnat
Hunter Gnat 2
Raptor
Robocopter
Perseus
Heron
Eagle Eye
Altus II
Hermes 1500
Fire Scout
Eagle 2
Predator
Predator B
Dragon Eye
Ceiling (ft)
Micro
Mini
Tactical
MALE
HALE
Rotary
10
100
1000
Max TO Weight (lb)
10000
Global Hawk
MIT
ICAT
Ceiling
Class
G
FL 600
Class
A
18,000 ft
Classes
B-E, G
100000
MIT
ICAT
Takeoff Method
Hand-launched: Aerovironment Pointer
Rocket-Assisted: Hunter UAV
Rail-Launched: Sperwar
Tilt-Rotor: Eagle Eye
Runway Takeoff: X-45 UCAV
MIT
ICAT
Basic Supervisory Control
Architecture
Controlled
Process
Human
Operator
Displays
Controls
Human Int.
Computer
Task Int.
Computer
Communications
Channel
Adapted from Sheridan, Humans and Automation
Vehicle
Sensors
MIT
ICAT
UAV Operation Basic
Functional Architecture
surveillance
Air Traffic Control
commands
Other A/C
reporting/
negotiation
direct
control
reporting
displays
operations
controller
feedback
operator
(vehicle)
(dispatch)
(customer)
(field commander)
operator
(sensors)
displays
vehicle
commands
controls
transmission
sensors
UAV
Payload
surveillance
sensors
sensors
Environment
(A)
MIT
ICAT
Pointer UAV
y Used for Short-Range Surveillance
† Battlefield commanders
† Law Enforcement
y Vehicle Capabilities
†
†
†
†
Manual Control
Autopilot
Sensor Integration and Display
Loss of Link Return to Base
y Bandwidth Requirements
† Transmission of Vehicle Commands
† Receipt of Sensor Intelligence, Vehicle State
MIT
ICAT
Pointer UAV Tasking & Control
experience,
training
goals
Commander
tasking
tasking
experience,
training
goals
speech
tasking/interpretation
Plan Actions
Implement
Plan Actions
sensor operator
Interpret
MonitorInterpret
Monitor
Aircraft Control
Control
Camera
pilot
CDU
traj
commands
Lost Link
Procedure
vehicle
MCP
state
commands
FMC
controls displays Recording displays
manual
control
sensor
autointegration
pilot
ground
station
camera
guidance
control
surfaces
UAV
sensors
Camera
General Atomics Predator
Medium Altitude, Endurance
MIT
ICAT
Predator Air Vehicle Operator (AVO) Station
from – M. Draper, Air Force Research Lab (2001)
Northrop-Grumman Global Hawk
HALE UAV
MIT
ICAT
Global Hawk Mission Control
Elements
MIT
ICAT
Air Vehicle Operator
Station
Command &
Control
y Interface with ATC
y Uplink Mission Changes
y Monitor Vehicle Health
and Status
y Monitor Threat Warning
and Deception
ATC
Battlefield
Intelligence
y
y
y
y
y
U.S. Air Force photo
Navigation Plan
Communications Plan
Sensor Plan
Dissemination Plan
Dynamic Retasking
Mission Planning Station
y
y
y
y
View Imagery
Monitor Sensor Status
Calibrate Sensors
Process & Disseminate
Imagery
Sensor Data and
Processing Station
y Maintain Health and
Status of Comm.
Subsystems
y Construct and Monitor
Comm. Plan
Communication and
Control Station
MIT
ICAT
Global Hawk MCE
MIT
ICAT
Boeing X-45 UCAV
Boeing X-45A Control Station
from – DARPA Website (2003)
MIT
ICAT
Multiple UAV Control Station for Simulated Scenario
from – J. Nalepka, Air Force Research Lab (2003)
MIT
ICAT
MIT
ICAT
MIT
ICAT
MIT
ICAT
UAV-Related Human Factors
Issues - (Partial List)
y
Allocation/ Level of Autonomy
y
Bandwidth/ Latency
y
Situation Awareness
y
Cognitive Complexity Limitations
† Single & Multiple UAVs
y
Information Saturation/ Boredom
y
Simulator Sickness
y
Operator Orientation Confusion
y
Culture Resistance
y
Judgment
† Acceptable Risk
† Weapons Release Authorization
MIT
ICAT
y
Situation (Battlespace)
Awareness
UAV Task Analysis
y
† Navigation
† Aircraft Configuration
† Sensor Operation
† Perception
† Comprehension
† Projection
y
Diagnosis
y
Strategic Planning/
Re-planning
† Goal Management
† Route planning
y
Tactical Decisions
† Weapons Authorization
† Avoidance of Hazards
† Systems Management
Monitoring
† Vehicle Health
† External Environment
‹ Threats, Targets, Traffic
† Risk Assessment
† Communications Link
† Sensor Data
† Environment
† Threat
† Targets
y
Control
y
Communication
†
†
†
†
Current State
Intent
Intelligence
Tasking
MIT
ICAT
AFRL Levels of Autonomy
1.
Remotely Guided
2.
Real Time Health Diagnosis
3.
Adapt to Failures & Flight
Conditions
4.
Onboard Route Replan
5.
Group Coordination
6.
Group Tactical Replan
7.
Group Tactical Goals
8.
Distributed Control
9.
Group Strategic Goals
Remote Manual Control
Currently
Realized
Full
Continuum
Supervisory
Control
10. Fully Autonomous Swarms
Fully Autonomous,
World Aware
MIT
ICAT
Level of Autonomy Trend
Source: DOD UAV
Roadmap, 2000
MIT
ICAT
MIT
ICAT
UAV Design Space - Military
@add pictures@
Level of Autonomy/
System Complexity
Group
Coord
Tactical
Replan
Health
Monitoring
Predator
Waypoint
Designation
Global Hawk
Manual
Pilotage
X-45
Pioneer
Shadow
Pointer
Tactical
Scout
Battlefield
Monitoring
Mission Complexity
Multiship
Coord
MIT
ICAT
Diagnosis Procedure Role
Goals
Sensors
Target
Detection
Intelligence
Experience
Training
Diagnosis
Procedure
Display
Human
Lethal Force Authorization
Importance of Situation Awareness
Command
Console
UAV
Platform
MIT
ICAT
Endsley Situation Awareness Model
•System Capability
•Complexity
•Interface Design
•Automation
•Stress & Workload
Task/System Factors
Feedback
Situation Awareness
State of the
Environment
Perception Of
Elements In
Current
Situation
Comprehension Of Projection Of
Current Situation Future Status
Decision
Level 2
Level 1
Performance
Of
Actions
Level 3
Individual Factors
Information Processing
Mechanisms
•Goals & Objectives
•Preconceptions
(Expectations)
Long Term Memory
Stores
Automatically
•Abilities
•Experience
•Training
MIT
ICAT
Bandwidth Limits
Controlled
Process
Human
Operator
Displays
Controls
Human Int.
Computer
Task Int.
Computer
Communications
Channel
Bandwidth Limits
Adapted from Sheridan, Humans and Automation
Vehicle
Sensors
MIT
ICAT
Bandwidth Limit
Downlink
y Video
† Forward View, Surveillance
y Imagery
† Reconnaissance, Target Selection
y Voice
† ATC Comm, Intelligence
y Schematic Data
† System Health, Location
Uplink
y Voice
† ATC Comm, Comm to Ground
y Manual Control
y Commands
† Waypoint/ Tasking Commands
MIT
ICAT
Task Performance & Bandwidth
Frame Rate
Color
Depth
Constant Task
Performance
Constant
Bitrate
Resolution
Diagram from Sheridan, Teleoperation
Communications Latency
Problems
MIT
ICAT
+
+
Air Traffic
Control
Other
Traffic
Operator
+
UAV
Satellite
Link
+
communication
over time
channel used
channel free
receipt
delay
transmission
delay
Satellite Latency Cycle Times : 2-5 sec
PIO Issues due to lags.
MIT
ICAT
Multiple Vehicle Control
y Situation Awareness
†
†
†
†
†
“Big Picture” Overview of Battlefield
Orientation Confusion Multiple Reference Frames
N Vehicle states
N Vehicle status
Kindergarten Model
y Human/ Machine Allocation
†
†
†
†
†
Level of Vehicle Autonomy
Need for Higher Level of Abstraction (Macro vs Micro Management)
Organizational vs Operator Model
Directed vs Behavioral Automation
Dynamic re-allocation
y Cognitive Workload - Taskload
† How many vehicles can be reliably managed
† Cognitive Complexity Limitations
† ATC Analogy (Acceptable Level of Traffic)
Complexity Concepts &
Controller Process Model
MIT
ICAT
AIR TRAFFIC CONTROLLER
ATC
OPERATIONAL
CONTEXT
AIR
TRAFFIC
SITUATION
SITUATION AWARENESS
Surveillance
Path
LEVEL 1
LEVEL 2
LEVEL 3
Perception
Comprehension
Projection
STRUCTURE
SITUATION
COMPLEXITY
COGNITIVE
COMPLEXITY
Command
Path
WORKING
MENTAL
MODEL
PERFORMANCE
OF ACTIONS
DECISION
PROCESSES
x Monitoring
x Evaluating
x Planning
“CURRENT
PLAN”
x Implementing
PERCEIVED
COMPLEXITY
Adapted from Endsley
Situation Awareness
Model, Pawlak Key ATC
Processes
MIT
ICAT
Human-System Interface
Issues
y Interface Comparison - UAV vs Commercial
† DARPA USAF Boeing X-45 Example
† Boeing B-777
y Source: Build 2 Operational Simulation Overview Briefing
† Caveats:
‹ Prototype not operational system
‹ Briefing may not reflect actual system
‹ PC based interface
MIT
ICAT
Interface Design Comparison
PC vs Commercial Avionics Conventions
MIT
ICAT
X-45
Primary Flight Display (PFD)
Analogue vs Digital Indications
Color Conventions
Readability
Hidden Info
MIT
ICAT
Commercial B-777
Primary Flight Display (PFD)
MIT
ICAT
•
Reduction of Clutter
•
No indications for
Quiet Dark Philosophy
“normal”
•
No “ON” indicators
•
No indications for
“do nothing”
•
Indicate limits, not
normal range
Elements of quiet dark …
So once the procedure for this failure is taken care of ….
When the gear is safely up and locked …
When the flaps are up …
Only the engine indications remain. Maybe in the future we can eliminate most of them as well.
MIT
ICAT
Mode Awareness
Cognitive Models
Operator Directed Process
MIT
ICAT
y
Example: Flight Automation
Mode Awareness is becoming a serious
issues in Complex Automation Systems
† automation executes an unexpected action
(commission), or fails to execute an action
(omission) that is anticipated or expected by one
or more of the pilots
y
Multiple accidents and incidents
† Strasbourg A320 crash: incorrect vertical mode
selection
† Orly A310 violent pitchup: flap overspeed
† B757 speed violations: early leveloff conditions
y
Pilot needs to
† Identify current state of automation
† Understand implications of current state
† Predict future states of automation
Reference: Aviation Week &
Space Technology. McGrawHill, January 30, 1995.
MIT
ICAT
MIT
ICAT
MIT
ICAT
MIT
ICAT
MIT
ICAT
MIT
ICAT
MIT
ICAT
MIT
ICAT
MIT
ICAT
MIT
ICAT
MIT
ICAT