June 10, 2002
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Adaptive Coordinated Control of Intelligent
Multi-Agent Teams
Shankar Sastry, Ruzena Bajcsy, Laurent El Ghaoui, Mike Jordan, Jitendra
Malik, Stuart Russell, Pravin Varaiya (Berkeley)
Vijay Kumar, Kostas Danillidis, James Ostrowski, George Pappas, C. J.
Taylor (Penn)
Howie Choset, Alfred Rizzi, Charles Thorpe (CMU)
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Background
ARO-MURI “Integrated Approach to Intelligent Systems”
1996-2001. Partners: Berkeley, Stanford, Cornell.
Highlights:
1. Creation of Field of Hybrid Systems: Foundations,
Methods, Analysis, Control
2. Vision Based Navigation and Control
3. Major Force Driving Bayesian Networks, Graphical
Models, Dynamical Probabilisitic Networks, Learning,
Rapproachment of AI and Control
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Integrated Approach to
Intelligent Systems
•
Disciplinary Evolution
– Control Theory
• Optimal control, linear control, nonlinear control, adaptive
control, stochastic control
• Mathematics: differential equations
– Artificial Intelligence
• Reasoning, adaptation, neural networks, natural language,
expert systems
• Mathematics: formal logic
– Computational Neuroscience & Cognitive Science
• Sensing, vision, taction, olfaction neural networks
• Mathematics (recently developed)
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Why Hierarchical Hybrid Control?
Central Control
Paradigm. that is
sensors and actuators
interacting locally,
breaks down when
dealing with
distributed systems
due to
• Complexity & scale
• Necessity of “tight” or
“optimal” operations
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Key Characteristics of
Distributed Intelligent
Systems
– Hierarchical or modular
to control complexity
– Globally organized
emergent behavior
– Robust, adaptive and
fault tolerant, and
degraded modes of
operation
– Architectural
organization involving the
use of compositionality
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Why Hybrid Hierarchical Control?
• Intelligence Augmentation for Human-Centered
Systems
• Autonomous Intelligence
Why integrative? Due to:
- the need to merge sensor fusion and hierarchies of sensing with
actuation across many agents, with desired emergent behavior
- the need to merge logical decision making and continuous action
- the need reconcile the need for safety of individual agents with
collective optimality
Control, artificial intelligence and cognitive neuroscience deal with
continuous action, logical reasoning and human/machine
understanding, respectively
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Technology Drivers:
Semi-Autonomous Multi-Agent Systems
The need for a theoretical framework for an integrative approach arises
from advances in computation, communication, intelligent materials,
visualization and other technologies which make it possible to expect more
from a multi-agent system than from a centralized control framework.
•
•
•
•
•
•
•
•
•
Distributed Command and Control
Distributed Communication Systems
Distributed Power Systems
Intelligent Vehicle Highway Systems
Air Traffic Management Systems
Intelligent Telemedical Systems
Intelligent Manufacturing Systems
Unmanned Aerial Vehicle Networks
Mobile Offshore Bases
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Theoretical Underpinnings
•
Architectural Design for Multi-Agent Systems
–
–
–
•
Perception Systems Sharing Many Representations
–
–
–
•
•
•
Hybrid Systems
Centralization for optimality
Decentralization for safety, reliability and speed of response
Hierarchical aggregation
Wide-area surveillance
Low-level perception
Frameworks for Representing and Reasoning with Uncertainty
Incorporation of Learning, Adaptation and Fault Tolerance
Parametric uncertainty with update and adaptation at the
continuous levels, learning of new “logical entities” -reinforcement learning at the logical levels and metal-learning for
redefining architecture
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
What Are Hybrid Systems?
Dynamical systems with interacting continuous and
discrete dynamics
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Why Hybrid Systems?
• Modeling abstraction of
– Continuous systems with phased operation (e.g. walking
robots, mechanical systems with collisions, circuits with
diodes)
– Continuous systems controlled by discrete inputs (e.g.
switches, valves, digital computers)
– Coordinating processes (multi-agent systems)
• Important in applications
– Hardware verification/CAD, real time software
– Manufacturing, communication networks, multimedia
• Large scale, multi-agent systems
– Automated Highway Systems (AHS)
– Air Traffic Management Systems (ATM)
– Uninhabited Aerial
Vehicles (UAV), Power Networks
June 10th, 2002: Topics
Adaptive Coordinated Control in 3-D
1,7, 17 Joint Kickoff
Dynamic Battlefield
Framework
Control Theory
Computer Science
Models of computation
Communication models
Discrete event systems
Control of individual agents
Continuous models
Differential equations
Hybrid Systems
x
y
2 [0[0;; 1]1]
2
q1
xç = f 1( x ; y)
yç = f 2( x ; y)
x
ô5
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
x > 4
à!
y > 10
à!
x
2 [0; 1]; y =
x = 0; y
q2
1
2 [1; 3]
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
xç = g1( x ; y)
yç = g2( x ; y)
y
ô 10
Control Challenges
• Large number of semiautonomous agents
• Coordinate to
– Make efficient use of common resource
– Achieve a common goal
• Individual agents have various modes of operation
• Agents optimize locally, coordinate to resolve conflicts
• System architecture is hierarchical and distributed
• Safety critical systems
Challenge: Develop models, analysis, and synthesis
tools for designing and verifying the safety of multi-agent
systems
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Taking Stock in Hybrid Systems
• Hybrid Systems and Control: established as a discipline, taught
to undergrads, grads. Monographs, textbooks being written by
all co-PIs: Lee and Varaiya, Henzinger and Alur, Lygeros, Tomlin
and Sastry. Workshop on Hybrid Systems established (first was
in Berkeley in 1998). Special Issues in IEEE Proceedings,
Systems and Control Letters, Automatica, IEEE Transactions on
Automatic Control, …
• Software: Programming languages, tools and frameworks for
Simulation and Control: Ptolemy II, Giotto, Massaccio all
developed. Ongoing work on verification tools.
• Hardware: in the loop demonstrations on the local UAVs,
formation flying to follow.
• Embedded software: EMSOFT established, new IEEE
Proceedings Special Issue 2003.
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
UCB/UCSF Laparoscopic Telesurgical Workstation
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
ANIMAL LAB TRIALS 1998
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Suturing with Unimanual System, 1998
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Berkeley BEAR Fleet: Ursa Maxima 1
Based on Yamaha RMAX
industrial helicopter
Integrated
Nav/Comm Module
Length: 3.63m
Width:0.72m
Height: 1.08m
Dry Weight: 58 kg Payload: 30kg
Engine Output: 21 hp
Rotor Diameter: 3.115m
Flight & system operation time: 60 min
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Flight Control System Experiments
Position+Heading Lock (Dec 1999)
Landing scenario with SAS (Dec 1999)
Attitude control with mu-syn (July 2000)
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Position+Heading Lock (May 2000)
Pursuit-Evasion Game Experiment Setup
Waypoint Command
Pursuer: UAV
Current Position, Vehicle Stats
Ground Command Post
Evader location detected
by Vision system
Current Position, Vehicle Stats
Evader: UGV
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Pursuit-Evasion Game Experiment
PEG with four UGVs
• Global-Max pursuit policy
• Simulated camera view
(radius 7.5m with 50degree conic view)
• Pursuer=0.3m/s Evader=0.5m/s MAX
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Experimental Results: Pursuit Evasion Games
with 4UGVs (Spring’ 01)
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Pursuit-Evasion Game Experiment
PEG with four UGVs and a UAV
• Global-Max pursuit policy
• Simulated camera view
(radius 7.5m with 50degree conic view)
• Pursuer=0.3m/s Evader=0.5m/s MAX
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Experimental Results: Pursuit Evasion Games
with 4UGVs and 1 UAV (Spring01)
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
What is Different Today?
1. The world and national security threats are different:
mobile operations in urban terrain, hostage rescue,
anti terrorism operations, homeland protection.
2. Use of robotic and mixed initiative forces, the need
for coordination of manned and unmanned forces
3. The need for dynamic strategies and tactics for
dealing with a determined and flexible adversary.
4. Exploitation of the 3rd dimension by organic UAVs
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
New Technical Innovations
• Control of the 3 D Digital battlefield: need to
use 3rd dimension, aerial forces, robotic and
mixed initiative forces, untethered
communications
• Adaptive Coordinated Control of Multiple
Agents: reconfiguration of teams dynamically
in response to adversarial action
• Intelligent coordination of multiple agents:
ability to discover intent and reconfigure
strategies adaptively
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Intellectual Organization:
Thrust Areas
• Architecture Design for Adaptive, Dynamic
Planning
• Integration of Rich Multi-Sensor Information
into Virtual Environments incorporating
human intervention
• Handling Uncertainty and Adversarial Intent in
Adaptive Planning
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Challenge Scenarios
• Reconaissance and Intelligence: robotic
ranger force for scouting fixed area for time
critical targets
• Mixed Initiative Engagement in urban
environments using micro-UAVs, UGVs.
Emphasis on immersive environments for
deploying
• Recognition and Tracking of Unfriendlies:
emphasis on networked vision for tracking.
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Hierarchical Architectures for Dynamic
Adaptive Planning
• Progess to date in hierarchical architectures for
decision making in “normal” modes of operation.
Main emphasis here will be on replanning in “fault” or
“degraded” modes of operation including deviations
from hierarchical operation.
• Key technical issues:
– Abstractions of Hybrid Systems for Architecture Design
• Hierarchical abstractions
• Assume-guarantee reasoning for abstractions
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Thrust I continued
• Control of Hybrid Systems
– Numerical Solutions for Controller Synthesis
– Hierarchical Solutions of Synthesis Procedures
– Liveness and other acceptance conditions
• Controller Libraries
– Many world semantics and hierarchy semantics
– Modal decomposition
– Exceptions
• Team and Task Allocation
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Integration of Multi-Sensor Information Into
Virtual Environments
• Adaptive Hierarchial Networks for Acquiring and
providing information
– Networked sensors
– Bandwidth utilitzation
• Extraction of 3 D Models from Distributed Sensors
– 3 D models from video data
– Integration of real and virtual environments
• Environments for Human Intervention & Decision
Making
– Situational awareness
– Display of uncertain data
– Triaging of data for decision making
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Smart Dust, Dot Motes, MICA Motes
Dot motes, MICA motes and smart dust
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Tiny OS (TOS)
Jason Hill, Robert Szewczyk, Alec Woo, David Culler
• TinyOS
• Ad hoc networking
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Last 2 of 6 motes are dropped from
MAV
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Field of wireless sensor nodes
• Ad hoc, rather than engineered placement
• At least two potential modes of observation
– Acoustic, magnetic, RF
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Subset of more powerful assets
• Gateway nodes with pan-tilt camera
– Limited instantaneous field of view
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Set of objects moving through
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Track a distinguished object
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Many interesting problems
• Targeting of the cameras so as to have objects of
interest in the field of view
• Collaborate between field of nodes and platform to
perform ranging and localization to create coordinate
system
• Building of a routing structures between field nodes
and higher-level resources
• Targeting of high-level assets
• Sensors guide video assets in real time
• Video assets refine sensor-based estimate
• Network resources focused on region of importance
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Abstraction of Sensorwebs
• Properties of general sensor nodes are described by
– sensing range, confidence on the sensed data
– memory, computation capability, clock skew
– Communication range, bandwidth, time delay, transmission loss
– broadcasting methods (periodic or event-based)
• To apply sensor nodes for the experiments with UAV/UGVs
introduce super-nodes (or gateways), which can
– gather information from sub-nodes ( filtering or fusion of the data from
sub-nodes for partial map building)
– communicate with UAV/UGVs
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield
Uncertainty and Adversarial Intent
• Models of Uncertainty
– Environmental: non deterministic and probabilistic
– Adversarial
• Guarantees of Success in the face of uncertainty
– Decision making in the presence of uncertainty
• Learning of Adversarial Strategy
– Probing strategies
– Games, partial information solution concepts
– Adaptation to changing utility functions of adversary
June 10th, 2002: Topics
1,7, 17 Joint Kickoff
Adaptive Coordinated Control in 3-D
Dynamic Battlefield