Modeling and Planning with
Robust Hybrid Automata
Cooperative Control of Distributed Autonomous
Vehicles in Adversarial Environments
2001 MURI: UCLA, CalTech, Cornell, MIT
Dahleh/Feron/Williams
May 14, 2001
UCLA
Brief update on MIT status
Investigators
• Dahleh
• Feron
• Massaquoi
• Williams
Students
• Z.-H. Mao (PhD)
• G. Kotsalis (PhD)
• K. Santarelli (PhD)
• T. Schouwenaars (PhD)
• M. Valenti (PhD)
• A. Walcott (PhD)
Outline
• Robust Hybrid Automaton concepts
• Model-Based Programming of autonomous
explorers
• Game-theoretic concepts
Problem Formulation
• Basic problem for autonomous vehicles/robots:
• Generate and execute a (sub)-optimal motion plan,
satisfying given boundary conditions, flight envelope and
obstacle avoidance constraints, in a dynamic and
uncertain environment
– Nonlinear control
• Steering of underactuated, non-holonomic systems
• Stabilization/tracking for nonlinear systems
• Flight envelope protection
– Robotics/Artificial Intelligence
• Path planning (obstacle avoidance) for non-holonomic dynamical
systems
– Computer science/Software Engineering
• Hard real-time constraints
Research supported by AFOSR, Draper, ONR
Hierarchical decomposition
• Need to introduce a hierarchical structure to achieve
computational tractability, e.g. (Stengel, 93):
– “Strategic layer”: Task scheduling, goal planning
– “Tactical layer”: Guidance, navigation
– “Reflexive layer”: Tracking, control, estimation
• General hierarchical systems, derived from arbitrary
decompositions, can be extremely hard to analyze and
verify
• Design a hierarchical system such that it offers safety and
performance guarantees by construction
– Analysis and verification: robustness analysis problem
• Consistent hierarchical system
System Quantization
• Quantization of feasible trajectories into trajectory
primitives
– formalization of the concept of “maneuver”
– Consistent abstraction of the system dynamics
• Hierarchical decomposition of the control tasks:
– Maneuver sequencing (guidance, trajectory planning)
– Maneuver execution (control, trajectory tracking)
• Control synthesis:
– Build a “maneuver library” (with feedback control)
– Behavioral programming: Solve a mixed-integer program on a “small”
space
– Hybrid control system with performance and safety guarantees by design.
Maneuver Automaton
• Two classes of trajectory primitives ( trim trajectories + maneuvers )
• Construct a “Maneuver Library”, with a finite number of primitives
• Generate trajectories by sequencing such primitives
– All generated trajectories are solutions of the system’s diff. equations
– All generated trajectories satisfy the flight envelope constraints (assuming
Steady left turn
F(x,u)=F(Yhx,u))
Hover
Forward flight
Steady right turn
Example of planning in a free
environment
400
300
actual position
actual velocity
commanded position
"maneuver switch"
200
100
0
-100
-200
-300
0
5
10
15
20
25
30
35
40
Model-based Autonomy
• How do we program explorers that reason
quickly and extensively from
commonsense models?
• How do we coordinate heterogeneous
teams of robots -- in space, air and land -to perform complex exploration?
• How do we couple reasoning, adaptivity
and learning to create robust agents?
• How do we incorporate model-based
autonomy into every day, ubiquitous
computing devices?
Model-based Autonomy
Programmers generate breadth of functions
from commonsense models in light of mission
goals.
• Model-based Reactive Programming
• Programmer guides state evolution at strategic levels.
• Commonsense Modeling
• Programmer specifies commonsense, compositional
models of spacecraft behavior.
• Model-based Execution Kernel
• Reason through system interactions on the fly,
performing significant search & deduction
within the reactive control loop.
Model-based Programming of
Cooperating Explorers
Managing Interactions for
Cooperation
Programmers and operators must reason through
system-wide interactions to :
• select among redundant
procedures
• Evaluate outcomes
• Plan contingencies
• select deadlines
• select timing
constraints
• allocate resources
Model-based Cooperative
Programming
• Model-based Programs
• Specify team behaviors as concurrent programs.

• Specify options using decision theoretic choice.

c
If c next A
Unless c next A
A, B
Always A
• Specify timing constraints between activities.

Choose reward

A in time [t-,t+]



• Model-based Execution
• Achieves correctness and economy
 Pre-plans threads of execution that are
optimal and temporally consistent.
• Responds at reactive timescales
 Perform planning as graph search
Decision-theoretic
Temporal Planner
Mission Scenario
TWO
ONE
HOME
Station: ABC
RENDEZVOUS
Enroute
RESCUE AREA
Diverge
Station: XYZ
RESCUE LOCATION
MEETING POINT
Enroute Activity:
Enroute
Corridor 2
Rendezvous
Corridor 1
Corridor 3
Rescue Area
Enroute Activity:
• Least cost threads of execution generated by extended auction algorithm
price = 425
price = 0
[450,540]
1
2
0
Extend Path
425
4
0
price = 425
5
0
9
0
price = 0
30
price = 30
3
10
price = 0
8
0
0
440
6
price = 440
0 price = 0
0
7
11
price = 0
price = 1
Path P = 1  3  4  5  8
Start Node : 1
End Node: 2
0
13
price = 0
price = 425
0
9  10
11  12
1
12
price = 0
13  2
Temporal planning is combined with randomized
path planning to find a collision free corridor
Path 1
xinit
4
xgoal
Xobs
5
Game-theoretic concepts
(Feron and DeMot)
Problem:
•Navigation of a number of vehicles to a target
•Target located at a position that is known with respect to the vehicles or
in a known region with a certain known probability distribution
•Vehicles have visual information about a local part of the environment
•Adversarial, unknown environment
Issues:
• Many cooperating vehicles vs. single vehicle missions
•Continuously updating available information
Approach:
•Game theory
Illustrative Example
Obstacle
Target
Adversary
Two-agent game
Single-agent
game
One
agent gets to
target fast
Getstrategy
to target fast
Pure
Requires mixed strategy
?
Agent
Agents
Initial Observations
• Multiple vehicles yield pure strategies whereas for
single vehicles a mixed strategy is optimal
• Continuously information updates?
Applicability of certainty equivalence principles
(eg Basar & Bernhardt, Birkhauser, 1991)
• More general setting: nature chooses the position
of an arbitrary amount of obstacles in the
unexplored areas - Need for well-defined models