THE PROBLEM
OF LONG-DURATION AUTONOMY
• One cannot foresee problems and opportunities in advance
• Humans cannot continuously monitor the state of affairs looking for
unexpected problems and opportunities
• Instead the system must be robust in the face of surprise
2
VISION AND APPROACH
Overall Objectives:
• To articulate a foundation for long-duration autonomy that integrates the MIDCA and T-REX architectures
• To formulate learning techniques that ensure robustness by adapting to uncertain and changing environments
• To provide a prototype implementation that exhibits this integration on actual physical platforms
Approach: Metareasoning combined with Goal-Driven Autonomy
• Allows agent to see contradictions/anomalies and remedies to current plans and actions
• Can be applied, for instance, to shipboard firefighting where new fires may break out
3
GOAL-DRIVEN AUTONOMY
(GDA)
• RECOGNIZE NOVEL PROBLEM (OR OPPORTUNITY)
• EXPLAIN WHAT CAUSES THE PROBLEM
• GENERATE A GOAL TO REMOVE THE PROBLEM
(OR TO LEVERAGE THE OPPORTUNITY)
4
THE GDA MODEL
5
METAREASONING
Action
Selection
Ground
Level
Meta-level
Control
Object
Level
Perception
Meta-Level
Introspective
Monitoring
Doing
Reasoning
Metareasoning
UUV
TREX
MIDCA
6
T-REX AGENT USED
ON THE DORADO UUV
• Goal-centric agent control
• Partitioned problem-solving with
continuous, parallel decision
and control loops
• Reactors encapsulate control
loops and share state variables
7
REACTOR PARAMETERS
• Planning look-ahead
ϑ
The expected number of ticks into
the future for which the reactor
will plan
• Planning latency
λ
The expected time a reactor will
take to produce a plan and to
integrate new goals
𝑃𝑤 = τ + λ, τ + λ + ϑ
8
METACOGNITIVE
INTEGRATED
DUAL-CYCLE
ARCHITECTURE
Goal Management
goal change
goal
insertion
subgoal
goal
input
goal change
Metagoal
Goals
insertion
subgoal
Intend
Meta
Goals
Reasoning Trace
( )
Plan
Intend
Meta-Level
Control
Strategies
Mental Domain = Ω
)
Intend
Problem
Solving
Task
Plan
Actions = Ω
Mental Domain
Act
(& Speak)
Interpret
Introspective
Monitoring
goal
Trace
insertion
Goals
MΨ
Monitor
MΨ
Evaluate
Memory
Mission &
Goals( )
World Model (MΨ)
Comprehension
Hypotheses
MΨ
Interpret
Episodic Memory
Semantic Memory
& Ontology
Plans( ) &
Percepts ( )
World =Ψ
9
Monitor
goal input
subgoal
Metaknowledge
Self Model (
)
Hypotheses
Episodic Memorygoal change
Introspective
Monitoring
Evaluate
Trace
Metaknowledge
Self Model (
Controller
Reasoning Trace
( )
Controller
Interpret
Episodic Memory
Memory
Activations
Hypotheses
Strategies
Activations
Task
Evaluate
Memory
Task
Plan
Meta-Level
Control
goal input
Percepts
Perceive
(& Listen)
MODEL OF SELF-REGULATED
PERFORMANCE
Task & goal (
confusion or
uncertainty
Trace ( )
cost &
benefits
goal insertion
Causal
Attribut.
(Assess)
Anomaly
Detect
(JOL)
)
int or
ext?
Strategy
Decision
choice & reasons
internal
MΨ
external
L
Models
Strategy
Execution
(Guide)
ignore
anomaly
∆
A
World
(Ψ)
= learning goal
A = attainment goal
= self model
L
10
MIDCA’S MODEL OF TREX (MΩ)
• A hierarchical graph of agent reactors and their connections
• For each reactor
• Internal and external variables
• Values for parameters (e.g., λ and ϑ)
• Current reactor state (e.g., failed)
• Representations of traces of TREX run-time behavior
• Control structure: Goal and parameter manipulation
11
THE MIDCA RESPONSE
• Reducing the window size of
Reactor2 constrains the planning
for goals internal to it
12
EXAMPLE UUV MISSION
COMMANDER’S INTENT:
END-STATE: QROUTE IS SAFE
TASKS: CLEAR MINES IN GA1 AND GA2
•
•
UUV-2, AN UNMANNED UNDERWATER
VEHICLE, IS ASSIGNED A MISSION REGARDING
THE GREEN AREA PORTION OF A HARBOR
DURING ITS TRANSITS, UUV-2 PASSES
UNEXPECTED OBJECTS OF INTEREST
13
PROBLEM RECOGNITION
UUV 2 Tasks:
Clear mines in Green
Sub-areas GA1 & GA2
New State
RECOGNIZE
Actual
Observed
State
Previous States
Compare
States
Expected
State
World
Model
Discrepancy
Transit
Problem
Data
PROBLEM EXPLANATION
EXPLAIN
Exp1
Transit Route
Problem:
• Exists mine
between
Sub-areas
• Mine should
have been
seen during
recon
• Mine threatens
mission
Exp2
Near
IsIsNear
QRoute
Ship
Action:
Action:
Intruder
Intruder
Damages
Lays
Ship
Mine
Knows
Knows
about
about
Target
Target
Can
See
After
Ship
Friendly
Activity
Recon
Action:
Action:
Intruder
Intrudermoves
moves
using
Vessel
using Vessel
Exp3
Expn
Select
Explanation
Explanation 3:
Intruder
Explanation
Laid
New Minefield
GOAL MANAGEMENT
MANAGE GOALS
Actions
DG
Goal Management can:
•
•
•
•
•
Create goals
Abandon goals
Transform goals
Prioritize goals
Assign resources
Explanation 3:
Intruder Laid
New Minefield
Search for
Intruder 7h94
Search for all
Intruders
Search for all
Intruders
Clear Mines in
Green Sub-areas
Clear Mines in
QRoute
Generate
& Change
Goals
• Survey QRoute in serpentine
patterns
• Maintain lane distances of 0.2 nM for
survey
• Establish communications
• Transmit obstacle sighting location
to Fleet HQ using frequency Bravo
• Use default MCM sweep pattern to
clear Minefield beginning at
coordinates (x,y)
AI Planner
Goals
Resources
Report Obstacle
Report
Obstacle
EVALUATION SANDBOX
System-wide Evaluation:
• Assign points and penalties to mission activities in Sandbox
• Compare GDA mission performance to 3 control conditions:
◦ Optimal, Conventional, & Random action UUV
• Metrics: Percentage of points scored on objectives over time
as problem complexity and amount of resources vary
• Software implementation for testing purposes
• Provides “discovery course” with various missionrelated objects and events linked to actual platforms
and sensors
Optimal
100%
GDA UUV
Conventional UUV
Consistently higher performance
Performance degrades smoothly
Performance degrades rapidly
Cannot adapt to loss of resources
Conventional UUV
Random Chance
100
100
Mission
Performance
GDA UUV
Mission
Performance
(% of max)
80
80
Mission
Performance
60
40
30
Time
60
40
Performance will be bracketed by the ideal
score and random score for fair comparisons
30
20
Resources
10
0
2
4
6
8
10
Complexity
20
Resources
10
0
2
4
6
8
10
Complexity
17
CONCLUSION
• The problem of long-duration autonomy is the inability to foresee all
problems and opportunities
• The solution involves problem recognition, goal generation, and
metareasoning
• The project contains both challenging problems and significant
opportunities, not all of which we can foresee
18