From: AAAI Technical Report SS-99-05. Compilation copyright © 1999, AAAI (www.aaai.org). All rights reserved.
Model-based Programmingof Reactive Systems:
The Journey of Deep Space One
Brian C. Williams
MIT and NASAAmes Research Center
A new generation of sensor rich, massively distributed systems are emergingthat offer the potential for profound economicand environmentalimpact,
ranging from building energy systems to autonomous
space probes. These"immobile"robots have the richness that comesfrom interacting with physical environments, together with the complexity of networked
software systems. The goal is to make these systemsefficient, capable and long lived, that is, able to
survive decades of autonomousoperation within unforgiving environments. This offers an overwhelming programmingchallenge. Hand coding functions
for maintainingthe system’sinternals traditionally requires the programmerto reason through system wide
interactions, along lengthy paths betweenthe sensors,
control processor and control actuators. This reasoning requires thinking about the behavior of a hybrid
system, composedof complexreal-time software constructs, distributed digital hardwareand continuous
physical processes. The resulting code typically lacks
modularity, is fraught with error and makes severe
simplifying assumptions. Severe limitations of these
codes are compensatedby large operations teams.
Model-directed autonomy meets this challenge
through two ideas. First, we note that programmers
generate the desired function from their commonsense knowledgeof how the software and hardware
modules behave. The idea of model-based programruing is to exploit this modularityby havingengineers
programmodel-directed systems by simply articulating and plugging together these commonsensemodels. The next challenge is the infeasibility of synthesizing a set of codesat compiletime that envision all
likely failure situations and responses. Our solution
is to developreal time systems, called model-directed
executives, that respondto novel situations on the order of hundredsof milliseconds, while performingextensive deduction, diagnosis and planning within the
reactive control loop.
For the last several years mywork, and that of
the autonomous systems group at NASAAmes, has
been directed towards developing an experimental autonomouscontrol system, called RemoteAgent (RA).
Remoteagent will soon be demonstrated as a tech-
236
nology experiment of the NewMillenniumspacecraft,
Deep Space One (DS1). DSI, launched recently during the fall of 98, will fly by an asteroid and a comet,
using its solar powered ion propulsion engine and
navigating optically by followingthe stars. To demonstrate fully autonomy, RemoteAgent will guide DSI
through a series of optical navigation maneuversduring Mayof 99, responding and replanning in response
to a series of injected failures and changing mission
goals.
In this talk I will examinethe lessons learned from
DSI about model-based programming and modeldirected execution. In the first part I will examinethe
representational needs of modeling DS1as a hybrid
system. Fromthis I will develop the Reactive Modelbased Programming Language (RMPL). RMPLuses
transition systems and co-temporal interactions as
a rallying point for unifying representational concepts from a diverse set of research areas, including
qualitative modeling, model-baseddiagnosis, hidden
Markovdecision processes, synchronousreactive languages and concurrent constraint programming.
In the second part I will discuss the development
of a series of increasingly more expressive modeldirected executives. Eachexecutive is formulated as
a deductive form of an optimal, model-based controller in which modelsare specified through a combination of hierarchical, probabilistic transition systems and propositional logic. This frameworkallows
us to achieve high levels of autonomyand responsiveness, by drawinginspiration from a diverse set
of algorithms taken from model-baseddiagnosis, hidden Markovprocesses, search, planning and real-time
propositional inference. I will concludeby discussing
the role model-directed autonomyis playing within a
variety of future NASA
applications, from Mars Exploration to the search for Earth-like planets around
other stars.
Model-based Programming
of Reactive Systems:
The Journey of Deep Space One
Brian C. Williams
Autonomous Systems, NASAAmes
MassachusettsInstitute of Technology
http://www,
i c. arc. nasa. gov/ic/proje cts/mba/index,
html
In collaboration with
P. Pandurang Nayak, Vineet Gupta, Jim Kurien
Outline
¯ Remote Agents and Deep Space One
¯ Model-based Autonomy
¯ Livingstone:Model-based Configuration Manager
(with Pandu Nayak).
¯ Unifying Model-based and Reactive Programming
(with Vineet Gupta).
¯ Conclusions
237
Emergenceof Long-lived Systems
Rovers
Interferometers
In situ
Propellant
Production
Life
Support
238
Explorers
Houston, We have a problem...
¯ Quintuple fault occurs
(three shorts, tank-line and
pressure jacket burst, panel flies
off).
Groundassembles novel repair.
Swagert & Lovell work on
Apollo 13 emergencyrig lithium
hydroxideunit.
Mattingly works in ground
simulator to identify novel
sequence handling severe power
limitations.
Autonomy should embody the innovation
in Apollo 13 and other missions.
Remote Agent
exemplified
AMES / JPL
www.rax. arc. nasa. gov
Mission Goal Scenario
¯ Goal-directed
¯ Closed-loop
on Goals
¯ Programmed
with models
¯ Highly responsive
¯ Heavily deductive
239
courtesy JPL
Outline
" Remote Agents and Deep Space One
* Model-based Autonomy
* Livingstone:Model-based Configuration Manager
(with Pandu Nayak).
¯ Unifying Model-based and Reactive Programming
(with Vineet Gupta).
¯ Conclusions
240
Challenge: Programmersreason through interactions:
¯
¯
¯
¯
¯
¯
¯
¯
¯
monitoring
confirming commands
tracking goals
detecting anomalies
diagnosing faults
reconfiguringhardware
recovering from faults
avoiding failures
coordinating control
policies
Model-based Autonomy
Programmers generate breadth of functions from
commonsensehardware and software models in light
of mission-level goals.
¯ Model-based Programming
¯ Programby specifying commonsense,
compositional
models.
¯ Model-directed Executives
¯ Performsreasoningthroughsysteminteractions, by
performingsignificant search &deductionwithin the
reactive loop. "
How do we model and reason about hybrid
software/hardware systems simply and uniformly?
241
A Unified Modeling Paradigm
Distributed
Hardware
Continuous
Processes
Software
RMPL
¯c
If c next A
g%~
Unless e next A
.A,B
¯ Always A
¯ Choosewith probability
u t~ty’+ncer’a’n’and Anomalies
¯ Choosewith reward
Model-directedExecution is Formulated
as Model-basedStochastic OptimalControl
Controller
o(0
Plant
/1 = argminC(s ’, y, g ’) s.t. /.t’ ~
242
Reformulating the Autonomy
Coding Challenge
Programmers
must reason through system-wide
interactionsto generatecodesfor:
¯
¯
¯
¯
¯
¯
monitoring
confirming commands
tracking goals
detecting anomalies
isolating faults
diagnosing causes
¯
¯
¯
¯
¯
reconfiguringhardware
recovering from faults
safing the system
avoiding failures
coordinating control
policies
4
4
Identifying Modes
Reconfiguring Modes
Livingstone and Burton
Model-directedExecutives
Model
Possible
Discretized
Sensed values
Command
q
Model-based,Stochastic, OptimalController where:
¯ variables have finite domains
¯ modelspecified in a propositional temporallogic
¯ implemented
as queries to a fast, best-first, inference core
243
Outline
" Remote Agents and Deep Space One
" Model-based Autonomy
¯ Livingstone:Model-based Configuration Manager
(with Pandu Nayak).
¯ Unifying Model-based and Reactive Programming
(withVineetGupta).
¯ Conclusions
A Unified Modeling Paradigm
Distributed
Hardware
Continuous
Processes
Software
Qualitative
Algebra &
State Diagrams
Uncertainty
and Anomalies
244
Modeling:Co-temporalInteractions
Valve
Driver
Valve
¯
Resettable
[-7 failure
Off [--]
Permanent
¯ failure
On
Open
Closed
vdriver=on => Out= In;
vdriver=off=>Out=close; ....
~x~ Stuck
open
N
Stuck
M closed
vlv--open => Sgn(Inflow)= Sgn(Outflow);
vlv=elosed=>Sgn(Outflow)--O;....
Formulae in propositional
state logic {xi
¯ Algebra on Signs : S = {+, 0, -,?}
- [infloW]s = [outflOW]s
¯ Algebra on Relative
~
= vij
}
Values: R = {high, nominal,
low,?}
- [inflow]R= [outflow]R
DS1 RCSfailure scenario
z facing thrusters
x facing thrusters
2£21
Dynamics
I
x att
error
,
I
I
y art
error
z art
error
ACS mode
Type 2
nominal~nominal
245
A Unified Modeling Paradigm
Distributed
Hardware
Concurrent
Automata
Continuous
Processes
Software
Qualitative
Algebra &
State Diagrams
Hidden Markov Models
Uncertainty
and Anomalies
Modeling: Constraint-based,
Concurrent,Probabilistic, Automata
Valve Driver
Valve
Open
On (~::~~Resettable
Turn
on~ ff
Off ~_J
~
o,e"
failure
"~.~ Permanent
~ failure
vdriver=on => Out= In;
vdriver=off=> Out=close; ....
cost
l
Stuck
~
"open
51//
Prob .9 ~.~
Closed
~
~kal Stuck
~ closed
vlv=open => Sgn(lnflow)= Sgn(Outflow);
vlv=closed=> Sgn(Outflow)=0;....
Compactencoding of a restricted
246
POMDP
Livingstone: Model-basedConfiguration Manager
Possible
Model
Command
Discretized
Sensed values
Command
ModeIdentification:
ReachableConsistent Next States
Current state
Possibl
t’l’On
wI t h2obt~rruVs~
Belief State
Enumerated by decreasing likelihood
247
Characterizing MI
Find feasible current states, given prior state,
commandsand current state observations.
Previous
Current
Si+l
Observed
Sol+
1
Predicted
¯ Probabilities computedas standard belief state update
with conditional independence of transitions.
ModeReconfiguration:
ReachingGoals in the Next State
~
,
~Current
o
state
o
Next states that provide
"nominal thrust"
Enumerated by decreasing
immediate reward
248
Characterizing MR
Find least cost commandsthat achieve the current
goal in the next state.
gi
Goal
Tn(S i ~ Suj)
Nominal
Next
Current
Propositional Reduction
MI: Characterizing possible next states
¯ MR: Characterizing
possible
249
commands
Statistically Optimal
Configuration Management
Statistical
ModeIdentification
¯ p(vj.] oi) = P(Oi]vj) p(v/) /
oc p(oi I 5) P( rJ)
Bayes Rule
¯ p(oil ~) is approximated from the model
- p(oil "9) =
if 5 (ai-1) entailsoi
if rj. (ai_l)is inconsistent
withio
- P(OiI rj.) =
- P(Oil ~j.) =
otherwise
Optimal ModeReconfiguration
t~i = argminc( pi ’) s.t. i ’ in Mi
OPSAT:
RISC-like Best-first, Deductive Core
Test
Best-first Agenda
Optimal
feasible
solutions
Checked
n;
Incorporate
conflicts
~~
Conflicts
(infeasible
subspaces)
~~
¯ Conflicts and unit propagation highly focus search
¯ (< 10 states visited per test)
¯ Unit propagation dominates, TMSsignificantly
250
reduces.
ITMSincremental unit
propagationclose to ideal
Data from 387 context switches on
DS-1 theory containing 12,693 clauses
300.
250¯
200¯
Ranges from 110% to
660% more label
changes than necessary
Number 150 ¯
10050.
o0
10
20
30 40
50
60
70
80
90
Percent extra label changes
100 110 > 110
Outline
" Remote Agents and Deep Space One
" Model-based Autonomy
" Livingstone:Model-based Configuration Manager
(with Pandu Nayak).
¯ Unifying Model-based and Reactive Programming
(with Vineet Gupta).
¯ Conclusions
251
Does a Unified Representation and
ParadigmExist ... at the Reactive Layer?
Whats Missing from Livingstone’s language?
¯ Goal Decomposition
¯ Strong Typing
¯ MethodSelection
¯ Encapsulations
¯ Concurrency
¯ Polymorphism
¯ Preemption
¯ Inheritance...
A Unified Modeling Paradigm
Distributed
Hardware
Concurrent Automata
Continuous
Processes
Software
Qualitative
Algebra &
State Diagrams
Hierarchical
Automata
Hidden Markov Models
Uncertainty and Anomalies
252
The Model-based Programming Language
Large-Scale Modeling Requires Good, Familiar
¯ Encapsulation Mechanisms.
Supports:
class Valve{
-Familiar syntax
public VaiveCmd
cmdln;
-encapsulation
private ValveFIow
inflow, outflow;
-polymorphism
private ValveNominalModes
mode;
valve 0 {
-strong typing
when(mode=open){inflow = outflow;
- multiple inheritance..
when(cmdIn=close)
next mode=closed;}
when(mode=closed){ inflow =
outflow= 0;
when(cmdIn=open)
next mode=open;}
}
To model software MPLmust support
complex imperative behaviors.
DS1 Optical Navigation:
Switch engine off,
tum MICASon
Do:
- Turn to first asteroid, take picture
- Turn to second asteroid, take picture
- Turn to third asteroid, take picture
Watching if thrusters are broken,
then switch to alternate set and repeat.
Computecurrent position and course correction
Watchingif pictures are corrupted,
then repeat sequence.
253
Execution Script
Component Model
AutoNav::{
TurnMicasO~)SwitchlPSStandb~’~)
do when IPSsta-ndby & MICASonteen{
{TurnToTarget(1
when Turndonedo TakePicture(1)};
/* samefor targets 2 and3 */
{TurnMicasOfl~)
OpticalNavigation0}
} watching PictureError(~
when PictureError donext AutoNav}
MICAS:: always choose {
with probability 0.99 {
if MICASonthen
unless TurnMicasOff
thennext MICASon,
with probability 0.01 {
next MICASfail,
if MicasTakePic
thennext PictureError
}}
Language Design Features
¯ Constraints
¯ Conditional execution
¯ Iteration
¯Preemption/defaults
¯ Nested Parallel
¯ Probability & reward
Composition
Reactive MPLCombinators
Support Design Features
¯
¯
¯
¯
¯
¯
¯ c
constraints
¯ If c next A
conditional execution
¯ Unless c next A
preemption/defaults
parallel composition ~ ¯
A,B
/
iteration
~l
¯ Always A
¯ Choose with probability
probability
¯ Choose with reward
and reward
Generalize
from TCC/HCC[Gupta, Saraswat]
254
Combinatorsof synchronouslanguages
like Esterel can be derived fromRMPL:
¯ do A watching c
¯ suspend A on c reactivate on d
¯ A;B
¯ next A
¯ P ::= A[P]
¯ when c do next A
¯ when c do A
¯ If c then A
EncodingRMPL
using Hierarchical
Probabilistic ConstraintAutomata
¯
¯
¯
¯
¯
¯
¯
Generalizefromconcurrentconstraint automata
States maybe automata(compositestates).
Multiplestart andcurrent states for eachautomata.
Conditionsallowedon negativeinformation(not entailed).
Transitionsare distributionsover next states, with rewards.
Transitionsfromprimitivestates only.
Constraintsassociatedwithprimitivestates only.
255
Combinators: Preemption &
Conditional Execution
¯ Introduces non-monotonicity,
but is stratified.
¯ Avoids causal paradoxes
found in languageslike
Esterel.
¯ Only combinator using
negative information.
¯ Livingstone used material
implication to encode:
- not c or next A
¯ RBurtonuses intuitionistic
interpretation,
Combinators:Iteration &
Parallel Composition
Starts newcopy of A at each
time instant.
Theability to markmultiple
states simultaneouslyenables a
compact encoding.
256
Executing Deterministic HCA
0 = {c,d,e}
Given marking:
¯ Markstart states of all newly markedcompositestates.
¯ Let theory 0 be the set of constraints of all markedstates.
¯ Find enabled transitions of markedstates.
- Labeld is satisfied if d is entailed by 0
- Label’dis satisfied ifd is not entailed by 0
¯ Take Transition
RBurton: Model-directed Executive
Reactive
MPL x,~
Derived state
Goals
Plant Model
Discrete
Sensed values
257
Commands
Probabilistic Execution: EnhancedMI
Standard
MI
Nested
Choose
A
B
C
Encodingof a partially
D
A CB CA D B D
observable MarkovProcess
Sequence Combinator:
Decision Theoretic Choice
Nested
Choose
RI/~R2R3///~
A
B
Encodingof a partially
R4
C
D
observable MarkovDecision Process
258
Does a Unified ParadigmExist
for Model-based Autonomy?
Model-based Programming
Model-based Execution
oC
¯ If c next A
¯ Unless c next A
.A,B
¯ Always A
¯ Choosewith probability
¯ Choosewith reward
Hierarchical, Probabilistic
Constraint Automata
RISC-like Deductive Kernel
259