Integrating Planning and Scheduling:
Status and Prospects
Subbarao Kambhampati
Arizona State University
rakaposhi.eas.asu.edu/yochan.html
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Planning vs. Scheduling
$&RQWLQXXP
Scheduling
– Set of jobs (may have of tasks in some
(partial) order)
– Temporal constraints on jobs
Planning
– Initial state & a set of Goals,
– A library of actions
» Preconditions/effects
» EST, LFT, Duration
–
O Discrete/Continuous
Contention constraints
» Each task can be done on a
subset of machines
» Resource requirements
Synthesize a sequence of actions capable
of satisfying goals
Find start times for jobs that are optimal
(wrt make-spans, resource
consumption etc)
&D
&DX
XVD O 5H
5HD
DVR QLQ
LQJ
J
5HVR
5HVRX
XUFH 5HDV
5HDVR
RQLQ
LQJ
J
I = initial state
7
,QWH
,QWHJJUDWLQJ 3ODQQLQJ
(prec) Oi
(effects)
/)7
'XU
(6 7
G = goal state
7
[I]
Oi
Oj
Ok
5HVHDUFKLQWRSODQQLQJDQGVFKHGXOLQJPHWKRGV
KDVODUJHO\EHHQGHFRXSOHG
6FKHGX
6FKHGXOLOLQQJ
Om [ G ]
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Need for Integration
* Most existing schedulers
concentrate only on
resource allocation,
ignoring action selection
– E.g. HSTS operation
scheduling
* Most existing planners
concentrate on action
selection, ignoring
resource allocation
– Plan-based interfaces
– Interactive decision
support
* Many real-world problems require both
capabilities
– Supply Chain Management problems
» I2, ILOG, Manugistics
– Planning in domains with durative actions, continuous
change
» NASA RAX experiment
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Why now?
* Significant scale-up in plan synthesis in last 4-5
years
– 5/6 action plans in minutes to 100 action plans in
minutes
– Breakthroughs in search space representation, heuristic
and domain-specific
* Significant strides in our understanding of
connections between planning and scheduling
– Rich connections between planning and CSP/SAT/ILP
» Vanishing separation between planning techniques
and scheduling techniques
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Approaches for Integration
* Extend schedulers to
* Extend planners to deal with
handle action and
resource choices
resources, durative actions
and continuous quantities
resource
alternative
job-shop choices
processes
scheduling (RCSP)
process8
process3
planning
process7
…… …
Task4
Task7
Task2
…… … …
Task5
Task1
Task8
Task3
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
1(
5
XO H
$1
3/
KG
– De-coupled
– Loosely Coupled (RealPlan System)
6F
* Coupled Architectures
&D
XVD
O3
ODQ
Task6
+(
6&
/(
'8
5
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Overview
:K\LQWHJUDWHSODQQLQJDQG
VFKHGXOLQJ"
· 3ODQQLQJ7KHVWDWHRIWKHDUW
* 6FKHGXOLQJ7KHVWDWHRIWKHDUW
* ,QWHJUDWLQJ3ODQQLQJDQG6FKHGXOLQJ
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
3ODQQLQJ
7KH6WDWHRIWKH$UW
Overview
*
*
*
*
Why integrate planning and scheduling?
Planning: The state of the art
Scheduling: The state of the art
Integrating Planning and Scheduling
,Q W HJ UDUDWW LQJ 33OOD Q L
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6X EE
EEDD UDR.DPE KD PSD WL
6
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
(Deterministic) Planning: The problem
At(A,M),At(B,M)
¬In(A), ¬In(B)
Appolo 13
*States are modeled in terms of (binary)
state-variables (factored rep.)
-- Complete initial state, partial goal state
*Actions are modeled as state
transformation functions
-- Syntax: ADL language (Pednault)
Earth
Earth
*Plans are sequences of actions
At(A,E), At(B,E),At(R,E)
Effects
In(o1)
Preconditions
Load(o1)
At(o1,l1), At(R,l1)
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
At(R,M), ¬At(R,E)
In
(
x ) ⇒ At ( x, M)
∀x
& ¬At(x, E)
Unload(o1)
Fly()
¬In(o1)
In(o1)
At(R,E)
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
The (too) many brands of classical planners
J
Q
L
\ UN
I
L
R
Q
8 PHZ
D
U
)
Planning as Theorem Proving
(Green’s planner)
Planning as Model Checking
Planning as Search
Search in the space of States
(progression, regression, MEA)
(STRIPS, PRODIGY, TOPI, HSP, HSP-R,
UNPOP, FF)
Search in the space of Plans
(total order, partial order,
protections, MTC)
(Interplan,SNLP,TOCL,
UCPOP,TWEAK)
Search in the space of
Task networks (reduction
of non-primitive tasks)
(NOAH, NONLIN,
O-Plan, SIPE)
Planning as CSP/ILP/SAT/BDD
(Graphplan, IPP, STAN, SATPLAN, BLackBOX,GP-CSP,BDDPlan)
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Refinement Planning: The idea
All action
sequences
P
Refine
All Sol
P’
P’
P
All Seq.
Refinements
All Solutions
Partial plans
Narrowing sets of action sequences
to progress towards sets of solutions
Remove non-solutions
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
[AIMAG-97]
6XEE
6XEED
.DPEKDDPS
DUDR .DPEK
PSDDWL
Plan Representation
Steps, Orderings, Aux. Constraints
Partial plan =
0
P:
1: Load(A)
4:Unload(A) ∞
2:Fly()
In(A)@2
At(R,E)
3: Load(B)
Semantics in terms of Candidate sets
--Candidate is an action sequence that satisfies all the
plan constraints (but can have additional actions)
Refinements split and prune candidate sets
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
At(A,E)
At(B,E)
At(R,E)
progression
1: Unload(A)
∞
0
2: Load(B)
1: Unload(A)
∞
0
2: Fly()
1: Unload(A)
∞
State-Space
Ext
end
Suf
fix
0
0
1: Fly()
Regression
∞
At(A,M)
At(B,M)
¬In(A)
¬In(B)
0
1: Fly()
2: Unload(A)
∞
0
1: Fly()
2: Unload(B)
∞
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
1:Unload(A)
∞
At(A,M)@ •
PSR
At(A,M)@•
At(A,M)
Ad
2: Load(A)
0
Plan-Space
mid
dle
∞
1: Unload(A)
the
0
d in
Ext
end
Pre
fix
(The) Three Refinement Strategies
¬At(A,M)
0
2:Fly()
3:Unload(A)
∞
In(A)@2
/H
/HD
D VW &
&R
RPPLWPHQ
LWPHQWW 5H JUHV
JUHVV
VLRQ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Tradeoffs among Refinements
FSR and BSR must commit
to both position and
relevance of actions
+ Give state information
(Easier plan validation)
- Too many states when
actions have durations
0
Plan-space refinement (PSR)
avoids constraining position
+ Reduces commitment
(large candidate set /branch)
- Increases plan-validation
costs
- Leads to premature
commitment
Pf
W
HD
V
JU
V
DU
H
Z
W
\
DV
RO
7
K
KH
QJ
7
OOL
)L
V
V
/H
+ Easily extendible to actions
with duration
∞
1: Fly()
Pp
Pb
0
1: Fly()
0
∞
1: Fly()
∞
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
A flexible Split & Prune search for
Refinement Planning
Null Plan set
Null Plan set
R1
R2
Refineplan( P : Plan)
Rk
P1
P2
Pk
R1
R2
Rk
P1
P2
Pk
0*. If «P » is empty, Fail.
1. If a minimal candidate of P
is a solution, terminate.
2. Select a refinement strategy R .
Appply R to P to get a new plan set P’
3. Split P’ into k plansets
4. Non-deterministically select one of the plansets P’ i
Call Refine(P’ i)
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Broad Themes in the Planning
Renaissance
* Disjunctive
Representations
* Reachability/Relevance
Analysis
* Connections to
combinatorial substrates
* Sophisticated domain
pre-processing
Techniques
optimistic projection of achievability
level 0
level 1
level 0
level 2
level 1
level 3
level 2
…
…
…
'L
'LV
VM XQF
QFWWLYH 3O D QQ LQJ
3ODQ
3ODQQ
QL QJ D V LLWWH UD
UDWWLYH
+ HXUL
HXULV
VW LFV
%RX
%RXQ
QGHG
HQJWK
GHGO HQJ
WK
&RPEL
&RPELQ
QDWRULDO 3UREO
3UREOH
H P 6R
6ROO Y LQ J
7X WR ULDO R Q 5H FH QW $GYDQ
$GYDQF
FH V LQ $, 3ODQQ
3ODQQLLQJ
,- &$
&$,,
$$$,
$,
KWWS
SOODQ
HDVD VXH GX
UDNDSR
KWWSUDNDS
GXS
RVK LHDV
DQQ
QLQJ
LQJWX WR ULDO
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Init:
Ontable(A),Ontable(B),
Clear(A), Clear(B), hand-empty
holding(A)
~Clear(A)
Ontable(A)
Pickup(A)
Ontable(B),
~Ontable(A)
Clear(B)
Clear(B)
~handempty
hand-empty
Goal:
~clear(B), hand-empty
Ontable(B),
Clear(A)
holding(B)
~Clear(B)
Pickup(B)
~Ontable(B)
Ontable(A),
$
%
Clear(A)
3U
3UR
RJUHV
JUHVV
VLRQ 6HDUFK
~handempty
5HJU
5HJUH
HVVLRQ 6H DUFK
~clear(B)
holding(A)
""+R
""+RZ
Z
SURPL
SURPLV
VLQ
LQJ
J D UH
WKH LQ
LQWWH UPHGLDWH
6WD WH V 3ODQV"
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
Putdown(A)
~clear(B)
hand-empty
Stack(A,B)
holding(A)
clear(B)
Putdown(B)??
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
h-A
Pick-A
Pick-B
Init:
Ontable(A),Ontable(B),
Clear(A), Clear(B), hand-empty
h-B
Goal:
~clear(B), hand-empty
~cl-A
~cl-B
~he
onT-A
onT-A
onT-B
onT-B
cl-A
cl-A
cl-B
cl-B
he
he
n
io
t
ec ty
j
ro ili
P
b
ic cha
t
is ea
m
ti f R
p
O o
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
h-A
Pick-A
Pick-B
on-A-B
on-B-A
St-A-B
h-B
h-A
h-B
~cl-A
~cl-A
~cl-B
~cl-B
St-B-A
~he
onT-A
onT-A
Ptdn-A
V
WLRQ
'/ OVDF
2
D
LWHU KH3*
6L]H D[ O IPtdn-B
R W
0
' HQJWK
//
onT-B
onT-B
cl-A
cl-A
cl-B
cl-B
he
he
Pick-A
~he
onT-A
onT-B
cl-A
cl-B
Pick-B
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
he
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Heuristics based on the Planning Graph
* lev(S): index of the first level where all props in S appear
nonmutexed.
– If there is no such level, then
If the graph is grown to level off, then ∞
Else k+1 (k is the current length of the graph)
Set-Level heuristic: h(S) = lev(S)
Admissible but not very informed
DO
N
7
nV
Sum heuristic: h(S) = ∑p∈
∈S lev({p}) Inadmissible
D\
P
UG
WH
V
H
\
VW
6
Assumes that sub-goals are independent
<H
OW
OW$
$
Adjusted Sum heuristic: [Sanchez et. al., 2000]
HAdjSum2M(S) = length(RelaxedPlan(S)) + max p,q∈S δ(p,q) Inadmissible
where δ(p,q) = lev({p,q}) - max{lev(p), lev(q)}
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Planning as Plangraph Solution Extraction
h-A
If there exists a k-length plan,
it will be a subgraph of the
k-length planning graph.
Pick-A
Pick-B
St-A-B
h-A
h-B
~cl-A
~cl-A
~cl-B
Planning is thus searching for
a “valid” subgraph of the
planning graph.
St-B-A
~cl-B
~he
onT-A
~he
onT-A
Ptdn-A
onT-B
Combinatorial search.
Can be cast into any
combinatorial substrate
(e.g. CSP, SAT, ILP…)
on-A-B
on-B-A
h-B
onT-B
onT-A
onT-B
Ptdn-B
cl-A
cl-A
cl-B
cl-B
he
he
Pick-A
cl-A
cl-B
Pick-B
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
he
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
(very) Quick overview of CSP/SAT concepts
x,y,u,v: {A,B,C,D,E}
w: {D,E} l : {A,B}
x=A ⇒ w≠E
y=B ⇒ u≠D
u=C ⇒ l≠A
v=D ⇒ l≠B
* Constraint Satisfaction Problem (CSP)
– Given
» A set of discrete variables
» Legal domains for each of the
variables
» A set of constraints on values
groups of variables can take
– Find an assignment of values to all the
variables so that none of the
constraints are violated
A solution:
x=B, y=C, u=D, v=E, w=D, l=B
x←A
N1: {x=A}
←B
y←
* SAT Problem = CSP with boolean
N2: {x= A & y = B }
←D
v←
variables
N : {x= A & y = B & v = D }
←C
u←
* TCSP = CSP where variables are time
N : {x= A & y = B & v = D & u = C }
points and constraints describe
←E
w←
←D
w←
N : {x= A & y = B & v = D & u = C & w= E }
allowed distances
3
4
5
N6: {x= A & y = B & v = D & u = C & w= D }
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Important ideas in solving CSPs
Variable order heuristics:
Pick the most constrained variable
--Smallest domain, connected to most other variables,
causes most unit propagation,
causes most resource contention,
has the most distance etc…
x ←A
N1: {x=A}
y← B
N2: {x= A & y = B }
v← D
N3: {x= A & y = B & v = D }
u← C
N4: {x= A & y = B & v = D & u = C }
Value ordering heuristics
Pick the least constraining value
w← E
N5: {x= A & y = B & v = D & u = C & w= E }
w← D
N6: {x= A & y = B & v = D & u = C & w= D }
Consistency enforcement
k-consistency; adaptive consistency. (pre-processing)
Forward Checking, unit propagation during search (dynamic)
x,y,u,v: {A,B,C,D,E}
w: {D,E} l : {A,B}
x=A Ÿ wzE
y=B Ÿ uzD
u=C Ÿ lzA
v=D Ÿ lzB
Search/Backtracking
DDB/EBL: Remember and use interior node failure explanations
Randomized search
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Posing Plangraph Solution
Extraction as a CSP/SAT
Variables: literals in proposition lists
Values: actions supporting them
Constraints: Mutex and Activation constraints
h-A
Pick-A
Pick-B
St-A-B
on-A-B
h-A
h-B
~cl-A
~cl-A
~cl-B
St-B-A
~cl-B
~he
onT-A
onT-A
onT-B
onT-B
~he
Ptdn-A
cl-A
cl-B
cl-B
onT-B
he
he
Pick-A
~cl-B-2: { #, St-A-B-2, Pick-B-2}
he-2:
{#, St-A-B-2, St-B-A-2,
Ptdn-A-2,Ptdn-B-2}
h-A-1: {#, Pick-A-1}
h-B-1: {#,Pick-B-1 }
….
onT-A
Ptdn-B
cl-A
Variables: Domains
on-B-A
h-B
cl-A
cl-B
Pick-B
he
Constraints:
he-2 = St-A-B-2 => h-A-1 !=#
{activation}
On-A-B-2 = St-A-B-2
=> On-B-A-2 != St-B-A-2
{mutex constraints}
Goals:
~cl-B-2 != # he-2 !=#
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Compilation to Integer Linear Programming
,/
,/3
3 * LYHQ D V
VH
HW R I UHDO YD
YDO
OX HG YDULD
YDULDE
EOHV D OLQHDU REMHFWLY
REMHFWLYH
H I XQFWLRQ RQ W
WK
KH YD
YDU
ULD
LDE
EOHV
OHV
D VHW R I O
OL
RI
LQ HDU LQHTXDOLWLHV R Q WKH YDULDE
I LQWHJ
LQWHJU
UDO
DOL
LW \ U HVW ULFWLRQV RQ W
YDULDEO
OHV DQ G D VHW R
WK
KH
YD
YDU
ULDEO
DEOH
HV )LQG WK H YDOXHV R I W
WK
KH I HD
HDV
VLE
LEO
OH YD
YDU
ULDEO
DEOH
HV I
IR
RU Z
ZK
K LF K W
WKH
KH REMH
REMHF
FW LYH I
IX
X Q FW L R Q
DWWDLQV WKH PD[L
D[LP
PXP YDOXH
LQWHJ
LQWHJH
HU SU
SUR
RJUDP
JUDPP
PLQJ FR UU HVS RQG
RQGV
V FORVHO\ W R 6$7 SU REOH
REOHP
P
* Motivations
– Ability to handle numeric quantities, and do optimization
– Heuristic value of the LP relaxation of ILP problems
* Conversion
– Convert a SAT/CSP encoding to ILP inequalities
» E.g. X v ~Y v Z => x + (1 - y) + z >= 1
– Explicitly set up tighter ILP inequalities
» If X,Y,Z are pairwise mutex, we can write x+y+z <= 1
DXW]
(instead of x+y <=1 ; y+z <=1 ; z +x <= 1)
UH . ,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
RORX
O
V
> :DVO VHQ HW D 'LPRS
9R FNPD\U
%R
V@
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Relative Tradeoffs Offered by
the various compilation substrates
* CSP encodings support implicit representations
*
*
*
– More compact encodings [Do & Kambhampati, 2000]
– Easier integration with Scheduling techniques
ILP encodings support numeric quantities
– Seamless integration of numeric resource constraints [Walser
& Kautz, 1999]
– Not competitive with CSP/SAT for problems without numeric
constraints
SAT encodings support axioms in propositional logic form
– May be more natural to add (for whom ;-)
BDDs are very popular in CAD community
– Commercial interest may spur effective algorithms (which we can use)
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Disjunctive Planning
* Idea: Consider Partial plans with disjunctive step,
ordering, and auxiliary constraints
* Motivation: Provides a lifted search space, avoids re*
generating the same failures multiple times (also, rich
connections to combinatorial problems)
Issues:
– Refining disjunctive plans
» Graphplan (Blum & Furst, 95)
Q
– Solution extraction in disjunctive plans
FWLR
[WUD ULDO
(
Q
W LR
DWR
» Direct combinatorial search
6RO X RPELQ
F
D
P
H
O
LV
» Compilation to CSP/SAT/ILP
3UR E
Refining disjunctive plans
0
1: Load(A)
or
2 : Load(B)
or
3 : Fly(R)
•
0
1: Load(A)
or
2 : Load(B)
or
3 : Fly(R)
1: Load(A)
or
2 : Load(B)
or
3 : Fly(R)
or
•
4 : Unload(A,E)
or
5 : Unload(B,E)
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Planning: Current Status
System
R
FF
1000
* Disjunctive planners as well
Seconds
38
39
40
41
34
35
36
37
32
33
31
30
29
27
28
26
25
23
24
22
21
19
20
GRT
17
18
– Plans with upto 100
actions synthesized in
minutes
– Impressive performances
in AIPS-1998 and AIPS2000 Competions
STAN
10
16
as heuristic state-search
planners scale well
HSP2
Mips
0.1
100
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
0.01
90
95
70
65
55
40
50
35
28
25
18
19
15
16
13
11
12
10
9
8
7
6
1
5
[Long & Kambhampati,
IJCAI-2001 (may be)]
4
comparable performance
are still to be developed
Seconds
* Plan-space planners with
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
6FKHGXOLQJ
7KH6WDWHRIWKH$UW
Overview
*
*
*
*
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
Why integrate planning and scheduling?
Planning: The state of the art
Scheduling: The state of the art
Integrating Planning and Scheduling
,Q W HJ UDUDWW LQJ 33OOD Q L
6
6X EE
EEDD UDR.DPE KD PSD WL
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Scheduling: Brief Overview
CSP Models
– Time point model
» Tasks as variables, Time
points as values
» EST, LFT, Machine
contention as constraints
– Inter-task precedences as
variables (PCP model)
Jobshop scheduling
– Set of jobs
» Each job consists of tasks
in some (partial) order
– Temporal constraints on jobs
» EST, LFT, Duration
– Contention constraints
» Each task can be done on
a subset of machines
CSP Techniques
– Customized consistency
enforcement techniques
» ARC-B consistency
» Edge-finding
– Customized variable/value
ordering heuristics
/)7
'XU
7
(67
» Contention-based
» Slack-based
0
– MaxCSP; B&B searches
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
Job Shop Scheduling as a CSP
Jobs
J1
O1 1 R 1
J2
O1 2 R 1
J3
O1 3 R 3
O2 1 R 2
Capacity
Constraints
O2 2 R 2
O2 3 R 1
Start Point Representation
Variables: Start time stil
Domain: [estil….lstil]
Precedence constraints:
stil + duil ≤ stjl
Capacity Constraints:
stil + duil ≤ stjl ∨ stjl + dujl ≤ stil
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
Precedence
Constraints
O3 1 R 3
6LPSO
PSOH
H 6H DUFK
O3 3 R 2
RPHV
LFDOO\ F VWUDLQWV
S
\
W
Q
WLR
RQ
'LVMXQFK FDSDFLW\ F
J
X
R
WKU
PCP Representation
Variables: Ordering(i,j,R) for task i and j
contending for resource R.
Domain: {i-before-j, j-before-i}
Constraints: Posting and propagation in
the underlying temporal constraint network
(time points and intervals)
0RU
0RUH
H )O H [ L E O H
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Constraint Propagation
$UF%RXQ
$UF%RXQG
GV
Through Precedence Relations:
O11R1
O21R2
O31R3
[0,12]
[0,12]
Forward Propagation
[0,12]
O11R1
O21R2
O31R3
[3,9]
[0,6]
[6,12]
Backward Propagation
(G JH)
JH)LLQGL
QGLQ
QJ
- S is a set of operations competing for resource R.
- O is an operation not in S also requiring R.
((LFT(S) - EST(S) < Dur(O) + Dur(S)))
š (LFT(S) – EST()) < Dur(O) + EST(O))
EST(O) t EST(S) + Dur(S)
O1
O2
7K UR
URX
XJK 5HVRX
5HVRXUUFH FR QVWUDLQ
VWUDLQWWV
O3
10
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
@
>6DGHK demand
20
S = {O2,O3}; O = O1
Æ Start Time O1 = 25
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Contention-based Ordering Heuristic
Individual Demand of O1 for Rj
demand
Aggregate Demand (of all O) for Rj
0
demand
3
6
time
Individual Demand of O2 for Rj
0
3
6
9
time
Most critical region
0
probability
0
6
3
9
time
Operation O1 Start Time Distribution
3
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6
Start time
Contention: Aggregated curves found for
each resource
Critical Region: Where a resource is
contended the most
Most Critical Unassigned Operation:
Contributes the largest area in critical
region
Variable Ordering Heuristic:
Choose the most critical unassigned
operation
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
PLWK >&KHQJ 6
@
Slack-based Ordering Heuristic
(Precedence constraint-posting slack)
(VW
(VWLL
* For two unordered operations I and J
– Slack(I
– Bslack(I
/IWL
GL
(VWM
GM
/IW
/IWMM
J) = Lftj – Esti – (Duri + Durj)
J) = Slack (I
J) / f(S), (f(S) is similarity measure)
* Min-Slack Selection (Variable Ordering)
– Choose opeations pairs with minimum value of
Min (Bslack(I
J), Bslack(J
I) )
* Max-Slack Posting (Value Ordering)
– Select the precedent constraint that leaves maximum remaining slack
Max(Bslack(I J), Bslack(J
I) )
* This slack-based heuristic performs competitively with
contention-based heuristic
* Significantly improved by combining with consistency
enforcement methods (Baptiste, Le Pape, Nuijten, 1995)
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Current State of Scheduling as CSP
* Constraint-based scheduling techniques are an
integral part of the scheduling suites by ILOG/I2
– Optimization results comparable to best approximation
algorithms currently known on standard benchmark
problems.
– Best known solutions to more idiosyncratic, “multiproduct hoist scheduling” application (PCB
electroplating).
* Better in most large-scale problems than IP formulations
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Integrating Planning & Scheduling
Overview
*
*
*
*
Why integrate planning and scheduling?
Planning: The state of the art
Scheduling: The state of the art
Integrating Planning and Scheduling
,Q W HJ UDUDWW LQJ 33OOD Q L
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6
6X EE
EEDD UDR.DPE KD PSD WL
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Approaches
* Decoupled
– Existing approaches
* Monolithic
– Extend Planners to handle time and resources
– Extend Schedulers to handle choice
* Loosely Coupled
– Making planners and schedulers interact
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Decoupled approaches
(which is how Project Mgmt Done now)
0DQDJHPHQW
7HFKQRORJ\ 'HYHORSPHQW
0LGORZHU PDQDJHU
WDVN SODQQLQJ
06 3URMHFW
VFKHGXOLQJ
,PSOHPHQWHUV
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Extending Planners
* ZENO [Penberthy & Weld], IxTET [Ghallab & Laborie],
HSTS/RAX [Muscettola] extend a conjunctive plan-space
planner with temporal and numeric constraint reasoners
* LPSAT [Wolfman & Weld] integrates a disjunctive statespace planner with an LP solver to support numeric
quantities
* IPPlan [Kautz & Walser; 99] constructs ILP encodings with
numeric constraints
* TGP [Smith & Weld; 99] supports actions with durations in
Graphplan
Engine
Search
engine
Domain Knowledge
Search
Control
Planning
Experts
Model
(DDL)
Goals
Initial state
from EXEC
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
HSTS
Plan Database
Plan
to EXEC
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Actions with Resources and Duration
Load(P:package, R:rocket, L:location)
Resources:
?h : robot hand
Preconditions: Position(?h,L)
Free(?h)
Charge(?h) > 5
Effects:
Constraints:
[?s, ?e]
?s
?s
holding(?h, P)
[?s, ?t1]
depositing(?h,P,R)
[?t2, ?e]
Busy(?h)
[?s, ?e]
Free(?h)
?e
Charge - = .03*(?e - ?s) ?e
?t1 < ?t2
?e - ?s in [1.0, 2.0]
&DSDFLW\ URERW ,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
"H
"V
>
>@
3RV "K/
+R OG "
"K
K 3
)UH
)UHH
H "K
%XV\ "K
G H S " K 3
)UH
)UHH
H "K
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
What planners are good for handling
resources and time?
* State-space approaches have an edge in terms of ease of monitoring
*
resource usage
– Time-point based representations are known to be better for multicapacity resource constraints in scheduling
Plan-space approaches have an edge in terms of durative actions and
continuous change
– Notion of state not well defined in such cases (Too many states)
– PCP representations are known to be better for scheduling with
single-capacity resources
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Extending Scheduling
resource
choices
job-shop
scheduling (RCSP)
process8
alternative
processes
process3
planning
process7
Task4
Task7
Task2
Task5
Task1
Task8
Task3
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
Ordering choices
Resource choices
Process choices
Task6
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Monolithic Architectures Scale Poorly
)
(
'
* Extended planning systems
&
are hard to control
– RAX uses a very error-prone
hand-coded search control
strategy
* Extended scheduling systems
*
tend to lose effectiveness due
to increased disjunction
Monolithic systems can
sometimes show counterintuitive behavior (by
multiplying search failures)
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
7RWDO
%
$
3HUIRUPDQFH ZRUVHQV ZLWK
LQFUHDVHG UHVRXUFHV
6HDUFK
'
$
*UDSK
(
%
)
&
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
6F
KG
XO H
(5
1
1
3/ $
&D
XVD
O3
O DQ
Loosely Coupled Architectures
+(
6&
/
'8
(5
Schedulers already routinely handle resources and
metric/temporal constraints.
– Let the “planner”concentrate on causal reasoning
– Let the “scheduler” concentrate on resource
allocation, sequencing and numeric constraints for
the generated causal plan
1HHGEHWWHUFRXSOLQJWRDYRLGLQWHUPRGXOHWKUDVKLQJe
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Making Loose Coupling Work
&D
XVD
O3
6F
KG
XOH
– Low level constraint propagation
» Loose path consistency on TCSPs
(5
» Bounds on resource consumption,
11
$
3/
» LP relaxations of metric constraints
– Pre-emptive conflict resolution
The more aggressive you do this, the less need for a
scheduler..
* How do the modules interact?
ODQ
* How can the Planner keep track of consistency?
(
8/
'
+(
5
6&
– Failure explanations; Partial results
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
RealPlan--Master/Slave
0DVWHU6ODYH
3ODQQHUGRHVFDXVDOUHDVRQLQJ
6FKHGXOHUDWWHPSWVUHVRXUFH
DOORFDWLRQ
,IVFKHGXOHUIDLOVSODQQHUKDVWR
UHVWDUW
6
9DULDEOHV $FWLRQV
9DOXHV 5HVRXUFHV
Planner’s CSP:
Variables: “goals”
Values: “actions”
Scheduler’s CSP:
Variables: “Actions”
Values: “Resources”
Re
ac sou
tiv rc
se at e c
lec ed o
ted by nst
ac the rain
ton
ts
s
3
5HDO3ODQ06
YD W L
DVWD KDPSD
Y
L
U
>6 DPE @
. & 3 ( $ $, $
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Performance of Master-Slave Coupling
QR
L O V L Y HQ
D
I
J
U
XOH FHLV
G
H
V F K X LG D Q Q
U
H
J
:K FLILF ODQQH
VSH WKHS
WR
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
RealPlan: Peer-to-Peer
F W HG HQ H
U
L
H
G
WLRQ EHWZ XOHU
D
Q
J
D
G
O
Q
([S WUDFNL G6FKH
N
EDF QHUDQ
3ODQ
9DULDEOHV )DFWV
9DOXHV $FWLRQV
7
9DULDEOHV $FWLRQV
9DOXHV 5HVRXUFHV
6
Se
ca t of
nn ac
ot tio
be ns
sc tha
he t
du
led
3
5HDO3ODQ33
3HHU3HHU
Planner’s CSP:
Variables: “goals”
Values: “actions”
Re
ac sou
t
se ivat rce c
lec ed o
ted by nst
ac the rain
ton
ts
s
Scheduler’s CSP:
Variables: “Actions”
Values: “Resources”
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Se
ca t of
nn ac
ot tio
be ns
sc tha
he t
du
led
Inter-module Dependency Directed
Backtracking
Planner’s CSP:
Variables: “goals”
Values: “actions”
Re
a c so u
t
r
se ivat ce c
lec ed on
ted by st
ac the rain
ts
ton
s
Scheduler’s CSP:
Variables: “Actions”
Values: “Resources”
Scheduler’s Task
Explanation Generation
Interface’s Task
Explanation Translation
Planner’s Task
Generate compact explanation of the
Scheduler’s failure in allocating resources
Translate the explanation into the form
that make sense to the planner.
Use the translated explanation to
generate plan that avoid this failure.
Generation of Alternative Plan
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Resource Domains:
A1, A2 , A3: {R1 , R2}
A4, A5: {S1 , S2 , S3}
Subset of variables that can not be
assigned values (reason of failure):
A1 = R1
Resource Constraints:
A1 ≠ A2 ; A2 ≠ A3 ;
A1 ≠ A3 ; A4 ≠ A5 ;
(A1 , A2 , A3)
N1: {A1 = R1}
A2 = R2
N1: {A1 = R1, A2 = R2}
A4 = S1
N1: {A1 = R1, A2 = R2 , A4 = S1}
A3 = R1
N1: {A1 = R1, A2 = R2 , A4 = S1 , A3 = R1}
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
A3 = R2
N1: {A1 = R1, A2 = R2 , A4 = S1 , A3 = R2}
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Shuffle 8 Blocks
100000
Time (sec)
10000
1000
Realplan-PP
Realplan-MS
100
10
1
2
3
4
5
Robot hand
Problem
Shuf_b8r4
Shuf_b8r4
RealPlan-PP
Class
Time
INFRES
FIX
#bk
BTPWF
Time
RealPlan-MS
Class
Time
Speedup (x)
BTPWF RealP
lanMS
2.52
4
>3 hr
SAMELEN
9373
>4170
3619
8.74
1
>3 hr
SAMELEN
9373
>1236
1072
Huge_r6
INFRES 4.53
4
976
INCRLEN
11056
215
2441
Huge_r6
FIX
17.48
3
991
INCRLEN
11056
56.7
632
Huge_r6
SAMELEN
20.59
2
983
INCRLEN
11056
47.7
537
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Executor
Mission
Profile
Execution status
Replanning requests
1H[W*HQHUDWLRQ5HDOSODQ
1$6$6SRQVRUHG
Plans with annotated
waypoints
Evolving specs
CSP-based
Finite capacity
resource
scheduler
interface
A temporal planner
supporting causal
plan synthesis
int
Mission modifications
Plan criticism
a
erf
int
erf
a
ce
ce
Mixed Integer/Linear
programming module
for metric constraints
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL
Summary & Conclusion
* Motivated the need for integrating Planning and
Scheduling
* Discussed the state of the art in Planning and
Scheduling
* Discussed approaches for Integrating them
– Loosely coupled architectures are a promising approach
,QWH
,QWHJJUDWLQJ 3ODQQLQJ 6FKHGX
6FKHGXOLOLQQJ
6XEE
6XEEDDUDR .DPEK
.DPEKDDPS
PSDDWL