Reviving Integer Programming Approaches
for AI Planning:
A Branch-and-Cut Framework
Menkes van den Briel
Thomas Vossen
Subbarao Kambhampati
Department of Industrial Engineering
Leeds School of Business
Department of Computer Science
Arizona State University
University of Colorado at Boulder
Arizona State University
Tempe, AZ
Boulder, CO
Tempe, AZ
menkes@asu.edu
vossen@colorado.edu
rao@asu.edu
Compilation approaches for AI planning
Planning
Integer
Programming
(IP)
Propositional
Satisfiability
(SAT)
Constraint
Satisfaction
(CSP)
• Very little focus on IP
• There were no IP-based planners in
IPCs 1998, 2000, 2002
Background
• Why should we pursue integer programming?
– Support for optimality
– Support for numeric constraints
– IP has shown to be a great success in solving large
scale optimization problems
Reviving integer programming
1. State variables are modeled by an appropriately defined
network
– The resulting IP formulation can be interpreted as a network
flow problem with additional side constraints
2. The common notion of parallelism based on planning
graphs is generalized
– This reduces the plan length of the solution plan
3. Action ordering constraints are dynamically generated
when needed by a branch-and-cut algorithm
– Branch-and-cut has been extremely successful for a large
number of optimization problems
Example: State change flow network
LOC1
LOC2
Example: State change flow network
LOC1
AT_LOC1
Package1
LOC2
I
t=1
t=2
t=3
AT_LOC2
G
IN_TRUCK1
AT_LOC1
I
Truck1
AT_LOC2
0
1
2
3
Example: State change flow network
LOC1
AT_LOC1
Package1
LOC2
I
t=1
t=2
t=3
AT_LOC2
G
IN_TRUCK1
AT_LOC1
I
Truck1
AT_LOC2
0
1
2
3
Action effects
• STRIPS (binary-valued)
LOC1
LOAD(Package1,Loc1)
PRE: Package1_AT_LOC1, Truck1_AT_LOC1
DEL: Package1_AT_LOC1
ADD: Package1_IN_TRUCK1
• SAS+ (multi-valued)
LOAD(Package1,Loc1)
State change effects
Package1: AT_LOC1  IN_TRUCK1
State change effect
Prevail conditions
Truck1: AT_LOC1  AT_LOC1
Prevail condition
Example: state change flow network
LOC1
LOC2
LOAD(Package1)  DRIVE(Truck1,Loc1,Loc2)  UNLOAD(Package1)
AT_LOC1
Package1
I
AT_LOC2
t=1
t=2
t=3
G
IN_TRUCK1
AT_LOC1
I
Truck1
AT_LOC2
Action effects link multiple networks together
Single state change (1SC) formulation
• Constraints
– State changes (network flow), for all c  C
gC ycf,g,t = 1{f  I}
hC ycg,h,t+1 = fC ycg,h,t
fC ycf,g,T = 1
for f  Dc
for f  Dc , 1  t < T
for g  G
– Effect implications, for all c  C, 1  t  T
aA:(f,g)SCa xa,t = ycf,g,t
xa,t  ycf,f,t
for f, g  Dc, f  g
for aA, (f,f)PRa
Reviving integer programming
1. State variables are modeled by an appropriately defined
network
– The resulting IP formulation can be interpreted as a network
flow problem with additional side constraints
2. The common notion of parallelism based on planning
graphs is generalized
– This reduces the plan length of the solution plan
3. Action ordering constraints are dynamically generated
when needed by a branch-and-cut algorithm
– Branch-and-cut has been extremely successful for a large
number of optimization problems
Generalized parallelism
•
Common notion of parallelism (Graphplan)
– Actions can be executed in the same plan step as long as they
do not interfere with each other (no action deletes the
precondition or add effect of another action)
– In an actual plan it is possible to arrange independent parallel
actions in any order with exactly the same outcome
•
Alternative concept of parallelism
– Allow actions to be executed in the same plan step as long as
there exists a feasible ordering of the actions such that:
• All the actions preconditions are met
• Within each state variable and for each plan step, the value
is changed at most once
Generalized parallelism
Increase in parallelism
requires “generalized”
state changes
Graphplan parallelism
Alternative parallelism
Example: Generalized parallelism
LOAD(Package1,Truck1)
Increase in parallelism
requires “generalized”
state changes
AT_LOC1
Package1
AT_LOC2
IN_TRUCK1
LOAD(Package1,Truck1)
AT_LOC1
Truck1
AT_LOC2
Action effect implications DRIVE(Truck1,Loc1,Loc2)
Implied precedences
• Implied precedences
A4
A1,A2
A3
A1
A3
A4
A2
Implied precedence graph
Implied precedences
• Implied precedences
A4
A1,A2
A3
A1
A3
A4
A2
A4
A1
Implied precedence graph
• Cycle elimination constraints need to ensure feasible
orderings
Generalized single state change (G1SC)
formulation
• Constraints
– State changes (network flow), for all c  C
gC ycf,g,t = 1{f  I}
hC ycg,h,t+1 = fC ycg,h,t
fC ycf,g,T = 1
– Effect implications, for all c  C, 1
for f  Dc
for f  Dc,1  t  T
for g  G Dc
tT
aA:(f,f)SCa xa,t= ycf,g,t
for f, g  Dc, f  g,
xa,t  ycf,f,t + gDc:f≠g (ycg,f,t + ycf,g,t) for aA, (f,f)PRa
– Ordering (Cycle elimination) constraints
 aV() xa,t  |V()| – 1
for all cycles G, 1  t  T
Reviving integer programming
1. State variables are modeled by an appropriately defined
network
– The resulting IP formulation can be interpreted as a network
flow problem with additional side constraints
2. The common notion of parallelism based on planning
graphs is generalized
– This reduces the plan length of the solution plan
3. Action ordering constraints are dynamically generated
when needed by a branch-and-cut algorithm
– Branch-and-cut has been extremely successful for a large
number of optimization problems
Branch-and-cut algorithm
•
Branch-and-cut (row generation with branch-andbound)
– Classes of valid inequalities are left out of the LP relaxation
because there are too many constraints to handle efficiently,
and most of them will not be binding in an optimal solution
anyway
– Branching occurs when no violated inequalities in a class are
found and the LP solution does not satisfy the integrality
constraints
Reviving integer programming
1. State variables are modeled by an appropriately defined
network
– The resulting IP formulation can be interpreted as a network
flow problem with additional side constraints
2. The common notion of parallelism based on planning
graphs is generalized
– This reduces the plan length of the solution plan
3. Action ordering constraints are dynamically generated
when needed by a branch-and-cut algorithm
– Branch-and-cut has been extremely successful for a large
number of optimization problems
Further generalizations
• Allow actions to be executed in the same plan step as long
as there exists a feasible ordering of the actions such that
– All the actions preconditions are met
– For each state variable, more than one state change is
allowed, e.g. “state change paths”
AT_LOC1
Package1
I
t=1
t=2
AT_LOC2
G
IN_TRUCK1
AT_LOC1
I
Truck1
AT_LOC2
0
0’
1
1’
Further generalizations
• There exists a feasible ordering of the actions such that
– All the actions preconditions are met
– For each state variable, more than one state change is
allowed, e.g. “state change paths”
LOAD(Package1)  DRIVE(Truck1,Loc1,Loc2)  UNLOAD(Package1)
AT_LOC1
Package1
I
AT_LOC2
t=1
G
IN_TRUCK1
AT_LOC1
I
Truck1
AT_LOC2
0
0’
State change path (kSC) formulation
• Constraints
– State changes (network flow), for all c  C
gC,fg ycg,f,1 + 1 {f  I} = ycf,f,1
ycf,f,1 = gC,fg ycf,g,1 + zcf,1 for f  C
gC,fg ycg,f,1 + zcf,t-1 = ycf,f,t
ycf,f,t = gC,fg ycf,g,t + zcf,t for f  C, 1  t  T – 1
gC,fg ycg,f,T + zcf,T-1 = ycf,f,T
ycf,f,T = gC,fg ycf,g,T + zcf,T for f  C
– Effect implications, for all c  C, 1  t  T
aA:(f,f)SCa xa,t= ycf,g,t
for f, g  Dc, f  g,
xa,t  ycf,f,t + gDc:f≠g (ycg,f,t + ycf,g,t)
for aA, (f,f)PRa
– Ordering (Cycle elimination) constraints
 aV() xa,t  |V()| – 1 for all cycles G, 1  t  T
Experimental Setup
• Four domains from the International Planning Competition
– Blocks-world, Logistics, Driverlog, Zenotravel
– Used Fast Downward preprocessor (translates PDDL to multi-valued
state variables)
• Compared results of 1SC, G1SC, and kSC to SATPLAN04
(SAT4)
– State-of-the art SAT-based planner
– First prize in the IPC-2004
• Resources
– 2.67GHz Linux machine, 1Gb memory
– ILOG CPLEX 8.1
– 30 minute maximum for each problem
Performance
10000
35
Plan length (# of actions)
SAT4
1SC
1000
Time (seconds)
Quality
G1SC
100
kSC
10
1
0.1
0.01
SAT4
30
1SC
25
G1SC
kSC
20
15
10
5
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35
Problem s (Blocksm ove)
140
10000
SAT4
Plan length (# of actions)
SAT4
1SC
1000
Time (seconds)
Problem s (Blocksm ove)
G1SC
100
kSC
10
1
0.1
120
1SC
100
G1SC
kSC
80
60
40
20
0
0.01
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
Problem s (Logistics)
20
21
22
23
24
25
26
27
28
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
Problem s (Logistics)
20
21
22
23
24
25
26
27
28
Performance
SAT4
1SC
Time (seconds)
100
G1SC
10
kSC
1
0.1
40
Plan length (# of actions)
1000
Quality
1SC
30
G1SC
25
kSC
20
15
10
5
0.01
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Problem s (Zenotravel)
Problem s (Zenotravel)
1000
100
60
SAT4
Plan length (# of actions)
10000
Time (seconds)
SAT4
35
1SC
g1SC
kSC
10
1
0.1
50
40
SAT4
1SC
g1SC
kSC
30
20
10
0
0.01
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Problem s (Driverlog)
Problem s (Driverlog)
Results: plan length
18
SAT4
Plan steps
16
1SC
14
G1SC
12
kSC
10
8
6
4
2
0
Blocks
Logistics
Driverlog
Zenotravel
Summary and conclusions
• First integer programming formulation that is competitive
with state-of-the art SAT-based planner:
– Uses multi valued states and represent state changes as flows in
an appropriately defined network
– Uses generalized notion of parallelism in a branch-and-cut
algorithm
• Future work
– Optimal planning (minimum number of actions, minimum cost
plans, minimum resource usage, minimum makespan)
– Expand the scope of planning problems, apply integer
programming techniques to planning with resources and temporal
planning