Monash University
Hybrid Algorithms, Local Search and
ECLiPSe
Overview
 Motivations
 From Problem to Solution
 Solvers
 Global Constructive Search Hybrids
 Hybrids with Separate Search Routines
 Mappings between Models
 Hybrid Search
2
Combinatorial Optimisation
 Theory
Why aren’t deterministic computers as efficient as
nondeterministic ones?
 Practice
Science and Engineering
Configuration and design
Resource and Task Optimisation
Industry and government organisations
3
Applications of CP in Melbourne
 Internet Routing and Balancing
Telstra
 Supply Chain
Patrick Corp
Berri Ltd
Linfox
Coles Myers
 Transportation
Translogix
IBM
CTI
NAB
 Construction Scheduling
Synctec
Bovis Lend Lease
 Health Systems
Southern Health
4
Four Case Studies
 Logistics with Depots
Wincanton Transport
 Patrol Despatcher
RAC
 Flight Schedule Retimer
BA
 Network resilience
Cisco Systems
5
Logistics with Depots
6
Logistics with Depots: Summary
 Objectives
Generate feasible schedule
Minimise driver/vehicle/travel costs
 Constraints
Collection and delivery time windows
Vehicle capacity data
Travel time data
Vehicle load/unload time data
Driver shift data
 Business Benefits
Strategy – which customers to take
Profitability – how much to charge
7
Patrol Despatcher
The RAC’s Birmingham Despatch Centre
8
Patrol Despatcher : Summary
 Strategic Objectives
Despatch a years’ worth of jobs
Simulate the company’s operation
Meet constraints/business criteria
 Business Benefits
Negotiation with patrols
Test benefits of onsite protocol
Test different resource levels
Test different objectives (cost/QoS)
9
 Operational Objectives
Meet real-time requirements
Decisions match business goals
Handle larger regions
Consistency, completeness
 Business Benefits (planned)
Cost savings
Quick, fair decisions
Better despatching at “borders”
Flexible business process
Flight Schedule Retimer
10
Flight Schedule Retimer: Summary
 Objectives
Retime
scheduled flights
Observing constraints
Minimising changes to existing schedule
 Business Benefits
Aircraft
utilisation gains
Slot profile change
Punctuality improvement
11
Network Resilience: Summary
 Objectives
For each route passing through a network component, find
an alternative that avoids the component
Separate shared link risk groups
Ensure there is sufficient bandwidth in alternative routes
 Business Benefits
Computes minimal network redundancy
Maintains quality of service
Exploits MPLS technology
12
Approaches – Research Communities
 Operations Research
Integer/linear programming
 Metaheuristics
Generate and improve
 Constraint Programming
Constraint programming
13
Combinatorial Optimisation Community:
Big Questions
 What problem features best map to what approaches?
Schedule optimisation problem
10,000 variables Time points
50,000 constraints
1 disjunction
Ti  Tj
(Ti1  Ti 2)  (Tj1  Tj 2)
 Can each approach be “informed” by the others?
Why do metaheuristics improve faster than branch and bound?
 How do we get from a problem definition to a problem solution?
14
Overview
 Motivations
 From Problem to Solution
 Solvers
 Global Constructive Search Hybrids
 Hybrids with Separate Search Routines
 Mappings between Models
 Hybrid Search
15
Problem Definition => Problem Solution
Conceptual Model
PhD
Design Model
Solver Platform
Executable Program
16
Mapping Conceptual to Design Model
Conceptual Model
Add Constraint Behaviour
Add Search
Design Model
17
Logical Transformations
Conceptual Model
 Decompose
 Transform
 Tighten
 Link
Conceptual Model
18
Logical Transformations
Conceptual Model:
 Decompose:
 Transform
 Tighten
P
P  S1  S 2  ...  Sn
Si( X )  S ' i ( X ' )
S ' i ( X ' )  L( X ' )
 Link
Si ( X )  S ' i ( X ' )  Ch( X , X ' )
(PC(X)  Ch(X, X' )  PC' (X' )
PC' (X' )  Ch(X, X' )  PC(X))
Conceptual Model
19
Final Conceptual Model to Design Model
 Conceptual Model
Logic
 Solver Choices
Add constraint behaviour
 Search Choices
Add search
 Design Model
Program
20
Add Constraint Behaviour
 Post constraints to chosen solvers
 Choose information to export from one solver
to another
 Choose information to export from solvers to
search engine
21
Constraint Programming – Nature and Scope
Model
Algorithm
Finite
Domain
Library
22
Interval
Reasoning
Library
Linear
Xpress-MP
CPLEX
Programming
Library
Repair
Library
Add Search
 Choose one global or multiple separate
searches
 Choose type of search
 Choose information to import from and export
to solvers
23
Research Challenge
Example: Workforce Scheduling
Routing
Subproblem
24
Scheduling
What search?
What information?
What control?
Subproblem
Overview
 Motivations
 From Problem to Solution
 Solvers
 Global Constructive Search Hybrids
 Hybrids with Separate Search Routines
 Mappings between Models
 Hybrid Search
25
Constraint Behaviour: Solvers
 Finite Domain
 Linear
 Interval
 Integer/linear (MIP)
 Set
 SAT
 One-way constraints
26
Communication Between Solvers
Import
Information
Events
constraints
wake(Change,Prior)
variables
Export
27
assignments
constraints
heuristics
interrupt(Prior)
return / failure
post(Change)
Finite Domain Solver Information
 Import
Variables and domains
Constraints – primitive,reified,global,…
 Export
Instantiation
Domain bounds
Domain sizes
“Active” constraints on a variable
Entailed constraints (disequalities, inequalities, reified)
Disentailed constraints
28
Constraint Behaviour: Solvers
 Finite Domain
 Linear
 Interval
 Integer/linear (MIP)
 Set
 SAT
 One-way constraints
29
Linear Solver Information
 Import
Linear constraints (2*X + 3*Y – 6*Z =< 24.3)
Optimisation expression (min(2*X –Y))
 Export
Optimum value
An optimal solution
Reduced costs, shadow prices
30
Linear Constraints
Optimal solution
Feasible Region
31
Shadow Prices
X-Y =< 12
Feasible Region
Shadow Price
32
Reduced Costs
Feasible Region
33
Reduced Costs
Variable X
Feasible Region
34
Reduced Costs
Feasible Region
Reduced Cost
35
ECLiPSe Linear Solver Interface: eplex
 lib(eplex)
 An interface between ECLiPSe and an external
LP/MIP solver


XPRESS-MP, a product by Dash Associates
CPLEX, a product by ILOG SA
 Motivation to have this link:

Use state-of-the-art linear programming tools
 Use ECLiPSe for modelling
 Build hybrid solvers
 Implementation

36
Tight coupling, using subroutine libraries
Objective Function
Mathematical Programming
Plant
capacity
Transportation
cost
10
1
 500
5
5
3
3
 400
10
400
B
300
C
7
Modelled as a matrix:
•Each row is one constraint
•Each column is one variable
A1
B1
C1
D1
subject to
A1 +
B1 +
C1 +
D1 +
8
100
37
A
5
 300
11
200
8
9
7
2
Client
demand
minimize
10
8
5
9
D
Constraints
+
+
+
+
7
5
5
3
A2
B2
C2
D2
+ 11 A3 +
+ 10 B3 +
+ 8 C3 +
+ 7 D3
A2
B2
C2
D2
+
+
+
+
A3
B3
C3
D3
=
=
=
=
200
400
300
100
A1 + B1 + C1 + D1  500
A2 + B2 + C2 + D2  300
A3 + B3 + C3 + D3  400
Example problem as a matrix
38
A1 A2 A3
B1 B2 B3 C1 C2 C3 D1 D2 D3
1
1
1
0
0
0
0
0
0
0
0
0
=
200
0
0
0
1
1
1
0
0
0
0
0
0
=
400
0
0
0
0
0
0
1
1
1
0
0
0
=
300
0
0
0
0
0
0
0
0
0
1
1
1
=
100
1
0
0
1
0
0
1
0
0
1
0
0

500
0
1
0
0
1
0
0
1
0
0
1
0

300
0
0
1
0
0
1
0
0
1
0
0
1

400
10
7
11
8
5
10
5
5
8
9
3
7
Modelling MP problems in Eplex
 Eplex can model multiple MP problems
simultaneously
 Each MP problem represented by an eplex
instance :
eplex instance
MP Problem
variables & bounds
X1 X 2
solution values
constraints
... Xm
c1
c2
=
=<
cn
>=
Obj
= Cost
cost
LP / MIP search
39
Transportation Problem in ECLiPSe
:- lib(eplex).
main1(Cost, Vars) :Vars = [A1, A2, A3, B1, B2, B3, C1, C2, C3, D1, D2, D3],
Vars :: 0.0..1.0Inf,
eplex:
eplex:
eplex:
eplex:
(A1
(B1
(C1
(D1
+
+
+
+
A2
B2
C2
D2
+
+
+
+
A3
B3
C3
D3
=:=
=:=
=:=
=:=
200),
400),
300),
100),
eplex: (A1 + B1 + C1 + D1 =< 500),
eplex: (A2 + B2 + C2 + D2 =< 300),
eplex: (A3 + B3 + C3 + D3 =< 400),
eplex: optimize(min(
10*A1 + 7*A2 + 11*A3 +
8*B1 + 5*B2 + 10*B3 +
5*C1 + 5*C2 + 8*C3 +
9*D1 + 3*D2 + 7*D3)).
40
Linear constraints
 Non-strict linear equalities and inequalities
eplex:(X=:=Y)
eplex:(X>=Y)
eplex:(X=<Y)
 No strict inequalities or disequalities allowed
 X and Y are linear expressions
X
123 3.4
+ Expr - Expr
E1 + E2 E1 - E2 Coeff * E2
sum( ListOfExpr )
41
Triggering the solver repeatedly
 eplex_solver_setup(+Objective, -Cost, +Options,
+Priority, +TriggerModes)
+Objective
min(Expr) or max(Expr)
-Cost
variable - it does not get instantiated, but only bounded by the solution cost.
+TriggerModes
inst - if a variable was instantiated
bounds - if a variable bound was changed
deviating_bounds - if a variable bound was changed such that its LPsolution was excluded by more than a tolerance.
new_constraint - when a new constraint appears
trigger(Atom) - explicit triggering
42
Retrieving results from eplex instance
eplex_get(+Handle, +What, -Value)
Solver results
typed_solution
 reduced_cost
 slack
 dual_solution

Statistics
simplex_iterations
 node_count
 statistics

Original setup input

43
vars, ints, constraints, objective
Constraint Behaviour: Solvers
 Finite Domain
 Linear
 Interval
 Integer/linear (MIP)
 Set
 SAT
 One-way constraints
44
Interval Solver Information –
for integer and continuous variables
 Import
Variables and domains
Integrality (or not…)
Constraints – primitive,reified,global,…
 Export
Instantiation
Domain bounds
Domain sizes
“Active” constraints on a variable
Entailed constraints (disequalities, inequalities, reified)
Disentailed constraints
 Interfaces Linear Relaxation and FD Solver
45
Integer/linear (MIP) Solver Information
Exponential Cost Solver
 Import
Linear constraints
Optimisation expression
Integrality constraints
 Export
Optimum value
An optimal solution
Reduced costs, shadow prices
Linear Constraints (“cutting planes”)
Fixed variables
Search node heuristics
Branching heuristics
46
Set Solver Information
Finite Sets of Values
 Import
Set variables and domains
Set cardinality variables and domains
Constraints – primitive,reified,global,…
 Export
FD information on cardinality variables
Values excluded from set
Values included in set
Entailed/disentailed set constraints
47
SAT Solver Information –
Exponential Cost Solver
 Import
Clauses
 Export
Feasible solutions
 If solver can be suspended, can “export”
Clauses (“nogoods”)
Variable choice heuristics
48
Constraint Behaviour: Solvers
 Finite Domain
 Linear
 Interval
 Integer/linear (MIP)
 Set
 SAT
 One-way constraints
49
One-way solver information
 Import
One-way constraints (functions)
Heuristic information
Variable (“tentative”) assignments
 Export
Inconsistency
Heuristic information
Variable (“tentative”) assignments
50
One-way Solver Examples
 X=Y+3
Y:= 2 => X := 5
Y:= 4 => X := 7
 Incremental Computation
S = sum{X,Y,Z}
Y := 4 => S := S_old + 4 – Y_old
 Application
Var = select(mindomain,{X,Y,Z})
Y \= 4 => ???
 Method
Invariant (specialised implementation of incremental one-way solver)
51
One-way Solver in ECLiPSe - repair
 Purpose
To maintain non-logical “tentative” information
 Library Invocation
lib(repair)
 Method
Attach “hidden” information to each variable
Associate events and actions with this info.
52
Tentative Assignments to Avoid Failing
Logical
Assignment
Define Decision Variables
X::1..10
53
X tent_set 8
“X>=5”
Constrain Decision Variables
Search for a Solution
Tentative
Assignment
X=4
X tent_set 4
Fail
Record Conflict
Problem Modelling and Solving
FD
Initialise Decision Variables
Vars::Domain
Constrain Decision Variables
ic: <Cons>
Search for a Solution
54
ic: labeling(Vars)
Repair
Vars tent_set Vals
<Cons> r_conflict Store
repair(Store)
Tentative Invariants
Initialise Decision Variables
[X,Y] tent_set [1,2]
Constrain Decision Variables
Either:
Y =:= X+1 r_conflict a
Or:
Y tent_is X+1
During Search …
X tent_set 3
Record Conflict
Y{4}
55
Overview
 Motivations
 From Problem to Solution
 Solvers
 Global Constructive Search Hybrids
 Mappings between Models
 Hybrid Search
56
Global Constructive Search Hybrid Forms
Add constraint
and wake solvers
Root Node
Node1
Node2
Some of these leaf nodes are our solutions
57
Cooperating Solvers: Intersecting Subproblems
Y
C1
X
Z
C2
Subproblem1
58
Subproblem2
Cooperating Solvers: Copied Variables
Y
C1
X2
X1
Solver1
Keep X1 and X2 in step
Communicate events
Communicate information
59
C2
Z
Solver2
Cooperating Solvers: Copied Variables
Y
C1
X2
X1
X1 >= 10.5
60
X2 >= 11
10.5
Z
integer(X2)
Linear Solver
Info on “X”
C2
Interval Solver
X >=
X1
>=10.5
11
FD and Linear Solver Cooperation
(within a Global Constructive Search)
 Motivation
 Linking through Channeling Constraints
 Forms of Cooperation
61
Motivation for FD Linear Comunication
 Performance
 Behaviour
62
Motivation for FD Linear Communication:
Performance
 FD
e.g. interval propagation
more general constraints
handle integers directly
Favourable example:
Variable Bounds:
[X1,…,X100] :: 1..100
 Linear
e.g.
Simplex, Gauss
restricted class of constraints
finds optimum without search
Favourable example:
Variable Bounds:
[X1,X2] :: 1..100
Previous Constraints:
X1 < X2 , … , X98 < X99
Resulting Bounds:
X1 :: 1..2, …, X99 :: 99..100
New Constraint:
X1 >= 3
Result (1 step): failure!
63
New Constraints:
X 1 > X 2, X 2 > X 1
Result (1 step): failure!
FD and Linear Constraint Solving Performance
 Add N variables and trivial constraints
N = 100, 200, 500, 1000, 2000, 5000, 10,000, 20,000, 50,000
 The Constraints
for each variable X[J] do
X[J]::1..N,
X[J] >= X[J-1].
 The Solvers
FD
CLPQ
CPLEX (run once, after posting ALL constraints)
CPLEX (incremental)
Performance Figures (FD) - X i+1>=Xi
0.5
0
Log.
Time per -0.5
Constraint -1
CLPQ
Inc-CPLEX
CPLEX
FD
-1.5
-2
-2.5
-3
-3.5
-4
2
2.3 2.7
3
3.3 3.7
4
4.3 4.7
Log. # Constraints Added (100 - 50000)
The Less Trivial Case N Propagation Steps per Constraint
 Add N variables and constraints
N = 100, 200, 500, 1000, 2000, 5000, 10,000, 20,000, 50,000
 The Constraints
for each variable X[J] do
X[J]::1..N,
X[J]-1 >= X[J-1].
 The Solvers
FD
CPLEX (run once, after posting ALL constraints)
CPLEX (incremental)
Performance Figures (FD) - X i+1>Xi
0
-0.5
Log.
Time per
-1
Constraint
FD - Xi+1>Xi
Inc-CPLEX
CPLEX
-1.5
-2
-2.5
-3
-3.5
2
2.3 2.7
3
3.3 3.7
4
4.3 4.7
Log. # Constraints Added (100 - 50000)
Motivation for FD Linear Comunication
 Performance
 Behaviour
68
One-machine Scheduling
Task 1 = Start-time 0..15, Duration 5
Task 2 = Start-time 0..15, Duration 15,
Either Task1 must precede Task2:
0
5
10
15
20
15
20
Or Task2 must precede Task1:
0
Constraint:
69
5
10
no_overlap(S1,5,S2,15).
25
Representing Disjunction
with Auxiliary Boolean Variables
no_overlap(S1,D1,S2,D2) :- S1 >= S2+D2.
no_overlap(S1,D1,S2,D2) :- S2 >= S1+D1.
Choose a “large” number M
Introduce variables [B1,B2] :: 0..1
Replace disjuction with conjunction:
no_overlap(S1,D1,S2,D2,B1,B2) :S1+M-B1*M >= S2+D2,
S2+M-B2*M >= S1+D1,
B1+B2>=1.
Example:
[S1,S2] :: 0..4, no_overlap(S1,3,S2,5,B1,B2), B1=1
=>
fail!
70
Communicating Linear and FD Solvers
Optimisation
Integer Feasibility
X,Y::0..10
Y>= 2*X
X+Y >= 5
Cost = min(X+2*Y)
S1,S2::0..4, B1,B2::0..1
S1 + 10 - B1*10 >= S2 + 3
S2 + 10 - B2*10 >= S1 + 5
B1 + B2 >= 1
Cost = min(S1)
FD
Linear
71
Cost ::0..30
B2=0, B1=1
S1 :: 3..4, S2 ::0..1
Cost >= 25/3
Cost >= 0
Hoist Scheduling
Each product must be dipped in a sequence of tanks.
72
Disjunctive Constraint in Hoist Scheduling
Period = P
1 Hoist
action b
0
T1
1
2
3
T2
4
action a
P
P+T1
1
2
3
P+T2
Disjunctive Constraint on actions a and b :
or
73
T2 >= P+T1+timefull12 + timeempty23
P+T1 >= T2+timefull34 + timeempty41
4
Disjunctive Constraint in Hoist Scheduling
Period = P
2 Hoists
action b: hoist H2
0
T1
1
2
3
T2
4
action a: hoist H1
P
P+T1
1
2
3
P+T2
Disjunctive Constraint on actions a and b :
or
or
74
T2 >= P+T1+timefull12 + timeempty23
P+T1 >= T2+timefull34 + timeempty41
H1 < H2
4
Hoist Performance
75
CP solver
MIP
solver
CP/MP
solver
1 hoist
1 track
96 s
10 s
51 s
2 hoists
1track
> 10 min 34 s
96 s
2 hoists
2 tracks
> 10 min > 10 min
152 s
Hoist Robustness -Algorithm
100 Randomly Generated 2-hoist 2-track Problems
76
Optimal Solutions Found
100
Minimum Time
167 secs
Maximum Time
1146 secs
Average Time
314 secs
Hoist Robustnes - Model
 Benchmark Hoist Scheduling Problem
 Tank Capacities
 Two hoists on one track
 Multiple hoists on multiple tracks
77
FD/Linear Hybrid
Previous approaches
Same basic model for
Completely different
each problem class
models
FD and Linear Solver Cooperation
(within a Global Constructive Search)
 Motivation
 Linking through Channeling Constraints
 Forms of Cooperation
78
Different Models in FD and Linear
Distances Table: Data
X1 X2 X3 X4 X5 X6
X2
X1
X5
X4
X3
X6
TSP Problem: Find shortest route
79
X1
0
5
2
4
1
7
X2
5
0
4
7
2
5
X3
2
4
0
4
6
1
X4
4
7
4
0
3
5
X5
1
2
6
3
0
2
X6
7
5
1
5
2
0
ECLiPSe Syntax
 Iteration
foreach(El,[a,b,c,d]) do write(El),write(‘ ‘).
for(K,1,4) do write(K), write(‘ ‘).
for(K,1,4),foreach(El,List) do El=K
abcd
1234
List = [1,2,3,4]
Const=a, (foreach(El,List), for(_,1,4), param(Const) do El=Const)
List = [a,a,a,a]
Const=a, (foreach(El,List), for(_,1,4) do El=Const)
List = [_,_,_,_]
1-1 2-2 3-3 4-4
for(K,1,4),for(J,1,4) do write(K-J),write(‘ ‘).
multifor([K,J],[1,1],[3,3]) do write(K-J),write(‘ ‘)
1-1 1-2 1-3 2-1 2-2 2-3 3-1 3-2 3-3
80
ECLiPSe Syntax
 Arrays
Struct = f(a,b,c,d), X is Struct[2].
X=b
Matrix = m(f(11,12),f(21,22),f(31,32)), X is Matrix[3,2]
X=32
Matrix = m(f(11,12),f(21,22),f(31,32)), X is Matrix[1..3,2]
X=[12,22,32]
dim(M,[2,2]), A is M[1,2]
M = []([](_,A),[](_,_))
dim(M,[2,3]),
( multifor([K,J],[1,1],[2,3]), param(M) do X is M[K,J], X=K-J)
M = []([](1-1, 1-2, 1-3), [](2-1, 2-2, 2-3))
81
TSP Problem – ECLiPSe FD Design Model
length(Vars,6), ic:(Vars :: 1..6), ic:alldifferent(Vars),
length(Dists,6), ic:(Dists :: 1..10),
( for(J,1,6),
foreach(SJ,Vars),
foreach(DJ,Dists),
param(Data)
do
ListJ is Data[J,1..6]), ic:element(SJ,ListJ,DJ) ),
ic: (Cost =:= sum(Dists))
For example:
ic:element(S3,[2,4,0,4,6,1],D3)
82
TSP Problem – ECLiPSe Linear Design Model
dim(Bools,[6,6]), Bools::0..1,
( for(J,1,6),
param(Bools)
do
eplex: (Bools::0..1),
eplex: (sum(Bools[J,1..6]) =:= 1),
eplex: (sum(Bools[1..6,J]) =:= 1) ),
( multifor([I,J],[1,1],[6,6]),
foreach(C,Costs),
param(Bools,Data)
do
C = Bools[I,J]*Data[I,J] ),
eplex:(Cost =:= sum(Costs)).
83
Mapping FD Variables to Booleans
[ic,eplex]: (Var :: 1..N),
length(Bools,N), eplex: (Bools::0..1), eplex: (sum(Bools) =:= 1),
(foreach(B,Bools), for(Val,1,N), for(E,Exprs) do E = B*Val),
eplex: (Var =:= sum(Exprs)).
Channeling constraint in linear solver
84
Cooperating Solvers: Copied Variables
Channeling constraint in linear solver
Bk
C1
Var
Bk=1
C2
Var
Z
Var=k
Linear Solver
Info on “Var”
85
FD Solver
Cooperating Solvers: Copied Variables
Channeling constraint in linear solver
Bk
C1
Var
Bk=1
Var
Linear Solver
Info on “Var”
86
C2
Var=k
FD Solver
Z
Mapping FD Variables to Booleans
ic:
(Var :: 1..N),
length(Bools,N), [ic,eplex]: (Bools::0..1), eplex:(sum(Bools) =:= 1),
( foreach(B,Bools), for(Val,1,N), param(Var) do
ic: =:=(Var, Val, B)
).
Channeling constraint in FD solver
87
FD and Linear Solver Cooperation
(within a Global Constructive Search)
 Motivation
 Linking through Channeling Constraints
 Forms of Cooperation
88
Information Communicated from FD to Linear Solvers
 Variable Bounds
 Variable instantiation
Variable remains in linear solver, with identical bounds
 Linear Constraints
Delay
MLLP
89
Communicating FD to LP: Delay
 X*B >= Y
As soon as B is instantiated to 1, send X>=Y to linear solver
As soon as B is instantiated to 0, send 0>=Y to linear solver
As soon as X is instantiated, send X*B >=Y to linear solver
 ECLiPSe Syntax
suspend(eplex: (X*B>=Y), 3, [X,B] -> inst)
90
Communicating FD to LP: MLLP
Hooker’s Mixed Logical Linear Programming
 FD(X1,…,Xn) => LP(Y1,…,Ym)
If search or propagation entails FD constraint,
then LP constraint is posted to linear solver
 ECLiPSe Syntax
mllp_1(X)  X#>5 | eplex: (Y >= X)
91
Information Communicated from Linear to FD Solvers
 Directly
Cost Bounds
 Indirectly
Reduced costs => Var ≠Val
often by fixing Bval=0
92
Reduced Cost Pruning
Cost
Values ruled out
by reduced cost
Infeasible
Region
Best solution so far
0
0.2
0.4
0.6
0.8
Value of boolean
93
1
Relaxed Optimum
Branch-and-bound (incremental)
Cost
First solution
Solutions
Better solution
Optimal solution
No solution (Proof of optimality)
Lower bound
(relaxed solution)
Iterations:
94
1
2
3
4
Communicating Linear and FD Solvers
Optimisation
X,Y::0..10
Y>= 2*X
X+Y >= 5
Cost = min(X+2*Y)
FD
Linear
95
Cost ::0..30
Cost >= 25/3
Branch-and-bound (incremental)
Cost
First solution
Solutions
Better solution
Optimal solution
No solution (Proof of optimality)
Lower bound
(relaxed solution)
Iterations:
96
1
2
3
4
Global Constructive Search – FD and Linear
Solvers
Add constraint
and wake FD and
linear solvers
Root Node
Node1
Node2
Some of these leaf nodes are our solutions
Reasoning at each node:
Repeat
Reduce FD domains
Tighten linear relaxed cost bound
Until fixpoint
97
Linear and FD Constraints
as Fixpoint Operators
FD fixpoints
Linear Fixpoints
FD&Linear
Information
98
Information Communicated from FD and Linear
to Search Engine
 FD
Domain size
Number of active constraints
 Linear
Relaxed solution
Reduced costs => “regret”
99
Combining Linear and FD – Horses for Courses
Problem
100
Progressive
Cabinet
Set
Party
Assignment
Partitioning
CP
> 5 mins
9.27 secs
64.68 secs
LP
> 5 mins
> 5 mins
0.18 secs
CP & LP
91.5 secs
13.2 secs
0.71 secs
Overview
 Motivations
 From Problem to Solution
 Solvers
 Global Constructive Search Hybrids
 Hybrids with Separate Search Routines
 Mappings between Models
 Hybrid Search
101
Hybridisation Forms with Separate
Subproblem Search Routines
Subproblem A
Search & Solver 1
102
Subproblem B
Search &Solver 2
Hybridisation using “Nogoods”
A
Submodel
Solver
B
Submodel
Solver
Solution to subproblem A is infeasible for subproblem B
=> Add constraint to subproblem A precluding similar solutions
103
Solver Requirements
 Extract “Minimal” Cause of Conflict
General
Application-specific
 Export Constraints
Treatable by solver A
Exclude current violation
 Leave choice points if necessary
Disjunction of alternatives should not preclude any feasible
solutions
104
Different Models in FD and Linear
Distances Table: Data
X1 X2 X3 X4 X5 X6
X2
X1
X5
X4
X3
Flow out of subset >= 1
 B
jk
1
K { 4 , 5 , 6} J {1, 2 , 3}
X6
TSP Problem: Find shortest route
105
X1 0
5
2
4
1
7
X2 5
0
4
7
2
5
X3 2
4
0
4
6
1
X4 4
7
4
0
3
5
X5 1
2
6
3
0
2
X6 7
5
1
5
2
0
Hybrisation of FD and One-way solver using nogoods
Subproblem A
Temporal Constraints
Interval Solver
106
Subproblem B
Resource Constraints
One-way Solver
ECLiPSe Repair library
 Tentative Values
Vars tent_set Vals
Vars tent_get Vals
 Conflict Sets
conflict_constraints(Cset,CCs)
conflict_vars(CVars,CCs)
 Constraint Annotations
Constraint r_conflict Cset
Var tent_is Expression
107
Example: Bridge Scheduling
 Schedule tasks to build a bridge
Schedule consists of different tasks that share several resources
(eg. excavator, pile-driver)
Some tasks depend on others
 Temporal Constraints
e.g time for concrete to set
 Disjunctive resource constraints
pile-driver can only be used in one task at a time
108
The Bridge Scheduling Problem
T1
M1
T2
T3
M3
M2
T5
T4
M6
M5
M4
B6
S6
B1
S1
A1
A6
B2
S2
A2
B3
S3
A3
B5
S5
A4
P1
109
B4
S4
P2
A5
Bridge Scheduling Problem
No. Name Description
Duration
1
PA
Beginning of project
0
2
A1
Excavation (abutment 1)
4
3
A2
Excavation (pillar 1)
2
…
8
P1
Foundation Piles 1
20
9
P2
Foundation Piles 2
13
10
U1
Erection of temporary housing
10
11
S1
Formwork (abutment 1)
8
…
17
B1 Concrete Foundation (abut. 1)
1
…
23
AB1 Concrete Setting Time (abut 1)
1
…
29
M1 Masonry work (abutment 1)
16
…
35
L
Delivery of preformed Bearers
2
36
T1
Positioning (preformed bearer 1) 12
…
42
V1
Filling 1
15
…
46
PE
End of Project
0
110
Resource
excavator
excavator
pile-driver
pile-driver
carpentry
concrete-mixer
-
bricklaying
crane
crane
caterpillar
-
Temporal constraints on the Schedule
 Precedence
before(Start1,Duration1,Start2)
 Maximal delay
before(Start2,-Delay,Start1)
before(Start1,Duration1,Start2) :ic: (Start1 + Duration1 =< Start2).
Resource Constraints
Machine
End
Start
D1
D2
D3
D4
D5
Tasks
noclash(S1,D1,S2,D2) :- ic: (S1>=S2+D2).
noclash(S1,D1,S2,D2) :- ic: (S2>=S1+D1).
112
Non-repair Search: Labeling
before(S1,D,S2),
…
noclash(S0,D0,S1,D1),
…
mylabeling(Starts) :( foreach(S,Starts)
do
mindomain(S,S)
).
113
Repair Search: Probe Backtracking
before(S1,D,S2)
…
noclash(S0,D0,S1,D1) r_conflict cs
…
tent_set_to_min(Vars) :( foreach(Var,Vars),
foreach(Min,Mins)
do
mindomain(Var,Min)
),
Vars tent_set Mins.
 Tentative Values are earliest
possible start times
 They satisfy temporal
constraints
 They may violate resource
constraints
114
A disjunctive constraint is in conflict
 Probe can cause violation (overlap)
repair_label :conflict_vars(cs,CVs),
tent_set_to_min(CVs),
conflict_constraints(cs,CCs),
( CCs == [] ->
true
;
CCs = [Constraint|_],
call(Constraint),
%% Fix the constraint
%% create choice point
%% tighten var bounds
repair_label
).
115
Repair Search: Top level
 Fix to tentative values after all conflicts resolved
minimize(
( repair_label, Starts tent_get Starts ),
End_date
).
116
Adding Multiples of Linear Constraints
(A) 2X + 3Y =< 7
(B) 3X + 6Y =< 12
A*3: 6X + 9Y =< 21
B*2: 6X + 12Y =< 24
A*3+B*2: 12X + 21Y =< 45
Optimisation expression: 12X + 21Y
Optimum value: 45
Dual values:
117
for constraint A:3
for constraint B:2
Adding Multiples of Linear Constraints
(A) 2X + 3Y =< 8
(B) 3X + 6Y =< 10
A*3: 6X + 9Y =< 24
B*2: 6X + 12Y =< 20
A*3+B*2: 12X + 21Y =< 44
Optimisation expression: 12X + 21Y
Optimum value: 44
Dual values:
118
for constraint A:3
for constraint B:2
Hybridisation Using Benders Decomposition
A
Submodel
Solver
B
Submodel
Solver computing duals
Solution to subproblem A is infeasible for subproblem B
=> Add Benders cut to subproblem A precluding similar solutions
119
Hybridisation using Column Generation
A
Allocate
Tasks
Solver computing duals
B
Assign a
resource
Solver
Solution to subproblem B is used to build solutions to A
=>
Add cost bound focussing B on useful assignments
120
Requirement
 Access to Dual Values
 Encapsulation of Hybridisation Technique
Automatic generation of Benders Cut/Column
Automatic imposition of cut/cost bound
Automatic detection of stopping condition
 Handling of Numerical Instability
Automatic avoidance of looping
User hooks to control tailing off behaviour
121
Plug-and-play with Column Generation
1-Dimensional Cutting Stock Example
Stock:
Boards of length 17
Required: 25 boards of length 3
20 boards of length 5
15 boards of length 9
Minimise: Wastage
122
Plug-and-play with Column Generation
:- lib(ic), lib(colgen).
solve(Cost) :Vars = [Threes,Fives,Nines], Vars #:: 0..10,
3*Threes+5*Fives+9*Nines #= Used,
Used #=< 17,
Waste #= 17-Used,
colgen: (sum(Threes) >= 25),
colgen: (sum(Fives) >= 20),
colgen: (sum(Nines) >= 15),
minimize(my_solve(C,Vars), sum(Waste), Cost).
123
Plug-and-play with Column Generation
my_solve(C,Vars) :labeling(Vars),
colgen: sol(C,Vars).
 my_solve routine written by problem solver
 Master problem dual automatically created by library
 Dual cost constraint automatically added to subproblem
124
Overview
 Motivations
 From Problem to Solution
 Solvers
 Global Constructive Search Hybrids
 Hybrids with Separate Search Routines
 Mappings between Models
 Hybrid Search
125
Logical Transformations
Conceptual Model
 Decompose
 Transform
 Tighten
 Link
Conceptual Model
126
Problem Decomposition
 Send variables to different solvers
 Send constraints to different solvers
 Send to multiple solvers
Variables
Constraints
127
Linking Constraints to Solvers in ECLiPSe
:- lib(ic).
solve(Digits) :Digits = [S,E,N,D,M,O,R,Y],
integers(Digits),
ic: (Digits :: 0..9),
SEND
+ MORE
= MONEY
ic: alldifferent(Digits),
ic: (S >= 1), ic: (M >= 1),
ic: (1000*S + 100*E + 10*N + D
+ 1000*M + 100*O + 10*R + E
=:= 10000*M + 1000*O + 100*N + 10*E + Y),
labeling(Digits).
128
Requirement: Generic Solver Interface
:- lib(eplex).
solve(Digits) :Digits = [S,E,N,D,M,O,R,Y],
eplex: integers(Digits),
eplex: (Digits :: 0..9),
SEND
+ MORE
= MONEY
alldifferent(0,9,Digits),
eplex: (S >= 1), eplex: (M >= 1),
eplex:
(1000*S + 100*E + 10*N + D
+ 1000*M + 100*O + 10*R + E
=:= 10000*M + 1000*O + 100*N + 10*E + Y),
optimize(min(sum(Digits),_Min).
129
Mapping “SEND+MORE=MONEY” to both ic and eplex
:- lib(ic), lib(eplex).
solve(Digits) :Digits = [S,E,N,D,M,O,R,Y],
ic:integers(Digits),
[ic,eplex]: (Digits :: 0..9),
SEND
+ MORE
= MONEY
ic: alldifferent(Digits),
[ic,eplex]: (S >= 1), [ic,eplex]: M >= 1,
[ic,eplex]: (1000*S + 100*E + 10*N + D
+ 1000*M + 100*O + 10*R + E
=:= 10000*M + 1000*O + 100*N + 10*E + Y),
eplex_solver_setup(min(sum(Digits)),_Min,[],0,[inst]),
labeling(Digits).
130
Logical Transformations
Conceptual Model
 Decompose
 Transform
 Tighten
 Link
Conceptual Model
131
Transform
 Global constraints
e.g. turn a set of disjunctions into an “atmost”
 Dual forms
e.g. Constraints -> variables, variables -> constraints
 “Manageable” constraints
e.g. eliminate disjunctions
 Separable subproblems
e.g. multiple resource scheduling problem -> schedule each
resource separately
132
A CLP(R) Model
solve(W,X,Y,Z) :imp(X,W), imp(Y,W), two(X,Y,Z).
imp(X,Y) :- X=<Y.
two(X,Y,Z) :- and(X,Y).
two(X,Y,Z) :- and(Y,Z).
two(X,Y,Z) :- and(Z,X).
and(X,Y) :- X>=1, Y>=1.
?- solve(W,X,Y,Z), W=0.
133
Removing Disjunction from a CLP Model
solve(W,X,Y,Z) :imp(X,W), imp(Y,W), two(X,Y,Z).
imp(X,Y) :- X=<Y.
two(X,Y,Z) :- and(X,Y).
two(X,Y,Z) :- and(Y,Z).
two(X,Y,Z) :- and(Z,X).
and(X,Y) :- X>=1, Y>=1.
?- solve(W,X,Y,Z), W=0.
134
solve(W,X,Y,Z, B) :imp(X,W, B), imp(Y,W, B), two(X,Y,Z, B).
imp(X,Y, B) :- X=<Y+ 2-2*B.
two(X,Y,Z, B) :[B1,B2,B3] :: 0..1,
and(X,Y, B1), and(Y,Z, B2), and(Z,X, B3),
B1+B2+B3 >= B.
and(X,Y, B) :- X+ 2-2*B >= 1, Y+ 2-2*B >= 1.
?- solve(W,X,Y,Z, 1), W=0.
Removing Disjunction from a CLP(R) Model
 Add boolean to all predicates
solve(W,X,Y,Z, B) :- imp(X,W, B), imp(Y,W, B), two(X,Y,Z, B).
 Call the goal with extra boolean=1
?- solve(W,X,Y,Z,1).
 Add M*(1-B) to inequations
imp(X,Y, B) :- X=<Y+ 2-2*B.
 Replace disjunction with boolean inequations
two(X,Y,Z, B) :[B1,B2,B3] :: 0..1,
and(X,Y, B1), and(Y,Z, B2), and(Z,X, B3),
B1+B2+B3 >= B.
135
Logical Transformations
Conceptual Model
 Decompose
 Transform
 Tighten
 Link
Conceptual Model
136
Tighten
 By programmer
Experience
Maths
 Automatically
Propagation
Intelligent methods
137
Redundant Constraints
 Cover a mutilated chess
board with dominoes:
dominoes(List) :length(List,31),
foreach(D,List) do domino(D),
guess(List).
domino(X-Y) :[X,Y] :: 2..63, Z :: 1..7,
(Y #= X+8) or Y #= X+1 and (X+Z)/8 #= _.
guess(List) :for(N,2,63), param(List) do
member(X-Y,List),
(X=N ; Y = N).
138
Redundant Constraints
 Cover a mutilated chess
board with dominoes:
dominoes(List) :length(List,31),
foreach(D,List) do domino(D),
bw(List,B,W), B#=30, W#=32,
guess(List).
bw([],0,0).
bw([X-Y|T],B,W) :- bw(T,BR,WR), B #=BR+1, W #= WR+1.
139
Logical Transformations
Conceptual Model
 Decompose
 Transform
 Tighten
 Link
Conceptual Model
140
Link
 Channeling constraints
 Communicating intermediate solutions
141
Summary
Conceptual Model
Add Constraint Behaviour
Add Search
Design Model
142
Overview
 Motivations
 From Problem to Solution
 Solvers
 Global Constructive Search Hybrids
 Hybrids with Separate Search Routines
 Mappings between Models
 Hybrid Search
143
Hybrid Search
 Incomplete Constructive Search
 Local Search
 Hybrid Schemes
 Current Framework
144
Incomplete Constructive Search - Labelling
 Select a variable
 Allocate credit to each value in the variable’s domain
 Choose a value
Credit Based Search
Incoming Credit:
Allocated credit:
145
16
8
4
2
1
1
0
Incomplete Constructive Search - Labelling
 Select a variable
 Allocate credit to each value in the variable’s domain
 Choose a value
Best N
Incoming Credit:
Allocated credit:
146
3
3
3
3
0
0
0
Incomplete Constructive Search - Labelling
 Select a variable
 Allocate credit to each value in the variable’s domain
 Choose a value, and take its credit
Limited Discrepancy Search
Incoming Credit:
Allocated credit:
147
2
2
1
0
0
0
0
Incomplete Constructive Search- Other
 Step search
X=a v X ≠ a
 2-variable constraints
S1 ≥ S2+D2 v S2 ≥ S1+D1
 Range splitting
X>4v X≤4
148
Based on Value Choice Heuristics
 Problem-specific
N-queens: middle-out
 Relaxed solution
Linear relaxation
Tree relaxation
Relaxed local search (Kamarainen)
 Previous solution
 Limited Discrepancy Search
Only count a discrepancy when two heuristics agree
149
Local Search
 Min conflicts
change a variable in conflict
(improves expected time to solve 2-SAT)
 Guided Local Search
Escape from local opt. by penalising solution “features”
? Can we change viewpoint so “features” are represented
by new-variable assignments ?
? Are feature variables also decision variables ?
150
Hybrid Schemes
 MIP and Row Generation
 Repair variables in conflict
 Iterated search methods
 Local search hybrids
Core/dependent variables
Large Neighbourhoods
151
MIP – Bridge Scheduling
before(S1,D,S2),
…
noclash(S0,D0,S1,D1),
…
optimize(End,Cost).
noclash(S1,D1,S2,D2) :eplex:[B1,B2]::0..1,
eplex:integers([B1,B2]),
eplex:(S1+M-B1*M>=S2+D2),
eplex:(S2+M-B2*M>=S1+D1),
eplex:(B1+B2 >= 1)
Solve linear relaxation
Find 0+e < Bool < 1-e
Branch on Bool =< 0 or Bool >= 1
effectively branch on S2>=S1+D1 or S1 >= S2+D2
152
Hybrid Bridge Scheduling
 Probe can cause violation (overlap)
repair_label :conflict_vars(CVs),
tent_set_to_min(CVs),
conflict_constraints(cs,CCs),
( CCs == [] ->
true
;
CCs = [Constraint|_],
call(Constraint),
%% Fix the constraint
%% create choice point
%% affect probe
repair_label
).
153
A repair heuristic
 Pick a constraint according to a heuristic
noclash(S0,D0,S1,D1) r_conflict cs - tasks(Task0,Task1)
select_constraint(TaskPairs,SelectedPair) :(
foreach(TaskPair,TaskPairs),
foreach(Key-TaskPair,KeyedCCs)
do
TaskPair = tasks(
task with [duration:D0] ,
task with [duration:D1]),
Key is min(D0,D1)
),
sort(1,>,KeyedCCs,[_- SelectedPair|_]).
154
Results – MIP vs Hybrid: LP Nodes
20000
18000
16000
LP Nodes
14000
12000
10000
8000
6000
4000
2000
0
10
20
Number of Activities
155
50Integer/Linear
Programming
Unimodular Probing
Exploring search spaces
partial assignments
Tree search:
• constructive (partial/total assignments)
• systematic
• complete or incomplete
“Local” search:
• move-based (only total assignments)
• random or systematic
• incomplete
156
A Tentative Assignment
 The Conflict Region
Remaining Variables
Changed Variables
157
Conflict
Region
of Violated
Constraints
Tree Search Employing Tentative Assignments
a
a
c
cb
b
c
c
c
158
a
Local
Tree Search with Tentative Assignments
And with Finite Domains
Initialise Decision Variables
Vars::Domain,
Vars tent_set Vals
Constrain Decision Variables
ic: <Cons>
<Cons> r_conflict a
Search for a Solution
159
search(a) :( find_conflict( a, Var) ->
change_tent_val(Var),
ic:indomain(Var),
search(a)
;
true
).
Local Search - algorithm template
 local_search:
set starting state
while global_condition
while local_condition
select a move
if acceptable
do the move
if new optimum
remember it
endwhile
set restart state
endwhile
160
Local Search instances
 Algorithm parameters:
starting (and restarting) state
global and local condition
possible moves and how to select one
when is a move accepted
 Different parameters yield different
algorithms:
random walk
hill climbing
simulated annealing
tabu search
... and many variants
161
Hill climbing































162
hill_climb(MaxTries, MaxIter, VarArr, Profit, Opt) :init_tent_values(VarArr, 0),
%
(
for(I,1,MaxTries),
%
fromto(0, Opt1, Opt4, Opt),
param(MaxIter,Profit,VarArr)
do
(
for(J,1,MaxIter),
%
fromto(Opt1, Opt2, Opt3, Opt4),
param(I,VarArr,Profit)
do
Profit tent_get PrevProfit,
(
flip_random(VarArr),
%
Profit tent_get CurrentProfit,
CurrentProfit > PrevProfit,
%
conflict_constraints(cap,[])
->
( CurrentProfit > Opt2 ->
%
printf("Found solution with
Opt3=CurrentProfit
%
;
Opt3=Opt2
%
)
;
Opt3=Opt2
%
)
),
init_tent_values(VarArr, 0)
%
).
starting solution
global condition
local condition
try a move
is it uphill?
% is it a solution?
is it new optimum?
profit %w%n", [CurrentProfit]),
accept and remember
accept
reject (move undone)
restart
Techniques used here
 Move operation and acceptance test:
They are within the condition part of the if-then-else construct, so:
If the acceptance test fails (no solution or objective not improved) the
move is automatically undone by backtracking!
 Detecting solutions:
Constraint satisfaction is checked by checking whether the conflict
constraint set is empty
 Monitoring cost/profit:
Retrieve tentative value of Profit-variable before and after the move to
check whether it is uphill
Since the move changes the tentative values of some variable(s),
tent_is/2 will automatically update the Profit variable!
163
Interleaved Search Methods
 Interleaved Construction and repair
1) Construct a partial solution till no consistent extension
2) Make this partial solution “tentative”
3) Choose a value for the “next” variable
4) Repair the tentative solution
 Instances
Greedy construction + local search repair
Weak commitment – repair by constructive search
Iterated search – retain completeness by distinguishing instantiated and
repairable variables
165
From Local to Global
 Local search on small set of “driving” variables
Constructive search to complete solution and cost
 Move by unfixing part of solution
Create move by constructive search
Pesant – best neighbour
Large neighbourhood search
Linear optimisation (Le Pape)
166
Current Framework
partial assignments
Move between (consistent) partial solutions
Move “upwards” by unfixing a variable
Move “downwards” by fixing a variable
Limitations of diagram:
Should go to ancestor node
Should rearrange tree after move
167
Current Framework
 Unfix
Which variable?
What dependency information should be kept?
 Fix
Which variable?
What propagation?
 Move by unfixing and fixing
Can this successfully encompass complex moves, e.g.2-swaps?
Is there a “viewpoint” that makes this equivalent to unfix+fix?
168
Summary
 Separate Model from Solution
No
 Separate Constraint Behaviour from Search
No
 Bring together MP, CP and SSP
No
 Consequence
There’s lots to be done 
169