Integer Programming, a Technology
Anureet Saxena
ACO PhD Student,
Tepper School of Business,
Carnegie Mellon University.
Acknowledgements
• Parents
• Dr. Egon Balas
• Thesis Committee
• Dr. Egon Balas
• Dr. Samuel Burer
• Dr. Gerard Cornuejols
• Dr. Francois Margot
• Faculty Members, staff and friends …
Anureet Saxena, TSoB
1
Disjunctive Cuts for Non-Convex Mixed Integer
Quadratically Constrained Problems
Anureet Saxena, TSoB
2
Disjunctive Cuts for Non-Convex MIQCP
• A. Saxena, P. Bonami and J. Lee, Disjunctive Cuts
for non-convex Mixed Integer Quadratically
Constrained Problems (to appear in IPCO 2008).
Anureet Saxena, TSoB
3
MIQCP
aT
0x
m in
st
x T A i x + aT
i x + bi ·
xj 2
l· x ·
0; i = 1 : : : m
Z; j 2 NI
u
Integer Constrained
Variables
Symmetric Matrices
NOT necessarily
positive semidefinite
Anureet Saxena, TSoB
4
MIQCP
aT
0x
m in
st
A i :Y + aT
i x + bi ·
xj 2
l · x ·
Y =
0;
i = 1:::m
Z;
j 2 NI
u
xx T
yi j = x i x j
Anureet Saxena, TSoB
5
Research Question?
Determine lower bounds on the
optimal value of MIQCP by
constructing strong convex
relaxations of MIQCP.
Anureet Saxena, TSoB
6
Disjunctive Programming
Polyhedral Relaxation
Disjunction
P = f x j A x ¸ bg
Separation Problem
Given x2P show that x2PD or find an inequality which
is satisfied by all points in PD and is violated by x.
Anureet Saxena, TSoB
7
Disjunctive Programming
T heorem : x 2 PD if and only if t he opt im al
value of t he f ollow ing cut generat ing linear program ( CGL P ) is non-negat ive.
m i n ®x ¡ ¯
s:t
® = ut A + vt D t
8t = 1 : : : q
¯ · u t b + v t dt
8t = 1 : : : q
ut ; vt ¸ 0
8t = 1 : : : q
CGLP
Pq
t
t t
t= 1( u » + v » ) = 1
Anureet Saxena, TSoB
8
Disjunctive Programming
Polyhedral Relaxation
Disjunction
P = f x j A x ¸ bg
Outer Approximation
of MIQCP defined by
the incumbent solution
Anureet Saxena, TSoB
9
Disjunctive Programming
Polyhedral Relaxation
Disjunction
P = f x j A x ¸ bg
What are the sources
of non-convexity in
MIQCP?
Anureet Saxena, TSoB
10
Disjunctive Programming
Polyhedral Relaxation
Disjunction
P = f x j A x ¸ bg
Integrality Constraints
Y=xxT
• xj2 Z j2 NI
• Elementary 0-1 disjunction
?
(xj · 0) OR (xj ¸ 1)
• Split Disjunctions
• GUB Disjunctions
Anureet Saxena, TSoB
11
Y=xxT
Y=xxT
All eigenvalues of
Y-xxT are equal to
zero.
Eigenvectors of Y-xxT
associated with non-zero
eigenvalues can be used as
sources of cuts
Anureet Saxena, TSoB
12
Y=xxT
Ohh!!
I don’t like fractional
components. I can use
them to get good cuts
MILP
Anureet Saxena, TSoB
13
Y=xxT
Ohh!!
I don’t like non-zero
eigenvalues. I can use
them to get good cuts
MIQCP
Anureet Saxena, TSoB
14
Negative Eigenvalues of Y-xxT
^ ¡ x^ x^ T ) c =
If ( Y
¸ c w here ¸ < 0 t hen
² ( cT x) 2 · Y:ccT is a c o n v ex quadrat ic cut
^)
w hich cut s o® ( x^ ; Y
² equivalent t o im p osing t he SD P condit ion
Y ¡ xx T ¸ SD P 0 by SO CP cut s.
Anureet Saxena, TSoB
15
Positive Eigenvalues of Y-xxT
^ ¡ x^ x^ T ) c =
If ( Y
¸ c w here ¸ > 0 t hen
Y:ccT · ( cT x) 2
is a n o n - c o n v ex quadrat ic cut w hich cut s o®
^ ).
( x^ ; Y
Y:ccT ·
t =
Univariate non-convex
expression
Anureet Saxena, TSoB
t2
cT x
16
Positive Eigenvalues of Y-xxT
m i n ( x;Y ) 2 OA cT x
m ax ( x;Y ) 2 OA cT x
( cT x) 2
cT x
Y:ccT · ( cT x) 2
Anureet Saxena, TSoB
17
Positive Eigenvalues of Y-xxT
p( cT x) + q
Secant
Approximation
Y.ccT· p(cTx) + q
cT x
Anureet Saxena, TSoB
18
Positive Eigenvalues of Y-xxT
µL
µ
p1 ( cT x) + q1
µU
p2 ( cT x) + q2
cT x
Anureet Saxena, TSoB
19
Positive Eigenvalues of Y-xxT
"
µL ( c) · cT x · µ
Y:ccT · p1 ( cT x) + q1
#
"
W
Anureet Saxena, TSoB
µ · cT x · µU ( c)
Y:ccT · p2 ( cT x) + q2
20
#
Cutting Plane Algorithm
^)
( x^ ; Y
E xt ract E igenvalues
and E igenvect ors of
^ ¡ x^ x^ T .
Y
¸ < 0
( cT x) 2 · Y:ccT
¸ > 0
Y:ccT · ( cT x) 2
Convex
Quadratic Cut
Derive Disjunction
CGLP
Derive Disjunctive
Cut
Anureet Saxena, TSoB
21
Cutting Plane Algorithm
Y.ccT · (cT x)2
Can we improve the disjunctive
cuts by choosing c more
Convex
carefully?
Quadratic Cut
Derive Disjunction
CGLP
Derive Disjunctive
Cut
Anureet Saxena, TSoB
22
A Lesson from MILP
Lift and Project
Cuts
Intersection
Cuts
Elementary 0-1
Disjunctions
Balas and Peregaard ‘02
Balas ’72
Anderson, Cornuejols & Li ‘04
Intersection cuts are canonical
disjunctive cuts which are used
to determine improving directions
or reduced costs for improving
L&P cuts.
Anureet Saxena, TSoB
23
A Lesson from MILP
Y.ccT · (cx)2
?
Anureet Saxena, TSoB
Lift and Project
Cuts
24
A Lesson from MILP
Secant
Approximation
Y.ccT · (cx)2
Lift and Project
Cuts
p( cT x) + q
Y.ccT· p(cTx) + q
cT x
Anureet Saxena, TSoB
25
A Lesson from MILP
Secant
Approximation
Y.ccT · (cx)2
Lift and Project
Cuts
P r o p o sit io n L et f ( t ) = t 2 f or t 2 [a; b], and let
g( t ) = t ( a+ b) ¡ ab b e t he secant approxim at ion
of f in [a; b]. T hen
µ
Error
cT x
a
m ax t 2 [a;b] ( g( t ) ¡ f ( t ) ) =
¶
b¡ a 2
2
b
E r r or /
wi dt h 2
width
Anureet Saxena, TSoB
26
A Lesson from MILP
W e are searching f or vect ors c w hich sat isf y,
This condition is always satisfied
if c belongs to vector space
spanned by eigenvectors of YxxT associated with positive
eigenvalues.
^ :ccT > ( cT x^ ) 2
1. Y
2. m ax ( x;Y ) 2 OA cT x¡ m i n ( x;Y ) 2 OA cT x is as sm all
as possible
This can be calculated by
solving a linear program
whose right hand side is a
linear function of c
Anureet Saxena, TSoB
27
A Lesson from MILP
W e are searching f or vect ors c w hich sat isf y,
1.
This condition is always satisfied
if c belongs
toavector
space
This
problem
can
be
formulated
as
mixed
T
T
2
^ :cc > ( c x^ )
Y
spanned by eigenvectors of Yinteger linear program!!
xxT associated with positive
eigenvalues.
Univariate Expression
Generating Mixed Integer
Program
T
T
2. m ax ( x;Y ) 2 OA c x¡ m i n ( x;Y ) 2 OA c x is (UGMIP)
as sm all
as possible
This can be calculated by
solving a linear program
whose right hand side is a
linear function of c
Anureet Saxena, TSoB
28
Cutting Plane Algorithm
^)
( x^ ; Y
E xt ract E igenvalues
and E igenvect ors of
^ ¡ x^ x^ T .
Y
¸ < 0
( cT x) 2 · Y:ccT
¸ > 0
Y:ccT · ( cT x) 2
UGMIP
Convex
Quadratic Cut
Derive Disjunction
CGLP
Derive Disjunctive
Cut
Anureet Saxena, TSoB
29
Cutting Plane Algorithm
^)
( x^ ; Y
E xt ract E igenvalues
and E igenvect ors of
^ ¡ x^ x^ T .
Y
¸ < 0
( cT x) 2 · Y:ccT
¸ > 0
Y:ccT · ( cT x) 2
Version 1
UGMIP
Convex
Quadratic Cut
Derive Disjunction
CGLP
Derive Disjunctive
Cut
Anureet Saxena, TSoB
30
Cutting Plane Algorithm
^)
( x^ ; Y
E xt ract E igenvalues
and E igenvect ors of
^ ¡ x^ x^ T .
Y
¸ < 0
( cT x) 2 · Y:ccT
¸ > 0
Y:ccT · ( cT x) 2
Version 2
UGMIP
Convex
Quadratic Cut
Derive Disjunction
CGLP
Derive Disjunctive
Cut
Anureet Saxena, TSoB
31
Cutting Plane Algorithm
^)
( x^ ; Y
E xt ract E igenvalues
and E igenvect ors of
^ ¡ x^ x^ T .
Y
¸ < 0
( cT x) 2 · Y:ccT
¸ > 0
Y:ccT · ( cT x) 2
Version 3
UGMIP
Convex
Quadratic Cut
Derive Disjunction
CGLP
Derive Disjunctive
Cut
Anureet Saxena, TSoB
32
Computational Results
Solvers
• Convex Relaxations – IpOpt
• Eigenvalue Computations – LAPACK
• Linear Programs & Mixed Integer Programs– CPLEX 10.1
• COIN-OR / Bonmin based implementation
Test Bed
• 160 GlobalLIB Instances
• 4 Chemical Process Design instances from Lee & Grossman
• 40 Box QP Instances
Anureet Saxena, TSoB
33
Computational Results
Experiment Setup
• 1 Hour Time limit on each instance
• Initial Relaxation strengthened by RLT inequalities
opt ( F i nal Rel axat i on) ¡ RL T
D ual i t y Gap =
£ 100
Opt ( M I QCP ) ¡ RL T
Anureet Saxena, TSoB
34
GlobalLIB Instances
160 = 129 + 24 + 7
• All MIQCP Instances with
upto 50 variables
Numerical Problems
• x1 x2 x3 x4 x5
Zero Duality Gap
• (x1+x2)/x3 ¸ 2x1
• x0.75
Non-Zero Duality Gap
Anureet Saxena, TSoB
35
GlobalLIB Instances
Summary of % Duality Gap Closed (129 Instances with non-zero Duality Gap)
V1
16
1
10
11
91
V2
23
44
23
22
17
V3
23
52
21
20
13
24.80%
76.49%
80.86%
>99.99 %
98-99.99 %
75-98 %
25-75 %
0-25 %
Average Gap Closed
Anureet Saxena, TSoB
36
GlobalLIB Instances
Summary of % Duality Gap Closed (129 Instances with non-zero Duality Gap)
V1
16
1
10
11
91
V2
23
44
23
22
17
V3
23
52
21
20
13
24.80%
76.49%
80.86%
>99.99 %
98-99.99 %
75-98 %
25-75 %
0-25 %
Average Gap Closed
Anureet Saxena, TSoB
37
Version (2,3) vs Version 1
Instance
st_qpc-m3a
st_ph13
st_ph11
ex3_1_4
st_jcbpaf2
st_ph12
ex2_1_9
prob05
st_glmp_kky
st_e24
st_ph15
st_bsj4
st_ph14
st_e08
st_ht
st_pan2
ex2_1_1
st_fp1
st_pan1
ex5_2_4
st_e02
st_kr
% Duality Gap Closed
V1
V2
V3
0.00%
98.10%
99.16%
0.00%
99.38%
98.80%
0.00%
99.46%
98.19%
0.00%
86.31%
99.57%
0.00%
99.47%
99.61%
0.00%
99.49%
99.62%
0.00%
98.79%
99.73%
0.00%
99.78%
99.49%
0.00%
99.80%
99.71%
0.00%
99.81%
99.81%
0.00%
99.83%
99.81%
0.00%
99.86%
99.80%
0.00%
99.85%
99.86%
0.00%
99.81%
99.89%
0.00%
99.81%
99.89%
0.00%
68.54%
99.91%
0.00%
72.62%
99.92%
0.00%
72.62%
99.92%
0.00%
99.72%
99.92%
0.00%
79.31%
99.92%
0.00%
99.88%
99.95%
0.00%
99.93%
99.95%
Instance
st_e33
st_z
st_qpc-m0
st_phex
st_e26
st_m1
ex2_1_6
st_fp6
st_e07
st_glmp_kk92
st_ph3
st_ph20
st_qpk1
st_bsj2
st_ph2
st_ph1
ex2_1_5
st_fp5
ex3_1_3
st_bpv2
st_qpc-m1
st_qpc-m3b
Anureet Saxena, TSoB
% Duality Gap Closed
V1
V2
V3
0.00%
99.94%
99.95%
0.00%
99.96%
99.95%
0.00%
99.96%
99.96%
0.00%
99.96%
99.96%
0.00%
99.96%
99.96%
0.00%
99.96%
99.96%
0.00%
99.95%
99.97%
0.00%
99.92%
99.97%
0.00%
99.97%
99.97%
0.00%
99.98%
99.98%
0.00%
99.98%
99.98%
0.00%
99.98%
99.98%
0.00%
99.98%
99.98%
0.00%
99.98%
99.96%
0.00%
99.98%
99.98%
0.00%
99.98%
99.98%
0.00%
99.98%
99.99%
0.00%
99.98%
99.99%
0.00%
99.99%
99.99%
0.00%
99.99%
99.99%
0.00%
99.99%
99.98%
0.00% 100.00% 100.00%
38
Version (2,3) vs Version 1
% Duality Gap Closed
% Duality Gap Closed
Instance
V1
V2
V3
Instance
V1
V2
V3
st_e33
0.00%
99.94%
99.95%
st_qpc-m3a
0.00%
98.10%
99.16%
st_z
0.00%
99.96%
99.95%
st_ph13
0.00%
99.38%
98.80%
st_qpc-m0
0.00%
99.96%
99.96%
st_ph11
0.00%
99.46%
98.19%
st_phex
0.00%
99.96%
99.96%
ex3_1_4
0.00%
86.31%
99.57%
st_e26
0.00%
99.96%
99.96%
st_jcbpaf2
0.00%
99.47%
99.61%
st_m1
0.00%
99.96%
99.96%
st_ph12
0.00%
99.49%
99.62%
Either
version
2
or
version
3
closes
>99%
of
the
duality
gap
on
44
ex2_1_6
0.00%
99.95%
99.97%
ex2_1_9
0.00%
98.79%
99.73%
which99.78%
version
1 is unable
gap. 99.92% 99.97%
st_fp6 to close any0.00%
prob05instances on0.00%
99.49%
st_e07
0.00%
99.97%
99.97%
st_glmp_kky
0.00%
99.80%
99.71%
st_glmp_kk92
0.00%
99.98%
99.98%
st_e24
0.00%
99.81%
99.81%
The relaxation
by adding
disjunctive cuts
be 99.98%
st_ph3
0.00% can
99.98%
st_ph15
0.00%obtained
99.83%
99.81%
st_ph20
st_bsj4substantially
0.00%
99.86% than
99.80%the SDP
stronger
relaxation!!0.00% 99.98% 99.98%
st_qpk1
0.00%
99.98%
99.98%
st_ph14
0.00%
99.85%
99.86%
st_bsj2
0.00%
99.98%
99.96%
st_e08
0.00%
99.81%
99.89%
st_ph2
0.00%
99.98%
99.98%
st_ht
0.00%
99.81%
99.89%
st_ph1
0.00%
99.98%
99.98%
st_pan2
0.00%
68.54%
99.91%
ex2_1_5
0.00%
99.98%
99.99%
ex2_1_1
0.00%
72.62%
99.92%
st_fp5
0.00%
99.98%
99.99%
st_fp1
0.00%
72.62%
99.92%
ex3_1_3
0.00%
99.99%
99.99%
st_pan1
0.00%
99.72%
99.92%
st_bpv2
0.00%
99.99%
99.99%
ex5_2_4
0.00%
79.31%
99.92%
st_qpc-m1
0.00%
99.99%
99.98%
st_e02
0.00%
99.88%
99.95%
st_qpc-m3b
0.00% 100.00% 100.00%
st_kr
0.00%
99.93%
99.95%
Observation
Anureet Saxena, TSoB
39
Version 2 vs Version 3
Instance
ex7_3_1
ex9_2_3
ex9_2_7
st_fp7b
st_rv3
ex9_2_1
st_pan2
ex2_1_1
st_fp1
ex5_2_4
ex8_1_7
% Duality Gap Closed
V1
V2
V3
0.00%
0.00% 85.43%
0.00%
0.00% 47.17%
42.31% 51.47% 86.25%
0.00% 22.06% 55.51%
0.00% 40.40% 72.68%
54.54% 60.04% 92.02%
0.00% 68.54% 99.91%
0.00% 72.62% 99.92%
0.00% 72.62% 99.92%
0.00% 79.31% 99.92%
77.43% 77.43% 95.79%
Instance
st_rv7
st_qpk3
st_rv8
st_e20
ex8_1_8
ex5_3_2
ex3_1_4
st_fp7c
st_qpk2
st_rv9
house
ex7_3_2
Anureet Saxena, TSoB
% Duality Gap Closed
V1
V2
V3
0.00% 45.43% 62.28%
0.00% 33.53% 50.04%
0.00% 29.90% 45.80%
0.00% 76.38% 90.88%
0.00% 76.49% 90.88%
0.00%
7.27% 21.00%
0.00% 86.31% 99.57%
0.00% 44.26% 57.10%
0.00% 71.34% 83.33%
0.00% 20.56% 31.64%
0.00% 86.93% 97.92%
0.00% 59.51% 70.26%
40
Version 2 vs Version 3
% Duality Gap Closed
% Duality Gap Closed
Instance
V1
V2
V3
Instance
V1
V2
V3
st_rv7
0.00% 45.43% 62.28%
ex7_3_1
0.00%
0.00% 85.43%
st_qpk3
0.00% 33.53% 50.04%
ex9_2_3
0.00%
0.00% 47.17%
0.00%
45.80%
Version 3 closes
dualityst_rv8
gap than version
2 on29.90%
23
ex9_2_7
42.31% 10%
51.47%more
86.25%
st_e20
0.00% 76.38% 90.88%
st_fp7binstances. 0.00% 22.06% 55.51%
ex8_1_8
0.00% 76.49% 90.88%
st_rv3
0.00% 40.40% 72.68%
ex5_3_2
0.00%
7.27% 21.00%
ex9_2_1
54.54% 60.04% 92.02%
on finding
width86.31%
pays 99.57%
ex3_1_4with small0.00%
st_pan2The effort spent
0.00% 68.54%
99.91%c vectors
st_fp7c
0.00% 44.26% 57.10%
ex2_1_1
0.00% 72.62% 99.92%
off!!
st_qpk2
0.00% 71.34% 83.33%
st_fp1
0.00% 72.62% 99.92%
st_rv9
0.00% 20.56% 31.64%
ex5_2_4
0.00% 79.31% 99.92%
house
0.00% 86.93% 97.92%
ex8_1_7
77.43% 77.43% 95.79%
ex7_3_2
0.00% 59.51% 70.26%
Observation
Anureet Saxena, TSoB
41
Linear Complementarity Disjunctions
• Some problems have linear complementarity constraints
xi xj = 0
• These constraints can be used to derive the linear complementarity
disjunctions
(xi=0) OR (xj=0)
which can be used with the medley of other disjunctions to derive
disjunctive cuts
Anureet Saxena, TSoB
42
Linear Complementarity Disjunctions
Instance
ex9_1_4
ex9_2_1
ex9_2_2
ex9_2_3
ex9_2_4
ex9_2_6
ex9_2_7
Without Using LCD
V2
V3
0.00%
1.55%
60.04%
92.02%
88.29%
98.06%
0.00%
47.17%
99.87%
99.89%
87.93%
62.00%
51.47%
86.25%
Using LCD
V2
100.00%
99.95%
100.00%
99.99%
99.99%
80.22%
99.97%
V3
99.97%
99.95%
100.00%
99.99%
100.00%
92.09%
99.95%
Observation
Linear Complementarity conditions can be exploited effectively
within a disjunctive programming framework to derive strong
cuts
Anureet Saxena, TSoB
43
Recap
Eigenvalue
computation
Disjunctive
Programming
Research Question
What is the marginal value of Disjunctive
Programming in this framework?
Anureet Saxena, TSoB
44
Marginal Value of Disjunctive Programming
p( cT x) + q
Secant
Approximation
Y.ccT· p(cTx) + q
cT x
Anureet Saxena, TSoB
45
Marginal Value of Disjunctive Programming
^)
( x^ ; Y
E xt ract E igenvalues
and E igenvect ors of
^ ¡ x^ x^ T .
Y
¸ < 0
( cT x) 2 · Y:ccT
¸ > 0
Y:ccT · ( cT x) 2
UGMIP
Convex
Quadratic Cut
DeriveApproximation
Disjunction
Secant
CGLP
Derive Disjunctive
Cut
Anureet Saxena, TSoB
46
Marginal Value of Disjunctive Programming
Summary of % Duality Gap Closed (Instances with non-zero Duality Gap)
V1
>99.99 %
98-99.99 %
75-98 %
25-75 %
0-25 %
Average Gap Closed
V2
V3
16
23
23
1
44
52
10
23
21
11
22
20
91
17
13
24.80% 76.49% 80.86%
Anureet Saxena, TSoB
V2-SA V3-SA
24
27
4
6
17
25
26
22
58
49
44.40% 52.56%
47
Marginal Value of Disjunctive Programming
Summary of % Duality Gap Closed (Instances with non-zero Duality Gap)
V1
>99.99 %
98-99.99 %
75-98 %
25-75 %
0-25 %
Average Gap Closed
V2
V3
16
23
23
1
44
52
10
23
21
11
22
20
91
17
13
24.80% 76.49% 80.86%
V2-SA V3-SA
24
27
4
6
17
25
26
22
58
49
44.40% 52.56%
32%
28%
Anureet Saxena, TSoB
48
Summary
xx T ¡ Y ¸ SD P 0
Disjunctive
Programming
Anureet Saxena, TSoB
Y ¡ xx T ¸ SD P 0
49
Optimizing over the Split Closure
Anureet Saxena, TSoB
50
Optimizing over the Split Closure
• E. Balas and A. Saxena, Optimizing over the Split
Closure, Mathematical Programming (Ser. A)., Vol
113 (2) Jan 2008, 219-240.
• A. Saxena, OSCLIB,
www.andrew.cmu.edu/user/anureets/osc/osc.htm
Anureet Saxena, TSoB
51
MIP Model
min cx
Ax ¸ b
xj 2 Z 8 j2N1
Contains
xj ¸ 0 j2N
xj · uj j2N1
N1: set of integer variables
Incumbent
Fractional
Solution
Anureet Saxena, TSoB
52
Cutting Planes
Cutting planes are linear valid inequalities which are used to
strengthen the Linear Programming relaxation of a MIP.
– Dynamic Approximation of the Integer Hull
Anureet Saxena, TSoB
53
Cutting Planes
Cutting planes are linear valid inequalities which are used to
strengthen the Linear Programming relaxation of a MIP.
– Dynamic
Approximation of the Integer Hull
Elementary Closure
Elementary closure of P w.r.t a family
 of cutting planes is defined by
intersecting P with all rank-1 cuts in
.
Eg: CG Closure, Split Closure
Anureet Saxena, TSoB
54
Elementary Closures
How much duality gap can be closed by
optimizing over elementary closures?
L&P Closure
CG Closure
Split Closure
Bonami and Minoux
Fischetti and Lodi
?
Anureet Saxena, TSoB
55
Elementary Closures
How much duality gap can be closed by
optimizing over elementary closures?
L&P Closure
CG Closure
Split Closure
Bonami and Minoux
Fischetti and Lodi
Balas and Saxena
Anureet Saxena, TSoB
56
Split Disjunctions
•
•
•
 2 ZN, 0 2 Z
j = 0, j 2 N2
0 <  < 0 + 1
 x · 0
 x ¸ 0 + 1
Split Disjunction
Anureet Saxena, TSoB
57
Split Cuts
u
u0
Ax ¸ b
 x · 0
Ax ¸ b
 x ¸ 0+1
L x ¸ L
R x ¸ R
x¸
Anureet Saxena, TSoB
v
v0
Split Cut
58
Split Closure
Elementary Split Closure of P = { x | Ax ¸ b } is the
polyhedral set defined by intersecting P with the valid
rank-1 split cuts.
C = { x2 P |  x ¸  8 rank-1 split cuts  x¸ }
Without Recursion
How to optimize over the split closure?
Anureet Saxena, TSoB
59
Algorithmic Framework
Add Cuts
min cx
Ax ¸ b
t x¸ t t2
Solve
Master LP
Integral Sol?
Unbounded?
Infeasible?
Yes
MIP Solved
No
Split Cuts
Generated
Rank-1 Split Cut
Separation
No Split Cuts
Generated
Anureet Saxena, TSoB
Optimum over
Split Closure
attained
60
Algorithmic Framework
Add Cuts
min cx
Ax ¸ b
t x¸ t t2
Solve
Master LP
Integral Sol?
Unbounded?
Infeasible?
Yes
MIP Solved
No
Split Cuts
Generated
Rank-1 Split Cut
Separation
No Split Cuts
Generated
Anureet Saxena, TSoB
Optimum over
Split Closure
attained
61
SC Separation Theorem
Theorem: lies in the split closure of P if and only if the optimal
value of the following parametric mixed integer linear program is
non-negative.
Parameter
Parametric
Mixed Integer
Linear Program
Anureet Saxena, TSoB
62
SC Separation Theorem
Theorem: lies in the split closure of P if and only if the optimal
value of the following parametric mixed integer linear program is
non-negative.
Research Questions
Parameter
1. Parametric MIP vs MIP?
2. Branch-and-Bound for PMILP? Relaxation?
Parametric
Mixed Integer
3. Sparse disjunctions with small coefficients vs Dense
disjunctions with large coefficients
Linear Program
4. Cuts Diversification?
Anureet Saxena, TSoB
63
Deparametrization
Parametric Mixed
Integer Linear
Program
Anureet Saxena, TSoB
64
Deparametrization
Parametric Mixed
Integer Linear
Program
If  is fixed,
then PMILP reduces
to a MILP
Anureet Saxena, TSoB
65
Deparametrization
MILP( )
Deparametrized
Mixed Integer
Linear Program
Maintain a dynamically
updated grid of parameters
Anureet Saxena, TSoB
66
Deparametrization
Cutting Planes Engineering
•
•
Penalize L1 norm of the
disjunction
•
sparse cuts
•
small coefficients
Cut Diversification via a setcovering model
•
•
Anureet Saxena, TSoB
solved heuristically
Re-derive cuts
•
Store disjunction
•
Short Proof of Validity!
67
Separation Algorithm
Initialize Parameter Grid (  )
For  2 ,
Diversification
• Solve MILP() using CPLEX 9.0
• Enumerate  branch and bound nodes
• Store all the separating split disjunctions which
are discovered
Strengthening
At least one
Grid Enrichment
no
split disjunction
yes
STOP
discovered?
Bifurcation
Anureet Saxena, TSoB
68
Implementation Details
Processor Details
• Pentium IV
• 2Ghz, 2GB RAM
COIN-OR
CPLEX 9.0
Core Implementation
• Solving Master LP
• Setting up MILP
• Disjunctions/Cuts Management
• L&P cut generation+strengthening
Anureet Saxena, TSoB
Solving MILP(  )
69
Computational Results
Termination Criterion
• Integral Solution Found
• No Cut in 3 hours
• Time-Limit Reached
• 4 days for each MIPLIB problem
Anureet Saxena, TSoB
70
MIPLIB 3.0 MIP Instances
Summary of MIP Instances (MIPLIB 3.0)
Total Number of Instances: 41
Number of Instances included: 40
No duality gap: noswot, dsbmip
Instance not included: rentacar
Results
98-100% Gap closed in 15 instances
75-98% Gap closed in 10 instances
25-75% Gap closed in 7 instances
0-25% Gap closed in 6 instances
Average Gap Closed: 72.78%
Anureet Saxena, TSoB
71
MIPLIB 3.0 Pure IP Instances
Summary of Pure IP Instances (MIPLIB 3.0)
Total Number of Instances: 24
Number of Instances included: 24
No duality gap: enigma
Results
98-100% Gap closed in 9 instances
75-98% Gap closed in 4 instances
25-75% Gap closed in 6 instances
0-25% Gap closed in 4 instances
Average Gap Closed: 72.47%
Anureet Saxena, TSoB
72
arki001
• MIPLIB 3.0 & 2003 instance
• Metallurgical Industry
Problem Stats
• Unsolved for the past 10 years [1995-2000-2005]
1048 Rows
1388 Columns
123 Gen Integer Vars
415 Binary Vars
850 Continuous Vars
Anureet Saxena, TSoB
73
Solution Strategy
Original
Problem
CPLEX 9.0
Presolver
Preprocessed
Problem
CPLEX 9.0
Emphasis on optimality
Strong Branching
Rank-1 Split Cut
Generation
Anureet Saxena, TSoB
Strengthened
Formulation
74
Strengthening + CPLEX 9.0
Solved to optimality
Crossover Point
(227 rank-1 cuts)
Anureet Saxena, TSoB
75
Strengthening + CPLEX 9.0
Solved to optimality
arki001 Solution Statistics
% Gap closed by rank-1 split cuts: 83.05%
Time spent in generating rank-1 split cuts: 53.76 hrs
Number of rounds of (rank-1 split) cuts generated: 92
Time taken by CPLEX 9.0 after strengthening: 10.94 hrs
No. of branch-and-bound nodes enumerated by CPLEX: 643425
Total time taken to solve the instance to optimality: 64.70 hrs
Anureet Saxena, TSoB
76
CPLEX 9.0
After 100 hours:
43 million B&B nodes
22 million active nodes
12GB B&B Tree
Anureet Saxena, TSoB
77
Comparison
Crossover
Point
Anureet Saxena, TSoB
78
OSCLIB
Split
Closure
{-1,0,1} Split
Disjunctions
{0,1} Split
Disjunctions
Elementary
0-1 Split
Disjunctions
Time
Limit
(sec)
Category
Source
No.
Instances
Capacitated Warehouse
Location Problem (Set 1)
OrLib
37
100.00%
99.95%
99.94%
98.32%
36000
Capacitated Warehouse
Location Problem (Set 2)
OrLib
12
94.54%
94.52%
94.51%
94.33%
36000
Capacitated p-Median
Problem
OrLib
20
99.92%
99.91%
99.91%
98.47%
18000
Single Source Capacitated
Facility Location Problem
Holmberg
71
98.56%
94.51%
92.73%
88.73%
18000
Fixed Charge Network Flow
Problem
BCOL
20
94.58%
31.36%
29.44%
2.35%
18000
Multi-Commodity
Capacitated Network Design
Problem (Splittable Version)
BCOL
15
78.24%
77.51%
69.97%
46.16%
86400
Multi-Commodity
Capacitated Network Design
Problem (Unsplittable
Version)
BCOL
20
67.40%
38.98%
20.41%
15.92%
86400
Capacitated Lot Sizing
Problem
BCOL
100
79.37%
78.49%
78.48%
26.46%
18000
Anureet Saxena, TSoB
79
OSCLIB
Split
Closure
{-1,0,1} Split
Disjunctions
{0,1} Split
Disjunctions
Elementary
0-1 Split
Disjunctions
Time
Limit
(sec)
Category
Source
No.
Instances
Capacitated Warehouse
Location Problem (Set 1)
OrLib
37
100.00%
99.95%
99.94%
98.32%
36000
Capacitated Warehouse
Location Problem (Set 2)
OrLib
12
94.54%
94.52%
94.51%
94.33%
36000
Capacitated p-Median
Problem
OrLib
20
99.92%
99.91%
99.91%
98.47%
18000
Single Source Capacitated
Facility Location Problem
Holmberg
71
98.56%
94.51%
92.73%
88.73%
18000
Fixed Charge Network Flow
Problem
BCOL
20
94.58%
31.36%
29.44%
2.35%
18000
Multi-Commodity
Capacitated Network Design
Problem (Splittable Version)
BCOL
15
78.24%
77.51%
69.97%
46.16%
86400
Multi-Commodity
Capacitated Network Design
Problem (Unsplittable
Version)
BCOL
20
67.40%
38.98%
20.41%
15.92%
86400
Capacitated Lot Sizing
Problem
BCOL
100
79.37%
78.49%
78.48%
26.46%
18000
Anureet Saxena, TSoB
80
OSCLIB
Category
Source
No.
Instances
Capacitated Warehouse
Location Problem (Set 1)
OrLib
37
Capacitated Warehouse
Location Problem (Set 2)
Capacitated p-Median
Problem
Split
Closure
{-1,0,1} Split
Disjunctions
{0,1} Split
Disjunctions
Elementary
0-1 Split
Disjunctions
Time
Limit
(sec)
99.94%
98.32%
36000
100.00%
99.95%
Data
Mining
A simple
designed94.52%
to sieve through
the94.33%
OrLib experiment
12
94.54%
94.51%
cuts and disjunctions generated during the
OrLib
20
99.92%
99.91%
99.91%
98.47%
experiment.
Single Source Capacitated
Facility Location Problem
Holmberg
98.56%
94.51%
18000
92.73%
88.73%
18000
•
BCOL
20
Sparse
Disjunctions
–94.58%
support31.36%
of at most29.44%
10
2.35%
18000
•
SmallBCOL
disjunction
coefficients
–77.51%
{0,1,-1}
15
78.24%
46.16%
86400
•
Cuts had nice coefficients
•
Very BCOL
few distinct
–38.98%
combinatorial
20 coefficients
67.40%
20.41% origin?
15.92%
86400
•
Some well-known facets identified
Main Findings
71
36000
Fixed Charge Network Flow
Problem
Multi-Commodity
Capacitated Network Design
Problem (Splittable Version)
69.97%
Multi-Commodity
Capacitated Network Design
Problem (Unsplittable
Version)
Capacitated Lot Sizing
Problem
BCOL
100
79.37%
78.49%
Anureet Saxena, TSoB
78.48%
26.46%
18000
81
OSCLIB
Split
Closure
{-1,0,1} Split
Disjunctions
{0,1} Split
Disjunctions
Elementary
0-1 Split
Disjunctions
Time
Limit
(sec)
Category
Source
No.
Instances
Capacitated Warehouse
Location Problem (Set 1)
OrLib
37
100.00%
99.95%
99.94%
98.32%
36000
Capacitated Warehouse
Location Problem (Set 2)
OrLib
12
94.54%
94.52%
94.51%
94.33%
36000
Capacitated p-Median
Problem
OrLib
20
99.92%
99.91%
99.91%
98.47%
18000
Single Source Capacitated
Facility Location Problem
Holmberg
71
98.56%
94.51%
92.73%
88.73%
18000
Fixed Charge Network Flow
Problem
BCOL
20
94.58%
31.36%
29.44%
2.35%
18000
Multi-Commodity
Capacitated Network Design
Problem (Splittable Version)
BCOL
15
78.24%
77.51%
69.97%
46.16%
86400
Multi-Commodity
Capacitated Network Design
Problem (Unsplittable
Version)
BCOL
20
67.40%
38.98%
20.41%
15.92%
86400
Capacitated Lot Sizing
Problem
BCOL
100
79.37%
78.49%
78.48%
26.46%
18000
Anureet Saxena, TSoB
82
OSCLIB
Split
Closure
{-1,0,1} Split
Disjunctions
{0,1} Split
Disjunctions
Elementary
0-1 Split
Disjunctions
Time
Limit
(sec)
Category
Source
No.
Instances
Capacitated Warehouse
Location Problem (Set 1)
OrLib
37
100.00%
99.95%
99.94%
98.32%
36000
Capacitated Warehouse
Location Problem (Set 2)
OrLib
12
94.54%
94.52%
94.51%
94.33%
36000
Capacitated p-Median
Problem
OrLib
20
99.92%
99.91%
99.91%
98.47%
18000
Single Source Capacitated
Facility Location Problem
Holmberg
71
98.56%
94.51%
92.73%
88.73%
18000
Fixed Charge Network Flow
Problem
BCOL
20
94.58%
31.36%
29.44%
2.35%
18000
Multi-Commodity
Capacitated Network Design
Problem (Splittable Version)
BCOL
15
78.24%
77.51%
69.97%
46.16%
86400
Multi-Commodity
Capacitated Network Design
Problem (Unsplittable
Version)
BCOL
20
67.40%
38.98%
20.41%
15.92%
86400
Capacitated Lot Sizing
Problem
BCOL
100
79.37%
78.49%
78.48%
26.46%
18000
Anureet Saxena, TSoB
83
Summary
Discover
Separation Oracle for
the split closure
Practical implementation of
the separation oracle
Cuts’ Diversification via Set
Covering Model
Demonstrate
MIPLIB Instances
• Pure 72.47%
• Mixed 72.78%
Diversify
Structured MIPs
• CWLP
94-100%
• p-Median 99.92%
• SSCFLP 98.56%
Difficult Instances
• arki001
• FCNF
94.58%
• Lot Sizing 79.37%
• Multi-Commodity
• Splittable 78.24%
• Unsplittable 67.40%
{0,1,-1} Split Closure
Anureet Saxena, TSoB
84
Probabilistic Set Covering Problem
Anureet Saxena, TSoB
85
Probabilistic Set Covering Problem
•
A. Saxena, V. Goyal and M. Lejeune: MIP Reformulations of the
Probabilistic Set Covering Problem, To Appear in Mathematical
Programming.
•
A. Saxena, A Short Note on the Probabilistic Set Covering Problem.
•
A. Saxena, MIP Reformulations of the Probabilistic Set Covering
Problem (II).
Anureet Saxena, TSoB
86
Probabilistic Set Covering
Random 0/1 Vector
(Joint Distribution)
Reliability
Level
Probabilistic
Deterministic
Anureet Saxena, TSoB
87
A Simple Algorithm
Random 0/1 Vector
(Joint Distribution)
1.
2.
Reliability
Level
Enumerate all possible 0/1 realizations of .
For each 0/1 realization whose cdf is greater than or equal
to p, solve the deterministic problem
Anureet Saxena, TSoB
88
Prekopa, Beraldi, Ruszczynski Approach
Anureet Saxena, TSoB
89
Prekopa, Beraldi, Ruszczynski Approach
111
110
101
011
100
010
001
000
Anureet Saxena, TSoB
90
Prekopa, Beraldi, Ruszczynski Approach
p-efficient frontier
Anureet Saxena, TSoB
91
2-Phase Algorithm
Enumeration of p-efficient points
Solve a Deterministic Problem for each
p-efficient point
Anureet Saxena, TSoB
92
2-Phase Algorithm
Enumeration of p-efficient points
Independent
Solve a Deterministic Problem for each
p-efficient point
Anureet Saxena, TSoB
93
Beraldi & Ruszczynski Approach
Explosive Growth
In computation
time
scp41
scp42
Anureet Saxena, TSoB
94
2-Phase Algorithm
Pitfall
Enumeration of p-efficient points
Solving a Deterministic Problem for each
p-efficient point
Anureet Saxena, TSoB
95
Our Approach
Integrate the 2phases
Enumeration of p-efficient points
Solving a Deterministic Problem for each
p-efficient point
Anureet Saxena, TSoB
96
Our Approach
Integrate the 2phases
Enumeration of p-efficient points
Independent
Solving a Deterministic Problem for each
p-efficient point
Anureet Saxena, TSoB
97
Our Model
Log of cumulative
probability of block t
MIPing
Non-Linear
Anureet Saxena, TSoB
98
Our Model
Log of cumulative
probability of block t
Anureet Saxena, TSoB
99
Our Model
Log of cumulative
probability of block t
Anureet Saxena, TSoB
100
Beraldi & Ruszczynski Approach:
Comparison
All instances solved in
less than 1sec by
CPLEX 9.0. CPLEX
enumerated less than
50 nodes solving
most instances at the
root node
scp41
scp42
Anureet Saxena, TSoB
101
Key Observations
•
•
•
•
Models any arbitrary distribution
Exponential number of constraints for each block
Linear in the input size for generic distribution
Encodes the enumeration phase as a Mixed Integer
Program
• Allows us to exploit state-of-art MIP solvers to perform
intelligent enumeration.
Anureet Saxena, TSoB
102
Key Observations
•
•
•
•
Models any arbitrary distribution
Exponential number of constraints for each block
Linear in the input
size for generic
distribution
Research
Question
Encodes the enumeration phase as a Mixed Integer
ProgramThe model has an exponential number of
constraints for each block. Is there a way
• Allows us
exploit
MIP solvers to perform
to to
reduce
thestate-of-art
number of constraints?
intelligent enumeration.
Eg: 50 blocks of size 10 – 50,000 constraints
Anureet Saxena, TSoB
103
The Answer is Yes
Anureet Saxena, TSoB
104
p-Inefficient Frontier
Anureet Saxena, TSoB
105
Refined Formulation
Add t constraints
only for lattice
points above the
frontier
Set-Covering
Constraint for
maximally pinefficient points
Anureet Saxena, TSoB
106
Refined Formulation
Block Size10
Anureet Saxena, TSoB
107
Summary
Discover
MIP Reformulation
of PSC
Demonstrate
BR
Hours
SGL
Seconds
Diversify
Set Covering
CWLP
p-Inefficient Frontier
p-Median
SSCFLP
Anureet Saxena, TSoB
108
Probabilistic SSCFLP
Transportation Cost
Fixed Cost
Demand Constraints
Capacity Constraints
Set of Customers
Set of Facilities
Anureet Saxena, TSoB
109
Probabilistic SSCFLP
Anureet Saxena, TSoB
110
Probabilistic SSCFLP
Holmberg Test Bed Instance – p31
• 30 Facilities & 150 Customers
• Can be solved by CPLEX 9.0 in 80 sec
Probabilistic SSCFLP p31
• 15 Blocks of size 10 each, p=0.8
• CPLEX 9.0 takes 33 hours to solve the instance to optimality and
enumerates 1.7 million B&B nodes
Anureet Saxena, TSoB
111
Probabilistic SSCFLP
Holmberg Test Bed Instance – p31
• 30 Facilities & 150 Customers
• Can be solved by CPLEX 9.0 in 80 sec
Research Question
Why is this instance so difficult to solve?
Probabilistic SSCFLP p31
• 15 Blocks of size 10 each
• CPLEX 9.0 takes 33 hours to solve the instance to optimality and
enumerates 1.7 million B&B nodes
Anureet Saxena, TSoB
112
Answer
Big-M Constraints
Anureet Saxena, TSoB
113
Polarity Cuts
Facets of P can
strengthen the model
Big-M Constraints
model P
Anureet Saxena, TSoB
114
Polarity Cuts
• We
know all the extreme points and extreme rays of P
• Compact description of polar
• Facets of P can be found by solving the linear program
derived from the polar.
Anureet Saxena, TSoB
115
Polarity Cuts Separation
Point to be cut off
Anureet Saxena, TSoB
116
Polarity Cuts Separation
Point to be cut off
Research Questions
1. Unbounded Linear Program?
2. Can we fix signs of the cut coefficients?
3. How to solve the resulting LP?
4. Can we improve the performance of the LP solver?
5. Can we guide the solver to generate sparse cuts?
Anureet Saxena, TSoB
117
Cutting Planes Engineering
L1 norm penalty
Normalization
Polyhedral
Analysis
• More rows than columns – Dual Simplex Algorithm
• Solve the separation problem for the following set
instead of for P
Anureet Saxena, TSoB
118
Polarity Cuts Separation
Anureet Saxena, TSoB
119
A Tough Instance - p31
Tough Instance Solved
• % Gap closed at Root Node 67.84%
• Time Spent in Strengthening 0.83 sec
1.7 million
• Time Spent in Solving Separation LP 0.30 sec
• Time Taken by CPLEX 9.0 after Strengthening 51.65 sec
• No. of Branch-and-Bound enumerated by CPLEX 9.0 2300
• Total time taken to solve the instance to optimality 53.04 sec
33 hours
Anureet Saxena, TSoB
120
Comparison
Anureet Saxena, TSoB
121
Polarity Cuts
Polarity
Enumeration
Separation Algorithm
Dual Simplex
Algorithm
Anureet Saxena, TSoB
122
Summary
Discover
MIP Reformulation
of PSC
p-Inefficient Frontier
Polarity Cuts
Demonstrate
BR
Hours
SGL
Seconds
Diversify
Set Covering
CWLP
Impact of
Polarity Cuts
p-Median
Fundamental Thm
Independent Dist.
SSCFLP
Balanced Matrices
Circular 1s property
Hierarchy of Relaxations
Anureet Saxena, TSoB
123
Our Contribution
Linear Programming
Spectral Decomposition
Outer Approximation
Disjunctive Programming
CGLP
Normalization
Cuts’ Diversification
Polarity
Cut Rank
CPLEX
XPRESS
LAPACK
COIN-OR
Bonmin
Integer Programming, a Technology
Split
Closure
Probabilistic
Set Covering
Set Covering
Polytope
Anureet Saxena, TSoB
Disjunctive
Programming
125
Thank you
Anureet Saxena, TSoB
126