Efficient Solutions for
2-Variables-per-Constraint
Integer Programming Problems
Reuven Bar-Yehuda &
Dror Rawitz
1999
Efficient Solution for 2VIP Problems
1
Heirarchy of Relevant Problem
Types
Vertex Cover (VC)
2-Satisfiability (2SAT)
Minimum Weight 2SAT
2-Variable-per-constraint Integer Programming
(2VIP)
Efficient Solution for 2VIP Problems
2
Vertex Cover
Input:
An undirected graph G(V,E)
Solution: A subset S  V which includes a vertex in
each arc in E
“What about
optimality?” Just wait…
Efficient Solution for 2VIP Problems
3
2-Satisfiability
Input:
A boolean formula in 2-CNF form,
with m constraints and n variables
(x 1  x 2 )  (x 1  x 3 )  (x 1  x 4 )  (x 4  x 5 )  (x 5  x 6 )
Solution:
A vector of n values for the n variables, so
that they satisfy all constraints
(FALSE, TRUE, FALSE, TRUE, FALSE, FALSE)
Efficient Solution for 2VIP Problems
4
Vertex Cover as a Type of 2SAT
Vertices
Boolean Variables
i
x i, x j, x k
k
j
Selected Vertices
Variables Assigned TRUE
xi = TRUE
i
Edges
Simple OR constraints
i
( xi  x j )
j
Efficient Solution for 2VIP Problems
5
Feasibility Algorithm for 2SAT

SAT (the general satisfiability problem) is NP-Complete.

3SAT (satisfiability of 3-CNF formulae) is NP-Complete.
…
But 2SAT is in P.
This difference is the basis
for our entire approach to the
approximation of 2VIP
Efficient Solution for 2VIP Problems
6
Feasibility Algorithm for 2SAT (continued)
2SAT-FEASIBILITY(F)
If all the variables are set, return SUCCESS
Select an unset variable xk
F1  F with the implications of xk=FALSE propagated
F2  F with the implications of xk=TRUE propagated
If both F1 & F2 cannot be satistfied return FAILURE
F3  the first of F1,F2 for which the propagation is completed without its being
determined as a FALSE formula
return 2SAT-FEASIBILITY(F3)
… 99% Pure Greed!
Efficient Solution for 2VIP Problems
7
Feasibility Algorithm for 2SAT (correctness)
Why does this work?
Observe an assignment effect propagation:
(x 1  x 2 )  (x 1  x 3 )  (x 1  x 4 )  (x 4  x 5 )  (x 5  x 6 )
x1 = FALSE
(F  x2 )  (T  x3 )  (T  x4 )  ( x4  x5 )  ( x5  x6 )
x2 = TRUE
(F  T )  (T  x3 )  (T  x4 )  ( x4  x5 )  ( x5  x6 )
Efficient Solution for 2VIP Problems
8
Feasibility Algorithm for 2SAT (correctness)
After the propagation is complete, we have:
one side of OR set FALSE,
other side set TRUE
no changes to either
side
one side of OR set TRUE,
other side unchanged
(F  T )  (T  x3 )  (T  x4 )  ( x4  x5 )  ( x5  x6 )
satisfied constraints
unaffected constraints
Efficient Solution for 2VIP Problems
9
Feasibility Algorithm for 2SAT (correctness)
But… every x1=TRUE feasible solution must also
satisfy these
constraints!
(F  T )  (T  x3 )  (T  x4 )  ( x4  x5 )  ( x5  x6 )
satisfied constraints
Unaffected Constraints
So… combine the values set here
with the values for this part
‘borrowed’ from an x1=TRUE feasible solution to create
an x1=FALSE feasible solution.
Efficient Solution for 2VIP Problems
10
Feasibility Algorithm for 2SAT (complexity)
Time Complexity - seemingly:
O(n) recursive calls  O(m) constraits checked in the
propagation process = O(mn) time
But, actually:
Each variable is set at most once
 Each constraint is checked at most twice
 Running the propagations in parallel binds the amount
of ‘wasted’ time to the amount of time used to propagate
adopted assignments.

Hence O(2·(m+n))= O(m+n) time in total.
Efficient Solution for 2VIP Problems
11
2SAT as an extension of Vertex Cover
In Vertex Cover, we only have constraints of a
single type:
Edges
Simple OR constraints
i
( xi  x j )
j
But there are 2 more types of constraints in 2SAT:
???
OR constraints with 1 NOT
i
??
j
???
( xi  x j )
( xi  x j )
OR constraints with 2 NOTs
i
??
j
Efficient Solution for 2VIP Problems
( xi  x j )
12
Extending 2SAT: Minimum Weight
(MIN-2SAT) Given a weight vector w, find a vector x
n
which minimizes
w x
i 1
ak  xi  bk  x j  ck
xi  0,1
i
i
, subject to:
k 1,........, m
i 1,........, n
2SAT: ( xi  x j )
generalization
ak  xi  bk  x j  ck
a, b  0
2SAT: ( xi  x j )
generalization
ak  xi  bk  x j  ck
a  0, b  0
2SAT: ( xi  x j )
generalization
ak  xi  bk  x j  ck
a, b  0
Efficient Solution for 2VIP Problems
13
Extending MIN-2SAT: Any Integer Value
(2VIP) Given a weight vector w, find a vector x which
n
minimizes
w x
i 1
i
i
, subject to:
ak  xi  bk  x j  ck
li  xi  ui
xi  N
9
8
7
6
5
4
3
2
1
0
k 1,........, m
i 1,........, n
i 1,........, n
9
8
7
6
5
4
3
2
1
0
Efficient Solution for 2VIP Problems
14
Feasible Polyhedron Convexity
The polyhedron bounding the feasible solutions of an IP
problem is convex.
Specifically, if the points (a,b),(c,d),(e,f),(g,h),(i,j) satisfy a
constraint involving variables i and j …
(e,f)
(a,b)
(g,h)
(c,d)
(i,j)
…so do all the points on the line and within the triangle.
Efficient Solution for 2VIP Problems
15
Extending the 2SAT Feasibility Algorithm:
Partitioning the Solution Space
2SAT:
unset variable xi
xi = FALSE
2VIP:
xi = TRUE
xi  li , ui  , l i  u i
  li  ui  
xi  li , 

2



  li  ui 

xi   
 1, ui 

 2 

Efficient Solution for 2VIP Problems
16
Extending the 2SAT Feasibility Algorithm:
Propagating Bound Effects
(This illustration courtesy of Dror Rawitz )
Efficient Solution for 2VIP Problems
17
Result: Feasibility Algorithm for 2VIP
2VIP-FEASIBILITY(P)
If l = u then
If l is a feasible solution, return SOLUBALE
The solution is infeasible; return INSOLUBLE
Select a variable xk for which lkuk
P1  P with the implications of x k   l k ,  l k  u k   propagated


 l u
2


k 
P2  P with the implications of x k   k
 1, u k  propagated

2



If both P1 & P2 are directly unsatisfiable return INSOLUBLE
P3  the first of P1,P2 for which the propagation is completed without its being
determined as directly unsatisfiable
return 2VIP-FEASIBILITY(P3)
Efficient Solution for 2VIP Problems
18
Feasibility Algorithm for 2VIP (correctness)
Q: Can choosing one ‘half’ of the solution space over the
other ‘half’ rule out feasible solutions?
A: Of course.
Q: Can choosing one ‘half’ of the solution space over the
other ‘half’ rule out ALL feasible solutions?
A: No! We can still ‘borrow’ parts of any feasible solution –
y – for the discarded ‘half’, creating a feasible solution – y’
– for the selected ‘half’:
y k  lk
 lk

k  n , y ' k   y k l k  y k  u k
…
u
uk  y k
 k
Efficient Solution for 2VIP Problems
19
Feasibility Algorithm for 2VIP (correctness)
9
8
7
6
5
4
3
2
1
0
k
bounds for
y – solution for
 l  u k
x k   k
 2
y’ – solution for



1,
u
k



Efficient Solution for 2VIP Problems
  l  u k 
x k  l k ,  k

  2 
 l  u k 

x k   k

1,
u
k


 2 

20
Feasibility Algorithm for 2VIP (correctness)
Why is y’ a valid solution?
Examine any one of the m constraint (say, constraint k),
involving variables i and j. One of the following holds:
1.
Both y’ variables are equal to the y variables.
2.
One of the y’ variables differs from its y counterpart.
3.
Both y’ variables differ from their y counterparts.
We will see how in all 3 cases, (y’i,y’j) satisfy the
constraint.
Efficient Solution for 2VIP Problems
21
Feasibility Algorithm for 2VIP (correctness)
yi  y'i  y j  y ' j
lower bound for y’i
Case 1:
(yi ,yj) = (y’I ,y’j)
lower bound for y’j
no changes to the y variables; since the y vector is a
feasible solution, constraint k is satisfied by (y’i,y’j) .
Efficient Solution for 2VIP Problems
22
Feasibility Algorithm for 2VIP (correctness)
( yi  y 'i  y j  y ' j ) 
( yi  y 'i  y j  y ' j )
we are demonstrating
correctness for
yi  y i  y j  y j
'
'
lower bound for y’i
Case 2:
(yi ,vj)
(yi ,lj)
(yi ,yj)
lower bound for y’j
since yi is within the bounds for variable i, there must exist a
value vi such that (yi,vj) satisfy the constraint; we can now
use the convexity to draw the line between (y’i,y’j) and (yi,vj)
within the feasible polygon… the line crosses (yi,lj)=(y’i,y’j) .
Efficient Solution for 2VIP Problems
23
Feasibility Algorithm for 2VIP (correctness)
( yi  y 'i  y j  y ' j )
we are demonstrating
correctness for
yi  y'i  y j  y ' j
lower bound for y’i
Case 3:
(li ,vj)
(li ,lj)
(yI ,yj)
(vi ,lj)
lower bound for y’j
we again use the definition of the lower and uppoer bounds:
there exist vi and vj so that (li,vj) and (vi,lj) satisfy the
constraint. Together with (yi,yj) , these points form a feasible
triangle within which we find (li, lj)= (y’i, y’j) .
Efficient Solution for 2VIP Problems
24
Feasibility Algorithm for 2VIP (complexity)
(Remembering 2SAT…)
U  max u i  l i 
i
Each lower/upper bound is set at most U times
 for every lower/upped bound update, at most m
constraints are checked.
 Running the constraint propagations in parallel binds
the amount of ‘wasted’ time to the amount of time used to
propagate adopted lower/upper bound changes.

Hence O(mU) time in total.
Efficient Solution for 2VIP Problems
25
Extending 2VIP Yet Again…
Q: How can we ensure we aren’t discarding all ‘good’
solutions by searching only one part of the solution
space?
A: We must extend the problem again, slightly, so that we
can ‘cut our losses’ when betting on one part of the
solution space.
(E2VIP) Instead of minimizing

n
i 1
wi  xi , we minimize the
 0,
xi  lˆi 


n
n
ˆ
ˆ
ˆ
ˆ
expression W ( x, l , uˆ )   i 1 wi  ( xi , li , uˆi )  i 1 wi   xi  li , li  xi  uˆi 


 uˆi  lˆi ,
uˆi  xi 


n
ˆ
ˆ
for some l , uˆ  Q , satisfying
l  l  uˆ  u .
Efficient Solution for 2VIP Problems
…
26
Decomposition of ‘Weights’
The W function is, in essence, the weight of the part of a
vector x which intersects with the (non-integral) range
(lˆ, uˆ ) (i.e. the weight of the part x of between lˆ and û ).
Now we can use a partitioning
of the solution space with
some vector m̂ to achieve a
sort-of-a-decomposition of
weights, without changing the
actual weight vector w:
W ( x, lˆ, uˆ ) 
W ( x, lˆ, mˆ )  W ( x, mˆ , uˆ )
û
m̂
m̂
lˆ
Efficient Solution for 2VIP Problems
27
Applying the Local Ratio Principle
We can now apply the Local Ratio theorem:
If, for given lˆ, uˆ we can find an m̂ so that every feasible
solution, in both parts of the solution space (i.e. in the
  lk  uk  
solution space with the effects of either xk  lk , 
or


  2 
  lk  uk 

xk   
 1, uk  propagated) is an r-approximation w.r.t.

 2 

(lˆ, mˆ ) then every feasible solution that is an r-approximation
ˆ , uˆ ) is also an r-approximation w.r.t. (lˆ, uˆ ).
w.r.t. (m
We are left with the task of finding a useful decomposition
of (lˆ, uˆ ) , i.e. a useful m̂ vector.
Efficient Solution for 2VIP Problems
28
Choosing a Useful ‘Weight’ Decomposition
What effects the Choice of
m̂?
We have partitioned the solution space, and will only be
searching (for example) in the bounds effected by setting
  l  uk  
xk  lk ,  k
(let’s call these bounds lleft , uleft ).


  2 
any feasible solution in
ˆ  lleft
Intuitive choice: m
lleft , uleft would be a
1-approximation with
ˆ , but…
respect to lˆ, m
Efficient Solution for 2VIP Problems
29
Choosing a Useful ‘Weight’ Decomposition
Problem: although we are sure there are feasible solutions
in (lleft , uleft ) – the only thing we know about their optimality
is that the cost of lleft in (lˆ, uˆ ) is lower than that of lright
(that’s why we would choose (lleft , uleft ) over (lright , uright ) ).
Conclusion: the selection of m̂ must take into account the
possibility that the optimal solution is in the discarded
part of the solution space: (lright , uright ).
We must make every solution in (lright , uright )‘pay’ enough in
(lˆ, mˆ ) so that it would still be worthwhile to only search for
solutions in (lleft , uleft ) .
Efficient Solution for 2VIP Problems
30
Choosing a Useful ‘Weight’ Decomposition
Solution:
ˆ  lleft . Now increase the values of m̂ in
Start with m
variables for which
l   l  … until l
left k
right k
right
‘pays’ in
(lˆ, mˆ ) as much as lleft ‘pays’ in (lˆ, uˆ ):
lright
lleft
mˆ  lleft
mˆ  lleft
Efficient Solution for 2VIP Problems
 
31
Choosing a Useful ‘Weight’ Decomposition
What have we achieved?
Loss:
We have added as much as
W (lleft , lˆ, uˆ ) to m̂ , which means
the minimum feasible solution
ˆ , uˆ ) is 2 times more
in (m
expensive then lleft (which is
cheaper-or-equal to any
feasible solution in (lleft , uleft ) ).
Remember Dr. Bar-Yehuda’s ‘pay 2
dollars for 1’ rule-of-thumb!
Gain:
Every solution in (lright , uright )
ˆ ) at
now ‘pays’ in the range (lˆ, m
least W (lleft , lˆ, uˆ ) , so it can’t be
more than 2 times cheaper,
ˆ ) , than the feasible
w.r.t (lˆ, m
ˆ , uˆ ) .
solution we shall find in (m
Thus any feasible solution in (m
ˆ , uˆ ) is a 2-approximation of
the optimum solution w.r.t the range (lˆ, m
ˆ ).
Efficient Solution for 2VIP Problems
32
The Approximation Algorithm for 2VIP
E2VIP-APPROX (l , u , lˆ, uˆ )
If lˆ  l then lˆ  l ; If u  uˆ then û  l
If l = u then:
if l is a feasible solution, return l
if l isn’t a feasible solution, return FAILURE
Select a variable xk for which lkuk
lleft,uleft  l,u with the implications of x k   l k ,  l k  u k   propagated
 2 



lright,uright  l,u with the implications of x k    l k  u k   1, u k  propagated
 2 




If both lleft,uleft and lright,uright are directly unsatisfiable return FAILURE
…
Efficient Solution for 2VIP Problems
33
The Approximation Algorithm for 2VIP
…
If lleft,uleft is directly unsatisfible, return E2VIP-APPROX (lright , uright , lˆ, uˆ )
If lright,uright is directly unsatisfible, return E2VIP-APPROX (l , u , lˆ, uˆ )
left
left
If W (lleft , lˆ, uˆ )  W (lright , lˆ, uˆ ) then
better  left , worse  right
else better  right , worse  left


Find m̂ so that lˆ  m
ˆ  max lworse , lˆ and W (lworse , lˆ, m
ˆ )  W (lbest , lˆ, uˆ)
If m̂ < lbest then m̂ lbest
Return E2VIP-APPROX (lbest , ubest , m
ˆ , uˆ )
Note that calling E2VIP-APPROX(l,u,l,u) means
approximating our original 2VIP problem.
Efficient Solution for 2VIP Problems
34
Approximation Algorithm for 2VIP (correctness)
At every level of recursion, we ‘cut out’ a piece of the
solution space, and with it – a part of the cost of the
solution.
Since we ensure that any feasible solution in the reduced
ˆ , uˆ ) ) is a 2-approximation in the cut-out range
range (i.e. (m
ˆ ) ), we end up ‘cutting out’ the entire solution
(i.e. (lˆ, m
space, so that we are left with a single, trivial, feasible
solution for lfinal=ufinal , and to it we add back the entire
solution space in the form of the ‘cut-outs’, while
maintaining the 2-effectiveness.
Thus we end up with a solution which is a 2-approximation
for the original problem.
Efficient Solution for 2VIP Problems
35
Approximation Algorithm for 2VIP (complexity)
Unlike 2VIP-Feasibility, we can waste a lot of time
propagating bound change effects in the discarded part of
the solution space. Thus every single execution of the
algorithm (not including the recursive call) takes O(mU)
time.

If we choose the xk’s in successive order (from x1 to xn
repeatedly), we will have a series of ‘runs’ over all n
variables, with each i’th run cutting the range (lk,uk) by half
and taking O(nmU/2i) time.
 The number of runs is bounded by log(U).
log U nmU
Hence  i 0
= O(nmU) time in total.
i
2

Efficient Solution for 2VIP Problems
36
Generalizations
We did not make use of the linearity of the constraints;
we only used the covexity of the feasible polyhedron. We
can therefore generalize the problem to any ‘axis-convex’
constraints.

We did not make use of the linearity of the cost function.
We can generalize the problem to (more or less) any nonnegative monotone weight function.

Efficient Solution for 2VIP Problems
37
The End.
Efficient Solution for 2VIP Problems
38