Instructor
Neelima Gupta
ngupta@cs.du.ac.in
Table of Contents
 Lp –rounding
 Dual Fitting
 LP-Duality
Linear Programming Problem
 A linear programming (LP) problem is an optimization problem in
which we minimize or maximize a linear objective function subject to a
given set of linear constraints.
Example:
Minimize 3x1 − 5x2 + + 3x3 + 2x4 subject to:
3x1 + 4x2 = 6
−x3 + 2x1 − x2 ≥ 22
x5 ≤ −3.5
x3 + .5x4 = .8
xi ≥ 0 for all i
Solutions
Feasible Solution
A feasible solution to a linear program is a solution
that satisfies all constraints.
Optimal Solution
An optimal solution to a linear program is a feasible
solution with the largest(smallest) objective function value
for a maximization(minimization) problem.
Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana
Biswas(05)
Many optimization problems involve selecting a subset
of a given set of elements.
 Examples:
A vertex cover is a subset of vertices.
A spanning tree is really a subset of edges.
A knapsack solution is a subset of items.
 Can be formulated as LPs with integrality constraints.
Integer Program
 An Integer Program (IP) is an LP with Integrality
Constraints
 Integrality Constraints: Some or all the variables are
constrained to be integers.
Solving Linear/Integer Programming
Problems
 LPs can be solved efficiently (polynomially but slowly).
 IPs generally cannot be solved efficiently (it is NP hard). Some
specific IPs can be solved efficiently. Actually, their LP optimal
is guaranteed to be integral.
Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana
Biswas(05)
Using Indicator Variables
Many selection problems can be formulated as IPs using
indicator variables (or 0-1 variables).
 An indicator variable is defined for each element . A
value of 1 indicating the selection of the element and a
value of 0 indicating otherwise.
Few Examples are :
 vertex cover
 Set Cover
 Knapsack
Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana
Biswas(05)
Example: Unweighted Vertex Cover
 Variables: {xv | v ∈ V }.
 The IP:
Minimize ∑ xv
s.t.
xu + xv ≥ 1 ∀ (u, v) ∈ E,
xv ∈ {0, 1} ∀ v ∈ V.
Example: Knapsack
 Let the item names be {1, . . . , n}.
 Variables: {xi | 1 ≤ i ≤ n}.
 The IP:
 Minimize ∑i cixi
s.t.
∑i sixi ≤ K,
xi ∈ {0, 1} ∀ 1 ≤ i ≤ n.
Programming Problems
 LPs can be solved efficiently (polynomially but slowly).
 IPs generally cannot be solved efficiently (it is NP hard).
Some specific IPs can be solved efficiently. Actually, their
LP optimal is guaranteed to be integral.
LP Relaxation (Drop the integrality
constraint)
 Example: Unweighted Vertex Cover
 The IP:
Minimize ∑v xv
s.t.
xu + xv ≥ 1 ∀ (u, v) ∈ E,
xv ∈ {0, 1} ∀ v ∈ V.
 The LP relaxation:
Minimize ∑v xv
s.t.
xu + xv ≥ 1 ∀ (u, v) ∈ E,
xv >= 0 ∀ v ∈ V.
Example: Weighted Vertex Cover
 Variables: {xv | v ∈ V }.
 The IP:
Min ∑Cv xv
where
Cv : cost associated with vertex
xv : indicator variable
s.t:
xu + xv ≥ 1
xv ∈ {0, 1}
∀ (u, v) ∈ E
∀v∈V
Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana
Biswas(05)
LP Relaxation (Drop the integrality
constraint)
 Example: Weighted Vertex Cover
 The IP:
Min ∑Cv xv
s.t:
xu + xv ≥ 1
xv ∈ {0, 1}
∀ (u, v) ∈ E
∀v∈V
 The LP relaxation:
Min ∑Cv xv
s.t:
xu + xv ≥ 1
xv ≥ 0
∀ (u, v) ∈ E
∀v∈V
Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana
Biswas(05)
LP rounding
 If xv ≥ ½, round it up to 1
Else round it down to 0.
 Here xv is the solution obtained from LP
 E.g:
LP: ¼ c1 + ½ c2 + ¾ c3 + 4∕5 c4
IP : c2 + c3 + c4
Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana
Biswas(05)
Claim 1: Solution Obtained is
feasible
Let (u,v) ∈ E
Since the solution of LP is feasible, values of xv , v ∈ V, satisfy
xu + xv ≥ 1
(1)
⇒ atleast one of xu and xv ≥ ½
Assume x’u and x’v be the solutions obtained after rounding, then at least one
of them must be 1, i.e.
x’u + x’v ≥ 1
So the solution, obtained after rounding, is feasible.
Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana
Biswas(05)
Claim 2: C(S) ≤ 2LOPT
 According to the strategy some of the variables have been
increased to a maximum of double & some have been reduced
to 0, i.e Cv’ <= 2Cv.
Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana
Biswas(05)
So,
C(S): cost of solution obtained by IP
C(S) ≤ ∑v’ Cv’ xv’
≤ 2 ∑v Cv xv
= 2 LPOPT
( x’v ≤ 2* Xv )
Hence claim 2 follows
Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana
Biswas(05)
Set Cover Problem
A finite set (universe) U of n elements, U= {e1, e2,…, en}, a
collection of subsets of U i.e. S1, S2, …., Sk with some cost,
select a minimum cost collection of these sets that covers all elements
of U.
Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana
Biswas(05)
 IP:
• Indicator variable xs, xs ∈ {0,1}
xs =0
xs =1
if set S is not picked
if set S is picked
Min ∑s Cs xs
s.t.
∑s:e belongs to S xs ≥ 1 ∀ e ∈ U
xs = {0,1}
LP Relaxation:
Min ∑s Cs xs
s.t.
∑s:e belongs to S xs ≥ 1 ∀ e ∈ U
xs > 0
Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana
Biswas(05)
LP rounding for SC
Let f denote the maximum frequency of any element in U Si
 Find an optimal solution to LP-Relaxation
xs >1/f
xs <1/f
round it to 1
discard the set, i.e. round it down to 0.
Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)
Claims
 Claim 1: solution is feasible
 Claim 2: It gives factor f approximation
Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana
Biswas(05)
Claim 1: Solution is feasible
Let,
ei ∈ U ,
1≤i≤n
S be the collection of subsets of U
em : 1<m <n belongs to l subsets of S where 1<l<k
Since the solution of LP is feasible i.e. values of xs s ∈ S obtained satisfies
xs1 + xs2 + xs3 + ….+ xsl >1
(1)
⇒ atleast one of xs1, xs2, xs3,…., xl >1/f
⇒ x’s1 + x’s2 + x’s3 +….+ x’l> 1
Where x’si is the solution obtained after rounding. Thus it is feasible.
Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)
Claim 2: Factor f approximation
 For each set s ∈ Collection of picked sets(S), xs has been increased by a
factor of atmost f.
Let C(s): Cost of our solution
Therefore,
C(S) ≤ ∑s Cs x’s
≤ f ∑s Cs xs
= f LPOPT
∀s∈S
( x’s ≤ f* xs)
Hence it is a factor ’f’ approximation.
Note: f factor could be large. Later we’ll see a technique of
rounding that gives O(log n) factor.
Thanks to Bhavya Dhingra(06),Garvita Sharma(09),Archikana Biswas(05)
 Linear Programming - Example
Minimize 8x1 + 5x2 + 5x3 + 2x4 subject to:
3x1 + 4x2 ≥ 6
3x2 + x3 + x4 ≥ 5
xi ≥ 0 for all i
x = (2, 1,0, 3) is a feasible solution.
8*2 + 5*1 + 2*3 = 27 is an upper bound.
What is the Lower Bound?
Minimize 8x1 + 5x2 + 5x3 + 2x4 subject to:
3x1 + 4x2 ≥ 6
3x2 + x3 + x4 ≥ 5
xi ≥ 0 for all i
LB: 8x1 + 5x2 + 5x3 + 2x4 ≥ 3x1 + 4x2 ≥ 6
Better LB:
8x1 + 5x2 + 5x3 + 2x4 ≥ (3x1 + 4x2 ) + (3x2 + x3 + x4) ≥ 6+5 = 11
How to compute a good LB
Minimize 8x1 + 5x2 + 5x3 + 2x4 subject to:
3x1 + 4x2 ≥ 6 ……………….y1
3x2 + x3 + x4 ≥ 5……………y2
xi ≥ 0 for all i
Assign a non-negative coefficient yi to every inequality
such that
8x1 + 5x2 + 5x3 + 2x4 ≥ y1 (3x1 + 4x2 ) + y2(3x2 + x3 + x4 )
Then, LHS ≥ 6y1 + 5y2.
We are interested in finding yi’s such that RHS is
maximum. This leads to our dual problem.
 The corresponding dual for the given example will be:
max 6y1 + 5y2
such that,
and,
3y1 < 8
4y1 + 3y2 < 5
y1 < 5
y2 < 2
yi > 0
for all i
Thanks to Divya Narang(8), Gautam Pahuja(10), Harshi Verma(11), Monika Bisla(14)
Weak Duality Theorem
Theorem: If x and y are feasible then,
n
c x
j
>
n
b y
i
j
i
i =1
j=1
Proof:
n
n
m
 cj xj >  ( ai, j yi) xj =
j=1
j 1
i =1
m
n
 ( a
i, j
i 1
m
xj) yi
>  bi yi
j=1
Thanks to Divya Narang(8), Gautam Pahuja(10), Harshi Verma(11), Monika Bisla(14)
i =1
Set Cover
xs is 1 iff set S in included in the cover.
The Primal :
Objective : min ∑ Cs xs
s.t  xs > 1
e  U
S :eS
xs = {0,1}
LP relaxation: xs > 0
Thanks to Divya Narang(8), Gautam Pahuja(10), Harshi Verma(11), Monika Bisla(14)
Introduce an indicator variable ye for each of the
constraints in primal.
The Dual :
objective: max
s.t
y < y C
e
e
e:eSi
e:eSi
Si
for i = 1 to k
Thanks to Divya Narang(8), Gautam Pahuja(10), Harshi Verma(11), Monika Bisla(14)
Example
S = { x, y, z, w}
S1 = { x, y}
S2 = { y, z}
S3 = { x, w, y}
Let xs , xs , xs be an indicator variable for S1 , S2 , S3
1
2
3
respectively.
Let Cs , Cs , Cs is the cost of S1 , S2 , S3 respectively.
1
2
3
Thanks to Divya Narang(8), Gautam Pahuja(10), Harshi Verma(11), Monika Bisla(14)
Primal
Min : Cs xs1 + Cs x2 + Cs x3
1
2
3
Subject to
xs + xs > 1
1
3
xs + xs + xs > 1
1
2
3
xs > 1
2
xs > 1
3
xs , xs , xs > 0
1
2
(yx)
(y y)
(yz)
(yw)
3
Thanks to Divya Narang(8), Gautam Pahuja(10), Harshi Verma(11), Monika Bisla(14)
Dual
Max: yx + y y + yz + yw
Subject to
yx + y y < Cs
1
y y + yz < Cs
2
yx + y y + yw < Cs
3
yx , y y , yz , yw > 0
Thanks to Divya Narang(8), Gautam Pahuja(10), Harshi Verma(11), Monika Bisla(14)
From set cover via lp
Complementary Slackness
Conditions
Relaxed Complementary
Slackness Conditions
Example: Weighted Vertex
Cover
Primal:
Min ∑Cv xv
s.t:
xu + xv ≥ 1
xv ∈ {0, 1}
Dual:
∀ (u, v) ∈ E
∀v∈V
Max ∑ye
s.t:
∑e:e is incident on v ye < Cv ∀ v ∈ V
ye ∈ {0, 1}
∀e∈E
Primal Dual Schema 1
 U = empty, y = 0
 For each edge e = (u, v)
 ye = min {c(u) − ∑e′:u∈e′ ye′ , c(v) − ∑e′:v∈e′ ye′ }
 U = U union argmin {c(u) − ∑e′:u∈e′ ye′ , c(v) − ∑e′:v∈e′ ye′ }
 Output U
3
5
2
7
4
3
1
2
Thanks to Neha& Neha Katyal
3
3
5
2
7
4
(1)
Ye =3
3 (0)
1
2
3
For every edge pick minimum of two vertices
Min{4,3} = 3 Set ye=3
U has vertex having red color
Thanks to Neha& Neha Katyal
3
5(4)
2
Ye =1
7
4 (1) (0)
Ye =3
3(0)
1
2
Min{1,5} = 1 Set ye=1
Thanks to Neha& Neha Katyal
3
3
5(4)
2
Ye =1
7
4 (1) (0)
Ye =3
3(0)
Ye
=0
1
Ye
=0
Ye
=0
2
Min{1,0} = 0 Set ye=0
Min{2,0} = 0 Set ye=0
Min{3,0} = 0 Set ye=0
Thanks to Neha& Neha Katyal
3
3
5(4)(0)
2
Ye =1
Ye =4
7(3)
4 (1) (0)
Ye =3
3(0)
Ye
=0
Ye
=0
Ye
=0
Thanks to Neha& Neha Katyal
3
Ye =0
5(4)(0)
2
Ye =1
Ye =4
7(3)
4 (1) (0)
Ye =3
3(0)
Ye
=0
Ye
=0
Ye
=0
Thanks to Neha& Neha Katyal
3 (1)
Ye =2
Ye =0
2 (0)
5(4)(0)
Ye =1
Ye =4
7(3)
4 (1) (0)
Ye =3
3(0)
Ye
=0
Ye
=0
Ye
=0
Red-colored nodes form a vertex-cover
Thanks to Neha& Neha Katyal
3 (1)
Ye =2
Ye =0
2 (0)
5(4)(0)
Ye =1
Ye =4
7(3)
4 (1) (0)
Ye =3
3(0)
Ye
=0
Ye
=0
Ye
=0
Red-colored nodes form a vertex-cover
Thanks to Neha& Neha Katyal
Solution is feasible
 Trivial, since the algorithm runs for every edge.
 Let e= (u,v) be an edge.
 Suppose if possible, none of the xu and xv has been set
to 1 i.e constraints corresponding to u and v have not
yet gone tight and we have a ye that can be raised. That
means the algorithm has not yet completed.
Solution is 2 factor
 For every xv > 0, dual constraint is tight (trivially).
 For every edge e = (u,v),
1 < xu + xv < 2
Hence, by relaxed CSC, cost of the solution is at most
twice the OPT.
Primal-Dual Schema 2 (Ignore)
 Raise the dual variables uniformly until one or more of
the constraints become tight. Freeze the dual variables
contributing to these constraints. Set the
corresponding primal variable to 1.
 If more than one constraint becomes tight, take them
one by one in an arbitrary order.
3
5
2
7
4
3
1
2
Thanks to Neha& Neha Katyal
3
3
Ye =3/4
Ye =3/4
5
Ye =3/4
2
Ye =3/4
7
4
Ye =3/4
3 (1)
Ye =3/4
Ye =3/4
Ye =3/4
1
2
3
3
Ye =3/2
Ye =3/2
5
2
Ye =3/2 Ye =3/2
7
4
Ye =3/4
3 (1)
Ye =3/4
Ye =3/4
Ye =3/4
1
2
3
3
Ye =3/2
Ye =3/2
5
2
Ye =7/4 Ye =7/4
7
4
Ye =3/4
3 (1)
Ye =3/4
Ye =3/4
Ye =3/4
1
2
Thanks to Neha& Neha Katyal
3
Solution is feasible
 Let e= (u,v) be an edge.
 Suppose if possible, none of the xu and xv has been set
to 1 i.e consraints corresponding to u and v have not
yet gone tight and we have a ye that can be raised. That
means the algorithm has not yet completed.
Solution is 2 factor
 For every xv > 0, dual constraint is tight (trivially).
 For every edge e = (u,v),
1 < xu + xv < 2
Hence, by relaxed CSC, cost of the solution is at most
twice the OPT.