Semidefinite Programming
Based Approximation
Algorithms
Uri Zwick
Tel Aviv University
UKCRC’02, Warwick University,
May 3, 2002.
Outline of talk
Semidefinite programming
MAX CUT (Goemans, Williamson ’95)
MAX 2-SAT and MAX DI-CUT (FG’95, MM’01, LLZ’02)
MAX 3-SAT (Karloff, Zwick ’97)
-function (Lovász ’79)
MAX k-CUT (Frieze, Jerrum ’95)
Colouring k-colourable graphs (Karger, Motwani, Sudan ’95)
Positive Semidefinite Matrices
A symmetric nn matrix A is PSD iff:
• xTAx  0 , for every xRn.
• A=BTB , for some mn matrix B.
• All the eigenvalues of A are non-negative.
Notation: A  0
iff A is PSD
Linear
Programming
Semidefinite
Programming
max cx
s.t. ai x  bi
x0
max CX
s.t. Ai X  bi
X0
Can be solved
exactly
in polynomial time
Can be solved
almost exactly
in polynomial time
LP/SDP algorithms
• Simplex method (LP only)
• Ellipsoid method
• Interior point methods
Algorithms work well in practice,
not only in theory!
Semidefinite Programming
(Equivalent formulation)
max  cij (vi vj)
(k)
(k)
s.t.  aij (vi vj)  b
v i  Rn
X ≥ 0 iff X=BTB.
If B = [v1 v2 … vn] then xij = vi · vj .
Lovász’s -function
(one of many formulations)
max JX
s.t. xij = 0 , (i,j)E
I X = 1
X0
Orthogonal
representation
of a graph:
vi vj = 0 ,
whenever (i,j)E
The Sandwich Theorem
(Grötschel-Lovász-Schrijver ’81)
 (G )   ( G )   (G )
Size of
max clique
Chromatic
number
The MAX CUT problem
Edges may be weighted
The MAX CUT problem: motivation
Given: n activities, m persons.
Each activity can be scheduled either
in the morning or in the afternoon.
Each person interested in two activities.
Task: schedule the activities to maximize the
number of persons that can enjoy both activities.
If exactly n/2 of the activities have to be held in
the morning, we get MAX BISECTION.
The MAX CUT problem: status
• Problem is NP-hard
• Problem is APX-hard (no PTAS unless P=NP)
• Best approximation ratio known, without SDP,
is only ½. (Choose a random cut…)
• With SDP, an approximation ratio of 0.878
can be obtained! (Goemans-Williamson ’95)
• Getting an approximation ratio of 0.942
is NP-hard! (PCP theorem, …, Håstad’97)
A quadratic integer programming
formulation of MAX CUT
Max  wij
s.t.
1  xi x j
2
xi {1,1}
An SDP Relaxation of MAX CUT
(Goemans-Williamson ’95)
Max
s.t.
w
ij
1  vi  v j
2
n
vi  R , || vi || 1
An SDP Relaxation of MAX CUT –
Geometric intuition
Embed the vertices of the graph on the unit sphere
such that vertices that are joined by edges are far apart.
Random hyperplane rounding
(Goemans-Williamson ’95)
To choose a random hyperplane,
choose a random normal vector
r
If r = (r1 , r2 , …, rn), and
r1, r2 , … , rn  N(0,1),
then the direction of r
is uniformly distributed
over the n-dimensional
unit sphere.
The probability that
two vectors are separated
by a random hyperplane
vi

vj


Analysis of the MAX CUT Algorithm
(Goemans-Williamson ’95)
exp 
sdp 
w
1
cos (vi  v j )

ij
w
1  vi  v j
ij
1
ratio
2
2 cos ( x )
 min
 0.87856...
1 x 1 
1 x
Is the analysis tight?
Yes!
(Karloff ’96)
(Feige-Schechtman ’00)
The MAX Directed-CUT problem
Edges may be weighted
The MAX 2-SAT problem
x1  x 2
x 2  x6
x4  x5
x 2  x5
x3  x5
x3  x7
x3  x4
x3  x4
x4  x7
A Semidefinite Programming
Relaxation of MAX 2-SAT
(Feige-Lovász ’92, Feige-Goemans ’95)
Max
s.t.
w
3  v0  v i  v0  v j  v i  v j
ij
4
v i  v j  v i  v k  v j  v k   1 , 0  i , j , k  2n
v n  i  v i
, 1 i  n
v i  R , || v i || 1
n
Triangle constraints
The probability that a
clause xi  xj is satisfied is :
v0
0i  0 j  ij
2
vi
 ij
vj
Pre-rounding rotations
(Feige-Goemans ‘95)
vi
v0
 0i
 v i
 vi
 0i
vi
 0i  f ( 0i )
f : [0,  ]  [0,  ]
f (   )    f ( )
Skewed hyperplanes
(Feige-Goemans ’95, Matuura-Matsui ’01)
Choose a random vector r
that is skewed toward v0.
Without loss of generality v0 = (1,0, …,0).
Let r = (r1 , r2 , …, rn), where r2 , …, rn ~ N(0,1).
Choose r1 according to a different distribution.
“Threshold” rounding
(Lewin-Livnat-Zwick ’02)
Choose a random vector r
perpendicular to v0.
Set xi=1 iff vi · r ≥ T( v0· vi ).
Results for MAX 2-SAT
Authors
Technique
Goemans-Williamson ‘95
Random
hyperplane
Pre-rounding
rotations
Skewed
hyperplanes
Threshold
rounding
Feige-Goemans ‘95
Matuura-Matsui ‘01
Lewin-Livnat-Zwick ‘02
Bound
0.878
0.931
0.935
0.941
Integrality ratio *
0.945
Inapproximability
0.954
The MAX 3-SAT problem
(Karloff-Zwick ’97 Zwick ’02)
A performance ratio of 7/8 is obtained using:




A more complicated SDP relaxation
The simple random hyperplane rounding.
A much more complicated analysis.
Computer assisted proof. (Z’02)
Approximability and
Inapproximability results
Problem
MAX CUT
MAX DI-CUT
MAX 2-SAT
MAX 3-SAT
Approx.
Ratio
Inapprox.
Ratio
Authors
0.878
16/17
0.941
Goemans
Williamson ’95
0.874
12/13
0.923
GW’95, FW’95
MM’01, LLZ’01
0.941
21/22
0.954
GW’95, FW’95
MM’01, LLZ’01
7/8
Karloff
Zwick ’97
7/8
What else can we
do with SDPs?
• MAX BISECTION (Frieze-Jerrum ’95)
• MAX k-CUT
(Frieze-Jerrum ’95)
• (Approximate) Graph colouring
(Karger-Motwani-Sudan’95)
(Approximate) Graph colouring
• Given a 3-colourable graph,
colour it, in polynomial time,
using as few colours as possible.
• Colouring using 4 colours is still NP-hard.
(Khanna-Linial-Safra’93 Khanna-Guruswami’01)
• A simple combinatorial algorithm can colour,
in polynomial time, using about n1/2 colours.
(Wigderson’81)
• Using SDP, can colour (in poly. time) using n1/4
colours (KMS’95), or even n3/14 colours (BK’97).
Vector k-Coloring
(Karger-Motwani-Sudan ’95)
A vector k-coloring of a graph G = (V,E) is
a sequence of unit vectors v1 , v2 , … , vn
such that if (i,j)E then vi · vj = -1/(k-1).
The minimum k for which G is
vector k-colorable is  (G )
A vector k-coloring, if one exists,
can be found using SDP.
Lemma: If G = (V,E) is k-colorable,
then it is also vector k-colorable.
Proof: There are k vectors v1 ,v2 , … , vk
such that vi · vj = -1/(k-1), for i ≠ j.
k=3:
Finding large independent sets
(Karger-Motwani-Sudan ’95)
Let r be a random normally distributed
vector in Rn. Let c  23 ln   13 ln ln  .
I  {i V | vi  r  c}
I’ is obtained from I by removing a
vertex from each edge of I.
Constructing a large IS
vi
r
vj
Colouring k-colourable graphs
Colouring k-colourable graphs
using min{ Δ1-2/k , n1-3/(k+1) } colours.
(Karger-Motwani-Sudan ’95)
Colouring 3-colourable graphs
using n3/14 colours.
(Blum-Karger ’97)
Colouring 4-colourable graphs
using n7/19 colours.
(Halperin-Nathaniel-Zwick ’01)
Open problems
• Improved results for the problems considered.
• Further applications of SDP.