Yuan Zhou, Ryan O’Donnell
Carnegie Mellon University
Constraint Satisfaction Problems
• Given:
– a set of variables: V
– a set of values: Ω
– a set of "local constraints": E
• Goal: find an assignment σ : V -> Ω to maximize
#satisfied constraints in E
• α-approximation algorithm: always outputs a
solution of value at least α*OPT
Example 1: Max-Cut
• Vertex set: V = {1, 2, 3, ..., n}
• Value set: Ω = {0, 1}
• Typical local constraint: (i, j) in E wants σ(i) ≠ σ(j)
• Alternative description:
– Given G = (V, E), divide V into two parts,
– to maximize #edges across the cut
• Best approx. alg.: 0.878-approx. [GW'95]
• Best NP-hardness: 0.941 [Has'01, TSSW'00]
Example 2: Balanced Separator
• Vertex set: V = {1, 2, 3, ..., n}
• Value set: Ω = {0, 1}
• Minimize #satisfied local constraints:
(i, j) in E : σ(i) ≠ σ(j)
• Global constraint: n/3 ≤ |{i : σ(i) = 0}| ≤ 2n/3
• Alternative description:
– given G = (V, E)
– divide V into two "balanced" parts,
– to minimize #edges across the cut
Example 2: Balanced Saperator (cont'd)
• Vertex set: V = {1, 2, 3, ..., n}
• Value set: Ω = {0, 1}
• Minimize #satisfied local constraints:
(i, j) in E : σ(i) ≠ σ(j)
• Global constraint: n/3 ≤ |{i : σ(i) = 0}| ≤ 2n/3
• Best approx. alg.: sqrt{log n}-approx. [ARV'04]
• Only (1+ε)-approx. alg. is ruled out assuming 3SAT does not have subexp time alg. [AMS'07]
Example 3: Unique Games
• Vertex set: V = {1, 2, 3, ..., n}
• Value set: Ω = {0, 1, 2, ..., q - 1}
• Maximize #satisfied local constraints:
{(i, j), c} in E : σ(i) - σ(j) = c (mod q)
• Unique Games Conjecture (UGC) [Kho'02, KKMO'07]
No poly-time algorithm, given an instance
where optimal solution satisfies (1-ε) constraints,
finds a solution satisfying ε constraints
• Stronger than (implies) "no constant approx. alg."
Open questions
Is UGC true?
Is Max-Cut hard to approximate better than
0.878?
Is Balanced Separator hard to approximate with in
constant factor?
Easier questions
Do the known powerful optimization algorithms
solve UG/Max-Cut/Balanced Separator?
SDP Relaxation hierarchies
• A systematic way to write tighter and tighter
SDP relaxations
BASIC-SDP
r-round SDPOrelaxation
(r )
in roughly n
time
?
…
UG(ε)
ARV SDP for Balanced Separator
GW SDP for Maxcut (0.878-approx.)
• Examples
– Sherali-Adams+SDP [SA'90]
– Lasserre hierarchy [Par'00, Las'01]
How many rounds of tighening suffice?
• Upperbounds
  (1 )
– n
rounds of SA+SDP suffice for UG
[ABS'10, BRS'11]
• Lowerbounds [KV'05, DKSV'06, RS'09, BGHMRS '12]
(also known as constructing integrality gap instances)
 (1)
– exp((log log n) ) rounds of SA+SDP needed
for UG
 (1)
exp((log
log
n
)
) rounds of SA+SDP needed
–
for better-than-0.878 approx for Max-Cut
 (1)
(log
log
n
)
–
rounds for SA+SDP needed for
constant approx. for Balanced Separator
From SA+SDP to Lasserre SDP
• Are the integrality gap instances for SA+SDP
also hard for Lasserre SDP?
• Previous result [BBHKSZ'12]
– No for UG
– 8-round Lasserre solves the Unique Games
lowerbound instances
From SA+SDP to Lasserre SDP (cont’d)
• Are the integrality gap instances for SA+SDP
also hard for Lasserre SDP?
• This paper
– No for Max-Cut and Balanced Separator
– Constant-round Lasserre gives better-than0.878 approximation for Max-Cut lowerbound
instances
– 4-round Lasserre gives constant
approximation for the the Balanced Separator
lowerbound instances
Proof overview
• Integrality gap instance
– SDP completeness: good vector solution
– Integral soundness: no good integral solution
• Show the instance is not integrality gap instance
for Lasserre SDP – no good vector solution
– we bound the value of the dual of the SDP
– interpret the dual as a proof system
(”SOSd/sum-of-squares proof system")
– lift the soundness proof to the proof system
What is
the SOSd proof system?
Polynomial optimization
• Maximize/Minimize p (x )
• Subject to
q1 ( x)  0, q2 ( x)  0, qm ( x)  0
r1 ( x)  0, r2 ( x)  0, rm' ( x)  0
all functions are low-degree n-variate polynomials
• Max-Cut example:
2
Maximize (i,j)EE ( xi  x j )
s.t.
xi (1  xi )  0, i
Polynomial optimization (cont'd)
• Maximize/Minimize p (x )
• Subject to
q1 ( x)  0, q2 ( x)  0, qm ( x)  0
r1 ( x)  0, r2 ( x)  0, rm' ( x)  0
all functions are low-degree n-variate polynomials
• Balanced Separator example:
Minimize E ( xi  x j ) 2
(i,j)E
s.t.
xi (1  xi )  0, i
E[ xi ] 
i
1
2
,
E
[
x
]

3
3
i
i
Certifying no good solution
• Maximize
• Subject to
p (x )
q1 ( x)  0, q2 ( x)  0, qm ( x)  0
r1 ( x)  0, r2 ( x)  0, rm' ( x)  0
• To certify that there is no solution better than
, simply
 say that the following equalities &
inequalities are infeasible
p(x)  
q1 ( x)  0, q2 ( x)  0, qm ( x)  0
r1 ( x)  0, r2 ( x)  0, rm' ( x)  0
The Sum-of-Squares proof system
• To show the following equalities & inequalities
are infeasible,
q1 ( x)  0, q2 ( x)  0, qm ( x)  0
r1 ( x)  0, r2 ( x)  0, rm' ( x)  0
• Show that
1 
 f ( x ) q ( x )  h( x )
i 1... m
i
i
• where h(x) is a sum of squared polynomials,
including ri (x)'s
• A degree-d "Sum-of-Squares" refutation, where
d  max {deg( f i )  deg( qi ), deg( h)}
i
Positivstellensatz
Subject to some mild technical conditions,
every infeasible system has such a refutation
Caveat: fi’s and h might need to have high degree.
Lasserre SDP and SOSd proof system
A degree-d SOS refutation
 O(d)-round Lasserre SDP is infeasible
Summary
• Defined the degree-d SOS proof system
• Remaining task
Integral soundness proof
 low-degree refutation in the SOS proof system
Example 1
• To refute
x2
x (1  x )  0
• We simply write
 1  x(1  x)  ( x  2)  ( x  1) 2
• A degree-2 SOS refutation
One-slide How-to
Thm: Max-Cut of this graph
is ≤ blah
Proof: … Invariance Principle …
… Majority-Is-Stablest…
“Check out these polynomials.”
Thm: Min-Balanced-Separator
in this graph is ≥ blah
Proof: … hypercontractivity…
“Check out these polynomials.”
Example 2: Max-Cut on triangle graph
• To refute
( x1  x2 ) 2  ( x2  x3 ) 2  ( x3  x1 ) 2  2  
x1 (1  x1 )  0, x2 (1  x2 )  0, x3 (1  x3 )  0
• We "simply" write
... ...
Example 2: Max-Cut on triangle graph (cont'd)


 ( x1  x2 ) 2  ( x2  x3 ) 2  ( x3  x1 ) 2  2  

 ( x1 x2  x2 x3  x1 x3  x2 ) 2  ( x1  x2  1) 2  ( x2  x3  1) 2
 x1 (1  x1 )( x22  x32  2 x2 x3  1)
 x2 (1  x2 )( x1  x32  2 x1 x3  2 x1  2 x3  3)
 x3 (1  x3 )( x1  x2  2 x1 x2  1)
• A degree-4 SoS refutation
Latest results
• Our theorem on Max-Cut is improved by
[DMN’12]
– Constant-round Lasserre SDP almost exactly
solves the known instances
• Constant-round Lasserre SDP solves the hard
instances for Vertex-Cover [KOTZ’12]
Open question
• Does constant-round Lasserre SDP solve the
known instances for all the CSPs?
– I.e. SOS-ize Raghavendra’s theorem.
Thank you!