Yuan Zhou
Carnegie Mellon University
Joint works with Boaz Barak, Fernando G.S.L. Brandão,
Aram W. Harrow, Jonathan Kelner, Ryan O'Donnell and
David Steurer
Constraint Satisfaction Problems
• Given:
– a set of variables: V
– a set of values: Ω
– a set of "local constraints": E
• Goal: find an assignment σ : V -> Ω to maximize
#satisfied constraints in E
• α-approximation algorithm: always outputs a
solution of value at least α*OPT
Example 1: Max-Cut
• Vertex set: V = {1, 2, 3, ..., n}
• Value set: Ω = {0, 1}
• Typical local constraint: (i, j) э E wants σ(i) ≠ σ(j)
• Alternative description:
– Given G = (V, E), divide V into two parts,
– to maximize #edges across the cut
• Best approx. alg.: 0.878-approx. [GW'95]
• Best NP-hardness: 0.941 [Has'01, TSSW'00]
Example 2: Balanced Seperator
• Vertex set: V = {1, 2, 3, ..., n}
• Value set: Ω = {0, 1}
• Minimize #satisfied local constraints:
(i, j) э E : σ(i) ≠ σ(j)
• Global constraint: n/3 ≤ |{i : σ(i) = 0}| ≤ 2n/3
• Alternative description:
– given G = (V, E)
– divide V into two "balanced" parts,
– to minimize #edges across the cut
Example 2: Balanced Seperator (cont'd)
• Vertex set: V = {1, 2, 3, ..., n}
• Value set: Ω = {0, 1}
• Minimize #satisfied local constraints:
(i, j) э E : σ(i) ≠ σ(j)
• Global constraint: n/3 ≤ |{i : σ(i) = 0}| ≤ 2n/3
• Best approx. alg.: sqrt{log n}-approx. [ARV'04]
• Only (1+ε)-approx. alg. is ruled out even assuming
3-SAT does not have subexp time alg. [AMS'07]
Example 3: Unique Games
• Vertex set: V = {1, 2, 3, ..., n}
• Value set: Ω = {0, 1, 2, ..., q - 1}
• Maximize #satisfied local constraints:
(i, j) э E : σ(i) - σ(j) = c (mod q)
• Unique Games Conjecture (UGC) [Kho'02,
KKMO'07]
No poly-time algorithm, given an instance
where optimal solution satisfies (1-ε)
constraints, finds a solution satisfying ε
constraints
• Stronger than (implies) "no constant approx.
Example 3: Unique Games (cont'd)
• Vertex set: V = {1, 2, 3, ..., n}
• Value set: Ω = {0, 1, 2, ..., q - 1}
• Maximize #satisfied local constraints:
(i, j) э E : σ(i) - σ(j) = c (mod q)
• UG(ε): to tell whether an instance has a solution
satisfying (1-ε) constraints, or no solution
satisfying ε constraints
• Unique Games Conjecture (UGC). UG(ε) is hard
for sufficiently large q
Example 3: Unique Games (cont'd)
• Implications of UGC
– For large class of problems, BASIC-SDP
(semidefinite programming relaxation) achieves
optimal approximation ratio
Max-Cut: 0.878-approx.
Vertex-Cover: 2-approx.
Max-CSP
[KKMO '07, MOO '10, KV '03, Rag '08]
Open questions
• Is UGC true?
• Are the implications of UGC true?
– Is Max-Cut hard to approximate better than
0.878?
– Is Balanced Seperator hard to approximate
with in constant factor?
SDP Relaxation hierarchies
• A systematic way to write tighter and tighter
SDP relaxations
BASIC-SDP
r rounds SDPO (rrelaxation
)
in roughly n
time
?
…
UG(ε)
ARV SDP for Balanced Seperator
GW SDP for Maxcut (0.878-approx.)
• Examples
– Sherali-Adams+SDP [SA'90]
– Lasserre hierarchy [Par'00, Las'01]
How many rounds of tighening
suffice?
• Upperbounds
  (1 )
– n
rounds of SA+SDP suffice for UG(ε)
[ABS'10, BRS'11]
• Lowerbounds [KV'05, DKSV'06, RS'09, BGHMRS '12]
(also known as constructing integrality gap instances)
(1)
– exp((loglog n) ) rounds of SA+SDP needed
for UG(ε)
(1)
exp((log
log
n
)
) rounds of SA+SDP needed
–
for better-than-0.878 approx for Max-Cut
(1)
(log
log
n
)
–
rounds for SA+SDP needed for
constant approx. for Balanced Seperator
Our Results
• We study the performance of Lasserre SDP
hierarchy against known lowerbound instances
for SA+SDP hierarchy, and show that
• 8-round Lasserre solves the Unique Games
lowerbound instances [BBHKSZ'12]
• 4-round Lasserre solves the Balanced Seperator
lowerbound instances [OZ'12]
• Constant-round Lasserre gives better-than0.878 approximation for Max-Cut lowerbound
instances [OZ'12]
Proof overview
• Integrality gap instance
– SDP completeness: a good vector solution
– Integral soundness: no good integral solution
• A common method to construct gaps (e.g. [RS'09])
– Use the instance derived from a hardness
reduction
– Lift the completeness proof to vector world
– Use the soundness proof directly
Proof overview (cont'd)
• Our goal: to prove there is no good vector
solution
– Rounding algorithms?
• Instead,
– we bound the value of the dual of the SDP
– interpret the dual of the SDP as a proof
system ("Sum-of-squares proof system")
– lift the soundness proof to the proof system
Remarks
• Connection between SDP hierarchies and
algebraic proof systems
• New insight in designing integrality gap instances
– should avoid soundness proofs that can be
lifted to Sum-of-Squares proof system
• Lasserre is strictly stronger than other
hierarchies on UG and related problems (as it
was believed to be)
Outline of the rest of the talk
Sum-of-Squares proof system
Relation between SoS proof system and
Lasserre SDP hierarchy
Lift the soundness proofs to the SoS proof
system
Sum-of-Squares proof system
Polynomial optimization
• Maximize/Minimize p(x)
• Subject to
q1 ( x)  0, q2 ( x)  0,qm ( x)  0
r1 ( x)  0, r2 ( x)  0,rm' ( x)  0
all functions are low-degree n-variate
polynomial functions
• Max-Cut example:
2
E
(
x

x
)
Maximize
i
j
(i,j)E
s.t.
xi (1  xi )  0, i
Polynomial optimization (cont'd)
• Maximize/Minimize p(x)
• Subject to
q1 ( x)  0, q2 ( x)  0,qm ( x)  0
r1 ( x)  0, r2 ( x)  0,rm' ( x)  0
all functions are low-degree n-variate
polynomial functions
• Balanced Seperator example:
2
E
(
x

x
)
Minimize
i
j
(i,j)E
s.t.
xi (1  xi )  0, i
E[ xi ] 
i
1
3
, E[ xi ]  2 3
i
Certifying no good solution
• Maximize
• Subject to
p(x)
q1 ( x)  0, q2 ( x)  0,qm ( x)  0
r1 ( x)  0, r2 ( x)  0,rm' ( x)  0
• To certify that there is no solution better than
, simply
 say that the following equations &
inequalities are infeasible
p(x)  
q1 ( x)  0, q2 ( x)  0,qm ( x)  0
r1 ( x)  0, r2 ( x)  0,rm' ( x)  0
The Sum-of-Squares proof system
• To show the following equations & inequalities
are infeasible,
q1 ( x)  0, q2 ( x)  0,qm ( x)  0
r1 ( x)  0, r2 ( x)  0,rm' ( x)  0
• Show that
1 
 f ( x)q ( x)  h( x)
i 1...m
i
i
• where h(x) is a sum of squared polynomials,
including ri (x)'s
• A degree-d "Sum-of-Squares" refutation, where
d  max {deg( f i )  deg( qi ), deg( h)}
i
Example 1
• To refute
x2
x(1  x)  0
• We simply write
1  x(1  x)  ( x  2)  ( x 1)2
• A degree-2 SoS refutation
Example 2: Max-Cut on triangle graph
• To refute
( x1  x2 )2  ( x2  x3 )2  ( x3  x1 )2  2  
x1 (1  x1 )  0, x2 (1  x2 )  0, x3 (1  x3 )  0
• We "simply" write
... ...
Example 2: Max-Cut on triangle graph
(cont'd)


 ( x1  x2 ) 2  ( x2  x3 ) 2  ( x3  x1 ) 2  2  

 ( x1 x2  x2 x3  x1 x3  x2 ) 2  ( x1  x2  1) 2  ( x2  x3  1) 2
 x1 (1  x1 )(x22  x32  2 x2 x3  1)
 x2 (1  x2 )(x1  x32  2 x1 x3  2 x1  2 x3  3)
 x3 (1  x3 )(x1  x2  2 x1 x2  1)
• A degree-4 SoS refutation
Relation between SoS proof system and
Lasserre SDP hierarchy
Finding SoS refutation by SDP
• A degree-d SoS refutation corresponds to
d
solution of an SDP with O(n ) variables
• The SDP is the same as the dual of (d ) -round
Lasserre relaxation
Bounding SDP value by SoS refutation
• An SoS refutation => upperbound on the dual of
optimum of Lasserre => upperbound on the value
of Lasserre
– e.g. 4-round Lasserre says that Max-Cut of
the triangle graph is at most 2
(BASIC-SDP gives 9/4)
Remarks
• Positivestellensatz. [Krivine'64, Stengle'73] If the
given equalities & inequalities are infeasible,
there is always an SoS refutation (degree not
bounded).
• The degree-d SoS proof system was first
proposed by Grigoriev and Vorobjov in 1999
• Grigoriev showed (n) degree is needed to
refute unsatisfiable sparse F2 -linear equations
– later rediscovered by Schoenbeck in Lasserre
world
Lift the proofs to SoS proof
system
Unique Games
Components of the soundness proof
(of known UG instances)
•
•
•
•
•
Cauchy-Schwarz/Hölder's inequality
Hypercontractivity inequality
Smallsets expand in the noisy hypercube
Invariance Principle
Influence decoding
Hypercontractivity Inequality
• 2->4 hypercontractivity inequality:
for low degree polynomial f ( x) 
we have

 S  xi
S [ n ],|S |d
2
2
E n[ f ( x) 4 ]  9d  E n[ f ( x)
x{1,1}
 x{1,1}
iS
] 

• Goal of an SoS proof:
write
2
9d  E n[ f ( x) 2 ]   E n[ f ( x) 4 ]  i h( , {1} , {2} ,) 2
 x{1,1}
 x{1,1}
Note that  S 's are indeterminates
Hypercontractivity Inequality (cont'd)
• 2->4 hypercontractivity inequality:
for low degree polynomial f ( x) 
we have

 S  xi
S [ n ],|S |d
2
2
E n[ f ( x) 4 ]  9d  E n[ f ( x)
x{1,1}
 x{1,1}
iS
] 

• Goal of an SoS proof:
write
2
9d  E n[ f ( x) 2 ]   E n[ f ( x) 4 ]  i h( , {1} , {2} ,) 2
 x{1,1}
 x{1,1}
• Prove by induction (very similar to the wellknown inductive proof of the inequality itself)...
Components of the soundness proof
(of known UG instances)
•
•
•
•
•
Cauchy-Schwarz/Hölder's inequality
Hypercontractivity inequality
Smallsets expand in the noisy hypercube
Invariance Principle
Influence decoding
A few words on Invariance Principle
• trickier
• "bump function" is used in the original proof
--- not a polynomial!
• but... a polynomial substitution is enough for UG
Max-Cut and Balanced Seperator
• An SoS proof for "Majority Is Stablest"
theorem is needed for Max-Cut instances
– We don't know how to get around the bump
function issue in the invariance step
– Instead, we proved a weaker theorem: "2/pi
theorem" -- suffices to give better-than0.878 algorithms for known Max-Cut instances
• Balanced Seperator. Key is to SoS-ize the proof
for KKL theorem
– Hypercontractivity and SSE is also useful
there
– Some more issues to be handled
Summary
• SoS/Lasserre hierarchy refutes all known UG
instances and Balanced Seperator instances, gives
better-than-0.878 approximation for known MaxCut instances,
– certain types of soundness proof does not work
for showing a gap of SoS/Lasserre hierarchy
Open problems
• Show that SoS/Lasserre hierarchy fully refutes
Max-Cut instances?
– SoS-ize Majority Is Stablest theorem...
• More lowerbound instances for SoS/Lasserre
hierarchy?
Thank you!