Constraint Satisfaction over a
Non-Boolean Domain
Approximation Algorithms and Unique Games Hardness
Venkatesan Guruswami
Prasad Raghavendra
University of Washington
Seattle, WA
Constraint Satisfaction Problem
A Classic Example : Max-3-SAT
( x1  x 2  x3 )( x2  x3  x5 )( x 2  x3  x5 )( x5  x4  x1 )
Equivalently the
largest fraction of
clauses
Given a 3-SAT formula,
Find an assignment to the variables that
satisfies the maximum number of clauses.
Constraint Satisfaction Problem
General Definition :
Example : Max-3-SAT
Domain : {0,1,.. q-1}
Predicates : {P1, P2 , P3 … Pr}
Domain :
Predicates :
Pi : [q]k -> {0,1}
Arity : Maximum number of
variables per constraint (k)
{0,1}
P1(x,y,z) = x ѵ y ѵ z
Arity
= 3
GOAL : Find an assignment satisfying maximum fraction
of constraints
Approximability
Most Max-CSP problems are
NP-hard to solve exactly.
Different Max-CSP problems
are approximable to varying
ratios.
Max-2-SAT : 0.94
[Lempel-LivnatZwick],[Austrin]
Max-3-SAT : 7/8
[Karloff-Zwick],[Hastad]
3-XOR :
½
[Hastad]
Max-Cut : 0.878
[Goemans-Williamson],
[Khot-Kindler-MosselO’donnel]
Refined
Question
:
Question :
Among all Max-CSP problems over
Which Max-CSP is the hardest to
domain [q] ={0,..q-1}, and arity k,
approximate?
which is the hardest to approximate?
Clearly, the problems become harder as domain size
or the arity grows
PCP Motivation
Completeness(C) :
If SAT formula is
satisfiable, there is a
proof that verifier accepts
with probability C
Soundness (S):
For an unsatisfiable
formula, no proof is
accepted with probability
more than S
( x1  x2  x3 )( x2  x3  x5 )( x2  x3  x5 )( x5  x4  x1 )
Probabilistically Checkable
Proof
(A string over alphabet {0,1,..q-1})
Random
bits
Verifier
ACCEPT/REJECT
PCP Motivation
What is the best possible gap between
completeness (c) and soundness (s) for a
PCP verifier that makes k queries over an
alphabet [q] = {0,1,..q-1} ?
Among all Max-CSP problems over
domain [q] ={0,..q-1}, and arity k,
which is the hardest to approximate?
Boolean CSPs
Hardness:
For every k, there is a boolean
CSP of arity k, which is NPhard to approximate better
than :
[Samorodnitsky-Trevisan 2000]
simplified by [Hastad-Wigderson]
[Engebresten-Holmerin]
[Charikar-Makarychev-Makarychev]
Every boolean CSP of arity k, can be
approximated to a factor :
2 2k
2k
Assuming Unique
Games Conjecture
[Samorodnitsky-Trevisan 2006]
Algorithm:
22 k
2k
[Hast]
[Trevisan]
Random Assignment
2k
2k
0.88k
2k
k / log k
2k
2
2k
1
2k
This Work : Non-Boolean CSPs
UG Hardness:
Assuming Unique Games Conjecture,
For every k, and a prime number q, there is a
CSP of arity k over the domain [q] ={0,1,2,..q-1},
which is NP-hard to approximate better than
Algorithm:
The algorithm of [Charikar-MakarychevMakarychev] can be extended to non-boolean domains.
Every CSP of arity k over the domain [q]
={0,1,2,..q-1} can be approximated to a factor
2
q k
qk
 k / q7 
 k 
 q 
Related Work
“Optimal approximation algorithms
and hardness results for every CSP, assuming
Unique Games Conjecture.”
[Raghavendra 08]
– “Every” so applies to the hardest CSPs too.
– Does not give explicit example of hardest CSP, nor the explicit value of
the approximation ratio.
[Austrin-Mossel 08]
“Assuming Unique Games conjecture,
For every prime power q, and k, it is NP-hard to
q(q  1)k
approximate a certain CSP over [q] to a factor >
”
k
q
– Independent work using entirely different techniques(invariance principle)
– Show a more general result, that yields a criteria for Approximation Resistance
of a predicate.
Techniques
• We extend the proof techniques of [Samorodnitsky-Trevisan
2006] to non- boolean domains.
• To this end, we
– Define a subspace linearity test.
– Show a technical lemma relating the success probability of
a function F to the Gower’s norm of F
(similar to the standard proof relating the number of
multidimensional arithmetic progressions to the Gower’s
norm)
• Along the way, we make some minor simplifications to
[Samorodnitsky-Trevisan 2006].
–
(Remove the need for common influences)
Proof Overview
Dictatorship Testing Problem
Given a function F : [q]R
[q],
• Make at most k queries to F
•Based on values of F, Output ACCEPT or REJECT.
Distinguish between the following two cases :
F is a dictator function
F(x1 ,… xR) = xi
Pr[ACCEPT ] =
Completeness
F is far from every
dictator function
(No influential coordinate)
Pr[ACCEPT ] =
Soundness
Goal : Achieve maximum gap between Completeness
and Soundness
UG Hardness Proofs
Test
For the restDictatorship
of the
talk, weOver
shall
focus F:[q]R -> [q]
functions
on Dictatorship
Completeness = C
Soundness = S
Testing. # of queries = k
Using [Khot-Kindler-Mossel-O’Donnell]
reduction.
UG Hardness Result:
Assuming Unique Games Conjecture, it is NPhard to approximate a CSP over [q] with
arity k to ratio better than C/S
Testing Dictatorships by Testing Linearity
[Samorodnitsky-Trevisan 2006]
Fix {0,1} : field on 2 elements
k = 2d
Given a function
F : {0,1}R -> {0,1}
• Pick a random affine
subspace A of dimension d.
• Test if F agrees with some
affine linear function on
the subspace A.
Random Assignment :
There are 2d+1 different
Every dictator
affine linear functions on A.
F(x1 , x2 ,.. xR ) = xi
2d
is
a
linear
function
There are 2 possible
functions
on A.space
over vector
{0,1}R
So a random function
satisfies the test with
probability 2 d 1 2k
2
2d

2k
Cubes
Gower’s Norm
x+y2
x
x+y1
For F : {0,1}R -> {0,1},
let f(x) = (-1)F(x) .
dth
Gowers Norm Ud(f) =
E[ product of f over C]
Expectation over random ddimensional subcubes C in {0,1}R
x+y1+y2
x
x+y3+ y2
x+y2
x+y1+y2+y3
x+y1+y2
x+y3
x
x+y1
x+y1+y3
x+y1
d-dimensional cube spanned
by {x,y1 ,y2 ,.. yd } is


C   x   yi | S  {1,..d }
 iS

Gower’s Norm
More Formally,
Intuitively, the dth Gower’s norm measures
the correlation of the function f with degree
d-1 polynomials.
Testing Dictatorships by Testing Linearity
[Samorodnitsky-Trevisan 2006]
Lemma : If F : {0,1}R -> {0,1} passes the test with
probability 2 k   then f = (-1)F has high dth Gowers
k
2
Norm.
(k=2d)
R -> {-1,1} has
Lemma : If a balanced function
f
:
{0,1}
Using Noise sensitivity,
high dth Gowers Norm, then
it has
There
are an
onlyinfluential
a FEW
coordinate
(k=2d)
influential coordinates.
Theorem : If a balanced function F : {0,1}R -> {0,1}
2k
passes the test with probability k   then it has an
2
influential coordinate
Extending to Larger domains
Fix [q]: field on q elements(q
is a prime). k = qd
Given a function
F : [q]R -> [q]
• Pick a random affine
subspace A of dimension d.
• Test if F agrees with some
affine linear function on
the subspace A.
Replace 2 by q in the
[SamorodnitskyTrevisan] dictatorship
test.
The Difficulty
Lemma : If F : {0,1}R -> {0,1} passes the test with
probability 2 k   then f = (-1)F has high dth Gowers
k
2
Norm.
(k=2d)
Over {0,1}R,
Subcube = Affine subspace.
Testing linearity over a random affine subspace, can be easily
related to expectation over a random cube.
Over [q]R ,
Subcube ≠ Affine subspace.
(2R points)
(qR points)
Success probability of a function
F : {0,1}R -> {0,1}, is related to :
let f(x) = (-1)F(x) .
E[ product of f over A]
Multidimensional
Progressions
x
x+y1
x+(q-1)y2
x+2y1
x+(q-2)y1x+(q-1)y1
x+(q-1)y2+y1 x+(q-1)y2+2y1x+(q-1)y2+ (q-2)yx+(q-1)y
1
2+ (q-1)y1
Expectation over random
d-dimensional affine subspace A in [q]R
(Affine subspaces are like
multidimensional arithmetic
progressions)
E[ product of f over C]
Expectation over random dq-dimensional
subcubes C in [q]R
x+(q-2)y2
x+(q-2)y2+y1 x+(q-2)y2+2y1x+(q-2)y2+ (q-2)yx+(q-2)y
2+ (q-1)y1
1
x+2y2
x+2y2+y1
x+2y2+2y1
x+2y2+ (q-2)y1
x+y2
x+y2+y1
x+y2+2y1
x+y2+ (q-2)y1 x+y2+ (q-1)y1
x
x+y1
x+2y1
x+ (q-2)y1
x+2y2+ (q-1)y1
x+ (q-1)y1
Alternate Lemma
Lemma : If F : [q]R -> [q] passes the test with
2
q
probability k
then f = (-1)F has high dqth Gower’s

k
Norm.
(k=qd)
q
•d-dimensional affine subspace test relates to
the dqth Gower’s norm
•The proof is technical and involves repeated use
of the Cauchy-Schwartz inequality.
• A special case of a more general result by
[Green-Tao][Gowers-Wolf], where they define
“Cauchy-Schwartz Complexity” of a set of
linear forms.
Open Questions
CSPs with Perfect Completeness:
Which CSP is hardest to approximate, under
the promise that the input instance is
completely satisfiable?
Approximation Resistance:
Characterize CSPs for which the best
approximation achievable is given by a
random assignment.
Thank You
Unique Games
A Special Case
x-y = 11 (mod 17)
x-z = 13 (mod 17)
…
….
z-w = 15(mod 17)
E2LIN mod p
Given a set of linear equations of the
form:
Xi – Xj = cij mod p
Find a solution that satisfies the
maximum number of equations.
Unique Games Conjecture [Khot 02]
An Equivalent Version
[Khot-Kindler-Mossel-O’Donnell]
For every ε> 0, the following problem is
NP-hard for large enough prime p
Given a E2LIN mod p system, distinguish
between:
• There is an assignment satisfying 1-ε fraction
of the equations.
• No assignment satisfies more than ε fraction
of equations.