22186
>> David Wilson: Okay. Well, today we're pleased to have Alexandra Kolla here. She
graduated from U.C. Berkeley, was at the Advanced Study before coming here for her
post-doc. She'll tell us about unique games.
>> Alexandra Kolla: Thanks for having me. I'm going to talk about joint results with
Konstantin Makarychev. And since I already gave a talk about unique here not too long
ago, I decided at least to change the color from white to blue so it looks different.
And probably you already heard my introduction previously, so the suspense is all killed
now. But let me recap some things about unique games.
Unique games conjecture is something, conjecture that [inaudible] Khot made in 2002.
And what's interesting about this conjecture is that it has been used since 2002 by many
researchers to prove optimal hardness approximation results.
An example is -- do you have a pointer here? Sorry. Last a laser pointer.
So an example is Khot and Regev proved that vertex cover is hard to approximate better
than two basically. And two is the best approximation algorithm we know. And Khot,
Kindler Mossel O'Donnel 2004 showed the hardness of 0.878 for max cut, which is also
the best algorithm, the best algorithm that we know by Gameson [phonetic] and
Williamson. And then 2008 a breakthrough that Raghavendra had that hardness that
every constraint satisfaction problem equals integrality gap of some constant epsilon.
And there are many other hardness results related on UTC for max cut vertex cover we
said. Graph coloring, max PC, quadratic forms, max sub graph, multi way cut, et cetera,
et cetera, the list goes on.
So that's why unique games conjecture is interesting. But what is the unique games
conjecture, what are unique games. Unique games you're given a graph with N vertices
and you're given K colors. And for each edge between two vertices U and V you're given
a constraint, a permutation pi UV of the colors.
And you're required to satisfy as many constraints as possible. So a constraint is
satisfied if you give a coloring to U and a color to U and a color to V such that these
colors match according to this permutation.
For example, for this particular instance, if it was color U with yellow and V with red, then
yellow maps to red according to pi UV. So this edge or this constraint is satisfied. So
this is then. And then I can ask to give a coloring of the graph. As to maximize the
number of satisfied constraints.
That should be clear from previous time anyway, what are unique games. So Khot in
2002 conjectured that given an instance of unique games with opt bigger than 1 minus
epsilon, so more than one minus epsilon fraction of constraints can be satisfied, then it is
NP hard to satisfy even a delta fraction of all constraints for every positive epsilon delta in
sufficiently large label size, alphabetize K. So even if the game was constraint
satisfaction problem is very, very satisfiable, called conjecture it's NP hard to even find an
assignment that satisfies a half or .1 of the constraints.
Okay. So that's kind of surprising, because we know how to satisfy, if an instance is
completely satisfiable we know how to find a satisfying assignment.
>>: Go over the information itself? It's NP hard but not for a given thing, not for a given
instance? It's NP hard besides ->> Alexandra Kolla: There is a K. An alphabet size K. Give me epsilon, I'll give you K.
And an instance we have to K subset.
>>: Written zero, NP hard for a specific instance.
>>: Given this [inaudible].
>> Alexandra Kolla: Right. I could see how it can be misread. But, no, that's ->>: The conjecture is that there sort of is a family of these hard instances that Khot
[inaudible] but still [inaudible].
>> Alexandra Kolla: I mean, if you give an instance we consider finite, it's not as in
constant time. Okay. So the nice thing about this conjecture is that not only that it was
so helpful in proving optimal hardness of approximation results, is that there has been
really the last ten years, even though people in theory have really thought about it hard
and worked on it for like really intensely, there's been no consensus really. It's not clear
if -- it's about 50/50 who believes it's true and who believes it's not true. So in fact there
have been efforts in both proving and disproving it, and let's see what's known.
So a question immediate question one can ask is do existing state-of-the-art methods
work for unique games. In fact Khot Vishnoi 2004 proved semidefinite programming
cannot disprove the unique games conjecture. They presented an integrality gap the
Khot Vishnoi instance, as it's called. That STP fails when randomly this distance.
And for other stronger semidefinite or linear programming relaxation entireties there's
been other results of this type that this could not be used to disprove unique games
conjecture by Charikar, Makarychev, Raghavendra, Steurer. And so what do we know?
Well, we do know some things. There's been some approximation algorithms around
even from the same paper that Khot made the conjecture.
So Khot has optimization algorithm Stravesand, Gupta, Tower and Charikar,
Makarychev, Makarychev. And we're going to stay here for a second and note that Khot,
Kindler, Mossel, O'Donnel showed that this algorithm is tied assuming the unique games
conjecture. So this algorithm given one minus epsilon satisfiable instance of unique
games can satisfy one minus square root epsilon log alphabetize of the constraints. And
that's tied. You cannot improve that unless you disprove UDC.
And another line of work, there has been investigation of what happens, what are special
families that maybe unique games is easy.
And then with [inaudible] Vishnoi, we had the result that says expander graphs, unique
games is expanders. And Makarychev proved that for delta taking into account the edge
expansion instead of the second eigen value. And then also Khot, Vishnoi and
[inaudible] by recent result of mine, and parallel petition by Barak, Hardt, Arora, Steurer.
So games that come from pi repetition are easy. And subexponential algorithms exist for
unique games, as seen by a late result by Barak Hardt Steurer. So that's all in the
algorithm side. It has been a lot of attempts to disprove UDC.
Now on the hardness side, there's also been some conjectures, stronger conjectures,
small set expansion conjecture by Raghavendra Steurer Trevisan and Tulsiani. And
weaker conjectures, equations already on the wheels by Moshkovitz, but all these
conjectures imply or are implied by unique games conjecture. And there have been no
work as far as I know. Finished at least. Algorithms for unique games. That's the
picture.
>>: So what does a code [inaudible] start?
>> Alexandra Kolla: So the three equations already on the wheel approximately
equations over the reel with three variables is UGC hard. Okay.
So expanders are easy. Khot Vishnoi is easy. Pi repetition is easy. In fact, we don't
know how hard instances of unique will look like.
And unlike other problems, we believe that random instances of SAP are NP hard
according to [inaudible] conjecture, factoring PQ for random P approximately same order
of Q is hard. Finding planted clicks [phonetic] is NP hard, roughly for less than square
root N to the one-half minus epsilon planted ->>: Another one where people disagree on ->> Alexandra Kolla: All right. Well, let me, let's put more things on my desk for now. I'll
be happy to be wrong. So but we really don't know what happens when it comes ->>: Sorry. You said -- I missed. So the first two that are random, you mean the third,
when you say it's NP hard, you mean [inaudible] maybe I missed it. We know that
finding ->> Alexandra Kolla: If a planted click is [inaudible].
>>: Are you talking about the random case?
>> Alexandra Kolla: Yes.
>>: Because you said NP hard. The word random is not [inaudible].
>> Alexandra Kolla: Oh, okay. Right.
>>: Random, that word.
>> Alexandra Kolla: Okay. I see. So then believe ->>: We know it's NP hard for the general graph.
>> Alexandra Kolla: Yes for random graph.
>>: Random graphs there's ->> Alexandra Kolla: There's no consensus either.
>>: There's no consensus.
>> Alexandra Kolla: I put it to it to the hard case for now and then we'll see. So natural
question to ask then let's try to extract the hard distributions for unique games where we
already know that completely random graphs with random constraints are easy, random
graph expanders. That's not very interesting anymore. But we can ask our semi random
unique games instances easy. And what does that mean?
So our model is as follows. For now, let's consider the general case of unique games be
adversarial mode. How to construct 1 minus epsilon instances in general in the worst
case, which is a constraint graph, which is a completely satisfiable set of constraints.
And we pick an epsilon fraction of all edges. Let's say epsilon prime, and replace every
constraint in epsilon prime with another constraint, which is now not satisfied anymore,
possibly.
And if all steps are adversarial, then we get the general worst case, 1 minus epsilon
satisfiable instance of unique games. So our model, the semi random model is one of
the steps is random.
Either the graph choice is random or the set of satisfiable constraints is random of the
epsilon fraction of edges you pick is random, or the constraints you choose to replace
with this epsilon fraction of edges is random and all other steps adversarial.
Then what we show is if we average degree of a graph is large enough, roughly greater
than log V alphabet size, then a semi random unique game is easy. There's the
polynomial time algorithm with high probability works and finds a half or some good
assignment.
And in fact this holds for all four models. So any of the steps is -- okay, the first step was
already known for expanders but the other three cases and for unique games in the
special case when equations are linear, unique games.
In this talk I'm going to just give the third step being random.
>>: Is that like four different ->> Alexandra Kolla: The first proof is already done in expander case, and it's three
different proofs, yes.
>>: Build up the suspense model, two for random and two for -- [laughter].
>>: You can prove the strongest.
>> Alexandra Kolla: Well, I figured this stuff would be more straightforward because last
talk it was building suspense. Anyway, let's just see how this case works. So now graph
is N graph, we start from any complete satisfiable set of constraints. So nature picks
epsilon fraction of edges at random and adversary place constraints on these edges with
some other constraints as the adversary wishes.
>>: So a question would be in the adversary -- the adversary at step two, for instance,
can't see what the [inaudible] is going to be at step three.
>> Alexandra Kolla: All steps are sequential. And we call these random edges
constraints. In order to analyze this case and see how the algorithm would work, let's
look at the standard semidefinite program that has been used many times for unique
games. So what is this semidefinite program?
Let's see how we would rewrite this optimization problem as quadratic vector program.
So we introduce an indicator vector UI for each vertex U and its color I. This is a flag. So
in the intended solution, in actual labeling, one of the UIs would be some unit vector and
all the rest are zero, and if we wanted to assign label I to vertex U, then U sub I for this
color would be the nonzero vector and all the others are zero. The number of satisfied
constraints equals this expression which is a half summation over all edges summation
over all Is, square normal view I minus V, PUV of I squared. Is that clear why this solves,
if we could solve this problem then we could solve the unique games problem?
>>: Solve which problem.
>> Alexandra Kolla: To find a solution like that that minimizes this quantity.
>>: Should E depend on I?
>> Alexandra Kolla: No. You can think of it as 1 or 0. It doesn't even have to be -- it can
be one dimensional if it's actually a solution. You don't have P, right?
>>: But in general E is to be 1.
>> Alexandra Kolla: No, I haven't said anything. I'm just saying if for some reason we
could solve -- we can find a solution that looks like that for every vertex, we have a flag
that's one in one label and 0 otherwise and minimize this quantity, then this is a quadratic
program basically that solves unique games, which is NP hard to do.
>>: V?
>> Alexandra Kolla: V is an edge. V sub permutation of I is the flag for the label. And so
if you could look at a particular edge and a particular label that says -- so they're like the
cases that both of them are 0. Both of them are 1, one of them is 0 and one of them is 1.
If both of them are 1 then it means the constraint is satisfied, because they gave the
correct label I was sent to pi UV of I. If both are 0 nothing happens. This is 0. If one of
them is 1 and the other is 0 it means the constraint is unsatisfied. So this counts the
number of unsatisfied constraints.
>>: But everything you're saying at the moment E is just E to the 1, little E. So that's
what you mean when you say [inaudible] you're saying ->> Alexandra Kolla: For one in particular, yes. I'm introducing vector notation for later.
But, yes.
>>: But it's not if you could solve this vector ->> Alexandra Kolla: You can think of it as 1 for now. Yes, that's enough. Okay. I just
wanted to make sure that you're convinced that this counts the number of unsatisfied
constraints. It's very simple what I'm saying. Cool. So now we can relax this 1/0
condition, and we can write an actual semidefinite, post semidefinite constraint. Instead
of one of them being 1 and the rest are 0 now the square of the norms of these vectors is
1.
So you can think of it as a probability distribution or something else. And these vectors
are orthogonal and we also impose triangular equalities. This is some technical condition
that I'm not going to deal with right now. This is semidefinite programming standard
relaxation that has been used many times. And we can solve this program efficiently,
and this program outputs a vector solution.
So for each vertex and for each color there's a vector. All these vectors are orthogonal.
They possibly have different lengths. Some of them might be 0. And some of them -and that's how this solution to this semidefinite program looks like.
So let's see if we route this semidefinite problem, try to solve it for semi random instance,
what happens, there's a problem because this technique fails. And why does it fail? A
way to easily see why it fails is the beforementioned Khot Vishnoi instance, which is the
instance that is integrality gap, problematic for the standard semidefinite program, and
this instance all you have to know is that the value of the semidefinite program, this
minimization quantities say epsilon over 10, however, the integral value is very large.
Almost all constraints are unsatisfied. They cannot be satisfied.
So it's a really highly unsatisfiable instance where the STP thinks it's really highly
satisfiable. And semi random instances might indicate hidden Khot Vishnoi instances as
follows. So let's assume this is a Khot Vishnoi graph, and alphabet size is K. Here's the
Khot Vishnoi constraints. And we can also think of a different completely satisfiable
instance on the same graph on alphabet size K, and superimpose these two instances
together so that now our alphabet size is 2-K and here's like a Khot Vishnoi layer and a
completely satisfiable layer. So we just sort of lift -- we create a new instance by
combining both. And just having the original Khot Vishnoi permutations in the first K
label, say, and different permutations in the next K labels.
And now we randomly pick epsilon fraction of edges. And since the adversary has a
choice to change this permutation on these edges, what the adversary does, he can pick
the completely satisfiable layer and only change that, which would make the STP answer
the same basically as the Khot Vishnoi . So the STP would concentrate all its weight of
the vectors in the first K labels since the value of the solution is epsilon over 10, which is
less than epsilon.
Is it clear why this is a problem? Yes? You look a little confused.
>>: How many of those instances [inaudible].
>> Alexandra Kolla: So the edges remain the same number of edges. We just double
the alphabet size. And we consider, sorry, direct sums.
>>: How many lightning bolts?
>> Alexandra Kolla: Epsilon fraction.
>>: Personally mean for all, right? For all the ways of constraints [inaudible].
>> Alexandra Kolla: Yes, so I'm giving you a way to change the constraints -- the STP
could not do anything.
>>: What you want is the pluralities.
>> Alexandra Kolla: This is a counter example. It's enough to give one to show that this
doesn't work. I'm just saying that the STP doesn't work.
>>: STP ->> Alexandra Kolla: Because here's an example that in fact the STP value has nothing
to do, has no information about the actual solution, because the actual solution would
assign labels to the satisfiable layer. So these K labels. But the actual value of this
objective function is less, is bigger than what's for the Khot Vishnoi STP solution. So the
STP would concentrate all its norm of the vector it outputs in the Khot Vishnoi layer.
Okay. So this is a problem. And what we did to solve the problem is we introduce a
different STP, which we call crude STP. And instead of having the summation of all
these things to be one, now we require that all these vectors that it's an output to a
specific vertex, have norm one. So all the vectors are unique vectors. And, again, the
orthogonal trigonal qualities.
>>: [inaudible].
>> Alexandra Kolla: UI minus UJ spare cuts. Do you want me to write it? Square
triangle. It's a square. L2 squared.
Typically that's what happens in the standard STP. We only need for two vectors of the
same, say, vertex UI and UJ and all the vectors of the other vertices. But L2 squared.
Okay. So this is not a relaxation. In fact, even if unique game instances is completely
very satisfiable, then this might have big value. It really, the value of that doesn't say
anything about unique games. It doesn't have any meaning.
So let's see why we would want to introduce that. To see why, let's first take a step and
define the label extended graph which would be useful. So the label extended graph is
just a graph sort of lifted out by K. So for its original vertex of the graph, you now
introduce K vertices. Each one vertex for each label. And there is an edge between this
blurb of K vertices and this blurb of K vertices. There's a matching that corresponds to
the constraint or the permutation that was there.
So what this all means is that now I have N times K vertices, and the edge is between UI
and VJ if there was an edge between U and V, the original graph, and I equals pi UV of J.
So that's how they label the extended graph would look like. This is just rearranging for
my pictures to look better. Okay.
So we have a definition. An edge of the label extended graph now can be seen together
with this STP as having assigned to it two vectors. So edge U comma I, V comma J has
two vectors assigned, UI and VJ. And we call this edge super short if UI minus PJ
squared is less than some constant than over log K. So if really those two vectors that
PSTP outputs are very close. So this definition is tied to a specific STP solution.
Is that clear what super short edges. Okay. So here would be a super short edge. Here
would be a super short edge because these two vectors have very small square
difference.
And this is a set of super short edges in my label extended graph. Why do I care about
that? Well, the nice thing we can show is that if X sub U was a satisfying solution to the
original list of the noncorrupted instance, the completely satisfiable instance, then 1
minus some epsilon prime, which is depending on epsilon, fraction of edges U comma
XU and V comma XV, in the label extended graph are super short.
So there are very many super short instances in the satisfiable layer, basically. That's
what our structural theorem says. And let's see why would we care about having a lot of
super short edges in the satisfiable layer. For that, let's step back and look at how typical
popular Makarychev, Makarychev type semidefinite programming rounding for unique
games go. So we have here STP solution. There K orthogonal vectors for each vertex.
And basically what we do is we interpret this semidefinite programming solution as a
probability distribution of assignments variables to variables. And we want to pick an
assignment to variables by sampling from this distribution such that variables connected
by constraints are strongly correlated.
And what does that mean is that we pick a random Gaussian vector, and we examine the
projections of all these vectors on this random Gaussian vector and pick the one that has
basically the largest projection. We modify this to more delicate analysis to take all this
original different lengths of the vectors into account, et cetera, but the main idea is
Gaussian projections like that. So that's what [inaudible] does. And if we go back to the
super short edges and the label extended graph, we can see that here was the STP
solution on a super short edge. What's nice about it is that these two vectors are so
close that CMM-like STP rounding algorithms would not cut that edge. So if you pick a
random Gaussian vector the projections of both of those vectors on it would be very close
to each other.
So it would be likely, if we picked this one as being the largest projection, we would also
pick the other one. So that's good news. Seems the remaining super short edges, it's
unlikely that super short edges are caught by STP rounding algorithms. But they're bad
news, too, because this is a crude STP. It's not a relaxation and has no information
necessary to actually run these rounding algorithms. But have the same probability
chosen have same length have unit vectors. So there's no clear way how to extract the
actual information out of this STP. So what do we do to remedy that? Well, what we do
is we, together with this STP, we write an LP to sort of recover the lengths of those
vectors to find the probabilities that the vertex label pair belongs to the solution.
So our LP just maximizes over all super short edges. The mean of pi UI PUI and PVJ.
And we require that the sum of for each vertex the sum of those PUIs is one. And this
might be a little confusing, but all it does, you can think of the PUIs as the lost lengths of
these STP solutions. And then we round the STP and LP together using techniques
developed in the Makarychev, Makarychev algorithm.
Okay. And let's see a little bit say a few words about the structural theorem. So,
remember, if XU is a completely satisfying solution of the original instance before the
corruption happened, then some large fraction of edges in the label extended graph
would be super short. So that's the structural theorem. And the way to show that, I'm not
going to get into the proof because it's mostly technical. But the idea is that if this doesn't
happen -- what? I can tell you the proof on the board. It's really difficult on the slides.
So if this doesn't happen, then we combine the STP solution with an integer solution and
we show this new combined solution has a better value with high probability. So in fact
roughly what we do is every fixed STP with few super short edges can be improved with
high probability with the degree of the graph is greater than log K and then we take a
union bound over possible STP solutions. So the two main ingredients to show this is
strong concentration bound on the probability and also need to bound the number of STP
solutions by clever use of [inaudible] Strauss projections and taking epsilon edge.
>>: Can you say [inaudible]. I mean ->> Alexandra Kolla: This part?
>>: No, how we use the [inaudible].
>> Alexandra Kolla: So we basically say there's a randomized embedding for say all but
1 over K fraction. Fix an STP solution. We project it down to say log K dimensions. And
such that all but one K over fraction of the vectors have some new distance that is
roughly close to the old distance. And then we say, okay, that's what we do for each STP
solution, and there's like N -- E to the N times log K many possible ways to take STP
solutions, basically.
>>: So your slides you say post dimensional.
>> Alexandra Kolla: Think of K. That's conjecture K some constant.
>>: Right. How many points of high dimensional space are you projecting? [inaudible]
don't want to [inaudible] all points.
>> Alexandra Kolla: Yeah, just all but one over K fraction. One over K fraction. Actually
one over log K fraction is -- because basically what we want to do, we would like to have
the only task to run CMM, Makarychev, Makarychev, which says, okay, only one over log
K fraction is corrupted then we can recover constant solution. So we always forget about
this 1 over log K easily.
Okay. So what we showed is that if the average degree of the graph is greater than log
K, then semi random unique games with some relatively small epsilon are easy. So for
epsilon up to one-third we can show.
In fact, we can show that assuming a two-to-one conjecture which is something a little bit
like unique games conjecture, a little different if we take epsilon to be a little bigger than
current one-half then this is NP hard. So it's basically really -- it's not tight. But sort of we
hope half is the correct value. We didn't really try that much to improve that. But this is
interesting to see that. You can think of it as being UGC hard.
And for the other models, proofs are much different. And I'm not going to get into them.
But basically what we do is we don't care about short edges in the label extended graph
anymore. We care about long edges. We remove those and solve the STP twice. The
standard STP this time. We have a much more complex probabilistic analysis for these
cases. And I'm pretty much done.
So we introduce the semi random models of unique games. We showed that these
models are easy. And, you know, we can read the paper if you want. And the question
now is can we use these techniques for other semi random type of models for
combinatorial other optimization problems, it's interesting to see this is not unique games
specific. This is keep working on unique games. [applause].
>>: So this is the easy one.
>> Alexandra Kolla: This is the easy one.
>> David Wilson: Any further questions?
>> Alexandra Kolla: I think I completely lost all of you. [applause]