>> Konstantin Makarychev: Today, Tselil Schramm will tell us about this small set conjecture.
Tselil is a grad year student at UC Berkeley and now she is interning at Microsoft research, so if
you want to prove or disprove the conjectures it's the right time to do so. Talk to her after the
talk about it.
>> Tselil Schramm: Thanks for the very optimistic introduction, Konstantin. Hello everyone,
thank you for coming to my talk. If at any point you have questions, please ask because I know
the crowd is of varying backgrounds. I'll begin with a couple of definitions to level the playing
field. I'll begin by defining the expansion of a graph or of a set in a graph. Basically, the
expansion of a set in the graph is a measure of the connectivity of the graph. It's the number of
edges leaving the set divided by the total number of edges inside the set in a d-regular graph.
The notation of the definition is not that important, but please remember this notation because
I'm going to be using it throughout. Here you can see an example, this set circled in red is
relatively non-extending and the set circled in blue is relatively extending set. It's interesting to
ask what is the minimum expanding set inside of a graph, so that is the parameter is the graph
extension. You can ask okay, what is the minimum expanding set inside a graph. In this case,
again, it's the set circled in red. You can kind of generalize this to ask what if I only want to find
non-extending sets of a specific side. Say I want to limit sets of size 1/6 the total number of
non-linear nodes in the graph, what is the minimum extending sets of Si’s. Here we can see in
this example that although there might be a non-extending set inside the graph, it's possible
that all sets of size 1/6 don't expand, or do expand a lot. This is an interesting extension of the
problem. You can design the problem, uniform sparsest cut, which is given a graph, find the
minimum expanding set. This is NP hard, but we have several approximation algorithms for
this. One of them is Cheeger's algorithm and this algorithm is going to be important in the talk.
How familiar, would anyone like a refresher of Cheeger's algorithm? Okay. I'll give you a
refresher of Cheeger's algorithm. Cheeger's algorithm is basically an algorithm with the
following guarantee. Say you give me a normalized adjacency matrix and a vertex and I will
return a set that has expansion at most this quantity relating to the vertex and the adjacency
matrix. This is like the Rayleigh quotient of the vector. Normalize meaning it's stochastic.
>> You said a vertex but you meant a vector.
>> Tselil Schramm: I said a vertex but I meant a vector, yes. And the way that the algorithm
works is you take the vector and you order the vertices for the entries of the vector according
to their size from least to greatest and then you output the set that maximizes this quantity.
You look at the first vertex and if this minimizes then you take the first two vertices, third
vertices and you output the set that minimizes this. It works by choosing a probability
distribution over the index at which you break the set and you bound the expectation and that
gives the desired approximation ratio. This is something that I will use later in the talk as well.
There is also other approximations, for example, [indiscernible] and they are better in some
regimes, but I won't discuss them today. Now there's the problem of approximating the sets of
size at most delta, so this goes back to the expansion profile and the question is what can we do
with this. There is some algorithms known, one of them due to someone sitting in this room, in
part, or two people sitting in this room, and the point is we don't have quite the Cheeger's
guarantee here. In the case of Ergamanister and Totali [phonetic] wear off by this factor which
is okay if the delta is constant but it gets bad if delta is really small [indiscernible] exponential
and here for the [indiscernible] Trevisan, there is a trade-off between how good the expansion
approximation is and the size of the set. If you are willing to output a larger set you can get a
pretty good approximation of the expansion. Why care about small set the expansion? I could
give you reasons and in an industry like Facebook and I want to test a product in a small group,
but actually, there are better heuristics than that, like, choose New Zealand, you know? Really,
the reason we care is because of theory. Small set expansion is kind of embroiled in this
important question in theoretical computer science about hardness of approximation. So you
have probably heard of the Unique Games conjecture. If you have heard of it but don't
remember it, or if you have never heard of it, don't worry. The point is, basically, that Unique
Games conjecture asserts that a problem called Unique Games is NP hard to approximate
within a certain ratio. If Unique Games is NP hard, we have reductions from it to vertex cover
and to a bunch of constraint satisfaction problems, think like 3SAT and MAX-Cut, but this is
really it, like a huge family of problems and we know that if Unique Games conjecture is true we
have all of these optimal approximation algorithms for vertex cover and all the constraint
satisfaction problems. Small set expansion is one of the few problems we know that reduces to
Unique Games, so if we can show the small set expansion is hard, that would imply that all of
these problems are hard to approximate and we have optimal bounds, and vice versa there's
kind of a weaker reduction from Unique Games to small set expansion that has some
assumption on the Unique Games instance that you have, namely, the small sets expand more
and so showing that small set extension is easy to approximate within a certain ratio could have
implications, I mean, it has implications that some version of Unique Games is not difficult
throwing that on to the full picture. Now in order to understand the rest of my talk, you will
have to understand this notion of gap problems. A gap problem is the following. Say that I am
handing you a graph and I promise you that either there exists a set of size delta V that has
expansion less than some parameter C, or I say all sets of size delta V has expansion of at least
S. This is known as a gap CS problem. I model it with small set expansion because that's the
one that we will need for today. Denote this problem in the following way, an algorithm that
decides between a CS gap problem which tells you which scenario you're in gives you an S over
C approximation algorithm. Does anyone have questions?
>>: Can you go back to the previous slide?
>> Tselil Schramm: Yeah.
>>: [indiscernible] bigger then C?
>> Tselil Schramm: Yes. In this case S is bigger than C. Yeah. It's a minimization problem. If
you can distinguish between these two cases, you can say whether you can get an
approximation for this vector. Gap amplification is when you take a gap problem and you
reduce it to a different gap problem where the gap is wider. In essence, you are taking an
approximation and saying it's hard to approximate within alpha and then saying okay. This
reduces to approximating within beta where beta is an easier thing to approximate. Here you
can see in the example a small set expansion. This would be like taking a set that kind of
expands, but not that much and then read the widening the gap so a set that expands
completely and a set that expands almost not at all. Gap amplification is a useful tool in
improving hardness of approximation results. If you want to show that a problem is hard to
approximate with a ratio of beta, what you can do is you can take a problem that is hard to
approximate with a ratio alpha, say you know of some problem that is hard to approximate
using alpha, and then you can do gap amplification until the ratio becomes beta. That implies
that the problem that was hard to approximate within alpha reduces to the beta problem and
so the beta approximation is also hard. Another reason you would do gap amplification is if you
want to show that a problem is actually easy to approximate within some ratio. You can take a
problem, the problem of approximating within ratio alpha and reduce this to approximating
within the ratio beta via gap amplification and if you have a beta approximation algorithm, you
have just disproved it's hard to approximate the problem within alpha. In our case, we would
be happier with a result like this because we don't know a problem that is hard to reduce to
Unique Games, but we could hope, or two small set expansion, but we could hope to take small
set expansion with the ratio that we want, reduce it to a ratio that we already know because of
the approximation algorithms and then show that it's easy. For the first reason it's important in
these two settings. Label Cover is very important problem and also in Unique Games and gap
amplification results are known for both of these problems. The way that you prove gap
amplification results, sorry. The way that you prove gap amplification results for these is you
do something like you take a graph times a power of the underlying instance. I will explain that
a little more later. Our method is not that and as a nice benefit we don't have any growth of
the vertex set and stuff like that, so it's kind of a novel gap amplification. Our result is gap
amplification results for the small set expansion problem matching the known Unique Games
gap amplification results. If you want the technical theorem, this is it. Basically, if we have a
function that grows slower than root epsilon, we can reduce between epsilon and that function
to any constant and beta. This is actually the main idea. Let me go back to this graph tensor
power. Normally, to do a reduction, to do a gap amplification in Label Cover or in Unique
Games, you take an underlying instance, which is a graph, and then you take its tensor power.
Taking the tensor power essentially amounts to taking the Cartesian product of the vertex set
and then adding edges between two vertices in the power graph if both of the endpoints have
edges in the underlying graph as well. Here, you can see the tensor power of this edge, which is
a simple graph. You would think since Unique Games and a small set expansion are kind of
related, probably, that you could try to apply the same sort of idea in order to achieve gap
amplification for small set expansion because this is the way that gap amplification is done for
Unique Games. There's an obstacle when you are doing this. The obstacle is the following. Say
I started with a gap that falls into the no case, falls into the case that every set with size delta
has expansion that is too large. Say that there is some set with size delta that has expansion
that is in the yes case, that has bounded expansion. Now I take the tensor product of the graph
and I look at the product of the set with itself. This set will have size delta in the new graph, in
the tensor power graph, but on the other hand the expansion doesn't change very much, so if it
had expansion epsilon here, it has expansion about epsilon here and so that means that we can
fail to reject graphs that we should have rejected. We can fail, in this case we can say totally
this has a set that is the size at most delta that expands by at most whatever, and on the other
hand here, we don't. This is a problem. What makes sense for gap amplification in small set
expansion? Random walks seem like kind of a good idea. Sort of by definition of expansion, if a
set is not expanding, then the probability maps of a random walk stays trapped inside the set
for a very long time like here. On the other hand, if a set is expanding then that means that a
random walk started from that set will spread really fast across the graph. We can try to think
of taking the graph power as in the matrix power to do the reduction instead of taking the
graph tensor power, which corresponds to the random walk. That's what we do. There's the
reduction. We start with given a d-regular graph with the lazy random walk matrix m. Our
graph, our output graph is going to be the TTH power of this matrix. Essentially, what that
means is that we take the original vertex set and on it we put edges with the probability of
starting at vertex A and ending at vertex B. Here is the reduction parameters are not very
important. The main point is that if we choose the parameters carefully we can amplify the
gap. Any questions at this point?
>>: So do you have [indiscernible] for the matrix m?
>> Tselil Schramm: Yeah. M is the matrix with self loop already added.
>>: [indiscernible]
>> Tselil Schramm: Exactly. If you start with 0 it's on and on.
>>: There must be some limit as to how much you can amplify the gap.
>> Tselil Schramm: There is.
>>: Where is that limit?
>> Tselil Schramm: We can get this gap amplification, but for some parameters it's
meaningless. For example, when this becomes 1.
>>: [indiscernible] corresponds to which [indiscernible]
>> Tselil Schramm: I see. This is sets that expand by at most A and this is all sets that expand
by at least B and it maps these sets that expand by at most 1/2ta or sets that expand by at least
tb squared.
>>: [indiscernible]
>> Tselil Schramm: Right. Delta maps in new values and stuff depending on the -- yeah. So
maybe this is actually a better way to think of the limiting. If you choose t too large then the
set becomes really, really small. Other questions?
>>: [indiscernible] specific [indiscernible]
>> Tselil Schramm: Yes. Well, it's not just a specific [indiscernible]. It's also, it's a lower band.
>>: I mean it doesn't [indiscernible]
>> Tselil Schramm: No, no. It doesn't do anything else. Yes.
>>: Is there some sort of spectral estimate on this?
>> Tselil Schramm: Yeah, completely. It's 100 percent spectral. I will get to that. Okay. The
proof proceeds as follows. First, we have to show that if you have a set that did not expand
very much in the base graph, when you take the power of the random walk matrix it won't
expand that much in the final graph. And then we also have to show that if the set expanded to
begin with then after t steps of the lazy random walk it will expand even more, so we get upper
and lower bounds for the expansion of the set after applying the graph power. This proof
proceeds in the following way. First, what we do is we say, we perceive the contradiction and
we say if the set did not expand enough in the final graph and after t steps of the lazy random
walk then there must have been some step, at least one step in which it didn't expand enough
along the way. Then what we want to do is we want to take the vector corresponding to that
set and apply Cheeger's algorithm to it, but there's some technical difficulties because the
support of the vector might be too big and there we use a method similar to Heer Bruckinster
[phonetic] to reduce the support of the vector and get the set. For the first part in which we
say that non-expanding sets don't expand too much, we can apply this bound which is known
and Oveis Gharan and Trevisan have a particularly nice exposition of it when they are trying to
prove something a little bit more sophisticated. Maybe I'll tell you guys a little bit about how
this goes in case you don't know, but in the end, the point is that you can bound the expansion
using a power of the expansion. What you essentially do is you turn this into a problem about
positive semi-definite matrices. The lazy random walk matrix is positive semi-definite and for
any positive semi-definite matrix p and Norm 1 vector x you can show that this quadratic form
is larger than the product of these two quadratic forms, so essentially, if you look at the
expansion after t steps of the random walk it should be greater than the, or it should be less
than the expansion after one set than the expansion after t -1 sets. Then what you can do is
you can turn this into a problem about eigenvalues. You can view this, which is actually this
quadratic form, as an expectation over the lambda i for the k and then you can apply Chevy
Chez Simon [phonetic] equality because the matrix is positive; and all of the eigenvalues are
positive, Not negative.
>>: You mean [indiscernible]?
>> Tselil Schramm: No. Maybe you could apply [indiscernible] inequality?
>>: If you wanted to use convexity of lambda i to the k [indiscernible]
>> Tselil Schramm: Yeah. You can do that. It turns out to be the same. This is just an
inequality that says if you have ordered a and b and you take a product like this it's positive, but
I think you can use convexity.
>>: If you don't have to, the [indiscernible] should not be important.
>> Tselil Schramm: It's not. The positivity isn't worth it.
>>: Is it?
>> Tselil Schramm: Yeah.
>>: Is the positivity important? Involved in all of the [indiscernible]
>>: The positivity is important to this proof, but not to the result.
>>: [indiscernible] if something is odd or even [indiscernible]
>>: [indiscernible] this proof does depend on positivity but the basic inequality there that…
>>: Is just reversibility. Just need reversibility.
>>: Yes.
>> Tselil Schramm: For this.
>>: Forward [indiscernible] odd or even?
>> Tselil Schramm: For this
>>: X prime p to the tx is bigger than x prime ex to the power of 2 [indiscernible]
>>: [indiscernible] works and then if you have [indiscernible] eigenvalues, but [indiscernible]
it's a special case of [indiscernible] conjecture, which is not known, but that special case known
from the [indiscernible] 40, 50. I once sent to Cheyenne like a whole list of reference and it
keeps being rediscovered many times. [indiscernible] told me that we discovered inequality,
wrote it up and then found it had, you know, ten earlier references.
>> Tselil Schramm: Thanks. Maybe you can arrange it? Okay. In any case, this suffices for the
upper bound on the expansion of a set in the target graph given the original expansion. In
order to show that sets that expand in the original graph expand more in the target graph, what
we do is we say the following. If by contradiction a set does not expand enough after t sets of
the random walk, then we look at the probability mass of the set each of the different walks
and we say okay. There must have been at least one step of the walk in which there wasn't
enough expansion and we try, well, we don't try to identify that set but just the existence of
that set is enough. Let's suppose by contradiction that there is some set S with bounded size
such that at most the expansion is bounded by beta in the target graph. Let 1 sub s be the
indicator vector for S and what we are going to do is we're going to try to extract from S a set
that did not expand enough in the original graph and get from that a contradiction. We know
that 1 minus beta is less than this quotient just by definition of expansion because one minus
this is the expansion of the set S. We can translate that into a statement about norms of
vectors. Now, what we do is we say if this product of the ratios, if we look at the product of the
norm of V1 over V0 where Vi is the distribution over probability mass of vectors at the ith set,
then if we look at the quotients of each of the sets and we look at the products what we end up
with is the quotient from the beginning to the end. This is actually the quantity that we dealt
with before which we have a bound on because of our assumption that the set did not expand
enough. Then we know that at least one of these terms has to be bounded by 2 over TTH root.
Now what we hope to do is we have this vector and we have a bound on the Rayleigh quotient
and we know that Cheeger's algorithm can give us a set with expansion at most root that. The
problem is that this vector might have really large support. This vector might have, you know,
all n vertices have known 0 support and then Cheeger's algorithm might output a set that has
size n over 2 and that’s something we want. We want to bound the number of vertices in the
set. We can trouble shoot this issue in the following way. We take our vector and we pick
some threshold theta and for every entry in the vector less than theta we just set it to 0 and
four entries greater than or equal to theta we just subtract theta. Using some bounds on theta
and the two norm of v which we get via the expansion, we can conclude that the Rayleigh
quotient, this, which is using in Cheeger's algorithm to get the bound is bounded by the
Rayleigh quotient of the vector before thresholding. In some sense reducing or thresholding
the vector did not affect its Rayleigh quotient too much. Okay. Then we can just use our
previous bound. We have an upper bound now on this Rayleigh quotient. From there we can
apply Cheeger's algorithm and we get a bound on the expansion of our new set and we also
have a bound on the size of the new set using Markov's inequality or what have you. And then
essentially by selection of parameters we can get the result. We have the following result,
small set expansion with sets less than or equal to A or all sets expanding by at least B if size
delta reduces to an instance of SSE where for all sets in size of 4 delta over tb squared the sets
of size half, there exists a set of size half ta or offsets are greater than tb squared. It's clearly
polynomial time because all we do is apply t stuff to the random walk. And the way that we do
this is first we showed that non-expanding sets don't expand too much and then we showed
that expanding sets ended up expanding a lot by contradiction and basically what we did is we
bound the expansion of the jth step of the random walk starting from S. We controlled the
support size of the vector by thresholding and then we applied Cheeger's algorithm, so pretty
simple. In order to get our kind of more human readable result, you can just set the
parameters like this and you get this result which is analogous to the unique gains. It's the
best-known gap application for Unique Games, but, you know, it's not like I think this isn't the
main message. I think the main idea is that you can do gap amplification using these random
walks.
>>: In this human readable way, what is eta?
>> Tselil Schramm: It is [laughter] it is any constant you want, so that for any eta that you want
here, there exists some epsilon that you can reduce from. Epsilon the way that we…
>>: I know what it says about eta, but I prefer the non-human readable. [laughter]
>> Tselil Schramm: Me too.
>>: [indiscernible]
>> Tselil Schramm: I think the non-human readable is kind of like the victors write the history
books kind of thing, yeah.
>>: [indiscernible] [laughter]
>> Tselil Schramm: I'd rather not say. Open problems from this, I think there are a lot of open
problems, but a lot of them seem very difficult. The first open problem which was the original
open problem for this work was can you get to more amplified gap amplification for small set
expansion. And also it would be interesting to get a tensor powerbase gap amplification for
small set expansion or random walk based gap amplification for unique gains. So a little bit
more [indiscernible] to these problems. If we can amplify the gap from small set expansion
with all sets, or with the existing set expanding by at most epsilon or all sets expanding by at
least 1 minus epsilon to some gap version of SSE that we know is easy to approximate, then this
would refute the hardness of more constrained version of Unique Games and that would be a
very interesting result because it sort of sheds light on the Unique Games conjecture and
maybe on the difference between small set expansion and Unique Games. We did not achieve
this gap amplification. Another interesting thing to do would be to get a tensor based gap
amplification for Unique Games or a random walk based, sorry a tensor based gap amplification
for small set expansion or a random walk based gap amplification for Unique Games. The
reason being that I think a lot of people think of these two problems as relatively equivalent, so
it's strange that the two problems aren't amenable to the same kind of gap amplification. In
applying tensor based gap amplification to small set expansion, one needs to overcome this
issue with the completeness, the problem that, you know, you can get new non-expanding sets
just by applying tensor products and I have thought a little bit about this but I haven't been able
to resolve it. On the converse side, maybe it would be possible to get a random walk based
reduction for Unique Games which I have also thought about a little bit. This would also be nice
because the random walk based reduction does not amplify the size of the vertex set and so
this could lead to better gap amplification because the size of the graph won't be a limitation as
it is usually. But the obstacle that one runs into here is say that your base graph is
disconnected, has several components and say that one of the components has a really, really
good assignment and the other component has like a so-so assignment. Then when you, so
what it would be natural to do would be to take the label extended graph and to apply the
random walk to it until the problem gets harder, I mean easier. But the problem is for the
perfect completeness component, when you apply the random walk it doesn't get anywhere. If
you have the component of size a third which is perfect, that's the lower bound for how much
you can amplify the gap. Whereas, in your other component things get worse and worse, it
doesn't make much of a difference. Overcoming this problem also would be interesting. I
thought about this a little. Maybe one could put some kind of expander on the whole instance,
but that didn't seem to work out. If you have any ideas about this I would be happy to discuss
them. Yes. Thank you.
>> Konstantin Makarychev: Questions?
>>: I guess one difference to the amplification for Unique Games is also so that, maybe it's just
always said that usually you want to reduce the gap, you want to amplify the gap so that it is
close to one. You said that you [indiscernible] to half.
>> Tselil Schramm: Sorry. I think for the Unique Games it should have been one minus the
corresponding.
>>: I mean, for small set expansion you could hope for that you could amplify the gap to 1.
>> Tselil Schramm: To 1.
>>: Did you think about, [indiscernible] one [indiscernible] of that is it gets Cheeger's
inequality. You usually use it as sort of is concentrates on expansion close to 0. But they are
also variance is an inequality that given the whole range of expansions are close to 1 and close
to 0. Just wondering if you try to increase the gap from half to close to 1?
>> Tselil Schramm: No. We didn't. We thought about it a little bit, but we didn't.
>>: So when you are raising random walks, are there other, is there any reason to suspect that
some kind of [indiscernible] random walk or other kind of random walk with the other, that you
could employ other [indiscernible] like inequalities that come with those random walks and give
you some other issue?
>>: Other random walks?
>>: [indiscernible] random walk like you walk [indiscernible] over this amount of biasing by a
degree or something.
>> Tselil Schramm: I mean, all of the results that I presented today were for d-regular graphs,
but actually we also have, I mean, we have analogous results for, you know, any degree
distribution and, again, we do a wave during the walk, actually.
>> Konstantin Makarychev: More questions? Let's thank the speaker again. [applause]