1
>> Nikil Devanur: Hi. It's my pleasure to introduce Zhiyi, who is from the
University of Pennsylvania and he's an intern here. It's his second time
interning here, and he will tell us about Exponential Mechanism For Social
Welfare from the point of privacy, efficiency and so on.
>> Zhiyi Huang: Yes. Thanks, Nikhil, and hi, everyone. Thanks for coming for
this talk. What I'm going to talk about today is how to design mechanism that
are both truthful and differential private, and this talk is based on a recent
paper with my advisor, Sampath Kannan, and some follow-up discussion with Aaron
Roth.
So this is what I'm going to talk about today. So first, I'll go through some
basic background about mechanism design, in particular designing truthful
mechanisms and as well as differential privacy. And if you are already
familiar with these two areas, feel free to take a nap for the first 15 minutes
of the talk. And then after I go through the basics and make sure we're on the
same page, I will go on and talk about our main result, which is a general
mechanism for getting both truthfulness and differential privacy
simultaneously.
And then at the end of the talk, I'll talk about some discussions which are
some extensions of our results, some discussion about model and conclude with a
few open problems.
So that's the plan of today's talk. First of all, let me start with a
motivating example, which is this oil field allocation scenario. Suppose the
government now want to allocate a bunch of oil fields to several companies, say
BP, Shell and so on. Now for each oil field, different companies may have
different values for various reasons. For example, maybe this gray oil field
is one on the Caribbean sea, and BP had the oil spill accident a few years
back, to BP has less incentive of getting this Caribbean sea, and maybe Shell
has done some extensive research on the field and figured out the amount of the
oil. And the fuel is much larger than everyone else's thinking so Shell has
more incentive for getting the fuel.
And similarly, we can make this assumption for all oil fields. So basically,
for each company and each oil field, there could be arbitrary value for the
company getting that particular oil field.
2
Now, what the government wants to do is to make allocation between the oil
fields and the companies. And just for the sake of presentation, I'll assume
that each company will get exactly one oil field. We can interpret this as
each company having limited resource and cannot start more than one new oil
field at a time. But this constraint is just for presentation purposes. It's
not a restriction for our result.
So what the government wants to do by making such allocation. Since it's the
government that's making the allocation, we can assume the government's goal is
not to maximize revenue, because after all, the government can just print some
money. What government want to do is maximize the overall goodness of the
society, and one natural objective is what's so-called social welfare, which is
defined to be the sum of each company's value on the oil field that it gets.
And it's not difficult to see that once we make the restriction that each
company get exactly one oil field, the allocations are simply matchings between
the companies and oil fields, and the objective of maximizing social welfare is
simply a max way matching problem.
So there are various ways of looking at this problem. For example, we can
think about this as an algorithm design problem, in which case we want to
design this red box in the middle, which is kind of an input/output interface.
What's the input to this red box is the private valuations of the agents. And
what's the output of this red box is some outcome from the feasible range,
which, in this example, is simply the set of all possible [indiscernible]
matchings.
And the goal is to maximize or minimize some objective function. In this case,
it's the total weight of all the address in the matching. And in particular,
there's nothing so special about max matching. We can replace this problem by
any algorithm design problem, say facility location, max cut, or Steiner Tree
or any of your favorite algorithm problem. And the algorithm design problem
can be interpret in this framework.
>>: So you have some minimization problems here.
[indiscernible].
>> Zhiyi Huang:
>>:
Right.
Minimization problems [inaudible].
You can understand
3
>> Zhiyi Huang: So first of all, minimization and maximization are the same up
to an [indiscernible]. And also, there could be a, say, mean cost flow or a
minimization problem that also fit into this picture.
>>:
Yeah, I don't understand how the [inaudible].
>> Zhiyi Huang:
>>:
[inaudible].
For example -What are the clear values?
>> Zhiyi Huang: Ah, that's right. So for minimization problem, it could be
that each player own one of the edges in the graph and then the player has a
cost for the maximum designer of choosing the edge. Let's say the maximum
designer is trying to purchase a set of edges that form a Steiner Tree that
connects a bunch of cities on the map. And it's kind of a procurement auction.
But so far, we haven't get into the auction setting. So far, it's just an
optimization problem and any algorithm problem can be kind of viewed in this
picture.
Okay. So if we are thinking about the max matching problem in terms of
algorithm design, then we are done, because we all know that this max-weight by
product matching can be solve in poly time and we can pick our favorite
algorithm and just run it.
But the real world situation is a bit different in the sense that all this
input data are private information held by these companies, and throughout this
talk, we will refer to this as agents in the market. And since all this
information are private to the agents, we need to incentivize the agent to
report their true values in order to pick a reasonable outcome based on this
underlying data.
So suppose we take this into account, what does the picture look like? Again,
we want to design this red box in the middle, and in order to make a difference
from the algorithm you point out call this red box mechanism is algorithm. So
as the input, instead of the true underlying data, what we will be getting as
input is simply the reported value from each of these individual agents, which
may or may not equal the true underlying data.
And based on this reported data, we need to pick some outcome, X, and along
4
with a payment vector, P. This payment vector P, we should interpret them as
some true that play important role in kind of incentivizing these agents
reporting the true value. And then our goal, again, is to maximize -- minimize
some objective with respect to the true underlying data.
Okay. So now what's the goal? The goal is to incentivize the agents to report
a true values so that we can make our decision and choose a reasonable outcome
to maximize the true underlying objective.
But in order to do so, we first need to understand why agents would lie about
valuations. What assumption in particular we need to make assumption about how
agents will behave in the market.
So as a standard assumption in game theory, and I guess in macroeconomics in
general, we assume that agents will aim to maximize the expected utility, which
has this quasi-linear form versus the evaluation of the outcome chosen by the
mechanism, minus the payment we charge the agent.
And the conceptual -- so what's the point? The point is that the agent will
lie if lying can actually increase this quasi-linear utility. So we want to
prevent that. The conceptual solution for this concern is to focus on the
so-called truthful mechanisms. What is the truthful mechanism? A truthful
mechanism is one where agents always maximize this notion of quasi-linear
utility by bidding the true value, no matter what the other agents do.
Suppose we have this nice property, then. We can say with confidence there's
no reason for the agents to lie, and therefore they should report their true
value and, therefore, simply base on their reported value we will be able to
pick something reasonable.
And one of the most famous example of such truthful mechanism is probably the
second prize auction, or its generalization, known as the VCG mechanism, named
after Vickrey, Clarke and Groves, which essentially choose an allocation which
maximize the overall happiness in the society; namely, the sum of the valuation
of all agents over this outcome. And what they say is that if you do that,
then there's some general method of deriving the payments to make it truthful.
So far, so good. We are able to handle truthfulness, at least for the social
welfare maximization problem. And arguably, most, if not almost all of the
previous work in economics as well as in algorithmic theory has been focused on
5
this truthfulness concern.
And, in fact, usually for most game theory talk, this slide is the end of the
introduction and we will get into the more precise model and results part. But
what I'm going to do today is slightly different. I want to argue that there's
actually something else we need to worry about if we want to incentivize an
agent to report their true values.
This other concern is that agent or people care about privacy. So over the
past few years, users has become more and more aware of how their private data
or their private information might be misused on the internet or by some other
third party databases. One famous example is the 2008 paper on the
de-anonymizing the NetFlix database. So the story is that NetFlix used to
release these database about which user put what kind of rate on each of these
movies. Of course, they also take an eye on privacy. But what they do is
simply make the database anonymous.
So intuitively, anonymous implies privacy, right? There's nothing you can
learn about individual agents' data once its anonymous. But what this paper
shows is that by combining this database with some other information from the
internet, they can actually de-anonymize the database and learn a lot about
individual agents' information from the database.
What they do is essentially using the fact that the agent not only submit rate
to NetFlix, they also submit rate to other places like IMDB or Amazon, which is
not completely anonymous. And they use that to identify who each of these
individual agent is, and then use that fact to deanonymize the whole database.
And a more recent example of how people are becoming aware of privacy is the
recent complaint filed by EPIC against Facebook, saying that Facebook has
misleading use of agent -- the use of private information and also has been
sharing more information about each individual agent than they should to third
parties and advertisers. And this has result in Facebook kind of has to fix
all these privacy issue and are subject to a privacy audit every other year for
something like 20 years from now.
And taking a step back to the example we considered at the beginning of this
talk in the oil field allocation example, all this valuations of the companies
are sensitive business secret in the following sense. So a company's value on
each individual oil field may comprise information about an extensive research
6
of the company has done on the area and also include information about, for
example, maybe a company has recent breakthrough and during technology and
stuff like that, which the company considers its competitive edge in the future
business and they do not want to reveal them to their competitors in the
market.
And suppose we run the traditional mechanism, say the VCG mechanism that almost
for sure that will leak nontrivial information about all these private values.
And therefore, even if the mechanism is truthful, a privacy-aware company or
privacy-aware agent may still choose to lie about a value or not participate in
the action in order to protect their privacy.
So the challenge here, motivated by all these examples, the challenge here is
how to get good privacy and at the same time still get nearly optimal
objective. And this has been a relatively new area known as differential
privacy over the past few years.
But so far, I haven't really defined what is privacy, because privacy is a wake
word. For example, in the first example, we have seen that making the database
anonymous is not enough to guarantee privacy. So what precisely do we mean by
privacy?
So ideally, what we want is that by participating in this database, any third
party should not be able to learn too much about my private information. So
more precisely, suppose I fix the participants and values of all our agents and
consider me reporting truthfully, reporting VI and reporting live all my
variation by reporting VI prime.
This mechanism, presumably will choose outcomes from two different
distributions. And what privacy means is that by simply looking at one or a
few samples from these two distribution, the adversarial should not be able to
distinguish these two cases. And more precisely, we will say the mechanism has
good privacy if the distribution of these two cases has some notion of distance
and most [indiscernible] for any fixed values of other agents and any way of
lying about my true value.
So what remains is that to define what notion of distance between distributions
that we choose to define this privacy. And there's a long story here, which
I'm not going to get into. So the notion of distance, we will use here it's
the infinite divergence between these two distributions.
7
So what does that mean? We will say a mechanism is differential private if I
fix the value of other agents and change my value from VI to VI prime, then the
probability that any subset of outcomes being chosen by the mechanism should
not change by more than a multiplicative E to the epsilon vector.
So what does that mean is that supposedly I look at this probability density
curve in the two cases where I lie about my value, and I report truthfully then
the probability then point-wise should be bounded by these E to the epsilon
vector.
>>: So when you combine this, when you look at it from the truthfulness point
of view, this says, okay, if I lie, let's say you're truthful, you don't gain
anything. But this is saying on the other hand, you also do not lose anything.
You also do not lose much. Because [indiscernible] even if I lie, I don't
lose.
>> Zhiyi Huang: Yeah, that a point I will get to in the second part. So I
guess what Nikhil says is essentially that if a mechanism is essentially
private, then it also implies it's approximately triple, because by lying, I
cannot change the outcome distribution by too much. Therefore, I cannot gain
too much by lying, right?
But there's a problem with that, because ideally, when we talk about
approximate truthfulness, what we want is we can get closer and closer to exact
truthfulness without hurting the objective that we are trying to maximize or
minimize.
However, by using this approach, if we want to get arbitrarily close to
truthfulness, then we need to get arbitrarily close to perfectly private. When
we get to perfect privacy, that essentially, all that we can do is the trivial
thing of essentially picking a random outcome from all possible outcomes
uniformly at random, which is very poor in terms of objective.
So ->>: So what I'm saying is actually, [indiscernible] then yes, let's say you
have truthfulness. Truthfulness is nice. Nobody can gain by lying.
>> Zhiyi Huang:
Okay.
8
>>:
But this constraint also says nobody can lose by lying.
>> Zhiyi Huang: Exactly. So yeah, that's another critic of directly using
differential privacy as a notion of truthfulness. But what we'll be doing is
by imposing payments and with the help of payment, we can actually incentivize
agent to tell the truth even if the outcome is kind of smooth and does not
change much, no matter what I ->>:
Outcome does not include the payment?
>> Zhiyi Huang:
Yeah, the outcome does not include the payment at this point.
>>: Is that the only -- [indiscernible] telling the truth of what you can do
right now.
>> Zhiyi Huang: No, it's actually, it should hold no matter what our agents
do. So I should probably put a VI -- V minus I prime just to say that fact.
So V minus I may not be the true value of other agents.
>>:
That is fixed --
>> Zhiyi Huang: Yeah. So and also, usually, we will assume epsilon is some
small constant, or even small O of 1. So each of the epsilon is really 1 plus
epsilon. But we choose this definition because for technical reason, and it's
usually the standard definition in differential privacy as well. But you can
imagine that it's one plus epsilon if that works more towards your intuition.
So I have given the definition of differential privacy, but let me also spend
two more slides to do some intuition about what is differential privacy and
what is the general way of getting differential privacy.
So first of all, it's not difficult to see that no nontrivial deterministic
mechanism can be differentially private because any event that's chosen with
probability zero at one particular input has to remain zero for all inputs. So
the best we can do is simply choose a fixed outcome no matter what the input
is.
That means we have to use randomness. The problem is how to use randomness.
So to get an intuition, I will briefly go through two general methods of using
9
randomness to get differential privacy, which is also particularly related to
our work. They are just input perturbation and exponential mechanism.
So let's first talk about the first method, input perturbation. The idea to
perturb the mark itself. So other than the original agent one to N, I will add
a bunch of dummy agents into the market whose value is drawn uniformly at
random from all possible valuations. And then what we will do is to run the
optimization problem on in enlarged market with the original agent and these
dummy agents to choose the outcome. And, if necessary, project the outcome
back to the original market.
So more concretely, think about this matching market. I can add a bunch of
dummy companies and find the max matching in that case and then only keep the
address that's adjacent to the actual company as my outcome.
And the hope is that by adding enough randomness by adding dummy agents with
random valuation, the whole market looks random enough and, therefore, it's
differentially private.
So what's the pro of this approach? The pro is that it's extremely simple, and
it's oblivious for which algorithm you're using in the middle and also
oblivious to the structure of the problem.
So we can take the algorithm as a black box and without knowing anything about
a problem, we can still use this approach to enforce differential privacy.
But, of course, that comes with a price. Since this method is not used any
specific property and structures of the problem, as you can expect, it usually
achieve very poor objective for most of the problems. In fact, it only works
for very restricted settings where, essentially, the total number of different
valuations for the agents is much smaller than the number of agents in the
market.
So what that tells us is that in order to get something more general and works
for more problems, we need to use specific structures of the problems. And
arguably, the only way we know of using specific structures of the problem
while still general enough for all problems is the exponential mechanism.
>>:
[inaudible].
10
>> Zhiyi Huang:
>>:
Only from the given ones or all possible?
>> Zhiyi Huang:
>>:
Uniformly, at random, from all possible valuations.
All possible.
So we know all possible?
We assume that we know?
>> Zhiyi Huang: Yeah, okay. We have to have some knowledge about problem.
Also, we need some knowledge to project this outcome back to the original
market, but that's pretty much it. I don't take any too specific structure of
the problems. Okay?
So the exponential mechanism is originally proposed by Frank McSherry and Kunal
Talwar in 2007. What the exponential mechanism do is to choose the outcome X
from the feasible range with probability proportional to the exponent of the
performance of this outcome, scaled by the epsilon, which is the privacy
parameter, and over divided by two delta. Where delta is the Lipschitz
constant of this F function in terms of V1 through VN. But for the purpose of
this talk, we can always assume that is 1, because I always scale the function
properly so that the Lipschitz constant is 1.
So let's ignore data for now.
>>:
[inaudible].
>> Zhiyi Huang: Exactly. But, yeah, right now, I'm assuming this is a
maximization problem. If it's a minimization problem, I'll have to take the
negation over here.
So there's some nice thing about this exponential mechanism. First of all,
it's always epsilon differentially private, no matter what the problem is. And
this is not difficult to verify. And moreover, it could be the right answer
for differential privacy if we ignore computational efficiency.
So Aaron Roth actually conjectured that this is the right answer, but I'm more
conservative towards this conjecture. As a matter of fact, it has proved to be
a symptotically optimal in terms of a trade-off between the objective function
and privacy for many problems. For example, counting queries, combinatorial
public projects, K-medians and set cover and many other problems. And we do
11
not know a single count example where this exponential mechanism is not a
symptotically optimal, ignoring the computational efficiency matter.
So actually, this is the mechanism we'll be playing with and get truthfulness
out of it so I think it's important that everyone understand the definition of
this mechanism and is there any question about?
Okay. If there's no question about settings and about mechanism design,
differential privacy, I'll move on to the second part, which is how to kill two
birds with one stone, kind of combine the techniques from mechanism design and
differential privacy to get both truthfulness and differential privacy at the
same time.
Okay. Let me first, you know, I hate definition, but let me spend one slide to
make sure we're on the same page. We assume there are N agents and some
feasible range of outcome, R. And each agent has some private values matching
from R to 0, 1. I choose this interval, 0, 1, because I want to make sure the
social welfare function has Lipschitz constant 1 in terms of each individual
agent's input.
And the objective is to choose an outcome from the range to maximize the social
welfare, which is defined to be the sum of the agent's value on this outcome
chosen. And since we will be considering randomized mechanism, the objective
will be maximizing the expected social welfare. And truthfulness and
differential privacy is, we defined in the previous slides. I want to make one
remark here about our definition of differential privacy. That is, I'm
assuming in this talk that we only consider the differential privacy issue of
the allocations. But in real world, the payments may leak information as well.
So we do handle payments in our paper, but the reason I don't want to talk
about that is the techniques for handling payment is quite standard, and
essentially just adds some Laplacian noise into the payments and I really feel
the differential privacy concern of the allocation is the more interesting
part. So that's what I'm doing to talk about in the talk and ignore the
payments.
So given the model, what's the general question we are trying to attack. What
we are trying to attack is that can we design mechanisms, can we find a general
way of designing mechanisms that simultaneously achieve all four of the
following. We want to have differential privacy. We want to have
12
truthfulness. We want to get near optimal social welfare subject to the
privacy constraint, and as computer scientists, we can't computational
efficiency.
But, of course, I'm being too greedy by saying I want to achieve all four of
these. Because even ignoring the differential privacy part, getting
truthfulness good social welfare and computational efficiency is the central
problem in algorithmic theory for the last ten, twelve years and we're still
far from being able to completely understand that.
So there's no hope to answer this question in one shot. So what our strategy
is, is to first focus on the first three objective, differential privacy,
truthfulness and good social welfare, and provide something as general as VCG.
And then, on a problem by problem basis, we can consider the computational
efficiency issue.
And in this talk, I'll only talk about the interaction of the first three part,
which I feel is the more interesting part. So there are different models of
getting truthfulness and privacy at the same time, but I feel I have already
throw in too many definitions right now. So instead of distracting you guys
with different definitions, I'll simply state my choice of model and then at
the end of the talk, I will justify why I feel this is the more appropriate
model to use.
So what we are going to assume is that agents will not participate in the
auction unless the mechanism is epsilon differentially private. And once the
agent choose to participate, then three will aim to maximize the usual notion
of quasi-linear utility. So under this assumption, what we need to do is
incentivize agent to report a true value is to design a mechanism whose
allocation rule is epsilon differentially private and the allocation, together
with the payment satisfies the usual notion of truthfulness.
Now, suppose we take this assumption. What is known already in this model. So
it turns out that suppose we only want any two of these three objectives we
already know what to do if we want welfare and truthfulness, we can simply run
the VCG mechanism, which gets optimal social welfare and perfectly truthful.
Suppose we want good social welfare and differential privacy. Then arguably,
we can use exponential mechanism to achieve of the optimal trade-off. At least
that's the case for many problems.
13
Suppose we only want truthfulness and privacy, then we can have the trivial
solution of always picking an outcome that's independent of the input, which is
perfectly truthful and perfectly private. Of course, has very poor social
welfare. So the real challenge is getting all three of those.
And even this has been considered before. So in the original paper that
McSherry and Talwar exposed this exponential mechanism, they also point out
that differential privacy also implies approximate truthfulness, like Nikhil
just points out. What that means is that by definition, lying cannot change
the outcome distribution by too much and, therefore, I cannot gain too much as
long as the mechanism is epsilon differentially private. And therefore, that
implies approximate truthfulness.
And Nikhil has also made some critics about this approach that although people
have less -- not much incentive to lie, they don't have much incentive to tell
the truth as well. And also, we cannot get arbitrarily to true exact
truthfulness without hurting the objective function.
So that's not as appealing a solution concept as we would like. So in order to
handle that, there's some follow-up paper by Nissim, et al., in 2012 and also
independently by [indiscernible] in 2010 that show how to convert this nearly
truthful mechanism into exactly truthful ones in some specific settings.
But first of all this way of converting it into actual mechanism only work for
very restricted settings and also after this conversion, the mechanism is no
longer differentially private so we're getting truthfulness. But on the same
time, we're losing privacy.
And as an attempt to getting truthfulness and privacy at the same time, David
Xiao studied mechanism design without payment, and proposed using input
perturbation as a general method of doing so. So what does that mean? Input
perturbation means I'll add a bunch of random agents into the market as before.
And in the middle, I'll use a truthful mechanism in the red box and, therefore,
this mechanism with respect to the original agent should still be truthful.
Of course, as we say that input perturbation only works for very restrictive
settings and this method is not as general as we want.
So what we prove is there's actually a very general method of getting
14
differential privacy and truthfulness at the same time for very general
settings. So offers, record the exponential mechanism is to choose an outcome
X proportional to the exponent of the social welfare scale by the privacy
parameter. What we show is that for any mechanism design problem, as long as
the objective is social welfare, the exponential mechanism can be coupled with
some proper payments to make it truthful. Exactly truthful.
So how we should interpret this, in some sense, this is a family of
generalization of VCG mechanism for which by scaling the epsilon from positive
infinity to zero, we can have a family of differentially private version of
VCG. Where epsilon goes to infinity, this is the VCG mechanism, because we
will always choose the outcome which maximizes the social welfare. When
epsilon goes to zero, we get this perfectly private but trivial kind of
uniformly random, picking an outcome uniformly at random from the feasible
range.
>>:
[inaudible] also like consider payments as also part of the --
>> Zhiyi Huang: Yes. So there are two parts of outcomes, actually. So the
first outcome is what I refer to as outcome is the outcome in the feasible
range, and that has something to do with the social welfare. And that part has
to be differentially private. But also, the payment part has to be
differentially private, right.
But as I mentioned, there are very standard trick of payments, standard tricks
for the payments to make it differentially private as we can add some zero min
noises to the payments. And since the agent only aim to maximize the expected
utility, that doesn't change, really change the utility, assuming risk neutral.
And therefore, I will focus on the differential privacy concern of the, simply
the outcome.
And also, depending on how much you believe in that conjecture, that
exponential mechanism is the right answer for differential privacy, we can say
that for many problems, differential privacy is compatible with truthfulness,
at least for this maximum design with payment setting.
So before I move on to the proof, is there any question about the statement?
Okay. So there are different ways of proving this theorem, in fact. The
original proof we have is a bit complicated, but later we found a very cool
15
proof by making a connection to physics.
So let me first introduce some background, essentially some high school or
college physics. So the notion I want to talk about is Gibbs Measure. Or
Boltzmann Distribution sometimes in physics. So consider come particles of a
gas in a container, and assume this gas has K energy state. You can't to EK
what the Gibbs Measure or the Boltzmann Distribution says is that suppose I
pick a random particle from the container. Then the probability that it has
stayed J is proportional to the exponent of negative energy of the state
divided by the Boltzmann constant and the temperature.
And this sometimes is also known, in a less precise constant, as nature prefers
low energy as lower the energy is, the higher this probability is. Or higher
temperature implies more chaotic system because T goes infinity will have a
uniform distribution over all possible states.
So then I would like to make the simple observation that the exponential
mechanism itself is a Gibbs Measure. So here's a -- I want you to verify this
observation by this table so I guess simply by staring at the probability
density function or mass function, we can already see the similarity between
these two guys. But let me make it more precise. So in the Gibbs Measure
setting, nature want to minimize the energy. And in terms of exponential
mechanism, they want to maximize this notion of social welfare.
So the social welfare and the negative energy are playing similar roles in the
probability max function. And also in both settings, we have some parameter
come specify how chaotic the system is. In Gibbs Measure, we have the
temperature. And in exponential mechanism, we have the privacy parameter,
where the smaller this privacy parameter is, the more chaotic the system needs
to be, because we want more privacy.
And these two guys are also playing similar role in the system. So where are
we going with this? The point is there has been a lot of study for Gibbs
Measure or Boltzmann Distribution by statistical physics and by making a
connection between these two guys, we can borrow some of the theorems and
truths from Gibbs Measure and use that to prove our result.
So more precisely, the notion I want to borrow from Gibbs Measure is the notion
of free energy.
16
So what's a free energy? Suppose we have a distribution, D, the free energy of
this distribution at temperature T is the expected energy supposed to stay is
drawing from this distribution, minus the Shannon entropy of the distribution,
multiplied by KB times T.
And it turns out that this fully characterized the Gibbs Measure. As Gibbs
Measure is the distribution that minimizes the free energy. And sometimes,
this is also known as nature maximizes the entropy given the expected energy
level.
And this can be easily verified either by taking the, first of all, the
condition of these minimization problem or there are various way. But I'm not
going to bother you with the math here. So just trust me, this is true for
now.
Since we have make a connection between Gibbs Measure and exponential
mechanism, we can translate this fact into the language of exponential
mechanism, right?
So what that means is that exponential mechanism actually is maximizing this
guy, which for fun I just call it the free social welfare. So the free social
welfare is the expected social welfare, suppose the outcome is chosen from that
distribution, plus the Shannon entropy of the distribution scaled by 2 over
epsilon.
This is simply by replacing the corresponding terms in the probability mass
function and translate a previous fact in the language of exponential
mechanism.
So if you are familiar with game theory and mechanism design, this actually
implies the exponential mechanism is a maximum distributed range allocation
and, therefore, there are standard techniques to make it truthful.
And in case you don't see that, I have one slide which explain why this is the
case. So in order to see why the exponential mechanism is truthful, imagine
the following -- imagine a market where instead of choosing outcomes, we are
choosing distribution of outcomes. And each of the agent in the original
market now translate to an agent which maximizes the expected valuation with
respect to that distribution. But I want to add an additional agent into the
market who is a pure risk lover, whose utility is simply the Shannon entropy of
17
the outcome distribution scaled by two over epsilon.
Now what the VCG mechanism will do in this imaginary market is to maximize the
social welfare with respect to the original agent, plus these additional risk
lover, right. So it turns out that the social welfare in this imaginary market
exactly the free social welfare, which characterized the exponential mechanism
and, therefore, the outcome is essentially the same for this imaginary market
with VCG mechanism and exponential mechanism in the original market. And
therefore, we can translate back the payments to the original market and make
it truthful.
And that's the end of the proof of the main theorem. Now, is there any
question about main results before I move on to further discussion part?
All right. So as I promised, I will talk about what are the other models for
capturing privacy and truthfulness at the same time and why we choose our model
instead of theirs. And also, talk about some extensions of our results and
conclude with a few open problems. So what's the other options of modern
privacy? The other options, seemingly a more natural opposition is to model
privacy via a utility function. In other words, we want to kind of capture how
much information has been leaked by the mechanism and then find this utility of
the agent, which is monotone in this privacy loss due to participating in the
mechanism and then assume the agent, trying to maximize the usual notion of
utility minus this dis-utility.
This is seemingly a more natural option of capturing privacy in to the
framework of mechanism design. And this has been considered by David Xiao and
Chung, et al., in two papers in 2011. However, Nissim, et al., actually point
out this assumption is a bit problematic for the following reason. In order to
compute this privacy loss, the agent not only need to know his own utility, his
own valuation, but also need to know what other agents report.
In other words, we're in this dilemma, where assuming -- suppose we are in the
perfect information setting where agents know each other's values, in which
case they have enough information to evaluate that dis-utility. But since
we're in this perfect information setting, there's not much incentive to taking
this privacy issue into the picture, because everything is public.
And suppose agents do not know each other's evaluation, then it's funny to say
that agents actually maximize the utility which they do not have enough
18
information to evaluate.
choosing the model.
>>:
So we need to be a bit more careful in terms of
[inaudible].
>> Zhiyi Huang: Okay. So because the usual notion of privacy loss, we can
define as some kind of distance between the probability distribution, whether I
lie or I tell the truth, right. And that distribution not only affect by my
behavior, but also depend on what other agents tells the mechanism. And
therefore, in order to evaluate how much information is leaked by the
mechanism, the agent also need to know what other agents report.
And actually, Nissim et al., and Chan, et al., provide some partial solution,
and they are quite generic. So what they do is they do not assume any specific
form of this dis-utility function and simply assume there is some dis-utility,
which agents have much information to evaluate. But this dis-utility is upper
bounded by this privacy loss epsilon. And then they consider problems where we
can design strictly truthful mechanisms.
And once we do that, then we can say that as long as the mechanism is private
enough, then the gain in privacy for lying is not enough to compensate the loss
in value, valuation by lying. Because strictly truthfulness means I will lose
some fixed amount if I lie about my valuation. And therefore, as long as the
mechanism is private enough, it will be truthful even for this privacy aware
agents.
However, since we do not assume any specific form of the dis-utility function,
arguably this is the best we can do. We cannot design really specific
mechanisms which take the form of the dis-utility function into the picture and
therefore, this approach only work for very specific problems.
And the line of attack I want to propose here also is the first open question
is how about Bayesian setting. Because in Bayesian setting, people have enough
information to evaluate their privacy loss in expectation and, therefore, it
seems okay to assume specific form of dis-utility function and, therefore,
there's hope to handle more general settings even by modeling privacy into the
utility function.
So this is the kind of first open questions from the talk. And next I'll talk
about some extensions of our main result. So first of all notice that the
19
connection between exponential connection and Gibbs Measure and our main
theorem does not really use the fact that we are using social welfare as our
objective function. So in general, for any problem, the exponential mechanism
is essentially maximizing the expected performance shifted by the Shannon
entropy of the outcome distribution, scaled by 2 over epsilon, right.
And this actually gives some intuition why it works so well for many problems.
Because, in some sense, exponential mechanism is maximizing entropy, given the
performance level, and privacy in some sense is trying to maximize uncertainty
in the system. And in a hand waving manner, entropy is approximately the level
of uncertainty in the system.
However, it seems very tricky to make this hand waving kind of intuition more
precise, because differential privacy is not defined in a way using entropy,
it's defined using like how much the distance between probability distribution
conditioned on whether this agent lies or not. So I think it's interesting
open question to trying to make this connection more precise, given that the
exponential mechanism actually achieved optimal differential privacy for so
many problems. I personally believe there has to be a more intriguing
connection between these two guys.
Another extension I need to use an alternative interpretation of our main
theorem. So it's well known that maximizing entropy is the minimizing the KL
divergence to uniform distribution so I can alternatively write the correct
exponential mechanism is actually maximizing the expected performance minus the
distance, KL divergence to uniform distribution scaled by some prior vectors.
The point here is that there's nothing so special about uniform, right.
Uniform is -- what uniform do here is serving as a default distribution over
all possible outcomes. And if the problem has some nice symmetric over a
different outcomes in the feasible range, maybe uniform is a reasonable choice.
However, for some problems, maybe some outcome is obviously worse off compared
to other outcomes. And in those cases, we should put less weight on the
outcome in the default distribution, even maybe put zero weight on the default
distribution.
So due to that observation, we can derive a more generalized version for this
characterization. So the generalized exponential mechanism, which take an
outcome X proportional to again E to the -- actually, that should not be social
welfare, but arbitrary performance of that outcome scale by the privacy
20
parameter, and then also kind of bias by this prior distribution P of X can
actually be characterized as maximizing the expected social performance minus
the divergence to this default distribution P. And this generalized version
actually captures most of the extensions and of the exponential mechanism in
the previous literature. Example, sometimes people just pick a subset of
outcomes which form a nice geometric covering of the outcome space in terms of
these objective function and then use exponential mechanism only on that subset
of outcomes. That can be captured by choosing P to be a uniform distribution
over that subset of outcomes.
And what this means is that all these previous extension of mechanism are also
truthful if the objective are social welfare. So our technique is actually
compatible with all those ad hoc tricks or extensions of exponential mechanism.
>>:
[inaudible].
>> Zhiyi Huang:
Sorry?
>>:
What was it again?
>>:
[inaudible].
So why was it [indiscernible] choosing?
>> Zhiyi Huang: Oh, yeah. So first of all,
may improve the computational efficiency for
could be exponentially large. And the naive
mechanism has running time kind of linear in
choosing only a subset of outcomes
that underlying outcome space
way of implementing exponential
the size of the outcome space.
So that could potentially improve the running time. And sometimes that could
improve the privacy and objective trade-off as well.
>>:
[indiscernible].
>> Zhiyi Huang: Yeah, sometimes some of the outcomes are obviously bad. For
example, in the matching setting, right, we can also include partial matchings
into the picture, but that's obviously bad. And therefore, I want to eliminate
those matchings into the -- in the outcome space.
Now, finally, let me conclude with two more open problems. The first open
problem is, has something to do with having differentially private mechanism
that answers query online. So now let's take a step back from the mechanism
21
design literature and back to this database and answering queries kind of
scenario.
Suppose we have a database about information, say, of everybody in this
building 99, and I want to answer queries such as what's the fraction of people
in this building who has blue eyes or who has brown hairs. This is like a
typical database query releasing scenario. And quite often, this queries
actually comes online. They're not given up front and you cannot pick the
optimal way of perturbing and using randomness to answer all of them and ensure
differential privacy.
And that's exactly what exponential mechanisms do. So a challenging area in
differential privacy is how to answer this query online and still being able to
achieve optimal trade-off between the error and the differential privacy.
So what you can obviously do is add independent noise to all these queries,
right, but sometimes that's the optimal, because maybe the first query is
what's the fraction who have blue eyes and the other is who do not have blue
eyes. Then it would be stupid to use two independent perturbation for these
two queries and you really want to use one of them. And this gap can be made
arbitrarily large. They could be highly correlated in which case you only want
to add a few noises into the picture.
So there has been some work done in this literature, and this online mechanism
actually performs very well. They actually get error bound close to this
offline exponential mechanism. But it's very mysterious why they are behaving
so well. So the next open problem is towards understanding that.
So we have essentially characterized the exponential mechanism as the optimal
solution of a convex program. And there are algorithms for solving convex
programs where the constraints comes online.
The problem is can we combine those technique and this characterization to
understand and maybe even improve this online differentially private mechanism.
>>: So what does the exponential mechanism mean for these database queries?
Because in optimization we had ->> Zhiyi Huang: Right, exactly. So now the objective will be to minimize, the
out infinite error of all these answers or maybe out to errors of all this
22
answer. And we have this -- we can view this as a minimization problem now,
and the exponential mechanism can be used to solve that.
And some of this online differentially private mechanism actually achieve
similar error bound, even comparing to this exponential mechanism, which
presumably is optimal if you're given all these queries up front up to some
small off vectors. So it's very mysterious why they are able to do so well.
Okay?
>>:
And the offline [indiscernible].
>> Zhiyi Huang: So it depends on individual settings. So, for example, for
the kind of counting queries that I'm talking about, say, what's the fraction
of people have blue eyes and stuff like that, if the queries are actually kind
of random enough, the exponential mechanism is optimal. And even for arbitrary
counter query, it's conjecture that that's optimal.
I mean, it's probably not that well accepted conjecture, but we don't have the
counter example where exponential mechanism is not optimal with respect to that
kind of query. Even offline, assume we leave the computational efficiency
issue aside, then we don't have a counter example where it is not optimal.
It's symptotically optimal.
And the last open question is how about mechanism design without payments. So
what our result essentially says is that differential privacy and truthfulness
are compatible if we are allowed to use payments. On the other hand, our
approach heavily relies on the use of payments.
In particular, David Xiao showed that the exponential mechanism is not truthful
without payments for some problems. And therefore, an interesting open problem
is to consider the literature of exponential mechanism without payments and can
we still get exact truthfulness and differential privacy at the same time
within this framework.
And that's all for the talk, and thank you for coming.
>>:
So your mechanism is truthful in expectation?
>> Zhiyi Huang:
Yeah, truthful in expectation, that's right.
23
>>:
And built on truthfulness?
>> Zhiyi Huang: Yeah, that's a very good question. So actually, our results
do not imply that there's a universal truthful mechanism, and to my knowledge,
I don't know any result in that regard. But that might be an interesting thing
to explore as well.
>> Nikil Devanur:
Any other questions?
>>: So the way you define differential privacy, you say that if you define it
in terms of [indiscernible], then it would be [indiscernible]. It would be
straightforward. The only problem is the ->> Zhiyi Huang:
>>:
Right.
The way you define differential privacy is [inaudible].
>> Zhiyi Huang: Right. So one thing, also very mysterious, is that
exponential mechanism, once we do it as characterized by this convex problem,
it's defined on a point-wise manner in the sense that given any input data, we
can compute a differential mechanism, kind of maximizing this convex program,
which completely determined by the input data at that particular point.
But differential privacy is defined on kind of a comparing the distribution of
any two neighboring databases so there's a mismatch here, and it's not clear
why, with this mismatch, exponential mechanism is still being able to do so
well with respect to the definition of differential privacy.
>>:
I had a comment and a question.
>> Zhiyi Huang:
Okay.
>>: So this kind of welfare plus the [indiscernible], that looks very much
like what they're doing ->> Zhiyi Huang:
Exactly, exactly.
Yeah.
>>: And so that's actually the right thing to do for online learning, where
you just maximize whatever you've seen so far plus some variation.
24
>> Zhiyi Huang:
>>:
And you can take [indiscernible] coming from doing such an operation.
>> Zhiyi Huang:
>>:
That's right.
Yeah, exactly.
So maybe [inaudible].
>> Zhiyi Huang: Yeah, so that's an interesting direction to explore as well.
So even without this result, there has been known that there's a close
connection between learning and differential privacy, and many of the result
can be translate from one [indiscernible] to the other. So it would be
interesting to see what that means in learning.
In some sense, differential privacy is trying to prevent someone from learning
your data. So they are like dual problems in some sense. I mean, of course,
in the hand waving sense. But we also kind of look ->>:
[inaudible].
>> Zhiyi Huang: Exactly. Yeah. So yeah, it's worth looking at this online
learning literature and see what that means. Actually, Aaron has a result
showing that any no-regret learning algorithm can be translated into an online
query release mechanism, which is differentially private, and different error
bound can be obtained for different no regret learning algorithm.