>> Max Chickering: So it's my pleasure to introduce Yagil to the packed audience here. Yagil and I worked together in Live Labs in 2007, and we did a bunch of work with paid search, and so Yagil is going to talk about pricing display ads now. He's in our research lab in Israel. >>Yagil Engel: Thank you Max. I will talk about the posted prices exchange for display advertising contracts. This is joint work with Moshe Tennenholtz, with MSR Israel. Display ads are those banner ads that are shown on webpages and more and more on apps. What they let us do is effective segmentation of the target audience according to criteria that are some subset of large number of available criteria which can be demographic, they can relate to the browsing history of the user, and they can relate to the current content that the user is accessing. I will start with some preliminaries about the current display advertising market with motivation and some basic notation, and afterwards I will get to the actual description of the market that we propose, equilibrium, and how we calculate demand and supply. The atomic good in the display ad market is called an impression, which is one view of one user of one ad. Impressions are sold in roughly 2 ways: today, either through a spot market, which takes place while the user is browsing, or through contracts for future impressions; a contract specifies quantity of impressions of specific segments and specific time framing in the future which is those contracts should be delivered. Those impressions should be delivered. And the market today of contracts is very dispersed, very decentralized. Contracts are sold by bilateral negotiation between sales force of the publisher. In our case, our salespeople and the advertisers. The problem with this is that, beside efficiency, in terms of economic efficiency that is of course not, does not exist in this market at all, it is, it will also not scale to a large number of publishers. So, today we are selling our own inventory and the inventory of some large publishers that are, for which we are providing these sales as a service like Wall Street Journal. But it will not let us take a bite of them, of the larger market where lots of little publishers of apps or webpages are selling their own inventory. Looking at the literature, we see there are, I know of two kinds of works. One is talking about the pricing of contracts under the framework of the current decentralized market, these are works that came from Yahoo and Google, and there is also work that came from Yahoo, which we can say today that came from MSR that, in which a publisher is selling fixed inventory via an auction. Our goal is to introduce a market that is, at the same time continuous, so trade can occur at any time, not as [inaudible] auction event that, to which advertisers and publishers have to wait, but something that exists all the time. But on the other hand, it also brings the economic efficiency into this picture so we can achieve some global optimality of a location. And it is also two-sided. So many advertisers, many publishers can participate easily. And the approach we chose for that is that of posted prices which are based on supply and demand. So prices that are will equate supply and demand on each impression type. Besides the advantages that I just mentioned, it will also let us elicit the demand in response to prices, rather than asking advertisers to report their willingness to pay for various types of impression, we give them prices and they tell us what they want to buy in these prices. And this lets us do effective utility elicitation, even if the utility structure that we assume is pretty expressive. And the second thing it let us do is to assume, it lets us be sloppy on the incentive side by assuming that the players, the advertisers and the publishers, are price takers, which means that they are not aware or they just cannot affect the price by their behavior because there are just too many players. So one player cannot, will not have any incentive to change his behavior, not to be behave optimally because it will not change the price anyway. So it will just optimize their behavior with respect to current prices. The challenges with this approach is that first, how do we define the prices? We cannot define price every atomic good because there are just too many of them. We have thousands of criteria, so the number of impression types is exponentially in the thousands. We also don't want to take some subset of the atomic groups and price them because then it will be too specific, the chance that it will get trades on those exact goods are slim. And the third challenge is that in order to find prices that equate supply and demand we have to do some kind of depend elicitation, and we have to consider how we are going to represent supply. Technically, our approach would be to price, we’ll solve this price structure problem by pricing sets of impressions. Sets that the demand does not distinguish between various items in the set. And this will, actually the set of prices that we’ll have is small enough to be polynomial in the input that the system gets in the demand that the system gets, but will still be expressive enough so that the demand cannot be distinguished between elements of the same price of the same set. The problem, the technical problem with this is will be that those sets of impressions that have the same price overlap. And then we’ll have to think about what to do with the impressions that can fit into two sets. I hope this will become clearer later. And then we will obtain demand quantities with respect to current prices, so if we have some initial set of prices, we elicit demand based on these current prices, from that we will be able to extrapolate the demand or the utility functions as functions of any price, so we will be able to assess the demand that is going to be in any other price. And this will let us calculate demand and supply for any price, and with this we can calculate equilibrium prices. Okay. So little bit of notation. We have a set of attributes which are those segmentation criteria. For example, gender, income, sports, location, the space of impressions is the Cartesian product of the domain of these attributes. The size of this domain, like I said, is the size of A is exponentially in the thousands. So what we do when we have a size that is too large to deal with, we define an even larger space, and this is going to be this space F which includes not only all of the fully specified impressions, but also all the partially specified impressions. So, for example, high income male would be a statement in F because it is a partial assignment to the attributes. Actually F is not that larger than A, it’s just that we can think about it as adding the null value to the domain of each attribute and then taking the Cartesian domain of those new attribute domain, the Cartesian product on those on those new attribute domains. But what this means is that every statement in F refers to a set of atomic impression types. For example, a high income male will refer to all the impression types of for each, if income is high and the gender is male, and it also gives rise to a satisfaction relation where we say that f prime satisfies f double prime if all the impressions in the set f prime are also in that set f double prime, and here's an easy example, if f prime is high-income male, then it satisfies the statement high income because any a high income male is also just high income person. >>: Can you explain more of what the [inaudible]? >>Yagil Engel: Yeah. Maybe this should have been domain prime. So this is, so I defined a new domain which includes, for each attribute I defined a new domain which includes the original domain [inaudible] plus the null value. Now I take the Cartesian product of this new domain and this is it. So any, so F includes all the full specification of all of the impressions that are fully specified, but also impressions that are partially specified, like high income male I'm only specifying the value of this attribute and of this attribute, so this refers to any full specification for which G is male and I is highly. >>: Okay. >>Yagil Engel: So these were the preliminaries, now I will define how this market is going to look like, what are the market goods, the demand, supply and the equilibrium? So we have some n advertisers, a1 through a, n. And we assume that each advertiser A, J cares about some set of demand statements. Those statements in F. So one advertiser wants to advertise to high income females and to male scientists. And the second advertiser wants to advertise to high income females and to cyclists and to males. And the third advertiser was advertise to female cyclists. Then we will just take all of those statements and define them as a set of market goods. So the second market goods here would be high income females, male cyclists, cyclists, males, and female cyclists. And this is of course, linear in the size of those, of the input that the system gets from advertisers which are those demand statements. And we will, just price those goods. So the market will assign a price vector of [inaudible] that assigns a price to each good in D. Now these are somewhat nonstandard market goods. Like I mentioned, each market good is a set of atomic goods. Cyclists refers to all the impressions in which sports equals cyclists. And males refers to all the impressions which genders equals male. And the problem is that we might have male cyclists and to each one of those goods a demand or supply of male cyclists will refer. So this is something we'll have to answer. For now, we'll just say that the semantics of this good is that if with we are purchasing a male cyclist and we're getting a guarantee to get an impression of a male cyclist, if I'm just purchasing an impression in which sports is cyclists, then I don't know anything about the gender, for example. So it might be a male or female, so it's not guaranteed to be random, male or female, it's just what it is so it just provides me the guarantee on those attributes that are mentioned in the statement that I'm purchasing. We'll see more examples so it will, it might be more clear. What does it mean in terms of demand? So we have this set of market goods, which we call F1 through F Omega. This is D. We know that advertiser A, J gets utility from some subset. Some subset of these goods. Those goods that he actually mentioned, that he cares about, which are, let's say it's we’ll index them F one through F Omega J. So he has utility function that is defined over quantities of those Omega J goods. So he has utility function U, J that is defined over Q1 through Q, Omega J. So some of the market goods are not mentioned in the utility function of these advertiser A, J, which normally this is okay because we can just assume that if the market good is not referred to by consumer’s utility function, that it’s just getting zero value from that good. But this is not the case here because maybe advertiser A, J wants to purchase cyclists, for example, so his utility function refers to cyclists. But then there are other market goods which are cyclists from Michigan or high income cyclists. And they are also cyclists. So they are part of this market good that is called cyclists. So it's not, and we assume that advertiser A, J will get the same utility from getting a cyclist for which Michigan is also specified, or high income is also specified, because, merely because they are also cyclists. So it's not true that [inaudible] getting zero value from those other market goods that are not mentioned by utility function. But what we will do, we'll just refer to the economic definition of demand and we’ll just leave it at that for now. And we’ll show later how this can be easily overcome. So the demand at prices pi prime given some budget constraint that each advertiser has, so A, J has budget constraint W, J. It's just the vector of quantities of the market goods, so we'll just stick to the economic definition of demand, is a vector over the market goods, which maximizes the utility of that advertiser, the utility that is defined over this subset of goods under prices pi prime and budget constraints W, J. And will just have to, we'll just assume that the utility maximization procedure will know how to deal with those market goods that are satisfying the goods that appear in our utility function. We’ll see later how it is done. And will also define the aggregate demand as the sum over the demand of all the advertisers. On the supply side, we have an analogous problem in which supply, one supplying impression type may satisfy multiple market goods. So for example, a supply may have high income cyclists, and there is one market good which is high income and one which market good which is cyclists. So to which of these do we assign that good? Well this is actually completely falls into the, was called in economics, production market. Because this advertiser just, it can be seen as a production choice, whether I produce with what I have I can the either produce one unit of the market good that is called high income are one unit of the market good that is called cyclists. And it's actually a very simple production problem because we don't need any inputs. And now we can define the supplier pi prime as profit maximizing production plan. So we'll have S, J, pi prime will be the quantities will also be a vector of quantities over the market goods which maximizes the profit of the supplier, and what we actually do, we are also pushing here the question of how to assign the goods, we are pushing it into this problem of profit maximizing production plan. And the aggregate supply is also, is again the sum over supply of all the publishers. So now we can define market equilibrium, which is just a vector of prices that equate supply and demand. And what we can also do is easily show that an equilibrium exists in this market because we can take the market model that we just defined and easily, pretty easily reduce it to was called an Arrow-Debreu production economy, which is a model where we have consumers, we have producers, and they also, there are a few things that we do not need from this more general model, but we can easily make the reduction. So an equilibrium is going to exist under some mild assumptions like convexity and continuity of the advertiser’s utility and we’re going to have that. And in addition, there are previous results, previous literature that show that equilibrium prices can be computed in polynomial time under some additional assumptions, mainly on the utility function and we're going to make those assumptions too, later. But we’ll also have to show, in order for this result to hold, that we can calculate, we can actually find out what is the demand and what is the supply in polynomial time for any given vector of prices. So this will be the key here, to be able to calculate, to find equilibrium in polynomial time. So this would be the next section where we’ll describe those demand and supply oracles. So the, in order to be able to describe how we calculate demand, we need to first come up with some model for the utility function of advertisers. How this utility function look like. So we have this set of market goods D, we know that advertiser A, J cares about some subset of this good, which is called B, J and these are F1 for F Omega J, so we have a utility function that is defined over quantities Q1 through Q Omega J, and there is some budget W, J to these advertisers. Now the first assumption that we are going to make is what called in economics constant elasticity of substitution. In the next slide I will describe what this assumption means. In the meantime, I will just give the mathematical interpretation of that, which is this, under this assumption, the utility function must looks like that which is a summation over weighted quantities where each quantity is also taken to the power of row which is constant that is related to the elasticity of substitution. And those weights sum up to one. Now what elasticity of substitution means is, elasticity of substitution is denoted by Sigma, which is actually one over one minus row, so this is how these two parameters relate to each other. It refers to how flexible the budget allocation of, how flexible the advertiser is in his allocation of budget between impression types. So on one extreme, if Sigma equals one, meaning that the elasticity is very small, is the smallest, and this happens when row is taken to zero at the limit, what you get is the Cobb-Douglas utility function, and under Cobb-Douglas, the budget is firmly allocated between the goods, which are impression types in this case. So if we have some budget of hundred thousand dollars and we decided to spend, and given one set of prices, we are spending 20,000 on cyclists from Michigan, and 85,000 on high income females, then even if prices will change of any of these goods, it will not change this budget allocation. So if the price of cyclists from Michigan will go up, we’ll just purchase less of these, and we will not transfer, it will not cause us to transfer budget between those impression types. And this is actually completely not how advertisers behave. They substitute all the time between different impression types according to the prices. So this is very limiting and on the other extreme, the elasticity is infinite, so Sigma equals infinite, and this happens when row equals one, what we get is the linear utility function. And you can see if we replace row here by one we get linear function. And with the linear utility function, there is no notion of allocating budget to goods. So it is, we'll just get, we’ll have some activity from each impression type and we will just get the combination of, or the collection of goods that maximizes our linear utility function. So if a price of one good becomes really, really low, then we’ll just purchase lots and lots of this impression and we might not purchase other impression types at all. So we might allocate all the budget to that cheap impression type. And this is also not how advertisers work, they do care about some balance between the types of audience that they are exposed to. So what our structure means is that we cover all this spectrum between those two extremes. So we can take Sigma anywhere between one and infinite. Or row anywhere between zero and one. So the elasticity of substitution can be anywhere, can be anything as long as it is constant. So we only assume that this elasticity is the same for any pair of goods, but we do not assume a particular value. And as you can see, it is much more general than the linear utility function which is taken an assumption in most of the previous literature. At least when we are talking about such a huge space like the display advertising space. The second assumption that we take is that the goods are gross substitute. This means that we do not allow advertisers to express complementarities between different impression types. So we will never, so an advertiser will never be willing to pay for getting two types of impressions more than he will pay, more than the sum of what he will pay for each one separately. And none of this assumption will strictly hold in practice, but we believe is that it is close enough, what this gross substitute assumption let’s us, this is exactly the assumption that is required to achieve this polynomial computation in previous literature. I will also show where, why this assumption is required for computation of purposes. In terms of the mathematical structure, what this assumption means is that row is between zero and one, if row is less than zero, then what we get are complements. >>: So the constant elasticity, is that for advertiser or for [inaudible]? >>Yagil Engel: It doesn't have to be the same elasticity for everybody. So next is the actual elicitation of demand. How we figure out what is the demand of an advertiser. And let's first assume that we know the elasticity of a given advertiser, row, and we assume that the input that we get is a demand vector at some price vector. So we have prices pi prime out there and advertisers coming and purchasing Q1 through Q Omega, some quantity of each market good. So now we use what's called first order condition, meaning we take the derivative of the utility and equate it to zero, so this must hold at the utility maximizing demand. And it can, this equation can be pretty easily obtained. It's called the tangency condition of the constant elasticity utility. It tells us about this ratio between prices and the weighted goods. And we are getting, we have Omega minus 1 equations like that, so when I equals one and K equals two, and I equals two and K equals three, and so on. And we also have, we also know that the betas sum up to one, so what we have are Omega linear equations with Omega unknowns, so we can solve them and get the betas, remember that what the knowns here are the pi’s and the Q’s because we are getting the demanded price at price pi prime. Actually, we don't even have to solve the system of linear equations, we can just start with any beta and instantiate the rest of the betas and then normalize it so that those betas sum up to one. And we also assume that we have the budget now because we assume that the advertisers are exhausting their budge by stating their demand vector. So the budget is just the sum of the expenditure of the advertiser in these prices. Now, initially we do not know what the elasticity role. So in order to be able to calculate it, we need two demand factors at two different prices. And then what we can do is some binary search over the values of row between zero and one for each one, we can find the betas fast and we’ll actually have too many equations when we come to solve, to find this vector of betas, and most likely we’ll not get an exact solution, but we’ll just find the best set of betas and find the row that gives us the smallest error over the betas. And how do we get two demand vectors at two different price points? This is going to be a market which is dynamic, which is, prices are out there, prices change, and even though, so when an advertiser comes and gives us the first demand vector, we will not have a very good assessment of the utility function because we'll have to guess some elasticity, maybe based on other advertiser’s elasticity, but then once price changed and we get another demand vector, we can get it much better assessment. >>: So how important, so I assume that you can solve for row or you can fit row, if only a subset of the [inaudible]. If you could change just a few of them, then you’re going to get kind of evidence of this>>Yagil Engel: Yeah, because actually because all we need is one more equation. So even if one price changes this would be enough. But we can't, I mean, we do not want to assume that only one price change. So the demand oracle now, first we’ll show that it's actually not a problem that we have those overlaps that I showed in the beginning, so we have several market goods that may satisfy a single demand statement. We'll define this set as Delta f. So, for example, if a statement is, if an advertiser wants to advertise to cyclists from California, and then there might be two market goods, one is cyclists from California and the other is female cyclists from California, they both satisfy this demand statement. And what we want to do is just minimize, so first, remember that we are looking for the demand set of an advertiser given the prices. This is completely greedy thing. So the advertiser wants to, in equilibrium, the demand set of each advertiser must be, must give that advertiser the highest profit or the highest gain. So what we want to find is the set of impressions that satisfy the demand that provides the highest utility to that advertiser at the lowest price. What I'm saying is that it we can safely pick the cheapest impression type of this set Delta f. And we call this f hat k, so for the impression type that is the cheapest way to satisfy the demand statement F, K is F, K hat. And now we can get utility maximizing demand factor in the current prices, which means it will just do maximization of this utility function given the budget constraint, and this is also pretty easy to solve. Again, we take first of the condition, we get that equation from a previous slide, this tangency condition, and now the betas are known and the pi’s are known and what we are looking for are the Q’s, but we can find it exactly the same way that we did before instead of the last equation, the normalization will not be the sum of betas because the betas are known, but will be that budget constraint. So after we instantiate the Q’s, we’ll just normalize it into the budget constraint. >>: So I’m a little bit confused. So suppose I’m the producer and I have a female cyclist from California. In dropping the female is cheaper, aren’t I never going to drop the female? >>Yagil Engel: You will. First of all, this is the demand side. We'll talk about the production in the next slide. But you will, definitely. The answer is yes. But we just didn't get there yet. This is on the demand side. If I want to advertise to a cyclist from California, and it’s cheaper to advertise to a female cyclist from California, then I will pick female cyclists from California because it satisfies my demand. This is on the demand side. What you said is true for the production, for the publisher side. >>: So when you're picking the cheapest, it’s probably the case that female isn’t going to be cheaper because if it were, it wouldn’t have been [inaudible]. >>Yagil Engel: Right. Not in equilibrium. So this can happen on the way to the equilibrium prices. In equilibrium we will not even have to. Yeah. Now I see where your question came from. So now looking at the supply side, the problem that we have here first is to match the supply with the market goods. The market goods came from the demand. The supply can look very differently. And what we are, what we need in order to answer this question is some way to represent the supply. The first representation that we considered is the proposal that came from MSR, to represent supply by Bayesian network. I think I will skip the, this is a very technical part, for which the result is that we do not, we can't or we don't know how to use Bayesian networks in this context, so I will skip this part. What we need in order to be able to match the supply with the market goods is what we called the more explicit supply representation as opposed to the Bayesian network which is a very, a concise representation. Then with an explicit representation, a publisher will submit a set of triplets where each triplet mentions an impression type, the quantity that he has from that impression type, and the impression type is in f, right? It can be a statement. It doesn’t have to specify all the attributes. So the impression type, the quantity, and the minimum price that you want for these impression types, for this impression type, the reason I think that minimum price is needed here is that it's not true that the costs of putting an ad is completely zero, at the very least, we have to consider the disturbance to the user. And the key about this representation is that those quantities are assumed to be disjoined. So we’ll say that the total number of impressions in the inventory of that supplier is the sum over those quantities. We can see it in order to understand it we can look at that tree. We have, at each node we are branching on one of the attributes. So at the top here, we branch over the location. And now we say that for those impression types that are from Michigan, we branch over the income. So we have 100,000 impressions of high income people from Michigan, and in addition, we have 300,000 impressions of low income people from Michigan, and then in addition to that, we have 150,000 impressions for which we don't know the income, 50,000 of which, are cyclists, and another hundred thousand we do not know whether they are cyclists or not or what kind of sports they like. So here we have those hundred, actually here we have another hundred thousand for which we know they are from Ohio, but we don't know anything else. And in addition, we have 200,000 for which we don't know the state but we know that they are females. So you get the point, each leaf here has a quantity which is disjoined from the rest of the quantities. So we can just to sum up over the all the quantities that we have and this is the total number of impressions. The key, whether this is a valid supply representation or not is whether supplies can even come up with such a representation. And we believe they can even though I forgot to mention that is a work in progress, and some things are not completely nailed down yet, but we believe that this is, that this representation can be learned from data. This actually looks like a decision tree and it can be learned like a decision tree where the class that we are, that this decision tree is classifying on is the price. Because the supply actually doesn't care, doesn't need to distinguish between different goods if they have the same or roughly the same price. So we can assume that we are going through the logs of the type of impressions that the advertisers have seen in the past. And we can obtain a price for each such type of impression just by querying our own price structure, or if we don't have it yet, we can look at bids in the spot market. So we have a price for each type of impression, and we learn this decision tree and I will was told that this is because the class is continuous, it is the price, then what we're getting is actually a regression tree. So we learn this tree from this past data, and what we get is that we only distinguish between a different; we only expand the tree as long as it can provide the supplier more revenue to do so. You will not expand it more if it doesn’t have the potential to give more income. If, at some point we are, it's possible that some advertisers will not get the type of impressions that they need even though such supply exists because it has the same price as a different kind of impression. But in that case, the price of that impression type that the advertiser really wants will go up. So there will be difference in the price at some point, and then the publisher will have a reason to distinguish between these impression types. Does is make sense? This is pretty new, the idea to build this as a decision tree, so this [inaudible] requires more thought. >>: The last point was, the question I had is I understand you’re representing all of the inventory [inaudible]. And the claim is that if the supplier doesn't care about distinguishing inventory, that means that neither, there is no demands [inaudible] as well. >>Yagil Engel: Right. Well, initially, it’s possible that you have two types of different goods which have the same price. And somehow, so let's say that one advertiser wants high income people from Michigan and another advertiser wants low income people from Michigan. But they have the same price. So what the supply does, so if they have the same price, the supplier does not need to branch here. But then, prices will go up because one of these goods will be in over demand. So at least in the next round, hopefully the supplier will have reason to branch over those, over the income. >>: But do you need to know Q for these two distinctions I don't care about in order to get to the [inaudible]? >>Yagil Engel: Well, what will happen in this scenario that I described is that you will have some quantity of people from Michigan. You will not have high income people from Michigan. And then there will be a demand statement for high income people from Michigan, which there will not be enough for supply for. So the price of that, of high income people from Michigan will go up. >>: So the supply is zero for that in that case until>>Yagil Engel: From this publisher. You have lots of publishers. Maybe the supply is zero. >>: So how do you, so are you going to need to take the M into account in your split criteria? >>Yagil Engel: I don't think so. No. M will be used in the next slide. But the fact that I have different minimum price for these two impression it types, I don't think really matters because I will, in any case, I will only sell an impression if its price is at least M. But maybe it makes sense to branch so I will supply those impressions that are, but it's not M, if it's considered cost then maybe, yeah, because I will want to supply the impressions that have, that are cheaper for me. But at least in our current semantics, we don't assume that M is cost. We do not in the optimization here, the profit maximization, we just take maximum price, we do not take price minus M. If we do, which might make sense, then yes. Then we should maybe branch, instead of the price, we should branch over price minus cost. So for each, so the supply oracle, when we have this explicit representation, would be pretty simple, again, we find the set of matching market goods for each supply good, so this is Delta f here. This is reversed to what we did in the demand. Here we’re looking for all the market goods that are satisfied by our supply goods. So if we have female cyclists from California, then we might have, this satisfies market good like cyclists from California or someone from California and so on. And of these, we picked the one with the highest price, and here we have, the reason this is so simple is that our utility function is linear here. We just look for, we'll just assign the quantities of our good, of our inventory in a way that will maximize the sum of the prices that we get. And we assign a good to a market good, we assign an inventory impression to a market good at the current price of that market good, this cheapest price that we got here is at least our minimum price. Now, what's left is finding the equilibrium, and I already mentioned that there is a result, that there is an algorithm for finding the equilibrium in polynomial time. And we have the supply oracle, the polynomial supply and demand oracle, and we have the gross substitution assumption, so we are fine there. But the problem is that algorithm is relying on the ellipsoid algorithm that is known not to perform well in practice, so we are also proposing this simple [inaudible] like procedure to find equilibrium. It's not strictly polynomial, it’s pseudopolynomial. But it might be the case, and of course it has to be tested against the ellipsoid algorithm. It's possible that this one will actually perform better. We start with zero prices, then we iterate, we query over the next iteration, we query what is the demand and what is the supply in the current prices, and once we hit a good for which the demand exceeds supply, then we increase the price of that good by an epsilon. The reason that this will work is that, first of all, when we increase the price, it will never reduce the demand of another good. This is actually the formal definition of gross substitute. So we’re fine there because we assume a gross substitute. This is exactly definition. It roughly means no complements, but not exactly. This is a definition that is exactly came to help computer scientists when they try to find an algorithm for calculating equilibrium. And this exactly matches, this definition exactly matches the, this range of elasticity between of row between zero and one. The second observation is that when we increase the price of one good it will never increase the supply of another good because if a price increase will cause any change in supply, it will be for more supply of that good. Of the good whose price increased. It will never cause any publisher to move his supply, to increase supply for different good. So these together means that this prices never have to be decreased because we are only increasing prices, it will never be the case that we get, that the supply somehow, it will never be the case that the demand will suddenly, that the demand was higher than the supply and suddenly it is the other way around that we need to decrease prices. So it's possible to show that this will converge to an equilibrium. So I have one more slide which is very not technical. And this is the part of the work that we believe we can do better. But for now, this is how we propose that this market will work in practice. We divide the time into steps. Into periods. And the contracts are sold throughout the period for the next period at this period’s prices. And then prices can be recalculated at the end of every period according to the supply and demand of this past period. So we have some assumption here that the demand and supply will not change that much from one period to another, so we can use the prices that we calculated based on the period that just ended. Now this is not going to be exactly the case, there will be excess demand or excess supply. The way to handle excess demand is we’ll have to, when we calculate equilibrium prices, we will ignore some fraction of the supplies, so it's like we'll set aside some of the supply and we’ll compensate advertisers for unsatisfied demand, so if they purchase something in the current prices and we actually don't have supply of that, for that they will get some kind of compensation. And we have excess supply, then we can still set it in the spot market. But like I said, it's probably too simplistic. It is probably the case that if you want more flexible periods, not fixed periods, but this is what we have for now. To summarize, what we are trying to obtain is a market in which we have posted prices for impression types. So this will give us both continuity and efficiency. It will accommodate two- sided markets where any advertiser, any publisher can join at any time. It's completely transparent, everybody knows what are the prices, there are no, it's not, we do not need the effort in which is done today to close a contract for each advertiser separately. We propose a compact price structure that is based on the actual demand. And we show that in this market there exists an equilibrium, and it can be computed in polynomial time. We allow a rather expressive demand in comparison to previous literature. And we can elicit this demand just by observing the behavior of advertisers. We do not need to explicitly ask advertisers or go through some kind of demand elicitation process with advertisers. We still have some open points with respect to the inventory, to the representation of inventory, especially whether we can or cannot use Bayesian networks in some acknowledgments. And I'll be happy to take if there are additional questions. >>: I have another question about the supply trade. So how is it that you, suppose the goods are all the time. Right. So I have somebody wants cyclists, someone wants Michigan, someone wants income, right? So all of>>Yagil Engel: So it's not how I use the word atomic. Meaning they're all>>: Singular. One-dimensional demand. But I have demand for all of the thousand bands. And they'll have separate prices. They all have different prices. Does the tree blow up? >>Yagil Engel: That's an interesting question; I need to think about it. So you’ll have, let's say you first branch over state. And you have some states that people mention and then you have the unknown. And then, actually I don't think so because you will first branch on state, but then any of the other, the only branch that you will need to keep extending is the unknown because for, once you know that someone is from Michigan, you don't need to know anything else. This is only sold to this advertiser that wants to purchase impressions from Michigan. Now advertiser that cares about gender, that would fall into the branch under the unknown state. So you'd, so you always have the branch for each attribute where you say that the value of this attribute is not known, right? >>: So what if I have a Michigan high income impression? And I'm interested in high income? Don't I have to follow the, I know it's not unknown because I know what it is>>Yagil Engel: I see what you’re saying. You know what it is but it satisfies the other branch. The branch of unknown. So the question really is how would the decision tree or regression tree algorithm work, whether it will try to blow up everything or it will do the right thing and I can't tell that. I thought sufficiently about that in order to tell you that. It will do the right thing. But it's possible to express it with a compact tree. Whether the algorithm that will actually achieve that or not, I need to think about because- >>: But if it does the right thing, then you no longer have the property that you sum the leaves, you get the total inventory, right? So if you have a Michigan high income, it's going to, that's impressions going to appear to>>Yagil Engel: No, no. If you expanded both. Well, if you expanded both, then the Michigan high income will appear under Michigan high income. It would not appear under the unknown high income. The question is whether, but what I was saying is that you can assign those Michigan high income to a branch where you don't know the state because no one cares about Michigan high income. But I'm really not sure that this is what will, I'm just saying that you can represent it through a compact tree where you don't even expand Michigan further because no one cares whether someone from Michigan has high income or not. So it doesn't have to appear in the tree. But it's a good point. I'll have to see whether the algorithm will do the right thing or not. Okay. Thanks for coming.