>> Ben Livshits: Okay. Thank you very much for [inaudible] back with me. I'm Ben Livshits.
Welcome to another meeting of our JavaScript book club or whatever we have here. We have an
exciting visitor today, Jan Vitek, who is visiting us from Purdue University. He'll talk about a
couple of recent results in the JavaScript space.
>> Jan Vitek: All right. Okay. So thanks a lot for having me. And as Ben said, this talk will be
about a couple of recent results, or it's more like work in progress, really, that we've been -- why
a couple of results? Well, I just felt these were two interesting things, and I couldn't decide
which one I wanted to present, so I said, well, let's try to do both. So we'll see if that works out.
What's the -- and there is sort of a thread running between them, other than the fact that they're
both on JavaScript. So let's start.
So the first part of the talk is work that we did on trying to understand how the eval operator of
the JavaScript language is used.
So JavaScript has a way to turn a random piece of text into executable code. It actually has
many ways to do that. Eval is the one that we will refer to in the talk, but there are other ways to
do this.
So basically you can at runtime turn text into code.
Now, the question then is does it happen often, what do people do, how bad is it and why should
we care.
So essentially what we did was -- this is the big concern, you know, this slide summarizes it all,
and essentially what we did was the following. We took the Safari WebKit browser and we
instrumented it. And we instrumented it to record execution traces. And the execution traces
contain a lot of information including the evals that you actually do.
Then we created three datasets. The first dataset, which I'll refer to in this talk as interactive, is
obtained by taking the hundred most popular Web sites and interacting with them for 15 minutes
or so a number of times. The second dataset is taken by taking the 10,000 most popular Web
sites and just loading the first page and seeing what happens, you know, recording the JavaScript
that runs on page load.
And the third dataset was created by essentially doing what I just said, and then once you have
loaded the page randomly clicking on clickables. The different between the first -- the last two is
that you have always some behavior that doesn't happen as you load the page but only happens
when you interact with it. So by randomly clicking we sort of did our best approximation of a
sort of interaction with the Web page.
And so out of this the two numbers that should jump out are 91 and 59. So 91 is out of 10,000
Web pages, 91 percent of them use JavaScript. So if you need convincing that this is important,
well, 91 is the number that you should use. And 59 is the percentage of Web sites in our
interactive dataset that use eval.
So majority of Web sites written in JavaScript. Most of the Internet uses eval. That's sort of the
takeaway. So can't ignore it. Can't ignore it. And so one of the other results we got out of this is
we have this database now of -- you know, this is maybe 300 -- well, is it 300 megabytes of eval
strings that we collected? So large numbers of those. So what -- maybe I should modulate this a
little bit further, say, well, why should we care about evals? Well, there's two reasons, and the
reasons are performance and security. So I'll argue for both.
So let's say we want to ignore eval. What happens. Well, here's a result we got in some other
work we did with people I can't name. This is Firefox behavior and this is the speedup of
Firefox over time on the standardized SunSpider benchmark. So what this tells you is 5 of 1.5 -from 1.5 to 3.6 you have a 13-fold speedup in performance, which is wonderful. Right?
And then you look at the performance on a real Web site, like Amazon, there's a 3.1 speedup.
And, more importantly, after Firefox 3, it's basically flat lined. So what does this tell us? Well,
it tells us there's something -- probably something wrong with the benchmark.
I mean, if this is representative, and it actually is, what is wrong with the benchmark? Well, the
benchmark is static. It is, you know, loops and traditional control flow. This guy is doing the
SMM Web site, so using eval, it's using all sorts of dynamic features.
So ignoring how people actually use the features of the language is going to give you wrong
results if you care about performance. So we can't do that.
So that was half of the motivation. The other half of the motivation for looking at eval is one of
security. So here is an example of what people do with eval. So imagine I have a programmer
that wants to access the message field or some field of an object called resources. And for
generality at development time, the programmer doesn't want to fix which of the many message
field is going to be accessed.
So it's sort what the programmer has in mind is that at the end of the day we will concatenate this
string with some other string to form something like resources dot message underscore hello.
That's the programmers' model here.
And of course if what we -- this is just string concatenation. I could also do something like this.
I could say this is resources dot message underscore method and vocation files the delete all. So
what will happen here is assuming there is such a thing as a file object that has a delete all
method.
We'll evaluate the argument of the method. We'll call the delete all, it will do something, and
then we'll find out that the resource object doesn't have a message method, and we'll return
undefined.
But this is -- both of these were fine pieces of JavaScript code that through eval give you
behavior that sometimes may surprise you.
So in this case, the programmer being careful implemented a validate method which hopefully
will segregate these two cases. It will parse the input string and make sure that the input string
does not actually delete your files. Or do whatever it is you don't want it to do.
Of course this is really dumb code, because in JavaScript you could have done this much more
safely by writing this. So in JavaScript any object is a hash map, so I can just create a selector
by concatenating to field, and there's no use of eval, I'm not invoking code, not compiling,
generating anything. So it's much safer. Could have been done on [inaudible].
So bottom line is eval is one of the sources of security breaches, and it's also part of what makes
it hard to compile JavaScript efficiently.
So the questions we wanted to answer are listed in no particular order here. So we wanted to
know, well, does eval occur frequently in the real world, not in the benchmarks. Does it occur -are there many eval sites in your program or is it only one place. Is the data that we're evaling
large? I mean, are we evaluating a few bytes or are we evaluating megabytes.
Why does it matter? Well, if you're evaluating a few bytes, you can imagine maybe filtering
those bytes, maybe parsing them, maybe statically analyzing them, proving that they're not
breaking your security policy. If you're evaluating megabytes, it becomes harder.
What is the scope of this eval? How far do they reach? I mean, eval will run code and that code
will touch your data structures. Is it going to touch the data structure throughout the system or is
it very local? What is the origin of these strings that we're feeding eval and is this stuff we're
getting from the inputs or -- all of these are questions we would like to answer.
So if I come back to our big slide I had in the beginning, there's some of the data there, which I
didn't repeat, but what you can see is that's the numberer of eval codes we saw sort of between.
And the interactive dataset was about 2,000, and overall on our big random Web site was 367.
And another thing -- so yesterday we had the talk on [inaudible]. One of the things to note is that
page load doesn't figure as many evals as clicking on the Web page. So that means we have to
actually interact with the page to get it to do its stuff.
So I'll give you now more data. So the first graph I want to talk about, looking at the number of
call sites in -- of sites where eval gets invoked in your program, and this number -- and this is for
our three different workloads or datasets, and you can see that in the interactive dataset the
average is about 5. So there's typically a handful of places where you're going to be calling eval
from your program. And it doesn't go over 30.
And if you take the random -- well, the random is a dataset. You see that it goes all the way to
466 places where you're calling eval. So why does that matter? Well, if I want to validate the
security of that Web site, that application, I have to look at 466 places where I may be getting
input from -- yes.
>>: What is static location?
>> Jan Vitek: So these are static locations.
>>: [inaudible] that you identified as being used.
>> Jan Vitek: These are -- okay. So to be precise, these were locations that were exercised. So
call sites in the text that were actually reached. So this is all tracing base data. So anything that
we don't exercise we don't see here.
So in some sense what this addressed is that, you know, the 100 top Web sites maybe better
written than the average, which is not too surprising, because they tend to have few of these, and
we can imagine that maybe half of them are doing JSON parsing, which is probably safe, and
then a few other places that are doing sort of weirder stuff. Yeah.
>>: [inaudible] inspect it by hand?
>> Jan Vitek: No, we'll do that. Yes. I'm ->>: [inaudible] not hard to imagine.
>> Jan Vitek: Yeah, that could be. That very well could be.
So the next question we're wondering about was how big is the data. How big is the data that
you're feeding eval. And it's sort of interesting. So this is size and this is the percentage that it
accounts for. So essentially what this says is that if our interactive dataset at 65 -- so 65 percent
of eval are less than 64 bytes. So over -- the majority of evals are less than 64 bytes.
Majority of evals are less than 64 bytes. You see that, you know, it grows. And, for instance, for
interactive dataset, 90 percent reaches, yeah, about 500 -- above 512 bytes. So evals tends to be
small. But, on the other hand, they go all the way to 256kB.
So what is happening at that end? Well, people using eval to do -- to LoadLibrary code. So it's a
way to do lazy loading. They just say, well, if I think I'm going to need -- you know, I'm not
going to tell the engine to actually compile or parse this JavaScript until I need it, and then I call
eval, and poof, I got the code.
So what does this tell us? Well, it tells us that, you know, there's certainly for -- on the small
sizes I believe we could online analyze those. We could probably analyze those online, could
verify it, validate them. But once you start talking about kilobytes of JavaScript code, it may be
hard if you're doing any interesting static analysis to do it online or not. I mean, that depends.
So depends what -- I mean, the speeds that were reported yesterday were quite encouraging, so it
all depends.
So the next thing we looked at was what kind of data. So we know the size, but what are they
doing with that, what is the actual payload of that eval. And we categorize the evals in sort of
buckets. And the first product that we see a lot is evaling JSON. So this is a JSON string, so this
is -- the braces are really telling us when you eval this, create an object that has a field X with
value 2. So this is this evaluation. That's a big bucket.
Then another kind of is evaling stuff that kind of looks like JSON but has small, you know, sort
of -- it's not exactly per the JSON spec, but it kind of creates objects too. We call this relaxed
JSON.
And the third big bucket is JSON assigned to a field. So there's a lot of that. So you assign that
to either a field of an object or a local variable.
The fourth big bucket is calls that have only a method -- a field identifier. That's the whole code.
And that's a big bucket. I'll tell you how big the bucket is later. But the only thing it's doing is
this. This is definitely a case of -- in many cases it's a case of ignorance, because you could just
rewrite this without the eval, put a square brace around this, and use F as a string and -- okay.
Okay. So that's the next.
So here's a weird bucket that we found a lot of and we have no understanding or explanation
why. So people seem to like to write type tests in evals. So the whole body of the eval is a type
of expression that just checks if the type of sum -- here in this case it's a constructed field -- is
undefined.
We have no good reason -- no good understanding why people would do this. We looked
through the Internet to see if somebody at some point had said this is a good idea, and we
couldn't find that. And it's not even the right way to do this, because you could say, you know,
this field is in -- is -- well, you could say -- I mean, you could use the in -- there's a keyword "in"
in JavaScript that let's you test if something is present in an object.
Next bucket, which is also weird, is this. So this is actually occurring enough that we found it
worthy to mention. And it is a piece of code that tries, that throws -- I mean, so the order of
evaluation is assign 14 to global variable V. Okay. Throw an exception, catch and discard it,
move on. Okay. So this whole thing could be replaced with V equals 14. Or eval V of 14,
rather.
So there is one tiny thing is when you're evaling, you're in global scope. So there is a scoping
issue. But other than that, the try and catch seems slightly unnecessary.
Then a big bin is calls. So people have an eval that is doing only one thing, which is calling a
method passing an argument. And then there's these library loads. So then when -- this is cases
where you're loading a bunch of functions. And finally there's a bin for other where you're doing
a combination of those and we just couldn't categorize it. Yes.
>>: [inaudible] like completely useless and how many of them are -- like the original validate
example that you had.
>> Jan Vitek: So the question is how many of the argument -- so all of the arguments I gave
here were extreme constants, just for the example. Right? And the question is how many times
is this exactly what happens. And we'll see that in a bit, some number. And I have a graph for
that.
>>: [inaudible]
>> Jan Vitek: Right. So we're getting there. So there's more data, and I'm sort of simplifying
some -- just for due to time, but basically the call bin, the one where you do a method call, is the
highest. JSON comes next. Then the bins with other stuff. And then it sort of decreases
slightly. So this is -- so, you know, we're selling -- calls are 100,000 occurrences, JSON are
about 800 -- 80,000 occurrences, and it goes down.
Oh, I didn't say that we found a number which were either empty strings or, you know,
actually -- either white spaces or empty strings.
All right. So that is the dynamic. So these numbers are the number of strings fed to eval in our
random dataset that had this [inaudible] dynamic.
>>: [inaudible] so the empty string might have been the result of something --
>> Jan Vitek: Absolutely. So it may be a code -- there are many reasons why you may have that
and you just -- well, you don't care, right, it's an empty string, so it doesn't matter. But it just
looks ugly and silly when you're looking at them from the outside.
So the next thing we looked at was we tried to understand how -- yeah.
>>: [inaudible] did you look at all the coverage, like how many different sites did the typo thing,
for example? What fraction of the [inaudible].
>> Jan Vitek: So we have that somewhere. I don't have it offhand. And there's another thing
that we have, which is we looked at how often the same call site, you know, emits the same kind
of, you know, instructions. Like, for instance, a call site, does it always do JSON? And we have
data that sort of suggests most call sites are consistent. But there's some number of them, and I
forget the exact one, where you can change. Sometimes it's one bin sometimes in the other.
But it is interesting that a lot of these call sites are consistent, so maybe what you could do is
once you've seen it do one pattern once, you can sort of specialize for whatever analysis or code
generation you're doing for that pattern.
Another thing to observe is if we got rid of JSON, if -- by that I mean if rather than evaling it we
were parsing it, you know, not such hard thing to do, we would reduce, you know, somewhat the
number of stuff that has to be taken care of.
So we can get rid of this because we can parse it. Member access, you shouldn't write them in
eval, so we should be able to get rid of that. Type of -- I can't figure out why. So the ones -- you
know, there's a number of these categories you could get rid of, and then it could reduce the
magnitude of the problem, not eliminate it unfortunately. So the next thing we did -- yeah?
>>: [inaudible] any of these kind of strange usages where the result of maybe [inaudible]
generate code or were they actually handwritten? Can you kind of get an idea as to ->> Jan Vitek: So, I mean, to some extent we could. We didn't. But we could because the -- you
could look at the code and try to figure it out. I mean, clearly they're out of those 10,000 pages,
there's a bunch that are [inaudible]. But yeah.
So the next thing we wanted to know was how far reaching the code that you're running in eval is
going to -- you know, how far is it going to reach in your program. And so we looked at this,
and the way to read this is 10 percent of the dynamic eval we're running are reading and writing
only to local scope.
So 10 percent of the evals are completely local. So let's say JSON would probably be a case of
stuff that should be completely local. It's creating a new object and it's storing to the fields of
that new object. So in some sense it's safe from a security standpoint. It's neither going to
endanger secrecy or integrate it.
Then there's like 60 percent that is writing to local and reading from the global scope. That
means, well, at least one bit of global data was red during the eval. But the nice thing is it's only
writing to local. So that 60 percent is not going to endanger integrity but it may get a secret out
of you. And then there is about 9 percent that is writing and reading global values. And I would
be probably a little bit leery of that. Right? That does not look good.
So the next thing we did is we took our bins and we ->>: [inaudible] stand for?
>> Jan Vitek: Write and read.
>>: [Inaudible] power.
>> Jan Vitek: Oh, that's a typo. That's a typo.
>>: What's the N?
>> Jan Vitek: You don't want to know. Okay. I'll tell you. So what we were trying do is say L
means you're the -- reading local variables in your call frame and M means reading variables in
sort of the call stack. But it's not that important.
So the next thing we did was we took our bins and we tried to see where do the strings that you
feed to eval come from per bin. So we have this categorization. We said they're Ajax, strings
that come through the Ajax protocol. There's strings that come either from calls to a native
function or through -- or that you grab from your document. There's strings that are constructed
by concatenating two or more constant strings. And there are strings that are constant, you
know, hard wire, which was the question asked earlier, in the program.
And so if you -- you categorize the -- and we did that for all bins. And how do we do that?
Well, we -- essentially we taint, so we change the interpreter to taint these strings at creation, and
then when you concatenate, so strings may start as a constant, then you concatenate it, then you
flag that string as tainted and you propagate that through the execution. So it's a very simple
form of taint analysis.
So what should we say? Well, for one thing, there's very little Ajax anywhere. Ajax shows up
here in the JSON bin, which makes sense. It shows up in the white space bin. I guess you're
sending an empty message, most likely, in the relaxed JSON. And that's about it, all of the other
places. And which is good. Right? That's probably what you would like.
If you look at a constant string, well, they account for -- they show up in JSON, so there's some
amount of JSON that is constructed as of constant, but they account for 50 percent of cases
where you're accessing a single variable. So the code had eval X in it. And it goes for 80
percent of method calls in your code. Again, the call had -- and it counts for almost all the try
catch. So you're not constructing these things, which is good probably.
So all right. So that's pretty much where we stand. So essentially we have, and I'll conclude for
this part, the thing -- well, we've looked at the behavior evals, and it occurs. It occurs a lot. It
can be big. There is encouraging bits of data suggesting that if your site is well written, it is sort
of more disciplined. And we're still working on trying to do more.
We want to get all the forms of eval. You know, there are other ways to actually get eval to
happen, so we want to know what they are and how often they occur. And we want to get better
provenance information so we can tell you better which is user input and stuff like that. Yes.
>>: So is one of the conclusions from this that from a performance standpoint one might want to
actually have a preprocessing step that we take -- where you take a page, run it through, do eval
for whatever you need to, and then actually execute and JavaScript using [inaudible] SunSpider,
right?
>> Jan Vitek: You said it. Definitely. I mean, if you look at -- look at the number of these that
are static once you get a Web page. That's not a silly thing do.
>>: With the analysis that you did, would that attract a renamed eval where someone will assign
[inaudible]?
>>: Yes.
>> Jan Vitek: Yes. Because we are in the browser. So as long as you end up in the eval, we
would have caught it.
>>: Did you look at all the documents I write in terms of [inaudible]?
>> Jan Vitek: So that's the other forms of eval, and we'll have that data soon. So it's coming.
>>: So do you have a sense of the distribution of the other?
>> Jan Vitek: Not yet. So we're running the experiment now. So we just -- yeah.
>>: In the case of [inaudible] as a preprocessing step, just [inaudible] what you're left with is
some legitimate forms of it [inaudible] used primarily [inaudible] still have to worry about
[inaudible] what's your overall feeling about where this is going? Do you feel like it will
basically eliminate a bunch of these things and then [inaudible] maybe through some [inaudible]
recognized primitives or will there be more?
>> Jan Vitek: So I guess -- so the question is if we remove JSON, if we remove the trivial cases,
how many things are people doing with eval that they couldn't do otherwise? And I don't have a
good feeling for that. Dynamic loading shouldn't be eval. I mean, we understand dynamic
loading and we don't need to give the full power of you introspecting over the environment to
load dynamic code. You just -- you know, you load a bundle of functions. That's one thing. But
you don't have compute while you're doing this.
So there is likely to be some call where you need it, and I'm not sure what will be left. But we
could whittle it down. We could whittle it down a lot, and that would make life, you know,
probably easier.
>>: So do you get a sense of the use of these idioms and how it's split between the actual
application unit and the libraries that they're loading, like J queries and stuff like that?
>> Jan Vitek: No, I don't have ->>: Because maybe the reason you're seeing like that typo thing is because it's in a library
everybody uses.
>> Jan Vitek: Yeah, that could be. Yeah.
>>: So [inaudible] I think our like estimate for the amount of evals are sort of frivolous, was
about 8 percent, something like that.
>> Jan Vitek: Yeah. Yeah. So we hope we could get rid of 80 percent of evals, and that leaves
20 percent you have to look at, yes.
>>: [inaudible] eval with less version of say J query [inaudible] very complex language
[inaudible] but perhaps you could take a site, Facebook, and try to have an eval-less version
[inaudible] free version of something.
>> Jan Vitek: And see what happens [inaudible]. All right. So I'll move to the second part,
which I'll go through faster, because we've takable more time than I've planned.
So the second part is another piece of work. It's looking at access control for JavaScript. And
it's kind of motivated by eval. So here's the code I want you to understand. And, well, this is the
MySpace worm. And if we had to focus on one -- on two lines, give me two lines, two lines, two
lines, on two lines you need this. This is the part you really need to understand.
So this is the worm, the MySpace worm. So what was going on. So somebody wanted to attack
MySpace. And in order to do that, so MySpace used to be a popular Web site, you know, maybe
familiar. And they were using -- so the attacker wanted to use JavaScript and to attack it. And
he found that, well, MySpace was filtering, so they were looking at the data and they were sort of
doing pattern matching and they would throw away JavaScript.
But the reason I said it's two line, there's actually a line break at the end here. And the filter that
MySpace was using considered the word Java and script and two words, which makes sense, you
know, it's Java, new line, script.
Now, of course the browser built by some company considered -- there were actually several
browsers built by a number of companies, considered Java the new line to be nonsignificant
character. So for the browser's perspective, Java and script because JavaScript. That's part of the
attack.
So you could -- they manage to get JavaScript recognized. And then you could have code. But
the problem is this was already quoted twice. So they ran out of quotes. So they use eval to -because they needed quotes for doing stuff inside the code. So they used eval to grab another
part of the HTML page, this expression here, and run that code. So eval was needed. Browser
differences were needed.
Another example from the same worm is in order for the attack to work, they had to send the
documents, they had to send the Web page. MySpace was looking for the word inner HTML
and not looking for the two string inner and [inaudible] HTML that could be computationally
added. And that was the attack.
So basically what's the problem? The problem is we're running untrusted code in our browser,
and we want to secure it. And one way to secure it is to try to filter the code you're given. But
as I've shown, the presence of eval makes it very hard to find static methods to filter that code
because the code can be constructed on the fly and life becomes hard.
So this project that I will try to talk to you about attempts to provide security for, you know, one
part of this problem. And here's my example. So this is a page, a Web page from some news
agency that you may choose to trust or not. But assumably, if you're going there, you have some
expectation that they're not going to attack you, so there is a trust relationship between you and
them.
But on this Web page there is these two bits here which don't belong to CNN. They're ads. And
the code that these ads is running is coming from a -- another code provider, which actually
sometimes offloads even to another provider. So the code comes from God knows where. Well,
actually for this page, we were loading places from all these URLs, some of them make sense,
others don't.
So the problem we're looking at is you're running a piece of code that is composed from different
origins. There is one source, CNN, which is the trusted source, right, at least in news, and there
are other places which may not be as trusted. So the question we were trying to solve is can we
secure, can you provide security from those ads, those little bits of code that you get that get
merged into your content.
And it's really an access control problem, right? Because what you want to do is control what
the JavaScript that is loaded through these ads can do unrestricted. And there have been a
number of attacks that have used ads to endanger the security of your server.
So the project we call JSLocker, because the Internet is minefield. And we -- our goal was to
protect users from untrusted ads with the constraints that we don't believe we can change the
code that is out there, so it has to run unchanged code, but we allow ourself, you know, the
freedom to mess with the browser. So we sort of say sure, we can change the browser, we can
change the code that's out there, what can we do, how far can we get.
So in terms of problem statement, you can think of the problem as really being the -- coming
from the fact that a Web application is really code that comes from multiple sources. Each of
these sources is a principal, and they're manipulating objects. They're manipulating resources,
your shared memory locations. And what we want do is control how they manipulate those
resources. So it's an access control problem.
And in an access control setting, you have two questions to ask: What is your access control
mechanism, how are we going to monitor access to data, and what are the policies, how do you
define who's allowed do what. So we propose a solution for both of those.
So to simplify things, let's assume there are only two principals. There's the Web page, the host
page, which we trust, say CNN, and then there is the untrusted codes, the ads. For simplicity
we'll just take two principals. And your resources [inaudible] your objects, your functions that
you're going to call on them. And what we propose as a mechanism is what I'll explain,
something we call delimited histories and revocation. And as policies we're going to have
predicates that can be written by users over these histories and revocations.
So how does this work? Basically you're faced with access control decisions of this kind. So
you're running in JavaScript and you get to a point in the code where you see somebody's trying
to store a value into the field of let's say the global object, let's say. And this is an operation that
is done by the untrusted part of the system.
So the question is do we allow this or do we reject this. Okay. Well, if I see only this, it's very
hard. I don't have much context. So what we're proposing is to say, well, we're going to give
you a window on the computation, a history, and you can actually look what happened before
that very instant.
So at time T minus 1, you see the code was incrementing a counter. Yeah, that still looks pretty
safe. And if you look at instant T minus 2, you'll see that the code read some user data and
stored it into NM. So what this is doing is it's reading something, say, some secret, and then
putting it in a globally visible object.
So now you have enough context, if you see all of this, to make an access control decision.
So the idea was of JSLocker is to say, well, we're going to record histories of computation
performed by the untrusted code, and then before -- but then at some particular point in time,
we're going to run a predicate over this history that will tell us this is valid or this is a security
violation.
So here's an example of a predicate which is the send after read policy, and the send after read
policy says when you want to check a history, look at the history in order, see if there was a read
of data that didn't belong to the untrusted principal. And if there was a read, remember that, and
see if there is an attempt communicate over the Internet.
And in the case a read proceeded it, stop. So very simple. If you've read a secret, we're not
going to let you talk on the Internet. Simple policy.
So the question is what does it mean to stop? Well, we could just stop the computation. With
what we propose is a mechanism closer to transactions, which is to undo all the effect performed
by the history.
So the reason is, well, you know, if I just stop here, now I have part of my state that is
inconsistent. I've incremented some stuff. And I would like to clean that up. Or maybe what I
did is if I stop here, well, I still have the opportunity to register a function and I'm polluting the
global name space. So want to be able to clean what the -- if we detect somebody is untrusted,
want to undo its effects. So the ->>: [inaudible]
>> Jan Vitek: Sorry?
>>: [inaudible]
>> Jan Vitek: Well, once we've ->>: [inaudible]
>> Jan Vitek: Right. We can block it, but we want also to undo what it has already done.
Because we already executed. We're not -- this is not static analysis. This is runtime. So we
have a history of what you have done in the past, and now we are going to undo that history.
So that's basically the thing. So we have a mechanism for in the browser recording histories of
untrusted computation, and at particle points of time we'll validate that history and either we
accept it and we say you can keep on, or we revoke it and we'll undo you.
And a couple of odds and ends. We have to define what is a history start and what is a history
end. When do we start recording.
So what we do is we tag every object by its source. So if you download something from a an ad,
it gets tagged as untrusted. And whenever you call a method of an untrusted object, we start the
history. And when you leave that method, we end the history. So we know when you're
executing untrusted code. And whenever you create new objects, we're going to tag them as
well.
And at the end of a history we're going to run the validation code. And if it succeeds, then you're
fine. If not, we'll undo what you've done during that piece of execution.
There are cases where in the middle of a history you may be trying to communicate, you try to
send something on the Internet. So what happens then? Well, we support a form of eager
validation. We know that the history is not over, but we're going to let you decide whether you
want to kill the computation right now or not.
Eval. So this is where we come back to eval. So one of the -- the danger is eval could be used to
get trusted code to run a string that was created by untrusted code. And that would defeat our
mechanism. So what we say is if you're running an eval, we're considered as untrusted. And all
evals are run with history recording and potential unrolling.
>>: So you're basically creating like a [inaudible] transaction log [inaudible] and rolling back to
different points. But doesn't that require that you do for a whole page [inaudible] those global
locates be operating on [inaudible] page? Not just frame itself.
>> Jan Vitek: It is done -- so we're recording what happens for the whole page, but the recording
only needs to -- so JavaScript is sequential, right? So the only -- it's only when you start
executing an untrusted object, an object that was loaded or created ->>: [inaudible] the untrusted object [inaudible] something's done. And then returned back.
>> Jan Vitek: Right. So if it's done by the trusted page, we assume it's trusted. So that's part -there's couple of things we don't do, and that's one of them.
So we implemented this in WebKit in Safari browser. It's about 5,000 lines of changes spread
over 29 files. I lied because I said -- I showed you a nice policy. The real policies are written in
C++ and they're ugly.
We validated this on real Web site. So our validation was done as follows. We started -- we
took 50 Web sites and we ran them with the empty policy, so the policy that always says yes.
And, as expected, 50 Web sites ran fine with the policy that always says yes. Then we ran the
Web sites with two other policies, one which says you're not allowed to update global variables.
You can create new ones, but you can't update a variable that somebody else created. So that's
the add-only policy.
And what we show here is that 36 Web sites ran just fine with that. There were eight Web sites
where we worked as a very good ad blocker, so the ad just disappeared, which from a user
standpoint is great; from the Web sites probably not acceptable, because they rely on these ads
being displayed for their benefit. Five Web sites were kind of partially working and one
completely died.
So a more interesting policy I think is the send after read that I described. And there what is
interesting is 42 of our Web sites just worked fine with this, so that means the ad never read a
global value -- a global piece of information and then communicated on the Internet. In seven
cases we saw ads that -- seven cases we saw ads that were trying to send data that they had read.
And in one case -- so we wrote an ad blocker in those cases. And in one case the behavior of the
Web sites was not working properly. Yes.
>>: Was it data that they actually read that they tried to send back, or just the fact that they
had ->> Jan Vitek: This is just a fact that they had. So a better -- so you sort of imagine since we're
keeping track of the computational history, we could do better than that. This is just the
simple-minded if you've read, we're not going to let you write.
>>: So if [inaudible] Google Analytics, what it does, it seems like it's included in many, many
Web sites. I'm curious if it violates this policy.
>> Jan Vitek: So there's some things that we -- so I don't remember about Google Analytics, but
I remember there is some cases where there were things that were -- that we had to just white list
and say, yeah, sure, we're not -- we're loading this and -- because part -- part of it comes from the
source, where you got the data. And some Web sites use other servers for their source and we
would treat their own code as untrusted, so there's some issues there.
Lastly, I want to say performance-wise, performance-wise the overheads of doing this are mostly
in the 11 and 13 percent range. And of course there is one site here which runs a 106 slower, is
two times slower. And why? Well, because most of it is run through eval. Okay?
So the only thing it's doing, it's creating code and evaling it. And since -- and this is trusted code
run by the trusted part of the Web site. But, as I said, our policy is all evals are considered evil
and we -- that's why we get the massive slowdown.
So -- yeah.
>>: So it seems like -- so this overhead, what policy?
>> Jan Vitek: Yeah, right. What policy was it? This was probably the empty policy we were
looking at the logging and just overhead of keeping track of the histories. I don't think it was
evaluating the actual policies themselves. Yeah.
>>: I think my -- going back to the previous slide, my first reaction is that if you compare this to
[inaudible] these are fairly low overheads [inaudible] if you compare this to browser based, it's
limitation and enforcement of policies, this is quite high because we've seen overheads and ad
space [inaudible] to 1 percent.
>> Jan Vitek: You get a lot -- so I think it is high for -- there's potentially two things to say. One
is we keep a lot of data, so it's doing quite a lot of actual recording. Secondly, we haven't
optimized it at all.
So I -- I think all I want to say with this is Web sites that are not eval based entirely don't slow
down to a crawl. And if you're doing only evals -- so this is really the worst pathological case
where we record everything and you have a 2X slowdown. So that gives you a range for naive
implementation. This is what you get.
Performance was not what we were after, so I don't want to make too much of a point of these
numbers.
So to conclude the talk, let's say there's different ways to do security for JavaScript. You can try
to filter the text. You can try to rewrite, or you can try to integrate yourself in the browser. And
what we're sort of arguing is the integration in the browser is the safest because that is where you
have all the information, you can't be tricked by parsing errors, you really are in control of what's
going.
You believe that delimited histories are quite nice and powerful, and there's more things that we
believe we could do. For instance, if we combine them with some static analysis, we believe we
could get information flow out of them. And revocation is an interesting mechanism because it
allows us to speculate over trust. We can say, well, we'll trust you, and if it turns out that we
shouldn't have, we can undo what we -- we can take that back.
So this work was done by these fine gentlemen. So Brian, who was at Purdue until recently,
Christian Hammer and Gregor Richards, which some of you have known.
And that's what I had to say today.
>> Ben Livshits: [inaudible]
[applause]
>>: I'm curious, have you looked at all of what the information that's going back was? It seems
like this would be a nice mechanism that you could just make available to people in general to
find out what kind of privacy stuff is leaking from their Web sites. But you don't have to
provoke ->> Jan Vitek: Yeah, you could just sort of say, oh, now that we know that you communicated,
we can look at the history and trace that info. Because we have it all provenance encapsulated
there. No, we haven't done that, but it's a ->>: [inaudible] even if you had to white list these things, I mean, it'd be useful to see what was
going through there, right?
>> Jan Vitek: A lot -- I suspect in our evaluation a lot of the broken cases are false conflicts,
right? You read something, but that's not what you send out. And we just didn't carefully
enough analyze this to check that there wasn't [inaudible]. So the false -- our false positive rate
is probably unacceptable at this level in some sense.
>>: One thing that Microsoft has had to deal with over the years is the degree to which any
leaked information -- so there was a big stink about the Media Player where it was sending back
two different pieces of information to two different sort of sources, that when you combine the
two, you can figure out what the person's movie viewing history was. And it was a mistake.
There was no intention to sort of correlate to, but somebody, some privacy guy, figured this out
and it caused a lot of [inaudible] and we changed our policy, which is probably good. But, you
know, one thing about the browsers is that there's all this JavaScript running on every Web site
you visit that's collecting all this data, and people now just have no clue about how much
information they're giving out, you know, Web source. And I think ->> Jan Vitek: The lack of isolation in JavaScript means that, you know, even an ad can
potentially get at data that it shouldn't.
So you can trust CNN to some extent. All of the other sites, you've never heard of them, right?
>>: Maybe that space, if you really want [inaudible] is to look at the quantitative information
from [inaudible] that's been out for quite a while. [inaudible] build a utility that would say, well,
this site is leaking 27 bits whereas this site is only leaking two bits. And that could be used as a
little, you know, browser plug that would sort of say, well, you know, maybe you take that into
consideration.
>> Jan Vitek: Well, yeah, yeah. Okay. So that's exactly where that started. So the NSF grant
that funded this work was about can we find -- information -- you know, information flow
doesn't work. It's useless. We know that. You can't program in it because even checking my
password is leaking a bit.
But it's only leaking a bit, right? So we should be able to -- so that was the idea. And, you
know, this is where we are right now. So, as I said, work in progress. Yes.
>>: Definitely valuable looking at the user input and whether that's then being translated
[inaudible].
>> Ben Livshits: Okay. All right. Thank you.
[applause]