>> Mark Marron: So I think we’re ready to get started here, so it’s my pleasure to introduce John Vilk
today. I think a lot of you know him; he’s kind of becoming a regular here, done internships the last two
summers now, so it’s good. He—you know—he’s also a PhD student at UMass Amherst with Emery
Berger, where he’s done the work that he’s gonna be talking about today. It was presented at PLDI last
year; it was a SIGPLAN Highlight Award there. So it’s really exciting; I think it’s pretty cool; and I’ll just
give it to John to tell you exactly what it is.
>> John Vilk: Thanks Mark. So this work is joint work with Emery, so let’s just get right to it. So as many
of you are aware, there are a number of programming languages out there. So for example, we have
Java, Python, and Ruby. And it turns out that there are actually a lot of useful programs and libraries
that are already written in these languages, but unfortunately, there’s only one language that runs
across all of the browsers, and that’s JavaScript. Wouldn’t it be nice if we could just take our existing,
well-tested programs and libraries and run them in the browser with no modifications? So to motivate
this problem, I’ll be discussing a nonexistent text chat client written in Java, as it illustrates the
challenges of bringing conventional languages and their programs into the browser. So our theoretical,
nonexistent chat client lets you connect to multiple chat servers at once; it logs your chats as text files;
and encapsulates each server connection in its own thread. What we would like to do is take this Java
chat client and run it in the browser with no modifications whatsoever. How might we go about doing
this? Now, you might reasonably expect that you could simply take the Java source code of the client
and just recompile it somehow into JavaScript, but unfortunately, this turns out to be infeasible for a
number of reasons. So for the first part of my talk, I’m going to discuss why this is the case.
The first problem has to do with the nature of Java itself. Java is a dynamic language; performing aheadof-time compilation on Java code will rule out dynamic class-loading and many parts of the Java
reflection API. The second problem is a bit more fundamental. Our chat client uses multiple threads, all
inhabiting the same address space, but JavaScript does not support threads; instead, it supports
workers. These workers do not inhabit the same address space, and they can’t share memory with each
other. So instead, they communicate by sending messages. So we can’t simply map threads directly to
workers, which poses a challenge to bringing our chat client to the web. Our chat client uses TCP
sockets to communicate with the chat servers. The browser doesn’t have support for TCP sockets;
instead, it supports WebSockets, which is a protocol implemented on top of TCP. As a result, our chat
client will not be able to communicate with the chat servers, which expect a regular TCP connection. To
log chats, the chat client writes text files into the file system, as you would expect, but the browser
doesn’t have a file system. Instead, it has a variety of other mechanisms for storing data, which are
unevenly supported across all the different browsers, and they also have a variety of restrictions. One
of these does use a file system abstraction, but it is asynchronous and does not support blocking
operations, and I’ll explain the importance of this little detail in a little bit. Unbeknownst to our chat
program, under the covers, the Java chat… Java class library—which has things like the string class and
all the other things that Java clients… Java programs take advantage of—it performs a number of unsafe
operations using explicit memory allocation, requiring an unmanaged heap, but the browser doesn’t
offer a mechanism for explicit memory allocation. And I want to note here, carefully, that I’m not
complaining about this; this is actually good, but I’m just stating a fact: it’s not there and it’s… something
needs to be there. So the chat client requires all these components in order to function properly. So if
we want to run our chat client in the browser, we need to somehow emulate all of these resources in
the browser. But there’s another challenge to doing this, and that’s browser diversity.
We’re not just trying to bring our chat client into one browser—such as Google Chrome, or Opera, or
something like that—we’re actually trying to bring it into multiple browsers, which unevenly implement
the various web standards we need to use in order to implement all of these missing abstractions. But
there also isn’t just one version of every browser out there; there’s actually multiple major versions in
use at any given time. And there are additional differences, too—there’s differences between desktop,
tablet, mobile, et cetera—and this becomes a major obstacle to bringing the resources that the chat
client needs into the browser, and I can illustrate this with an example. Also, I want to pause for a
second: if anyone has any questions along the way, feel free to interrupt me and let me know, ‘cause I’ll
be happy to answer them. So let’s ignore the version differences to simplify this discussion a bit, and
we’ll focus on the latest version of each browser. As I mentioned earlier, our chat client needs to write
log files into a… some sort of file system, but… and there are a few persistent storage mechanisms
available in the browser that we could use to emulate the file system. First, all browsers implement a
simple key-value store that gives programs a paltry five megabytes of storage. Safari allows web pages
to store larger amounts of data into a SQL database. Chrome and Opera offer the same functionality,
but they do not persist a database; it is temporary storage. Firefox and Internet Explorer allow web
pages to persist data in an object database. And once again, Chrome and Opera, they put it into
temporary storage; they don’t persist it; and Internet Explorer’s version is not fully spec-compliant. And
then finally, we have the HTML5 file system, which is primarily used in WebKit browsers, so only Chrome
and Opera support this interface. And so the takeaway is this: if our chat client wants to persist more
than five megabytes of logs in the browser and cares about browser compatibility, then it needs to
support three completely different storage interfaces. SQL database, an object database… Yep?
>>: So I thought index.db… is persistent among all browsers now, right?
>> John Vilk: So it’s in, like, temporary storage, so it’s shared between… there’s like a pool of storage
that’s shared among all, like, web pages, and then if you exceed… if that, like, pool fills up, it’ll just knock
out the last thing in the storage. So it’s not totally persistent.
>>: That’s horrible.
>> John Vilk: Yep?
>>: Safari’s not WebKit-based.
>> John Vilk: Oh, sorry, I meant to say Blink-based, yeah. For some reason, even though it’s in the
WebKit sources, Safari didn’t have it. It may have it now, but I haven’t checked recently, but I suspect it
has the same restrictions as the other ones.
Okay. So yeah, this is painful, ‘cause these are all completely different interfaces. So that’s one
challenge; another challenge is the event-driven runtime model in JavaScript, so this results in pervasive
asynchrony in all JavaScript code. So in particular, JavaScript I/O requires callbacks, which proves to be
painful to bringing conventional languages into the browser. Let’s walk through an example, just to
make this very clear. So our chat client needs to periodically send a ping command to the chat server to
ensure its connection is alive, and waits for a pong response. We’ll walk through the code in Java, and
then we’ll try to directly translate it to JavaScript. Let’s say that somewhere in our main method, the
chat client fires off of a… fires off a ping. So the sendPing function might look something like this, but I
want to note that I’m simplifying this for presentation purposes; I realize this is not totally valid code.
On the first line, the chat client writes the ping command on the socket to the server, then it issues a
blocking read to wait for a response. Finally, once the pong comes back, it returns the data that it read
off the wire, and our chat client continues on its merry way. We’ll try to do this… oh, yep?
>>: A question on your Java code: the way you present it makes it read like it’s completely synchronous
and will block until the response arrives.
>> John Vilk: Yes.
>>: Is that what your client does, or is there another subtlety linked to the presence of a UI thread
running in parallel or something?
>> John Vilk: So this is what, like, this theoretical nonexistent chat client does, and this is actually what a
lot of Java programs do; they do completely synchronous operations on sockets. What was your other
question?
>>: So the Java client, if it blocks—like if the network is stuck or something—that means the UI—the
user interface—is going to be frozen for the duration of the read call on the socket, right?
>> John Vilk: In JavaScript, if you were to naïvely map it into JavaScript, that is the case; the way that we
deal with it in Doppio means that that does not happen. I’ll describe how that works in a little bit, but
the way that it works in Doppio is that we’re able to have the same effect as a synchronous call from… in
the… from the Java perspective. But from the JavaScript perspective, that call is actually asynchronous
and causes things to wind down, so the GUI does have a chance to repaint, and everything stays
responsive. And that… but you raise a good point. That’s actually the key issue I’m trying to hammer
home here, ‘cause a lot of people aren’t familiar with JavaScript or the browser, and they might be very
surprised to see that some synchronous I/O would actually block the GUI and cause things not to render
properly.
So let’s see if we can translate this into JavaScript somehow. So like before, the main function fires off a
call to sendPing, and sendPing sends a ping command on the WebSocket like before. But then we need
to assign a callback function on the socket object, which the browser will call once it receives data from
the chat server, but here, we have a problem. The main function expects that sendPing will return the
response from the server, but sendPing does not have the response yet. It needs to wait for the
browser to call the callback we just registered. What do we have to do here to return the value that
main expects? JavaScript doesn’t have a sleep function that we can use to suspend execution and
persist the activation records on the call stack for sendPing and main. If we try to busy-wait, and the
server does send the WebSocket message to the browser, it will end up in the browser event queue,
which the program has no access to. This is also where mouse clicks, timer fires, and other events
queue up, including repainting the browser window while JavaScript is executing. In addition, if we try
to busy-wait, the browser will try to kill our script, because we are blocking progress in the browser
event queue. One result of this is that JavaScript programs have no choice but to execute in some sort
of continuation-passing style, where they periodically yield to the browser event queue by scheduling
the next phase of the program to occur as a new event. In any case, the only way to unblock the
browser event queue and receive the WebSocket message that is queued up in there is to empty the
JavaScript call stack, which means we have to return from sendPing, and then we have to return from
main. Now the JavaScript call stack is empty, and the browser will fire its next event, which triggers our
callback. But now, we have completely disrupted the program, and we have no way to get this data
back to main. If we wanted to port the Java code to JavaScript, the program would need to be
completely restructured into continuation-passing style to allow for breaks in execution for callbacks to
execute. This is not what… this is just a fundamental requirement here.
So I’ve discussed these challenges, and I want to describe how the Doppio runtime system overcomes
them. Yes?
>>: I have a question about the file system. Why… another alternative would be to use a file system in
the cloud, and just have a local cache. Why don’t you want to explore that possibility?
>> John Vilk: Oh, I’m not limiting to that, in fact, when I describe it later, our file system does support
Dropbox cloud storage, but in that case, it’s still going to be asynchronous, so it’s doesn’t… isn’t going to
support blocking operations.
>>: Oh yeah, I just mean instead of having to support the three different… you could just use the heap.
>> John Vilk: It’s true. Yeah, you could do that. But the thing is you also have limited local caching,
because you have to cache to something local in the browser, and so then you’re back… you’re stuck
back to the original problem. You have those four different interfaces, one of which is supported
everywhere, but only gives you five megabytes of cache and…
>>: Assume you’re caching on disk and not memory.
>>: Right. You could cache the memory and…
>> John Vilk: Yeah, you can cache and memory, but if want to persist it… if you’re a web application—
right—and you want to a make sure that the caching persists across reloads—if you have, like, a large
amount of data, and you want to cache—you don’t want to grab it off the wire every time, then you
have these very limited storage options that are everywhere and then these better ones that support
large files that are completely separate; I need to support all of them to have full coverage. So that’s
like a huge issue there. Yep? Sure.
>>: So the limits are for storage. If you do the cloud-based option, and then you just load in the data as
JavaScript files, you could presumably exceed those limits.
>> John Vilk: Yes, but what I was saying is that if you wanted to cache them, so you avoid… yeah
>>: I’m saying you can cache them that way.
>> John Vilk: Right and wrong.
>>: So when you… the next time you load the app, it loads a big-ass JavaScript file with all of the stuff
that you wanted cached. JavaScript gets cached over the five… that five megabit limit… or megabyte
limit.
>> John Vilk: That’s true, but you don’t control the semantics of that, so you don’t really know when the
cache will be empty and such like that. But you’re right in that you can piggyback on the browser’s
existing caching mechanism—which works for scripts, images, et cetera—to cache your assets. You’re
correct there, but you also don’t exactly control the semantics of that, so you don’t know when that will
be emptied and stuff like that. But I suspect in a lot of cases that might be good.
>>: I don’t you ever fully control it, ‘cause the user can empty their web history at any time.
>> John Vilk: That’s true. Yep?
>>: Just this comment, I guess; I mean, if you are mapping from a model that’s synchronous and
that’s—you know—I mean, pretty much what you described [indiscernible] and so on and so forth—
right—so you have a fundamental mismatch, as opposed to having something that’s a promise-based
and… I mean, could still be Java—right—it’s just so talk about the programing style and frameworks one
uses. So any thoughts on that, I mean as far as the… of closing the gap, if you will, in terms of the
programming paradigm?
>> John Vilk: So it sounds like what you’re suggesting would eliminate the ability to run existing
programs that are already written in, like, the blocking style. Is that correct?
>>: Yeah, it would.
>> John Vilk: Yeah. So if you wanted to go that route, there’s already things like Google Web Toolkit
and things like that that work for things that you write programs in JavaScript style, where you write
things with callbacks, and you write things so that they carefully manage how long they execute so that
they don’t block the browser. But if you have existing code, then you have no… you’d have to rewrite all
of it, essentially, to get it to run in the browser. And so our… that’s our pain point here that we’re trying
to address. We don’t want people to have to port gigantic applications like javac or things like that to
get them to work in the browser.
Okay, so I guess I’ll continue, and again, feel free to interrupt me in the future if you have any further
questions. So now… I’ve discussed these challenges, and now I’m going to talk about how the Doppio
runtime system overcomes them. So first off, Doppio is a one hundred percent JavaScript runtime
library for bringing conventional languages to the web. There’s no applets; there’s no native escape
hatch; it’s complete JavaScript. We implement threads on top of the JavaScript event-driven runtime
model, and language implementations use these threads to keep programs responsive, support
synchronous I/O, and to run multi-threaded programs. We also expose a TCP socket interface on top of
WebSockets, a file system interface on top of the various storage mechanisms available to the browser,
and we emulate an unmanaged heap. These three services handle the challenges of browser diversity
that I’ve discussed previously, and they let programs running in Doppio to function across all of the
major browsers. And using this infrastructure in Doppio, we built DoppioJVM, which is a proof-ofconcept Java Virtual Machine interpreter, written in a hundred percent JavaScript that can run
unmodified programs straight from their JAR files. And DoppioJVM contains all of the features required
to bring our chat client to the web.
So I’ve discussed all those components; now, let’s talk about each of them. First, let’s talk about
Doppio’s threads. So as discussed previously, the chat client encapsulates each individual server
connection in its own thread, and this is problematic in the browser, because JavaScript only has one
thread. How might we map multiple language-level threads on to one hardware thread? So first off,
Doppio threads require language implementations to explicitly manage their stack state, and this allows
Doppio threads to suspend execution by saving the stack information into a JavaScript object and
resume later, using the same stack. And then on top of this basic mechanism, we build a time-sliced,
priority-based thread scheduler that adjusts its quanta dynamically for responsiveness. And using this
scheduler, not only are we able to support multi-threaded programs, but—and with a preemptive
semantics from the language level—we also run… we also allow long-running threads to execute over
multiple quanta, and that prevents them from freezing the webpage and keeps the application
responsive to I/O, such as mouse clicks, et cetera. But wait, there’s actually more. So using this
mechanism, we can support the synchronous blocking I/O by simply suspending threads when they use
JavaScript’s asynchronous I/O, and then resuming them once the needed information comes back. So
now we can actually directly hook up JVM threads to Doppio threads and that lets our chat clients
spawn and run across multiple threads without needing to be modified at all. So that… yep?
>>: The cost here is: maintain your own stack, right?
>> John Vilk: Yep.
>>: That’s a… that might… that’s a big thing. You really…
>> John Vilk: It’s true, yes. It does have an impact on performance, but the thing is there’s actually…
there’s a lot—a large class—of applications out there, that aren’t computationally intensive, that run
just fine with this overhead. And if you think about the web platform right now, most applications on
there are not computationally intensive; they’re just frontends to something—stuff like that—and so
they run just fine—they operate fine—in this environment with the overhead.
>>: Have you thought also about… there was this other project; in there, they generated code—little bit
more special when they mapped to JavaScript—such that they could—at yield points—they could kind
of recover the stack when you needed to save it from whatever the JavaScript was in. So it became
more like trampoline style.
>> John Vilk: Yeah. Was that the Racket-based Whalesong work or something else?
>>: No, I think this is, like, the Tesla work.
>> John Vilk: The Tesla?
>>: You mean the Volta?
>>: Volta, right.
>> John Vilk: Oh, was that the MSR work? First…
>>: It was the Microsoft, mostly, work which…
>> John Vilk: Yeah, I looked into that. They never actually implemented that. I looked at their code, and
then… if they discussed it, it’s not part of the actual project that they dumped onto GitHub, and I
couldn’t find any information on that.
>>: Right, right.
>> John Vilk: Yep?
>>: Isn’t OCaml JS already doing this kind of thing for dealing with tier recursion?
>>: JS OCaml is doing trampoline, but not for mutually recursive functions, I think.
>> John Vilk: Yeah, not that I know of. The only work that I’m aware of right now that does something
similar for threads is Whalesong—which is for Racket—but they don’t deal with multi-threaded
programs; they don’t deal with operating system things; they just deal with, like, single-threaded
programs.
>>: So when you say, “Explicitly manage your stack,” are you saying it’s CPS’d?
>> John Vilk: In a way, so right now, the one language that we fully support—the JVM—it just has an
explicit thread object that manages its state, but that’s not a requirement for Doppio threads. All
Doppio threads require is that at a suspend point, you can suspend by somehow recovering your stack,
either through explicitly managing it during execution or by dynamically recovering it at a point where…
when you need to.
>>: So that’s up to the application program to figure out how it wants your…?
>> John Vilk: No. It’s up to the language porter, not to the application developer. The language itself
supports it, so the application has no idea it’s happening at all.
>>: So you could try another strategy.
>> John Vilk: Yep. Oh yeah, you could. It’s not a limitation of our framework whatsoever.
>>: Because now, it’s like if you can use the JavaScript stack as most as possible, you can take advantage
of all the optimizations people do for JavaScript engines, right? If you maintain your own stack, now
you’re—you know—you’re pushing on this stack array…
>> John Vilk: Yeah, but the JavaScript stack, there’s only… you introspect on local variables or state like
that, so you would need to basically have custom logic in every function or something that would persist
that data somehow…
>>: Yes, right.
>> John Vilk: … and that would be another expense. In addition, if you do something like trampolining
with exceptions, the JavaScript JIT engines are very sensitive to where you place throws and catches,
and so it might degrade performance.
>>: Right, right. Yeah. No, I agree, it’s a difficult question.
>> John Vilk: It is. It’s a very interesting space to explore, but I haven’t fully explored it in this work. I
just laid down all the requirements, and I’ve chosen one just for the proof-of-concept.
>>: Right. No, yeah, great, yeah.
>> John Vilk: Yeah. Another question? Yeah?
>>: Yeah, two questions. The first one is: you have to have a precise understanding of where you insert
calls to your custom yield function, right?
>> John Vilk: Yeah.
>>: Okay.
>> John Vilk: So the yield function only comes into play during methods that need to be written directly
in JavaScript—so like native methods. So in the JVM, there are native methods: things like socket.read
and write. They eventually map down to a method that’s normally written in C that interacts with the
operating system environment. So within those, the person who creates the language and rewrites
these native methods needs to be aware of how the threads work and needs to say, “Hey, I’m gonna do
an asynchronous operation. You can schedule another thread now.” And then the thread schedule will
go ahead and pick another thread to schedule while it’s waiting for whatever to come back. There’s
also… we also dynamically… so using method call counts right now as a proxy for program runtime, we
use that to determine when to suspend after, like, a certain number, and we measure… we also
dynamically adjust that count, depending on how long it took to get to the count in the first place, using
a moving average. And so we try to stay responsive regardless of what the program is doing. So if it
stays within Java code for a very long time—it doesn’t hit a native method that says, “I need you to
pause for a bit and then come back”—it will eventually decide on its own: “I think it’s time to pause and
let the GUI repaint and not freeze the page.”
>>: And my other question was: what you describe is actually pretty similar to the ES7—I think—yield
generators…
>> John Vilk: Yeah, so…
>>: …that do exactly what [indiscernible] yield keyword that suspends the thread, and then you just
need to have an outer function that says…
>> John Vilk: I agree. So James Mickens here actually implemented some sort of, like, threading
mechanism on top of those, and they’re very slow right now, and they’re not very well optimized.
>>: So yeah, you can… at a high level—yeah—the yield keyword allows you to implement some of the
ideas that were discussed back here—about the idea of trying to leverage the native JavaScript stack to
remember activation records. Now, the time that we wrote this maybe, like, a year ago, yields were
super unoptimized…
>> John Vilk: They were interpreted entirely in most browsers…
>>: They were interpreted the whole way…
>> John Vilk: …and they weren’t well-supported across browsers at all, so it only worked in Firefox, and
then Chrome eventually got it.
>>: Yeah. Now recently, they’ve hit sort of like the first-level slow JIT in a lot of the browsers, so I think
they improved by like an order of magnitude or something like that. We can talk about that more
offline, but you’re definitely right the yield will allow you to do… to leverage some of the automatic
JavaScript stack.
>> John Vilk: Yeah, and that’s another thing you can explore with the Doppio framework if you want to.
I didn’t explore it at the time the paper was written, because it was just too slow. And before I get to
another audience question I actually have one here on the screen here. It asks, “How do we switch
threads when their quantum has expired?” I kind of answered that a little bit. So there’s two situations
or three situations—I guess—in which a thread will be switched. One is if the program’s been running
for a long time, and we use method call counts as a proxy for prog… measuring program runtime… oh,
okay, they got an answer; never mind. Okay, so next audience question?
>>: The issue of timeouts—right—browser timeouts, and predicting how long something might run in
order to avoid those, what’s your overall thinking on that? I didn’t quite…
>> John Vilk: Oh, yeah, so…
>>: Do you have an answer to that?
>> John Vilk: Yeah, so we… so what we do is we just do the method call counts to figure out how… to
estimate kind of how long it’s been running. And we assume that the method durations, they fall like a
nice distribution, and so they’re not going to take forever to… between method calls. Now right now, in
our proof-of-concept, if you have a, like, long-running loop that does not make a method call, it can
freeze the browser GUI, but we could fix that by augmenting our metric to incorporate loopback edges.
But right now, we haven’t encountered a program that has actually done that.
>>: Mmhmm.
>> John Vilk: Yeah. I realize it’s imprecise, but I haven’t encountered much that has, like, taken down
the browser yet.
>>: Alright. ‘Cause it’s kind of a… somewhat of a terrible possibility, right?
>> John Vilk: It is.
>>: That, like, it can just terminate the current event loop and then just proceed.
>> John Vilk: Yeah. It is, yeah. And in the future, you can use web workers, which would completely
eliminate that possibility of taking down the browser, ‘cause the web worker is completely separate
from the GUI context.
>>: Mmhmm.
>> John Vilk: But the issue is: you still have to occasionally suspend, because it gets messages from the
main context—from the GUI, like mouse clicks, et cetera—in its own event queue, and it will not receive
those until it winds down. So you still have to periodically suspend to be responsive to user input, but in
a web worker context, you would not be able to take down the GUI if our proxy for program runtime
turned out to be inaccurate in some case, and it took a long time to suspend. Anything else? Okay, I’ll
continue with the next bit then.
So that’s the Doppio threads, and next, I’m just gonna discuss the file system. So our text chat client
that I discussed earlier, it writes files into the file system for log files, and that’s normally implemented
in the UNIX file system class in Java when you’re on UNIX-based systems. And the Doppio file system
which I… it’s actually called Browser FS, it exposes a POSIX file system interface in JavaScript. And this
file system supports many backends, including most of the persistent storage mechanisms mentioned
earlier—Dropbox cloud storage, ZIP files, in-memory storage, et cetera—all through the same interface.
And then, you can actually mount these into arbitrary locations in the Doppio file system, which lets
you… it gives you great flexibility in where the chat client writes its log files, and lets you… it lets you
determine dynamically which one you want to use in case, for some reason, you don’t have access to
some of the better storage mechanisms at the time. And so these are actually… some of these are
synchronous; some of these are asynchronous, depending on the interface. Now, that’s not a problem
for the Java programs, because I… as I mentioned before, our threading implementation lets you map
the synchronous I/O to the asynchronous I/O—it’s not a problem at all. So that was the overview of the
file system. And next, I’ll discuss network sockets, unless someone wants to know more about that.
So our chat client expects to connect to the chat server through a TCP connection, but there’s a problem
to porting the setup to the browser that I‘ve sort of inferred earlier. The browser does not give you raw
access to TCP at all, and instead, you must use WebSockets, the WebSo… and WebSockets are… is a
protocol on top of TCP. The server has no idea how to handle these WebSocket connections. So using
Doppio threads, we can implement the JVM synchronous socket abstraction on top of asynchronous
JavaScript WebSockets. But the server still has no idea what a WebSocket connection even is. But we
can fix that with a pre-existing Python program called WebSockify. So WebSockify wraps the server
program and transparently proxies a WebSocket connection to the server as a standard TCP connection,
and so neither program needs be modified. You only have to have cooperation with the server runner
to wrap the program like this, and everybody wins. On both sides, you have a unmodified chat client
running to… connecting to an unmodified server, and neither have any idea of the mechanisms that are
in between each other. Yep?
>>: Wait, but you have to go to, like, Microsoft and ICQ and tell them to run all their servers in
WebSockify?
>> John Vilk: If you want to do that, yeah. You could also run a local proxy, if you wanted to, in some
unsafe escape hatch, but ideally, if you are bringing something to the web, you control the server
portion as well, and so you can simply wrap your program like this. Yep?
>>: Firefox had the same problem with Firefox OS when they wrote the mail client. They needed raw
TCP access; what they did was create a privileged API that’s only available through privileged Firefox
code called mozTCPSocket. Have you gone, put in a call, and tried to pressure them into—you know—
putting it into a standard [indiscernible] like, “Hey makes me cry every time I see all these layers of
abstraction. Please do something about it.”
>> John Vilk: No, I haven’t.
>>: I mean, it’s very time-consuming, so… and difficult, so totally understand if you didn’t, but just
curious.
>> John Vilk: Yeah, I haven’t pressured them, ‘cause I’m not entirely certain if it’s a good idea just to let
web programs connect to arbitrary servers through TCP. I haven’t really thought through all of the
issues with that. They’re very sensitive to these things, but—you know—I… maybe.
>>: Okay.
>> John Vilk: Not sure. Yeah. I’m more… I was more concerned about the threading, and what I can do
to improve that in the web platform than the OS stuff, ‘cause I assume that eventually, all of these
services are going to expose some sort of WebSocket connection, once that permeates through all the
browsers and becomes more commonplace.
Okay, so tho… that’s were the sockets. Now, I’m just going to talk about this unmanaged heap. So
when our chat client starts up, it pulls in a wide variety of libraries in the Java class library. And if you
invoke this unsafe Java API when they statically initialize—which lets the program do many really fun
things—and supporting this library is a requirement to bringing the Java platform and the chat client
into the browser. So all we really need here is an implementation of our old friends or enemies—
depending on how you feel about them—malloc and free. And implementing these is very wellunderstood and won’t be covered in the talk in detail, but you need to operate somehow on buffers of
memory. And funnily enough, therein lies the problem: representing binary data efficiently across all of
these browser variants. So we actually use three different representations, depending on the browser
and what support it has for these things. So ArrayBuffer and its predecessor, CanvasPixelArray, are
guaranteed to compactly represent our heap in memory. And for those in the audience who are familiar
with the HTML5 Canvas, the CanvasPixelArray is used as… to back the image data that is stored on the
canvas, when we’re just completely abusing it, ‘cause it’s the only thing available in that situation that
has a very compact memory representation, where every byte that you write to it maps to one byte of
memory. Now unfortunately, if the browser doesn’t support those two objects—such as Internet
Explorer 8—we fall back to a vanilla JavaScript array of numbers. And this has a two-x memory
overhead, since JavaScript numbers are sixty-four-bit doubles, but it’s the best universal fallback that we
can possibly do, since all of the JavaScript bit operations—like OR, AND, et cetera—they coerce the
numbers into a thirty-two bit form. And so with these buffers of memory, we can implement a… the
unmanaged heap using the data structures, and we can also implement the JVM’s malloc and free
interfaces directly on top of this unmanaged heap, and that lets the Java class library perform all of its
unsafe voodoo at program startup and during runtime.
So I’ve covered the runtime system, and now I’m gonna move into DoppioJVM and how it hooks up into
all of this to bring the JVM to the browser. So DoppioJVM uses an unmodified copy of the OpenJDK Java
class library to provide the applications with all of the library support that they expect. And as I
mentioned before, the various Java class library interfaces within Doppio are mapped… sorry, let me
start over. The various Java class library interfaces are mapped directly onto the Doppio operating
system support, and using Doppio threads, these interfaces can keep their blocking JVM semantics, even
when they need to invoke an asynchronous JavaScript operation. And we also implement all of the
native portions of these classes, which we admit on here, but they’re… that’s a part of it as well.
So we covered that; let’s talk about the more fundamental details. So we map the core JVM data
structures into JavaScript features in a fairly straightforward manner. Java objects become JavaScript
objects; Java arrays become JavaScript arrays; most numeric types become JavaScript numbers, which
act as both thirty-two bit integers and sixty-four bit doubles. But sixty-four bit integers are not natively
provided in the browser, so we emulate them using Google’s gLong Library, which means that if you
have any code that operates with long data types, we have no choice but to allocate an object to
represent that long in memory. And due to the straightforward mapping, we actually get garbage
collection completely for free, because we are already executing within a garbage-collected
environment.
Finally, there are some features of the JVM that we had to just completely re-implement for the
browser. This includes the core bytecode interpreter for JVM bytecodes, the JVM ClassLoader and JVM
monitors, which are used for synchronization across JVM threads. And so DoppioJVM is completely real;
it exists; it’s been evaluated by the PLDI Artifact evaluation committee; it got the Distinguished Artifact
Award; and this is the demo here; and this is our mock terminal interface that you can see yourself at
doppiojvm.org and play around with. And in tradition with all of my Microsoft Research talks, I’m
actually gonna show you a live demo.
If I can figure out what side of the screen… there it is. Okay, so this is the interface. I’m serving it locally,
just to avoid any potential network shenanigans to interfere with the demo. So we have a
straightforward terminal interface here; we’re in temporary storage, but if I go to the root directory, you
see we have this mount directory, which has a bunch of browser storage things mounted. Because
we’re in Chrome, I’ve mounted HTML5 file system in local storage in here, and you can go in here, and
you can write file. So let’s create a simple Java file here. I have a very blurry screen over here so forgive
me in fumbling about. So let’s write a little Hello World program here. So system.out.printline Hello
Microsoft Research. Actually, let’s put a syntax error in here just for fun, save, and close. So now, I
actually have a choice of two Java compilers that I can use that are completely unmodified. We have
javac and the Eclipse compiler. Let’s invoke javac on this; see what it tells us. And so now, what it’s
doing is it’s actually pulling in individual class files off the wire, so the first run’s always slower. And you
can see here it tells me… oh, I’m missing a semicolon here—oops. So let me edit that and add that back
in so we compile that bad boy.
>>: So you improving this editor—right—and…
>> John Vilk: That’s actually an existing JavaScript library called Ace Editor.
>>: Right.
>> John Vilk: Yep, so we just…
>>: And you implement this—you know…
>>: Shell?
>>: …from the shell, basically, right? It was [indiscernible]
>> John Vilk: Yeah, the shell is actually implemented in JavaScript. I was debating rewriting it in Java at
some point just for fun, but…
>>: That’s fine—right—but then… and that works directly with the Doppio VM or… right?
>> John Vilk: Yeah. This works directly with the Doppio VM and the file system to bring all of it
together, so the Doppio file system is accessible nicely within JavaScript.
>>: As I understand it, it’s cool you made this in JavaScript, because a problem with the ASM JavaScript
stuff—they may really compile C programs in these magic ASM things—it doesn’t work well with the
other JavaScript, right?
>> John Vilk: No it doesn’t at all.
>>: But here, it’s cool, but everything actually is JavaScript, so you can easily interface between them,
right?
>> John Vilk: Yes. Yeah, there’s no… yeah, ‘cause with that… people who aren’t familiar: asm.js, it puts
certain JavaScript engines like Firefox into a different mode where, strangely enough, you can’t use
objects, which is kind of a requirement for most JavaScript code. And so if you want to interface
between something in asm.js and regular JavaScript, you have to use an FFI between the two which, as
far as I can tell, is undocumented, and I couldn’t get it to work. But in addition, all of the stuff like
Emscripten and things like that that compile things to asm.js, they don’t do any of the threading things
that I do, and so as a result, they can’t map their file system to HTML5 file system or any of the large
storage mechanisms now in the browser.
>>: Right.
>> John Vilk: And in fact, if you want to use them, you have to manually, like, implement sync
operations that tells it when to do certain things. Yep?
>>: On the other hand—you know—being inside asm.js, you get some isolation properties, apart from
these undocumented FFIs…
>> John Vilk: Yeah.
>>: …that you know that your Ambient JavaScript isn’t going to muck with the invariance of your
compile language. Whereas here, you have JavaScript—you know—interfacing with the JVM
implemented inside JavaScript, and you may worry that—you know—potentially malicious or buggy
JavaScript just completely screws with the invariance of your system.
>> John Vilk: That’s true, but that’s also true of any other JavaScript code that you have on the page.
>>: Yes, absolutely.
>> John Vilk: Yeah. So the way that we view this is that if you are deploying Doppio on the page, you
are also controlling what data you feed to it. And much like any other JavaScript library, you guard the
global variable that we expose that you use to interface with it to do all the JVM things. And so as a
result, we have, like, the same sort of security properties as any other JavaScript library as far as I’m
concerned. Right?
>>: Really? I mean, for example, if you make a call from inside your JVM abstraction to a JavaScript
function…
>> John Vilk: Yep.
>>: …what prevents… I mean how… the calling convention—for example—that you’re emulating, that’s
Java’s calling convention inside JavaScript, which isn’t really the same; those differences may be
observable to a native JavaScript program that wasn’t compiled, or it wasn’t run inside your
environment. For example, it may try to do—you know—caller dot—you know—callee.stack.blah—
right—which may not exist or may be set up differently in your environment than what that JavaScript
program expects. And it may be able to notice that difference and do something with it.
>> John Vilk: Okay. So you’re talking about if a Java program’s interfacing with a JavaScript library that’s
on the page.
>>: That’s right, that’s right. Yes, as this program seems to be doing, right?
>> John Vilk: Yeah.
>>: Like you are calling into javac from here—which is a Java program—and maybe this JavaScript
program, I mean, you wrote it, so it’s not particularly malicious, but I mean, if you got it wrong, for
example, it… your program may be able to… your JavaScript program may be able to break the
invariance of your compiled Java program.
>> John Vilk: Yes, that is true.
>>: So I’m just saying… which is perfectly… I mean, I think in the setting that you have it’s quite a succ…
you know, it’s okay, but I mean, it’s coming back to the counterpoint with asm.js, this kind of isolation
between asm.js and the rest of the environment…
>> John Vilk: Which is only maintains…
>>: …is sort of for a reason, right?
>> John Vilk: It’s only maintained, really, within Firefox, as far as I can tell. In other browsers that don’t
support the special use ASM flag, they just treat it as regular JavaScript, and so there’s nothing blocking
you from mucking about with that. In Firefox, I’m not sure what happens if you break the invariance
that they assume… I don’t know if the page reloads and just executes in regular JavaScript mode, or if it
fails to execute, but that wasn’t really designed as a security issue; it was just… that wasn’t really
designed as a security invariant to maintain; it was designed as… we can turn on extra optimizations
now and make it fast.
>>: That’s true, that’s true.
>> John Vilk: Yeah, so I would not use that as a… as some sort of sandbox mechanism. Okay, so… yep?
>>: I mean, just to next point, I guess. I mean, it’s not an… doesn’t sound like you’re making particularly
strong promises about the fidelity of the translation just given how difficult it is, and so this is one of the
issues there, which is within an environment that could be hostile.
>> John Vilk: Mmhmm.
>>: You could have all sorts of funny things happening. You could get…
>> John Vilk: Absolutely.
>>: …it is what it is.
>> John Vilk: To build off that, we haven’t implemented the bytecode verifier; so simply malformed
class files, if you don’t control that input, would throw the JVM off track, ‘cause we don’t consider that
to be an important, hot issue in the overall contributions we’re trying to make here.
>>: What about things like—you know—mapping Java’s memory model precisely to your model of
threads and the native heap and…?
>> John Vilk: That’s a good question.
>>: Because this is not difficult enough, clearly.
>> John Vilk: Yeah, clearly, yeah. [laughter] I didn’t really consider that to be a priority when I was
working on this. So I would not be surprised if you could find byways to the memory model. Could be
considered future work, but at the moment, I haven’t found any programs that have triggered any weird
memory model issues, so…
>>: Okay.
>>: Could you show us how much code there is underneath, like the debugger maybe, or just tell us?
>> John Vilk: So in terms of lines of code, if you include all the native bits with the JVM and everything—
the DoppioJVM—it’s, like, less than twenty-five K of JavaScript from what I remember—last time I
measured. That includes the file system, and DoppioJVM, and its native implementations, and
everything else. That does not include the Google Long library that I’m importing; I don’t know how
large that is. But overall, it’s not an unreasonable amount of JavaScript. Now, the big part is actually the
class files that I’m pulling in. The Java Class library itself—the core part of it compressed in ZIP form—is
thirty megabytes, and uncompressed, it’s sixty, but that’s only if you pull in the entire thing. Every
program, typically, used probably about, like, five megabytes, ten megabytes of that—and that’s
uncompressed sized. So if you Gzip that, you get huge savings there. So it really depends on the
program, and how much of the JCL it uses—that’s the biggest cost. The JavaScript is almost nothing
compared to that.
But… so yeah, let’s see; I fixed my syntax error; I’m just gonna compile this using javac. Takes about—I
think—eight seconds, last time I measured. And now, if I run it with Java, it’ll just print that out. And
also, we have ECJ, which is the Eclipse compiler, which—yeah—all those little options are here. And
that’s just… we didn’t modify any of this, it’s just… it’s direct class files. So in addition to being able to
do that, let’s see if javap works—which is the disassembler—so we can just see that this actually has the
structure that we desire. And in addition, other languages that work on the JVM, work. So for example,
Kawa-Scheme is an implementation of Scheme on the JVM, so we can boot that up. This is loading all
the files from the JAR file, which takes a bit for the first time, but… now we have it’s little REPL, and
gonna do one plus one, give us two. I’m not much of a Scheme hacker, so I can’t do anything really cool
here, but you can mess around with it at doppiojvm.org, and let me know if you break anything. And
also, finally—just for one final thing before I switch back to the slides—Rhino is a JavaScript
implementation on top of the JVM, so naturally, we wanted to go a little bit meta. So if we run Rhino,
we’re gonna get to a JavaScript REPL that is executing on our JVM that is executing in JavaScript.
[laughter] I have yet to complete the loop, because I have to… the only thing remaining to complete the
loop in which I can run Doppio in Rhino on Doppio—oh, this thing went to sleep, it’s back—is I have to
actually port a file system backend to my file system that maps to Rhino, which maps to Doppio, which
maps to browser—all that stuff. But I’m very close to completing the loop; I’m sure next demo, I’ll have
it ready. So yeah, we have a JavaScript REPL here.
>>: It’s amazing.
>>: It’s also amazing, like that… like, when you do javac, it runs an entire Java compiler inside your
browser, right? And JavaScript interprets it.
>> John Vilk: Yep.
>>: And so that means that JavaScript interpreters are… they’re good. This is… I mean, it is amazing.
>> John Vilk: It is.
>>: It works, right?
>> John Vilk: Yeah.
>>: Yeah, right.
>> John Vilk: Oh, what did I forget?
>>: Well, maybe you don’t have much experience with JavaScript. [Laughter]
>>: You kidding [commotion]
>> John Vilk: Oh, oops.
>>: Everybody’s a critic.
>> John Vilk: Ah, I thought it was WriteLine. Maybe it’s just write.
>>: Document [indiscernible]
>>: Sprite, I think.
>> John Vilk: No, the… ‘cause it’s a… it has different interface completely from...
>>: It’s [indiscernible]
>>: Print, print.
>> John Vilk: Oh, it’s print.
>>: Print, print.
>> John Vilk: It’s print.
>>: This is amazing that we have this issue.
>> John Vilk: This is like peer programming, guys. [Laughter]
>>: Right brain [indiscernible] just print.
>> John Vilk: Yeah, there we go. Print doesn’t add the new line, so I think it’ll mess up the console a
little bit… so if I do this it’ll just…
>>: Do alert. [laughter]
>> John Vilk: I don’t think you have access to alert from this, ‘cause there’s no DOM. Yeah, look at that.
Okay, so let’s exit… I think it’s exit or it’s quit.
>>: How about barcode toolkits for Java, like Swing, AWT, STWT?
>> John Vilk: Good point. So we tried to port AWT—which is the Abstract Window Toolkit, which is
what Swing is implemented on top of on the web—but there’s a well-documented issue with OpenJDK
in that the AWT is hard-coded, funnily enough, to the platforms it supports. Even though it’s called the
Abstract Window Toolkit, it’s not abstract at all. In fact, it calls a bunch of final Sun classes that are hardcoded to different platforms, and so if you wanted to port it to the web, we’d have to modify the Java
class library, which we don’t want to do. With that said, I did find that someone on the internet is
porting JavaFX to Doppio, and they have a basic demo working with, like, shapes moving around the
canvas, both with a complete Java software renderer, running at full speed in the interpreter, and also
one that’s has some native methods implemented, so it’s using the canvas and hardware acceleration
directly.
>>: And how about SWT?
>> John Vilk: What was that?
>>: SWT.
>> John Vilk: I haven’t looked into that at all.
>>: Okay. Because in that case, we could open Eclipse. [Laughs]
>> John Vilk: Yes. Yeah. So the only thing that you have to do is write that in the methods for that, and
if it requires a special backend for the platform, you have to do that as well. So it’s the same as porting
it to a different… you have to put in as much effort as porting that to a different operating system, or a
different flavor of Linux, or something. But yeah, I haven’t looked into that in particular.
>>: Thanks.
>> John Vilk: So that’s the little demo. And again, this is at doppiojvm.org; feel free to mess around
with it. You can also mount Dropbox in here—yeah—which takes a bit sometimes, ‘cause it’s hitting
Dropbox server every time I do a file system thing. But I have little… is this actually…? So this is actually
downloading this from my Dropbox account. I’m not sure if this’ll work—meaning, I don’t know if the
network connection will cut out. Ah… it’s taking too long. So this is actually downloading this JAR file
from my Dropbox account, and control C kills it. So yeah, feel free to mess around doppiojvm.org, and
its—also—source is on GitHub. And if you want to see that it’s just static files, you can clone the demo
repository and look at the gh-pages branch, and you’ll see it’s just files. So let me go back to my
presentation to finish up.
Okay, yes, so as I showed you, doppiojvm… oh this thing, like, just minimized. Sorry, okay. I was
reconnecting with the question interface. So DoppioJVM runs with the real, unmodified programs that
they just showed you. These… those four programs were actually our little benchmark programs, and in
some cases, they’re not so little—they’re quite large. Those are the… that’s how large they are in Java.
And we’ll jump into the benchmark results. So we’re comparing DoppioJVMs performance to the
HotSpot interpreter, because DoppioJVM’s also an interpreter. And we want to note that DoppioJVM is
totally untuned; it’s just a proof-of-concept; and HotSpot is highly optimized, and it’s written natively.
But despite this, in our best browser—Google Chrome—DoppioJVM, on these benchmark programs, is
twenty-four to forty-two times slower than the HotSpot interpreter.
>>: Is the HotSpot interpreter?
>> John Vilk: Oh, the HotSpot interpreter is when you get Java from Oracle; it’s their Java
implementation, and the interpreter is with JIT compilation turned off.
>>: Running on…?
>> John Vilk: Running on the same server… so these are both running on a MAC Mini with a Core i7 at
two gigahertz, I think. It’s in the paper, but yeah, they’re running on the same system. In fact, I don’t
think I could find out how to turn off multi-threading, so the HotSpot interpreter’s actually using multithreading to get ahead in some cases. But any case, we’re comparing against that.
>>: So I had a quick question on this.
>> John Vilk: Yeah?
>>: So I assume all these benchmarks are run in a very compute-intensive manner.
>> John Vilk: Yes.
>>: So this is sort of the worst-case scenario.
>> John Vilk: Yes.
>>: You wouldn’t even want to really do this. You’re more interested in sort of the UI-bound…
>> John Vilk: No, you would not.
>>: Do you have any examples of like...?
>> John Vilk: I do.
>>: Okay.
>> John Vilk: I’m actually getting to that point.
>>: Okay, good.
>> John Vilk: Yeah. Yeah. So yeah, these are the results. You’ll see that it differs a lot between
browsers. We developed this in Chrome, and we profiled it in Chrome, and so we probably are a bit
tightly coupled with it. And because it’s an interpreter, any small changes in the… how we dispatch op
codes and such, is going to have a huge impact on program runtime. Also, you’ll notice that Safari had
an issue with javap. We found an issue in Safari, where it wasn’t garbage-collecting certain things, and
that’s been fixed now; so that’s no longer an issue. But yeah, these are all compute-intensive code, like
you’d give it a command, and it goes on forever; it’s the worst-case. And the browsers don’t generally
run this type of code. And so we found the DoppioJVM’s fast enough for many people already who have
discovered it. So for example, the University of Illinois came to me with some… for some help, and
they’ve launched a website called CodeMoo.com, which uses DoppioJVM to teach Java programming—
or basic Java programming—skills to children. So this is the little game that they can play; this is like a
level-one game. So children go here; they string together function calls to guide the character in red to
the goal without bumping into bad guys. And when ready, after arranging all of these things, the child
clicks the green button and watches the character carry out the instructed actions. Under the covers,
these instructions are placed into a Java file in the Doppio file system, compiled with the Eclipse
compiler running within Doppio, and then the resulting class file is run in DoppioJVM again, and that all
harnesses the power of the visitor’s own browser. So existing things like this, existing platforms people
use—and universities use—to teach children how to program in Java use server backends to run the
Java code. In this case, it’s just using the browser directly, and all they have to do is serve the static files.
So yeah, in this talk, I introduced Doppio—a full runtime system that bridges the impedance mismatch
between conventional languages and the browser—I showed off our proof-of-concept Java Virtual
Machine—DoppioJVM—which can run unmodified Java programs in the browser. I also didn’t mention
that our file system can sometimes integrate into Emscripten programs, and so that’s been used in a
variety of web pages. And I encourage you to check out Doppio at doppiojvm.org and thank you all for
your attention. [applause]
>> Mark Marron: Thanks a lot. So I think we got a lot of questions throughout, but definitely time for
more.
>> John Vilk: Yeah.
>>: So that execution was from February; what’s been going on in the last eleven months?
>> John Vilk: So we been looking at speed; we been looking at supporting other languages; and I also, in
my spare time, got Java8 working. So variety of things on the plate right now that I’m exploring—not
much to talk about just yet, but I’m hoping I’ll have some news on that in a few months. Yep?
>>: You mentioned compatibility with fairly old browsers, such as IE8.
>> John Vilk: Yes.
>>: Why is that important for you?
>> John Vilk: Yeah, I wanted to see what all the issues were with browser diversity. Now, I think I broke
IE8; I think it’s broken on our demo right now, but IE9 definitely works. But yeah, I wanted to see what
web developers currently face in bringing programs to the web, and I wanted to see if IE8 could work.
And I did have it working at one point; I think there’s a weird bug in there somewhere.
>>: Fair enough.
>> John Vilk: Yep?
>>: I’m curious what the take loop for executing individual Java bytecodes is. Are you… is it like a array
index into a bunch of function, or is it…?
>> John Vilk: Me think… um… as of the demo—I’ve rewritten it recently—so as of the demo that I
showed you, that actually has an object allocated for every interpreter instruction, which has, like, a run
command, and so that allows for some nice allotting in some cases. The version that I have now is a lot
simpler, and it has an array of functions for every interpreter op code, and dispatches it to… using the
number at a certain op set in the bytecode directly. So those are the two that I’ve used, and they have
similar performance. I don’t know if that was clear but…
>>: Is it a big switch statement or is it a chief…?
>> John Vilk: No.
>>: Do you know?
>> John Vilk: No, no. So actually, switch statements, in some JavaScript JIT engines, are not optimized if
you have more than a hundred and twenty-eight cases.
>>: So it… you index an array of functions?
>> John Vilk: Yeah, I index an array. Yep.
>>: Okay.
>>: Alright. Well let’s thank John again. [applause]
>>: Thank you, Johnny.