>> Ben Livshits: So let's get started. I'm Ben Livshits. I'm excited to host a visitor
from other than San Francisco visiting us for the day. He'll talk about type
inference for dynamically typed languages. Avik, the podium is yours.
>> Avik Chaudhuri: Thanks.
>> Ben Livshits: And by the way, if you want to talk to Avik one-on-one, we still
have, I think, a slot of two somewhere sometime in the afternoon.
>> Avik Chaudhuri: Thanks, Ben. Just a brief introduction on who I am. So I
wear various at Adobe. One of the hats is designing ActionScript 4 which is the
next version of ActionScript. ActionScript is a language that analyzes Flash
applications. And this was part of a larger refocus of Flash towards gaming. So
Flash was traditionally targeted towards ads, but now we want to focus towards
game programming.
And ActionScript 4 went a long way to achieve that with a cleaner and richer type
system and faster byte code. So we are recently more interested in JavaScript
and this is again part of a larger effort to advance the web. This in particular
means that I've been starting to get involved in TC39 which is the JavaScript
design committee. We are thinking of adding various kinds of features to
JavaScript to make it better such as concurrency.
I am also interested in all sorts of optimizations that you can do in compiling
JavaScript. But the second part is fairly new; we're just getting started there. In
this talk I'm going to focus on type inference. Type inference means very different
things to very different people. One thing that I want to focus on in this talk is the
kind of techniques that will allow you describe rather than prescribe behaviors.
So we're not trying to enforce a very strict discipline on code; instead, we are
going to take code as it is and try to gain information about that code, in
particular to use that information for various purposes like performance
optimization, security and [inaudible] and so on, also maybe just to understand
the program better.
So if you think about why people write dynamically typed code, there are at least
two reasons and sometimes the same piece of code can manifest both reasons.
One is just avoid verbosity, right, Java-like verbosity. So you're doing something
that's fairly simple but you don't write down types every time you come back from
a function or declare a variable and so on.
The other one is where you want to avoid rigidity in the type system. So in the
first case you are writing implicitly typed code but you just don't want to write
down types. In the second class of programs you are actually trying very hard to
get around the type system because the type system is getting [inaudible]. Right?
So if you don't have generics in your system either you write multiple versions of
type code or you use a dynamic type, and you get around the type system. If you
have generics but not dependent types, maybe you want to again go out of the
type system. So the dynamic type offers you kind of an escape hatch to write
programs that would otherwise be inexpressible in your type system.
So going along this intuition, we really have at least two problems. One problem
is when you are given implicitly typed code -- And I would call this kind of code
the good code -- you don't have types in there but it's still implicitly typed. Can I
make it run faster without breaking the program?
So you have the program. You're not trying to change it any way; you just want to
get some information to run it faster. So this is based on a paper that was
published POPL 2012, joint work with Aseem Rastogi and Basil Hosmer. And the
second line of work I'll discuss a technique where you are given explicitly untyped code, highly dynamic code, and I'll call this code the bad code. Although,
I'm not passing judgment, right, I'll just give you very good reasons why you
wouldn't want to write such code.
It's bad just because it breaks most traditional static analyses. Given such code,
can I discover types by observing it as it runs? So I'm going to throw some
dynamic analysis at this problem. And this is based on an earlier paper that we
published in POPL 2011. This is joint work with David An, Jeff Foster and Mike
Hicks.
All right. Static Analysis: beautiful word. Why do we want -- So the motivation
behind this line of work is this whole buzz around script to program evolution. So
the idea is that you start off with a dynamically typed script. And by script I mean
just anything that you've just, you know, written down fairly fast for typing
purposes. And then, this script needs to be gradually moved into a program. A
program is something that you want to maintain for a long time and it has to be
robust and performing and so on.
And there's this thing called gradual typing that many people advocate which is
really just incrementally adding types to you program. And as you incrementally
type, you get more checking and you get more guarantees. That's the hope, at
least. So adding annotations would give you more performance. That's the
promise of gradual typing. I mean not experience; this is more or less a fantasy.
And we have a lot of experience in this domain. ActionScript is a production
gradual type language, and it's used by millions of people. And really they don't
use it in this manner. They either write script or they write programs, and it's
unfortunate because this whole story actually makes sense but nobody uses it.
Why doesn't anybody use it? If you think about it, there's an annoying tradeoff
that you are providing programmers with. You either annotate your program
which means you have to take on a lot of burden or if you don't you get a
performance penalty. And people choose what they want and they go with either
one or the other and they don't get rid of both at the same time. So the problem is
that if you leave out the type annotation, the compiler thinks that you really want
a dynamic type. So missing annotations of dynamic types will default; although,
what you really meant was that a dynamic type has to be there as a fall-back and
most of the time you are not writing dynamically typed code. You are writing
statically typed code but implicitly statically typed.
Okay. Yeah?
>>: So if you have a tracing, wouldn't you be able to get [inaudible] marginally
performance penalty? If you don't make an assumption, right, if you just trace
through the code then [inaudible] the types and have [inaudible] for other things.
>> Avik Chaudhuri: Yes. You get part of the way there, I would claim....
>>: But [inaudible].
>> Avik Chaudhuri: Yeah. If you are trying to discover types of runtime -- And
we'll go into this -- you know, most of the time you are getting local information
and not getting cross function opportunity. Yeah?
>>: Is performance really the -- I can see performance as being one of the main
concerns here but isn't it also a question of maintaining code and being able to...
>> Avik Chaudhuri: Yes.
>>: ...have modularity, [inaudible]...
>> Avik Chaudhuri: Of course.
>>: ...[inaudible]?
>> Avik Chaudhuri: Of course. Cite all the good arguments for static typing and I
am with you. But if you have to really sell it, you know, talk performance and this
idea immediately sells. Right? People are sometimes -- You know, most
developers are okay with working hard to write badly written code, but if they can
get some performance out of doing something then that's a larger motivation for
them to try something out.
>>: So that is the big carrot? To get people to annotate their stuff with...
>> Avik Chaudhuri: Oh, yeah. Yeah, yeah, yeah. I mean especially because
there are a lot of low hanging fruit and ActionScript 3 is not great at performing.
So if you dangle that carrot, they immediately [inaudible].
Okay, so we're going to talk about gradual type inference then. Practical
motivation is to improving the performance of existing Flash applications by
eliminating as many type casts as possible. And one of the key goals in this
work, which is important because this [inaudible] what techniques we are going
to use, is that we want to be backward compatible which means that we cannot
recompile program and have it break in arbitrary context and so on.
A non-goal is to increase safety or eliminate errors, which means that, yes, we
can of course reject programs. So maybe this is too harsh a statement. But we
definitely don't want to fail early or pull up checks in places -- If we are not
guaranteed to fail, we should not fail. So this is a very different kind of modality
than the usual way of thinking about type languages. You conservatively prove
that you may fail and you reject a program.
And this is because we should admit code that is reasoning outside the static
type system; otherwise, we will be solving a different problem than what people
require. So here is an example of a gradual type program. You can see some
annotations and some missing annotations. And by default these missing
annotations are taken to be the dynamic type which I denote by a star here. And
what that does is it -- So if you think about it, how is a dynamically typed value
represented at runtime? It is represented with a type tag, so you do a lot of
boxing and un-boxing all over the place. So here zero was up-cast from number
to star which is what you would calling boxing in most compilers.
And you also have the opposite, down-cast, which is un-boxing. And typically
some operation may involve a whole lot of boxing and un-boxing. So this is the
kind of byte code that an ActionScript compiler will generate.
And you pay heavily for this. Of course [inaudible] would typically take out some
of these but not all. So instead of assuming that a missing type is a dynamic
type, what we do here is that we assume that it's an unknown type. So we invent
type variables. I represent type variables with capital letters here and italics. And
we will try to enforce static types for these unknown types, very possible;
although, the dynamic type is perfectly [inaudible] as well as a solution. So if we
cannot type a certain part of the program, we will not reject it. We will just give it
the dynamic type. So we won't be worse off than what we were before, but
hopefully we will be much better.
So this is an architecture of the system. You start up with a program annotated
with type variables and these type variables are obviously auto-generated.
There's a compilation phase that translates this program to a program with
coercions. These coercions are just like type casts that the compiler will insert for
you except that in this case they also have type variables in there. And next we
interpret these coercions as small flow constraint, and we do a closure
computation on top of these flow constraints to get a flow relation. And then, we
solve this relation. We get back solutions for these type variables. We plug back
the solutions in the coercions. So you get concrete coercions and then, you can
illumination many of these coercions.
Okay, so this is a notation I'm going to use for a coercion, T1 to T2. This
essentially means that I have a term of type T1 that is flowing into a context that
expects a term of type T2. And I will distinguish two kinds of constraints. One
kind of constraints is the constraints that are flowing into a type variable which I
will call inflows, and the opposite one would be outflows.
Again, I want to distinguish between these two because they are going to
determine how we solve for these variables. So instead of star now I'm going to
generate coercions to type variables and coercions from type variables, right? So
the coercions that are generated are completely trivial.
Now, again, looking at the type variable you will have a whole bunch of things
that are flowing in and a whole bunch of things that are flowing out of the type
variable. So how do typical type inference algorithms work? You see what is
flowing into a type variable and that determines all the things that that variable
can contain. So these are the definitions of the type variable, and if you take the
least upper bound of these types then you get some sort of tight bound of what
that variable can contain, how it is defined. If you look at the outflows on the
other hand, okay, if you take a greatest lower bound or intersection of all the
places where the type variable flows out, essentially you're talking about how that
variable is used in your program, so you're talking about safety there.
And typically a solution is somewhere in between this least upper-bound and
greatest lower bound. And if you have certain defined type variables sometimes
the solution is fairly [inaudible], it's either this or that. But some other type
variables may have ambiguous polarities and you can't decide which one to use.
Sometimes these bounds will even cross over means that something funny is
going on in your program. Your program may not be completely wrong but it may
involve reasoning outside the type system.
So anyway, we just said that the theme here would be to describe and not
prescribe behaviors. So we will focus on least upper bounds here because we
want to define exactly what the tightest representation for a type variable is so
that's going to determine our solutions.
So as applied to the simple example, if you look at the inflows into S there's just
one so S is number. Two things flow into I; if you take the union it's still number.
One thing flows into the return type of this function and that's also number. So we
allow flows but it's not important in this example but just note that we did ignore
all the outflows. So you -- Yeah?
>>: Was the bang significant in foo?
>> Avik Chaudhuri: That's just notation. So we are using the notation foo bang to
denote the return type of the foo function. We'll also use foo question mark to
denote the parameter type of foo. It's not significant; it's just a name, naming
convention.
So we get this program now once we insert all the solutions in. And you can see
a lot of redundant stuff here, right? So number-to-number coercions can be
eliminated. So we end up with this program and that's much simpler than the
original program, the compile program that we had earlier.
So this is a key idea: only inflows determine solutions in the system. And this is
precisely because we want our type inference to be backward compatible. We
don't want to, you know, do early errors. And you'll see why I'm saying this in
more examples later.
So how does type inference for higher order types work? ActionScript is a dialect
of JavaScript so you have first class functions flying around all over the place. So
this is kind of a simple example that is indicative of larger examples. Where you
have a function that takes anything and I assign it to a variable called X. And
under some condition I override this function with something that is more
specialized. It takes an "int." And I can call it with integers in that context, but in
other contexts I can call it with things that are not ints, for example, doubles.
So if I proceed as before I'll generate two coercions here: start the void goes to X
and int to void goes to X. These are two kinds of functions that are flowing into X.
And if you are doing kind of a traditional static type inference because we are
interested in taking the greatest lower bound of the things that are flowing into X - Sorry, I should have said least upper bound. Sorry. And the least upper bound
for functions is defined as the greatest lower bound of the parameter types arrow
the least upper bound of the return types. Right? The polarities switch at the
parameter site. That's how I get int to void, but this is completely unsound in the
definition of unsound that we have because it's going to break the program.
It's going to break the program because now, you know, what happens to this call
where I'm passing on a double to this X. So then you would think, "Oh, I am
doing something wrong so maybe I should just take the union off parameter
types and infer X to be star to void." But this doesn't make sense. It may work for
this example but it makes me nervous. Right? Why did you do that? Yeah?
>>: I just noticed something: you said break the program at this call, when you
call X with 0.42. I mean, the inferred types, how do they manifest themselves?
Are they inserted into the program as runtime type tests?
>> Avik Chaudhuri: You would start assuming that X would only hold functions
that expect ints and, therefore, you might use this information to convert things
only ints and so on. So the fact that you are inferring this type would then mean
that the compiler is free to treat X as a function, as holding functions that int
[inaudible]. And who knows what it might do with that information? It might wrap
around that function with an early conversion to int, for example. And that would
be completely wrong because then this double would be converted to an int
before being passed into the original function.
>>: But you're thinking about runtime not static check, right? Not [inaudible]?
>> Avik Chaudhuri: So I'm talking about inferring information out of this program
that a compiler could use. Right? And so...
>>: The confusion might be that I thought you said you were going to not
prescribe but describe. So I thought it was perfectly okay to say int to void as
long as you run the if branch. And you can determine that dynamically and then
optimize X for the int to void case. And so, that's perfectly good type depending
on the context. So you seem to be saying now you're prescribing. I mean, so I'm
a little confused. It sounds like you're going back.
>> Avik Chaudhuri: So I do want to infer some things, some function type for X
that is static because I want to compile my program, right, based on that
information....
>>: [inaudible] -- I mean, because if we're thinking in the context of git, I can
think of specializing it in the then branch.
>> Avik Chaudhuri: In that case I would split the variable X into two variables,
yes.
>>: So, I don't know, it seems to be when you're just describing, it's in the eye of
the beholder and I think that's a perfectly good type as long as I'm in the then
branch.
>>: I guess the way I understand it, you are saying that you want your
description to be a sound approximation of all runtime [inaudible] off that...
>> Avik Chaudhuri: Yes.
>>: ...off the value that you describe at the time.
>> Avik Chaudhuri: Yes.
>>: And then if you describe it as int to void, your git may allocate just one
word...
>> Avik Chaudhuri: Yes.
>>: ...on the stack for the argument...
>> Avik Chaudhuri: Yes.
>>: ...but if you pass 0.42 it's going to...
>> Avik Chaudhuri: Yes.
>>: ...[inaudible]. Okay.
>> Avik Chaudhuri: And if you're not getting a static approximation that you can
use in your static compiler then really there's no point in doing this exercise.
>>: So how do you describe soundness less formally then in the system? You're
not formalizing the git and everything are you?
>> Avik Chaudhuri: Soundness is only defined to the extent that if your program,
before inference, ran for some input then it should continue to run once you have
inferred these types.
>>: So these coercions that you're inserting have some runtime effect.
>> Avik Chaudhuri: Yes.
>>: And if you insert...
>> Avik Chaudhuri: Yes, so you would assume that -- Yes, you would enforce
the static types that you just got either by inserting more and more coercions or
by breaking function coercions into, you know, argument coercions and return
coercions at function calls and so on.
>>: And in this case if you input int to void and you insert the coercion around
the 0.42...
>> Avik Chaudhuri: Then you...
>>: ...a float to each coercion would fail or...
>> Avik Chaudhuri: Yeah, you change...
>>: ...change the semantics or something.
>> Avik Chaudhuri: ...the semantics. Yes.
>>: I think the real problem here is that you're getting this type. So the least
[inaudible] of these two functions is not int to void because my dynamic
[inaudible] are not compatible. They're completely different types and when you
intersect them you get the empty type. So the least upper bound here is the
function that takes nothing to void, and so that's the problem.
>> Avik Chaudhuri: Yeah, you could think of star as a completely different type
but you could also think of star as a union.
>>: [inaudible] all types because at runtime it's not compatible if you pass an int
versus [inaudible], it's not the same.
>> Avik Chaudhuri: That's an implementation detail, I would think.
>>: [inaudible] the coercion at runtime to go from an int to a dynamic thing, right?
So this is why this isn't working here. So you have to be careful about your loads.
>> Avik Chaudhuri: You could -- So I would argue that this example would also
work if you take something else rather than star.
>>: [inaudible] then it's obvious because then you're going to get bottom and
you're not going to be...
>> Avik Chaudhuri: No, if you take this to be A and star to be object, but it is
some class, then this example would also work. So you're right but you're
partially right. Yes, star at runtime implementation-wise it could be thought of as
a different type, not a super type of end. But for the purpose of this example, it is
perfectly fine to think of star as a union type. And you can replace that with other
kinds of things and that one would still hold.
>>: So are you aware of early work on Lisp and dynamic typing [inaudible]...
>> Avik Chaudhuri: Soft typing.
>>: ...exactly the same problem.
>> Avik Chaudhuri: Soft typing.
>>: No, not soft typing. I'm thinking of [inaudible] work.
>> Avik Chaudhuri: [inaudible] work was, as far as I understand it, it was doing...
>>: It was doing exactly this. It was thinking of every place as a coercion from
either an un-boxed in to dynamic or vice versa. Right? And then they would just
infer things and figure out how to get the minimum number of coercions.
>> Avik Chaudhuri: Yeah, I have to remember what the differences are. It's
certainly compared in the paper somewhere. It was my understanding that they
were doing -- the guarantee that they were shooting for is slightly different. It's
not the same guarantee.
>>: [inaudible] as an optimization.
>> Avik Chaudhuri: But it was more restrictive. It was not as --. Okay, so going
back to star now where you're just trying to get a solution that works by taking the
union of int and star and for now assuming that star is the union of all types and
representations do not matter here.
What you're getting is at best something that is imprecise and at worst something
that is potentially doing unsound reasoning. The real type that you want to infer
here is number to void, I would argue. A number is the type that we have for
double. The reason is that at runtime if you have to precisely capture the
behaviors that were there in a program to begin with, at runtime all you’re doing
is [inaudible] checking, right? So the only way you can enforce a function type is
by checking that it is a function and then, depending on the uses of its parameter
and the uses of its return type, very lazily you would enforce types on the
parameters and the return types. So what is important when you're trying to
preserve behaviors is to look at the ways in which that function is called at
runtime because that's going to give you the most precise description of your
program. You have a question.
>>: [inaudible] some kind of escape analysis [inaudible].
>> Avik Chaudhuri: I'll come to that, yeah. That's perfectly true, yeah.
So, you know, this is the type that we would like to have, and we now need to
formalize how we arrived at this. There has to be a systematic way in which you
have to derive these types. So definitely the solution of X is a function type. So
let's split X into X question mark arrow X bang where X question mark is a type
variable for the parameter type and X bang is the type of the return value.
And because we want to adhere to this principle that you want to infer everything
based on inflows because inflows are going to give you the tightest description of
how these variables are defined, we will infer both parts of this function type
based on their inflows. And inflows of parameter types in particular correspond to
function applications of the thing that is type X.
So we're going to introduce a kind of every type constructor essentially. So you
have zero type constructors like numbers and Booleans and so on, but you also
have functions and objects. And so we are going to introduce this kind of pattern
for every kind of type. And then, all parts of this type are going to be solved
based on inflows.
Okay, and this is very related to a notion of naïve subtyping that was invented by
Philip Wadler and Findler a few years back. And they were also studying gradual
type systems, so it's not a big surprise that we arrived at the same, more or less,
concept from a different angle.
So if we go through this example once more, here I am splitting X into this
function type. And then, I am applying the usual contravariant and covariant rules
for splitting the function types. And here I generate some more coercions based
on the function calls, and eventually I will get the parameter type to be number
and the return type to be void. Okay?
>>: Sorry. So when you say you're using the contravariant rule, apparently
you're not, right, because you don't care about outflows. Right? Or are you just
not going...
>> Avik Chaudhuri: So...
>>: You're just going to ignore those in the end?
>> Avik Chaudhuri: That's true but they could, in turn, generate some more
coercions which would lead to more splits and more flows. So I am only going to
ignore outflows when I am deriving the solution not during closure computation.
>>: Okay, so [inaudible] second to last know that X question mark has to be less
than what flows into int. How is that used during the closure computation?
>> Avik Chaudhuri: That is not used because that's kind of a leaf constraint. But
if I...
>>: I see. So if it's higher order taking a function argument, that's where
[inaudible].
>> Avik Chaudhuri: Yes.
>>: Okay. Got it.
>>: But the outflow is the inflow to something else.
>> Avik Chaudhuri: Yeah, so if something else flows to an X and an X flows to a
Y then I would treat this to flow to Y.
>>: That's with the closure.
>> Avik Chaudhuri: Yes. But I would typically ignore constraints of the form X
goes to a function type because I might have had two different things flow into X
under two different conditions. So the additional constraint that X is used as a
function is not that important. Right? You could be reasoning outside the type
system and you don't want that to influence the solution.
However, if there was a function type here and this is used as a function, it could
be that I still want the parameter types of this one to flow into the parameter
types of that one and so on.
Okay, so going back to our next observation. Now let's see what happens with
public functions here. By public function I mean a function that is callable from a
context outside your program. So I'm using this notation again: parameter type of
foo is foo question mark and the return type is foo bang.
So we go through the motions; you get number to number as the inferred types.
But now you can break this program, right? So say you have some external code
-- So let me go through this function a little bit.
Here you are checking a global Boolean variable and depending on that you are
either adding one to X or your returning zero. Okay, so if you set B to false and
then pass through to foo, you will go down this path and your program will work.
Right? So you're passing Boolean to something that was explicitly dynamically
typed, as I said was boxed, so everything works here.
However, in the code that is generated on the right this will fail very early
because here I'm going to try to convert a Boolean to a number and that's not
going to work.
>>: So I have a question about ActionScript. So it seems to be more strict that
JavaScript?
>> Avik Chaudhuri: Actually, yeah. So this is a lie. There would be an implicit
coercion here. So it corresponds to JavaScript semantics. So choosing Boolean
here was not the right thing to do.
>>: [inaudible] cluster and add a string and a number in JavaScript...
>> Avik Chaudhuri: Yes. You can do the same here. So a simplification of
ActionScript is not -- Yeah.
>>: [inaudible].
>> Avik Chaudhuri: Right. So we're assuming that, yeah, all the primitive types
are not [inaudible].
>>: [inaudible] basically talking about [inaudible] byte code because you have
[inaudible]?
>> Avik Chaudhuri: Yes.
>>: [inaudible]?
>> Avik Chaudhuri: Yes. So...
>>: Can you talk more about that because I'm just sort of missing that
connection. Because through JavaScript, which we are accustomed to, we don't
have byte code; there is just source and execution.
>> Avik Chaudhuri: Yeah. Sorry. So I should've made that connection. So the
ActionScript looks like JavaScript plus types at the source level but then it's
compiled by this compiler. It's a very simple compiler, just convert everything to
byte code. And that's how things are shipped around as Flash applications and
then, there's a VM that [inaudible]
this byte code and [inaudible].
>>: But are you thinking about -- I mean you're talking about sort of sending stuff
off to runtime. Does that involve changing the byte code?
>> Avik Chaudhuri: Yeah, so this compilation phase would be implemented in
the source compiler.
>>: I see.
>> Avik Chaudhuri: And you would generate [inaudible]...
>>: Okay, so it's not [inaudible]...
>> Avik Chaudhuri: [inaudible].
>>: ...it's part of the compile.
>> Avik Chaudhuri: Yeah. All right. I just showed you an example of why this
entrance is unsound. And the reason is that, yes, we were relying on inflows to
determine our solutions. But obviously here we are not seeing all inflows into the
parameter type of this function precisely because that function is a public
function, so we of course we cannot analyze all sorts of external code that will
use this function.
>>: Is it only public functions or like JavaScript, does ActionScript have a higher
order store?
>> Avik Chaudhuri: It also has that. Yeah, I'm going to -- You're moving ahead
of...
>>: Okay, sorry for that.
>> Avik Chaudhuri: Not sorry. So this is exactly where this is going next. So
where do we not see all inflows? Formally these are type variables that are in
negative positions in the type of the program. So if you think of your program as
kind of a module that is exporting a lot of functions and objects and so on, a
public function is something in that module signature that is at the top level and
the parameter type is in a negative position in that type. So these types are
unsafe to infer so we have to be very careful about not trying to infer these types.
Either we require the programmer to provide the explicit annotation or we just
infer these to be stars.
So these are kind of the roots of our analysis and everything else flows from
these things. So if you want to control these roots, you have to explicitly annotate
these positions; otherwise, we'll just want to infer star. Okay? However, we can
do better for local functions. So this is a local function; foo is a local function now
that is wrapped inside bar and now we can see all calls to foo. Okay? So here it's
perfectly sound to infer number to number because this function cannot be called
from external code. However then, there are local functions that escape. So here
I am calling this function but also returning foot, and now if you are leaking these
local functions from public functions then they are as good as public functions.
So again this is unsound now because I can construct a similar counter example
to break this code with some arbitrary context. So this would mean that perhaps
we need some escape analysis, right? And this is getting more and more
complicated and tedious. However, fortunately we do not need an escape
analysis and that's one of the good things about this technique. So if you go
through the motions and generate all the constraints and the right form then you
will notice that some variables -- So this is a complicated expression that says
that I'm talking about the return type of bar which is this and the parameter type
of whatever bar is returning. So if you look at this code, bar is returning a function
and this is the parameter type of the function that is being returned.
And by rules that is in a negative position as a type of the program. And because
it's in a negative position now because have set up our -- you know, the thing that
Tom was talking about, right? Two function types, the parameters are flowing
back and forth. This thing, by annotating this as star or inferring this as star, this
will flow back some more constraints and eventually you will infer the parameter
type of foo to be star.
So this is safe, right? So we do not end up inferring the parameter type to be
number because that would be unsound.
So this is a good thing because this does not add any more weight to the system.
The system remains pretty lightweight. You just have to have a position-based
restriction on the type of the program plus some sound rules for closure
computation and that suffices.
So objects are similar. You have potentially unsound results if you don't take into
account things that escape. So it turns out that writeable fields of objects escape
via the constructor so they have to be treated as -- Yeah?
>>: So you're assuming that if a structure is actually finite -- otherwise, it can't do
this kind of expansion. So what -- can't you write functions that take an arbitrary
number of arbitrary argument type functions?
>> Avik Chaudhuri: I would not go into inferring the types of those functions.
>>: [inaudible]? I mean if you just run this algorithm you will never terminate
because you constantly build up bigger and bigger functions. It will keep splitting
a variable into the bang version and the question mark version and then
recursively figure out, "Oh the question mark is again a bang and a question
mark," and so on and so forth.
>> Avik Chaudhuri: So the rules dominant because there are -- So at every step
of the closure computation something is going down. We guarantee that and
there's a proof in the paper to that effect.
>>: [inaudible] Because I mean clearly your rule is that if you have some function
flowing into a type code, right, you have to split the type code. Right?
>> Avik Chaudhuri: Yes. So...
>>: [inaudible] during the closure computation, right?
>>: Avik Chaudhuri: Yes.
>>: You have to prove that you're not going to split them indefinitely, right?
>> Avik Chaudhuri: So in typical transitive -- So we do not compute a full
transitive closure. Again, I would reference you to the paper to see this. What
happens is that because these X's are only split into their shapes and the shapes
determine how things flow across functions. So if there's a function type that
goes to an X and that X goes to another function type, you first split that X into X
question mark and X bang and the flows go through that. And you stop when you
have mismatch on the two sides.
So it's kind of stunted. You don't take either cross-products across type variables
which is also why we have one order less than the usual running type. Our
running type is not [inaudible]....
>>: [inaudible] Because take the Y combinatory, okay? JavaScript, you can write
the Y combinatory, right? There is nothing that prevents that. [inaudible] has an
infinite size type if you're to expand it. So I do not see how you're getting
[inaudible]. If you're trying to type these programs using this type structures, I
don't see how you do it.
>> Avik Chaudhuri: Can we go through that...
>>: Yeah, we can do it...
>> Avik Chaudhuri: ...example offline...
>>: ...offline because this doesn't seem [inaudible].
>> Avik Chaudhuri: Okay. Okay. I can more or less guarantee that we will stop
early and assume...
>>: [inaudible]
>> Avik Chaudhuri: Yeah, yeah, yeah, so we'll go through the rules to see that.
So the time complexity of this algorithm is N-squared rather than the usual bound
of N-cubed. This is precisely because we do not take a full transitive closure and
we are kind of clever in the way we terminate transitive closures.
And we prove that this preserves the runtime semantics and programs do not
break and they can be composed with external code. All right. So in order to
evaluate this: so this is a very early evaluation. We later implemented this in the
ActionScript compiler, so the results that I'm showing are old results where we
ran this through a bunch of benchmarks.
So this is how we set the experiments up. We have fully typed versions of these
benchmarks. We also have what we call partially typed benchmarks which are
basically we take the fully typed benchmarks and we only retain the annotations
that are required for our analysis which is annotations at negative positions. And
then, we run this algorithm.
And on an average we had a lot of improvement over these programs, and many
benchmarks we actually recovered 100 percent performance of the fully typed
benchmarks. So the promise here is that you can leave out these types and still
get the performance as if you fully annotated your programs. And in some cases
you also get better than fully typed benchmarks because we manage to infer
something that is more precise and more compact than the annotation that we
had.
In some cases we are hopelessly bad. For example, for [inaudible] we really can't
guarantee anything that comes out of an ActionScript area. It was just more or
less like [inaudible]. Yeah?
>>: I'm a little surprised by your benchmarks [inaudible]. I mean not only are they
relatively small, these are also fairly numeric in nature which is not entirely
common if you look at client-side JavaScript. And there are benchmarks in their
V8 suite that are fairly different from like [inaudible]. So like if I want to run this in
JCreator, what would come up?
>> Avik Chaudhuri: That would be possibly the subject of the next part of the
talk.
>>: Okay.
>> Avik Chaudhuri: So here we are focusing more or less on game-like
ActionScript programs which are either fully typed to begin with and use a lot of
numerics and so on, and they don't use a lot of prototype chains and lots of
dynamic constructs that JavaScript has. Those things do exist in ActionScript, as
a subset, but most people use the classes in ActionScript and that kind of subset
which is better behaved than for JavaScript.
>>: So you said JavaScript programs border to ActionScript...?
>> Avik Chaudhuri: Yeah, so these...
>>: And then are annotated with types?
>> Avik Chaudhuri: So these are part of the ActionScript VM benchmarks which
evolved from being a JavaScript VM to an ActionScript VM so these benchmarks
still exist there. Yes, and some of these have been portered to use classes
instead of generics and so on.
We also don't do well with numbers versus integers but some range analysis can
improve that. Okay. How are we doing on time?
>>: Good.
>>: We have time.
>> Avik Chaudhuri: Okay. All right. So in summary, this is a technique that
improves the performance of existing Flash applications, and the key idea here
was to consider only inflows for solutions and infer solutions by parts for higherorder types. And this is backward compatible, and the way to do that is only by
having a very lightweight restriction on the types that we can infer or the parts of
types that we can infer. And escape analysis that seems necessary for this
analysis is actually not necessary because it's encoded by the way we compute
flows.
Okay. So this is fine but as we will see, as we move to the second way in which
dynamically typed code appears not everything can be solved through static
analysis. In particular if you're deliberately trying to go outside the type system,
there are ways in which you can break the static analysis very soon. But we
would still need static typing I claim.
And there are many reasons to desire static typing in these languages. I would
want to check for type errors or all paths and keep documentations synchronized.
I also want to ease code maintenance and know how to call functions. Of course
I want efficient implementations and that's always there.
Okay. So the kind of the static analysis that I just described fails miserably when
you throw very dynamic constructs at it, right? In particular things like eval and
prototypes just don't go very well with the kind of static analysis I showed.
It's also true that a lot of libraries use these dynamic constructs. Essentially they
do a lot of metaprogramming with the dynamic language. And libraries could
highly benefit from types. So it turns out that the kind of code that would defeat
static analysis would benefit the most with, you know, inferred types.
Now the idea behind the second piece of the talk is that we can do dynamic
analysis for these kinds of programs. And the saving grace is that dynamic
programs come with lots of tests, so we can exploit these tests for doing precise
analysis. By observing only the feasible parts through your program and then
trying to abstract whatever observations we make by generalizing these types
and predicting type errors.
So this is a small function that we'll use as a running example. I'm switching
notation here because this is a different paper and we used different notations,
just to keep notations consistent with that paper. Here I have things called alpha
which denote type variables. So there you have a formal argument X and the
type variable for that formal argument is alpha X. There's also a corresponding
type variable for the return type of bar.
So the key idea behind this dynamic analysis -- So what are we trying to do? We
are going to take a program and we are going to instrument it to observe things
as it's running. So one of the key concepts here is one of wrapping a value. Let's
say that a value, V, is something that you have at runtime. There's also a
wrapped value which is also a value at runtime which is a value, V, wrapped with
some type variable.
So you can think of this wrapped value as some sort of proxy that more or less
behaves as V but it also carries along this type variable with it. So type variables
express some static invariance and the values wrapped with these type variables
are carrying these static invariants. And we want to make sure that this wrapping
has some meaning.
So there are two dual operations here: one of constraining and one of wrapping.
So how does this work? Let's look at this code. Start with this call here to bar with
the new instance of C. So we create an object of type C and this now flows to X,
the parameter of bar. So X now contains this object. So I insert this constraint
now which says that the type of whatever X contains has to be a subtype of
alpha X which is the type parameter for X. And because X at runtime actually
contains a C here, I'm going to insert that coercion, C to alpha X.
The next step now is to wrap this object. So recall that we are at the entry point
of bar. We have just inserted a coercion. Now we want to wrap this value that just
came in with this abstract type, alpha X.
So the reason we are doing this is that we now want to monitor wherever this
parameter is flowing and we want to capture the fact that this is typed alpha X.
So the subsequent constraints that we generate are constraining alpha X rather
than the concrete type of C. So this becomes important because in the next step
we are calling foo and then, we insert the [inaudible] coercion that says that X
has to accept a foo method.
And the type of the X now note is alpha X because we have just wrapped this
value with alpha X, so the type of that value is now this abstract type alpha. Then
we go to the return value which is 7 and we again admit a coercion that says type
of 7 is the type of the return value. Again, that's concrete up to this point. But in
the next step, we wrap this value with the type variable that denotes the return
type of bar. And from that point on we will track how this value flows out, right?
So the return value that comes out of bar will go to Y and from that point on it will
carry this invariant that it came from return of bar.
>>: Can you double that so that you have something that's already wrapped
[inaudible]...?
>> Avik Chaudhuri: Yes. Yes, you can double wrap. That's the next slide.
>>: Okay.
>> Avik Chaudhuri: So these are the subtype constraints that you generate,
right? And this is the picture that you were alluding to, right? So this is more or
less how things look at runtime, how code looks. You have a value that is
wrapped by an alpha X that's wrapped around alpha Y and so on. Now the thing
to notice is that we always constrain a wrap at the same time. So when we
wrapped V with alpha X we also did constrain the type of V to be a subtype of
alpha X. The next time we wrap it with alpha Y we say that alpha X which is a
type of the wrapped value is a subtype of alpha Y.
And the next time we wrap we do the same. So this is kind of a chain of things
that we would generate and that's the meaning of wrapping essentially. Now it's
also true that you can optimize this and just keep around one level of wrapping.
So this is a summary of what happens at a function call. Say we have a function
MX with body E and it's called with a value U. We constrain the argument and
wrap the argument, and we evaluate the body. We constrain the return and wrap
the return.
And this is how, you know, the program instruments itself and carries on, carrying
essentially these type variables as the types of most values in the program as it
runs.
So once we finish our execution with some test, we would get this system of
constraints and we can find most the general solutions for these things. How we
solve this is not important in this work because, again, you can take a transitive
closure and this time we decided to try to infer some polymorphic types. So we
take both lower bounds as well as upper bounds into consideration and we try to
infer polymorphic types for functions to some extent. These are not the best
polymorphic types that you can infer. And it's a fruitless exercise anyway.
>>: Do most general types exist?
>> Avik Chaudhuri: No. They don't exist. So we have a pattern that we try to
satisfy which is first infer the return types in terms of the parameter types and
then generalize the parameter types as far as possible. For fields it's
unambiguous which one to do so we pick one; we pick the lower bound. And we
also check for consistency in this one. So we actually check whether the outflows
correspond to -- So we actually check whether the outflows are consistent and
this gives us some type errors so we can suggest to the programmer that maybe
your program is not well typed. So the good features of this are what I want to
focus on which are: one, path sensitivity. So if you are deliberately trying to go
outside the type system you may have code like this which under some
conditions you treat something as an integer and some other conditions, you
treat the same thing as a string.
A standard static type inference would completely get confused because it would
see that there's a potential for an integer being used as a string, so you would
give unsatisfiable constraint system. So you would say that this doesn't work.
However, if you look closely this actually corresponds to an infeasible path
because if P holds in this program, we consistently use it as an integer. And if P
does not hold then we use it as a string.
So the dynamic type inference technique that I just talked about will actually not
be stuck with this inconsistency. So we would go down one of the paths and
things would be completely consistent down that path. In our other test we might
go down a different consistent path and the constraints that we would generate
would be also consistent. So eventually we would infer something like Boolean to
number for this function which is exactly the type you want.
One might think that there would be an exponential blow up in the path coverage
that is required for this to work; however, because we are doing a modular
analysis, for every function we have a type parameter and a return value and we
are not doing like inter-procedural analysis.
We kind of contain the path coverage. So here we have two functions, and
potentially you would think that there are four paths to be covered here.
However, with one test you could cover one side of that branch. With another test
you might cover the remaining paths and that's sufficient for soundness, again by
some definition of soundness that I will talk about informally here.
Further tests are redundant, so they are not driving your program through
different paths. And so this is good because what we are doing here is dealing
with type variables rather than concrete types and that kind of generalizes the
observations a little bit so that you don't need an exponential number of tests to
cover all paths in your program.
>>: Then you don't cover many paths ever, right? In...
>> Avik Chaudhuri: That's true.
>>: ...[inaudible].
>> Avik Chaudhuri: In particular...
>>: [inaudible]
>> Avik Chaudhuri: ...if those paths are infeasible, you would not cover them.
You may not also cover some paths that are feasible but are not tested.
>>: Right. But [inaudible]...
>> Avik Chaudhuri: Yeah, so that...
>>: ...[inaudible]...
>> Avik Chaudhuri: That's a price to pay for this. Sorry, random numbers you
said?
>>: Right. Like if the random number is 1.2 then...
>> Avik Chaudhuri: Yeah.
>>: ...you know, do this [inaudible]....
>> Avik Chaudhuri: Yeah. So there's a caveat there. Of course you're not going
to potentially cover some paths. However, the cover is that we require here is
sort of very abstract coverage. It's fine to cover the control flow graph of your
program to some extent. And the soundness guarantee that we will get -- Of
course you can't even think about soundness in this inference, right? So we
managed to prove something that is a qualified soundness theorem which says
that if your subsequent executions follow the abstract paths that you covered
during your test phase or training phase then you would be fine. So the
generalization that we do is sound, but you don't get an absolute soundness
guarantee.
>>: What do you mean by the abstract paths? I mean, you only observe
concrete paths.
>> Avik Chaudhuri: Concrete paths, but the information that we generate on the
side, the constraints that we generate are abstract, right, because we did this
wrapping of these concrete values with these type variables so the constraints
that we generate involve these type variables for the most part. Which means
that we exercise both paths here, for example, we will get some very general
constraint about the parameter type of bar and the two users that correspond to
the two types. So the type that we'll infer for the parameter type of bar would
cover those two abstract users in terms of how they [inaudible].
>>: You mean edge coverage? Because you keep...
>> Avik Chaudhuri: Yeah, I don't know what...
>>: ...saying path coverage but obviously you're not covering paths. [inaudible]...
>> Avik Chaudhuri: Yeah, so...
>>: ...so do you mean edge coverage?
>> Avik Chaudhuri: Maybe, yeah. I don't know exactly what term [inaudible].
>>: So in this example you had two paths...
>> Avik Chaudhuri: Yeah, edge coverage is what I meant there. Yeah.
>>: Okay. [inaudible]
>> Avik Chaudhuri: Yeah, yeah, yeah.
>>: But [inaudible] you're saying that if you follow the path, exactly the paths that
you saw in your training phase then your types are sound for those paths. But if
in this case, for example, I went left first and then right...?
>> Avik Chaudhuri: Yeah, I should have said edges probably. That was -- So
inside a function, I do mean that you need to cover all paths though.
>>: Whole paths? Not just edge coverage?
>> Avik Chaudhuri: Yeah, inside of functions. This is across functions you can
mix and match paths across functions.
>>: [inaudible]
>> Avik Chaudhuri: Yeah, so in the size of individual methods this is still
exponential. You can do whatever you want to do, right? So you can potentially
just exercise one path through your program and your soundness here will then
hold for just that path.
Fields obviously can propagate state across functions, so you can [inaudible]
break this technique with fields if you're not careful. In particular this is an
example where you can assign both a string and an integer to a field at different
times. And if you have just the right sequence of calls in your testing phase, you
would potentially infer something that is unsound. You would infer the field to be
an integer here because you called bar then qux then foo which means that you
wrote an integer and you used it as an integer. Again, this plus is not polymorphic
in this example.
However, you can then break the program with a subsequent test. You know, the
execution remains in the set of paths that were covered but you're still not
typable. So the solution of course is to constrain and wrap fields as well. So you
have to intercept field reads and writes just like you intercept method calls and
returns. So you can think of every field read and write as an access or call.
So this is the soundness theorem that we have which is if the tests cover all
feasible paths per method then the inferred types are sound for all executions.
You can weaken this, you can qualify this by coverage, by saying that as long
executions fall under the abstract paths through your program, the infer types will
be sound.
Another observation that we made when implementing and testing this is that
most types turn out to be path insensitive. So even when we did not have full
coverage we inferred types that were still correct. So this is -- Yeah?
>>: I have a question on this. So because a lot of these operations are
polymorphic, right -- So plus for example -- isn't it the case that even if you cover
all paths of a function what you really need to cover all paths even of the
polymorphic operators? And that seems much harder to actually achieve, right?
>> Avik Chaudhuri: That's true. Yeah. So when we did this work this wasn't done
in the context of Ruby and from that point on I didn't rethink this in terms of
JavaScript. I think many more problems need to be solved in order to apply this
to JavaScript.
>>: [inaudible]
>> Avik Chaudhuri: Yes. But for override it has classes, and we added some
more rules that would check consistency of overrides, right, just like you would
do in most class-based languages. So that got rid of the problem for Ruby. When
you have arbitrary inheritance change then it's harder to see how that works.
However, you could do it, right, and make some more assumptions.
All right. So one of the cool things about the implementation which I'm not very
proud of but it may seem cool to some people is that because Ruby is highly
dynamic you can use it to instrument itself. So the implementation was [inaudible]
you don't need like a whole static analysis to set up a whole static analysis
framework to implement this.
And similar thing could be done in JavaScript as well probably after solving some
other problems. So because we were using Ruby to instrument itself, there was a
lot of proxies being generated. So this is slow but of course this is not going to be
the program that you will ultimately run. You're just going to use this infer types
and that's going to be one-time thing probably.
But most of the types that we inferred were correct even though we do not have
full coverage in most of these programs. We also managed to find a type error.
Okay, so in conclusion we have static analysis and we have dynamic analysis.
And as I mentioned we are just getting started on JavaScript, and I think that a
combination of both techniques will be necessary. And this is almost like a
[inaudible]; I don't want to take ownership of this idea of course.
On the static analysis end, especially with tooling, because Adobe is a tooling
company we are in the web browser business, we don't want to analyze any
JavaScript that you want to throw to us. We can make your JavaScript programs
run through our tools and, you know, check for errors early on and force you to
write in certain ways and so on.
We can imagine that static analysis would be very fruitful. There are more and
more sophisticated analyses that we can use. So one particularly interesting one
is Dimitris Vardoulakis' work on CFA2. He's probably trying to introduce that to
Google Closure compiler. And this seems to work pretty well, especially with
JavaScript programs. Some benchmarks set up inheritance hierarchies using
prototypes but they are not crazy programs; they are dynamic programs but they
kind of stabilize after a while. CFA2 works pretty well for those kinds of programs.
Dynamic analysis can fill in the gaps. It's actually used in a very rudimentary form
in most gits today. Right? So most gits work, JavaScript gits work by running a
program for a while and based on the profiles that you generate, you speculate
on types. So there is hope that the kind of dynamic type inference technique that
I just talked about could be used to generalize those profiles.
There's also the view that dynamic checks can always support unsound static
analysis. And this is the subject of Brian Hackett's work in Firefox where his
argument is that JavaScript is just too hard to do sound static analysis on, but
that does not mean that you cannot do unsound, yet useful, static analysis and
then, you know, add enough checks to support that unsound analysis.
Future directions that I'm looking at: one direction that is particularly interesting is
whether you can compile JavaScript ahead of time. As I said we have tools; we
can force people to write programs in certain ways. But it would be really cool if
you could force people or have people write games in JavaScript. JavaScript is a
nice function programming language, if you think about it. It's not very crazy.
Some people completely hate it, but I think if you just use the functions and
records part of JavaScript, it's actually not that bad. And most games are written
in that form anyway. So the idea behind this project is can we do the same kinds
of things that a typical JavaScript git would do but only broaden the scope of this
speculation? So typical gits speculate and add guards all over the place. If you
do some more or less sound static analysis, you can eliminate many of the
guards and you can pull up the checks early on so that you can pre-specialize
functions to either go down a slope or a fast path very early on.
So that's some of the ideas that we are exploring in that project. The other idea
which I don't know how it's going to pan out is, as I said, try to apply this dynamic
type inference technique to generalize the kind of speculation that you do in git.
And I'll be happy to brainstorm or collaborate.
[applause]
>> Ben Livshits: So I mean, I think you'll stick around until we go to lunch. Right?
>> Avik Chaudhuri: Yeah.
>> Ben Livshits: So if you have questions, feel free to engage with the speaker.