1
>> Tom Ball: Okay. Welcome, everybody. I'm really pleased to have Gabriel Dos Reis
here from Texas A&M where he's a professor. He has a very interesting history. He has
a Ph.D. in mathematics from Université Paris-VII, sorry, and École Normale Supérieure
de Cachan, France. And his research interests range from mathematical software, applied
formal methods, programming languages and libraries to generic programming.
In 2012 he received an NSF CAREER award for his research in compilers for dependable
computational mathematics. He's an active member of the ISO C++ standards
committee, and he's authored and coauthored several features of the 2011 ISO C++
standard as well as doing research in axiomatic programming and his OpenAxiom system
for scientific computation, and also at some point being the release manager for GCC.
So can you imagine all of that in one person. Here he is to tell us about his recent work.
>> Gabriel Dos Reis: Thank you much, Tom, and thank you for inviting me to the
lecture.
So, yes, when people ask me what I do, I say I do programming and I say I also do math.
And I'd like to tell you today why I believe that doing programming is actually doing
math. For a moment I have to qualify, I have to say doing good programming is doing
math, but eventually I hope that we'll just say programming is math.
So in this talk essentially I want to give an overview of what I'm doing in programming,
[inaudible] math, and I started from computer algebra. And it is all connected. And I'll
try to explain and try to convince you that, yes, this is very connected, and then I'll focus
near the end on my -- one of my recent projects, which is Liz. It's an experimental
language for doing research in structured generic programming.
So I started as a student working on geometry and then get seduced by using computers
to do geometry and then quickly spent too much time in a computer room and now I'm
going to back again.
Okay. So I did internship [inaudible] at INRIA working in European Commission project
called Frisco which was a project to write libraries to solve huge polynomial systems that
people found in science and engineering.
And I was mentored at the time by Victor Pan, who is one of the experts on structured
matrix computations and so forth. And had to work on the one of the fastest algorithm
for uniform approximation of univariate polynomial roots.
And at the time, first of all, the project consisted a few working at INRIA, Sophie
Antipolis [inaudible] northwest and people in Italy and Spain. But essentially we had
only two implementation languages, Fortran and C++. So I didn't really find myself very
comfortable with Fortran so I chose C++. And this '96.
So you have to remember at the time I had GCC, but it was horrible. And the other
choice I had was Sun Microsystems compiler, CC. It's pretty good. And in the end I end
up actually reimplementing a component of the C+ standard because it wasn't part of
GCC and I need it to get the library done so that my colleagues could use my library
without having to reinstall everything.
2
And while I work on the implementation of valarray I found several bugs, and the only
way I could actually get them fixed in the center was to join the AFNOR group, which is
the equivalent to the NC group, or French group, and then no [inaudible] valarray, GCC,
and as a punishment they asked me to become a tenure and later become Liz manager
from 200 until 2007. We stop because can't be professor and Liz manager at the same
time.
But what I got out of it was real interest in supporting generic programming. Because a
lot of computer algebra and using C++ of course templates, discover that you need to do
something different. And the reason why I got in GCC was I thought, well, it's just a
software so it should be fixed. And that turns out to be a bit more complicated than I
realized at the time.
Okay. So I'm going to show the connection between generic programming with C++ and
computer algebra.
So generic programming is essentially methodology for developing reusable form of
libraries, and you want efficiency and really want to parameterize the components so that
you can reuse them and you want to manage complexity.
And so my background is in computer algebra where we have [inaudible] mathematical
structures so it is essential that we find a way to manage complexity.
So since you're here we have one function called find. I can use it in a vector, I can use it
in lists, I can use it in any dimensional sequence. And it just works the same. I can use
the same wisdom working on vectors or list.
Now, why is that important? This is important because it is a principled way of
constructing software. We need principles. We need things that scale.
And the other thing is that actually it is not just some [inaudible] model methodology
[inaudible] it's just not mechanics, playing with for all or exist. It is there, real code.
You know, you here at Microsoft, Adobe, Google, Bloomberg, A9. One of the things
that's common with Adobe and A9 is that Alexa Stepanov who invented [inaudible]
worked Adobe till two years ago and now he's at A9 and still working on general
problem and trying to get -- educate programmers to get things done.
And actually it's effective. There are many places where object-oriented programming
[inaudible] in the '90s failed where using generic programming actually succeeds pretty
well. Languages like Java [inaudible] started as pure object-oriented and then they added
generic as the reason for that.
Now, a connection with computer algebra. So actually computer algebra is one of the
original areas of generic programming because of the complexity in real structures that
mathematicians, computational mathematicians have had to deal with.
You can think of GCD and if you think of the Euclidean algorithm, the state of the
Euclidean algorithm is very abstract if you have Euclidean domain. Essentially a
connection of things that look like numbers, you can divide them and you can have a
3
process that tells you when you can terminate, so you have some clear termination
criteria. You can do GCD there.
Or you wanted to compute Gaussian elimination. This is what people who do linear
algebra like to do it, you can do Gaussian elimination with floating point numbers which
is kind of approximation of fields. But what if you don't have division? Well, it can still
work, you know, use Gaussian elimination on rings that happen to have the right
properties.
So you see that if you set up your algorithm and you understand the conditions and very
well be you can get a variety of complexity and very good algorithm that actually scale.
And the last thing, and I probably said before and I'll repeat it again, is that mathematical
structures are very, very complex. They're sort of the things that you learn. Math is
great, but I bet everyone would understand that there's a lot of complicated stuff out
there. And generic programming helps with complexity.
There are many ways of doing generic programming. If your language is Haskell or
more [inaudible] oriented, there's going to be a lot of good stuff out there. And there is
now several experimental systems to look at several ways of doing generic programming
from Haskell point of view.
Most of these have in common a focus on the orientation of data. Okay. And I try to
explain this [inaudible] or sometimes situation from category theory and type theory.
And this is actually what you'll see in most academic papers. And then there is the
[inaudible] aspect that where you focused on algorithm.
So as I said, if you look at computer algebra, it is mostly having to do with algorithms.
So Stepanov explained this [inaudible] STL using [inaudible] templates. It focuses on
algorithms [inaudible] when you go to industry, that's what you find predominantly, even
whether you use C#, Java, its focus on the algorithm. And I learn it from the computer
algebra community.
And the idea here is that you may have a general formulation of an algorithm, but you
can add a series of requirements and refinement to have better complexity.
So, for example, if you want to search an item in a sequence, you can do linear search.
But if you happen to have a -- if you have a single link list, then you have to do linear
search. But if you, let's say, have an array and you know that the array is sorted, you can
do binary search and there you have better complexity, on average, than you have linear
search.
The connection between data type generic programming and algorithm-focused generic
programming is still something that's done. In the Haskell community [inaudible] Peyton
Jones and Ralf Lämmel have all been trying to bridge the gap. And so we have these
paper series of [inaudible] try to bridge the gap. And Jaakko Järvi and myself trying to
bring some ideas from Haskell into the [inaudible] community because it's always good
to get infected by a lot of people.
>>: Can I ask a question?
4
>> Gabriel Dos Reis: Yes.
>>: So for the example that you mentioned about changing the implementation from,
say, a -- from one representation of an array to another, I mean, that kind of thing is very
naturally done with type classes is Haskell, right?
>> Gabriel Dos Reis: Yes. So ->>: And I wonder about your view of your statement that generic programming Haskell
is about data. It seems to be at least for type classes it's more about operations that that
data supports.
>> Gabriel Dos Reis: So if you look at type classes, type classes focus on type, not on
algorithms. The -- I'll have some slides on the contrast I want to show. But if you give
the example of sort algorithm in C++, if you want to parameterize your sorting
[inaudible] function, you can do it in Haskell, but you start getting into hoops. You have
to use multitype-type classes which are not standard.
And once you get there, you have a lot of ambiguities in how you solve that. You can try
to use functional dependencies that gets more problems. You can try to solve this
problem in Haskell, but as you do it, you're building more machinery complexity which
is making this program more and more complex.
Because at the very beginning the focus was on type, not on functions. Yes, you can use
type classes, but you are going to only go in one direction, not try and have at least twoor three-dimensional view of what the problem is.
>>: I'll wait for more ->> Gabriel Dos Reis: Yeah. Sure. We can have more discussion on this. But, yeah,
very good question.
So going to give brief example of interaction application of generic programming in
computer algebra. It's algorithmic differentiation. So we all know this thing about
differentiation when you take a calculus class, which is, you know, we have [inaudible]
law, and so it sounds very simple problem.
The problem you're looking here is different. It is that you have a program that computes
some mathematical function, and you're right to have another problem like computed
derivative of that problem. So you don't have a closed formula. You have an algorithm
that computes that function. And you like to generate automatically. And in a program
that computes it already. So that's algorithm differentiation.
So transformation phi here, if you accept that you have some semantic function that gives
meaning to your program in some mathematical domain and you liked -- and you have
the mathematical properties, they can get derivative, and you like to have a
transformation of your source program to something else. If you take the meaning of
that, you get derivative. It's what you're looking at.
5
Well, the first thing is this is not divide the difference. Meaning this is -- it is not
computing the value of a program, a function, two different point and then divide and -by the difference. This [inaudible] that scheme is very unstable and not very interesting.
It is not taking an expression tree and just apply Leibnitz rule indefinitely. Assume you
have a closed formula. This is about you mention you take the [inaudible]
implementation of square roots. Okay? And you would like to have a function that
computes the derivative of that that's done automatically, not using different equation.
So if you want to do that, the first thing you need to do is to get a better understanding of
what is going on. And you have to work at a very high level, not the concrete
representation but a very high level set of properties of your function and also have a
[inaudible] semantics of your language so that you understand how control flow is
expressed and work at the mathematical level and then design a transformation that gives
you the actual representation. So it is something that I did, I work on with my colleague
Jaakko Järvi and student Jacob Smith. He's now at Intel.
So, one of the reasons why this is actually a good thing to do is that most functions we
use in size, they don't closed form. Now, I want to take at least zero, two, or three
classes, they always come with very nice functions and they teach you how to compute
derivative.
So you move to the real world where I have to supplement, and suddenly you realize,
well, most of your functions don't have closed form. Okay? They're probably defined by
some differential equations and you devise algorithms to compute the function but you
don't have a closed form.
And sometimes, even if you have a closed form, you don't really want to present it as the
expression tree and derive it. If you take this sequence series of functions, so we have
zero, we have the identity, and then the old thing [inaudible] it's like Fibonacci function
but it is not because you have some log there.
The interesting thing about log is that when you derive it, you get quotients. And it keeps
going on. And so, well, if you derive -- start to derive computer derivative each element
here, essentially the representation doubles at every step. So you don't want to that.
Whereas if you resolve it as an algorithm that computes, the algorithm is compact, fixed
size, doesn't change. So you prefer the algorithmic representation over the expression
tree of presentation for doing the [inaudible] computation.
And the rules I actually use is not the Leibnitz rule that says essentially this thing plus
linearity. What's actually used is the chain rule. So if you mention that you have
sentiment S1, sentiment S2, you run this on the machine, essentially transformation of the
statist of your machine.
So if phi is this function in the transformation that compute the derivative, well, you have
to use the chain rule. If you compose F and G, then the derivative of this whole thing is
the derivative of F, apply what was there earlier, which is function F1.
So this is what you use in practice. This really sometimes you want to have X times Y,
6
you probably need that, but we'll [inaudible].
And this also implemented as a library. And the reason why this is very successful is that
in the OpenAxiom system we have this notion of categories supposed to be equivalent to
type classes except that they are mostly parameter type classes and well behaved. And
the whole library in Axiom is generic and structured in a very mathematical way. And
Axiom was designed to prove a point, which is that you can do structured computer
algebra with strong typing. If you take Maple Mathematica, it's not the case.
Mathematica you have expression tree and you substitute to get something. If you get the
answer, great, most every time you get something; you don't know exactly what it is.
Maple is close to that.
And it turns out that this actual paper was an improvement over what existed in Maple in
2007. I think they are beating us now. But in 2007 it was a huge improvement over
what -- yes.
>>: Are you assuming that the functions are continuous?
>> Gabriel Dos Reis: Very good question. So the question was whether I'm assuming
the functions are continuous. Not only I'm assuming that they are continuous, I'm
assuming that they can be different theories.
And if I remember, I'm assuming that the algorithm representation is very natural. So to
give you an example, if you take the identity function, you can decide that, well, it's F of
X equal return X, something like this, or you can say, well, it is return X except if X is
42, in which case you return 42 you have a sweet statement, so only you have some kind
of artificial discontinuity. Right?
So I'm not assuming any of those things. It is a restriction on the form of the input
program. In practice, it is not a big deal. Because in practice most people write functions
natural way, and if you write it in a way that is not natural, if you write it, well, I'll return
X except if X is 42 then return 42, you'll see that you get derivative -- if you apply the
rules, you'll get 1 for everything except for it you will get 0 because it's custom, right?
But, yeah, it is one of the shortcomings. And I was hoping that, you know, [inaudible]
continue and try to use ideas from abstract interpretation to try to -- kind of symbolic
limits at some artificial discontinuities. But yeah. And in practice, again, this is not a
real -- is not a big deal because most codes there are written in appropriate way.
But you handle four loops like you read in C. So things just work beautifully. It's
amazing. It's very, very simple and amazing.
And another thing is that -- sorry. The -- huh. Yeah. So a very different situation
[inaudible] numerical stable. It inherits the same array so we can actually prove a
theorem if you write down the informal rules that if you do this program transformation,
the numerical error you get is the same as, you know -- the accuracy is the same as the
original input. So it is actually preferred way. And as a matter of fact I wish that when
we teach calculus classes, we teach the algorithmic way of differentiating a program than
the formula that people learn. It is very good work in an algorithm.
7
And the rules are very, very simple. Only need one general. And you can do computer
science programming and calculus at the same time.
Okay. So next thing. How do I get from -- so during programming I'm doing that for
C++, also I did that from there to the other part of my research which is trustworthy
systems. There are just two slides because I want to move very quickly to this.
The first thing is I really like to trust my libraries. First I want to trust the
implementation of some library and I would like to trust your library. And you probably
want to trust my library and how do we get there. It is program correctness [inaudible] is
really easier if you know you don't depend on some implementation details. And that is
assured by working at the most abstract level. So you need to generate programming.
And another thing is that I actually believe that C++ is a fine representation language.
Okay. I was raised in France, so in the good French tradition I had to learn Pascal and
the next was OCaml, of course, from Guy Cousineau who among the people who -Francesco can tell you more -- people who are at the theoretical foundation of, okay,
well, so Caml stands for Categorical Abstract Machine Language, and [inaudible] but
does come from Guy Cousineau [inaudible] and so forth.
But C+ is very fine language to work with. You just have to be principled and
disciplined. So one project I have, longtime project is to try to build a formally verified
C++ compiler the same way that we built the CompCert compiler, which I think is a very
good thing. It's for both the C community and the formal methods community, which is
that, well, hey, you can actually build complex stuff and improve it to be correct. So it's
good stuff.
I would like to do that for C++, too, because the language, it's complex, but there are key
things there that I -- that I would like to see more in mainstream languages. The first is
the notion of construction/destruction. Many of these oriented language have the
construction, but they don't have the deconstruction part. They have [inaudible] and it's
messy.
Systematic reliable destruction [inaudible] resource is something very important. You
can depend on it. And it is something that I think should be preserved in official
languages. So with Xavier and Tahina we formalized that notion. We've got dynamic
location but just know the principle.
It was originally introduced by [inaudible] in early '80s, but [inaudible] to be correct, and
in the process we found some bugs in the standard. So this is one of the cases where you
get some academic paper and you fix the real-world thing. So it's really good. And then
the bugs have to with virtual function [inaudible] during construction.
In the next step I would like to formalize -- sorry.
>>: I want to go back just a little bit to the stuff on differentiation, and started to make
the connection that how does this relate to the generic programming stuff you were
talking about? So far should I understand this differentiation stuff as an compiler that's
transforming abstract syntax trees to abstract syntax trees?
8
>> Gabriel Dos Reis: So the concrete representation and implementation was on a C
library. But it works at a fully typed abstract semantics graph, not abstract syntax free. I
wish we say abstract semantic graph as opposed to ASC, because, you know, when you
have the representation, it is no longer a tree, it is really a graph. And it is typed because
types give meaning to the programs. Okay. When I have integer and I say, well, integer
satisfies a notion of ring, that's not the same as having string.
So being able to distinguish an integer from a string is very important. So you don't just
do some transformation on expression trees, you look at a type and this satisfies some
constraints which is that you have to have ring, not just a ring, you have to have a
differentiation of ring. Not only that, you ought to have an algorithmic differential ring.
So the details actually in the paper. I'll -- sorry, I was going very fast on this.
>>: [inaudible] provided a way to introspect on the ->> Gabriel Dos Reis: Yeah. So if you go to the OpenAxiom Web site, you actually
get -- I developed a library for program analysis. So it has -- it comes from the syntax.
The compiler is available at runtime. So, yeah, I didn't go much into detail about Axiom,
but it has parameterized types. It is theoretically typed. But types are instantiated at
runtime which means that you actually have [inaudible] of types at runtime if you wanted
to have introspection. Yes. The compiler is available at runtime. But the program is
aesthetically typed.
>>: But don't you worry about things like breaking abstractions, like if I find a function
and I can introspect on its syntactic structure, very week notion of abstraction in the -that is not a problem. One, you're doing symbolic mathematics. Because what you have
you see [inaudible] X is of type 8. But that's something you discover at runtime.
Interestingly, when you do math, if you want to formulize that, it is most of the time
runtime. If you want to do everything statically, you get into a lot of trouble. If you want
to integrate a function, you first need to understand whether that function, for example, is
rational, is elementary, or it has some -- defined by ODE. This is something you don't
just look at the form of the function and say, oh, yeah, it is ODE, you try to put into the
function. This is something you do at runtime by looking at this structure, running the
function over some kind of test point set of polynomials or set of equations to see which
one it's the root of, it's the zero of, and then discover that. You come back, oh, here is
actually what it is, and then you go on.
So it is a lot of dynamic thing going on. It is not -- computational automatic is not static.
Is it very, very dynamic. So I'm not worried.
And, yeah. I can tell you more about OpenAxiom, how it works. And study typing is a
shift through a -- two-level type system where you have domains that are instantiated at
runtime, and then you have categories that will be cumulative-type classes that you use to
predict what might happen at runtime. And since you do your reasoning not on a type but
on the abstract mathematical structure, you can still do type checking. And you don't
break abstraction because things like concrete representation is not accessible at runtime.
What is accessible at runtime is algebraic properties and [inaudible] properties. Thanks.
So, yeah, I was here. And so now the things that I would like to work on, well, of course
9
I like templates, I would like to have more formal accounts with templates, very
complicated, I know, but I think we can get there.
And one thing I want to announce is that I have a new student for work who has been
working on code extraction from tough development to C++. And he's in C++ 11, and it
makes things much simpler. He's quite well advanced and I hope that we soon will have
something that is part of the Coq main distribution as opposed to just being he and me
having something we put on the Web but part of the -- that way I could develop part of
libraries in Coq, get them [inaudible] to C++, integrate that into my programs and say,
well, I can say this part of my C++ program has been proven to be correct.
Okay. So next thing. So how I go from trustworthy system to Liz and computer algebra.
So Liz is this experimental language I'm working on. And interestingly enough it brings
me back to computer algebra. Not that I want to solve polynomials, but trying to write a
complier for this language brings me back to old problems that I was trying to avoid.
Okay. So what is Liz. So it's system for structured -- what I call structured generic
programming. The core semantics is based on that core of C++ I think it is okay. There
are a bunch of things that you can find complex about C++. There is a core that's actually
well designed that I think is good for -- as a basis for a new polynomial language.
It is inspired by the book by Alex Stepanov that was published in 2009, "Elements of
Programming," where he tried to explain that generic programming is really, really math.
And of course I know I was hugely influenced by my experience on using the Axiom
system. The first time I hear about the Axiom was when I did my internship at INRIA,
was '96. Then I became a developer maintainer of the compiler. And so there was a lot
of implementation techniques that actually should find there way through mainstream.
So it's a good polynomial language I believe for starting efficient and scalable generic
programming and lessons in terms of design and implementation technique. I would like
to use that to second effort at designing C++ concept. We probably were in the C+ OX
we tried to have concept for C++ which was kind of type system for templates. They get
in the standard draft and then they had to be taken out again because of various problems,
some of them, really, to growing complexity. And this time around Ben and I got funds
from planning from NSF to do this thing again properly. And I believe you know how to
do it, but I don't want to have all the complexity of C++ while I'm looking at this
problem. So I'm taking some part of it and looking at them in this system.
And the other thing is the constraints that we put for C++ concepts, we believe that for
this to succeed, we have to break with conventional type checking. What I call by
conventional type checking is you have a nice interface and you're going to represent this
by some kind of V tables and you're going to [inaudible] V tables at runtime. And so
we're going to fly for C++.
The reason I'm going to fly for C++ is that if people have to move to improve system and
lose inefficiency, they're not going to do it because the disciplined programmer knows
how to use templates effectively. What you're trying to do is get the masses do the same
thing, get the benefits. So we have to break with conventional type checking. And it is
being developed as open source on a BSD-type license when you just look at the codes
and do whatever you want to do with it. That's fine with me.
10
So it started with Carla. She was at the time one of the grads. I had her write the first
parser for Liz. She's now at Microsoft as programmer manager. Then after that Erik
Katzen. I had him rewrite part of designer library using that facilities that I added. And
based on his work, I decided to rewrite the parser myself and change completely the
syntax. So since last summer I've been busy rewriting parser.
Jasson is a Ph.D. student with me. He's actually from ECE being coadvised. And he's
interested in network and secure network protocol design. And he wanted to use the
ideas from Liz to look at goods designed for the network. And Jordan Donais is
undergrad doing research, and he has been very good at finding bugs in latest
implementation.
Now, the first thing is that I'd like to go back to Hindley-Milner, and I suspect this
probably is related to some of your questions about Haskell.
So we have ideal in programming language which is the Hindley-Milner Type system.
It's a masterpiece. It is great. And it has only four rules, sometimes, right, five.
Originally it was written five, but it can have four rules.
And it was based on very simple ideas. Look at your programs and what you're doing,
essentially, you have functions. You apply them. And you have variables. That's
essentially all. And the idea is that you don't need to write types for your program, just
write your program. And based on this structure, we can tell you whether it is waveform.
We have rules. It is waveform. And the compiler is always the winner. It's going to
infer a type and if you can supply the type.
If you supply the type, it's going to be a smaller version of what a compiler can deduce,
small version in terms of instantiation, going to instantiation of the most general type. So
what the first is always the best you could possibly write. So you write your program,
your types, and if you believe that type give meanings to program, then the compiler is
going to find meaning for your program. And because can have several type, that means
can have several meaning. It's a very simple idea.
But what happens, well, it's [inaudible]. What happens is that it kind of puts the focus on
some kind of a bottom-up view of how we look at types. Usually we take most
traditional papers on type systems. They're going to give you some initial algebra and
here's the way you construct things, then it's what did you do with it.
One problem with that is if you try to develop a generic library very rapidly, very
quickly, you run into having several type parameters. So just as an instance, when
people, you know, develop type classes for Haskell and they wanted to write container
libraries, they realize, well, they need more than just one parameter, because containers
happen to carry a lot of things we don't talk about for most of the time.
And so McJones introduced the notion of multitype parameter -- sorry.
Multiparameter-type classes. And originally sounds like a great idea until realize there's
some problems with ambiguity. So he introduced functional dependency. They said
well, we can have two types, but one is going to determine the other, and then you can
solve ambiguities. But only partially. And things started, you know, following and so
11
forth.
Eventually in the 2000s people decided, well, why don't we just go back to those C++
guys and borrow some of the ideas, ideas being top traits. That's how you get [inaudible]
types.
So when the paper on [inaudible] types were published originally explicit information of
the inference of C++, and I thought that never again because now people have to refer
back to the more formal version.
So the point here is that when you base your system on a Hindley-Milner Type system,
which is very nice, and you build generic libraries, you get a lot of extra parameters.
That you have to mention again and again and again. It is a distraction. It makes the user
interface, the software complexity very thick.
This is Microsystem. This isn't off the system. [inaudible] good, it's just that when we
want to build generic libraries, it is complicated. So it is not just familiar. The same is
true for C++. I'm going to show you an example.
So I'm going to present something, propose something different. So it's my experimental
system. There are ideas there that probably don't scale. But I do believe there is a good
idea there that we should look at and actually go inference the rest of the languages.
Okay. So the first thing is when you write your program, your library, think of it as a
mathematical object. Even if you're not [inaudible] think of it as a kind of geometrical
object.
Here you have a surface and you have some -- this sphere and you have some curves,
algebraic curves of degree over 40. And how would you describe this thing? Would you
like to use explicit parameters to describe it? Or would you prefer simplicity questions.
Take this sphere, for example. A simplicity question you just need one. You just say,
well, X square plus Y square plus Z square call [inaudible] square. Some number. That's
all. If you prefer the parametric version, you can just use one system of parametric
equation to describe the entire sphere. That's just a mathematical theorem. You need at
least two.
Now, if you need two, you're going to have relevancy. Now you need covariance,
conditions between. It's complicated.
What I'm saying is there is some analogy here between this situation and how we develop
programs. Look at your generic algorithms. Are you happy about them? Do you have
your right number of parameters? Do you have two more? A lot? Few? Just enough?
How do you specify constraints. Okay?
Now, if you take the [inaudible] parameterization [inaudible] this is a parameterization of
this sphere, I'm just using the [inaudible] projection. And compare that with the equation.
This one is very nice work if you're just looking locally at a portion of this sphere. But in
general it is a little bit complicated.
Okay?
12
On the other hand, the implicit equation is very nice, just one thing, all the other points at
equal distance from a center. Good. Okay? But sometimes it's very hard to get things
like what's the tangents to the sphere at a certain point. That's nice to do there to here.
Okay?
So what actually happened is that you can't just say one is better on the order
systematically. What you have is different ways of looking at the same problem. And
mathematicians have been very successful at this because they've developed tools to
convert from one to the other. And you need combination to be successful.
Now, this one from here is that I would like to propose a way of looking at type systems
differently from the initial algebraic view of type systems. So that's what I'm going to
talk about next.
What I want is some kind of co-algebraic view. So each algebra essentially have some
structure. And you look at it. The co-algebraic, you poke, you ask questions, can I do
this, can I do that. If you look at the ST, for example, one of the reasons why it is very
successful is that it is purely co- algebra. It doesn't have any memory allocation. Okay?
It is very functional in nature. Even if it is written in C+, it is very functional in nature.
But it doesn't have any memory allocation. You copy to another place. The iterator
knows how to allocate. It is not the algorithm's job. Okay? Do you know transform?
You [inaudible] many things are done without knowing how they're done, just you have
to specify what's going on. Okay.
So here is what I don't want people to do with explicit parameter. So I don't make this -I don't make this up. You can take GCC and you look at the internal implementation of
the hash table. This is what you get. You get at least 11 template parameters. How do
you know you have them in the right order? How do you know you have all of them
correctly? I'm not -- this is not artificial. It's taken directly from the GCC source code. I
don't touch that part of GCC because I think it is just insane. So I don't want this. So if I
don't want this, what should people do?
>>: This is pretty recent code, right?
>> Gabriel Dos Reis: I'm sorry?
>>: This is pretty recent code?
>> Gabriel Dos Reis: Oh, yeah. So yeah. If you -- I'm sorry?
>>: Because [inaudible].
>> Gabriel Dos Reis: Possibly. Or just look at -- you know, I have a copy of GCC on
my hard drive, just look at it and -- I'm pretty sure I can come up with something similar
to this in high scale. Okay. But I thought this was much easier.
The fact that this is recent is not the issue. The fact is that this is how people write it.
Yes, you're right. Hash tables were not in C++ 3 and they are in C++ 11. So this is
recent version. But the point is this is what people do because they want to control every
13
part, so they have to pass at least 11 parameters. They have to mention them -- so here -so everything is real expect this dot dot because they are his parameters and they won't fit
on this slide, so just written like that. But, you know, if you have the source code, you
can look at it. It's ugly. But apparently it works.
So I don't want this. What I want is the first thing is we spent so much time especially in
object-oriented programming world asking what is this thing as opposed to how can I use
this and when I use it, what does it do. That is actually what is most important. The STL
is successful because it does not ask me to be something. It just say this is your
conditions. When you use this thing, this is how it should -- all it should produce.
So you focus on a behavior as opposed to some kind of glorious state of being something.
Okay. So, for example, if you take this soft function from the STL, it takes two iterators
that designates some sequence. And you want to sort. You can provide your own
[inaudible] function. Here is the comp, so it is just some kind of -- it is value of some
type.
And I can use my own comparator that way or I can provide point of degree of function.
It will work great. The problem is if I messed up and press a wrong argument, well, I get
very terrifying error message and I don't know where to start. So the question is how can
I specify constraints on these template parameters that reflect what they're supposed to
do, not what they are, but what they're supposed to do.
So to look at that problem, well, we try that in past decades and didn't work exactly as we
wanted. I'm very persistent. You had a question? Sorry. Oh. Very persistent, so I'm
trying again. And so I want to isolate one aspect of it and study it in some experimental
language, gain some understanding and see what scales and get that back to C++. Okay.
So the idea is to impact a real-world language just for some language but looking at
fundamental problem.
So the call to write field is that, well, I'm going to look at type checking differently.
When look at type checking algorithms, there is certain questions we ask implicitly. I
want primitive for that [inaudible] language so that the programmer can state what's
going on. So the first thing is understanding something that can call from something that
shouldn't be called at all. So call that function.
Then function, if I have [inaudible] function, well, I have to know whether I'm calling to
the right number of arguments. So I need a primitive that gives me arity of a function
type. Then I want to be able to look at its argument positions. So this is also -- an input
type is an operation, binary operation that gives me the type of a parameter at a given
position. And then the return type.
So if you -- again, if you look at, take a geometric view, essentially a function is an object
of N dimension or N plus 1 dimension where N is the arity and then you use input type
and codomain ask the coordinate access to talk about what's going on. Right? Okay. So
that's -- it's that geometric view that informs what I'm doing.
So, for example, if I take the functional type, now a point vector and I construct a vector
field, for example, one of the arity is 2 and the input type at 0 is point and input type at 1
14
is vector and the codomain is vector field. So that's essentially what these primitives are
about.
Now, my claim is that these four primitives, and five because I also need a way to know
when two types are equivalent, are sufficient to conduct type checking of an algorithm at
most abstract level. So that's my claim. Okay. And that's what's the basis of the latest
project.
So here I have the mathematical, so, you know, I want first to prove that you can actually
have a mathematical statement of something and have that encode that surrounds. So
here I take notion of a homogenous function. So a function type is said to be
homogeneous if it takes at least one input type and all its input types are the same and we
don't say anything about the return type.
So the arity must be zero -- must be positive, sorry, and for all inputs type you have the
same -- here I have the equality. Earlier on the previous slide I say how can I say two
types are equal. So have relation of equality of type.
So here I'm defining what I all a concept. A concept essentially predicates over
something. And that predicate is supposed to define an entity in the most fundamental
way. Anything else I should derive that from the definition as opposed to having it in my
head or in some documentation. Okay?
So here, for example, if I take the function distance between two points, well, it is
homogenous function, but if the function translate that translated point by a vector and
produce another point, it is not homogeneous function because the two input types are not
the same. But distance is a homogeneous function. Even the return type is different.
So this is what this notion of concept says. Okay? So essentially I have sequence of
requirement predicates. You can think of a concept as a predicate over its arguments.
And, again, my claim is that concepts would consider basis of type checking.
So if all the input type are the same, actually have the notion of domain of the object
here, I have the row that gives me -- I can define definition of domain, just like a role, I
can think of either some kind of -- something I added to C++ 11 which is template ASs, a
way of having function types, type functions; namely, the function type directly.
Or way of doing computations with values and types at compile time. So those two
things are in C++ 11. I implemented myself this one directly, GCC, so it is a [inaudible]
recent version of GCC, and this one was implemented by someone else.
Now, an operation. What is an operation? An operation is a homogeneous function that
have the same where you're doing type is the same as input type. So here I can state
succinctly what that thing is. It is code. So the compiler understand it. Compiler come
easy to do type checking. That's the good part. And, again, I'm using type equality, an
example of an operation. So if my multiplication is nonoperational, it's an operation, but
computing normal for vector is not an operation.
And binary operation, now I have to fix the arity here. I don't fix the arity. Now the
input types are free to go in direction, but they're all the same. So I have some kind of
15
[inaudible] view. But here I fix a number of parameter. And, again, these are sufficient
to do type checking.
Now, here is a function called square. So if you know -- if you have an operation, you
can actually use it to do [inaudible] squaring and so on. So here just have a function that
takes an operation. And a value of an int domain and return that function -- sorry, that
binary function apply to the argument twice.
So if you apply to call these functions square with three and multiplication, you get nine.
And if you call it with three in addition you get six. It is basis of [inaudible] squaring
algorithm that is so basic.
And interestingly the type of this function is, well, I take some type alpha and I require
that it satisfies a predicate binary operator and any input types are not referenced to const
D, the domain, and alpha.
The thing here is that the real parameter is what I want to emphasize, which is that I'm
applying an operation to change an int domain. Not the parameter -- not the domain type,
but the operation. This is where I want to put the emphasis.
If I want to use the classical view, then I have to introduce this, ref const D as the
parameter, and then I have to somehow specify directly the representation of the domain.
I don't want that. I want the representation of the function to be as abstract as possible.
Okay. So here example. I can apply some multiplication or addition.
Now, more on this. So it is customary to every time want something to define a type and
say, oh, I have something of that type. But type, I just carry a set, right? They help us
represent values. And carrier set could be carry set for many algebraic structures. For
example, if you take monoids, for example, your notion of monoids, which essentially
have a binary operation and then you have one element when you combine [inaudible]
just return that element and then have [inaudible] things going on.
Well, integers, infinite integers [inaudible] forms a monoid. But if you take infinite
integers and GCD, for example, it's a monoid structure. So you can't just say integer is
the monoid. We have to be very precise. You have to say for which operation.
And then you see that, well, the property is very interesting, but what you want, you want
to put them on operations as opposed to the carrier sets.
So here how do I set an operation is [inaudible]. I can set that specifically. So this is of
taking right encode, get the compiler to check the definition, and at some point I can
inform the compiler that, well, operation -- actually I'm going to assume GCD is
associative.
I could -- in a more advanced system, I could actually provide a definition of GCD and
try to prove that GCD, but I want something that can scale, something that I can scale, I
can get back to original C+. So I can have everybody do proof. Most people prefer to
write algorithms, not proofs.
16
So the type checker, I can also have the type checker use Axioms to design semantic -runtime semantic properties to aid with other resolution. So if I want to have a notion of
semigroup modeling, as I said earlier, so I need a notion of semigroup. So a semigroup
operation is parameterized over an operation. And I need a type for that operation, so I
put a [inaudible]. Can think of these as the templates. So essentially it's the guessable
type. What area it wants to apply is the operation I don't care about the type, but I need a
type so that I can get some other things moved.
And then I can define a notion of neutral value, so it's an Axiom. So and if I -- there's
some value E and if I put it in different places, I get the other stuff back. And now I can
state what it means for an operation to be a monoid operation.
Well, essentially you need to have some distinguished value. The neutral value had to
satisfy that property. And, by the way, I would like to be able to refer to that later on, so
I introduce a rule that says, well, I want to name that neutral value E later.
So implicitly this is actually a function of the operation because I have some extensions
here, I get [inaudible] and it turns into a function.
This is a principled way of doing type functions or associated values. Currently in C+ we
have traits or in Haskell you have type families. And you state axioms. They are not
even controlled. You can control, but most of them, they are not controlled. And they
are the root causes of many of the headaches that Haskell people have, and many of the
headache that similar people have trying to build a skill generate programming.
We want type functions, but we want them in a structure. We want to know when you
introduce a type trait at a given point what's the controlling condition. It is there. So it is
not something that is defined by any programer willy-nilly. It has to satisfy some
constraint. Any complier is going to check that.
So what do I get out of this? Well, first thing is that I got shorter and clearer codes. It
have higher level specification and it aids verification. And I really want to get rid of
these things. They look nice, but they are not good for software construction, at least
scalable software construction.
And so one thing that's quite interesting was I had a student [inaudible], and I said, well,
let's look at this in a [inaudible] system. Because, you know, we have [inaudible]
structures, we should be able to do something with it. So the first thing is can we do
some form of low hanging fruit, some kind of implicit parallelization.
Many operations happen to be [inaudible] in [inaudible] setting. So can we get this thing
actually. So at the PASCO conference, we were able to quantify opportunities for
implicit parallelization if we can inform the complier [inaudible] properties like
[inaudible] and so forth. So GCD, which we use very frequently in [inaudible] system
solving is [inaudible]. And then in a subsequent work, I got him to actually get the
compiler [inaudible] and generate code implicitly. And it was pretty good. Yes.
>>: Do you have a result for the computational complexity of type checking?
>> Gabriel Dos Reis: Yes. I hope soon. Yeah. Yeah, I know I'm running a little bit
17
time -- yes. I'll get to that. And this is where it is still ongoing work. And then I would
like to see what scale and put it back. But I do believe that reducing the number of
[inaudible] is good way to go.
>>: Your system is basically first order. First order logic?
>> Gabriel Dos Reis: Yeah.
>> Right.
>> Gabriel Dos Reis: But you can combine things. I will show you the formal version.
Yeah, I had the great part of it ->>: So is it even decidable?
>> Gabriel Dos Reis: Oh, yeah. Good question. Is it even decidable? So the system is
implemented so that I can experience [inaudible] but currently I do have restrictions like,
well, certain quantifiers cannot be nested and so that I can decide things.
And the other thing is it will be very interesting from software engineering point of view
what kind of restrictions. If you put certain restrictions on certain formulas, you got
formula, what kind of programming technique you can develop, what is supported, what
becomes too complicated.
Interestingly enough, [inaudible] is very first order and, as many things just, you know,
work beautifully. And of course I would like to test this in other fields and get an idea for
it, what actually [inaudible]. Yeah. But yeah. In full generality, [inaudible] that's fine.
Even in Haskell. Because have type classes and, again, two type-level computations and
it will terminate and so forth.
Okay. So good thing is that part of the algorithm that I developed for Liz were used by
Andrew Sutton who is a postdoc with me and [inaudible] on this new version of C++
concept. So we got the -- two weeks ago we got the ISO C++ standards committee
approve a technical -- they request for a technical specification, so we are going to say we
want this, but a committee want to have a -- some formal presentation first, and if we all
agree on it, then we just drop it into ->>: [inaudible] some sort of Axiom specification based on first order logic and C++?
>> Gabriel Dos Reis: Yes, that's the goal. That's the goal. And actually have another
student [inaudible] working on that with GCC.
>>: Good luck with that [inaudible].
>> Gabriel Dos Reis: Just have to keep asking. Get something. So last year I showed a
design and near implementation at CICM. And so the thing is that concept actually
elaborated into a set of proof obligations. When you define a generic function, you use
them as hypothesis so that you can check the body. And when you call a generic
function, you have to provide evidence that the [inaudible] was satisfied.
18
So that's essentially what the idea is about. So you can have a predicate like this, so
essentially a propositional formula, okay, you can ask that a set of operations exist. This
is a result through name lookup. And you can postulate axioms inside the concept
definition. This essentially requires the compiler to go up and see whether there is a
statement by user that certain property holds.
And then you have how to get rid of type tricks. Okay. Now, so essentially have the
quantifier, so asking you a question [inaudible], oh, so this is -- so the full
implementation is of course on this side of it because I want to experiment in several
directions, but here I'm showing essentially the part that I'm comfortable about.
So we have a constraint essentially. I have a quantified formula with the types, and
sometimes I want to be able to have some type deduced like in templates, you know, the
template parameters are usually deduced, so the question I have to say is I want the
deducible parts and expression, so this is just a simple expression. I can do a function
call, function definition, and then instantiated type.
I have a formulas and variables of course. Any program is just a definition of functions
or definition of concept. And when I'm doing type checking, I have to maintain all
declarations and I have to maintain the axioms, properties about entities. And of course
one thing that doesn't show here is that this is all since I have the codes between types, I
also need a congruence closure to keep track of when things become equal. Like when I
say codomain of F is the same as domain and I have to maintain that in all type checking.
So [inaudible] transition a little bit tricky, but it works. And it's actually earlier things I
did, and I was very happy about it and pretty cool.
So the implementation. So this specification here, I didn't say much about the monoid
type, but essentially because they are expressions and then becomes type by putting
restrictions on what actually it's like when you have dependent types, you have
to [inaudible] you actually evaluate to see what's going on and you put restrictions on the
structure. Yeah, Ben, you had ->>: Can you have infinite types, or is the issue compiler [inaudible]?
>> Gabriel Dos Reis: Yeah, so compiler does some evaluations like compile time.
>>: Okay.
>> Gabriel Dos Reis: Yes, but on the rules, for example, the rules essentially functions
that evaluate compile time. And we already have that in C++ 11 already. So I put that in
C++ 11 last time around. Yeah, it can have certain functions that are simple enough, and
actually it makes now completions go forward, but we already do that with templates.
This is a just better way, a simple way of doing the same thing.
I think in the '90s, there was -- a decidable type system was fashionable. But decidable
type system means a lot of things. Once something -- sorry. Once something is
undecidable, it can be undecidable for many reasons, right, because we don't know the
algorithm or sometimes certain computations are just gone forever. And if it puts a limit
on upper bound on how long you can spend time doing a computation, you get some
19
feedback. And you can increase. And, you know, even Haskell people now are very
okay with that idea. So I think it is okay now if undecidable means some computations
can go forever.
So doing type checking after [inaudible] closures for type equality. Just to give you an
idea about how I elaborate this. So if you check this source level definition, so this is
very pretty simple. So homogeneous is a -- here it is a concept constructor. So it has
type. It takes a type and there is a predicate on a type.
So when a predicate is unary, I can use it as a type. When I say F colon function actually
means F is a type such that the predicate function of F holds. Okay? So just short
notation. And define a concept. And in the body have that everything has to be -- has to
be positive and then elaborate this.
So I put some constraints on -- so question that I get very often, both implement in
position Y when I present this is what is the theory that we have for integers? Is it
everything? Can I test, you know, primality or that kind of stuff?
Okay, the problem is you'll say I can order integers and [inaudible] and have this built-in
notion of the integers are bounded. I can go more. I can do more, but I wanted
something, something pretty simple. And once you start writing, only the basic concept a
little bit complicated to get, but also get [inaudible] find it's pretty, pretty simple, right?
Here I'm just saying that codomain has to be the same as domain is the rule, so we get to
evaluate compile time, it get into this form and so on. So, again, equality constraint I
maintain them throughout type checking.
And one slide, something very interesting, is that when I say well, I have a notion of
monoid operation. So the idea is that you have this function, so certain types a semigroup
operation, and there also exists an element in its domain such that I have this relation
between F and that element.
And, by the way, I would like to refer to that element later on as neutral element. So
when I elaborate this, I get two things. I get the first definition of monoids, which is just
essentially that. I have -- I have a semigroup -- okay, it's alpha, so it's -- I have to be able
to defer alpha up here is inferrible from the argument F. Here is a bug. Sorry. Alpha
should be there. And then the actual function goes from F to -- from F to three concept.
And the requirement is just this. That I ought to be able to find this. And this is solvable
by just looking at database, query database and find.
Now, the other interesting is natural value. So this value is a function [inaudible] this is
the type families. But in a principled way. It is implicitly defined by the compiler and
the compiler knows exactly when defined because there is conditions on where it is.
And one other problem is we have with type families in Haskell is that most of the time
they're unprincipled. So when you design in an algorithm do type checking, you have to
imagine that you'll get institutions where the some Axiom just popped out of thin air.
No, in real programs, when people write real program, they don't pop out of air. They
have conditions. So if you take that ability out of the program, put it in a complier, you
20
can actually do type checking. Okay.
>>: [inaudible] predicate relating to the function and the neutral element, or do you
actually say somewhere, you know, that if you take an element from the group and you
apply F to that element in the neutral element you get back the ->> Gabriel Dos Reis: So for -- okay, substitute. [inaudible] but neutral element, no, it's
defined here, but I could have -- assume a neutral value GCD.
>>: But these F things are C++ functions, right?
>> Gabriel Dos Reis: Which one?
>>: The operation ->> Gabriel Dos Reis: Yeah, the function. Yes. So these are the points where I trust the
user.
>>: Well, but, you know, I mean, this thing could have like effects and ->> Gabriel Dos Reis: Oh, you're talking about -- the neutral value, no, the [inaudible]
function. They are side-effect free. The function that I'm talking about can have side
effect, that is fine. But these actions are pure functions.
>>: And so when you say F of X of E is equal to XX, why is it ->> Gabriel Dos Reis: So when I say this, I'm not running the program. I'm stating a
property that the compiler is free every time it is type checking and it sees this
expression. It is free to simplify to this. It is like these optimizations that compiled.
>>: And it's yours and it's total and it's F of X of I G ->> Gabriel Dos Reis: Yeah. Yeah. Yeah. So the point is this. It is already happening.
What I'm doing is to try to reduce the clutter. If I have the luxury of requiring everybody
to do proof, then it is okay. Okay so the question is this is designed how much you
require people to do and how much you tolerate as noise.
Okay.
If I can get away with -- if I can remove type traits and just have this, I'm very happy with
that. Because this is evil. The [inaudible] are evil. They're useful currently because we
don't have a better way of doing it, but if I can take that and represent this, that's fine.
We do this all the time. Even from the computer algebra point of view. [inaudible] so
usually people say, oh, 14-point numbers, they come from a field or whatever it is and
multiplying -- sorry, addition on 14 point is [inaudible]. Which is true. You can provide
examples and show [inaudible]. But if you know your input data, this is set. On that, that
assumption becomes true.
As a matter of fact, this is a real thing. One of the fastest modular algorithms working on
21
integers use 14-point double to present integers because it is relevant [inaudible] and you
get 53 bits.
And when you're running your program on your machine, the CPU is busy doing
arithmetic operations and Euclidean instruction. Why the 14-point units just sit there
looking at what's going on.
So if you move a little, show it to the [inaudible], well, it works. Because you know your
input sets. Of course that assumption is not true in the absolutely mathematical sense, but
you know your input sense, and that's what matters.
And this is okay library [inaudible]. It's one of the fastest library on modular semantics.
Pretty neat idea.
Okay. So where am I? Oh, okay. So you get, you know, this. And currently you have
some feedback. Andrew Sutton, who is a postdoc with me, implementing part of this,
part of the ideas, some of these scale of ideas into GCC. And we want this to get into the
concept of technical support. And Michael has been working with me for a year and a
half. He's working on axiomatic paths in GCC. So things are moving pretty good. I
hope that we'll get some [inaudible] paper soon. That's going to be hard [inaudible] PGI
is pretty good.
So for future work. I would like to have less segmentation faults in a published
[inaudible] and get [inaudible] pretty neat idea about how to use some of the constraints
for [inaudible] protocol. And so people like contract, programming by contact and that
kind of stuff, so I would like to understand much better [inaudible] integrating axioms,
things that you just assume for optimization purpose are things that you actually check at
runtime and what part of it.
You know, some of them you cannot check at runtime [inaudible] defined algorithm, for
example, assumes that the two iteratives you give, one is originally from the first and that
it essentially is running the algorithm. So you can check that system at runtime.
And of course I want for more account for this. And, yeah, can we trust the user all the
time? This is design point [inaudible] to look at. Okay. I'm ready to take questions.
>> Tom Ball: Thanks, Gabriel.
[applause].
>> Tom Ball: Any more questions? We asked quite a few. Okay. Great, well Gabby
will be here for the next two days. And thank you again very much for taking time to
come.
[applause.]