1
>> Sebastian Burckhardt: Hello, welcome to this session. It's my pleasure to
introduce Eelco Visser from Delft University of Technology. Eelco is an
associate professor, and I met him at a PC meeting for Splash. And at the
time, I was just starting to get interested into mobile programming and
research problems and it was a shock to discover his paper, which essentially
looked to me like he did everything already.
So in the meantime, I've relaxed a little so there's plenty of research left to
do. But he's here today to tell us about his special research on mobile
development, but also on general background in program languages and on how to
build domain specific languages.
>> Eelco Visser: Thanks. Thanks for having me. Right, so the title of this
talk is software language design and engineering, which is sort of the mission
statement of my research group in Delft. And so I wasn't quite sure what
audience I would get here. And I'll be here today and tomorrow morning. So I
wanted to give a sort of introduction to the sort of work we do in my group, as
we give you an impression of what we're doing so that we can -- well, as a
starting point for talking and seeing if there's leads for collaboration. And
one of them is this mobile, work on mobile.
So the general problem I'm addressing is this. We are only software
engineering, and the problem there is how to get from problem domains to code
[indiscernible] on the machine, and we have to bridge this gap and so as a
community, we've been developing high level programming languages that reduce
that group so we don't have to [indiscernible] assembly code anymore. And this
works very well, pretty well, but there are still some gap left there.
And the idea is that with between domain specific languages, we can further
reduce that gap, a lot of them saying well, we design languages that are
designed for arbitrary applications. We design languages that are specialized
for a particular problem domain so that we can better, well, reduce accidental
complexity. So sort of the background of my work.
And then our mission is to, or our research is to enable software engineers to
effectively design, implement and apply domain specific languages in software
engineering. And that falls into two categories.
So traditionally, I've been working in compiler construction tools.
That's
2
sort of my background. So what we try to do is if you want to make domain
specific languages or linguistic abstraction as sort of an engineering methods,
then you should have good tools to make DSLs. And so making DSLs should be as
easy as making [indiscernible] general programming. That's the goal.
So anyway, to do that, what we need is declarative language definitions that
allow us to automatically derive efficient, scaleable, incrementable compilers
and also, these days, usable IDEs so that we can get good productivity. So
that is the aim and we are getting pretty good at it now. But there's still
lots of work to do.
And so one of the domain results of this is the Spoofax language workbench
which is an eclipse log in for making languages. So you can define languages
in the sort of syntax and semantics of your language using DSLs for language
for language definition. And from that, you compile and built to get a new
[indiscernible] plug-in that is special for your language and it has all the
usual IDE features and so on. I'll talk about it in the second part of this
talk.
And so I've been doing that work on making tools for making languages for a
pretty long time, and we're actually getting somewhere with that, and we now
have tools where you can actually make languages. So now if you can do that,
then the question becomes, well, how do you actually do this, making a new
language.
So at the beginning, we used to try our tools on well-known programming
languages. Said, well, can we make a compiler for Haskell or source to source
transformations for Haskell programs or that sort of stuff. But it's much more
interesting to make new DSLs, and then the question becomes given some domain,
how do I systematically and effectively design a language for that domain that
will allow me to have better productivity in programming that domain.
And there's much less technical stuff like here's a syntax -- here's a grammar
and [indiscernible]. It's more about language design and how to do that.
So we started studying systematic design of domain specific languages where you
have to find a good trade-off in these different properties of languages.
Expressivity, so how easy it is to express something in language. The
completeness, how well do you -- I mean, do you generate complete codes, or do
you generate a bit of code and then you do the program some other stuff in your
3
target language.
Portability, can you actually address, generate code for different platforms.
Coverage, how well do you cover the domain. Can you express everything that
you want. And maintainability. If your target platforms move, can you quickly
move your compiler to also address those changes without changing the models in
your DSLs.
And since I moved to Delft in 2006, I've been starting to do work in this
domain, which is case studies in DSL design. In particular, so my students do
lots of little languages, but in particular, we'd be looking in the domain of
web programming, and we've designed and built two languages. One is called
WebDSL, which is a web programming on the server, so a web application where
you generate pages that people interact with via browser.
And but most of the action happens on the server. So this is one of the
applications. So once you build a language, you also have to build
applications with it so this application is [indiscernible] with your digital
library application. It has over a million and a half publication records, and
you can actually use. So that has become another interest of mine that I'm
interested to talk about with people.
And more recently, we've been looking at the mobile web, and so nowadays when
you have a web application, you also need to have an application that runs on
your mobile phone, and for that we have developed Mobl, which is a client side
stateful web application. And that's what I'm going to talk about now.
So this talk is really two talks. In the first part of the talk I'm going to
talk about languages on case study Mobl. So declaratively programming the
mobile web with Mobl. And second half of the talk I'm going to talk about the
Spoofax, language workbench. How do you make languages technically with ->>:
Can you tell me where Spoofax comes from?
>> Eelco Visser: Yeah, so it has been invented by engineers, by Karl
[indiscernible], who was a Norwegian Ph.D. student who worked with me for a
year, and he actually came up with the name, and he engineered it so it was
Google unique at the time. So there was no hits in Google. And that wasn't
the main criteria. I didn't think about it so much at the time. Spoof, maybe
that's -- I'm not sure the connotations are actually right. But it has stuck.
4
So that's -- yeah, so there's no meaning, basically.
So this is sort of an [indiscernible] for giving talks and it may be a bit
longish. Also, not going very deep in both topics so I'd welcome questions you
may have about any of these things. I know that these guys are very interested
in the Mobl side of things and services. You may want to have that discussion
now or later. We'll see. I'm flexible.
All right. So first talk. Declaratively programming the mobile web with Mobl.
So this is joint work with Zef Hemel, who actually -- this is work by Zef
Hemel. He's my graduate student who has just submitted his Ph.D. thesis and
design and implementation is mostly his work.
So it may very well be that there are questions that I'm not able to answer
about, especially about the programming model, the deeper details there.
All right. So what is the domain? So we had been developing this web
programming language, and I will become interested in also making applications
for mobile phones and tablets. And Zef had actually been doing some work on
PIL, which stands for platform independent language, and the goal was to have
portability for DSLs. So if you make a high-level program, you would like to
be able to port it to multiple different platforms, and we, at the time we were
interested, we were doing web DSL. We had a Java [indiscernible], the Google
app engine came up and it had [indiscernible] program so we thought, well, we
have two back ends for web DSL. It would be pressing to have a single
intermediate language from which we could generate all those things.
We developed PIL as an example, and that works fine. So I mean, these target
languages are very similar. So actually, you can abstract the tiny syntactic
details. So we had approve concept, implementation that works.
It turned out there were some pretty deep problems afterwards. So one of the
problems was called hibernate, on the Java platform we use hibernate for object
ratio mapping, and that sort of determines the semantics for our data models.
There was no hibernate, there were other frame works for doing operational
mapping, but they had quite a defendant semantic. So you had to do explicit
saves to save objects, which you didn't have to do in the hibernate platform.
So it turned out that this -- so although the languages that we targeted were
pretty similar semantically, the language itself wasn't the real problem. It's
5
the platform. So all the APIs that you program against. And they we thought
again about this web domain which doesn't really matter that you can
[indiscernible] your web program. On a server, as a developer, you control the
server and it doesn't really matter, you can determine what platform you're
running on.
So we thought, well, so there's two developments we would like to have an
example of portability and, well, there's smart phones were coming up and this
was a pretty interesting domain itself.
And so this looked like a pretty good area for studying portability, right. So
there's all these different mobile platforms and if you are developing for
these platforms, you make an iPhone application that you provide in Objective-C
and then for Android, you have to program Java, and there are other platforms
as well. So that means you have to do a lot of double work.
So that was sort of the background of this work. We're going to make a
language for mobile applications and targeting different programming languages.
Then I came across a book on mobile web, and it said, well, actually, you
should program mobile applications with client side web applications. It was
actually many of these platforms have very powerful browsers that run HTML 5,
the universal user interface engine that actually runs on all these platforms.
And it's much more than HTML.
That's why we have -- so the architecture, you have the standard HTML and Java
script, and CSS and you connect with the web service to transport data. But
now there's also to access things in HTML 5, you have a local database in the
[indiscernible] device that you can store data in. You have access to GPS, and
while there's all sorts of other APIs that you can access. So there's local
information, you have a canvas you can paint on. There's multitouch support,
offline support. You can run these things in full screen so you don't actually
see the browser.
And if you then capture your [indiscernible] application Phone Gap, then you
can even have access to the native phone specific APIs if you're off the
platform.
>>:
What is it called again?
6
>> Eelco Visser: It's a tool that's -- and that's a web application, so a
client-side web application as a native application. So you can, for instance,
for iPhone, you can sell it in the Apple app store, and it looks like a native
application.
>>:
[inaudible].
>> Eelco Visser: And since you're running as a native application, you then
have access to these APIs. Which if you're just running in the browser, you
don't have access to all of these things. And it's not just -- I mean, it
works on many platforms. So you actually get portability.
So the platform here, we have HTML 5, it actually gives you a very rich
platform for building client-side phone applications. But, well, what does
that mean? You have to program a HTML 5. So it gives you sort of a platform
for doing this, but how you do it is pretty gruesome, right.
So you have to write the HTML in Javascript. So Javascript is the language
there. And so there are some frameworks for doing this, but, well, they use
this model view controller paradigm. Very little integration. No F-sections
and a lot of accidental complexity.
And then so this looks very much like the web programming on the server that we
looked at before. So for studying WebDSL, we looked at a bunch of different
web frameworks. We looked at Java, Seam, Ruby on Rails, and [indiscernible]
Lift so quite different styles of programs. And we looked at how they behave
if you make programming errors.
So even in the [indiscernible] time fonts like Lift and Java, it turns out that
many programming problems are reported as very late. So they appear at run
time and often in a very unclear way. And so the problem is that you have, in
addition to the general purpose programming language that you're programming
in, there's these other languages that you're programming in.
There's HTML and Javascript that is stuff that runs in the clients and you are
probably doing some sort of SQL, which is often embedded as strings in your
general purpose programming language. So it's [indiscernible] programming in a
bunch of different languages that are glued together in a not very well way.
So you get -- and our programming environments don't provide support about the
7
integration of these languages and they perform pretty poorly.
So with WebDSL, the slogan that we came up with was separation of concerns,
which is provided by all these different languages, right. So HTML is good for
presentation, SQL is good for database queries. But linguistically integrated.
You want to integrate these languages and let your programming environments be
aware of these languages and their integration.
And so Mobl is another -- grew from that experience with WebDSL, but it targets
this different domain of client-side applications. So rather than the untyped
environment of Javascript, it's statically typed. Rather than having these
separate languages for separate concerns which are not integrated, we have
different languages for different concerns, but they're integrated into a
single language, and it's declarative and concise, as I hope you'll agree with
me after this talk.
And just the results, you can program web applications for touch, right. So
these are all screen shots from applications built with Mobl that's run in a
phone and you can -- well, they have the usual touch interface, and they feel
like native applications.
And they can actually run on different devices so they can run on Androids and
tablets and they can use different form factors as well.
So, yeah, so the platform gives us portability and the language gives us a nice
way to perform this platform.
As far as the architecture, we have an IDE in which you write Mobl programs.
These Mobl programs are compiled to a combination of HTML, CSS and Javascript,
mainly a lot of Javascript. You deploy that as an HTML file on your server.
The Mobl device fetches the HTML file and as a result all the Javascript files,
which may be cached on the device so you should be able to use it offline.
Locally, the application can use the local database and other APIs, and it
typically then talks to a web server to JSON serves to transport data. Okay?
So this is the [indiscernible] world Mobl. Let me say, so this is a complete
application. And, well, I should -- you should first look at the code, because
it's very complicated.
8
So we say well, there's an application called tipcalculator that imports the
Mobl library and the application is a single screen, it's a root screen, that's
the thing that opens the IDE folder when you open the application, and we have
two local variables, amount and percentage, and this is a bit of [inaudible].
There's a header titled tipcalculator. And there's a group with three items.
The first shows the value of the amount variable. The second shows the value
of the percentage variable, and third is a complicated mathematical formula
that performs a computation based on these two.
And now these num fields actually is an edit interface so you can edit these
values. And whenever you edit one of these values, then the computed value
changes. So let me -- I should actually be able to show that. I think it's
15% here, right? So you see that whenever I change the percentage, the
calculated value changes automatically.
>>:
Not necessarily to the right value.
>> Eelco Visser:
I'm sorry?
>>: 1% of 245 is 287?
>> Eelco Visser:
17%.
Change that to 1% again.
It's a total.
>>:
The printing is only printing 24 and not 30.
>>:
I see.
[multiple people speaking]
>> Eelco Visser: I don't necessarily understand this.
total, you have to pay, that's the --
This is math.
The
>>: So one way to understand this screen root is it's a little bit -- so it's
declaring the user interface and but it's also binding data to code?
>> Eelco Visser:
>>:
Yeah, so it does data binding.
Does data binding?
>> Eelco Visser:
Yeah, so there's data bonding from the UI to variables.
So
9
that's all that's going on here. Also, there's a number field that says show
that value, but it's also num field is also an edit field. It's like an input
field in a web forum that you can edit. And as soon as you edit, change the
value, then that changed value is bound to the available and it's propagated to
wherever that variable is used.
>>:
If things are circular, your compiler gives an error?
>> Eelco Visser:
>>:
No.
[indiscernible].
Okay.
>> Eelco Visser: That's another question I got in a previous talk and we don't
do that currently. That would be interesting. But it's reactive programming
so this value reacts to changes of the model, and we could do interesting
analogies there. And I'm not sure, we haven't looked into that yet.
>>: But you use it to compute a fixed, point, though? If you know that's the
value, it doesn't actually change the value. Does it continue propagate?
[inaudible].
>> Eelco Visser:
>>:
If it's circular, yeah.
I think it loops.
So how do you deal with more complex expressions [inaudible].
>> Eelco Visser: So the idea is that this should be [indiscernible] values.
It's very straightforward, reactive programming. Yes.
>>:
Are these styles the UI have CSS-like support?
>> Eelco Visser: Yeah, so there's a styling language that allows you to style
these things. The library has default style attached to it, ask. You can
change that, and it does use CSS to do styling, but you can do it dynamically
and compute it, actually. And there are some [indiscernible] sections for
doing this.
All right. So let's move on. So what we use is the model-view pattern, as
we've called it. Normally, use a model-view controller pattern. And the
controller is something in web programming that takes the changes. In the
view, you make the changes and you send it to the controller. It analyzes the
10
changes and it updates them all and then you update the view.
And we found that that's often the controller is responsible for a lot of the
accidental complexity in programming. And here, you can see that it is absent,
right. So we just say the Mobl is these two variables. We have a presentation
of these variables that shows, well, given these values, show this on the
screen, but it also takes care of data binding. If an event happens to the UI,
we take care of taking it back to the model, updating the model, and, as a
consequence, updating the view.
And so we get rid of this intermediate controller. And, of course, the model
can be a bit more complex than a bunch of number variables so we have data
models, entities and logic. And in view, we have event handling and
presentation. So let's look at a more complex example.
So to-do list, we have a list of tasks that you can check. So what we have is
this task model. So tasks have a name and a dom a done field, a due date, and
we have categories and tags. And so in -- so in our language in Mobl, you can
represent this where it's an entity language where you can say we have an
entity task, which is a name, and a done field and due dates and a category
which is a single category and we have a tags, which is a collection of tags
and then we have entities for category and tags. And you see here that these
are inverse, which means that the properties are automatically synchronized.
So if you add a tag to a task, then automatically the task is added to the
tasks of the tag.
You see the name field is annotated as searchable, which means that we maintain
automatic search index in the device so we can do search. And so these data
models are all that you need to represent data, structure data in your
application, and persistence of that data in a local database is done
automatically for you. So if you create objects and they're automatically
added to the database, and you make changes to them, they're also recorded in
the database. So you don't have to think about persistence in your
programming.
>>: I'm sorry. Could you go back?
are we [inaudible]?
>> Eelco Visser:
So this is all, it's all value based?
So these are references to objects.
Or
11
>>:
And what's the inverse?
>> Eelco Visser: That makes sure that you have an automatic dual binding. So
if you add a -- if you say I have a category object and I add a task to it, so
category, task, adds a task, then the category field of the task is set to that
category. So you get an automatic ->>:
I see.
>>: So what if you added two different categories.
second one?
Then you gonna clear the
>> Eelco Visser: That's right, so since category one too many, you can only
add one category.
>>:
[inaudible].
>> Eelco Visser:
>>:
It is still going to be in the other category, right?
>> Eelco Visser:
>>:
Okay.
That's what the --
Remove it?
>> Eelco Visser:
Yeah.
So when you --
>>: So if I take the task and add it to Category 1, right, and then I add it
to Category 2, that second add will automatically remove it from Category 1?
>>:
[indiscernible].
>> Eelco Visser:
>>:
That's [indiscernible].
You can think of [indiscernible] just one data piece.
>> Eelco Visser: Basically, in the database, there's a single relation that
represents these.
>>:
And what if you have a category and you remove a task from a tasks field,
12
what happens with the task?
>> Eelco Visser:
>>:
The category field, though, the task becomes null?
>> Eelco Visser:
>>:
Yes.
So these things can be null or have values?
>> Eelco Visser:
>>:
It becomes null.
Right.
Are all the values nullable, or only --
>> Eelco Visser: Only the data references. Well, in WebDSL, we do the same
thing, where we're -- it depends, basically. Yeah, whether -- typically, if
you carried a new object, then value fields are initialized to default values.
So an empty string or zero or fold or something. But you can actually set them
to null as well. Whether you want that, you may want to configure in your data
model as well.
>>:
I see.
>> Eelco Visser: You could say some fields are not nullable and that's always
have a value. Yeah, so always a discussion whether you want that or not. It's
application dependent.
Yeah, so these are sort of things that you really would like to have in regular
programming languages as well. I mean, a lot of accidental complexity in
object or in programming comes in maintaining these sorts of relations.
All right. So you can add methods to entities. So basically, it's simple
[indiscernible] language, and, yeah, simple imperative language inspired by
Javascript. Details are not so important.
But the important thing is it's statically typed so we have a data model and we
check the functions or [indiscernible] for the data model and so on, which you
don't get in Javascript. Yeah?
>>:
So it looks like you've got some daytime in here.
Are you, like,
13
leveraging the Javascript built-in library and adding annotations to that, or
did you create your own?
>> Eelco Visser: No, this is really -- no, so underneath, things are mapped to
Javascript and the language is actually, you can directly import Javascript
libraries and you can also embed Javascript and then encapsulate that as a
function, as a Mobl function. So in those parts, we don't have static talking,
of course. But the idea is in the code that you write yourself, so this is
real Mobl codes and that is checked statically.
>>: But did you have to write get full year [indiscernible] yourself around
the Javascript.
>>:
The only thing that was annotated was the typed, right?
>> Eelco Visser:
>>:
No, you have to make those [indiscernible], yes.
So the day times were [indiscernible]?
>> Eelco Visser:
I suppose, yes.
>>: So what about the [indiscernible].
asynchronous thing, right?
>> Eelco Visser:
>>:
Obviously, that's going to be some
Um-hmm.
So how do you deal with asynchrony in this?
>> Eelco Visser: Well, so we do an automatic [indiscernible] transform. And
the only pattern in Javascript that's problematic is that you have to write
everything in [indiscernible] style. Writing sequential codes, you write a
function that wraps everything and takes the code back and then you get this
terribly [indiscernible] codes. We do that transformation automatically. So
you write sequential codes and for those things that are marked as
asynchronous, it turns into ->>: An example, basically between the initial request and the [indiscernible],
anything can happen, right?
>> Eelco Visser:
Yeah, so basically you turn it into a function with a call
14
back that caused the results.
>>: So what I'm asking, it's imperative to be [indiscernible], right? The
system is imperative. We have no clue as a programmer what data can be changed
[inaudible]. That's not a concern?
>> Eelco Visser: No, so this really takes, I mean, this is part of the
call-back. So this is turned into a function that takes the rest of the code
as a function as code when you get the results.
>>: Other events can happen in between, right?
between those.
So the state can change
>> Eelco Visser: Right. Okay, then well we saw the reactive UI so this is a
bit more complex example. So again we have a root screen with a local phrase
variable that we're going to use for searching. We have a header which has a
button and if we click it, we will go to a normal screen. We see that on the
next slide. We have a search box that passed the phrase variable, that passed
the reference. And we have group where we list the results of searching for
this phrase.
So we're searching for phrases for tasks that have the name that match this
phrase. I mean, and 20 of those. And we show a check box and a label. So
what happens if we type something in the search box, then the value of the
phrase variable is updated. So this phrase is passed by reference. So if
search is like the number field we saw before, it's like a normal user
interface element that you can with data binding. It's bound to the variable,
this phrase, and again the value of the search depends on the value of the
phrase. So as soon as phrase is updated, then the results of this search is
updated, and the list is specialized to list off things you're searching for.
Yeah?
>>: So do you do some incrementalization, or do you just re-run the entire
code?
>> Eelco Visser: Well so what we do here is we say that these -- it's a
[indiscernible] model so we note which variables are depending on which
variables and if they change, we recompute those things.
And there may be room for doing that more in batches.
So this is done locally,
15
but if you would go to the server, then you may want to do this on every key
stroke with [indiscernible] bits. So there may be different policies you want
to apply in different circumstances.
So now if you click the adds button, what we do is call an auto screen so add
task is defined as another screen and it's, again, it creates a new task object
and which you can edit and when you're done, there's a done button over here
which, actually, I say that is automatic, but if you want to add an object to
the database, you have to do that explicitly. Otherwise, you might not want to
save this particular object and all these [indiscernible] objects in the
database.
So only for creating new objects, you have to expressly say this should be
added.
And then you do a screen return, which is like a return, but then it also does
a UI return. So we go back to the previous screen and the nice thing here is
it's a stack model so you don't have to remember what auto screen this things
was culled from. Usually in web programming, you have this continuation
passing style where you explicitly need to know what context you are so that
you can return there. Here, we maintain a stack.
And the nice thing here, we can do this in this domain because it's all
stateful, and it's client side. In normal web programming, you have to go from
the client to the server and then back. Since here we can maintain a stack of
screens that you have visited and rewind that screen, that stack.
Okay. And, in fact, you can -- screens can return a value like normal
functions so you can have a button that's calls screen to ask for results and
returns that result as a result of the screen. So it does local data binding,
right, so the screen is a local variable that is bound in the text fields and
the value of the field is returned on the return stack, and so you can have
expressions that call for multiple new screens.
So one thing we learned from the WebDSL experience is in WebDSL, we made too
many things as primitives, and the philosophy of the design of Mobl is that it
should be growable and you should be able to define as much as possible in the
library of the language. Which means that in Mobl you can define things like
tabs and these sort of UI, which is as library definitions. A lot of nice
primitives in the language. So you can make them reusable controls.
16
So here we see a tab set so what is a tab? It has two thingies, it has a row
of labels that you can select tabs with, and then given a particular tab, you
want to show different contents. So we see here that the tab sets have a list
of controls, so pairs of labels and controls and a current tab. And then each
tab is just a normal control. So control is like a screen, but you can define
controls to build up more complex screens.
And you can pass these to the tab sets control. So the tab set control is
defined in Mobl itself so it takes a list of pairs of strings and controls and
an active tab so the currently active tab, which is a string. And then it has
two lists, one to create the tab index at the top. [indiscernible] over the
list of tabs, where we match each tab to a tab name and tab control, and we
create a block where we -- that is clickable so when you click it, we change
the value of the active tab variable to the name of the current tab that you
control, and we change the style from -- to active, and we display the name of
the label.
And then in the contents of the tab sets, we also iterate the tab.
the tab control for the active tab.
So we call
>>: Seems like declaring some new structure, right, as a code that you laid
out in a certain way, that as you show here, you call, basically, a code. So
what sort of -- I have a hard time understanding exactly the model of this type
of call here. Is that just think of it as streamlining the declarations for
the tab control in there, or what does that mean, exactly?
>> Eelco Visser: Yes, you see what is a control? It's a bit of UI, a bit of
rendering. So you've given some values of its parameters. It shows some text
and structure on the screen and it has data binding so it can handle events.
And, while that's a bit off code that you're abstracting from, that you make a
reusable abstraction of.
And basically, you're realigning that, but [indiscernible]. That's the idea.
So rather than having to define each page completely, you can make abstractions
over smaller bits of a page, of a screen, and you can make a library of those
things and compose complex library pages from screens from ->>: So you know in Mobl that this is a call to something that's just UI?
not -- it doesn't actually compute something? It's not --
It's
17
>> Eelco Visser: It's a control. The control is the abstraction here.
screens and the controls are the UI ->>:
So the
Typing takes care of knowing what this all means?
>> Eelco Visser: That's right, yes. So you hear me saying that
[indiscernible] set, we're expecting that there is a control, and controls are,
well, things like tab sets itself. It produces something of control. Those
things can be passed in these, yes.
And these things are complex things. That's what domain is about. So in
WebDSL, we have this motion of templates, which is similar. It is rendering
some data, reacting to events that are passed to the UI. And that is the thing
we are abstracting about, sort of one of the strengths of this language, that
we have this -- that the abstracts are a combination of rendering and behavior.
>>:
Could you [inaudible] twice, tab control, tab control?
>> Eelco Visser:
>>:
Yes.
And you just get two instances [inaudible].
>> Eelco Visser:
Yes, so you would see the contents twice.
>>: But would it be sharing the underlying data binding, or would it be
[inaudible] independent?
>> Eelco Visser: I suppose if the tab control has references to data, those
references would be shared. Let's say if you would say here, we could call
search box phrase twice, thine get two search boxes.
>>: But the tab control, the tabs that you're passing on the next page have
their own ->> Eelco Visser: But that's the same thing. So the point here is that if you
duplicate the search box, then you get two search boxes. And if you type in
one, the value of the other should be updated to the other.
>>:
[indiscernible] control, yeah, control -- one of those variables in the
18
control tab locally ->> Eelco Visser:
tab sets, then -somewhere. Here,
say this variable
page, as control,
be close.
Right. Local variables, those are local. If you have two
where was a local variable? We had a local variable
for instance. So this -- okay, so it's not a control. But
is local to this screen, and if you would call it twice on a
then each would have its own local variable and, yeah, would
Yeah, so I also detail other example of higher order controls, where you say,
well, we want to have this pattern where you have a list of items and when you
click on a particular item, your get the details of that. And in that pattern,
you can define as well. And again, the same sort of higher level, higher order
control where we pass -- we have a number of items in collection that we want
to display, and we have the master item control and the detail control for each
of those. So we call for each item.
So in the list we call the master item for each item and then you click on it
and you call the detail screen, I suppose, that should be detail rather than
detail screen. Oh, right, that's over here.
>>:
A question [indiscernible].
>> Eelco Visser:
>>:
Yeah, so this is for --
[indiscernible].
>> Eelco Visser: Here, you can have value one, but you can in general have
more. Well, so this is not really typed over the generic variable.
>>:
[inaudible].
>> Eelco Visser:
it.
>>:
You can write T instead of the question mark, and --
>> Eelco Visser:
>>:
Well, so we don't give it a name so we don't actually match
That's right.
And then use or maybe task or you know.
19
>> Eelco Visser: Yeah. And so then you can actually implement this master
detail thingy in this way, so if you have a small screen, it's a single list.
And if you click it, you go to the details. But if you have a device with more
screen real estate, you can have a list and the details on the side as you
click it and it changes to the -- and this can be done automatically, right.
So without changing the implementation, it will adapt to the particular
application or to the device that you're running this on.
So this actually has been used by a bunch of people outside our group, and so
people from companies are developing all sorts of interesting applications with
it, and even music player application and a wellness application. So they have
really adapted the style.
>>:
So they just messed with the CSS?
>> Eelco Visser: Right. So you can make -- so the nice thing is the widgets,
you can make your own library and implement and, yeah, and provide it a
completely different look and feel.
So at the bottom, Mobl doesn't provide any building widgets, so it's all the
library defines these higher order controls. And at the bottom, implements
these things in terms of data binding, abstraction and HTML fragments and
Javascript goes. And so it's really flexible, but it's also leak in the sense
that there you have the leak to -- you can do anything in Javascript, but we
don't have control over it either. So we cannot penalize it.
So I'm always a bit ambivalent about it. I like DSLs that are closed and you
control the whole language. The problem with that is that there's no escape,
and if you want to make new features, you have to add it to the language that
makes your extension process very slow. And ->>: That's always the big problem, right, the DSL? You find a DSL.
[indiscernible] may not have thought about the big, like the [indiscernible].
>> Eelco Visser: Yeah, so it's a trade-off between coverage and -- so in Mobl,
you have very good coverage. You can write any web application codes that you
can write in HTML and Javascript and embed it in an application, but you lose
portability in the sense that now if you decide, well, actually, rather than
generating web code, you would like to generate native IOS codes, then, well,
20
all those libraries are not portable.
We actually did have a student who did write a back end for Mobl to IOS, also
generating objective-C and what he did is take the library and did the library
interface and make implementations of all those over a certain number of
widgets in objective-C directly and then you can actually do generate native
cause.
If you decide to stick to a certain API, then you can still get portability,
yeah.
All right. So high level overview of Mobl. And so a couple of things to point
out. This idea of linguistic integration, so we integrate data modeling, which
you would normally do with RM or SQL. User interface, abstraction style, I
didn't show that, and application logic. And they're all different aspects
which have their own linguistic support, but they're integrated and the type
checker knows about them and compiler knows about them and can do interesting
stuff with them.
And there are more interesting things to do, like actually analyzing the
reactive programming model that you have.
It really makes writing these applications a lot easier. So there have been
many people from outside who have played with this and really like the
programming model.
So some points are data binding, where you write a field and it's automatically
tried to combine displaying the value and binding changes to the value. The
reactive programming model and these reusable higher order controls. We also
find that it is model view better and works very well. So getting rid of this
controller that you have to do the program yourself to -- how you have to react
to the two events.
And another strong point of Mobl is that it's IDE, which in -- well does all
this checking for you. And it's highlighting and it does check these variants
and has code completion and, well, all the things you would expect from a
[indiscernible].
And yeah. So and that brings me automatically to the second talk, if you still
have the energy for it. Well, so maybe there are some questions about the Mobl
21
part first.
>>: So the whole user interface part seemed -- I haven't seen a previous talk
where the user interfaces are designed using [indiscernible] syntactically and
semantically very similar to function calls, yet they are not actually
functions. Did you come up with that yourself, or was there prior ->> Eelco Visser: No, that sort of emerged in WebDSL, basically. So there we
had this idea of page definitions for web pages and navigates that are links to
pages. And in WebDSL, you can write these things and then so you build up a
page with the vendors and does data binding as well. And there is indeed the
analogy where the page definition is like a function call. To navigate a link
is like the function definition, and navigate is like a function call.
And if you declare these things explicitly in your language, rather than doing
URL or arithmetic, which you normally do in web programming, where you create a
URL by string compositions, much like pointer arithmetic in C, you can
actually, we have this [indiscernible] where you can navigate to a page. That
page is declared these arguments where you pass objects to this thing. You can
statically check that the link you are generating to a page that actually
exists and it has the right types of arguments and it automatically does
binding of -- knows how to pass an account object to another page. Lots of
stuff you can abstract from and have nice properties.
And then the screen is like a page and it's the full, the full page and control
in WebDSL is a template that says well if you divide it and make a play,
there's actually often patterns that you want to reuse, and we want to make
abstractions for those so that we can, in many places in your application, you
can use the same pattern. You can compose pages from smaller building blocks,
basically.
>>:
What's the difference between a page and a control?
>> Eelco Visser: Well, it's like in Mobl, it's a screen and a control. A
screen is always the full screen. And a control is a part, a part of a screen.
>>: But I guess you could think of a control, you could just go to a control
if you want to go to a control that has a screen.
>> Eelco Visser:
No, the idea is that you need to make a screen.
22
>>: Do you have first class functions too, or are they hidden in your
[indiscernible], or is there some way to make it land on [inaudible].
>> Eelco Visser:
>>:
It's in controls.
>> Eelco Visser:
>>:
So we have higher order controls, right, which are --
Oh, but do you have [indiscernible] controls?
Yeah.
>> Eelco Visser:
I haven't seen them, actually.
>>: If you don't need them, that's fine, right?
wondered.
>> Eelco Visser:
I don't care.
I just
It should be easy to do, but I have to admit I don't know.
>>: I'm more interested, because in Javascript, like it's first and foremost,
right? You have to have all these things all the time. You've now in a way
made it so you don't really need them, right? Which is easier.
>>:
I don't think it would be very easy either.
>> Eelco Visser: No, but you have closures. And so with WebDSL, we were
targeting Java, and there we don't have the higher order templates, for
instance, because mapping to Java is much harder. Mapping to Javascript, we
have much more power there and it's easier to do define higher order
abstractions than those with Java.
So if you have a few minutes, I'd like to talk about this.
ask.
>>:
But maybe I should
[indiscernible].
>> Eelco Visser:
No, I should be done by 12:00.
>> Sebastian Burckhardt: So anybody who wants to meet with Eelco, send me an
e-mail. He's here today and tomorrow morning.
23
>> Eelco Visser:
But feel free to.
>> Sebastian Burckhardt:
the talk is up.
I'm just saying that now so if people leave before
>> Eelco Visser: Okay. So yeah, so we've made this -- we have made Mobl with
Spoofax, which is a language workbench. In the past we would could this a
programming environment for creating programming environments.
So this is the other part of the challenge, automatically deriving language
implementations from declarative language definitions. And so if you're -- so
who is a compiler writer here? I know you are, right? Written -- well, you
have written compilers, I suppose. But you know this. So, I mean, traditional
compilers, you need the grammar for the syntax or at least the [indiscernible]
to text your strings to an ASD. You need to define the static semantics, to do
error checking and name resolution and type analysis. You may have some
model-to-model transformation, where you transform your ISD that you can put in
your code generator and a code generator that's [indiscernible] some final
codes.
So these days, languages are not accepted if they don't have a proper IDE to
support them, so IDEs give a lot of extra productivity over the language
abstractions that you have. See things like all sorts of editor services, like
syntax checking and bracket matching and syntax highlighting, code folding and
outline view.
And semantic editor services that show a long list of errors in your terminal.
You get in-line errors and you get reference resolving so if you click on an
identifier, you jump to the declaration. Code completion and refactoring and
so on.
So if you want to make a language, you want to make your traditional compiler
and also make an IDE. And what we want to do is make those things from a
single language definition. So rather than having [indiscernible] batched on
top of your compiler, we want to make them in one go.
So we do that in two parts. We have a language for grammars, called SDF for
syntax definition formalism, and a language called [indiscernible] for program
transformation that rebuilds languages within Spoofax.
24
So SDF is a fairly typical grammar formulaism, but what it does, it integrates
-- it's a character level grammar so it integrates lexical syntax in the
[indiscernible] rather than defining a separate lexical syntax with regular
expressions. We have a gel grammar that defines both the tokens and the
characters. We still mark things as lexical syntax and context-free syntax but
the production that we use are actually the same.
The difference is that symbols that occur in context-free syntax can be
separated by layout. So we can have wide space increments between these places
in context-free syntax and it's implicitly declared humorous, and in lexical
syntax, these things should be concatenated with the intermediate layouts.
What we also do here is declare constructers for the extra syntax, so given a
syntax, we get a mapping to an ASD.
And, well, so given the grammar you already get a lot. You get a syntax aware
editor and the nice thing about Spoofax is you can develop those things in the
same environment. So in eclipse, you develop a grammar, you develop your
language, you build it in the same eclipse instance you, get an editor for your
language with highlighting and so on.
Actually, the grammar I showed you was ambiguous, if you would type ->>: So when I give a grammar, I get this editor.
project?
>> Eelco Visser:
>>:
A project?
Like a language project?
>> Eelco Visser:
>>:
Do I also get like a
Yes, you do.
Okay.
>> Eelco Visser: So, yeah, well, you define these things in an eclipse
project. In that project, you create a grammar and you have transformations
and so on. What do you mean? Like a project for your language?
>>:
Yeah, like do you generate some, you know, where I can see my language
25
files and stuff.
>> Eelco Visser:
show in files ->>:
In eclipse that's jen rake so you can just make projects and
Digital studio, like [indiscernible] hidden fields.
>> Eelco Visser: Yeah, yeah, but you can add. So, for instance, Mobl has
buildable safe, which is built into the language so if you save a file in a
Mobl file, it will automatically compile that ->>:
You get like separate descriptions, like --
>> Eelco Visser:
>>:
Part of the language definition.
Okay.
>> Eelco Visser: Actually, SDF can deal with ambiguous grammar so you can
[indiscernible] grammar is an issue. You don't get an error, but you can each
get -- debug those ambiguities by inspecting the resulting tree. So here we
have X equal Y times Z, and we didn't specify priorities so what you get is a
tree that has bugs, protections. So rather than having to debug conflicts in a
parse table, you can inspect the ambiguities and resolve them at that level.
We have declarative disambiguation for things like priorities that you can now
use to resolve these ambiguities. And then if you have resolved them, you get
the proper results.
So a nice detail, while I'm here, is that this idea of showing -- so what's
happening here, I selected a part of my programming in my language, and I
clicked the show ISD item in my transform menu and I get the term that
represents ISD here and it was inspired by the Oslo project which has project
which had this -- you know Oslo, right? Yeah. It had this thing where you
typed the program and you got the ISD on the right side. I thought that was a
nice feature so I added it to the Spoofax.
All right. So something we added is error recovery for generalized parsing so
SDF is based on [indiscernible] generalized [indiscernible] parsing, which
allows you to pars any kind of arbitrary [indiscernible] grammars. In the
past, that wasn't supported by -- we didn't have error recovery for that.
26
Recently, we added that. So now you can write programs that have syntax errors
in them. So here we have a missing closing parenthesis and here this screen,
keyword is not finished. And because of this error recovery, we can actually
continue parsing and we get an ISD. And based on the ISD, we can provide
semantic services. So we still see error messages and we can do resolution
even in the face of syntactic errors.
And if anyone would be interested in this, I can have more to say about it.
So that's the syntactic part. And for the semantic parts, we do
transformations with the Stratego transformation language, which we use for all
sorts of semantic operations. So typical normalization. So syntactic
normalization, let's say we have a special case of the if statement without an
else statement. Yes?
>>: So this used to have another rewriting formula, ASL. Is Stratego like a
replacement for that, the next generation? Is that how to think of it?
>> Eelco Visser: If you want the history, I used to be a Ph.D. student in a
group of [indiscernible] in Amsterdam where we developed [indiscernible] for
SDF. I worked in SDF while I was there. Got this extension of SDF, and ASF
was a pure rewriting language so it only has rewrite rules. I became
frustrated with this, and so after my Ph.D., I went to Portland and developed
Stratego there.
Stratego has rewrite rules, but it also has programmable rewriting strategies,
so you can define your own -- rather than just having, say, give an
[indiscernible] strategy that takes all the rules in your whole program and
applies all of them exhaustively, you can say, well, now I want to apply -- I
want to take these rules and apply [indiscernible] reduction to this tree.
>>:
Okay.
So you can control the --
>> Eelco Visser:
>>:
Yes.
Okay, thanks.
>> Eelco Visser: So it's a rewriting language and it has strategies to control
those. So we do, well, typical desugaring and syntactic normalization, CPS
transform, type analysis. Given an AST, rewrite it to an -- an AST that
27
represents a type. So we have a string, the type of that is simple type with
the qualified ID Mobl string. We can do type checking, so given type analysis,
we can say, well, a simple type should, if you use a type somewhere, it also
has a type of a property and a declaration, for instance, or a type of an
argument so it should be declared. So if we fail to look up that type, then we
want to give an error message on that type expression that says that the type
is not defined.
Or if we have a property expressed of expression dot fields, and of the
expression has a certain type, but the whole thing doesn't have a type, that
means, well, that type doesn't have that property, and we want to give an error
message. And we can see here that we -- and these constraint errors are
interpreted by the IDE, and displayed by us in line errors. So here we get an
error message. If we hover it, it says type is not defined.
And then the next thing is in our language definition, we don't have to deal
with the binding of these error messages to the IDE. We automatically get file
origin tracking. We say the error message should be placed on this AST, and
the editor of the environment knows where to place the -- put the squiggly
lines in the IDE.
And then we can bind these analyses to semantic error editor services. So, for
instance, the editor analysis says, well, analyze my program and we turn a list
of errors and warnings and notes. And then true origin tracking displays those
in the editor. We can do completion. And so given a typing, the term that
we're typing, propose -- so, for instance, here if we -- so this is the Mobl
editor. If I say, you know, so propose some value or propose a type, then it
will look into the current program and propose valid types.
And it does reference resolving. So if I control click on an item, it will
jump to the declaration and that sort of stuff.
And if we make errors, then it will still continue in -- well, it will give
error messages based on that and it will still do that in the face of syntactic
errors as well.
So I can, if I approve this, I will still be able to do reference resolution
even if we have syntactic errors.
And so the nice thing here is that we have a very thin interface between the
28
semantic analysis and the IDE framework. You just write your analyses in
transformation Stratego and then using an interface like this, you can bind
that to the IDE. So the interface for this editor analysis is that we can AST
and a bat and after we [indiscernible] with these things, then it knows how to
deal with those and is not concerned with the actual analysis.
And we have some languages for declaring these services so you can declare
binds, a combination of syntax definition and transformations to an IDE using
so-called billers and that's on the interface between the IDE implementation
and your language definition.
And so as a regular language developer, you have very little concerns about
details of eclipse implementations.
Recently, we added a testing language. You can actually express tests for the
language you built. Typically, you test these things by making programs and
clicking around them, seeing if everything works. Now you can make these tests
in a unit testing style where you quote a small piece of program and define
what it should, should it test syntax but also reference resolution and type
checking and so on.
And the nice thing is, you can do this in an editor and get syntax highlighting
for the language that you're developing in the testing language as well.
All right. That's it. So couple of ideas. Linguistic abstraction and this
idea of linguistic integration is important, and we have a nice workbench for
creating DSLs. Okay. Thanks.