1
>>: Welcome. My name is [indiscernible] representing Zapfest. Tonight you
are here to listen to Peli to talk about John Peli talk about [indiscernible]
or apps for your phone. Without further ado, John Peli.
>> Nikolai Tillmann: I'm Nikolai Tillmann, and I'm here together with Peli de
Halleux. We are two members of the TouchDevelop team, and we are going to talk
about a new way how to write programs on your phones.
So what we are all very much aware of now, in our everyday life, is computers
are shrinking. We have seen different input devices and the way we surf the
internet now is totally different than what it used to be.
You can just touch on a
invention of the mouse,
think about is that all
everyday tasks, but how
screen and move things around. Before that, with the
we could drag around things. And so what I want you to
of these new input devices, they are not just fun for
can we actually use them for programming?
There are many applications out there, and writing programs is one of them. So
devices are getting smaller and smaller, so you might think that they're
getting more limited. But in many ways, the latest generation of devices
smartphones is more powerful than ever and one point that we point out is that
unlike a PC, you have your phone always with you. That's dramatically
different from how you interacted with a PC in the past. You always have the
phone with you.
And also no secret is that if you look at sales numbers, smartphones are taking
over. PCs kind of not going so fast anymore. So if you just look at the
numbers here, there's a billion units sold. In a few years, literally every
person on earth will have a phone. So you always have it with you, and
everyone on earth to some extent will have a smartphone, which is a general
purpose computing device. This is a full-blown computer.
And in another aspect, it's a much more fun computer than the PC ever was,
because it has many more sensors. The PC had, what, a screen, a keyboard and a
mouse. And what does a smartphone come with? Touch screen, of course,
camera's not on front but also back, always connected to the cloud. It can
talk to other devices with Bluetooth, it has GPS, it knows how fast you are
going, how much you're accelerating. Much more sensors than ever before.
2
As great as this computing device is, if you think about the state of the art
of creating software for smartphones, there's a big disconnect. So you create
smartphone software by using a PC, using a using a traditional set up. These
languages are existed before mobile devices ever existed.
You're supposed to write text with a keyboard. You can use a mouse if you
want. And then to try the programs out, you use an emulator and maybe deploy
it on the actual phone.
So that is state of the art, and so I guess everyone here
been writing Windows phone apps already. Is there anyone
a Windows phone app in the past? Oh, a bunch, actually.
I can ask everyone who did write a Windows phone app, did
marketplace? Who submitted it to the marketplace? Oh, a
in the audience has
who has never written
Okay. So then maybe
you submit it to the
few people.
Who got it accepted the first time? Ooh, one person out of the entire
audience. So that actually maps our experience and statistics. It is pretty
tricky to get apps out there, not just because of the software you need, the
skill set you need to have but also because of the process you have to go
through.
So if you go back in time, then when I go up and I'm sure many of you have had
similar experiences, that is what my computer looked like when I turned it on
for the first time. It wasn't a smartphone, but pretty cool, and it wanted me
to write code. That's really -- it's not the greatest [indiscernible]
experience, I admit, but it shaped an entire generation. You turn it on, you
get exposed to the idea of writing code. And how is it now you get such a
powerful device and what's been an experience provides is that you just get
finished apps from the marketplace, and you are a consumer.
By default, you are a consumer, whereas it used to be much more permanent to
actually create content, create apps. Can we change that? That is the
research question that our group here in Building 99 took on, and we came up
with a radically new development environment that we call TouchDevelop. It is
actually a Windows phone app. It's available for free in the marketplace right
now. And it's full development environment that also pegs social collaboration
features.
It allows you to write code, run code and share your programs with other
people. And after all the talking, I will now hand over to Peli, who will give
3
you demo. And keep one thing in mind, everything we're going to show you is on
the phone only. There's no PC required. The whole idea behind our project was
that you might not even have a PC. And yet, you should be able to write
software. So Peli.
>> Peli de Halleux: All right, so my name is Peli de Halleux, and I'm also a
developer in the TouchDevelop team. And today we're going to write an app
together. So before we write the app together, we should get the phone here.
So if I had a live stream, I could actually write the app and keep walking
around. I do sometimes, and sometimes I find myself coding while biking. But
I won't do that today.
So TouchDevelop allows you to write apps everywhere, and they allow you to
write apps for your phone. It's a pretty cool environment to be creative while
you have something to do. So I'm just going to remove the auto focus here.
So it's an app and we're in Windows phone. So I'm going to launch
TouchDevelop, and you land on the hub where you can browse the news scripts
that have been published by other users and you can look at whether you've got
more points or so forth. But we're going to show that later.
We're going to start by creating a new script and writing some code. So I'm
going to tap on the plus button here, and I can pick a cool name. I'm going to
call it Zapfest 2012 and create. So we have a script as an app. It bundles
code. It might bundle events, and a bunch of assets and we're going to go into
the code editor, which we call action. So there's an action called main here
and I can just tap on it to open the editor.
So here we are in what looks like a traditional editor. So we have a typed
imperative language. It's a pretty simple scripting language. If you have
ever done C, java, language like that it's pretty intuitive. We use the arrow
because you don't see dots on small screens.
So I can run this code. So I can hit the play button down there, and I can
just run the code, right. And at the time of Nikolai, this would have been
excite be to be able to display strings, but these days not so much. So we can
make it a bit more interesting. So I can select this, and, for example, I can
refactor that into a flow.
So we have control flow, we have things like -- let's go back and do that
4
again. So I'm go to add a new four loop. You can see we have if statements,
we have 4 H by 4, all the good stuff that you know. So I'm going to do four
loop. And here, you see that there's not a lot of typing to do. All the
boilerplate code is there for you. You can't touch it and you just have to
provide the upper bound.
So in this case, I'm going to use this little number editor and we are here in
the expression editor, which is a specialized keyboard to enter expressions.
So if I press back, I'm back into my little function here, and then I'm going
to say -- I'm going to tap on do nothing here, and then I'm going to add one
more statement. It's going to be a command, and I need to add a string, and
strings live here.
And as you can see, I'm using a big finger on a small screen. In fact, I'm
doing typos here. Cool. And we're going to post the string to the wall. So
TouchDevelop tries to be simple for end users and we try to use part that they
understand. We are in the phase book era so we actually don't consult write
line or print F. We post things to the wall. And that is pretty cool. So we
just did a little loop and we're displaying text.
So we have a rich language. We can really do whatever we want. And we can
also leverage all the cool stuff that's on the phone. The sensors, the media,
and also maybe a calendar and so forth. So instead of doing something that is
cool but cool a long time ago, I'm just going to cut that, what I'm going to do
is maybe pick a random picture from the phone.
So the phone has all my pictures. I'm just going to go and look for media. So
there's a media, and in media I have an easy access to the picture albums, the
pictures. Maybe I can also get the songs. I'm going to grab the pictures.
And in this case, there's a handy method called random.
So this action returns a picture object, and there is a refactoring button here
that says store in variable. So I can just do that, and now I've declared a
local variable and I can come back to my code.
So we've picked a picture and we're going to just display that to the wall
again, adding a new statement, a command. And now you can see that pick is
right at the front of a [indiscernible] so we actually, we've done quite a bit
of work trying to make the best guess and reorganize those buttons. So as you
type, we change the order in order to predict what you're going to try to use
5
in that case.
So we've done that demo quite a bit. So it knows that we want to post it to
the wall. All right. So we're running this and it picked a picture. And it
can run it again and again and again.
So that's a good start. We already did something that wasn't trivial. But
we're going to make it more interesting. So there's been recently, well, it
seems that people want to see a picture, but with some sepia filtering. They
want to see the old school. Yes?
>>:
Will this entire presentation be recorded?
>> Peli de Halleux: So this entire presentation is it recorded?
recorded and it will be ->>:
[indiscernible].
>>:
So what are those arrows [indiscernible].
>> Nikolai Tillmann:
>> Peli de Halleux:
C-Sharp.
>>:
Yes, it is
The question is what are those arrows.
The question is what are the arrows.
They are dots in
They are dots?
>> Peli de Halleux:
What do you call those?
>> Nikolai Tillmann: So it's like the dot in C-Sharp. We, in fact, went
through different iterations and the dot is too small on the screen to see
well. So the arrow is more expressive to select a property of an object.
>>:
Got it.
>> Peli de Halleux: Yes. All right. So what we want to do is tint the
picture. So I'm going to tap on the first statement and I'm going to sneak in
another line here. So I'm going to add another command. And I know that there
is probably a tint method defined on pick, but I'm not sure exactly where it is
so I'm going to search for it.
6
And here I'm going to do tint. And there is actually a method called tint, and
it will tint the picture. So if we go back now and run it, we got a familiar
effect where you can't see anything on the screen almost.
So, in fact, sepia is rather boring. So let's go ahead and change that and,
instead, we're going to pick a color ourselves. So we're going to use a
specialized editor to pick colors, because many times we're going to customize
our colors, a bright one. Pick green, favorite color here. And there we go.
So we've got our green color. It can run again. And now we've got that runs,
and that is actually a picture that runs over this.
Okay. So, so far so good. We've written a little app that's funny. What we
want to do is now maybe make it more interactive, so whenever I shake the
phone, I want to go that again and again and again and see more pictures. And
to make it more artistic, I'm actually going to go back here instead using a
random color, which is going to be more interesting. Random. There we go.
All right. So back to the script editor, we looked at code, but we also have
events. So events are first class, and they allow you to respond to what the
environment is doing to the phone so we can tap on plus to see what we have.
So you can see that we have events like shake or phone face up and down, which
are related to the [indiscernible] meters. We process the [indiscernible]
meter and we turn that into an event for you. And you have more events down so
you have events related to the music you're playing, whether you're starting to
play music or what's the active song on the phone.
You also have events for the camera, because there is a camera. And you can be
notified which the user tries to press the camera. So there's a whole lot of
interesting events that you can register to.
So we're going to do shake today. Tap here and we're back into the statement
editor. And in this case, what I'm going to do is I'm just going to call the
main action that we just wrote. So the way you do that is you access -- you
look for a single [indiscernible] code, and there we go. And then I'm going to
do it again and notice how the button here, as a triangle, and you call code.
But when we put it in the expression editor, we don't put the code. And the
reason is space. The reason is that after a while, you recognize the symbol
and it saves a lot of space. We're dealing with very small screens here.
7
So I can run this. We get a picture, and every time I shake it, shake it,
shake it, we get another picture.
>> Nikolai Tillmann:
>>:
There was a question in the audience?
Yeah, so I see this on [indiscernible].
>> Nikolai Tillmann: So the question is there are green and red arrows at the
top corner of the buttons if you go into the expression editor. What do the
colorful buttons mean?
>>:
[inaudible].
>> Peli de Halleux: You mean some are green and some are red? There is some
internal coloring scheme. These are, the green ones here are singletons and
art is a special one. And based on the return type, you'll have different
colors. And I deleted the code I was supposed to do. Undo. Perfect.
All right. Undo works. We're good to go. And I can shake, right. So the
last thing I want to do is maybe I see a variation of my pictures that I want
to save. So I want to save that to the phone. So we can add one more event.
So let's go back to the script editor and you can see here how interactive
building an app is when you're already on the phone. Again, really play
around, tune it, trial kind of things and delete. It's really playful.
So I'm going to add another event and it's possible to register tap events on
certain kinds. So I can actually be notified when a picture is tapped on the
wall.
So the tap wall picture will trigger whenever the user taps on a picture. And
in that case, it's very easy to save it to the phone. So I'm going to do item
and save to library. And that will save the picture into a special album in
the Windows phone.
All right. So we launched the app and we generate a couple pictures. In fact,
I like this one with the bike one. So I'm just going to tap on it, and I Don't
Know whether it did it. All right. So let's change the app. And let's have
the phone vibrate or maybe play some music. Even more exciting. Let's pick a
random song. And play. So media, songs, random, play. That's pretty
8
straightforward. So let's do it again. Come on. Whoops. Probably pressed a
button. And we're back, and let's see. So I have to find my script, which is
probably called Zapfest 2012. And let's try it again.
All right. I like this one. Okay. It did it. It saved the picture so we're
good. We can actually go into -- stop the music. All right. Going to go into
the pictures and if you go into the save pictures library, you can see that
it's already there. Actually, I saved the previous one. So you can -- these
are all the things that you can do with touch pretty easily. You can play
around, build your app. And then when you're happy with your app, you want to
share it. We all want to show off what we have, and we have to have users.
So in TouchDevelop, in order to submit an app, you just have to press a button.
It's here at the bottom. And it will publish your script to our bazaar.
There's a little disclaimer, don't store passwords in there, because it's all
public. So you hit publish and you have to be connected to internet, of
course. Let's try that again.
>>:
You're in flight mode.
>> Peli de Halleux: I am in flight mode, but I should be also on the WiFi,
which is turned off. Yeah, all right. So we won't publish today. But if we
had published, then the script would show up in the new section. Maybe we'll
finish the demo later when I figure out why I lost the WiFi.
>> Nikolai Tillmann: Publishing experience, I have a different phone here,
which happens to have an internet connection. So I won't be able to recreate
this elaborate script that Peli just taught, but let's create another one.
Maybe kind of a picture. So I want to take a picture with the camera, tint it,
and post it to the wall.
And that would be pretty easy to do. I misspelled the name, but it doesn't
really matter. So go in and add a statement to take a picture. So the camera
is one of many sensors in the phone so I can just go to sensors, take a
picture, store it, and then operating on the variable, I can tint it, similar
to what Peli already showed, I'm really walking fast through the program.
And tint it in a random way and then I will throw it on the screen. So this
program actually does something interesting. Okay. So that's my entire
program. In this program, it will ask to take a picture. So it just took a
9
picture, and let's accept that and here it is.
So cool app, and to share it with the world, indeed, I just press the upload
button for the script and let's correct the spelling mistake. Upload.
So it's now sent to our cloud, and our scripts actually automatically analyzed
in the cloud. We are looking for certain privacy concerns, what information
does it potentially leak. We thought of everything. And now if anyone looks
for all of the scripts that we have in the script bazaar, you see what's
currently happening. So 60 seconds ago, I posted this script and other people
are publishing other scripts.
Okay.
So we just had fun writing scripts.
>> Peli de Halleux:
>> Nikolai Tillmann:
Let's finish the experience.
You're back on the internet?
>> Peli de Halleux: I'm taking another phone.
looking at new scripts.
So that's another phone.
I'm
>> Nikolai Tillmann: And if you have a Windows phone right now, you can do the
same thing. Open TouchDevelop, go to new, and you will find it.
>> Peli
this is
and see
there's
can fix
de Halleux: So this is very similar to a marketplace experience where
very similar to a marketplace experience where you can look at the app
what it's doing. So you can get an idea of what's happening. Oh,
no screen shots. There's a forum. So there's no screen shot. Maybe I
that.
>> Nikolai Tillmann:
Can you like it too?
>> Peli de Halleux: I can also like it, but I haven't run it yet. So I can
actually go and look at the source code directly from the preview page and kind
of get a feeling what's happening here.
So I'm going to download it. This will download the full code of the app and
open the editor, and I can run the app. So it turns out that it's taking a
picture. Nikolai, returning you the favor. There we go. All right. And we
got Nikolai in green. That's pretty cool.
10
What I'm going to do is I'm going to take a screen shot. So this allows to
take a screen shot of the app and upload it to the cloud. So it's very easy to
actually make your app more appealing to have people download it. It's very
important. And to like it, I'm also going to go back to the review page and
then I'm going to add a little light to that.
So I'm going to say that, yeah -- oh, we got already two like, three likes.
This app is on fire.
>> Nikolai Tillmann:
later.
>> Peli de Halleux:
cool. All right.
That's very good.
We will come back to this script
I'm going to drop a message to Nikolai, saying this is
>> Nikolai Tillmann: I just got a notification that somebody liked my script.
So it's a fully integrated experience. When he writes a comment, I should get
another notification.
>> Peli de Halleux:
>> Nikolai Tillmann:
This is cool.
It comes with a bit of delay.
>> Peli de Halleux: And now, in fact, when we come back to the new page, you
can see that there's actually a little preview of the screen shot in the list
of apps to make it more attractive.
>> Nikolai Tillmann: So in just a few seconds, we wrote a script and we
published it. We decorated it with all kinds of metadata. How cool was that?
It was never easier to get programs out there, even on PCs.
So we showed you how we can write some script in a few seconds. Let's look at
what other people have been doing. So since half a year, it is possible for
people to share scripts. And since then we have seen many downloads of our
app, we are really overwhelmed by the response. So we almost had 200 app
downloads per Windows phone. And there have been 6,000 scripts and different
script variations published by all of the users.
And you can see that it's really very diverse.
There are many games.
But
11
also, utilities and you can use Bing maps and you can draw charts. It's a
general purpose forming environment. So let me show you some of the cool
scripts in action that people have been writing.
So I already installed them on my phone, and let's start with a pretty simple
one. You probably recognize this game immediately. So I can use the
[indiscernible] orbiter to tell the phone that I'm tilting it so the game works
by tilting the phone. And then when I tap on the screen, the ball gets going,
and we take away some breaks. Once in a while, I can also use a ripple effect,
but that depends on configuration.
So somebody wrote this entire script on the phone with no PC. I think it's
pretty amazing. And how difficult is that? Probably must be a lot of work,
right? Let's look at the code behind this.
So I can go into the code editor and first of all, you see that there are just
three functions here. This entire game consists of three functions. Main,
draw bricks and game. So when we look at main, there's some setup work in the
beginning. It's basically two pages of code. And then the second page is the
main game loop. So while running, it does something, it basically moves around
the pedal depending on the comp accelerometer reading. And then it counts your
lives and you have three lives and when all of your lives are gone, then the
game ends, obviously.
So we can look at this other function that is invoked here. What does that
function do? So when you are out of lives and it takes you high score and it
posts it to a leaderboard. So they have many built-in cloud APIs in
TouchDevelop and one of them is a centralized leaderboard. If you write a
game, you don't have to worry about where to maintain the high score list. We
do that for you.
So there's one more function, and that is called draw bricks. This one is
interesting because it's very easy here to see how the grid of bricks is
created. So there are two nests of four loops and in the loops, rectangles are
created with different colors. So those are the bricks.
That is the entire game. And you've seen how to add code, right? So half an
hour, after a bit of practice, you can write such a game. And sell for 99
cents, as we will see later. If you keep producing apps every half hour, that
actually amounts to something.
12
Let's look at another one. So here is another famous game. One of our users
loaded. Every day, we are just surprised by what our users can do on the
phone. So I can swipe down to accelerate. I can tap on it to rotate. And I
can swipe to the sides and this is not going to look very good. That's, again,
that a high score, fully integrated experience.
Last game, this one is called cloud hopper. And again, it use the
accelerometer to move around the little guy here. So it takes the phone -careful. I've done so much better. And then, there's a centralized
leaderboard. This actually has shown what people are playing the game. Can
you beat that high score. That's really tough.
>>:
Boy, you're in the top ten.
>> Nikolai Tillmann: Yeah, I've been playing this a lot. What's really cool
is whatever you find on the TouchDevelop, all of these scripts are free. So
you can just play these games for the fun of it. Yes?
>>: If they're all free, is there some sort of license?
license when you have what?
You have reached some
>> Nikolai Tillmann: So the question is when the script is uploaded, is there
some kind of license that says that everything is free. Indeed. So there's
the terms of use that basically says when you upload a script, you give it to
the public domain. Read the terms of use.
So we recently added a feature that allows to create rich data types using a
graphical editor and a really cool script that user set feature called my bird
log.
So when I run this script, the first thing it does is it does a [indiscernible]
first to download a list of all birds in the world and it puts them into a
dictionary. Okay. Now the app is running. I can browse all of the birds that
are known to the app, and let's say outside you just saw an eagle. Then we can
tip on the E, and select eagle. What kind of eagle? Well, I've seen bald
eagles here, but I'm not really sure if it's a bald eagle.
So the next thing the app does, I can ask for Bing pictures that match this
description of bald Eagles. So it's easy to connect to Bing, Bing Search, Bing
13
Maps. Yeah, that's exactly what I saw. So now I can log a bird sighting.
There's a plus button at the bottom. You can clear it. Simple UI is in
TouchDevelop and when I press the plus button, the phone will use GPS to take
my current location. It's just one function called in the code, get current
location, and it will store it in the table.
And now whenever I want to know again where I saw that bird, I can pop up a map
where it shows me where we are right now. So that is a pretty rich application
that has a little database in its back end, and you can create this entire
application on the phone while riding in the bus to work.
So to just show you how this database looks like, so far, we have only looked
at code, but there are also events. We added shake events in the code.
There's more. So there's data. Data is like your regular global static vowel
words and other programming languages.
One difference is that they are persistent. You don't have to deal with
tombstoning and serializing all the state. We do all of that. And it's very
easy and that's a way to maintain global state.
Now, on top of that, there's a concept of structure data types here under the
title of records, where you can configure tables in any place, which are
basically connections of records. You can also add [indiscernible] and their
[indiscernible] concept to create modular applications.
Yeah, that's another pretty cool script. Any questions about scripts that can
be created with TouchDevelop so far? Yeah?
>>: So how can you [indiscernible] logic for apps. What kind of support does
this [indiscernible] UIs. What kind of UIs can I create?
>> Nikolai Tillmann: What kind of UIs can you create? So today, it is a bit
limited. You have seen the concept of the wall, you just post stuff. And then
there's a concept of pages. You have a stack of words that show different
things. And on top of that -- at the bottom of it, you can have application
bar buttons. That is what the app allows you to do today.
We right now have an intern who is actually just walking in. He is working on
a better way to create UIs using basically as an extension of our
[indiscernible] in language. So we do not want that you have to learn XML and
14
another language. But we will have a completely integrated experience to
create even little UIs.
But we have found that for many click applications, what we have is fully
sufficient. What we also have, and they're first used in all of the games is
the concept of a bot, which is basically a canvas on which you can paint
anything. But it's a bit richer. It's actually, it has a built-in physics
engine. You can place objects, there's physical properties like speed,
friction, and that allows you to draw anything you wish. Another question?
>>: [indiscernible] do we have to all these programs from inside TouchDevelop
or can we just publish it as normal?
>> Nikolai Tillmann: Do you have to learn the program from within TouchDevelop
or can you publish them as regular apps? Excellent question. There is a
[indiscernible]. I will talk about it in a moment.
>>: The second question is about [indiscernible] consumption of these scripts.
[indiscernible] function for the scripts. What seems to be consuming?
>> Nikolai Tillmann: How about power consumption of scripts. That is the
question. So what we do is we actually interpret the scripts. So they are
running a bit slower than you're used to, which also means they consume a bit
more power.
On the other hand, we have very high level APIs that live on a very high
abstraction level. So we have a bid in physics engine. So for many task, you
don't have to add low-level code that does things in tight loops, but you can
use our high-level APIs so that power consumption is not an issue. But indeed,
in the big picture, it's part of our research agenda how we can get the power
consumption done even further.
Okay. Then back to -- so we looked at many scripts. Some of the many scripts
that people have written. There's a link on this page. So this is a fun video
of an event we did recently. So we went to UW and we had a similar event to
the [indiscernible] that we are planning to do inside of Microsoft.
And here is an experience report.
>>:
So UW stands for the University of Washington right here in Seattle.
15
[Music playing]
>> Nikolai Tillmann:
So we gave everyone Windows phones for a weekend.
>>: I had an [indiscernible] and I guess to make up for it we decided to try
to make one. [inaudible]. Now it's normal and now it's hungry. And then you
see clouds. These are well animated. Pretty impressive [inaudible].
>>: So I developed a facial recognition app using TouchDevelop. And the
purpose was to train it on a set of people that I already recognized. And I
gave it two or three images for each person. And then whenever I took an image
that I didn't know about of them later on, later onwards, or I took a Facebook
image of them, it would recognize the person in the image.
>>: I really enjoyed working in TouchDevelop. I never knew you could do so
much on a touch device without actually using an IBE on a laptop or a computer.
I was really impressed with the language and the API was really well thought
through. I really enjoyed working on it.
>>: My friend, Ian Finder, thought of this cool
because you can go look at the number on the pay
wanted to call his friend when he walked by. So
need to make this into an app. I run around and
Microsoft phone they gave me, and of each of the
around campus.
I'd to call in pay phones
phone and you call them. He
he did this, and he said we
I take a picture with the
pay phones on campus and
Then we built an app that like pegs the user's location, tells them what the
closest to them pay phone is so you can walk to the pay phone or near it in a
bush, and then call it, right. So it's like a pay phone prank.
>>: Hello? Hey. How's it going. What's your name? Yeah, you just got the
phone. Acceleration [inaudible] it's kind of like the whole choreography
thing.
>>:
I just pranked you.
[indiscernible].
>> Nikolai Tillmann: So we had tons of fun on that weekend and so did the
students. And I think it's, again, it was pretty amazing what they came up
with. Very diverse. Games, utilities, prank apps. You name it.
16
So what's really different about TouchDevelop is really the language and the
editor. So we designed a new language that actually fits a touch screen
editor. So it really needed to be strongly typed so that we can do really good
intelli-sense predictions. Otherwise, it's a traditional imperative procedural
language that's the kind of language that basically took over the world, all of
the software is written in such languages.
What's really, really cool is that pretty much every API that is on the phone
is available in TouchDevelop. So that includes, of course, all of the sensors
we already talked about, but also all of the data sources that sit on the
phone. Peli was interacting with pictures and the songs, but you can also
access the calendar, contacts, everything.
So one app that used to be our top app, I think now it's number two, is called
My Online Meetings. It was actually written by a manager in Microsoft. He's
not a full-time program [indiscernible]. He's a [indiscernible]. So what it
does, it looks at the current time, looks at your calendar, picks out current
meeting you're supposed to be in. And if it's an online meeting, then it gets
out the phone number you're supposed to call and the access code and it
combines it into a huge phone number with pauses in between and dials it. And
then you can pin that to the start screen of your phone and run it with a
single button click.
So that really shows nicely how you can combine all the different data sources
in a phone scripted together in a way that no other building app does, and you
can do it in a few lines of code. Very useful.
So I also mentioned the physics engine that makes it very
There's full support to pin apps to tiles and to actually
[indiscernible] on tiles. We also showed that you can do
full support for JSON objects and XML so you can interact
easy to add games.
change
web queries. There's
with rich web APIs.
It's very easy to share scripts. We showed you how that works. And by making
it really, really easy to share scripts, we also potentially made it easy to
share malware. Somebody could write a malicious script that takes your current
location and sends it to his own web service.
What about that? We don't really want that. So what we do is whenever you
upload your script, you can only share it with other people by going through
17
our center script buzzer. And in that script buzzer, we automatically run an
information for analysis on the script to determine what data sources go to
what data syncs. And then there are two cases. Either we classify such a flow
as safe, when basically you get a chance to review the data before it goes out.
Then everything behaves as normal. So if your script takes a picture from
Facebook, or if the script takes a picture of this camera and then shares it on
Facebook, you get a dialogue that asks do you really want to share that. You
click yes. Then you're in an excellent position to decide whether it's okay to
share or not.
But if the picture gets tainted. If it encodes your current position now or if
this dialogue is missing, then by default you would run -- if you download the
script, you would run it with anonymized information and you have to go to some
settings menu to override it to run it with your real information.
So by default, you're always safe. You can also manually inspect scripts
because they always come with source code. And everything we do is fully
automatic. There's no manual review process. Your script will not be stuck in
a waiting room for a while.
So we already showed you some of the social features. People can give hearts
to scripts, write comments, there are screen shots, and there's a notification
system that notifies you whenever something happens that is related to your
scripts. So it's really interactive.
So we were early with TouchDevelop over a year ago and then something
interesting happened. On other platform, other IDEs popped up. And so there's
AIDE for Android and Codea for iOS, and they are really cool IDEs. You should
check them out if you have such devices. But there's something fundamentally
different from TouchDevelop and those IDEs. For those IDEs, basically knee a
keyboard. Attach it with Bluetooth or however you can do it, because they
require you to enter either java code [indiscernible] or newer code. So it's
basically just using your mobile device as a PC without changing the UI.
And so with TouchDevelop, we really tried to go a step forward and see how
should the programming experience be if this is the first computer you ever
hold in your hand. We are really used to keyboards, and initially, you might
miss your keyboard a lot because you're so used to it. But after a while, our
intelli buttons become very intuitive, and we think it's pretty close to a
great experience for entering code on a touch device should be.
18
If you want to learn more, we have created a book with the help of Microsoft
Research connections. It's available to download as an eight megabyte PDF file
from our wish page. There's also a set of slides, basically one slide per API
area to learn how to use the APIs that come with TouchDevelop.
So one question was how to run these scripts. Do I need TouchDevelop to run
the script. And at first, the answer is yes. If you publish a script and you
want to share it with other people, that other person needs to have a Windows
phone and a TouchDevelop app. And then sharing is very, very quick. Just a
matter of seconds, publishing script and then it's out.
But there's also a process to sell apps and get rich. So when you publish an
app, it gets a unique ID, and let's actually look at that. So if I go to our
web page, the TouchDevelop web page, then you can browse through everything
that is happening right now. There's a book and many other materials, but you
can see what is happening right now. It's a similar view as we had on the
phone. You can look at news scripts. We also keep track of top scripts that
get run a lot. And there are featured scripts that contain novel features that
haven't been used like that before.
But here new, we see the script that we just published. So I published this
one. That is me. And when I click on the script and if you log in under that
page with the same credentials that you did on the phone, then new things are
possible, because you are the [indiscernible] script.
So we have a little system where want to make sure that before you turn the
script into an app, it's actually of high quality. So we require that three
people like it or give it a heart. Fortunately, that already happened for my
script, and unless some people take away the heart, no? And then I get is this
new button, get started. This is your own app. And then see if I can still do
it. And then a new page will pop up.
So the process was entering some metadata. And at the end of the process, I
can download this app file. So these are all basically metadata items that are
required to submit an app to the marketplace. There's some descriptions
regarding the text. It asks who am I. There's some legal disclaimers, if you
want to add a privacy statement. And you have to upload certain images.
So basically, if you want to public an app in the marketplace, you have to
19
follow the rules of the marketplace. You also have to register as a developer,
and then you can publish your app file.
So at the end of the download process, you get a zap file and you submit it to
the regular app hub and then you're free to actually charge money for it. We
also have an option integrate ads into your app so that's another potential
revenue stream.
So what we do to generate the zap file is we basically bundle the script code
you wrote with our interpreter engine. So it will behave exactly the same as
it behaved in the TouchDevelop app. Does that answer your question?
>>:
Yes.
>> Nikolai Tillmann: Okay. So that is basically everything that is possible
today in TouchDevelop and I want to talk about one really cool feature that is
coming up soon. And that is distributed state in the cloud.
So if you think of all the latest mobile applications, they typically store all
of your information in the cloud. And it doesn't matter on what device you
access your data. It's always the same. And in the background, everything
gets synchronized and magic.
But if you want to implement such an app yourself, it's actually pretty
challenging. You need to set up a server that hosts the data. You need to pay
someone for that. You need to worry about user identities, no syndication.
And then when you have conflicting data coming from different devices, you have
to think about how to merge these conflicting updates.
There's a lot of work that has to go into getting a rich, distributed state
application. We want to change that. So in one of the next updates of
TouchDevelop, what we will allow you to do is to basically take a check box
called cloud next to a global [indiscernible]. And when you do that, we know
who you are, because you have to log in, in TouchDevelop with your Windows Live
ID or Facebook account. And then we will host data for you in the cloud.
And a lot of research has been done in our group to figure out a scheme, how we
can automatically deal with structured data that gets sent to different
clients. So the ideas are bold from coat repository systems, where you can
check out data, work on it, and then later when connectivity is restored, you
20
check it in.
And we basically lifted these ideas of merging systems to such data types. So
the different devices can do it. And when they come together again, the data,
the updates get merged.
How do they get merged? Well, for every data type, we have built-in merging
functions. You don't have to define them. And they usually do the right
thing. So for integers, cloud integers have special operations. You can
adjust to read and write them, but we provide a new application called add.
And then you can accumulate changes and merging is basically error-free.
And this also extends to searcher types like records. If different clients add
to records to [indiscernible], that's no problem, no conflict. They just get
added as you would expect it.
So that's really exciting.
That's coming up in one of the future updates.
Okay. So to summarize, we were really surprised by how popular TouchDevelop
became over the course of the last year. We had many, many downloads and also
another interesting data point is that most of our users actually rate it very
highly. So that tells us that the editing experience actually works.
So if you haven't tried it yourself yet, you're still suspicious, try it. You
will be surprised how easy it actually is. TouchDevelop is the only mobile
editing environment for code that really takes the new form vector seriously.
You have a touch screen and great apps on mobile devices have to rethink the
UI. Otherwise, they are not great. TouchDevelop makes available all of the
sensors so conceptually, there is no application you couldn't write in
TouchDevelop that you cannot write in C-Sharp or in the studio.
We believe that TouchDevelop is great for also teaching and learning. So we
find that many of our users, they just find TouchDevelop on the Windows phone
marketplace, because they're trying out apps. So many people for the first
time in their life actually get exposed to programming.
There are tutorials that guides you step by step through the button clicks to
create a little program. So we see that a third of all of the scripts that are
written are by such users who probably write programs for the first time in
21
their life.
But we also think that it's great for teaching in the classroom. So if you
think about the next generation of students, when they come into a classroom,
what kind of computer they most used to? It's mobile devices. Everyone needs
a phone. And then if they get an exercise for the teacher, says no, go to the
PC and implement a sorting algorithm, then it's hard to get taken seriously.
So we believe that it's much more fun for students to actually do exercises
also on mobile devices, and we have some great experience already in high
school. It's Peli has been teaching an entire class in a high school. So it's
a great way to introduce people to programming.
And finally, if there are any interns in the room and you want to take these
ideas to your university, we have a phone loan program. So you can write us an
email and then we might make available to you a number of Windows phones for a
classroom so that you can -- or your class at the university can try it out.
That's everything we have to say, and if there are questions for the general
presentation, then you are free to ask now. Yes?
>>: So my question is about errors. So is there a chance with these scripts
to write a script that is [indiscernible], for example, or run time error? Can
you say how this is handled?
>> Nikolai Tillmann: What about errors. Excellent question. Many things come
to my mind. So in general, we follow the philosophy that is similar to most
mobile operations. When it caches you don't get a dialogue, but it basically
goes back to the start screen. So it's no elaborate exception handling, but
there's a concept of invalid values that you can test for.
So by default, you assume that if something goes horribly wrong, just let the
user start over and things will bear out better. You can test for invalid
values if you want to.
So there are, of course, potential compilation errors. We have the same
structured code editor, but it doesn't guarantee that your code is correct.
You can make certain kinds of air reports, but you can still have mismatching
parenthesis at the expression level.
22
And then you actually get shown type errors in the code. If you try out
TouchDevelop and you type something that doesn't make sense, you will see it
immediately.
We currently working on debugger support. So today when the app caches,
because there was an invalid value, for example, you take a picture and then in
of the program, you assume that there's a picture. But maybe the user actually
cancelled picture taking and the program cannot continue, right.
So today, when that happens, you get very little feedback in the phone what
actually went wrong. But we are working right now with one of our interns on a
debugger on the phone. Any other error-related answers?
>>: Is TouchDevelop done with third party APIs and silverlight or is there any
special access?
>> Nikolai Tillmann: Is TouchDevelop done with only the public Silverlight
APIs, or did we get special access? We didn't get any special treatment.
Everything in TouchDevelop, you could write in C-Sharp, which is a studio using
public knowledge.
>>: Some of these applications I've seen up, like media [indiscernible], can
those be created in TouchDevelop, or have people made applications in
TouchDevelop to make these things, like pictures?
>> Nikolai Tillmann:
What about media?
Peli wants to answer.
>> Peli de Halleux: Yes, so there's two things to this answer. We don't
bundle any kind of media. So we don't bundle any picture or video, but you can
easily create links to videos that are hosted somewhere on the web.
Now, there is an anecdote to that. A user actually wrote a sprite editor in
TouchDevelop and then you would encode the sprite as a big string and then he
wrote, of course, the encoder and a decoder for that, all in TouchDevelop. And
then he could actually now, he could write and create his little sprites, and
he started creating games that had sprite in them. And then you had this
massive piece of text in the code where you had all these images encoded.
So you had a lot of fun, I guess, during that.
23
>> Nikolai Tillmann:
Any more questions?
>>: Didn't I see in the library that one of the things you could put in there
was media as part of the program?
>> Nikolai Tillmann:
media in the screen.
So the media -- so the question is whether there was a
But I think it means out.
>>: So you had the code and then you had events and then down at the bottom
there was an app.
>> Nikolai Tillmann: Exactly. So that is basically a mechanism where you can
supply links to sounds and pictures and you get a nice preview of what it
actually stands for and then you can use them in code just like built-in songs
and pictures. But there will actually fetched at run time through those links.
So if you distribute this program, any other end users also will download the
pictures from the iteming source.
>>:
Is the run time check, are they still there?
>> Nikolai Tillmann: Yes. Otherwise, it's going to be an invalid value.
either you check for it or the program will terminate.
And
>>: When it showed the [indiscernible] that you support, I only saw the high
level ones, like shake, something like that. Do you have some of the low-level
ones like [indiscernible] on the screen or something like that. Or dutch move.
>> Peli de Halleux: So the question is whether we have more targeted event
like touch, tap, swipe, drag on particular elements. So we have that for the
2D game board. So the game engine. You will, as soon as you declare it, you
would actually see new events that allows you to register to tap, swipe, drag
on the board. But also in every individual sprite or sprite set.
But we don't give you tap or drag on wall elements, for example.
>>: And one other question. I really like the idea of the app. It is
amazing. But is it possible to take a step back and use it on the PC? Like in
emulate if you don't have Windows phone on you at the moment?
>> Nikolai Tillmann:
So you cannot use it on your PC.
And we do not want that
24
people use it in an emulator, because it really takes away the whole point.
You get immediate feedback when you use the accelerometer. And all of our
buttons on the PC, that would be huge.
And you have this mouse, it's a super-fine selection ability. It doesn't make
sense. So you really want to different experience on the PC than on a phone.
And we didn't want to weaken our philosophy.
Actually, it's a great exercise for us to constrain ourselves to this phone
factor. If you only have that much space, you really have to seriously rethink
what you're doing.
>>: Some additional question too. But application has reached some limitation
of the platform. It will be really cool if you can get some C-Sharp or code
from [indiscernible] developer and add some [indiscernible] which is not
[indiscernible].
>> Nikolai Tillmann: So the question is, what if you reach a limitation and
you want to continue outside of TouchDevelop, how can we get the code. So what
is available today is you can actually look at the code on the website in the
TouchDevelop language. Actually, not exactly how it shows up on the phone, but
here we -- this is our internal presentation with [indiscernible].
And there's currently a masters student working on a compiler to C-Sharp, and
then you could use that. But we don't provide any built-in functionality to
turn it into a C-Sharp app.
It's actually pretty tricky, because there are two ways you could do it.
There's a naive way. But in the naive way, I talked about tombstoning. And we
serialize all of the data and take care of tombstoning. But that wouldn't
quite work if you translate in an [indiscernible] to C-Sharp.
And if you do it in a way that is bulletproof than the C-Sharp would look
horrible. There's tons of continuation everywhere. So tricky subject.
>> Peli de Halleux: For example, if you look at the sample here, we take a
picture. We take a camera picture. In reality, the application is terminated.
We go into the photo application, takes a photo, it comes back and drops a
photo some well-known place and then the developer in C-Sharp has to pick it up
and then continue with that.
25
So we made it a single call, but in reality, it's much more tricky than that
when you're implementing it in C-Sharp. So the code gen would be a bit more
ugly.
>>: Is it possible to have these scripts like running in the background, like
calling somebody.
>> Nikolai Tillmann: Is it possible to run scripts in the background? Today,
that is not possible because we haven't implemented that part yet. If you want
to help us, we are the research team. Since you're working for Microsoft, you
are happy to join us to implement all of those little missing pieces.
But there's no inherent reason why you shouldn't be able to run things in the
background.
>>: Is it question, like if the events or, you know, the extension points that
they provide are not enough, can we code some additional extension for it and
integrate them? Is it extensible at the moment?
>> Nikolai Tillmann: Is it extensible. Can you add integration points. So
what we do have is a library concept. So if you, let's say, talk to a web
service and you want to reuse a piece of code in different scripts, than you
can wrap that into a library and reference it again and again from different
scripts.
What we currently do not allow is that you kind of mix and match C-Sharp and
TouchDevelop code. It's an interesting idea. We haven't really thought about
it yet.
>> Peli de Halleux: So in the phone, we are a third party application and we
are bound to all the rules and we can't just drop a DLL and start using it. It
has to be in the ->> Nikolai Tillmann: That is true. It would have to be then a standalone app
and no longer purely phone development.
>>: You could implement it as a co-app to web service, right?
implement your library as a web service somewhere and --
So you could
26
>> Nikolai Tillmann: So the comment is that you can always implement extra
functionality as a web service. Indeed. So an example, one of the students,
there's a face recognition, he actually, the face recognition, he didn't
implement it on the phone. He was using an existing service site
implementation.
>> Peli de Halleux: And we do support HTP requests and JSON parsing and XML
parsing. So you can definitely intract with all the rich web APIs out there.
>>: Is it possible to, like, stream data from the internet?
you were making a Pandora application or music?
>> Nikolai Tillmann: Is it possible to stream data?
have built-in support for --
[inaudible] like
To some extent, yes.
We
>> Peli de Halleux: You can definitely, I mean, you can open a video stream
and then the native phone player will kick in and play that.
>> Nikolai Tillmann:
>> Peli de Halleux:
Play a song.
We don't support that in the API.
>> Nikolai Tillmann: Okay. Then thank you very much for staying here.
is the conclusion of the general presentation.
This