>> Nikolai Tillmann: Hello. I'm Nikolai Tillmann, and with Microsoft
Research. It's my pleasure today to announce Arno Puder
from San Francisco State University. Have been working with Arno over
the past month on a project called Hatch Develop.
And mobile development environment that runs on many platforms. And
Arno is an expert in cost compilation and dealing with many different
mobile platforms. So I'm excited to hear more about today what Arno
has been doing now.
>> Arno Puder: Thanks, Nickolai, for the introduction and thanks for
inviting me to give a talk here. So the title of the talk is
WebAppBooster, giving access to your smartphone.
This project was a result of a collaboration with Microsoft Research as
Nickolai mentioned, and I want to give you a bit about the motivation
on why we did it and also some technical details on what WebAppBooster
is all about. So in terms of agenda I will talk about using HTML 5 for
apps it's an exciting time if you look at the space. I will talk about
PhoneGap, the pioneer of using HTML 5 for mobile app development, and
I'll go into some more details on WebAppBooster. And also I'm going to
be focusing on security. Because that obviously is a very important
topic.
Last month there was a study, it was released talking about HTML 5 and
mobile development. And if you kind of look back on the history of
using HTML 5 for any kind of development, we had Web 1.0 basically
using static HTML pages. I think most of us have used an editor to
write your own little home page.
Then we had Web 2.0 I think Google really opened people's eyes by doing
something like Google Maps that really showed people the potential of
what it can do inside browser, not just merely rendering static content
like in Web 1.0 but having a highly interactive application. And now
lately there's something I call Web 3.0 so HTML 5 going mobile.
So you see actually more and more platforms that use HTML 5 as the
first class citizen for writing mobile applications. So there was kind
of like an early example that unfortunately didn't quite take off Web
OS from perm and now HP.
There's Chrome OS from Google. Mozilla is right now extremely active
in this space. You may have heard of Firefox OS. So basically
bringing the gecko rendering engine of Firefox to a mobile device so
the entire experience of Firefox OS is in effect based on HTML 5. So
starting from the home screen and they even go down to low level system
applications that are partially written in HTML 5.
Tizen is another example of collaboration between Samsung and Intel
bringing HTML 5 to mobile platforms and last but not least Windows
Phone 8 is also -- Windows 8 have elected JavaScript and HTML as a
first-class citizen.
So if you look at the space with regards to who is developing and in
what language, what platform for mobile devices, there was an
interesting market study that was released in January of this year.
And the link can see down here develop economics like a market research
company. And I would highly recommend you take a look at the study
because it has some very interesting points.
I'm pulling out two slides from the study. And the first slide here
talks about the distribution of developers and for which platform they
develop and the change within half a year.
First of all, you can see that the majority of the developers do target
androids. And it has grown from 68 to 72 percent of developers that
were being questioned as part of the study that go for Android.
The next not much big surprise is IOS. Although they have seen a
little bit of a decline, the study cites some changes, some shifts in
Asia. Asia is heavily biased towards Android. So that accounts for
the slight drop on developers for IOS.
Now, summer of last year, middle of last year, the developers were not
yet asked about HTML 5. So there is no comparison how this looked like
back in the middle of 2012. But still I find it very interesting that
50 percent of developers said yes they also target, they also use HTML
5 for developing mobile apps. And it would be interesting to see how
then in subsequent months this changes.
Well, the other platforms you can see is also less popular, the next
one is Windows Phone, and followed by some other platforms. I wanted
to highlight that apart from the big elephants in the room, the IOS and
Android, HTML 5 has also become a premier technology for developing
mobile applications.
Now, the second slide I wanted to cite from this market study deals
with some thoughts about using HTML 5 for mobile applications. And,
first of all, the developers were asked the reasons for selecting HTML
5. And the number one reason for using HTML is cross-platform
portability. It's pretty well known that mobile platforms become these
walled gardens where also the owner of that walled garden have strong
choice on what technology should be used for this mobile platform.
And it's difficult to use anything but the first technology. So Java
is for Android. You have objective C for IOS. Windows Phone 8, C# and
also HTML 5, but if you have a nice idea for an application, you as a
developer, you want that to be present in as many platforms as
possible.
revenue.
And as many marketplaces as possible to maximize your
So if you have to rewrite your application from scratch, that's a big
amount of work.
So HTML is kind of like this unifying factor. So developers see HTML 5
as a technology that enables them to more quickly port their
application to different platforms. Now, at the same time, developers
were also asked what are the biggest down sides or omissions of HTML 5.
And the number one point that was mentioned is access to the latest
native API. So you have HTML 5 as an execution platform. But one
thing that is really missing is accessing some device-specific things.
So smartphones are known for special hardware devices like
accelerometer, magnetometers on one side but they also have some
interesting data, so think of music library or your photo gallery that
you might want to access from the application.
So if you run inside a Web browser you're sandboxed you do not have
access to the API. So that's kind of like the landscape as it is of
today.
Now I want to give you some examples of some of the advanced Web API
currently developing under the umbrella of w3C and it's amazing to see
how the Web is developing to cater to these needs.
So there are a slew of new API currently being standardized by w3C and
I'm sure you've seen or heard of many of those APIs. Like for multi
touch you have API by now. WebSockets being directional communication
and Web workers allowing parallelism in Web applications, Web RTC read
communication and even a new standard for accessing near [inaudible]
communication from Web applications.
Now I wanted to give you two examples because these are like
cutting-edge APIs that are still in the process of being standardized.
And the first one is called Web intends or Web activities.
So the intent activity paradigm has been introduced by Android. And
it's basically a late binding mechanism where one application can make
use of another application's functionality. So a nice example is if
you want to pick a contact from your contact list. In Android, you
send an intent that will be answered by an activity inside the Android
platform.
So it enables more code reuse, and makes this late binding mechanism
easier. So now there's something called Web activities. And Mozilla
right now is pushing this very heavily for their Firefox OS platform.
I have a code snippet at the bottom so you can see how it looks like.
So there's a new browser object called Moz activity you can instantiate
it give it a name and some additional data like mine types. So
additional the code snippet here I have on this slide, what you want to
do is you want to pick images. And you specify what interested in PNG
and JPEG images and you can fire this Moz activity.
Now let's just assume that at least on Firefox OS, if you have
different application installed that manage images, for example, the
wallpaper application or the gallery application, what happens then is
the same thing that happens in Android. You're being shown what is
called an activity chooser. So that's the screen shot here on the
right. Essentially these are all the applications that are able to
service or that are able to provide you with images.
So then the user says, well, I want to pick, for example, image from
the gallery, and you click on okay. And then basically that
application will be launched. You can pick an image that will be
returned to the application. Same here late binding mechanism but now
as part of the HTML 5 platform. The second example I wanted to give
are WebSockets. So for the longest time Web developers were limited to
HTP as a communication protocol. If you want to do something like push
notifications from server down to a Web application, you have to resort
to all kinds of tricks. For example, deferred reply.
So you deliberately did not send an htp response, and only when you had
an event on the server side you would piggyback it on this deferred
reply to the browser. So eventually got around to doing something like
WebSockets. And WebSockets is a new standard that allows bidirectional
between the server and browser.
The interesting thing is they managed to do so in a compatible way with
current Web standards. So a WebSocket in terms of protocol starts out
as http protocol that gets upgraded. Once you've upgraded the
communication link then you can actually also send data down to the
browser. And the way this looks like, again given some little code
snippet here. So there's another browser object called WebSocket. And
there's a new URI scheme called WS. You can give it a domain name and
a port number here.
And then you can bind different call-back messages, call-back functions
to this object. So, for example, the on message call back will then be
invoked inside your browser, whenever the serve out is pushing some
data down to your browser.
Likewise, you can send data from the browser to the server by just
using the sent API here. So it's a very nice way of pushing data back
and forth between server and browser.
So next I wanted to talk about the kind of like the grandfather of Web
development for mobile devices, a project called PhoneGap. An open
source project. They were acquired by adobe and actually they turned
it over to the Apache Foundation. And PhoneGap is now, I believe,
still at incubator status with the Apache foundation and they also
renamed it. Gave it a different name. It's called C ordova, but I
hardly see the name mentioned and everyone still refers it to it as
PhoneGap. I'll do the same here.
What PhoneGap essentially did is a very clever idea. PhoneGap creates
a native app for the respective platform, but what they do is they
instantiate a Web view widget in full screen mode. And the HTML 5
application that was developed by a mobile app developer is piggybacked
or is bundled as data into this native app. And at runtime, what
PhoneGap would do, it would inject the HTML 5 in JavaScript into this
Web view that is part of the full application. What you had in fact is
a native app hosted in an app store like the IOS Web store or Google's
play store, and you download it like a native app but the moment you
run the application what it does it ininstantiates the full screen
browser window and then injects the HTML in JavaScript.
And they also give you a lot of CSS styles to really mimic the native
look and feel. So they actually went down to mimic, for example, the
IOS fancy widgets that IOS is known for. But it's all HTML 5 and HTML
and CSS. So some more details on at least how PhoneGap works on
Android. So there are two ways of communication. How we can break out
of the sandbox, because you still are running inside a browser widget.
So if you are JavaScript running inside this browser widget and you
want to make a call to the native layer. Let's say you wanted to
access the camera API, then what Android allows you to do is on the
Java side, there is something called a Web view client. And you can
override a method that's called should overwrite URL loading. And
basically the HTML 5 application has a certain URL scheme that it then
uses to make a request, for example, to take a picture and you can
intercept that on the Java layer, inside this boilerplate code that
comes with PhoneGap. And this boilerplate code simply forwards that
unmarshals the parameters and then forwards that to the native API. So
that's how we can make a down call so from JavaScript down to the
native layer.
Now, an up call is even simpler. The Web view has something called a
method called load URL and essentially you can just inline your
JavaScript code that is being executed in the context of your
application.
And if you have, for example, events like accelerometer or GPS you can
then upload this to or you can return that back to the application
running in your Web view. And for different platforms like IOS that do
this, it does the same thing, same trick. They always instantiate a
widget that implements a native browser. And then they inject the
JavaScript.
And PhoneGap does support many different platforms and it's a widely
used project. Now, Nickolai mentioned that we have been doing this
project in the context of TouchDevelop. So TouchDevelop, there is an
HTML 5 version of TouchDevelop. And the first thing that we try to do
is we try to use PhoneGap by taking all the JavaScript in HTML 5 of
TouchDevelop and bundling it with PhoneGap. Again, as a way of turning
it into a native app that can be uploaded, for example, to the Google
play store.
However, we ran into a whole bunch of problems, when we tried that
route. And, first of all, surprisingly there are quite a few widgets
that we're missing. So many things that TouchDevelop wanted to do,
we're not able. At least PhoneGap did not immediately provide any
solutions.
Now, PhoneGap does have a nice plug-in architecture. It's just a
matter of implementing additional plug-ins. But the bigger problem
that we encountered is that PhoneGap uses the browser widget that comes
with Android, which is all good and fine. However, this browser widget
is not Chrome. It's not the Chrome browser. It's based on different
technology. And you can tell that it's not maintained as vigorously as
the Chrome browser. In particular, we found that TouchDevelop
applications run extremely laggy using the PhoneGap approach. So at
the same time, we noticed that Chrome as an external browser actually
had much better performance in terms of executing PhoneGap. So the
idea was why don't we simply use Chrome as the rendering engine for
TouchDevelop. Well, that's all good and fine, except that Chrome is
not a widget. Chrome is an external application.
So the next question is, okay, then how can we then still run
TouchDevelop in a way that gives access to a native API? And while
that was basically then the birth place or the birth time of this
project that we have called WebAppBooster. A bit of a geeky name but I
guess it captures the essence of what it tries to do.
So WebAppBooster is essentially a free Android app. It's already
available in the Google play store. And what it does, it acts as a
gateway for Web applications that run inside of Chrome to capabilities
and functionality that you have on your device.
So, for example, if you want to access contacts, you want to access the
gallery or song library, so PhoneGap gives you access to those things
inside your smartphone.
So it makes use of WebSockets for bidirectional communication, and it
also has a plug-in architecture that makes it easy to add additional
components.
So on the next slide I have an architectural overview on how what app
booster looks like. So basically so WebAppBooster we're talking about
Android here. WebAppBooster is a native app that runs on Android. And
here you see another native app that is Chrome, also running on
Android. These two applications run on the same device. And what
happens now is Web app booster when you run it it's a background
server. Once you activate it runs in the background and there are two
different servers that run there. One is the WebSocket server that's
used for bidirectional communication. That is how, for example, we
push sensor data. So if you have a continuous stream of sensor data
like accelerometer or magnetometer, that's how we push that to the Web
application.
We also have an embedded HTP server. And the reason for that is we
used HTP server for returning more volume data. So, for example, if
you want to return an image over to your application, we could have
done this also using the WebSocket server here by doing data URL
encoding. But we found that it was too inefficient. So it's much more
efficient if you actually return this data via embedded HTP server.
The idea is when you have Chrome, visit a page like touchdevelop.com,
which is like a Web app. First in step one, you download all the HTML
and JavaScript belongs that the TouchDevelop application. Then there's
some JavaScript code that's similar to the code we had earlier that
opens a WebSocket to the WebSocket server that is owned by
TouchDevelop. And then there's a protocol that the Web app can talk to
Web booster in order to accomplish certain things.
Now, obviously security is of big concern here, because if anyone could
just simply connect to this WebSocket server and start pulling down
your contact information, it would not be a good idea.
And we have taken a lot of steps to ensure this doesn't happen and
we'll talk through that in a second.
Question?
Yes.
>>: Have you tried that approach in other platforms? I know that, for
example, with Windows Phone not be able to expose a server or WebSocket
server.
>> Nikolai Tillmann: The question is if we had explored the same
approach for different platforms. And the questions are often for IOS
or Windows Phone 8. And well, yes, we have explored it. However, it
is not possible.
And for legal reasons. Not for technological reasons. So IOS, for
example, does allow background processes. So, again, WebAppBooster is
a background server, it runs in the background and IOS does allow
background services however only for very limited kinds of
applications.
So, for example, for voice over IP applications or for media
applications that play music in the background.
So something like WebAppBooster would not pass Apple's review process,
and Windows Phone 8 I believe it's similar. But you guys probably know
better.
So I don't have a live demo, but I do have a series of screen shots
that give you kind of a flavor on what it looks like using
WebAppBooster on Android. And, well, if you download WebAppBooster
from Google Play Store you have to explicitly enable it. It's an opt
in thing, also to be very clear that you know what you're getting
there.
And initially so that's the overview screen of WebAppBooster.
Initially there's no application that uses it. So there's no
connection. So next what -- so from this point on, WebAppBooster runs
in the background. So then you can switch to Chrome and you can visit
a Web application that makes use of WebAppBooster.
So I wrote a little demo. So there's also domain WebAppBooster/Demo.
It gives you a nice way to test it see how it works, a Web interface
where you can simply select what kind of action you want to perform
like accelerometer or pick a contact. So the very first thing that Web
application has to do that wants to use WebAppBooster is to request
permissions.
Again, you don't want to have uncontrolled access to all kinds of
sensitive data. But you want the user to be aware of what is
happening. Also the user has to give explicit permission. So this
little demo application you can have a button here that says request
permissions.
And then what happens here in the next step, if you click on this
button. So there's a little dialogue that pops open where the user is
being prompted. Okay. So there's a website that's called
WebAppBooster.org and it requests these kind of permissions here. So
access to all kinds of different data. Do you allow this once, do you
always allow it or do you reject it. So the user has to make a choice.
Now I do want to point out one technical detail at this stage here.
And this dialogue, modal dialogue that you see here, this is not part
activity Chrome application.
What happens here is the Web application sends a request for these
permissions to the background service that is implemented by
WebAppBooster. And then WebAppBooster opens the modal dialogue but
it's able to do so overlaying this modal dialogue over the Chrome
browser.
If you look carefully you can still see this darkened background so
actually this modal dialogue here is overlaid over the Chrome browser.
But it was not open by Chrome. It was opened by this background
service.
You may say well this is also a kind of an interesting feature of
Android that I duck out in terms of API feature, background service is
able to overlay something over the current application. But it is
possible using official Android API. It's not a hack. If someone is
interested I can explain how this works. But just to be clear, there's
no way to tamper with this. It's not like part of some JavaScript
where you might do things differently. But security is guaranteed by
this background service so there's no way how the Web app can
manipulate this dialogue here. At least if you trust WebAppBooster as
an application.
So let's assume that the user gave permission. So you say, okay, I
allow this application that runs on this domain to use these
permissions and then you can start using the API.
Before I do that, if you switch -- if at this point you switch back to
WebAppBooster you can see there's one active connection. So you can
also see the list of domains that have currently open connections to
WebAppBooster. You can then tap on one of those domains, and you can
see what kind of permissions have been requested by the user, by this
Web app and you can also revoke those permissions if you choose to do
so.
But I guess the more interesting thing is let's go back to Chrome,
because now we have permission to use a different API. So what I'm
doing now is the Web app now requests or says, okay, let me pick a
contact from the user's contact list. And the same thing. So the
request is being routed near the WebSocket to the background service,
and similarly what I mentioned before, with this dialogue that pops
over the Chrome, the same way now I create an intent for picking a
contact but I do it in such a way that it overlays the Chrome browser.
So what you see again here, you can see just barely in the background
you can still see the Chrome browser, but then the white window you see
over here, that is the activity in Android, the standard dialogue in
Android that lets you pick a contact.
So my demo device I have two contacts here. And then if I pick one of
those contacts, then the background service sends a little bit of JSON
back to the Web application and what the Web app simply does it simply
prints, logs out this JSON code here.
It gives you an idea on how WebAppBooster works, and we already do
support a bunch of APIs. So contacts is one. We do support several
sensors. So accelerometer, gyroscope are supported.
We support media. So you can browse all the MP3s stored on your
device. You can play media, and we also -- there was one more.
Gallery. So we also allow you to browse the gallery of -- and also
taking a picture is also supported by WebAppBooster in the current
version. Okay. I'm going to switch back and show you more internal
details on WebAppBooster. So here is essentially if you wanted to make
use of WebAppBooster, here is how you would do it. So you would just
simply create a WebSocket and you connect to a WebSocket that runs on
local host, on port 8042. And if you don't get a connection to this
port, that basically means that WebAppBooster either is not installed
or has been deactivated. In that case if you wanted to, you could pop
open a little dialogue you say well to enhance your experience you
might want to download WebAppBooster and enable it.
But if this connect succeeds, that means that you are running
WebAppBooster on the device, and you can make use of enhanced
capabilities.
Now, you have to do through an authentication authorization process,
I'm not showing on this particular slide here, but once you have
authenticated and authorized the connection, you can start making use
of additional features, and what I do here is I show how you would
start the gyroscope, for example. And so there's a bit of JSON, and
you just have an action that's called start gyro. You have to give
every request a unique ID. You simply use the WebSocket to send this
JSON over to WebAppBooster. And from then on you will get a continuous
stream of gyro data as the building sensor has updates.
As I mentioned before there's this on message callback that you can
hook into. And if you see the same message ID for the response of 42
here that I use also for the request, then you know that you've
received back some gyroscope data.
So some more detailed -- so here if you wanted to extend a new plug-in,
so I mentioned before that WebAppBooster does have a plug-in
architecture so as a base class called plug-in on the Android source
code. And each plug-in has a lifecycle. So when the first Web
application connects and uses this plug-in, there's an on create. So
you can see for each origin that is being used so plug-in essentially
is intended for each origin. And also once you disconnect, once you
close the tap in Chrome then also you go through a lifecycle on
destroy, if you wanted to make some data persistent, for example.
But the main function here, the main callback is execute. And that
basically whenever you send some JSON like the start gyro I had
earlier, then WebAppBooster would be calling this execute here, and you
get the request ID, you get the action and you get the original JSON
object. So if there's additional parameters you need to access.
You can register on new plug-ins. There's a kind of a like a manifest
WebAppBooster, manifest plug-in XML, a bit of XML that lists all the
different plug-ins that are available, and I use Java reflection
actually at runtime to instantiate the plug-ins and register them.
You can see the kind of actions that each plug-in responds to. But
also for certain plug-ins you can see there's some permissions here.
So the plug-in gyroscope plug-in requests permission called gyro. So
before Web app can actually use that, it first has to request this
particular permission.
So I mentioned that we did this project as part of the TouchDevelop
project. So if you are TouchDevelop user and if you wanted to actually
give it a try on Android. So what you can do is you can on your
Android device first from Google Play Store you have to download and
install WebAppBooster. It's a free app again and once you've
downloaded it, you can then switch back to TouchDevelop. And here's
like a very small script that gives you an idea on how this works.
So essentially what you see in this little box down here, you see a
TouchDevelop script that if you run it you can see it, it uses social
API choose contact. And actually this little TouchDevelop script would
then pop open the same contact chooser dialogue that I had earlier in
my screen shot.
So I wanted to spend some time to talk about security. Because that is
now the big thing that I mentioned is important. But I really haven't
talked about it at this point.
And I wanted to explain what we have done in kind of in terms of
security. So, first of all, there's authorization, there's
authentication. And authorization, you have seen the permissions
dialogue. So before a Web app can use certain API, it has to request
permission to do so. So that's the same model that Android does so
that at install time, there's a dialogue that pops open and the user
has to say yes or no if you want, if you want to give that Web app
these kind of permissions.
So that's all good and fine. The more difficult part is the
authentication. Like how do we actually trust if there's a dialogue
box that pops open and says well there's a Web page XYZ.com and it
requests these permissions, how can you trust the origin, how can you
trust it actually, domain that's mentioned. Basically the screen shot
I had here, so it's a WebAppBooster will take the origin of the URL,
that's also given to you as part of the WebSocket protocol and it says
there's a Web page, Web application called WebAppBooster.com. .org and
do you give this application the permission. Now, if you know a little
bit about Web protocols, you know that you can easily fake these bits
of information.
I mean you have a WebSocket server that runs on your device, and many
bad things that can happen here. First of all, since you're running a
WebSocket server, what could happen is that someone like drive by
malware that someone from the outside that's not even running on your
device can access, connect to your WebSocket server.
The next thing is that if you have something that is not even the
Chrome browser, let's say malware is on your device, but it's just
using plain socket API. You can actually spoof the course head, cross
origin resource headers, and you can pretend that you're a trusted
domain. If you trust TouchDevelop we just simply fake this
information.
So that is what we have to guard against and make sure that just going
back one slide here so that people can actually at least trust this
information about the origin of the domain.
So one thing that we have done is at least to limit the scope of
security. We only allow connections, the WebSocket server only allows
connections from local hosts. So that already takes care of drive by
malware so nobody outside of your device in the same network can
connect to your socket server.
And also one limitation we have right now is we only allow one
connection per origin. So if someone already like if TouchDevelop
already has done a connection, then you cannot piggyback by saying,
okay, I'm also claiming to be TouchDevelop.
But an unsolved problem again is the faking of course headers, and I
want to go through the protocol that we've implemented to explain how
we take care of this particular problem.
So this whole protocol essentially kicks off by a connection that we
receive inside of WebAppBooster, and then there's a connection and
someone claims to be touchdevelop.com. And, again, this could be a
legitimate request coming originating from Chrome, and we can trust
Chrome not to do something bad with the course headers. But this could
potentially be some malware that just takes advantage of a trusted
domain.
So at this point we have an incoming request, and we see, well, someone
who claims to be touchdevelop.com but we cannot trust this piece of
information at this point in time.
So what WebAppBooster does it tears down the connection. So it closes
the connection. It creates a unique token for this new origin. And it
then gives like the main activity of TouchDevelop. Now, what happens
of WebAppBooster. But then what happens in the next step, the
background service uses the same intense system that I showed earlier
for picking a contact.
Web page.
It uses the same intense system for opening a
And the rationale here is we trust the intense system. It's built into
Android that will actually be answered by the Chrome browser and what
you can see in this intent for opening a Web page, what we do here is
we have the original -- the URL. We say okay someone who claims to be
TouchDevelop, let's go to this Web page, and we also append the token
that we created in the previous step.
If you know a little more about Android development, what you can do,
if you have multiple applications on your Android device that are able
to open Web pages, so let's just say you download Opera or Firefox for
Android, and you have potentially multiple browsers that can respond to
this intent, what happens here is the same thing I mentioned earlier
for the Moz activity. You have something called an activity chooser
where the user has to then say I want to use this application to open
this URL.
So if you wanted to intercept this. If you were malware, what you can
do is you can say, okay, malware, and I'm also able to open a Web page,
and then but the user will have to click on a button here.
So and for the built-in applications like Chrome and then also
additional third-party applications like Firefox, people trust that.
So, again, like this step here, WebAppBooster opening this Web page
relies on the trust you have in certain applications. And if there's a
suspicious application that all of a sudden also claims to be able to
open this Web page, at least the thinking is that people will not click
on that but it will rather click on the Chrome browser.
But then the next step what happens is so Chrome then goes to
touchdevelop.com and it downloads for a second time this Web app but
now there's this token in the URL, now a second time TouchDevelop runs
inside Chrome. It can extract the token. It can open up the WebSocket
connection a second time. By the second time it will present this
token that was created in step number one.
So which means that at this point in time, we actually know that this
connection has to come from a Web app that originated from legitimate
source named touchdevelop.com. So at this stage the connection is
authenticated and from then on we allow more communication like
starting gyroscope and picking contacts.
So any questions regarding security?
Ben?
>>: I think it makes sense. I don't know. We talked about this
offline so there's actually a paper about UI spoofing on Android.
I
wondered if they have a more principled solution that people can use
because this seems like ->> Nikolai Tillmann:
Well WebAppBooster does --
>>: [inaudible].
>> Nikolai Tillmann: I do agree. I think WebAppBooster does not add
more problems here than what we already have on the spoofing side. I
think the fact that ->>: Open platform, to say that they suffer from the [inaudible] they
suffer from the need to support this, the platform just ignore this
issue.
>> Nikolai Tillmann: Yeah. Well, I guess the fact that you can
overlay a dialogue from a background service, I wonder, I'm surprised
that no one picked up on that yet in the security community.
>>: There is a commission on that -- not [inaudible].
>>: [inaudible].
>> Nikolai Tillmann: Okay. Then let me come to my conclusions here.
So in terms of status quo. So we do have a stable version of
WebAppBooster that is currently hosted in the Android market. We
already have several plug-ins that we support and we plan to do some
more. There's a home page. So you can go to WebAppBooster.org and it
also gives you some more detail on how it works. And also if you want
to use WebAppBooster, if you wanted to integrate it and it also has
some code snippets that explains how to do that.
As the next step, we want to think a bit more in depth on the security
model and there's actually an interesting feature of or not so much a
feature but requirement of TouchDevelop in particular, because
TouchDevelop is a container for running small apps but different apps
potentially. So the question is here, okay, how do we request security
permissions? Is it kind of like request all possible permissions or
only on whatever the current app you're running, and what is the scope
of a permission like when do we revoke permissions.
So especially for TouchDevelop, we need to think a little bit more in
depth on requesting permissions.
Right now I only do it on a per domain basis. But I think we may need
to make it a little more fine grained to better support the needs of
TouchDevelop.
Well, as kind of an outlook, and this might be a bit way off, I guess
we have to think a little bit more in depth about it. But I mentioned
earlier that the WebSocket server right now only allows connections
from originating from the device itself for security reasons. But what
if in some constraints we were to allow connections from different
devices? So you could actually build up some kind of sensor network.
Imagine that you could use your phone or let's say your platform to
access someone else's sensor. A reason might be maybe your own phone
doesn't have the sensor. Let's say if you have a special purpose
sensor you only have on a different device you might want to access
that.
Or you might want to actually correlate data coming from different
sensors, like a mini sensor network. This might be, could be used for
some distributed gaming here, for example.
So there's a little thing what I have down here. Basically you have
one phone here that has multiple connections to phones nearby or maybe
in different locations. But do something like remote sensing or
distributed gaming. So just some ideas on how to push maybe
WebAppBooster beyond what we use it for right now.
Well, with that I come to my last slide here. So there's a screen shot
of the home page. Again, I have put up some more information on what
it is and how to use it. So you're welcome to contribute in different
ways.
Just maybe to use it in your own Web application or contribute some
more plug-ins if you're so inclined. But with that, I would like to
conclude and thank you for your attention.
>>: Let's thank Arno.
[applause]
>> Nikolai Tillmann:
Any questions?
Anything else?
>>: Does it have an impact on battery or consumption?
>> Nikolai Tillmann: Impact on battery. Android is known to pull down
the battery because of all the background services. I mean, it's a
nice feature. You know, it's a nice flexible system. Actually, they
have -- I talked much about activities. But Android also likewise has
this notion of a service. So it's deliberate, something running in the
background. There's no constraints. And many applications make use of
that. WebAppBooster, it will drain battery only if you use it. So if
it is just passive in the background you'll not use any battery. Only
if someone opens a WebSocket connection. But then, of course, it will
add additional drain on the battery. But I think it's fairly limited.
Let's say if you want to pick a contact, that's a one-time dialogue.
If you were to let's say connect to a sensor and you had a continuous
stream of sensor data, that probably would have a higher hit on the
battery.
>>: Am I able to tweak the sensing rate?
>> Nikolai Tillmann: Yes. In Android when you register for a sensor,
you can say what sampling rate you want. If you're only interested in
orientation changes you don't need 100 hertz. You just need 410 hertz.
>>: Is there an adjustment for Windows 8 log. I think Windows 8 has a
file where you can get sensors and it's [inaudible].
>>: No, unless you write it as ->>: Metro app sure can get this.
>>: Metro app is a store.
>>: [indiscernible] on five can get that table five.
>>: [inaudible].
>>: [indiscernible].
>>: Is there something similar to that in Android?
>> Nikolai Tillmann:
No.
Not in Android.
>>: You would write against mean RTs and across platforms?
>>: It's not Web platform, [indiscernible] but it is.
around the limitations of Android platform.
We are getting
>>: And you're a native app.
>> Nikolai Tillmann: Essentially recorded. So there was a question
about the same thing for Windows Phone 8 and it was a discussion
amongst the Microsoft people here, which you guys are -- better experts
on that topic.
>>: I'm not [inaudible] Windows 8.
>>: Different question. So you've been talking about the platinum
model, spans the idea I have to compile the plug-ins to my better
booster that runs on the device. Sort of -- technically I can send
Java code over the wire to have the managed plug-in so technically
possible. Any way to get in security?
>> Nikolai Tillmann: The question is I talked
And right now you have to create a new version
someone were to contribute a new plug-in, then
recompile WebAppBooster and resubmit it to the
about the plug-in model.
of WebAppBooster. So if
I have to actually
Google Play Store and
the question is there some way to do it a different way? Yes, there
is. Android has a virtual machine called Dulvic has the same
capabilities as the Sun hot spot virtual machine. Mainly they have a
class loader, which means if you compile a plug-in or any Java code to
Dulvic's decks code, you can download it from somewhere and just use
the regular class loader on Android to then execute this code.
There was a time, I believe, where the Facebook application did that.
People at some point noticed that Facebook updated itself without you
actually having to go through the Google Play Store. I'm not quite
sure, but someone told me that maybe a couple of years ago that that
was the case. And they used this little technique.
>>: The idea in that sense so you could maintain your own little
plug-in store that you or someone else, you certify submissions, and so
instead of the Web page submitting random Java code it could come from
a semi tested ->> Nikolai Tillmann: I could open my little plug-in store, but you
know the thing is this has a different quality. I mean, we're talking
about like some system plug-ins, like very low level stuff. It's not
like an end user plug-in. So I think it does make sense to republish
the application in the Google Play Store. It has a different quality.
There's no icon for a plug-in. Not like -- it's a very different thing
than the regular app store.
>>: [inaudible] once you start covering them [inaudible].
>>: Yeah.
>> Nikolai Tillmann:
Other questions?
>> Arno is visiting us today and tomorrow; if you want to talk to him
we can schedule an appointment for tomorrow.
>> Nikolai Tillmann:
[applause]
Thanks, guys.