>> Ranveer Chandra: So it is my pleasure to introduce Romit who is an
assistant professor at Duke. He got his Ph.D. from UIUC in 2006 and within one,
one half years he's already gotten the career award an is doing very well.
He received a lot of press for this work that he'll be talking about today, like
newspapers in India calling him like the next generation great scientist. So
Romit, I'd like you to describe your work.
>> Romit Roy Choundhury: Well, that was an exaggeration. My father called me
up and said Indian-American scientist, what does it mean. So I was like okay.
Okay. So let me get started. I have 75 slides, so I don't want to be too long. But
again, don't worry, most of my slides are one liners and two liners. Okay. So the
topic for today is designing of virtual information tell scope using mobile phones
and social participation, and before I actually jump into the technical stuff, let me
talk a little bit about ourselves.
We are the Systems and Networking Research Group at Duke, and we are five
Ph.D.s, three undergrads, including one who is interning over here and one of
the Ph.D.s over here. We mostly work in the several, several layers of the
protocol stack.
Instead of going into the details, if you abstract it out, what we are really doing
has essentially two threads. One is we're looking at physical layer capabilities
and exploiting them at the MAC layer and higher level towards better protocol
design.
And the other approach is we are looking at applications and services at the
highest layer, the application layer and figuring out what are the challenges that
need to be resolved in order to actually support these kinds of applications.
So again, if you abstract what we are doing two teams, one from the higher
levels, what are the visions, what are the top down research problems and one
from the lower left, what are the physical layer capabilities that can be exploited
at the protocol design on the higher list, okay.
So that's essentially the team of our research. And we have four ongoing
projects right now, and one of them is called shuffle and let me just give you a
one line description of each of them. Shuffle is basically looking at interference
cancelation and message and message capabilities in which the basic punch line
is interference is not all that bad a thing if you understand interference you can
actually cancel it and do lots of interesting stuff with it.
So it's like keep your enemies close to your heart in some sense, try to
understand the interference and there by cope better with it. And we have been
designing protocols with those capabilities.
Spotlight is about using beam forming antennas in wireless mesh networks and
we are designing hybrid TDMA CMSA kind of protocols in that space.
Smart cost is about link layer multi-cast for video and audio kind of applications.
Again, with beam forming antennas. And today's talk is about information
telescope, which is essentially building mobile social participatory network using
mobile phones.
So this is essentially the organization of the talk. I'll start off with a little bit about
the bigger vision, the systems and applications, some of the research challenges
and some of the ongoing work. Okay. So let's get started.
To set up the context, we believe that next generation mobile phones will have a
large number of embedded sensors in them. There are already cameras,
microphones, accelerometers GPS, et cetera. Next generation there might be
compasses, health monitors, pull-in monitors, air quality monitors, water quality
monitors, a whole bunch of them.
And our view is each phone may be viewed as a virtual microlens and this
microlens essentially exposes a microview of the world to the Internet, right.
And now with three billion active phones in the world today, right, and it's actually
even growing faster by end of 2009 it's supposed to surpass laptop sales, so with
three billion active phones our vision is really metaphorically captured in this
diagram where all these phones are actually working as this microlens and we
want to design a software virtual information telescope which will allow you to
look at the entire world through the eyes and ears of the phone, through the
Internet, right.
So using the -- the users of this telescope should be able to zoom into any part of
the world and look at that part of the world in realtime through the sensors in the
phone, and those sensors need not necessarily be cameras only, it can be is
there WiFi at this location, what is the light sound temperature of this location, et
cetera. It can be the entire spectrum of signals, right.
Now, several applications can come up, including disaster recovery, health care,
education, et cetera, and but perhaps more fundamentally this might change how
we browse, query, learn, process information from across the space, right.
Now, that's a kind of a broad vision, kind of a metaphor if you will. One instance
of this vision is what we do in the project Micro-Blog and it has essentially three
steps which is content sharing, content querying and content floating, right.
And let me quickly go through these tabs again through pictures and hopefully it
should be clear. So content sharing, the key idea is instead of having people log
on their laptops and desktops, right, we are encouraging people to blog on their
mobile phones.
And so a person goes to a California beach, takes a video, talks about the beach,
likes the beach, and presses the send button on the phone, the phone client, the
Micro-Blog client takes all this multimedia information, timestamps it, location
stamps it and sends it to our server which then positions that blog at the right
place on the map, on say Google maps right now, right.
And as a user of the Internet, say a virtual telescope, the user goes to Google
maps, for example, zooms it into California beach and sees what people are
saying may be videos, audios, et cetera, et cetera, right. Well, this is when we
started with this, it was kind of new and exciting, but I think now you've seen a
whole bunch of applications which are very similar to this, right.
So what we thought is if you want to make this a little bit more novel, what we
said, hey, you can actually add querying to it. So the idea is this user is trying to
plan his vacation, say, to California beach, and sees quite a few could videos
about what are the cool places to go to, but no one talks about say parking in the
California -- around California beach.
So what Micro-Blog allows to you do is you can go to Google maps, select a
region using your mouse, and then record a query saying hey, I'm Romit from
Duke, can someone tell me if there's good parking in this region and then send
the query from the interface, which will actually viewcast that query to all phones
in that region that was demarcated by the user, right.
So all -- so people who are around the beach at that point in time will have these
queries pop up on their phones and let's assume for now there are some
incentives to reply, someone among them might take a picture of the parking lot,
might say well, five dollars for the day, here you go. And that's another
Micro-Blog that comes back and gets positioned on the map as well.
So if you generalize it a little bit, it's like strangers of the world asking questions
of each other on a spatial domain and that might be interesting.
Okay. So we said okay, good, but we might be actually able to take it one step
further and the idea is, okay, by the way, before I forget, now these kind of
queries can be participatory in which the user actually takes a picture or says
something, or it can be automatic where, for example, you say is there WiFi at
San Diego beach, right, and the phone can actually result a query for you and
say here, there are five for you WiFi, so on, so on, and then send back the query
and reply and you can get information.
So queries need not be only participatory, they can be automatic also, much like
the typical notion of sensor networks. Okay.
So we tried to extend this to the next step, and we said, okay, till now what we've
said is essentially taking information and super imposing on virtual space, in this
case GoogleMaps, right.
We could also take electronic information and superimpose on physical space.
For example I go to a restaurant, I like the food, I make a Micro-Blog about the
food and float it at the restaurant itself. And I push off.
Someone else comes to the restaurant and that information automatically gets
downloaded on the phone and he says oh, Romit had kung pow chicken at this
restaurant, looks good, he recommends it, maybe I should order it.
So it's as if you're floating post-its in the air, you're writing post-its and floating it
in the air and moving away and someone else who is going by that location reads
the post-it and might even revise it, add more to that post-it. And that's why we
call it post-its in the air, essentially floating information on physical space.
The implementation as you can imagine essentially there's some server which
is -- which knows where you are and accordingly pushes information based on
your phones's location.
Good. So please, please do, please do tell me if something is not clear or if feel
free to interrupt me. I'd love to have -- actually have interactions during the talk
as opposed to questions at the end.
Okay. So we believe that if you design this carefully, there's a whole bunch of
applications that can be enabled, and we have been thinking about a few. One
of them is tourism clearly. You can plan your tour based on other people's
feedback. Microreporters when everyone can be a journalist and actually CNN
already does this I report.
Another application we've been kind of excited about is on the fly ride sharing
where the idea is hey, how about if I get out of my office and I want to go home I
pull out my phone and I kind of see all people around me in my two mile radius
who can give me a ride towards my home. Right.
And if I can do this electronically, maybe it's like an eBay, a realtime eBay for ride
sharing and in the face of current gas prices people might appreciate that.
Now, the fourth one which we are also thinking about is essentially pitched for
the developing regions, maybe rural regions and we call this virtual order on
physical disorder, and the basic idea again is similar. Suppose -- so a lot of
people who go to India is not as much structure in India like in US for example
roads are not labeled, you don't know where to go, where to go for taxi, what to
do, et cetera, and for an American who goes to India they don't even speak the
same language, right.
So maybe one idea would be to take information and superimpose on physical
space so that you get off the flight your phone tells you walk straight, take the
door on your right, that's the taxi stand. And as you keep doing what the phone
tells you, your phone recognizes that you are in this location and tells you what to
do next. So it's -- so if you really push it to the extreme, traffic lights should be on
your car should not -- there should not be an infrastructure, your car dashboard
should tell you well it's green or it's red and you should go accordingly. But yeah,
that's extreme. People get psyched out when I say that.
Okay. So a whole bunch of applications. And we actually prototype Micro-Blog
with Nokia and 95 phones and there's a server and I can skip a little bit of that in
the interest of time.
And actually this -- the better version is live at this point at synergy.ee.duke.edu,
and you can actually get a link to that from my Web page.
But to give you a quick idea of what it looks like, so we were demoing this in
census last year and we in Australia, in Sydney, and there were phones in this
region, and so we drew this box and we send a -- typed a query which says other
restaurants close to the opera house within walking distance and my students
have these phones and they were walking around in this opera house region,
and the phone actually -- the query actually showed up on the phone.
And one of them typed out yes there are plenty of restaurants, and when they
typed that out, that reply came back, I said okay, now I can go. Well, this was an
orchestrated demo, but you can potentially you can imagine what it would -should look like in real life. Okay.
So then we thought okay, good, we have the demo set up, why not give it to
some of the Duke students and see what they feel about it. So we did a survey
kind of stuff with 12 volunteers, four phones. We had only four phones then, so
we gave them out in three rounds. Two I was not great and people complained,
et cetera.
But still, from the exit interview, I think we had a few very good, very interesting
comments which we were not quite expecting. So here are a couple of excerpts
from the exit interview and most of them said hey, this is fun, but we need a
cooler GY, right, and we absolutely recognize that.
People also said that we had to carry two phones and we often forgot, et cetera,
so those kinds of problems were underlying this kind of a survey.
But more interestingly people said of course privacy control is vital, I don't want to
give away all my pictures to others, all my videos, but people said unanimously,
we don't care about incentives.
And when asked why, people said hey, it's very interesting to reply to people,
especially if you know who is asking. It's kind of a service that they are doing to
others. Again, this is the undergrad crowd which might be a biased crowd, but
they felt it's cool to reply and they're kind of doing -- they're kind of helping other
people getting replies to questions that they might have.
The third one is people said voice is personal, text is impersonal. They did not
want to record their Micro-Blog and then if they did not like it they then want to
delete it and all that. They said, hey, it's easier to write text and correct it the way
you want it.
And the last one was logs mostly turned out to be 5 p.m. to 9 p.m. and well, that
was the time and they were really off from their regular schedule and that
probably says something when it comes to factory usage of the phone, et cetera.
Okay. So so far so good. We believe that Micro-Blog is a rich pace for
developing applications and services, but the question that most of you might
already be thinking in your minds, so what exactly is the research here, yeah,
that's the question we also actually struggle for some time, but then I think we hit
into a whole bunch of stuff that we are really convinced are very important
research problems. Especially when we were demoing it, things were pretty
good. Well, yeah, not much research, but as we started dishing them out to -- for
people to use, lots of things started coming out that we didn't see in the demo
phase.
So we are going to -- I'm going to essentially focus on these three research
challenges. One is energy efficient localization, one is symbolic localization, and
the other is location privacy. There are several others but I believe we do not
have the skill set to actually address them well, so right now they are low in our
priority. Okay?
So please feel free to ask questions at any point in time.
A quick disclaimer. Most of this work is ongoing, so it won't be mature but I think
the talk is more about identifying the problems, understanding the problems
space and essentially getting a feel of the solution sketch as opposed to
full-fledged mature research solutions.
Good. So problem one is energy efficient localization and we call this project
M-Lock. Okay.
So we gave out these phones to undergrads and they came back within 10 hours
and said hey, the battery is dead. And we said why? And we figured out through
extensive measurements that GPS is extremely energy hungry and with the
latest Nokia phoneware upgrades it comes to about 8.5 hours without any other
applications running. When you run other applications it's even less.
>>: [inaudible]?
>> Romit Roy Choundhury: The screen was off, too except maybe a couple of
times it turned off for some weird reason, maybe the application was doing
something, but, yes, the screen was off, too.
So and we did all these measurements and the GPS consumes, we saw it
consumes 400 milliwatt. We did AGPS as well, which is assisted GPS that was
marginally better but nothing -- something like half an hour better.
And then the idle power was really 55 milliwatt, which is really the green line over
here. The red line is the GPS floor and the orange is WiFi.
>>: Does it [inaudible] GPSes or --
>> Romit Roy Choundhury: So we try across two phones and ->>: The same Nokia?
>> Romit Roy Choundhury: Yes. We don't know what it's across different
platforms. But we have heard complaints from other people, also.
>>: [inaudible] I was wondering if some of the GPS that you're using this
changes drastically?
>> Romit Roy Choundhury: I don't know. So then people said hey, of course,
there are a lot of other alternate localization mechanisms Place Lab from Intel,
Skyhope, they are doing WiFi or GSM based localization. Yes, it saves energy.
WiFi gives you around 20 hours, GSM gives you around 40 hours. That's great.
But, hey, there's a cost to that. And that was with WiFi even in dense regions
you get around 40 meters of localization accuracy. With GSM, it's 400 meters if
you turn on iPhone they give you a big circle of where you are.
>>: This is [inaudible].
>> Romit Roy Choundhury: What is that?
>>: This is outdoors?
>> Romit Roy Choundhury: In indoors GPS is not working, so if you come to my
office you'll see phones lined up on my window all trying to get a line of sight to
the satellites.
Maybe some applications -- yes?
>>: [inaudible] why is it [inaudible] for locating restaurant.
>> Romit Roy Choundhury: Not enough?
>>: It should be enough for locating restaurants and stuff like that, right?
>> Romit Roy Choundhury: But there might be several applications which might
want to know where you are, say location specific advertisements. 40 meters is
not enough because you might be one of five stores in the mall and you don't
know where -- which store you are in. You don't want to advertise toothpaste to
a coffee drinker in some sense.
So let -- I'll come to that in a little bit, right. So again, there might be a class for
which it's fine but we believe there's also a class for which 40 meters may not be
fine and by the way, this is for dense urban kind of areas where there is pretty
good amount of WiFi density.
So the trade-off summary is something like this, 10 hours, 10 meters, 20 hours,
40 meters, 40 hours, 400 meters. And the research question we started to bump
into is can we achieve the best of both worlds? And 10 meters of accuracy may
be 40 hours of life, battery lifetime.
So the basic intuition that we are trying to use over here is that, hey, human
mobility is not brown in, people don't like go from here to there in seconds. And
there are a lot of back-turns in people's behavior, in group behavior there's a lot
of persistence that you can actually exploit to predict mobility and if you can't
predict then you need not do energy of localization sampling, right.
And that's the basic idea. But well, that's still kind of loose. Let's actually try to
formulate this a little bit better. So what we say is okay, given a trace of human
mobility, we are trying to convert that into an error versus time graph, and this
graph is like follows suppose this is your LT0 is your low case at time T0 and
suppose you're over here and you start moving and say you pause over here for
three time units and then again you start moving and you pause for two time
units and expose you know the location, the original location from where you
start.
You can convert that into this graph which says you're original error is zero and it
keeps increasing to T2, this one, and then you have paused there for two time
units, it's stationary, again it increases, it is pauses. So that's a pretty quick
translation of this trace into the error time graph, right.
And now we said okay, suppose you took a GPS reading at time T4, then
essentially your error drops instantaneously to zero and then again when you
start moving your error starts going up.
I have simplified this a little bit. People might be coming back to their original
position and this thing might decease and all kinds of things have happened but
without loss of generalization, this should give you the idea that we are trying to
get. So if you do a GPS reading over here, you've expended some energy but
you've gained accuracy. And that's the shaded reach, right.
If you do say another WiFi reading at T6, you have again expended energy but
you have gained accuracy by that amount. Essentially your total error is the area
under this curve, right.
And as you take more and more readings you're kind of pushing that envelope
downwards so that the area under the curve starts reducing, right. And now the
question is given an energy budget E, a trace D, and a location reading cost, say
what is the cost of GPS, cost of WiFi, cost of GSM, right, can you come up with a
schedule which will actually optimize your localization error for a given energy
budget?
So where all should you be cutting this graph down so that your overall or your
aggregate area under the curve is minimum, right. And that seemed like to us
like to a well formulated problem, and we have a dynamic program solution to
that with which we can actually find out what actually is the optimal that you can
achieve for a given trace and given energy budget.
Not going to go into that, but I can talk about that offline. But what it gave us was
the offline optimal offers the lower bound on error for a given energy budget. But
we need online algorithms to actually do it while mobile phones are moving. The
online optimal is difficult, so we need to design heuristics. Okay.
So we started looking at heuristics and we've got -- not gotten too far, but we
have some ideas. And again, the main idea is you do prediction based on
human mobility. You also do prediction based on how the accelerometer of the
phone is changing. If you are static and you started moving, the accelerometer
can give you hints. If you take a right or a left turn your accelerometer can tell
you stuff, so you use all that information to actually do prediction, okay.
By the way, the predictive heuristics can actually be fed into the optimizer,
essentially because if you do better prediction your this black curve would start
coming down because your error rate will start decreasing, right.
So we fed predictors into the system, and we have essentially three predictors
right now. They're all very simple one is you do simple interpolation. So
essentially you say well I'm going to take one GPS reading and then couple of
WiFi readings and I'm going to essentially extrapolate my GPS readings in the
direction of WiFi.
So I take a GPS reading at time T0 and I take a WiFi reading at time T2, say, I
extrapolate my location or my movement pattern towards location at time T2 but
using my previous GPS reading. And that is better than just saying okay my
current location is LW T2.
Again, simple but well, let's see, I mean it might work, it may not.
The second thing we are doing, you say hey, how about recording users' mobility
profile and at any given time if your current mobility is similar to your historical
mobility pattern, maybe you use your historical patterns more often, right. Yes,
that gives you something.
And then we are using accelerometers, we are saying okay at this traffic
intersection you're accelerometer says with .78 confidence that you've taken a
right turn, maybe I should believe that. And we are actually getting good results
out of the accelerometer about taking turns.
>>: [inaudible] on them at all times because [inaudible] when you are driving in
car versus when you are walking in the mall. How do you [inaudible] those
because you can't -- you can't takes those that combined across the two.
>> Romit Roy Choundhury: Yes, absolutely. So again, each of these historical
patterns you cannot rely completely upon them. They all give you hints and you
take all these hints and you try to predict.
And the other thing is if you see that there's good input or correlation in your
history, morning 8:30 to 9, for me it's 9:45 to 10:15, you drive to your office. If
you can use some of the temporal correlation, you can actually benefit. But, yes,
right now we have not used that in one of the heuristics or any of the heuristics
we have done.
So but accelerometer we are getting good data because you can predict whether
you've taken a right turn or a left turn, yes. Okay.
Now, the third thing that we are doing -- my Power Point is stuck.
>>: [inaudible].
>> Romit Roy Choundhury: What is that? Oh, I don't know. Okay. Oh, okay.
Thanks. [laughter].
Okay. And the third thing you are quickly doing is okay, we are saying hey, if a
person is going to a new place, we don't know what he's going to do, so why not
use population statistics from there. So if you have an intersection and you know
90 percent of the people take a right turn, maybe you can use that information
towards your prediction. And we are trying -- we are trying to gather that kind of
a distribution for every intersection in the Duke campus and we've used some
tricks that I can talk about offline.
And so we did -- so this very early results came in yesterday night from my
student and this is the optimal -- this is a heuristic which essentially takes a K
random set from the optimal sequence. By the way, the optimal sequence is like
a time T1 take GPS time T2, take WiFi at time T3, take GPS time T4, take GSM.
So it's a schedule you should take to minimize your accuracy -- to maximize your
accuracy for a given energy budget.
And then this random K you take subsequence, a part of that sequence and you
repeat that for your localization. And this is if you have the energy budget you
divide it among GPS readings and you keep using that GPS readings and it turns
out that actually GPS is pretty bad, your optimal can be much better.
And my belief is we can actually -- by the way, this is the best case in the other
graphs that we have, this is actually higher and we believe that you can actually
use all these accelerometer stuff to do better prediction.
Okay. So localization cannot be taken for granted. I think the energy
perspective is important and it will be a critical trade-off for the all kinds of
applications for the future. I think there's substantial room for saving energy
especially sustaining reasonably -- while sustaining good accuracy.
However, after we've done this to some extent my feeling was physical
localization may not be the way to go. In other words, an application may not
care what exactly is your coordinate. People might care what exactly is your
context and that's we are trying to look at symbolic localization where the idea is
your location is essentially your context and the application cares about that.
Okay?
So let me motivate what the problem is in a little bit more. So services may not
care about physical location, the service might want to know for example if it's a
location specific advertisement, the service might want to know if it's a coffee
shop, if it's Wal-Mart, if it's a movie theater, if it's a parking lot, et cetera, right.
Now, one way to do this is hey, why don't you take your physical location and
reverse map it to find out what is the context, right. So you take the location of
the phone, you look up a database and you say aha, this location is in Starbucks
and therefore you know the context, right. But then unless your physical location
is remarkably price, you'll never be able to say whether it's -- whether this phone
is on this side of the wall or that side of the wall and your context might change
dramatically. And that's where I was saying the person in Starbucks does not
want a coupon for grocery, right?
>>: [inaudible].
>> Romit Roy Choundhury: What is that?
>>: Send them both, send them the Wal-Mart coupon and the Starbucks.
>> Romit Roy Choundhury: Well, you don't want to do spam there, also, you
want to restrict it as much as possible, especially with energy interest matching
and all kinds of stuff.
So my belief is with physical infrastructure based location submeter accuracy is
difficult, maybe we should do something completely different, and that is what we
call symbolic localization. And the idea over here is why don't you sense the
ambience around you and try to say where you are. And that leads to this project
that we call surround sense where the basic idea is the phones microphone
listens to all kinds of noises around you, sounds around you, lights around you
and forms a soft signature of what this place is.
You train your system with these signatures or these fingerprints, and then you
localize accordingly, right.
So you gather fingerprints from mobile phones. You might use Micro-Blog for
that, and then you match a phone's fingerprint to your trained database, right.
So the basic idea is your GSM localization says you're in the mall, say 400
meters accuracy or 300 meters accuracy and your surround sense augments
that by saying you are in the apple store or you're in Best Buy just by
understanding the fingerprint of that particular store, okay.
Okay. So before we didn't believe that this is going to work, so we said hey, two
mobile phones we don't want to invest that into surround sound right away, let's
start off with very crude sensors in T mode invents and we use the light and
sound sensor on these [inaudible] and we actually we got good results so we are
currently prototyping it on Nokia 95s.
So what we did was we took the sound signals and we transformed that in the
frequency domain for a full year transform, and then we looked at different
frequency bands. Very interestingly we saw that they are very good signatures
like if a microwave is on, certain bands are high. If people are talking certain
bands are high, vacuum cleaners have signatures, refrigerators have signatures
and the idea is if you know, hey, the vacuum cleaner is on, the microwave is on,
the light is dim, maybe this is a Starbucks or a restaurant or something like that.
Yes?
>>: Are these sound sensors comparable to the mics that are ->> Romit Roy Choundhury: No, these are way more cheaper, so while belief is
once we go to the form platform we can do much better in terms of this
fingerprinting. But even at these lower frequency this is from 20 hertz to 230
hertz, so it's very low frequencies. We do not a catch the higher frequency ones,
yes.
But we've extracted 48 features from the light and sound where most of the
features are different frequency bands and then we do a 48 dimensional
fingerprint and then we do basic very simple data-mining classification of
[inaudible]. Yes?
>>: Does it matter [inaudible].
>> Romit Roy Choundhury: So right now we do not know that. But right now
with our sensors we've actually turned it in all kinds of orientation, including the
other way. And it did not matter much with the sound. With the light it does.
But light we are not depending on it too much anyway, simply because your
phone is going to be mostly in your pockets, and I don't think the camera would
actually pick up light too much, but sound itself is doing pretty good.
So let me get to the results a little bit. Okay. So this is essentially the blog
diagram. We took -- we take raw audio data, divide it into blogs, do FFT, extract
features similar for sound, we have a fingerprint with 48 features, we do a
nearest neighbor go to the basic mining nearest neighbor algorithm and we get
what is the location of the phone, right.
Again, remember, you cannot say the globally unique location, we assume that
GSM tells you you are in the mall, we are essentially discriminating between
shops in the mall. We are not saying you are in their own Starbucks versus
Seattle Starbucks, we cannot do that. Right. Okay.
So here are results. And these are whole bunch of restaurants and the packed
coffee shops, and this is Chic and this is Whole Foods, this is Twainese, which is
Duke cafe, which sucks, Blue Express which is actually a good cafe. [inaudible].
Yeah, I'm on record. So yeah. So we actually did auto correlation and cross
correlation and you see that for details -- so all of them, you are able to localize it
very well with sound and light together.
We were having some problems with streets. We are not able to say although so
this data shows you were able to but later we got a whole bunch of other data in
which the street auto correlation is not turning out too well and I'll tell what you
the intuition is just in the next slide.
But the point is when we use both sensors, light and sound, it turned out better
than just using sound, which was much better than just using light. Because
sound proved to be a very good fingerprint off the place. Okay.
So why this really worked and we didn't expect it to work, but why it worked was
when you look at a whole cluster of stores or shops in an area, they're designed
to be different. You don't want to have five Chinese restaurants, one beside the
other with the same ambience, with the same lighting, same sound, same music
playing, et cetera. Economic reasons want you to design them differently so that
the business gets distributed, so on, so forth, so there's diversity in spatially
clustered business locations, and we are trying to explore that diversity over
here.
If they were very similar, it would be hard, right. Okay. Streets unfortunately
does not have that property. So most of the streets seemed to be similar in
terms of light and sound with cars passing by and we are actually having a tough
time classifying streets, one street from the other. Question.
>>: [inaudible] to train this database, how would you [inaudible] to actually ->> Romit Roy Choundhury: So I spend around $500 paying for my students'
dinners and cafes right now and train these -- they go around these coffee shops
and these locations getting fingerprints.
But my assumption is, my hope is when we use Micro-Blog, if people are
blogging, maybe we can extract some of the ambient data from there and keep
training the database. And maybe if you get location over there, you might be
able to do good. So, yeah, maybe we can use Micro-Blog peer-to-peer,
something to actually train it. But for now, we've trained it just to figure out
whether this at all works or not, and we are in that stage right now.
>>: [inaudible] dependency, right, because for Micro-Blog location to work
protocol you need a location and you need morning blogs for the location system
to work.
>> Romit Roy Choundhury: Right. Right. So essentially what I meant over there
was suppose someone goes to Starbucks and blogs and says I'm in Starbucks,
you can use that to say ->>: You don't actually expect that Micro-Blogs --
>> Romit Roy Choundhury: Parse the data, parse the [inaudible]. But I'm
assuming this can be done, this training of it.
Okay. And maybe if a phone has a lot of battery, right, maybe the phone can use
the GPS and tell me exact what is.
>>: [inaudible].
>> Romit Roy Choundhury: It would work, yes, it's probably true.
Okay. So then we said okay, we can actually augment symbolic localization with
accelerometer. So if you can't say where this guy is through light and sound,
maybe you can look at the accelerometer reading and if the guy is actually the
accelerometer reading says the guy is walking up and down, maybe it's a
Wal-Mart, this guy walking up and down aisles.
If this guy has been sitting and the lights are dim and the noise is low, maybe it's
a restaurant. So we added accelerometer on top of this, and I'm not going to talk
too much about this, but the idea is people's activity can be guessed through
accelerometer readings and if you can correlate that to the context, maybe that
also gives you a localization information.
So again, very similar approach over there, you train that database with the -with the accelerometer. Let me jump quickly through this. Again, this is K
nearest neighbor classification and [inaudible].
What we've done to save energy a little bit is the phone extracts the feature from
the accelerometer and only sends the three features to the server which then
runs the matching algorithm with the database and essentially feeds it to the
classifier. And that's because you did not want the phone to actually export all
the accelerometer data to the server.
And this is something we have built in the phone right now, and we have the
demo, and this is in a modicum workshop paper this year about GPS free
localization.
And we call this project AAMPL, which is Accelerometer Augmented Mobile
Phone Localization.
So here are again some stores, restaurants, fast food, and different kind of
stores, and we see that there are sufficient diversity in them to be actually able to
say something about it, and we actually beat Google in the results where Google
are all these blue ones and Google said this is one of the streets in 9th Street in
the Duke campus and these are the three stores and Google said these are the
three stores.
And they were clearly wrong. And the red push pins are really what AAMPL
predicted and they are definitely correct because the movement patterns in these
three stores are different.
And again, we are hoping that these stores would be by design different so that
we can exploit that diversity. Okay. Good.
So thoughts over there. Main idea is the surrounding is a fingerprint effective for
separating out nearby context. Reality specialty cluster shops are diverse by
design and that really is AAMPL and surround sets. And we are trying to
augment that with cameras as well.
We go trying to say, hey, suppose you still cannot say where this guy is, how
about the camera taking a couple of pictures and maybe even the camera has a
compass which tells you what is the orientation of the camera. And if the
camera's pointing upwards maybe you know, you know it's the picture of the
ceiling and the camera's pointing sideways, maybe you know it's the picture of
the walls and then you use that information with very basic image processing and
you know, hey, this is a place when has green ceiling, must be a panera bread or
something else, you know.
And we are trying to kind of add everything that you can sense in your
surrounding, even peer-to-peer if you know who other people around you and
they are your social network buddies, maybe you're at the restaurant, right.
Things like that. And we are trying to essentially sense the ambience in all kinds
of ways and localize accordingly. And this has implications in other kinds of
research like activity recognition, advertisements, all kinds of stuff.
Okay. So finally the third problem is location privacy, and I think I am running out
of time, so I'll quickly tell you what the intuition over here is.
>>: [inaudible].
>> Romit Roy Choundhury: I have 10, 15 minutes. Oh, that's good. Okay. So
location information reveals context clearly and that's one of the key questions
people ask me. And then there's a thin line between utility and privacy.
If I tell you I'm in Seattle, tell me what restaurant I should go to, you as an
application it's hard for you to reply. If I tell you I am in room number XYZ at
Microsoft Research and tell me what I should be doing next, hey, that's a privacy
bridge, people might hit the panic button, right.
So it's kind of a thin line where is privacy and where is utility, and we are trying to
actually address that problem. Now, people have looked at this problem in
various angles. One is using pseudonyms. So people said hey, why don't you
say you're Jack or John or Susan, right.
But then if there are applications which need continuous querying or tracking,
pseudonyms is not good enough. So for example suppose this is my path, my
mobility path and I'm saying hey, I'm John, then I say I'm Leslie, Jack, Susan,
Alex, and then I get to my office and people know where my office is, people will
easily know all these pseudonyms correspond to Romit right. So that doesn't
work in a whole class of applications.
People said, well, what he should do is key anonymity where you say suppose
this is you, you draw a box with say four other people. This is a bounding box.
And to the application you give this box and say tell me the restaurants in this
box. Hey, well in a sparse region, this is not going to work. Then you have to
wait for other people to come into this box before you give it to the application
doesn't really have the realtime nature of it, right.
So we said well, it's good for certain applications but maybe not for another class
of applications. Okay.
But then what was interesting was the recent work by Mark [inaudible] in CCS
this year, last year, and that the intuition is actually simple. It says hey, if you can
cloak some reasons in mobility traces, you can actually give some amount of
anonymity. For example, A is moving, B is moving, if I put my hand on top and
then you only see who is coming out, you don't know which was A, which was B.
And that can be used towards confusing the attacker because the attacker does
not know what happened in this cloaked region.
But this still has a problem. The problem is you're implicitly assuming that A and
B are intersecting in both space and time. In a sparse region that may not
happen.
Also, you'll to have do this offline. You would not know who all got intersect at
what point, so you have to get all the traces, figure out where people have
intersected and cloak them. Also, if you had a query in this point that query is not
going to get result, right.
So what we're saying is hey, we want to tackle all these problems, you want it to
be realtime, you want it to have high quality of localization, which means I don't
want to [inaudible] the location to a big bounding box, I want to exactly have that
location. I want to entropy guarantees, I want some privacy guarantees from the
system and I want it to work even in a sparse region.
Okay. So what we did was what we called cash cloak, and again the intuition is
exploit mobility predictions to create confusions. And let me actually jump to the
idea, okay, by the way the architecture is here. We are proposing a kind of a
third party cash cloak system in which we assume that the users will trust cash
cloak which is a limitation. But assuming users are willing to do that, cash cloak
will do some an optimizing and will actually give an API to the location
applications and the location applications won't be able to figure out or track you
while you're moving.
Okay. So given that, our basic idea is as follows: So let's say, okay, maybe I
should just jump to the figure and actually do that. So let's say user A is driving
this path and user B is driving this path, right. What cash cloak does is cash
cloak knows the individual locations but cash cloak gives this entire path to the
attacker or to the application.
So the application's view is this. The entire path of A and the entire path of B,
okay. The application replies to each of these locations, say the query is what
are the restaurants around this place. So the application replies to -- replies to
that query for every location here and for here, okay.
Now, suppose this user has taken a turn and goes to this new path. What cash
cloak does is it predicts the user's path and makes sure that that predicted path
intersects with some other existing path. So and then it actually gives this entire
set of paths to the application so the application now sees this. Now, open that
the application cannot figure out whether the user turned from here, from here,
from here, or from here.
So as the users are taking turns, it's more and more difficult to actually track the
user, right. You can actually give privacy guarantees on top of this if you want to
achieve certain guarantees you can do spurious branching. So for example, you
can take a branch out of this and make it connect to this region. So then you do
not know whether some guy came from here and came like that or some guy
came from here or some guy came from here and so on, so forth.
So if you look at it from entropy perspective, at every intersection the entropy is
increasing, right, or in other words the probability that you can track this guy is
kind of diffusing with time. And that kind of gives -- gives you anonymity to quite
a good extent.
And the thing to notice, you are getting high quality responses because the
application is telling cash cloak what are the restaurants for each of this location?
Cash cloak is cashing that and when you get to that point, cash cloak is telling
you here are the locations, here are the restaurants for this location. This is a
trade-off, it's not all win-win. There's a lot of overhead that comes in because the
application is responding to all these locations, right.
But then we are saying we are thinking that that's on the wide infrastructure,
maybe we can live with that if that buys us privacy, that's a trade-off we are ready
to live with, right.
And -- question?
>>: I'm not sure. It seems that cash cloak knows everything at all times, right?
>> Romit Roy Choundhury: Yes.
>>: So why is this a solution to privacy?
>> Romit Roy Choundhury: So you're assuming that cash cloak is trusted.
>>: Why?
>> Romit Roy Choundhury: Verizon knows exactly your locations all the time.
[brief talking over]
>>: But this will give [inaudible].
>>: Either Verizon knows and therefore you might have to tell everybody
because you know federal government can query Verizon and they know where
you are at all times, or you don't tell anybody.
>> Romit Roy Choundhury: Our take is that if you trust one service and if that
guy knows -- yes? I mean if you do not want to do that, then you have to make
this you diffuse that out in some sense. But if you can trust one service you can
do the rest.
>>: [inaudible] more that you can't trust the -- essentially if you can trust one
service you're telling everybody because the US government can always come
and [inaudible].
>>: Yeah, I mean you [inaudible].
>>: So I think it's a [inaudible] you can trust -- you can either trust everybody or
nobody.
>> Romit Roy Choundhury: So that's the limitation of this approach. Yes. But,
okay, anyone way if we go through and we calculated the entropy for each of
these locations and then we see that over time at every intersection the entropy
actually goes down very quickly and then within around -- okay, so we, by the
way, we simulated this in a simulator and we took Durham's map and we used
the US sensor bureau trace data, which has all kinds of traces for vehicle mobility
with the speed limits, traffic lights, et cetera, and we found out that the entropy
actually increases very quickly with time. So within five minutes you can achieve
a pretty high amount of entropy and you can still get accurate pretty good quality
of service in terms of the responses to your query.
And again, the overhead is not too much simply because cash cloak can cash
this information. And if your query has an expires time of say 20 minutes, that is
huge. I mean, that brings down the overhead a lot. Even with one minute we
saw that the overhead is not too much, especially if it's on the wired backbone.
Yes. But again that is definitely true. We do assume that you have this trusted
server whom you can trust and that may not be true as you've pointed it out. And
again if you want some amount of privacy guarantees, you can do adaptive
branching. You branch anywhere you want, or intelligently, of course, and you
can also introduce dummy users into the system, right, and as if there are other
users which the attacker doesn't know but which are actually dummy and if they
can emulate human mobility patterns you can actually even confuse the attacker
even more.
Okay. So closing thoughts, two nodes may intersect in space but not in time,
hence mixing is not possible. There's a whole bunch of papers which have
actually kind of assumed this model. Our belief is that mobility prediction can
actually allow mixing in both space time. Right? Simply because you're
predicting, so even if you are not there in this room at the right time or we are not
in this room together by prediction we might be intersecting in this room at the
same space and time and that is what really underlies cash cloak and that gives
you the benefits. Okay.
So in conclusion, the virtual information telescope is a generalization of mobile
location based social participatory computing and this is essentially the vision
you envision every mobile phone is a lens and you see the world through the
eyes and ears of these lenses. Just developing applications is not enough. We
believe there is a whole bunch of research challenges that underlie these
applications especially if you want to go past the demo stage.
And we have actually been looking at the space of research through the project
Micro-Blog and the several projects coming up after that, and the current
snapshot of our research are these four projects, the Micro-Blog is the overall
system and application, M-Lock is energy efficient, localization, GPS is too power
hungry, you are to optimize that. Surround sense and AAMPL is essentially
symbolic localization that is aware of your context sensing everything around in
your surrounding and trying to say where you are, and cash cloak is a location
privacy via some kind of mobility prediction.
So we have a couple of other things that we are kind of emerging, that are kind of
emerging in our labs. So for more information please stay tuned to our lab,
which is Systems Networking Research Labs, synergy.ee.duke.edu. And I'll stop
here and be glad to talk questions. Thanks a lot.
[applause]
>> Romit Roy Choundhury: Thank you.
>>: About the cash cloak aren't those like fancy precedents for people trusting
like the bank or similar things like that? There's a big difference between
advertising to everybody where my [inaudible] is and my bank [inaudible] right?
So I would say that this mobile doesn't like seem to be bogus to me [inaudible].
>>: But there's no [inaudible] location. There's no [inaudible] other people
[inaudible]. So that is [inaudible]. Nobody's managing verification.
>>: And if people are willing to give that.
>>: I think it's just a question really to create privacy, right. I mean [inaudible]
pointed out [inaudible] is absolutely valid. If somebody has the information and
then there's reason to need to get it, then you can get it through the legal system.
>>: Yeah, but that goes for you know [inaudible] institutions, too, right.
>>: Oh, yeah.
>>: Then there are well-defined legal protections for, that's why [inaudible].
[brief talking over]
>>: So you shouldn't assume that your [inaudible] [laughter].
>>: [inaudible].
>>: Actually that's true. [laughter].
>>: [inaudible] trying to protect against. There's a big difference between
protecting against the government knowing things or just a random person out
there writing application and can figure out my information.
>>: Yes. And why -- I guess the whole [inaudible] whole point of telling this one
trusted entity is so that you don't have to trust you know your provider, Verizon or
whoever. I mean, who else is going to know? I mean, who are you protecting it
from by having this [inaudible]? Why would you trust them more than your
[inaudible]. That's what was [inaudible].
>>: I was assuming that the applications [inaudible] just written by or ->> Romit Roy Choundhury: Yes, the applications could be written by anyone
else like WebSphere or something. They would like to have your [inaudible] but
you don't want to trust them but you want to trust this one entity which builds the
reputation.
>>: [inaudible] you trust the certification.
>>: So are you requiring that applications have to use this interface, right?
>>: Assuming that someone would buy interface, someone would buy cash
cloak means that that interface will get the privacy or verifications.
>>: So why would applications want to use it, applications -- if the application
developers or the.
>> Romit Roy Choundhury: Well, it depends on whether the user are willing to
go outside -- suppose cash cloak is successful. Users may not be willing to go
to -- users may want to get associated to cash cloak only and that user base
might be the reason why applications want to tap into that.
>> Ranveer Chandra: Thank you.
>> Romit Roy Choundhury: Thanks a lot.
[applause]