>> Helen Wang: Good morning everyone. It's my pleasure to welcome back Serge Engelman,
and Serge, we all know him very well. He interned here twice and he’s a frequent collaborator
with some of us. And today Serge is interviewing with a security privacy research group, and
he's going to tell us about his work over the past couple years at UC Berkeley as a postdoc.
Thank you, Serge.
>> Serge Egelman: Thanks for the introduction. I was going to preface this with, so this is
mostly work that I've been doing over the past year and a half at Berkeley, looking at
permissions on mobile devices and helping users to make better decisions about how they
grant third-party applications access to their personal data. Helena graciously invited me to
speak about this in July, and so I've been agonizing about what news data to put in here,
whereas what background information to leave, and so I'll warn you, for those of you who saw
the talk in July, that the first 20 minutes or so might be a repeat. So feel free to take a nap, and
I’ll wake you when I get to the new stuff.
>>: [inaudible].
>> Serge Egelman: That works too. So to step back for a second, so most of the stuff that I do
is within the usable security space, and what that means is I try and solve computer security
problems by designing interfaces that are better suited for humans, so applying HCI
methodologies to solve security problems. And within that space, there are generally two
recognized major problems that we see frequently. So the first problem is there are a lot of
unnecessary security interactions and these result in habituation, and what that means is
people become accustomed to seeing lots of security information that they otherwise might
not care about, to the point that when they see the information that they do care about,
they’re used to just swatting it away.
The second problem is frequently users are asked to make decisions that very simply aren’t
unqualified to make. So that's another big problem. And a lot of the solutions to these though,
can be learned from looking at the classic hazard avoidance literature, and so looking at that,
we see there are really three different ways of dealing with hazards, in order of preference. So,
in the best case when we can eliminate a hazard we should do so, that way there's nothing
really for the user to fall victim to or anything extra for them to do. If we can't do that, we
should try and mitigate the hazard in some way. And then in the worst case, if we can't do any
sort of mitigation, we are stuck with warning of the user. And frequently what we see within
the computer security space is that we jump to the third one, warning, when we could probably
be doing other things that are preferable.
And so looking at the mobile space, why is mobile interesting? Well, 50 percent of the US
population has a smart phone, which means if they have a mobile device which has the ability
to install third-party apps, and you know these apps can take advantage of all sorts of sensor
data, which create really rich user experiences, but with great power comes great
responsibility. We see threats such as malware and grayware, so malware, unwanted apps that
can cause damage, so taking your personal information, maybe holding your data hostage, stuff
that users don't want on their devices. But that's pretty rare since most of these marketplaces
are centralized; when that stuff gets identified it's removed quickly and pulled from the users’
devices.
More frequently what we see as grayware. So these are apps that users actually desired, they
intentionally installed on their devices, but the apps might be using data in ways that were
unexpected to the user. And so the question is how can we make them aware of this? So
looking at one scenario, say a user finds unexpected SMS charges on their bill. They suspect an
app might be responsible. What could they do to determine what app is responsible? So here's
a screenshot from Android, a lot of this work was done on Android, but I firmly believe that the
results are generalizable to most of the mobile platforms out there. So the first step would be,
you know, selecting the menu, brings up all the apps, and now we have to find the settings app,
they scroll down, click the settings app, it brings up this whole new screen, they have to know
to click the applications sub panel, which then brings up more information to read, they have to
know to click manage applications, and then that brings up a list of applications on the device.
And then they have to sequentially go through, app by app, bringing up those app settings,
knowing to scroll to the bottom to see the list of permissions that that app has been granted.
And so ostensibly, the user would do this for every single app on their device to determine a
sub set of apps that have the ability to send SMS messages. And even then, once they've done
all that, it's not clear what app is actually responsible; they've just narrowed the list down.
So there's a lot we can do to better convey information about how apps are using their data.
So in the talk for today, I was going to talk about some of these problems with the current
choice architectures: how users make decisions about what apps do with their data, how we
can improve this by applying some better intelligence, in terms of choosing permission
mechanisms, and then finally, showing that in some cases, it's much better to simply beg for
forgiveness than ask permission. And then I'm going to talk about some future work.
So with regard to the current problems, we did some work about a year ago, looking at whether
Android users understand permissions on their devices. So we did this online survey of a little
over 300 actual Android users, we were sure that they were Android users because we
recruited them using AdMob so we could be assured that they were seeing the survey on their
device because we advertise within another Android app. And we showed every participant
three different permission requests, and we have multiple-choice questions asking them to
define what this permission allowed an application to do. So in this example we have granting
Internet access, there are four options plus none of these and I don't know, we removed the
people who left these completely blank since we had an explicit none of these option. We also
removed people who answered I don't know. So we only looked at people who selected one of
the five options.
In this case, the correct answer would be loading advertisements and sending information to
the application server. So every participant had three of these, and what we found was, on
average of the three, they got .6 questions correct. And so we can, you know, calculate the
distribution of expected answers from random guessing, we have four multiple-choice
responses, three times the expected value would be, I guess about a third of what we observed.
So this is statistically significant; it indicates the people weren't just randomly guessing, they
put some effort into this. But, at the same time, the majority of people weren't able to identify
any of these permissions. Only eight people got all three correct. So there are some problems
here with comprehension.
We followed this up with a laboratory experiment to get qualitative data on why these
comprehension rates were so low. We recruited 24 Android users from Craigslist, and we had
them do three different things. The first two tasks, using their own devices, we observed them
install two applications. We gave them scenarios, for instance, install a coupon saver
application. And so they went to the market and were allowed to download whatever
application they wanted that they felt fulfilled the scenario, and we observed whether they
viewed the permission request screen during this process. They did that twice, and then the
third task was, using their device again, we asked them about an application, their choice, that
they've used frequently. And we had them open that application, interact with it, then we had
them go to the permission screen that I showed you at the beginning of the talk, which list all
the permissions that that application had been granted, and we simply asked them whether the
application had the ability to send SMS messages.
So what we observed was, during the installation task, fewer than 20 percent viewed the
permissions during the install. The request came up after they clicked the install button, and
then they simply click through them without reading them which was kind of expected. This is
also likely an upper bound. We performed this in the laboratory, so it's likely that they were
under pressure to maybe apply more diligence to the task than they would've otherwise. And
then also, when we asked them whether this app that they were already familiar with and for
which they could view the permissions, could send SMS messages, a majority of them were
unable to answer this correctly. And the reason for this is they knew the permissions that it
was granted, but they didn't know that the permissions it hadn't requested. So they didn't
know that SMS was a separate permission. So being able to state definitively whether an app
can do a particular action requires some sort of familiarity with both the granted permissions as
well as the ones that weren't granted.
So from this we came up with several suggestions on how permissions could be improved more
generally. So one, people were habituated to the requests since they'd seen these requests
with every single application they’ve installed, they were used to clicking through them. So one
suggestion might be only prompting when necessary.
>>: You mentioned that these application stores are well curated and [inaudible] taking down,
so might not be rational to ignore the permission screen?
>> Serge Egelman: Yes and no. So they’re curated for stuff that's, you know, outright bad,
right? So things that’s malicious. But things where it comes down to user preference, so
specifically privacy stuff, so people have a wide range of privacy preferences, in some cases
some people care about disclosing location, others do not. And so basically it comes down to
user choice. So the next suggestion was people were, a plurality of our participants were
simply unaware that these things even existed. Since they were so habituated to clicking this
after the install screen, a lot of them said, well, they thought this was just another EULA, and it
wasn't really about granting abilities to an application. So the information was maybe being
presented at the wrong time during the installation process. Another thing is there are possible
cognitive biases, so maybe after clicking install seeing this; they were subject to choice
confirmation bias. They don't want to revisit that choice they are ready made. So maybe this
information needs to be provided at a different time, maybe earlier when they’re browsing
applications, or maybe even later at runtime, for instance, when the permissions are actually
used.
And finally, there's this problem of understanding what an application is doing requires
understanding what it's not doing or what is not asking to do. So maybe we should narrow the
list of possible permissions. So in Android they have over 100 different permissions. I think
Window’s phone is a lot better; there are fewer than 20. iOS has, they've changed. It used to
be that only location and notifications were prompted at runtime, and they’ve expanded that
to add photo, access the photo library and contacts. And so this brought about a much broader
discussion on permission granting mechanisms generally. So these were all problems with
install time mechanisms. But maybe there are other ways of asking for permission. And this is
applicable to more than just smart phones. Frequently, I guess we’re heading towards an
environment where there's a lot more centralized app distribution, such as having a store or
bundled with the OS. We see this both in Windows now as well as on the Mac. Certainly their
permission requests are integrated in that.
So again, looking at install time warnings, there are pros and cons to these. They're adaptable
to many different permissions because it's simply just a little text box here where you describe
what's being requested. So there's nothing really dynamic about that; it's easy to write these.
They could also be used to grant future abilities. So at the time that the permission’s granted
isn't the time that it's actually used. The problem is, especially in the mobile space, there's
limited real estate. So in this case, there are just three different permissions that the user sees
by default unless they scroll all the way down, so it's easy to overlook. The other problem is
they lack context. So when these are actually used, it's not clear what they're going to be used
for as opposed to maybe a runtime warning which adds contextual information, so when the
user sees a request for location, they were doing something to trigger that request, and so they
might have a better idea of why it's being requested. Likewise, these could made dynamic by
including other environmental information. So over here we have a Bluetooth request, we
could state the name of the device. Obviously you can't do that at install time. But these have
problems too. So everyone's probably familiar with this.
So habituation. When users see these frequently they stop paying attention to them and so
they're used to the benign situations, but when there's a potentially malicious one, they've now
become trained to dismissing these without reading because they see them as a barrier to
completing a primary task. But at the same time there are also things that we could do to grant
permission without interrupting the user. So there's curation. So if we have in our market
some sort of review process so that every app that's distributed has gone through some sort of
heuristics, maybe that's a way of preventing the user from being inundated with too much
security information. The problem with this is it’s very resource intensive, so it's really hard to
do this automatically, there needs to be some sort of human oversight, especially if you're
going to be either revisiting decisions and pulling apps off devices or rejecting apps from paying
developers.
The process is also completely opaque to both users and developers alike. This is what Apple
has been doing, but we've seen lots of problems with it. So everyone's probably familiar with
the Path fiasco. So Path was a sort of social networking app. Turns out that in Apple's terms of
service, they prevent developers from accessing the contact list on the device, but there is no
actual enforcement of this. It's supposed to be enforced through the curation process,
somehow this slipped through the cracks. Millions of users were using this application when it
became clear that it had been pretty frequently violating the terms of service agreement. So
curation might not be the best way.
But there are other emerging approaches for granting permission without interrupting the
user. So what I call Trusted UI elements. So these are system drawn components or access
control gadgets. The developer essentially specifies an area on their app to draw a button or
some other component and then the OS is responsible for drawing that. And these are
advantageous because the only way of granting the permission is by interacting with the OS
drawn widget rather than made developer drawn one, so there's no real way to spoof them.
Certainly developers could spoof the look and feel, but interacting with that spoofed widget
doesn't actually grant permission. Another thing you could do with this is granular access
control which doesn't really, hasn't really been enforced before. So maybe you have a widget
for adding stuff to the calendar, an app requests this, the user then interacts with the system
drawn widget and can add a subset of the events or can make changes to the request before
it's granted, and then the app can't override those decisions.
But these have some drawbacks. So one case is you can't, it's hard to grant staff in perpetuity,
also it's hard to grant stuff where the initial request wasn't generated by the user. Certainly
you could do this with the various callbacks, but I argue that this breaks the user's flow and sort
of these devolve into just another confirmation dialog if they are explicitly performing some
permission granting task to grant a future permission. And then this came, this idea of implicit
access came up. So in certain cases, if our goal is to minimize habituation by not bothering the
user with unwanted interactions, when the risks are really minor or can be completely
reversed, maybe we shouldn't ask a priority, instead we should grant the permissions, but allow
the user to audit what happens so that when the misbehavior occurs, they can go back and
undo it.
And so from this, we've created sort of a mechanism hierarchy in terms of desirability. The
base case are the install time warnings, which are generally undesirable, but they're sort of a
one-size-fits-all approach for applicable to pretty much any permission. The other end of the
spectrum there's the implicit access, which are most desirable, but you can’t always use them.
Or to put another way, so then the issue is deciding between which one of these mechanisms is
most appropriate for a given situation. Or to put another way, grant me the serenity to accept
that some permission requests cannot be eliminated, the courage to eliminate the ones that I
can, and the wisdom to know the difference.
So then that brings up the question is: how do we know the difference? So we did this thought
experiment, me and a couple grad students in the group, where we amassed a collection of all
of the possible permission requests that a user might see. We came to this by gathering all the
permissions from Android, Windows phone, iOS, and the Mozilla WebAPI. We removed all the
duplicates, and we also removed the ones that the user probably has no business seeing. You
know, things like enabling multicast, most users that's one of those decisions that most users
just aren't qualified to make. And for each of these, we individually asked what’s the best way
of granting permission. We went through and then met together to try and reach consensus.
And we found that the way we'd reasoned about this, there were a couple common questions
that arose.
So the first factor that we looked at is what’s the risk level of granting one of these? What are
the consequences if this were to be abused? So one example might be a permission to
permanently disable the device. So you might have some sort of security software on there, so
if your device is stolen, you go to some website and it destroys all your data. So this is pretty
severe, especially now that we see a lot of malware, which is holding devices hostage hoping to
try and profit. Is this reversible? Well, it's specifically designed to not be reversible. Is there an
incentive for widespread abuse? Yeah. So the consensus here is since we’re granting a future
permission, really the only way of doing this is would be an install time warning.
Another example on the other end of the spectrum is changing the time. So what's the risk
level here? Well, certainly you can come up with some sort of James Bond style plot that
hinges on changing the time on someone's phone, but for the most part, that's not really
applicable to most people. And really the risk is just an annoyance. The user could go and
change the time back.
>>: So slides ago you asked the question what’s the best way, and then you give this example.
Do you come up with objective way to decide what’s the best and how it's coming out?
>> Serge Egelman: Next slide.
>>: Good. I’ve been waiting two slides.
>> Serge Egelman: Okay. And so based on the patterns that emerged in terms of how we
reasoned about these individually, we came up with a flowchart for determining the most
effective way for granting permissions. I'm not going to say this is the best way, this is a formal
way that we came up with based on a lot of discussions, and I think this still needs to be
validated. But I think this is an improvement, and it provides a process to aid developers,
platform developers in choosing what mechanisms to use. So we have five questions here.
And so the first is, and we have this notion of revert ability, so could the device be reverted to a
previous state if something were to go wrong? If the answer is yes, then you should always just
grant those permissions implicitly but offer some sort of audit mechanism so that users can
identify misbehaving apps. Likewise, if it can't be necessarily undone but it is just an
annoyance, then we should do the same thing there. But if the risk level rises above that, the
question is did the user initiate the request, and if so, then in every case we should be able to
use Trusted UI components or ACG's. And if the user didn't necessarily initiate it but maybe we
can allow the user to grant a subset of the request, we could still use system drawn
components or choosers to allow the user to do so. And then finally, if the answer to that is no,
we are stuck with either a runtime warning or an install time warning and the way that you
differentiate those is whether the ability needs to be granted in the future, in which case you
really need an install time warning.
So using this model, we went back and looked at our list of permissions, and what we found
was that the majority of them could simply be done away with and replaced with audit
mechanisms. We found that a quarter could also be replaced with Trusted UI, and at the other
end of the spectrum, there are the install time warnings. Now recall that on Android, that was
100 percent. We've now reduced that to six percent by applying this model. So this could
dramatically reduce unnecessary interactions.
>>: So presumably the reason why implicit access you do more often. You put that at the top
of the flowchart as opposed to Trusted UIs. Every Trusted UI component creates a cognitive
load, they to recognize a certain type of widget; they have to know about it. So 23 percent
sounds like a lot of widgets to know about.
>> Serge Egelman: It's not necessarily the user has to know about it because honestly if the
only way of granting, well, okay. It needs to be intuitive that that widget does the desired
thing, so maybe a button for the shutter of the camera. But if the only way of achieving those
actions is through the Trusted UI, it just needs to be made obvious that those components will
complete the primary task. The user doesn't have to memorize all the Trusted UI components.
>>: Why not have the flowchart to start with Trusted UI. Shouldn't Trusted UI be the first to go
to thing, I mean because if it's so easy.
>>: Can I make a first clarification?
>> Serge Egelman: Sure.
>>: The user does not need to recognize the Trusted UI.
>> Serge Egelman: Right.
>>: If the Trusted UI is not used, permission is not granted. Now they will not be achieved.
>> Serge Egelman: Well, it needs to be, so the user is trying to complete a primary task, right?
And so the Trusted UI components need to facilitate that. But the user doesn't have to be
cognizant of the fact that these are, they’re granting permission through these.
>>: Given that, why isn’t Trusted UI at the beginning of your flowchart instead of partway
through?
>> Serge Egelman: Because there are also, you know, other constraints, so balancing needs of
developers. So simply telling developers that you no longer have the flexibility to do this, you
have to use a system drawn component, might be overreaching and might stifle some
innovation. So for the things that are truly minor, we should just not care about that.
>>: Can you give an example, something that [inaudible] implicit access and [inaudible] Trusted
UI [inaudible]?
>> Serge Egelman: Connecting to the Internet.
>>: So why do user needs to care about connecting to the Internet?
>> Serge Egelman: They don't. That's my point.
>>: So we when were doing ACG work, we only spoke to what we called the user home
resources. I don’t think the network is[inaudible].
>> Serge Egelman: Sure. I mean, we arrived at this by starting with, what are all the
permissions that users are confronted with now, rather than>>: [inaudible].
>> Serge Egelman: Yeah. Absolutely. You're absolutely right. But I mean, so take connecting
to the Internet, you're not going to have a system drawn a button for every app so that it can
get Internet access. That's ridiculous. Instead>>: What about something that is in response to user interaction, so it would go past that
before it [inaudible]?
>>: Why does that belong to the denominator?
>> Serge Egelman: The point is, the harm from that is sufficiently minor if you're not say,
internationally roaming. Most people have pretty good data plans, often unlimited, if there's
an app that's abusing this we can about audit medic mechanism, as we’re going to get to, so
that people can identify the apps that are abusing this even though it's been granted without
consent. And so this sets up this research agenda, which is first looking at well, which
permissions are necessary to grant, which ones aren't. And then of those that are necessary,
how could we design, that aren’t necessary to be explicitly granted, how can we design better
audit mechanisms? And the other side of that is for the requests that do require user
interaction, how can we optimize those? And so in the rest of the talk is current stuff that I've
been working on to improve the audit mechanisms, or what I like to refer to as begging
forgiveness instead of asking for permission because I guess what we're doing here is not asking
for the permission a priority but allowing the user to revisit those decisions.
So the first question in exploring this is: are audit mechanisms likely to be effective? So I
conducted this online survey using people on Mechanical Turk. We didn't specifically screen
based on what smart phone platform they used, instead we did that post doc. We had a free
response, this way they weren’t incentivized to cheat one way or the other, and we just
screened out Android and iOS users, the reason being that both have some existing audit
mechanisms; and we also got too few responses from Windows phone and BlackBerry users to
really include them. We only got like two or three from each.
Anyway, so there are two questions that we hoped to answer through this survey. The first is,
so do users understand that applications running the background can access the same
resources as those running in the foreground? This is important because if you're not aware of
this when a misbehavior occurs on their device, they're likely to just blame whatever app was
running in the foreground not understanding that it could've been anything that was also
running at the same time. And if that's the case, that can set the need for better audit
mechanisms so that they can disambiguate the cause of misbehaviors. The second is that both
these platforms have some existing audit mechanisms built-in. And so we wanted to look at
whether users were aware that these existed.
>>: Why do you think the users [inaudible] aware this audit mechanisms? Average user, I don’t
need to know what’s happening on the platform. This should be the platform like IO has a
Windows phone or [inaudible]. You don’t need to know.
>> Serge Egelman: If something happens on your device you probably want to find what app’s
responsible so you can take corrective action.
>>: Not really in many of the scenarios in the real world. If I own something, I don’t know how
to fix it. I would call someone or find someone who's an expert to fix the problem. It doesn't
necessarily put everything in the user’s hands.
>> Serge Egelman: Sure. But if we can do it in an unobtrusive way so that it’s simpler for the
user, I mean that's still a net win because it's empowering more users, certainly not every user
is going to take advantage of that, but it's empowering more users to make better decisions
about how their data is being used.
>>: It seems like the background, running in background and [inaudible], they sound
orthogonal to me. If I'm using Angry Birds and it's sending SMS messages while I’m running it,
it’s just as annoying as>> Serge Egelman: Absolutely. But if you get-
>>: So why is this question in a survey about audit mechanisms? Why is it [inaudible] in the
background? What does that matter?
>> Serge Egelman: Because there needs to be a way of disambiguating whether it really was
Angry Birds or something else that happened to be running at the same time. I mean, you're
not going to know. You're just going to get some charges on your bill.
>>: So you think that there will be a charge and it says 2:35 PM and the user will remember, oh,
I was running Angry Birds at 2:35 PM.
>> Serge Egelman: SMS is a bad example because the way if they get notified is a distant time
in the future. But say your device just starts vibrating.
>>: So just to be clear, so you're suggesting auditing as an alternative to asking for permission?
>> Serge Egelman: I mean SMS is a bad example because that's not something that's benign.
You would grant permission for SMS, and users are concerned with running up costs.
>>: Could you give like two or three examples of the kind of things [inaudible]?
>> Serge Egelman: Yeah, so like vibrating the device, having it make noise. So these are
annoyances, and the user will want to maybe stop these at the moment that they start if it's an
ongoing event. So the first question was we had a multiple-choice question about what
happens to apps in the background. We had five possible answers from I don't know, it simply
exits, it runs with fewer abilities, it goes into a suspended state and doesn't actually do anything
until you resume it, or runs with the same abilities, which was the correct answer. So 22
percent selected the same abilities, but we see, we do a binomial task and see that if people
were randomly guessing, there's still a 25 percent chance that at least this many would've
guessed this. So what this illustrates is people really have no idea what happens when an app
goes in the background which means that maybe we should have some sort of provenance for
apps when they do misbehaviors on the device.
So, to give one example, data usage on Android. So Android 4.0 added the ability to view the
data usage. So in the settings app there's a panel, there’s a thing here for data usage; if the
user selects that they get a sub panel. And all the way at the bottom here, if they scroll down,
it will show it sorted by individual apps. So they can see which app was responsible for using
most of their data. This has been in deployment for over a year. There haven't been any public
education campaigns about this. So our question was: are Android users aware of this? Have
they discovered this on their own just because it's been on their devices?
So what we did was we asked a question. We had a scenario that you get a hefty bill and it’s
due to data usage, how would you go about finding the cause of this using your device? This
was a free text response. So we just had like an essay box and we wanted to see whether they
would describe the steps involved in getting to the settings app unprompted. And what we
found was almost three quarters of Android 4.0 users were aware of this. They wrote out all
the steps involved in getting to the data usage screen. They were aware of its existence. If we
look at the 95 percent confidence interval, this means as high as 85 percent of users are simply
aware of this because they've had it on their devices. Looking at users of prior versions of
Android as well as iOS who didn't have this ability, their responses were all over the place. A
plurality simply said they would blame whatever app they most recently used, though the next
popular response was they would inspect device settings which indicates that maybe if, on
these other platforms that feature were available, users would've stumbled upon it.
Another 13 percent said that they check app reviews, and while this is probably the most viable
strategy of the ones listed, it's not a guaranteed solution either. And so what we found from
this is users generally expect to find this provenance information within the settings app,
regardless of whether or not it's actually there. So this might be a good place to have
annotations to indicate what app most recently changed a certain setting.
>>: Yeah, so that's exactly the question I was going to ask. So this is not really the auditing
mechanism or, it's more of a, it’s almost like it's even more general than just privacy [inaudible].
It's basically about configurations and configure [inaudible].
>> Serge Egelman: Yeah, I guess that’s what I’ve been saying; provenance, I guess, rather than
auditing.
>>: There may not be any misbehavior, it's just [inaudible] does something, and then later I
think, I want to know how to turn it off.
>> Serge Egelman: Sure.
>>: [inaudible].
>> Serge Egelman: Yes, and it could be used to audit but it's not in and of itself necessarily.
>>: So this setting is very natural for all the configurations to happen.
>> Serge Egelman: Yeah. And so then that brings the second question. So all of this was, well,
a follow up question, all of this was about changes to settings and where do users go to find the
cause of that. What about ongoing events? So maybe the device starts vibrating. Is it
reasonable to expect that they go to the settings app to do that, to figure out what's causing
that? And that brings about notifications. So pretty much every platform has some sort of
notification bar where it has icons, which indicate what apps are doing. So here on Android,
this is the SMS one, and if I were to drag that down, it would expand and show me that I have a
text, and it will even show me some information from that text. So the question is: are users
aware that these exist? Do they understand that they can interact with them to yield more
information about the cause of certain behaviors?
So we followed up this survey with a qualitative interview with 8 different participants
recruited from Craigslist, all were existing Android users, and the gender was pretty well split,
as were the ages. We asked them to perform three different scenarios using a smart phone
that we provided them. So the first the scenario was, we asked them the same scenario that
we gave to our online participants: how would you identify the cause of a data overage? And
just based on that information, half our participants immediately navigated to this screen. So
that sort of validated those results. The remaining four immediately said things like well, I
would use my phone to call my provider or I would look at my bill online. Of course those
strategies won’t actually break it down on a per app basis, and so then after we prompted them
to investigate the device, they too immediately navigated to the settings. So this sort of
validates that this is an intuitive location for that type of information.
In the second scenario, we told them to imagine that the wallpaper has changed on their
device. What would they do to figure this out? Now currently there's no way of auditing this.
But the purpose of this scenario was to see whether they too would go to the settings panel
where such changes are generally made to see if it would yield any information. And of course
all of them did that. There are two different ways, so they can either click the desktop for a
prolonged period which brings up a chooser, or from the settings panel there's this display
settings, they can configure wallpaper from there. So all of them did one of these two things to
see if any information would be there about the cause of such a change, but only one
participant understood that applications could be responsible for these type of changes. The
rest of the participants said well, they expected maybe somebody else borrow their phone and
changed it. They didn't understand that this is something that apps could do unexpectedly.
And finally, we looked at notifications. So we had them read a newsreader on the phone and
while they're in the process of doing that task, we made a notification appear with a little video
icon here, and if they were to expand it, it said that another app was accessing video. We
wanted to observe whether they would notice these and expand them on their own.
>>: [inaudible] Facebook?
>> Serge Egelman: What?
>>: Was it on Facebook?
>> Serge Egelman: No.
>>: Did they say anything about Facebook verses other apps?
>> Serge Egelman: Yeah. And so what we found was, so without prompting, only three of them
investigated this on their own and we suspect that’s because it wasn't their phone, they were in
a public place when they were using it, so a lot of them said that after we prompted them they
knew exactly what it meant, they just didn't really care; it wasn't alarming enough. But when
we prompted them, everyone but one person knew that they could expand that menu to see
what app was responsible.
>>: My question is, did they say anything about like oh, it's okay because it’s Facebook and I
trust Facebook to record my life.
>> Serge Egelman: We didn't actually test that. The whole point was whether they would know
to expand it, whether they would know that there's more information available about what app
is causing that. So from this, we had a couple observations. So the first was that users expect
to find this provenance information in either the system wide settings app or wherever else
those changes can be made. We also found that users are familiar with the notification bar and
know that they can use it to get more information.
So we decided to instrument our phones with some of these, with these two audit mechanisms
to observe whether people would do this in the lab, interact with them when they're put in a
position to motivate them to do so. I guess, I refer to this as adversarial usability testing where
we try and mimic a threat model and see whether users will use the provided mitigations to
counter that threat. So we had 76 Android users recruited from Craigslist. It's skewed towards
male; the average age was mid-30s, so we think this was relatively generalizable. And we
performed a controlled experiment. We randomly assigned people to two different conditions,
we ran several sessions, we had 20 different phones, half of them were flashed with an
experimental condition, the other half were flashed with a control where they didn't have audit
mechanisms, and we didn't actually know what condition people were assigned to until we
pulled the logs off the devices at the end of the experiment.
And the questions we wanted to answer were: when devices misbehave and they're provided
with audit mechanisms, will people use those audit mechanisms to find the cause of the
misbehaviors? We intentionally designed this study to make this task ambiguous in the control
condition by having them interact with applications that all requested the same abilities. So we
specifically looked at the ability to vibrate the device, so we had three different apps that we
interacted with that all had this ability, and three different apps that could change the
wallpaper. And so obviously when this misbehaves, we wanted to make sure that every single
one of these apps was already running in the background so that it's plausible that it could've
been any of them that was causing the misbehavior. And so what we did was, we used to some
deception and we told them that they were there to simply write reviews for apps, that way we
could be assured that they had run several apps in a short amount of time and the apps were
running in the background.
So with regard to the vibration audits, so these were only available in the experimental
condition, when the device started vibrating, since this was an ongoing annoyance, we had a
notification icon appear, and if they were to interact with that, it would attribute that to a
particular app that they had used. In the controlled condition, the only way of identifying the
app would be to launch the task list and individually kill apps until the vibration stops. And we
did observe that behavior. With regard to>>: [inaudible] shortcut?
>> Serge Egelman: Hmm?
>>: So the vibration icons are the shortcut to the system setting?
>> Serge Egelman: No. We actually didn't implement that. So we didn't ask them to stop the
vibration. The only data point that we collected was whether they interacted with it, and then
in their questionnaires, we asked them which app was responsible. But, yeah. That was a
oversight that we talk about in the paper that several people did try and click that button. In
the exit survey we asked them what behavior they would've expected after clicking the button,
and it was about evenly split between launching the application responsible or going to some
central settings panel to disable it. Yes. That's something that we should've instrumented but
didn't.
>>: Is your proposal to make this easier tests?
>> Serge Egelman: Yes. Exactly. I think that having that go to the settings panel is probably the
best way of handling it because if these are going to be drawn by the OS, and if it's a potentially
malicious app, it should be the OS that handles terminating the behavior. With regard to the
wallpaper changes, we simply added annotations to the settings panels. There were two of
them; so in the display settings we added it here, right under the wallpaper, where they would
go to change the wallpaper as well as that chooser that I showed you from the desktop. And so
again, this was only available in the experimental condition. And so actually in the controlled
condition, there was no good way of determining what app is responsible for changing the
wallpaper. In a best case scenario, people could go through and see what apps have the ability
to do so, but that only narrows it down to a subset and doesn't definitively tell them. Yeah.
>>: So you give like three or four different places of different permissions on different events.
But they are currently where working on the best app, I mean Windows has this system lock,
right? So [inaudible] that ever happened.
>> Serge Egelman: Sure.
>>: [inaudible] search. Do you think it make sense in any way, like we should provide a central
place for people to look at the events and permissions because the good thing if I know where
to look for wallpaper change, it's obvious. On the other hand, when there are a [inaudible]
away, even as a [inaudible] user as I am, I find it’s pretty hard to [inaudible] going on there. It is
there a central place I can easily search for it?
>> Serge Egelman: Yeah. Potentially. I think that's a separate question. And I agree
completely. I actually, in the future work that I wanted to discuss, I’ve thought about that a
little bit. I'll try and get to that, I should get to that at the end.
>>: I have a question about iPhone.
>> Serge Egelman: Okay.
>>: If this is not a good time>> Serge Egelman: Yeah, it’s fine.
>>: My question is that iPhone also has, in the setting, it already, so when your iPhone is
[inaudible] information system, right? So it also tells you, do you want this application to send
you notification? Then later you can also go [inaudible]. So is that pretty close to what you're
proposing?
>> Serge Egelman: Yes. Exactly. We actually looked at that a little bit. So we did a previous
experiment. Summer undergrads actually set up a Mechanical Turk task to get people to
upload screenshots of that panel just to observe whether by default people just have that all
enabled or if they've actually done some, if there's any intelligence applied to disabling that on
a per app basis. They were trying to take a machine learning approach to see whether they
could predict what applications would have those things disabled, and all of the classifiers we
used weren't able to find any trends, which was kind of interesting that it seemed that on just,
it all came down to preference. But it was surprising that people actually have taken steps to
disable that.
In our, the first online survey that we did with this that I presented a few slides ago, I did have
some questions about iOS 6, which now has those audit mechanisms for sharing contacts and
photos. And while we found that on average more people were aware of those, we had them
select multiple-choice options of what audits are available on their device, and we included a
whole bunch of ones that didn’t actually exist, on average, people were more likely, the means
were higher for those in among iOS 6 users; but there wasn't a statistically significant
difference, but also only 40 of our 200 some participants were confirmed using iOS 6, so it's
hard to drawing any conclusions whether, I guess those have moved into the public
consciousness that those options are available at this point. I guess it's only been available for
six months.
Anyway, moving on, so in the lab when we examined this we had them do, each misbehavior
we had occurred twice. The first time we referred to it as the implicit misbehavior, so we had
them doing some other task and then we triggered the misbehavior, and that was simply to
observe whether they would've noticed that it was occurring and interact with the audits
unprompted. And that we followed that up with an explicit misbehavior after they completed
all the other tasks, we told them your device is now, one of the apps is causing your device to
vibrate, please identify which one it is. So that was sort of to get both a lower bound and an
upper bound on the effectiveness of these audit mechanisms.
So the full sequence of steps was, so we randomized whether the wallpaper or vibration one
occurred first, so this listing is just as an example, but had the wallpaper one been first, once
they arrived at the lab we told them that they needed to write reviews for three different
drawing apps; and as a first step, all these apps are installed on the phone, they should spend
two or three minutes interacting with each one to become familiar with it. And after they did
all three of those in random order, we then asked them to interact with a desktop widget. And
the reason why we had them interact with a desktop widget was so that we can be assured that
they could view the desktop when changed the background to Justin Bieber. And then after
that, we didn't prompt them about this, we just did it, and then we observed, we asked them to
write the reviews for the apps; and afterwards we read through those reviews to see if they
documented this misbehavior within their app reviews.
We followed this up with the vibration misbehavior, so we had them do the same thing with
the three different timer apps. Again, we chose these apps because they all had the ability to
vibrate the device, and then as the fourth distractor task, we asked them to play with a dice
app. The reason for this was so that we could be assured that they were holding the device in
their hand because it was accelerometer-based, so they had to shake the device to roll the dice,
that way when we would start vibrating incessantly we knew that they would be able to detect
that because it was in their hands. Again, after that, we asked them to write reviews for the
apps to observe whether they would include that misbehavior in any particular app review.
After they did those, all those tasks, we then had them go back and we told them okay, one of
the apps just change the wallpaper on your device, please indicate which app was responsible
and also report the degree of confidence you have in your selection using a five point Likert
scale. We did that again with the vibration, and then we had an exit survey. So regarding the
unprompted vibration, which was represented with a notification icon, we found that about
half of them commented on the vibration in their reviews. So they noticed it; but the
difference with regard to identifying a particular app was significantly different between, so 17
of the 21 who noticed it identified it, whereas only two of the six who noticed it in the control
condition were able to identify it. So of the people who noticed it but didn't identify it, they
wrote in multiple reviews that one app was vibrating, I'm not sure which one was responsible.
Going through the logs that we stored on the devices, every single one of the 17 who correctly
identified the app did so by expanding the notification bar. So this was intuitive to them; they
didn't need to be prompted to do it.
Regarding the explicit misbehavior, so we told them your device is now vibrating, please report
which app is responsible, over 80 percent in the experimental condition were able to correctly
do so, whereas only about 30 percent in the control condition. Of course though, of the
multiple-choices that we provided, the number in the control condition was not statistically
significant, then simply randomly guessing and arriving at the correct one.
And so finally, looking at the strategies that people took, almost everyone opened the
notification bar after the device started vibrating, even the people in the control condition.
They expected that there was some sort of something going on. Maybe there was something in
the notification bar, even though there wasn't. Of the 12 participants who identified the
correct app in the exit survey, 10 of them outright that it was due to random guessing, whereas
two of them killed apps individually until they found the right app. And so finally, of the
incorrect ones, there was a plurality who admitted random guessing because they didn't know
what to do. 29 percent named the last app; and what's interesting both on this misbehavior
and the other one, there was a small minority who simply inferred the abilities based on the UI.
So they said, within this particular app there's a settings panel for controlling a vibration
feature, I don't see this in the other apps, so therefore this must be the one that's responsible.
So they're inferring abilities based on what UI features are available as opposed to what
permissions have been actually granted the application. And then also in the experimental
condition, it was the only one where we would see significant correlation between confidence
and the correct answer and actually having a correct answer.
So finally, wallpaper; we were a little disheartened to observe that very few people actually
documented the change. So we, after we Biebered our participants, only about 15 percent
noted it in their reviews. And afterwards we asked them in the exit resurvey, and a lot more
people had been aware of it but thought maybe it was something they did with the widget they
were interacting with, and so they blamed the thing that they were using at the time that the
misbehavior occurred.
>>: For all these, it wasn’t their fault; it was your fault [inaudible] using, right?
>> Serge Egelman: Yeah.
>>: So could they just think, I’m in a drawing app, maybe this is what drawing apps do, right?
>> Serge Egelman: And so then when we explicitly called their attention this and told them it
was one of the apps they had previously used, please identify it and specify your confidence,
there was a statistically significant improvement and difference in the experimental condition,
but it was only about 30 percent. So of the 12 who correctly identified it, two admitted to
randomly guessing and arriving at the answer, so there were really only 10 who got here
through the audit mechanism, and this was confirmed through the logs. Six of them went to
the settings app and observed the annotation, three saw the wallpaper chooser, and one saw
both. And so the incorrect ones, again, it came down to random guessing, looking at the last
use app. And again, there was this phenomenon where they're inferring abilities based on the
UI features available. So some of them said, you know a particular app, they saw the option to
set wallpaper, they didn't notice this in the other apps, therefore it must be this app. And
again, confidence correlated with correctness.
So from this there were a couple open questions. So we confirmed our previous findings
regarding people intuitively going to the settings app to look at the cause of changes. But the
question is: should that 30 percent in the wallpaper, during the wallpaper misbehavior, should
that be considered a success? While this was a significant improvement over the status quo, 30
percent seems like a low number. I would argue that yes, this is a success. What we've
observed in previous studies is most users don't actually read the direct disclosures from apps
about how data is being used and instead rely on third parties, such as reviews or word-ofmouth. And so actually having a small number of savvy users aware of these features, that
information is then eventually going to get disseminated into reviews and more users will
become aware if that particular application is systematically causing misbehaviors.
And so some of the future questions are: if apps are drawing these notifications, we have
system drawn notifications, we also have app drawn ones, how do we make this difference
intuitive to end-users? Yeah.
>>: [inaudible] should we be designing these prompts to target [inaudible] users in that case if
they’re the ones most likely to use it?
>> Serge Egelman: Potentially. I mean this was one misbehavior, and so it's hard to dry
conclusions from that. I mean, what we also found is, so in the stuff in the settings app with the
data usage, so that had been deployed for a year, right? Up to 85 percent are aware of that, so
maybe if these notifications were in widespread use, maybe in a year that number’s going to be
greatly improved. It's hard to say. But I think this is promising.
>>: I have a question. Who are the participants? [inaudible]?
>> Serge Egelman: No, no, no. We recruited from the general public. I usually avoid recruiting
people from campus. We had them, we collected ages, gender, and also occupation. And we
found that there was actually a pretty good distribution with regard to occupation. There
weren't many in technical jobs, but there was a huge range from a blue collar to white collar,
people with higher education degrees. So I think this is relatively generalizable. One problem
that Cormac sort of hit at is the moral hazard element. So they were using our phones and not
their phones. So maybe some of these misbehaviors they simply didn't have an incentive to
look at because they weren’t the ones who were bearing any lasting harm. So maybe these are
considered lower bounds as a result of that.
So future work, I'm just about done, is that the grand vision for all of this, and I think more
generally with previous work that I've done, is in the usable security space we've gone leaps
and bounds over say, 10 years ago where things were sort of designed without the user in
mind. And now, a lot of systems have been improved by focusing on the user doing user
centered design; but at the same time, all of the experiments that we do sort of cater to the
average user, but no such user actually exists. And I think that there are probably clusters of
behaviors. And I think one of the future things that I'm really interested in working on over the
next few years is trying to cater security mitigations to individuals. And this comes back to your
question about having some thing where all that information is aggregated, well, there's the
habituation problem, and obviously that could cause of some sort of overload, I think less so
than with the warnings because it’s the push versus pull nature of it. People are going there to
seek information out, but there's still some intelligence that we could apply in terms of
displaying the things that we think a particular user finds most important. And also that's the
case with if a lot more, if 50 percent of permissions are replaced with notifications, obviously
that could overwhelm the status bar and create a lot of clutter; but if we know what an
individual cares about, we can prioritize and only show the things that are relevant to them.
And the amount of data that's out there, especially on mobile devices and backing up stuff to
the cloud, makes this really possible.
So to give an example. So Gustavo has a smart phone. He uses it to make public check ins on
Foursquare, anyone can read his posts on Foursquare. He sends tweets that include his
location, but having some sort of middleware or even at the OS level, the smart phone can
infer that he probably doesn't care about location and further prompting him to disclose,
whether he wants to disclose his location is probably just going to habituate him to request that
he might care about more. And so we could do a lot in this space by learning what individual
users care about, or maybe as a first cut, just segmenting the population into a couple discreet
personas and catering, security mitigations around that before we get all involved in a machine
learning approach to>>: [inaudible] contradict this. I like the Facebook and all the social apps that access my photo
album. Doesn’t mean I would like [inaudible] to access [inaudible].
>> Serge Egelman: Yeah. Sure.
>>: This is what really happened to me. [inaudible] app on all my pictures. Once I give that
information it’s what I>> Serge Egelman: Yeah. No. I mean that’s a good example, and I don't think, this isn't
supposed to be, I'm not designing a system around this particular use case, but I mean this is
just to illustrate how we can do customization and make inferences about individual users to
minimize the number of unnecessary security interactions.
So the conclusion is that the current choice architectures are really failing users because there
are too many requests that they don't care about which means that the important ones tend to
go unnoticed. At the same time, we know that users do want some control of how information
is shared with third parties, and by performing iterative human subjects experiments, we can
yield generalizable knowledge to try and give them that.
>>: I have a question. I have the same sense that is basically what you said auditing can be
used beyond [inaudible]. In some sense, I mean this, I'm [inaudible] people, but in some sense,
people care about the usability of fixing a problem more than security and for information
percentage [inaudible]. So have you thought about looking at this applies the auditing. How
could the user resolve an issue problem instead of [inaudible] related issues? Is that something
like>> Serge Egelman: I'm not sure I understand the question.
>>: For instance, you apply, you are looking at auditing because you want to allow users to
reverse permissions.
>> Serge Egelman: Sure.
>>: And when I look at auditing work I say, man, if you can help 50 percent people or 60
percent people to figure out wallpaper problem, I still have the same problem on my wife's
phone, my daughter plays with it. I mean it's a real-world issue. I mean that happens way
more often than information issues.
>> Serge Egelman: Yeah. Absolutely. I mean, I think there's a permission for saying wallpaper.
But I understand your point, and I think that more generally what this points to is, I guess using
same permissions is a misnomer. I think that what we are really concerned with are outcomes,
and unexpected outcomes or undesirable outcomes. And whether that's through
inappropriately granting one permission, a set of permissions, or just unexpected behavior that
had nothing to do with permissions, yeah, you're absolutely right. And that sort of falls under
though this notion of provenance and trying to make it easier to understand for users how we
change or unexpected event occurred generally.
>>: So I have follow up my early question about iphone.
>> Serge Egelman: Okay.
>>: So based on your study here, what would you think that Apple should do in addition to
what they have right now because they seem to already have a mechanism, like in setting you
can kind of undo the things that you've done, you configure your apps.
>> Serge Egelman: Yeah.
>>: [inaudible]. So what do you think we should do in addition based on your findings?
>> Serge Egelman: I think that actually based on some of the earlier research, which I didn't
include in this talk, we've done a couple surveys looking at unexpected outcomes in order to
sort of rank consequences to see what people are most concerned with. And of the four things
that Apple has provided audit mechanisms for, those tend to end up at the bottom of the list in
terms of user concern, and so I think that the way that we are doing it is intuitive to people.
What we’ve found is having it in the settings panel, it does get found by users, but I think that
they're focusing on the wrong things.
>>: Can you give an example?
>>: So just quickly, have you thought about maybe using third parties that might have a view
across multiple users. Example: at [inaudible], imagine the OS in your carrier, which app is
sending it, and your carrier could say, he Serge, this chess timer, this is really>> Serge Egelman: Yeah. I mean I've thought about that. I haven't thought about that more
generally; I've thought about the SMS issue, and I think that that's a larger economics problem.
So on some level, the carriers sort of have an incentive to not do anything about that because
they're getting money. At the same time-
>>: [inaudible], right?
>> Serge Egelman: What?
>>: Can't figure out>> Serge Egelman: The platform>>: That you can send [inaudible], that you can send at this time.
>>: So, replace the carrier with LookOut.
>> Serge Egelman: Sure.
>>: Or, you know, [inaudible] or Symantec or some other antivirus. So you see a new antivirus
model where I let these people sort of see what apps are sending SMS>> Serge Egelman: Yeah. I think that's the model that we are going to now. There used to be
this notion that if anything bad can happen then the system’s insecure and should be
abandoned. But I think what we are, the model that we’re going to now, is there are always,
when new malware comes out or a new vulnerability is discovered there's some sort of, there
are some expected losses, but then the crowd learns of it and adjusts. And so we see that with,
I say malware isn't it really a problem on most of these platforms. Certainly when malware gets
uploaded there are a few users who are impacted, right? But eventually, it gets detected and
gets pulled off the devices. They had a few users are adversely impacted before it gets
detected is bad, but I think we are, we’d be poorly allocating resources if that's the main
problem we are focusing on.
>>: Okay. Thank you.
>>: Does Windows eight, do they seem to be moving towards a storage model for the desktop
as well?
>> Serge Egelman: Yeah.
>>: Are the usability problems the same? Are they easier, harder?
>> Serge Egelman: I haven't actually spent much time looking at them, but I think that most of
this is applicable to that. Certainly with respect to, again, I don't think malware, in detecting
malware is really problem, it's simply empowering users to decide how their information is
shared with third parties; and certainly the central app repositories on the desktop could go
leaps and bounds to improve that.
>>: The notification perks, things like useful place to notify people about vibration, it's intuitive
because vibration’s typically used for notifying people. So imagine that when my phone
vibrates, I think oh, it's probably something I should be notified about.
>> Serge Egelman: Sure.
>>: But it seems not to fit so much with the phone is getting hot, or the time just changed, or
do you just>> Serge Egelman: I don't know. We haven't investigated that. So the time change I don't
think is, I don’t know. I mean, that might be more appropriate for settings. The idea that we
are thinking about is using the notification bar for ongoing things because you have a passive
indicator, and if the user is concerned about something that they notice that's currently
happening, it's just, they just have to glance up.
>>: Right. I was just wondering, will that work on iPhones that are>> Serge Egelman: Yeah. I honestly don't know. And I think that's an interesting test case.
>>: All right. Any last questions? We do have time for more I think, but we can also
[inaudible]. Okay. Let's thanks Serge.
>> Serge Egelman: Thank you.