An Architecture for Privacy-Sensitive Ubiquitous Computing Jason I. Hong Computer Science Division University of California, Berkeley Ubicomp Presents New Benefits Advances in wireless networking, sensors, devices – Greater awareness of and interaction with physical world Ubicomp can help in coordination, efficiency, safety Find Friends E911 Incident Command Example Location-enhanced Instant Messenger Instant messaging used by 250m people, 20% growth / yr Clients are moving to mobile devices (Phones, PDAs) – Will be capable of determining your location Potential risks? – Stalking – Constant surveillance by boss – Location-based spam Ubicomp Presents New Privacy Risks These ubicomp systems could also be used to: – Commit fraud – Draw embarrassing or inaccurate inferences – Discriminate against users Everyday Risks Friends, Family Extreme Risks Employers Government _________________________________ _________________________________ __________________________ Over-protection Social obligations Embarrassment Over-monitoring Discrimination Reputation Civil liberties Stalkers, Muggers _________________________________ Well-being Personal safety Ubicomp Privacy is a Serious Concern “[It] could tell when you were in the bathroom, when you left the unit, and how long and where you ate your lunch. EXACTLY what you are afraid of.” - allnurses.com What’s Hard about Ubicomp Privacy? Scope and scale of ubicomp – Past: costly to collect, store, and use info – Future: everywhere, always on, far easier to collect data – New Domains: family, marketplace, workplace, healthcare… Many issues must be addressed simultaneously – Social and Organizational, Interaction Design, Technical Problem Hard to Create Privacy-sensitive Ubicomp Apps Hard to analyze privacy – What concerns do people have? – How to design effective user interfaces for privacy? Hard to implement privacy-sensitive systems – What are the basic abstractions? – What are the privacy mechanisms? Solution Confab Privacy Toolkit Informed by End-User Needs Hard to analyze privacy – Analysis of end-user needs for ubicomp privacy Interviews, surveys, postings on message boards – Analysis of interaction design for ubicomp privacy Pitfalls in designing user interfaces for privacy Hard to implement privacy-sensitive systems – Confab toolkit for privacy-sensitive ubicomp apps Capture, processing, and presentation of personal info – Evaluation thru building three apps and user studies Location-enhanced messenger, location-enhanced web proxy, emergency response app Outline Motivation End-user Privacy Needs Pitfalls in User Interfaces for Privacy Confab Toolkit for Privacy-Sensitive Ubicomp Applications Built An HCI Perspective on Privacy “The problem, while often couched in terms of privacy, is really one of control. If the computational system is invisible as well as extensive, it becomes hard to know: – what is controlling what people –Empower what is connected to what so they to share: –choose where information is flowing – how it is being used • the right information can • with the right people or services • at the right time The Origins of Ubiquitous Computing Research at PARC in the Late 1980s Weiser, Gold, Brown Analysis of End-User Privacy Needs Lots of speculation about ubicomp privacy, little data Published Sources – Examined papers describing usage of ubicomp systems – Examined existing and proposed privacy protection laws EU Directive, Location Privacy Act 2001, Wireless Privacy Act 2003 – Theoretical analysis, asymmetric information flows [Ubicomp 2002] Surveys and Interviews – Analyzed survey data of 130 people on ubicomp privacy prefs – Interviewed 20 people on location-based services Existing Systems – Analyzed postings on nurse message board on locator systems Summary of End-User Privacy Needs Clear value proposition Simple and appropriate control and feedback Plausible deniability Alice’s Location Limited retention of data Decentralized control Special exceptions for emergencies Bob’s Location Outline Motivation End-user Privacy Needs Pitfalls in User Interfaces for Privacy Confab Toolkit for Privacy-Sensitive Ubicomp Applications Built Pitfalls in Designing for Privacy What kinds of user interfaces work? What kinds do not? – Analyzed ~40 apps for common user interface mistakes – Pitfalls in Designing for Privacy [PUC 2004] Privacy Pitfalls Obscuring Actual Flow Users should understand what is being disclosed to whom – Many ubicomp systems are “invisible” by default – Systems should provide appropriate visibility Who is querying my location? “Bob will see this request” How often? “Alice has requested your location” Privacy Pitfalls Configuration over Action Designs should not require excessive configuration – Configuration a typical “solution”, but hard to predict right settings – Manage privacy in the actual context of use Privacy Pitfalls Lacking Coarse-Grain Control Fine-grained controls should be secondary, not primary “[T]raveling employees may want their bosses to be able to locate them during the day but not after 5 p.m. Others may want to receive Did I set it right? coupons from coffee shops before 9 do I know? a.m. onHow weekdays but not on This is a lotthey of work… weekends when sleep in. Some may want their friends alerted only when they are within one mile, but not 10 miles.” Protecting the Cellphone User's Right to Hide New York Times, Feb 5 2004 Simple, does exactly what I think it does Privacy Pitfalls Inhibiting Established Practices Designs should not inhibit established social practices “Smart” Answering Machine “Lee has been motionless in a dim place with high ambient sound for the last 45 minutes. Continue with call or leave a message.” Outline Motivation End-user Privacy Needs Pitfalls in User Interfaces for Privacy Confab Toolkit for Privacy-Sensitive Ubicomp Applications Built Confab Toolkit for Privacy-Sensitive Ubicomp Confab for privacy-sensitive ubicomp apps – Cover end-user privacy needs – Avoid pitfalls in user interface design wrt privacy – Provide solid technical foundation for privacy-sensitive ubicomp Presentation Infrastructure Physical / Sensor …but notpresent help choices I might developers process it well to users… …but have control safely not or provide Iover might acquire how info was visibility tothe end-users information privately… acquired or processed A toolkit needs to support all three of these layers – Must capture, store, process, & share in privacy-sensitive manner Past Work Addresses at Most One Layer Presentation P3P, Privacy Mirrors Infrastructure ParcTab System, Context Toolkit Physical / Sensor Cricket Location Beacons, Active Bats Today, building privacy-sensitive apps would have to be done in an ad hoc manner Architectural Requirements Low barrier to entry – Make it simple for programmers, admin, end-users Easy to add or modify app-specific privacy controls Easy for end-users to control and understand Easy for end-users to share info at a comfortable level Confab High-Level Architecture Capture, store, and process personal data on my computer as much as possible (laptops and PDAs) Provide greater control and feedback over sharing Name Loc Source Sources In Operators Logging Check Privacy Tag Personal Data Store On Operators My Computer Out Operators App Invisible Mode Enforce Access Garbage Collect User Interface Periodic Reports Example Built-in Confab Operator Flow Control Goal: Disclose different info to different requestors Conditions – – – – Age of data Requestor Domain Requestor ID Requestor Location – Data Format – Data Type – Current Time Actions – – – – Lower Precision Set (fake value) Invisible (no out data) Interactive – Allow – Hide (data is removed) – Timeout (fake network load) – Deny (forbidden) Outline Motivation End-user Privacy Needs Pitfalls in User Interfaces for Privacy Confab Toolkit for Privacy-Sensitive Ubicomp Physical layer for acquiring location Infrastructure layer Presentation layer Applications Built Physical / Sensor Layer Intel’s Place Lab Location Source Determine location via local database of WiFi Access Points – Unique WiFi MAC Address -> Latitude, Longitude – Periodically update your local copy A –Works indoors and C in urban canyons –Works with encrypted nodes B –No special equipment –Privacy-sensitive –Rides the WiFi wave PlaceLab Data at SF Bay Area SF Bay Area ~60000 Nodes (~4 Megs) PlaceLab Data at UC Berkeley Berkeley Campus ~1000 Nodes Outline Motivation End-user Privacy Needs Pitfalls in User Interfaces for Privacy Confab Toolkit for Privacy-Sensitive Ubicomp Physical layer for acquiring location Infrastructure layer Presentation layer Applications Built Infrastructure Layer Confab’s Built-in MiniGIS Operator People and apps need semantically useful names – “Meet me at 37.875, -122.257” Country Name Region Name City Name ZIP Code Place Name Latitude/Longitude = United States = California = Berkeley = 94709 = Soda Hall = 37.875, -122.257 Preferred MapPoint MiniGIS operator transforms location info locally – Using network-based services would be privacy hole Whittled down to 30 megs from public sources – Places hardest to get, 3 ugrads + me scouring Berkeley Infrastructure Layer Confab’s InfoSpace Data Store InfoSpace like a diary that stores your personal info – Static info (ex. name and phone#) – Dynamic info (ex. current location and activity) Runs on your personal device or on a trusted service – Can choose to expose different parts to people & services Confab Architecture Name PlaceLab Source Loc InfoSpace Data Store Out Operators My Computer • Flow Control • MiniGIS Request Location Messenger Tourguide How to make users aware of and be able to control the flow of personal info? Outline Motivation End-user Privacy Needs Pitfalls in User Interfaces for Privacy Confab Toolkit for Privacy-Sensitive Ubicomp Physical layer for acquiring location Infrastructure layer Presentation layer Applications Built Presentation Layer Observations on Disclosure Prefs Want visibility and control without overwhelming users – IP Address, domain name, current location? Services – Judged mainly by perceived value and risk People – Judged mainly by who is making request “Either I trust someone with my information or I don't.” – Common secondary criteria is time “Work people can know my information during work hours. Home/SO people can know my information always.” Prefs should be set during or after a request Presentation Layer Notification for IM Request from Person Four iterations with seven people – Location-enhanced messenger, location-enhanced tourguide Avoiding the Pitfalls • Actual flow of information • Minimal configuration • Coarse-grain control • Plausible deniability Presentation Layer Notification from Tourguide Service Presentation Layer PlaceBar for Tourguide Service People thought of tourguide as discrete push of info – Ex. Information only sent when link is clicked on PlaceBar for sharing location on per-transaction basis Confab Architecture My Computer PlaceLab Source Name Loc Pull InfoSpace Data Store Location Messenger Push Tourguide How to control what happens to your info once it leaves your InfoSpace? Privacy Tags Digital Rights Management for Privacy – – – – Like adding note to email, “Please don’t forward” Notify address - notify-abc@cs.berkeley.edu Time to live - 5 days Max number of sightings - last 5 sightings of my location Provide libraries for making it easy for app developers Requires non-technical solutions for deployment – Market support thru TrustE, Consumer Reports – Legal support thru data retention laws Implementation #Classes Confab Shared Libs PlaceLab MiniGIS Total 330 230 10 15 575 Lines of Lines of code Comments (SLOC Count) 20000 32000 16000 23000 900 1700 2300 3300 39200 59500 I wrote ~95% of this over ~2.5 years – Uses Java 1.5, Tomcat Web Server, MySql, Jaxen XPath Distributed querying system (3 grads) [Ubicomp 2003] – Ex. Update “location.occupant.age” as people move in and out Two course projects outside Berkeley Outline Motivation Analysis of End-user Privacy Needs Analysis of Interaction Design for Privacy Confab Toolkit for Privacy-Sensitive Ubicomp Applications Built Putting it Together #1 Location-Enhanced Messenger Putting it Together #1 Location-Enhanced Messenger Putting it Together #2 Location-Enhanced Web Proxy Auto-fills location information on existing web sites PageModification URL =http://www.starbucks.com/ txtCity =CityName txtState =RegionCode txtZip =ZIPCode MapQuest Starbucks Putting it Together #2 Location-Enhanced Web Proxy Location-aware web sites – Different content based on your current location Putting it Together #3 Emergency Response Service Field studies and interviews with firefighters [CHI2004] Finding victims in a building – “You bet we’d definitely want that.” – “It would help to know what floor they are on.” But emergencies are rare – How to balance privacy constraints with utility when needed? Putting it Together #3 Emergency Response Service Trusted third party (MedicAlert++ or home server) Loc “ABC” Medic Alert++ “ABC” On Emergency Application Details Location-enhanced Instant Messenger – Uses Hamsam library for cross-platform IM – ~2500 LOCs across 23 classes, about 5 weeks (mostly GUI) – Acquiring location, InfoSpace store (and prefs), location queries, automatic updates, access notifications, MiniGIS + dataset Location-enhanced web proxy – Added ~800 LOCs to existing 800 LOCs, about 1 week – Location queries, automatic updates, MiniGIS + dataset, PlaceBar Emergency Response – ~200 LOC in 2 days (no GUI, just raw client) – Location queries, update both servers, access notifications Confab reduces what would be a lot of duplicated work User Evaluations Ongoing task-based eval with 9 people – Proficient with web and IM, but not computer experts – Location-enhanced messenger, location-enhanced tourguide Can they accomplish basic tasks correctly? – Do they understand the choices? – Can they use the interfaces to make the decisions they want? Is their conceptual model correct? – Does the system work roughly the way they think it does? Do they still have privacy concerns? – Would they want to use it? User Evaluations (The Good) All assumed location information started with them, no third parties involved (even with IM) – Correct for Confab, not always for other systems Options understandable and could make desired choice – Pretty much everyone chose “Just for now” – Only real issue was what others saw on “Ignore for now” These apps fit well in users’ existing comfort zone Request for disclosure options of “work” and “home” Enthusiastic about new possibilities – Checking length of movie lines, restaurant lines, bus lines – Making sure children are safe User Evaluations (The Not So Good) PlaceBar merged too many ideas together – Understandable, but collapses too many features in one place – “Home” and “work” location rather than current place too Some terminology and displays confusing Confab Recap Clear value proposition Simple and appropriate control and feedback – Access notifications and PlaceBar Plausible deniability – Default is “unknown”, can’t tell why Limited retention of data – Privacy tags, automatic deletion of data Decentralized control – PlaceLab source for capturing location info – MiniGis service for processing location info Special exceptions for emergencies Contributions Set of end-user needs for ubicomp privacy Pitfalls in user interfaces for ubicomp privacy [PUC 2004] Confab toolkit for facilitating construction of privacysensitive ubicomp applications [Mobisys 2004] – Introduces idea of privacy protection at physical, infrastructure, and presentation layers – Introduces alternative architecture for ubicomp, doing as much work as possible on end-user’s computer – Greater choice, control, and feedback Evaluation thru building three apps + user tests [DIS2004] Future Work in Ubicomp Design Book on web design patterns – Shopping Carts, Action Buttons – Over 13,000 copies in use – Used in several classes Design patterns for ubicomp? – Faster design cycles? – Higher-quality apps? – Privacy-sensitive systems? DIS 2004 Future Work in Ubicomp Prototyping Developed SATIN toolkit – Ink, interpretation, & zooming – Downloaded 1600+ times Helped develop DENIM – Sketch and “run” web designs – Downloaded 47000+ times CHI Prototyping for ubicomp? – What techniques? – Tools for aiding deployment? Future Work in Ubicomp Evaluation Started WebQuilt Project – Remote Web site usability testing & analysis tool – Downloaded 800+ times WWW10 Evaluating ubicomp apps? – New methods & tools? Ubicomp apps often mobile, remote evaluation tools may work well! Conclusions Confab toolkit for privacy-sensitive apps Privacy just one aspect of my work in ubicomp – Tools / methods for designing, prototyping, and evaluating high-quality ubicomp apps “Use technology correctly to enhance life. It is important that people have a choice in how much information can be disclosed. Then the technology is useful.” Acknowledgements Thanks to: DARPA Expeditions NSF ITR Intel Fellowship Siebel Systems Fellowship PARC Intel Research http://placelab.org John Canny Anind Dey James Landay Scott Lederer Jennifer Ng Bill Schilit Many, many others… Jason I. Hong jasonh@cs.berkeley.edu http://guir.berkeley.edu/confab Backup Slides Hypothesis: The Privacy Hump fears Pessimistic Many legitimate concerns Many alarmist rants “Right” way to deploy? Value proposition? Rules on fair use? time Optimistic Things have settled down Few fears materialized Market, Social, Legal, Tech We get tangible value Evidence: The Privacy Hump “[T]he right to be let alone” “[T]he telephone permitted intrusion… by solicitors, purveyors of inferior music, eavesdropping operators, and even wiretransmitted germs.” Initial ecommerce scares – SSL, and credit card liabilities limited to $50 – Pew Internet study, more experience => trust Stakes higher with ubicomp, let’s do it right – Mistakes could raise the hump for future work – Easy to reduce privacy, hard to add it back in Privacy Metrics? User-perceived privacy metrics – Do they feel in control? – Do they understand who can see what about them? – Can they make choices they want? Without being overwhelmed? Location privacy metrics – Minimal inferences from machine learning algorithms? – Information theoretic, ie sends the minimal amount of data required for a service to work? However, serious limits to these approaches – Can we really measure a civil right? Presentation Layer Access Notifications Evaluations over four iterations with seven people – Location-enhanced messenger, location-enhanced tourguide For most part, worked well – Understood all choices correctly, but “too much text!” Some distinctions in how often information is shared Go t “Giving a GPS location once or twice does not provide enough information for an invasion of privacy… [but] if GPS location is shared every 2 seconds, there is a potential for an invasion of privacy.” Wh Iteration 1 Users’ Conceptual Model Continuous Push Pull Access Notifications Emergency somewhat Response ok here Discrete Find Friend Worked here Tourguide Didn’t work E911 here IOthers continuously requestshare my current my location location I send my location to others with another from me just person this once or service Access Notifications (revised) Reduced text Added info for 1-time vs continuous disclosure PlaceBar for Discrete-Push case Continuous Push Discrete Tourguide E911 Pull PlaceBar for sharing location on per-transaction basis Handling the After Case Who is seeing what about me? Who has seen what about me? Turn it all off Putting it Together #2 Location-Enhanced Web Proxy Service Description default index.html rect tower.html 37,-12 36,-13 Web Site rect soda.html 38,-11 39,-12 … Web Proxy Browser InfoSpac e Diary For location+ sites Page Mods For existing sites Gathering MiniGis Data USGS State Gazetteer – Names in USA – 2m records ~650 megs – States, Cities, Places GEOnet Names Server – Names outside USA – 5.5m records ~700megs – Regions, Cities, Places Whittled down to ~30 megs “Places” hardest to get – Airports & schools useful, lava and quicksand less so – 3 ugrads & I are scouring Berkeley for places (and WiFi too) Cafes, buildings, landmarks Requirements Check Value proposition Limited retention of data Simple and appropriate control and feedback Decentralized architectures Plausible deniability Special exceptions for emergencies Addresses majority of issues with previous systems – Greatly informed by end-user needs – Better interaction design, avoids common problems Stronger technical foundation than previous systems – Protection at physical, infrastructure, and presentation layers – Greater choice, control, and feedback Putting it Together #2 Location-Enhanced Web Proxy Kinkos Fedex Google High-Level Architecture Sources Operators – Run asynchronously – Manage flow of tuples – Update tuples (ex. – Simple location) form of extensibility Source Sources In Operators Name Loc Out Operators App InfoSpace Diary Services Services InfoSpace Diary On Operators Services only) – Runs(localhost on a trusted computer – Ex. GPS -> City of Name – Stores tuples personal info – Ex. Location proxy parts to others – Can exposeweb different Architectural Analysis Prevent – Capture and process personal information locally – PlaceLab, MiniGis – Minimizes risk of mission creep (ex. SSNs) Avoid – Interfaces for feedback and control over personal information – Access Notifications / PlaceBar Detect – Finding problems – Access Notifications – Privacy Tags (processed on requestor’s side) Application Developer Support Want to make it easy for app developers too Low barri Make Easy to a Easy for e Easy to s Extensibility through chainable operators In Operators Out Operators InfoSpace Diary Check Privacy Tag On Operators Invisible Enforce Collect Access Garbage Coalesce Periodic Reports Application Developer Support ConfabClient – Java client-side API for accessing InfoSpaces – add, remove, query Active Properties – Stores and can automatically update values Berkeley, CA localuser.location OnDemandQuery localuser.activity PeriodicQuery localuser.name Busy Static Jason Requirements Check Value proposition Simple and appropriate control and feedback – Access Notifications (pull) and PlaceBar (push) Plausible deniability – No action, “Ignore for now”, and “Never Allow” appear same Limited retention of data – Privacy Tags, Automatic deletion of old data Decentralized architectures – PlaceLab and MiniGis Special exceptions for emergencies Related Work Consumer Privacy Preferences Privacy surveys since 1990s show three groups [Westin] Risk / benefit sweet spot? – Privacy for Safety – Privacy for Convenience Fundamentalists 25% Pragmatists 63% Unconcerned 12% Fair Information Practices Notice - Notice of data collection Choice - Consent over collection Onward Transfer - Consent over secondary use Access - See data about self Security - Reasonable safeguards Data Integrity - Data is accurate Enforcement - Enforcing policies and redress OECD Fair Information Practices Collection Limitation - Limited collection with consent Data Quality - Relevant and up-to-date Purpose Specification - Purpose at time of collection Use Limitation - Restrict use to said purposes Security Safeguards - Reasonable security Openness Principle - Existence of data known Individual Participation - Obtain and correct the data Accountability - Someone accountable Fair Information Practices Comments FIPs meant for governments and corporations – Family, friends, co-workers? Spectrum of apps require different kinds of practices – Commercial apps vs. Firefighter apps vs. National Security apps – App running at home vs. App running at work Notification and Consent impractical in some cases – Cannot always readily notify (ex. traffic monitoring) – Possibly no alternatives (cannot opt out of building security cameras) – Pervasive sensors significantly increases scale Need a framework that considers: – Risks / Benefits, Identifiability, Quality, Quantity, and Scope of data Categorizing Privacy Techniques Prevent Location Support Garbage Collection Wearables Anonymization Pseudonymization Access Control Strategies for Protecting Data P3P Avoid Lowering Precision User Interfaces for Feedback, Notification, and Consent Privacy Tags Privacy Mirrors Detect Logging and Periodic Reports Audits Collection Access Data Lifecycle Second Use Privacy Perspectives Quotes on Privacy “You know it when you lose it” “My own hunch is that Big Brother, if he comes to the United States, will turn out to be not a greedy powerseeker but a relentless bureaucrat obsessed with efficiency” [Vance Packard] Privacy is relatively new concept in society, “ultimately a psychological construct, with malleable ties to specific objective conditions” [Grudin 2001] Why Privacy? Idealistic Reasons UN Universal Declaration of Human Rights Article 12 – "No one shall be subjected to arbitrary interference with his privacy, family, home or correspondence, nor to attacks upon his honour and reputation. Everyone has the right to the protection of the law against such interference or attacks." Hippocratic Oath – "What I may see or hear in the course of the treatment or even outside of the treatment in regard to the life of men, which on no account one must spread abroad, I will keep to myself, holding such things shameful to be spoken about." Why Privacy? Pragmatic Reasons Natural form of protection from others – Identity theft, stalkers, abusive husbands – Government intrusion – Wrong interpretations by others Taylorism – Too easy to start treating others as a cog in a machine Data for one purpose tends to be used for others – Ex. SSNs – Ex. Could place GPS in all cars to eliminate speeding – Is this what we really want? Why Privacy? Cannot Always Reject Technology Oakland nurses successfully rejected active badges – Admin wanted it for efficiency and accountability – People at desk liked it to find people – Nurses hated it because no immediate benefit to them However, nurses could reject only because they had economic upper hand, ie a shortage of nurses As researchers in a democratic society, we should ensure our work promotes democratic ideals Why Privacy? Privacy and Technology, Gary Marx Anonymity important for honesty and risk-taking Confidentiality can improve communication flows – Doctors, lawyers, AIDS American ideal of “starting over” Some information can be used unfairly – Ex. Religious discrimination Mental health and creativity Totalitarian systems lack respect for individuals Why Privacy? Medical Record Risks Insiders who make innocent mistakes and cause accidental disclosure of confidential information Insiders who abuse their record access privileges Insiders who knowingly access information for spite or for profit An unauthorized physical intruder who gains access to information Vengeful employees and outsiders Arguments Against Privacy “I have nothing to hide” Overlooks that there are degrees of privacy – So why close the door when changing clothes? Overlooks civil rights and human dignity – Surveillance gives the impression that an activity is not proper – Surveillance can be a “velvet glove” of repression – Privacy protects us from excessive societal norms [Goffman] Arguments Against Privacy The Transparent Society, by David Brin Openness and accountability are key to a democratic society – The technology is coming… – So let’s opt for complete transparency Problems: – – – – Ignores social power imbalances Ignores spheres of private and public Ignores degrees of transparency Goes completely against human nature Arguments Against Privacy Communitarian Argument Ex. Public safety – HIV testing for newborns – Megan’s laws for sexual predators Communities and Ubicomp? – We do not enough experience with ubicomp to make these decisions yet – Ubicomp can be easily abused if underlying tech not built right