Introduction to Mobile Sensing
with Smartphones
Uichin Lee
KSE 801
Nov. 16, 2011
iPhone 4 - Sensors
Applications
• Transportation
– Traffic conditions (MIT VTrack, Nokia/Berkeley
Mobile Millennium)
• Social Networking
– Sensing presence (Dartmouth CenceMe)
• Environmental Monitoring
– Measuring pollution (UCLA PIER)
• Health and Well Being
– Promoting personal fitness (UbiFit Garden)
Citysense
MacroSense
CabSense
Eco-system Players
• Multiple vendors
– Apple AppStore
– Android Market
– Microsoft Mobile Marketplace
• Developers
–
–
–
–
Startups
Academia
Small Research laboratories
Individuals
• Critical mass of users
Scale of Mobile Sensing
Sensing Paradigm
• Participatory: active sensor data collection by users
– Example: managing garbage cans by taking photos
– Advantages: supports complex operations
– Challenges:
• Quality of data is dependent on participants
• Opportunistic: automated sensor data collection
– Example: collecting location traces from users
– Advantages: lowers burden placed on the user
– Challenges:
• Technically hard to build – people underutilized
• Phone context problem (dynamic environments)
INFORM, SHARE,
PERSUASION
LEARN
SENSE
Mobile Sensing Architecture
Mobile Computing Cloud
Sense
INFORM, SHARE,
PERSUASION
LEARN
SENSE
• Programmability
– Managing smartphone sensors with system APIs
– Challenges: fine-grained control of sensors, portability
• Continuous sensing
– Resource demanding (e.g., computation, battery)
– Energy efficient algorithms
– Trade-off between accuracy and energy consumption
• Phone context
– Dynamic environments affect sensor data quality
– Some solutions:
• Collaborative multi-phone inference
• Admission controls for removing noisy data
Learn
INFORM, SHARE,
PERSUASION
LEARN
SENSE
• Integrating sensor data
– Data mining and statistical analysis
• Learning algorithms
– Supervised: data are hand-labeled (e.g., cooking,
driving)
– Semi-supervised: some of the data are labeled
– Unsupervised: none of the data are labeled
• Human behavior and context modeling
• Activity classification
• Mobility pattern analysis (place logging)
• Noise mapping in urban environments
Learn: Scaling Models
INFORM, SHARE,
PERSUASION
LEARN
SENSE
• Scaling model to everyday uses
– Dynamic environments; personal differences
– Large scale deployment (e.g., millions of people)
• Models must be adaptive and incorporate people into the
process
• If possible, exploit social networks (community guided
learning) to improve data classification and solutions
• Challenges:
– Lack of common machine learning toolkits for smartphones
– Lack of large-scale public data sets
– Lack of public repository for sharing data sets, code, and tools
Inform, Share, Persuasion
INFORM, SHARE,
PERSUASION
LEARN
SENSE
• Sharing
– Data visualization, community awareness, and social
networks
• Personalized services
– Profile user preferences, recommendations, persuasion
• Persuasive technology – systems that provide tailored
feedback with the goal of changing user’s behavior
– Motivation to change human behavior (e.g., healthcare,
environmental awareness)
– Methods: games, competitions, goal setting
– Interdisciplinary research combining behavioral and social
psychology with computer science
Privacy Issues
• Respecting the privacy of the user is the most
fundamental responsibility of a mobile sensing
system
• Current solutions
– Cryptography
– Privacy-preserving data mining
– Processing data locally versus cloud services
– Group sensing applications is based on user
membership and/or trust relationships
Privacy Issues
• Reconstruction type attacks
– Reverse engineering collected data to obtain invasive
information
• Second-hand smoke problem
– How can the privacy of third parties be effectively
protected when other people wearing sensors are
nearby?
– How can mismatched privacy policies be managed
when two different people are close enough to each
other for their sensors to collect information?
• Stronger techniques for protecting people’s
privacy are needed
Understanding Smartphone Sensors:
accelerometer, compass, gyroscope,
location, etc
Smart Phone/Pad Sensors
Nexus One
Nexus S
iPhone4
Samsung
Galaxy S
HTC
Incredible
Galaxy Tab/
iPad2
Accelerometer
O
O
O
O
O
O
Magnetometer
O
O
O
O
O
O
O
O
?
Gyroscope
O
Light
O
O
O
O
O
O
Proximity
O
O
O
O
O
O
Camera
O
O
O
O
O
O
Voice
O
O
O
O
O
O
GPS
O
O
O
O
O
O
Accelerometer
Mass on spring
1g
-1g
Gravity
1g = 9.8m/s2
Free Fall
Linear Acceleration
Linear
Acceleration
plus gravity
Compass
• Magnetic field sensor (magnetometer)
Z
Y
Y
X
X
Z
Geographic
north
Magnetic
north
Horizontal
Gravity
3-Axis Compass?
Magnetic declination
Magnetic
field
vector
Magnetic inclination
Orientation: Why Both Sensors?
• Two vectors are required to fix its orientation!
(i.e., gravity and magnetic field vectors)
Tutorial: http://cache.freescale.com/files/sensors/doc/app_note/AN4248.pdf
Gyroscope
• Angular velocity sensor
– Coriolis effect – “fictitious force” that acts upon a freely moving object as
observed from a rotating frame of reference
Accelerometer vs. Gyroscope
• Accelerometer
– Senses linear movement, but worse rotations, good for tilt
detection,
– Does not know difference between gravity and linear
movement
• Shaking, jitter can be filtered out, but the delay is added
• Gyroscope
– Measure all types of rotations
– Not movement
– Does not amplify hand jitter
• A+G = both rotation and movement tracking possible
Other Sensors
• Light: Ambient light level in SI lux units
• Proximity: distance measured in centimeters
(sometimes binary near-far)
• Temperature
• Pressure
• Barometer, etc…
Global Positioning System (GPS)
•
•
•
•
27 satellite constellation
Powered by solar energy
Originally developed for US military
Each carries a 4 rubidium atomic clocks
– locally averaged to maintain accuracy
– updated daily by US Air Force
• Ground control
– Satellites are precisely synchronized with each other
• The orbits are arranged so that at any time, anywhere on Earth,
there are at least four satellites "visible" in the sky.
• A GPS receiver's job is to locate three or more of these satellites,
figure out the distance to each, and use this information to
deduce its own location. This operation is based on a
mathematical principle called tri-lateration
Tri-lateration: GPS, Cell-tower
• Imagine you are somewhere in
Korea and you are TOTALLY lost -for whatever reason, you have
absolutely no clue where you are.
• You find a friendly local and ask,
"Where am I?" He says, "You are
70 km miles from 대전“
• You ask somebody else where you
are, and she says, "You are 60 km
from 대구"
• Now you have two circles that
intersect. You now know that you
must be at one of these two
intersection points.
• If a third person tells you that you
are 100 km from 광주, you can
eliminate one of the possibilities.
You now know exactly where you
are
70km
60km
100km
Assisted-GPS: GPS + Network
• Reduce satellite search
space by focusing on
where the signal is
expected to be
• Other assistance data
from cellular nets
–
–
–
–
Time sync
Frequency
Visible satellites
Local oscillator, etc..
• MS-based vs. assisted
GPS
GPS Errors (in Smartphones)
• In-vehicle signal attenuation
• Smartphone’s inferior antenna
(worse!)
– PND uses Spiral helix; Microstrip
antenna vs. Galaxy S (single
wire)
• low GPS reading under high
speed environments
– 4800bps (600 B/s)
– http://www.hadaller.com/dave/
research/papers/MitigatingGPS
Error-UWTechReport08.pdf
Galaxy S
GPS Antenna
V.S.
PND GPS antennas
Network Positioning Method
• Cell-tower localization with tri-lateration
Cell-tower
Cell-tower
Cell-tower
Wi-Fi Positioning System
• Fingerprinting (e.g., RADAR, Skyhook)
• Training phase (building a fingerprint table): for each location, collect
signal strength samples from towers, and keep the average for each
location
RSSI: Received Signal
Strength Indicator
RSSI (x, y, z) = (-20, -10, -15)
(-15, -12, 18)
…………
 L1=avg(x, y, z) = (xx, yy, zz)
RSSI (x, y, z) = (-21, -40, -18)
(-16, -42, 12)
…………
 L2=avg(x, y, z) = (xx’, yy’, zz’)
• Positioning phase:
– Calculate the distance in signal strength space between the measured signal
strength and the fingerprint DB
– Select k fingers with the smallest distance, and use arithmetic average as the
estimated location
Mobile Sensing Applications
Pothole Patrol
• Acceleration data gathering from vehicles (geo-tagged)
• Simple data processing to detect a pothole, and statistical
processing (clustering) for accurate detection
Smooth Road
Pothole
The Pothole Patrol: Using a Mobile Sensor Network for Road Surface Monitoring, Eriksson et al, MobiSys, 2008
Community Awareness:
Health and Wellness
• Personal environmental impact report (PIER) on
“health and wellness”
• Participants use mobile phones to gather location data
and web services to aggregate and interpret the
assembled information (e.g., air pollution, CO2
emission, fast food exposure)
"Sensing Pollution without Pollution-Sensors”
Existing
Infrastructure
Annotation
/Inferences
Scientific
Models
Activity Classification
e.g., staying, walking, driving
Air pollution CO2
exposure emissions
(PM 2.5)
Fast food
exposure
Tracklog format
GIS Data Annotation
e.g. weather, traffic
Weather, traffic data
Impact and Exposure
Calculation
School,hospital,fast
food restaurant
locations
User profile
Data Aggregation
PEIR, the Personal Environmental Impact Report as a Platform
for Participatory Sensing Systems Research, Mun et al., Mobisys 2009
SoundSense
Admission Control
Acoustic Features
Decision Tree Classifier
Markov Model Recognizer
Voice
Analysis
Music
Analysis
Ambient
Sound
Learning
Social Sensing with Twitter
Event detection from twitter
detect an
earthquake
search and
classify them into
positive class
some users posts
“earthquake
right now!!”
Object detection in
ubiquitous environments
detect an
earthquake
Probabilistic model
Probabilistic model
values
Classifier
tweets
・・・ ・・・
・・・ ・・・
・・・
observation by twitter users
some earthquake
sensors
responses
positive value
observation by sensors
earthquake
target event occurrence
target object
Earthquake shakes Twitter users: real-time event detection by social sensors, Takeshi et al, WWW 2010
Summary
Mobile Computing Cloud
LEARN
SENSE
Mobile Sensing Architecture
INFORM, SHARE,
PERSUASION