Wearable Computing : Present
and Future
Swadhin Pradhan
Reading Group Presentation
Business
• In 2013, investors poured $458 million into 49
wearable company deal ( CB Insights )
• $50 Billion Industry by 2017 !! ( Credit Susse )
• Major tech companies like Apple, Google,
Samsung and Intel investing heavily in
wearables, with non-tech giants like Nike,
Under Armour, Adidas, lululemon etc.
Popularity Example
• Pebble
• Kickstarter Campaign
• Seeks : $100K
• Raises : $10+ Million
Factors in Wearable Tech Today
•
•
•
•
•
•
•
Faster and Cheaper Hardware
Cloud Storage
Location Data
Quantified Self Activity
Gaming Industry
Visual & Voice Technology
User Experience
State-of-the-art
• Devices
• Inputs
• Applications
• Algorithms
Devices
Audio-Video Capturing Devices
Logging Life Events
Google Glass (Smart Glasses)
Narrative Clip (First Person Camera)
Autographer (First Person Camera
with Sensors and GPS)
Kapture (Autonomic Audio Capture)
Body Data Gathering Devices
Monitoring Health Status
Jawbone (Activity Monitor)
Fitbit (Fitness Tracker)
Pebble (Smart Wrist Watch)
Emotiv (Brain Activity Tracker –EEG)
Muse (Brain Monitor)
OmSignal (Smart Shirt)
Sensoria (Smart Socks)
FootLogger (Shoe Sole for Fitness
Tracking)
Mimo (Baby Status Monitoring)
MC10 (Flexible wearable sensors
for health monitoring)
Gesture Recognizing Devices
Myo (Muscle Activity Tracker)
SmartyRing (Smart Ring)
Fin (Another Smart Ring)
Ring GINA
(MobiCase 2013) for
3D gestures
Scanadu (Health – heart rate – ECG
– Blood Pressure etc. Checker)
SkinPut (MSR)
Wearables discussed in Research
Papers
New Wearables – New Applications
• BodyScope: A Wearable Acoustic Sensor
for Activity Recognition (UbiComp 2012)
New Wearables – New Applications
Detecting Cocaine Usage through Wearable ECG Sensor (
UbiComp 2013)
New Wearables – New Applications
• An Amulet for Trustworthy Wearable mHealth
(HotMobile 2012)
New Wearables – New Applications
• Wearable Smartphone: Wearable Hybrid
Framework for Hand and Foot Gesture
Interaction on Smartphone (CVF, ICCV 2013)
New Wearables – New Applications
• An Interactive Belt Worn Badge (CHI 2012)
New Wearables – New Applications
• Wearable Device for Visualizing Knee
Rehabilitation Exercises (CHI 2013)
And Many More ……
Classification of Wearable Devices
(Function Wise)
• Life Logger
• Gesture Recognizers
• Entertainer
– Video
– Gaming
• Assistant
– For Chore Jobs
– For Creative Jobs
– For Emergency Jobs
Classification of Wearable Devices
(Creation Wise)
• Replacing Daily Wearables with Smarter
Alternatives
– Watches, Shirts, Shoes, Socks etc.
• Creating New Wearables
– Armband, Headband, Shirt Clippers etc.
Inputs
Ambient Sensors
• Accelerometer, Gyroscope, Magnetometer,
Barometer, Thermometer, Relative Humidity
Sensor, Light Sensor etc.
Accelerometer
Gyroscope
Physical Activity Sensor
• MC10 Sensor, EEG sensor etc.
Interaxon Muse EEG
reader
MC10 Sensor
Traditional Inputs/Capture
• Audio(microphone), Video, Photos (camera –
head mounted/ body-attached/ smartphone)
...
• Wi-Fi, Cellular, GPS …
Applications
Life Logging (Augmented Memory)
• Google Glass
• Narrative Clip Syncing with cloud
• Getting summary from the video or audio or
images – an important problem !!
• Automatic diary creation for the users or
taking notes for the forgetful users is
important.
Activity Tracking/Monitoring
• Calorie Used (Wireless Health ‘13 paper used
to guess correctly with accelerometer)
• Sleep Pattern
• Steps walked
• Detecting Eye Contact using Wearable EyeTracking Glasses (UbiComp 2012)
• Wearable Activity Recognition for Dogs !!
(UbiComp 2013)
Healthcare
• Monitors different vitals of users and help
them to take informed decisions.
– Calory Count using multi-modal Wearable Sensors
(Wireless Health ’12, Hail Kalnatarian et. al.)
– SpiroSmart: Using a Microphone to Measure Lung
Function on a Mobile Phone (UbiComp 2012)
• Emergency Patient observation and
immediate healthcare notification.
– Cognitive Assistance through Wearables
(Offloading through Cloudlets) [Kiryong Ha et. al.]
Gesture Recognizing
• Free from Gesture used for authentication
(MobiSys 2014 – MPI,Rutgers)
• Identifying Emotions Expressed by Mobile
Users through 2D Surface and 3D Motion
Gestures (UbiComp 2012)
• Unobtrusively Wearable Sensor Suite for
Inferring the Onset, Causality, and
Consequences of Stress in the Field (Sensys
2013)
Assistant
• Anywhere, Anytime Information and
Communication
• Sensor-Assisted Facial Recognition: An
Enhanced Bio-metric Authentication System
for Smartphones (MobiSys 2014) [Trick :
Relative Position estimation using sensors]
• Navigation using multimodal sensors
(NaviComf, PerCom 2012)
• A Smartphone-Based Obstacle Detection and Classification
System for Assisting Visually Impaired People (CVF, ICCV 2013)
Assistant to the special people
• Geometric Layout Analysis in a Wearable Reading Device for
the Blind and Visually Impaired (MobiCase 13)
• Parent-Driven Wearable Cameras for Autism Support,
CMU, UbiComp poster
Augmented Reality
• Google Glass like view which adds layer of
virtual view to normal view.
• After integrating gesture recognition and voice
commands, augmented reality can impact
retail industry, social networks, and gaming
industry.
Algorithms
General Flow
• Feature Selection and Extraction (PCA etc.)
• Noise removal and Smoothing (Local
Averaging, DTW etc.)
• Peak Detection and Filtering (Butterfly
Filtering etc.)
• Unsupervised (K-Means, EM Maximization
etc.) and Supervised (SVM, LDA etc.)
Needs Better Sampling Algorithms
• Wireless Health 2012 paper on better
sampling for Body Area Networks (Goudar et.
al.)
• How much to sample ? When ? What is the
accuracy needed ?
• Should it be application based or activity
based ?
Needs Better Robustness for
Context Awareness/ Activity Recog
• Wireless Health 2012 paper tries to find
bound of Dynamic Time Wrapping Technique
to perform moderately (Nimish Kale et. Al.)
• Combination of Wearable Sensor Data and
Physiological data to estimate calorie count
(Wireless Health 2013) [Marco Altini et. al.]
Needs Customized Machine
Learning Algorithms
• Unsupervised Activity Clustering using Single
body Sensor and estimating energy
consumption (Wireless Health 2013) – Similar
Activity Clusters have similar regression
models.
• Energy Efficient HMM and KNN for embedded
classifiers (Dawud Gordon et. al.)
• Google’s Nine Level Neural Network to read
Road Signs, which actually breaks 99.99%
Captcha !!
Needs Customized Machine
Learning Algorithms
• PhotoOCR: Reading Text in Uncontrolled
Conditions (ICCV 2013)
• Motion Primitive-Based Human Activity
Recognition Using a Bag-of-Features Approach
(IHI 2012)
Interesting Marraiges
Wearable Computing and Smartphone
• Most Common and Obvious
• Hub, Connector, and Storage
• Bluetooth, Wi-Fi Direct, NFC ( Audio NFC (Dhawni,
SIGCOMM ‘13) & Visible Light (Hotnets 2013) !! )
Wearable Computing and IoT
Wearable Computing and Smart
Home
Wearable Computing and Gaming
• Facebook buys Oculus Rift to give user a
virtual social gaming experience …
• Microsoft buys Osterhout Design Group in San
Francisco, which creates virtual gaming
environments.
• Wearable computing can make a daily skype
or phone call a direct virtual interaction
experience !!
Wearable Computing and Search
• Google Acquires Nest
• Physical Graph - Web Graph – Knowledge
Graph
• Better and personalized results
• Closer to the actual Information Need
Wearable Computing and Social
• Physical Graph – Facebook Graph
• Fusing Mobile, Sensor, and Social Data To Fully
Enable Context-Aware Computing (HotMobile
2010)
• Socio-Technical Network Analysis from
Wearable Interactions (Katayoun Farrahi et.
al.)
Multiple Wearables in sync
MIThrill (MIT Media Lab)
Sixth Sense ( An Old One ?)
Oculus Rift and Myo
Google Glass and Muse
• Muse rejected Google’s buy-out offer 
• Muse will read your mind and Google Glass
will show you content accordingly.
– Show videos to appease query or release tension
( much better than HotMobile 2014 paper
QuiltView which only shows videos for a given
query )
Concerns
Privacy
• Expectation and Purpose: Understanding
Users’ Mental Models of Mobile App Privacy
through Crowdsourcing (Privacy depends on
Context, UbiComp 2012)
• Give whole level characteristics to the service
provider not each user level specific
information.
Security
• Wearable Device can be hacked and attacked
wirelessly. Patients may die.
• Spoofing and altering are dangerous
phenomenon which can actually derail the
whole purpose. May create panic.
• Side channel attack through power trace
analysis is possible.
Energy
• Less is More: Energy-Efficient Mobile Sensing
with SenseLess (MobiHeld 2012)
• Power Constrained Sensor Sample Selection
for Improved Form Factor and Lifetime in
Localized BANs (Wireless Health 2012)
• Power will come from human body energy !!
(Thad Starner, IBM Systems Journal)
• Energy-Efficient Continuous Activity Recognition on
Mobile Phones ( Your Activity will define Model
(Energy <->Classification), ISWC 2012)
Intrusion
• Too much personalization or assistance will
repel users
• Users will be overwhelmed by the huge
amount of data and can easily be panicked by
misinterpreting any vital health data
• May curb creativity and reduce recall rate
New Ideas
Automatic Text Tagging with
Emotions (Google Glass + Muse +
Jawbone)
• Each story can be automatically tagged with
emotions by tracking the eye movement,
sounds, activity etc.
• User can easily search according to his mood
or can be automatically given reading
suggestions of a particular position of a book
depending upon his mood !!
AutoRemember : A Google for Daily
Things (Tile + Google Glass +
Smartphone)
• We forget. We can’t find important docs
when we need
• Can we use our mobile, our sensors, google
glass, rfids, tiles or qr codes to automatically
keeping track of our things?
• This system will also automatically categorize
the things for us; sometimes also
opportunistically scan some docs to store
these in cloud for ubiquitous access.
MindDoctor : Body Language
Detection - Mood Inference - Mental
health Suggestion
• By intelligently and energy efficiently sensing
our activities and context, a system can easily
infer our mood and can set the color &
background music of my smart home
accordingly.
• The system can suggest some exercises like
deep breathing when we are really tensed.
• The system can also detect our body language
or postures, and make suggestions according to
context – like be confident when in meeting.
Random Thoughts
• Give wearable like benefits using feature
phones for developing countries …
• Communication through visible light (hotnets
2013) or audio (Dhawni, SigComm 2012) or
omnipresent signals in environment (aereo
catches TV signals and SigComm 2013 best
paper BackScatter uses it for powerless
communication) can be leveraged …
What is Next …
Edible Computers
• "I expect to see edible computers pills, which
will act like little medical monitors,
downloading information about your state of
health to a computer you wear.“
– Nicholas Negroponte, MIT Media Lab, 1999
• Motorola Password Pills & Tatoos ..
Planting Computers
(Transhumanism?)