WalkSafe: A Pedestrian Safety App
Presented by,
Ramya Deepa Palle
CS 541
Paper Overview:

Summarizes a historical view on a conference about mobile sensors
development

Discuss about path toward cognitive phones

Explains few phone applications based on mobile sensors

Discuss problems and solution while analyzing Big Sensor Data
Sensors in mobile:

Mobile phone or smartphone becomes the most important communication device in people lives.

A smartphone is a mobile phone offering advanced capabilities, often with PC-like functionality


Hardware (Apple iPhone 3GS as an example)

CPU at 600MHz, 256MB of RAM

16GB or 32GB of flash ROM

Wireless: 3G/2G, WiFi, Bluetooth

Sensors: camera, acceleration, proximity, light
Functionalities

Communication

News & Information

Socializing

Gaming

Schedule Management etc.
Embedded sensors to phone as a sensor

The view moved away from traditional embedded sensors to phone as a
sensor.
Small history:

In early 2007 Nokia released a phone with an embedded accelerometer.

They did not mention about this in N95 because it was only there for video
stabilization and photo orientation

When Nokia’s peter Boda (the SensorPlanet leader) visited Dart-mouth
college, he casually mentioned that the N(% included an embedded
accelerometer.

This, along with the phone’s GPS sensors, led researchers to study and
develop new phone-based sensing applications, such as the CenceMe
app.
CenceMe

CenceMe, the first social sensing application running on mobile phones.

Sensor application


Accelerometer

Microphone

GPS

Bluetooth
Track user information

Physical activity


Social interaction


Sitting, walking, running or driving
Public conversation
Share using social networking applications

Ex. Facebook
CenceMe App Analysis

Data pushed to servers for analysis

Reconstruct status of users

Phone Classifiers:

Audio - Fourier transform to determine talking pattern


Understand behavior, mood, and health
Activity - Accelerometer data to determine if user is sitting, standing, walking, or
running
CenceMe App Analysis Cont…

Backend Classifiers:

Conversation



Social Context

Neighborhood conditions - are there CenceMe friends around?

Social status - combination of classifiers

Partying

Dancing
Mobility Mode


combines multiple audio primitives to determine whether someone is in a conversation, which may contain pauses
GPS to determine if user is traveling in a vehicle or not
Am I Hot

Nerdy - often alone and spends time in locations such as libraries

Party animal - often at parties and often near others

Cultured - often visits theaters/museums

Healthy - often walking, jogging, cycling, or at the gym

Greeny - low environmental impact - not typically in a car and often walks, cycles, or runs
Smart phone sensing:

Today’s top-end smartphones come with 1.4-GHz quad-core processors
and a growing set of inexpensive yet powerful embedded sensors.

They include:

An accelerometer, a digital compass, a gyroscope, a GPS, quad
microphones, dual cameras, near-field communication, a barometer, and
light, proximity, and temperature sensors.

They also have multiple radios for body, local, and wide area
communications; 64 Gbytes of storage; and the touchscreen.
BeWell: A Mobile Health App

The BeWell app continuously tracks user behaviors along three key health dimensions without requiring
any user input — the user simply downloads the app and uses the phone as usual.

Monitor and promote

Physical and emotional well-being

Persuasive feed back techniques

Tracks along three dimensions


Physical activity

Social interaction

Sleep duration
Weighted scorecard

Between 0 to 100

Ex. 8 Hrs/day sleep is score 100
BeWell : Health App Cont…


Physical activity tracking

Walking, stationary or running

Daily metabolic equivalent of task value

Centers for Disease Control and Prevention guidelines
Sleep tracking

Over 24 hrs period

No phone interaction

Daily bed time activity


Recharging phone

Keeping at same place longer time
Guidelines by National Sleep Foundation
BeWell : Health App Cont…


Social Isolation tracking

Total speech time during day

Phone microphone usage

No recordings

Social apps presence on the phone and usage

Low score linking to depression
Work as stand-alone and interwork with cloud
BeWell : Health App Cont…

Persuasive feedback techniques

Animated aquatic ecosystem as wallpaper

Orange clown fish


Blue clown fish


Reflects user’s social interaction
Ocean’s ambient lighting


Reflects user’s activity
Reflects user’s sleep duration
Score history accessibility
Ambient Display:
BeWell+

Store data on clouds

Share with friends on social networks

Compare with others

Targeted messages

Challenges

Low energy consumption

More accurate classification

Population Diversity Problem
BeWell : Big Sensor Data

No raw data to server

Upload features, inferences, scores and usability data

20MB data traffic per day per user

Population Diversity Problem


Walking patterns changes for a child and an adult
Community Similarity Networks (CSN)

Personalized model per user

Measure similarity between people
WalkSafe : Pedestrian Safety App

Research in social science

Safety app for pedestrians crossing roads while on phone

Back camera to detect vehicles

Machine learning algos

Difficulty

Minimum battery utilization


During active calls only
Trained model

Decision tree
WalkSafe: Pedestrian Safety App

A mobile-phone user deep in conversation while
crossing a street is generally more at risk than other
pedestrians not engaged in such behavior.

WalkSafe uses the smartphone’s back camera to
detect vehicles approaching the user, alerting the user
of any potentially unsafe situations.
Algorithms used:

More specifically, WalkSafe uses machine-learning algorithms
implemented on the phone to detect moving vehicles. It also exploits
phone APIs to save energy by running the vehicle-detection algorithm only
during active calls.

The WalkSafe app offers real-time detection of the front and back views of
cars, noting when a car is approaching or moving away from the user,
respectively . It alerts the user using sound and phone vibration.
Image recognition:

The core WalkSafe car-detection
technology is based on imagerecognition algorithms. Image
recognition is a computationally
intensive process that, if not
carefully designed, can easily
drain the smartphone’s
computational resources and
batteries.
Continued……

To address this, WalkSafe bases its vehicle recognition process on a model
that’s first trained offine and then uploaded to the phone and used for
online vehicle recognition, which runs automatically whenever there’s an
ongoing phone call.

WalkSafe activates the smartphone’s camera and captures video of the
surroundings. Each video frame is preprocessed to compensate for the
phone tilt and illumination variations, and is then analyzed by the decision
tree model built during the offline training phase. If the decision tree
detects a car in the picture, it triggers an alert to warn the user of possible
danger.
NeuralPhone: A Brain-to-Smartphone
Interface

There’s a growing interest in new
handsfree interfaces for
smartphones based on voice and
face recognition systems.

They developed the EyePhone,
which lets the user select and
activate applications with the blink
of an eye.1 We then wondered if a
thought could also drive a
smartphone application—and it
turns out it can.
NeuralPhone : A brain to smartphone
interface

Hands free interfaces for smart phone

Thought driving smart phone app

EEG headsets

Brain controlled address book dialing

P300
Summary

Towards cognitive phones

Pushing intelligence towards phone

Robust classification techniques

Mobile sensors to improve life conditions
Queries?
If time permits:

A small video based on how accelerometer inside the cell phone are
made and How a Smartphone Knows Up from Down.

http://www.youtube.com/watch?v=OjwLnAZoDFE