1
Pose Invariant Activity Classification
for Multi-floor Indoor Localization
Saehoon Yi1, Piotr Mirowski2,3, Tin Kam Ho2,4, Vladimir Pavlovic1
1Computer
Science Department, Rutgers University
2Statistics and Learning Research Department, Bell Labs, Alcatel-Lucent
3Now at Microsoft Bing
4Now at IBM Watson
shyi@cs.rutgers.edu
“Mapping while walking”
Outline




Related work
 Indoor localization
 Pedestrian Dead Reckoning
 GraphSLAM
Motivation
 “Mapping while walking”
Methods
 Pose-invariant sensor features
 SVM classification of actions
 HMM temporal smoothing
Results
2
Indoor Localization

Indoor localization
 Various practical use
 Radio Frequency (WiFi, 4G cell) maps
“where am I in this building?”
 Network deployment optimization:
“where should we place that 4G cell in the building?”

No GPS

Smartphone sensors
 Accelerometer
 Gyroscope
 Magnetometer
 Barometer
 WiFi, cell
3
Pedestrian Dead Reckoning



Step detection
on vertical acceleration
xt-1
error at
position
reset
[Steinhoff et al., Pervasive Computing and
Communications 2010]
3-axis orientation
xt+1
[Madgwick et al., Rehabilitation Robotics 2011]

xt-2
xt
Trajectory updates
xt-3
xt-4


step length d, offset angle β
xt-5
Subject to drift
due to orientation error
 Sensor measurement noise
 Magnetic field perturbations
angle offset
beta β
direction
of walk
4
Yaw of the phone
w.r.t. horizontal
X axis
longitude or
human/horizontal X
(“towards East”)
GraphSLAM

Modify trajectory to minimize estimation error
xj
𝒆𝒊𝒋 (𝒙𝒊 , 𝒙𝒋 )


[Grisetti et al., Transportation Systems Magazine 2010]
xj*
Challenges
 Requires landmark detection
 How do we know that two different observations
are actually taken at the same location?
 Hand-placed landmarks, e.g.,: QR code or NFC tag
 Manually installed and maintained
5
𝒛𝒊𝒋 (𝒙𝒊 , 𝒙𝒋 )
zij
xi
Motivation




Detect and provide natural landmark for GraphSLAM
 Stairs and elevators are accurately detected
 They are non moving, distinct structures, which is ideal for landmarks
Classify human activities using a smartphone in the pocket
 Pose invariant features extracted from smartphone sensors
Jointly infer activity and floor information
 Focus on activities that incur floor change
“Mapping while walking”
 Facilitate radio-frequency map building for network engineers
6
Methods



SVM:
Classify activity at each time
point
HMM:
Smoothing SVM activity
classification and jointly
infering floor
Activities






walking
stair up
stair down
stand still
elevator up
elevator down
7
Pose invariant features for IMU sensors

Pose invariant features from A

[Kobayashi et al., ICASSP, 2011]

Invariant to rotation R
8
Statistical features for barometer




Rotation does not affect air
pressure
Fluctuate over time
Depend on weather and
temperature
Detects ascending/descending
air pressure
9
SVM classification


Features are extracted from sliding window of sensor observations
Linear SVM for 6 activity classes
 Fast and storage efficient
 Linear classification


Able to implement real time classification in Android OS
Provide activity probability
 Platt’s scaling algorithm
 Required to obtain HMM observation probability
10
HMM smoothing

6 activities for each floor
 State transit for strong evidence
 Smooth sporadic brief misclassification

Augment inference of activity with floor from Viterbi algorithm

Observation probability


from SVM confidence level(Platt’s scaling)
from mixture of Gaussian
11
HMM smoothing

Transition probability
 Manually design transition probabilities
 Higher probability of transition to the same state
 Floor changes only for stair and elevator
12
Experiment set up


Input data
 Sensor data recorded at 50Hz
Feature extraction
 Sliding window
 IMU sensor features
 Length: 64 frames
 Step size: 35 frames
 Barometer features
 Length: 192 frames

Train data: 10271 seconds of each class repeatedly performed

Test data: 6160 seconds of 12 natural sequence
13
Activity classification result

For SVM,
 Walking is confused to taking stairs
 Standing still is confused to taking elevators
 Leg dynamics are similar
 Air pressure does not change over short period of time
 Each sliding window is considered independently
14
Activity classification result


HMM removes sporadic
misclassification between walking
and taking stairs
Rectification: rectifies stairs to
walking when it does not incur
floor change
16
Landmarks match




Types of landmark
 Stair
 Elevator
GraphSLAM requires matching of the same landmark along the trajectory
Training phase
 Obtain information from reference landmarks
 WiFi access point visibility
Testing phase
 Landmarks matching
 Get current WiFi AP visibility
 Calculate
distance to reference landmarks
 Take the closest corresponding landmark
18
Initial PDR trajectory



Initial trajectory
obtained from PDR
Rotation angle
underestimated for
every turn
Need to be modified
using GraphSLAM
19
Multi-floor GraphSLAM result
20
Conclusion



Rotation invariant features were able
to capture different dynamics of motion activities
Our approach improves classification accuracy
and jointly infers activity and corresponding floor information
GraphSLAM successfully modifies multi-floor trajectory
using natural landmarks detected by our framework.
21
Q&A
Thank you
22