E-eyes: Device-free Location-oriented
Activity Identification Using Fine-grained
WiFi Signatures
Presenter: Yan Wang
Yan Wang†, Jian Liu†, Yingying Chen†,
Marco Gruteser‡, Jie Yang#, Hongbo Liu*
†Dept.
of ECE, Stevens Institute of Technology
‡ WINLAB, Rutgers University
# Dept. of CS, Florida State University
* Indiana University-Purdue University Indianapolis
MobiCom 2014
Maui, Hawaii
Sept. 7th – 11th 2014
Motivation and Applications
2
Coarse-grained
Device-free passive Localization using
localization
Nonintrusive
Intrusive
off-the-shelf WiFi
RSS-based
approach
Localization
Localization/classification
using specialized devices
RTI
WiSee, WiTrack
Fine-grained
Granularity of the solutions
Our Goal: Low-Cost Fine-Grained Activity
Identification
Our E-eyes
Attached sensors
Non-attached sensors
Optimal solution
Low cost
Activity sensors
Scalability / Infrastructural cost
3
High cost
Intuition and Basic Idea
Sub-carrier 1 Sub-carrier 2
Sub-carrier P-1 Sub-carrier P
Wall
...
Reflected
rays
CSI.rate
30
20
10
15
20
25
Subcarrier Index
30
0
Access
1290 point
Reflected
1285
rays
Wall
Smart 10
Appliance
5
1295
Direct path
Packet index
WiFi
device
SNR
CSI Amplitude
40
5
10
15
20
25
Subcarrier Index
30
1280
0
5000
100
environmentsCSI.rate
 Increasing availability of WiFi signals in home
 WiFi provides fine-grained channel state information (CSI)
 Use CSI to capture changes of multipath environment
4
Uniqueness of CSI Comparing to RSS
CSI Amplitude
30
Wall
Reflected
rays
Washing dishes
25
Talking on the phone
20
15
10
5
0
Direct path
WiFi
device
Reflected
Wall rays
RSS amplitude
Access 25
point
1
5
9
13
17
Washing dishes
20
Talking on the phone
15
10
5
0
1
5
5
9
13
17
30
30
25
25
Sub-carriers
Sub-carriers
E-eyes System Challenges
20
15
10
5
0
0
20
15
10
5
50
100
150
Packet index
200
0
0
250
50
 Profile uniqueness and Robustness
 Generality to different types of activities
 Assisting the profile generation and updating
6
100
150
Packet index
200
250
System Overview
Access Point Signal Time Series
Increase robustness to
real environments
Data Pre-processing
Activity Identification
Generality to
different Activities
Coarse Activity Determination
Walking Activity
Walking
activity
In-place activity
Activity
In-place
Tracking using
MD-DTW
Identification
using EMD
Data Fusion
Crossing Links
Known Activity
Assisting the
profile generation
and updating
Profile Construction and
Updating
Construc
tion
Adaptive
Profile
Updating
matching
Unknown Activity
7
None
Profile
Based
User Feedback
Coarse Activity Determination
Time
CSI Amplitude
…
In-place
activity
Subcarrier P
Walking
…
activity
CSI Amplitude
Subcarrier p
CSI Amplitude
CSI Amplitude
Subcarrier 1
Time
V (1)
… V ( p) …
V 
TimeV ( p )
Time
V (P)
P
p 1
 Walking activity
 Large moving variance due to significant body movements and
location changes
 In-place activity
 Small moving variance due to smaller body movements
8
Characteristics of CSI Measurements from
Walking Activity
Trajectory 2
30
30
25
25
Sub-carriers
Sub-carriers
Trajectory 1
20
15
10
5
0
0
20
15
10
5
50
100
150
Packet index
200
0
0
250
50
100
150
Packet index
 CSI pattern is dominated by walking activities’ path
 Doorway profile can facilitate walking activity tracking
9
200
250
CSI Amplitude
Subcarrier 1
Walking Activity
Classifier
…
Subcarrier P
…
Time
Time
Multi-Dimensional Dynamic
Time Warping Distance
Derivation
Time
Activity
Profiles
Time
…
Time
10
Subcarrier P
CSI Amplitude
…
CSI Amplitude
Subcarrier p
CSI Amplitude
Subcarrier 1
DTW distance
Subcarrier p
CSI Amplitude
CSI measurements
CSI Amplitude
Walking Activity Tracking
Time
Characteristics of CSI Measurements from
In-Place Activity
15
Counts
Counts
15
10
5
0
10
5
0
0
5
10
15
CSI Amplitude Bins
20
0
5
10
15
CSI Amplitude Bins
 Different in-place activities cause different distributions of CSI
 Different rounds of same in-place activities result in similar
distributions of CSI
11
20
In-Place Activity Identification
CSI measurements
Counts
In-place Activity
Classifier
Distribution of CSI
Amplitudes Extraction
CSI Amplitude
CSI measurements
Raw data
CSI Amplitude Bins
Quantized data
Time
Subcarrier Earth Mover’s
Distance Derivation
Counts
Counts
Activity
Distribution
Profiles
Activity profile
CSI Amplitude Bins
CSI Amplitude Bins
EMD distance
12
Non-profiling Clustering
Activity
Identification
Profile Construction and
Updating
Constructing
profiles
Adaptive
Updating
Unknown
Activity
Nonprofiling
Clustering
User Feedback
 Semi-supervised approach to cluster daily activities and update CSI
profiles
 Construct CSI profiles when our system starts
13
Questions
 How robust is the system in typical indoor
environments?
 Can two different activities be distinguished at
the same location?
 Is WiFi traffic in home environment feasible to
identify activities?
14
Experimental Setup
 WiFi devices
 Intel 5300 NIC + Thinkpad T500 and T 51
 Cisco E2500
 Scenarios
 Small apartment with one bedroom
 Large apartment with two beddoms
15
Questions
 How robust is the system in typical indoor
environments?
 Can different activities be distinguished at the
same location?
 Is WiFi traffic in home environment feasible to
identify activities?
16
Performance of In-place Activity
Identification in Two Different Apartments
1-bedroom apartment
Mult. WiFi device
Single WiFi device
1
1
0.8
0.8
Mult. WiFi device
TPR
TPR
Single WiFi device
2-bedroom apartment
0.6
0.6
0.4
0.4
0.2
0.2
0
0
a
b
c
d
e
f
g
o
a
Activity types
b
f
g
h
i
j
o
Activity types
False positive rate: less than 5%
17
Performance of Walking Activity Tracking and Doorway
Identification
1-bedroom apt.
A
B
C
D
Unknown
Door
Door1
Door2
Door3
A
1
0
0
0
0
Door1
1
0
0
B
0
1
0
0
0
C
0
0
0.95
0.05
0
D
0
0
0
1
0
Door2
0
0.975
0.025
O
0
0
0.1
0
0.9
None
0
0
1
2-bedroom apt.
E
F
G
H
Unknown
Door
Door1
Door2
Door3
E
1
0
0
0
0
Door3
1
0
0
F
0.15
0.85
0
0
0
G
0
0
0.9
0.1
0
H
0
0
0
1
0
Door4
0
0.875
0.125
O
0.05
0
0.1
0
0.9t
None
0
0
1
18
Questions
 How robust is the system in typical indoor
environments?
 Can different activities be distinguished at the
same location?
 Is WiFi traffic in home environment feasible to
identify activities?
19
Performance of Identifying Different
Activities at the Same Location
 Four in-place activities




True Positive Rate
0.5
0.8
0.4
0.6
0.3
FPR
TPR
1
Sleeping on the bed
Sitting on the bed
Receiving calls nearby the sink
Washing dishes nearby the sink
0.4
0.2
0.2
0.1
0
0
4 5 6 7 8 9 10 11 12
Number of EMD bins
False Positive Rate
4 5 6 7 8 9 10 11 12
Number of EMD bins
20
Questions
 How robust is the system in typical indoor
environments?
 Can different activities be distinguished at the
same location?
 Is WiFi traffic in home environment feasible to
identify activities?
21
Performance of Different Packet Rate
 Packet transmission rate (PTR): 5 pkts/s - 20 pkts/s
Average true positive rate
Average false positive rate
1
0.1
0.95
0.08
0.9
0.06
0.85
0.04
0.8
0.02
0.75
0
5
10
15
20
5
22
10
15
20
Conclusion
 Show that the channel state information (CSI) from offthe-shelf 802.11n devices can be utilized to identify and
distinguish in-place activities inside a home
 Develop a monitoring framework that can run on a single
WiFi AP and use the associated profile matching
algorithms to compare amplitude profiles against those
from known activities
 Explore dynamic profile construction to accommodate
the movement or replacement of wireless devices and
day-to-day profile calibration
 Extensive experiments in two apartments of different size
demonstrates the generality of our approach
23
Yan Wang
ywang48@stevens.edu
24