Whole-Home Gesture Recognition
Using Wireless Signals
——MobiCom’13
Author: Qifan Pu et al. University of Washington
Presenter: Yanyuan Qin & Zhitong Fei
Background
• Why interactive technique?
Control music volume with gesture when showering or cooking.
Turn on the thermostat with gesture while in bed.
The disabled need it more.
• Options?
Kinect at each room? Expensive…
Wearable Devices? Wear a device in a shower?…
• Can we use
？
Can we use it to recognize gestures?
How to detect gestures using Wi-Fi?
• Wi-Fi Doppler shifts
• Humans reflect Wi-Fi signals, thus can be treated as signal sources
• Human motion introduce Wifi Doppler shifts
• Different gestures exhibit different patterns.
Two Challenges
•Challenge 1: How to detect small Doppler shifts within
WiFi’s bandwidth (typically 20MHz)
f  5GHz


f  2 fv / c,  v  0.5m / s
c  light _ speed

f  17 Hz
•Challenge 2: How to determine the relationship between
Doppler shifts and gestures
In WiSee the target human performs a repetitive gesture, which
we use as that person’s preamble.
WiSee
• WiSee is a wireless system that enable whole-home gesture
reconigtion.
• Three main questions:
First, how does WiSee extract Doppler shifts from conventional
wireless signals like Wi-Fi?
Second, how does it map the Doppler shifts to the gestures
performed by the user?
Third, how does it enable gesture recognition in the presence of
other humans in the environment?
Extracting Doppler shifts from Wireless
Signals
Doppler shift is the change in the observed frequency as the transmitter and
the receiver move relative to each other.
• Move towards
crests arrive at a faster rate
• Move from
crests arrive at a slower rate
• Where c is the speed of light in the medium and f is the transmitter’s center
frequency.
Extracting Doppler shifts from Wireless Signals
How does WiSee deal with frequency offsets?
Mapping Doppler Shifts to Gestures
WiSee extracts the Doppler information by computing the frequencytime Doppler profile of the narrowband signal.
Segmentation
WiSee leverages the stucture of the Doppler profiles.
Set of segments that have positive and negative Doppler shifts.
Gestures
Gestures Classification
The receiver can classify gestures by matching the pattern.
Positive
Negative
Both
“1”
“-1”
“2”
Challenge 3: Interference from other
people in the environment
• WiSee uses MIMO in 802.11n to improve accuracy
• But MIMO requires a known preamble
• In WiSee, a repeated gesture acts as preamble to specify certain user
User pushes
hand toward
and away
WiSee detect
target user
WiSee
iteratively finds
MIMO channel
that maximizes
Doppler energy
Multi-path Effects
The reflections usually arrive at the receiver along multiple
paths.
A gesture towards
receiver can create
both positive and
negative Doppler shifts
at the receiver.
WiSee automatically finds a
proper MIMO direction
because of iteration algorithm
Strong reflectors
(metal) can flip the
positive and negative
Doppler shifts.
The receiver can calibrate the
sign of the subsequent
Doppler shifts based on
preamble gestures.
Experiments
•Scenarios
Office building
Two-bed apt
•Many conditions
Line-of-sight, non-line-of-sight, through-the-wall,
through-the-corridor, through-the-room
Feasibility of Wireless Gesture Detection
Wall
TX: transmitter RX: Wisee receiver
Feasibility of Wireless Gesture Detection
Gesture Recognition in Two-bed apt
Gesture Recognition Accuracy
Confusion matrix
Gestures in the Presence of Other Humans
(No target user, only other 12 people)
False detection rate for a whole day
Influence from the number of interfering user
(Target user and other people)
Influence from the distance of interfering user
Summary
• Design and evaluate, WiSee, a gesture recognition
system that leverages wireless signals to enable
whole-home sensing and recognition of human
gestures
• WiSee can extract a rich set of gesture information
from wireless signals and enable whole-home
gesture recognition using only two wireless sources
placed in the living room
Comments on WiSee
Advantages
Whole-home gesture recognition with few wireless sources.
Without sensing device on human body or many devices in
environment
High accuracy: 94%(on average) for 9 gestures.
Disadvantages
Heavily influenced by the number of other people.
The number of gestures are limited (9 gestures) and hard to
extend.
Using scale is smaller than wearable devices.
Future Works
•Security?
•Information fusion: Add other information
to make the system more robust, like sound
•Explore whether the shape of body will
affect the performance.