A Preliminary Study of Sensing Appliance Usage for
Human Activity Recognition Using Mobile Magnetometer
Mi Zhang and Alexander A. Sawchuk
Department of Electrical Engineering
University of Southern California
mizhang@usc.edu
UbiMI workshop at ACM Ubicomp Conference, September 8, 2012
1
Introduction
• Background
 Human activity recognition is one of the most basic problems in Ubiquitous
Computing.
 Applications: surveillance, security, health care, etc.
 In this work, we focus on recognizing household activities by detecting appliance
usage.
• Existing Technology
Computer Vision
Courtesy to Opportunity Project
RFID
Courtesy to Intel Research
Smart Meter
Courtesy to IBM
2
Magnetic Field Sensing
• Idea
 When household appliance is in operation, it generates electromagnetic waves.
 The electric component has been demonstrated to infer appliance usage at home.
 How about the magnetic component?
• Magnetometer
 The magnetometer measures the strength and the direction of the earth’s magnetic
field in 3D space.
 It is mainly used for outdoor navigation
(referred to as compass).
 Magnetometer can also be used for detecting
magnets and ferromagnetic materials
(referred to as metal detector).
3
Key Observation
• More Interestingly
 When household appliances are in operation, the magnetic field around them
presents a different pattern compared to the scenarios when these devices are
turned off.
Hair Dryer is OFF
Hair Dryer is ON
These changes exhibit different patterns for different devices and act as the
signatures of the devices.
4
Our Framework
Magneto
meter
Sliding
Window
at 6s
Feature
Extraction
Overall Magnetic
Field Strength
M (t )  mag x (t ) 2  mag y (t ) 2  mag z (t ) 2
Classification
 f1 
f 
F   2
 
 
 f n  n1
Standard Deviation, Mean Derivatives,
Mean Crossing Rate, Dominant Frequency,
Dominant Frequency Magnitude, Energy,
Spectral Entropy
5
Evaluation
• Sensing Hardware and Software
 Hardware: 3-axis magnetometer in iPhone 4GS,
sampling at 15Hz.
 Software: techBASIC mobile application.
 Data analysis is performed using MATLAB.
• Household Activities
Laptop
Hair Dryer
TV
Mobile Phone
Microwave
6
Preliminary Results
• Scatter Plots
Statistical Features
Energy Features
• Recognition Accuracy
Laptop
Microwave
TV
Hair Dryer
Mobile
Phone
84.3%
100%
81.6%
92.7%
83.4%
7
Limitations and Future Work
• Sampling Rate is LOW
 High frequency components in the magnetic field signal that may contain important
information are not captured.
• Test on More Appliances
• Combine with Other Environmental Sensors
 light sensor, temperature sensor, and motion sensors
8
Any Questions?
9
Thank You
10