Extracting Places and Activities from GPS Traces
Using Hierarchical Conditional Random Fields
Lin Liao, Dieter Fox, and Henry Kautz, In International Journal of Robotics Research
(IJRR), 26(1), 2007
2012311529
Yong-Joong Kim
Dept. of Computer Science
Yonsei University
Contents
• Motivation
• Hierarchical Activity Model
• Preliminaries : Conditional Random Fields
– Overview
– Inference
– Parameter Learning
• Conditional Random Fields for Activity Recognition
– GPS to street map association
– Inferring activities and types of significant places
– Place detection and labeling algorithm
• Experimental Results
–
–
–
–
Experimental environment
Example analysis
Extracting significant places
Labeling places and activities using models learned form others
• Conclusions
Motivation (cont’)
• Application areas of learning patterns of human behavior from sensor data
– Intelligent environments
– Surveillance
– Human robot interaction
• Using GPS location data to learn to recognize the high-level activities
• Difficulties in previous approaches
– Restricted activity models
– Inaccurate place detection
Motivation
• A novel, unified approach to automated activity and place labeling
– High accuracy in detecting significant places by taking a user’s context into
account
– By simultaneously using CRF (Conditional Random Field)
• Estimating a person’s activities
• Identifying places
• Labeling places by their type
• Research goal
– To segment a user’s day into everyday activities
– To recognize and label significant places
Hierarchical activity model (cont’)
• GPS readings
– Input to proposing model
– Segmenting a GPS trace spatially in order to generate a discrete sequence of
activity nodes
• Activities
– Being estimated for each node in the spatially segmented GPS trace
– Distinguishing between navigation activities and significant activities
• Significant places
– Playing a significant role in the activities of a person
Hierarchical activity model
• Two key problems for probabilistic inference
– Complexity of model
• Solved by approximating inference algorithm
– Not clear how to construct the model deterministically from a GPS trace
• Solved by constructing the model as part of this inference
Preliminaries :
Conditional Random Fields
Preliminaries: Conditional random fields
Overview (cont’)
• Definition of CRFs
– Undirected graphical models developed for labeling sequence data
– Properties
• Directly represent the conditional distribution over hidden states
• No assumptions about the dependency structure between observations
• Nodes in CRFs
– Observation :
– Hidden states :
– Defining conditional distribution
over hidden states y
• Cliques
– Fully connected sub-graphs of a CRF
– Playing a key role in the definition of conditional distribution
Preliminaries: Conditional random fields
Overview
• Conditional distribution over hidden state :
where
Preliminaries: Conditional random fields
Inference (cont’)
• Inference in CRF can have two tasks :
– To estimate the marginal distribution of each hidden variable
– To estimate the most likely configuration of the hidden variables
(i.e. the maximum a posteriori, or MAP, estimation)
– Using Belief propagation to solve these tasks
• Two types of BP algorithms :
– Sum-product for marginal estimation
– Max-product for MAP estimation
Preliminaries: Conditional random fields
Inference (cont’)
• Sum-product for marginal estimation
– Message initialization :
• Initializing all messages
as uniform distr. over
– Message update rule :
– Message update order :
• Iterating the message update rule until it (possibly) converges
– Convergence conditions :
– After convergence, calculation of marginals
Preliminaries: Conditional random fields
Inference
• Max-product for MAP estimation
– Very similar to the sum-product
– Replaced summation with maximization in the message update rule
– After convergence, calculating the MAP belief
– Then, each component of
Preliminaries: Conditional random fields
Parameter learning (cont’)
• Goal of parameter learning
– To determine the weights of the feature functions
– Learn the weights discriminatively
• Two method
– Maximum likelihood (ML) estimation
– Maximum pseudo-likelihood (MPL) estimation
• Parameter sharing
– Learning algorithm to learn the same parameter values (weights) for different
cliques in the CRF
Preliminaries: Conditional random fields
Parameter learning (cont’)
• Maximum likelihood (ML) estimation
– Object function
– The gradient of object function
Preliminaries: Conditional random fields
Parameter learning (cont’)
• Maximum pseudo-likelihood (MPL) estimation
•
•
: local feature counts involving variable
– Object function
– The gradient of object function
Preliminaries: Conditional random fields
Parameter learning
• Parameter sharing
– Learn a generic model that can take any GPS trace and classify the locations in
that trace
– Achieved by making sure that all the weights belonging to a certain type of
feature are identical
– Calculating gradient for a shared weight by the sum of all the gradients
computed for the individual cliques
Conditional Random Fields for
Activity Recognition
Conditional Random Fields for Activity Recognition
GPS to street map association (cont’)
• Desirable to associate GPS traces to a street map
– (e.g.) to relate locations to addresses in the map
• Constructing a CRF
– Taking into account the spatial relationship between GPS readings
– Generating a consistent association
Conditional Random Fields for Activity Recognition
GPS to street map association (cont’)
• Distinguishing tree types of cliques
– Measurement cliques (dark grey)
– Consistency cliques (light grey)
– Smoothness cliques (medium grey)
Conditional Random Fields for Activity Recognition
GPS to street map association
• Using these feature function, conditional distribution can be written as :
– 𝑊𝑚 : measurement feature function weight
– 𝑊𝑐 : consistency feature function weight
– 𝑊𝑠 : smoothness feature function weight
Conditional Random Fields for Activity Recognition
Inferring activities and types of significant places (cont’)
• Generating a new CRF, to estimate
– Activity performed at each segment
– A person’s significant places
Conditional Random Fields for Activity Recognition
Inferring activities and types of significant places
• Activity node’s features
–
–
–
–
Temporal information such as time of day, day of week, duration of the stay
Average speed through a segment
Information extracted from geographic databases
Connected to its neighbors
• Place node’s feature
– Activities that occur at a place strongly (consider weekly frequency)
– A limited number of different homes or work places
• Possibility of generating very large cliques
– Resolve this problem by converting to tree-structured CRFs
Conditional Random Fields for Activity Recognition
Place detection and labeling algorithm
Experimental Results
Experimental Results
Experimental environment
• Collected GPS data from four different persons
– Seven days of data
– Roughly 40,000 GPS measurements (10,000 segments)
– Manually labeled all activities and significant places
• Using leave-one-out cross-validation for evaluation
– Training data : 3 persons (MPL estimation for learning)
– Testing data : 4 persons
Experimental Results
Example analysis
Experimental Results
Extracting significant places
• Comparing experiment
– Proposing system
– A widely-used approach (time threshold)
Experimental Results
Labeling places and activities
using models learned form others (cont’)
Labeling places and activities
using models learned form others
Experimental Results
Conclusions
• A novel approach to performing location-based activity recognition
–
–
–
–
One consistent framework
Iteratively constructing a hierarchical CRF
Discriminative learning using pseudo-likelihood
Being performed the Inference efficiently using loopy BP
• Achieving virtually identical accuracy both with and without a street map