Activity Recognition
Aneeq Zia
Agenda
• What is activity recognition
• Typical methods used for action recognition
• “Evaluation of local spatio-temporal features for action recognition”,
Heng Wang et all
• “Action Recognition by Dense Trajectories”, Heng Wang et all
• Summary
• References
Typical methods used for action recognition
Evaluation of local spatio-temporal
features for action recognition
Result
Action Recognition by Dense
Trajectories
Dense Trajectories
• Feature trajectories have shown
to be efficient for representing
videos
• Extracted using KLT tracker or
matching SIFT descriptors
between frames
• However, the quantity and quality
is generally not enough
• This paper proposes an approach
to describe videos by dense
trajectories
Dense Trajectories
• The trajectories are obtained by tracking densely sampled points
using optical flow fields
• A local descriptor is introduced that overcomes the problem of
camera motion
• The descriptor extends the motion coding scheme based motion
motion boundaries developed in the context of human detection
Dense Trajectories
• Feature points are sampled on a grid spaced by W (=5) pixels and
tracked in each scale separately
• 8 spatial scales used
• Each point in a certain frame is tracked to the next frame using
median filtering in a dense optical flow field
Tracking
• Points of subsequent frames are concatenated to form a trajectory
• Trajectories are limited to ‘L’ frames in order to avoid drift from their
initial location
• The shape of a trajectory of length ‘L’ is described by the sequence
where
• The resulting vector is normalized by
Trajectory descriptors
• Histogram of Oriented Gradient (HOG)
• Histogram of Optical Flow (HOF)
• HOGHOF
• Motion Boundary Histogram (MBH)
• Take local gradients of x-y flow components and compute HOG as in static images
Bag of Features
• Codebook of descriptors (trajectories, HOG, HOF, MBH) constructed
• Number of visual words = 4000
• 100,000 randomly selected training features used
• Each video described by a histogram of visual word occurances
• Non-linear SVM with Chi-Square kernel used to classify the actions
Results
Summary
• Action recognition using
• HMM’s
• Temporal Template Matching
• Spatio Temporal Interest Points
• Bag of Visual Words Technique for action recognition
• Dense Trajectories
References
• “Evaluation of local spatio-temporal features for action recognition”,Heng Wang
et all
• “Action Recognition by Dense Trajectories”, Heng Wang et all
• CVPR 2011 tutorial on “Human Activity Analysis”
• CVPR 2014 tutorial on “Emerging topics in Human Activity recognition”