MVL (Machine Vision Lab)
HUMAN MOTION VIDEO
DATABASE
Scripts, Queries, Recognition
Jezekiel Ben-Arie
ECE Department
University Of Illinois at Chicago
UIC
MVL (Machine Vision Lab)

Composition of interactive motion queries.

Analysis and Recognition of human activities.

Human body parts labeling.

Human detection.
UIC
MVL (Machine Vision Lab)
UIC
HUMAN ACTIVITY CAPTURE AND REGONITION
UIC
MVL (Machine Vision Lab)
Visual
Feedback
User
Motion Query
Video Retrieval
Videos
Video Database
Video Analysis and
Recognition
Retrieved videos
MVL (Machine Vision Lab)
UIC
HUMAN BODY PART LABELING

Objective: Identify the roles of parts that appear as
bars.

Labeling : Using the spatial locations and orientations.

Method : Finding maximum conjunction of partial
hypotheses.
MVL (Machine Vision Lab)
HUMAN BODY PART LABELING
Theoretical Foundations
UIC
UIC
MVL (Machine Vision Lab)
HUMAN BODY PART LABELING
Illustration of Theoretical Foundations
(a)
Overlap of Spatial distribution for
(a) Correct Labeling (b) Incorrect Labeling
(b)
UIC
MVL (Machine Vision Lab)
HUMAN BODY PART LABELING
(a)
Mesh diagram of Overlap of Spatial distribution for
(a) Correct Labeling (b) Incorrect Labeling
(b)
MVL (Machine Vision Lab)
HUMAN BODY PART LABELING
Experimental Results
Silhouette Extraction


Bar detection
Using Gabor signatures.
Parsing silhouettes

90 different human poses
98.7% correct labeling.

UIC
MVL (Machine Vision Lab)
HUMAN BODY PART LABELING
Experimental Results
UIC
MVL (Machine Vision Lab)
HUMAN BODY PART LABELING
Experimental Results
UIC
MVL (Machine Vision Lab)
HUMAN BODY PART LABELING
Silhouette Extraction
UIC
MVL (Machine Vision Lab)
HUMAN BODY PART LABELING
Silhouette Extraction
Illustration of variation of chromaticity and brightness distortion
UIC
MVL (Machine Vision Lab)
HUMAN ACTIVITY RECOGNITION
Introduction

Poses indicative of actions taking place
Poses involved in walking
Indexing based recognition using sparse
frames
 Extends this technique with optimal
constrained sequencing based voting

UIC
MVL (Machine Vision Lab)
UIC
HUMAN ACTIVITY RECOGNITION
Introduction

Temporal sequence of pose vectors

Multidimensional hash tables for model activities

Individual hash tables for each body part
Identifying input pose vectors as samples of densely
sampled model activity and create vote vectors

Vote vectors are temporal depiction of the loglikelihood that indexed pose belongs to a model

Dynamic programming based constrained sequencing
to recognize activities

MVL (Machine Vision Lab)
HUMAN ACTIVITY RECOGNITION
Creating Vote Vectors
Illustration of the entire voting process
UIC
MVL (Machine Vision Lab)
UIC
HUMAN ACTIVITY RECOGNITION
Experimental Results
Videos of sitting action overlaid with skeleton superposed with the help of
tracking information
Sparse samples of jump activity adequate for robust recognition
UIC
MVL (Machine Vision Lab)
HUMAN ACTIVITY RECOGNITION
Experimental Results
Average votes for 5 test videos of each
activity along with the votes for other
activities.
Rows – Test Activity
Columns – Model Activity
Recognition rate under various
conditions of occlusion
MVL (Machine Vision Lab)
UIC
HUMAN ACTIVITY RECOGNITION
Experimental Results
Performance of the approach under conditions of view point variance
UIC
MVL (Machine Vision Lab)
FACE DETECTION
Original Image
Skin detection
Detection by the Gabors
Regions passing the
ellipse area criterion
Detected Faces
UIC
MVL (Machine Vision Lab)
FACE DETECTION
Original Image
Detected faces with
medium threshold
(0.7)
Detected faces with
maximum threshold
(0.8)
MVL (Machine Vision Lab)
UIC
GUI for Queries Composition




Motion query is composed by using model motion data
clips.
An example of a model motion data clip is a walk cycle
consisting of a sequence of poses in one basic cycle of
left-right steps.
Model motion data clip can also be non-cyclic such as
sitting.
Model motion data clip is obtained from a motion capture
library or can be interactively composed by the user.
UIC
MVL (Machine Vision Lab)
INTERACTIVE GUI
Specify Trajectory Key-points
Interpolate by Splines
Specify Activities
Calculate Segments
Calculate Position and Orientations
Generate Motion Sequences(Scripts)
Display
MVL (Machine Vision Lab)
UIC
Theoretical Foundations
•
•
•
•
•
Parameterization of 3-D rotations (Euler Quaternions)
Splines (Catmull Rom)
Interpolation (SLERP, Quaternions)
Human body model
Motion composition techniques
(Inverse Kinematics, Mocap)
MVL (Machine Vision Lab)
Limb Pose Vocabulary
UIC
MVL (Machine Vision Lab)
Example of complete body poses
UIC
MVL (Machine Vision Lab)
Inverse kinematics based key framing
tool
UIC
MVL (Machine Vision Lab)
Implementation
UIC