A Discriminative Key Pose Sequence
Model for Recognizing Human
Interactions
Arash Vahdat, Bo Gao, Mani Ranjbar,
and Greg Mori
ICCV2011
Goal
Outline
•
•
•
•
•
•
•
Introduction
Related methods
Modeling Human Interactions
Single Subject Key Pose Sequence Model
Interaction Key Pose Sequence Model
Learning the parameters
Experiments
Introduction
• This paper focuses on recognizing interactions
between individuals.
• The sequences of key poses between peoples
will be combined into activity level.
• The activities to be recognized include
hugging ,shaking hands,pointing, punching,
kicking,pushing each others.
Introduction
• Not every movement or pose by the target is
relevant to the activity to be recognized.
• We use an examplar-based model by giving
the pose a score to match the key poses.
• When people do some activities, they will do
the key poses in a chronological order.
Related methods
• Ryoo and Aggarwal [13]develop a matching
kernel that considers spatial and temporal
relations between space-time interest points.
• Yao et al. [23] use a Hough transform voting
scheme from an interest point representation.
[13] M. Ryoo and J. Aggarwal. Spatio-temporal relationship match: Video structure
comparison for recognition of complex human activities. In ICCV, 2009.
[23] A. Yao, J. Gall, and L. Van Gool. A hough transform-based voting framework for
action recognition. In CVPR, 2010. 2, 6
Recognition result in [13]
Modeling Human Interactions
• There are four things to know:
• 1. Who is involved in the interaction?
– Subject or object
• 2. When do the key poses occur?
– The interval of the key poses
• 3. How are the key poses executed?
– With hand or leg , powerful or weak
• 4. Where are the people when the key poses
occur?
Modeling Human Interactions
• we will assume F maximizes a model G that
includes the latent variables H:
• The variables H are the answer of the four
questions above.
Single Subject Key Pose Sequence Model
Single Subject Key Pose Sequence Model
• We represent each key pose by h:
• Denote K key poses of a sequense by H
Single Subject Key Pose Sequence Model
• Exemplar Matching Link:
• Compute the pose connection strength to the
exemplar poses.
Single Subject Key Pose Sequence Model
• Activity-Key Pose Link:
– Compute the sequence similarity in the activity to
give a score.
• Direct Root Model:
Interaction Key Pose Sequence Model
• Update the model from individual to two
people interaction.
• We should recognize who is subject or object.
• The scoring function will be:
Learning the parameters
• Define the scoring function E(x,y):
• Use multiclass linear SVM classifier to find the
best parameters.
Experiments
• The UT-Interaction dataset contains videos of
6 classes of human-human interactions.
• Set 1 is with a stationary background,and
Set 2 is with slight background movement and
camera jitter.
Experiments
Experiments
Experiments
Experiments
Conclusion
• This paper focuses on the key poses method
to recognize the interaction.
• The precision is over 90% and outperform
other methods in UT-interaction dataset.