Multiple Human Objects
Tracking in Crowded Scenes
Yao-Te Tsai, Huang-Chia Shih,
and Chung-Lin Huang
Dept. of EE, NTHU
International Conference on Pattern
Recognition (ICPR’06)
Outline






Introduction
Initialization and Pixel Classification
Single Object Tracking
Tracking Occluded Objects
Experimental Results
Conclusion
Outline






Introduction
Initialization and Pixel Classification
Single Object Tracking
Tracking Occluded Objects
Experimental Results
Conclusion
Introduction
 human objects tracking systems
 Pfinder
 Utilize stochastic, region-based feature
 McKenna et al.
 Adaptive Gaussian mixture to model color
distribution
 M2Tracker
 Combine presence probability with color model
to classify each pixel
 Tsutsui et al.
 Exchange the optical flow information within
multiple cameras
Multiple human objects tracking
system
 System consist of





Model-based object segmentation
Remove noise of segmented region
Optical flow-based position estimation
Occlusion detection
Object separation from occlusion
 Contribution
 Track occlusion, separate object and
track it individually afterwards
Outline






Introduction
Initialization and Pixel Classification
Single Object Tracking
Tracking Occluded Objects
Experimental Results
Conclusion
Gaussian Mixture Color Model
 Condition probability for pixel i belong to
object O is
1
m

( i   j )T  j 1 ( i   j )
1
p(i O)  
e 2
 ( j)
j 1 2  j
 Parameters:
mean  , and covariance matrix 
 ( j )  0,  j ( j )  1
 Expectation-maximization (EM) algorithm
 To determine the maximum likelihood parameters
of a mixture of m Gaussian
Color Model
 Use HIS space to reduce ambient illumination
change
 Each pixel i has 2-D feature vector vi  (hi , si )
where hi is the hue, si is the saturation
 Likelihood pixel i belonging to torso (n=0) or the
bottom (n=1) of a person O is
1
 ( vi   j )T  j 1 ( vi   j )
1
p(i O )  
e 2
 ( j)
j 1 2  j
m
n
Color similarity
 The color of the torso of object 1 is
similar to the color of the bottom of
object 2
 (b) is the result of applying the torso
color model of object 2 for all pixels
Initialize Presence Map
 Presence map
 The set of presence probabilities of the pixels inside
the object
 Head line
 Scan the torso projection profile H0(yi)
 yHL=arg minyiH0(yi)
 Torso line
 Central vertical axis
 Probabilities of the pixels will be larger
Bayesian Classification
 Only consider pixels in the neighborhood of an
object
Pposterior(Okn i )  Pprior, ( xr , yr ) (Ok ) P(i Okn )
 Pposteriori(Ok|i) : posterior prob. of pixel i belong to
object Ok
 P(i|Ok) : probability defined by torso or bottom model
 Ppriori(Ok) : presence probability of Ok
 Relative coordinate
 Defined by the head line and central axis
 Color model selection for torso or bottom
 If Pposteriori(Ok|i) >=0.05, then i classified to Ok
Outline






Introduction
Initialization and Pixel Classification
Single Object Tracking
Tracking Occluded Objects
Experimental Results
Conclusion
Single Object Tracking
 Flow chart of single object tracking
 Newcomer detection
 By using background subtraction
Tracking process
 Calculate angles and magnitudes of the flow vectors
in the neighborhood of window
 Quantize the direction into 12 bins (30 degree/bin)
and determine which bin object belong to
 Find the most significant bin and calculate average
flow
 Shift object window by average flow
Update presence map
 Size and shape of a moving object change
over time
 Need to update the presence map
 If pixel at (xr, yr) classified correctly,
increase the corresponding priori prob. for
every 10 frames
Outline






Introduction
Initialization and Pixel Classification
Single Object Tracking
Tracking Occluded Objects
Experimental Results
Conclusion
Tracking Occluded Objects
 Optical flow and presence probability
are unreliable
 Only use color models to estimate
object’s central vertical axis
 Use distance between central axes to
determine object becomes separable
Occlusion detection
 Each individual object has five attributes
based on its activity
Two object windows touch
and form an occlusion
window
Two object windows
overlap and form an
occlusion window
A single object joins
an occlusion and form
a new occlusion
window
Occlusion group separation
 Compute distance between every two
objects in an occlusion group as
d k  xck 1  xck
 If di  max  d k

 Two extreme objects Oi and Oi+1
 If |di| > threshold, then determine Oi or Oi+1 can
be separate from the original occlusion
Object separation example
separate
separate
Resume tracking
 One an object separate from occlusion, we
need to update:
 Object window location, head line, and torso line
 Central vertical axis
 Left and right boundary
 Scan the vertical projection profile of Ok
 From the central vertical axis leftward and then
rightward
 Head line and torso line
 Analyze the horizontal projection profile
Outline






Introduction
Initialization and Pixel Classification
Single Object Tracking
Tracking Occluded Objects
Experimental Results
Conclusion
Tracking example 1
 Format:
Occlusion 2
 Image frame is 160x120x24 bits, 15 frames/sec
Occlusion 1
Object 2 separate
and join occlusion 1
single object 4
occlusion 1
Tracking example 2
 Two occlusion groups merge as one and then separate
to another two occlusion groups
System evaluation and error
analysis
 Three separation events:
 2-object, 3-object, and 4-object separation event
 Define separation occurs’ accuracy based on:
 A single object leaves an occlusion and track him
correctly afterward
 If an occlusion splits into two, system identify the
correct objects in the two pairs.
 More 2-object separation events
Outline






Introduction
Initialization and Pixel Classification
Single Object Tracking
Tracking Occluded Objects
Experimental Results
Conclusion
Conclusion
 Object tracking consists of:
 Gaussian mixture model
 Presence probability
 Optical flow
 Objects under occlusion
 Use color model to distinguish each object and
locate central vertical axes
 Object separation
 Determine by distances between the central
vertical axes of objects