Different Tracking Techniques
1.Gaussian Mixture Model:

1.Construct the model of the
Background.

2.Given sequence of background
images find the mean and variance of
mixture of the Gaussians.

3.Given a new pixel find out the
probability of the pixel value having
come from the Gaussians.

2.NCC:Normalized cross correlation

1.For each pixel find out cross
correlation with the background.

2.Normalize by dividing the
variances.

3.Sometimes better results obtained
by not normalizing.


3.Entropy:

1. The entropy is defined as the
amount of information needed to exactly
specify the state of the system, given
what we know about the system.

2. If the two images are exactly alike
their entropy will be low.

3. The lower the entropy the higher
the likelihood that the pixel belongs to
the background.

4.Mutual Information:

1. Mutual information of two random
variables is a quantity that measures the
mutual dependence of the two variables.

2. The higher the mutual information
the higher the likelihood of the pixel
belonging to the background.

3. Considered to be a stronger
concept than entropy theoretically.

Gaussian Mixture Model
1.Technique developed by Chris Stauffer
and W.E.L. Grimson during CVPR 1999.
 2. The technique is so powerful and
omnipresent in tracking,HCI,etc. that it
won the Longuet-Higgins award at
CVPR 2009.

Gaussian Mixture
Model(continued)
We model the pixel history with K Gaussian
Probability Density Functions.
 The history of a pixel {x,y} is modeled as
time series.


(vc.cs.nthu.edu.tw/.../Background_Estimati
on_with_Gaussian_Distribution_for_Image
_Segmentation,_a_fast_approach.ppt)
Gaussian Mixture
Model(continued)

The probability of observing the current
pixel is given by
is the weight parameter that is used to
describe by the Gaussian distribution

is a Gaussian distribution that has two
parameters: is the mean of the
Gaussian distribution at time t and
is
the covariance matrix at time instant t

Gaussian Mixture
Model(continued)

A new pixel is said to match a
distribution
 If it is within 2.5 standard deviations from
mean of the distribution
 This distribution are updated with this pixel
value
Gaussian Mixture
Model(continued)
is the learning rate defined by the user.
I have set it as 0.1 which gives us good results.
Gaussian Mixture
Model(continued)

The weight parameters of all
distributions are updated as

is 1, for matched distribution
0, for unmatched distribution
Gaussian Mixture
Model(continued)

If the current pixel didn’t match with any of
the K distributions
 The distribution with smallest weight is replaced
by a new distribution

The new distribution with
 The mean is set to the value of the current pixel
 The variance is set large
 The weight is set to a small value
Gaussian Mixture
Model(continued)

Define which of distributions describing the
history of a pixel result from background
 1. Order all distributions by a factor
 2. B first distributions are marked as background
distribution

If a pixel is matched with one of these B
distributions it is marked as a background
pixel
Gaussian Mixture
Model(continued)
1. Remove the singular pixels and small
blobs by eroding.
 2.Use connected components to
segment out the persons.

Results
Here are the results to a normal video
sequence.
 As you can see the shadows have been
removed so has the noise.
 Further improved results can be
obtained by edge detecting.

A Color Similarity Measure for Robust
Shadow Removal in Real-Time
1. This is a paper by Daniel Grest, JanMichael Frahm, and Reinhard Koch.
 2. Under normal conditions it gives us
suitable results.
 It is suitable for indoor use. However
with the complexities of outside lighting,
etc. it fails to live up to background
modeling.

A Color Similarity Measure for Robust
Shadow Removal in Real-Time
1. To detect foreground regions we use a
background reference image and calculate
the difference image with the current one.
 2. To acquire a background reference a
ground truth image is incorporated as a
mean image of a sequence of the
interaction area without the user.
 3. The segmentation of the foreground is
often obtained by applying a threshold to
the difference values.

A Color Similarity Measure for Robust
Shadow Removal in Real-Time(Continued)

1. Instead of calculating the variance for
each pixel over time, we measure the
variance of the pixel noise over the
whole difference image each frame,
leaving the segmented foreground
region out.
A Color Similarity Measure for Robust
Shadow Removal in Real-Time(Continued)

Let d(x,y) = r(x,y) - a(x,y) be the
difference image value, r(x; y) the
background reference image intensity
and a(x,y) the current image intensity at
image positions (x,y). The variance of
noise in an image with size M X N is

Variance= Sum over the window
( d(i,j)-Mean Image)^2
A Color Similarity Measure for Robust
Shadow Removal in Real-Time(Continued)
Assuming a Gaussian distribution of
noise we set the segmentation threshold
to 2 * Variance . Therefore 97.5% of
noise are in the segmentation interval.
 Each pixel belongs to the foreground, if
 |d(x,y)-Mean Image| > 2 * Variance.
 Now we have to remove the shadow.

A Color Similarity Measure for Robust
Shadow Removal in Real-Time(Continued)

To measure the similarity of image parts
it is possible to calculate the cross
correlation over a given window size .
Usually brightness invariance is desired,
as given by the cross covariance, which
is the average-free cross correlation.
A Color Similarity Measure for Robust
Shadow Removal in Real-Time(Continued)
The cross covariance with window size
M X N for two images a,b at image
position (x,y) is then defined as
 CC(a,b)(x,y) =(1/MN) Sum over window
 (Pixel(x,y) of Image a-Mean window a) *
(Pixel(x,y) of Image b-Mean window b)
 To make it contrast invariant divide by
the variance of each of the windows.

A Color Similarity Measure for Robust
Shadow Removal in Real-Time(Continued)

For shadow pixels this correlation is
near to one, so we eliminate the darker
pixels in the segmented image, if the
NCC of the pixel in the current image
and the pixel in the reference image is
greater than a threshold NCC. After
removing pixels with the NCC
thresholding, we apply a 3x3 closening
operator to fill small holes in the contour.
A Color Similarity Measure for Robust
Shadow Removal in Real-Time Results





Given below are the NCC results.
Using only NCC doesn’t take care of
moving objects like trees blowing in
wind,moving shadows,etc.
It is suitable only for indoors with restricted
environment.
Using edge detection and NCC can give us
improved results.
However, it is not as powerful as
background modeling.
Entropy Results
If two images are similar their entropies
should be close.
 Hence, by taking the differences in
entropy over a window, we can find the
similarity.
 This method doesn’t give us results as
good as those for Background Modelling
or even NCC.

Conclusion
Background Modelling gives us the nest
results.
 Done on a GPU it gives us real time
speed.
