Visual Tracking via Locality
Sensitive Histograms
Shengfeng He
Qingxiong Yang
Rynson W.H. Lau
Jiang Wang
(CityU, Hong Kong)
Ming-Hsuan Yang
(UC Merced, California, U.S.A. )
• Introduction
• Locality Sensitive Histograms
• Illumination Invariant Features
• Multi-Region Tracking
• Experiments
• Conclusion
2
• Introduction
• Locality Sensitive Histograms
• Illumination Invariant Features
• Multi-Region Tracking
• Experiments
• Conclusion
3
• Due to their simplicity, intensity histograms are widely used to represent
objects for recognition and tracking.
• However, spatial information of object appearance is missing in this holistic
representation, which makes it sensitive to noise as well as occlusion in
tracking applications.
• To address this problem, algorithms based on multi-region representations
have been proposed.
• The fragment-based tracker divides the target object into several regions and
represents them with multiple local histograms.
4
• Propose a novel locality sensitive histogram algorithm, which takes into
account contributions from every pixel in an image instead of from pixels only
inside local neighborhoods as the local histogram algorithm.
• It operates in a way similar to conventional image histograms.
• However, instead of counting the frequency of occurrences of each intensity
value by adding ones to the corresponding bin, a floating-point value is added
to the corresponding bin for each occurrence of an intensity value
• The floating-point value declines exponentially with respect to the distance to
the pixel location.
5
6
7
• A histogram that takes into account contributions from every image pixel
adaptively and facilitates a real-time tracking framework.
• An efficient illumination invariant feature for tracking.
• A multi-region tracking algorithm that outperforms state-of-the-art algorithms
both in terms of accuracy and speed.
8
• Introduction
• Locality Sensitive Histograms
• Illumination Invariant Features
• Multi-Region Tracking
• Experiments
• Conclusion
9
• Let I denote an image. The corresponding image histogram H is a Bdimensional vector defined as:
W is the number of pixels
B is the total number of bins
Q(Iq, b) is zero except when intensity value Iq (at pixel location q) belongs to bin b
10
• Let 𝑯𝑰𝒑denote the integral histogram computed at pixel p
𝑯𝑰𝒑 contains the contributions of all the pixels to the left of pixel p
• The local histogram between pixel p and another pixel q on the
left of p is computed as
11
• Object tracking, pixels further away from the target center should be weighted
less as they more likely contain background information or occluding objects.
Hence, their contribution to the histogram should be diminished.
α ∈ (0, 1) is a parameter controlling the decreasing weight as a pixel moves away from the target center
12
• Similar to the integral histogram
• The summation of
𝐸,𝑙𝑒𝑓𝑡
𝐻𝑝
and
𝐸,𝑟𝑖𝑔ℎ𝑡
𝐻𝑝
combines the contribution from all
pixels and the weight of the contribution drops exponentially with respect to
the distance to pixel p
13
• Normalization factor at pixel p
14
• 𝑛𝑝 can also be computed in the same way:
15
• Introduction
• Locality Sensitive Histograms
• Illumination Invariant Features
• Multi-Region Tracking
• Experiments
• Conclusion
16
• Propose a new method for extracting dense illumination invariant features
• It is basically an image transform to convert the original image into a new
image, where the new pixel values do not change when the illumination
changes.
• Let 𝐼𝑝 and 𝐼𝑝′ denote the intensity values of pixel p before and after an affine
illumination change. We have:
where a1,p and a2,p are two parameters of the
affine transform Ap at pixel p.
17
• LSH Intensity value :
κ = 0.1 is a constant
𝑏𝑝 denote the bin corresponding to the intensity value 𝐼𝑝
𝑟𝑝 controls the interval of integration at pixel p , 𝑟𝑝 = 𝜅 𝐼𝑝 − 𝐼𝑝
18
19
• Introduction
• Locality Sensitive Histograms
• Illumination Invariant Features
• Multi-Region Tracking
• Experiments
• Conclusion
20
• The proposed tracking algorithm represents a target object with multiple
overlapping regions, each of which describes some local configuration.
• Based on the location of the target object center in the previous frame, we aim
to locate the new center location within the search region.
• Exhaustive search within the search region is performed in this work, where
every pixel is considered as a candidate target center.
21
22
• Each region at the candidate location is matched correspondingly against the
template to produce a score, based on the Earth Mover’s Distance (EMD)
𝑆1 and 𝑆2 are the corresponding regions of the template
(centered at 𝑝1 ) and the candidate (centered at 𝑝2 )
𝐶1 and 𝐶2 are the cumulative histograms of 𝐻𝑝𝐸1 and 𝐻𝑝𝐸2
• Once the matching scores of all regions within a candidate region are
computed, a vote map is obtained.
• Use a least-median-squares estimator to accumulate all the vote.
23
• Once the new target location is determined, the local histograms are updated
as follows:
M is the median distance of all the regions at the new position
𝐹1 and 𝐹2 are the forgetting factors, which define the appearance model update rate
• We set the forgetting factors 𝐹1and 𝐹1 as 0.96 and 1.04, respectively. For a
target object with 1,000 regions, about 30 of them are updated in each frame.
24
• Introduction
• Locality Sensitive Histograms
• Illumination Invariant Features
• Multi-Region Tracking
• Experiments
• Conclusion
25
Red, green, yellow,azure, purple, olive and blue rectangles correspond to results from
DFT, CT, MTT, MIL, Struck, TLD and ours.
26
27
28
• Introduction
• Locality Sensitive Histograms
• Illumination Invariant Features
• Multi-Region Tracking
• Experiments
• Conclusion
29
• In this paper, we propose a novel locality sensitive histogram method and a
simple yet effective tracking framework.
• Experimental results show that the proposed multi-region tracking algorithm
performs favorably against numerous state-of-the-art algorithms
• http://www.shengfenghe.com/visual-tracking-via-locality-sensitivehistograms.html
30