Hierarchical Method for Foreground
DetectionUsing Codebook Model
Jing-Ming Guo, Yun-Fu Liu, Chih-Hsien Hsia,
Min-Hsiung Shih, and Chih-Sheng Hsu
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR
VIDEO TECHNOLOGY, VOL. 21, NO. 6, JUNE 2011
Outline
• Background Model Construction
– Block-Based Background Subtraction
– Pixel-Based Background Subtraction
• Hierarchical Foreground Detection
• Background Models Updating with the
Short-Term Information Models
• Experimental Results
Background Model Construction
• This method involves two types of
codebooks(CBs), block-based and pixel-based
CBs.
• The modeling of two CBs is similar to the
former CB[14]
[14] K. Kim, T. H. Chalidabhongse, D. Harwood, and L. Davis, “Real-time foregroundbackground segmentation using codebook model,” Real- Time Imaging, vol. 11, no. 3, pp.
172–185, Jun. 2005.
Background Model Construction
Background Model Construction
• There are two different time intervals for
training (xt).
• (1 ≤ t ≤ T) and (t > T) for training the
background models and foreground detection.
• The updating algorithms are separated into
two parts for different time zones.
The Features Used in Block-Based
Background Subtraction
• A frame xt of size P x Q is divied into multiple
non-overlapped blocks of size M x N.
• The former block truncation coding(BTC)
reduce the frame into two means,high-mean
and low-mean.
• In this paper ,we have four means to
represent a frame, high-top mean (μht ), highbottom mean (μhb), low-top mean (μlt ), and
low-bottom mean (μlb).
The Features Used in Block-Based
Background Subtraction
The Features Used in Block-Based
Background Subtraction
• Each means have three colors(RGB),so each
codebook have 12 dimensions.
Updating Block-Based Background
Models (CBs) in the Training Phase
• a specific block can be represented as a vector
Vb = {vbt|1 ≤ t ≤ T }.
• A CB for a block can be represented as
C = {ci|1 ≤ i ≤ L}, consisting of L codewords
• An additional weight wi is geared for
indicating the importance of the ith codeword.
• Codebook size is (P/M)x(Q/N)
Updating Block-Based Background
Models (CBs) in the Training Phase
Updating Block-Based Background
Models (CBs) in the Training Phase
Updating Pixel-Based Background
Models (CBs) in theTraining Phase
• The same as block-based method.
• Codebook size is P x Q.
• Each codebook is 3 dimensions (RGB)
Hierarchical Foreground Detection
• After the background models training as
indicated before the time point T, the two CBs
are applied to the proposed hierarchical
foreground detection.
• The foreground is obtained by background
subtraction.
Foreground Detection with the BlockBased CB
• the input vector (vbt) extracted from a block is
compared with the ith block-based codeword
(ci) to determine whether a match is found
• When a vbt is classified as background, the
corresponding block is also used to update the
pixel-based CB.
Foreground Detection with the PixelBased CB
• This subsection introduces how to classify a
pixel in a block to foreground or background.
• The foregrounds are classified into one true
foreground and two fake foregrounds (shadow
and highlight).
Foreground Detection with the PixelBased CB
Foreground Detection with the PixelBased CB
Background Models Updating with the
Short-Term Information Models
• an additional variable timeics is involved to
store the updated time for estimating whether
the corresponding ith codeword (cis ) has been
updated for a specific period or not.
• If the duration is longer than a predefined
parameter Dsdelete, the corresponding cis is
simply a temporary foreground.
Background Models Updating with the
Short-Term Information Models
• When cis , is favor to strong stationary (
wics ≥ Dadd), the short-term information model
can be considered as a part of the true
background model.
• This additional value is employed for filtering
out ci which meets the states of eventually
moving as foregrounds with the predefined
parameter Ddelete.
Experimental Results
• λ = 5 for block-based , λ = 6 for pixel-based, η =
0.7, θcolor = 3, β = 1.15 ,γ = 0.72
,Dupdate = 3, and α = 0.05, Dadd = 100,
Dsdelete = 200, and Ddelete = 200
Experimental Results
•
•
•
•
[9]MOG
[5]color model
[11][25] hierarchical MOG
[14]CB
[9] C. Stauffer and W. E. L. Grimson, “Adaptive background mixture models for real-time tracking,” in Proc. IEEE
Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2. Jun. 1999, pp. 246–252.
[5] R. Cucchiara, C. Grana, M. Piccardi, and A. Prati, “Detection moving objects, ghosts, and shadows in video streams,”
IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 10, pp. 1337–1342, Oct. 2003.
[11] Y.-T. Chen, C.-S. Chen, C.-R. Huang, and Y.-P. Hung, “Efficient hierarchical method for background subtraction,”
Pattern Recognit., vol. 40, no. 10, pp. 2706–2715, Oct. 2007.
[25] C.-C. Chiu, M.-Y. Ku, and L.-W. Liang, “A robust object segmentation system using a probability-based background
extraction algorithm,” IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 4, pp. 518–528, Apr. 2010.
[14] K. Kim, T. H. Chalidabhongse, D. Harwood, and L. Davis, “Real-time foreground-background segmentation using
codebook model,” Real- Time Imaging, vol. 11, no. 3, pp. 172–185, Jun. 2005.
•
C)MOG
d)Color model
e)CB
f)g) hierarchical MOG
C)MOG
d)Color model
e)CB
f)g) hierarchical MOG
C)MOG
d)Color model
e)CB
f)g) hierarchical MOG
Experimental Results
Experimental Results
Experimental Results
Conclusion
• The block-based stage can enjoy high speed
processing speed and detect most of the
foreground without reducing TP rate.
• Pixel-based stage can further improve the
precision of the detected foreground object
with reducing FP rate.
• Short-term information is employed to
improve background updating