Performance Investigation on Bilateral Filter with
Confidence Based over Spatial Correlation-based
Optical Flow for Image Reconstruction
Darun Kesrarat↑ and Vorapoj Patanavijit
Abstract— Optical flow in image sequences gives essential
data on motion structure. It is relevant in several areas such as
robot vision, video coding, and super resolution reconstruction.
Even though this realm has been concentrated for more than a
decade, reduction the glitch in estimation stands a difficult
trouble. Many techniques were proposed to enhance the
performance and one of the most is bilateral filter. This paper
presents the performance study of bilateral filter and bilateral
filter together with confidence based over spatial correlationbased optical flow where the quality of the reconstructed images
from the result of motion vector is highly focused. We perform
an experiment over several standard sequences with several
noise levels in contamination to study the robustness of each
robust technique and used PSNR as an indicator.
I. INTRODUCTION
Optical flow is used to determine the motion vector (MV) of
the movement sequences in a level of pixel based. Several
areas such as motion tracking, video coding, image super
resolution reconstruction, edge detection, and object
segmentation use optical flow techniques in advance to
archive the outcome. From the past decade, a lot of optical
flow algorithms are proposed and this paper focuses on the
achievement over spatial correlation based optical flow
(SCOF) [1].
Under noisy condition, the performance of optical flow is
dropped. There are many techniques were proposed to
increasing the performance such as gradient orientation
information [7] and confidence-based optical [2, 5, 6]. And
one of the most techniques is bilateral filter [8,9,10]. Bilateral
filer is a robust edge-preserving filter that was referenced in
many computer vision and image processing.
According to the performance investigation of D. Kesrarat
and V. Patanavijit [3, 4] over confidence-based optical flow
for high reliability [2], we found that it presented the
performance upgraded over several model. Then, we also
investigate the performance of bilateral filter in combination
with this confidence based model.
In our experiment, we address the performance investigation
over different standard sequences with 3 simulated levels of
Additive White Gaussian Noise (AWGN) (25, 20, and 15 dB)
by using PSNR as a performance indicator. The rest of the
↑
Department of Information Technology, Vincent Mary School of
Science and Technology, Assumption University, Bangkok, Thailand.
Email: darunksr@gmail.com
↑↑
The Corresponding Author, Department of Computer and Network
Engineering, Vincent Mary School of Engineering, Assumption University,
Bangkok, Thailand. Email: patanavijit@yahoo.com
978-616-361-823-8 © 2014 APSIPA
↑↑
paper is grouped as follows. Section II clarifies the related
optical flow algorithm. Section III introduces the model of
bilateral in combination with confidence based model (BC).
Section IV introduces the performance result and conclusion.
II. RELATED ALGORITHMS OF OPTICAL FLOW
This section clarifies related algorithms of optical flow.
A. Spatial Correlation-Based Optical Flow (SCOF) [1]
The mechanic in correlation-based optical flow is similar to
block matching that partition an image into non-overlapped
with the fix size of rectangular block. Then, the best candidate
of each block in pixel-based level is determined as motion
vector.
B. Confidence Based Model
This algorithm was proposed in 2008 by R. Li and F. Yu. It
presents the use of bidirectional symmetry in bi-direction
(forward velocity (MV of frame ff+1) and backward
velocity (MV of frame f+1f)). Both directions of MV are
used to determine the reliable rate (R) by confident
measurement defines as:


|| vln s, f   vln ( s  vln ( s, f ), f  1) ||
 (1)
Rln s, f   exp  
n
n
n
 (|| vl ( s, f ) ||  || v  ( s  vl ( s, f ), f  1) ||) / 2   
l


Where l is forward, l- is backward,  prevents the
denominator from zero, s is position of current pixel, and v is
MV in horizontal axis.
Next, the final MV ( v and u ) is computed in average
result from reliability base on pre-denoted neighborhood
(N(s0)) defines as :

 

(2)
vln ( s0 )  
Rln ( si )vln ( si ) 
/
Rln ( si ) 

 s 



 i N ( s0 )
  si N ( s0 )

This model shows very effective performance over several
conditions even noise or non noise contamination sequences.
C. Bilateral Filter [10]
This model was introduced by C. Tomasi and R. Manduchi
that was applied several computer vision and image
processing [8,9,10]. The bilateral kernel is defined as:-
APSIPA 2014
 ( x  n)  exp (
| n |2
| I ( x  n)  I ( x ) 2 |

)
2 a2
2 b2
(3)
Where δa is standard deviation of signal v(x) multiple by 7
and δc is standard deviation of signal I(x).
Bilateral filter is applied to adjust the computation as
follows:vb ( x) 
1
W
 v( x) ( x  n)
(4)
|n|  N
Where ɸ() is bilateral Gaussian kernel, and N is number of
neighborhood. In our experiment, we set N equal to ±7. W is
the kernel normalization factor is defined as:W 

gradient of image frame f and f+1
III. BILATERAL IN COMBINATION WITH CONFIDENCE BASED
MODEL (BC)
Under BC, the result of bilateral filter bidirectional
symmetry is applied to lift the performance in achievement of
MV under the noisy condition.
In BC, the MV on both direction in forward and backward
are obtained by using BCOF based on the concept of
bidirectional symmetry in CHR. Next, the bilateral filter from
is used to compute over the MV of previously forward MV
and backward MV as the unit vector orientation. Next, the
final MV is computed by using the reliability concept in
confidence based model (Eq. 1 and Eq. 2) but the values of
the unit vector orientation on bi-direction are used instead of
the traditional bi-direction MV in computation as shown in
Fig.1.
IV. PERFORMANCE RESULT AND CONCLUSION
We conduct an experiment over 4 conventional QCIF
(176x144 with 100 frames) sequences. There are AKIYO,
COASTGUARD, CONTAINER, and FOREMAN. We also
simulate AWGN in 3 levels at 25dB (little noise), 20dB
(medium noise), and 15dB (huge noise) over these sequences.
So, 16 sequences totally are used in an experiment. For SCOF
algorithm, we set ±3 for neighbors block size and set ±7 for
window search area with 0.5 bilinear interpolation sub-pixel
displacements for an experiment. Lastly, the performances
evaluation over the effectiveness in PSNR is focused on our
experiment by reconstruction image frames in comparison
with their corresponding one in the original sequence defines
as.
1 m 1

mn i  0
n 1
  A ( i , j)  B ( i, j)
2
(6)
j 0
PSNR  10 x log 10 (
G 2I
)
Err
Take Image Sequences
(Frame f and f+1)
(5)
 ( x  n)
|n|  N
Err 
In Fig. 2-17 show the values of PSNR of the experiment
sequences.
From the experiment result, we found that the bilateral
impacts the performance on reconstruction image especially
over noisy sequence. With increasingly of noise, the bilateral
generates better performance than SCOF.
We also found that the confidence model helps to
increasing the performance of bilateral filer.
(7)
Where Err is mean squared error. A is rebuilding image and
B is original image at resolution (m×n). GI is 255 (value of 8
bits image).
Compute MV on
forward direction (f -> f+1) and backward
direction(f+1 -> f)
Based on SCOF algorithm
MV in forward and backward direction
Apply bilateral filter on forward MV and
backward MV
Bilateral vector in forward and backward
Compute Final MV with reliability rate by using
bilateral vector result of forward and backward
Fig. 1 Process of BC algorithm.
Fig. 2 PSNR of AKIYO (no noise) sequence.
Fig. 3 PSNR of AKIYO (AWGN 25 dB) sequence.
Fig. 4 PSNR of AKIYO (AWGN 20 dB) sequence.
Fig. 5 PSNR of AKIYO (AWGN 15 dB) sequence.
Fig. 6 PSNR of COASTGUARD (no noise) sequence.
Fig. 8 PSNR of COASTGUARD (AWGN 20 dB) sequence.
Fig. 7 PSNR of COASTGUARD (AWGN 25 dB) sequence.
Fig. 9 PSNR of COASTGUARD (AWGN 15 dB) sequence.
Fig. 10 PSNR of CONTAINER (no noise) sequence.
Fig. 12 PSNR of CONTAINER (AWGN 20 dB) sequence.
Fig. 14 PSNR of FOREMAN (no noise) sequence.
Fig. 16 PSNR of FOREMAN (AWGN 20 dB) sequence.
Fig. 11 PSNR of CONTAINER (AWGN 25 dB) sequence.
Fig. 13 PSNR of CONTAINER (AWGN 15 dB) sequence.
Fig. 15 PSNR of FOREMAN (AWGN 25 dB) sequence.
Fig. 17 PSNR of FOREMAN (AWGN 15 dB) sequence
REFERENCES
[1] Reed TR, “Digital Image Sequence Processing,
Compression, and Analysis,” Chemical Rubber Company
(CRC) Press, 2005.
[2] R. Li and S. Yu, “Confidence Based Optical Flow
Algorithm for High Reliability” In proceeding of IEEE
International Conference on Acoustics, Speech, and
Signal Processing (ICASSP), pp. 785-788, 2008.
[3] D. Kesrarat. and V. Patanavijit, “Performance Evaluation
of Differential Optical Flow Algorithms Based on High
Confidence Reliability with Sub-Pixel Displacement,” In
proceeding of the 33rd Electrical Engineering Conference
(EECON), pp. 1305-1308, 2010.
[4] D. Kesrarat. and V. Patanavijit, “Experimental
Performance Analysis of High Confidence Reliability
Based on Differential Optical Flow Algorithms over
AWGN Sequences with Sub-Pixel Displacement,” In
proceeding of Intelligent Signal Processing and
Communication Systems (ISPACS), pp. 1-6, 2011.
[5] D. Kesrarat and V. Patanavijit, "A Novel Robust and
High Reliability Spatial Correlation Optical Flow
Algorithm Based on Median Motion Estimation and
Bidirectional Symmetry Flow Technique", In proceeding
of IEEE International Symposium on Intelligent Signal
Processing and Communication Systems (ISPACS), pp.
1-6, 2011.
[6] D. Kesrarat and V. Patanavijit, " Experimental Analysis
of Performance Comparison on Both Linear Filter and
Bidirectional Confidential Technique for Spatial Domain
Optical Flow Algorithm", The ECTI Transactions on
Computer and Information Technology (ECTI-CIT), vol.
7(2), pp. 157-168, 2013.
[7] T. Kondo and W. Kongpawechnon, "Robust Motion
Estimation Methods Using Gradient Orientation
Information", ScienceAsia 35 (Journal of The Science
Society of Thailand), pp 196-202, 2009.
[8] J.Sun, Y. Li, S. Kang, and H.-Y. Shun, "Symmetric
stereo matching for occlusion handling," In proceeding of
IEEE Conf. Comput. Vis.Pattern Recognit., pp. 399-406,
2005.
[9] S. Paris, P. Kornprobst, J. Tumblin, and F. Durand,
"Bilateral filtering: Theory and Applications,"Found.
Trends Compt. Graph. Vis., Vol.4, No.1, pp.1-73, 2008.
[10] M. G. Mozerrov, "Constrained Optical Flow Estimation
as a Matching Problem," IEEE Transactions on Imiage
Processing, vol.2, No.5, pp. 2044-2055, 2013.