International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: editor@ijaiem.org
Volume 4, Issue 1, January 2015
ISSN 2319 - 4847
A detailed analysis on object tracking technique
with newly evolved optical flow methods
Siva kumar.karpurapu1, Sravani.boggavarapu2
1
Affiliated to the VRS&YRN Engineering & Technology, vadaravu road, Chirala
2
Affiliated to the Vellore Institute Of Technology, Vellore
ABSTRACT
A common approach for real-time segmentation of slow moving regions in image sequences involves “background
subtraction,” or thresholding the error between an estimate of the image without moving objects and the current image. The
numerous approaches to this problem differ in the type of background model used and the procedure used to update the model.
This paper discusses modeling each pixel as an “adaptive Gaussian mixture models” with object detection techniques as
advance improvements in Lucas and KanadeOptical flow method. In General moving object can be detected by using either
background subtraction or optical flow methods. In all the cases we have seen so far are dealing with moving camera, by these
methods we may get some false-positives in stationary viewpoint assumption. So we implemented a new approach and
compared the results with the existing design.
Keywords :- LK Optical Flow(Lucas and Kanade Optical flow)
1. INTRODUCTION
In modern era, vehicles have been developed with safety critical functions and advance driver assistance features, such
as lane departure warning, forward collision warning system , and park assistance systems. Fundamentally, By visionbased on-boardelectronics can able to provide these intelligent functions which utilize moving object detection
algorithms with aid of other technologies [2].In the past, many computational barriers had limited the real-time video
processing applications with many complexities. As a consequence, most successful systems were either too slow to be
practical, or implemented with restricting themselves to very controlled situations. Recent computers have developed
gradually in researchhad considered more complex, robust models for real-time analysis along with streaming data.
These new methods may allowengineers to begin modeling in real world processes under different environmental
conditions .Our goal is to create a robust, adaptive tracking system that is flexible enough to handle variations in
lighting, moving scene clutter, multiple moving objects and other arbitrary changes to the observed scene. The resulting
tracker is primarily geared towards scene-level video surveillance applications.
2. PREVIOUS WORK AND CURRENT SHORTCOMINGS
In the past years of computer vision arena, many number of conclusions made to detect moving objects. One among
those is the estimation of moving objects by accumulating frame differences [3]. The frame difference method has
modest computational loads with highly adaptive background model, which can adapt to changes on background very
fast because the background is based solely on the previous frame. A standard method of adaptive back grounding is
averaging the images over time, this method creates a background approximation which is similar to the current static
Frame except where motion occurs. While this become effective in some situations where objects move continuously
and the background is visible a significant portion of the time. This may not be robust to some frames with many
moving objects particularly if they move slowly. It also cannot handle bimodal backgrounds, which recovers slowly
when the background is uncovered, and has a single, predetermined threshold for the entire Frame. Changes in scene
lighting can cause problems for many adaptive back grounding methods. Ridder et al.[12] modelled each pixel with a
Kalman Filter which made their system more robust to lighting changes in the scene. By considering these
disadvantages in Adaptive Back ground Models, we came across with the other alternative by Lucas and Kanade
Optical flow method. To cope with the above characteristics To cope with this characteristics, approximated median
[4] and adaptive Gaussian mixture model [5], [6] had been suggested. Unlike the frame difference model which only
accumulates the differences between two subsequent frames, these models keep tracks on background changes in
theform of a frame consisted by median values and parameters for Gaussian functions respectively.
Once the
background frame has been established, these models become generally strong at suppressing background noises such
as waving trees, movingclouds, and rain drops.
Volume 4, Issue 1, January 2015
Page 68
International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: editor@ijaiem.org
Volume 4, Issue 1, January 2015
ISSN 2319 - 4847
3. BLO4CK DIAGRAM
In this Paper we proposed a new design to Optimize the previous designs [1]. The main characteristic of this design
gives the Pre-processing of the images (Frames) before applying to the Optical flow algorithm, this gives the better
result as if one considered as without Pre-Processing the frames. The below fig(a). shows the detail flow of the new
proposed algorithm for optimization by using an Adaptive Gaussian Mixture Models and also by Optimal filtering
Techniques.
Fig(a): Overall Detection Flow for Proposed Design
On Determining the movements of an object, which has been studied in the form of finding optical flow vectors of the
object [7], [8]. This may focus more on the moving foreground object, rather than background image, by sorting out
pixels with unique magnitude and direction vectors. It is later improved to implement non-linearized constraint
ofoptical flow using “warp” theory [9], [10]. By using the “Wrap” Theory, one can visualise the Non-Linear
characteristics of Pixels at the slowlyMovements also. Friedman and Russell [11] have recently implemented a pixelwise EM framework for detection of vehiclesthat bears the most similarity to our work. Their method attempts to
explicitly classify the pixel values into three separate, predetermined distributions corresponding to the road color, the
shadow color, and colors corresponding to vehicles. Their attempt to mediatethe effect of shadows appears to be
somewhat successful, but it is not clear what behaviour their system would exhibit for pixels which did not contain
thesethree distributions. For example, pixels may present a single background color or multiple background colors
resulting from repetitive motions, shadows, or reflectance’s.
4. OUR APPROACH
Rather than applying the filters after the Optical Flow method , we simply added one optimal Filter which can be
made in such a way that to reduce the noise and all the frame errors and act like as Pre-processing the frames before .
We have concluded not only on the fact that an optimal filtering must be performed at every pyramidal level, but also
introduced a novel method for according the filter to the processed context. Also, from our study we have found that the
Gaussian filter performs considerably better among different other filters. So we have proposed the new algorithm by
adding Gaussian filtering at each Pyramidal Level in optical Flow technique. Generally, in pyramidal optical flow
computation, the input images are filtered only at the beginning and at the following levels, the images are being
resized from that base. Our first experiment concerned the proper use of filtering, not only at the initial level, but
subsequently at each pyramidal level. As several test sequences, together with the reference ground truth were
available, an improvement of the error was obtained.
5. MOVING OBJECT DETECTION
Based on the constant and the variance of each mixture of Gaussians, we determine which Gaussians may correspond
to background colors. Pixel values that do not fit the background distributions are considered foreground until there is a
Gaussian that includes them with sufficient, consistent evidence supporting. Our system adapts to deal robustly with
lighting changes, repetitivemotions of scene tracking elements, tracking through cluttered regions, slow-moving
objects, and introducing and removing objects from the scene.Slowly moving objects take longer to be incorporatedinto
Volume 4, Issue 1, January 2015
Page 69
International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: editor@ijaiem.org
Volume 4, Issue 1, January 2015
ISSN 2319 - 4847
the background, because their color has a largervariance than the background. Moving detection of objects may
depends on mainly on the Background subtraction and Optical flow methods until the movement of entire frame has
been corrected first. This correction is done by background compensation routine which takes the two subsequent
frames by the adaptive Gaussian mixture model.
6. THE MODEL
The model here we adapted is vary with the normal Gaussian mixture model which can take the If each pixel resulted
from a particular surface under particular lighting, a single Gaussian would be sufficient to model the pixel value while
accounting for acquisition noise. If only lighting changed over time, a single, adaptive Gaussian per pixel would be
sufficient. In practice, multiple surfaces often appear in the view frustum of a particular pixel and the lighting
conditions change. Thus, multiple, adaptive Gaussians mixture models are necessary. We use a mixture of adaptive
Gaussians to approximate this process. Our backgrounding method contains two significant parameters – α, the
learning constant and T, the proportion of the data that should be accounted for by the background. Where α is the
learning rate2, and T is the critical parameter for this model which gives a measure of the minimum portion of the
data that should be accounted for by the background. This takes the “best” distributions until a certain portion, T, of the
recent data has been accounted for. If a small value for T is chosen, the background model is usually unimodel. If this is
the case, using only the most probable distribution will save processing. If T is higher, a multi-modal distribution
caused by a repetitive background motion (e.g. leaves on a tree, a flag in the wind, a construction flasher, etc.) could
result in more than one color being included in the background mode. As the parameters of the mixture model of each
pixel change, we would like to determine which of the Gaussians of the mixture are most likely produced by
background processes.
7. GAUSSIAN FILTERING
Since in our extensive research on the subject, we could not find some specifications of the optimal type of filter.so, we
have considered the most referenced 2D ones, as Gaussian, Mean, Median, Bilateral, Adaptive Noise Removal or the
High Boost filter. From these experimental results we had concluded that theGaussian filtering is the most suitable in
regarding to our design, on the basis ofcomputed average angular error and average endpoint error. As the Gaussian
filter was the suitable for pre-Processing the input images, we have investigated the relation between the standard
deviation values of the Gaussian function and the image contents. From the plotted results, we have observed that there
is no fixed σ value achieving the lowest error for any input sequence. Based on empirical observations, as the variation
of error with standard deviation, we have established a correlation. Our novel method for selecting the σ value consists
in observing the shape of the Gaussian function using a standard deviation of 1 and extracting the highest value.
Comparing this reference point with the image average intensity can give an indication on the suitable value to be used.
Also, we have found that the ratio between the image intensity and the highest Gaussian value can give an indication
on the proper σ value. Finally, we concluded on the fact that computing the filter standard deviation from image
characteristic offers a more accurate optical flow computation.
We made the optical flow as robust and accurate by the below three constrains:
 Local image constraints on motion
 Robust statistics
 Spatial coherence
7.1 localimage constraints on motion
 Brightness constancy: I ( x  u, y  v, t  1)  I ( x, y, t )  0 equation (1)
 Linearized brightness constancy: Deviation from brightness constancy (we want to minimize it).
  I ( x  u, y  v, t  1)  I ( x, y, t ) equation (2)
 LIN  I x () u ()  I y () v()  I t ()

 Ix

Iy
2
LIN
It

u 
 v  ()  ( x, y, t )
 
1 
u 
 [u v 1] J  v   0
1 
 Averaged Linearized brightness constancy
Volume 4, Issue 1, January 2015
Page 70
International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: editor@ijaiem.org
Volume 4, Issue 1, January 2015

2
LIN  GAUSS
u 
 [u v 1] (G  J )  v   0
1 
ISSN 2319 - 4847
equation (3)
 gradient constancy
I ( x  u, y  v, t  1)  I ( x, y, t )  0
These are the major constraints with which the optical flow method will work effectively. We have studied on these
three constraints and identified some failures of these three constraints.
a) Motion is not observable in the uniform regions.
b) We can estimate only one flow component i.e. aperture problem
c) Occlusions- we have not seen where some points were moved.
Occluded regions are marked in red
7.2 Robust statistics
It can be obtained by the combination of two flow constraints.Here, one can visualize the difference of usual and robust
statistics.
Easy to analyse and minimize
Sensitive to outliers
Fig(b): usual Flow statistics
Volume 4, Issue 1, January 2015
Page 71
International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: editor@ijaiem.org
Volume 4, Issue 1, January 2015
ISSN 2319 - 4847
Smooth: easy to analyse
Robust: in presence of outliers
Fig(c): robust flow statistics
7.3 Spatial coherence
Information about motion is missing in some points at that time we need to check with the spatial coherency first.
Constraints do not hold everywhere we need methods to combine them robustly.
 Need homogenous propagation
 Robust statistics in homogenous propagation.
 Bilateral filtering- combines information from regions with similar flow and similar intensities Handles occlusions
The global optimization can be done very fast through multi grid approach and Full multi grid approach.
Here, we have introducing some new optical flow approaches,
 Patch comparison techniques.
 Gradient based techniques
These are the two different approaches which will be implemented in the optical flow by considering all the parameter
failures and can be implemented in a new manner.
8. EXPERIMENTAL RESULTS
To evaluate our approaches some experiments were carried out, in our experiments our approach is performed for the
three test videos. And compared the results of the existing designs given in [1][2][3] and our experimental results and
parameters are tabulated below.
Fig(d)Test videos
8.1 Tables
Table 1:parameters of the test video
Volume 4, Issue 1, January 2015
Page 72
International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: editor@ijaiem.org
Volume 4, Issue 1, January 2015
ISSN 2319 - 4847
We estimated the processing times for the existing methods given [1][2] and with our proposed algorithm in table2.
Table 2:processing times for different background modeling method
Background
model
Change
Detection
Method
Median value
Histogram
based
adaptive
Gaussian
mixture
models
Gait
video1
VIDEO
Gait
video2
Car
parking
13.9098
161.6029
367.4683
1.272
26.16
11.0673
0.4282
3.7902
4.7036
0.352
2,56
3.45
Table 3:processing time for the foreground extraction for the different methods
9. CONCLUSION
In this paper we have tested back ground modeling and the optical flow techniques on various videos and compared,
the various pros and corns of the various constraints of the optical flow methods which has given a numerous inputs for
further study of implementing a robust optical flowdesigns, modeling each pixels as an adaptive Gaussian mixtures has
given more accurate results than the other modeling techniques. And concentrates mainly on the light change
conditions of the observed scenes which will gives a better results for climatic calculations of frames.
REFERENCES
[1] J. Kwon and D.-S. Kim, Member, IEEE Korea Electronics Technology Institute, Seongnam, Gyeonggi-do, Rep. of
Korea “Detecting Moving Objects From Slowly Moving Viewpoint for Automotive Rear View Camera
Systems”.
[2] L. Li, J. Song, F.-Y. Wang, W. Niehsen, and N.-N. Zheng, “IVS 05: new developments and research trends for
intelligent vehicles”, Intelligent Systems, IEEE, vol.20, no.4, pp.10,14, July-August 2005.
[3] R. Jain, and H.-H. Nagel, “On the Analysis of Accumulative Difference Pictures from Image Sequences of Real
World Scenes”, Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol.PAMI-1, no.2, pp.206,214,
April 1979.
Volume 4, Issue 1, January 2015
Page 73
International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: editor@ijaiem.org
Volume 4, Issue 1, January 2015
ISSN 2319 - 4847
[4] N. McFarlane, and C. Schofield, “Segmentation and tracking of piglets in images”, British Machine Vision and
Applications, pages 187-193, 1995.
[5] N. Friedman, and S. Russell, “Image segmentation in video sequences: a probabilistic approach”, in Proc. 13th
Conf. on Uncertainty in Artificial Intelligence, 1997.
[6] C. Stauffer, and W. Grimson, “Adaptive background mixture models for real-time tracking”, Proceedings IEEE
Conference on Computer Vision and Pattern Recognition, CVPR 1999, pages 246-252, 1999.
[7] B. Horn, and B. Schunck, “Determing optical flow”, Artificial Intelligence, vol. 17, pp.185-203, 1981.
[8] B. Lucas, and T. Kanade, “An iterative image registeration technique with an application to stereo vision”, in Proc.
Seventh International Joint Conference on Artificial Intelligence, pp.674-679, Vancouver, Canada, August 1981.
[9] M. J. Black, and P. Anandan, “Robust dynamic motion estimation over time”, in Proc. 1991 IEEE Computer
Society Conference on Computer Vision and Pattern Recognition, pp. 292-302, Maui, HI, June 1991.
[10] T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, “High accuracy optical flow estimation based on a theory for
warping”, in Proc. European Conference on Computer Vision (ECCV), vol. 3024, pp.25-36, May 2004.
[11] Nir Friedman and Stuart Russell. “Image segmentation invideo sequences: A probabilistic approach,” In Proc. of
theThirteenth Conference on Uncertainty in Artificial Intelligence(UAI), Aug. 1-3, 1997.
[12] Christof Ridder, Olaf Munkelt, and Harald Kirchner.“Adaptive Background Estimation and Foreground
Detectionusing Kalman-Filtering,” Proceedings of InternationalConference on recent Advances in Mechatronics,
ICRAM’95, UNESCO Chair on Mechatronics, 193-199, 1995
AUTHOR
Siva Kumar Karpurapu received theB.Techfromchiralaengineeringcollege in 2010 and
M.TechfromVRS&YRN Engineering & Technology, vadaravu road, Chirala in 2014. During 20102013 he stayed in embedded processing research laboratory (EPL)in JNTUH to study the recent
technologies in the automotive domain and also in the communication systems .he now with L&T
Technology Services in automotive department.
Volume 4, Issue 1, January 2015
Page 74