International Journal of Engineering Trends and Technology (IJETT) – Volume 5 Number 6 - Nov 2013
Video Stabilization and Enhancement
* V.S.R Kumari 1
Anusha Korrapati 2
Professor & HOD, Dept. of ECE, Sri Mittapalli College of Engineering, Guntur, A.P, India.
PG Student (M. Tech), Dept. of ECE, Sri Mittapalli College of Engineering, Guntur, A.P, India.
1
2
Abstract: The poor image quality of many video surveillance cameras effectively renders them
useless for the purposes of identifying a person, a license plate, etc. Hence, many researchers
study such drawbacks to enhance the quality of casual videos. In this paper propose a
matching algorithm to stabilize causal videos directly without the need to estimate the motion.
A stable output video will be attained without any unanticipated effects during video recording.
Keywords: Image processing, video stabilization
The first phase the purpose is to
1. Introduction
Recently, the market of handheld
estimate the motion between frames. After
camera has growth rapidly. However,
that, the parameters of estimated motion
video capturing by non-professional user
which is obtained from the first phase will
normally
unanticipated
be sent to motion compensation, where
effects. Let us assume a camera rigidly
removing the high frequency distortion
mounted on a vehicle in motion. If the
and calculating the global transformation,
motion of the vehicle is smooth, so will be
which is very important to stabilize the
the corresponding image sequence taken
current frame. Next, warping will be done
from the camera.
by image composition for the frame under
will
lead
to
In the case of small unmanned aerial
processing.
2. Architecture of the system
imaging system, and off road navigating
ground vehicles, the onboard cameras
experience
sever
jitter
and
vibration.
Figure
diagram
1
shows
of
video
the
basic
block
stabilization
and
Consequently, the video images acquired
enhancement. It mainly consists feature
from
selection;
these
preprocessed
platforms
to
have
eliminate
induced
variations
analysis.
Generally
before
the
the
to
be
jitter
human
processes
of
stabilization have to go through three
phases namely
1. Motion estimation
select
the
correspondence
between points, estimated transform, and
smoothening.
A)
Feature
selection
(Detection
of
corners)
This stabilization algorithm requires
features
to
estimate
motion
between
2. Motion smoothing
frames. Since features are required to
3. Image composition.
persist from frame to frame, it is a good
idea to have a feature selection method
that will choose parts of the image that
ISSN: 2231-5381
http://www.ijettjournal.org
Page 298
International Journal of Engineering Trends and Technology (IJETT) – Volume 5 Number 6 - Nov 2013
will likely not change in appearance from
corner detectors have been developed in
frame to frame. An ideal feature is distinct
that
and unique, is identifiable in the next
Detector System Object is used to find
frame, and provides new information to
corner
the stabilization process.
Detection which is one of the fastest
Many applications require relating
Corner
detector
values
using
system.
Corner
Harris
Corner
algorithms to find corner values.
two or more images in order to extract
B)
Select
information from them. For example, if
points
correspondence
between
two successive frames in a video sequence
After the salient points from each
taken from a moving camera can be
frame are obtained the correspondence
related,
extract
between the points. For each point, the
information regarding the depth of objects
matching of lowest cost between the
in the environment and the speed of the
points that existed in frame A and B are
camera.
of
also needed to be found for all points.
comparing every pixel in the two images is
Hence, it is necessary to divide the
computationally
sequence of frames image into 9×9 block.
it
is
The
possible
brute
force
to
method
prohibitive
for
the
majority of applications.
The matching cost means the distance
Intuitively, one can image relating
two images by matching only locations in
the
image
that
To find this cost, the technique of
interesting. Such points are referred to as
Sum of Squared Differences (SSD) can be
interest points and are located using an
used
interest
images.
relationship
in
detector.
between
some
pixel.
way
point
are
between frame A and B measured in
Finding
point
consecutive
in
frame
frame
A
is
compared with the points in frame B to
performed using only these points. This
find the lowest matching cost or in other
drastically
words the shortest distance between them
the
is
Each
the
then
reduces
images
a
between
required
computation time.
Many
different
measured in pixels.
interest
point
detectors have been proposed with a wide
range of definitions for what points in an
image are interesting. Some detectors find
points of high local symmetry; others find
areas of highly varying texture, while
others locate corner points. Corner points
are interesting as they are formed from
two or more edges and edges usually
define the boundary between two different
objects or parts of the same object. Many
ISSN: 2231-5381
Figure 1 Block diagram of video stabilization and
enhancement
http://www.ijettjournal.org
Page 299
International Journal of Engineering Trends and Technology (IJETT) – Volume 5 Number 6 - Nov 2013
C) After that, we determine the affine
image
transformations
between
C.
Select correspondence between points
all
After the salient points from each
neighboring frames of a video sequence
frame
using
and
between the points. For each point, the
Consensus (RANSAC) procedure applied
matching of lowest cost between the points
to point correspondences between two
that existed in frame A and B are also needed
images and finally we warp the video
to be found for all points. Hence, it is
frames to achieve a stabilized video.
necessary to divide the sequence of frames
3. RESULTS AND DISCUSSION
image into 9×9 block. The matching cost
a
Random
Sampling
In this section, we discuss the results
attained based proposed methodology.
are
obtained
the
correspondence
means the distance between frame A and B
measured in pixel.
A. Read the frame from captured video
To find this cost, the technique of
In this paper, we read first two frames
Sum of Squared Differences (SSD) can be
from
captured
video
for
stabilization
used between the consecutive frame images.
images
Each point in frame A is compared with the
because of grayscale images improve the
points in frame B to find the lowest matching
speed. Below figure2 shows captured
cost or in other words the shortest distance
frames from a video.
between them measured in pixels. Figure 4
process.
We
read
intensity
Frame A
represents the corresponding points between
Frame B
frames.
A
B
Figure 2 Captured frames from a video
B. Detection of corners
Initially, an algorithm is developed
based on FAST corner detection algorithm
to identify all strong corner points from
each frame. Sample of detected corner
Figure 4 corresponding points between frames
D. Estimate
from
correspondences
points obtained from each frames as
shown in below figure2.
transformation
Many of the point correspondences
obtained in above step are incorrect. But we
can still derive a robust estimate of the
geometric transform between the two images
using
the
Random
Sample
Consensus
(RANSAC) algorithm. This algorithm searched
through
Figure 3 Detected corner points obtained from each frame
ISSN: 2231-5381
the
correspondences
given
set
specifically
of
valid
point
linear
correspondence as in Figure 5.
http://www.ijettjournal.org
Page 300
International Journal of Engineering Trends and Technology (IJETT) – Volume 5 Number 6 - Nov 2013
A
B
Figure 7 Stabilized output
Figure 5 Correct correspondences according to RANSAC
4. Quality Measurements
E. Transform and smoothening
The
The main goal of this step is to correct
the distortion between the two frames by
finding a transformation that will be done by
applying an object system which returns
affine
transform.
Figure
6
shows
color
composite of affine transform and its S-R-T
output
video
quality
is
also
measured based on the proposed methods.
This is evaluated based on Singular Value
Decomposition (SVD) based grayscale Image
value and graphical measurement.
1. SVD Based Grayscale Image Quality
Singular value decomposition (SVD) is
transform.
Color composite of affine and s-R-t transform outputs
developed as a new measurement that can
express the quality of distorted images either
graphically that is in 2D measurement or
50
numerically as a scalar measurement, both
near and above the visual threshold. The
100
experiments
150
here
utilized
SVD
based
measurement that outperformed the normally
used PSNR and corresponding results as
200
shown in table 1. The following Equation
50
100
150
200
250
300
Figure 6 color composite of affine transform and S-R-T
transform.
represented the computed value for this
purpose:
F. Stabilized output
Further, the raw mean video frames
and
the
mean
of
corrected
frame
are
computed as in Figure 6. The left image
showed the mean of the raw input frames
that resembled the distorted original video
frame due to extreme jittery. On the right side
is the mean of the corrected frames with less
distortion. This proven that the stabilization
Where:
Dmid represents the midpoint of the sorted DiS
k is the image size
n is the block size
M –SVD is the measurement of Singular
value decomposition.
algorithm worked well.
ISSN: 2231-5381
http://www.ijettjournal.org
Page 301
International Journal of Engineering Trends and Technology (IJETT) – Volume 5 Number 6 - Nov 2013
2. C. Harris and M.J. Stephens, “A combined
Table 1 Criteria of sample video & Results (Type: RGB &
Extension: AVI)
Sample Frame
Frames
Raw input mean
corner and edge detector”, Proc of Alvey
Vision Conference, pp 147–152, 1988.
Computational
Numerical
Quality
Time (S)
Measure
Value
8.078
52.98%
47.02%
70
3. Anu Suneja and Gaurav Kumar . “An
Experimental
Study
of
Edge
Detection
Methods in Digital Image”, Global Journal
of Computer Science and Technology, 10(2),
2010.
4. http://www.mathworks.com/products/com
putervision/demos.html?file=/products/de
4. Conclusion
We
have
described
a
mos/shipping/vision/videostabilize_pm.ht
simple
and
computationally efficient technique for
ml.
5. Fischler, MA; Bolles, RC. "Random Sample
video stabilization and enhancement. The
Consensus: A Paradigm for Model Fitting
motion between video frames is modeled
as
a
global
affine
transform
with Applications to Image Analysis and
whose
Automated Cartography." Comm. of the
parameters are estimated using standard
matching techniques. A temporal mean or
ACM 24, 1981.
6. Tordoff, B; Murray, DW. "Guided sampling
median filter is then applied to this
and consensus for motion estimation." 7th
stabilized video to yield a single high
quality
frame.
We
have
shown
European Conference on Computer Vision,
the
effectiveness of this technique on both
2002.
7. J. Jin, Z. Zhu, and G. Xu. “Digital video
synthetically generated and real video.
sequence
This technique should prove useful in
motion
enhancing the quality of low-grade video
stabilization
estimation
and
based
on
inertial
2.5D
motion
filtering”, Real-Time Imaging, 7(4):357–365,
surveillance cameras.
2001.
8. http://siddhantahuja.wordpress.com/tag/
Acknowledgements
sum-of-squared-differences/
The authors would like to thank the
anonymous reviewers for their comments
which were very helpful in improving the
quality and presentation of this paper.
References:
1. M.
Gleicher
9.
M. Pilu. “Video stabilization as a variation
problem and numerical solution with the
Viterbi
method”.
In
Proceedings
of
Computer Vision and Pattern Recognition,
pp 625–630, 2004.
and
F.
Liu.,
“Re
10. Aleksandra
Shnayderman,
Alexander
cinematography: Improving the camerawork
Gusev, and Ahmet M. Eskicioglu “An SVD-
of casual video,” ACM Transactions on
Based Grayscale Image Quality Measure for
Multimedia Computing, Communications,
Local and Global Assessment “,IEEE 15(2),
and Applications, 5(1), pp 1- 28, 2008.
2006.
ISSN: 2231-5381
http://www.ijettjournal.org
Page 302
International Journal of Engineering Trends and Technology (IJETT) – Volume 5 Number 6 - Nov 2013
Authors Profile:
V.S.R KUMARI is working as a
Professor
&
department
Head
of
the
of
Electronics
&
Communication
Engineering
in
College
of
Sri
Mittapalli
Engineering, Guntur, A.P, India.
She has over 18 years of teaching experience and
she is carrying out her research under Andhra
University, Vishakhapatnam, AP, India.
Anusha Korrapati is Pursuing
her M. Tech from Sri Mittapalli
College of Engineering, Guntur,
A.P, India in the department of
Electronics & Communications
Engineering (ECE).
ISSN: 2231-5381
http://www.ijettjournal.org
Page 303