Anomaly Detection from video using
Artificial Neural Network
Sanjivani Shantaiya
Dept. of CSE,DIMAT
Raipur
sanjivaninice@gmail.com
Kesari Verma
Dept. of Application,NIT
Raipur
keshri.verma@gmail.com
Kamal Mehta
Dept. of CSE,NIRMA University
Ahmedamad
kmehta28@yahoo.co.in
Abstract
Neural Network has been widely used for classification purpose. In this paper we propose a
novel method for anomaly detection in video dataset using Neural Network. In this study Gabor
filter is used for feature extraction of anomaly and normal data of video. Video data are itself
large in size and gabor filter extraction creates a large set of features for each frame of video. In
order to reduce the dimension we have used principal component analysis for further processing.
Neural Networks iteratively trained the neurons for normal and anomaly dataset. Anomalous
object can be detected effectively through this trained network.
Keywords:- Anomaly Detection, Gabor Filter, Artificial Neural Network, Machine Vision.
I.
Introduction
Machine vision is recently introduced for machine based automatic inspection and
analysis for applications such as automatic inspection, robot guidance in industry, fault detection
and others. Video surveillance is used for an automated detection precisely predefined concepts
of abnormal behavior, such as an intruder in a prohibited zone. Alternatively, the video streams
are constantly recorded or observed by operators. For many visual surveillance applications
however, ideal analysis software would automatically interpret the entire scene and alert in case
of any suspicious situation[11]. The developments presented in this paper aim at an automatic
and independent detection of abnormal behavior in surveillance videos. Present study focuses on
development of methods and techniques that enable to automatically examine and detect the
abnormal behavior in the surveillance scene. A large part of the work is devoted to the visual
analysis of human behavior and abnormal event detection in outdoor scenarios. In this context, a
working system would alert of dangerous situations and improve the personal safety of people to
avoid hazardous situation.
Fig.1 Proposed Methodology for anomaly detection
II.
Motivation
The Kargil war was an armed conflict between India and Pakistan that took place between May
and July 1999 in the Kargil district along the Line of Control (LOC). The cause of the war was
the infiltration of Pakistani soldiers and Kashmiri militants into positions on the Indian side of
the LOC. Kargil was targeted partly because the terrain was conducive to the preemptive
seizure of several unoccupied military positions and the difficulties were be exacerbated by the
high altitude and freezing temperatures. During the winter season, due to extreme cold in the
snow-capped mountainous areas of Kashmir, it was a common practice for both the Indian and
Pakistan Armies to abandon some forward posts on their respective sides of the LOC and to
reduce patrolling of areas that may be avenues of infiltration. When weather conditions became
less severe, forward posts would be reoccupied and patrolling resumed. During February 1999,
the Pakistan Army began to re-occupy the posts it had abandoned on its side of the LOC in the
Kargil region, but also sent forces to occupy some posts on the Indian side of the LOC . These
incursions were not detected by India as Indian patrols were not sent into some of the areas
infiltrated by the Pakistani forces. This was a serious problem for the security of India or for any
nation or organization. The range of Himalayan mountain is approximately 3,000 km.In this
wide range it is quite difficult to manually observe each location of Loc. The problem can be
solved with the help of satellite images and machine vision techniques which may help to detect
anomalous behavior in loc and alarm the system to avoid any mishap.
III.
Anomaly Detection
Anomaly detection (or outlier detection) is the identification of items, events or observations
which do not conform to an expected pattern or other items in a dataset. Traditionally, video
streams from surveillance cameras are watched by human operators who are trained to spot
abnormal events. An alternative is to record the video streams on storage devices for future
viewing in case something interesting had happened. Recent approaches inspect the streams with
rule-based processing[12]. In this paper, we investigate towards a fully automated analysis of the
surveillance video for a robust detection of abnormal events. Consequently, detecting unusual or
suspicious activities, uncommon behaviors, or irregular events in a scene is the primary objective
of an automated video surveillance system. We refer to this activity as anomaly detection
because the sought-after situations are not observed frequently. Although the term anomaly
cannot be defined explicitly, all such systems are based on the implicit assumption that events
that occur occasionally are potentially suspicious, and thus may be considered as being
anomalous. By formulating anomaly detection as a reconstruction process, anomaly detection is
reduced to being defined as an outlier detection problem, i.e., finding the events that are not
similar enough to the previously observed events in the video. Therefore, given a video
sequence, V, containing a set of events V= {𝑒𝑖 }𝑁
𝑖=1 and a similarity measure S, the concept of an
anomaly is defined for a particular event eq, as follows:
𝑒𝑞 ∈V
𝑠𝑞,𝑖 = 𝑠(𝑒𝑞 , 𝑒𝑖 ); 𝑒𝑖 ∈ 𝑉 − {𝑒𝑞 }
𝑒𝑞 is anomaly iff ∀𝑖; 𝑠𝑞,𝑖 ≤ 𝛾
where 𝛾 is a threshold. This implies that the event 𝑒𝑞 is not similar enough to any of the observed
events[14].
IV.
Previous Work
Anomaly detection in video surveillance is an important task for public security and safety and
attracts more and more researchers’ attentions recently. Meanwhile, there is a challenge task due
to many difficult problems, such as inevitable noise illumination change and deformation in the
scenes, diversity of event, and interaction between multiple events. Visual surveillance has been
investigated worldwide under several large research projects by various researchers. In most
computer vision approaches, abnormal events are identified as outliers to previously trained
models of normality (e.g. [Johnson and Hogg 1995,Makris and Ellis 2005]). If motion patterns in
the scene are available, statistical outliers are then detected as abnormal events based on low
level motion features (e.g. [Adam et al. 2008, Stauffer and Grimson 2000]), or high-level
specific object tracking [Hu et al. 2006, Basharat et al. 2008, Hendel et al. 2010].In crowded
scenes, optical flow vectors are quantized and modeled in order to detect deviations to normality,
as for example in [Kim and Grauman 2009,Kratz and Nishino 2009]. Due to the fact that trained
models of normality can hardly contain all expected configurations, Boiman and Irani [Boiman
and Irani 2005] attempt to compose the current observation from space-time fragments
in the database in order to detect irregular situations. [Vijay Mahadevan, Weixin Li, Viral
Bhalodia and Nuno Vasconcelos(2010)] presented a novel framework for anomaly detection in
crowded scenes. The model for normal crowd behavior is based on mixtures of dynamic textures
and outliers under this model are labeled as anomalies. M. Javan Roshtkhari, M.D. Levine
(2013) presented an approach for detecting suspicious events in videos by using only the video
itself as the training samples for valid behaviors. Xing Hu, Shiqiang Hu, Xiaoyu Zhang,
Huanlong Zhang, and Lingkun Luo(2014) proposes a novel LNND descriptor to represent the
video event for anomaly detection in crowded scenes.
V.
Neural Network Based Classifier For Anomaly Detection
The proposed anomaly detection method consists of two phases. First phase is preprocessing of
images in dataset and to train the system with Neural Network. Images presented to the network
go through a pre-processing step to improve the classification rate of the system. In order to
equalize the intensity values of the pixels present in the examination window histogram
equalization is performed to expand the range of intensities, in order to compensate for
differences in lighting and camera input gains. Second phase is testing phase which includes
inputing video frames iteratively to detect anomalous activity. The operation of the anomaly
detection through system can be broken down into three main steps.
 Initialization (design and creation of a neural network)
 Training (choice of training data, parameters, and training)
 Classification (scanning images to locate anomalies)
A. Algorithm for training
Start
1. Different images are collected and stored for anomaly image.
2. Features of images are collected through Gabor filter.
3. Step (1) and (2) are applied for nonanomalous images also.
4. Initalize the neural network parameters.
numInputs: 2160
numLayers: 500
transferFcn = 'trainscg';
net.trainParam.lr = 0.4;
net.trainParam.epochs = 1000;
net.trainParam.show = 200;
net.trainParam.goal = 1e-3;
5. Call matlab train function(Network Input, Target input);
6. Input a Test video frames iteratively
7. Compute the Minmax of scan Image, template1 image and template2 image.
8. Apply the convolution operator using both the templates.
9. Apply the trained net to detect the images we can obtained the point.
10. Create the window based on obtained points
Stop
The result obtained from above algorithm is saved in training dataset; whenever a test data
appears it apply template based matching and principle component analysis in order to
detect the right image from the training dataset.
B. Algorithm for Feature Extraction Using Gabor.
The core of Gabor filter based feature extraction is the 2D Gabor filter function which can be
written as
2
2
f 2 (γf 2 x′ 2+ηf 2 y′ 2 ) j2πfx′
ψ(x, y) =
e
e
ΠΥn
x ′ = xcosθ + ysinθ
y ′ = −xsinθ + ycosθ
f is the central frequency of the filter, θ the rotation angle, γ sharpness (bandwidth) along the
Gaussian major axis, and η sharpness along the minor axis (perpendicular to the wave). In the
given form, the aspect ratio of the Gaussian is η/γ.Gabor features, referred to as Gabor jet,
Gabor bank or multi-resolution Gabor feature, are constructed from responses of Gabor filters by
using multiple filters on several frequencies fm and orientations θn.
Start
1. Initialize parameters for window, scale and orientation
2. W(m,n) : Window [128 128]
3. nu : Scale [0 ...4];
4. mu : Orientation [0...7]
𝜋
𝑛𝑢
𝜋
5. K = 2 /√2 . 𝑒 𝑖.𝑚𝑢. 2
6. NK = 𝑟𝑒𝑎𝑙(𝐾)2 2+𝑖𝑚𝑎𝑔(𝐾)2 ;
7. Psi = zeros(m,n);
Stop
Fig. 2 Gabor Features
C. Algorithm for Feature Reduction
After feature extraction, the resulting output of each image contains several features, each
containing several filters .Such an output results in two problems for image retrieval: 1) Those
features without discrimination will undermine the retrieval performance; 2) For each feature,
a large bank of filters produces an output feature-vector with a large number of dimensions.
Image retrieval tasks may not require a large bank of filters for effective performance or may not
tolerate the computational burden imposed by a large filter bank.PCA also known as KarhunenLoeve (KL) transformation or eigenspace projection, a frequently used statistical technique for
optimal lossy compression of data under least square sense, provides an orthogonal basis vectorspace to represent original data. The eigenspace is a subspace of the image space spanned up by
a set of eigenvectors of the covariance matrix of the trained images. These eigenvectors are also
called eigenfaces because of their face-like appearance. The projection of an image into
eigenspace will transform the image into a representation of a lower dimension which aims to
hold the most important features of the face and make the comparison of images easier.
Start
1. Reshape the whole training set in One Dimensional Dataset
2. Computing the average all images
3. Calculating the deviation of each image from mean image
4. Computing the difference image for each image in the training set.
5. Merge all centered images.
6. Get the Eigen Values.
7. Sorting and eliminating Eigen values.
8. Apply thresholding function.
9. Calculate the eigenvectors of covariance matrix 'C'
10. Get the Eigen faces as centered image vectors.
Stop
D. Matching Algorithm for Anomaly Detection
Start
1. Input video frames as test image
2. Projection of centered test images into Eigenfaces.
3. Append all the projected test Images
4. Reshape the Centered test image
5. Find out difference from test image and mean
6. Find feature vector of test image
7. For each image in the training set
8. Find out min Euclidean Distance between test image and projected trainingg
9. The image that is minimum Euclidean distance is matching image.
Stop
VI. Experimental Results
The experiment for proposed system is performed in Windows operating system ,
Pentium V machine, 340 GB memory , 2.33 GHz processor, 1 GB RAM and software
Matlab2012 with image processing and Neural Network toolbox. . The images are extracted in
.png format. Database contains 220 normal images and 220 abnormal images. For testing single
camera was installed near the boundary wall for continuous surveillance. Video containing only
boundary wall is considered as normal event but as soon as video detects some faces of human
beings which is suspicious activity it consider it as anomalous activity and immediately alarms
about the dangerous situations and improve the personal safety of people to avoid hazardous
situation by detecting the same. The experimental results are shown in Figure 3, 4 and 5
respectively.
Fig.3 Random frames from video showing normal boundary wall
Fig.4 Random frames from video showing intruders near the boundary wall
Fig.5 Random frames from video showing anomaly detection of faces of intruders.
VII.Conclusion and Future Scope
This paper presents a novel method for anomaly detection in video dataset using Neural
Network. In this research Gabor filter is used for feature extraction of anomaly and normal data
of video. Video data are itself large in size and Gabor filter extraction creates a large set of
features for each frame of video. In order to reduce the dimension we have used principal
component analysis for further processing. Neural Networks iteratively trained the neurons for
normal and anomaly dataset. Anomalous object can be detected effectively through this trained
network. The experimental results shows that if the number of image for test dataset is less the
system elapsed time is less, when the number of faces increase in the image the scanning time
increases relatively. The present work has been carried out by installing single static camera only
and thus it is considering frontal faces of intruders only which is limitation of our work. So this
work can be extended by installing more number of cameras for better accuracy and also
considering activities of intruders from different angles to capture any kind of anomaly from the
video.
References
[1] N. Johnson and D. Hogg. Learning the distribution of object trajectories for event recognition. In Proceedings
British Machine Vision Conference, 1995.
[2] D. Makris and T. Ellis. Learning semantic scene models from observing activity in visual surveillance. IEEE
Trans. on Systems,Man, and Cybernetics, Part B: Cybernetics, 35(3):397–408, 2005.
[3] A. Adam, E. Rivlin, I. Shimshoni, and D. Reinitz. Robust real-time unusual event detection using multiple fixedlocation monitors.IEEE Transactions on Pattern Analysis and Machine Intelligence,30(3):555–560, 2008.
[4] D. H. Hu, V. W. Zheng, and Q. Yang. Cross-domain activity recognition via transfer learning. Pervasive Mobile
Computing, 7:344–358,2011
[5] C. Stauffer andW. E. L. Grimson. Learning patterns of activity using real-time tracking. IEEE Transactions on
Pattern Analysis and Machine Intelligence, 22(8):747–757, 2000
[6] A. Basharat, A. Gritai, and M. Shah. Learning object motion patterns for anomaly detection and improved object
detection. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition, 2008.
[7] A. Hendel, D. Weinshall, and S. Peleg. Identifying surprising events in video using bayesian topic models. In
Proceedings Asian Conference on Computer Vision, 2010.
[8] J. Kim and K. Grauman. Observe locally, infer globally: A space-time mrf for detecting abnormal activities with
incremental updates. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition, 2009.
[9] L. Kratz and K. Nishino. Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern
models. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition,2009.
[10] O. Boiman and M. Irani. Detecting irregularities in images and in video. In Proceedings International
Conference on Computer Vision, 2005.
[11] Weiming Hu, Tieniu Tan, Liang Wang, and Steve Maybank,” A Survey on Visual Surveillance of Object
Motion and Behaviors”, Ieee Transactions On Systems, Man, And Cybernetics—Part C: Applications And Reviews,
Vol. 34, No. 3, August 2004
[12] Chandola, Banerjee and Kumar,” Anomaly Detection : A Survey”, ACM Computing Surveys, 09 2009, Pages
1-72.
[13] Vijay Mahadevan Weixin Li Viral Bhalodia Nuno Vasconcelos,” Anomaly Detection in Crowded Scenes,”
IEEE Conf. on Computer Vision and Pattern Recognition, San Francisco, 2010.
[14] M. Javan Roshtkhari, M.D. Levine, An on-line, real-time learning method for detecting anomalies in videos
using spatiotemporal compositions, Comput. Vis. Image Understand. (2013),
http://dx.doi.org/10.1016/j.cviu.2013.06.007
[15] Xing Hu, Shiqiang Hu, Xiaoyu Zhang, Huanlong Zhang, and Lingkun Luo,” Anomaly Detection Based on
Local Nearest Neighbor Distance Descriptor in Crowded Scenes”, The Scientific World Journal Volume 2014,
Article ID 632575, 12 pages http://dx.doi.org/10.1155/2014/632575