Advanced Science and Technology Letters
Vol.87 (Art, Culture, Game, Graphics, Broadcasting and Digital Contents 2015), pp.127-132
http://dx.doi.org/10.14257/astl.2015.87.26
Abnormal Behavior Recognition using Non-Overlapped
Multi-Camera – focused on Stalking and Violence
Hwa-Jin Park1, Hye-Young Kim2
1
Dept. of Multimedia, Sookmyung Women’s University,
Seoul, Korea
phj2000@sookmyung.ac.kr
2
Department of Game Software, School of Games, Hongik University, Jochiwon-eup,
Chungcheongnam-do, 339-701,Korea
hykim@hongik.ac.kr
Abstract. Proposed in this paper is an algorithm which determines whether
violence and stalking are being carried out in the images captured from multi
cameras. To decide the presence of violence and stalking, each object’s features
are identified and classified into three groups. Recognition processes for each
dangerous behavior are systemized for implementation, which would be one of
future research.
Keywords: abnormal behavior recognition, stalking, violence, multi-camera
surveillance
1
Introduction
As reflected social concerns on safety, social requests for surveillance system with
CCTV or IP camera are increasing recently. But the existing surveillance system
where cameras simply record and save images, doesn’t recognize anything. It detects
someone who is moving using IR sensors and gives an alarm if considered as
unlawful entry.
Recently lots of research has been in progress on detection of abnormal or
dangerous behaviors before the occurrence of accidents based on images taken from
CCTV or IP camera. Some research used a stereo camera to solve an occluding issue
easily. But it is not applicable since a single camera is installed in most of existing
installed cameras. Other research aimed at recognizing human behaviors such as
walking, running, sitting, standing from a single camera, which is a basis of
recognizing human intention. Some of dangerous behaviors such as unlawful entry,
smoking, and loitering can be detected from one camera, while stalking, tracing and
tailing need some other multi cameras’ image to understand. Hence, this research
focuses on recognition of abnormal behaviors such as trace, tail, stalking and violence
by a way of using multi cameras as a final goal. The first step to achieve this goal
would be to identify required features and plan the extracting method, and classify
them systemically.
ISSN: 2287-1233 ASTL
Copyright © 2015 SERSC
Advanced Science and Technology Letters
Vol.87 (Art, Culture, Game, Graphics, Broadcasting and Digital Contents)
2
Multi cameras in networked surveillance system
Generally each area can be monitored by one camera. This research performed in
such a situation that each camera has set in a sequence of id number in one direction
so that a passenger can be expected to appear in the next camera when he keeps upon
going in the same direction. Each camera will provide information about exiting and
entering objects to the next camera in the direction of each object. It means we can
continuously trace all objects even though there is a gap between cameras.
Fig.1. shows the diagram of networked surveillance system.
3 Violence and stalking recognition from non-overlapped multi
cameras
3.1 Definition of violence and stalking behaviors
The word “stalking” means a following someone stealthily or waiting in ambush. The
stalker follows someone to the extent that one feels attacked or murdered, which can
be defined as violent behavior. Such stalking behavior can be prevented in advance by
a way of detecting and warning what might be lead to dangerous situations such as
detaining, assaulting, and murdering. Following or shadowing can be recognized as
stalking, if such behavior is characterized as below. (1) Following a target in the same
direction for at least 2 or 3 minutes. (2) If the target stops, the follower also stops, and
if the target turns, the follower hesitates to go further for a second and starts to follow
again. (3) The follower always comes close to the target within a certain distance. (4)
The approaching speed is maintained at a fixed level. (5) Although the follower
possibly passes the target, he always goes behind the target.
The term “violence” is generally defined as direct and physical compelling power
such as physical attacks, which usually occur in a provocative manner between two
persons. Such violent behavior which would be detected from screens is defined in
this research as below. If the size and frequency of changes of the object abruptly
increase since the shape of an object heavily changes in the occasion of violence, it is
regarded as occurrence of violence. In other words, the system would recognize
128
Copyright © 2015 SERSC
Advanced Science and Technology Letters
Vol.87 (Art, Culture, Game, Graphics, Broadcasting and Digital Contents)
changes in the size of the object such as height and width and if they exceed the preset rate of the changes and the frequency of the changes, it alarms.
3.2 Classification of features
We have to interpret, save, and share the characteristics data of the object in all videos
as well as the relating data near the object with other cameras so that we can observe
the stalking behavior for a long time and detect it as stalking. The characteristics of
the object would be detected in each individual camera as below. Firstly, group
characteristics can be categorized by the characteristics which identify a follower, the
follower’s behaviors, and the characteristics that the follower goes within a certain
distance. The identifying characteristics form positions, heights, and colors (hair,
shirt-sleeves, and pants). The behavioral characteristics consist of speeds and tracks.
All these characteristics data will be saved in every second. The group characteristics
of each individual n will be extracted from objects, followers and following within a
certain distance. The newly-defined, Member feature of a specific area means all
objects with a certain distance. Follower (n) feature means all objects which are ahead
of n in the same direction, while following (n) means all objects which are behind n in
the same direction.
Each feature are expressed in mathematics,
• Features for identification
Position: p(t) = (x(t), y(t), z(t))
Color: haircolor = color(top(blob)),
upcloth = color(mid(blob)), lowcloth= color(low(blob))
Height: height = (top(blob)) – bottom(blob))
• Features for behaviors
dp
Velocity: v(t) =
dt
Trace =tr ∑v(t)i = trace(∑v(t)i ), i = 1,2
• Features for group(n):
Member(n, m, t, min, max) = {m | n, m∈image(t), min<dist(n, m)<max},
where n, m are blobs
Follower(n, m) = {m |∀m ∈ member(n, m, t, min, max),
cos(angle(v(n), (p(m)-p(n))> 0, cos(angle(v(n),v(m)) > 0},
Following(n, m) = {m |∀m ∈ member(n, m, t, min, max),
cos(angle(v(n),(p(m)-p(n)) <0, cos(angle(v(n),v(m)) > 0}
Member does not include a companion of n by setting a min value because friends
usually go close to each other. To reduce a processing time, features regarding two
objects n and m can simultaneously perform since if m is a member of n at a certain t,
n is also a member of m at the same time. Similarly, Follower and Following are also
reciprocal.
Copyright © 2015 SERSC
129
Advanced Science and Technology Letters
Vol.87 (Art, Culture, Game, Graphics, Broadcasting and Digital Contents)
3.3 Detecting Process of Stalking Behavior
We will be able to save an identifying time by providing neighboring cameras with
the data containing the information of objects who exit from each camera’s screen
rather than tracing out the identical object by analyzing the object data of all cameras.
We can estimate the identicalness and an expected entering time considering a
distance at a certain time and an ongoing speed. We can also use each individual’s
behavioral information for detecting out the identicalness.
With the basis of these, we perform the following detecting process so as to find
out whether there is a stalker or not.
1. Find out whether there is an object who goes close by above 90 percentage,
using member which has group characteristics data saved for the last 3 minutes
by each individual object.
2. If there is an object in 1, find out whether he or she has identical tracks with
individual objects and if one does, it is considered as stalking.
Violence can occur more frequently in the case of being followed by someone. But
once violence occurs, a process of detecting it is dependent upon the fluctuation of a
change in the object. In other words, if the position of the object is not much different
but its vertical and horizontal size changes a lot at the same time, such behavior
would be regarded as violence.
4
Implementation design
We should extract three dimensional data from one source camera. Hence, we will
adopt a three dimensional space modeling by using a position and an angle of a
camera. z=0 because moving objects walk in the street, buildings and on the ground.
The position and serial numbers of all cameras should be set in advance. Each camera
should extract moving objects by using binarization, AVD (absolute value difference),
morphological filtering, and group them as a blob. To get rid of overlapped blobs or
shadows, a single blob should be extracted utilizing the height information of each
individual. Also the values of three dimensional characteristics will be saved by a way
of three dimensional modeling.
This is an algorithm for recognizing violence and stalking using multi cameras.
For each camera
1
Assign serial number to each camera in order according to location
2
Set 3 D tracker referring to 3D model using camera’s height and angle
3
For each frame
3.1
Detect objects
3.2
For each object
3.2.1 Convert to 3D data
3.2.2 Record all features in section 3.2
3.2.3 In case of an incoming object, check where it is coming from.
130
Copyright © 2015 SERSC
Advanced Science and Technology Letters
Vol.87 (Art, Culture, Game, Graphics, Broadcasting and Digital Contents)
3.2.3.1 In case of a new object, record an entering time
3.2.3.2 In case of a continuous object from the previous camera, link
it into the data of previous one in order to keep upon tracing.
3.2.4 Check a violence occurrence of the object
3.2.4.1 Increase potential violence counting if there is a big change
in a size of the object (pre-defined)
3.2.4.2 Give an alarm if potential violence counting is greater than
the pre-defined metric data for violence
3.2.5 Check a stalking occurrence of object comparing the elapsed time
with a pre-defined duration time (ex. 10 min)
3.2.5.1 With member feature for predefined duration time, list
objects appeared more than 90 %
3.2.5.2 Consider as stalking and give an alarm if one, a member
which satisfies 3.2.5.1, has the same trajectory with the
object’s
3.3
Save the outgoing object’s list if an object in the previous frame is not
seen in this frame
Fig 2, 3, and 4 show cases of violence occurrence, alarm message, and success rate of
violent behavior recognition.
Fig. 2. Cases of violence
Fig. 3. Alarm message
Table 1. Success rate of violent behavior recognition
Situation Category
Recognition rate
success
90%
failure
10%
success
80%
failure
20%
Violent
Non-Violent
5
Conclusion and Future Studies
This paper studied the recognition system detecting out abnormal behaviors that needs a longterm observation using multi cameras. It also defined characteristics and calibration
methodologies that would be required in the system. Though the possibility of errors would not
Copyright © 2015 SERSC
131
Advanced Science and Technology Letters
Vol.87 (Art, Culture, Game, Graphics, Broadcasting and Digital Contents)
be so low due to surrounding lights and shaking cameras, it is expected to be solved if censors
are adopted in this preliminary phase of the study. The future work could be done so as to
increase the possibility of recognition by a way of integrating censors as well as implement the
recognition system provided in this paper
References
1. Zhao, T., Nevatia, R.: Tracking multiple humans in complex situations. IEEE transactions
on pattern analysis and machine intelligence, vol.26, no.9, pp.1208-1220, 2004
2. Krumm, J., Harris, S., Meyers, B., Brumitt, B., Hale, M., Shafer, S.: Multi-Camera multiperson tracking for easyliving. Third IEEE International Workshop on Visual Surveillance,
2000
3. Park, J., Do, Y.: Dividing Occluded Pedestrians in wide angle images for the vision-based
surveillance and monitoring,” Journal of Sensor Science and Technology, vol.24, no.1,
pp.54-61, 2015
4. Venetianer, P. L., Zhang, Z., Yin, W., Lipton, A. J., Venetianer, P. L., Zhang, Z., Yin, W.
and Lipton, A. J.: Stationary target detection using the object video surveillance system. in
Proc. IEEE Int. Conf. Advanced Video Signal Based Surveillance, pp. 242-247, London,
UK (2007)
5. Oh, S., Moon, S., Choi, S.: Intelligence security and surveillance system in sensor network
environment using integrated heterogeneous sensors. The Journal of Korean institute of
communications and information sciences, vol.38C,no.07,pp.551-562, 2013
6. Niu, W., Long, J., Han, D., Wang, Y.: Human Activity Detection and Recogntion for Video
Surveillance. IEEE ICME 2004, vol.1, pp.719-722,
7. Merad, D., Aziz, K.: Fast people counting head detection from skeleton graph. IEEE
international conf. on advanced video and signal based surveillance, pp. 233-240, 2010.
132
Copyright © 2015 SERSC