See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/303930696
A real-time surveillance mini-rover based on OpenCV-Python-JAVA using
Raspberry Pi 2
Conference Paper · November 2015
DOI: 10.1109/ICCSCE.2015.7482232
CITATIONS
READS
11
3,376
5 authors, including:
Nazmul Hossain
Fahim Salauddin
North South University
La Trobe University
2 PUBLICATIONS 147 CITATIONS
11 PUBLICATIONS 18 CITATIONS
SEE PROFILE
All content following this page was uploaded by Fahim Salauddin on 13 June 2016.
The user has requested enhancement of the downloaded file.
SEE PROFILE
2015 IEEE International Conference on Control System, Computing and Engineering, 27 - 29 November 2015, Penang, Malaysia
A Real-time Surveillance Mini-rover Based on
OpenCV-Python-JAVA Using Raspberry Pi 2
Nazmul Hossain, Mohammad Tanzir Kabir, Tarif Riyad Rahman, Mohamed Sajjad Hossen, Fahim Salauddin
Department of Electrical and Computer Engineering
North South University
Dhaka, Bangladesh.
nazmul.hossain@northsouth.edu, tanzir.kabir@northsouth.edu, trahman@northsouth.edu, hmohamedsajjad43@gmail.com,
fahim.salauddin.bd@ieee.org
II. RELATED WORK
Raspberry Pi series computers are relatively new in the
market. The first Raspberry Pi 1 Model B+ was introduced
commercially only 3 years ago on February, 2012. On the
official Raspberry Pi site (www.raspberrypi.org), there are lots
of helpful resources for one to start using Raspberry Pi in
relevant projects. Even though the Raspberry Pi has been in
the technological arena for relatively shorter period of time,
people had already been using it to make various interesting
and marvelous projects in their own ways. Some of those
projects’ demonstration videos which are similar to our
project, can be watched on the YouTube.
Abstract-- Real-time surveillance is a vital component in any
situation or environment where there is a high need of security
for both personal and commercial property and assets.
Technology today is used in many different ways in order to
provide us such surveillance. An ideal solution for surveillance
involves not only the assortment of necessary sensors and devices
using appropriate tools, but also should be provided in the most
optimized way for obtaining feed and data while keeping the
expenditure minimized. In our project, we aimed at developing a
low-cost, real-time video surveillance mini-rover which will be
capable of providing real-time video footage and roam around in
the area which we want to observe. Our target was to use
hardware which is very low of cost and easily available.
One such interesting project introduced a fuzzy based smart
monitoring system using Raspberry Pi, Python & JAVA in [2].
Keywords—Mini-rover, Video Surveillance, Real-time,
Networking, OpenCV, Python, JAVA, Raspberry Pi 2.
Loscri, Mitton & Compagnone applied OpenCV “Object
Detection” and “Face Recognition” to implement the
behavioral algorithm described in their paper for Arduino
based platform [3].
I. INTRODUCTION
In our modern day to day life, security and surveillance
plays a very pivotal role. An effective security and
surveillance system can provide crucial early warnings in case
of any kind of emergency. However, if the surveillance system
is capable of roaming inside the area of surveillance, more
area of interest can be observed with optimum number of
surveillance equipment under constrained budget.
In [4], line tracking algorithm was implemented using
Raspberry Pi in marine environment for detecting fault in
underwater communication/ oil pipelines.
In [5], a rocker-bogie suspension rover with scooping arm
was controlled using cellphone technology (DTMF) where the
main component was an Atmega32 board. However, its video
streaming range was only limited to 100 feet.
Moreover, if the surveillance system is easily modifiable
with various sensors (digital temperature/ humidity/ flame/
sonic/ air quality/ gas sensors, etc.), obtaining necessary data
becomes much easier. This availability of various kinds of
necessary data in turn helps us in decision making process
while managing various kinds of situations.
However, the project implemented and described in [6] is
very similar to ours compared to other projects from [2], [3],
[4] and [5]. In [6], a Raspberry Pi based rover was controlled
from smart devices using web browser via WLAN/ Internet.
But, according to the system overview given in [6], the user
can access the rover control & video feed webpage running on
the webserver hosted on the Raspberry Pi but the user agent
(smart devices) has to be connected to the same Wi-Fi
network where the Raspberry Pi is connected or has to connect
to the internet through the same subnet used by the Raspberry
Pi. In contrast to [6], our Mini-rover can be controlled either
from a Laptop/ Desktop/ Android devices using our JAVA or
Android-JAVA application either via a Wi-Fi LAN in local
areas or, when using the cloud server & Internet, the minirover can be placed anywhere in the world with Internet
connectivity and controlled from the remote control platform
(Laptop/ Desktop/ Android devices) connected to the Internet.
In the latter case (with cloud server and Internet) , both the
mini-rover and the “Remote Control” platform needn’t to be
According to Kevin Ashton in [1], loss, waste and cost
management would be a lot easier if we had computers with
information on everything that we use were in charge of
counting and tracking items without any intervention from any
human. RFID technology would enable computers with those
kinds of powers without the limitations of human-entered data
so replacing, repairing or recalling faulty items would be a lot
more convenient.
978-1-4799-8252-3/15/$31.00 ©2015 IEEE
476
2015 IEEE International Conference on Control System, Computing and Engineering, 27 - 29 November 2015, Penang, Malaysia
connected to the Internet via the same subnnet, only the cloud
server will need to have a “Public/ Global IP
P Address”.
III. MOTIVATION & BACKGROU
UND
It’s needless to say that security plays a pivotal
p
role in our
modern life. Whether it’s for business purrposes or personal
reason, the need of surveillance and securityy has always been
much talked about. On top of that, remotee surveillance can
play a very effective role in providing securrity in our modern
ways of life.
Most of the present day surveillance systtems provide with
fixed-point CCTV view of certain area. To
T record/ obtain
surveillance footage from various viewing anngle, one must use
PTZ (Point-Tilt-Zoom) CCTV cameras whicch are quite costly
and moreover, they remain fixed at one siingle point – they
can’t roam around and provide coverage of a larger area.
Besides, once installed, modification/ uppgrading of those
systems are quite costly and time-consumingg.
Fig. 1. Diagrammatic Overview of
o the Project
V. HARDWARE USEED IN THE PROJECT
For implementing our projject, we used various hardware
which are low of cost, reliabble and readily available in the
local market. We also wanted to recycle any old parts if
possible. Below here is givenn a list and short descriptions of
various principal hardware useed in our project:
So, we wanted to develop a simple video surveillance minirover system which can be remotely controlled to move
around and provide the user with real-time video
v
footage of an
area of interest.
IV. SALIENT FEATURES OF THE PROJECT
R
Followings are the salient features of our project:
p
•
It should be inexpensive in comparisson to present day
surveillance equipment. For example, only a single
PTZ CCTV camera will cost arouund $100 - $450
depending on the manufacturing company (as of
August 30, 2015).
•
The mini-rover’s chassis should be
b able to move
forward, backward, leftward & righhtward on its four
wheels with all the equipment on boarrd.
•
The USB Webcam should be able to
t provide with a
clear, real-time view of the area at its front.
•
Video stream data from the webcam should
s
be sent to a
remote controlling platform over the Internet/ Wireless
network for being displayed to the useer’s screen.
•
The mini-rover should be able to receive commands
from the remote controlling platform
m (i.e.: Desktop or
Laptop) over the Internet/ Wireless network
n
and move
around according to the given commaand.
•
A. Raspberry Pi 2 Model B Veersion 1.1:
To be able to handle all the process and computations
needed for the mini-rover suurveillance system, we chose to
use a Raspberry Pi 2 (Model B,
B Version 1.1) for our project.
Fig. 2. A Raspberry Pi 2 Model B Version 1.1
We chose Raspberry Pi 2 ass the “Brain” of our mini-rover
because of the following reasonns:
• Raspberry Pi 2 doesn’t require
r
much power to operate.
• It is very low-cost (USD
D $35 only).
• It runs on Raspbian OS, an optimized version of
Debian Linux (which is Open-source).
O pins to interact with various
• It has in total 40 GPIO
sensors & modules. Soo, even after connecting all the
hardware of our projecct, there are enough GPIO pins
left to connect variouss types of sensors in future if
modification becomes necessary.
n
The system should be easily moddifiable for future
integration of various types of sensoors with the video
surveillance module on board the minni-rover.
The next image shows a diagrammatic overview of our
project:
477
2015 IEEE International Conference on Control System, Computing and Engineering, 27 - 29 November 2015, Penang, Malaysia
•
It’s a full-fledged CPU capable of connecting to a
Local Network / Internet via its Ethernet port or using a
Raspberry Pi 2 compatible USB modem or USB Wi-Fi
adapter which is ideal for implementing our project.
B. L298N H-Bridge Dual Motor Controller:
To control all the dc motors onboard the mini-rover’s
chassis, we used a L298N H-Bridge Dual Motor Controller.
Fig. 5. Apacer 4400mAh mobile power bank used in our project
E. Power Supply for Mini-Rover’s Driving Motors:
To provide external power to the two DC motors driving the
mini-rover, we used a small plastic battery pack capable of
holding 5(five) AA batteries (1.5V each, total 7.5V).
F. Four-wheeled RC Toy-car Chassis:
Fig. 3. A L298N H-Bridge dual motor controller
As for constructing the mini-rover’s body, we salvaged a
thrown-out four-wheeled RC Toy-car chassis, replaced its
Bluetooth RC Module with our L298N H-Bridge Dual Motor
Controller and assembled all other components on top of that
chassis.
C. A4Tech USB Mini Webcam:
To obtain video footage, we used a standard, plug-n-play
A4Tech USB Webcam.
Fig. 6. Vertical (left) & lateral (right) view of the mini-rover
VI. MAJOR SOFTWARE & LANGUAGES USED
Fig. 4. A4Tech USB webcam used in our project
The deliverance of smooth movement control and good
quality video capture was a priority in choosing the proper
software for our project. The list of all the software used in our
project is given below:
D. Apacer 4400mAh Mobile Power Bank:
To supply power to the Raspberry Pi 2 on-board the minirover, we used a 4400mAh Apacer Mobile Power Bank.
A. Raspbian OS:
There are several different operating systems for the
Raspberry Pi. Among these, Raspbian stands out to be the
most popular Debian Linux based operating system for the
Raspberry Pi [7].
478
2015 IEEE International Conference on Control System, Computing and Engineering, 27 - 29 November 2015, Penang, Malaysia
• Cardboard and paper for making the rover body
A. Hardware Setup Sequence::
After obtaining all the necesssary hardware and accessories,
we began to setup and connecct the hardware together for the
mini-rover as follows:
After enclosing the R-Pi 2 innside a protective plastic casing,
we established required connnections between the R-Pi 2’s
GPIO pins and L298N’s inputt pins as shown in the Figure 8
below:
Fig. 7. A typical Raspbian desktop
B. Python:
In the context of our project, Pythhon is the best
programming language that can be useed to allow the
Raspberry Pi to interact (in our case alloowing control for
movement and video capture) with the user or
o client.
C. OpenCV:
For the purposes of our project, OpenC
CV library was an
essential tool for the video capturing aspect of the mini-rover.
Due to the library’s flexible compatibiliity and hardware
acceleration standards, OpenCV was easily integrated
i
into our
Raspberry Pi 2.
Fig. 8. Connections between Raspberrry Pi 2’s GPIO & L298N’s pins
After connecting the two DC
C motors of the chassis with the
motor controller & then the motor
m
controller with Raspberry
Pi 2, we connected the USB Webcam and a Wi-Fi pocket
router (which will provide network connectivity to the
Raspberry Pi 2) to the Raspbeerry Pi 2’s USB ports. One can
use a USB Wi-Fi adapter com
mpatible with the Raspberry Pi 2
instead of a Wi-Fi pocket routeer for network connectivity.
We then put all the connected hardware so far on the
salvaged toy-car chassis as shoown in Figure 6. We connected
the power bank with Raspbeerry Pi 2 at the last stage of
hardware assembly to ‘switch-oon’ the Raspberry Pi 2.
D. JAVA:
All the programs running on the central server
s
and also on
the ‘remote control’ laptop for this project were written and
implemented using JAVA. Data transmissioon (both video and
rover movement commands) was achieved using
u
JAVA socket
programming both on the central server and the ‘control’
laptop.
E. SSH, Putty and Xming:
B. Software Setup Sequence:
SSH stands for “Secure Shell”. SSH is used to establish
cryptographic network-protocol based ‘shell’ sessions from a
remote platform on a Linux-based machinee. Putty & Xming
are two third party software used in our prroject to establish
SSH sessions between our Windows-baseed laptop and the
Linux-based R-Pi 2 for remotely accesssing the desktop/
command line of the R-Pi 2 on board the minni-rover.
After powering up the R-Pii 2, we establish a SSH session
from a remote laptop using an Ethernet cable, Putty & Xming
server program.
After that, we opened up tw
wo separate consoles on the R-Pi
2 using the SSH session and obtained root privileges (using
the command “sudo su”).
Then, we ran the central serrver program (written in JAVA)
on a separate laptop which iss connected to the same Wi-Fi
network provided by the Wi-Fii pocket router connected to the
Raspberry Pi 2.
From the previously openedd up consoles on the Raspberry
Pi 2, we accessed two python scripts
s
(stored on the Raspberry
Pi 2) for controlling the mini--rover’s movement & obtaining
video footage from the USB webcam and by running them, we
connect the Raspberry Pi 2 onn-board the mini-rover with the
central server.
Then, we connected the ‘reemote control’ laptop with the
same Wi-Fi network where thee central server & the R-Pi 2 are
VII. SETUP & IMPLEMENTATIO
ON
Alongside the major hardware mentioneed previously, we
used the following miscellaneous items too to implement our
project:
• A robust encasing for protecting the Raaspberry Pi
• 5 AA size batteries for the motor control battery pack
• Jumper cables (male-to-female, femalee-to-female)
• Ethernet cable
• Utility tape for attaching components of
o the rover
479
2015 IEEE International Conference on Control System, Computing and Engineering, 27 - 29 November 2015, Penang, Malaysia
IX. ACHIEVEMENT
connected
and
ran
the
two
JAVA
programs
“RoverControl.java” & “VideoFrame.java” to access the
webcam feed and the mini-rover’s movement control panel as
shown in Figure 9:
The main objective of our project was to provide a real-time
surveillance mini-rover that can be remotely controlled and at
the same time obtain video feed of the desired area. In our
project, we used the Raspberry Pi 2 as the main processing
unit for overall mini-rover control and functions.
We successfully achieved using the internet as the medium
of connection required for transferring control signals and
obtaining video feed. If we had used standard radio signals or
Bluetooth connectivity instead of Internet or Wi-Fi as medium
of connectivity that would not have enabled the mini-rover to
achieve its virtually global operating range.
X. LIMITATIONS
However, as most of real-world things, our mini-rover has
some limitations, too. For example:
•
When using a local Wi-Fi network, mini-rover’s
response time is almost instantaneous. However, when
using the Internet to remotely control and obtain
footage from the mini-rover (while using a cloud server
with public IP address), the response time will vary
(might be few milliseconds in case of movement
control and a few seconds in case of video-streaming if
Internet bandwidth being used is low). It can be
overcome if high-speed Internet connectivity through
cellular data network is ensured.
•
As a RC toy car chassis was used for the mini-rover’s
body, in case of muddy or sandy surface, our minirover in its current design isn’t suitable. However, if a
stronger, robust chassis with greater mobility is
designed and used with our mini-rover’s concept, then
it will be able to handle almost all kinds of terrain quite
well.
•
Due to its current RC toy car chassis, the mini-rover
can’t take any extra payload. It can be overcome by
using the solution described in the previous point.
•
The viewing angle of the USB webcam is fixed. This
can be overcome by mounting the USB webcam on a
tilt and rotation enabled platform (this platform can be
constructed using servo motors and other necessary
things).
•
As there was no Raspberry Pi 2 compatible USB Wi-Fi
dongle available in the local (Bangladeshi) market (it’s
available in the international market), we were
compelled to use a 3G Mobile Pocket Router for
providing Wi-Fi connectivity to the Raspberry Pi 2.
This only limits the Raspberry Pi 2 to connect to the
Internet. The 3G Mobile connection had bandwidth
limitations and at times was intermittent, causing a
problem for continuous motor control and video feed
capturing of our mini-rover. In order to overcome this
Fig. 9. Mini-rover’s movement control panel (on the left) & Video feed from
the Mini-rover’s webcam (on the right) as displayed on the remote control
laptop’s screen
VIII. RESULTS & ANALYSIS
After assembling all the hardware correctly and installing &
running all the software, we successfully demonstrated that the
mini-rover can:
•
•
•
Provide us with a “Black and White” video footage of
the area at its front.
It can receive commands from the “remote control”
laptop/pc/android device and move Left/ Right/ Up/
Down according to the given commands.
When connected with the central server via Internet
(using a USB Wi-Fi dongle connected to the Raspberry
Pi 2’s USB port and available Wi-Fi network), the
mini-rover can be placed anywhere in the world where
Wi-Fi Internet connectivity is available & send/
receive data to/ from the “remote control” platform
(which is also connected to the central server via
Internet) .
The gray-scale image format was chosen because videoframes in gray-scale format is much faster to stream and
consumes one-third of the bandwidth required for streaming
same video-frames in RGB or any other color format similar
to RGB format. As we used cellular/ mobile network for
connecting our mini-rover to the Internet while connecting to
the cloud-server, a good amount of bandwidth was saved as a
result of using gray-scale image format.
And as a direct result of using Internet via cellular data
network, we were able to achieve virtually global operating
range for our mini-rover.
480
2015 IEEE International Conference on Control System, Computing and Engineering, 27 - 29 November 2015, Penang, Malaysia
problem, we need to use a Raspberry Pi 2 compatible
USB Wi-Fi dongle.
XII. CONCLUSION
This project provided an inexpensive solution for mobile
surveillance in any necessary environment. Total cost of our
project was estimated to be around $90 which is much cheaper
compared to presently available mobile surveillance solutions.
Our mini-rover can indeed be controlled fully to move in all
directions. By using the USB webcam, we can also obtain
video-footage in real time while the mini-rover is on the move.
And above all, the mini-rover is fully controlled through
internet connection, whether it is through a local network, WiFi, 3G or a cloud server, which makes our mini-rover the best
option for integrating an IoT application into the issue of
security and surveillance services. By adding necessary
modifications as mentioned before, our project can be
delivered as a full-fledged device which can be used for home
or corporate surveillance purposes.
XI. FUTURE WORKS
In future, our project can be expanded in a number of ways.
Below here is some of the possible future works suggestion
for our project:
•
We can modify the mini-rover chassis to be able to
handle all kinds of terrain and all kinds of inclement
weather.
•
Any necessary sensor can be installed on the mini-rover
(i.e.: Digital temperature & humidity sensor, Gas
sensor, Air quality sensor, Sonic sensor, etc.) to obtain
necessary information on the surveillance area.
•
Solar panel and compatible battery modules can be
installed on the modified chassis to make the minirover to be solar-powered.
•
•
•
REFERENCES
[1]
We may devise and add a module involving a laser
sensor for measuring distance so that the mini-rover
can scan and generate a 3D map representation of the
surroundings, hence providing the person controlling
the rover a 3D map representation of the surrounding
area.
[2]
[3]
Artificial Intelligence (i.e.: Machine-Learning, Pattern/
Object Recognition, etc.) can be added to enable the
mini-rover identify obstacles/ hazardous environment
and notify & suggest alternatives to the operator/
control station while making the mini-rover fully
automated for roaming around.
[4]
[5]
With necessary sensors and modules installed and any
other modifications, the mini-rover can be used to
monitor various high-risk large areas like various
industrial plants, factories or can be used to constantly
monitor vast areas like various agricultural fields/ farms
where constant human monitoring can be too costly.
[6]
[7]
481
View publication stats
‘The Internet of Things’ Thing by Kevin Ashton’ – Retrieved from
http://www.rfidjournal.com/articles/view?4986
Fahim Slauddin and Tarif Riyad Rahman . “A Fuzzy based low-cost
monitoring module built with raspberry pi – python – java architecture”.
International Conference on Smart Sensors and Application (ICSSA) ,
2015.
Valeria Loscri, Nathalie Mitton, and Emilio Compagnone. “OpenCV
WebCam applications in an arduino-based rover”. International
Workshop on Wireless Sensor, Actuator and Robot Networks
(WiSARN), Jun 2014, Benidorm, Spain. 2014.
S. Amir, A.A. Siddiqui, N. Ahmed, and B.S. Chowdhury.
“Implementation of line tracking algorithm using raspberry pi in
maritime environment”. IEEE International Conference on Industrial
Engineering and Engineering Management (IEEM), 2014.
S. Divakar. “Cell phone controlled rocker-bogie suspension type rover
with a scooping arm”. 8th International Conference on Ubiquitous
Robots and Ambient Intelligence (URAI), 2011.
Dadi Anil Kumar and Mrs.Dr.M.Sangeetha. “ Controlling raspberry pi
rover through any smart device using web browser via WLAN/
internet”. International Journal of Science, Engineering and Technology
Reasearch (IJSETR), Vol. 4, Isuue 4, April 2015.
Harrington, N. ‘Learning Raspbian’. Packt Publishing – ebooks
Account.2015.