International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: editor@ijaiem.org
Volume 3, Issue 5, May 2014
ISSN 2319 - 4847
Vision Based Gesture Recognition Systems - A
survey
Miss. Medha Joshi1, Prof. Sonal Patil 2
1
PG Student, Department of Computer Science and Engineering, G.H.Raisoni Institute of Engineering and Management,
Shirsoli Road, Mohadi, Jalgaon. 425002, Maharashtra
2
Asstt.Proffesor, Department of Computer Science and Engineering, G.H.Raisoni Institute of Engineering and Management,
Shirsoli Road, Mohadi, Jalgaon. 425002, Maharashtra
Abstract
In last few decades, the Human Computer Interaction has gain importance. The new developed HCI systems are slowly replacing
the traditional input devices such as the keyboard and mouse. To communicate between user and computer, hand gesture
recognition plays an important role. It provides separately a complementary modality to speech to express an idea by the user.
Hand gesture, a non-verbal technique provides free expressions while communicating between human and computer as compared
to any traditional devices. Hence for designing an HCI system hand gestures recognition is seen to be much efficient. This paper
emphasis mainly on the survey of some different vision based hand gesture recognition systems
Keywords: Gesture recognition, vision based hand gesture recognition, hand gesture, human computer interaction.
1. INTRODUCTION
Presently in the day-to-day life for interactive and intelligent computing, there is a need of an effective and efficient
human– computer interaction. By using different hand gesture recognition approaches, this non-verbal communication
technique can be used also for Human-Computer Interaction but not only Human-Human Interaction. Many different
types of prototypes which are developed are easy to use and understand and much cost effective than the traditional
interface devices such as the keyboard and the mouse. Hand Gestures are very expressive with the intent of interacting
with the environment or conveying any information. There are two types of hand gestures 1) static where assumption of
some specific pose is determined 2) dynamic which is rum time and can’t be determined. There are different types of
approaches used for hand gesture recognition. Some of the approaches are listed below:
a) Data glove based approach
b) Vision Based approach
c) Color glove based approach
a) Data glove based approaches: In this approach a device of glove type is worn by the user. This glove like device is
having sensors which sense the movements of finger and hands. The information is then passed to the computer for
processing [3].
b) Vision based approaches: Only a single camera is used which captures the image of hands which is used for interaction
between human and computers. This approach as compare to data glove based approach is the convenient, simple and
natural [4] .
In this approach user not require to wear anything. Instead
C. Color glove based approaches: This approach compromise between data glove based approaches and vision based
approaches. Special Marked gloves or special colored gloves are used by the user for the process of hand tracking and
locating of hand and fingers [5].
In this paper we are discussing some vision based HCI system
.
2. VISION BASED HAND GESTURE RECOGNITION SYSTEMS
Following are the different vision based hand recognition systems.
1) Wei Du and Hua Li [6] proposed system which is taking input using a CCD camera. This image goes through
segmentation step where, extraction of users hand from the input image is done using skin color filters. The binary output
of this step is received. After this step the feature extraction is done. Both global and local are used for the enhancement
of the robustness of system. Using the finite state machine the cluster is done
2) In 2007, a multi-angle hand-gesture recognition method was proposed by Chen and Tseng [7]. In this method three
webcams were used which were set at right, left and at front of the hand to capture the gestures. The three SVM (Support
Vector Machine) classifiers trained respectively. One voting and two plans for fusion fused the constructed classifiers.
This process was carried out after the training process. The methods recognition rate of hand gestures which included
different angles, sizes, and different skin colors is achieved more than 93%. In their research, there were only three hand
gestures.
Volume 3, Issue 5, May 2014
Page 321
International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: editor@ijaiem.org
Volume 3, Issue 5, May 2014
ISSN 2319 - 4847
3) A system which recognizes American Sign Language alphabets using hand gestures was proposed by Amin and
Yan[8]. This system used PCA and Gabor filters to achieve the task. From the experiment it is demonstrated that the best
result was obtained from the combination of the two principle components i.e fourth and seventh. This best combination
of principal components out of the top 20 principal components is determined by finding the best fuzzy cluster for the
corresponding PCs of the training data. The average recognition rate of the system is 93.23% but for similar alphabets
this recognition rate is relatively low.
4) Deng-Yuan Huang, Wu-Chih Hu, Sung-Hsiang Chang [9] proposed a Vision Based system for hand gesture
recognition. Many simple hand gestures were recognized by the system using the method of Gabor filter, PCA (principal
components analysis) and SVM in complex background. Firstly, from a sequence of video images extracted the hands
using the skin color information. After that using Gabor filter the images of hand gesture were coped and to minimize the
dimension of the data space they used PCA method. SVM classifier was used at the last to recognize the hand gestures.
The proposed system recognition rate of 95.2% is achieved. The processing time in this system is 0.2 second for every
frame, which did not achieved the requirement of a real-time system.
5) Amit Gupta, Vijay Kumar Sehrawat, Mamta Khosla[10], proposed a Real Time Human Hand Gesture Recognition
System based on FPGA. This system recognizes total 10 different static hand gestures. This hand gesture recognition
system is modelled in Verilog HDL and synthesized using Xillinx® ISE. The FPGA board used is the Virtex2 Pro FPGA
development board In the system firstly the image is captured using the CMOS image sensor camera. The image which is
captured is retrieved in R,G,B components by the camera interfacing module setup to communicate between camera and
the FPGA board. The image stored on board DDR RAM of Virtex2 Pro board is magnified to the 320×240 pixels
resolution around the region of interest (ROI) as the resolution of the camera may be set to different value. To acquire the
images in real time, the FPGA is operated at 100 MHz. After acquiring the image the pre- processing is done. In
preprocessing firstly the illumination compensation using the spectral distribution is carried to get the proper R,G,B
components. After the illumination compensation the skin color segmentation on that image using YCbCr color space is
achieved. The morphological image filtering is used to remove the noisy holes from the image due to nails and other hand
accessories like ring. The gesture detection is carried out in two steps first is the feature extraction and second is the
classification of the gestures. Feature Extraction is carried out on the shape based approach in which some shape based
features are used such as Area of hand, Perimeter of hand, Thumb detection and Radial profile and angular position. The
classification of gesture criterion is defined by analyzing the images of hands of 25 individuals. The overall accuracy of
the system is 94.40 %.
6) Maciej Czupryna, Michal Kawulok [11] proposed a system which completely replaced a mouse as a pointing device
and extended the mouse capabilities. The system is a vision based interface which uses a simple webcam. The system
recognizes a set of predefined hand poses, presented either in the gesture mode or the mouse mode. The image is acquired
using the webcam. After acquiring the image, skin detection is performed on the HSV color space. Subsequently, the
hand feature points are extracted using the Connected Component Labeling, Distance Transform and Hough Transform,
where the thresholds are determined using DTmax. The hand gesture is classified form a feature vector which is obtained
from the values such as angle between two fingers (af), fingers length ratio (rf), and finger base distance (db). The next
step is carried out when the dynamic gesture occurs where the trajectory recognition is performed to recognize the
gesture. The recognition rate of the system differs for angular set and circular set. Hence the overall recognition rate is
78%.
7) Siddharth S. Rautaray, Anupam Agrawal[12] designed a gesture recognition system which recognizes static and
dynamic hand gestures. The hand image is captured from camera. As the image capturing is over, the haar cascade
classifier is used for locating hand position and classifying gestures (open, close, pointing, etc.). The tracking of hand
which is the next phase is done using the CAMSHIFT technique. Contour along with the corresponding hand is mapped
as soon as the hand is tracked which further extracts a corresponding convex hull (area). The recognition has been done
through modeling of the hand by mapping it to the number of defect formed in it. Afterwards system tracks the number of
defects that have been generated by the hand and maps it to a meaningful command.
3. APPLICATIONS
3.1 Sign Language
The physically impaired person in hearing and speech uses sign language as a natural way for communication. The
different types of vision based gesture recognition methods are being embedded into sign language. A captured device
finds, track hands and record the shapes and trajectories of hands are represented by feature vector. The captured image is
matched from the signs stored in the feature vector and recognizes the specified character.
3.2 Gaming purpose
The hand gestures are also used for computer games where player’s hands are tracked to control movement and even
orientation of interactive game objects such as cars. The motion of the body or hand is also tracked to navigate the
gaming environment. The gestures are also used gestures to control the movement of avatars in a virtual world [14].
3.3 Medical Purpose
Volume 3, Issue 5, May 2014
Page 322
International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: editor@ijaiem.org
Volume 3, Issue 5, May 2014
ISSN 2319 - 4847
The hand gesture recognition systems helps doctors by manipulating digital images during medical procedures.With the
help of this technique the image are access without using touch screens or computer keyboards [14]
3.4 Alternative computer interfaces
Using strong gesture recognition system the traditional input devices such as keyboard and mouse can be completely
replaced [14].
4. CONCLUSION
In this survey paper, an overview of different hand gesture recognitions approaches is provided. We have reviewed several
existing vision based hand gesture recognition systems using different approaches and techniques. The survey enlists
some the common enabling technologies of hand gesture recognition for supporting vision-based human-computer
interaction based on the recognition of hand gestures. The application of the vision based hand gesture recognition system
in different fields is also specified.
References
[1] S. Mitra, and T. Acharya, 2007. “Gesture Recognition: A Survey”. IEEE Transactions on systems, Man and
Cybernetics, Part C: Applications and reviews, vol. 37 (3), pp. 311-324, doi: 10.1109/TSMCC.2007.893280.
[2] Sanjay Meena, 2011 “A Study on Hand Gesture Recognition Technique”, Master thesis, Department of Electronics
and Communication Engineering, National Institute of Technology, India
[3] Harshith.C, Karthik.R.Shastry, Manoj Ravindran, Srikanth, Navee Lakshmikhanth “Survey on various
Gesture
Recognition Techniques for Interfacing Machines based on Ambient Intelligence” (IJCSES) Vol.1, No.2, November
2010
[4] PragatiGarg, Naveen Aggarwal and SanjeevSofat, 2009. Vision Based Hand Gesture Recognition, World Academy of
Science, Engineering and Technology 49, pp. 972-977
[5] Mokhtar M. Hasan, and Pramod K. Mishra, 2012. “Hand Gesture Modeling and Recognition using Geometric
Features: A Review”, Canadian Journal on Image Processing and Computer Vision Vol. 3,No.1.
[6] Wei Du Hua Li “Vision based gesture recognition system with single camera” IEEE Proceedings of ICSP2000,
0~7803-5747-7/00/$10.0002000
[7] Y. T. Chen, and K. T. Tseng, “Multiple-angle hand gesture recognition by fusing SVM classifiers”. In: IEEE
conference on AutomationScience and Engineering, Scottsdale, AZ, USA, Sep 2007, pp. 527-530
[8] M. A. Amin, and H. Yan, “Sign language finger alphabet recognition from Gabor-PCA representation of hand
gestures”. In: Proc. of the Sixth International Conference on Machine Learning and Cybernetics, Hong Kong, August
2007, pp. 2218-2223
[9] Deng-Yuan Huang, Wu-Chih Hu, and Sung-Hsiang Chang, “Vision-based hand gesture recognition using PCA +
Gabor filters and SVM”. Proceedings of the 5th International Conference on Intelligent Information Hiding and
Multimedia Signal Processing, Kyoto, Japan, Sep 2009, pp. 1-4
[10] Amit Gupta, Vijay Kumar Sehrawat, Mamta Khosla “FPGA Based Real Time Human Hand Gesture Recognition
System” 2nd International Conference on Communication, Computing & Security [ICCCS-2012] The Authors.
Published by Elsevier Ltd. Selection and/or peer-review under responsibility of the Department of Computer Science
& Engineering, National Institute of Technology Rourkela doi: 10.1016/j.protcy.2012.10.013.
[11] Maciej Czupryna, Michal Kawulok “Real-Time Vision Pointer Interface “ IEEE 54th International Symposium
ELMAR-2012, 12-14 September 2012, Zadar, Croatia
[12] Siddharth S. Rautaray, Anupam Agrawal “Real Time Gesture Recognition System for Interaction in Dynamic
Environment” a Indian Institute of Information Technology, Allahabad, 211012, India 2212-0173 © 2012 Published
by Elsevier Ltd
[13] H. Birk and T. B. Moeslund, “Recognizing Gestures From the Hand Alphabet Using Principal Component Analysis,”
Master Thesis, Laboratory of Image Analysis, Aalborg University, Denmark, 1996
[14] Siddharth S. Rautaray • Anupam Agrawal “Vision based hand gesture recognition for human computer interaction: a
survey” Information Technology, Indian institute of Information Technology, Allahabad, India © Springer
Science+Business Media Dordrecht , November 2012
AUTHOR
Miss Medha Joshi has received her degree in computer science and engineering form Sant Gadge Baba
University, Amravati 2010. She is currently perusing her Master’s of Engineering in Computer Science and
Engineering from G.H.Raisoni Institute of Engineering and Management, Jalgaon under North Maharashtra
University, Jalgaon. Her area of interest in research is Image Processing
Volume 3, Issue 5, May 2014
Page 323
International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: editor@ijaiem.org
Volume 3, Issue 5, May 2014
ISSN 2319 - 4847
Prof. Sonal Patil received B.E. degree in Computer Science & Engineering from S.S.B.T.’S
COET,Bambhori from North Maharashtra University in 2008 and M.Tech In CSE From TIT,Bhopal from
Rajiv Gandhi Prodyogiki Vishvavidyala in 2012.She is currently working as a Assistant Professor in
G.H.raisoni Institute Of Enginnering & Management, Jalgaon. She has published 7 articles in National &
International Jornals and 17 papers in National & International Conferences.Member of ISTE.
Volume 3, Issue 5, May 2014
Page 324