International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 04 | Apr 2019 p-ISSN: 2395-0072 www.irjet.net A Survey on Vision based Fall Detection Techniques Shahna Sherin P1, Anju J2 1MTech Student, Dept. of Computer Science and Engineering, LBS Institute of Engineering and Technology for Women, Kerala, India. 2Professor Anju.J, Dept. of Computer Science and Engineering, LBS Institute of Engineering and Technology for Women, Kerala, India. ---------------------------------------------------------------------***---------------------------------------------------------------------- Abstract - Falls can occur anytime ,anywhere and human body conditions are prone to severe accidents in such situations. Brownsel et al.[1] analysed how much effect automatic fall detection systems could have on people’s fear of falling. The people subject to the experiments had experienced falls in the previous six months. From the study it was understood that the people felt more confident and more safe to walk around after using the fall detectors. falls can be highly dangerous even leading to death.Unobserved or unattended falls increase the chance of casualties.Physically weak or disabled people and aged people have more chances to fall,faint or injure themselves on the busy roads ,slippery floors etc.Thus it is very important to design and develop efficient fall detection systems that would help to provide quick assistance to the victims .A number of studies have been made in the area of fall detection and the fall detection systems are often classified based on the type of sensors used to record or collect the data.This paper reviews different fall detection systems based on camera vision that have been developed over the years to detect human falls. The main purpose of an automated fall detection system is to identify that a fall has occured and to provide assistance in a suitable way .Time is a very crucial thing and if timely assistance is provided to the fallen people, it would also be easier to provide fast medical care. Existing health conditions may worsen when the older people lay down without being noticed for a long period of time on the roads or on the floor. If the fall is severe, blood loss for a long time can even lead to death and it is highly unfortunate that thousands of people die this way, unattended. It is thus very important to develop efficient fall detection systems that would help save lives. Timely detection and timely alerts can definitely reduce the grave situation of people dying unattended. Key Words: Human Fall,Fall detection, computer vision, Image Processing, Video Content Analysis etc 1. INTRODUCTION Falls are a great cause of fatal injuries, especially for the people who are old, disabled and weak. According to the reports of World Health Organization approximately 646000 people die from accidental falls each year globally and most of these cases are reported from poor countries .Falls are accountable for most of the deaths due to accidental injuries. It is also understood that falls occur frequently among people of age group 65 and above. In the case of older people biological changes in their body make them more weak and thus more prone to fainting and falling. The effects of such falls can be highly dangerous and it may even lead to long term hospitalization or much worse, early death. The critical phase of the fall is when a person meets the ground or the lower surface with a shock or an impact. After a fall the person shouldn’t lie down on the ground for too long. The fall detectors work towards giving fast detection of the fall event and giving quick alerts to nearby help centres or concerned individuals so that the people could be saved at the earliest. Fear of fall is always associated with the fall. People who have already experienced such accidental falls tend to develop negative feelings in their mind and may refrain from daily life activities due to their fear of falling. Apart from biological changes in the body that make people go weak and prone to falls, physical factors are also responsible for this accidents. Slippery floors and roads ,open drainages or sewage channels, potholes on the roads etc are a major threat and people irrespective of their age or © 2019, IRJET | Impact Factor value: 7.211 1.1 Different Fall Detection Systems Human fall detection has always been an area of constant research and the number of studies in this area has also increased drastically over the years. The fall detection systems are classified according to a variety of criterias. Some researchers have classified them based on the types of sensors used to collect fall data such as sensors that are wearable and that can be embedded somewhere for monitoring like camera ,pressure sensors etc. Other classifications are based on detecting the phases after the fall event and the impact of fall etc. Basically, all fall detection systems have a similar structure. The data that is needed to be analysed have to be collected first with the help of various sensors. Fall detection systems have to efficiently differentiate between activities such as sitting, walking, running and falling. The ambience sensor based fall detection could be implemented only in places equipped with the ambience sensors. This creates the problem of noise often getting mixed with the sensed data. Wearable device based systems are found to be used more in case of outdoor fall detection systems. However when it comes to older people they might not be that interested in being watched or monitored al- | ISO 9001:2008 Certified Journal | Page 4090 International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 04 | Apr 2019 p-ISSN: 2395-0072 www.irjet.net ways. Apart from that, people may forget to wear such sensing devices and it also has the added disadvantage of long term exposure to skin. Vision based fall detection systems overcome this problem of ambience based and wearable device based sensors to an extent. It is very complex and has also various factors that limit its performance such as effects of lighting on the photos or video ,quality of the camera used for vision, multiple objects that appear in the background etc.However with the introduction of latest technologies such as convolutional neural networks, vision based fall detection techniques have improved a lot. Neural networks are a good example of advanced machine learning approaches used for fall detection. Neural networks simulate the working of our brain and consists of so many layered networks. The inputs pass through all these layers and finally the output is obtained. The nodes of these networks have activation functions such as ReLu (Rectified Linear Unit) activation function, sigmoid function etc which are chosen according to the type of the problems. There are several neural network classifiers that have been developed over the years and CNNs(Convolutional Neural Networks) have proven to be efficient for image and video classifications. Instead of doing normal matrix multiplications as it was in the case of old classifiers, convolutional neural networks as the name suggests, perform convolutions over frames which helps in a more detailed classification of the input images. The number of filters used, play a very significant role in the performance of neural networks. While dealing with neural networks, computational complexity, feasibility of in- corporating so many layers etc have to be checked. Otherwise it would result in reduced quality of the system. Vision based fall detection or sensing systems can be seen as a category of context aware systems. The context aware systems as the name suggests involve sensors that record or sense data from the surroundings by continously monitoring them. Cameras and sensors embedded on floors or on other surfaces etc thus come under this category. Comparison of different fall detection systems is very difficult as they cannot be easily combined under one single section. Even though the fall detection based on computer vision is difficult, it has proven to be one of the best ways to detect falls as cameras could be placed anywhere indoors or outdoors. Many researchers have taken in to consideration, the issues of privacy while designing indoor fall detection systems, making them user friendly. It is very important to address the privacy issues as the vision based systems detects falls, after capturing pictures or videos of the people in focus.This paper presents a study on different vision based fall detection approaches. Following are some of the studies on vision based fall detection techniques using deep learning concepts: Lesya Anishchenko in [3] used deep learning and transfer learning concepts in the videos obtained from surveillance cameras. AlexNet was the convolutional neural network used for classification, classifying fall events from 30 sets of data records. The method used in here was transfer learning. The method showed good results in terms of sensitivity values, specificity values etc. Zhou et al.[22] developed a fall detection system using the concepts of convolutional neural networks and multi sensor fusion. This system involved a combination of radar and optical camera. Optical camera captured the pictures of sequence of human actions. Using STFT(Short Time Fourier Transform),time frequency(TF) micromotion characteristics of the radar were obtained. Two kinds of CNNs, Alex-Net and Single Shot Multi-box Detector(SSD)Net were used for classification and recognition of the action sequences. Lu et al.[4] developed an intelligent human fall detection system that could detect falls from video surveillance. Vibe algorithm was used to detect human bodies. The gabor features were selected as the observation feature and SVM was the classifier used. Nadi et al.[5] developed another system using LDA (Linear Discriminant Analysis).Aspect ratios and angles were calculated and noise removal was done to delete shadows. A fall detection method based on 3D CNN(three dimensional convolutional neural network) was developed by Lu et al.[6] in which an automatic feature extractor was trained by kinetic data alone. By applying three dimensional convolutions over the frames, motion information was obtained from the videos apart from the spatial features obtained from 2D images. 2. VISION BASED FALL DETECTION APPROACHES Vision based fall detection systems capture images and videos using a good quality camera and using an efficient fall detection algorithm they classify the actions or events in the captured data as ”fall” or ”not fall”. The cameras have to be attached in appropriate places and the approaches based on vision are computationally complex, as they need high speed processors. With the development of technologies like neural networks, a lot of intelligent fall detection systems have been made. While designing fall detection systems it is crucial to reduce the number of false alarms, that is the number of times the system detect a ’non fall ’ event as a fall event should be reduced. The number of false positive cases provided by machine learning methods are very less compared to many other methods. It is highly complex, but it is the current trend as it provides much better results. Wang et al.[2] defined a method which was based on human characteristic matrix. The classifier used here to differentiate between the fall and other activities was SVM(Support Vector Machine) .The human silhouette was extracted using background subtraction and human characteristic matrices were formed based on human posture informations and Hu-moment invariant. These features were then used for the training of the classifier. © 2019, IRJET | Impact Factor value: 7.211 The vision based fall detection techniques can also be classified based on the type of cameras used for motion or image capture. Following are some of the studies conducted | ISO 9001:2008 Certified Journal | Page 4091 International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 04 | Apr 2019 p-ISSN: 2395-0072 www.irjet.net with single, multiple and depth cameras: Ko et al.[7] described a single camera based fall detection method where both 3D depth tracking and EKF(Extended Kalman Filter) based approach were combined. This paper described the 2D image based detection procedure of a person in movement which involved background subtrac- tion, binarization, filtering of the noise and finally human detection. EKF was used to get the depth informations from an image. Using a set of trained images depth maps were computed with the help of discriminatively-trained Markov Random Field (MRF). Bian et al.[8] proposed a fall detection approach based on a depth camera. This method was not affected by the changes in illumination of light. Improved Randomized Decision Tree algorithm was used here to extract 3D joints. Depth cameras helped to reduce the ambiguity of silhouettes and depth images were used to apply pose corrections on the extracted 3D joints of the human body. SVM classifier was then applied on these extracted 3D joints to confirm falls. Suneung Kim et al.[9] developed a method based on Extended Kalman Filter that used both two dimensional and three dimensional information of a dynamic scene. RGB image was taken as the input and depth map was created from that input using learning based methods. Particle Swarm Optimization was used for tracking humans. Three dimensional human tracking was done with the help of Extended Kalman Filter. Detection of fall event was followed by the ringing of an emergency alarm. detected using the three dimensional velocities which were obtained from the 3D trajectory. 2)Spatiotemporal features based approach: Using spatiotemporal features, shape modeling is used to get detailed information from the human activities. Foroughi.et.al[17] has developed a method combining GMM(Gaussian Mixture Models) and HMM (Hidden Markov Models) to detect falls. Adaptive GMM method was used to distuingish humans from other moving images and HMM was used to recognize the fall events. Chih-Yang Lin et al[13] described a method to identify human falls through different shape features. Here again, foreground extraction to obtain human shapes were done using GMM and Motion History Image(MHI) was used to analyze the behavior of the fall. An automated system for fall detection was developed by Sehairi et al.[14] that does simp-SOBS(Simplified Self Organized Background Subtraction).The different classifiers used here were RBF SVM(Radial Basis Function Support Vector Machine )classifier, KNN(K Nearest Neighbor)classifier and BPNN(Back Propagation Neural Network).RBF SVM performed slightly better than KNN as it is an improved version of KNN and also better than BPNN as the later works more efficiently with larger amounts of data. Durga Priya et al.[15] analysed fall detection in the obtained videos by combining motion and shape features.This method focuses on forming contours of the target human bodies based on morphological skeletons in depth images.From this contours it then extracts local dynamic shape and motion features.For improving the accuracy of the fall detection method various shape and motion features are combined together.S.M.Naqvi et al.[16] proposed one class boundary classifier method for fall detection from captured videos.Centroid and orientation of voxel persons were selected as the features to be extracted and a comparison was made between 4 different boundary methods which are one class Support Vector Machine method ,k Nearest Neighbor method and SCMPM(Single Class Minimax Probability method)by testing on different datasets. The fall detection techniques could vary depending upon the features analysed such as spatiotemporal features, head position, body posture, body shape change, body state change etc. 1) Three Dimensional Head Position Analysis Based Approach: Three dimensional head position analysis involves the evaluation of head position. This helps in determining the occurence of large scale movement in the video sequences. It is dependant on head monitoring. Hazelhoff et al in [10] designed a fall detection system to detect falls in an unobserved home situation. The foreground region extraction was done here by using 2 fixed uncalibrated cameras. A Gaussian multiframe classifier was used for the classification of fall events and the rejection of false positives were done by the head tracking module. PCA(Principal Component Analysis) was used to find the direction in which the the body’s main axis lie and also the differences that occured in the x and y directions. The system gave a 100% accuracy on un- occluded videos and about 90% accuracy on occluded videos. Rougier and Meunier in [11] used monocular camera images in their proposed system. Head 3D Trajectories were the main feature used for detecting falls in this paper. Here, head tracking was done as often there would be large amount of head movements during a fall and the head would be always visible in the images.3D trajectory was extracted using particle filter and using this the head was tracked and finally fall was © 2019, IRJET | Impact Factor value: 7.211 3)Shape and Posture Variation Based Approaches: Foroughi et al.[12] proposed a method for human fall detection using variations in the human shape. During feature extrac- tion, ellipses were approximated around the human bodies and histograms of both vertical and horizontal projections were constructed. Head position was also noted here. Motion classification was done using multi class SVM and for- ward, backward and sideways falls were detected. Posture changes were determined by OAA(OneAgainst-All Method) that utilized k binary SVMs to classify k classes and OAO(One-Against-One) method which used k(k-1)/2 binary SVMs to identify k classes. Khandoker et al.[18] tried to find out how wavelet based analysis on gait variables could be effectively used for dealing with the balance problems among the elderly. Signal recording was done to obtain MFC(Minimum Foot Clearance),gait data. | ISO 9001:2008 Certified Journal | Page 4092 International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 04 | Apr 2019 p-ISSN: 2395-0072 www.irjet.net Feature extraction involved analysis of the histogram or the wavelet analy- sis. SVM was used as the classifier which helped in the recognition of fall patterns. Miaou et al used an omni camera or MapCam in [19] to capture pictures and image processing was done over these images. MapCam has a 360 degree scene capture capacity and the speciality of this system is that it is personalized. Different people have different body figures and according to this there would be notable differences in the normal state and fall states. The personal informations of an individual such as weight, height etc helped to make the detection much more easy and to decrease the number of false alerts. 6 7 Thome and Miguet in [20] proposed a two layered HHM model (Hierarchical Hidden Markov Model. The first layer’s elementary motion pattern helped to detect sudden changes in the body posture such as falls. Lin and Ling [21] discussed a fall event detection in a domain that was compressed. Global motion estimation and local motion clustering methods were applied here to perform object segmentation for extracting moving objects. The paper mainly focused on the extraction of the features such as the fall occurence’s short time period range, quick change in the centroid of the falling person, and also the falling human’s vertical projection histogram. 8 9 Table -1:Analysis Of Fall Detection Techniques Sl.No. 1 2 Contribution Techniques 10 Reference Improved fall detection accuracy by focusing on specified angles rather than the normal ellipse. Hu-moment invariant and SVM classifier. [2] Studied the feasibility of usage of deep learning techniques in fall detection. Pre-trained CNN AlexNet [3] 11 12 3 4 5 Identified that gabor feature can effectively describe human bodies for fall detection. Detected falls by extracting features: aspect ratios and fall angles.Moving objects alone are detected in the frame by effective removal of shadows. Achieved high accuracy on multiple cameras fall data set by using visual guide 3D CNN. © 2019, IRJET | Vibe algorithm and SVM classifier Linear Discriminant Analysis and Cross Validation LSTM and 3D CNN [4] [5] 13 [6] Impact Factor value: 7.211 | Successfully detected human falls in the outdoor scenarios just by using single cameras and also applying depth informations. Utilised depth cameras to avoid the illumination problems and overcame the difficulties of existing methods in extracting joints. Detected head positions and tracks heads which helps in obtaining good robustness and also helps to remove false detections. Identified the potential of three dimensional tracking based fall detection with a single camera.Depth maps were gener- ated from the outdoors. Experimented with realistic datasets and achieved excellent results even with low quality images.Identified that Human shape deformation is a very effective tool for detection of falls. Combined the motion information and shape change information to detect falls. Employed cost effective methods for detection that could be applied in real time.Acceleration and angular acceleration calculated to improve accuracy of fall detection. Detected silhouettes of moving persons and used effective algorithms to estimate head positions. Extended Kalman Filtering based tracking and depth based fall de- tection. Enhanced RDT algorithm and SVM classifier. Gaussian multiframe classifier and PCA. Extended Kalman Filtering and Particle Swarm Optimization. Gaussian Mixture Model and Pro-crustes shape analysis. Multi class SVM classifier. Gaussian Mixture Background Model and Motion History Image. FSM algorithm and RBFSVM,KNN and Back propagation Neural Networks classifiers. ISO 9001:2008 Certified Journal | [7] [8] [9] [10] [11] [12] [13] [14] Page 4093 14 15 16 17 18 19 20 21 International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 04 | Apr 2019 p-ISSN: 2395-0072 Extracted local features from contours of depth images and obtained high fall detection accuracy by fusion of several extracted features. Extracted 3D video features by constructing a voxel person and classifiers based on boundary methods were constructed Fall detection successfully done by analysing histograms segmented silhouettes and noting changes in head positions Detected balance impairments among old people.Gait variables were used to extract the features to be fed in to the classifier Improved fall detection accuracy by incorporating personal informations like height,weight etc. Accurate recognition of fall pose from sitting and walking pose and well capable of working in unspecified configurations. Fall events identified and detected by changes in the human centroid position,changes in ratio of vertical histograms and the time duration of fall event. Informations from multiple sensors are fused inorder to improve the accuracy of fall detection systems © 2019, IRJET | Shape and motion dynamics analysis and SVM classifier. SCMPM method selected as the optimal method. MLP Neural Network Minimum Foot Clearance(MFC) used as the gait variable and SVM classifier www.irjet.net 3. CONCLUSION [15] The paper presents a survey on various vision based approaches used in fall detection systems. Vision based fall detection methods based on different aspects such as the features used to analyze the fall patterns like spatiotemporal features, head position, change of body shape and posture are discussed here. The survey provides insight in to the wide variety of methods and techniques utilised in the vision based fall detection systems. It cannot be said that one particular method is better than the other, each has its own advantage and disadvantage and the methods are selected according to their usability and feasibility. But still we could say that usage of convolutional neural networks could give us a more accurate fall detection as they outperform almost all the existing classifiers in image and video classifications. [16] [17] REFERENCES [18] [1] [2] MapCam with 360 degree scene capturing ability used for im- age capture.Fall detection done by comparing with threshold states. Hierarchical Hidden Markov Model [19] [3] [4] [20] [5] Global Motion Estimation and object clustering [21] [6] [7] Informations from multiple sensors are fused inorder to improve the accuracy of fall detection systems [22] Impact Factor value: 7.211 [8] [9] | Brownsell S ,Hawley MS .Automatic fall detectors and the fear of falling. Journal of Telemedicine and Telecare,2004. Rui-dong Wang,Yong-liang Zhang ,Ling-ping Dong, Jiawei Lu , Zhi- qin Zhang, Xia He.Fall detection algorithm for the elderly based on human characteristic matrix and SVM,2015 15th International Confer- ence on Control, Automation and Systems (ICCAS). Lesya Anishchenko ; Machine Learning In Video. Surveillance For Fall Detection, 2018 Ural Symposium on Biomedical Engineering, Radioelectronics and Information Technology (USBEREIT). Hong Lu,Bohong Yang,Rui Zhao,Pengliang Qu,Wenqiang Zhang.Intelligent Human Fall Detection for Home Surveillance. 2014 IEEE 11th Intl Conf on Ubiquitous Intelligence and Computing and 2014 IEEE 11th Intl Conf on Autonomic and Trusted Computing and 2014 IEEE 14th Intl Conf on Scalable Computing and Communications and Its Associated Workshops. Mai Nadi ,Nashwa El-Bendary,Aboul Ella Hassanien,Taihoon Kim.Falling Detection System Based on Machine Learning.2015 4th International Conference on Advanced Information Technology and Sensor Application (AITS). Na Lu,Xiaodong Ren,Jinbo Song,Yidan Wu.Visual guided deep learning scheme for fall detection.2017 13th IEEE Conference on Automation Science and Engineering (CASE). Myeongseob Ko ,Suneung Kim, Kyungchai Lee,Mingi Kim, Kwangtaek Kim.Single camera based 3D tracking for outdoor fall detection toward smart healthcare. 2017 2nd International Conference on Bio-engineering for Smart Technologies (BioSMART). Zhen-Peng Bian,Junhui Hou,Lap-Pui Chau,Nadia Magnenat- ThalmannFall.Detection Based on Body Part Tracking Using a Depth Camera. IEEE Journal of Biomedical and Health Informatics ( Volume: 19,Issue: 2,March 2015). Suneung Kim, Myeongseob Ko,Kyungchai Lee,Mingi Kim, Kwangtaek Kim.3D fall detection for single camera ISO 9001:2008 Certified Journal | Page 4094 [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 06 Issue: 04 | Apr 2019 p-ISSN: 2395-0072 www.irjet.net surveillance systems on the street.2018 IEEE Sensors Applications Symposium (SAS). Lykele Hazelhoff,Jungong Han and Peter H.N. de. VideoBased Fall Detection in the Home Using Principal Component Analysis.10th International Conference, ACIVS 2008, Juan-les-Pins, France, October 20-24, 2008. Proceedings. Caroline Rougier, Jean Meunier, Alain St-Arnaud, and Jacqueline Rousseau. Robust Video Surveillance for Fall Detection Based on Human Shape Deformation.IEEE Transactions on circuits and systems for video technology,vol.21,No.5,May 2011 611. Homa Foroughi , Alireza Rezvanian ,Amirhossien Paziraee, Robust Fall Detection Using Human Shape and Multi-class Support Vector Machine, 2008 Sixth Indian Conference on Computer Vision, Graphics and Image Processing. Chih-Yang Lin , Shang-Ming Wang , Jia-Wei Hong , Li-Wei Kang , Chung-Lin Huang,Vision-Based Fall Detection Through Shape Feature, 2016 IEEE Second International Conference on Multimedia Big Data. Kamal Sehairi, Fatima Chouireb, Jean Meunier, Elderly fall detection system based on multiple shape features and motion analysis, 2018 International Conference on Intelligent Systems and Computer Vision (ISCV), Fez, Morocco, 2018, pp. 1-8. Durga Priya Kumar , Yixiao Yun , Irene Yu-Hua Gu, Fall detection in RGB-D videos by combining shape and motion feature,2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). M. Yu , S.M. Naqvi , A. Rhuma ,J. Chambers ; One class boundary method classifiers for application in a videobased fall detection system, IET Computer Vision ( Volume: 6 , Issue: 2 , March 2012 ). Homa Foroughi,Baharak Shakeri Aski,Hamidreza Pourreza. Intelligent video surveillance for monitoring fall detection of elderly in home environments.2008 11th International Conference on Computer and Information Technology. Ahsan H. Khandoker ,Daniel Lai , Rezaul K. Begg , Marimuthu Palaniswami.A Wavelet-Based Approach for Screening Falls Risk in the Elderly using Support Vector Machines.2006 Fourth International Conference on Intelligent Sensing and Information Processing. S.-G. Miaou, Pei-Hsu Sung, Chia-Yuan Huang. A Customized Human Fall Detection System Using OmniCamera Images and Personal Infor- mation.1st Transdisciplinary Conference on Distributed Diagnosis and Home Healthcare, 2006. D2H2. Nicolas Thome, Serge Miguet.A HHMM-Based Approach for Robust Fall Detection.2006 9th International Conference on Control, Automa- tion, Robotics and Vision. Chia-Wen Lin , Zhi-Hong Ling. Automatic Fall Incident Detection in Compressed Video for Intelligent Homecare.2007 16th International Conference on Computer Communications and Networks. Xu Zhou ,Li-Chang Qian ,Peng-Jie You, Ze-Gang Ding , YuQi Han.Fall Detection Using Convolutional Neural Network With Multi- Sensor Fusion.2018 IEEE International Conference on Multimedia and Expo Workshops (ICMEW). © 2019, IRJET | Impact Factor value: 7.211 | ISO 9001:2008 Certified Journal | Page 4095