A Vision-Based System that Detects the Act of Smoking a Cigarette
Xiaoran Zheng, University of Nevada-Reno, Dept. of Computer Science
Dr. Mubarak Shah, Dr. Niels Lobo, University of Central Florida, Computer Vision Lab
Dr. George Bebis, Dr. Dwight Egbert, University of Nevada-Reno, Dept. of Computer Science
Results:
Abstract:
Tracking of facial features
3. Tracking of the head region during occlusion:
This project attempts to develop an automatic system that processes a sequence
of images obtained from a video camera to detect if a person is smoking a
cigarette. So far, we found methods to track mouth and arm movements through
a number of image frames with the smoker’s upper body occupying 1/4 of the
image size. Further work on interpreting the information extracted from tracking,
and detecting smoke in an image will ultimately produce such a system.
The number of skin regions indicates whether occlusion is present. The method
employed here is correlation. Since we have already found the head in the previous
frame where occlusion does not occur, the search window can be established to find
the head in the current frame where occlusion occurs. The method is further extended
by using the boundary points of the head, which have already been defined, to correct
the small variations that result from correlation.
Problem Statement:
4. Tracking of facial features:
Cigarette smoking poses dangers to our health, environment and safety.
Therefore, when smoking is prohibited in certain areas, the enforcement of this
rule is important. Using physical smoke detectors poses constraints. A smoke
detector is not suitable for the level of smoke produced by a cigarette, especially
in large areas. Letting a human monitor a designated area through surveillance
camera is rather inefficient. If a computer is programmed to recognize this
activity, an automatic system can be set up to alert humans when the computer
“sees” someone smoking through a video camera.
Facial features are extracted using Sobel edge detector. The size and shape of these
edges are used to locate eyes. A new color space is calculated from the RGB space to
identify and track lips.
Tracking of arms
Edges
Current Approaches:
New Color Space
5. Tracking of arms:
There does not exist a system in which the activity of cigarette smoking can be
automatically recognized by a machine.
A best-fit line that passes through the center of mass of an arm region provides
orientation, as well as the end points of that region. By keeping a history of previous
centroids and end points, we can track the two arms. If an arm is occluding the other
arm, the program goes to the previous frame where all skin regions are separated, and
takes the separated arm as a template. The template is rotated and is matched against a
mask that contains all the skin regions in the current frame. If an arm is merged with
the head, the program utilizes the method of rotating template for the first three
frames, then uses correlation. It is also necessary to keep track of the size of each arm
region, which is used to detect body rotation.
Methodology:
1. Segmentation of human skin:
Skin regions are segmented out using color predicate. These regions are
manually extracted in the first frame and used as a mask in the training process.
Successive frames are tested against the color space generated through training.
Conclusion:
Training Image
Mask
The goal of this research project is to experiment with different ideas and methods that
will enable a computer to detect if a person is smoking a cigarette. So far, we have
found methods to track mouth and arms through a sequence of images with the
smoker’s upper body occupying 1/4 of the image size.
Skin Regions
2. Locating the head region:
Connecting components is performed to separate skin regions. The head is
identified by examining the circularity of each region.
Skin Regions
Boundary Points
Head Region
Future Work:
Acknowledgements:
The continuation of this project includes the analysis of the time a hand is positioned
at the mouth, the interpretation of the gesture of an arm moving up and down, and the
detection of smoke around the head region.
•The Nation Science Foundation
References:
•University of Central Florida, Computer Vision Lab
Further information can be found at http://www.cs.ucf.edu/~zheng
•University of Nevada-Reno, Computer Science Department
•University of Nevada-Reno, Office of Research