A Drowsy Driver Alert Interface using Facial Expression Recognition
Paper Presenter: Sunilduth Baichoo, Dept. of Computer Science and Engineering
Author(s):




Sunilduth Baichooo, Dept. of Computer Science and Engineering
Shakuntala Baichoo, Dept. of Computer Science and Engineering
Sundeep Ganeshe, Dept. of Computer Science and Engineering
Yadassen Sandiren Palaniandy, Dept. of Computer Science and Engineering
Extended Abstract
A facial expression is a visible manifestation of the affective state, cognitive activity,
intention, personality, and psychopathology of a person (Fasel et al., 2003). Facial
expressions are natural means of communications between humans. By looking at the
facial expression of a human face the emotional state of that person can be understood.
Research in Facial Expression Recognition has considered seven basic facial expressions
namely anger, disgust, fear, happy, sad, surprise and neutral (Ekman et al., 1971).
With the advancement in the field of face recognition and face detection and the
increased availability of relatively cheap computer power (Turk et Al., 1991), there has
been an increasing interest in the study of automatic facial expression recognition
(AFER) in the last decade. While human beings can easily interpret facial expressions, it
is a challenging task for computers to interpret the latter. An AFER system is one which
can automatically determine the expressions of a particular user. AFER consists of four
main steps: face detection, pre-processing, facial feature extraction and lastly facial
expression classification and recognition.
For the Driver Alert Interface the basic expressions fear, disgust, anger and an additional
expression “sleepy” are used. The proposed system is developed using the four
components of the AFER system, as follows:
Face detection involves locating the face from an image or video stream from a webcam
or camera. Here Face Detection is carried out using Local Successive Mean Quantization
Transform (SMQT) Features and Split Up Sparse Network of Windows (SNOW)
Classifier developed by Mikael Nilsson, Nordberg Jorgen and Ingvar Claesson (Nilsson
et al., 2007).
Pre-processing involves improving the quality of the image.
Facial feature extraction: The prominent facial features of the face-eyebrows, eyes, nose,
mouth, and chin are extracted using the model developed by Matthew Turk and Alex
Pentland.
Classification and recognition: A feed-forward back-propagation neural network has been
used. The classifier is used to categorize the facial features extracted from the previous
step to determine the expression being conveyed by the face by feeding the corresponding
weight vectors to the input layer of the neural network. The number of neurons in the
hidden layer is 20 and the number of neurons in the output layer is 4 corresponding to 3
basic facial expressions (fear, disgust and anger) and also the sleepy state. The back
propagation algorithm has been used for the Neural Network learning process.
After the Neural Network has learned the four expressions and recognized the emotion, it
can simulate an unknown data. If this recognition result corresponds to the true emotion
of the user, then the recognition result is said to be correct.
Two facial expressions databases were used to train the system and it was found that the
accuracy of the system was higher using the JAFFE database than MMI. The highest
recognition rate for the JAFFE facial expressions on static images was 85.7 % and at realtime 56%. When integrated with the Drowsy Driver Alert Interface (DDAI), the overall
accuracy of the system was acceptable, about 60%. The system was also tested for
images with occlusions with rather good results.
Though the system yielded good performance, some expressions were confused with
each other e.g. if a person is surprised with the mouth open, the identified expression is
that of ‘happiness’.
References
Ekman, P., Friesen, W. V., “Constants across cultures in the face and emotion”, Journal
of Personality and Social Psychology, 1971, 17, 124-129.
Fasel B., Luettin J., “Automatic facial expression analysis: a survey”, Pattern
Recognition, 36(1): 259-275 (2003)
Nilsson M., Nordberg J., Claesson I., “Face Detection using Local SMQT Features and
Split Up SNoW Classifier”, pp. in proc. IEEE International Conference on Acoustics,
Speech, and Signal Processing (ICASSP), Honolulu, 2007.
Turk, M., and Pentland, A., “Eigenfaces for recognition”, Journal of Cognitive
Neuroscience, Vol, 3, pp. 71-86, (1991).