School of Electrical, Computer and Energy Engineering
M.S. Final Oral Defense
Grassmannian Learning for Facial Expression Recognition from Video
by
Anirudh Yellamraju
11/17/2014
1:00 pm
GWC 409
Committee:
Chaitali Chakrabarti, Co-Chair
Pavan Turaga, Co-Chair
Lina Karam
Abstract
In this thesis we consider the problem of facial expression recognition (FER) from
video sequences. Our method is based on subspace representations and Grassmann
manifold based learning. We use Local Binary Pattern (LBP) at the frame level for
representing the facial features. Next we develop a model to represent the video sequence
in a lower dimensional expression subspace and also as a linear dynamical system using
Autoregressive Moving Average (ARMA) model. As these subspaces lie on Grassmann
space, we use Grassmann manifold based learning techniques such as kernel Fisher
Discriminant Analysis with Grassmann kernels for classification. We consider six
expressions namely, Angry (AN), Disgust (Di), Fear (Fe), Happy (Ha), Sadness (Sa) and
Surprise (Su) for classification. We perform experiments on Cohn-Kanade (CK+) facial
expression database to evaluate the expression recognition performance. We achieve an
average recognition accuracy of 97.41% using a method based on expression subspace,
kernel-FDA and Support Vector Machines (SVM) classifier. By using a simpler
classifier, 1-Nearest Neighbor (1-NN) along with kernel-FDA, we achieve a recognition
accuracy of 97.09%. We find that to process a group of 19 frames in a video sequence,
LBP feature extraction requires majority of computation time (97 %) which is about
1.662 seconds (Intel Core i3, dual core platform). When only three frames (onset, middle
and peak) of a video sequence are used, the recognition accuracy drops to 92.88 %,
however the computational complexity is reduced by about 83.75 % (260 milliseconds).
Overall by using this framework we demonstrate good expression recognition
performance outperforming other state of art FER algorithms.