Aframeworkfor Face Classification under
Pose Variations
Alwin D. Anuse
Jagdish P. Sarode
Dept. of E & TC Engineering
Maharashtra Institute of Technology
Pune, India
jagdish.sarode@mitpune.edu.in
Dept. of E & TC Engineering
Maharashtra Institute of Technology
Pune, India
alwin. anuse@mitpune. edu.in
Ab stract- Automatically verifying a person from a video
frame or a digital image using computer application is known as
a Face Recognition system. With changes in facial pose, face
appearance changes drastically . Recognition of faces under pose
variations is much more challenging. Now a day's recognizing
human faces in unconstrained or wild environment is emerging
as a critically important and technically challenging computer
vision
problem.
gradually
Recently
shifting
unconstrained
its
setting.
Database called
as
face
focus
A
"My
new
recognition
on
much
community
more
unconstrained
is
challenging
human
unconstrained Database"
has
face
been
developed in this paper. A model based approach is used and the
Moment based feature extraction techniques (Hu's, Zernike and
Legendre Moments) are implemented on three different face
databases containing different poses of the faces. This paper
proposes a modified method called "Genetic Algorithm based
Transfer Vectors" for generation of features of a frontal face
from the features of different poses of image. Next, the generated
called "Genetic Algorithm based Transfer vectors" for
generation of features of a frontal face from the features of a
different poses of image. Then the generated frontal features
are matched with the actual frontal features [1]. Also this paper
introduces new unconstrained human face Database called as
"My unconstrained Database" which is inspired from IMFDB
[7]. For feature extraction Hu, Zernike and Legendre moments
are used [2] [6]. Feature vectors are obtained for the images in
frontal and a non-frontal poses.
This paper is organized as follows, Section II mentions
flowchart and overview of the Face recognition System,
section III discuss about three different Face Databases used in
this paper, section IV gives details about feature extraction
methods and their algorithms, section V discuss about
proposed Genetic Algorithm based Transfer Vectors method,
section VI and VII gives results and conclusion respectively.
frontal features are matched with the actual frontal features.
II.
Extracted feature are classified by three different methods: kNN
classifier, LDA and Transform Vector with Distance Metric and
finally Correct Recognition Rate is determined.
Index
Terms-
Face
recognition,
Pose
Variation,
Hu's
Moments, Zernike Moments, Legendre Moments, kNN classifier,
LDA,
Genetic Algorithm,
Transfer vectors,
unconstrained face
database.
I.
INTRODUCTION
Automatically verifying or identifying a person from a
video frame or a digital image using computer application is
known as A Face Recognition system. Face Recognition
provides a more direct, friendly and convenient identification
method and it is more acceptable to users as compared to other
biometric methods. Because of variation in face pose angle,
illumination, expression and occlusion there are many
challenges in face recognition. Face appearance changes
drastically with change in facial pose and hence recognition of
faces under pose variations has proved to be a difficult
problem. A pose tolerance technique is needed because most of
the time, for matching purpose of each test face under
recognition, there exists only a one corresponding image of
frontal face in data base. Hence, generation of either the frontal
face image or the frontal face features from the non-frontal face
image is necessary. To generate front face features there is a
need to obtain such a transfonn through learning and finally
transfonnation is used. This paper proposes a modified method
978-1-4799-3080-7114/$31.00 ©2014 IEEE
FACE RECOGNITION SYSTEM FLOW CHART
Flow chart of face recognition system is given in figure 1.
In the preprocessing stage RGB to grayscale conversion,
image resizing, histogram equalization to improve contrast of
the image and padding operations are done. For feature
extraction Hu moments, 4th order Zernike moments and 4th
order Legendre moments are used. Hu moments feature vector
contents seven moments, 4th order Zernike moments gives
feature vector of nine moments and 4th order Legendre
moments gives a vector of fifteen moments. kNN classifier
with k 1, Linear Discrimenant Analysis are used to classify
the face images. Also proposed Genetic Algorithm based
Transfer Vectors method is used to transfer non frontal face
moments to frontal face moments and classification is done
using minimum distance method. Performance of the system
is calculated on the basis of, out of the total given faces how
many faces are correctly recognized and its percentage is
given in the result tables.
=
III.
A.
FACES DATABASES
The FERET Database, USA
The FERET Database was created between 1993 to 1996
[3] [4]. The Database contains 196 subjects that include 100
males and 96 females. The images were collected in a semi­
controlled environment. The size of each image is 256 X 386
pixels and each pixel is of 256 grey levels. The files are in jpg
1886
Authorized licensed use limited to: MIT-World Peace University. Downloaded on May 30,2022 at 06:09:52 UTC from IEEE Xplore. Restrictions apply.
much more challenging unconstrained settings. My
unconstrained Database (MUDB) consist 270 faces of
eighteen well-known personalities in India collected from
web. Fifteen different poses of each person are collected.
Manual selection of faces from web resulted in high degree of
variability (in pose, scale, illumination, expression, age, image
size etc). In the laboratory setting it difficult to capture variety
of natural expressions. There are six female subjects and
twelve male subjects in this Database. All images are in . jpg
format. This can help in designing face recognition algorithms
that can robustly recognize an individual.
Image
Acauisition
(RGB to GreyscaJe, Image Resize,
Histogram equalization, Padding)
I.mage Preprocessing
Summary of Databases used in this paper is given in table 1.
Fig. 2 Subject of My unconstrained Database (MUDB)
Performance Analysis
Fig. 3 Different poses of a particular Subject
TABLE I: SUMMARY OF DATABASE USED FOR TESTING
Fig.1 Face recognition System flow chart
format. Each subject is having following nine different poses
of face rotation 0°, + 60°, +40°, + 2So, +I So, I SO, 2So, 4 0°,
60°
_
_
B.
-
.
Indian Face Database (IIT Kanpur)
This Database is created in campus of Indian Institute of
Technology, Kanpur [S]. It Database is of S7 distinct subjects
having eleven different poses for each person. Images of the
subjects in database are in an upright, frontal position and
captured on bright homogeneous background. For each
individual subject, following looking poses are captured:
front, left, right, up, up towards left, up towards right, down,
laughter, neutral, sad/disgust, smile. The files are in . jpg
format.
C.
FERET
Database
Indian Face Database
(lIT Kanpur)
My Unconstrained
Database (MUDB)
Total Persons
196
57
18
Poses/Person
9
II
15
Poses used
for Training
7
8
12
Poses used
for Testing
2
3
3
-
My unconstrained Database (MUDB)
This paper introduces new unconstrained human face
Database called as "My unconstrained Database" which is
inspired from IMFDB [7]. Recognizing human faces in the
wild environment is emerging as critically important and
technically challenging computer vision problem. With the
introduction of Databases such as LFW, Pubfigs and IMFDB
face recognition community is gradually shifting its focus on
IV.
FEATURE EXTRACTION
Feature extraction is a dimensionality reduction technique.
In this paper Hu's Moments, Zemike Moments and Legendre
Moments are used for feature extraction.
A.
Hu 's Moments:
Hu, in 1962 introduced 'Hu's moments'. These moments
are translation, scale and rotation invariants. Hu's moments
are a set of seven moments. These seven moments are
nonlinear combinations of normalized central moments up to
order three.
20I4International Conference on Advances in Computing, Communications and Informatics (ICACCl)
Authorized licensed use limited to: MIT-World Peace University. Downloaded on May 30,2022 at 06:09:52 UTC from IEEE Xplore. Restrictions apply.
1887
A 19orithm for Feature Extraction by using Hu 's Moments:
I.
2.
3.
4.
Hu's moments of subjects of FERET database for frontal pose
are given in table 2.
Start
Acquire the image
Convert colored image to grayscale image
Calculate Geometrical Central Moments
B.
.
. the moment of order (P+Q),p,q = 0, 1,2, . . .
Where " t,.q IS
and MxN is size of image
5.
Calculate Normalized Central Moments
l flpq =
E!'=l L�l (X - X),P (y- y),C4f(x, y)
(2)
Here x and j are the image centroid and are defined by:
x
=
tll:t .o
tllj).O
Y=
-
and
-'
6.
ij -
Mz
M3
M4
M5
=
=
=
=
=
'Ill
(I !ti)
�ltl() + 2
(4)
1120 + 110z
2
(1120 -1]02) + 411il
(lh o - 31112)Z + (31h1 - lJ03)2
2
(1130 -1]1 ) + (1121 - 1103)Z
2
z
(1]30 - 31112)(1130 + 1112 )[(1130 + 1hz) 2
3(1121 + 1103) ] + (31]z1. - 1103)(11z1. + 1103)[3(1130 1]12)Z - (11z1 + 1103)2]
M6
M7
7.
8.
=
2
(1120 -1]02)[(1130 + 11'12) - (1lz1. + 1103)2 ] +
4711.1(1130 + 1112) (lJ21 + 1103)
=
2
(31121 - 1103)(1130 + 111 ) [(1]30 + 1]I ) 2
Z
3(1121 + lJ03)2] - (1130 - 311'12)(1121 + lJ03)[3(11 30 +
z
1112) - (11z1 + 1103)2]
(5)
Combine these Seven Invariants to form the feature vector
of the image
Stop
1888
I. Start
2. Define the order and repetition of the moment.
3. Acquire the image.
4. Convert colored image to grayscale image.
5. Resize image to 120 x 120 pixels
6. Converts the intensity image I to double precision.
7. Calculate the Radial Polynomial using "(6)"
(3)
Calculate Seven Moments Invariants using Normalized
Central Moments given by Equation (5).
Ml
Algorithm for Feature Extraction by using Zernike Moments:
m� l
HiO , o
Above central moments are origin independent and hence
they are translation invariants. For scale invariance divide
corresponding central moment by ooth moment i. e. scaled
energy of the original moment as given in Equation (4)
11
Zernike Moments (ZM):
Prof. Frits Zemike introduced a set of complex polynomials
called the Zemike polynomials. These polynomials are
defined on the interior of the unit disc, i. e. , x1\2+ y1\2=1 and
form a complete orthogonal basis set ZM is geometrical
orthogonal moments. Orthogonal moments are rotation as well
as scale invariants and very robust in the presence of noise.
(6)
8. Calculate Zernike Moments using "(7)"
Ztun = un 1
E!:-l �=-lf(x, y) V* nm (X, y)
(7)
9. Combine these invariants to form feature vector of the
image.
10. Stop.
Zemike moments of subjects of FERET database for frontal
pose are given in table 3.
C.
Legendre Moments:
These moments are based on the Legendre polynomials
[7]. These are orthogonal Moments.
The two dimensional Legendre Moments of order (m + n), are
defined as "(8)"
.
L_
- pq
=
(
I
) I a. . f a.
p+1) q+1)
4
1
-]
,. .... (1d. I"''I' )f(:A!, y) d� d},
(8)
The image is scaled in the region [-I, 1]. Hence Xi and Yj can
be normalized as "(9)"
2j
2;
x· =
1
,
Yj
eN- I )
1
(M-1)
(9)
and
-
---
=
-
In this paper up to 4th order Legendre moments are calculated.
Legendre moments of subjects of FERET database for frontal
pose are given in table 4.
2014International Conference on Advances in Computing, Communications and Informatics (ICACCl)
Authorized licensed use limited to: MIT-World Peace University. Downloaded on May 30,2022 at 06:09:52 UTC from IEEE Xplore. Restrictions apply.
TABLE 2 HU MOMENTS OF FERET DATABASE FOR FRONTAL POSE
Moments of the subject
Subject of
Database
M,
M,
M3
M.
Ms
M.
M7
Subject 1
6.37421
14.43838
24.17941
23.12982
46.96479
30.61291
47.48578
Subject 2
6.40062
14.64574
23.12028
22.63719
45.57875
29.98992
46.68987
Subject 3
6.30489
14.52329
24.89738
24.91324
49.99588
32.24101
50.78289
Subject 4
6.33768
14.49911
23.90741
25.5165
51.49041
3 3.05424
50.36829
Subject 5
6.41553
14.69512
23.95389
23.99652
48.06056
31.39594
48.98018
TABLE 3 ZERNIKE MOMENTS OF FERET DATABASE FOR FRONTAL POSE
Moments of the subject
Subject of
Database
M,
M,
M3
M.
Ms
M.
M7
Ms
M.
Subject 1
0.6005
0.0906
0.4134
0.0604
0.1096
0.0259
0.9430
0.0611
0.0385
Subject 2
0.5869
0.0963
0.4362
0.0245
0.0366
0.0231
0.9935
0.0349
0.0107
Subject 3
0.5150
0.0284
0.2833
0.0179
0.0762
0.0203
0.7370
0.0195
0.0429
Subject 4
0.6030
0.0229
0.3406
0.0338
0.0652
0.0349
0.8815
0.0084
0.0433
Subject 5
0.6097
0.0413
0.4293
0.0095
0.0433
0.0025
0.9600
0.0279
0.0116
TABLE 4 LEGENDRE MOMENTS OF FERET DATABASE FOR FRONTAL POSE
Moments of the subject
Subject of
Database
V.
M,
M,
M3
M.
Ms
M.
M7
Ms
M.
M,.
Mll
M12
Subject 1
0.1170
-0.0230
0.0234
0.0002
0.0187
0.0544
0.0094
-0.0305
0.0222
-0.0146
0.0175
-0.0104
Subject 2
0.1145
-0.0131
0.0385
-0.0065
0.0177
0.0371
0.0059
-0.0136
0.0046
-0.0134
0.0141
-0.0015
Subject 3
0.1006
-0.0057
-0.0049
-0.0025
0.0115
0.0114
0.0053
-0.0446
0.0000
-0.0283
0.0340
0.0047
Subject 4
0.1175
-0.0085
0.0211
-0.0122
0.0448
0.0610
0.0141
0.0417
-0.0055
0.0335
-0.0196
0.0210
Subject 5
0.1188
-0.0056
0.0260
0.0026
0.0269
0.0383
0.0109
0.0076
0.0062
0.0049
0.0037
0.0020
GENETIC ALGORITHM BASED TRANSFORM VECTORS
This paper proposes new method to transform feature
vector of non frontal face pose to the feature vector of frontal
face pose called "Genetic Algorithm based Transform
Vectors" [J]. Let, matrix Ap contains feature vectors of all
non frontal poses of face and matrix Af contains frontal face
feature vectors. Let U be the desired optimized transform
multiplier to be learned.
U is calculated such that erl"(H' = Af - Ap' * U
will be minimum. The optimized transformation multiplier
"Uop" is calculated iteratively by using Genetic Algorithm.
Algorithm for the fitting function of Genetic Algorithm
used is given below.
1.
2.
Read frontal face vector to variable af
Read feature vectors of all non frontal poses of face
to variable ap(n) respectively where n J, 2, . . . . , No.
of different poses of face.
Find err(n) vector by using Equation
err(n) (af(n) - (ap(n) * U»;
At the first iteration store err_old any high value
in Excel sheet.
=
Thus,
Al
Therefore
=
(10)
, U - Ap
U =
((ApT _ Ap) - 1 (ApT An )
3.
(lJ)
4.
=
=
2014International Conference on Advances in Computing, Communications and Informatics (ICACCl)
Authorized licensed use limited to: MIT-World Peace University. Downloaded on May 30,2022 at 06:09:52 UTC from IEEE Xplore. Restrictions apply.
1889
5.
6.
7.
8.
Find err_new by taking summation of all the
elements of err(n) matrix.
Read err old from excel sheet.
If err_new < err_old, then err_new err_old, else
keep err_old as it is.
Send err_new to genetic algorithm.
=
After obtaining optimized U i. e. Uop, it is used to
transform the vectors of non frontal pose of face to the
corresponding vectors of frontal pose of the face. These
transformed victors are used for feature matching. Euclidean
distance is used for feature matching.
(12)
As given in "12", Euclidean distance between two vectors
is used to give the matching score.
TABLE 7
(%) RECOGNITION RATE OF MY UNCONSTRAINED
DATABASE (MUDB)
Database
Classifier
kNN
Classifier
My
Database
Discriminant
Analysis
Transfonn
Vector with
Distance Metric
Moments
(%) Recognition Rate
Hu
7.97
Zemike
16.72
Legendre
23.05
Hu
9.17
Zemike
25.32
Legendre
25.76
Hu
6.55
Zemike
10.92
Legendre
15.05
Graphical representation of Results are shown in figure 4.
A minimum value of the Score along with its
corresponding frontal pose vector is stored. The frontal posed
vector which gives minimum Score value is the closer match
of two vectors and are said to be matched or of same class.
VI.
TABLE 5
(%) RECOGNITION RATE OF FERET DATABASE
Database
Classifier
kNN
Classifier
FERET
Discriminant
Analysis
Transfonn
Vector with
Distance Metric
TABLE 6
Database
(%) Recognition Rate
Hu
22.45
Zemike
90.45
Legendre
91.66
Hu
52.47
Zemike
85.38
Legendre
91.81
Hu
12.58
Zemike
62.98
Legendre
38.03
Classifier
Moments
(%) Recognition Rate
kNN
Hu
21.21
Zemike
71.80
Legendre
72.28
Discriminant
Analysis
Transfonn
Vector with
Distance Metric
1890
Moments
(%) RECOGNITION RATE OF lIT K DATABASE
Classifier
IIT K
RESULTS
Hu
36.36
Zemike
71.80
Legendre
80.69
Hu
12.25
Zemike
54.85
Legendre
49.99
Fig. 4 Graphical representation of results in Table 5, 6 and 7.
VII.
CONCLUSION
This paper proposes a new method called "Genetic
Algorithm based Transfer Vectors" for generation of features
of a frontal face from the features of a different poses of image.
Also this paper introduces new unconstrained human face
database called as "My unconstrained Database" which is
inspired from IMFDB. In this paper face recognition using
moments has been presented. The proposed method along with
two other standard methods is implemented by authors and
tested on all the three different databases mentioned in this
paper. Nine experiments were conducted. Features were
extracted using three different moments (Hu moments, Zemike
moments and Legendre moments). Two different standard
classifiers (kNN Classifier, Linear Descriminant Classifier)
were used for classification. "Genetic Algorithm based
Transform Vectors" method is used to transform non frontal
vectors to frontal vectors and matching of two vectors is done
using Euclidean distance method.
From the experimental results it is observed that, the
Legendre moments are the best among three moments and it
gives 91. 81% recognition rate with LDA classifier. Results of
Zemike Moments give maximum 90.45% recognition rate.
Zemike Moments are good and can be used for face
recognition. Hu moment gives maximum 52.45% recognition
2014 International Conference on Advances in Computing, Communications and Informatics (ICACCl)
Authorized licensed use limited to: MIT-World Peace University. Downloaded on May 30,2022 at 06:09:52 UTC from IEEE Xplore. Restrictions apply.
rate. The perfonnances of Hu's moments are not satisfactory
and can't be used for face recognition. Proposed "Genetic
Algorithm based Transfer Vector" method perfonnance is
lower than the other two standard methods and need to be
improved
further.
Newly
introduced
"My
unconstrained
Database" is useful to further boost the face recognition
systems under unconstrained pose variations as all three
methods gives lower recognition rate as compared to other two
face Databases.
VIII. ACKNOWLEDGMENT
Jagdish P. Sarode thanks the Management, the Director of
MAEER's
M.I.T.
Pune
for
sponsorship
of
Master
of
Engineering (Electronics - Digital Systems) course and Prof.
(Dr.) G. N. Mulay, for financial help.
IX. REFERENCES
[I]
[2]
[3 ]
[4 ]
Jeet Kumar, Aditya Nigam, Surya Prakash, Phalguni Gupta, "An
efficient pose invariant face recognition system", International
Conference on Soft Computing for Problem Solving (SocProS 2011)
December 20-22, 2011. Vol. -2, pp 145-152, Online ISBN 978-81-3220491-6.
Rajiv Kapoor, Pallavi Mathur, "Face recognition using moments and
wavelets", International Journal of
Engineering and Applications
(UERA) Vol. 3, Issue 4, Jul - Aug 2013, pp. 82 - 95 (ISSN:2248 9622).
http://www.itl.nist.govliadlhumanid/feret/feret_master.html
http://www.itl.nist.govliadlhumanid/feret/
[5]
Vidit Jain, Amitabha Mukherjee. The Indian Face Database. http://viswww.cs.umass.edul-viditiIndianFace Database.
[6]
Jan Flusser, Thomas Suk and Barbara Zitova, "Moments and Moments
Invariants in Pattern Recognition", John Wiley & Sons, Ltd. ISBN: 9780-470-69987-4
[7]
Shankar Setty, et al. "Indian movie face Database: a benchmark for face
recognition under wide variations", 4th IEEE National Conference on
Computer Vision, Pattern Recognition, Image Processing and Graphic
(NCVPRIPG), 18 - 21 December, 2013. Jodhpur, ISBN: 978-1-47991586-6
[8]
R. Rajalakshmi, M. K. Jeyakumar, "A review on classification used in
face recognition methods under pose and illumination variation",
International Journal of Computer Science Issues (UCS!), Vol 9, Issue 6,
No 2, November 2012, (rSSN:1694-0814)
[9 ]
Shan Du, Rabab ward, "Face recognition under pose variations",
ScienceDirect, Journal of the Franklin Institute 343 (2006) 596 613
[10] Simon
Xinmeng
Liao,
"Image
analysis
by
moments",
the
department of electrical and computer engineering, the University
of Manitoba Winnipeg, Manitoba, Canada, 1993.
2014International Conference on Advances in Computing, Communications and Informatics (ICACCl)
Authorized licensed use limited to: MIT-World Peace University. Downloaded on May 30,2022 at 06:09:52 UTC from IEEE Xplore. Restrictions apply.
189 1