ADAPTIVE MATCHING WAVELET NETWORKS FOR FACE RECOGNITION
Yang Zhi
Gu Ming
School of Software, Tsinghua University, Bei Jing , P.R.China 100084
yangz02@mails.tsinghua.edu.cn
guming@mail.tsinghua.edu.cn
ABSTRACT
This paper describes a novel Adaptive Matching method
for face recognition. We employ an Face Bunch
Graph(FBG), which has reconstructed Gabor feature on
each FBG nodes. The reconstructed Gabor feature comes
from the orthogonal analysis of a family of Gabor
wavelet coefficients, which reduces the redundant
information. After selecting fiducial points roughly by
elastic bunch graph matching(EBGM) algorithm, the
reconstructed feature was used for the relocation. The
Wavelet Networks(WN) and best matching fiducial
points’ location contain the discriminative information of
faces. We proposed a new approach to compute the
similarity between two faces on both the global means and
topological means. The experimental results show that our
algorithm is an effective method compared with EBGM,
PCA, HMM face recognition approaches.
1.
INTRODUCTION
Face Recognition had been researched for nearly two
decades and many effective methods had been developed.
Today, successful methods can be mainly divided into
two classes: one focuses on the global senses (such as
eigenface and fisherface),the other prefers to analyze the
constitutive local feature of faces( eyes, nose or mouth) .
It is no doubt that among those local approaches, the
EBGM algorithm[1] is the most successful one by
applying Gabor Wavelet analysis to local feature points.
This method compute the related value among feature
points to form an elastic grid graph. The similarity can be
computed by comparing the difference between two
elastic grid graphs. Some smart improvement of EBGM
algorithm were achieved. María José Escobar and Javier
Ruiz-del-Solar
applied EBGM on a Log Polar
Transformation to locate faces’ landmarks[2]; Anastasios
Tefas,Constantine Kotropoulos, and Ioannis Pitas utilized
Support Vector Machine(SVM) to enhance performance
of EBGM[3]; based on EBGM, Stefano Arca, Paola
Campadelli, Raffaella Lanzarotti proposed an completely
This project is sponsored by National 973 Plan of China ( Project
Number: 2004CB719400).
0-7803-9134-9/05/$20.00 ©2005 IEEE
automatic face-recognition system and they reported
good performance of this system[4]. However, to our
knowledge, our work that using Adaptive Matching
Wavelet Networks, which is also based on EBGM, is a
new Face Recognition approach.
We utilize the wavelet analysis and energy function
as supervision to direct the matching process. Besides, we
also utilize Wavelet Networks, which is a kind of
universal approximator and achieves faster convergence
over radial basis function networks (RBFN). In WN, the
radial basis functions of RBF-networks are replaced by
wavelets. WN inherit the properties of wavelet
decomposition and mention especially their universal
approximation property, the availability of convergence
rates and the explicit link between the network
coefficients and the wavelet transform, and during the
training phase, the network weights as well as the degrees
of freedom (position, scale, orientation) of the wavelet
functions are optimized [5].
The outline of this paper is as follows: Section 2 will
provide some background of the Elastic Bunch Graph
Matching. An improved method on matching facial
landmarks will be presented in Section3. In Section4, a
Wavelet Network will be applied to Face Recognition. In
the final Section, the experiment results will be shown.
2 . RELATED BACKGROUND
In order to express a face’s important feature, we can
extract a group of wavelet coefficients from the
convolution of a family of Gabor Wavelet Kernels with
input image. The 2-Dimension Gabor Wavelet Kernel can
be defined as follows:
r
ψ j ( x) =
k 2j
σ
exp(−
2
k 2j x 2
rr
σ2
)[exp(ik j x ) − exp(− )]
2σ
2
(1)
2
v+2
−
π
r  k jx   k cos φu 
π
 kv = 2 2 φu = u
k j =   =  v
k
8
k
cos
φ
u 
 jy   v
σ is the standard deviation of the Gaussian, with ν the
frequency parameter. The latter term in the second
bracket makes the function ψ j ( xr )d 2 xr. converge.
∫
At one selected point(fiducial point) of face
image ,we extract a series of Gabor Wavelet coefficients
as the local feature, namely a Jet. For example, 5
frequencies and 8 orientations means a Jet with 40
coefficients.
After having chosen all the Jets, the fiducial points
have constructed the nodes of an FBG with its edges.
Because all the fiducial points in FBG have the identical
structure, different distance value of one elastic gragh
represent different appearance of the same human’s face.
So on a fiducial point, there will be a group of value,
each indicating an appearance. The group is called a
bunch. All the edges are labeled with average value
between two fiducial points: ∆xre = ∑ ∆xrem / M .
m
3. ADAPTIVE WAVELET FEATUER MATCHING
In [1], a heuristic algorithm is used to find the image
graph which maximizes the graph similarity function.
First, the FBG is resized and aspect ratio to adapt to the
right format of the face. Then all nodes are moved locally
and relative to each other to optimize the graph
similarity(see (2)) further[1].
r
r
( ∆ x eJ − ∆ xeB ) 2
1
λ
S B (G j , B ) = ∑ max( S φ ( J nj , J nBm )) − ∑
rB 2
N
where
E
n
Sφ ( J , J ' ) =
e
rr
∑ a j a ' j cos(φ j − φ ' j − d k )
( ∆ xe )
(2)
j
∑ a ∑ a'
2
j
2
j
(3)
j
j
Their results could be better convinced if they have
considered the redundant information, so some fiduicial
points may be misplaced although they get the maximal
similarity. Here we proposed an adaptive remedial method
to find more reasonable matching fiducial points.
3.1 Reconstruction Of Jets
As mentioned above, an input face image I can be
decomposed into a family of Gabor Wavelet coefficients.
On the contrary , the coefficients can not easily
reconstruct the image Iˆ by a linear combination ,because
the Gabor wavelet functions are not orthogonal. If we
really want to do so and let wi be the weight of
corresponding Gabor wavelet ψ i , we have
N
r
r
(4)
Iˆ ( x ) = ∑ w iψ i ( x )
r
r
r
r r
Jˆ j ( x ) = ∫ Iˆ( x )ψ j ( x − x ' )d 2 x '
r
r r
r
= ∑ wi ∫ψ i ( x )ψ j ( x − x ' )d 2 x '
i =1
In order to reconstruct Ĵ j ,we must compute wi first,
but it is not possible to calculate it by a simple projection
of the Gabor wavelet ψ j onto the image[5] (as it is being
done for orthogonal wavelets). Instead we have to
~
consider the family of dual wavelets Ψ
= {ψ~1 ,...ψ~ N }. The
~
wavelet ψ j is the dual wavelet to the wavelet ψ j if :
if i = j
if i ≠ k
(6)
ψ~i = ∑ (Ψi , j ) −1ψ j where Ψi , j = ⟨ψ i ,ψ j ⟩
(7)
1
r
r r
⟨ψi ,ψ~j ⟩ = ∫ψi (x)ψ~k (x)d2x =δi, j = 
0
ψ~i can be computed as follows:
j
This function’s proof can be found in [5].Then we can
(8)
calculate wi directly:
wi = ⟨ I ,ψ~i ⟩
3.2 Adjusting The Weight Of Wavelet
The reconstructed image is not completely identical to the
original image, thus there is an difference between
them[6]. We use an energy function to express the
r
r
1
difference:
(9)
E = ∑ [ I ( x ) − Iˆ( x )]2
2
Applying conjugate gradient method can minimize E and
adjust wk .
3.3 Adaptive Matching
The FBG is constructed as follows: firstly, compute all
fiducial points based on EBGM; secondly, compute all
the dual wavelets and get their corresponding weights
wi according to (8); thirdly, minimize the difference (9)
by applying conjugate gradient method; fourthly, compute
the reconstructed Jet according to (5); finally, add
reconstructed Jet into bunch.
The matching process includes three stages. In the
first stage, matching all the fiducial points based on
EBGM; in the second stage, computing the reconstructed
Jet; in the third stage, all nodes are moved locally and
relative to each other by computing among reconstructed
Jets to optimize the graph similarity by (2) .
i =1
where Iˆ is the reconstructed image and i=1,….,N means a
set of
Gabor Wavelet(in our experiment N=40).
Therefore a Jet can be approximated as:
(5)
N
4. FACE RECOGNITION
4.1
An Introduction To Wavelet Networks
Recently, wavelet networks(WN) was applied in many
research fields, such as Pose Estimation [6], Facial
Feature Detection [7], Face Alignment [8],etc. A Wavelet
Networks can be defined as:
Letψ ni i = 1,....,N be a set of wavelets, f a DC-free
image and wi and ni chosen according to the energy
functional (10). The two vectors Ψ = (ψ n1 ,..., ψ nN )T ,
and W = ( w1 ,..., wN )T , define then the wavelet network
(Ψ,W) for image f.
Where parameter vectors n = (c x , c y ,θ , s x , s y ) .The
c x , c y defines the translation of the wavelet, s x , s y defines
the dilation and θ defines the orientation of the wavelet.
The parameters vector n (translation, orientation and
dilation) of the wavelets maybe chosen arbitrarily at this
point[5]. According to wavelet theory, any function
f ∈ L2 ( R 2 ) can be losslessly represented by their
continuous wavelet transform and thus, with arbitrary
precision, by a wavelet network. We therefore interpret
the image f to be a function of the space f ∈ L2 ( R 2 ) and
assume further, without loss of generality that f is DC-free.
In order to find the WN for image f , we minimize the
energy function:
N
(10)
E = min f − wψ
∑
ni , wi for
i =1
all
i
ni
N
i
i =1
N
ni
− ∑ wiψ ni
(11)
i =1
algebraic transformations lead to:
v−w


:= ∑ (vi − wi )(v j − w j )⟨ψ i ,ψ j ⟩ 
Ψ

 i, j
= (v − w)T ( Ψ )(v − w)
12
(12)
Where • computes the Euclidean distance between the
Ψ
two appropriate points in < Ψ > and thus considers the
different parameters of the wavelets. For orthogonal
wavelets, the matrix (Ψ )i , j = ⟨ψ ni ,ψ nj ⟩, is the unity matrix
and no weighting is needed[5].
4.2
Ψm .
We have many methods to compute the difference
between two faces, among them the simplest method is to
calculate the sum of all Dm . As mentioned in[1], we can
calculate the similarity between two faces on an average
over the difference between pairs of corresponding face
appearance. For an image graph G I , the similarity
between it and a stored FBG B with edges e = 1, ..., E is
defined as :
(13)
S (G I , B ) =
r
r
( ∆xeI − ∆xeB ) 2
λ
1
− ∑ Dm (V , Wm ) − ∑
r
M m
E e
(∆xeB ) 2
Where m means the m-th appearance’s (Ψm ,Wm ) stored
in the FBG B. λ determines the relative importance of
difference and the topography term.
5. EXPERIMENTS
i
If we want to compute the distance between two vectors
of wavelet coefficients v and w , for example , we can
compute the Euclidean distance in the (image) subspace
< Ψ > between the v and w :
∑vψ
and then optimize them based on function(9) (this two
steps have been done by the adaptive matching). After
getting Vn we will compare it with Wm of each person ,
the difference is defined as: Dm (V , W m ) = V − W m .
To check the validity of our algorithm, experimental
studies are carried out on the ORL face image database of
Cambridge University. 400 face images from 40
individuals in different states from the ORL have been
used to evaluate the performance of the proposed method.
The sample images vary in position, rotation, scale and
expression. In this database each person has changed his
face expression in each of 10 samples (open/close eye,
smiling/not smiling). For some individuals, the images
were taken at different times, varying facial details
(glasses/no glasses). Fig.1 shows some matching results
on ORL database.
The first experiment was tested for comparing the
recognition rate between EBGM and our proposed
method, and that maybe indicates the matching rate of two
algorithms. We used 25 individuals totally 250 images for
the test and extracted face feature from 13 fiducial points
Face Recognition Based On WN
In order to apply face recognition algorithm, we first
extract Wm = ( w1 ,..., wN )T m , and Ψ m = (ψ 1 ,..., ψ N ) T m .
of the m-th appearance of each person’s face. The wavelet
network (Ψ , W ) = ((Ψ1 ,..., ΨM ), (W1 ,..., WM )), of each person
is regarded as his or her prototype sample and stored in
this person’s FBG.
When a new face image is input, we extract its
coefficient vectors V = (v1 ,..., vN )T , according to (8),
Figure1. The matching results on ORL Face Database: the top row
shows the matching results of our proposed method, the results of EBGM
on the bottom. The matching points on top row was adjusted from the
position of matching points of bottom row’s images.
Figure2. The Recognition rate curves of Adaptive Matching and EBGM.
to 23 fiducial points. Each time after having added
another fiducial point we randomly chose 20 faces to test
Recognition Rate. The comparison results is shown in
Fig.2. We can see that our method gain a higher
recognition rate when number of ficucial points increase.
Another experiment is to test the robustness of
classification capacity of our algorithm. The other three
effective methods were tested in our experiment--- PCA,
HMM and EBGM, because they were proved to be valid
in the Face Recognition applications[9]. We increased the
trainning gallery from 25 to 40 individuals in the
experiment and chose randomly among them to test
recognition rate, regardless the expression or pose, each
new individual with 10 images. After being added
another new individual, the test was carried on by
randomly choosing 40 images from the current face
database. The HMM algorithm utilized EHMM model and
selected 8 DCT coefficients as its input feature. EBGM
and AM used 23 fiducial points. The Fig.3 show that
HMM method and our AM method achieved higher
performance than the other two approaches.
proposed an approach to resolve it. This method
computes the dual wavelets to get weight of each wavelet,
then optimizes the weights to reconstruct the Gabor Jets .
Through the adjusted Jets we can find the fiducial points
whose locations are more accurate. The Wavelet
Networks was used for the face-feature’s classification.
For doing so we proposed a new approach for computing
similarity between two faces, and this method not only
consider the global representation of face but also take
topological information into account. The algorithm was
tested on the ORL database and proved to be useful for
face-feature’s location and face recognition. Future work
will be carried on estimating similarity based on a
statistical view over wavelet difference.
REFERENCES
[1] Laurenz Wiskott, Jean-Marc Fellous, Norbert Krüger,
Christoph von der Malsburg , “Face recognition by elastic bunch
graph matching,” IEEE trans on pattern analysis and machine
intelligence, Vol.19, NO.7,pp.775-779, July 1997.
[2] Escobar, M.J., and Ruiz-del-Solar, “Biologically –based face
recognition using Gabor Filters and Log-Polar Images,” IJCNN
2002, Honolulu, USA, pp.1143 – 1147, 2002.
[3] Anastasios Tefas,Constantine Kotropoulos, and Ioannis Pitas,
“Using support vector machines to enhance the performance of
elastic graph matching for frontal face authentication,” IEEE
trans on pattern analysis and machine intelligence, Vol. 23, NO.
7, pp.735-745, 2001.
[4] Stefano Arca, Paola Campadelli, Raffaella Lanzarotti, “An
automatic feature-based face recognition system,” WIAMIS
2004, Portugal ,May 2004.
[5] Volker Krüger, Sommer G, “Wavelet networks for face
processing.” JOSA-A(19),No. 6, pp. 1112-1119,June 2002.
[6] Volker Krüger, Gerald Sommer, “Gabor wavelet networks
for efficient head pose estimation,” Image Vision Compute.
20(9-10), pp. 665-672,2002.
[7] R. S. Feris, J. Gemmell, K. Toyama and V. Krueger, “Facial
feature detection using a hierarchical wavelet face database,”
Microsoft Research Technical Report MSR-TR-2002-05, 2002.
Figure3. Face recognition performance of PCA, EGBM, HMM and our
proposed method on the ORL database as number of individuals increase.
6.CONCLUSION
In this paper, we present a novel method for Face
Recognition. Based on EBGM, we analysis the problem
of redundant information of elastic matching and
[8] Volker Krüger, “Gabor wavelet networks for object
representation,” Technical Report CS-TR-4245, University
of Maryland, CFAR, May 2001.
[9] W. Y. Zhao, R. Chellappa, A. Rosenfeld, and P. J. Phillips.
“Face recognition: a literature survey”. UMD CfAR Technical
Report CAR-TR-948, 2000.