FACIAL APPEARANCE MODEL
Ing. Miroslav Kasár
supervisor: prof. Ing. Ján Mihalík, PhD.
Laboratory of Digital Image Processing and Videocommunication
Dept. of Electronics and Multimedia Communications, FEI TU Košice
Park Komenského 13, 041 20 Košice, Slovak Republic.
Tel: +421-55-6024309, E-mail: Miroslav.Kasar@tuke.sk
ABSTRACT
This paper deals with construction of the face
appearance model (FAM) from training set of face
images. Because the training images can be obtained
from different sources, they have to be normalized
before applying principal component analysis (PCA).
Significant task by construction FAM has a wireframe
model Candide-3.
1
INSTRUCTION
Appearance model is combination of shape model
and model of normalized textures [1]. For
construction FAM are required face images with
adapted model Candide-3. The adapted models have
been normalized (geometrically, energically) and then
PCA is apllied to normalized textures.
In section 2 the wireframe model Candide-3 and
its parameterization are described. Geometrical
normalization of the human face is described in
section 3 and in section 4 FAM is presented.
2
CANDIDE-3 AND ITS
PARAMETERIZATION
Candide-3 is 3D wireframe model (Fig. 1), which
is improved version of Candide-0. Candide-0 was
created by Mikael Rydfalk at Linkőping University in
1987. This work was motivated by first attempts to
perform image compression through animation, later
called model-based, object-based or semantic coding
[2]. Improvement was realised by Jőrgen Ahlberg
from Linkőping Univerzity in 2001 [3]. Candide-3
contains 113 vertices and 184 triangles (polygons).
The geometry of the wireframe model is
parameterized according to
x  x  S  A
(1)
where the resulting vector x contains the (x, y, z)
coordinates of vertices of the model. Vector x is
standard shape of the model, and the columns of S and
A matrices are the Shape and Animation Units
respectively, and thus  and  contain the shape and
animation parameters.
To perform the global motion six more parameters
for rotation, scaling and translation is needed to
append. Thus, (1) is replaced with


x  sR x  S  A  t
(2)
where R = R(Θx, Θy, Θz) is rotation matrix, s is the
scale, and t = t(tx, ty) is the 2D translation vector. The
geometry of model is thus parameterized by the
parameter vector
p
Θx , Θ y , Θz , s , t x , t y , ,  
T
(3)
The texture g is mapped on the surface of
wireframe model and it is represented as a standardshaped image, being a linear combination of a set of
Texture Units consisting of geometrically normalized
eigenfaces. This is formulated as
g  g  Pg b g
(4)
where g is the mean texture, the columns of Pg are
the TUs and bg is the vector of texture paratemers.
3
a)
b)
Fig. 1: 3D model Candide-3 a) front view, b) profile.
GEOMETRICAL NORMALIZATION OF
HUMAN FACE
Geometrical normalization of human face uses to
obtain its normalized texture, removes texture
variations caused with global and local motion and
geometrical
differences
between
individuals.
Geometrical
normalization
is
a
non-linear
transformation, which warps source image of human
face with adapted wireframe model Candide-3 to the
standard shape of the model Candide-3 with given
scale (Fig. 2).
Fig. 2: (from left) Adapted model, standard shape.
4
DESIGN OF FACIAL APPEARANCE
MODEL BY PCA
Before applying PCA to the relevant section of
normalized textures is necessary to minimize the
effect of global lighting variation. Therefore
geometrically normalized textures are centred (the
sum of elements is zero) and energically normalized
(the variance of elements is unity). By applying PCA
[4] to these normalized data a linear FAM (4) is
obtained. Fig. 5 displays 10 normalized textures from
training set and 10 eigenfaces corresponding to the
largest eigenvalues.
Texture mapping on the standard geometry shape
is realised by following algorithm:
1. For each pixel coordinate (x, y) in the destination
image compute the barycentric coordinates (a, b,
c) regarding to the first triangle in the destination
mesh.
2. Compute the source coordinates (x’, y’) from the
barycentric
coordinates
applied
to
the
corresponding triangle in the soure mesh.
3. Interpolate the source image by the destination
image.
This approach is repeated for all triangles of given
model. Results of texture mapping are illustrated in
Fig. 3. Note that the faces are more similar to each
other after the normalization process than before.
Fig. 3: (from left) Adapted model, standard shape
with texture, normalized texture.
After obtaining normalized texture is important to
select its relevant section, which contains the most
important parts of human face (eyes, lip, nose…). Fig.
4 displays normalized textures (top) and its relevant
sections (bottom).
Fig. 5: (top) Normalized faces, (bottom) eigenfaces.
5
CONCLUSION
Using the FAM is possible to represent human
faces, which are not present in the training set.
Presented FAM is very important element of active
face appearance model (AFAM), which is used for
tracking the motion of face in head-and-shoulders
videosequences and other analysis/synthesis tasks.
The work was supported by the Grant Agency
of Ministry of Education and Slovak Academy of
Science under Grant No. 1/0384/03.
REFERENCES
[1] Cootes,T.F.-Taylor,C.J.: Statistical Models of
Appearance for Computer Vision. Imaging
Science and Biomedical Engineering, University
of Manchester, 2004.
[2] Mihalík,J.: Image Coding in Videocommunication.
Mercury-Smekal, ISBN 80-89061-47-8, Košice,
2001. (In Slovak)
[3] Ahlberg,J.: Candide-3 – an Updated Parameterized
Face. Report No.LiTH-ISY-R-2326, Dept. of EE,
Linköping University, 2001.
Fig. 4: (top) Normalized textures, (bottom) relevant
sections of normalized textures.
[4] Kasár,M.: Principal Component Analysis of
Images. Zborník IV. Doktorandskej Konferencie
a ŠVOS, FEI TU Košice, 2004, p.57-58.