Automatic 3D Face
Recognition System
Biometric Authentication
695410042 邱彥霖
491410044 龔士傑
Outline





Introduction
Nose Tip extraction
Pose Correction
Face Segmentation
Recognition
Introduction
Nose Tip extraction
Reference
[1] X. Lu and A. K. Jain. "Multimodal facial feature extraction for
automatic 3D face recognition," Technical Report MSU-CSE-0522, Department of Computer Science, Michigan State University,
East Lansing, Michigan, August 2005.
[2] X. Lu and Anil K. Jain, "Automatic Feature Extraction for
Multiview 3D Face Recognition," Proc. 7th IEEE International
Conference on Automatic Face and Gesture Recognition
(FG2006), pp. 585-590, Southampton, UK, Apr. 2006.
[3] M.L. Koudelka, M.W. Koch, T.D. Russ, A prescreener for 3D face
recognition using radial symmetry and the Hausdorff fraction, in:
IEEE Workshop on Face Recognition Grand Challenge
Experiments, June 2005.
[4] A. S. Mian, M. Bennamoun and R. A. Owens, “Automatic 3D
Face Detection, Normalization and Recognition”, 3DPVT, 2006.
Face Segmentation
[1] X. Lu and A. K. Jain. "Multimodal facial feature extraction for automatic 3D face recognition.”
Nose Tip Extraction :
Frontal scan
[1] X. Lu and A. K. Jain. "Multimodal facial feature extraction for automatic 3D face recognition.”
Nose Tip Extraction :
Frontal scan
[1] X. Lu and A. K. Jain. "Multimodal facial feature extraction for automatic 3D face recognition.”
Nose Tip Extraction :
Pose change


For a frontal facial scan, nose tip usually has
the largest z value.
But, in the presence of large pose changes,
this heuristic does not hold.
[2] X. Lu and Anil K. Jain, "Automatic Feature Extraction for Multiview 3D Face Recognition."
Nose Tip and Pose Estimation
Pose quantization

The yaw angle change ranges from -90 degrees
(full right profile) to 90 degrees (full left profile) in
the X-Z plane.
[2] X. Lu and Anil K. Jain, "Automatic Feature Extraction for Multiview 3D Face Recognition."
Nose Tip and Pose Estimation
Directional maximum
 xi j
 
 yi j
  j
 zi
  cos  j
 
 0
 
   sin  j
sin  j  xi 
 
1
0  yi 
0 cos  j  zi 
0
[2] X. Lu and Anil K. Jain, "Automatic Feature Extraction for Multiview 3D Face Recognition."
Nose Tip and Pose Estimation
Pose correction
 x'   cos 
  
 y'    0
 z '    sin 
  
sin   x  p x 


1
0  y  p y 
0 cos   z  p z 
0
[2] X. Lu and Anil K. Jain, "Automatic Feature Extraction for Multiview 3D Face Recognition."
Nose Tip and Pose Estimation
Nose profile extraction
[2] X. Lu and Anil K. Jain, "Automatic Feature Extraction for Multiview 3D Face Recognition."
Nose Tip Extraction :
Pose change

Nose Tip Extraction :


Radial Symmetry Map
Gradient and Zero-Crossing Map
[3] M.L. Koudelka, M.W. Koch, T.D. Russ, A prescreener for 3D face recognition using radial symmetry and the Hausdorff fraction.
Radial Symmetry Map

First, the gradient of the image, g is computed at
each pixel p.
[3] M.L. Koudelka, M.W. Koch, T.D. Russ, A prescreener for 3D face recognition using radial symmetry and the Hausdorff fraction.
Radial Symmetry Map

For each pair of affected pixels, the corresponding
point P+ve in the orientation projection image is
incremented by 1, respectively, while the point
corresponding to P-ve is decremented by 1.
[3] M.L. Koudelka, M.W. Koch, T.D. Russ, A prescreener for 3D face recognition using radial symmetry and the Hausdorff fraction.
Gradient and Zero-Crossing
Map

The shape of the face is another effective indicator of
key facial features.
[3] M.L. Koudelka, M.W. Koch, T.D. Russ, A prescreener for 3D face recognition using radial symmetry and the Hausdorff fraction.
Nose Tip Extraction




Each 3D face is horizontally sliced at multiple steps dv.
The nose tip is detected using a coarse to fine approach.
Circles centered at horizontal intervals dh on the slice.
The point which has the maximum altitude is considered
to be a potential nose tip and assigned a confidence
value equal to the altitude.
[4] A. S. Mian, M. Bennamoun and R. A. Owens, “Automatic 3D Face Detection, Normalization and Recognition.”
Face Detection

A sphere of radius r (80 mm) centered at the nose tip
is then used to crop the 3D face and its corresponding
registered 2D face.
[4] A. S. Mian, M. Bennamoun and R. A. Owens, “Automatic 3D Face Detection, Normalization and Recognition.”
Pose Correction
Reference
[1] A. S. Mian, M. Bennamoun and R. A. Owens, “Automatic 3D
Face Detection, Normalization and Recognition”, 3DPVT,
2006.
Pose Correction

Pose is corrected using the Hotelling transform.

To calculate the mean vector and covariance matrix.

The matrix of eigenvectors V of the covariance matrix C
[1] A. S. Mian, M. Bennamoun and R. A. Owens, “Automatic 3D Face Detection, Normalization and Recognition.”
Pose Correction

V is also a rotation matrix that aligns the point cloud
P on its principal axes.
[1] A. S. Mian, M. Bennamoun and R. A. Owens, “Automatic 3D Face Detection, Normalization and Recognition.”
Face Normalization
[1] A. S. Mian, M. Bennamoun and R. A. Owens, “Automatic 3D Face Detection, Normalization and Recognition.”
Face Segmentation
Reference
[1] Ajmal S. Mian, M. Bennamoun and R. Owens, "An Efficient
Multimodal 2D-3D Hybrid Approach to Automatic Face
Recognition", to appear in IEEE Transactions in Pattern
Analysis and Machine Intelligence (IEEE TPAMI), 2007.
[2] K.-C. Wong, W.-Y. Lin, Y. H. Hu, N. Boston, and X. Zhang, "
Optimal Linear Combination of Facial Regions for Improving
Identification Performance", IEEE Trans. Systems, Man, and
Cybernetics Part B: Cybernetics, Accepted, 2007.
Face Segmentation and
Recognition


Robustness to facial expressions by automatically
segmenting the face into expression sensitive and
insensitive regions.
To measure the variance in the depth of the corresponding
pixels (neutral and non-neutral expression).
[1] Ajmal S. Mian, M. Bennamoun and R. Owens, "An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition."
Face Segmentation and
Recognition

The features were automatically segmented by
detecting the inflection points around the nose tip.
[1] Ajmal S. Mian, M. Bennamoun and R. Owens, "An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition."
Results and Analysis
[1] Ajmal S. Mian, M. Bennamoun and R. Owens, "An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition."
Multiple Region Face
Recognition
[2] K.-C. Wong, W.-Y. Lin, Y. H. Hu, N. Boston, and X. Zhang,
" Optimal Linear Combination of Facial Regions for Improving Identification Performance."
Similarity Score
ROC
[2] K.-C. Wong, W.-Y. Lin, Y. H. Hu, N. Boston, and X. Zhang,
" Optimal Linear Combination of Facial Regions for Improving Identification Performance."
Results and Analysis
[2] K.-C. Wong, W.-Y. Lin, Y. H. Hu, N. Boston, and X. Zhang,
" Optimal Linear Combination of Facial Regions for Improving Identification Performance."
Recognition
Reference
[1] M. Worring and A. W. M. Smeulders, " Digital curvature
estimation ", CVGIP: Image Understanding, 58(3):366–382,
1993.
[2] N. Gelfand, N. J. Mitra, L. J. Guibas, and H. Pottmann, "Robust
global registration", In Proc. Symp. Geom. Processing, pages
197–206, 2005.
[3] Robust Curvature Estimation Through Line Integrals.
Gaussian Convolution
 (i )  tan 1 (
y (i  1)  y (i )
)
x(i  1)  x(i )
k (i )   (i )  G '
B(i, r ) 
k  k (i )
100%, k  1/ r
k
[1] M. Worring and A. W. M. Smeulders, " Digital curvature estimation ."
Results and Analysis
[1] M. Worring and A. W. M. Smeulders, " Digital curvature estimation ."
Area Integrals
[2] N. Gelfand, N. J. Mitra, L. J. Guibas, and H. Pottmann, "Robust global registration".
Results and Analysis
[2] N. Gelfand, N. J. Mitra, L. J. Guibas, and H. Pottmann, "Robust global registration".
Line Integrals
[3] Robust Curvature Estimation Through Line Integrals.
Results and Analysis
[3] Robust Curvature Estimation Through Line Integrals.