Facial Features Extraction
Amit Pillay
Ravi Mattani
What Are We Doing !
 Finding Features on a Face
 Eyes
 Mouth
 Nose
Why Facial Feature Extraction?




Biometrics
Facial recognition system
Video Surveillance
Human Computer Interface
Difficulties !




Face Variation
Physical characteristic vary
Non-uniform lighting
Face position
Previous Work.

Many Face Extraction Methods

Main Trend : Combine image information and knowledge of face

Ian-Gang Wang, Eric sung in their article have proposed a morphological
procedure to analyze the shape of segmented face region. Several rules
have been formulated for the task of locating the contour of the face.

Terrillon et al., 1998 mentions the problem of how other body parts such
as neck may lead to face localization error

Haalick and Shapiro, 1993 demonstrate how morphological operations
can simplify the image data while preserving their essential shape
characteristics and can eliminate irrelevances.
Our Process
Input
image
skin color
segmentation
Morphological
image-processing
Skeletonization
Line
segmentation
and contour
detection
Facial feature
extraction using
facial geometry.
Output
image
Skin Segmentation
Depends on color space
Used the finding by Yang & Waibel(1995,1996)
Normalized r-g color plane.
Took seed pixel
Classified the pixels based on whether the pixel
lies within the threshold
 Same process carried out for the R and G plane





Skin color segmentation
Morphological Image Processing
 Dilation
 Fills the holes
 Erosion
 Restores the shape of the face
Morphological Image Processing
Skeletonization
 Reduces binary image objects to a set of
thin strokes.
 Retains important information about the
shape of the original object
Skeletonization
Contour Tracing
 Certain vertices of these skeleton lines called
fitting points can fit the contour of the human
face.
 Certain rules are then applied to deduce these
fitting points by analyzing the skeleton lines.
Contour Tracing
 Rule 1 - The contour fitting points should be the vertices of the
roughly horizontal skeleton line segments that are long enough.
 Rule 2 - The left vertex will be selected as candidate for contour
fitting if most of the horizontal line segments are positioned at
the left of the symmetry axis and vice versa
 Rule 3 - The contour points should be above a vertical position
that is set at 3/4 of the height from the top of the symmetry axis
 Rule 4 - The point set satisfying the above will be doubled using
symmetry axis
Contour Tracing
ROI
Feature extraction within the ROI
 Edge Detection using Sobel Operator
 Vertical position by horizontal integral
projection
 Lip line maximizes the projection
 Bounded by a rectangular box
 Same process is repeated for nose and
eyes regions within the fixed vertical
positions
Results
Conclusion
 No. of images experimented with = 30
 No. of images in which features are
correctly identified = 27
 Percentage correctly identified = 90
 Average time taken to get the output in
MATLAB = 15-20 secs
Future Work
 More robust and dynamic
 Extended for profile views of image
 More efficient code for faster execution
(applicable especially for MATLAB !!!)
References
 Frontal-view face detection and facial feature
extraction using color and morphological
operations by Jian-Gang Wang, Eric Sung
 A Model-Based Gaze Tracking System by Rainer
Stiefelhagen, Jie Yang, Alex Waibel
 Digital Image Processing Using MATLAB by
Gonzalez, Woods &Eddins,Prentice
 Images taken from www.faceresearch.org
 Prof. Gaborski’s lecture slides
 www.wikipedia.com
Questions???