Facial Component Detection For Efficient
Facial Characteristic Point Extraction.
Part I
Presenter: 馮氏芳翠 <Lisa>
Professor: Dr. Shih-Chung Chen
Introduction To Facial Recognition
System Applications
Part II
Contents – Part I
1 Basic Concepts
2 Facial Component Detection
1 Facial Characteristic Point Extraction
3
1 Verification Experiment and Result
4
3
What Is an Image ?
A common method is to define an image I as a rectangular matrix (called
image matrix )
I = [ f (x, y) ]
Image rows (defining the row counter or row index x).
Image columns (column counter or column index y).
One row value together with a column value defines a small image area
called pixel (picture element, image element), which is assigned a value
representing the brightness of the pixel.
* Illustration *
4
Image Processing Levels
Perform the cognitive functions. Normally
associated with vision
High Level
Extracted Attributes, Segmentation,
Description.
* Exp: Cup Rim (Script) *
Mid Level
Reduce noise, contrast
enhancement, sharpening.
Low Level
6
Examples of Low Level
Low Contrast
Original
Noise
Reduced Noise
Reduced
ReducedNoise
Noise
High Contrast
Sharpening images
7
Facial Recognition System (FRS)
A computer application for automatically identifying or
verifying a person from a digital image or a video source.
One of the ways to do this is by comparing selected
facial features from the image and a facial database.
It is typically used in security & surveillance systems
and can be compared to other biometrics such as
fingerprint, palm scan or iris recognition systems.
8
Main Processes in FRS
Identify and locate human faces in an
image regardless of their
 Position
 Scale
 In plane rotation
 Orientation
 Illumination
Detect the presence
and location of
features such as eyes,
such as eyes, nose,
nostrils, eyebrow,
mouth, lips, ears, etc
Identity individual
9
Face Detection: A Solved Problem ?
Fig.1 Example of rotation
invariant
face
detection
Fig.1
Example
of rotation
face detection
Fig.2 Detection result of faces
with various poses
Fig.3 Detection result
Fig.3ofDetection result of
occluded faces occluded face
Fig.4 Detection result of faces
with different face sizes
Fig.5 Detection result of face
when changes expression
10
Why face recognition is hard ?
Many styles of
Madonna’s Face
11
What is the facial components ?
Title
12
Facial Expression
Worried
Sick
Sad
Excited
Angry
Difference?
Happy
Embarrassed
Scared
Tired
Disgusted
13
Detection Processes
FRD
ERD
EbRD
MRD
FE
Fig. 1. Block diagram for facial component detection
14
Facial Region Detection
Y’CbCr Color Space
Set threshold for Cb[77,127] & Cr[133,173]
Combine 2 images after setting threshold
Fig. 2. The Input image
Fig. 3. The detected facial region.
(Skin Color Region)
When the hair has the bright color like the skin area?
Improvement
15
“Y’CbCr” Definition
Y’CbCr is a family of color spaces used as a part of the color image
pipeline in video and digital photography systems.
Y’ is the luma component
Cb is the blue-difference chroma component.
Cr is the red-difference chroma component.
A color image
Y Component
Cb Component
Cr Component
http://en.wikipedia.org/wiki/Color_space
http://en.wikipedia.org/wiki/Chrominance
http://en.wikipedia.org/wiki/Luminance_(video)
http://en.wikipedia.org/wiki/Luminance
16
Facial Region Detection
- without hair effect Y’CbCr Color Space
Fig. 5. Facial region
detected with hair effect.
Fig. 4. Input Image
Eq1: Represents the luminance
variation at coordinates (x,y)
Thresholding
Fig. 6. Hair region detected
by luminance variation.
Fig. 7. Facial region detected
17
without hair effect
Eye Region Detection Using Template
Template
Facial region
image
Inverse & extract
Fig. 8. Extracted eye region
Fig. 9. Real eye region
Boundary
Rectangular
Apply the fact that the eyes are
located symmetrically in the
upper facial region and under
the eyebrows
Fig.10. Detected eye regions
using template matching
Sometimes the eyes shape is not accurate?
Improvement
18
Eye Region Detection Using Weighted Templates
Custom-Masks
Get more accurate eye shape by assigning a new search region.
Remark:
Black pixels are assigned with -1
Dark gray assigned with 1
Light gray assigned with 2
White assigned with 0
Fig. 10. Weighted templates for left, right, up, and down sides
The eye region is
extracted more
accurately (left & right)
Fig.11. Detected eye regions
using template matching
Fig.12. Detected eye regions after
using the weighted template
19
Eyebrow Region Detection
Fig.12. Detected eye regions after
using the weighted template
Luminance histograms
Estimated
Fig.13. Eyebrow search region
Thresholding using a luminance histogram
Fig.14. Processes of modifying a histogram and
determining a threshold value
Width = 2.5 eye Width
Height = 2.5 eye Height
Fig.15. Detected eyebrow region
20
Mouth Region Detection
A mouth search region is specified by the positions of the detected eyes
and the statistical data regarding the geometric information of a face.
Fig.16. Geometric structure of eyes and mouth
Eq2: Coordinates of mouth search region
21
Mouth Region Detection
The mouth region also has a large luminance variance
Eq1: Represents the luminance variation at coordinates (x,y)
Fig.17. Input image for mouth region detection and a detected mouth region
22
FCP Extraction
Appoint 34 points for FCPs in the facial region
10 points
Eyes Region
16 points
Edge Detection
Eyebrows Region
Mouth Region
8 points
Fig.18. Appointed FCPs
23
Verification Experiment & Result
 Condition: The input image must be a bust shot (portrait), including a
front view of the face without glasses, and the background has to be
simple.
 The experiment is carried out with 150 images. It extracts valid FCPs
in 122 / 150 images (81.333 %)
Fig.19. The FCPs extracted
by the proposed algorithm
24
Verification Experiment & Result
1. The first case was due to background effects: The background with skin-color is
detected as the facial region.
2. The second case was because of long hair: Long hair covering the eyes and
eyebrows causes the wrong eye region detection and makes it impossible to detect the
remaining facial components.
3. The third case was affected by viewpoint (poses): The input images disagreed with
the geometric information of a face, the facial components cannot be normally detected.
4. The fourth case regards a problem with skin-color range: The skin-color of several
non-Caucasian people was out of the assumed Caucasian skin-color range and the
facial region could not be detected.
The front three cases were solved by cautious images acquisitions
The last case solved by adjusting a skin-color range to a race.
25
Conclusion-I
 The research proposed the improved method to detect the
facial components that used for extracting FCP-an important
information for facial expression and recognition.
 Future Work: Extract facial components using LabVIEW
& Vision Assistant
 Challenges ???
26
Why face detection is difficult ?
27
Facial Recognition Application
FaceCheck_Server
1
FaceCheck_Verify
6
2
FaceSnap_Recorder
Around
the world
5
FaceSnap_Fotomodul
FaceSnap_IsoShot
3
4
FaceSnap_FotoShot_TwainShot
28
FaceCheck_Server
Automatic Recognition and Comparison of Facial Images, and
Notification when a Person of Interest is Identified
29
FaceCheck_Server
FaceCheck Server receives facial images from FaceSnap imaging units
within an IP network. For optimal performance, real-time facial recording
queues all images for subsequent identification.
Watch lists
(Portrait Samples)
First In –First Out
FaceSnap imaging units
30
FaceCheck_Verify
The Reliable Facial Recognition System for High Security Access
Control and Identity Checks Based on ID Photos
The images of an enrolled user may be retrieved from a database or from a chip card. The verification
process is fully automatic, optionally allowing visual monitoring by an operator and automatic image
recording on all verification attempts (providing a data file for future reference)
Live Verification
of ID Photos
31
FaceSnap Fotomodul
A Time and Cost Saving Automatic Photo Cropping Tool
Recognizes and records facial images
 Automatically crops the image according to a pre
selected portrait format.
 Automatic brightness, contrast and color
corrections for best image impression.
 Automatic background removal
.The images can be acquired either by a TWAIN
interface or directly from a file in jpg format.
32
FaceSnap_FotoShot/TwainShot
Reliable Face Detection System for High Quality Portrait Capture
• The image is taken automatically when the facial recognition software “sees” a person
who poses correctly for an ID photo.
• Locates the face and then crops automatically, resizes and color corrects the image
for maximum user convenience.
33
FaceSnap_ IsoShot
Facial Detection and Quality Assessment Software
for ISO 19794-5 Compliant Portraits
Uses facial detection technology to
automatically generate standardized portraits
Automatically set facial
landmarks to adjust
image geometry
Captures live images through remote camera control
• Instantly enhances and resizes images
• Configures camera and facial image cropping settings.
35
FaceSnap_ Recorder
An Indispensable Facial Recognition Tool for Law Enforcement,
Post-Event Analysis, Business Security.
 Automatically recognize and record facial images from different viewing angles. Users
are ensured of receiving the visual information they need quickly, efficiently and reliably.
 For Identity checks, video observation, access monitoring.
36
Conclusion-II
 Facial recognition systems is very useful for maintaining security and safety of
visitors and employees in many organizations.
 Facial recognition technology is an ideal solution for high-traffic public areas
where access control and law enforcement are of paramount importance.
Airports and railway Stations
Cash machines
Casinos
Financial institutions
Government Offices
Public transportation facilities
Stadiums
Businesses of all types
37
References
[1]. R. Chellappa, C. H. Wilson, and S. Sirohey: Human and Machine Recognition of Faces: A
Survey. Proc. of the IEEE, vol. 83, no. 5 (1995) 705-740
[2]. Y. H. Han and S. H. Hong: Recognizing Human Facial Expressions and Gesture from Image
Sequence. Journal of Biomedical Engineering Research, vol. 20, no. 4 (1999) 419-425
[3]. R. Brunelli and T. Poggio: Face Recognition: Feature versus Templates. IEEE Trans. PAMI, vol.
15, no. 10 (1993)
[4]. G. Chow and X. Li: Towards a System for Automatic Facial Feature Detection. Pattern
Recognition, vol. 26, no. 12 (1993) 1739-1775
[5]. V. Govindaraju, S. N. Srihari, and D. B. Sher: A Computational Model for Face Location. Proc.
3rd Int. Conf. Computer Vision (1990) 718-721
[6]. R. C. Gonzalez and R. E. Woods: Digital Image Processing. Addison Wesley New York (1992)
[7]. J. C. Russ: The Image Processing Handbook, 3rd Ed.. IEEE Press (1999)
[8]. D. Chai and K. N. Ngan: Face Segmentation Using Skin-color Map in Videophone
Application. IEEE Trans. Circuits and Systems for Video Technology (1999) 551-564
[9]. H-S. Yoon, M. Wang, and B-W. Min: Skew Correction of Face Image Using Eye Components
Extraction. The Journal of the Korea Institute of Telematics and Electronics, vol. 33- B, no. 12 (1996)
71-83
38
References
http://en.wikipedia.org/wiki/YCbCr
http://en.wikipedia.org/wiki/Color_space
http://en.wikipedia.org/wiki/Luminance_(video)
http://en.wikipedia.org/wiki/Chrominance
http://en.wikipedia.org/wiki/Luminance
http://www.azooptics.com/Details.asp?ArticleID=154
http://www.crossmatch.com/FaceCheckVerify.html
39