Triangle-based approach to the
detection of human face
March 2001
PATTERN RECOGNITION
Speaker Jing. AIP Lab
Outline
 Introduction
 Segmentation of potential face regions
 Face verification
 Experimental results and discussion
Introduction 1/3
Problem
Given a still or video image,
detect and localize an
unknown number of faces
Applications
–
–
–
–
–
–
Security mechanism (replace key, card,passwd)
Criminology (find out possible criminals)
Content-based image retrieval
video coding
video conferencing
Crowd(大眾) surveillance and intelligent human-computer
interfaces.
Introduction 2/3
Requirement
* achieve the task regardless of
- illumination, orientation, and camera distance
* Speedy and correct detection rate
Why difficult ?
Human face is a dynamic object
High degree of variability in appearance (面孔的多變性)
Introduction 3/3
 Drawbacks of the papers until now
– Free of background
– Cannot detect a small face ( < 50 * 50)
– Cannot detect multiple face ( >3)
– Cannot handle the defocus and noise
– Cannot conquer the partial occlusion of
mouth or wear sunglasses
– Cannot detect a face of side view
A classified algorithms
Begin the method
Overview of the system
1. Form 4-connected components
2. Find the center for each one
1
. Search any 3 center th
form an isosceles or
triangle
1. Normalize the size o
face regions
1. Calculate the weight
by mask function
Segmentation
4 step for segmenting the potential face

–
–
–
–
Convert the input image to a binary image
Find the blocks using 4-connected component
Search the triangle
Clip the satisfy triangle region
Step1: Convert the image
 RGB Color Image
– Eliminating the hue and saturation
– Gray-level  binary image
Gray-level < T are labelled as black
Gray-level > T are white
– Remove noise using opening operation
– Eliminate holes by the closing operation
Step 2:Form the blocks &
Searching triangle
 Form the blocks by using 4-connected components
algorithm
 Locate the center of each block
 Searching the triangle
– Frontal view (isosceles triangle)
– Side view (right triangle)
Step 3:
Frontal view (isosceles triangle)
 Isosceles triangle:
D(ij)=D(jk)
a
i
c
b
 Matching rule:
mouth to mouth
| b  c | 0.25 max( b, c)
Eye to mouth
| b  a | 0.25 max( b, c)
k
j
Clipping the region 2/4
x2
x1
Xi,Yi
X1=X4=Xi – 1/3 d
X2=X3=Xk + 1/3 d
Y1=Y2=Yi + 1/3 d
Y3=Y4=Yj – 1/3 d
d
Xj,Yj
Xk,Yk
Side view (right triangle) 3/4

Right triangle
b
i
2
3
j
1
c
a

k
Matching Rules: (25% derivation)
1.
2.
3.
0.4 a < | a-c | < 0.6 a
0.13 a < | a-b | < 0.19 a
0.29 a < | b-c | < 0.44 a
Clipping the region 4/4
d/6
1.2d
X1=X4=Xi-d/6
X2=X3=Xi+1.2d
Y1=Y2=Yi+d/4
Y3=Y4=Yi-d
d/4
d
Speedup of searching
 How many triangles ?
C
n
3
 If the mouth & right eye are already known,
=> the left eye should be located in the near area.
k
i
j
Face verification
 3 steps in verification
Step1: Normalization the potential facial areas
– 60 * 60 pixels
Step 2: Calculating the weight by masking function
Question 1 . How to generate the face mask ?
Question 2 . How to calculate the weight ?
Step 3:Verification by thresholding the weight
Question 1 . How to generate the face mask ?
 Read the 10 binary training masks
 Add the corresponding entries
 Binarized the added mask
Ex:
Question 2 . How to calculate the weight
 Eye and mouth are labeled as black, others as
white
– If the pixels in the P is equal to T
• Both Black: Weight + 6
• Both White : Weight + 2
– White in P and black in T
• Weight –2
– White in T and black in P
• Weight - 4
P: potential facial region
T: Training mask
Verification
 For each potential facial regions
– Threshold value is given for decision making
• Front view => 4000 < threhold < 5500
• Side view => 2300 < threhold < 2600
 Finally, eliminate the regions that
– Overlap with the chosen facial region
Result—frontal view
Original
clipping
Binary
Normalized
Isosceles triangle
Result – Side View
Original
clipping
Binary
Normalized
Isosceles triangle
Experimental results
 500 test images
– included 450 different persons
– 600 faces that are used
11 faces cannot be found correctly
98% success rate
Experiment result
 Scaling: 5*5 to 640*480
 Light condition
Experiment Result
 Distinct position
 Defocus face
Experiment Result
 Changed expressions
Experiment Result
Noise
Occlusion
Sunglasses
cartoon Chinese doll
Experiment Result
Target machine: PII 233 PC
2.5 sec
28 sec
Experiment Result
Multi-faces and video stream
Experiment Result
False cases
Too Dark
Right eye being occluded
Conclusion
 Manage different sizes, changed light
conditions, varying pose and
expression
 Cope with partial occlusion problem
 Detect a side-view face
 In the future, using this algorithm
for solving face recognition problem
My opinions
 The processing time depend on the
complexity of the image.
 Real-time requirement was unachievable.
(some images need 28 sec to process)