Face Processing System
Presented by: Harvest Jang
Group meeting Fall 2002
Outline
Introduction
System Architecture
Pre-processing
Face detection
Face tracking
Problems
Future work
Introduction
An automatic system is needed to provide more
human-computer interactive services to improve the
life style
Security System
Interactive application/game
Robot visualization
This is a challenging task for integrating different
techniques
face detection
face tracking
face recognition
System Architecture
Three main components
Pre-processing
Face detection
Face tracking
Fig 1: System architecture
Pre-processing
Transform to YCrCb color space
2. Use ellipse color model to locate the fresh
color
1.
Fig 2: 2D projection in the CrCb
subspace (gray dots represent skin
color samples and black dots
represent non-skin tone color)
Pre-processing
Perform morphological
operation to reduce
noise
4. Skin segmentation to
find face candidates
3.
Fig 3: Pre-processing step (a) binary skin mask,
(b) original images (c) binary skin mask after
morphological operation and (d) Face candidates
Comparison on three face detection
approach
Motivation for the comparison
Using different infrastructure systems for detecting
faces in a large image
Only shows one result pair of detection rate and
false alarm rate
Using experimental result to compare
Sparse Network of Winnows (SNoW)
Support Vector Machines (SVM)
Neural Network
Comparison on three face detection
approach
Sparse Network of Winnows (SNoW)
Primitive features architecture
256  ( y  w  x)  I ( x, y )
Winnow update rule
learner’s
prediction
correct
response
update action
update
name
1
0
wi: wi/ if xi1
wi unchanged if xi0
demotion step
0
1
wi: wi if xi1
wi unchanged if xi0
 >1 fixed parameter
promotion step
Table 1: Winnow update rule
Comparison on three face detection
approach
Support Vector Machines (SVM)
Find a hyperplane that leaves the maximum
margin between two classes which will have
the smallest generalization error
The margin is defined as the sum of the
distances of the hyperplane from the closest
point of the two classes
Neural Network
Back-propagation method
x1
x2
w1
w2
xn
wn
x0 = 1
w0

i 0
n

1
if
wi xi  0


o  x 1 , x 2 , x n   
i 0


- 1 otherwise

n
w
Fig 4: SVM
i
xi
 


 
1 if w  x  0
Vector notation : o x   sgn x, w   

- 1 otherwise
Fig 5: Neural Network
Comparison on three face detection
approaches
CBCL face database from MIT
Training set (2429 face pattern, 4548 non-face
pattern with 19x19 pixel)
Testing set (472 face pattern, 23573 non-face
pattern with 19x19 pixel)
To better represent the detectability of each model,
ROC curve is used to replace single point of criterion
response
Comparison on three face detection
approaches
Similar signal for
SNoW
Neural Network
SVM model with linear
kernel
SVM model with
polynomial kernel
much stronger signal
the classification
between face and nonface patterns is much
better
Fig: 6: ROC curves of different face detection method
Comparison on three face detection
approaches
In our system, success to detect face is more
important than processing more windows
Face tracking method to help for tracking the
detected face
SVM model with polynomial kernel of second degree
is chosen.
(in sec)
NN
SNoW SVM with linear
SVM with polynomial
Mean
35.57
63.60
65.22
61.13
S.T.D.
0.029
1.378
0.306
0.410
Table 2: The average processing time of each model
for testing 24045 images ten times
Face detection
To detect different size of faces, the region is resized
to various scales
A 19x19 search window is searching around the resized regions
Histogram equalization is performed to the search
window
After the first presence of a face at the larger scale,
face recognition using SVM classifier will be
performed to retrieve the information of that person.
Face detection - Example
Transform to
various scale
Histogram
equalization
Apply a 19x19
search window
Face tracking
Why using face tracking
Face detection method has missing rate and false
alarm rate
The missing rate and false alarm rate for losing an
object will be reduced
The performance speed is much better than face
detection method
Conditional Density Propagation –
Condensation algorithm
Sampling-based tracking method
Face tracking
The posterior probability density at timestep t
p ( xt | Z t )
using a set of N random samples, denoted as
{S t( n ) , n  1,, N } with weights  t(n )
There are three phases to compute the
density iteratively at each time step t for the
set of random samples to track the
movement
Face tracking
Selection phase:
(n )
S
The element with high weights in the set t
has a higher probability to be chosen to the
predictive steps
Predictive phase:
(n)
The sample set S t 1 for the new time-step is
generated by independent Brownian motion
(n)
The weights  t 1 are approximately from the
effective prior density p( xt 1 | Z t ) for time-step t+1
p( xt 1 | Z t )   p ( xt 1 | xt ,  t ) p ( xt | Z t )dxt
Face tracking
Measurement (Update) phase
Given in terms of likelihood p( zt 1 | xt 1 )
The model is expressed as a histogram for face
color HUE (HSV color space)
Calculate the measurement
p( xt 1 | Z t 1 )  ct 1 p( zt 1 | xt 1 ) p( xt 1 | Z t )
where ct 1 is a normalization factor
Face tracking
Fig 7: One time step in the Condensation algorithm
Face tracking
The color input image is being masked
by the binary skin mask
reduce the localization period
converge less sensitive from the
background noise
converge to the face boundary much faster.
Problems
The face detection rate is still low
The condensation algorithm doesn’t
provide verification for the tracking
object
Two adjacent object with similar color will
cause problem
Future work
Improve the accuracy of face detection
Implement the face tracking verification
step using face recognition
Thank you!