Face Recognition Using
aces
Eigenf
Mathew A.Turkand AlexP.Pentland
Presenter: Polina Golland
Pr
obem
l Def
inition
• Recognize f
ac
es vs.non-f
aces
aces ofa particular person vs.
• Recognize f
f
aces ofother people
• Do it f
ast
02/7/5
0 0
6.
881
2
Basic Idea
• PCAon f
ac
e images
œ Face images lie in a low dimensional space
œ I
mages ofthe same person are close to eachother
œ I
f
erent people are f
arther away
mages ofdif
02/7/5
0 0
6.
881
3
Recognition
• Given a new image, proj
ec
t onto f
ac
e space
œ I
fthe residual is too high, it‘s not a f
ace
œ I
tion is close to one ofthe —prototypes“,
fthe proj
ec
assign it to that class
œ Otherwise, it‘s a new f
e
ac
02/7/5
0 0
6.
881
4
Il
l
ustr
ation ofFace Space
4
u2
Ω2
3
Ω3
Face Space
Ω1
u1
1
2
Figure by MIT OCW.
02/7/5
0 0
6.
881
5
Tr
aining
• I
nput: M input images in a vector f
orm
• Mean f
ace
= M-1
ƒ
i
• Zero-mean data i = i • Zero-mean data matrixA = [ 1
• Covariance matrixC = M-1
02/7/5
0 0
i
6.
881
ƒ
i
2…
T
i
M]
= AAT
6
Tr
aining cont‘d
• Eigenf
aces uk are the eigenvectors ofC
• Keepa small number (M‘) ofeigenf
aces
œ I
n the experiments, M = 16, M‘ = 7
• Resultingmodel:
œ Mean f
ac
e
ac
œ M‘ eigenf
es uk (k = 1,…,M‘)
02/7/5
0 0
6.
881
7
Detection
• Given a new image , compute its weights
T
)
k = uk ( • The f
ace is represented bythe weight vector
= [ 1 2 … M‘]T
• I
f|| -
• I
f||
02/7/5
0 0
-
œ
n
ƒ
kuk
|| <
,
|| <
ε,
this is a f
ace image.
this is a f
ace f
rom class n.
6.
881
8
4
3
u2
Ω2
|| Γ − Ψ − Σ ωk uk ||
Ω3
Face Space
Ω1
u1
1
2
|| Ω − Ω3 ||
Figure by MIT OCW.
C
Exampes
l ofpr
oj
ections
Image removed due to copyright considerations. Please see Figure 3 in:
M.E. Leventon, E.L. Grimson, and O. Faugeras, "Statistical Shape Influence in Geodesic
Active Contours," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 316-323, 2000.
02/7/5
0 0
6.
881
10
Notes
• Use Euclidean distance
œ not Mahalanobis distance
• Not clear how the classes
kare
constructed
œ here used j
ust one image per person
• Free parameters:
œ number ofeigenf
aces M‘ and thresholds
œ in the experiments, M‘ = 7, and the thresholds varied
• Eigenf
ac
es looklike f
aces
02/7/5
0 0
6.
881
11
Eigenf
aces
Image removed due to copyright considerations. Please see Figure 2 in:
M.E. Leventon, E.L. Grimson, and O. Faugeras, "Statistical Shape Influence in Geodesic
Active Contours," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 316-323, 2000.
02/7/5
0 0
6.
881
12
—Faceness“ Map
• For eachwindow location, evaluate its
rom the f
distance f
ace space
• Slow, f
aster algorithms exist
Image removed due to copyright considerations. Please see Figure 3 in:
M.E. Leventon, E.L. Grimson, and O. Faugeras, "Statistical Shape Influence in Geodesic
Active Contours," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 316-323, 2000.
02/7/5
0 0
6.
881
13
Exper
imentalResul
ts
• I
mage Database
œ Controlled experiment
• Real-time f
ac
e recognition
œ Unconstrained recognition, closer to the real-world
02/7/5
0 0
6.
881
14
Image Database Resul
ts
• I
mage Database
œ 16 people
œ 3 illuminations, 3 angles, 3 head sizes
• Use one image per person to train
œ M = 16, M‘ = 7
œ all at the same illumination, angle, size.
• Test on all other images, f
orcingrecognition
œ 96% across illuminations
œ 85% across orientations
œ 64% across sizes
02/7/5
0 0
6.
881
15
Image Database Resul
ts cont‘d
• Varythe threshold (ROC, precision/
recall)
œ 100% accuracy, checkf
or —unknown“ labels
» 19% across illuminations
» 39% across orientations
» 60% across sizes
œ 20% —unknown“, checkthe accuracy
» 100% across illuminations
» 94% across orientations
» 74% across sizes
02/7/5
0 0
6.
881
16
Real
-time f
ace r
ecognition
• Motion detection, plus head localization: small
blobon topofa bigger blob
• Run the recognizer on the head window
• 2-3 times/
second (longtime ago, so might
ast on a modern computer)
actuallybe reallyf
• No accuracyp
/recision reported
02/7/5
0 0
6.
881
17
Notes on Peror
f mance
• More sensitive to head size than other f
actors
œ Normalization, windowing
œ Related to the —f
eness“ map
ac
• Should be sensitive to orientation, as parts of
the f
ace get occluded
œ Maybe the proj
ec
tion is helpingit
• What happens when we varyall f
ac
tors at
once?
02/7/5
0 0
6.
881
18
Summar
y
• Faces have a lot ofstructure, let‘s tryto
capture it automatically.
• Prior workf
ocused on craf
ted templates
œ This paper learns the structure throughPCA
• Strongassumptions on the structure
œ Face images can be approximated bya linear low
dimensional space
r.
œ Theycluster in that space w.
t.the person
• Fairlynaïve classif
ication
02/7/5
0 0
6.
881
19
Since Then
• More explicit models ofthe obec
j t shape
œ Active Shape/
Appearance Models
• Non-linear subspace/
manif
old modeling
œ
œ
œ
œ
Mixture ofGaussians
Kernel PCA
LocallyLinear Embedding, I
somaps
non-negative decomposition, etc.
• Classif
ic
ation
œ Fisher f
ac
es (Fisher Linear Discriminant)
ication methods
œ Other classif
02/7/5
0 0
6.
881
20