Kuo Hao (David) Chen
Electrical and Computer Engineering
Introduction
Newton’s Method
Supervised Descend Method (SDM)
Evaluations of SDM
Questions
Many computer vision problems are solved
through a nonlinear optimization method
Gradient descent for dimensionality reduction
Gauss-Newton for image alignment
The Newton’s method is the major approach for
nonlinear optimization for smooth functions
Newton’s Method
A smooth function f(x) can be approximated by
a quadratic function
𝑓 𝑥0 + ∆𝑥 ≈ 𝑓 𝑥0 + 𝑱𝑓 (𝑥0
)𝑇 ∆𝑥
1 𝑇
+ ∆𝑥 𝑯(𝑥0 )∆𝑥
2
The minimum can be found by solving a system
of linear equations if the Hessian is positive
definite
A sequence of updates
𝑥𝑘+1 = 𝑥𝑘 − 𝑯−1 (𝑥𝑘 )𝑱𝑓 (𝑥𝑘 )
Pros
The convergence rate is quadratic
Guarantees to converge if the initial estimate is
sufficiently close to the minimum
Cons
Requires the function to be twice differentiable
The dimension of the Hessian matrix might be large
and not positive definite
SDM learns from
training data a set
of generic descent
directions {Rk}
X. Xiong and F. De la Torre. Supervised Descent Method and its Applications to Face
Alignment. In CVPR, 2013.
Minimize the below function over Δx
𝑓 𝑥0 + ∆𝑥 = 𝒉 𝒅 𝑥0 + ∆𝑥
− ∅∗
2
2
d(x) = p landmarks in the image
h = a non-linear feature extraction function
(e.g. SIFT)
Ø * = h(d(x*))
X. Xiong and F. De la Torre.
Supervised Descent Method and its
Applications to Face Alignment. In
CVPR, 2013.
Similar to Newton’s method:
∆𝑥1 = − 𝑯−1 𝑱𝑓 = −2𝑯−1 𝑱𝑇ℎ (∅0 − ∅∗ )
In SDM form:
∆𝑥1 = 𝑹0 ∅0 + 𝒃0
𝑥𝑘 = 𝑥𝑘−1 + 𝑹𝑘−1 ∅𝑘−1 + 𝒃𝑘−1
SDM learns a sequence of {Rk} and {bk}
The succession of xk converges to x*
X. Xiong and F. De la Torre. Supervised Descent Method and its Applications to Face
Alignment. In CVPR, 2013.
Rk and bk are obtained by minimizing the
expected loss between the predicted and the
optimal landmark displacement under many
possible initializations
𝑝(𝑥𝑘𝑖 )
arg min
𝑅𝑘 ,𝑏𝑘
∆𝑥
𝑘𝑖
− 𝑅𝑘 ∅𝑖𝑘
− 𝑏𝑘
2
𝑑𝑥𝑘𝑖
𝑑𝑖
Approximate the integration with Monte Carlo
sampling
∆𝑥∗𝑘𝑖
arg min
𝑅𝑘 ,𝑏𝑘
−
𝑅𝑘 ∅𝑖𝑘
− 𝑏𝑘
2
𝑑 𝑖 𝑥𝑘𝑖
X. Xiong and F. De la Torre. Supervised Descent Method and its Applications to Face
Alignment. In CVPR, 2013.
SDM on analytic scalar functions
The NLS problem is
min 𝑓 𝑥 = (ℎ 𝑥 − 𝑦 ∗ )2
𝑥
Measure of Accuracy
𝑥𝑘 − 𝑥 ∗
𝑥∗
SDM converges
faster in each
iteration
SDM is more
robust against
bad initializations
and ill-conditions
SDM has less
accurate
estimation
X. Xiong and F. De la Torre. Supervised Descent Method and its Applications to Face
Alignment. In CVPR, 2013.
LFPW dataset, 29
landmarks
LFW-A&C dataset, 66
landmarks
RU-FACS dataset
29 video sequences (ave. 6300 frames/sequence)
66 landmarks
The ground truth is given by a person-specific
AAMs
Person-specific AAM is unreliable when there is
an occlusion
X. Xiong and F. De la Torre. Supervised Descent Method and its Applications to Face
Alignment. In CVPR, 2013.
Youtube Celebrities dataset
SDM is reliable under large pose, occlusion, and
illumination changes
https://www.youtube.com/user/xiong828/vide
os
X. Xiong and F. De la Torre. Supervised Descent Method and its Applications to Face
Alignment. In CVPR, 2013.
1. Please list some nonlinear optimization
methods.
2. What are the advantages of Newton’s method?
3. What are the drawbacks of Newton’s method?
4. What are two things that SDM learns during
the face alignment training?
5. Under what condition is the person-specific
AAM unreliable?