Lena Gorelick
Joint work with Frank Schmidt and Yuri Boykov
Rochester Institute of Technology,
Center of Imaging Science
January 2013
1
E(S)
 f(x)s(x)
f, S  B(S)
E(S)

xS
S
Pr(I | Fg)
Pr(I | Bg)
 Pr(I(x)| fg) 

f(x) ln
 Pr(I(x)| bg) 
I
2
Resulting
Target
Appearance
Fg
Probability
Distribution
Bg
Intensity
3
 Pr(Ip | fg) 

 l n



Pr(I
|
bg)
pS
p


R(S) ||
S
- T || 2
L
R(S) KL( S || T )
R(S) Bha( S , T )
Non-linear
harder to optimize
regional term
4
E(S)  R(S) B(S)
non-linear
regional term
S

 complex appearance models
 shape
5

Can be optimized with gradient descent
 First order (linear) approximation models
Ben Ayed et al. Image Processing 2008,
Foulonneau et al., PAMI 2006
Foulonneau et al., IJCV 2009

We use more accurate non-linear
approximation models based on trust region
6

General class of
non-linear regional functionals
R(S)  F( f1 , S , , fk , S )

Optimization algorithm based on trust region
framework – Fast Trust Region
7


Non-linear Regional Functionals
Overview of Trust Region Framework
 Trust region sub-problem



Lagrangian Formulation for the sub-problem
Fast Trust Region method
Results
8
Volume Constraint
R(S)  ( 1, S  V0 )
| S | 1, S
f(x) 1
2
fi , S
fi (x)for bin i
9
Bin Count Constraint
k
R(S) Σ( fi , S  Vi )
i 1
| S | 1, S
f(x)  1
fi , S
fi (x)for bin i
10
2

Histogram Constraint
Pi (S ) 
| S | 1, S
f(x)  1
fi , S
1, S
fi , S
fi (x)  1 for bin i
11

Histogram Constraint
R(S) ||
S
- T || 2
L
R(S) Σ Pi (S)  Vi 
k
2
i 1
Pi (S ) 
fi , S
1, S
12

Histogram Constraint R(S) KL(
S
|| T )
P i (S)
R(S) Σ P i (S)log
i1
Vi
k
Pi (S ) 
fi , S
1, S
13

Histogram Constraint R(S) Bha( S ,
T
)


R(S)  log Σ Pi (S) Vi 
 i 1

k
Pi (S ) 
fi , S
1, S
14

Volume Constraint is a very crude shape prior

Can be encoded using a set of shape
moments mpq
p+q is the order
15

Volume Constraint is a very crude shape prior
fpq (x,y)  x y
12 02
3
m pq (S)  f pq , S
f pq (x, y)  x y
p
q
16
m00  Volume
(m10 , m01 )  CenterOf Mass
 m 20

 m11
...
m11  PrincipalOrientation
 
m02 
Aspect Ratio
17

Shape Prior Constraint
R(S)
(m
p q k
pq
R(S)  Dist( S ,
(S) mpq (T))
T
)
2
mpq (S)  fpq , S
fpq (x,y)  x y
p
q
18
E(S)  R(S) B(S)
19




E(S)  R(S) B(S)
Gradient Descent
First Order Taylor Approximation for R(S)
First Order approximation for B(S)
(“curvature flow”)
Only robust with tiny steps
 Slow
 Sensitive to initialization
Ben Ayed et al. CVPR 2010,
Freedman et al. tPAMI 2004
http://en.wikipedia.org/wiki/File:Level_set_method.jpg
20

Speedup via energy- specific methods
 Bhattacharyya Distance
 Volume Constraint

Ben Ayed et al. CVPR 2010,
Werner, CVPR2008
Woodford, ICCV2009
In contrast:
 Fast optimization algorithm
for general high-order energies
 Based on more accurate non-linear
approximation models
21

The goal is to optimize E(S)  R(S) B(S)
d
Trust
region
Trust Region
Sub-Problem
S0
~
min E(S)  U0 (S) B(S)
||S S 0 || d
• First Order Taylor for R(S)
• Keep quadratic B(S)
22

The goal is to optimize E(S)  R(S) B(S)
d
Trust Region
Sub-Problem
S0
~
min E(S)  U0 (S) B(S)
||S S 0 || d
23

Constrained optimization
~
minimize E
(S)  U 0 (S)  B(S)
s.t. || S  S 0 || d


Unconstrained Lagrangian Formulation
~
minimize Lλ (S)  E
λB(S)
|| S Sλ0 |||| S  S0 ||
U(S)
0 (S)
Can be approximated
with unary terms
Boykov et al. ECCV 2006
Can be optimized globally using graph-cut
24
• Newton step
• “Gradient Descent”
• Exact Line Search (ECCV12)
25

Repeat
 Solve Trust Region Sub-problem
around S0 with radius d
 Update solution S0
 Update Trust Region Size d

Until Convergence
28

General Trust Region
 Control of
the distance constraint d
Lagrangian Formulation
 Control of
the Lagrange multiplier λ
λ

1
λ
d
29




Simulated Gradient Descent
Exact Line-Search (ECCV 12)
Newton step
Fast Trust Region (CVPR 13)
31
R (S )
0
Log-Lik. + length
Vmin
Initializations
Vmax
|S|
Log-Lik. + length + volume
Fast Trust Region
35
Fast Trust Region
Log-Likelihoods
No Shape Prior
Second order geometric moments computed
for the user provided initial ellipse
36
Init
Fast
Trust Region
Exact
Line Search
“Gradient Descent”
Appearance model is obtained from the ground truth
37
“
“
Fast Trust Region
Exact Line Search
“Gradient Descent”
Appearance model is obtained from the ground truth
38
Log-Lik. + length + Shape Prior
Fast Trust Region
Second order Tchebyshev moments computed
for the user scribble
39
Ground Truth
BHA. + length
Fast Trust Region
Appearance model is obtained from the ground truth
40


Multi-label Fast Trust Region
Binary shape prior:
 affine-invariant Legendre/Tchebyshev moments
 Learning class specific distribution of moments

Multi-label shape prior
 moments of multi-label atlas map

Experimental evaluation and comparison
between level-sets and FTR.
41
42