Lena Gorelick
joint work with
Y. Boykov
O. Veksler
I. Ben Ayed
A. Delong
E (x) 

p
fp  xp 
1
p , q Ν
x  0 ,1

[ x p  xq ]
f
Potts Model
2
E (x) 
u
p
p
xp 
v
pq
x p xq
x  0 ,1

p , q
v pq  0
 v pq
Potts Model
Submodular Energy
global optimum with graphcut (Boros & Hammer, 2002)
3
E (x) 
u
p
p
v
xp 
pq
x p x q  const.
p , q
 pq ,
v pq  0
Middlebury
Non-Submodular Energy
NP-hard
Image credit: Carlos Hernandes
4
General energy - NP-hard
 Approximate methods:
 Global Linearization:
QPBO, TRWS, SRMP (Kolmogorov et al. 2006, 2014)
 Local Linearization:
parallel ICM, IPFP (Leordeanu, 2009)
 Message Passing: BP (Pearl 1989)
5

QPBO, TRWS, SRMP (Kolmogorov et al. 2006, 2014)
E (x )
Linearize introducing
large number of
variables and constraints
~
min E ( y )
s .t . y C
Solve relaxed LP
or its dual
relaxed y
*
Rounding
integer
x
*
 Integrality Gap
6

~
parallel ICM (Leordeanu, 2009)
Et(x)
 large steps  weak min

IPFP (Leordeanu, 2009)
 controls step size by relaxation
E (x )
x t 1
xt
  Integrality Gap
{0 ,1}
N
x
Bounded domain of
discrete configurations
7

Local
Submodular
Approximation model

Non-linear

Two ways to control step
size
~
Et(x)
E (x )
xt
x t 1
{0 ,1}
N
x
Bounded domain of
discrete configurations
8

Trust Region
 Local submodular approximation
LSA-TR

Auxiliary Functions = Surrogate Functions =
Upper Bounds = Majorize-Minimize
 Local submodular upper bound
LSA-AUX

Never leave the discrete domain
9

Trust Region:
 Discrete High Order Energies
Gorelick et al. 2012,2013
 Relaxed Quadratic Binary Energies
 Levenberg Marquardt

Olsson et al. 2008
Hartley & Zisserman 2004
Auxiliary Functions=Surrogate Functions
=Upper Bounds = Majorize-Minimize
 Discrete High Order Energies
Narasimhan & Bilmes 2005
Rother et al. 2006
Ben Ayed et al. 2013
10
E (x ) 
u
p
xp 
p
v
E (x)
pq
x p xq
xt
p , q
x
+
E (x)  E
sub
(x)  E
sup
(x)
11
E (x)
xt
x
E (x)  E
sub
(x)  E
sup
(x)
12
E (x)
E (x)  E
sub
(x)  E
sup
(x)
~
E (x)
t
xt
x
Approximate E (x ) around x t
~
sub
E t (x)  E ( x ) 
13
E (x)
E (x)  E
sub
(x)  E
sup
~
E (x)
t
xt
(x)
x
Approximate E (x ) around x t
Linear
Approximation
~
sub
approx
E t (x)  E ( x )  E t
(x)
Submodular function
LSA
14
E
sup
( x )   v pq x p x q ,
pq  
v pq  0
15
v pq x p x q
16
  xy
 0
17
  xy
 0
0,1
0,0
1

y
0
1,1
1,0
1
x
18
  xy
 0
1

y
0
1,0
1
x
19
0  x    y  const

Linear (Unary)
approximation
0,0
1
y
0
1,1
1,0
1
x
20
u  x  v  y  const

1
y
0
1
x
21
E (x)  E
sub
(x)  E
sup
(x)
E (x )
xt
x
22
~
sub
approx
E t (x)  E ( x )  E t
(x)
E (x )
~
Et(x)
x t 1
xt
Newton Step
x
23
~
sub
approx
E t (x)  E ( x )  E t
(x)
E (x )
~
s.t.
|| x  x t ||  d t
Et(x)
xt
Trust Region
x
Trust Region
Sub-Problem
Constrained
Submodular
NP-hard!
Optimization
24
L t (x)  E
sub
(x)  E
approx
t
  t || x  x t ||
Submodular
(x) 
Gorelick et al. 2013
Unary Terms
Boykov et al. 2006
t
fixed in each iteration
inversely related to trust region size
adjusted based on quality of approximation
25
26


Binary De-convolution
All pairwise terms supermodular
?
Original Img
Convolved
Convolved+Noise
27
QPBOI
LBP
Noise:
N(0,0.05)
QPBO
(0.1 sec.)
TRWS
TRWS:
5000 iter.
E=65.07
QPBO-I
(0.2 sec.)
E=66.44
SRMP
SRMP:
5000 iter.
E=39.06
LBP
5000 iter.
E=40.15
LSA-AUX
(0.04 sec)
E=34.70
IPFP
(0.4 sec.)
E=32.90
FTR-L
LSA-TR
(0.3 sec.)
E=21.13
28
Image
QPBO
QPBO-I
E= -77.08
LBP
E= -84.54
IPFP
E= 163.25
Repulsion = Reward different
across high
contrast LSA-TR
edges
TRWS labels
SRMP
LSA-AUX
E= -67.21
Potts, v<0
(submodular)
E= -101.61
SRMP
E= -120.03
with edge repulsion, v>0
(non-submodular)
E= -175.05
29

dtf-chinesechar database
Kappes et al., 2013
Input Img Ground Truth
LSA-TR
30
31

Efficient Squared Curvature model –
(Nieuwenhuis et al. 2014, poster on Friday)
Potts Model
Elastica
Heber et al. 2012
90-degree curvature
El-Zehiry&Grady, 2010
Our curvature
Using LSA-TR 32

Two novel discrete optimization methods
 Simple, efficient, state-of-art results
 The code is available online -
http://vision.csd.uwo.ca/code/

Extensions:
 Find new applications
▪ Convexity Shape Prior (in ECCV14)
 Alternative optimization framework with LSA
▪ Pseudo-Bounds (in ECCV14)

Please come by our poster
33