MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Cambridge, Massachusetts
DIY: Construct/Analyze Submodular and Other
Discrete Energy Functions
Srikumar Ramalingam
© MERL mm/dd/yy
1
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
“In the world of discrete mathematics, we encounter a
bewildering variety of topics with no apparent connection
between them. But appearances are deceptive.”
Ezra Brown
© MERL mm/dd/yy
2
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Outline
•
•
•
•
•
•
•
Pseudo-Boolean Functions
Submodularity
Encoding Schemes for Multi-label Functions
Graph Construction for Higher Order Functions
Alpha-Expansion
Alpha-Beta Swap
Range Moves
© MERL mm/dd/yy
3
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Pseudo Boolean Functions (PBF)
• Variables:
x1 , x2 ,..., xn  0,1
• Negations:
xi  1  xi  0,1
n
f
:
{
0
,
1
}
R
• Pseudo-Boolean Functions (PBF):
» Maps a Boolean vector to a real number.
• Has unique multi-linear representation:
» For example:
f ( x1 , x2 , x3 , x4 )  2  3x2 x4  5x1 x2 x3
[Boros&Hammer’2002]
© MERL mm/dd/yy
4
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Posiforms for Pseudo-Boolean functions (PBF)
• Posiforms: Non-negative multi-linear polynomial except
maybe the constant terms.
f ( x1 , x2 , x3 , x4 )  2  3 x2 x4  5 x1 x2 x3
 2  3(1  x2 ) x4  5 x1 x2 x3
 2  3 x4  3 x2 x4  5 x1 x2 x3
 2  3(1  x4 )  3 x2 x4  5 x1 x2 x3
  1  3x4  3x2 x4  5x1 x2 x3
• Several posiforms exist for a given function.
• Provides bounds for minimization, e.g.   1
[Boros&Hammer’2002]
© MERL mm/dd/yy
5
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Set Functions are Pseudo Boolean Functions (PBF)
• Finite ground set V  {1,2,..., n}
• Set function (Input - subset of V , output - real number)
fs : 2  R
V
• 1-1 correspondence exists between x1 , x2 ,..., xn  0,1
and subset S of V .
V  {1,2,3,4}
xi  1
{x1  1, x2  1, x3  0, x4  1}  (1,2,4) xi  0
© MERL mm/dd/yy
iS
iS
6
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Set Functions are Pseudo Boolean Functions (PBF)
• Consider a PBF
f ( x1 , x2 , x3 , x4 )  2  3 x2 x4  5 x2 x3
• Equivalent to a set function
f s ({1,2})  2  3(1)(0)  5(1)(0)  2
f s ({2,3})  2  3(1)(0)  5(1)(1)  7
© MERL mm/dd/yy
7
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Submodular set functions (Union-Intersection)
A
B
A, B  V
A B
• A set function f : 2  R is submodular if and only if:
V
f ( A)  f ( B)  f ( A  B)  f ( A  B), A, B  V
© MERL mm/dd/yy
8
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Submodular set functions (Union-Intersection)
f ( A)  f ( B)  f ( A  B)  f ( A  B), A, B  V
Let us consider a very simple case with only two
variables x1 and x2 .
V  {1,2}, A  {1}, B  {2}
f (0,0) f (0,1)
Using submodularity, we have:
f (1,0)
f (1,1)
f ( x1  1, x2  0)  f ( x1  0, x2  1) • Main diagonal elements
are smaller than offf ( x1  1, x2  1)  f ( x1  0, x2  0)
f (1,0)  f (0,1)  f (1,1)  f (0,0)
© MERL mm/dd/yy
diagonal ones.
• Blue is larger than red.
9
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Quadratic Pseudo Boolean Functions (QPBF)
• Example of quadratic pseudo Boolean functions
f ( x1 , x2 , x3 , x4 )  1  x1  3 x2  x1 x2  5 x3 x4
[Boros&Hammer’2002]
© MERL mm/dd/yy
10
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Submodular Quadratic Pseudo Boolean Functions
• A QPBF is submodular if and only if all quadratic coefficients
are non-positive.
f 3 ( x1 , x2 , x3 )  15  x1  3 x2  x1 x2  5 x2 x3
© MERL mm/dd/yy
11
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Example for submodular QPBF
f 3 ( x1 , x2 , x3 )  15  x1  3 x2  3 x1 x3  5 x2 x3
V  {1,2,3}, A  {1,2}, B  {2,3}
A  B  {1,2,3}, A  B  {2}
f ( A)  15  1  3(1)  3(1)(0)  5(1)(0)  13
f ( B)  15  0  3(1)  3(0)(1)  5(1)(1)  7
f ( A  B)  15  1  3(1)  3(1)(1)  5(1)(1)  5
f ( A  B)  15  0  3(1)  3(0)(0)  5(1)(0)  12
 f ( A)  f ( B)  f ( A  B)  f ( A  B), (13  7  5  12)
© MERL mm/dd/yy
12
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Network model for submodular QPBF
• A submodular QPBF f can be associated with a network Gv .
• There is 1-1 correspondence every edge in network and
every term in f .
• Let us denote source by s  0 and sink by t  1.
• An edge that goes from x1 to x2 is denoted by x1 x2 .
x1
x2
sx1
x1 x2
x1
© MERL mm/dd/yy
s
x2
t
x2 t
13
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Network model for submodular QPBF
x1
x1 x2
x2
s
x1  sx1
x2
x1
x2  x2 t
t
• Given a QPBF we rewrite it using a posiform representation
using only three types of terms: xi x j , xi ,
xi ,
f  3 x1  x2  4 x1 x2
f  3 x1  x2  (4 x1 x2  4 x2  4 x2 )
f  3 x1  x2  4(1  x1 ) x2  4 x2
f  3 x1  3 x2  4 x1 x2
3
x1
s
4
f  3 x1  (3 x2  3  3)  4 x1 x2
f  3  3 x1  3(1  x2 )  4 x1 x2
t
x2
3
f  3  3sx1  3 x2t  4 x1 x2
© MERL mm/dd/yy
14
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Network model for submodular QPBF
• There is a one-one correspondence between values of f
and s-t cut values of Gv . [Hammer 1965]
s
3
sx1
x1
4 x1 x2
x2
3
t
x2 t
f ( x1  0, x2  1) 
C ({x1 , x2 })  4
© MERL mm/dd/yy
s-t mincut
[Ford&Fulkerson’62,
Goldberg&Tarzan86]
15
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Network model for submodular QPBF
• There is a one-one correspondence between values of f
and s-t cut values of Gv . [Hammer 1965]
s
3
x1
4
x2
3
t
f ( x1  1, x2  0) 
C ({x2 , s},{x1 , t})  3  3  6
Thus we can compute the minimum of f using
maxflow/mincut algorithm on the associated Gv .
© MERL mm/dd/yy
s-t mincut
[Ford&Fulkerson’62,
Goldberg&Tarzan86]
16
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Simple MRF problems with 2-labels
[Boykov and Jolly’2001,
Rother et al. 2004]
[Kohli&Torr’2005]
[Ramalingam et al. 2009]
© MERL mm/dd/yy
17
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Network model for non-submodular QPBF
• A non-submodular QBPF f can be associated with a
network Gv as follows:
f  3x1  x2  4 x1 x2
3
x1
s
-4
f  3x1  5 x2  4(1  x1 ) x2
f  3sx1  5sx 2 4 x1 x2
5
x3
t
• There is no polynomial-time algorithm for s-t mincut on a
network with negative edge capacities.
• A submodular QBPF can always be associated with a
network with non-negative edge capacities.
© MERL mm/dd/yy
18
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Minimizing Quadratic Pseudo Boolean Functions
• If QPBF is submodular, use maxflow algo..
[Ford&Fulkerson’62,
Goldberg&Tarzan86]
• If QPBF is non-submodular, use QPBO or message passing
algorithms.
[Boros&Hammer’2002]
© MERL mm/dd/yy
19
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Multi-label Problems
• Choose the disparities from the discrete set: (1,2,..., L)
© MERL mm/dd/yy
20
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Multi-label Problems
Exact Methods:
Transform the given multi-label problems to Boolean
problems and solve them using maxflow/mincut algorithms
or QPBO techniques.
Approximate Methods:
Develop iterative move-making algorithms where each
move corresponds to a Boolean problem.
© MERL mm/dd/yy
21
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Transforming multi-label functions to Boolean ones
1. Use 2 or more Boolean variables to denote each
multi-label variable.
2. Write the original multi-label energy function.
3. Replace multi-label variables with Boolean ones
using encoding functions.
© MERL mm/dd/yy
22
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Boolean Energy Function
• Variables x1 , x2 ,..., xn  0,1.
 xj - cost of assigning xi  j  {0,1}.
i
 xlmx - cost of jointly assigning xi  l and x j  m.
i
j
Energy function:
1
1
j 0
j 0
1
1
E ( x1 , x2 )    xj1 xj1    xj2  xj2    xij1x2  xi1 xj2
© MERL mm/dd/yy
i 0 j 0
23
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Multi-label Energy Function
• Variables y1 , y2 ,..., ym  0,1,..., L.

l
yi
 yl
-
i
 ylmy
i
1
yi  l
0
otherwise.
cost for assigning a single variable
yi  l.
- cost of jointly assigning yi  l and y j  m.
j
Energy function:
L
L
j 1
j 1
L
L
E ( y1 , y2 )    yj1 yj1    yj2  yj2    yij1 y2  yi1 yj2
© MERL mm/dd/yy
i 1 j 1
24
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Transforming multi-label to Boolean functions
• Use 2 or more Boolean variables to encode the states of a
single multi-label variable.
y
x1 x2 x3
1
1
1
1
2
0
1
1
3
0
0
1
4
0
0
0
Using 3 Boolean
variables to denote a
4-label variable
• There is a one-one correspondence at their respective
minima:
arg min E ( y1 ,..., ym )  arg min ( x1 ,..., xn )
yi ,i {1,.., m}
xi ,i {1,...n}
[Ramalingam et al.2008]
© MERL mm/dd/yy
25
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Encoding multi-label variables using Boolean ones
1.
Choose the encoding.
y
x1 x2 x3
1
1
1
1
2
0
1
1
3
0
0
1
4
0
0
0
Using 3 Boolean
variables to denote a
4-label variable
2. Generate encoding functions  yi
using Boolean variables.
l
 1y  x1 ,
 y2  x2  x1 ,
 y3  x3  x2 ,
 y4  1  x3
3. Only certain Boolean assignments are allowed,
i.e., xi  xi 1 , i  {1,2}. Penalty term such as H xi 1 xi
avoids ( xi 1  0, xi  1) using a high cost H .
[Ramalingam et al.2008]
© MERL mm/dd/yy
26
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Graphical Interpretation of the Encoding
y 1
y
x1 x2 x3
1
1
1
1
2
0
1
1
3
0
0
1
4
0
0
0
Using 3 Boolean
variables to denote a
4-label variable
y2
s
x1
H
x2
y3
H
x3
t
• A network based on Boolean variables. y  4
• Restricted Boolean configurations are inhibited using high
edge costs shown as H .
• st-cuts on the Boolean network and the associated states of
y are shown.
[Ishikawa’03, Schlesinger & Flach’06]
© MERL mm/dd/yy
27
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Submodularity for Multi-Label Functions
• Variables y1 , y2 ,..., ym  0,1,..., L.
 ylmi y j - cost of jointly assigning yi  l and y j  m.
•

l ( m 1)
yi y j

2-label case:
 ylmy
i

•
•
© MERL

( l 1) m
yi y j
j
( l 1) m
yi y j
lm
yi y j

L
( l 1)( m 1)
yi y j
 x01x   x10x   x00x   x11x
i
j
i
j
 yl (ym 1)
i

j
( l 1)( m 1)
yi y j
Main diagonal elements are
smaller than off-diagonal ones.
Blue is larger than red.
i
j
i
j
L
m 1
m
l 1
l
0
0
yi
yj
[Ishikawa’03, Schlesinger & Flach’06]
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Multi-Label Energy with Unary Terms
• Consider the following energy function with only one
variable:
4
E ( y1 )               
j 1
j
y1
j
y1
1
y1
1
y1
2
y1
2
y1
3
y1
3
y1
4
y1
4
y1
• Transform the multi-label function to a Boolean one using
encoding functions
E ( y1 )   1y1 1y1   y21 y21   y31 y31   y41 y41
  1y1 ( x1 )   y21 ( x2  x1 )   y31 ( x3  x2 )   y41 (1  x3 )
 x1 ( y21   1y1 )  x2 ( y21   y31 )  x3 ( y31   y41 )   y41
• All unary functions are submodular with this encoding!
© MERL mm/dd/yy
29
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Multi-Label Energy with Pairwise Terms
A simple pairwise energy function with two variables:
4
4
E ( y1 , y2 )    yij1 y2  yi1 yj2
i 1 j 1
Encoding each 4-label variable using 3 Boolean ones:
yi  ( x1yi , x2yi , x3yi ), i  {1,2}
Encoding functions
 1y  x1y ,
i
i
 y2  x2y  x1y ,
i
i
i
 x x ,
3
yi
yi
3
yi
2
 y4  1  x3y
1
1
© MERL mm/dd/yy
30
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Multi-Label Energy with pairwise terms
• Multi-label pairwise energy with two 4-label variables:
4
4
E ( y1 , y2 )   yij1 y2  yi1 yj2
i 1 j 1
Substituting the encoding functions:
3
3
E ( y1 , y2 )   xiy1 x jy2 ( y(1i y21)( j 1)   yij1 y2   y(1i y21) j   yi1( yj21) ) 
i 1 j 1
…unary terms
If non-positive, then the multi-label energy is a
submodular QPBF!
© MERL mm/dd/yy
31
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Multi-Label Energy with pairwise terms
3
3
E ( y1 , y2 )   xiy1 x jy2 ( y(1i y21)( j 1)   yij1 y2   y(1i y21) j   yi1( yj21) ) 
i 1 j 1
…unary terms
• Submodularity condition for multi-label potentials is given
by:

l ( m 1)
yi y j

( l 1) m
yi y j

lm
yi y j

( l 1)( m 1)
yi y j
If the original multi-label function is submodular, then the
transformed Boolean energy is also submodular!
© MERL mm/dd/yy
32
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Multi-Label Energy with pairwise terms –
Graphical Interpretation
3
3
E ( y1 , y2 )   xiy1 x jy2 ( yi1( yj21)   y(1i y21) j   yij1 y2   y(1i y21)( j 1) ) 
i 1 j 1
s
y1
1
…unary terms
x1y2
x
H
H
x2y1
x2y2
For a submodular multi-label
energy, the associated Boolean
network has non-negative edge
weights.
H
H
x3y1
x3y2
t
© MERL mm/dd/yy
33
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Transforming submodular multi-label functions to
submodular Boolean ones.
Submodular
multi-label
functions
y
x1 x2 x3
1
1
1
1
2
0
1
1
3
0
0
1
4
0
0
0
Submodular
Boolean
functions
Using 3 Boolean
variables to denote a
4-label variable
To encode a L-label variable we use (L-1) Boolean nodes.
© MERL mm/dd/yy
34
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Why don’t we use a more compact encoding?
Encoding functions
y
x1
x2
1
0
0
2
0
1
3
1
0
4
1
1
Using 2 Boolean variables to denote a
4-label variable
Energy
  (1  x1 )(1  x2 ),
1
y
 y2  (1  x1 ) x2 ,
 y3  x1 (1  x2 ),
 y4  x1 x2
E ( y1 , y2 )  xiy1 x jy2 xky1 xly2 yab1 y2 .....
• Second degree multi-label problems are transformed to
fourth degree Boolean problems.
• Submodular multi-label problems need not be submodular
[Ramalingam et al.2008]
Boolean networks.
© MERL mm/dd/yy
35
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Higher order functions for image denoising
• Higher Order Energy Functions
Unary
Pairwise
Higher
order
MRF for Image
Denoising
Original
© MERL mm/dd/yy
Pairwise MRF Higher order MRF
Images Courtesy: Lan et al. ECCV06
36
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Triple clique terms
• Encoding each 4-label variable using 3 Boolean ones:
yi  ( x , x , x ), i  {1,2,3}
yi
1
yi
2
yi
3
y
1


x
• Encoding functions: y
1 ,
i
i
 y2  x2y  x1y ,
i
i
i
 y3  x3y  x2y ,
i
i
i
 y4  1  x3y
i
i
© MERL mm/dd/yy
37
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Triple clique terms
• Energy function with triple clique terms:
4
4
4
E ( y1 , y2 , y3 )   yijk1 y2 y3  yi1 yj2  yk3
i 1 j 1 k 1
3
3
3
E ( y1 , y2 , y3 )    ijk xiy1 x jy2 xky3  L2
i 1 j 1 k 1
where L2 denotes the second degree and lower
degree energy terms.
© MERL mm/dd/yy
38
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Triple clique terms
• If aijk  0
3
3
E ( y1 , y2 , y3 )    ijk xiy1 x jy2 xky3  L2
i 1 j 1 k 1
aijk xiy1 x jy2 x 
y3
k
s
min aijk ( xiy1  x jy2  xky3  2) z
z{0 ,1}
3
x1y1
x1y2
aijk
z
aijk
x1y3
aijk
aijk
x2y1
x2y2
x2y3
x3y1
x3y2
x3y3
t
© MERL mm/dd/yy
39
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Triple clique terms
• If aijk  0
s
x1y1
x1y2
x1y3
x2y1
x2y2
x2y3
x3y1
x3y2
x3y3
aijk
aijk
z
t
© MERL mm/dd/yy
aijk
aijk
40
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Submodularity for higher-order functions
Submodularity for a k-clique function
For a k-clique function, if we fix (k-2) variables, then the
remaining pairwise function with the left-over two variables
should be submodular. The condition should hold true for
all possible combinations of (k-2) variables.
f (_, _ 1, _,., _,0, _)  f (_, _ 0, _,., _,1, _) 
f (_, _ 0, _,., _,0, _)  f (_, _ 1, _,., _,1, _)
fixed
© MERL mm/dd/yy
41
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Geometric layout estimation
[Hoiem, Efros, Hebert,
IJCV, 2007 ]
Sky
Vertical
Ground
We use a 3rd degree
prior based on
ordering of sky,
building and ground.
CRF
© MERL mm/dd/yy
42
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Original
© MERL mm/dd/yy
Superpixel
Ground truth
Hoiem et al.
Ramalingam et al.
43
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Exact methods for submodular functions
• A third degree Boolean submodular function can be transformed to a
pairwise one [Billionnet’1981, Kolmogorov&Zabih’2004]
• A second degree multi-label submodular function can be solved using s-t
mincut [Ishikawa’2003, Schlesinger&Flach’2006].
• Any higher order multi-label function can be transformed to Boolean
second degree function (with no guarantee on submodularity after 3rd
degree case) [Ramalingam et al. 2008].
• Not all fourth order submodular functions can be transformed to pairwise
ones [Zivny et al. 2009].
• A k-variable submodular function needs at most D(k) (the Dedekind
number ) auxiliary variables to quadratize, if possible [Ramalingam,
Russell, Ladicky and Torr, 2009].
© MERL mm/dd/yy
44
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Energy
Move Making Algorithms
Solution Space
[Image courtesy: Pushmeet Kohli, Phil Torr]
© MERL
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Move Making Algorithms
Current Solution
Search
Neighbourhood
Energy
Optimal Move
Solution Space
[Image courtesy: Pushmeet Kohli, Phil Torr]
© MERL
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
  Expansion
  building
[Boykov et al. 2001]
© MERL mm/dd/yy
47
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
  Expansion
• Let yi and y j be two adjacent variables whose labels are
not  .
retain
yi
la

y j retain
lb

In the move space, we compute if the two variables should
retain the same labels or move to label  .
[Boykov et al. 2001]
© MERL mm/dd/yy
48
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
  Expansion
• In the move space, we use two Boolean variables xi and x j to
denote yi and y j respectively. The encoding is shown below:
yi  la  xi  0
y j  la  x j  0
yi    xi  1
yj    xj 1
• Submodularity condition states that the sum of main
diagonal elements is greater than the sum of elements in
the off-diagonal:


00
xi x j
10
xi x j

01
xi x j

11
xi x j

=
l a lb
yi y j
 yly
b
i
j

la
yi y j
 yy
i
j
[Boykov et al. 2001]
© MERL mm/dd/yy
49
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
  Expansion
• Submodularity condition states that the sum of main
diagonal elements is greater than the sum of elements
in the off-diagonal:
00
xi x j

01
xi x j
 x10x

11
xi x j

i
j

=
l a lb
yi y j
lb
y y
i

la
yi y j


j
If the multi-label potentials
satisfy metric condition:
y y
i

  yi y j  
l a lb
yi y j
la
yi y j
lb
  yi y j  0
j
la , lb  L,
 yl ly  0,
a a
1 2
 yl ly   yl ly  0,
a b
b a
1 2
1 2
 yl ly   yl ly   yl ly
© MERL mm/dd/yy
a b
b c
a c
1 2
1 2
1 2
[Boykov et al. 2001]
50
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
  Expansion
[Image courtesy: Lubor Ladicky]
© MERL mm/dd/yy
[Boykov et al. 2001]
51
MITSUBISHI ELECTRIC RESEARCH LABORATORIES

Swap
• The variables having the labels  and
labels or retain their previous states.
retain
yi
y j retain


 can swap their


[Boykov et al. 2001]
© MERL mm/dd/yy
52
MITSUBISHI ELECTRIC RESEARCH LABORATORIES

Swap
• In the move space, we use two Boolean variables xi and x j
to denote yi and y j respectively. The encoding is shown
below: y    x  0
y   x 0
i
j
i
yi    xi  1
j
yj    xj 1
• Submodularity condition states that the sum of main
diagonal elements is greater than the sum of elements in
the off-diagonal:
 x00x
i

j
10
xi x j
 x01x
i

j
11
xi x j
 yy  yy
j
 yy
j
i
=
i
j
j
i
 yy
i
[Boykov et al. 2001]
© MERL mm/dd/yy
53
MITSUBISHI ELECTRIC RESEARCH LABORATORIES

Swap
• Submodularity condition states that the sum of main
diagonal elements is greater than the sum of elements in
the off-diagonal:
 x00x
i

j
10
xi x j
 x01x
i
 x11x
i
j
j
 yy  yy
=
i
 yy
i
j
j
• Semi-metric condition:
i
 yy
i
 yy   yy   yy   yy  0
j
i
j
i
j
i
j
i
j
j
la , lb  L,
 yl ly  0,
a a
1 2
 yl ly   yl ly  0
a b
b a
1 2
1 2
[Boykov et al. 2001]
© MERL mm/dd/yy
54
MITSUBISHI ELECTRIC RESEARCH LABORATORIES

Range Swap
• The variables having the labels between  and  can
retain their labels or move to any labels between and  .
• In the move space, we compute if the variables retain their
old states or move to new states as shown below




• The optimal labeling in the move space can be computed
using a variant of Ishikawa’s graph construction.
[Veksler et al. 2007]
© MERL mm/dd/yy
55
MITSUBISHI ELECTRIC RESEARCH LABORATORIES
Summary
• It is possible to construct/analyze energy functions purely
using pseudo-Boolean techniques.
• The techniques provide valuable insights on the nature of
an energy function.
© MERL mm/dd/yy
56