Cutting-Plane Training of
Non-associative Markov Network for
3D Point Cloud Segmentation
Roman Shapovalov, Alexander Velizhev
Lomonosov Moscow State University
Hangzhou, May 18, 2011
Semantic segmentation of point clouds
• LIDAR point cloud without color information
• Class label for each point
System workflow
feature
computation
CRF
inference
segmentation
graph
construction
Non-associative CRF
  (x
i
, yi ) 
i N
node features
  (x
ij
, y i , y j )  max
( i , j ) E
edge features
y
point labels
• Associative CRF:  ( x ij , y i , y j )   ( x ij , k , k )
• Our model: no such constraints!
[Shapovalov et al., 2010]
CRF training
CRF
inference
• parametric model
• parameters need to be
learned!
Structured learning
[Anguelov et al., 2005; and a lot more]
• Linear model:
 ( x i , y i )  w n ,i x i y
T
 ( x i , y i , y j )  w e ,ij x ij y i , k y j , l
T
• CRF negative energy: w  ( x , y )  max
y
• Find w such that
T
w  (______, ______)  w  (______, ______)
T
T
w  (______, ______)  w  (______, ______)
T
T
w  (______, ______)  w  (______, ______)
T
…
T
w  (______, ______)  w  (______, ______)
T
T
Structured loss
• Define x  features (______)
• Define structured loss, for example:
 (y , y ) 
• Find w such that
[y
i
 yi ]
i N
w  ( x , ______)  w  ( x , ______)   (______, ______)
T
T
w  ( x , ______)  w  ( x , ______)   (______, ______)
T
T
w  ( x , ______)  w  ( x , ______)   (______, ______)
T
T
…
w  ( x , ______)  w  ( x , ______)   (______, ______)
T
T
Cutting-plane training
• A lot of constraints (Kn)
w  ( x , y )  w  ( x , y )   ( y , y ),  y
T
T
• Maintain a working set
• Add iteratively the most violated one:

y  arg max w  ( x , y )   ( y , y )
y
T

• Polynomial complexity
• SVMstruct implementation [Joachims, 2009]
Results
[Munoz et al., 2009]
Our method
Results: balanced loss
better than Hamming loss
1
0.9
0.8
0.7
0.6
0.5
SVM-HAM
0.4
SVM-RBF
0.3
0.2
0.1
0
Ground recall
Building recall
Tree recall
G-mean recall
Results: RBF better than linear
1
0.95
0.9
0.85
0.8
0.75
SVM-LIN
0.7
SVM-RBF
0.65
0.6
0.55
0.5
Ground recall
Building recall
Tree recall
G-mean recall
Results: fails at very small classes
1
0.9
0.8
0.7
0.6
0.5
[Munoz, 2009]
0.4
SVM-LIN
0.3
SVM-RBF
0.2
0.1
0
Ground fscore
Vehicle fscore
Tree f-score
Pole f-score
0.2% of
trainset
Analysis
• Advantages:
– more flexible model
– accounts for class imbalance
– allows kernelization
• Disadvantages:
– really slow (esp. with kernels)
– learns small/underrepresented classes badly