N
I
P
S
Analysis of Contour Motions
Neural Information
Processing Systems
Conference 2006
Ce Liu
1. Introduction
3. Forming edgelets & boundary fragments
Existing algorithms cannot correctly analyze the motion
of textureless objects under occlusion
William T. Freeman
Edward H. Adelson
• Spatial boundary fragment extraction
– Steerable filters to obtain edge energy for each orientation band
– Track boundary fragments in frame 1 (using Canny-like threshold)
– Boundary fragments: lines or curves with small curvature
• Edgelet temporal tracking with uncertainties
– Frame 1: edgelet (x, y, q)
– Frame 2: orientation energy of q
One frame of
motion sequence
Output of the state-of-the-art
optical flow algorithm [1]
5. Inference
• Two-step inference
– Contour grouping: set switch variables to optimize Pr(S; B,O)(hard)
– Global motion given grouping (easy, least squares)
• Importance sampling to estimate the marginal of the switch variables
– Bidirectional proposal density
– A Gaussian pdf is fit with the weight of orientation energy
– 1D uncertainty of motion (even for T-junctions)
Output of our contour
motion algorithm
• Conditioned on the grouping, the graphical model for motion is a
Gaussian MRF
• Use marginals to obtain a best grouping
6. Results
A T-junction edgelet,
shown in frame 1.
The same edgelet,
shown in frame 2.
The relevant orientation Visualization of the
energy, frame 2
Gaussian pdf.
4. Forming contours:
graphical model for grouping & motion
Problem regions caused by the
occlusions of textureless objects
All results generated using the same parameter settings. The running
time varies from ten seconds to a few minutes in MATLAB, as a function
of the number of boundary fragments.
• Grouping machinery: switch variables (attached to every end
of the fragments)
• Corners: spurious T- or L-junctions
– Exclusive: one end connects to at most one other end
– Reversible: if end (i,ti) connects to (j,tj), then (j,tj) connects to
(i,ti), i.e. S(i,ti) =(j,tj), S(j,tj)=(i,ti), or S(S(i,ti))=(i,ti)
• Lines: boundary ownership
• Flat regions: illusory boundaries
b3
b3
Our approach: simultaneous grouping and
motion analysis
b1
b2
b1
b1( 0 )
b2
b1
– Multi-level contour representation
Example fragments Grouping ambiguity
Reciprocity
constraint
– Formulate graphical model that favors good
contour and motion criteria
• Affinity metric
terms:
– Inference using importance sampling
b3
b3
b2
b1
b3
b2
b2
b1
Legal contours
More legal contours
(21, 21 )
– Motion similarity
b2
(11, 11)
b1
– Curve smoothness

b2
r
2.Three levels of contour representation
b1
– Contrast consistency
h21
h22
b2
h11
b1
h12
• The graphical model for grouping:
Edgelet
Boundary fragment
Contour
Affinity
Reversibility
No self-intersection
[1] T. Brox et al. High accuracy optical flow estimation based on a theory for warping. ECCV 2004