Michael Bronstein Heat diffusion descriptors
Heat diffusion descriptors
for deformable shapes
Michael Bronstein
Institute of Computational Science
Universita della Svizzera Italiana
Lugano, Switzerland
Weizmann Institute of Science, 4 November 2010
1
2
Michael Bronstein Heat diffusion descriptors
Alex Bronstein
TAU
Maks Ovsjanikov
Stanford
Leo Guibas
Stanford
Iasonas Kokkinos
ECP Paris
Dan Raviv
Technion
Ron Kimmel
Technion
3
Michael Bronstein Heat diffusion descriptors
The next challenge
Text
Visual data
Geometric data
Michael Bronstein Heat diffusion descriptors
Shape retrieval today
4
Michael Bronstein Heat diffusion descriptors
Bags of words
construction
architecture
Italy
France
cathedral
church
basilica
Paris
Rome
Gothic
Roman
Notre Dame de Paris is a Gothic cathedral in the fourth
quarter of Paris, France. It was the first Gothic
architecture cathedral, and its construction spanned
the Gothic period.
St. Peter’s basilica is the largest church in world,
located in Rome, Italy. As a work of architecture, it is
regarded as the best building of its age in Italy.
5
6
Michael Bronstein Heat diffusion descriptors
Outline
Geometric
words
Feature
descriptor
“
Bag of geometric
words
Scale invariance
Spatially-sensitive
bag of words
Volumetric
descriptors
”
“
”
Geometric
expressions
7
Michael Bronstein Heat diffusion descriptors
Shape descriptors
Representation
Curvature
Integral
volume1
Volume/Mesh
Any
Shape context3
Any
HKS4
Any
SI-HKS5
Any
1 Gelfand
Scale Bending Topology
Any
Spin image2
vHKS6
Rigid
Volume/Mesh
et al. 2005; 2 Johnson, Hebert 1999; 3 Belongie et al. 2002; 4 Sun et al. 2009
5 B, Kokkinos 2010; 6 Raviv, BBK 2010
8
Michael Bronstein Heat diffusion descriptors
Diffusion geometry
Heat equation
where
- positive semidefinite Laplace-Beltrami
operator
- heat distribution
Fundamental solution (heat kernel,
) – heat equation solution
for initial conditions
Amount of heat transferred from point x to point y in time t
Spectral expression
Michael Bronstein Heat diffusion descriptors
Heat kernel interpretation
Geometric interpretation: “multiscale Gaussian curvature”
Probabilistic interpretation: the probability of a random walk to remain
at point x after time t.
Sun, Ovsjanikov, Guibas, 2009
9
Michael Bronstein Heat diffusion descriptors
Heat kernel signature
Multiscale descriptor
Time (scale)
■ Intrinsic, hence deformation-invariant
■ Provably informative
■ Efficiently computable on different shape representations
■ Multiscale
Sun, Ovsjanikov, Guibas, 2009
10
11
Michael Bronstein Heat diffusion descriptors
Shape
Geometric vocabulary
Ovsjanikov, BB, Guibas, 2009
BB. Ovsjanikov, Guibas 2010
Bag of geometric words
12
Michael Bronstein Heat diffusion descriptors
Bags of geometric words
1
Ovsjanikov, BB, Guibas, 2009
BB. Ovsjanikov, Guibas 2010
Index in geometric vocabulary
64
Michael Bronstein Heat diffusion descriptors
SHREC 2010: Robust shape retrieval benchmark
Query set
Transformation
Database (>1K shapes)
B et al. 2010
13
14
Michael Bronstein Heat diffusion descriptors
Query
B et al. 2010
Toldo et al. 2009
Shape
15
Michael Bronstein Heat diffusion descriptors
Performance results
Toldo et al. 2009
Bags of words using spin image
descriptor
Shape
Bags of words using HKS descriptor,
vocabulary of size 48
Performance criterion: mean average precision (mAP) in %
Toldo et al. 2009
B et al. 2010
16
Michael Bronstein Heat diffusion descriptors
Scale invariance
Original shape
Scaled by
Not scale invariant!
17
Michael Bronstein Heat diffusion descriptors
Scale-invariant HKS
Log scale-space
0
-5
log + d/d
Fourier transform
magnitude
0
4
-0.01
3
-0.02
2
-0.03
-10
1
-0.04
-15
0
100
200

300
Scaling = shift and
multiplicative constant
B, Kokkinos CVPR 2010
0
100
200

300
Undo scaling
0
0
2
4
6
8 10 12 14 16 18 20
=2k/T
Undo shift
18
Michael Bronstein Heat diffusion descriptors
Scale invariant HKS
HKS
B, Kokkinos 2010
SI-HKS
19
Michael Bronstein Heat diffusion descriptors
Query
B, Kokkinos 2010
HKS
SI-HKS
20
Michael Bronstein Heat diffusion descriptors
HKS vs SI-HKS
HKS, vocabulary of size 48
SI-HKS, vocabulary of size 48
Performance criterion: mean average precision (mAP) in %
B, Kokkinos 2010
21
Michael Bronstein Heat diffusion descriptors
Expressions
Ovsjanikov, BB & Guibas 2009
Matrix is a science fiction
movie released in 1999.
Matrix
refers
to
a
simulated reality created
by machines in order to
subdue
the
human
population.
matrix decomposition
matrix factorization
science fiction
canonical form
In biological science,
decomposition is the
process of organisms to
break down into simpler
form of matter. Usually,
decomposition
occurs
after death.
matrix
decomposition
is
a
the
of
in
to
by
science
form
In math science, matrix
decomposition
is
a
factorization of a matrix
into some canonical
form. Each type of
decomposition is used in
a particular problem.
22
Michael Bronstein Heat diffusion descriptors
Expressions
Ovsjanikov, BB & Guibas 2009
In particular matrix used
type a some science,
decomposition form a
factorization
of
is
canonical. matrix math
decomposition is in a
Each problem. into of
matrix decomposition
matrix factorization
science fiction
canonical form
matrix
decomposition
is
a
the
of
in
to
by
science
form
In math science, matrix
decomposition
is
a
factorization of a matrix
into some canonical
form. Each type of
decomposition is used in
a particular problem.
23
Michael Bronstein Heat diffusion descriptors
Geometric expressions
Yellow
“Yellow Yellow”
No total order between points (only “far” and “near”)
Geometric expression = a pair of spatially close geometric words
Ovsjanikov, BB & Guibas 2009
Michael Bronstein Heat diffusion descriptors
Spatially-sensitive bags of words
Ovsjanikov, BB & Guibas 2009
24
25
Michael Bronstein Heat diffusion descriptors
HKS vs SI-HKS
HKS, vocabulary of size 48
Spatially-sensitive HKS,
vocabulary of size 8x8
Performance criterion: mean average precision (mAP) in %
B et al. 2010
26
Michael Bronstein Heat diffusion descriptors
Is our shape model good?
Boundary ∂X
Interior
X
Raviv, BBK 2010
27
Michael Bronstein Heat diffusion descriptors
Is our shape model good?
Boundary isometry
Preserves geodesic distances on
the boundary surface
Camel illustration from Sumner et al.
Raviv, BBK 2010
Volume isometry
Preserves geodesic distances
inside the volume
28
Michael Bronstein Heat diffusion descriptors
Diffusion equation
Boundary diffusion
Volumetric diffusion
where
- Laplace-Beltrami operator
- Euclidean Laplacian
- normal to boundary surface
Raviv, BBK 2010
29
Michael Bronstein Heat diffusion descriptors
Heat kernels
Boundary heat kernel
where
Geometric interpretation
“Multiscale Gaussian curvature”
Raviv, BBK 2010
Volumetric heat kernel
30
Michael Bronstein Heat diffusion descriptors
Heat kernel signatures
HKS
Boundary+volume isometry
Boundary isometry
Raviv, BBK 2010
vHKS
Boundary+volume isometry
Boundary isometry
31
Michael Bronstein Heat diffusion descriptors
HKS
Raviv, BBK 2010
vHKS
32
Michael Bronstein Heat diffusion descriptors
HKS vs vHKS
HKS, vocabulary of size 48
vHKS, vocabulary of size 48
Performance criterion: mean average precision (mAP) in %
Raviv, BBK 2010
33
Michael Bronstein Heat diffusion descriptors
Summary
Geometric
words
Feature
descriptor
“
Bag of geometric
words
Scale invariance
Spatially-sensitive
bag of words
Volumetric
descriptors
”
“
”
Geometric
expressions
Michael Bronstein Heat diffusion descriptors
Thank you
34