Stochastic Systems Group
Some Rambling (Müjdat)
and
Some More Serious Discussion
(Ayres, Junmo, Walter)
on
Shape Priors
Stochastic Systems Group
Desire to use
Shape Priors in Segmentation
• The posterior:
Segmenting
curve
Observed image
data
• The most common (implicit) prior used is the
curve length penalty:
• Want to be able to use better prior models
Stochastic Systems Group
Challenges and remarks
• Need probabilistic descriptions in the space of
shapes
– A non-linear, infinite-dimensional manifold
– Distance (similarity) measures in the shape space
• Of course, a statistical description for shapes
has uses other than segmentation as well
– Sampling from a shape density
– Recognition of objects
– Completion of incomplete shapes
Stochastic Systems Group
The PCA World
• Cootes & Taylor
– Shape representation using marker points
• Leventon, Tsai
– PCA and level sets
• More…
• Want a more principled approach
Stochastic Systems Group
Some Literature
•
“Active shape models - their training and application,”
T. Cootes, C. J. Taylor, D.H. Cooper, J. Graham, 1995.
•
“Embedding Gestalt laws in Markov random fields,”
Song-Chun Zhu, 1999.
•
“On the incorporation of shape into geometric active contours,”
Y. Chen, H.D. Tagare et al., 2001.
•
“Image segmentation based on prior probabilistic shape models,”
A. Litvin and W. C. Karl, 2002.
•
“Shape priors for level set representations,”
M. Rousson and N. Paragios, ECCV, 2002.
•
“Nonlinear shape statistics in Mumford-Shah based segmentation,”
D. Cremers, T. Kohlberger, and C. Schnorr, 2002.
•
“Geometric analysis of constrained curves for image understanding,”
A. Srivastava, W. Mio, E. Klassen, X. Liu, 2003.
•
“Analysis of planar shapes using geodesic paths on shape spaces,”
E. Klassen, A. Srivastava, and W. Mio (in review)
•
“Gaussian distributions on Lie groups and their application to stat. shape analysis,”
P. T. Fletcher, S. Joshi, C. Lu, and S. Pizer, 2003.
Stochastic Systems Group
Overview of Anuj Srivastava’s Work
• Specify a space of continuous curves with constraints (e.g. simple
closed) and exploit the differential geometry of this space
• Use geodesic paths for deformations between curves and to
compute distances – do not have analytical exps. for geodesics
• To move in the shape manifold, first move in a linear space and
then project back (using tangents/normals)
• Given two curves, solve an optimization problem to find the
geodesic path between them (find a local min)
• Build statistical descriptions based on Karcher means;
covariance of (Fourier coefficients of) tangent vectors
• Use such descriptions as priors (very preliminary)
• Some current limitations:
– Cannot handle topological changes
– Extension to surfaces in 3-D not straightforward
Stochastic Systems Group
Outline
• Some metrics proposed for shape similarity (Junmo)
• Anuj Srivastava’s work
– Geometric representations of curves
and shape spaces (Walter)
– Tangents, normals, geodesics (Ayres)
– Statistical models and application to segmentation (Junmo)
• Brief highlights from Pizer’s work (Walter)
• Discussion on all of this
Stochastic Systems Group
Some Metrics on the Space of
Shapes
• Notion of “similarity” between shapes
– Basic task of vision system is to recognize similar
objects which belong to the same category.
• Describing shapes : landmarks, level set
• Shape can be described as
• There are several metrics
for two shapes
Stochastic Systems Group
Hausdorff Metric
• Given
• Very sensitive to any outlier points in
and
Stochastic Systems Group
Template Metric
•
• Totally insensitive to outliers
Stochastic Systems Group
Transport Metric
• Fill
with ‘stuff’ and find the shortest paths
along which to move this ‘stuff’ so that it now
fills
Stochastic Systems Group
Optimal Diffeomorphism
• If
and
are topologically different, the
distance would be infinite.
– E.g.
is
minus a pinhole
– E.g. A small break cuts a shape into two
Stochastic Systems Group
Geometric Representations of Curves
and Shape Spaces
• Restrict attention to closed curves in R2.
• Classify curves which differ only by
orientation preserving rigid motions (rotation
and translation) and uniform scaling as the
same shape.
• Consider two different representations of
planar curves for simple closed curves:
– Using direction functions.
– Using curvature functions.
Stochastic Systems Group
Preliminaries
• First, the issue of scaling is resolved by fixing
the length of all curves to 2.
• Curves are parameterized by arc length
with period 2 &
• Define the unit tangent function
where S1 is the unit circle, &
where (s) is the direction function.
• (s+2) - (s) = 2n (n = rotation index { = 1})
Stochastic Systems Group
Curves to Consider
• Consider entire set of curves with rotation
index 1 because this set is complete.
– Contains its own limit points.
• Note that the set of simple closed curves is an
open subset of this larger set.
Stochastic Systems Group
Shape Using Direction Functions
• Direction function for S1 is 0(s) = s. All other
closed curves have direction function
 = 0 + f, where f is L2 periodic on [0,2].
• To adjust for rigid rotations, restrict attention
to  s.t.
• To ensure curve closure, require
Stochastic Systems Group
More Direction Functions
• Define a map
by
• The pre-shape space C1 is 1-1(,0,0).
• Multiple elements of C1 may denote the same
shape. An adjustment of the reference point
(s=0) handles this.
Stochastic Systems Group
Geometry of Shape Manifolds
• Constraints define a manifold embedded in
0 + L2
• Move along manifold by moving in tangent
space and projecting back to manifold
• Tangent space is infinite dimensional, but
normal space is characterized by three
constraints defined in f1
Stochastic Systems Group
Tangents and Normals
• The derivative of f1 in the direction of f at  is:
• Implies df1 is surjective
• If f is orthogonal to {1, sin , cos }, then df1=0 in the
direction of f and hence f is in the tangent space
Stochastic Systems Group
Projections
• Want to find the closest element in C1 to an
arbitrary   0 + L2
• Basic idea: move orthogonal to level sets so
projections under f form a straight line in R3
• For a point b  R3, we define the level set as:
• Let b1=(,0,0). Then its level set is the
preshape space C1
Stochastic Systems Group
Approximate Projections
• If points are close to C1, then one can use a
faster method
• Let d be the normal vector at  for which
f(+d)=b1. Can do first order approximation
to compute this
• Approximate Jacobian as:
Stochastic Systems Group
Iterative algorithm
• Define the residual (error) vector as
• Then:
where
• Iteratively update  + d  until the error goes to
zero
• Call this projection operator P
Stochastic Systems Group
Example Projections
Fig. 1: Projections of arbitrary curves into C1
Stochastic Systems Group
Geodesics
• Definition: For a manifold embedded in Euclidean space, a
geodesic is a constant speed curve whose acceleration vector is
always perpendicular to the manifold
• Define the metric between two shapes as the distance along the
manifold between the shapes with respect to the L2 inner
product
• Nice features:
– Defined for all closed curves
– Interpolants are closed curves
• Finds geodesics in a local sense, not necessarily global
Stochastic Systems Group
Paths from initial conditions
• Assume we have a  in C1 and an f in the
tangent space
• Approximate geodesic along manifold by
moving to +fDt and projecting that back onto
the manifold (Dt is step size)
• So (t+Dt) = P((t)+f(t)Dt)
Stochastic Systems Group
Transporting the tangent vector
• Now f(t) is not in the tangent space of (t+Dt)
• Two conditions for a geodesic:
– The acceleration vector must be perpendicular to the
manifold: simply project f into the next tangent space
– The curve must move at constant speed: renormalize so
||f(t+1)||=||f(t)||
• hk is the orthonormal basis of the normal space
Stochastic Systems Group
Geodesics on shape spaces
• S1 is a quotient space of C1 under actions of S1 by
isometries, so finding geodesics in S1 equivalent to
finding geodesics in C1 which are orthogonal to S1
orbits
• S1 acting by isometries implies that if a geodesic in
preshape space is orthogonal to one S1 orbit, it’s
orthogonal to all S1 orbits which it meets
• So now normal space has one additional component
spanned by
• The algorithm is the same as detailed earlier except
with an expanded normal space
Stochastic Systems Group
Geodesics between shapes
• We know how to generate geodesic paths given
 and f
• Now we want to construct a geodesic path from
1 to 2
• So we need to find all f that lead from 1 to an S1 orbit
of 2 in unit time, and then choose the one that leads
to the shortest path
• Let Y define the geodesic flow, with Y(1,0,f)=1 as
the initial condition
• We then want Y(1,1,f)=2
Stochastic Systems Group
Finding the geodesic
• Define an error functional which measures
how close we are to the target at t=1:
• Choose the geodesic as the flow Y which has
the smallest initial velocity ||f||
• i.e., min ||f|| s.t. H[f]=0
• Hard because infinite dimensional search
Stochastic Systems Group
Fourier decomposition
• f  L2, so it has a Fourier decomposition
• Approximate f with its first m+1 cosine
components and its first m sine components:
• Let a be the vector containing all of the
Fourier coefficients
• Now optimization problem is min ||a|| s.t.
H[a]=0
Stochastic Systems Group
Geodesic paths
Fig. 2: Geodesic paths between two shapes
Stochastic Systems Group
Statistical Modeling of Shapes
• Given example shapes
–
–
–
–
–
Mean shape
Shape variation
Shape prior
Sampling from the prior
Using shape prior for segmentation of occluded
images
Stochastic Systems Group
Mean Shape
• Given the geodesic distance function
,
• Karcher mean of shapes
is defined to
be a shape
for which the variance function
is a local minimum
• The Karcher mean exists, but may not be unique
Stochastic Systems Group
Variances on Shape Spaces
• Model the variation from the mean shape
as
,
an element of the tangent space at the mean shape
• Represent by its Fourier expansion:
• Model
as multivariate normal with
mean 0 and covariance matrix
Stochastic Systems Group
Shape Sampling
• Sample Fourier coefficients of tangent vectors
from the multivariate normal distribution
• Move along the geodesic path starting from
the mean shape in the direction of
by
distance
Stochastic Systems Group
Shape Sampling Examples
Observed shapes
Mean
shape
Random samples
from the
Gaussian model
Stochastic Systems Group
Shape Prior
•
•
: the space of curves (larger than shape space)
can be represented as pairs
–
: parameters for translation, rotation, and
scaling
–
: the shape
• Gaussian density with a mean shape
with the
shape dispersion
Stochastic Systems Group
Bayesian Discovery of Objects
Stochastic Systems Group
Some closing thoughts
on Anuj Srivastava’s work
• Most energy has been spent on manipulating the
shape manifold
• Work on using these models as priors preliminary
– Non-diagonal covariance – explore modes of variation
– Non-Gaussian?
– Mean in manifold?
• Other thoughts
– Non-Fourier representations for tangents – KL expansion?
– Could similar ideas be used with representations based on
signed distance functions?
Stochastic Systems Group
Overview of Steve Pizer’s Work
• Medial representations (m-reps) are used to model the geometry
of anatomical objects.
• Medial parameters are not in a Euclidean space; so, PCA cannot
be used. However, m-reps model parameters are elements of a
Lie group.
• Gaussian distributions on this Lie group are considered, with
the max likelihood estimates of mean and covariance derived.
• Similar to PCA for Euclidean spaces, principal geodesic analysis
(PGA) on Lie groups are defined for the study of anatomical
variability.
• Framework is applied to hippocampi in a schizophrenia study.
–
–
–
–
86 m-rep figures are first aligned (translation/rotation/scaling)
Intrinsic mean is then computed
PGA (modes of variability) are then computed
Results yield smoother deformations when compared with PCA
Stochastic Systems Group
Medial Representation
• Introduced by Blum (1978), a 3-D object is
represented by a set of connected continuous
medial manifolds formed by the centers of all
spheres are are interior to the object and
tangent to the object boundary at two or more
points. The figure below illustrates this:
Stochastic Systems Group
Medial Atom & Lie Groups
• A medial atom is represented by
–
–
–
–
The location in space (R3)
The radius of the sphere (R+)
The local frame (SO(3))
The object angle (SO(2))
• R3 is a Lie group under vector addition, R+ is a
multiplicative Lie group, and SO(2) & SO(3) are Lie
groups under composition of rotations.
• The direct product of Lie groups is a Lie group.
Stochastic Systems Group
Lie Groups and Lie Algebras
• A Lie group is a group G that is a finite-dim
manifold such that the two group operations
of G, multiplication and inverse, are C2
mappings.
• If e is the identity of G, the tangent space at e
forms a Lie algebra. The exponential map
provides a method for mapping vectors in the
tangent space into the Lie group.
Stochastic Systems Group
Alignment and PGA
• Translation: each model is situated so that the
average of its medial atoms is at the origin
• Rotation and scaling are done in a manner
which minimizes the total sum-of-squared
distances between m-rep figures.
• After alignment, principal directions in the
geodesic are computed, and the analog to
PCA is performed.
Stochastic Systems Group
Results
• The analysis, performed on 86 aligned
hippocampus m-reps, shows smooth
deformations (compared with PCA). The
mean shape is top left, the medial atoms are
overlaid lower left, and the first 3 PGA modes
are shown right.
Stochastic Systems Group
Shape Using Curvature Functions
• Alternatively, curves can be represented by
curvature fcns. Since the rotation index is 1,
• Using
, the closure condition
Stochastic Systems Group
More Curvature Functions
• Define a map
then the pre-shape space C2 is 2-1(2,0,0).
• As with direction functions, different
placements of s=0 result in different C2 shapes.
Thus, re-parameterization is needed.
by