CSE 554
Lecture 7: Deformation II
Fall 2011
CSE554
Deformation II
Slide 1
Review
• Rigid-body alignment
Source
• Non-rigid deformation
Target
– Intrinsic methods: deforming the
Before
boundary points
– An optimization problem
• Minimize shape distortion
• Maximize fit
After
– Example: Laplacian-based
deformation
CSE554
Deformation II
Slide 2
Extrinsic Deformation
• Computing deformation of each point in the plane or volume
– Not just points on the boundary curve or surface
Credits: Adams and Nistri, BMC Evolutionary Biology (2010)
CSE554
Deformation II
Slide 3
Extrinsic Deformation
• Applications
– Registering contents between
images and volumes
– Interactive spatial deformation
CSE554
Deformation II
Slide 4
Techniques
• Thin-plate spline deformation
• Free form deformation
• Cage-based deformation
CSE554
Deformation II
Slide 5
Thin-Plate Spline
• Given corresponding source and target points
• Computes a spatial deformation function for every point in
the 2D plane or 3D volume
Credits: Sprengel et al, EMBS (1996)
CSE554
Deformation II
Slide 6
Thin-Plate Spline
• A minimization problem
– Minimizing distances between source and
target points
– Minimizing distortion of the space (as if
bending a thin sheet of metal)
• There is a closed-form solution
– Solving a linear system of equations
CSE554
Deformation II
Slide 7
Thin-Plate Spline
• Input
pi
qi
– Source points: p1,…,pn
– Target points: q1,…,qn
• Output
p
– A deformation function f[p] for any point p
f[p]
CSE554
Deformation II
Slide 8
Thin-Plate Spline
• Minimization formulation
E
Ef
Ed
– Ef: fitting term
• Measures how close is the deformed source to the target
– Ed: distortion term
• Measures how much the space is warped
– : weight
• Controls how much non-rigid warping is allowed
CSE554
Deformation II
Slide 9
Thin-Plate Spline
• Fitting term
– Minimizing sum of squared distances between deformed source points
and target points
n
Ef
f pi
qi
2
i 1
CSE554
Deformation II
Slide 10
Thin-Plate Spline
• Distortion term
– Minimizing a physical bending energy on a metal sheet (2D):
Ed
2f
x
2
2
2
2f
2
xy
2f
y
2
2 2
x
y
– The energy is zero when the deformation is affine
• Translation, rotation, scaling, shearing
CSE554
Deformation II
Slide 11
Thin-Plate Spline
• Finding the minimizer for
E
Ef
Ed
– Uniquely exists, and has a closed form:
n
f p
M
p
p
pi
vi
i 1
where
r
r2 Log r
• M: an affine transformation matrix
2.5
• vi: deformation vectors
1.5
• Both M and vi are determined by pi,qi,
1.0
0.5
2
CSE554
r
2.0
Deformation II
1
1
2
Slide 12
Thin-Plate Spline
• Result
– At higher , the
deformation is closer
to an affine
transformation
0
0.001
Credits: Sprengel et al, EMBS (1996)
CSE554
Deformation II
0.1
Slide 13
Thin-Plate Spline
• Application: image registration
– Manual or automatic feature pair detection
Source
Target
Deformed source
Credits: Rohr et al, TMI (2001)
CSE554
Deformation II
Slide 14
Thin-Plate Spline
• Advantages
– Smooth deformations, with physical analogy
– Closed-form solution
– Few free parameters (no tuning is required)
• Disadvantages
– Solving the equations still takes time (hence cannot perform “Interactive”
deformation)
CSE554
Deformation II
Slide 15
Free Form Deformation
• Uses a control lattice that embeds the shape
• Deforming the lattice points warps the embedded shape
Credits: Sederberg and Parry, SIGGRAPH (1986)
CSE554
Deformation II
Slide 16
Free Form Deformation
• Warping the space by
“blending” the deformation at
the control points
– Each deformed point is a weighted
sum of deformed lattice points
CSE554
Deformation II
Slide 17
Free Form Deformation
• Input
– Source lattice points: p1,…,pn
– Target lattice points: q1,…,qn
• Output
– A deformation function f[p] for any
p
f[p]
point p in the lattice grid:
n
f p
wi p qi
pi
i 1
qi
• wi[p]: determined by relative
position of p with respect to pi
CSE554
Deformation II
Slide 18
Free Form Deformation
• Desirable properties of the weights wi[p]
– Greater when p is closer to pi
• So that the influence of each control point is local
– Smoothly varies with location of p
• So that the deformation is smooth
–
1
n
i 1
p
n
i 1
f[p]
wi p
• So that f[p] =  wi[p] qi is an affine combination of qi
–
p
pi
qi
wi p pi
• So that f[p]=p if the lattice stays unchanged
CSE554
Deformation II
Slide 19
Free Form Deformation
• Finding weights (2D)
– Let the lattice points be pi,j for i=0,…,k and j=0,…,l
– Compute p’s relative location in the grid (s,t)
• Let (xmin,xmax), (ymin,ymax) be the range of grid
s
t
CSE554
px
xmax
py
ymax
xmin
xmin
p0,2
p1,2
p2,2
p3,2
p
t
p0,1
p1,1
p2,1
p3,1
p0,0
p1,0
p2,0
p3,0
s
ymin
ymin
Deformation II
Slide 20
Free Form Deformation
• Finding weights (2D)
– Let the lattice points be pi,j for i=0,…,k and j=0,…,l
– Compute p’s relative location in the grid (s,t)
– The weight wi,j for lattice point pi,j is:
ui,j p
wi,j p
i,j
ui,j p
k
i 0
l
j 0 ui,j
p1,2
Deformation II
p3,2
p0,1
p1,1
p2,1
p3,1
p0,0
p1,0
p2,0
p3,0
s
p
• B: Bernstein basis function: Bk
i s
p2,2
p
t
Bki s Blj t
• i,j: importance of pi,j
CSE554
p0,2
k
i
si 1
s
k i
Slide 21
Free Form Deformation
• Finding weights (2D)
– Weight distribution for one control point (max at that control point):
w1,1 p
p3,2
p3,1
p1,1
p0,2
p3,0
p0,1
p2,0
p1,0
p0,0
CSE554
Deformation II
Slide 22
Free Form Deformation
• A deformation example
CSE554
Deformation II
Slide 23
Free Form Deformation
• Registration
– Embed the source in a lattice
– Compute new lattice positions over the target
– Fitting term
• Distance between deformed source shape and the target shape
• Agreement of the deformed image content and the target image
– Distortion term
• Thin-plate spline energy
• Non-folding constraint
• Local rigidity constraint (to prevent stretching)
CSE554
Deformation II
Slide 24
Free Form Deformation
• Registration example
Deformed w/o rigidity
Source
Deformed with rigidity
Target
Credits: Loeckx et al, MICCAI (2004)
CSE554
Deformation II
Slide 25
Free Form Deformation
• Advantages
– Smooth deformations
– Easy to implement (no equation solving)
– Efficient and localized controls for interactive editing
– Can be coupled with different fitting or energy objectives
• Disadvantages
– Too many lattice points in 3D
– The lattice structure is not suitable for organic, non-cubical shapes
CSE554
Deformation II
Slide 26
Cage-based Deformation
• Use a control mesh (“cage”) to embed the shape
• Deforming the cage vertices warps the embedded shape
Credits: Ju, Schaefer, and Warren, SIGGRAPH (2005)
CSE554
Deformation II
Slide 27
Cage-based Deformation
• Warping the space by
“blending” the deformation at
the cage vertices
pi
p
n
f p
wi p qi
i 1
– wi[p]: determined by relative
position of p with respect to pi
f[p]
CSE554
Deformation II
qi
Slide 28
Cage-based Deformation
• Finding weights (2D)
– Problem: given a closed polygon (cage) with vertices pi and an interior
point p, find weights wi[p] such that:
• 1) 1
n
i 1
wi p
pi
• 2) p
n
i 1
wi p pi
p
• 3) They vary smoothly with p
CSE554
Deformation II
Slide 29
Cage-based Deformation
• Finding weights (2D)
– A simple case: the cage is a triangle
– The weights are unique, and are the barycentric coordinates of p
p1
w1 p
Areap,p2 ,p3
Areap1 ,p2 ,p3
p
p3
p2
CSE554
Deformation II
Slide 30
Cage-based Deformation
• Finding weights (2D)
– The harder case: the cage is an arbitrary (possibly concave) polygon
– The weights are not unique
• A good choice: Mean Value Coordinates (MVC)
• Can be extended to 3D
ui p
wi p
Tan
i
2
Tan
pi
pi-1
i 1
2
pi
p
αi
ui p
n
i 1 ui
p
p
αi+1
pi+1
CSE554
Deformation II
Slide 31
Cage-based Deformation
• Finding weights (2D)
– Weight distribution of one cage vertex in MVC:
wi p
pi
CSE554
Deformation II
Slide 32
Cage-based Deformation
• Application: character animation
– Using improved weights: Harmonic Coordinates
CSE554
Deformation II
Slide 33
Cage-based Deformation
• Registration
– Embed source in a cage
– Compute new locations of cage vertices over the target
• Minimizing some fitting and energy objectives
• Not seen in literature yet… future work!
CSE554
Deformation II
Slide 34
Cage-based Deformation
• Advantages over free form deformations
– Much smaller number of control points
– Flexible structure suitable for organic shapes
• Disadvantages
– Setting up the cage can be time-consuming
– The cage needs to be a closed shape
• Not as flexible as point handles
CSE554
Deformation II
Slide 35
Further Readings
• Thin-plate spline deformation
– “Principal warps: thin-plate splines and the decomposition of deformations”, by Bookstein (1989)
– “Landmark-Based Elastic Registration Using Approximating Thin-Plate Splines”, by Rohr et al.
(2001)
• Free form deformation
– “Free-Form Deformation of Solid Geometric Models”, by Sederberg and Parry (1986)
– “Extended Free-Form Deformation: A sculpturing Tool for 3D Geometric Modeling”, by
Coquillart (1990)
• Cage-based deformation
– “Mean value coordinates for closed triangular meshes”, by Ju et al. (2005)
– “Harmonic coordinates for character animation”, by Joshi et al. (2007)
– “Green coordinates”, by Lipman et al. (2008)
CSE554
Deformation II
Slide 36