Speaker:James Chang

• Non-rigid surface registration, particularly registration of human face, finds a wide variety of application in computer vision and graphics.
• This study present a new automatic surface registration method utilized both attraction forces originating from geometrical and textural similarities, and stresses due to nonlinear elasticity of the surface.

3D
Reference
Surface
2D
Reference
Plane
3D Target
Surface
2D Target
Plane
Mapping Surfaces onto Image Planes
Registration via Triangular
Mesh
Deformation
Deformed
Surface
Image Matching with
Triangular Meshes
Nonlinear Elastic Deformation
Multiresolution Registration
Algorithm
result

Mapping Surfaces onto Image Planes

• Blanz and Vetter: A morphable model for the synthesis of 3-D faces.
Proc. SIGGRAPH’99, Los Angeles, USA,
187–194, 1999.
- Cylindrical projection: Project their vertices onto planes.
- Large distortions in the regions where the angle between the surface normal and the projection direction is large.
- Worse still, a bijective mapping is not guaranteed.

• Briefly the algorithms are categorized according to mapping distortion criterion that they minimize.
• Length-preserving, angle-preserving , arepreserving parameterization approaches.
• Least squares conformal mapping technique.(LSCM),angle-preserving.
B. Levy, S. Petitjean, N. Ray and J. Maillot. Least squares conformal maps for automatic texture atlas generation. ACM Transactions on Graphics, 21(3):362–
371, 2002. Proceedings of SIGGRAPH 2002.

Registration via Triangular Mesh Deformation in
2D Domain

• Lets denote the reference surface as A which is to be mapped to a target surface B .
• Their planar parameterizations yield 2D domains
• If we can find a mapping from to ,this leads to the mapping between the surfaces in
3D space.

• Image Matching with Triangular Meshes.
• Nonlinear Elastic Deformation.
• Multiresolution Registration Algorithm.

A mapping from to can be expressed via a vector field.
B A
Where denotes the 2D image coordinates and is a constant displacement

Image Matching with Triangular Meshes
• For each triangle of the reference mesh, we have a mapping function that maps a point in a triangle of the reference to the point in the target.
• This function interpolates the mapped coordinates of the triangle vertices by

Image Matching with Triangular Meshes
Where barycentric coordinates are obtained by
P1
P
P2 P3 is the area of triangle

Image Matching with Triangular Meshes
• An image matching energy , ,which accounts for the total square matching error over domain becomes
Where is the set of triangles, , is the domain of triangle , and represent multi-modal image values, and is a weighting matrix.

Image Matching with Triangular Meshes
• To exploit different surface attributes during matching , weighted sums of image attributes are used by means of diagonal weighting.
• The weights are determined according to both variance and importance of the channel.

Image Matching with Triangular Meshes
• The driving forces that deform a reference mesh to a target come from minimization of the image matching energy.
• This energy can be minimized by using the gradients at each mesh node with respect to its mapped coordinate s

Image Matching with Triangular Meshes
• The gradient at vertex is obtained through the chain rule as is the set of triangles connected to the node , is the .vertex of triangle that corresponds to node , and is thus the .barycenteric coordinate for the point .

Image Matching with Triangular Meshes
• The gradients are evaluated at each node of the mesh to update the mapping vector, , in a gradient decent scheme.
• The image gradients are evaluated using 3 X 3
Scharr mask.
J. Weickert and H. Scharr. A scheme for coherenceenhancing diffusion filtering with optimized rotation invariance.Jornal of Vis. Comm. Im. Rep., 13:103–118, 2002.
• We approximate the integrals in (4) and (5) by sampling at the recursively subdivided triangle centers.

Nonlinear Elastic Deformation

Nonlinear Elastic Deformation
• St. Venant Kirchoff material model.
M. Bro-Nielsen. Medical image registration and surgery simulation.IMM-DTU
Ph.D. Dissertation, 1996 .
• The potential energy for St. Venant Kirchoof material is given by
Where and are the Lame material constants.

Nonlinear Elastic Deformation
• Green-Lagrange strain tensor

Nonlinear Elastic Deformation
• We use minimization of the Green-Lagrange strain tensor to attain elasticity. The deformation energy is defined as the
Froebenius norm of this strain,

Nonlinear Elastic Deformation
• Analytic expressions for gradients of with respect to vertex coordinated mappings must be obtained for the energy minimization task.
• With triangular mesh elements nonlinearities are not too complex and can be solved directly with a gradient descent scheme.
• During this minimization, stresses at each node of the mesh deform its neighboring triangles to regularize the displacements due to forces generated by image mating errors.

Multiresolution Registration Algorithm.

Multiresolution Registration Algorithm.
• The total energy minimized over the reference triangulation T is
In this equation controls the amount of elasticity and thus the regularization during the registration
• We minimize the total energy via gradient descent methods in a coarse-to-fine approach .

Multiresolution Registration Algorithm.
• To solve the problem in different scales:
• First, Gaussian image pyramids are constructed one at the reference, the other at the target.
• Second, at each resolution, meshes are generated adapted to the scale of the pyramid level.

Multiresolution Registration Algorithm.
• Registration starts from the coarsest level , and the estimated deformations are subsequently transferred to a finer level .
• The deformation transfer from one mesh to another is realized by barycentric mapping .
• For each node of a finer level, its barycentric coordinates at the coarser mesh are calculated, and thus mapping of that node is obtained by a weighted sum of the previous mesh node value.

Multiresolution Registration Algorithm.
• To produce meshes:
• First, constrained Delaunay triangulation is performed . The boundary vertices , are used as constraints in this triangulation. Feature points can also be used as additional constraints.
J. D. Boissonnat, O. Devillers, S. Pion, M. Teillaud, and M. Yvinec.
Triangulations in CGAL. Comput. Geom. Theory Appl., 22:5–19, 2002.

Multiresolution Registration Algorithm.
• Second, a mesh generation algorithm which limits the maximum allowed triangle edge length is applied to the triangulation.
J. R. Shewchuk. Mesh generation for domains with small angles. Proc. 16th
Annu., AACM Sympos. Comput. Geom.,1-10, 2000.

Multiresolution Registration Algorithm.
• Energy minimization restarts in one higher resolution level initialized with the values of the previous step.
• This procedure continues until the finest and final deformation estimates are found.

Finest level
Multiresolution Registration Algorithm.
Coarsest level
Reference
Mesh deformation
Barycentric mapping target

Multiresolution Registration Algorithm.

• This algorithm is quite efficient.
• The number of iterations till convergence were in between 50 and 200 , and total registration duration changed from 3 to 7 seconds using 3.0GHz CoreDuo processor.

• A new automatic algorithm to solve the nonrigid registration problem of two surfaces is presented.
• Registration is realized by deforming one surface onto the other one after mapping the onto a plane.

• This is achieved by first using a bijective mapping , and then utilizing finite element method to obtain the matching deformation.
• The differences of geometric and texture related surface attributes generates deformation forces, while elastic stresses regularize the resulting deformations.

• Nunlinear elasticity is used to cope with the large deformations by minimizing Green-
Lagrange strain tensor .
• Also, following a multiresolution approach, good results can be obtained in a short time.

• Currently the algorithm is not designed to handle expression with open mouth.
• This team also plan to experiment using different surface attributes and investigate better strategies to combine these modalities.