An Integrated Pose and
Correspondence Approach to Image
Matching
Anand Rangarajan
Image Processing and Analysis Group
Departments of Electrical Engineering and Diagnostic Radiology
Yale University
Motivation I
• Human Brain Mapping:
– Different subjects.
• Statistical analysis.
• Normal vs. abnormal.
– Different times.
• Detect significant change, help diagnosis.
– Different modalities.
• Combine complementary information.
Motivation II
• Difficulty :
– Variability in pose, size, shape and acquisition.
• Brain registration :
– Common coordinate frame.
– Data comparable.
– Quantitative analysis.
Results
Interactive 3D Sulcal Tracing
Overview
• Extract features:
– Sulcal traces represented as point sets.
– Labeling, ordering information [optional].
• Jointly solve feature correspondence and
spatial mapping.
Overview II
• Part II: Information Analysis:
– Measurements.
– Learn from the data, construct statistical
models.
• e.g., probabilistic atlas for structures / functions.
– Make inference for new data based on the
learned models.
• e.g., automated sulcal labeling, segmentation, computer aided
diagnosis.
Outline
• Related work.
• The approach.
– Point-based representation of sulci.
– Robust point matching algorithm.
• Results and examples.
• Future work.
Other Work in Brain Registration
• Voxel-based methods:
– Volumetric Warping: Christensen et al., Gee et
al., Collins et al.
• Feature-based methods:
–
–
–
–
Landmarks: Bookstein.
Curves: Sandor and Leahy, Collins et al.
Surfaces: Thompson et al., Davatzikos et al.
Sulcal Graphs: Lohmann and von Cramon.
Approach Rationale
• Voxel intensity matching does not ensure
that corresponding sulci indeed match.
• Landmarks hard to define.
• Extraction, representation and matching of
cortical curves / surfaces / graphs is
difficult.
Our Approach
Point-based Representation
• Hundreds of points, statistically more robust
than just a few landmarks.
• Additional information can be used:
– Major sulcal labels.
• Further analyses made easy:
– Procrustes mean.
– Eigen-analysis of the error covariance matrix.
Our Approach
Robust Point Matching (RPM)
• Estimation :
– Correspondence and spatial mapping.
• Softassign:
– Soft correspondence.
– Allows partial matching, noise.
– Less sensitive to local minima.
• Handles outliers.
Robust Point Matching
Alternating Optimization
• When correspondence M is known, standard
least squares solution for spatial mapping A.
• When spatial mapping A is fixed,
assignment solution for correspondence M.
– Softassign - soft correspondence.
– Deterministic Annealing - temperature T.
Robust Point Matching
Energy Function
Robust Point Matching
Step I. Solve Spatial Mapping
• Given correspondence M, find the optimal
spatial mapping A (affine):
• Standard least-squares solution.
• Gradually relaxed regularization on l.
Robust Point Matching
Part II. Softassign
• Given spatial mapping A, solve the Linear
Assignment Problem:
subject to
Robust Point Matching
Positivity
M ij =exp( b Qij )
Two-way constraints
Row Normalization
M ij
M ij
Si M ij
Col. Normalization
M ij
M ij
Sj M ij
Step II. Softassign
•Step I: Mij = exp ( - Qij/T).
•Step II: Double Normalization.
Sinkhorn’s Algorithm.
Outlier rejection using slack
variables.
Robust Point Matching
Part II. Softassign
• Deterministic Annealing :
– T as an extra parameter.
– F = Eassign - TS =
• Gibbs Distribution :
– Positivity ganranteed.
– High T, insensitive to Q, uniform M .
– Low T, sensitive to Q, binary M .
Robust Point Matching
Algorithm Summary
• Start: uniform M, high temperature T.
• Do until final temperature is reached.
– Given M, solve for spatial mapping A.
– Given A, use Softassign to update M.
• Decrease temperature.
Experiment on Brain Sections
Results of Method
Results
Interactive 3D Sulcal Tracing
Results
RPM Example
Two labeled sulcal point sets, initial position.
RPM without label information
Results
Visual Matching Comparison
Results
Visual Matching Comparison
Quantitative Comparison
Quantitative Comparison
Future Work
• Error measure on the entire volume.
• Fully non-rigid 3D spatial mapping.
– Thin-plate spline and correspondence.
•
•
•
•
Automated sulcal extraction, Zeng et al.
Investigate partially labeled case.
Automated labeling.
Atlas construction.
The End
Thin-plate-spline Implementation
Thin-plate-spline Implementation
Results
Visual Matching Comparison
TPS