Development and Dissemination of
Robust Brain MRI Measurement Tools
(1R01EB006733)
Dinggang Shen
IDEA
Department of Radiology and BRIC
UNC-Chapel Hill
Team
• UNC-Chapel Hill
- Dinggang Shen
- 1/2 Postdoctoral fellow(s)
• UPenn
- Christos Davatzikos
• GE
- Jim Miller
- Xiaodong Tao
Goal of this project
• To further develop HAMMER registration and
white matter lesion (WML) segmentation
algorithms, for improving their robustness and
performance.
• To design separate software modules for these two
algorithms and incorporate them into the 3D Slicer.
Overview of Our Brain Measurement Tools
• To further develop
HAMMER
registration and
WML segmentation
algorithms, for
improving their
robustness and
performance.
• To design separate
software modules
for these two
algorithms and
incorporate them
into the 3D Slicer.
Format
Converter
PACS
Database
Data importer
Data
processing
Skull Stripping
Multimodality
Registration
Tissue Classification
Data
processing
Learn Best
Features
Models
Complexity
Levels
MI
Q-MI
Skull Stripping
Intensity
Normalization
HAMMER
Deformation
Constraints
Parameter
Tuning
Registration
Tissue
Density Maps
ROI
Labeling
Manual
Segmentation
Voxel-wise
Segmentation
Training SVM
Classifier
False-Positive
Elimination
Application
Training
WML Atlas
SPM
Group
Analysis
ROI-based
Analysis
Applications
HAMMER Registration Algorithm
WML Segmentation Algorithm
Visualization
Engine
HAMMER
Matching attribute vectors
Image registration and
warping

Shen, et al., “HAMMER: Hierarchical Attribute Matching Mechanism for Elastic Registration”, IEEE
Trans. on Medical Imaging, 21(11):1421-1439, Nov 2002.
(2006 Best Paper Award, IEEE Signal Processing Society)
Registration – HAMMER
(1) Formulated as correspondence detection
Individual:
Model:
How can we detect correspondences?
Difficulty: High variations of brain structures
Solution: Use both global and local image
features to represent anatomical structures,
such as using wavelets or geometrical
moments.
 Xue, Shen, et al., “Determining Correspondence in 3D MR Brain Images Using Attribute Vectors as
Morphological Signatures of Voxels”, IEEE Trans. on Medical Imaging, 23(10): 1276-1291, Oct 2004.
Distinctive character of attribute vector:
toward an anatomical signature of every voxel
Brain A
Brain B
Similarity Map
Examples of attribute vector similarity maps, and point correspondences
HAMMER
(2) Hierarchical registration – reliable points first
To minimize the effect of local minima
Few driving voxels
Smooth approximation of
the energy function
Roots of sulci
Crowns of gyri
Voxels with
distinct
attribute
vectors.
Many driving voxels
Complete energy function
All boundary voxels
HAMMER
(2) Hierarchical registration – reliable points first
Beginning of registration
End of registration
158 brains we used to construct average brain
158 subjects
Average
Template
3D renderings
Model brain
A subject before warping and after warping
HAMMER in labeling brain structures:
Model
HAMMER
Subject
HAMMER
- Cross-sectional views
Model
Subject
Registration – HAMMER
- Label cortical surface
Inner
cortical
surface
Outer
cortical
surface
Model
Subject
Simulating brain deformations for validating registration methods
Template
Simulated

Xue, Shen, et al., “Simulating Deformations of MR Brain Images for Evaluation of Registration
Algorithms”, Neuroimage, Vol. 33: 855-866, 2006.
Successful applications of HAMMER:
10+ large clinical research studies and clinical
trials involving >8,000 MR brain images:
• One of the largest longitudinal studies of aging in the world to date,
(an 18-year annual follow-up of 150 elderly individuals)
• A relatively large schizophrenia imaging study (148 participants)
• A morphometric study of XXY children
• The largest imaging study of the effects of diabetes on the brain to date,
(650 patients imaged twice in a 8-year period)
• A large study of the effects of organolead-exposure on the brain
• A study of effect of sustained, heavy drinking on the brain
Improving: Learning Best Features for Registration
Best-scale moments:
Criteria for selecting best-scale moments of each point:
• Maximally different from those of its nearby points.
(Distinctiveness)
• Consistent across different samples. (Consistency)
• Best scales, used to calculate best-scale features,
should be smooth spatially. (Regularization)
Moments w.r.t. scales:

Wu, Qi, Shen, “Learning Best Features for Deformable
Registration of MR Brains”, MICCAI, 2005.
Improving: Learning Best Features for Registration
Results:
• Visual improvement:
Model
Ours
HAMMER’s
• Average registration error:
Histogram of deformation estimation errors
0.07
0.06
0.05
0.04
0.03
Improved method
HAMMER

Wu, Qi, Shen, “Learning-Based
Deformable Registration of MR Brain
Images”, IEEE Trans. Med. Imaging,
25(9):1145-1157, 2006.

Wu, Qi, Shen, “Learning Best Features
and Deformation Statistics for
Hierarchical Registration of MR Brain
Images”, IPMI 2007.
0.02
0.01
0
Error 2mm
0.66mm 0.95mm
Improving: Statistically-constrained HAMMER
HAMMER
Registration
Template
Statistical
Model of
Deformations,
using waveletPCA
Subject
Normal brain deformation
captured from 150 subjects

Xue, Shen, et al., “Statistical Representation of High-Dimensional Deformation Fields with Application to
Statistically-Constrained 3D Warping”, Medical Image Analysis, 10:740-751, 2006.
Improving: Statistically-constrained HAMMER
Results:
• More smooth deformations:
• Detection on simulated atrophy:
Comparison of Histograms of Jacobian Determinants
2.0%
HAMMER
SMD+HAMMER
Percentage
1.5%
1.0%
0.5%
0.0%
0
1
2
3
4
Jacobian Determinant
HAMMER
SMD+HAMMER
White Matter Lesion (WML)
Segmentation
WML Segmentation
• WMLs are associated with cardiac and vascular disease,
and may lead to different brain diseases, such as MS.
• Manual delineation
• Computer-assisted segmentation
-
Fuzzy-connection
Multivariate Gaussian Model
Atlas based normal tissue distribution model
KNN based lesion detection
• Lao, Shen, et al "Computer-Assisted Segmentation of White Matter Lesions in 3D MR images Using
Support Vector Machine", Academic Radiology, 15(3):300-313, March 2008.
Our approach
• Image property: serious intensity overlap in WMLs
T2
T1
WML
PD
FLAIR
Attribute Vector
• Attribute vector for each point v
FLAIR
PD
T2
T1
Neighborhood Ω (5x5x5mm)
F v   I tm  | tm  vm , m {T1 , T2 , PD, FLAIR}
• SVM
 To train a WML segmentation classifier.
• Adaboost  To adaptively weight the training samples
and improve the generalization of WML segmentation
method.
Overview of Our Approach
Co-registration
Manual Segmentation
Skull-stripping
Training SVM model via
training sample and Adaboost
Intensity normalization
Pre-processing
False positive
elimination
Post-processing
Training
Voxel-wise evaluation &
segmentation
Testing
Results
Results – 45 Subjects
10 for training, and 35 for testing
• Paired Spearman Correlation (SC)
Gold standard (rater 1)
Gold standard
(rater 1)
Rater 2
Computer
Mean+dev. of the
lesion volume
1.0
0.95
0.79
1494+/-3416 mm3
mm3
Rater 2
0.95
1.0
0.74
2839+/-6192
Computer
0.79
0.74
1.0
1869+/-3400 mm3
• Coefficient of variation (CV)
Coefficient
of Variation
Rater 1
189%
Rater 2
218%
Double
To investigate the variation of the lesion load’s
distribution of the 35 evaluated subjects
Defined as CV=/.
Close
Computer
182%
• Lao, Shen, et al "Computer-Assisted Segmentation of White Matter Lesions in 3D MR images Using
Support Vector Machine", Academic Radiology, 15(3):300-313, March 2008.
Improvement in this project
• Improve the robustness of multi-modality image
registration (for T1/T2/PD/FLAIR) by using a novel
quantitative and qualitative measurement for mutual
information, where salient points will be considered
more during the registration.
• Design region-adaptive classifiers, in order to allow each
classifier for capturing relative simple WML intensity
pattern in each region; we will also develop a WML atlas
for guiding the WML segmentation.
• Lao, Shen, et al "Computer-Assisted Segmentation of White Matter Lesions in 3D MR images Using
Support Vector Machine", Academic Radiology, 15(3):300-313, March 2008.
Conclusion
Further develop HAMMER registration and WML
segmentation algorithms  improve their
robustness and performance
3D Slicer
Thank you!
http://bric.unc.edu/IDEAgroup/
http://www.med.unc.edu/~dgshen/
IDEA