AUTOMATIC SEGMENTATION OF
THALAMIC NUCLEI USING LABEL FUSION
Sep 9, 2014
Jason Su
Motivation
Segmentation of thalamic
nuclei is an important task
• Diseases and disorders like Multiple
Sclerosis and Essential Tremor have
effects on the thalamus
• Helpful for surgical guidance
Manual delineation takes 1-2
days to do manually
How can we make it
automatic?
1Tourdias
Manual segmentation of
thalamic nuclei using a
WMnMPRAGE acquisition
from a normal control.
et al. Neuroimage. 2013 Sep 7;84C:534-545. 2Niemann et al. Neuroimage. 2000 Dec;12(6):601-16.
Outline
Background and Tools
• Label fusion
• Nonlinear registration
• Automatic segmentation
Application to thalamic nuclei segmentation
• Using white matter nulled MPRAGE (WMnMPRAGE)
Label Fusion
Scenario: I have a structure manually
segmented by many different
radiologists/raters
Differing
Opinions
• How do I combine their differing
opinions into one solution?
Solution
?
STAPLE3 is the most widely adopted
standard algorithm
• Try to estimate the posterior
distribution by determining how
much you trust each rater
3Warfield et
al. IEEE Trans Med Imaging. 2004 Jul;23(7):903-21.
···
Easy approach: majority wins
Nonlinear Registration
Scenario: I want to compare how
subjects differ from each other at
every location in the brain.
• How can I align their brains so that
there is voxel-wise correspondence?
Easy approach: linear registration
Common algorithms:
• FSL’s FNIRT
• SPM8
• ANTS
Enables the creation of detailed
mean brain templates: MNI152
Linear
MNI152
Nonlinear
MNI152
Automatic Segmentation
Scenario: I have a structure
manually segmented by
one or more radiologists
across many subjects
• This was a lot of work!
• How can I do it automatically?
Easy approach: hope
there’s existing software
Idea: Labelfusion + Nonlinearregistration =
Automaticsegmentation
• Register our prior subjects to the
target subject, along with their
manual ROIs
• Prior ROIs are now an
automatically generated set of
candidate solutions
• Use label fusion to combine
these into a single automatic
segmentation
• Incorporate local registration
accuracy as a measure of trust
Application: Thalamic Nuclei
Automatically segment the whole thalamus and its nuclei
using a library of manual-defined ROIs
Compare the performance of some existing label fusion
algorithms
Assess its accuracy against the manual truth with the Dice
coefficient and center of mass displacement
Background
White matter nulled MPRAGE
(WMnMPRAGE) at 7T shows
remarkable contrast in the
thalamus
Enabled detailed delineation of
16 thalamic nuclei guided by the
Morel atlas1,2
Manual segmentation of
thalamic nuclei using a
WMnMPRAGE acquisition
from a normal control.
1Tourdias
et al. Neuroimage. 2013 Sep 7;84C:534-545. 2Niemann et al. Neuroimage. 2000 Dec;12(6):601-16.
Scanning Methods
WMnMPRAGE data are from two
different studies with varying protocols
• 7T, 32ch head coil, 1mm3 isotropic, TS
6000ms, TI 680ms, TR 10ms, α 4°, BW 12
kHz
Nuclei
Labeled
6 controls
8 controls and
15 MS patients
16
13
ARC
Trajectory
Time
(min)
No
1D centric
16
2D centric,
1.5x1.5 radial fan
beam
5.5
29 total subjects with 14 controls, 15
patients
• 20 (9C: 11P) used as our atlas of priors
• 9 (5C:4P) reserved for validation testing
Fully Sampled
16min
Accelerated
5.5min
Processing Methods: Label Fusion
• Dice overlap is the most
important performance
measure
2| A B |
D
| A|  | B |
• ANTS is the premier registration
tool for this application
4Wang, H.
5Cardoso
and Yushkevich, PA. Front Neuroinform. 2013 Nov 22;7:27
et al. Med Image Anal. 2013 Aug;17(6):671-84.
Processing Methods: Label Fusion
MICCAI 2012 Multi-Atlas
Grand Challenge Ranking
1.
2.
3.
4.
5.
6.
4Wang, H.
5Cardoso
PICSL_BC
NonLocal STAPLE
MALP_EM
PICSL_Joint
MAPER
STEPS
and Yushkevich, PA. Front Neuroinform. 2013 Nov 22;7:27
et al. Med Image Anal. 2013 Aug;17(6):671-84.
Processing Methods: Label Fusion
Apply published and publically
available solutions:
MICCAI 2012 Multi-Atlas
Grand Challenge Ranking
1.
2.
3.
4.
5.
6.
4Wang, H.
5Cardoso
PICSL_BC
NonLocal STAPLE
MALP_EM
PICSL_Joint
MAPER
STEPS
and Yushkevich, PA. Front Neuroinform. 2013 Nov 22;7:27
et al. Med Image Anal. 2013 Aug;17(6):671-84.
Majority voting
PICSL MALF4 (UPenn)
STEPS5 (UCL)
Processing Methods: Registration
A mean brain template was created
with ANTS from 17 (6C:11P)
subjects in the MS study group
• N4 bias field correction used to compensate
for some B1- and B1+ inhomogeneity
Convergence after 16 iterations
(1wk on 12-core CPU)
Axial slice from the 1mm
isotropic resolution
WMnMPRAGE template
formed from 17 subjects
• Cortical registration challenging, as usual
• Preserves excellent detail in the thalamus
Pipeline
Prior subject 1:
WMnMPRAGE
13 nuclei ROIs
Prior subject 2:
WMnMPRAGE
13 nuclei ROIs
Mean WMnMPRAGE
Template
Label Fusion
Majority
Voting
ANTS
ANTS
Target subject:
WMnMPRAGE
···
Prior subject 20:
WMnMPRAGE
13 nuclei ROIs
STEPS
PICSL
MALF
• Not using direct nonlinear registration
– Subjects are registered to one another via the
template
Pipeline
Prior subject 1:
WMnMPRAGE
13 nuclei ROIs
Prior subject 2:
WMnMPRAGE
13 nuclei ROIs
···
Prior subject 20:
WMnMPRAGE
13 nuclei ROIs
Mean WMnMPRAGE
Template
Label Fusion
Majority
Voting
ANTS
ANTS
Target subject:
WMnMPRAGE
STEPS
PICSL
MALF
• Warp priors to the template, then apply the
inverse warp to reach the target subject
Pipeline
Prior subject 1:
WMnMPRAGE
13 nuclei ROIs
Prior subject 2:
WMnMPRAGE
13 nuclei ROIs
···
Prior subject 20:
WMnMPRAGE
13 nuclei ROIs
Mean WMnMPRAGE
Template
Label Fusion
Majority
Voting
ANTS
ANTS
Target subject:
WMnMPRAGE
STEPS
PICSL
MALF
• Reduces 20 nonlinear registrations to only 1
• 3-4 hr/registration on a 4-core CPU
Processing Methods: STEPS Optimization
Pul
STEPS has control
parameters that can be
optimized
Use cross-validation to
grid search over the
parameter space
• σ, the Gaussian kernel
size
• Measures local
registration quality in
a window with
normalized crosscorrelation
• X, the number of locally
well-registered priors to
use
• β, Markov random field
regularization (not
optimized, set to 0)
• 29 data sets split into
20 for training and 9 for
testing
• Maximize the mean
Dice overlap for each
ROI
• 44,200 total calls of
STEPS
• 20 hours on Stanford
Sherlock Cluster
(sherlock.stanford.edu)
MTT
Validation
With the three label fusion algorithms, validate the
approach using the test data
• Produce automatic segmentations in 9 subjects using manual priors
from 20 others
Evaluate the automatic technique vs. manual tracing
• Displacement distance of the centers of mass between the
predicted and truth labels
• Dice overlap
Results
Whole thalamus and
nuclei segmentations
Automatic result as
filled region
Manual truth as
yellow outline
Overlaid in an MS
patient.
See [1] for the
abbreviation glossary
Probabilistic segmentation from a
DTI-based method6
1Tourdias
6Mang
et al. Neuroimage. 2013 Sep 7;84C:534-545.
et al. MRM. 2012 Jan;67(1):118-26.
Results: Dice Overlap
Results: Dice Overlap
Results: Dice Overlap
Results: Dice Overlap
Results: Center of Mass
Results: Center of Mass
Results: Center of Mass
Results: Center of Mass
Median PICSL
Mean
Median
ΔCoM Median
Dice in [6] Dice in [7]
(mm)
Dice
Whole
Thalamus
AV
VA
VLa
VLP
VPL
Pul
LGN
MGN
CM
MD
Hb
MTT
0.328
0.924
N/A
N/A
0.741
0.813
1.292
0.825
1.187
0.639
0.617
0.331
0.448
0.519
0.277
0.999
0.786
0.689
0.607
0.782
0.649
0.871
0.730
0.705
0.783
0.871
0.797
0.714
0.726
N/A
N/A
N/A
N/A
N/A
0.725
0.405
0.515
N/A
N/A
N/A
N/A
0.874
N/A
0.814
N/A
N/A
N/A
0.771
N/A
N/A
Performance of PICSL MALF compared to a
DTI and multi-modal techniques
6Ye
et al. Proc SPIE. 2013 Mar 13;8669
et al. Proc IEEE Int Symp Biomed Imaging. 2013:852-855.
7Stough
Discussion
STEPS is underperforming despite optimization efforts
We achieve ≈1mm accuracy in estimating the center of mass for most
nuclei
Whole thalamic segmentation is excellent
Nuclei segmentation at this performance is a starting point to reduce
manual editing
• A couple are good enough to be for completely automatic segmentation (Pul and MD)
• Processing time for a new subject is dominated by the registration,
3-4 hours
Future Work
Correction of label fusion using machine learning
has been highly successful in other anatomies8
PICSL MALF optimization
Comparison of indirect template vs. direct
registration performance
8Yushkevich
et al. Neuroimage. 2010 Dec;53(4):1208-24.
Novelty
This is the first work to do many things:
• A detailed automatic segmentation for 13 thalamic
nuclei
• All evaluated against ground truth manual
segmentation
• Most previous works used DTI with unsupervised
learning methods (clustering)
• Here we’ve shown higher resolution with a faster
acquisition and better segmentation performance
Questions?
• Slides available at
http://mr.jason.su/
• Special thanks to Manoj, Thomas, and
Stanford Research Computing Facility
PICSL with SegAdapter
(AdaBoost)
Results: Dice Overlap
Results: Dice Overlap
Results: Dice Overlap
Results: Dice Overlap
Results: Center of Mass
Results: Center of Mass
Results: Center of Mass
Results: Center of Mass
STEPS with SegAdapter
(AdaBoost)
Results: Dice Overlap
Results: Dice Overlap
Results: Dice Overlap
Results: Dice Overlap
Results: Center of Mass
Results: Center of Mass
Results: Center of Mass
Results: Center of Mass
Old Versions
Results: Dice Overlap
Results: Center of Mass