IDEA
LINKS: Learning-based multi-source IntegratioN
frameworK for Segmentation of infant brain images
Li Wang, Yaozong Gao, Feng Shi, Gang Li, Dinggang Shen
Presented by Li Wang
09-18-2014
Department of Radiology and BRIC, UNC-Chapel Hill
Content

Motivation
 Proposed method
 Experimental results
 Conclusion
Department of Radiology and BRIC, UNC-Chapel Hill
Motivation
T1
T2
FA
Manual segmentation
Fractional anisotropy
(FA) was calculated
from Diffusion MRIs.
2-weeks
6-months
12-months
Limitations of multi-atlas label fusion
1. nonlinear registrations
2. simple intensity patch
3. equal weight for different modality
Our proposed work will
1. linear registrations
2. appearance features and context features
3. adaptive weights for different modality
Department of Radiology and BRIC, UNC-Chapel Hill
Flowchart of our proposed work
T1
T2
FA
Appearance
features
Ground
truth
Random
forests
Classifier 1
Haar-like featuresContext
Classifier 2
features
Appearance
features
Context
features
Appearance
features
Feature vectors
Classifier τ
Sequence classifier
Probability maps
Department of Radiology and BRIC, UNC-Chapel Hill
Result of an unseen target subject
T1
T2
FA
Original images
Iteration 1
Iteration 2
Iteration 10
Ground truth
Department of Radiology and BRIC, UNC-Chapel Hill
Post-processing: Anatomical constraint
To deal with the possible artifacts due to independent voxel-wise classification, we use patch-based
sparse representation to impose an anatomical constraint [1] into the segmentation.
Ground truth of
training images
Probabilities of
training image by
the random forest
𝛼1
𝛼𝑖
Probabilities of
target image by
the random forest
Without anatomical
With anatomical
Ground truth
1. Wang, L., Shi, F., Gao, Y., Li, G., Gilmore, J.H., Lin, W., Shen, D., 2014. Integration of sparse multi-modality representation and anatomical
constraint for isointense infant brain MR image segmentation. NeuroImage 89, 152-164.
Department of Radiology and BRIC, UNC-Chapel Hill
Dataset



Dataset 1: UNC 119 infants consisting of 26, 22, 22, 23, and 26
subjects at 0-, 3-, 6-, 9- and 12-months of age, respectively.
Dataset 2: NeoBrainS12 MICCAI2012 Challenge.
Dataset 3: SATA MICCAI2013 Challenge.
Department of Radiology and BRIC, UNC-Chapel Hill
Importance of the context features
Iterations
Iterations
Iterations
Department of Radiology and BRIC, UNC-Chapel Hill
Importance of the multi-source
Department of Radiology and BRIC, UNC-Chapel Hill
Dataset 1: UNC 119 infants
(a)
(b)
(c)
(d)
(e)
(f)
Majority voting (MV)
Nonlocal label fusion [1]
Atlas forest [2]
Patch-based sparse labeling [3]
Proposed1 (Random forest)
Proposed2 (Random forest + Anatomical constraint)
1. Coupé, P., Manjón, J., Fonov, V., Pruessner, J., Robles, M., Collins, D.L., 2011. Patch-based segmentation using expert priors: Application to
hippocampus and ventricle segmentation. NeuroImage 54, 940-954.
2. Zikic, D., Glocker, B., Criminisi, A., 2013. Atlas Encoding by Randomized Forests for Efficient Label Propagation. MICCAI 2013, pp. 66-73.
3. Wang, L., Shi, F., Gao, Y., Li, G., Gilmore, J.H., Lin, W., Shen, D., 2014. Integration of sparse multi-modality representation and anatomical
constraint for isointense infant brain MR image segmentation. NeuroImage 89, 152-164.
Department of Radiology and BRIC, UNC-Chapel Hill
Slice comparisons
T1
T2
FA
Ground truth
Segmentation
Difference
maps with the
ground truth
(a) Majority voting
(b) Nonlocal
label fusion
(c) Atlas forest
(d) Patch-based
(e) Proposed1 (f) Proposed2
sparse labeling
Department of Radiology and BRIC, UNC-Chapel Hill
Inner surface comparisons
(a) Majority voting
(b) Nonlocal
label fusion
(c) Atlas forest
(d) Patch-based
sparse labeling
(e) Proposed1
(f) Proposed2
(g) Ground truth
Department of Radiology and BRIC, UNC-Chapel Hill
Quantitative measurement
Methods
MV
Time cost
0
3
WM 6
9
12
0
3
GM 6
9
12
0
3
CSF 6
9
12
1h
81.6±0.28
76.6±1.48
80.1±0.83
79.2±0.98
82.5±1.05
78.6±1.02
77.3±1.42
79.9±1.04
83.6±0.69
84.9±1.01
76.6±1.57
80.6±1.55
71.2±0.71
68.7±1.27
65.2±3.69
Nonlocal
Patch-based
Atlas Forest
label fusion
Sparse labeling
1.2h
12m
2h
89.0±0.74 88.9±0.60
89.7±0.59
85.0±1.21 85.1±1.33
85.3±1.71
83.6±0.80 82.1±0.91
84.2±0.78
86.1±2.00 84.2±1.34
87.1±1.89
88.6±1.22 87.2±1.29
90.3±1.42
85.1±0.78 87.1±0.76
86.7±0.81
83.4±0.78 85.5±1.12
85.3±0.51
83.9±0.83 83.1±0.93
84.8±0.77
88.1±0.75 87.4±0.66
87.4±0.54
89.3±0.90 88.8±1.02
88.9±0.57
80.2±1.87 77.7±4.52
76.1±2.59
84.1±1.88 82.4±2.17
80.1±1.10
79.2±1.69 86.7±1.16
83.0±0.77
80.6±2.40 84.1±1.57
81.0±2.27
81.5±1.66 83.6±1.83
81.7±2.59
Proposed1
Proposed2
5m
91.7±0.64
88.8±1.09
86.4±0.79
89.0±0.78
90.7±0.74
89.6±0.66
88.1±1.00
88.2±0.77
90.0±0.49
90.3±0.74
83.9±2.20
83.7±1.52
92.7±0.63
85.8±1.53
84.1±1.90
1.8h
92.1±0.62
89.1±0.95
87.9±0.68
89.4±0.56
91.8±0.65
90.8±0.42
88.3±0.90
89.7±0.59
90.3±0.54
90.4±0.68
84.2±2.02
85.4±1.49
93.1±0.55
86.7±1.09
85.2±1.69
Department of Radiology and BRIC, UNC-Chapel Hill
Dataset 2: NeobrainS12 MICCAI Challenge
 2 training images with the manual segmentations.
 3 target images for testing.
Department of Radiology and BRIC, UNC-Chapel Hill
Our results of 3 target images
Department of Radiology and BRIC, UNC-Chapel Hill
Quantitative measurement
Table 1. Dice ratios (DC) and modified Hausdorff distance (MHD) of different
methods on NeoBrainS12 MICCAI Challenge data.
(Bold indicates the best performance)
WM
Team Name DC MHD
UNC-IDEA 0.92 0.35
Imperial 0.89 0.70
Oxford
0.88 0.76
UCL
0.87 1.03
UPenn
0.84 1.79
CGM
DC MHD
0.86 0.47
0.84 0.73
0.83 0.61
0.83 0.73
0.80 1.01
BGT
DC MHD
0.92 0.47
0.91 0.8
0.87 1.32
0.89 1.29
0.8 4.18
BS
DC MHD
0.83 0.9
0.84 1.04
0.8 1.24
0.82 1.3
0.74 1.96
CB
DC MHD
0.92 0.5
0.91 0.7
0.92 0.63
0.9 0.92
0.91 0.85
CSF
Placed
DC MHD
0.79 1.18
1
0.77 1.55
2
0.74 1.82
3
0.73 2.06
4
0.64 2.46
5
http://neobrains12.isi.uu.nl/mainResults_Set1.php
Department of Radiology and BRIC, UNC-Chapel Hill
Dataset 3: SATA MICCAI2013 Challenge
 35 training images with the 14 ROIs in subcortical regions.
 12 target images for testing.
Department of Radiology and BRIC, UNC-Chapel Hill
Our results on one target image
Department of Radiology and BRIC, UNC-Chapel Hill
Quantitative measurement
Table 2. Dice ratios (DC) and Hausdorff distance (HD) of different methods on
SATA MICCAI Challenge data.
Team Name
Submission
Date/Time
Mean (Median)
DSC
Mean (Median) Hausdorff
Distance (mm)
UPENN_SBIA_MAM
12-Jul-2013
0.8686 (0.8772)
3.3043 (3.1006)
PICSL
02-Jul-2013
0.8663 (0.8786)
3.5381 (3.2369)
LINKS
04-May-2014
0.8613 (0.8722)
3.6453 (3.3637)
deedsMIND
12-Jul-2013
0.8402 (0.8573)
4.1027 (3.8983)
MSRC_AF_NEW
18-Feb-2014
0.8247 (0.8392)
3.8437 (3.6799)
MSRC_AF_NEW_STAPLE
18-Feb-2014
0.8063 (0.8169)
4.6494 (4.3760)
deedsMIND no marginals
15-Jul-2013
0.7216 (0.7539)
6.1614 (5.5120)
http://masi.vuse.vanderbilt.edu/submission/leaderboard.html
Department of Radiology and BRIC, UNC-Chapel Hill
Conclusion


We have presented a learning-based method (LINKS) to
effectively integrate multi-source images and the tentatively
estimated tissue probability maps for infant brain image
segmentation.
Experimental results on 119 infant subjects and MICCAI grand
challenge show that the proposed method achieves better
performance than other state-of-the-art automated segmentation
methods.
Department of Radiology and BRIC, UNC-Chapel Hill

Thanks for your attention!
http://www.unc.edu/~liwa/
Google scholar
Department of Radiology and BRIC, UNC-Chapel Hill