Multilevel Segmentation and Integrated Bayesian Model
Classification with an Application to Brain Tumor Segmentation
J. J. Corso, E. Sharon, and A. Yuille. Medical Imaging Informatics and Dept. of Statistics
University of California, Los Angeles
Main Contribution: A mathematical formulation for incorporating generative models into the calculation of affinities in graph-based
segmentation. We extended the Segmentation by Weighted Aggregation algorithm using the proposed model-aware affinities. The
implementation is applied to brain tumor segmentation; it is very accurate and extremely efficient (2 minutes for a 256x256x25 volume).
I Segmentation Background
II Bayesian Model-Aware Affinity
Graph-based Segmentation
Define binary random variable to capture probabilistic region membership:
Define the problem on a graph:
Augment with node statistics
or
and class
.
Incorporate models as hidden variables and marginalize over them:
Affinities measured on each edge:
Model Evidence Weights
Model Specific Affinity
brain
Goal: find cuts that minimize criterion:
tumor
edema
Generative Model-based Segmentation
Define likelihood and prior:
E
N
A
N
Goal: maximize posterior:
Model specific affinity is a function of the class variables.
For example: class dependent weighted distance:
Grammatical Relations
III Segmentation by Weighted Aggregation
IV Generative Models and Class Likelihood
Example
Efficient multiscale process based on algebraic multigrid.
Approximates the normalized cut criterion .
Class label is associated with each node, a by-product of model-aware affinity.
Standard SWA gives a segmentation hierarchy but it does not say which elements to select; our integrated model classification provides a selection.
Currently use i.i.d. models: mixture of Gaussians over 4D multichannel image.
Edema
Tumor
Model-aware affinities modulate multiscale process:
Original SWA
SWA with Models
Gives Output in
Multiple Scales
V Application to Brain Tumor
Important to quantify tumor size.
T1
T2
FLAIR
T1CE
T1
Channel weights on the model specific affinity; set from domain knowledge.
Enhancement Necrosis
T1
T1 with C ontrast
FLAIR
State of the art is manual.
20 Studies: Glioblastoma Multiforme.
Debris
Highly varying in appearance,
size, shape and location.
Ring
Edema
Use of multiple MR channels needed.
1
2
ND, ND
1/3
1/3
1/3
3
ND, Brain
0
1/2
1/2
4
5
Brain , T umor
0
1
0
6
7
(a)
(b)
All processing in 3D.
Typical Example:
(c)
Hierarchy
Example:
(d)
T1
T2
FLAIR
T1CE
T1 w/ Contrast
Flair
T2
Algorithm
Single Voxel Classifier
Saliency-Based Extractor
Model-Based Extractor
Algorithm
Single Voxel Classifier
Saliency-Based Extractor
Model-Based Extractor
Results on Training Set
Tumor
Jac
Prec
Rec
42%
48%
85%
44%
51%
64%
62%
75%
81%
Results on Testing Set
Tumor
Jac
Prec
Rec
49%
55%
81%
48%
61%
63%
66%
80%
79%
Jac
43%
47%
54%
Edema
Prec
Rec
49%
78%
55%
76%
66%
72%
Jac
56%
56%
61%
Edema
Prec
Rec
66%
76%
66%
71%
78%
71%
Extremely efficient processing; of a 256 x 256 x 25 volume
full segmentation and classifcation in about 2 minutes.
Brain , Edema
0
0
1
8
9
T umor, E dema
0
1/2
1/2
10
11