Colon Gland Segmentation with
Deep Convolutional Neural Networks and
Total Variation Segmentation
Team vision4GlaS
Philipp Kainz1,2, Michael Pfeiffer2, and Martin Urschler3,4
1
Institute of Biophysics, Medical University of Graz, Graz, Austria
Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland
3 Institute for Computer Graphics and Vision, Graz University of Technology, Graz, Austria
4 Ludwig Boltzmann Institute for Clinical Forensic Imaging, Graz, Austria
2
philipp.kainz@medunigraz.at
Oct. 5, 2015
Page 1
Colon Gland Segmentation Algorithm Overview
Color Deconvolution
Hematoxylin-Eosin
Contrast Limited Adaptive
Histogram Equalization
CLAHE
Learn
Glandular Objects
Learn
Separator Structures
OBJECT NET
SEPARATOR NET
Combine Model
Predictions
Oct. 5, 2015
Preprocessing
Pixel
Probabilities
Total Variation Segmentation (TVseg)
Active Contour Energy Minimization
Learning & Prediction
Postprocessing
Colon Gland Segmentation with Deep Convolutional Neural Networks and Total Variation Segmentation, P. Kainz et al.
Page 2
Image Preprocessing: Tissue Decomposition
Color Deconvolution [Ruifrok and Johnston, 2001]
red channel
source image (RGB)
deconvolved image (RGB)
green channel
Contrast Limited Adaptive
Histogram Equalization
[Zuiderveld, 1994]
blue channel
red channel, CLAHE
Oct. 5, 2015
Colon Gland Segmentation with Deep Convolutional Neural Networks and Total Variation Segmentation, P. Kainz et al.
Page 3
Learning Glandular Objects
C0: background benign
C1: gland benign
4-Class Classification
OBJECT NET
C2: background malignant
C3: gland malignant
predicts the label of the
center pixel in a
101x101 px patch
Deep Convolutional Neural Network [LeCun et al., 2010]
C1: 80@91x91
C2: 96@40x40
C3: 128@16x16
C4: 160@6x6
F5: 1024
S1: 80@46x46
Input: 101x101
S3: 128@8x8
F6: 512
Output: 4
convolutions
11x11
Oct. 5, 2015
S2: 96@20x20
max-pooling
2x2
convolutions
7x7
max-pooling
2x2
convolutions
5x5
max-pooling
convolutions
2x2
3x3
Colon Gland Segmentation with Deep Convolutional Neural Networks and Total Variation Segmentation, P. Kainz et al.
full connection
Page 4
Predicting Glandular Objects
input image
annotation image
Oct. 5, 2015
background benign
background malignant
gland benign
gland malignant
Colon Gland Segmentation with Deep Convolutional Neural Networks and Total Variation Segmentation, P. Kainz et al.
Page 5
Learning Separator Structures
Binary Classification
SEPARATOR NET
predicts the label of the
center pixel in a
101x101 px patch
C1: 64@93x93
C2: 96@41x41
C3: 128@17x17
C4: 160@7x7
F5: 1024
S1: 64@47x47
Input: 101x101
S3: 128@9x9
F6: 512
Output: 2
convolutions
9x9
Oct. 5, 2015
S2: 96@21x21
max-pooling
2x2
convolutions
7x7
max-pooling
2x2
convolutions
5x5
max-pooling
2x2
convolutions
3x3
Colon Gland Segmentation with Deep Convolutional Neural Networks and Total Variation Segmentation, P. Kainz et al.
full connection
Page 6
Predicting Separator Structures
input image
annotation image
Oct. 5, 2015
non-separator structures
separator structures
Colon Gland Segmentation with Deep Convolutional Neural Networks and Total Variation Segmentation, P. Kainz et al.
Page 7
Combining Model Predictions
background probability map
background benign
background
background malignant
separator
structures
gland benign
gland
gland malignant
Oct. 5, 2015
foreground probability map
Colon Gland Segmentation with Deep Convolutional Neural Networks and Total Variation Segmentation, P. Kainz et al.
Page 8
Total Variation Segmentation
Based on a convex geodesic active contour model, a figure-ground
segmentation u is computed:
background probability map
input I(x)
segmentation u
foreground probability map
Oct. 5, 2015
Globally optimal solution giving minimal contour length
Colon Gland Segmentation with Deep Convolutional Neural Networks and Total Variation Segmentation, P. Kainz et al.
Page 9
Segmentation Results: Test Set A (off-site)
Oct. 5, 2015
Colon Gland Segmentation with Deep Convolutional Neural Networks and Total Variation Segmentation, P. Kainz et al.
Page 10
Segmentation Results: Test Set A (off-site)
Oct. 5, 2015
Colon Gland Segmentation with Deep Convolutional Neural Networks and Total Variation Segmentation, P. Kainz et al.
Page 11
Thanks for your attention!
Team vision4GlaS
Philipp Kainz1,2, Michael Pfeiffer2, and Martin Urschler3,4
1
Institute of Biophysics, Medical University of Graz, Graz, Austria
Institute of Neuroinformatics, University of Zurich and ETH Zurich, Zurich, Switzerland
3 Institute for Computer Graphics and Vision, Graz University of Technology, Graz, Austria
4 Ludwig Boltzmann Institute for Clinical Forensic Imaging, Graz, Austria
2
philipp.kainz@medunigraz.at
Oct. 5, 2015
Page 12
References
[Ruifrok and Johnston, 2001]
A. C. Ruifrok and D. A. Johnston. Quantification of histochemical staining by color deconvolution. Anal Quant
Cytol Histol, 23(4):291–299, August 2001.
[Zuiderveld, 1994]
K. Zuiderveld. Contrast limited adaptive histogram equalization. In Graphics gems IV, pages 474–485.
Academic Press Professional, Inc., 1994.
[LeCun et al., 2010]
Y. LeCun, K. Kavukcuoglu, and C. Farabet. Convolutional networks and applications in vision. In ISCAS, pages
253–256, May 2010.
[Hammernik et al., 2015]
K. Hammernik, T. Ebner, D. Stern, M. Urschler, and T. Pock. Vertebrae segmentation in 3D CT images based
on a variational framework. In Recent Advances in Computational Methods and Clinical Applications for Spine
Imaging, pages 227–233. Springer, 2015.
[Bresson et al., 2007]
X. Bresson, S. Esedoglu, P. Vandergheynst, J.-P. Thiran, and S. Osher. Fast global minimization of the active
contour/snake model. J Math Imaging Vis, 28(2):151–167, 2007.
[Chambolle and Pock, 2011]
A. Chambolle and T. Pock. A first-order primal-dual algorithm for convex problems with applications to imaging.
J Math Imaging Vis, 40(1):120–145, 2011.
Oct. 5, 2015
Colon Gland Segmentation with Deep Convolutional Neural Networks and Total Variation Segmentation, P. Kainz et al.
Page 13