Supporting Phenotyping through
Visualization and Image Analysis
Raghu Machiraju,
Computer Science & Engineering,
Bio-Medical Informatics
The Ohio State University
About Myself
 Associate Professor, Computer Science and
Engineeering, BioMedical Informatics
 7th Year at OSU
 Research Interests – Imaging, Graphics and
Visualization
 Notable Points
 Co-Chair of Visualization 2008 Conference, Columbus OH
 Alumni in video gaming/animation industry (Pixar, EA), National
Government Labs (Lawrence Livermore), Industrial Research
(Samsung, IBM, Mitsubishi Electric), Medical Schools (Harvard
Medical School)
Research Activities
Medical, Biological Imaging and Visualization
 Optical Microscopy
 In-vivo, fluorescence imaging
 Structural/Functional Magnetic Resonance Imaging
 Diffusion Tensor Imaging
•Mostly interested in:
• Segmentation, Registration, Tracking
• Applications: phenotyping, longitudinal studies
Reconstruction of Microscopic Architecture
Stained (H&E) Light Microscopy Stack
Confocal Microscopy Stack
Cellular structures near
mammary gland of a female
mouse
Source: Dr. Leone, Cancer
Genetics, OSU
Embryonic Structure of Zebra Fish,
Source: Dr. Sean Megason, Harvard Medical School
My Colleagues …
Kishore Mosaliganti, 5th year
Bioinformatics/Cancer Genetics
Gustavo Leone, Mike Ostrowski
Human Cancer Genetics Program
Kun Huang, Biomedical
Informatics
The Usual Imaging Pipeline
Harvest Rb- & Rb+ mice
Sectioning - 5 microns
Imaging
Visualization
An Advanced Role for Imaging Support
 Mouse Placenta
 Role of Rb tumor suppressor gene
 Changes in placental
morphology
 Fetal death and miscarriages
 Large data size
 High resolution image (~1 GB)
 800~1200 slides/dataset
 Quantification
 Surface area/volume of
different tissue layers
 Infiltration between tissue
layers
Need More - Morphometric Differences
Labyrinth-Spongiotrophoblast Interface
Wild Type (Top) vs. Mutant (Bottom)
Yet Another (A)Typical Example 
 Mouse Mammary Gland
 PTEN phenotyping
 Data characteristics
 High resolution 20X images (~1 GB)
 500 slides/dataset
 Mammary duct segmentation and 3D reconstruction
Digging In - Tumor Micro-Environment
 Mouse Mammary Gland
 More comprehensive system biology study
 Data characteristics
 Confocal, multi-stained
 50 slides/dataset
 Multi-channel segmentation and 3D reconstruction
The Last One - Zebrafish Embryogenesis
A 2D image plane
Final 3D segmentation
 Identifying and tracking development in the embryo
 Presence of salient structures
 3D cell segmentations and tracking required
 Different in-plane and out-plane resolutions
 800 Time steps available
The Underlying Premise
Is there an unified way to visualize and analyze the
various microscopic image modalities ?
The Essentials Of Microstructure
Premise - you can measure, visualize and
analyze cellular structures if you characterize
and build virtual microstructure
Component
Distributions
Packing
Arrangements
Material Interfaces
Essential I- Component Distributions
& Packing
 Tissue layers differ in spatial distributions
 Characteristic packing of RBCs, nuclei, cytoplasm - phases
 Differ in porosity, volume fractions, sizes and arrangement
 NOT JUST ANOTHER TEXTURE !
 Use spatial correlation functions !
Essential II - Component Arrangements
 Arrangements
 Complex tessellations which can better characterize changes.
 A step ahead of looking at only nuclei their packing
 Complex geometry
 Concentric arrangement of epithelial cells
 Torturous 3D ducts and vasculature
Essentials III – Material Interfaces
Labyrinth-Spongiotrophoblasts Interface
The Holy Grail – Virtual Cellular Reconstructions
Before using cellular segmentation
Using N-pcfs and cellular segmentations
1 TeraByte
Pipelines
1Gb x 1 Gb x 900
20 x magnification
Image Registration (3-D alignment)
Feature extraction
Image Segmentation
3-D Visualization
Quantification
NIH Insight Tool Kit (ITK), NA-MIC Tools (microSlicer3)
Conclusions
 Highly multi-disciplinary approach.
Need scalability and robustness
 Useful workflows need to be constructed
Much application-domain knowledge has to be
embedded in algorithms
Validation of methods and proving robustness is a
pre-occupation.
The final goal of a virtual cellular architecture is
not that elusive 
Destroying The Amazon Rain Forest 











K. Mosaliganti and R. Machiraju et al. An Imaging Workflow for Characterizing Phenotypical
Change in Terabyte Sized Mouse Model Datasets. Journal of Bioinformatics, 2008 (to appear)
K. Mosaliganti and R. Machiraju et al. Visualization of Cellular Biology Structures from Optical
Microscopy Data. IEEE Transactions in Visualization and Computer Graphics, 2008 (to appear)
K. Mosaliganti, R. Machiraju et al. Tensor Classification of N-point Correlation Function features for
Histology Tissue Segmentation. Journal of Medical Image Analysis, 2008 (to appear)
K. Mosaliganti and R. Machiraju et al. Geometry-driven Visualization of Microscopic Structures in
Biology. Workshop on Knowledge-Assisted Visualization, Proceedings of EuroVis2008 (to
appear).
K. Mosaliganti, R. Machiraju et al. “Detection and Visualization of Surface-Pockets to Enable
Phenotyping Studies”. IEEE Transactions on Medical Imaging, volume 26(9), pages 1283-1290,
2007.
R. Sharp, K. Mosaliganti et al. “Volume Rendering Phenotype Differences in Mouse Placenta
Microscopy Data”. Journal of Computing in Science and Engineering, volume 9 (1), pages 38-47,
Jan/ Feb 2007.
P. Wenzel and K. Mosaliganti et al. Rb is critical in a mammalian tissue stem cell population. In
Journal of Genetics and Development, volume 21 (1), pages 85-97, Jan 2007.
K. Mosaliganti and R. Machiraju et al. Automated Quantification of Colony Growth in Clonogenic
Assays. Workshop on Medical Image Analysis with Applications in Biology, 2007, Piscatway,
Rutgers, New Jersey, USA.
R. Ridgway, R. Machiraju et al. Image segmentation with tensor-based classification of N-point
correlation functions. In MICCAI Workshop on Medical Image Analysis with Applications in Biology,
2006.
O. Irfanoglu, K. Mosaliganti et al. “Histology Image Segmentation using the N-Point Correlation
Functions”. International Symposium of Biomedical Imaging, 2006.
Acknowledgements
 Joel Saltz, BMI
 Richard Sharp, Okan Irfanoglu, Firdaus Janoos,
CSE OSU
Weiming Xia, Sean Megason, Harvard Medical
school
 Jens Rittscher, GE Global Research
NIH, NLM Training Grant
NSF ITR grant
Thank You !
Questions ?