NA-MIC
National Alliance for Medical Image Computing
http://na-mic.org
Techniques for
Visualization of Anatomy
and Function from MRI
Steve Pieper, PhD
VizBi 2010
Heidelberg
Outline
• Motivation
• Standard Clinical MRI Visualization
• Research Imaging
– Registration
– Segmentation and Parcellation
– Functional Imaging
• Applications and Open Issues
National Alliance for Medical Image Computing
http://na-mic.org
Example: Morphometry
Group Statistics
• Regional Cortical Thickness Correlation with
Aging and Cognitive Impairment
– Greater Sensitivity than Clinical Tests
– Localize Disease within Brain Anatomy
– Impact of Treatment on Imaging Biomarkers
National Alliance for Medical Image Computing
http://na-mic.org
Fischl, Greve, et al - MGH
Example: Multimodality
Image Guided Neurosurgery
• Detailed Pre-Operative Model
– Registered to Tracked
Instruments
– Superimposed on Intraoperative
MRI
• Extracted Anatomy and
Function
– Roadmap for Surgical Decision
Making
• What is the Best Surgical
Approach to Preserve
Function?
National Alliance for Medical Image Computing
http://na-mic.org
Golby, Kikinis, Lemaire, BWH Neurosurgery
MRI Context
• MRI: Typically Volumetric at ~1mm
Resolution
– Unlike CT, No Physical Units for MR
– Contrast Determined by Scan Protocol
• Much of clinical MRI is pattern recognition
– Seeing things that “don’t look right”
• Anatomical Conventions
– Like Maps having North at the Top
– Axial (Transverse), Sagittal, Coronal
• Analysis Results
– Spatially Localized Data
– Often visualization is used as a
‘reasonableness check’ on the automated
calculations
National Alliance for Medical Image Computing
http://na-mic.org
Wikipedia
Basic Clinical Visualization
•
•
•
•
•
Window/Level
Corner Annotations
Pseudocolor
Mosaic/Lightbox
Cine
National Alliance for Medical Image Computing
http://na-mic.org
3D Clinical Visualization
• Ray Casting Through Volume
– Summation (Simulated X-Ray)
– MIP (Maxiumum Intensity Projection)
– SSD (Shaded Surface Display)
• Color Transfer Function and Opacity
Transfer Function
• Pseudocolor + Gradient Lighting
• Less Applicable for MRI than CT
• Reference Labels for Standard
Views
– Left/Right, Anterior/Posterior,
Inferior/Superior
National Alliance for Medical Image Computing
http://na-mic.org
MR Angiogram MIP from Siemens
Leonardo Workstation
MRI Contrast Enhancement
• Vascular / Oral: Gadolinium, Iron
Oxide, Manganese
– Change MR Properties of Tissues
• Dynamic Contrast Tissue
Enhancement (4D)
Gilbert et al
University of Wisconsin
– Vascularized Tumor
– Stroke Related Ischemia
• Perfusion, Diffusion, Bloodflow
• Mismatch Indicates Brain Tissue that
May Recover
Gonzalez et al MGH
National Alliance for Medical Image Computing
http://na-mic.org
Longitudinal Imaging (4D)
• Volumes Acquired Over
Multiple Visits
– “Watchful Waiting” Prior to
Intervention
– Monitor Treatment (Multiple
Sclerosis, Cancer, Lupus…)
• Comparison View
– Linked Cursors
– Subtraction Imaging and
Quantification
National Alliance for Medical Image Computing
http://na-mic.org
Guttmann, Meier, Fedorov – BWH
Miller - GE
Registration
• Intra-subject
– Pre-Intra-Post Procedure
– Longitudinal Tracking of Disease Progression
• Inter-subject
– Support Group Comparison (fMRI)
– Map Anatomical Atlas to Individual
• Degrees of Freedom (DOFs)
–
–
–
–
–
–
Rigid (Rotation + Translation)
Similarity (Rigid + Uniform Scale)
Affine (Rigid + Nonuniform Scale and Shear)
Polyaffine (Locally Affine Interpolation)
B-Spline (Cubic Displacement)
Vector Field
National Alliance for Medical Image Computing
http://na-mic.org
Sota - BWH
Registration
Considerations
•
Registration is Typically an Ill-Posed Problem
– Requires Nonlinear Optimization
– Distortions, Anatomy, Pathology Means No
Exact Correspondence
– Multimodality Registration Requires Statistical
Metric (e.g. Mutual Information)
– When No “Right” Answer
• Only What “Looks Right”
or
• What is Reproducible and Statistically
Significant
•
Insight Toolkit (ikt.org) for Details and
Software
Ibanez - Kitware
Registration Method
Fixed
Image
Metric
Interpolator
Moving
Image
Transform
National Alliance for Medical Image Computing
http://na-mic.org
Optimizer
Registration Visualization
• Cross Fade /
Toggle
• Color / Skeleton
Overlay
• Checkerboard
• Vector Fields
National Alliance for Medical Image Computing
http://na-mic.org
Segmentation
• Definition: Assignment of Anatomical Labels to
Image Regions
– Not an Exact Science
• Anatomists Disagree
• Definition Depends on Scale
• Techniques
– Intensity Driven: Function of Image Measurements
• Thresholding is Most Common (Typically Bad for MRI)
– Rule Based:
• E.g. “Skin is always on the outside”
• Obviously not always the case in clinical scans
– Atlas Driven: Registration of Manually Labeled Data
• Also difficult for clinical scans
– Hybrid Approaches Typically Required
• E.g. Expectation Maximization (EM)
National Alliance for Medical Image Computing
http://na-mic.org
Pohl – IBM
Kikinis, Shenton - BWH
Segmentation Visualization
• Label Map Overlay
– Cross Fade / Toggle
– Solid or Outline
• 3D Surface Models
– Leverage Commodity Graphics
Cards
– Material Properties, Lighting,
Transparency…
National Alliance for Medical Image Computing
http://na-mic.org
Parcellation
• Functional or Anatomical Subdivisions (e.g. Cerebral
Hemisphere Surface)
• Obtained from Curvature and Landmarks on Surface
– Flattened to Plane or Sphere for Tractability
– Displayed Inflated to Show Sulci (Valleys) and Gyri (Ridges)
National Alliance for Medical Image Computing
http://na-mic.org
Fischl et al - MGH
Functional MRI
• Blood Oxygen Level Dependent (BOLD)
• Volume Time Series ~2mm Resolution
• Activation Statistics Volume: Typically
Finger Tapping
Generalized Linear Model (GLM) of
– Hemodynamic Response Function (HRF)
– Stimulus / Paradigm
– Intensity Trends
– Physiology
– Motion…
• Statistics Become Difficult
– Noisy Data
– Multiple Comparisons / Multiple Regressors
– Detects Metabolism, not Neural Activity Directly
National Alliance for Medical Image Computing
http://na-mic.org
Plesniak, Liu - BWH
fMRI Visualization
•
•
•
•
Statistics Volume
Multi Volume Rendering
Cortical Surface Map
Group Comparisons in Atlas Space (Spatial
Normalization)
–
–
Group Contrasts (Left/Right, Schizophrenics/Normal Control,
Active/Resting…)
Positive and Negative Correlations
Plesniak, Liu – BWH
Hernell – Linköpings U.
National Alliance for Medical Image Computing
http://na-mic.org
Multi-Modality Imaging
• Integrated Visualization of
What is Known About the
Subject
– Anatomical Space as
Common Coordinate
System
– Segmented Anatomy and
Volume Rendering for
Context
– Statistics Volumes
– Interactive Visualization
(View, Visibility, Cropping,
Slicing… )
• Image Guided Therapies
National Alliance for Medical Image Computing
http://na-mic.org
Plesniak, Aucoin et al - BWH
Jakab and Berenyi - University of Debrecen
Group Comparisons
• Visualization for MRI Data Mining
– Images, Clinical Data,
Demographics on Web Database
– Target Population and Hypothesis
Specify Batch Computation
– Group Statistics Overlay on Inflated
Cortical Surface of Atlas
– Click to Get Scatter Plot of Subjects
• Supports Alzheimer’s Disease
Research
• Anatomy Labels Linked to Web
Resources and Journal Papers
National Alliance for Medical Image Computing
http://na-mic.org
Morphometry BIRN Consortium
Open Challenges in MRI
Visualization
• Stereo / Virtual Reality (VR) / Augmented
Reality (AR) / Telepresence
– No real traction in spite of significant
investment
– Probably due to lack of mainstream hardware
• Dynamic multimodal volume rendering
– Large volumes – CUDA vs. Cluster
– Encode segmentation in transfer function
• General Information Overload
• Display of Uncertainty in Analysis Results
– Many techniques require discretization
National Alliance for Medical Image Computing
http://na-mic.org