NA-MIC National Alliance for Medical Image Computing http://na-mic.org Techniques for Visualization of Anatomy and Function from MRI Steve Pieper, PhD VizBi 2010 Heidelberg Outline • Motivation • Standard Clinical MRI Visualization • Research Imaging – Registration – Segmentation and Parcellation – Functional Imaging • Applications and Open Issues National Alliance for Medical Image Computing http://na-mic.org Example: Morphometry Group Statistics • Regional Cortical Thickness Correlation with Aging and Cognitive Impairment – Greater Sensitivity than Clinical Tests – Localize Disease within Brain Anatomy – Impact of Treatment on Imaging Biomarkers National Alliance for Medical Image Computing http://na-mic.org Fischl, Greve, et al - MGH Example: Multimodality Image Guided Neurosurgery • Detailed Pre-Operative Model – Registered to Tracked Instruments – Superimposed on Intraoperative MRI • Extracted Anatomy and Function – Roadmap for Surgical Decision Making • What is the Best Surgical Approach to Preserve Function? National Alliance for Medical Image Computing http://na-mic.org Golby, Kikinis, Lemaire, BWH Neurosurgery MRI Context • MRI: Typically Volumetric at ~1mm Resolution – Unlike CT, No Physical Units for MR – Contrast Determined by Scan Protocol • Much of clinical MRI is pattern recognition – Seeing things that “don’t look right” • Anatomical Conventions – Like Maps having North at the Top – Axial (Transverse), Sagittal, Coronal • Analysis Results – Spatially Localized Data – Often visualization is used as a ‘reasonableness check’ on the automated calculations National Alliance for Medical Image Computing http://na-mic.org Wikipedia Basic Clinical Visualization • • • • • Window/Level Corner Annotations Pseudocolor Mosaic/Lightbox Cine National Alliance for Medical Image Computing http://na-mic.org 3D Clinical Visualization • Ray Casting Through Volume – Summation (Simulated X-Ray) – MIP (Maxiumum Intensity Projection) – SSD (Shaded Surface Display) • Color Transfer Function and Opacity Transfer Function • Pseudocolor + Gradient Lighting • Less Applicable for MRI than CT • Reference Labels for Standard Views – Left/Right, Anterior/Posterior, Inferior/Superior National Alliance for Medical Image Computing http://na-mic.org MR Angiogram MIP from Siemens Leonardo Workstation MRI Contrast Enhancement • Vascular / Oral: Gadolinium, Iron Oxide, Manganese – Change MR Properties of Tissues • Dynamic Contrast Tissue Enhancement (4D) Gilbert et al University of Wisconsin – Vascularized Tumor – Stroke Related Ischemia • Perfusion, Diffusion, Bloodflow • Mismatch Indicates Brain Tissue that May Recover Gonzalez et al MGH National Alliance for Medical Image Computing http://na-mic.org Longitudinal Imaging (4D) • Volumes Acquired Over Multiple Visits – “Watchful Waiting” Prior to Intervention – Monitor Treatment (Multiple Sclerosis, Cancer, Lupus…) • Comparison View – Linked Cursors – Subtraction Imaging and Quantification National Alliance for Medical Image Computing http://na-mic.org Guttmann, Meier, Fedorov – BWH Miller - GE Registration • Intra-subject – Pre-Intra-Post Procedure – Longitudinal Tracking of Disease Progression • Inter-subject – Support Group Comparison (fMRI) – Map Anatomical Atlas to Individual • Degrees of Freedom (DOFs) – – – – – – Rigid (Rotation + Translation) Similarity (Rigid + Uniform Scale) Affine (Rigid + Nonuniform Scale and Shear) Polyaffine (Locally Affine Interpolation) B-Spline (Cubic Displacement) Vector Field National Alliance for Medical Image Computing http://na-mic.org Sota - BWH Registration Considerations • Registration is Typically an Ill-Posed Problem – Requires Nonlinear Optimization – Distortions, Anatomy, Pathology Means No Exact Correspondence – Multimodality Registration Requires Statistical Metric (e.g. Mutual Information) – When No “Right” Answer • Only What “Looks Right” or • What is Reproducible and Statistically Significant • Insight Toolkit (ikt.org) for Details and Software Ibanez - Kitware Registration Method Fixed Image Metric Interpolator Moving Image Transform National Alliance for Medical Image Computing http://na-mic.org Optimizer Registration Visualization • Cross Fade / Toggle • Color / Skeleton Overlay • Checkerboard • Vector Fields National Alliance for Medical Image Computing http://na-mic.org Segmentation • Definition: Assignment of Anatomical Labels to Image Regions – Not an Exact Science • Anatomists Disagree • Definition Depends on Scale • Techniques – Intensity Driven: Function of Image Measurements • Thresholding is Most Common (Typically Bad for MRI) – Rule Based: • E.g. “Skin is always on the outside” • Obviously not always the case in clinical scans – Atlas Driven: Registration of Manually Labeled Data • Also difficult for clinical scans – Hybrid Approaches Typically Required • E.g. Expectation Maximization (EM) National Alliance for Medical Image Computing http://na-mic.org Pohl – IBM Kikinis, Shenton - BWH Segmentation Visualization • Label Map Overlay – Cross Fade / Toggle – Solid or Outline • 3D Surface Models – Leverage Commodity Graphics Cards – Material Properties, Lighting, Transparency… National Alliance for Medical Image Computing http://na-mic.org Parcellation • Functional or Anatomical Subdivisions (e.g. Cerebral Hemisphere Surface) • Obtained from Curvature and Landmarks on Surface – Flattened to Plane or Sphere for Tractability – Displayed Inflated to Show Sulci (Valleys) and Gyri (Ridges) National Alliance for Medical Image Computing http://na-mic.org Fischl et al - MGH Functional MRI • Blood Oxygen Level Dependent (BOLD) • Volume Time Series ~2mm Resolution • Activation Statistics Volume: Typically Finger Tapping Generalized Linear Model (GLM) of – Hemodynamic Response Function (HRF) – Stimulus / Paradigm – Intensity Trends – Physiology – Motion… • Statistics Become Difficult – Noisy Data – Multiple Comparisons / Multiple Regressors – Detects Metabolism, not Neural Activity Directly National Alliance for Medical Image Computing http://na-mic.org Plesniak, Liu - BWH fMRI Visualization • • • • Statistics Volume Multi Volume Rendering Cortical Surface Map Group Comparisons in Atlas Space (Spatial Normalization) – – Group Contrasts (Left/Right, Schizophrenics/Normal Control, Active/Resting…) Positive and Negative Correlations Plesniak, Liu – BWH Hernell – Linköpings U. National Alliance for Medical Image Computing http://na-mic.org Multi-Modality Imaging • Integrated Visualization of What is Known About the Subject – Anatomical Space as Common Coordinate System – Segmented Anatomy and Volume Rendering for Context – Statistics Volumes – Interactive Visualization (View, Visibility, Cropping, Slicing… ) • Image Guided Therapies National Alliance for Medical Image Computing http://na-mic.org Plesniak, Aucoin et al - BWH Jakab and Berenyi - University of Debrecen Group Comparisons • Visualization for MRI Data Mining – Images, Clinical Data, Demographics on Web Database – Target Population and Hypothesis Specify Batch Computation – Group Statistics Overlay on Inflated Cortical Surface of Atlas – Click to Get Scatter Plot of Subjects • Supports Alzheimer’s Disease Research • Anatomy Labels Linked to Web Resources and Journal Papers National Alliance for Medical Image Computing http://na-mic.org Morphometry BIRN Consortium Open Challenges in MRI Visualization • Stereo / Virtual Reality (VR) / Augmented Reality (AR) / Telepresence – No real traction in spite of significant investment – Probably due to lack of mainstream hardware • Dynamic multimodal volume rendering – Large volumes – CUDA vs. Cluster – Encode segmentation in transfer function • General Information Overload • Display of Uncertainty in Analysis Results – Many techniques require discretization National Alliance for Medical Image Computing http://na-mic.org