Applied Anatomical Diagnostic
Framework for Visualizing the
Human Body in a 3-D, Immersive,
Navigable and Interactive VR
Environment
May 4, 2012
Steven Beaudoin, BS
George D. Lecakes Jr., MS
Tony Aita, BS
Lawrence Weisberg, MD
Michael Goldberg, MD
Vijay Rajput, MD
Shreekanth Mandayam, PhD
Contributing Authors
Steven
Beaudoin
Michael
Goldberg, MD
George
Lecakes, MS
Vijay
Rajput, MD
Shreekanth
Mandayam, PhD
Lawrence
Weisberg, MD
Overview
•
•
•
•
•
•
•
Background
Objectives
Approach
Results
Future Goals
Conclusions
Acknowledgements
Cooper Medical School
“ To consider integration of 21st
century competencies, expertise, such
as critical thinking, complex problem
solving, collaboration, multimedia
communication, and technological
competencies demonstrated by
professional disciplines”
-US Department of Education, 2010
CAVE® at Medical School
• Basic anatomy
• Clinical correlation
• Systems based practice educational tool
• Simulation scenarios (simulated patients /
conditions)
Background
The Rowan
University CAVE®
Background
The Rowan
University CAVE®
•
•
•
•
•
•
•
7.5'h x 10'w x 10'd tracked virtual environment
covering 3-walls and a floor
4- active stereo DLP projectors, 3,000 ANSI lumens,
1400x1050 resolution, with short-throw zoom lens
CAVE® structure fabricated with extruded &
powdercoated 80/20 aluminum,
8-camera WorldViz IR position tracking system
CrystalEyes® active shutter glasses
6 - PC computer cluster
WorldViz Vizard, Autodesk 3D Studio Max / Maya
Objectives
• Provide intuitive correlation of clinical information
• Utilize a CAVE® virtual reality (VR) environment to
enhance viewing of data sets
• Ask “what if?” questions and simulate multiple scenarios
for pathological/anatomical diagnosis
• Utilize multiple diagnostic imaging procedures for
prognostic capability
Approach
Data preparation
• CT- dataset of volunteer torso from the skull to the pelvis
• Consisted of 909 images at 1mm slices
• Provided in DICOM data format with each image having a different window
center and window width
Data
Preparation
Data
Preprocessing
Data
Visualization
Real-Time
Processing
Approach
Data preprocessing
•
Utilize image processing tool (MATLAB) to convert images into a standard window center
and window width
•
Converted the various ranges into standard grayscale range (0-255)
•
•
•
0 – Black
255 – White
Export images to CAVE compatible file format (tga,png,jpeg)
Data
Preparation
Data
Preprocessing
Data
Visualization
Real-Time
Processing
Approach
Raw data
Processed Data
Data
Preparation
Data
Preprocessing
Data
Visualization
Real-Time
Processing
Approach
Data Visualization
• Load images (textures) sequentially
• Textures are applied to a series of
evenly spaced polygonal planes
• Each plane represents one crosssectional slice of patient data
• Images are given transparency
• Images are rendered back to front
Data
Preparation
Data
Preprocessing
Data
Visualization
Real-Time
Processing
Approach
Pixel Values
• Values from a CT scan
represent different densities
• Radiolucent regions are low
density – air
• Radiodense regions are
higher density – bone
Data
Preparation
Data
Preprocessing
Data
Visualization
Real-Time
Processing
Approach
Data Manipulation and Interface
1. Grayscale histogram – Cull pixel values from the histogram
using a range slider.
2. Pseudo-color histogram – Cull pixel values and color the
ranges with red, green, or blue ranges.
3. Clip Plane Mode – Clip the polygonal planes with an
OpenGL clipping plane in any orientation.
Xbox 360® Controller – All functionality mapped to buttons
Data
Preparation
Data
Preprocessing
Data
Visualization
Real-Time
Processing
Educational / Future Medical Applications
•
Learn anatomy / clinically applied anatomy in
3D virtual reality
•
Anatomic pathology and radiological
integration
•
Correlation of surface and organ anatomy for
surgical procedure
•
Application of 3D virtual reality to understand
system based practice – e.g. Assess Home
environment for safety for geriatric patient
Other uses of CAVE®
Work In Progress
Fuse multiple image data sets (MRI, CT, PET) to
distinguish redundant and complementary
information for clinically applied anatomical
education
Work In Progress
Acknowledgments
We gratefully acknowledge the assistance of:
Dr. H. Warren Goldman, MD, PhD
Professor and Chair of Neurosurgery
Cooper Hospital, Camden, NJ
Mr. Anthony Aita
Neurosurgery Department
Cooper Hospital, Camden, NJ
Approach
Approach