Biomedical Image Processing
The State of the Art
1500 ||||||||||||||||||||||||||||||||| 2003
Joerg Meyer
jmeyer@uci.edu
March 14, 2016
DEPARTMENT OF
BIOMEDICAL ENGINEERING
UCIrvine
Biomedical Image Processing
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Biomedical Image Processing
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Biomedical Image Processing
X-Ray
(Gundelach Tube, 1898 – 1905)
Oak Ridge Associated Universities
Health Physics
Instrumentation Museum
(Eberhart’s Manual of High Frequncy Currents, Ch. 10, 1911)
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Biomedical Image Processing
Week 0
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Week 4
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Biomedical Image Processing
Scaphoid Bone
Week 0
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Week 4
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Biomedical Image Processing
Step 1: Incision Marking
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Biomedical Image Processing
Step 2: Exposure of Fracture Line
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Biomedical Image Processing
Step 3: Screw Insertion Site
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Biomedical Image Processing
Before
2 months postop
(Images courtesy of: Electronic Textbook of Hand Surgery, http://www.eatonhand.com)
Step 4: X-Ray
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Biomedical Image Processing
Portable System
High Frequency
X-Ray Tube
(Eberhart’s Manual of High Frequncy Currents, Chap. 10, 1911)
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Biomedical Image Processing
Scaphoids
(1mm)
MagneVu 1000
(MRI)
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Biomedical Image Processing
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Biomedical Image Processing
• Problems:
– Time
– Cost factor
• Solution:
– Correlate X-ray images with 3-D models
– Database (typical bones)
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Biomedical Image Processing
Scaphoid Bones
Various shapes and sizes (Zimmer, 1968)
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Biomedical Image Processing
Scaphoid Bone Fractures
Classification (Herbert)
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Biomedical Image Processing
A
B
A - C: Scaphoid view 1 with forearm pronated
45deg. to view profile of
scaphoid & STT joint
D - Scaphoid view 2 (ulnar
oblique view) showing
radioscaphoid joint (from
Rockwood & Green)
DEPARTMENT OF
BIOMEDICAL ENGINEERING
C
D
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Biomedical Image Processing
• Solution:
– Superimpose
radiographic
scan and
3-D model
– Select best
model from
database
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Biomedical Image Processing
• General
Question:
– How to combine
different
modalities?
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Visible Human (CT, frozen)
Joseph Paul Jernigan
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Visible Human (CT, frozen)
Slice 1125
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Visible Human (MRI)
Slice 1125
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Visible Human (RGB color)
Slice 1125
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Visible Human (RGB color)
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Visible Human
DEPARTMENT OF
BIOMEDICAL ENGINEERING
(3-D Reconstruction)
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
CT Scanner
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
CT Scanner
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
CT Scanner
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
CT Scanner
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
CT Scanner
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
CT Scanner
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
CT Scanner
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
CT Scanner
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
CT Scanner
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
CT Scanner
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
CT Scanner
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Image Acquisition
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
CT Scanner
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Image Acquisition
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Image Acquisition
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Image Acquisition
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Image Acquisition
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Image Acquisition
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Image Acquisition
Principle of a CT Scanner
X-Ray Source
Rotation
Translation
Object
Detector
a) Translation
DEPARTMENT OF
BIOMEDICAL ENGINEERING
b) Rotation
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Image Acquisition
Principle of an MRI Scanner
a) Random Spin
DEPARTMENT OF
BIOMEDICAL ENGINEERING
b) directional magnetic field
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Image Acquisition
Principle of an MRI Scanner
a) Directional Puls (orthogonal)
DEPARTMENT OF
BIOMEDICAL ENGINEERING
b) Relaxation
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Image Acquisition
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Rhesus Monkey Brain
• High-resolution large-scale image data
• Resolution: 2666dpi
Pixel spacing: 9 mm
• Enables zooming
down to the cell
level.
• Total data size:
76 GB
RGB image series (real-color, dyed), 5037 x 3871 x 1400, 76 GB
(data courtesy of Edward G. Jones, Center for Neuroscience, UC Davis)
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Image Acquisition
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
3-D Reconstruction
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Image Acquisition
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
From 2-D Cross-sections to 3-D
RGB image series (real-color, human brain), 1472 x 1152 x 753, 3.57 GB
(data courtesy of Arthur W. Toga, Dept. of Neurology, UCLA School of Medicine)
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
3-D Reconstruction
• 3-D Volume Rendering of a Human Brain
RGB image series (real-color, human brain), 1472 x 1152 x 753, 3.57 GB
(data courtesy of Arthur W. Toga, Dept. of Neurology, UCLA School of Medicine,
image courtesy of Eric B. Lum, Ikuko Takanashi, CIPIC, UC Davis)
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
From 2-D Cross-sections to 3-D
?
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
From 2-D Cross-sections to 3-D
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
From 2-D Cross-sections to 3-D
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
3-D Reconstruction
CT head (512 cross-sections, 1024 planes)
rendered with different transparency transfer functions
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
3-D Reconstruction
Human brain
(128 crosssections,
220 planes)
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Cancer cell
(256 crosssections,
512 planes)
Ice Block
(Human brain)
(128 cryo-sections,
256 planes)
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
3-D Reconstruction
MRI scan of a human skull
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
3-D Reconstruction
CT scan of a human skull
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
3-D Reconstruction
CT scan of a human skull
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
How to scale down large-scale data?
100k polygons
120 CD-ROMs
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Space Decomposition
• Data arranged
in slices
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Space Decomposition
• Goal: extract subvolume
• Problem: data arranged in slices or
unstructured grids
• Necessity to touch a lot of files or data
• Solution: breaking up the data set into
bricks
• Data in bricks still too big for
transmission/
interactive rendering
• Solution: multiresolution representation
 idea: combination of octree/wavelet
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Space Decomposition
• Data arranged
in slices
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Space Decomposition
• Data arranged
in slices
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Space Decomposition
• Breaking up the
data into bricks
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Space Decomposition
• Breaking up the
data into bricks
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Space Decomposition
• Breaking up the
data into bricks
...
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Space Decomposition
• Breaking up the
data into bricks
...
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Space Decomposition
• Breaking up the
data into bricks
...
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Space Decomposition
• Breaking up the
data into bricks
...
DEPARTMENT OF
BIOMEDICAL ENGINEERING
ROI
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Combination: Octree/Wavelet
• Leaf encoding
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Wavelet Compression
...
original (256 x 256)
step 1
step 2
image pyramid
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Wavelet Compression
original (256 x 256)
n=8
n=7
n=6
n=5
n=4
n=3
n=2
DEPARTMENT OF
BIOMEDICAL ENGINEERING
MRI scan
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Wavelet Compression
Original
image array
Original Volume
HLH
LLH
L
H
First run: x-direction
DEPARTMENT OF
BIOMEDICAL ENGINEERING
LL
HL
LH
HL
Second run: y-direction
LHH
LLL
HLL
LHL
HLL
HHH
Third run: z-direction
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Progressive Reconstruction
Initial stage
Second level of detail
Third level of detail
Final reconstructed image
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Application Areas
• Cancer Research
– Image processing (segmentation, classification)
– Multi-modal imaging (CT/MRI/cryosection/confocal)
• Neuroscience
– Rhesus Macaque Monkey Brain Atlas (NIMH)
– Scalable Visualization Toolkits for Bays to Brains (NPACI)
• Cell Physiology
– Connectivity in Leech Giant Glial Cells
– Correspondence Analysis in Time-variant Microscopic
3D Image Data
• Molecular Diagnostics
– Genomics, Proteomics, Phylogenetic Trees
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Infrastructure
• Projection Systems
–
–
–
–
Concave Reality Projection System
RealityCenter
Interactive White Boards
Active/Passive Stereo Projectors
• Spatial Tracking Hardware
– Nest-of-Birds (electro-magnetic)
– VICON Motion Tracking System
(vision)
• Interaction Hardware
– PinchGloves (data gloves)
– Stylus (6dof pointing device)
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Conclusions
• Engineering solutions can help
to make health care affordable
• Economic feasibility
• Data set size and
image quality increases
• Scalable solutions
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
Acknowledgements
•
Center for Neuroscience,UC Davis
•
Engineering Research Center, Mississippi State University
•
Center for Image Processing and Integrated Computing
(CIPIC), UC Davis
•
The Scripps Research Institute (TSRI), La Jolla, CA
•
San Diego Supercomputer Center
•
These projects are funded in part by the National
Institute of Mental Health (NIMH) through the Center for
Neuroscience at UC Davis, and the National Partnership
for Advanced Computational Infrastructure (NPACI)
(award #10195430 00120410).
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine
That‘s all, folks!
Joerg Meyer
University of California, Irvine
Questions
BME/EECS Department
?
644E Engineering Tower
Irvine, CA 92697-2625
jmeyer@uci.edu
http://www.eng.uci.edu/~jmeyer
DEPARTMENT OF
BIOMEDICAL ENGINEERING
Biomedical Image Processing – The State of the Art
Joerg Meyer - jmeyer@uci.edu
UCIrvine