Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Multimodality & 4D
Image Registration:
Mehau Kulyk /spL
Methods &
By 9:25 AM, you will ….
Understand the basic mechanics of
multimodality and 4D image registration
techniques
Clinical Use
Understand the different techniques used
to combine, display and interact with
multimodality and 4D image and dose data
Marc L Kessler, PhD
The University of Michigan
Understand the clinical use and limitations
of these techniques for Tx planning, Tx
delivery and plan adaptation
Look here!
By 9:25 AM, you will ….
ing
n
pe d” …
p
ha hoo used
Understand the s
different techniques
i
- with
to combine,tdisplay and interact
e
a
h
multimodality
-4Dtimage and dose data
wh and
r
…
Understand
the
use and limitations
declinical
n
of these techniques
for Tx planning, Tx
u
“
delivery and plan adaptation
Understand the basic mechanics of
multimodality and 4D image registration
techniques
Acknowledgements!
Many wonderful people have contributed
material for this presentation!
Marc L Kessler, PhD
1
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Outline
Motivation
Precision radiation therapy requires
accurate delineation of the tumor
and normal tissues in the planning
phase and accurate localization of
these structures during the delivery
phase
Motivation
Mechanics !
Clinical Use
…with the aid of imaging
Gregoire
Gregoire // St-Luc
St-Luc
Motivation
Multimodality Targeting
entire
Optimization of the radiotherapy process
requires that we anticipate, measure &
adapt to changes in the patient
Imaging
Planning
?
Delivery
on-line
off-line
Imaging
Physics
X-ray
CT
Anatomy
MRI
Physiology
Nuc Med
We now have many cameras available
… which provide complementary data!
Marc L Kessler, PhD
2
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Repeat Imaging
4-D Imaging
Balter
Balter // UM
UM
?
?
… assess motion
Normal Tissues
Target Volumes
Dawson
Dawson // PMH
PMH
Image Guided Treatment
Varian
OBI™
Elekta
Synergy™
Siemens
PRIMATOM™
TomoTherapy
Hi-Art™
ViewRay
Renaissance™
Resonant
Restitu™
The Big Picture
CT
Tx Plan
3D Dose
MR
NM
“Adapting”
Patient
Model
US
3D Dose
Day n
Marc L Kessler, PhD
Portal
Images
CBCT
1…n
US
4D
CBCT
3
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
The Goal
7/26/2007
Ideally, we would like to have a time
dependent vector of information for
every “point” in an anatomic object
image information (MR, CT, NM, … )
physiologic information (τ )
anatomic label information
dose information … with time stamp !
The Goal
Past / Present
PET
MR
Registration
CT
Segmentation
CT+ MR + NM + Dose (τττ)
Scalar Data
4-D Vectors
Marc L Kessler, PhD
4
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Present / Future
Outline
Motivation
Registration
Mechanics !
Segmentation
Clinical Examples
Mechanics
Transformation
… determine the geometric transformation
that maps corresponding points from one
image series to another
… determine the geometric transformation
that maps corresponding points from one
image series to another
Form of the transformation T
… from rigid to fully freeform
Number of degrees of freedoms β
… from 3 to 3 x N *
**N
XB = T ( XA , { ß })
(x,y,z) coordinates
of a point in Series B (x,y,z) coordinates
of a point in Series A
= number of voxels
Marc L Kessler, PhD
5
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
What is T ?
Degrees of Freedom
PET/CT
MR - CT
4D CT
Rigid / Affine
Global, regional, or piecewise
Full 3D / 4D Deformation
Parametric models
Free-form models
None ?
Few
Many
www.gnome.org
www.gnome.org
What is T ?
Affine Transformations
Rigid / Affine
Global, regional, or piecewise
xB =
A xA + b
y = m x+ b
Study A
A
A Square
Square
Study B
(up to 12 DOF)
… in 3D
Translation
Translation
Rotation
Rotation
Scaling
Scaling
Shearing
Shearing
3
3
3
3
Parallel lines stay parallel !
Marc L Kessler, PhD
6
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
www.gnome.org
www.gnome.org
Affine Transformations
Study A
6 DOF
Translation
Translation
Study A
Affine Transformations
Study A
A
A Square
Square
Rotation
Rotation
A
A Square
Square
Study B
Scaling
Scaling
Shearing
Shearing
Parallel lines stay parallel !
non- Affine
www.gnome.org
www.gnome.org
Transformations
Study B
Parallel lines don’t stay parallel!
Study B
Translation
Translation
Rotation
Rotation
Scaling
Scaling
Shearing
Shearing
3 or 4 DOF
Parallel lines stay parallel !
non- Affine
Study A
Transformations
Study B
XB = T ( XA , { ß })
Marc L Kessler, PhD
7
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
non- Affine
Study A
Transformations
non- Affine
Study B
Transformations
Study A
Study B
Transformation parameters
to apply to a particular point
depends on the location of
the point !
XB = T ( XA , { ß(XA) })
non- Affine
Transformations
Balter
Balter // UM
UM
XB = T ( XA , { ß(XA) })
Full 3D / 4D Deformation
… up to 3 x N
(DICOM!)
phase dependent ?
XB = T ( XA , { ß(XA, φ )})
Parametric
Marc L Kessler, PhD
Freeform
8
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Full 3D / 4D Deformation
Parametric
Full 3D / 4D Deformation
Each have some distinct properties
… local
Various splines ( TPS , B-splines )
B-Splines
Other basis functions
Thin-Plate splines … global
Freeform
Finite element models
Flow models ( optical, viscous )
Full 3D / 4D Deformation
Warp Space /
… Drag Objects
Warp Objects /
… Drag Space
Brock / PMH
Parametric
Finite element
… bio-mechanical
Intensity flow
… image forces
( mono-modality )
How Do We Compute { β } ?
1 Construct a metric that measures the
mismatch (or similarity) between a
pair of datasets
2 Apply an optimization algorithm to
determine the parameters (DOF) that
minimize (maximize) this metric
Freeform
Marc L Kessler, PhD
9
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
How Do We Compute { β } ?
{β}
Registration Metrics
?
Geometry-Based Metrics
Point Matching
?
Least Squares
Surface Matching
Chamfer Matching
Geometry-based
Intensity-based
Marc L Kessler, PhD
Σ( X
B
B
- XAA ) 2
Σ min distance 2
… depends on image segmentation!
10
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Intensity-Based Metrics
Mono-modality
Sum Squared Difference
Σ(I
B
B
How About An Example?
Transformation
- IAA ) 2
Rotate - Translate
PET
Registration Metric
Multimodality Data
Mutual Information
Σ
p(IAA, IBB)
p(IAA, IBB) log
p(IAA) p(IBB)
CT Mutual Information
Optimizer
Simplex Algorithm
… depends on the image characteristics!
How About An Example?
How About An Example?
PET
CT
PET
CT
Marc L Kessler, PhD
11
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
How About An Example?
Balter
Balter // UM
UM
How About Deformations ?
PET
CT
Multiphasic CT Data
How About Deformations ?
Transformation
B-Splines ( multi-resolution )
Multiresolution Deformations
Successively increase the resolution of
the knot spacing
Registration Metric
Sum Squared Difference
Optimizer
Exhale
State decent
Gradient
Inhale State
Only small additional computation cost
when increasing the number of knots.
Marc L Kessler, PhD
12
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Multiresolution Deformations
Multiresolution Deformations
Successively increase the resolution of
the image data
Successively increase the resolution of
the image data
¼
Resolution
¼ Resolution
Full
Resolution
Full Resolution
60
60 xx 48
48 mm
mm
60 xx 60
4
4 xx 3
3 mm
mm
4 xx 4
Coarse
Fine
Coarse
Fine
Multiresolution Deformations
Multiresolution B-Splines
Registration Metric vs. Iteration
Multiphasic CT Data
Registration Metric
2.5
2.5
Change in
knot spacing
2.4
2.4
Low Res
2.3
2.3
2.2
2.2
2.1
2.1
High Res
2.0
2.0
1.9
1.9
Exhale State
1.8
1.8
0
0
20
20
40
40
60
60
80
80
100
100
120
120
140
140
160
160
180
180
Inhale State
deformed
Iteration Number
Marc L Kessler, PhD
13
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Ruan
Ruan // UM
UM
Multiresolution B-Splines
We Are Not Really Splines !
Multiphasic CT Data
No “stiffness”
information
Exhale State
Inhale State
deformed
Add Some Physics?
Extracted
Ribcage
Exhale
Deform Inhale
Spatially Variant Stiffness
Etotal = Esimilarity + α Estiffness
intensity similarity measure
tissue-dependent regularization
Evol =
wc(x) |det JT(x) – 1|2 dx
Marc L Kessler, PhD
14
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Ruan
Ruan // UM
UM
“Stiffness” Weighting
Using “Prior” Information
wc(x)
using “stiffness”
information
Extracted
Ribcage
Tissue Sliding
Balter
Balter // UM
UM
Deal with different organs individually?
Exhale
Deform Inhale
Tissue Sliding
Balter
Balter // UM
UM
Deal with different organs individually?
Marc L Kessler, PhD
15
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Segmentation + Registration
No masking
Masking
Brock
Brock // UM
UM
Finite Element Modeling
Exhale
Exhale
Inhale
Inhale
Ribs driven by large
lung deformations
Ribs not affected
by lung registration
Brock
Brock // PMH
PMH
Finite Element Modeling
Take into account physical
tissue properties (directly)
The Future ?
Family of
Generalized,
Customizable,
Patient Models
… thorough segmentation is necessary
Marc L Kessler, PhD
16
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Is The Future Here Already?
Is It Here Already?
II have
have no
no commercial
commercial interest
interest in
in this
this company
company
www.mimvista.com
www.mimvista.com
…from Atlas to Individual
II have
have no
no commercial
commercial interest
interest in
in this
this company
company
Thompson
Thompson // UCLA
UCLA
…from Individuals to Atlas
Brain
Brain Mapping:
Mapping: The
The Disorders
Disorders,, Academic
Academic Press,
Press, 1999
1999
Marc L Kessler, PhD
17
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Meyer/UM
Meyer/UM
without
Segment /register / average
In The Meantime …
with
with
Segment using atlas
Anatomy Mapping
Image
Anatomy
Dose
Anatomy Mapping
Dong
Dong // MDACC
MDACC
Drawn Contours
Simple Overlay
(no transform)
Planning CT
“Delivery” CT
… map to CT
Boolean OR
Use superior MR contrast for targeting
Marc L Kessler, PhD
18
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Anatomy Mapping
Drawn Contours
Dong
Dong // MDACC
MDACC
Transformed and
resampled
Segmentation
of a registration!
Planning done
CT w/ the aid“Delivery”
CT
Pouliot
Pouliot // UCSF
UCSF
Dose Mapping / Analysis
CBCT 1
Dose Mapping
Dong
Dong // MDACC
MDACC
CBCT 2
Transforming All Voxels!
More Than Deformations
∆ Dose
deformation
weight loss
resection
Change in shape
Dose Difference (%)
>5%
Increased cord dose
… not just
deformation!
shrinkage
∆ vascular
>10%
Marc L Kessler, PhD
19
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Dose Mapping
Validation
Dealing with volume elements that may:
change shape / appear / disappear
… need proper spatial re-sampling
How do we know how well these
registration methods perform?
build phantoms and test them
don’t necessarily add in a linear fashion
we can know the truth!
… need some sort of radiobiology
provide tools to examine results
exist in homogenous intensity regions
we don’t know the truth!
… hard to evaluate registration
Validation Phantoms
1986
1986
Validation Phantoms
Kashani
Kashani // UM
UM
CT
MR
Marc L Kessler, PhD
20
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Validation Tools
Qualitative Tools
Validation Tools
Quantitative Tools
Study A
Color gel or wash
overlay
Split /dual screen
displays
Point
1
2
3
4
5
6
Description
2nd branch of bronchial tree
3rd branch of bronchial tree
4th branch of bronchial tree
Vessel bifurcation 1
Vessel bifurcation 2
Vessel bifurcation 3
X
-5.37
-5.73
-6.50
-8.12
-8.06
-10.69
Exhale
Y
0.98
2.12
2.77
3.37
-1.95
2.47
Z
-3.42
-5.42
-8.42
A
A
-9.92
-4.42
0.58
Z
-2.92
-5.92
-9.42
-11.42
-3.92
1.08
Exhale' - Inhale
∆Y
-0.25
-0.16
-0.11
-0.49
0.37
-0.29
∆Z
-0.44
0.09
-0.09
-0.18
0.29
0.16
0.29
0.26
X
-4.62
-5.40
-6.24
A
A
A -8.12
A
-7.67
-10.78
(x , y , z )
Study B
all values in cm.
Anatomic boundary
overlay!
Exhale' ( w/ TPS alignment )
-4.71
-0.47
-3.36
-5.35
0.58
-5.83
-6.27
0.69
-9.51
B
B
-8.19
0.91
-11.60
-7.27
-2.83
-3.63
-10.85
0.87
1.24
*
∆X
-0.09
0.05
-0.03B
B
B
B
-0.07
0.40
-0.07
(x , y , z )
σ
AAPM Task Group 132
Inhale
Y
-0.22
0.74
0.80
1.40
-3.20
1.16
*
0.19
Opportunities & Challenges
T22
Use of Image Registration and Data
Fusion Algorithms and Techniques in
Radiotherapy
Methods to assess the accuracy
of
i ng
image registration andmfusion
o
C o ‘08
in
T11
Flair
Gd
Diff
Issues related to acceptance testing
and quality assurance for image
registration and fusion
Marc L Kessler, PhD
21
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
More than just mechanics!
Summary
What Now ?
Taxonomy of Registration Process
Geometry
Intensity
Interactive
Automated
Affine
non-Affine
MR volumes mapped to CT study
Summary
Product Comparison
Tools are now available to register and
integrate image, anatomy & dose for
both Tx planning and Tx delivery
www.ITNonline.net
These tools can be used to help build
better models of the patient and to
help customize and adapt therapy
Work towards more standard and
robust tools and validations methods
(for non-rigid) situations continues
Marc L Kessler, PhD
22
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Don’t try this at home!
Thank you
for your
time !
There’s more
than one way
to scan a cat
www.itk.org
Winter Institute of Medical Physics
February 9-13, 2008
30th Year Anniversary
www.utmem.edu/WIMP/
Summit County, Colorado
~ Breckenridge, Keystone, Vail, A-Basin, Copper ~
Marc L Kessler, PhD
23
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Practical Aspects
Degrees of Freedom
… do whatever you can to reduce the
number of degrees of freedom of the
image registration problem !
PET/CT
MR - CT
4D CT
Using too many degrees of freedom will …
increase computation time
increase the likelihood of local minima
likely decrease the overall accuracy
None ?
Degrees of Freedom
PET
Few
Many
Not Always None!
CT
PET / CT Hybrid
GE Discovery LS
T = Identity ?
XPET = XCT ?
PET
Marc L Kessler, PhD
XPET = XCT
CT
24
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
JNM
JNM 46:1488-96
46:1488-96 2005
2005
Not Always None!
Not Always None!
User Beware!
CT artifact from respiration “burned in”
to attenuation corrected PET
Not Always None!
Dawson
Dawson // PMH
PMH
What About Using Just a Few?
MR - CT phantom
in head frame
Most of the motion
of the liver seems
to be rigid or affine!
… some deformation
does occur though.
Mechanically attach
a coordinate system
You still need
to be careful !
Marc L Kessler, PhD
25
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
What About Using Just a Few?
What About Using Just a Few?
maybe ignore over a limited field-of-view
maybe ignore over a limited field-of-view
?
particularly
poor
MR
CT
( Diagnostic )
( Therapy )
anatomic-based
data cropping
CT registered to MR
split-screen display
What About Using Just a Few?
What About Using Just a Few?
maybe ignore over a limited field-of-view
maybe ignore over a limited field-of-view
anatomic-based
data cropping
anatomic-based
data cropping
CT registered to MR
image-switch display
Marc L Kessler, PhD
CT registered to MR
image-switch display
26
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Limited Field-of-View
Limited Field-of-View
Rigid assumption used
for regional registration
Oops!
Pick Your Battles Wisely!
Successively increase the resolution of
the image data and spline knot density
60
60 xx 60
60 xx 48
48 mm
mm
4
4 xx 4
4 xx 3
3 mm
mm
Coarse
Fine
Knot Spacing
Multiresolution Deformations
Registration Metric vs. Iteration
2.5
2.5
Registration Metric
Start Off Slow, … Speed Up!
Increase in
knot density
2.4
2.4
2.3
2.3
2.2
2.2
2.1
2.1
2.0
2.0
1.9
1.9
1.8
1.8
0
0
20
20
40
40
60
60
80
80
100
100
120
120
140
140
160
160
180
180
Iteration Number
Marc L Kessler, PhD
27
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Multiresolution Deformations
Multiresolution Deformations
4-D CT Example
4-D CT Example
Exhale State
Exhale State
Inhale State
Validation of Registration
Validation of Registration
4-D CT Example
Study A
Point
1
2
3
4
5
6
Description
2nd branch of bronchial tree
3rd branch of bronchial tree
4th branch of bronchial tree
Vessel bifurcation 1
Vessel bifurcation 2
Vessel bifurcation 3
X
-5.37
-5.73
-6.50
-8.12
-8.06
-10.69
Exhale
Y
0.98
2.12
2.77
3.37
-1.95
2.47
X
-4.62
-5.40
-6.24
A
A
-8.12
-7.67
-10.78
Inhale
Y
-0.22
0.74
0.80
1.40
-3.20
1.16
Z
-2.92
-5.92
-9.42
-11.42
-3.92
1.08
*
∆X
-0.09
0.05
-0.03
B
B
-0.07
0.40
-0.07
Exhale' - Inhale
∆Y
-0.25
-0.16
-0.11
-0.49
0.37
-0.29
∆Z
-0.44
0.09
-0.09
-0.18
0.29
0.16
σ
0.19
0.29
0.26
*
Z
-3.42
-5.42
-8.42
A
A-9.92 A
A
-4.42
0.58
(x , y , z )
Inhale State
deformed
Study B
all values in cm.
Exhale' ( w/ TPS alignment )
-4.71
-0.47
-3.36
-5.35
0.58
-5.83
-6.27
0.69
-9.51 B
B
B
B
-8.19
0.91
-11.60
-7.27
-2.83
-3.63
-10.85
0.87
1.24
(x , y , z )
Exhale State
Marc L Kessler, PhD
Inhale State
28
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Balter/
Balter/ UM
UM
4-D Deformable Phantom
Multiresolution Deformations
4-D CT Example
Exhale State
Inhale State
deformed
Ruan
Ruan // UM
UM
We Are Not Really Splines !
Two Part Cost Function
• intensity similarity measure
• tissue-dependent deformation
regularization
No “stiffness”
information
Extracted
Ribcage
T * = arg min ESSD(T ) + α Evol(T )
T
Exhale
Deform Inhale
Marc L Kessler, PhD
29
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Two Part Cost Function
Regularization Term
Similarity term: Sum squared differences
ESSD =
( f(x) - g(T.x) )2 dx
Regularization term: Volume preservation
Evol =
wc(x) |det JT(x) – 1|2 dx
Ruan
Ruan // UM
UM
“Stiffness” Function
Using “Prior” Information
wc(x)
using “stiffness”
information
Extracted
Ribcage
Marc L Kessler, PhD
Exhale
Deform Inhale
30
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Balter
Balter // UM
UM
Tissue Sliding
Deal with different organs individually?
Mean Square Difference
Segmentation + Registration
No
No masking
masking
Mask
Mask using
using segmented
segmented lung/abdomen
lung/abdomen
Note
ribs (and
(and tissue
tissue )) near
near diaphragm
diaphragm
Note ribs
Ribs
by lung
lung registration
registration
Ribs not
not affected
affected by
Product Comparison
Mean
Mean Square
Square Difference
Difference
The
The mean
mean square
square difference,
difference, MSD,
MSD, is
is aa simple
simple method
method to
to evaluate
evaluate the
the similarity
similarity between
between two
two images,
images, A
A
and
and B,
B, as
as is
is shown
shown in
in equation
equation 7.
7. MSD
MSD is
is aa direct
direct approach
approach that
that can
can only
only be
be used
used when
when the
the intensities
intensities
between
between images
images A
A and
and B
B are
are the
the same,
same, but
but provides
provides aa simple
simple minimization
minimization problem
problem for
for optimizing
optimizing
13,14
registration.
registration.13,14
MSD( A, B) =
A− B
www.ITNonline.net
2
Marc L Kessler, PhD
31
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Don’t try this at home!
Practical Aspects
… do whatever you can to reduce the
number of degrees of freedom of the
image registration problem !
Using too many degrees of freedom will …
increase computation time
increase the likelihood of local minima
likely decrease the overall accuracy
www.itk.org
Degrees of Freedom
PET/CT
MR - CT
Degrees of Freedom
4D CT
PET
CT
PET / CT Hybrid
GE Discovery LS
None ?
Few
Many
T = Identity ?
Marc L Kessler, PhD
XPET = XCT ?
32
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Not Always None!
PET
XPET = XCT
JNM
JNM 46:1488-96
46:1488-96 2005
2005
Not Always None!
CT
Not Always None!
Not Always None!
User Beware!
MR - CT phantom
in head frame
CT artifact from respiration “burned in”
to attenuation corrected PET
Mechanically attach
a coordinate system
Marc L Kessler, PhD
You still need
to be careful !
33
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
The Mechanics!
Automated Registration
FDG PET
11C
Original
PET
C PET
Original 11
CT
Image
Registration
)
H(IPET
PET
1.83
Data
Fusion
PET
X-ray CT
Synthetic
Synthetic
MR-PET
MR-PET Image
Image
Original
Original MR
MR
H(ICT)
3.98
Registered
Registered MR
MR
DICOM 3 Parts 3 & 17
Manipulate CT to
match FDG-PET
- rotate / translate -
DICOM 3 Parts 3 & 17
From
From DICOM
DICOM 3.3-2007,
3.3-2007, page
page 209
209
A.39.2.1
A.39.2.1 Deformable
Deformable Spatial
Spatial Registration
Registration IOD
IOD Description
Description
The
The Deformable
Deformable Spatial
Spatial Registration
Registration Information
Information Object
Object Definition
Definition (IOD)
(IOD) describes
describes
spatial
spatial relationships
relationships between
between images
images in
in one
one or
or more
more frames
frames of
of reference
reference via
via
deformation
grids
and
transformation
matrices.
The
deformations
and
deformation grids and transformation matrices. The deformations and
transformations
transformations describe
describe to
to an
an application
application how
how to
to sample
sample data
data from
from one
one or
or more
more
Source
Source RCSs
RCSs into
into the
the Registered
Registered RCS.
RCS.
The
The Registered
Registered RCS
RCS is
is the
the Frame
Frame of
of Reference
Reference specified
specified within
within an
an instance
instance of
of this
this
IOD.
Source RCS
RCS Frame
Frame of
of
IOD. The
The IOD
IOD may
may specify
specify that
that only
only aa subset
subset of
of the
the entire
entire Source
Reference
is
affected
by
the
transformation,
by
specifying
specific
frames
of
Reference is affected by the transformation, by specifying specific frames of image
image
SOP
Source Frame
Frame of
of Reference.
Reference.
SOP Instances
Instances that
that use
use the
the Source
The
The deformation
deformation is
is described
described as
as aa grid
grid of
of offset
offset vectors.
vectors. Each
Each grid
grid element
element
contains
contains 3
3 values
values representing
representing offset
offset distances
distances in
in the
the X,
X, Y,
Y, and
and Z
Z directions
directions at
at the
the
center
center position
position of
of the
the deformation
deformation grid
grid element.
element. The
The relationship
relationship between
between the
the
data
data being
being deformed
deformed and
and the
the deformation
deformation grid
grid is
is purely
purely spatial.
spatial. Therefore
Therefore the
the
resolution
resolution of
of the
the grid
grid is
is independent
independent of
of the
the data
data being
being deformed.
deformed.
DICOM handles only up to affine
... (and most Tx planning systems)
Marc L Kessler, PhD
34
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
New DICOM Objects ?
Prospective
OK Marc
PET / CT Hybrid
GE Discovery LS
F = Identity
Not Always Perfect!
XB = XA
User Beware!
User Beware!
CT artifact from respiration “burned in”
to attenuation corrected PET
Marc L Kessler, PhD
35
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Why did that work?
User Beware
MR - CT phantom
… in
attach
a
head frame
ouch
… reproduce patient orientation as closely
as possible using an immobilization device
coordinate
system to
the patient!
… Stereotactic Radiosurgery
Why did that work?
… reproduce patient orientation as closely
as possible using an immobilization device
Virtual
CT device
device
Virtual PET
PET -- CT
Custom-molded
Styrofoam cradle
Thorax Board
Sinmed BV
Full 3D / 4D Deformation
Warp Space /
… Drag Objects
Warp Objects /
… Drag Space
18FDG is not very specific - it lights up
… 18
a lot of tissues to some extent
- it contains anatomic information
… breath hold exhale CT similar to free
breathing PET
fairly
Brock
Brock // PMH
PMH
Parametric
Marc L Kessler, PhD
Freeform
36
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
How ?
Prospective
Prospective
reproduce imaging geometry exactly
attach coordinate system to patient
• frames / fiducials
Retrospective
PET / CT Hybrid
GE Discovery LS
patient intrinsic
• anatomy / shape / image intensities
F = Identity
Prospective
ouch
XB = XA
Retrospective
… attach a
coordinate
system to
the patient!
How do we determine T ?
Interactive Tools
… let the experienced user drive
Automated Tools
… let the computer drive
… Stereotactic Radiosurgery
Marc L Kessler, PhD
37
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Interactive Registration
Multi-resolution B-Splines
Should we really describe this?
Translate
Yes, … go to next slide
Rotate
? Deform
No, … skip past this section
Provide tools to transform and visualize!
B-Spline Transformation Model
B-Spline Transformation Model
Transformation is built up using
a set of weighted basis splines
Transformation is built up using
a set of weighted basis splines
∆X
basis spline
∆X
basis splines
w
w11
knot
k11
w
w11
X
knots
… these are “just like” the
beamlets we use in IMRT
Marc L Kessler, PhD
k11
w
w22
k22
X
… these are “just like” the
beamlets we use in IMRT
38
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
B-Spline Transformation Model
B-Spline Transformation Model
Transformation is built up using
a set of weighted basis splines
Transformation is built up using
a set of weighted basis splines
∆X
basis splines
w
w11
k11
knots
∆X
w
w22
k22
basis splines
w
w33
w
w11
k33
X
knots
k11
w
w22
k22
w
w33
k33
w
w44
k44
X
… these are “just like” the
beamlets we use in IMRT
… these are “just like” the
beamlets we use in IMRT
B-Spline Transformation Model
B-Spline Transformation Model
Transformation is built up using
a set of weighted basis splines
Transformation is built up using
a set of weighted basis splines
∆X
weighted
sum
∆X
w
w11
knots
k11
w
w22
k22
w
w33
k33
w
w11
w
w44
k44
X
β(X-kii)
X’ = X + ∆X = X + Σ wiiWβ
weights
knots
k11
w
w22
k22
w
w33
k33
w
w44
k44
X
β(X-kii)
X’ = X + ∆X = X + Σ wiiWβ
splines
Marc L Kessler, PhD
weights
splines
39
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
B-Spline Transformation Model
B-Spline Transformation Model
Transformation is built up using
a set of weighted basis splines
Transformation is built up using
a set of weighted basis splines
∆X
w
w11
knots
k11
w
w33
∆X
w
w44
w
w11
w
w22
k22
k33
k44
X
knots
β(X-kii)
X’ = X + ∆X = X + Σ wiiWβ
weights
k11
w
w33
w
w44
w
w22
k22
k33
k44
X
β(X-kii)
X’ = X + ∆X = X + Σ wiiWβ
splines
weights
splines
B-Spline Transformation Model
B-Spline Transformation Model
Transformation is built up using
a set of weighted basis splines
Tighter knot spacing allows
more local deformation
∆X
w
w11
knots
k11
w
w33
∆X
w
w44
w
w22
k22
k33
k44
X
knots
k11 k22 k33 k44
k55 k66 k77 k88
k99 k10
k11
10
11
X
β(X-kii)
X’ = X + ∆X = X + Σ wiiWβ
… linear with respect to weights!
Marc L Kessler, PhD
40
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
B-Spline Transformation Model
B-Spline Transformation Model
Tighter knot spacing allows
more local deformation
Tighter knot spacing allows
more local deformation
∆X
∆X
w
w77
knots
k11 k22 k33 k44
k55 k66 k77 k88
w
w77
k99 k10
k11
10
11
X
knots
k11 k22 k33 k44
k55 k66 k77 k88
k99 k10
k11
10
11
X
B-Spline Transformation Model
B-Spline Transformation Model
Tighter knot spacing allows
more local deformation
Tighter knot spacing allows
more local deformation
∆X
knots
∆X
w
w77
k11 k22 k33 k44
k55 k66 k77 k88
k99 k10
k11
10
11
X
knots
k11 k22 k33 k44
k55 k66 k77 k88
k99 k10
k11
10
11
X
However, more degrees of
freedom can lead to local minima
Marc L Kessler, PhD
41
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Regional Registration
Can you tell what is different in the 2 images?
Regional Registration
Bones aligned, prostate region not aligned
No Cropping
The “answer” depends on the region defined!
Regional Registration
Dawson
Dawson // PMH
PMH
What About Using Just a Few?
Bones ignored, prostate region aligned
Most of the motion
of the liver seems
to be rigid or affine!
… some deformation
does occur though.
Cropping
The “answer” depends on the region defined!
Marc L Kessler, PhD
42
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
What About Using Just a Few?
What About Using Just a Few?
maybe ignore over a limited field-of-view
maybe ignore over a limited field-of-view
?
particularly
poor
MR
CT
( Diagnostic )
( Therapy )
anatomic-based
data cropping
CT registered to MR
split-screen display
What About Using Just a Few?
What About Using Just a Few?
maybe ignore over a limited field-of-view
maybe ignore over a limited field-of-view
anatomic-based
data cropping
anatomic-based
data cropping
CT registered to MR
image-switch display
Marc L Kessler, PhD
CT registered to MR
image-switch display
43
Multimodality Imaging in Radiation Oncology:
Imaging vs. Imagining
Limited Field-of-View
Limited Field-of-View
Rigid assumption used
for regional registration
Pick Your Battles Wisely!
Marc L Kessler, PhD
Oops!
44