MIT OpenCourseWare
http://ocw.mit.edu
HST.582J / 6.555J / 16.456J Biomedical Signal and Image Processing
Spring 2007
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
Harvard-MIT Division of Health Sciences and Technology
HST.582J: Biomedical Signal and Image Processing, Spring 2007
Course Director: Dr. Julie Greenberg
Medical Image Registration I
HST 6.555
Lilla Zöllei and William Wells
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
The Registration Problem
Spring 2007
HST 6.555
2
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
The Registration Problem
Spring 2007
HST 6.555
3
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
The Registration Problem
Spring 2007
HST 6.555
4
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Applications
„
„
multi-modality fusion (intra-subject)
time-series processing
‰
„
„
„
e.g.: fMRI experiments, cardiac ultrasound
warping across patients (inter-subject, uni-modal)
warping to / from atlas for anatomical labeling
image-guided surgery:
‰
‰
Spring 2007
modeling tissue deformation,
comparing pre- and intra-operative scans,…
HST 6.555
5
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
The Registration Problem
Tinit
Tfinal
Spring 2007
HST 6.555
6
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Medical Image Registration
Medical image data sets
Transform (move around)
Compare with objective function
score
initial
value
Spring 2007
motion
parameters
Optimization algorithm
HST 6.555
7
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Roadmap
„
„
„
Data representation
Transformation types
Objective functions
‰
‰
„
„
Feature/surface-based
Intensity-based
Optimization methods
Current research topics
Spring 2007
HST 6.555
8
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Data Representation
z
y
object
„
Object represented as a function
‰
example:
coordinates Æ intensity
x
Spring 2007
HST 6.555
9
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Data Representation
x2
„
„
f(x): space Æ intensity
representation of “moved” object
g ( x ') = f (T ( x '))
x1
x2’
x1’
Spring 2007
HST 6.555
10
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Data Representation
x2
„
x0
focus on a feature
‰
‰
before motion: f ( x0 ) = f
0
after motion
„ What value of x0’ : g ( x0 ') = f 0
?
f (T ( x0 ')) = f 0
T ( x0 ') = x0
x0 ' = T −1(x0 )
x2’
x1
x0’
Feature of f 0 now is located at T −1 ( x0 ) −1
⇒ motion of object is T (.)
„
x1’
Spring 2007
HST 6.555
11
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Medical Image Registration
Medical image data sets
Transform (move around)
Compare with objective function
score
initial
value
Spring 2007
motion
parameters
Optimization algorithm
HST 6.555
12
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Space of transformations
Data dimensions
2D-2D
3D-3D
2D-3D
Spring 2007
Example
cardiac ultrasound,
x-ray patient repositioning,
histology – MRI, …
MR/MR, MR/CT, CT/CT,
PET-MR, …
X-ray/CT, fluoroscopy-CT,
surface model/video
HST 6.555
13
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Class of transformations
„
What motions or distortions are allowed to
merge datasets?
‰
Rigid Transformations:
„
„
‰
displacement
rotation & displacement
Non-Rigid Transformations:
„
parametric
‰
‰
‰
„
Spring 2007
affine
piecewise-affine
….
non-parametric
HST 6.555
14
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Displacement only
T ( x) ≡ x + D „
2D: 2 parameters
3D: 3 parameters
„
For example:
„
⎛ ⎡ x1 ⎤ ⎞ ⎡ x1 ⎤ ⎡ D1 ⎤
T ⎜⎢ ⎥⎟ = ⎢ ⎥ + ⎢ ⎥
⎝ ⎣ x2 ⎦ ⎠ ⎣ x2 ⎦ ⎣ D2 ⎦
⎛ ⎡ x1 ⎤ ⎞ ⎡ x1 ⎤ ⎡ D1 ⎤
⎜⎢ ⎥⎟ ⎢ ⎥ ⎢ ⎥
T ⎜ ⎢ x2 ⎥ ⎟ = ⎢ x2 ⎥ + ⎢ D2 ⎥
⎜ ⎢x ⎥⎟ ⎢x ⎥ ⎢D ⎥
⎝⎣ 3⎦⎠ ⎣ 3⎦ ⎣ 3⎦
in 2D
in 3D
Spring 2007
HST 6.555
15
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Rigid Motion
T ( x) ≡ R( x) + D
„
„
„
„
„
both rotations and displacements are allowed
length-preserving transformation
order of transformations does matter!
2D: 3 parameters; 3D: 6 parameters
for example: in 2D (non-linear in θ)
T ( x) = Rx + D
⎡cosθ −sinθ ⎤
R=⎢
⎥
sin
cos
θ
θ
⎣
⎦
Spring 2007
R: valid rotation matrix
RT R = 1 and R = +1
HST 6.555
16
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
„
in 3D: rotate by θ about axis N̂
‰
if q is a unit quaternion, such that
⎛ q02 + q12 − q22 − q32
⎜
R= ⎜ 2 ( q0 q3 + q2 q1 )
⎜ 2(−q0 q2 + q3 q1 )
⎝
θ
θ⎞
⎛
q = ⎜ cos ; N̂ sin ⎟
2
2⎠
⎝
2(−q0 q3 + q1q2)
2
2
2
2
q0 − q1 + q2 − q3
2(q0 q1 + q3 q2 )
2(q0 q2 + q1q3 ) ⎞
⎟
2(−q0 q1 + q2 q3 ) ⎟
q02 − q12 − q22 + q32 ⎠⎟
Horn, B.K.P., Closed Form Solution of Absolute Orientation using Unit Quaternions, Journal
of the Optical Society A, Vol. 4, No. 4, pp. 629--642, April 1987.
Spring 2007
HST 6.555
17
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Non-rigid transformations
„
Parametric
‰
‰
‰
„
affine
piecewise-affine
others
Non-parametric
Spring 2007
HST 6.555
18
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Affine Transformation
T ( x) ≡ M ( x) +
D
„
M: square matrix
‰
‰
„
„
beyond rigid motion, allows shears and scaling
preserves notion of parallel lines
2D: 6 parameters
3D: 12 parameters
Spring 2007
HST 6.555
19
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
„
2D affine:
⎛ ⎡ x1 ⎤ ⎞ ⎡ m11
T ⎜⎢ ⎥⎟ = ⎢
⎝ ⎣ x2 ⎦ ⎠ ⎣ m21
m12 ⎤ ⎡ x1 ⎤ ⎡ D1 ⎤
+⎢ ⎥
⎥
⎢
⎥
m22 ⎦ ⎣ x2 ⎦ ⎣ D2 ⎦
OR
⎛ ⎡ x1 ⎤ ⎞ ⎡ m11 m12
⎜⎢ ⎥⎟ ⎢
T ⎜ ⎢ x2 ⎥ ⎟ = ⎢ m21 m22
⎜⎢ 1 ⎥⎟ ⎢ 0
0
⎝⎣ ⎦⎠ ⎣
D1 ⎤ ⎡ x1 ⎤
D2 ⎥⎥ ⎢⎢ x2 ⎥⎥
1 ⎥⎦ ⎢⎣ 1 ⎥⎦
linear form
T ( X ) = QX
homogeneous coordinate
Spring 2007
HST 6.555
20
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
2D affine: determined by control points
„
[Y1
Y1 = QX 1
Y2 Y3 ] = Q [ X 1
X 2
X 3 ] Y2 = QX 2
Q = [Y1 Y2 Y3 ][ X1 X2 X3
] Y3 = QX 3
(if the inverse exists)
−1
Y1
X1
X2
X3
Y2
Y3
Spring 2007
HST 6.555
21
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
„
Affine transformation for other points:
determined by motion of control points
X
Spring 2007
⎡ y1 ⎤
⎡ x1 ⎤
⎢ y ⎥ = Q ⎢x ⎥
⎢ 2⎥
⎢ 2⎥
⎢⎣ 1 ⎥⎦
⎢⎣ 1 ⎥⎦
HST 6.555
Y
22
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Piecewise affine
„
2D example:
‰
‰
subdivide space of X into triangles
use different affine transformation for each
triangle (determined by vertices…)
-e.g.: some use in breast registration
Spring 2007
HST 6.555
23
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Other Parametric Transformations
„
Finite element models
‰
„
engineering methods to simulate mechanical /
electrical systems (discretization of the space;
integral problem formulation turned into
system of linear equations)
Spline models
‰
Thin-plate splines, B-splines, Cubic splines
Fred Bookstein, Morphometric Tools for Landmark Data : Geometry
and Biology
http://www.iog.umich.edu/faculty/bookstein.htm
Spring 2007
HST 6.555
24
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
S. Timoner: Compact Representations for Fast Non-rigid Registration of Medical Images (MIT PhD`03)
Spring 2007
HST 6.555
25
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
S. Timoner: Compact Representations for Fast Non-rigid Registration of Medical Images (PhD`03)
Spring 2007
HST 6.555
26
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Non-parametric models
„
Continuum mechanics ‰
elastic solid models, fluid transport, …
G. E. Christensen* and H. J. Johnson: “Consistent Image Registration.”
IEEE TMI, VOL. 20, NO. 7, JULY 2001
© 2000 IEEE. Courtesy of IEEE. Used with permission.
Spring 2007
HST 6.555
27
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
© 2000 IEEE. Courtesy of IEEE. Used with permission.
Christensen, G. E., et al. “Large-Deformation Image Registration using Fluid Landmarks.”
Image Analysis and Interpretation 2000, Proceedings of 4th IEEE Southwest Symposium, pp. 269 -273 Spring 2007
HST 6.555
28
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Medical Image Registration
Medical image data sets
Transform (move around)
Compare with objective function
score
initial
value
Spring 2007
motion
parameters
Optimization algorithm
HST 6.555
29
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Objective functions
„
„
„
measure how well things are lined up
assumption: the input datasets (U,V) are related to
each other by some transformation T
define: energy function to be optimized
E= f (U ( x ),V (T ( x )))
„
2 main styles:
¾
¾
Spring 2007
feature- or surface-based
intensity-based
HST 6.555
30
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Feature-based measures
„
„
compute alignment quality based upon the agreement of 2 sets of landmark features assumption:
‰
‰
‰
landmarks visible in both images
they can be reliably located and
they can guide the alignment of the whole
image
Spring 2007
HST 6.555
31
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Surface-based measure
„
a simple measure of
alignment
discretized curve points
E = ∑ min xi − y j
i
j
2
= ∑ C 2 (xi )
i
C y ( x) ≡ min x − y j
xi
j
yi
Spring 2007
chamfer function / distance transform
HST 6.555
32
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Chamfer function
C(x): distance to nearest point on curve
∃ fast way to calculate C(x)
on a grid
curve
1
2
3
…
C(x) =
G. Borgefors. Hierarchical chamfer matching: a parametric edge matching algorithm. IEEE
Trans. on Pattern Analysis and Machine Intelligence, PAMI- 10(6):849-865, November 1988
Spring 2007
HST 6.555
33
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
„
if structure is missing, not a robust measure of
alignment; large penalties may swamp the
measure
x
y
Alternative solution:
• “robust chamfer matching”
C y '( x) ≡ min(d 0 , C y ( x))
not close to any point on y
Spring 2007
(penalty saturates)
HST 6.555
34
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
„
feature-based techniques used at the BWH:
‰
‰
„
laser data
locater data
patient skin in MR
similar methods pioneered by
‰
Charles Pelizzari (Dept Radiation Oncology, U.
Chicago)
„
Spring 2007
“Head in Hat” – MR-SPECT/PET registration; extract
surfaces of heads; register the surfaces
HST 6.555
35
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Intensity-based measures
compute alignment quality based upon the
intensity profiles of the input images
„ no landmark or feature selection is
necessary
„
Spring 2007
HST 6.555
36
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
SSE and correlation (I)
„
SSE: sum of squared errors
E= ∑ U ( xi ) − V (T ( xi
))
2
xi
=∑ ⎡⎣U 2 ( xi ) − 2U ( xi )V (T ( xi )) + V 2 (T ( xi )) ⎤⎦
xi
„
1st term: no dependency over T; 3rd term
contributes a constant* ⇒ equivalent problem:
E ' ≡ ∑ U ( xi)V (T ( xi ))
⇒ classical correlation
xi
Spring 2007
HST 6.555
37
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
SSE and correlation (II)
„
For translation only: T(x) = X + D
E ' = ∑ U ( xi )V ( xi + D)
xi
Convolution ⇒ can use Fourier methods…
Spring 2007
HST 6.555
38
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
SSE and correlation (III)
„
CAVEAT:
problems can surface if some structure is
missing or extra-large quadratic penalties
can swamp the measure
E = ∑ U ( xi ) −V (T ( xi
))
2
xi
Spring 2007
HST 6.555
39
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
More robust measures (I)
„
SAV: summed absolute value
E ( x) = ∑ U ( xi ) −V (T ( xi ))
xi
‰
Amy Gieffers, 6A HP Andover*: cardiac
ultrasound registration
„
Spring 2007
*later Agilent, later Phillips
HST 6.555
40
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
More robust measures (II)
„
Saturated SSE
penalty saturates
(
E ( x) = ∑ min d , U ( xi ) −V (T ( xi ))
xi
Spring 2007
HST 6.555
2
)
41
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Multimodal inputs…
Spring 2007
HST 6.555
42
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Multimodal Inputs
„
„
correlation fails for multi-modal registration
when intensities are different; e.g.: MR-CT
one solution:
⇒ apply a special intensity transform to the MRI
to make it look more like CT; then compute the
correlation measure
e.g.: Petra Van Den Elsen
Spring 2007
HST 6.555
43
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
MR-CT situation
bone
white matter
fat
CT
gray matter
CSF
air
MR
Spring 2007
HST 6.555
44
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Histograms
Data
1.2
1.4
1.0
3.4
Bins or buckets
1.0
3.0
10.0
1.3
1.6
0
Counts:
Rel. frequency:
Spring 2007
1
2
3
4
5
6
7
8
9
10
0 6 0 3 0 0 0 0 0 0 1
0
6
10
0
HST 6.555
3
10
0 0 0 0 0 0
1
10
45
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
11
Histogram Joint Intensity of Images
Images:
histogram
U
Y
V
1
2
1
2
2
2
1
2
1
3
3
3
1
2
1
2
2
2
Y
X
V
2
1
2
U
X
intensities
relative freq.
Joint intensities:
(1,2)(2,2)(1,2)(1,3)(2,3)(1,3)(1,2)(2,2)(1,2)
Spring 2007
3
HST 6.555
2
9
1
9
4
9
2
9
46
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
MRI & CT pairs
misaligned
Spring 2007
slightly misaligned
HST 6.555
aligned
47
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Joint histogram: MRI & CT registered
MRI
MRI
CT
Spring 2007
CT
HST 6.555
48
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Joint histogram: MRI & CT; slightly off
MRI
MRI
CT
Correct registration
Spring 2007
CT
Slight mis-registration
HST 6.555
49
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Joint histogram:
MRI & CT; significantly off
MRI
MRI
CT
Correct registration
Spring 2007
CT
Significant mis-registration
HST 6.555
50
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
entropy:
H ( p ( x)) ≡ −∑ p ( x) log 2 ( p ( x))
x
e.g.:
predictable coin no uncertainty,
lowest entropy
biased coin -
moderate uncertainty,
fair coin most uncertain,
moderate entropy
highest entropy
1
p
.5
0
H
Spring 2007
T
H
T
HST 6.555
H
T
51
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Examples of joint intensity distributions
1
4
V
1
4
1
4
1
4
V
U
1
6
1
6
1
6
1
6
1
6
U
H = log 2 (6)
H = log 2 (4)
Spring 2007
1
6
HST 6.555
52
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
“Real” CT-MR registration:
3D starting position
Spring 2007
HST 6.555
53
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
CT-MR registration final result
Spring 2007
HST 6.555
54
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
CT-MR registration movie
From: Wells, W. M., et al. "Multi-modal Volume Registration by Maximization of Mutual Information."
Medical Image Analysis 1, no. 1 (March 1996): 35-51. ��
Courtesy Elsevier, Inc., http://www.sciencedirect.com. Used with permission.
Spring 2007
HST 6.555
55
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
Registration by minimization of joint entropy
≅
Alignment by maximization of mutual information
„
„
„
„
Viola, PhD 1995
Pluim, PhD 2001
many more papers (2002: more than 100)
West Fitzpatrick et al 1998 JCAT
‰
‰
Spring 2007
…clear winner MR/CT…
…also good for MR/PET…
HST 6.555
56
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
„
Chuck Meyer et al. (Radiology Dept, U
Mich.)
‰
‰
‰
Non-rigid mutual information registration
Thin-plate spline warp model
Downhill simplex optimizer
„
„
Spring 2007
rat brain auto-radiograph ↔ CT
breast MRI ↔ breast MRI
HST 6.555
57
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].
END
Spring 2007
HST 6.555
58
Cite as: Lilla Zöllei and William Wells. Course materials for HST.582J / 6.555J / 16.456J, Biomedical Signal and Image Processing, Spring 2007.
MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD Month YYYY].