From: AAAI Technical Report SS-94-05. Compilation copyright © 1994, AAAI (www.aaai.org). All rights reserved.
Medical Image Registration using Voxel Similarity Measures
tDerek LGHill & David J Hawkes
1.
Introduction
1.1.
Patients frequently undergomultiple radiological imaging
investigationsthat providecomplementary
information.It is
becomingincreasingly widely recognised that the use of
imageprocessing techniques to combinethis information
into a single representationof the patient can assist in the
interpretation of the relevant radiological information[1].
Thedevelopment
of clinically useableautomatic3Dmedical
imageregisuation algorithms is therefore an important
researcharea.
Medicalimagesof the sameregionof a patient acquiredwith
different imagingmodalitiesare usually recognisablysimilar, even to a non-expert. For example,MRand CTimages
ofmanyparts ofthe bodycontain broadlythe samefeatures,
andobserverswhoare newto these types of imageswill frequentlyrecognisethe part of the bodythat has beenimaged,
but mistakethe modality. In these visually-similar images
fromdifferent modalities, imagefeatures in the different
modalitieswill appearwith different intensity andtexture,
and someimagefeatures visible in one modality will be
absent from another modality. There are important exceptions: projectionangiograms
(especially subtractedones)are
difficult to relate to cross-sectionalimagesof the samepart
of a patient becamethey share almost no imagefeatures due
to their different dimensionality;nuclear medicineimages
acquiredusingvery specific functionaltracers providelittle
anatomicalinformationto relate to other modalities.
Cross Correlation of Intensity Values
The cross correlation of two functionsis frequentlyused in
signal processingand imageprocessingas a measureof how
well two functions A and B matchup whenthey are transformedwith respect to each other. ThetransformationT that
providesthe best matchbetweenthe functions has the highest con’elationvalue.
For two3Dreal valuedimagesAand B with p x q x r voxels,
the correlation value for a Wansformation
Tis given by equation 1.
p-lq-lr-I
[A~B]r-- ~, ~_~_Aqt. T(Bii
k)
(I)
i=Oj=Ok=O
To determinethe transformationthat most closely matches
the twofunctions,it is necessaryto find the valueof T that
maximisesthe right hand side of this equation. Thetransformedvalueof Bi]kis unlikelyto lie ona samplepoint in the
data, so interpolation is normallyused in the evaluationof
T(Bok).
Published work in medicalimageregistration using cross
correlation of intensity valueshas beenless successfulthan
alternative approaches, such as landmarkregistration and
surface matching.Tworecently developedtechniques, which
are closely related to cross correlation, havebeenmuchmore
successful.
1.1.1 Cross Correlation of Image Gradients
Wherevisually-similar images are being combined,image Vanden Elsen [7] proposedan algorithmfor imageregistraregistration can be accomplishedby manually,or perhaps tion by correlation of imageintensity gradients. In her algorithm, the secondderivative of the imagein the direction
automatically,identifying a small number
of equivalentfeatures suchas points or surfacesin the images[2,3,4,5,6]. An normalto the local intensity gradientis usedas a ridge detector. Bychoosingan appropriatescale at whichto calculate
alternativeapproach
is to performregistrationusingall, or at
the derivative, the bone ridge from MRand CTimagescan
least a large number
of the voxelsin the imagesrather thana
be
automaticallyextractedfromthe images,and the resulting
smallnumberof derivedfeatures. Thebasis of this approach
is the assumptionthat somearithmetic combination
of voxel gradient imagescan be cross correlated to determinethe correct registration transformation.
values in two images,whenappfied to each imagevoxel in
turn, provides a similarity measurethat has an optimum
1.1.2 Minimisingthe Coefficient of Variation of Intenvalue whenthe imagesare aligned. The best knownalgosity Ratios
rithm of this type is cross correlation. This paper reviews
RogerWoods
[8,9] has proposedan algorithmthat is related
algorithmsusing voxel similarity measuresthat havei~en
to
cross
correlation,
but incorporatesan importantmodificaapplied to medical images, and presents new work on a
tion.
His
algorithm
is
basedon an idealised assumptionthat
methodology
for devising improvedsimilarity measures.
states: if twoimagesare accuratelyaligned, thenthe valueof
anyvoxel in one imageis related to the value of the corretRadiologicalSciences,UMDS,
Guy’s&St Thomas’
Hospitals,
spondingvoxel in the other imageby a multiplicative factor
St Thomas"St. LondonSEI 9RT, UK. D.Hill@umds.ac.uk R. In other words,for all voxelsai andbi in imagesAand B
Fundedby the UKScienceand EngineeringResearchCouncil.
34
2.2.
Qi
~spcctively, ~ = R. WhenA and B am acquired from the
;ame patient using the samemodafityat different times,
here will be a single value of R for all intensity values.
0VhenAandB are acquiredfromthe samepatient using dif’erent modalities, there mightbe a different valueof R for
¯ ach intensity value in either image.TheWoods
algorithm
msbeendevelopedspecifically for registration of multiple
~ETimagesfromthe samepatient, and for registration of
’ETimagesto MRIimagesof the samepatient. Clearly, the
dealisedassumption
will not holdin either of these applicaions, but, if R is moreuniform(has a lowervariance 2)
vhenthe imagesare in registration than whenthe imagesare
,ot, and if 02 increases as the degree of misregistration
ncreases, then imageregistration can be accomplishedby
ainimisingthe coefficientof variationof the intensityratios.
Vehavepreviously shownhowthis techniquecan be modited in order to automaticallyregister MRandCTimagesof
~e head,providedthere is sufficient axial sampling[10].
"he successof these techniquesfor solvingspecific registraon problemsencouragedus to devise a methodologyfor
arther investigatingvoxelsimilarity measures.
’,.
Similarity measure plots
Aquantitative indication of the performance
of voxel based
registration algorithmscan be gainedby studyingthe wayin
whichthe similarity measurechangeswith misregiswafion.
Usingregistered referenceimages,the similarity measureis
evaluatedfor the imagesat registration, andwhenmisregistered by knowntransformations in each of the degrees of
freedomof the desiredregistration transformation.For rigid
bodyregistration, this providesa series of six one dimensional curves, eachof whichis a plot of similarity measure
valueagainst misregistrationfor a single degreeof freedom.
Weterm the resulting graphssimilarity measureplots. The
similarity measuresare formulatedas cost functions, so an
ideal similarity measureplot has a minimum
value at registration, and is a monotonically
increasing function of misregistration. It must be emphasisedthat these similarity
measureplots do not sampleall of the parameterspace, and
there are likely to be manylocal minima
(or eventhe global
minimum)
that are not visible in the similarity measure
plots.
2.3.
Initial
Evaluation of Similarity Measures
These techniques have been used to evaluate similarity
measureson preregistered MR,CTand PETimages.
3. Results
Method
Ve have accurately registered manydozens of medical
nages using anatomicallandmarks[1]. This provides us
,ith a large number
of referencedatasetswith whichto evalate alternative registration algorithms.Wehavedevisedtwo
:chniquesfor assessingpossible voxelsimilarity measures:
¯ ,ature spacesequences
andsimilarity measure
plots.
3.1.
Feature Space Sequences
ides a wayof studyingthe changein the appearanceof a
.ature space with misregistration. Alternative similarity
teasures canbe calculateddirectly fromthe feature spaces.
A
l similar feature space sequence generated from T
weightedMRand CTimagesof the skull base is shownin
figure2.
A feature space generatedfromtwoidentical, perfectly registered images,
isa line ofunit
gradient. Theappearanceof
the feature space changesin a similar waywith misregistralion in eachdegreeof freedomin turn. Thedistinctive diagonal line gradually blurs in the horizontal and vertical
directions, Horizontalandvertical lines appear,intersecting
.1. Feature Space Sequences
at peaksin the original feature space, oncethe misregistra¯ qualitativewayof consideringthe effect of misregistration tion is sufficiently great that a givenintensity valuein one
n voxel similarity measures,is to use feature spaces. For imagecan underlie any intensity value in the secondimage.
~e workpresentedhere, feature spacesare constructedfrom Feature space sequenceswere also generatedfor other more
nageintensifies. Extendingthis workby generatingfeature relevant imagecombinations.Afeature space sequencewas
paces fromimagegradients or texture mightproveuseful.
generated from two similar but not identical MRimages.
, feature spacesequenceis a series of feature spacescalcu- One MRimagewas generated from the original by adding
tted from a pair of imagestransformedrelative to each Gaussiannoise with a standard deviation similar to the
standarddeviationof the air in this image,followedby addther. Oneimagein the sequenceis calculated whenthe
ing an offset of 100 to each voxel value. The second MR
nages
arecorrectly
registered,
theothers
arecalculated
’ithknown
transformations
ina chosen
degree
offreedom.imagewasgeneratedfromthe sameoriginal by setting the
¯ feature
space
sequence
canbecalculated
foreach
degree left mostthird of the voxelvaluesto O. Theresulting feature
f freedom
of the rigid bodytransformation:
It thereforepro- spacesequenceis shownin figure 1.
35
These plots demonstrate that the cost increases almost
monotonically
with misregistrafionin eachof the degrees of
freedom.Theequivalentplots obtainedusing cross correlation andthe coeffecientof variation of intensity ratios conrain local minimain each degreeof freedom.
le-04
Rgure1. A feature space sequencegenerated
fromtwosimilar MRimages
registered(a) andtranslatedlaterallyby3ram
(b), 9ram
(c) and25ram
8e.05
E
R~6e-05
0
i
4e-05
¯ ~,2e-05
O
Rgure2. A feature space sequencegenerated
fromMRandCTimages
registered(a) androtated
2.5"(b), 7.5"(c) and20"(d) abouta cranio-caudal
axis.
Thet’eature space sequencesshownin ligures 1 and 2, and
others gennratedfrom alternative imagecombinations,look
quite different. However,
there are common
characteristics:
1.
diagonalfeaturesin the imagesat registration disperse whenthe imagesare misregistered.
2.
exceptat the origin, the highestintensitypixels get
less bright with misregistration.
3.
the number
of lowintensity pixels increaseswith misregistration
horizontalandvertical lines appearin the feature
4.
spaceswhenthe imagesare significantly misregistered.
Wedevised a newsimilarity measure,designedto be sensifive to someof the changes in feature space appearance
listed above:the third order moment
of the intensity histogramof the feature space. Thehistogramof the feature space
contains informationaboutthe distribution of feature space
intensities. It is weightedtowardshigh values if a small
numberof feature space pixels havehigh intensity, and is
weightedtowardslow values if a large numberof feature
space pixels have low intensity. Thehigher order moments
of this histogramquantify its distribution. Wechoseto use
the third order moment
of the feature space histogram,but
the choiceof this rather than anyother moment
of order 2 or
abovewasarbitrary.
Figure3 showsthe similarity measureplots for pre andpost
GadoliniumMRimages. These imageswere acquired in the
normalclinical routine. The patient wasremoved from the
MRscanner for injection of contrast betweenacquisitions,
and the voxel dimensionsweredifferent in the two images.
36
o
0°+0030-20 - 10
0
10
20
30
mismgisb’aUon - ~’~slat~n (ram) or m~fion (degrees)
Rgure3. A similarity measureplot calculated
using the 3rd ordermoment
of the feature spacehistogram, for two T1 weightedMRimages(pre and post
Gadolinium).
Pie and post GadoliniumMRimageshave been successfully
registered automaticallywith this similarity measureusing a
genetic algorithm[11] with a populationsize of I00 and 30
generations,at twoscales.
4.
Discussion
In order to register imagesusingequivalentor similar features it is first necessaryto identifythese features. Registration algorithmsof this type require considerableinteraction
froma trained operator, becauseautomaticsegmentationand
labelling of anatomicalfeatures in medicalimagesremainsa
difficult problem.Registration algorithms that use voxel
similarity measurespotentially overcomethis difficulty by
using imagevoxels rather than derived geometricfeatures
for registration. Both RogerWoodsat UCLA
and Petra van
den Elsen at Utrecht have implementedalgorithms using
voxel similarity measuresand successfully registered MR
and PET, and MRand CTimages respectively. The former
technique requires presegmentationof the brain from the
MRimages, and the latter has only beenapplied to images
with considerablyhigher resolution than those that are roufinely acquiredclinically.
Wehave previously demonstratedthat RogerWoods’algorithm can be extended to the registration of MRand CT
images,by modifyingthe algorithmto use only voxel inten-
dries within certain ranges. However,the algorithm failed to
"egister imageswith insufficient axial sampling.
Wehave devised a methodologyfor further evaluation of
~oxel similarity measures,in particular the generationof feame space sequences. All the feature space sequences proluced shared common
characteristics that might be used for
egistration. The coefficient of variation of intensity ratios
algorithm makes one dimensional measurementsin feature
:pace: it calculates the coefficient of variation alongthe ordiDate axis. Oneconsequenceof this is that in MRand CTregstration, there is a high cost associated with soft tissue from
¢IR overlying bone from CT, but there is not a correspondng high cost for soft tissue from CToverlying bone from
dR. An algorithm that operates on both dimensions of the
eamre space might be more reliable.
Ye devised an alternative measureof the change in appearnee of the feature space images. This measure, the third
,rder momentof the feature space histogram, was successally used for the automatic registration of MRimages pre
nd post injection of Gadolinium.This is an important appliation of image registration, as subtraction of these images
an provide useful clinical information [12], and patients
ormally movebetween~theacquisition of these sequences.
: is an exampleof a large class of imageregistration prob.’ms, in whicha time series of images need to be related.
)bvious examples include monitoring disease progression
¯ .g: plaque volumein patients with multiple sclerosis), and
orrecting for movement
artifacts in functional MR[13].
earching parameter space for the minimumevaluation of a
oxel similarity measureis difficult. There can be an enorious numberof local minima. Even if the similarity messre plots suggest that the similarity measure increases
mnotonically with misregistration, there can be local
dnima, or even a global minimumin an unmeasuredpart of
ammeterspace, One reason for this is that the similarity
teasures tend to assign a high cost to air overlying tissue.
or axial images, misregistration caused by translations in
te lateral and posterior-anterior directions, and rotations
bout all three axes lead to air overlying tissue. However,
tany incorrect transformations will also reduce the amount
f air overlying tissue, leading to a local minimum.
lore workis neededto devise and test appropriate similarity
~easures, but the approach showsgreat promise of prodnctg an accurate, automatedmethodfor registration of voxel
ttasets in 3Dmedical imaging.
¯
References
Hill DLG,HawkesDJ, Hussain Z, Green SEM,Ruff CF,
Robinson GP. Accurate combination of CT and MRdata of
the head:Validation and applications in surgical andther-
37
apy planning. Computerized Medical Imaging 17:35%362
1993
2. Hill DLG,HawkesDJ, Crossman JE, Gleeson MJ, Cox
TCS,Bracey EECML,
Strong A J, Graves P. Registration of
MRand CT imagesfor skull base surgery using point-like
anatomicalfeatures. Br J Radiology.64:1030-1035.1991
3. EvansAC,Marrett S, Torrescorzo J, KuS, Collins L. MRIPETCorrelation in Three DimensionsUsing a Volume-ofInterest (VOI)Arias. J CerebBloodFlow Metab; 11 :A69A78. 1991
4. Pelizzari CA,Chert GTY,Spelbring DR, Weichselbaum
RR,
ChenC-T. Accurate three dimensional registration of CT,
PETand~or MRimages of the brain. J ComputAssist
Tomogr13:20-26. 1989
5. Jiang H, RobbRA,Holton KS. Newapproachto 3-1) registration of multimodality medical imagesby surface matching. In: Visualisaation in BiomedicalComputing,Proc Soc
Photo-opt Instrum Eng 1808:196-213.1992
6. Hill DLG,HawkesDJ. Medical image registration using
knowledgeof adjacency of anatomical structures. Image
and Vision Computing
12 (3) 1994(in press)
7. van den Elsen PA. Mnltimodality matching of brain
Images.Utrecht University Thesis 1993.
8. WoodsRP, Cherry SR, Mazziotta JC. A rapid automated
algorithm for accurately aligning and resliceing PET
images. J CompAssis Tomogr16:620-633 1992
9. WoodsRP, Mazziotta JC, Cherry SR. MRI-PETregistration
with automated algorithm. J CompAssis Tomogr17: 536346 1993
10. Hill DLG,HawkesDJ, Harrison N, Ruff CF. A strategy for
automatedmultimodality registration incorporating anatomical knowledgeand imagercharacteristics. In: Barrett
HH,Gmitro AF, eds. Information Processing in Medical
ImagingIPMI’93. Lecture Notes in ComputerScience 687
Springer-Verlag,Berlin. pp 182-196.1993
11. GoldbergDE. Genetic algorithms in search optimisation
and machinelearning. AddisonWesley, Mass. USA.1989
12.Lloyd GAS,Barker PG, Phelps PD. Subtraction gadolinium
enhancedmagnetic resonance for head and neck imaging.
Br J Radiology 66:12-16 1993
13.Hajnal YV,Oatridge A, SchwiesoJ, CowanFM,YoungIR,
BydderGM.Cautionaryremarkson the role of veins in the
variability of functional imagingexperiments. Proc. SMRM
p166 1993