From: AAAI Technical Report SS-94-05. Compilation copyright © 1994, AAAI (www.aaai.org). All rights reserved.
Matching 3-D Anatomical Surfaces with
Non-Rigid Volumetric Deformations
Sttphane Lavali~
TIMB - TIMC - IMAG,
Facult6 de Mtdecine de Grenoble,
38 700 La Tronche, France
Richard Szeliski
Digital Equipment Corporation,
Cambridge Research Lab,
One Kendall Square, Bldg. 700,
Cambridge, MA02139
Abstract
This paper presents a newmethodfor determiningthe
minimalnon-rigid deformationbetweentwo 3-Dsurfaces, such as those whichdescribe anatomicalstructures in 3-Dmedicalimages.Althoughwematchsurfaces, werepresent the deformationas a volumetric
transformation. Our methodperformsa least squares
minimizationof the distance betweenthe two surfaces
of interest. To quickly and accurately computedislances betweenpoints on the two surfaces, weuse a
precomputed
distance maprepresentedusing an octree
spline whoseresolution increasesnear the surface. To
quickly and robustly computethe deformation,weuse
a volumetricspline to modelthe deformationfunction.
Wepresent experimentalresults on both synthetic and
real 3-Dsurfaces.
Introduction
Thematchingof 3-Danatomicalsurfaces betweenanatomical atlases andpatient data, or between
differentpatientdata
sets, is an importantelementof 3-Dmedicalimageanalysis andquantification.Matching
betweenatlases andpatient
data enablesmoreaccurateandreliable segmentation
andthe
functionallabelingof medicalimages,as well as multimodality data registrationandintegration.In computer
vision, this
problemcorrespondsto finding the non-rigid deformation
betweentwo surfaces, with applications to model-based
object recognitionanddeformableobject tracking.
In previouswork[1 ], wedevelopeda fast and accurate
techniquefor determiningthe rigid transformationbetween
twosurfaces,andalso between
a 3-Dsurfaceandits 2-Dprojections. In this paper, weextendour techniqueto recover
smoothnon-rigid deformationsbetween3-D surfaces. Our
approachis basedon describingthe deformationas a warping of the spacecontainingoneof the surfaces.In particular,
weuse a multiresolutionwarpor displacementfield based
on conceptsfromfree-formdeformations[2]. Ourapproach
enablesus to locally adapt the resolution of the deformation field to bring the twosurfacesinto registration, while
maintainingsmoothnessand avoidingunnecessarycomputation. Theresult is a rapidandefficient registrationalgorithm
whichdoes not require the extraction of features fromthe
22
two surfaces (see [3] for moredetails on our algorithmand
results).
The mainapplication of our technique is model-based
segmentation of 3-D medical images. Segmentingmedical structures is importantboth in medicaldiagnosis and
as a component
of computerassisted medicalinterventions
[4]. Whileunsupervisedsegmentationbased on purely local operatorscan be very difficult, the a priori knowledge
containedin anatomicalmodelscan makethe segmentation
processeasier and morerobust.
Asecondapplicationof our techniqueis in quantification
of normalanatomicalstructures and deviations fromnormal
(morphoraetrics
[5]), e.g., studies of brain asymmetry,
andin
deviationfromself over time, e.g., changesin liver tumors.
In somecases, a volumeregistration betweena patient data
set (e.g., a 3DMRIdata set) and a model(e.g., a brain
atlas) is madepossible by the existence of somecommon
referencesurfacesin bothdata sets (e.g., the surfaceof brain
ventricles).Thisregistrationcanbe usedto infer the location
of specificfeatures(e.g., thalamus
brainnuclei)in the patient
d~t~set. Suchan inferencewouldnot bepossibleiftheelastic
registration wasa surface deformationinstead of a volume
deformation.
Previous work
Theregistration of 3-D and 2-I) imageshas been widely
studied in both medicalimageprocessingand computervision. In medicalimaging,the problemof imageregistration
is usually solvedusing external fiducial markersplacedon
the bodyof the patient[6] or by interactivelyselectingpairs
of matchingpoints [7]. Ourpreviousalgorithmsolvedthis
problemby minimizingthe sumof squared distances betweenthe transformedpoints on one surfaceanda stationary
descriptionof the other surface[ 1]. It also solvedthe more
difficult problemof registering a 3-Dsurface with its 2-D
projections[I].
The registration of 3-Dmedicalimagesunder non-rigid
deformationshas been studied by Bajcsy and Kovacic[8]
usingvolumetricdeformationsbasedon physical properties.
Anotherapproachfrom Evanset al. [9] is based on approximatinga 3Dwarpingfunction by 3Dthin-plate splines
fitted to manuallymatchedreferencepoints. Jacq andRoux
[I0] estimate a global warpingfunction by minimizingan
distance in 3Dimagesusinggenetic algorithms.In comer graphics, non-rigiddeformationsare widelyused for
deling andanimationpurposes[11]. In computervision,
t-rigid deformations
havebeenusedfor fitting flexible
clels to both image[12] and volumetricdata [13]. The
.roach weuse in this paperis basedonfree-formdeformais [2], whichuse volumetric,3-Dtensor-productsplines
lescribe the warpingor displacementof points embedded
he space.
Problem formulation
formulateour deformablematchingproblemas follows.
en a modelobject andsenseddata, estimate the transnation T parameterizedby a parametervector p which
tes the two coordinate systems. Morespecifically, we
~methat both data sets are surfaces,withthe senseddata
"esentedas a collection of points {q~, i - 1...N}, and
modelsurface S representedin somearbitrary way.The
mariontask is then to find a geometrictransformationT
that the transformedcoordinatesri = T(qi; p) all lie
he surface,5.
practice, dueto noise andthe inability to perfectly
ster twosurfaces,this conditionwill neverbe satisfied.
ead, weposethe problemas a minimizationof the cost
:tion
,re d(r,, S) = mina,slit, - ,11is theminimum
Euean distancefromthe point rl to S ando~2 is the variance
~ciatedwithpointi [1].
b solve the minimization
problem,werequire three coments: a suitablerepresentationfor the geometric
transforion T(q~;p), an iterative minimization
algorithm,and
:lent methodfor computing
d(r, S) alongwith its graditbal polynomial deformations
simplestrepresentationfor ’r(qi; p) is a rigid body
sformation whichcan be parameterizedby 6 degrees
reedom
(3 for the translation, and3 for rotation.) In our
,ious work[I], weused the Euler angle representation
the rotation matrixR. Therigid bodyrepresentationis
ropriate whenworkingwith rigid anatomicalstructures
,re onlythe poseof the patient is unknown.
¯ moregeneral class of transformationsare the affine
sforms,whichcan be parameterizedwith 12 degrees of
dom.Thetrilinear class of transformationsaddsanother
tegreesof freedom
for a total of 24. Finally,the q~dratic
ily of transformations
has 30free parameters(see [3] for
Jls). Forall of these transformations,
the positionof the
sformedpoint r~ = T(q~;p) is li nearfunction ofthe
~netersin p, i.e., the transformationcan be written as
: Mip[11].
~1 spline deformations
)brain a widerrangeof moreflexible deformations,we
id continueincreasingthe order of the polynomial.
How-
23
ever, this suffers fromthe sameproblemsas high degree
polynomial
fitting, e.g., instability andthe presenceof ringing. Instead, wemodelthe deformationusing a family of
volumetrictensor productsplines,
T(q,; p) = dj k,Bj(z,)Bk(~,)B,(zi),
(2
j,k,t
wherethe djki are the spline coefficients whichcomprise
the parametervector p, and B~, Bh, and Bt are B-spline
basis functions[2]. Withsplines, the deformations
required
to bringa local areaof twosurfacesinto registrationwill not
affect the registrationat far-awayportionsof the surface.
Forour initial experiments,
wefirst find a set of global
transformations,
starting witha rigid transformation
andthen
fitting an affine transformation.Wethen estimate a local
trilinear splinedeformation.
Tobetter decouple
local elastic
deformationsfromglobal effects such as scale change,we
use bothtransformations
in series.
Least squares minimization
Toperformthe nonlinearleast squaresminimization,weuse
the Levenberg-Marquardt
algorithmbecauseof its goodconvergenceproperties[ 14]. Leastsquarestechniquesworkwell
whenwehave manyuncorrelated noisy measurements
with
a normal(Ganssian)distribution. In orderto updatethe current estimate of the parametersp(k) Levenberg-Marquardt
requiresthe evaluationof the distancefunctiond(ri, S) along
withits derivativewith respectto all of the unknown
parameters. Efficienttechniquesfor computing
the distancefunction
d, as well as its spatial gradientg = Vrd,are presentedin
the next section. Theevaluationof the derivativeinvolvesa
straightforward
applicationof the chainrule.
pOncethe distance samplesd~ and their derivatives 0dr0
have been computed,the Levenberg-Marquardt
algorithm
forms the approximateHessianmatrix Aand the weighted
error gradientvector b [14], andthen computes
an increment
6p towardsthe local minimum
by solving
(A + ),l)Sp (k) = b,
(3)
where~ is a stabilizing factor whichvaries over time [14].
Aftersetting p(k+l)= p(k) + Ap(S,),this processis repeated
until C(p)is belowa fixed threshold,the differencebetween
parametersIp(k) - p(k-l)[ at two successiveiterations
belowa fixed threshold, or a maximum
numberof iterations
is reached.
Whenthe numberof parametersbeing estimated is reasonablysmall(as in global deformations),weuse the GaussJordaneliminationalgorithm[14] to solve (3). For systems
with moreparameters, such as local deformations,weuse
conjugategradientdescent[ 14].
Fast distances using octree splines
Themethoddescribedin the previoussection relies on the
fast computation
of the distanced(r, S) andits gradient.
speed up this computation,weprecomputea 3-Ddistance
map,whichis a function that gives the minimum
distance to
$ fromanypoint r inside a bounding
volumeVthat encloses
S[151.
~:"~p~5:’£.:
~:.....
.~...’:. ~.. .~.~
- .:.~-~
, ~:~.;-.:.
": ~;; ..-
~
~::.j..!~
c
b
d
Figure1: Registrationof twoface ~ta sets:
(a)model
da~.
set(george1)
Co)sensor
da~set(heidi)
(c)final
deformed
sensor
d~t~(d)final
registered
a
In looking for an improvedtrade-off betweenmemory
¯ see that the two dat~ sets are registered well, except for the
space, accuracy, speed of computation,and speed of coneyebrows,whichwouldrequire a moredetailed deformation.
slniction, wedevelopeda newkind of distance mapwhich
Wealso note that the deformedface of Figurelc resembles
wecall the octree spline [1]. Theintuitive idea behindthis
that of Figure la morethan its former(undeformed)self
geometricalrepresentationis to havemoredetailed informa(Figurelb).
tion (i.e., moreaccuracy)nearthe surfacethanfar awayfrom
As a second exampleof our algorithm, we matchedthe
it. Westa/’t withthe classical octreerepresentationassocisurfaceof a real patient vertebrato the surfaceof a plasated with the surface5 [16] andthen extendit to represent
tic "phantom"vertebra (both 3-Dimagessets wereacquired
a continuous3-Dfunction that approximatesthe Euclidean
with a CTscanner).Figure2 showsthe result of our matchdistance to the surface. This representation combinesading. After affine registration, a fair amountof discrepancy
vantagesof ~,ptive spline functions andhierarchical data
remains.After the local spline registration, mostof the pastructures(see[ 1, 3] for details).
tient vertebra (contour lines) matchesthe phantommodel
(cloudof dots), exceptfor the tips of the vertebrawhichhave
Experimental
results
not beenpulledinto registration.
Todeterminethe reliability of our global and local deforDiscussion and Conclusions
mationestimates, wefirst performed
a series of experiments
on both real and synthetic surfaces undersimulated(known)
Wehavedevelopeda techniquefor registering 3-Dsurfaces
with non-rigid deformations.Ourapproachrepresents the
motion. For a given surface, wefirst computethe octree
distancemap.Next, weselect a subset of the surfacepoints
deformationusing a combinationof global polynomialde(typically 5%)and transformthese throughthe inverse
formationsand local displacementsplines (free-formdeforthe deformationweare simulating. Wethen initialize the
mations). Weuse an algorithm whichdirectly minimizes
Levenberg-Marquardt
algorithmwith someinitial rigid pose
the squareddistances betweenthe twosurfaces, rather than
estimateandset the non-rigidparameterestimatesto O. Fiidentifyingsparsefeatures(e.g., ridge lines or featurepoints
nally, werun the Levenberg-Marquardt
algorithmin stages,
[17]) andthen trying to matchthem.
first computing
the best rigid match,thenthe best globalnonWebelieve that the direct matchingof surfaces has betrigid match,andfinally the best local displacement
warp.
ter accuracyand removesthe needfor a feature detection
To demonstratethe local non-rigid matching,weuse two
stage, whichmaynot always operate reliably. Twoargusets of range data acquired with a Cyberwarelaser range
mentswhichfavored feature-based approachesin the past
scanner(Figures la andlb). Theoctree spline distance map
were computationalcomplexityand global correspondence
is computed
on the larger of the twodata sets (Figure la),
search. Usingthe octree spline distance map,the complexity
andthe smallerof the twodata sets is deformed
(Figurelb).
of eachiterative adjustment
step in our algorithmis linear in
In their initial positions, the data sets overlapby about50%
the numberof sensed surface points. In our approach,we
and differ in orientation by about 10°. Wefirst perform
use a modifiedgradient descentwhichavoidscombinatorial
8 iterations of rigid matchingand8 iterations of non-rigid
search but only finds locally optimalmatches.In practice,
affine matching.Wethen perform8 iterations at each of
wehavefoundthat false local minimaare usually far away
4 levels of the local displacementspline. Thefinest ocl~ee
fromthe true solution. In medicalapplications, a priori
spline level has (24 + 1)3 ~ 5000nodesfor a total of about
knowledge
aboutposition andshapeis usuallyavailable, so
15000degrees of freedom.Evenwith this large numberof
that only small displacementsand local deformationshave
parameters,the algorithmconvergesvery quickly,becauseit
to be estimated.
is alwaysin the vicinityof a goodsolution(a typicaliteration
Our approachembedsone of the surfaces being matched
at the finest level takes about2 seconds).FromFigureld, we
into a deformablespace, rather than equippingit directly
24
." ¯ - ""* ....
..
:.
IL
a
b
C
Figure2: Registration
of patientvertebrawithplastic "phantom":
fter affineregistration
Co)afterfinal localsplineregistration
(c) selectedlines in deformation
spline
elastic properties.Whilethe latter approach
maybe
physicallyrealistic if an anatomically
andbiomechany correctmodelis constructed,ourapproach
enablesus
witharbitrarysurfaceswhichmaynot evenhavea
~th connectedrepresentation.Volumetric
deformations
yield auxilliaryinformation
aboutthe motionof nearby
:tures whichdid not participatein the matching,
e.g.,
;trationsperformed
oncertaineasily identifiablebrain
:tures canbe usedto estimatethe registrationfor the
le brain.
ur experimentsto date havebeen performed
on pre~ented3-D surfaces. Weare planningto extendour
rithmto workdirectly with unsegmented
3-Dmedical
~es. Otherareasof futureinvestigationincludeanadap~lgorithm
for decidinghowto refinethe local octreederationspline,the choiceof theorderof theoctreespline,
Lhechoiceof varioussmoothness
constraints(regularizReferences
S. Lavall~e, R. Szeliski, and L. Brunie. Matching3-D smooth
surfaces with their 2-D projections using 3-Ddistance maps.
In SPIE Vol. 1570 Geometric Methods in ComputerVision,
pages 322-336, San Diego, CA, July 1991.
T.W. Sederberg and S.R. Parry. Free-form deformations
of sofid geometric models. Computer Graphics (SIGGRAPH’86),20(4):151-160, 1986.
R. Szeliski and S. Lavali~e. Matching 3-D anatomical surfaces with non-rigid deformations. In SPIE Vol. 2031 Geometric Methodsin ComputerVision II,pages 306-315, San
Diego, July 1993. Society of Photo-Optical Instrumentation
Engineers.
S. Lavall~e, L. Brunie. B. Mazier, and P. Cinquin. Matching
of medical images for computerand robot assisted surgery.
In IEEE EMBSConference, pages 39-40, Orlando, Florida,
November 1991.
E L. Bookstein. Principal warps: Thin-plate splines and
the decomposition of deformations. IEEE Transactions on
Pattern Analysis and MachineIntelligence, 11(6):567-585,
June 1989.
[7]
[8]
[9]
[10]
teractive surgery. In SPIE, Medical Imaging, VoL767, pages
27-35, 1987.
C. Schiers, U. Tiede, end K.H. Hohne. Interactive 3Dregistration of imagevolumes from different sources. In H.U.
Lemke,editor, ComputerAssisted Radiology, CAR89, pages
667--669, Berlin, June 1989. Springer-Veriag.
R. Bajcsy and S. Kovacic. Multiresolution elastic matching.
ComputerV’tsion, Graphics, and ImageProcessing, 46:1-21,
1989.
A. C. Evans, W.Dai, L. Collins, P. Neelin, end S. Marett.
Warpingof a computerized 3-d atlas to match brain image
volumesfor quantitative neuroanatomicaland functional analysis. In SPIE Vol. 1445, Medical ImagingV, pages 236-247,
1991.
J. J. Jacq and C. Roux. Automatic registration of 3Dimages using a simple genetic algorithm with a stochastic performance function. In IEEE Engineering Medicine Biology
Society (EMBS),pages 126-127, San Diego, October 1993.
[11] A. Witkin and W. Welch. Fast animation and control of
nonrigid structures. Computer Graphics (SIGGRAPH’90),
24(4):243-252, August 1990.
[12] D. Terzopoulos, A. Witkin, and M. Kass. Constraints on deformable models: Recovering 3Dshape and nonrigid motion.
Artificial Intelligence, 36:91-123,1988.
[13] B. Horowitz and A. Pentland. Recovery of non-rigid motion and structure. In IEEEComputerSociety Conference on
Computer~rtsion and Pattern Recognition (CVPR’91),pages
325-330, Maul, Hawaii, June 1991. IEEE ComputerSociety
Press.
[14] W.H.Press, B. P. Flarmery, S. A. Tenkolsky, and W.T. Vet,
terling. NumericalRecipes in C : The Art of Scientijic Computing. CambridgeUniversity Press, Cambridge, England,
secondedition, 1992.
[15] P.-E. Danielson. Euclidean distance mapping. Computer
Graphics and Image Processing, 14:227-248, 1980.
[16] H. Samet. TheDesignandAnalysisofSpatialDataStractures.
Addison-Wesley,Reading, Massachusetts, 1989.
[17] A. Gu~ziec and N. Ayache. Smoothing and matching of 3-d
space curves. In G. Sandini, editor, SecondEuropeanConference on Computer Vision (ECCV’92), pages 620-629, Santa
Margherita, Italy, May1992. Springer Verlag, LNCSSeries
Vol. 588.
B.A. Kall, P.J. Kelly, and S.J. Goerss. Comprehensive
computer-assisted data collection treatment planning and in-
25