Methods for elastic adaptation of segmented volumes of
interest for adaptive radiotherapy planning
Urban Malsch, Christian Frühling, Rolf Bendl
Department of Medical Phyiscs, German Cancer Research Center (DKFZ), 69120 Heidelberg, Germany
Abstract
In this paper we present an approach to calculate displacement vector fields between planning CT image series and verification CT
image series acquired during fractionated radiotherapy. After successful calculation these vector fields can be applied to all
volumes of interest segmented for initial treatment planning. This way it is possible to deduce volumes of interest which conform to
the verification image without conventional time consuming segmentation. Fast generation of a precise model of the patient’s
anatomy is considered to be an inevitable pre-requisite for adaptive radiotherapy, which should allow evaluation and adaptation of
irradiation parameters based on verification images immediately before of each fraction.
Keywords
Template Matcher, Optical Flow, adaptive Radiotherapy
Introduction
Conformal radiotherapy together with three-dimensional
inverse and forward planning has shown that precise adaptation
of dose to the shape of the target volume can allow dose
escalations in the tumor and in this way tumor control
probability can be increased while severe side effects can be
reduced simultaneously. Even if it is technically possible to
adjust dose distributions very precisely to a given shape, it is
still necessary to add security margins around the tumor to
compensate for deviations in size, shape and position of organs
during therapy.
Those deviations are the result of positioning errors, organ
movements due to variations in filling of hollow organs
(bladder, rectum), tissue dependent reaction on radiation and
motion artifacts e.g. caused by respiration during therapy.
The goal of adaptive radiotherapy is to consider those
deviations which are quite common during fractionated
radiotherapy. Several levels of adaptivity were proposed so far.
One strategy is to calculate more individualized security
margins by measuring and estimating organ motions based on
a couple of different image series acquired before therapy or
during the first fractions. The methods described in this paper
are designed to support a more complex level of adaptivity.
The goal of our approach is to measure deviations before each
fraction and to compensate them directly by patient
repositioning or by modifying the beam setup. That means,
after having acquired a verification image series major
planning steps must be repeated before each fraction. Since
delineation of anatomy is a time consuming task - but always
necessary to enable plan evaluation based on dose-volumehistograms - we have focused our efforts on methods to speed
up this crucial preprocessing step. To be able to solve this
segmentation task in a very short time frame, our strategy is to
calculate deviations between verification and planning CTs and
to apply the calculated displacement vector fields to the already
segmented volumes of interest. The hypotheses is, that this
approach of adjusting the already generated three-dimensional
patient model is faster than repeated conventional delineation
of the anatomy based on the verification images. To allow
adjacent plan evaluation and modification before applying the
daily fraction the adjustment of volumes of interest should not
exceed more than 10 minutes.
To solve that difficult task we have examined several
concurrent strategies and we are presenting an approach which
is based on a combination of two promising ones.
Material and methods
The first approach which we have investigated is the estimation
of a motion or displacement vector field between planning and
verification CTs by calculating the optical flow between the
different image series [1]. While the optical flow approach tries
to track all image gradient information over the time, the
second approach tries to identify a restricted number of small
corresponding sub-volumes (“templates”) in both data sets [2].
After calculating displacement vectors between these
corresponding sub-volumes a global displacement vector field
can be derived by applying some interpolation strategies.
Since both methods have inherent advantages and
disadvantages a combination of both should produce more
reliable results. To optimize start conditions for both strategies
we first apply a rigid transformation either calculated based on
evaluation of stereotactic markers or on the maximization of
Mutual Information [3].
Optical Flow
Optical flow is an elegant method for calculating deviations
between consecutive images by solving a system of partial
differential equations. The equation system describes the
motion by analyzing the changes in image gradient information.
That means image intensities are differentiated in 3D space and
time.
If f(k,0) is a image at time t0 we are looking for a time
dependent vector field u(k,t) which solves the equation
f(k - u(k,t), 0) = f(k, t)
The displacement vector field u(k,t) can be determined exactly
under several conditions:
1. Image structure is linear: f (k  u )  f (k )  u  f
2. locally constant:   0 with u(l )  constl  Be (k )
3. sufficient image structure:
Rank ( M ( f ,  ))  3 , with M ( f ,  ) 
 f (f )
T
Be ( k )
If these conditions hold, u can be estimated by:
u2
f (k )  g (k )
f   g
2
(f  g ) with =g(k) = f(k, t)
While the first two conditions are normally fulfilled, the main
problem is condition three, which can also be deduced from the
equation above. To get a solution we need sufficient gradient
information, that means optical flow cannot be calculated in
homogenous or nearly homogenous regions and only the
spatial components in direction of the gradients can be
determined locally.
Motion estimations with optical flow are especially reliable for
small movements, if movements exceed the size of moving
objects corresponding gradients are not mapped properly and
the calculated displacement or motion vectors are wrong.
Hierarchical optical flow
To overcome these problems, we have implemented a
hierarchical approach. In a preprocessing step we are
calculating Gaussian pyramids for both data sets [4], that
means we resample the image series with lower resolutions. In
images with lower resolution homogenous regions are reduced
resp. eliminated and this way an coarse estimation for the
displacement vectors can be established. This estimation can be
used to fill in gaps where no results can be obtained at a higher
resolution. Of course, flow calculations on low resolution
images will introduce errors in the size as well as in the
orientation of the calculated vectors but this information is
used only, if it is not possible to calculate more precise values
at finer resolutions.
The Template Matcher
The global strategy of this approach is quite different. A global
displacement vector field is estimated (interpolated) based on a
small number of displacement vectors which are determined by
identifying small corresponding sub-volumes in both data sets.
The general procedure can be divided into four individual
steps:
In the first step, a small number of suitable regions in the
planning dataset will be identified. These regions are called
“templates” [2]. In the second step, they will be continuously
relocated and/or rotated, to find the best correspondence in the
verification dataset. As a result of this iterative search a motion
vector is assigned to each template. The pool of all motion
vectors describes an elastic transformation between both
datasets. Based on this incomplete vector set a displacement
vector for each single voxel is interpolated in the third step.
After the complete vector field is determined the volumes of
interest, segmented on the planning data set are adjusted
according to that vector field and this way they should fit to the
new situation. The four steps will be explained in detail:
Selection of suitable template positions:
The quality of the outcome of this approach and its
performance depends strongly on the number and position of
the selected templates. A low number of templates cannot
describe complex deformations. A large number of templates
will result in excessive computation time. Unsuitable regions
cannot be reliably matched and will result in wrong movement
vectors.
Our strategy is to prefer positions on the edge of organs or
tissues, since their movements can be observed more easily. To
avoid clustering of templates, a minimal distance between the
templates is required. Templates inside organs or inside regions
with homogenous intensity are difficult to match and are
rejected. Since it can be assumed that segmentations on the
initial planning data set was based on perceivable image
structures, the corresponding points are promising candidates.
But of course a restriction to only those points will often not
sufficiently describe a complex transformation.
A raw identification and localization of edges of organs or
tissues borders is a prerequisite for template positioning. For
that purpose we have developed a fast algorithm which
identifies edges between tissue classes. Different types of
tissue (fat, muscle, bone, etc) are classified according to the
associated Hounsfield-units in the CT image series using either
static or dynamic thresholds, calculated on a global intensity
histogram. To generalize this registration method and to use it
to register series acquired with other image modalities
corresponding intensity levels must be determined.
After the classification of tissues, edges are easily identified by
evaluating the proximity of each voxel. An edge voxel is
characterized by less than four (or eight) neighbors within the
same tissue class.
Upon identification of edges template positions are determined.
To get a reasonable number of templates and to avoid
accumulations, the condition of the minimal distance between
templates was introduced. That means a candidate position is
added only to the list of templates, if no other template in a
given neighborhood was already selected. To reduce the
number of templates further, possible template positions are
neglected, where only minimal motion is observable.
Two methods for a fast estimation of movements were
implemented: The first method compares the perpendicular
intensity profiles of corresponding edges in both image
sequences and identifies that position where the correlation
between both profiles becomes maximal. The necessary
translation is used as an estimation of motion. The second
method simply identifies the distance to the next edge with the
same tissue class in the second dataset. By using this
information the number of templates can be reduced
significantly, but due to reasons during final interpolation
which are described below, it is recommended to keep some of
the templates in regions where no movements can be observed.
The last criterion in selecting template positions are the local
contrast of possible candidates. To be able to identify them
easily in the second image the candidates with a high local
contrast should be preferred.
Bladder
Left femur
Right femur
Rectum
Figure 1: Template-matcher. A original planning dataset, B original verification dataset (one slice of a 256×256×52 CT-Volume, which shows
bladder, prostate, rectum and both femurs). C a raw segmentation of edges from A. The bright dots highlight template positions. D the
segmented edges from B. E is the unmatched superposition of the contours displayed in D onto the image displayed in A. F shows the edges of
the verification dataset D projected onto the transformed planning data set. Bladder and muscles fit quite well, rectum was adjusted but shows
still some deviations, largest deviations at femurs due to missing placement of templates.
Search for corresponding template positions:
After the initial template selection, the corresponding regions
in the verification dataset must be identified. We use a local
correlation coefficient as a similarity measure.


 Ax   A B x   B
T
CC 
i

xi TA , B
i
T



 Ax   A   B x   B
2

xi TA , B
i
T

xi TA , B
i
T

2
The correlation coefficient will reach its maximum in case of
positive linear dependency (dark/bright regions in image BT are
projected to dark/bright regions in image A). In an iterative
optimization procedure each template is translated and rotated
until the similarity measure reaches a maximum. Therefore an
optimization in six variables (tx, ty, tz, and rx, ry, rz) must be
performed. In our implementation we use Powell’s approach to
determine that point where our optimization function will reach
its maximum [5].
Transformation vectors for each voxel:
After the motion vectors for all templates are found, for all
other voxels interpolated vectors can be determined with the
help of Bookstein’s thin-plate spline algorithm [6, 7] and based
on the resulting vector field the planning cube can be
elastically deformed to match the verification image.

N predetermined anchor points p i (the template positions in

the planning data set) and N corresponding target points v i
(verification data set) are given. Due to the interpolation, each

voxel r of the resultant cube will receive a new gray value




F (r )  ( Fx (r ), Fy (r ), Fz (r )) .
Where each component (k = x, y, z) is calculated as

 
Fk (r )   ai ,k u r  pi  .
N
Since our first results have identified a series of starting points
for further enhancements a more detailed evaluation on various
patient data sets is deferred until this work is done.
i 1
u is a specific base function R3R1 describing the nature of
the thin-plate splines and ai,k are the coefficients, which are
calculated by solving the three equation systems Rk (with
k = x,y,z):

 
Rk ( pi )   a j ,k u  pi  p j   vi ,k .
N
j 1

Where p i are the i=1...N anchor points and vi,k the three

components of the N target points. The base function u( t ) is
defined according to [6]:


u( t ) = || t ||
which describes the minimization of the bending energy of
thin-plate splines in the 3D case.
The thin-plate spline interpolation is global in nature and one
can not avoid the global effect of each template. Among others,
this means that the transformation of each voxel is affected of
each anchor point. This is easy to understand, since the number

of coefficients ai is equal to the number of templates.
To avoid that image structures, which haven’t changed their
position in both data sets, are moved due to the influence of
remote templates, it is necessary to add templates in regions
with no observable motion. This way a fixation of these
structures at their initial positions can be assured.
Results
The result of the template matcher depends largely on the
number of templates. The required number of templates
depends on the size of the data set and on the occurred
deformations. With our test data sets (~ 50 slices, each with
256×256 pixels) we have observed acceptable correction of the
occurred motions when using about 1000 or more templates. A
sample of achievable results is displayed in figure 1. The
necessary processing time varies from ranges from 8 to 10
minutes. The necessary processing time is at the upper
tolerance level. If we reduce the number of templates, larger
deviations between the transformed planning data set and the
verification image are observed. Therefore current work is
focused on less time consuming interpolation methods which
should allow us to use more templates.
We expect a more noticeable gain in performance from
synergy effects by using motion estimations when searching
corresponding template positions. The current implementation
does not consider any a priori information about occurred
movements. By using results of optical flow calculations on resampled images with coarse resolution a first hint about
possible motion direction can be introduced which should
improve the starting point for the optimization step. The
hierarchical optical flow algorithm as stand-alone method
doesn’t always seem be able to generate a reliable result. Here
the quality depends heavily on the frequency of which it is
necessary to fall back to the motion calculations in low
resolution images.
1000
900
800
700
600
500
400
300
200
100
0
Correspondence
Bookstein Coefficients
Transformation
100
33
1
15
Sum
50
320
121
1
40
161
620
211
30
83
294
1350
394
79
162
635
1720
497
86
193
863
Figure 2: Calculation time (in s) for selected steps of the template
matcher in dependency of the number of templates. The calculation
time of the Bookstein coefficients ai,k increase superproportional with
the number of templates
Discussion
In this contribution we present an approach to calculate
displacement vector fields between planning and verification
CTs based on optical flow calculations and based on a template
matching strategy. After successful calculation these vector
fields can be applied to deform already segmented anatomical
structures. This way it is not necessary to perform conventional
segmentation on verification image series in fractionated
radiotherapy. The run time of the approach is currently at the
upper limit of the tolerated range. Therefore current work is
focused on methods resulting in an increase of performance to
get additional possibilities for enhancements in quality, too.
References
[1] Horn B K P; Schunk B G, 1981 Determining optical flow
Artificial Intelligent 17 185-204
[2] Rösch, P 2001 Template Selection and Rejection for Robust
Non-Rigid 3D Registration in the Presence of large
deformations Proceedings of the SPIE 545–556
[3] Maes et al 1997 Multimodality Image Registration by
Maximization of Mutual Information. IEEE Transactions on
Medical Imaging, Vol. 16, No. 2: 187 - 198
[4] Likar B, Pernus F, 1999 A Hierarchical Approach to Elastic
Registration Based on Mutual Information.
Image and Vision Computing, pages 33-44.
[5] Press W H, Teukolsky S A, 1995 Numerical Recipies in C
Cambridge University Press 545–556
[6] Bookstein F, 1989 Principal Warps: Thin-Plate Splines and
the Decomposition of Deformations IEEE Transaction on
Pattern Analysis and Machine Intelligence 11 567–585
[7] Arad N, Resifeld D, 1995 Image Warping Using few
Anchor Points and Radial Basis Functions Computer
Graphics Forum 14 1:35–46