Available online at www.sciencedirect.com
R
Microvascular Research 66 (2003) 113–125
www.elsevier.com/locate/ymvre
Automated tracing and change analysis of angiogenic vasculature
from in vivo multiphoton confocal image time series
Muhammad-Amri Abdul-Karim,a Khalid Al-Kofahi,a Edward B. Brown,b
Rakesh K. Jain,b and Badrinath Roysama,*
a
b
Department of Electrical, Computer and Systems Engineering, Rensselaer Polytechnic Institute, Troy, NY 12180, USA
Edwin L. Steele Laboratory, Department of Radiation Oncology, Massachusetts General Hospital and Harvard Medical School,
Boston, MA 02114, USA
Received 15 January 2003
Abstract
Automated methods are described for tracing and analysis of changes in angiogenic vasculature imaged by a multiphoton laser-scanning
confocal microscope. Utilizing chronic animal window models, time series of in vivo 3-D images were acquired on approximately the same
target volume of the same specimen while undergoing angiogenic change (typically every 24 h for 7 days). Objective, precise, 3-D, rapid,
and fully automated vessel morphometry was performed using an adaptive tracing algorithm that is based on a generalized irregular cylinder
model of the vasculature. This algorithm was found to be not only adaptive enough for tracing angiogenic vasculature, but also very efficient
in its use of computer memory, and fast, taking less than 1 min to trace a 768 ⫻ 512 ⫻ 32, 8-bit/pixel 3-D image stack on a Dell Pentium
III 1-GHz computer. The automatically traced centerlines were manually validated on six image stacks and the average spatial error was
measured to be 2 pixels, with an average concordance of 81% between manual and automated traces on a voxel basis. The tracing output
includes geometrical statistics of traced vasculature and serves as the basis of statistical change analysis. The computer methods described
here are designed to be scalable to much larger hypothesis testing studies involving quantitative measurements of tumor angiogenesis, gene
expression relative to known vascular structures, and impact of drug delivery.
© 2003 Elsevier Science (USA). All rights reserved.
Keywords: In vivo change analysis; Angiogenesis; Tumor vasculature; Vasculature tracing; Vasculature segmentation; Automated morphometry; Multiphoton microscopy
Introduction
An automated method is described for statistical change
analysis of in vivo tumor vasculature. It consists of three
phases: intravital microscopy, automated vasculature morphometry, and statistical change analysis. Changes occurring in tumor vasculature, and in preexisting vasculature bed
next to a tumor, provide insight into tumor pathophysiology,
which includes gene expression, angiogenesis, vascular
transport, and drug delivery (Jain et al., 2002; Folkman,
2001; Carmeliet and Jain, 2000; Auerbach et al., 1991).
* Corresponding author. ECSE Department, Rensselaer Polytechnic
Institute, Troy, NY 12180-3590, USA. Fax: ⫹1-518-276-8715.
E-mail address: roysam@ecse.rpi.edu (B. Roysam).
First, in vivo image acquisition by intravital microscopy is
performed using the multiphoton laser-scanning microscope
(MPLSM), aided by a variety of chronic animal window preparations described by Brown et al. (2001). The live specimen,
while undergoing angiogenesis, is imaged on approximately
the same volume, over a period of time (typically every 24 h
for 7 days), producing a time series of 3-D image stacks (see
Figs. 1– 4). These images reveal how a preexisting vascular
bed is altered as a tumor grows into the imaged region.
Precise vasculature segmentation is then performed on
the image stacks using a fully automated 3-D vasculaturetracing algorithm. This model-based algorithm extends our
prior work on tracing dye-injected neuron images (Al-Kofahi et al., 2002) and retinal angiograms (Shen et al., 2001;
Can et al., 1999), using a set of directional edge detectors to
0026-2862/03/$ – see front matter © 2003 Elsevier Science (USA). All rights reserved.
doi:10.1016/S0026-2862(03)00039-6
114
M.-A. Abdul-Karim et al. / Microvascular Research 66 (2003) 113–125
Using the statistics generated by the tracing algorithm on
time series images, change analysis is performed by comparing
morphometric statistics of the vasculature within a region of
interest common to all images in the sequence. Tabulation of
the statistics together with visual displays in the form of progression graphs is performed to reveal and describe vasculature
changes (Table 1 and Fig. 6). These change measurements
may then be used for hypothesis testing, pattern classification,
or other decision-making processes.
Materials and methods
Specimen preparation and imaging
3-D image stacks were acquired in vivo using an
MPLSM on Severe Combined Immunodeficiency Disease
Fig. 1. (A) Sample 768 ⫻ 512 ⫻ 32, 8-bit/pixel 3-D in vivo image of a
fluorescently labeled vasculature of skin altered by growth of a nearby
tumor, presented by its projections (x–y, y–z, and x–z). Note the amount of
background noise and intensity nonuniformities within the vasculature. The
size scale varies from 5 to 20 pixels wide. Assumption of Gaussian
intensity profile within vasculature is not valid in this image. Volumes
highlighted are common overlapping regions across all images in this set of
time series images. (B) Projections of the traces on each image projection.
For illustrative purposes, each vasculature segment is labeled. The tracing
algorithm adds several extra slices before the top slice and after the bottom
slice for computational consistency issues. Note that the program ignores
most of the background noise.
trace and segment vasculature that satisfy a generalized
cylinder model (Fig. 5). For this study, the algorithms were
modified to handle higher tortuosity, high size-scale variability, nonuniform brightness, and irregular structure of
tumor vasculature, which are not typical features in neuron
images. Being model-based, the algorithm introduced in this
paper overcomes limitations of intensity-based methods by
having the built-in notion of a physical object model, rather
than being based solely on intensity. Additionally, the algorithm overcomes the limitations of line-filtering methods
by avoiding the Gaussian cross-sectional profile assumption, and by performing calculations only on the image
foreground. Finally, the algorithm does not suffer from the
subjectivity and tedium associated with manual tracing.
Fig. 2. (A) The image of approximately the same volume as in Fig. 1 on
Day 2. The volume that represents the common overlapping region among
all images in this time series is highlighted. The background noise and
intensity nonuniformities within the vasculature are more visible in this
image compared to the image of the first day (Fig. 1). (B) Projections of the
traced image. As expected, the traced centerlines are minimally affected by
the imaging artifacts.
M.-A. Abdul-Karim et al. / Microvascular Research 66 (2003) 113–125
115
Table 1
Experimental change analysis results for time series tumor vasculature imagesa,b,c
Day
1
2
3
4
Total vessel segments
Total length of vessels (␮m)
Average horizontal width (␮m)
Average vertical width (␮m)
Data
Change index
Data
Change index
Data
Change index
Data
Change index
140
160
170
95
1.143
1.063
0.559
6572.1
7001.0
7334.8
4541.1
1.065
1.048
0.619
4.81
5.13
5.75
5.81
1.068
1.120
1.011
2.89
2.82
2.95
3.02
0.976
1.046
1.022
—
—
—
—
a
A set of four time series images taken every 24 h, shown in Figs. 1– 4.
Measurements are restricted to the common overlapping volume among all images.
c
Change Index is the ratio of the current data over the data on the previous day.
b
(SCID) mice prepared using various chronic window preparations. The reader is referred to a recent paper by Brown
et al. (2001) for a much more detailed description of the
specimen preparation methods. In Figs. 1– 4, a murine
mammary adenocarcinoma (MCaIV) was implanted in the
center of the dorsal skinfold window. Under anesthesia,
blood vessels were highlighted using fluorescently labeled
dextrans injected intravenously and a volume of preexisting
vasculature was imaged every 24 h for 7 days as the tumor
grew over the site. In Fig. 7, vasculature in the cranial
window was imaged after highlighting using fluorescent
dextrans injected intravenously. The resulting 3-D images
are typically 22–141 optical slices of size 768 ⫻ 512 each,
with typical intraslice resolution of 0.72 ␮m and interslice
resolution of 5 ␮m per voxel. Each 4-D dataset has typically
seven 3-D images, taken 24 h apart over a week.
The cranial window images (Fig. 7) required smoothing
before tracing could be attempted. To process such images,
a morphological preprocessor toolkit was integrated into the
implementation of the tracing algorithm. It consists of standard 3-D grayscale mathematical operations such as erosion, dilation, closing, and opening, as well as rank filters
such as the median filter. The built-in structuring elements
for these operations are ellipsoids and cubes with configurable size and axial scales. For example, a z scale of 1 is a
special case corresponding to 2-D structuring elements on
the x–y plane. These operations can be performed prior to
tracing if necessary to remove unwanted image artifacts.
Unless stated otherwise, all results presented in this paper
were produced without any preprocessing.
Automated vessel tracing methods
Segmentation is an essential first step for vessel morphometry, which in turn is used for statistical quantitation of
vascular changes (Jain et al., 1997; Barbareschi et al., 1995;
Leunig et al., 1992). Methods for vascular segmentation can
be grouped into three main categories: (i) intensity-based
segmentation; (ii) line-filtering methods; and (iii) manual
and semiautomatic methods. Methods in the first category
utilize intensity-based thresholds and are generally susceptible to background noise (e.g., Brey et al., 2002; Holmes et
al., 2002; Brown et al., 2001; Wild et al., 2000; Parsons-
Wingerter, et al., 1998; Seifert et al., 1997; Tjalma et al.,
1998; Toi et al., 1996; Fox et al., 1995; Jakobsson et al.,
1994; Avinash et al., 1993; Kowalski et al., 1992; Rohr et
al., 1992). Some methods utilize frequency and size thresholds (Schoell et al., 1997; Iwahana et al., 1996; Nissanov et
al., 1995). Some improvements were reported by preprocessing the images with morphological filters (e.g., KumarSingh et al., 1997; Merchant et al., 1994). Methods in the
second category use a line filter as the vasculature model
(Streekstra and Pelt, 2002; Frangi et al., 1999; Sato et al.,
1998). These methods assume a Gaussian cross-sectional
vessel profile, and perform expensive calculations on each
image voxel; therefore they scale poorly with image size.
Methods in the third category involve manual tracing,
counting, or visual inspection of the vasculature (Dellas et
al., 1997; Dellian et al., 1996; Kirchner et al., 1996; Li et al.,
1994; DeFouw et al., 1989; Endrich et al., 1979). These
methods, despite being the gold standard, are unavoidably
subjective and overly labor-intensive (Al-Kofahi et al.,
2003).
The vessel-tracing algorithm used in this work is an
extension of prior work in the context of tracing 3-D confocal images of dye-injected neurons, which was shown to
yield a 97% accuracy compared to multi-user gold standards
(Al-Kofahi et al., 2003; Al- Kofahi et al., 2002). The tumor
vasculature images of interest exhibit several structural
complexities that necessitated several improvements to the
previously developed algorithms. Specifically, they exhibit
a much higher tortuosity, size-scale variability, and irregularity of geometrical structure. Furthermore, images of tumor vasculature often exhibit nonuniform intensity levels
within vessels, mostly due to the dynamics of blood flow
relative to the spatiotemporal imaging window. There is
also significantly higher variability from one vessel to the
next.
Change analysis errors often originate from segmentation or tracing errors, which are caused mainly by the type
and density of imaging artifacts present in an image. Since
the image acquisition phase is performed in vivo and over
several days, several types of imaging artifacts are inevitable, caused partly by flow of red blood cells, dye leakage,
variability of dye injection, and vasculature structural
changes due to respiration, among others. The tracing algo-
116
M.-A. Abdul-Karim et al. / Microvascular Research 66 (2003) 113–125
defined by a base point b, which is the center location of the
first LPD correlation kernel in the template, e.g., the point
bR. The template orientation in 3-D is defined by the set ␪
⫽ {␪V, ␪H}, where ␪H is a rotation about the line HH⬘ (i.e.,
horizontal) and ␪V is a rotation about the line VV⬘ (i.e.,
vertical) (see Fig. 5). Lines VV⬘ and HH⬘ are orthogonal to
each other. In this work, each element in ␪ is discretized to
32 discrete values, giving an angular precision of 11.25°,
therefore yielding a total of 1024 unique 3-D directions. Our
software implementation allows the user to select other
discretization values to best fit the images of interest. The
template length, k, is defined by the number of LPD correlation kernels stacked together to form the template. Fig. 5
illustrates templates of length 8 in 3-D and Fig. 8 illustrates
templates of various lengths in 2-D. The template that is
most closely oriented along the generalized cylinder and
centered on the cylinder boundary produces the maximum
template response, i.e., correlation coefficient (Al-Kofahi et
al., 2002).
Fig. 3. (A) Day 3 image of approximately the same volume as in Fig. 2.
The common overlapping region is highlighted. At this time, more vasculature segments are visible. The amount of background noise is much more
apparent than images of previous days. (B) Projections of the traced image.
As expected, the tracing algorithm is robust to structural irregularities and
intensity nonuniformities present in the image.
rithms presented here are designed to achieve a high automation level while being robust to these imaging artifacts.
A 3-D mathematical model for describing vessels
Al-Kofahi et al. (2002) used the generalized three-dimensional cylinder as a geometric model for dye-injected
neurons. This geometrical model is illustrated in Fig. 5, and
is sampled at four edges, denoted T ⫽ {tT, tB, tL, tR}, with
the subscripts denoting their locations either at the top,
bottom, left, or right edge of the vessel cross-section. The
four edges are detected using a set of directional templates
adapted from the work of Sun et al. (1995) and Can et al.
(1999). Each template is composed of a stacked set of
directional 1-D low-pass differentiator (LPD) (Sun et al.,
1995) correlation kernels of the form [⫺1, ⫺2, 0, 2, 1]T.
They are illustrated in Fig. 5.
The location of a template in the 3-D image space is
Fig. 4. (A) Day 4 image of approximately the same volume as in Fig. 3.
The common overlapping region is highlighted. The number of visible
vasculature is much less compared to images of previous days. (B) Projections of the traced image. Geometrical statistics generated from these
traces are shown in Fig. 6.
M.-A. Abdul-Karim et al. / Microvascular Research 66 (2003) 113–125
117
Fig. 5. Isometric view of the vasculature model over a short distance with structural irregularity. Each template shown here is of length 8. The two angular
directions, ␪ ⫽ {␪H, ␪V} are rotations around the line HH⬘ (parallel to the z axis) and VV⬘ (orthogonal to HH⬘, i.e., on the x–y plane). Other than the strength
of the edge at the four boundaries, this model does not have any inherent assumption on the cross-sectional shape or the intensity profile of the vasculature.
This figure also illustrates the robustness of using median statistics for the template responses to handle vascular structural irregularities.
As noted in the Introduction, the above model was originally developed for neuron tracing. It must be modified to
handle the higher tortuosity, high size-scale variability, nonuniform brightness, and irregular structure of tumor vasculature. As illustrated in Fig. 5, tumor vessels can have a
rough surface and a nonuniform cross-section, making the
generalized cylinder model inexact. In other words, the need
exists for a systematic method for handling limited deviations from the pure generalized cylinder model. In keeping
with the need to keep the tracing algorithms scalable to
larger data sets, it is also desirable to seek computationally
inexpensive methods. A simple method for meeting these
requirements is described below.
In the prior body of work (Al-Kofahi et al., 2002; Can et
al., 1999; Sun et al., 1995), the response of each template
was length-normalized by averaging correlation kernel responses along the template length. It is well-known in the
statistical literature that the average response is greatly
affected by intensity nonuniformities within the structures
(Huber, 1981). A more robust alternative to the average
response is the median response, which requires computing
the median value of correlation kernel responses along the
template length. By definition, the median value is robust to
as many as 50% outliers (Huber, 1981). The median response R for a template of empirically chosen lengths defined in the set K, at a 3-D location b ⫽ [x y z]T, along the
118
M.-A. Abdul-Karim et al. / Microvascular Research 66 (2003) 113–125
M.-A. Abdul-Karim et al. / Microvascular Research 66 (2003) 113–125
119
Fig. 6. The statistics generated by traces of the images in Figs. 1– 4, also shown in Table 1. (A) Statistics for total vasculature segments present in an image.
(B) Statistics for total length of vasculature. (C) Statistics for average horizontal widths. (D) Statistics for average vertical widths (See Fig. 5 for width
definitions). Notice that the changes highlighted by the statistics qualitatively agree with the image contents. For example, the average horizontal
width increased in the first 3 days, but stayed relatively the same on Day 4 although 38% of the vessels “disappeared.” Currently, the accuracy of vertical
width measurements is greatly affected by axial blurring in the images.
orientation defined by the unit vector u, is expressed mathematically as:
R共b,u,K兲 ⫽ arg max {median (r(b ⫹ ju,u ⬜ ))}, (1)
k僆K
j⫽1. . .k
where r(b,u⬜) is the response of a single 1-D LPD correlation kernel at b along the direction u⬜ that is perpendicular to u. Recall that a template of length k is comprised of
k 1-D LPDs stacked together, hence, r is essentially a
template of length 1. Notice that this median version of R is
also length-normalized since it can only have values returned by r, independent of all k僆K. The maximum median
value over all lengths k 僆 K is chosen to be the median
response of the template at location b along the direction u.
Another aspect of model generalization and refinement
originated from observing that tumor vessels resemble a
deformed cylinder over a short length in most cases. In other
words, over a short distance, linear approximations of vasculature boundaries are not parallel to each other. This
violates the assumption of parallel edges in our previous
work (Al-Kofahi et al., 2002), which stated that each element in the set of boundary points B must have the same
orientation ␪. Here, each template is allowed to shift, ex-
Fig. 7. (A) An optical section from a 768 ⫻ 512 ⫻ 141 image stack. Observe the intensity nonuniformity within the vasculature. This was due to moving
red blood cells (RBC) during horizontal scan of the specimen. (B) The same optical section, filtered 20 times using a median filter with a spherical structuring
element (x–y diameter ⫽ 7 voxels, z diameter ⫽ 1 voxel). The horizontal scan artifacts due to moving RBCs are minimized, while vascular boundaries are
preserved. (C) Optical sections 1 through 17 of the image stack in the x–y projection. (D) Segmented vasculature of the optical sections in (C), obtained by
tracing, also in the x–y projection, which highlights the accuracy of boundary detection by the tracing algorithm. (E) The entire image stack, 705 ␮m deep,
in red– blue anaglyph. (F) Segmented vasculature of the entire image stack, in red– blue anaglyph. This shows that the tracing algorithm is fully 3-D, and
applicable to deep image stacks in addition to relatively shallow image stacks shown in Figs. 1– 4.
Fig. 10. Illustrates both manual and automated centerline traces superimposed on a maximum intensity projection image for qualitative performance
evaluation (quantitative manual-automated concordance measure is 85%), highlighting several areas of interest. (A) False-negative; the vessel has poorly
defined edges and poor contrast relative to the local background. (B) Here is where the automated centerline traces are smoother than the manual traces,
showing its “steady-hand” effect. (C) Both manual and automated traces agree here in this image, but in subsequent images in the time series dataset, this
section becomes more convincing as dye-leakage, hence creating false-positive errors for the automated tracing algorithm in those subsequent images. (D)
False-positive; here the strip of dye leakage closely resembles a vessel. (E) False-positive; this relatively dim vessel is occluded by brighter optical slices,
hence missed by manual tracing, but not by the automated tracing algorithm. Note the robustness of the automated tracing algorithm to irrelevant blob-like
structures, and nonuniform dye distribution within and among vessels.
120
M.-A. Abdul-Karim et al. / Microvascular Research 66 (2003) 113–125
Fig. 8. Example of two pairs of templates that gives the highest response at each iteration i in 2-D. One pair of templates in 2-D consists of left and right
templates. Note that at a single iteration i, each template is allowed to elongate, shift, and rotate independently subject to constraints of the generalized
cylinder model.
pand, and rotate independent of other templates (shown in
2-D in Fig. 8). With this in mind, the strict generalized
cylinder model is relaxed to account for cross-sectional
expansion and shrinkage as one traces along a vessel.
3-D vessel tracing algorithm
The tracing algorithm traces the vasculature in an exploratory manner starting from automatically detected starting points (seed points). To find the seed points, a 2-D
maximum intensity projection of the 3-D volumetric image
is created and a search for local maxima is performed along
the lines of a coarse grid on this 2-D image. Each local
maxima discovered becomes a seed candidate. For each
seed candidate, the z value is found by performing an axial
search corresponding to the lateral coordinates of each candidate. Notice that the search for seed point candidate has
only been 1-D thus far. Next, each seed point candidate is
validated in 3-D using the generalized cylinder model and
unfit candidates are rejected. The validation process begins
with finding the four boundaries as illustrated in Fig. 5,
using the templates and template response function as in Eq.
(1), exhaustively at all directions and at all widths. Seed
point candidates without the four almost-parallel boundaries, i.e., with relatively low maximal template response,
are therefore rejected. At this stage, each validated seed
point is associated with vertical and horizontal width infor-
mation as well as a direction unit vector corresponding to
the morphometrics of the generalized cylinder model where
it fits best. Tracing begins at the seed points in an iterative
manner until one of the stopping criteria is met, i.e., where
the generalized cylinder model is violated. A seed point is
used as a starting point twice, along the opposing directions
of the cylinder axis.
A single iteration of tracing is defined as moving from a
centerline point pi to the next centerline point pi⫹1 (p0 is a
seed point) with the distance between these two points
defined as the step size si, the angle between them as ␪i, and
the unit vector along the direction ␪i as ui. At pi, the
locations of the corresponding boundary points are denoted
by the set Bi ⫽ {bTi, biB, bLi, biR} and are ordinarily computed
as the points at which the template responses are maximum.
To better handle tortuous structures of interest, this direct
method was modified as follows. First, at each point in the
template shifting process, the responses of templates along
adjacent directions are also computed by rotating the templates. Second, template lengths are adjusted to fit local
image features. This is because longer templates are more
robust to noise than shorter ones since they perform more
averaging along the vessel edge. On the other hand, shorter
templates are more accurate around curved vasculature segments. This means that at each shifting and rotating step, the
algorithm computes the responses of templates of different
lengths (Al-Kofahi et al., 2002). A length-normalized tem-
M.-A. Abdul-Karim et al. / Microvascular Research 66 (2003) 113–125
121
Fig. 9. This figure illustrates the intuitions expressed mathematically in Eq. (4) that describes the maximum expansion (or shrinkage) tolerance angle ␾ of
the vasculature when tracing from pi to pi⫹1 as a function of shift constraint ␣, rotation constraint ␤, step size si, and template length k.
plate response defined earlier in Eq. (1) is used to set the
tracing direction. Mathematically, this can be written as
follows:
共b i,u i,k i兲
⫽
arg max
兵R共b,u,K兲其,
(2)
M
{(b,u,k) 兩 b⫽pi⫹mu⬜,m⫽1, . . . , ,u僆U,k僆K}
2
where U is the set of unit vectors along directions in the
neighborhood of ui and K is the set of all template lengths.
The vector u is a unit vector along a particular ␪, while u⬜
is the unit vector perpendicular to u. The parameter M is the
user-defined diameter of the widest expected vessel. Values
(bi, ui, ki) are the results of this exhaustive search at iteration
i, each representing the location, orientation, and length,
respectively, of the template that returns the maximum
response R. This search is performed four times corresponding to the four templates that make up the generalized
cylinder model. The centerline point pi, cylinder direction
ui, and cylinder length ki are calculated as a function of the
sets {bTi, biB, bLi, biR}, {uTi, uiB, uLi, uiR}, and {kTi, kiB, kLi, kiR},
respectively. Continuing with centerline tracing, we proceed
from pi to pi⫹1 using the equation
p i⫹1 ⫽ 共p i共B i兲 ⫹ s iu i兲 ⫹ c i⫹1 共B i⫹1兲,
(3)
where c i⫹1(B i⫹1) is the correction (refinement) vector as a
function of the set of boundary points B i⫹1 (Al-Kofahi et
al., 2002). In other words, the location of the centerline
point at i ⫹ 1 is not exactly known until the algorithm
determines the corresponding boundary points at i ⫹ 1. The
step size s i acts as the scaling factor for the unit vector u i .
It is adaptive and calculated from the length of the shortest
template among the four corresponding templates with maximum responses.
To allow small deviations from the generalized cylinder
model, the template expansion is allowed to be fully independent, while template shift and rotation are subject to a set
of constraints. For a template of length k, its shift is limited
to an empirically set range ⫾ ␣ and its rotation is similarly
limited to the range ⫾ ␤. The following equation defines the
maximum expansion (or shrinkage) tolerance angle ␾ when
tracing from pi to pi⫹1 as a function of shift constraint ␣,
rotation constraint ␤, step size si, and template length k (see
Fig. 9).
冉 冉 冊 冉 冊冊
␾ ⫽ max tan⫺1
␣
␤
, sin⫺1
si
k
.
(4)
To prevent multiple traces of the same vessel, traced
vessels are marked using ellipsoids that are individually
sized according to the estimated horizontal and vertical
widths at each traced point. Seed points located within
traced vessels are deemed invalid and hence ignored.
After tracing all structures in an image, individual traces
are merged to form branching points. For a detailed
description of these issues, the reader is referred to AlKofahi et al. (2002).
Change analysis results
Our primary intent was to quantitate temporal vessel
changes in a set of time series images. There are two broad
methods for change analysis. One method is to compute
122
M.-A. Abdul-Karim et al. / Microvascular Research 66 (2003) 113–125
morphometric data from images at each temporal sampling
point, and perform statistical comparison of these data. A
more ambitious approach is to register the images over time
and extract detailed changes on a vessel segment by vessel
segment basis. In this work, the less ambitious approach
was adopted as a starting point. Vessel lengths, widths, and
count can be readily generated from the traces generated by
the automatic tracing algorithm described in the previous
section. Fig. 6 and Table 1 show the statistics corresponding
to the four time series images shown in Figs. 1– 4. Naturally, only vasculature segments located in the volume common to all four images contributed to these statistics. An
overall Change Index is calculated as the simple ratio of the
current measurement over the previous measurement.
For this study, 30 vasculature images, of varying planar
dimensions and volume depth, were traced. Typical examples are presented here. SCID mice with window preparations were implanted with MCaIV at the center of the
window and fluorescent labels were injected intravenously
to highlight the vasculature. Imaging was performed on
anesthetized mice using the MPLSM which consisted of a
Spectra-Physics MilleniaX pumped Tsunami Ti:Sapphire
laser, Bio-Rad MRC600 scanhead, and the Zeiss Axioskop20 microscope (Brown et al., 2001). Several images
with a relatively higher degree of intensity nonuniformity
were preprocessed using a median filter prior to tracing.
These images reveal how a preexisting vascular bed is
altered as a tumor grows into the imaged region.
In addition to the trace projections, the program generates the actual traces drawn onto the 3-D input image and
the segmented vasculature in a 3-D binary image (generated
by drawing ellipsoids along the centerline, and sizing the
ellipsoid according to the local vertical and horizontal width
estimates). Results in Fig. 7 are examples of such binary
images. The program also generates the length and width
statistics in an output text file.
Of the 30 images used in this study, 28 images belong to
four sets of time series images (768 ⫻ 512 ⫻ 32 stacks, 8
bits/voxel, 7 days, 1 stack/day). The images were traced and
statistics from each temporal set were gathered. A region of
interest was defined for each image by the intersection of
the image and all other images in the set. Statistics collected
within these regions include total vasculature length, average vasculature segment length, average horizontal width,
and average vertical width. Statistics and traces generated
outside these regions were ignored. The generated statistics
were entered into a spreadsheet and plotted to highlight the
changes. Fig. 6 and Table 1 show the change analysis results
for the first 4 images corresponding to the first 4 days from
the second time series set that are shown in Figs. 1– 4.
Actual change measurements such as percentage of reduction in total number of vasculature segments or percentage
of increase in total length of vasculature can be obtained
directly from the spreadsheet.
Method for validating trace results
Quantitative validation of the tracing algorithm requires
the availability of a ground truth, which must be established
manually. Manual tracing of 3-D structures is very difficult
and time-consuming. It also suffers from a greater degree of
tracer variability. To establish the ground truth, it is vital
that one accounts for this intertracer variability by having
multiple manual traces of the same image. This constitutes
an unreasonable burden and we argue that, for the purposes
of this study, it is sufficient to validate the results based on
their 2-D maximum intensity projections. In other words,
the 3-D automated trace results are projected on a 2-D
plane, and validated against manual traces of 2-D maximum-intensity projections of the 3-D image.
The tracing results were validated using two performance metrics. The two metrics cater to different user
concerns regarding the accuracy of the automated traces. To
further avoid the factor of subjectivity in validating the trace
results, the comparisons between the manual and automated
traces were performed automatically. This validation study
is based on six image stacks consisting of the four image
stacks shown in Figs. 1– 4 that belong to a time series set,
and two other single-shot vasculature image stacks.
The first performance metric is the average distance
between the manually traced and automatically traced vasculature centerlines. To evaluate this metric, we calculated
the Euclidean distance between every traced pixel in the
manual traces and every traced pixel in the automated traces
that were within a certain Euclidean distance, which was
chosen to be 10. This metric is suitable for users concerned
with the pixel-wise accuracy of the traces. The average
distance over the six image stacks ranged from 1.81 to 2.38
pixels, with an overall average of 2.11 pixels.
The second performance metric is the concordance, or
agreement between the manual and automated traces. This
metric was evaluated by calculating the number of automatically traced pixels that were within a certain distance from
corresponding manually traced pixels. The end result tells
how much editing may need to be performed manually by
the user after the image has been traced automatically.
Editing may be as easy as deleting false-positive segments,
putting seed points to trace false-negative segments, or
manually retracing the false-negative segments. Note that
the factor of subjectivity of manual tracing is reduced down
to the correspondence between the manual and automated
traces. This metric is suitable for users that are concerned
about coverage of the automated traces. The concordance
measures for the six image stacks ranged from 72 to 89%,
with an overall average of 81%.
Other than the traces of the vasculature centerlines, the
width statistics need to be validated as well. Given the
nature of 3-D vasculature images, it is very time-consuming
to manually gather the width statistics and, since the results
will be subjective, it further reduces the value of the effort.
Instead, we used several synthetic images containing tubes
M.-A. Abdul-Karim et al. / Microvascular Research 66 (2003) 113–125
123
Table 2
Width statistics results for phantom/synthetic imagesa,b,c
Image Description
Sine tube on x–y plane
Sine tube on x–z plane
Sine tube on x–y plane
Sine tube on x–z plane
Straight tube with parallel boundaries
Straight tube with sinusoidal boundaries
Horizontal width
Vertical width
Actual
Detected
17.00
19.00
19.00
11.00
17.00
8.64
18.18
19.05
21.77
10.54
16.00
8.62
Error (%)
7.0
0.2
14.6
4.2
5.9
0.2
Average ⫽ 5.4
Actual
Detected
5.00
19.00
19.00
11.00
17.00
8.64
6.28
23.73
18.73
12.48
18.95
8.62
Error (%)
25.7
24.9
1.4
13.5
11.5
0.2
Average ⫽ 12.9
a
Phantom/synthetic images are generated by rolling an ellipsoid of specified horizontal and vertical width along a specified path (either a straight line or
a sinusoid).
b
The actual width is the known width, and the detected width is the width measurement reported by the tracing algorithm.
c
Error is the percentage of absolute difference between the actual and detected measurements.
of known geometrical dimensions. The tubes were generated by drawing ellipsoids of known horizontal and vertical
widths along a known path, which can be straight or tortuous. We stimulated tortuous vasculature by using sinusoidal
paths, and vasculature wall irregularities were simulated by
using sinusoidal vasculature walls. The deviation of the
reported width statistics from the known true values averaged 5.4% for horizontal width and 12.9% for vertical width
(see Table 2 ). These measurements are greatly affected by
the degree of angular discretization in the tracing algorithm,
where finer angular discretization yields higher accuracy at
the expense of computational speed. Also, the location of
the true boundary was discretized to a voxel location, causing slight deviation from the “true” boundary location,
which theoretically lies between the foreground voxel and
the neighboring background voxel. Nevertheless, these deviations from the true measurements are reproducible and
consistent from one image to another, which is critical for
change analysis studies.
Conclusions and discussion
The change analysis study presented in this paper is
based on geometrical statistics generated by a fully automatic 3-D tracing algorithm. The tracing algorithm is fast,
accurate, and precise, making it applicable for large-scale
applications where speed and reproducibility are important.
It is robust to intensity nonuniformities, structural irregularities, and background noise. This work extends our previous work (Al-Kofahi et al., 2002) with more attention
given to handling imaging artifacts and structural irregularities which are more apparent in angiogenic vasculature
images than dye-injected neuron images (Al-Kofahi et al.,
2002) and retinal angiograms (Can et al., 1999). Execution
time is up to five times higher than our previous implementation, mainly due to the use of median statistics.
Clearly, more intellectual and computational efforts are
required in the vasculature tracing (segmentation) phase
compared to the change analysis phase. This is because the
whole change analysis phase is greatly simplified by just
analyzing meaningful statistics and measurements that were
extracted from raw image data by the tracing algorithm. It
also implies that the accuracy of the change analysis relies
heavily on the accuracy of the tracing algorithm. Regardless, the use of an automated tracing algorithm in quantitative change analysis yields reproducible results, which minimizes the factor of subjectivity in generating the
measurements analyzed for changes. Consequently, collections of change analysis results can be fairly compared
between research groups conducting all kinds of different
vasculature-related assays.
The tracing algorithm does not require any special hardware. The results presented here were obtained using a Dell
Pentium III 1-GHz computer. For a typical 8-MB image as
shown in Fig. 1, it takes 53 s, and its speed varies depending
on the amount of structures present in the image since it
only processes the image foreground. At the time of this
writing, the authors are considering dissemination plans for
the PC-compatible implementation of the algorithm, but
they have not been finalized.
Nevertheless, the tracing algorithm is still not perfect and
suffers from some drawbacks, most noticeably false-negative errors (see Fig. 10). These errors are caused by (1) dim
vasculature that has poor contrast with image background,
(2) poorly defined edges which can be more appropriately
modeled as ramp edges instead of the step edges which are
built into our vasculature model, and (3) absence of a seed
point on that particular vasculature segment. On the other
hand, stretches of dye leakage that closely resemble discontinuity of dye in vasculature and spurious seeds contribute
to false-positive errors. Overall, most tracing errors are
caused by artifacts caused by the imaging process itself. For
example, in the datasets considered for this study, the fluorescent dye is injected over a period of several days,
leaving punctate deposits of extravasated dye in the subsequent images. Nevertheless, the erroneous results are reproducible, which means that they can be easily reproduced to
124
M.-A. Abdul-Karim et al. / Microvascular Research 66 (2003) 113–125
further refine the tracing algorithm to be more robust to
causes of these errors. Furthermore, unlike manual tracing,
the tracing algorithm was shown to have the “steady hand”
effect. In fact, Al-Kofahi et al. (2002) have shown that the
algorithm is more accurate than a manual tracer in as far as
locating the “true centerline” is concerned. The tracing
algorithm can be viewed as having a consensual standard
where expert human observers agree on a set of criteria to
classify image voxels as part of the vasculature or the
background. However, it is often not straightforward to
formalize these criteria into mathematical forms that can be
implemented as computer algorithms. Currently, only the
notions of local parallel edges and local contrast are being
incorporated in the tracing algorithm.
In summary, the statistical change analysis methods described in this paper are just an example of possible utilizations of the tracing output. Other utilizations of geometrical information of curvilinear structures extracted by the
tracing algorithm include geometrical change analysis, feature extraction, and image registration. Medical applications
of change analysis include testing the efficacy of anti-angiogenic therapies and image-based diagnosis. A potential
application in biology is to derive vessel growth parameters
which may be correlated with physiological and gene expression profiles in tumors.
Acknowledgments
Various portions of this research were supported by the
Center for Subsurface Sensing and Imaging Systems, under
the Engineering Research Centers Program of the National
Science Foundation (Award No. EEC-9986821), the NSF
Partnerships in Education and Research Program, MicroBrightField Inc. (Williston, VT), the Ministry of Entrepreneur Development of Malaysia (via MARA), and by grants
from the NCI (P01CA80124 and R24CA85140) and the
Rensselaer Polytechnic Institute. The authors thank colleagues George Nagy, Charles V. Stewart, Richard Radke,
Qiang Ji, Omar Al-Kofahi, Vijay Mahadevan, and Kenneth
Fritzsche, for discussions and valuable suggestions on the
broad topic of tracing algorithms.
References
Al-Kofahi, K.A., 2000. Algorithms for Rapid Automated Tracing of Neurons from 2-D and 3-D Confocal Images: Applications to Nanobiotechnology. Ph.D. thesis. Rensselaer Polytechnic Institute, Troy, NY.
Al-Kofahi, K.A., Can, A., Lasek, S., Szarowski, D.H., Dowell, N., Shain,
W., Turner, J.N., Roysam, B., 2003. Median based robust algorithms
for tracing neurons from noisy confocal microscope images. IEEE
Trans. Inform. Technol. Biomed., in press.
Al-Kofahi, K.A., Lasek, S., Szarowski, D.H., Pace, C.J., Nagy, G., Turner,
J.N., Roysam, B., 2002. Rapid automated three-dimensional tracing of
neurons from confocal image stacks. IEEE Trans. Inform. Technol.
Biomed. 6 (2), 171–187.
Auerbach, R., Auerbach, W., Polakowski, I., 1991. Assays for angiogenesis: a review. Pharmac. Ther. 51, 1–11.
Avinash, G.B., Quirk, W.S., Nuttall, A.L., 1993. Three-dimensional analysis of contrast-filled microvessel diameters. Microvasc. Res. 45 (2),
180 –192.
Barbareschi, M., Gasparini, G., Morelli, L., Forti, S., Dalla Palma, P.,
1995. Novel methods for the determination of the angiogenic activity of
human tumors. Breast Cancer Res. Treat. 36 (2), 181–192.
Brey, E.M., King, T.W., Johnston, C., McIntire, L.V., Reece, G.P., Patrick
Jr., C.W., 2002. A technique for quantitative three-dimensional analysis of microvascular structure. Microvasc. Res. 63 (3), 279 –294.
Brown, E.B., Campbell, R.B, Tsuzuki, Y., Xu, L., Carmeliet, P., Fukumura, D., Jain, R.K., 2001. In vivo measurements of gene expression,
angiogenesis and physiological function in tumors using multiphoton
laser scanning microscopy. Nature Med. 7 (3), 864 – 868.
Can, A., Shen, H., Turner, J., Tanenbaum, H., Roysam, B., 1999. Rapid
automated tracing and feature extraction from live high-resolution
retinal fundus images using direct exploratory algorithms. IEEE Trans.
Inform. Technol. Biomed. 3 (2), 125–138.
Carmeliet, P., Jain, R.K., 2000. Angiogenesis in cancer and other diseases.
Nature 407, 249 –257.
DeFouw, D.O., Rizzo, V.J., Steinfeld, R., Feinberg, R.N., 1989. Mapping
of the microcirculation in the chick chorioallantoic membrane during
normal angiogenesis. Microvasc. Res. 38 (2), 136 –147.
Dellas, A., Moch, H., Schultheiss, E., Feichter, G., Almendral, A.C.,
Gudat, F., Torhorst, J., 1997. Angiogenesis in cervical neoplasia: microvessel quantitation in precancerous lesions and invasive carcinomas
with clinicopathological correlations. Gynecol. Oncol. 67 (1), 27–33.
Dellian, M., Witwer, B.P., Salehi, H.A., Yuan, F., Jain, R.K., 1996. Quantitation and physiological characterization of angiogenic vessels in
mice: effect of basic fibroblast growth factor, vascular endothelial
growth factor/vascular permeability factor, and host microenvironment.
Am. J. Pathol. 149 (1), 59 –71.
Endrich, B., Reinhold, H.S., Gross, J.F., Intaglietta, M., 1979. Tissue
perfusion inhomogeneity during early tumor growth in rats. J. Natl.
Cancer. Inst. 62 (2), 387–395.
Folkman, J., 2001. Angiogenesis, in: Braunwald, E., Fauci, A.S., Kasper,
D.L., Hauser, S.L., Longo, D.L., Jameson, J.L. (Eds.), Harrison’s
Principles of Internal Medicine, fifteenth ed., McGraw-Hill, New York,
pp. 517–530.
Frangi, A.F., Niessen, W.J., Hoogeveen, R.M., Walsum, T., Viergever,
M.A., 1999. Model-based quantitation of 3-D magnetic resonance
angiographic images. IEEE Trans. Med. Imag. 18 (10), 946 –956.
Fox, S.B., Leek, R.D., Weekes, M.P., Whitehouse, R.M., Gatter, K.C.,
Harris, A.L., 1995. Quantitation and prognostic value of breast cancer
angiogenesis: comparison of microvessel density, Chalkley count, and
computer image analysis. J. Pathol. 177 (3), 275–283.
Holmes III, D.R., Moore, M.J., Mantilla, C.B., Sieck, G.C., Robb, R.A.,
2002. Rapid semi-automated segmentation and analysis of neuronal
morphology and function from confocal image data. Proc. IEEE Intl.
Symp. Biomed. Imag. 233–236.
Huber, P.J., 1981. Robust Statistics. Wiley, New York.
Iwahana, M., Nakayama, Y., Tanaka, N.G., Goryo, M., Okada, K., 1996.
Quantification of tumour-induced angiogenesis by image analysis. Int.
J. Exp. Pathol. 77 (3), 109 –114.
Jain, R.K., Munn, L.L., Fukumura, D., 2002. Dissecting tumour pathophysiology using intravital microscopy. Natl. Rev. Cancer 2 (4), 266 –
276.
Jain, R.K., Schlenger, K., Höckel, M., Yuan, F., 1997. Quantitative angiogenesis assays: progress and problems. Nature Med. 3 (11), 1203–
1208.
Jakobsson, A.E., Norrby, K., Ericson, L.E., 1994. A morphometric method
to evaluate angiogenesis kinetics in the rat mesentery. Int. J. Exp.
Pathol. 75 (3), 219 –224.
Kirchner, L.M., Schmidt, S.P., Gruber, B.S., 1996. Quantitation of angiogenesis in the chick chorioallantoic membrane model using fractal
analysis. Microvasc. Res. 51 (1), 2–14.
M.-A. Abdul-Karim et al. / Microvascular Research 66 (2003) 113–125
Kowalski, J., Kwan, H.H., Prionas, S.D., Allison, A.C., Fajardo, L.F.,
1992. Characterization and applications of the disc angiogenesis system. Exp. Mol. Pathol. 56 (1), 1–19.
Kumar-Singh, S., Vermeulen, P.B., Weyler, J., Segers, K., Weyn, B., Van
Daele, A., Dirix, L.Y., Van Oosterom, A.T., Van Marck, E., 1997.
Evaluation of tumour angiogenesis as a prognostic marker in malignant
mesothelioma. J. Pathol. 182 (2), 211–216.
Leunig, M., Yuan, F., Menger, M.D., Boucher, Y., Goetz, A.E., Messmer,
K., Jain, R.K., 1992. Angiogenesis, microvascular architecture, microhemodynamics, and interstitial fluid pressure during early growth of
human adenocarcinoma LS174T in SCID mice. Cancer Res. 52 (23),
6553– 6560.
Li, V.W., Folkerth, R.D., Watanabe, H., Yu, C., Rupnick, M., Barnes, P.,
Scott, R.M., Black, P.M., Sallan, S.E., Folkman, J., 1994. Microvessel
count and cerebrospinal fluid basic fibroblast growth factor in children
with brain tumours. Lancet 344 (8915), 82– 86.
Merchant, F.A., Aggarwal, S.J., Diller, K.R., Bovik, A.C., 1994. In-vivo
analysis of angiogenesis and revascularization of transplanted pancreatic islets using confocal microscopy. J. Microsc. 176 (3), 262–275.
Nissanov, J., Tuman, R.W., Gruver, L.M., Fortunato, J., 1995. Automatic
vessel segmentation and quantification of the rat aortic ring assay of
angiogenesis. Lab. Invest. 73 (2), 734 –739.
Parsons-Wingerter, P., Lwai, B., Yang, M.C., Elliot, K.E., Milaninia, A.,
Redlitz, A., Clark, J.I., Sage, E.H., 1997. A novel assay of angiogenesis
in the quail chorioallantoic membrane: stimulation by bFGF and inhibition by angiostatin according to fractal dimension and grip intersection. Microvasc. Res. 55 (3), 201–214.
Rohr, S., Toti, F., Brisson, C., Albert, A., Freund, M., Meyer, C., Cazenave, J.P., 1992. Quantitative image analysis of angiogenesis in rats
implanted with a fibrin gel chamber. Nouv. Rev. Fr. Hematol. 34,
287–294.
Sato, Y., Nakajima, S., Shiraga, N., Atsumi, H., Yoshida, S., Koller, T.,
Gerig, G., Kikinis, R., 1998. Three-dimensional multi-scale line filter
125
for segmentation and visualization of curvilinear structures in medical
images. Med. Image Anal. 2 (2), 143–168.
Schoell, W.M.J., Pieber, D., Reich, O., Lahousen, M., Jenicek, M., Guecer,
F., Winter, R., 1997. Tumor angiogenesis as a prognostic factor in
ovarian carcinoma: quantification of endothelial immunoreactivity by
image analysis. Cancer 80 (12), 2257–2262.
Seifert, W.F., Verhofstad, A.A.J., Wobbes, T., Lange, W., Rijken,
P.F.J.W., van der Kogel, A.J., Hendriks, T., 1997. Quantitation of
angiogenesis in healing anastomoses of the rat colon. Exp. Mol. Pathol.
64 (1), 31– 40.
Shen, H., Roysam, B., Stewart, C.V., Turner, J.N., Tanenbaum, H.L., 2001.
Optimal scheduling of tracing computations for real-time vascular
landmark extraction from retinal fundus images. IEEE Trans. Inform.
Technol. Biomed. 5 (1), 77–91.
Streekstra, G.J., Pelt, J.V., 2002. Analysis of tubular structures in threedimensional confocal images. Network Comput. Neural Syst. 13, 381–
395.
Sun, Y., Lucariello, R., Chiaramida, S., 1995. Directional low-pass filtering
for improved accuracy and reproducibility of stenosis quantification in
coronary arteriograms. IEEE Trans. Med. Imag. 14 (2), 242–248.
Tjalma, W., Van Marck, E., Weyler, J., Dirix, L., Van Daele, A., Goovaerts, G., Albertyn, G., van Dam, P., 1998. Quantification and prognostic relevance of angiogenic parameters in invasive cervical cancer.
Br. J. Cancer 78 (2), 170 –174.
Toi, M., Kondo, S., Suzuki, H., Yamamoto, Y., Inada, K., Imazawa, T.,
Taniguchi, T., Tominaga, T., 1996. Quantitative analysis of vascular
endothelial growth factor in primary breast cancer. Cancer 77 (6),
1101–1106.
Wild, R., Ramakrishnan, S., Sedgewick, J., Griffioen, A.W., 2000. Quantitative assessment of angiogenesis and tumor vessel architecture by
computer-assisted digital image analysis: effects of VEGF-toxin conjugate on tumor microvessel density. Microvasc. Res. 59 (3), 368 –376.