Machine learning based automatic detection of pulmonary trunk
Hong Wu, Kun Deng, and Jianming Liang
Department of Biomedical Informatics, Arizona State University
ABSTRACT
Pulmonary embolism is a common cardiovascular emergency with about 600,000 cases occurring annually and causing
approximately 200,000 deaths in the US. CT pulmonary angiography (CTPA) has become the reference standard for PE
diagnosis, but the interpretation of these large image datasets is made complex and time consuming by the intricate
branching structure of the pulmonary vessels, a myriad of artifacts that may obscure or mimic PEs, and suboptimal bolus
of contrast and inhomogeneities with the pulmonary arterial blood pool. To meet this challenge, several approaches for
computer aided diagnosis of PE in CTPA have been proposed. However, none of these approaches is capable of
detecting central PEs, distinguishing the pulmonary artery from the vein to effectively remove any false positives from
the veins, and dynamically adapting to suboptimal contrast conditions associated the CTPA scans. To overcome these
shortcomings, it requires highly efficient and accurate identification of the pulmonary trunk. For this very purpose, in
this paper, we present a machine learning based approach for automatically detecting the pulmonary trunk. Our idea is to
train a cascaded AdaBoost classifier with a large number of Haar features extracted from CTPA image samples, so that
the pulmonary trunk can be automatically identified by sequentially scanning the CTPA images and classifying each
encountered sub-image with the trained classifier. Our approach outperforms an existing anatomy-based approach,
requiring no explicit representation of anatomical knowledge and achieving a nearly 100% accuracy tested on a large
number of cases.
Keywords: Pulmonary Embolism, Pulmonary Trunk, Automatic machine learning based detection, Haar, AdaBoost,
Cascade
1. INTRODUCTION
Pulmonary embolism (PE) is the third most common cause of death in the US [1]. Computed tomography pulmonary
angiography (CTPA) has become the test of choice for PE diagnosis. However, interpreting these images is complex and
time consuming. Several approaches for Computer Aided Diagnosis (CAD) of PE in CTPA have been proposed [2-7].
However, none of these approaches are capable of detecting central emboli. To detect central emboli, the pulmonary
trunk must be first identified and segmented automatically. Automated identification of the pulmonary trunk is also of
significant value for the automated separation of the pulmonary arteries from the veins to remove the false positives
generated in the pulmonary veins (nearly half of all the false positives), and automatic adaptation to suboptimal bolus of
contrast to increase the robustness in detecting PE, etc. To this end, this paper presents a machine learning based
approach to automatically detecting the pulmonary trunk, achieving a nearly 100% accuracy on a large number of cases.
Our idea is to train a Boosting based classifier with a large number of image samples, so that the pulmonary trunk
can be identified by sequentially scanning the CTPA images and classifying each encountered sub-image with the
trained classifier. As a result, this approach requires no explicit representation of anatomy knowledge. Our approach
automatically scans the CTPA images at multiple scales to handle size variations of the pulmonary trunks. To solve the
extremely imbalanced data, and quickly reject the negatives, we adopt the approach which is derived from method
[8],[9]. We also explore alternative schemes of injecting negative examples and forming cascade structures.
2. DETECTING THE PULMONARTY TRUNK
Pulmonary trunk, as shown in Figure 1, is the main pulmonary artery that arises from the
right ventricle of the heart, extends upward, and divides into the right and left pulmonary
arteries carrying blood to the lungs. The significance of a highly accurate approach for
identifying pulmonary trunk is multi-fold:
1.
2.
Detecting central PEs: All the existing approaches for the automated PE detection in
the literature require the lung segmentation, which isolates the lungs but Figure 1: Pulmonary trunk
unfortunately also eliminates the pulmonary trunk along with the heart; (red dot) with central PEs
consequently, it is not possible for them to detect central PEs.
(indicated by arrows)
Removing false positive in the pulmonary veins: PEs are only found in the
pulmonary artery, but all those approaches have no capabilities in distinguishing the pulmonary arteries from
3.
the pulmonary veins, resulting in many false positives from the veins, for instance, nearly 50% of the false
positives found in the veins [4]. Automatic detection of the pulmonary trunk provides the start point to
distinguish the artery from the vein. As a matter of fact, radiologists are often confused by the artifacts in the
veins; therefore, automated separating the pulmonary artery from the vein is not only valuable for detecting PE
by computer but also by radiologists.
Improving robustness in PE detection: It is typical that CTPA scans have suboptimal bolus of contrast and
inhomogeneities with the pulmonary arterial blood pool. Therefore, automated detection of PEs must be selfadaptive to the contrast conditions. All those approaches except [10] are incapable of adapting contrast
conditions, but it discloses nothing about the approach used for detecting the pulmonary trunk.
2.1 Anatomy-based approach
To our knowledge, the only approach in the literature for detecting the pulmonary trunk is reported by [11], which
employs the anatomical knowledge of the lungs and the heart. Anatomically, the aortic arch lies immediately above both
the main pulmonary trunk approximately at the same level as the tracheal bifurcation. Computationally, it is easier to
detect air within the lumen of the trachea than adjacent vessels in the CTPA images, therefore, they proposed to first
identify and trace the trachea beginning at the level of the thoracic inlet inferiorly towards the point of division of the
trachea into right and left main bronchi (i.e., the tracheal bifurcation) to facilitate precise and reproducible identification
of the aortic arch. Once the aortic arch is localized, the pulmonary trunk can be robustly identified and segmented in a
bounding box defined by the aortic arch. Their evaluation showed that that this anatomy-based approach could correctly
identify the pulmonary trunk and provide acceptable segmentation in 60 (about 90%) of 67 cases.
However, this performance is not sufficient as a common foundation for automatically detecting central PEs,
separating the pulmonary artery from the veins, and adapting to the contrast conditions. Therefore, in this paper, we take
a different path and present a boosting-based approach to achieve an extremely high accuracy (nearly 100%) in detecting
the pulmonary trunk.
3. BOOSTING BASED APPROACH
Our pulmonary trunk detection algorithm is inspired by Viola’s face detector method [9], which was based on the
AdaBoost algorithm [12].
3.1 AdaBoost and Viola’s Face Detector
AdaBoost, short for Adaptive Boosting, is historically the first and perhaps the most popular member of the boosting
algorithm family. The basic idea behind AdaBoost is to repeatedly pick a weak learner with respect to a distribution and
add them together to form a final strong classifier. A weak learner, intuitively, is a classifier that only slightly correlates
with the true classification. A commonly used weak classifier, namely the decision stump, makes a prediction based on
the value of a single input feature. The first real-world application of AdaBoost came from Viola’s breakthrough work
on face detection. Their learning algorithm, based on AdaBoost, selects the most visually relevant features from
thousands of Haar features [13], each corresponding to a single decision stump weak learner. Their final face detector is
a cascade of boosted strong classifiers, which is used to scan the images and obtain candidate detections.
3.2 Cascade
Classifier cascade is a common technique for accelerating the object detection process. The idea is to arrange a series of
classifiers in a chain or cascade. If at any point in the chain, a candidate is rejected, no further processing is performed
and the search moves on to the next region. Clearly, an efficient classifier cascade should reject as many negative
candidates as possible, at the earliest possible stage of the cascade. However, several questions still remain. It is not
known how many strong boosted classifiers are needed, how many weak learners each boosted classifier should have,
and which combinations of ROC operating points would yield optimal performance. Therefore, in the rest of the paper,
we will experiment with several variations of cascading schemes in comparison with a single AdaBoost classifier.
4. METHODS
4.1 Viola’s Cascading Scheme
Our implementation of Viola’s AdaBoost uses TPR (True Positive Rate) 𝛼𝑖 , FPR (False Positive Rate) 𝛽𝑖 and max
number of weak learners 𝜂𝑖 as the criterion of training a cascade stage. Shown in Figure 2, 𝐷𝑖+ , 𝐷𝑖− refer to the positive
and negative sub-images and their initial weights for training an AdaBoost classifier at stage 𝑖. During training, more and
more weak learners are added to the stage until the target performance (𝛼𝑖 , 𝛽𝑖 ) or the maximum number of weak learners
𝜂𝑖 is reached, after which we start the next stage 𝑖 + 1. The output
𝑇𝑖
of stage 𝑖 is a boosted classifier containing {𝑓𝑖 }𝑖=𝑇
weak
𝑖−1 +1
learners. Upon terminating the current stage, the whole CTPA
dataset is scanned to search for false positives, so as to be used as
negative samples in the subsequent stage.
Figure 2: Viola’s cascading scheme
4.2 Multi-exit Cascading Scheme
Multi-exit AdaBoost is another way of cascading and injecting
negative samples. The difference from the Viola’s cascading
scheme is that the boosted classifier for each stage contains not
only the weak learners from its own but also all the previous
stages, as shown in Figure 3. For example, stage 𝑖 contains the
weak learners from 𝑓1 to 𝑓𝑇𝑖 .
4.3 Detection
Figure 3: Multi-exit cascading scheme
Detection candidates are generated by scanning the CTPA dataset slice by slice using one of the above cascading
schemes. A candidate sub-window, which is classified as positive by the cascading scheme, is characterized by a
particular location and size. Candidates that share over 25% of overlaps are merged as a single candidate. Candidates
are also merged in z-axis if their distance is less than 5 pixels. The final detection is the highest response candidate
among all the merged candidates in the CTPA dataset.
5. EXPERIMENTS
5.1 Data Preparation
Our experiments utilize 80 CTPA datasets for training and 77 CTPA datasets for testing. Each dataset contains 450 ~
600 slices of 512-by-512-pixels. The voxel size of these slices ranges from 0.5 to 0.7 mm in the axial plane, and the slice
thickness is around 0.5 mm. Of course, every dataset has one and only one pulmonary trunk. The position of a
pulmonary trunk is marked by a center point with a bounding box. Positive training examples are obtained by shifting
the bounding box 0~5 pixels, along the x, y, and z axes, and then resized to 25-by-25 pixels. All the following methods
share the same positive samples. Negative samples are selected randomly from the training datasets outside the bounding
box of the pulmonary trunk. Examples of positive and negative samples are shown in Figure 4 and Figure 5.
Figure 4: Positive samples.
5.2 Different approaches
5.2.1 Single AdaBoost
Figure 5: Negative samples.
Table 1. Detection accuracy of several approaches
Single AdaBoost
71/80 (88.75%) 60/77 (77.92%)
Viola’s AdaBoost
80/80 (100%)
76/77 (98.70%)
For comparison, 100 positive and 500 negative samples
Multi-exit AdaBoost
79/80 (98.75%) 77/77 (100%)
were chosen from each CTPA dataset to train a single
AdaBoost. No extra training samples are injected during
77/80 (96.25%) 74/77 (94.10%)
Single AdaBoost with
the training of 100 weak learners. 100% performance
selected negative samples
can be achieved on the training samples, while only
88.75% detection rate in the training cases and 77.92% in the testing are observed, as shown in Table 1.
5.2.2 Viola’s AdaBoost
In this approach, we started off with 100 positive and 100 negative samples from each training case. The training of a
cascade stage is terminated when 𝛼𝑖 is over 0.99 and 𝛽𝑖 is below 0.05 or 𝜂𝑖 hits 30. After exiting from one cascade, the
program scans the whole CTPA dataset to select at most 100 false positives from each case as the negative samples for
the next stage. The number of weak learners from cascade 1 to 6 is 1, 9, 22, 30, 30 and 8. In Figure 7, the accuracy
drops and FPR soars when new negative samples are injected. The FPR increases more dramatically in later cascade as
the negative samples in that cascade are more similar to the positive and hard to be classified correctly. The accuracy in
the detection of the testing cases hits 98.70%, shown in Table 1.
5.2.3 Multi-exit AdaBoost
We use the same parameters 𝛼𝑖 , 𝛽𝑖 , 𝜂𝑖 as the previous method. The number of weak
learners for stage 1 to 7 is 1, 9, 23, 40, 62, 90 and 100, respectively. Figure 8 is the
performance trend of multi-exit AdaBoost with cascade. The accuracy in the testing
cases reaches 100%.
5.2.4 Single AdaBoost trained with selected negative samples
To verify the representativeness of the samples and their influence on AdaBoost
algorithm, we inject false positive samples collected from the previous methods. The Figure 7: The performance of Viola’s
AdaBoost with cascade in training
negative samples are all the negative samples used in the 5.2.2 including all the
phrase.
negatives injected in between cascade stages. We also use the same process as in
Section 5.2.1 but only 50 weak learners (half of that in Section 5.2.1) to train a model
and use it to detect the PT. The accuracy of the detection increases significantly as
shown in Table 1.
6. CONCLUSION
In this paper, a machine learning based system has been developed to identify the
pulmonary trunk, which is opaque to anatomy knowledge of the underlying interesting
subjects. Our system is also insensitive to the position, size and intensity variations of
the pulmonary trunk and it can achieve near 100% accuracy in testing cases. Through
the experiments, we observe that the quality of the training data (i.e. representativeness
of the negative training examples) seems to be the utmost important factor for achieving
accurate detection rather than the cascade schemes.
Figure 8: The performance of multiexit AdaBoost with cascade in
training phrase.
ACKNOWLEDGMENTS
We thank Sebastien Paris for his help with our implementation.
REFERENCES
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
“The Surgeon General's Call to Action to Prevent Deep Vein Thrombosis and Pulmonary Embolism,” US Department of
Health and Human Services, 2008.
H. Bouma, J.J. Sonnemans, A. Vilanova, and F.A. Gerritsen, “Automatic Detection of Pulmonary Embolism in CTA Images,”
IEEE Transactions on Medical Imaging, vol. 28, Aug. 2009, pp. 1223-1230.
C. Zhou, et al., “Automatic Pulmonary Vessel Segmentation in 3D Computed Tomographic Pulmonary Angiographic (CTPA)
Images,” Proceedings of SPIE, San Diego, CA, USA: 2006, pp. 61444Q-61444Q-7.
J. Liang and J. Bi, “Computer Aided Detection of Pulmonary Embolism with Tobogganing and Multiple Instance
Classification in CT Pulmonary Angiography,” Information Processing in Medical Imaging, vol. 20, 2007, pp. 630-641.
R. Sebbe, “Computer-aided Diagnosis of Pulmonary Embolism in Opacified CT Images,” PhD thesis, Faculte Polytechnique
de Mons, Feb. 2007.
Y. Masutani, H. MacMahon, and K. Doi, “Computerized Detection of Pulmonary Embolism in Spiral CT Angiography Based
on Volumetric Image Analysis,” IEEE Transactions on Medical Imaging, vol. 21, Dec. 2002, pp. 1517-1523.
H. Bouma, “Vessel-Diameter Quantification and Embolus Detection in CTA Images.” PhD. Thesis, 2008, Technische
Universiteit Eindhoven, Eindhoven, The Netherlands.
M. Pham, V.D. Hoang and T. Cham, “Detection with Multi-exit Asymmetric Boosting,” IEEE Computer Society Conference
on Computer Vision and Pattern Recognition, Los Alamitos, CA, USA: IEEE Computer Society, 2008, pp. 1-8.
P. Viola and M. Jones, “Rapid Object Detection using a Boosted Cascade of Simple Features,” IEEE Computer Society
Conference on Computer Vision and Pattern Recognition, Los Alamitos, CA, USA: IEEE Computer Society, 2001, p. 511.
M.S. Dinesh, et al, “Adaptive Contrast-based Computer Aided Detection for Pulmonary Embolism,” Medical Imaging 2009:
Computer-Aided Diagnosis, SPIE, 2009, p. 726010.
X. Zou, J. Liang, et al, “Anatomy-based Automatic Detection and Segmentation of Major Vessels in Thoracic CTA Images,”
International Journal of Computer Assisted Radiology and Surgery, vol. 2, Jun. 2007, pp. 68-81.
Y. Freund and R.E. Schapire, “A Decision-theoretic Generalization of On-line Learning and an Application to Boosting,” J.
Comput. Syst. Sci., vol. 55, 1997, pp. 119-139.
F.C. Crow, “Summed-area Tables for Texture Mapping,” SIGGRAPH Comput. Graph., vol. 18, 1984, pp. 207-212.