1
Automatic differentiation of melanoma and clark nevus skin lesions
R.W. LeAnder, A. Kasture, A. Pandey, S.E. Umbaugh
Proceedings of the SPIE
Conference, Vol. 8, No. 33
Medical
Imaging
2007
ABSTRACT
Skin cancer is the most common form of cancer in the United
States. Although melanoma accounts for just 11 % of all types
of skin cancer, it is responsible for most of the deaths,
claiming more than 7910 lives annually. Melanoma is visually
difficult for clinicians to differentiate from Clark nevus lesions
which are benign. The application of pattern recognition
techniques to these lesions may be useful as an educational
tool for teaching physicians to differentiate lesions, as well as
for contributing information about the essential optical
characteristics for identifying them. Purpose: This study
sought to find the most effective features to extract from
melanoma, melanoma in situ and Clark nevus lesions, and to
find the most effective pattern-classification criteria and
algorithms for differentiating those lesions, using the
Computer Vision and Image Processing Tools (CVIPtools)
software package. Methods: The color differences between
images that occur because of differences in ambient lighting
during the photographic process were minimized by the use of
dermoscopic images. Differences in skin color between
patients was minimized by using normalizing them by means
of converting them to relative-color images, and differences in
ambient lighting during photography, and the photographic
and digitization processes, original color images were
normalized by converting them into relative-color images.
Tumors in the relative-color images were then segmented out
and morphologically filtered. The filtered-tumor features were
then extracted and various pattern-classification schemes were
applied. Results: Experimentation resulted in four useful
pattern classification methods, the best of which was a
classification rate of 100% for melanoma and melanoma in
situ (grouped) and 60% for Clark nevus.
This project is an extension of work done by Iris Cheng [1]
who developed a computer imaging system to enhance skin
tumor images to facilitate the differentiation between benign
and malignant tumors [2]. Cheng developed a relative-colorbased method which reduces the misclassification of tumors
caused by image-to-image variations in skin color between
patients, ambient lighting and the photographic and
digitization processes. Cheng’s relative-color approach
enlisted the following steps: 1) create the relative color images
from standard photographic images, 2) segment the images
using a PCT/median algorithm [3], 3) morphologically filter
the resultant image, 3) extract relative-color features from the
morphologically-filtered image to create two different feature
spaces –tumor-feature space and object–feature space, and 4)
find a statistical analysis model of the relative color features to
classify the various types of skin tumors. A database of 160
melanoma, 42 dysplastic nevi, and 80 nevus images was used.
Cheng was able to classify 86% of the malignant melanomas
successfully compared to an approximately 75% success rate
of dermatologists [1].
The significance of this present work is that it extends Cheng’s
use of the relative-color method to a database of dermoscopic
images rather than using photographic images. Dermoscopic
images allow more control over lighting variations which
contribute to lesion misclassification. The combination of
dermoscopic images with the relative-color method should
further reduce misclassification, thereby resulting in a higher
lesion-classification success rate. Analysis and control of
parameters that contribute to successful lesion classification
should contribute significantly to the education of physicians
needing to learn lesion differentiation as well as contribute to
the literature regarding important features of lesion
differentiation. Another significant aspect of this work is the
development of new classification algorithms as well as preprocessing and post-processing methods that can be used in
future research.
I. INTRODUCTION
III. MATERIALS AND METHODS
Of the various types of skin cancer that plague mankind,
melanoma is the most serious form. Having the greatest
potential to spread to other bodily tissues, it affects deeper
layers of the skin. Medical professionals classify melanoma as
being in the in-situ (localized) stage or the invasive stage. The
in-situ stage is the precursor of invasive melanoma. The
principal objective of this project was to automatically identify
and differentiate melanoma, melanoma in situ and Clark nevus
skin lesions.
Image Database: A database of 60 color dermascopic lesion
images was used. The database was subdivided into three
categories consisting of 20 images each: melanoma, melanoma
in situ and Clark nevus. All the images were 768 x 512 pixels
and stored in raw, compressed format, to reduce border image
artifacts.
II. BACKGROUND
Software: CVIPtools (Computer Vision and Image Processing
tools), version 4.4c, was used to perform most of the image
processing and computer vision operations.
A. MATERIALS
2
Border images: Border images (BIs) are binary images having
a black background and a white object of interest. They are
generated by using a mouse to manually draw a border around
an object of interest (tumor) in the original color image. The
border images were used to mask tumors out of the original
color images to retrieve “skin-only” images. A logical NOT of
the BIs was used to mask patient skin out of the original color
images to generate “tumor-only” images.
Relative-Color Images: Cheng’s algorithm in Figure 1 was
used to generate relative-color images from the “tumor-only”
and skin-only images. For one original-color image, the
average RGB values of its skin-only image were subtracted
from its tumor-only image to generate a relative-color image.
Due to differences in lighting, skin color between individuals
and the photographic and digitization processes, original-color
images were normalized by converting them into relative-color
images.
B. METHODS
Image preprocessing, generation of relative-color images,
feature extraction and pattern classification proceeded as
follows (Refer to Figure 1.):
1) Preprocess the original-image: In order to minimize
artifacts dermoscopic-gel bubbles and camera flash were
removed from dermoscopic images using a second-order,
Contra-Harmonic Filter (CHF) that had a size-5 mask [3].
2) Generate border images. (See the Materials section.)
3) Generate relative-color images: For one original-color
image, the average RGB values of its skin-only image were
subtracted from its tumor-only image to generate a relativecolor image. (See Figure 1.)
4) Reduce the number of objects in the lesions using
segmentation and morphological filtering. Segmentation:
CVIPtools’ Principal-Color-Components Transform algorithm
was used to segment the relative-color images into four,
principal, homogeneous regions of color. Morphological
filtering: In order to smooth the shapes of image objects, their
projections were pared and holes were filled, using a 9-pixeldiameter circle as a structuring element [3].
experiments, features were extracted from the whole lesion
which was treated as one object.
6) Normalize the data: A variety methods were used to
normalize the data, including the unit-vector, statisticallybased (statistic-based), min-max and softmax scaling (softmax)
methods [3].
7) Choose a metric to be used in the pattern-classification
algorithm: Three different distance metrics were used to
compare the training-set feature vectors to those in the test
sets, including the city-block, Euclidean-distance and
maximum-value (max-value) metrics. For one experiment a
similarity measure was used called the vector inner product
[3].
8) Develop a pattern-classification algorithm: The database
of 60 lesion images was divided into sets of 30 images each.
One 30-image set was divided into training and test sets of 15
images each. After training and testing, if the classification
results were consistent, the algorithm that was developed was
executed on the other set of 30 images. If the consistency
prevailed on that set of 30, the algorithm was then executed on
a set of 50 images (30 original and 20 chosen from the test set
of 30).
Input Image
Preprocessing with Contra
harmonic filter. Mask size = 5
Order = 2
Skin only
Tumor only
Calculate average RGB value of
skin only image.
Subtract
PCT/Median segmentation. Number
of desired colors = 4
Morphological filtering, Structuring
element = circle size = 9
Feature extraction
5) Extract features from the lesion’s objects: A variety of
feature vectors were generated using various combinations of
the following features: area (ar), thinness (th), perimeter (per)
and histogram features, including the mean (m), standard
deviation (sd), energy (en), and entropy (ent). The texture
features that were extracted included inertia (tx-inr) and
entropy (tx-ent). In some experiments, the above features were
extracted from the two, largest objects in the lesion; in other
Pattern classification
END
Fig. 1: Flowchart of algorithm for generating relative-color images,
extracting tumor features and classifying them. The relative-color-imagegeneration segment mitigates differences in skin color between individuals, as
3
well as any differences in the photographic and digitization processes, thereby
normalizing original-color images.
120
100 100
100
100
A variety of combinations of feature measurements and
pattern-classification methods were then applied to “the 50”.
Using the training set of 50, 200 experiments were performed,
using various image processing, feature extraction and patternclassification methods (See Table 1). Out of the 200
experiments, the four with the best results are graded,
compared and reported here (See Table 1 and the Results
section).
Success Rate 80
(%)
70
Exp 1
Exp 2
Exp 3
Exp 4
Segmented Image
Used As
relativecolor
relativecolor
relativecolor
BI mask
BI mask
morph.filtered
morph.filtered
3
1
whole
tumor
3
1
whole
tumor
3
2
largest
objects
2
2
largest
objects
ar, per, th,
ent, sd, m
ar, per, th,
ent, sd, m,
en
ar, per, th,
ent, sd, m,
en
min-max
statisticbased
softmax
scaling
vector
inner
product
No. of Classes to
Identify
No. of Objects
Analyzed per
Lesion
ar, per, ent,
sd, m, txinr, tx-ent
Features
Extracted
Data
Normalization
Method
Distance or
Similarity Metric
Used
unit-vector
city-block
max-value
K-nearestK-nearestnearestneighbor
neighbor
nearest
Classification
centroid
(K = 5)
(K = 3)
centroid
Method
Table 1. Experimental set-ups of the best four feature-extraction &
pattern-classification experiments.
61
60
53
50
43
In_situ
501
40
Clark Nevus
Total Avg %
33
20
3
20
0
0
1
2
3
4
90
60
100
60
0
20
33
100
0
87
70
50
50
53
43
61
100 %
Melanoma
Clark Nevus
Total Avg %
original
Melanoma
60
60
In_situ
Parameters
Used
Image Used
90
87
Experiment No 2
7
2
Fig 3. Bar graph for results of Experiment 2. Same training and test set
schemes as in Fig. 2.
100
90
80
90 87
89
70
80
Success Rate
60
(%)
40
45
70
47
45
23
20
43
54
50
30
20
0
1
2
3
4
45
23
43
in situ
90 %
80
20
30
50
clark
90
70
89
70
Total %
87
45
47
54
Melanoma
Experiment No 3
Fig 4. Bar graph for results of Experiment 3. Same training and test set
schemes as in Fig. 3.
150
IV. RESULTS
100
Success Rate
(%)
100
Success Rate
50
(%)
00
100
90
80 80
67
40
5660
80
69
53
33
30 30
6
54
50
0
1
2
3
4
Melanoma
80 %
56
30
50
in situ
80
60
30
33
clark
40
0
67
0
90
100
80
69
53
54
Total
7
Experiment No 1
Fig 2. Bar graph for results of Experiment 1. For Bar Set #1 (four bars):
Training set = test set = 15 images; Bar Set #2: Training set = test set = 30
images; Bar Set #3: Swapping training and test set of 30 images; Bar Set #4:
Training set = 50 images, test set = 10 images.
88 81 84
100
93
61
77
80
60
50
0
150
75 75 75
1
2
3
4
melanoma + In situ
75 %
88
93
100
clark
75
81
61
60
Total %
75
84
77
80
Experiment no 4
Fig 5. Bar graph for results of Experiment 4. Same training and test set
schemes as in Fig. 4.
4
Feature
Extraction &
Pattern
Classification
Experiment
No. of Lesions
in
Training/Test
Set
%Melanoma
+ In Situ
Identified
% Clark
Nevus
Identified
Avg
%
1
15/15
75
75
75
2
3
30/30
Swapping
30/30
50/10
88
93
81
61
84
77
100
60
80
4
Table 2. Results from the best four feature-extraction & patternclassification experiments.
V. DISCUSSION
Comparison of Cheng’s with the Present Results
When Cheng used the Multi-Layer Perceptron method, she got
better classification results for classifying melanoma than
when she used the Principal Component Analysis method [3].
Her feature space included histogramic features, but not any
texture features. In contrast, this present study included texture
features. Cheng’s best results for classifying melanoma with
nevus lesions was 78%, overall. Separately classified, her
algorithm correctly identified melanoma, 86%, dysplastic
nevus, 56%, and nevus 62% of the time. Figures 2-5 show the
results of the feature-extraction and pattern classification
experiments in this present study which are summarized in
Table 2. As can be seen in Table 2, the best classification
scheme was in Experiment #4. Note that in contrast to Cheng’s
study, the texture features, inertia and entropy, were included.
The pattern classification algorithm used here (Exp. #4) that
gave the best results included the Softmax Scaling
normalization, the Vector Inner Product similarity metric and
the Nearest Centroid classification functions in CVIPtools [3].
The overall success rate for classifying melanoma, melanoma
in situ and Clark nevus was 80%. The success rate for
melanoma with melanoma in situ grouped together was 100%.
The classification success rate for Clark nevus alone was 60%.
.
The main difference between the present method and Cheng’s
is that the present method uses dermoscopic instead of
photographic images and texture features. The dermoscopic
images helped minimize the differences in ambient lighting
that is caused by photographic methods. Dermoscopic images
also provided more details of the skin lesions than
photographic images. By the inclusion of texture features, the
better results presented here indicate that tumors’ texture
features are salient features and important to include in tumor
feature extraction, analysis and classification.
Clark nevus. So, grouping them together increases the
classification rate.
The results of this study were consistent with theory in that as
the training-set size increases, the results improve. During the
course of experimentation it was observed that better results
were more consistent when the training set was expanded to 30
or 50 tumor images, compared to using a training set of only
15 images. This indicated that our data set was self-consistent
and complete. Misclassification of melanoma and melanoma in
situ for an expanded training set, suggests that there is a
considerable overlap between the two categories of melanoma.
This indicates that some additional features are required to
automatically differentiate those classes.
Images of a Tumor at Various Stages of Processing Before
Feature Extraction and Pattern Classification
Fig 6. Original Image. Lesion w/ skin, Fig 7. Effects of preprocessing.
camera flashes, gel bubbles and hair.
Fig 8. Relative Color Image.
Fig 10. Morph.-filetered image
REFERENCES
1.
Yue (Iris) Cheng, Ragavendar Swamisai, Scott Umbaugh,
Randy Moss, Willliam V. Stoecker, Saritha Teegala,
Subhashini K. Srinivasan, “Skin lesion classification
using relative color features”, Skin Research and
Technology (Denmark); accepted for publication.
2.
Iris Cheng, “Skin tumor classification using relative color
features”; Thesis submitted in partial fulfillment for
Master’s degree in Electrical Engineering at Southern
Illinois University, Edwardsville, Illinois, 2006.
VI. SUMMARY AND CONCLUSION
In differentiating melanoma from Clark nevus lesions,
grouping melanoma and melanoma in situ together, got the
best results because those two sets of tumors (melanoma and in
situ) have features and feature values that are similar enough to
be considered one class of tumor that significantly differs from
Fig 9. PCT-Segmented image
5
3.
S.E. Umbaugh, “Computer Imaging: Digital Image
Analysis and Processing”, CRC Press; January 2005.
4.
US 2004 Cancer Facts and Figures, American Cancer
Society, 2004, Published by American Cancer Society,
Inc,
http://www.cancer.org/docroot/STT/content/STT_1x_Ca
ncer_Facts__Figures_2007.asp”.