Color Image Segmentation for Automatic Alignment of Atomic Force
Microscope.
by
Pradya Prempraneerach
B.S., Mechanical Engineering
Carnegie Mellon University, 1998
Submitted to the Department of Mechanical
in Partial Fulfillment of the Requirements for the Degree of
Master of Science in Mechanical Engineering
at the
Massachusetts Institute of Technology
February 2001
ENPI
MASSA C HLUSEUTS INaTITUTE
OF TECHNOLOGY
© 2001 Massachusetts Institute of Technology
All rights reserved
JAN 16 2002
LIBRARIES
Signature of Author......
V
......................
I
Department of Mechanical Engineering
February 1, 2001
Certified by...................
-fV
Prof. Kamal Youcef-Toumi
Professor, Mechanical Engineering
Thesis Supervisor
Accepted by ........................................
Dr. Ain A. Sonin
Professor, Mechanical Engineering
Chairperson, Departmental Committee on Graduate Students
I
Color Image Segmentation for Automatic Alignment of Atomic Force
Microscope.
by
Pradya Prempraneerach
Submitted to the Department of Mechanical Engineering on
February 1, 2001 in Partial Fulfillment of the Requirements for the
Degree of Master of Science in Mechanical Engineering
ABSTRACT
Traditionally, an alignment process of the Atomic Force Microscopy's (AFM)
probe and sample manually and accurately is slow and tedious procedure. To be able to
carry out this procedure repeatedly in large volume process, an image recognition system
is needed to process this procedure autonomously. An image recognition system is
mainly composed of two separated parts, segmentation and classification methods.
In this research, a new color-image segmentation procedure is proposed to
overcome the difficulty of using only gray-scale image segmentation and oversegmentation. To extract color information similar to human perception, a perceptually
uniform color space and to separate a chromatic value from an achromatic value,
CIELUV and IHS color spaces are used as basis dimensions to obtain the coarse
segmentation. The intervals of each dimension: intensity, hue and saturation are
determined by the histogram-based segmentation with our valley-seeking technique.
Then the proposed three-dimensional clustering method combines this coarse structure
into distinct classes corresponding to objects' region in the image.
Furthermore, the artificial neural networks is employed as classification method.
The training and recognition processes of desired images are applied using unsupervised
learning neural networks: competitive learning algorithm. Typically, images acquired
from the AFM are restricted only to similarity and affine projections so that main
deformations of the objects are planar translation, rotation and scaling. The segmentation
method can identify the position of samples, while the objects' orientation must be
correctly adjusted before processing the recognition algorithm.
Thesis Supervisor: Prof. Kamal Youcef-Toumi
Title : Professor of Mechanical Engineering
2
Acknowledgements
These thesis and research become successful with many supports of several
persons, whom I am really appreciated and would like to thank here. First of all, my
advisor, Prof. Kamal Youcef-Toumi, I would like to thank him for giving me an
opportunity to work on this exciting research topic and also his motivation and
suggestions to make this research become possible. It has been an invaluable experience
for me to accomplish this project.
Furthermore, I would like to extend my thanks to my family, especially my father,
mother and grandmother for supporting and taking care of me through my life. I am very
grateful to have them as my family. Also, I like to thank my sister for her thoughtfulness
and cheer.
In addition, I would like to give special thanks to my friends, both in Thailand and
at MIT. I would like to thank my lab-mates, particularly Bernardo D. Aumond, El Rifai
Osamah M, and Namik Yilmaz, in the "Mechatronics Research Laboratory" for many
recommendations on how to operate the equipment as well as the software and a friendlyworking environment. I also like to thank Susan Brown for several suggestions on editing
this thesis. Lastly, I like to give a special thank to my friend, who I do not want to
mention name here, in Thailand for understanding and support me during this time.
3
Table of Contents
ABSTRACT
ACKNOWLEDGEMENTS
TABLE OF CONTENTS
2
3
4-5
Chapter 1: - Introduction
1.1)
Introduction
Background and Motivation
1.2)
1.3)
Objective of this research
1.4)
Thesis Organization
6-8
8-10
10-11
11
Chapt -r 2: - Recognition System and Comparison
2.1)
Introduction
2.2)
Recognition system
Segmentation algorithm
2.3)
2.4)
Feature selection for segmentation
2.5)
Clustering
2.6)
Summary
12
12-13
13-15
15-18
18
19
Chapt -r3: - Science of color
Science of color
3.1)
3.2)
The CIE system
3.3)
Uniform Color spaces and color difference formulas
3.4)
Different color space for representing color image
3.5)
Summary
20
20-31
31-35
35-45
45-46
Chapt r 4: - Proposed Segmentation Technique
4.1)
Introduction
4.2)
Histogram-based segmentation
Filtering: Morphological filtering operations and
4.3)
Edge-preserving filtering operation
4.4)
Histogram valley-seeking technique
4.5)
Three-dimensional color-space cluster detection
Experimental results, discussion and evaluation
4.6)
4.6.1) Evaluation measurement function
4.6.2) Experimental results and evaluation
4.6.3) Segmentation results of natural scenes
4.7)
Summary
47-48
48-52
52-56
56-62
62-67
67-76
76-77
77-92
93-103
104-106
107-108
Chapter 5: - Conclusion and future works
5.1)
Conclusion
Further improvement and future works
5.2)
109-110
110-111
Appendix A: Introduction of Morphological operations.
112-115
4
Appendix B:
B.1)
B.2)
B.3)
B.4)
Classification - Neural Networks
Introduction to Neural Networks
Competitive learning Neural Networks
Experimental results and discussion
Summary
116-117
117-122
123-127
128
Appendix C: Segmentation program
129-176
References
177-179
5
Chapter 1: - Introduction
1.1: Introduction
For the past decade, profilometry and instrumentation techniques for imaging
material surface at atomic level with outstanding accuracy have been developed rapidly.
All these techniques are known as Scanning Probe Microscopy (SPM), including
Scanning Tunneling Microscopy (STM), Atomic Force Microscopy (AFM) and etc.
However, in this research, the main focus is only on the AFM. The AFM operation is
carried out by detection of a displacement of cantilever's probe due to an inter-atomic
interaction known as van der Waals forces between the probe and the samples' surface,
while the probe scans the samples' surface. Generally, the AFM cantilevers have
dimensions in the order of 100 micrometers and the tip of the probe that is mounted on
the cantilever is usually in the 10 micrometers range. The main components of the AFM
are the cantilever with mounted probe, a large photodetector array for measuring the
cantilevers' deflection, sample holder and positioner, a piezoelectric actuator, and a
control system to maintain the position of the probe with the samples' surface. In
addition, the AFM in our experimental setup also consists of a ccd camera, motorized and
non-stacked x-y translation stages and z-axis motorized stage for a visual purpose of
aligning and positioning the probe over certain samples' part to be scanned.
Traditionally, a manually probe-alignment process between the tip of cantilever
with the particular features of sample is very tedious and repeated process. This manual
procedure uses the adjustable microscope eyepiece for visualization and then adjusts the
samples' position with a manual x-y-z stage. In macroscopic manufacturing processes
such as semiconductor industries also require a high dimension accuracy with small range
of tolerance; thus, high productivity rate and rapid improvement become very difficult to
achieve with manual probe-sample alignment of the AFM. Then, the computerized
control system with the vision feedback is desirable to achieve the automatic probesample alignment.
With the improvement in computer hardware and software performance,
especially in computer graphic, the real-time image processing have just been
6
accomplished in the past few years. One of the most important image-processing
implementation is an object recognition system. Even though, the first object recognition
research started in the early 1960s, the primary goals of the object recognition have not
been altered since then.
The basic objective of object recognition system is the identification of what and
where the objects are presented in either a single image or a sequence of images. In the
literature, there are two major kinds of the recognition systems. Both approaches suffer
from different difficulties. The first approach is known as the geometric or invariant
approach, such as Grimson's [13], LEWIS (Rothwell
[27]), Vosselman's
[33]
approaches, this method can solve the objects' pose or orientation simultaneously along
with specify the objects' location. This procedure requires objects' contours or edges that
are often obtained as small fragments of curve; thus, an efficient search algorithm is
required to reduce the search time for the objects' geometric correspondence. Typically,
the edges are computed from gradient of illumination only. On the other hand, the second
technique is composed of two main algorithms: segmentation and classification. The
segmentation is used to answer the question of where the locations of each object are in
the image, while the classification is employed to identify what objects are existing in the
viewing area. The efficiency and accuracy of this approach depend upon both of these
algorithms. In this research, the attention is focused on the second approach, since it is
not strictly constrained with geometric shapes of objects. Nevertheless, all algorithms
need the objects' geometric pose or orientation for interpretation of objects in three
dimensions from planar images. In our experimental setup, the images viewing from the
AFM are only affected by an orthographic projection in Euclidean plane. Therefore, a
problem with the large distortion in the image caused by perspective projection does not
apply in our case.
In general, the segmentation technique is a method that partitions the images into
independent and uniform regions corresponding to parts of object or objects of interest to
satisfy certain criterions. Then these segmented regions can be used to answer where each
objects' position is in the field of view of the camera. A three-dimensional clustering
technique in the XYZ normalized color space studied by Sarabi and Aggarwal [30] is one
of the segmentation techniques. In their method, the clusters of each class are formed, if
7
the value of each pixel locates within some distance determining from certain thresholds.
Our proposed segmentation technique is explained later in more detail.
The classification method employs the resulting regions from segmentation
technique in such a way that the classification algorithm can react purposefully to these
segmented regions or signal-like inputs. In other words, classification is a decisionmaking process deriving from particular functions so that the computer can perform
certain intelligence for assigned tasks. One of the simplest decision-making functions is a
correlation function. If the value of correlation is above a predetermined threshold value,
then that object has certain characteristic that most likely match with the desired object.
Other
existing
classification
methods,
such
as
parametric
and nonparametric
classification (from Fukunaga [9]) and neural network, have been extensively researched;
however, they are beyond the scope of this thesis.
In this thesis, we will cover the two main issues. The first topic is the formulation
of the proposed segmentation technique and the second one is the probe-sample
alignment of the AFM with use of our segmentation technique. The following chapter
begins with the motivation of this research and the issues of segmentation and the AFM
alignment.
1.2: Background and Motivation
The AFM is one of the most popular profilometry instruments among the SPM
technique, since the AFM operation bases on the contact force between tip-sample
interaction. Therefore, the AFM does not have a limitation to image a non-conductive
sample. Comparing to the STM, which is relied on the electrical current flowing through
between a tip-sample separation, is restrict to operate on conducting samples' surface
only. As mentioned in last section that in manufacturing process of large quantities with
small-error tolerance, the probe-sample alignment is time-consuming process and in
certain case it must be done repeatedly. The computerized recognition system becomes
essential in this alignment procedure. In addition, our great interest is focused on the
capability in aligning the probe with the straight edge of sample, because with this
particular sample, the capability of our algorithm can be tested easily for an aligning
8
accuracy. Our experiments use the AFM machine, model Q-Scope 250, with additional
motorized x-y translation stage manufactured by Quesant Instruments Corporation. The
picture of this model of AFM is shown in Figure 1.1.
phctosensibve array
piezo-tube
Image
X
Figure 1.1: The AFM machine model Q-Scope 250 manufactured by Quesant
Instrumentation Corporation (On the right - from "Quesant Instrument Corporation") and
the typically schematic diagram shows how the AFM operates (On the left - from "Image
Processing for Precision Atomic Force Microscopy." by Y. Yeo).
In the past, several segmentation techniques commonly produced a oversegmentation problem caused by illumination variation of the images, since most
segmentation algorithms depended on gray-scale images which contain only intensity
information. These illumination effects include shadow, bright reflection, gradual
variation of lighting. The primary reason is that computer have not been powerful enough
to incorporate the images' color information into those algorithms. Recently, the
computation of large number of data can be accomplished within remarkable small
amount of time. Hence, it is our advantage to combine color information into
segmentation algorithm, similar to other works done in [5] and [30], to overcome the
variation of lighting. However, the most appropriate color space must be taken into the
consideration. In this thesis, the uniform CIELUV color space that was recommended by
the "Commission Internationale de l'Eclairage" (CIE) for graphical industry applications
9
is used as for the color conversion. Then, the perceptual color space, IHS, which is
similar to human perception, is derived from the CIELUV. The transformations of
different kinds of color spaces, including CIELUV and IHS, are explained in more detail
in chapter 3.
Most progress in object recognition system that is currently used in industrial
applications strictly applies to simple and repeatable tasks in the controlled environments.
For instance, the packaging industries employ a vision system to classify the different
products so that robots or machines can separate and pack them accordingly. However,
the vision system that can recognize any general objects or tasks is far more complicate
because of requirement of tremendous amount of data bases and computational
complexity.
Therefore, in this study, a proposed segmentation algorithm, that is one of
significant part of recognition system, is concentrate on method that can apply with any
general images, as described in more detail in chapter 4. Another major advantage of our
segmentation over many existing segmentation methods is that the prior information of
objects in the image as well as the number of classes of objects and user interaction is not
required. Moreover, our proposed procedure makes use of coarse-scale segmentation and
then refine these results with proposed clustering technique to achieve fast computational
time for real-time implementation.
1.3: Objective of this research
According to the need of the general purposed objection recognition system, in
this research, the proposed segmentation technique is the first phase to achieve this
complex system in the near future. Currently, most existing recognition systems operate
in controlled environment for certain specific tasks. Therefore, our formulation technique
is concentrate on 1) partition any acquired images into meaningful regions corresponding
to specific part of objects or object of interest, 2) fast computational time for real-time
implementation as well as 3) incorporation of color information to overcome
conventional problems with use of gray-scale information only.
10
Another goal of this research is to apply the proposed segmentation algorithm to
accomplish the probe-sample alignment of the AFM. Our segmentation algorithm must
correctly detect and locate both the AFM cantilever and the desired sample from the
image viewing from the AFM so that the position of the tip of the AFM cantilever can be
aligned with the particular position of the sample. Both of these positions is then used as
feedback signals to control motorized x-y translation stage to reposition the sample
accordingly.
Finally, in the literature, most segmentation results are evaluated visually;
therefore, the evaluation methods for determining the segmentation results are proposed
to satisfy our objectives based upon the general segmentation criteria suggested by
Haralick and Shapiro [14].
1.4: Thesis Organization
Chapter 2 briefly describes the recognition system of this study and several
segmentation techniques are compared and discussed. In chapter 3, the color science,
including colorimetry, color matching functions similar to the human perception, the CIE
chromaticity diagrams and etc., will be describe in great detail. Furthermore, the
explanation, formulation and comparison of different kinds of color spaces; RGB, XYZ,
CIELAB, CIELUV, YUV, YIQ, YCbCr, 111213 and IHS, are also given in chapter 3. Then,
in chapter 4, the three main steps of our proposed segmentation technique are presented
along with the evaluation and discussion of this method tested on both synthetic and real
images. The experimental results of the probe-sample alignment of the AFM with use of
this algorithm are shown and discussed in section following the formulation of the
proposed algorithm. The evaluation functions to justify the segmentation results are given
in chapter 4 as well. Finally, the conclusion and further improvements of this proposed
technique are presented in chapter 5.
11
Chapter 2: Recognition System and Segmentation Comparison
2.1: Introduction
This chapter briefly introduces the structure of the second type of recognition
system in section 2.2. Then, the different segmentation algorithms that have been
researched in the literature are presented along with their advantages and disadvantages
in section 2.3. In section 2.4, the several segmentation results are compared to choose the
most appropriate method in the sense of feature selection. Various clustering techniques,
given in section 2.5, are introduced as basic for the proposed clustering algorithm in
chapter 4. Finally, the summary of selection is discussed in section 2.6.
2.2: Recognition system
As mentioned briefly in last chapter, answering the questions of what and where
the objects are existing in the image are the primary purposes of general recognition
systems. The reason for using recognition system is to automatically perform the tasks
that are required high accuracy and done repeatedly in a short period of time. The second
type of recognition system, which is consisted of segmentation and classification
algorithms, is major focus of our interest due to its adaptation for a general-purpose
system. The schematic diagram of this type of recognition system is shown in Figure 2.1
below. Generally, one of the most important image enhancements is the noise reduction
process. Several kinds of filtering process; such as low-pass, band-pass and gaussian
filters, have been used for smoothing out the images. A feature selection is predetermined
alternative for different segmentation and classification techniques, as described later in
this chapter. For the past few years, classification algorithms have been extensively
researched and formulated; such as several standard methods in Fukunaga's book [9];
nevertheless, their inputs are based on the good and meaningful segmentation results.
Therefore, in this research, a new segmentation algorithm is proposed and implemented
to fulfill this need.
12
Enag en
,e Featr
1nSegmentation
Classification
Figure 2.1: The schematic diagram of the second type of the recognition system.
2.3: Segmentation algorithms
The main objectives of image segmentation are to partition an original image into
homogeneous separated regions corresponding to desired objects and then to identify the
position of the segmented regions representing the existing objects in the image. Interior
regions should be uniform. And each region should not contain too many small holes.
Boundaries of each segment should be simple and the spatial information should be
correctly located on the regions' boundary. For illustration purpose, if the original image
is composed of a single object, which is blue color, on a white background; this is
obvious that blue region must correspond to the existing object in the image. Therefore,
this is a very simple task to label all pixels with blue color in the original image as one
object and pixels with white color as the other object. However, in natural scenes, there
are many variations, such as, shadow, change in lighting, color fading and etc, in the
viewing image. All variations come from both environment and viewing equipment.
These uncertainties make the segmentation problems more challenging. Satisfying all of
the desired requirements is very difficult task. In the literature, image segmentation
algorithms were classified mainly into four categories: pixel-based, edge-based, regionbased and physically-basedsegmentation techniques.
The pixel-based segmentation uses the gray-scale or color information from each
pixel to group them into certain classes for specifying or labeling objects in image. There
are many ways to classify each pixel corresponding to objects of interest and each
method has different benefits and drawbacks. Histogram-based segmentation, distancebased pixel classification and maximum likelihood pixel classification are some examples
of the pixel-based segmentation technique. All of these techniques use only the global
information described in a certain space to classify each pixel in the original image. In
this study, the histogram-based segmentation approach is used as a basis to obtain a
coarse segmentation, since histogram can be considered as a density distribution function.
13
Therefore, the pixels that are in the same interval of histogram show the same specific
characteristic. The benefit of this approach is neither user's input nor "priori" information
of the image required, more details of this method are described in section 4.2.
The major difficulty of the edge-based segmentation is to obtain closed and
connected contours of each object. This technique first computes the edge features of the
image and assigns these features in a vector form. Then, the segmentation results are
achieved by clustering these edge vectors to form close-contour regions. As mentioned
earlier, the edges are calculated from a gradient in specific area of image; therefore, the
edges usually scatter as a fragment in the image as the result of images' variation.
Examples of this technique are the edge tracking and edge clustering (Z. Wu and R.
Leahy [34]). The computation of this approach is very expensive; for example, Wu's and
Leachy's approach take about 10 minutes to accomplish the good segmentation result of
the brain image acquired from the Magnetic Resonance Imaging (MRI) process.
Primary advantage of the region-based segmentation over the pixel-based
segmentation is that spatial information is conserved; therefore, there is less small
isolated noise or improper classified regions from images' variation of color and
illumination within uniform regions. The most important criterion of this method is to
partition the original image into separated regions that are smooth or homogeneous. The
homogeneity measurement, seed-based region growing (R. Adams and L. Bischof [1]),
recursive
split-merge
segmentation,
neighborhood-based
approach
using Markov
Random Fields (J. Liu and Y. Yang [19]) and watershed algorithm (J.M. Gauch [10]) are
some examples of this type of segmentation technique. Most of these methods,
particularly region growing, demand a prior information, posteriori probability, and
iterative processes with large computational expense. To deal with image processing in
real time, iterative process is a major disadvantage.
The elementary physical models of the color image formation to produce color
differences are used in the physically-based segmentation. However, in this method, the
main drawback is that segmented regions do not follow the objects' boundaries,
segmented perimeters instead follow the variation in lighting and color similar to the
edge finding process and the pixel-base segmentation. Even though, in 1990 Klinker,
Shafer and Kanade [17] proposed a model called "Dichromatic Reflection Model
14
r
-
-
_
_ ___
_.
I
-
-
-
-
--
-
-
. ......
- ....
. ........
........
.............
-t......
...
.
. ............
....
. ..
. .............
(DRM)", this model yields good segmented results only in restrict viewing environment
because of rigid assumptions of the model.
2.4: Feature selection for segmentation
According to all segmentation techniques explained in the last section, so far no
report of the segmentation techniques is confirmed to be the best approach. In our
opinion, the region-based segmentation seems to give the best results in the sense of their
feature selection, region base, instead of using edge or pixel information. However,
typically the region-based segmentation required iteration process to achieve good
results. For the feature-selection comparison purpose, the examples of edge detection,
watershed algorithm [10] and histogram-based segmentation [29] are applied to the AFM
images with poor and with good illumination conditions, the photograph of parrots and
Lena, as shown in Figures 2.3, 2.4, 2.5 and 2.6 respectively. Before applying these
segmentation algorithms, all of the original images are filtered with the edge-preserving
filter, described later in section 4.3. All original images are shown in Figure 2.2. The
edge detection is done using the canny operator in Matlab and the watershed algorithm
based on the Gauch's algorithm is implemented using the gaussian filter with G=20 in
C++ language. The Matlab's command of the edge detection operation used in these
examples is "edge(I,'canny')", where I is a gray-scale image. The histogram-based
(a)
(b)
(c)
(d)
Figure 2.2: (a) An original AFM image with poor lighting condition; (b) An original
AFM image with good lighting condition; (c) An original parrots image; (d) An original
Lena image.
15
........
..............
-.........
...............
.........
. . ....
(b)
(a)
(c)
Figure 2.3: (a) The canny edge detection; (b) Watershed segmentation; (c) Histogrambased segmentation.
I
(a)
(b)
(c)
Figure 2.4: (a) The canny edge detection; (b) Watershed segmentation; (c) Histogrambased segmentation.
(a)
(b)
(c)
Figure 2.5: (a) The canny edge detection; (b) Watershed segmentation; (c) Histogrambased segmentation.
16
(a)
(b)
(c)
Figure 2.6: (a) The canny edge detection; (b) Watershed segmentation; (c) Histogrambased segmentation.
segmentation, done in C++ language, uses our histogram valley-seeking technique,
described later in section 4.4, to separate the intensity histogram of each picture into
certain intervals. The computational time of the canny edge detection, watershed
algorithm and histogram-based segmentation are in the order of less than 10 seconds, 2
minutes and less than 10 seconds respectively.
From the above results, the canny edge detection gives very poor performance,
since usually the edges are not connected across the regions that have low value of
gradient. Besides that there are some cases that two edges corresponding to the same line
are obtained. Hence, the close boundaries of objects are remarkably difficult to recover
from the edges. On the other hand, with use of the watershed algorithm, close contours of
objects are guaranteed, since the segmentation results are often small-segmented regions.
Although, most segmented regions preserve all edge information due to their local
minimum property, its main disadvantage is over-segmented. The histogram-based
segmentation of intensity also results in closed contour and most of the boundaries of
segmented regions are corresponded to the objects' edge. Nevertheless, the images'
variations, especially illumination, lead this method to over-segmentation problems.
Particular, the AFM image with poor illumination condition separated into many
segmented areas due to the shadow, illumination fading and noise effect. As a result of
comparison above, the histogram-base segmentation is more suitable in several aspects:
fast computational time, closed boundary and less over-segmentation of segmented
17
regions. Thus, this technique is then used as the coarse segmentation of the proposed
algorithm, described later in chapter 4.
2.5: Clustering
In this section, several clustering techniques are briefly introduced along with
their benefits and drawbacks as a background for the proposed clustering method in
chapter 4. One of the well-known unsupervised classification methods that do not
requires an assist of training sets is the clustering technique. Most clustering methods do
not require any prior information of the sample data; therefore, certain characteristic in
those data must be used to separate them into distinct groups or classes. The particular
characteristic for defining the clustering is the most significant and basic issue that is
predetermined to satisfy the desired objective.
In the literature, the clustering algorithm is categorized into two main approaches:
the parametricapproach and the nonparametricapproach. Most parametric approaches
require a specific clustering criterion and a given number of samples so as to optimize the
separation of samples according to that desired criterion. The class separability measures
method is the best parametric approach. However, all parametric approaches demand
iterative process to classify sampled points in specific spaces. In case of the image
processing, these processes need to classify each pixel according to their criterion in each
iteration. The convergence to satisfy that particular criterion is another problem of
iterative procedure. Furthermore there is no discrete theory for determining the number of
classes. The examples of the parametric clustering are the nearest mean reclassification
algorithm, branch and bound procedure and maximum likelihood estimation (Fukunaga
[9]). For the nonparametric approach, the valleys of density function are used for
separation of each class. Nonetheless, the problem associates with this approach occurs
when the shape of density functions becomes very complex to define the boundaries. The
estimation of density gradient and reduced parzen classifier (Fukunaga [9]) and the threedimensional clustering technique based on a binary three of RGB values (Rosenfeld [26])
are the examples of nonparametric procedures.
18
2.6: Summary
The image segmentation is the first component of the object recognition system,
because the classification methods are based upon good segmentation results for solving
the question of where each object does exist in the viewing image. Even though, various
segmentation algorithms were proposed in last few years, the most common criterion of
these algorithms is to partition the image into meaningful and uniform regions
corresponding to the object presenting in the picture. All other general criteria are
proposed and summarized by Haralick and Shapiro [14]. Several segmentation
techniques, edge detection, watershed algorithm and histogram threshold are shown in
this chapter for a feature-selection purpose. From the comparison, the histogram-base
segmentation yields most suitable result in several aspects: fast computation, producing
closed contour and less over-segmentation. Therefore, our proposed segmentation
technique uses the histogram-base segmentation method as coarse segmentation. Then,
the proposed clustering technique, described in chapter 4, refines this coarse
segmentation to obtain better segmentation results.
19
Chapter 3: - Science of color
3.1: Introduction
In this chapter, we discuss a science of color that have been developed for
standard measurements and representations by the "Commission International de
l'Eclairage" (CIE). Most researchers have searched for the color representations that
closely matches with human perception. This chapter covers the most important color
representations, RGB, XYZ, YUV, YIQ and etc. The CIE recommended the well-known
CIELAB and CIELUV color spaces that closely represent human visualization. Both of
these color spaces are suitable for industrial-graphical applications. Our proposed
algorithm, in chapter 4, also uses CIELUV color space as basic space for clustering.
Lastly, the advantage and disadvantage of each color space are discussed.
3.2: Science of color
According to Isaac Newton's experiment, he discovered that a prism separates
white light into strips of light or "visible spectrum". These strips of light, consisting of
violet, blue, green, yellow, orange, and red, include all color in human visible range.
Subsequently, these visible spectrums form white light by directing it through a second
prism. This is the first evidence that spectral colors in the visible spectrum are the basic
component of white light. Later on, light is discovered as a form of electromagnetic
radiation that can be described by its physical property, wavelength (symbol X).
Wavelength has units of nanometer (nm). The electromagnetic radiation varies from the
shortest cosmic rays (10-6 nm) to radio waves with wavelength of several thousands
meter. Therefore, the visible spectrum is a small range of electromagnetic spectrum from
380 to 780 nm.
Color science study can be divided into many topics such as radiometry,
colorimetry, photometry, and color vision. It also covers how human beings distinguish
color. However, this study is concentrate only on colorimetry, which numerically
specifies color stimulus of human's color perception. Due to the recent development of
20
cheaper and more advanced image processing devices so that concepts and methods of
colorimetry have become more practical.
The basis of colorimetry is developed by the "Commission Internationale de
l'Eclairage" (CIE), well know as "CIE system". According to CIE, colorimetry is divided
mainly into three areas: color specification, color difference and color appearance. In this
study, we will focus on color difference and appearance. Most of the basis colorimetry
theories have been developed by the CIE.
3.3: CIE system
There are a lot of existing color specification systems and color spaces; for
example, RGB, XYZ, YIQ, LUV, however, the most significant of all color specification
systems is developed by the CIE. A standard procedure for specifying a color stimulus
within controlled viewing conditions was provided by the CIE system. It was first
established in 1931 and further developed in 1964. Both of these systems are well known
as the "CfIE 1931" and "CIE 1964" supplementary systems. Three important components
of color perception are light, object and eye. These also were studied and standardized
according to the CIE system as explained in following sections.
3.3.1: Light
All color perception came from lighting. A light source can be identified by
measuring its "spectral power distribution" (SPD - S(X)) using a spectroradiometer. This
SPD is defined in term of the relative distribution; so, the absolute SPD is not significant
for colorimetric formulation. There are many kinds of light sources. For example,
daylight is the most common one, it is a mixture of direct and sky-scattered sunlight.
Daylight's SPD varies from quite blue to quite red. In addition, artificial light sources are
also widely used such as fluorescent, incandescent, and etc. The SPD variation of light
sources is varies in wide range, therefore, it is not necessary to make color measurement
using all possible light sources. The CIE suggested a set of standard illuminants in form
21
of a SPD-value table for industrial applications, but the illumination may not have a
corresponding light source.
Moreover, the light source can be specified in term of "color temperature", which
has unit of Kelvins, K. Color temperatures are separated mainly into three types,
distribution temperature, color temperature and correlated color temperature. Firstly, a
distribution temperature is temperature of a "Planckian" or black-body radiator. When the
Planckian radiator, an ideal furnace, is kept at a particular temperature, it will discharge
radiation of constant spectral composition. Secondly, a color temperature is the
temperature of a Planckian radiator whose radiation has the same chromaticity as that of a
given color stimulus. This is mainly used to prescribe a light source that has SPD similar
to a Planckian radiator. Thirdly, a correlated color temperature is the temperature of a
Planckian radiator whose color is match closely to a given color stimulus. This is
commonly used for light source that has SPD quite different from a Planckian radiators
such as fluorescent lamps.
According to CIE in 1931, three standard illuminants are known as A, B, and C
which corresponding to the representation of incandescent light, direct sunlight, and
average daylight respectively. Color temperature of A, B, and C are 2856, 4874 and 6774
K correspondingly and their SPD are shown in Figure 3.1. The CIE suggested a gas-filled
tungsten filament lamp denoting the standard source A. The suggested standard sources B
and C can be produced by using the standard source A combining with liquid color filters
of specified chemical compositions and thickness. Later on, the CIE found that the
illuminants B and C were unsatisfied for certain applications due to very small power in
the ultraviolet region. Therefore, the CIE recommended another series of the D
illuminants in 1964. The SPD of D65 also display in Figure and noticeably there is large
distinction between illuminants of D65 and B or C around the blue-color wavelength. The
most common used of D illuminants are D50 and D65 for the graphics arts and the
surface color industries respectively.
22
A
1200
0900
B
600
..
300
04
300
700
600
500
400
Wavelength
800
900
( nm )
Figure 3.1: Relatives SPDs of CIE standard illuminants: A, B, C and D65 (from "The
Colour Image Processing Handbook" by S.J. Sangwine and R.E.N. Home [29]).
3.3.2: Object
Reflectance, R(X), of an object defines the objects' color as a function of
wavelength. Reflectance can be described by the ratio of the light reflected from a sample
to the light reflected from a reference white standard. Moreover, there are two types of
reflectance, "specular" and "diffuse". Generally, some part of the light that direct into
the surface will reflect back with the same angle as incident angle respect to the surfaces'
normal axis, this is called specular reflection. Some part of the light will penetrate into
the object. The rest will be absorbed and scattered back to the surface, this scattered light
is called diffuse reflection. Measuring surface reflectance can be done using a
spectrophotometer. The CIE also specifies standard methods to achieve instrumental
precision
and accuracy
measurement.
Firstly,
a perfect
reflecting
diffuser is
recommended as the reference white standard. Secondly, four different types of
illuminating
and
viewing
geometries,
45
degree/Normal,
Normal/45
degree,
Diffuse/Normal, and Normal/Diffuse are recommended as shown in Figure 3.2. These
four illuminating and viewing geometries can be divided into two categories: with and
without an integrating sphere. Therefore, to obtain accurate reflectance result, it is very
essential to specify the viewing geometry and illumination together with inclusion and
exclusion of specular reflectance.
23
Viewing
Illuminating
Integrating
Viewing
a
Gloss trap
sphere
Illuminating
Baffle
d/O
0/45
Viewing
Illuminating
Illuminating
Gloss
Integrating
sphere
Baffle
Viewing ------
45/0
trap
0/d
Figure 3.2: Schematic diagram showing the four CIE standard illuminating and viewing
geometries for reflectance measurement.
3.3.3: Eyes
The most important components of the CIE system is the color matching functions. These
functions characterize how human eyes match color stimuli using a standard set of red,
green, and blue references. Also, the color matching properties of the "CIE Standard
Colorimetric Observers" are described by these functions.
The CIE color matching functions are based on the fundamentals of additive color
mixing. In additive colorimetry, red, green and blue are the primary colors. For example,
the addition of red and green light fluxes in appropriate proportions produces a new
color, yellow. In general, a color "C" is produced by adding "R" units of the red primary,
"G" units of the green primary and "B" units of the blue primary. Thus, color match can
be formulated as the following equation:
C[C] = R[R] + G[G] + B[B]
Where R, G, and B are the amount of red [R], green [G], and blue [B] references stimuli
required to match the C units of [C] target stimulus. The + and = sign represent an
additive mixture and matching or defining respectively. Above equation is a unique
24
definition of the color C or known as Grassman's first law of color mixture. Another
fundamental law of color mixture, Grassman's second law, defines a color as a mixture of
two stimuli 1 and 2. It can be expressed by:
Stimulus 1
C1[C] =
R1[R] +
G1[G] +
B1[B]
Stimulus 2
C2[C] =
R2[R] +
G2[G] +
B2[B]
Resulting Color
(C1+C2)[C] =
(R1+R2)[R] + (G1+G2)[G] + (B1+B2)[B]
Generally, to represent a particular color required three independent quantities, red, green
and blue primaries, so graphical representations have difficulty in representing a threedimensional quantities onto two-dimensional plane. Thus, chromaticity coordinates, r, g,
and b represent the proportions of red [R], green [G] and blue [B] instead of their
absolute values become advantageous. Thus, only two of these variable are need for color
representation. The chromaticity coordinates are calculated using:
r = R/(R+G+B)
g = G/(R+G+B)
b = B/(R+G+B)
Where r, g, and b are the proportions of the three reference basics, [R], [G], and [B], and
r, g and b sum to unity. Color mixture is one of the significant property of a chromaticity
diagram, For example, colors Cl and C2 are mixed additively, the resulting color C3 is
on the straight line drawn from CI to C2.
According to the CIE in 1931, a set of standard color matching functions, known
as the "CIE 1931 Standard Colorimetric Observer" (or 2 degree Observer) for use with
visual color matching between 1-degree and 4-degree field sizes. These functions were
derived from two equivalent sets of visual experiment carried out by "Wright" (19281929) and Guild (1931) based upon 10 and 7 observers respectively. In order to combine
the results from these two sets of experimental data, they are normalized and averaged.
The functions are denoted as r(X), g(X) and b(X) which corresponding to color stimuli
of wavelengths 700 nm, 546.1 nm, and 435.8 nm for red, green, and blue primaries
25
.
. ..
. .........
..
...........................
...
.. .............
respectively. Their units were adjusted to match an "equal energy white", which has a
constant SPD over the spectrum. These functions are shown in Figure 3.3 below.
3,
2.5
2
1.5
0
-0.5 1
4
5
Wavelength (nm)
Figure 3.3: The color matching functions for the CfIE 1931 Standard Colorimetric
Observer expressed using R, G, and B basic at 700, 546.1 and 435.8 nm, respectively.
From the plot, negative tristimulus value show that the light was added to the test color
instead of to the red, green, and blue mixture. Consequently, these CIE 1931 2-degree
color matching functions seriously underestimates sensitivity at wavelengths below 460
nm or at short wavelengths. Therefore, these functions, r(X),
g(X) and
b(X), were
linearly transformed to a new set of functions, x(X), y(X) and z(X), to get rid of the
negative coefficients in the previous set. The x(X), y(X) and z(X) functions are also
plotted on Figure 3.4. Furthermore, in 1964, a new set of color matching functions were
formulated by the CIE for use with a more accurate correlation with visual color
matching of fields of larger than 4 degree at the eye of the observer. This set of new
functions was also derived from the experimental results by Stiles and Burch in 1959 and
by Speranskaya in 1959. These functions is called the "CIE 1964 Supplementary
Standard Colorimetric Observer" (or 10 degree Observer), and is denoted by xio(X),
yio(X) and z1o(X). These functions are also plotted on the same Figure 3.4 below.
26
1
2.5111
1
2
0.5 05
0
350
-
400
450
50
--
--
-
6
650
550
Wavelength (nm)
..
4.._
70
-
---
750
'--
WO
Figure 3.4: The color matching function for the 1931 Standard Colorimetric Observer
(solid lines) and for the 1964 Supplementary Standard Colorimetric Observer (dashed
lines).
3.3.4: Tristimulus values
In the previous section, three essential components of color perception have been
expressed in terms of functions of the visible spectrum. The standard colorimetric
observer is defined by either the functions of x(X), y(X)and z(X) or the function of
xio(X),
ylo(X) and
zio(X), to represent the person having normal color vision. The
illuminants are standardized in terms of the SPD, S(X). In addition, the CIE standardized
the illuminants and viewing conditions for measuring a reflecting surface. Surface
reflectance is defined by R(X). Thus, the "tristimulus values", (X,Y,Z) are the triple
numbers that described any color. These X, Y, and Z values define a color in the amounts
of red, green and blue CIE basics required to match with a color by the standard observer
under a certain CIE illuminant. So, these values are the integration of the products of
S(X), R(X), and x(X), y(X) and z(X) in three components over the visible spectrum.
X = kJS(X)R(X) x(X) d%
Y = kfS(X)R(X) y(X) d%
Z = kfS(X)R(X) z(X) dX
27
Where the k constant was chosen such that Y = 100 for the perfect reflecting diffuser.
And either ( x, y, z ) or ( x10 , yio, Z10 ) can be use for small and large angle observer
respectively.
For measuring self-luminous colors such as light-emitting colors or light sources,
the following equation must be applied instead of the above equations, since the object
and illuminant are not defined.
X = kIP(X) x(X) d
Y = kIP(X) y(X) d
Z = kfP(X) z(X) dX
Where P(X) is the function representing the spectral radiance or spectral irradiance of the
source. Similarly to previous equations, the k constant is chosen such that Y = 100 for the
appropriate reference white. Usually, the areas color display have quite small angular
subtense so that the CIE 1931 Standard Colorimetric Observer (x,y,z) is more accurate
functions to be applied.
The color described in the CIE system generally plots on the "chromaticity
diagram" shown in Figure 3.5 below. And the chromaticity coordinates are calculated
using the formulation as followed.
x = X / (X+Y+Z)
y = Y / (X+Y+Z)
z = Z / (X+Y+Z)
and
x+y+z=1
Thus, only two variables from x, y and z coordinates are need to represent the color on
the chromaticity diagram.
28
I......
. ..................
.................
_ - I- - -
Figure 3.5: The CIE chromaticity diagrams for the 1931 2-degree standard colorimetric
observer.
Noticeably, the region of all perceptible color is bounded by the horseshoe-shaped curve
of pure monochromatic spectral colors or "spectrum locus", with a straight line
connecting the chromaticity of extreme red and blue. This line is known as the "purple
line". Point E in the middle of the spectrum locus is denoted the white color and other
colors become more saturated toward the outer curve. An important feature of this
chromaticity diagram is that only point, that lies on any straight lines, comes from an
additive mixture of two color lights, that lies on the same straight line. This property is
similar to the property in r, g and b coordinates. For example, if the amounts of two given
colors are known, the position of the mixture of these two can be calculated.
Consequently, color specification can be specified by tristimulus values and
chromaticity coordinates. Nevertheless, these representations cannot characterize color
appearance, such as, lightness, saturation, and hue, it is very conceptual to visualize color
from either tristimulus values or chromaticity coordinates. From tristimulus value,
lightness or intensity can be calculated from Y value. The brighter the color is, the larger
Y value will be. So, the CIE recommended another variables, "dominant wavelength"
and "excitation purity" to represent the hue and saturation attributes respectively. These
29
............
.
two parameters can be calculated from chromaticity diagram as the following example,
which specified attribute values plotted on Figure 3.6 below. Let a surface color, denoted
by point C, it observes with a particular illuminant, denoted by point N, on diagram
below. Draw a straight line from point N pass through point C to intersect with the
spectrum locus at point D, so a dominant wavelength of color C is about 485 nm, which
is equivalent to a greenish blue color. Also, excitation purity, (Pe), is a ratio of line NC to
line ND. The excitation purity is closely related to saturation attribute. In this example, it
is about 0.7, representing a quite saturation color.
Moreover, for a certain color that lie close to the purple line, like the color C', a
line NC' doesn't join the spectral locus. The line NC' must be extended in the opposite
direction until it intersect the spectrum locus. The intersection point on the spectrum
locus denotes the "complementary wavelength", in this case is about 516 nm. Also, its
excitation purity, which is about 0.85, is still a ratio of line NC' to line ND'.
Figure 3.6: Variable values plotted on the CiIE 1931 chromaticity diagram to demonstrate
the calculation of the dominant wavelength, complementary wavelength and excitation
purity. (from -"The Colour Image Processing Handbook" by S.J. Sangwine and R.E.N.
Home [29]).
30
. ............
3.4: Uniform Color spaces and color difference formulas
To standardize color in industrial applications, the CIE proposed a way to
represent the color under a standard viewing conditions as described in previous section.
Colorimetry is also useful for evaluating color difference in color quality control.
However, after 1964, several color discrimination experimental results had shown that
the XYZ space in chromaticity diagram from CIE 1931 and 1964 shown very poor
uniformity results. Since, for all color, color differences are not represented by lines or
circles of equal size from the white point E in Figure 3.5. The lines representing
noticeable vary in size by about 40 to 1, being shortest in the violet region and longest in
the green region on 1931 chromaticity diagram. This showed that attempt have been
made to find a diagram that approaches more nearly "uniform chromaticity diagram". In
1937, MacAdam suggested the transformation below from (x,y) coordinate to (u,v)
coordinate and was accepted by the CIE in 1964.
u=4x /(-2x+ 12y+3)
v=6y/(-2x +12y+3)
This transformation has been used until 1974, when more recent investigations,
particularly from Eastwood, suggested that the v coordinate should increase by 50%,
while the u coordinate remain unchanged. Then, in 1976, the CIE restrain this
modification as a new chromaticity diagram called the "CIE 1976 uniform color space
diagram" or " the "CIE 1976 UCS diagram, which represent uniform space perceptually.
This new space, (u',v') coordinates, is a projective transformation from the tristimulus
values or (x,y) coordinates as the following equations:
u' = 4X / (X + 15Y + 3Z)= 4x / (-2x + 12y + 3)
v' = 9Y / (X + 15Y + 3Z)= 9y / (-2x + 12y + 3)
Unfortunately, this leads to an ambiguous decision from users' point of view to choose
between (u,v) and (u',v') coordinates for more accurate color representation. Therefore,
31
. ............
................
. ........
-46A
this subject still continues to be under active consideration by the CIE colorimetry
committee. The CIE 1976 UCS diagram, shown in Figure 3.7 below, also has the additive
mixture property as well as the CIE 1931 and 1964 diagrams.
Figure 3.7: The CIE u'v' chromaticity diagram.
CIELAB and CIELUV color spaces
In 1976, the CIE recommended two new uniform color spaces, CIE L*a*b*
(CIELAB) and CIE L*u*v* (CIELUV). CIELAB is a nonlinear transform of the
tristimulus space to simplify the original ANLAB formulation by Adams in 1942, widely
used in color industrial. CIELAB and CIELUV is given as the following equations:
3.4.1: CIELAB color space
L*=ll6f(-
-16
a*=500[f(
-f
32
b* =200 f=
O[f (YY
f (Zj]
And
x 1/3
if x >0.008856
f(x)= 7.787x +
otherwise
Where X, Y, Z and Xo, Yo, Zo are the tristimulus values of the sample and a specific
reference white respectively. Generally, the value of Xo, Yo, Zo can be obtained from the
perfect diffuser illuminated by a CIE illuminant or a light source. L* denotes the
lightness value and it is orthogonal to a* and b* spaces, while a* and b* represent
relative redness-greenness, yellowness-blueness respectively. Converting from a*, b*
axes to polar coordinate related to hue and chroma attributes as the following.
hab*
tan-,lb
(a*
Cab*=
a*
2
+b* 2
The equation of color difference between two sets of L*a*b* coordinates (L*i,a*i,b*i)
and (L* 2 ,a*2 ,b* 2) is :
AEab
2
=
+A*2+
*2
where AL* = L*2 - L* 1 , Aa* = a*2 - a*i and Ab* = b*2 - b* 1 .
or
A*2 +AC
ab
*ab2
+
*ab 2
where Al *ab = PV2(C *ab,1 C *ab,2 -a Ia
=
{IP
2
*1 b* 2 ) and
if a*, b*2 >a*2 b*1
otherwise
and subscripts 1 and 2 represent the considered standard pair and considered sample pair
respectively.
33
3.4.2: CIELUV color space
In CIELUV color space, L* is the same as in CIELAB color space. The other coordinate
are given as the following:
U*= 13L *(u - u o )
V* = 13L * (v - vo)
h
= tan-
C.. =
u* 2 +v *2
Where u, v, Y and uO, vo, Yo are the u, v coordinates of samples and a suitable chosen
white respectively. Variables, u and v in above equation can be substituted by either u
and v or u' and v'.
The equation of color difference is:
AEU = VAL *2 +Au * 2 +Av * 2
where AL* = L* 2 - L*i, Au* = u*2 - u*I and Av* = v*2 - v*i.
AL *2 +AC*
or AE
where AH *
=-1
2
+AH *
= pJ2(C *uvl
2
C *uv,2
-U*
U *2 -V * V *2)
and
ifu* 1 v*2 >u* 2v*I
Iotherwise
P
and similarly subscripts 1 and 2 represent the considered standard pair and considered
sample pair respectively.
Also, the saturation attribute can be calculated from CIELUV color space as
followed:
SU =13 (u-u) 2 +(v -vO)2
34
The three dimensional representation of the CIELAB color space is shown in
Figure 3.8 in L*a*b* coordinates. The value of L* vary from 0 to 100 corresponding to a
black and reference white respectively. And a* and b* values represent the rednessgreenness and yellowness-blueness values correspondingly. In CIELUV color space, u*
and v* have the similar representation as a* and b*. The
Cab*
scale is an open scale on
horizontal plane with zero at the origin on the L* vertical line. The hue angle has range
between 0 to 360 degrees; however, a pure red, yellow, green and blue colors are not
located at 0 degree, 90 degree, 180 degree and 270 degree angles. Instead, hue angle is
corresponding to the rainbow-color sequence.
L*
100
White
Yellow+)
C*ab
Green ()1
Red (+)
Blue()
0
Black
Figure 3.8: A three dimensional representation of the CIELAB color space.
3.5: Different color space for representing color image
Generally, color space or coordinate system usually describes in three dimensions
depending upon image processing application.
Certain color space and color
transformation is more suitable for one image processing system than regular RGB. This
is discussed and explained in the following section. However, each color can be
presented in one of these three-dimensional spaces. Different color spaces were derived
from various sources or domains or developed particularly for image processing.
35
3.5.1: RGB color space
RGB color space is the most basic space for other types of color space in image
processing. Also, RGB is the most commonly used one for both direct input and output
signals in color graphics and displays. However, different devices provide variation in
RGB outputs signals, so certain characteristic for each device must be taken into account
during processing. Generally, most TV system uses either standard PAL/RGB or
NTSC/RGB in encoder and decoder operations. The RGB color gamut can be model as a
cube in Figure 3.9 below such that a point either on a surface or inside of this cube
designates each RGB color. Then, the gray color can be describe on the main diagonal of
this cube from black color, which has R, G and B values equal to 0, to white color, which
has R, G and B values equal to maximum value of the cube.
B
Blue (O,OBmax)
Magent
(Rmax,O,Bma )
Cyan (0,Gmax,Bmax)
White (Rbaax,Gmax,Bmax)
Gray color
Line
Black (0,0,C)
Red (Rmax,0,0)
~
Green (0,Gmax,0)
G
Yellow (Rmax,Grnax,0)
R
Figure 3.9: RGB color space for representing each color.
The main disadvantage of RGB color space in most applications involves in
natural high correlation between its elements: about 0.78 for R-B, 0.98 for R-G and 0.94
for G-B elements. Their components' unorthogonality is the reason why RGB color space
is not desirable for compression purpose. Furthermore, other drawbacks of RGB color
space are hard for visualization of color based on R, G and B components and its nonuniformity in evaluating the perceptual difference depending upon distance in RGB color
36
gamut. As mentioned in section 3.3.3 that normalization is required to transform from R,
G and B to r, g and b coordinate for illumination independence. According to some
experiments by Ailisto and Piironen in 1987 and Berry in 1987 verified that rgb
coordinates are much more stable with changes in illumination level than RGB
coordinates. Importantly, dark color pixels can produce incorrect rgb values because of
noise effect, so that these pixels should be ignored in image segmentation process
(Gunzinger, Mathis and Guggenbuehl, 1990). This dark color effect appears in the AFM
image with poor illumination condition; thus, the noise level is very high as shown in
section 4.6.
3.5.2: XYZ color space
As described in section 3.3.3, the XYZ color space was accepted by the CIE in
1931 and its main objective was to eliminate negative tristimulus values in each r, g and b
coordinate. The transformation from RGB to XYZ color space is described as the
following equations given by Wyszecki and Stiles in 1982:
X = 0.490 R + 0.310 G + 0.200 B
Y = 0.177 R + 0.812 G + 0.011 B
Z = 0.000 R + 0.010 G + 0.990 B
For the chromaticity coordinate system, the XYZ coordinate also has the normalized
transformation equations given by Wyszecki and Stiles in 1982, similar to RGB
coordinate. Thus, only two variables are independent and need to describe color in twodimensional plane. The Y element in this color space represents the image illumination.
x = X / (X + Y + Z)
y
=
Y / (X + Y + Z)
z
=
Z / (X + Y + Z)= 1 - x - y
37
From above transformation, reference stimuli in the CIE RGB space, R, G, B and white
reference, convert into the following values:
Reference Stimuli
x
y
Reference Red
0.735
0.265
Reference Green
0.274
0.717
Reference Blue
0.167
0.009
Reference White
0.333
0.333
There are another two versions for transforming from RGB to XYZ color space,
they are called EBU RGB (European Broadcasting Union) and FCC RGB (Federal
Communications Commission, USA). Both of these transformations, based on the
tristimulus values of cathode ray tube phosphors, have been employed in color
reproduction of most color cameras. The transformation from RGB to EBU color space is
given as the following:
X = 0.430 R + 0.342 G + 0.178 B
Y = 0.222 R + 0.707 G + 0.071 B
Z
=
0.020 R + 0.130 G + 0.939 B
And the transformation from RGB to FCC color space is given as the following:
X = 0.607 R + 0.174 G + 0.200 B
Y = 0.299 R + 0.587 G + 0.114 B
Z = 0.000 R + 0.066 G + 1.116 B
According to, both transformations, reference stimuli in EBU and FCC color
space, R, G, B and white reference, are calculate as the following values:
38
FCC
EBU
Reference Stimuli
x
y
x
y
Reference Red
0.640
0.330
0.670
0.330
Reference Green
0.290
0.600
0.210
0.710
Reference Blue
0.150
0.060
0.140
0.080
Reference White
0.313
0.329
0.310
0.316
The reference white for EBU color space is the illuminant D65, while the FCC's is the
illuminant C, mentioned in section 3.3.1.
The XYZ color space often used as an intermediate space in the process of
calculating uniform color space, either CIELAB (L*a*b*) or CIELUV (L*u*v*). Also, in
this study, EBU color space is used as the intermediate space before converting to
CIELUV color space.
3.5.3: Television color spaces
The television color spaces define a luminance element and two chromaticity
elements, depending upon color-difference signals: R-Y and B-Y, separately such that
they were known as "opponent" color spaces. This separation is designed to minimize the
bandwidth of the composite signals. This is major advantage of television color space.
According to Slater study in 1991, a spatial detail in luminance is far more sensitive than
a spatial detail in chromaticity. Thus, the bandwidth of the chromaticity signal can be
reduced tremendously comparing to luminance signals'.
3.5.3.1: YUV color space
The YUV color space is the basis for the PAL TV signal coding system, most
commonly used in Europe. Their transformation from RGB coordinates is given below:
39
Y= 0.299R+0.587G+0.114B
U
=
-0.147 R - 0.289 G + 0.437 B
=
0.493 (B - Y)
V= 0.615 R - 0.515 G - 0.100 B = 0.877 (R- Y)
The luminance, Y, element is the same as the Y element in XYZ color space. IHS space
can be transformed from YUV space as the equations below. Note that both Y and I
represent the illumination signal.
H. = tan-ij
Due to chromaticity signal required less information than luminance signal, the
bandwidth can be arranged as follows: 80% of bandwidth used for Y component and 10%
for each of the U and V components (Monro and Nicholls, 1995). Therefore, special
chips for digital video processing and other image compression hardware often use YUV
color space with either Y:U:V 4:2:2 or Y:U:V 4:1:1 bandwidth ratio. YUV have been
used of coding for color images and videos as well as color image segmentations.
3.5.3.2: YIQ color space
The YIQ color space is the basis of the NTSC TV signal coding system, used in
America. Y element is the same as in Y in YUV color space and in XYZ color space.
This YIQ relates to FCC RGB coordinates the following equation given below:
I = 0.596 R - 0.274 G - 0.322 B
=
0.74 (R - Y ) - 0.27 (B - Y)
Q = 0.211 R - 0.523 G + 0.312 B= 0.48 (R - Y)+ 0.41 (B - Y)
Both I and Q components express jointly hue and saturation ratio. Similarly, IHS space
can be transformed from YIQ space as the equations below. Similarly, the expressions for
Y and I components are the same.
40
HQ = tan
SJQ =
I2 +Q
2
Several image processing has been used YIQ color space as basis for object recognition,
image coding, edge detection and color image segmentation. For instance, the I
component was used for extraction of human faces from color images given by Dai and
Nakano in 1996.
3.5.3.3: YCbCr color space
This color space is independent of TV signal coding systems. It is currently used
in image compression such as JPEG format and in process of video sequences encoding
on videodisks. The Y component is also identical to Y component of YUV and YIQ color
space. Their transformation from RGB coordinates is given below:
Cb= -0.169 R - 0.331 G + 0.500 B =0.564 (B - Y)
0.500 R - 0.418 G - 0.081 B
Cr
0.713 ( R - Y)
3.5.4: OHTA 111213 color space
According to Ohta, Kanade and Sakai in 1980, they presented this color space in
the following equations:
I
12
13
=(R
+ G + B) /3
(R - B ) /2
=(2G
- R- B
) /4
Similar to the television color space, the luminance, Ii, is defined separately from
chromaticity elements, 12 and 13. These three components are good approximation of the
41
results of the Karhunen-Loeve transformation, best in aspect of decorrelation of RGB
components. This color space had been used in color recognition, image segmentation,
image compression and image analysis as well.
3.5.5: IHS and related perceptual color spaces
Though, human color receptor absorb light with the highest sensitivity in the
spectrum range, red, green and blue, then signals from the cones are processed further in
the visual system. In the perception process, a human can easily recognize basic attributes
of color: intensity (brightness or lightness) - I, hue - H and saturation - S. The hue and
saturation values are very correlated to the dominant wavelength of the color stimulus
and color purity (lack of white in the color) respectively. So, a pure color has saturation
value equal to 100%, while a gray-scale color has saturation value equal to 0%. In case of
intensity, a pure white has a maximum value and a pure black has a minimum value. The
IHS color space can be described visually in cylindrical coordinate as in Figure 3.10.
Intensity
Saturation
Figure 3.10: IHS color space in cylindrical coordinate.
In the literature, the hue was expressed in various forms owing to its compromise
between accuracy and simplicity of the calculation. The first derivation was done by
Tenenbaum, Garvey, Weyl and Wolf in 1974 and later on was acknowledged by Haralick
42
and Shapiro in 1991 as the basic formula for hue. This formula, shown below, is quite
complex.
2r - g -
H = cos
6
(r
-
3J 2 +
(g
b
3)
+
(b-
3)
Where H = 360 - H, if b > g and H has unit of degrees. This formula can also be
calculated in RGB coordinate system as the following:
H
=
- G )+ (R - B)]
Hcs
cos = -'0.5[(R
V(R - G )(R - G ) + (R - B )(G - B )
Where H = 360 - H, if B > G. The faster version for calculating hue, shown below, is
done by Kender in 1976. This formula doesn't contain any square root and it is composed
of a few multiplications.
For achromatic case: R = G = B then H is undefined
For chromatic case:
If min(R,G,B)
=
B then
(G-R
H = -r+ tan3
(_G-B)+(R-B)
If min(R,G,B) = R then
If min(R,G,B)
=
G then H
=z+tn
H~_
V3(B - G)
((B -R) +(G -R))
53r
3(R - B)
= + an(R -G)+(B-
G),
Where min(*) is the minimum of * variables. Later on, hue formulae were simplified by
Bajon, Cattoen and Kim in 1985 so that the formulae do not contain any trigonometric
43
functions. However, these new formulae produce slightly different result from previous
two hue formulae. The Bajon's formulae are expressed as followed:
If min(R,G,B)
=
B then H=
If min(R,G,B)
=
R then
If min(R,G,B)
=
G then
H
(G - B)
3(R + G - 2B)
1
3(G+B-2R) 3
(B - R)
(R - G)
2
3 (R+B-2G)
3
For the saturation element, S, the basic equation is:
S =1-3min(r,g,b) = 1 -
3
min(RG,B)
R+G+B
Nonetheless, the above equation was modified to minimize instability in the region near a
point S
=
(0,0,0) by Gordillo in 1985.
-3
If I<= 'max then S= I
3
"
min(Ima - R, Ima G, Imax B)
3 - (R + G + B)
If I > '"x then S = Imax - 3( min (R, G, B)
3 R+G+B
3
For the intensity element, I, the basic equation is:
I = (R + G + B) / 3 or equivalently I = R + G + B
The major benefits of IHS color space over other color spaces are compatibility
with human perception process and separability of chromatic values from achromatic
values similar to the opponent color space. Moreover, only hue space can be used for fast
segmentation and recognition of color object alone as been done by Gagliardi, Hatch and
44
Sarkar in 1985. Intensity invariant method is accomplished by employing only hue and
saturation spaces. On the other hand, IHS color space also has some important
disadvantages. First, transformation from RGB to HIS color space leads to an
unavoidable singularities (H is undefined for achromatic case and S = 0, if R = G = B =
0). Secondly, the instability of a small change near singularity points results in incorrect
transformation. Another problems with trigonometric function may produce infinity
values for hue space. Lastly, converting from RGB directly to IHS color space causes
perceptually non-uniformity. So far, existing number of ways to transform from RGB to
IHS color space have purposed, conversions are either direct transform from RGB to IHS
space or double transform from RGB to intermediate space; such as, YUV or YIQ, and
then to IHS space. Both of these transformations are known as "IHS-type" color spaces.
3.5.6: Perceptually uniform color spaces
Originally, color space with property that color differences, recognized by human
eyes, equally express by Euclidean distances was needed for simplicity of color
representation. In 1976, the CIE suggested two perceptually uniform color spaces, which
have this property, known as CIELAB and CIELUV. The CIELAB space is composed of
L*, a* and b* components and suitable for reflected light. Similarly, the CIELUV space
is composed of L*, u* and v* components and appropriate for emitted light. Both of
these color spaces are described earlier in section 3.4. Because of their property that can
be expressed equivalently by Euclidean distance, both are effective for color image
segmentation using clustering method in either rectangular coordinate or cylindrical
coordinate. In this study, CIELUV color space is used as a basis in the purposed
clustering procedure, described in chapter 4, for color image segmentation.
3.6: Summary
Each color space transformations are appropriate for different applications in
color image processing; therefore, a proper alternative of color space can lead to better
outcome than the other color spaces. Even though, practically there is no perfect color
45
space for all image- processing tasks, understanding of each color spaces' properties can
result in easier decision-making.
For the segmentation purpose, our proposed technique in chapter 4 bases upon the
uniform color space, CIELUV, since its color difference property is closely related with
human perception and it can express equivalently with Euclidean distance. In addition,
CIELUV is then transformed to IHS color space. The primary reason for selecting IHS
color space is its capability of separation of chromatic from achromatic values so that the
invariant of most of the intensity variation is achievable. Moreover, its non-linearity of
the transformation due to noise can be prevented by the filter operation, described in
section 4.3. The following table summary properties of each color space transformations
(from "The Colour Image Processing Handbook" by S.J. Sangwine and R.E.N. Horn
[29]).
Color space
Linearity of
Stability of
Perceptual
transformation
calculation
uniformity
rgb
No
No
No
XYZ
Yes
Yes
No
xyz
No
No
No
YUV
Yes
Yes
No
YIQ
Yes
Yes
No
YCbCr
Yes
Yes
No
Ohta
Yes
Yes
No
IHS
No
No
No
CIELAB
No
Yes
Yes
CIELUV
No
Yes
Yes
46
Chapter 4: - Proposed Color Image Segmentation Technique
4.1: Introduction
In this chapter, the three main components of our proposed segmentation
technique are described in depth. The first section, 4.2, explains the conventional
histogram-based segmentation method that is one of the most popular segmentation
approach because of its property that does not require either users' input or "prior"
information of the original image. In our technique, individual three histograms of each
color space, IHS, are considered separately. I, H and S are respectively represent the
intensity, hue and saturation. However, the major drawback of this technique is a
discontinuity of the histogram, if it is considered as a distribution function. So as to
overcome this difficulty, a morphological filtering and edge-preserving filtering
operations, as described in section 4.3, are applied to histogram signals and the original
image respectively. Both of these filtering operations smooth out the histogram curves
remarkably. Therefore, the histogram valley-seeking method, proposed in section 4.4, can
be employed to recover the intervals between the valley points of each histogram. These
intervals of each histogram represent the coarse segmentation. The proposed threedimensional clustering technique, discussed in section 4.5, apply this coarse segmentation
to obtain distinct clusters in a refine manner. Nonetheless, the limitations of this
technique are that a decent segmentation result can be obtained from the original image
with noise less than a certain level, the under-segmentation might occur in case that the
objects have very similar color and illumination.
Another essential issue for image segmentation is lack of uniform evaluation
criteria of the result. In the literature, most segmented results are evaluated visually; so,
this is very subjective depending upon individual's opinion to justify the results. One of
the criterions is to concentrate on one or more of desired properties such that they
compromise one preferred property against another. J. Liu and Y. Yang [19] suggested a
way to evaluate the segmentation results quantitatively as explained in section 4.5. Both
47
synthetic images with and without additional noise are used to evaluate the proposed
algorithm numerically.
Then, the experimental results of the probe-sample alignment of the AFM are
shown and discussed in section 4.6 following the noise tolerance performance testing.
Finally, the segmentation results of the natural scene images are shown and discussed as
well.
4.2: Histogram-based segmentation
In the past, most researchers have concentrated only on the segmentation of grayscale images because of limitation of computer system. Therefore, the most troublesome
of using monochrome image is the illumination variation. As the result, over-segmented
regions are often produced. Recently, computer systems become more powerful to
process the large information of the image, especially the three dimensions of the color.
In this study, color space is one of the most essential components of image segmentation.
The CIELUV color space, described in section 3.4.2, is used, since its color difference is
highly correlated with human color recognition and it can be equally expressed with the
Euclidean distances. In spite of perceptually uniformity of the CIELUV color space, the
conversion from RGB space to LUV space is a nonlinear transformation. Even if, the
characteristic of noise in RGB space is homogeneous, after this transformation the
noise's characteristic in LUV space becomes inhomogeneous. Any small quantity of
noise in RGB space may be highly amplified by this transformation depending upon
those RGB values (L. Shafarenko [31]). Thus, in the case of noisy picture, the appropriate
filter requires to smooth out individual
RGB
spaces before performing this
transformation. Then, LUV color space is transformed into IHS color space that separates
the chromatic values, hue and saturation, from the achromatic value. Properties of I, H
and S are described earlier in section 3.5.5. Besides that histograms of I, H and S
components are constructed to examine the distribution of each attribute independently in
case that histograms consider to be the discrete approximations of the probability density
function described in certain ranges. The accuracy of the results of the histogram-base
segmentation depends on how well the object of interest separate and cluster into distinct
48
ranges or intervals in that dimension of the histogram. In our implementation, the
following ranges [0-256], [0-300] and [0-200] are used respectively for intensity, hue and
saturation. We now illustrate how the histogram-base segmentation is done. First,
considering the simple synthetic image that contains only three distinct colors, red, green
and blue; therefore, there are only three distinct peaks in all histograms of hue, intensity
and saturation, as shown in Figure 4.1.
Ma - 32640
0
Hue Histogram in range 0-300
An original image
Max 32640
Max- 32640
0
256
Intensity Histogram in range 0-256
200
0.
Saturation Histogram in range 0-200
Figure 4.1: Synthesized cross image and its I, H and S histograms.
From the above example, the original image's size is 256x256 pixels and it is composed
of 18496 red pixels, 14400 green pixels and 32640 blue pixels. All three histograms have
three peaks with maximum peaks in each histogram corresponding to blue color pixels.
49
.......
. ...
......
..................
..
.......
........
. .......
For instance, considering hue histogram, if two thresholds are set at middle points
between first-second peak and second-third peak, then three labels were assigned to each
pixel according to its corresponding intervals from the hue histogram. The following
assignment of three labels: red, green and blue, are obtained as shown in Figure 4.2.
Figure 4.2: three segmented regions from threshold hue histogram.
Likewise, if the thresholds are set for intensity and saturation histogram, the same results
as Figure 4.2 are obtained. Basically, the histogram-base segmentation is a threshold
process in an one-dimensional space assuming that homogeneous regions in the image
cluster into certain ranges or intervals in histogram. The threshold values can be
determined from the valleys of histogram and each interval between two valleys is
assigned a specific label. The resulting segmentation of the image can be achieved by
mapping the threshold ranges back to the spatial domain of the image as three segmented
regions shown above. If the histogram has distinct peaks like in this example, the
segmentation with this approach is straightforward so that only one of the spaces of IHS
color space alone can segment original image into distinct regions accurately. In addition,
if the valley points that separate the two peaks can be located correctly, the prior
information would not be required for homogeneous segmented regions from original
image. Another advantage of the histogram-based segmentation is that all color
information from each pixel need to be process only once for conversions of these three
histograms; therefore, computational time of this technique is far less than other iterative
methods. Unfortunately, this histogram threshold procedure suffers from the lack of local
spatial information in the image. As shown in Figure 4.2, all red pixels are considered to
be one single object, so as to green and blue pixels. Practically, in the natural image, hue ,
50
.. - -
- --_- - - -
Z-
-, - - __
- _:__ -
- -
- - __ -- __
.........
.....
intensity and saturation histograms are very noisy, some of the peaks in histograms would
not distinctly correspond to any homogeneous region or object of interest as shown in the
following example of parrots image of Figure 4.3:
Hue histogram in the range 0-300
2500
1500
-
100
10
50
0
100
2)0
150
300
250
An original image
Intensity histogram in the range 0-256
1AM.
Saturation histogram in the range 0-200
120
200
1500
80
60
10
400
2)0
0
so
100
150
200)
250
n0
2
40
0
80
100
120
140
160
180
2M
Figure 4.3: Parrots image, representing natural scene, and its I, H and S histograms.
The hue, intensity and saturation histograms, shown above, are very noisy because they
are a discrete approximation of each dimension density function. Thus, it becomes very
difficult to decide which peaks correctly represent uniform and homogeneous regions and
which peaks come from variation in original image. Typically, the slow-vary peaks
buried within the high-frequency variation are peaks denoting uniform regions in the
image. For instance, the intensity histogram seems to have at least five outstanding peaks,
overlaid by high-frequency noise. If the thresholds are set at the valleys between these
51
.
..
....................
five noticeable peaks and labels are assigned for each interval, the segmentation result
from intensity-histogram threshold is shown in Figure 4.4. In case of hue histogram, it is
even more troublesome to separate the peaks visually.
Figure 4.4: Five segmented regions resulting from threshold intensity histogram.
In this case, a body of the middle parrot has same intensity values as middle section of
the background so that histogram threshold method labels both of these areas as a
uniform region. Therefore, segmentation into homogeneous regions is not necessarily an
appropriate solution in some circumstance, including the case above. Our valley-seeking
technique, described in section 4.4, depends on the derivative or zero-crossing property
of the signal-like histogram. This technique is used to locate both peaks and valleys of
histograms for further segmentation methods. Noisy histograms usually result in too
many peaks and valleys. So, smoother histograms are required to locate valley points
more precisely and set threshold correctly.
4.3: Filtering
In this section, morphological filtering operations and filtering technique for edge
preservation are discussed. First, the morphological filter is a method that bases upon the
operation of a certain geometric structure, known as a structuring element. In our case, a
52
predetermined geometric shape is used as filtering structure and it is suitable for discrete
one-dimensional signal, such as histogram. Moreover, the spatial filtering technique,
similar to adaptive smoothing, is used to remove noise that has particular characteristic in
the original image before any further processes.
4.3.1: Morphological filtering operations
Like other filtering techniques, morphological filters are used to strengthen
certain characteristic of input signals or images and attenuate other characteristics. The
morphological filtering operation, based on the predetermined structuring elements, does
not require any transformation, like Fourier, Hilbert transforms and etc., before applying
filtering process. However, the particular features; such as salient or high-frequency
properties, that is important or irrelevance for the desired criteria must be realized and
taken into consideration so as to apply any kinds of filters, including morphological
filters. In this case, histograms can be recognized as a one-dimensional signal; therefore,
the morphological filter operation can capture the salient feature of signal. Moreover, it
can operate in any given ranges of domain. As mention in last section, each histogram is
composed of slow-vary signal buried within the high frequency noise due to their
discretization. Therefore, the main objective of using the morphological filters is to
reduce the high-frequency noise in histograms with opening and closing operations. Both
opening and closing operations of morphology are the combination of dilation and
erosion. All morphological operations are described briefly in appendix A. In this study,
the structuring element, used for filtering histograms, is a triangle with window size of
three shown in Figure 4.5. Since the domains of I, H and S histograms in our procedure
are 256, 300 and 200, which are quite small; thus, the window size of three is the most
effective for all histograms. The values of this triangle structure are 1, 5 and 1. First, the
demonstration of the effect of the opening and closing operations alone on the hue
histogram of the parrots image are shown in Figure 4.6. Then, consider each histogram
from parrots image's example in the last section, the opening-closing filtering operations
of these signals are shown in Figure 4.7.
53
. ..............
6
4
2
0
2
11
-2
Figure 4.5: A triangle structuring element that is used in our opening-closing operation.
The effect of opening operation on the hue histogram
The effect of closing operation on the hue histogram
1500-
1000
500
50
Sao
0
50
10
150
2M
250
3)
0
50
10
150
2M
250
M
Figure 4.6: The effect of the closing and opening operation on the hue histogram on of
the parrots image.
According to Figure 4.6 above, using only the closing operation with this triangle
element is comparable to fill in the gap between two consecutive peaks of the onedimensional signal. While, using only the opening operation with the same triangle
element is similar to smooth out the one-dimensional signal with gaussian function with
large window size. Thus, applying the closing operation alone the high-frequency noise is
not eliminated. On the other hand, applying the opening operation alone the significant
peaks might be gotten rid of as well as the high-frequency noise.
As shown in three histograms in Figure 4.7, the opening-closing operation with
use of this triangle element acts like a low-pass filtering operation. All high-frequency
noise in histograms is eliminated so that the salient features or peaks become prominent.
If a larger size structuring element than three is used, small concentrated peaks will be
removed from histogram completely. However, noticeably, certain parts of histogram are
54
more stepwise, similar to discrete one-dimensional signal. This discrete effect can be
overcome by convoluting this discrete histogram with a gaussian filter. This filtering
operation yields better results than the results from a regular low-pass filter or the
gaussian filter, as shown on hue histogram of the previous example in Figure 4.8.
Hue histogram in the range 0-300
2500
25M
2M -
1500
1500
1000
500
50
301
250
200
150
100
50
0
Intensity histogram in the range 0-256
nn.
0
1400
1200
1200 -
1000
1000
8m
800 -
600
600
400
400
200
200
50
0
150
100
250
200
Saturation histogram in the range 0-200
100
150
Opening-Closing operation on intensity
0
200
histogram
250
30
0
of the parrots image
250
200
150
100
50
0
t1rr.
2500
50s
Opening-Closing operation on saturation histogram of the parrots image
1600
0
1400
1200
150
1000
100 0600
400
so
200
0
0
20
40
60
80
100
120
140
150
150
2UJ
0
20
40
0
0
1W
120
140
160
1W
2M
Figure 4.7: Opening-closing filtering effect on I, H and S histograms of the parrots
image.
55
..........
.-----.......................
.....
.....
...
......
Comparison between opening-closing filter and low-pass filter on hue histogram
2500
Blue solid line: Original histogram
2000 -
Red -. line: opening-closing histogram
Black - line: low-pass filtering histogram
1500
1000
-
&
500
00
5
00
150
V
200
250
300
Figure 4.8: Comparison between opening-closing filter and low-pass filter on histograms.
4.3.2: Edge-preserving filtering operation
A blurring/sharpening algorithm that enhances strong edges, eliminates noise and
weak edges and preserves corners and T-junctions is applied to smooth out the original
picture even before finding histograms. This algorithm follows similar concepts of the
adaptive smoothing, first used for noise cleaning in television images by Graham in late
1950s and then rediscovered by others. The adaptive smoothing uses a local edge-ness
estimation to adjust a variable weighting function for averaging out the image. Therefore,
the Graham's algorithm is described as a fundamental concept for further development.
The Graham's algorithm bases upon the concept that noise is less apparent in
certain regions that contain more details. Thus, this method blurs the image more in the
smooth regions than in the busy regions. A local measure of degree of busyness or level
of detail can be determined from directional difference of each pixel's nearest neighbors.
Consider I(x,y) as sampled gray-scale input image, the Graham's procedure approximates
the directional difference, Ixx and Iyy, by the discrete convolutions with the following 3 x
3 kernels respectively:
56
From I,
1/6
1/6
1/6
1/6
-1/3
-1/3
-1/3
1/6)
1/6
1/6
1/6
1/6
-1/3
-1/3
1/6,
1/6
-1/3
and
1/6)
and Iyy calculations, the weighting variables of each pixels, pixel(ij), are
computed and substituted into each pixels of the original image. These variables obtain
from averaging original value of each pixel and certain neighbor pixels depending on
values of Ixx and Iyy. In Graham's algorithm, neighbors of pixel(ij) include only eight
directional pixels: (i+1,j), (i+1+j+1), (ij+1), (i-1,j+1), (i-1,j), (i-1,j-1), (ij-1) and (ij+1).
The formula for weighting variables are given below:
( pixel(ij) + eight neighbor pixels ) / 9
if Lxx < T and Iyy < T
( pixel(ij) + (i+1,j) + (i-1,j) ) / 3
if IXX < T and Iyy >= T
(pixel(ij) + (ij+1) + (ij-1) ) / 3
if Ixx>= T and Iyy < T
pixel(ij)
if lxx >= T and Iyy >= T
Where T is a selected threshold, in the Graham's algorithm, T was set in the range
between 0.1 and 0.5 percent of intensity value. With this range of threshold, the original
image is generally blur about 25 percent in both x and y direction. The low value of
threshold results in blur image in all direction, while the high value of threshold results in
the original image itself. However, this filtering algorithm blurs across both prominent
edges and corner. This special characteristic that blurs in direction with less detail leads
to the development of the adaptive smoothing or edge-preserving filter by M. Nitzberg,
D. Mumford and T. Shiota [22], discussed in this section.
First, let consider the intensity of the original image, I(x), in two-dimensional
intensity space and x denotes (x,y) coordinate. Define the edge to represent the boundary
that separates two relatively homogeneous regions of the image. This edge is determined
from value of the gradient of I(x). So, the adaptive smoothing filter replaces each pixels,
xO, of the original image, I(xo) with I0ut(xo), this Iot(xo) is obtained by applying a variable
Gaussian kernel that is shaped, scaled and displaced according to the gradient of I
computed from neighbor pixel of xO. The formula for Iout(xo) given as the following:
iot(xo ) =
1
N(xo)
Jk(xo, x)I(x)dx
=
57
where k(xo,x) denotes a variable Gaussian kernel and N(xo) is a normalizing factor. This
normalizing factor defines so that the maximum and minimum values of IOt(xo) is inside
the range of maximum and minimum of I(xo). The Neumann type boundary condition is
used to define the boundary of the original image so that the boundary of the image is
extended to from original region, A, to larger region, A' by reflection of images'
boundary. Thus, above formula can apply to every pixel in original image without
modifying the formula for boundary pixels of the original image.
The variable kernel for the point xo can be expressed as the following:
k(xo,x) =
N(xo)
1
A(x- xO +a(xo))eQxO(Xxo+a*xo))
N(xo)
where Qxo represents a quadratic form, and cc is a vector that gives the negative of the
displacement of the kernel from xO. If the origin is at xo-a(xo), then the kernel can be
written as:
2
-(Ex 2 +2Fxy+Gy )/.2
1
N(xo)
where E, F and G values are calculated once for each xO, their expressions are described
in the following discussion. The p, is a cut-off function centered at the origin and it limits
the effective size of the kernel, called the kernel size. All the kernel sizes, in this
implementation, have property of rotational symmetric and have unit volume. The a is a
scaling variable of the Gaussian kernel.
The kernel shape is represented in the quadratic form as the following:
1E
Q O~2
(F
F
G)
where E, F and G are gradient components of I in a neighborhood of U of xO against a
weighting function, P2. The diameter of the neighborhood U, called the window size, sets
a length scale of features in covering area of filter. The P2 is also a gaussian function that
gives more weight to the gradient near xO. Thus, E, F and G can be calculated as the
following:
E = JI(x)2 P 2 (X - XO)dx
U
58
F
=
JI(x)I,(x)p 2 (x - xo)dx
U
G= JI(x)2p 2(x- x0 )dx
U
This quadratic form produces three type of kernel. The first type of kernel has a
broad shape so that it can smooth uniform regions in case that the gradient is small
throughout U. Another kind of kernel must be narrow to smooth along edges and not
across edges in the dominant gradient direction in case that the gradient is large in the
neighborhood U. Lastly, the third type of kernel must be small and concentrated to
maintain corners or triple points, if the gradient is large in many directions throughout U.
The shapes of kernels for different characteristic gradients, as described above, are shown
in Figure 4.9.
The first type
of kernel
The second type
of kernel
The third type
of kernel
Figure 4.9: The shapes of three kernels producing from the quadratic form corresponding
to different characteristic of the gradients in the neighborhood of U. (from M. Nitzberg,
D. Mumford and T. Shiota [22])
The basic concept of using quadratic form for kernel shape can be justified as following
example. Consider a positive semi-definite quadratic form:
5
Ixi
and then integrate this quadratic form of neighborhood U for every points of original
image. If there is a strong gradient or prominent edge in one direction in the window size
U, above quadratic form yields an ellipse, which is wide in the edge direction and narrow
in the gradient or across-edge direction. If gradients are equally strong or weak in all
59
directions in the window size U, the above quadratic form yields a concentrated circle or
wide shallow circle respectively.
Moreover, the variable kernel k(xo,x) enhances edges by pushing the kernel away
from nearby edge. Therefore, the kernel directs the image values of neighborhood regions
toward the edge. This effect is represented in variable kernel as a vector -a(x) at each
point. For each xo e A, the vector a(xo), given in the formula below, is calculated as a
function of the weighted average vector, V, of gradient of I(xO) over neighborhood U of
xO and this weighted average is scaled by its projection onto the radial direction from xO.
a(xO) = #(V)
(VI(xO + y)- y)VI(xO + y)p 3(y)dy
V =
where p3 is cut-off function similar to pi and P2, and < is a smooth, vector-valued
function that compresses vector V to limit displacement.
#(V)=cV/
p2+
In this implement,
V
and ci is about quarter of kernel size and p is a positive constant, equal to 10 in our
implementation. Let consider the effect of above weighting average vector, V always
point away from xo because of the VI(xo+y) term; therefore, at the corner point,
combining values from both side of comer point make V pointing outward from the
corner. The corners and junction points are enhanced more with this weighted average
vector, V.
This filter is very useful for our desired application, since the images taken from
the AFM with ccd camera are very noisy. Sample image from ccd camera is shown in
Figure 4.10 and 4.11 on the right. Noticeable from these two pictures, Noise from AFM
image is chromatic noise. So as to get rid of this chromatic noise, we apply above
adaptive filter to individual R, G and B color spaces with both kernel size and the
window sizes of 13 pixels. In the other word, both regions A and U cover the range of 13
pixels. These specific kernel and window sizes are tested to be appropriate for AFM
images as shown in next section. Then, the transformation from RGB to CIELUV color
spaces uses these smoothed R, G and B spaces as inputs. Combining of R, G and B color
spaces results in blur images, as shown in Figure 4.10 and 4.11 on the left.
60
Figure 4.10: First example on comparison between original image from AFM (on the left)
and image after filter with adaptive filtering technique (on the right).
Figure 4.11: Second example on comparison between original image from AFM (on the
left) and image after filter with adaptive filtering technique (on the right).
As seen from the pictures above, the images after processing with this adaptive
filter display smoothness in uniform regions and output images do not blur across the
edge or at the corner and junction points as explain in the filtering operation above.
4.3.2.1: Kernel and window sizes selection for AFM images
Using of the adaptive filter technique explained in last section, the appropriate
kernel size, pi, and window size, P2, must be selected for fast computational time and
61
noise elimination purpose. In addition, we must realize that both kernel and window sizes
are highly correlated to values of -of pi and P2, representing the gaussian bumps. Since,
the larger the kernel and window sizes, as variable gaussian function in this case, are, the
longer the computational time is. Therefore, it would be suitable to choose an appropriate
size so that it can be done in real time processing. The appropriate in our case means that
the filter must eliminate noise as much as possible and its size also should be smallest
too. Another consideration is noise elimination. These kernel and window sizes denote
neighborhoods of region A and U in the formula discussed in last section respectively. If
small kernel and window sizes are selected, there might be some high-frequency noise
left in the filtering image. Because the filtering algorithm might cover a small area so that
may cause noise at some parts of the edge or corner point to be preserved. If large kernel
and window sizes are selected, all high-frequency noise is eliminated. But, the filtering
algorithm will spend a lot of time trying to cover the same regions that do not possess
high-frequency noise. In our implementation, both kernel and window sizes of 13 are the
most effective and filtered images from AFM are shown in Figure 4.10 and 4.11 in
previous section. Moreover, all histograms of I, H and S spaces, described in section 4.2,
also become smoother due to this adaptive filtering affect. These histograms of AFM
image in Figure 4.12 are shown below:
4.4: Histogram valley-seeking technique
The main purpose of this histogram valley-seeking technique is to divide all
pixels of the original image into certain groups according to the valley of histogram or
density function of each color space. In this case, certain groups in the segmentation
mean uniform and homogeneous regions in the original image. The intervals between
each valleys of histogram can determine the number of clusters in image; however, this
procedure produces coarsely segmented clusters that will be used as basis for refining
with the proposed three-dimensional clustering, described in next section. The major
advantage of these coarse segmented clusters is significantly time-reduction in threedimension clustering comparing to the other existing clustering methods that use
individual pixel property for clustering in three dimensions.
62
Max = 3836
Max 4316
g
H
300
l300
Hue histogram of filtering image
Hue histogram of original image
Max='
Max- 11 6
022
0
256
25
Intensity histogram of filtering image
Intensity histogram of original image
Max=1355
Max = 1147
200
200
Saturation histogram of original image
Saturation histogram of filtering image
Figure 4.12: Adaptive filtering effect on I, H and S histograms of AFM image
Our histogram valley-seeking technique is comparable to a zero-crossing method,
which relies on finding the first derivative of the signal. Initially, the one-dimensional
gaussian bump of predetermined size is constructed along with its derivative. In this
implementation, discrete gaussian function with sigma value of 1 and kernel size of 9 and
63
its derivative are used because the reason similar to the small size of structuring element
using in the morphological filter to avoid removing small concentrated peaks completely.
First, we convolve each histogram after applying the opening-closing operation with the
created gaussian function, this smoothing histogram is then convoluted with the
derivative of gaussian bump. The basic idea of this process is to locate where the
resulting histograms after convoluting with the derivative of gaussian function cross a
zero-horizontal line. This zero crossing points represent either the maximum or minimum
points in each histogram. To determine whether these zero crossing points is maximum or
minimum point, its nearest neighborhood points on both sides must be taken into
consideration. If there is a change from positive to negative value either from one to its
next point on the right or from left to right of the middle point, this sampled point is a
maximum point. On the other hand, if there is a change from negative to positive value
either from one to its next point on the right or from left to right of the middle point, this
sampled point is a minimum point. Because each histogram is discrete signal, a threshold
value is required to distinguish between small change due to noise or large change from
the outstanding peaks for zero-crossing points. In our implementation, this threshold is
set at 1.0. For example, if the difference between one to its next point is greater than 1.0,
then this point is the interested zero-crossing point that need to be considered. Moreover,
we can obtain the coarse segmented regions of each color space from the range between
valleys of its histogram as mention in section 4.2. Let apply this valley-seeking technique
to the histograms of AFM image after filter with adaptive filter in last example of section
4.3. The blue circles and red crosses denote maximum and minimum points respectively
or both are the zero-crossing points, as shown in Figure 4.13.
However, there is a minor problem using morphological filter operation on
histogram, if there is a peak either at the first or last point in the histogram before
applying the opening-closing operation. If these two peaks exist in histogram, the
opening operation will completely eliminate these peaks. Our histogram valley-seeking
technique can solve this drawback by detecting occurrence of peaks at the first and last
points in the original as well as other peaks from the opening-closing histograms. In case,
there is a peak at the first point of histogram, then add one valley point between this first
point in the original histogram and the first peak obtained from the opening-closing
64
. .........
.....................
histogram. And similar method is applied to identify the peak at the last point in original
histogram. Let consider another example of synthetic image of 6 color characters on
white background shown in Figure 4.14 along with its I, H and S histograms. Notice that
there are peaks at the first point in both hue and saturation histogram, while there is a
peak at the last point in the intensity histogram. Then, this histogram valley-seeking
technique is used to locate the maximum and minimum points on the same histograms
after the opening-closing operation, as shown in Figure 4.15. Additional valley points are
added in hue and saturation histogram for detecting peaks at the first point on both of
their original histograms. Also, another valley point is added in intensity histogram for
detecting peak at the last point of its original histogram.
Zero crossing of Intensity Histogram
Zero crossing of histogram of Hue Histogram
Red solid line: opening-closing histogram
Blue dash line: first derivative of opening-closing histogram
30M
an
-
Red solid line: opening-closing histogram
700
-
Blue dash line: first derivative of opening-closing histogram
600
200
Ism
400
10C
200
5W
>
L1
100
j- i
0-
0A
00
so
200
150
100
250
300
50
100
150
200
250
300
Zero crossing of Saturation Histogram
Red solid line: opening-closing histogram
Blue dash line: first derivative of opening-
1200 -
closing histogram
1000
1
1
j
600
0
J
0
50
100
150
200
250
Figure 4.13: Maximum and minimum points located on each I, H and S histograms after
the opening-closing operation, represented by red lines, using valley-seeking technique
discussed above, while blue lines in each histogram represent resulting signals after
convolving each histogram with the derivative of gaussian.
65
..........
- .....
. . . ..........
..
. ............................
.....
..
..........
x1(i
A
Hue histogram in range 0-300
........... I------------- ----------
----------- -----------------------
4 ------------L ------------I ----------------------- I -----------
3 ------------------------ ----------- --------
----------- ---------
2 ----------------------------------- --------- .........................
----------- ----------- ----------- ----------- ----------- -----------
u0
10
so
100
Intensity histogram in range 0-255
x
160
200
300
250
Saturation histogram in range 0-200
x
S
......................................... ------------- - --------------
4 ------ ....... ------ .............. 4 ------ --------------
4 ------------- ------------- --------------------------------------------
3 --------------------------------------------------------
--------------
2 ........................... .........................
.............. L
---- ------ ------
..................... L ...... ..............I ...... ...... ....... ......
------------- ------------- ------------- -------------- --------------
-0
50
100
150
2M
------------- ------ ------ ------- ------
2M
-0
20
40
60
80
100
120
140
160
180
MO
Figure 4.14: Synthetic image of 6 color characters and its 1, H and S histograms.
Zero crossing of intensity Histogram
Zero crossing of histogram of Hue Histogram
35(
3SOO
.......... ........... ........... ........... ...........
3000 -----------
....................... ---------
30(
---------------- - ------------
2000 ----------L -----------L ----------- I ---------I --------- I ...........
20(
...........
. ......... --------- ...........
15(
----------- ---------------
2500
1000
-----
500 ---- -----
.........
-- ------ - --------- ---------
------ ......
.......
--------
.............k ...... ----------
....... .....
-10000
60
100
150
200
)0
300
----------------------
.........
. -----------
-----
---------
-
0
-----------
-I-----------
..
.................
----------...............
i'A
---------------------------
...........
250
------- -----------
----------------- -------- ---------
10(
01
.500 .....
------ ---------
- --------------------------------------------
25
100
.....
.........
-----------
........
...........
150
200
250
300
66
.....
............
...
......
.
...
. .......
..
...
. .... ........
...
. .............
......
..
. ......
Zero crossing of Saturation Histogram
3502
3W
gur
--4.---
-sy5teti
I,
H ---
hist-grams
50
10
as
this-----aysekn
Iitga
haa.ters..Using
f .
mage-. .......
an
af-er
160
2W0
te-hnique-peak
250
Figure 4.15: 1, H and S histograms after applying opening-closing operation of the
synthetic image of 6 color characters. Using this histogram valley-seeking technique peak
and valley points of each histogram can be located as shown above.
4.5: Three-dimensional color-space clustering
As mentioned in chapter 2, the clustering technique is a method to classify the
samples into distinct classes. Most of these kinds of classification do not require prior
information or training set, so they are known as clustering or unsupervised classification.
In our implementation, samples for classification are the pixels in the original image. In
the clustering, the resulting classes of samples are required to possess certain properties
within the same class. The criterion for selecting these properties is the most important
and fundamental issue to distinguish between suitable or unsuitable classification of the
clustering method. For our proposed approach, the clustering criterion is to group the
connecting nodes according to their normalized variance in this three dimensional color
space, IHS.
The main objective of our proposed clustering method is to be able to perform
clustering of sampled data in three dimensions within a short period of time and without
any iteration process so that each distinct class can be identified. First, we project the
resulting valley points of each color spaces, I, H and S, into three dimensions to construct
a connecting-box structure. These connecting boxes can be considered as connecting
nodes, as shown in Figure 4.17. Then, the sampled data is assigned into each connected
67
. ....
. ....
..............
. .............................
.... ........
..
......
box owing to their color properties: I, H and S. Let consider a simple example of
synthesis image of 6 color characters of last section. There are 7, 6 and 5 intervals for
hue, intensity and saturation spaces, shown in Figure 4.15. Thus, there are total of 210
connecting boxes, as shown in Figure 4.16. The seven red dots represent 6 color
characters and white background in the original image. Since there is not any variation in
these seven synthetic-color regions, each color then gathers into a single dot.
IHS Color Spaces
120,
100,
--.
80,
15-0I-~
-
10.--
2
I
-
60,
40
200
4
50
Q
-2
0
-4
H
Figure 4.16: Each sampled points of the 6 color characters image are plotted on threedimensional color spaces, I, H and S and blue lines that construct the connecting boxes
are the corresponding valleys in I, H and S histograms.
Notice that the unit of hue space varies from -n to n, however, hue histogram is in the
range from 0 to 300. Originally, hue space is described in polar coordinate as mentioned
in section 3.5.5, thus, this connecting-box structure has wrap around property on hue axis.
The boxes or nodes that are on the left-face with hue value equal to -n are connected with
the boxes or nodes that are on the right-face with hue value equal to n. And unit of I and
S are range from 0 to 256 and 0 to 200 respectively. Each pixel in the image locates
either inside or at the faces of these connecting boxes. For the synthetic image of 6 color
68
..
. ......
.....................
. ......
characters, only seven boxes out of 210 boxes contain the sampled data or pixels of the
image, while the other boxes are vacant. Thus, it is very clear to separate the original
image into seven distinct classes corresponding to these seven boxes. These seven classes
are red T character, green A character, yellow C character, dark blue 0 character, light
blue E character, purple I characters and white background. Or even without these
connecting nodes, six distinct classes can be specified by each dot. However, in case that
there is a lot variation in the image, sampled data in the IHS color space scatter and do
not assemble into distinct groups or classes as shown in previous example of 6 color
character image. Each box or node is connected to its 26 neighborhood nodes as shown in
Figure 4.17. Each circle represents the box or node of the connecting-box structure. The
red circle, labeled with number 27, denotes the interested box, while black and blue
circles are its neighborhood nodes in the same and different levels of intensity
respectively. The labels from 1 to 7 are assigned to nodes in the same intensity level as
node 27, while labels from 9 to 17 are assigned to nodes in one lower level of intensity
than node 27. Also, one higher level of intensity than node 27 is labeled with number
range from 18 to 26.
25
248
26
22
1
2
68
19
21
14
4
16
2
133
12
10
11
Figure 4.17: Connecting-box structure representing neighborhood nodes.
When the current interested node is at the boundary of connecting-box structure, only
certain neighbor nodes have to take into consideration. For instance, if the current
69
......
.
. ......
. ...................
.........
interested node is at the highest intensity level, only neighbor nodes with labels 1 to 17
are adjoined to current node.
Let consider another example, if additional chromatic noise with gaussian
distribution characteristic is added to the original synthetic image of 6 color characters.
This noise covers 25 percent of pixels in original image or 25 percent of pixels in the
original image is distorted by the noise. Applying our histogram valley-seeking technique
to each histogram, 5, 12 and 4 intervals are obtained from I, H and S histograms
respectively, as shown in Figure 4.18 below. Then the connecting boxes and each pixel
are plot on this IHS space as shown in Figure 4.19.
Zero crossing of histogram
1i0r
-
-
-
__
of Hue Histogram
_
__
_
8am ----------- ----------- ----------- ----------- ---- ----- -----------
6WO ----------- -----------
-----------
-----------
4W0............ .............. .......................... .................
0
1(O
50
0
3O
25)
2W
Hue histogram in range 0-300
Cross image with 25 percent additional noise
Zero
Zero crossing of Intensity Histogram
crossing of Saturation Histogram
4000
. ......... .............. ............. .............. .............
35002
. ....................................................
-------------
-------------- ------------- ............. ..............
3000
-------------- ------------- ------------- ..............
2500
41000
1000
150
.--- - -- -- - -- . .. . .. -- - -- - -- - -- - -- -- -------------------------
1500
............. - -------------
------------- ........................... --------------------------- ------------- 4 -------------- --------------
100
.................. -
. I
0
50
100
10
2O
250
Intensity histogram in range 0-256
3W
__
. .
60o
------ --------------
1, -1
150
200
250
Saturation histogram in range 0-200
Figure 4.18: Synthetic 6 color characters image with additional chromatic noise adding to
25 percent of its original area and its I, H and S histograms.
70
- - - -
.......
...
.
....
......
.
.
IHS Color Spaces
IHS Color Spaces
120
-
120
-- -
-.- .
50
-50
-
40-
50
....
...
w100 ..... .....- .
200
...
-
.
60 .......
100
150
4
S
3
0
1
H
2
-4
-3
-2
1
0
4
-2
-3
0
1
3
2
1
4
H
Top view on H and S plane
110 ---
------
IHS Color Spaces
---
...---------
ih arcoor
tpaces
a
----.
100 ... .
100 ...---.
90 . ---
70 ....--
...------.-.--
......
---60 ...---
. ---- . .40 ....-
20,
-4
-3
..-. .. - -
-2
-
-- -- ---
*
....
50 ----.
-1
0
H
-.
1
..-..-.--.-.-.
-
2
Front view on H and I plane
3
4
20
2MO
150
150
140
120
100
S
80
50
40
20
0
Side view on I and S plane
Figure 4.19: Each sampled points of 6 color characters image with 25 percent additional
noise are plotted on three dimensional color space along with its connecting box
structure.
Notice in hue histogram that the additional chromatic noise has a gaussian
distribution characteristic, since all six prominent peaks have shapes similar to normal
distribution function. While, the first peak is eliminated by the opening-closing operation.
From three-dimensional spaces, all seven colors, including white background color, have
a lot of variations, especially a cluster located at the base of the H-S plane. However,
these seven clusters, corresponding to seven dots in Figure 4.16, are still simple enough
to identify by human. Using connecting-box structure, the clustering criteria must be
established to group each connecting box into classes with certain desired property.
The proposed clustering criterion is to identify each connecting node either to be a
root or branch node of a tree formation in three-dimensional IHS spaces. For simplicity,
71
.... . ........
...............
__
__ ..........
. ......
let consider one of the intensity levels of the connecting-box structure with 6 intervals in
both H and S spaces, the red and white circles in the grid of the H-S plane represent roots
and branches of tree-like structure respectively as shown in Figure 4.20 below. The
boundary of two classes is drawn in a darker line separating these two clusters. So, in
three dimensional space, nodes in both top and bottom level might direct to these two
minimum as well.
Figure 4.20: Example of proposed clustering method of two distinct classes in one of the
levels of the H-S plane.
To form the tree-like structure for clustering method, we need an algorithm to
determine whether each node is the root node or the predecessor node in the connectingbox structure. So, the predecessor node becomes an initial or intermediate node and leads
into a root node along particular series of branch, representing by series of arrows in the
above example. Each branch points from one node to its predecessor until reaching the
root node. First, the variances of each node, Ni, are computed from the following
equation:
Var(N,)=
I
N(N,)
N(N,)
Var(RRu (N, ))+Var(R ,(N, )) +Var( Rs,(N 1 ))
QVlN)'
Vol(N,)
Where N(Ni) represents numbers of sampled points that fall inside or on the face of the
connecting box, Ni, and Vol(Ni) denotes the volume of the connecting box, Ni. And
R*(Ni) is the range of each color space of node, Ni, taken from the histogram. Using these
72
4%
normalized variance values to construct series of branch as paths in a steepest gradient
descent sense forms the tree-like structure from the connecting nodes. By following the
series of branch from each node, the set of paths that lead to each node to its root node
according to the local minimum of normalized variance can be identified. Therefore, the
main goal is to find the branch directions for each node by determining the most steepest
direction of the minimum of the normalized variance among its 26 neighborhood nodes.
Then, these disjoint groups of nodes can separate into the distinct classes along their
boundaries. However, the problem associates with finding the branch directions with this
method might occur. If two nodes, N, and N2 , that belong to different classes are the
neighbors of one another and the normalized variance of the first node, N1 , is the smallest
among 26 neighbors of the second node, N2. Therefore, the measurement of number of
the sampled points in IHS spaces in the specific volume between any two nodes must be
taken into consideration. In case of above two nodes, N1 and N2 , that came from different
classes, if the specific volume between these two nodes does not contain any sampled
points. The branch direction of the second node, N2 , will point to the second smallest
normalized variance of its neighbors, N3 , instead of N 1. Furthermore, if the measurement
of the sampled point within this specific volume between node N 2 and N3 also empty, the
third smallest variance of neighbors of N2 will be considered and so on. In case that there
is not any sampled points in the specific volume between the initial node and its 26
neighborhood nodes. This initial node is then identified to be a new distinct class. The
specific volumes between the initial node and its 26 neighborhood nodes are shown in
Figure 4.21.
From Figure 4.21, the thickness of the boxes forming the specific volume that
measure the number of the sampled point between the initial node and its neighbors is set
to be 5/6 of the length of one face of connecting box corresponding to those neighbors'
directions. The forming process of this tree-like structure that classifying each node into
distinct classes begins with identifying all minimum nodes from all nodes in the threedimensional color spaces as the root node. For all remaining nodes in IHS spaces, the
gradient value of the normalized variance of each node is used to construct the series of
branch in such a way that it reaches one of the pre-identify minimum nodes. Then, these
series of branch are specified in the clustering output space. Once all nodes in IHS space
73
are associated with their minimum or root nodes, then each distinct class in color spaces
is assigned with unique label for simplicity of identified each class in later process.
Volumes between initial
node and nodes 4 and 8
Volumes between initial node
and nodes 9, 13, 18 and 22
Volumes between initial
node and nodes 21 and 25
Volumes betw een initial
node and nodces 2 and 6
Volumes between initial node
and nodes 11, 15, 20 and 24
Volumes between initial
node and nodes 12 and 16
Volumes bet ween initial
node and no(des 17 and 26
Volumes between initial Volumes between initial
node and nodes 3 and 7 node and nodes 1 and 5
Volumes between initial
node and nodes 19 and 23
Volumes between initial
iode and nodes 10 and 14
Figure 4.21: The volumes between the initial node and its 26 neighborhood nodes for
determining its branch direction.
The advantages of this proposed clustering algorithm are that this process is not
iterative, thus, clustering technique can be done very quickly. Also, this method does not
require a number of classes assignment or a predetermine number of classes. Let consider
the clustering result of the synthetic image of 6 color characters in Figure 4.19 using this
proposed clustering technique. Total of 11 distinct classes are obtained as shown in both
views of Figure 4.22.
74
. .. .. .............................
Cluster 1
Cluster 3 Cluster 4
Cluster 2
IHS Color Spaces
...... ......
110
- - --
----------
*
. .......
... . .
70,
1>
*5
.
.
------ - -- - -2-0 -.-
7
+: +
4
>+
4..
-
...
.
60
2
40
31-
20
0
+ --
---
In
1
...........
4
S
2
3
Cluster 5
0
1
H
1
Cluster 6
-2
-3
Cluster 7
4
*...............
A.~..........
~
First vie,
IHS Color
Cluswr1
.......
Spaces Cluster 2 Cluster 3
'~ ;
~
*~11
.
*
0
Cluster 4
00
0 0
0
5
122
144Hug
100-
120 -
40
12
14
6
4
le
-
400
2/
Ckuster 5
0
-2
-2
s
.4
Cluater 6
2
S0
S
Second view
Figure 4.22: From the proposed clustering method, the 11 distinct classes represented by
each cluster of same symbols are the segmentation result of the 6 color characters image.
From Figure 4.22, the two views of clustering result in IHS spaces, red squares,
black stars, blue multiple signs, black dots, blue x signs and red circles represent cluster
1, 2, 4, 5, 6 and 7 respectively. However, the cluster 2 is over-segmented into blue star,
magenta right-triangles and + signs, red upside-down triangles and black diamonds, these
over-segmented clusters result from the 25 percent additional noise. The result of
segmentation of this example is given below in Figure 4.23. Each gray-scale color
corresponds to one of the clusters in HIS spaces. From this example, the main drawback
of this proposed clustering technique is over-segmentation in case that there is a lot of
noise in the original image. This problem is caused by the huge scattering of pixels' color
property in the three-dimensional IHS space.
75
.
..
.
......
.
............
Original image
.....
.......
......
Segmentation result
Figure 4.23: Segmentation result with 11 gray-scale colors corresponding to 11 clusters
obtained from our proposed clustering technique.
4.6: Experimental results, discussion and evaluation
The proposed image segmentation technique is tested on both synthetic images
and real images of size 256 x 256. Synthetic images of cross and 6 color characters given
as examples in last three sections are used to evaluate the noise-tolerance performance of
our proposed algorithm. The noise-tolerance performance is tested with only synthetic
image because the correct segmented regions can be obtained precisely and the condition
of noise can be control. While in the real image, correct segmented regions are very
difficult or impossible to know true corresponding regions of each pixel owing to the
variation and noise from both equipment and environment. Furthermore, the segmented
results from real images, especially images obtained from our AFM machine, are only
concentrate on detecting and locating the desired object correctly.
The softwares developed in this implementation are written in both C++
programming language and Matlab. The results, shown in this research, from both C++
language and Matlab are equivalent. The reasons that implementation is done in both
languages are that Matlab is more powerful for graphical display and its results are easy
for visual comprehension. On the other hand, the C++ language is employed for real-time
76
. ...
_ _ -...
. .........
implementation owing to its fast computation. The experiments are performed using a
Pentiumll-400 MHz computer.
4.6.1: Evaluation measurement function
Usually, the evaluations of image segmentation results are done visually and there
is not any uniform criterion to justify the results numerically. Nonetheless, the basic
criteria, given by R. M. Haralick and L. G. Shapiro [14] for justifying the segmentation
results, are 1) uniform or homogeneous, 2) simple and does not contain too many holes,
3) distinctly different from each other and 4) the boundary of segmented regions should
be smooth and located at their edges. According to J. Liu and Y. H. Yang's experiment
[19], they proposed a novel evaluation function following Haralick and Shapiro's
guidelines. Their evaluation function, given below, determines the accuracy of
segmentation both locally and globally.
R
e2
F(I)=Vk.Z e'
F(I
=
-vr
where I is the original image to be segmented, R is the number of segmented regions, Ai
is the area of the segmented region, i, and ei is the color error of region i. Or ei is defined
as the sum of the Euclidean distance of the color vectors between the original image and
the segmented image of each pixel in each segmented region. The -1K and e, values are
the global and local measurement respectively. If there are too many segmented regions,
global measurements will cause the evaluation function to be large. Similarly, if there are
too many errors in assigning segmented region, the local measurement will lead to a large
value of F. Therefore, the smaller the value of F is, the better the segmentation regions
are. In our implementation, the noise-tolerance performance of the proposed algorithm
will be determined based upon similar evaluation measurement of the evaluation
function, proposed by Liu and Yang. We pay more attention on correctly locating the
segmented regions of interest rather than obtaining homogeneous regions; thus, our
evaluation function is composed of three main measurements and defined as the
following:
77
F'FRI=
RC
R
RC - M + 1
.
__
Ainside
i=1 Ainside (i)
RC
Aou tside
(i)
Aoutside (
where M denotes the numbers of correct class that is clustered into the same group as
another correct class or classes. Rc represents the number of true or correct classes in the
original image. The leAinside(i) and IeAoutside(i) are the mislabeling error of each region, i,
that are inside and outside of each true distinct region. In our segmentation results, each
segmented region corresponds to a cluster or class in the IHS space. Ainside(i) is the area
or pixels inside each true segmented region, while Aoutside(i) is the total area of image
subtracted by the area inside that true segmented region. Both
"i'i' and
Ainside
""'lsd
Aoutside
are
local measurements of mislabeling area of segmented regions. The evaluation functions,
F and F', include the basic criteria of segmentation except for smooth boundary
condition. The terms
eAinside(i), eAoutside(i)
and ei directly evaluate the simplicity of each
region, satisfied the second condition of Haralick and Shapiro, as well as they indirectly
measure how uniform and distinct each region is. And the first measurement,F,
indirectly determines how difference each region should be. In our implementation, the
RC
second measurement,
RC-
, denotes the accuracy of algorithm to classify the coarse
M +1
segmentation so that each cluster matches with the correct classes in the image. The last
measurement,
-
Aoutside
, corresponds to the mislabeling error of
each segmented region. The percent of mislabeling error is defined to be a product of
mislabeling ratios of regions inside and outside of the true regions of interest. And the
mislabeling ratio is defined as a ratio of the number of pixels that are assigned the wrong
class labels to the total number of pixels in the original image. The value of our
evaluation function, F', varies for different image; therefore, the normalization of F' with
the worst or maximum value of F' is required for comparison of different images. Both
the synthetic cross image, in Figure 4.1, and the 6 color characters image, in Figure 4.14,
are used for testing the noise-tolerance performance of our proposed algorithm. Figure
78
4.24 shows segmentation results of the cross image and the 6 color characters image with
increasing percent of additional chromatic noise from 0 to 75 percent. This additional
chromatic noise has the same characteristic as the noise, mentioned in the last example.
However, additional chromatic noise range from 0 to 100 percents is used for noisetolerance testing. Only the boundaries of each segmented region are shown in synthetic
images with additional chromatic noise, since it enhances the distinct edge between each
small-segmented region. Showing only the boundaries is better than displaying the
segmentation result in the gray-scale formation. Table 4.1 displays the maximum (worst)
and the minimum (best) values of each measurement in the evaluation function, F' of
both the cross and the 6 color characters images.
Measurement terms of the
evaluation function, F'
Maximum
or worst
value of
cross image
2562
Minimum or
best value
of cross
image
Maximum
or worst
value of 6
characters
Minimum or
best value
of 6
characters
1
2562
1
37
R
=
147.8017
3
RC
RC -M+1
R1
Ainside(i)
=
3
96.7589
7
4
Aoutside
7
8
)
Combination of 3 terms: F'
443.4050
0
677.3123
0
Table 4.1: The range of each measurement term in the evaluation function, F', of both the
cross and the 6 color characters images.
From the above Table, the reason that the value of the evaluation function varies from
different images is the variation of the number of true regions in each image. The larger
the number of distinct regions in the image is, the greater the value of the evaluation
function, F', will be. Therefore, the value of F' must be normalized by 443.4050 and
677.3123 for the cross and the 6 color characters image respectively so that the F' is
comparable between these two images. The percent mislabeling error from the third
measurement term and the normalized F' of both images are plotted versus the percent
additional noise as shown in Figure 4.25.
79
. ......
.......
................................
.......................................
Cross image
without noise
Segmentation result
with 3 regions
Cross image with 25
percent additional noise
Segmentation result
with 9 regions
Cross image with 50
percent additional noise
Segmentation result
with 9 regions
Cross image with 75
percent additional noise
Segmentation result
with 12 regions
6 color characters image
without noise
Segmentation result
with 7 regions
6 color characters image with
50 percent additional noise
6 color characters image with Segmentation result
with 11 regions
25 percent additional noise
Segmentation result 6 color characters image with Segmentation result
with 9 regions
with 8 regions
50 percent additional noise
Figure 4.24: Segmentation results of the synthetic cross image and the 6 color characters
image with increasing percent of additional noise.
80
.u
X103
Noise-tolerance performance determining by mislabeling error
45 -.
8
1.6
1.4
3 -
re 1.2 -
10
20
30
7
E/
0.4
-
10
0
Noise-tolerance performance determining by evaluation function, F
-
40 .
5
... ......
.........
..... . .......
-
<35 -~
20.
. .
-
/
/
70
60
50
40
percent of additional noise
0.2
0
901 10
0
10
20
3
70
0
50
40
percent of additional noise
80
90
10
Figure 4.25: Effect of additional chromatic noise determining by both the percent
mislabeling error and the normalized evaluation function, F', of the synthetic cross image
and the 6 color characters image. The red dash lines and the blue solid lines are the
results of the cross and the 6 color characters images respectively.
From the above graphs, the greater percent of additional chromatic noise is added
into the original picture is, the larger the percent of mislabeling error and the normalized
value of the evaluation function, F', becomes. This percent of mislabeling error is the
average percent error of each segmented region that correctly matched with its true
regions. From the results of the cross image shown in Figure 4.27, 4.29, 4.31, 4.33 and
4.35, the percent mislabeling error and the normalized evaluation function, F', in the
cross image with 25 and 50 percent additional noise are increase. This occurs because of
the expansion of each cluster caused by the scattering noise in the IHS space. In case of
75 percent additional noise in the image, the segmented regions that match with red and
blue regions are the same large regions. This merging effect comes from the fact that the
clusters of the red and blue regions merge into a large cluster due to the large variance or
distribution of chromatic noise in the IHS space. Consequently, the percent mislabeling
error and the normalized evaluation function, F', are respectively smaller and larger than
it should be as shown in Figure 4.25. The reason that the normalized evaluation function,
F', is larger because the second term of F',
RC
has higher penalty value owing to
Rc- M+1
the merging effect. Similarly, in case of the cross image with 100 percent additional
81
noise, there is not any merging effect of the segmented regions, matched with red, green,
and blue regions. Moreover, the 100 percent additional noise distorts the characteristic of
the hue and intensity histograms in Figure 4.34 noticeably; thus, the valley points from
our histogram valley-seeking method might not specify the correct intervals for the
proposed clustering technique. This is a limitation of our proposed segmentation
technique.
Furthermore, in case of the segmentation results of the 6 color characters image as
shown in Figure 4.37, 4.39, 4.41, 4.43 and 4.45, there is not any merging effect in the
image with 25 percent additional noise. However, with 50 percent noise added, the
clusters of A and C characters are merged into a single cluster and so the clusters of 0, I
and T characters are. This effect can be seen from the IHS space in Figure 4.40.
Therefore, the percent mislabeling error and the normalized evaluation function, F',
become larger as seen in Figure 4.25, these are originated from the high penalties of the
third and second term of F' respectively. With 75 and 100 percent additional chromatic
noise, the merging and over-segmentation effect become severer, especially I and T
characters, since their clusters in the IHS space are very close together as shown in Figure
4.43 and 4.45.
According to both values of the percent mislabeling error and the normalized
evaluation function, F', our proposed algorithm yields the best segmentation results, in
case that there is no any additional noise. However, it is also produced a good result in
case that the additional chromatic noise is less than 25 percents of area in the original
image. If the percent of additional noise is larger than 25, the clusters that represent each
distinct class might merge into a smaller class. This merge effect causes different objects
to be represented by the same region and this can be determined distinctly from the
normalized value of the evaluation function, F'.
82
......
...................
....... .. ...
..............
, ...
...............
..........
. ...........
Zero crossing of Intensity Histogram
Zero crossing of histogram of Hue Histogram
20000
20000
- --
15000 --------- -----------
-----------.....
-----------...
..
-------------
10000 ---------- --------- ----
-..-------- ..
---------- ----------
----------- ----------- -----------5000 ------.----- ----------- ------ --1500-
- ----- - -- ----- - - -----
--..
... - ------
1s A I ---------- ---
.
--- - - -- --- -..
--- ----------- ------- ---10000 -----
--..
----------- --- ------- ------- ----.-...--.
500. - ....
S0t)
10
50
0
200
150
100
Zero crossing of
250
0
301
50
150
100
250
200
300
IHS Color Spaces
Saturation Histogram
S~t).
80.----------------------- ----------- -------------------10000...........
70_-50
t-
40
--
-
-
-
50 < -
230
4
0
50
10
150
2)
250
S1
0
-.
100
2
-2
o
4
H
Figure 4.26: 1, H and S histograms and clusters in three-dimensional space of the
synthetic cross image without any additional noise.
I
U U U
UUU U U U
U U U
U
I
I
U
The percent mislabeling
error =0 %
The percent mislabeling
error = 0 %
The percent mislabeling
error = 0 %
Figure 4.27: The segmented regions, representing by white regions, correspond to red,
green and blue colors overlaying on their matching regions. The percent of mislabeling
error of these 3 segmented regions is Opercent and the value of normalized F' is 0.
83
.
.....
. ......
..........
.....
. ............................
Zero crossing of Intensity Histogram
Zero crossing of histogram of Hue Histogram
1600
1400 - - -
.. . .. .. . .. . .. .. . .. .. . .. . .. .. . ..
-------
-------------------- r- -- -------------14 0 ------ ----
42 0 ------ -- --- - - - - ---- -
- -- -- ----
-- ---- --- -
100
-------- -
--
-
-- -- -
-
-
400
----- - -
-- --
..
-..
.
--
0
0
3C0
250
200
150
100
50
50
0
100
----- - --- - ----- -- ------
3000
-
- -- -
-
300
250
200
150
--
---.
-.-.-.
----------------
---------
2500
--
IHS Color Spaces
Zero crossing of Saturation Histogram
3f5D
-
---- ----r---
-
-----
0
---
-------- --- ----- - -----------
200
-200
2UO - ------ ----- ------ --------
-
100-
2tXD
.-.-.-.- ..
- ------------------------- ------------
1500
------.......
------
80
-----
--------- -------------- ----------- ---- --.--- --------- --....
.... ........
. .- -
--- ......
--------.
4
22
2
0
0
so
100
1
150
250
200
0
150
-2
-4
2W
S
Figure 4.28: 1, H and S histograms and clusters in three-dimensional color space of the
synthetic cross image with 25 percent additional noise.
UUKEKU~U
19,M
M
M MNM
M
MM
M
MM
M
The percent mislabeling
error = 55.36 %
The percent mislabeling
error = 0 %
The percent mislabeling
error= 23.11 %
Figure 4.29: The segmented regions, representing by white regions, correspond to red,
green and blue colors overlaying on their matching regions. The percent of mislabeling
error of these 3 segmented regions is 26.15 percent and the value of normalized F' is
0.0006757.
84
of
Zero crossing
histogram
of
Hue Histogram
-----------
-- -- ----- ----------- ----------- -----------------------
-----------
600 ........... . ..................
Zero crossing of Intensity Histogram
1rnn
1fM
..........
..
- ..
.....
..
.....
...
.......
I
.
----------- ----------- ----------- ------- --------------
A --.........
-------------- ..... .........
---------
6M0----------- ----------- ... .......................... ...............
......... -----------
.
AIM
------------
------ --- ----------- -
00
200
.............. . ........
200 ............ ........ ............ ..
..
-----------
....... ----------- -- -- -----
.............. .. ..
----------- ------
---- --- - -----------
.........
....
100
50
250
200
150
3
0
50
100
150
IHS
Zero crossing of Saturation Histogram
-------------
- -- -----------
200
.........
250
300
Color Spaces
------------ ------------- --------------
------------
:-j
2000 ------------- ------------
1500
------------- j --------------I---------------
..............
-------------- ------------ - - ----------- -------------
100 ----------------------- --
-----------------------------------------
.......
.............
100-
------------------------
---------------------
U
0
50
100
150
4
200
2
0
20
2
0
250
-2
-4
S
200IS
Figure 4.30: 1, H and S histograms and clusters in three-dimensional color space of the
synthetic cross image with 50 percent additional noise.
~EUIEI~UEL
-
U IDE
HMEN
-0'
The percent mislabeling
error = 40.36 %
I
E~ULE ~
-MW
The percent mislabeling
error = 3.42 %
20A
a
FA
W:q
24N - OR C
4-1-1
MIN
IN
W --
-Wn-r
U-M -
The percent mislabeling
error = 33.03 %
Figure 4.31: The segmented regions, representing by white regions, correspond to red,
green and blue colors overlaying on their matching regions. The percent of mislabeling
error of these 3 segmented regions is 25.60 percent and the value of normalized F is
0.000750.
85
..........
- .- ........
.......
-.............
............
..
..........
. ............
Zero crossing of histogram of Hue Histogram
500 ----------- -
Zero crossing of Intensity Histogram
- -----..
- -..
--- . . .- ------- -- - -- -..
-
2 00~0
0
1W
50
-------------
--
W
0
-----
400
.
1W
-2
250
2
150
3W
0
T
------
----- - ---
-
-------- --- ---- --
-- -- -- - -----------
-----------
--
--
1W
50
--- - - -
---- - --------------------
150
20
250
3W
IlaS Color Spaces
Zero crossing of Saturation Histogram
2500T 100------------
-
-- -- - -- -- - -- - -- --
- -- -- - -- -- -- - -- -
------ --0
252W
0
-------
---
0 ------- --- ----- -----..--.-.--.--.-.-.----..--------.-.
40
200
-----
-
8W -------------- -----------------------
-- --------- ----------- ----------- -----------
---------
---
--.
........
05
1W
err=5.94
2
150
IIIINhI
Tesegmetdramsin
Figure 4.323:Han
The percent mislabeling
error
=
35.94 %
-
clutresein bythre-ienseional colrrespacetof te
The percent mislabeling
error
=
25.05
%
The percent mislabeling
error
= 37.08 %
Figure 4.33: The segmented regions, representing by white regions, correspond to red,
green and blue colors overlaying on their matching regions. The percent of mislabeling
error of these 3 segmented regions is 32.69 percent and the value of normalized F' is
0.0019.
86
.
. . .
. ..........
.
Zero crossing
Zero crossing of histogram of Hue Histogram
1500
2500T
of Intensity
Histogram
1400
1200
- ----------- -------- ---------- - - ---------- ---- ------ ---
00
400
---- -- --
------ ----
--
-
----
200
0
0
-200
dc~..+..7;...0.230
1100
100
c
Zero crossing
trn
400
50
100
150
IHS Color
of Saturation Histogram
- -- -- ----
---------------
0
300
250
200
150
-------------
200
250
300
Spaces
---------
350
300
250
------- ---
-... . .. .
200
150
-------- --
- - -- -- -- -----
... .
---- ---- ---- ----------------
. .. . .
. . . ..
--
550
00
--- ------------- -- I---
100
S100
50
150
-2
-d
50
100
150
200
250
S
200
-4
Figure 4.34: 1, H and S histograms and clusters in three-dimensional color space of the
synthetic cross image with 100 percent additional noise.
-~
U~U~U~U~U~UV
EEJEEKE I -I--I
'qp
-~
~
u~ ~
%,'
7
'U
E~E7IiU [11131
1.
,14
41,
X
Pak
,, ,
-4,
The percent mislabeling
The percent mislabeling
The percent mislabeling
error = 70.25 %
error = 31.09 %
error = 43.34 %
RP
Figure 4.35: The segmented regions, representing by white regions, correspond to red,
green and blue colors overlaying on their matching regions. The percent of mislabeling
error of these 3 segmented regions is 48.23 percent and the value of normalized F' is
0.0016.
87
...........
..
..
.......
. .....
Zero crossing of Intensity Histogram
Zero crossing of histogram of Hue Histogram
35M
2500
--
------
-------
-- -----
-
- . --
- --------
.-.--------i5m - --
- - - -
- -- _
-
---
-
------
-
-
-
1500
-1 0
0 0 --- -----
-------- ------
-- --- ...2...
-2---
-----____1_
0
.5W1500
. . . ...-..
- -- -----
10
... -..
... . ---
------
I
200
-in
0
0
------------- ----
----------- --25000 ---- ..5.. - -- --- - ----
50
10
100
------- - ...... .----------. - .-.---- ---- ------ ----0--
30
15----Wi------0 - -----25M - - -
t
12010
W-
---- --- -------
. ... .
20 0 - ------ 4----
3W
LHS Space
Zero crossing of Saturation Histogram
SM
3500
250
--- -- - -- --- --- -- -- -__
_
_
-----------------
-----2M
50
s
0
100
150
20
250
S
0
-2
-4
Figure 4.36: 1, H and S histograms and clusters in three-dimensional color space of the
synthetic 6 color characters image without any additional noise.
The percent mislabeling
error of A, C, E, I, 0, T
and background are 0, 0,
0, 0,0,0 and 0 %
respectively.
Figure 4.37: The segmented regions, representing by white and red regions for each
characters and background respectively, correspond to 6 color characters overlaying on
their matching regions. The percent of mislabeling error of these 7 segmented regions is 0
percent and the value of normalized F' is 0.
88
Zero crossing of Intensity Histogram
Zero crossing of histogram of Hue Histogram
4WD0
----------- ----------- ----------- ----------- ---- ----- -----------
3)00
........... ........... ...................... ---- ----- -----------
2000
-
........... .. ......
------------- --.......... ..
........ . ..
400
a-a----------- ---------.....
0
0
q
100
0
0
200
160
260
0
30
5
100
150
2
260
30D
IHS Color Spaces
Zero crossing of Saturation Histogram
AAM
-----
----------------
----
---
400
100
3500
-- ----------- - ------- -- ---------
------------
100
200
400
20I
0
0
100
150
2M
250
S
Figure 4.38: 1, H and S histograms and clusters in three-dimensional color space of the
synthetic 6 color characters image with 25 percent additional noise.
LINg
The percent mislabeling
error of A, C, E, I, On T
and background are
29.16, 0.23, 30.46, 0.23,
0.55 and 67.71
respectively.
Figure 4.39: The segmented regions, representing by white and red regions for each
characters and background respectively, correspond to 6 color characters overlaying on
their matching regions. The percent of mislabeling error of these 7 segmented regions is
18.41 percent and the value of normalized F' is 0.0002982.
89
Zero crossing of Intensity Histogram
Zero crossing of histogram of Hue Histogram
in.
AM r.
.
3500
----------------------- ------------------ ----------- ----
....
...-...-.
--.-.
.
301M
So
.-.-...-- -- ---- - ------- ----..
...
-..
--- ------- .........
---
-25M0 ---.. ......
..
...
400
- -----
..... ----
---------------------- ----- ----- ------
30
- ------------ - -------------- ---------- ---------- -----
2002
1500
...------ - ------ ----- --------- --------- - -------- -
203
500
----
100
- ~-- ----
.-- ~------
0
T-
a
0
50
150
10
250
2W
0
30 D
----------
- -- ------- --- -- - - --- -- - -- -- - ------
-_5
10
50
Zero crossing of Saturation Histogram
150
250
20
300
IHS Color Spaces
120 1500
60 --
1000
20-
----- -------4-------------
S0
20-
- ------------
100
0
0
s0
100
150
20
-2
250
.4
200
S
Figure 4.40: 1, H and S histograms and clusters in three-dimensional color space of the
synthetic 6 color characters image with 50 percent additional noise.
The percent mislabeling
error of A, C, E, I, 0, T
and background are
39.29, 7.38, 51.05,
16.67, 28.41, 38.71 and
66.76 respectively.
~;
4.~U
~4~$\
-
-~
Figure 4.41: The segmented regions, representing by white and red regions for each
characters and background respectively, correspond to 6 color characters overlaying on
their matching regions. The percent of mislabeling error of these 7 segmented regions is
35.47 percent and the value of normalized F' is 0.0009796.
90
...
..................
......
................
......
......
...................
.
Zero crossing of histogram
of Hue Histogram
Zero crossing of Intensity Histogram
30W
2500
4W0
- ---- - .---------------- ------ ----------
2000
. ...
- -.
---------- ----- ----------------- --- --
1500
----- -------- ---------
2W
------------- - ----1000
200
---..--- .---..
....----
------------- - -------
-- ---So
100
----------------- -- -------- - --- ----- -
-500
0
50
10
r
ismc
150
250
20
30
50
0
150
10
250
2M
30
IHS Color Spaces
Zero crossing of Saturation Histogram
100 ) - - -- --I-- -- -- - ---- -- -I--- -- -
-- -- -150
Fr-
0
-150
-50
0
50
100
150
200
250
4
~
~3-
2
-3
-4
Figure 4.42: 1, H and S histograms and clusters in three-dimensional color space of the
synthetic 6 color characters image with 75 percent additional noise.
NJ4
percent mislabeling
The~AW;
error of A, C, E, I, 0, T
and background are
58.65, 4.39, 35.10,
25.76, 31.10, 49.73 and
76.22 respectively.
Figure 4.43: The segmented regions, representing by white and red regions for each
characters and background respectively, correspond to 6 color characters overlaying on
their matching regions. The percent of mislabeling error of these 7 segmented regions is
40.14 percent and the value of normalized F' is 0.0009406.
91
.........
..
of histogram of Hue Histogram
Zero crossing
crossing of Intensity Histogram
Z s hmsZero
AM
..........
........... ...........
-----------------------------------
700
360M
600
400
2500
ics
---- - -
--
-- -- - - ---
- - - - -- - -
---
----
..........
..............
---....--...- -........
-----
------- ----- ------------ -
-----------
----------- I ----------- A .......................
...........
-----------
----------- -----------
-----------
-----------
...........
-
-----------
----------- -----------
100
...........
.......................
-----------
-----------
100
0-
0
...........
-100
0
0
100
150
----------
260
20
3M
...-.....
12m-
................ *---------------I--------------- --------------
.......... ...............
100
........... -----------
- -------------
__-__
------------------------
60
...........
_
_
_
.
i- 0I
IHS Color Spaces
Zero crossing of Saturation Histogram
i'r.
i
300
250
200
150
----------
-----------
............. .............
-
.............
60-
---------------------------- -------
400
-
- ----------------
------ --------------
40
...........
200
0
0
so
20-
--------------------------------
100
150
200
260
2
-20
0t
-2
-4
200
100
S
Figure 4.44: 1, H and S histograms and clusters in three-dimensional color space of the
synthetic 6 color characters image with 100 percent additional noise.
M
j
The percent mislabeling
error of A, C, E, 1, 0, T
and background are
45.46, 32.17, 57.66,
61.50, 49.12, 34.55 and
66.33 respectively.
Figure 4.45: The segmented regions, representing by white and red regions for each
characters and background respectively, correspond to 6 color characters and background
overlay on their matching regions. The percent of mislabeling error of these 7 segmented
regions is 49.54 percent and the value of normalized F' is 0.00 15.
92
4.6.2: Experimental results and evaluations
The objective of this experimental study is to correctly detect and position the
objects of interest with use of the proposed segmentation technique such that the AFM
cantilever and sample can be aligned. In this case, the desired objects are the AFM
cantilever and the straight edge of samples. The AFM cantilever, displayed in Figure
4.46, is a triangular shape thin beam with a standard Silicon Nitride probe mounted on its
tip. This probe is used for scanning the sample's surface. Not only, the detection of these
desired objects in the viewing image is desired, but how accurate the location of these
interested objects is also essential. So as to evaluate the detection performance of
segmentation results, an evaluation criterion similar to the one discussed in last section is
used. However, the acquired images from our AFM's camera are seriously buried with
high-frequency chromatic noise came from the variation in lighting. Especially, the
illumination of samples' edge is very poor because the direction of light in this system
comes from the top of the AFM's cantilever, as shown in Figure 4.47. This causes the
shadow in the camera's field of view directly below the AFM's probe. To eliminate these
variation effects, additional light sources are added to both right and left directions within
the range of 30-60 degree angle with the samples' edge. Besides that the edge-preserving
filter, as discussed in section 4.3, applies three times to individual R, G and B color
spaces to get rid of the chromatic noise. In these experiments, the AFM machine is
manufactured by Quesant Instrument Corp, while the Silicon Nitride scanning probes
mounted on the cantilever is obtained from Digital Instruments.
\35r
35'
3T.
rp
<
35.
Figure 4.46: Standard Silicon Nitride Probe and its diagram of sidewall angle. These
images obtain from "Digital Instruments" Web page (http://www.di.com/).
93
.
......
_ _
....
...........
. ....
.
........
deflection
probe
cantilever
sample
Figure 4.47: The diagram of illumination viewing under original setup of AFM machine.
4.6.2.1: Effect of the additional illumination and the edge-preserving filter
The images' condition can be improved significantly with additional illumination
and the edge-preserving filter, described in section 4.3. The AFM images with different
lighting conditions are compared in Figure 4.48 below. The images without additional
illumination are shown in Image 1 and Image 2, while the images with additional
illumination added to both right and left directions are displayed in Image 3 and Image 4.
In this experiment, the light sources that used for additional illumination are the FiberLite High Intensity illuminator (series 180) from Dolan-Jenner Industries Inc. A level 10
of intensity, which is the highest intensity of this light source, is used in the experiments.
Image 1
Image 2
Without additional light source
Image 3
Image 4
With additional light source from both
Figure 4.48: Effect of additional light source on the images obtained from AFM.
94
...
...
. ......
.
. .
.. .......
.
..............
............
From above figures, the AFM images, Image 1 and 2, from original setup without
any additional light are very dark and blur, only the AFM cantilever, which have a
triangle shape, can be identified. The location of straight samples' edge is very difficult to
position even with human perception. The edge position of samples become more distinct
and easily to be detected by both human and algorithm, when the additional light sources
are added as shown in Image 3 and 4. Also, the AFM cantilevers can still be located
easily because their surface is coated with reflecting material. Figure 4.49 displays
zoomed images of Image 1 and 4 above as well as their smoothing images after filtering
with the edge-preserving filter. According to section 4.3, this adaptive filter blurs image
more in uniform region, while it blurs less in busy region to preserve the edges. In Figure
4.49, the uniform region on around the AFM cantilevers become very smooth, thus the
edges of the cantilever beam and the samples become more sharpen.
Original Image
Original Image
Filtered Image
Filtered Image
Figure 4.49: Effect of the edge-preserving filter on the AFM images.
95
4.6.2.2: Segmentation results
We now apply the proposed segmentation technique to the filtered images
acquired from AFM. The valley points are located on each I, H and S histogram using our
histogram valley-seeking method, described in section 4.4. The connecting box structures
of each image are constructed based on valley points from their histograms. These
connecting nodes are then clustered into distinct classes corresponding to segmented
regions in resulting image. Figure 4.50 through 4.52 show the segmented results of Image
1, 3 and 4 along with their I, H and S histograms. Each segmented region is represented
by different gray-scale region. As an aid for visualization of the boundary of each
segmentation result, the boundary of segmented regions is plotted overlaying the
segmented images in red color. Also, the red vertical lines that are superimposed on each
histogram are used for specifying locations of the valley points. If each histogram is
considered as the distribution in different color dimensions, the segmentation results
using histogram-threshold method, which separates the original image into segmented
regions according to separated intervals of that histogram, shown below each histogram.
In this case, the separated intervals for histogram threshold method are range between
each valley locations.
In the segmentation results of three AFM images, Image 1, 3 and 4, if the
illumination condition of the image is very poor, like Image 1, a noise level becomes very
high and unacceptable. Thus, the recovery of the homogeneous regions without too many
small holes inside is more difficult, particularly with only use of histogram-threshold
method of different color spaces. As shown in Figure 4.50, histogram-threshold images
of hue, intensity and saturation are segmented into scattered small regions due to this
noise effect. On the other hand, if the illumination condition is improved by the
additional light sources from both right and left directions below the AFM cantilever, the
noise level becomes lower so that the segmentation results with use of the histogramthreshold method yield more uniform segmented regions. Nonetheless, in hue histogramthreshold images, the small-segmented regions caused by noise are not completely
eliminated because of non-linear transformation of hue space as explained in section
3.5.5. The segmented regions using only intensity histogram threshold can distinguish
96
most of uniform regions similar to human perception; however, typically segmented
regions from the histogram-threshold method of intensity is over-segmented. For
example, the region corresponding to the AFM cantilever is partitioned into few
segmented regions. Therefore, the proposed clustering technique
combines the
information from all three-color dimensions to refine the segmentation result as shown in
Figure 4.51 and 4.52. The segmented region matched with the AFM cantilever region
becomes more accurate as well as the edge of segmented region that corresponds to the
edge of the sample.
Another significant issue is a computational time of the proposed method. The
CPU time for performing the segmentation of most AFM images is between 50 seconds
to 1 minutes, which is acceptable for real-time image processing. The range of the
computational time depends on how many connecting nodes are created from valley
points in their histograms.
97
...........
Filtered Image 1
4ax -17M
I
I
'Vk
1 1
Segmentation result of Image 1
with 9 distinct regions
I
I
Hue histogram in range 0-300
6 segmented region from hue
histogram threshold
2WX
256
3W3
Intensity histogram in range 0-256 Saturation histogram in range 0-200
12 segmented regions from
intensity histogram threshold
3 segmented regions from
saturation histogram threshold
Figure 4.50: The segmented result of Image 1 and its I, H, S histograms along with
histogram threshold images of each color space.
98
pylyggq
;ii - -, - -
-
--
--
.
........
. ......
Segmentation result of Image 3
with 8 distinct regions
Filtered Image 3
Max -3336
D
1
M~au - 11
1I11Il
Pim
I I I
.
300
200
256
.
Hue histogram in range 0-300
6 segmented region from hue
histogram threshold
Intensity histogram in range 0-256 Saturation histogram in range 0-200
11 segmented region from
intensity histogram threshold
2 segmented region from
saturation histogram threshold
Figure 4.51: The segmented result of Image 1 and its I, H, S histograms along with
histogram threshold images of each color space.
99
...........
-- =VA
MWRPW - -
=-
--
- ---
.....- ....
Segmentation result of Image 4
with 9 distinct regions
Filtered Image 4
ax
..............
.....
-- -- - - --.-
. .......
,- .............
. ...............
-
[L I-________
2816
F
Max-1343
I
-
L
2WX
1
I--
300
Hue histogram in range 0-300
Intensity histogram in range 0-256
6 segmented regions from hue
histogram threshold
11 segmented regions from
intensity histogram threshold
, I
, ., ,
.1 ,
,
Saturation histogram in range 0-200
1 segmented regions from
saturation histogram threshold
Figure 4.52: The segmented result of Image 1 and its I, H, S histograms along with
histogram threshold images of each color space.
100
-
- - -----------
4.6.2.3: Evaluation of segmentation results
As described earlier, the main purposes of this experiment are to correctly detect
and locate the objects of interest. Thus, the segmentation results from our proposed
segmentation technique must detect the desired objects and position them accurately. In
order to verify our objectives, the true regions of the AFM cantilever and the sample are
segmented out from each original image by hand as shown in Figure 4.53 through 4.55.
These segmented regions from the original image represent true regions that can be
identified similarly by the human perception. Then, the result of segmented regions
corresponding to regions of the AFM cantilever and the sample are matched with its true
regions. This is done for justification purpose of accuracy measurement only. An
evaluation criterion similar to the evaluation function, F', is a percent of mislabeling
error, F", or the third term of the evaluation function, F'. In this case, only the segmented
region of the cantilever is evaluated using the F". The percent of mislabeling error is
given in the equation below:
F"(I )
=
r
eAinsideI
rIeAoutside
I
Ainside
Aoutside
)
where F" is the percent of mislabeling error and I, represents the AFM cantilever image.
Ainside
and Aoutside are the area inside and outside the true region of the AFM cantilever
respectively.
eAinside
Ainside )
are the mislabeling ratios of regions inside and
and
( Aoutside )
outside the true regions of the cantilever. These percent mislabeling errors obtained from
these three examples are the relative values; thus, the comparison between these three
cases can be made. The percent of mislabeling error of the AFM cantilever of Image 1, 3
and 4 are 2.18, 0.18 and 0.087 percents respectively. The segmented region of the
cantilever from the good illumination image is around 12 times better than the segmented
region from the poor illumination image.
However, this percent of mislabeling error cannot be applied to identify the
location of samples' edge, since we are interested only on the edge of the sample.
Therefore, the pixels at the right edge of segmented region that overlap with the pixels at
101
........
. ...............
....
.
.
....................
......
........
......
. .....
..
.......
...... .......
.....
the left edge of the blade are labeled with red color as shown below. The number of
pixels of the left edge of the samples that segmented by hand of Image 1, 3 and 4 are 103,
106 and 76 respectively. The result of segmented region corresponding to the sample's
edge in Image 1 does not have a smooth boundary with total pixels of 60 matched with
the sample's edge. While the segmented results of Image 3 and Image 4 have smooth
edges with total pixels of 104 and 79 matched with their samples' edges, as shown in
Figure 4.54 and 4.55. Thus, with the good illumination condition, we can specify the
region of the AFM cantilever and the location of the samples' edge in the image accurate
enough so as to align the probe at the tip of the AFM cantilever with the samples' edge.
IV
Filtered Image 1
True segmented region of the
cantilever in Image 1
Zerror
True segmented region of the
sample in Image 1
The percent of mislabeling
of the cantilever
region is 2.18 %
Figure 4.53: The segmented regions corresponding to the desired objects in Image 1.
102
............
. ...............
I
V
True segmented region of the
cantilever in Image 3
Filtered Image 3
V
True segmented region of the
sample in Image 3
V
The percent of mislabelin g
error of the cantilever
region is 0. 18 %
Figure 4.54: The segmelnited regions corresponding to the desired objects in Image3.
V
Filtered Image 4
V
True segmented region of the
cantilever in Image 4
I
True segmented region of the
sample in Image 4
The percent of mislabeling
error of the cantilever
region is 0.0867 %
Figure 4.55: The segmented regions corresponding to the desired objects in Image4.
103
4.6.3: Segmentation results of natural scenes
Our proposed segmentation technique has also been applied to segment natural
scenes. Figure 4.56 shows a photograph of parrots and its I, H and S histograms as well
as its segmentation result with 13 distinct regions, shown in similar manner as
segmentation results of the AFM images in last section. We can see that its histograms
after opening-closing operation of the images, using the edge-preserving filter with the
same kernel and window sizes as mentioned in section 4.3, is much smoother than its
histograms after opening-closing operation of the original image. Figure 4.57 also
displays another a photograph of Lena and its I, H and S histograms as well as its
segmentation result with 10 distinct regions. The Lena picture has been one of the most
widely used standard test images, that can be found in most image processing researches,
for over 25 years.
As we can see from the segmentation results of both parrots' and Lena's images
that the segmented regions are quite uniform and simple. However, the segmentation
result of parrots image can still capture the detail regions around the parrots eyes due to
color distinction of that region, while the parrots' body regions and background are
segmented into distinct homogeneous classes. In case of Lena's image, there is a lot of
variation, especially in the fur area on the hat; therefore, the segmented results of the fur
area are partitioned into very small segments. While the segmented results of the
background area are composed of only homogeneous and simple regions. Noticeable that
Lena's eyes, nose and mouth are presented by distinct classes in the segmented image.
104
...
..........
...................
.............
I
Segmentation result
Filter image of parrots
Max - 125
Max -21!
Ii
III
I
Hue histogram in range 0-300
Segmentation result with its boundary
zW
256
Intensity histogram in range 0-256 Saturation histogram in range 0-200
I
9 segmented regions from hue
histogram threshold
7 segmented regions from
intensity histogram threshold
5 segmented regions from
saturation histogram threshold
Figure 4.56: The segmented result of parrots image with 13 distinct regions and its I, H, S
histograms along with histogram threshold images of each color space.
105
-
=7
--
--
-
Segmentation result
Original picture of Lena
------- -
.. ...............
. ........
Segmentation result with its boundary
Max-150
256
Hue histogram in range 0-300
7 segmented regions from hue
histogram threshold
Intensity histogram in range 0-256
8 segmented regions from
intensity histogram threshold
Saturation histogram in range 0-200
6 segmented regions from
saturation histogram threshold
Figure 4.57: The segmented result of Lena's image with 10 distinct regions and its I, H, S
histograms along with histogram threshold images of each color space.
106
----
. .........
-
4.7: Summary
In this chapter, our proposed color image segmentation technique is presented as
well as the experimental results and their evaluations. The two major objectives are to
achieve good segmentation results that satisfy the Haralick and Shapiro's basic
segmentation criteria. And then using these segmented results to correctly detect and
locate the AFM cantilever and the edge of the samples in a short-period of time so that
the real-time alignment between the tip of cantilever and the edge of the sample can be
done accurately in a later process. Thus, the segmented results of the image can be used
as a position feedback signal. To be able to accomplish the segmentation quickly, the
histogram-based segmentation applying with our proposed histogram valley-seeking
method is applied to the three-color dimensions to obtain the coarse segmentation by
forming the connecting nodes structure. Then, our proposed clustering technique in three
dimension spaces of IHS refines this coarse segmentation. The main advantages of this
proposed algorithm are that this technique does not require any prior number of class
assignments of the given images and it is not an iterative operation. Therefore, the
computational time from this technique is quite fast. Nevertheless, the large variance or
distribution of color information from each pixel due to the chromatic noise effect is a
major limitation of our segmentation technique.
Furthermore, the evaluation function, F', that satisfy the conventional criteria is
proposed for the justification of the noise-tolerance performance using our proposed
segmentation technique with the synthetic images. As a result, our algorithm can produce
good segmentation results, if the percent of additional chromatic noise in the original
image is less than 25 percents.
A similar evaluation function, F", or percent of mislabeling error is used to
evaluate the segmentation results of the AFM images. If the images have good
illumination condition, the segmented region that matched with the AFM cantilever
yields higher accuracy than the segmented region from the images with poor lighting
condition. The percent of mislabeling error of the AFM cantilever region is about 12
percents lower in case that the viewing images have better lighting. In addition, the
position of the edge of the sample is located more preciously in the images with good
107
lighting condition as well. Practically, both of the evaluation functions, F' and F", cannot
be applied with the natural scenes, since their formulations need the prior information of
the image. Therefore, the evaluation function, F, given by Liu and Yang, is more
appropriate for the natural images since its objective is to evaluate only the difference of
each pixel's color information. Besides that it is not concentrate on the detection of any
specific objects in the image.
108
Chapter 5: Conclusion and recommendations
5.1: Conclusion
To be able to achieve a general-purpose object recognition system, our proposed
segmentation technique is a first step to satisfy this objective. Since most object
recognition systems are generally consisted of the segmentation and classification
processes and the output of the segmentation is considered as an input of the
classification method. Thus, the three main purposes of our segmentation technique are to
partition any image into regions representing the objects of interest, to perform its
computation within a small amount of time and to incorporate the color information into
our segmentation approach.
As mentioned in chapter 3, the suitable color space must be selected so that the
performance of our segmentation technique is the most desirable. In our approach, the
CIELUV and IHS color spaces are used because of the highly correlation with the human
perception and the separation of the chromatic and achromatic values respectively. From
the feature selection in chapter 2, the histogram-based segmentation is the most proper in
several aspects: fast computation, producing closed contour and less over-segmentation
such that our method bases coarsely upon the segmented intervals of the IHS histograms.
Then, our proposed clustering algorithm relied on the normalized variance in the IHS
space combines the node-like structure into distinct clusters according to previous coarse
segmentations. The significant benefits of our segmentation technique are that no prior
information of any image or users' input is required and it is not an iterative procedure.
However, the large scattering of each distinct-class cluster caused by severe noise is
primary drawback of our segmentation approach.
The automatic alignment of the AFM probe and sample accurately using our
proposed segmentation technique is another goal of our research. Using our segmentation
algorithm, the segmented regions of the AFM image with good illumination condition
corresponding to the AFM cantilever and the sample's edge can be identified and
specified their locations accurately. Determining by the percent mislabeling error or the
evaluation function, F", of the segmented region that matches with the AFM cantilever is
109
...
.
...........
.........
. - ..
-
_
-
only about 0.1 percent. Hence, the images' position can be employed as a feedback signal
to control motorized x-y translation stage to move the sample to the desired position in
the later process.
Finally, the evaluation functions, F' and F", are proposed for justification of the
noise-tolerance performance and the accurate alignment of the AFM probe with the
sample correspondingly. The general criteria for segmentation followed from Haralick
and Shapiro's guidelines are incorporated into our evaluation functions. Our algorithm
yields a good segmentation result determined by the normalized evaluation function, F',
when the additional chromatic noise with gaussian distribution is less than 25 percents of
the area in the original image.
5.2: Furtherimprovement and future works
First, one might combine our segmentation technique with the watershed
algorithm so that not only the closed contour of each segmented region is achieved, but
also the edge information of the objects and the spatial information are directly taken into
account. Nonetheless, there are a few issues that need to be carefully considered: faster
implementation and what features that are similar and distinct between our approach and
the watershed algorithm. For instance, if each small segmented region from the watershed
algorithm is grouped into larger region according to their most matched from our
segmentation result. For demonstration, the segmentation result of the AFM image in
Figure 4.52 combined with the watershed algorithm is displayed in Figure 5.1 below:
Segmentation result from our
method
The watershed regions plotted
overlaying our result
Combining watershed regions
with our result
Figure 5.1: The combination of our segmentation result with the watershed algorithm.
110
-
- ==IU
In addition, a diffusion process that is very similar to a heat diffusion process
might be another approach that can improve our segmentation result, because this
operation can remove the scattering noise in the image completely. Typically in this
process, the regions of each object in the original image will gradually merge into more
distinct regions. Although the result from this diffusion method is simple for the
segmentation, its computation is required several iterations to converge to the desired
result.
The classification method must be implemented to classify each segmented region
correctly. One of the classification algorithms is a Neural Networks. We implemented the
competitive learning neural network and tested them with English characters as explained
in more detail in appendix C. Even though, the recognition result is acceptable, the
training process is very computational expensive. At the present time, other existing
classifications; such as a statistical hypothesis testing, a bayes classifier and etc., need to
be further examined and tested. In spite of need of the very huge training set of interested
data, most classification method can be applied restrictively with certain applications.
Nevertheless, in our opinion, the classification for very large data base could been
accomplished with use of the scale-space property to form class and sub-class structure.
Lastly, the control algorithm; such as PID control, must be implemented for
stepping motors attached to the motorized x-y translation stage of the AFM so that the
samples can be reposition accurately and automatically according to the samples'
position computed from the segmentation results. Therefore, the probe-sample alignment
can be accomplished with use of images' locations as the feedback signal.
111
Appendix A: Morphological operations
First, let describe the augmented morphological basis for a gray-scale sampled
signal. To deal with the Euclidean signals, which are real-valued functions in the real
domain R, the supremum (least upper bound) and the infimum (greatest lower bound) are
often used in this mathematical analysis. Simply, the sup and inf operations are closely
related to max and min respectively. Given a collection of signals, fk, with the possible
domain of infinite, the following operations, which can be found in reference [11], can be
defined as followed:
[EXTSUP(fk) ](t) = sup[ fk(t)]
,if there are at least one k such that
fk
is defined at t,
and the supremum apply for all k.
or undefined ,if fk(t) is undefined for all k
[ INF(fk) ](t)
or undefined
=
inf[
fk(t) ]
,if fk(t) is defined for all k
,otherwise
where the domain of EXTSUP(fk) and INF(fk) are the union and the intersection of the
input domains respectively.
For any signal f with its domain Df and a point x in R, fx+y represents a signal f translated
x units to the right and y units up. In addition, the domain of Dfx+y is equal to Df + x.
Then, given the Euclidean signals, f and g with their domain Df and Dg, the dilation of f
by g is defined as the following:
D( f,g)
=
EXTSUPxEDf gx+ f(X) ]
Geometrically, the dialation of f by g is obtained by taking an extended supremum of all
copies of g that are translated to the right x units and up f(x) units. The dual of dilation is
the erosion; thus, the erosion of f by g is defines as the following:
112
[ E( f, g)] = sup{ y: gx + y << f }
The erosion operation is undefined for any point x, if the domain of gx is not a subdomain of Df. Geometrically, the erosion of f by g at the point x can be found by shifting
the function, g, so that the function, g, is centered at x and then finding the largest value
of the vertical translation, y, that still has gx + y values less than f values. Carefully, if Dg
+ x, the domain of gx, is not a subset of the domain of f, then the erosion is undefined at
x.
Therefore, the opening for one-dimensional signal can be defined as the following:
( f , g )= D[ E( f , g ), g ] or
O(f, g)= EXTSUP{ gx + y: gx + y << f }
The domain of the opening is a subset of the input domain, f, but it is not necessarily a
proper subset. The geometrical description of the opening for morphological filtering can
be explained by fitting g, shifted by x and y values, underneath the graph of f.
Similarly, the closing for one-dimensional signal can be defined by taking the dual of the
opening as the following:
C( f, g ) = -O( -f, -g )
The closing of f by g can be achieved by reflecting f and g signals through the x-axis,
applying the opening operation and then reflecting back the resulting signal. The
following three examples, in Figure A.1, A.2 and A.3, show the opening, closing and
opening-closing (O(C( f , g )) effect on the one-dimensional signal that represents the hue
histogram of parrots image in section 4.2.
113
..
. ...........
-4w
Opening operation on hue histogram from parrots image
2500
Blue lihe Original histogram
2000 ---------
- - - -pe-i
- --- -----
----------- - -- - --------
1500
.............
............
1000
500
01
0
....
.........
..
h-50
100
150
200
250
300
Figure A. 1: The opening operation applying to the one-dimensional signal, hue histogram
of the parrots image.
Closing operation on hue histogram from parrots image
2500
Blue line : Original histogram
2000 --------- Red 4
C-lesirng- ist-agr
-- -----
--.--
--..
-.
------
1500
1000
500
-- -------------------
0
-500
F i u------e---i--------------
01
50
100
--------+~--------- -- --- ----- ----- --- -----
the--o--- d--s------
150
200
---
250
s----t----------
300
Figure A.2: The closing operation applying to the one-dimensional signal, hue histogram
of the parrots image.
114
Opening-Closing operation on hue histogram from parrots image
2500
Blue
i ie
2000 -----Red ii-
:Original histogram
-Openmn
vlesing hst grsmt--
r
1500
--
1000
---
----
~1'~
IT
I
---
.
--r--------
- ------
-
500
n
0
50
100
150
200
-I
250
300
Figure A.3: The opening-closing operation applying to the one-dimensional signal, hue
histogram of the parrots image.
115
Appendix B: Classification - Neural Networks
In this appendix B, a section B. 1 introduces a general concept and different types
of neural networks and then in section B.2, three types of competitive learning neural
networks: adaptive template matching (ATM), frequency-sensitive competitive learning
(FSCL), rival-penalized competitive learning (RPCL) algorithms, are explained in more
detail along with their simulation results. The experimental procedure and results of
recognition process with 26 English characters are described in section B.3. Besides that
the proposed orientation adjustment method of binary image is also presented in section
B.3. In section B.4, the advantages and disadvantages of the FSCL algorithm and our
proposed pose-adjustment method for recognition purpose are concluded and suggested
for further improvements.
B.1: Introduction to Neural Networks
Artificial neural networks or Neural Networks has been an active research topic in
several areas, such as classification, clustering, optimization and etc. Many practical
applications, especially pattern recognition, were accomplished with use of neural
networks. Generally, neural networks are computing methods that imitate the biological
neural system of animals. Their basic components are a set of nodes and a set of
connections among the nodes forming a graph structure. Certain criteria, known as
"learning" algorithms, are specified for different types of network structure. Thus, each
node performs some computation according to its particular criterion and then passes
those values, called "weight", through their connections or links to nodes in the next
layer.
Neural networks can be categorized partly as the non-parametric estimators, as
mentioned in chapter 2. Thus, typical learning algorithms are not require any prior
knowledge of input sets and most of them base their computation upon a non-linear node
functions. On the other hand, non-parametric and non-linear statistical procedures are
more difficult and complex than neural networks. However, the statistical results can be
evaluated with more confidence because of the support of existing statistical theorems. Its
116
results are also more understandable with careful interpretation than neural networks.
Thus, the statistical process can then be used to justify the neural networks' results.
Two main types of learning algorithms are supervised and unsupervised learnings.
The examples of supervised learning algorithms are backpropagation, adaptive multilayer
networks, and etc.; while the examples of unsupervised learning algorithms are
competitive learning, adaptive resonance theory and vector quantization learning. In this
study, our main focus is on the competitive learning algorithms because of their
simplicity of its structure and implementation, convergence in a training process and a
fast recall or recognition procedure of its algorithm.
B.2: Competitive Learning Neural Networks
The competitive learning algorithm behaves like the clustering technique;
therefore, data set can be classified into distinct classes that are different from each other
as much as possible. In additional, both training and recalled data sets are not required to
be mutually orthogonal with use of this learning rule and this algorithm yields better
clustering-like performance in high-dimensional spaces. Basically, the structure of
competitive learning networks is consisted of three layers of feedforward networks: input,
hidden and output layers. For example, let consider the simple example of 3-2-3
competitive learning network as shown in Figure B. 1. This network has 3, 2 and 3 nodes
in the input, hidden and output layers respectively.
Layer 0
(Input layer)
Layer 1
(Hidden layer)
Layer 2
(Output layer)
Figure B. 1: A simple 3-2-3 competitive learning neural network.
117
The competitive learning networks have a few weight update rules: adaptive
template matching (ATM), frequency-sensitive competitive learning (FSCL) and rivalpenalized competitive learning (RPCL). A learning mechanism of ATM is a basic for the
development of FSCL and RPCL; thus, the ATM algorithm is first described in detail in
this section and then the FSCL and RPCL learning rules are introduced in that order.
Three of these algorithms are described in more detail in Korn [18]. The structure of
adaptive template matching is an m-n-m competitive learning networks, shown in Figure
B.2. The input layer or layer 0 receives an m-dimensional pattern vector X as an input
and a n-dimensional vector V produces from the input layer. Therefore, there is an n-bym "template matrix"- W associated with the connections or links from input to hidden
layer.
V
X
Z
X[1]
Z[1]
X[2]
Z[2]
X[3]
Z[3]
W
Layer 0
(Input layer)
size: m
WT
Layer 1
(Hidden layer)
size: n
Layer 2
(Output l ayer)
size: m
Figure B.2: A structure of the ATM learning network. Generally, the input and output
layers are an m-dimensional vectors, X and Z respectively and the hidden layer is a ndimensional vector, V. In this illustration, m=3 and n=4.
The values of W are not known in advance, they are initialized with random values.
Then, only one row out of n rows of W that corresponds to the smallest squared templatematching error, E, is updated in each training iteration. This is similar to a "winner-take-
118
all" rule. Thus, V[i]=1 where i is the row of W that corresponds to the row of E with the
smallest value, while the other values or rows of vector V are set to be zeros.
E(i, x) = I (W[i, j] - x[j])2 where i =1,2,..., n
j=1
The updating or learning algorithm for the matrix W is given by the following formula:
W[i, j] = W[i, j]+ Rate(x[j]- W[i, j]) where i =1,2,..., n
where Rate>=O and it is defined as a learning rate. This update law can be described as a
"steepest-descent rule" that minimizes the sample average error of E over all the inputpattern dimensions. With this learning algorithm, the matrix W generally converges
pretty fast, if the learning rate, Rate, reduces so that it approaches zero and the sum of
this learning rate is diverge over all training iterations. The result of this updating rule is
similar to a well-known k-means clustering in statistics. Then, the output vector, Z, yields
a best-input-match output as expressed by the following equation:
Z=WIV
The output vector, Z, is usually used for supplementary computation and resulting
display.
However, the problem associated with this ATM method is that sometime the
outuput pattern does not match any patterns in a set of input data or matches more than
two input patterns. In some cases, it matches with one of the input patterns and then
forgets and changes to match with another input pattern. The simulation result of 9 input
patterns in two dimension (x,y) with this ATM method is shown in Figure B.3. In this
example, the ATM structure is composed of 9 nodes in all input, hidden and output
layers. And the 9 input patterns are (0.8,0.8), (0.8,0), (0.8,-0.8), (0,0.8), (0,0), (0,-0.8), (0.8,0.8), (-0.8,0), (-0.8,-0.8), but they are perturbed by uniform random noise with
amplitude of 0.05. Its initial learning rate is 0.08 and the learning rate increases with rate
of 0.99 in each learning step. The red "plus" signs represent the location of input patterns
in each iteration and the blue circles denote the location of the output vector
corresponding to given input patterns in each iteration. First, we assign all initial values
of W to be very small closed to zeros or one of the input pattern, (0,0).
119
.
.........
..
.............
-
--
--
--------
From the simulation result of Figure B.3, we can see the problems of this ATM
algorithm that stated earlier. Some output vectors converge directly to match with their
corresponding input patterns. Or some output vectors converge to the middle points
between two input patterns before converging again to correct pattern. For instance, the
output vector on the top-left position converges directly to correct input pattern and then
forgets and drifts to match the middle location instead.
For all of the competitive learning algorithm, the best way to prevent the output
vector from converging and staying at the local minimum of sample average error of E is
to perturb the input patterns with uniform random noise in every iterations.
Unsupervised competitive learning of n=9 noisy patterns after
000
iterations
0.8
0 -2..-...4
-..
2.....2
. 40
0
.
0.4
0.6
00
-040
60
-0
-0. 8...
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.8
1
Figure B.3: The simulation result of the ATM algorithm with 9 input patterns.
The frequency-sensitive competitive learning is another learning algorithm of
competitive learning of neural networks. This learning law bases upon the ATM method,
since in the ATM method, the winner row of E that used to modify one of the rows of W
is selected with equal probability. As a result, the output vector might not converge
directly to the input patterns or forget the correct previous pattern and shift to another
pattern. Therefore, an additional bias vector, called "conscience"- H, can be used to
multiply with E before choosing the smallest error from one of the rows of E*H vector.
As a result of this bias vector, the probabilities of all input patterns are distributed evenly,
120
-_-
_--____-_
........
....
.... .....
..............
........
- .........
. ...........
. ...............
.........
. ............
since the input with the smallest probability has more chance to be updated by the
learning rule. Different conscience algorithms were introduced, in our case, only the
"Frequency-sensitive competitive learning" (FSCL) algorithm is discussed briefly. For
FSCL algorithm, the bias vector, H, is a template selection counts for each input pattern.
Then, the smallest value of the E*H vector is picked for deciding which row of W need to
be updated in that iteration. This FSCL algorithm normally yields equal selection-count
values of H; thus, the problems with converge and drifting effects almost eliminate
completely. The simulation test similar to the previous example is done using the FSCL
algorithm and the result is shown in Figure B.4. Two of the position of input patterns are
move from (-0.8,0) and (0.8,0) to (-0.4,0.8) and (-0.4,-0.8) respectively so that in this
example some input patterns are closer and harder to distinguish. Nonetheless, the output
patterns using the FSCL learning rule converge directly to correct input patterns within
4000 iterations. Also, there is not any problem with drifting effect or convergence to
wrong output pattern using this algorithm.
Unsupervised competitive learning of n=9 noisy patterns after 4000 iterations
0.2
00
---.----.. -.------ --.
-0.2 ...-
-.--- ..
-.-..--
--
.. --.
-0.2 -- +
-04
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
Figure B.4: The simulation result of the FSCL algorithm with 9 input patterns.
The rival-penalized competitive learning is an algorithm that modified from the
FSCL algorithm. The different of RPCL and FSCL is in the RPCL updating rule that is
composed of two formulas. The first learning law is the same as the FSCL algorithm and
121
-.
..........
................
. ...............
.....
- 11,11,1111"..","","",
....... .
.........
....
the second or additional learning rule applies to adjust another row of matrix W, which
corresponds to the second smallest row of the E*H vector, with a negative learning rate.
The additional learning rule is given below:
W[k,
j] =
W[k, j] - Const -Rate(x[ j] - W[k, j]) where i =1,2,..., n
where Const is a positive constant and its appropriate value is about 0.05. Using this
additional learning rule, if there are two output patterns that converge to one of the input
patterns, this learning rule will reject the second best match of output vectors so that it
moves into the opposite direction. Therefore, in case that there are input patterns more
than output patterns, only the best-match results of inputs will converge to the output
patterns. A similar simulation test similar to the previous example is shown in Figure B.5.
Unsupervised competitive learning of n=16 noisy patterns after 4000 iterations
-.-.-
-.
0 .8 -
066
.. -.--
-0 .26 --- -- -
040
-0 .4 - ------- -- -- --..-- -
---.
-..- -- -- ---.--
---.
o
04
-06
--
0.6
0.8
1
-08
Figure B.5: The simulation result of the RPCL algorithm with 9 input patterns.
From the above simulation result, all values of the output vector converge correctly to
input patterns, but three of the output patterns are shifted from the center of the input
clusters. Furthermore, some values of the output vector seem to converge to wrong input
patterns at first and then they are rejected with the learning rule in such a way that they
finally converge to correct clusters.
122
B.3: Experimental results and discussion
According to the three simulation results shown formerly, the result from FSCL
algorithm converge directly and quickly to the desired output patterns without any
drifting. Therefore, we use this FSCL algorithm in an experimental testing for pattern
recognition of 26 English characters. The objective of this experiment is to demonstrate
that the FSCL can be useful for a classification purpose of the segmentation results
obtained from our proposed segmentation technique in chapter 4. First, 26 English
characters of size 50x50 pixels were created, all these characters are shown in Figure B.6.
ABCDEFGHI JK
LMNOPQRSTUV
wXYZ
Figure B.6: The 26 English characters used as training inputs for the FSCL algorithm.
In this experiment, each pixel of every character is put into a column vector, so it can be
used for an input vector of the FSCL algorithm; therefore, the dimension of input vectors
is 2500. While, the output dimension is 26 representing each English character. Using
this approach, all characters must be oriented so that their poses do not change with their
orientation before input into the training process. Hence, we propose a method that can
correct the pose of each character. Our method of orientation adjustment bases upon the
major and minor axis of second moment. The normalized second moments of each binary
character are computed as the following:
a= A
JJ(x -
Area f
b=
x) 2
b(x,y)dxdy
2 f(x-_x)(y-_y)b(x,y)dxdy
Area ,
123
C
Area J(y
)2b(xy)dxdy
where I denotes each character's binary image and b(x,y) is the value of binary input (0
or 1) form each pixel (x,y) of the character's image. x and y are the positions of the
center of each character's area and Area is the area or the zero moment of that character.
Then, the angle, 01, between the horizontal or x-axis and the minor axis of the second
moment and the angle, 02, between the horizontal axis and the major axis of the second
moment can be computed as the following equations:
D = b 2 +(a -c)
sin29 =
1=
01 = tan
2
D-(a-c)
2D
and cos2=
sin 2O
si 0
sgn(b) -cos 2)
D+(a- c)
2D
di
and 02 = tan-,
n i2O
sn2
- sgn(b) -cos 20
where sgn(b) is a sign of variable of b. After calculating the angle 01 and 02, the axis of
the largest and smallest second moment can be plotted overlaying each characters as
shown in Figure B.7. Subsequently, their major axis lines divide the area of each
character into two sub-sections, green and blue regions as shown in Figure B.7. The areas
of these sub-sections of each character are determined. Consequently, each character's
orientation is assigned in such a way that the larger sub-section is always located at the
lower position of the character's image as shown in Figure B.8. This orientation
adjustment of each character accomplish with use of a rotational matrix:
cos0
3
-sin90 3
Lsin0
3
cos9 3
_
where 03 is computed so as to locate of the larger sub-area at the bottom of each picture.
All pixels of each original character are transformed with this rotation matrix; however,
because of the discretization of the binary image, each character after the rotational
transformation might contain some small holes inside. The closing operation from
morphological method is one possible solution to eliminate these small holes from the
binary images as shown in Figure B.9. The explanation of the closing operation with
binary image can be found in Giardina and Dougherty. Then, these orientation-adjusted
124
-
----
characters after filling with closing operation are then employed as input patterns for
training the FSCL algorithm.
Figure B.7: The minor axis (red color) and major axis (yellow color) are superimposed on
the 26 English characters so that each character is divided into two separated sub-areas
according to their axis of the largest second moment.
A11
-
19:.11
C
UL
K
a.
1+1
x
SU
Figure B.8: The 26 English characters after their rotational adjustment with used of our
proposed method.
125
------------ --
Figure B.9: Examples of the closing operation of binary image perform on 11 characters
after their rotational adjustment.
Figure B.10: The characters A through P with additional uniform noise obtained from one
of the training iterations.
Then, during the training process, additional random noise with uniform
distribution is added to each pixel of every characters for perturbation purpose. The
amplitude of the additional noise is between 0 and 0.25. Some characters, A through P,
with additional noise from one of iterations of the FSCL training process are shown in
Figure B.10. In this experiment, the FSCL algorithm, done in Matlab, is trained for 750
iterations. The initial learning rate of the FSCL is 0.08 and the decreasing rate of the
learning rate is equal to 0.9. The computational time for this training process is in the
order of 6 hours. However, after training, the recall process to recognize each character
takes only about 5 seconds. Examples of recall processes of V characters in different
shape conditions are shown in Figure B. 11. The V characters in various conditions are
tested in the recall processes because its shape is similar to segmented region that we
obtained from the AFM cantilever as shown in the experimental section of chapter 4.
126
-~1
Therefore, this can proof that although the segmented regions are partially occluded, the
FSCL algorithm can still recognize and output them as correct objects of interest.
A
Case 1:
Case 2:
Case 3:
Case 4:
Different input
patterns of
V character
Orientation
adjusted
Closing operation
of orientation
adjusted images
[raining inputs
with additional
The recalled or
output patterns
Figure B. 11: The examples of the recall process of V ch aracters in different conditions
after training with the FSCL algorithms for 750 iterations.
From the above examples of the recall processes with different V-shape
characters, the FSCL algorithm implemented in this way is very tolerance to noisy input
patterns owing to their large input dimension. As shown in case 1 to 3, the recall process
successfully recognizes the different shape of V characters; even though, the original
shape are partially occluded by the other objects. However, in the case 4, the recall
process fails to recognize it as the V character, since the original image is more similar to
L character than V character. In addition, our proposed orientation adjustment rotates it
accordingly because our proposed pose adjustment method relies only on the global
property of the input binary images. In case of large occlusion, our pose adjustment
cannot reposition the orientation correctly and this is the main drawback of our propose
orientation adjustment.
127
B.4: Summary
The FSCL learning algorithm from the competitive learning is very effective
algorithm, because the training process can be done off-line. The recall process can be
done in short period of time. The primary advantages are that this algorithm is highly
tolerance with severely noise effect and partially occlusion in the recall or recognition
process, as shown in the experimental results. Furthermore, the fast and direct
convergence of this FSCL algorithm is another major advantage, as shown in the
simulation result. The main disadvantage is that its training process in very timeconsuming procedure due to the large input dimension. If a smaller input dimension is
used, the training time as well as its noise-tolerance will be decrease. Therefore, it is a
trade-off between noise-tolerance performance and training time. Moreover, if different
input type with smaller dimension; such as invariant function, is used, the noise-tolerance
performance of this FSCL algorithm will totally depends on the performance of the
invariant function instead. Typically, most invariant functions cannot tolerant level of
noise as well as noise-tolerance performance of our approach shown in our experimental
section. Therefore, from the experimented recognition results, it is very promising that
this algorithm can be implemented as the classification method and employed to
recognize the segmented regions from our proposed segmentation method.
Our proposed orientation adjustment method bases only upon the global
information of the binary image. Thus, our method can correct the orientation of patterns
in case that none or small partially occlusion of the original input occurs. Our pose
adjustment method can be improved by including both the local and global information,
but this is out of our scope at this time.
128
Appendix C: Segmentation Program
In this appendix, our segmentation program that is implemented in C++ language
is given below along with the edge-preserving filter. Note that since we program in the
C++ Builder 4 by Borland, which is categorized as a visual programming; therefore,
some commands might be different from standard C++ language. However, the
comments given in each section will explain most of the concepts.
Define Section:
#define rH 300.0 / the range of the hue histogram
#define rI 256.0 // the range of the intensity histogram
#define rS 200.0 / the range of the saturation histogram
Global Variables:
TImage *Imagel;
int w;
int h;
int minindex;
int *rvalleyh, *rvalleyi, *rvalleys;
int Irvalleyh, Irvalleyi, Irvalleys;
int ***minimumw;
float maxH, minH, rangeH;
float maxL, minL, rangeL;
float maxS, minS, rangeS;
float **Int, **Hue, **Sat;
float **redM, **greenM, **blueM;
// End of Global Variables=======
// 2D array containing the original Image
// Width of the original image
// Height of the original image
// Number of distinct classes in the 3D IHS space
// Vectors containing the intervals of hue, intensity
// and saturation histograms
/Number of the intervals of each histogram
// 3D array containing clustering classes
// Maximum, minimum and range of hue
// Maximum, minimum and range of intensity
// Maximum, minimum and range of saturation
// 2D array of hue, intensity and saturation values
// 2D array of red, green and blue color
Data Structures:
typedef struct THistogram {
int *FHistogram;
int GetCounto;
void Create(float range);
void Clear(float range);
void Increment(int index);
} THistogram;
// End of Data Structure======================================================
// The internal functions of Thistogram******************************************************
// For the histogram display codes, both internal and public functions, base on one function of the
// "ShowImage" program by E.F. Glynn II in Delphi 3 language
int
THistogram::GetCounto
{
int result=O;
int i;
for (i=0;i<255;i++)
result = result + FHistogram[i];
return result;
129
}
------------------------------------------------------
//
void
THistogram::Create(float range) {
Clear(range);
}
//-------------------------------------------------void
THistogram::Clear(float range)
{
int i;
for (i=0;i<range;i++)
FHistogram[i] = 0;
}
/----------------------------------------------void
THistogram::Increment(int index)
{
FHistogram[index]++;
}
// End of THistogram internal functions====================================-----------=
// The public functions of Thistogram *
void DrawHistogram(THistogram *Histogram, TCanvas *Canvas, float range)
{
// range : the range of each specific histogram
const MajorTickSize = 8;
int BarLength, Delta;
int i, j, Height, Width, MaxValue;
TColor Color;
String s;
Color = clBlack;
// {avoid compiler warning about initialization)
Height = Canvas->ClipRect.Bottom;
Width = Canvas->ClipRect.Right;
// For now only paint on a canvas exactly 256 pixels wide. If
// MaxValue is zero, array was not filled in correctly and is ignored.
MaxValue = Histogram->FHistogram[O];
for (i=0;i<range;i++) {
if (MaxValue<Histogram->FHistogram[i]) MaxValue=Histogram->FHistogram[i];
}
if (MaxValue > 0) {
for (i = 0; i < range; i++) {
Color = (TColor)RGB(0,0,1);
//Color = (TColor)RGB(i*0.75, i*0.75, i*0.75);
Canvas->Pen->Color = Color;
BarLength = floor((Height*Histogram->FHistogram[i]) / (float)(MaxValue));
Canvas->MoveTo(i, Height-1);
Canvas->LineTo(i, Height-i -BarLength);
}
130
Canvas->Pen->Color = clDkGray;
// Vertical Lines for visual estimation
for (i = 0; i <= 25; i++) {
Canvas->MoveTo(10*i, Height-1);
if (flnod(i, 5.0) == 0) Delta = MajorTickSize;
else Delta = (MajorTickSize/2.0);
Canvas->LineTo(10*i, Height-I-Delta);
}
// Horizontal Lines
for(j = 0;j <= 4;j++) {
Canvas->MoveTo(0, j*(Height/5.0));
Canvas->LineTo(Width-1, j*(Height/5.0));
}
Canvas->Brush->Style = bsClear;
Canvas->TextOut(2,2, "Max = " + IntToStr(MaxValue));
Canvas->TextOut(2, Height-(Canvas->TextHeight('X')) - MajorTickSize, "0");
s=FloatToStr(range);
Canvas->TextOut(Width-(Canvas->TextWidth(s)),
Height-(Canvas->TextHeight('X')) - MajorTickSize, s);
}
}
--------------------------------------------------------------------------void GetHistogram(float **Bmap, THistogram *Histogram, float range)
{
int index,row, col;
float minI, maxI;
index = 0;
Histogram->Clear(range);
// get min and max values
minl=maxI=Bmap[0][0];
for (row=0;row<h;row++) {
for (col=0;col<w;col++) {
if (minI > Bmap[row][col])
minl=Bmap[row][col];
if (maxI < Bmap[row][col])
maxl=Bmap[row][col];
}
}
// Step through each row of image.
for (row=0;row<h;row++) {
for (col=O;col<w;col++) {
index=(int)(0.5+((Bmap[row][col]-minI)*range/(maxI-minI)));
if (index > (range-1)) index=(range-1);
else if (index < 0) index=O;
Histogram->Increment(index);
}
}
}
//-----------------------------------------------------------
131
void DrawMinVOnHist(THistogram *Histogram, TCanvas *Canvas, float range, int rvalley, int *valley)
{
const MajorTickSize = 8;
int BarLength, Delta;
int i, j, Height, Width, MaxValue;
TColor Color;
String s;
Color = clRed;
Canvas->Pen->Color = Color;
Height = Canvas->ClipRect.Bottom;
Width = Canvas->ClipRect.Right;
// For now only paint on a canvas exactly 256 pixels wide. If
// MaxValue is zero, array was not filled in correctly and is ignored.
MaxValue = Histogram->FHistogram[0];
for (i=0;i<range;i++) {
if (MaxValue<Histogram->FHistogram[i]) MaxValue=Histogram->FHistogram[i];
}
j=0;
if (MaxValue > 0) {
for (i = 0; i <= range; i++) {
/Color = (TColor)RGB(i*0.75, i*0.75, i*0.75);
Color = (TColor)RGB(0,0,1);
if (i==valleyj]) {
Canvas->Pen->Color = clRed;
Canvas->MoveTo(i, Height-1);
Canvas->LineTo(i, 0);
j++;
} else {
Canvas->Pen->Color = Color;
BarLength = floor((Height*Histogram->FHistogram[i]) / (float)(MaxValue));
Canvas->MoveTo(i, Height-1);
Canvas->LineTo(i, Height-i -BarLength);
}
}
Canvas->Pen->Color = clDkGray;
// Vertical Lines for visual display
for (i = 0; i <= 25; i++) {
Canvas->MoveTo(10*i, Height-1);
if (finod(i, 5.0) == 0) Delta = MajorTickSize;
else Delta = (MajorTickSize/2.0);
Canvas->LineTo(10*i, Height-I-Delta);
}
// Horizontal Lines
for (j = 0; j <= 4; j++) {
Canvas->MoveTo(O, j*(Height/5.0));
Canvas->LineTo(Width-1, j*(Height/5.0));
}
Canvas->Brush->Style = bsClear;
Canvas->TextOut(2,2, "Max = " + IntToStr(MaxValue));
132
Canvas->TextOut(2, Height-(Canvas->TextHeight('X')) - MajorTickSize, "0");
s=FloatToStr(range);
Canvas->TextOut(Width-(Canvas->TextWidth(s)),
Height-(Canvas->TextHeight('X')) - MajorTickSize, s);
}
}
// End of THistogram public functions===========================================-----Load Image:
void LoadImageo
{
int shiftx, shifty;
Graphics::TBitmap *BitmapRGB = new Graphics::TBitmap(;
Graphics::TBitmap *BitmapClear = new Graphics::TBitmapo;
shiftx=StrTolnt(Edit I->Text);
shifty=StrToInt(Edit2->Text);
try
{
if (OpenPictureDialog->Executeo) {
w=256; h=256;
Image 1->Width=w; Image I ->Height=h;
BitmapRGB->Freelmageo;
BitmapRGB->LoadFromFile(OpenPictureDialog->FileName);
BitmapRGB->PixelFormat = pf32bit;
if ((shiftx>0)jj(shifty>0)) {
for (int i=0;i<w;i++)
for (intj=0j<hj++)
Image 1->Canvas->Pixels[i][j]=BitmapRGB->Canvas->Pixels[i+shiftx][j+shifty];
} else {
Image 1 ->Picture->Assign(BitmapRGB);
}
}
}
catch (...) {
MessageBeep(- 1);
}
BitmapRGB->Freelmageo;
BitmapClear->Freelmageo;
delete BitmapRGB; delete BitmapClear;
}
/--------------------------------------------------------------------------Convert the original values in the hex format to RGB color and gray scale value:
float GrayConvert(int temp 1, float *red, float *green, float *blue)
{
float redi ,red2;
float green l,green2;
float bluel,blue2;
blue 1=floor(temp 1/1048576.0);
temp 1 =fiod(temp 1,1048576.0);
blue2=floor(temp 1/65536.0);
temp I =finod(temp 1,65536.0);
greenl=floor(templ/4096.0);
133
temp 1 =finod(temp 1,4096.0);
green2=floor(templ/256.0);
temp 1=fmod(temp1,256.0);
red 1=floor(temp 1/16.0);
red2=fmod(temp 1,16.0);
red[0]=(redI * 16.0)+red2;
green[0]=((((green I*4096.0)+(green2*256.0))-256.0)*(255.0/65024.0));
blue[0]=((((blue 1 * 1048576.0)+(blue2*65536.0))-65536.0)*(255.0/16646144.0));
return ((0.298936*red[0])+(0.5870431 *green[0])+(0. 1140209*blue[0]));
}
/--------------------------------------------------------Convert the original Image to RGB space and apply the edge-preserving filter:
void filterRGBO
{
int i, j;
float *red, *green, *blue, sigma2;
red = new float[1]; green = new float[1]; blue = new float[ I];
redM = new float*[h]; greenM = new float*[h]; blueM = new float*[h];
for (int i = 0; i < h; i++)
{
redM[i] = new float[w]; greenM[i] = new float[w]; blueM[i] = new float[w];
}
for (i=0;i<h;i++) {
for (j=0j<wj++) {
G 1=GrayConvert(Image 1->Canvas->PixelsUj][i],red,green,blue);
if (red[0]<0) red[0]=0;
if (red[0]>255)red[0]=255;
if (blue[0]<0) blue[0]=0;
if (blue[0]>255)blue[0]=255;
if (green[0]<0) green[0]=0;
if (green[0]>255) green[0]=255;
redM[i]U]=red[0]; greenM[i][j]=green[0]; blueM[i]j]=blue[0];
}
}
sigma2=3;
filter(redM, 13, sigma2, 10, (4*sigma2));
filter(greenM, 13, sigma2, 10, (4*sigma2));
filter(blueM, 13, sigma2, 10, (4*sigma2));
for (i=0;i<h;i++) {
for (j=0j<wj++) {
Image2->Canvas->Pixels[][i] = (TColor)RGB(redM[i][j],greenM[i][j],blueM[i][j]);
}
}
delete red; delete green; delete blue;
}
//---------------------------------------------The edge-preserving filter: (This filter operation in the C++ code based on the code by M. Nitzberg, D.
Mumford and T. Shiota)
void filter(float **I, int N, float sigma, float mu, int ker)
{
134
// x,y : the current pixel location of filtering
// N : a window size of bluring kernel of guassian function
// ker : a kernel size of bluring kernel of guassian function
// sigma: a breadth of the kernel
// mu : a displacement attenuation
float dx, dy, ex, fx, fy, gy, p, q;
float **EE, **F, **G;
EE = new float*[h]; F = new float*[h]; G = new float*[h];
for (int i=0;i<h;i++) {
EE[i] = new float[w]; F[i] = new float[w]; G[i]= new float[w];
I
// Compute Ix2, IxIy, and Iy2
for (int row=O;row<h;row++) {
for (int col=O;col<w;col++) {
if ((row==O)&&(col==0)) {
dx=(I[1][0]-L[0][0])/2.0;
dy=(I[0][1]-I[O][0])/2.0;
} else if ((row==0)&&(col==(w-1))) {
dx=(I[1][(w- 1)]-I[0][(w- 1)])/2.0;
dy=(I[0][(w-1)1-I[0][(w-2)])/2.0;
} else if ((row==(h-1))&&(col==0)) {
dx=(I[(h- 1)][0]-I[(h-2)][0])/2.0;
dy=(I[(h-1)][l]-I[(h-1)][0])/2.0;
} else if ((row==(h-1))&&(col==(w-1))) {
dx=(I[(h-1)][(w-1)]-I[(h-2)][(w-1)])/2.0;
dy=(I[(h-1)][(w-1)]-I[(h-1)][(w-2)])/2.0;
} else if (row=0) {
dx=(I[1][col]-I[0][col])/2.0;
dy=(I[O][col+1]-I[0][col-1])/2.0;
} else if (row==(h-1)) {
dx=(I[(h- I)][col]-I[(h-2)][col])/2.0;
dy=(I[(h-1)][col+1]-I[(h-1)][col-1])/2.0;
} else if (col==0) {
dx=(I[row+1][0]-I[row-1][0])/2.0;
dy=(I[row][1]-I[row][0])/2.0;
} else if (col==(w-1)) {
dx=(I[row+ I ][(w- 1)]-I[row- I ][(w- 1)])/2.0;
dy=(I[row][(w- 1 )]-I[row] [(w-2)])/2.0;
} else {
dx = (I[row+I][col] - I[row-I][col])/2.0;
dy = (I[row][col+1] - I[row][col-1])/2.0;
}
EE[row][col] dx*dx;
F[row][col]= dx*dy;
G[row][col] = dy*dy;
}
/ Blur with the gaussian function
BlurGaussian(EE,sigma,ker);
BlurGaussian(F,sigma,ker);
BlurGaussian(G,sigma,ker);
135
#define phi(x) (N/4)*x/sqrt(mu*mu + x*x)
for (int row=0;row<h;row++) {
for (int col=O;col<w;col++) {
// Compute the kernel shape, Q,O, that represent in the quadratic form:
if ((row==0)&&(col==0)) {
ex = (EE[1][0] - EE[O][0])/2.0;
fx = (F[1][0] - F[O][0])/2.0;
fy = (F[O][1] - F[O][0])/2.0;
gy = (G[O][l] - G[O][0])/2.0;
} else if ((row==O)&&(col==(w-1))) {
}
}
}
ex = (EE[1][(w-1)] - EE[0][(w-1)])/2.0;
fx = (F[1][(w-1)] - F[O][(w-1)])/2.0;
fy = (F[0][(w-1)] - F[0][(w-2)])/2.0;
gy = (G[O][(w-1)] - G[0][(w-2)])/2.0;
else if ((row==(h-1))&&(col==O)) {
ex = (EE[(h-1)][0] - EE[(h-2)][0])/2.0;
fx = (F[(h-l)][0] - F[(h-2)][0])/2.0;
fy = (F[(h-1)][1] - F[(h-1)][0])/2.0;
gy = (G[(h-1)][1] - G[(h-1)][0])/2.0;
else if ((row==(h-1))&&(col==(w-1))) {
ex = (EE[(h-1)][(w-1)] - EE[(h-2)][(w-1)])/2.0;
fx = (F[(h-1)][(w-1)] - F[(h-2)][(w-1)])/2.0;
fy = (F[(h-1)][(w-1)] - F[(h-1)][(w-2)])/2.0;
gy = (G[(h- 1)][(w- 1)] - G[(h- 1)][(w-2)])/2.0;
else if (row==O) {
ex = (EE[1][col] - EE[0][col])/2.0;
fx = (F[1][col] - F[0][col])/2.0;
fy= (F[O][col+1] - F[0][col-1])/2.0;
gy = (G[0][col+1] - G[0][col-1])/2.0;
} else if (row==(h-1)) {
ex = (EE[(h-1)][col] - EE[(h-2)][col])/2.0;
fx = (F[(h-1)][col] - F[(h-2)][col])/2.0;
fy = (F[(h-1)][col+1] - F[(h-1)][col-l])/2.0;
gy = (G[(h-1)][col+1] - G[(h-1)][col-1])/2.0;
} else if (col==0) {
ex = (EE[row+1][0] - EE[row-1][0])/2.0;
fx = (F[row+I][0] - F[row-I][0])/2.0;
fy = (F[row][1] - F[row][0])/2.0;
gy = (G[row][1] - G[row][0])/2.0;
} else if (col==(w-1)) {
ex = (EE[row+1][(w-1)] - EE[row-I][(w-1)])/2.0;
fx = (F[row+1][(w-1)] - F[row-1][(w-1)])/2.0;
fy= (F[row][(w-1)] - F[row][(w-2)])/2.0;
gy = (G[row][(w-1)] - G[row][(w-2)])/2.0;
} else {
ex = (EE[row+1][col] - EE[row-1][col])/2.0;
fx = (F[row+1][col] - F[row-1][col])/2.0;
fy = (F[row][col+1] - F[row][col-1])/2.0;
gy = (G[row][col+1] - G[row][col-1])/2.0;
}
p = phi((ex+fy)/2.0);
q = phi((fx+gy)/2.0);
I[row][col] = filterjoint(I,row,col,EE[row][col],F[row][col],G[row][col],p,q,N,sigma);
}
136
}
// de-allocate
for (int i=0;i<h;i++) {
delete[] EE[i]; delete[] F[i]; delete[] G[i];
I
delete[] EE; delete[] F; delete[] G;
}
//--------------------------------------------------------------------------float filter_point(float **I,int x,int y,float ee,float ffloat g,float p,float q,int N,float sigma)
{
int dx,dy, r;
double v,norm,total;
r=(int)(N- 1)/2;
norm=total=0.0;
for (dx=-r;dx<=r;dx++) {
for (dy=-r;dy<=r;dy++) {
v=exp(-(ee*dx*dx + 2*f*dx*dy + g*dy*dy + p*x + q*y)/(2*sigma*sigma));
if (((x+dx)<=0)&&((y+dy)<=0))
total=total+I[0][O]*v;
else if (((x+dx)<=0)&&((y+dy)>=(w-1)))
total=total+I[0][w- ]*v;
else if (((x+dx)>=(h-1))&&((y+dy)<=O))
total--total+I[h-1][0]*v;
else if (((x+dx)>=(h- 1))&&((y+dy)>=(w- 1)))
total=total+I[h-1][w-1]*v;
else if ((x+dx)<=0)
total-total+I[0][y+dy]*v;
else if ((x+dx)>=(h-1))
total=total+I[h- 1I][y+dy]*v;
else if ((y+dy)<=O)
total=total+I[x+dx][0]*v;
else if ((y+dy)>=(w-1))
total=total+I[x+dx][w-1]*v;
else
total=total+I[x+dx][y+dy]*v;
norm=norm+v;
}
}
return total/norm;
}
/-------------------------------------------------------------------------void BlurGaussian(float **Im, float sigma, int ker)
{
float **J,*ki;
float norm;
int index, i, j;
// Initialize the J matrix
J = new float*[h];
for (i=0;i<h;i++)
J[i] = new float[w];
137
for (i=O;i<h;i++)
for (j="d<wj++)
J[i][]=0.O;
ki = new float[(2*ker)+1];
for (i=0;i<(2*ker)+1;i++)
ki[i]=0;
// Build the kernel in k
norm=O;
index=O;
for (i=-ker;i<=ker;i++) {
if (abs(i) <= 2*sigma)
ki[index]=exp(-(i*i)/(2.0*sigma*sigma));
else if (((2*sigma) < abs(i))&&(abs(i) <= (4.0*sigma)))
ki[index]=(l/(16.0*ME*ME))*pow((4.0-abs(i)/sigma),4);
else
ki[index]=O;
norn=norm+ki[index];
index++;
}
index=O;
for (i=-ker;i<=ker;i++) {
ki[index]=ki[index]/norm;
index++;
}
Convolve ld(Im,J,ki,"Vertical",ker,h,w);
Convolve 1 d(J,Im,ki, "Horizontal",ker,h,w);
delete ki;
for (i=0;i<h;i++)
delete[] J[i];
delete[] J;
}
//---------------------------------------------------------------------void Convolve l d(float **I, float **J, float *k, String axis, int ker, int hh, int ww)
{
// Perform convolution in ID
int row=O,col=0,=0;
int trow,tcol,index;
float total;
if (axis == "Horizontal") {
for (row=O;row<hh;row++) {
for (col=O;col<ww;col++) {
total=O;
index=O;
for (l=-(ker);l<=(ker);l++) {
//for (l=-(2*ker);l<=(2*ker);l++) {
trow = row + 1;
tcol = col;
if ( (trow>=O) && (trow<=(hh- 1)) && (tcol>=O) && (tcol<=(ww- 1)))
total=total+(I[trow][tcol]*k[index]);
138
index++;
}
J[row][col]=total;
}
}
} else {
for (row=O;row<hh;row++) {
for (col=O;col<ww;col++) {
total=O;
index=O;
for (l=-(ker);l<=(ker);l++) {
/for (l=-(2*ker);l<=(2*ker);l++)
{
trow = row;
tcol = col + 1;
if ( (trow>=O) && (trow<=(hh-1)) && (tcol>=O) && (tcol<=(ww-1)))
total--total+(I[trow][tcol]*kfindex]);
index++;
}
J[row][col]=total;
}
}
}
}
// End of the edge-preserving filter codes================================--------=-The opening-closing filtering operation of the morphological method:
void ClOp(int *f, int *oc, int m)
{
/ f : an original 1D signal of histogram
// oc : an output of ID signal after applying the opening-closing operation
int i, j, k, n=3,ml, nl, mintrof, maxtrof;
int *g, *flninus, *gminus, *etemp, *efg, *ofg;
int **trof;
g = new int[n];
fminus = new int[m]; gminus = new int[n];
= new int[m-n+1]; ofg
trof= new int*[n];
for (i=0;i<n;i++)
trofi] = new int[m+n-I]; / TRAN-OFF operation
etemp = new int[m+n-I]; efg
=
new int[m];
g[]=1; g[1]=5; g[2]=1;
/CLOSING OPERATION-(-OPEN(-f,-g))
for (i=0;i<m;i++)
flninus[i]=- *f[i];
for (i=0;i<n;i++)
gminus[i]=- *g[i];
/ initialize trof
for (i=0;i<n;i++)
for (j=Oj<(m+n-)j++)
trofli][j]=O;
for (i=O;i<n;i++) {
k=O;
for (j=ij<(m+i)j++) {
139
trof[i][]=fminus[k]-gminus[n-i- 1];
k++;
}
}
for (i=0;i<(m+n- 1);i++) {
min trof=trof[O][i];
for (j=1;j<nj++)
if (trofj]j[i]<min trof) min-trof=trof[j][i];
etemp[i]=mintrof;
}
for (i=(n-1);i<m;i++)
efg[i-n+1]=etemp[i];
// delete trof
for (i=;i<n;i++)
for (j=0;j<(m+n-1)j++)
trof[i][j]=O;
ml=m-n+1; nl=n;
for (i=0;i<nl;i++) {
k=O;
for (j=ij<(ml+i)j++) {
trof[i][]=efg[k]+gminus[i];
k++;
}
}
for (=Oj<(n1-1)j++) {
maxtrof=O;
for (i=O;i<nl;i++) {
if (trof[i][j]!=O)
maxtrof=trofji][j];
}
if(max trof!=O)
for (i=0;i<nl ;i++)
if ((trof[i]U]>max trof)&(trof[i][j]!=O))
maxtrof=trofli][j];
ofgj]=max_trof;
}
for (j=(nl-1)j<ml;j++) {
maxtrof=O;
for (i=O;i<nl;i++) {
if (trof[i][j]!=O)
maxtrof=trofi]jU];
}
if (max trof!=O)
for (i=0;i<nl;i++)
if ((trof[i][j]>max-trof)&(trof[i][j]!=0))
maxtrof=trof[i][j];
ofgj]=max-trof;
}
for (j=ml;j<mj++) {
maxtrof=0;
for (i=O;i<nl;i++) {
if (trof[i][j]!=O)
maxtrof=trofli]U];
140
}
if (max trof=O)
for (i=0;i<nl;i++)
if ((trof[i][j]>max trof)&(trofji][j]!=O))
maxtrof=trofli]U];
ofgj]=max_trof;
for (i=0;i<m;i++)
fli]=- I *ofg[i];
// delete trof
for (i=O;i<n;i++)
for (j=0;j<(m+n-1)j++)
trofli]U]=O;
// delete etemp, efg, ofg
for (i=O;i<(m+n- 1);i++)
etemp[i]=0;
for (i=O;i<(m-n+ 1);i++)
efg[i]=O;
for (i=0;i<m;i++)
ofg[i]=0;
//OPENING OPERATION-(ERODE&DILATE)
// initialize trof
for (i=0;i<n;i++)
for (=0j<(m+n-1)j++)
trofi][j]=0;
for (i=0;i<n;i++) {
k=0;
for j=ij<(m+i)j++) {
trof[i][j]=fk]-g[n-i-1];
k++;
}
}
for (i=0;i<(m+n- 1);i++) {
min _trof-trof10][i];
for (j=lj<nj++)
if (troflj][i]<min trof) min-trof=troflj][i];
etemp[i]=min trof;
for (i=(n-1);i<m;i++)
efg[i-n+1]=etemp[i];
// delete trof
for (i=0;i<n;i++)
for (j0j<(m+n-1)j++)
trofti][j]=0;
ml=m-n+1; nl=n;
for (i=0;i<n1;i++) {
k=0;
for (j=ij<(ml+i-1)j++) {
trof[i][]=efg[k]+g[i];
k++;
141
}
}
for (j=0j<(nl-1)j++) {
maxtrof=O;
for (i=0;i<nl;i++) {
if (trof[i][]!=O)
maxtrof=trofli][];
}
if (max trof!=O)
for (i=O;i<nl;i++)
if ((trofli][j]>max trof)&(trofli]U]!=O))
maxtrof=trof[i][j];
ofgj]=max_trof;
}
for (j=(nI-1);j<ml j++)
maxtrof=O;
for (i=0;i<nl;i++)
if (trofji][j]!=O)
{
{
maxtroftrofli][j];
}
if (max trof!=0)
for (i=O;i<n 1;i++)
if ((trofli]U]>max trof)&(trofi]j] !=0))
maxtrof=troffi]u];
ofgj]=max_trof;
}
for (j=ml j<mj++) {
maxtrof=O;
for (i=0;i<nl;i++) {
if (trofji][j]=0)
maxtrof=trofi]U];
}
if(max trof!=O)
for (i=0;i<nl;i++)
if ((trofli]U]>max trof)&(trofti]U]!=0))
maxtrof=trofti][j];
ofg[j]=maxtrof;
}
for (i=0;i<m;i++)
oc[i]=ofg[i];
delete etemp; delete efg; delete ofg; delete fminus; delete gminus; delete g;
for (i=0;i<n;i++) {
delete[] trofli];
}
delete[] trof;
}
/-----------------------------------------------------The propose segmentation technique begins here:
void Operators(TImage *Image2, float **Strength, float **BlurI, float sigma)
{
// fl, f2, 3 : ID signal of hue, intensity and saturation histogram respectively
// H1,L 1, SI : ID signal of hue, intensity and saturation histogram after opening-closing operation
142
int i, j, 1;
int *fl, *f2, *f3;
int maxfl, maxf2, maxf3, minfl, minf2, minf3;
int *LI, *Hl, *Sl;
int kerl=4;
int index, ti, sigmal=1;
float x1, yl, zt, Y, temp1, temp2, temp3, al, b1, sum;
float uO, vO, u, v, x, yO, dent, den2;
float **G;
float *red, *green, *blue;
float *k, **t3, *gau, *dgau, *t;
int N=2*sigma, ker=(2*sigma);
int thresh=100;
int row, col;
float mu=100.0;
maxH=0.0; rangeH=0.0; maxL=0.0; rangeL=0.0; maxS=0.0; rangeS=0.0;
G = new float*[h]; Ix2 = new float*[h]; Ixly = new float*[h]; Iy2 = new float*[h];
for (int i = 0; i < h; i++) {
G[i] = new float[w]; Ix2[i] = new float[w]; txly[i] = new float[w]; Iy 2 [i]= new float[w];
}
red = new float[I]; green = new float[t]; blue = new float[I];
fl = new int[rH]; f2 = new int[rI]; f3 = new int[rS]; f4 = new int[rW];
Ht = new int[rH]; Lt = new int[rl]; St = new int[rS]; Wt = new int[rW];
// The values of uO and vO for the reference white D65
x0=95.05/303.93; yO=100.0/303.93;
u0=(2.0*x0)/(6.0*y0-x0+1.5); // uO for D65 White
v0=(3.0*y0)/(6.0*y0-x0+1.5); ! vO for D65 White
for (row=0;row<h;row++) {
for (col=0;col<w;col++) {
G[row][col]=GrayConvert(Imagel->Canvas->Pixels[col][row],red,green,blue);
// X
yl=(0.222*redM[row][col])+(0.707*geenM[row][col])+(0.071*blueM[row][col]); // Y
z =(0.020*redM[row][col])+(0. 130*greenM[row][col])+(0.939*blueM[row][col]); // Z
if (xl<0.0) xI=0.0;
if (yl <0.0) yt=0.0;
if (zt<0.0) zl=0.0;
xl=(0.430*redM[row][col])+(0.342*greenM[row][col])+(0.178*blueM[row][col]);
// Conversion from RGB color space to LUV (CIELUV) color space
Y=yl;
denl=xt+yI+z1; // X+Y+Z
if (dent!=0.0) {
xl=xl/denl; yl=yl/denl;
}
den2=(6.0*yl)-xl+1.5;
u=(2.0*xt)/den2;
v=(3.0*yl)/den2;
templ=1 6.0*pow(Y,(t.0/3.0))-l6.0;
temp2=3.0*tempt *(u-uO); // u*
temp3=13.0*temp I*(v-vO); // v*
143
// Conversion from LUV (CIELUV) color space to IHS color space
Int[row][col]=temp1;
Hue[row][col]=atan2(temp3,temp2);
Sat[row][col]=pow((pow(temp2,2.0)+pow(temp3,2.0)),O.5);
if ((row==O)&&(col==O)) {
minH=Hue[O][0]; minL=Int[O][O]; minS=Sat[O][0];
} else {
if (minH>Hue[row][col]) minH=Hue[row][col];
if (minL>Int[row][col]) minL=Int[row][col];
if (minS>Sat[row][col]) minS=Sat[row][coll;
}
if (maxH<Hue[row][col]) maxH=Hue[row][col];
if (maxL<Int[row][col]) maxL=Int[row][col];
if (maxS<Sat[row][col]) maxS=Sat[row][col];
if (G[row][col]<O) {
G[row][col]=O;
} else if (G[row][col]>255)
G[row][col]=255;
}
}
{
}
rangeH=maxH-minH; rangeL=maxL-minL; rangeS=maxS-minS; //rangeW=maxW-minW;
// de-allocate
delete red; delete green; delete blue;
for (i = 0; i < h; i++) {
delete[] redM[i]; delete greenM[i]; delete blueM[i];
}
delete[] redM; delete[] greenM; delete[] blueM;
// Show Histograms ********************************************************************
// Show hue histogram
THistogram *Histogram;
Histogram = new THistogram;
Histogram->FHistogram = new int[rH];
Screen->Cursor = crHourGlass;
try {
Histogram->Create(rH);
try {
GetHistogram(Hue, Histogram, rH);
DrawHistogram(Histogram, HueHist->Canvas, rH);
for (i=O;i<rH;i++)
fl [i]=Histogram->FHistogram[i];
} _finally {
delete Histogram;
}
} _finally {
Screen->Cursor = crDefault;
}
// Show intensity histogram
144
Histogram = new THistogram;
Histogram->FHistogram = new int[rI];
Screen->Cursor = crHourGlass;
try {
Histogram->Create(rI);
try {
GetHistogram(Int, Histogram, rI);
DrawHistogram(Histogram, IntHist->Canvas, rI);
for (i=0;i<rI;i++)
f2[i]=Histogram->FHistogram[i];
} _finally {
delete Histogram;
}
} _finally {
Screen->Cursor = crDefault;
}
// Show saturation histogram
Histogram = new THistogram;
Histogram->FHistogram = new int[rS];
Screen->Cursor = crHourGlass;
try {
Histogram->Create(rS);
try {
GetHistogram(Sat, Histogram, rS);
DrawHistogram(Histogram, SatHist->Canvas, rS);
for (i=O;i<rS;i++)
f3[i]=Histogram->FHistogram[i];
} _finally {
delete Histogram;
}
} _finally {
Screen->Cursor = crDefault;
}
/ Find maximum and minimum value of each histogram
maxfl=fl [0]; minfl =fl [0];
maxf2=f2[0]; minf2=f2[0];
maxf3=f3[0]; minf3=f3[0];
for (i=1;i<rH;i++) {
if (i<rI) {
if (maxf3<f3[i]) maxf3=f3[i];
if (minf3>t3[i]) minf3=f3[i];
if (maxf2<f2[i]) maxf2=f2[i];
if (minf2>f2[i]) minf2=f2[i];
}
if (maxfl <fl [i]) maxfl =fl [i];
if (minfl >fl [i]) minfl =fl [i];
*************************************
}
// Morphology filter ********************************************************
ClOp(fl,H 1,rH);
ClOp(f2,L 1,rI);
ClOp(f3,Sl,rS);
// Creating the ID guassian Filter*********************************************************
/ initialize
145
k = new float[(4*kerl)+1]; / ID signal of the convolution of the gaussian with its derivatinve
t = new float[(2*kerl)+ 1];
gau = new float[(2*kerl)+1]; / ID signal of the gaussian function
dgau = new float[(2*kerl)+1]; // ID signal of the derivative of gaussian function
t3 = new float*[3];
for (i=0;i<3;i++)
t3[i] = new float[(2*kerl)+1I];
// Build the kemal in k
index=O;
for (i=-kerl;i<=ker1;i++)
t[index]=i;
index++;
}
{
for (i=O;i<3;i++) {
for (j=0j<(2*kerl)+1j++)
switch (i) {
case 0:
t3[0]U]=t[j]-0.5;
case 1:
t3[l][]W=tj];
case 2:
t3[2][]=tUj]+0.5;
}
}
{
}
for (i=0;i<(2*kerl)+1;i++) {
sum=0;
for (j0j<3j++)
sum+=exp((t3[j][i]*t3[j][i])/(-2*sigmaI *sigmal));
gau[i]=sum/(6*MPI*sigmal *sigmal);
dgau[i]=(-1 *t[i]*exp((t[i]*t[i])/(-2*sigmal *sigmal ))/(sigmal *sigmal));
}
// de-allocate
delete t;
for (i=0;i<3;i++)
delete[] t3[i];
delete[] t3;
// Convolution of gau and dgau
for (i=0;i<(4*kerl)+l;i++) {
sum=0.0;
for (j=;j<(2*kerl)+1 j++) {
if ( ((i-j)>=0) && ((i-j)<(2*kerl)+ 1))
sum+=gauj]*dgau[i-j];
}
k[i]=sum;
}
// de-allocate
delete gau; delete dgau;
// Performing our histogram valley-seeking and clustering technique
146
Cluster(maxfl,maxf2,maxf3,fl,f2,f3,H1,L 1,S1 ,k);
delete fl; delete f2; delete f3; delete f4;
delete LI; delete HI; delete SI; delete Wi;
delete k;
}
void Custer(int maxfl, int maxf2, int maxf3, int *fl, int *f2, int *f3, int *H1, int *L1, int *Sl, float *kl)
{
/ kI : the convolution of gaussian and its derivative
int i, j, k, 1, row, col;
int Lh, Ls, Li;
int *Npt;
int ***minwater2; //, **regionl;
float ***minwater;
float **meanhls, **varhls;
float sumh, suml, sums, sum2h, sum2l, sum2s;
float sizeH, sizeL, sizeS;
float tbegin, tend;
//Hue Declaration *************************
int ker=4;
int index, index2, count, incr, Caselncr;
float sum, incr2;
float mthresh=0.5; / threshold of histogram valley-seeking technique
float *coh2; // The convoluted signal of hue histogram after the opening-closing operation with ki
int *peakh, *valleyh; // The locations of peak and valley points in the convolution signal
int *rpeakh, *tempvalleyh; / temporary signal for peak and valley points of the hue histogram
//
Int Declaration **********************************************************************
float *coi2; // The convoluted signal of histogram after the opening-closing operation with ki
int *peaki, *valleyi; // The locations of peak and valley points in the convolution signal
int *rpeaki, *tempvalleyi; / temporary signal for peak and valley points of the intensity histogram
// Sat Declaration **********************************************************************
float *cos2; / The convoluted signal of histogram after the opening-closing operation with kI
int *peaks, *valleys; // The locations of peak and valley points in the convolution signal
int *rpeaks, *tempvalleys; / temporary signal for peak and valley points of the saturation histogram
/Minimum Declaration ****************************************************************
bool terminate;
int areal, tx[10], ty[10], tz[10];
int rangelndex;
/***********************************************************************************
// Operation on hue space *
coh2 = new float[rH+(4*ker)];
peakh = new int[rH+(4*ker)]; valleyh = new int[rH+(4*ker)];
// Perform the convolution of the hue signal after opening-closing operation and ki
for (i=0;i<(rH+(4*ker));i++) {
sum=0.0;
for (j=0;j<rH+(4*ker)+ 1j++) {
147
if ( ((i-j)>=0) && ((i-j)<(4*ker)+ 1))
sum+=(H1[j]*kl[i-j]);
}
coh2[i]=sum;
I
// Initialize the peak and valley of the convoluted signal
for (i=0;i<(rH+(4*ker));i++) {
peakh[i]=0; valleyh[i]=0;
}
//Look for the zero crossings: +-, -+
/We choose the edge to be the negative point
for (i=0;i<(rH+(4*ker));i++) {
if (i<(rH+(4*ker)-1)) {
if ( (coh2[i]<0) & (coh2[i+1]>0) & (fabs(coh2[i]-coh2[i+1])>mthresh)) { // [(-)+]
valleyh[i]=10;
}
}
if (i>0) {
if ( (coh2[i-l]>0) & (coh2[i]<0) & (fabs(coh2[i-l]-coh2[i])>mthresh))
{ // [(+)-]
peakh[i]= 10;
}
}
}
//Most likely this covers all of the cases. Just check to see if there are any points
//where the col was precisely zero:
for (i=1;i<(rH+(4*ker)-1);i++) {
if (fabs(coh2[i])<0.0001) {
if ( (coh2[i-1]<0) & (coh2[i+1]>0) & (fabs(coh2[i-1]-coh2[i+1])>2*mthresh) ) // [- 0 +]
valleyh[i]=10;
if ( (coh2[i-1]>0) & (coh2[i+l]<0) & (fabs(coh2[i-l]-coh2[i+1])>2*mthresh) ) // [+ 0 -]
peakh[i]= 10;
}
i
index=0; I/ Number of the peak point
index2=0; // Number of the valley point
for (i=(2*ker)-1 ;i<(rH+(2*ker))+ 1;i++) {
if (peakh[i]>1) index++;
if (valleyh[i]>1) index2++;
}
/ initialize temporary signal
rpeakh = new int[index]; tempvalleyh = new int[index2];
index=O;
for (i=(2*ker)-l;i<(rH+(2*ker))+l;i++) {
if (peakh[i]>1.0) {
rpeakh[index]=i-(2*ker);
index++;
}
}
index--;
index2=0;
for (i=(2*ker)-1;i<(rH+(2*ker))+l;i++) {
if (valleyh[i]>1.0) {
tempvalleyh[index2]=i;
148
index2++;
}
}
index2--;
// Initialize the signal that contain the final valley points
if (index<=0) index= 1;
if (index2<0) index2=0;
Caselncr=0;
if ((fl[0]>=(0.5*maxfl)) && (fl[(int)(rH-l)]>=(0.5*maxfl)))
( (fabs(fl[2]-fl[1])>=1000.0 ) && ( fabs(fl[(int)(rH-2)]-fl[(int)(rH-3)]) >= 1000.0))) {
rvalleyh=new int[index+3];
incr=2; Irvalleyh=index+3;
for (i=0;i<lrvalleyh;i++)
rvalleyh[i]=0;
rvalleyh[ I ]=rpeakh[0]/2; rvalleyh[index+ 1]=(rH+rpeakh[index])/2; rvalleyh[index+2]=rH;
} else if ( ((fl[0]>=(0.5*maxfl)) && (fl[(int)(rH-1)]<(0.5*maxfl)))II
( ( fabs(fl [2]-fl [1 ])>= 1000.0 ) && ( fabs(fl [(int)(rH-2)]-fl [(int)(rH-3)]) < 1000.0))) {
rvalleyh=new int[index+2];
incr=2; Irvalleyh=index+2;
for (i=0;i<lrvalleyh;i++)
rvalleyh[i]=0;
rvalleyh[1]=rpeakh[0]/2;
} else if ((fl[0]<(0.5*maxfl)) && (fl[(int)(rH-l)]>=(0.5*maxfl)))
( (fabs(fl[2]-fl[1])<1000.0 ) && ( fabs(fl[(int)(rH-2)]-fl[(int)(rH-3)]) >= 1000.0))) {
rvalleyh=new int[index+2];
incr=l; lrvalleyh=index+2;
for (i=0;i<lrvalleyh;i++)
rvalleyh[i]=0;
rvalleyh[index]=(rH+rpeakh[index])/2;
} else {
rvalleyh=new int[index+1];
incr=l; Irvalleyh=index+1; Caselncr=1;
for (i=;i<lrvalleyh;i++)
rvalleyh[i]=0;
}
rvalleyh[0]=0;
for (i=0;i<index;i++) {
count=0;
if (index2>0)
for (j=0;j<index2;j++)
if ( ((tempvalleyhj]-(2*ker))>rpeakh[i]) & ((tempvalleyh[j]-(2*ker))<rpeakh[i+1]))
rvalleyh[i+incr]=tempvalleyh[j]-(2*ker);
count++;
}
if ((count == 0)&&(trvalleyh[i+incr]==0))
rvalleyh[i+incr]=(rpeakh[i]+rpeakh[i+l])/2;
{
}
if (Caselncr==1) rvalleyh[index]=rH;
else rvalleyh[index+incr]=rH;
// Delete Findvalley Hue *
delete peakh; delete valleyh;
delete rpeakh; delete tempvalleyh;
delete coh2;
149
// Operation on intensity space ***********************************************************
coi2 = new float[rI+(4*ker)];
peaki = new int[rI+(4*ker)]; valleyi = new int[rI+(4*ker)];
// Perform the convolution of the intensity signal after opening-closing operation and kI
for (i=0;i<(r+(4*ker));i++) {
sum=0.0;
for (j0j<rI+(4*ker)+ 1 j++) {
if ( ((i-j)>=0) && ((i-j)<(4*ker)+ I)
sum+=(Ll[j]*kl [i-j]);
}
coi2[i]=sum;
}
// Initialize the peak and valley of the convoluted signal
for (i=0;i<(rI+(4*ker));i++) {
peaki[i]=0; valleyi[i]=0;
}
//Look for the zero crossings: +-, -+
//We choose the edge to be the negative point
for (i=0;i<(rI+(4*ker));i++) {
if (i<(rI+(4*ker)- 1)) {
if ( (coi2[i]<0) & (coi2[i+l]>O) & (fabs(coi2[i]-coi2i+l])>mthresh)) { // [(-)+]
valleyi[i]=10;
}
}
if (i>0) {
if ( (coi2[i-l]>0) & (coi2[i]<0) & (fabs(coi2[i-l]-coi2[i])>mthresh))
peaki[i]=10;
{ // [(+)-]
}
}
}
//Most likely this covers all of the cases. Just check to see if there are any points
/where the co 1 was precisely zero:
for (i=1 ;i<(rI+(4*ker)- 1);i++) {
if (fabs(coi2[i])<0.000I) {
if ( (coi2[i-l]<0) & (coi2[i+l]>0) & (fabs(coi2[i-l]-coi2[i+l])>2*mthresh) ) // [- 0 +]
valleyi[i]=10;
if ( (coi2[i-l]>0) & (coi2[i+l]<0) & (fabs(coi2[i-1]-coi2[i+l])>2*mthresh) ) // [+ 0 -]
peaki[i]=10;
}
}
index=0; // The number of peak points
index2=0; // The number of valley points
for (i=(2*ker)-l;i<(rI+(2*ker))+l;i++) {
if (peaki[i]>I) index++;
if (valleyi[i]>1) index2++;
}
/ initialize temporary signal
rpeaki = new int[index]; tempvalleyi = new int[index2];
index=0;
for (i=(2*ker)-l;i<(rI+(2*ker))+l;i++) {
if (peaki[i]>1.0) {
rpeaki[index]=i-(2*ker);
150
index++;
}
}
index--;
index2=0;
for (i=(2*ker)-1;i<(rI+(2*ker))+1;i++)
if (valleyi[i]>1.0) {
tempvalleyi[index2]=i;
index2++;
}
{
}
index2--;
// Initialize the signal that contain the final valley points
if (index<=0) index=1;
if (index2<0) index2=0;
Caselncr=0;
if( ((f2[0]>=(0.5*maxf2)) && (f2[(int)(rI-1)]>=(0.5*maxf2)))
( (fabs(f2[2]-f2[1])>=1000.0 ) && ( fabs(f2[(int)(rl-2)]-f2[(int)(rI-3)]) >= 1000.0))) {
rvalleyi=new int[index+3];
incr=2; lrvalleyi=index+3;
for (i=0;i<lrvalleyi;i++)
rvalleyi[i]=0;
rvalleyi[1]=rpeaki[0]/2; rvalleyi[index+ 1 ]=(rI+rpeaki[index])/2; rvalleyi[index+2]=rI;
} else if ( ((f2[0]>=(0.5*maxf2)) && (f2[(int)(rI-1)]<(0.5*maxf2))) 11
( ( fabs(f2[2]-f2[1])>=1000.0 ) && ( fabs(f2[(int)(rI-2)]-f2[(int)(rI-3)]) < 1000.0)))
rvalleyi=new int[index+2];
incr=2; lrvalleyi=index+2;
for (i=;i<lrvalleyi;i++)
rvalleyi[i]=0;
rvalleyi[1]=rpeaki[0]/2;
} else if ((f2[0]<(0.5*maxf2)) && (f2[(int)(rI-1)]>=(0.5*maxf2))) 11
( (fabs(f2[2]-f2[l])<l000.0 ) && ( fabs(f2[(int)(rI-2)]-f2[(int)(rI-3)]) >= 1000.0)))
rvalleyi=new int[index+2];
incr=1; lrvalleyi=index+2;
for (i=0;i<lrvalleyi;i++)
rvalleyi[i]=0;
rvalleyi[index]=(rI+rpeaki[index])/2;
} else {
rvalleyi=new int[index+1];
incr=1; Irvalleyi=index+1; Caselncr=1;
for (i=0;i<lrvalleyi;i++)
rvalleyi[i]=0;
{
{
}
rvalleyi[0]=0;
for (i=0;i<index;i++) {
count=0;
if (index2>0)
for (j=0j<index2;j++)
if ( ((tempvalleyij]-(2*ker))>rpeaki[i]) & ((tempvalleyi[j]-(2*ker))<rpeaki[i+1]))
rvalleyi[i+incr]=tempvalleyiU]-(2*ker);
count++;
}
if ((count == 0)&&(rvalleyi[i+incr]==0))
{
151
rvalleyi[i+incr]=(rpeaki[i]+rpeaki[i+1])/2;
}
if (Caselncr==1) rvalleyi[index]=rI;
else rvalleyi[index+incr] =rI;
//Delete Findvalley Int ****************************************************
delete peaki; delete valleyi;
delete rpeaki; delete tempvalleyi;
delete coi2;
// Sat space ****************************************************************
cos2 = new float[rS+(4*ker)];
peaks = new int[rS+(4*ker)]; valleys = new int[rS+(4*ker)];
// Perform the convolution of the saturation signal after opening-closing operation and k1
for (i=0;i<(rS+(4*ker));i++) {
sum=0.0;
for (j=0;j<rS+(4*ker)+1 j++) {
if ( ((i-j)>=0) && ((i-j)<(4*ker)+1))
sum+=(SlU]*kl[i-j]);
}
cos2[i]=sum;
}
/Initialize the peak and valley of the convoluted signal
for (i=0;i<(rS+(4*ker));i++) {
peaks[i]=0; valleys[i]=0;
I
//Look for the zero crossings: +-, -+
//We choose the edge to be the negative point
for (i=0;i<(rS+(4*ker));i++) {
if (i<(rS+(4*ker)-1)) {
if ( (cos2[i]<0) & (cos2[i+l]>0) & (fabs(cos2[i]-cos2[i+l])>mthresh))
valleys[i]=10;
}
}
if (i>0) {
if ( (cos2[i-1]>0) & (cos2[i]<0) & (fabs(cos2[i-1]-cos2[i])>mthresh))
peaks[i]=10;
}
}
{ // [(-)+]
{ // [(+)-]
}
//Most likely this covers all of the cases. Just check to see if there are any points
/where the co 1 was precisely zero:
for (i=1;i<(rS+(4*ker)-1);i++) {
if (fabs(cos2[i])<0.0001) {
if ( (cos2[i-1]<0) & (cos2[i+l]>O) & (fabs(cos2[i-1]-cos2[i+1])>mthresh) ) // [- 0 +]
valleys[i]=10;
if ( (cos2[i-1]>0) & (cos2[i+1]<0) & (fabs(cos2[i-1]-cos2[i+l])>mthresh) ) // [+ 0 -]
peaks[i= 10;
}
index=0; // The number of peak points
index2=0; // The number of valley points
for (i=(2*ker)-1;i<(rS+(2*ker))+1;i++) {
152
if (peaks[i]>1) index++;
if (valleys[i]>1) index2++;
}
// initialize temporary signal
rpeaks = new int[index]; tempvalleys = new int[index2];
index=0;
for (i=(2*ker)-1;i<(rS+(2*ker))+1;i++)
if (peaks[i]>1.0) {
{
rpeaks[index]=i-(2*ker);
index++;
}
}
index--;
index2=0;
for (i=(2*ker)-1;i<(rS+(2*ker))+1;i++)
if (valleys[i]>1.0) {
tempvalleys[index2]=i;
{
index2++;
}
}
index2--;
/ Initialize the signal that contain the final valley points
if (index<=O) index=1;
if (index2<0) index2=0;
Caselncr=0;
if ( ((f3[0]>=(0.5*maxf3)) && (f3[(int)(rS-1)]>=(0.5*maxf3)))|1
( (fabs(f3[2]-f3[1])>=1000.0 ) && ( fabs(f3[(int)(rS-2)]-f3[(int)(rS-3)]) >= 1000.0)))
{
rvalleys=new int[index+3];
incr=2; lrvalleys=index+3;
for (i=0;i<lrvalleys;i++)
rvalleys[i]=0;
rvalleys[1]=rpeaks[0]/2; rvalleys[index+1 ]=(rS+rpeaks[index])/2; rvalleys[index+2]=rS;
} else if ( ((f3[0]>=(0.5*maxf3)) && (f3[(int)(rS-1)]<(0.5*maxf3))) 1
( ( fabs(S3[2]-f3[1])>=1000.0 ) && ( fabs(f3[(int)(rS-2)]-f3[(int)(rS-3)]) < 1000.0)))
{
rvalleys=new int[index+2];
incr=2; rvalleys=index+2;
for (i=0;i<lrvalleys;i++)
rvalleys[i]=0;
rvalleys[ 1]=rpeaks[0]/2;
} else if ( ((f3[0]<(0.5*maxf3)) && (f3[(int)(rS-1)]>=(0.5*maxf3)))
( ( fabs(f3[2]-f3[1])<1000.0 ) && ( fabs(f3[(int)(rS-2)]-f3[(int)(rS-3)]) >= 1000.0)))
{
rvalleys=new int[index+2];
incr=1; Irvalleys=index+2;
for (i=0;i<lrvalleys;i++)
rvalleys[i]=0;
rvalleys[index]=(rS+rpeaks[index])/2;
} else {
rvalleys=new int[index+ 1];
incr= 1; lrvalleys=index+1; Caselncr=1;
for (i=0;i<lrvalleys;i++)
rvalleys[i]=0;
}
153
rvalleys[0]=0;
for (i=0;i<index;i++) {
count=O;
if (index2>0)
for (=0;j<index2;j++)
if ( ((tempvalleys[j]-(2*ker))>rpeaks[i]) & ((tempvalleys[j]-(2*ker))<rpeaks[i+1]))
{
rvalleys[i+incr]=tempvalleysUj]-(2*ker);
count++;
}
if ((count == 0)&&(rvalleys[i+incr]==0))
rvalleys[i+incr]=(rpeaks[i]+rpeaks[i+1 ])/2;
}
if (Caselncr==1) rvalleys[index]=rS;
else rvalleys[index+incr]=rS;
//Delete Findvalley Sat *
delete peaks; delete valleys;
delete rpeaks; delete tempvalleys;
delete cos2;
// Clustering ***************************************************************
//
minwater = new float**[lrvalleyi- ];
for (i = 0; i < Irvalleyi-1; i++) {
minwater[i] = new float*[lrvalleyh- 1];
for (j = 0; j < Irvalleyh-1; j++) {
minwater[i]U] = new float[lrvalleys-1];
}
}
/initialize minwater
for (i=0;i<lrvalleyi-I;i++)
for (j=0j<lrvalleyh- 1;j++)
for (l=0;l<lrvalleys-1;l++)
minwater[i]U][]=0.O;
1=0;
Lh=lrvalleyh-1; Ls=lrvalleys-1; Ll=lrvalleyi-1;
Npt = new int[Lh*Ll*Ls];
meanhls = new float*[3]; varhls = new float*[3];
for (i = 0; i < 3; i++) {
meanhls[i] = new float[Lh*Ll*Ls]; varhls[i]= new float[Lh*Ll*Ls];
}
/initialize meanhls, varhls
index=0;
for (i=0;i<3;i++)
for (j=j<(Lh*Ll*Ls)j++) {
meanhls[i][j]=0.0; varhls[i]U]=0.O;
if (i==0) {
Npt[index]=0; index++;
}
}
for (i=0;i<Ll;i++) {
for (jO=j<Lhj++) {
for (k=0;k<Ls;k++)
{
154
if ( (k==(Ls-1)) && (j!=(Lh-1)) && (i!=(L1-1)) ) {
index=O; sumh=0.0; suml=0.0; sums=0.0; sum2h=0.0; sum2=0.0; sum2s=0.0;
for (row=O;row<h;row++) {
for (col=O;col<w;col++) {
if ( (Int[row][col]>=(((rvalleyi[i]/rI)*rangeL)+minL)) &&
(Int[row][col]<(((rvalleyi[i+1]/rI)*rangeL)+minL)) &&
(Hue[row][col]>=(((rvalleyh[j]/rH)-O.5)*2*MPI)) &&
(Hue[row][col]<(((rvalleyh[j+1]/rH)-0.5)*2*MPI)) &&
(Sat[row][col]>=(((rvalleys[k]/rS)*rangeS)+minS)) &&
(Sat[row][col]<=(((rvalleys[k+1]/rS)*rangeS)+minS))) {
sumh+=Hue[row][col]; suml+=Int[row][col]; sums+=Sat[row][col];
sum2h+=pow(Hue[row][col],2); sum2l+=pow(Int[row][col],2);
sum2s+=pow(Sat[row][col],2);
index++;
}
}
}
} else if ( (k!=(Ls-1)) && (j==(Lh-1)) && (i!=(Ll-1))) {
index=O; sumh=0.0; suml=0.0; sums=0.0; sum2h=0.0; sum2l=0.0; sum2s=0.0;
for (row=O;row<h;row++)
for (col=O;col<w;col++)
if ( (Int[row][col]>=(((rvalleyi[i]/rI)*rangeL)+minL)) &&
(Int[row][col]<(((rvalleyi[i+1]/rl)*rangeL)+minL)) &&
(Hue[row][col]>=(((rvalleyhU]/rH)-0.5)*2*M PI)) &&
(Hue[row][col]<=(((rvalleyhU+1 ]/rH)-O.5)*2*MPI)) &&
(Sat[row][col]>=(((rvalleys[k]/rS)*rangeS)+minS)) &&
(Sat[row][col]<(((rvalleys[k+1]/rS)*rangeS)+minS))) {
sumh+=Hue[row][col]; suml+=Int[row][col]; sums+=Sat[row][col];
sum2h+=pow(Hue[row][col],2); sum2l+=pow(Int[row][col],2);
sum2s+=pow(Sat[row][col],2);
index++;
}
} else if ( (k!=(Ls-1)) && (j!=(Lh-1)) && (i==(L1-1))) {
index=O; sumh=0.0; suml=0.0; sums=0.0; sum2h=0.0; sum2l=0.0; sum2s=0.0;
for (row=O;row<h;row++)
for (col=O;col<w;col++)
if ( (Int[row][col]>=(((rvalleyi[i]/rI)*rangeL)+minL)) &&
(Int[row][col]<=(((rvalleyi[i+1]/rI)*rangeL)+minL)) &&
(Hue[row][col]>=(((rvalleyh[j]/rH)-0.5)*2*MPI)) &&
(Hue[row][col]<(((rvalleyh[j+1]/rH)-0.5)*2*MPI)) &&
(Sat[row][col]>=(((rvalleys[k/rS)*rangeS)+minS)) &&
(Sat[row][col]<(((rvalleys[k+1]/rS)*rangeS)+minS))) {
sumh+=Hue[row][col]; suml+=Int[row][col]; sums+=Sat[row][col];
sum2h+=pow(Hue[row][col],2); sum2l+=pow(Int[row][col],2);
sum2s+=pow(Sat[row][col],2);
index++;
}
} else if ((k==(Ls-1)) && (j==(Lh-l)) && (i!=(Ll-1)) ) {
index=O; sumh=0.0; suml=0.0; sums=0.0; sum2h=0.0; sum2l=0.0; sum2s=0.0;
for (row=O;row<h;row++)
for (col=O;col<w;col++)
if ( (Int[row][col]>=(((rvalleyi[i]/rI)*rangeL)+minL)) &&
(Int[row][col]<(((rvalleyi[i+1]/rI)*rangeL)+minL)) &&
155
(Hue[row][col]>=(((rvalleyh[j]/rH)-0.5)*2*MPI)) &&
(Hue[row][col]<=(((rvalleyh[j+1]/rH)-0.5)*2*MPI)) &&
(Sat[row][col]>=(((rvalleys[k]/rS)*rangeS)+minS)) &&
(Sat[row][col]<=(((rvalleys[k+ I ]/rS)*rangeS)+minS)) )
{
sumh+=Hue[row][col]; suml+=Int[row][col]; sums+=Sat[row][col];
sum2h+=pow(Hue[row][col],2); sum2l+=pow(Int[row][col],2);
sum2s+=pow(Sat[row][col],2);
index++;
}
} else if ( (k==(Ls-1)) && (j!=(Lh-1)) && (i==(Ll-1))) {
index=O; sumh=0.0; suml=0.0; sums=0.0; sum2h=0.0; sum2l=0.0; sum2s=0.0;
for (row=O;row<h;row++)
for (col=O;col<w;col++)
if ( (Int[row][col]>=(((rvalleyi[i]/rI)*rangeL)+minL)) &&
(Int[row][col]<=(((rvalleyi[i+I]/rI)*rangeL)+minL)) &&
(Hue[row][col]>=(((rvalleyhU]/rH)-O.5)*2*MPI)) &&
(Hue[row][col]<(((rvalleyh[j+1]/rH)-0.5)*2*MPI)) &&
(Sat[row][col]>=(((rvalleys[k]/rS)*rangeS)+minS)) &&
(Sat[row][col]<=(((rvalleys[k+1]/rS)*rangeS)+minS)))
{
sumh+=Hue[row][col]; suml+=Int[row][col]; sums+=Sat[row][col];
sum2h+=pow(Hue[row][col],2); sum2l+=pow(Int[row][col],2);
sum2s+=pow(Sat[row][col],2);
index++;
}
} else if ( (k!=(Ls-1)) && (j==(Lh-1)) && (i==(Ll-1))) {
index=O; sumh=0.0; suml=0.0; sums=0.0; sum2h=0.0; sum2l=0.0; sum2s=0.0;
for (row=O;row<h;row++)
for (col=O;col<w;col++)
if ( (Int[row][col]>=(((rvalleyi[i]/rI)*rangeL)+minL)) &&
(Int[row][col]<=(((rvalleyi[i+1]/rI)*rangeL)+minL)) &&
(Hue[row][col]>=(((rvalleyh]/rH)-O.5)*2*M PI)) &&
(Hue[row][col]<=(((rvalleyh[j+1]/rH)-0.5)*2*MPI)) &
(Sat[row][col]>=(((rvalleys[k]/rS)*rangeS)+minS)) &&
(Sat[row][col]<(((rvalleys[k+1]/rS)*rangeS)+minS))) {
sumh+=Hue[row][col]; suml+=Int[row][col]; sums+=Sat[row][col];
sum2h+=pow(Hue[row][col],2); sum2l+=pow(Int[row][col],2);
sum2s+=pow(Sat[row][col],2);
index++;
}
} else if ( (k==(Ls-1)) && (j==(Lh-1)) &&(i==(Ll-1))) {
index=O; sumh=0.0; suml=0.0; sums=0.0; sum2h=0.0; sum2l=0.0; sum2s=0.0;
for (row=O;row<h;row++)
for (col=O;col<w;col++)
if ( (Int[row][col]>=(((rvalleyi[i]/rl)*rangeL)+minL)) &&
(Int[row][col]<=(((rvalleyi[i+1 ]/rI)*rangeL)+minL)) &&
(Hue[row][col]>=(((rvalleyh[j]/rH)-0.5)*2*MPI)) &&
(Hue[row][col]<=(((rvalleyhUj1]/rH)-0.5)*2*MPI)) &&
(Sat[row][col]>=(((rvalleys[k]/rS)*rangeS)+minS)) &&
(Sat[row][col]<=(((rvalleys[k+]/rS)*rangeS)+minS)))
{
sumh+=Hue[row][col]; suml+=Int[row][col]; sums+=Sat[row][col];
sum2h+=pow(Hue[row][col],2); sum2l+=pow(Int[row][col],2);
sum2s+=pow(Sat[row][col],2);
156
index++;
}
} else {
index=0; sumh=0.0; suml=0.0; sums=0.0; sum2h=0.0; sum2l=0.0; sum2s=0.0;
for (row=0;row<h;row++)
for (col=O;col<w;col++)
if ( (Int[row][col]>=(((rvalleyi[i]/rI)*rangeL)+minL)) &&
(Int[row][col]<(((rvalleyi[i+1]/rI)*rangeL)+minL)) &&
(Hue[row][col]>=(((rvalleyh[j]/rH)-0.5)*2*MPI)) &&
(Hue[row][col]<(((rvalleyh[j+1 ]/rH)-0.5)*2*MPI)) &&
(Sat[row][col]>=(((rvalleys[k]/rS)*rangeS)+minS)) &&
(Sat[row][col]<(((rvalleys[k+1]/rS)*rangeS)+minS))) {
sumh+=Hue[row][col]; suml+=Int[row][col]; sums+=Sat[row][col];
sum2h+=pow(Hue[row][col],2); sum2l+=pow(Int[row][col],2);
sum2s+=pow(Sat[row][col],2);
index++;
}
}
// Compute the mean and variance
Npt[l]=index;
if (index > 0.0) {
meanhls[O][1]=(sumh/(float)(index));
meanhls[1][1]=(suml/(float)(index));
meanhls[2][l]=(sums/(float)(index));
varhls[0][l]=(sum2h/(float)(index))-pow(meanhls[0][],2);
varhls[1][l]=(sum2l/(float)(index))-pow(meanhls[1][1],2);
varhls[2][l]=(sum2s/(float)(index))-pow(meanhls[2][1],2);
}
// Compute the normalize variance and assign to the minwater
if (Npt[] <(0.003*h*w))
minwater[i]U][k]=-1;
else {
sizeL=fabs((((rvalleyi[i]/rl)*rangeL)+minL)-(((rvalleyi[i+1]/rI)*rangeL)+minL));
sizeH=fabs((((rvalleyh[j]/rH)-0.5)*2.0*MPI)-(((rvalleyh[j+1]/rH)-0.5)*2.0*MPI));
sizeS=fabs((((rvalleys[k]/rS)*rangeS)+minS)-(((rvalleys[k+1 ]/rS)*rangeS)+minS));
minwater[i][j][k]=sqrt(pow(varhls[0][1],2)+pow(varhls[1][1],2)+pow(varhls[2][1],2))*
(Npt[l]/(float)(h*w))*((rangeL*rangeH*rangeS)/(sizeL*sizeH*sizeS));
}
1++;
}
}
}
1=1;
minindex=30;
minwater2 = new int**[lrvalleyi-1]; // The minimum or root nodes in IHS space
minimumw = new int**[lrvalleyi-1]; // The direction that assign to the branch node in IHS space
for (i = 0; i < Irvalleyi-1; i++) {
minwater2[i] = new int* [lrvalleyh-1];
minimumw[i] = new int*[lrvalleyh-1];
for (j = 0; j < Irvalleyh-1; j++) {
minwater2[i][j] = new int[lrvalleys-1];
minimumw[i]U] = new int[lrvalleys-1];
}
157
}
/ initialize minwater2 & minimumw
for (i=0;i<lrvalleyi-1;i++)
for (j=Oj<lrvalleyh- 1j++)
for (k=O;k<lrvalleys- 1;k++) {
minwater2[i][][k]=0;
minimumw[i]j][k]=O;
}
// Assign the direction of branch node from the normalize variance
for (i=O;i<Ll;i++) {
for (j=Oj<Lhj++) {
for (k=O;k<Ls;k++) {
if ((lrvalleyi>2)&&(lrvalleys>2)) {
// bottom level
if ( (1>=1) && (l<=Ls)) {
if (1==1)
minwater2[i]U][k]=suml(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,l,Lh,L1,Ls);
else if (1==Ls)
minwater2[i]U][k]=sumLs(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,,Lh,L1,Ls);
else
minwater2[i][j][k]=sumbedgeright(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,1,
Lh,Ll,Ls);
}
if ( (l>=Ls+1) && (l<=((Lh*Ls)-Ls))) {
if (finod(,Ls)== 1)
minwater2[i]U][k]=sum bedge back(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,1,
Lh,Ll,Ls);
else if (finod(,Ls)==0)
minwater2[i][j][k]=sum-bedgefront(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,1,
Lh,L1,Ls);
else
minwater2[i]U][k]=sum bottommiddle(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,l,
Lh,L1,Ls);
}
if ( (l>=((Lh*Ls)-Ls)+1) && (l<=(Lh*Ls))) {
if (l==((Lh*Ls)-Ls+1))
minwater2[i]U][k]=sum bcornerback(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,1,
Lh,L1,Ls);
else if (l==(Lh*Ls))
minwater2[i]U][k]=sum-bcomerfront(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,l,
Lh,L1,Ls);
else
minwater2[i]U][k]=sum bedgeleft(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,1,
Lh,L1,Ls);
}
//levelj
if ( (l>=((i*Lh*Ls)+1)) && (l<=((i*Lh*Ls)+Ls)) && ((i+1)>l) && ((i+1)<(lrvalleyi-1)))
if (l==((i*Lh*Ls)+1))
minwater2[i]U][k]=sumil (minwater,minimumw,rvalleyh,rvalleyi,rvalleys,l,Lh,L1,Ls);
else if (l==((i*Lh*Ls)+Ls))
minwater2[i][j][k]=sumils(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,,Lh,L,Ls);
else
minwater2[i][][k]=sum-iedge _right(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,l,
Lh,Ll,Ls);
{
158
}
if ( (l>=((i*Lh*Ls)+Ls+ 1)) && (1<=(((i+1)*Lh*Ls)-Ls)) && ((i+1)> 1) && ((i+1)<(lrvalleyi- 1)))
if (finod(,Ls)==1)
{
//'sumledgeBack'
minwater2[i][j][k]=sum-iedgeback(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,1,
Lh,LL,Ls);
else if (fmiod(l,Ls)==O)
//'sumledgeFront'
minwater2[i][][k]=sum-iedgefront(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,1,
Lh,L1,Ls);
else
//'sumLeveliMiddle'
minwater2[i]U][k]=sum-leveli-middle(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,l,
Lh,Ll,Ls);
}
if ( (l>=(((i+l)*Lh*Ls)-Ls+l)) && (l<=((i+1)*Lh*Ls)) && ((i+1)>1) && ((i+l)<(Irvalleyi-1)))
if (l==(((i+1)*Lh*Ls)-Ls+1))
{
minwater2[i]U][k]=sum icornerback(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,1,
Lh,L1,Ls);
else if (1==((i+1)*Lh*Ls))
minwater2[i][j][k]=sum icornerfront(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,l,
Lh,Ll,Ls);
else
minwater2[i][][k]=sumiedgeleft(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,l,
Lh,Ll,Ls);
}
/top level
if ( (l>=((i*Lh*Ls)+1)) && (l<=((i*Lh*Ls)+Ls)) && ((i+1)==(Irvalleyi-1)))
if (l==(i*Lh*Ls)+1)
{
minwater2[i]U][k]=sumtopl(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,,Lh,L,Ls);
else if (l==((i*Lh*Ls)+Ls))
minwater2[i]U][k]=sumtopls(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,,Lh,L1,Ls);
else
minwater2[i][][k]=sum-tedgeright(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,l,
Lh,L1,Ls);
}
if ( (l>=((i*Lh*Ls)+Ls+ 1)) && (l<=(((i+1)*Lh*Ls)-Ls)) && ((i+1)==(lrvalleyi-1)))
if (fimod(,Ls)== 1)
{
minwater2[i][j][k]=sum tedge _back(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,1,
Lh,L1,Ls);
else if (fimod(l,Ls)==O)
minwater2[i][][k]=sum tedge-front(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,1,
Lh,L1,Ls);
else
minwater2[i]U][k]=sum topmiddle(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,l,
Lh,L1,Ls);
}
if ( (1>=(((i+ 1)*Lh*Ls)-Ls+ 1)) && (l<=((i+ 1)*Lh*Ls)) && ((i+1)==(rvalleyi- 1)))
if (l==(((i+1)*Lh*Ls)-Ls+1))
{
minwater2[i]U][k]=sum tcomerback(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,1,
Lh,L1,Ls);
else if (l==((i+1)*Lh*Ls))
minwater2[i]U][k]=sum tcomerfront(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,l,
Lh,Ll,Ls);
else
159
minwater2[i][j][k]=sum-tedgeleft(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,l,
Lh,Ll,Ls);
}
} else if ((Irvalleyi==2)&&(Irvalleys==2)) {
/ If there is only 1 Level
if ( (1>=1) && (l<=Ls) ) {
minwater2[i][j][k]=sumRightEdge(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,l,Lh,Ll,Ls);
}
if ( (l>=Ls+1) && (l<=((Lh*Ls)-Ls))) {
minwater2[i][j][k]=sumLeftRight(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,,Lh,L1,Ls);
}
if( (l>=((Lh*Ls)-Ls)+l) && (l<=(Lh*Ls))) {
minwater2[i][][k]=sumLeftEdge(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,,Lh,L,Ls);
}
} else if ((lrvalleyi>2)&&(lrvalleys==2)) {
// If Ls == 1
// Bottom or level 0
if( 1==l )
minwater2[i][j][k]=LS 1_(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,l,Lh,Ll,Ls);
if ( (1>=2) && (l<=(Lh-1)) )
minwater2[i][U][k]=LS lmiddle(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,,Lh,L1,Ls);
if( l==Lh )
minwater2[i]U][k]=LS lls(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,1,Lh,L,Ls);
/ levelj
if ( (l==((i*Lh)+ 1)) && ((i+ 1)> 1) && ((i+l1)<(lrvalleyi-1)))
minwater2[i][j][k]=LS lil (minwater,minimumw,rvalleyh,rvalleyi,rvalleys,,Lh,L1,Ls);
if ( (l>=((i*Lh)+2)) && (l<=(((i+1)*Lh)-l)) && ((i+1)>1) && ((i+1)<(Irvalleyi-1)) )
minwater2[i][U][k]=LS 1imiddle(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,,Lh,L1,Ls);
if ( (l==((i+1)*Lh)) && ((i+1)>1) && ((i+1)<(lrvalleyi-1)) )
minwater2[i]U][k]=LS _ils(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,,Lh,L,Ls);
/top level
if ( (1==((i*Lh*Ls)+ 1)) && ((i+ 1)==(Irvalleyi- 1)))
minwater2[i][U][k]=LS Itop(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,,Lh,L1,Ls);
if ( (l>=((i*Lh)+2)) && (1<=(((i+l)*Lh)-l)) && ((i+l)==(lrvalleyi-1)) )
minwater2[i]j][k]=LS ltopmiddle(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,l,
Lh,L1,Ls);
if ( (1==((i+1)*Lh*Ls)) && ((i+1)==(lrvalleyi-1)))
minwater2[i][j][k]=LS ltopls(minwater,minimumw,rvalleyh,rvalleyi,rvalleys,,Lh,L,Ls);
}
1=1+1;
}
}
}
II Assign the distinct clusters with different labels in 3D array of minimumw according to the minimum
// or root node (minimumw) and the direction of branch node of minwater2.
for (i=0;i<Ll;i++) {
for (j=0j<Lhj++) {
for (k=O;k<Ls;k++) {
if( minwater2[i][j][k]>0) {
areal=0;
index=0;
for (1=0;1<10;1++) {
tx[l]=-1; ty[l]=-1; tz[l]=-1;
160
}
tx[index]=i; / i
ty[index]=j; // h
tz[index]=k; // s
terminate=true; // not reach the minimum
while (terminate) {
if ( (minwater2[tx[index]][ty[index]][tz[index]] >= 30)||
(minimumw[tx[index]][ty[index]][tz[index]]!=0)) {
terminate=false;
if (minwater2[tx[index]][ty[index]][tz[index]]>=30)
area I=minwater2[tx[index]][ty[index]][tz[index]];
else if (minimumw[tx[index]][ty[index]][tz[index]]!=O)
areal=minimumw[tx[index]][ty[index]][tz[index]];
} else {
switch (minwater2[tx[index]][ty[index]][tz[index]]) {
case I :
index++; tx[index]=tx[index-1];
ty[index]=ty[index-l]-l;
if (ty[index]<0) ty[index]=(Lh-1); else if (ty[index]>(Lh-1))
tz[index]=tz[index-l]-1; break;
case 2 :
index++; tx[index]=tx[index- I];
ty[index]=ty[index-l]-1;
if (ty[index]<0) ty[index]=(Lh-1); else if (ty[index]>(Lh-I))
tz[index]=tz[index-I]; break;
case 3 :
index++; tx[index]=tx[index-1];
ty[index]=ty[index-l]-l;
if (ty[index]<0) ty[index]=(Lh-1); else if (ty[index]>(Lh-1))
tz[index]=tz[index-1]+1; break;
case 4 :
index++; tx[index]=tx[index-1];
ty[index]=zty[index- 1];
if (ty[index]<0) ty[index]=(Lh-1); else if (ty[index]>(Lh-1))
tz[index]=tz[index-1]+1; break;
case 5 :
index++; tx[index]=tx[index-1];
ty[index] =ty[index-I]+ 1;
if (ty[index]<0) ty[index]=(Lh- 1); else if (ty[index]>(Lh- 1))
tz[index]=tz[index-l]+l; break;
case 6 :
index++; tx[index]=tx[index-1];
ty[index] =ty[index-1 ]+1;
if (ty[index]<0) ty[index]=(Lh-1); else if (ty[index]>(Lh-1))
tz[index]=tz[index-1]; break;
case 7 :
index++; tx[index]=tx[index-1];
ty[index]=ty[index- 1]+1;
if (ty[index]<0) ty[index]=(Lh- 1); else if (ty[index]>(Lh- 1))
tz[index]=tz[index-1]-1; break;
case 8 :
index++; tx[index]=tx[index-1];
ty [index]=ty [index- I];
if (ty[index]<0) ty[index]=(Lh- 1); else if (ty[index]>(Lh- 1))
tz[index]=tz[index-l]-l; break;
case 9 :
ty[index]=0;
ty[index]=0;
ty[index]=0;
ty[index]=0;
ty[index]=0;
ty[index]=0;
ty[index]=0;
ty[index]=0;
161
index++; tx[index]=tx[index-11-1;
ty[index]=ty[index-1 ]-1;
if (ty[index]<0) ty[index]=(Lh-1); else
tz[index]=tz[index-1]-1; break;
case 10 :
index++; tx[index]=tx[index-1]-1;
ty[index]=ty[index- 1]- 1;
if (ty[index]<0) ty[index]=(Lh-1); else
tz[index]=tz[index-1]; break;
case 11:
index++; tx[index]=tx[index-1 ]- 1;
ty[index]=ty[index-l ]-1;
if (ty[index]<O) ty[index]=(Lh- 1); else
tz[index]=tz[index-1]+1; break;
case 12:
index++; tx[index]=tx[index-1]-1;
ty[index]=ty[index- 1];
if (ty[index]<0) ty[index]=(Lh- 1); else
tz[index]-tz[index-1]+1; break;
case 13 :
index++; tx[index]=tx[index-1]-1;
ty[index]=ty[index- 1]+ 1;
if (ty[index]<0) ty[index]=(Lh-1); else
tz[index]=tz[index-11+1; break;
case 14 :
index++; tx[index]=tx[index-1]-1;
ty [index] =ty [index-1I]+ 1;
if (ty[index]<0) ty[index]=(Lh- 1); else
tz[index]=tz[index-1]; break;
case 15 :
index++; tx[index]=tx[index- 1]-1;
ty[index]=ty[index- 1]+1;
if (ty[index]<0) ty[index]=(Lh-1); else
tz[index]=tz[index-1]-1; break;
case 16 :
index++; tx[index]=tx[index-1 ]-1;
ty[index]=ty[index- 1];
if (ty[index]<0) ty[index]=(Lh-1); else
tz[index]=tz[index-1]-l; break;
case 17 :
index++; tx[index]=tx[index-11-1;
ty[index]=ty[index-1];
if (ty[index]<0) ty[index]=(Lh- 1); else
tz[index]=tz[index-1]; break;
case 18 :
index++; tx[index]-tx[index-1]+1;
ty[index]=ty[index-1]- 1;
if (ty[index]<0) ty[index]=(Lh-1); else
tz[index]=tz[index-1]-1; break;
case 19 :
index++; tx[index]=tx[index-1]+1;
ty[index]=ty[index- ]-1;
if (ty[index]<0) ty[index]=(Lh-1); else
tz[index]=tz[index- 1]; break;
case 20 :
index++; tx[index]=tx[index-1]+1;
if (ty[index]>(Lh-1)) ty[index]=0;
if (ty[index]>(Lh-1)) ty[index]=0;
if (ty[index]>(Lh- 1)) ty[index]=0;
if (ty[index]>(Lh- 1)) ty[index]=0;
if (ty[index]>(Lh-1)) ty[index]=0;
if (ty[index]>(Lh- 1)) ty[index]=0;
if (ty[index]>(Lh-1)) ty[index]=0;
if (ty[index]>(Lh-1)) ty[index]=0;
if (ty[index]>(Lh- 1)) ty[index]=0;
if (ty[index]>(Lh-1)) ty[index]=0;
if (ty[index]>(Lh-1)) ty[index]=0;
162
ty[index] =ty[index- 1]-1;
if (ty[index]<0) ty[index]=(Lh- 1);
tz[index]=tz[index-1]+1; break;
case 21:
index++; tx[index]=tx[index-1]+1;
ty[index]=ty[index- 1];
if (ty[index]<0) ty[index]=(Lh- 1);
tz[index]=tz[index-1]+1; break;
case 22 :
index++; tx[index]-tx[index-1]+1;
ty[index]=ty[index- 1]+1;
if (ty[index]<0) ty[index]=(Lh-1);
tz[index]=tz[index-1]+1; break;
case 23 :
index++; tx[index]=tx[index-1]+1;
ty[index]=ty[index- 1]+1;
if (ty[index]<0) ty[index]=(Lh- 1);
tz[index]=tz[index-1]; break;
case 24 :
index++; tx[index]=tx[index-1]+1;
ty[index]==ty[index- I ]+1;
if (ty[index]<0) ty[index]=(Lh- 1);
tz[index]=tz[index-1]-1; break;
case 25 :
index++; tx[indexj=tx[index-1]+1;
ty[index]=ty[index- 1];
if (ty[index]<0) ty[index]=(Lh- 1);
tz[index]=tz[index- 1]-1; break;
case 26 :
index++; tx[index]=tx[index-1]+1;
ty[index]=,ty[index- 1];
if (ty[index]<0) ty[index]=(Lh-1);
tz[index]=tz[index-1]; break;
}
}
else if (ty[index]>(Lh- 1)) ty[index]=0;
else if (ty[index]>(Lh- 1)) ty[index]=0;
else if (ty[index]>(Lh-1)) ty[index]=0;
else if (ty[index]>(Lh- 1)) ty[index]=0;
else if (ty[index]>(Lh- 1)) ty[index]=0;
else if (ty[index]>(Lh-1)) ty[index]=0;
else if (ty[index]>(Lh-1)) ty[index]=0;
}
1=0;
while ( ((tx[l]>=0) 1 (ty[l]>=0) 1 (tz[l]>=0)) && (1<10)) {
minimumw[tx[1]][ty[l]][tz[l]]=areal; 1++;
}
}
}
}
}
/Display the segmentation result on the Imagel *
// rangelndex : the number of distinct classes obtained from the clustering in IHS space
rangelndex=minindex-29; incr2=255.0/rangelndex;
tempI = "Resulting segmentation with "+ IntToStr(rangelndex) + " regions";
Label5->Caption-temp 1;
for (i=0;i<rangelndex;i++) {
for =0=j<Llj++) {
for (k=0;k<Lh;k++) {
for (l=0;l<Ls;l++) {
if (minimumw][k][1]==(29+(i+l))) {
for (row=0;row<h;row++)
163
for (col=O;col<w;col++) {
if ( (Int[row][col]>=(((rvalleyilj]/rI)*rangeL)+minL)) &&
(Int[row][col]<=(((rvalleyij+1]/rI)*rangeL)+minL)) &&
(Hue[row][col]>=(((rvalleyh[k]/rH)-O.5)*2*MPI) ) &&
( Hue[row][col]<= (((rvalleyh[k+I]/rH)-0.5)*2*MPI) ) &&
( Sat[row][col]>=(((rvalleys[l]/rS)*rangeS)+minS) ) &&
( Sat[row][col]<=(((rvalleys[l+1]/rS)*rangeS)+minS))) {
Image5->Canvas->Pixels[col][row] = (TColor)RGB(incr2*(i+1),incr2*(i+1),incr2*(i+1));
}
}
}
}
}
}
}
// Deallocate Clustering variables *
delete Npt;
for (i=0;i<3;i++) {
delete[] meanhls[i]; delete[] varhls[i];
}
delete[] meanhls; delete[] varhis;
for (i=O;i<(lrvalleyi-1);i++) {
for (j0j<(rvalleyh-1)j++)
delete[] minwater[i]U];
delete[] minwater[i];
}
delete[] minwater;
for (i=0;i<(rvalleyi-1);i++) {
for (j=Qj<(Irvalleyh-I)j++)
delete[] minwater2[i]U];
delete[] minwater2[i];
}
delete[] minwater2;
}
/------------------------------------------------------------------------The directions of each node:
//The following codes are used to identify the direction of each node from their normalized variance values
// For a particular node, if there is not any suitable neighborhood node assigned for the direction of the
// smallest normalized variance, that particular node will be specified as a new distinct class.
// Note: only the function for finding the direction of the node in the middle of level i is given here; since
the functions for finding the direction of the other node can be modified according to their
//
neighbors nodes.
int sumlevelimiddle(float ***minwater, int ***minimumw, int *rvalleyh, int *rvalleyi, int *rvalleys,
int 1, int Lh, int Ll, int Ls)
{
bool empty=true;
int i, j, row, col, count, ptc=27, suml, tindex, twater min, range1, range2;
float temp=0.0;
int tl[27], th[27], ts[27];
int index[27];
float twater[27], *trll, *trl2, *trhl, *trh2, *trsl, *trs2;
164
trIl
=
trhl
=
trsl
new float[2]; trl2 = new float[2];
new float[2]; trh2
new float[2]; trs2
=
=
new float[2];
new float[2];
index[O]=1-Ls-1-1;
index[1 ]=1-Ls-1;
index[2]=l-Ls+ 1-1;
index[3]=1+1-1;
index[4]=+Ls+ 1-1;
index[5]=l+Ls-1;
index[6]=+Ls- 1-1;
index[7]=- 1-1;
index[8]=-(Lh*Ls)-Ls1 -1;
index[9]=1-(Lh*Ls)-Ls- 1;
index[ 10]=-(Lh*Ls)-Ls+1 -1;
index[1]=-(Lh*Ls)+L+-1;
index[ 1 2]=1-(Lh*Ls)+Ls+ -1;
index[13]=-(Lh*Ls)+Ls-1;
index[ 14]=l-(Lh*Ls)+Ls1-1;
index[ 15]=I-(Lh*Ls)-I-1;
index[ 16]=l-(Lh*Ls)-1;
index[ 1 7]=1+(Lh*Ls)-Ls- -1;
index[ 19]=+(Lh*Ls)-Ls-1;
index[I9]=1+(Lh*Ls)-Ls+1-1;
index[20]=+(Lh*Ls)+I -1;
index[2I]=1+(Lh*Ls)+Ls+I-1;
index[22]=l+(Lh*Ls)+Ls-1;
index[23 ]=I+(Lh*Ls)+Ls- I- 1;
index[24]=1+(Lh*Ls)1 -1;
index[25]=+(Lh*Ls)- 1;
index[26]=- 1;
for (i=0;i<ptc;i++) {
tl[i]=floor(index[i]/((float)Lh*Ls));
th[i]=finod(floor(index[i]/((float)Ls))+1,Lh)-1; if (th[i]==-1) th[i]=(Lh- 1);
ts[i]=fmod((index[i]+1),Ls)-1; if (ts[i]==-1) ts[i]=(Ls-1);
twater[i]=minwater[tl[i]][th[i]][ts[i]];
}
suml=O; tindex=1;
while ((suml==O) && (tindex<ptc)) {
empty=true;
for (i=O;i<ptc;i++) {
if (i==O) {
j=1; temp=twater[i];
while ((temp<0.0000000000000000000000000000000000000001) && (j<ptc))
temp-twaterj]; j++;
}
if (temp>0.0000000000000000000000000000000000000001) {
twater-min=(j-1); empty=false;
{
}
} else {
if ((temp>twater[i]) && (twater[i]>0.0000000000000000000000000000000000000001))
temp=twater[i];
twater-min=i;
{
165
empty=false;
}
}
}
if (empty)
suml=-1;
count=0;
for (i=;i<ptc;i++)
if (fabs(twater[i]-temp)<0.0000000000000000000000000000000000000000000 1) count++;
if (count>1)
suml=-1;
else {
switch (twater min) {
case 0 :
Lnormal(rvalleyi,trll,tl[ptc-1]); Lnormal(rvalleyi,trl2,tl[twater-min]);
HLeft(rvalleyh,trhl,th[ptc-1]); HRight(rvalleyh,trh2,th[twater min]);
SFront(rvalleys,trsl,ts[ptc-1]); SBack(rvalleys,trs2,ts[twatermin]);
range 1=0; range2=0;
for (row=0;row<h;row++)
for (col=O;col<w;col++) {
if ( (Int[row][col]>=trll [0]) && (Int[row][col]<=trl 1 [1]) &&
(Hue[row][col]>=trhl [0]) && (Hue[row][col]<=trhl [1]) &&
(Sat[row][col]>=trsl [0]) && (Sat[row][col]<=trs [1]) )
range I ++;
if ( (Int[row][col]>=trl2[0]) && (Int[row][col]<=trl2[1]) &&
(Hue[row][col]>=trh2[0]) && (Hue[row][col]<=trh2[1]) &&
(Sat[row][col]>=trs2[0]) && (Sat[row][col]<=trs2[1]))
range2++;
}
if ((rangel!=0) && (range2!=0)) suml=1; else twater[twater-min]=-1;
break;
case 1 :
Lnormal(rvalleyi,trll,tl[ptc-1]); Lnormal(rvalleyi,trl2,tl[twater min]);
HLeft(rvalleyh,trhl,th[ptc-1]); HRight(rvalleyh,trh2,th[twatermin]);
Snormal(rvalleys,trsl,ts[ptc-1]); Snormal(rvalleys,trs2,ts[twater-min]);
range 1=0; range2=0;
for (row=0;row<h;row++)
for (col=0;col<w;col++) {
if ( (Int[row][col]>=trl 1 [0]) && (Int[row][col]<=trl 1 [1]) &&
(Hue[row][col]>=trhl [0]) && (Hue[row][col]<=trhl [1]) &&
(Sat[row][col]>=trs 1 [0]) && (Sat[row][col]<=trs [1]) )
range 1++;
if ( (Int[row] [col]>-trl2 [0]) && (Int[row][col]<=trl2[1]) &&
(Hue[row][col]>=trh2[0]) && (Hue[row][col]<=trh2[1]) &&
(Sat[row][col]>=trs2[0]) && (Sat[row][col]<=trs2[1]))
range2++;
}
if ((rangel!=0) && (range2!=0)) suml=2; else twater[twater-min]=-1;
break;
case 2
Lnormal(rvalleyi,trll,tl[ptc-1]); Lnormal(rvalleyi,trl2,tl[twater-min]);
166
HLeft(rvalleyh,trhl,th[ptc-1]); HRight(rvalleyh,trh2,th[twater min]);
SBack(rvalleys,trsl,ts[ptc-1]); SFront(rvalleys,trs2,ts[twater min]);
range 1=0; range2=0;
for (row=0;row<h;row++)
for (col=O;col<w;col++) {
if ( (Int[row][col]>=trll [0]) && (Int[row][col]<=trll [1]) &&
(Hue[row][col]>=trhl[0]) && (Hue[row][col]<=trhl[1]) &&
(Sat[row][col]>=trs 1 [0]) && (Sat[row][col]<=trsl [1]) )
range 1++;
if ( (Int[row][col]>=trl2[0]) && (Int[row][col]<=trl2[1]) &&
(Hue[row] [col]>=trh2 [0]) && (Hue[row][col]<=trh2[1]) &&
(Sat[row][col]>=trs2[0]) && (Sat[row][col]<=trs2[1]))
range2++;
}
if ((rangel!=0) && (range2!=0)) suml=3; else twater[twater-min]=-1;
break;
case 3 :
Lnormal(rvalleyi,trll,tl[ptc-1]); Lnormal(rvalleyi,trl2,tl[twater min]);
Hnormal(rvalleyh,trhl,th[ptc-1]); Hnormal(rvalleyh,trh2,th[twater min]);
SBack(rvalleys,trs1,ts[ptc-1]); SFront(rvalleys,trs2,ts[twater min]);
range I =0; range2=0;
for (row=0;row<h;row++)
for (col=0;col<w;col++) {
if ( (Int[row][col]>=trll [0]) && (Int[row][col]<-trll [1]) &&
(Hue[row][col]>=trhl[0]) && (Hue[row][col<=trhl[1]) &&
(Sat[row][col]>=trsl [0]) && (Sat[row][col]<=trs [1]))
rangeI++;
if ( (Int[row][col]>=trl2[0]) && (Int[row][col]<=trl2[1]) &&
(Hue[row][col]>=trh2[0]) && (Hue[row][col]<=trh2[1]) &&
(Sat[row][col]>=trs2[0]) && (Sat[row][col]<-trs2[1]))
range2++;
}
if ((range 1!=0) && (range2!=0)) suml=4; else twater[twater min]=- 1;
break;
case 4 :
Lnormal(rvalleyi,trll,tl[ptc-1]); Lnormal(rvalleyi,trl2,tl[twater min]);
HRight(rvalleyh,trhl,th[ptc-1]); HLeft(rvalleyh,trh2,th[twater-min]);
SBack(rvalleys,trsl,ts[ptc-1]); SFront(rvalleys,trs2,ts[twater-min]);
range 1=0; range2=0;
for (row=0;row<h;row++)
for (col=0;col<w;col++) {
if ( (Int[row][col]>=trll [0]) && (Int[row][col]<=trll [1]) &&
(Hue[row][col]>=trhl [0]) && (Hue[row][col]<=trhl [1]) &&
(Sat[row][col]>=trs 1 [0]) && (Sat[row][col]<-trs 1[1]) )
rangeI++;
if ( (Int[row][col]>=trl2[0]) && (Int[row][col]<=trl2[1]) &&
(Hue[row][col]>-=trh2[0]) && (Hue[row][col]<=trh2[1]) &&
(Sat[row][col]>=trs2[0]) && (Sat[row][col]<=trs2[1]))
range2++;
}
167
if ((range 1 !=0) && (range2!=0)) suml=5; else twater[twater min]=- 1;
break;
case 5 :
Lnormal(rvalleyi,trll,tI[ptc-1]); Lnormal(rvalleyi,trl2,tl[twater min]);
HRight(rvalleyh,trhl,th[ptc-1]); HLeft(rvalleyh,trh2,th[twater-min]);
Snormal(rvalleys,trsl,ts[ptc-1]); Snormal(rvalleys,trs2,ts[twater-min]);
range 1=0; range2=0;
for (row=0;row<h;row++)
for (col=0;col<w;col++) {
if ( (Int[row][col]>-trl1 [0]) && (Int[row][col]<=trl I[1]) &&
(Hue[row][col]>=trhl [0]) && (Hue[row][col]<=trhl [1]) &&
(Sat[row][col]>=trs 1 [0]) && (Sat[row][col]<=trs [1]) )
range 1++;
if ( (Int[row][col]>=trl2[0]) && (Int[row] [col]<=trl2[1]) &&
(Hue[row][col]>=trh2[0]) && (Hue[row][col]<=trh2[1]) &&
(Sat[row][col]>=trs2[0]) && (Sat[row][col]<=trs2[1]))
range2++;
}
if ((range l!=0) && (range2!=0)) suml=6; else twater[twater-min]=-1;
break;
case 6:
Lnormal(rvalleyi,trll,tl[ptc-1]); Lnormal(rvalleyi,trl2,tl[twater min]);
HRight(rvalleyh,trhl,th[ptc-1]); HLeft(rvalleyh,trh2,th[twater min]);
SFront(rvalleys,trsl,ts[ptc-1]); SBack(rvalleys,trs2,ts[twater-min]);
range 1=0; range2=0;
for (row=0;row<h;row++)
for (col=0;col<w;col++) {
if ( (Int[row][col]>=trll [0]) && (Int[row][col]<=trll [1]) &&
(Hue[row][col]>=trh1 [0]) && (Hue[row][col]<=trhl [1]) &&
(Sat[row][col]>=trs 1 [0]) && (Sat[row][col]<=trs [1]) )
range I ++;
if ( (Int[row][col]>=trl2[0]) && (Int[row][col]<=trl2[1]) &&
(Hue[row][col]>=trh2[0]) && (Hue[row][col]<=trh2[1]) &&
(Sat[row][col]>=trs2[0]) && (Sat[row][col]<=trs2[1]))
range2++;
}
if ((rangel!=0) && (range2!=0)) suml=7; else twater[twater-min]=-1;
break;
case 7 :
Lnormal(rvalleyi,trll,tl[ptc-1]); Lnormal(rvalleyi,trl2,tl[twater-min]);
Hnormal(rvalleyh,trhl,th[ptc-1]); Hnormal(rvalleyh,trh2,th[twater min]);
SFront(rvalleys,trsl,ts[ptc-1]); SBack(rvalleys,trs2,ts[twater min]);
range I =0; range2=0;
for (row=0;row<h;row++)
for (col=0;col<w;col++) {
if ( (Int[row][col]>=trl I[0]) && (Int[row][col]<=trl 1 [1]) &&
(Hue[row] [col]> =trhl [0]) && (Hue[row][col]<=trh1[1]) &&
(Sat[row][col]>=trsl [0]) && (Sat[row][col]<=trsl [1]))
range 1++;
if ( (Int[row][col]>=trl2[0]) && (Int[row][col]<-trl2[1]) &&
(Hue[row][col]>=trh2[0]) && (Hue[row][col]<=trh2[1]) &&
168
(Sat[row][col]>=trs2[0]) && (Sat[row][col]<=trs2[1]))
range2++;
}
if ((rangel!=0) && (range2!=0)) suml=8; else twater[twatermin]=-1;
break;
case 8 :
LUp(rvalleyi,trll,tl[ptc-1]); LDown(rvalleyi,trl2,tl[twater min]);
HLeft(rvalleyh,trhl,th[ptc-1]); HRight(rvalleyh,trh2,th[twater-min]);
SFront(rvalleys,trsl,ts[ptc-1]); SBack(rvalleys,trs2,ts[twater-min]);
range 1=0; range2=0;
for (row=0;row<h;row++)
for (col=0;col<w;col++) {
if ( (Int[row][col]>=trll [0]) && (Int[row][col]<=trll [1]) &&
(Hue[row][col]>=trhl [0]) && (Hue[row][col]<=trhl [1]) &&
(Sat[row][col]>=trs1 [0]) && (Sat[row][col]<=trsl [1]) )
range 1++;
if ( (Int[row][col]>=trl2[0]) && (Int[row][col]<=trl2[1]) &&
(Hue[row] [col]>=trh2[0]) && (Hue[row][col]<=trh2[1]) &&
(Sat[row][col]>=trs2[0]) && (Sat[row][col]<=trs2[1]))
range2++;
}
if ((range 1!=0) && (range2!=0)) suml=9; else twater[twater min]=- 1;
break;
case 9:
LUp(rvalleyi,trll,tl[ptc-I]); LDown(rvalleyi,trl2,tl[twater min]);
HLeft(rvalleyh,trhl,th[ptc-1]); HRight(rvalleyh,trh2,th[twater min]);
Snormal(rvalleys,trsl,ts[ptc-1]); Snormal(rvalleys,trs2,ts[twater-min]);
range 1=0; range2=0;
for (row=o;row<h;row++)
for (col=0;col<w;col++) {
if ( (Int[row][col]>=trll [0]) && (Int[row] [col] <trll [1]) &&
(Hue[row][col]>=trhl [0]) && (Hue[row][col]<=trhl [1]) &&
(Sat[row][col]>-trs 1 [0]) && (Sat[row][col]<=trs [1]) )
range 1++;
if ( (Int[row][col]>=trl2[0]) && (Int[row][col]<=trl2[1]) &&
(Hue[row][col]>=trh2[0]) && (Hue[row][col]<=trh2[1]) &&
(Sat[row][col]>=trs2[0]) && (Sat[row][col]<=trs2[1]))
range2++;
}
if ((rangel!=0) && (range2!=0)) suml=10; else twater[twater-min]=-1;
break;
case 10:
LUp(rvalleyi,trl1,tl[ptc-1]); LDown(rvalleyi,tri2,tl[twater min]);
HLeft(rvalleyh,trhl,th[ptc-1]); HRight(rvalleyh,trh2,th[twater min]);
SBack(rvalleys,trsl,ts[ptc-1]); SFront(rvalleys,trs2,ts[twater-min]);
range 1=0; range2=0;
for (row=0;row<h;row++)
for (col=O;col<w;col++) {
if ( (Int[row][col]>=trll [0]) && (Int[row][col]<=trll [1]) &&
(Hue[row][col]>=trhl [0]) && (Hue[row][col]<=trhl [1]) &&
169
(Sat[row][col]>=trs 1 [0]) && (Sat[row][col]<-trs 1[1]))
range I++;
if ( (lnt[row][col]>=trl2[0]) && (Int[row][col]<=trl2[1]) &&
(Hue[row][col]>=trh2[0]) && (Hue[row][col]<=trh2[1]) &&
(Sat[row] [col]>=trs2 [0]) && (Sat[row] [col]<=trs2[ 1]))
range2++;
}
if ((range 1!=0) && (range2!=0)) suml=11; else twater[twater min]=- 1;
break;
case 11 :
LUp(rvalleyi,trll,tl[ptc-1]); LDown(rvalleyi,trl2,tl[twater-min]);
Hnormal(rvalleyh,trhl,th[ptc-1]); Hnormal(rvalleyh,trh2,th[twater min]);
SBack(rvalleys,trsl,ts[ptc-1]); SFront(rvalleys,trs2,ts[twater-min]);
range 1=0; range2=0;
for (row=0;row<h;row++)
for (col=0;col<w;col++) {
if ( (Int[row][col]>=trl 1 [0]) && (Int[row][col]<=trl 1 [1]) &&
(Hue[row][col]>=trhl [0]) && (Hue[row][col]<=trhl [1]) &&
(Sat[row][col]>=trs1 [0]) && (Sat[row][col]<trs1 [1]) )
range I++;
if ( (Int[row][col]>=trl2[0]) && (Int[row][col]<-trl2[1]) &&
(Hue[row][col]>=trh2[0]) && (Hue[row][col]<=trh2[1]) &&
(Sat[row][col]>-trs2[0]) && (Sat[row][col]<=trs2[1]))
range2++;
}
if ((rangel!=0) && (range2!=0)) suml=12; else twater[twater-min]=-1;
break;
case 12:
LUp(rvalleyi,trll,tl[ptc-1]); LDown(rvalleyi,trl2,tl[twater-min]);
HRight(rvalleyh,trhl,th[ptc-1]); HLeft(rvalleyh,trh2,th[twater-min]);
SBack(rvalleys,trsl,ts[ptc-1]); SFront(rvalleys,trs2,ts[twater-min]);
range 1=; range2=0;
for (row=0 ;row<h;row++)
for (col=0;col<w;col++) {
if ( (Int[row][col]>=trl 1 [0]) && (Int[row][col]<=trll [1]) &&
(Hue[row][col]>=trhl [0]) && (Hue[row][col]<=trhl [1]) &&
(Sat[row][col]>=trs1 [0]) && (Sat[row][col]<=trs 1[1]) )
range I++;
if ( (Int[row][col]>=trl2[0]) && (Int[row][col]<=trl2[1]) &&
(Hue[row][col]>=trh2[0]) && (Hue[row][col]<=trh2[1]) &&
(Sat[row][col]>=trs2[0]) && (Sat[row][col]<=trs2[1]))
range2++;
}
if ((range 1!=0) && (range2!=0)) suml= 13; else twater[twater min]=- 1;
break;
case 13
LUp(rvalleyi,trll,tl[ptc-1]); LDown(rvalleyi,trl2,tI[twater min]);
HRight(rvalleyh,trhl,th[ptc-1]); HLeft(rvalleyh,trh2,th[twater-min]);
Snormal(rvalleys,trsl,ts[ptc-1]); Snormal(rvalleys,trs2,ts[twater-min]);
range 1
0; range2=0;
170
for (row=0;row<h;row++)
for (col=0;col<w;col++) {
if ( (Int[row][col]>=trl 1 [0]) && (Int[row][col]<=trl 1 [1]) &&
(Hue[row][col]>=trhl [0]) && (Hue[row][col]<=trh 1 [1]) &&
(Sat[row][col]>=trsl [0]) && (Sat[row][col]<=trs [1]) )
rangeI++;
if ( (Int[row][col]>=trl2[0]) && (Int[row][col]<=trl2[1]) &&
(Hue[row][col]>=trh2[0]) && (Hue[row][col]<=trh2[1]) &&
(Sat[row][col]>=trs2[0]) && (Sat[row][col]<=trs2[1]))
range2++;
}
if ((rangel!=0) && (range2!=0)) suml=14; else twater[twater-min]=-1;
break;
case 14:
LUp(rvalleyi,trll,tl[ptc-1]); LDown(rvalleyi,trl2,tl[twater min]);
HRight(rvalleyh,trhl,th[ptc-1]); HLeft(rvalleyh,trh2,th[twater-min]);
SFront(rvalleys,trsl,ts[ptc-1]); SBack(rvalleys,trs2,ts[twater-min]);
range I=0; range2=0;
for (row=0 ;row<h;row++)
for (col=0;col<w;col++) {
if ( (Int[row][col]>=trl1 [0]) && (Int[row][col]<=trl I[1]) &&
(Hue[row][col]>=trhl [0]) && (Hue[row][col]<=trhl [1]) &&
(Sat[row][col]>=trs 1 [0]) && (Sat[row][col]<-trs 1[1]) )
rangeI++;
if ( (Int[row][col]>=trl2[0]) && (Int[row][col]<=trl2[1]) &&
(Hue[row][col]>=trh2[0]) && (Hue[row][col]<=trh2[1]) &&
(Sat[row][col]>=trs2[0]) && (Sat[row][col]<=trs2[1]))
range2++;
}
if ((range 1!=0) && (range2!=0)) suml= 15; else twater[twater-min]=-1;
break;
case 15
LUp(rvalleyi,trll,tl[ptc-1]); LDown(rvalleyi,trl2,tl[twater-min]);
Hnormal(rvalleyh,trhl,th[ptc-1]); Hnormal(rvalleyh,trh2,th[twater min]);
SFront(rvalleys,trsl,ts[ptc-1]); SBack(rvalleys,trs2,ts[twater-min]);
range 1=0; range2=0;
for (row=0;row<h;row++)
for (col=0;col<w;col++) {
if ( (Int[row][col]>=trl 1 [0]) && (Int[row][col]<=trll [1]) &&
(Hue[row][col]>=trhl[0]) && (Hue[row][col]<=trhl [1]) &&
(Sat[row][col]>-trs1 [0]) && (Sat[row][col]<=trs 1 [1]) )
range I ++;
if ( (Int[row][col]>=trl2[0]) && (Int[row][col]<=trl2[1]) &&
(Hue[row][col]>=trh2[0]) && (Hue[row][col]<=trh2[1]) &&
(Sat[row][col]>-trs2[0]) && (Sat[row][col]<=trs2[1]))
range2++;
}
if ((rangel!=0) && (range2!=0)) suml=16; else twater[twater-min]=-1;
break;
case 16:
LUp(rvalleyi,trll,tl[ptc-1]); LDown(rvalleyi,trl2,tl[twater-min]);
171
Hnormal(rvalleyh,trhl,th[ptc-1]); Hnormal(rvalleyh,trh2,th[twater min]);
Snormal(rvalleys,trsl,ts[ptc-1]); Snormal(rvalleys,trs2,ts[twatermin]);
rangeI=0; range2=0;
for (row=0;row<h;row++)
for (col=0;col<w;col++) {
if ( (Int[row][col]>=trll [0]) && (Int[row][col]<==trll [1]) &&
(Hue[row][col]>=trhl [0]) && (Hue[row][col]<=trhl [1]) &&
(Sat[row][col]>=trsl [0]) && (Sat[row][col]<=trs [1]) )
range 1++;
if ( (Int[row][col]>=trl2[0]) && (Int[row][col]<-trl2[1]) &&
(Hue[row][col]>=trh2[0]) && (Hue[row][col]<=trh2[1]) &&
(Sat[row][col]>=trs2[0]) && (Sat[row][col]<=trs2[1]))
range2++;
}
if ((range l!=O) && (range2!=0)) suml=17; else twater[twater-min]=-1;
break;
case 17:
LDown(rvalleyi,trll,tl[ptc-1]); LUp(rvalleyi,trl2,tl[twater min]);
HLeft(rvalleyh,trhl,th[ptc-1]); HRight(rvalleyh,trh2,th[twater min]);
SFront(rvalleys,trsl,ts[ptc-1]); SBack(rvalleys,trs2,ts[twatermin]);
range I =0; range2=0;
for (row=0;row<h;row++)
for (col=O;col<w;col++) {
if ( (Int[row][col]>=trll[0]) && (Int[row][col]<=trlI[1]) &&
(Hue[row][col]>=trhl[0]) && (Hue[row][col]<=trh1 [1]) &&
(Sat[row][col]>=trsl [0]) && (Sat[row][col]<=trsl [1]) )
rangeI++;
if ( (Int[row][col]>=trl2[0]) && (Int[row][col]<=trl2[1]) &&
(Hue[row][col]>=trh2[0]) && (Hue[row][col]<=rih2[1]) &&
(Sat[row][col]>=trs2[0]) && (Sat[row][col]<-trs2[ 1]))
range2++;
}
if ((range 1!=0) && (range2!=0)) suml= 18; else twater[twater min]=- 1;
break;
case 18:
LDown(rvalleyi,trll,tl[ptc-l]); LUp(rvalleyi,trl2,tl[twater min]);
HLeft(rvalleyh,trhl,th[ptc-1]); HRight(rvalleyh,trh2,th[twater min]);
Snormal(rvalleys,trsl,ts[ptc-1]); Snormal(rvalleys,trs2,ts[twater min]);
range 1=0; range2=0;
for (row=0;row<h;row++)
for (col=O;col<w;col++) {
if ( (Int[row][col]>=trll [0]) && (Int[row][col]<=trll [1]) &&
(Hue[row][col]>=trhl [0]) && (Hue[row][col]<=trhl [1]) &&
(Sat[row][col]>=trs 1 [0]) && (Sat[row][col]<=trs [1]) )
range 1++;
if ( (Int[row] [col]>-trl2 [0]) && (Int[row][col]<=trl2[ 1]) &&
(Hue[row][col]>-trh2[0]) && (Hue[row][col]<=trh2[ ]) &&
(Sat[row] [col]>=trs2 [0]) && (Sat[row][col]<=trs2[ I)
range2++;
}
172
if ((range 1!=0) && (range2!=0)) suml= 19; else twater[twater min]=- 1;
break;
case 19:
LDown(rvalleyi,trll,tl[ptc-1]); LUp(rvalleyi,trl2,tl[twater-min]);
HLeft(rvalleyh,trhl,th[ptc-1]); HRight(rvalleyh,trh2,th[twater-min]);
SBack(rvalleys,trsl,ts[ptc-1]); SFront(rvalleys,trs2,ts[twater min]);
range 1=0; range2=0;
for (row=O;row<h;row++)
for (col=0;col<w;col++) {
if ( (Int[row][col]>=trll [0]) && (Int[row][col]<=tr1 [1]) &&
(Hue[row][col]>=trhl [0]) && (Hue[row][col]<=trhl [1]) &&
(Sat[row][col]>=trs 1 [0]) && (Sat[row][col]<=trs 1[1]) )
range I++;
if ( (Int[row][col]>-trl2[0]) && (Int[row][col]<=trI2[1]) &&
(Hue[row][col]>=trh2[0]) && (Hue[row][col]<-trh2[1]) &&
(Sat[row][col]>=trs2[0]) && (Sat[row][col]<=trs2[1]))
range2++;
}
if ((rangel!=0) && (range2!=0)) suml=20; else twater[twater-min]=-1;
break;
case 20:
LDown(rvalleyi,trll,tl[ptc-1]); LUp(rvalleyi,trl2,tl[twater-min]);
Hnormal(rvalleyh,trhl,th[ptc-1]); Hnormal(rvalleyh,trh2,th[twater min]);
SBack(rvalleys,trsl,ts[ptc-1]); SFront(rvalleys,trs2,ts[twater-min]);
range I =0; range2=0;
for (row=0;row<h;row++)
for (col=0;col<w;col++) {
if ( (Int[row][col]>=trl 1 [0]) && (Int[row][col]<=trl 1 [1]) &&
(Hue[row][col]>=trhl [0]) && (Hue[row][col]<=trhl [1]) &&
(Sat[row][col]>=trs 1 [0]) && (Sat[row][col]<=trs [1]) )
range I ++;
if ( (Int[row][col]>=trl2[0]) && (Int[row][col]<=trl2[1]) &&
(Hue[row][col]>=trh2[0]) && (Hue[row][col]<=trh2[1]) &&
(Sat[row][col]>=trs2[0]) && (Sat[row][col]<-trs2[1]))
range2++;
}
if ((range l!=0) && (range2!=0)) suml=21; else twater[twater min]=-1;
break;
case 21
LDown(rvalleyi,trll,tl[ptc-1]); LUp(rvalleyi,trl2,tl[twater-min]);
HRight(rvalleyh,trhl,th[ptc-1]); HLeft(rvalleyh,trh2,th[twater min]);
SBack(rvalleys,trsl,ts[ptc-1]); SFront(rvalleys,trs2,ts[twatermin]);
range 1=0; range2=0;
for (row=0;row<h;row++)
for (col=0;col<w;col++) {
if ( (Int[row][col]>-trl1 [0]) && (Int[row][col]<=trll [1]) &&
(Hue[row][col]>=trhl [0]) && (Hue[row][col]<=trh 1[1]) &&
(Sat[row][col]>=trs1 [0]) && (Sat[row][col]<=trs [1]) )
range 1++;
if ( (Int[row][col]>=trl2[0]) && (Int[row][col]<=trl2[1]) &&
(Hue[row][col]>=trh2[0]) && (Hue[row][col]<=trh2[1]) &&
173
(Sat[row][col]>=trs2[0]) && (Sat[row][col]<=trs2[1]))
range2++;
}
if ((rangel!=0) && (range2!=0)) suml=22; else twater[twater-min]=-1;
break;
case 22:
LDown(rvalleyi,trll,tl[ptc-l]); LUp(rvalleyi,trl2,tl[twater min]);
HRight(rvalleyh,trhl,th[ptc-1]); HLeft(rvalleyh,trh2,th[twater min]);
Snormal(rvalleys,trsl,ts[ptc-1]); Snormal(rvalleys,trs2,ts[twater min]);
range I =0; range2=0;
for (row=0;row<h;row++)
for (col=0;col<w;col++) {
if ( (Int[row][col]>=trl 1 [0]) && (Int[row][col]<=trl 1[1]) &&
(Hue[row][col]>=trh 1 [0]) && (Hue[row][col]<=trhl [I]) &&
(Sat[row][col]>=trs1 [0]) && (Sat[row][col]<=trs 1[1]) )
range l++;
if ( (Int[row][col]>=trl2[0]) && (Int[row][col]<=trl2[1]) &&
(Hue[row][col]>=trh2[0]) && (Hue[row][col]<=trh2[1]) &&
(Sat[row][col]>=trs2[0]) && (Sat[row][col]<=trs2[1]))
range2++;
}
if ((range 1!=0) && (range2!=0)) suml=23; else twater[twater min]=- 1;
break;
case 23
LDown(rvalleyi,trl1,tl[ptc-1]); LUp(rvalleyi,trl2,tl[twater min]);
HRight(rvalleyh,trhl,th[ptc-1]); HLeft(rvalleyh,trh2,th[twater-min]);
SFront(rvalleys,trsl,ts[ptc-1]); SBack(rvalleys,trs2,ts[twater-min]);
range 1=0; range2=0;
for (row=0;row<h;row++)
for (col=0;col<w;col++) {
if ( (Int[row][col]>=trl1 [0]) && (Int[row][col]<=trl I[1]) &&
(Hue[row][col]>=trhl [0]) && (Hue[row][col]<-=trhl [1]) &&
(Sat[row][col]>=trs 1 [0]) && (Sat[row][col]<=trs 1 [1]) )
rangeI++;
if ( (Int[row][col]>=tr2[0]) && (lnt[row][col]<=trl2[1]) &&
(Hue[row][col]>=trh2[0]) && (Hue[row][col]<=trh2[1]) &&
(Sat[row][col]>=trs2[0]) && (Sat[row][col]<=trs2[1]))
range2++;
}
if ((rangel!=0) && (range2!=0)) suml=24; else twater[twater-min]=-1;
break;
case 24:
LDown(rvalleyi,trll,tl[ptc-1]); LUp(rvalleyi,trl2,tI[twater min]);
Hnormal(rvalleyh,trhl,th[ptc-1]); Hnornal(rvalleyh,trh2,th[twater min]);
SFront(rvalleys,trs1,ts[ptc-1]); SBack(rvalleys,trs2,ts[twatermin]);
range 1=0; range2=0;
for (row=0;row<h;row++)
for (col=0;col<w;col++) {
if ( (Int[row][col]>=trl 1 [0]) && (Int[row][col]<=trl I [1]) &&
(Hue[row] [col]>=trhl [0]) && (Hue[row][col]<=trh 1 [1]) &&
174
(Sat[row][col]>=trs1 [0]) && (Sat[row][col]<=trs 1[1]))
range I++;
if ( (Int[row] [col]>=trl2 [0]) && (Int[row][col]<=trl2[1]) &&
(Hue[row][col]>=trh2[0]) && (Hue[row][col]<=trh2[1]) &&
(Sat[row][col]>=trs2[0]) && (Sat[row][col]<=trs2[1]))
range2++;
}
if ((rangel!=0) && (range2!=0)) suml=25; else twater[twater-min]=-1;
break;
case 25
LDown(rvalleyi,trll,tl[ptc-1]); LUp(rvalleyi,trl2,tl[twater min]);
Hnormal(rvalleyh,trhl,th[ptc-1]); Hnormal(rvalleyh,trh2,th[twater min]);
Snormal(rvalleys,trsl,ts[ptc-1]); Snormal(rvalleys,trs2,ts[twatermin]);
range 1=0; range2=0;
for (row=0;row<h;row++)
for (col=O;col<w;col++) {
if ( (Introw][co]>=trl 1 [0]) && (Int[row][col]<--trl 1 [1]) &&
(Hue[row][col]>=trhl [0]) && (Hue[row][col]<=trh1 [1]) &&
(Sat[row][col]>=trs 1 [0]) && (Sat[row][col]<=trs [1]) )
range 1++;
if ( (Int[row][col]>=tr2[0]) && (Int[row][col]<=trl2[1]) &&
(Hue[row][col]>-=trh2[0]) && (Hue[row][col]<=trh2[1]) &&
(Sat[row][col]>=trs2[0]) && (Sat[row][col]<=trs2[I]))
range2++;
}
if ((range l!=O) && (range2!=0)) suml=26; else twater[twater min]=-1;
break;
case 26:
suml=minindex;
minimumw[tl[ptc-1]][th[ptc-1]][ts[ptc-1]]=minindex;
minindex++;
}
}
}
}
tindex++;
delete trl 1; delete trl2; delete trhl; delete trh2; delete trs 1; delete trs2;
return suml;
}
------------------------------------------------------------------------------
void Lnormal(int *rvalleyi, float *trl, int tl)
{
trl[0]=(((rvalleyi[tl]/rI)*rangeL)+minL);
trl[1]=(((rvalleyi[tl+1]/rI)*rangeL)+minL);
}
//-----------------------------------------------------------void LUp(int *rvalleyi, float *trI,
{
int tl)
175
trl[O]=(((rvalleyi[tl]/rI)*rangeL)+minL);
trl[1]=(((rvalleyi[tl]/rI)*rangeL)+minL)+((rangeL/6.0)*(rvalleyi[tl+1]-rvalleyi[tl])/rI);
/up
}
//-----------------------------------------------void LDown(int *rvalleyi, float *trl, int tl)
{
trl[O]=(((rvalleyi[tl+1]/rI)*rangeL)+minL)-((rangeL/6.0)*(rvalleyi[tl+1]-rvalleyi[tl])/rI);
trl[1 ]=(((rvalleyi[tl+1]/rI)*rangeL)+minL);
/down
}
/----------------------------------------------void Hnormal(int *rvalleyh, float *trh, int th)
{
trh[0]=(((rvalleyh[th]/rH)-0.5)*2.0*MPI);
trh[1]=(((rvalleyh[th+1]/rH)-0.5)*2.O*MPI);
}
//--------------------------------------------void HRight(int *rvalleyh, float *trh, int th)
{
trh[O]=((( (rvalleyh[th]/6.0+(5.0/6.0)*rvalleyh[th+ 1]) /rH)-0.5)*2.0*MPI); / to right H
trh[1]=(((rvalleyh[th+1]/rH)-0.5)*2.0*MPI);
}
//--------------------------------------------void HLeft(int *rvalleyh, float *trh, int th)
{
trh[O]=(((rvalleyh[th]/rH)-0.5)*2.0*MPI);
trh[1]=((( ((5.0/6.0)*rvalleyh[th]+rvalleyh[th+1]/6.0) /rH)-0.5)*2.0*MPI); / to left H
}
/---------------------------------------------
void Snormal(int *rvalleys, float *trs, int ts)
{
trs[O]=(((rvalleys[ts]/rS)*rangeS)+minS);
trs[ 1]=(((rvalleys[ts+ 1]/rS)*rangeS)+minS);
}
//--------------------------------------------void SFront(int *rvalleys, float *trs, int ts)
{
trs[0]=(((rvalleys[ts]/rS)*rangeS)+minS);
trs[1]=(((rvalleys[ts]/rS)*rangeS)+minS)+((rangeS/6.0)*(rvalleys[ts+1]-rvalleys[ts])/rS);
/ to front S
}
//--------------------------------------------void SBack(int *rvalleys, float *trs, int ts)
{
trs[O]=(((rvalleys[ts+1]/rS)*rangeS)+minS)-((rangeS/6.0)*(rvalleys[ts+1]-rvalleys[ts])/rS);
trs[ 1]=(((rvalleys[ts+1 ]/rS)*rangeS)+minS);
/ to back S
}
//--------------------------------------------------
176
......
............
.......
.....
....................
...........
.......
... .. ....
....
..............
References:
1.
R. Adams and L. Bischof, "Seeded Region Growing", IEEE transactions on pattern
analysis and machine intelligence, vol. 16, no. 6, 1994.
2. R. Alferez and Y.F. Wang, "Geometric and Illumination Invariants for Object
Recognition", IEEE Trans. PAMI, Vol. 21, No. 6, p.505-535, June 1999.
3. B.D. Aumond, "High Precision Profilometry.", Mechanical Engineering M.S. Thesis,
(M.I.T., 1994)
4. CIE Standards, http://cvision.ucsd.edu/cie.htm, for raw data.
5. C.H. Chen, Jiann-Der Lee, Jenq-Rong Wand and C. W. Mao, "Color Image
Segmentation for Bladder Cancer Diagnosis", Mathl. Comput. Modelling, vol. 27, no.
2, 1998.
6. A. Daniele, S. Salapaka, M.V. Salapaka and M. Dahleh, "Piezoelectric Scanners for
Atomic Force Microscopes: Design of Lateral Sensors, Identification and Control",
American Control Conference, p. 253-257, 1999.
7. "Digital Instruments" (http://www.di.com/) for the images of AFM cantilevers and
probes.
8. D.G. Fink (Consulting Editor), Selected papers and records of the National Television
System Committee, Color Television Standards, McGraw-Hill Book Company, INC.,
1s edition, 1955.
9. K. Fukunaga, Introduction to Statistical Pattern Recognition, Computer Science and
Scientific Computing, Academic Press, 2 "d edition, 1990.
10. J. M. Gauch, "Image Segmentation and Analysis via Multiscale Gradient Watershed
Hierarchies", IEEE Trans. on Image Processing, vol. 8, no. 1, 1999.
11. C.R. Giardina and E.R.Dougherty, Morphological Methods in Image and Signal
Processing, Prentice Hall, Englewood Cliffs, New Jersey, 1988.
12. E.F. Glynn II, CIE Chromaticity Diagrams, Maxwell Triangle and color range, efg99,
http://www.efg2.com/Lab/Default.htm
13. W.E.L. Grimson, T. Lozano - P e rez and D.P. Huttenlocher, Object Recognition by
Computer: The Role of Geometric Constraints, The MIT Press, Cambridge,
Massachusetts, 1990.
14. R.M. Haralick and L.G. Shapiro, "SURVEY Image Segmentation Techniques",
Computer vision, graphics, and image processing, vol. 29, 1985.
177
.
.
......
....
..
. ... ...........
......................
. ............
.............
-..................
15. B.K.P. Horn, Robot Vision, The MIT Press, Cambridge, Massachusetts, 12t printing,
1998.
16. C. Imelin'ska, M.S. Downes, W. Yuan, "Semi-automated color segmentation of
anatomical tissue", Computerized Medical Imaging and Graphics, Vol. 24, p.173180, 2000.
17. G.J. Klinker, S.A. Shafer and T. Kanade, "A physical approach to color image
understanding", Int. J. Comput. Vision, vol. 4, 1990.
18. G.A. Korn, Neural Networks and Fuzzy-Logic Control on Personal Computers and
Workstations, The MIT Press, Cambridge, Massachusetts, 1st edition, 1992.
19. J. Liu and Y.H. Yang, "Multiresolution Color Image Segmentation", IEEE
transactions on pattern analysis and machine intelligence, vol. 16, no. 7, 1994.
20. K. Mehrotra, C.K. Mohan, S. Ranka, Elements of Artificial Neural Networks,
edition The MIT Press, Cambridge, Massachusetts, 1997.
1 st
21. F. Mosteller, R.E.K. Rourke and G.B. Thomas JR., Probability and Statistics,
Addison-Wesley Publishing Company, Inc., 1961.
22. M. Nitzberg, D. Mumford and T. Shiota, "Filtering, Segmentation and Depth",
Lecture Notes in Computer Science, Springer-Verlag Berlin Heidelberg, 1993.
23. "Quesant Instrument Corporation" (http://www.quesant.com/) for an image of the
AFM.
24. K. Reisdorph, et al., Borland C++ BuilderTM4 - Unleashed, Sams Borland Press, 1 't
edition, 1999.
25. "The Rest of Lena Story" (http://www.cs.cmu.edu/~chuck/lennapg/lenna.shtml) by
chuck+@cs.cmu.edu for Lenna's image.
26. A. Rosenfeld and A.C. Kak, Digital Picture Processing, Academic Press, 2"d edition,
New York, New York, 1982.
27. C. A. Rothwell, Object Recognition through Invariant Indexing, Oxford University
Press, 1 st Printing, 1995.
28. P. Salembier and M. Parda's, "Hierarchical Morphological Segmentation for Image
Sequence Coding", IEEE Trans. on Image Processing, Vol. 3, No. 5, p. 639-651,
September 1994.
178
29. S.J. Sangwine and R.E.N. Home, The Colour Image Processing Handbook, Chapman
& Hall, 1st edition, 1998.
30. A. Sarabi and J.K. Aggarwal, "Segmentation of chromatic image", PR, Vol.13, p.417427, 1981.
31. L. Shafarenko, M. Petrou, and J. Kittler, "Histogram-Based Segmentation in a
Perceptually Uniform Color Space", IEEE transactions on image processing, vol. 7,
no. 9, 1998.
32. W.N. Sproson, Colour Science in Television and Display Systems, Adam Hilger Ltd,
Bristol, 1st edition, 1983.
33. G. Vosselman, Relational Matching, Lecture Notes in Computer Science, SpringerVerlag, 1 " edition, p. 1-43 and p. 88-109, 1992.
34. Z. Wu and R. Leahy, "An Optimal Graph Theoretic Approach to Data Clustering:
Theory and Its Application to Image Segmentation", IEEE Trans. on PAMI, Vol. 15,
No. 11, p.1101-1113, November 1993.
35. Y. Yeo, "Image Processing for Precision Atomic Force Microscopy.", Mechanical
Engineering M.S. Thesis, (M.I.T., 2000)
179