2012 International Conference on Computer Technology and Science (ICCTS 2012)
IPCSIT vol. 47 (2012) © (2012) IACSIT Press, Singapore
DOI: 10.7763/IPCSIT.2012.V47.76
Object-oriented Classification of High-resolution Remotely Sensed
Imagery
Li Fu1+, Jian Cheng1, 2, 3 and Yongheng Zheng1
1
School of Electronic Engineering, University of Electronic Science and Technology of China, Chengdu,
611731, China
2
Institute of Geo-Spatial Information Science and Technology, University of Electronic Science and
Technology of China, Chengdu, 611731, China
3
State Key Laboratory of Remote Sensing Science, Beijing, 100875, China
Abstract. An object-oriented classification method using Region Adjacency Graph (RAG) is introduced to
process high-resolution remotely sensed imagery in this paper. The proposed method with the nearest
neighbor classifier utilizes valuable spatial and structural information complementary to pixel-based spectral
information. The key of the proposed method is to get meaningful objects through image segmentation. A
hybrid segmentation method combining marker-based watershed transformation and region merging
algorithm is adopted in this paper, which provides one-pixel wide, closed and accurately localized contours.
In the region merging stage, RAG is used to represent the regions and an automatic stopping merging
criterion using spectral divergence is proposed. Experimental results indicate that the proposed method is
superior to pixel-based method in classifying high-resolution remotely sensed imagery.
Keywords: high-resolution; remotely sensed imagery; object-oriented classification; watershed
transformation; region merging
1. Introduction
The constant increase in the amount and resolution of remotely sensed (RS) imagery necessitates the development of
automatic processing and analysis methods for high-resolution imageries. Land-use classification is a basic work for RS
imagery analysis and understanding. The most common kind of classification methods are pixel-based, such as
Maximum Likelihood Classification (MLC), Minimum Distance Classification (MDC), Parallelepiped Classification, KMeans Clustering, Fuzzy C-Means Clustering and so on[1]. These traditional pixel-based classification techniques are
more suitable for hyper-spectral or multi-spectral RS imagery with rich spectral information. High-resolution RS imagery
contains other abundant information such as shape, spatial and structural information, which has potential to further
enhance image classification. If we still use pixel-based classification methods utilizing only spectral information to
classify high-resolution RS imagery, it will cause large amount of waste in resources, even misclassified or unclassified
pixels [2]. Therefore, an improved classification approach for high-resolution imagery is doomed to emerge with better
accuracy. That is why object-oriented classification method has turned up. Image object is a self-existent, resoluble entity
consisting of neighboring pixels with similar characteristics. The fundamental distinction of object-oriented and pixelbased analysis method is that the former operates on an entity, while the latter on per-pixel.
We obtain image object through image segmentation. Multi-scale segmentation technique is popular in objectoriented classification researches. In this paper, we adopt a hybrid segmentation method combining watershed
transformation and region merging algorithm, which also produces excellent segmentation result. The rest of this paper is
organized as follows. An overview of the proposed classification method is depicted in section II. In section III,
experimental results and analysis are given. Conclusions are drawn in section IV.
+
Email: fuli_babyblue87@163.com
411
2. Methodology Overview
The proposed object-oriented classification method contains three major parts. These are image segmentation, object
feature extraction and the nearest neighbor classification. In the segmentation stage, morphological filtering is conducted
as pre-processing and the gradient and marker image are generated to be the input of the hybrid segmentation algorithm
as illustrated in Fig. 1.
2.1.
Image segmentation
Segmentation is a technique partitioning of a given image into a number of homogeneous segments, which is the
premise for object-oriented classification. The result of image segmentation will directly impact the performance of
subsequent classification. Watershed transformation is a newly-used segmentation technique based on mathematical
morphology, which produces one-pixel wide, closed and accurately localized contours with high accuracy and low
computational complexity [3]. Simultaneously, it causes sever over-segmentation problem due to noise and local
irregularities in the image. To overcome this problem, some pre-processing and post-processing work have been done as
depicted in the following subsections.
1) Morphological filtering
Smoothing technique should be applied to the original image to reduce the noise and local irregularities. Studies
indicate that morphological operations, opening and closing by reconstruction especially, do better in smoothing the
image and preserving the boundary information than linear filters. In this phase the input image is subjected to a group of
morphological operations as preprocessing before generating the gradient image. Basic morphological operations are
dilation, erosion, opening, closing and reconstruction [4]. The kernel of these operations is commonly known as
structuring element (SE). The shape and the size of SE can influence the segmentation result. As for the size selection of
SE, too small size will produce little effect on smoothing and too big size will merge different objects. After experiments
using different SE, we finally choose a disk kernel with radius of 3.
Input Image
Morphological Filtering
Gradient and Marker image
Segmentation
Feature Extraction
Classification
Output Image
Fig. 1 Flow chart of the proposed object-oriented classification.
2) Gradient image and Marker image generation
The local variation of gray value in the image can well be given by the gradient. It helps detecting ramp edges and
avoids thickening and merging of edges providing edge-enhancements. The gradient image in this paper is obtained by
Sobel operator [4].
The concept of markers is used to solve over-segmentation problem, whose goal is to detect the presence of
homogeneous regions from the image by a set of morphological simplifications [3]. We get a binary marker image, which
contains a set of components (set to 0) marking the core regions and a large number of pixels remaining unassigned (set
to 1). The next step is to label the unassigned pixels by watershed algorithm dealing with markers to get the primitive
partition.
412
3) Watershed transformation
Watershed transformation of an image is a mapping from the original image to a labeled image such that all points in
a given catchment basin have the same unique label. With the marker image generated from the previous stage, the splitand-merge fast watershed algorithm developed by Dobrin et al. is applied to the gradient image. The algorithm is
described in [5].
4) Region merging
This step is a post-processing of watershed transformation to solve the over-segmentation problem. It can be
achieved by merging neighboring regions based on a certain similarity measure. The regions obtained by watershed
algorithm are represented using region adjacency graph (RAG) [6]. RAG is a kind of data structure to represent regions
in which the nodes correspond to regions and adjacent regions are connected by edges. Each node is associated with the
relevant properties of the region it represents. Such basic properties are size, mean spectral values of different satellite
bands, brightness and so on. Additional properties, like shape index, compactness, and texture measures, will be utilized
in the process of object feature selection.
Before the merging operation, a weight must be assigned to each graph edge expressing the dissimilarity between
two regions. Then two regions with the minimum weight are merged. A common description of the dissimilarity function
is [7]:
δ (i, j ) =
|| Ri || ⋅ || R j ||
|| Ri || + || R j ||
[ μ ( Ri ) − μ ( R j )] 2 ⋅ I (i, j )
(1)
Where || Ri ||,|| Rj || denote the number of pixels in region i, j; μ ( Ri ), μ ( R j ) denote the mean spectral value of Ri , R j ; If
Ri , R j is adjacent, I (i, j ) = 1 else I (i, j ) = +∞ .
When to stop the merging procedure? Two simple criteria to stop iteration are edge weight threshold and region
count threshold. Iteration stops when either minimum edge weight or region count exceeds their corresponding threshold.
Actually it is rather difficult to choose approximate values for these thresholds, especially for complex images. In this
paper, an automatic stopping criterion is proposed using spectral divergence that describes the total spectral value
difference degree of all the regions in the image. After each iteration, let K l be the number of regions left, J l be the
spectral divergence at this time, J wr be the divergence of region r(r=0,1… K l ), J t be the divergence of the whole
r
image, N r , N be the number of pixels in region r and the whole image, x i , x i be the spectral value of pixel i in
r
region r and in the image, μ , μ be the mean spectral value of region r and the image, then
Kl
Jl =
∑
J wr
r=0
Jt
Kl
=
Nr
∑ ∑ [x
r =0 i=0
N
∑ [x
i
r
i
− μ r ]2
− μ]
(2)
2
i=0
Let K be the initial number of regions produced by watershed algorithm, kl = Kl / K . From the aspect of image
segmentation, we hope the region number as few as possible, i.e., the smaller k l the better; while from the aspect of
regions obtained by segmentation, we hope the information loss in each region as little as possible, i.e., the smaller J l
the better. These two problems are contradictory, which can not reach their corresponding smallest value simultaneously.
Considering the relation between information loss and the number of remaining regions in the merging stage, we define
Jl + kl as the merging cost to compromise the contradiction. When the cost reaches the minimum value the merging
procedure terminates.
The proposed region merging algorithm works as follows:
Input: RAG of the K-partition
Iteration: For i = 0 to n-1
Find the most similar pair of neighboring regions in the (K-i)-RAG;
Merge the regions of the pair to get the (K-i-1) RAG;
Compute the merging cost, if it is the minimum then stop the iteration;
Output: RAG of the (K-n)-partition and the segmented image
2.2.
Object feature extraction
413
In this step, we need to extract features of the objects to conform a feature space. Except spectral features, we extract
shape and textural features to improve the effectiveness of our classification method. Some frequently-used shape and
textural features are given in [8].
A common way to describe texture is using Gray Lever Concurrence Matrix (GLCM) proposed by Haralick [9]. The
size of GLCM is determined by the number of gray levels of the study image. For instance, if the number of gray levels is
N, the size of GLCM is N×N. In our study image, the number of gray levels is 256 so that it is time-consuming to
compute GLCM. Therefore we quantize it to 64 to reduce computational load, meanwhile the image will not lose too
much information at this level.
Feature selection is important to later classification stage. Both too many and too few features may decrease the
classification accuracy unexpectedly. Therefore we need to select meaningful features which can distinguish different
land uses effectively, especially those with similar spectral features.
2.3.
The nearest neighbor classification
The nearest neighbor classification is the specialization of Maximal Likelihood Classification (MLC), where the
covariance matrix of every type is a unit matrix and equivalent [1]. Suppose there are n classes { ω 1 , ω 2 , " , ω n } and
each class ω i has mi samples, whose corresponding feature vector is x ik (k=1,2,… mi ). Let x be the unclassified
feature vector, the discrimination function of class ωi is defined as
g i ( x ) = min || x − x ik || 2 ( k = 1, 2 " m i )
(3)
The decision rule of the classifier is represented as follows:
If g j ( x ) = min g i ( x )
i = 1,2 " , n , then x ∈ ω j .
3. Experimental Results and Analysis
We take a ground locating in Beijing as the study area, and there are six types of land-use in general: building, grass
land, wheat land, vegetable land, bare land and road. High resolution imagery acquired by satellite SPOT5 in Apr.2005
was used, including data from green, red, near-infrared (NIR) bands, with the spatial resolution of 10 meter.
Several results of the proposed classification method and MLC are showed in Fig. 2. The initial region number
produced by watershed algorithm in (b) is 8717 and after region merging algorithm, there are 1219 regions in (c). These
two results demonstrate the effectiveness of the hybrid segmentation method in this paper, which solve the oversegmentation problem greatly. The final classification result of the proposed method is showed in (d). MLC is
commonly-used and excellent in pixel-based classification methods; so we apply MLC to the same image as a
comparison. Fig. 2 (e) shows the classification result using MLC in ENVI4.2. From visual effect, our classification
method avoids a large number of unclassified pixels (black points), i.e., the pepper noise.
In the study area, we randomly access 235 samples to assess the classification accuracy of the proposed method and
MLC by analysis their Confusion Matrix. Table I and Table II are the Confusion Matrix of the proposed object-oriented
classification method and Maximum Likelihood Classification (MLC). The accuracy assessment of two methods is
showed in Table III, where we can easily find out that the overall accuracy of the proposed method reaches to 90.21%,
17% higher than that of MLC (73.19%) and Kappa coefficient also improves from 0.6727 to 0.8774. Especially, the
producer accuracy of building recognized by our method increases 20%, road increases 15%, and bare land also increases
27.50%. This illustrates object-oriented method can better recognize land uses with similar spectral value for full use of
shape and textural information.
4. Conclusion
An object-oriented classification method was presented for high-resolution RS imagery. Experimental results
demonstrate the effectiveness of the proposed method. The key of the proposed method is the segmentation stage, which
integrates several image processing techniques: morphological operations, computation of the gradient image, makerbased watershed transformation and region merging using RAG. We have introduced a function named spectral
divergence function as an automatic stopping criterion in region merging algorithm, which avoids the difficulty of
selection of appropriate parameter value. This segmentation method is easy to be tailored for different applications, such
as processing other kinds of image or be input of higher level image analysis etc. Future work will concentrate on
414
applying other classification techniques instead of the nearest neighbor classifier, such as Support Vector Machine
(SVM), to get better classification result.
Table 1 Confusion Matrix of The Proposed Method
Classified
data
building
grass
wheat
vegetable
bare land
road
Column
total
building
grass
wheat
45
2
0
0
2
1
1
12
0
0
2
0
0
1
36
2
1
0
50
15
40
vegetable
road
0
2
3
64
1
0
bare
land
2
0
0
0
37
1
70
40
20
road
2
0
0
0
0
18
Table 2 Confusion Matrix of MLC
Classified
data
building
grass
wheat
vegetable
bare land
road
unclassifi
ed data
Column
total
building
grass
wheat
vegetable
35
1
0
0
8
5
0
10
0
2
2
0
0
3
30
3
1
0
0
10
2
56
0
0
bare
land
5
4
0
0
26
1
1
1
3
2
4
1
50
15
40
70
40
20
1
2
0
0
1
15
Table 3 Accuracy Assessment of Two Methods
The proposed method
Accuracy
building
grass
wheat
vegetable
bare land
road
Overall
accuracy
Kappa
Producer
accuracy
90.00％
80.00％
90.00％
91.43％
92.50％
90.00％
User
accuracy
90.00％
70.59％
92.31％
94.12％
88.10％
90.00％
90.21％
0.8774
Maximum Likelihood
Producer
accuracy
70.00％
66.67％
75.00％
80.00％
65.00％
75.00％
User
accuracy
85.37％
33.33％
93.75％
91.80％
68.42％
71.43％
73.19％
0.6727
5. Acknowledgment
This work was supported by Open Fund of State Key
Laboratory for Remote Sensing Science, Institute of Remote Sensing Applications of Chinese Academy of Sciences
and Beijing Normal University (NO.2009KFJJ004), sponsored by grants from the National Basic Research (973)
Program of China (NO.2007CB714406), and supported by China Postdoctoral Science Foundation (NO.20080441198).
6. References
[1] C. M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006.
[2] Yongxue Liu, Manchun Li, Liang Mao,Feifei Xu, and Shuo Huang, “Review of remotely sensed imagery
classification patterns based on objected-oriented image analysis”, Chinese Geographical Science, vol.16(3), 2006,
pp. 282-288.
[3] K. Nallaperumal, K. Krishnaveni, J. Varghese, S. Saudia, S. Annam, and P. Kumar, “An efficient multiscale
morphological watershed segmentation using gradient and marker extraction” IEEE, India Conference, 2006.
[4] R. C. Gonzales, and R. E. Woods, Digital Image Processing, 2nd ed., Beijing: Publishing House of Electronics
Industry, 2007.
[5] B. P. Dobrin, T. Viero, and M. Gabbouj, “Fast watershed algorithms: analysis and extensions”, Proc.IS&T/SPIE
Symposium on Electronic Image Science & Technology, Nonlinear Image Processing V, vol.2180, 1994, pp. 209220.
415
[6] K. Saarinen, “Color image segmentation by a watershed algorithm and region adjacency graph processing”, IEEE,
Image processing, vol.3, 1994, pp. 1021-1025.
[7] K. Haris, S. N. Efstratiadis, and N. Maglaveras, “Watershed-based image segmentation with fast region merging”,
IEEE, Image Processing, vol.3, Oct.1998, pp.338-342.
[8] Zhong Chen, “Research on high resolution remote sensing image classification technology”, Beijing: Institute of
Remote Sensing Applications of Chinese Academy of Science, 2006.
[9] R. M. Haralick, Dinstein, and K. Shanmugam, “Textural features for image classification”, IEEE, Transactions on
Systems, Man, and Cybernetics, vol. SMC-3, 1973, pp.610-621.
Building
Grass Land
Wheat Land
(a)
Building
Vegetable Land
Bare Land
(b)
Grass Land
Wheat Land
Vegetable Land
(d)
Road
(c)
Bare Land
Road
unclassified pixel
(e)
Fig. 2 Experimental results (a) original false color image (b) segmentation result of watershed algorithm (c) result of
region merging algorithm (d) classification result of the proposed method (e) classifiction result using MLC in ENVI4.2
416