The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. 38, Part II
OBJECT-BASED IMAGE CLASSIFICATION UTILIZING BACKGROUND
KNOWLEDGE: A CASE STUDY OF LAND USE CLASSIFICATION
T. Zhang, X.M.Yang, C.H.Zhou, F.Z.Su, J.M.Gong, Y.Y.Du
State Key Laboratory of Resources and Environmental Information System, IGSNRR, CAS, Beijing, China –
(zhangt, yangxm, zhouch, sufz, gongjm, duyy)@lreis.ac.cn
KEY WORDS: object-based classification, background knowledge, change detection, land use, CBERS
ABSTRACT:
Object-based image classification and information extraction approaches are rapidly developing from the beginning of this century,
but the automatic procedure for land use mapping are still problematic facing with land use complexity. This paper presents a
method of incorporating background knowledge into the object-based image classification procedure, intending to improve the classification accuracy and boundary consistency of land use data. Two forms of knowledge are used in this paper: the expert interpreted
land use polygons and the land use change rules. The idea of this paper is tested on the platform of Definiens Developer 7 (trial version). The proposed approach mainly contains three parts: segmentation supported by land-use thematic layer, classification and
change detection supported by land-use change rules. The experiment result shows that the proposed procedures have good potential
for automatic land use mapping.
certain land use type, and features may be time variant for certain land use type. Take the land use type “cropland” as an example, the spectral features are probably quite different before
and after the crop harvested. In addition, land parcel boundaries
are generally vague in image, and it is hard to get consistent
segmentation result for certain land parcel. Furthermore, the
same land parcel may have different boundaries, because of the
difference of the radiation characteristics among sensors and
the time variation influence.
1. INTRODUCTION
Object-based remote sensing imagery classification and information extraction approach have been widely tested and proved
to be a promising method for automatic image classification
and interpretation. The object-based approach has two main
advantages over the traditional pixel-based classification
method: one is reducing the salt-pepper effect, and the other is
utilizing multi features extracted from image objects to improve the classification accuracy. These features include the
spectral features, shape features, texture features, spatial relational features and semantic relations multi-level image object
hierarchy and class hierarchy.
Intending to improve the object-based image classification result for land use mapping, this paper introduces an idea of incorporating background knowledge into the object-based land
use classification procedures. The knowledge presented in this
paper mainly refers to two kinds: one is the knowledge persisted by expert in the process of manual interpretation, the
other is the land use change pattern knowledge.
Many efforts have been endeavoured into the application of object-based approach into image classification and information
extraction, including vegetation classification using very high
resolution satellite imagery or airborne imagery (Yu et al.,
2006; Mathieu et al., 2007; Mallinis et al., 2008), mangrove
and tree mortality mapping from IKONOS imagery (Wang et
al., 2004; Guo et al., 2007), vehicles detection from high resolution aerial photography (Holt et al., 2009), burn area and fire
type mapping from IKONOS imagery and NOAA-AVHRR
data (Gitas et al., 2004; Mitri and Gitas, 2006).
Traditionally, land use was mapped through field investigation.
With the remote sensing technique developing, land use can be
mapped from remotely sensed imagery by the means of manual
interpretation by domain experts. Experience shows that the
manually interpretation by trained interpreters with field investigation experience can obtain good classification accuracy.
There is certain knowledge an interpreter possess for image interpreting, including spectral, texture, shape, spatial relations,
semantic relations, and other features unknown. It is justifiable
to think that part of the unutterable features and rules remains
in the land use data which the interpreter produced. This research is intend to seek for a procedure to utilize the interpreted
land use as background knowledge to aid the object-based classification and information extraction, and it is the first kind of
knowledge used in this research.
In the field of land use and land cover mapping, as the land use
and land cover pattern of certain region (like the coastal region)
change rapidly and fiercely, it is of great importance to develop
automatic procedures to do the time consuming land use and
land cover mapping work. The object-based approaches show
great potential of automatic land information extraction in imagery, and several attempts have been made for land use mapping and change detection (Walter, 2004; Stow et al., 2007;
Rahman and Saha, 2008), but it still has a long way to go before reaching the goal of automatic procedures of land use
mapping and change detection.
The second kind of knowledge refers to the land use change
pattern. Land use types do not change randomly in space, but
they change following certain rules in certain regions. For example, “built-up land” is not inclined to change to other land
use types in a temporal scale of year. Land use change rules
When using the object-based approach for land use mapping, it
is hard to develop a consistent threshold of multi features for
525
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. 38, Part II
difference between the CBERS panchromatic (2.36) and multispectral image (20m). The relatively lower image quality of
the CBERS image place some challenges for the effective land
use extraction.
may vary from region to region, and the techniques to extract
these rules from data sources are still under development. We
employ simple land use change rules in this study, just for the
illustration of the procedure and framework of knowledge aided
object-based automatic land use mapping.
2. DATASETS
2.1 Study area
The study area is located in west coastal area of Chinese Pearl
River Estuary (figure 1). It lies in the west of Hongkong and in
the administration zone of Zhongshan city. Rapid urbanization
undergoes in this area since the China’s reform and opening up.
(a) Spot 5 pseudo-color image (NIR, R, G)
(b) CBERS 02B pseudo-color image (NIR,
R, G)
Figure 1. Location of the study area
2.2 Imagery
Two image sources are selected for this research: one is the
SPOT 5 image and the other is CBERS 02B image. The SPOT
5 image is a fusion image of SPOT 5 panchromatic (2.5m) and
multispectral image (10m). The fusion algorithm is Pansharpen
implemented in PCI software package, and the final image
resolution is 2.5m. Three bands are included in the SPOT 5 fusion image, NIR, Red, and Green. The pseudo colour combined
image (NIR, R, and G) is shown in figure 2(a). The imaging
time is October 23, 2003, and SPOT image is the source image
of the old land use polygon obtained by expert manual interpretation.
(c) CBERS 02B true colour image (R, G,
B)
Figure 2. Remote sensing imagery of the study area
2.3 Land use polygon data
The land use thematic data is from the Chinese Coast and Island Remote Sensing Investigation Project (Sun, 2008). The
land use is mapped through manually image interpretation from
SPOT 5 imagery and other ancillary data, the scale is about
1:50 000. The verification result from field investigation shows
that this dataset have more than 91% classification accuracy of
level 2 land use type. Five land use type appear in the study
area, namely cropland, shrub and grassland, forest land, builtup land, and aquaculture (figure 3).
The CBERS image is a fusion image of CBERS 02B High
Resolution (HR) (2.36m) and multispectral image (20m). The
fusion algorithm is also Pansharpen. The final image resolution
is 2.5m. Four bands are included in the CBERS fusion image,
namely NIR, Red, Greed, and Blue. The combined colour image is shown in figure 2(b, c). The scene date is January 5,
2009. The CBERS imagery is distributed by China Centre for
Resources Satellite Data & Application. The CBERS image is
used to generate the new land use classification.
By manually interpreting images in figure 2, one can see that
SPOT image have better detailed texture information than the
CBERS image, one possible reason is that too much resolution
526
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. 38, Part II
thematic layer. Then, based on the coarse level objects, more
detailed land parcels at a fine level (level 2) are segmented.
Level 2 objects are copied and they make a new level at level 3.
Level 3 objects are the sub-objects of level 2 objects, but in fact
they hold the same image pixels. In this research, level 3 objects are used for change detection.
The good segmentation result and the definite land use attribute
the level 1 objects hold will greatly benefit the classification
procedure in section 3.2.
Figure 3. Land use data of the study area
3. METHODS
3.1 Background knowledge aided image segmentation
The knowledge aided image segmentation methods is achieved
by incorporating land use thematic layer into image segmentation, and force the segmented image object to have the edge
which the land use thematic layer delineated.
This research employs multiresolution segmentation algorithm
(Baatz and Schäpe, 2000) implemented by Definiens (Definiens, 2007) to do the segmentation. Through the multiresolution segmentation, individual pixels are perceived as the initial
regions, which are sequentially merged pairwise into larger
ones with the intent of minimizing the heterogeneity of the resulting objects. The sequence of the merging objects, as well
the size and shape of the resulting objects, are empirically determined by the user. Initially, the layers, as well as their
weight, the parameters for homogeneity/heterogeneity, and the
crucial scale parameters are specified by the analyst (Benz et
al., 2004).
Figure 4. Image segmentation procedure utilizing background
knowledge.
3.2 Knowledge aided image classification
In this approach, image objects are classified not only by the
traditional multi features, but also by land use change rules.
First, the level 1 image objects land use are classified based on
the background land use thematic layer attribute “land use
type”, and the level 1 image objects classification represents
the old land use type. Then, the level 2 image objects are classified with the land use change rules and multi features. The land
use change rules used in this research is described in table 1,
and the features used for classification is shown in table 2. The
feature brightness and vegetation index in table 2 refer to the
object mean value.
The homogeneity criterion is calculated as a weighted combination of color and shape properties of both the initial and the
resulting image objects of the intended merging. The color homogeneity is based on the standard deviation of the spectral
colors. The shape homogeneity is based on the deviation of a
compact or a smooth shape. The computation of homogeneity
criterion f is illustrated in equation(1):
f = (1 − w1 )σ color + w1 ((1 − w2 )σ smooth + w2σ compact )
(1)
Where σ color refers to the color homogeneity, σ smooth refers to
Table 1. Description of land use change rules.
the smoothness shape homogeneity, and σ compact refers to the
Old land use
compactness shape homogeneity.
Cropland
Segmentation on several scales with different scale parameters
can be carried out leading to the formation of a hierarchical
network of objects. This procedure is constrained so that spatial
shape of objects in one level fits hierarchically into objects of
another level enabling consideration of sub-objects and superobjects and their relationships in the classification step.
Shrub and
grassland
Forest land
As in the introduction section explained, expert knowledge is
contained in manually interpreted land use data. This research
uses the land use polygon data as a thematic layer to help delineate proper land parcels. Detailed procedures are in figure 4.
Firstly, image is segmented at a coarse level (level 1) to delineate the old land use type which is indicated by the background
Aquaculture
Built-up land
527
New land use
Built-up land
Aquaculture
Cropland
Built-up land
Aquaculture
Shrub and
grassland
Built-up land
Aquaculture
Forest land
Built-up land
Shrub and
grassland
Aquaculture
Built-up land
Condition
BT > 50, VI < 0.1, Entropy >6
BT < 45, VI <0.05
Other condition
BT > 50, VI < 0.1, Entropy >6
BT < 45, VI <0.05
Other condition
BT > 50, VI < 0.1, Entropy >6
BT < 45, VI <0.05
Other condition
BT > 50, VI < 0.1, Entropy >6
NDVI > 0.15
Other condition
All condition
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. 38, Part II
Table 2. Description of the various object features used in the
study.
4.1.2
Abbreviation of
the feature
BT
VI
Entropy
As described in the method section 3.1, the coarse level objects
(level 1) are segmented first, then based on the old land use
polygon, fine level objects (level 2) are segmented. Level 3 objects are used for change detection, and they are copied from
level 2 objects. The detailed parameters used for segmentation
is shown in table 3. The segmentation result is shown in figure
6. The boundaries of level 1 object are consistent with the land
use map shown in figure 3.
Name of the feature
Brightness
Vegetation index
GLCM entropy
Computation method
(or reference)
(Red + Greed + Blue) / 3
(Nir – Red) / (Nir + Red)
Refers to (Definiens, 2007)
3.3 Change detection
Change detection is achieved by comparing the classification of
objects at level 1 and that of level 2. Classification at level 1 refers to the Old land use, and classification at level 2 refers to
the new land use. If the land use type of a level 2 object is different with the land use type of its super object at level 1, then
the sub-object at level 3 of this image object is classified as
“changed”, otherwise, the sub-object at level 3 is classified as
“unchanged”.
Segmentation result at multi level
Table 3. Parameters used for segmenting four image layer of
the multi-scale object
level
Level 1
Level 2
Level 3
scale
500
200
200
bands
Band 1-4
Band 1-4
Band 1-4
thematic
yes
no
no
shape
0.1
0.1
0.1
compactness
0.5
0.5
0.5
4. EXPERIMENTS AND RESULTS
4.1 Knowledge aided image segmentation
4.1.1 Comparison of segmentations without and with land
use polygons
To demonstrate the effect of segmentation method supported
by land use thematic layer, a test is conducted to compare the
segmenting result with or without the old land use polygons.
Segmentations are performed at CBERS layer 1-4, with scale
parameter 100. Result is shown in figure 5. The segmentation
result without land use delineate land parcels too fine boundaries, and it does not in consistent with the map generalization
rule; but the segmentation result supported by land use polygon
preserves the old land parcel boundaries very well, and some
possible type changed land parcel are delineated as well without conflicting with the old land boundary.
(a) Segmentation result at level 1
(b) Segmentation result at level 2
Figure 6. Segmentation result at level 1 and level 2.
(a) Segmenting without old land use
4.2 Knowledge aided image classification
Land use change rules are employed in the process of classification. Objects on level 2 are classified according to the rules
described in table 2. The features used for classification are described in table 1. Take the classification procedure of land use
type “aquaculture” for example, the detailed classification tree
is shown in figure 7. Classification procedure of other land use
types is about the same of type “aquaculture”, only the land use
change rules varied. The final classification result is shown in
figure 8.
(b) Segmenting with land use aided
Figure 5. Comparison of segmenting result of CBERS image
without and with land use polygons.
528
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. 38, Part II
4.3 Change detection
By comparing the land use type of level 2 objects (new land
use) with the land use type of their super-object (old land use),
the land use change status for each land parcel in level 3 are defined. The final result is as figure 9 shows, one land parcel
changed the land use type.
By manually interpreting the image in figure 1, the spectral feature of land use type “cropland” in the CBERS image is quite
different from that in the SPOT image. Generally used objectbased change detection approaches are hard to tell this kind of
difference, normally they result in false change detection in this
research condition. The pixel-based change detection approach
are not suitable for this condition, because the radioactive characteristics of the sensors (SPOT 5 and CBERS 02B) are quite
different.
Figure 9. Change detection result.
5. CONCLUSION
Object-based approaches are promising for automatic land
mapping, but there is still a long way to go to reach the goal
automatic. This paper introduced an object-based approach for
land mapping and change detection by utilizing background
knowledge. In the case study, the old manually interpreted land
use thematic layer and user defined expert land change rules
are employed as background knowledge, and aiding the segmentation, classification, and change detection in the proposed
procedures. The experiment result shows that there is good potential for the proposed knowledge aided object-based classification approach to become automatic procedure for land mapping.
The result of the automatic procedure of our approach is quite
consistent with manual interpretation. As described in section 2,
the image quality of SPOT 5 image is better than that of
CBERS 02B image, but high price of SPOT image limit the access of research community for such data. The procedure proposed in this paper also shows good potential for classification
and change detection for less quality images.
However, it has to acknowledge that the framework proposed
in this paper is only a prototype, and the exact knowledge used
in this paper is problematic when applying for other regions.
Further work has to be done to reach full automatic procedure
for land mapping by knowledge aided object-based approach.
Firstly, proper knowledge discovering techniques have to be
developed to extract effective knowledge, either in the form of
rules or patterns, which can be used for automatic land classification. Secondly, the optimal feature space for the classification of certain land type need to be discovered. Thirdly, knowledge and feature variation among different regions need to be
considered for further automatic procedure for large area (regional) land mapping.
Figure 7. Classification tree for land use type “aquaculture”.
ACKNOWLEDGMENTS
The research is supported by “863” Project of China
(No.2008AA121706, No.2009AA12Z148, No.2007AA12Z222).
We are grateful to the anonymous referees for their comments
and suggestions, and thanks are also due to Definiens Company
for supporting Definiens Developer trial version software package.
REFERENCES
Baatz, M., and Schäpe, A., 2000, Multiresolution Segmentation-an optimization approach for high quality multi-scale image segmentation. In Angewandte Geographische InformationsVerarbeitung XII, Karlsruhe: Wichmann Verlag, pp. 12-23.
Figure 8. Classification result for level 2 objects.
Benz, U.C., Hofmann, P., Willhauck, G., Lingenfelder, I., and
Heynen, M., 2004, Multi-resolution, object-oriented fuzzy
analysis of remote sensing data for GIS-ready information.
Isprs Journal of Photogrammetry and Remote Sensing, 58(3-4),
pp. 239-258.
529
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. 38, Part II
Definiens, 2007, Definiens Developer 7 - Reference Book,
Germany: München.
Gitas, I.Z., Mitri, G.H., and Ventura, G., 2004, Object-based
image classification for burned area mapping of Creus Cape,
Spain, using NOAA-AVHRR imagery. Remote Sensing of Environment, 92(3), pp. 409-413.
Guo, Q.H., Kelly, M., Gong, P., and Liu, D.S., 2007, An object-based classification approach in mapping tree mortality using high spatial resolution imagery. Giscience & Remote Sensing, 44(1), pp. 24-47.
Holt, A.C., Seto, E.Y.W., Rivard, T., and Gong, P., 2009, Object-based Detection and Classification of Vehicles from Highresolution Aerial Photography. Photogrammetric Engineering
and Remote Sensing, 75(7), pp. 871-880.
Mallinis, G., Koutsias, N., Tsakiri-Strati, M., and Karteris, M.,
2008, Object-based classification using Quickbird imagery for
delineating forest vegetation polygons in a Mediterranean test
site. Isprs Journal of Photogrammetry and Remote Sensing,
63(2), pp. 237-250.
Mathieu, R., Aryal, J., and Chong, A.K., 2007, Object-based
classification of ikonos imagery for mapping large-scale vegetation communities in urban areas. Sensors, 7(11), pp. 28602880.
Mitri, G.H., and Gitas, I.Z., 2006, Fire type mapping using object-based classification of Ikonos imagery. International Journal of Wildland Fire, 15(4), pp. 457-462.
Rahman, M.R., and Saha, S.K., 2008, Multi-resolution Segmentation for Object-based Classification and Accuracy Assessment of Land Use/Land Cover Classification using Remotely Sensed Data. Photonirvachak-Journal of the Indian
Society of Remote Sensing, 36(2), pp. 189-201.
Stow, D., Lopez, A., Lippitt, C., Hinton, S., and Weeks, J.,
2007, Object-based classification of residential land use within
Accra, Ghana based on QuickBird satellite data. International
Journal of Remote Sensing, 28(22), pp. 5167-5173.
Sun, X.Y., 2008, Analysis of Exploitative Intensity of Coastal
Zone Area - A Case Study on the Coastal Zone of Eastern Part
of GuangDong (Ph.D Thesis): Graduate School of Chinese
Academy of Science, Beijing.
Walter, V., 2004, Object-based classification of remote sensing
data for change detection. Isprs Journal of Photogrammetry
and Remote Sensing, 58(3-4), pp. 225-238.
Wang, L., Sousa, W.P., and Gong, P., 2004, Integration of object-based and pixel-based classification for mapping mangroves with IKONOS imagery. International Journal of Remote Sensing, 25(24), pp. 5655-5668.
Yu, Q., Gong, P., Clinton, N., Biging, G., Kelly, M., and Schirokauer, D., 2006, Object-based detailed vegetation classification. with airborne high spatial resolution remote sensing imagery. Photogrammetric Engineering and Remote Sensing,
72(7), pp. 799-811.
530