ON-BOARD CHANGE DETECTION WITH NEURAL NETWORKS
M. Spigaia, *, G. Oller a,
a
ALCATEL SPACE INDUSTRIES, 26 av. JF. Champollion, BP 1187, 31037 Toulouse, France (marc.spigai@space.alcatel .fr, guillaume.oller@space.alcatel .fr)
KEY WORDS: Change Detection, Neural, Networks, Classification, Edge.
ABSTRACT:
The aim of this paper is to describe a potential on-board change detection chain by earth observation satellites (optical and SAR).
The benefits of such an on-board chain are multiple : reduction of the amount of data to transmit from board to ground and
autonomy of the system for example. We describe particularly one component of the chain : a detection/classification module based
on the neural network, this type of algorithm being a part of a more global future fusion-based classification module. First
experiments have allowed to validate the algorithm based on neural networks and first results are satisfying. The continuation
consists first to enlarge the classification module with the implementation of other classification algorithms and to compare them
with a more exhaustive set of data.
1. INTRODUCTION
With the increase of on-board satellite computation capabilities,
certain tasks, processed nowadays on ground, will be treated
directly on-board in the next future.
The paper is in the field of on-board change detection between
two images taken at different times by optical or radar SAR
earth observation satellite sensors.
The benefits of an on-board change detection is multiple. By
determining in an image what change of interest (in regard to a
particular mission) happened, it will be possible, among others :
To reduce the amount of data to transmit from board to ground
by sending only the changes,
To have a hierarchy in data transmission by sending relevant
information first, and less relevant information after,
To enhance autonomy of systems,
To exploit at best on-board capacities ...
In the paper, we propose a potential on-board change detection
chain by earth observation satellites (optical and SAR) and we
describe and validate the principle of one component of the
detection/classification module based on the neural network.
First experiments of validation are also given.
used for the information where less relevant information is
present in regard to the mission.
High compression
ratio
Change of interest : Low
compression ratio
Compression
Transmission
Decompression
2. PROBLEMATIC OF ON-BOARD CHANGE
DETECTION
2.1 Problematic
We suppose that an observation satellite has on-board two
images of a given area taken at time t1 and t2 with t2>t1. The
image 1 is the reference image. The problem is to detect onboard changes of interest between the two images for a given
mission (ex.: research of planes).
As we have introduced it in the previous section, this process
has many interests. We illustrate hereafter the reduction of
amount of data transmitted to the ground by the use of selective
compression : low compression ratio (few distortions) is applied
to the changes of interest detected and high compression ratio is
Figure 1. Selective compression after change detection
2.2 Hypothesis
The two images are supposed to come from spatial optical or
radar sensor. In this paper, we focus on two images of the same
nature : optical/optical (visible) or SAR/SAR (radar). We place
us in the field of high and very high resolution sensors : metric
ON-BOARD CHANGE DETECTION WITH NEURAL NETWORKS
Pecora 15/Land Satellite Information IV/ISPRS Commission I/FIEOS 2002 Conference Proceedings
and submetric, the resolution of images 1 and 2 being very
closed.
A typical on-board processing chain using change detection is
illustrated below :
Data
Under the terms change detection, we put in the paper the
detection/classification. The reason is that the classification is
used here to enhance the global performances of the detection.
2.3 Limitations
Pre-processing
Change detection with Automatic Target Recognition (ATR)
has a number of well-known problems, in particular:
•
The environment is by nature unstable: humidity,
temperature, luminosity,
Change detection
•
There are residual errors after pre-processing which can be
interpreted as changes,
•
The number of target classes and aspects is very high,
•
There are inconsistencies in the SAR signature of target.
Compression
Figure 2. Typical on-board processing chain using change
detection
When data have been acquired, a set of pre-processing is
necessary to put the two images in the same space
representation :
Image geometric co-registration allows to represent data in
the same geometric space. We suppose that the two
different orbits at time t1 and t2 are close enough to avoid
strong distortions due to very different satellite points of
view.
Resolution of the two images is supposed to be very
closed. If there is a slight difference, resolution will be put
in a same common value.
Radiometric calibration can be applied in order to
compensate radiometric non relevant differences.
SAR processing allows to transform radar SAR raw data
into an interpretable SAR image. This task is very
exepnsive in computation time. Concerning the case of
SAR images, only the magnitude of the complex
information is taken in account in our work, no phase
information has been introduced in algorithms and no
multi-polarization images are used.
An image filtering can be applied before the change
detection, a compromise should be found between
smoothing and loss of relevant information.
The set of pre-processing is supposed to be done before the
change detection.
Everything can change between the two images and the
problem of change detection is a real challenge, even on ground
with big computation resources. On-board ATR is subject to the
CPU and memory limitations due to resources which are limited
compared to the ground. To be realistic, we deliberately restrict
the problem and we suppose in the stage of the work that the
user as defined a kind of target of interest, the number of targets
of interest being equal or less than two (even equal to one in the
first experiments we have done).
3. OVERVIEW OF A POTENTIAL CHANGE
DETECTION PROCESSING CHAIN
In the on-board change detection that we describe here, there
are three main steps illustrated hereafter: detection,
classification and selection of change of interest.
Image 1
pre-processed
Image 2
pre-processed
Detection
Classification
Targets of
interest
Concerning the change detection, one can distinguish :
Detection: a changed is detected in an image,
Classification: the change detected is classified (ex:
plane/not plane),
Selection of change of
interest
Identification: the recognized object is identified (ex: a
plane of type A),
Figure 3 : on-board change detection processing chain
Analysis: parameters of the classification (ex: speed and
direction of the plane).
ON-BOARD CHANGE DETECTION WITH NEURAL NETWORKS
Pecora 15/Land Satellite Information IV/ISPRS Commission I/FIEOS 2002 Conference Proceedings
Detection: The aim of this step is to detect as many as possible
“real” changes of interest between the two images.
Performances are generally expressed in terms of probability of
detection (Pd) and probability of false alarm (Pfa). Typical
algorithms are used and described in the next paragraph.
Classification: Many classifiers exist. On a general point of
view, our aim is to test N algorithms, each of them having its
own properties in terms of performances, robustness and
reliability. A final fusion algorithm will combine the results of
each classifier.
This classification structure allows to take in account different
classifiers and to active them in function of the context and the
situation (including CPU time available for example). In the
article we test one of the N classifier : a neural network
classifier with a pre-processing based on edge extraction
combined with mathematical morphology.
Detection
A typical processing applied to detect changes between two
images is done by the computation of a correlation coefficient ρ
and its comparison to a threshold:
ρ=
E [I 1 I 2 ] − E [I 1 ]E [I 2 ]
var( I 1 ) var(I 2 )
(1)
E =expectation, var = variance, Ii = window in the image i.
Two geographic corresponding areas in image 1 and 2 have a
high correlation coefficient if there is no relevant change
between the two areas.
In practice, to avoid bad computation due to imperfection in the
data pre-processing (residual errors in co-registration for
example) an area is determined in image 2, the correlation
kernel, and the maximum ρ is computed on a neighborhood of
the corresponding area in image 2, as illustrated in the next
figure.
In the case of SAR, correlation computation is often applied
even if the noise is multiplicative for SAR image. An
alternative is to use a feature based correlation measure rather
than a classical correlation coefficient.
Pre-processing 1
...
Pre-processing N
...
Classifier 1
Classifier N
Fusion of classification results
Correlation kernel
Figure 4: Fusion of classifiers
Selection: after changes being classified, the selection keeps the
changes of interest in regards of the mission. This step is
considered in its simplest version and is not described in our
paper.
4. TECHNICAL DESCRIPTION OF THE CHAIN
Neighborhood of research of maximum ρ
In this paragraph, we describe the detection and
pre-processing/classification steps of the change detection chain
described in the previous paragraph.
In practice, the image 2 is split in blocks, and at the output of
the detection a decision is taken for each block: change
detected/change not detected.
Figure 5: Correlation computation
4.1 Detection
As input of the detection, we have the images 1 and 2 preprocessed.
The output of the detection will be the image 2 split in blocks, a
decision being taken for each block : change detected/change
not detected.
= Changes detected in the image 2
Figure 6: Output of the detection
ON-BOARD CHANGE DETECTION WITH NEURAL NETWORKS
Pecora 15/Land Satellite Information IV/ISPRS Commission I/FIEOS 2002 Conference Proceedings
4.2 Classification pre-processing: edge extraction,
morphological operations
As input of the classification pre-processing, there are blocks of
image 2 where changes have been detected. The output will be
a classification pre-processing of these “change” blocks.
Whatever the classifier, classification pre-processing is a crucial
step where data are represented in a space which, hopefully,
enhances classification performances. We have chose to apply
an edge extraction as a pre-processing.
Concerning optical images, for extracting edges a Sobel filter is
applied [3]. The principle consists to find places in the image
where intensity values change rapidly. The Sobel filter
computes gradient on horizontal and vertical axis at the position
[i,j] of the image by convolution:
∂A ∆A
≈
= hi * A[i, j ] ,
∂y ∆i
∂A ∆A
≈
= h j * A[i, j ]
∂x ∆j
with
2
1
1

0
0  ,
hi =  0
− 1 − 2 − 1
(2)
1 0 − 1 
h j = 2 0 − 2
1 0 − 1 
detection. In fact edges too close to others risk not to be
detected. So Fjørtoft proposed to used the ROEWA operator.
The intensity values in each windows are weighted
exponentially to compute the averages (µ1 and µ2). The
maximum ratio is chosen :
µ µ 
rmax = max 1 , 2 
 µ 2 µ1 
(5)
Fjørtoft showed that it is an optimal tradeoff between
localization precision and speckle reduction when the
reflectivity jumps follow a Poisson distribution. ROEWA is
computed vertically and horizontally, then with analogy to
gradient-based operators for optical images, the magnitude is
computed.
As for the correlation coefficient computation, in practice, to
avoid bad computation due to imperfection in the data preprocessing or in the edge extraction, a morphological operation
based on dilation-erosion is applied to the edge image. So
finally the classification pre-processing is edge extraction and
dilate/erosion morphological operations.
We illustrate hereafter the classification processing of a plane in
an optical image:
hi et hj, also called masks, are the convolution kernel of an
impulse finite response filter.
Gradient norm is then given by:
2
∂A
∂A
+
∂x
∂y
∇A[i, j ] =
Figure 7: optical Image and classification pre-processing
2
4.3 Classification with neural networks
(3)
As input of the classification, there are pre-processed blocks of
image 2 where changes have been detected.
Concerning the SAR, operators used in optics are not adapted to
SAR imagery because their False Alarm Ratio (FAR) depends
on the radiometry [4], which makes impossible a good
threshold setting for edge extraction. Adaptations have been
proposed (log transformation, normalization) to have a Constant
False Alarm Ratio (CFAR), but considering an edge as a
maximum of the first derivative is not adapted to the
multiplicative nature of the speckle noise.
We used the Ratio Of Exponentially Weighted Averages
(ROEWA) operator proposed by Fjørtoft [4].
Let us consider two windows on both sides of a pixel, with Î1
and Î2 as average intensity. The Ratio Of Averages (ROA)
operator can be defined as :
 Iˆ
rm = max 1
ˆ
 I2
,
Iˆ2  or

Iˆ1 
 Iˆ
rn = min 1
ˆ
 I2
,
Iˆ2 

Iˆ1 
As output of the classification, each block will be classified as
target of interest/target not of interest.
4.3.1
Neural networks
The first motivation of the development of neural networks was
to reproduce the human capabilities for some tasks that
computers reproduce imperfectly and with heavy computations.
The recognition of objects independently of their size,
orientation and environment is particularly eloquent. This idea
failed in 1960 because of the lack of mathematics tools adapted
to the conception and analysis of complex networks. Research
had restarted around 1980, with success. But even today, a real
reproduction of the human neural network is still not reached
for the simple reason that nobody really knows the real working
of human neural networks.
(4)
A formal neuron makes a non linear function with parameters w
(weights) of the input x : Y = f(x1, x2,..xN ; w1,w2, ...wp). As
an example, the sigmoid function hyperbolic tangent is often
used
This operator exhibits a CFAR and better results than
differential methods. But large sizes of windows are necessary
to filter the speckle, and it raises the problem of multi-edge
ON-BOARD CHANGE DETECTION WITH NEURAL NETWORKS
Pecora 15/Land Satellite Information IV/ISPRS Commission I/FIEOS 2002 Conference Proceedings

y = th  w0 +


n −1
∑ w x 
(6)
i i
i =1
The process of estimation of the weights w is called learning. In
the paper, the learning is supervised : that means that some
points (x,y) are known and allow to estimate the weights. The
estimation is often done by methods based on the gradient
algorithm. When the weights are estimated, the neural network
is able to give an estimate of the output y for a given x (a priori
not known).
The neuron simply realizes a parametric non-linear function of
its input. One interest comes from their association in networks
and the properties which follow from this composition of nonlinear functions: it can be shown that the number of adjustable
parameters is as small as possible. This can be important when
the CPU resources are limited for example (on-board).
As illustrated on the next figure, a typical neural network
structure is composed of input, output, weights and hidden
layers. The structure of neural networks, including the number
of hidden layers, is generally determined by numerical methods
of model conception.
Block image
•
•
•
•
...
•
•
LVQ
Neural
Network
Classification target :
Of interest /
Not of interest
Figure 9 : Classification by LVQ
For a particular object (ex: a plane): the structure of the neural
networks is determined by a set of example of target of interests
and non-target of interest. The size of input, number of hidden
neurons depends on the problem.
5. EXPERIMENTS
A few first experiments have been done to validate the principle
of this chain and the use of neural networks. We show here two
typical application of the chain on a optical image and on a
SAR image.
Both cases are changes simulated : the image 1 is a real image
and the image 2 with changes has been created by adding noise
and removing or adding targets of interest.
Images 1 comes both for optical and SAR from airborne data
and are representative of high or very high resolution earth
observation satellite.
For the optical case, the image (390x650) comes from airborne
and is shown here comes from airborne with high resolution
around 2m. In the case of SAR, the image (215x195) shown
comes from airborne with resolution very high resolution of
0.1m and has been downloaded at [6].
Block sizes for correlation processing are (15x25) for optical
and (40x40) for SAR.
Figure 8: Typical neural networks structure
4.3.2
Neural networks and classification
Target of interest are planes for the optical image and tank for
the SAR image.
Neural networks are used in many domains including the
classification. For this particular case, because of progress
carried out in the comprehension of fundamental properties of
neural networks, it has been shown that neural networks not
only give a binary decision : they can give for an object its
probability to belong to every class, this allow neural networks
to integrate classification systems based the fusion of many
classifiers, as we intend to test it for on-board change detection.
For both chain the detection has been done by correlation.
Sobel filter is use for optical edges extraction and ROEWA for
SAR edges extraction. The LVQ neural network has 10 neurons
on the input layer. A set of 20 objects have been used for
learning in the both cases.
4.3.3
The classification with neural networks gives good results (in
terms of performance of classification) in this first stage of
simulation. The structure used with LVQ seems a good
compromise bias-variance and it yields a network base that can
now be grown.
Neural networks for on-board change detection
In the change detection chain described in our paper, to classify
objects of interest with neural networks we used the principle of
Learning Vector Quantization (LVQ) developed by Kohonen in
its work on self-organizing maps [5].
For both cases, we present hereafter typical images given by the
detection/classification.
ON-BOARD CHANGE DETECTION WITH NEURAL NETWORKS
Pecora 15/Land Satellite Information IV/ISPRS Commission I/FIEOS 2002 Conference Proceedings
A first comparison with an algorithm based on the direct
correlation with elements of the database gives promising
results: performances in terms of classification are similar for
both algorithms but the neural network needs less operations
(around 30% or more) and less memory because its does not
need to have the database in memory. This was already
mentioned in the paragraph on neural network : the number of
adjustable parameters is as small as possible, this explains the
gain in complexity of the neural networks.
The idea was to validate the principle of this chain on a set of
first simulations. The limitations of the neural network are usual
ones: results are highly dependant on the exhaustivity and
quality of the learning data set relative to the problem to solve.
This should be study in a more complete performance analysis:
we intend to implement other classifiers in the change detection
chain and compare them in terms of performances, robustness
and reliability with a more exhaustive set of simulated data.
5.1 Change detection on the optical image
The following figures represent:
•
The blocks where a change has been detected in image 2
after thresholding the correlation measures.
•
Figure 11 : Image 2 after classification of plane
5.2 Change detection on the SAR image
The following figures represent:
•
The blocks where a change has been detected in image 2
after thresholding the correlation measures.
•
The blocks where a tank has been recognized in image 2
after classification: the result of the classification is
accurate because only the tanks are kept on the image.
The blocks where a plane has been recognized in image 2
after classification : the result of the classification is
accurate because only the plane is kept on the image.
Figure 12 : Image 2 after detection of changes by correlation
Figure 10 : Image 2 after detection of changes by correlation
Figure 13 : Image 2 after classification of tanks
6. CONCLUSION
The aim of this paper was to describe a potential change
detection chain by earth observation satellites (optical and
radar) and to validate the principle of one component of the
detection/classification module based on the neural network.
First experiments allowed to validate the algorithm based on
neural networks and first results are satisfying.
ON-BOARD CHANGE DETECTION WITH NEURAL NETWORKS
Pecora 15/Land Satellite Information IV/ISPRS Commission I/FIEOS 2002 Conference Proceedings
The continuation of this paper consists first to enlarge the
classification module with the implementation of other
classification algorithms and to compare them in terms of
performances, robustness, reliability and implementability with
a more exhaustive set of data.
7. REFERENCES
Chen, C.H., 1989. Information processing for remote sensing.
Chen, , pp. 317-336.
Javidi B., 2002. Image recognition and classification. Dekker,
New-York, pp. 1-36, 61-100.
J-P. Cocquerez , S. Philipp, 1995, Analyse d'Images : Filtrage
et Segmentation, Masson, Paris.
R. Fjørtoft, P. Marthon, A. Lopès, E. Cubero-Castan, 1998, An
Optimum Multiedge Detector for SAR Image Segmentation,
IEEE. Tr. on Geoscience and Remote Sensing, 36(3):793-802.
Kohonen, T., 1987. Self-Organization and Associative
memory, 2nd Edition. Springer-Verlag, Berlin.
General Atomics. http://www.airforcetechnology.com/contractors/surveillance/general/index.html
(accessed Nov. 2001)
ON-BOARD CHANGE DETECTION WITH NEURAL NETWORKS
Pecora 15/Land Satellite Information IV/ISPRS Commission I/FIEOS 2002 Conference Proceedings