Robust Estimation of 3-D Line Segments from Satellite Images
for Model Building and Change Detection
Ibrahim Eden and David B. Cooper
Division of Engineering, Brown University
184 Hope St, Providence, RI 02912, USA
{ieden, cooper}@lems.brown.edu
Introduction
matching and 3-D line segment reconstruction from multiple satellite images. We show that the proposed line matching method substantially decreases the complexity of the
problem size in comparison with other methods. This paper also describes an approach for automatically computing
free-form wireframe models based on individual 3-D line
segments.
This paper addresses how to automatically match and reconstruct line segments from multiple satellite images and detect changes in man-made structures by using reconstructed
3-D line segments. We propose a new approach to line reconstruction and use wireframe models to better estimate the
actual 3-D geometry by using the incidence relations between reconstructed line segments. Our change detection
method is widely applicable as man-made structures are the
main focus of most change detection applications.
Detection of structural changes in images of the same
scene has become a key issue due to a large number of
applications in computer vision. Important applications of
change detection are video surveillance, urban planning,
agricultural analysis and military intelligence (Huertas &
Nevatia 1998; Radke et al. 2005). The core problem of
change detection is to identify sets of features (pixels, regions, lines, etc.) that are significantly different than previous images of the same scene.
Some of the early work on change detection focuses on intensity values of pixels and regions. The main drawback of
these methods is their high likelihood to create false alarms
in cases where pixel values are affected by viewpoint, illumination, seasonal and atmospheric changes. This is the reason why intensity (pixel and region) based change detection
algorithms such as simple differencing methods (Bruzzone
& Prieto 2002) and background modeling methods (Stauffer
& Grimson 1999) fail in such scenarios.
The notion of “important change” varies according to the
focus of the application. Generally the main focus is on
man-made structures, especially buildings. Considering that
most man-made structures consist of 3-D line segments, a
change detection system based on modeling the objects in a
scene using 3-D line segments is promising because it may
provide more accurate and reliable results.
In this study we have been interested in developing a line
segment based change detection method. This rarely studied
approach requires attention to develop more efficient and accurate algorithms that can be generalized to complex problems. In order to make our change detection approach more
efficient and accurate, we propose novel techniques for line
Satellite Imagery and 3-D Line Segment
Reconstruction
In this section, we first give a brief description of the cubic
Rational Polynomial Coefficient (RPC) camera model that
maps world coordinates to the image plane in which each
component of image coordinates is the ratio of two cubic
polynomials of latitude, longitude and height. The purpose
of this model is to represent the relationship between the image plane and the ground surface more accurately by using
a simple, generic set of equations (Dial & Grodecki 2005).
One important property of the RPC camera model is normalizing image and ground coordinates to a range of ±1.
The entire complexity of a satellite camera is described by
10 normalizing parameters (a scale and an offset value for
each of the object space coordinates) and 80 polynomial coefficients (20 coefficients for each cubic polynomial).
As the reader might notice, the RPC camera model is different than the projective camera model in many ways. One
difference, and probably the most important one, is that unlike projective cameras, for the RPC cameras, a point in the
image plane is not back projected into the 3-D space as a
straight line. On the other hand, it is possible to transfer a
point in the image plane into a point that lies on a known
3-D surface in the world coordinate system using surface
parametrization and gradient based nonlinear optimization
techniques. For simplicity, we restrict our surface models to
the set of planes normal to the Z-axis.
Line segment matching over multiple images is known to
be a difficult problem due to its exponential complexity requirement and challenging inputs. Some methods have been
proposed in order to make this process more efficient and
more accurate (Baillard et al. 1999). Unfortunately, most
of these methods are not applicable to our domain, since the
epipolar geometry does not exist between pairs of satellite
images. On the other hand, by exploiting the fact that the
c 2007, Association for the Advancement of Artificial
Copyright Intelligence (www.aaai.org). All rights reserved.
1854
matching problem in satellite images is similar to the matching problem for small baseline aerial images, we can explain
most of the correspondences with a local planar homography. Our method uses local planar homographies between
pairs of satellite image patches as a guideline during the
matching of corresponding 2-D line segments. It should be
noted that these geometric relations between pairs of satellite images are also supported by photometric consistencies
along the support regions of matching line segments.
The geometry of each 3-D line segment is estimated using the matching results between pairs of images. This process is done in two separate steps: a consistent and robust
initial estimation step with 2 degrees-of-freedom (2-DOF),
and a fine tuning step with 6-DOF. First we choose a reference (base) image and optimize the 3-D line segment, so
that it perfectly fits the 2-D line segment in the reference
image. The second step of the reconstruction process is the
estimation of the actual line segment in the unconstrained
3-D space where the line segment has 6-DOF. The result of
the first step is used as the initial estimation in the second
optimization process.
tions. These methods are unreliable as they fail to provide
desirable results around high frequency regions. Considering that most man made structures consist of linear features,
a model that makes use of such features promises to provide more robust and consistent results (Huertas & Nevatia
1998).
The change detection method we propose in this work is
a generic finite state machine for transducing the results of
reconstruction for each 3-D line segment to detect changes.
Note that, this method uses the final result of 3-D reconstruction and assumes that the images are sorted in chronological
order.
Conclusion and Future Work
Our algorithm paves the way for a completely automated
3-D reconstruction system for RPC camera models and a
line segment based change detection system. In this paper,
we present a novel approach for change detection based on
matching and reconstruction of 3-D line segments from multiple satellite images. While our preliminary results show
that the algorithm is capable of matching and reconstructing
line segments in selected regions efficiently and accurately,
there still remain several issues such as robust detection of
2-D line segments from low resolution satellite images.
The main contribution of this paper is the design of algorithms for efficient and accurate reconstruction of line segments from multiple satellite images and the design of a general change detection engine based on the results of reconstruction. We believe that feature based change detection
algorithms will perform better than intensity based change
detection algorithms. Future directions include the design
and analysis of algorithms to reconstruct more complicated
models that can capture the geometry of different types of
buildings and improving the change detection machinery by
building a dynamic Bayesian network.
Figure 1: Left: Reference satellite image. Center: Region
of reconstruction, red lines indicate the projection of reconstructed 3-D line segments. Right: reconstructed 3-D line
segments
In some cases, minimizing the sum of projection error
may not provide the best 3-D reconstruction. It is also useful
to consider incidence relations between close line segments.
To achieve a better estimation of the actual 3-D geometry,
we form a wireframe model based on the matching results
and reconstructed 3-D line segments. For each wireframe
model, we further optimize the 3-D structure where each
incidence relation puts more constraints on the estimation
procedure. It is interesting that using the wireframe model
leads to worse results in terms of total projection errors and
better results in terms of capturing the real 3-D geometry.
References
Baillard, C.; Schmid, C.; Zisserman, A.; and Fitzgibbon,
A. W. 1999. Automatic line matching and 3D reconstruction of buildings from multiple views. In ISPRS Congress,
69–80.
Bruzzone, L., and Prieto, D. F. 2002. An adaptive semiparametric and context-based approach to unsupervised
change detection in multitemporal remote-sensing images.
IEEE Transactions on Image Processing 11(4):452–466.
Dial, G., and Grodecki, J. 2005. RPC replacement cameral
models. In ASPRS.
Huertas, A., and Nevatia, R. 1998. Detecting changes in
aerial views of man-made structures. In ICCV, 73–82.
Radke, R. J.; Andra, S.; Al-Kofahi, O.; and Roysam,
B. 2005. Image change detection algorithms: a systematic survey. IEEE Transactions on Image Processing
14(3):294–307.
Stauffer, C., and Grimson, W. E. L. 1999. Adaptive background mixture models for real-time tracking. In CVPR,
246–252.
Line Segment Based Change Detection
Change detection between different images of a scene taken
at different times has been important in many computer vision applications in detecting the effect of different events.
A number of different algorithms for the change detection
problem have been proposed in the literature; they span a
wide range of approaches from difference images to background modeling methods based on Gaussian mixture models (Stauffer & Grimson 1999). Generally pixel or region
based methods fail to detect changes in the case of severe
environmental changes such as lighting and weather condi-
1855