MULTISTAGE IMAGE MATCHING
Luisa Maria Gomez Pereira, Branko Makarovic
International Institute for
Survey
and Earth Sciences (ITC)
350, Boulevard 1945
7500 AA Enschede
Commission II
ABSTRACT
In this paper, attention is
to the two
matching of strings of image primitives in
tree matching from coarse to fine. The
digitised di
in
lines.
for off-line preprocessing and image
The development reported here is
I. INTRODUCTION
The
impact
of
rapidly
photogrammetry is especially
related operations.
One
matching is automatic modelling
to devise an effective overall
automatic measurement of the
images.
Because the amount
attain
ty,
process has to be time
paramount
the
accuracy
and
reliability
of image matching are
significance. A flexible multistage strategy of processing "from coarse to
fine", with adaptive capability incorporated, tends to provide an adequate
solution of the problem.
The strategy proposed here addresses four main sequent
stages. The first
stage predicts image match approximately by exploiting a priori knowledge
about the geometry of a photogrammetric stereo-model. The second stage
proceeds with course matching by means of the dynamic programming technique
applied to low frequency (spatial) image informat
The corrected
parallax values are entered in the next matching stage.
third
...........--::---:---:::-"''-uses as the input image segments structured in several
The coarse to fine matching proceeds down
branches of
cal
tree.
The fourth stage is intended for fine matching
the required
accuracy is very high. It uses the a priori information, including that
gained in the preceding matching stages, in conjunction with a set of
selectable algorithms for fine matching.
The first matching stage, concerning match prediction,
st
foreward and is therefore considered here only marginally. Hence, attention
has been focused on the two intermediate s
(2 and 3), which
for
sufficient "pull-in" and thus reliability.
attainable
will
presumably meet
requirements for
common applicat
terrain models (DTM). The development of
two stages
not yet
[ 2] ;
fully completed. The initial software has been developed and tes
some optimization, however, is an issue for further development.
last
stage, pertaining to fine matching, is beyond the scope of
paper.
The paper concerns the concepts for a multistage st
and for the
individual matching stages.
Most of these concepts
emerged in the
course the authors' work since 1975, [3], [4], [5], [6]. An outline is
presented of the procedures, and of
software and its testing. Moreover,
consideration is also given to further development.
II. CONCEPTS FOR MULTISTAGE MATCHING
The concepts concern the
strategy, the
a
the operations
photogrammetric model, including
geometry,
involved in the four main matching stages. In the follwing a review is
presented of these concepts and their interrelationships.
1) Process strategy
The concepts concern the arrangement of the four main
their interrelationships (
1)
Stage 1:
s
and
Match prediction
j
Fig. 1: Sequent
Stage 2:
Coarje matching
Stage 3:
Coarse to fine rna tching __ j
Stage 4:
Fine matching
~
J
matching stages
In stages 2 and 3 their main phases (preprocess, mat
and
form feedback loops which provide for optimization (figure 2).
postprocess)
~
Prep1ocess3J
t
Matching
.-:-J
feedback loops
-- - - --for
optimizing
___j
Post~rocess
Fig. 2: Optimization in stages 2 and 3
Table 1 presents a list of the concepts for each of the
stages. The lists are open ended and liable to alterations.
four
matching
2) Geometry of photogrammetric images and stereomodels
The basic geometric concepts can be found in photogrammetric textbooks and
will therefore not be considered here.
Implementation of the epipolar
geometry in analytical stereoplotters for digitising of stereo-images is,
however, of special significance. A simple geometric situation
attained
when using only the rotation parameters for the relative orientation.
Hence, the base B is parallel to the X-axis in virtual model space XYZ
(figure 3).
II
1
Stage 1: Match
prediction
matching
Photogrammetric
Epipolar geometry
geome
Edge extraction
Edge description
String matching (dynamic programming)
Epipolar parallax
le
Other related
epipolar
parallax
profiles
to fine
) profiles
target
Hierarchical bynary tree (of image segments)
Search segment size
Edge extract
ion (as in s
2)
techniques
Adjacency
Epipolar parallax
le
Other related
matching
1: List
. 3: Simpli
Joint analys
of parallaxes
of the corresponding images,
Matching techniques
ion of the technique
Quality control
Edit
Other
multis
geometry (X!IB!IE)
u. .
The
to
The
the line
c is the
realised
X
program
set of
cover
entire
of the sensor
information
subroutine
line
rs e'
further use
3
) is essential. Af
relative
correct
it
base to
ction
convergency
some
ter can
the
1)
is direc
linked with
3. A fur
level
built in
cal
t
)
.
t
Gaussian)
st
lines)
Fig. 4:
strip
In contrast to stage 3, the strips in stage 2 are not segmented (for a
binary tree structure . The
for the extraction and description
of edges
tives , however, are identical in both stages.
Hence, in s
2 (
n) the
are
)
where
level
is the
size
tal images in the lowest
The
levels
n-1) are built
by
the same composite
of modified Gauss
the
(or more)
between
acent lines is tripli
in
to the next
level
For the extraction
to the filtered
(in level
process the
value can be automat
adjusted.
subroutine attributes to each edge a set of
(vide stage
limit the search
to matching, the maximum expected
correction (to
cted one in stage 1) should be assessed. By means
of a mirror s
and a parallax bar, the approximate parallax
difference can be measured between the highest
the lowest
in the
stereo-image, and a safety margin is added.
First the similarity is
Matching comprises two steps.
potentially conjugate edge-pair within the search range. To
, the
corresponding edge
ptors are compared.
These
ptors can be
weighted according to their impact on the assessment of similari
Edge
pairs with opposite polarity (e.g.
of the
or of
second
di
)
ected. The similari
in other
is tangeble
and
by the
merit.
ine measures the
degree
s
ty for each descri
separately, and converts it,
a
look-up table, into the corresponding figure of merit. If the merit is less
than the value of the corresponding acceptance threshold, the edge-pair is
rejected.
Otherwise an additional composite similarity is assessed, e.g.,
the weighted mean (S) of the similarity
est
for
individual
descriptors.
mean values (
are
against
threshold.
The second s
concerns matching a string of potentially conjugate edgepairs
in the first stage. A suitable technique for string-matching
is dynamic programming with constraints [1]. To this end, the composite
similari
estimates (S) are arranged in a matrix (
5).
The
t
rs is obtained by the path of the
to
maximum
computation of
potent
paths is by
algorithm [1]. The geometric constraints restrict
the number of variants in each state (
) to the angular segment (quadrant)
between the
itive directions of bo
epipolar lines. After matching the
conjugate st
of edges, the coarse epipolar parallaxes are known
all
of the edge-pairs.
Together they form a coarse epipolar
profile
which serves as the input for the matching stage 3.
II-
RH
line
He ected
edge
-x
range
LH
line
.
5:
maximum simi
ty
2)
The
the
core in
which is
'
coarse
in
rect
lines
(level 1)
X
6a:
tree of
lines
. 6b:
the search range in the
corrections)
termined by
The
of a
the
are
of
red
the
n). If
search
In the lower hierarchical levels
similar
from
ts
the search
in
of
level i) are
and a half
to the left of
uniform structure of the
ion of common
s
'
accepted pairs. This
segments
ordered
conjugate hierarchi
accumulat
rs
a)
In each
matched
matched, the mean
within
tree
correction value
This mean
is
is
of
range
segments on
olo.r line
)
. 7:
b)
The
are
matched
( 1).
t
A
the process
the branches of
in the lowest level
correction
lines, from
For
r
paper.
For
may not be
IV INITIAL TESTS
the subroutines for
this
tal
Co.
array
up and s
li
of
both video cameras
Some
time,
process
carried out
for preprocess
) the
tware modules were
address
of
tion
ion and ege
the
for s
,
3.
the baseratio, the convergency (
, the
thms and measures for similari
assessment, and
were tested one
Then the
optimization, however, have not yet
of further
These
one.
V. FINAL CONSIDERATIONS
is
res
REFERENCES
[ 1]
[2] Pereira,
G.D., "The Vi
L.M.G.
, M Sc
"Automat c
Thesis ITC
", IEEE
, Nr. 3,
[3] Makarovic, B.,
Fall
[4]
c,
Com.II,
[5] Makarovic,
B.,
B.
Image Registration by Means of
Meet
, Little
, USA,
"Automatic
' 1980.
Correlation
1980.
[6]
B , "Automatic
ISPRS
Generat
Data",
DTM", ISP
thms", ISP
, Com.II,