Marco Scaioni
CONCEPT AND TESTING OF AN AUTOMATIC SYSTEM FOR AERIAL TRIANGULATION
Gianfranco FORLANI*, Livio PINTO**, Marco SCAIONI**
*
Università degli Studi di Parma, Italy
Dipartimento di Ingegneria Civile
forlani@parma1.ing.unipr.it
**
Politecnico di Milano, Italy
Dipartimento di Ingegneria Idraulica, Ambientale e del Rilevamento
livio@mail.polimi.it, marco.scaioni@polimi.it
Working Group II/8
KEY WORDS: Photogrammetry, Automatic Aerial Triangulation, Automation.
ABSTRACT
This paper presents the latest release of the system TRIADIGIT for Automatic Aerotriangulation jointly developed at
the Politecnico di Milano and at the University of Parma. The system is made up of three main modules: automatic
location, measurement and identification of fiducial marks (inner orientation); automatic tie point extraction; assisted
ground control point measurement. Here the second module is analyzed in detail, describing the procedure for tie point
selection and transfer, for improving the number of multi-ray points and for outlier rejection. Results of a series of
experimental tests are reported, showing the performances of the system in dealing with block featuring different scale,
image content, topography and texture.
1 INTRODUCTION
Automation of the Aerial Triangulation has been a hot research topic in the ninenties, in the scientific community as
well as among photogrammetric companies. Several tests have been organized to compare the performances of different
strategies and to highlight advantages and disadvantages of the new concept with respect to Analytical Aerial
Triangulation. The announcement of one of such tests, jointly organized by ISPRS and OEEPE in 1997 (Heipke and
Eder, 1998), prompted our groups to join efforts in the design and implementation of an Automatic Aerial Triangulation
program. The very first release of the program, named TRIADIGIT, only takes care of the selection and recognition of
tie points and was completed just in time to take part into the above mentioned test (Forlani et al., 1998). Later, two new
modules were implemented: ORINT, which performs the automatic interior orientation, and APPBLOCK, which allows
interactive measurement of ground or additional tie points through a graphic interface.
The core of the program, i.e. the strategy for tie point extraction and transfer, has since been updated to improve either
the number of multi-ray points and the point distribution within the images, both not completely satisfactory in the
preceeding tests. The original approach, based on the direct search for correspondencies in a bundle block, has not been
changed, but several stages have gone through a comprehensive review, leading to a more complex iteration sequence,
where bundle block adjustments are performed also at the higher level of the pyramid, to improve initial values of the
orientation parameters and outlier rejection.
2 THE TRIADIGIT SYSTEM FOR AUTOMATIC AERIAL TRIANGULATION
2.1 System overview
Automatic Aerotriangulation is a complex task, involving different basic procedures. The existing systems for AAT are
structured in modules, each of them performing a single stage, and TRIADIGIT is not exception. Its main components,
which execute the fundamental tasks (interior orientation, tie point extraction, interactive measurement of GCPs, and
bundle adjustment) may exist as stand-alone program, allowing their use in other packages.
The automatic interior orientation of every image of the block, a pre-requisite of every metric operation on the imagery,
is performed first. Then follows the measurement of a minimum set of the available GCPs, carried out interactively by
the module APPBLOCK, to define the photogrammetric block datum, which is needed throughout the tie point
486
International Archives of Photogrammetry and Remote Sensing. Vol. XXXIII, Part B2. Amsterdam 2000.
Marco Scaioni
extraction stage and to compute intermediate bundle adjustments. The last preparation stage is the image pyramids
generation.
Tie points are selected and matched in a hierarchical procedure from-coarse-to-fine. At the end of each level, a robust
procedure to discard gross errors is carried out, followed by a bundle adjustment to reject those outliers which have still
survived and to update the EO of the block. Here a check of the results by the user is required, because if the geometry
of tie/ground control points is weak, the solution may be not stable, causing the subsequent failure of the whole AAT.
After this stage, the remaining GCPs can be quickly measured, because the improved EO of the block is exploited to
assist the user in their localization. A final bundle adjustment ends the AAT.
2.2 Strategy for tie point extraction
Tie point selection, measurement and transfer within a photogrammetric block is based on some a priori information,
namely a rough EO and a ground model. Apart from blocks in the mountains, where a DTM is strictly necessary, a
horizontal plane and the maximum height difference in the area is enough elsewhere. The strategy for tie point transfer
is based on a multi-resolution approach, starting from a pixel size of about 0.8 mm and using all the levels of the image
pyramids.
The workflow is made up of three main stages:
1.
2.
3.
improvement of the approximate exterior orientation of the block;
tie point densification;
improvement of tie point accuracy and of the number of multi-ray points.
The first stage can be skipped if a set of accurate values for the EO parameters is available (e.g. if GPS has been used
during the flight). In this case the procedure may start from stage 2.
2.2.1 Preliminary data setup. The dream of an AAT program only requiring to supply the images into the storage unit,
to push a button, to have lunch and then to enjoy the results of aerotriangulation is still utopian. Data preparation is
required also by the best commercial packages and the knowledge of some a priori information about the
photogrammetric block (number of images, pixel size), the interior orientation elements and yhe camera calibration
data, is necessary. A number of processing parameters must be fixed (number of image pyramid levels, l.s. matching
parameters, etc.).
The search for homologous points is directly performed on the whole block, using a bundle approach. Since the
collinearity model is used in point transfer, initial values of the EO elements are required; their accuracy defines the
extent of the search area.
A ground model must also be given, although a simple horizontal plane is enough for most blocks. In this case the mean
height in the block area has to be provided as well as the maximum height difference.
2.2.2 Improvement of the approximate exterior orientation. This stage is carried out at the top of the image
pyramids, in one or may be two levels, depending on the resolution of the original images. As a rule of thumb, with
imagery having a pixel size less than 20 µm two levels are necessary, otherwise one is enough.
A set of interest points is extracted by means of the Förstner operator on every image at the lowest resolution and is
stored in a file; hereafter we will refer to these points as secondary points. A sub-set of primary points is derived from
each set of secondary points. If there is no a priori knowledge about block geometry, however a case seldom happening,
the primary points are selected by taking the np points with the highest interest value w, hopefully all over the image. If
on the contrary the geometry of the block is approximately known, the primary areas are selected in such a way that
they do not overlap. Besides, areas where only twofold points might be found are neglected, apart from the border
images. The primary points act as templates in the image matching, the secondary points as slaves.
The search for homologous points is built up on two nested loops. The external loop runs on all images of the block,
which become in turn primary images. The inner loop runs on all the remaining images, which become secondary
images. When an image is primary, each of its primary points defines a template. Its homologous in the secondary
images are looked for in two steps.
International Archives of Photogrammetry and Remote Sensing. Vol. XXXIII, Part B2. Amsterdam 2000.
487
Marco Scaioni
The first one defines a search window on the secondary image by projecting the template point down to the DTM and
then back onto the secondary image. Window orientation and size depend on the uncertainty of EO parameters of both
images and the ground model. All secondary points falling in the search window are selected as candidates and matched
one at a time by l.s. template matching. The match with the largest correlation coefficient yields the (likely)
homologous point, if larger than a threshold. By this procedure, the conjugate points of each primary point are looked
for on the whole block, so that multi-ray points might be found.
Hopefully, a first set of tie points has now been found. Unfortunately, as many as 30÷40% of them are outliers: due to
wrong EO the conjugate points were searched for also in areas far from the correct position. To cope effectively with
the large number of mismatchings, a robust procedure for gross error rejection is implemented. Every feasible pair of
images is considered and the set of common points is checked against outliers by means of two methods: a robust
relative orientation (Scaioni, 2000) or a parallax analysis; using the former or the latter depends on the number of
points in the pair.
The AAT procedure is now stopped and the set of “clean” tie points is adjusted by the bundle program CALGE to
further remove outliers by data snooping and to improve the initial EO parameters. The top level of the image pyramids
usually gives tie points with an accuracy of about 0.25÷0.30 pixels. Optionally, a DTM can be computed from the
object coordinates of tie points, to reduce the search window in the next level.
2.2.3 Tie point densification. Experience tells that the number of tie points found in the first stage is not enough to get
in the end a strong photogrammetric block. Transfer through all the remaining levels of the image pyramids shrinks the
set of about 10÷20% of the total amount in each level. Therefore point densification is carried out, at level n-1 (or n-2).
Thanks to the larger resolution of the images, more interest points can be extracted and, by exploiting the improved EO,
true matches should be found more easily.
From an operational point of view, the procedure for point transfer is the same as in the first stage, but the goal is
different. The improvement of the EO is now less important, while seeking more tie points is crucial to give stability to
the block. At the end of this stage the tie points found are transferred “vertically” to the next level after the bundle
adjustment. Moreover, the object coordinates of tie points are backprojected from the ground onto all the images of the
block, giving raise to new secondary points, so that the number of rays per point should increase wherever possible. All
the tie points found (directly and by the backprojection) are then transferred to the next level.
2.2.4 Tie point refinement and improvement of multi-rays. Two tasks are executed in the third stage: first, since tie
points have been measured at a coarse resolution (some hundreds micrometers) the accuracy of their measurement must
be improved; second, the number of manifold points should be increased as much as possible.
Every primary point has now assigned its
corresponding tie points on the other
images and these don’t have to be looked
for any more. Interest points extracted at
the top level of the image pyramids are
located on well defined features, allowing
a good determinability of the geometric
parameters of l.s. matching. Points
transferred to the next level should keep
this peculiarity, but this is not possible by
direct transfer (simply multiplying the
image coordinates of the point by a factor
2). The coordinates of each primary and
secondary point are therefore redefined on
the new level by applying the Förstner
operator on a window centered on the
points transferred directly (see Figure 1).
Obviously, the measurement at current
level may improve the precision, but may
also fail, due to inaccurate backprojection
or to the appearance of new details which
cause mismatchings.
Figure 1. Example of a 6-ray tie point correctly traced along 5 level
image pyramids
After the measurement stage, outlier
488
International Archives of Photogrammetry and Remote Sensing. Vol. XXXIII, Part B2. Amsterdam 2000.
Marco Scaioni
rejection and a new bundle adjustment are performed. The remaining points are backprojected on every image of the
block and then transferred to the next level. This scheme is repeated until the level 0 of the image pyramids is reached.
To save time, the adjustment and the backprojection are carried out only after the levels 4 and 2, whilst after the other
levels all the points found by image matching are transferred without any further processing.
2.3 Interior Orientation
The reconstruction of the interior orientation of every image is carried out in automatic manner by the program ORINT
(Forlani et al., 1999), which performs either the measurement and the identification of the fiducial marks. Based on a
multi-resolution approach, the fiducial marks are first located by using gray value cross-correlation; their position is
then refined by l.s. matching. If the marks are provided by symbols, they are automatically associated to the calibration
file. Finally the computation of the transformation between pixel and image coordinate systems is carried out with a
robust method.
2.4 Measurement of Ground Control Points
The program APPBLOCK performs the interactive measurement of GCPs with subpixel accuracy, thanks to image
resampling. This solution is the best suited to natural GCPs, which are widely used in Italy. Apart from the
measurement in strict sense, the program assists the user in the management of photogrammetric observation files.
Windows from all images where the GCP is likely to be are displayed, based on the available exterior orientation and
DTM. At the beginning of the AAT of a block, a minimum number of GCPs is measured to constrain the block during
the intermediate bundle adjustments; this task is performed with only a rough information about the block orientation.
After tie point extraction, other GCPs may be added, this time by using a more accurate exterior orientation.
This procedure may be applied also to the measurement of tie points in interactive aerotriangulation or to introduce
additional observations in areas of the block where the automatic extraction failed.
3 RESULTS OF EXPERIMENTAL TESTS
3.1 Description of the tests
In this section some tests on the performance of the latest release of TRIADIGIT are reported. The theoretical accuracy
of tie point extracted has been investigated by looking at the outcome of the bundle block adjustment and an
independent check of their accuracy has been obtained by comparing the results of AAT to those of interactive
aerotriangulation. Keeping into account the above-mentioned goals, only small blocks have been considered, resulting
in manageable data volumes and not involving too much computation time. Operational aspects of AAT have been
neglected. According to these guidelines, four blocks have been selected among those delivered on the occasion of the
OEEPE-ISPRS test (Heipke and Eder, 1998) and those available at the University of Parma: they differ by scene
contents, topography, cameras, scales, film material and overlap. Table 2 reports the main features of the blocks; Figure
3 shows for each block an image and a patch representative of the texture.
Project name
Scene content
Scene topography
Average image scale
Camera (focal lenght, mm)
Film material
No. of images per strip
Overlaps
Pixel size (µ
µ m)
Source
No. and kind of GCPs
Lucera
country, small town
flat, partly hilly
1:7500
Wild RC10 (150)
B/W
6×3
l=60%,q=60%
25
DIIAR, Milan
Echallens
open, partly forest
flat
1:5000
Wild RC10 (150)
B/W
3×3
l=60%, q=30%
20
EPFL, Lausanne
Botticino
OSU
rock quarry
built-up, partly trees
very steep
flat, buildings
1:5600
1:4000
Zeiss RMKA (150) Wild RC10 (150)
B/W
FIR
3×2
3×3
l=80%, q=60%
l=60%,q=60%
25
25
Comune di
The Ohio State
Botticino
Univ. / TU Munich
8 (natural)
44 (signalized)
8 (natural)
not available
Table 2. Description ot the data sets used in the tests
International Archives of Photogrammetry and Remote Sensing. Vol. XXXIII, Part B2. Amsterdam 2000.
489
Marco Scaioni
Whole image
Detail of the texture
Lucera (scale 1:7500, pixel size 25 µm)
Whole image
Detail of the texture
Botticino (scale 1:5600, pixel size 25 µ m)
Echallens (scale 1:5000, pixel size 20 µ m)
OSU (scale 1:4000, pixel size 30 µ m)
Figure 3. Examples of images and details of the texture from the blocks used for the tests
3.2 AAT setup and preprocessing
The inner orientation, computed by program ORINT, gave a mean sigma nought of the transformation from pixel to
image coordinates for each block ranging from 0.2 to 0.5 [pel]. A minimum set of GCPs was measured in each block, to
compute the bundle adjustments of each hierarchical level, except for block “OSU”, where no GCPs are available and
the adjustment has been computed by constraining 7 EO parameters.
3.2.1 Approximate exterior orientation. The blocks “Echallens” and “OSU”, delivered on the occasion of the
OEEPE-ISPRS test, were already provided with approximate values of the exterior orientation parameters, accurate to
about 50 m for the projection centers and 2° for the attitude angles. Besides, the overlaps were given as well. These
values have been directly used. In the other two blocks (“Lucera” and “Botticino”) every image has been roughly
georeferenced on a map at the scale 1:25000, obtaining approximate values for the horizontal coordinates of the
projection centres with an accuracy of about 100 m, and values for angles κ with an accuracy of about 3°. For the
elevation Z0 the instrumental flight heights registered on each photograph have been used (accuracy about 50 m), while
the angles ω and φ have been put equal to zero. This demonstrates that the derivation of the approximate exterior
parameters without GPS on board is not critical, because the procedure converged despite the low accuracy .
3.2.2 Ground model. For all the blocks a horizontal plane has been adopted as ground model. For blocks “Lucera” and
“Botticino” the average height and the maximum variation in elevation have been derived from an existing map. For
blocks “Echallens” and “OSU”, the average height of the ground was given, while the variation has been guessed
according to the kind of topography.
3.2.3 Program control parameters. We tried to find standard values for the control parameters, independent of the
block characteristics: e.g. we found that a 6 levels image pyramid may be used with image resolutions in the range
20÷30 µm. Another critical point is the size of the window for l.s. matching: after many trials, we concluded that the
best results may be obtained by increasing by two pixels the size of the window at each level of the image pyramid,
starting from a size of 9 pixels.
3.2.4 AAT strategy. By AAT strategy we mean the setup of tie point extraction workflow, i.e. the number of image
pyramid levels involved in the refinement of initial EO parameters (see par. 2.2.2), how many times intermediate bundle
adjustments are computed and the backprojection of tie points is performed to increase multi-ray points. For all the
blocks we followed this strategy, except for “Botticino”:
1.
490
improvement of approximate EO involving only the initial level (level 5) of the image pyramids;
International Archives of Photogrammetry and Remote Sensing. Vol. XXXIII, Part B2. Amsterdam 2000.
Marco Scaioni
2.
3.
4.
tie point densification at level 4: after the bundle adjustment, all the tie points found are backprojected from the
ground to the images;
in the remaining levels (3-0), tie points are traced along the image pyramids; an intermediate bundle adjustment
followed by the backprojection of tie points is executed after level 2;
after level 0 the final bundle adjustment is computed.
This solution still recovers many multi-ray points which have been lost, avoiding to compute the bundle adjustment at
the end of each level.
In case of block “Botticino”, flown over a hilly area, a DTM has been computed at the end of the initial level by all the
tie points found and used in the next level for tie point densification. In case of complex topography, where the
availability of a rough DTM would facilitate tie point transfer, the test on block “Botticino” has shown that also the
ground model defined by the tie points represent an effective solution.
3.3 AAT outcomes
3.3.1 Tie point distribution. The geometric distribution of tie points over the block and the amount of manifold points
is the key for an accurate and stable aerotriangulation. Table 4 reports the number of tie points found and that of multiray points. For the blocks “Lucera”, “Echallens” and “OSU” a consistent set of points has been found, as you can see in
Figure 6; only multi-ray points have been drawn. The best results have been gained in case of block “Echallens”, where
the flat terrain and the well-contrasted image texture represent an ideal condition for AAT. On the contrary, a smaller
set of tie points has been found for block “Botticino”, which features many critical conditions, such as a complex
topography with many breaklines, woods, areas whith poor texture.
3.3.2 Outcomes of bundle adjustment. Table 5 reports the results of the bundle adjustment in terms of sigma nought
and the accuracy of tie points in object space, which are representative of the block accuracy. As far as σ0 is concerned,
we expected lower values, because the potential accuracy of l.s. matching is better than 0.4 [pel] and the tie point set is
consistent and well distributed. The reason of this result is likely to arise from the bundle adjustment, which should use
a robust method; indeed, the internal blunder detection procedure copes effectively with gross errors, but does not the
same with small outliers. For the block “Echallens” the result is satifying, but also in this case it might be further
improved by adopting a robust bundle adjustment.
The theoretical accuracy of object coordinates obtained of the blocks are sufficiently accurate according to image scale
and terrain characteristics.
3.3.3 Independent accuracy check. Though standard deviations of tie points and external reliability may highlight
weaknesses in the block, deformations cannot be traced without an external check. We therefore performed an
independent test, adopting the same procedure of the OEEPE-ISPRS test (Heipke and Eder, 1998).
Project
Lucera
Echallens
Botticino
OSU
Project
Lucera
Echallens
Botticino
OSU
No. of image
pyramids levels
6
6
6
6
Av. no. of rays
No. of multi-ray points in object space
per points
Total
2
3
4
5
6
7
8
2.7
2838
1679
669 284
157
47
2
0
2.5
1607
1041
335 164
27
20


2.5
306
173
103
25
5
0


2.8
1330
772
289 121
63
58
22
5
Table 4. Multi-ray tie points at the end of AAT
9
0


0
Theoretical accuracy
RMS of residuals on
Ratio
No. of
No. of
σ0 of bundle
[cm]
check points [cm]
GCPs
check
adjustment equations/
used
points
[pel] [µ
µ m] unknowns
σZ
σ X-Y
σZ
σ X-Y
0.50 12.6
1.75
8
106
8.8
22.8
8.4
16.1
0.27
5.4
1.67
8
55
3.4
7.5
3.8
7.9
0.54 13.4
1.65
8
59
10.1
30.5
6.2
13.4
0.32
9.6
1.86
60
5.1
9.2
8.8
11.3

Table 5. Results of bundle adjustment and residuals on check points
International Archives of Photogrammetry and Remote Sensing. Vol. XXXIII, Part B2. Amsterdam 2000.
491
Marco Scaioni
Lucera
# of rays per point
3
0
OSU
492
4
5
5
6
7
8
0
0.5
1
1.5
2 km
Botticino
Echallens
# of rays per point
3
4
6
0.5
1 km
Figure 6. Distribution of
manifold points (more than two
rays per point) in object space
for the block used in
experimental tests. 3-D GCPs
are represented by triangles,
height GCPs by circles.
International Archives of Photogrammetry and Remote Sensing. Vol. XXXIII, Part B2. Amsterdam 2000.
Marco Scaioni
A set of well distributed tie points has been interactively measured by means of the program APPBLOCK in each
block. The same interior orientation parameters were applied, so both sets of image coordinates (“automatic” and
“manual”) are homogeneous. The “manual” block has been adjusted by using the same constraints and the object
coordinates of tie points have been computed. Then the image and object coordinates of tie points belonging to the
“manual” block have been inserted as check points in the adjustment of the block made up by the “automatic”
measurements. The discrepancies between these two data sets are reported in Table 5. The RMS computed in this way
may be assumed as the accuracy of “automatic” measurements. Differences with respect to the theoretical accuracies
(Table 5), are always very small, ensuring that the ties within the “automatic” block are strong enough. The larger
discrepancies in height might be due also to the weakness of the “manual” blocks, which should be strenghtened by
introducing more tie points.
4 CONCLUSIONS
In this paper we described the new features of our AAT program TRIADIGIT. The tests made show that the program is
now operational, though not ready for mass production, and can cope with demanding environments. As far as the
procedure is concerned, outlier rejection should be slightly modified, since too many points are rejected on the border of
the block. As far as the workflow is concerned, the switch from bundle block adjustment to point transfer should be
improved, ensuring a larger degree of automation. For the time being, the performance in terms of computing time is
still unsatisfactory: the average time in the four block turned out to be around 6 hours on a Pentium I 200MHz: software
optimization is required. A graphic user interface is also being developed, to improve the data preparation.
ACKNOWLEDGMENTS
The authors are grateful to the TU of Munich, the EPFL, Lausanne, the Ohio State University, and the Comune di
Botticino for providing the images and the ground control point coordinates used in this test. Thanks go also to Istituto
Geografico Militare Italiano, in particular to Ing. Antonio Arrighi, for the digitization of the photogrammetric block
“Lucera”. We are indebted to Ing. Fabio Remondino, who carried out his Degree Thesis at Department IIAR working
on Automatic Aerial Triangulation.
REFERENCES
References from Journals
Forlani G., Pinto L., Scaioni M., 1998. The TRIADIGIT program for automatic aerial triangulation, IAPRS, Vol. 32,
Part 2, Cambridge, UK, pp. 67-73.
Forlani G., Pinto L., Scaioni M., 1999. A procedure for Automatic Inner Orientation, IAPRS, Vol. 32, Part 2W5,
Munich, Germany, pp. 131-135.
Scaioni M., 2000. Robust Estimation of Relative Orientation through the Fundamental Matrix, in printing on IAPRS,
Vol. 32, Part 6W8/2, Ljubljana, Slovenia.
Heipke C., Eder K., 1998. Performance of tie point extraction in automatic aerial triangulation. OEEPE Official
Pubblication n. 35, pp. 125-185.
References from Other Literature
Scaioni M., 1999. A System for Automatic Aerotriangulation: Concept, Implementation and Testing. Ph.D. Thesis,
Politecnico di Milano, Italy.
International Archives of Photogrammetry and Remote Sensing. Vol. XXXIII, Part B2. Amsterdam 2000.
493