Institut für Photogrammetrie
ifp
Research and Development at ifp
Keynote dedicated to the 80th Birthday of Professor Dr. mult. Fritz
Ackermann
Universität Stuttgart
Dieter Fritsch, Institute for Photogrammetry (ifp), Universität Stuttgart
Copyright: Institut fuer Photogrammetrie, Univ Stuttgart
1
70th Birthday Fritz Ackermann
ifp
 Prof. Ackermann created the world-renowned Institute for
Photogrammetry at Univ. Stuttgart
 Supervised many national and international PhD students
Universität Stuttgart
 Gave outstanding contributions to Analytical and Digital
Photogrammetry
 Was a visionary and pragmatic engineer who always combined
basic research with technology transfer and applications!
His heritage is our commitment!!!
2
Contents
ifp
Introduction
From Oberschwaben to Vaihingen/Enz
Universität Stuttgart
Mapping by High Definition Airborne Imaging
Come-back of Image Matching – Multiray Photogrammetry
Automated Processing of Virtual City Models
Conclusions
Copyright: Institut fuer Photogrammetrie, Univ Stuttgart
3
1. Introduction – ifp Activities
ifp
Universität Stuttgart
Institute for Photogrammetry (ifp)
Universitaet Stuttgart
Teaching,
Further
Education
L3






Geodesy & Geoinformatics
GEOENGINE
Infrastructure Planning
WAREM
Aerospace Engineering
Environmental Engineering
Research
&
Development
(R&D)



Photogrammetry, RS
Geoinformatics
Terrestrial Positioning,
Close Range Photogr.
Signal Processing,
Statistical Inference

4
Technology
Transfer




The Photogrammetric
Week Series (bi-annual)
SW Development
Consultancy
Workshops (eg QuaCon,
bi-annual)
Fritz Ackermann at Photogrammetric Week ‘09
ifp
 Prof. Ackermann created the world-renowned Institute for
Photogrammetry at Univ. Stuttgart
 Supervised many national and international PhD students
Universität Stuttgart
 Gave outstanding contributions to Analytical and Digital
Photogrammetry
 Was a visionary and pragmatic engineer who always combined
basic research with technology transfer and applications!
His heritage is our commitment!!!
5
1. Introduction - Research through PhD Theses
at ifp
ifp
Universität Stuttgart




At present, 13 doctoral theses under preparation
National and international PhD students (50/50)
Duration of a PhD Thesis: 3-4Y
Individual regular structure and best-fit supervision
2. From Oberschwaben to Vaihingen/Enz
Universität Stuttgart
ifp
Testsite Oberschwaben
7
2. From Oberschwaben to Vaihingen/Enz
ifp
The Airborne Pushbroom Camera DPA (1995)

Universität Stuttgart

8
stereo module
focal length
line array
pix/line
10 mm
resolution
convergence angle
spectral range
spectral module
focal length
line array
pixelsize
data resolution
spectral range
80 mm
2x6000
pixelsize
data
8 bit
± 25°
515 -780nm
40 mm
6000 pix/line
10 mm
8 bit
440 - 525nm
520 - 600nm
610 - 685nm
770 - 890nm
2. From Oberschwaben to Vaihingen/Enz
German Camera Evaluation Test
ifp
GSD 20cm
Universität Stuttgart
GSD 8cm
Tutorial Phowo 2009
© Institute for Photogrammetry, Univ. Stuttgart
2. From Oberschwaben to Vaihingen/Enz
Test flights, evaluated by ifp
ifp
Universität Stuttgart
Testf flight
Sensor
Date
month/year
GPS/inertial components
Digital line scanner
DPA
07/95, 08/96
10/96, 11/98
Digital line scanner
WAAC
11/97
WAAC – system specific
Digital line scanner
HRSC-A
02/98
POS/AV-510 DG, LR86
Analog camera
RMK-Top15
12/98
POS/AV-510 DG, LR86
Analog camera
RMK-Top15
06/00, 09/02
DMC
04/03
POS/AV-510 DG, AIMU
ADS40
06/04
POS/AV-510 AIMU /
LN200, AEROcontrol-IId
dIGIcam-K14
06/04
AEROcontrol-IId
large / medium format
DMC,
dIGIcam-H39
09/07
AEROcontrol-IId
4Head Frame Rollei
AIC4
12/07
Applanix POS/AV 510
several
06/08ff
Applanix, AEROcontrol
Digital frame camera
large format
Digital line scanner
Digital frame camera
medium format
Digital frame camera
DGPF DAC Test
DPA – system specific
AEROcontrol-IId, IMU-IId
3. Mapping by High Definition (HD) Digital
Airborne Cameras
Today’s situation
ifp
Leica Geosystems
ADS40/ADS80
Intergraph/ZI
DMC
DiMAC Systems
DiMAC
Universität Stuttgart
Vexcel Imaging
Ultracam series
D, X, Xp
Rolleimetric
AIC series
IGI
DigiCAM series
Applanix
DSS series
11
3. Mapping by HD Digital Airborne Cameras
Empirical results – dual DigiCAM H/39, 17.12.2007
ifp
Dual DigiCAM
principle
Universität Stuttgart
System installation
Block layout – GSD 7cm, focal length 82mm, Dpix 6.8mm
Tutorial Phowo 2009
© Institute for Photogrammetry, Univ. Stuttgart
12
3. Mapping by HD Airborne Digital Cameras
Empirical results – dual DigiCAM H/39
ifp
GCP
#
ChP
#
s0
DG
0
65
AT no
32
AT 44
AT io
Vers.
Universität Stuttgart
RMS [m]
DEast
DNorth
DUp
4.08
0.045
0.075
0.130
33
1.50
0.033
0.070
0.134
32
33
1.41
0.022
0.037
0.088
32
33
1.47
0.022
0.039
0.096
Absolute accuracy from check point analysis, GSD 7cm
© Kremer & Cramer, 2008
http://www.ifp.uni-stuttgart.de/publications/2008/kremer_cramer_beijing2008.pdf
Tutorial Phowo 2009
© Institute for Photogrammetry, Univ. Stuttgart
13
3. Mapping by HD Digital Airborne Imaging
ifp
Innovative Sensor Geometries
Universität Stuttgart
VisionMap A³
Digital sweep-framing double lens
metric camera
© VisionMap, 2008
3. Mapping by HD Digital Airborne Cameras
Universität Stuttgart
ifp
VisionMap A³ Camera Design
Camera parameters
 Digital sweep-framing
double lens metric camera;
 Cross-track sweep motion;
 Focal lens – 300 mm;
 Folded optics;
 Maximal sweep FOV – 104
degree;
 FMC, SMC, Vibration –
mirror based optical
compensation and
stabilization;
 Weight – 15 kg;
 Size – 50*50*40 cm.
Tutorial Phowo 2009
© Institute for Photogrammetry, Univ. Stuttgart
27 double frames per sweep;
Along strip overlap between frames 2%
Cross strip overlap between frames 15%;
Overlap between two sweeps – 56%.
© VisionMap, 2008
3. Mapping by HD Digital Airborne Cameras
ifp
Innovative sensor geometries, VisionMap A³
VisionMap A³
Universität Stuttgart
Test flight Vaihingen/Enz,
June 13, 2009
3. Mapping by HD Digital Airborne Cameras
ifp
Innovative sensor geometries, VisionMap A³
VisionMap A³
Universität Stuttgart
Test flight Vaihingen/Enz,
June 13, 2009
3. Mapping by HD Digital Airborne Cameras
ifp
What comes next?
 Towards “all-by-one” mapping by photogrammetric image
flights:
Universität Stuttgart
 Automatic camera alibration, Direct Georeferencing, True
Orthophoto and Digital Surface Model in one flight
Determine a very dense DSM by multi-ray image matching
and compute the true ortho with the DSM
 Unwrap hidden buildings in the DSM using automated
reconstruction algorithms, compute DTM
Reconstruct the 3D building vectors, texture the
reconstructed building
 The “all-by-one” approach will improve accuracy and
efficiency, cost-saving
No need for Airborne LiDAR in map revision processes
anymore. Flight-on-demand!
Copyright: Institut fuer
Universität Stuttgart
ifp
4. Come-back of Image Matching – Multi-Ray
Photogrammetry
standard design
film-based ACs
60% forward and 20% across overlap
Test flight digital AC
80% forward and 60% across overlap
 large overlap of dense HD photo blocks
 comprehensive support of automatic image matching
29.05.2016
19
4. Come-back of Image Matching – Multi-Ray
Photogrammetry
Universität Stuttgart
ifp
Usage of multiple overlaps, 3D point measurements
 Look at all possible stereo combinations
10 
k!
 No possible permutations  k  

 
   45
r
r!
k

r
!


 
2
 Dense sampling of 3D mass points using
 Epipolar geometry
 Test of consistency by forward – backward matching
20
4. Come-back of Image Matching
DTM/DSM 3D Point Clouds from Image Matching (N. Haala, ifp
ifp
DMC
2009)
Ultracam-X
DigiCAM
Universität Stuttgart
8cm GSD
20cm GSD
DGPF-DSM-Jena
29.05.2016
21
4. Come-back of Image Matching
Sportfield Rosswag
RMS of 3D Point Clouds (N. Haala, ifp 2009)
ifp
RMS [cm]
all Pts
1.92
Density
[Pts/m2]
8.25
[%]
Elim. Pts
0.54
ZMean [m]
Ref.
LiDAR
GSD
8cm
DMC
5.16
9.74
19.67
1.30
253.065
Ultracam-X
6.84
8.04
19.04
0.49
253.067
DigiCAM
10.18
11.28
20.83
0.68
253.023
RMK
17.23
27.27
0.77
3.18
253.002
DMC
17.23
25.36
2.71
1.10
252.896
Ultracam-X
22.57
34.21
1.62
0.43
252.980
DigiCAM
34.13
48.24
2.64
2.45
253.030
RMK
60.56
66.21
0.31
0.73
253.187
GSD
20cm
Universität Stuttgart
RMS [cm]
Filt. Pts
1.83

Image matching generates 3D point clouds of good quality (GSD 8cm)



253.036
RMS=7.4cm, 19.8[Pts/m2]
Average of 3 investigated digital photogrammetric airborne camera systems
Partly problems using GSD 20cm GSD

RMS= 24.6cm, 2.3[Pts/m2]
29.05.2016
22
4. Come-back of 2D Image Correlation
(Numerical Simulation)
ifp
PhD Thesis F. Blumrich, ifp 2009
Amount of the max error (in pixel) for sub-pixel detection
Universität Stuttgart
1D sub-pixel detector (state-ofthe-art)
general 2D linear Gaussian
regression (new)
The systematic error of the existing 1D sub-pixel detectors were
analytical described and validated
A general 2D linear sub-pixel detector has been developed and tested,
which is very fast (real-time apps) and up to 1000x more precise
ifp
5. Automated Processing of Virtual City Models The ifp Approach
Universität Stuttgart
3D Point
Cloud
(>4 points/m2)
PhD Theses:






3D Building
Unwrapping
(Vectorisation)
Apps
Apps
3D Building
Texturing
(Rendering)
3D Building
Generalisation
3D Building ReTexturing
(Re-Rendering)
3D Building
Structures
(Features)
Apps
Apps
N. Haala (1996)
B. Ameri (2000)
C. Brenner (2001)
M. Kada (2007)
S. Becker (2010)
M. Peter (2011)
24
5. Automated Processing of Virtual City Models
3D Berlin, Project Overview
ifp
 Project execution:
 virtualcitySYSTEMS
 Reconstruction of building geometry
 Autodesk GmbH
 Texturing from oblique images
Universität Stuttgart
 Project duration:
 East Berlin (March 2008 – July 2008)
 West Berlin (November 2008 – February 2009)
 3D-Reconstruction software developed by
 virtualcitySYSTEMS
 Institute for Photogrammetry (ifp), University of Stuttgart
5. Automated Processing of Virtual City Models
Universität Stuttgart
ifp
3D Berlin - Examples of Simple Shapes and their Combination
to 3D-Building Models
5. Automated Processing of Virtual City Models
3D Berlin, Reconstruction Approach
ifp
Universität Stuttgart
Basic Shapes
Flat
Shed
Connecting Shapes
Gabled
Hipped
„Berliner“
Special Shapes
5. Automated Processing of Virtual City Models
ifp
Building Unwrapping, Reconstruction Approach (ifp 1998, 2007)
 Automatic reconstruction of building geometry from
Universität Stuttgart
 Digital surface model (4 ppsqm for Berlin)
 Ground plans (e.g. from the authoritative real estate cadastre)
1. Generate a generalized ground plan decomposition in disjoint
(quadrilateral) cells
2. Classify the LIDAR points according to the orientation of their
local regression plane
3. Find for each cell the shape that best fits the classified points
5. Automated Processing of Virtual City Models
ifp
Rendering and Image Abstraction (ifp 2009)
 Hiding window contents
 Detection of window locations by
approaches from building reconstruction
 Gray out / set to high reflectance
Universität Stuttgart
 Removal of objects on the façade
 Detection using OCR techniques
 Copy other parts of the texture
 Non-photorealistic visualization
 Cartoon-like textures
 Recognize spatial situation  hide details
 Generic textures, single color
 Determine number and arrangement of
windows
Copyright: Inst. Photogrammetry, Univ. Stuttgart
5. Automated Processing of Virtual City Models
ifp
Privacy-Enabling Abstraction,Geometry abstraction (ifp 2009)
 Geometric simplification
 e.g. removed dormer
 Façade typification
Universität Stuttgart
 Change number and arrangement of
façade elements
 Standard roof shapes, 3D building symbols
 Typification of building groups
 Aggregation of building blocks
 Hide borders of individual buildings
Copyright: Inst. Photogrammetry, Univ. Stuttgart
5. Automated Processing of Virtual City Models
Universität Stuttgart
Data driven
bottom-up
 Interpretation of
measured data
+
 Flexible
+
 Accuracy and quality
control
- Sensitive to erroneous or
incomplete data
Explicit geometric
facade modelling
Fully automatic
Copyright: Institut fuer Photogrammetrie, Univ Stuttgart
Knowledge based topdown
Facade
Reconstruction
ifp
Automated Feature Extraction - Facade Reconstruction
Combined
Knowledge
Propagation
 Formal grammars (e.g.
shape rules)
+ Topological correctness
+ Completeness
- Variety of generated
structures is restricted to
knowledge base
Quality sensitive
5. Automated Processing of Virtual City Models
Automated Feature Extraction through
Knowledge Propagation (PhD Thesis S. Becker, ifp 2010)
ifp

Data Driven Facade Reconstruction
bottom - up



Knowledge Inference
top - down




Detection of dominant or repetitive
features and regularities
Interrelationship between geometries
Inference of production rules
Knowledge Propagation

top - down
Universität Stuttgart

Cell decomposition
Extraction and modelling of facade
geometries from terrestrial LiDAR data
Refinement based on facade imagery

Top-down prediction for verification and
completion
Generation of synthetic facades
Copyright: Institut fuer Photogrammetrie, Univ Stuttgart
Facade
Grammar
5. Automated Processing of Virtual City Models
Automated Feature Extraction, Results PhD Thesis S. Becker,
ifp 2010
ifp
Schillerplatz, Stuttgart
Terrestrial LiDAR (Leica HDS 3000)
Facade imagery (NIKON D2x)
Universität Stuttgart


Copyright: Institut fuer Photogrammetrie, Univ Stuttgart
Lindenmuseum, Stuttgart


Terrestrial LiDAR (StreetMapper)
Facade imagery (NIKON D2x)
6. Virtual City Models and Image Understanding
Universität Stuttgart
ifp
Stuttgart, Schillerplatz
Copyright: Institut fuer Photogrammetrie, Univ Stuttgart
ifp
6. Conclusions ... Thanks to Fritz Ackermann
 Prof. Ackermann created the world-renowned Institute for
Photogrammetry at Univ. Stuttgart
 Supervised many national and international PhD students
Universität Stuttgart
 Gave outstanding contributions to Analytical and Digital
Photogrammetry
 Was a visionary and pragmatic engineer who always combined
basic research with technology transfer and applications!
His heritage is our commitment!!!
Copyright: Institut fuer Photogrammetrie, Univ Stuttgart
35
ifp
6. Happy Birthday Fritz Ackermann - The next
Milestone: the 90th Birthday
 Prof. Ackermann created the world-renowned Institute for
Photogrammetry at Univ. Stuttgart
 Supervised many national and international PhD students
Universität Stuttgart
 Gave outstanding contributions to Analytical and Digital
Photogrammetry
 Was a visionary and pragmatic engineer who always combined
basic research with technology transfer and applications!
His heritage is our commitment!!!
Copyright: Institut fuer Photogrammetrie, Univ Stuttgart
36