Imaging, Integration and Visualization of
Multi-sensor Data for Road Terrain
Mapping
Imaging, Robotics and Intelligent Systems Lab
The University of Tennessee, Knoxville
Sreenivas Rangan
Sijie Yu
David Page
Mongi Abidi
September 2005
ii
Abstract
In this report, we present our experience in building a mobile imaging system that
incorporates multi-modal sensors for road surface mapping and inspection
applications. Our proposed system leverages 3D laser-range sensors, video cameras,
global positioning systems (GPS) and inertial measurement units (IMU) towards the
generation of photo-realistic, geometrically accurate, geo-referenced 3D models of
road surfaces. Based on our summary of the state-of-the-art systems, we address the
need and hence several challenges in the real-time deployment, integration and
visualization of data from multiple sensors. We document design issues concerning
each of these sensors and present a simple temporal alignment method to integrate
multi-sensor data into textured 3D models. The 3D models generated by our mobile
system on the one hand serve as inputs to simulators that study vehicle-terrain
interaction and also possess the required accuracy even for crack detection towards
road surface inspection in airfields and highways.
iii
Contents
1
INTRODUCTION .......................................................................1
1.1
1.2
1.3
1.4
2
LITERATURE REVIEW ...........................................................7
2.1
2.2
2.3
2.4
3
Road Terrain Profiling for Vehicle Terrain Simulators......................................... 1
Airfield/ Highway Pavement Distress Survey ....................................................... 3
Proposed Approach ............................................................................................... 4
Document Organization......................................................................................... 6
Overview of the State-of-the-art ............................................................................ 7
Commercial Systems – Techniques and Methodologies ....................................... 9
Summary ............................................................................................................. 11
Remarks ............................................................................................................... 12
DATA COLLECTION SYSTEM ............................................13
3.1
Hardware Components ........................................................................................ 13
3.1.1
3.1.2
3.2
3.3
4
System Design ..................................................................................................... 18
Hardware and Software Interaction Architecture ................................................ 21
DATA INTEGRATION ............................................................23
4.1
4.2
4.3
4.4
5
Range Profile Alignment ..................................................................................... 23
Direct Pose Measurement .................................................................................... 26
Filtering Redundancy .......................................................................................... 28
Smoothing and Triangulation .............................................................................. 29
ANALYSIS AND RESULTS ....................................................31
5.1
Modeling Sensor Noise ....................................................................................... 31
5.1.1
5.1.2
5.1.3
5.2
Modeling GPS and IMU Errors ......................................................................... 31
Laser Range Sensor Measurement Error ........................................................... 35
3D Modeling Accuracy ..................................................................................... 42
Video Mosaic Results .......................................................................................... 47
5.2.1
5.3
6
Navigation Equipment ....................................................................................... 13
Range and Intensity Imaging ............................................................................. 17
Texture Mapping ............................................................................................... 48
Examples and Discussion .................................................................................... 52
CONCLUSIONS........................................................................61
BIBLIOGRAPHY.............................................................................62
iv
Chapter 1: Introduction
1
1
INTRODUCTION
The need for accurate 3D road terrain models arises from two different yet significant
applications: (a) simulators for vehicle-environment interaction and (b) automated
distress survey in airfields and highways. The generation of such photo-realistic, georeferenced, geometrically accurate 3D terrain models begins with the design of a mobile
mapping system. The mobile acquisition system collects multi-sensor position,
orientation and geometric data which is later processed and integrated into 3D models
suitable for virtual reality simulations and distress survey algorithms. Before getting into
any further details on the construction of our system and real-time data collection, we
very briefly summarize the expectations on our system and the 3D models for each of
the aforementioned applications.
1.1
Road Terrain Profiling for Vehicle Terrain Simulators
The U.S Army has constructed several proving grounds scattered all over the country.
The idea being that army vehicles (tanks and carrier equipment) can be tested on
rough and uncertain terrain before deployment in real world scenarios. The testing is
performed by driving assembled and fully equipped vehicles across different types of
surfaces such as the Belgian blocks, Perryman surfaces etc. and making measurements
for fatigue, damage, wear and tear of different components in the automobile. Such an
assessment gives an idea of the robustness of the vehicle under test. Also, the vehicle
dynamics can be better understood through experimental measurements from surfaces
of different material (grass, concrete etc.), road roughness and topography (hilly,
plain, and rugged). These test drives also help in analyzing the driver behavior and
form the feedback loop towards the improvement of army vehicles and making them
suitable for battlefield conditions. Recently, the U.S army concluded that experiments
using real vehicles were very expensive and proposed to move towards virtual reality
testing using 3D CAD models of automobiles and road terrain. The Virtual Proving
Grounds (VPG) for driver/soldier training, soil-tire interaction, vehicle terrain
interaction and component behavior analysis is an effort in that direction. The terrain
models for these vehicle-terrain simulators are presently generated using
profilometers. In Figure 1.1, we show how profilometers are used to generate
graphical primitives and emphasize the difference between reality testing and virtual
reality testing in Figure 1.2.
Chapter 1: Introduction
Figure 1.1: Generating 3D models for vehicle-terrain interaction simulators using
profilometers.
Figure 1.2: Reality testing vs. virtual reality testing.
2
Chapter 1: Introduction
3
As seen, the models generated from profilometers do not embed the real-world
uncertainty into the simulation. Hence, the need for generating 3D models of dynamic
environments such as hilly terrain, a speed breaker, a gravel road arises. To meet such
a requirement, we propose in this report a data collection system capable of collecting
and processing data of any arbitrary terrain almost real-time and generating models
suitable for vehicle-terrain interaction analysis. The requirement imposed by such
simulators are that the 3D models be in a commonly used format that is easy to
visualize and are also suitable for modeling vehicle dynamics using finite element
analysis.
1.2
Airfield/ Highway Pavement Distress Survey
Traditionally, general aviation airfield pavements are maintained based on the
inspection staff’s judgment and experience [Walker, 2004]. The inspection personnel
walks or drives slowly through asphalt and concrete pavements observing surface
defects and degradation to make recommendations for immediate and long term
maintenance. The manual inspection procedure (as shown in Figure 1.3) is not only
cumbersome, time consuming and expensive but is also susceptible to human error
and inefficiency. With safety of aircrafts and passengers in mind, this functional and
important process of inspection can be significantly improved using a formalized
imaging system that will ease the effort required to inventory airfield conditions
through periodic evaluations and subsequent surface distress management without
compromising safety.
Figure 1.3: Airfield pavement inspection. (a) Manual inspection procedure. (Image
from the FAA conference) (b) Types of distress on the pavement. (c) Damage that
pavement distress can cause on aircrafts.
Chapter 1: Introduction
4
The key to successful road surface evaluation lies in identifying different types of
distress and linking them to the cause. Recognizing the defects and also understanding
their cause based on their appearance helps rate pavement conditions and select cost
effective repair measures. As a first step towards automation, high speed digital
imaging sensors deployed on mobile vehicles have been successfully demonstrated in
combination with image processing algorithms for crack detection. Such video based
vision systems have two major drawbacks in extension to airfield inspection. They do
not provide sufficient depth information and also have ambient illumination
requirements. The depth information is of particular significance in airfields because
the rating scheme for the runway surface is not just dependent on the length and width
of the cracks alone as is the case with pavement distress applications but also on the
depth. Crack depths in the order of a few millimeters require high precision distance
measurements. Hence, the design requirements for a comprehensive airfield data
collection system should address accuracy and precision in three dimensions of
measurement, speed of acquisition, time required for post processing, ease of
visualization and evaluation. To that end, we present a prototype mobile 3D data
acquisition system in this paper. Our system integrates visual range and color data
with position and orientation information through hardware measurements and
provides better accuracy for fast digitization of large scale road surfaces at almost
equal acquisition and processing time. We are able to generate accurate georeferenced 3D models that are compensated for sensor motion caused by the changing
physical environment. The multi-sensor integrated 3D models improve automatic
crack detection and classification. Furthermore, accurate archives of such 3D models
of road surfaces over time can be used for statistical wear and tear analysis.
1.3
Proposed Approach
We are looking to digitize road surfaces as accurately and quickly as possible with
available technology and processing equipment. Our primary goal being able to image
at high accuracy capable of measuring depth of cracks along the road, a 3D range
sensor on a mobile platform directly solves this problem. We have tried to use three
different 3D acquisition methods (triangulation-based, time-of-flight and structured
lighting) for the data collection and document our efforts in this regard. We concluded
that the triangulation based system matched our requirements for high speed and high
accuracy.
Though the 3D information alone is sufficient for crack detection and surface rating,
we need spatial information for crack localization and repair. We collect physical
location information by setting up a GPS base station and placing a receiver on the
mobile platform. The GPS data is accurate up to 4 cm in the motion direction and
gives us 10 samples of position information in one second. The GPS can be thought of
Chapter 1: Introduction
5
as sampling the 3D motion of the mobile platform that houses the sensors. In the
schematic shown in Figure 1.4, we have shown a video camera mounted on a rod,
whose image axis is orthogonal to the road surface. We prefer the orthogonal field-ofview because it makes the registration of range and intensity profiles trivial and
considerably improves integration time without having to consider CCD calibration
and rectification of images. The video data in addition to providing visual cues for
crack detection also helps in estimating the motion. The video data is particularly
useful when the GPS satellite signals are intermittently not available from the satellites
during certain time intervals of the day.
The system components that we have described thus far are sufficient for scanning
planar road surfaces. For roads with sufficient distress, varying terrains and embanked
pavements, the effect of driving on such terrain and roads with bumps needs to be
compensated. The oscillations on the mobile platform caused by the suspension
system also have to be considered. We have hence used the IMU for measuring the
orientation Euler angles (roll, pitch and yaw) of the sensor mount during data
collection. We have used a high performance computer with a Pentium 4 processor
that supports hyper threading with 1GB of RAM and with special high speed serial
interface cards as the processing equipment. Our multi-threaded, multi-document
graphical user interfaces written in C++ are capable of real-time buffer memory
management, storage and initial processing.
Figure 1.4: Schematic of our proposed system.
Chapter 1: Introduction
1.4
6
Document Organization
We have organized this report to address the design challenges in the construction of
the multi-modal integrated imaging system that is capable of real-time data collection
without restriction on the planarity of road surfaces. We have begun by listing the
specific requirements as a problem statement. In Section 2, we summarize
contemporary commercial systems targeting road surface inspection. The literature
survey emphasizes on the design methods implemented thus far and also serves as a
reference study to understand the difficulty in building systems for real-time
deployment. We introduce our prototype system and explain the idea behind using
multi-modal sensors in Section 3. After the data acquisition, we deal with the
integration of multi-modal data in Section 4. The integration involves the
representation of range and visual data into a spatially meaningful form using the
information from position and motion sensors. We show the 3D models generated
using our system driving a van along a test area containing different types of cracks in
Section 5 and conclude with recommendations for possible improvements and
reproducibility of our system in Section 6.
Chapter 2: Literature Review
2
7
LITERATURE REVIEW
In presenting the state-of-the-art and available technology we begin this chapter with a
brief overview of the state-of-the-art on road terrain mapping in Section 2.1 and later
delve into details of commercial systems, and available technologies in the Section 2.2.
We draw conclusive remarks based on our survey in Section 2.3
2.1
Overview of the State-of-the-art
Related work towards pavement distress, especially on airport runways and army
maintained highways dates back to early 1980’s. The pavement management system
(PMS) idea was proposed by the U.S Army [U.S Army, 1984] and has since then
undergone metamorphosis keeping pace with improving imaging technology.
However, transportation departments met with limited real-time success using digital
imaging techniques towards automatic crack detection and filling [NHRCP Report,
2002], until the late nineties. Non-visual sensors and several improvements on imagebased methods were proposed during this period. We summarize these methods in
Figure 2.1 and discuss the advantages and disadvantages of the different types of
sensing methodologies. Analog films have been completely replaced by digital
cameras and digital video systems are preferred to high resolution line scan methods
for the ease of use without special illumination requirements, though line scan
methods offer very high resolution data. Range sensors that directly give depth
measurements have limited field of view while profilometers and acoustic sensors
though inexpensive can only provide low resolution and a low dynamic range.
In 1987, Mendelsohn [Mendelsohn, 1987] listed several of these methods including
acoustic sensors and profilometers and suggested that the imaging modality was a
promising approach. At that time, the processing and image acquisition speeds
challenged the feasibility of a fast and efficient inspection system. Several surveys
were conducted to make an assessment of the feasibility of incorporating image
acquisition and processing methods for both development and implementation of
automated road surface inspection [Howe, 1998]. The conclusions of the survey
encouraged by improving hardware and processing equipment have led to most of the
commercial video-based systems available today that basically consist of an array of
high speed imaging sensors supported with illumination equipment. The video data
Chapter 2: Literature Review
8
from such systems though promises to be sufficient for distress detection [Meignen,
1997], requires spatial information for crack filling after detection and maintenance. A
potential solution AMPIS [Chung, 2003] was proposed that combined GPS
information with video to create GIS-like databases of road surfaces. AMPIS claims
improved road network identification, pavement inspection for better maintenance and
data management over the base framework of PMS.
Taking a robotic approach, Hass et al. [Haas, 1992] proposed a system to overcome
the shortcomings of the video-based system to make depth measurements by
incorporating a laser range sensor. Hass et al. concluded that combining laser range
data and video image data can provide overall better accuracy and speed of crack
detection although due to the time consuming aspect of laser range sensing in 1992,
they demonstrated range imaging for crack verification after an initial pass of the
video based system. Several 3D approaches have then been demonstrated since then.
Laurent et al. propose a synchronized laser scanning mechanism to capture high
precision 3D range and texture profiles. Bursanescu and Blais [Bursanescu, 1997]
reiterate a 3D optical sensor as the answer to high resolution and high accuracy
acquisition and redesign a Biris sensor to meet the specific requirements of the
pavement inspection application. They demonstrate six such sensors mounted on a
mobile platform acquiring data at normal highway speeds. Now that we have given a
brief introduction about methodologies already implemented, we shift into the details
about the sensor and system development in the following subsection.
Figure 2.1: Available technologies for road surface mapping.
Chapter 2: Literature Review
2.2
9
Commercial Systems – Techniques and Methodologies
We begin with imaging approaches, which include using analog and digital imaging to
acquire pavement textures; and then discuss sensing approaches that use acoustics and
infrared laser beams to acquire 3D geometric information. We trace the evolution of
multi-sensor approaches by combining two or more of these sensing methods and
clearly list the advantages over a single sensing strategy.
Analog imaging refers to the obsolete yet simple film photography (usually with 35
mm film) and videotaping (e.g. Super VHS). Analog Photographing or photologging it
is popularly was adopted as the method for Long Term Pavement Performance
(LTPP) program. Photographing using a 35-mm film requires human observation for
identification of distress. “Photologging mobile” as a system is typically a van with a
downward facing camera (to acquire distress data) and one or more facing forward or
in another direction (e.g. to estimate trajectory). “Photologging mobile” is reported to
function at nights using lighted cameras to reduce shadows cast by mobiles, traffic or
roadside features. In most cases photographing consciously samples only a section of
the roadway and with a maximum accuracy archived is cracks that are 1 mm wide
[McGhee, 2004] at speeds up to 60 mph when controlled illumination is available.
Digital imaging is similar to video logging in concept but uses a charge-coupled
device (CCD) for imaging. With such devices, though shadow is still a problem that
needs to be overcome using special lighting, the motivation to switch from the analog
to digital approach is that less or no more human intervention is required in processing
the acquired data. With digital images, computers can be programmed for distress
survey. Yet another convenience that the digital data offers is in terms of storage,
backup, archival and retrieval.
Digital imaging currently is more popular than analog imaging. Their lower price and
higher degree of automation make them the preferred method in most transportation
agencies [McGhee, 2004]. There are two types of cameras currently used: “area scan”
and “line scan”. Area scanning gives a 2D array of pixels which describes a snapshot
of objects. A sequence of snapshots is captured sequentially and periodically. For a
single image it normally covers half or full-lane width and 3-5 m long section of the
road, depending on camera intrinsic parameters (e.g. lens and field-of-view) and
system setup. The distance between successive frames however is dependent on the
motion of the mobile platform. We note that in most attempts using a digital camera, a
downward facing camera orthogonal to the surface of interest is preferred to minimize
pixel distortion.
Line scanning gives a single line of pixels (1D) that can be thought of as an intensity
profile of the road surface. Accumulating such profiles over a pre-determined motion
Chapter 2: Literature Review
10
path can be integrated as a high resolution image. One single large scale 2D image (an
array of pixels) is acquired by aligning pixel lines based on the mobile trajectory.
Each transverse line usually covers one full-lane of a highway, but the width depends
on the height of camera above the surface. Image length is determined by the motion
trajectory. Shadow is still a problem and needs to be overcome in the system design
phase.
Acoustic sensing was demonstrated for measuring smoothness or roughness of the
road surface using the definition of the International Roughness Index. The acoustic
sensing approach emerged in 1981 from the South Dakota department of
transportation (DOT). South Dakota DOT first used it as a roughness measuring
device that became so popular that by 1991, the number increased to 25 DOT agencies
choosing this system for real-time deployment. However, with increasing speed of
imaging-base sensors, a much more recent survey in 2004 for Synthesis of Highway
Practice reports that only 3 agencies are still using it. The South Dakota profiler device
simultaneously collects three ultrasonic profiles, one for each wheel path and another
for the lane center. Three profiles are mathematically integrated to generate roughness
measuring results and rutting at specified intervals along a moving trajectory.
Laser sensing is the rapidly developing and widely used technique for extracting 3D
geometry that has been replacing most of the aforementioned methods. Generally,
laser range scanners are more accurate and are easy to use and require no special
illumination requirements. Laser scanning as the name suggests, leverages an active
light source in the form of a laser to obtain range information, such as height and
distance. The choice of the type of sensor (triangulation, time-of-flight, stereo etc.) is
based on the accuracy and field-of-view required by our problem.
With these sensing modalities, modern approaches have evolved into integrated
sensing techniques which can be described in one word as multi-discipline combined,
multi-sensor integrated, multi-platform compensated, or multi-data fused to mean that
several sensors were involved in the modeling pipeline [Tao, 1998]. A simple example
would be a system that combines laser sensing and digital imaging to yield
photorealistic 3D surface information. Such integrated systems, essentially consist of
three sub systems: navigation, imaging and geometry. The navigation component
usually comprises one or more of accelerometers, GPS, Differential-GPS, or IMU.
Navigation equipment provides a sample version of the actual trajectory in 3D space.
Imaging components, as mentioned before, include digital cameras and video cameras.
It images texture information of the surface of interest. The geometry component uses
one or more of three types of sensors: laser, acoustic, or infrared. It yields 3D
geometric information of the road surface.
11
Chapter 2: Literature Review
Of the several integrated approaches, a summary of the system characteristics and
accuracy that we present in Table 1 reveals more information on how each modality
contributes to the accuracy and photo-realism requirements.
2.3
Summary
The main goal being able to generate large scale terrain models as fast and accurate as
possible, current data collection methods still necessitate integration of several
heterogeneous technologies. We further identify the scope for improvements in system
design targeting the time of acquisition and processing and list the important
characteristics of a real-time deployable system. An ideal road data collection system
must operate in real time gathering and post processing speeds. The duration required
for data analysis should not overwhelm the time required for acquisition. Furthermore,
much of the state-of-the-art appears to be restricted to small section image processing
using visual images when 3D scanning methods can significantly contribute to
accurate and realistic digitization. A single pass data collection strategy for costeffective distress identification and localization for the airfield survey applications,
and high accuracy dynamic terrain modeling applications have not been explored with
critical focus on the accuracy and robustness of the system and its extendibility to
arbitrary terrain. To that end, with all these system requirements in mind we now
present our prototype system in the Section 3.
Table 2.1: Summary of state-of-the-art integrated approaches.
System/
Group
Modalities used
Komatsu
Video, Line scan
WiseCrax
Dual video cameras
GPSVan
Stereo and analog
camera, GPS , INS
Resolution
of
imagery
4
mega
pixel
image.
Detects 3mm wide
cracks
Built for large scale
imaging
3D range
< 0.5 mm
National
Optics
NRC
(Canada)
AMPIS
RoadCrack
Video, GPS
3mm
wide,4mm
deep cracks
0.3 mega pixels
Array of CCD
1mm crack width.
3D range
Special notes
Collects data in the night with
argon lights at 10 km/hr.
Can collect data at 80 km/hr.
Acquires geo-spatial data for urban
planning.
Novel synchronized laser scanning
approach proposed.
An array of Biris 3D sensor used.
Limited field of view.
Can collect 400 km of data in one
day at highway speeds.
Chapter 2: Literature Review
2.4
12
Remarks
The main goal being able to generate large scale terrain models as fast and accurate as
possible, current data collection methods still necessitate integration of several
heterogeneous technologies. We further identify the scope for improvements in system
design targeting the time of acquisition and processing and list the important
characteristics of a real-time deployable system. An ideal road data collection system
must operate in real time gathering and post processing speeds. The duration required
for data analysis should not overwhelm the time required for acquisition. A single pass
data collection should be sufficient for cost-effective distress identification and
localization for the airfield survey applications, the critical aspect being the accuracy
and robustness of the system and its extendibility to arbitrary terrain. With all these
system requirements in mind we now present our prototype system in the Section 3.
Chapter 3: Data Collection System
3
13
DATA COLLECTION SYSTEM
From the state-of-the-art summary, we identified that the integrated sensing approach of
using multiple sensors was the right direction towards a fast mobile scanning system
capable of generating 3D terrain models for simulators providing high accuracy to the
extent of detecting cracks on the road surface. In this chapter, we present our prototype
system that we demonstrate as a potential solution for high accuracy road terrain
mapping. We discuss each component of our system along with its hardware
specifications and limitations in Section 3.1. Later in Section 3.2, we discuss the
software user interface and present a very brief tutorial on how to interface the hardware
with the software to acquire data real time.
3.1
Hardware Components
Our system also leverages the three subsystem approach and includes navigation,
imaging and geometry acquisition equipment. Our navigation equipment consists of a
differential GPS system from Leica Geosystems to measure the position of the sensing
equipment at a given point of time and an inertial measurement unit (Xsens) to track
the orientation of the sensors during the data collection process. Our imaging
components that include a handheld video camera, a triangulation based 3D scanner
(IVP) and a time of flight scanner (SICK) together provide the flexibility to switch
between high accuracy and large field of view. In Figure 3.1, we show the picture of
components that constitute our prototype system and list the hardware specifications in
Table 3.1.
3.1.1
Navigation Equipment
GPS and IMU technologies have been widely used in a variety of positioning and
navigation applications. The usual integration method for GPS with IMU data is
through the implementation of a Kalman filter. In all these integration modes the INS
error states (attitude, rate-of-turn), together with GPS navigation state (position,
velocity) are estimated using a mathematical stochastic model.
14
Chapter 3: Data Collection System
(a)
(b)
(c)
(d)
(e)
(f)
Figure 3.1: Components used in our prototype system. (a) SICK LMS 200 range
sensor (b) SONY digital camera. (c) IVP triangulation-base range sensor (d) Xsens
IMU. (e)Leica GPS-500. (f) Differential GPS system and radio transmission
equipment.
Chapter 3: Data Collection System
Table 3.1: Hardware specifications.
Sensor
SICK LMS200
(Indoor version)
Sony DCR-TRV730
Digital 8™ Camcorder
Leica System 500
SR530 sensor
Xsens MT9
IVP MAPP 2500
Specifications
Sampling frequency: 37 profiles per second
Range: 150 m – 0.2 m
Resolution: 5 mm
Error range: + 15 / - 15 mm, RMS 5mm in mm mode within 8 m
Sampling frequency: up to 30 frames per second, normally work
at 10 frames per second
View angle: 45 degree (max)
Pixel size: 720 x 480
Image format: BMP
Focal length in memory mode: 39 - 702 mm
Sampling frequency: 0.1 s-60 s (for NMEA83 sentences)
3D accuracy : (DGPS) 0.1 cm for North and East, Accuracy in
height = 2 x accuracy in position
Output streaming data format: NMEA 0183 V2.20 and Leica
proprietary
Sampling frequency: 25 Hz – 512 Hz
Angular resolution: 0.05 degree
Angular accuracy: 3 degree RMS
Given configured for a baseline of 70cm and stand off 70cm and
a triangulation angle of 45 degrees,
Sampling frequency: 2000 profiles per second
Resolution: 1 mm
Range: 1m width of view
Accuracy: RMS 1 mm
15
16
Chapter 3: Data Collection System
Differential GPS (DGPS) is the most common approach to significantly improve
measurement accuracy. A DGPS system uses a reference GPS and a rover GPS. The
position information of such a system is computed based on the signal reception from
four different satellites. The absolute location in terms of latitude and longitude at a
particular point is localized as the intersection of four spheres (ellipsoids) associated
with each satellite. Hence, the differential-GPS system requires that there be at least
four satellites to interact with. We have used two Leica GPS-500 station transceivers
that can provide centimeter accuracy on altitude which translates to about two
centimeter accuracy on position. The maximum rate for phase and code measurements
is 10 Hz. The GPS 3D coordinates sampling frequency for our system is one of 1Hz, 5
Hz or 10 Hz. We usually operate the system at 10 Hz data acquisition rate to ensure
high accuracy in position and also reduce uncertainty while modeling.
As shown in figure 4, DGPS involves 2 GPS receivers, one is a base receiver and the
other one serves as a roaming receiver. Base receiver obtains GPS signals and then
generates correction to roaming (or rover) receiver. Then roaming receiver uses
corrections to update and corrects its position. The final output is downloaded from
roaming receiver. Baseline length is an important factor influencing GPS position
accuracy. Based on experiments we proved that the measurement accuracy changes
linearly with respect to baseline length. Therefore, when placing GPS receivers, it is
better to keep them as close as possible.
Satellites
Roaming GPS
Reference GPS
Baseline length
Figure 3.2: Simple schematic diagram to understand the Differential-GPS.
17
Chapter 3: Data Collection System
3.1.2
Range and Intensity Imaging
Range Sensing: The 3D sensing being a significant component in our project, we start
with the description of our laser range sensor SICK LMS 200. A real 3D range image
is obtained by moving the laser relative to the scene. SICK has a adjustable scanning
angle of 100 or 180 degree, sampling resolution of 0.25, 0.5, or 1.0 degree, maximal
range distance of up to 8m with 0.5 degree configuration, average distance error of
+15 / -15 mm, profiling rate of up to 37 profiles/s (statistically estimated) and 5 mm
standard deviation (based on manufactory specifications). The data transfer rate can be
programmed to be 9.6, 19.2, 38.4, or 500K baud. When setting as 500K baud, a
RS422 PCI card is needed for desktop and a PCMCIA card for laptop.
The scanner emits a safe laser beam outside in a fan-shaped patter from right to left.
The emitted laser beam is reflected back when it strikes some object on its path. The
range is computed from the time taken by the beam to travel from the scanner to the
object and back to the scanner. Or, it is based on the computation of time of flight of
the beam. If the beam is not reflected back, it gets lost and the scanner inputs an error
flag. The beam gets lost when it passes through the glass windows or due to a long
range distance. When the beam pass through a glass window, sometimes it is reflected
by inner objects but the reflected beam can not be received by the scanner due to the
reflection in different direction or multiple reflections. Errors are generated when the
sun light directly falls on the scanner mirror. The scanner gets confused by the direct
fall of sunlight and can not compute the true range. This is one of the problems of
outdoor operations. Suppose a single SICK profile line i is generated by connecting a
series of points with distance and angle (r j ,  j ) . The local spatial information ( s i , j ) of
each point (i, j ) can be calculated as below:
S i , j  (0,ri  cos  i ,ri  sin  i )
(1)
where ri is the measured range and  i is calculated by angular range and resolution. If
the
angular
range
is
180
degree
with
resolution
0.5
degree,
then  i  0.5(i  1), i  1,..,361 , that is 361 points are measured for each scan i .
The manufactory specification states Sick has transversal range capability of 8 m.
Because points which are further away from laser mirror center distributes more
sparsely, we use partial points in each scan. Therefore the actual transversal range is
less than 8 m. For example, if the sensor mirror center point is 80 cm away from
object surface, to keep the spatial resolution between 2 closet points within 10 cm, we
use 298 points out of 361 points and the actual transversal range is 6 m.
Chapter 3: Data Collection System
18
Intensity imaging using a SONY video camera: A SONY video camcorder is used
in this prototype. It is used for capturing video images in a sequence when moving
forward. The image frames from the video provide the texture information for the
profiles from the range sensor to generate textured 3D geometric models. Our SONY
video camera is capable of imaging frames with 720 pixels horizontal x 480 pixels
vertical resolution. The CCD in the video camera has a diagonal chip cell size of 5.4
mm and a focal length 4.1 mm. Based on the camera focal length, we decided to
mount the camera at a 194 cm above the road surface (mounted the camera on a van
roof), so that a single image frame covered over a 129 cm wide x 194 cm long
rectangular flat area.
3.2
System Design
The system is configured to be downward looking with 4 equipments mounted on a
rigid flat metal plate. Each sensor has a local coordinate frame centered at its mass
center. Data fusion actually is a set of transformations from sensor’s local coordinate
frame to a global coordinate frame. Each sensor’s body-centered coordinate frames are
explained as below,

IMU local coordinate frame with X going through its body (also parallel to
moving direction), Y following right-handed rule, and Z being upward.

Camera local coordinate frame or image pixel frame with X denoting pixel
column number and Y denoting row number.

Laser local coordinate frame is centered at the body, with X pointing to
moving direction, Y following right-handed rule and Z being upward.

The global coordinate frame which is also the local earth fixed coordinate
frame. The GPS outputs are based on this frame, with X parallel to North, Y
parallel to East and Z being upward. The XY plane is the tangent
approximation of the local earth surface. Range and video measurements are
transformed into this frame.
The camera is placed downward facing with a small incidence angle (from 0 to 10
degrees). Enlarging the angle will obtain bigger field of view; while the sacrifice is the
image distortion and the camera may not capture thinner crack. If the camera is
perpendicularly downward looking at pavement the field of view is 45 degree. We
initially set the orientation angle at 0 degree.
19
Chapter 3: Data Collection System
To achieve high speed acquisition we mount the system on a van, but actually it can
also be mounted on a cart. Thus the outputs will be denser. There are 3 things for
considering the sensor placement, supposing the system is mounted on a van:

The distance from laser mirror center to pavement surface, which affects
significantly on ranging accuracy. Based on our extensive indoor experiments, we
found the optimal distance is between 60 cm and 80 cm. The distance herein is set
as 65 cm. The optimal distance is obtained from indoor calibration experiments,
which will be explained in the Chapter 5.

The distance from video camera focus to pavement surface. This parameter affects
camera field of view. If the distance is set as 194 cm, given camera field of view is
45 degree, a single image can cover a flat area of 129 cm x 194 cm. Larger
distance will generates wider field of view.

Based on the above set up, given the measurement van moves at a speed of 5 mph,
video images are captured at intervals of 22.3 cm, range scan lines are profiles at
intervals of 6.2 cm, and GPS are measured at intervals of 12.4 cm with IMU at
intervals of 0.7 cm. cubic spline interpolation is used for adding missing GSP and
IMU samples.
The sensor placement for the system design is illustrated in Figure 3.3 and the
instrumentation mounted on the van are shown in Figure 3.4
GPS antenna
Video
camera
Camera focal
Sensor mirror
center
194 cm
IMU
Mobile
65 cm
Sick
Surface
Figure 3.3: System design. (a) Sensor placement. (b) Design parameters.
Chapter 3: Data Collection System
Figure 3.4: Our prototype data acquisition system and its close up views.
20
21
Chapter 3: Data Collection System
3.3
Hardware and Software Interaction Architecture
Sick binary data is downloaded from a RS-422 serial communication port on a
QuadTech 2-port PCI serial communication card. GPS/IMU steaming data is
downloaded from two RS-232 serial communication ports respectively (e.g. com1 and
com2). Video images are grabbed from IEEE 1394 frame grabber. IEEE 1394 is a
standard definition for a high speed serial bus. This bus is also named as FireWire by
Apple. It is ideal for consumer electronics audio/video (A/V) appliances, storage
peripherals, and portable devices. The software architecture is shown in Figure
3.5. Our programs are multi-threaded windows architecture based programs that have
built in routines for acquiring and storing data from the IEEE 1394 ports for the video,
COM serial ports for the range sensor and pose estimation hardware. Our processing
computer which houses a hard drive stores the data from all these ports in a format
readily usable for integration. In Figure 3.6, we have shown the snapshots of the GUI
that we have created for real-time data acquisition. These interfaces are extremely easy
to use with quick buttons to trigger and stop data transfer from the sensing device to
the computer. The GPS/ IMU acquisition program outputs data GPS position in the
NEMA standard format that and the IMU data in an easy three column ASCII format.
So far, in this chapter we have listed the hardware components required for such a
road terrain mapping system and also demonstrated the hardware software interaction.
We have used Windows API commands in a Visual C++ environment to create
interfaces to enable easy and real-time data collection.
Imaging
Laser
Video camera
Navigation
I/O
0.027s/p
0.100s/f
0.100s
GPS
0.010s
IMU
Storage
RS-422
Serial
ASCII file
Serial
Comm
class
IEEE 1394
RS-232
Serial
RS-232
Serial
DirectX
show
BMP
Still
images
Serial
Comm.
COM
component
Figure 3.5: Hardware/ Software interaction.
+
ASII
Time
File
ASCII file
22
Chapter 3: Data Collection System
(a)
(b)
(c)
Figure 3.6: GUI built for data collection (a) GPS/IMU navigation data collection
GUI snapshot. (b) Snapshot of the GUI for frame grabbing. (c) Snapshot of the range
data acquisition program.
Chapter 4: Data Integration
4
23
DATA INTEGRATION
In this chapter, we describe the algorithmic procedure to reconstruct multi-sensor
integrated 3D information. We begin with the noise filtering from the range points and
then discuss about the range profile alignment based on position information. We
present the equations governing the alignment.
As mentioned before the SICK range sensor outputs range information. The third
degree information is obtained from measurement vehicle motion (combination of
translation and rotation). Given the moving direction points to positive x direction, y
axis follows right handed rule, and z axis is upward, laser 3D coordinates is computed
as ( x, y, z )T  (0,r cos ,r sin  )T , in which r is the range value and  is the scanning
angle. The laser scans in a constant angular speed therefore for each profile the
distance interval between two closest points is not uniform. To partially compensate
for the non-uniformly scanning of the laser, in each scan, the range values in each
profile are firstly interpolated by cubic splines. This particular interpolation method
can guarantee C 2 continuity along each profile. This allows scan rows have a regular
grid structure. Therefore it is easily to identify the neighbors to right, left, above and
below for each point, as seen later, is essential for the generation of a depth image and
triangulation, or future operations. For the raw range measurements are rather noisy, a
1D median filter is applied to remove noise. After the above operation the resulting
range image has a resolution of 1cm along profile direction (i.e. Y axis).
The measurement vehicle moves in a dynamic mode therefore the distance interval
between two close range profiles is also non-uniform. In our experiment, this scan row
interval varies from 3cm to 7cm. In Figure 4.1 we show the conceptual idea to
generate a grid range image.
4.1
Range Profile Alignment
To obtain a real 3D range image, the individual 2D range scans (or profiles) are
spatially aligned using motion estimated from GPS/IMU. This process can be
explained as an affine transformation for each laser scan. Scans are based on local
laser coordinate system, while alignment process is to transform them into a global
24
Chapter 4: Data Integration
(a)
(b)
Profile Interval
Figure 4.1: Range profile processing. (a) Original range profile. (b) Filtered range
profile (c) Non-uniform accumulation of profiles.
25
Chapter 4: Data Integration
coordinate system (e.g. the local earth-fixed coordinate system). Affine transformation
computation is illustrated in equation (5) and (6) [Zhao, 2003].
To obtain a real 3D range image, the individual 2D range scans (or profiles) are
spatially aligned using motion estimated from GPS/IMU. This process can be
explained as an affine transformation for each laser scan. Scans are based on local
laser coordinate system, while alignment process is to transform them into a global
coordinate system (e.g. the local earth-fixed coordinate system). Affine transformation
computation is illustrated in equation (5) and (6) [Zhao, 2003].
( X ,Y , Z )T  TsgThg ( x, y, z)T
Thg  Tgps  Rimu
(4-1)
(4-2)
Where ( x, y, z )T be the laser measurements in the local laser coordinate frame, which
can be calculated as ( x, y, z )T  (0, r cos ,r sin  )T ,  (i  1)0.5, i  1,...,361.
Thg be the transformation from GPS to the global coordinate frame (e.g. the local
earth-fixed coordinate system).
T sg be the translation from laser to GPS, which is manually measured.
T gps be the translation from GPS to global coordinate frame (e.g. the local earth-fixed
coordinate system), which is also the absolute GPS measurements.
Rimu be the rotation from GPS to global coordinate frame (e.g. the local earth-fixed
coordinate system), which is also the absolute IMU measurements.
Thus by aligning a set of range scans into a global coordinate system a 3D point cloud
is obtained. But because of non-uniformly vehicle motion, the distance interval
between the two closest scans varies. Therefore the range data is scattered and sparse.
In human visualization it looks rather blurring and fails to present road surface
features (e.g. ridge, crease, and jump). In general there is no guarantee that the
sampling density is even sufficient for a correct reconstruction. Moreover measured
range values are noisy, and they may contain outliers. The noise sometimes lies in the
same level of small road features. Therefore post-processing is applied to deal with the
problems described above. Post-processing herein includes redundancy filtering,
interpolation, and spatial filtering. The goal of the processing is to reconstruct an
accurate and eye-appeasing geometry and topology of the scanned surface.
Chapter 4: Data Integration
4.2
26
Direct Pose Measurement
As mentioned in the section 4.1 individual laser scan is spatially transformed into a
global coordinate system. The transformation parameters actually are directly obtained
from GPS and IMU. If we use 6 DoF (Degree of Freedom), the three translation
parameters Thg are measured from GPS and the three orientation parameters Rimu are
measured from IMU. That is from GPS we obtain X, Y and Z and from IMU we
obtain Euler angles (i.e. roll, pitch and yaw). We will discuss the measurement
Actually the raw data downloaded from GPS is National Marine Electrics Association
(NMEA) 0183 v 2.20 sentences. The NMEA is a standard describing a set of
massages that can be output by navigation sensors. The Leica is configured to output
Global Positioning Fix Data, or called GGA sentence for short. The information GGA
provides is listed below:
a.
b.
c.
d.
e.
f.
g.
h.
i.
j.
k.
Time of position (in Universal Time Code, or UTC);
Latitude coordinate;
North or South;
Longitude coordinate;
East or West;
GPS quality;
Number of satellites in use;
Antenna altitude above/below mean sea level;
Geodetic separation;
Age of differential GPS data;
DGPS station ID number.
The latitude and longitude (degree) can be extracted from NEMA sentences. Those
coordinates are based on geodetic WGS84 ellipsoid coordinate system. For fusing
with other sensor data they are transformed into the Earth Centered Earth Fixed
(ECEF) rectangular coordinates (meter) [Guo, 2002]. ECEF rectangular coordinates
are further transformed to the local earth-fixed coordinates. This coordinate frame is a
local approximation of earth surface.
The raw data downloaded from IMU are 3D accelerations and 3D gyroscopes. We use
the development kit provided by Xsens to compute 3D Euler angles. The MT9
software component is implemented in a COM object (.dll) and it supports the
IDispatch interface. The rotation matrix Rimu can be explained as shown in the
following equations.
27
Chapter 4: Data Integration
Rimu  RX RY RZ
4-3
0
0
1
0 cos(ex) sin( ex)
RX  
0  sin( ex) cos(ex)

0
0
0
cos(ey )
 0
RY  
 sin( ey )

 0
0  sin( ey )
1
0
0 cos(ey )
0
0
 cos(ez ) sin( ez )
 sin( ez ) cos(ez )
RZ  
 0
0

0
 0
0
0
1
0
0
0
0

1
0
0
0

1
0
0
0

1
4-4
4-5
4-6
In general the multi-sensory data are loaded into computer asynchronously at multirate. For example, laser scan samples at 0.02s, GPS points samples at 0.2s, and IMU
Euler angles samples at 0.01s. Before registration, or future fusion their data are
needed to be synchronized. The synchronization is performed by an interpolation. That
is interpolating both GPS and IMU for missing data at the laser sampling time t . To
ensure C 2 continuity of vehicle trajectory, we chose 1D Cubic spline to define the
interpolants. Here we assume the moving path is a smooth curve. Cubic spline is a
piecewise interpolation by 3rd polynomials, between n-adjustment points. Therefore
they take into account all trajectory behaviour on each piece of interval, while
respecting continuity conditions on position, velocity and acceleration on each
adjustment point. It can be shown that Cubic spline minimizes, among all interpolation
functions, acceleration on the curve [Gontran, 2005]. Considering these specifications,
such curves are particularly well suited for road axis modeling in fast kinematic
surveys. If we use GPS north measurement as an example, the interpolation is
explained as below,
N (ti )  ati  bti  cti  d
3
2
4-7
Where ti be the GPS time and a, b, c, d be the Cubic Spline coefficients.
One is important that in our synchronization scheme, if a function N(t) has been
selected which, for a given value of t, will produce a value, i.e., north = N(t). If
28
Chapter 4: Data Integration
position is being interpolated then 3 functions are used in the following manner. The
north, west and up coordinates are considered independently. For example, the points
(x, t) are used as control points for the interpolating curve so that x = Nx(t) results,
where N denotes an interpolating function and the subscript x is used to denote that
this specific curve was formed using the north (or x) coordinates of the key positions.
Similarly, y=Wy (t) and z=Uz(t) are formed so that at any time, t, a position (x, y, z) can
be produced.
4.3
Filtering Redundancy
As the van drives forward the range scans are captured in an order with respect to
temporal order. But when turning a big heading angle the order may be reversed,
which is illustrated in Figure 4.2. From figure 4.2 (a) we can see if the range point has
a longer transversal distance (i.e. ri cos  i ) than critical distance (i.e. d crit ), the reverse
order will happens. But if the transversal distance is less than critical distance, shown
in figure 13 (b) the scan order is kept. For a given moving translation distance s and
a heading angle difference  between two scans, the critical distance is computed as
below,
d crit 
s
sin(  )
4-8
Therefore the redundancy filtering can be described as for any range point has longer
transversal distance than critical distance, it will be removed from 3D model.
Point i in scan 2
ri cos  i
ri cos  i
Point i in scan 2
Scan 1
Scan 1
s

Scan 2
Scan 2
d crit
d crit
Moving path
Moving path
Figure 4.2: Scanning real-time (a) Reverse order. (b) Normal serial profile order.
Chapter 4: Data Integration
4.4
29
Smoothing and Triangulation
So far the 3D point cloud still has noise and outliers. To remove those unexpected
range values a 2D median filter is applied. For the points have connectivity
relationship with their neighbors, a median filter is performed in a neighborhood
defined by a window with 3 x 3 point size.
Triangulation is performed on the “clean” 3D range point cloud and 3D models are
obtained by rendering the triangulated results. The triangulation algorithm is very
simple: it divides the grid point cloud into small cells and then divides the cells into 2
triangles. We call this method as uniform triangulation. Actually most of the model
surface patches are flat and with small curvatures. This corresponds to the real
pavement surface which is also mostly flat and only cracks generate big curvatures.
Therefore uniform triangulation can not represent such surface characteristics; we
chose to use multiples sizes of triangles for representing surface area and cracks
separately. This method is called multi-level triangulation with level of detail (LOD).
The algorithm can be described as whenever there is a point with big curvature; we
divide the triangulation which includes the point into 4 smaller triangulations. The
criterion for estimating curvature is defined as “mean distance”, which is the mean of
distances from points to the plane defined by their outer triangle. If the mean distance
is bigger than a threshold value then the triangulation is divided into 4 smaller ones.
This LOD algorithm is firstly introduced in [Sequeira, 1999]. It can be explained as
below,

The point cloud is projected on a 2D big grid. In the previous step we interpolate
the points along the transversal direction (or Y axis); the result should be a grid
with each point at the corner of a small virtual cell inside of the grid.

The big grid is divided into, say 50 x 81 cells. For each cell 2 triangles are
generated by connecting diagonal points.

Then for each triangle “mean distance” is computed for all points falling inside it.
If the mean distance is larger than a predefined value, the triangle is divided into 4
smaller triangles. This step is performed iteratively until the triangulation can not
be divided any more.
The figure 4.3 shows an example of uniform triangulation and multi-level of
triangulation. As a comparison we also applied 2D Delaney triangulation to the point
cloud. The main advantage of uniform triangulation over 2D Delaney is its efficiency;
while the LOD triangulation result reduces significantly the model size. The point
cloud has 65,325 points representing a virtual grid of 201x325 point size.
30
Chapter 4: Data Integration
(a)
(e)
(d)
Figure 4.3: An example of the reconstructed 3D model (a) Uniform triangulation
result. (b) Triangulated with level of details.
Chapter 6: Conclusions
5
31
ANALYSIS AND RESULTS
In this section we begin with the characterization of the sensors that are deployed in the
system, to enable us to use these datasets in a much better fashion.
5.1
Modeling Sensor Noise
Totally eight types of measurements are acquired asynchronously from four sensors
during operation time. They are 3D positions (North, West and Height); 3D
orientations (Euler angles: roll, pitch and yaw), 1 intensity image and 1 range value.
Because of equipment physics and moving dynamics, we expect noise and outliers
come along with true measurements. To precisely create surface 3D models, we need
to model sensor output instead of assuming that we can measure the 8 types of
information exactly. In the following sections DGPS, IMU and TOF laser are
investigated respectively. Their error models are created to estimate their electronic
errors precisely. This work has been ignored by most 3D reconstruction work.
Equipment noise is usually obtained from manufactory specifications.
5.1.1
Modeling GPS and IMU Errors
The original outputs from an Inertial Measurement Unit (IMU) are 3D linear
accelerometers and 3 ring laser gyroscopes. Euler angles (or attitudes) are computed
from raw measurements using a custom program provided by Xsens. The Euler angles
then are used to compute motion orientation matrix (refer to section C). We expect
errors generated from fusion process and equipment electronics. Manufactory
specifications the attitude error is less than 1 degree when sensor being placed
stationary and 3 degree when the sensor is moving. But actually errors are larger than
the specifications.
The DGPS errors are caused by many sources. In [] describes the major sources, be
satellite clock error, ephemeris error, receiver errors, and atmospheric delay. DGPS is
much more accurate than a standard GPS measurement. The table 5.1 lists common
GPS errors.
32
Chapter 6: Conclusions
The GPS outputs are pseudo-range and carrier phase. A dual frequency receiver
outputs range and phase measurements for each carrier frequency. These four outputs
are combined to obtain earth centered coordinates (degree) and mean altitude (m), e.g.
WGS84 coordinates. Then off-line transformation is performed to obtain local earthfixed coordinates (degree). During this process we expect errors are generated from
equipment electronics and from transformations. By using DGPS, Leica system 500
can achieve 2cm accuracy in North and East direction. Altitude accuracy is normally
double worse, i.e. 4cm.
Besides the error source mentioned above there are also other factors influencing GPS
accuracy. For example the satellites can be blocked by high buildings, tunnels and
overpasses. If this happens the GPS will not be able to estimate position. The GPS
updating rate is usually slower than other sensors used in system. Therefore we have
to synchronize GPS measurements with respect to faster sensors, i.e. TOF laser.
Synchronization strategy is basically an interpolation. We use Cubic spline for
estimating missing GPS coordinates. This method can reconstruct a 3D smooth curve
and then corresponding positioning information is extracted from this curve. But if
discrete sampling process is under-sampling, the reconstruction can’t represent the
moving trajectory, and then positioning information is not accurate in which undersampling happens.
But IMU and GPS random error characteristics have been well studies. It is possible to
use a mathematical model to estimate their values. The model is constructed on static
experimental data. The sensor measurement residual can represent their errors if the
measuring process is long enough. In [] the residual is analyzed and then the paper
suggests using an Autoregressive (AR) model to estimate it. The reason of doing this
is that the sensor residual is closely similar to an empirical higher order Gauss-Markov
Process. The paper emphasizes that the 1st GM process can not model most residual
sequence. In figure 15 we give an example of IMU yaw angle residual with its
histogram and autocorrelation and compare it with a simulated 1st GM sequence. We
can see how closely these two sequences similar to each other. For a more general
case higher order GM is used. But paper further points out that an AR model can be
easily constructed instead of higher order GM. The modeling process can be
illustrated as below:

Feed the measurement residual into a inverse z-transformation equation and then
the residual at current time n can be computed as,
p
y(n)   ak y(n  k )  bw(n)
k 1
5-1
33
Chapter 6: Conclusions

Determine AR model parameters ( ak , b, k  1,... p ). This is performed by
minimizing the error between the original signal represented by the AR and the
estimated signal

p
y   a k y ( n  k )
5-2
k 1
The cost function for this minimization problem can be computed as below,
N
E   e 2 ( n)
n 1
N

  [ y (n)  y (n)]
2
5-3
n 1
N
  b 2 w(n)  min
n 1
In which, N be the measurement size; p be the AR model order, could be 1, 2, 3, or 4
(higher ones are not practical for computation); w(n ) be white Gaussian sequence with
zero mean and unity variance.
From equation (9) we can find that the minimization is b 2 . It is also the estimated
variance  w2 of the white noise input the AR model, or more generally, it represents the
prediction Mean Square Error (MSE)  2 .
Using Yule-Walker method the AR parameters are optimally estimated by solving the
following,
E
0
a k
5-4
If assuming we have the autocorrelation computed from residual sequence, which is
represented as R (m ) , where m=-N,…,N.
Then the parameters are computed as below,
Ra  r
1
5-5
a  R r
5-6
a  (a1 ,..., a p )T
5-7
34
Chapter 6: Conclusions
r  ( R(1),..., R( p)) T
5-8
... R( p  1) 

... R( p  2) 
...
... 

...
R(0) 
R( p) 1    2 
R(1)
 R(0)

R(0)
 R(1)
R
...
...

 R( p  1) R( P  2)

R(1)
 R(0)

R(0)
 R(1)
 ...
...

 R( p) R( P  1)

...
  
... R( p  1)  a1   0

...
...  ...   ...
  
...
R(0)  a p   0






5-9
5-10
After the parameters are obtained, we can model the measurement errors as a
correlated sequence. If this is combined with a linear Kalman filter, the data set could
be smoothed correctly.
The Kalman filter can be illustrated as an optimization method such that the following
statistical conditions hold:

On average the estimate of the state will equal the true state;

On average the estimate error is minimized.
There are many alternative to formulate Kalman filter equations. The most important
thing here is to find an appropriate model for this specific application. Optimal
parameters such as measurement noise and state noise are the other critical
consideration for implementation. We initially chose a linear Kalman which is given
in the following equations [Simon, 1998].
xk 1  Axx  Fak
5-11
zk 1  Hxk  vk
5-12
S k  Pk  R
5-13
K K  FPk S k1
5-14
Pk 1  APk AT  Q  APk S k1 Pk AT
5-15



x k 1  A x k  K k ( zk 1  H x k )
5-16
35
Chapter 6: Conclusions
In which, k denotes states and measurements happen at time k; x k  1 be the initial state
estimation from current state with accelerations estimated as a k . z k  1 be the real
measurement with errors vk . Here the error sequence can be represented by a AR
p
model as vk   ai v(k  i)  bw(k ) . As suggested by the [], the order p is optimally 2,
i 1
3, or 4. S k called covariance of the innovation. K k be kalman gain matrix. Pk  1 be

covariance of the prediction error. x k  1 be the optimally estimated state. The
computation is fairly intuitive. The first term means the state estimate at time k+1 is
just state estimate at time k. this term would be the state estimate if we don’t have a
measurement. In other words the state estimate propagates in time just like the state
vector. The second term means corrector. It corrects the propagated estimate using
measurement. This term means that if the measurement noise is much grater than the
process noise, K k would be small, then the measurement will contribute to final
estimate less; otherwise K k is large, then the measurement will be trusted more.
From the error estimation result, we observe that,

Pitch performance is statistically better than roll and yaw. When being absolute
static the measurement accuracy is better than 1 degree. But in a moving mode
accuracy decrease significantly, especially heading angles (yaw). Therefore denoising is necessary before data fusion,

The IMU sensor needs around 3s to initialize. We use the data after initializing
time,

Heading accuracy is worse, which is sensitive to vehicle maneuver.

GPS Z value has larger error than X, and Y values. This phenomenon happens
normally in a GPS receiver.
5.1.2
Laser Range Sensor Measurement Error
The TOF laser range value is computed from the time period during which laser
travels from scanner to target object and then back to scanner. The manufactory
specifications state the scanner can measure up to 8 m transversal range with +/- 15
mm system error and 5 mm standard deviation. Actually range error dynamically
changes with respect to many factors, including target surface properties (reflectivity,
gray level, color), distance from laser center to object, and orientation degrees. Sick
36
Chapter 6: Conclusions
LMS 200 has been calibrated in [Ye, 2002] by extensive indoor experiments. Those
experiments are conducted in an empirical environment (indoor with temperature
24°c). The scanner is configured to work at 1 degree angular resolution and with 500K
Baud rate. The paper concludes that,

When operating at 500kbaud, occasionally data packages may lose, while no
package is lost at the slower 38.6K baud rate. However the lost data packets in the
experiment never exceed 0.08%.

There is noticeable drift until 3 hours after start-up, but that drift related
fluctuations stabilize after 3 hours.

Mixed pixels may occur in range measurements, which is described as “when a
laser spot is located at the very edge of an object, the measured range is that of a
combination of the foreground object and the background object”.

Target surface reflectance properties dramatically influence measurement errors.
The [Sick, 2002] list 9 common object surfaces and their reflectivity. From the list
asphalt pavement surface has medium reflectivity capability.
Incident angle of the laser beam affects on measurement errors. The paper concludes
that when the angle between +/- 20 degrees will not cause a significant error in the
range measurements. A relatively larger error was caused by the angel of 30 degrees
and the error is small again from 30 to 60 degrees. In the experiments scanner is put
parallel to the horizontal floor surface. But in our case we put the scanner downward
and we noticed significant errors happening on the center points (around 20 points get
effects). Even if we orient the scanner a small angle (10 degrees) away from horizontal
plane the center points are still noisy. The only way to get rid of this is to put the
scanner farther away from target object, e.g. 100 cm, but in that case range resolution
dramatically decreases.

The most important contribution of this paper is set up a mathematical model
between true range value and mean measured value. Therefore the model can be
used for estimating measurement errors. The equation is listed below,

r  k  b
where,
k, b are parameters;

r is the true estimation of range;
(24)
37
Chapter 6: Conclusions
 is the mean measured value.
Based on the paper we conduct extensive experiments both indoor and outdoor to
calibrate the TOF laser. The above equation is used to estimate true measurement
values and then roughly evaluate the ranging accuracy. All experiments are conducted
with laser angular resolution of 0.5 degree, transfer
rate of 500 Kbaud and profiling rate of 37 p/sec. If higher angular resolution is
selected the profiling rate will consequently decreases, which is not suitable for high
speed scanning application. Our experiments are illustrated as below,
(a)
(b)
(c)
(d)
(e)
(f)
Figure 5.1: Simulated 1st order Gaussian – Markov function with Yaw measurements. (a) 1st
order Gaussian Markov samples; (b) Histogram; (c) Autocorrelation sequences. (d)Yaw
measurements residual; (e) Histogram; (f) Autocorrelation sequence.
38
Chapter 6: Conclusions
Table 5.1: GPS and IMU Measurement Error Modeling Results
Measurements
Max
error
Min
error
*RMS
X (cm)
Y (cm)
Z (cm)
Roll (deg)
Pitch (deg)
Yaw (deg)
2.5
3.1
2.9
2.7
2.4
3.7
-2.4
-2.8
-3.3
-2.0
-2.3
-3.6
0.9
1.0
1.0
0.9
0.9
1.1
N
 (measurement
*RMS =
i 1
N
2
i
)
2nd AR Model
Parameters
b
a1
a2
1.35 -0.38
0.17
1.31 -0.37
0.28
1.34 -0.41
0.28
1.13 -0.13
0.04
1.06 -0.07
0.05
0.83
0.15
0.89
Manufactory
specifications
<2
<2
<4
<1
<1
<1
Chapter 6: Conclusions
39
40
Chapter 6: Conclusions
(a)
(b)
(c)
(d)
Figure 5.2: Kalman filter is used to smooth measurement errors; (a) Yaw measurements before
smoothing and after (unit: degree); (b) Yaw errors estimated using 2nd AR model (unit: degree); (d) Z
measurements before and after smoothing (unit: cm); (d) Z errors estimated using 2 nd AR model (unit: cm).
41
Chapter 6: Conclusions

An indoor experiment to calibrate optimal beam number (or point number in a
single scan). We mount the system on a cart and push it smoothly, therefore the
motions dynamics will be minimized but still simulate real data acquisition
process. The height from cart to surface is 88.1 cm. According to [Sick, 2002], the
distance from laser back to the mirror center is 9.3 cm. Therefore the true
measurement distance should be 78.8 cm ( d1 ). The laser incident angle is 0 degree
and scanner is place downward looking at the target surface. Totally 8,472 scans
are acquired. Then the scanner is lift up to 94.8 cm ( d 2 ) away from floor surface.
The D which is the difference between d1 and d 2 ( d1  d 2 ) is 16 cm. we compute
mean values for all beams from number 1 to number 361 at distance d1 and d2
respectively; compute their mean value difference. We found that the 171 th beam
(i.e. the 171th measured value of a scan) was the beam that most accurately
represents the true distance ( d1  d 2 = 16.9 cm) and that beam is subsequently
used in the following experiments.

A set of indoor experiments in which the sensor is downward looking and with its
scanning plane parallel to surface plane. The purpose of these experiments is to
calibrate optimal distance from laser to target surface. The sensor firstly is placed
stationary and then is moved by a cart smoothly. The laser is then tilted so that
there is a small incident angle, i.e. 10 degrees. Ye states that if the incident angle is
between – 20 / + 20 degree there will be no significant changes in range
measurements.

Outdoor experiments on a flat asphalt pavement surface and a flat concrete
pavement surface, respectively. We mount the system on a cart and the height
from laser window center to surface is measured as 73 cm. The sensor incident
angle is 0 degree and scanner is place downward looking at the target surface.
We then use the linear model (i.e. equation 24) to estimate true range value.
Parameters values have been calibrated in [Ye, 2002], with k  1.0002, b  3.6 (unit
is mm). Those values are estimated from an empirical indoor environment with data
acquired from a medium grey target (RGB = 127) at 0 degree orientation angle. It
represents the median situation. Then errors are computed respectively and compared

with measured ones. The result is shown in figure 16 with r r vs. the scan number.

Where r and r denote true and measured value respectively. Table 9 summarizes
measurement errors for optimal distance experiment. We can conclude that when the
sensor is place around 80cm from target the range accuracy is maximized. Figure 17
shows experiment results with distance vs. error. The global minimum error is
achieved with the distance of 73cm. Starting from distance of 60cm the error
decreases fast until passing after distance of 100cm. that is we can say the optimal
distance range is 60cm – 100cm.
42
Chapter 6: Conclusions
Table 10 shows the pavement surface properties effects on range accuracy. The indoor
concrete surface is the most flat one therefore its accuracy is the smallest. On the other
hand when scanning a coarse asphalt surface but still flat we expect the accuracy
performance worse. Outdoor scanning has many environmental factors to influence
ranging, that is for almost the same flat concrete surface indoor accuracy is better than
outdoor one.
Table 5.2: Errors Influenced by Surface Properties
Experiment
Indoor concrete flat surface
Outdoor asphalt surface
Outdoor concrete surface

Error upper
bound (mm)
16.8
30.4
19.6
Error lower
bound (mm)
13.2
-19.6
-11.4
Error (mm)
5.6
7.8
6.1
The manufactory specification states the laser has +/- 15 mm system error with 5
mm statistical error when being set as (mm-mode with 1…8m range). Compared
with our experiments, most static experiments generate better accuracy while all
kinematic experiments have worse performance. The motion dynamics add
significant errors into measurement results. Therefore when a van is used as
measurement vehicle the driving speed should not be moderate otherwise errors
will be introduced because of vehicle vibration and tire friction.
Table 11 summarizes the system equipment accuracy performance. The true value of
GPS and IMU are estimated using Kalman filter and AR model. Then the measured
values are used to compute RMS. The range accuracy is estimated from outdoor
asphalt flat surface scanning, which can be the typical pavement surface encountered
outdoor.
5.1.3
3D Modeling Accuracy
We use our system to scan many indoor and outdoor objects. In general indoor objects
are well structured and have well-defined features therefore they are better to test
system capability; while outdoor targets can best simulate real cracks inspection
applications. To compute accuracy we firstly select features from 3D models, such as
point to point distance, maximum or minimum depth, and round object radius. Then
we manually measured features and compare results from estimated values from 3D
models. 3D model manipulation is conducted in Rapidform (www.rapidform.com), a
commercial 3D scan and metrology software. Actually we estimated features 10 times
43
Chapter 6: Conclusions
for each comparison and then compute the mean values as the finally results. Table 10
lists our results and figure 18 explains experimental objects and their features.
One can observe that, the depth accuracy is worse than width and length
measurements. The measurement error is worse than manufactory specifications.
Vehicle dynamics add a large degree of noise into the measuring process.
Table 5.3: Optimal Distance Experimental Results
Experiment
Distance
(cm)
Placement
Mobile
Error upper
bound (mm)
Error lower bound
(mm)
RMS
(mm)

n
 (r
i 1
i
 r)2
n
31+parallel
31
Cart and static
14.7
-6.3
5.2
Cart and static
14.8
-5.1
4.9
Cart and static
13.3
-4.7
4.9
Cart and static
13.1
-7.0
5.1
Cart and static
11.7
-6.3
5.2
Cart and static
14.4
-3.6
5.7
73
Parallel to horizontal
plane
Parallel to horizontal
plane
Parallel to horizontal
plane
Parallel to horizontal
plane
Parallel to horizontal
plane
Parallel to horizontal
plane
Downward
51+parallel
51
76+parallel
76
101+parallel
101
152+parallel
152
203+parallel
203
79+downward
Cart and static
12.8
-5.1
4.1
95+downward
89
Downward
Cart and static
135
-8.5
4.3
Chapter 6: Conclusions
44
Chapter 6: Conclusions
45
Figure 5.3 Optimal distance experiment with distance vs. errors. The distance herein is
measured manually from sensor window center to the target surface. Errors are computed as RMS
(root mean square).
46
Chapter 6: Conclusions
Table 5.4 Laser-range, GPS, and IMU Measurement Error Estimation
Sensor
GPS
Experiment
Measurement
Outdoor static with 3,011
points obtained in 5 minutes
X (cm)
Y (cm)
Z (cm)
X and Y
Z
Roll (degree)
Pitch (degree)
Yaw (degree)
Roll, pitch
and yaw
(degree)
Range (cm)
Manufactory specifications
IMU
Outdoor static with 19,985
points obtained in 5 minutes
Manufactory specifications
Sick
Outdoor asphalt flat surface
Manufactory specifications
n
* RMS

r
i 1
ture
 restimated
n
Error upper
bound
+ 1.8
+ 2.4
+ 4.0
/
/
+ 0.5
+ 0.3
+ 0.5
/
Error lower
bound
- 1.8
- 2.5
- 3.0
/
/
- 0.3
- 0.2
- 0.3
/
*RMS
+ 3.0
+ 1.5
- 2.0
- 1.5
7.8
0.5
0.6
1.0
1.6
2
4
2.4
1.8
5.3
<1
47
Chapter 6: Conclusions
5.2
Video Mosaic Results
The main purpose of this part of work is to prepare a large range texture image
covering whole scanning area for 3D models. According to [Chen, 1998] mosaic is
divided into two parts, image registration and image merging. In this project we
proposed two methods to do image registration. The first registration process select
feature points and intensity difference based registration and this approach requires
selecting a set of feature points from reference image and then exhaustively
calculating position difference between them. According to [Szeliski, 1996], a
homograph transformation matrix can be set up between two images taken at
successive time. This matrix explains spatial translation and rotation from first image
pixel coordinates to the second image ones or vise versa.
Suppose in image frame i pixel p( x, y ) transforms to image frame i  1 with p ' ( x ' , y ' ) ,
the transformation between them can be expressed as below,
p ' ( x ' , y ' )  T ( R( p( x, y)))
(25)
Where
p ' ( x ' , y ' ) be the image projection of pixel p( x, y ) ;
T be the translation matrix;
R be the rotation matrix.
By solving (18) image frame i is resample into the image frame i  1 ’s local coordinate
system. The two successive image frames then is registered as a single large image
frame. Feature points are selected respectively from two frames. Then transformation
is solved to minimize spatial position difference function (19) between points and its
projection using rotation and transformation.
E( R, T )   ( x '  x) 2   ( y '  y) 2
(26)
To minimize the above summation, we could either apply gradient-based optimization
techniques nongradient-based techniques such as down hill Simplex. Here we use
Levenberg-Marquardt interactive nonlinear minimization [Lee, 2002] and thus
compute correspondence ( R, T ) in the target image.
48
Chapter 6: Conclusions
The second registration method is rather straightforward. It uses GPS and IMU
measurement to compute transformation matrix. The advantage of this method is its
efficiency, especially for images which don’t include features. The disadvantage is the
measurement errors generate not so accurate transformation as one computed from
matching features.
5.2.1
Texture Mapping
Texture mapping is to assign a 2D texture image coordinate to each 3D range point.
The texture image coordinate is then interpolated between range points. The
assignment process is to compute a transformation matrix between 3D model points
and image pixels. To do this calibration of the camera with respect to the model is
required. Calibration uses feature points in 3D models and their correspondences in
2D texture images, and then find the transformation matrix parameter. The
transformation projects a model point X laser  ( X , Y , Z ) , in 3D laser local coordinates,
onto a 2D image point xcamera  (u , v) as the below:
X 
u 
    M Y 
v 
 Z 
(27)
Actually transformation matrix is determined by cameral models used. Thus in turn
determines the number of types of parameter in M . If a pin pole camera model of
perspective projection is used, according to [Tsai, 1987], the transformation M has 11
parameters, in which 6 parameters denote the translation and rotation of the camera
with respect to the local laser coordinate frame. 5 parameters denote the optical
properties of the cameras, which are also camera intrinsic parameter. They include a
1st order approximation of a coefficient to correct radial distortion.
If we can find 7 feature points from 3D model and their correspondences in 2D texture
image, then the camera model parameters can be solved. Corresponding information
can be supplied by cracks. When more than the minimum numbers of correspondence
are found, we can use the RANSAC parameter estimation method described in
[Fischler, 1981]. The paper attempts to identify and discard erroneous data, and thus
use only the most accurate data to find a solution.
49
Chapter 6: Conclusions
Table 5.5 3D Modeling Errors
3D Models
Features
Manually
measured

Estimated from
models m (cm)
Error

m  m (cm)
m (cm)
Indoor Flat indoor with a long
wooden log
Log width
8.2
7.8
-0.4
Log thickness
4.0
5.1
+1.1
Outdoor flat asphalt with a rounded
plastic box
Radius
12.5
11.9
-0.6
Outdoor flat asphalt with a handicap
mark
Mark width
12.3
Mark length
12.4
Outdoor flat asphalt with a L shape
cutting
D1
69.3
75.3
+6.0
D2
106.0
110.9
+4.9
Max slope
3.0
8.9
+6.9
Manhole radius
66.4
63.2
+3.2
Grass shape
N/A
Can be measured
N/A
Outdoor grass and cutting
50
Chapter 6: Conclusions
(a) Indoor Flat indoor with a long wooden log
(b) Features: log with and depth
(c) Outdoor flat asphalt with a rounded plastic box
(d) Box radius
(e) Outdoor flat asphalt with a longitudinal cutting
(f) Min and max cutting depth
51
Chapter 6: Conclusions
D2
Slope
D1
(i) Outdoor flat asphalt with a L shape cutting
(j) Corner point to point distance
(k) Outdoor flat rugged surface with a big crack
(l) Min and max depth and width
Slope
(m) Outdoor grass and cutting
Figure 5.4: 3D models and their features for accuracy estimation.
D
(n) A manhole radius
Chapter 6: Conclusions
5.3
52
Examples and Discussion
We conduct 2 outdoor large scale experiments on campus to evaluate system stability
and capability. Post-processing is performed off line for obtaining 3D geometry
models and 2D texture information. The below describes the 2 data collection
processes in details.

The first data set is collected by driving a van along an asphalt roadway and then
turns into a mix surface parking lot. The test site is carefully selected therefore the
data have varied features which can be found on a normal pavement surface. These
features are such as cracks, cuttings, grass, holes etc. The measurement vehicle
ran a course of 419 m at a speed of 3 mph to 5 mph. A GIS map showing testing
site and the vehicle trajectory plot are presented in figure 20. The path covers an
area of 104 m x 321m. Totally 1,291 GPS points, 8,840 laser scans and 2,543
images are measured respectively as the vehicle moved forward. The spacing
between two closest scans is 3 cm to 6 cm. For each scan we select 313 from 361
points and 1D cubic interpolation is used to uniformly resample them up to 601
points. The spacing between 2 closet points is 1cm. The moving path is shown in
figure 19 and modeling results are shown in figure 20.

The second experiment is conducted in a parking lot. The measurement van drives
in a U shape curve. The test area covers a rectangular of 267 m long and 120 m
width. The 3D model is more than 20 Mbytes. We split it into 4 smaller pieces,
with each piece of 7 Mbytes (with redundancy). IVP ranger is also used for crack
modeling. For the ranger data is rather large we just use it as an aid in a small
cracking area. Figure 19 shows the experiment results.

Then we show a set of small models which can be normally found on pavement
surface. As a comparison IVP high resolution models are also shown. These high
resolution models are obtained by moving a cart not a driving a van instead. Their
results are shown in figure 22.

Finally video image results are shown in figure 23, figure 24 and figure 25. Figure
23 shows a large scale mosaic result. Figure 24 and figure 25 otherwise show
“crack slit” which can be used for crack detection and classification. Observing the
results we can notice that light illumination influence image quality significantly.
Choosing an optimal time and lighting condition are critical considerations for
texture image data acquisition. Table 13 summaries mosaic results.
53
Chapter 6: Conclusions
TABLE 13
IMAGE MOSAIC RESULTS
Figure
Figure 23 (a)
Figure 23 (b)
Figure 24 (a)
Figure 24 (b)
Figure 25
No. of Frames
45
45
2
3
4
Pixel size
754 x 4744
720 x 4840
735 x 638
738 x 811
734 x 951
(a)
(b)
Figure 5.5: Moving trajectory. (a) 2D plot; (b) Plot in a high resolution
satellite image of UT campus. The van moves on a roadway along Knoxville
river, and then turns right into a parking lot.
54
Chapter 6: Conclusions
104 m
372 m
(a)
(b)
(c)
(d)
Figure 5.6: A large scale data (a) Large scale 3D model; (b) 1st round zoom and show
more details; (c) The scale of 1st round zoom operation; (d) Colorcoded surface patch.
55
Chapter 6: Conclusions
(a)
(b)
(c)
(d)
(e)
(f)
Figure 5.7: Large scale (a) Large scale 3D model; (b) Trajectory plot; (c) The scale of
117.1m
1st;
(d) plot trajectory in GIS image; (e) 2nd round zoom to show small details about
roughness of surface obtained from IVP; (f) Small crack patch obtained from IVP.
26.2 m
56
Chapter 6: Conclusions
(a) grass
(b) handicapped mark
(c) manhole
(d) Texture model
57
Chapter 6: Conclusions
(g) IVP surface patch in a L curve
(f) IVP cracking slit
(g) IVP crack slit
Figure 5.8: Varied 3D models built from Sick TOF laser and
IVP ranger.
Chapter 6: Conclusions
Figure 5.9: Mosaic result. The right result is obtained from GPS/IMU pose
estimation. The left one is obtained from matching features.
58
59
Chapter 6: Conclusions
(a)
(c)
Figure 5.10: Mosaic result. (a) Using 2 frames; (b) Using 3 frames.
Chapter 6: Conclusions
Figure 5.11: Mosaic result using 4 frames.
60
Chapter 6: Conclusions
6
61
CONCLUSIONS
The paper shows fusing laser, GPS, IMU data can achieving accurate and eyeappealing 3D surface models. These models together with image information can be
used for many applications, including terrain modeling, pavement inspection, tunnel
inspection and even urban architecture reconstruction. The integrated approach can be
great potential in fast digitizing and representing physical world with less human
efforts. That is our integrated system is capable of scanning both terrain and its
accompanying structures, at the desired level of resolution. For example, the micro
system scanning configuration is appropriate of inspection tasks of pavement surface.
The macro system is efficient for larger scale terrain profiling with little human
intervention kilometers’s worth of data is captured and represented in real time within
a few minutes. The system has been developed modularly that is the system can be
reconfigured according to the application with minimal amount
of effort and changes to the processing.
Latter efforts will go on system accuracy improvement and obtaining eye-appealing
3D models. Some of efforts will be:

A decent and robust algorithm for fast triangulating range points and
representing flat road surface;

Improving the navigation estimation algorithm in the ways of de-nosing,
fusing and interpolation;

An accurate algorithm of enhancing small-scale detail regions in the presence
of noise. Such details are as cracks, cutting, and rocks.
62
Bibliography
BIBLIOGRAPHY
Bibliography
63
[Belkasim, 1991] S. O. Belkasim, M. Shridhar, and M. Ahmadi, “Pattern Recognition
with Moment Invariants: a Comparative Study and New Results,” Pattern
Recognition, Vol. 24, No. 12, 1991, pp. 1117-1138.
[Chen, 1998] C. Y. Chen, “Image Stitching - Comparisons and New Techniques,”
1998.
[Cheng, 1990] H. D. Cheng, C. Tong, and Y. J. Lu, “VLSI curve detector,” Pattern
Recognition, Vol. 23, No.1/2, 1990, pp.35–50.
[Cheng, 1999a] H. D. Cheng, J. R. Chen, C. Glazier, and Y. G. Hu, “Novel Approach
to Pavement Cracking Detection Based on Fuzzy Set Theory”, J. of Computing in
Civil Eng., Vol. 13, No. 4, October 1999, pp. 270-280.
[Cheng, 2003] H. D. Cheng, X. J. Shi, and C. Glazier, “Real-Time Image
Thresholding Based on Sample Space Reduction and Interpolation Approach”, J. of
Computer in Civil Eng., Vol. 17, Issue 4, October, 2003, pp. 264-272.
[Cross, 1983] G. Cross and A. Jain, "Markov Random Field Texture Models," IEEE
Trans. PAMI, Vol. 5, No.1, 1983, pp.25-39.
[Feijoo, 1991] C. Feijoo, A. Asesio, and F. Perez, “New Practical Method for
Measurement Error Determination in Radar Systems under Real Traffic Conditions,”
IEEE Proc. –F, Vol. 138, No. 6, Dec. 1991, pp. 525 – 530.
[Feijoo, 1992] C. Feijoo, F. Perez, and A. Asesio, “Real Time Estimation of GPS
Measurement Errors from Vehicle Position Data,” Vehicle Navigation and
Information Systems, VNIS., The 3rd International Conference on September 2 - 4,
1992, pp. 369 – 374.
[Fischler, 1981] M. A. Fischler, and R. C. Bolles, “Random Sample Consensus: A
Paradigm for Model Fitting With Applications to Image Analysis and Automated
Cartography,” Communications of the ACM, Vol. 24, No. 6, 1981, pp. 381-395.
[Fu, 1981] K. S. Fu and J. K. Mui, “A Survey on Image Segmentation,’’ Pattern
Recognition, Vol.13, 1981, pp.3–16.
[Fukuhara, 1990] T. Rukuhara, K. Terada, M. Nagao, A. Kasahara, and S. Ichihashi,
“Automatic Pavement-Distress-Survey System,“ J. of Transp. Eng., Vol. 116, No.1,
January, 1990, pp. 280-286.
Bibliography
64
[Fundakowski, 1991] R. A. Fundakowski et al., “Video Image Processing for
Evaluating Pavement Surface Distress”, Final Report, National Cooperative Highway
Research Program1-27, September, Washington, D.C., 1991.
[Goldberg, 1989] D. E. Goldberg, “Genetic Algorithm in Searching, Optimization, and
Machine Learning,” Addison-Wesley, Reading, Mass, 1989.
[Gonzalez, 1992] R. C. Gonzalez and R. E. Woods, “Digital image processing”, 3rd
Ed., Addison-Wesley, Reading, Mass, 1992
[Grinstead, 2005] B. .Grinstead, A. Koschan, D. Page, A. Gribok, and M. A. Abidi,
“Vehicle-borne Scanning for Detailed 3D Terrain Model Generation,” SAE
International, 2005.
[Groeger, 2003] J. L. Groeger, P. Stephanos, P. Dorsey, and M. Chapman,
“Implementation of Automated Network-level Crack Detection Processes in
Maryland,” Transportation Research Record 1860, Transportation Research Board,
National Research Council, Washington, D.C., 2003, pp. 109-116.
[Guo, 2002] L. Guo, Q. Zhang and S. Han, “Position Estimate of Off-Road Vehicles
Using a Low-Cost GPS and IMU,” 2002 ASAE (The Society for Engineering in
Agriculture, Food and Biological System) Annual International Meeting, July, 2002.
[Hass, 1990] C. Haas, S. McNeil, "Criteria for Evaluating Pavement Imaging
Systems," Transportation Research Record, No. 1260, 1990.
[Howe, 1997] R. Howe, and G. G. Clemena, “Assessment of the Feasibility of
Developing and Implementing an Automated Pavement Distress Survey System
Incorporating Digital Image Processing,” Final Report for Virginia Department of
Transportation, Report No. VTRC 98-R1, Project No. 9128-010-940, Virginia
Transportation Research Council, Charlottesville, VA, November, 1887.
[Hu, 1962] M. K. Hu, “Visual Pattern Recognition by Moment Invariants,” IRE
Transactions on Information Service, Vol. 1, IT-8, February, 1962, pp. 179-187.
[Hu, 2000] R. Hu and M. M. Fahmy, “Texture Segmentation Based on a Hierarchical
Markov Random Texture Field,” Signal Processing, No. 26, Vol.3, March, 1992, pp.
285-305.
[Javidi, 2003] B. Javidi, “Pilot for Automated Detection and Classificationo of Road
Surface Degradation Features,” Report for Project JHR 03-293, Optical/Imaging
Processing Laboratory, The University of Connecticut, November, 2003.
Bibliography
65
[Javidi, 2004] B. Javidi, D. Kim and S. Kishk, “A Laser-Based 3D Data Acquisition
System for the Analysis of Pavement Distress and Roughness,” Report for Project
JHR 03-300, Optical/Imaging Processing Laboratory, The University of Connecticut,
December, 2004.
[Laurent, 1997] J. Laurent, M. Talbot, and M. Doucent, “Road Surface Inspection
using Laser Scanners Adapted for the High Precision Measurements of Large Flat
Surfaces”, International Conference on Recent Advances in 3-D Digital Imaging and
Modeling (3DIM '97), Ottawa, Canada, Dec., 1997.
[Lee, 2002] S. C. Lee and P. Bajcsy, “Multi-Instrument Analysis from Point and
Raster Data”, Tech. Report, No. alg04-02, National Center for Supercomputing
Applications, Champaign, IL, 2002.
[Leica, 2003] Leica AG. Heerburgg, “Outside World Interface (OWI) Interface
Control Document, Project GPS System 500,” No. TUH 3421, May 2003.
[Li, 1991] L. Li, P. Chan, A. Rao, and R. L. Lytton, “Flexible Pavement. Distress
Evaluation Using Image Analysis,” Proc., 2nd Int. Conf. on Applications of Advanced
Technologies in Transp. Engrg., 1991, pp. 473-477.
[Lu, 2002] J. J. Lu, X. Mei and M. Cunaratne, “Development of an Automatic
Detection System for Measuring Pavement Crack Depth on Florida Roadways,” Final
Research Report to Florida Dept. of Transp., Department of Civil and Environment
Engineering, University of South Florida, January, 2002.
[Luhr, 1999] D. R. Luhr, "A proposed methodology to Quantify and Verify
Automated Crack Survey measurements," the 78th Annual Transportation Research
Board Meeting, Washington, D.C., January, 1999.
[Meignen, 1997] D. Meignen, M. Bernadet,, and H. Briand, “One Application of
Neural Network for Detection of Defects Using Video Data Bases: Identification of
Road Distress,“ J. IEEE Computer Society, 1997, pp. 459-464.
[McGhee, 2004] K. H. Mcghee, “Automated Pavement Distress Collection
Techniques – A Synthesis of Highway Practice,” Report for National Cooperative
Highway Research Program (Synthesis 334), Transportation Research Board of the
National Academies, 2004.
[Monti, 1995] M. Monti, “Large-area laser scanner with holographic detector optics
for real-time recognition of cracks in road surfaces,” Optical Engineering, Vol. 34, No.
7, Brian J. Thompson, Ed., 1995, pp. 2017-2023.
Bibliography
66
[Nassar, 2003] S. Nassar, K. P. Schwarz, A. Noureldin and N. E. Sheeimy, “Modeling
Inertial Sensor Errors Using Autoregressive (AR) Models”, Proc. ION NTM-2003,
Anaheim, USA, January 22-24, 2003, pp.116-125.
[Ritchie, 1990] S. G. Ritchie, “Digital Imaging Concepts and Applications in
Pavement Management,” J. of Transp. Eng., Vol. 116, No.1, January, 1990, pp. 287298.
[Rumelhart, 1986] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning
Representations by Back-Propagating Errors,” Nature, Vol. 323, No. 9, 1986, pp. 533536.
[Sheimy, 2004] N. El-Sheimy and N. Nassar, “Wavelet De-Nosing for IMU
Alignment,” IEEE Aerosp. Electron. Syst. Mag., Vol.19, No.10, October 2004, pp.32 39.
[Sick, 2002] Sick AG, “LMS 200 / LMS 211 / LMS 220 / LMS 221 / LMS 291 Laser
Measurement Systems Technical Description,” Division Auto Indent., Germany, 2002.
[Simon, 1998] D. Simon, “Kalman Filtering”, Innnovatia software, 1998.
www.Innovatia.com/software
[Skaloud, 1999] J. Skaloud, “Optimizing Georeferencing of Airborne Survey Systems
by INS/GPS,” Ph.D. Thesis, Dept. of Geometrics Engineering, Univ. of Calgary,
Calgary, Alberta, Canada, UCGE Reports No. 20126.
[Standard, 2001] Standard Practice for Quantifying Cracks in Asphalt Pavement
Surface, AASHTO Designation PP44-01, American Association of State Highway and
Transportation Officials, Washington, D.C., Apr. 2001.
[Szeliski, 1996] R. Szeliski, “Video Mosaics for Virtual Environment”, IEEE
Computer Graphics and Applications, vol. 16, No. 2, March, 1996.
[Tao, 1994] S. Tao, N. Kehtarnavaz, and R. Lytton, “Image-Based Expert-System
Approach to Distress Detection on CRC Pavement,” J. Transportation Engineering,
Vol. 120, No. 1, January/February 1994, pp. 52-64.
[Tao, 1998] C. V. Tao, “Mobile Mapping Technology for Road Network Data
Acquisition,” International Symposium of Urban Multimedia/3D Mapping, Tokyo,
Japan, June 8-9, 1998.
Shape
Histograms
Bibliography
67
[Teague, 1993]
M. R. Teague, “Image Analysis via the General Theory of
Moments,” Optical Society American Journal, Vol. 70, No. 8, August, 1993, Aviation
Administration, the University of Wisconsin-Madison, 2004, pp. 920-930.
[Tsai, 1987] R. Y. Tsai, "A versatile Camera Calibration Technique for HighAccuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Cameras and
Lenses", IEEE Journal of Robotics and Automation, Vol. RA-3, No. 4, 1987, pp. 323344.
[Walker, 2004] D. Walker, “Pavement Surface Evaluation and Rating – Asphalt
Airfield Pavements,” Manual for the Federal
[Wang, 2003] K. C. P. Wang, “Transportation Research Circular: Automated Imaging
Technologies for Pavement Distress Survey”, Committee A2B06, Transportation
Research Board, National Research Council, Washington, D.D., draft, June, 2003.
[Ye, 2002] C. Ye and J. Borenstein, “Characterization of a 2-D Laser Scanner for
Mobile Robot Obstacle Negotiation,” Proceedings of the 2002 IEEE International
Conference on Robotics and Automation, Washington DC, USA, May, 2002, pp.
2512-2518.
[Zadeh, 1968] L. A. Zadeh, “Probability Measures of Fuzzy Events,” J. Mathematical
Anal. and Applications, Vol. 23, 1968, pp. 421–427.
[Zgonc, 1996] K. Zgonc K and J. D. Achenbach, “A neural network for crack sizing
trained by finite element calculations”, NDT&E Int., Vol. 29, 1996, pp. 147-55.
[Zhao, 2003] H. Zhao and R. Shibasaki, “Reconstructing a Textured CAD Model of
an Urban Environment Using Vehicle-Borne Laser Range Scanners and Line
Cameras”, Machine Vision and Applications, Vol. 14, No. 1, April, 2003, pp. 35 - 41.