IGARSS 2011, Vancouver, Canada
July 24-29, 2011
Mark Pritt, PhD
Lockheed Martin
Gaithersburg, Maryland mark.pritt@lmco.com
Kevin LaTourette
Lockheed Martin
Goodyear, Arizona kevin.j.latourette@lmco.com




Georegistration is the assignment of 3-D geographic coordinates to the pixels of an image.



It is required for many geospatial applications:
Fusion of imagery with other sensor data
Alignment of imagery with GIS and map graphics
Accurate 3-D geolocation
Inaccurate georegistration can be a major problem:
Correctly aligned
Misaligned
GIS
2



Our solution is image registration to a high-resolution digital elevation model (DEM):


A DEM post spacing of 1 or 2 meters yields good results.
It also works with 10-meter post spacing.
Works with terrain data derived from many sources:




LIDAR: BuckEye, ALIRT, Commercial
Stereo Photogrammetry: Socet Set® DSM
SAR: Stereo and Interferometry
USGS DEMs
3




Create predicted images from the DEM, illumination conditions, sensor model estimates and actual images.
Register the images while refining the sensor model.
Iterate.
Aerial Video
Sensor
Illumination
Occlusion
Scene
Shadow
Predicted
Images
4

Predicted
Image from DEM
Predicted
Image from
Aerial Image
The algorithm identifies tie points between the predicted and the actual images by means of NCC
(normalized cross correlation) with RANSAC outlier removal.
Registration
Tie Point
Detections
5

 The algorithm uses the refined sensor model as the initial guess for the next video frame:
Initial
Camera
• Estimate camera model
• Use camera focal length
& platform
GPS if avail.
Register
• Predict images from
DEM and camera
• Register images with
NCC
Refine
• Compose registration fcn & camera
• LS fit for better cam estimate
• Iterate
Next
Frame
• Register to previous frame
• Compose with cam of prev. frame for init. cam estimate
Iterate
• Iterate for each video frame
Finish
• Trajectory
• Propagate geo data from DEM
• Resample images for orthomosaic
 The refined sensor model enables georegistration.
 Exterior orientation: Platform position and rotation angles
 Interior orientation: Focal length, pixel aspect ratio, principal point and radial distortion
6

Inputs:
Aerial Motion Imagery over
Arizona, U.S.
1/3 Arc-second
USGS DEM
16 Mpix, 3.3 fps, panchromatic
Area: 64 km 2
Post Spacing: 10 m
7

Problem: Too shaky to find moving objects
Zoomed to full resolution (1 m)
8





Outputs:
Sensor camera models
Images georegistered to DEM
Platform trajectory
9

ATV
Vehicle
Pickup
Truck
Human
Video is now stabilized, and as a result, moving objects are easily detected.
10

Inputs:
Oblique Motion Imagery Over
Arizona, U.S.
LIDAR DEM
16 Mpix, 3.4 fps, pan
Area: 24 km 2
Post Spacing: 1 m
11

Stabilized
Video Inset
Aligned
Map
Graphics
Target
Tracking
Background
LIDAR DEM
Map coordinates
Orthorectified
Video
Aligned
Map
Graphics
12

 How fast does the algorithm converge?
IMAGE 1
Num tie points:
RMSE:
Mean Δx:
Mean Δy:
Sigma Δx:
Sigma Δy:
IMAGE 591
Num tie points
RMSE
Mean Δx
Mean Δy
Sigma Δx
Sigma Δy
1
Camera Iteration
2 3
319 318 282
17.4
1.4
-3.8
15.8
6
4.8
-0.7
-0.1
4
2.6
2.9
0.1
0
2.5
1.5
1
Camera Iteration
2 3
681 687 681
2.7
1
0.9
2.1
0.9
0.6
0
0
0.5
0.2
0.3
0
0
0.3
0.1
3
2,5
2
1,5
1
0,5
0
20
18
16
14
12
10
8
6
4
2
0
Tie Point Residuals
RMSE mean sigma
1 2
Camera Iteration
3
Tie Point Residuals
RMSE mean sigma
1 2
Camera Iteration
3
The initial error is high, but it decreases after only several iterations.
Subsequent frames have better initial sensor model estimates and require only 2 iterations.
13

Inputs:
Aerial Video Over
Arizona, U.S.
LIDAR DEM
720 x 480 Color 30 fps
Area: 24 km 2
Post Spacing: 1 m
14

Background
Image
Draped Over
DEM
Map coordinates
Orthorectified
Video
Aligned
Map
Graphics
15

Map Graphics Stay Aligned with Features in Video
16

Inputs:
MWIR Video Over White
Tank Mountains in Arizona
Commercial
LIDAR DEM
1 Mpix, 3.3 fps
Post Spacing: 2 m
17

Video
Mosaic
Background
LIDAR DEM
Inset:
Original
Video with Map
Graphics
Overlay
Video Mosaic
Georegistered and
Draped Over Mountains in Google Earth
18

Click picture to play video
19






We have introduced a new method for aerial video georegistration and stabilization.
It registers images to high-resolution DEMs by:



Generating predicted images from the DEM and sensor model;
Registering these predicted images to the actual images;
Correcting the sensor model estimates with the registration results.
Processing speed is 1 sec per 16-Mpix image on a PC.
Absolute geospatial accuracy is about 1-2 meters.
 We are developing a rigorous error propagation model to quantify the accuracy.
Applications:
 Video stabilization and mosacs


Cross-sensor registration
Alignment with GIS map graphics
20