Mata kuliah : T0283 - Computer Vision
Tahun
: 2010
Lecture 12
Stereo Reconstruction II
Learning Objectives
After carefully listening this lecture, students will be able
to do the following :
demonstrate 3D stereo computation by solving pointcorrespondence problems and fundamental matrix.
Calculate object-depth information using disparity and
triangulation techniques
January 20, 2010
T0283 - Computer Vision
3
An algorithm for stereo reconstruction
1. For each point in the first image determine the
corresponding point in the second image
(this is a search problem)
2. For each pair of matched points determine the 3D
point by triangulation
(this is an estimation problem)
January 20, 2010
T0283 - Computer Vision
4
Epipolar line
Epipolar constraint
• Reduces correspondence problem to 1D search
along an epipolar line
January 20, 2010
T0283 - Computer Vision
5
Algebraic representation of epipolar geometry
We know that the epipolar geometry defines a
mapping
/
x
l
point in first
image
epipolar line in
second image
• the map only depends on the cameras P, P/ (not on structure)
• it will be shown that the map is linear and can be written as
January 20, 2010
T0283 - Computer Vision
6
Stereo correspondence
algorithms
January 20, 2010
T0283 - Computer Vision
7
Problem statement
Given: two images and their associated cameras compute
corresponding image points.
Algorithms may be classified into two types:
1. Dense: compute a correspondence at every pixel
2. Sparse: compute correspondences only for features
The methods may be top down or bottom up
January 20, 2010
T0283 - Computer Vision
8
Top down matching
1. Group model (house, windows, etc) independently in each image
2. Match points (vertices) between images
January 20, 2010
T0283 - Computer Vision
9
Bottom up matching
• epipolar geometry reduces the correspondence search from 2D
to a 1D search on corresponding epipolar lines
• 1D correspondence problem
A
B
C
b/ c/
b
a
January 20, 2010
T0283 - Computer Vision
c
a/
10
Example image pair – parallel cameras
January 20, 2010
T0283 - Computer Vision
11
First image
January 20, 2010
T0283 - Computer Vision
12
Second image
January 20, 2010
T0283 - Computer Vision
13
Dense correspondence algorithm
Parallel camera example – epipolar lines are corresponding rasters
epipolar
line
Search problem (geometric constraint): for each point in the left image, the
corresponding point in the right image lies on the epipolar line (1D ambiguity)
Disambiguating assumption (photometric constraint): the intensity neighbourhood
of corresponding points are similar across images
Measure similarity of neighbourhood intensity by cross-correlation
January 20, 2010
T0283 - Computer Vision
14
Intensity profiles
• Clear correspondence between intensities, but also noise and ambiguity
January 20, 2010
T0283 - Computer Vision
15
Normalized Cross Correlation
region A
region B
write regions as vectors
a
b
vector a
January 20, 2010
T0283 - Computer Vision
vector b
16
Cross-correlation of neighbourhood regions
epipolar
line
January 20, 2010
T0283 - Computer Vision
17
left image band
right image band
1
cross
correlation
0.5
0
x
January 20, 2010
T0283 - Computer Vision
18
target region
left image band
right image band
1
cross
correlation
0.5
0
x
January 20, 2010
T0283 - Computer Vision
19
Why is cross-correlation such a poor measure in
the second case?
1. The neighborhood region does not have a “distinctive” spatial
intensity distribution
2. Foreshortening effects
front-parallel surface
imaged length the same
January 20, 2010
slanting surface
imaged lengths differ
T0283 - Computer Vision
20
Limitations of similarity constraint
Textureless surfaces
Occlusions, repetition
Non-Lambertian surfaces, specularities
January 20, 2010
T0283 - Computer Vision
21
Results with window search
Data
Window-based matching
January 20, 2010
T0283 - Computer Vision
Ground truth
22
Sketch of a dense correspondence algorithm
For each pixel in the left image
compute the neighbourhood cross correlation along the
corresponding epipolar line in the right image
the corresponding pixel is the one with the highest
cross correlation
Parameters
size (scale) of neighbourhood
search disparity
Other constraints
uniqueness
ordering
smoothness of disparity field
Applicability
textured scene, largely fronto-parallel
January 20, 2010
T0283 - Computer Vision
23
Example dense correspondence algorithm
left image
January 20, 2010
right image
T0283 - Computer Vision
24
3D Reconstruction
right image
January 20, 2010
T0283 - Computer Vision
depth map
intensity = depth
25
Pentagon example
left image
right image
range map
January 20, 2010
T0283 - Computer Vision
26
Rectification
For converging cameras epipolar lines are not paralle
e
January 20, 2010
e/
T0283 - Computer Vision
27
Project images onto plane parallel to baseline
epipolar plane
January 20, 2010
T0283 - Computer Vision
28
Rectification continued
Convert converging cameras to parallel camera
geometry by an image mapping
Image mapping is a 2D homography (projective transformation)
January 20, 2010
T0283 - Computer Vision
29
Example
original stereo pair
rectified stereo pair
January 20, 2010
T0283 - Computer Vision
30
Example: depth and disparity for a parallel camera stereo rig
Then, y/ = y, and the disparity
Derivation
x
x/d
Note
• image movement (disparity) is inversely proportional
to depth Z
• depth is inversely proportional to disparity
January 20, 2010
T0283 - Computer Vision
31
Triangulation
January 20, 2010
T0283 - Computer Vision
32
Problem statement
Given: corresponding measured (i.e. noisy) points x and x/ ,
and cameras (exact) P and P/, compute the 3D point X
Problem: in the presence of noise, back projected rays do not intersect
x
x/
rays are skew in space
C/
C
Measured points do not lie on corresponding epipolar lines
January 20, 2010
T0283 - Computer Vision
33
1. Vector solution
C
C/
Compute the mid-point of the shortest line between the two rays
January 20, 2010
T0283 - Computer Vision
34
2. Linear triangulation (algebraic solution)
January 20, 2010
T0283 - Computer Vision
35
January 20, 2010
T0283 - Computer Vision
36
3. Minimizing a geometric/statistical error
January 20, 2010
T0283 - Computer Vision
37