Mata kuliah : T0283 - Computer Vision
Tahun
: 2010
Lecture 11
Stereo Reconstruction I
Learning Objectives
After carefully listening this lecture, students will be able
to do the following :
demonstrate 3D Stereo Computation by solving pointcorrespondence problems and fundamental matrix
Calculate object-depth information using disparity and
triangulation techniques.
January 20, 2010
T0283 - Computer Vision
3
Stereo Reconstruction
Shape (3D) from two (or more) images
known
camera
viewpoints
January 20, 2010
T0283 - Computer Vision
4
Examples
images
shape
surface
reflectance
January 20, 2010
T0283 - Computer Vision
5
Scenarios
The two images can arise from
A stereo rig consisting of two cameras
the two images are acquired simultaneously
OR
A single moving camera (static scene)
the two images are acquired sequentially
The two scenarios are geometrically equivalent
January 20, 2010
T0283 - Computer Vision
6
Stereo head
Camera on a mobile vehicle
January 20, 2010
T0283 - Computer Vision
7
Imaging geometry
? scene
central projection
point
camera centre, image point and
scene point are collinear
image
point
an image point back projects to a
ray in 3-space
depth of the scene point is
unknown
January 20, 2010
camera
centre
T0283 - Computer Vision
image plane
8
The objective
Given two images of a scene acquired by known cameras, compute
the 3D position of the scene (structure recovery)
Basic principle: triangulate from corresponding image points
• Determine 3D point at intersection of two back-projected rays
January 20, 2010
T0283 - Computer Vision
9
Corresponding points are images of the same scene point
Triangulation
C/
C
The back-projected points generate rays which intersect at the 3D scene point
January 20, 2010
T0283 - Computer Vision
10
An algorithm for stereo reconstruction
1. For each point in the first image determine the
corresponding point in the second image
(this is a search problem)
2. For each pair of matched points determine the 3D
point by triangulation
(this is an estimation problem)
January 20, 2010
T0283 - Computer Vision
11
The correspondence problem
Given a point x in one image, find the corresponding point in
the other image
This appears to be a 2D search problem, but it is reduced to a 1D search
by the epipolar constraint
January 20, 2010
T0283 - Computer Vision
12
Notation
The two cameras are P and P/, and a 3D point X is imaged
as
X
P
x
C
P/
x/
C/
Warning
for equations involving homogeneous quantities ‘=’ means ‘equal up to scale’
January 20, 2010
T0283 - Computer Vision
13
Epipolar geometry
January 20, 2010
T0283 - Computer Vision
14
Epipolar geometry
Given an image point in one view, where is the
corresponding point in the other view?
?
epipolar line
C
/
C
epipole
baseline
• A point in one view “generates” an epipolar line in the other view
• The corresponding point lies on this line
January 20, 2010
T0283 - Computer Vision
15
Epipolar line
Epipolar constraint
• Reduces correspondence problem to 1D search along an
epipolar line
January 20, 2010
T0283 - Computer Vision
16
Epipolar geometry continued
Epipolar geometry is a consequence of the
coplanarity of the camera centers and scene point
X
x
x/
C/
C
The camera centers, corresponding points and scene point lie
in a single plane, known as the epipolar plane
January 20, 2010
T0283 - Computer Vision
17
Nomenclature
X
l/
left epipolar
line
x
right epipolar
line
x/
e
e/
C/
C
• The epipolar line l/ is the image of the ray through x
• The epipole e is the point of intersection of the line joining the
camera centres with the image plane
this line is the baseline for a stereo rig, and
the translation vector for a moving camera
• The epipole is the image of the centre of the other camera:
e = PC/ , e/ = P/C
January 20, 2010
T0283 - Computer Vision
18
The epipolar pencil
X
e
e
/
baseline
As the position of the 3D point X varies, the epipolar planes
“rotate” about the baseline. This family of planes is known as an
epipolar pencil. All epipolar lines intersect at the epipole.
(a pencil is a one parameter family)
January 20, 2010
T0283 - Computer Vision
19
The epipolar pencil
X
e
e
/
baseline
As the position of the 3D point X varies, the epipolar planes
“rotate” about the baseline. This family of planes is known as an
epipolar pencil. All epipolar lines intersect at the epipole.
(a pencil is a one parameter family)
January 20, 2010
T0283 - Computer Vision
20
Epipolar geometry ex. 1: parallel cameras
Epipolar geometry depends only on the relative pose (position and
orientation) and internal parameters of the two cameras, i.e. the position
of the camera centres and image planes. It does not depend on the scene
structure (3D points external to the camera).
January 20, 2010
T0283 - Computer Vision
21
Epipolar geometry ex. 2: converging cameras
e
e
/
Note, epipolar lines are in general not parallel
January 20, 2010
T0283 - Computer Vision
22
Homogeneous notation for lines
January 20, 2010
T0283 - Computer Vision
23
• The line l through the two points p and q is l = p x q
Proof
• The intersection of two lines l and m is the point x = l x m
Example: compute the point of intersection of the two lines l
and m in the figure below
y
1
l
m
2
which is the point (2,1)
January 20, 2010
T0283 - Computer Vision
x
24
Matrix representation of the vector cross product
January 20, 2010
T0283 - Computer Vision
25
Example: compute the cross product of l and m
January 20, 2010
T0283 - Computer Vision
26
Algebraic representation of epipolar geometry
We know that the epipolar geometry defines a mapping
January 20, 2010
/
x
l
point in first
image
epipolar line in
second image
T0283 - Computer Vision
27
Derivation of the algebraic expression
Outline
Step 1: for a point x in the first image
P
back project a ray with camera P
/
P
Step 2: choose two points on the ray and
project into the second image with camera P/
Step 3: compute the line through the two
image points using the relation l/ = p x q
January 20, 2010
T0283 - Computer Vision
28
• choose camera matrices
internal
calibration
rotation
translation
from world to camera
coordinate frame
• first camera
world coordinate frame aligned with first camera
• second camera
January 20, 2010
T0283 - Computer Vision
29
Step 1: for a point x in the first image
back project a ray with camera
A point x back projects to a ray
P
where Z is the point’s depth, since
satisfies
January 20, 2010
T0283 - Computer Vision
30
Step 2: choose two points on the ray and
project into the second image with camera P/
Consider two points on the ray
/
• Z = 0 is the camera centre
•Z=
P
is the point at infinity
Project these two points into the second view
January 20, 2010
T0283 - Computer Vision
31
Step 3: compute the line through the two
image points using the relation l/ = p x q
Compute the line through the points
Using the identity
F is the fundamental matrix
F
January 20, 2010
T0283 - Computer Vision
32
Example 1: compute the fundamental matrix for a parallel camera
X Y f
Z
f
• reduces to y = y/ , i.e. raster correspondence (horizontal scan-lines)
January 20, 2010
T0283 - Computer Vision
33
X Y f
Z
f
January 20, 2010
T0283 - Computer Vision
34
Example 2: compute F for a forward translating camera
f
X Y f
Z
January 20, 2010
T0283 - Computer Vision
35
f
X Y f
Z
first image
January 20, 2010
second image
T0283 - Computer Vision
36
January 20, 2010
T0283 - Computer Vision
37
January 20, 2010
T0283 - Computer Vision
38
Summary: Properties of the Fundamental matrix
January 20, 2010
T0283 - Computer Vision
39