Mata kuliah : T0283 - Computer Vision
Tahun
: 2010
Lecture 13
Structure from Motion
Learning Objectives
After carefully listening this lecture, students will be able
to do the following :
Show how to determine scene structures and camera
motions using point-correspondence and fundamental
matrix
January 20, 2010
T0283 - Computer Vision
3
Structure and Motion
Unknown
camera
viewpoints
Reconstruct
• Scene geometry
• Camera motion
January 20, 2010
T0283 - Computer Vision
4
Recap : Stereo reconstruction from 2 views
Given cameras
Epipolar geometry:
compute fundamental matrix
Correspondence search:
1D search for corresponding points x  x/
along epipolar line l/ = F x
Triangulation:
compute 3D point X from x  x/ , and P, P/
Now, structure and motion …
January 20, 2010
T0283 - Computer Vision
5
Structure and Motion: Problem statement
Given 2 (or more) images of a scene, compute the
scene structure and the camera motion
January 20, 2010
T0283 - Computer Vision
6
First image
January 20, 2010
T0283 - Computer Vision
7
Second image
January 20, 2010
T0283 - Computer Vision
8
Second motion example
January 20, 2010
T0283 - Computer Vision
9
First image
January 20, 2010
T0283 - Computer Vision
10
Second image
January 20, 2010
T0283 - Computer Vision
11
Interest points computed for each frame
• Harris corner detector
January 20, 2010
T0283 - Computer Vision
12
The geometric motion problem
Given image point correspondences, xi  xi/,
determine R and t
x/
x
C
January 20, 2010
C/
T0283 - Computer Vision
13
Outline of structure and motion computation
1. Compute the fundamental matrix F from point
correspondences xi  xi/
2. Compute the cameras (motion) from the
fundamental matrix (recall
).
Obtain
3. Compute the 3D structure Xi from the cameras P,
P/ and point correspondences xi  xi/
(triangulation)
January 20, 2010
T0283 - Computer Vision
14
Computing the
fundamental matrix
January 20, 2010
T0283 - Computer Vision
15
Problem statement
Given: n corresponding points
compute the fundamental matrix F such that
Solution
January 20, 2010
T0283 - Computer Vision
16
For n points
• For 8 points, A is an 8 x 9 matrix and f can be computed as the
null-vector of A, i.e. f is determined up to scale
January 20, 2010
T0283 - Computer Vision
17
Example: compute F from 8 point correspondences
Just consider first three points
January 20, 2010
T0283 - Computer Vision
18
January 20, 2010
T0283 - Computer Vision
19
The “8-point” algorithm – Least squares
solution
January 20, 2010
T0283 - Computer Vision
20
Automatic Computation of
the fundamental matrix
January 20, 2010
T0283 - Computer Vision
21
January 20, 2010
T0283 - Computer Vision
22
Step 1: interest points
Harris corner detector
100’s of points per image
January 20, 2010
T0283 - Computer Vision
23
Step 2a: match points – proximity
• proximity - search within disparity window
January 20, 2010
T0283 - Computer Vision
25
Step 2b: match points – cross correlate
• cross-correlate on intensity neighbourhoods
January 20, 2010
T0283 - Computer Vision
26
Correlation matching results
• Many wrong matches (10-50%), but enough to compute F
January 20, 2010
T0283 - Computer Vision
27
Robust line estimation - RANSAC
Fit a line to 2D data containing outliers
There are two problems
1. a line fit which minimizes perpendicular distance
2. a classification into inliers (valid points) and outliers
Solution: use robust statistical estimation algorithm RANSAC
(RANdom Sample Consensus) [Fishler & Bolles, 1981]
January 20, 2010
T0283 - Computer Vision
28
RANSAC robust line estimation
Repeat
1. Select random sample of 2 points
2. Compute the line through these points
3. Measure support (number of points within
threshold distance of the line)
Choose the line with the largest number of inliers
Compute least squares fit of line to inliers
(regression)
January 20, 2010
T0283 - Computer Vision
29
January 20, 2010
T0283 - Computer Vision
30
January 20, 2010
T0283 - Computer Vision
31
January 20, 2010
T0283 - Computer Vision
32
January 20, 2010
T0283 - Computer Vision
33
January 20, 2010
T0283 - Computer Vision
34
January 20, 2010
T0283 - Computer Vision
35
January 20, 2010
T0283 - Computer Vision
36
January 20, 2010
T0283 - Computer Vision
37
January 20, 2010
T0283 - Computer Vision
38
Algorithm summary – RANSAC robust F
estimation
Repeat
1. Select random sample of 7 correspondences
2. Compute F (1 or 3 solutions)
3. Measure support (number of inliers within
threshold distance of epipolar line)
Choose the F with the largest number of inliers
January 20, 2010
T0283 - Computer Vision
39
Correlation matching results
• Many wrong matches (10-50%), but enough to compute F
January 20, 2010
T0283 - Computer Vision
40
Correspondences consistent with epipolar
geometry
January 20, 2010
T0283 - Computer Vision
41
Computed epipolar geometry
January 20, 2010
T0283 - Computer Vision
42
Determining cameras
from the fundamental
matrix
January 20, 2010
T0283 - Computer Vision
43
Decomposing the fundamental matrix
Form the Essential matrix
January 20, 2010
T0283 - Computer Vision
44
The four camera solutions
The 3D point is only in front of both cameras in one case
January 20, 2010
T0283 - Computer Vision
45
Computing the rotation matrix from the Essential matrix
• Compute the SVD of
• Set
• Solutions are
January 20, 2010
T0283 - Computer Vision
46