Uncalibrated Camera
Lecture 5
Uncalibrated Geometry
& Stratification
MASKS © 2004
Invitation to 3D vision
Uncalibrated Camera
calibrated
coordinates
Linear transformation
pixel
coordinates
MASKS © 2004
Invitation to 3D vision
Overview
•
Calibration with a rig
•
Uncalibrated epipolar geometry
•
Ambiguities in image formation
•
Stratified reconstruction
•
Autocalibration with partial scene knowledge
MASKS © 2004
Invitation to 3D vision
Uncalibrated Camera
Calibrated camera
• Image plane coordinates
• Camera extrinsic parameters
y
c
• Perspective projection
Uncalibrated camera
• Pixel coordinates
• Projection matrix
MASKS © 2004
Invitation to 3D vision
x
Taxonomy on Uncalibrated Reconstruction
•
is known, back to calibrated case
•
is unknown
 Calibration with complete scene knowledge (a rig) – estimate
 Uncalibrated reconstruction despite the lack of knowledge of
 Autocalibration (recover
from uncalibrated images)
•
Use partial knowledge
 Parallel lines, vanishing points, planar motion, constant intrinsic
•
Ambiguities, stratification (multiple views)
MASKS © 2004
Invitation to 3D vision
Calibration with a Rig
Use the fact that both 3-D and 2-D coordinates of feature
points on a pre-fabricated object (e.g., a cube) are known.
MASKS © 2004
Invitation to 3D vision
Calibration with a Rig
• Given 3-D coordinates on known object
• Eliminate unknown scales
• Recover projection matrix
• Factor the
into
• Solve for translation
MASKS © 2004
Invitation to 3D vision
and
using QR decomposition
Uncalibrated Epipolar Geometry
• Epipolar constraint
• Fundamental matrix
• Equivalent forms of
MASKS © 2004
Invitation to 3D vision
Properties of the Fundamental Matrix
• Epipolar lines
• Epipoles
Image
correspondences
MASKS © 2004
Invitation to 3D vision
Properties of the Fundamental Matrix
A nonzero matrix
is a fundamental matrix if
a singular value decomposition (SVD)
with
for some
.
There is little structure in the matrix
MASKS © 2004
Invitation to 3D vision
except that
has
Estimating Fundamental Matrix
• Find such F that the epipolar error is minimized
Pixel coordinates
• Fundamental matrix can be estimated up to scale
• Denote
• Rewrite
• Collect constraints from all points
MASKS © 2004
Invitation to 3D vision
Two view linear algorithm – 8-point algorithm
• Solve the LLSE problem:
• Solution eigenvector associated with
smallest eigenvalue of
• Compute SVD of F recovered from data
• Project onto the essential manifold:
•
cannot be unambiguously decomposed into pose
and calibration
MASKS © 2004
Invitation to 3D vision
Calibrated vs. Uncalibrated Space
MASKS © 2004
Invitation to 3D vision
Calibrated vs. Uncalibrated Space
Distances and angles are modified by S
MASKS © 2004
Invitation to 3D vision
Motion in the distorted space
Calibrated space
Uncalibrated space
• Uncalibrated coordinates are related by
• Conjugate of the Euclidean group
MASKS © 2004
Invitation to 3D vision
What Does F Tell Us?
•
can be inferred from point matches (eight-point algorithm)
•
Cannot extract motion, structure and calibration from one
fundamental matrix (two views)
•
allows reconstruction up to a projective transformation (as
we will see soon)
•
encodes all the geometric information among two views
when no additional information is available
MASKS © 2004
Invitation to 3D vision
Decomposing the Fundamental Matrix
• Decomposition of the fundamental matrix into a skew
symmetric matrix and a nonsingular matrix
• Decomposition of
is not unique
• Unknown parameters - ambiguity
• Corresponding projection matrix
MASKS © 2004
Invitation to 3D vision
Projective Reconstruction
•
From points, extract , followed by computation
of projection matrices
and structure
Canonical decomposition
•
Given projection matrices – recover structure
•
Projective ambiguity – non-singular 4x4 matrix
•
Both
and
are consistent with the epipolar geometry – give the
same fundamental matrix
MASKS © 2004
Invitation to 3D vision
Projective Reconstruction
• Given projection matrices recover projective structure
• This is a linear problem and can be solve using linear least-squares
• Projective reconstruction – projective camera matrices and
projective structure
MASKS ©Euclidean
2004
Structure
Invitation to 3D vision
Projective Structure
Euclidean vs Projective reconstruction
•
•
•
•
•
Euclidean reconstruction – true metric properties of objects
lenghts (distances), angles, parallelism are preserved
Unchanged under rigid body transformations
=> Euclidean Geometry – properties of rigid bodies under
rigid body transformations, similarity transformation
Projective reconstruction – lengths, angles, parallelism are NOT
preserved – we get distorted images of objects – their distorted 3D
counterparts --> 3D projective reconstruction
=> Projective Geometry
MASKS © 2004
Invitation to 3D vision
Homogeneous Coordinates (RBM)
3-D coordinates are related by:
Homogeneous coordinates:
Homogeneous coordinates are related by:
MASKS © 2004
Invitation to 3D vision
Homogenous and Projective Coordinates
•
•
Homogenous coordinates in 3D before – attach 1 as the last
coordinate – render the transformation as linear transformation
Projective coordinates – all points are equivalent up to a scale
2D- projective plane
3D- projective space
• Each point on the plane is represented by a ray in projective space
• Ideal points – last coordinate is 0 – ray parallel to the image plane
• points at infinity – never intersects the image plane
MASKS © 2004
Invitation to 3D vision
Vanishing points – points at infinity
Representation of a 3-D line – in homogeneous coordinates
When  -> 1 - vanishing points – last coordinate -> 0
Similarly in the image plane
MASKS © 2004
Invitation to 3D vision
Ambiguities in the image formation
•
•
Potential Ambiguities
Ambiguity in K (K can be recovered uniquely – Cholesky or QR)
•
Structure of the motion parameters
•
Just an arbitrary choice of reference frame
MASKS © 2004
Invitation to 3D vision
Ambiguities in Image Formation
Structure of the (uncalibrated) projection matrix
•
For any invertible 4 x 4 matrix
•
In the uncalibrated case we cannot distinguish between camera
imaging word
from camera
imaging distorted world
•
In general,
•
In order to preserve the choice of the first reference frame we can
restrict some DOF of
MASKS © 2004
is of the following form
Invitation to 3D vision
Structure of the Projective Ambiguity
• 1st frame as reference
• Choose the projective reference frame
then ambiguity is
•
MASKS © 2004
can be further decomposed as
Invitation to 3D vision
Stratified (Euclidean) Reconstruction
•
General ambiguity – while preserving choice of first reference
frame
•
Decomposing the ambiguity into affine and projective one
•
Note the different effect of the 4-th homogeneous coordinate
MASKS © 2004
Invitation to 3D vision
Affine upgrade
•
Upgrade projective structure to an affine structure
•
Exploit partial scene knowledge

Vanishing points, no skew, known principal point
Special motions

Pure rotation, pure translation, planar motion, rectilinear
motion
Constant camera parameters (multi-view)
•
•
MASKS © 2004
Invitation to 3D vision
Affine upgrade using vanishing points
How to compute
Maps the points
To points with affine coordinates
Vanishing points – last homogeneous affine coordinate is 0
MASKS © 2004
Invitation to 3D vision
Affine Upgrade
Need at least three vanishing points
3 equations, 4 unknowns (-1 scale)
Solve for
Set up
structure
MASKS © 2004
and update the projective
Invitation to 3D vision
Euclidean upgrade
•
We need to estimate remaining affine ambiguity
Alternatives:
•
In the case of special motions (e.g. pure rotation) – no
projective ambiguity – cannot do projective reconstruction
•
Estimate
directly (special case of rotating camera –
follows)
Multi-view case – estimate projective and affine ambiguity
together
Use additional constraints of the scene structure (next)
Autocalibration (Kruppa equations)
•
•
•
MASKS © 2004
Invitation to 3D vision
Euclidean Upgrade using vanishing points
•
Vanishing points – intersections of the parallel lines
•
Vanishing points of three orthogonal directions
•
Orthogonal directions – inner product is zero
•
Provide directly constraints on matrix
•
S – has 5 degrees of freedom, 3 vanishing points – 3
constraints (need additional assumption about K)
Assume zero skew and aspect ratio = 1
•
MASKS © 2004
Invitation to 3D vision
Geometric Stratification
MASKS © 2004
Invitation to 3D vision
Special Motions – Pure Rotation
•
Calibrated Two views related by rotation only
•
Mapping to a reference view – rotation can be estimated
•
Mapping to a cylindrical surface
MASKS © 2004
Invitation to 3D vision
Pure Rotation - Uncalibrated Case
•
Calibrated Two views related by rotation only
•
Conjugate rotation
•
Given C, how much does it tell us about K (or S) ?
•
Given three rotations around linearly independent
axes – S, K can be estimated using linear techniques
Applications – image mosaics
•
MASKS © 2004
Invitation to 3D vision
can be estimated
Overview of the methods
MASKS © 2004
Invitation to 3D vision
Summary
MASKS © 2004
Invitation to 3D vision
Direct Autocalibration Methods
The fundamental matrix
satisfies the Kruppa’s equations
If the fundamental matrix is known up to scale
Under special motions, Kruppa’s equations become linear.
Solution to Kruppa’s equations is sensitive to noises.
MASKS © 2004
Invitation to 3D vision
Direct Stratification from Multiple Views
From the recovered projective projection matrix
we obtain the absolute quadric contraints
Partial knowledge in
(e.g. zero skew, square pixel) renders
the above constraints linear and easier to solve.
The projection matrices can be recovered from the multiple-view
rank method to be introduced later.
MASKS © 2004
Invitation to 3D vision
Summary of (Auto)calibration Methods
MASKS © 2004
Invitation to 3D vision