Single-view metrology
Magritte, Personal Values, 1952
Many slides from S. Seitz, D. Hoiem
Review: Camera projection matrix
camera
coordinate
system
 x  PX
world
coordinate
system
 x   x
  
 y  

1 
  
y
 x   r11
r12
r13
y
r22
r23
r32
r33

r21

1   r31
intrinsic
parameters
extrinsic
parameters
X
t1  
 Y
t2 
 Z
t 3  
 1







P  K R
Camera parameters
t
• Intrinsic parameters
•
•
•
•
•
Principal point coordinates
Focal length
Pixel magnification factors
Skew (non-rectangular pixels)
Radial distortion
m x

K 


my
 f


1  
f
p x   x
 
py 
 
1  
x
y


1 
y
Camera parameters
P  K R
• Intrinsic parameters
•
•
•
•
•
Principal point coordinates
Focal length
Pixel magnification factors
Skew (non-rectangular pixels)
Radial distortion
• Extrinsic parameters
• Rotation and translation relative to world coordinate system
t
Camera calibration
 x  K R
 x  *
  
 y  *
  
 1  *
t X
*
*
*
*
*
*
*

*

* 
X

Y

Z

1






Source: D. Hoiem
Camera calibration
• Given n points with known 3D coordinates Xi
and known image projections xi, estimate the
camera parameters
Xi
xi
P?
Camera calibration: Linear method
 x i  PX i
x i  PX i  0
 0

T
X
i

 y X T
i
i

 Xi
T
0
T
xi X i
T
 x i   P1 X i 

   T
y  P X 0
 i  2 i
 1   P3T X i 


T
y i X i   P1 
 
T 
 x i X i   P2   0
0   P3 
Two linearly independent equations
Camera calibration: Linear method
 0T
 T
X 1

 T
0
X T
 n
X
T
1
0
T

T
Xn
0
T
 y1 X 
T 
 x1 X 1   P1 
 
   P2   0

T 

 y n X n   P3 
T
 x n X n 
T
1
Ap  0
• P has 11 degrees of freedom
• One 2D/3D correspondence gives us two linearly
independent equations
• Homogeneous least squares: find p minimizing ||Ap||2
• Solution given by eigenvector of ATA with smallest eigenvalue
• 6 correspondences needed for a minimal solution
Camera calibration
• What if world coordinates of reference 3D
points are not known?
• We can use scene features such as vanishing
points
Vertical vanishing
point
(at infinity)
Vanishing
line
Vanishing
point
Vanishing
point
Slide from Efros, Photo from Criminisi
Recall: Vanishing points
image plane
vanishing point v
camera
center
line in the scene
• All lines having the same direction share the same
vanishing point
Computing vanishing points
v
X0
 X1

X2

Xt 
X3


•
•
•
 tD 1 

 tD 2

 tD 3 

1

 X1

X2


X3


Xt
/ t  D1 

/ t  D2

/ t  D3 

1/ t

X
 D1 
 
D2



 D3 
 
 0 
X∞ is a point at infinity, v is its projection: v = PX∞
The vanishing point depends only on line direction
All lines having direction D intersect at X∞
Calibration from vanishing points
• Let us align the world coordinate system with three
orthogonal vanishing directions in the scene:
v2
v1
.v
3
Calibration from vanishing points
*

P  *

 *
•
•
•
•
*
*
*
*
*
*
*

*  p 1

* 
p2
p3
p4
p1 = P(1,0,0,0)T – the vanishing point in the x direction
Similarly, p2 and p3 are the vanishing points in the y
and z directions
p4 = P(0,0,0,1)T – projection of the origin of the world
coordinate system
Problem: we can only know the four columns up to
independent scale factors, additional constraints
needed to solve for them
Calibration from vanishing points
• Let us align the world coordinate system with three
orthogonal vanishing directions in the scene:
1 
 
e1  0 ,
 
 0 
0 
 
e2  1 ,
 
 0 
ei  R K
T
T
vi K
T
T
1
0 
 
e3  0
 
 1 
vi,
ei e j  0
1
T
RR K
e i 
 v i  K R | t     KRe
0
i
T
v j  vi K
T
K
1
vj 0
• Each pair of vanishing points gives us a constraint
on the focal length and principal point
Calibration from vanishing points
Cannot recover focal
length, principal point is
the third vanishing point
Can solve for focal length, principal point
Rotation from vanishing points
e i 
 v i  K R | t     KRe
0
1
 K v 1  Re 1  [ r1
1
r2
i
1 
 
r3 ] 0  r1
 
 0 
Thus,  K v i  ri .
Get λ by using the constraint ||ri||2=1.
Calibration from vanishing points: Summary
•
•
•
Solve for K (focal length, principal point) using three
orthogonal vanishing points
Get rotation directly from vanishing points once
calibration matrix is known
Advantages
•
•
•
No need for calibration chart, 2D-3D correspondences
Could be completely automatic
Disadvantages
•
•
•
Only applies to certain kinds of scenes
Inaccuracies in computation of vanishing points
Problems due to infinite vanishing points
Making measurements from a single image
http://en.wikipedia.org/wiki/Ames_room
Recall: Measuring height
5
5.3
4
Camera height
3.3
3
2.8
2
1
Measuring height without a ruler
O
Z
ground plane
Compute Z from image measurements
• Need more than vanishing points to do this
The cross-ratio
• A projective invariant: quantity that does not change
under projective transformations (including perspective
projection)
The cross-ratio
• A projective invariant: quantity that does not change
under projective transformations (including perspective
projection)
• What are invariants for other types of transformations
(similarity, affine)?
• The cross-ratio of four points:
P3
P2
P1
P4
P 3  P1
P4  P2
P3  P 2
P 4  P1
Measuring height

TB
R
R B
T

H
R
scene cross ratio
T
(top of object)
t  b vZ  r
t
r
C
vZ
b
R
H
(reference point)
R
B
ground plane
(bottom of object)
rb
vZ  t
image cross ratio

H
R
Measuring height without a ruler
vz
r
vanishing line (horizon)
t0
vx
t
vy
v
H
R
b0
t  b vZ  r
H

r  b vZ  t
R
image cross ratio
b
H
2D lines in homogeneous coordinates
• Line equation: ax + by + c = 0
l x  0 where
T
a 
 
l b ,
 
 c 
x
 
x y
 
 1 
• Line passing through two points: l  x 1  x 2
• Intersection of two lines: x  l 1  l 2
•
What is the intersection of two parallel lines?
vz
r
vanishing line (horizon)
t0
vx
t
vy
v
H
R
b0
t  b vZ  r
H

r  b vZ  t
R
image cross ratio
b
H
Measurements on planes
4
3
2
1
1
2
Approach: unwarp then measure
What kind of warp is this?
3
4
Image rectification
p′
p
To unwarp (rectify) an image
• solve for homography H given p and p′
– how many points are necessary to solve for H?
Image rectification: example
Piero della Francesca, Flagellation, ca. 1455
Application: 3D modeling from a single image
J. Vermeer, Music Lesson, 1662
A. Criminisi, M. Kemp, and A. Zisserman, Bringing Pictorial
Space to Life: computer techniques for the analysis of paintings,
Proc. Computers and the History of Art, 2002
http://research.microsoft.com/en-us/um/people/antcrim/ACriminisi_3D_Museum.wmv
Application: 3D modeling from a single image
D. Hoiem, A.A. Efros, and M. Hebert, "Automatic Photo Pop-up", SIGGRAPH 2005.
http://www.cs.illinois.edu/homes/dhoiem/projects/popup/popup_movie_450_250.mp4
Application: Image editing
Inserting synthetic objects into images:
http://vimeo.com/28962540
K. Karsch and V. Hedau and D. Forsyth and D. Hoiem, “Rendering Synthetic Objects into
Legacy Photographs,” SIGGRAPH Asia 2011
Application: Object recognition
D. Hoiem, A.A. Efros, and M. Hebert, "Putting Objects in Perspective", CVPR 2006