Cameras, Resectioning,
and Triangulation
CS 4495/7495
Computer Vision
Frank Dellaert, Fall 07
Photographs are Projections
Vertical vanishing
point
(at infinity)
Vanishing
line
Vanishing
point
Slide Courtesy Antonio Criminisi
Vanishing
point
3D to 2D: Cameras
• First photograph due to Niepce
• Basic abstraction is the pinhole camera
– lenses required to ensure image is not too
dark
– various other abstractions can be applied
Pinhole cameras
• Abstract camera
model - box with
a small hole in it
• Pinhole cameras
work in practice
Pinhole Camera
• The better wayx toydo cameras:
(x, y, z) " ( f , f )
P
z z
C
f
O
j
p
q
i
Q
k
Perspective Camera Model
Linear transformation of homogeneous (projective) coordinates
&X #
&u #
&1 0 0 0# $ !
Y!
$
!
$
!
$
p = $ v ! = [ I 0]P = $0 1 0 0!
$Z !
$% w!"
$%0 0 1 0!" $ !
%T "
Recover image (Euclidean) coordinates by normalizing :
u
X
=
w
Z
v
Y
vˆ =
=
w
Z
uˆ =
Parallel lines meet
We can see infinity !
Railroad: parallel lines
P=(0,0,1,0)T
p=MP=(0,0,1)T
Normalized Image
coordinates
1
O
u=X/Z = dimensionless !
P
Pixel units
Pixels are on a grid of a certain dimension
f
O
u=k f X/Z = in pixels !
P
[f] = m (in meters)
[k] = pixels/m
Pixel coordinates
We put the pixel coordinate origin on topleft
f
O
u=u0 + k f X/Z
P
Pixel coordinates in 2D
(0.5,0.5)
640
"
X
Y%
$ u0 + kf ,v 0 + lf '
#
Z
Z&
480
(u0,v0)
i
!
(640.5,480.5)
j
Important: MATLAB
Convention
(1,1) !
Just as good as any other convention !
Summary: Intrinsic
Calibration
3 " 3 Calibration Matrix K
#X &
# u&
#) s u0 &#1 0 0 0&% (
% (
%
(%
(%Y (
p = % v ( = K[I 0]P = %
* v 0 (%0 1 0 0(
%Z (
%$w('
%$
1 ('%$0 0 1 0('% (
$T '
Recover image (Euclidean) coordinates by normalizing :
u
)X + sY + u0
ˆu =
=
w
Z
skew
v
*Y + v 0
vˆ =
=
w
Z
5 Degrees of Freedom !
Camera Pose
In order to apply the camera model, objects in the scene
must be expressed in camera coordinates.
y
c
w
T
!
x
!
!
z
!
z
World
!
Coordinates
x
y
Camera
Coordinates
!
!
Calibration
target looks tilted from camera
viewpoint. This can be explained as a
difference in coordinate systems.
3D-2D Projective mapping
Projection
Matrix
(3x4)
Projective Camera Matrix
Camera = Calibration " Pr ojection " Extrinsics
# u&
#) s
% (
%
p = % v ( = K[I 0]TP = %
*
%$w('
%$
#X &
u0 &#1 0 0 0&
% (
#
&
(%
( R t %Y (
v 0 (%0 1 0 0(%
(
0
1
$
'% Z (
1 ('%$0 0 1 0('
% (
$T '
= K [ R t ] P = MP
5+6 DOF = 11 !
!
Projective Camera Matrix
p = K [R t ]P = MP
& u # & m11 m12
$ v ! = $m
$ ! $ 21 m22
$% w!" $% m31 m32
m13
m23
m33
&X #
m14 # $ !
Y
!
m24 ! $ !
$Z !
m34 !" $ !
%T "
5+6 DOF = 11 !
Columns & Rows of M
& m1 #
M = $$m2 !! = m1
$% m3 !"
[
m2
m3
m4
]
m1 Pi
ui =
m3 Pi
m2 Pi
vi =
m3 Pi
m2P=0
O
Arbitrary 2-View
Triangulation
P?
 Given p and p’ find P
p’
C’
p
C
Linear Triangulation
Method
• Take Projection equations
• Apply cross-product trick:
$ [ p] M '
$0'
# p $ MP
# p $ MP = 0
#
%"
"&
)P = & )
"
%0(
%[ p']# M'(
! p' $ M ' P
! p '$M ' P = 0
 SVD
!
 Smallest Eigenvector !
 Generalizes to N points
Resectioning = Finding a
Camera Given Known points
• SVD: 6-point algorithm
• Apply cross-product trick
• Take Notes