10/04/11
Pinhole Camera Model
Computational Photography
Derek Hoiem, University of Illinois
Take photos for project 4
• >= 1024x1024, level, centered on face, blank
background
• Annotate your faces by 10/13, details to come
Reminders
• Project 3 due Monday
• I’m out of town Wed to Fri
• Amin Sadeghi is teaching Thurs: important
and interesting stuff
Next classes: Single-view Geometry
How tall is this woman?
How high is the camera?
What is the camera
rotation?
What is the focal length of
the camera?
Which ball is closer?
Today’s class
Mapping between image and world
coordinates
– Pinhole camera model
– Projective geometry
• Vanishing points and lines
– Projection matrix
Image formation
Let’s design a camera
– Idea 1: put a piece of film in front of an object
– Do we get a reasonable image?
Slide source: Seitz
Pinhole camera
Idea 2: add a barrier to block off most of the rays
– This reduces blurring
– The opening known as the aperture
Slide source: Seitz
Pinhole camera
f
c
f = focal length
c = center of the camera
Figure from Forsyth
Camera obscura: the pre-camera
• First idea: Mo-Ti, China (470BC to 390BC)
• First built: Alhacen, Iraq/Egypt (965 to 1039AD)
Illustration of Camera Obscura
Freestanding camera obscura at UNC Chapel Hill
Photo by Seth Ilys
Camera Obscura used for Tracing
Lens Based Camera Obscura, 1568
First Photograph
Oldest surviving photograph
– Took 8 hours on pewter plate
Joseph Niepce, 1826
Photograph of the first photograph
Stored at UT Austin
Niepce later teamed up with Daguerre, who eventually created Daguerrotypes
Dimensionality Reduction Machine (3D to 2D)
3D world
2D image
Point of observation
Figures © Stephen E. Palmer, 2002
Projection can be tricky…
Slide source: Seitz
Projection can be tricky…
Slide source: Seitz
Projective Geometry
What is lost?
• Length
Who is taller?
Which is closer?
Length is not preserved
A’
C’
B’
Figure by David Forsyth
Projective Geometry
What is lost?
• Length
• Angles
Parallel?
Perpendicular?
Projective Geometry
What is preserved?
• Straight lines are still straight
Vanishing points and lines
Parallel lines in the world intersect in the image at a
“vanishing point”
Vanishing points and lines
Vanishing Point
Vanishing Line
o
Vanishing Point
o
Vanishing points and lines
Vertical vanishing
point
(at infinity)
Vanishing
line
Vanishing
point
Credit: Criminisi
Vanishing
point
Vanishing points and lines
Photo from online Tate collection
Vanishing objects
Projection: world coordinatesimage coordinates
.
Optical
Center
(u0, v0)
.
.
v
u
u 
p 
v 
f
.
Camera
Center
(tx, ty, tz)
Z
Y
X 
P  Y 
 
 Z 
Homogeneous coordinates
Conversion
Converting to homogeneous coordinates
homogeneous image
coordinates
homogeneous scene
coordinates
Converting from homogeneous coordinates
Homogeneous coordinates
Invariant to scaling
 x   kx 
kx
x







kw
w
k  y    ky    ky    y 
 w kw  kw   w 
Homogeneous
Coordinates
Cartesian
Coordinates
Point in Cartesian is ray in Homogeneous
Basic geometry in homogeneous coordinates
ai 
• Line equation: ax + by + c = 0
line i   bi 
 ci 
• Append 1 to pixel coordinate to get
ui 
homogeneous coordinate
pi   vi 
 1 
• Line given by cross product of two points
line ij  pi  p j
• Intersection of two lines given by cross
product of the lines
qij  linei  line j
Another problem solved by homogeneous
coordinates
Intersection of parallel lines
Cartesian: (Inf, Inf)
Homogeneous: (1, 1, 0)
Cartesian: (Inf, Inf)
Homogeneous: (1, 2, 0)
Slide Credit: Saverese
Projection matrix
R,T
jw
kw
Ow
iw
x  KR t  X
x: Image Coordinates: w(u,v,1)
K: Intrinsic Matrix (3x3)
R: Rotation (3x3)
t: Translation (3x1)
X: World Coordinates: (X,Y,Z,1)
Interlude: when have I used this stuff?
When have I used this stuff?
Object Recognition (CVPR 2006)
When have I used this stuff?
Single-view reconstruction (SIGGRAPH 2005)
When have I used this stuff?
Getting spatial layout in indoor scenes (ICCV
2009)
When have I used this stuff?
Inserting photographed objects into images
(SIGGRAPH 2007)
Original
Created
When have I used this stuff?
Inserting synthetic objects into images: http://vimeo.com/28962540
Projection matrix
Intrinsic Assumptions Extrinsic Assumptions
• No rotation
• Unit aspect ratio
• Optical center at (0,0)
• No skew
x  KI 0 X
Slide Credit: Saverese
• Camera at (0,0,0)
K
u   f
wv    0
  
1   0
0
f
0
 x
0 0  
y


0 0 
 z 
1 0  
1 
Remove assumption: known optical center
Intrinsic Assumptions Extrinsic Assumptions
• No rotation
• Unit aspect ratio
• No skew
x  KI 0 X
• Camera at (0,0,0)
u   f
wv    0
  
1   0
0
u0
f
v0
0
1
x
0  
y

0  
 z 
0  
1 
Remove assumption: square pixels
Intrinsic Assumptions Extrinsic Assumptions
• No skew
x  KI 0 X
• No rotation
• Camera at (0,0,0)
u  
wv    0
  
1   0
0
u0

v0
0
1
 x
0  
y

0  
 z 
0  
1 
Remove assumption: non-skewed pixels
Intrinsic Assumptions Extrinsic Assumptions
• No rotation
• Camera at (0,0,0)
x  KI 0 X
u  
wv    0
  
1   0
s
u0

v0
0
1
Note: different books use different notation for parameters
 x
0  
y

0  
 z 
0  
1 
Oriented and Translated Camera
R
jw
t
kw
Ow
iw
Allow camera translation
Intrinsic Assumptions Extrinsic Assumptions
• No rotation
x  KI t X
u  
wv    0
  
1   0
0

0
 x
u0  1 0 0 t x   
y



v0 0 1 0 t y  
 z 

1  0 0 1 t z   
1 
Slide Credit: Saverese
3D Rotation of Points
Rotation around the coordinate axes, counter-clockwise:
0
1
Rx ( )  0 cos
0 sin 
p’

y
z
p

 sin  
cos 
 cos  0 sin  
R y (  )   0
1
0 
 sin  0 cos  
cos   sin  0
Rz ( )   sin  cos  0
 0
0
1
0
Allow camera rotation
x  KR t  X
u  
wv    0
  
1   0
s

0
u0   r11 r12
v0  r21 r22

1  r31 r32
r13
r23
r33
 x
tx   
y

ty  
 z 
t z   
1 
Degrees of freedom
x  KR t  X
5
u  
wv    0
  
1   0
6
s

0
u0   r11 r12
v0  r21 r22

1  r31 r32
r13
r23
r33
 x
tx   
y

ty  
 z 
t z   
1 
Vanishing Point = Projection from Infinity
 x
 xR 
 x
 y






p  KR t 
 p  KR y  p  K y R
 
 
z
 z R 
 z 
 
0 
u   f
wv    0
  
1   0
0
f
0
u0   xR 
v0   y R  
 
1   z R 
fxR
u
 u0
zR
fyR
v
 v0
zR
Orthographic Projection
• Special case of perspective projection
– Distance from the COP to the image plane is infinite
Image
World
– Also called “parallel projection”  u  1 0 0 0  x 
 y
– What’s the projection matrix?  v   0 1 0 0  
 
 w

 z 
0 0 0 1  
1 
Slide by Steve Seitz
Scaled Orthographic Projection
• Special case of perspective projection
– Object dimensions are small compared to distance to
camera
u   f
– Also called “weak perspective”   
v  0
– What’s the projection matrix?   
w  0
Illustration from George Bebis
0
f
0
 x
0 0  
y


0 0 
 z 
0 s   
1 
Slide by Steve Seitz
Take-home question
Suppose we have two 3D cubes on the ground
facing the viewer, one near, one far.
1. What would they look like in perspective?
2. What would they look like in weak perspective?
Photo credit: GazetteLive.co.uk
Beyond Pinholes: Radial Distortion
Corrected Barrel Distortion
Image from Martin Habbecke
Things to remember
Vanishing
line
• Vanishing points and
vanishing lines
• Pinhole camera model
and camera projection
matrix
Vanishing
point
Vertical vanishing
point
(at infinity)
Vanishing
point
x  KR t  X
Next two classes
• Thurs
– Fun with faces, with Amin Sadeghi
• Tues
– Single-view geometry: recovering size in the world
– Tricks with focus and aperture, Vertigo effect
Questions