General Imaging Model
Michael Grossberg and Shree Nayar
CAVE Lab, Columbia University
ICCV Conference
Vancouver, July 2001
Partially funded by NSF ITR Award, DARPA/ONR MURI
Imaging
Scene
Imaging System
Images
?
• What is a general imaging model ?
• How do we Compute its Parameters ?
Perspective Imaging Model
Camera Obscura
rays selected
rays become
image points
Systems that are not perspective
compound eyes
catadioptric
system
multiple camera
system
fisheye lens
General Imaging Model
• Essential components:
– Photosensitive elements
– optics
i
Pi
• Maps incoming pixels to rays
Raxel = Ray + Pixel
• Small perspective camera
– Simple lens
– One pixel photo-detector
Raxel
symbol
Index
Geometry
Position
Direction
Radiometry
Point Spread
Fall-off
• Most general model is a list of raxels
Response
Ray Surfaces
Position: (pX, pY, pZ)
Direction: (qq, qf)
virtual detectors
(raxels)
(pX, pY, pZ)
(qq, qf)
physical detectors
(pixels)
ray surface
imaging optics
Rays in 2D
• Singularity of rays called a caustic
perspective
non-perspective
position-direction
space
q
Y
X
position
space
caustic
Computing Caustics
• Change coordinates
– (x,y,d)
(X,Y,Z)
• Solve for d
¶pX
¶q
+d X
¶x
¶x
¶pY
¶q
det( J ) =
+d Y
¶x
¶x
¶pZ
¶q
+d Z
¶x
¶x
¶pX
¶q
+d X
¶y
¶y
¶pY
¶q
+d Y
¶y
¶y
¶pZ
¶q
+d Z
¶y
¶y
qX
qY =0
qZ
Caustic Ray Surface
• Caustic is a singularity or envelope of incoming rays
• Caustic represents loci of view-points
imaging optics
raxels
Caustic curve
Simple Examples
perspective
single viewpoint
multi-viewpoint
Raxel Radiometry
h(x)
• Linear fall-off of optical
elements
Normalized
Fall-off
Raxel index
g(e)
• Non-linear response of
photosensitive element
Normalized
Response
Normalized Exposure (e)
Point Spread
• Elliptical gaussian model of point spread.
– Major and minor deviation lengths, sa (d), sb (d)
– Angle of axis y (when sa (d), sb (d) are different)
Chief ray
y
sb
d, Scene depth
Image plane
sa
Impulse at Scene point
Finding the Parameters
• Known optical components: Compute
• Unknown optical components: Calibration Environment
?
Calibration Apparatus
• Structured light at two planes
– Geometry from binary patterns
– Radiometry from uniform patterns
pf
qf
z
pn
i
Finding the parameters:
Perspective System
video camera with perspective lens
laptop LCD
translating stage
sample image
Computed Raxel Model: Geometry
180
160
140
120
X in mm
100
80
60
180
160
Y in mm
140
120
100
80
260
280
300
320
340
360
Z in mm
Computed Raxel Model: Radiometry
• Radiometric response g(e)
• Pointwise fall-off h(x,y)
1.0
0.9
1.0
0.8
0.8
normalized
response
0.7
normalized
fall-off
0.6
0.4
0.6
0.5
0.4
0.3
0.2
0.2
0.0
0.1
0.0
0.0
0.1 0.2 0.3 0.4 0.5 0.6 0.7
normalized exposure
0.8 0.9 1.0
0
50
100
150
200
radius in pixels
250
300
Finding the parameters:
Non-single Viewpoint System
video camera with perspective lens
laptop LCD
translating stage
parabolic Mirror
sample image
Computed Raxel Model: Geometry
• Rotationally symmetric
10
5
0
-5
-10
mm from caustic max
-15
-20
-25
-30
-35
-60
60
-40
40
-20
mm from axis of symmetry
20
0
0
-20
20
-40
40
60
-60
mm from axis of symmetry
Computed Raxel Model: Radiometry
• Fall-off toward edge as resolution increases:
– less light collected
1.0
0.9
0.8
0.7
normalized
fall-off
0.6
0.5
0.4
0.3
0.2
90 110 130 150 170 190 210 230 250 270 290
radius in pixels
Summary
• Most general model simply list of raxels
Index
Geometry
Position
x, y
pX, pY, pZ
Radiometry
Direction
Point Spread
qq, qf
sa, sb, y
Fall-off
h
Response
g(e)
• Caustics summarize geometry
• Simple procedure for obtaining parameters from
a black box system