Photometric Image Formation
CSE 559: Computer Vision
Guest Lecturer: Austin Abrams
Images/Demo from Steve Seitz, Wikipedia
How are images made?
• One half: geometric vision
– “how the pixel projected onto the image”
• Today: photometric vision (aka radiometric)
– “how the pixel got its color”
Vision and Graphics
Computer Graphics
Properties
of a scene
Image
Vision
Image Formation Approach
• Come up with a model for how the scene was
created
• Given images, find the most likely properties
that fit that model
Diffuse Surfaces
Brightness of a pixel depends on:
• object color
• lighting direction
• surface normal
But NOT view direction!
Lambertian Cosine Law
• The intensity of an observed diffuse object is
proportional to the cosine of the angle
between the normal and lighting direction
L
θ
N
I = ρ cos θ
= ρ |L||N| cos θ
=ρL
N
=
LN

= L
N
x
=
I
=
ρ
L N
Recovering Albedo and Normals
• Can you decompose a single image into its
albedo and normal images?
x
=
x
x
Photometric Stereo
• Given multiple images taken with varying
illumination, recover albedo and normals.
– take pictures in dark room with varying
illumination.
– estimate lighting directions L.
– recover albedo and normals.
Side note 1:
How to get the lighting direction?
• Put a shiny sphere in the scene
• Sphere’s geometry (normals) are known
• Find specular highlight
Side-note 2:
Why “Stereo”?
Surface normals provide constraints on depth differences
Photometric Stereo
• If L is known, and albedo is grayscale this is a
linear problem.
I = ρ(L  N)
= ρ (Lx Nx + Ly Ny + Lz Nz )
= Lx Nxρ + Ly Nyρ + Lz Nzρ
= Lx a + Ly b + Lz c
I = ρ(L  N)
= L x a + Ly b + Lz c
For each pixel:
I1
I2
I3
…
In
=
Lx1 Ly1 Lz1
Lx2 Ly2 Lz2
Lx3 Ly3 Lz3
…
Lxn Lyn Lzn
Then:
ρ = sqrt(a2 + b2 + c2)
N = (a,b,c) / ρ
a
b
c
Demo
When does this model fail?
I ≠ ρ (L
N)
Attached shadows
L
N=0
L
L
N<0
I = ρ max(L
N, 0)
N>0
Cast Shadows, Ambient Light
I = ρ (S L
N + a)
S = 0 or 1
Radiometric Camera Calibration
• Pixel intensities are usually not proportional to
the energy that hit the CCD
RAW image
Published image
Radiometric Camera Calibration
Published
f
RAW
Radiometric Camera Calibration
Observed = f(RAW)
(Grossberg and Nayar)
f -1 (Observed) = RAW
Radiometric Camera Calibration
• How do you model f -1?
f -1(x) = xγ
f -1(x) = c0 + c1x + c2x2 + c3x3 + …
f -1(x) = f0(x) + f1(x) c1 + f2(x)c2 + …
mean camera curve
basis camera curves
Radiometric Camera Calibration
I = f (ρ (S L
N + a))
Adding exposure:
I = f (e ρ (S L
N + a))
Heliometric Stereo
Given lots of images from a stable webcam,
use lighting from the sun to recover:
I = f (e ρ (S L
N + a))
Heliometric Stereo
Heliometric Stereo
Heliometric Stereo