Lighting affects appearance
How do we represent light? (1)
Ideal distant point source:
- No cast shadows
- Light distant
- Three parameters
- Example: lab with controlled
light
How do we represent light? (2)
Sky
Environment map: l(q,f)
- Light from all directions
- Diffuse or point sources
- Still distant
- Still no cast shadows.
- Example: outdoors (sky and sun)
`
Lambertian + Point Source


 l is direction of light
l  l l 
l is intensity of light

i  max( 0,  (l  nˆ )
i is radiance
lˆ
 is albedo
nˆ is surface normal
Surface
normal q
Light
Lambertian, point sources, no
shadows. (Shashua, Moses)
Whiteboard
Solution linear
Linear ambiguity in recovering scaled
normals
Lighting not known.
Recognition by linear combinations.
Linear basis for lighting
Z
X
Y
A brief Detour: Fourier Transform,
the other linear basis
Analytic geometry gives a coordinate
system for describing geometric objects.
Fourier transform gives a coordinate
system for functions.
Basis
P=(x,y) means P = x(1,0)+y(0,1)
Similarly:
f (q )  a cos(q )  a sin(q )
11
12
 a cos( 2q )  a sin( 2q )  
21
22
Note, I’m showing non-standard basis, these
are from basis using complex functions.
Example
c, a , a such that :
1
2
cos(q  c)  a cosq  a sin q
1
2
Orthonormal Basis
||(1,0)||=||(0,1)||=1
(1,0).(0,1)=0
Similarly we use normal basis elements eg:
cos(q )
cos(q )
While, eg:
cos(q ) 
2
 cos
0
 cosq sin q
2
0
2
q dq
dq  0
2D Example
Convolution
f ( x)  g  h   g ( x  x )h( x )dx
0
0
Imagine that we generate a point in f by
centering h over the corresponding point in
g, then multiplying g and h together, and
integrating.
0
Convolution Theorem
f  g  T F *G
1
• F,G are transform of f,g
That is, F contains coefficients, when
we write f as linear combinations of
harmonic basis.
Examples
cosq  cosq  ?
cosq  cos 2q  ?
cosq  f  ?
(cosq  .2 cos 2q  .1cos 3q )  f  ?
Low-pass filter removes low frequencies from
signal. Hi-pass filter removes high
frequencies. Examples?
Shadows
Attached Shadow
Cast Shadow
With Shadows: PCA
(Epstein, Hallinan and Yuille;
see also Hallinan; Belhumeur and Kriegman)
#1
#3
#5
#7
#9
Ball
48.2
94.4
97.9
99.1
99.5
Face
53.7
90.2
93.5
95.3
96.3
Phone Parrot
67.9
42.8
88.2
76.3
94.1
84.7
96.3
88.5
97.2
90.7
Dimension: 5  2D
Domain
Lambertian
Environment map
n
l
q
lmax (cosq, 0)
Reflectance
Lighting
Images
...
Lighting to Reflectance:
Intuition
1
0.5
0
0
1
2
3
1
2
3
2
1.5
r
1
0.5
0
0

 
(See D’Zmura, ‘91; Ramamoorthi and Hanrahan ‘00)
Spherical Harmonics
Orthonormal basis, hnm , for functions on the sphere.
n’th order harmonics have 2n+1 components.
Rotation = phase shift (same n, different m).
In space coordinates: polynomials of degree n.
S.H. used for BRDFs (Cabral et al.; Westin et al;).
(See also Koenderink and van Doorn.)
(2n  1) (n  m)!
hnm (q , f ) 
Pnm (cos q )eimf
4 (n  m)!
(1  z 2 ) m 2 d n  m 2
n
Pnm ( z ) 
(
z

1
)
2n n! dz n m
S.H. analog to convolution theorem
• Funk-Hecke theorem: “Convolution” in
function domain is multiplication in
spherical harmonic domain.
• k is low-pass filter.
k
Harmonic Transform of Kernel

k (q )  max(cosq , 0)   kn hn 0
n 0
 

 2
 

kn   3
n
1 ( n  2)! (2 n  1)

2
( 1)
n
n n
2 (  1)!(  1)!

2
2

0
n0
n 1
n  2, even
n  2, odd
Amplitudes of Kernel
1.5
1 0.886
An
0.591
0.5
0.222
0.037
0.007
0.014
0
0
1
2
3
4
n
5
6
7
8
Energy of Lambertian Kernel in
low order harmonics
Accumulated Energy
120
100
99.2
99.81
99.93
99.97
2
4
6
8
87.5
80
60
40
37.5
20
0
0
1
Reflectance Functions Near
Low-dimensional Linear Subspace

n
r  k l  
 (K

 (K
n  0 m  n
2
n
n 0 m   n
nm
nm
Lnm )hnm
Lnm )hnm
Yields 9D linear subspace.
How accurate is approximation?
Point light source

r  k l  
n
 (K
n 0 m   n
2
nm
Lnm )hnm  
 (K
n 0 m   n
L )hnm
nm nm
Amplitude of l = point source
Amplitude of k
3.5
1.2
3
1
2.5
2
0.8
0.6
1.5
0.4
1
0.2
0.5
0
-0.5
n
0
0
1
2
4
6
8
0
1
2
4
6
9D space captures 99.2% of energy
8
How accurate is approximation?
(2)
Worst case.
Amplitude of l = point source
Amplitude of k
1.2
3.5
3
2.5
2
1.5
1
0.5
0
-0.5
1
0.8
0.6
0.4
0.2
0
0
1
2
4
6
8
0
1
2
4
6
DC component as big as any other.
1st and 2nd harmonics of light could have zero energy
9D space captures 98% of energy
8
Forming Harmonic Images
bnm (p)=λrnm (X,Y,Z)

2λ(Z2 -X2 -Y2 ) λ(X2 -Y2 )
Z
X
Y
XY
XZ
YZ
Compare this to 3D Subspace
Z
X
Y
Accuracy of Approximation of
Images
Normals present to varying amounts.
Albedo makes some pixels more important.
Worst case approximation arbitrarily bad.
“Average” case approximation should be
good.
Models
Find Pose
Harmonic Images
Query
Compare
Matrix: B
Vector: I