Computer and Robot
Vision II
Chapter 12
Illumination
Presented by: 傅楸善 & 張庭瑄
0963 331 533
r95922102@ntu.edu.tw
指導教授: 傅楸善 博士
12.1 Introduction
two key questions in understanding 3D image formation


What determines where some point on object will
appear on image?
Answer: geometric perspective projection model
What determines how bright the image of some
surface on object will be?
Answer: radiometry, general illumination models,
diffuse and specular
DC & CV Lab.
CSIE NTU

refraction of light bouncing off a surface patch:
basic reflection phenomenon
12.1 Introduction
I : proportional to scene radiance

image intensity

scene radiance depends on
the amount of light that falls on a surface
the fraction of the incident light that is reflected
the geometry of light reflection,
i.e. viewing direction and illumination directions
DC & CV Lab.
CSIE NTU
12.1 Introduction

image intensity
I  gJ f r CS  b
i






J
i
: incident radiance
f r : bidirectional reflectance function
C : lens collection
S : sensor responsivity
g : sensor gain
b : sensor offset
DC & CV Lab.
CSIE NTU
DC & CV Lab.
CSIE NTU
Joke
DC & CV Lab.
CSIE NTU
12.2 Radiometry

is the measurement of the flow and transfer
of radiant energy in terms of both the power
emitted from or incident upon an area and the
power radiated within a small solid angle
about a given direction.

is the measurement of optical radiation.
DC & CV Lab.
CSIE NTU
12.2 Radiometry

irradiance:
the amount of light falling on a surface
power per unit area of radiant energy falling on a surface
measured in units of watts per square meter.

radiance:
the amount of light emitted from a surface
power per unit foreshortened area emitted into a unit solid angle
measured in units of watts per square meter per steradian

radiant intensity:
of a point illumination source power per steradian
measured in units of watts per steradian
may be a function of polar and azimuth angles
DC & CV Lab.
CSIE NTU
12.2 Radiometry

z-axis: along the normal to the surface element dA
at 0

polar angle: measured from the z-axis
(pointing north)

azimuth angle: measured from x-axis
(pointing east)
DC & CV Lab.
CSIE NTU
12.2 Radiometry

The solid angle subtended by a surface patch
is defined by the cone whose vertex is at the
point of radiation and whose axis is the line
segment going from the point of radiation to
the center of the surface patch.
DC & CV Lab.
CSIE NTU
12.2 Radiometry

size of solid angle: area intercepted by the cone on a
unit radius sphere centered at the point of radiation

solid angle: measured in steradians

total solid angle about a point in space:
4
DC & CV Lab.
CSIE NTU
steradians
DC & CV Lab.
CSIE NTU
12.2 Radiometry
cos  A

2
d

A : surface area

d : distance from surface area to point of radiation
( d  A )
2

 : angle the surface normal makes w.r.t. the cone
axis
DC & CV Lab.
CSIE NTU
12.2 Radiometry

2
surface irradiance ( w m ) :
I 0 A cos  0 d 2 I 0 cos  0

A
d2

A : area of surface patch

I 0 ( w sr ) : constant radiant intensity of point
illumination source
DC & CV Lab.
CSIE NTU

law of inverse squares:
irradiance varies inversely as square of
distance from the illuminated surface to source


infinitesimal slice on annulus on sphere of
radius r , polar angle  , azimuth angle 
slice subtends solid angle d ,
since cos 0  1, d  r , A  (r sin d ) * (rd )
d  sin dd
12.2.1
Bidirectional Reflectance Function

The bidirectional reflectance distribution function
f r is the fraction of incident light emitted in one
direction when the surface is illuminated from
another direction.

ratio of the scene radiance to the scene irradiance
DC & CV Lab.
CSIE NTU
12.2.1
Bidirectional Reflectance Function

differential reflectance model:
dJ (  e , e , i , i )  dJ (  i , i ) f r (  i , i , e , e )
r







i
 : polar angle between surface normal and lens center
 : azimuth angle of the sensor
e : emitting from
i : incident to
J ri: irradiance of the incident light at the illuminated surface
J : radiance of the reflected light
f r: ratio of the scene radiance to the scene irradiance
DC & CV Lab.
CSIE NTU
12.2.1
Bidirectional Reflectance Function

2
(
w
/
m
 sr ) in the
The differential emitted radiance
direction (  e , e ) due to the incident differential
irradiance in the direction (i , i ) is equal to the
incident differential irradiance dJ i (  i , i )( w / m 2 )
times the bidirectional reflectance distribution
function f r ( i , i , e , e )( 1 sr ) .
DC & CV Lab.
CSIE NTU
12.2.1
Bidirectional Reflectance Function

For many surfaces the dependence of f r on the
azimuth angles i and e is only a dependence on
their difference
f r ( i , i , e , e )  f r (i , e ;e  i )

except surfaces with oriented microstructure
e.g. mineral called tiger’s eye, iridescent feathers
of
some birds
DC & CV Lab.
CSIE NTU
12.2.1
Bidirectional Reflectance Function

An ideal Lambertian surface is one that
appears equally bright from all viewing
directions and reflects all incident light
absorbing none

Lambertian surface:
perfectly diffusing surface with
matte appearance
DC & CV Lab.
CSIE NTU
12.2.1
Bidirectional Reflectance Function

reflectivity r:
unitless fraction called reflectance factor

white writing paper: r = 0.68
white ceilings or yellow paper: r = 0.6
dark brown paper: r = 0.13


DC & CV Lab.
CSIE NTU

white blotting paper: r = 0.8

dark velvet: r = 0.004
DC & CV Lab.
CSIE NTU
12.2.1
Bidirectional Reflectance Function

bidirectional reflectance distribution function for
Lambertian surface
f r (  i , i ,  e , e ) 
DC & CV Lab.
CSIE NTU
r

I
rE   d 
A
2
E : irradiance,

: polar angel,
2  2

0 0
In
cos  sin dd  L
A
In
L  : radiance
A
 : azimuth angle,
d  sin dd
DC & CV Lab.
CSIE NTU
r : reflectivity
DC & CV Lab.
CSIE NTU
12.2.1
Bidirectional Reflectance Function

differential relationship for emitted radiance for
Lambertian surface
i
rdJ
dJ r (  e , e ) 
w m 2  sr


Lambertian surface: consistent brightness no matter
what viewing direction

power radiated into a fixed solid angle: same in any
direction
DC & CV Lab.
CSIE NTU
Example 12.1
DC & CV Lab.
CSIE NTU
Joke
DC & CV Lab.
CSIE NTU
12.2 Photometry






photometry: study of radiant light energy resulting in
physical sensation
brightness: attribute of sensation by which observer
aware of differences of observed radiant energy
radiometry  radiant energy
photometry  luminous energy
radiometry  power( radiant flux )
photometry  luminous flux
DC & CV Lab.
CSIE NTU
On Internet

Photometry: is the science of measuring
visible light in units that are weighted
according to the sensitivity of the human eye.
DC & CV Lab.
CSIE NTU
12.2 Photometry


lumen: unit of luminous flux
luminous intensity: ( w.r.t. radiance intensity )
luminous flux leaving point source per unit solid angle
has units of lumens per steradian


candela: one lumen per steradian
illuminance: ( w.r.t. irradiance )
luminous flux per unit area incident upon a surface
in units of lumens per square meter


one lux: one lumen per square meter
foot-candle: one lumen per square foot
DC & CV Lab.
CSIE NTU
12.2 Photometry

one foot =0.3048 meter
1
1 lux =
2 foot - candles = 10.76 foot - candles
(0.3048)

luminance: ( w.r.t. radiance )
luminous flux per unit solid angle per unit of projected area
in units of lumens per square meter per steradian
DC & CV Lab.
CSIE NTU
12.2.3 Torrance-Sparrow Model

J dr : diffuse reflection from Lambertian surface facets

J sr :
specular reflection from mirrorlike surface facets
r
dependent on the view point whereas J d is not

J r : reflected light from roughened surface

consider surfaces:
f r ( i , i , e , e )  f r (i , e ;e  i )
DC & CV Lab.
CSIE NTU

Torrance-Sparrow model:
J r ( i ; e ,  ;  )  sJ sr ( i ; e ,  ;  )  (1  s) J dr ( i ;  )

s (0  s  1) : proportion of specular reflection

depending on surface
s=0: diffuse Lambertian surface
s=1: specular surface



: wavelength of light
DC & CV Lab.
CSIE NTU




N : unit surface normal

L : unit positional vector of the light source
V : unit positional vector of the sensor
 
cos  i  N  L
 
cos  e  N  V
DC & CV Lab.
CSIE NTU
12.2.4 Lens Collection

lens collection: portion of reflected light coming
through lens to film

f : distance between the image plane and the lens
r1 : distance between the object and the lens
 r : distance between the lens and the image of the
2

object
 a : diameter of the lens
  : angle between the ray from the object patch to
the lens center
DC & CV Lab.
CSIE NTU
a2
a1
12.2.4 Lens Collection



irradiance incident on differential area da2 coming
from differential area da1, having radiance dJ i,
and passing through a lens having
2
aperture area A  a 4
A cos  : foreshortened area of aperture stop
seen by da1
r1  s1 cos  : distance from da1 to the aperture
DC & CV Lab.
CSIE NTU
12.2.4 Lens Collection

solid angle  subtended by aperture stop as seen
from da1:
A cos  A cos 3 


2
r1
s12

differential radiant power d passing through
aperture due to da1
d  dJ da1 cos 
i
DC & CV Lab.
CSIE NTU
12.2.4 Lens Collection

radiant power passing through aperture from da1
dJ A cos da1
d 
2
s1
irradiance incident to da2:
i

4
(radiant power reaching da2 is d )
i
4
d

dJ
A
cos
da1
r
dJ 

da2
s12 da2
DC & CV Lab.
CSIE NTU
12.2.4 Lens Collection


assume s1  s2 , then
magnification is s1 / s2
s2  f , thus lens
2
2
2
hence da1 / da2  ( s1 / s2 )  s1 / f , therefore
i
4
2
i
4
dJ
A
cos

da
s
dJ
A
cos

r
1 1
dJ 

2
2
s1 da2
f
f2
DC & CV Lab.
CSIE NTU
12.2.4 Lens Collection

since
A  a 2 / 4
dJ r 

dJ i cos 4  a 2
4
f
2
then the lens collection C is given by
dJ r  a 2
C  i  ( ) cos 4 
dJ
4 f
DC & CV Lab.
CSIE NTU
12.2.5 Image Intensity

The image intensity gray level I associated
with some small area of the image plane can
then be represented as the integral of all light
collected at the given pixel position coming
from the observed surface patch, modified by
sensor gain g and bias b
DC & CV Lab.
CSIE NTU
12.2.5 Image Intensity
I ( ,  )  g  CS ( ) J ( ,  ;  ) r d  b
r
2 2







 : light wavelength
S ( ) : sensor responsivity to light at wavelength 
J r ( ,  ;  ) : radiance of observed surface patch
( watt / m2  sr )
 : solid angle subtended by the viewing cone of camera for the pixel
r : distance to the observed patch
J r ( ,  ;  )(r 2 ) : power received for the pixel position
DC & CV Lab.
CSIE NTU
12.3 Photometric Stereo

In photometric stereo there is one camera but K light
sources having known intensities i1 ,..., iK and incident
vectors v1 ,..., vK to a given surface patch.

In photometric stereo the camera sees the surface
patch K times, one time when each light source is
activated and the remaining ones are deactivated.
DC & CV Lab.
CSIE NTU
12.3 Photometric Stereo

f1 ,..., f K : observed gray levels produced by the model
of Lambertian reflectance
f k  grik vk  n  b, k  1,..., K




n: surface normal vector of the surface patch having
Lambertian reflectance
r: reflectivity of the Lambertian surface reflectance
g: sensor gain
b: sensor offset
DC & CV Lab.
CSIE NTU
12.3 Photometric Stereo



if camera has been photometrically calibrated, g, b
known
fk  b
*
f

 rvk  n and
let k
gik
in matrix form
 f1* 
 v1' 
 
 
*
f    V   
 *
 ' 
 fK 
 vK 
f  rVn
*
DC & CV Lab.
CSIE NTU
12.3 Photometric Stereo

if surface normal n known then least-squares solution
for reflectivity r:
*'
f Vn
r
(Vn)' (Vn)
f  rVn
*

if K = 3 a solution for unit surface normal n:
V 1 f *
n  1 *
V f
f  rVn
*
DC & CV Lab.
CSIE NTU
V : K 3
f * : K 1
n : 3 1
12.3 Photometric Stereo

if K > 3, a least-squares solution:
(V 'V ) 1V ' f *
n
(V 'V ) 1V ' f *
DC & CV Lab.
CSIE NTU
12.3 Photometric Stereo

if g, b unknown camera must be calibrated as
follows:

geometric setup with known incident angle of
light source to surface normal surfaces of
known reflectivities illuminated by known
intensity light source
DC & CV Lab.
CSIE NTU
12.3 Photometric Stereo

ik : known intensity of light source for kth trial

vk : known incident direction of light source for kth trial
n : known unit length surface normal vector
 r : known reflectivity of surface illuminated for kth trial
k


y k : observed value from the camera
yk  grk ik vk  n  b  gxk  b, xk  rk ik vk  n, k  1,..., K
DC & CV Lab.
CSIE NTU
12.3 Photometric Stereo

let xk  ik rk vk  n then unknown gain g and offset b
satisfy
 x1 1 


 x2 1 
  


 x 1
 K 
 y1 
 
 g   y2 
  
 
b    
y 
 K
yk  gxk  b, xk  rk ik vk  n, k  1,..., K
DC & CV Lab.
CSIE NTU
12.3 Photometric Stereo

this leads to the least-squares solution for g, b

  xk2
 g   k 1
    K
b   x
 k
 k 1
K

xk 

k 1



K

K
DC & CV Lab.
CSIE NTU
1
 K

  xk y k 
 k 1

 K

  yk 
 k 1

Joke
DC & CV Lab.
CSIE NTU
12.4 Shape from Shading

nonplanar Lambertian surfaces of constant
reflectance factor: appear shaded

this shading: secondary clue to shape of the
observed surface

shape from shading: recovers shape of
Lambertian surface from image shading
DC & CV Lab.
CSIE NTU
12.4 Shape from Shading

a, b, c: unit vector of distant point light source
direction



assume surface viewed by distant camera so
perspective projection approximated by orthographic
projection
surface point position x, y, z  : projected to image
position x, y 
z  g x, y  : surface expression
DC & CV Lab.
CSIE NTU
12.4 Shape from Shading

unit vector normal to the surface at x, y  :
 g 
 
 x 
 g 
1
 
y 
g 2 g 2

( )  ( ) 1
 1 
x
y
 
 
DC & CV Lab.
CSIE NTU
12.4 Shape from Shading

gray level at x, y , within multiplicative constant
I ( x, y ) 
ap ( x, y )  bq( x, y )  c
p 2 ( x, y )  q 2 ( x, y )  1

Where p  g / x  and q  g / y 

R p, q  : reflectance map
R  p, q  
ap  bq  c
p2  q2 1
DC & CV Lab.
CSIE NTU
12.4 Shape from Shading


 : penalty constant
relaxation method: minimizing original error and a
smoothness term criterion function to be minimized by
choice of p, q
 
2
r
 I (r , c)  R p(r , c), q(r , c)
2
c
 p (r  1, c)  p (r , c)2   p (r , c  1)  p (r , c)2
 
 q (r  1, c)  q (r , c)2  q (r , c  1)  q (r , c) 2

DC & CV Lab.
CSIE NTU




Horn Robot Vision Fig
10.19

two orthographic shaded view of the same
surface caption
DC & CV Lab.
CSIE NTU
Horn Robot Vision Fig
10.18

a block diagram of Dent de Morcles region in
southwestern Switzerland
DC & CV Lab.
CSIE NTU
12.4 Shape from Shading

uniform brightness if planar surfaces since px, y  ,
qx, y  constant surfaces with curvature: surfaces with
px, y  , qx, y  provide information about surface
ap ( x, y )  bq( x, y )  c
height g x, y 
I ( x, y ) 
p 2 ( x, y )  q 2 ( x, y )  1

first-order Taylor expression for g:
g
g ( x  1, y )  g ( x, y ) 
 g ( x, y )  g ( x  1, y )  p( x, y )
x
g
g ( x, y  1)  g ( x, y ) 
 g ( x, y )  g ( x, y  1)  q( x, y )
y
DC & CV Lab.
CSIE NTU
12.4 Shape from Shading

with boundary conditions on g x, y , we can solve
unknown surface height and partial derivatives
px, y  , qx, y 
DC & CV Lab.
CSIE NTU
12.4.1 Shape from Focus



possible to recover shape from the shading
profile of object edges
basic idea: cameras do not have infinite
depth of field
The degree to which edges may be
defocused is related to how far the 3D edge
is away from the depths at which the edges
are sharply in focus.
DC & CV Lab.
CSIE NTU
Joke
DC & CV Lab.
CSIE NTU
12.5 Polarization

illumination source characterized by four factors
directionality: relative to surface normal in bidirectional
reflectance
intensity: energy coming out from source
spectral distribution: function of wavelength 
polarization: time-varying vibration of light energy in certain
direction
DC & CV Lab.
CSIE NTU
Examples
DC & CV Lab.
CSIE NTU
12.5 Polarization




polarization: time-varying vibration of the light energy
in certain direction
linearly polarized: changes direction by 180 every
period
circularly polarized: phase angle difference

of 90 ,thus cos wt  i sin wt
elliptically polarized phase angle difference of 90
and different amplitude a cos wt  ib sin wt
DC & CV Lab.
CSIE NTU
Mathematical Meaning of
Polarization

polarization of light mathematically described
by using wave theory
DC & CV Lab.
CSIE NTU
Linearly Polarized
DC & CV Lab.
CSIE NTU
Circularly Polarized
DC & CV Lab.
CSIE NTU
Usefulness of Polarization in
Machine Vision

At Brewster’s angle, the parallel polarized
light is totally transmitted and the
perpendicularly polarized light is partially
transmitted and partially reflected.
DC & CV Lab.
CSIE NTU
Usefulness of Polarization in
Machine Vision

This effect can be used to remove the
specular reflections from the window or metal
surfaces by looking through them at
Brewster’s angle.
DC & CV Lab.
CSIE NTU
DC & CV Lab.
CSIE NTU
http://www.tiffen.com/polarizer_pics.htm
No Filter
With Polarizer
DC & CV Lab.
CSIE NTU
With Warm Polarizer
12.5.1 Representation of Light
Using the Coherency Matrix

natural light: completely unpolarized

Coherency Matrix: Representation method of
polarization
DC & CV Lab.
CSIE NTU
12.5.2 Representation of Light
Intensity

The intensity of any light can be represented
as a sum of two intensities of two orthogonal
polarization components.

S-pol: component polarized perpendicularly
to the incidence plane

P-pol: component polarized parallel to the
incidence plane
DC & CV Lab.
CSIE NTU
12.6 Fresnel Equation
DC & CV Lab.
CSIE NTU
12.7 Reflection of Polarized Light

ergodic light: time average of the light
equivalent to its ensemble average
DC & CV Lab.
CSIE NTU
12.8 A New Bidirectional
Reflectance Function
DC & CV Lab.
CSIE NTU
12.9 Image Intensity

image intensity can be written in terms of
illumination parameters
sensor parameters
bidirectional reflectance function
DC & CV Lab.
CSIE NTU
12.10 Related Work

reflectance models: have been used in
computer graphics and image analysis
DC & CV Lab.
CSIE NTU
課程網站




http://140.112.31.93
Account: CV2
Password: DCCV
Ps.注意都是大寫
DC & CV Lab.
CSIE NTU
Project due Mar. 7


use correlation to do image matching
find dx, dy  to minimize
 | PIX (ima, x, y)  PIX (imb, x  dx, y  dy) |
( x , y )R
DC & CV Lab.
CSIE NTU
DC & CV Lab.
CSIE NTU
DC & CV Lab.
CSIE NTU
DC & CV Lab.
CSIE NTU
P.S. 1
f *  rVn  (Vn) ' f *  r (Vn) ' (Vn)
'
*
*'
(Vn) f
f Vn
 r

'
'
(Vn) (Vn) (Vn) (Vn)
V : K 3
*'
f Vn
r
(Vn)' (Vn)
f * : K 1
n : 3 1
DC & CV Lab.
CSIE NTU
P.S. 2
1
f  rVn  V f  rn
*
*
V 1 f *
 nr  1 *
V f
normalize
V : K 3
f * : K 1, K  3
n : 3 1
DC & CV Lab.
CSIE NTU
V 1 f *
 n  1 *
V f