Presentation Slide
Scene illumination and surface albedo recovery via L1-norm total variation minimization
Hong-Ming Chen hc2599@columbia.edu
Advised by: John Wright
Decomposition of a scene
= .* scene illumination Reflectance
(albedo)
.* :
Matlab element multiplication operation
2
Image Formation
Pixel i
.*
=
E
R
signals
E
G
E
B
integration illumination reflectance scene intensity response Sensor response
S
G
S
B
ˆ
Light source power spectrum
E
R
E
B
E
G
q
ˆ
R q
ˆ
G
q
ˆ
B
S
R
ˆ
Object reflectance
N
i
S
R
ˆ
N
i
S
G
d d
Sensor response (camera or eyes)
q
ˆ k
: shutter speed, aperture size, quantization factor
ˆ
N
i
S
B
d
etc
3
It is VERY HARD to directly model / simulate / solve this problem!
4
Narrowing down our target problem
•
Simplification:
– mean wavelength response (impulse response)
•
Assumption
(on surface reflectance)
:
–
Lambertian Surface (Perfect diffuse reflection, no specular light)
•
Simulation
(of light source model)
:
–
We need a formula to describe the behavior of the light source
–
Blackbody radiation : parameterize the light source with:
•
Light color (color temperature)
•
Light intensity
5
Problem formulation:
E
R
E
G
E
B
q
ˆ
R q
ˆ
G
q
ˆ
B
ˆ
N
i
S
R
d
ˆ
N
i
S
G
d
ˆ
N
i
S
B
d
E
R
q
R
L
R
I T i
R
, i
E
G
q
G
L
G
I T i
E
B
q
B
L
B
I T i
, i
G
, i
B
, i
6
E
R
q
R
L
R
I T i
R
, i
E
G
q
G
L
G
I T i
E
B
q
B
L
B
I T i
, i
G
, i
B
, i
Assume: λ
R
λ
G
λ
G are known
L
IC
1
5 exp(
C
2
/
T ) log ln R i
ln q
R
ln I i
ln C
1
R
5
T i
C
2
R
ln
Ri ln G i
ln q
G
ln I i
ln C
1
G
5
T i
C
2
G
ln
Gi ln B i
ln q
B
ln I i
ln C
1
B
5
T i
C
2
B
ln
Bi
If there are N pixels in an image:
3N observations
5N unknowns (I, T, ref )
+ 3 quantize factors
Ax
b underdetermined system!
7
Ax
b
A A
A
I
A
T
A q
3 N
5 N
3
A
1
1
3 N
3 N
A q
1
1
1
1
1
1
3 N
3
A
I
1
1
1
1
1
1
1
1
1
3
A
T
C
2
C
C
2
2
R
G
B
C
2
C
2
C
2
R
G
B
3 x
ln ln ln ln
I
I
1
R 1
BN
N
T
1
1
T
N
1 ln ln ln q
R q
G q
B
5 N
3 b
ln ln ln
ln ln ln
R
G
B
R
B
1
1
1
N
G
N
N
ln ln ln ln ln ln
C
1
C
1
C
1
C
C
C
1
1
1
R
5
G
5
5
B
5
R
5
G
5
B
3 N
8
Recovering unknown
x
•
Previous approach
•
–
Introducing regularization terms into objective function min
– x
Ax b
2 s C p
s
Current approach w sp
ln
s
ln
p
2
Minimizing L1-norm total variation x
ln
ln ln ln
T
T ln ln ln
1
N
I
I
BN
1
1 q q q
R 1
1
N
R
G
B
5 N
3
9
Previous Approach
min x
Ax b
2 s C p
s w sp
ln
s
ln
p
2 w sp
e
p s
p p
2
2
2 w
A result of:
Intrinsic images by entropy minimization ,
Finlayson, ECCV2004
0 p p p s
255 s p 1-D grayscale visualization
A segmentation-like result
10
Drawbacks of this approach
•
There are at least 2 parameters (λ, σ) to be fine tuned.
•
• The results of Finlayson’s approach heavily affects the accurateness of our prior.
–
1. Its Achilles heel: projection problem
–
2. it is still an open problem to find the best rotation angle.
11
(λ =50 , σ = 10)
(λ =120 , σ = 5)
(λ =10 , σ = 30)
(λ =120 , σ = 8)
12
A brief review of Finlayson’ solution
•
Its Achilles heel:
13
L1 norm Total Variation Minimization
V b a
b a f
n x n
1
x n
Image From Wikipedia
14
L1 norm Total Variation Minimization
•
Widely used in image denoise / Compressive sensing
–
E ( x , y ) + λ TV ( y ).
n x n
1
x n
1
2
n
x n
y n
2
Image From Wikipedia
15
Current approach: L1 TV norm
• min
w i
1 st Ax
b w i
D i ln
i i
•
Applying L1-norm total variation on albedo term,
•
The L1-norm encourages a spiky result on gradient
–
Which means: we want most of the albedo gradients are 0 unless necessary x
=> when albedo changes
ln ln ln ln
R 1
BN
I
1
I
N
T
1
1
T
N
1 ln ln ln q
R q
G q
B
5 N
3
16
Results
Original image Albedo (reflectance) image
Light intensity image Light color (temperature) image
17
Results
Original image Albedo (reflectance) image
Light intensity image Light color (temperature) image
18
Results
Original image
Albedo (reflectance) image Light color (temperature) image
19
Results
Original image
Albedo (reflectance) image Light color (temperature) image
20
Editing
Average T = 3940
Average T-1000 Original image Average T+1000
Average T+2000 Average T+3000 Average T+4000 21
THANK YOU
22
