Scene illumination and surface albedo recovery via L1-norm total variation minimization
Hong-Ming Chen hc2599@columbia.edu
Advised by: John Wright

= .* scene illumination Reflectance
(albedo)
.* :
Matlab element multiplication operation
2

Pixel i
.*
=
E
R
  signals
E
G
 
E
B
  integration illumination reflectance scene intensity response Sensor response
S
G
S
B
ˆ  
Light source power spectrum
E
R
 
E
B

E
G
  
 q
ˆ
R q
ˆ
G

 q
ˆ
B

 
S
R
ˆ
Object reflectance
   
N
  i

S
R
 
ˆ    
N
 i

S
G
 
 d d
Sensor response (camera or eyes)

 q
ˆ k
: shutter speed, aperture size, quantization factor
ˆ    
N
 i

S
B
  d
 etc
3

It is VERY HARD to directly model / simulate / solve this problem!
4

Narrowing down our target problem
•
Simplification:
– mean wavelength response (impulse response)
•
Assumption
(on surface reflectance)
:
–
Lambertian Surface (Perfect diffuse reflection, no specular light)
•
Simulation
(of light source model)
:
–
We need a formula to describe the behavior of the light source
–
Blackbody radiation : parameterize the light source with:
•
Light color (color temperature)
•
Light intensity
5

E
R
 
E
G
 
E
B


 q
ˆ
R q
ˆ
G

 q
ˆ
B

ˆ    
N
  i

S
R
  d

ˆ    
N
 i

S
G
  d

ˆ    
N
 i

S
B
  d

E
R
   q
R
L
 
R
I T i
 
R
, i

E
G
   q
G
L
 
G
I T i
E
B
   q
B
L
 
B
I T i
, i
 
G
, i

 
B
, i

6

E
R
   q
R
L
 
R
I T i
 
R
, i

E
G
   q
G
L
 
G
I T i
E
B
   q
B
L
 
B
I T i
, i
 
G
, i

 
B
, i

Assume: λ
R
λ
G
λ
G are known
L
 
IC
1
 
5 exp(

C
2
/

T ) log ln R i
 ln q
R
 ln I i
 ln C
1

R

5 
T i
C
2

R
 ln

Ri ln G i
 ln q
G
 ln I i
 ln C
1

G

5 
T i
C
2

G
 ln

Gi ln B i
 ln q
B
 ln I i
 ln C
1

B

5 
T i
C
2

B
 ln

Bi
If there are N pixels in an image:
3N observations
5N unknowns (I, T, ref )
+ 3 quantize factors
Ax
 b underdetermined system!
7

Ax
 b
A A

A
I
A
T
A q
3 N

5 N

3
A


1
 


1




3 N

3 N
A q





1




1
1
1
1
1











3 N

3
A
I





 



1

1




1
1
1
1
1
1














1
3
A
T










C
2

C
C
2
2

R

G

B

C
2

C
2

C
2

R

G

B










3 x















 ln ln ln ln


I
I
1
R 1
BN
N
T
1

1
T
N

1 ln ln ln q
R q
G q
B


















5 N

3 b
 




 ln ln ln



 ln ln ln
R
G
B
R
B
1
1
1
N
G
N
N





 ln ln ln ln ln ln
C
1
C
1
C
1
C
C
C
1
1
1

R

5

G

5





5
B

5
R

5
G

5
B










3 N
8

x
•
Previous approach
•
–
Introducing regularization terms into objective function min
– x
Ax b
2     s C p
  s
Current approach w sp
 ln
 s
 ln
 p

2
Minimizing L1-norm total variation x






 








 ln

 ln ln ln
T
T ln ln ln
1

N

I
I
BN

1

1 q q q
R 1
1
N
R
G
B


















5 N

3
9

min x
Ax b
2     s C p
  s w sp
 ln
 s
 ln
 p

2 w sp
 e
 p s
 p p

2

2
 2 w
A result of:
Intrinsic images by entropy minimization ,
Finlayson, ECCV2004
0 p p p s
255 s p 1-D grayscale visualization
A segmentation-like result
10

•
There are at least 2 parameters (λ, σ) to be fine tuned.
•
• The results of Finlayson’s approach heavily affects the accurateness of our prior.
–
1. Its Achilles heel: projection problem
–
2. it is still an open problem to find the best rotation angle.
11

(λ =50 , σ = 10)
(λ =120 , σ = 5)
(λ =10 , σ = 30)
(λ =120 , σ = 8)
12

A brief review of Finlayson’ solution
•
Its Achilles heel:
13

L1 norm Total Variation Minimization
V b a
    b a f
  
  n x n

1
 x n
Image From Wikipedia
14

L1 norm Total Variation Minimization
•
Widely used in image denoise / Compressive sensing
–
E ( x , y ) + λ TV ( y ).

  
 n x n

1
 x n
1
2
 n
 x n
 y n

2
Image From Wikipedia
15

• min
 w i
1 st Ax
 b w i

D i ln
 i i
•
Applying L1-norm total variation on albedo term,
•
The L1-norm encourages a spiky result on gradient
–
Which means: we want most of the albedo gradients are 0 unless necessary x
=> when albedo changes















 ln ln ln ln


R 1
BN
I
1
I
N
T
1

1
T
N

1 ln ln ln q
R q
G q
B


















5 N

3
16

Results
Original image Albedo (reflectance) image
Light intensity image Light color (temperature) image
17

Results
Original image Albedo (reflectance) image
Light intensity image Light color (temperature) image
18

Results
Original image
Albedo (reflectance) image Light color (temperature) image
19

Results
Original image
Albedo (reflectance) image Light color (temperature) image
20

Average T = 3940
Average T-1000 Original image Average T+1000
Average T+2000 Average T+3000 Average T+4000 21

22