1
Resolution Loss
without
Optical Blur
Tali Treibitz
Alex Golts
Yoav Y. Schechner
Technion , Israel
14
total
intensity
I
L object
airlight
A
D
direct
transmission
scattering
D
I ( x, y)  Lobject ( x, y) t ( z) 
A( x, y)
A
1
e z
0
object
radiance
1 e z
z
Schechner, Narasimhan, Nayar
0
z
15
Haze
I ( x)  l
object
object
Schechner et al., Applied Optics ‘03
( x)t ( x)  A( x)
transmittance
airlight
17
Pointwise Degradations
noise
I ( x )  l object 
t( x)
 A( x )
 n( x )
object
pointwise attenuation:
additive component:
vignetting
reflection
atmosphere attenuation
glare
path radiance
Treibitz & Schechner, Recovery Limits in Pointwise Degradations
18
Pointwise Degradations
reduce SNR
even if known
I ( x)  l
object

t( x)
 A( x )
noise
 n( x )
object
pointwise attenuation
additive (positive) bias
Treibitz & Schechner, Recovery Limits in Pointwise Degradations
19
Noise: Object size matters?
20
Noise: Object size matters?
  0.5
21
Noise: Object size matters?
 1
22
Noise: Object size matters?
  1.5
Visibility Under Noise
depends on:
noise level
object  background intensity difference
object size
quantify this dependency !
Prior art: resolution limits due to optical blur
here: no optical blur
23
Previous criteria
• Is there something there?
• Is it a tank?
• What type is it?
Johnson charts:
detection
tank
0.75
orientation recognition
1.2
3.5
minimum line pairs for 50% success
identification
7
NIIRS- National Image Interpretability Rating Scales
0
Interpretability of the imagery is precluded by obscuration, degradation,
or very poor resolution
1
Detect a medium-sized port facility and/or distinguish between taxi-ways and
runways at a large airfield .
2
Detect large hangars at airfields. Detect large static radars (e.g., AN/FPS-85,
COBRA DANE, PECHORA, HENHOUSE), Detect military training areas...
3
Identify the wing configuration (e.g., straight, swept, delta) of all large aircraft
(e.g., 707, CONCORD, BEAR, BLACKJACK) ...
9
Identify small light-toned ceramic insulators that connect wires of an antenna.
Identify vehicle registration numbers (VRN) on trucks .
Identify screws and bolts on missile components ...
24
pattern visible
Treibitz & Schechner, Recovery Limits in Pointwise Degradations
25
Where is the Cutoff?
pattern visible
calculated analytically!
pattern
invisible
0.5
Input SNR
| S (u ) |

 noise
10
0.1
u (frequency)
0.5
Treibitz & Schechner, Recovery Limits in Pointwise Degradations
26
Cutoff Per Success Rate
success rate
50%
0.5
Input SNR
| S (u ) |

 noise
1
10
0.1
ucutoff
u (frequency)
0.5
Treibitz & Schechner, Recovery Limits in Pointwise Degradations
28
Noise Suppression in the HVS
response of receptive field
Theoretical Neuroscience,
Dayan & Abbott
low noise
high noise
frequency (cycles/degree)
We derive: fundamental analytical model
Model: simple linear denoising
not a denoising method
29
SNR Improvement by Averaging
signal
HW
SNR output
- SNR change after averaging
C(u,W) =
input
SNR
noise

Treibitz & Schechner, Recovery Limits in Pointwise Degradations
 /W
30
Different Sizes of Windows
too big for signal
too small for noise
Treibitz & Schechner, Recovery Limits in Pointwise Degradations
31
Averaging by Optimal Window
Treibitz & Schechner, Recovery Limits in Pointwise Degradations
32
SNR Improvement by Averaging
C max (u )
output
SNRmax
(u)  SNRinput 
depends on
frequency!
u
same plot for a Gaussian filter
Treibitz & Schechner, Recovery Limits in Pointwise Degradations
33
Output SNR
SNRoutput  1
SNRoutput  0.5
0.32
Input SNR
| S (u ) |

 noise
1
6.5
0.1
ucutoff
u (frequency)
Treibitz & Schechner, Recovery Limits in Pointwise Degradations
0.5
34
Cutoff Per Success Rate
success rate
70%
success rate
40%
0.32
Input SNR
| S (u ) |

 noise
6.5
0.1
ucutoff
u (frequency)
Treibitz & Schechner, Recovery Limits in Pointwise Degradations
0.5
35
Vision Success is Probabilistic
visible
invisible
SNR=2/3
SNR determines chances of visibility
Treibitz & Schechner, Recovery Limits in Pointwise Degradations
36
Success within a Confidence Interval
Object is visible if N ( x)  S
N S
prob  N ( x )  S 
…depends on SNR and..
 - success rate
68%
0.68
2 
0
 2
N ( x)
SNR
Treibitz & Schechner, Recovery Limits in Pointwise Degradations
25
Success within a Confidence Interval
visibility is kept if edge keeps sign
prob  N background ( x )  N object ( x )  S 
depends on SNR
noisy
clear
background object
pixel
pixel
what is the probability for correct detection?
 - success rate
%(sign kept) - %(wrong sign)
0.5
SNR
Treibitz & Schechner, Recovery Limits in Pointwise Degradations
37
Determining Resolution Limits
cutoff for
ρ=70% success
0.32
system
input SNR
| S (u ) |

 noise
6.5
0.1
ucutoff
frequency
0.5
Treibitz & Schechner, Recovery Limits in Pointwise Degradations
38
Pointwise Degradations
noise
I ( x )  l object 
t( x)
 A( x )
 n( x )
pointwise attenuation:
additive component:
vignetting
reflection
atmosphere attenuation
glare
haze
Treibitz & Schechner, Recovery Limits in Pointwise Degradations
39
Noise Model
2 2  I / g
2
Nikon D100
 2  200
I
photon noise dominates
Treibitz & Schechner, Recovery Limits in Pointwise Degradations
9
Photon (shot) Noise
Photon
either
{
Electrons
e
or
nothing
Schechner
10
Photon (shot) Noise
Photons
either
{
Electrons
e
e
e
nothing
Schechner
41
SNR per size (frequency)
l
l
object
l
background
background
l object
SNR =
|S|

,
S l
Treibitz & Schechner, Recovery Limits in Pointwise Degradations
object
l
background
t ( x)
42
Resolution Limits in Haze
minimal visible
object size[m]
reciprocal
to
ucutoff
limit due to
atmosphere
limit due to
pixel size
distance
[km]
Treibitz & Schechner, Recovery Limits in Pointwise Degradations
43
Cutoff Per Success Rate
success rate
50%
0.5
Input SNR
| S (u ) |

 noise
1
10
0.1
ucutoff
u (frequency)
0.5
Treibitz & Schechner, Recovery Limits in Pointwise Degradations
44
Haze in the Galilee
raw frame
average of 50 frames
limit due to noise and not blur
Treibitz & Schechner, Recovery Limits in Pointwise Degradations
45
What now?
What are the reconstruction limits?
What is the minimal detectable object size?
What camera noise properties are acceptable for detection?
…
46
Imaging in Haze
Treibitz & Schechner, Polarization- Beneficial for Visibility Enhancement?
Haze Through a Polarizer
47
best
polarized
image
Treibitz & Schechner, Polarization- Beneficial for Visibility Enhancement?
Haze Through a Polarizer
48
best
polarized
image
increased
exposure time
single frame- used by photographers
Treibitz & Schechner, Polarization- Beneficial for Visibility Enhancement?
Dehazing using a Polarizer
49
best
polarized
image
worst
polarized
image
post-processing
2 frames
two frames- Schechner et al.
Treibitz & Schechner, Polarization- Beneficial for Visibility Enhancement?
Is it worth using a polarizer?
50
rarely!
under the constraint of
equal acquisition time
unpolarized image
best polarized
image
 goal: object detection
 local contrast stretch- OK
Treibitz & Schechner, Polarization- Beneficial for Visibility Enhancement?
post-processing
2 frames
51
Using a Single Polarized Image
I min
Best
polarized
image
Imin (x, y ) 
D ( x, y )
1 p
 A( x, y )
2
2
Treibitz & Schechner, Polarization- Beneficial for Visibility Enhancement?
p  0,1
degree of polarization
52
SNR Comparison
I unpolarized
Ibest

Treibitz & Schechner, Polarization- Beneficial for Visibility Enhancement?
53
A Single Saturated Frame
SNRpolarized
SNRunpolarized
p
maximal value
in nature
Treibitz & Schechner, Polarization- Beneficial for Visibility Enhancement?
54
SNR Comparison
I unpolarized
Ibest


equal acquisition time
acquisition time = exposure time X number of frames
55
Same Total Acquisition Time
SNRpolarized
SNRunpolarized
p
p<0.4
in our experiments
Treibitz & Schechner, Polarization- Beneficial for Visibility Enhancement?
maximal value
in nature
56
Experiment
Wide field of view
I best
I unpolarized
average of 2 frames
same total acquisition time
Treibitz & Schechner, Polarization- Beneficial for Visibility Enhancement?
57
SNR Comparison
I unpolarized
dehazing
 1 p 


 p 


 1 p 


 p 

equal acquisition time
optimal
exposures tbest

tworst
58
Advantages of Polarization
• contrast stretch in non-uniform distances
• restoring color
• compensating for attenuation
distance map
59
Limits in Pointwise
Degradations
• Freq cutoff – due to noise – without imaging blur
• Relation between cutoff and success rate
• Application: limits in pointwise degradations
• Case study of performance trade-offs