Overview of Haze Removal Methods
Matteo Pedone
Machine Vision Group, University of Oulu, Finland
Overview of Haze Removal Methods
1. Description of the problem
2. Overview of current approaches found in literature
3. Strengths and weaknesses of present methods
4. Description of our method
The Atmospheric Scattering Model
•
A particle scatters incident light
•
The nature of scattering depends on material properties,
shape and size
•
The exact form and intensity of the scattering pattern varies
dramatically with particle size
The Atmospheric Scattering Model
Haze:
• constituted of aerosol (small particles suspended in gas)
• Main sources: volcanic ashes, foliage exudation, combustion
products, sea salt…
• Haze particles are larger than air molecules but smaller than
fog droplets.
• produce a distinctive gray or bluish hue and affects visibility.
• Extends to altitudes of several Km
The Atmospheric Scattering Model
Fog:
• Same origins as haze, associated with an increase in relative
humidity of an air
• Size of water droplets increases
• Haze can turn into fog (transition state: mist)
• Reduces visibility more than haze
• Extends to altitudes of few hundred meters.
The Atmospheric Scattering Model
The Atmospheric Scattering Model
Important physical mechanisms:
• Attenuation: radiance of a scene-point falls
as its distance from the observer increases
• Airlight: Atmosphere behaving like a source
of light. Due to multiple scattering. Increases
with distance.
The Atmospheric Scattering Model
Mathematical Model:
•
I(x) is the observed radiance at x
•
J(x) is the original scene radiance at x
• A is the airlight
• t(x), scalar called transmission: describes how the radiance of a point in the
scene is attenuated according to its distance d from the observer
• Note that I, J, A are (R,G,B) triplets
The Atmospheric Scattering Model
Mathematical Model:
•
In order to remove the effect of haze, one must recover J(x)
•
Quantities A and t are typically unknown
• I(x) is known
The Atmospheric Scattering Model
References
Narasimhan & Nayar, “Vision and the Atmosphere”, International
Journal of Computer Vision, 2001
Overview of Haze Removal Methods
1. Description of the problem
2. Overview of current approaches found in literature
3. Strengths and weaknesses of present methods
4. Description of our method
Current Methods for Haze Removal
• Can be grouped into several categories
1. With multiple images
2. With one image + depth-map
3. Single image
•
Subcategories of the ones above are:
1. Requires user interaction
2. Fully automatic
• We are mostly interested in Single-Image methods
Multiple Image Approaches
Assume 2+ images of the same scene are taken:
 Under different weather conditions [1]
or
 With different polarization filters [2]
[1] Narasimhan & Nayar, “Vision and the Atmosphere”, 2001
[2] Schechner et al. 2003, “Polarization-based vision through haze”, Applied Optics 42
Multiple Image Approaches
Narasimhan & Nayar’s method
• Assumes 2+ bad weather images are given
• Uses geometric constraints to estimate A
• The airlight component [1-t(x)] is estimated from corresponding pixels of the
two bad weather images
Multiple Image Approaches
Narasimhan & Nayar’s method, RESULTS
(a),(b) Foggy images
(c) Dehazed image, (d) Clear weather images
One Image + Depth + Texture
Kopf et al. Method: Deep Photo project from SIGGRAPH 2008
• Assumes a 3D model of the scene is given (e.g.: from Google Maps)
• Assumes textures of the scene are given (from satellite or aerial photos)
• Requires user interaction to align the 3D model with the scene
• Very accurate results
Single Image Approaches
• Do not require information extracted from additional images
• Do not require depth-information
• Typically rely upon statistical assumptions, and or the nature
of the scene (e.g. part of the sky is visible)
• Sometimes they require user interaction
• Most relevant:
•
•
Fattal’s ”Single-Image Dehazing”, SIGGRAPH 2008
He’s ”Single Image Haze Removal Using Dark Channel Prior”, CVPR 2009
Single Image Approaches
He’s Method (with Dark Channel Prior)
• Assumes a portion of the scene is dominated by airlight
• STATISTICAL ASSUMPTION: ”in most of the non-sky
patches, at least one color channel has very low intensity at
some pixels. In other words, the minimum intensity in such
a patch should have a very low value”
• The 1st assumption is used to estimate airlight, the 2nd
assumption is used to estimate the transmission
Single Image Approaches
He’s Method (with Dark Channel Prior)
• Dark Channel:
• Airlight is estimated by picking up the pixels of the image
corresponding to the 0.1% brightest pixels in the dark channel,
and then choosing the one with maximum intensity.
Single Image Approaches
He’s Method (with Dark Channel Prior)
• Dark Channel:
• He shows that the transmission can be estimated by
calculating:
Single Image Approaches
He’s Method (with Dark Channel Prior)
• Dark Channel:
• He shows that the transmission can be estimated by
calculating:
Dark channel of the
image divided by the
airlight color
Single Image Approaches
He’s Method (with Dark Channel Prior)
• Airlight and transmission are sufficient to invert the model and
retrieve the original radiance of the scene.
• Dark channel is computed on square neighborhoods  Block
artifacts and halos are reduced by using a soft-matting
algorithm.
Overview of Haze Removal Methods
1. Description of the problem
2. Overview of current approaches found in literature
3. Strengths and weaknesses of present methods
4. Description of our method
Summary
Multiple image methods
• require special equipment (polarizers) or same scene under
different weather conditions.
• They don’t necessarily produce better results than single-images
approaches
Summary
One Image + 3D model + textures
• Accurate and does not require special equipment
• Requires a considerable amount of special information (3D
model, and aerial photos of the scene)
• Requires user interaction
Summary
Single-Image methods
• do not require special equipment, nor extra information
• They either make assumption on the nature of the scene, or
require little interaction by the user
Summary
Single-Image methods
• It is known what are the consequences of a bad estimate for the
transmission  haze is not completely removed, or it is
removed where there is no haze (overboost contrast)
Summary
• None of the aforementioned authors shows what happens when
the airlight estimate is inaccurate (motivation of our work)
Overview of Haze Removal Methods
1. Description of the problem
2. Overview of current approaches found in literature
3. Strengths and weaknesses of present methods
4. Description of our method
Estimation of Airlight
• Collect statistics of airlight colors from 100+ natural images (daylight and
twilight hazy scenes)
• Manually select 32x32 pixel patch with “full haze”
• Airlight colors are scattered around a 29.8784 degrees line in hue-saturation
plane, and most are close to the origin (=> low saturation).
Estimation of Airlight
• Extract patches of 13x13 pixels from hazy image according to the following
criteria:
1.
The patch contains pixels with same transmission and hue but with
different shades (=> same direction for R but different magnitudes)
2.
The pixels in the patch do not have too low or too high transmission
(avoid degenerate cases)
Estimation of Airlight
• Extract patches of 13x13 pixels from hazy image according to the following
criteria:
1.
The patch contains pixels with same transmission and hue but with
different shades (=> same direction for R but different magnitudes)
2.
The pixels in the patch do not have too low or too high transmission
(avoid degenerate cases)
3.
Pixels in the patches do not have too low saturation (hue would not be
reliable).
4.
Pixels in the patches are not too dark or too bright in average, and
variance should not be too high (noise) or too low (homogeneous areas
with no shades).
Estimation of Airlight
• Solve minimization problem: find an airlight color vector that enforces:
1.
2.
3.
Perpendicularity to patches albedos
Closeness to natural airlight hue line
Low saturation
• ni : normal to the plane containing the [R,G,B] values of the pixels in the i-th patch
extracted
• nsky : unit-vector in RGB space having direction corresponding to the statistical hue of
airlight in natural images
• w : unit vector [1,1,1]/31/2
• c(ni ): scalar associated with the i-th patch (based of residual error, see paper)
•
: weight parameters (respective default values: 3, 0.02)
Estimation of Airlight
Results with artifical haze (Middlebury dataset)
Estimation of Airlight
Results with real images
THANK YOU