(1) A probability model respecting
those covariance observations:
Gaussian
• Maximum entropy probability distribution for a
given covariance observation (shown zero mean for
notational convenience):
Inverse covariance matrix



P( x )  exp(  12 x T Cx1 x )
Image pixels
• If we rotate coordinates to the Fourier basis, the
covariance matrix in that basis will be diagonal.
So in that model, each Fourier transform
coefficient is an independent Gaussian random
variable of covariance
D( )  E (| F ( ) |2 )
Power spectra of typical images
Experimentally, the
power spectrum as a
function of Fourier
frequency is observed
to follow a power law.
E (| F ( ) | ) 
2
http://www.cns.nyu.edu/pub/eero/simoncelli05a-preprint.pdf
A

Random draw from Gaussian spectral model
http://www.cns.nyu.edu/pub/eero/simoncelli05a-preprint.pdf
Noise removal (in frequency domain), under
Gaussian assumption
Posterior
Observed Fourier
component
Estimated Fourier
probability for X component
Power law prior
probability on
estimated Fourier
component
P( X | Y )  exp(  || Y  X || 2 ) exp(  X (
2
2
n
A

1
) X 2)
Variance of white,
Gaussian additive noise
Setting to zero the derivative of the the log probability of
X gives an analytic form for the optimal estimate of X (or
just complete the square):
Xˆ ( ) 

A
Y ( )

2
A  n
Noise removal, under Gaussian assumption
original
With Gaussian noise of
std. dev. 21.4 added,
giving PSNR=22.06
(try to ignore JPEG compression artifacts from the PDF file)
http://www.cns.nyu.edu/pub/eero/simoncelli05a-preprint.pdf
(1) Denoised with
Gaussian model,
PSNR=27.87
(2) The wavelet marginal model
Histogram of wavelet coefficients, c, for various images.
http://www.cns.nyu.edu/pub/eero/simoncelli05a-preprint.pdf
P(c)  exp(  | c s | )
p
Wavelet coefficient value
Parameter determining
peakiness of distribution
Parameter determining
width of distribution
Random draw from the wavelet marginal model
http://www.cns.nyu.edu/pub/eero/simoncelli05a-preprint.pdf
And again something that is reminiscent of operations found in V1…
An application of image pyramids:
noise removal
Image statistics (or, mathematically, how can
you tell image from noise?)
Noisy image
Clean image
Pixel representation,
image histogram
Pixel representation,
noisy image histogram
bandpassed representation
image histogram
Pixel domain noise image and
histogram
Bandpass domain noise image and
histogram
Noise-corrupted full-freq and bandpass images
But want
the
bandpass
image
histogram
to look like
this
Bayes theorem
By definition of
conditional probability
P(x, y) = P(x|y) P(y)
so
Using that twice
P(x|y) P(y) = P(y|x) P(x)
and
P(x|y) = P(y|x) P(x) / P(y)
The parameters
you want to
estimate
Constant w.r.t.
Likelihood
parameters x.
function
Prior probability
What you observe
Bayesian MAP estimator for clean bandpass coefficient
values
Let x = bandpassed image value before adding noise.
Let y = noise-corrupted observation.
By Bayes theorem
y = 25
P(x|y) = k P(y|x) P(x)
y
P(x)
P(y|x)
P(y|x)
P(x|y)
P(x|y)
Bayesian MAP estimator
Let x = bandpassed image value before adding noise.
Let y = noise-corrupted observation.
By Bayes theorem
y = 50
P(x|y) = k P(y|x) P(x)
y
P(y|x)
P(x|y)
Bayesian MAP estimator
Let x = bandpassed image value before adding noise.
Let y = noise-corrupted observation.
By Bayes theorem
y = 115
P(x|y) = k P(y|x) P(x)
y
P(y|x)
P(x|y)
y = 25
y = 115
y
P(x)
P(y|x)
P(y|x)
P(x|y)
y
P(x|y)
For small y: probably it is due to noise and y should be set to 0
For large y: probably it is due to an image edge and it should be kept untouched
MAP estimate, x̂ , as function of
observed coefficient value, y
x̂
y
http://www-bcs.mit.edu/people/adelson/pub_pdfs/simoncelli_noise.pdf
Simoncelli and Adelson, Noise Removal via
Bayesian Wavelet Coring
original
With Gaussian noise of
std. dev. 21.4 added,
giving PSNR=22.06
(2) Denoised
with wavelet
marginal model,
PSNR=29.24
http://www.cns.nyu.edu/pub/eero/simoncelli05a-preprint.pdf
(1) Denoised with
Gaussian model,
PSNR=27.87
M. F. Tappen, B. C. Russell, and W. T. Freeman, Efficient
graphical models for processing images IEEE Conf. on
Computer Vision and Pattern Recognition (CVPR) Washington,
DC, 2004.
Motivation for wavelet joint models
Note correlations between
the amplitudes of each
wavelet subband.
http://www.cns.nyu.edu/pub/eero/simoncelli05a-preprint.pdf
Statistics of pairs of wavelet coefficients
Contour plots of the joint histogram of various wavelet coefficient pairs
Conditional distributions of the corresponding wavelet pairs
http://www.cns.nyu.edu/pub/eero/simoncelli05a-preprint.pdf
(3) Gaussian scale mixtures

P( x )  

1 
exp(  x ( z) x )
1 T
2
N 2
(2 )
| z |
12
Pz ( z ) dz
observed
Wavelet
coefficient
probability
A mixture of
Gaussians of
scaled
covariances
z is a spatially varying hidden variable
that can be used to
(a) Create the non-gaussian histograms
from a mixture of Gaussian densities, and
(b) model correlations between the
neighboring wavelet coefficients.
Gaussian scale
mixture model
simulation
original
With Gaussian noise of
std. dev. 21.4 added,
giving PSNR=22.06
(1) Denoised with
Gaussian model,
PSNR=27.87
(3) Denoised with
Gaussian scale
mixture model,
PSNR=30.86
http://www.cns.nyu.edu/pub/eero/simoncelli05a-preprint.pdf
(2) Denoised with wavelet
marginal model,
PSNR=29.24