Noise Estimation from a Single Image
Ce Liu
William T. Freeman
Richard Szeliski
Sing Bing Kang
Parameter Tweaking in Computer Vision

Computer vision algorithms suffer from hand tuning
parameters for particular images or image sequences

We want vision algorithms that behave properly under
varying lighting conditions, blur levels and noise levels

Our work is one step in that direction
 Given an image, estimate the noise level
 Modify vision algorithms to be independent of noise
Image Noise Is Important in Vision

In image denoising the noise is assumed to be known as
Additive Gaussian White Noise (AWGN)

However, in real applications the noise is unknown and
non-additive

Many other computer vision algorithms also explicitly or
implicitly assume the type and level of image noise

Hard to make vision algorithms fully automatic without
knowing noise
Noise Level Function (NLF)

The standard deviation of noise s is a function of image
brightness I

Measurable by fixing the camera and taking multiple
shots of a static scene
s

For each pixel:
 Mean: I
 Standard deviation: s

NLF depends on camera, ISO, shutter speed, aperture

Our goal is to estimate NLF from a single image
 How to estimate noise without separating noise and signal?
I
An Example Image
Piecewise Smooth Image Prior
Affine model
=
+
Patch
Signal
Brightness
mean I
Standard deviation
For each RGB
channel:
s
s
Red
Residual
s
Green
0.2
0.2
0.2
0.1
0.1
0.1
0
0
0.5
1
I
0
0
0.5
Brightness
1
I
0
Standard
deviation s
Blue
0
0.5
1
I
Piecewise Smooth Image Prior
=
+
Standard deviation
Patch
s
s
Red
Signal
Residual
s
Green
0.2
0.2
0.2
0.1
0.1
0.1
0
0
0.5
1
I
0
0
0.5
Brightness
1
I
0
Blue
0
0.5
1
I
Piecewise Smooth Image Prior
=
+
Standard deviation
Patch
s
s
Red
Signal
Residual
s
Green
0.2
0.2
0.2
0.1
0.1
0.1
0
0
0.5
1
I
0
0
0.5
Brightness
1
I
0
Blue
0
0.5
1
I
Segmentation-based Approach
Observed image
Segmentation-based Approach
Oversegmentation
Segmentation-based Approach
Signal
Segmentation-based Approach
Residual=
noise +
unmodelled
image variation
Estimate NLFs
s
s
Residual std. dev.
s
I
I
Brightness



Assume brightness mean I is accurate estimate
Standard deviation s is an over-estimate: (may contain
signal)
The lower envelope is the upper bound of NLF
I
Issues
s
s
Residual std. dev.
s
I
I
Brightness

Should the curve be strictly and tightly below the points?
I
Issues
s
s
Residual std. dev.
s
I
I
Brightness

Should the curve be strictly and tightly below the points?

How to handle the missing data?
I
Issues
s
s
Residual std. dev.
s
I
I
Brightness

Should the curve be strictly and tightly below the points?

How to handle the missing data?

Correlation between RGB channels?
I
Solutions
s
s
Residual std. dev.
s
I
I
Brightness

Formulate the inference problem in a probabilistic
framework

Learn the prior of noise level functions
I
Outline

Over-segmentation and per-segment variance analysis

Learning the priors of noise level functions (NLF)
 Synthesize CCD noise
 Sample noise level functions
 Learn the prior of noise level functions

Inference: estimate the upper bound of NLF
 Bayesian MAP to estimate NLFs for RGB channels

Applications
 Adaptive bilateral filtering
 Canny edge detection
Camera Noise
L
Scene
Radiance
Atmospheric
Attenuation
Lens/
geometric
Distortion
CCD Imaging/
Bayer Pattern
Fixed Pattern
Noise
Quantization
Noise
Digital
Image
I


A/D
Converter
Shot
Noise
Dark Current
Noise
Camera
Irradiance
Thermal
Noise
Gamma
Correction
White
Balancing
Interpolation/
Demosaic
t
Noise model I  f ( L  ns  nc )
Tsin et. al. Statistical calibration of CCD image
process. ICCV, 2001
Dependent noise: E(ns )  0, Var(ns )  Ls s2
Independent noise: E(nc )  0, Var (nc )  s c2
Camera response function (CRF) f: download from Columbia camera
response function database (used 196 typical CRFs)
Synthesize CCD Noise
IN
I
Estimate NLF  ( I ; f , s s , s c )  E ( I N  I )
Camera response function: f
Dependent noise: s s
Independent noise: s c
Sample NLFs by Varying the Parameters
Camera response
function (CRF) f
ss
sc
0.02
0.18
Dependent noise: s s
Independent noise: s c
0.02
0.18
0.02
0.04
0.06
The Prior of NLFs
Likelihood Function

The estimated standard deviation sˆ should be
probabilistically bigger than and close to the true value s
s
sˆ
s
(Iˆn ,sˆ n )
sˆ
s
p(sˆ | s )
s

sˆ
Bayesian MAP inference
nh
(n+1)h
I
Validation (1): Synthetic Noise

Add synthetic CCD noise, estimate, compare to the
ground truth
— ground truth
—
—
— estimated
Validation (2): Measure NLF of a Real Camera

29 images were taken under the same settings (the camera is
not in the database for training)

The real NLF is obtained by computing mean and variance
per pixel
Validation (3): Robustness Test

Verify that different images from the same camera give
the same estimated NLF (camera not in the database for
training)
Application (1): Adaptive Bilateral Filtering

Bilateral filter is an edge-preserving low-pass filter
 Spatial sigma and range sigma
Input noisy image

Smoothing kernel
Denoised image
From Durand and Dorsey, SIGGRAPH 02
Adaptive bilateral filter
 Down-weigh RGB values by signal and noise covariance matrices
 The range sigma is set to be a function of the estimated standard
deviation of the noise
Test on Low and High Noise
low noise
Red
high noise
Green
Blue
Red
Green
Blue
Results—Adaptive Bilateral Filtering
Standard
bilateral
filtering
Adaptive
bilateral
filtering
low noise
high noise
Results—Adaptive Bilateral Filtering
Standard
bilateral
filtering
Adaptive
bilateral
filtering
Zoom in
high noise
Application (2): Canny Edge Detection
low noise
Red
high noise
Green
Blue
Red
Green
Blue
Results—Canny Edge Detection
Parameters
adapted in
MATLAB
Parameters
adapted by
estimated
noise
low noise
high noise
Conclusion

Piecewise-smooth image prior model
to estimate the upper bound of noise
level function (NLF)

Estimate the space of NLF by
simulating CCD camera on the
existing CRF database

Upper bounds are verified by both
synthetic and real experiments

An important step to automate vision
algorithms independent of noise
Thank you!
Noise Estimation from a Single Image
Ce Liu
William T. Freeman
CSAIL MIT
Rick Szeliski
Sing Bing Kang
Microsoft Research