Image Denoising with the Non-local Means Algorithm
CS766 Homework #4
Fall 2005
Westley Evans
Abstract
Over the years a variety of methods have been introduced to remove noise from digital
images, such as Gaussian filtering, anisotropic filtering, and Total Variation minimization.
However, many of these algorithms remove the fine details and structure of the image in addition
to the noise because of assumptions made about the frequency content of the image. The nonlocal means algorithm does not make these assumptions, but instead assumes that the image
contains an extensive amount of redundancy. These redundancies can then be exploited to
remove the noise in the image. This project will implement the non-local means algorithm and
compare it to other denoising methods using the method noise measurement.
1.0 Introduction
All digital images contain some degree of noise. Often times this noise is introduced by
the camera when a picture is taken. Image denoising algorithms attempt to remove this noise
from the image. Ideally, the resulting denoised image will not contain any noise or added
artifacts.
Major denoising methods include Gaussian filtering, Wiener filtering, and wavelet
thresholding. Many more methods have been developed; however, most methods make
assumptions about the image that can lead to blurring. This paper will explain these assumptions
and present a new method, the non-local means algorithm that does not make the same
assumptions. The non-local means method will then be compared to other denoising methods
using several measurements on the output images. One of the measurements used will be the
method noise, which is the difference between the image and denoised image [2].
1.1 Previous Methods, Motivation, & Problem Statement
Most denoising algorithms make two assumptions about the noisy image. These
assumptions can cause blurring and loss of detail in the resulting denoised images. The first
assumption is that the noise contained in the image is white noise. This means that the noise
contains all frequencies, low and high. Because of the higher frequencies, the noise is oscillatory
or non-smooth. The second assumption is that the true image (image without the noise) is
smooth or piecewise smooth [1]. This means the true image or patches of the true image only
contain low frequencies.
Previous methods attempt to separate the image into the smooth part (true image) and the
oscillatory part (noise) by removing the higher frequencies from the lower frequencies.
However, not all images are smooth. Images can contain fine details and structures which have
high frequencies. When the high frequencies are removed, the high frequency content of the true
image will be removed along with the high frequency noise because the methods cannot tell the
difference between the noise and true image [1]. This will result in a loss of fine detail in the
denoised image. Also, nothing is done to remove the low frequency noise from the image. Low
frequency noise will remain in the image even after denoising. Because of this loss of detail
Baudes et al. have developed the non-local means algorithm [1, 2].
2.0 Non-local Means Theory
The non-local means algorithm does not make the same assumptions about the image as
other methods. Instead it assumes the image contains an extensive amount of self-similarity.
Efros and Leung originally developed the concept of self-similarity for texture synthesis [4]. An
example of self-similarity is displayed in Figure 1 below. The figure shows three pixels p, q1,
and q2 and their respective neighborhoods. The neighborhoods of pixels p and q1 are similar,
but the neighborhoods of pixels p and q2 are not similar. Adjacent pixels tend to have similar
neighborhoods, but non-adjacent pixels will also have similar neighborhoods when there is
structure in the image [1]. For example, in Figure 1 most of the pixels in the same column as p
will have similar neighborhoods to p's neighborhood. The self-similarity assumption can be
exploited to denoise an image. Pixels with similar neighborhoods can be used to determine the
denoised value of a pixel.
Figure 1: Example of self-similarity in an image. Pixels p and q1 have similar neighborhoods, but pixels p and q2
do not have similar neighborhoods. Because of this, pixel q1 will have a stronger influence on the denoised value of
p than q2.
2.1 Non-local Means Method
Each pixel p of the non-local means denoised image is computed with the following
formula:
NL V
p
w p ,q V q
q V
(1)
where V is the noisy image, and weights w(p,q) meet the following conditions 0 w p ,q
and
q
w p ,q
1 . Each pixel is a weighted average of all the pixels in the image. The
weights are based on the similarity between the neighborhoods of pixels p and q [1, 2]. For
1
example, in Figure1 above the weight w(p,q1) is much greater than w(p,q2) because pixels p and
q1 have similar neighborhoods and pixels p and q2 do not have similar neighborhoods. In order
to compute the similarity, a neighborhood must be defined. Let Ni be the square neighborhood
centered about pixel i with a user-defined radius Rsim. To compute the similarity between two
neighborhoods take the weighted sum of squares difference between the two neighborhoods or as
a formula d p , q
V Np
V Nq
2
2, F
[1, 2]. F is the neighborhood filter applied to the
squared difference of the neighborhoods and will be further discussed later in this section. The
weights can then be computed using the following formula:
1
e
Z p
w p ,q
Z(p) is the normalizing constant defined as Z p
d p ,q
h
e
q
d p, q
h
[1,2]. h is the weight-decay
control parameter and will be further discussed in section 2.2.
As previously mentioned, F is the neighborhood filter with radius Rsim. The weights of
F are computed by the following formula:
Rsim
1
Rsim i
1 2 i 1
2
m
where m is the distance the weight is from the center of the filter. The filter gives more weight to
pixels near the center of the neighborhood, and less weight to pixels near the edge of the
neighborhood. The center weight of F has the same weight as the pixels with a distance of one
[3]. Despite the filter's unique shape, the weights of filter F do sum up to one.
Figure 2: Shape of filter F when Rsim = 4.
Equation (1) from above does have a special case when q = p. This is because the weight
w(p,p) can be much greater than the weights from every other pixel in the image. By definition
this makes sense because every neighborhood is similar to itself. To prevent pixel p from overweighing itself let w(p,p) be equal to the maximum weight of the other pixels, or in more
mathematical terms w p , p
max w p , q
p q
[3].
2.2 Non-local Means Parameters
The non-local means algorithm has three parameters. The first parameter, h, is the
weight-decay control parameter which controls where the weights lay on the decaying
exponential curve. If h is set too low, not enough noise will be removed. If h is set too high, the
image will become blurry. When an image contains white noise with a standard deviation of 
h should be set between 10and 15.
The second parameter, Rsim, is the radius of the neighborhoods used to find the similarity
between two pixels. If Rsim is too large, no similar neighborhoods will be found, but if it is too
small, too many similar neighborhoods will be found. Common values for Rsim are 3 and 4 to
give neighborhoods of size 7x7 and 9x9, respectively [1, 2].
The third parameter, Rwin, is the radius of a search window. Because of the inefficiency
of taking the weighted average of every pixel for every pixel, it will be reduced to a weighted
average of all pixels in a window. The window is centered at the current pixel being computed.
Common values for Rwin are 7 and 9 to give windows of size 15x15 and 19x19, respectively [1,
2]. With this change the algorithm will take a weighted average of 152 pixels rather than a
weighted average of N2 pixels for a NxN image.
3.0 Results and Comparisons
Gaussian, Wiener, and SUSAN filtering will be compared to the non-local means method
on several test images. These methods will be compared using three different criteria. 1) visual
quality, 2) mean-square-error (MSE), and 3) the method noise. Visual inspection is the best tool
to determine the quality of the denoised image. The image should be clear and contain no
artifacts or noise. The MSE between the denoised image and true image will show
mathematically how close the resulting denoised image is to the true image. A lower MSE does
not guarantee that one image will look better than another image. The method noise, also
introduced by Buades et al., is the difference between the image and the denoised image.
Ideally, the method noise should only contain white noise. It should not contain any structure or
detail from the image because denoising should only remove the noise [1, 2].
Gaussian and Wiener filtering were computed using the functions from the Matlab Image
Processing Toolbox [7]. The source code to compute the Susan filter was found on Matlab
Central [6]. The source code listing for the non-local means method is attached to the end of the
report. All test images were found in the image databases of Ohio State University's Signal
Analysis and Machine Perception Lab and University of Southern California's Signal and Image
Processing Institute [8, 9].
Because of the large number of images and the need of exact representation of the
images, the test cases and their analyses are only available on line [5]. Four test cases are
available. The first two test cases (Barb and Mandrill) involve adding noise and then trying to
remove it with the denoising methods. The other two test cases (Camera and Walter) do not
involve the addition of artificial noise. Instead the denoising methods are directly applied to the
image. It will be assumed that these images already contain white noise with a standard
deviation of 2.5.
4.0 Conclusion
As expected, Gaussian filtering performed poorly on all the test cases. The resulting
images show little detail and still contain noise. The method noise for the Gaussian filter
contained extensive structure and detail from the image.
The Wiener filter performed marginally better than the Gaussian filter. More noise was
removed by this method, but the images were still blurry. Once again the method noise
contained detail and structure from the image.
The SUSAN filter performed much better than the other two filters. The results from the
Walter and Camera cases show no noise and smooth edges. The SUSAN filter performed better
when there is less noise in the image. However, despite the good results the method noise still
contained some structure from the image.
The non-local means method performed exceptionally well. As
expected, the non-local means did a better job of preserving edges
than the other methods. It performed best on periodic textures like
the stripped pants from the Barb test case. In the Camera and Walter
test cases where no noise was added, the denoised images looked
clear and smooth. In all test cases for the non-local means method,
the method noise contained little structure from the image. The nonlocal means algorithm accomplished its goals of removing noise and
preserving detail.References
[1]
A. Buades, B. Coll, and J Morel. On image denoising methods. Technical Report 200415, CMLA, 2004.
[2]
A. Buades, B. Coll, and J Morel. A non-local algorithm for image denoising. IEEE
International Conference on Computer Vision and Pattern Recognition, 2005.
[3]
A. Buades. NL-means Pseudo-Code.
http://dmi.uib.es/~tomeucoll/toni/NL-means_code.html
[4]
A. Efros and T. Leung. Texture synthesis by non parametric sampling. In Proc. Int.
Conf. Computer Vision, volume 2, pages 1033-1038, 1999.
[5]
W. Evans. Image denoising with the non-local means algorithm.
http://www.cs.wisc.edu/~evanswes/cs766.html
[6]
B. Gustavsson. Matlab Central - gen_susan.
http://www.mathworks.com/matlabcentral/files/6842/gen_susan.m
[7]
Mathworks. The Matlab image processing toolbox.
http://www.mathworks.com/access/helpdesk/help/toolbox/images/
[8]
The Ohio State University Signal Analysis and Machine Perception Lab. Image database.
http://sampl.ece.ohio-state.edu/database.htm
[9]
University of Southern California Signal and Image Processing Institute. Image database.
http://sipi.usc.edu/database/database.cgi?volume=sequences