BGU - Computational Vision Course

advertisement
Image noise filtering using Neural Networks
Final project by
Arie Ohana, 33771635
arie.ohana@gmail.com
Introduction
Scanning images, photographing and other image related operations almost always
involve the corruption of the output image due to the addition of noise. When related
to imaging, noise is a high frequency random perturbation in the image pixels. You
can think of noise as the subtle background hiss you hear at a stereo system at high
volume.
There are many methods to achieve noise elimination and reduction and total
elimination can rarely be found. Moreover, one of the biggest problems in noise
reduction is that the more noise we would like to reduce, the blurrier the image
becomes. Blurring an image as a way of reducing noise isn’t always such a good
idea, a blurred image makes it hard to distinguish between small objects in the
image, not to mention the uneasiness observing that image.
Noise comes in different flavors… Each one can be formulated and characterized.
There is an additive noise, speckle and others. In this project we tried to reduce the
Salt and Pepper noise only. Different types of noise will be reduced by this project's
software in exactly the same method. Further discussion of this can be seen further
in this paper.
This project offers to reduce image noise using a Neural Network. A neural network
is a computing paradigm that is loosely modeled after cortical structures of the brain.
It consists of interconnected processing elements called neurons that work together
to produce an output function. The output of a neural network relies on the
cooperation of the individual neurons within the network to operate. A given neural
network achieves its goal by a learning process. When it is given a sufficient number
of inputs and their desired outputs respectively, the network will adjust itself, by
correcting the current weights on every input, according to a predefined formula.
Running this process for a large enough number of iterations will make the network
express some function that can map each input to its output, using the network's
weights. The more examples this network is shown and the more versatile the
examples are, the larger the network's ability is to generalize the problem.
Neural Networks
A neural network, as explained earlier, models a function by a learning process. By
introducing exemplars to the network, it can fine tune itself in an iterative process,
so that eventually the network will converge into a steady state, and this steady
state behavior is the function we want to model. By looking at the given input and its
desired output, it's often hard to come up with a general function to map those input
and outputs. Instead, the learning process of the neural network will "somehow" find
a pretty close approximation to that unknown function.
There are several methods of building a neural network. The simplest is by using a
single neuron, also called a perceptron, which models a single neuron in the brain.
Its abilities, as you probably guessed, are therefore very limited, it can be used to
model logic functions such as logical OR and logical AND. Any function that can be
linearly separated can be implemented using a single neuron. A function that cannot
be linearly separated needs a more sophisticated structure of the network. A more
sophisticated structure means using more neurons, and more layers. Generally, we
can implement every function using a network that has at least two hidden layers. A
hidden layer is a sequence of neurons that don’t meet the input or output directly.
A perceptron as a basic neural network for
implementing a linearly seperatable function
How many layers should the network consist of? How many neurons in each layer?
What are the initial weights on the neurons' inputs? In order to answer those entire
questions one must first estimate the difficulty of the problem at hand. For instance,
filtering noise in an image is obviously not a simple task, its pretty innocent to
assume that a single perceptron might implement the function. Moreover, it's safe to
assume that we'll need at least to hidden layers since filtering is a complex task. The
initial weights we usually use are randomly chosen small figures close to zero
(usually in the interval [-1, 1]).
The correction of the weights in the learning process might be done in several
methods. The one used in this application is the Backpropagation algorithm. The
algorithm introduces an exemplar to the network, and while doing this, we measure
the output on each neuron, not just the output layer. Using a few formulations we
can calculate the correction delta that needs to be applied to each neuron. Those
formulations are relatively simple and are based on progress of the input in the
network until the output layer is met, and the correction is done in reverse – form
output to input, hence the name Backpropagation. The formulations can be found in
the literature.
Approach and Method
The project consists of two parts. In the first part we learn our network to specialize
at reducing noise. Then we receive a matrix of weights from this learning process
which defines our network. The second part evolves processing a given input (a
noisy image) with that matrix by letting the network perform its function but without
learning (as opposed to the first part).
First stage / Learning
Learning (or training) the network to reduce noise is done by presenting the network
a pixel P(corrupt, x, y) and all its 8 neighbors taken from the corrupted image and
"telling" it that the desired output for that specific input is the pixel P(source, x, y).
This is done for every pixel in every image in the training set (shown below). In our
project a pixel means a value in the range of [0, 255].
The process begins with writing a general neural network. The network is a 10-50-1,
i.e.:
1. The first layer is the input layer, the layer that will read 8 neighbors of a
given pixel and the pixel itself from the noisy image, and the same
coordinates pixel in the source image (10 neurons).
2. The second layer consists of 50 neurons that have 10 inputs each, form the
previous layer. Experiments showed that using 50 neurons for that layer is
sufficient.
3. The last layer is a single neuron that receives 50 inputs from the second layer
and simply calculates a weighted sum on its inputs.
So the network layout is:
Every neuron outputs its result through a sigmoid function. The sigmoid function has
the form: f(x) = 1 / (1 + e^-2.7x), as often used in NN (neural networks). In the
very same way we could use 5x5 or 7x7 as the neighbors instead of 3x3, which is
used in this project. Using MATLAB, the addition of Salt and Pepper noise to a given
image is as easy as calling the function imnoise, so we can easily produce a set of
training images: a clean source image and a corrupted noisy image with a density of
0.05. So part of that training set was:
The trick is to use different images as the learning set, meaning images that are
loosely related, and will represent a large percent of any image that can ever be
presented to the network to filter. Moreover, the order of the inputs at the learning
stage is also crucial, its highly recommended to show the network the training set
but to give it a random pixel from a random image at every iteration, so the network
wont converge too fast to a local optimum point, or ruining the fine-tuned weights
achieved from previous learned images.
Second stage / Filtering an image
When learning is over, the network is ready to start filtering images. As mentioned,
at first MATLAB was used to add noise to images so that the training set can be
created. MATLAB also dumped that noisy image to a simple file containing those 9
pixels from the corrupted along with the desired pixel from the clean one. Then I
used a java software to implement the network, and at the end the java program
produced a matrix of weights representing the neural network. Then I used MATLAB
again to receive a noisy image, the weights matrix from java and by simulating the
neural network operation, MATLAB filtered the noisy image. The only service used
directly from MATLAB was the imnoise function, all other operations were written as
part of the project.
Since MATLAB has a demo for filtering noise from images, it was a good idea to
incorporate the neural network algorithm into the existing demo so we make some
observations and comparisons.
Results
Running the network on images used in the MATLAB demo produced:
Some examples were run outside MATLAB, mainly to test some simple and
dichotomy images, for instance:
A dichotomy noisy image
Filtered image using NN
Conclusions
General
It is pretty obvious that the existing methods for filtering Salt and Pepper noise are
preferable than the one presented in this project. Although when presented a simple,
dichotomies image the NN method seems to produce a pretty good result.
Results improvement
The neural network for filtering noise can be improved for better performance by:
1. Fine tuning the middle layer – adding or removing neurons, testing images
and then deciding the optimized number of neurons.
2. Adding hidden layers – in this implementation, we have a single layer
between the input and the output layers. Adding another layer or two might
improve filtering dramatically.
3. Presenting the network a much larger and richer training set.
Analyzing the network
The method by which our network chose to filter a noisy image was simply to blur it
a little and to brighten it. All the above examples show clearly the higher level of
brightness the NN produces. In the example below we can observe that the NN was
also blurring the image as a mean for filtering:
When filtering is done right, we expect the distribution of the grayscale values of the
source (cleared) image not to be that different from the distribution of that in the
filtered image. A dichotomy image, much like those presented above, is filtered in a
reasonable way when comparing to other type of images, since the network
damaged the grayscale distribution less:
The histogram of a typical
image (the guitar above).
The histogram of a dichotomy
image (the bars above).
Grayscale histogram of the image as
produced by the NN. The damage is
pretty large.
The histogram the NN produced which
very similar to the source.
The grayscale was normalized to be in the range of [0, 1].
The resulting histogram of the grayscale
values.
Filtering noise from an image containing
dichotomy and gradient properties.
Future work
Since noise filtering methods operate on pixels in the image that are not necessarily
noise, they damage the image even in regions in the image that noise wasn't present
in the source image. A neural network approach for filtering noise can improve that;
we can build a neural network that specializes at recognizing noise. This kind of NN
is to have impressive performance since its output is a binary value (noise / nonoise) and network as such can specialized faster and easier. Given an image to be
filtered, the NN will show where the presence of noise is, and then we could apply
some method for filtering noise (e.g. Averaging, Adaptive, NN or others) that will
filter locally the noise. This way we avoid "filtering" noise that does not exist and by
this we damage the image minimally. Of course, the NN can also be wrong here and
then but the average percent of error is reasonably low when dealing with that kind
of network (provided the NN is well trained, which is mainly an issue of time).
References



Fundamentals of Neural Networks, Laurene Fausett
Building Neural Networks, David Skapura
A nonlinear filtering technique for digitized images degraded by film-grain
noise, Warren Willman
Download