INF-VERK 3830/4830 project assignment: Image
denoising using simple diffusion operators
Image denoising refers to the removal of noises from a noise-contaminated
image, such that the restored image becomes closer to the original noise-free
image. Since noisy images are present in many real-life situations, image denoising has become an important task in modern use of computers. An example
of image denoising is illustrated in Figure 1 below.
Figure 1: Left: a noisy image; Light: a denoised image after applying a simple
diffusion operator.
The purpose of this project assignment is to let the students get familiarized
with the following important aspects of scientific programming:
1. Translation of simple mathematical formulas to a working code.
2. Input and output of data files in both ascii and binary format.
1
3. Simple program structure design via object orientation.
4. Compilation and use of external software libaries.
1
Introduction
An image can be thought as a mathematical function u(x, y), which only has
discrete values on a rectangle of pixels. For an image with m × n pixels, this
means that u has the following values


u(m, 1) u(m, 2) · · · u(m, n)


..
..
..
..


.
.
.
.


 u(2, 1) u(2, 2) · · · u(2, n) 
u(1, 1) u(1, 2) · · · u(1, n)
For simplicity, the discrete values of u(x, y) can also be denoted by u1,1 , u1,2 , . . . , um,n .
By a noisy image we mean that
u(x, y) = s(x, y) + f (x, y),
(1)
where s(x, y) is the original image without noise, whereas f (x, y) is the noisy
addition having a random characteristic. The purpose of image denoising is
to restore s(x, y) from u(x, y), on the basis of some (limited) knowledge of the
random characteristic of f (x, y). Often, the noisy addition f (x, y) is a fastoscillating random function, so the contaminated image u(x, y) displays a noisy
feature.
Image denoising is a non-trivial task, because we at the best only have some
partial description of f (x, y). (For example, when f (x, y) is a Gaussian noise
we may be lucky to know its mean value and the standard deviation.)
2
Denoising by Diffusion
There exist many image denoising methods, with varying degrees of complexity
and effectiveness. We will in this project only concentrate on a simple class of
denoising methods that make use of diffusion operators.
The simplest diffusion-based denoising method is defined as follows:
ū(x, y) = u(x, y) + ∆t∇2 u(x, y),
(2)
where ū(x, y) denotes the resulting denoised image, ∆t is a prescribed constant,
and ∇2 u(x, y) denotes the Laplacian of the noisy image u(x, y). More specifically,
∇2 u = ∇ · (∇u) =
2
∂2u ∂2u
+ 2.
∂x2
∂y
(3)
Note that the result of the³Laplacian
´ can be equally achieved by first computing
∂u ∂u
the gradient vector ∇u = ∂x , ∂y , and then taking the divergence (∇ · ) of the
gradient vector.
The hope is that ū, arising from formula (2), becomes a better representation
of the original image s. This is because the above denoising method is able
to “smooth out” the noises to a certain extent. In real-life applications, the
operation of formula (2) needs to be repeated several times to achieve a desirable
denoising effect. That is,
For i = 1, 2, . . . , until a given number of iterations:
Apply one iteration of denoising: ū(x, y) = u(x, y) + ∆t∇2 u(x, y)
Copy ū(x, y) into u(x, y):
u(x, y) = ū(x, y)
Go back and repeat
The above image denoising strategy can be called isotropic diffusion because
the amount of diffusion happens isotropically, independent of the spatial location
(x, y). A replacement of formula (2) thus arises from using anisotropic diffusion
as follows:



ū(x, y) = u(x, y) + ∆t∇ · 
3
∇u

¡ ∂u ¢2 ³ ∂u ´2  .
1 + ∂x + ∂y
(4)
Numerical Differentiation
To implement the two image denoising strategies described above, we have to
be able to compute the partial derivatives of u(x, y) numerically, because the u
function only has values on the discrete pixels.
Recall that we have the m×n discrete values of u(x, y) as u1,1 , u1,2 , . . . , um,n .
The following finite difference formulas should be used to compute the needed
numerical partial derivatives:

∇2 u(xi , yj )
∂u
(x1 , yj )
∂x
∂u
(xi , yj )
∂x
∂u
(xm , yj )
∂x
∂u
(xi , y1 )
∂y
∂u
(xi , yj ),
∂y
∂u
(xi , yn )
∂y
≈
ui,j−1 + ui−1,j − 4ui,j + ui+1,j + ui,j+1 ,
0,
≈
u2,j − u1,j ,
1≤j≤n
≈
ui+1,j − ui−1,j
,
2
≈
um,j − um−1,j ,
≈
ui,2 − ui,1 ,
≈
ui,j+1 − ui,j−1
,
2
≈
ui,n − ui,n−1 ,
2 ≤ i ≤ m − 1, 2 ≤ j ≤ n − 1
(5)
otherwise
(one-sided difference)
2 ≤ i ≤ m − 1, 1 ≤ j ≤ n
1≤j≤n
1≤i≤m
(centered difference)
(one-sided difference)
(one-sided difference)
1 ≤ i ≤ m, 2 ≤ j ≤ n − 1
1≤i≤m
3
(centered difference)
(one-sided difference)
(6)
(7)
(8)
(9)
(10)
(11)
It is clear that the numerical Laplacian needed by formula (2) is given
straightforwardly by formula (5). However, the computation needed in formula
(4) should be carried out by first computing an intermediate discrete vector
function (gx , gy ) as
gx (x, y)
=
1+
gy (x, y)
∂u
∂x
¡ ∂u ¢2
+
∂x
∂u
∂y
=
1+
¡ ∂u ¢2
∂x
+
³
³
∂u
∂y
∂u
∂y
´2 ,
´2 ,
and then completing the compuation of formula (4) as
µ
¶
∂gy
∂gx
ū(x, y) = u(x, y) + ∆t
+
.
∂x
∂y
4
The Project Assignment
The above two diffusion-based image denoising strategies should be implemented
in the project assignment, together with supporting functionality. The implementation should be done in both C and C++, where the C++ version is
basically an object-oriented redesign of the C version.
4.1
The C Implementation
The following data structure should be adopted for representing an image with
m × n pixels:
struct image
{
float** image_data;
int m;
int n;
};
/* a 2D array of float */
/* # pixels in x-direction */
/* # pixels in y-direction */
Based on struct image, the following three functions should be implemented
for memory allocation (of the image data array), memory deallocation, and copy
from an image to another, respectively:
void allocate_image (image *u, int m, int n);
void deallocate_image (image *u);
void copy_image (const image *a, image *b); /* copy from a to b */
Moreover, the following functions should be implemented for reading and
writing an image from/to data file:
void
void
void
void
read_ascii_file (const char* filename, image* a);
write_ascii_file (const char* filename, const image* a);
read_binary_file (const char* filename, image* a);
write_binary_file (const char* filename, const image* a);
4
where the format of an ascii image file is assumed to be as follows:
m=xxx
n=xxx
u(1,1)=xxx
u(1,2)=xxx
u(1,3)=xxx
...
The format of a binary image file should be determined by the student herself/himself.
Finally, the following two functions should implement the isotropic and
anisotropic diffusion-based image denoising strategies, respectively:
void iso_duffsion_denoising (image* u, image* u_bar, float dt, int num_iters);
void ani_duffsion_denoising (image* u, image* u_bar, float dt, int num_iters);
All the above C fucntions are considered by the project as the C version of
a simple user-defined library for image-denoising.
4.2
The C++ Implementation
A C++ class with name Image should be implemented to replace the C data
structure struct image. This new C++ class should also incorporate similar
member functions as the above C functions (allocate image, deallocate image,
copy image, read ascii file, write ascii file, read binary file, write binary file).
Moreover, another C++ class with name DiffusionDenoising should be implemented with the following structure:
class DiffusionDenoising
{
protected:
int num_iters;
float dt;
public:
DiffusionDenoising ();
~DiffusionDenoising ();
setNumIters (int iters);
setDt (float dt);
virtual void denoising (Image& u, Image& u_bar) =0;
};
Note that class DiffusionDenoising is to act as a pure virtual base class,
whose member function denoising has to be explicitly implemented in its subclasses. As two concrete subclasses, class IsotropicDiffusionDenoising and
class AnisotropicDiffusionDenoising should be develped, each having a concrete implementation of the member function denoising.
The above four C++ classes thus constitute the C++ version of a simple
user-defined library for image-denoising.
5
4.3
Main Program in C
A main program should be implemented in C to test all the functions belonging
to the C version of the user-defined library. The following ascii file contains a
100 × 100 pixel image s(x, y) without noise:
http://heim.ifi.uio.no/~xingca/inf-verk3830/noise_free_image.txt
To generate a noisy image u(x, y) = s(x, y) + f (x, y), the standard C function
rand should be used to add a random noise of range −20 ≤ f (x, y) ≤ 20 on
top of s(x, y). Note that each call of rand returns a random number between
0 and RAND MAX, so some kind of scaling and shifting must be applied to obtain
random values between -20 and 20 (see more info by e.g. google rand function).
When u(x, y) is generated, five iterations of both the isotropic and anisotropic
denoising strategies should be carried out, with ∆t = 0.5. The five denoising
iterations should be carried out one by one, so that the deviation from ū(x, y) to
the original noise-free image s(x, y) can be checked (by computing the Euclidean
norm of [ū1,1 − s1,1 , ū1,2 − s1,2 , . . . ūm,n − sm,n ]).
The main program in C should also write one of the denoised images to a
binary data file (using function write binary file). This binary data file should
then be read in (using function read binary file) to make sure that the two
functions and the format of the binary data file works as intended.
4.4
Main Program in C++ Using an External Library
Another C++ main program, making use of the C++ version of the user-defined
library, should also be developed. All the actions implemented in the C main
should be repeated.
In addition, a real-life noisy image (the left picture of Figure 1) should be
tested. However, this is a JPEG formatted image,
http://heim.ifi.uio.no/~xingca/inf-verk3830/noisy-paprika.jpg
which can not be read directly by either read ascii file or read binary file.
To enable import and export of JPEG formatted images, an extenal C library
package is to be used. More specifically, the following gzipped tar file
http://heim.ifi.uio.no/~xingca/inf-verk3830/simple-jpeg.tar.gz
should be downloaded and packed out under a separate directory (e.g. with
directory name external code). It will be seen that the external C library package contains a set of header files and C files. A makefile should be written
so that all the C files are compiled as an external library file (e.g. with name
libsimplejpeg.a). Only two functions within the external library are meant for
use in the project:
void import_JPEG_file (const char* filename, unsigned char** image_chars,
int* image_height, int* image_width,
int* num_components);
void export_JPEG_file (const char* filename, const unsigned char* image_chars,
int image_height, int image_width,
int num_components, int quality);
6
These two functions can be used, respectively, to read and write a data file of
the JPEG format. We remark that each pixel in a grey-scale JPEG image uses
one byte, and a one-dimensional array of unsigned char (of total length mn) is
used to contain all the pixel data of a grey-scale JPEG image. (In the case of
a color JPEG image, a 1D unsigned char* array of rgbrgbrgb... values is read
in.) This means that a conversion must be carried out to translate from a 1D
unsigned char* array to a 2D float** array, required by our denoising functions.
We remark also that a pixel in a JPEG image can only have value between 0
and 255. Moreover, the integer variable num components will contain value 1 after
the import JPEG file function finishes reading a grey-scale JPEG image. The
value 1 should also be given to num components when invoking export JPEG file
to export a grey-scale JPEG image. We remark that the last argument quality
of the export JPEG file function is an integer indicating the compression level
of the resulting JPEG image. A value of 75 is the typical choice of quality.
Once the external library libsimplejpeg.a is ready compiled, it can be used
in the C++ main program as follows:
#include <Image.h>
#include <IsotropicDiffusionDenoising.h>
extern "C" {
void import_JPEG_file(const char *filename, unsigned char** image_c,
int* m, int* n, int *k);
void export_JPEG_file(const char * filename, unsigned char* image_c,
int m, int n, int k, int quality);
}
int main (int nargs, const chars* args)
{
// repeat the tasks in the C main program
int m,n,k,i,j;
unsigned char* image_chars; // 1 byte per pixel in a grey JPEG image
// read from file a JPEG formatted image
import_JPEG_file ("noisy-paprika.jpg", &image_chars, &m, &n, &k);
//
//
//
//
create u and u_bar objects with dimension [m,n]
convert data inside ’image_chars’ into the ’image_data’ array of u
create an object of IsotropicDiffusionDenoising and call ’denoising’
convert the ’image_data’ array of u_bar back to ’image_chars’
// write the denoised image to a new JPEG formatted image file
export_JPEG_file ("denoised-paprika.jpg", image_chars, m, n, k, 75);
}
4.5
return 0;
Submission
Each student should independently work with the project assignment. Before
the deadline, Friday May 25, 2007 (12:00 o’clock), each student should submit
a gzipped tar-file (using commands tar and gzip) with the following content:
README
src/C
src/C++
src/external_code
7
test/C
main.c
make.sh
makefile
test/C++
main.cpp
make.sh
makefile
clean.sh
The README file should contain a simple overview of all the files submitted. The
source code of the C and C++ user-defined libraries goes into the src/C and
src/C++ directories, respectively. The src/external code directory contains the
external C library package for handling JPEG formatted image files. The two
test directories test/C and test/C++ contain the main programs and have either
a make.sh or a makefile. The latter is a true makefile to be used with the make
command, while the former is a simple shell script that builds the user-defined
library, and compiles and links the main program. The clean.sh file is a shell
script for traversing the entire directory tree and removing all binary files (which
can easily be regenerated). The relevant Unix command for clean.sh is
find . \( -name ’*.o’ -o -name ’*.a’ -o -name ’*.so’ \) \
-print -exec rm -rf {} \;
Remember to run clean.sh before you pack the tar file and submit it to the
teacher! (Otherwise big object and library files make the tar file much larger
than necessary.)
8