Image Analysis
Image Restoration
Image Restoration
g ( x, y )  h( x, y) * f ( x, y)  n( x, y)
Image enhancement tries to improve subjective image quality.
Image restoration tries to recover the original image.
Bahadir K. Gunturk
EE4780
2
Noise Models
• Noise may arise during image due to sensors, digitization, transmission, etc.
• Most of the time, it is assumed that noise is independent of spatial
coordinates, and that there is no correlation between noise component and
pixel value.
• Noise may be considered as a random variable, its statistical behavior is
characterized by a probability density function (PDF).
Gaussian noise
1
p( z ) 
e
2
Bahadir K. Gunturk
2
z 


2 2
EE4780
3
Noise Models
Uniform noise
Bahadir K. Gunturk
Impulse (salt-and-pepper) noise
EE4780
4
Noise Models
Original
Noisy images and
their histograms
Bahadir K. Gunturk
EE4780
5
Noise Models
How to estimate noise parameters?
• If imaging device is available
• Take a picture of a flat surface.
• See the shape of the histogram; decide on the noise model.
• Estimate the parameters. (e.g., find mean and standard deviation.)
• When only images already generated are available
• Get a small patch of image with constant gray level
• Inspect histogram
• Estimate the parameters
Bahadir K. Gunturk
EE4780
6
Restoration When There is Only Noise
g ( x , y )  f ( x , y )  n ( x, y )
Low-Pass Filters:
• Smoothes local variations in an image.
• Noise is reduced as a result of blurring.
• For example, Arithmetic Mean Filter is
ˆf ( x, y )  1
g ( s, t )

mn ( s ,t )N xy
Bahadir K. Gunturk
EE4780
 Convolve with a uniform
filter of size m-by-n.
7
Restoration When There is Only Noise
Adaptive, 2local noise reduction filter
• Let  n ( x, y ) be the noise variance at (x,y).
• Let  L2 ( x, y ) be the local variance of pixels around (x,y).
mL be the local mean of pixels around (x,y).
• Let
We want a filter such that
• If noise variance is zero, it should return g(x,y).
• If local variance is high relative to noise variance, the filter should
return a value close to g(x,y). (Therefore, edges are preserved!)
• If two variances are equal, the filter should return the average of
the pixels within the neighborhood.
2

ˆf ( x, y )  g ( x, y )  n ( x, y)  g ( x, y )  m 
L
 L2 ( x, y)
Bahadir K. Gunturk
EE4780
8
Restoration When There is Only Noise
Bahadir K. Gunturk
EE4780
9
Restoration When There is Only Noise
Median Filter
• Replaces the value of a pixel by the median of intensities in the
neighborhood of that pixel.
• Is very effective against the salt-and-pepper noise.
Bahadir K. Gunturk
EE4780
10
Restoration When There is Only Noise
Adaptive Median Filter: The basic idea is to avoid extreme values
• Let
• z_min: minimum gray level value in a neigborhood of a pixel at (x,y).
• z_max: maximum gray level value…
• z_med: median…
• z(x,y): gray level at (x,y).
• Is z_med=z_min or z_med=z_max? (That is, is z_med an extreme value?)
• No:
• Is z(x,y) an extreme value? (Is z(x,y)=z_min or z(x,y)=z_max?)
• No: Output is z(x,y)
• Yes: Output is z_med.
• Yes: Increase window size (to find a non-extreme z_med) and go to the
first step. (When a maximum allowed window size is reached, stop and
output z(x,y).)
Bahadir K. Gunturk
EE4780
11
Restoration When There is Only Noise
Bahadir K. Gunturk
EE4780
12
Restoration When There is Only Noise
Removing Periodic Noise with Band-Reject Filters
Spikes are due to noise
Periodic Noise
Band-reject filter
Bahadir K. Gunturk
EE4780
13
Restoration When There is Only Noise
Finding Periodic Noise from the Spectrum and Using Notch Filters
Filter out these
spikes
Noise due to
interference
Bahadir K. Gunturk
EE4780
14
Image Restoration
Spatial domain:
g ( x, y )  h( x, y) * f ( x, y)  n( x, y)
Frequency domain:
G (u, v)  H (u, v) F (u, v)  N (u, v)
Bahadir K. Gunturk
EE4780
15
Image Restoration
Inverse Filtering
G (u, v)  H (u, v) F (u, v)  N (u, v)
Fˆ (u, v) 
Bahadir K. Gunturk
1
N (u, v)
G(u, v)  F (u, v) 
H (u, v)
H (u, v)
EE4780
This could dominate
signal.
16
Image Restoration
Bahadir K. Gunturk
EE4780
17
Image Restoration
Cut off the inverse filter for large
frequencies. (Signal-to-noise
ratio is typically low for large
frequencies.)
Bahadir K. Gunturk
EE4780
18
Image Restoration
Minimum Mean Square Error (Wiener) Filtering:
• Find Fˆ (u , v) such that the expected value of error is minimized:

Error  E Fˆ (u, v)  F (u, v)

2
• Solution is


2
H
(
u
,
v
)
1

Fˆ (u, v)  
2
H (u, v)
N
(
u
,
v
)
2

H (u , v) 
2

F
(
u
,
v
)




 G (u, v)



Investigate this equation for different signal-noise ratios.
Bahadir K. Gunturk
EE4780
19
Original image
Bahadir K. Gunturk
EE4780
20
Image Restoration
Bahadir K. Gunturk
EE4780
21
Least Squares Filtering
G (u, v)  H (u, v) F (u, v)  N (u, v)
Find F(u,v) that minimizes the following cost function:
Cost  G (u , v)  H (u , v) F (u , v)
2
The solution is
 H * (u, v) 
Fˆ (u, v)  
G(u, v)
2
 H (u, v) 
(Unconstrained
solution)
(See the derivations in the classroom)
Bahadir K. Gunturk
EE4780
22
Least Squares Filtering
G (u, v)  H (u, v) F (u, v)  N (u, v)
Find F(u,v) that minimizes the following cost function:
Cost  G (u , v)  H (u, v) F (u, v)   P(u, v) F (u, v)
2
2
Choose a P(u,v) to have a smooth solution. (A high-pass
filter would do the trick.)
Bahadir K. Gunturk
EE4780
23
Least Squares Filtering
The frequency domain solution to this optimization problem is
2
 1

H
(
u
,
v
)
Fˆ (u, v)  
G(u, v)
2
2
 H (u, v) H (u, v)   P(u, v) 
(Constrained
solution)
where P(u,v) is the Fourier Transform of p(x,y), which is typically chosen
as a high-pass filter.
Example:
Bahadir K. Gunturk
 0 1 0 
p( x, y )   1 4 1
 0 1 0 
EE4780
24
Least Squares Filtering
Bahadir K. Gunturk
EE4780
25