Digital Image Processing
Chapter 5: Image Restoration
A Model of the Image
Degradation/Restoration Process

Degradation
⚫
⚫
⚫
Degradation function H
Additive noise  ( x, y)
Spatial domain
g ( x, y) = h( x, y) * f ( x, y) + ( x, y)
⚫
Frequency domain
G(u, v) = H (u, v) F (u, v) + N (u, v)

Restoration
g ( x, y )  Restoratio n Filter  fˆ ( x, y )
Noise Models

Sources of noise
⚫

White noise
⚫

Image acquisition, digitization,
transmission
The Fourier spectrum of noise is
constant
Assuming
⚫
⚫
Noise is independent of spatial
coordinates
Noise is uncorrelated with respect to
the image itself
Gaussian Noise


Random noise – affects both dark
and light areas of image
Gaussian noise
⚫
The PDF of a Gaussian random variable,
z,
1
− ( z −  ) 2 / 2 2
p( z ) =
e
2 
⚫
⚫
⚫
Mean: 
Standard deviation:
2
Variance: 


Rayleigh noise
⚫
The PDF of Rayleigh noise,
2
−( z − a ) 2 / b
 ( z − a )e
p( z ) =  b

0

⚫
⚫
Mean:
 = a+  b/4
Variance:
b( 4 −  )
 =
4
2
for z  a
for z  a

Erlang (Gamma) noise
⚫
The PDF of Erlang noise,
is a positive integer,
 a b z b −1 −a z

e
p( z ) =  (b − 1)!

0

b
=
⚫ Mean:
a
b
2
⚫ Variance:
 = 2
a
a0
,
for z  0
for z  0
b
Exponential noise- occurs due to
illumination changes

Exponential noise- occurs due to
⚫
The PDF of exponential noise,
ae − a z
p( z ) = 
 0
⚫
⚫
Mean:
Variance:
1
=
a
1
 = 2
a
2
a0
for z  0
for z  0
,

Uniform noise
⚫
The PDF of uniform noise,
 1

if a  z  b
p( z ) =  b − a

otherwise
 0
a+b
⚫ Mean:
=
2
2
(
b
−
a
)
2
⚫ Variance: 
=
12

Impulse (salt-and-pepper) noise
⚫
The PDF of (bipolar) impulse noise,
Pa

p( z ) =  Pb
0

⚫
⚫
for z = a
for z = b
otherwise
b  a : gray-level b
will appear as a
light dot, while level a will appear like
a dark dot
Unipolar: either Pa or Pb is zero
⚫
Usually, for an 8-bit image, a =0
(black) and b =255 (white)

Modeling
⚫
Gaussian

⚫
Rayleigh

⚫
Electronic circuit noise, sensor noise due
to poor illumination and/or high
temperature
Range imaging
Exponential and gamma

Laser imaging
⚫
Impulse

⚫
Quick transients, such as faulty switching
Uniform
Least descriptive
 Basis for numerous random number
generators


Periodic noise
⚫
⚫
⚫
Arises typically from electrical or
electromechanical interference during
the image capturing process
Reduced significantly via frequency
domain filtering
An image affected by periodic noise will
look like a repeating pattern has been
added on top of the original image.
Restoration in the Presence of Noise
Only-Spatial Filtering

Degradation
⚫
Spatial domain
g ( x, y) = f ( x, y) + ( x, y)
⚫
Frequency domain
G(u, v) = F (u, v) + N (u, v)

Mean filters
⚫
Arithmetic mean filter – removes
gaussian noise(Low pass filter)
ˆf ( x, y ) = 1
g ( s, t )

mn ( s ,t )S xy
⚫
Geometric mean filter – removes
1
gaussian noise

 mn
fˆ ( x, y ) =   g ( s, t )
( s ,t )S xy

⚫
Harmonic mean filter

Works well for salt noise, but fails fpr
pepper noise
fˆ ( x, y ) =
mn

( s ,t )S xy
1
g ( s, t )
⚫
Contraharmonic mean filter


Q0
Q0
: eliminates pepper noise
: eliminates salt noise
fˆ ( x, y ) =
Q +1
g
(
s
,
t
)

( s ,t )S xy
 g ( s, t )
( s ,t )S xy
Q

Usage
⚫
⚫
Arithmetic and geometric mean filters:
suited for Gaussian or uniform noise
Contraharmonic filters: suited for
impulse noise

Order-statistics filters
⚫
Median filter

Effective in the presence of both bipolar
and unipolar impulse noise
fˆ ( x, y ) = median{g ( s, t )}
( s ,t )S xy
⚫
Max and min filters

max filters reduce pepper noise
fˆ ( x, y ) = max {g ( s, t )}
( s ,t )S xy

min filters salt noise
fˆ ( x, y ) = min {g ( s, t )}
( s ,t )S xy
⚫
Midpoint filter

Works best for randomly distributed noise,
like Gaussian or uniform noise
ˆf ( x, y ) = 1  max {g ( s, t )} + min {g ( s, t )}

( s ,t )S xy
2 ( s ,t )S xy
⚫
Alpha-trimmed mean filter
Delete the d/2 lowest and the d/2 highest
gray-level values
 Useful in situations involving multiple
types of noise, such as a combination of
salt-and-pepper and Gaussian noise

fˆ ( x, y ) =
1
mn − d
 g ( s, t )
( s ,t )S xy
r

Adaptive, local noise reduction filter
2

⚫ If
 is zero, return simply the value
of
g ( x, y)
2
2
⚫ If 
, return a value close to
 L
g ( x, y)
2
2
⚫ If 
 =  L , return the arithmetic
mean value
mL
ˆf ( x, y ) = g ( x, y ) −   g ( x, y ) − m 
L
2
2
L

Adaptive median filter
⚫
⚫
⚫
⚫
⚫
z min
S xy
zmax
S xy
zmed
z xy
S max
= minimum gray level value in
= maximum gray level value in
= median of gray levels in S xy
= gray level at coordinates ( x, y)
= maximum allowed size of S xy
⚫
⚫
⚫
⚫
⚫
⚫
⚫
⚫
⚫
Algorithm:
Level A: A1= zmed − zmin
A2= zmed − zmax
If A1>0 AND A2<0, Go to
level B
Else increase the window size
If window size  S max
repeat level A
Else output zmed
⚫
⚫
⚫
⚫
Level B: B1= z xy − z min
B2= z xy − z max
If B1>0 AND B2<0, output
Else output zmed
z xy
⚫
Purposes of the algorithm
Remove salt-and-pepper (impulse) noise
 Provide smoothing
 Reduce distortion, such as excessive
thinning or thickening of object
boundaries

Periodic Noise Reduction by Frequency
Domain Filtering

Bandreject filters
⚫
Ideal bandreject filter
W

if D(u, v)  D 0 −
1
2

W
W

H (u, v) = 0 if D 0 −  D(u, v)  D 0 +
2
2

W
1
if D(u, v)  D 0 +

2

2
2 1/ 2
D(u, v) = (u − M / 2) + (v − N / 2)


⚫
Butterworth bandreject filter of order n
H (u, v) =
⚫
1
 D(u, v)W 
1+  2
2
 D (u, v) − D0 
Gaussian bandreject filter
H (u, v) = 1 − e
1  D 2 ( u ,v ) − D02 
− 

2  D ( u ,v )W 
2
2n

Bandpass filters
H bp (u, v) = 1 − H br (u , v)

Notch filters
⚫
Ideal notch reject filter
0 if D1 (u, v)  D 0 or D 2 (u, v)  D 0
H (u, v) = 
otherwise
1

D (u, v) = (u − M / 2 + u )

+ (v − N / 2 + v ) 
D1 (u , v) = (u − M / 2 − u0 ) + (v − N / 2 − v0 )
2
0
2
2
2 1/ 2
2 1/ 2
0

Butterworth notch reject filter of
order n
H (u, v) =
1


D
1+ 

 D1 (u, v) D2 (u, v) 
2
0
n

Gaussian notch reject filter
H (u, v) = 1 − e
1  D1 ( u ,v ) D2 ( u ,v ) 
− 

2
2 
D0


Notch pass filter
H np (u , v) = 1 − H nr (u , v)

Optimum notch filtering
⚫
Interference noise pattern
N (u, v) = H (u, v)G(u, v)
⚫
Interference noise pattern in the spatial
domain
 ( x, y) =  {H (u, v)G(u, v)}
−1
⚫
Subtract from g ( x, y) a weighted
portion of  ( x, y) to obtain an
estimate of f ( x, y)
fˆ ( x, y ) = g ( x, y ) − w( x, y ) ( x, y )
⚫
⚫
⚫
Minimize the local variance of fˆ ( x, y )
The detailed steps are listed in Page
251
Result
w( x, y ) =
g ( x, y ) ( x, y ) − g ( x, y ) ( x, y )
 2 ( x, y ) −  2 ( x, y )
Linear, Position-Invariant Degradations

Input-output relationship
g ( x, y) = H [ f ( x, y)] + ( x, y)
 ( x, y) = 0
g ( x, y) = H [ f ( x, y)]

H is linear if
H [af1 ( x, y ) + bf 2 ( x, y )]
= aH [ f1 ( x, y )] + bH [ f 2 ( x, y )]
⚫
Additivity
H [ f1 ( x, y ) + f 2 ( x, y )]
= H [ f1 ( x, y )] + H [ f 2 ( x, y )]
⚫
Homogeneity
H [af1 ( x, y)] = aH[ f1 ( x, y)]
⚫
Position (or space) invariant
H[ f ( x −  , y −  )] = g ( x −  , y −  )]
⚫
In terms of a continuous impulse
function
f ( x, y ) = 



− −
f ( ,  ) ( x −  , y −  )d d
g ( x, y ) = H [ f ( x, y )]
 

= H   f ( ,  ) ( x −  , y −  )d d 
 − −

g ( x, y )
=


= 
= 

− −


− −


− −
H  f ( ,  ) ( x −  , y −  )d d
f ( ,  ) H  ( x −  , y −  )d d
f ( ,  )h( x,  , y,  )d d
⚫
Impulse response of H
h( x,  , y,  ) = H [ ( x −  , y −  )]
⚫
⚫
In optics, the impulse becomes a point
of light
Point spread function (PSF)
h( x,  , y,  )
⚫
All physical optical systems blur
(spread) a point of light to some
degree
⚫
Superposition (or Fredholm) integral of
the first kind
g ( x, y ) =

 

− −
f ( ,  )h( x,  , y,  )d d
⚫
If H is position invariant
H [ ( x −  , y −  )] = h( x −  , y −  )
⚫
Convolution integral
g ( x, y ) =

 

− −
f ( ,  )h( x −  , y −  )d d

In the presence of additive noise
g ( x, y ) =

 

− −
⚫
f ( ,  )h( x,  , y,  )d d +  ( x, y )
If H is position invariant
g ( x, y ) =

 

− −
f ( ,  )h( x −  , y −  )d d +  ( x, y )
⚫
If H is position invariant
g ( x, y) = h( x, y)  f ( x, y) + ( x, y)
G(u, v) = H (u, v) F (u, v) + N (u, v)
⚫
Restoration approach
Image deconvolution
 Deconvolution filter

Estimating the Degradation Function

Estimation by image observation
⚫
In order to reduce the effect of noise in
our observation, we would look for
areas of strong signal content
Gs (u, v)
H s (u, v) =
Fˆs (u, v)

Estimation by experimentation
⚫
Obtain the impulse response of the
degradation by imaging an impulse
(small dot of light) using the same
system settings
G (u, v)
H (u, v) =
A
⚫
⚫
Observed image G(u, v)
The strength of the impulse
A

Estimation by modeling
⚫
⚫
Hufnagel and Stanley
Physical characteristic of atmospheric
turbulence
H (u, v) = e
5
−k (u 2 +v 2 ) 6
⚫
Image motion
T
g ( x, y ) =  f [ x − x0 (t ), y − y0 (t )]dt
0
G (u, v) = 



− −
g ( x, y ) e
− j 2 ( u x + v y )
dx dy
 f [ x − x (t ), y − y (t )]dt 
0
0


−  −  
0

− j 2 ( u x + v y )
e
dx dy
=


T
=     f [ x − x0 (t ), y − y0 (t )]
0 
 − −
− j 2 ( u x + v y )
e
dx dy dt
T



T
G (u , v) =  F (u , v)e
− j 2 [ u x0 ( t ) + v y0 ( t )]
0
T
= F (u , v)  e
− j 2 [ u x0 ( t ) + v y0 ( t )]
0
dt
= F (u , v) H (u , v)
⚫
Where
T
H (u, v) =  e
0
− j 2 [ u x0 ( t ) + v y0 ( t )]
dt
dt
⚫
If
x0 (t ) = at / T and
T
H (u, v) =  e
y 0 (t ) = 0
− j 2 [ u x0 ( t )]
0
T
= e
0
− j 2 [ u at / T ]
dt
dt
T
− j ua
=
sin( ua)e
 ua
⚫
If
x0 (t ) = at / T and y0 (t ) = bt / T
H (u, v)
T
− j ( ua + vb)
=
sin[ (ua + vb)]e
 (ua + vb)
Inverse Filtering

Direct inverse filtering
⚫
Limiting the analysis to frequencies
near the origin
G (u , v)
ˆ
F (u , v) =
H (u , v)
N (u , v)
= F (u, v) +
H (u, v)
Minimum Mean Square Error (Wiener)
Filtering

Minimize
2
ˆ
e = E{( f − f ) }
2

Terms
⚫ H (u, v) = degradation function
⚫
⚫
⚫
⚫
⚫
H  (u, v) = complex conjugate of
2
H (u , v ) = H  (u, v) H (u, v)
H (u, v)
S (u, v) = N (u , v) = power spectrum
2
⚫
of the noise
⚫
S f (u , v) = F (u, v)
2
= power spectrum
of the undegraded image
Wiener filter


H
*
(
u
,
v
)
S
(
u
,
v
)
f
G (u, v)
Fˆ (u, v) = 
2
 S f (u, v) H (u, v) + S (u, v) 



H * (u, v)
G (u, v)
=
2
 H (u, v) + S (u, v) / S f (u, v) 
2
 1

H (u, v)
G (u, v)
=
2
 H (u, v) H (u, v) + S (u, v) / S f (u, v) 
⚫
White noise
2
 1

H
(
u
,
v
)
Fˆ (u, v) = 
G (u, v)
2
 H (u, v) H (u, v) + K 
Constrained Least Squares Filtering

Vector-matrix form
g ( x, y) = h( x, y) * f ( x, y) + ( x, y)
⚫
⚫
⚫
g = Hf + η
g , f , η : MN 1
H : MN  MN

Minimize
M −1 N −1

C =   f ( x, y )
2
x =0 y =0
⚫
Subject to
2
2
ˆ
g − Hf = η

2

The solution


H
*
(
u
,
v
)
Fˆ (u, v) = 
G (u, v)
2
2
 H (u, v) +  P(u, v) 
⚫
Where P(u, v) is the Fourier transform
of the function
 0 −1 0 
P( x, y ) = − 1 4 − 1
 0 − 1 0 

Computing  by iteration
r = g − Hfˆ
⚫
Adjust

so that
r = η a
2
2
⚫
Computation
M −1 N −1
r =  r 2 ( x, y )
2
x =0 y =0
1
m =
MN
M −1 N −1
1
 =
MN
M −1 N −1
2
 ( x, y)
x =0 y =0
  ( x, y) − m 
x =0 y =0
η = MN [ 2 + m2 ]
2
2
⚫
⚫
⚫
⚫
⚫
Algorithm
1: Specify an initial value of
2: Compute
2
2
3: Stop if r = η  a is satisfied;
otherwise return to Step 2 after
2
2
increasing  if
r  η − a or
decreasing

if
r  η +. a
2
2
Geometric Mean FIlter
Fˆ (u , v) =




 H * (u , v)  
H * (u, v)


 
2



S
(
u
,
v
)
 H (u , v)  
2

H (u , v) +  



S
(
u
,
v
)


f




1−
G (u , v)
Geometric Transformations

Spatial transformations
x' = r ( x, y)
y' = s( x, y)
⚫
Tiepoints
⚫
Bilinear equations
x' = r ( x, y) = c1 x + c2 y + c3 xy + c4
y' = s( x, y) = c5 x + c6 y + c7 xy + c8

Gray-level interpolation
v( x' , y' ) = ax'+by'+cx' y'+d