Yao Wang
Polytechnic University, Brooklyn, NY11201 http://eeweb.poly.edu/~yao

Outline
• Noise removal by averaging filter
• Noise removal by median filter
• Sharpening (Edge enhancement)
• Deblurring
©Yao Wang, 2006 EE3414: Image Filtering 2

Noise Removal (Image Smoothing)
• An image may be “dirty” (with dots, speckles,stains)
• Noise removal:
– To remove speckles/dots on an image
– Dots can be modeled as impulses (salt-and-pepper or speckle) or continuously varying (Gaussian noise)
– Can be removed by taking mean or median values of neighboring pixels (e.g. 3x3 window)
– Equivalent to low-pass filtering
• Problem with low-pass filtering
– May blur edges
– More advanced techniques: adaptive, edge preserving
©Yao Wang, 2006 EE3414: Image Filtering 3

Example
©Yao Wang, 2006 EE3414: Image Filtering 4

Averaging Filter
• Replace each pixel by the average of pixels in a square window surrounding this pixel
• Trade-off between noise removal and detail preserving:
– Larger window -> can remove noise more effectively, but also blur the details/edges
©Yao Wang, 2006 EE3414: Image Filtering 5

Example: 3x3 average
100
100
100
100
100
100 100
200 205
195 200
200 205
100 100
100
203
200
195
100
100
100
100
100
100
©Yao Wang, 2006 EE3414: Image Filtering
100
100
100
100
100
100 100
144 167
167 200
144 166
100 100
100
145
168
144
100
100
100
100
100
100
6

Example
©Yao Wang, 2006 EE3414: Image Filtering 7

Weighted Averaging Filter
• Instead of averaging all the pixel values in the window, give the closer-by pixels higher weighting, and far-away pixels lower weighting. g ( m , n ) = l =
L
− L k =
L
− L h ( k , l ) s ( m − k , n − l )
• This type of operation for arbitrary weighting matrices is generally called “2-D convolution or filtering”. When all the weights are positive, it corresponds to weighted average.
• Weighted average filter retains low frequency and suppresses high frequency = low-pass filter
©Yao Wang, 2006 EE3414: Image Filtering 8

Graphical Illustration
©Yao Wang, 2006 EE3414: Image Filtering 9

Example Weighting Mask
1
9
×
1
1
1
1
1
1
1
1
1
1
16
×
1
2
1
2
4
2
1
2
1
©Yao Wang, 2006
All weights must sum to one
EE3414: Image Filtering 10

Example: Weighted Average
1
9
×
1
1
1
1
1
1
1
1
1
100
100
100
100
100
100 100
200 205
195 200
200 205
100 100
100
203
200
195
100
100
100
100
100
100
100
100
100
100 100
144 167
167 200
100
145
168
100
100
100
100
100
144 166
100 100
144
100
100
100
©Yao Wang, 2006 EE3414: Image Filtering
1
16
×
1
2
1
2
4
2
1
2
1
100
100
100
100
100
100 100
156 176
174 201
156 175
100 100
100
158
175
156
100
100
100
100
100
100
11

Filtering in 1-D: a Review
• Continuous-Time Signal
Time Domain : g ( t ) =
Frequency Domain : G ( s ( t ) ∗ h ( t ) f ) = S ( f
=
) H
∞ ∫
− ∞
( f h ( τ
)
) s ( t
Filter frequency response : H ( f ) =
− τ ) d τ
∫
−
∞
∞ h ( t ) e − j 2 π ft dt
• Extension to discrete time 1D signals
Time Domain (linear convolutio n) : s ( n ) = s ( n ) ∗ h ( n ) = ∑ m h ( m ) s ( n
Frequency Domain : G ( f ) = S ( f ) H ( f )
Filter frequency response (DTFT) : H ( f
H ( f
) = ∑ n h ( n ) e − j 2 π fn
) is periodic, only needs to look at the range f ∈ (1 / 2 , 1 / 2 )
− m )
1/2 correspond s to f s
/ 2
©Yao Wang, 2006 EE3414: Image Filtering 12

Filtering in 2D
Weighted averaging = 2D Linear Convolutio n g ( m , n ) = l l
1 ∑
= l
0 k k
1
=
∑
0 k h ( k , l ) s ( m − k , n − l )
In 2D frequency domain G ( f
1
, f
2
) =
Frequency response of the 2D Filter
S ( f
1
, f
2
) H ( f
1
, f
2
)
H ( f
1
, f
2
) = m l
1 ∑
= l
0 n k
1
=
∑
0 k h ( m , n ) e − j 2 π ( f
1 m + f
2 n )
H ( f
1 f
1
, f
2
∈ ( − 1 /
) is periodic, only needs to look at the square region
2 , 1 / 2 ), f
2
∈ ( − 1 / 2 , 1 / 2 ).
©Yao Wang, 2006 EE3414: Image Filtering 13

Frequency Response of Averaging
Filters
• Averaging over a 3x3 window h ( m , n ) =
1
9
 0


0


0


0
0
0
1
1
1
0
0
1
1
1
0
0
1
1
1
0
0 
0


0 

0
0


+
=
=
+
H (
1
9 f
1
, f
2
) =
1
9
( e − j 2 π ( f
1
⋅ ( − 1 ) + f
2
⋅ ( − 1 ))
( e − j 2 π ( f
1
⋅ ( − 1 ) + f
2
⋅ ( 0 ))
+ e − j 2 π ( f
1
⋅ ( 0 ) + f
2
⋅ ( − 1 ))
+ e − j 2 π ( f
1
⋅ ( 0 ) + f
2
⋅ ( 0 )) + e − j 2 π ( f
1
⋅ ( 1 ) +
+ e − j 2 π ( f
1
⋅ ( 1 ) + f
2
⋅ ( 0 ))
) f
2
⋅ ( − 1 ))
)
1
9
1
9
1
9
(
(
( e − j 2 π ( f
1
⋅ ( − 1 ) + f
2
⋅ ( 1 )) e e j 2 π f
1 j 2 π f
1
+ 1 + e − j 2 π f
1
+ 1 + e − j 2 π f
1
+ e − j 2 π ( f
1
⋅ ( 0 ) + f
2
⋅ ( 1 ))
)
)( e e j 2 π f
2 j 2 π f
2
+
1
9
( e j 2 π f
1
+ e − j 2 π ( f
1
⋅ ( 1 ) + f
2
⋅ ( 1 ))
+ 1 + e − j 2 π f
1
+ 1 + e − j 2 π f
2
)
=
1
9
(
1 +
⋅
)
)
1 +
1
9
( e j 2 π f
1
+ 1 + e − j 2 π f
1
2 cos 2 π f
1
)(
1 + 2 cos 2 π f
2
)
) e − j 2 π f
2
©Yao Wang, 2006 EE3414: Image Filtering 14

H ( f
1
, f
2
) = H ( f
1
) H ( f
2
)
H ( f
1
) =
1
3
(
1 + 2 cos 2 π f
1
)
, H ( f
2
) =
1
3
(
1 + 2 cos 2 π f
2
)
Sketch H(f1)
1
0.5
0
-0.5
30
20
10
0 0
5
10
15
20
25
©Yao Wang, 2006 EE3414: Image Filtering 15

Frequency Response of Weighted
Averaging Filters
H =
H ( u , v )
1
( 1 + b ) 2
=
( b +



1 b

 1
2 b b b
2
1 b



1 
 cos( 2 π u
=
)
)( b
( 1
+
+
1
2 b ) 2 cos(



1 b

 1
2 π v




 [
1
)
) b
/( 1 +
1 ;
] b ) 2
©Yao Wang, 2006
1
0.8
0.6
0.4
0.2
0
30
20
10
0 0
5
10
15
20
25
EE3414: Image Filtering 16

Averaging vs. Weighted Averaging
H
H
=
( u ,
1
9 v )


1

1
 1
=
1
1
1
(
1 +
1
1



1 
2
= cos(




1
1
1




2 π
[
1 u )
)(
1
1 +
1
]
2
; cos( 2 π v )
)
/ 9
H =
( 1 +
H ( u , v )
1
= b ) 2
( b +


1
 b
 1
2 b b b
2
1 b



1  cos( 2 π u )
=
)( b
( 1
+
+
1
2 b ) 2




1 b
1




[
1 cos( 2 π v )
) b
/( 1 +
1 ;
] b ) 2
0
-0.5
30
1
0.5
20
10
0 0
5
10
15
20
25
©Yao Wang, 2006 EE3414: Image Filtering
0.4
0.2
0
30
1
0.8
0.6
20
10
0 0
5
10
15
20
25 b=2
17

Interpretation in Freq Domain
Filter response
Low-passed image spectrum
0
Original image spectrum
Noise spectrum f
Noise typically spans entire frequency range, where as natural images have predominantly lower frequency components
©Yao Wang, 2006 EE3414: Image Filtering 18

Median Filter
• Problem with Averaging Filter
– Blur edges and details in an image
– Not effective for impulse noise (Salt-and-pepper)
• Median filter:
– Taking the median value instead of the average or weighted average of pixels in the window
• Median: sort all the pixels in an increasing order, take the middle one
– The window shape does not need to be a square
– Special shapes can preserve line structures
• Order-statistics filter
– Instead of taking the mean, rank all pixel values in the window, take the n-th order value.
– E.g. max or min
©Yao Wang, 2006 EE3414: Image Filtering 19

Example: 3x3 Median
100
100
100
100
100
100 100
200 205
195 200
200 205
100 100
100
203
200
195
100
100
100
100
100
100
©Yao Wang, 2006
100
100
100
100
100
100 100
100 200
200 200
100 195
100 100
100
100
200
100
100
100
100
100
100
100
EE3414: Image Filtering
Matlab command: medfilt2(A,[3 3])
20

Example
©Yao Wang, 2006 EE3414: Image Filtering 21

Matlab Demo: nrfiltdemo
Original Image Corrupted Image Filtered Image
Can choose between mean, median and adaptive (Wiener) filter with different window size
©Yao Wang, 2006 EE3414: Image Filtering 22

Noise Removal by Averaging Multiple
Images
©Yao Wang, 2006 EE3414: Image Filtering 23

Image Sharpening
• Sharpening : to enhance line structures or other details in an image
• Enhanced image = original image + scaled version of the line structures and edges in the image
• Line structures and edges can be obtained by applying a difference operator (=high pass filter) on the image
• Combined operation is still a weighted averaging operation, but some weights can be negative, and the sum=1.
• In frequency domain, the filter has the “highemphasis” character
©Yao Wang, 2006 EE3414: Image Filtering 24

Frequency Domain Interpretation
Filter response (high emphasis) sharpened image spectrum
0
Original image spectrum f
©Yao Wang, 2006 EE3414: Image Filtering 25

Highpass Filters
• Spatial operation: taking difference between current and averaging
(weighted averaging) of nearby pixels
– Can be interpreted as weighted averaging = linear convolution
– Can be used for edge detection
• Example filters



0


1
0
1
− 4
1
0
1
0





;





−
0
0
1
− 1
4
− 1
−
0
0
1



;






1
1
 1
1
− 8
1
1


1

;
1 






−
−
−
1
1
1
− 1
−
8
1
−
−
−
1


1

;
1 

– All coefficients sum to 0!
©Yao Wang, 2006 EE3414: Image Filtering 26

Example Highpass Filters





0
− 1
0
− 1
4
− 1
−
0
0
1










−
−
−
1
1
1
− 1
8
− 1
−
−
−
1
1
1 




8
6
30
0
4
2
20
10
0
0
5
10
15
20
25
©Yao Wang, 2006
8
6
12
10
0
30
4
2
20
10
0 0
5
10
15
20
25
EE3414: Image Filtering 27

Example of Highpass Filtering
Original image Isotropic edge detection Binary image
©Yao Wang, 2006 EE3414: Image Filtering 28

Designing Sharpening Filter Using High
Pass Filters
• The desired image is the original plus an appropriately scaled high-passed image
• Sharpening filter f s
= f +
f h h s
( m , n ) =
( m , n ) +
h h
( m , n ) f(x) g(x)=f(x)*h h
(x) f s
(x)=f(x)+ag(x)
©Yao Wang, 2006 x
EE3414: Image Filtering x
29 x

f(x)
Example Sharpening Filters g(x)=f(x)*h h
(x) f s
(x)=f(x)+ag(x)
H h
=
1
4




−
0
0
1
− 1
4
− 1 x
−
0
0
1




⇒ H s
=
1
4




−
0
0
1 x
− 1
8
− 1
−
0
0
1



 with λ = 1 .
x
H h
=
1
8





−
−
−
1
1
1
− 1
8
− 1
−
−
−
1
1



1 

⇒ H s
=
1
8





−
−
−
1
1
1
− 1
16
− 1
−
−
−
1
1



1 
 with λ = 1 .
©Yao Wang, 2006 EE3414: Image Filtering 30

Example of Sharpening
H h
=
1
4





−
−
1
1
− 1
− 1
8
− 1
−
−
−
1
1
1 




©Yao Wang, 2006 EE3414: Image Filtering
H s
=
1
8





−
−
−
1
1
1
− 1
16
− 1
−
−
−
1
1
1 




31

λ = 4
λ = 8
Example of Sharpening
©Yao Wang, 2006 EE3414: Image Filtering 32

Challenges of Noise Removal and
Image Sharpening
• How to smooth the noise without blurring the details too much?
• How to enhance edges without amplifying noise?
• Still a active research area
©Yao Wang, 2006 EE3414: Image Filtering 33

Wavelet-Domain Filtering
©Yao Wang, 2006
Courtesy of Ivan Selesnick
EE3414: Image Filtering 34

Feature Enhancement by
Subtraction
Taking an image without injecting a contrast agent first. Then take the image again after the organ is injected some special contrast agent (which go into the bloodstreams only).
Then subtract the two images --- A popular technique in medical imaging
©Yao Wang, 2006 EE3414: Image Filtering 35

Image Deblurring
• Noise removal considered thus far assumes the image is corrupted by additive noise
– Each pixel is corrupted by a noise value, independent of neighboring pixels
• Image blurring
– When the camera moves while taking a picture
– Or when the object moves
– Each pixel value is the sum of surrounding pixels The blurred image is a filtered version of the original
• Deblurring methods:
• Inverse filter: can adversely amplify noise
• Wiener filter = generalized inverse filter
• Many advanced adaptive techniques
©Yao Wang, 2006 EE3414: Image Filtering 36

Example of Motion Blur
©Yao Wang, 2006 EE3414: Image Filtering 37

Inverse Filtering vs. Wiener Filtering
©Yao Wang, 2006 EE3414: Image Filtering 38

Constrained Least Squares Filtering
©Yao Wang, 2006 EE3414: Image Filtering 39

What Should You Know
• How does averaging and weighted averaging filter works?
• How does median filter works?
• What method is better for additive Gaussian noise?
• What method is better for salt-and-pepper noise?
• How does high-pass filtering and sharpening work?
• For smoothing and sharpening:
– Can perform spatial filtering using given filters
– Deriving frequency response is not required, but should know the desired shape for the frequency responses in different applications
• What is the challenge in noise removal and sharpening?
• What causes blurring?
• Principle of deblurring: technical details not required
©Yao Wang, 2006 EE3414: Image Filtering 40

References
Gonzalez and Woods, Digital image processing , 2 nd edition,
Prentice Hall, 2002. Chap 4 Sec 4.3, 4.4; Chap 5 Sec 5.1 – 5.3
(pages 167-184 and 220-243)
©Yao Wang, 2006 EE3414: Image Filtering 41