Digital Image Analysis and Processing
CPE 0907544
Image Enhancement –Part II
Spatial Filtering
Chapter 3
S ti
Sections
: 3.4-3.7
3437
D Iyad
Dr.
I d Jafar
J f
Outline
y Introduction
y Mechanics of Spatial Filtering
y Correlation and Convolution
y Linear Spatial Filtering
y Spatial filters for Smoothing
y Spatial filters for Sharpening
y Nonlinear Spatial
S
Filtering
y Combining Spatial Enhancement Techniques
2
Background
y Filtering is borrowed from the frequency domain
processing and refers to the process of passing or
rejecting certain frequency components
y Highpass,
Highpass lowpass,
lowpass band-reject , and bandpass
filters
y Filtering is achieved in the frequency domain by
g g the pproper
p filter ((Chapter
p 4))
designing
y Filtering can be done in the spatial domain also by
3
using filter masks (kernels, templates, or windows)
y Unlike frequency domain filters, spatial filters can
be nonlinear !
Spatial Filtering Mechanics
y A spatial filter is characterized by
y A rectangular neighborhood of size mxn (usually m and n are odd)
y A predefined operation that is specified by the mask values at each
position.
Origin
w(-1,-1)
w(-1,0)
w(-1,1)
w(0,-1)
w(0,0)
w(0,1)
w(1,-1)
w(1,0)
w(1,1)
y
(x, y)
Neighbourhood
3x3 filter mask example
x
y Spatial filtering Operation
Image
g f (x, y)
y
y The filter mask is centered at each pixel in the image and the output
4
pixel value is computed based on the operation specified by the
mask
Oi i
Origin
Spatial Filtering Mechanics
y
w(-1,-1)
w(-1,0)
w(-1,1)
w(0,-1)
w(0,0)
w(0,1)
w(1,-1)
w(1,0)
w(1,1)
f(x-1,y-1)
f(x-1,y)
f(x-1,y+1)
f(x,y-1)
f(x,y)
f(x,y+1)
f(x+1,y-1)
f(x+1,y)
f(x+1,y+11)
3x3 Filter
x
Image f (x, y)
Filter
a
g( x,
x y)=
5
b
∑ ∑ w( s,ts t ) f ( x + s,s y + t )
s =− a t =−b
x
Original Image Pixels
∑
Vector Representation of
S ti l Filt
Spatial
Filtering
i
y The previous filtering equation can be written as
R = w1 z1 + w2 z2 + ... + wmn zmn
mn
=
∑
wk zk
k =1
w1
w2
w3
w4
w5
w6
w7
w8
w9
y If we represent the coefficients of the filter mask as a
row vector w = [w1 w2 w3 w4 w5 w6 w7 w8 w9] and
the image pixels under the mask by z = [z1 z2 z3 z4 z5
z6 z7 z8 z9],
] then
h the
h filtering
fl
equation can be
b written
as
R = wz
6
T
Spatial Correlation
y The
filtering operation just described is called
correlation
y In 1-D
Origin
f
1) Function and mask
0 0 0 1 0 0 0 0
w
1 2 3 2 8
0 0 0 1 0 0 0 0
2) Initial Alignment
3) Zero Padding for f by m-1
1 2 3 2 8
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
1 2 3 2 8
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
7
4) Position after one shift
1 2 3 2 8
Spatial Correlation
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
5) Position after 4 shifts
6) Repeat
Final Position
1 2 3 2 8
.
.
.
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
1 2 3 2 8
Full correlation result
8
Cropped correlation
result
0 0 0 8 2 3 2 1 0 0 0 0
0 8 2 3 2 1 0 0
Spatial Convolution
y A strongly related operation to correlation is
convolution (an important tool in linear systems
theory)
y The mechanics of convolution is similar to those
of correlation, except that the filter mask is
rotated by 180o before sliding
9
Spatial Convolution
Origin
f
1) Function and mask
0 0 0 1 0 0 0 0
w
1 2 3 2 8
0 0 0 1 0 0 0 0
2) Initial Alignment
after rotation of
mask
3) Zero Padding for f by m-1
8 2 3 2 1
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
8 2 3 2 1
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
4) Position after one shift
10
8 2 3 2 1
Spatial Convolution
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
5) Position after 4 shifts
6) Repeat
Final Position
8 2 3 2 1
.
.
.
0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0
8 2 3 2 1
Full convolution result
11
Cropped convolution
result
0 0 0 1 2 3 2 8 0 0 0 0
0 1 2 3 2 8 0 0
Extension to 2D
y Extension to 2D is straight forward
y In convolution,, the mask is flipped
pp verticallyy and
horizontally
y Zero padding is done in both directions by m-1 and n-1
0 0 0 0 0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
f
Zero padding by m-1
m 1 and n
n-1
1
0 0 0 0 0 0 0 0 0
0 0 1 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0
0 0 0 0 1 0 0 0 0
0 0 0 0 0 0 0 0 0
1 2 3
w
4 5 6
7 8 9
12
9 8 7
Mask rotation
6 5 4
3 2 1
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
Extension to 2D
13
1 2 3 0 0 0 0 0 0
9 8 7 0 0 0 0 0 0
4 5 6 0 0 0 0 0 0
6 5 4 0 0 0 0 0 0
7 8 9 0 0 0 0 0 0
3 2 1 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 1 0 0 0 0
0 0 0 0 1 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
Initial position in correlation
Initial position in convolution
Extension to 2D
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 9 8 7 0 0 0
0 0 0 1 2 3 0 0 0
0 0 0 6 5 4 0 0 0
0 0 0 4 5 6 0 0 0
0 0 0 3 2 1 0 0 0
0 0 0 7 8 9 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
Full correlation result
14
Full convolution result
Extension to 2D
0 0 0 0 0
0 0 0 0 0
0 9 8 7 0
0 1 2 3 0
0 6 5 4 0
0 4 5 6 0
0 3 2 1 0
0 7 8 9 0
0 0 0 0 0
0 0 0 0 0
Cropped correlation result
Cropped
pp convolution result
Convolution
Correaltion
a
g(( x, y ) =
b
∑ ∑ w(( s,t ) f ( x + s, y + t )
s =− a t =−b
15
a
g(( x, y ) =
b
∑ ∑ w(( s,t ) f ( x − s, y − t )
s =− a t =−b
Treatment of Pixels at Edges
y In the previous slides, we padded the image with zeros
in both directions in order to compensate for
unavailable values.
e
e
y Other approaches
x
y Replicate edge pixels
y Consider only available
pixels that fall under the
mask in the computation
of the new values
y Allow pixels to wrap
around
16
e
e
e
y
e
Image f (x, y)
e
Treatment of Pixels at Edges
y The result depends on the used approach
y Example
E
l 3.6
36
Filtered Image:
Z
Zero
Padding
P ddi
Original
Image
Filtered Image:
Replicate Edge Pixels
Filtered Image:
Wrap Around Edge Pixels
17
Smoothing Spatial Filters
y A smoothing (averaging, blurring) filter replaces each
pixel
i l with
i h the
h average value
l off allll pixels
i l under
d the
h
mask
y Uses of smoothing filters
y The smoothed image correspond to the gross details
of the image or the low frequency content of the image
y Can
C be
b used
d to
t reduce
d
noise
i as it is
i characterized
h
t i d with
ith
sharp transitions
y However, smoothing usually result in degradation of
ed e quality
edge
alit (sharpness
(sha p ess of the image)
i a e)
18
Smoothing Spatial Filters
y Common smoothing masks
1
1
1
1
1
1
1
1
1
Standard averaging mask
X 1/9
1
2
1
2
4
2
1
2
1
X 1/16
Weighted average mask
y Notes
y The weighted average filter gives more weight to pixels near the
center
y It is hard to see the visual difference between the processing
results of the two filters; however, the weighted average filter
summation is 16, which make it more attractive for computers
19
Smoothing Spatial Filters
y Example: 3.7 Smoothing with different mask sizes. Notice the
loss of details as the mask size increases
Original
20
Smoothing by 9x9 Mask
Smoothing by 3x3 Mask
Smoothing by 15x15 Mask
Smoothing by 5x5 Mask
Smoothing by 35x35 Mask
Smoothing Spatial Filters
y Example 3.8: smoothing highlights gross details.
Could
C
ld bbe useful
f l in
i providing
idi bbetter
tt segmentation
t ti
results
Original Image
21
Smoothed Image
Thresholded Image
Smoothing Spatial Filters
y Example 3.9: Noise Reduction
Original
22
Smoothed
Sharpening Spatial Filters
y The principle objective of sharpening is to highlight
transitions in intensity which usually correspond to
edges in images; thus sharpening is the opposite of
smoothing
y If we examine the smoothing operation we can think
off it as integration
i t
ti
y Thus to perform sharpening in the spatial domain, it is
intuitive to use differentiation
y In
23
the following few slides we examine the
fundamental properties of the first and second
pp
on different intensityy
derivative when applied
transitions
Sharpening Spatial Filters
y Lets discuss differentiation for 1-D case
y We are concerned about the behavior of 1st and 2nd
derivatives in the following areas
y Constant intensity
y Onset and end of
discontinuities (ramps and steps discontinuities)
y Intensity ramps
y Properties of 1st derivative
y Zero in areas of constant intensity
y Nonzero at the onset of a step and intensity ramp
y Nonzero along intensity ramp
24
y P
Properties 2ndd derivative
d
y Zero in areas of constant intensity
y Nonzero
N
att the
th onsett and
d end
d off a step
t and
d intensity
i t it ramp
y Zero along intensity ramp
Sharpening Spatial Filters
y Derivatives can be approximated as differences
y 1st derivative at x
∂f
= f ( x +1) − f ( x )
∂x
y 2nd derivative at x
∂ f
2
∂x
25
2
= f ( x +1) + f ( x −1) − 2 f ( x )
Sharpening Spatial Filters
y Investigation of derivatives behavior
26
Sharpening Spatial Filters
y Notes
y Examining the 1st and 2nd derivatives plots shows that all
of their properties are satisfied
y 1st derivative produce thicker edges than 2nd derivatives
y 2nd derivative produce double edge separated by a zero
crossing
y 2nd derivative is commonly used in sharpening since it
has simpler implementation and finer edges
27
Sharpening Using
D i ti
Derivative
nd
2
y The second derivative (Laplacian) in 2-D is defined as
∇f 2 =
∂2 f
∂x 2
+
∂2 f
∂y 2
If we define
∂2 f
∂x
2
= f ( x + 1, y ) + f ( x − 1, y ) − 2 f ( x, y )
∂2 f
∂y
2
= f ( x, y + 1 ) + f ( x, y − 1 ) − 2 f ( x, y )
Then the discrete second derivative can be
approximated by
28
∇f 2 =
f ( x + 1, y ) + f ( x − 1, y ) + f ( x, y + 1 ) + f ( x, y − 1 ) − 4 f ( x, y )
Sharpening Using
D i ti
Derivative
nd
2
y The Laplacian can be implemented as a filter mask
0
1
0
1
-4
1
0
1
0
1
1
1
1
-8
1
1
1
1
y Or
29
Sharpening Using
D i ti
Derivative
nd
2
y Computing
p
g the Laplacian
p
doesn’t p
produce a sharpened
p
image. However, grayish edge lines and discontinuities
superimposed on a dark background
30 y
It is common practice to scale the Laplacian image to [0,255] for better display
Sharpening Using
D i ti
Derivative
nd
2
y Alternatively,
y to obtain a sharpened
p
image
g g(x,y),
g( y)
subtract the Laplacian image from the original image
g(( x, y ) = f ( x, y ) − ∇ f ( x, y )
2
31
Original
Image
=
Laplacian
Filtered Image
Sharpened
Image
Sharpening Using
D i
Derivative
i
nd
2
y The two steps required to achieve sharpening can be
combined into a single filtering operation
g ( x, y ) = f ( x, y ) − ∇ f
= f ( x, y ) − [ f ( x + 1, y ) + f ( x − 1, y )
+ f ( x, y + 1) + f ( x, y − 1)
− 4 f ( x, y )]
= 5 f ( x, y ) − f ( x + 1, y ) − f ( x − 1, y )
− f ( x, y + 1) − f ( x, y − 1)
2
32
Sharpening Using 2nd
Derivative
y As a filter mask
mask, the previous equation can be represented
as
33
0 -1 0
-1 5 -1
0 -1 0
Sharpening Using 1st Derivative
y The first derivative in image processing is implemented
as a ggradient which is defined as a vector
⎡ ∂f ⎤
⎡Gx ⎤ ⎢ ∂x ⎥
∇f = ⎢ ⎥ = ⎢ ∂f ⎥
⎣G y ⎦ ⎢ ⎥
∂y ⎥⎦
⎢⎣ ∂y
y The gradient points in the direction of greatest rate of
change
h
at location
l
i (x,y)
( )
yThe magnitude of the gradient is defined as
34
⎡⎛ ∂f ⎞2 ⎛ ∂f ⎞2 ⎤
M ( x,, y ) = ⎢⎜ ⎟ + ⎜ ⎟ ⎥
⎢⎣⎝ ∂x ⎠ ⎝ ∂y ⎠ ⎥⎦
1
2
Or, approximately
| ∇f |≈ Gx + G y
Sharpening Using 1st Derivative
y Computation of the gradient using Roberts
g
operators
p
cross-gradient
y Roberts operators for computing Gx and Gy are shown
below. The upper-left pixel in the operator is overlaid over
the pixel (x,y) or z5 in the original image
Z1
Z2
Z3
-1
1
0
0
-1
1
Z4
Z5
Z6
0
1
1
0
Z7
Z8
Z9
Horizontal Operator
Pixel z5 and its neighbours
Gx ( x, y ) = [ z9 − z5 ] ≈ z9 − z5
2
G y ( x,, y ) = [ z8 − z6 ] ≈ z8 − z6
2
35
M ( x, y ) ≈ z9 − z5 + z8 − z6
Vertical Operator
Sharpening Using 1st Derivative
y Computation of the gradient using Sobel
p
operators
y Sobel operators for computing Gx and Gy are shown
below. The upper-left pixel in the operator is overlaid over
the pixel (x,y) or z5
Z1
Z2
Z3
-1
1
-2
2
-1
1
-1
1
0
1
Z4
Z5
Z6
0
0
0
-2
0
2
Z7
Z8
Z9
1
2
1
-1
1
0
1
Pixel z5 and its neighbours
Mask to Compute Gx
Mask to Compute Gy
Gx ( x, y ) ≈ ( z7 + 2 z8 + z9 ) − ( z1 + 2 z2 + z3 )
G y ( x, y ) = ( z3 + 2 z6 + z9 ) − ( z1 + 2 z4 + z7 )
36
M ( x, y ) = Gx ( x, y ) + G y ( x, y )
Sharpening Using 1st Derivative
y Example 3.10
y Gradient is widely
y used in industrial inspection
p
as it
produce thicker edges in the result, which make it easier
for machines to detect artifacts
37
Optical Image for a contact lens
Gradient Image
Nonlinear (Order-statistic)
S ti l Filt
Spatial
Filters
y Order-statistic filters are nonlinear filters whose
response is based on ordering the pixels under the mask
and then replacing the centre pixel with the value
determined by the ranking result
y Examples
p
y minimum filter
Z1
Z2
Z3
Z4
Z5
Z6
Z7
Z8
Z9
Rank
y maximum filter
New value
y median filter
y popular and useful in removing impulse or salt-and-pepper noise
with less blurring than linear smoothing
38
Nonlinear (Order-statistic)
S ti l Filt
Spatial
Filters
y Example
p 3.11
Original Image
With Noise
Image After
Averaging Filter
Image After
Median Filter
Note how the median filter has reduced the noise
significantly with less blurring than smoothing filter
39
Combining Spatial
Enhancements
h
y It is very common to combine
different enhancement
techniques in order to achieve
the desired goal
y Example 3.12
y We want to enhance/sharpen
the bones in this image
g
y Direct application of the
Laplacian may result in noise
amplification
40
Combining Spatial
Enhancements
h
Compare to enhancement by single method
Processed by a
combination of
methods
41
Sharpened by
Laplacian
Histogram
Equalization
Power-Law with
gamma = 0
0.5
5
Combining Spatial
E h
Enhancement
t
Enhancement by multiple methods
(a)
Laplacian filter of
bone scan (a)
42
(b)
Sharpened version of
bone scan achieved
(c)
y subtracting
g ((a))
by
Sobel filter of bone
and (b)
scan (a)
(d)
Combining Spatial
E h
Enhancement
t
The product of (c)
and (e) which will be
used as a mask
(e)
43Image
Sharpened image
which is sum of (a)
and (f)
(d) smoothed with
a 5*5 averaging filter
(f)
Result of applying a
power law trans.
power-law
trans to
(g)
(g)
(h)
Combining Spatial
E h
Enhancement
t
Compare
p
the original
g
and final images
g
44
Readings
y Read section 3
3.6.3
6 3 titled “Unsharp
Unsharp Masking
and Highboost filtering”.
45
Related Matlab Functions
y Check Matlab documentation for the
following functions
y conv2
2
y corr2
y fillter2
y imfilter
y padarray
y median
46
y medfilt2