Image Filtering
• Many basic image processing techniques are
based on convolution.
• In a convolution, a convolution filter W
is applied to every pixel of an image I to create a
filtered image I*.
• The filter W itself is a 2D matrix of real values.
• To simplify the mathematics, we could consider W
to have a center [0, 0] and extend from –m to m
vertically and –n to n horizontally.
• This means that W is of size (2m + 1)×(2n + 1).
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
1
Convolution
Example: Averaging filter:
1/9
1/9
1/9
1/9
1/9
1/9
1/9
1/9
1/9
j
i
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
2
Convolution
Grayscale Image:
Averaging Filter:
1
6
3
2
9
1/9
1/9
1/9
2
11
3
10
0
1/9
1/9
1/9
5
10
6
9
7
1/9
1/9
1/9
3
1
0
2
8
4
4
2
9
10
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
3
Convolution
Original Image:
Filtered Image:
11/9 61/9 31/9 2
9
0
0
21/9 11
1/9 31/9 10
0
0
5
51/9 10
1/9 61/9 9
7
0
0
3
1
0
2
8
0
0
4
4
2
9
10
0
0
0
0
0
0
0
0
0
value = 11/9 + 61/9 + 31/9 + 21/9 + 111/9 + 31/9
+ 51/9 + 101/9 + 61/9 = 47/9 = 5.222
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
4
Convolution
Original Image:
Filtered Image:
1
6
3
2
9
1/9 1/9 1/9
0
0
0
2
11
3
10
0
1/9 1/9 1/9
0
5
7
5
10
6
9
7
1/9 1/9 1/9
0
0
3
1
0
2
8
0
0
4
4
2
9
10
0
0
0
0
0
0
0
0
value = 61/9 + 31/9 + 21/9 + 111/9 + 31/9 + 101/9
+ 101/9 + 61/9 + 91/9 = 60/9 = 6.667
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
5
Convolution
Original Image:
Filtered Image:
1
6
3
2
9
0
0
0
0
0
2
11
3
10
0
0
5
7
5
0
5
10
6
9
7
0
5
6
5
0
3
1
0
2
8
0
4
5
6
0
4
4
2
9
10
0
0
0
0
0
Now you can see the averaging (smoothing) effect of
the 33 filter that we applied.
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
6
Convolution
• It needs to be noted that for convolution the filter
needs to be rotated by 180 before starting the
computations (otherwise it’s a correlation).
• An intuitive explanation is that we would like
convolution to be like a “local multiplication of
patterns.”
• For example, if our image contains a few 1s and
otherwise 0s, we would expect the convolution
result to contain a copy of the filter pattern
centered at each 1.
• Let us look at an image with one 1-pixel:
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
7
Convolution
Image:
Filter:
Result:
0
0
0
0
0
1
2
3
0
0
0
0
0
0
0
0
0
0
4
5
6
0
9
8
7
0
0
0
1
0
0
7
8
9
0
6
5
4
0
0
0
0
0
0
0
3
2
1
0
0
0
0
0
0
0
0
0
0
0
Oops! The copy of the filter is rotated by 180 !
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
8
Convolution
• If we rotate the filter by 180 beforehand, we get
the desired result.
• This leads to the following definition of convolution
for image I, filter W, and result I*:
• This formula needs to be applied to all coordinates
[i, j] in I in order to create the complete image I*.
• Convolution is commutative, i.e., W and I are
exchangeable.
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
9
Image Filtering
More common: Gaussian Filters
Continuous version:
W ( x, y )  G ( x, y ) 

1
2
2
e
x2  y2
2 2
• implement decreasing influence
by more distant pixels
Discrete version:
April 19, 2016
1/273
1
4
7
4
1
4
16
26
16
4
7
26
41
26
7
4
16
26
16
4
1
4
7
4
1
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
10
Image Filtering
Effect of Gaussian smoothing:
original
April 19, 2016
33
99
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
1515
11
Different Types of Filters
• Smoothing can reduce noise in the image.
• This can be useful, for example, if you want to find
regions of similar color or texture in an image.
• However, there are different types of noise.
• For so-called “salt-and-pepper” noise, for example,
a median filter can be more effective.
• Note that it is not a convolution filter, but it works
similarly.
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
12
Median Filter
• Use, for example, a 33 filter and move it across
the image like we did before.
• For each position, compute the median of the
brightness values of the nine pixels in question.
– To compute the median, sort the nine values in
ascending order.
– The value in the center of the list (here, the fifth
value) is the median.
• Use the median as the new value for the center
pixel.
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
13
Median Filter
• Advantage of the median filter: Capable of
eliminating outliers such as the extreme brightness
values in salt-and-pepper noise.
• Disadvantage: The median filter may change the
contours of objects in the image.
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
14
Median Filter
original image
April 19, 2016
33 median
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
77 median
15
Why Edge Detection?
How can an algorithm extract relevant information
from an image that is enables the algorithm to
recognize objects?
• The most important information for the
interpretation of an image (for both technical and
biological systems) is the contour of objects.
• Contours are indicated by abrupt changes in
brightness.
• We can use edge detection filters to extract
contour information from an image.
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
16
Types of Edges
One-dimensional profiles of different edge types
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
17
Types of Edges
One-dimensional profile of actual edges
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
18
Edge Detection
First we need some definitions:
An edge point is a point in an image with coordinates
[i, j] at the location of a significant local intensity
change.
An edge fragment corresponds to the i and j
coordinates of an edge and the edge orientation ,
which may be the gradient angle.
An edge detector is an algorithm that produces a set
of edges (edge points or edge fragments) from an
image.
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
19
Edge Detection
A contour is a list of edges or the mathematical
curve that models the list of edges.
Edge linking is the process of forming an ordered list
of edges from an unordered list. By convention,
edges are ordered by traversal in a clockwise
direction.
Edge following is the process of searching the
(filtered) image to determine contours.
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
20
Gradient
In the one-dimensional case, a step edge
corresponds to a local peak in the first derivative of
the intensity function.
In the two-dimensional case, we analyze the
gradient instead of the first derivative.
Just like the first derivative, the gradient measures
the change in a function.
For two-dimensional functions it is defined as
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
21
Gradient
In order to compute Gi and Gj in an image F at
position [i, j], we need to consider the discrete case
and get:
Gi = F[i+1, j] – F[i, j]
Gj = F[i, j+1] – F[i, j]
This can be done with convolution filters:
1
Gi =
Gj = 1 -1
-1
To be precise in the assignment of gradients to
pixels and to reduce noise, we usually apply 33
filters instead (next slide).
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
22
Sobel Filters
• Sobel filters are the most common variant of
edge detection filters.
• Two small convolution filters are used
successively:
1
2
1
1
0
-1
0
0
0
2
0
-2
-1 -2 -1
1
0
-1
Si
April 19, 2016
Sj
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
23
Sobel Filters
Sobel filters yield two interesting pieces of
information:
• The magnitude of the gradient (local change in
brightness):
• The angle of the gradient (tells us about the
orientation of an edge):
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
24
Gradient vs. Edge Orientation
Note: Edge and gradient orientation are perpendicular
to each other:
Here, the gradient orientation is horizontal (pointing to
the right) and the edge orientation is vertical.
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
25
Sobel Filters
Calculating the magnitude of the brightness gradient
with a Sobel filter. Left: original image; right: filtered
image.
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
26
Sobel Filters and Thresholding
April 19, 2016
Introduction to Artificial Intelligence
Lecture 21: Computer Vision I
27