Colorado School of Mines
Image and Multidimensional Signal
Processing
Professor William Hoff
Department of Electrical Engineering and Computer
Science
Colorado School of Mines
http://inside.mines.edu/~whoff/
Department of Electrical Engineering and Computer Science
Boundary Finding
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Edges vs Boundaries
• Edges
– Local intensity discontinuities
– Points
– Not dependent on models
• Boundaries
– Extensive
– Composed of many points
– May be dependent on models
• Typically our goal is to
reconstruct the boundary from
local edge elements
Local edge
point or
edge
element
Object
boundary
To group edge
points, we may
need a domain
or object model
3
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Boundaries
• If we can group individual local edge elements, we can fit
a line or curve to them, to find the object boundary
Edge elements
Grouping
Best fit line
4
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Less
Knowledge
More
Knowledge about Boundary
• Can extract the complete closed
contour of the object if its shape is
known
• Can extract pieces of the boundary
using general line or curve models
• Can just try to find a connected series
of edge elements, using only
heuristics on say, edge curvature
5
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Finding Line Segments by Fitting to Contours
• Linear approximation: approximate a segment by a line
between its endpoints; split at point with largest error,
repeat until approximation is good
N=1
N=2
N=4
N=8
6
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Finding Lines via Hough Transform
• Useful for detecting any parametric curves (eg, lines,
conics)
• Relatively unaffected by gaps in curves, and noise
• Given a set of edge points, find the line(s) which best
explain the data
7
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Hough Transform (continued)
• A line has two parameters (m,b)
y
x
• Given a point (x0,y0), the lines that could pass through this point are
all (m,b) satisfying
The equation b = -x0 m + y0
is a line in (m,b) space
y0 = m x0 + b
y
• Or
b = -x0 m + y0
x0,y0
x
8
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Hough Transform (continued)
• All points on a line in image space, yield lines in
parameter space which intersect at a common point
– This point is the (m,b) of the line in image space
x
m
x1,y
1
b = -x2 m + y2
x2,y
2
m’,b’
b = -x1 m + y1
y
y=m’x+b’
b
9
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Hough Transform Algorithm
•
•
•
Initialize an accumulator array A(m,b) to zero
For each edge element (x,y), increment all cells that satisfy b = -x m + y
Local maxima in A(m,b) correspond to lines
mmin
mmax
m
bmin
1
10 points
voted for this
line (m,b)
3
0
10
4
2
1
bmax
b
Colorado School of Mines
10
Department of Electrical Engineering and Computer Science
Polar Coordinate Representation of Line
• ρ = x cos θ + y sin θ
– Avoids infinite slope
– Constant resolution
q
r
y
x
The parameter space
transform of a point is a
sinusoidal curve
qmin
qmax
rmin
rmax
A(r,q)
11
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Hough Transform
x
•
q
Angle, axis conventions
– angle range is -90°..+89°
– rho range is –dmax..+dmax
r
ρ = x cos θ + y sin θ
• dmax is the largest possible distance
•
Example of a point at (x,y) = (50,100)
y
(50,100)
q =0
r =50
q = -45
+45
r = 50 cos(-45) + 100 sin(-45)
= 50 √2 /2 – 100 √2 /2 = -25√2
q =-90
r =-100
r = 50 cos(45) + 100 sin(45)
= 50 √2 /2 + 100 √2 /2 = 75√2
12
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Example
qmin
q=0
qmax
rmin
r=0
Image of size
100x100, containing 5
points (at the corners)
rmax
13
Colorado School of Mines
Department of Electrical Engineering and Computer Science
14
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Pseudo Code
for all x
for all y
if edge point at (x,y)
for all theta
rho = x*cos(theta) + y*sin(theta)
increment (add 1 to) the cell in H corresponding to (theta,rho)
end
end
end
end
15
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Hough transform in Matlab
dMax is the
length of
the
diagonal
x
• Create arrays
– theta holds all possible values of
theta (i.e., -90 .. +89)
– rho holds all possible values of
rho (i.e., –dMax .. +dMax)
• Allocate storage for the
accumulator array H
y
iTheta=1
(q = -90)
iTheta=length(theta)
(q = +89)
iRho=1
(r = -dMax)
– H has size (length(rho),
length(theta))
iRho=length(rho)
(r = +dMax)
H(iTheta, iRho)
Colorado School of Mines
Department of Electrical Engineering and Computer Science
16
Hough transform in Matlab
•
If we find an edge point at (ix,iy), we loop through all possible values of
theta
for iTheta=1:length(theta)
t = theta(iTheta);
:
•
theta: -90 -89 -88 … 0 …
iTheta 1 2 3 …
88 89
179 180
For each value of t, we compute r using
r = ix*cos(t) + iy*sin(t);
•
We next find the closest value to r in the rho table:
– We subtract r from each value in the rho table
– The location of the minimum in the difference table is where the closest value
of rho was
•
We can find this using “min”:
[d, iRho] = min(abs(rho-r));
Example:
rho: -144 -143 -142 … 0 … 143 144
If r = -142 then
rho-r: -2 -1 0 1 2 …
285 286
then the result is
d=0 (minimum distance), iRho=3 (index)
17
Colorado School of Mines
Department of Electrical Engineering and Computer Science
[iHeight,iWidth] = size(E);
distMax = round(sqrt(iHeight^2 + iWidth^2));
theta = -90:1:89;
rho = -distMax:1:distMax;
% Maximum possible distance from origin
% range of theta values
% range of rho values
H = zeros(length(rho),length(theta));
% Allocate accumulator array
% Scan through edge image
for ix=1:iWidth
for iy=1:iHeight
if E(iy,ix) ~= 0
% Fill in accumulator array
for iTheta = 1:length(theta)
t = theta(iTheta)*pi/180;
% get angle in radians
% Calculate distance from origin, given this angle
% Let's use (1,1) as the "origin"
r = (ix-1)*cos(t) + (iy-1)*sin(t);
% Find rho value that is closest to this
[d, iRho] = min(abs(rho-r));
if d <= 1
H(iRho,iTheta) = H(iRho,iTheta) + 1;
end
% Inc accumulator array
end
end
end
end
figure, imshow(H, []), impixelinfo;
Colorado School of Mines
Department of Electrical Engineering and Computer Science
18
Exercise
• Create a synthetic image E (200x200)
– Put a non-zero point somewhere
– Look at the resulting H matrix
% Make a synthetic image E
E = zeros(200,200);
E(100,50) = 1;
imshow(E,[]);
pause
19
Colorado School of Mines
Department of Electrical Engineering and Computer Science
To visualize how the H
matrix is formed:
•
Every time an
accumulator cell is
incremented, draw the
line corresponding to
that value of (rho, theta)
•
Equation of the line is
ax + by + c = 0, where
– a = cos(t)
– b = sin(t)
– c = -rho
•
We’ll use Matlab’s line
function to draw a line
from (x1,y1) to (x2,y2)
%%%%%%%%%%%%%
% Draw the line represented by (r=rho,t=theta)
% Equation of the line is ax+by+c=0, where
% a = cos(t), b = sin(t), c = -r
imshow(E);
title(sprintf('theta = %f, rho = %f\n', t, r));
if abs(t) > pi/4
% Line is mostly horizontal. Pick two values of x,
% and solve for y = (-ax-c)/b
x1 = -iWidth;
y1 = (-cos(t)*x1+r)/sin(t);
x2 = 2*iWidth;
y2 = (-cos(t)*x2+r)/sin(t);
else
% Line is mostly vertical. Pick two values of y,
% and solve for x = (-by-c)/a
y1 = -iHeight;
x1 = (-sin(t)*y1+r)/cos(t);
y2 = 2*iHeight;
x2 = (-sin(t)*y2+r)/cos(t);
end
line([x1 x2], [y1 y2], ...
'Clipping', 'off', ...
'Color', 'r');
% Draw the rho vector in green
line([1 r*cos(t)], [1 r*sin(t)], ...
'Clipping', 'off', ...
'Color', 'g');
pause
%%%%%%%%%%%%%%
20
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Exercise
• Put two non-zero points
– Find the peak in the resulting H matrix
– What (rho, theta) does that peak correspond to?
% Make a synthetic image E
E = zeros(200,200);
E(100,50) = 1;
E(50,50) = 1;
imshow(E,[]);
pause
21
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Finding peaks in the Hough array
• The simplest way is to get all values greater than a
threshold
[iRhoPeaks,iThetaPeaks] = find( H > thresh);
– this yields all the locations in H where it exceeds the threshold
• You can then draw the lines corresponding to the peaks
for i=1:length(iRhoPeaks)
r = rho(iRhoPeaks(i));
t = theta(iThetaPeaks(i)) *pi/180;
fprintf('r = %f, t = %f\n', r, t);
:
end
insert the code to
draw the line
corresponding to
(r,t)
22
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Example – synthetic image
• Perform Canny edge
detection to get edge
points
• Perform Hough
transform on edge
image
• Notice multiple lines
detected for each true
line
image “diamond.tif”
23
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Non-maximal suppression
• Instead of taking all points greater than a threshold, take points that
are local maxima
• You can find the largest point within a neighborhood by doing a
grayscale dilation
• Then find those points that
Dilation of H (gives the
local max within a small
equal the local maxima
H
neighborhood)
6
Hmax = (H==imdilate(H, ones(M,N));
5
4
3
2
H values
x
1
• Finally, take the local
maxima that are also
greater than the
threshold
Hpeaks = Hmax & (H>thresh);
0
1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16
24
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Example – synthetic image
H
Hpeaks
(local
maxima)
Size of neighborhood:
10x5
Threshold: 0.5*max(H(:))
25
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Matlab Hough Transform Functions
• [H, theta, rho] = hough(bw)
Output Hough array,
size NRho x NTheta
Vectors of theta
and rho values
Input binary
image of edge
points
• peaks = houghpeaks(H,numpeaks, …
’Threshold’,thresh, ‘NHoodSize’, [M N]);)
Output array (row,col) of
peaks (up to numpeaks)
The (rho, theta) values for the ith peak are
r = rho(peaks(i,1));
t = theta(peaks(i,2));
• lines = houghlines(bw, theta, rho, peaks)
Output structure array of lines. Each line has fields:
(endpoint1, endpoint2, rho, theta)
26
Colorado School of Mines
Department of Electrical Engineering and Computer Science
clear all
close all
I = imread('gantrycrane.png');
G = rgb2gray(I);
E = edge(G, 'canny');
imshow(E)
[H,theta,rho] = hough(E);
figure, imshow(H,[]);
peaks = houghpeaks(H,50,'Threshold',30);
figure, imshow(G,[]), hold on
lines =
houghlines(E,theta,rho,peaks,'FillGap',5,'MinLength',15);
for k = 1:length(lines)
xy = [lines(k).point1; lines(k).point2];
line(xy(:,1),xy(:,2),'LineWidth',1,'Color','r');
end
27
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Other Shapes
• Hough transform can be used to find any parameterized curve
• Example - circles (x0,y0,r)
28
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Example – detecting parabolas
• A parabola centered
at (x0,y0) has the
equation
2
y  y0 = a  x  x0 
• So we need to
search for three
parameters: x0,y0,a
Grenouilles.jpg
• Can limit range of
“a” to search over
29
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Pseudocode
Input a binary edge image E(x,y)
Initialize accumulator array A(i,j,k) to zeros
for all values of (x,y)
if E(x,y) == 1
for all values of a between amin and amax
for all values of x0
Compute y = y  a  x  x 2
0
0
Increment A(i,j,k) where (i,j,k) corresponds to the cell
associated with x0i , y0 j , ak
end


end
end
End
Search for peaks in A(i,j,k) – the corresponding values of
are the parameters of the detected parabolas
30
Colorado School of Mines
Department of Electrical Engineering and Computer Science
% Find parabolas in binary image I(N,M).
% A parabola is y=rx^2
clear all
close all
I = rgb2gray(imread('Grenouilles.jpg'));
imshow(I,[]);
[N,M] = size(I);
[E,thresh] = edge(I, 'canny', 0.45);
figure, imshow(E,[]);
rvals = [ 0.003, 0.004, 0.005, 0.006, 0.007, 0.008 ];
R = length(rvals);
% number of sizes to try
% Fill accumulator array A(N,M,R)
A = zeros(N,M,R);
[yIndex xIndex] = find(E);
% get edge points
for cnt = 1:length(xIndex)
for r=1:R
for x0 = 1:M
y0 = yIndex(cnt)-rvals(r)*(xIndex(cnt)-x0)^2;
y0 = round(y0);
if y0 < N & y0 >= 1
A(y0,x0,r) = A(y0,x0,r)+1;
end
end
end
end
figure, imshow(A(:,:,round(R/2)),[]);
title(sprintf('A slice of the accumulator array, for r=%f', ...
rvals(round(R/2)) ));
pause
Colorado School of Mines
Department of Electrical Engineering and Computer Science
31
Finding peaks in a 3D accumulator array
• The accumulator array is three
dimensional A(y,x,a)
• Similarly to the case of a 2D accumulator
array, we want to find peaks that are local
maxima, and are greater than a threshold
y
– In this case, we look for local maxima over a
three dimensional neighborhood
a
x
• We can still use “imdilate”, but use a
3D structuring element containing all 1’s
– For example, use a block of size (W,H,D)
– Dilation will find the local maximum with
the neighborhood of that size
H
W
D
32
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Extracting peak coordinates from 3D array
• Assume
– We have found local maxima in the 3D accumulator array A
– Also we found values in A that are greater than a threshold
– Let Apeaks contain 1’s where both conditions are true (ie, they are the
peaks we want)
• Then we want to find the coordinates of the nonzero points in
Apeaks
– Similarly to the 2D case, we can use Matlab’s “find” function
• However, you can’t just do
[rowIndices, colIndices, depthIndices] = find(Apeaks);
– For some reason, “find” only works for 1D and 2D arrays
33
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Extracting peak coordinates (continued)
• We can always treat a multidimensional array as a
1D array
– This is actually how multidimensional arrays are stored
inside the computer
– Then every element of Apeaks has a one dimensional index
row indices
col indices
• So if you do
– you get the one dimensional indices of the non-zero points
• To calculate the equivalent subscripts in the 3D
array, from the one dimensional indices, you can
use Matlab’s “ind2sub” function
[rowIndices,colIndices,depthIndices] = ...
ind2sub(size(Apeaks), indices1D);
one dimensional indices
indices1D = find(Apeaks);
34
Colorado School of Mines
Department of Electrical Engineering and Computer Science
% Find the local maxima in a 3D neighborhood
Amax = imdilate(A, ones(20,20,4));
% We want those places where A = AMax and A > thresh
thresh = 95;
Apeaks = (A==Amax) & (A > thresh);
% Get indices of the peaks (the nonzero points).
% These are the indices of the array, which is treated
% like a one dimensional array.
indices1D = find(Apeaks);
% The array is actually three dimensional, not one dimensional.
% This function will calculate the equivalent indices of the three
% dimensional array.
[rowIndices,colIndices,depthIndices] = ...
ind2sub(size(Apeaks), indices1D);
figure, imshow(E,[]);
for i=1:length(rowIndices)
x0 = colIndices(i);
y0 = rowIndices(i);
r0 = rvals( depthIndices(i) );
for x=1:M
y = round(y0 + r0*(x-x0)^2);
if y<=N && y >= 1
rectangle('Position', [x y 1 1], ...
'EdgeColor', 'r');
end
end
end
35
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Extension to use Gradient Direction
• If you know the gradient direction at
each edge point, it reduces the
complexity of the Hough transform
q
r
• Namely, the angle of the gradient
already gives the angle of the line
Edge
point
Gradient
direction
• This eliminates one “for” loop – the
loop over the angles
• Potential problem
•
Image noise can cause uncertainty in the gradient angle
•
You can increment multiple cells corresponding to several angles near the
estimated angle
36
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Generalized Hough Transform
• Can find arbitrary shapes
– The shape is represented by a sequence of boundary points
– For each point, we specify the distance and angle to the center
(x1,y1)
R1
R2
(x2,y2)
(xc,yc)
• We create an “R-table”
– For each gradient direction,
store the vector(s) R to the
object
The two boundary points (x1,y1) and
(x2,y2) have the same gradient
direction. The vector from each
boundary point to the object’s center
(xc,yc) is R1 and R2, respectively.
Gradient angle
q1
Vectors pointing to object origin
R1,R2
q2
R3,R4,R5,…
:
qN
…
37
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Generalized Hough Transform
• To find the shape
– Compute the gradient direction at each edge point
– Look up the vectors R from the R-table ... these vectors point to the possible
location of the object center
– Increment an accumulator array at those point(s)
• Example
Template object
Test image
Hough accumulator array
38
Colorado School of Mines
Department of Electrical Engineering and Computer Science
Summary / Questions
• Edge points can be linked to form meaningful region boundaries.
• Lines and line segments can be fitted to edge points.
– If a connected series of edge points is available (e.g., contours
produced by the LoG or Canny edge detector), then you can recursively
split the contour and fit line segments to each portion.
– If edge points are not necessarily connected, you can use the Hough
transform method to find lines.
• The Hough transform can be applied to find any parameterized
curve. It is robust to noise and missing points.
– Why is it desirable to parameterize lines using the form 𝜌 = 𝑥𝑐𝑜𝑠𝜃 +
𝑦𝑠𝑖𝑛𝜃 instead 𝑦 = 𝑚𝑥 + 𝑏?
39
Colorado School of Mines
Department of Electrical Engineering and Computer Science