CSCI 510/EENG 510
Image and Multidimensional Signal Processing
Fall 2015
Homework Assignment 5 - SOLUTIONS
Due Monday, November 9, 2015
Notes: Please email me your solutions for these problems (in order) as a single Word or PDF document. If you do
a problem on paper by hand, please scan it in and paste it into the document (although I would prefer it typed!).
1. (20 pts) Do Principal Components Analysis (PCA) on the pixels of the RGB image
“pears.png” from the Matlab image processing toolbox and show how it can be
represented with two values per pixel. Give:
a. The 3 eigenvalues and 3 eigenvectors for this image.
b. Show the color RGB image reconstructed from the top two principal
components.
c. Plot the reconstructed pixels in RGB space.
Solution:
We first convert the image, to a collection of (R,G,B) vectors. The image is 3-dimensional, of
size (height, width, depth), where depth is the number of bands (3). We convert it to a 2dimensional array that has size (height*width, depth). So each row represents an (R,G,B) pixel.
We can do this using Matlab’s “reshape” function. According to the help page for this function,
“reshape(X,M,N) returns the M-by-N matrix whose elements are taken columnwise from X.”
(a) We compute the mean and covariance of the collection of vectors. The principal components
are the eigenvectors of the covariance matrix. The Matlab code:
% HW5, problem 2
% Do principal component analysis on an RGB image
clear all
close all
RGB = im2double(imread('pears.png'));
% Convert 3-dimensional array array to 2D, where each row is a pixel (RGB)
X = reshape(RGB, [], 3);
N = size(X,1); % N is the number of pixels
% Get mean and covariance
mx = mean(X);
Cx = cov(X);
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Get eigenvalues and eigenvectors of Cx.
% Produces V,D such that Cx*V = V*D.
% So the eigenvectors are the columns of V.
[V,D] = eig(Cx);
e1 = V(:,3);
disp('Eigenvector e1:'), disp(e1);
e2 = V(:,2);
disp('Eigenvector e2:'), disp(e2);
e3 = V(:,1);
disp('Eigenvector e3:'), disp(e3);
d1 = D(3,3);
1
CSCI 510/EENG 510
Image and Multidimensional Signal Processing
Fall 2015
disp('Eigenvalue d1:'), disp(d1);
d2 = D(2,2);
disp('Eigenvalue d2:'), disp(d2);
d3 = D(1,1);
disp('Eigenvalue d3:'), disp(d3);
The resulting values:
Eigenvector e1:
0.6533
0.6041
0.4564
Eigenvector e2:
-0.6798
0.2026
0.7049
Eigenvector e3:
-0.3333
0.7708
-0.5430
Eigenvalue d1:
0.0679
Eigenvalue d2:
0.0033
Eigenvalue d3:
5.3952e-004
(b) We next transform the input vectors using the equation y = A(x – mx) where mx is the mean
of the vectors, and A is the matrix formed from the eigenvectors of Cx, such that the 1st row of
A is the eigenvector corresponding to the largest eigenvalue, the 2nd row is the eigenvector
corresponding to the second largest eigenvalue, etc. The Matlab code:
%
%
%
A
Construct matrix A such that the 1st row of A is the eigenvector
corresponding to the largest eigenvalue, the 2nd row is the eigenvector
corresponding to the second largest eigenvalue, etc.
= [e1'; e2'; e3'];
%
%
%
%
%
%
%
%
Y
Project input vectors x onto eigenvectors. For each (column) vector x,
we will use the equation y = A*(x - mx).
To explain the Matlab commands below:
X is our (N,3) array of vectors; each row is a vector.
mx is the mean of the vectors, size (1,3).
We first subtract off the mean using X - repmat(mx,N,1).
We then transpose that result so that each vector is a column.
We then apply our transform A to each column.
= A*(X - repmat(mx,N,1))';
% Y has size 3xN
% Display y vectors as images
[height,width,depth] = size(RGB);
Y1 = reshape(Y(1,:), height, width);
Y2 = reshape(Y(2,:), height, width);
Y3 = reshape(Y(3,:), height, width);
figure;
2
CSCI 510/EENG 510
Image and Multidimensional Signal Processing
Fall 2015
subplot(1,3,1), imshow(Y1,[]);
subplot(1,3,2), imshow(Y2,[]);
subplot(1,3,3), imshow(Y3,[]);
The Y1,Y2,Y3 vectors, displayed as images:
Next we reconstruct the original vectors using only the top two principal components, using the
equation xr = AT*y + mx. The Matlab code:
% Reconstruct image using only Y1 and Y2. For each (column) vector y,
% we will use the equation x = A'*y + mx.
% To explain the Matlab commands below:
%
Y is our (3,N) array of vectors; where each column is a vector.
%
A(1:2,:) is the first two rows of A.
%
Y(1:2,:) is the first two rows of Y.
%
A(1:2,:)' * Y(1:2,:) produces our transformed vectors (3xN); we
then
%
transpose that to make an array of size Nx3, and add the mean.
Xr = ( A(1:2,:)' * Y(1:2,:) )' + repmat(mx,N,1);
% Xr has size Nx3
% Display reconstructed image
Ir(:,:,1) = reshape(Xr(:,1), height, width);
Ir(:,:,2) = reshape(Xr(:,2), height, width);
Ir(:,:,3) = reshape(Xr(:,3), height, width);
figure, imshow(Ir);
The original image and the reconstructed image:
Original
Reconstructed
3
CSCI 510/EENG 510
Image and Multidimensional Signal Processing
Fall 2015
For this image, the reconstructed image is a very good approximation to the original image,
probably because it consists mostly of shades of green. We can predict the squared error of the
reconstructed image ... it is just the sum of the eigenvalues corresponding to the principal
components we discarded. For this image, that third eigenvalue was very small.
2. (20 pts) (This is exercise 6.15 in the Gonzalez and Woods textbook.) Consider the
following image composed of solid color squares. For discussing your answer, choose a
gray scale consisting of eight shades of gray, 0 through 7, where 0 is black and 7 is
white. Suppose that the image is converted to HSI color space.
a. Sketch the hue image and provide the values in the image.
b. Sketch the saturation image and provide the values in the image.
c. Sketch the intensity image and provide the values in the image.
If you cannot assign a specific value to a point in the resulting image, explain why.
Red
Green
Blue
Magenta
Cyan
Yellow
White
Black
Solution: The original image is
We use equations 6.2-2 through 6.2-4 to convert RGB to HSI. The following is from an Excel
spreadsheet:
4
CSCI 510/EENG 510
Color
red
green
blue
magenta
cyan
yellow
white
R
1
0
0
1
0
1
1
G
0
1
0
0
1
1
1
B
0
0
1
1
1
0
1
Image and Multidimensional Signal Processing
H
S
0.00
0.33
0.67
0.83
0.50
0.17
#DIV/0!
I
1.00
1.00
1.00
1.00
1.00
1.00
0.00
0.33
0.33
0.33
0.67
0.67
0.67
1.00
theta
0.00
120.00
120.00
60.00
180.00
60.00
#DIV/0!
Fall 2015
Scaled
H S I
0
7 2
2
7 2
5
7 2
6
7 5
4
7 5
1
7 5
### 0 7
The hue, saturation, and intensity images are shown below, from left to right. Note that the hue
for white (or any shade of gray) is undefined.
or, the values of these regions are:
3. (20 pts) In class you trained a decision tree to classify two types of shapes (“plus” and
“hearts”), and applied the tree to classify an unknown shape. Another type of classifier
is the “minimum distance” classifier, which chooses the class with the minimum
distance to the class center. Apply this classifier to the unknown shape, and find which
class it belongs to. Use the Mahalanobis distance as described in the lecture notes, and
give the value of the distance to the closest class. Use the same features that we used in
class (i.e., “area” and “solidity”). Although Matlab has a function to compute
Mahalanobis distance, don’t use that; compute it yourself instead.
Solution: The Mahalanobis distance from a vector x to the (mean) class center xc is
dm =
( x − xc ) C−1 ( x − xc ) where C is the covariance matrix of the class,
T
(
C = E ( x − xc )( x − xc )
T
) or C =
ij
1
N
∑( x − µ )( x
i
i
Using the Matlab code below, I get:
d1 (distance to the “hearts” class) is 5.75
5
j
− µj ) .
CSCI 510/EENG 510
Image and Multidimensional Signal Processing
d2 (distance to the “plus” class) is 0.86
So the shape should be classified as a “plus”.
% HW6 problem 1 - closest distance classifier
clear all
close all
% Read in images with training data
% Invert so that symbols are 1's and background is 0's
I1 = ~imread('hearts.bmp');
I2 = ~imread('plus.bmp');
[L1,n1] = bwlabel(I1);
[L2,n2] = bwlabel(I2);
% Ok, extract a vector of features for each symbol.
% We will use the features returned by "regionprops"
props1 = regionprops(L1, 'all');
props2 = regionprops(L2, 'all');
% Create a set of feature vectors. The elements of the feature vector are
X1(:,1) = cat(1, props1(:).Area);
X1(:,2) = cat(1, props1(:).Solidity);
X2(:,1) = cat(1, props2(:).Area);
X2(:,2) = cat(1, props2(:).Solidity);
m1 = mean(X1);
m2 = mean(X2);
% mean
C1 = cov(X1);
C2 = cov(X2);
% covariances
% Read the test image
I3 = ~imread('test.bmp');
[L3,n3] = bwlabel(I3);
if n3 ~= 1
fprintf('Hey! Too many objects!\n');
% Ok, extract a vector of features for the symbol.
% We will use the features returned by "regionprops"
props3 = regionprops(L3, 'all');
Xtest(1) = props3(1).Area;
Xtest(2) = props3(1).Solidity;
% Find the Mahalanobis distance to each centroid
d1 = sqrt( (Xtest-m1) * inv(C1) * (Xtest-m1)' )
d2 = sqrt( (Xtest-m2) * inv(C2) * (Xtest-m2)' )
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Let's plot the training vectors and their means
hold on
plot(X1(:,1), X1(:,2), '+r');
plot(X2(:,1), X2(:,2), '+g');
plot(m1(1), m1(2), 'or');
plot(m2(1), m2(2), 'og');
xlabel('Area'), ylabel('Solidity');
% Plot the contours of equal probability
6
end
Fall 2015
CSCI 510/EENG 510
Image and Multidimensional Signal Processing
Fall 2015
Xmin = min( cat(1,X1,X2) );
Xmax = max( cat(1,X1,X2) );
dX = (Xmax-Xmin)/50;
[f1,f2] = meshgrid(Xmin(1):dX(1):Xmax(1), Xmin(2):dX(2):Xmax(2));
Cinv = inv(C1);
detCsqrt = sqrt(det(C1));
for i=1:size(f1,1)
for j=1:size(f1,2)
x = [f1(i,j) f2(i,j)];
fX(i,j) = (1/(2*pi*detCsqrt)) * exp( -0.5*(x-m1)*Cinv*(x-m1)' );
end
end
fmax = 1/(2*pi*detCsqrt);
contour(f1,f2,fX, ...
'LevelList', [0.4*fmax 0.6*fmax 0.8*fmax]);
Cinv = inv(C2);
detCsqrt = sqrt(det(C2));
for i=1:size(f1,1)
for j=1:size(f1,2)
x = [f1(i,j) f2(i,j)];
fX(i,j) = (1/(2*pi*detCsqrt)) * exp( -0.5*(x-m2)*Cinv*(x-m2)' );
end
end
fmax = 1/(2*pi*detCsqrt);
contour(f1,f2,fX, ...
'LevelList', [0.4*fmax 0.6*fmax 0.8*fmax]);
% Plot the new shape on the feature space
plot(Xtest(1), Xtest(2), '+b');
The plot below confirms that the test shape (the blue “+”) is closer to the “plus” class center (the
green “o”) than the “hearts” class center (the red “o”). Contour lines are drawn at 0.4, 0.6, and
0.8 of the maximum.
7
CSCI 510/EENG 510
Image and Multidimensional Signal Processing
Fall 2015
0.95
0.9
0.85
0.8
Solidity
0.75
0.7
0.65
0.6
0.55
0.5
0.45
3000
4000
5000
6000
7000
Area
8000
9000
10000
11000
Note – another way to do this is with Matlab's function "mahal" which is in the statistics toolbox.
This returns the squared distance, so you have to take the square root:
sqrt(mahal(Xtest, X1))
sqrt(mahal(Xtest, X2))
4. (20 pts) Below is a small region of an image consisting of a noisy step edge. (The
original image had value=30 on the left of the step, and value=10 to the right of the
step.) It was corrupted by additive Gaussian noise with zero mean and variance = 6.5.
32
28
32
27
32
25
31
28
28
30
29
32
30
33
29
30
31
33
32
29
27
28
31
27
31
13
11
10
11
13
14
7
11
12
9
16
12
8
12
12
12
6
14
6
13
7
11
12
12
10
At the three points indicated by the boxes, compute the value of the adaptive filter
shown in Equation 5.3-12 in the textbook. (The size of the filter is 5x5.)
(Note to me: if you compute variance by dividing by N-1 instead of N, you get a slightly different result.
Not sure which is correct??)
Solution:
The noisy image looks like the figure below, with the points of interest marked with red boxes:
8
CSCI 510/EENG 510
Image and Multidimensional Signal Processing
Fall 2015
σ
The equation for the adaptive filter is: fˆ ( x, y ) =
g ( x, y ) − η2 [ g ( x, y ) − mL ] . We compute the
2
σL
local mean and variance in a 5x5 window surrounding each of the three points of interest. You
can do this by hand or write a small piece of Matlab code.
% Input data
g = [
32
25
28
31
32
28
27
28
32
30
29
32
30
33
29
30
31
33
32
29
27
28
31
27
31
13
11
10
11
13
14
7
11
12
9
16
12
8
12
12
12
6
14
6
13
7;
11;
12;
12;
10 ];
n = 2;
% size of filter is n-2:n+2
r = 3; c = 3;
% Point about which to compute
gsub = g(r-n:r+n,c-n:c+n);
mL = mean2(gsub);
% Local mean
vL = var(gsub(:));
% Local variance
f = g(r,c) - (vN/vL)*(g(r,c) - mL);
fprintf('At point (r,c)=(%d,%d), local mean = %f, local var = %f, filter
value = %f\n', ...
r,c, mL, vL, f);
r = 3; c = 8;
% Point about which to compute
gsub = g(r-n:r+n,c-n:c+n);
mL = mean2(gsub);
% Local mean
vL = var(gsub(:));
% Local variance
f = g(r,c) - (vN/vL)*(g(r,c) - mL);
fprintf('At point (r,c)=(%d,%d), local mean = %f, local var = %f, filter
value = %f\n', ...
r,c, mL, vL, f);
r = 3; c = 6;
% Point about which to compute
gsub = g(r-n:r+n,c-n:c+n);
mL = mean2(gsub);
% Local mean
vL = var(gsub(:));
% Local variance
f = g(r,c) - (vN/vL)*(g(r,c) - mL);
fprintf('At point (r,c)=(%d,%d), local mean = %f, local var = %f, filter
value = %f\n', ...
r,c, mL, vL, f);
The results are:
At point (r,c)=(3,3), local mean = 29.800000, local var = 4.666667, filter value = 29.721429
At point (r,c)=(3,8), local mean = 10.960000, local var = 6.623333, filter value = 10.904882
At point (r,c)=(3,6), local mean = 18.800000, local var = 90.250000, filter value = 10.633795
9
CSCI 510/EENG 510
Image and Multidimensional Signal Processing
Fall 2015
So you can see that the filter is working correctly. Where the local variance is comparable to the
estimated noise variance (i.e., on the left and right of the step edge), the filter outputs the local
mean. Where the local variance is much higher than the estimated noise variance (i.e., where the
window overlaps the step edge), the filter outputs a value approximately equal to the value at that
point.
As described in section 5.3.3, it is assumed that σ η2 ≤ σ L2 , although due to the random nature of
the local noise, it is possible for this condition to be violated in places. This might cause
problems such as the filter returning negative intensity values. The book suggest two approaches
for dealing with this: (1) test the ratio of σ η2 / σ L2 and set it to 1 if it is greater than 1, or (2) apply
the filter as shown and just rescale the intensity values at the end to eliminate negative values.
In our example, the filter does not generate any negative values. However, if you took approach
#1, you could change the statement that computes the filter to:
f = g(r,c) - min(1.0, vN/vL)*(g(r,c) - mL);
In this case, you would get the following output:
At point (r,c)=(3,3), local mean = 29.800000, local var = 4.666667, filter value = 29.800000
At point (r,c)=(3,8), local mean = 10.960000, local var = 6.623333, filter value = 10.904882
At point (r,c)=(3,6), local mean = 18.800000, local var = 90.250000, filter value = 10.633795
5. (20 pts) Give a summary of your progress so far on your final project. Describe your
approach, any findings or results obtained so far, and any problems that you are
encountering. Points will be given based on how much progress has been made, and
how specific the description is.
10