advertisement

Digital image processing and analysis Laboratory 6: Detection of image features Vedrana Baličević, Pavle Prentašić, Hrvoje Kalinić, Sven Lončarić 2014. 6.1 Image features 6.1.1 Spatial features Features can be differentiated based on the space where they exist, for example they can be spatial, temporal, frequency features etc. Since the word ”image” usually refers to an image in 2D space, for most of applications we use spatial features. Based on these features the computer can analyze, describe and interpret the image content. Therefore, image feature retrieval is the first step for computational image analysis. 6.1.2 Amplitude features For each pixel in an image, amplitude features are defined as some parameters calculated from neighborhood pixel intensities. In MATLAB, there are several functions that can apply a selected operation on each pixel’s neighborhood (eg. blkproc(), colfilt(), nlfilt()). We are going to use the function colfilt(). To do this, we need to understand function pointers concept and how to use them in MATLAB. This is described in the following example. An example >> >> >> >> img = imread(’texture.tif’); fun = @(x) mean(x); img2 = colfilt(img,[32 32],’sliding’,fun); figure; imagesc(img2); colormap(gray) % % % % read the image function pointer (function handle) image is being processed display the resulting image The same way we defined the function pointer f un, pointing to the function mean(), we can define function pointers to some other existing functions or our own functions. 6.1.3 Problems 1. Select an image with smaller dimensions (such as 256 × 256) - or scale a larger image to this size. For a selected image calculate several amplitude features (using functions: max(), min(), mean(), std()). 2. How do the results depend on the selected block size? 1 6.2 First order histogram features Amplitude values within a neighborhood of a pixel can be interpreted as the results of a random experiment. In that case the first order histogram represents the estimation of the probability density function (PDF) for the selected neighborhood. Based on this estimation, we can determine some statistical properties (features) such as moments and central moments of the PDF. Besides moments, we find entropy and energy of the histogram interesting in image processing as well. To do this, we first need to learn how to calculate histogram in MATLAB, which is given with the following example. Since the function hist() calculates the histogram of a 1D vector, in this example we transform an image into a vector (img(:)) and provide this to the function in the same line of code: An example >> img = imread(’texture.tif’); % read the image >> hist(im2double(img(:)),30); % calculate and display the histogram of the image, % by grouping the values into 30 bins 6.2.1 Problems 1. Write the functions for calculating moments, entropy and energy, in such way that they can be used with a function colfilt(). Mathematical expressions for calculating these features are given in the lectures. 2. Select an image with smaller dimensions (such as 256 × 256) - or scale a larger image to this size. For a selected image calculate 1st order histogram features (entropy and energy) on the block / window of size 5 × 5, using the funciton colfilt(). Display the results and comment on them. 6.3 Second order histogram features As described, pixel intensity values within a neighborhood of a pixel can be interpreted as the results of a random experiment. In that case the first order histogram represents the estimation of the probability density function (PDF) for the selected neighborhood. But, instead of observing a single pixel, we can observe a pair of pixels, whose distance (shift) is defined with a specific relation. In that case the random experiment becomes two-dimensional, and probability density function can be estimated using the second order histogram as described here: An example >> >> >> >> >> >> >> [img,map] = imread(’testpat1.tif’); % read the image img = im2double(img); % convert the image to double format % Try the shift (2,3): img1 = img(4:end,3:end); % crop the images to obtain the (2,3) shift between them img2 = img(1:end-3,1:end-2); hist3([img1(:), img2(:)]) % calculate and display the 2nd order histogram of the image % Try the shift (5,1): 2 >> img1 = img(2:end,6:end); % crop the images to obtain the (5,1) shift between them >> img2 = img(1:end-1:1:end-5); >> hist3([img1(:), img2(:)]) % calculate and display the 2nd order histogram of the image 6.3.1 Problems 1. Read the image 4.2.07.tiff from the USC-SIPI database. Calculate and display the 2nd order histograms for the shifts (1, 1) and (5, 5). 2. Why are the results grouped around a diagonal? 3. Why are they more dispersed in the second case? 4. Read the image saturn.tif and calculate the 2nd order histograms for several shifts of your choice. Display the results and comment on them. 6.4 Edge detection Edge detection is one of the most important tasks in image processing since the edges actually define borders of objects. Therefore we can use them to segment, register and/or identify objects in the image. Since the edges in an image are characterized by a sudden change of the amplitude values of the pixels, we can employ the gradient estimation methods for their detection. 6.4.1 Sobel and Prewitt operator Gradient-based methods usually estimate the gradient value in 2 directions: x and y. These two values define a gradient vector, ie. its direction and its value for a selected pixel. If we’re interested only in the gradient value, we can calculate it as the square root of the each direction’s square value, as shown in the Fig.6.1. Figure 6.1: Gradient calculation procedure. Masks for the Prewitt and the Sobel operator for the detection of horizontal edges can be obtained in MATLAB with the function fspecial(): An example >> sy = fspecial(’sobel’) % sobel mask for horizontal edges >> sx = rot90(sy) % rotate it to obtain the mask for vertical edges >> sy, sx 3 sy = 1 2 1 0 0 0 -1 -2 -1 sx = 1 2 1 0 -1 0 -2 0 -1 Now that we have the masks for the horizontal and the vertical edge detection, we can calculate the convolution of an image with these masks, ie. estimate the gradient in x−direction and y−direction. An example >> >> >> >> >> [img,map]=imread(’moon.tif’); gx=conv2(img,sx,’same’); % estimation of gradient in x-direction gy=conv2(img,sy,’same’); % estimation of gradient in y-direction g=sqrt(gx.*gx+gy.*gy); % estimation of gradient in total imagesc(g) % display the obtained gradient 6.4.2 Problems 1. Read the image 4.2.07.tiff from the USC-SIPI database. Apply Sobel and Prewitt operator to this image to estimate its edges. Display the results and comment on them. 6.4.3 Compas operators for edge detection Simple gradient-based methods estimate the gradient in two directions. Compas operators detect edges leaned at a certain angle. Examples of the masks (of size 3 × 3) for angles 0, 45 and 90 degrees can be generated as explained: An example >> >> >> >> h0 h0=[1 1 1; 0 0 0; -1 -1 -1]; % mask for the angle of 0 degrees h45=[1 1 0; 1 0 -1; 0 -1 -1]; % mask for the angle of 45 degrees h90=[1 0 -1; 1 0 -1; 1 0 -1]; % mask for the angle of 90 degrees h0, h45, h90 = 1 1 1 0 0 0 -1 -1 -1 h45 = 1 1 0 4 1 0 -1 0 -1 -1 h90 = 1 0 -1 1 0 -1 1 0 -1 Convolution of an image with a selected compas operator results in an image with emphasized edges of a selected angle. 6.4.4 Problems 1. Read the image 4.2.07.tiff from the USC-SIPI database. Apply several compas operators on this image (for different directions). 2. What happens with the edges positioned in the direction that does not correspond to the specific operator? 3. Calculate and display the frequency characteristics for the selected compas operators. (This can be done with the function fft2() as described in the Exercise 3.) 4. Why is their resolution so small? 6.4.5 Laplacian operator for edge detection Gradient masks give the best result in cases when there are sudden edges, ie. large change in the pixel intensity values. For the ”softer” edges, methods for second derivative estimation give better results. An example of such estimation is the Laplacian operator. Laplacian of size 3 × 3 can be simply generated with function fspecial: An example >> lap = lap = 0 1 1 -4 0 1 6.4.6 fspecial(’laplacian’,0) % mask that approximates the Laplacian operator 0 1 0 Problems 1. Read the image 4.2.07.tiff from the USC-SIPI database. Apply several Laplacian operator approximations on this image (note: change the second input parameter of the fspecial() function). Display the results and comment on them. 5 6.5 Texture features Texture are present in almost every image. Even though we have an intuitive ”knowledge” about textures, despite their importance there is no formal definition of a texture. In this part of the exercise, we will use the second order histogram features to represent texture features. For that purpose, we will use the function nlfilter() to calculate each feature on the block (of the defined size) in the input image. An example >> >> >> >> >> >> >> [img,map] = imread(’testpat1.tif’); % read the image img = im2double(img); % convert it to double format p = [2,2]; % define the shift nhood = [15, 15]; % define the neighborhood size fun = @(x) inertia(x,p); % function pointer (function handle) img2 = nlfilter(img,nhood,fun); % process the image figure; imagesc(img2); colormap(gray) % display the resulting image You can notice that for calculating the second order histogram, we need to define a shift vector, p = [Dy , Dx ], which corresponds to the shift between the two pixels under observation. This can be a problem since even for the small neighborhoods (eg. neighborhood size = [M,N]), the number of possibilities is very large. Therefore, to reduce the calculation time, select the images with smaller dimension and define the smaller neighborhood sizes. Also, have in mind that the shift vector [Dy , Dx ] has to be smaller than the neighborhood size. 6.5.1 Problems 1. Read the image texture.tif, which contains 5 different areas: 3 textures and 2 plain areas. Calculate the following features: absolute(), inertia(), covariance(), energy(), entropy() (with the corresponding functions) for these shifts: [2,2], [3,3] and [3,5]. Define the block size as 12 × 12. Display the results and comment on them. 2. Select few images that display different textures (you can crop the image to work with images of smaller dimensions). Select a single feature and calculate it for each of the chosen images. 3. Do the features exhibit a sufficient difference to use them for the texture classification? 4. Which feature seems the best to perform the classification of the textures in the image? 6