advertisement

Image Measurements Davi Geiger INTRODUCTION TO IMAGE MEASURMENTS Images can be represented as surfaces, where I(x,y) can be thought as a surface (see figure 1). In this representation surface discontinuities represent edges. I1(x,y) I(x,y) Figure 1. Images as surfaces. Edges as surface discontinuities. It makes no sense to extract features at pixel level as the image is noisy and the discretization/quantization introduces artifacts. The minimum resolution in which we can look an image is an order of magnitude higher than a pixel, i.e., it is a few pixels, say 10 pixels. In an image there are various intensity discontinuities, and these are important information to preserve. Thus, when we are near intensity/surface discontinuities one must be careful not to mix signals from different surfaces, i.e., we cannot simply average the pixel values within a window. There are two basic distinct types of discontinuities: (i) edges (ii) junctions. Edges are located where two surface intensities meet, and junctions are located where edges meet/join and they can divide two or more surface intensities. There are two distinct low level processes with images. The first one is to obtain measures of images and the other one is to extract features such as edges and junctions. Measures are quantities (data manipulations) that can be observed in images and do not require any model about the scenes and do not require any decision process. For example, the average intensity value of an image is a measure of the image. The second quantity, extracting features, requires a model of features and a decision if such a model is present in the image. Decisions require comparisons and/or parameters/thresholds to evaluate if the fitting of the model to the data is above 1 such threshold, i.e., if a feature is present or not. Models can only be evaluated if appropriate measures of the image are available. How can we know if an image is bright or dark without computing the average intensity value? However, a decision if an image is bright or not requires a threshold parameter for brightness. Therefore, before we proceed to detect edges and junctions we elaborate on measures that will be useful to address the discontinuity problem (edges and junctions). 1. Filtering/Averaging Along Different Orientations We start by noting that a complete multiscale representation of images is given by the wavelet representation, while the Radon transform provides a complete representation of images according to intensity integration along different orientation. These representations effectively average pixel values of the original data and are general tools for representing the data. We here are interested on average according the orientations and also on eliminating pixel noise. ~ Intensity accumulation I: I ( x, y, , s ) For the first measure, for each pixel (x,y) we evaluate the accumulation of the intensity along a direction and at different scales. Typically we evaluate the accumulation for 16 directions, namely and at 5 scales, namely s= 3, 5, 7, 9. Since we will use this angles many times, we may refer to them as {=1,2,…,16}. In the continuous the line accumulation formula is given by s 1 ~ I ( x, y, , s) I ( cos , sin ) d s0 and when we discretize we obtain for s=3,5,7,9 (see figure 2.) 1 s 1 ~ I ( x, y, , s ) I ( x i x cos , y i y sin ) (1) s i 0 where, for x 1 and y 1 x 2 and y 2 2 and y 1 x 3 2 x 1 and y 3 where the subpixel values for for for 1 I ( x , y) 2 and 1 I ( x, y ) 2 are defined as 1 1 1 1 I ( x , y ) I ( x 1, y ) I ( x, y ) and I ( x, y ) I ( x, y 1) I ( x, y ) . 2 2 2 2 We point out that there are other techniques to discretize the integral above and can give more accurate approximations to the integral, but at a larger computational expense. For example, by tracing a line at angle one may consider the length of the line segment within each pixel (each box), divided by the total length, to weight the contribution of that pixel intensity for the sum/average. 2 In some applications we may prefer to remove the center pixel, I(x,y) and simply compute 1 s 1 ~ I c ( x, y , , s ) I ( x i x cos , y i y sin ) s 1 i 1 (2) s ~ 1 I ( x, y , , s ) I ( x, y ) s 1 s The approximate location of these quantities can be evaluated as s 1 s 1 1 1 ~ x I~ ( x, ) ( x i x cos ) , ~ y I~ ( y, ) ( y i y sin ) s i 0 s i 0 and we can compile the above information in a table as follows ~ x I~ ~y ~ I ~ x I~ ~y ~ I x + s/2 x + s/2 x + s/2 x + s/4 x x - s/4 x - s/2 x - s/2 y y + s/4 y + s/2 y + s/2 y + s/2 y + s/2 y + s/2 y + s/4 x - s/2 x - s/2 x - s/2 x - s/4 x x + s/4 x + s/2 x + s/2 y y - s/4 y - s/2 y - s/2 y - s/2 y - s/2 y - s/2 y - s/4 ~ Table 1. The values of x I~ and y I~ where the quantity I ( x, y , , s ) is located, for different angles /4 /3 / /3 /4 /6 /6 s=3 pixels s=7 pixels Figure 2. A 10 x 16 pixels lattice. 8 directions, where line accumulation ~ ( I ( x, y , , s ) ) are computed. The other 8 directions are obtained by adding to each of the 8 angle direction. ~ Fast Computation of I ( x, y, , s ) 3 ~ There is a very fast method to compute I ( x, y, , s) for an image I ( x, y ) of size N x N in O(N2) (it is independent of the size s of the window). ~ Compute - I ( x, y, , s ) : pseudo code for t 0 …N-1 Initialization ~ ~ ~ ~ SI (0, t ,0) SI (0, t , / 4) SI (0, t , ) SI (0, t , ) I (0, t ) 6 3 ~ SI (1, t , ) I (1, t ) 6 ~ ~ ~ ~ ~ ~ ~ SI (t , 0, ) SI (t , 0, ) SI (t , 0, ) SI (t , 0, ) SI (t , 0, 2 ) SI (t , 0, 3 ) SI (t , 0, 5 ) I (t ,0) 2 4 6 3 3 4 6 ~ ~ SI (t ,1, ) SI (t ,1, 2 ) I (t ,1) 3 3 ~ ~ ~ SI ( N 1, t , 3 ) SI ( N 1, t , 2 ) SI ( N 1, t , 5 ) I ( N 1, t ) 4 3 6 ~ 5 SI ( N 2, t , ) I ( N 2, t ) 6 (end loop for t) end for y 1 …N-2 for x 1 …N-2 ~ ~ SI ( x, y, 0) SI ( x 1, y, 0) I ( x, y ) ~ ~ SI ( x, y, ) SI ( x, y 1, ) I ( x, y ) 2 2 ~ ~ SI ( x, y, ) SI ( x 1, y 1, ) I ( x, y ) 4 4 ~ ~ SI ( x, y, 3 ) SI ( x 1, y 1, 3 ) I ( x, y ) 4 4 for s 3,5,7 if (y s and x s and x N-s-1) 1 ~ ~ ~ I ( x, y, , s ) SI ( x, y, 0) SI ( x s, y, 0) s ~ ~ I ( x s 1, y, 0, s ) I ( x, y, , s ) 1 ~ ~ ~ I ( x, y, 3 , s ) SI ( x, y, ) SI ( x, y s, ) 2 2 2 s ~ ~ I ( x, y s 1, , s ) I ( x, y, 3 , s ) 2 2 1 ~ ~ ~ I ( x, y, 5 , s ) SI ( x, y, ) SI ( x s, y s, ) 4 4 4 s ~ ~ I ( x s 1, y s 1, , s ) I ( x, y, 5 , s ) 4 4 1 ~ ~ ~ I ( x, y, 7 , s ) SI ( x, y, 3 ) SI ( x s, y s, 3 ) 4 4 4 s ~ ~ I ( x s 1, y s 1, 3 , s ) I ( x, y, 7 , s ) 4 4 4 end (end loop for s) if (y >1 and x > 1 and y < N-2 and x < N-2) 1 ~ ~ SI ( x, y, ) SI ( x 2, y 1, ) I ( x 1, y ) I ( x 1, y 1) I ( x, y ) 6 6 2 1 ~ ~ SI ( x, y, ) SI ( x 1, y 2, ) I ( x, y 1) I ( x 1, y 1) I ( x, y ) 3 3 2 1 ~ ~ 2 2 SI ( x , y , ) SI ( x 1, y 2, ) I ( x, y 1) I ( x 1, y 1) I ( x, y ) 3 3 2 1 ~ ~ SI ( x, y, 5 ) SI ( x 2, y 1, 5 ) I ( x 1, y ) I ( x 1, y 1) I ( x, y ) 6 6 2 for s 3,5,7 if (y s and x s and x N-s-1) 1 ~ ~ ~ I ( x, y, 7 , s ) SI ( x, y, ) SI ( x s 1, y ( s 1) , ) I ( x s 1, y ( s 1) ) 6 6 2 6 2 s ~ ~ I ( x s 1, y ( s 1) , , s ) I ( x, y, 7 , s ) 2 6 6 1 ~ ~ ~ I ( x, y, 4 , s ) SI ( x, y, ) SI ( x ( s 1) , y s 1, ) I ( x ( s 1) , y s 1) 3 3 2 3 2 s ~ ~ I ( x ( s 1) , y s 1, , s ) I ( x, y, 4 , s ) 2 3 3 1 ~ ~ ~ I ( x, y, 5 , s ) SI ( x, y, 2 ) SI ( x ( s 1) , y s 1, 2 ) I ( x ( s 1) , y s 1) 3 3 2 3 2 s ~ ~ I ( x ( s 1) , y s 1, 2 , s ) I ( x, y, 5 , s ) 2 3 3 1 ~ ~ ~ I ( x, y,11 , s ) SI ( x, y, 5 ) SI ( x s 1, y ( s 1) , 5 ) I ( x s 1, y ( s 1) ) 6 6 2 6 2 s ~ ~ I ( x s 1, y ( s 1) , 5 , s ) I ( x, y,11 , s ) 2 6 6 (end loop for s) (end loop for x) (end loop for y) end end end 5 ~ Also we can compute - I c ( x, y, , s) : pseudo code for y 1 …N-2 for x 1 …N-2 for s 3,5,7 if (y s and x s and x N-s-1) for s ~ 1 ~ I c ( x, y , , s ) I ( x, y , , s ) I ( x , y ) s 1 s end (end loop for s) end (end loop for x) end (end loop for y) ~ Homogenity measure: H ( x, y, s ) ~ It is clear that if an intensity discontinuity occurs along the value of I c ( x, y, , s) will vary significantly as s varies, where s ~ 1 ~ I c ( x, y , , s ) I ( x, y , , s ) I ( x , y ) . s 1 s ~ We can define the meaningful values of I ( x, y, , s ) as the ones for which the values do not vary significantly across s. ~ ~ ~ ~ R( x, y, , s) | I c ( x, y, , s) I c ( x, y, , s s) | | I c ( x, y, , s s) | where is a small parameter, typically =1, to make sure the division is stable. One can also ~ ~ think that R ( x, y, , s ) is computed from the gray values I c ( x, y, , s) offset by =1, i.e., ~ ~ I c ( x, y, , s) I c ( x, y, , s) + for every s. Then the normalized differences are computed to ~ obtain R ( x, y, , s ) . Next, we can average this quantity over all 16 angles and define homogeneity as ~ H ( x, y , s ) 11 6 ~ R ( x, y , , s ) (3) 0 ~ This measure will be used to insure that meaningful quantities of I ( x, y, , s) are being considered. ~ ~ The line representation of I ( x, y, , s ) or I c ( x, y, , s) is particularly useful to measure junctions, as we will discuss. However, for measuring edges/contours, one may want to have a more robust measure by further averaging on each orientation 6 Intensity Accumulation II : Iˆ( x, y, , s ) In order to obtain more robust measures, for an angle we can average the value of ~ I ( x, y, , s ) along as shown in figure 2 and figure 3 to obtain Iˆ( x, y, , s ) . The quantities Iˆ( x, y, , s ) are particularly useful to measure intensity edges along any direction when no junctions exist, as we will discuss next. /4 s=7 s=3 s=5 s=7 s=5 s=3 /3 s=7 /6 s=7 s=5 s=5 s=3 s=3 Figure 3. A 16 x 16 pixels lattice. We show four windows of 4 pixels width (represented by the darkest gray pixels). The reach of the pixels contributing for the average value of the window for different scales is shown as: s=3 pixels (dark gray), s=5 pixels (medium gray) and s=7 pixels (light gray) for directions The ~ value of Iˆ( x, y, , s ) are computed by averaging I ( x, y , , s ) over the four pixels window and the center (x,y) coordinate is shown as a white dot. The precise formula for Iˆ( x, y, , s ) for four pixels windows (figure 2) is given by, 1 ~ ~ ~ ~ Iˆ( x, y, , s ) = ( I ( x, y, , s ) I ( x, y 1, , s ) I ( x, y 1, , s ) I ( x 1, y, , s )) 4 for 1 ~ ~ ~ ~ = ( I ( x, y, , s ) I ( x 1, y, , s ) I ( x 1, y, , s ) I ( x, y 1, , s )) 4 for 1 ~ ~ ~ ~ = ( I ( x, y, , s ) I ( x 1, y, , s ) I ( x, y 1, , s ) I ( x 1, y 1, , s )) 4 for 1 ~ ~ ~ ~ = ( I ( x, y, , s ) I ( x 1, y, , s ) I ( x, y 1, , s ) I ( x 1, y 1, , s )) 4 for 7 1 ~ ~ ~ ~ ( I ( x, y, , s) I ( x 1, y, , s) I ( x, y 1, , s ) I ( x 1, y 1, , s )) 4 for 1 ~ ~ ~ ~ = ( I ( x, y, , s ) I ( x 1, y, , s ) I ( x, y 1, , s ) I ( x 1, y 1, , s )) 4 for (4) ~ ˆ Analogously we can compute I c ( x, y, , s) from I c ( x, y, , s) (equation 2). = s=7 /4 s=3 s=5 s=7 s=5 s=3 s=7 s=7 /6 /3 s=5 s=5 s=3 s=3 Figure 4. A 16 x 18 pixels lattice. We show four windows with width of 9 pixels (represented by the darkest gray). The reach of the pixels contributing for the average value of the window for different scales is shown as: s=3 pixels (dark gray), s=5 pixels (medium gray) and s=7 pixels (light gray) for directions ~ The value of Iˆ( x, y, , s ) are computed by averaging I ( x, y , , s ) over the four pixels window and the center (x,y) coordinate is shown as a white dot. ~ ~ Once I ( x, y, , s ) (or I c ( x, y, , s) ) and Iˆ( x, y, , s ) (or Iˆc ( x, y, , s) ) have been computed we can produce a rich description of an image. A description should be taught as a set of measurements of the image and are motivated by (i) human vision (ii) computer vision systems. We can also associate to the measurements the detections of features (such as edges, corners, junctions, curvature) by providing thresholds/parameters and making decisions according to how the measurements compare against these thresholds. 2. Measuring Intensity Changes - 1st and 2nd Derivatives Measuring first derivative: 8 In order to estimate intensity discontinuities we generate a set of measurements of the intensity derivatives at different orientations and different scales. We construct two perpendicular directions ̂ and ˆ , where ˆ (cos , sin ) and ˆ (-sin , cos ) in (x,y) coordinates. These are natural directions to take derivatives of Iˆ( x, y, , s ) . Thus, Iˆ( x, y, , s) Iˆ( x, y, , s) D Iˆ( x, y, , s) Iˆ( x, y, , s) (cos , sin ) cos sin x y Iˆ( x, y, , s) Iˆ( x, y, , s) , (5) where the second expression is a lattice approximation to the derivative. Analogously we ob~ tain DI ( x, y, , s) . The location of these derivatives are in the center of pixel (x,y). y x We have now a bank of eight (8) distinct oriented filters at different scales, namely Iˆ( x, y ) = { D Iˆ( x, y, , s) ; and s. The other 8 filters can be obtained straightforwardly, since from (4) D Iˆ( x, y, , s) D Iˆ( x, y, , s) . These filters respond to intensity discontinuities along as we will discuss next. D Iˆ( x, y,0 , 5) D Iˆ( x, y, , 5) 2 D Iˆ( x, y, , 5) 6 D Iˆ( x, y, , 5) 4 2 D Iˆ( x, y, , 5) 3 D Iˆ( x, y, 3 , 5) 4 D Iˆ( x, y, , 5) 6 5 D Iˆ( x, y, , 5) 6 Figure 4. Display of D Iˆ( x, y, , s) for the black and white image from figure 1, for the eight directions and scale s=5 pixels. Negative values are shown in dark and positive values in light. 9 /3 Figure 5. 16 x 16 pixels lattices. An intensity edge at an angle of The four pixels window measurement D Iˆ( x, y , 6 , s ) , shown at different locations, will have maximum response (highest value) among the possible orientations, while the measurement D Iˆ( x, y , 2 , s ) , not shown, will have the min3 imum response. The derivative along the other direction ˆ ( sin , cos ) will give us estimates for junctions (see figure 6). Junctions are location in images where different surface projection meet. Corners are special type of junctions where only two surfaces meet and they often appear in images caused by the intersection/projection of an object surface and its background surface. A corner will contain two discontinuities along ˆ ( sin , cos ) /4 /6 Figure 6. 16 x 16 pixels lattices. A junction between the angles and angle of The intensity accumula- ~ tion I ( x, y, , s ) provides reliable information to detect the junction. 10 Let us define the derivative along ˆ ( sin , cos ) ~ ~ D I c ( x, y, , s ) I c ( x, y, , s ) ( sin , cos ) ~ ~ I c ( x, y , , s ) I c ( x, y , , s ) where 6 12 (6) for for Measuring second derivative: ~ We may also want to look at how D Iˆ( x, y, , s) (or D I c ( x, y, , s) ) varies along In the continuous we make use of the Hessian matrix 2 Iˆ 2 Iˆ 2 x y x H ( x, y ) 2 ˆ 2 Iˆ 2 I y x y in order to compute the (second) derivative along a direction v̂ of a first derivative along direction we compute û Dvu Iˆ( x, y, , , s) vˆ H ( x, y) uˆ T . For example for 2ˆ uˆ (1,0) xˆ; vˆ (1,0) xˆ we obtain Dvu Iˆ( x, y, , , s) Dxx Iˆ( x, y, , , s) I 2 , for x 2ˆ uˆ (1,0) xˆ; vˆ (0,1) yˆ we obtain Dvu Iˆ( x, y, , , s ) D yx Iˆ( x, y, , , s ) I . When yx considering the (second) derivative along a direction of a (first) derivative along a direction we obtain a continuous and a lattice approximation as D Iˆ( x, y, , , s) (cos , sin ) H ( x, y) (cos , sin ) T DIˆ( x, y, , s) DIˆ( x x cos , y y sin , , s) where | ( x cos , y sin ) | (2 x cos 2 2 y sin 2 )1 / 2 . The location of the sec1 ond derivative is at ( x , y ) ( x Iˆ , y Iˆ ) (x cos , y sin ) . A common measurement of 2 the second derivative is along an edge orientation /2. (see figure in detail in power point presentation) y x x 11