Image Measurements
Davi Geiger
INTRODUCTION TO IMAGE MEASURMENTS
Images can be represented as surfaces, where I(x,y) can be thought as a surface (see figure 1).
In this representation surface discontinuities represent edges.
I1(x,y)
I(x,y)
Figure 1. Images as surfaces. Edges as surface discontinuities.
It makes no sense to extract features at pixel level as the image is noisy and the discretization/quantization introduces artifacts. The minimum resolution in which we can look an image
is an order of magnitude higher than a pixel, i.e., it is a few pixels, say 10 pixels. In an image
there are various intensity discontinuities, and these are important information to preserve.
Thus, when we are near intensity/surface discontinuities one must be careful not to mix signals
from different surfaces, i.e., we cannot simply average the pixel values within a window.
There are two basic distinct types of discontinuities: (i) edges (ii) junctions. Edges are located
where two surface intensities meet, and junctions are located where edges meet/join and they
can divide two or more surface intensities.
There are two distinct low level processes with images. The first one is to obtain measures of
images and the other one is to extract features such as edges and junctions. Measures are quantities (data manipulations) that can be observed in images and do not require any model about
the scenes and do not require any decision process. For example, the average intensity value of
an image is a measure of the image. The second quantity, extracting features, requires a model
of features and a decision if such a model is present in the image. Decisions require comparisons and/or parameters/thresholds to evaluate if the fitting of the model to the data is above
1
such threshold, i.e., if a feature is present or not. Models can only be evaluated if appropriate
measures of the image are available. How can we know if an image is bright or dark without
computing the average intensity value? However, a decision if an image is bright or not requires a threshold parameter for brightness. Therefore, before we proceed to detect edges and
junctions we elaborate on measures that will be useful to address the discontinuity problem
(edges and junctions).
1. Filtering/Averaging Along Different Orientations
We start by noting that a complete multiscale representation of images is given by the wavelet
representation, while the Radon transform provides a complete representation of images according to intensity integration along different orientation. These representations effectively
average pixel values of the original data and are general tools for representing the data. We
here are interested on average according the orientations and also on eliminating pixel noise.
~
Intensity accumulation I: I ( x, y,  , s )
For the first measure, for each pixel (x,y) we evaluate the accumulation of the intensity along a
direction and at different scales. Typically we evaluate the accumulation for 16 directions,
namely
and at 5
scales, namely s= 3, 5, 7, 9. Since we will use this angles many times, we may refer to them as
{=1,2,…,16}. In the continuous the line accumulation formula is given by
s
1
~
I ( x, y,  , s)   I ( cos  ,  sin  ) d
s0
and when we discretize we obtain for s=3,5,7,9 (see figure 2.)
1 s 1
~
I ( x, y, , s )   I ( x  i  x cos , y  i  y sin  )
(1)
s i 0
where,
for 
 x  1 and  y  1
 x  2 and  y  2
2
and  y  1
 x 
3
2
 x  1 and   y 
3
where
the
subpixel
values
for 
for 
for 
1
I ( x  , y)
2
and
1
I ( x, y  )
2
are
defined
as
1
1
1
1
I ( x  , y )  I ( x  1, y )  I ( x, y )  and I ( x, y  )  I ( x, y  1)  I ( x, y )  .
2
2
2
2
We point out that there are other techniques to discretize the integral above and can give
more accurate approximations to the integral, but at a larger computational expense. For example, by tracing a line at angle one may consider the length of the line segment within each
pixel (each box), divided by the total length, to weight the contribution of that pixel intensity
for the sum/average.
2
In some applications we may prefer to remove the center pixel, I(x,y) and simply compute
1 s 1
~
I c ( x, y ,  , s ) 
 I ( x  i  x cos , y  i  y sin  )
s  1 i 1
(2)
s ~
1


 I ( x, y ,  , s )  I ( x, y ) 
s 1
s

The approximate location of these quantities can be evaluated as
s 1
s 1
1
1
~
x I~ ( x, )   ( x  i  x cos ) , ~
y I~ ( y, )   ( y  i  y sin  ) 
s i 0
s i 0
and we can compile the above information in a table as follows

~
x I~
~y ~
I

~
x I~
~y ~
I








x + s/2
x + s/2
x + s/2
x + s/4
x
x - s/4
x - s/2
x - s/2
y
y + s/4
y + s/2
y + s/2
y + s/2
y + s/2
y + s/2
y + s/4








x - s/2
x - s/2
x - s/2
x - s/4
x
x + s/4
x + s/2
x + s/2
y
y - s/4
y - s/2
y - s/2
y - s/2
y - s/2
y - s/2
y - s/4
~
Table 1. The values of x I~ and y I~ where the quantity I ( x, y ,  , s ) is located, for different angles 
/4
/3
/
/3 /4
/6
/6
s=3 pixels s=7 pixels
Figure 2. A 10 x 16 pixels lattice. 8 directions, where line accumulation
~
( I ( x, y ,  , s ) ) are computed. The other 8 directions are obtained by adding to each of the 8 angle direction.
~
Fast Computation of I ( x, y, , s )
3
~
There is a very fast method to compute I ( x, y, , s) for an image I ( x, y ) of size N x N in
O(N2) (it is independent of the size s of the window).
~
Compute - I ( x, y, , s ) : pseudo code
for t  0 …N-1
Initialization
~
~
~
~
SI (0, t ,0)  SI (0, t ,  / 4)  SI (0, t ,  )  SI (0, t ,  )  I (0, t )
6
3
~
SI (1, t ,  )  I (1, t )
6
~
~
~
~
~
~
~
SI (t , 0,  )  SI (t , 0,  )  SI (t , 0,  )  SI (t , 0,  )  SI (t , 0, 2 )  SI (t , 0, 3 )  SI (t , 0, 5 )  I (t ,0)
2
4
6
3
3
4
6
~
~
SI (t ,1,  )  SI (t ,1, 2 )  I (t ,1)
3
3
~
~
~
SI ( N  1, t , 3 )  SI ( N  1, t , 2 )  SI ( N  1, t , 5 )  I ( N  1, t )
4
3
6
~
5

SI ( N  2, t ,
)  I ( N  2, t )
6

 (end loop for t)
end
for y  1 …N-2
for x  1 …N-2
~
~
SI ( x, y, 0)  SI ( x  1, y, 0)  I ( x, y )
~
~
SI ( x, y,  )  SI ( x, y  1,  )  I ( x, y )
2
2
 ~

~
SI ( x, y,  )  SI ( x  1, y  1,  )  I ( x, y )
4
4
~
~
SI ( x, y, 3 )  SI ( x  1, y  1, 3 )  I ( x, y )
4
4
for s  3,5,7
if (y  s and x  s and x  N-s-1)
1 ~
~
~
I ( x, y,  , s )  SI ( x, y, 0)  SI ( x  s, y, 0)
s
~
~
I ( x  s  1, y, 0, s )  I ( x, y,  , s )




1 ~
~
~
I ( x, y, 3 , s )  SI ( x, y,  )  SI ( x, y  s,  )
2
2
2
s
~
~
I ( x, y  s  1,  , s )  I ( x, y, 3 , s )
2
2
1
~
~
~
I ( x, y, 5 , s )  SI ( x, y,  )  SI ( x  s, y  s,  )
4
4
4
s
~
~
I ( x  s  1, y  s  1,  , s )  I ( x, y, 5 , s )
4
4
1
~
~
~
I ( x, y, 7 , s )  SI ( x, y, 3 )  SI ( x  s, y  s, 3 )
4
4
4
s
~
~
I ( x  s  1, y  s  1, 3 , s )  I ( x, y, 7 , s )
4
4



4

end
 (end loop for s)
if (y >1 and x > 1 and y < N-2 and x < N-2)
1
~
~
SI ( x, y,  )  SI ( x  2, y  1,  )  I ( x  1, y )  I ( x  1, y  1)   I ( x, y )
6
6
2
1
~
~
SI ( x, y,  )  SI ( x  1, y  2,  )  I ( x, y  1)  I ( x  1, y  1)   I ( x, y )
3
3
2

1
~
~
2

2

SI ( x , y ,
)  SI ( x  1, y  2,
)  I ( x, y  1)  I ( x  1, y  1)   I ( x, y )
3
3
2
1
~
~
SI ( x, y, 5 )  SI ( x  2, y  1, 5 )  I ( x  1, y )  I ( x  1, y  1)   I ( x, y )
6
6
2
for s  3,5,7
if (y  s and x  s and x  N-s-1)
1 ~
~
~
I ( x, y, 7 , s )   SI ( x, y,  )  SI ( x  s  1, y  ( s  1) ,  )  I ( x  s  1, y  ( s  1) ) 
6
6
2
6
2 
s
~
~
I ( x  s  1, y  ( s  1) ,  , s )  I ( x, y, 7 , s )
2
6
6
1 ~
~
~
I ( x, y, 4 , s )   SI ( x, y,  )  SI ( x  ( s  1) , y  s  1,  )  I ( x  ( s  1) , y  s  1) 
3
3
2
3
2

s
~
~
I ( x  ( s  1) , y  s  1,  , s )  I ( x, y, 4  , s )
2
3
3
1 ~
~
~
I ( x, y, 5 , s )   SI ( x, y, 2 )  SI ( x  ( s  1) , y  s  1, 2 )  I ( x  ( s  1) , y  s  1) 
3
3
2
3
2

s
~
~
I ( x  ( s  1) , y  s  1, 2 , s )  I ( x, y, 5 , s )
2
3
3
1 ~
~
~
I ( x, y,11 , s )   SI ( x, y, 5 )  SI ( x  s  1, y  ( s  1) , 5 )  I ( x  s  1, y  ( s  1) ) 
6
6
2
6
2 
s
~
~
I ( x  s  1, y  ( s  1) , 5 , s )  I ( x, y,11 , s )
2
6
6
 (end loop for s)
 (end loop for x)
 (end loop for y)
end
end
end


5
~
Also we can compute - I c ( x, y, , s) : pseudo code
for y  1 …N-2
for x  1 …N-2
for s  3,5,7
if (y  s and x  s and x  N-s-1)
for   
s ~
1
~

I c ( x, y ,  , s ) 
 I ( x, y ,  , s )  I ( x , y ) 
s 1
s

end
 (end loop for s)
end
 (end loop for x)
end
 (end loop for y)


~
Homogenity measure: H ( x, y, s )
~
It is clear that if an intensity discontinuity occurs along the value of I c ( x, y, , s) will vary
significantly as s varies, where
s ~
1
~

I c ( x, y ,  , s ) 
 I ( x, y ,  , s )  I ( x , y )  .
s 1
s

~
We can define the meaningful values of I ( x, y,  , s ) as the ones for which the values do not
vary significantly across s.
~
~
~
~
R( x, y, , s)  | I c ( x, y, , s)  I c ( x, y, , s  s) | | I c ( x, y, , s  s)   |
where  is a small parameter, typically =1, to make sure the division is stable. One can also
~
~
think that R ( x, y, , s ) is computed from the gray values I c ( x, y, , s) offset by =1, i.e.,
~
~
I c ( x, y, , s)  I c ( x, y, , s) +  for every s. Then the normalized differences are computed to
~
obtain R ( x, y, , s ) . Next, we can average this quantity over all 16 angles and define homogeneity as
~
H ( x, y , s ) 
11
6
~
 R ( x, y ,  , s )

(3)
0
~
This measure will be used to insure that meaningful quantities of I ( x, y, , s) are being considered.
~
~
The line representation of I ( x, y,  , s ) or I c ( x, y, , s) is particularly useful to measure junctions, as we will discuss. However, for measuring edges/contours, one may want to have a
more robust measure by further averaging on each orientation


6

Intensity
Accumulation II : Iˆ( x, y,  , s ) 
In order to obtain more robust measures, for an angle we can average the value of
~
I ( x, y,  , s ) along as shown in figure 2 and figure 3 to obtain Iˆ( x, y,  , s ) . The quantities Iˆ( x, y,  , s ) are particularly useful to measure intensity edges along any direction when
no junctions exist, as we will discuss next.


/4
s=7
s=3 s=5 s=7
s=5
s=3 /3
s=7
/6
s=7
s=5
s=5
s=3
s=3
Figure 3. A 16 x 16 pixels lattice. We show four windows of 4 pixels width (represented by the darkest gray pixels). The reach of the pixels contributing for the average value of the window for different scales is shown as: s=3
pixels (dark gray), s=5 pixels (medium gray) and s=7 pixels (light gray) for directions  The
~
value of Iˆ( x, y,  , s ) are computed by averaging I ( x, y ,  , s ) over the four pixels window and the center
(x,y) coordinate is shown as a white dot.
The precise formula for Iˆ( x, y,  , s ) for four pixels windows (figure 2) is given by,
1 ~
~
~
~
Iˆ( x, y,  , s ) = ( I ( x, y, , s )  I ( x, y  1, , s )  I ( x, y  1, , s )  I ( x  1, y, , s ))
4
for 
1 ~
~
~
~
= ( I ( x, y, , s )  I ( x  1, y, , s )  I ( x  1, y, , s )  I ( x, y  1, , s ))
4
for 
1 ~
~
~
~
= ( I ( x, y, , s )  I ( x  1, y, , s )  I ( x, y  1, , s )  I ( x  1, y  1, , s ))
4
for 
1 ~
~
~
~
= ( I ( x, y, , s )  I ( x  1, y, , s )  I ( x, y  1, , s )  I ( x  1, y  1, , s ))
4
for 
7
1 ~
~
~
~
( I ( x, y, , s)  I ( x  1, y, , s)  I ( x, y  1, , s )  I ( x  1, y  1, , s ))
4
for 
1 ~
~
~
~
= ( I ( x, y, , s )  I ( x  1, y, , s )  I ( x, y  1, , s )  I ( x  1, y  1, , s ))
4
for 
(4)
~
ˆ
Analogously we can compute I c ( x, y, , s) from I c ( x, y, , s) (equation 2).
=
s=7
/4
s=3 s=5 s=7
s=5

s=3
s=7
s=7 /6
/3
s=5
s=5
s=3
s=3
Figure 4. A 16 x 18 pixels lattice. We show four windows with width of 9 pixels (represented by the darkest
gray). The reach of the pixels contributing for the average value of the window for different scales is shown as:
s=3 pixels (dark gray), s=5 pixels (medium gray) and s=7 pixels (light gray) for directions 
~
The value of Iˆ( x, y,  , s ) are computed by averaging I ( x, y ,  , s ) over the four pixels window and the center
(x,y) coordinate is shown as a white dot.

~
~
Once I ( x, y,  , s ) (or I c ( x, y, , s) ) and Iˆ( x, y, , s ) (or Iˆc ( x, y, , s) ) have been computed we
can produce a rich description of an image. A description should be taught as a set of measurements of the image and are motivated by (i) human vision (ii) computer vision systems. We
can also associate to the measurements the detections of features (such as edges, corners, junctions, curvature) by providing thresholds/parameters and making decisions according to how
the measurements compare against these thresholds.
2. Measuring Intensity Changes - 1st and 2nd Derivatives
Measuring first derivative:
8
In order to estimate intensity discontinuities we generate a set of measurements of the intensity
derivatives at different orientations and different scales. We construct two perpendicular directions ̂
and ˆ , where ˆ  (cos  , sin  ) and ˆ  (-sin  , cos ) in (x,y) coordinates.
These are natural directions to take derivatives of Iˆ( x, y, , s ) . Thus,
Iˆ( x, y, , s)
Iˆ( x, y, , s)
D Iˆ( x, y, , s)  Iˆ( x, y, , s)  (cos  , sin  ) 
cos 
sin 
x
y
 Iˆ( x, y, , s)  Iˆ( x, y,    , s)
,
(5)
where the second expression is a lattice approximation to the derivative. Analogously we ob~
tain DI ( x, y, , s) . The location of these derivatives are in the center of pixel (x,y).

y





 x
We have now a bank of eight (8) distinct oriented filters at different scales, namely Iˆ( x, y ) =
{ D Iˆ( x, y, , s) ; and s. The other 8 filters can be obtained

straightforwardly, since from (4) D Iˆ( x, y,   , s)   D Iˆ( x, y, , s) . These filters respond to
intensity discontinuities along as we will discuss next.
D Iˆ( x, y,0 , 5)

D Iˆ( x, y, , 5)
2


D Iˆ( x, y, , 5)
6
D Iˆ( x, y, , 5)
4
2
D Iˆ( x, y,
, 5)
3
D Iˆ( x, y,
3
, 5)
4

D Iˆ( x, y, , 5)
6
5
D Iˆ( x, y, , 5)
6
Figure 4. Display of D  Iˆ( x, y, , s) for the black and white image from figure 1, for the eight directions and
scale s=5 pixels. Negative values are shown in dark and positive values in light.
9
/3
Figure 5. 16 x 16 pixels lattices. An intensity edge at an angle of The four pixels window measurement D  Iˆ( x, y , 

6
, s ) , shown at different locations, will have maximum response (highest value)
among the possible orientations, while the measurement D  Iˆ( x, y , 
2
, s ) , not shown, will have the min3
imum response.
The derivative along the other direction ˆ  ( sin  , cos ) will give us estimates for junctions (see figure 6). Junctions are location in images where different surface projection meet.
Corners are special type of junctions where only two surfaces meet and they often appear in
images caused by the intersection/projection of an object surface and its background surface. A
corner will contain two discontinuities along ˆ  ( sin  , cos ) 
/4
/6
Figure 6. 16 x 16 pixels lattices. A junction between the angles and angle of The intensity accumula-
~
tion I ( x, y, , s ) provides reliable information to detect the junction.
10
Let us define the derivative along ˆ  ( sin  , cos )
~
~
D I c ( x, y, , s )  I c ( x, y, , s )  ( sin  , cos  )
~
~
 I c ( x, y ,  , s )  I c ( x, y ,      , s )
where    
  

6

12
(6)
for 
for 

Measuring second derivative:
~
We may also want to look at how D Iˆ( x, y, , s) (or D I c ( x, y, , s) ) varies along In the
continuous we make use of the Hessian matrix
  2 Iˆ

 2 Iˆ
2

x y 

x
H ( x, y )   2 ˆ

 2 Iˆ 2 
 I
y x
y 

in order to compute the (second) derivative along a direction v̂ of a first derivative along direction
we
compute
û
Dvu Iˆ( x, y, ,  , s)  vˆ H ( x, y) uˆ T . For example for
2ˆ
uˆ  (1,0)  xˆ; vˆ  (1,0)  xˆ we obtain Dvu Iˆ( x, y, ,  , s)  Dxx Iˆ( x, y, ,  , s)   I 2 , for
x
2ˆ
uˆ  (1,0)  xˆ; vˆ  (0,1)  yˆ we obtain Dvu Iˆ( x, y, ,  , s )  D yx Iˆ( x, y, ,  , s )   I
. When
yx
considering the (second) derivative along a direction of a (first) derivative along a direction
 we obtain a continuous and a lattice approximation as
D Iˆ( x, y, ,  , s)  (cos  , sin  ) H ( x, y) (cos  , sin  ) T
 DIˆ( x, y, , s)  DIˆ( x    x cos  , y    y sin  , , s) 
where  | ( x cos  ,  y sin  ) | (2 x cos 2   2 y sin 2  )1 / 2 . The location of the sec1
ond derivative is at ( x , y )  ( x Iˆ , y Iˆ )  (x cos  , y sin  ) . A common measurement of
2
the second derivative is along an edge orientation /2.
(see figure in detail in power point presentation)
y

x
x


11