Feature detection and estimation:
Differential geometric structures
in Images
Davi Geiger and Rosalba Giugno
INTRODUCTION TO FEATURE DETECTION
~
~
We have obtained image measurements such as D Iˆ( x, y, , s) , D I ( x, y, , s) , H ( x, y, s ) and
D Iˆ( x, y, ,  , s) . We now wish to make detections and estimations, which are decisions

based upon our models. Decisions are the result of some comparison followed by a choice.
Examples (i) if a measurement is above a threshold we accept, not otherwise; (ii) if a measurement is the largest compared to others, we select it.
In particular we will discuss edge detection, orientation, curvature, and junction detection
and classification.
1. Edgels: edge detection, edge orientation (tangent)
In order to detect an intensity edge we require that there is a value DIˆ( x, y, , s) in the
set Iˆ( x, y ) above some threshold value, say T . In order to detect the orientation of a detected
e
edge element, at a given scale, we look for the angle  that simultaneously maximizes the value DIˆ( x, y,   / 2, s ) in magnitude and minimizes the value DIˆ( x, y, , s ) in magnitude.
This is essence our model of extracting/detecting the orientation of a hypothesized contour
present at a pixel (x,y). It requires a decision, and one possible criteria is to extract the orientation, at a given scale, via

s
max
( x, y)  arg max  ( | DIˆ( x, y,   / 2, s) |  | DIˆ( x, y, , s) | )
where only eight (8) directions are considered, and for locations where
there is an angle  such that
( | DIˆ( x, y,   / 2, s) |  | DIˆ( x, y, , s) | )  Te
Figure 1 (right). Edgels Results of detecting and estimating 
s
max
( x, y) .
Dark levels represent 0 degrees orientation, while the brightest values
represent 
1
~
Figure 2. Shows a step edge at  The value | DI ( x, y,   / 2, s ) | is (equally) large for both and
as shown (in red) for scale s=3 pixels. It is also large for values not shownHowever, the quan-
~
~
tity | DI ( x, y,   / 2, s ) |  | DI ( x, y, , s ) | is significantly larger for 



~
Figure 3. Shows a step edge at  The value | DI ( x, y,   / 2, s ) | at “edge” locations is (equally)
large for both and as shown (in red) for scale s=3 pixels. It is also (equally) large for values
~
~
not shown)However, the quantity | DI ( x, y,   / 2, s ) |  | DI ( x, y, , s ) |
is significantly
larger for 




Figure 4. Shows a step edge at  As the measurements are made away from the location of the edge, The quan-
~
~
tity | DI ( x, y,   / 2, s ) |  | DI ( x, y, , s ) | decreases, and also spurious response are obtained, as in this
case the response is equal for both and homogeneity measure will detect these locations.
2

We can further filter out spurious detections by removing locations where
~
H ( x, y, s)  TH ,
and TH is a threshold to be estimated. We can define a routine called Edgel(x,y,s) which returns
NIL
if
the
location
does
not
pass
either
the
ˆ
ˆ
test ( | DI ( x, y,   / 2, s) |  | DI ( x, y, , s) | )  Te or the homogeneity test, and returns
 s ( x, y ) and | DIˆ( x, y,   / 2, s ) |  | DIˆ( x, y, , s ) | otherwise.
max
Pseudocode
Edgel(x,y,s)
D max  0
π
5π
for θ  0, , ...,
6
6
DÎ  ( | DÎ(x, y, θ  π/2, s) |  | DÎ(x, y, θ, s) | )
if (DÎ  D max )
θ smax  θ
D max  DÎ
~
if ( ( D max  Te ) and ( H(x, y, s)  TH ) )
return ( θ smax , D max )
else NIL
end
Note that 
s
max
( x, y ) is selected among eight (8) directions from 
2. Curvatures or changes of Edgel orientations
We use the concept of the second derivatives D Iˆ( x, y, ,  , s) to construct a detector of angle
changes (which is related to the curvature of a contour passing by pixel (x,y), as we will describe). Consider the quantity (change of angle)
 NIL
 s max ( x, y )   s
s
 max ( x N , y N )   max ( x, y )
where
if Edgel( x N , y N , s )  NIL
otherwise
,
x N  x  x s max cos s max ( x, y), and y N  y  y s max sin  s max ( x, y) and x and y will
vary according to as
x  1 and y  0
for 
3
for 
x  0 and y  1
for 
x  2 and y  2
4
and y  2
for 
x 
3
4
x  2 and y 
for 
3
Note that the displacement length changes for different angles and for some angles such as
 the displacement is much larger in length as to reach a center of another pixel. The
s
( x, y ) quantity can be best understood in figure 2.
change in angle  max
s
 max
( x, y ) 

contour
2
ontour
 s max ( x   x cos  , y   y sin  ) 
y

2
s
   max
( x, y )

Figure 2. The geometrical quantity
s
max
( x, y)
 s max ( x, y) provides an estimation of how does the hypothetical contour
x
present in a pixel changes its tangent angle. It is proportional to the curvature of the contour.
We can make the estimate symmetric by averaging over both sides of the pixel (x,y). More
precisely, by considering
s
 s max ( x, y)  s max ( x  x cos  , y  y sin  )   max
( x  x cos  , y  y sin  )
,
where    s max ( x, y) . From  s max ( x, y) we can readily compute the curvature of the hypothetical contour passing by pixel (x,y) as
 s max ( x, y )

where  | ( x cos  ,  y sin  ) | (2 x cos 2   2 y sin 2  )1 / 2 .
 ( x, y, s) 
4
3. Junctions and Corners
For each pixel location we examine how well a simple junction model can fit the data. We
~
examine locations where large values of D I ( x, y, , s) occurs, i.e., where there is
~
D I ( x, y, , s)  T
for some and T is a threshold that needs to be estimated. We can further filter out spurious
detections by removing locations where
~
H ( x, y, s)  TH ,
and TH is a threshold to be estimated. Note that this filter also responds to edges, and so let us
further filter out the junctions by characterizing them. Once a junction is detected one can also
~
compute the number n of directions  such that D I ( x, y, , s) is above the threshold T.
Thus, Corners or L-junctions will have n=2, T-junctions and L-Junctions will have n=3, Xjunctions will have n=4. More precisely, corners are expected to yield only two different an~
~
gles,  1 and  2 , such that D I ( x, y,1 , s)  T and D I ( x, y, 2 , s)  T . Moreover we must require that
min(  2 | 1  1 | , 2   2 )  5 ,
6
otherwise it is an edgel, not a corner. It is possible to detect T-junction or Y-junctions, where
~
there are three distinct angles with D I ( x, y, , s)  T , i.e., there are three distinct regions
 2 ,  3 , 2   2   3 . In these cases the specific angles will define if it is a T- or Yjunction. T-junctions will have one    while Y-junctions will not. T-junctions and Y-
junctions usually characterize an occlusion, a surface on top of other surfaces. In this notation,
corners are many times called L-junctions. X-junctions correspond to four angles detected such
~
that D I ( x, y, , s)  T and they usually characterize a transparent surface intersecting other
surfaces.
~
We next present a pseudocode for Junction detection and classification. The Input is Dθ I .
5
for Junctions Pseudocode
~
Junctions( Dθ I )
j=1
for y  s …N-s
for x  s …N-s
n(j)=0 ;
list- θ( j) =NIL ;
~
if ( H(x, y, s)  TH )
for i  1,2, ...,16
 (loop for the sixteen different angles)
~
if ( | Dθ I (x, y, θ i , s) |  Tθ )
n(j)  n(j)  1 ;
θn j  θi ;
add θ n j  list- θ( j) ;
 (end loop for i)
end
if (n(j)  2)
if (n(j) = 2)
Δθ  θ 2 j  θ1 j ;
 (check it is corner not edgel)
if ( MIN (Δ , 2 - Δ θ )  5
6
)
continue ;
x(j)  x ;
y(j)  y ;
j=j+1 ;
 (end loop for x)
 (end loop for y)
end
end
return (j, x(j), y(j), n(j), list- θ( j) )
end
Note that the final value of j carries the total number of junctions detected, while x(j), y(j) n(j)
and list- θ( j) are arrays with entries for each junction.
4. Connecting Edgels
Say we have detected edgels, and we wish to connect them to construct a contour representation. We are now moving from features represented at the pixel level to larger objects such as
contours.
Let us examine a simple recursive algorithm to link edgels. Say we start with an edgel located
at (xc , yc) and with orientation of max(xc , yc). The idea of the algorithm is to examine the pixel
neighbors defined by the orientation of the centered pixel and check if they are edgels, and if
they are, link them and recursively apply the algorithm to the neighbors. So a pseudocode
would be
6
Link-neighbors+(xc , yc ,max)
xn+ = xc +  xmax cosmax ;
yn+ = yc +  ymax sinmax ;
if (Edgel(xn+ , yn+)  NIL)
Link((xc , yc), (xn+ , yn+))
Link-neighbors+( xn+ , yn+ ,max(xn+ , yn+) )
end
Analogously we construct a routine that link neighbors in the other direction, more precisely
Link-neighbors-(xc , yc ,max)
xn- = xc -  xmax cosmax ;
yn- = yc -  ymax sinmax ;
if (Edgel(xn- , yn-)  NIL)
Link((xc , yc), (xn- , yn-))
Link-neighbors-( xn- , yn- ,max(xn- , yn-) )
End
To apply a contour follower, it would have to select a seed to start. One possible approach is,
given a seed we apply
Contour-Follower(xc , yc)
if (Edgel(xc , yc ))
Link-neighbors+(xc , yc ,max)
Link-neighbors-(xc , yc ,max)
end
One possible method of selecting such a seed is with the detection of junctions and following
them. Here is how a contour follower could be designed
~
Contour-Follower( D I )
~
Junctions( D I )
for i  1,2, ...,j
for m  1,2, ...,n(j)
if (  m i  5
6
)
Link-neighbors-(x(j) , y(j) ,m i)
else
Link-neighbors+(x(j) , y(j) ,m i)
end
 (end loop for m)
end
 (end loop for i)
7