Acta Mathematica Academiae Paedagogicae Nyı́regyháziensis
31 (2015), 5–14
www.emis.de/journals
ISSN 1786-0091
NOTE ON THE RIEMANNIAN GEOMETRY OF IMAGE
PROCESSING
HOWARD E. BRANDT
Dedicated to the distinguished Finsler geometer Prof. Dr. Lajos Tamássy
on his 90th birthday
Abstract. A brief review is given of the Riemannian geometry of image
processing. Methods are exposited for treating images as Riemannian manifolds in general embedding spaces for application to linear and nonlinear
scale-space image processing. In particular, the Beltrami operator approach
to obtaining a nonlinear differential equation for processing of gray-scale
images is treated in some detail. A minor error is found in earlier work
appearing in the literature.
1. Introduction
In the Riemannian geometry of image processing, differential geometric
methods are exploited for treating images as Riemannian manifolds in general Euclidean embedding spaces for applications in linear and nonlinear scalespace image processing, and for developing improved mathematical procedures
for enhancement, smoothing, and segmentation of multi-spectral and texture
images [4]-[2]. In the present work, the Beltrami operator approach is reviewed
in some detail for obtaining a nonlinear differential equation for processing of
gray-scale images. A minor error is found in earlier work appearing in the
literature [3]
2. Gray-scale image processing
A gray-scale image can be represented by a two-dimensional curved surface
in which the light intensity I(x, y) is given as a function of coordinates x
and y. The surface is referred to as the image manifold Σ which is taken to
be a Riemannian manifold embedded in a higher dimensional spatial-feature
manifold M . One has the following embedding map:
2010 Mathematics Subject Classification. 68U10.
Key words and phrases. Image processing, scale space, Riemannian geometry, Beltrami
flow, harmonic map.
5
6
HOWARD E. BRANDT
(1) (X 1 (x1 , x2 ) = x1 , X 2 (x1 , x2 ) = x2 , X 3 (x1 , x2 ) = I(x1 , x2 ))
≡ (x1 , x2 , I(x1 , x2 )) ≡ (x, y, I(x, y))).
Here, X i , i = 1, 2, 3 are the embedding-space coordinates, and x1 ≡ x and
x2 ≡ y are the image-space coordinates. The line element in the isometric
three-dimensional embedding space is
(2) ds2 = ηij dX i dX j = ηij ∂µ X i ∂ν X j dσ µ dσ ν = gµν dxµ dxν ,
{i, j = 1 − 3}, {µ, ν = 1, 2},
in which ηij is the metric in the embedding-space manifold, {∂µ } ≡ {∂1 , ∂2 } ≡
{∂/∂x1 , ∂/∂x2 } = {∂/∂x, ∂/∂y}, and gµν is the metric in the two-dimensional
image-space manifold. For the rectilinear flat Euclidean embedding-space manifold which is treated below, one has
1, i = j
(3)
ηij = δij =
,
0, i 6= j
where δij is the Kronecker delta. In this case then the image-space metric is
(4)
gµν = δij ∂µ X i ∂ν X j = ∂µ x1 ∂ν x1 + ∂µ x2 ∂ν x2 + ∂µ I(x, y)∂ν I(x, y).
Then
(5)
g11 = ∂1 x1 ∂1 x1 + ∂1 x2 ∂1 x2 + ∂1 I(x, y)∂1 I(x, y) = 1 + (∂x I)2 ,
(6)
g12 = g21 = ∂1 x1 ∂2 x1 + ∂1 x2 ∂2 x2 + ∂1 I(x, y)∂2 I(x, y) = ∂x I∂y I,
(7)
g22 = ∂2 x1 ∂2 x1 + ∂2 x2 ∂2 x2 + ∂2 I(x, y)∂2 I(x, y) = 1 + (∂y I)2 ,
or in matrix form:
(8)
1 + Ix2 Ix Iy
[gµν ] =
,
Ix Iy 1 + Iy2
∂I
∂I
and Iy ≡ ∂y
.
where Ix ≡ ∂x
Given an input image I(0) for scale parameter t = 0, image information is
to be processed locally and forwarded to t + dt. The output image I(t) will be
a solution to a differential equation of the form:
(9)
∂t I = OI,
where O is a local (generally nonlinear) differential operator, and the initial
condition is given by the input image I(0). The measure on the space of
embedding manifolds is given by the following functional integral [4], [3]:
Z
i
√
(10)
S X , gµν , hij = d2 x gg µν ∂µ X i ∂ν X j hij ,
where d2 x ≡ dxdy, and for a given gray-scale image in a rectilinear embedding
space, one has Eq. (3).
RIEMANNIAN GEOMETRY OF IMAGE PROCESSING
7
In the following, the normal vector to the two-dimensional image surface is
needed. A two-dimensional surface embedded in three-dimensional rectangular
Cartesian space with coordinates X, Y, Z can be represented by
(11)
G(X.Y.Z) = 0.
The normal to a surface plane at any point is given by
(12)
(∂G/∂x, ∂G/∂y, ∂G/∂z) .
The gray-scale image, expressed in terms of the image-space coordinates, is
given by
(13)
I = I(x, y),
and representing the image as a two-dimensional surface embedded in threedimensional space with (X, Y, Z) ≡ (x, y, I), one has
(14)
G(X, Y, Z) = I − I(x, y) = 0.
Therefore, the normal to the surface is given by the following vector:
(15)
(∂G/∂x, ∂G/∂y, ∂G/∂z) = (−Ix , −Iy , ∂I/∂I) = (−Ix , −Iy , 1).
One proceeds by varying the functional integral Eq. (10) with respect to
the embedding, keeping the image metric gµν constant since it is completely
determined by the image I(x, y), and is given by Eq. (8). One easily obtains
the corresponding inverse of the image metric, namely,
1 g22 −g12
1 1 + Iy2 −Ix Iy
µν
(16)
[g ] =
=
,
g −g12 g11
g −Ix Iy 1 + Ix2
where the determinant of the image metric is given by
(17)
g = 1 + Ix2 + Iy2 .
Also, one has the following:
1
√
(18)
∂x g = √ (Ix Ixx + Iy Iyx )
g
and
(19)
1
√
∂y g = √ (Ix Ixy + Iy Iyy ) .
g
Next, varying the functional integral Eq. (10) with respect to the embedding.
One has
Z
δ
δ
j
i δ
2 √
µν
i
j
∂
X
h
+
∂
X
hij
(20)
S
=
d
x
gg
∂
X
∂
X
ν
ij
µ
µ
ν
δX k (x0 )
δX k
δX k
i
j δ
+∂µ X ∂ν X
hij ,
δX k
in which the metric gµν in the image space is given and fixed, and the metric
in the embedding space hij is taken to be constant. Equation (20) becomes
8
(21)
Z
HOWARD E. BRANDT
δ
2 √
µν
2
0
S
=
d
x
gg
∂
δ
δ
(x
−
x
)
∂ν X j hij
µ
ik
δX k (x0 )
+ ∂µ X i ∂ν δkj δ 2 (x − x0 ) hij +∂µ X i ∂ν X j ∂k hij δ 2 (x − x0 ) ,
where δ 2 (x) is the two-dimensional Dirac delta function. Integrating by parts,
with the support for integration vanishing at the boundaries of the image,
Eq. (21) becomes
Z
√ µν
δ
j
2
0
2
(22)
gg
∂
X
h
δ
(x
−
x
)
S
=
d
x
∂
ν
kj
µ
δX k (x0 )
√ µν
−∂µ
gg ∂ν X j hkj δ 2 (x − x0 )
√ µν
+∂ν
gg ∂µ X j hkj δ 2 (x − x0 )
√ µν
−∂ν
gg ∂µ X i hik δ 2 (x − x0 )
√
+ gg µν ∂µ X i ∂ν X j ∂k hij δ 2 (σ − σ 0 )
√
√
= 0 − ∂µ gg µν ∂ν X j hkj − gg µν ∂ν X j ∂µ X l ∂l hkj
√ µν
√
+0 − ∂ν
gg ∂µ X i hik − gg µν ∂µ X i ∂ν X l ∂l hik
√
+ gg µν ∂µ X i ∂ν X j ∂k hij
√ µν
√
= −∂µ
gg ∂ν X j hkj − gg µν ∂ν X j ∂µ X l ∂l hkj
√ µν
√
−∂ν
gg ∂µ X j hkj − gg µν ∂µ X j ∂ν X l ∂l hjk
√
+ gg µν ∂µ X i ∂ν X j ∂k hij .
Next using the symmetries of the image-space and embedding-space metrics,
and renaming dummy indices, then Eq. (22) becomes
(23)
δ
√ µν
j
gg
∂
X
hkj
S
=
−2∂
ν
µ
δX k (σ 0 )
√
− gg µν ∂µ X j ∂ν X l (∂l hkj + ∂l hkj − ∂k hjl )
√
= −2∂µ gg µν ∂ν X j hkj
√
− gg µν ∂µ X j ∂ν X l (∂l hjk + ∂j hkl − ∂k hjl )
√
= −2∂µ gg µν ∂ν X j hkj
√
− gg µν ∂µ X j ∂ν X l (∂l hjk + ∂j hlk − ∂k hjl ) .
Therefore from Eq. (23), with both metrics positive definite, and substituting
Eq. (3), one conveniently has
1 δ
1
√
S = √ −2∂µ gg µν ∂ν X j hnk hkj
(24) √ hnk
k
0
2 g
δX (x )
2 g
√
−hnk (∂l hjk + ∂j hlk − ∂k hjl ) gg µν ∂µ X j ∂ν X l
1
√
= − √ ∂µ gg µν ∂ν X j δjn
g
RIEMANNIAN GEOMETRY OF IMAGE PROCESSING
9
1
− hnk (∂l hjk + ∂j hkl − ∂k hjl ) g µν ∂µ X j ∂ν X l .
2
Next in accord with Eqs. (9) and (24), and following convention, one writes [3]
(25)
dX n
1
δ
= − √ hnk
S ,
dt
2 g
δX k (σ 0 )
and substituting Eq. (24) in Eq. (25), one obtains for the scale-space evolution
of the image processing in terms of the scale-space parameter t:
dX n
1
√
(26)
= √ ∂µ ( gg µν ∂ν X n ) + Γnjl ∂µ X j ∂ν X l g µν ,
dt
g
where Γnjl is the Christoffel symbol of the second kind in the embedding space,
namely,
1
(27)
Γnjl = hnk (∂l hjk + ∂j hlk − ∂k hjl ) .
2
For the gray-scale image surface embedded in three-dimensional Euclidean
space with rectangular Cartesian coordinates, with the embedding-space metric given by Eq. (3), one sees by substituting Eq. (3) in Eq. (27) that the
Christoffel symbols in the embedding space are vanishing, namely,
Γnjl = 0.
(28)
In this case one obtains using Eqs. (26) and (28) the so-called Beltrami operator
equation for the evolution of the image processing [3]
(29)
dX n 1
√
= √ ∂µ ( gg µν ∂ν X n ) , n = 1, 2, 3,
dt
g
where it is to be understood that
(30)
{∂1 , ∂2 , } = {∂x , ∂y } .
It is well to note that the associated map is harmonic. Then according to
Eq. (29) for X 1 = x, in accord with Eq. (1), one has for the evolution of the x
coordinate of the image:
dx
1
√ µν
(31)
= √ ∂µ
gg ∂ν X 1
dt
g
1
1
√
√
= √ ∂µ ( gg µν ∂ν x) = √ ∂µ gg µ1 ∂1 x
g
g
1
√ µ1
= √ ∂µ
gg ∂x x
g
1
√ µ1 1 √ 11 √ 21 = √ ∂µ
gg = √ ∂x
gg +∂ y
gg
g
g
i
1 h √
√
√
√
= √ ∂x gg 11 + g∂x g 11 +∂ y gg 21 + g∂y g 21 .
g
10
HOWARD E. BRANDT
Next using Eqs. (16)-(19) in Eq. (31), one obtains
"
!
!
1 + I 2y
1 + I 2y
1
1
dx
√
(32)
= √ √ (Ix Ixx +I y Iyx )
+ g∂ x
dt
g
g
g
g
1
−I x Iy
−Ix Iy
√
+ √ (Ix Ixy +I y Iyy )
+ g∂y
g
g
g
1 = 3/2 (Ix Ixx +I y Ixy ) 1 + I 2y −2 (Ix Ixx +I y Ixy ) 1 + I 2y
g
+2gI y Ixy − (Ix Ixy +I y Iyy ) Ix Iy
1
+2 (Ix Ixy +I y Iyy ) Ix Iy −g (Iy Ixy +I x Iyy )] √ ,
g
or substituting Eq. (17) and simplifying, one obtains
1 dx
(33)
= 3/2 (Ix Ixx +I y Ixy ) 1 + I 2y −2 (Ix Ixx +I y Ixy ) 1 + I 2y
dt
g
+2 1 + I 2x +I 2y Iy Ixy − (Ix Ixy +I y Iyy ) Ix Iy +2 (Ix Ixy +I y Iyy ) Ix Iy
1
− 1 + I 2x +I 2y (Iy Ixy +I x Iyy ) √
g
1 = 3/2 Ix Ixx +I y Ixy +I x Iy2 Ixx +I 3y Ixy −2I x Ixx −2I y Ixy
g
−2I x Iy2 Ixx −2I 3y Ixy
+2I y Ixy +2I 2x Iy Ixy +2I 3y Ixy −I 2x Iy Ixy −I x Iy2 Iyy +2I 2x Iy Ixy
+2I x Iy2 Iyy −I y Ixy
1
−I x Iyy −I 2x Iy Ixy −I 3x Iyy −I 3y Ixy −I x Iy2 Iyy √
g
1
1 = 3/2 −I x Ixx −I x Iy2 Ixx +2I 2x Iy Ixy −I x Iyy −I 3x Iyy √
g
g
1
1
= 3/2 −I x Ixx 1 + I 2y +2I 2x Iy Ixy −I x Iyy 1 + I 2x √ .
g
g
Thus one has for the evolution of the x coordinate of the image surface:
(−I x )
dx
1 (34)
= 3/2 Ixx 1 + I 2y −2I x Iy Ixy +I yy 1 + I 2x
√ .
dt g
g
Next according to Eq. (29), one has for the evolution of the X2 = y coordinate of the image:
i
1 h √ 12 √
dy
√ 22 √
12
22
(35)
= √ ∂x gg + g∂x g +∂ y gg + g∂y g
dt
g
1
−I x Iy
1
−I x Iy
√
= √ √ (Ix Ixx +I y Ixy )
+ g∂ x
g
g
g
g
RIEMANNIAN GEOMETRY OF IMAGE PROCESSING
1 + I 2x
1
1 + Ix2
√
+ √ (Ix Ixy +I y Iyy )
+ g∂y
g
g
g
1
−I x Iy
1
= √ √ (Ix Ixx +I y Ixy )
g
g
g
√
g
− 2 2 (Ix Ixx +I y Ixy ) (−I x Iy )
g
1
1
1 + I 2x
+ √ (−I xx Iy −I x Ixy ) + √ (Ix Ixy +I y Iyy )
g
g
g
√
g
1
− 2 2 (Ix Ixy +I y Iyy ) 1 + Ix2 + √ (2Ix Ixy ) ,
g
g
or
(36)
dy
1
= 3/2 [(Ix Ixx +I y Ixy ) (−I x Iy ) −2 (Ix Ixx +I y Ixy ) (−I x Iy )
dt
g
+ 1 + I 2x +I 2y (−I xx Iy −I x Ixy ) + (Ix Ixy +I y Iyy ) 1 + I 2x
1
−2 (Ix Ixy +I y Iyy ) 1 + I 2x + 1 + I 2x +I 2y (2I x Ixy ) √
g
1 = 3/2 (Ix Ixx +I y Ixy ) Ix Iy − 1 + I 2x +I 2y (Ixx Iy +I x Ixy )
g
1
− (Ix Ixy +I y Iyy ) 1 + I 2x +2 1 + I 2x +I 2y Ix Ixy √
g
1
= 3/2 Ix2 Iy Ixx +I x Iy2 Ixy −I xx Iy −I x Ixy −I 2x Iy Ixx −I 3x Ixy −I 3y Ixx
g
−I x Iy2 Ixy −I x Ixy −I y Iyy −I 3x Ixy −I 2x Iy Iyy
1
+2I x Ixy +2I 3x Ixy +2I x Iy2 Ixy √ ,
g
or further simplifying, then
(37)
dy
1
= 3/2
dt
g
1
= 3/2
g
1
= 3/2
g
1
√
g
i 1 h
−I xx Iy (1 + I 2y ) − I yy Iy (1 + I 2x ) + 2I x Iy2 Ixy
√
g
h
i −I y
(1 + I 2y )I xx −2I x Iy Ixy +(1 + I 2x )I yy
.
√
g
−I xx Iy −I 3y Ixx −I y Iyy −I 2x Iy Iyy +2I x Iy2 Ixy
Thus one has for the evolution of the y coordinate of the image:
i −I dy
1 h
y
2
2
(38)
=
(1 + I y )I xx −2I x Iy Ixy +(1 + I x )I yy
.
√
dt g 3/2
g
11
12
HOWARD E. BRANDT
Next according to Eq. (29), one has for X 3 = I , the equation whose solution
yields the evolution of the image surface:
(39)
1
d
√
I = √ ∂µ ( gg µν ∂ν I)
dt
g
1 √ 11 √ 12 = √ ∂x
gg Ix +∂ x
gg Iy
g
1 √ 22 √ 21 + √ ∂y
gg Ix +∂ y
gg Iy
g
1
1
1
2
2
= √ ∂x √ 1 + I y Ix −∂ x √ Ix Iy
g
g
g
1
1
1
2
+ √ ∂y √ (−I x Iy )I x +∂ y √ (1 + I x )I y ,
g
g
g
or
dI
1
1
(40)
= √ − 3/2 (2I x Ixx +2I y Ixy ) 1 + I 2y Ix −I x Iy2
dt
g
2g
1 + √ Ixx +I 2y Ixx +2I x Iy Ixy −I xx Iy2 −2I x Iy Ixy
g
1
1
2
2
+ √ − 3/2 (2I x Ixy +2I y Iyy )(−I x Iy +I y +I x Iy )
g
2g
1 2
2
− √ 2I x Ixy Iy +I x Iyy −I yy −2I x Ixy Iy −I x Iyy
g
1 1 = − √ 3/2 (I x Ixx +I y Ixy )[ 1 + I 2y Ix −I x Iy2 ]− 1 + I 2x +I 2y Ixx
gg
1 1
− √ 3/2 (I x Ixy +I y Iyy )(−I 2x Iy +I y +I 2x Iy )
gg
+ 1 + I 2x +I 2y (−I yy )],
or
dI
1 1
= − √ 3/2 Ix2 Ixx +I x Iy Ixy +I 2x Iy2 Ixx +I x Iy3 Ixy
dt
gg
−I 2x Iy2 Ixx −I x Iy3 Ixy −I xx −I 2x Ixx −I xx Iy2 −I 3x Iy Ixy
−I 2x Iy2 Iyy +I x Iy Ixy +I 2y Iyy +I 3x Iy Ixy +I 2x Iy2 Iyy
−I yy −I 2x Iyy −I 2y Iyy )
1 1
= − √ 3/2 (2I x Iy Ixy −I xx −I xx Iy2 −I yy −I yy Ix2 ).
gg
RIEMANNIAN GEOMETRY OF IMAGE PROCESSING
13
Thus one has for the evolution of the image:
(41)
1
1 dI
= 3/2 1 + I 2x Iyy −2I x Iy Ixy + 1 + I 2y Ixx √ ,
dt g
g
which agrees with Eq. (36) of [2].
Summarizing Eqs. (34), (38), and (41), one has the following nonlinear differential equation for the gray-scale image flow:
−
→
−
→
dX
(42)
= HN,
dt
where the nonlinear operator H is given by
(43)
H≡
1 g 3/2
1 + I 2x Iyy −2I x Iy Ixy + 1 + I 2y Ixx ,
−
→
and the vector X is defined in terms of the x and y coordinates of the image,
and the image I itself:
 
x
−
→  
(44)
X= y ,
I
−
→
and the vector N is defined by
(45)

−Ix
−
→ 1 
−Iy  .
N =√
g
1

→
−
Comparing Eq. (45) with Eq. (15), and using Eq. (17), one sees that N is
the unit normal vector orthogonal to the image surface at any point. Also, it
can be shown that H in Eq. (43) is proportional to the mean curvature of the
image surface at any point. Equation (45) is in disagreement with Eq. (27) of
[3], since the latter erroneously has Ix interchanged with Iy in Eq. (45) above.
The error would of course result in erroneous processing of the image (unless
the image is such that Ix = Iy , which, of course, is not generally the case).
3. Conlusion
A brief review has been given of the Riemannian geometry of Image processing. Images are treated as Riemannian manifolds in general embedding
spaces for application to linear and nonlinear scale-space image processing.
The Beltrami operator approach to obtaining a nonlinear differential equation
for processing of gray-scale images is treated in some detail, and a minor error
is found in earlier work appearing in the literature.
14
HOWARD E. BRANDT
References
[1] G. Aubert and P. Kornprobst. Mathematical problems in image processing, volume 147
of Applied Mathematical Sciences. Springer-Verlag, New York, 2002. Partial differential
equations and the calculus of variations, With a foreword by Olivier Faugeras.
[2] R. Kimmel. Numerical geometry of images. Springer-Verlag, New York, 2004. Theory,
algorithms, and applications.
[3] N. Sochen, R. Kimmel, and R. Malladi. Report, lbnl-39213. Technical report, Lawrence
Berkeley National Laboratory, University of California, Berkeley, CA, August 1996.
[4] A. Spira, R. Kimmel, and N. Sochen. A short-time Beltrami kernel for smoothing images
and manifolds. IEEE Trans. Image Process., 16(6):1628–1636, 2007.
Received September 19, 2013.
United States Army Research Laboratory,
Maryland