INF5300 Image Analysis
Contents
Object Shape Representation
• Basic region descriptors
• Area, perimeter and circularity
• From bit quads, chain codes and polygons
• Non-orthogonal moments
• Central moments
Lecture 27.02.2008
•
•
•
•
•
•
Fritz Albregtsen
• Literature:
• F. Albregtsen: ”Object shape descriptors”, v.1.0
20.02.2008
INF5300 Lecture 3
1
• Orthogonal moments
• Legendre, Chebyshev and Zernike moments
20.02.2008
What is shape
INF5300 Lecture 3
INF5300 Lecture 3
2
Assumptions
• Numerical description of spatial configurations in an image.
• No generally accepted methodology of shape description.
• Location and description of high curvature points give
essential information.
• Many useful, application-dependent heuristics.
• Shape description of 2D planar objects is “easy”.
• Shape is often defined in a 2D image, but its usefulness in
a 3D world depends on 3D -> 2D mapping.
• Invariance is an important issue.
20.02.2008
Moments of inertia
Radius of gyration
Object orientation
Best fitting ellipse
Scaling invariance
Hu’s invariant set of moments...
3
• We have a segmented, labeled image.
• Each object that is to be described has been identified.
• The image objects can be represented as
–
–
–
–
–
–
–
–
–
–
binary image (whole regions)
contour (region boundaries)
through a run length code
through a chain code
through a quad tree
in cartesian coordinates
in polar coordinates
in some other coordinates
as coefficients of some transform
...
20.02.2008
INF5300 Lecture 3
4
Considerations
Shape invariants
• Shape descriptors depend on viewpoint,
=> object recognition may often be impossible if
object or observer changes position.
• Shape description invariance is important
•
•
•
•
•
•
•
Input representation form, boundaries or whole regions?
Object reconstruction ability?
Incomplete shape recognition ability?
Local/global description?
Mathematical or heuristic techniques?
Statistical or syntactic object description?
Robustness of description to translation, rotation, and scale
transformations?
• Shape description properties in different resolutions?
– shape invariants represent properties which remain
unchanged under an appropriate class of transforms.
• Stability of invariants is a crucial property which
affects their applicability.
• The robustness of invariants to image noise and
errors introduced by image sensors is of prime
importance.
– description changes discontinously.
• Robustness against
– image noise
– geometric sampling (discretization)
– intensity quantization
20.02.2008
INF5300 Lecture 3
5
20.02.2008
Invariance of features
• Generally, the area is defined as:
– Position invariant
=> independence of the position of the object within the image.
– Scaling invariant
=> independence of the size of the object.
– Rotation invariant
=>independent of the orientation of the object.
– Warp invariant
=>independent of a “rubber-sheet” deformation of the object.
• In most cases we want position invariant features.
• The other depend on the application.
INF5300 Lecture 3
6
Area and perimeter
• Assume that we want to extract some object features.
• We may wish that the features are:
20.02.2008
INF5300 Lecture 3
A=
∫ ∫ I ( x, y )dxdy
XY
I(x,y) = 1 if the pixel is within the object, and 0 otherwise.
• In digital images:
A = ∑∑ I ( x, y ) ∆A
X
Y
∆A = area of one pixel. If ∆A = 1, area is simply measured in pixels.
• Area changes if we change the scale of the image
–
change is not perfectly linear, because of the discretization of the image.
• Area ≈ invariant to rotation (small discretization errors).
• Digital perimeter ≠ length of the original contour.
7
20.02.2008
INF5300 Lecture 3
8
Circularity and irregularity
Pattern matching - bit quads
• Circularity may be defined by C = 4πA/P2.
• C = 1 for a perfect continuous circle; betw. 0 and 1 forother shapes.
• Let n{Q} = number of matches between image pixels and pattern Q.
• Then area and perimeter of 4-connected object
is given by:
⎧0⎫
A = n{1 }, P = 2n{ 0 1 }+ 2n ⎨ ⎬
⎩1⎭
• In digital domain, C takes its smallest value for a
– digital octagon in 8-connectivity perimeter calculation
– digital diamond in 4-connectivity perimeter calculation
• Dispersion may be given as the major chord length to area
• Irregularity can be defined as:
π max ( xi − x )2 − ( yi − y )2
D=
A
(
Bit Quads handle 8-connected images:
)
• Gray (1971) gave area and the perimeter as
AG =
– where the numerator is the area of the centered enclosing circle.
• Alternatively, ratio of maximum and minimum centered circles:
2
2
max ⎛⎜ ( xi − x ) − ( yi − y ) ⎞⎟
I=
20.02.2008
⎝
min⎛⎜
⎝
⎠
•
⎠
9
The boundary is followed from a starting point in a given direction.
The number c indicates the direction to the next boundary pixel.
•
“Mid-crack” chain coding uses the mid-points of the sides of the square pixels.
•
Freeman (1970) computed the area and perimeter of the chain by
AF =
•
c
⎛
⎞
c ix ⎜ y i −1 + iy ⎟ , PF = n E + n O 2
∑
2⎠
⎝
i =1
Vossepoel and Smeulders (1982) improved length estimate by a corner count nC,
defined as the number of occurrences of consecutive unequal chain elements:
PK =
20.02.2008
π
8
(1 + 2 )(n
E
+ 2n O
•
)
INF5300 Lecture 3
10
C
The region can also be represented by n polygon vertices
where the sign of the sum reflects
the polygon orientation.
11
A = ∫∫ dxdy = ∫ xdy
S
20.02.2008
1
Aˆ =
2
INF5300 Lecture 3
N −1
∑ (x
k =0
k
0⎤
0⎥⎦
0⎤
0⎥⎦
1⎤ ⎡1 0⎤ ⎡1 1⎤
1⎥⎦ ⎢⎣1 1⎥⎦ ⎢⎣1 0⎥⎦
The surface integral over S (having contour C) is given by Green’s theorem:
Kulpa (1977) gave the perimeter as
.
0⎤ ⎡0
1⎥⎦ ⎢⎣1
0⎤ ⎡1
1⎥⎦ ⎢⎣1
⎡1 1⎤
Q4 : ⎢
⎥
⎣1 1⎦
⎡1 0⎤ ⎡0 1⎤
QD : ⎢
⎥⎢
⎥
⎣ 0 1 ⎦ ⎣1 0 ⎦
INF5300 Lecture 3
s := 0.0;
n := n + 1;
pkt[n].x := pkt[1].x;
pkt[n].y := pkt[1].y;
for i:=2 step 1 until n do
begin
dy := pkt[i].y - pkt[i-1].y
s := s + (pkt[i].x + pkt[i-1].x)/2 * dy;
end;
area := if (s > 0) then s else -s;
N
PVS = 0.980 n E + 1.406n O − 0.091 n C
•
1⎤ ⎡0
0⎥⎦ ⎢⎣0
1⎤ ⎡0
1⎥⎦ ⎢⎣1
Object area from contour
where N is the length of the chain, cix and ciy are the x and y components of the ith chain
element ci (cix, ciy = {1, 0, -1} indicate the change of the x- and y-coordinates), yi-1 is the
y-coordinate of the start point of . nE is the number of even chain elements and nO the
number of odd chain elements.
•
0⎤ ⎡ 0
0⎥⎦ ⎢⎣0
⎡1 1 ⎤ ⎡ 0
Q2 : ⎢
⎥⎢
⎣0 0⎦ ⎣0
⎡1 1⎤ ⎡0
Q3 : ⎢
⎥ ⎢
⎣0 1⎦ ⎣1
1
7
1⎡
⎤
[n{Q2 }+ n{Q3}+ 2 n{QD }]
AD = ⎢n{Q1}+ 2n{Q2 }+ n{Q3 }+ 4n{Q4 }+ 3 n{QD }⎥ , PD = n{Q2 }+
2
4⎣
2
⎦
20.02.2008
Chain code
•
•
0⎤
0⎥⎦
• More accurate formulas by Duda :
(xi − x )2 − ( yi − y )2 ⎞⎟
INF5300 Lecture 3
1
[n{Q1}+ 2n{Q2 }+ 3 n{Q3}+ 4n{Q4 }+ 2 n{QD }], PG = n{Q1}+ n{Q2 }+ n{Q3}+ 2 n{QD }
4
⎡0
Q0 : ⎢
⎣0
⎡1
Q1 : ⎢
⎣0
y k +1 − x k +1 y k
12
)
Recursive boundary splitting
•
1.
2.
3.
4.
5.
•
•
•
•
Sequential polygonization
Draw straight line between contour points that are farthest apart.
These two points are the initial breakpoints.
For each intermediate point:
Compute the point-to-line distance
Find the point with the greatest distance from the line.
If distance is greater than given threshold, we have a new breakpoint.
The previous line segment is replaced by two, and 1-4 above is
repeated for each of them.
The procedure is repeated until all contour points are within the
threshold distance from a corresponding line segment.
The resulting ordered set of breakpoints is then the set of vertices of a
polygon approximating the original contour.
This is probably the most frequently used polygonization method.
Since it is recursive, the procedure is fairly slow.
20.02.2008
INF5300 Lecture 3
• Start with any contour point as first “breakpoint”.
• Step through ordered sequence of contour points.
• Using previous breakpoint as the current origin,
area between contour and approximating line is:
1
Ai = Ai −1 + ( yi xi −1 − xi yi −1 ), si = xi2 + yi2
2
• Let previous point be new breakpoint if
– area deviation A per unit length s of approximating line segment
exceeds a specified tolerance, T.
• If |Ai|/si < T, i is incremented and (Ai, si) is recomputed.
• Otherwise, the previous point is stored as a new breakpoint,
and the origin is moved to new breakpoint.
• This method is purely sequential and very fast.
13
20.02.2008
A comparison of methods
•
We have tested the methods on circles, R ={5,…,70}.
•
Area estimator :
•
Perimeter estimator :
INF5300 Lecture 3
Statistical moments
•
The general form of a moment of order (p + q), over image plane ξ is:
m pq =
– Duda is slightly better than Gray.
∫∫ξ ψ
pq
( x, y ) f ( x, y )
where the weighting kernel or basis function is ψpq.
– Kulpa is more accurate than Freeman.
•
Circularity :
•
Errors and variability largest when R is small.
•
Best area and perimeter not computed simultaneously.
•
Gray’s area can be computed using discrete Green’s
theorem, suggesting that the two estimators can be
computed simultaneously during contour following.
– Kulpa’s perimeter and Gray’s area gave the best result.
20.02.2008
14
INF5300 Lecture 3
•
•
This produces a weighted description of f(x, y) over ξ.
The basis functions may have a range of useful properties.
•
In digital images, moments must be expressed in discrete form.
– Assume square pixels of area A with constant intensity I.
– So if Px,y is a discrete pixel value then:
Pxy = I ( x, y ) ∆A
– Thus,
M xy =
∑∑ψ ( x, y ) P( x, y ) ;
x
•
15
p, q = 0, 1, 2, ..., ∞
y
Choice of basis function depends on application and desired properties.
20.02.2008
INF5300 Lecture 3
16
Non-orthogonal moments
Low order Cartesian moments
• Continuous 2D (p + q)-th order Cartesian moment defined as:
m pq =
∞ ∞
∫ ∫x
p
q
y f ( x, y )dxdy
− ∞− ∞
• We assume that
– f(x, y) is piecewise continuous, bounded, having non-zero values
only in the finite region of the xy plane.
• Then, moments of all orders exist and “uniqueness theorem” holds:
Moment sequence mpq with basis xpyq is uniquely defined by f(x, y)
and f(x, y) is uniquely defined by mpq.
• Discrete version of Cartesian moment for image with pixels Pxy :
m pq =
M −1 N −1
∑∑x
p
y q P( x, y )
x =0 y =0
• mpq is a two dimensional Cartesian moment, where M and N are the
image dimensions and the monomial product xpyq is the basis function.
20.02.2008
INF5300 Lecture 3
17
Central moments
µ pq =
∑∑ (x − x ) ( y − y )
p
q
x =0 y =0
• The two first order moments are used to find
the Centre Of Mass (COM) of an image.
• If applied to a binary image and normalized by m00,
the result is the centre co-ordinates of the object:
m10 =
M −1 N −1
∑∑ x f ( x, y ),
m01 =
x =0 y =0
20.02.2008
M −1 N −1
∑∑ y f ( x, y ),
x=
x =0 y =0
INF5300 Lecture 3
m10
,
m00
y=
18
• Moments µpq (p + q ≤ 3) are given by mpq by:
f ( x, y )
x =0 y =0
x=
m10
m
, y = 01
m00
m00
Central moments from mpq
• The definition of a 2D discrete central moment is:
M −1 N −1
• The zero order moment m00 is defined as the total mass
(or power) of the image.
• For a binary M ×N image, this gives
M −1 N −1
m 00 = ∑ ∑ f ( x, y )
the number of pixels in the object:
• The 3D µpqr, are expressed by mpqr:
m01
m00
p
q
r
µ00 = m00 , µ10 = 0, µ01 = 0
µ20 = m20 − x m10
µ02 = m02 − ym01
µ11 = m11 − ym10
µ30 = m30 − 3x m20 + 2 x 2m10
⎛ p⎞ ⎛ q⎞ ⎛ r ⎞
µ pqr = ∑∑∑ − 1[D − d ] ⎜⎜ ⎟⎟ ⎜⎜ ⎟⎟ ⎜⎜ ⎟⎟ ∆x p − s ∆y q −t ∆z r −u m stu µ12 = m12 − 2 ym11 − x m02 + 2 y 2m10
S =0 t =0 u =0
⎝ s ⎠ ⎝ t ⎠ ⎝u⎠
• Corresponds to computing ordinary Cartesian moments
after translating object so that origin = center of mass.
µ21 = m21 − 2 x m11 − ym20 + 2 x 2m01
• where
µ
• D = (p + q + r); d = (s + t + u)
• and the binomial coefficients are given by
03
=> Central moments are invariant under translation.
v!
⎛v⎞
, w<v
⎜⎜ ⎟⎟ =
⎝ w ⎠ w ! (v − w )!
• Central moments are not scaling or rotation invariant.
20.02.2008
INF5300 Lecture 3
= m03 − 3x m02 + 2 y 2m01
19
20.02.2008
INF5300 Lecture 3
20
Moments of inertia
Parallel-axis theorem
• The two second order central moments
µ20 =
M −1 N −1
M −1 N −1
x =0 y =0
x =0 y =0
•
– moment of inertia Ic of mass M about axis through center of mass
– and moment of inertia Ip about any other parallel axis displaced by distance d :
∑∑ (x − x )2 f ( x, y ), µ02 = ∑∑ ( y − y )2 f ( x, y )
are “moments of inertia” around the coordinate axes
• ”Cross moment of inertia” is given by
µ11 =
There is a simple relationship between
IP =
∑ mR =
2
∑ m(r + 2 yd + d
2
2
= ∑ mr 2 + 2d ∑ my + d 2 ∑ m
M −1 N −1
∑ ∑ (x − x )( y − y ) f ( x, y )
= I C + 2dMy + Md
Z
)
Zc
d
P
x =0 y =0
r
x
• Moments of inertia do not reflect true shape of object,
as they are not orientation invariant.
• µpq (p+q = 2) can easily be made invariant to rotation.
20.02.2008
C
R
2
INF5300 Lecture 3
•
But since the Zc-axis is placed in the center of mass,
the mean value of y is zero, so
X
20.02.2008
Moments of inertia: rectangle
INF5300 Lecture 3
22
Moments of inertia: square
Y
b
y
a
X
x
• For a continuous binary square object of
size 2a · 2a, the moments of inertia
around the X- and Y-axes are equal:
a
a
⎡ x3 ⎤
⎡a3 a3 ⎤ 4
I 20 = 2 ∫ x 2 b dx = 2b ⎢ ⎥ = 2b ⎢ + ⎥ = a 3 b
3⎦ 3
⎣ 3 ⎦ −a
⎣3
−a
• Moment of inertia around X-axis:
b
⎡b
⎡y ⎤
b ⎤ 4
I 02 = 2 ∫ y 2 a dx = 2a ⎢ ⎥ = 2a ⎢ + ⎥ = ab 3
3
3
3⎦ 3
⎣
⎣ ⎦ −b
−b
b
3
3
3
• If size of rectangle is given as a · b,
moments are a3b/12 and ab3/12.
20.02.2008
INF5300 Lecture 3
M
I P = I C + Md 2
21
• Given continuous binary rectangle.
• Moment of inertia around the y-axis
is found by integration:
a
Y
y
23
I 20 = I 02 = 2 ∫ x 2 a dx =
−a
Y
a
y
a
X
x
4 4
a
3
• If size of square is given as a · a,
the moments are:
a
I 20 = I 02
20.02.2008
a
⎡ x3 ⎤ 2
⎡a3 a3 ⎤ 1 4
a
dx = a ⎢ ⎥ = a ⎢ + ⎥ =
a
=2 ∫ x
2
3 ⎦ −a
24 24 ⎦ 12
⎣
⎣
−a
2
2
2
2
INF5300 Lecture 3
24
Moments of inertia: ellipse
Y
• For a homogeneous (binary)
ellipse we see that:
2
Moments of inertia: circular disc
b
y
a
X
x
2
b 2 2
⎛ x⎞ ⎛ y⎞
a −x
⎜ ⎟ + ⎜ ⎟ =1 ⇒ y = ±
a
⎝a⎠ ⎝b⎠
x + y =R ⇒ y = ± R −x
2
a
a
b ⎡x
a4
b ⎡ a 4 ⎛ π π ⎞⎤ π
b 2 2 2
⎛ x ⎞⎤
x a − x dx = 2 ⎢ (2 x 2 − a 2 ) a 2 − x 2 + sin −1 ⎜ ⎟⎥ = 2 ⎢ ⎜ + ⎟⎥ = a 3b
a ⎣8
a
a
8
a −∫a
⎝ ⎠⎦ −a
⎣ 8 ⎝ 2 2 ⎠⎦ 4
INF5300 Lecture 3
25
20.02.2008
Perpendicular-axis theorem
• Given a homogeneous 2D object in the XY-plane.
• Let origin O be any point in XY-plane.
• Moment of inertia around Z-axis is given by
∫r
object
2
dr =
∫
( x 2 + y 2 )dr =
object
∫
object
x 2 dr +
∫y
2
object
IX =
∫x
2
dr,
IY =
object
∫y
2
X
Y
y
• Thus, the moment of inertia around the Z-axis passing
through the center of the object is IZ = L4/6.
P
dr
• Obviously, the moment of inertia around the Z-axis must
be independent of rotation of X&Y-axis in the object plane.
object
• A very simple relation between the three moments of inertia:
I Z = I X + IY
• Therefore, moment of inertia about ANY AXIS in the plane
that passes through the center of a square is L4/12.
• This is known as the “perpendicular-axis theorem”.
• For 3D objects it is only valid for thin plates in the XY-plane.
20.02.2008
INF5300 Lecture 3
26
• For a continuous binary square object with side L,
moments of inertia around the X- and Y-axis passing
through the center of the object are equal, IX = IY = L4/12.
O
x
INF5300 Lecture 3
A consequence ...
Z
r
dr
• Moments of inertia relative to X- and Y-axes are
X
x
• This is the moment of inertia of a binary circular object
around any axis that lies in the XY-plane and that
passes through the centre of the object.
b
IZ =
y
2
R
b
a
a⎡y
b4
a ⎡ b 4 ⎛ π π ⎞⎤ π
⎛ y ⎞⎤
y 2 b 2 − y 2 dy = 2 ⎢ (2 y 2 − b 2 ) b 2 − y 2 + sin −1 ⎜ ⎟⎥ = 2 ⎢ ⎜ + ⎟⎥ = ab3
b −∫b
8
b ⎣8
b ⎣ 8 ⎝ 2 2 ⎠⎦ 4
⎝ b ⎠⎦ −b
20.02.2008
2
R
⎡x
⎡ R 4 ⎛ π π ⎞⎤ π
R4
⎛ x ⎞⎤
I 20 = I 02 = 2 ∫ x 2 R 2 − x 2 dx = 2 ⎢ (2 x 2 − R 2 ) R 2 − x 2 + sin −1 ⎜ ⎟⎥ = 2 ⎢ ⎜ + ⎟⎥ = R 4
8
⎝ R ⎠⎦ − R
⎣8
⎣ 8 ⎝ 2 2 ⎠⎦ 4
−R
• Smallest moment of inertia is:
I 02 = 2
2
R
• We see that the moments of inertia
around the X- and Y-axes are now equal:
• Moment of inertia around Y-axis:
I 20 = 2
Y
• For a homogeneous (binary) circular
object where the perimeter is given by
27
20.02.2008
INF5300 Lecture 3
28
Radius of gyration, K
A table of K2
• The radius of a circle where we could concentrate all the
mass of an object without altering the moment of inertia
about its center of mass.
IZ
I = µ 00 Kˆ 2 ⇒ Kˆ =
µ 00
I X + IY
=
µ 00
=
µ 20 + µ 02
µ 00
• This feature is invariant to rotation.
• A very useful quantity because it can be determined,
for homogeneous objects, entirely by their geometry.
• Squared radius of gyration, K2,
may be tabulated for simple object shapes,
to help us compute the moments of inertia.
20.02.2008
INF5300 Lecture 3
Rectangle: K2 = b2/3
Disk:
K2 = R2/4
Ellipse:
K2 = b2/4
29
2b
2a
K2 = R2/2
R
a
b
b
K2 = (a2+b2)/3
a
K2 = (a2+b2)/4
INF5300 Lecture 3
30
Object orientation - I
•
2b
Parallelepiped :
b
R
20.02.2008
K2 of some solid objects
2c
b
2a
K2 = (a2+b2)/3
•
2a
Orientation is defined as the angle, relative to the X-axis,
of an axis through the centre of mass
that gives the lowest moment of inertia.
Orientation θ relative to X-axis found by minimizing:
Y
I (θ ) = ∑∑ β 2 f (α , β )
L
Cylinder :
α
K2 = R2/2
α
β
θ
X
β
where the rotated coordinates are given by
R
α = x cos θ + y sin θ ,
β = − x sin θ + y cos θ
L
Cylinder :
•
K2 = R2/4 + L2/12
The second order central moment of the object around the α-axis,
expressed in terms of x, y, and the orientation angle θ of the object:
R
I (θ ) = ∑∑ [ y cos θ − x sin θ ] f ( x, y )
2
Sphere :
20.02.2008
R
K2 = 2R2 /5
INF5300 Lecture 3
x
•
•
31
y
We take the derivative of this expression with respect to the angle θ
Set derivative equal to zero, and find a simple expression for θ :
20.02.2008
INF5300 Lecture 3
32
Object orientation - II
Best fitting object ellipse
Y
• Second order central moment around the α-axis:
• The object ellipse is defined as the ellipse
whose least and greatest moments of inertia
equal those of the object.
I (θ ) = ∑∑ [ y cos θ − x sin θ ] f ( x, y )
2
x
y
• Derivative w.r.t. Θ = 0 =>
∂
I (θ ) = ∑∑ 2 f ( x, y ) [ y cos θ − x sin θ ][− y sin θ − x cos θ ] = 0
∂θ
x
y
⇓
[
]
[
Y
x
y
x
y
α
β
]
∑∑ 2 f ( x, y ) xy (cos 2 θ − sin 2 θ ) = ∑∑ 2 f ( x, y ) x 2 − y 2 sin θ cos θ
θ
(aˆ, bˆ)=
X
⇓
2 µ11
2 tan θ
2 sin θ cos θ
=
=
= tan (2θ )
(µ 20 − µ 02 ) (cos 2 θ − sin 2 θ ) 1 − tan 2 θ
20.02.2008
[
where θ ∈ 0, π
2
]if µ
11
[
]
> 0, θ ∈ π , π if µ 11 < 0
2
INF5300 Lecture 3
33
Object bounding rectangles
(µ20 + µ02 )2 + 4µ112 ⎤⎥
⎦
µ00
20.02.2008
INF5300 Lecture 3
34
• Transform an image by changing scale by α in X-direction
and β in Y-direction, we get a new image
f’(x,y) = f(x/α,y/β).
• The relation between µpq and µ’pq is
– smallest bounding rectangle having sides parallel to image edges.
– represented by the coordinates of two opposite corners:
• e.g. (xmin,ymin) and (xmax, ymax).
– size and elongation depend on object orientation.
• “object-oriented” bounding rectangle:
µ 'pq = α 1+ p β 1+ q µ pq
– smallest bounding rectangle having its longest side
parallel to the orientation of the object.
– transform pixels along perimeter from (X,Y) to (α,β) by
• For β = α we have
α = x cos θ + y sin θ , β = − x sin θ + y cos θ
– search for the minimum and maximum value of α and β.
– represented by coordinates of two opposite corners:
• e.g. (αmin,βmin) and (αmax, βmax).
– size and shape is invariant to rotation.
INF5300 Lecture 3
2 ⎡ µ 20 + µ02 ±
⎢⎣
Scaling invariant central moments
• “image-oriented” bounding rectangle:
20.02.2008
X
• The numerical eccentricity of the best fit ellipse is given by
aˆ 2 − bˆ 2
εˆ =
aˆ 2
• These expressions hold for both gray scale and binary objects.
• So the object orientation is given by:
⎡ 2 µ 11 ⎤
1
θ = tan −1 ⎢
⎥,
2
⎣ (µ 20 − µ 02 )⎦
α
• The semimajor and semiminor axes of this ellipse are given by
⇓
2 µ11 (cos 2 θ − sin 2 θ ) = 2(µ 20 − µ 02 ) sin θ cos θ
β
µ 'pq = α 2 + p + q µ pq
• We get scaling invariant central moments by:
µ pq
p+q
η pq =
, γ=
+ 1 , ∀( p + q ) ≥ 2
γ
2
(µ 00 )
35
20.02.2008
INF5300 Lecture 3
36
Symmetry
Skewness
• To detect symmetry about center of mass, use central moments.
•
• For invariance of scale, use scale-normalised central moments
– (η11, η20, η02, η21, η12, η30, η03).
• Objects symmetric about either x or y axis will produce η11 = 0.
• Objects symmetric about y axis will give η12 = 0 and η30 = 0.
• Objects symmetric about x axis will give η21 = 0 and η03 = 0.
• X axis symmetry: ηpq = 0 for all p = 0, 2, 4, ... ; q = 1, 3, 5, …
η11
.
0
+
+
-
0
0
-
C
O
0
0
+
+
+
+
0
0
+
0
+
0
0
0
20.02.2008
INF5300 Lecture 3
n
µ
−3 =
a 4 = 40
µ 202
⎛
37
•
•
•
•
•
2
20.02.2008
INF5300 Lecture 3
38
2 The method of absolute moment invariants.
This is a set of normalized central moment combinations,
which can be used for scale, position, and rotation
invariant pattern identification.
4
2
INF5300 Lecture 3
3
1 Find principal axes of object, rotate coordinates.
This can break down if object has no unique principal axes.
−3
2
⎞
⎜ ∑ (x − x ) ⎟
⎝ i =1
⎠
Assuming unimodal and symmetric projections:
Use kurtosis to distinguish between shapes:
Uniform (U), kurtosis = -1.2
Semicircular (W), kurtosis = -1
Normal (N), kurtosis = 0
20.02.2008
n
2⎞
⎜ ∑ (x − x ) ⎟
⎝ i =1
⎠
Hu’s rotation invariant moments
i =1
n
3
i =1
• NB: 1D projection histograms cannot distinguish point
symmetry from axis symmetry.
Defined by fourth moment normalized by square of variance.
Kurtosis of the normal distribution equal to zero.
Higher kurtosis => more infrequent extreme x-values.
Find orientation of object and rotate. Then, kurtosis is given by:
n ∑ (x i − x )
µ
a 3 = 303 =
σ
⎛
• a1 = (mean – mode)/σ, (Pearson’s first coefficient).
• a2 = (mean – median)/σ, (Pearson’s second coefficient).
Kurtosis
•
•
•
•
n
n ∑ (x i − x )
• Mean & median on same side of mode as the longer tail.
η20 η02 η21 η12 η30 η03
M
• A measure of departure from symmetry.
• A dimensionless measure using third central moment:
• For second order (p+q=2), there are two invariants:
φ2 = (η20 - η02)2 + 4η112
φ1 = η20 + η02
39
20.02.2008
INF5300 Lecture 3
40
Third order Hu moments
•
Hu moments of simple objects
For third order moments, (p+q=3), the invariants are:
• In the continuous case, the two first Hu moments of a binary
rectangular object of size 2a by 2b, are given by
φ3 = (η30 - 3η12)2 + (3η21 - η03)2
φ4 = (η30 + η12)2 + (η21 + η03)2
φ1 =
φ5 = (η30 - 3η12)(η30 + η12)[(η30 + η12)2 - 3(η21 + η03)2]
+ (3η21 - η03)(η21 + η03)[3(η30 + η12)2 - (η21 + η03)2]
2
• Similarly, the two first Hu moments of a binary elliptic object with
semi-axes a and b, are given by
φ7 = (3η21 - η03)(η30 + η12)[(η30 + η12)2 - 3(η21 + η03)2]
- (η30 - 3η12)(η21 + η03)[3(η30 + η12)2 - (η21 + η03)2]
φ1 =
φ7 is skew invariant, and may help distinguish between mirror images.
INF5300 Lecture 3
1
4π
⎛a b⎞
⎜ + ⎟,
⎝b a⎠
2
⎛ 1 ⎞ ⎛a b⎞
⎟ ⎜ − ⎟
⎝ 4π ⎠ ⎝ b a ⎠
2
φ2 = ⎜
while the remaining five Hu moments are all zero.
These moments are not independent, and do not comprise a complete set.
20.02.2008
2
⎛ 1 ⎞ ⎛a b⎞
⎟ ⎜ − ⎟
⎝ 12 ⎠ ⎝ b a ⎠
φ2 = ⎜
while the remaining five Hu moments are all zero.
φ6 = (η20 - η02)[(η30 + η12)2 - (η21 + η03)2] + 4η11(η30 + η12)(η21 + η03)
•
1 ⎛a b⎞
⎜ + ⎟,
12 ⎝ b a ⎠
41
20.02.2008
Φ1 and φ2 versus a/b
INF5300 Lecture 3
42
Hu and compactness – simple objects
compactness versus a/b
• Relative difference in φ1 of ellipse and its object
oriented bounding rectangle is constant, 4.5%.
ellipse
rectangle
1
γ ellipse =
•
•
0,1
1
10
100
1000
a/b
• Relative difference in φ2 of ellipse and its object
oriented bounding rectangle is constant, 8.8%.
1⎛a b⎞
⎜ + ⎟,
4⎝b a⎠
γ rec tan gle =
200
For ellipses, a linear relationship: φ1 = γ/π
For rectangles: φ1 = γ (π/12)(a2+b2)/(a+b)2
ellipse
rectangle
0
0
1 ⎛a b
⎞
⎜ + + 2⎟
π ⎝b a
⎠
200
400
a/b
Hu's first moment versus compactness
50
=> Using both γ and φ1 to characterize simple objects
seems redundant, regardless of size and elongation.
Hu's second moment versus a/b
10000
ellipse
rectangle
0
0
50
100
150
200
Compactness (P2/(4piA))
1000
Hu's second moment vs compactness
100
ellipse
10
rectangle
•
•
For an ellipse: φ2 = γ (1/4π2ab)(a2-b2)2/(a2+b2)
For rectangle: φ2 = γ (π/144)(a/b -1)(1- b/a)
1
0,1
0,01
1
10
100
1000
=> φ2 and γ is a better combination than φ1 and γ
to characterize elliptical and rectangular objects.
2000
Hu's second moment
Hu's second moment
• Relative differences given above are also true
when comparing an ellipse with a same-area
rectangle having the same a/b ratio, regardless
of the size and eccentricity of the ellipse.
•
10
Compactness γ = P2/(4πA).
High value for complex and very elongated objects,
like rectangles and ellipses where the a/b ratio is high.
For ellipses and rectangles, compactness in the
continuous case is given by:
compactness
Hu's first moment
• Notice that even in the continuous case it may be
hard to distinguish between an ellipse and its
bounding rectangle using these two moments.
•
•
Hu's first moment versus a/b
100
Hu's first moment
• Only (φ1, φ2) are useful for these simple objects.
ellipse
rectangle
0
0
50
100
150
200
Compactness
a/b
20.02.2008
INF5300 Lecture 3
43
20.02.2008
INF5300 Lecture 3
44
Contrast invariants
Affine invariants
• Abo-Zaid et al. have defined a normalization that cancels both scaling
and contrast.
( p+q )
• The normalization is given by
2
µ
⎛
⎞
µ
pq
'
00
η pq =
⎜
⎟
µ00 ⎜⎝ µ 20 + µ02 ⎟⎠
• This normalization also reduces the dynamic range of the moment
features, so that we may use higher order moments without having to
resort to logarithmic representation.
• Abo-Zaid’s normalization cancels the effect of changes in contrast, but
not the effect of changes in intensity:
f ( x, y ) = f ( x, y ) + b
'
• Flusser and Suk (1993) give set of four moments are
invariant under general affine transforms:
I1 =
µ 20 µ 02 − µ112
µ 004
I2 =
µ 302 µ 032 − 6µ 30 µ 21 µ12 µ 03 + 4 µ 30 µ123 + 4 µ 213 µ 03 − 3µ122 µ 212
10
µ 00
I3 =
µ 20 (µ 21 µ 30 − µ122 ) − µ11 (µ 30 µ 03 − µ 21 µ12 ) + µ 02 (µ 30 µ12 − µ 212 )
µ 007
3
I 4 = ( µ 20
µ 032 − 6µ 202 µ11 µ12 µ 03 − 6µ 202 µ 02 µ 21 µ 03 + 9 µ 202 µ 02 µ122 + 12 µ 20 µ112 µ 21 µ 03
2
+ 6µ 20 µ11 µ 02 µ 30 µ 03 − 18µ 20 µ11 µ 02 µ 21 µ12 − 8µ112 µ 30 µ 03 − 6µ 20 µ 02
µ 30 µ12
• In practice, we often experience a combination:
2
11
+ 9 µ 20 µ112 µ 21 µ 03 + 12 µ 20 µ 02
µ 212 + 12 µ112 µ 02 µ 30 µ12 − 6µ11 µ 022 µ 21 + µ 023 µ 302 ) / µ 00
f ' ( x, y ) = cf ( x, y ) + b
20.02.2008
INF5300 Lecture 3
45
20.02.2008
Orthogonal moments
INF5300 Lecture 3
46
Monomials as basis set
• Moments produced using orthogonal basis sets have the
advantage of needing lower precision to represent
differences to the same accuracy as the monomials.
• The orthogonality condition simplifies the reconstruction of
the original function from the generated moments.
• Orthogonality means mutually perpendicular:
– ym and yn are orthogonal over an interval a ≤ x ≤ b if and only if:
• Cartesian moments are given by:
m pq =
M −1 N −1
∑∑x
p
y q P( x, y )
x =0 y =0
• M and N are the image dimensions
• Basis function: monomial product xpyq
1
• Monomials are highly correlated.
b
∫y
m
( x ) yn ( x )dx = 0; m ≠ n
a
• In discrete images, integrals within moment descriptors
are replaced by summations.
• Two such orthogonal moments are Legendre and Zernike.
20.02.2008
INF5300 Lecture 3
47
• We need more moments to describe
an object than if the basis functions
were uncorrelated.
0
-1
0
1
-1
20.02.2008
INF5300 Lecture 3
48
Legendre moments
•
Legendre moments of order (m+n):
λ mn =
•
(2m + 1)(2n + 1)
∑∑ L
4
x
• Discrete moments defined by
m
χ mn =
( x ) Ln ( y ) f ( x, y ) , m, n = 0, 1, 2, ..., ∞
∑∑ T
x
y
m
( x ) Tn ( y ) f ( x, y ) , m, n = 0, 1, 2, ..., ∞
y
• Tm, Tn, and f(x,y) defined over [-1,1]:
Lm, Ln, and f(x,y) defined over [-1,1]:
Lo ( x ) = 1
L1 ( x ) = x
To ( x ) = 1
1
1
L2 ( x ) = (3x 2 − 1)
L3 ( x ) = (5 x 3 − 3x )
2
2
1
L4 ( x ) = (35 x 4 − 30 x 2 + 3 ) L
8
M
2n + 1
n
Ln +1 ( x ) =
x Ln ( x ) −
Ln −1 ( x )
n +1
n +1
•
•
Chebyshev moments
T1 ( x ) = x
T2 ( x ) = 2 x − 1
T3 ( x ) = 4 x 3 − 3x
2
1
T4 ( x ) = 8 x − 8 x + 1
4
0
-1
0
1
2
L
M
Tk ( x ) = 2 x Tk −1 ( x ) − Tn −2 ( x )
-1
INF5300 Lecture 3
49
∑B
R mn ( x, y ) =
mnk
k =n
r , Bmnk = (− 1)
m −k
2
⎛m+k⎞
⎜
⎟!
⎝ 2 ⎠
⎛m−k ⎞ ⎛k +n⎞ ⎛k −n⎞
⎜
⎟!⎜
⎟!⎜
⎟!
⎝ 2 ⎠ ⎝ 2 ⎠ ⎝ 2 ⎠
• Z can be calculated from central µ moments
Z pq =
p +1
π
20.02.2008
p
t
q
∑∑ ∑ (− j )
k = q l =0 m =0
m
⎛t ⎞
⎜⎜ ⎟⎟
⎝l ⎠
20.02.2008
INF5300 Lecture 3
50
• The higher the moment order, the more complex
is the pattern of spatial contribution to moment.
V mn ( x, y ) = Rmn ( r ) e ( jnθ ) , j = − 1, m ≥ 0, n ≤ m, m − n = even
k
1
Zernike moments, example
• Projections of image onto V-functions:
m
0
-1
Complex Zernike moments
• where
0
-1
• Central / peripheral polynomial weights
different from Legendre polynomials.
This set of moments is not correlated.
Need linear mapping of object to [-1,1].
20.02.2008
1
• Only a few moments are needed to reconstruct
object, and distinguish between different objects.
⎛q⎞
⎜⎜ ⎟⎟ B pqk µ (k − 2 l − q + m )(q + 2 l − m ) , t = ( k − q) / 2
⎝m⎠
INF5300 Lecture 3
51
20.02.2008
INF5300 Lecture 3
52