Invariants to convolution
Motivation – template matching
Image degradation models
Space-variant
Space-invariant  convolution
Two approaches
Traditional approach: Image restoration (blind
deconvolution) and traditional features
Proposed approach: Invariants to convolution
Invariants to convolution
for any admissible h
The moments under convolution
Geometric/central
Complex
Assumptions on the PSF
PSF is centrosymmetric (N = 2) and has a unit integral
supp(h)
Common PSF’s are centrosymmetric
Invariants to centrosymmetric convolution
where (p + q) is odd
What is the intuitive meaning of the invariants?
“Measure of anti-symmetry”
Invariants to centrosymmetric convolution
Invariants to centrosymmetric convolution
Convolution invariants in FT domain
Convolution invariants in FT domain
Relationship between FT and moment invariants
where M(k,j) is the same as C(k,j) but with
geometric moments
Template matching
sharp image
with the templates
the templates (close-up)
Template matching
a frame where
the templates were
located successfully
the templates (close-up)
Template matching performance
Boundary effect in template matching
Valid region
Boundary effect in template matching
Invalid region
- invariance is
violated
Boundary effect in template matching
zero-padding
Other types of the blurring PSF
• N-fold rotation symmetry, N > 2
• Axial symmetry
• Circular symmetry
• Gaussian PSF
• Motion blur PSF
The more we know about the PSF, the more
invariants and the higher discriminability we get
Discrimination power
The null-space of the blur invariants = a set of
all images having the same symmetry as
the PSF.
One cannot distinguish among symmetric
objects.
Recommendation for practice
It is important to learn as much as possible
about the PSF and to use proper invariants
for object recognition
Combined invariants
Convolution and rotation
For any N
Robustness of the invariants
Satellite image registration by
combined invariants
Registration algorithm
• Independent corner detection in both images
• Extraction of salient corner points
• Calculating blur-rotation invariants of a circular
neighborhood of each extracted corner
• Matching corners by means of the invariants
• Refining the positions of the matched control
points
• Estimating the affine transform parameters by
a least-square fit
• Resampling and transformation of the sensed
image
Control point detection
( v11, v2
v2
min distance((
v11,k,v3
v21k, ,…
v3)k, … ) , ( v1m, v2(mv1
, v3
,…
)) 2, … )
2, m
2, v3
k,m
Control point matching
Registration result
Application in MBD
Combined blur-affine invariants
• Let I(μ00,…, μPQ) be an affine moment
invariant. Then I(C(0,0),…,C(P,Q)),
where C(p,q) are blur invariants, is a
combined blur-affine invariant (CBAI).
Digit recognition by CBAI
CBAI
AMI
Documented applications of
convolution and combined invariants
• Character/digit/symbol recognition in the
presence of vibration, linear motion or
out-of-focus blur
• Robust image registration (medical, satellite,
...)
• Detection of image forgeries
Moment-based focus measure
• Odd-order moments  blur invariants
• Even-order moments  blur/focus measure
If M(g1) > M(g2)  g2 is less blurred
(more focused)
Other focus measures
•
•
•
•
Gray-level variance
Energy of gradient
Energy of Laplacian
Energy of high-pass bands of WT
Desirable properties of a focus measure
•
•
•
•
Agreement with a visual assessment
Unimodality
Robustness to noise
Robustness to small image changes
Agreement with a visual assessment
Robustness to noise
Sunspots – atmospheric turbulence
Moments are generally worse than wavelets
and other differential focus measures
because they are too sensitive to local
changes but they are very robust to noise.
Invariants to elastic deformations
How can we recognize objects on curved surfaces ...
Moment matching
• Find the best possible fit by minimizing the
error and set
The bottle
Orthogonal moments
- set of orthogonal polynomials
Motivation for using OG moments
• Stable calculation by recurrent relations
• Easier and stable image reconstruction
Numerical stability
How to avoid numerical problems with high
dynamic range of geometric moments?
Standard powers
Orthogonal polynomials
Calculation using recurrent relations
Two kinds of orthogonality
• Moments (polynomials) orthogonal on a unit
square
• Moments (polynomials) orthogonal on a unit
disk
Moments orthogonal on a square
is a system of 1D orthogonal polynomials
Common 1D orthogonal polynomials
•
•
•
•
•
•
Legendre
Chebyshev
Gegenbauer
Jacobi
(generalized) Laguerre
Hermite
<-1,1>
<-1,1>
<-1,1>
<-1,1> or <0,1>
<0,∞)
(-∞,∞)
Legendre polynomials
Definition
Orthogonality
Legendre polynomials explicitly
Legendre polynomials in 1D
Legendre polynomials in 2D
Legendre polynomials
Recurrent relation
Legendre moments
Moments orthogonal on a disk
Radial part
Angular part
Moments orthogonal on a disk
•
•
•
•
•
•
Zernike
Pseudo-Zernike
Orthogonal Fourier-Mellin
Jacobi-Fourier
Chebyshev-Fourier
Radial harmonic Fourier
Zernike polynomials
Definition
Orthogonality
Zernike polynomials – radial part in 1D
Zernike polynomials – radial part in 2D
Zernike polynomials
Zernike moments
Mapping of Cartesian coordinates x,y to
polar coordinates r,φ:
• Whole image is mapped inside the unit disk
• Translation and scaling invariance
Zernike moments
Rotation property of Zernike moments
The magnitude is preserved, the phase is shifted by ℓθ.
Invariants are constructed by phase cancellation
Zernike rotation invariants
Phase cancellation by multiplication
Normalization to rotation
Image reconstruction
• Direct reconstruction from geometric moments
• Solution of a system of equations
• Works for very small images only
• For larger images the system is ill-conditioned
Image reconstruction by direct
computation
12 x 12
13 x 13
Image reconstruction
• Reconstruction from geometric moments via FT
Image reconstruction via Fourier transform
Image reconstruction
• Image reconstruction from OG moments on
a square
• Image reconstruction from OG moments on
a disk (Zernike)
Image reconstruction from Legendre
moments
Image reconstruction from Zernike
moments
Better for polar raster
Image reconstruction from Zernike
moments
Reconstruction of a noise-free image
Reconstruction of a noisy image
Reconstruction of a noisy image
Summary of OG moments
• OG moments are used because of their favorable
numerical properties, not because of theoretical
contribution
• OG moments should be never used outside the area
of orthogonality
• OG moments should be always calculated by
recurrent relations, not by expanding into powers
• Moments OG on a square do not provide easy
rotation invariance
• Even if the reconstruction from OG moments is
seemingly simple, moments are not a good tool
for image compression