Multidimensional and Directional Filter Banks
Minh N. Do
Department of Electrical and Computer Engineering
University of Illinois at Urbana-Champaign
www.ifp.uiuc.edu/∼minhdo
minhdo@uiuc.edu
Joint work with Martin Vetterli (EPFL and UC Berkeley),
Arthur Cunha, Yue Lu, Jianping Zhou (UIUC)
Outline
1. Multidimensional filter banks
• Sampling lattices
• Filter design
2. Laplacian pyramid frames
3. Iterated directional filter banks
1. Multidimensional Filter Banks
1
1. Multidimensional Filter Banks
Motivation
Looking for...
• Discrete-domain expansions for sampled data
• Structured transforms with fast algorithms
• Seamless connection with continuous-domain expansions
Answer: Filter banks and associated bases and frames.
1. Multidimensional Filter Banks
2
Filter Banks
Wavelets and filter banks...
fundamental link between continuous and discrete domains
synthesis
analysis
H1
Mallat
2
y1
2
G1
+
x
H0
2
2
+
G1
+
2
G1
2
x1
2
y0
2
G0
x̂
x0
Daubechies
G0
G0
1. Multidimensional Filter Banks
3
Multidimensional Filter Banks
• Common approach: use 1-D techniques in a separable fashion
• Limitations:
– very constrained filter design (separable filters)
– only rectangular frequency partition possible
1. Multidimensional Filter Banks
4
“True” Multidimensional Filter Banks
H
H1L
π
Q
x
−π
H
H0Η
Q
ω2
π
ω1
−π
• More flexibility:
– True multidimensional filters → directional filters
– Sampling via lattices → discrete rotations
1. Multidimensional Filter Banks
5
Sampling via Lattices
In 1D (downsample by 2):
n1
In 2D (downsample by 2):
1. Multidimensional Filter Banks
6
Multidimensional Signals and Filtering
• Discrete-time d-dimensional signal:
x[n],
def
n = (n1, . . . , nd)T ∈ Zd
• The z-transform:
X(z) =
X
x[n]z −n
where z n =
d
Y
zini
i=1
n∈Zd
• Filtering with h[n] yields
y[n] = x[n] ∗ h[n] =
X
x[k]h[n − k]
k∈Zd
Y (z) = X(z)H(z)
1. Multidimensional Filter Banks
7
Multidimensional Sampling
Sampling operation is represented by a d × d nonsingular integer matrix M .
Downsampling
xd [n] = x[M n].
Upsampling
xu[n] =
(
x[k] if n = M k, k ∈ Zd
0
otherwise.
Sampling lattice
def
LAT (M ) = n : n = M k, k ∈ Z
d
= all linear combinations of columns of M with integer coefficients
def
Sampling density is equal to |M | = |det M |
1. Multidimensional Filter Banks
8
Examples of Sampling Lattices
Separable lattice in two dimensions:
D1 =
D2 =
2
0
2
0
0
2
2
2
Quincunx lattice in two dimensions:
1 1
1 −1
1 2
D2 =
1 0
D1 =
1. Multidimensional Filter Banks
9
Sampling Lattices and Matrices
Theorem 1. LAT (A) = LAT (B) if and only if A = BE where E is a
unimodular (i.e. det E = ±1) integer matrix.
Example: Four basic unimodular matrices (shearing operators)
1 0
1 0
1 −1
1 1
.
, R3 =
, R2 =
, R1 =
R0 =
−1 1
1 1
0 1
0 1
Theorem 2. [Smith form] Any integer matrix M can be factorized as
M = U DV where U and V are unimodular integer matrices and D is an
integer diagonal matrices.
Example: For quincunx matrices
1 −1
,
Q0 =
1 1
Q1 =
1 1
−1 1
Q0 = R1D 0R2 = R2D 1R1 and Q1 = R0D 0R3 = R3D 1R0,
where D 0 = diag(2, 1) and D 1 = diag(1, 2)
1. Multidimensional Filter Banks
10
Examples of Multidimensional Sampling
↓R
R3 =
1 0
−1 1
1. Multidimensional Filter Banks
Q1 =
↓Q
1 1
= R0 D 0 R3
−1 1
11
Multidimensional Sampling in Frequency-Domain (1/2)
Downsampling by M
1
Xd(ω) =
|det(M )|
X
X(M −T (ω − 2πk)).
k∈N (M T )
Here N (M ) is the set of integer vectors of the form M t, t ∈ [0, 1)d.
1. Multidimensional Filter Banks
12
Multidimensional Sampling in Frequency-Domain (2/2)
Upsampling by M
Xu(ω) = X(M T ω),
Xu(z) = X(z M ).
def
Here z M = (z m1 , . . . , z md )T , where mi is the i-th column of M .
1. Multidimensional Filter Banks
13
Polyphase Representation
• Polyphase components with respect to the sampling matrix M :
xi[n] = x[M n + li]
where {li} is the set of |M | integer vectors of the form M t, t ∈ [0, 1)d
• Each polyphase component “lives” on a coset.
def
Ci = m : m = M n + l i , n ∈ Z
|M |−1
[
d
Ci = Z ,
Ci
i=0
• In the z-domain:
\
d
,
0 ≤ i ≤ |M | − 1
Cj = ∅, i 6= j.
|M |−1
X(z) =
X
z −li Xi(z M )
i=0
1. Multidimensional Filter Banks
14
Examples of Polyphase Representations
Separable lattice
M=
2 0
0 2
X(z1, z2) = X00(z12, z22)+z1−1X10(z12, z22)+z2−1X01(z12, z22)+z1−1z2−1X11(z12, z22)
Quincunx lattice
M=
1 1
−1 1
X(z1, z2) = X0(z1z2, z1z2−1) + z1−1X1(z1z2, z1z2−1)
1. Multidimensional Filter Banks
15
Polyphase-Domain Analysis
Filter and downsample
x[n] −→ H(z) −→ (↓ M ) −→ c[n]
|M |−1
C(z) =
X
Hi(z)Xi(z) = H(z)x(z)
i=0
def
def
where x(z) = (X0(z), . . . , X|M |−1(z))T , and H(z) = (H0(z), . . . , H|M |−1(z))
Upsample and filter
c[n] −→ (↑ M ) −→ G(z) −→ p[n]
Pi(z) = Gi (z)C(z)
def
or p(z) = G(z)C(z)
def
where G(z) = (G0(z), . . . , G|M |−1(z))T , and p(z) = (P0(z), . . . , P|M |−1(z))T
1. Multidimensional Filter Banks
16
Filter Bank in the Polyphase-Domain
ANALYSIS
ANALYSIS
SYNTHESIS
SYNTHESIS
H0
D
D
G0
z −l0
D
D
z l0
H1
D
D
G1
z −l1
D
D
z l1
X
+
HN −1
D
D
GN −1
X̂
H p Gp
X
z
−l|D|−1 D
+
D
z
X̂
l|D|−1
• Critically sampled: |D| = N
• Perfect reconstruction: Gp(z)H p(z) = I|D|
• Biorthogonal: Perfect reconstruction + critically sampled
• Orthogonal: H p(z) = GTp (z −1) and Gp(z)GTp (z −1) = I
(i.e. Gp(z) is a paraunitary matrix)
1. Multidimensional Filter Banks
17
A Reduction Theorem for Biorthogonal Filter Banks
Theorem 3. [ZhouD:04] Suppose G(z) is an N × N matrix and
GN −1(z) is its submatrix obtained by deleting the last row of G(z).
Suppose H(z) is another N × N matrix and H N −1(z) is its submatrix
obtained by by deleting the last column of H(z).
Then G(z)H(z) = I N if and only if
GN −1(z)H N −1(z) = I N −1,
(1)
GN,i(z) = α(z)(−1)i+N det H i,N −1 (z),
(2)
Hi,N (z) = α−1(z)(−1)i+N det GN −1,i (z),
(3)
where α(z) = det G(z) is an arbitrary nonzero filter, H i,N −1 (z) is the
submatrix of H N −1(z) obtained by deleting its ith row, and GN −1,i (z) is
the submatrix of GN −1(z) obtained by deleting its ith column.
If G(z) and H(z) have FIR entries, then additionally α(z) = αz k
Corollary 1. A N -channel biorthogonal FIR filter bank is completely
determined by its first N − 1 channels, and a scaled delay in the last
channel.
1. Multidimensional Filter Banks
18
Example: Quincunx Filter Bank
H0
Q
y0
Q
G0
x
+
H1
Q
y1
Q
x̂
G1
Possible frequency partitions:
ω1
(π, π)
000000000000
111111111111
000000000000
111111111111
1
000000000000
111111111111
000000000000
111111111111
000000000000
111111111111
0
000000000000
111111111111
000000000000
111111111111
ω0
000000000000
111111111111
000000000000
111111111111
000000000000
111111111111
000000000000
111111111111
000000000000
111111111111
ω1
(π, π)
000000000000
111111111111
000000000000
111111111111
0
000000000000
111111111111
000000000000
111111111111
1
000000000000
111111111111
000000000000
111111111111
000000000000ω0
111111111111
000000000000
111111111111
000000000000
111111111111
000000000000
111111111111
000000000000
111111111111
000000000000
111111111111
Equivalent biorthogonal condition for FIR filters:
H0(z)G0(z) + H0(−z)G0(−z) = 2,
H1(z) = z −k G0(−z),
1. Multidimensional Filter Banks
and
G1(z) = z k H0(−z).
19
Filter Design Problem
Filter design amounts to solve
P (z) + P (−z) = 2,
(easy)
P (z) = H0(z)G0(z)
(hard)
where
Fundamental problem in multi-dimensions: no factorization theorem.
How to avoid MD factorization?
One possibility: Map 1D filters to MD filters...
1. Multidimensional Filter Banks
20
Design via Mapping
1. Start with a 1D solution:
P (1D)(z) + P (1D)(−z) = 2 and P (1D)(z) = H (1D)(z)G(1D) (z)
2. Apply a 1D to MD mapping to each filter:
F (1D)(z) → F (MD)(z) = F (1D)(M (z))
• If M (z) = −M (−z) then the perfect reconstruction condition is
preserved
P (MD)(z) + P (MD)(−z) = P (1D)(M (z)) + P (1D)(M (−z)) = 2
• Can design the mapping to preserve and obtain other properties, like
linear phase, vanishing moments, directional vanishing moments, ...
(McClellan:73, TayK:93, ChenV:93, CunhaD:04)
This only works for biorthogonal filter banks...
1. Multidimensional Filter Banks
21
MD Orthogonal FB Design using Cayley Transform
Orthogonal
Filter Banks
Paraunitary
matrices
Cayley
Transform
Para-skew-Hermitian
matrices
Definition. [Cayley transform] The Cayley transform of a matrix U (z) is
H(z) = I + U (z)
−1
I − U (z) .
Definition. H(z) is a para-skew-Hermitian (PSH) matrix if it satisfies
H(z −1) = −H T (z),
for real coefficients.
Proposition 1. [ZhouDK:04] Suppose U (z) is an N × N matrix. Then
there exists at least one N × N diagonal matrix Λ whose diagonal entries
are either 1 or −1 such that I + ΛU (z) is nonsingular.
Theorem 4. [ZhouDK:04] The Cayley transform of a paraunitary matrix
is a PSH matrix. Conversely, the Cayley transform of a PSH matrix is a
paraunitary matrix.
1. Multidimensional Filter Banks
22
Example: Two-Channel Filter Banks
Polyphase-domain:
U (z) =
U00(z) U01(z)
U10(z) U11(z)
Cayley-domain:
The paraunitary condition
U (z)U T (z −1) = I
becomes
U00(z)U00(z −1) + U01(z)U01(z −1 ) = 1,
U00(z)U10(z −1) + U01(z)U11(z −1 ) = 0,
U10(z)U10(z −1) + U11(z)U11(z −1 ) = 1.
→ nonlinear equations
1. Multidimensional Filter Banks
H(z) =
H00(z) H01(z)
.
H10(z) H11(z)
The PSH condition
H(z −1) = −H T (z)
becomes
H00(z −1) = −H00(z),
H11(z −1) = −H11(z),
H01(z −1) = −H10(z).
→ linear equations
23
Outline
1. Multidimensional filter banks
2. Laplacian pyramid frames
• Framing pyramids
• New reconstruction (dual frame)
3. Iterated directional filter banks
2. Laplacian Pyramid Frames
24
2. Laplacian Pyramid Frames
decomposition
reconstruction
• Reason: avoid “frequency scrambling” due to (↓) of the HP channel.
• Laplacian pyramid as a frame operator → tight frame exists.
• New reconstruction: efficient filter bank for dual frame (pseudo-inverse).
2. Laplacian Pyramid Frames
25
Decomposition in the Laplacian Pyramid
c
x
H
M
M
G
p
_ +
d
G
H
C(z) = H(z)x(z)
d(z) = x(z) − G(z)H(z)x(z) = (I − G(z)H(z))x(z).
Combining gives
C(z)
H(z)
=
x(z).
d(z)
I − G(z)H(z)
| {z } |
{z
}
y(z)
2. Laplacian Pyramid Frames
A(z)
26
Usual Reconstruction in the Laplacian Pyramid
c
M
G
+
x̂
d
x̂(z) = G(z)C(z) + d(z)
Or,
C(z)
x̂(z) = G(z) I
.
| {z } d(z)
| {z }
S 1 (z)
y(z)
Note that S 1(z)A(z) = I (perfect reconstruction) for any H(z) and
G(z).
But... what about noisy pyramids: ŷ = y + e ?
2. Laplacian Pyramid Frames
27
Frame Analysis
• LP is a frame operator (A) with redundancy.
• It admits an infinite number of left inverses.
• Let S be an arbitrary left inverse of A,
x̂ = S ŷ = S(y + e) = x + Se.
• The optimal left inverse (minimizing kSk) is the pseudo-inverse (or dual
frame) of A:
A† = (AT A)−1AT .
• If the noise is white, then among all left inverses, the pseudo-inverse
minimizes the reconstruction MSE.
2. Laplacian Pyramid Frames
28
A Tight Frame Pyramid
Proposition. The Laplacian pyramid with orthogonal filters is a tight
frame with frame bounds equal to 1.
Proof: Orthogonal condition means G∗(z)G(z) = 1 and H(z) = G∗(z).
With this, we can directly verify that
∗
∗
∗
∗
A (z)A(z) = H (z) I − H (z)G (z)
Geometrical interpretation:
X
hx[•], g[• − M k]i g[n − M k].
p[n] =
|
{z
}
k∈Zd
H(z)
I − G(z)H(z)
= I.
x
c[k]
d
Using the Pythagorean theorem:
kxk2 = kpk2 + kdk2 = kck2 + kdk2.
2. Laplacian Pyramid Frames
V
p
29
New Reconstruction for the Laplacian Pyramid
For tight frame with frame bounds 1, the pseudo-inverse of A(z) is
†
T
A (z) = A (z) =
H(z)
I − G(z)GT (z)
T
T
= G(z) I − G(z)G (z) .
Which leads to the optimal reconstruction:
c
d
H
M
_ +
M
G
+
x̂
This also works for biorthogonal filters H and G.
2. Laplacian Pyramid Frames
30
Laplacian Pyramids as an Oversampled Filter Bank
From the polyphase representation of A(z) and S(z)
c
M
M
H
G
K0
x
K|M |−1
d0
M
M
d|M |−1
M
M
F0
+
x̂
F|M |−1
d
For the usual reconstruction (REC-1)
[1]
Fi (z) = z −ki ,
for i = 0, . . . , |M | − 1.
For the new reconstruction (REC-2)
[2]
Fi (z) = z −ki − G(z)Hi(z M ),
2. Laplacian Pyramid Frames
for i = 0, . . . , |M | − 1.
31
Multilevel Laplacian Pyramids
2
G
2
G
2
F0
2
F0
2
F1
2
F1
+
+
x̂
3.5
3.5
3
3
2.5
2.5
Magnitude
Magnitude
Frequency responses of equivalent synthesis filters: REC-1 and REC-2
2
1.5
2
1.5
1
1
0.5
0.5
0
0
0.2
0.4
0.6
0.8
Normalized Frequency (×π rad/sample)
1
0
0
0.2
0.4
0.6
0.8
1
Normalized Frequency (×π rad/sample)
The new reconstruction is crucial for building multiscale directional frames.
2. Laplacian Pyramid Frames
32
PSfrag
Experimental Results (1/2)
x
LP
dec.
y
+
ŷ
threshold
usual
rec.
new
rec.
x̂1
x̂2
Nonlinear approximation: SNR’s of the reconstructed images from the M
most significant LP coefficients (after 6 levels of decomposition)
M
Barbara REC-1
REC-2
Goldhill REC-1
REC-2
Peppers REC-1
REC-2
2. Laplacian Pyramid Frames
212
9.68
9.87
12.30
12.60
15.06
15.62
214
12.56
13.18
15.79
16.23
20.81
21.33
216
20.94
21.75
21.55
22.19
26.77
27.32
33
PSfrag
Experimental Results (2/2)
x
LP
dec.
y
+
e
ŷ
usual
rec.
new
rec.
x̂1
x̂2
With additive uniform white noise in [0, 0.1] (non-zero mean)
usual
rec.
SNR =
6.28 dB
2. Laplacian Pyramid Frames
new rec.
SNR =
17.42 dB
34
Outline
1. Multidimensional filter banks
2. Laplacian pyramid frames
3. Iterated directional filter banks
• How to obtain directional decomposition with 2D tree-structured filter
banks
• Local directional bases
3. Iterated Directional Filter Banks
35
3. Iterated Directional Filter Banks
• Feature: division of 2-D spectrum into fine slices using tree-structured
filter banks.
ω2
0
1
(π, π)
2
3
7
4
6
5
5
6
4
ω1
7
3
2
1
0
(−π, −π)
• Background: Bamberger and Smith (’92) cleverly used quincunx FB’s,
modulation and shearing.
• We propose:
– a simplified DFB with fan FB’s and shearing
– use DFB to construct directional bases
3. Iterated Directional Filter Banks
36
Our Simplified DFB: Two Building Blocks
• Frequency splitting by the quincunx filter banks (Vetterli’84).
Q
y0
Q
x
+
Q
y1
x̂
Q
• Shearing by resampling
3. Iterated Directional Filter Banks
37
Illustration: The First Two Levels of the DFB
Q1
0
Q1
1
Q1
2
Q1
3
Q0
Q0
Using the multirate identity:
M
H(ω)
≡
H(M T ω)
M
The support configurations of equivalent filters are:
ω1
(π, π)
000000000000
111111111111
000000000000
111111111111
0
000000000000
111111111111
000000000000
111111111111
1
000000000000
111111111111
000000000000
111111111111
000000000000ω0
111111111111
000000000000
111111111111
000000000000
111111111111
000000000000
111111111111
000000000000
111111111111
000000000000
111111111111
First level
3. Iterated Directional Filter Banks
ω1
(π, π)
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
1
0
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
ω0
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
Second level
ω1
0
(π, π)
1
2
3
3
2
1
ω0
0
Overall filters
38
How Frequency is Divided into Finer Direction?
(l+1)
H2k
M (l+1)
Q
(l)
Hk
≡
M (l)
M (l+1)
Q
(l+1)
111111
000000
000000
111111
000000
111111
000000
111111
000000
111111
1
0
000000
111111
000000
111111
000000
111111
000000
111111
000000
111111
000000
111111
Overall sampling:
H2k+1
(l)
M k = [2 · D il−2] · Rsl(k)
≡ separable sampling, then shearing (doesn’t change lattice
3. Iterated Directional Filter Banks
39
Multichannel View of the Directional Filter Bank
H0
S0
H1
S1
x
H2l−1
S 2l−1
y0
y1
y2l−1
G0
S0
G1
S1
+
x̂
G2l−1
S 2l−1
Use two separable sampling matrices:
#
"

2l−1 0

l−1

0
≤
k
<
2
(“near horizontal” direction)

 0
2
#
"
Sk =

2
0

l−1
l


2
≤
k
<
2
(“near vertical” direction)
 0 2l−1
3. Iterated Directional Filter Banks
40
Why Critical Sampling Works?
ω2
Hk
π
π
Sk
ω1
Frequency tiling in the MDFB:
X
δRk (ω − 2πS −T
k m) = 1
for all k = 0, . . . , 2l − 1; ω ∈ R2.
m∈Z2
where Rk is the ideal frequency region for Hk .
3. Iterated Directional Filter Banks
41
General Bases from the DFB
An l-levels DFB creates a local directional basis of l2(Z2):
o
n
(l)
(l)
gk [· − Sk n]
l
2
0≤k<2 , n∈Z
(l)
• Gk are directional filters:
• Sampling lattices (spatial tiling):
2
2l−1
2
2l−1
3. Iterated Directional Filter Banks
42
Example of DFB Impulse Responses
32 equivalent filters for the first half channels (basically horizontal directions)
of a 6-levels DFB that use the Haar filters
3. Iterated Directional Filter Banks
43
References
• Multidimensional filter banks
– P. P. Vaidyanathan, Multirate Systems and Filter Banks, PrenticeHall, 1993.
– M. Vetterli and J. Kovačević, Wavelets and Subband Coding,
Prentice-Hall, 1995.
– J. Zhou, M. N. Do, and J. Kovačević, Multidimensional orthogonal
filter bank characterization and design using the Cayley transform,
IEEE Trans. on Image Proc., to appear.
• Laplacian pyramid frames
– M. N. Do and M. Vetterli, “Framing pyramids,” IEEE Trans. on
Signal Proc., pp. 2329-2342, Sep. 2003.
• Iterated directional filter banks
– M. N. Do,“Directional multiresolution image representations,” Ph.D.
thesis, EPFL, December 2001. (Chapter 3)
• Software and downloadable papers: www.ifp.uiuc.edu/∼minhdo
44