Outline
Multidimensional and Directional Filter Banks
1. Multidimensional filter banks
• Sampling lattices
• Filter design
Minh N. Do
2. Laplacian pyramid frames
Department of Electrical and Computer Engineering
University of Illinois at Urbana-Champaign
3. Iterated directional filter banks
www.ifp.uiuc.edu/∼minhdo
minhdo@uiuc.edu
Joint work with Martin Vetterli (EPFL and UC Berkeley),
Arthur Cunha, Yue Lu, Jianping Zhou (UIUC)
1. Multidimensional Filter Banks
1
1. Multidimensional Filter Banks
Filter Banks
Motivation
Wavelets and filter banks...
fundamental link between continuous and discrete domains
Looking for...
synthesis
analysis
Mallat
• Discrete-domain expansions for sampled data
H1
2
H0
2
y1
2
G1
2
G0
x1
+
x
• Structured transforms with fast algorithms
y0
x̂
x0
• Seamless connection with continuous-domain expansions
Answer: Filter banks and associated bases and frames.
2
2
G1
2
G0
+
+
1. Multidimensional Filter Banks
2
G1
2
1. Multidimensional Filter Banks
Daubechies
G0
3
Multidimensional Filter Banks
“True” Multidimensional Filter Banks
H
H1L
• Common approach: use 1-D techniques in a separable fashion
π
Q
x
ω2
−π
H
H0Η
π
Q
ω1
−π
• More flexibility:
– True multidimensional filters → directional filters
– Sampling via lattices → discrete rotations
• Limitations:
– very constrained filter design (separable filters)
– only rectangular frequency partition possible
1. Multidimensional Filter Banks
4
Sampling via Lattices
1. Multidimensional Filter Banks
5
Multidimensional Signals and Filtering
• Discrete-time d-dimensional signal:
In 1D (downsample by 2):
n1
x[n],
def
n = (n1, . . . , nd)T ∈ Zd
• The z-transform:
In 2D (downsample by 2):
X(z) =
X
x[n]z
−n
n
where z =
d
Y
zini
i=1
n∈Zd
• Filtering with h[n] yields
y[n] = x[n] ∗ h[n] =
X
x[k]h[n − k]
k∈Zd
Y (z) = X(z)H(z)
1. Multidimensional Filter Banks
6
1. Multidimensional Filter Banks
7
Multidimensional Sampling
Examples of Sampling Lattices
Sampling operation is represented by a d × d nonsingular integer matrix M .
Separable lattice in two dimensions:
Downsampling
xd [n] = x[M n].
Upsampling
xu[n] =
(
2 0
D1 =
0 2
2 2
D2 =
0 2
x[k] if n = M k, k ∈ Zd
0
otherwise.
Quincunx lattice in two dimensions:
Sampling lattice
def LAT (M ) = n : n = M k, k ∈ Zd
= all linear combinations of columns of M with integer coefficients
def
Sampling density is equal to |M | = |det M |
1. Multidimensional Filter Banks
8
Sampling Lattices and Matrices
1 1
D1 =
1 −1
1 2
D2 =
1 0
1. Multidimensional Filter Banks
9
Examples of Multidimensional Sampling
Theorem 1. LAT (A) = LAT (B) if and only if A = BE where E is a
unimodular (i.e. det E = ±1) integer matrix.
Example: Four basic unimodular matrices (shearing operators)
1 0
1 0
1 −1
1 1
.
, R3 =
, R2 =
, R1 =
R0 =
−1 1
1 1
0 1
0 1
Theorem 2. [Smith form] Any integer matrix M can be factorized as
M = U DV where U and V are unimodular integer matrices and D is an
integer diagonal matrices.
Example: For quincunx matrices
1 −1
,
Q0 =
1 1
Q1 =
1 1
−1 1
↓R
Q0 = R1D 0R2 = R2D 1R1 and Q1 = R0D 0R3 = R3D 1R0,
where D 0 = diag(2, 1) and D 1 = diag(1, 2)
1. Multidimensional Filter Banks
R3 =
10
1 0
−1 1
1. Multidimensional Filter Banks
Q1 =
↓Q
1 1
= R0 D 0 R3
−1 1
11
Multidimensional Sampling in Frequency-Domain (1/2)
Multidimensional Sampling in Frequency-Domain (2/2)
Upsampling by M
Downsampling by M
1
Xd(ω) =
|det(M )|
Xu(ω) = X(M T ω),
X
X(M −T (ω − 2πk)).
Xu(z) = X(z M ).
k∈N (M T )
def
Here z M = (z m1 , . . . , z md )T , where mi is the i-th column of M .
d
Here N (M ) is the set of integer vectors of the form M t, t ∈ [0, 1) .
1. Multidimensional Filter Banks
12
1. Multidimensional Filter Banks
Polyphase Representation
13
Examples of Polyphase Representations
Separable lattice
• Polyphase components with respect to the sampling matrix M :
xi[n] = x[M n + li]
M=
where {li} is the set of |M | integer vectors of the form M t, t ∈ [0, 1)
d
• Each polyphase component “lives” on a coset.
def C i = m : m = M n + l i , n ∈ Zd ,
|M |−1
[
C i = Zd ,
Ci
i=0
\
0 ≤ i ≤ |M | − 1
Quincunx lattice
Cj = ∅, i 6= j.
1 1
−1 1
|M |−1
X(z) =
X
z −li Xi(z M )
X(z1, z2) = X0(z1z2, z1z2−1) + z1−1X1(z1z2, z1z2−1)
i=0
1. Multidimensional Filter Banks
2 0
0 2
X(z1, z2) = X00(z12, z22)+z1−1X10(z12, z22)+z2−1X01(z12, z22)+z1−1z2−1X11(z12, z22)
M=
• In the z-domain:
14
1. Multidimensional Filter Banks
15
Polyphase-Domain Analysis
Filter Bank in the Polyphase-Domain
Filter and downsample
ANALYSIS
ANALYSIS
SYNTHESIS
H0
D
D
G0
H1
D
D
G1
x[n] −→ H(z) −→ (↓ M ) −→ c[n]
X
D
z −l1
D
SYNTHESIS
D
z l0
D
z l1
H p Gp
X
X̂
+
z −l0
+
X̂
|M |−1
C(z) =
X
Hi(z)Xi(z) = H(z)x(z)
HN −1
i=0
def
D
D
GN −1
z
−l|D|−1 D
D
z
l|D|−1
def
where x(z) = (X0(z), . . . , X|M |−1(z))T , and H(z) = (H0(z), . . . , H|M |−1(z))
Upsample and filter
• Critically sampled: |D| = N
• Perfect reconstruction: Gp(z)H p(z) = I|D|
c[n] −→ (↑ M ) −→ G(z) −→ p[n]
• Biorthogonal: Perfect reconstruction + critically sampled
Pi(z) = Gi (z)C(z)
or p(z) = G(z)C(z)
def
def
where G(z) = (G0(z), . . . , G|M |−1(z))T , and p(z) = (P0(z), . . . , P|M |−1(z))T
1. Multidimensional Filter Banks
16
• Orthogonal: H p(z) = GTp (z −1) and Gp(z)GTp (z −1) = I
(i.e. Gp(z) is a paraunitary matrix)
1. Multidimensional Filter Banks
A Reduction Theorem for Biorthogonal Filter Banks
Example: Quincunx Filter Bank
Theorem 3. [ZhouD:04] Suppose G(z) is an N × N matrix and
GN −1(z) is its submatrix obtained by deleting the last row of G(z).
Suppose H(z) is another N × N matrix and H N −1(z) is its submatrix
obtained by by deleting the last column of H(z).
Then G(z)H(z) = I N if and only if
GN −1(z)H N −1(z) = I N −1,
17
(1)
GN,i(z) = α(z)(−1)i+N det H i,N −1 (z),
(2)
Hi,N (z) = α−1(z)(−1)i+N det GN −1,i (z),
(3)
where α(z) = det G(z) is an arbitrary nonzero filter, H i,N −1 (z) is the
submatrix of H N −1(z) obtained by deleting its ith row, and GN −1,i (z) is
the submatrix of GN −1(z) obtained by deleting its ith column.
H0
Q
y0
Q
y1
Q
G0
x
+
H1
Q
x̂
G1
Possible frequency partitions:
ω1
(π, π)
111111111111
000000000000
000000000000
111111111111
1
000000000000
111111111111
000000000000
111111111111
000000000000
111111111111
0
000000000000
111111111111
000000000000
111111111111
ω0
000000000000
111111111111
000000000000
111111111111
000000000000
111111111111
000000000000
111111111111
000000000000
111111111111
ω1
(π, π)
111111111111
000000000000
000000000000
111111111111
0
000000000000
111111111111
000000000000
111111111111
1
000000000000
111111111111
000000000000
111111111111
000000000000
111111111111
ω0
000000000000
111111111111
000000000000
111111111111
000000000000
111111111111
000000000000
111111111111
000000000000
111111111111
Equivalent biorthogonal condition for FIR filters:
If G(z) and H(z) have FIR entries, then additionally α(z) = αz k
Corollary 1. A N -channel biorthogonal FIR filter bank is completely
determined by its first N − 1 channels, and a scaled delay in the last
channel.
1. Multidimensional Filter Banks
18
H0(z)G0(z) + H0(−z)G0(−z) = 2,
H1(z) = z
1. Multidimensional Filter Banks
−k
G0(−z),
and
k
G1(z) = z H0(−z).
19
Filter Design Problem
Design via Mapping
1. Start with a 1D solution:
Filter design amounts to solve
P (1D)(z) + P (1D)(−z) = 2 and P (1D)(z) = H (1D)(z)G(1D) (z)
P (z) + P (−z) = 2,
(easy)
2. Apply a 1D to MD mapping to each filter:
where
P (z) = H0(z)G0(z)
F (1D)(z) → F (MD)(z) = F (1D)(M (z))
(hard)
• If M (z) = −M (−z) then the perfect reconstruction condition is
preserved
Fundamental problem in multi-dimensions: no factorization theorem.
P (MD)(z) + P (MD)(−z) = P (1D)(M (z)) + P (1D)(M (−z)) = 2
How to avoid MD factorization?
• Can design the mapping to preserve and obtain other properties, like
linear phase, vanishing moments, directional vanishing moments, ...
(McClellan:73, TayK:93, ChenV:93, CunhaD:04)
One possibility: Map 1D filters to MD filters...
This only works for biorthogonal filter banks...
1. Multidimensional Filter Banks
20
MD Orthogonal FB Design using Cayley Transform
Orthogonal
Filter Banks
Paraunitary
matrices
Cayley
Transform
−1
Para-skew-Hermitian
matrices
Polyphase-domain:
I − U (z) .
U (z) =
U00(z) U01(z)
U10(z) U11(z)
Cayley-domain:
The paraunitary condition
U (z)U T (z −1) = I
becomes
Definition. H(z) is a para-skew-Hermitian (PSH) matrix if it satisfies
H(z −1) = −H T (z),
21
Example: Two-Channel Filter Banks
Definition. [Cayley transform] The Cayley transform of a matrix U (z) is
H(z) = I + U (z)
1. Multidimensional Filter Banks
for real coefficients.
Proposition 1. [ZhouDK:04] Suppose U (z) is an N × N matrix. Then
there exists at least one N × N diagonal matrix Λ whose diagonal entries
are either 1 or −1 such that I + ΛU (z) is nonsingular.
U00(z)U00(z −1) + U01(z)U01(z −1 ) = 1,
U00(z)U10(z −1) + U01(z)U11(z −1 ) = 0,
U10(z)U10(z −1) + U11(z)U11(z −1 ) = 1.
→ nonlinear equations
H(z) =
H00(z) H01(z)
.
H10(z) H11(z)
The PSH condition
H(z −1) = −H T (z)
becomes
H00(z −1) = −H00(z),
H11(z −1) = −H11(z),
H01(z −1) = −H10(z).
→ linear equations
Theorem 4. [ZhouDK:04] The Cayley transform of a paraunitary matrix
is a PSH matrix. Conversely, the Cayley transform of a PSH matrix is a
paraunitary matrix.
1. Multidimensional Filter Banks
22
1. Multidimensional Filter Banks
23
Outline
2. Laplacian Pyramid Frames
1. Multidimensional filter banks
2. Laplacian pyramid frames
• Framing pyramids
• New reconstruction (dual frame)
3. Iterated directional filter banks
decomposition
reconstruction
• Reason: avoid “frequency scrambling” due to (↓) of the HP channel.
• Laplacian pyramid as a frame operator → tight frame exists.
• New reconstruction: efficient filter bank for dual frame (pseudo-inverse).
2. Laplacian Pyramid Frames
24
2. Laplacian Pyramid Frames
Decomposition in the Laplacian Pyramid
25
Usual Reconstruction in the Laplacian Pyramid
c
x
H
M
M
G
p
_ +
d
c
G
+
x̂
d
G
H
M
x̂(z) = G(z)C(z) + d(z)
C(z) = H(z)x(z)
Or,
d(z) = x(z) − G(z)H(z)x(z) = (I − G(z)H(z))x(z).
C(z)
x̂(z) = G(z) I
.
| {z } d(z)
|
{z
}
S 1 (z)
y(z)
Combining gives
Note that S 1(z)A(z) = I (perfect reconstruction) for any H(z) and
G(z).
C(z)
H(z)
=
x(z).
d(z)
I − G(z)H(z)
| {z } |
{z
}
y(z)
2. Laplacian Pyramid Frames
But... what about noisy pyramids: ŷ = y + e ?
A(z)
26
2. Laplacian Pyramid Frames
27
Frame Analysis
A Tight Frame Pyramid
• LP is a frame operator (A) with redundancy.
Proposition. The Laplacian pyramid with orthogonal filters is a tight
frame with frame bounds equal to 1.
• It admits an infinite number of left inverses.
Proof: Orthogonal condition means G∗(z)G(z) = 1 and H(z) = G∗(z).
With this, we can directly verify that
• Let S be an arbitrary left inverse of A,
A (z)A(z) = H (z) I − H (z)G (z)
∗
∗
x̂ = S ŷ = S(y + e) = x + Se.
• The optimal left inverse (minimizing kSk) is the pseudo-inverse (or dual
frame) of A:
A† = (AT A)−1AT .
• If the noise is white, then among all left inverses, the pseudo-inverse
minimizes the reconstruction MSE.
2. Laplacian Pyramid Frames
28
New Reconstruction for the Laplacian Pyramid
†
T
A (z) = A (z) =
H(z)
I − G(z)GT (z)
T
H(z)
I − G(z)H(z)
Geometrical interpretation:
X
hx[•], g[• − M k]i g[n − M k].
p[n] =
|
{z
}
k∈Zd
= I.
x
c[k]
d
Using the Pythagorean theorem:
2
2
2
2
V
p
2
kxk = kpk + kdk = kck + kdk .
2. Laplacian Pyramid Frames
29
From the polyphase representation of A(z) and S(z)
c
M
M
H
G
T
K0
M
K|M |−1
M
x
d0
M
F0
+
x̂
= G(z) I − G(z)G (z) .
d|M |−1
Which leads to the optimal reconstruction:
M
F|M |−1
d
c
d
∗
Laplacian Pyramids as an Oversampled Filter Bank
For tight frame with frame bounds 1, the pseudo-inverse of A(z) is
∗
For the usual reconstruction (REC-1)
H
M
_ +
M
G
+
[1]
Fi (z) = z −ki ,
x̂
for i = 0, . . . , |M | − 1.
For the new reconstruction (REC-2)
[2]
Fi (z) = z −ki − G(z)Hi(z M ),
This also works for biorthogonal filters H and G.
2. Laplacian Pyramid Frames
30
2. Laplacian Pyramid Frames
for i = 0, . . . , |M | − 1.
31
Multilevel Laplacian Pyramids
2
G
2
G
2
F0
2
F0
2
F1
2
F1
+
PSfrag
+
Experimental Results (1/2)
x̂
x
LP
dec.
y
+
ŷ
threshold
usual
rec.
new
rec.
x̂1
x̂2
3.5
3.5
3
3
2.5
2.5
Magnitude
Magnitude
Frequency responses of equivalent synthesis filters: REC-1 and REC-2
2
1.5
1.5
1
0.5
0.5
0.2
0.4
0.6
0.8
0
0
1
Normalized Frequency (×π rad/sample)
M
Barbara REC-1
REC-2
Goldhill REC-1
REC-2
Peppers REC-1
REC-2
2
1
0
0
Nonlinear approximation: SNR’s of the reconstructed images from the M
most significant LP coefficients (after 6 levels of decomposition)
0.2
0.4
0.6
0.8
1
Normalized Frequency (×π rad/sample)
212
9.68
9.87
12.30
12.60
15.06
15.62
214
12.56
13.18
15.79
16.23
20.81
21.33
216
20.94
21.75
21.55
22.19
26.77
27.32
The new reconstruction is crucial for building multiscale directional frames.
2. Laplacian Pyramid Frames
PSfrag
32
2. Laplacian Pyramid Frames
Outline
Experimental Results (2/2)
x
LP
dec.
y
+
e
ŷ
usual
rec.
new
rec.
33
x̂1
1. Multidimensional filter banks
x̂2
2. Laplacian pyramid frames
3. Iterated directional filter banks
• How to obtain directional decomposition with 2D tree-structured filter
banks
• Local directional bases
With additive uniform white noise in [0, 0.1] (non-zero mean)
usual
rec.
SNR =
6.28 dB
2. Laplacian Pyramid Frames
new rec.
SNR =
17.42 dB
34
3. Iterated Directional Filter Banks
35
3. Iterated Directional Filter Banks
Our Simplified DFB: Two Building Blocks
• Feature: division of 2-D spectrum into fine slices using tree-structured
filter banks.
ω2
0
1
• Frequency splitting by the quincunx filter banks (Vetterli’84).
(π, π)
2
Q
3
7
4
6
5
5
6
4
2
1
Q
x
+
ω1
Q
7
3
y0
y1
x̂
Q
0
(−π, −π)
• Shearing by resampling
• Background: Bamberger and Smith (’92) cleverly used quincunx FB’s,
modulation and shearing.
• We propose:
– a simplified DFB with fan FB’s and shearing
– use DFB to construct directional bases
3. Iterated Directional Filter Banks
36
Illustration: The First Two Levels of the DFB
Q1
3. Iterated Directional Filter Banks
37
How Frequency is Divided into Finer Direction?
0
(l+1)
Q0
Q1
1
Q1
2
Q1
3
H2k
Q0
M (l+1)
Q
(l)
Hk
≡
M (l)
M (l+1)
Q
Using the multirate identity:
(l+1)
M
H(ω)
≡
T
H(M ω)
111111
000000
000000
111111
000000
111111
000000
111111
000000
111111
1
0
000000
111111
000000
111111
000000
111111
000000
111111
000000
111111
000000
111111
M
The support configurations of equivalent filters are:
ω1
(π, π)
111111111111
000000000000
000000000000
111111111111
0
000000000000
111111111111
000000000000
111111111111
1
000000000000
111111111111
000000000000
111111111111
000000000000ω0
111111111111
000000000000
111111111111
000000000000
111111111111
000000000000
111111111111
000000000000
111111111111
000000000000
111111111111
First level
3. Iterated Directional Filter Banks
ω1
(π, π)
1111111
0000000
0000000
1111111
0000000
1111111
0000000
1111111
0
0000000 1
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000ω0
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
0000000
1111111
Second level
ω1
0
(π, π)
1
2
3
3
2
1
ω0
H2k+1
0
Overall sampling:
Overall filters
38
(l)
M k = [2 · D il−2] · Rsl(k)
≡ separable sampling, then shearing (doesn’t change lattice
3. Iterated Directional Filter Banks
39
Multichannel View of the Directional Filter Bank
H0
S0
H1
S1
x
H2l−1
S 2l−1
y0
y1
y2l−1
Why Critical Sampling Works?
G0
S0
ω2
G1
S1
+
Hk
x̂
π
π
Sk
G2l−1
ω1
S 2l−1
Use two separable sampling matrices:
#
"

2l−1 0


0 ≤ k < 2l−1 (“near horizontal” direction)

 0
2
#
Sk = "

2
0



2l−1 ≤ k < 2l (“near vertical” direction)
 0 2l−1
3. Iterated Directional Filter Banks
Frequency tiling in the MDFB:
X
δRk (ω − 2πS −T
k m) = 1
for all k = 0, . . . , 2l − 1; ω ∈ R2.
m∈Z2
where Rk is the ideal frequency region for Hk .
40
General Bases from the DFB
3. Iterated Directional Filter Banks
41
Example of DFB Impulse Responses
An l-levels DFB creates a local directional basis of l2(Z2):
o
n
(l)
(l)
gk [· − Sk n]
l
2
32 equivalent filters for the first half channels (basically horizontal directions)
of a 6-levels DFB that use the Haar filters
0≤k<2 , n∈Z
(l)
• Gk are directional filters:
• Sampling lattices (spatial tiling):
2
2l−1
2
2l−1
3. Iterated Directional Filter Banks
42
3. Iterated Directional Filter Banks
43
References
• Multidimensional filter banks
– P. P. Vaidyanathan, Multirate Systems and Filter Banks, PrenticeHall, 1993.
– M. Vetterli and J. Kovačević, Wavelets and Subband Coding,
Prentice-Hall, 1995.
– J. Zhou, M. N. Do, and J. Kovačević, Multidimensional orthogonal
filter bank characterization and design using the Cayley transform,
IEEE Trans. on Image Proc., to appear.
• Laplacian pyramid frames
– M. N. Do and M. Vetterli, “Framing pyramids,” IEEE Trans. on
Signal Proc., pp. 2329-2342, Sep. 2003.
• Iterated directional filter banks
– M. N. Do,“Directional multiresolution image representations,” Ph.D.
thesis, EPFL, December 2001. (Chapter 3)
• Software and downloadable papers: www.ifp.uiuc.edu/∼minhdo
44