Extensions of wavelets
ECE 802
M-Band Wavelet Systems
• Generalization of dyadic wavelets
• Scale factor of M
• More flexible tiling of the time-frequency
plane
Properties
• Scaling Equation:
 (t )   h(n ) M  ( Mt  n )
1
H ( / M ) ( / M )
M
 ( ) 
• Existence and Orthogonality:
 h(n )  M
 h(n)h(n  Mm)   (m)
H ( )  H (  2 / M )    H (  2 ( M  1) / M )  M
2
2
• M-1 wavelets:
 l (t )   M hl (n ) ( Mt  n )
n
2
MRA
• At each scale j, there are M-1 wavelet
functions and one scaling function
V1  V0  W1,0  W2,0  W3,0
Wl , j  span l ( M j t  k )
• If the wavelets are orthogonal to the
scaling function at the same scale
h(n)h (n  Mk )  0
l
Analysis and Synthesis
• The expansion is
M 1
f (t )   c(k ) (t  k )   dl , j (k ) M j / 2 l ( M jt  k )
k
j
l 1
• The filter bank structure will now have M
branches
• Gives a mixture of a logarithmic and linear
frequency resolution.
• Easier to design for M=2k
Wavelet Packets
• M=2 results in a logarithmic frequency
resolution. The low frequencies have
narrow bandwidths and the high
frequencies have wide bandwidths.
• Wavelet packet system proposed by
Coifman
• Adjustable resolution of frequencies at
high frequencies
• Computational complexity O(NlogN)
Wavelet Packet Decomposition
• In order to have higher resolution decomposition at high
frequencies, iterate the highpass wavelet branch
• Split both the lowpass and highpass bands at all stages
• Evenly spaced frequency resolution
• In DWT we consider the outputs of each channel.
• In WPD, we have more outputs than inputs redundant
system
• Choose an independent set as basis (not one unique
basis)
Optimization Criteria
• Search based on minimizing a cost function on the
transform coefficients.
• Binary search algorithm for additive cost function
• How do we choose the ‘best’ basis?
–
–
–
–
Shannon entropy
Thresholding the coefficients
Log Energy
Norm of the coefficients
• Two approaches:
– Choose a particular decomposition based on the signal class
– Adapt the decomposition to each signal
Complexity
• P(J): The number of J-scale orthonormal
wavelet packet transforms
• P(1)=1
• P(J)=P(J-1)2+1
• Application: FBI standard for fingerprint
image compression
Haar Wavelet Packets
Wavelet Packet Tree
Optimization Functions
•
•
•
•
Shannon Entropy:
Norm:
Log-Energy:
Threshold Entropy: Number of times the
coefficient is larger than a threshold
Example: Minimum Entropy
Decomposition
•
•
Start with a constant original signal. w00 = ones(1,16)*0.25;
Compute entropy of original signal.
–
•
Then split w00 using the haar wavelet.
–
•
[w10,w11] = dwt(w00,'db1');
Compute entropy of approximation at level 1.
–
•
e10 = wentropy(w10,'shannon') e10 = 2.0794
The detail of level 1, w11, is zero; the entropy e11 is zero. Due to the additivity property the
entropy of decomposition is given by e10+e11=2.0794. This has to be compared to the initial
entropy e00=2.7726. We have e10 + e11 < e00, so the splitting is interesting.
Now split w10 (not w11 because the splitting of a null vector is without interest since the
entropy is zero).
•
–
•
[w20,w21] = dwt(w10,'db1');
We have w20=0.5*ones(1,4) and w21 is zero. The entropy of the approximation level 2 is
–
•
•
e20 = wentropy(w20,'shannon') e20 = 1.3863
Again we have e20 + 0 < e10, so splitting makes the entropy decrease.
Then
–
•
•
e00 = wentropy(w00,'shannon') e00 = 2.7726
[w30,w31] = dwt(w20,'db1'); e30 = wentropy(w30,'shannon') e30 = 0.6931 [w40,w41] =
dwt(w30,'db1') w40 = 1.0000 w41 = 0 e40 = wentropy(w40,'shannon') e40 = 0
Perform wavelet packets decomposition of the signal s.
t = wpdec(s,4,'haar','shannon');
Best Tree
Overcomplete Representations,
Frames, Redundant Transforms
• There are many cases where a single basis is
not effective for signal representation.
• Example: Fourier basis is good for sinusoids, but
bad for transients
• Efficiency of the transform can be improved by
combining several basis systems.
• Combination of basis systemsOvercomplete
• Collection of basis systems is called a dictionary.
Desired Criteria
• Sparsity: Efficient representation
• Separation: Better ability to separate a
mixture of signals
• Superresolution: Higher resolution or detail
compared to a single basis
• Stability: Robust under noise, the selected
atoms do not change
• Speed
Definitions
• Frame: Generalization of a basis, a
collection of functions that span the vector
space, but are not linearly independent
• The frame condition:
• 0<A<B<∞
• If A=B, tight frame
• If A=B=1, orthonormal basis
Frame Examples
• Tight Frame: 4 basis functions in 3dimensional space
1 1  1  1 a0 
g
(
0
)


1  1 1  1  a 
1
1
 g (1)  

 1 

 A 3 1 1
1
1   a2 
 g (2)

 

 a3 
1
T
g  FF g
A
Matching Pursuit
• Matching pursuit (MP) algorithm finds a
sub-optimal solution to the problem of an
adaptive approximation of a signal in a
redundant set (dictionary) of functions.
• Look for a linear expansion of a signal in
terms of elements (atoms) of a dictionary.
Algorithm
[Mallat, Zhang 1993]
• At each step, try to find the element of the
dictionary that ‘best’ fits the signal.
• Energy Conservation:
• For a complete dictionary as M∞, the residue
should go to zero.
• Stopping Criteria: Threshold the residue or predetermine M
Dictionary
• Commonly used dictionary: Gabor
functions, dictionary of time-frequency
atoms
• General and compact model for
oscillations
• Compact time-frequency localization
• Restrict the search to a range of time,
frequency, and scale values
Applications
• EEG Spike Parametrization:
Extensions
• Multichannel MP: Jointly represent a class of
signals using the same elements of the
dictionary
• Orthogonal Matching Pursuit (OMP):
– Efficient greedy algorithm
– Applies Gram-Schmidt orthogonalization to the
selected atoms before computing the residue
– The selected atoms are orthogonalized with respect
to the residue
– Faster convergence than MP
Basis Pursuit
[Chen, Donoho]
• Convex optimization: Find the
representation that minimizes the l1 norm
of the coefficients
• Solved using linear programming
• Nearly linear time
• l1 norm guarantees sparsity, l2 norm does
not (Method of Frames)
Examples
• FM-Cosine signal:
Comparisons
• MP and OMP are iterative algorithms.
• MP starts from an ‘empty’ signal model
and builds it up one atom at a time
• BP starts from a ‘full’ model and iteratively
improves the full model.
• Wavelet Packet Decomposition (Best
Orthogonal Basis) focuses only on the
orthogonal bases.
• MOF l2 solution, not sparse, can be noisy.