1 Lecture 13: Multirate processing and wavelets fundamentals Instructor: Dr. Gleb V. Tcheslavski Contact: gleb@ee.lamar.edu Office Hours: Room 2030 Class web site: http://ee.lamar.edu/gleb/ds p/index.htm ELEN 5346/4304 DSP and Filter Design Fall 2008 2 Filter banks A digital filter bank (FB) is a set of digital bandpass (usually) filters with either a common input (analysis FB) or a summed output (synthesis FB). Analysis FB separates the analyzed signal xn into a set of M spectral bands. A uniform filter bank – equal passband widths Let a causal IIR LPF with a real impulse response h0,n have a transfer function: H 0 (z) h0 , n z n (13.2.1) n0 Assume that the filter has its passband and stopband frequencies around /M as indicated in the picture, where M is an arbitrary integer. ELEN 5346/4304 DSP and Filter Design Fall 2008 3 Filter banks Let us consider next the casual filter with the impulse response obtained by modulating h0,n with the exponent: h k , n h0 , n e j 2 kn M (13.3.1) The corresponding transfer function is H k (z) h n0 k ,n z n h n0 o ,n j 2M k ze n j 2M k H 0 ze (13.3.2) ELEN 5346/4304 DSP and Filter Design Fall 2008 4 Filter banks The frequency response of the filter is H k e j j 2 k H 0 e M ;0 k M 1 (13.4.1) Therefore, the frequency response is obtained by shifting the frequency response of the LPF “to the right” by the amount 2k/M as shown. By designing M-1 filters obtained by modulating (shifting) the low pass prototype, we get M-1 uniformly shifted (in frequency) versions of the prototype H0. These M filters can be arranged into a uniform filter bank. Note: a DFT algorithm can be viewed in terms of a uniform filter bank. ELEN 5346/4304 DSP and Filter Design Fall 2008 5 Filter banks Two-channel Quadrature Mirror (QM) filter bank In many applications, a discrete-time signal xn is first split into a number of subband signals {vk,n} by an analysis FB; the subband signals are then (somehow) processed and finally combined by a synthesis FB to form an output yn. If the bandwidths of the subband signals are (chosen to be) much smaller than the bandwidth of the original signal, subband signals can be downsampled (sampled at a lower sampling rate) before the processing. This allows more efficient processing. Downsampling by the factor Q p s ELEN 5346/4304 N DSP and Filter Design Qp ’N = N/Q Qs Fall 2008 6 Filter banks After processing, these signals are upsampled and then combined by the synthesis bank into the output (higher rate) signal. The structure performing such operations is called a quadrature-mirror (QM) filter bank. Assuming no processing, a two-channel QM FB is: Note that for two channels, Q = 2. Analyzing this structure: Vk ( z ) H k ( z ) X ( z ) (13.6.1) Downsampling and upsampling can be represented as: U k (z) 1 V k z 1 2 V k z 1 2 2 2 Vˆk ( z ) U k ( z ) Here, k = 0,1. ELEN 5346/4304 DSP and Filter Design Fall 2008 (13.6.2) (13.6.3) 7 Filter banks The last equation can be rewritten as: 1 1 Vˆk ( z ) V k ( z ) V k ( z ) H k ( z ) X ( z ) H k ( z ) X ( z ) 2 2 (13.7.1) Therefore, the output of the FB is: Y ( z ) G 0 ( z )Vˆ0 ( z ) G 1 ( z )Vˆ1 ( z ) 1 2 H 0 ( z ) G 0 ( z ) H 1 ( z ) G1 ( z ) X ( z ) 1 2 H 0 ( z ) G 0 ( z ) H 1 ( z ) G1 ( z ) X ( z ) (13.7.2) Which can be rewritten as: Y ( z ) T ( z ) X ( z ) A( z ) X ( z ) T (z) where 1 H 0 ( z ) G 0 ( z ) H 1 ( z ) G1 ( z ) 2 is the distortion transfer function 1 A ( z ) H 0 ( z )G 0 ( z ) H 1 ( z ) G1 ( z ) 2 is the aliasing term. ELEN 5346/4304 DSP and Filter Design Fall 2008 (13.7.3) (13.7.4) (13.7.5) 8 Filter banks In general, QM FB are Linear Time-Varying systems. However, by selecting the analysis and synthesis filters such that aliasing effects are canceled, QM FB can be made an LTI system. Aliasing cancellation condition A ( z ) 0 H 0 ( z ) G 0 ( z ) H 1 ( z ) G1 ( z ) 0 Will hold when, for instance: In this situation: (13.8.1) G 0 ( z ) H 1 ( z ); (13.8.2) G1 ( z ) H 0 ( z ) (13.8.3) Y (z) T (z) X (z) (13.8.4) If T(z) is an allpass filter, i.e. |T(z)| = c 0 Y (z) c X (z) (13.8.5) Therefore, amplitude distortions are possible. We can also show that no phase distortions will be introduced if T(z) has a linear phase. ELEN 5346/4304 DSP and Filter Design Fall 2008 9 Filter banks Perfect reconstruction condition An alias-free QM FB having no amplitude and phase distortions is called a perfect reconstruction QM FB. In this case: H 0 ( z ) G 0 ( z ) H 1 ( z ) G1 ( z ) 2 z which implies that: Therefore: and T (z) z l l l (13.9.1) (13.9.2) Y (z) z X (z) (13.9.3) y n xn l (13.9.4) Indicating that the reconstructed output is a delayed by l samples replica of the input. ELEN 5346/4304 DSP and Filter Design Fall 2008 10 Filter banks (QM FB example) Let us consider a two-channel QM FB with the analysis filters: 1 H 0 (z) 2 1 z a LPF ; H 1 (z) 1 1 2 1 z a H PF (13.10.1) 1 z a H PF (13.10.2) 1 and the synthesis filters: G0 ( z ) 1 2 1 z 1 a LPF ;G1 ( z ) 1 2 1 Therefore: A ( z ) H 0 ( z ) G 0 ( z ) H 1 ( z ) G 1 ( z ) 1 2 1 z 2 1 2 1 2 1 1 z 0 2 DSP and Filter Design 1 1 1 2 (13.10.3) the two-channel QM FB is an alias-free system. ELEN 5346/4304 1 1 z 1 z 1 z 1 z Fall 2008 11 Filter banks (QM FB example) On the other hand: T (z) 1 2 H 0 ( z ) G 0 ( z ) H 1 ( z ) G1 ( z ) 1 4 1 z 1 1 z 1 1 4 1 z 1 z 2 z 1 1 1 (13.11.1) Therefore, this FB is a perfect reconstruction system. In a time domain, the outputs of the analysis filters are: u 0 ,n Note: the scaling factor 1 1 2 x n x n 1 ,u1, n 1 2 x n x n 1 2 preserves energy. In a matrix form: u 0 ,n xn H2 u x 1, n n 1 where H2 1 1 2 1 The reconstruction: ELEN 5346/4304 DSP and Filter Design (13.11.2) 1 1 is the Haar transform matrix u 0 ,n xn t H2 u 1, n x n 1 Fall 2008 (13.11.3) (13.11.4) (13.11.5) 12 Filter banks From alias-free and perfect reconstruction requirements: G0 ( z ) H 0 ( z ) Therefore: For a real filter: (13.12.1) H 1 ( z ) G0 ( z ) H 0 ( z ) (13.12.2) G1 ( z ) H 1 ( z ) H 0 ( z ) (13.12.3) H1 e j H0 e j (13.12.4) Indicating that if h0 is a LPF, than h1 is a HPF; in fact, its mirror image with respect to the /2 – the quadrature frequency. Therefore, two analysis filters and two synthesis filters in the QM FB are essentially determined from one prototype transfer function. QM FB can be build from FIR or IIR filters. ELEN 5346/4304 DSP and Filter Design Fall 2008 13 Filter banks Orthogonal QM FB G0 ( z ) z N 1 H 0 ( z ),G 1 ( z ) z N 1 H 1 ( z ), H 1 ( z ) z N 1 H 0 ( z ) (13.13.1) Implying that if H0(z) is a causal FIR filter, three other filters must be causal FIR filters. Also, if H0(z) is a LPF, than H1(z) is a HPF. Third-order FIR filters forming analysis QM FB 7-order FIR filters forming analysis QM FB Biorthogonal QM FB – QM FB obtained for the perfect reconstruction condition and represented by nonorthogonal transfer matrices (and filters). ELEN 5346/4304 DSP and Filter Design Fall 2008 14 Filter banks L-channel QM FB is a generalization of a two-channel QM FB. 0 k L 1 Vk ( z ) H k ( z ) X ( z ) U k (z) 1 L 1 L Hk z 1 L (13.14.1) e j 2 l L X z 1 L e j 2 l L (13.14.2) l0 L Vˆk ( z ) U k ( z ) ELEN 5346/4304 DSP and Filter Design (13.14.3) Fall 2008 15 Filter banks The vector of downsampled subband signals of an L-channel QM FB can be expressed in the matrix form as: u(z) 1 H L z 1 L t x (m ) z 1 L u ( z ) U 0 ( z )U 1 ( z )...U L 1 ( z ) Where: The input is: x (m ) H 0 (z) H0 z e (m ) H (z) H z e j 2 L 1 L 0 t (13.15.2) H1(z) H1 z e j 2 L DSP and Filter Design (13.15.1) j 2 L 1 L j 2 L ( z ) X ( z ) X ( z e )... X ( z e ) The analysis FB modulation matrix: ELEN 5346/4304 (m ) H1 z e Fall 2008 j 2 L t (13.15.3) H L 1 ( z ) j 2 L 1 L H L 1 z e H L 1 z e j 2 L j 2 L 1 L (13.15.4) 16 Filter banks The output of the QM FB is L 1 G Y (z) k ( z )Vˆk ( z ) (13.16.1) k 0 which in the matrix form is y where the output is: y (m ) (m ) (z) G ( z ) Y ( z )Y ( z e The synthesis FB modulation matrix: G0 ( z) T(z) 1 L ELEN 5346/4304 DSP and Filter Design j 2 L G1 z e j 2 L ( z )u z L (13.16.2) )...Y ( z e j 2 L 1 L G1 ( z ) G0 z e (m ) G (z) G z e j 2 L 1 L 0 The transfer matrix: (m ) G1 z e G (m ) Fall 2008 (13.16.3) G L 1 ( z ) j 2 L j 2 L 1 L ( z ) H ) t (m ) G L 1 z e ( z ) G L 1 z e j 2 L j 2 L 1 L (13.16.4) t (13.16.5) 17 Filter banks From the alias cancellation condition, the set of synthesis filters can be determined: G ( z ) H (m ) ( z ) 1 t( z) t ( z ) L T ( z )...0 Where and the distortion transfer function is: T ( z ) a0 ( z ) 1 (13.17.1) t L 1 H L k ( z )G k ( z ) k 0 If T(z) has a constant magnitude, the QM FB is magnitude-preserving. If T(z) has a linear phase, the QM FB has no phase distortion. If T(z) is a pure delay, it is a perfect reconstruction QM FB i.e.: T (z) z n0 (13.17.4) Three-channel FIR QM FB ELEN 5346/4304 DSP and Filter Design (13.17.2) Fall 2008 (13.17.3) 18 Filter banks Multilevel FB A multichannel FB can be replaced by a multilevel FB called a tree-structured FB. Analysis and synthesis filters can be related between these two structures: H 00 ( z ) H 0 ( z ) H 0 ( z ), H 01 ( z ) H 0 ( z ) H 1 ( z ) 2 2 H 10 ( z ) H 1 ( z ) H 0 ( z ), H 11 ( z ) H 1 ( z ) H 1 ( z ) 2 2 G 00 ( z ) G 0 ( z ) G 0 ( z ),G 01 ( z ) G 0 ( z ) G 1 ( z ) 2 2 G 10 ( z ) G 1 ( z ) G 0 ( z ),G 11 ( z ) G 1 ( z ) G 1 ( z ) 2 ELEN 5346/4304 DSP and Filter Design 2 Fall 2008 (13.18.1) 19 Filter banks FB considered so far have equal passband widths (full tree FB). Sometimes, it is beneficial to make FB with unequal passband widths (partial tree FB). H 000 ( z ) H 0 ( z ) H 0 ( z ) H 0 ( z ), H 001 ( z ) H 0 ( z ) H 0 ( z ) H 1 ( z ) 2 4 2 4 H 01 ( z ) H 0 ( z ) H 1 ( z ), H 1 ( z ) (13.19.1) 2 G 000 ( z ) G 0 ( z ) G 0 ( z ) G 0 ( z ),G 001 ( z ) G 0 ( z ) G 0 ( z ) G 1 ( z ) 2 4 2 G 01 ( z ) G 0 ( z ) G 1 ( z ),G 1 ( z ) 2 Corresponding filters’ magnitude responses ELEN 5346/4304 DSP and Filter Design Fall 2008 4 20 Discrete Wavelet Transform (DWT) It turns out that perfect reconstruction octave band QM FB can be represented by a certain discrete wavelet transform (DWT). For this binary tree, there are two “parent” analysis FIR filters: H0(z) – LPF, and H1(z) – HPF. Which can also be represented by the equivalent structure where 0 k L and: i k 1 k 2 2 2 H k ,1 ( z ) H 0 z H 1 z i0 k 1 H k ,0 ( z ) i0 ELEN 5346/4304 DSP and Filter Design H0 z 2 (13.20.1) i (13.20.2) Fall 2008 21 Discrete Wavelet Transform (DWT) Assuming that the input xn is a sequence of length M, the outputs will have the following approximate lengths and bandwidths: The downsampling leads to: 2 downsampling by 2 2 In the wavelet decomposition, u0,n represents the “finest details” (the highest frequency components) of the input xn, while u4,n is its “coarsest” representation (its lowest frequency components). ELEN 5346/4304 DSP and Filter Design Fall 2008 22 Discrete Wavelet Transform (DWT) The process of generating the set of output subsequences uk,n where 0 k L from the finite-length input xn by the structure that can be represented as an Llevel octave-band perfect reconstruction QM FB is called the discrete wavelet transform (DWT). The analysis sequences for the equivalent representation are: X k ( z ) H k ,1 ( z ) X ( z ),0 k L 1 X k ( z ) H L ,1 ( z ) X ( z ), k L or in the time domain: (13.22.1) xk ,n h k ,1, n m x m ,0 k L 1 m (13.22.2) x L ,n h L ,0 , n m x m , k L m The subband sequences: u k ,n m h k ,1, 2 k 1 n m x m ,0 k L 1 (13.22.3) u L ,n m ELEN 5346/4304 DSP and Filter Design h L ,0 , 2 k n m x m ,k L Fall 2008 23 Discrete Wavelet Transform (DWT) The process of reconstruction (synthesis) of the output sequence yn – which is a replica of the input xn – from the subband sequences un can be accomplished by the L-level octave-band perfect reconstruction QM FB: Which has an equivalent structure, where i k 1 k 2 2 2 G k ,1 ( z ) G 0 z G 1 z i0 k 1 G k ,0 ( z ) i0 ELEN 5346/4304 DSP and Filter Design G0 z 2 (13.23.1) i (13.23.2) Fall 2008 24 Discrete Wavelet Transform (DWT) Therefore, the output will be: G L Y (z) G k ,1 ( z )U k 1 z k 1 2 (13.24.1) g L ,0 , n 2 L m u L , m (13.24.2) k L ,0 ( z )U L z 2 L or, in the time domain: L yn k 1 m Using the notation: g k ,1, n 2 k 1 m u k , m k , m , n g k ,1, n 2 k 1 L , m , n g L ,0 , n 2 the output will be: L yn L m m m ,0 k L 1 , k L (13.24.3) k ,m ,n u k ,m (13.24.4) k 0 m Which is the inverse DWT. uk,m are the wavelet coefficients of xm with respect to the basis function k,m,n ELEN 5346/4304 DSP and Filter Design Fall 2008 25 Discrete Wavelet Transform (DWT) Biorthogonal wavelets The biorthogonal wavelets are generated from biorthogonal octave-band perfect reconstruction QM FB. The simplest examples are Haar wavelets: ELEN 5346/4304 DSP and Filter Design Fall 2008 26 Discrete Wavelet Transform (DWT) Orthogonal wavelets The orthogonal wavelets are generated from orthogonal octave-band perfect reconstruction QM FB. One example would be the four-level Daubechies 4/4-tap wavelet: ELEN 5346/4304 DSP and Filter Design Fall 2008 27 On the Uncertainty Principle Observations: • A short-duration (“narrow”) waveform (signal) yields a long-duration (“wide”) spectrum. • A narrow spectrum yields a long-duration (“wide”) waveform. • Waveform and spectrum cannot be made arbitrary small simultaneously. These observations are explained by the Uncertainty Principle: C onst 2 t 2 1 (13.27.1) 4 “=“ for Gaussian signals Here t2 is a variance in time – a measure of the duration of the signal; 2 is a variance in frequency – a measure of the bandwidth of the signal (most of the signal energy will be concentrated in 2B = 2 rad/s. Consequence: the same signal cannot have both: fine resolution in time and fine resolution in frequency. ELEN 5346/4304 DSP and Filter Design Fall 2008 28 Non-stationary… So what? Signals Estimated spectra 2 1.5 Case 1: 1 Stationary: 0.5 0 20 0 -20 -1.5 0 300 400 500 600 700 800 900 0 100 1000 1 0.8 0.6 Case 2: 0.4 0.2 Non – stationary: 0 -0.2 -0.4 S1 S2 -0.6 -0.8 200 400 ELEN 5346/4304 600 800 1000 1200 DSP and Filter Design 1400 1600 1800 2000 Fall 2008 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Normalized Frequency ( rad/sample) 0.9 1 50 0 -50 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Normalized Frequency ( rad/sample) 0.9 1 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Normalized Frequency ( rad/sample) 0.9 1 5 1 x 10 0 -1 -2 0 0.9 -1 Magnitude (dB) 200 0.3 0.4 0.5 0.6 0.7 0.8 Normalized Frequency ( rad/sample) 0 Phase (degrees) 100 0.2 x 10 -2 0 0.1 5 1 Phase (degrees) -1 -1 40 S1 + S2 -0.5 -2 Magnitude (dB) 60 0 29 Non-stationary… So what? Ideally: from “Modern DSP” by Cristi ELEN 5346/4304 DSP and Filter Design Fall 2008 30 Short-Time Fourier Transform* The Short-Time Fourier Transform (STFT) of a windowed signal with the window centered at t is: X (t , ) w x ( ) w ( t ) e * j d (13.30.1) where w(t) is a low pass analysis window function, and (typically): 2 w ( t ) dt 1 (13.30.2) The inverse STFT is defined as: x (t ) 1 2 X ( , ') ( t ) e w j 't d d ' (13.30.3) Given that: w ( t ) ( t ) dt 1 * (13.30.4) * - based on the lecture notes for “Advanced topics in Signal Processing” by Dr. Ben H. Jansen, UH, 2006 ELEN 5346/4304 DSP and Filter Design Fall 2008 31 Short-Time Fourier Transform A natural choice for the synthesis window (t), is to make it equal to the analysis window w(t). 1. STFT is a linear time-frequency representation; 2. STFT depends on the choice of w(t); 3. STFT is generally complex. Defining a spectrogram as: X (t , ) w 2 which is a representation of the energy distribution in the time-frequency plane, i.e. a quadratic time-frequency representation. We can show that: ELEN 5346/4304 2 E X (t , ) d w X ( t , ) dt w DSP and Filter Design X ( t , ) dtd w 2 2 x ( ) 2 X ( ') 2 2 x ( t ) dt w ( t ) dt 2 w ( t ) d x ( t ) 2 2 (13.31.1) 2 W ( ') d ' X ( ) Fall 2008 (13.31.2) 2 (13.31.3) 32 Short-Time Fourier Transform We can notice that: e j t x (t ) w * ( t ) x ( ) w ( t ) e * j d (13.32.1) Therefore, the STFT can be viewed as a filtering operation by a LPF with impulse response w*(-t) as shown in (A): Also: * j t x ( t ) w ( t ) e e x ( ) w ( t ) e * j t X (t , ) j e j t d (13.32.2) Therefore, the STFT can also be viewed as a filtering operation with a BPF with impulse response ejt w*(-t) as shown in (B): ELEN 5346/4304 DSP and Filter Design Fall 2008 33 Discrete STFT Define a sampled version of STFT as: X ( nt , kF ) w x ( ) w ( nT ) e * jk 2 F d (13.33.1) Therefore, the DSTFT: X n ,k w x j * m wm n e 2 km , k 0,1, ... L 1 L (13.33.2) m The IDSTFT: x (t ) n or: xm n X ( nT , kF ) ( t nT ) e w jk 2 Ft (13.33.3) k X n ,k m n e w j 2 kn L ,k 0,1, ..., L 1 (13.33.4) k The IDSTFT holds if: 1 F ELEN 5346/4304 DSP and Filter Design t n * nT w t nT ( k ) for all t F k Fall 2008 (13.33.5) 34 STFT: Gabor expansion (transform) Note that (13.33.3) can be viewed as a series expansion of a 1-D signal in terms of 2-D time-frequency functions. The Gabor expansion is defined as: x (t ) G n x ( n , k ) g n ,k (t ) (13.34.1) k where g(t) is a 1-D function: g n ,k (t ) g (t n T )e jk 2 F t (13.34.2) If the basis functions gn,k(t) are selected such that they are well localized in time and well concentrated in frequency, then the Gabor coefficients Gx(n,k) will reflect the signal’s time-frequency content around (nT, kF). Gabor proposed g (t ) which is a Gaussian signal with e t 2 (13.34.3) t 1 4 2 2 The Gabor expansion is always possible if FT 1, but the Gabor coefficients are not unique (the DSTFT is just one possible solution). Note: FT of a Gaussian is a Gaussian. ELEN 5346/4304 DSP and Filter Design Fall 2008 35 Time-Frequency resolution of STFT Using the filter representation of STFT, we observe: 1. All the BPFs have the same duration impulse response, hence the time resolution of the STFT is the same at all frequencies and is proportional to window length. 2. All the BPFs have the same bandwidth, hence the frequency resolution is the same at all times and is inversely proportional to window length. 1/T T Although, according to the uncertainty principle, 2 t2 C onst , t could be made large for low frequency components and small for high frequency components… leading to corresponding changes in the frequency resolution. ELEN 5346/4304 DSP and Filter Design Fall 2008 36 Continuous Wavelet Transform (CWT) Rather than using constant bandwidth BPFs (as STFT does), we use constant Q BPFs, where Q c B center frequency (13.36.1) bandwidth In other words, the impulse response will decrease in duration (and the bandwidth increase) as center frequency increases: therefore, use scaled versions of w*(-t). For simplification, we will use (as window functions) the scaled versions of the same prototype (t): (13.36.2) (t ) t c c Here (t) is the analyzing (mother) wavelet function: a bandpass function centered around c. ELEN 5346/4304 DSP and Filter Design Fall 2008 37 Continuous Wavelet Transform (CWT) The CWT is defined as: ( ) Wx (t , ) x ( ) * t d c c (13.37.1) Using b = c/, the CWT can also be defined as a time-scale representation: ( ) Wx (t , b ) x ( ) * t d b b 1 The energy distribution in the time-domain is evaluated by the scalogram: W ELEN 5346/4304 DSP and Filter Design ( ) x 2 (t , b ) Fall 2008 (13.37.2) 38 STFT vs. CWT STFT estimates the similarity (cross-correlation) between x(t) and g,(t): X ( , ) Where: * x ( t ) g , ( t ) dt g , ( t ) w ( t ) e j t (13.38.1) (13.38.2) CWT estimates the similarity (cross-correlation) between x(t) and hb,(t): W ( , b ) Where: ELEN 5346/4304 DSP and Filter Design hb , ( t ) * x ( t ) hb , ( t ) dt (13.38.1) t b b (13.38.2) 1 Fall 2008 39 Same time and frequency resolutions STFT vs. CWT STFT: FT = const, F = const, T = const CWT: FT = const, F const, T const from “Modern DSP” by Cristi ELEN 5346/4304 DSP and Filter Design Fall 2008