Lecture 4: Frequency domain representation, DTFT, IDTFT, DFT, IDFT Instructor: Dr. Gleb V. Tcheslavski Contact: gleb@ee.lamar.edu Office Hours: Room 2030 Class web site: http://ee.lamar.edu/gleb/dsp/ind ex.htm ELEN 5346/4304 DSP and Filter Design Fall 2008 1 2 Some history Jean Baptiste Joseph Fourier was born in France in 1768. He attended the Ecole Royale Militaire and in 1790 became a teacher there. Fourier continued his studies at the Ecole Normale in Paris, having as his teachers Lagrange, Laplace, and Monge. Later on, he, together with Monge and Malus, joined Napoleon as scientific advisors to his expedition to Egypt where Fourier established the Cairo Institute. In 1822 Fourier has published his most famous work: The Analytical Theory of Heat. Fourier showed how the conduction of heat in solid bodies may be analyzed in terms of infinite mathematical series now called by his name, the Fourier series. ELEN 5346/4304 DSP and Filter Design Fall 2008 3 Frequency domain representation Letxn Acos(0n )a (co)sinusoidal input (4.3.1) frequency A j (0n ) A j (0n ) A j j0n A j j0n xn e e e e e e 2 2 2 2 complex exponent of 0 complex exponent of -0 A sinusoidal signal is represented by TWO complex exponents of opposite frequencies in the frequency domain. ELEN 5346/4304 DSP and Filter Design Fall 2008 (4.3.2) 4 Frequency domain representation (cont) Letxn e j0n yn hk e j0 ( nk ) hk e j0k e j0n k k H e j0 xn The frequencyresponse :H e j hk e jk H e j e (4.4.1) j H e j phase (4.4.2) k magnitude Iff it exists! Forarealsystem :H e j hn e j n hn*e j n n H e n H e e e H e H e H e H e j H e j j j j j DSP and Filter Design j H e j (4.4.3) j j H e j has a period of 2 ELEN 5346/4304 * hn e jn H * e j n Fall 2008 (4.4.4) 5 Frequency domain representation (cont 2) 10 For an arbitrary real LTI system: Magnitude (dB) 5 0 -5 -10 Symmetric with respect to -15 0 0.2 0.4 0.6 0.8 1 1.2 1.4 Normalized Frequency ( rad/sample) 1.6 1.8 2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 Normalized Frequency ( rad/sample) 1.6 1.8 2 Anti-symmetric with respect to Phase (degrees) 100 50 0 -50 -100 ELEN 5346/4304 DSP and Filter Design Fall 2008 6 Frequency domain representation (cont 3) Combining (4.3.2) and (4.4.4) – back to our sinusoid! A j A e H e j0 e j0n e j H e j0 e j0n realsystem 2 2 j H e j0 j n j H e j0 j n A j A j j0 j0 0 e H e e e e H e e e 0 2 2 j 0 n H e j0 j 0 n H e j0 A j0 H e e e 2 Stady-state response: yn LTI filtering: yn A H e j0 cos 0 n H e j0 due to the same as from the input the input input change due to the system phase change due to the system Via design, we manipulate H(ej), therefore, hn, and, finally, manipulate the coefficients in the Linear Constant Coefficient Difference Equation (LCCDE) ELEN 5346/4304 DSP and Filter Design Fall 2008 (4.6.1) (4.6.2) (4.6.3) (4.6.4) 7 Frequency domain representation (cont 4) xn e j0n ,n 0 j n n ytot ,n ci i ke 0 initialconditions 0 i LCCDE: ytot ,n ke j0n for large enough n: yn hk e j0 n k un k k n j0 k j0 n j0 j0 n hk e e H ( e ) e k 0 (4.7.1) (4.7.2) (4.7.3) n H (e j ) hne jn hn if thesystemisBIBO,thefrequencyresponseexists n ELEN 5346/4304 DSP and Filter Design n Fall 2008 (4.7.4) 8 Frequency domain representation (cont 5) for an LTI: xn e j0nun yn ytr ,n yss,n (4.8.1) We don’t need systems of order higher than 2: can always make cascades. yss,n H (e j0 )e j0n existsonly forLTI ,BIBOsystems for a real, LTI, BIBO system: xn Acos(0 n ) yss ,n A H (e j0 ) cos(0 n H (e j0 ) effects of filtering We cannot observe ANY frequency components in the output that are not j present in the input (in steady state). We may see less when H (e r ) 0 ELEN 5346/4304 DSP and Filter Design Fall 2008 (4.8.2) 9 Frequency domain representation (cont 6) In continuous time: xt st nt {wewant} yt c st distortion lesstransmission signal noise Y () const yt e jt dt (4.9.1) delay j ( t ) j j c s e d ( t ) e ce S () H ()S () t We need a constant magnitude and linear phase for the frequencies of interest. Ideal filters: LPF H (e j0 HPF H (e ) j0 ) H (e ) H (e j0 BSF Fall 2008 H (e j0 ) ) Ideal filters are non-realizable! DSP and Filter Design BPF ELEN 5346/4304 j0 (4.9.2) 10 CTFT and ICTFT X () CTFT: x(t )e jt dt (4.10.1) X ()e jt d (4.10.2) ICTFT: 1 x(t ) 2 Examples: ELEN 5346/4304 DSP and Filter Design Fall 2008 11 DTFT X e j x e n j n n if exists acomplex function, periodic,continuouson (4.11.1) (4.11.2) X re e j X e j cos X e j , X im e j X e j sin X e j X e j X re 2 e j X im 2 e j 2 (4.11.3) What’s about convergence??? 1. Absolute convergence: if xn l1i.e. xn X e j convergesabsolutelyoruniformly n j Let X k (e ) ELEN 5346/4304 k n k DSP and Filter Design xn e jn lim X e j X k e j 0 k abs .error Fall 2008 (4.11.4) (4.11.5) 12 DTFT (cont) Absolutely summable sequences always have finite energy. However, finite energy sequences are not necessary absolutely summable. 2 an are not l . In fact , they are not l : x 1 2 n n Acos(0 n ) must be un 2. Mean-square convergence: lim k X e j X k e j 0 (4.12.1) energyof theerror The total energy of the error must approach zero, not an error itself! ELEN 5346/4304 DSP and Filter Design Fall 2008 13 IDTFT Let p 2 isa period , X e j xe Combining (4.11.1) and (4.12.2) 1 xl l 2 X e j ELEN 5346/4304 e j ( n l ) 2 n p n n 1 1 j j n x X ( e ) e d IDTFT: n p p 2 j X (e j )e jn d (4.13.2) 1 xn 2 jl jn x e l e d l sin (n l ) d xl xl nl xn (n l ) l shows where xn “lives” in the frequency domain. DSP and Filter Design (4.13.1) Fall 2008 (4.13.3) (4.13.4) 14 Back to ideal filters 1 Ideal LPF: H (e j0 ) 0 c 2 H LP e j 1,0 c 0,c (4.14.1) Using IDTFT: hLP ,n 1 2 c e j n c 1 e jc n e jc n d 2 jn jn si n c n , n n (4.14.2) 1. The response in (4.14.2) is not absolutely summable, therefore, the filter is not BIBO stable! 2. The response in (4.14.2) is not causal and is of an infinite length. As a result, the filter in (4.14.1) is not realizable. Similar derivations show that none of the ideal filters in slide 9 is realizable. ELEN 5346/4304 DSP and Filter Design Fall 2008 15 DTFT properties Linearity :ag n bxn aG e j bX e j (4.15.1) Shiftintime :g n n0 e j n0 G e j (4.15.2) DTFT DTFT dG e Differentiation :n g j Shiftin frequency :e j0 n g n G e j ( 0 ) (4.15.3) DTFT j d Linearconvolution :g n xn G e j X e j n (4.15.4) DTFT (4.15.5) DTFT 1 Periodicconvolution :g n wn DTFT 2 Parseval ' stheorem : g n w*n n ELEN 5346/4304 DSP and Filter Design j j ( ) G e W e d (4.15.6) 1 2 continuous, periodic functions j * j G e W e d Fall 2008 (4.15.6) 16 DTFTs of commonly used sequences ELEN 5346/4304 DSP and Filter Design Fall 2008 17 DTFT examples Letxn e j0n X e j 2 ( 0 2 k ) (4.17.1) 0 xn 1 X e j 2 ( 2 k ) (4.17.2) DTFT k DTFT k un un 1 n U e j 1 e j 1 U e j DTFT 1 c 2 ( l 2 ) j 1 e l (4.17.3) ½ of DC value of un 1 un ( l 2 ) DTFT 1 e j l (4.17.4) * * j n j n * j h e h e H e n n n n 1 1 j n j j n j n h g e H e e g e d n n n n 2 n 2 1 2 ELEN 5346/4304 DSP and Filter Design H e G e j j ( ) d Fall 2008 (4.17.5) H e g e j n j ( ) n d n (4.17.6) 18 DTFT examples (cont) We can re-work the Parseval’s theorem (4.15.6) as follows: Energy En hn 2 n 1 2 1 H e H e d 2 j * j H e j 2 d (4.18.1) Sh ( e j ) energy density (spectrum) Autocorrelation function: rgg ,l gn gnl * gn g(l n )* gl gl * G e j G* e j Sg e j n ELEN 5346/4304 DSP and Filter Design n DTFT Fall 2008 (4.18.2) 19 DTFT examples (cont 2) One obvious problem with DTFT is that we can never compute it since xn needs to be known everywhere! which is impossible! Therefore, DTFT is not practical to compute. Often, a finite dimension LTI system is described by LCCDE: N M N ai yni bm xnm ai e i 0 DTFT m 0 i 0 M Y e b e j X e j m0 N i 0 j b e M m0 m j m m a e Y e ji ji H e j i Prediction of steady-state behavior of LCCDE ELEN 5346/4304 DSP and Filter Design Fall 2008 j m X e j (4.19.1) practical (finite dimensions) (4.19.2) How to measure frequency response of an actual (unknown) filter? 20 1. Perform two I/O experiments: 1)xn ( c ) Acos(0 n ) yn ( c ) A H e j0 cos 0 n H e j0 ytr ,n ( c ) LTI j j (s) (s) (s) 2)xn Asin(0 n ) yn A H e 0 sin 0 n H e 0 ytr ,n LTI (4.20.1) (4.20.2) 2. Analyze these measurements and form: xn xn ( c ) jxn ( s ) Ae j (0 n ) j 0 n H e j0 j0 (c) (s) e ytr ,n yn yn jyn A H e jH e j0 yn j0 Finally H e e ztr ,n xn That’s a good way to measure/estimate a frequency response for every . ELEN 5346/4304 DSP and Filter Design Fall 2008 (4.20.3) (4.20.4) (4.20.5) 21 DFT and IDFT Consider an N-sequence xn (at most N non-zero values for 0 n N-1) X e j x e j n n xn e jn X e j n 0 n N 1 DFT: N 1 X k xn e 2 j kn N n 0 Usingnotation:WN e k 2 k N X k ,k0,1,..., N 1 (4.21.1) uniformly spaced frequency samples ,k0,1,..., N 1 (4.21.2) Finite sum! Therefore, it’s computable. j 2 N (4.21.3) N 1 (4.21.1) can be rewritten as: X k xnWN kn (4.21.4) n 0 1 N 1 IDFT :xn X kWN kn ,n 0,1,..., N 1 N n 0 Btw, DFT is a sampled version of DTFT. ELEN 5346/4304 DSP and Filter Design Fall 2008 (4.21.5) 22 DFT and IDFT (cont) Let us verify (4.21.5). We multiply both sides by WN l n N 1 x W n 0 n N N 1 l n 1 N 1 N 1 1 N 1 kn l n X kWN WN X kWN ( k l ) n N n 0 k 0 n 0 N k 0 N 1 xnWN l n n 0 1 N 1 N 1 ( k l ) n X k WN N k 0 n 0 N , fork l rN ,risInteger SinceWN ( k l ) n n 0 0,otherwise (4.22.1) (4.22.2) N 1 (4.22.3) N 1 xnWN l n X l (4.22.4) n 0 ELEN 5346/4304 DSP and Filter Design Fall 2008 23 FFT In the matrix form: where: X DN xDFT 1 x DN X IDFT (4.23.2) x[ x0 x1...xN 1 ]T timedomainsignal (4.23.3) X [ X 0 X1... X N 1 ]T NDFT samples (4.23.4) 1 1 1 1 W 1 WN 2 N DN 1 WN 2 WN 4 1 WN N 1 WN 2( N 1) (4.23.5) WN N 1 WN 2( N 1) DFT matrix ( N 1)( N 1) WN 1 1 1 1 W 1 WN 2 N 1 DN 1 1 WN 2 WN 4 N 1 WN ( N 1) WN 2( N 1) 1 WN ( N 1) WN 2( N 1) IDFT matrix ( N 1)( N 1) WN D 1 1 N This is actually FFT… ELEN 5346/4304 (4.23.1) DSP and Filter Design Fall 2008 (4.23.6) 1 DN * N (4.23.7) 24 Relation between DTFT and DFT 1. Sampling of DTFT Letxn X e j DTFT Sample X e j atk 2 k ,k 0,1,...N 1N frequencysamples X e jk Yk N point DFT N Yk yn ,n0,1,..., N 1 IDFT Yk X e jk 1 N 1 yn YkWN kn N k 0 j 2 Nk kl X e xlWN l 1 N 1 1 N 1 k ( nl ) kl kn xlWN WN xl WN N k 0 l l N k 0 Considering yn x m n mN ,n0,1,..., N 1 (4.24.1) (4.24.2) (4.24.3) yn is an infinite sum of shifted replicas of xn. Iff xn is a length M sequence (M N) than yn = xn. Otherwise, time-domain aliasing xn cannot be recovered! ELEN 5346/4304 DSP and Filter Design Fall 2008 25 Relation between DTFT and DFT (cont) 2. DTFT from DFT by Interpolation Let xn be a length N sequence: xn N pointDFT X k Let us try to recover DTFT from DFT (its sampled version). N 1 X e j xne jn n 0 N 1 j 2 k n N Since e n 0 N 1 N 1 j 2 kn 1 N 1 N 1 1 X kWN kn e jn X k e N e jn N n 0 k 0 N k 0 n 0 1 e j ( N 2 k ) e j ( N 2 k )/2 1 e j ( 2 k / N ) e j ( 2 k )/2 N (4.25.1) N 2 k N 2 k sin sin j 2 k N 1 2 2 e N 2 (4.25.2) N 2 k N 2 k sin sin 2N 2N N 2 k sin j 2 k N 1 1 N 1 1 2 e N 2 X e j X k N k 0 N N 2 k sin 2N N sin N 1 N 1 2 j 2 Xk e 1 k 0 sin 2 (4.25.3) It’s possible to determine DTFT X(ej) from its uniformly sampled version uniquely! ELEN 5346/4304 DSP and Filter Design Fall 2008 26 Relation between DTFT and DFT (cont 2) 3. Numerical computation of DTFT from DFT Let xn is a length N sequence: We wish to evaluate X e j xn X e j DTFT defined by N uniformly spaced samples at more dense frequency scale. k 2 k / M ,k 0,1,..., M 1,whereM N 1 N 1 X e jk xn e jk n xne n 0 Define: xl , n j 2 k N n M (4.26.1) n 0 xn ,0 n N 1 0, N n M 1 X e jk M 1 x n 0 l ,n e j 2 k zero-padding n M M 1 xl ,nWM kn n 0 No change in information, no change in DTFT… just a better “plot resolution”. ELEN 5346/4304 DSP and Filter Design Fall 2008 (4.26.2) (4.26.3) 27 A note on WN Recallthat:WN e j 2 N (4.27.1) WN is also called an Nth root of unity, since WN e j Im Im 2 N Re Re n ELEN 5346/4304 DSP and Filter Design Fall 2008 2 N 2 N 1 (4.27.2) 28 DFT properties 1. Circular shift xn is a length N sequence defined for n = 0,1,…N-1. An arbitrary shift applied to xn will knock it out of the 0…N-1 range. Therefore, a circular shift that always keeps the shifted sequence in the range 0…N-1 is defined using a modulo operation: xc ,n x( n n0 moduloN x nn0 N xnn0 ,n0 n N 1 xN n0 n ,0 n n0 x n n0 N N 1 xn ELEN 5346/4304 x n1 x n5 6 DSP and Filter Design 6 x n4 x n 2 6 1 N (4.28.1) N 1 k ( n n0 ) X W k N k 0 (4.28.2) 1 kn0 kn0 kn W X W W Xk N k N N DFT N k 0 6 Fall 2008 29 DFT properties (cont) 2. Circular convolution A linear convolution for two length N sequences xn and gn has a length 2N-1: N 1 yL,n g m xn m ,0 n 2 N 2 (4.29.1) m0 A circular convolution is a length-N sequence defined as: N 1 yC ,n gm x nm gn xn xn gn N g m 0 N 1 m x nm N 1 N 1 1 N 1 N 1 k ( nm) g m X kWN glWN km X kWN kn N k 0 N k 0 m0 m 0 x nm N 1 N 1 Gk X kWN kn Gk X k DFT N k 0 Procedure: take two sequences of the same length (zero-pad if needed), DFT of them, multiply, IDFT: a circular convolution. ELEN 5346/4304 (4.29.2) N m 0 N 1 N DSP and Filter Design Fall 2008 (4.29.3) 30 DFT properties (cont 2) Example: X e j xne jn (4.30.1) n Take N frequency samples of (4.30.1) and then IDFT: 2 2 N 1 j kl j 1 N 1 1 kn kn N N xn X kWN xl e WN WN e N k 0 N k 0 l 1 N 1 k ( n l ) xl WN xn rN N k 0 l r (4.30.2) aliased version of xn nl rN (4.22.3) r The results of circular convolution differ from the linear convolution “on the edges” – caused by aliasing. To avoid aliasing, we need to use zero-padding… ELEN 5346/4304 DSP and Filter Design Fall 2008 31 Linear filtering via DFT Often, we need to process long data sequences; therefore, the input must be segmented to fixed-size blocks prior LTI filtering. Successive blocks are processed one at a time and the output blocks are fitted together… We can do it by FFT: IFFT{FFT{x}FFT{h}}… Problem: DFT implies circular convolution – aliasing! yn hn xn (4.31.1) Assuming that hn is an M-sequence, we form an N-sequence (L - block length): xm,n xn mL n mL L 1 0otherwise N >> M; L >> M; N = L + M - 1 and is a power of 2 ELEN 5346/4304 DSP and Filter Design Fall 2008 (4.31.2) 32 Linear filtering via DFT (cont) Next, we compute N-point DFTs of xm,n and hn, and form Ym,k Hk X m,k k 0,1,...N 1 Ym , k ym ,n IDFT (4.32.1) - no aliasing! Since each data block was terminated with M -1 zeros, the last M -1 samples from each block must be overlapped and added to first M – 1 samples of the succeeded block. An Overlap-Add method. ELEN 5346/4304 DSP and Filter Design Fall 2008 33 Linear filtering via DFT (cont 2) Alternatively: Each input data block contains M -1 samples from the previous block followed by L new data samples; multiply the NDFT of the filter’s impulse response and the N-DFT of the input block, take IDFT. Keep only the last L data samples from each output block. The first block is padded by M-1 zeros. An Overlap-Save method. ELEN 5346/4304 DSP and Filter Design Fall 2008 34 DFT properties: General from Mitra’s book Btw, g[n] = gn ELEN 5346/4304 DSP and Filter Design Fall 2008 35 DFT properties: Symmetry from Mitra’s book xn is a real sequence ELEN 5346/4304 DSP and Filter Design Fall 2008 36 DFT properties: Symmetry (cont) from Mitra’s book xn is a complex sequence ELEN 5346/4304 DSP and Filter Design Fall 2008 37 N-point DFTs of 2 real sequences via a single N-point DFT Let gn and hn are two length N real sequences. Form xn = gn + jhn Xk (4.37.1) 1 X k X * k N 2 1 Hk X k X * k N 2j Gk Notethat X * k ELEN 5346/4304 DSP and Filter Design N X * N k (4.37.2) (4.37.3) (4.37.4) N Fall 2008 38 Summary Algorithm Time Frequency CTFT Continuous Continuous DTFT Discrete Continuous DFT Discrete Discrete ELEN 5346/4304 DSP and Filter Design Fall 2008