ELEN E4810: Digital Signal Processing Topic 3: Fourier domain 1. 2. 3. 4. The Fourier domain Discrete-Time Fourier Transform (DTFT) Discrete Fourier Transform (DFT) Convolution with the DFT Dan Ellis 2013-09-23 1 1. The Fourier Transform Basic observation (continuous time): A periodic signal can be decomposed into sinusoids at integer multiples of the fundamental frequency i.e. if x˜ (t) = x˜ (t +T ) we can approach x˜ with M ak cos x̃(t) k=0 Dan Ellis 2 k t+ T 2013-09-23 k Harmonics of the fundamental 2 M Fourier Series ak cos k=0 For a square wave, = 0; k i.e. ak = x(t) = cos 2 t T ( 1) 0 k 2 1 1 cos 3 1 k 2 k t+ T k k = 1, 3, 5, . . . otherwise 2 1 3t + cos T 5 2 5t T ... 1 0.5 0 0.5 1 1.5 Dan Ellis 1 0.5 0 2013-09-23 0.5 1 1.5 3 M Fourier Domain x is equivalently described by its Fourier Series parameters: ak = ( 1) k 2 1 1 k k = 1, 3, 5, . . . ak 1.0 ¡k k 1 2 3 4 5 6 7 k π Negative ak is equivalent to phase of º M 1 2 3 4 5 6 7 Complex form: x̃(t) j 2T k t ck e k= M Dan Ellis 2013-09-23 4 M Fourier Analysis x̃(t) ck e k= M How to find {|ck |}, {arg{ck }} ? Inner product with (conjugate) complex sinusoids: 1 ck = T Dan Ellis T /2 x(t)e j 2T k t dt T /2 2013-09-23 j 2T k t 5 M Fourier Analysis 1 ĉk = T T /2 x(t)e x 1 = 2 e cl ejl dt ; call jk e 1 2 ej(l k) 2 = t T d jk l cl l T 2 ck e k= M T /2 1 = 2 = j 2T k t x̃(t) d d Integral of (l-k) complete cycles of a complex sinusoid; = 0 for l≠k ∵ real (cos) part is complete cycles, imag (sin) part is odd; = 1 for l=k ∵ ∫ 1dτ = ck Dan Ellis j 2T k t 2013-09-23 6 Fourier Series Analysis 1 Thus, ck = T T /2 x(t)e j 2T k t dt T /2 j 2T k t because complex sinusoids e pick out the corresponding sinusoidal components linearly combined in M j 2T k t x(t) = ck e k= M Dan Ellis 2013-09-23 7 Fourier Transform Fourier series for periodic signals extends naturally to Fourier Transform for any (CT) signal (not just periodic): dt Fourier Transform (FT) j t Inverse Fourier Transform (IFT) X(j ) = x(t)e 1 x(t) = 2 X(j )e j t d Discrete index k → continuous freq. Ω Dan Ellis 2013-09-23 8 Fourier Transform Mapping between two continuous functions: |X( )| 0 level / dB -20 0.02 x(t) -40 -60 -80 0 0.01 4000 6000 8000 freq / Hz arg{X( )} 0 -0.01 0 2000 0.002 0.004 0.006 0.008 time / sec /2 0 /2 2π ambiguity Dan Ellis 2013-09-23 0 2000 4000 6000 8000 freq / Hz 9 Fourier Transform of a sine Z Assume x(t) = e 1 Now, since x(t) = j 0 t • X(W)e 2p • ...we know X(W) = 2pd(W jWt dW W0) ...where ±(x) is the Dirac delta function δ(x-x0) (continuous time) i.e. ( x x0 ) f ( x) dx = f ( x0 ) → x(t) = Ae Dan Ellis j 0 t x0 f(x) x X() = A ( 0 ) 2013-09-23 10 Fourier Transforms Fourier Series (FS) Fourier Transform (FT) Discrete-Time FT (DTFT) Discrete FT (DFT) Dan Ellis Time Frequency Continuous periodic ~x(t) Discrete infinite ck Continuous Continuous infinite x(t) infinite X(Ω) Discrete Continuous infinite x[n] periodic X(ejω) Discrete Discrete finite/pdc ~x[n] finite/pdc X[k] 2013-09-23 11 2. Discrete Time FT (DTFT) FT defined for discrete sequences: j X(e ) = x[n]e jn DTFT n= Summation (not integral) Discrete (normalized) frequency variable ! Argument is ej!, not j! Dan Ellis 2013-09-23 12 DTFT example e.g. x[n] = Æn·μ[n], |Æ| < 1 X(e ) = n= µ[n]e j = n=0 (e Re n |X(ej )| ) 2 0 2 3 Dan Ellis 4 5 3 4 5 n jn S = c cS = c n n n =1 S cS = c = 1 ( |c | < 1 ) 1 S = 1c 0 2 1 1 2 3 4 5 6 7 n =0 arg{X(ej )} 3 n j n 1 = j 1 e 1 - e-j Im -1 2013-09-23 13 Periodicity of X(ej!) X(ej!) has periodicity 2º in ! : X(e j ( +2 ) ) = x[n]e = x[n]e |X(ej )| j ( +2 )n jn e j 2 n j = X(e ) arg{X(ej )} 3 2 2 1 2 0 3 4 3 4 5 5 Phase ambiguity of ej! makes it implicit Dan Ellis 2013-09-23 14 Inverse DTFT (IDTFT) Same basic “Fourier Synthesis” form: 1 x[n] = 2 X(e j )e jn d IDTFT Note: continuous, periodic X(ej!) discrete, infinite x[n] ... IDTFT is actually Fourier Series analysis (except for sign of !) Dan Ellis 2013-09-23 15 IDTFT Verify by substituting in DTFT: 1 x[n] = 2 1 = 2 X(e j )e jn d (l x[l]e )e jl jn d 1 j (nl ) = l x[l] e d 2 = l x[l]sinc (n l)= x[n] Dan Ellis 2013-09-23 = 0 unless n=l i.e. = δ[n-l] 16 sinc 1 = 2 j (n e 1 l) dw = 2 Same as ∫cos Dan Ellis ej (n l) j(n l) 1 ej (n l) e j (n = 2 j(n l) 2j sin (n l) = sinc (n j(n l) ∴ 1 2 l) l) imag jsin part cancels 2013-09-23 17 sinc sin x sincx x ∆ = 1.0 4 3 sin(x)/x 2 2 3 4 x sin(x) y=x = 1 when x = 0 = 0 when x = r·º, r ≠ 0, r = ±1, ±2, ±3,... Dan Ellis 2013-09-23 18 DTFTs of simple sequences j X(e ) = x[n] = ±[n] x[n]e n= j 0 =e i.e. ±[n] Dan Ellis 1 X(ejω) x[n] -3 -2 -1 (for all !) X(ej!) x[n] =1 jn 1 2 3 n -º 2013-09-23 º ! 19 DTFTs of simple sequences IDTFT 1 j jn x[n] = e : x[n] = X(e )e d 2 j X(e ) = 2 ( 0 ) over -º < ! < º but X(ej!) must be periodic in ! j 0 n e k 2 ( 0 2k) j 0 n If !0 = 0 then x[n] = 1 ∀ n so Dan Ellis 1 k 2 ( 2k) 2013-09-23 X(ej!) 0 2º 4º 20 DTFTs of simple sequences From before: µ[n] n 1 j 1 e ( |α | < 1) μ[n] tricky - not finite µ[n] 1 + k ( + 2k) j 1 e DTFT of 1/2 Dan Ellis 2013-09-23 21 DTFT properties Linear: j j g[n] + h[n] G(e ) + H (e ) Time shift: g[n n0 ] e Frequency shift: e Dan Ellis j 0 n jn 0 g[n] G (e 2013-09-23 j G(e ) j ( 0 ) ) ‘delay’ in frequency 22 DTFT example x[n] = ±[n] + Æn μ[n-1] Dan Ellis ? = ±[n] + Æ(Æn-1 μ[n-1]) j 1 1 X(e ) = 1+ e j 1 e e j 1 e j + e j = 1+ = j 1 e 1 e j 1 = x[n] = Æn μ[n] j 1 e j 2013-09-23 23 DTFT symmetry If x[n] x[-n] x*[n] Re{x[n]} j X(e ) = x[n]e jn n= X(ej!) then... from summation X(e-j!) X*(e-j!) (e-j!)* = ej! 1 j ! XCS(e ) = [ X (e j ) + X * (e j )] 2 conjugate symmetry cancels Im parts on IDTFT jIm{x[n]} xcs[n] xca[n] Dan Ellis 1 j * j = [ X (e ) X (e )] 2 Re{X(ej!)} jIm{X(ej!)} XCA(ej!) 2013-09-23 24 DTFT of real x[n] When x[n] is pure real, XCS XR(ej!) = XR(e-j!) XI(ej!) = -XI(e-j!) Imag xcs[n] ≡ xev[n] = xev[-n] xca[n] ≡ xod[n] = -xod[-n] X(ej!) = X*(e-j!) x[n] real, even X(ej!) even, real xim[n] Real xre[n] n Dan Ellis 2013-09-23 25 DTFT and convolution Convolution: x[n] = g[n] h[n] X(e ) = n = ( g[n] h[n]) e j jn = n ( k g[k]h[n k])e jn = k ( g[k]e jk n h[n k]e j (n k ) ) j j = G(e ) H (e ) g[n] h[n] G(e j )H (e j ) Dan Ellis 2013-09-23 Convolution becomes multiplication 26 Convolution with DTFT j j Since g[n] h[n] G(e )H (e ) we can calculate a convolution by: g[n] h[n] Dan Ellis finding DTFTs of g, h → G, H multiply them: G·H IDTFT of product is result, g[n] h[n] DTFT DTFT G(e j ) Y (e j ) IDTFT y[n] H (e j ) 2013-09-23 27 M DTFT convolution example 1 x[n] = X(e ) = 1 e j h[n] = ±[n] - Ʊ[n-1] j Æn·μ[n] H (e ) = 1 (e j j 1 ) 1 y[n] = x[n] ∗ h[n] j j j Y (e ) = H (e )X(e ) 1 j = 1 e =1 ) j ( 1 e y[n] = ±[n] i.e. ... Dan Ellis 2013-09-23 28 DTFT modulation Modulation: x[n] = g[n] h[n] Could solve if g[n] was just sinusoids... 1 j j jn jn X(e ) = n G(e )e d h[n]e 2 write g[n] as IDTFT 1 j j ( )n = G(e ) n h[n]e d 2 1 j j ( ) g[n] h[n] G(e )H (e )d 2 [ ] Dual of convolution in time Dan Ellis 2013-09-23 29 Parseval’s relation “Energy” in time and frequency domains are equal: 1 * j * j g[n]h [n] = G(e )H (e )d 2 n If g = h, then g·g* = |g|2 = energy... Dan Ellis 2013-09-23 30 Energy density spectrum Energy of sequence g = g[n] 2 n 1 g = 2 G(e By Parseval Define Energy Density Spectrum (EDS) j j 2 Sgg (e ) = G(e ) Dan Ellis 2013-09-23 j 2 ) d 31 EDS and autocorrelation Autocorrelation of g[n]: rgg [] = g[n]g[n ] = g[n] g[n] n= DTFT {rgg []} = G(e )G(e j j ) If g[n] is real, G(e-j!) = G*(ej!), so j 2 j DTFT {rgg []} = G(e ) = Sgg (e ) no phase info. Mag-sq of spectrum is DTFT of autoco Dan Ellis 2013-09-23 32 3. Discrete FT (DFT) Discrete FT (DFT) Discrete Discrete finite/pdc x[n] finite/pdc X[k] A finite or periodic sequence has only N unique values, x[n] for 0 ≤ n < N Spectrum is completely defined by N distinct frequency samples Divide 0..2º into N equal steps, 2 k { k} = N Dan Ellis 2013-09-23 33 DFT and IDFT Uniform sampling of DTFT spectrum: N 1 X[k] = X(e j ) = 2 k = x[n]e N j 2 k n N n=0 N 1 2º/N 1 kn X[k] = x[n]W DFT: N WN n=0 where WN = e Dan Ellis j 2 N i.e. -1/Nth of a revolution 2013-09-23 34 IDFT N 1 1 nk Inverse DFT: IDFT x[n] = X[k]WN N k=0 Check: 1 kl nk x[n] = k l x[l]WN WN N Sum of complete set N 1 N 1 1 k (ln ) of rotated vectors = x[l] WN = 0 if l ≠ n; = N if l = n N l=0 k=0 im or finite re W = x[n] geometric series ( ) N WN2 0≤n<N Dan Ellis 2013-09-23 = (1-WNlN)/(1-WNl) 35 DFT examples Finite impulse x[n] = N 1 X[k] = 1 0 n=0 n = 1...N x[n]WNkn = WN0 = 1 1 k n=0 Periodic sinusoid: ⇥ 2 rn N x[n] = cos X[k] = (0 ≤ k < N) Dan Ellis 1 2 = (r Z) = 1 W rn + W rn ⇥ N N N 1 rn (W N n=0 2 + WNrn )WNkn k = r, k = N 0 otherwise N 2 2013-09-23 r 36 DFT: Matrix form X[k] = n=0 x[n] W N 1 X[0] X[1] X[2] .. . ⇧ ⇧ ⇧ ⇧ ⇧ ⇤ X[N ⇥ 1 ⌃ ⇧ 1 ⇧ ⌃ ⇧ ⌃ ⇧1 ⌃=⇧ ⌃ ⇧. ⌅ ⇤ .. 1] 1 WN1 WN2 .. . (N 1) 1 WN i.e. Dan Ellis kn N as a matrix multiply: 1 WN2 WN4 .. . ··· ··· ··· .. . 2(N 1) ··· WN 1 ⇥ (N 1) ⌃ WN ⌃⇧ 2(N 1) ⌃ ⇧ WN ⌃⇧ ⌃⇧ .. . (N WN ⌃⇧ ⌅⇤ 1)2 x[N X = DN x 2013-09-23 x[0] x[1] x[2] .. . 37 ⇥ ⌃ ⌃ ⌃ ⌃ ⌃ ⌅ 1] Matrix IDFT If X = DN x 1 then x = DN X i.e. inverse DFT is also just a matrix, DN1 1 ⇧1 ⇧ 1 ⇧ ⇧1 = N⇧ ⇧ .. ⇤. 1 WN 1 WN 2 .. . 1 WN (N 1) 1 WN 2 WN 4 .. . WN 2(N 1) =1/NDN* Dan Ellis 2013-09-23 ··· ··· ··· .. . ··· 1 ⌃ ⌃ 1) ⌃ ⌃ ⌃ ⌃ ⌅ (N 1) WN 2(N WN .. . WN (N 1)2 38 ⇥ DFT and MATLAB MATLAB is concerned with sequences not continuous functions like X(ej!) Instead, we use the DFT to sample X(ej!) on an (arbitrarily-fine) grid: Dan Ellis X = freqz(x,1,w); samples the DTFT of sequence x at angular frequencies in w X = fft(x); calculates the N-point DFT of an N-point sequence x 2013-09-23 39 M DFT and DTFT DTFT x[n]e j X(e ) = jn • continuous freq ! • infinite x[n], -∞<n<∞ n= N 1 X[k] = x[n]W • discrete freq k=N!/2º • finite x[n], 0≤n<N kn N DFT n=0 DFT ‘samples’ DTFT at discrete freqs: X[k] j X[k] = X(e ) = 2 k N Dan Ellis k=1... 2013-09-23 X(ej!) ! 40 DTFT from DFT N-point DFT completely specifies the continuous DTFT of the finite sequence N 1 jn 1 j kn X(e ) = X[k]WN e n=0 N k=0 N 1 j ( 2 k ) n 1 N “periodic = X[k] e sinc” N k=0 n=0 k N 1 sin N 2 j ( N21) k 1 = X[k] e k N k=0 sin 2 N 1 k = 2Nk interpolation Dan Ellis N 1 2013-09-23 41 Periodic sinc jN N 1 1e j n = e j 1e k k k n =0 jN k / 2 jN k / 2 e e e = j k / 2 j k / 2 e e e j k / 2 k ( N 1) j 2 k sin N 2 pure real =e k pure phase sin 2 = N when Δ!k = 0; = (-)N when Δ!k/2 = º = 0 when Δ!k/2 = r·º/N, r = ±1, ± 2, ... other values in-between... factor out half the angle jN k / 2 Dan Ellis 2013-09-23 42 Periodic sinc sin Nx sin x N sinNx/sinx sinNx 0 -N X[k] = X(ej2 (N = 8) k/N) X(ej 0) = X[k]· 1.5 DFT→ DTFT = interpolation by periodic sinc X[k]→X(ej!) Dan Ellis x 2 sinx sin N N sin X[3]· 1 sin N N sin k/2 k/2 3/2 3/2 0.5 0 -0.5 -1 freq 0 k=1 = 2 /N 2013-09-23 0 k=3 = 6 /N k=4 = 8 /N 43 DFT from overlength DTFT If x[n] has more than N points, can still j form X[k] = X(e ) = 2 k N IDFT of X[k] will give N point x̃[n] How does x̃[n] relate to x[n]? Dan Ellis 2013-09-23 44 DFT from overlength DTFT x[n] DTFT -A ≤ n < B X(ejω) 1 x̃[n] = N = sample x[ ]WNk k=0 = = 0≤n<N Dan Ellis x[n r= x˜[n] 0≤n<N N 1 x[ ] x̃[n] = X[k] IDFT 1 N N 1 k( WN WN nk =1 for n-l = rN, r∈I = 0 otherwise n) k=0 all values shifted by rN ] exact multiples of N pts to lie in 0 ≤ n < N 2013-09-23 45 DFT from DTFT example If x[n] = { 8, 5, 4, 3, 2, 2, 1, 1} (8 point) We form X[k] for k = 0, 1, 2, 3 by sampling X(ej!) at ! = 0, º/2, º, 3º/2 IDFT of X[k] gives 4 pt x̃[n] = r= Overlap only for r = -1: x̃[n] = Dan Ellis 8 + 2 5 4 + + 2 2 3 + 1 2013-09-23 = {10 rN ] x[n 7 (N = 4) 5 4} 46 DFT from DTFT example x[n] -1 x[n+N] (r = -1) -5 -4 -3 -2 -1 1 2 3 4 5 6 7 8 n 1 2 3 4 5 x˜[n] 1 2 3 n n x˜[n] is the time aliased or ‘folded down’ version of x[n]. Dan Ellis 2013-09-23 47 Properties: Circular time shift DFT properties mirror DTFT, with twists: Time shift must stay within N-pt ‘window’ g[ n n0 N ] WNkn0 G[k] Modulo-N indexing keeps index between 0 and N-1: g[ n n0 N] = g[n n0 ] g[N + n n0 ] n n0 n < n0 0 ≤ n0 < N Dan Ellis 2013-09-23 48 Circular time shift Points shifted out to the right don’t disappear – they come in from the left g[n] ‘delay’ by 2 n 5-pt sequence 1 2 3 4 g[<n-2>5] 1 2 3 4 n Like a ‘barrel shifter’: origin pointer Dan Ellis 2013-09-23 49 Circular time reversal Time reversal is tricky in ‘modulo-N’ indexing - not reversing the sequence: x˜ [n] 5-pt sequence made periodic -7 -6 -5 -4 -3 -2 -1 1 2 3 4 5 6 7 8 9 10 11 x˜ [ n N ] Time-reversed periodic sequence -7 -6 -5 -4 -3 -2 -1 1 2 3 4 5 6 7 8 9 10 11 Zero point stays fixed; remainder flips Dan Ellis 2013-09-23 n 50 n Duality DFT and IDFT are very similar both map an N-pt vector to an N-pt vector Duality: if then Circular time reversal g[n] G [k ] G [n] N g[ k N ] i.e. if you treat DFT sequence as a time sequence, result is almost symmetric Dan Ellis 2013-09-23 51 M 4. Convolution with the DFT IDTFT of product of DTFTs of two N-pt sequences is their 2N-1 pt convolution IDFT of the product of two N-pt DFTs can only give N points! Equivalent of 2N-1 pt result time aliased: (0 ≤ n < N) i.e. y [n ] = y [n + rN ] c l r= must be, because G[k]H[k] are exact samples of G(ej!)H(ej!) This is known as circular convolution Dan Ellis 2013-09-23 52 Circular convolution Can also do entire convolution with modulo-N indexing Hence, Circular Convolution: N 1 g[m ]h[ n m N ] G[k]H[k] m=0 N h[n] Written as g[n] Dan Ellis 2013-09-23 53 Circular convolution example 4 pt sequences: g[n]={1 2 0 1} h[n]={2 2 1 0} N 1 g[m ]h[ n m N ] 1 2 3 m=0 h[<n - 0>4] h[<n - 1>4] h[<n - 2>4] h[<n - 3>4] Dan Ellis 1 1 2 3 n 2 1 2 3 n 1 2 3 n 1 2 3 n 0 1 2013-09-23 n 1 2 3 n g[n] 4 h[n]={4 7 5 4} 1 2 3 n check: g[n] * h[n] ={2 6 5 4 2 1 0} 54 DFT properties summary Circular convolution N 1 m=0 g[m ]h[ n m Modulation g[n] h[n] Duality Parseval Dan Ellis 1 N N ] G[k]H[k] m=0 G[m]H[ k m N ] N 1 G [n] N g[ k n=0 x[n] N 1 2 = 1 N N ] k=0 X [k ] 2013-09-23 N 1 2 55 Linear convolution w/ the DFT DFT → fast circular convolution .. but we need linear convolution Circular conv. is time-aliased linear conv.; can aliasing be avoided? e.g. convolving L-pt g[n] with M-pt h[n]: y[n] = g[n] * h[n] has L+M-1 nonzero pts Set DFT size N ≥ L+M-1 → no aliasing Dan Ellis 2013-09-23 56 Linear convolution w/ the DFT Procedure (N = L + M - 1): g[n] pad L-pt g[n] with (at least) n M-1 zeros L h[n] → N-pt DFT G[k], k = 0..N-1 pad M-pt h[n] with (at least) n M L-1 zeros yc[n] → N-pt DFT H[k], k = 0..N-1 Y[k] = G[k]·H[k], k = 0..N-1 n N IDFT{Y[k]} = yL [n + rN ] = yL [n] (0 ≤ n < N) r= Dan Ellis 2013-09-23 57 Overlap-Add convolution Very long g[n] → break up into segments, convolve piecewise, overlap → bound size of DFT, processing delay Make gi [n] = h[n] i · N n < (i + 1) · N otherwise g[n] 0 g[n] = g[n] = i gi [n] i h[n] gi [n] Called Overlap-Add (OLA) convolution... Dan Ellis 2013-09-23 58 Overlap-Add convolution g[n] h[n] n g0[n] M n g0[n] * h[n] n g1[n] n g2[n] L 2L N= L+M-1 g1[n] * h[n] n g2[n] * h[n] n 3L h[n] * g[n] valid OLA sum ML Dan Ellis 2013-09-23 n 2L 3L 59 n n