The Fourier Transform
(and more…)
Nimrod Peleg
Nov. 2005
1
Outline
• Introduce Fourier series and transforms
• Introduce Discrete Time Fourier
Transforms, (DTFT)
• Introduce Discrete Fourier Transforms
(DFT)
• Consider operational complexity of DFT
• Deduce a radix-2 FFT algorithm
• Consider some implementation issues of
FFTs with DSPs
2
1
The Frequency Domain
• The frequency domain does not carry any
information that is not in the time domain.
• The power in the frequency domain is that it is simply
another way of looking at signal information.
• Any operation or inspection done in one domain is
equally applicable to the other domain, except that
usually one domain makes a particular operation or
inspection much easier than in the other domain.
• frequency domain information is extremely important
and useful in signal processing.
3
3 Basic Representations for FT
• 1. An Exponential Form
• 2. A Combined Trigonometric Form
• A Simple Trigonometric Form
4
2
The Fourier Series: Exponential Form
Periodic signal expressed as infinite
sum of sinusoids.
Complex Numbers !
x p (t) =
ck =
1
Tp
∞
∑c
k = −∞
∫
k
e jk ω 0 t ,
where
x p ( t )e − jk ω 0 t dt
Tp
z Ck’s are frequency domain amplitude and phase representation
z For the given value xp(t) (a square value), the sum of the first four terms of
trigonometric Fourier series are: xp(t) ≈ 1.0 + sin(t) + C2 sin(3t) + C3sin(5t)
5
The Combined Trigonometric Form
• Periodic signal: xp (t) = xp(t+T) for all t
and cycle time (period) is: T = 1 = 2π
f0
ω0
f0 is the fundamental frequency in Hz
w0 is the fundamental frequency in radians: ω = 2π f
xp(t) can be expressed as an infinite sum of orthogonal functions.
When these functions are the cosine and sine, the sum is called the
Fourier Series. The frequency of each of the sinusoidal functions in
the Fourier series is an integer multiple of the fundamental frequency.
Basic frequency + Harmonies
6
3
Fourier Series Coefficients
jkω t
• Each individual term of the series, Cke
,is the
frequency domain representation and is generally
complex (frequency and phase), but the sum is
real.
• The second common form is the combined
∞
trigonometric form:
0
xp ( t ) = C 0 + ∑ 2 Ck sin( kω 0t + θk )
k =1
θk = tan −1
Im( Ck )
Re( Ck )
Again: Ck are Complex Numbers !
7
The Trigonometric Form
Reminder:
∞
xp (t )=A0 + ∑ AkCos (kω 0t )+BkSin(kω 0t )
k =1
1
A0 = ∫ xp (t )dt = DC = Average value of xp (t )
Tp Tp
Ak = ∫ xp (t )Cos (kω 0t )dt
Tp
Bk = ∫ xp (t )Sin(kω 0t )dt
Tp
e jθ + e-jθ
Cosθ =
2
e jθ -e-jθ
Sinθ =
2j
All three forms are identical and are related using Euler’s identity:
e ± jθ = Cosθ ± jSinθ
Thus, the coefficients of the different forms are related by:
2Ck = Ak − jBk ; C 0 = A0
Im( Ck )
Bk
θk = tan −1
= tan −1
Re( Ck )
Ak
8
4
The Fourier Transform
1/3
The Fourier series is only valid for periodic signals.
For non-periodic signals, the Fourier transform is used.
Most natural signals are not periodic (speech).
We treat it as a periodic waveform with an infinite period.
If we assume that TP tends towards infinity, then we can produce
equations (“model”) for non-periodic signals.
If Tp tends towards infinity, then w0 tends towards 0.
Because of this, we can replace w0 with dw, and it leads us to:
1
Tp
= fp = ω
2π
1 dω
=
t →∞ Tp
2π
lim
The Fourier Transform
C (ω ) =
9
2/3
dω ∞
x ( t ) e − jωt dt
2π −∞
z
1
ω
dω
=
⇒
Tp 2π
2π
z Increase TP = Period Increases : No Repetition:
z Discrete frequency variable becomes continuous:
z Discrete coefficients Ck become continuous:
kω 0 ⇒ ω
Ck ⇒ C ( ω )
∞
C (ω )
2π
= ∫ x (t )e − jωt dt
dω −∞
10
5
The Fourier Transform
2π
We define:
X (ω ) =
zx
C (ω )
X (ω ) = F{ x(t )}
dω
∞
−∞
3/3
(t )
e
− jωt
1
dt ⇔ x ( t ) =
2π
zx
∞
(ω )
e jωt dω
−∞
11
Signal Representation by Delta Function
Instead of a continuous signal we have a “collection of samples”:
This is equivalent to sampling the signal with one Delta
Function each time, moving it along X-axis, and summing
all the results:
∞
xs ( t ) = ∑ x ( t )δ ( t − nTs )
−∞
Note that the Delta is “1” only
If its index is zero !
12
6
Discrete Time Fourier Transform
1/3
• Consider a sampled version, xs(t) , of a continuous
∞
signal, x(t) :
xs ( t ) = ∑ x ( t )δ ( t − nTs )
−∞
Ts is the sample period. We wish to take the Fourier transform of this
sampled signal. Using the definition of Fourier transform of xs(t)
and some mathematical properties of it we get:
∞
z Replace continuous time t with (nTs)
z Continuous x(t) becomes discrete x(n)
z Sum rather than integrate all discrete samples
xs(ω ) = ∑ x ( nTs) e − jωnTs
n = −∞
13
Discrete Time Fourier Transform
Fourier
Transform x(ω ) =
∞
∫
ωt
x(t )e − j dt
∞
∑ x(n)e
x( Ω ) =
− jΩn
n = −∞
−∞
Inverse
1 ∞
x(ω)e j dω
Fourier x( t ) =
2π −∫∞
Transform
ωt
π
2/3
Discrete Time
Fourier Transform
( nΩ )
1
x(Ω )e j dΩ
x(n) =
2π −∫π
Inverse Discrete
Time Fourier
Transform
z Limits of integration need not go beyond ±π because the spectrum
repeats itself outside ±π (every 2π): X ( Ω ) = X ( Ω + 2 π )
z Keep integration because X ( Ω ) is continuous: Ω = ωTs
means that Ω is periodic every Ts
14
7
Discrete Time Fourier Transform
3/3
• Now we have a transform from the time
domain to the frequency domain that is
discrete, but ...
DTFT is not applicable to DSP because it
requires an infinite number of samples and
the frequency domain representation is a
continuous function – impossible to represent
exactly in digital hardware.
15
1st result: Nyquist Sampling Rate
1/2
• The Spectrum of a sampled signal is
periodic, with 2*Pi Period: X (Ω) = X (Ω + 2π )
Easy to see:
X (Ω + 2π ) =
∞
∑
x(n)e − jn (Ω+ 2π ) =
n =−∞
=
∞
∑ x ( n )e
− jnΩ
∞
∑ x ( n )e
− jnΩ − j 2π n
e
n =−∞
= X (Ω )
n =−∞
e− j 2π n = cos(2π n) − j sin(2π n) = 1
16
8
1st result: Nyquist Sampling Rate
2/2
• For maximum frequency wH :
Ω|ω = ω =π
Ω = ωTs
H
π = ω HTs
π
Ts =
ωH
⇒ ωs =
BUT : Ts =
2π
=2ω H
Ts
π
ωs
17
Nyquist Conclusion…
18
9
Practical DTFT
z Take only N time domain samples
x (Ω ) =
∞
∑ x (n)e − j
N −1
x (Ω ) = ∑ x ( n )e − j
Ωn
n = −∞
z Sample
Ωn
n=0
the frequency domain, i.e. only evaluate x(Ω) at N
discrete points. The equal spacing between points is ∆Ω = 2π/N
N −1
2 πkn / N
2πk
x(
) = ∑ x ( n) e − j
N
n=0
k = 0,1,2,..., N − 1
19
The DFT
Since the only variable in 2πk / N is k , the DTFT is written:
N −1
x ( k ) = ∑ x ( n) e − j
2 πkn / N
k = 0,1,2,..., N − 1
n=0
− j 2π / N
Using the shorthand notation: WN = e
(Twiddle Factor)
The result is called Discrete Fourier Transform (DFT):
N −1
X N ( k ) = ∑ x (n)W Nkn
n=0
and
x ( n) =
1 N −1
∑ X N ( k )W N− kn
N k =0
20
10
Usage of DFT
• The DFT pair allows us to move between
the time and frequency domains while using
the DSP.
• The time domain sequence x[n] is discrete
and has spacing Ts, while the frequency
domain sequence X[k] is discrete and has
spacing 1/NT [Hz].
21
DFT Relationships
Time Domain
Frequency Domain
|x(k)|
N Samples
N Samples
≈
≈
0
0
Ts
1
2Ts 3Ts (N-1)Ts
2
3
N-1
≈
X(n)
t
n
0
1
2
N/2
0
Fs
N
2Fs
N
Fs
2
N-2 N-1
k
− 2Fs
N
f
− Fs
N
22
11
Practical Considerations
N −1
∑x
X N (k ) =
z Standard DFT:
( k )W Nkn
0 ≤ k ≥ N −1
N
( k )W 7kn
k = 0,1, 2,..., 7
n=0
z An example of an 8 point DFT:
X N (k ) =
n
7
∑x
n=0
z Writing this out for each value of n :
X n (k ) = x(0 )W7k 0 + x(1)W7k1 + ....... + x(7 )W7k 7 , k = 0,1,...,7
k0
z Each term such as x (0) W7
requires 8 multiplications
z Total number of (complex !) multiplications required: 8 * 8 = 64
z 1000-point DFT requres 10002 = 106 complex multiplications
And all of these need to be summed….
23
Fast Fourier Transform
Symmetry Property
WNk +N / 2 = − WNk
Periodicity Property
WNk +N = WNk
N
−1
2
XN (k) = ∑ x(2r ).W
Splitting the DFT in two
(odd and even)
2rk
N
r =0
or
N
−1
2
+∑ x(2r + 1).WN(2r +1)k
r =0
N
−1
2
N
−1
2
r=0
r=0
XN (k ) = ∑ x(2r ).( WN2 )rk + WNk ∑ x(2r + 1).( WN2 )rk
Manipulating the twiddle factor
WN2 = e
j( −
2π
2)
N
=e
j( −
2π
)
N/ 2
= WN / 2
THE FAST FOURIER TRANSFORM
X n (k ) =
N
−1
2
∑
r =0
N
−1
2
x ( 2r ) W Nrk 2 + W Nk ∑ x ( 2r + 1) W Nrk 2
r =0
24
12
FFT complexity
xN (k ) =
N −1
2
∑ x(2 r )W
rk
N /2
+W
k
N
N −1
2
r =0
∑ x(2 r + 1)W
rk
N/2
r =0
(N/2)2 multiplications
(N/2)2 multiplications
N/2 Multiplications
z For an 8-point FFT, 42 + 42 + 4 = 36 multiplications, saving 64 - 36 = 28
multiplications
z For 1000 point FFT, 5002 + 5002 + 500 = 50,500 multiplications, saving
1,000,000 - 50,500 = 945,000 multiplications
25
Time Decimation
z Splitting the original series into two is called decimation in time
z Let us take a short series where N = 8
z Decimate once Called Radix-2 since we divided by 2
n = {0, 1, 2, 3, 4, 5, 6, 7}n = { 0, 2, 4, 6 } and { 1, 3, 5, 7 }
z Decimate again
n = { 0, 4 } { 2, 6 } { 1, 5 } and { 3, 7 }
z The result is a savings of N2 – (N/2)log2N multiplications:
1024 point DFT = 1,048,576 multiplications
1024 point FFT = 5120 multiplication
z Decimation simplifies mathematics but there are more twiddle
factors to calculate, and a practical FFT incorporates these extra factors
into the algorithm
26
13
Simple example: 4-Point FFT
z Let us consider an example where N=4:
3
X 4( k ) = ∑ x (n)W4kn
0
z Decimate in time into 2 series:
n = {0 , 2} and {1, 3}
1
1
X 4( k ) = ∑ x (2r )W2rk + W4k ∑ x (2r + 1)W2rk
r =0
r =0
= {x (0) + x (2)W } + W {x (1) + x (3)W2k }
k
2
z We have two twiddle factors.
Can we relate them?
WNk = e
W2k = e
−j
2π
k
N
−j
2π
k
2
k
4
=e
−j
2π
2k
4
= W22 k
z Now our FFT becomes:
X 4( k ) = {x (0) + x (2)W42 k } + W4k {x (1) + x (3)W42 k }
27
4-Point FFT Flow Diagram
The 2 DFT’s:
for k=0,1,2,3
X 4( k ) = {x (0) + x (2)W42 k } + W4k {x (1) + x (3)W42 k }
X 4( 0) = {x (0) + x (2)W40} + W40 {x (1) + x ( 3)W40}
For k=0 only:
A ‘flow-diagram’ of it:
x(0)
x(2)
W40
x(1)
W4
+
W40
+
0
x(3)
+
This is for only 1/4 of the
whole diagram !
28
14
A Complete Diagram
X 4( 0) = {x (0) + x (2)W40} + W40 {x (1) + x ( 3)W40}
X 4(1) = {x (0) + x (2)W42 } + W41{x (1) + x (3)W42 }
X 4( 2) = {x (0) + x (2)W40 } + W42 {x (1) + x (3)W40 }
X 4( 3) = {x (0) + x (2)W42 } + W43 {x (1) + x ( 3)W42 }
X4(0)
x(0)
0
x(2)
WNk = e
−j
2π
k
N
0
2
1
2
x(1)
0
3
2
x(3)
Note:
X4(1)
4
4
W =e
X4(2)
−j
2π
4
4
−j
2π
6
4
W46 = e
X4(3)
= 1 = W40
= −1 = W42
29
The Butterfly
Twiddle Conversions
A Typical Butterfly
x1
0
W4 = 1
k
X1
X1 = x1 + WN x2
X2
X2 = x1 – WN x2
1
W4 = -j
2
x2
k
WN
W4 = -1
k
3
W4 = j
4 Point FFT Equations
0
X0 = (x0 + x2) + W4 (x1+x3)
4 Point FFT Butterfly
x0
X0
1
X1 = (x0 – x2) + W4 (x1–x3)
0
x2
1
x1
X2 = (x0 + x2) – W4 (x1+x3)
X3 = (x0 – x2) – W4 (x1–x3)
X1
0
W4
X2
1
x3
W4
X3
30
15
Summary
z Frequency domain information for a signal is important for processing
z Sinusoids can be represented by phasors
z Fourier series can be used to represent any periodic signal
z Fourier transforms are used to transform signals
z From time to frequency domain
z From frequency to time domain
z DFT allows transform operations on sampled signals
z DFT computations can be sped up by splitting the original series into
two or more series
z FFT offers considerable savings in computation time
z DSPs can implement FFT efficiently
31
Bit-Reversal
• If we look at the inputs to the butterfly FFT, we can see
that the inputs are not in the same order as the output.
• To perform an FFT quickly, we need a method of
shuffling these input data addresses around to the
correct order.
• This can be done either by reversing the order of the
bits that make up the address of the data, or by pointer
manipulation (bit reversed addition).
• Many DSPs have special addressing modes that allow
them to automatically shuffle the data in the
background.
32
16
Bit-Reversal example
X4(0)
x(0)
0
x(2)
0
2
1
x(1)
2
X4(2)
3
X4(3)
0
x(3)
2
X4(1)
• To obtain the output
in ascending order
the input values must
be loaded in the
order: {0,2,1,3}
• for 512 or 1024 it is
much more
complicated...
33
8-point Bit-Reversal
• Consider a 3-bit address (8 possible locations).
• After starting at zero, we add half of the FFT
length at each address access with carrying from
left to right (!)
Start at 0 = 000
000+100 = 100
100+100 = 010
010+100 = 110
110+100 = 001
001+100 = 101
101+100 = 011
011+100 = 111
=x(0)
=x(4)
=x(2)
=x(6)
=x(1)
=x(5)
=x(3)
=x(7)
Note that reversing the order
of the address bits gives same
result !
34
17
And what about DCT ???
 π (n + 12 )k 
b(k)
x
(
n
)
cos
∑


N
n =0
N −1
 π ( n + 12 )k 
II
b(k)X
(k)
cos
∑


N
k =0
N −1
II
X (k)=
2
N
DCT Type II*:
x(n) =
2
N
if k=0
1
b(k)=  2
 1 if k=1,...,L-1
The rest of the math is quite similar…..
Note: at least 4 types of DCT !!!
* after: A course in Digital Signal Processing, Boaz Porat
DCT Type II
Type I
Type II
35
Most used for compression:
JPEG, MPEG etc.
Type III
DCT Basis Vectors for N=8
Type IV
36
18
DCT Features
•
•
•
•
•
Real transformation
Reversible transformation
2D Transformation exists and separable
Better than the DFT as a “de-correlator”
Fast algorithm exists (NlogN Complexity)
37
Aliasing, Transforms and More…
• Nyquist's Sampling Theorem is explained here:
– http://www.cs.cf.ac.uk/Dave/Multimedia/node149.html
• The effect of the above and aliasing is also shown here:
– http://www.efunda.com/designstandards/sensors/methods/dsp_
nyquist.cfm
• And examples to aliasing in images is shown here:
– http://www.cogs.susx.ac.uk/users/ianw/teach/ms/node19.html
• Java examples of various signal processing:
–
http://www.jhu.edu/~signals/
38
19