The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
Fourier transform of real discrete data
Notes
Today we will discuss how to apply Fourier transform to real data,
which is always sampled at discrete times and is finite in duration.
Note: Fourier analysis also used in image processing. Images have
finite resolution. Instead of discrete time steps, one has discrete
points in space (pixels).
Real data is sampled in finite time steps ∆t: y (t) → y (m∆t),
where m = 0, 1, 2, . . . N − 1 and N is the number of points.
Only consider equally spaced time steps.
Data is of finite duration: T = N∆t
Notation:
Time steps tm ≡ m∆t
Function/Signal at discrete times: y (m∆t) = y (tm ) ≡ ym
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
How to discretize the Fourier transform
Notes
Since our data is finite, cannot evaluate the Fourier integral below
Z ∞
1
g (ω) = √
f (t)e −iωt dt
(1)
2π −∞
Task: Compute a measure from finite data sets sampled on
discrete time steps that resembles/approximates the Fourier
transform.
In the following we will develop the discrete Fourier transform.
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
Developing the discrete Fourier transform
Notes
Let’s go back to the complex Fourier series for periodic functions
with period T . Consider the Fourier representation of y (t) on an
interval from 0 to T . (Before, in lecture 3 we considered −T /2 to
T /2)
∞
X
y (t) =
cn e inωt , where
n=−∞
cn =
1
T
Z
T
y (t)e −i2πnt/T dt =
0
1
T
T
Z
y (t)e −in∆ωt dt,
0
where the discrete frequencies of the Fourier series representation
are
2π
n∆ω ≡ n
T
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
Developing the discrete Fourier transform
approximateP
the integral for cn by a summation:
RLet’s
T
N−1
i=0 f (i∆x).
0 f (x)dx ≈ ∆x
cn =
1
T
Z
T
y (t)e −in∆ωt dt ≈
0
N−1
∆t X
y (m∆t)e −in∆ωm∆t
T
m=0
≈
N−1
1 X
y (m∆t)e −i2πnm/N
N
m=0
|
{z
}
Y (n∆ω)
where we used ∆t = T /N and ∆ω =
out in the exponential.
2π
T .
Note that ∆t cancels
Notes
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
The discrete Fourier transform
Notes
We define the discrete Fourier transform (DFT) as:
Y (n∆ω) =
N−1
X
y (m∆t)e −i2πnm/N ,
m=0
This can be written more concisely as
Yn =
N−1
X
ym e −i2πnm/N ,
(2)
m=0
since y (m∆t) = y (tm ) = ym .
Still need the inverse discrete transform. Want it to be exact, such
that we recover the yn ’s from the Ym ’s. Next we will find the
inverse transform by using orthogonality.
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
The inverse discrete Fourier transform
Consider the sum
SN =
N−1
X
Notes
e ikα
k=0
If α = 0 then SN = N, since every term is 1. The sum above is
just a geometric series:
N−1
X
r k = r 0 + r 1 + . . . + r N−1 =
k=0
1 − rN
1−r
Therefore,
SN =
N−1
X
e ikα =
k=0
1 − e iαN
1 − e iα
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
The inverse discrete Fourier transform
Notes
Want SN to be zero for α 6= 0. This means
e iαN = 1
so
α = 2πl/N, where l is integer
Therefore,
SN =
N−1
X
e i2πkl/N =
k=0
N, l = 0
0, l =
6 0
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
The inverse discrete Fourier transform
Notes
Now, let’s express integer l as difference between two integers m
and n (l = m − n) to obtain orthogonality relation:
N−1
X
e i2πkm/N e −i2πkn/N = Nδm,n
k=0
Let’s return to the discrete Fourier transform (eqn. (2))
Yn =
N−1
X
ym e −i2πnm/N ,
m=0
and multiply both sides with e i2πkn/N and sum over n.
(3)
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
The inverse discrete Fourier transform
N−1
X
Yn e i2πkn/N
=
n=0
=
=
N−1
X N−1
X
ym e
n=0 m=0
N−1
N−1
X
X
−i2πnm/N i2πkn/N
e
e −i2πnm/N e i2πkn/N
ym
m=0
N−1
X
Notes
n=0
ym Nδk,m
m=0
= Nyk ,
where we used the orthogonality of the sums (eqn. (3))
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
The inverse discrete Fourier transform
Notes
Therefore, we can identify the inverse discrete Fourier transform
Yn =
ym =
N−1
X
ym e −i2πnm/N
m=0
N−1
X
1
N
Yn e i2πnm/N
,
DFT
(4)
,
inverse DFT
(5)
n=0
Similar to Fourier transform, prefactors and sign convention in the
exponential can vary in different books and code libraries. The
product of the prefactors always has to be 1/N.
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
Comments on the DFT
Notes
Discrete Fourier transform can be thought of as a truncated
complex Fourier series. This means higher frequency
components are missing. This is important when dealing with
real signals (more later) → How well does the DFT
approximate the Fourier transform ?
Even though it arose from an approximation of the Fourier
series, mathematically it is well defined (orthogonality of the
sums).
Let’s look at the computational implementation
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
Fast Fourier Transform
To transform N data points, need to compute N summations
over order N points. Therefore, computation time goes as N 2 .
For higher dimensions d, it goes as N 2d .
The fast Fourier transform (Cooley and Tukey 1965), can
reduce the computational effort dramatically: N 2 → Nlog2 N.
e.g. for N = 216 = 65536 need ≈ 106 instead of ≈ 4 · 109
evaluations.
The FFT is the most important and widely used algorithm!
Notes
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
The Fast Fourier Transform
Notes
Algorithm is based on regrouping terms in the summation. Let’s
assume, that N is a power of 2. Now consider the discrete
transform and decompose into even and odd terms:
g (n∆ω) =
=
N−1
X
f (m∆t)e −i2πmn/N
m=0
N−2
X
N−1
X
f (m∆t)e −i2πmn/N +
m=0,even
f (m∆t)e −i2πmn/N
m=1,odd
N/2−1
=
X
f (2j∆t)e −i2π(2j)n/N
j=0
N/2−1
+
X
f ((2j + 1)∆t)e −i2π(2j+1)n/N
j=0
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
The Fast Fourier Transform
Notes
The two sums can be rewritten as two DFT’s with N/2 points that
run over even and odd numbered points, respectively.
N/2−1
g (n∆ω) =
X
f (2j∆t)e −i2πjn/(N/2)
j=0
N/2−1
+e −i2πn/N
X
f ((2j + 1)∆t)e −i2πjn/(N/2)
j=0
= geven (n∆ω) +e −i2πn/N godd (n∆ω)
|
{z
}
| {z }
N/2 points
N/2 points
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
The Fast Fourier Transform
Notes
Instead of N 2 operations, we now have 2(N/2)2 operations.
Continue decomposition into even and odd points. If N = 2k , then
after k steps, there will be N transforms each containing just one
point. Therefore, need N · k = N log2 N operations.
FFT
The FFT requires N = 2k and the computing time goes as
N log2 N
Actual implementation a bit more tricky (see Giordano, Nakanishi
Appendix C3).
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
Properties of DFT - Periodicity
Notes
The DFT is periodic in time and frequency space. Shift indices by
N:
Yn+N
=
N−1
X
ym e −i2π(n+N)m/N =
m=0
ym+N
=
N−1
X
m=0
ym e −i2πnm/N e| −i2πm
{z } = Yn
=1
N−1
N−1
1 X
1 X
Yn e i2π(m+N)n/N =
Yn e i2πnm/N e| i2πn
{z } = ym
N
N
n=0
n=0
=1
If one continues summation beyond (N − 1), then Yn and ym
repeat.
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
Properties of DFT - Parseval’s theorem
Notes
Using the orthogonality of the sums (eqn. (3)) one can show
N−1
X
∗
xm ym
=
m=0
N−1
1 X
Xn Yn∗
N
(6)
n=0
analogous to what we had for Fourier transform. If x = y , then we
recover Parseval’s theorem:
N−1
X
|xm |2 =
m=0
N−1
1 X
|Xn |2
N
(7)
n=0
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
Properties of DFT - Convolution
Notes
Using the orthogonality of the sums (eqn. (3)) one can show the
Convolution theorem for the DFT:
(x ⊗y )n =
N−1
X
∞
X
xl
!
yn−l−pN
=
p=−∞
l=0
|
{z
}
periodic extension of yn−l
N−1
1 X
Xk Yk e i2πkn/N (8)
N
k=0
|
{z
}
inverse DFT of X ·Y
where p is an integer.
Note that Convolution needs N 2 operations. By performing it in
frequency domain using FFT it is much faster! However, need to
be careful since convolution “wraps around” due to periodicity.
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
Properties of DFT - Correlation
Notes
Using the orthogonality of the sums (eqn. (3)) one can show the
Correlation theorem for the DFT (Wiener-Khinchin):
!
N−1
∞
N−1
X
X
1 X ∗
(x y )n =
xl∗
yn+l−pN
=
Xk Yk e i2πkn/N
N
p=−∞
l=0
k=0
|
{z
}
|
{z
}
periodic extension of yn+l
inverse DFT of X ∗ ·Y
(9)
Again, get performance increase by doing correlation in frequency
space using FFT, especially for higher dimensions.
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
Properties of DFT - Shifting
Notes
Analogous to the Fourier transform, shifting the time axis or
frequency axis by l gives rise to an additional phase factor:
DFT [ym−l ]k
DFT [ym e
i2πml/N
]k
= Yk e −i2πkl/N
= Yk−l
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
What about the frequency cutoff?
Notes
Since we truncated the complex Fourier series to arrive at the
DFT, the DFT has a frequency limit!
For N data points, the discrete frequencies are ωn =
n
fn = N∆t
, where n = 0 . . . (N − 1).
n2π
T
=
n2π
N∆t
or
However, fN−1 = N−1
N∆t is NOT the highest frequency that the DFT
can resolve!!
To illustrate that consider the symmetry of the DFT.
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
DFT for a real data points
Notes
The DFT is given by
Yn =
N−1
X
ym e −i2πnm/N
m=0
∗ = y ). Then we can write
Let’s assume that the ym ’s are real (ym
m
YN/2−n =
∗
YN/2+n
=
N−1
X
m=0
N−1
X
ym e −i2πm(N/2−n)/N =
ym e i2πm(N/2+n)/N =
m=0
N−1
X
ym e −iπm e i2πmn/N
m=0
N−1
X
ym e iπm e i2πmn/N
m=0
∗
but e −iπm = e iπm = cos(πm). Therefore, YN/2−n = YN/2+n
.
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
Nyquist frequency
Notes
For a real signal, half of the points are redundant because of the
1
1
symmetry around fN/2 = N2 N∆t
= 2∆t
. It’s enough to compute Yn
from n = 0 . . . (N/2 − 1) for even N.
The highest independent frequency is therefore:
Nyquist frequency
fNyquist
N/2 − 1
1
≈
N∆t
2∆t
This is called the Nyquist frequency.
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
Nyquist frequency and sampling theorem
The frequencies f ≥ fNyquist are actually the negative frequencies!
(Remember that a simple sin/cos function was represented by a
one positive and one negative frequency component in the Fourier
transform/complex Fourier series. It is the same for the DFT).
Shannon-Nyquist sampling theorem
A function f (t) that contains frequencies less than fmax is
completely determined if it is sampled at points that are
1
∆t = 2fmax
apart.
Notes
The discrete Fourier Transform (DFT)
The fast Fourier transform - FFT
Nyquist frequency and sampling theorem
Notes
Example: Human ear can hear up to 20, 000 Hz. In order to
capture the whole audio spectrum, need a sampling rate that is at
least twice as large: 2 · 20, 000 = 40, 000 Hz. CD’s use 44, 100 Hz,
for example.
Notes
Notes
Notes