Fourier Series and the
Fast Fourier Transform
Anna-Karin Tornberg
Mathematical Models, Analysis and Simulation
Fall semester, 2011
Fourier series of a periodic function
Section 4.1 in Strang.
- Consider a periodic function, with periodic length 2�, i.e.
f (x + 2�) = f (x).
- The full Fourier series of f (x) on the interval −� < x < � is defined
as
∞ �
�
1
nπx
nπx �
f (x) = a0 +
an cos
+ bn sin
2
�
�
n=1
- Can use orthogonality properties of the sin and cos functions to
obtain explicit expressions for the coefficients.
Orthogonality properties
The following holds:
�
�
nπx
mπx
sin
dx = 0,
�
�
−�

� �

nπx
mπx
cos
cos
dx =

�
�
−�
�
� �
nπx
mπx
sin
sin
dx =
�
�
−�
cos
for all integers n, m,
0,
�,
2�,
0,
�,
if n �= m,
if n = m > 0,
if n = m = 0.
if n �= m,
if n = m > 0.
Verify these properties! Useful trigonometric identities:
sin nx sin kx = (cos(n − k)x − cos(n + k)x)/2,
cos nx cos kx = (cos(n − k)x + cos(n + k)x)/2,
sin nx cos kx = (sin(n − k)x + sin(n + k)x)/2,
sin2 α = (1 − cos(2α))/2, cos2 α = (1 + cos(2α))/2.
The coefficients
∞ �
�
1
nπx
nπx �
f (x) = a0 +
an cos
+ bn sin
2
�
�
n=1
Multiply through by cos kπx
� and integrate. Use orthogonality properties.
(Different for k = 0 and k > 0). Yields expression for coefficients
1
ak =
�
�
�
f (x) cos kx dx,
k = 0, 1, 2, . . .
−�
Multiply through by sin kπx
� and integrate. This yields:
1
bk =
�
�
�
f (x) sin kx dx,
−�
k = 1, 2, . . .
Fouries Series: The Complex Version
- Moivre’s theorem: e iα = cos α + i sin α.
- Define ck = (ak − ibk )/2, c−k = (ak + ibk )/2.
- Then:
ck e ikx + c−k e −ikx = ck (cos kx + i sin kx) + c−k (cos kx − i sin kx)
= (ck + c−k ) cos kx + i(ck − c−k ) sin kx
= ak cos kx + bk sin kx.
- The Fourier series can be equivalently written as
f (x) =
∞
�
cn e inx .
n=−∞
Odd and even functions
- The function f is odd if f (x) = −f (−x).
- The function f is even if f (x) = f (−x).
- If f is even, then f � is odd. Similarly, if f is odd, then f � is even.
If f is odd, then
1
ak =
�
�
�
−�
f (x) cos kx dx = 0 ∀k.
and f (x) is represented by a Fourier sine series on (−�, �).
Similarly, if f is even
1
bk =
�
�
�
−�
f (x) sin kx dx = 0 ∀k.
and f (x) is represented by a Fourier cosine series on (−�, �).
Diffusion equation - separation of variables.
Consider the diffusion equation: ut = kuxx
with initial conditions u(x, 0) = g (x).
Assuming u on the form u(x, t) = T (t)X (x) yields
T�
X”
=
= −λ = const
kT
X
The solutions to
X ” = −λX ,
have the form,
X (x) = C cos
√
in 0 < x < 1
λx + D sin
√
λx,
and the solution to T � = −λT is T (t) = e −λkt .
Diffusion equation - separation of variables, contd
X (x) = C cos
√
λx + D sin
√
λx,
With homogeneous Dirichlet boundary conditions, u(0, t) = u(1, t) = 0
and hence X (0) = X (1) = 0,
λn = (nπ)2 ,
Xn (x) = bn sin nπx,
(n = 1, 2, 3, . . .).
With homogeneous Neumann boundary conditions,
ux (0, t) = ux (1, t) = 0 and hence X � (0) = X � (1) = 0,
λn = (nπ)2 ,
Xn (x) = an cos nπx,
(n = 0, 1, 2, 3, . . .).
With periodic boundary conditions, u(0, t) = u(1, t), ux (0, t) = ux (1, t)
and hence X (0) = X (1), X � (0) = X � (1),
λn = (2nπ)2 ,
with b0 = 0.
Xn (x) = an cos 2nπx + bn sin 2nπx,
(n = 0, 1, 2, 3, . . .).
Symmetric boundary conditions
Consider the eigenvalue problem
X ” + λX = 0
in (a, b)
Any pair of boundary conditions
α1 X (a) + β1 X (b) + γ1 X � (a) + δ1 X � (b) = 0
α2 X (a) + β2 X (b) + γ2 X � (a) + δ2 X � (b) = 0
are called symmetric if
x=b
f � (x)g (x) − f (x)g � (x)|x=a = 0,
for any pair of functions f (x) and g (x) both of which satisfy the pair of
boundary conditions.
The Dirichlet, Neumann and periodic BC considered on the previous slide
are the most common examples of symmetric boundary conditions.
Orthogonality and General Fourier series
Consider the eigenvalue problem
X ” + λX = 0
in (a, b) with any symmetric BC
Then, all eigenvalues are real, and eigenfunctions corresponding to
different eigenvalues are orthogonal. If λn is an eigenvalue with
multiplicity k, its k linearly independent eigenfunctions can be rechosen
to be orthogonal and real.
I.e. if X1 and X2 are any two eigenfunctions (X1 �= X2 ), then
� b
(X1 , X2 ) =
X1 (x)X2 (x)dx = 0.
a
For any function f (x) on (a, b) its Fourier series is
�
An Xn (x),
n
where the Fourier coefficients are defined as
An =
(f , Xn )
.
(Xn , Xn )
Notions of convergence
�∞
Definition. We say that an infinite series n=1 fn (x) converges to
f (x) pointwise in (a, b) if it converges to f (x) for each a < x < b.
That is, for each a < x < b we have
�
�
∞
�
�
�
�
�
fn (x)� → 0 as N → ∞
�f (x) −
�
�
n=1
Definition. We say that the series converges uniformly to f (x) in
[a, b] if
�
�
∞
�
�
�
�
�
max �f (x) −
fn (x)� → 0 as N → ∞
�
a≤x≤b �
n=1
Definition. We say that the series converges in the mean-square (or
L2 ) sense to f (x) in (a, b) if
�
b
a
�
�2
∞
�
�
�
�
�
f
(x)
−
f
(x)
�
� dx → 0
n
�
�
n=1
as N → ∞
Convergence theorems
Theorem: Uniform
� convergence.
The Fourier series n An Xn (x) converges to f (x) uniformly on [a, b]
provided that
i) f (x), f � (x) and f ”(x) exist and are continuous for a ≤ x ≤ b, and
ii) f (x) satisfies the given boundary conditions.
For the classical Fourier series (sine, cosine or full), it is not required
that f ”(x) exist.
Theorem: Pointwise convergence.
The classical Fourier series (sine, cosine or full) converges to f (x)
pointwise on (a, b), provided that f (x) is a continuous function on
a ≤ x ≤ b and f � (x) is piecewise continuous on a ≤ x ≤ b.
Theorem: L2 convergence.
The Fourier series converges to f (x) is the mean-square sense in
(a, b) provided only that f (x) is any function for which
�
b
a
|f (x)|2 dx
is finite.
Piecewise continuous functions
- A function f (x) has a jump discontinuity at a point x = c if the
one-sided limits f (c+) and f (c−) exist but are not equal. (It does
not matter whether f (c) is defined or not).
- A function f (x) is called piecewise continuous on an interval [a, b] if
it is continuous at all but a finite number of points and has jump
discontinuities at those points.
Theorem: Pointwise convergence.
If f (x) itself is only piecewise continuous on a ≤ x ≤ b and f � (x) is
also piecewise continuous on a ≤ x ≤ b, then the classical Fourier
series (sine, cosine or full) converges at every point x
(−∞ < x < ∞). The sum is
�
An Xn (x) =
n
1
[f (x+) + f (x−)]
2
for all a < x < b.
Examples
- Square wave:


−1,
SW (x) = 1,


0,
if x ∈ (−π, 0),
if x ∈ (0, π),
if x = −π, 0, π.
Fourier sine series:
�
�
4 sin x
sin 3x
sin 5x
sin 7x
SW (x) =
+
+
+
+ ···
π
1
3
5
7
Partial sums overshoot near jumps: Gibbs phenomenon.
- Repeating Ramp: RR is obtained by integrating SW :
RR(x) = |x|.
Fourier cosine series:
�
�
π π cos x
cos 3x
cos 5x
cos 7x
RR(x) = −
+
+
+
+ ···
2
4
12
32
52
72
Note: The coefficients are equal to those obtained by term-wise
integration of the sine series for SW .
The Gibbs phenomenon
- The Gibbs phenomenon is the overshoot that occurs when we
approximate a jump discontinuity - a step function - with a
truncated Fourier series. We denote this by a partial sum, SN ,
including N terms.
- The oscillations move closer and closer to the jump as we include
more terms in the partial sum (N increases), but they do no
disappear.
- Gibbs showed that the amplitude of the overshoot approaches a
value of 9% of the size of the jump discontinuity.
- The width of the overshoot goes to 0 as N → ∞, while the extra
height remains at 9% (top and bottom).
Again, the complex Fourier series
- Let φk (x) = exp(ikx) for k = . . . , −2, −1, 0, 1, 2, . . . .
- Every f ∈ L2 (−π, π) (complex!) has a representation
f (x) =
+∞
�
fˆk e ikx ,
k=−∞
1
1
with fˆk =
(f , φk ) =
2
�f �
2π
�
π
f (x)e −ikx dx
−π
in the sense of L2 (−π, π).
- Orthogonality in L2 (−π, π):
� π
�
ikx −ijx
(φk , φj ) =
e e
dx =
−π
π
e i(k−j)x dx = 2πδkj
−π
- The regularity of f determines what type of convergence we can
expect.
- ”Regularity” also include regularity across periodic boundary.
Decay of Coefficients
- Parseval’s identity:
� π
−π
2
2
|f (x)| dx = �f � = 2π
Consequently, f ∈ L2 (−π, π) ⇔
- For the derivatives, we have
f
(p)
(x) =
�+∞
k=−∞
+∞
�
+∞
�
k=−∞
|fˆk |2
|fˆk |2 < ∞.
(ik)p fˆk e ikx .
k=−∞
p
- Let Hper
= {v ∈ H p (−π, π)|v is 2π-periodic}.
(f and the p first derivatives of f are square integrable).
f ∈
p
Hper
⇔
+∞
�
k=−∞
k 2p |fˆk |2 < ∞.
This is the generalization of the decay property for Fourier
coefficients.
Decay of Coefficients, contd.
coefficients
no decay
1/k decay
1/k 2 decay
1/k 4 decay
r k decay (r < 1)
functions
Delta functions
Step functions (with jumps)
Ramp functions (with corners)
Spline functions (jumps in f ��� )
Analytic functions
I.e, for each additional continuous derivative of f , the power of decay of
Fourier coefficents increases by one.
The partial sums for analytical functions (c ∞ ) converge exponentially
fast! This is the basis for fast solution methods for certain partial
differential equations.
The Discrete Fourier Transform (DFT)
Read: Strang, section 4.2
Without loss of generality assume the basic interval to be [0, 2π].
Let [0, 2π] be subdivided into N equidistant intervals,
h=
2π
,
N
xj = jh.
For a periodic function, f (0) = f (2π) = f (xN ) such that the trapezoidal
rule reads
� 2π
N−1
N−1
1
h � −ikxj
1 � � −ih �jk
−ikx
f (x)e
dx ≈
fj e
=
fj e
2π 0
2π
N
j=0
[let w = e
ih
j=0
and so w̄ = e
−ih
N−1
1 �
]=
fj w̄ jk := ck
N
j=0
The discrete version of the inverse transformation is,
f˜j =
N−1
�
ck e
ikxj
=
k=0
N−1
�
ck w kj
k=0
DFT, contd.
Theorem. One transformation is the inverse of the other,
fj ≡ f˜j
Proof Compute:
f˜j =
N−1
�
ck w
kj
=
k=0
Since
N−1
�
k=0
N−1
�
k=0
the result follows.
w
(j−l)k
N−1
N−1 N−1
1 �
1 � � (j−l)k
lk kj
fl w̄ w =
fl
w
N
N
l=0
=
�
1−w (j−l)N
1−w
N,
l=0
= 0,
k=0
if j �= l,
if j = l,
The Fast Fourier Transform (FFT)
Strang, section 5.1.
- The naive application of the discrete Fourier transform has
complexity O(N 2 ). (matrix-vector multiplication)
- Example: N = 212 .
- The naive approach requires 224 ≈ 1.7 · 107 complex multiplications.
- The FFT requires only 6 × 212 ≈ 2.4 · 104 multiplications.
- The basic idea: Let N be a power of 2 and M = N/2.
fj =
N−1
�
ck w kj =
k=0
=
M−1
�
k � =0
�
�
ck w kj +
k even
2 k�j
c2k � (w )
��
=:fj
�
�
ck w kj
k odd
+w
j
M−1
�
k �� =0
�
�
c2k �� +1 (w 2 )k
��
=:fj
��
��
j
�
- fj� and fj�� are discrete Fourier transform of half the original size M!
FFT (cont.)
- This formula can be simplified:
- For j = 0, . . . , M − 1: Take it as it stands.
- For j = M, . . . , N − 1: Let j � = j − M.
It holds w M = e ihM = e ihN/2 = e iπ = −1 and w N = w 2M = 1;
�
�
�
w j = w M+j = w M w j = −w j ,
�
(w 2 )kj = (w 2 )kj .
- This gives the identities
fj = fj� + w j fj��
fj+M = fj� − w j fj��
�
j = 0, . . . , M − 1
This can be applied recursively: compute each of fj� and fj ” by applying
this formula again. The number of terms in the sums is halved in each
step.
With N = 2n , log2 (N) levels until at ”the bottom”.
Computational complexity: O(N log N)
FFT: Computational Complexity
Assumptions:
- The exponentials w j are precomputed.
- Let W (N) be the number of complex operations for a FFT of length
N.
W (2M) = 2W (M) + 4M, W (1) = 0.
Denote wj = W (2j ) and N = 2n :
w0 = 0, wj = 2wj−1 + 2 · 2j .
Multiply the equation by 2N−j and sum up:
n
�
n−j
2
wj = 2
j=1
n
�
j=1
n−j
2
�
wj−1 + 2
j
�
n
= 2n2 +
n−1
�
2n−j wj−1
j=0
Consequently,
W (N) = wn = 2n2n = 2N log2 N
Shifted DFT
Using the base interval [0, 2π] leads to the standard DFT. What happens
if we use [−π, π] instead? (Let N be even.)
N/2−1
N−1
N−1
�
1 � −ikjh
1 �
−ikjh
i2π 1
ck =
fj e
=
fj e
+e
fj e −ikjh
N
N
N
j=0
j=0
j=N/2
N/2−1
N−1
1 �
1 �
−ikjh
=
fj e
+
fj e −ik(j−N)h
N
N
j=0
N/2−1
1 �
1
=
fj e −ikjh +
N
N
j=0
j=N/2
−1
�
l=−N/2
fl e
−iklh
1
=
N
Similarly, for the inverse DFT it holds,
�N/2−1
fl = k=−N/2 ck e iklh , l = −N/2, . . . , N/2 − 1.
N/2−1
�
l=−N/2
Order of coefficients:
DFT: (c0 , c1 , . . . , cN−1 )
shifted DFT: (cN/2 , cN/2+1 , . . . , cN−1 , c0 , . . . , cN/2−1 )
This is what matlab’s fftshift does.
fl e −iklh