Fourier Series, Discrete Fourier Transforms and Fast Fourier Transforms
Math 572
1.
Complex Numbers
We start with some basics of complex numbers:
If z = a + bi is a complex number, then the complex conjugate of z is the complex number z = a − bi .
From Euler’s theorem, we have a relationship between e x , cos x and sin x : e ix
= cos x + i sin x and e
− ix
= cos x − i sin x
We will define ω
N
= e
− 2 πi
N for N a positive integer.
ω
N is called a primitive N th root of unity . In particular, ω
N is a root of x N − 1 and is not a root of x k − 1 for k < N . Some useful properties of ω
N are:
ω k
N
= ω
M if N = kM
N/ 2
ω
N
= − 1 if N is even
ω
M
N
= ω r
N if M = kN + r
2.
Fourier Series over
[0 , 2 π ]
In this section we consider the function space C [0 , 2 π ]. Our initial goal will be to approximate a given function f ( x ) in C [0 , 2 π ] using linear combinations of functions of the form cos kx and sin kx where k is a non-negative integer. This is obviously useful for periodic functions. Our setup is as follows: h f ( x ) , g ( x ) i =
1 Z
2 π f ( x ) g ( x ) dx
π
0 and
S n
= Span(
1
√
2
, sin x, . . . , sin nx, cos x, . . . , cos nx )
We can check that the following are true:
Therefore h 1 , cos αx i =
1 Z
2 π cos αx dx = 0
π
0 h cos αx, cos βx i =
1 Z
2 π h 1 , sin βx i =
1
π
0
π
0 sin βx dx = 0

Z
2 π cos αx cos βx dx =


1 if α = β
0 if α = β h sin αx, sin βx i =
1
π

Z
2 π sin αx sin βx dx =

0

1 if α = β
0 if α = β h cos αx, sin βx i =
1 Z
2 π cos αx sin βx dx = 0
π
0
{
1
√
2
, sin x, . . . , sin kx, cos x, . . . , cos kx } is an orthonormal basis for S k
.
1

If we define: a
0
= h f ( x ) , 1 i =
1 Z
2 π f ( x ) dx
π
0 a k
1
= h f ( x ) , cos kx i =
π
Z
2 π
0 f ( x ) cos kx dx and b k
1
= h f ( x ) , sin kx i =
π
Z
2 π
0 f ( x ) sin kx dx then the least squares approximation of f ( x ) in S n will be: s n
( f ( x )) = a
0
+ n
X
( a k cos kx + b k sin kx )
2 k =1
This sum is an example of a trigonometric polynomial and is known as the n th degree Fourier series for f ( x ) over [0 , 2 π ] .
It is generally convenient to write this series in exponential form, which is what we proceed to do:
Let c k
=
1
2
( a k
− ib k
) and c
− k
= c k
With these definitions, we have the following relations: a k
= c k
+ c
− k and b k
= i ( c k
− c
− k
)
So c k
=
1
2
( a k
− ib k
) =
1
2 π
Z
2 π f ( x )(cos kx − i sin kx ) dx
0
Using Euler’s theorem, we have: c k
=
1
2 π
Z
2 π f ( x ) e
− ikx
0 dx
Finally, notice that c k e ikx
+ c
− k e
− ikx
= c k
(cos kx + i sin kx ) + c
− k
(cos kx − i sin kx ) = ( c k
+ c
− k
) cos kx + i ( c k
− c k
) sin kx
This gives: c k e ikx
+ c
− k e
− ikx
= a k cos kx + b k sin kx
Therefore the n th degree Fourier series: s n
( f ( x )) = a
0
+ n
X
( a k cos kx + b k sin kx )
2 k =1 can be rewritten as: s n
( f ( x )) = n
X c k e ikx k = − n
Fourier series have many applications. One of the most important is in the area of signal processing. The
Fourier series of a noisy function can be used to reduce this noise. Noise in this case refers to the terms of s n
( f ( x )) where | k | is large. We can eliminate this noise by setting the corresponding c k
2 to 0.

3.
Discrete Fourier Transform
In applications, we often do not know f ( x ) explicitly, rather we have to approximate f ( x ) by sampling. This will lead to an approximation of s n
( f ( x )).
First, we partition the interval [0 , 2 π ] into N equal parts: x
0
= 0 , x
1
=
2 π
, . . . , x
N
N
= 2 π
In particular, x j
=
2 πj
N
For each x j we measure the output y j and set y j
= f ( x j
)
Since we do not know f ( x ), we can not compute c k
= 1
2 π
R
2 π
0 f ( x ) e − ikx dx . Rather we will approximate this integral using the left endpoint Riemann sum given the partition above: c k
≈
N − 1
X f ( x j
) e
− ikx j
N j =0
1
Note that e ikx j = e
−
2 πikj
N
= ω jk
N where ω
N is the primitive N th root of unity defined in section 1. Therefore
N − 1
X f ( x j
) e
− ikx j
N j =0
1 1
=
N
N − 1
X y j
ω jk
N j =0
We define d k
=
N − 1
X y j
ω jk
N j =0
The sequence { d
0
, d
1
, . . . , d
N − 1
} is called the discrete Fourier transform or DFT of { y
0
, y
1
, . . . , y
N − 1
}
We will now show how to compute d k using matrix multiplication. Let F
N entry is ω
( j − 1)( k − 1)
N so that be the N × N matrix whose k, j
F
N

1 1
=



1 ω
N



1 ω
2
N



1 ω
3
N



.
..

1 ω
N − 1
N
ω
2( N − 1)
N
1
ω
2
N
ω
4
N
ω
6
N
.
..
ω
N
1
ω
3
N
ω
6
N
ω
9
N
.
..
3( N − 1)

· · · 1
· · ·
· · ·
· · ·
ω
N − 1
N
2( N − 1)
ω
N
3( N − 1)
ω
N
.
..












· · · ω
( N − 1)( N − 1)
N

Note that F
N is a symmetric matrix (ie.
F T
N
= F
N
).
If we define y and d by:
  y
0 y =






 y
..
.
1 y
N − 1







  d
0 and d =






 d
..
.
1 d
N − 1







Then a direct calculation shows that
F
N y = d
3

4.
Fast Fourier Transform
Using the matrix calculation above to compute the discrete Fourier transform of a sequence { y
0
, y
1
, . . . , y
N − 1
} requires approximately 8 N 2 operations. There are many algorithms which can be used to speed this processes up, they are collectively known as the fast Fourier transform or FFT . The best known of these is the Cooley-
Tukey algorithm. This algorithm was developed by Cooley and Tukey in the 1960’s, though it was later found to be originally developed by Gauss in the early 1800’s. This algorithm works when N is even and relies upon reordering the columns of F
N
.
To get started, we define two matrices as follows: Let { e
1
, e
2
, . . . , e
N
} be the standard basis for
R n . Then
P
N is defined to be:
P
N
= ( e
1 e
3 e
5
· · · e
N − 1 e
2 e
4
· · · e
N
)
Notice that P
N is a N × N permutation matrix, in particular P
− 1
N
N/ 2 × N/ 2 diagonal matrix whose j, j entry is ω j − 1
N
.
= P T
N
. We also define D
N/ 2 to be the
Our first goal is to rearrange the columns of F
N so that all of the odd numbered columns occur before all of the even columns. This can be done by multiplying F
N on the right by P
N
. This has the effect of arranging the columns so that the first N/ 2 columns contain only even powers of ω
N
. We then use the relations ω 2
N and ω k ( N/ 2+1)
N/ 2
= ω k
N/ 2
= ω
N/ 2 to reduce the entries to powers of ω
N/ 2
. These columns are now seen to consist of two copies of F
N/ 2
. For the remaining columns, we can do a similar reduction, with the added complication that we have a copy of D
N/ 2
F
N/ 2 and of − D
N/ 2
F
N/ 2
. Once we do this, we have the following block decomposition of
F
N
:
F
N
P
N
=

F
N/ 2

F
N/ 2

D
N/ 2
F
N/ 2
− D
N/ 2
F
N/ 2

So,
F
N y = ( F
N
P
N
)( P
T
N y )
If we write
P
T
N y =
 w
1

 w
2
 where w
1 is the first half of of P T
N y and w
2 is the second half of this vector and use the properties of block multiplication (see section 1.5 of Leon):
F
N y =

F
N/ 2

F
N/ 2
D
N/ 2
F
N/ 2
  w
1

− D
N/ 2
F
N/ 2
  w
2

=

F
N/ 2 w
1

F
N/ 2 w
1
+ D
N/ 2
F
N/ 2 w
2

− D
N/ 2
F
N/ 2 w
2

Therefore, to compute F
N y we only need to compute F
N/ 2 w
1 and D
N/ 2
F
N/ 2 w
2
. This process can be used recursively, in stages: one stage for each power of 2 which divides N . This algorithm is most efficient when N is a power of 2. In this case the FFT takes approximately 5 N log
2
N operations (as opposed to 8 N 2 operations for the DFT). For example, if N = 1024 then the dft takes 2 23 operations whereas the FFT will take 2 11 ∗ 25 operations. This is speedup by a factor of 2 12 / 25.
4

Example:
Let N = 6 and y = (1 , 2 , 3 , 4 , 5 , 6)
T
.
Then

1 1
F
6
=



1 ω
6



1 ω 2
6



1 ω 3
6



1 ω 4
6

1 ω 5
6
ω
ω
ω
ω
1
2
6
4
6
6
6
8
6
ω 10
6
ω
1
ω
ω
ω
3
6
6
6
9
6
12
6
ω 15
6
ω
ω
ω
ω
1
4
6
8
6
12
6
16
6
ω 20
6

1
ω
5
6



ω 10
6



ω 15
6



ω 20
6



ω 25
6

It is advantageous to reduce the various powers of ω
6 to smaller powers of ω
6 and powers of lower order primitive roots of unity (also note that ω
3
6
= − 1 , ω
5
6
= − ω
3 and ω
6
= − ω
2
3
):

1 1 1 1 1 1

F
6
=






1 − ω
1 ω
3
2
3
ω
3
ω
2
3
−
1
1 ω
ω
2
3
3

− ω
3


ω
2
3



1 − 1 1 − 1 1 − 1



1 ω
2
3
ω
3
1 ω
2
3
ω
3










1 − ω
3
ω 2
3
− 1 ω
3
− ω 2
3

P
6

1 0 0 0 0 0

=



0 0 0 1 0 0 






0 1 0 0 0 0 

 


0 0 0 0 1 0 

 


0 0 1 0 0 0 


0 0 0 0 0 1

F
3
=

1 1 1




1 ω

1 ω
3
2
3

1 1
ω
2
3



ω
4
3

=



1 ω
3

1 ω
2
3
1
ω
2
3
ω
3






1 0 0
 
1 0 0

D
3
=



0 ω
6
0




0 0 ω
2
6

=



0 − ω 2
3
0



0 0 ω
3

 and

D
3
F
3
1
=



− ω
2
3

ω
3

1 1
− 1 − ω
3




1 ω 2
3
5

Now compute F
6
P
6
:
Therefore

1 1 1 1 1 1

F
6
P
6
=



1 ω
3



1 ω
2
3



1 1 1 − 1 − 1 − 1



1 ω
3
ω
ω
2
3
3
ω 2
3
−
ω
ω
3
ω 2
3
2
3

− 1 − ω
3


1
1
ω
ω
2
3
3










1 ω 2
3
ω
3
− ω
3
− 1 − ω 2
3

=

F
3

F
3

D
3
F
3
− D
3
F
3
 d = F
6 y = F
6
P
6
P
6
T y =

F
3

F
3
 
1
 
2
 

1

D
3
F
3

− D
3
F
3

 


3 




5



 


2 

 


4 


6

=



F
3












F
3




3


1


5


3

5








+ D
3
F
3
− D
3
F
3
  


4  
 

6
 






2
 


  


4  
 

6
 
Finally,

1

F
3
 


3 


5

=

1 1 1



1 ω

1 ω
3
2
3
 
1

ω
2
3



ω
3




3




5

=
 
9



1 + 3 ω
3

1 + 3 ω 2
3
+ 5 ω
2
3



+ 5 ω
3
 and

2
 
1
D
3
F
3
 


4 


6

=



− ω 2
3

ω
3
1 1
 
2
 
12


− 1 − ω
3


 


4 

1 ω 2
3
 
6

=



− 2 ω 2
3

2 ω
3
− 4 − 6 ω
+ 4 + 6 ω 2
3

3



So d = F
6 y =
 
9 + 12
1
6









− 3 − 3 ω
3



5 + 7 ω
3
+ 3 ω
+ 9 ω
2
3
2
3






9
5 + 9
−
ω
3
12
+ 7 ω 2
3







− 3 + 3 ω
3
− 3 ω 2
3

=
 
1
6
21
 


− 3 .
0000 + 5 .
1962 i 

 


− 3 .
0000 + 1 .
7321 i 





− 3 .
0000






− 3 .
0000 − 1 .
7321 i 


− 3 .
0000 − 5 .
1962 i

6