The Fast Fourier Transform and
Applications to Multiplication
Analysis of Algorithms
Prepared by
John Reif, Ph.D.
Topics and Readings:
- The Fast Fourier Transform
• Reading Selection:
• CLR, Chapter 30
Advanced Material :
- Using FFT to solve other Multipoint Evaluation
Problems
- Applications to Multiplication
Nth Roots of Unity
• Assume Commutative Ring (R,+,·, 0,1)
• ω is principal nth root of unity if
– ωk ≠ 1 for k = 1, …, n-1
– ωn = 1, and
n-1
jp
w
å = 0 for 1 £ p £ n
j=0
• Example:
w=e
2p i/n
for complex numbers
Example of nth Root of Unity for
Complex Numbers
w=e
2p i/8
is the 8th root of unity
Fourier Matrix
æ1 1 … 1
ö
ç
÷
n -1
ç1 w … w
÷
M n (w ) = ç1 w 2 … w 2( n -1) ÷
ç
÷
ç
÷
çç
n -1
( n -1)( n -1) ÷
÷
1
w
…
w
è
ø
so M(w )ij = w ij for 0 £ i, j<n
æ a0 ö
ç
÷
given a = ç
÷
ça ÷
è n-1 ø
Discrete Fourier Transform
Input a column n-vector a = (a0, …, an-1)T
Output an n-vector which is the product of
the Fourier matrix times the input vector
DFTn (a) = M(w ) x a
æ f0 ö
ç ÷
= ç ÷ where
çf ÷
è n-1 ø
n-1
fi =
ik
a
w
å k
k=0
Inverse Fourier Transform
DFT (a) = M(w ) x a
-1
n
-1
1 -ij
Theorem M(w ) = w
n
proof We must show M(w ) × M(w )-1 = I
-1
ij
1
n
n-1
ik -kj
w
å w
k=0
1
=
n
n-1
k(i-j)
w
å
k=0
ì0 if i-j ¹ 0
= í
î1 if i-j = 0
n-1
using identity å w kp = 0, for 1 £ p < n
k=0
Fourier Transform is Polynomial
Evaluation at the Roots of Unity
Input a column n-vector a = (a0, …, an-1)T
Output an n-vector (f0, …, fn-1)T which are the
values polynomial f(x)at the n roots of unity
æ f0 ö
ç ÷
DFTn (a) = ç ÷ where
çf ÷
è n-1 ø
i
f i = f(w ) and
n-1
f(x) = å a j × x j
j=0
Fast Fourier Transform
• Viewed as Evaluation Problem:
naïve algorithm takes n2 ops
• Divide and Conquer gives FFT
with O(n log n) ops for n a power of 2
• Key Idea:
• If ω is nth root of unity
then ω2 is n/2th root of unity
• So can reduce the problem to two
subproblems of size n/2
Algorithm FFTn
•
Input a = (a0, …, an-1)T, n a power of 2
[1] If n=1 then ouput
T
 '

'
[2]  f 0 ,..., f n   FFTn (( a0 , a2 ,..., an 2 )T )
1
2 
2

T
 "
" 
T
 f 0 ,..., f n 1   FFTn (( a1 , a3 ,..., an 1 ) )
2 
2

n
[3] For i=0, ...,  1 do f i  f i'   i f i"
2
f n  f i'   i f i"
i+
[4] Output (f 0 , f1 , ..., f n-1 )
2
FFT Circuit
(also known as Butterfly Network)
• Total Recursion depth = log n
• Communication Distance 2d at depth d
f i = a 0 + a1 i + a 2 2i +...+a n-1 (n-1)i
f i = f i' +  i f i" where
i(n-2)
2
2
f i' = a 0 + a 2 ( 2 )i + a 4 ( 2 ) 2i +...+ a n-2 ( )
i(n-2)
2
2
f i" = a1 + a 3 ( 2 )i +...+ a n-1 ( )
 ' 
 a0 
f 0 


a

  M ( 2 )  2   DFT ((a , a ,..., a )T )
n
n
0
2
n2




2
2
f n' 


 2 1 
 an  2 
 " 
 a1 
f 0 
a 

  M ( 2 )  3   DFT (( a , a ,..., a )T )
n
n
1
3
n 1




2
2
f n" 


 2 1 
 an 1 
Note: f
'
n
+i
2
'
i
=f , f
"
n
+i
2
n
= f , i=0, ..., - 1
2
"
i
n
+1
2 2
But w n =1, so (w )
= w n × (w 2 )i = w 2i
n
for i=0, ..., - 1
2
n
'
i
"
Thus, fi = f i + w f i for i=0, ..., - 1
2
and f
i+
'
=
f
n
i +w
n
2
× f i"
2
=f -w f
'
i
i+
i
n
2 2
"
i
n
for i=0, ..., - 1
2
n
2
since (w ) = w n = 1, so w = -1
Operation Counts for FFT Algorithm
• Assume n = 2k
• # additions
Add(n) = 2· Add(n/2) + n
= n log n
• # multiplications
Mult(n) = 2· Mult(n/2) + n/2
= ½ n log n
• Total Time
O(n log n)
• Note in complex FFT,
# real ops is 5 n log n
Multipoint Polynomial Evaluation
• Input polynomial f ( x) =
n -1
i
a
x
å i
i =0
• Problem evaluate f(x) at x0, x1, …, xn-1
• Easy Cases: eval at roots of unity
FFT Case xk = ωk for k=0,…,n
Multipoint Polynomial Evaluation
(cont’d)
Summary of FFT:
method f ( x)  f '( y )  x f "( y )
where y  x 2
f '( x), f "( x) both degree halved
 needed to only evaluate at half as many points
Other Polynomial Evaluation
Problems Solved by FFT
Each costs O(n log n) time
•
Evaluate at points Xi = bai + d for i=0,…, n-1(Chirp Transfom)
– Reduced to FFT
•
Single point evaluation of all derivatives of a polynomial
– Solve by reduction to above Chirp Transform of case 2)
•
Evaluate at points Xi = b(ai)2+ cai + d for i=0,…, n-1
– Solve by divide and conquer similar to FFT
Single Point Evaluation of all
Derivatives of Polynomial
• Input
n -1
f ( x) = å ai x i
and point x0
i =0
• output
k
d
f ( x)
k
f ( x) =
x = x0 for k = 0,..., n - 1
dx
Single Point Evaluation of all
Derivatives of Polynomial (cont’d)
• Taylor Series Representation of
n -1
f ( x) = å ci ( x - x0 )i
i =0
Then
f ( k ) ( x0 ) = k !ck
This reduces to case of evaluation at points
xi  abi for i  0,..., n  1
• Solve this Chirp Transform problem by
reduction FFT
Advanced Material:
Further Applications of FFT
1) Convolution: Products and Powers of
Polynomials
• Used for for Integer Multiplication
Algorithms
• Also used for Filtering on infinite
input streams
2) Division and Inverse of Polynomials
3) Multipoint Evaluation and
Interpolation
Advanced Material: Products and
Powers of Polynomials
• Input vectors
a = (a0, a1, …, an-1)T
b = (b0, b1, …, bn-1)T
• Definition of Convolution c = a
⊗b
n-1
Where
ci = å a jbi-j
for i=0, …, 2n-1
j=0
define ak = bk = 0
if k< 0 or k>n
Products and Powers of Polynomials
(cont’d)
• Convolution Theorem
a Ä b = FFT
-1
2n
( FFT
2n
(a)× FFT2n (b)
• Application to Polynomial Products:
n-1
p(x) = å a i x i
i=0
n-1
q(x) = å b j x j
j=0
2n-2
n-1
i=0
j=0
p(x) × q(x) = å ci x i where ci = å a jbi-j
)
Products of m Polynomials
n-1
for k=1, ..., m let Pk (x) = å a k,i x i
i=0
m
m(n-1)
k=1
i=0
Õ Pk (x) =
å
m
ci x i , where ci = å Õ a k,jk
k=1
å jk =1
• Generalized Convolution Theorem
a1  a 2  ...  a m =
FFTn-1m  FFTn m (a1 )  FFTn m (a 2 )  ...  FFTn m (a m ) 
Wrapped Convolutions
• a = (a0, a1, …, an-1)T , b = (b0, b1, …, bn-1)T
• Positive wrapped convolution is
c = (c0, c1, …, cn-1)T
i
n-1
j=0
j=i+1
ci =å a jbi-j + å a jb n+i-j
• Negative wrapped convolution is
d = (d0, d1, …, dn-1)T
i
n-1
j=0
j=i+1
di = å a jbi-j - å a jbn+i-j
Application of Wrapped Convolution
to Modular Polynomial Products
n-1
p(x) = å a i x
i
i=0
n-1
q(x) = å b j x
j
j=0
p(x) × q(x) mod(x +1)
n
n-1
= å d i x since x = -1 mod(x +1)
i
i=0
n
n
Computing Positive Wrapped
Convolution
• Let ω = principal nth root of unity
• Assume n has multiplicative inverse,
Theorem
-1
n
c = FFT
(FFTn (a) × FFTn (b) )
is the positive wrapped convolution of
n-vectors a and b.
Computing Negative Wrapped
Convolution
• Also
(
ˆd = FFT-1 FFT (a)
ˆ
ˆ
×
FFT
(b)
n
n
n
)
is the negatively wrapped convolution of
n-vectors a and b
where
(
b̂ = (b , Yb , ...,Y
â = a 0 , Ya1 , ...,Y a n-1
n-1
0
1
n-1
)
)
b n-1
T
T
and Ψ2 = ω = principal nth root of unity
Integer Multiplication by Polynomial
Product (solved via FFT)
• Input n bit integers a,b
define polynomials degree k = n/L
k-1
a(x) = å a i x ,
i
0 £ ai £ 2
L
i=0
k-1
b(x) = å bi x i ,
0 £ bi £ 2L
i=0
L
L
so a = a(2 ), b = b(2 )
Integer Multiplication by Polynomial
Product (cont’d)
• Idea
1) Compute c(x) = a(x)· b(x)
by convolution
2) Evaluate c(2L) = a· b
Integer Multiplication Algorithms using
Reduction to Polynomial Product
• Pollard Mult Algorithm
Î
2
O(n(logn) )(loglogn) use L = logn
• Karp Mult Algorithm
O(n(logn)2 ) use L = n
• Schönage-Strassen Mult Algorithm
O(n(logn)(loglogn)) use L = n
and wrapped convolution
Pollard Multiplication Algorithm
• n = kL, L = 1 + log k
1) Choose primes P1, P2, P3 where
P1 × P2 × P3 ³ 4 × k
3
and Pi = á i × 2 + 1, á i = O(1)
L
2) Compute C(x) by convolution over
finite field Zpi for i =1,2,3
(requires k mults on 2L bit integers)
Pollard Multiplication Algorithm
(cont’d)
3) Evaluate C(2L)
• Time Bounds
recursive mults
FFT
T(n) = 3kT(2L) + O(k logk) × O(L)
= 3kT (2(1 + logk)) + O(k(log k)2 )
£ O(n(log n)2 (log log n)Î ) for any Î > 0
Korp Multiplication Algorithm
n = 2s = kL
ì 2s
if s even
ï2
k = í (s-1)
ï2 2 else
î
1) Compute C(x) modulo k by
convolution
2) Compute C(x) modulo (22L+1) by
convolution
3) Compute C(x) coefficients from C(x)
mod k, C(x) mod (22L+1) by Chinese
remaindering
Korp Multiplication Algorithm
(cont’d)
4) Compute C(2L)
• Time
recursive mults
FFT
T(n) = 2kT(2L) + O(k logk)O(L)
= 2 nT (2 n ) + O(n log n)
= O(n(log n) )
2
Schönage-Strassen Multiplication
Algorithm
(2’) Compute C(x) mod (xk+1) modulo
(22L+1) by wrapped convolution
Requires only k recursive mults on 2L
bit numbers
• Time
recursive mults
FFT
T(n) = kT(2L) + O(k logk)O(L)
= nT (2 n ) + O(n log n)
= O(n log n)(log log n)
Still Open Problem: How Fast Can
You Multiply Integers?
• Can you mult n bit integers in
O(n log n) time?
The Fast Fourier Transform and
Applications to Multiplication
Analysis of Algorithms
Prepared by
John Reif, Ph.D.