The Fast Multipole Method: It’s All
about Adding Functions
Alan Edelman
MIT: Dept of Mathematics,
Lab for Computer Science
FOCM 2002
Saturday, August 10
5/28/2016
1
Outline





Multipole: What are we adding?
The exclusive add without the multipole
Representing functions on a computer
“Adding” functions and the Pascal Matrix
Research Opportunities
5/28/2016
2
The matrix-vector viewpoint:
f =V(y, x) q
potentials at
y1 , ..., yN Vij = r(yi - xj)
charges at
x1 , ..., xN
qj
Example: fi=S
j ||yi-xj||
2
O(N )
Direct evaluation:
work
Greengard, Rokhlin FMM (1987): (N)
5/28/2016
3
Some applications:
•N-body Problems
•Potential Evaluation
•Future Fast Fourier Transform
•Divide and Conquer Eigenvalue Solvers
•Polynomial Roots
•More waiting to be found ….
5/28/2016
4
Outline





Multipole: What are we adding?
The exclusive add without the multipole
Representing functions on a computer
“Adding” functions and the Pascal Matrix
Research Opportunities
5/28/2016
5
It’s all about adding functions:
qj
fi=Sj ||y -x ||
i j
f(y)=Sj fj(y)
fj(y)=
5/28/2016
qj
||y-xj||
6
What might these functions look like?
f(y)=Sj fj(y)
f1(y)
f2(y)
5/28/2016
7
Outline





Multipole: What are we adding?
The exclusive add without the multipole
Representing Functions on a computer
“Adding” functions and the Pascal Matrix
Research Opportunities
5/28/2016
8
The Exclusive Add
yi = Sji xj
y =
01111111
10111111
11011111
11101111
11110111
11111011
11111101
11111110
x
•Why
not sum and subtract each element?
sum(x)-x
•What if NaN’s or numbers on different
scales are present?
5/28/2016
9
The Exclusive Add
yi = Sji xj
y =
01111111
10111111
11011111
11101111
11110111
11111011
11111101
11111110
x
A Good Algorithm is worth seeing five ways!
• 1. Pseudocode
• 2. MATLAB
• 3. On a tree
• 4. Matrix Notation
• 5. Kronecker Product Notation
5/28/2016
10
Exclusive Add (pseudocode)
yi = Sji xj
1 2 3 4 5 6 7 8 1)Pairwise Sums
3 7 11 15 2)Recursive Excl Add
33 29 25 21 3)Update “evens & odds”
35 34 33 32 3130 29 28
5/28/2016
11
Exclusive Add in MATLAB
yi = Sji xj
1 2 3 4 5 6 7 8 1)Pairwise Sums
3 7 11 15 2)Recursive Excl Add
33 29 25 21 3)Update “evens & odds”
35 34 33 32 3130 29 28
function y=exclude(v)
n=length(v);
if n==1, y=0*v; else
w=v(1:2:n)+v(2:2:n);
% Pairwise adds
w=exclude(w);
% Recur
y(1:2:n)=w+v(2:2:n); y(2:2:n)=w+v(1:2:n); % Update Adds
end
5/28/2016
12
Exclusive Add on a Tree
0
36
10
26
3 7
11 15
1 2 3 4 5 6 7 8
Pairwise sums “up the tree”
5/28/2016
yi = Sji xj
26
10
33 29 25 21
35 34 33 32 3130 29 28
Modify evens/odds down
13
With Matrices
01111111
10111111
11011111
11101111
11110111
11111011
11111101
11111110
5/28/2016
=
1000
1000
0100
0100
0010
0010
0001
0001
0111
1011
1101
1110
1000
1000
0100
0100
0010
0010
0001
0001
T
+
01000000
10000000
00010000
00100000
00000100
00001000
00000001
00000010
14
Kronecker product Approach
01111111
10111111
11011111
11101111
11110111
11111011
11111101
11111110
A8
=
1000
1000
0100
0100
0010
0010
0001
0001
=(I4(
1
1
0111
1011
1101
1110
1000
1000
0100
0100
0010
0010
0001
0001
1
1
T
+
01000000
10000000
00010000
00100000
00000100
00001000
00000001
00000010
)) A4 (I4( ) + I4  ( )
)T
01
10
1)Pairwise Sums
2)Recursive Excl Add
3)Update “evens & odds”
5/28/2016
15
Kronecker product Approach
01111111
10111111
11011111
11101111
11110111
11111011
11111101
11111110
A8
=
1000
1000
0100
0100
0010
0010
0001
0001
=(I4(
1
1
0111
1011
1101
1110
1000
1000
0100
0100
0010
0010
0001
0001
1
1
T
+
01000000
10000000
00010000
00100000
00000100
00001000
00000001
00000010
)) A4 (I4( ) + I4  ( )
)T
01
10
1)Pairwise Sums
2)Recursive Excl Add
3)Update “evens & odds”
5/28/2016
16
Kronecker product Approach
01111111
10111111
11011111
11101111
11110111
11111011
11111101
11111110
A8
=
1000
1000
0100
0100
0010
0010
0001
0001
=(I4(
1
1
0111
1011
1101
1110
1000
1000
0100
0100
0010
0010
0001
0001
1
1
T
+
01000000
10000000
00010000
00100000
00000100
00001000
00000001
00000010
)) A4 (I4( ) + I4  ( )
)T
01
10
1)Pairwise Sums
2)Recursive Excl Add
3)Update “evens & odds”
5/28/2016
17
Kronecker product Approach
01111111
10111111
11011111
11101111
11110111
11111011
11111101
11111110
A8
=
1000
1000
0100
0100
0010
0010
0001
0001
=(I4(
1
1
0111
1011
1101
1110
1000
1000
0100
0100
0010
0010
0001
0001
1
1
T
+
01000000
10000000
00010000
00100000
00000100
00001000
00000001
00000010
)) A4 (I4( ) + I4  ( )
)T
01
10
1)Pairwise Sums
2)Recursive Excl Add
3)Update “evens & odds”
5/28/2016
18
The
All Algorithms
1)Pairwise Sums
2)Recursive “foo”
3)Update
Directions Inclusive
Exc=0
Prefix
Left
Suffix
Right
Left/Right Reduce
5/28/2016
Family
Exclusive
Exc=1
Exc Prefix
Exc Suffix
Exc Add
Neighbor Exc
Exc=2
Left Multipole
Right " " "
Multipole
19
Multipole is a doubly exclusive add
00111111
00011111
10001111
11000111
11100011
11110001
11111000
11111100
•We add functions with poles not numbers
•We exclude ourselves and our nearest neighbors
For reasons analogous to the floating point example:
The function can not be represented accurately near
the singularities.
•This is 1d, higher dimensions analogous
•Algorithm: Pairwise Add, Recursion, Update Missing Pieces
5/28/2016
20
Outline





Multipole: What are we adding?
The exclusive add without the multipole
Representing Functions on a computer
“Adding” functions and the Pascal Matrix
Research Opportunities
5/28/2016
21
“Rounding” functions to p coefficients
Best you can do:
1)Pick an interval
2)Pick a function
space inside or
outside the interval
3)Find the best
approximation on
that function space.
5/28/2016
22
“Rounding” functions to p coefficients
•Ideal polynomial Approximation
•Truncated Taylor Series
•f(x) = S ak(x-c)k
•Theory: Converges in a disk from c to nearest
singularity
•Practical: Accurate where smooth, far from the
singularity
•Alternative Polynomial: Interpolate rather than
Taylor
5/28/2016
23
“Rounding” functions to p coefficients
•Ideal Multipole Approximation
•Truncated Multipole Series
•f(x) = S ak/(x-c)k+1
•Theory: Converges outside a disk from c to
nearest singularity
•Practical: Accurate where smooth, far from the
singularity
•Alternative Multipole: Interpolate
5/28/2016
24
“Rounding” functions to p coefficients
•Examples
•R multipole/Taylor
Gu
•R Chebyshev interpolation Dutt, Gu, Rokhlin
•S1 Chebyshev interpolation Dutt, Rokhlin
•R singular funs of int ops
Yarvin, Rokhlin
•C multipole/Taylor
Greengard, Rokhlin
•C discretized Poisson form Anderson
•C singular funs of int ops
Hrycak, Rokhlin
•R3 multipole/Taylor
Greengard
•R3 discretized Poisson
Anderson
•R3 singular funs of int ops
Greengard, Rokhlin
•Any5/28/2016
Virtual Charges
E, McCorquodale
25
Finite Precision Arithmetic
•Idea adding real numbers
•We all know what this means
•x=1+1
•x=e+
•On a computer:
•Storage: must round to d-bit precision
•Arithmetic: need a finite precision algorithm
5/28/2016
26
Finite Precision Arithmetic
•Idea adding functions
•We all know what this means
•f(x)=sin2(x)+cos2(x) = 1
•f(x)=log(x)+log(x) = log(x2)
•On a computer:
•Storage: must round to d-bit precision
•Arithmetic: need a finite precision algorithm
5/28/2016
27
Functions to add
•Slowly growing singularity
•f(x)=q/(x-c)
•f(x)=q*log(x-c)
•f(x)=q*cot(x-c) but not f(x)=exp(-(x-c)2)
5/28/2016
28
Outline





Multipole: What are we adding?
The exclusive add without the multipole
Representing Functions on a computer
“Adding” functions and the Pascal Matrix
Research Opportunities
5/28/2016
29
Adding Functions
•Issues with adding polynomials
•common center
•accuracy
•Same issues appear with multipole
5/28/2016
30
The Pascal Matrix
•P=pascal(5)
1 1 1
1 2 3
1 3 6
1 4 10
1 5 15
( )
•Pij=
i+j
i
1
4
10
20
35
1
5
15
35
70
i,j = 0,…,n-1
•Gil Strang’s latest favorite matrix
5/28/2016
31
The Pascal Matrix
•P=pascal(5)
1 1 1
1 2 3
1 3 6
1 4 10
1 5 15
( )
•Pij=
5/28/2016
i+j
i
1
4
10
20
35
1
5
15
35
70
i,j = 0,…,n-1
32
The Pascal Matrix
•P=pascal(5)
1 1 1
1 2 3
1 3 6
1 4 10
1 5 15
( )
•Pij=
5/28/2016
i+j
i
1
4
10
20
35
1
5
15
35
70
i,j = 0,…,n-1
33
The Pascal Matrix
•P=pascal(5)
1 1 1
1 2 3
1 3 6
1 4 10
1 5 15
( )
•Pij=
5/28/2016
i+j
i
1
4
10
20
35
1
5
15
35
70
i,j = 0,…,n-1
34
The Pascal Matrix
•P=pascal(5)
1 1 1
1 2 3
1 3 6
1 4 10
1 5 15
( )
•Pij=
5/28/2016
i+j
i
1
4
10
20
35
1
5
15
35
70
i,j = 0,…,n-1
35
The Pascal Matrix
•P=pascal(5)
1 1 1
1 2 3
1 3 6
1 4 10
1 5 15
( )
•Pij=
5/28/2016
i+j
i
1
4
10
20
35
1
5
15
35
70
i,j = 0,…,n-1
36
The Pascal Matrix
•P=pascal(5)
1 1 1
1 2 3
1 3 6
1 4 10
1 5 15
( )
•Pij=
5/28/2016
i+j
i
1
4
10
20
35
1
5
15
35
70
i,j = 0,…,n-1
37
The Pascal Matrix
•P=pascal(5)
1 1 1
1 2 3
1 3 6
1 4 10
1 5 15
( )
•Pij=
5/28/2016
i+j
i
1
4
10
20
35
1
5
15
35
70
i,j = 0,…,n-1
38
The Pascal Matrix
•P=pascal(5)
1 1 1
1 2 3
1 3 6
1 4 10
1 5 15
( )
•Pij=
5/28/2016
i+j
i
1
4
10
20
35
1
5
15
35
70
i,j = 0,…,n-1
39
The Pascal Matrix
•P=pascal(5)
1 1 1
1 2 3
1 3 6
1 4 10
1 5 15
( )
•Pij=
5/28/2016
i+j
i
1
4
10
20
35
1
5
15
35
70
i,j = 0,…,n-1
40
Pascal and Cholesky(Pascal)
•P=pascal(5)
1 1 1 1 1
1 2 3 4 5
1 3 6 10 15
1 4 10 20 35
1 5 15 35 70
•L=chol(pascal(5))
10000
11000
12100
13310
14641
( ) i,j = 0,…,n-1
•Pij=
i+j
i
5/28/2016
i+j
=S
j
P=LL’ ( )
( ) i,j = 0,…,n-1
•Lij=
i
k
i
j
j
j-k
( )( )
41
Binomial Identities
i+j
Si=0 ( i )

•
(1-x)-(j+1)
=
•
(1+x) j
= Si=0 ( ) xi
•
(x-1)-(j+1)
5/28/2016
j
i
j

xi
P
 L’
= Si=j ( ) x-(i+1)  L
i
j
42
“Flipping” (multipoletaylor)
w=Fv
wi (x-d)i = S vj / (x-c) j+1
S
i=0
j=0
Fij = (c-d)-i Pij (d-c)-(j+1)
Proof: Differentiate i times then evaluate at x=d
or cleverly use (1-x)-(j+1) = S( i+j)xi
i
5/28/2016
43
“Shifting” (multipolemultipole)
w=Sv
wi /(x-d) i+1 = S vj / (x-c) j+1
S
j=0
i=0
Sij = (c-d) (i+1) Lij (c-d)-(j+1)
5/28/2016
44
“Shifting” (taylortaylor)
w=Sv
S wi (x-d)i = S vj(x-c)j
i=0
j=0
Sij = (d-c)-i LTij (d-c) j
5/28/2016
45
Outline





Multipole: What are we adding?
The exclusive add without the multipole
Representing Functions on a computer
“Adding” functions and the Pascal Matrix
Research Opportunities
5/28/2016
46
Group Representations
Let V be a vector space of functions of x
e.g. polynomials, rational functions, etc.
Let G be a group acting on {x},
e.g. rotations, non-singular matrices.
Clearly the map r from f(x) to h(x)=f(g-1x) is
linear.
Clearly r(gh)= r(g) r(h).
We say that r is a representation of G.
5/28/2016
47
Group Representations Research
Generalize Fast Multipole to Arbitrary
Representations!
Algebraic Multipole Theory???
5/28/2016
48