.
.
.
.
..
.
Chebfun2: Bivariate function approximation
the ACA way
Alex Townsend
Supervised by Nick Trefethen
Chebfun and Beyond Workshop
.
Alex Townsend
Chebfun2
.
.
.
.
.
. What is Chebfun2?
Project: An attempt to generalise Chebfun to rectangles.
Mathematical idea: A continuous analogue of low-rank
matrices.
Software: 12,000 lines of Matlab code, 203 m-files and 2
users.
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Approximate
Internally, we represent a function by a matrix of values on a
Chebyshev tensor grid.
.
Definition
..
A rank-k matrix A ∈ Rm×n is of low rank if
k · (m + n) < m · n
.
..
We’re interested in low-rank functions, i.e. values from a
Chebyshev tensor grid form a low rank matrix.
.
Alex Townsend
Chebfun2
.
.
.
.
.
.
.
.
. Approximate

 
1
a11 a12 . . . a1n
a21 a22 . . . a2n   a21

  a11
 ..
.. . .
..  =  ..
 .
. .   .
.
an1
an1 an2 . . . ann
a11
(1)
aij = aij −


a11 a12 . . . a1n
(1)
(1) 

1
0 a22 . . . a2n 



.
..
.. 
..
..

. 
.
.
. 
 ..
(1)
(1)
1
0 an2 . . . ann
ai1
aj1 ,
a11
2 ≤ i, j ≤ n.
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Approximate
 1
a11
a11
a21 1



A= .

..
 ..
. 
an1
1



a11 a12 . . . a1n
(1)
(1) 

0 a22 . . . a2n 
1



.. 
.
..
..
..

.
. 
. 
.
 ..
(1)
(1)
1
0 an2 . . . ann
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Approximate
 1
a11
a11
a21 1



A= .

..
 ..
. 
an1
1



a11 a12 . . . a1n
(1)
(1) 

0 a22 . . . a2n 
1



.. 
.
..
..
..

.
. 
. 
.
 ..
(1)
(1)
1
0 an2 . . . ann
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Approximate
 1
a11
a11
a21 1



A= .

..
 ..
. 
an1
1



a11 a12 . . . a1n
(1)
(1) 

0 a22 . . . a2n 
1



.. 
.
..
..
..

.
. 
. 
.
 ..
(1)
(1)
1
0 an2 . . . ann



0 0 ... 0
a11

(1)
(1) 

) 0 a22 . . . a2n 
1 
a21  (
A−
 .  a11 a12 . . . a1n = 
..
.. 
..

 ..
a11  .. 
.
. 
.
.
(1)
(1)
an1
0 an2 . . . ann

.
Alex Townsend
Chebfun2
.
.
.
.
.
. Approximate
 1
a11
a11
a21 1



A= .

..
 ..
. 
an1
1



a11 a12 . . . a1n
(1)
(1) 

0 a22 . . . a2n 
1



.. 
.
..
..
..

.
. 
. 
.
 ..
(1)
(1)
1
0 an2 . . . ann



0 .0. . . . 0
a11

(1)
(1) 

. 
) .0. a22 . . . .a2n
1 
a21  (

A−
 ..  a11 a12 . . . a1n = 
..
..
.
.

..
.. 
a11  . 
.
.

(1)
(1)
an1
0 .an2
. . . . ann

..
Movie.
.
Alex Townsend
Chebfun2
.
.
.
.
.
. History of ACA by mugshots
Eugene Tyrtyshnikov
Bebendorf, Goreinov,
Oseledets, Savostyanov,
Zamarashkin.
Mario Bebendorf
Keith Geddes
Carvajal, Chapman.
Gesenhues, Griebel,
Hackbusch, Rjasanow.
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Integrate
∫
1
∫
1
f (x, y )dxdy .
−1
−1
Clenshaw−Curtis weights
wT
C(y)
U
R(x)
.
Alex Townsend
Chebfun2
w
.
.
.
.
.
. Integrate
∫
1
∫
1
f (x, y )dxdy .
−1
−1
Clenshaw−Curtis weights
wT
C(y)
U
R(x)
w
Integration takes O(n log n + kn) operations. See Waldvogel’s
talk for computing weights on Wednesday 10:30.
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Differentiate
.
Definition (Tensor product operators)
..
L = Ly ⊗ L x ,
Lx and Ly linear.
.
..
Example of tensor product operators
.. Differentiation
1
.
.
.
.
L=
∂
,
∂y
L = D ⊗ I,
O(kn) operations.
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Differentiate
.
Definition (Tensor product operators)
..
L = Ly ⊗ L x ,
Lx and Ly linear.
.
..
Example of tensor product operators
.. Differentiation
1
.
.
.
.
L=
...
2
∂
,
∂y
L = D ⊗ I,
O(kn) operations.
Discrete Cosine Transform
F2 : vals → coeffs,
F2 = F⊗F ,
O(kn log n) operations.
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Differentiate
.
Definition (Tensor product operators)
..
L = Ly ⊗ L x ,
Lx and Ly linear.
.
..
Example of tensor product operators
.. Differentiation
1
.
.
.
.
L=
...
2
∂
,
∂y
L = D ⊗ I,
Discrete Cosine Transform
F2 : vals → coeffs,
...
3
O(kn) operations.
F2 = F⊗F ,
O(kn log n) operations.
Evaluation
Ef = f (x, y ),
E = Ey ⊗ Ex ,
O(kn2 ) operations.
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Orthogonalise: Singular Value Decomposition
C(y)
U
R(x)
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Orthogonalise: Singular Value Decomposition
( QC(y) RC )
...
1
U
R(x)
Reduced QR factorisation of columns (Trefethen 2009,
Gonnet).
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Orthogonalise: Singular Value Decomposition
( QC(y) RC )
...
...
1
2
U
( RTR
QR(x)T )
Reduced QR factorisation of columns (Trefethen 2009,
Gonnet).
Reduced QR factorisation of rows.
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Orthogonalise: Singular Value Decomposition
QC(y) ( RC
...
...
...
1
2
3
U
RTR )
QR(x)T
Reduced QR factorisation of columns (Trefethen 2009,
Gonnet).
Reduced QR factorisation of rows.
Discrete singular value decomposition.
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Orthogonalise: Singular Value Decomposition
( QC(y) UC )
...
...
...
1
2
3
S
( VTR
QR(x)T )
Reduced QR factorisation of columns (Trefethen 2009,
Gonnet).
Reduced QR factorisation of rows.
Discrete singular value decomposition.
Total Operations ∼ 12k 2 n + 20k 3 .
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Analyse
Convergence of ACA
0
10
Relative error in L2
Best rank approx.
ACA
−5
10
Differentiable functions
−10
10
Analytic functions
−15
10
0
10
20
30
40
50
60
Terms in truncation
In practice, ACA computes the near-optimal rank approximation
to a smooth function.
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Optimise: Global Maximum
max {f (x, y ) : x, y ∈ [−1, 1]} .
...
...
1
Find the maximum on a Chebyshev tensor grid.
2
Constrained Newton iteration.
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Optimise: Global Maximum
max {f (x, y ) : x, y ∈ [−1, 1]} .
...
...
1
Find the maximum on a Chebyshev tensor grid.
2
Constrained Newton iteration.
Combinatorial Optimisation: If c ∈ Rk and S ⊂ Rk then
{
}
{
}
max c > x : x ∈ S = max c > x : x ∈ conv(S) .
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Optimise: Global Maximum
max {f (x, y ) : x, y ∈ [−1, 1]} .
...
...
1
Find the maximum on a Chebyshev tensor grid.
2
Constrained Newton iteration.
Combinatorial Optimisation: If c ∈ Rk and S ⊂ Rk then
{
}
{
}
max c > x : x ∈ S = max c > x : x ∈ conv(S) .
Moreover, for any S, T ⊂ Rk
{
}
{
}
max y > x : x ∈ S, y ∈ T = max y > x : x ∈ conv(S), y ∈ conv(T ) .
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Optimise: Global Maximum
C(y)
R(x)
If k = 1, 2, 3 then computing convex hull requires O(n log n)
operations.
If k ≥ 4 then it is more efficient to just compute the discrete
maximum in O(n2 ).
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Optimise: Global Maximum
S
C(y)
R(x)
If k = 1, 2, 3 then computing convex hull requires O(n log n)
operations.
If k ≥ 4 then it is more efficient to just compute the discrete
maximum in O(n2 ).
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Associate, illustrate and demonstrate
Let’s play. Demo time.
.
Alex Townsend
Chebfun2
.
.
.
.
.
. So many questions...
Is Chebfun2 a good starting point? Discussion group at 4:30
What about representations away from the rectangle?
Huybrechs at 11:30
Can we extend things to Partial Differential Equations?
Driscoll at 9am and Olver at 10am tomorrow
How to find roots, extrema and contours of bivariate
polynomials? Boyd now
.
Alex Townsend
Chebfun2
.
.
.
.
.
. References
M. Bebendorf, Hierarchical Matrices, Springer, 2008.
O. A. Carvajal, F. W. Chapman and K. O. Geddes, Hybrid
symbolic-numeric integration in multiple dimensions via
tensor-product series, ISSAC ’05 Proceedings, (2005),
pp. 84–91.
S. A. Goreinov, E. E. Tyrtyshnikov and N. L. Zamarashkin, A
theory of pseudo-skeleton approximations, Linear Algebra
Appl., 261 (1997), pp. 1–21.
.
Alex Townsend
Chebfun2
.
.
.
.
.
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Approximate
Singular
∑n Value Decomposition:
A = j=1 σj uj vj>
Karhunen–Loève
Expansion:
∑
f (x, y ) = ∞
σ
j=1 j ψj (y )φj (x)
Best rank-r approximation:
∑
kA − Ar k2F = nj=r +1 σj2 , where
∑
kAk2F = ni,j=1 |aij |2 .
Best
∫ ∫ rank-r2approximation:
∑
2
(f − fr ) = ∞
j=r +1 σj .
Compute truncated K-L expansion.
Singular Value Decomposition is expensive O(n3 )!
We use Adaptive Cross Approximation with complete pivoting
(k steps of Gaussian elimination with complete pivoting)
.
Alex Townsend
Chebfun2
.
.
.
.
.
.
. Analyse: Symmetric Analytic functions
.
Theorem (Little and Reade, 1984)
.
..
If f (x, y ) is symmetric, analytic, bounded by M and for each
y0 ∈ [−1, 1] the function f (x, y0 ) is analytically continuable to the
open Bernstein ellipse Eρ . Then we have
N
∑
2Mρ−N−1
f −
σ
φ
⊗
ψ
≤
.
j j
j
1−ρ
j=1
L2 ([−1,1]2 )
.
..
.
Bernstein ellipse of cos(10x)
b
a
ρ=a+b
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Analyse: Symmetric Analytic functions
f (x, y ) =
1
,
1 + γ(x 2 + y 2 )
2D Runge function.
Convergence of K−L and ACA
0
10
Relative error in L2
K−L
ACA
−5
10
−10
10
γ=1
γ=100
γ=10
−15
10
0
5
10
15
20
25
30
Terms in truncation
.
Alex Townsend
Chebfun2
.
.
.
.
.
. Analyse: Supergeometric convergence
100
∑
qk e −γ ((x−xk )
2 +(y −y )2
k
),
xk , yk ∈ [−1, 1]2 , arbitrary.
k=1
Convergence of K−L and ACA
0
10
K−L
ACA
Relative error in L2
f (x, y ) =
−5
10
−10
10
γ=100
γ=10
γ=1
−15
10
0
10
20
30
40
50
60
Terms in truncation
N
∑
f −
σ j φj ⊗ ψ j j=1
)
( N
≤ O e − 2 log N .
L2 ([−1,1]2 )
Alex Townsend
Chebfun2
.
.
.
.
.
.
. Analyse: Symmetric Differentiable functions
fk (x, y ) = φ3,k (kx − y k2 ) ∈ C 2k ,
Wendland’s CSRBFs
Convergence of K−L and ACA
0
10
Relative error in L2
K−L
ACA
φ3,0∈ C0
−5
10
φ3,1∈ C2
−10
10
φ3,3∈ C6
−15
10
0
50
100
150
200
Terms in truncation
N
∑
fk −
σ j φj ⊗ ψ j j=1
≤ O(N −2k+ 2 ), k ≥ 1.
1
L2
.
Alex Townsend
Chebfun2
.
.
.
.
.