The Poisson matrix and Kronecker
Products
Tom Lyche
University of Oslo
Norway
The Poisson matrix and Kronecker Products – p. 1/2
Plan for the day
Numerical solution of the Poisson problem
The finite difference scheme
Formulation as a matrix equation
Formulation in standard form Ax = b.
The Poisson matrix A is block tridiagonal
LU factorization of a block tridiagonal matrix
Solving the Poisson problem, complexity
Kronecker product
Relation to Poisson problem
Basic properties
Eigenvalues and eigenvectors
Eigenpairs for the Poisson matrix
The Poisson matrix and Kronecker Products – p. 2/2
Numerical solution of the Poisson Problem
u=0
u=0
−(u +u ) =f
xx yy
u=0
u=0
m ∈ N, h = 1/(m + 1),
vj,k ≈ u(jh, kh)
(m+1)h
j,k+1
(jh,kh)
kh
j-1,k
0
j,k
j+1,k
j,k-1
0
jh
(m+1)h
The Poisson matrix and Kronecker Products – p. 3/2
Discrete Equation
g ′′ (t) ≈
g(t − h) − 2g(t) + g(t + h)
h2
v j,k+1
vj+1,k
vj-1,k vj,k
vj,k-1
From differential equation for j, k = 1, 2, . . . , m
(−vj−1,k + 2vj,k − vj+1,k ) + (−vj,k−1 + 2vj,k − vj,k+1 ) = h2 fj,k
(1)
From boundary conditions
vj,k = 0 if j = 0, m + 1 or k = 0, m + 1
(2)
The Poisson matrix and Kronecker Products – p. 4/2
2
Matrix Equation T V + V T = h F


2 −1 v11


−1 2
v21

v11

 .
V :=  ..

vm1
v12
v22

+
v11
v21
v12

2

v22
−1


−1
f
 = 1  11
9 f21
2
f12

f22

T := T m = tridiagm (−1, 2, −1) ∈ Rm,m



· · · v1m
f11 · · · f1m



.. 
.. 
 ..
m,m
m,m
∈
R
F
:=
∈
R



. 
. 
 .
· · · vmm
fm1 · · · fmm
(1), (2)
TV + V T

jk
=
m
X
i=1
⇔ T V + V T = h2 F
T ji vik +
m
X
vji T ik = l.h.s in (1) = h2 (F )jk
i=1
The Poisson matrix and Kronecker Products – p. 5/2
2
Convert T V + V T = h F to Ax = b
1,3
2,3
3,3
1,2
2,2
3,2
1,1
2,1
3,1
>
7
8
9
4
5
6
1
2
3
x i in grid
v in grid
j,k
-v j,k+1
-v
-v j+1,k
j-1,k 4 vj,k
-vj,k-1
-x i+m
-->
-x i-1
4x i
-xi+1
-x i-m
4xi − xi−1 − xi+1 − xi−m − xi+m = bi
The Poisson matrix and Kronecker Products – p. 6/2
Linear system Ax = b
For m = 3 or n = 9 we have the following linear system:
4x1 −x2
−x4
= h2 f11
−x1 4x2 −x3
−x5
= h2 f21
−x2 4x3
−x6
= h2 f31
−x1
4x4 −x5
−x7
= h2 f12
−x2
−x4 4x5 −x6
−x8
= h2 f22
−x3
−x5 4x6
−x9 = h2 f32
−x4
4x7 −x8
= h2 f13
−x5
−x7 4x8 −x9 = h2 f23
−x6
−x8 4x9 = h2 f33
The Poisson matrix and Kronecker Products – p. 7/2
Blockstructure of A

4 −1
0

 −1
4 −1


 0 −1
4


 −1
0
0


A =  0 −1
0


0 −1
 0

 0
0
0


 0
0
0

0
0
0


A=

−1
0
0
0
0
0
0
−1
0
0
0
0
0 −1
0
0
0
−1
0 −1
0
0
0
0
−1
4
0 −1
0
−1
−1
0
0
4 −1
0
0
4
−1
0
0
4 −1
−1
0
0 −1
0 −1
0 −1
−I
0
−I
T + 2I
−I
0
−I
T + 2I
T + 2I
4
−1
4

























Block tridiagonal
The Poisson matrix and Kronecker Products – p. 8/2
LU -factorization of a tridiagonal matri

d1
c1

a2







d2
..
.

c2
..
.
..
an−1
dn−1
.
an



1
 
 
 l2
=
 
 

cn−1 

dn
1
..
.
..
.
ln







1
u1
c1
..
.

..
.
un−1



.

cn−1 

un
(3)
Note that L has ones on the diagonal, and that A and U have the same
super-diagonal.
By equating entries in (3) we find
d1 = u1 ,
ak = lk uk−1 ,
dk = lk ck−1 + uk ,
k = 2, 3, . . . , n.
The Poisson matrix and Kronecker Products – p. 9/2
The Algorithm
A = LU (LU -factorization)
Ly = b (forward substitution)
U x = y (backward substituion)
u1 = d1 ,
lk =
y 1 = b1 ,
xn = yn /dn ,
ak
uk−1 ,
uk = dk − lk ck−1 ,
yk = bk − lk yk−1 ,
xk = (yk − ck xk+1 )/uk ,
k = 2, 3, . . . , n.
k = 2, 3, . . . , n,
k = n − 1, . . . , 2, 1.
This process is well defined if A is strictly diagonally
dominant.
The number of arithmetic operations (flops) is O(n).
The Poisson matrix and Kronecker Products – p. 10/2
LU of Block Tridiagonal Matrix
 D1
A2


C1
D2
C2
..
..
.
.
..

.
Am−1 D m−1 C m−1
Am
Dm
U 1 = D1 ,
Lk = Ak U −1
k−1 ,

 
=
  U1
1
L2 1
.. ..  
. .
C1
..
.
..
.
U m−1 C m−1
Um
Lm 1
U k = D k − Lk C k−1 ,

.
k = 2, 3, . . . , m. (4)
To solve the system Ax = b we partition b conformally with A in the form
bT = [bT1 , . . . , bTm ]. The formulas for solving Ly = b and U x = y are as
follows:
y 1 = b1 ,
xm =
−1
Um
ym ,
y k = bk − Lk y k−1 ,
xk =
U −1
k (y k
k = 2, 3, . . . , m,
− C k xk+1 ),
k = m − 1, . . . , 2, 1.
(5)
The solution is then xT = [xT1 , . . . , xTm ].
The Poisson matrix and Kronecker Products – p. 11/2
Poisson Problem
The discrete Poisson problem can be solved writing it as a block
tridiagonal matrix.
We can also use Cholesky factorization of a band matrix.
√
The matrix has bandwidth d = n.
Cholesky requires O(2nd2 ) = O(2n2 ) flops.
The work using block tridiagonal solver is also O(n2 ).
Need better methods (next time).
The Poisson matrix and Kronecker Products – p. 12/2
Kronecker Product
Definition 1. For any positive integers p, q, r, s we define the Kronecker product of two
matrices A
∈ Rp,q and B ∈ Rr,s as a matrix C ∈ Rpr,qs given in block form as

Ab1,1

 Ab2,1

C=
..

.

Abr,1
Ab1,2
Ab2,2
..
.
Abr,2
···
Ab1,s
···
Ab2,s
···
Abr,s
We denote the Kronecker product of A and B by C
..
.




.


= A ⊗ B.
Defined for rectangular matrices of any dimension
# of rows(columns) = product of # of rows(columns) in A and B
The Poisson matrix and Kronecker Products – p. 13/2
Kronecker Product Example


2 −1

,
T =
−1 2

T ⊗I =

I ⊗T =
T
0
2I
−I



1

I=
0


2 −1
0
0
2
0
0
2

0 


−1 

2

 −1
0
=

 0
T

0


0

1
0 −1
2
0 −1

 0
2
−I
=

 −1
2I
0

0 −1
0
2
0
0


−1 


0 

2
The Poisson matrix and Kronecker Products – p. 14/2
Poisson Matrix = T ⊗ I + I ⊗ T


A=


T
T

2I
−I
 
 +  −I 2I
 
T
0 −I
0


−I 
=T ⊗I +I ⊗T
2I
∈ Rr,r , B ∈ Rs,s and I k be the
identity matrix of order k . The sum A ⊗ I s + I r ⊗ B is known as the Kronecker sum of
A and B .
Definition 2. Let for positive integers r, s, k , A
The Poisson matrix is the Kronecker sum of T with itself.
The Poisson matrix and Kronecker Products – p. 15/2
Matrix equation ↔ Kronecker
Given A ∈ Rr,r , B ∈ Rs,s , F ∈ Rr,s . Find V ∈ Rr,s such that
AV B T = F
For B ∈ Rm,n define


b
 1
 
 b2 
mn

vec(B) := 
 ..  ∈ R ,
.
 
bn


b
 1j 


 b2j 

bj = 
 .. 
 . 


bmj
jth column
Lemma 1.
AV B T = F
⇔
(A ⊗ B) vec(V ) = vec(F )
(6)
AV + V B T = F
⇔
(A ⊗ I s + I r ⊗ B) vec(V ) = vec(F ).
(7)
The Poisson matrix and Kronecker Products – p. 16/2
Proof
We partition V , F , and B T by columns as V = (v 1 , . . . , v s ),
F = (f 1 , . . . , f s ) and B T = (b1 , . . . , bs ). Then we have
(A ⊗ B) vec(V ) = vec(F )

Ab11

 .
⇔  ..

Abs1
X
···
Ab1s
..
.
···
Abss
   
v
f
  1  1
  ..   .. 
 .  =  . 
   
vs
fs
bij vj = fi , i = 1, . . . , m
⇔
A
⇔
[AV b1 , . . . , AV bs ] = F
j
⇔
AV B T = F .
This proves (6)
The Poisson matrix and Kronecker Products – p. 17/2
Proof Continued
(A ⊗ I s + I r ⊗ B) vec(V ) = vec(F )
⇔
(AV I Ts + I r V B T ) = F
⇔
AV + V B T = F .
This gives a slick way to derive the Poisson matrix. Recall
AV + V B T = F
⇔
(A ⊗ I s + I r ⊗ B) vec(V ) = vec(F )
Therefore,
T V + V T = h2 F ⇔ (T ⊗ I + I ⊗ T ) vec(V ) = h2 vec(F )
The Poisson matrix and Kronecker Products – p. 18/2
Mixed Product Rule
(A ⊗ B)(C ⊗ D) = (AC) ⊗ (BD)
Proof





If B ∈ Rr,t and D ∈ Rt,s for some integers r, t then
Ab1,1
..
.
···
Ab1,t
..
.
Abr,1
···
Abr,t





Cd1,1
..
.
···
Cd1,s
..
.
Cdt,1
···
Cdt,s


 
 
=
 
E 1,1
..
.
···
E 1,s
..
.
E r,1
···
E r,s



,

where for all i, j
E i,j =
t
X
k=1
bi,k dk,j AC = (AC)(BD)i,j = ((AC) ⊗ (BD))i,j .
where in the last formula i, j refers to the ij-block in the Kronecker
product.
The Poisson matrix and Kronecker Products – p. 19/2
Properties of Kronecker Products
The usual arithmetic rules hold. Note however that
(A ⊗ B)T = AT ⊗ B T
(A ⊗ B)−1 = A−1 ⊗ B −1 if A and B are nonsingular
A ⊗ B 6= B ⊗ A
(A ⊗ B)(C ⊗ D) = (AC) ⊗ (BD) (mixed product rule)
The mixed product rule is valid for any matrices as long as the products
AC and BD are defined.
The Poisson matrix and Kronecker Products – p. 20/2
Eigenvalues and Eigenvectors
The Kronecker product of two vectors u ∈ Rp and v ∈ Rr is a vector
T
T
pr
T
u ⊗ v ∈ R given by u ⊗ v = u v1 , . . . , u vr Suppose now A ∈ Rr,r
and B ∈ Rs,s and
Aui = λi ui ,
then for i = 1, . . . , r,
i = 1, . . . , r,
Bv j = µj v j ,
j = 1, . . . , s,
j = 1, . . . , s
(A ⊗ B)(ui ⊗ v j ) = λi µj (ui ⊗ v j ),
(A ⊗ I s + I r ⊗ B)(ui ⊗ v j ) = (λi + µj )(ui ⊗ v j ).
(8)
(9)
Thus the eigenvalues of a Kronecker product(sum) are the products
(sums) of the eigenvalues of the factors. The eigenvectors of a Kronecker
product(sum) are the products of the eigenvectors of the factors.
The Poisson matrix and Kronecker Products – p. 21/2
Proof of Eigen-formulae
This follows directly from the mixed product rule. For (8)
(A ⊗ B)(ui ⊗ v j ) = (Aui ) ⊗ (Bv j ) = (λi ui ) ⊗ (µj v j ) = (λi µj )(ui ⊗ v j ).
From (8)
(A ⊗ I s )(ui ⊗ v j ) = λi (ui ⊗ v j ),
and
(I r ⊗ B)(ui ⊗ v j ) = µj (ui ⊗ v j )
The result now follows by summing these relations.
The Poisson matrix and Kronecker Products – p. 22/2
The Poisson Matrix A = T ⊗ I + I ⊗ T
Recall eigenvalues and eigenvectors of the T matrix. Let h = 1/(m + 1).
1. We know that T sj = λj sj for j = 1, . . . , m, where
sj = (sin (jπh), sin (2jπh), . . . , sin (mjπh))T ,
(10)
jπh
).
λj = 2 − 2 cos (jπh) =4 sin (
2
(11)
2
2. The eigenvalues are distinct and the eigenvectors are orthogonal
sTj sk
1
δj,k ,
=
2h
j, k = 1, . . . , m.
(12)
The Poisson matrix and Kronecker Products – p. 23/2
Eigenvalues/-vectors A = T ⊗ I + I ⊗ T
1. We have Axj,k = λj,k xj,k for j, k = 1, . . . , m, where
xj,k = sj ⊗ sk ,
(13)
sj = (sin (jπh), sin (2jπh), . . . , sin (mjπh))T ,
λj,k
jπh
2 kπh
) + 4 sin (
).
= 4 sin (
2
2
2
(14)
(15)
2. The eigenvectors are orthogonal
xTj,k xp,q
1
= 2 δj,p δk,q ,
4h
j, k, p, q = 1, . . . , m.
(16)
3. A is symmetric AT = (T ⊗ I + I ⊗ T )T = T T ⊗ I T + I T ⊗ T T = A
4. A is positive definite (positive eigenvalues)
The Poisson matrix and Kronecker Products – p. 24/2
A = T ⊗ I + I ⊗ T for m = 2
 
√
sin( π3 )
3 1


s1 =
=
,
2π
2
1
sin( 3 )


π
λ1 = 4 sin2 ( ) = 1,
6

s2 = 
sin( 2π
3 )
sin( 4π
3 )
λ2 = 4 sin2 (

=
√


3 1 
2 −1
2π
)=3
6
Axij = µij xij , µij = λi + λj , xij = si ⊗ sj
i, j = 1, 2
µ11 = 2, µ12 = µ21 = 4, µ22 = 6.
3
[1, 1, 1, 1]T ,
4
3
s2 ⊗ s1 = [1, −1, 1, −1]T ,
4
s1 ⊗ s1 =
xTj,k xp,q
3
[1, 1, −1, −1]T ,
4
3
s2 ⊗ s2 = [1, −1, −1, 1]T
4
s1 ⊗ s2 =
9
= δj,p δk,q ,
4
j, k, p, q = 1, 2.
The Poisson matrix and Kronecker Products – p. 25/2
Check

 
2 −1
1



A=
⊗
−1 2
0

4

 −1


 −1

0

 
 

0
1 0
2 −1
+
⊗

1
0 1
−1 2
0
−1 −1
4
0
0 −1
4 −1
4
−1 −1
4 −1 −1
0
 
 
1
1
 
 
1
 1
 
 
  = 2 ,
1
 1
 
 
1
1

1


1


 
 
 −1



 1
4
0 −1   1 

 

  = 4 
 −1
 −1
−1
0
4
−1

 
 
0 −1 −1
4
−1
−1
etc.
The Poisson matrix and Kronecker Products – p. 26/2
Summary
Studied the Poisson matrix A
Shown that A can be written as a Kronecker sum
studied properties of general Kronecker products and Kronecker
sums
used this to derive properties of A
The Poisson matrix and Kronecker Products – p. 27/2