Gaussian Elimination and LU Decomposition1
A lower triangular matrix has elements such that ci,j = 0 for i < j.
The LU Theorem is that a square matrix A of order n, made of submatrices Ak with
det(Ak)  0 for k=1,…,n-1 has a unique L = (mij), with mii = 1 and a unique U = (uij) such
that LU = A and det(A) = uii
Proof by induction a11 = 1 u11 and det(A) = u11. Assume that theorem is true for n = k-1. Fon n=k
A
A   k 1
 r
c 

akk 
where r is a row of k-1 components and c is a column of k=1 components.
Write
0
U k 1 u 
L
L   k 1
, U  

 m 1
 0 uk ,k 
Then
 Lk 1U k 1
LU  
 mU k 1
Lk 1u 

mu  uk ,k 
The hypothesis is that
Lk 1u  c, mU k 1
Lk 1U k 1  Ak 1 , thus the requirement that LU=A is equivalent to
 r , mu  uk ,k  ak ,k
These can be determined in this order. Finally
k
det  A   det  L   det(U )  1  det U k 1   uk ,k   uk ,k
i 1
In a similar way one can prove that A=LDU, where L and U are upper and lower triangular matrices with 1’s on the diagonal.
Next, if A is symmetric, then A=LDU  U=LT. If A is positive definite then det(Ar)>0 for r = 1, 2, … ,n, and hence in the LDU
theorm one has all dr,r > 0. If we let
D1/ 2
 d11

 0

 0
0
0
0 

0 

d nn 
1/2
Then G=LD , and A = GGT
Corollary. If A is a symmetric positive definite matrix, then it can be decomposed
uniquely into GGT, where G is a lower triangular matrix with positive diagonal
components.
Skip to page 114
23. Positive Definite (Symmetric) Matrices
A=GGT
1
Computer Solution of Linear Algebraic Systems, George Forsythe and Cleve B. Moler, Prentice-Hall (1967) p.27
 a11

a21
a
 31
a12
a22
a23
 g112

  g 21 g11
g g
 31 11
a13   g11
 
a23    g 21
a33   g31
0
g 22
g 32
g11 g12
g12 g 21  g 22 2
0   g11

0  0
g 33   0
g12
g 22
0
g13 

g 23 
g 33 



2
g31 g13  g 32 g 23  g 33 
g11 g13
g 21 g13  g 22 g 23
g31 g12  g32 g 22
If a is on the diagonal
a j , j  g 2j ,1  g 2j ,2 
 g 2j , j Note that this puts a bound on the maximum size of gjj. gjj < ajj.
If a is below the diagonal
ai , j  gi ,1 g j ,1  gi ,2 g j ,2 
 gi , j g j , j
The procedure is to
A.
1.
g11  a11
2.
a21  g 2,1 g1,1  g 21  a21 / g11
3.
a31  g31 g11  g31  a31 / g11
B.
1.
2
a22  g12 g21  g22
 g22  a22  g12 g 21
The algorithm is
For j := 1 step 1 until n do
j 1
Begin gjj := sqrt(ajj-
g
k 1
2
j ,k
);  note that this implies that there will be an advantage to starting with the largest a ii in the 1
position. Reorder the array.htm .doc– slowed robmin convergenge down.
For i := j+1 step 1 until n do
gi , j
j 1

gi ,k g j ,k


k

1
:  ai , j 
g j, j





;



end
Notice that the elements of A above the diagonal are not involved because of the symmetry.