7.3 Positive Definite Matrices
Definition
Very frequently in engineering and statistics applications,
we end up dealing with symmetric (& square) matrices
which have positive eigenvalues. These matrices are said
to be positive definite.
Theorem 1
If A is positive definite, then it is invertible and det A > 0.
Proof: A (n x n) with eigenvalues 1, 2,…, n
det A = 12…n > 0 (since all eigenvalues are pos)
Since det A≠ 0, A is invertible.
Theorem 2
A symmetric matrix A is positive definite iff XTAX > 0 for
every column X ≠ 0 in n.
Proof: let PTAP = D = diag(1, 2,…, n) where P-1=PT and
the i are the eigenvalues of A (since princ axes thm says that
symmetric A is orthogonally diagonalizable).
Let Y = PTX = [y1 y2…yn]T and we know A=PDP-1
so XTAX = XT PDP-1 X=YTDY=1y12+…+ nyn2 (*)
i>0 since A is pos def. Also, if Y=0, we have the
homogeneous system PTX=0 which only has the solution X=0
since P is invertible. But X≠0 so Y≠0. This implies that
some yi≠0, so XTAX > 0 (from * above)
Theorem 2 (cont)
Given XTAX >0 for every X≠0, we will select an X≠0 for
which each i≠0: let X = PEj ≠ 0 (where Ej is col j of In-- so
we have a col of P). Then Y = PTX = PTPEj=P-1PEj=Ej
So recall (*) XTAX = XT PDP-1 X=YTDY=1y12+…+ nyn2 >0
Becomes: j >0 which tells us that A is positive definite.
Example
If A = CTC where C is invertible, show that A is positive definite.
Solution: Try to show that XTAX>0 for all X≠0.
XTAX = XTCTCX=(CX)T(CX)=CX•CX=||CX||2>0: reason:
with C invertible, the only solution to the homogeneous system
CX = 0 is the trivial solution X=0, but X≠0, so CX ≠0, which
means ||CX||2>0.
Definition
Every positive definite matrix A can be factored as A=UTU,
where U is upper triangular with positive entries on the main
diagonal--called the Cholesky Decomposition (which is a
unique factorization).
We find the Cholesky Decomposition using elementary row
operations (as in the next example).
Cholesky Decomposition Algorithm
First be sure that your matrix A is positive definite. Then:
1. Add multiples of a row to lower rows in order to carry A to
an upper triangular matrix U1. (Only multiply the higher row
times a value.)
2. Divide the entries in each row by the square root of the
diagonal entry in the row.
You will now have U such that A=UTU. It gets messy to
verify this in the general case, but we can show it works for
various pos. def. matrices.
Example
Find the Cholesky Decomposition of
We only add multiples of a row to
rows below it.
2
4 3  
0 11  
 4  0

Can verify that UTU=A
4 3
A  

3
5


3 
2 
11 

2 
Definition
If A is (n x n), and 1r n, let rA be the (r x r) matrix obtained
from A by deleting the last (n - r) rows and columns. The
matrices 1A, 2A,…,nA = A are called the principle submatrices
of A.
1
a11
A  a11 , A  
a21
2
a11
a12 3
a
,
A

 21
a22 


a31
a12
a 22
a32
a13 
a23 

a33 

Theorem 3
If A is positive definite, so is rA for each r = 1,2,…,n
 A P
Y 
r
n
Proof: A  
,letY  0  ; X   
Q R
0 
r
0 < XTAX (since A is pos def)
X AX  Y
T
T
r A PY  T r
0
  Y AY
Q R0 
So YTrAY >0 so rA is positive definite.
Lemma
Let A be an (n x n) matrix for which det rA>0 for each
r=1,2,…,n. Then LA =U where L is (n x n) lower triangular
with 1’s on the diagonal, and U is (n x n) upper triangular with
positive entries on the diagonal.
Proof: (by induction).
n=1: A=[a], a>0, so let L=[1] and U=A.
n>1: let A=[aij] and note that a11=det1A>0. So A can be taken
by row operations that add multiples of row 1 to lower rows to:
a11
0

X 
The row operations to accomplish this are left-mult
B 

by elementary matrix that is lower triangular with
1’s on the diagonal. (verify)
Proof (cont)
Let L1 be the product of these elementary matrices, which will
also be lower triangular with 1’s on the main diagonal. (verify)
a11
L1 A  
 0
X 
B 

We could also write L1 and A in block form:
r L1
L1 A  
M
0 r A C  r L1 r A P

 

N D E   Q
R
Looking at the two versions of L1A if r2, we can see that
r
r
a11
L1 A (L1 A) 
0
r
r
X  a11
 

B  0
X  
r1 
B
Proof (cont)
r
r
a11
L1 A (L1 A) 
0
r
r
X  a11
 

B  0
X  
r1 
B
det(rL1rA)= a11det(r-1B)
Since det rL1=1 (since L1 is triangular w/ 1’s on diag),
det(rL1rA)= det(rL1) det( rA)= detrA
So a11det(r-1B)= detrA
Since detrA>0 (given) and a11>0 (given), det(r-1B)>0.
Proof (cont)
By induction, let L0B=U0 be as in the lemma.
X  a
1 0 
1 0 a
If L2= 0 L ,then : L L A  0 L 
 0
11

0

2
1

0
0
11
B  
X 
U0 

L2L1A is upper triangular with positive diagonal entries since U0
is upper triangular with pos entries on diag (given) and a11>0.
Since L2L1 is lower triangular with 1’s on the diagonal,
Let L = L2L1 and U = L2L1A which satisfies the condition.•
Theorem 4
The following conditions are equivalent for an (n x n) symmetric
matrix A:
1. A is positive definite.
2. det rA>0 for each r=1,2,…,n
3. A = UTU where U is upper triangular with positive diagonal
entries.
Proof: (1) (2): thm 3 says that if A is pos. def, then so is rA
for each r=1,2,…,n and thm 1 says that if A is pos def, then det
A>0. So done.
Proof (cont)
(2)(3): Let LA = U0 as in the previous lemma (i.e U0 is
upper triangular w/ pos entries on the diagonal, L is lower tri
w/ 1’s on diag.)
Let D = diag(d1,d2,…,dn) be the diagonal mtx w/ the same
diagonal as U0.
Then U1=D-1U0 is upper triangular w/ 1’s on the diagonal (we
can show this). Then also U0 = DU1
LA = U0  A =L-1U0 = L-1DU1
Also, A = AT = (L-1DU1)T = U1TDT(L-1)T = U1TD(LT)-1
Proof (cont)
So L-1DU1 = A = U1TD(LT)-1
So DU1=LU1TD(LT)-1
So DU1LT=LU1TD
So DU1LTD-1=LU1T
DU1LTD-1 is upper triangular since D is diagonal, U1 is upper
triangular, and LT is upper triangular.
LU1T is lower triangular w/ 1’s on main diag since both L and
U1T are lower tri with 1’s on the main diagonal (verify).
Since DU1LTD-1=LU1T they are upper and lower triang (so
diag) w/ 1’s on diag (thus we have In)
Proof (cont)
So LU1T=In  L-1 = U1T
Let D0=
diag( d1 ,..., dn )
So D02 = D.
Then U=D0-1U0 is upper triangular with positive diagonal
entries since D0-1 is diagonal w/ pos entries and U0 is upper
triangular w/ pos entries on diagonal.
And UTU =(D0-1U0)T(D0-1U0)=U0T (D0-1)(D0-1)U0
(D0-1)2 is diagonal w/ 1/di on diag which is D-1
So = U0TD-1U0=(D-1U0)TU0 = U1TU0=L-1U0=A
(done)
Proof (cont)
(3)(1) basically the same as in our first example.•
Also, proofs of the lemma and theorems in this section prove the
legitimacy of the Algorithm for the Cholesky Decomposition.