Linear Algebra - November 28
Topics: Projections, reflections, Rayleigh quotient, Courant Max/Min theorem (minimax principle), Cholesky decomposition, symmetric matrices, positive
definite matrices.
Problems:
J07#1, S95#5, J97#2, J07#4, S94#5, J96#3, S02#4, S96#5.
———————————————————————————————–
Projections The projection matrix P is the matrix such that P b is the orthogonal projection of b onto the column space of P , ie (b − P b) ⊥ P b. It can
be calculated from the basis of vectors onto which it projects.
• project onto a single vector a: P =
aaT
.
aT a
• project onto the column space of A: P = A(AT A)−1 AT .
P
• project onto orthonormal vectors {qi }: P = qi qiT .
The following characterizes a projection:
P 2 = P, P = P T
Rayleigh Quotient
R(x) =
⇔
P is a projection.
xT Ax
.
xT x
Rayleigh’s principle If A is symmetric, then minx6=0 R(x) = λ1 , where λ1 is
the smallest eigenvalue.
Minimax Principle (Know how to prove this!) Suppose A is symmetric with
eigenvalues λ1 ≤ λ2 ≤ · · · ≤ λn . Let Sj denote any j-dimensional subspace of
Rn . Then
λj = min
max
Sj x∈Sj ,x6=0
Symmetric/Hermitian matrix
• has real eigenvalues
• has orthonormal eigenvectors
1
R(x).
Positive definite matrices Each of the following is a necessary and sufficient
condition for the real symmetric matrix A to be positive definite:
• xT Ax > 0 for all x 6= 0.
• All the eigenvalues of A satisfy λi > 0.
• All the upper left submatrices Ak have positive determinants.
• All the pivots (without row exchanges) satisfy di > 0.
• There exists a matrix R with independent columns such that A = RT R.
A matrix is positive semi -definite if 0 can be among its eigenvalues. In the
above, all strict inequalities are replaced with non-strict inequalities.
2