advertisement

Linear Algebra - November 28 Topics: Projections, reflections, Rayleigh quotient, Courant Max/Min theorem (minimax principle), Cholesky decomposition, symmetric matrices, positive definite matrices. Problems: J07#1, S95#5, J97#2, J07#4, S94#5, J96#3, S02#4, S96#5. ———————————————————————————————– Projections The projection matrix P is the matrix such that P b is the orthogonal projection of b onto the column space of P , ie (b − P b) ⊥ P b. It can be calculated from the basis of vectors onto which it projects. • project onto a single vector a: P = aaT . aT a • project onto the column space of A: P = A(AT A)−1 AT . P • project onto orthonormal vectors {qi }: P = qi qiT . The following characterizes a projection: P 2 = P, P = P T Rayleigh Quotient R(x) = ⇔ P is a projection. xT Ax . xT x Rayleigh’s principle If A is symmetric, then minx6=0 R(x) = λ1 , where λ1 is the smallest eigenvalue. Minimax Principle (Know how to prove this!) Suppose A is symmetric with eigenvalues λ1 ≤ λ2 ≤ · · · ≤ λn . Let Sj denote any j-dimensional subspace of Rn . Then λj = min max Sj x∈Sj ,x6=0 Symmetric/Hermitian matrix • has real eigenvalues • has orthonormal eigenvectors 1 R(x). Positive definite matrices Each of the following is a necessary and sufficient condition for the real symmetric matrix A to be positive definite: • xT Ax > 0 for all x 6= 0. • All the eigenvalues of A satisfy λi > 0. • All the upper left submatrices Ak have positive determinants. • All the pivots (without row exchanges) satisfy di > 0. • There exists a matrix R with independent columns such that A = RT R. A matrix is positive semi -definite if 0 can be among its eigenvalues. In the above, all strict inequalities are replaced with non-strict inequalities. 2