Chapter 7 Eigenvalues and Eigenvectors 7.1 Eigenvalues and Eigenvectors 7.2 Diagonalization 7.3 Symmetric Matrices and Orthogonal Diagonalization Elementary Linear Algebra R. Larsen et al. (6 Edition) 投影片設計編製者 淡江大學 電機系 翁慶昌 教授 7.1 Eigenvalues and Eigenvectors Eigenvalue problem: If A is an nn matrix, do there exist nonzero vectors x in Rn such that Ax is a scalar multiple of x? Eigenvalue and eigenvector: Geometrical Interpretation A:an nn matrix :a scalar x: a nonzero vector in Rn Eigenvalue Ax x Eigenvector Elementary Linear Algebra: Section 7.1, pp.421-422 1/53 Ex 1: (Verifying eigenvalues and eigenvectors) 2 0 A 0 1 1 0 x1 x2 0 1 Eigenvalue 2 0 1 2 1 Ax1 2 2 x1 0 1 0 0 0 Eigenvector Eigenvalue 2 0 0 0 0 Ax2 1 (1) x2 0 1 1 1 1 Eigenvector Elementary Linear Algebra: Section 7.1, p.423 2/53 Thm 7.1: (The eigenspace of A corresponding to ) If A is an nn matrix with an eigenvalue , then the set of all eigenvectors of together with the zero vector is a subspace of Rn. This subspace is called the eigenspace of . Pf: x1 and x2 are eigenvectors corresponding to (i.e. Ax1 x1, Ax2 x2 ) (1) A( x1 x2 ) Ax1 Ax2 x1 x2 ( x1 x2 ) (i.e. x1 x2 is an eigenvecto r correspond ing to λ) (2) A(cx1 ) c( Ax1 ) c(x1 ) (cx1 ) (i.e. cx1 is an eigenvecto r correspond ing to ) Elementary Linear Algebra: Section 7.1, p.424 3/53 Ex 3: (An example of eigenspaces in the plane) Find the eigenvalues and corresponding eigenspaces of 1 0 A 0 1 Sol: If v ( x, y ) 1 0 x x Av 0 1 y y For a vector on the x-axis Eigenvalue 1 1 1 0 x x x 0 1 0 0 10 Elementary Linear Algebra: Section 7.1, p.425 4/53 For a vector on the y-axis Eigenvalue 2 1 1 0 0 0 0 0 1 y y 1 y Geometrically, multiplying a vector (x, y) in R2 by the matrix A corresponds to a reflection in the y-axis. The eigenspace corresponding to 1 1 is the x-axis. The eigenspace corresponding to 2 1 is the y-axis. Elementary Linear Algebra: Section 7.1, p.425 5/53 Thm 7.2: (Finding eigenvalues and eigenvectors of a matrix AMnn ) Let A is an nn matrix. (1) An eigenvalue of A is a scalar such that det( I A) 0 . (2) The eigenvectors of A corresponding to are the nonzero solutions of (I A) x 0 . Note: Ax x (I A) x 0 (homogeneous system) If (I A) x 0 has nonzero solutions iff det( I A) 0. Characteristic polynomial of AMnn: det(I A) (I A) n cn 1n 1 c1 c0 Characteristic equation of A: det( I A) 0 Elementary Linear Algebra: Section 7.1, p.426 6/53 Ex 4: (Finding eigenvalues and eigenvectors) 2 12 A 1 5 Sol: Characteristic equation: det( I A) 2 12 1 5 2 3 2 ( 1)( 2) 0 1, 2 Eigenvalue s : 1 1, 2 2 Elementary Linear Algebra: Section 7.1, p.426 7/53 3 12 x1 0 (1)1 1 (1I A) x 1 4 x2 0 x1 4t 4 3 12 1 4 ~ t , t 0 1 4 0 0 x2 t 1 4 12 x1 0 (2)2 2 (2 I A) x 1 3 x2 0 x1 3t 3 4 12 1 3 ~ t , t 0 1 3 0 0 x2 t 1 Check : Ax x i Elementary Linear Algebra: Section 7.1, p.426 8/53 Ex 5: (Finding eigenvalues and eigenvectors) Find the eigenvalues and corresponding eigenvectors for the matrix A. What is the dimension of the eigenspace of each eigenvalue? 2 1 0 A 0 2 0 0 0 2 Sol: Characteristic equation: 2 1 0 I A 0 2 0 ( 2)3 0 0 0 2 Eigenvalue: 2 Elementary Linear Algebra: Section 7.1, p.428 9/53 The eigenspace of A corresponding to 2: 0 1 0 x1 0 (I A) x 0 0 0 x2 0 0 0 0 x3 0 0 1 0 0 1 0 x1 s 1 0 0 0 0 ~ 0 0 0 x2 0 s 0 t 0, s, t 0 0 0 0 0 0 0 x3 t 0 1 1 0 s 0 t 0 s, t R : the eigenspace of A correspond ing to 2 0 1 Thus, the dimension of its eigenspace is 2. Elementary Linear Algebra: Section 7.1, p.428 10/53 Notes: (1) If an eigenvalue 1 occurs as a multiple root (k times) for the characteristic polynominal, then 1 has multiplicity k. (2) The multiplicity of an eigenvalue is greater than or equal to the dimension of its eigenspace. Elementary Linear Algebra: Section 7.1, p.428 11/53 Ex 6:Find the eigenvalues of the matrix A and find a basis for each of the corresponding eigenspaces. 1 0 0 0 0 1 5 10 A 1 0 2 0 3 1 0 0 Sol: Characteristic equation: 1 0 0 0 0 1 5 10 I A 1 0 2 0 1 0 0 3 ( 1) 2 ( 2)( 3) 0 Eigenvalue s : 1 1, 2 2, 3 3 Elementary Linear Algebra: Section 7.1, p.429 12/53 (1)1 1 0 0 1 1 0 0 (1I A) x 1 1 0 1 0 5 10 0 ~ 0 1 0 0 0 0 2 0 0 0 0 x1 0 0 5 10 x2 0 0 1 0 x3 0 0 0 2 x4 0 0 0 2 x1 2t 0 2 0 1 2 x2 s 1 0 s t , s, t 0 0 0 0 x3 2t 0 2 0 0 0 x4 t 0 1 0 0 0 2 1 0 , is a basis for the eigenspace 0 2 of A corresponding to 1 0 1 Elementary Linear Algebra: Section 7.1, p.429 13/53 1 0 (2 I A) x 1 1 0 1 0 1 0 0 0 1 5 10 0 1 ~ 1 0 0 0 0 0 1 0 0 1 0 0 (2)2 2 0 x1 0 1 5 10 x2 0 0 0 0 x3 0 0 0 1 x4 0 0 0 x1 0 0 5 0 x2 5t 5 t , t 0 0 1 x3 t 1 0 0 x4 0 0 0 0 0 5 is a basis for the eigenspace 1 of A corresponding to 2 0 Elementary Linear Algebra: Section 7.1, p.429 14/53 (3)3 3 2 0 1 1 0 2 0 0 2 0 (3I A) x 1 1 0 0 1 0 5 10 0 1 ~ 1 0 0 0 0 0 0 0 0 2 0 0 0 0 1 0 0 x1 0 5 10 x2 0 1 0 x3 0 0 0 x4 0 0 x1 0 0 5 x2 5t 5 t , t0 0 x3 0 0 0 x4 t 1 0 0 5 is a basis for the eigenspace 0 of A corresponding to 3 1 Elementary Linear Algebra: Section 7.1, p.429 15/53 Thm 7.3: (Eigenvalues of triangular matrices) If A is an nn triangular matrix, then its eigenvalues are the entries on its main diagonal. Ex 7: (Finding eigenvalues for diagonal and triangular matrices) 1 0 0 0 0 2 0 0 0 2 0 0 0 (a) A 1 1 0 (b) A 0 0 0 0 0 0 0 0 4 0 5 3 3 0 0 0 0 3 2 0 0 Sol: ( a ) I A 1 1 0 ( 2)( 1)( 3) 5 3 3 1 2, 2 1, 3 3 (b) 1 1, 2 2, 3 0, 4 4, 5 3 Elementary Linear Algebra: Section 7.1, p.430 16/53 Eigenvalues and eigenvectors of linear transformations: A number is called an eigenvalue of a linear tra nsformatio n T : V V if there is a nonzero vector x such that T (x) x. The vector x is called an eigenvecto r of T correspond ing to , and the setof all eigenvecto rs of (with the zero vector) is called the eigenspace of . Elementary Linear Algebra: Section 7.1, p.431 17/53 Ex 8: (Finding eigenvalues and eigenspaces) Find the eigenvalue s and correspond ing eigenspace s 1 3 0 A 3 1 0 . 0 0 2 Sol: 1 I A 3 0 3 1 0 0 0 ( 2) 2 ( 4) 2 eigenvalue s : 1 4, 2 2 The eigenspace s for these two eigenvalue s are as follows. B1 {(1, 1, 0)} Basis for 1 4 B2 {(1, 1, 0), (0, 0, 1)} Elementary Linear Algebra: Section 7.1, p.431 Basis for 2 2 18/53 Notes: (1) Let T:R 3 R 3 be the linear transformatio n whose standard matrix is A in Ex. 8, and let B' be the basis of R 3 made up of three linear independen t eigenvecto rs found in Ex. 8. Then A' , the matrix of T relative to the basis B' , is diagonal. 0 4 0 A' 0 2 0 B ' {(1, 1, 0), (1, 1, 0), (0, 0, 1)} 0 0 2 Eigenvecto rs of A Eigenvalue s of A (2) The main diagonal entries of the matrix A' are the eigenvalue s of A. Elementary Linear Algebra: Section 7.1, p.431 19/53 Keywords in Section 7.1: eigenvalue problem: 特徵值問題 eigenvalue: 特徵值 eigenvector: 特徵向量 characteristic polynomial: 特徵多項式 characteristic equation: 特徵方程式 eigenspace: 特徵空間 multiplicity: 重數 20/53 7.2 Diagonalization Diagonalization problem: For a square matrix A, does there exist an invertible matrix P such that P-1AP is diagonal? Diagonalizable matrix: A square matrix A is called diagonalizable if there exists an invertible matrix P such that P-1AP is a diagonal matrix. (P diagonalizes A) Notes: (1) If there exists an invertible matrix P such that B P 1 AP, then two square matrices A and B are called similar. (2) The eigenvalue problem is related closely to the diagonalization problem. Elementary Linear Algebra: Section 7.2, p.435 21/53 Thm 7.4: (Similar matrices have the same eigenvalues) If A and B are similar nn matrices, then they have the same eigenvalues. Pf: A and B are similar B P 1 AP I B I P 1 AP P 1IP P 1 AP P 1 (I A) P P 1 I A P P 1 P I A P 1 P I A I A A and B have the same characteristic polynomial. Thus A and B have the same eigenvalues. Elementary Linear Algebra: Section 7.2, p.436 22/53 Ex 1: (A diagonalizable matrix) 1 3 0 A 3 1 0 0 0 2 Sol: Characteristic equation: 1 3 0 I A 3 1 0 ( 4)( 2) 2 0 0 0 2 Eigenvalue s : 1 4, 2 2, 3 2 1 (1) 4 Eigenvecto r : p1 1 (See p.403 Ex.5) 0 Elementary Linear Algebra: Section 7.2, p.436 23/53 (2) 2 Eigenvector : p2 P [ p1 p2 Notes: (1) P [ p2 p1 (2) P [ p2 p3 1 p3 ] 1 0 1 p3 ] 1 0 1 p1 ] 1 0 Elementary Linear Algebra: Section 7.2, p.436 1 0 1, p3 0 (See p.403 Ex.5) 0 1 1 0 0 4 0 1 0 P 1 AP 0 2 0 0 0 2 0 1 1 0 2 0 0 1 0 P 1 AP 0 4 0 0 0 2 0 1 0 1 2 0 0 0 1 P 1 AP 0 2 0 0 1 0 0 4 24/53 Thm 7.5: (Condition for diagonalization) An nn matrix A is diagonalizable if and only if it has n linearly independent eigenvectors. Pf: () A is diagonaliz able there exists an invertible P s.t. D P 1 AP is diagonal Let P [ p1 p2 pn ] and D diag (1 , 2 ,, n ) PD [ p1 p2 1 0 0 2 pn ] 0 0 0 0 n [1 p1 2 p2 n pn ] Elementary Linear Algebra: Section 7.2, p.437 25/53 AP A[ p1 p2 pn ] [ Ap1 Ap2 Apn ] AP PD Api i pi , i 1, 2,, n (i.e. the column vec tor pi of P are eigenvecto rs of A) P is invertible p1 , p2 ,, pn are linearly independen t. A has n linearly independen t eigenvecto rs. () A has n linearly independen t eigenvecto rs p1 , p2 , pn with correspond ing eigenvalue s 1 , 2 , n i.e. Api i pi , i 1, 2,, n Let P [ p1 p2 pn ] Elementary Linear Algebra: Section 7.2, p.438 26/53 AP A[ p1 [ Ap1 p2 pn ] Ap2 Apn ] [1 p1 2 p2 n pn ] 1 0 0 2 pn ] 0 0 0 0 [ p1 p2 PD n p1 , p1 ,, pn are linearly independen t P is invertible P 1 AP D A is diagonaliz able Note: If n linearly independent vectors do not exist, then an nn matrix A is not diagonalizable. Elementary Linear Algebra: Section 7.2, p.438 27/53 Ex 4: (A matrix that is not diagonalizable) Show that the following matrix is not diagonaliz able. 1 2 A 0 1 Sol: Characteristic equation: 1 2 I A ( 1) 2 0 0 1 Eigenvalue : 1 1 0 2 0 1 1 I A I A ~ Eigenvecto r : p1 0 0 0 0 0 A does not have two (n=2) linearly independent eigenvectors, so A is not diagonalizable. Elementary Linear Algebra: Section 7.2, p.439 28/53 Steps for diagonalizing an nn square matrix: Step 1: Find n linearly independent eigenvectors p1 , p2 ,, pn for A with corresponding eigenvalues 1 , 2 ,, n Step 2: Let P [ p1 p2 pn ] Step 3: 1 0 0 2 P 1 AP D 0 0 0 0 , where Api i pi , i 1, 2,, n n Note: The order of the eigenvalues used to form P will determine the order in which the eigenvalues appear on the main diagonal of D. Elementary Linear Algebra: Section 7.2, p.439 29/53 Ex 5: (Diagonalizing a matrix) 1 1 1 A 1 3 1 3 1 1 Find a matrix P such that P 1 AP is diagonal. Sol: Characteristic equation: 1 I A 1 3 1 1 3 1 ( 2)( 2)( 3) 0 1 1 Eigenvalue s : 1 2, 2 2, 3 3 Elementary Linear Algebra: Section 7.2, p.440 30/53 1 2 2 2 1 1 1 1 0 1 1I A 1 1 1 ~ 0 1 0 3 1 3 0 0 0 x1 t 1 1 x2 0 t 0 Eigenvecto r : p1 0 x3 t 1 1 1 1 0 14 3 1 2 I A 1 5 1 ~ 0 1 14 3 1 1 0 0 0 x1 14 t 1 1 x2 14 t 14 t 1 Eigenvecto r : p2 1 x3 t 4 4 Elementary Linear Algebra: Section 7.2, p.440 31/53 3 3 2 1 1 1 0 1 3I A 1 0 1 ~ 0 1 1 3 1 4 0 0 0 x1 t 1 1 x2 t t 1 Eigenvecto r : p3 1 x3 t 1 1 1 1 1 Let P [ p1 p2 p3 ] 0 1 1 1 4 1 2 0 0 P 1 AP 0 2 0 0 0 3 Elementary Linear Algebra: Section 7.2, p.440 32/53 Notes: k is a positive integer d1 0 (1) D 0 0 d2 0 d1k 0 0 0 k D d n 0 0 d 2k 0 0 0 d nk (2) D P 1 AP D k ( P 1 AP) k ( P 1 AP)( P 1 AP) ( P 1 AP) P 1 A( PP 1 ) A( PP 1 ) ( PP 1 ) AP P 1 AA AP P 1 Ak P Ak PD k P 1 Elementary Linear Algebra: Section 7.2, p.445 33/53 Thm 7.6: (Sufficient conditions for diagonalization) If an nn matrix A has n distinct eigenvalues, then the corresponding eigenvectors are linearly independent and A is diagonalizable. Elementary Linear Algebra: Section 7.2, p.442 34/53 Ex 7: (Determining whether a matrix is diagonalizable) 1 2 1 A 0 0 1 0 0 3 Sol: Because A is a triangular matrix, its eigenvalues are the main diagonal entries. 1 1, 2 0, 3 3 These three values are distinct, so A is diagonalizable. (Thm.7.6) Elementary Linear Algebra: Section 7.2, p.443 35/53 Ex 8: (Finding a diagonalizing matrix for a linear transformation) Let T:R 3 R 3 be the linear transformatio n given by T ( x1,x2 ,x3 ) ( x1 x2 x3 , x1 3 x2 x3 , 3 x1 x2 x3 ) Find a basis B for R 3 such that the matrix for T relative to B is diagonal. Sol: The standard matrix for T is given by 1 1 1 A T (e1 ) T (e2 ) T (e3 ) 1 3 1 3 1 1 From Ex. 5, there are three distinct eigenvalues 1 2, 2 2, 3 3 so A is diagonalizable. (Thm. 7.6) Elementary Linear Algebra: Section 7.2, p.443 36/53 Thus, the three linearly independent eigenvectors found in Ex. 5 p1 (1, 0, 1), p2 (1, 1, 4), p3 (1, 1, 1) can be used to form the basis B. That is B { p1 , p2 , p3} {( 1, 0, 1), (1, 1, 4), (1, 1, 1)} The matrix for T relative to this basis is D [T ( p1 )]B [T ( p2 )] B [T ( p3 )]B [ Ap1 ]B [ Ap2 ]B [ Ap3 ]B [1 p1 ]B [2 p2 ]B [3 p3 ]B 2 0 0 0 2 0 0 0 3 Elementary Linear Algebra: Section 7.2, p.443 37/53 Keywords in Section 7.2: diagonalization problem: 對角化問題 diagonalization: 對角化 diagonalizable matrix: 可對角化矩陣 38/53 7.3 Symmetric Matrices and Orthogonal Diagonalization Symmetric matrix: A square matrix A is symmetric if it is equal to its transpose: A AT Ex 1: (Symmetric matrices and nonsymetric matrices) 0 1 2 A 1 3 0 (symmetric) 2 0 5 4 3 B (symmetric) 3 1 3 2 1 C 1 4 0 (nonsymmetric) 1 0 5 Elementary Linear Algebra: Section 7.3, p.447 39/53 Thm 7.7: (Eigenvalues of symmetric matrices) If A is an nn symmetric matrix, then the following properties are true. (1) A is diagonalizable. (2) All eigenvalues of A are real. (3) If is an eigenvalue of A with multiplicity k, then has k linearly independent eigenvectors. That is, the eigenspace of has dimension k. Elementary Linear Algebra: Section 7.3, p.447 40/53 Ex 2: Prove that a symmetric matrix is diagonalizable. a c A c b Pf: Characteristic equation: a c I A 2 (a b) ab c 2 0 c b As a quadratic in , this polynomial has a discriminant of (a b) 2 4(ab c 2 ) a 2 2ab b 2 4ab 4c 2 a 2 2ab b 2 4c 2 ( a b ) 2 4c 2 0 Elementary Linear Algebra: Section 7.3, p.448 41/53 (1) (a b) 2 4c 2 0 a b, c 0 a 0 A is a matrix of diagonal. 0 a (2) (a b) 2 4c 2 0 The characteristic polynomial of A has two distinct real roots, which implies that A has two distinct real eigenvalues. Thus, A is diagonalizable. Elementary Linear Algebra: Section 7.3, p.448 42/53 Orthogonal matrix: A square matrix P is called orthogonal if it is invertible and P 1 PT Ex 4: (Orthogonal matrices) 0 1 0 1 1 T (a) P is orthogonal because P P . 1 0 1 0 3 5 0 (b) P 0 1 4 0 5 4 3 0 5 5 1 T 0 is orthogonal because P P 0 1 4 3 0 5 5 Elementary Linear Algebra: Section 7.3, p.449 4 5 0 . 3 5 43/53 Thm 7.8: (Properties of orthogonal matrices) An nn matrix P is orthogonal if and only if its column vectors form an orthogonal set. Elementary Linear Algebra: Section 7.3, p.450 44/53 Ex 5: (An orthogonal matrix) 13 2 P 5 2 3 5 0 5 3 5 2 3 1 5 4 3 5 2 3 Sol: If P is a orthogonal matrix, then P 1 PT PPT I 1 32 T PP 5 3 25 2 3 1 5 4 3 5 13 2 0 3 5 2 3 5 3 2 3 2 5 1 5 0 2 3 5 4 3 5 5 3 5 1 0 0 I 0 1 0 0 0 1 13 23 23 Let p1 25 , p2 15 , p3 0 325 345 3 5 5 Elementary Linear Algebra: Section 7.3, p.451 45/53 produces p1 p2 p1 p3 p2 p3 0 p1 p2 p3 1 { p1 , p2 , p3} is an orthonorma l set. Elementary Linear Algebra: Section 7.3, p.451 46/53 Thm 7.9: (Properties of symmetric matrices) Let A be an nn symmetric matrix. If 1 and 2 are distinct eigenvalues of A, then their corresponding eigenvectors x1 and x2 are orthogonal. Elementary Linear Algebra: Section 7.3, p.452 47/53 Ex 6: (Eigenvectors of a symmetric matrix) 0 1 Show that any two eigenvecto rs of A 1 0 correspond ing to distinct eigenvalue s are orthogonal . Sol: Characteristic function 3 1 I A 2 6 8 ( 2)( 4) 0 1 3 Eigenvalue s : 1 2, 2 4 1 1 1 1 1 (1) 1 2 1I A ~ x 1 s , s 0 1 1 0 0 1 1 1 1 1 1 (2) 2 4 2 I A ~ x 2 t , t 0 1 1 0 0 1 s t x 1 x 2 st st 0 x 1 and x 2 are orthogonal . s t Elementary Linear Algebra: Section 7.3, p.452 48/53 Thm 7.10: (Fundamental theorem of symmetric matrices) Let A be an nn matrix. Then A is orthogonally diagonalizable and has real eigenvalue if and only if A is symmetric. Orthogonal diagonalization of a symmetric matrix: Let A be an nn symmetric matrix. (1) Find all eigenvalues of A and determine the multiplicity of each. (2) For each eigenvalue of multiplicity 1, choose a unit eigenvector. (3) For each eigenvalue of multiplicity k2, find a set of k linearly independent eigenvectors. If this set is not orthonormal, apply GramSchmidt orthonormalization process. (4) The composite of steps 2 and 3 produces an orthonormal set of n eigenvectors. Use these eigenvectors to form the columns of P. The matrix P 1 AP PT AP D will be diagonal. Elementary Linear Algebra: Section 7.3, p.453 49/53 Ex 7: (Determining whether a matrix is orthogonally diagonalizable) 1 1 1 A1 1 0 1 1 1 1 5 2 1 A2 2 1 8 1 8 0 Symmetric matrix Orthogonally diagonalizable 3 2 0 A3 2 0 1 0 0 A4 0 2 Elementary Linear Algebra: Section 7.3, p.454 50/53 Ex 9: (Orthogonal diagonalization) Find an orthogonal matrix P that diagonaliz es A. 2 2 2 A 2 1 4 2 4 1 Sol: (1) I A ( 3)2 ( 6) 0 1 6, 2 3 (has a multiplici ty of 2) v1 (2) 1 6, v1 (1, 2, 2) u1 ( 13 , 32 , 23 ) v1 (3) 2 3, v2 (2, 1, 0), v3 (2, 0, 1) Linear Independent Elementary Linear Algebra: Section 7.3, p.455 51/53 Gram-Schmidt Process: v3 w2 w2 v2 (2, 1, 0), w3 v3 w2 ( 52 , 54 , 1) w2 w2 w3 w2 2 1 u2 ( 5 , 5 , 0), u3 ( 325 , 3 4 5 , 3 5 5 ) w2 w3 13 (4) P [ p1 p2 p3 ] 32 23 2 5 1 5 0 4 3 5 5 3 5 2 3 5 6 0 0 P 1 AP P T AP 0 3 0 0 0 3 Elementary Linear Algebra: Section 7.3, p.456 52/53 Keywords in Section 7.3: symmetric matrix: 對稱矩陣 orthogonal matrix: 正交矩陣 orthonormal set: 單範正交集 orthogonal diagonalization: 正交對角化 53/53