Chapter 7: Eigenvalues and Eigenvectors 1 Chapter 7: Eigenvalues and Eigenvectors SECTION A Introduction to Eigenvalues and Eigenvectors By the end of this section you will be able to understand what is meant by an eigenvalue and eigenvector determine eigevalues and eigenvectors prove properties of eigenvalues and eigenvectors This is an important chapter in linear algebra because it has many applications in the areas of physical sciences and engineering. This section is straightforward but it does rely on a number of topics in linear algebra such as matrices, determinants, vectors etc. You need to thoroughly know how to evaluate determinants to understand this chapter. A1 Definition of Eigenvalue and Eigenvector Before we define what is meant by an eigenvalue and eigenvector we do an example which involves these. Example 1 4 2 2 Let A and u then evaluate Au . 1 1 1 Solution. Multiplying the matrix and column vector we have 4 2 2 6 2 Au 3 1 1 1 3 1 What do you notice about your result? Well we have Au 3u . In geometric terms we have that the matrix A scalar multiplies the vector u by 3: y 3 2.5 3 2 1 2 1.5 u= 1 2 1 0.5 1 2 3 4 5 Fig 1 In general terms this can be written as (7.1) Au u where A is a square matrix, u is a column vector u 6 n x and (lambda) is a scalar. Can you think of a vector, u, which satisfies equation (7.1)? The zero vector u O . In this case we say we have the trivial solution. In this chapter we consider the non-trivial solutions, u O , and these solutions are powerful tools in linear algebra. For non-zero vector u the scalar is called an eigenvalue of the matrix A and the vector u is called an eigenvector belonging or corresponding to which satisfies Chapter 7: Eigenvalues and Eigenvectors 2 Au u . In most linear algebra literature the Greek letter lambda, , is used for eigenvalues. These terms eigenvalue and eigenvector are derived from the German word ‘eigenwerte’ which means ‘proper value.’ The word eigen is pronounced “i-gun”. Eigenvalues was initially developed in the field of differential equations by Jean D’ Alembert. Alembert 1717 to 1783 was a French mathematician and the illegitimate son of Madam Tencin and an army officer, Louis Destouches. His mother left him on steps of a local church but he was sent to a home for homeless children. His father recognised his son’s difficulties and placed him under the care of Madam Rousseau. He always thought of Rousseau as his mother. Fig 2 Alembert 1717 to 1783 However Alembert’s father died when he was only nine years old but his father’s family looked after his financial situation so that he could be educated. In 1735 Alembert graduated and thought a career in law would suit him but his real thirst and enthusiasm was for mathematics and did this in his spare time. Three years later he did qualify for an advocate but did not pursue a career in this field but chose mathematics instead. For most of his life he worked for the Paris Academy of Science and the French Academy. He was well known to have a short fuse and loved to argue with most of his contemporaries. Example 2 1 1 Let A . Verify the following: 2 4 1 (a) u is an eigenvector of the matrix A corresponding to the eigenvalue 1 2 . 1 1 (b) v is an eigenvector of the matrix A corresponding to the eigenvalue 2 3 . 2 Solution. (a) Multiplying the given matrix A and column vector u we have 1 1 1 2 1 Au 2 2 4 1 2 1 1 Thus u is an eigenvector of the matrix A belonging to 1 2 because Au 2u . 1 (b) Similarly we have 1 1 1 3 1 Av 3 2 4 2 6 2 1 Thus v is an eigenvector of the matrix A belonging to 2 3 because Av 3v . 2 Chapter 7: Eigenvalues and Eigenvectors 3 What do you notice about your results? A 2 by 2 matrix can have more than one eigenvalue and eigenvector. Note that we can have eigenvalues and eigenvectors for any square matrices such as 3 3 , 4 4 , 5 5 , etc which satisfy Au u . Example 3 5 0 0 0 Let A 9 4 1 and u 1 . Determine an eigenvalue of this matrix A. 2 6 2 1 Solution. Multiplying the matrix and column vector we have 5 0 0 0 0 0 Au 9 4 11 2 2 1 6 2 1 2 4 2 We have Au u where 2 . Hence 2 is an eigenvalue of the matrix A with an eigenvector u. A2 Characteristic Equation From the above equation (7.1) Au u we have Au I nu [ I nu u Multiplying by the identity keeps it the same] where I n is the n by n identity matrix. We can rewrite this as Au I nu O A I n u O Under what condition is the non-zero vector u a solution of this equation? By Proposition (2.30)?? of chapter 2 we have a non-zero vector u if and only if det A I n 0 This is an important equation and is called the characteristic equation. We give this a reference number (7.2) det A I 0 We drop the subscript n because we assume the identity matrix I is of appropriate size. In some linear algebra texts you will see (7.2) written as det I A 0 . Both of these are equivalent so it does not matter which we apply to find the eigenvalues. This second approach does have the advantage of not expanding brackets like 1 2 But if you remove 2 minus signs then it should easier to expand this: 1 2 1 2 1 2 Advantage of using (7.2) is that you don’t need to subtract the matrix A, it just boils down to subtracting along the leading diagonal. We use this characteristic equation to find the eigenvalues and corresponding eigenvectors. The procedure for this is as follows: 1. Solve the characteristic equation for the scalar . 2. For the eigenvalue determine the corresponding eigenvector u by solving the system A I u O . Chapter 7: Eigenvalues and Eigenvectors 4 Let’s follow this procedure for the next example. Note that eigenvalues and eigenvectors come in pairs. You cannot have one without the other. Example 4 2 0 Determine the eigenvalues and corresponding eigenvectors of A . 1 3 Also sketch the effect of multiplying the eigenvectors by matrix A on 2 . Solution. What do we find first, the eigenvalues or eigenvectors? Eigenvalues because eigenvectors are born out of eigenvalues. Remember we carry out the procedure outlined above: Step 1. Solve the characteristic equation for the scalar . We need to find λ in det A I . First we obtain A I : 2 A I 1 2 1 0 1 0 3 0 1 0 0 2 0 3 0 1 3 Substituting this into det A I gives 0 2 det A I det 2 3 0 3 1 For eigenvalues we equate this determinant to zero: 2 3 0 1 2 or 2 3 Step 2. For the eigenvalue determine the corresponding eigenvector u by solving the system A I u O . 2 0 Let u be the eigenvector corresponding to 1 2 . Substituting A and 1 3 1 2 into A I u O gives 2 0 2 0 u O 1 3 0 2 0 0 u O 1 1 A I u 0 x Remember O and let u and so we have y 0 0 0 x 0 1 1 y 0 Multiplying out gives 00 0 x y 0 Remember the eigenvector cannot be the zero vector, therefore at least one of the values, x or y, must be non-zero. From the bottom equation we have x y . Chapter 7: Eigenvalues and Eigenvectors 5 2 3 5 Simplest solution is x 1, y 1 but we can have , , , etc 2 3 5 We have an infinite number of solutions. We need to write down the general eigenvector u. How? Let x s then y s where s 0 and is a scalar. Thus the eigenvectors belonging to 2 are 1 s 1 u s where s 0 is the simplest eigenvector s 1 1 Similarly we find the general eigenvector v corresponding to the other eigenvalue 2 3 . Putting 2 3 into A I v O gives 2 0 3 0 1 0 v O simplifies to v O 1 3 0 3 1 0 0 [different x and y from those above] and O we obtain 0 1 0 x 0 1 0 y 0 A I v x By writing v y Multiplying out: x 0 0, x 0 0 From these equations we must have x 0 . What is y equal to? We can choose y to be any real number apart from zero. Thus y s where s 0 The general eigenvector belonging to 2 3 is 0 x 0 0 v s where s 0 is the simplest eigenvector y s 1 1 Summarizing the above we have for the eigenvalue 1 2 the eigenvector 1 0 u s and for 2 3 the eigenvector v s . What does all this mean? 1 1 It means that we must have Au 1u and Av 2 v which in this case is Au 2u and Av 3v Plotting these eigenvectors and the effect of multiplying by the matrix A is shown: y 3 Av = 3v u and v shown are the simplest eigenvectors. 2 1 v= 0 1 0.5 -1 Fig 3 -2 u= 1 1 -1 1.5 2 Au = 2u x Av = 3v Chapter 7: Eigenvalues and Eigenvectors 6 The effect on the eigenvectors of multiplying by the matrix A is to produce scalar multiples of itself as you can see in Fig 3. Eigenvectors are non-zero vectors which are transformed to scalar multiples of themselves by a square matrix A. Next we find the eigenvalues and eigenvectors of a 3 by 3 matrix. Follow the algebra carefully because you will have to expand brackets like 1 3 To expand this it is easier to take out 2 minus signs and then expand, that is 1 3 1 3 13 Because Example 5 1 0 4 Determine the eigenvalues of A 0 4 0 3 5 3 Solution. We have 0 4 1 0 4 0 0 1 A I 0 4 0 0 0 0 4 0 3 5 3 0 0 3 5 3 It is easier to remember that A I is actually matrix A with along the leading diagonal (from top left to bottom right). We need to evaluate det A I . What is the easiest way to find this? From the properties of determinants of chapter 2 we know that it will be easier to evaluate the determinant along the second row, containing the elements 0, 4 and 0. Why? Because it has two zeros so that we do not have to evaluate the 2 by 2 determinants associated with these zeros. From above we have 0 4 1 det A I det 0 4 0 Row 2 3 5 3 1 4 4 det 3 3 4 1 3 3 4 Expanding the Second Row By Determinant of 2 by 2 4 1 3 12 Taking Out Minus Signs 4 3 2 3 12 Opening Brackets and 4 2 2 15 Simplifying 4 5 3 Factorizing By the characteristic equation (7.2) which is det A I 0 , means that we equate all the above to zero: 4 5 3 0 Chapter 7: Eigenvalues and Eigenvectors 7 Solving this gives the eigenvalues 1 4, 2 5 and 3 3 . Example 6 Determine the eigenvectors associated with 3 3 for the matrix A given in the above Example 5. Solution. 1 0 4 Substituting the eigenvalue 3 3 and the matrix A 0 4 0 into 3 5 3 A I u O (subtract 3 from the leading diagonal) gives: 0 4 1 3 A 3I u 0 4 3 0 u O 3 5 3 3 where u is the eigenvector corresponding to 3 3 . What is the zero vector, O, equal to? 0 x Remember this zero vector is O 0 and also let u y . Substituting these into the 0 z above and simplifying gives 2 0 4 x 0 0 1 0 y 0 3 5 6 z 0 Expanding this yields the linear system 2 x 0 4 z 0 † 0 y0 0 3x 5 y 6 z 0 †† ††† From the middle equation †† we have y 0 . From the top equation † we have 2x 4z which gives x 2z If z 1 then x 2 ; or more generally if z s then x 2s where s 0 [Not Zero]. x 2s 2 The general eigenvector u y 0 s 0 where s 0 and corresponds to z s 1 3 3 . The eigenvectors corresponding to 1 4 and 2 5 are part of Exercise 7a. A3 Eigenspace Note that for the 3 3 in the above Example 6 we have an infinite number of eigenvectors by substituting various non-zero values of s: Chapter 7: Eigenvalues and Eigenvectors 8 2 4 1 4 u 0 or u 0 or u 0 or u 0 etc 1 2 1/ 2 2 These solutions are given by all the points (apart from x y z 0 ) on the line shown below: 4 0 2 4 0 2 Fig 4 In general if A is a square matrix and is an eigenvalue of A with an eigenvector u then every scalar multiply (apart from 0) of the vector u is also an eigenvector belonging to the eigenvalue . We write this as a proposition: Proposition (7.3). If is an eigenvalue of a square matrix A with an eigenvector u then every non-zero scalar multiplication of u is also an eigenvector belonging to . Proof. By (7.1) we have Au u where is the eigenvalue and u is the eigenvector belonging to . Consider an arbitrary non-zero scalar k, then A k u k Au By rules of Matrices k u ku By 7.1 Thus we have A k u k u which means that k u is an eigenvector belonging to the eigenvalue . Since k was arbitrary therefore every non-zero scalar multiple of u is an eigenvector of the matrix A belonging to the eigenvalue . ■ Proposition (7.4). If A is a square n n matrix with an eigenvalue of , then the set of all eigenvectors of A belonging to together with the zero vector, O, that is the following set S O u u is an eigenvector belonging to is a subspace of n . How do we prove the given set is a subspace of n ? Chapter 7: Eigenvalues and Eigenvectors 9 We can use proposition (4.7) of chapter 4 which says that any non-empty subset S of a vector space is a subspace if and only if (a) The zero vector, O, is in S (b) If vectors u and v are in S then for any scalars k and c we have k u c v is also in S. Proof. What do we need to prove? We need to prove the zero vector, O, is in the set S and if u and v are eigenvectors belonging to the eigenvalue then k u c v is also an eigenvector belonging to . Clearly by the definition of the set S we have the zero vector in S. Let u and v be eigenvectors belonging to the eigenvalue and k and c be any non-zero scalars. Then by the above Proposition (7.3) we have A k u k u and A c v c v We need to show that A k u c v k u c v : Applying the rules of matrices By the above A ku cv A ku A cv ku cv ku cv Thus k u c v is an eigenvector belonging to which means it is a member of the set S. By Proposition (4.7) we conclude that the set S is a subspace of n . ■ This subspace S of Proposition (7.4) S O u u is an eigenvector belonging to is called an eigenspace of λ and is denoted by E , that is E S . For example the eigenspace relating to Example 4 for the eigenvalue 1 2 is the 1 0 eigenvector u s and for 2 3 the eigenvector v s is shown below: 1 1 y 3 E3 Vectors u and v shown are the simplest eigenvectors. 2 1 -3 -2 v= -1 1 -1 u= -2 Fig 5 0 1 -3 1 -1 2 3 E2 x Chapter 7: Eigenvalues and Eigenvectors 10 0 1 Note that the vector is a basis for the eigenspace E3 and is a basis for the 1 1 eigenspace E2 . The simplest eigenvectors are a basis for each eigenspace. The line in Fig 4 with arrows pointing to it is the eigenspace E3 of Example 5. We can also use the numerical software MATLAB to find eigenvalues and eigenvectors as the following commands verify: 1 2 3 Let A 4 5 6 and enter this matrix in MATLAB. The command eig(A) gives you 7 8 9 the eigenvalues of this matrix which is 16.1168, 1.1168 and 0 . The following command >> [V,d]=eig(A) should show V= Reading each column down gives 0.2320 0.7858 0.4082 the eigenvectors and they have been 0.5253 0.0868 0.8165 normalized. 0.8187 0.6123 0.4082 d= The diagonal entries give the 16.1168 0 0 eigenvalues 0 1.1168 0 0 0 0.0000 MATLAB gives the normalized eigenvectors. Note that we can normalize our eigenvectors by dividing by the length or norm of the vector. By reading the above MATLAB output we have the normalized eigenvectors are 0.2320 0.7858 0.4082 , and u 0.5253 v 0.0868 w 0.8165 0.4082 0.8187 0.6123 SUMMARY Consider (7.1) Au u where A is a square matrix, u is a column vector and is a scalar. If u is non-zero vector then we call u an eigenvector belonging (or corresponding) to the eigenvalue of the matrix A. The following equation (7.2) det A I 0 is called the characteristic equation. To find the eigenvalues and eigenvectors we use the following procedure: 1. Solve the characteristic equation for the scalar . 2. For the eigenvalue determine the corresponding eigenvector u by solving the system A I u O . Proposition (7.3). If is an eigenvalue of a square matrix A with an eigenvector u then every non-zero scalar multiplication of u is also an eigenvector belonging to . Proposition (7.4). If A is a square n n matrix with an eigenvalue of , then the following set E O u u is an eigenvector belonging to is a subspace of n and is called the eigenspace.