De…nitions of Linear Algebra Terms In order to learn and understand mathematics, it is necessary to understand the meanings of the terms (vocabulary words) that are used. This document contains de…nitions of some of the important terms used in linear algebra. All of these de…nitions should be memorized (and not just memorized but understood). In addition to the de…nitions that are given here, we also include statements of some of the most important results (theorems) of linear algebra. The proofs of these theorems are not included here but will be given in class. There will be a question on each of the exams that asks you to state some de…nitions. It will be required that you state these de…nitions very precisely in order to receive credit for the exam question. (Seemingly very small things such as writing the word “or” instead of the word “and”, for example, can render a de…nition incorrect.) The de…nitions given here are organized according to the section number in which they appear in David Lay’s textbook. Here is a suggestion for how to study linear algebra (and other subjects in higher level mathematics): 1. Carefully read a section in the textbook and also read your class notes. Pay attention to de…nitions of terms and examples and try to understand each concept along the way as you read it. You don’t have to memorize all of the de…nitions upon …rst reading. You will memorize them gradually as you work the homework problems. 2. Work on the homework problems. As you work on the homework problems, you should …nd that you will need to continuously refer back to the de…nitions and to the examples given in the textbook. (This is how you will naturally come to memorize the de…nitions.) You will …nd that you have to spend a considerable amount of time working on homework. When you are done, you will probably have read through the section and your course notes several times and you will probably have memorized the de…nitions. (There are some very good True–False questions among the homework problems that are good tests of your understanding of the basic terms and concepts. Such True–False questions will also appear on the exams given in this course.) 3. Test yourself. After you are satis…ed that you have a good understanding of the concepts and that can apply those concepts in solving 1 problems, try to write out the de…nitions without referring to your book or notes. You might then also try to work some of the supplementary problems that appear at the end of each chapter of the textbook. 4. One other tip is that it is a good idea to read a section in the textbook and perhaps even get started on working on the homework problems before we go over that section in class. This will better prepare you to take notes and ask questions when we do go over the material in class. It will alert you to certain aspects of the material that you might not quite understand upon …rst reading and you can then ask questions about those particular aspects during class. 1.1: Systems of Linear Equations De…nition 1 A linear equation in the variables x1 ; x2 ; : : : ; xn is an equation of the form a1 x 1 + a2 x 2 + : : : + an x n = b where a1 ; a2 ; : : : ; an and b are real or complex numbers (that are often given in advance). Example 2 4x + 4y 7z + 8w = 8 is a linear equation in the variables x; y; z; and w. 3x + 5y 2 = 6 is not a linear equation (because of the appearance of y 2 ). De…nition 3 Suppose that m and n are positive integers. A system of linear equations in the variables x1 ; x2 ; : : : ; xn is a set of equations of the form a11 x1 + a12 x2 + : : : + a1n xn = b1 a21 x1 + a22 x2 + : : : + a2n xn = b2 .. . am1 x1 + am2 x2 + : : : + amn xn = bm 2 where aij (i = 1; : : : ; m, j = 1; : : : n) and bi (i = 1; : : : ; m) are real or complex numbers (that are often given in advance). This system of linear equations is said to consist of m equations with n unknowns. Example 4 2x1 + 2x2 3x1 5x3 + 4x4 4x1 + x2 2x2 + 4x4 2x5 = 4 5x4 = 1 2x5 = 4 is a system of three linear equations in …ve unknowns. (The “unknowns” or “variables” of the system are x1 ; x2 ; x3 ; x4 ; and x5 .) De…nition 5 A solution of the linear system a11 x1 + a12 x2 + : : : + a1n xn = b1 a21 x1 + a22 x2 + : : : + a2n xn = b2 .. . am1 x1 + am2 x2 + : : : + amn xn = bm is an ordered n–tuple of numbers (s1 ; s2 ; : : : ; sn ) such that all of the equations in the system are true when the substitution x1 = s1 ; x2 = s2 ; : : : ; xn = sn is made. Example 6 The 5–tuple 0; 1; 2x1 + 2x2 3x1 12 ; 0; 3 5 5x3 + 4x4 4x1 + x2 2x2 + 4x4 is a solution of the linear system 2x5 = 4 5x4 = 1 2x5 = 4 because all equations in the system are true when we make the substitutions x1 = 0 x2 = 1 12 x3 = 5 x4 = 0 x5 = 3. The 5–tuple 1; 3; 53 ; 0; 21 is also a solution of the system (as the reader can check). 3 De…nition 7 The solution set of a system of linear equations is the set of all solutions of the system. De…nition 8 Two linear systems are said to be equivalent to each other if they have the same solution set. De…nition 9 A linear system is said to be consistent if it has at least one solution and is otherwise said to be inconsistent. Example 10 The linear system 2x1 + 2x2 5x3 + 4x4 4x1 + x2 2x2 + 4x4 3x1 2x5 = 4 5x4 = 1 2x5 = 4 is consistent because it has at least one solution. (It in fact has in…nitely many solutions.) However, the linear system x + 2y = 5 x + 2y = 9 is obviously inconsistent. De…nition 11 Suppose that m and n are positive integers. An m n matrix is a rectangular array of numbers with m rows and n columns. Matrices are usually denoted by capital letters. If m = 1 or n = 1, then the matrix is referred to as a vector. In particular, if m = 1, then the matrix is a row vector of dimension n, and if n = 1 then the matrix is a column vector of dimension m. Vectors are usually denoted by lower case boldface letters or (when hand–written) as lower case letters with arrows above them. Example 12 is a 5 4 matrix. 2 6 6 A=6 6 4 8 10 5 7 0 5 1 3 4 8 r=! r = 2 3 4 6 6 10 10 1 8 6 1 5 10 1 9 8 3 7 7 7 7 5 is a row vector of dimension 5. 2 3 4 c=! c =4 5 5 10 is a column vector of dimension 3. Remark 13 If A is an m n matrix whose columns are made up of the column vectors c1 ; c2 ; : : : ; cn , then A can be written as A = [c1 cn ] . c2 Likewise, if A is made up of the row vectors r1 ; r2 ; : : : ; rm , then A can be written as 2 3 r1 6 r2 7 6 7 A = 6 .. 7 . 4 . 5 rm Example 14 The 5 can be written as 4 matrix 2 8 10 6 5 7 6 5 A=6 6 0 4 1 3 4 8 6 6 10 10 1 A = [c1 c3 c2 8 6 1 5 10 3 7 7 7 7 5 c4 ] where 2 6 6 c1 = 6 6 4 8 5 0 1 4 3 2 7 6 7 6 7 , c2 = 6 7 6 5 4 10 7 5 3 8 3 2 7 6 7 6 7 , c3 = 6 7 6 5 4 5 6 6 10 10 1 3 2 7 6 7 6 7 , c4 = 6 7 6 5 4 8 6 1 5 10 3 7 7 7 7 5 or can be written as 2 r1 r2 r3 r4 r5 6 6 A=6 6 4 where r1 = 8 10 7 7 7 7 5 6 8 5 7 6 r2 = r3 = 0 5 r4 = 1 r5 = 3 10 1 3 4 6 10 5 8 1 10 . De…nition 15 For a given linear system of equations a11 x1 + a12 x2 + : : : + a1n xn = b1 a21 x1 + a22 x2 + : : : + a2n xn = b2 .. . am1 x1 + am2 x2 + : : : + amn xn = bm , the m n matrix 2 6 6 A=6 4 a11 a21 .. . a12 a22 .. . am1 am2 is called the coe¢ cient matrix of 2 a11 6 a21 6 [A b] = 6 .. 4 . am1 ... a1n a2n .. . amn 3 7 7 7 5 the system and the m 3 a12 a1n b1 a22 a2n b2 7 7 .. .. .. 7 .. . . . . 5 am2 amn bm is called the augmented matrix of the system. 6 (n + 1) matrix Example 16 The coe¢ cient matrix of the linear system 2x1 + 2x2 5x3 + 4x4 4x1 + x2 2x2 + 4x4 3x1 is 2 2 A=4 4 3 2 1 2 and the augmented matrix of this 2 2 4 4 [A b] = 3 5 0 0 2x5 = 4 5x4 = 1 2x5 = 4 4 5 4 3 2 0 5 2 4 5 4 2 0 2 system is 2 1 2 5 0 0 3 4 1 5. 4 De…nition 17 An elementary row operation is performed on a matrix by changing the matrix in one of the following ways: 1. Interchange: Interchange two rows of the matrix. If rows i and j are interchanged, then we use the notation ri ! rj . 2. Scaling: Multiply all entries in a single row by a non–zero constant, k. If the ith row is multiplied by k, then we use the notation k ri ! ri . 3. Replacement: Replace a row with the sum of itself and a non–zero multiple of another row. If the ith row is replaced with the sum of itself and k times the jth row, then we use the notation ri + k rj ! ri . Example 18 Here are some examples formed on the matrix 2 2 5 4 7 4 A= 8 7 1. Interchange rows 2 2 4 7 8 1 and 3: 5 4 7 7 8 8 of elementary row operations per7 8 8 3 7 2 5. 2 3 2 7 8 7 r1 !r3 5 4 2 7 4 ! 2 2 5 7 8 8 7 3 2 2 5 7 2. Scale row 1 by a factor of 5. 2 3 2 2 5 7 7 10 5 r1 !r1 4 7 4 8 2 5 ! 4 7 8 7 8 2 8 25 4 7 35 8 8 3 35 2 5 2 3. Replace row 2 with the sum of itself and 2 times row 3. 2 3 2 3 2 5 7 7 2 5 7 7 r +2 r3 !r2 4 7 4 4 9 18 8 8 2 5 2 ! 2 5 8 7 8 2 8 7 8 2 De…nition 19 An m n matrix, A, is said to be row equivalent (or simply equivalent) to an m n matrix, B, if A can be transformed into B by a series of elementary row operations. If A is equivalent to B, then we write A B. Example 20 As can 2 2 4 7 8 2 2 5 4 7 4 8 7 and 2 be seen by studying the previous 3 2 5 7 7 8 7 8 5 4 4 8 2 7 4 8 7 8 2 2 5 7 3 2 7 7 10 25 35 5 4 8 2 7 4 8 8 2 8 7 8 2 5 4 7 4 8 7 7 8 8 3 7 2 5 2 2 2 5 7 4 9 18 8 8 7 8 Example 21 The sequence of row operations 2 3 2 3 2 3 6 6 12 6 1 !r1 3 !r3 4 1 6 5 2 r! 4 1 6 5 4 r! 4 1 7 1 7 1 28 shows that 2 3 4 1 7 3 6 6 5 1 2 28 4 1 6 8 3 4 6 5. 12 example 3 2 2 5, 7 3 35 2 5, 2 3 7 2 5. 2 3 2 12 28 r !r 6 5 1 !3 4 1 4 6 3 4 6 5 12 Remark 22 (optional) Row equivalence of matrices is an example of what is called an equivalence relation. An equivalence relation on a non–empty set of objects, X, is a relation, , that is re‡exive, symmetric, and transitive. The re‡exive property is the property that x x for all x 2 X. The symmetric property is the property that if x 2 X and y 2 X with x y, then y x. The transitive property is the property that if x 2 X, y 2 X, and z 2 X with x y and y z, then x z. If we take X to be the set of all m n matrices, then row equivalence is an equivalence relation on X because 1. A A for all A 2 X. 2. If A 2 X and B 2 X and A B, then B 3. If A 2 X, B 2 X, and C 2 X and A A. B and B C, then A C. 1.2: Row Reduction and Echelon Forms De…nition 23 The leading entry in a row of a matrix is the …rst (from the left) non–zero entry in that row. Example 24 For the matrix 2 6 6 6 6 4 0 0 5 0 9 1 0 0 0 7 3 4 8 4 3 7 7 0 10 7 7, 0 1 5 2 8 the leading entry in row 1 is 1, the leading entry in row 2 is 4, the leading entry in row 3 is 5, the leading entry in row 4 is 1, and the leading entry in row 5 is 9. (Note that if a matrix happens to have a row consisting of all zeros, then that row does not have a leading entry.) De…nition 25 A zero row of a matrix is a row consisting entirely of zeros. A nonzero row is a row that is not a zero row. De…nition 26 An echelon form matrix is a matrix for which 1. No zero row has a nonzero row below it. 9 2. Each leading entry is in a column to the right of the leading entry in the row above it. De…nition 27 A reduced echelon form matrix is an echelon form matrix which also has the properties: 3. The leading entry in each nonzero row is 1. 4. Each leading 1 is the only nonzero entry in its column. Example 28 The matrix 2 6 4 6 1 2 6 4 4 4 0 10 is not an echelon form matrix. The matrix 3 10 10 7 7 2 5 1 0 0 0 4 8 0 0 0 0 3 is an echelon form matrix but not a reduced echelon form matrix. The matrix 2 3 0 0 1 0 6 0 0 0 1 7 6 7 6 0 0 0 0 7 6 7 4 0 0 0 0 5 0 0 0 0 is a reduced echelon form matrix. Theorem 29 (Uniqueness of the Reduced Echelon Form) Each matrix, A, is equivalent to one and only one reduced echelon form matrix. This unique matrix is denoted by rref(A). Example 30 For 2 6 4 6 1 2 A=6 4 4 4 0 10 10 3 10 10 7 7; 2 5 1 we can see (after some work) that 2 For 1 6 0 rref (A) = 6 4 0 0 A= 3 0 0 7 7. 1 5 0 0 1 0 0 0 0 0 4 8 0 0 0 0 3 , we have 0 0 0 1 0 0 0 0 0 1 rref (A) = For we have 2 6 6 A=6 6 4 0 0 0 0 0 2 6 6 rref (A) = 6 6 4 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 . 3 7 7 7, 7 5 0 1 0 0 0 3 7 7 7. 7 5 De…nition 31 A pivot position in a matrix, A, is a position in A that corresponds to a row–leading 1 in rref(A). A pivot column of A is column of A that contains a pivot position. Example 32 By studying the preceding example, we can see that the pivot columns of 2 3 6 4 10 6 1 2 10 7 7 A=6 4 4 4 2 5 0 10 1 are columns 1, 2, and 3. (Thus all columns of A are pivot columns). 11 The pivot columns of A= 0 0 0 4 8 0 0 0 0 3 are columns 4 and 5. The pivot columns of 2 6 6 A=6 6 4 are columns 3 and 4. 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 3 0 1 0 0 0 7 7 7 7 5 De…nition 33 Let A be the coe¢ cient matrix of the linear system of equations a11 x1 + a12 x2 + : : : + a1n xn = b1 a21 x1 + a22 x2 + : : : + a2n xn = b2 .. . am1 x1 + am2 x2 + : : : + amn xn = bm . Each variable that corresponds to a pivot column of A is called a basic variable and the other variables are called free variables. Example 34 The coe¢ cient matrix of the linear system 2x1 + 2x2 3x1 is Since 5x3 + 4x4 4x1 + x2 2x2 + 4x4 2 2 4 4 A= 3 2 1 2 5 0 0 2 1 0 0 rref (A) = 4 0 1 0 0 0 1 12 2x5 = 4 5x4 = 1 2x5 = 4 4 5 4 6 5 1 5 2 5 3 2 0 5. 2 2 5 8 5 6 5 3 5, we see that the pivot columns of A are columns 1, 2, and 3. Thus the basic variables of this linear system are x1 , x2 ; and x3 . The free variables are x4 and x5 . Theorem 35 (Existence and Uniqueness Theorem) A linear system of equations is consistent if and only if the rightmost column of the augmented matrix for the system is not a pivot column. If a linear system is consistent, then the solution set of the linear system contains either a unique solution (when there are no free variables), or in…nitely many solutions (when there is at least one free variable). Example 36 The augmented matrix of the system 2x1 + 2x2 3x1 is 5x3 + 4x4 4x1 + x2 2x2 + 4x4 2 Since 2 [A b] = 4 4 3 2 1 2 5 0 0 2 1 0 0 rref ([A b]) = 4 0 1 0 0 0 1 2x5 = 4 5x4 = 1 2x5 = 4 4 5 4 2 0 2 6 5 2 5 8 5 6 5 1 5 2 5 3 4 1 5. 4 6 5 19 5 6 5 3 5, we see that the rightmost column of [A b] is not a pivot column. This means that the system is consistent. Furthermore, since the system does have free variables, there are in…nitely many solutions to the system. Here is a general procedure for determining whether a given linear system of equations is consistent or not and, if it is consistent, whether it has in…nitely many solutions or a unique solution: 1. Compute rref([A b]) to determine whether or not the rightmost column of [A b] is a pivot column. If the rightmost of [A b] is a pivot column, then the system is inconsistent. 13 2. Assuming that the rightmost column of [A b] is not a pivot column, take a look at rref(A). (This requires no extra computation.) If all columns of A are pivot columns, then the linear system has a unique solution. If not all columns of A are pivot columns, then the system has in…nitely many solutions. 1.3: Vector Equations De…nition 37 Suppose that 2 6 6 u=6 4 u1 u2 .. . un 3 7 7 7 5 2 6 6 and v = 6 4 v1 v2 .. . vn 3 7 7 7 5 are column vectors of dimension n. We de…ne the sum of u and v to be the vector 2 3 u1 + v1 6 u2 + v2 7 6 7 u+v =6 7. .. 4 5 . un + vn De…nition 38 Suppose that 2 6 6 u=6 4 u1 u2 .. . un 3 7 7 7 5 is a column vectors of dimension n and suppose that c is a real number (also called a scalar). We de…ne the scalar multiple cu to be 2 3 cu1 6 cu2 7 6 7 cu = 6 .. 7 . 4 . 5 cun 14 De…nition 39 The symbol Rn denotes the set of all ordered n–tuples of numbers. The symbol Vn denotes the set of all column vectors of dimension n. (The textbook author does not make this distinction. He uses Rn to denote both things. However, there really is a di¤erence. Elements of Rn should be regarded as points whereas elements of Vn are matrices with one column.) If x = (x1 ; x2 ; : : : ; xn ) is a point in Rn , then the position vector of x is the vector 2 3 x1 6 x2 7 6 7 x = 6 .. 7 . 4 . 5 xn Example 40 x = (1; 2) is a point in R2 . The position vector of this point (which is an element of V2 ) is x= 1 2 . De…nition 41 Suppose that u1 ; u2 ; : : : ; un are vectors in Vm . (Note that this is a set of n vectors, each of which has dimension m). Also suppose that c1 ; c2 ; : : : ; cn are scalars. Then the vector c 1 u1 + c 2 u2 + + c n un is called a linear combination of the vectors u1 ; u2 ; : : : ; un . Example 42 Suppose that u1 = 1 2 , u2 = 0 5 , u3 = 1 4 . 2 0 5 1 4 = Then the vector 4u1 2u2 + 3u3 = 4 1 2 +3 7 30 is a linear combination of u1 ,u2 , and u3 . De…nition 43 Suppose that u1 ; u2 ; : : : ; un are vectors in Vm . The span of this set of vectors, denoted by Spanfu1 ; u2 ; : : : ; un g, is the set of all vectors that are linear combinations of the vectors u1 ; u2 ; : : : ; un . Formally, Span fu1 ; u2 ; : : : ; un g = fv 2 Vm j there exist scalars c1 ; c2 ; : : : ; cn such that v = c1 u1 + c2 u2 + 15 + c n un g . De…nition 44 The zero vector in Vm is the vector 2 3 0 6 0 7 6 7 0m = 6 .. 7 . 4 . 5 0 Remark 45 If u1 ; u2 ; : : : ; un is any set of vectors in Vm , then 0m 2 Spanfu1 ; u2 ; : : : ; un g because 0m = 0u1 + 0u2 + + 0un . De…nition 46 A vector equation in the variables x1 , x2 ,. . . ,xn is an equation of the form x1 a1 + x2 a2 + + xn an = b where a1 ,a2 ,. . . ,an and b are vectors in Vm (which are usually known beforehand). An ordered n–tuple (s1 ; s2 ; : : : ; sn ) is called a solution of this vector equation if the equation is satis…ed when we set x1 = s1 , x2 = s2 ,. . . , xn = sn . The solution set of the vector equation is the set of all of its solutions. Example 47 An example of a vector equation is x1 1 2 + x2 0 5 + x3 1 4 = 4 2 . The ordered triple (x1 ; x2 ; x3 ) = ( 9; 4=5; 5) is a solution of this vector equation because 4 0 1 4 1 +5 = . 9 + 5 4 2 2 5 Remark 48 The vector equation x1 a1 + x2 a2 + + xn an = b is consistent if and only if b 2 Spanfa1 ; a2 ; : : : ; an g. Remark 49 The vector equation x1 a1 + x2 a2 + 16 + xn an = b where and 2 6 6 a1 = 6 4 a11 a21 .. . am1 3 2 7 6 7 6 7 , a2 = 6 5 4 3 a12 a22 .. . am2 2 6 6 b=6 4 2 a1n a2n .. . 7 6 7 6 7 , . . . ,an = 6 5 4 amn 3 b1 b2 .. . bm 3 7 7 7 5 7 7 7 5 is equivalent to the system of linear equations a11 x1 + a12 x2 + : : : + a1n xn = b1 a21 x1 + a22 x2 + : : : + a2n xn = b2 .. . am1 x1 + am2 x2 + : : : + amn xn = bm . This means that the vector equation and the system have the same solution sets. Each can be solved (or determined to be inconsistent) by row–reducing the augmented matrix a1 a2 an b . Example 50 Let us solve the vector equation x1 1 2 + x2 0 5 + x3 1 4 4 2 = This equation is equivalent to the linear system 2x1 x1 + x3 = 5x2 + 4x3 = 4 2. Since [A b] = 1 2 0 1 5 4 4 2 and rref ([A b]) = 1 0 0 1 17 1 2 5 4 6 5 , . we see that the vector equation is consistent (because the rightmost column of [A b] is not a pivot column) and that it in fact has in…nitely many solutions (because not all columns of A are pivot columns). We also see that the general solution is x1 = 4 x3 6 2 x2 = + x3 5 5 x3 = free. Note that if we set x3 = 5, then we get the particular solution x1 = 9 4 5 x3 = 5 x2 = that was identi…ed in Example 47. 1.4: The Matrix Equation Ax = b De…nition 51 For a matrix of size m n, 2 a11 6 a21 6 an = 6 .. A = a1 a2 4 . a12 a22 .. . ... am1 am2 and a vector of dimension n, 2 6 6 x=6 4 x1 x2 .. . xn we de…ne the product Ax to be 3 7 7 7, 5 Ax = x1 a1 + x2 a2 + 18 + xn an . a1n a2n .. . amn 3 7 7 7, 5 Example 52 Suppose 2 4 A= and 3 3 4 3 2 3 4 x = 4 1 5. 0 Then Ax = 4 2 4 3 3 1 +0 4 3 = 11 19 . De…nition 53 Suppose that A= is an m a1 a2 2 6 6 =6 4 an n matrix and that 2 Then 6 6 b=6 4 b1 b2 .. . bm a11 a21 .. . a12 a22 .. . ... am1 am2 3 7 7 7 2 Vm . 5 Ax = b is called a matrix equation. Remark 54 The system of linear equations a11 x1 + a12 x2 + : : : + a1n xn = b1 a21 x1 + a22 x2 + : : : + a2n xn = b2 .. . am1 x1 + am2 x2 + : : : + amn xn = bm , 19 a1n a2n .. . amn 3 7 7 7 5 the vector equation x1 a1 + x2 a2 + + xn an = b, and the matrix equation Ax = b are all equivalent to each other! The only subtle di¤erence is that solutions of the system and of the vector equation are ordered n–tuples (x1 ; x2 ; : : : ; xn ) = (s1 ; s2 ; : : : ; sn ) 2 Rn ; whereas solutions of the matrix 2 x1 6 x2 6 6 .. 4 . xn equation are the corresponding vectors 3 2 3 s1 7 6 s2 7 7 6 7 7 = 6 .. 7 2 Vn . 5 4 . 5 sn 1.5: Solution Sets of Linear Systems De…nition 55 A homogeneous linear system is one of the form Ax = 0m where A is an m n matrix. A non–homogeneous linear system is a linear system that is not homogeneous. Remark 56 Homogenous systems are consistent! That’s because they have the solution x = 0n . De…nition 57 The solution x = 0n is called the trivial solution of the homogeneous system Ax = 0m . A non–trivial solution of the homogeneous system Ax = 0m is a solution x 2 Vn such that x 6= 0n . Example 58 The system 5x + 2y 3z + w = 0 3x + y 5z + w = 0 x 3y 5z + 5w = 0 is homogeneous. A non–trivial solution of this system (as the reader can check) is (x; y; z; w) = ( 18; 34; 1; 25). 20 Theorem 59 If every column of A is a pivot column, then the homogeneous system Ax = 0m has only the trivial solution. If not every column of A is a pivot column, then the homogeneous system Ax = 0m has in…nitely many non–trivial solutions (in addition to the trivial solution). Example 60 The system given in Example 58 has coe¢ cient matrix 2 3 5 2 3 1 5 1 5. A=4 3 1 1 3 5 5 Without doing any computations, we can see that it is not possible that every column of A can be a pivot column. (Why?) Therefore the homogeneous system Ax = 0m has in…nitely many non–trivial solutions. Theorem 61 Suppose that A is an m n matrix and suppose that b 2 Vm . Also suppose that v 2 Vn is a solution of the homogeneous system Ax = 0m and that w 2 Vn is a solution of the system Ax = b. Then v + w is also a solution of Ax = b. 1.7: Linear Independence De…nition 62 A set of vectors fv1 ; v2 ; : : : ; vn g in Vm is said to be linearly independent if the vector equation x1 v1 + x2 v2 + + xn vn = 0m has only the trivial solution. Otherwise the set of vectors is said to be linearly dependent. Example 63 Let v1 and v2 be the vectors 2 3 2 4 v1 = 4 1 5 , v2 = 4 4 We solve the vector equation x1 v1 + x2 v2 = 03 21 3 1 1 5. 4 by observing that 2 3 2 3 4 1 1 0 1 5 4 0 1 5. A=4 1 4 4 0 0 Since each column of A is a pivot column, the vector equation has only the trivial solution. Therefore the set of vectors fv1 ; v2 g is linearly independent. De…nition 64 Suppose that fv1 ; v2 ; : : : ; vn g is a linearly dependent set of vectors and suppose that (c1 ; c2 ; : : : ; cn ) is a non–trivial solution of the vector equation x1 v1 + x2 v2 + + xn vn = 0m . Then the equation c1 v1 + c2 v2 + + cn vn = 0m is called a linear dependence relation for the set of vectors fv1 ; v2 ; : : : ; vn g. Example 65 Consider the set of vectors 2 3 2 3 2 3 4 1 2 v1 = 4 1 5 , v2 = 4 1 5 , v3 = 4 3 5 , 4 4 4 2 3 4 v4 = 4 3 5 . 3 We solve the vector equation x1 v1 + x2 v2 + x3 v3 + x4 v4 = 03 by observing that 2 4 4 1 4 1 1 4 2 3 4 3 4 3 5 3 2 1 0 0 4 0 1 0 0 0 1 11 40 13 20 9 8 3 5. Our conclusion is that the vector equation has non–trivial solutions and that the general solution is x1 x2 x3 x4 = 11t = 26t = 45t = 40t where t can be any real number. By choosing t = 1, we see that a linear dependence relation for the set of vectors fv1 ,v2 ,v3 ,v4 g is 11v1 + 26v2 + 45v3 + 40v4 = 03 . 22 1.8: Introduction to Linear Transformations De…nition 66 A transformation (or function or mapping), T , from Vn to Vm is a rule that assigns each vector x 2 Vn to a vector T (x) 2 Vm . The set Vn is called the domain of T and the set Vm is called the codomain (or target set) of T . If T is a transformation from Vn to Vm , then we write T : Vn ! Vm . De…nition 67 Suppose that T : Vn ! Vm . For each x 2 Vn , the vector T (x) 2 Vm is called the image of x under the action of T . The set of all images of vectors in Vn under the action of T is called the range of T:In set notation, the range of T is fy 2 Vm j y = T (x) for at least one x 2 Vn g . Example 68 This example illustrates the terms de…ned in the preceding two de…nitions: Consider the transformation T : V2 ! V3 de…ned by 2 3 x2 x T = 4 4 5. y y x The domain of T is V2 and the codomain of T is V3 . The image of the vector 4 x= 1 is 2 3 16 T (x) = 4 4 5 . 5 The range of T consists of all vectors in V3 that have the form 2 3 s 4 4 5 t where s can be any non–negative real number and t can be any real number. For example, the vector 2 3 6 y=4 4 5 9 23 is in the range of T because y=T p 6p 9+ 6 . Note that it is also true that y=T p 9 6p 6 . Are there any other vectors x 2 V2 such that T (x) = y? The answer to this does not matter as far as determining whether or not y is in the range of T is concerned. We know that y is in the range of T because we know that there exists at least one vector x 2 V2 such that T (x) = y. De…nition 69 A transformation T : Vn ! Vm is said to be a linear transformation if both of the following conditions are satis…ed: 1. T (u + v) = T (u) + T (v) for all vectors u and v in Vn . 2. T (cu) = cT (u) for all vectors u in Vn and for all scalars c. Example 70 Let T : V3 ! V2 be de…ned by 02 31 x x @ 4 y 5A = T x + y + 2z z . We will show that T is a linear transformation: Let 3 2 2 3 x1 x2 u = 4 y1 5 and v = 4 y2 5 z1 z2 24 be two vectors in V3 . Then 02 31 x1 + x2 T (u + v) = T @4 y1 + y2 5A z1 + z2 = x1 + x2 (x1 + x2 ) + (y1 + y2 ) + 2 (z1 + z2 ) = x1 + x2 (x1 + y1 + 2z1 ) + (x2 + y2 + 2z2 ) = x1 x1 + y1 + 2z1 + x2 x2 + y2 + 2z2 = T (u) + T (v) : Now let 2 3 x u =4 y 5 z be a vector in V3 and let c be a scalar. Then 02 31 cx T (cu) = T @4 cy 5A cz = =c cx cx + cy + 2 (cz) x x + y + 2z cT (u) . We have shown that T is a linear transformation. De…nition 71 The zero transformation is the linear transformation T : Vn ! Vm de…ned by T (u) = 0m for all u 2 Vn . De…nition 72 The identity transformation is the linear transformation T : Vn ! Vn de…ned by T (u) = u for all u 2 Vn . We leave it to the reader to prove that the zero transformation and the identity transformation are both linear transformations. 25 Example 73 The transformation T : V2 ! V3 de…ned by 2 3 x2 x T =4 4 5 y y x is not a linear transformation. We leave it to the reader to explain why not. 1.9: The Matrix of a Linear Transformation De…nition 74 A transformation T : Vn ! Vm is said to be onto Vm if the range of T is Vm . De…nition 75 (Alternate De…nition) A transformation T : Vn ! Vm is said to be onto Vm if each vector y 2 Vm is the image of at least one vector x 2 Vn . De…nition 76 A transformation T : Vn ! Vm is said to be one–to–one if each vector y 2 Vm is the image of at most one vector x 2 Vn . De…nition 77 A transformation T : Vn one–to–one is called a bijection. ! Vm that is both onto Vm and Remark 78 In reference to the “dart” analogy that we have been using in class: Supposing that Vn is our box of darts and Vm is our dartboard, a transformation T : Vn ! Vm is onto Vm if every point on the dartboard gets hit by at least one dart; whereas T is one–to–one if every point on the dartboard either does not get hit or gets hit by exactly one dart. If T is a bijection, then every point on the dartboard gets hit by exactly one dart. 2.2: The Inverse of a Matrix a b is a 2 2 matrix. The quantity c d bc is called the determinant of A. It is denoted either by det (A) or by De…nition 79 Suppose that A = ad jAj. De…nition 80 The n n identity matrix, denoted by In , is the matrix that has entries of 1 at all positions along its main diagonal and entries of 0 elsewhere. 26 Example 81 The 3 3 identity matrix is 2 3 1 0 0 I3 = 4 0 1 0 5 . 0 0 1 De…nition 82 An n n matrix, A, is said to be invertible if there exists an n n matrix, B, such that BA = AB = In . Proposition 83 If A is an invertible n n matrix and B and C are n n matrices such that BA = AB = In and CA = AC = In , then B = C. In other words, if A has an inverse, then this inverse is unique. De…nition 84 Suppose that A is an invertible n n matrix. The inverse of A is de…ned to be the unique matrix, A 1 , such that A 1 A = AA 1 = In . Example 85 The inverse of the matrix A= is A 1 = 3 0 1 3 1 3 0 1 9 1 3 . (The reader should perform the computations to check that A 1 A = AA I2 .) 1 = De…nition 86 An elementary matrix is a matrix is a matrix that can be obtained by performing a single elementary row operation on an identity matrix. Example 87 The matrix 2 3 1 0 0 E=4 0 1 0 5 9 0 1 is an elementary matrix because it is obtained from I3 by the elementary row operation 9 R1 + R3 ! R3 . 27 2.3: Characterizations of Invertible Matrices De…nition 88 A transformation T : Vn ! Vn is said to be invertible if there exists a transformation S : Vn ! Vn such that S (T (x)) = x for all x 2 Vn and T (S (x)) = x for all x 2 Vn . Proposition 89 If T : Vn ! Vn is invertible, then there is a unique transformation S : Vn ! Vn such that S (T (x)) = x for all x 2 Vn and T (S (x)) = x for all x 2 Vn . This unique transformation is called the inverse of T and is denoted by T 1 . Example 90 If T : V2 ! V2 is the linear transformation that rotates all vectors in V2 counterclockwise about the origin through an angle of 90 , then T is invertible and T 1 : V2 ! V2 is the linear transformation that rotates all vectors in V2 clockwise about the origin through an angle of 90 . Theorem 91 Suppose that A is an n n matrix and let T : Vn ! Vn be the linear transformation de…ned by T (x) = Ax for all x 2 Vn . Then the following statements are all equivalent to each other (meaning that all of the statements are true or all of them are false). 1. A is invertible. 2. T is invertible. 3. T is onto Vn . 4. T is one–to–one. 5. A In . 6. The columns of A span Vn . 7. The homogeneous equation Ax = 0n has only the trivial solution. 8. For every b 2 Vn , the equation Ax = b has a unique solution. 28 4.1: Vector Spaces and Subspaces De…nition 92 A vector space is a non-empty set of objects, V , called vectors, on which are de…ned two operations, called addition and multiplication by scalars (real numbers), subject to the ten axioms listed below. The axioms must hold for all vectors u; v; and w in V and for all scalars c and d. 1. The sum of u and v, denoted by u + v is in V . 2. The scalar multiple of u by c, denoted by cu, is in V . 3. u + v = v + u. 4. (u + v) + w = u + (v + w). 5. There is a zero vector, 0, in V such that u + 0 = u. 6. There is a vector u in V such that u + ( u) = 0. 7. c (u + v) = cu + cv. 8. (c + d) u = cu + du. 9. c (du) = (cd) u. 10. 1u = u. De…nition 93 If V is a vector space and W is a non–empty subset of V , then W is said to be a subspace of V if W is itself a vector space under the same operations of addition and multiplication by scalars that hold in V . De…nition 94 Suppose that v1 ,v2 ,. . . ,vn are vectors in a vector space, V , and suppose that c1 ,c2 ,. . . ,cn are scalars. Then c1 v1 + c2 v2 + + cn vn is said to be a linear combination of the vectors v1 ,v2 ,. . . ,vn . De…nition 95 Suppose that fv1 ; v2 ; : : : ; vn g is a set of vectors in a vector space, V . This set of vectors is said to be linearly independent if the equation c1 v1 + c2 v2 + + cn vn = 0 has only the trivial solution (c1 = c2 = = cn = 0). Otherwise the set of vectors is said to be linearly dependent. 29 De…nition 96 Suppose that S = fv1 ; v2 ; : : : ; vn g is a set of vectors in a vector space, V . The span of S, denoted by Span (S), is de…ned to be the set of all possible linear combinations of the vectors in S. 4.2: Linear Transformations De…nition 97 Suppose that V and W are vector spaces. A transformation T : V ! W is said to be a linear transformation if T (u + v) = T (u) + T (v) for all u and v in V and T (cu) = cT (u) for all u in V and all scalars c. Remark 98 As in our previous discussions of transformations, the vector space V is called the domain of T and the vector space W is called the codomain (or target set) of T . Remark 99 In the above de…nition, since V and W might be two di¤erent vectors spaces and hence might have two di¤erent operations of addition and multiplication by scalars, we might want to use di¤erent symbols to denote these di¤erent operations. For example we might use + and to denote addition and scalar multiplication in V and use and to denote addition and scalar multiplication in W . With these notations, the conditions for T to be a linear transformation would appear as T (u + v) = T (u) T (v) for all u and v in V and T (c u) = c T (u) for all u in V and all scalars c. In most cases it is not necessary to use di¤erent notations such as this since the operations that are intended are clear from the context of whatever is being discussed. De…nition 100 The nullspace (also called the kernel) of a linear transformation T : V ! W is the set of all vectors in V that are transformed by T into the zero vector in W . This space (which is a subspace of V ) is denoted by ker (T ). Thus ker (T ) = fv 2 V j T (v) = 0W g . 30 De…nition 101 The range of a linear transformation T : V ! W is the set of all vectors in W that are images under T of at least one vector in V . This space (which is a subspace of W ) is denoted by range(T ). Thus range (T ) = fw 2 W j T (v) = w for at least one v 2 V g . Another (shorter) way to write this is range (T ) = fT (v) j v 2 V g . De…nition 102 In the case of a linear transformation T : Vn ! Vm whose standard matrix is the m n matrix A, the nullspace of A, denoted by Nul(A), is de…ned to be the nullspace (or kernel) of T . Thus Nul(A) = ker (T ). Remark 103 Nul(A) is the solution set of the matrix equation Ax = 0m . De…nition 104 In the case of a linear transformation T : Vn ! Vm whose standard matrix is the m n matrix A, the column space of A, denoted by Col(A), is de…ned to be the range of T . Thus Col(A) =range(T ). Remark 105 If A = [a1 a2 an ], then Col(A) = Span (fa1 ; a2 ; : : : ; an g). 4.3: Bases De…nition 106 A basis for a vector space, V , is a …nite indexed set of vectors, B = fv1 ; v2 ; : : : ; vn g, in V that is linearly independent and also spans V (meaning that Span (B) = V ). Remark 107 Some vector spaces have “standard” bases – meaning bases that are rather obvious without having to do to much checking. For example, the standard basis for Vn is B = fe1 ; e2 ; : : : ; en g where 2 3 2 3 2 3 1 0 0 6 0 7 6 1 7 6 0 7 6 7 6 7 6 7 e1 = 6 .. 7 , e2 = 6 .. 7 , , en = 6 .. 7 . 4 . 5 4 . 5 4 . 5 1 0 0 31 The standard basis for Pn (R) (the set of all polynomial functions with domain R and degree less than or equal to n) is B = fp0 ; p1 ; p2 ; : : : ; pn g where p0 (x) = 1 for all x 2 R p1 (x) = x for all x 2 R p2 (x) = x2 for all x 2 R .. . pn (x) = xn for all x 2 R. The vector space F (R) (the set of all functions with domain R) does not have a basis because no …nite set of functions spans this space. 4.4: Coordinate Systems De…nition 108 Suppose that V is a vector space, that B = fv1 ; v2 ; : : : ; vn g is a basis for V , and that v is a vector in V . The coordinates of v relative to the basis B are the unique scalars c1 ; c2 ; ; cn such that c1 v1 + c2 v2 + + cn vn = v. The coordinate vector of v relative to the basis B is the vector 2 3 c1 6 c2 7 6 7 [v]B = 6 .. 7 2 Vn . 4 . 5 cn The transformation T : V ! Vn de…ned by T (v) = [v]B is a linear transformation that is called the coordinate transformation (or coordinate mapping) on V determined by the basis B. De…nition 109 If V and W are vector spaces and T : V ! W is a linear transformation that is one–to–one and maps V onto W , then T is said to be an isomorphism of V and W . De…nition 110 If V and W are vector spaces and if there exists an isomorphism T : V ! W , then V is said to be isomorphic to W . Remark 111 Isomorphism is an equivalence relation on the set of all vector spaces. This is because: 32 1. Every vector space is isomorphic to itself. (Why?) 2. If V is isomorphic to W , then W is isomorphic to V . (Why?) 3. If V is isomorphic to W and W is isomorphic to X, then V is isomorphic to X. (Why?) 4.5: The Dimension of a Vector Space De…nition 112 If V is a vector space that is spanned by a …nite set of vectors, then V is said to be …nite–dimensional and the dimension of V , denoted by dim (V ), is de…ned to be the number of vectors in a basis for V . The dimension of the zero vector space, V = f0g, is de…ned to be 0. (Note that the zero vector space has no basis.) If V 6= f0g and V is not spanned by any …nite set of vectors, then we say that V is in…nite–dimensional. 4.6: The Rank of a Matrix De…nition 113 If A is an m n matrix, then the rank of A, denoted by rank (A), is de…ned to be the dimension of the column space of A. Thus rank (A) = dim (Col (A)). Remark 114 Since we know that the column space and the row space of any matrix have the same dimension, it is also true that rank (A) = dim (row (A)). Direct Sums of Subspaces De…nition 115 If V is a vector space and H and K are subspaces of V such that H \ K = f0g, then the direct sum of H and K, which is denoted by H K and which is also a subspace of V , is de…ned to be H K = fu + v j u 2 H and v 2 Kg . Remark 116 In order for the direct sum, H K, to be de…ned, it is necessary that H \ K = f0g. (This is part of the de…nition of direct sum.) Whether or not H \ K = f0g, the set H + K = fu + v j u 2 H and v 2 Kg 33 is called the sum of H and K. H + K is a subspace of V whether or not H \ K = f0g. The di¤erence is that when H \ K = f0g (and hence H + K = H K), each vector, w, in H K can be written uniquely in the form w = u + v where u 2 H and v 2 K; whereas this unique representation might not be possible for all vectors in H + K if H \ K 6= f0g. De…nition 117 Two subspaces, H and K, of a vector space V are said to be complementary to each other if H K = V . De…nition 118 Two vectors, u and v, in Vn are said to be orthogonal to each other if uT v = vT u = [0]. Remark 119 If u and v are any two vectors in Vn , then uT v = vT u. Thus we could simply state the above de…nition by saying that u and v are orthogonal to each other if uT v = [0]. Remark 120 If [a] is a 1 1 matrix, then we often adopt the convention to just view [a] as a scalar and hence just write a instead of [a]. With this convention, we can say that u and v are orthogonal to each other if uT v = 0 (without writing the brackets around 0). Also, with this convention, uT v (for any two vectors, u and v, in Vn ) is called the dot product of u and v and is denoted by u v. For example, if 2 3 2 3 4 1 u = 4 1 5 and v = 4 1 5 , 4 4 then u v = uT v = 4 1 2 4 4 3 1 1 5 = (4) ( 1)+( 1) ( 1)+( 4) ( 4) = 13. 4 De…nition 121 Two subspaces, H and K, of Vn are said to be orthogonal to each other if uT v = 0 for all u 2 H and v 2 K. De…nition 122 Two subspaces, H and K, of Vn are said to be orthogonal complements of each other if H and K are orthogonal to each other and H K = Vn . 34 Fundamental Subspaces of a Matrix Let A = [a1 2 6 6 an ] = 6 4 a2 b1 b2 .. . bm 3 7 7 7 5 be an m n matrix (where a1 , a2 , . . . , an are the column vectors of A (each of which has size m 1) and b1 , b2 , . . . , bm are the row vectors of A (each of which has size 1 n). The following four subspaces (two of which are subspaces of Vn and the other two of which are subspaces of Vm are called the fundamental subspaces of the matrix A: 1. The row space of A, row (A) = Span bT1 ; bT2 ; : : : ; bTm is a subspace of Vn . 2. The null space of A, N ul (A) = fx 2 Vn j Ax = 0m g is a subspace of Vn . 3. The column space of A, Col (A) = Span (a1 ; a2 ; : : : ; an ) is a subspace of Vm . 4. The null space of AT , N ul AT = x 2 Vm j AT x = 0n is a subspace of Vm . 35