In this section, matrices are expressed using brackets such as [ ]. Column or row matrices (or vectors) are expressed by the notation given below: = ⋮ ̂=[ is a column matrix, ⋯ ] is a row matrix. Row Echolon Form and Rank Consider a linear system of equations [ ] = where [ ] is the coefficient matrix of size ( × ) and is the solution vector of size ( × 1). A linear system of size ( × ) is represented as an augmented matrix = [ ]| which can be row reduced to matrix [ ] | . This means [ ] | and [ ] | are equivalent so they have the same solution if either one has a solution. At the end of Gauss elimination, the row echolon form [ ] | of the augmented matrix will be: ⎡ ⎢ ⎢ ⎢ ⎢ ⎢ ⎣ ⋯ ⋯ ⋱ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋯ ⋮ | | | | | | | ⎤ ⎥ ⋮ ⎥ ⎥ ⎥ ⋮ ⎥ ⎦ Note that ≤ , ≠ 0 and all the elements in the shaded triangular and rectangular regions are zero. In the row reduced coefficient matrix [ ], the number of nonzero rows is r and r is the rank of [ ] and also the rank of [ ]. → The system is inconsistent (i.e. there is no solution) if ,⋯⋯, is not zero. (See Example4, where = 3 and < and at least one of the numbers = 4). → The system is consistent (i.e. solution is available), if either ,⋯⋯, are all zero. → There is unique solution, if the system is consistent and ( = 3) ). = , or < and the numbers = . (See Example3, where ( = , , = 2) < → If there are infinitely many solutions, then the system is consistent and in order to obtain any of these solutions, one can choose values of ,⋯, arbitrarily (i.e. choose the values of ( − ) unknowns arbitrarily). Then, one can solve for using the equation (in terms of the arbitrary values) and solve for , , ⋯ ⋯ in a similar manner. Doing further simplifications, the augmented matrix can also be put into row-reduced echolon form. An ( × ) matrix is in row-reduced echolon form if and only if, i) The first nonzero entry in any row is 1 and the column in which it occurs is a column of [ ] i.e. contains the number 1 and ( − 1) zeros. ii) Every zero row vector has a higher row-index than every nonzero row vector. iii) If there are nonzero row vectors and if for 1 ≤ ≤ , the first 1 in row occurs in column , then < <⋯< . The procedure to obtain the row-reduced echolon form is known as Gauss-Jordan elimination. In Matlab, use the command rref to obtain row-reduced echolon form of any matrix. For example, the below matrix is in row-reduced echolon form. 1 0 0 0 1 −2 0 0 0 0 0 1 0 6 3 0 Example5a: In Matlab, the system given in Example5 can be solved as follows: >> A=[3 2 2 -5; 0.6 1.5 1.5 -5.4; 1.2 -0.3 -0.3 2.4] A = 3.000000000000000 2.000000000000000 2.000000000000000 -5.000000000000000 0.600000000000000 1.500000000000000 1.500000000000000 -5.400000000000000 1.200000000000000 -0.300000000000000 -0.300000000000000 2.400000000000000 >> b=[8; 2.7; 2.1] b = 8.000000000000000 2.700000000000000 2.100000000000000 >> r=rank(A) r = 2 >> rref([A b]) ans = 1 0 0 1 2 0 1 1 -4 1 0 0 0 0 0 Linear Independence and Rank of a Matrix Consider m vectors , ,⋯⋯, which have the same number of components. A linear combination of these m vectors is defined as + + ⋯⋯+ where , , ⋯ ⋯ , are any scalars. The set of vectors , ,⋯⋯, is linearly independent if and only if when ’s are all zero i.e. = = ⋯⋯ = = 0. + + ⋯⋯+ =0 The set of vectors , ,⋯⋯, is linearly dependent if and only if + + ⋯⋯+ =0 with scalars not all zero (i.e. at least one scalar is not zero). Then it is possible to express at least one of the vectors as a linear combination of the other vectors. For instance, if ≠ 0, then = + ⋯⋯+ where = − / (some ’s can be zero). Example 6: Consider the following vectors: = [1 1 −2 −2], = [2 −3 0 2], = [−2 0 2 2], = [3 −3 a) Determine whether these four vectors are linearly independent or linearly dependent. b) Determine whether the first three vectors are linearly independent or linearly dependent. Solution 6: a) Solve + + + −2 2] =0 +2 −2 +3 = 0 −3 +0 −3 = 0 −2 + 0 + 2 − 2 = 0 −2 + 2 + 2 + 2 = 0 ⟹ 1 2 −2 −3 0 1 = [ ]|0 = −2 0 2 −2 2 2 3 | 0 −3 | 0 −2 | 0 2 | 0 2 − ℎ 1 0 0 3 | 0 | ⟹ 0 1 0 2 0 0 0 1 2 | 0 0 0 0 0 | 0 [ ] = 3) < ( = = 4), the system is consistent and there are infinitely many Note that ( = solutions. Choose = , then = −2 , = −2 and = −3 . Let = 1, then −3 − 2 − 2 + = 0. Thus, the given four vectors are linearly dependent. b) Solve + + =0 +2 −2 =0 −3 +0 =0 −2 + 0 + 2 = 0 −2 + 2 + 2 = 0 ℎ 1 2 −2 | 0 0 | 0 ⟹ [ ] = [ ] | 0 = 1 −3 −2 0 2 | 0 −2 2 2 | 0 | 1 0 0 0 | − ⟹ 0 1 0 0 [ ] 0 0 1 | 0 0 0 0 | 0 [ ] = = 3) < ( = 4) and there is unique solution which is = See that ( = is a trivial solution, therefore , and form a set of linearly independent vectors. Notice that in both parts (a) and (b), there is a homogenous system. = = 0. This → The rank of a matrix [ ] is denoted by rank [ ] and it is the maximum number of linearly independent row vectors of [ ]. Theorem: Rank does not change as a result of elementary row operations; then row-equivalent matrices have the same rank. (Note that a matrix [ ] is row-equivalent to a matrix [ ] if [ ] is obtained from [ ] by elementary row operations). → Therefore, the rank of a matrix can be determined by reducing the matrix to row-echolon form and when the matrix is in row-echolon form, we count the number of nonzero rows which is exactly the rank of the matrix. Theorem: Consider p vectors each having q components. If the matrix formed, with these vectors as row vectors, has rank p, then these vectors are linearly independent. If that matrix has rank less than p, then these vectors are linearly dependent. (See the textbook for examples) Theorem: Consider p vectors each having q components. If the matrix formed, with these vectors as column | vectors, has rank p (i.e. | ⋯| = ), then these vectors are linearly independent. If that matrix has rank less than p, then these vectors are linearly dependent. Hence, rank [ ] is equal to the maximum number of linearly independent column vectors of [ ]. Therefore, [ ] and [ ] have the same rank. (See Example 6) Theorem: Consider p vectors each having q components. If dependent. < , then these vectors are linearly Example 7: Show that o Two nonzero parallel vectors are linearly dependent. o Three coplanar nonzero vectors are linearly dependent. o Four or more vectors in 3-D space are linearly dependent. 3 Solutions of Linear Systems: Existence, Uniqueness, Rank Rank gives complete information about the existence, uniqueness and general structure of the solution of linear systems. Consider a linear system [ ] = having m equations in n unknowns ( , , ⋯ , ). Then, the augmented matrix is = [ ]| . Theorem: [ ]= 1) The system is consistent (i.e. has at least one solution) if and only if = . [ ]= 2) The system has a unique solution if and only if = = . [ ]= 3) The system has infinitely many solutions if and only if = < . Then, arbitrary values are assigned to − unknowns in the complete solution. [ ]< 4) The system is inconsistent (i.e. has no solution) if . Then, the row echolon form of involves at least one equation of the form 0 = with ≠ 0. Example 8: 4 − 8 + 11 3 −6 +8 2 −4 +9 −2 +3 − 11 + 6 = 4 −9 +5 = 3 ⟹ +5 −4 = 2 −2 + = 1 4 3 2 1 −8 11 −11 6 | 4 −6 8 −9 5 | 3 −4 9 5 −4 | 2 −2 3 −2 1 | 1 Add -4 times row4 to row1. Add -3 times row4 to row2. Add -2 times row4 to row3. Interchange the new row1 and row4. Then you get the following matrix: 1 −2 3 −2 1 | 1 0 0 −1 −3 2 | 0 0 0 3 9 −6 | 0 0 0 −1 −3 2 | 0 Add 1/3 times row3 to rows 2 and 4. Subtract row3 from row1. Multiply row3 by 1/3. Interchange the new rows 2 and 3. Then you get the following matrix: 1 −2 0 0 0 0 0 0 0 −11 1 3 0 0 0 0 7 | 1 −2 | 0 0 | 0 0 | 0 [ ]= See that = = 2 < ( = 5). There are infinitely many solutions and − = 3 arbitrary values are present in the complete solution. So, set = , = and = . Using the first two rows (i.e. the first two equations in the row reduced matrix), =2 + 11 −7 + 1 and = −3 +2 Finally, the complete solution is: ⎡ ⎢ ⎢ ⎢ ⎣ 1 ⎤ ⎡0⎤ ⎥ ⎢ ⎥ ⎥ = ⎢0⎥ + ⎥ ⎢0⎥ ⎦ ⎣0⎦ 2 ⎡1⎤ ⎢0⎥ ⎢ ⎥+ ⎢0⎥ ⎣0⎦ 11 ⎡ 0⎤ ⎢−3⎥ ⎢ ⎥+ ⎢ 1⎥ ⎣ 0⎦ −7 ⎡ 0⎤ ⎢ 2⎥ ⎢ ⎥ ⎢ 0⎥ ⎣ 1⎦ The given system can be solved in Matlab using the following statements: >> A=[4 -8 11 -11 6; 3 -6 8 -9 5; 2 -4 9 5 -4; 1 -2 3 -2 1]; >> b=[4 3 2 1]'; >> At=[A b] 4 At = 4 -8 11 -11 6 4 3 -6 8 -9 5 3 2 -4 9 5 -4 2 1 -2 3 -2 1 1 >> r=rank(A), rt=rank(At) r = 2 rt = 2 >> rref(At) ans = 1 -2 0 -11 7 1 0 0 1 3 -2 0 0 0 0 0 0 0 0 0 0 0 0 0 Example 9: 1 −2 1 1 | 2 0 0 1 1 | 3 0 0 0 1 | −4 [ ]= For the above system, = = 3 < ( = 4). There are infinitely many solutions and − = 1 arbitrary value exists in the complete solution. From the last two equations, = −4 and = 7. Let = , then = 2 − 1. Finally, the complete solution is: −1 = 0 + 7 −4 2 1 0 0 Homogenous Linear Systems A homogenous linear system is given below where [ ] = = 0 and = [ ]|0 . + + ⋯⋯+ =0 + +⋯⋯+ =0 ⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯ + +⋯⋯+ =0 It can be seen that a homogenous linear system always has the trivial solution = =⋯= = 0 i.e. [ ]= = 0 . = 0 indicates that , thus a homogenous linear system is always consistent. Theorems: 1) The trivial solution i.e. = 0 is the unique solution if and only if [ ]= . 2) There are nontrivial solutions in addition to the trivial solution if and only if there are − arbitrary values. = [ ] < . Then, 5 3) If and are solution vectors of [ ] = 0 , then = + with any scalars )= solution vector of [ ] = 0 . This means [ ] = 0 and [ ] = 0 then [ ]( ) [ ]( [ ] [ ] [ ]( [ ] and = =0 . = + ) = 0 , thus =0 . and is a [ ] =0 4) A homogenous linear system with fewer equations than unknowns always has nontrivial solutions. (This [ ]≤ [ ] < ). is because and if < , then 5) If a nonhomogenous linear system [ ] = is consistent, then its complete solution is = + where is any particular (fixed) solution of [ ] = and is a complete solution of the homogenous [ ] system = 0 . (See that = − is the difference of any two solutions of the system [ ] = and is a solution of the system [ ] = 0 . Hence, [ ] = [ ] − = [ ] − [ ] = − = 0 ). Determinants A determinant of order n is a scalar value and it is associated with an ( × ) matrix as given below. = = = [ ]= (−1) (−1) ⋮ ⋯ ⋯ ⋮ ⋯ ⋮ = 1, 2, ⋯ , = 1, 2, ⋯ , ⋮ ( ) ( ) is called the minor of is called the cofactor of and given as = (−1) is a determinant of order ( − 1) obtained by omitting the can be expanded using any row or column Note that a determinant of 2nd order is calculated as [ ]= row and = column of [ ] − Example: [ ]= = = , = (−1) =− = (−1) , , = = , = (−1) = Example 10: 1 2 [ ]= 4 3 0 −1 1 6 3 4 2 1 . Expand using the 3rd row. 2 3 4 −2 Solution 10: 2 3 4 1 3 4 1 2 4 1 2 3 [ ] = (0) 3 2 1 − (−1) 4 2 1 + (2) 4 3 1 − (3) 4 3 2 = 150 6 4 −2 1 4 −2 1 6 −2 1 6 4 6 → In Matlab, use the function det(A) to calculate the determinant of a square matrix [ ]. Definition: Consider an ( × ) matrix [ ], the ( × ) matrix element is the cofactor of . [ ]= [ ] 1 [ ] [ ]) = ( [ ])[ ] and if [ ] =[ ] ⟹ [ ] 1 [ ] = ⋯ ⋯ ⋮ ⋯ ⋮ [ ]) = ( Theorem: [ ] is an ( × ) matrix, then [ ]( Knowing that [ ]( ⋮ [ ] is called the adjoint of [ ] whose ⋮ [ ])[ ] = ( [ ])[ ] [ ] ≠ 0, [ ] General Properties of Determinants 1) If [ ] is a triangular matrix of size ( × ), then product of the diagonal elements. 2) For every square matrix [ ], [ ] = [ ]= ⋯ i.e. [ ] is equal to the [ ]. 3) If a square matrix [ ] has a zero row or a zero column, then [ ] = 0. 4) If [ ] and [ ] are two square matrices of the same size,then det([ ][ ]) = det([ ][ ]) = 5) If a square matrix [ ] has proportional rows (or columns), then [ ] [ ] [ ] = 0. 6) Interchanging two rows multiplies the value of the determinant by -1. 7) Addition of a multiple of a row to another row does not change the value of the determinant. 8) Multiplication of a row by a constant c multiplies the value of the determinant by c. (Note that for an [ ]). ( × ) matrix [ ], det( [ ]) = 9) Properties 6, 7, 8 also apply to columns. Example 11: 3 1 −1 0 3 1 = 1 4 2 5 −1 −3 −1 1 2 2 4 3 2 3 1 2 1 5 2 Add the 3rd column to the 2nd and to the 5th columns. Add twice the 3rd column to the 4th column. Add three times the 3rd column to the 1st column to get 0 0 −1 0 3 4 1 6 = 7 6 2 7 −4 −4 −3 −4 5 3 2 7 Expand 0 3 3 2 4 by the 1st row to get 7 3 4 6 7 6 7 −4 −4 −4 5 3 7 = (−1)(−1) 3 3 2 4 Add twice the 4th column to the 1st, 2nd, and 3rd columns to get 9 10 12 3 13 12 13 3 = (−1) 0 0 0 2 13 11 15 4 Expand by the 3rd row to get 9 10 12 13 12 13 13 11 15 = −(2)(−1) Doing further simplifications, the result becomes = −166. The Relationship between Rank and Determinant A submatrix is any matrix obtained by omitting some rows and/or columns of matrix [ ] with size ( × ). Example 12: [ ]= [ ] has three (2 × 2) submatrices: , , [ ] has two (1 × 3) submatrices: [ ], [ [ ] has six (1 × 2) submatrices: [ ], [ [ ] has three (2 × 1) submatrices: [ ] has six (1 × 1) submatrices: [ , ], [ ] ], [ ], [ ], [ ], [ ] , ], [ ], [ ], [ ], [ ] → The only matrices with rank 0 are the zero matrices. Theorem: An ( × ) matrix [ ] has rank if and only if it contains an ( × ) submatrix with a nonzero determinant while the determinant of any square submatrix with more than rows contained in [ ] has a value equal to zero. Theorem: An ( × ) square matrix [ ] has rank equal to if and only if [ ] ≠ 0. → Note that elementary row operations change neither the rank nor the property of a determinant being nonzero. Example 13: [ ]= 1 3 −1 2 −1 3 4 0 −1 has 0 −2 7 = 2 because i) each of the (3 × 3) submatrices written below has determinant equal to zero. 8 1 3 −1 2 4 0 −1 1 0 , 3 −2 −1 2 3 1 4 −1 , 3 0 7 −1 −1 3 0 −1 , −2 7 2 −1 3 4 0 −1 0 −2 7 ii) not all (2 × 2) submatrices have zero determinant. For example 1 2 has nonzero determinant. 3 4 Alternative solution: It is easier to find the rank of [ ] when [ ] is transformed into row echolon form [ ] as follows. Add -3 times row1 to row2. Add 1 times row1 to row3 to get 1 2 −1 3 0 −2 3 −10 0 2 −3 10 Add 1 times row2 to row3 to get 1 2 −1 3 0 −2 3 −10 = [ ] 0 0 0 0 [ ] and [ ] are row-equivalent matrices and they have the same rank. Solution of Linear System of Equations in Matlab In Matlab, the following functions can be used to solve linear systems of equations: rref, linsolve, mldivide, \ , inv Example15: 3 − 0.1 − 0.2 = 7.85 0.1 + 7 − 0.3 = −19.3 0.3 − 0.2 + 10 = 71.4 % The given system can be expressed as Ax=b where x is the vector of unknowns. >> A=[ 3 -0.1 -0.2; 0.1 7 -0.3; 0.3 -0.2 10]; >> b=[7.85; -19.3; 71.4]; >> rref([A b]) ans = 1.000000000000000 0 0 3.000000000000000 0 1.000000000000000 0 -2.500000000000000 0 0 1.000000000000000 7.000000000000000 % rref(A) gives the row-reduced echelon form of A using Gauss Jordan elimination with partial pivoting. >> x=linsolve(A,b) x = 3.000000000000000 -2.500000000000000 7.000000000000002 % X=linsolve(A,B) solves the linear system A*X=B using LU factorization with partial pivoting when A is a square matrix and QR factorization with column pivoting otherwise. >> x=A\b x = 3.000000000000000 -2.500000000000000 7.000000000000002 9 >> x=mldivide(A,b) x = 3.000000000000000 -2.500000000000000 7.000000000000002 % X=mldivide(A,B) and X=A\B are equivalent commands and they solve the linear system A*X=B using Gauss elimination with partial pivoting. >> x=inv(A)*b x = 3.000000000000000 -2.500000000000000 7.000000000000001 % inv(A) is the inverse of the square matrix A. x=inv(A)*b can be used to solve the linear system A*x=b when A is a square matrix. However, using the command x=A\b is a better way of solving this system considering numerical accuracy and the required number of computations (i.e. execution time). % Important note: You must be careful when using mldivide, \ and linsolve if A is not a square matrix in the linear system A*X=B. For example, if A is any m-by-n matrix with m≠n, then x=A\B is the solution in the least squares sense to the underdetermined or overdetermined system of equations A*X=B. Please read the help pages for further information. Further Properties of Matrices → If [ ] and [ ] are nonsingular ( × ) matrices, then ([ ][ ]) = [ ] [ ] Similarly if [ ], [ ], ⋯ ⋯ [ ], [ ] are nonsingular ( × ) matrices, then ([ ][ ] ⋯ ⋯ [ ][ ]) = [ ] [ ] ⋯ ⋯ [ ] [ ] Proof: [ ][ ]([ ][ ]) =[ ] ⟹ [ ] [ ][ ]([ ][ ]) [ ] [ ]([ ][ ]) → ([ ] ) =[ ] [ ] ⟹ =[ ] [ ] ([ ][ ]) ⟹ [ ]([ ][ ]) =[ ] =[ ] [ ] =[ ] Proof: Let [ ] = [ ] . Using the definition of matrix inverse [ ] [ ] = [ ] ⟹ ([ ] ) [ ] Also [ ] [ ] = [ ][ ] then by comparison ([ ] ) [ ] = 1/ [ ] → Proof: [ ][ ] = [ ] ⟹ det([ ][ ] ) = → ([ ] ) [] =[ ] =[ ] ⟹ [ ] [ ] =1 = ([ ] ) → Consider that [ ], [ ], [ ] are ( × ) matrices. i) [ ][ ] = [ ][ ] does not generally mean [ ] = [ ] (even when [ ] ≠ [0]). But if [ ][ ] = [ ][ ], then [ ] = [ ]. [ ] = . Then [ ] [ ][ ] = [ ] [ ][ ] ⟹ [ ] = [ ]. Proof: [ ] exists if [ ]= and ii) [ ][ ] = [0] does not generally mean [ ] = [0] or [ ] = [0] or [ ][ ] = [0]. 10 [ ] = , then [ ][ ] = [0] means [ ] = [0]. The reason is that when However, if exists and [ ] [ ][ ] = [ ] [0] ⟹ [ ] = [0] . ([ ]) < and ([ ]) < If [ ][ ] = [0], but both [ ] ≠ [0] and [ ] ≠ [0], then are both singular). Example 16: 2 1 1 −2 0 0 [ ][ ] = = 6 3 −2 4 0 0 [ ]= ,[ ] (i.e. [ ] and [ ] iii) If [ ] is singular, then [ ][ ] and [ ][ ] are also singular. (See the proof given on pp.307 of the textbook). Symmetric, Skew-symmetric and Orthogonal Matrices Consider a real square matrix [ ]. o If [ ] = [ ] (i.e. = ), then [ ] is symmetric. o If [ ] = −[ ] (i.e. = − and = 0), then [ ] is skew-symmetric. o If [ ] = [ ] , then [ ] is orthogonal. → Any real square matrix [ ] can be written as the sum of a symmetric matrix [ ] and a skew-symmetric matrix [ ] i.e. [ ] = [ ] + [ ] where 1 = ([ ] + [ ] ) 2 1 = ([ ] − [ ] ) 2 Theorem: The eigenvalues of a symmetric matrix are real. The eigenvalues of a skew-symmetric matrix are pure imaginary or zero. Orthogonal Transformations and Orthogonal Matrices Orthogonal transformation is a transformation of the form =[ ] where [ ] is an orthogonal matrix. Theorem: The value of the inner product (dot product) of any two vectors ⃗ and ⃗ stays the same after an orthogonal transformation is performed as follows: =[ ] , =[ =[ ] ⋯ ⟹ ] ⋮ ⃗∙ ⃗ = ⃗∙ ⃗ = i.e. where and [ ] is an ( × ) orthogonal matrix This transformation also preserves the length (norm) of any vector ⃗ given by | ⃗ | = √ ⃗ ∙ ⃗ = √ Proof: [ ] = [ ] then = ([ ] ) [ ] = [ ] [ ] = [ ] = then ⃗ ∙ ⃗ = ⃗ ∙ ⃗ Theorem: A real square ( × ) matrix [ ] is orthogonal if and only if its column vectors also its row vectors) form an orthonormal system defined as: ⃗ ∙⃗ = Proof: , ,⋯, , 0 1 ,⋯, (and ≠ = are the column vectors of [ ], then [ ]=[ ] [ ]=[ ] [ ]= Notice that = , ,⋯, ⋮ [ ⋯ ⎡ ] ⟹ [ ]=⎢ ⎢ ⎣ ⋮ ⋮ ⋯ ⋯ ⋯ ⋮ ⋯ 1 0 ⋯ ⎤ ⎥= 0 1 ⋯ ⋮ ⋮ ⋯ ⎥ 0 0 ⋯ ⎦ 0 0 ⋮ 1 are all unit vectors. 11 → Show that the inverse of an orthogonal matrix [ ] is also orthogonal. Then, the column vectors of [ ] (= [ ] ) are the row vectors of [ ]. Thus, the row vectors of [ ] also form an orthonormal system. Theorem: The determinant of an orthogonal matrix is +1 or -1. ([ ][ ]) = [ ] [ ] , [ ] = [ ] and Proof: It is known that ([ ][ ] ) = ([ ][ ] ) = [ ] is orthogonal then 1 = []= [ ] [ ] = 1. ([ ] ) = ( [ ]) Theorem: If [ ] is an orthogonal matrix, then its eigenvalues are real or complex conjugates in pairs. These eigenvalues have an absolute value of 1. Exercise: 2/3 [ ] = −2/3 1/3 1/3 2/3 2/3 1/3 is orthogonal. Using [ ], verify the given properties of orthogonal matrices. 2/3 −2/3 12
0
You can add this document to your study collection(s)
Sign in Available only to authorized usersYou can add this document to your saved list
Sign in Available only to authorized users(For complaints, use another form )