Chapter 1 Systems of Linear Equations 1.1 Introduction to Systems of Linear Equations 1.2 Gaussian Elimination and Gauss-Jordan Elimination 1.3 Applications of Systems of Linear Equations ※ The linear algebra arises from solving systems of linear equations ※ The last section of each chapter illustrates how to apply the knowledge you learn in that chapter to solving problems in different fields 1.1 1.1 Introduction to Systems of Linear Equations A linear equation (線性方程式) in n variables: a1 x1 a2 x2 a3 x3 an xn b ai : real-number coefficients xi : variables needed to be solved b : real-number constant term a1: leading coefficient (領先係數) x1: leading variable (領先變數) Notes: (1) Linear equations have no products or roots of variables and no variables involved in trigonometric, exponential, or logarithmic functions (2) Variables appear only to the first power 1.2 Ex 1: Linear or Nonlinear 1 x y z 2 2 Linear (a) 3 x 2 y 7 (b) Linear (c) x1 2 x2 10 x3 x4 0 (d) (sin ) x1 4 x2 e 2 Linear Linear x is the exponent Nonlinear (e) xy z 2 (f) e x 2 y 4 Nonlinear 1 1 (h) 4 x y Nonlinear product of variables Nonlinear (g) sin x1 2 x2 3x3 0 trigonometric function not the first power 1.3 A solution (解) of a linear equation in n variables: a1 x1 a2 x2 a3 x3 an xn b x1 s1 , x2 s2 , x3 s3 , , xn sn s.t. a1s1 a2 s2 a3 s3 an sn b Solution set (解集合): the set of all solutions of a linear equation ※ In most cases, the number of solutions of a linear equation is infinite (無限大), so we need some methods to represent the solution set 1.4 Ex 2: Parametric representation (參數化表示) of a solution set x1 2 x2 4 with a solution (2, 1), i.e., x1 2, x2 1 ※ If you solve for x1 in terms of x2, you obtain x1 4 2 x2 (in this form, the variable x2 is free) By letting x2 t (the variable t is called a parameter), you can represent the solution set as x1 4 2t , x2 t , t is any real number The set representation for solutions: (4 2t, t ) | t R ※ If you choose x1 to be the free variable, the parametric representation of the solution set is ( s, 2 12 s) | s R 1.5 A system of m linear equations in n variables: a11 x1 a21 x1 a31 x1 a12 x2 a22 x2 a32 x2 am1 x1 am 2 x2 a13 x3 a23 x3 a33 x3 am3 x3 amn xn a1n xn a2 n xn a3n xn b1 b2 b3 bm A solution of a system of linear equations (線性方程式系 統) is a sequence of numbers s1, s2,…,sn that can solve each linear equation in the system 1.6 Notes: Every system of linear equations has either (1) exactly one solution (2) infinitely many solutions (3) no solution Consistent (一致性(有解)): A system of linear equations has at least one solution (for cases (1) and (2)) Inconsistent (非一致性(無解、矛盾)): A system of linear equations has no solution (for case (3)) 1.7 Ex 4: Solution of a system of linear equations in 2 variables (1) (2) (3) x y 3 x y 1 two intersecting lines exactly one solution x y 3 2x 2 y 6 two coincident lines inifinitely many solution x y 3 x y 1 two parallel lines no solution 1.8 Ex 5: Using back substitution (倒序替換法) to solve a system in row-echelon form (列梯形形式) x 2y 5 y 2 (1) (2) ※ The row-echelon form means that it follows a stair-step pattern (two variables are in Eq. (1) and the first variable x is the leading variable, and one variable is in Eq. (2) and the second variable y is the leading variable) and has leading coefficients of 1 for all equations ※ The back substitution means to solve a system in reverse order. That is, you solve Eq. (2) for y first, and then substitute the solution of y into Eq. (1). Next, you can solve x directly in Eq. (1) since x is the only unknown variable Sol: By substituting y 2 into Eq. (1), you obtain x 2(2) 5 x 1 The system has exactly one solution: x 1, y 2 1.9 Ex 6: Using back substitution to solve a system in rowechelon form x 2 y 3z 9 (1) y 3z 5 (2) z 2 (3) Sol: Substitute z 2 into (2), y can be solved as follows y 3(2) 5 y 1 and substitute y 1 and z 2 into (1) x 2( 1) 3(2) 9 x 1 The system has exactly one solution: x 1, y 1, z 2 1.10 Equivalent (等價): Two systems of linear equations are called equivalent if they have precisely the same solution set Notes: Each of the following operations (運算) on a system of linear equations produces an equivalent system O1: Interchange two equations O2: Multiply an equation by a nonzero constant O3: Add a multiple of an equation to another equation Gaussian elimination: A procedure to rewrite a system of linear equations to be in row-echelon form by using the above three operations 1.11 Ex 7: Solve a system of linear equations (consistent system) x 2 y 3z 9 (1) x 3y 4 (2) 2 x 5 y 5 z 17 (3) Sol: First, eliminate the x-terms in Eqs. (2) and (3) based on Eq. (1) (1) (2) (2) (by O3) x 2 y 3z 9 y 3z 5 2 x 5 y 5 z 17 (1) (2) (3) (3) (by O3) x 2 y 3z 9 y 3z 5 y z 1 (4) (5) 1.12 Second, eliminate the (-y)-term in Eq. (5) based on Eq. (4) (4) (5) (5) (by O3) x 2 y 3z 9 y 3z 5 2z 4 (6) (6) 12 (6) (by O2) x 2 y 3z 9 y 3z 5 z 2 Since the system of linear equations is expressed in its row- echelon form, the solution can be derived by the back substitution: x 1, y 1, z 2 (only one solution) 1.13 Ex 8: Solve a system of linear equations (inconsistent system) x1 3x2 x3 1 2 x1 x2 2 x3 2 x1 2 x2 3x3 1 (1) (2) (3) Sol: (1) ( 2) (2) (2) (by O3) (1) ( 1) (3) (3) (by O3) x1 3x2 x3 1 5 x2 5 x2 4 x3 4 x3 0 2 (4) (5) 1.14 (4) (1) (5) (5) (by O3) x1 3x2 x3 1 5 x2 4 x3 0 0 2 (a false statement) So, the system has no solution (an inconsistent system) 1.15 Ex 9: Solve a system of linear equations (infinitely many solutions) x2 x1 x1 Sol: x3 3 x3 3 x2 (1) (2) (by O1) x1 3 x3 x2 x3 x1 3 x2 0 1 1 (1) (2) (3) 1 0 1 (1) (2) (3) (1) (3) (3) (by O3) x1 3 x3 1 x2 x3 0 3 x2 3 x3 0 (4) 1.16 Since Equation (4) is the same as Equation (2), it is not necessary and can be omitted x1 x2 3x3 x3 1 0 x2 x3 , x1 1 3x3 Let x3 t , then x1 3t 1 x2 t tR x3 t This system has infinitely many solutions. 1.17 Keywords in Section 1.1: linear equation: 線性方程式 system of linear equations: 線性方程式系統 leading coefficient: 領先係數 leading variable: 領先變數 solution: 解 solution set: 解集合 free variable: 自由變數 parametric representation: 參數化表示 consistent: 一致性 (有解) inconsistent: 非一致性 (無解、矛盾) row-echelon form: 列梯形形式 equivalent: 等價 1.18 1.2 Gaussian Elimination and Gauss-Jordan Elimination mn matrix (矩陣): a11 a21 a31 a m1 a12 a22 a32 am 2 a13 a1n a23 a2 n a33 a3n am 3 amn m rows (列) n columns (行) Notes: (1) Every entry (元素) aij in the matrix is a real number (2) A matrix with m rows and n columns is said to be with size mn (3) If m n, then the matrix is called square matrix (方陣) of order n (4) For a square matrix, the entries a11, a22, …, ann are called the main (or principal) diagonal (主對角線) entries 1.19 Ex 1: Matrix [ 2] Size 1 1 ※ A scalar (any real number) can be treated as a 11 matrix. 0 0 0 0 22 1 3 0 1 2 1 4 e 2 2 7 4 3 2 Note: One very common use of matrices is to represent a system of linear equations (see the next slide) 1.20 A system of m equations in n variables: a11 x1 a21 x1 a31 x1 am1 x1 am 2 x2 a12 x2 a22 x2 a32 x2 a13 x3 a23 x3 a33 x3 am3 x3 amn xn bm a13 a1n a23 a2 n a33 a3n am 3 amn x1 x x 2 xn b1 b b 2 bm a1n xn a2 n xn a3n xn b1 b2 b3 Matrix form: Ax b a11 a21 A a31 a m1 a12 a22 a32 am 2 1.21 Coefficient matrix (係數矩陣): a11 a12 a 21 a22 a31 a32 am1 am 2 a13 a23 a33 am3 a1n a2 n a3n A amn Augmented matrix (增廣矩陣): formed by appending the constant-term vector to the right of the coefficient matrix of a system of linear equations a11 a12 a 21 a22 a31 a32 am1 am 2 a13 a23 a33 a1n a2 n a3n am3 amn b1 b2 b3 [ A b] bm 1.22 Before introducing the Gaussian elimination and the GaussJordan elimination, we need some background knowledge, including the elementary row operations (基本列運算) and the rules to identify the row-echelon or reduced row-echelon forms (簡化列梯形形式) for matrices Elementary row operations (基本列運算): (1) Interchange two rows: I i , j Ri R j (2) Multiply a row by a nonzero constant: M i( k ) (k ) Ri Ri (3) Add a multiple of a row to another row: Ai(,kj) (k ) Ri R j R j Row equivalent (列等價): Two matrices are said to be row equivalent if one can be obtained from the other by a finite sequence of above elementary row operations 1.23 Ex 2: Elementary row operations 0 1 3 4 1 2 0 3 2 3 4 1 2 4 6 2 1 3 3 0 5 2 1 2 1 2 4 3 0 3 2 1 2 1 5 2 I1,2 M 1 ( ) 2 1 ( 2) A1,3 1 2 0 3 0 1 3 4 2 3 4 1 1 2 3 1 1 3 3 0 5 2 1 2 1 2 4 3 0 3 2 1 0 3 13 8 1.24 Row-echelon form (列梯形形式): (1), (2), and (3) Reduced row-echelon form (簡化列梯形形式): (1), (2), (3), and (4) (1) All rows consisting entirely of zeros occur at the bottom of the matrix (若有全部為0之row,會出現在matrix的最下方) (2) For each row that does not consist entirely of zeros, the first nonzero entry from the left side is 1, which is called as leading 1 (對不全為0之row,從左到右,第一個出現非0的entry,其 值為1,且此entry稱為leading 1) (3) For two successive nonzero rows, the leading 1 in the higher row is further to the left than the leading 1 in the lower row (對 兩相鄰不全為0之row,越上方的row中的leading 1,會在越 左邊) (4) Every column that contains a leading 1 has zeros everywhere else (每個有leading 1的column中,其它的entry均為0) 1.25 Ex 4: Row-echelon form or reduced row-echelon form 1 2 1 4 0 1 0 3 0 0 1 2 (row-echelon form) 0 1 0 5 0 0 1 3 0 0 0 0 0 1 0 2 1 3 0 0 1 5 2 1 3 0 0 1 3 2 (row-echelon 0 0 0 1 4 form) 0 0 0 0 1 1 0 0 0 1 2 3 4 0 2 1 1 0 0 1 3 1 2 1 2 0 0 0 0 0 1 2 4 Violate the second criterion 0 1 0 0 (reduced rowechelon form) (reduced rowechelon form) Violate the first criterion 1.26 Gaussian elimination: The procedure for reducing a matrix to a row-echelon form by performing the three elementary row operations Gauss-Jordan elimination: The procedure for reducing a matrix to its reduced row-echelon form by performing the three elementary row operations Notes: (1) Every matrix has an unique reduced row-echelon form (2) A row-echelon form of a given matrix is not unique (Different sequences of elementary row operations can produce different row-echelon forms) ※ The above two statements will be verified numerically in Ex. 8 1.27 Ex: Procedure of the Gaussian elimination and Gauss-Jordan elimination Produce leading 1 0 0 2 0 7 12 2 4 10 6 12 28 2 4 5 6 5 1 I1,2 2 4 10 6 12 28 0 0 2 0 7 12 2 4 5 6 5 1 The first nonzero column Leading 1 M 1 1 2 5 3 6 14 0 0 2 0 7 12 2 4 5 6 5 1 ( 12 ) Produce leading 1 ( 2) 6 14 1 2 5 3 A1,3 7 12 0 0 2 0 0 0 5 0 17 29 Nonzero entries below leading 1 The first nonzero column Submatrix 1.28 The first nonzero column Leading 1 M ( 12 ) 2 6 14 1 2 5 3 0 0 1 0 7 2 6 0 0 5 0 17 29 6 14 1 2 5 3 0 0 1 0 7 2 6 0 0 0 0 12 1 ( 5) 2,3 A Nonzero entries below leading 1 Produce Submatrix leading 1 Non-zeros M (2) 3 6 14 1 2 5 3 0 0 1 0 7 2 6 0 0 0 0 1 2 ( 6) 3,1 A ( 72 ) 3,2 A (5) A2,1 1 2 0 3 0 7 0 0 1 0 0 1 0 0 0 0 1 2 Leading 1 (row-echelon form) (reduced rowechelon form) 1.29 Ex 7: Solve a system by the Gauss-Jordan elimination method (only one solution) x 2 y 3z 9 x 3y 4 2 x 5 y 5z 17 Sol: Form the augmented matrix by concatenating A and b 1 2 3 9 A(1) A( 2) 1 2 3 9 A(1) M (1/ 2) 1 2 3 9 2,3 3 1 3 0 4 1,2 1,3 0 0 1 3 5 1 3 5 2 5 5 17 0 1 1 1 0 0 1 2 (row-echelon form) 1 ( 3) ( 3) (2) 1 0 0 A3,2 A3,1 A2,1 x 1 y 1 0 1 0 1 0 0 1 2 z 2 (reduced row-echelon form) 1.30 Ex 8:Solve a system by the Gauss-Jordan elimination method (infinitely many solutions) 2 x1 4 x2 2 x3 0 3 x1 5 x2 1 Sol: For the augmented matrix, 2 4 2 3 5 0 1 M 2( 1) 0 0 M1( 12 ) 1 2 1 0 A1,2( 3) 1 2 1 0 1 3 5 0 1 0 1 3 1 2 1 0 A2,1( 2) 1 0 5 2 1 3 1 0 1 3 1 (row-echelon form) (reduced row-echelon form) 1.31 Then the system of linear equations becomes x1 5 x3 2 x2 3x3 1 It can be further reduced to x1 2 5 x3 x 2 1 3 x3 Let x3 t , then x1 2 5t , x2 1 3t , tR x3 t , So, this system has infinitely many solutions 1.32 ※ In order to show that the reduced row-echelon form is unique, I conduct an experiment by performing a redundant elementary row operation of interchanging the first and second rows in advance 2 4 2 3 5 0 ( 2 ) 1 A1,2 0 0 I1,2 3 5 0 1 M1(1/ 3) 1 5 / 3 0 1/ 3 1 2 4 2 0 2 4 2 0 5/3 0 1/ 3 M 2( 3 / 2 ) 1 5 / 3 0 1/ 3 2 / 3 2 2 / 3 0 1 3 1 1 0 5 2 0 1 3 1 ( 5 / 3) A2,1 (row-echelon form) (reduced row-echelon form) ※ Comparing with the results on Slide 1.31, we can infer that it is possible to derive different row-echelon forms, but there is a unique reduced row-echelon form for each matrix 1.33 Homogeneous systems of linear equations (齊次線性系統) : A system of linear equations is said to be homogeneous if all the constant terms are zero a11 x1 a12 x2 a21 x1 a22 x2 a31 x1 a32 x2 am1 x1 am 2 x2 a13 x3 a1n xn 0 a23 x3 a2 n xn 0 a33 x3 a3n xn 0 am 3 x3 amn xn 0 1.34 Trivial (obvious) solution (顯然解): x1 x2 x3 xn 0 Nontrivial solution (非顯然解): other solutions Theorem 1.1 (1) Every homogeneous system of linear equations is consistent. Furthermore, for a homogeneous system, exactly one of the following is true: (a) The system has only the trivial solution (b) The system has infinitely many nontrivial solutions in addition to the trivial solution (In other words, if a system has any nontrivial solution, this system must have infinitely many nontrivial solutions) (2) If the homogenous system has fewer equations than variables, then it must have an infinite number of solutions 1.35 Ex 9: Solve the following homogeneous system (also verify the two statements in Theorem 1.1 numerically) x1 x2 2 x1 x2 3x3 3x3 0 0 Sol: For the augmented matrix, ( 13 ) ( 2) (1) A M A 1,2 2 2,1 1 1 3 0 2 1 3 0 x1 x2 2 x3 x3 1 0 2 0 0 1 1 0 (reduced rowechelon form) 0 0 Let x3 t, then x1 2t , x2 t , x3 t , t R When t 0, x1 x2 x3 0 (trivial solution) When t 0, there are infinitely many nontrivial solutions 1.36 Keywords in Section 1.2: matrix: 矩陣 row: 列 column: 行 entry: 元素 size: 大小 square matrix: 方陣 order: 階 main diagonal: 主對角線 coefficient matrix: 係數矩陣 augmented matrix: 增廣矩陣 1.37 elementary row operation: 基本列運算 row equivalent: 列等價 row-echelon form: 列梯形形式 reduced row-echelon form: 簡化列梯形形式 leading 1: 領先1 Gaussian elimination: 高斯消去法 Gauss-Jordan elimination: 高斯-喬登消去法 homogeneous system: 齊次系統 trivial solution: 顯然解 nontrivial solution: 非顯然解 1.38 1.3 Applications of Systems of Linear Equations Polynomial Curve Fitting in Examples 1, 2, and 4 See the text book or “Applications in Ch1.pdf” downloaded from my website The polynomial curve fitting problem is that given n points (x1, y1), (x2, y2),…, (xn, yn) on the xy-plane, find a polynomial function of degree n–1 passing through these n points Once equipped with this polynomial function, we can predict the value of y for some missing values of x A typical application of the curve fitting problem in the field of finance is to derive the interest rate (y) for different time to maturity (x) 1.39