Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Math 407A: Linear Optimization Lecture 2 Math Dept, University of Washington January 7, 2010 Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Linear Programing Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Solving Systems of Linear equations Let A ∈ Rm×n and b ∈ Rm . Find all solutions x ∈ Rn to the system Ax = b. . Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Solving Systems of Linear equations Let A ∈ Rm×n and b ∈ Rm . Find all solutions x ∈ Rn to the system Ax = b. The set of solutions is either . Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Solving Systems of Linear equations Let A ∈ Rm×n and b ∈ Rm . Find all solutions x ∈ Rn to the system Ax = b. The set of solutions is either empty, . Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Solving Systems of Linear equations Let A ∈ Rm×n and b ∈ Rm . Find all solutions x ∈ Rn to the system Ax = b. The set of solutions is either empty, a single point, . Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Solving Systems of Linear equations Let A ∈ Rm×n and b ∈ Rm . Find all solutions x ∈ Rn to the system Ax = b. The set of solutions is either empty, a single point, or an infinite set. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Solving Systems of Linear equations Let A ∈ Rm×n and b ∈ Rm . Find all solutions x ∈ Rn to the system Ax = b. The set of solutions is either empty, a single point, or an infinite set. If a solution x0 ∈ Rn exists, then the set of solutions is given by x0 + Nul (A) . Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination and the 3 Elementary Row Operations We solve the system Ax = b by transforming the augmented matrix [ A | b] into upper echelon form using the three elementary row operations. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination and the 3 Elementary Row Operations We solve the system Ax = b by transforming the augmented matrix [ A | b] into upper echelon form using the three elementary row operations. This process is called Gaussian elimination. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination and the 3 Elementary Row Operations We solve the system Ax = b by transforming the augmented matrix [ A | b] into upper echelon form using the three elementary row operations. This process is called Gaussian elimination. The three elementary row operations. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination and the 3 Elementary Row Operations We solve the system Ax = b by transforming the augmented matrix [ A | b] into upper echelon form using the three elementary row operations. This process is called Gaussian elimination. The three elementary row operations. 1. Interchange any two rows. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination and the 3 Elementary Row Operations We solve the system Ax = b by transforming the augmented matrix [ A | b] into upper echelon form using the three elementary row operations. This process is called Gaussian elimination. The three elementary row operations. 1. Interchange any two rows. 2. Multiply any row by a non-zero constant. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination and the 3 Elementary Row Operations We solve the system Ax = b by transforming the augmented matrix [ A | b] into upper echelon form using the three elementary row operations. This process is called Gaussian elimination. The three elementary row operations. 1. Interchange any two rows. 2. Multiply any row by a non-zero constant. 3. Replace any row by itself plus a multiple of any other row. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination and the 3 Elementary Row Operations We solve the system Ax = b by transforming the augmented matrix [ A | b] into upper echelon form using the three elementary row operations. This process is called Gaussian elimination. The three elementary row operations. 1. Interchange any two rows. 2. Multiply any row by a non-zero constant. 3. Replace any row by itself plus a multiple of any other row. These elementary row operations can be interpreted as multiplying the augmented matrix on the left by a special matrix. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Exchange and Permutation Matrices An exchange matrix is given by permuting any two columns of the identity. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Exchange and Permutation Matrices An exchange matrix is given by permuting any two columns of the identity. Multiplying any 4 × n matrix on the left by the exchange matrix 1 0 0 0 0 0 0 1 0 0 1 0 0 1 0 0 will exchange the second and fourth rows of the matrix. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Exchange and Permutation Matrices An exchange matrix is given by permuting any two columns of the identity. Multiplying any 4 × n matrix on the left by the exchange matrix 1 0 0 0 0 0 0 1 0 0 1 0 0 1 0 0 will exchange the second and fourth rows of the matrix. (multiplication of a m × 4 matrix on the right by this exchanges the second and fourth columns.) Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Exchange and Permutation Matrices An exchange matrix is given by permuting any two columns of the identity. Multiplying any 4 × n matrix on the left by the exchange matrix 1 0 0 0 0 0 0 1 0 0 1 0 0 1 0 0 will exchange the second and fourth rows of the matrix. (multiplication of a m × 4 matrix on the right by this exchanges the second and fourth columns.) A permutation matrix is obtained by permuting the columns of the identity matrix. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Notes on Matrix Multiplication Let A = [aij ]m×n ∈ Rm×n . Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Notes on Matrix Multiplication Let A = [aij ]m×n ∈ Rm×n . Left Multiplication of A: Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Notes on Matrix Multiplication Let A = [aij ]m×n ∈ Rm×n . Left Multiplication of A: When multiplying A on the left by an m × m matrix M, it is often useful to think of this as an action on the rows of A. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Notes on Matrix Multiplication Let A = [aij ]m×n ∈ Rm×n . Left Multiplication of A: When multiplying A on the left by an m × m matrix M, it is often useful to think of this as an action on the rows of A. For example, left multiplication by a permutation matrix permutes the rows of the matrix. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Notes on Matrix Multiplication Let A = [aij ]m×n ∈ Rm×n . Left Multiplication of A: When multiplying A on the left by an m × m matrix M, it is often useful to think of this as an action on the rows of A. For example, left multiplication by a permutation matrix permutes the rows of the matrix. However, mechanically, left multiplication corresponds to matrix vector multiplication on the columns. MA = [Ma•1 Ma•2 · · · Ma•n ] Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Notes on Matrix Multiplication Let A = [aij ]m×n ∈ Rm×n . Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Notes on Matrix Multiplication Let A = [aij ]m×n ∈ Rm×n . Right Multiplication of A: Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Notes on Matrix Multiplication Let A = [aij ]m×n ∈ Rm×n . Right Multiplication of A: When multiplying A on the right by an n × n matrix N, it is often useful to think of this as an action on the columns of A. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Notes on Matrix Multiplication Let A = [aij ]m×n ∈ Rm×n . Right Multiplication of A: When multiplying A on the right by an n × n matrix N, it is often useful to think of this as an action on the columns of A. For example, right multiplication by a permutation matrix permutes the columns of the matrix. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Notes on Matrix Multiplication Let A = [aij ]m×n ∈ Rm×n . Right Multiplication of A: When multiplying A on the right by an n × n matrix N, it is often useful to think of this as an action on the columns of A. For example, right multiplication by a permutation matrix permutes the columns of the matrix. However, mechanically, right multiplication corresponds to left matrix vector multiplication on the rows. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Notes on Matrix Multiplication Let A = [aij ]m×n ∈ Rm×n . Right Multiplication of A: When multiplying A on the right by an n × n matrix N, it is often useful to think of this as an action on the columns of A. For example, right multiplication by a permutation matrix permutes the columns of the matrix. a1• N However, mechanically, right a2• N multiplication corresponds to AN = . .. left matrix vector multiplication am• N on the rows. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination Matrices The key step in Gaussian elimination is to transform a vector of the form a α , b where a ∈ Rk , 0 6= α ∈ R, and b ∈ Rn−k−1 , into one of the form a α . 0 Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination Matrices The key step in Gaussian elimination is to transform a vector of the form a α , b where a ∈ Rk , 0 6= α ∈ R, and b ∈ Rn−k−1 , into one of the form a α . 0 This can be accomplished by left matrix multiplication as follows. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination Matrices a ∈ Rk , 0 6= α ∈ R, and b ∈ Rn−k−1 Ik×k 0 0 0 0 a α 1 0 −1 −α b I(n−k−1)×(n−k−1) b Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination Matrices a ∈ Rk , 0 6= α ∈ R, and b ∈ Rn−k−1 Ik×k 0 0 0 0 a α = 1 0 −1 −α b I(n−k−1)×(n−k−1) b Lecture 2 Math 407A: Linear Optimization . Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination Matrices a ∈ Rk , 0 6= α ∈ R, and b ∈ Rn−k−1 Ik×k 0 0 0 0 a a . α = 1 0 −1 −α b I(n−k−1)×(n−k−1) b Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination Matrices a ∈ Rk , 0 6= α ∈ R, and b ∈ Rn−k−1 Ik×k 0 0 0 0 a a α = α . 1 0 −1 −α b I(n−k−1)×(n−k−1) b Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination Matrices a ∈ Rk , 0 6= α ∈ R, and b ∈ Rn−k−1 Ik×k 0 0 0 0 a a α = α . 1 0 −1 −α b I(n−k−1)×(n−k−1) 0 b Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination Matrices The matrix Ik×k 0 0 0 0 1 0 −1 −α b I(n−k−1)×(n−k−1) is called a Gaussian elimination matrix. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination Matrices The matrix Ik×k 0 0 0 0 1 0 −1 −α b I(n−k−1)×(n−k−1) is called a Gaussian elimination matrix. This matrix is invertible with inverse Ik×k 0 0 0 . 1 0 −1 0 α b I(n−k−1)×(n−k−1) Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination Matrices The matrix Ik×k 0 0 0 0 1 0 −1 −α b I(n−k−1)×(n−k−1) is called a Gaussian elimination matrix. This matrix is invertible with inverse Ik×k 0 0 0 . 1 0 −1 0 α b I(n−k−1)×(n−k−1) Note that a Gaussian elimination matrix and its inverse are both lower triangular matrices. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Matrix Sub-Algebras Lower (upper) triangular matrices in Rn×n are said to form a sub-algebra of Rn×n . Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Matrix Sub-Algebras Lower (upper) triangular matrices in Rn×n are said to form a sub-algebra of Rn×n . A subset S of Rn×n is said to be a sub-algebra of Rn×n if Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Matrix Sub-Algebras Lower (upper) triangular matrices in Rn×n are said to form a sub-algebra of Rn×n . A subset S of Rn×n is said to be a sub-algebra of Rn×n if I S is a subspace of Rn×n , Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Matrix Sub-Algebras Lower (upper) triangular matrices in Rn×n are said to form a sub-algebra of Rn×n . A subset S of Rn×n is said to be a sub-algebra of Rn×n if I S is a subspace of Rn×n , I S is closed wrt matrix multiplication, and Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Matrix Sub-Algebras Lower (upper) triangular matrices in Rn×n are said to form a sub-algebra of Rn×n . A subset S of Rn×n is said to be a sub-algebra of Rn×n if I S is a subspace of Rn×n , I S is closed wrt matrix multiplication, and I if M ∈ S is invertible, then M −1 ∈ S. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Transformation to echelon (upper triangular) form. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Transformation to echelon (upper triangular) form. Lecture 2 Math 407A: Linear Optimization 1 1 2 A = 2 4 2 . −1 1 3 Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Transformation to echelon (upper triangular) form. 1 1 2 A = 2 4 2 . −1 1 3 Eliminate the first column with a Gaussian elimination matrix. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Transformation to echelon (upper triangular) form. Eliminate the first column with a Gaussian elimination matrix. Lecture 2 Math 407A: Linear Optimization 1 1 2 A = 2 4 2 . −1 1 3 1 0 0 G1 = −2 1 0 1 0 1 Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Transformation to echelon (upper triangular) form. Eliminate the first column with a Gaussian elimination matrix. 1 1 2 A = 2 4 2 . −1 1 3 1 0 0 G1 = −2 1 0 1 0 1 1 0 0 1 1 2 G1 A = −2 1 0 2 4 2 1 0 1 −1 1 3 Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Transformation to echelon (upper triangular) form. Eliminate the first column with a Gaussian elimination matrix. 1 1 2 A = 2 4 2 . −1 1 3 1 0 0 G1 = −2 1 0 1 0 1 1 0 0 1 1 2 G1 A = −2 1 0 2 4 2 = 1 0 1 −1 1 3 Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Transformation to echelon (upper triangular) form. Eliminate the first column with a Gaussian elimination matrix. 1 1 2 A = 2 4 2 . −1 1 3 1 0 0 G1 = −2 1 0 1 0 1 1 1 0 0 1 1 2 2 4 2 G1 A = −2 1 0 = 1 0 1 −1 1 3 Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Transformation to echelon (upper triangular) form. Eliminate the first column with a Gaussian elimination matrix. 1 1 2 A = 2 4 2 . −1 1 3 1 0 0 G1 = −2 1 0 1 0 1 1 1 1 0 0 1 1 2 2 4 2 G1 A = −2 1 0 = 1 0 1 −1 1 3 Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Transformation to echelon (upper triangular) form. Eliminate the first column with a Gaussian elimination matrix. 1 1 2 A = 2 4 2 . −1 1 3 1 0 0 G1 = −2 1 0 1 0 1 1 1 1 0 0 1 1 2 2 4 2 G1 A = −2 1 0 = 1 0 1 −1 1 3 Lecture 2 Math 407A: Linear Optimization 2 Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Transformation to echelon (upper triangular) form. Eliminate the first column with a Gaussian elimination matrix. 1 1 2 A = 2 4 2 . −1 1 3 1 0 0 G1 = −2 1 0 1 0 1 1 1 1 0 0 1 1 2 2 4 2 G1 A = −2 1 0 = 0 1 0 1 −1 1 3 Lecture 2 Math 407A: Linear Optimization 2 Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Transformation to echelon (upper triangular) form. Eliminate the first column with a Gaussian elimination matrix. 1 1 2 A = 2 4 2 . −1 1 3 1 0 0 G1 = −2 1 0 1 0 1 1 1 1 0 0 1 1 2 2 4 2 G1 A = −2 1 0 = 0 2 1 0 1 −1 1 3 Lecture 2 Math 407A: Linear Optimization 2 Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Transformation to echelon (upper triangular) form. Eliminate the first column with a Gaussian elimination matrix. 1 1 2 A = 2 4 2 . −1 1 3 1 0 0 G1 = −2 1 0 1 0 1 1 1 1 0 0 1 1 2 2 4 2 G1 A = −2 1 0 = 0 2 1 0 1 −1 1 3 Lecture 2 Math 407A: Linear Optimization 2 −2 Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Transformation to echelon (upper triangular) form. Eliminate the first column with a Gaussian elimination matrix. 1 1 2 A = 2 4 2 . −1 1 3 1 0 0 G1 = −2 1 0 1 0 1 1 1 1 0 0 1 1 2 2 4 2 G1 A = −2 1 0 = 0 2 0 2 1 0 1 −1 1 3 Lecture 2 Math 407A: Linear Optimization 2 −2 5 Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Now do Gaussian 1 0 0 Lecture 2 Math 407A: Linear Optimization eliminiation on the second column. 1 2 2 −2 2 5 Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Now do Gaussian 1 0 0 Lecture 2 Math 407A: Linear Optimization eliminiation on the second column. 1 0 0 1 2 1 0 2 −2 G2 = 0 0 −1 1 2 5 Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Now do Gaussian 1 0 0 eliminiation on the second column. 1 0 0 1 2 1 0 2 −2 G2 = 0 0 −1 1 2 5 1 0 0 1 1 2 0 1 0 0 2 −2 = 0 −1 1 0 2 5 Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Now do Gaussian 1 0 0 eliminiation on the second column. 1 0 0 1 2 1 0 2 −2 G2 = 0 0 −1 1 2 5 1 1 0 0 1 1 2 0 1 0 0 2 −2 = 0 −1 1 0 2 5 Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Now do Gaussian 1 0 0 eliminiation on the second column. 1 0 0 1 2 1 0 2 −2 G2 = 0 0 −1 1 2 5 1 1 0 0 1 1 2 0 1 0 0 2 −2 = 0 0 0 −1 1 0 2 5 Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Now do Gaussian 1 0 0 eliminiation on the second column. 1 0 0 1 2 1 0 2 −2 G2 = 0 0 −1 1 2 5 1 1 1 0 0 1 1 2 0 1 0 0 2 −2 = 0 0 0 −1 1 0 2 5 Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Now do Gaussian 1 0 0 eliminiation on the second column. 1 0 0 1 2 1 0 2 −2 G2 = 0 0 −1 1 2 5 1 1 1 0 0 1 1 2 0 1 0 0 2 −2 = 0 0 0 −1 1 0 2 5 Lecture 2 Math 407A: Linear Optimization 2 Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Now do Gaussian 1 0 0 eliminiation on the second column. 1 0 0 1 2 1 0 2 −2 G2 = 0 0 −1 1 2 5 1 1 1 0 0 1 1 2 0 1 0 0 2 −2 = 0 2 0 0 −1 1 0 2 5 Lecture 2 Math 407A: Linear Optimization 2 Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Now do Gaussian 1 0 0 eliminiation on the second column. 1 0 0 1 2 1 0 2 −2 G2 = 0 0 −1 1 2 5 1 1 1 0 0 1 1 2 0 1 0 0 2 −2 = 0 2 0 0 −1 1 0 2 5 Lecture 2 Math 407A: Linear Optimization 2 −2 Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gaussian Elimination in Practice Now do Gaussian 1 0 0 eliminiation on the second column. 1 0 0 1 2 1 0 2 −2 G2 = 0 0 −1 1 2 5 1 1 1 0 0 1 1 2 0 1 0 0 2 −2 = 0 2 0 0 0 −1 1 0 2 5 Lecture 2 Math 407A: Linear Optimization 2 −2 7 Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gauss-Jordan Elimination, or Pivot Matrices What happens in the following multiplication? Ik×k −α−1 a 0 a 0 α α−1 0 −1 0 −α b I(n−k−1)×(n−k−1) b Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gauss-Jordan Elimination, or Pivot Matrices What happens in the following multiplication? Ik×k −α−1 a 0 a 0 α = α−1 0 −1 0 −α b I(n−k−1)×(n−k−1) b Lecture 2 Math 407A: Linear Optimization . Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gauss-Jordan Elimination, or Pivot Matrices What happens in the following multiplication? Ik×k −α−1 a 0 0 a 0 α = . α−1 0 0 −α−1 b I(n−k−1)×(n−k−1) b Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gauss-Jordan Elimination, or Pivot Matrices What happens in the following multiplication? Ik×k −α−1 a 0 0 a 0 α = 1 . α−1 0 0 −α−1 b I(n−k−1)×(n−k−1) b Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gauss-Jordan Elimination, or Pivot Matrices What happens in the following multiplication? Ik×k −α−1 a 0 0 a 0 α = 1 . α−1 0 0 −α−1 b I(n−k−1)×(n−k−1) 0 b Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gauss-Jordan Elimination, or Pivot Matrices What happens in the following multiplication? Ik×k −α−1 a 0 0 a 0 α = 1 . α−1 0 0 −α−1 b I(n−k−1)×(n−k−1) 0 b What is the inverse of this matrix? Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Gauss-Jordan Elimination, or Pivot Matrices What happens in the following multiplication? Ik×k −α−1 a 0 0 a 0 α = 1 . α−1 0 0 −α−1 b I(n−k−1)×(n−k−1) 0 b What is the inverse of this matrix? Ik×k a 0 . 0 α 0 0 b I(n−k−1)×(n−k−1) Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Math 407 Begins! MATH 407 NOW BEGINS!! Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line What is optimization? Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line What is optimization? A mathematical optimization problem is one in which some function is either maximized or minimized relative to a given set of alternatives. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line What is optimization? A mathematical optimization problem is one in which some function is either maximized or minimized relative to a given set of alternatives. I The function to be minimized or maximized is called the objective function. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line What is optimization? A mathematical optimization problem is one in which some function is either maximized or minimized relative to a given set of alternatives. I The function to be minimized or maximized is called the objective function. I The set of alternatives is called the feasible region (or constraint region). Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line What is optimization? A mathematical optimization problem is one in which some function is either maximized or minimized relative to a given set of alternatives. I The function to be minimized or maximized is called the objective function. I The set of alternatives is called the feasible region (or constraint region). I In this course, the feasible region is always taken to be a subset of Rn (real n-dimensional space) and the objective function is a function from Rn to R. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line What is linear programming (LP)? Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line What is linear programming (LP)? A linear program is an optimization problem in finitely many variables having a linear objective function and a constraint region determined by a finite number of linear equality and/or inequality constraints. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line What is linear programming (LP)? Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line What is linear programming (LP)? A linear program is an optimization problem Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line What is linear programming (LP)? A linear program is an optimization problem in finitely many variables Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line What is linear programming (LP)? A linear program is an optimization problem in finitely many variables having a linear objective function Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line What is linear programming (LP)? A linear program is an optimization problem in finitely many variables having a linear objective function and a constraint region determined by a finite number of constraints Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line What is linear programming (LP)? A linear program is an optimization problem in finitely many variables having a linear objective function and a constraint region determined by a finite number of constraints that are linear equality and/or linear inequality constraints. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line I A linear function of the variables x1 , x2 , . . . , xn is any function of the form c 1 x1 + c 2 x2 + · · · + c n xn for fixed ci ∈ R i = 1, . . . , n. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line I I A linear function of the variables x1 , x2 , . . . , xn is any function of the form c 1 x1 + c 2 x2 + · · · + c n xn for fixed ci ∈ R i = 1, . . . , n. A linear equality constraint is any equation of the form a1 x1 + a2 x2 + · · · + an xn = α, where α, a1 , a2 , . . . , an ∈ R. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line I I A linear function of the variables x1 , x2 , . . . , xn is any function of the form c 1 x1 + c 2 x2 + · · · + c n xn for fixed ci ∈ R i = 1, . . . , n. A linear equality constraint is any equation of the form a1 x1 + a2 x2 + · · · + an xn = α, I where α, a1 , a2 , . . . , an ∈ R. A linear equality constraint is any inequality of the form a1 x1 + a2 x2 + · · · + an xn ≤ α, or a1 x1 + a2 x2 + · · · + an xn ≥ α, where α, a1 , a2 , . . . , an ∈ R. Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Compact Representation minimize c 1 x1 + c 2 x2 + · · · + c n xn subject to ai1 xi + ai2 x2 + · · · + ain xn ≤ αi bi1 xi + bi2 x2 + · · · + bin xn = βi Lecture 2 Math 407A: Linear Optimization i = 1, . . . , s i = 1, . . . , r . Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Vector Inequalities: Componentwise Let x, y ∈ Rn . x = x1 x2 .. . xn y = y1 y2 .. . yn We write x ≤ y if and only if xi ≤ yi , i = 1, 2, . . . , n . Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Matrix Notation c 1 x1 + c 2 x2 + · · · + c n xn = c T x c = c1 c2 .. . cn Lecture 2 Math 407A: Linear Optimization x = x1 x2 .. . xn Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Matrix Notation ai1 xi + ai2 x2 + · · · + ain xn ≤ αi i = 1, . . . , s ⇐⇒ Ax ≤ a A= Lecture 2 Math 407A: Linear Optimization a11 a12 . . . a1n a21 a22 . . . a2n .. .. . . .. . . . . as1 as2 . . . asn a= α1 α2 .. . αn Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Matrix Notation bi1 xi + bi2 x2 + · · · + bin xn = βi i = 1, . . . , r ⇐⇒ Bx = b B= Lecture 2 Math 407A: Linear Optimization b11 b12 . . . b1n b21 b22 . . . b2n .. .. . . .. . . . . br 1 br 2 . . . brn b= β1 β2 .. . βs Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line LP’s Matrix Notation minimize c T x subject to Ax ≤ a and Bx = b c1 α1 β1 .. .. .. c = . , a = . , b = . cn αn βs a11 . . . a1n b11 . . . . .. .. A= , B = . as1 . . . asn Lecture 2 Math 407A: Linear Optimization . b1n br 1 . . . brn Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Applications of Linear Programing A short list: Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Applications of Linear Programing A short list: I resource allocation Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Applications of Linear Programing A short list: I resource allocation I production scheduling Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Applications of Linear Programing A short list: I resource allocation I production scheduling I warehousing Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Applications of Linear Programing A short list: I resource allocation I production scheduling I warehousing I layout design Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Applications of Linear Programing A short list: I resource allocation I production scheduling I warehousing I layout design I transportation scheduling Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Applications of Linear Programing A short list: I resource allocation I production scheduling I warehousing I layout design I transportation scheduling I facility location Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Applications of Linear Programing A short list: I resource allocation I production scheduling I warehousing I layout design I transportation scheduling I facility location I supply chain management Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Applications of Linear Programing A short list: I I resource allocation I production scheduling I warehousing I layout design I transportation scheduling I facility location I supply chain management Lecture 2 Math 407A: Linear Optimization flight crew scheduling Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Applications of Linear Programing A short list: I resource allocation I production scheduling I warehousing I layout design I transportation scheduling I facility location I supply chain management Lecture 2 Math 407A: Linear Optimization I flight crew scheduling I portfolio optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Applications of Linear Programing A short list: I flight crew scheduling I resource allocation I portfolio optimization I production scheduling I cash flow matching I warehousing I layout design I transportation scheduling I facility location I supply chain management Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Applications of Linear Programing A short list: I flight crew scheduling I resource allocation I portfolio optimization I production scheduling I cash flow matching I warehousing I I layout design currency exchange arbitrage I transportation scheduling I facility location I supply chain management Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Applications of Linear Programing A short list: I flight crew scheduling I resource allocation I portfolio optimization I production scheduling I cash flow matching I warehousing I I layout design currency exchange arbitrage I transportation scheduling I crop scheduling I facility location I supply chain management Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Applications of Linear Programing A short list: I flight crew scheduling I resource allocation I portfolio optimization I production scheduling I cash flow matching I warehousing I I layout design currency exchange arbitrage I transportation scheduling I crop scheduling I facility location I diet balancing I supply chain management Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Applications of Linear Programing A short list: I flight crew scheduling I resource allocation I portfolio optimization I production scheduling I cash flow matching I warehousing I I layout design currency exchange arbitrage I transportation scheduling I crop scheduling I facility location I diet balancing I supply chain management I parameter estimation Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington Outline Gaussian Elimination Matrices Gauss-Jordan Elimination (Pivoting) What is linear programming? Applications of Line Applications of Linear Programing A short list: I flight crew scheduling I resource allocation I portfolio optimization I production scheduling I cash flow matching I warehousing I I layout design currency exchange arbitrage I transportation scheduling I crop scheduling I facility location I diet balancing I supply chain management I parameter estimation I ... Lecture 2 Math 407A: Linear Optimization Math Dept, University of Washington