CHEE 3334
Numerical and Statistical Methods
in Chemical Engineering
Solving Algebraic Equations: An (almost) Self-Contained Introduction
Lecture Notes
© Michael Nikolaou
Chemical & Biomolecular Engineering Department
CHEE 3334 Lecture Notes
Solving Algebraic Equations
Michael Nikolaou
TABLE OF CONTENTS
1.
2.
3.
4.
5.
6.
7.
1.1
1.2
1.3
2.1
2.2
2.3
2.4
2.5
2.6
2.7
3.1
3.3
3.4
3.5
3.6
3.7
3.8
4.1
4.2
4.3
4.4
4.5
4.6
4.7
5.1
5.2
5.3
5.4
6.1
6.2
6.3
6.4
6.5
6.6
6.7
7.1
7.2
PITFALLS IN COMPUTATION “THE MATH BOOK IS NOT ENOUGH” (REVIEW) .............................. 7
Floating-point arithmetic and round-off error .............................................................................. 7
Sensitivity of solution to problem data ....................................................................................... 18
Instability: Solution diverges after large number of iterations .................................................... 19
MATRICES AND SYSTEMS OF LINEAR ALGEBRAIC EQUATIONS (REVIEW).................................... 21
Why systems of linear algebraic equations? ............................................................................... 21
Why vectors and matrices? ......................................................................................................... 22
What are vectors and matrices? ................................................................................................. 23
Matrix operations ....................................................................................................................... 25
Special matrices .......................................................................................................................... 35
Systems of linear algebraic equations in vector/matrix form ..................................................... 39
Geometric interpretations .......................................................................................................... 43
SOLUTION METHODS FOR SYSTEMS OF LINEAR ALGEBRAIC EQUATIONS (REVIEW) .................. 45
Basic Gauss elimination: Introduce by example ......................................................................... 46
Forward elimination as a series of elementary row and/or column operations ......................... 50
The two phases of basic Gauss elimination................................................................................. 51
Visual representation of forward elimination via elementary row (and/or column) operations 52
Basic Gauss elimination algorithm for 𝒏𝒏 equations with 𝒏𝒏 unknowns ........................................ 54
Gauss elimination for 𝒎𝒎 equations with 𝒏𝒏 unknowns ................................................................ 55
Iterative methods for systems of linear equations...................................................................... 65
VECTOR SPACES (REVIEW) .......................................................................................................... 66
Basic definitions .......................................................................................................................... 67
Constructing a vector space using the span of given vectors ...................................................... 69
Linear independence of vectors in a vector space ...................................................................... 71
Matrix rank ................................................................................................................................. 78
Matrix rank and solution of systems of linear equations ............................................................ 82
Basis and dimension for a vector space ...................................................................................... 85
Applications ................................................................................................................................ 89
SENSITIVITY OF SOLUTION OF LINEAR SYSTEM OF ALGEBRAIC EQUATIONS .............................. 92
Accuracy and precision ............................................................................................................... 93
Vector and matrix norms ............................................................................................................ 98
Matrix condition number .......................................................................................................... 104
Numerical precision and pivoting ............................................................................................. 116
ADDITIONAL USES OF GAUSS ELIMINATION (REVIEW) ............................................................. 121
LU decomposition: An economical way to store and use the results of forward elimination .. 121
Computing the inverse of a matrix (if you must)....................................................................... 124
Computing the determinant of a matrix (if you must) .............................................................. 129
Time requirements for Gauss elimination ................................................................................. 133
Cramer's rule for a linear system of equations ......................................................................... 134
Time requirements for Cramer's rule using the Leibniz formula ............................................... 135
Gauss elimination software ...................................................................................................... 136
Solve all examples by any method
NUMERICAL SOLUTION OF NONLINEAR ALGEBRAIC EQUATIONS .............................................
139preferred
(e.g. Excel Solver)
Why numerical solution of nonlinear equations? ..................................................................... 139
Graphical method ..................................................................................................................... 143
-5-
CHEE 3334 Lecture Notes
Solving Algebraic Equations
Michael Nikolaou
7.3 Optimization-based methods.................................................................................................... 144
7.4 The bisection method (a.k.a. Binary chopping, Interval halving, Bolzano's method) ................ 146
7.5 General recursive (open) method ............................................................................................. 153
7.6 When does 𝒙𝒙𝒙𝒙 + 𝟏𝟏 = 𝒈𝒈(𝒙𝒙𝒙𝒙) converge to the solution 𝒙𝒙 of 𝒙𝒙 = 𝒈𝒈(𝒙𝒙)? .................................. 158
7.7 Geometric interpretation of general recursive (open) method ................................................ 159
7.8 The Newton-Raphson method .................................................................................................. 164
7.9 Why is Newton–Raphson preferred? ........................................................................................ 170
7.10 Potential problems with Newton–Raphson .............................................................................. 173
7.11 The secant method ................................................................................................................... 182
7.12 Solution of polynomial equations ............................................................................................. 184
8.
EIGENVALUE / EIGENVECTOR PROBLEMS (REVIEW) ................................................................. 186
8.1 Review ...................................................................................................................................... 186
8.2 Why eigenvalues and eigenvectors? ......................................................................................... 188
8.4 Iterative methods for calculation of matrix eigenvalues and eigenvectors............................... 189
8.5 Software for eigenvalues .......................................................................................................... 189
8.6 Software for polynomial equations ........................................................................................... 190
9.
NUMERICAL SOLUTION OF SYSTEMS OF NONLINEAR ALGEBRAIC EQUATIONS ....................... 191 Solve all
9.1 Why numerical solution of systems of nonlinear algebraic equations? .................................... 191 examples
9.2 Basic notation ........................................................................................................................... 191 by any
method
9.3 General recursive (open) method ............................................................................................. 192
preferred
9.4 When does 𝐱𝐱(𝒌𝒌) = 𝐠𝐠(𝐱𝐱(𝒌𝒌 − 𝟏𝟏)) converge to a solution 𝐱𝐱 of 𝐱𝐱 = 𝐠𝐠(𝐱𝐱)? ................................. 192 (e.g.
9.5 Newton –Raphson for systems of nonlinear equations ............................................................ 195 Excel
9.6 Potential problems with Newton–Raphson .............................................................................. 204 Solver)
9.7 How to find a good initial guess for an open method: Continuation (homotopy) methods ..... 205
9.8 Software for nonlinear equations ............................................................................................. 208
10.
SINGULAR-VALUE DECOMPOSITION (SVD) ............................................................................... 209
10.1 Why SVD? ................................................................................................................................. 209
10.2 What is singular-value decomposition (SVD)? .......................................................................... 210
10.3 Approximation with SVD: Basis for data compression ............................................................. 215
10.4 Linear least squares and SVD (OPTIONAL) ................................................................................ 217
10.5 Software for SVD ....................................................................................................................... 221
Notation:
Uppercase, boldface: Matrices. e.g. 𝐌𝐌
Lowercase, boldface: vectors. e.g. 𝐯𝐯
Lowercase, italics: scalars. e.g. 𝑓𝑓
-6-