10.34, Numerical Methods Applied to Chemical Engineering
Professor William H. Green
Lecture #4: When Algorithms Run Into Problems: Numerical Error; Illconditioning, and Tolerances
Homework
•
•
•
find balance between concise and detailed Æ manager is audience
include solutions in word document
include any issues you had
Factorization
All NxN real matrices can be written: A Æ PTLU
(L: lower triangular, U: upper, P: permutation)
MATLAB is an offshoot of LAPACK (linear algebra package) Linear algebra is a WELL-DEVELOPED
area of study
• Can download LAPACK from netlib
Ax = b
in MATLAB: x = A \ b
‘\’ is an “amazing function” – very powerful
* WARNING: do not always know what it’s doing
Uniqueness and Existence of Solution
if rank(A) = N
NxN –- solution exists
if rank(A) < N, det(A) = 0
*find additional equation*
--
singular, rank deficient
Az = 0, where z ≠ 0
at least one eigenvalue = 0
no inverse: z ≠ A-1*0
does rank(A) = rank([A b])?
Ax = b
if yes, there is a solution
if yes and rank(A)<N, there are an infinite number of solutions
Symmetry
AT = A
Also called Hermitian Matrix if all coefficients are real
General Hermitian matrix: transpose and complex-conjugate.
positive definite: xTAx > 0 for all x ≠ 0 (all eigenvalues are positive)
If symmetric and positive definite: A = UTU; O(N2) operation (takes less memory)
L = UT
Cholesky factorization: U = chol(A)
if not symmetric/positive definite: ‘chol’ gives incomplete factorization
Vector Norms
Recall definition of vector norms: || v || p =
(∑ v )
i
1
p p
- vector norm
Matrix Norms
A = max
x≠0
|| A ∗ x || P
|| x || p
finds x that stretches matrix A the most
Cite as: William Green, Jr., course materials for 10.34 Numerical Methods Applied to Chemical Engineering, Fall
2006. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD
Month YYYY].
N
|| A ||1 = max
j
∑a
add up elements of columns and find biggest value
ij
i =1
(||A||1 = max column sum)
N
|| A || ∞ = max
i
∑a
ij
(||A||∞ = max row sum)
i =1
When p = 1 or ∞, easy to compute matrix norm;
when p is anything else, very complex solution
||A||2 = sqrt(largest eigenvalue of ATA) = largest singular value of A
||A|| > 0 if A ≠ 0
||cA|| = |c| ||A||
||A+B|| ≤ ||A|| + ||B|| (triangle inequality)
||A·B|| ≤ ||A|| · ||B||
||A●x|| ≤ ||A|| · ||x||
⎛ x⎞
A⎜⎜ ⎟⎟ = b
⎝ y⎠
⎛ − m1 1⎞⎛ x ⎞ ⎛ b1 ⎞
⎜⎜
⎟⎟⎜⎜ ⎟⎟ = ⎜⎜ ⎟⎟
⎝ − m2 1⎠⎝ y ⎠ ⎝ b2 ⎠
y
Ill-conditioning
y = m1x + b1
y = m2x + b2
(x,y)true
x
Figure 1. As long as the matrix is not
singular, there is a solution (intersection).
y
b1 ≠ b2
If m1 = m2
y
b1 = b2
b1
b2
x
Figure 2. 2 vectors that
have no solutions.
x
Figure 3. 2 vectors that have
an infinite number of solutions.
Figure 4. Poor conditioning.
y
Slopes are very similar: big range of (x,y) where:
*may be unable to discern difference numerically
y≈m1x+b1
y≈m2x+b2
x
10.34 Numerical Methods Applied to Chemical Engineering
Prof. William Green
Lecture 4
Page 2 of 3
Cite as: William Green, Jr., course materials for 10.34 Numerical Methods Applied to Chemical Engineering, Fall
2006. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD
Month YYYY].
Atrue·xtrue = btrue
Residual: r = b – Ax
||r|| < tolerance -- “good enough”
Absolute tolerance: # with units defined for
specific problem
Relative tolerance: ~# of significant figures,
scales with size of characteristic quantity
tolerance:
“a” – number (atol)
ε||b|| - % (rtol)
- best you can do to being exact
||Δx|| < rtol * ||xtrue + Δx||
- but you don’t know xtrue
guaranteed
||r|| < 2N-1 Nεmach||A|| ||x||
εmachine = usually 10-14
in reality, typically
||r|| < Nεmach||A|| ||x||
Axsol ≈ b :
Even if matrix is N = 109, can still achieve an rtol of 10-5
just because you satisfy the equation, doesn’t mean it is the exact solution
(ill conditioning)
xsoln = xtrue + Δx
Figure 5. 2 vectors with similar slopes.
y
Δx
x
(A + δA)(x + δx) = (b + δb)
⎛ || δA || || δb || ⎞
|| δx ||
⎟⎟
< || A || ⋅ || A −1 || ⎜⎜
+
|| x ||
⎝ || A || || b || ⎠
Condition number of A Æ cond(A)
If the condition number is large, a small change in A or b leads to a large change in δx .
If A is singular, there is no A-1 and cond(A) = ∞
Start next time: how scaling affects the condition number
10.34 Numerical Methods Applied to Chemical Engineering
Prof. William Green
Lecture 4
Page 3 of 3
Cite as: William Green, Jr., course materials for 10.34 Numerical Methods Applied to Chemical Engineering, Fall
2006. MIT OpenCourseWare (http://ocw.mit.edu), Massachusetts Institute of Technology. Downloaded on [DD
Month YYYY].