2.29 Numerical Fluid Mechanics
Spring 2015 – Lecture 7
REVIEW Lecture 6:
• Direct Methods for solving linear algebraic equations
– LU decomposition/factorization
• Separates time-consuming elimination for A from that for b / B
• Derivation, assuming no pivoting needed:
• Number of Ops: Same as for Gauss Elimination
• Pivoting: Use pivot element “pointer vector”
• Variations: Doolittle and Crout decompositions, Matrix Inverse
– Error Analysis for Linear Systems
• Matrix norms
• Condition Number for Perturbed RHS and LHS:
– Special Matrices: Intro
2.29
Numerical Fluid Mechanics
PFJL Lecture 7,
1
TODAY (Lecture 7): Systems of Linear Equations III
• Direct Methods
– Gauss Elimination
– LU decomposition/factorization
– Error Analysis for Linear Systems
– Special Matrices: LU Decompositions
• Tri-diagonal systems: Thomas Algorithm
• General Banded Matrices
– Algorithm, Pivoting and Modes of storage
– Sparse and Banded Matrices
• Symmetric, positive-definite Matrices
– Definitions and Properties, Choleski Decomposition
• Iterative Methods
– Jacobi’s method
– Gauss-Seidel iteration
– Convergence
2.29
Numerical Fluid Mechanics
PFJL Lecture 7,
2
Reading Assignment
• Chapters 11 of “Chapra and Canale, Numerical Methods for
Engineers, 2006/2010/2014.”
– Any chapter on “Solving linear systems of equations” in references on
CFD references provided. For example: chapter 5 of “J. H. Ferziger
and M. Peric, Computational Methods for Fluid Dynamics. Springer,
NY, 3rd edition, 2002”
2.29
Numerical Fluid Mechanics
PFJL Lecture 7,
3
Special Matrices
• Certain Matrices have particular structures that can be exploited, i.e.
– Reduce number of ops and memory needs
• Banded Matrices:
– Square banded matrix that has all elements equal to zero, excepted for a band
around the main diagonal.
– Frequent in engineering and differential equations:
• Tri-diagonal Matrices
• Wider bands for higher-order schemes
– Gauss Elimination or LU decomposition inefficient because, if pivoting is not
necessary, all elements outside of the band remain zero (but direct GE/LU
would manipulate these zero elements anyway)
• Symmetric Matrices
• Iterative Methods:
– Employ initial guesses, than iterate to refine solution
– Can be subject to round-off errors
2.29
Numerical Fluid Mechanics
PFJL Lecture 7,
4
Special Matrices:
Tri-diagonal Systems Example
Forced Vibration of a String
Example of a travelling pluse:
f(x,t)
xi
Y(x,t)
Consider the case of a Harmonic excitation
f(x,t) =- f(x) cos(wt)
Applying Newton’s law leads to the wave equation:
With separation of variables, one obtains the
equation for modal amplitudes, see eq. (1) below:
Ytt  c 2Yxx  f ( x, t )

Y ( x, t )   (t ) y ( x )
Differential Equation for the amplitude:
(1)
Boundary Conditions:
2.29
Numerical Fluid Mechanics
PFJL Lecture 7,
5
Special Matrices: Tri-diagonal Systems
Forced Vibration of a String
Finite Difference
f(x,t)
xi
+ O(h2)
Y(x,t)
Discrete Difference Equations
+
Harmonic excitation
f(x,t) = f(x) cos(wt)
Matrix Form:
Differential Equation:
y
(1)
Boundary Conditions:
Tridiagonal Matrix
If kh  1 or kh 
3 symmetric, negative or positive definite: No pivoting needed
Note: for 0< kh <1 Negative definite => Write: A’=-A and y '   y ' to render matrix positive definite
2.29
Numerical Fluid Mechanics
PFJL Lecture 7,
6
Special Matrices: Tri-diagonal Systems
General Tri-diagonal Systems:
Bandwidth of 3
LU Decomposition
Three steps for LU scheme:
1. Decomposition (GE):
2. Forward substitution
3. Backward substitution
2.29
Numerical Fluid Mechanics
PFJL Lecture 7,
7
Special Matrices: Tri-diagonal Systems
Thomas Algorithm
By identification with the general LU decomposition,
one obtains,
1. Factorization/Decomposition
2. Forward Substitution
3. Back Substitution
i
Number of Operations: Thomas Algorithm
LU Factorization:
Forward substitution:
Back substitution:
Total:
2.29
Numerical Fluid Mechanics
3*(n-1) operations
2*(n-1) operations
3*(n-1)+1 operations
8*(n-1) ~ O(n) operations
PFJL Lecture 7,
8
Special Matrices:
General, Banded Matrix
p
General Banded Matrix (p ≠ q)
0
q
Banded Symmetric Matrix (p = q = b)
0
p super-diagonals
q sub-diagonals
w = p + q + 1 bandwidth
2.29
w = 2 b + 1 is called the bandwidth
b is the half-bandwidth
Numerical Fluid Mechanics
PFJL Lecture 7,
9
Special Matrices:
General, Banded Matrix
LU Decomposition via Gaussian Elimination
If No Pivoting: the zeros are preserved
j
j
p
i
i
0
q
0
=
=
0
0
aij( j )
mij ( j ) 0
a jj
2.29
(i )

uij a
aij( i 1)  mi,i1 ai(i1,1)j
ij
if
j  i or i  j  q
(as gen. case)

uij 0
(banded)
Numerical Fluid Mechanics
if
i  j or j  i  p
(as gen. case)
(banded)
PFJL Lecture 7,
10
Special Matrices:
General, Banded Matrix
LU Decomposition via Gaussian Elimination
With Partial Pivoting (by rows):
Consider pivoting the 2 rows as below:
p
q
Then, the bandwidth of L remains unchanged,
q
0
=

mij 0
if
j  i or i  j  q
but the bandwidth of U becomes as that of A
p
q
0

uij 0
if
i  j or j  i  p  q
w
w = p + 2 q +1 bandwidth
2.29
Numerical Fluid Mechanics
PFJL Lecture 7,
11
Special Matrices:
General, Banded Matrix
Compact Storage
Needed for
Pivoting only
Diagonal (length n)
p
q
q
q
0
0
i
p
i
n–q
0
0
j
j’=j – i+ q
Matrix size: n2
2.29
Matrix size: n (p+2q+1)
Numerical Fluid Mechanics
PFJL Lecture 7,
12
Special Matrices:
Sparse and Banded Matrix
‘Skyline’ Systems
(typically for symmetric matrices)
0
…..
‘Skyline’
…..
Storage
0
Pointers
1 4 9 11 16 20
…..
Skyline storage applicable when no pivoting is needed, e.g. for banded,
symmetric, and positive definite matrices: FEM and FD methods. Skyline solvers
are usually based on Cholesky factorization (which preserves the skyline)
2.29
Numerical Fluid Mechanics
PFJL Lecture 7,
13
Special Matrices:
Symmetric (Positive-Definite) Matrix
Symmetric Coefficient Matrices:
• If no pivoting, the matrix remains symmetric after Gauss Elimination/LU decompositions
Proof: Show that if
aij( k )  a (jik )
then
aij( k 1)  a (jik 1)
using:
• Gauss Elimination symmetric (use only the upper triangular portion of A):
aij( k 1)  aij( k )  mik akj( k )
aki( k )
mik  ( k ) ,
akk
i  k  1, k  2,..., n
j  i, i  1,..., n
• About half the total number of ops than full GE
2.29
Numerical Fluid Mechanics
PFJL Lecture 7,
14
Special Matrices:
Symmetric, Positive Definite Matrix
1. Sylvester Criterion:
A symmetric matrix is Positive Definite if and only if:
det(Ak) > 0 for k=1,2,…,n, where Ak is matrix of k first lines/columns
Symmetric Positive Definite matrices frequent in engineering
2. For a symmetric positive definite A, one thus has the following properties
a) The maximum elements of A are on the main diagonal
b) For a Symmetric, Positive Definite A:
c) The elimination is stable:
No pivoting needed
aii( k 1)  2 aii( k ) .
To show this, use akj2  akk a jj in
aij( k 1)  aij( k )  mik akj( k )
aki( k )
mik  ( k ) ,
akk
2.29
i  k  1, k  2,..., n
j  i, i  1,..., n
Numerical Fluid Mechanics
PFJL Lecture 7,
15
Special Matrices:
Symmetric, Positive Definite Matrix
The general GE
aij( k 1)  aij( k )  mik akj( k )
aki( k )
mik  ( k ) , i  k  1, k  2,..., n
akk
j  i, i  1,..., n
becomes:
Choleski Factorization
Complex Conjugate
where
No pivoting needed
Complex Conjugate and Transpose
2.29
Numerical Fluid Mechanics
PFJL Lecture 7,
16
Linear Systems of Equations: Iterative
Methods
Sparse (large) Full-bandwidth Systems (frequent in practice)
x
0 x
x
0
0
x
x
0
0
x
0
0
x
x
x
0
0
x
Iterative Methods are then efficient
Analogous to iterative methods obtained for roots of equations,
i.e. Open Methods: Fixed-point, Newton-Raphson, Secant
Example of Iteration equation
A x  b  A xb 0
x xA xb 
0
x k 1  x k  A x k  b  ( A  I) x k  b
0
0
0
General Stationary Iteration Formula
xk 1  B x k  c
k  0,1,2,...
0
0
ps: B and c could be
function of k (non-stationary)
2.29
Compatibility condition for Ax=b to be the solution:
Write c  C b

1

(
I

B
)
A
 C or B  I  C A

1
1
A b  B A b  Cb 
Numerical Fluid Mechanics
PFJL Lecture 7,
17
Linear Systems of Equations: Iterative Methods
Convergence
Convergence
Convergence Analysis
Iteration – Matrix form

Sufficient Condition for Convergence:
2.29
Numerical Fluid Mechanics
PFJL Lecture 7,
18
||B||<1 for a chosen matrix norm
Infinite norm often used in practice
“Maximum Column Sum”
“Maximum Row Sum”
“The Frobenius norm” (also called Euclidean
norm)”, which for matrices differs from:
“The l-2 norm” (also called spectral norm)
2.29
Numerical Fluid Mechanics
PFJL Lecture 7,
19
Linear Systems of Equations: Iterative Methods
Convergence: Necessary and Sufficient Condition
Convergence
Convergence Analysis
Iteration – Matrix form

Necessary and Sufficient Condition for Convergence:
Spectral radius of B is smaller than one:  (B)  max i  1, where i  eigenvalue(Bnn )
i 1... n
(proof: use eigendecomposition of B)
2.29
(This ensures ||B||<1)
Numerical Fluid Mechanics
PFJL Lecture 7,
20
MIT OpenCourseWare
http://ocw.mit.edu
2.29 Numerical Fluid Mechanics
Spring 2015
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.