Engineering Computational Methods I
2016 spring
Prof. Dr. Aurél Galántai
Óbuda University
05-05-2016
2
Contents
1 Elements of Matlab
Matrices and vectors in MATLAB . .
Commands, functions and procedures
Commands . . . . . . . . . . .
Functions . . . . . . . . . . . .
M-…les and procedures . . . . .
. . . . . . . .
in MATLAB
. . . . . . . .
. . . . . . . .
. . . . . . . .
2 Solution of linear systems
Linear systems of equations . . . . . . . . . .
Linear systems with triangular matrices . . .
The Gauss method . . . . . . . . . . . . . . .
Gauss elimination with pivoting . . . . . . . .
The LU-decomposition . . . . . . . . . . . . .
The LU-decomposition and the Gauss method
The Gauss method on band matrices . . . . .
Sparse matrices . . . . . . . . . . . . . . . . .
The three row or coordinate-wise method
Sparse matrices in MATLAB . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
3 Approximate solution of nonlinear equations
The bisection algorithm . . . . . . . . . . . . . . .
Fixed point iteration . . . . . . . . . . . . . . . . .
The Newton method . . . . . . . . . . . . . . . . .
The Newton method in high dimension . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
5
. 5
. 9
. 9
. 10
. 12
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
15
15
17
18
20
22
25
25
27
29
30
.
.
.
.
31
32
33
35
37
.
.
.
.
.
.
.
41
41
42
44
45
47
53
61
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
4 Numerical solution of Ordinary Di¤erential Equations
Explicit one-step methods . . . . . . . . . . . . . . . . . . . .
The explicit Euler method . . . . . . . . . . . . . . . . .
Explicit one-step methods . . . . . . . . . . . . . . . . .
The convergence of explicit one-step methods . . . . . .
Runge-Kutta methods and implementation matters . . .
Linear multistep methods . . . . . . . . . . . . . . . . . . . .
Di¤erence methods for boundary value problems . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
3
CONTENTS
4
Existence and unicity results for boundary value
Numerical di¤erentiation . . . . . . . . . . . . .
Solution of linear boundary value problems . . .
Solution of nonlinear boundary value problems .
The ODE programs of Matlab (A quick reminder) . .
5 Numerical methods for partial di¤erential
Parabolic equations . . . . . . . . . . . . . . . .
The method of separation of variables . . .
Finite di¤erence methods . . . . . . . . . .
The explicit Euler method . . . . . . . . .
Hyperbolic equations . . . . . . . . . . . . . . .
D’Alembert’s method . . . . . . . . . . . .
The method of separation of variables . . .
Explicit di¤erence method . . . . . . . . .
Elliptic equations . . . . . . . . . . . . . . . . .
problems
. . . . . .
. . . . . .
. . . . . .
. . . . . .
equations
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
61
63
64
68
74
.
.
.
.
.
.
.
.
.
75
78
79
81
82
89
89
92
95
98
Chapter 1
Elements of Matlab
Matrices and vectors in MATLAB
The mathematical software MATLAB got its name from the expression MATrix LABoratory.
The most important data type of MATLAB is the real or complex matrix
The scalars and vectors are also matrices, however it is not necessary (but possible) to
invoke them by two indices
The simplest way to set up matrices is by enlisting their elements row-wise.
The rows must be separated by
; or enter,
The entries of row must be separated by a space or
,
the enlisted elements must be bracketed by [. . . ].
Example 1
2
3
1 0 3
A=4 3 1 2 5
4 5 6
A can be de…ned as follows
A=[1 0 3;3 1 2
4,5,6]
This is also an example of a simple command.
5
Elements of Matlab
6
Declaration of matrices is automatic if a reference is made to them.
Names of variables (data) can be selected in the usual ways.
There are …xed identi…ers used by Matlab. These can be detected with the HELP command.
Example 2 eps is an occupied identi…er, the so called machine epsilon (
2:2204
10
16
).
Programming Matlab we cannot use the …xed identi…ers for other purposes. Their use may
result in errors not easily detectable.
A matrix can be built from smaller matrices (blocks).
Example 3 The command
B=[A,[-1 -2;-1-2;-1-2];7,7,7,-7,-7]
with the previous A gives the matrix
2
1
6 3
B=6
4 4
7
0
1
5
7
3
2
6
7
1
1
1
7
We refer to any entry (i; j) of a matrix A by A (i; j).
3
2
2 7
7:
2 5
7
Example 4 A(2,3), B(1,4).
For the earlier A and B, A(2,3)=2, B(1,4)=-1.
For the elements of row or column vectors we can refer to only with one index.
In certain cases it has no signi…cance that a vector is column or row vector. In many cases
however it has signi…cance.
If a vector is de…ned by one index and its elements are given separately, then the vector will
be a row vector.
Scalar data can be de…ned in the usual way.
Example 5 -1.23e-2.
If k,h,m are already de…ned numbers, then we can build vectors in the following way:
c=k:h:m
This command de…nes a row vector c whose elements are in order
k; k + h; k + 2h; : : : ; k + th:
Elements of Matlab
7
The last element comes from the relations
k + th
m < k + (t + 1)h;
or
k + (t + 1)h < m
k + th;
depending the sign of h.
h can be omitted from the command, if h = 1.
A vector or matrix can be empty.
Earlier we built a matrix from submatrices. We can do the reverse.
The command
C=A(p:q,r:s)
results in the matrix
2
apr
6 ..
C=4 .
aqr
3
aps
.. 7 :
. 5
aqs
An important application of submatrices is the command
A(i,:)=A(i,:)+t A(k; :)
which gives t times the row k to the row i of matrix A.
The submatrix can be de…ned in a more general way in MATLAB: if u is a vector with r
elements, v is a vector with s elements (either row or column vectors), then the command
H=A(u,v)
produces a r
s type matrix H such that H(i; j) = A(ui ; vj ).
Example 6 Using only one command we can swap two rows of A:
A([j,k],:)=A([k,j],:)
If the elements of u or v are not strictly increasing numbers, then A(u,v) will not be a
submatrix of A.
Special matrices can be generated by built in Matlab functions. Some examples:
eye(m,n); ones(m,n); zeros(m,n); rand(m,n)
These functions produce in order an m n size
"identity" matrix,
a matrix whose entries are 1’s,
the zero matrix,
random matrix.
Function A RAND produces pseudo-random numbers with uniform distribution in (0; 1).
Elements of Matlab
8
For m = n, it is enough to use one argument: for example, ones(4).
Three other functions:
TRIL(A),
TRIL(A),
DIAG.
Function TRIL(A) takes out the lower triangular part of A.
Function TRIU(A) takes out the upper triangular part of A.
Before we specify DIAG we mention that we can refer to the diagonals of a matrix by their
indices.
The main diagonal of the m n type matrix A has the index 0, the diagonals above the main
diagonal are numbered as 1; : : : ; (n 1); the diagonals under the main diagonal are numbered
as 1; : : : ; (m 1).
The k-th diagonal is formed by those entries aij that satisfy the condition k = j
Let v be a vector of length t. The command
i.
DIAG(v,k)
generates matrix D 2 R(t+jkj)
other elements are 0.
(t+jkj)
whose k-th diagonal contains the elements of v and its
DIAG(A,k)
generates a column vector whose elements are elements of the k-th diagonals of matrix A.
If k = 0, we can omit k.
We can perform operations
+; ;
jelekkel with matrices, vectors and scalars provided that the operands are compatible with the
operations.
Exception: We can add (subtract) a scalar to any matrix. The scalar will be added to
(subtracted from) every entries of the matrix.
^
The sign
denotes the power operation. Operation A^k multiplies A with itself k times,
if k is positive integer, A^ ( 1) gives the inverse if it exists.
The operations + and
.(dot) sign.
are performed element-wise if the operation sign is preceded by a
Example 7 The result of [2,-2]. [2,3] is [4,-6], while the result of [-2,3].^3 is [-8,27].
Special operation is the transpose: A’.
Complex conjugation: conj(A)
Complex transpose (Hermitian transpose): ctranspose(A), or A´
Elements of Matlab
n).
9
For compatible matrices we can apply operations / and n (for details, see help / or help
We just mention the most important case. The result of command
x=Anb
is the (numerical) solution of linear system Ax = b (if the system is consistent).
Commands, functions and procedures in MATLAB
MATLAB has lots of built in functions and procedures to help program development.
Accordingly a MATLAB program generally consists of a few commands and several function
or procedure calls..
Commands
There are only a few commands:
simple command,
IF command,
two types of loops (for and while loops).
The general form of the IF command:
if condition
commands
{elseif
commands}
<else
commands>
end
Section {
Section <
} can be repeated many times (it can also be omitted),
> can be omitted.
Parts written in separate lines can be written in one line, but they must be separated by
or .
The conditions are logical expressions. Elementwise relational operators:
;
<, <=, >, >=, = =, ~=
,
Elements of Matlab
10
The last two are the ”equal” and the ”non-equal” operation. The logical or, and negation
are in order:
j, &, ~
The for loop has the form:
for i = v
commands
end
Variable i is the loop variable, v is a vector that is often has the form k:h:m (or k:m).
Semantics (if v has n elements):
the loop variable i takes the values v1 ; v2 ; : : : ; vn in order and for each value the loop
kernel is executed.
(If v is a matrix, then v1 ; v2 ; : : : ; vn stands for the column vectors of v.)
The while loop has the form:
while condition
commands
end
Functions
There are plenty of functions in MATLAB.
The following functions produce outputs from existing matrices.
Their inputs can be scalars as well..
Certain functions are de…ned only for scalars. In such cases the function is de…ned elementwise.
Example 8 The command
sin(pi/4*ones(2,3))
p
gives the 2 3 matrix whose all elements are 2=2:
The most important functions (using the HELP we can discover the others as well).
Elementary mathematical functions: SIN, COS, . . . ,ROUND, etc.
DET(A) produces det (A),
COND(A) produces the condition number of A (in ”2”-norm),
RANK(A) produces the rank of A.
[m,n]=size(A), h2=norm(A),
hinf=norm(A,inf)
The …rst command gives the size of matrix (vector) A. The second and third command give
kAk2 and kAk1 , respectively
Elements of Matlab
11
kAk1 and kAkF can also be computed by the calls NORM(A,1) and NORM(A,’fro’), respectively.
The outputs of the following functions are row vectors (or scalars):
MAX, MIN, SUM, PROD.
If the input is a matrix, but not a row vector, then MAX(A) gives the greatest elements
of each column, MIN(A) gives the smallest elements of each column of A, SUM(A) gives the
sums of the elements of each column, and PROD(A) gives the products of the elements of each
column.
If the input is a row vector, then we obtain in order the greatest, the smallest element, the
sum and the product of the elements (the result is clearly scalar).
Functions MAX and MIN can be invoked with two inputs, as well.
Example 9 Let
2
1
6 2
A=6
4 0
2
3
2
2
0
2
2
3
4
1
2
9
5
3
2
15 7
7:
1 5
1
Here max(A) gives the vector [2; 3; 4; 5; 15], while the commands
[c, d]=max(A), [c1,d1]=max(abs(A(3:4,2:4)))
give the results
c = [2; 3; 4; 5; 15]
d = [2; 1; 4; 4; 2]
c1 = [2; 4; 9]
d1 = [1; 2; 1]
Example 10 The command
[p,j]=max(max(A))
gives
p = 15;
j = 5;
which is the greatest element of A and its column index. The row index of the maximal element
is given by d(j), which is 2.
Many of the built in functions of MATLAB execute very complicated algorithms of numerical
linear algebra. The most important ones are the following
[L,U,P]=lu(A),
[T,F]=lu(A),
H=chol(C)
The …rst two functions perform two versions of the LU -decomposition, the last function produces the Cholesky-decomposition of matrix C (C must be symmetric and positive de…nite).
Elements of Matlab
12
The results satisfy the following properties: L; P; T; H are lower triangular matrices, U; F
are upper triangular matrices, P is a permutation matrix, and
L = P T;
X=inv(A),
U = F;
LU = P A;
T F = A;
HH T = C:
[Q,R]=qr(A)
Here X is the inverse of A (if it exists), Q is orthogonal, R is upper triangular and QR = A.
s=eig(A),
[V,D]=eig(A)
The …rst command gives the eigenvalues of A.
The second command puts the eigenvalues into the diagonal matrix D, while the i-th column
of matrix V is an eigenvector that belongs to the eigenvalue dii .
We mention here some numerical functions that are not built in functions but very useful:
- FSOLVE solves nonlinear systems of equations,
- QUAD computes approximate integral with an adaptive Simpson-formula,
- SPLINE computes the natural spline interpolation.
M-…les and procedures
A sequence of MATLAB commands form a MATLAB program.
The program must be included in a text …le with extension.M (M -…le).
The invocation of the program is simply done its name. For example, the command
prog
loads and execute the program contained in the …le prog.m.
The variables of a simple program (script-…le) are global variables.
It is also possible to de…ne functions in M -…le form in the following way:
The sequence of commands must be started with the line (function header)
function [k1 ; : : : ; kt ] = f name(b1 ; : : : ; bs )
The number of input parameters k1 ; : : : ; kt and the number of output parameters b1 ; : : : ; bs
is arbitrary (in principle).
It is advised to have the same function name and …le name (fname.m).
We can put the
RETURN
command anywhere in the program, with the usual e¤ect.
The non-parameter variables in the program of the function are local variables.
Elements of Matlab
13
A variables can be de…ned as global with the command
GLOBAL
A function name can be input parameter. In such a case we have to invoke the FEVAL
procedure within the function.
Example 11 If parameter f is a two variable function, then the command z = f(x; y) is error.
The correct way is z = feval(f; x; y).
A few other possibilities.
The command
DISP(variable)
or
DISP(’string’)
puts the value of variable or the string onto screen.
The command
DIARY …lename
puts all screen messages to the text …le …lename.
This can be stopped by the command
diary off
The command
diary on
continues writing.
If x and y are two vectors of the same length, then command
plot(x,y)
draws a curve on the screen from the points (xi ; yi ).
The procedure PLOT can be called in other forms as well.
We can also measure CPU time as shown by the next example
t=clock %or t=cputime
{commands}
tt=etime(clock,t) %or tt=cputime-t
The
CLOCK
returns a six element date vector containing the current time and date in decimal form:
[year month day hour minute seconds]
The
14
Elements of Matlab
ETIME
returns the time in seconds that has elapsed.
CPUTIME
returns the CPU time in seconds that has been used by the MATLAB process since MATLAB started.
Chapter 2
Solution of linear systems
Linear systems of equations
The general (canonical) form of linear systems of equations in case of m equations and n
unknowns is
a11 x1 + : : : + a1j xj + : : : + a1n xn = b1
..
.
ai1 x1 + : : : + aij xj + : : : + ain xn
=
..
.
bi
(2.1)
am1 x1 + : : : + amj xj + : : : + amn xn = bm
A compact form is
Ax = b;
(2.2)
where
m
A = [aij ]m;n
i;j=1 2 R
n
; x 2 R n ; b 2 Rm :
For m < n, the system is called underdetermined, and for m > n, it is overdetermined. If m = n
the system is called quadratic.
The geometric content of linear systems can be written in the following way.
De…nition 12 Let x0 2 Rn be a point of a hyperplane and let d 2 Rn be a nonzero vector that
is perpendicular (orthogonal) to the hyperplane. The points of this hyperplane are given by the
vectors x 2 Rn satisfying the equation
(x
x0 )T d = 0:
(2.3)
15
Solution of linear systems
16
We can rewrite the hyperplane equation in the form
xT d = xT0 d:
(2.4)
Using the row-wise partition of matrix A
2 T 3
a1
6 .. 7
6 . 7
6
7
A = 6 aTi 7
aTi = [ai1 ; : : : ; ain ]
6 . 7
4 .. 5
aTm
we can rewrite equation Ax = b in the form
aT1 x = b1 ;
..
.
aTm x
(2.5)
= bm :
We can see that the solution of the linear system is the common subset of m hyperplanes.
So there are three possibilities:
(i) Ax = b has no solutions,
(ii) Ax = b has exactly one solution,
(iii) Ax = b has in…nitely many solutions.
De…nition 13 If Ax = b has at least one solution, the system is called consistent. If it has no
solution, then the system is called inconsistent.
Example 14 The system
x + 2y = 1;
x + 2y = 4
is inconsistent.
Using the column-wise partitioning of A we can write Ax = b in the form
n
X
xi ai = x1 a1 + : : : + xn an = b;
i=1
where ai stands for the i-th column of A. The linear system can be solved if and only if b can
be expressed as a linear combination of the columns of A.
De…nition 15 The rank of a matrix A 2 C m
independent column or row vectors.
n
is de…ned by the maximal number of linearly
Solution of linear systems
17
Theorem 16 (a) Ax = b has a solution if and only if rank(A) =rank([A; b]).
(b) If rank(A) =rank([A; b]) = n, then Ax = b has exactly one solution.
Matlab: function rank(A) computes the rank of A.
From now on we assume that m = n.
Theorem 17 Equation Ax = b (A 2 Rn n , b 2 Rn ) has exactly one solution if and only if
there exists A 1 . I this case the solution is given by x = A 1 b.
Theorem 18 The homogeneous equation Ax = 0 (A 2 Rn
if and only if det (A) = 0.
n
) has a nontrivial solution x 6= 0
Linear systems with triangular matrices
De…nition 19 Matrix A = [aij ]ni;j=1 is lower triangular if for all i < j, aij = 0.
De…nition 20 Matrix A = [aij ]ni;j=1 is upper triangular if for all i > j, aij = 0.
A matrix which is both lower and upper diagonal is a diagonal matrix.
For a triangular matrix A = [aij ]ni;j=1 , det(A) = a11 a22 : : : ann .
The solution of systems with triangular coe¢ cient matrices is quite simple.
Consider the lower triangular linear system
a11 x1
..
.
= b1
..
.
..
.
ai1 x1 + : : : +aii xi
..
..
.
.
an1 x1 + : : : +ani xi
..
= bi
..
.
.
: : : +ann xn = bn
This can be solved uniquely if and only if a11 6= 0,: : : ; ann 6= 0.
From equation i, one obtains
!
i 1
X
x i = bi
aij xj =aii :
j=1
Hence the solution algorithm:
x1 = b1 =a11
for i = 2 : n
Pi 1
xi = (bi
j=1 aij xj )=aii
end
(2.6)
Solution of linear systems
18
Matlab programs: lowtri1.m, lowtri2.m, testlin1.m
Consider the upper triangular system
a11 x1 + : : : +a1i xi + : : : +a1n xn = b1
..
..
..
..
.
.
.
.
aii xi + : : : +ain xn = bi
..
..
...
.
.
ann xn = bn
This can be solved uniquely if and only if a11 6= 0,: : : ; ann 6= 0.
From equation i, one obtains
!
i 1
X
x i = bi
aij xj =aii :
j=1
Hence the solution algorithm (backward substitution):
xn = bn =ann
for i = n 1P
: 1:1
n
xi = (bi
j=i+1 aij xj )=aii
end
(2.7)
Matlab programs: upptri.m, testlin2.m
The Gauss method
The Gauss method consists of two phases:
I. We transform the system Ax = b to an equivalent upper triangular system:
II. We solve this upper triangular system using Algorithm (2.7).
Transformation to upper triangular form is done in the following way.
If a11 6= 0, then make zero the coe¢ cients of x1 under the coe¢ cient a11 so that we subtract
an appropriate multiple of the …rst row from row i (i = 2; : : : ; n):
(ai1
a11 )x1 + (ai2
a12 )x2 + : : : + (ain
a1n )xn = bi
b1 :
(2.8)
Solution of linear systems
Condition ai1
19
a11 = 0 gives us
ai1
:
a11
Hence the zeroing of coe¢ cients under a11 is done by the algorithm
9
= ai1 =a11
=
aij = aij
a1j (j = 2; : : : ; n)
(i = 2; : : : ; n)
;
bi = bi
b1
(2.9)
=
(2.10)
This algorithm overwrites the elements of submatrix A (2 : n; 2 : n) and vector b (2 : n).
However the zeros are not written into the matrix.
Using the original notation, the reduced linear system has the form
a11 x1 + a12 x2 + : : : + a1n xn = b1
a22 x2 + : : : + a2n xn = b2
..
..
.. :
.
.
.
an2 x2 + : : : + ann xn = bn
(2.11)
We can decompose this into the …rst equation and the smaller (n 1) (n 1) linear subsystem.
If a22 6= 0, we repeat the procedure on the subsystem, and so on.
Assume that we already made the zeroing in the …rst k 1 columns and we have a reduced
linear system of the form
a11 x1 + : : : : : : +a1k xk + : : : + a1n xn = b1
..
..
..
..
.
.
.
.
..
..
..
..
.
.
.
.
akk xk + : : : + akn xn = bk
..
..
..
.
.
.
aik xk + : : : + ain xn = bi
..
..
..
.
.
.
ank xk + : : : + ann xn = bn
If akk 6= 0, then we make zero the coe¢ cients of xk under akk .
Subtracting the multiple of row k from row i we obtain the equation
(aik
Condition aik
akk )xk + (ai;k+1
ak;k+1 )xk+1 + : : : + (ain
akk = 0 gives us
=
aik
:
akk
Hence the required algorithm is
= aik =akk
aij = aij
akj
bi = bi
bk
akn )xn = bi
(j = k + 1; : : : ; n)
bk :
(2.12)
(2.13)
9
=
;
(i = k + 1; : : : ; n)
Solution of linear systems
20
We can continue the procedure until akk 6= 0 and k n 1 hold.
If we succeeded to transform Ax = b into the equivalent upper triangular form, then we
apply Algorithm (2.7).
Notation (Matlab): A (i; j) stands for element aij of matrix A.
The Gauss method:
I. (elimination) phase:
for k = 1 : n 1
for i = k + 1 : n
= A (i; k) =A (k; k)
A (i; k + 1 : n) = A (i; k + 1 : n)
A (k; k + 1 : n)
b (i) = b (i)
b (k)
end
end
II. (backward substitution) phase:
x (n) = b (n) =A (n; n)
for i = n 1 : 1 : 1
x(i) = (b(i) A(i; i + 1 : n) x(i + 1 : n))=A(i; i);
end
Matlab programs: gauss1, testlin3.m
Theorem 21 The Gauss method requires 32 n3 + O(n2 ) arithmetic operations.
Theorem 22 (Klyuyev, Kokovkin-Shcherbak, 1965). Any algorithm which uses only rational
arithmetic operations (solely row and column operations) to solve a general system of n linear
equations requires at least as many additions and subtractions and at least as many multiplications and divisions as does the Gaussian elimination.
Gauss elimination with pivoting
In the …rst phase of the Gauss method it may happen that akk becomes zero (or near zero)
after the elimination of coe¢ cients under the main diagonal in the …rst k 1 columns of A.
Example 23 In the system
4x2 + x3 =
x1 + x2 + 3x3 =
2x1
2x2 + x3 =
a11 = 0.
9
6
1
Solution of linear systems
21
In such a case we can try to exchange rows or columns so that we should have a nonzero
element in place. In the above case we can swap the …rst and the third rows:
2x1
2x2 + x3 =
x1 + x2 + 3x3 =
4x2 + x3 =
1
6
9
If we swap the …rst and the second columns (and the respective variables) we obtain
4x2
+ x3 =
x2 + x1 + 3x3 =
2x2
2x1 + x3 =
9
6
1
Generally, there are several possibilities for the exchange of rows or columns.
If all coe¢ cients aj;k (j k) or ak;j (j k) are zero, then the submatrix [aij ]n;n
i;j=k is singular
and so is A. In such a case we clearly cannot continue the elimination.
Element akk is called the pivot element. The exchange of rows or columns so that the new
pivot element should be di¤erent from zero is called pivoting.
The selection of the pivot element greatly in‡uences the reliability of computational results.
Problem 24 Solve the linear system
10
17
x + y = 1
x
+ y = 2
with our program gauss1.m.
The obtained result is x = 0, y = 1, while the correct result is
x=
1
1
10
1;
17
y=1
10 17
1 10 17
1:
Swap the two equations:
x
10
17
+ y = 2
x + y = 1
The same program now gives the result x = 1, y = 1.
The last result is near to the solution, while the …rst one is a catastrophic result.
It is true generally that a good pivoting can improve the precision of the numerical solution
signi…cantly. It is a general rule to choose pivot elements with absolute values as big as possible.
There are two basic pivoting strategies.
Partial pivoting: Step k of the …rst phase: we choose row i (i
jaik j = max fjajk j : k
j
k) so that
ng :
Then we swap rows k and i so that after the pivoting jakk j = maxk
i n
jaik j holds.
Solution of linear systems
22
Complete pivoting: Step k of the …rst phase: we chose entry (i; j) so that
jaij j = max fjats j : k
t; s
ng :
Then we swap rows k and i, and columns k and j so that after the pivoting jakk j = maxk
holds.
i;j n
jaij j
In such a case we also swap the variables.
In the case of Gauss elimination with pivoting we make a pivoting at each step of the …rst
phase.
The complete pivoting is considered as a safe strategy but expensive.
The partial pivoting is also considered as a safe but less costly strategy, if some extra
technique (for example, iterative improvement) is added.
In practice the latter is preferred.
The LU-decomposition
The …rst phase of the Gauss method produces an upper triangular matrix. We seek for the
connection of A and this upper triangular matrix.
We use the following facts:
(i) The inverse of a triangular matrix is also triangular (of the same type);
(ii) The product of two triangular matrices of the same type is also triangular of the same
type
De…nition 25 The LU -decomposition of a matrix A 2 Cn n is a representation of the matrix
in the product form A = LU , where L 2 Cn n is lower triangular matrix and U 2 Cn n is
upper triangular matrix.
Lemma 26 If a nonsingular matrix A has two LU -decompositions A = L1 U1 and A = L2 U2 ,
then there exists a diagonal matrix D such that L1 = L2 D, U1 = D 1 U2 .
Proof. Assume that we have two LU -decompositions A = L1 U1 = L2 U2 , which implies
L2 1 L1 = U2 U1 1 . The left matrix is lower triangular, while the right matrix is upper triangular.
Hence they must be diagonal, that is L2 1 L1 = U2 U1 1 = D, which implies the claim.
Corollary 27 If L is unit lower triangular (each entry in its main diagonal is 1), then the
LU-decomposition is unique.
Solution of linear systems
23
Let
A(r)
2
3
a11 : : : a1r
6
.. 7
= 4 ...
. 5
ar1 : : : arr
(r = 1; : : : ; n
1) :
be the leading principal submatrix of A.
Theorem 28 (Turing). A nonsingular matrix A 2 Cn n has an LU -decomposition if and only
if
(2.14)
det A(r) 6= 0 (r = 1; : : : ; n 1):
There are cases when a matrix is nonsingular, and it has no LU -decomposition.
Example 29 Matrix
0 1
1 0
has no LU -decomposition.
Assume the contrary:
0 1
1 0
=
1 0
l 1
a b
0 c
=
a
b
al lb + c
:
Then a = 0, b = 1, 1 = al = 0 l = 0, which is impossible.
De…nition 30 A matrix P 2 Rn n is a permutation matrix, if it has exactly one entry 1 in
each row and each column, while the other entries are 0.
Example 31
2
1
6 0
6
4 0
0
0
0
0
1
0
1
0
0
3
0
0 7
7:
1 5
0
Let ei = [0; : : : ; 0; 1; 0; : : : ; 0]T 2 Rn be the i-th unit vector. Any permutation matrix can
be written in the form
3
2
eTi1
6 eT 7
6 i 7
P = 6 ..2 7 ;
4 . 5
eTin
where i1 ; i2 ; : : : ; in is a permutation of the numbers 1; 2; : : : ; n.
For the above example, P = [e1 ; e3 ; e4 ; e2 ]T .
Solution of linear systems
24
Consider the product of a matrix A by a permutation matrix P . Using eTi A = aTi one
obtains
2
3
2
3 2
3
eTi1
eTi1 A
aTi1
6 eT 7
6 eT A 7 6 aT 7
6 i2 7
6 i
7 6 i 7
P A = 6 .. 7 A = 6 2.. 7 = 6 .. 2 7 :
4 . 5 4 . 5
4 . 5
T
ein
eTin A
aTin
It follows that product P A exchanges the rows of A.
Let P be partitioned column-wise, that is
P = [ej1 ; : : : ; ejn ] ;
where j1 ; j2 ; : : : ; jn is a permutation of the numbers 1; 2; : : : ; n. Also assume that A is partitioned column-wise, that is A = [a1 ; : : : ; an ] (ai 2 Cn ). Since Aei = ai we have
AP = A [ej1 ; : : : ; ejn ] = [Aej1 ; : : : ; Aejn ] = [aj1 ; : : : ; ajn ] :
For arbitrary permutation matrix P , P T P = I.
In the k-th step of the …rst phase of the Gauss method with partial pivoting we swap rows
k and i (i > k). This operation is equivalent with the operation Pk A(k 1) x = Pk b(k 1) , where
2 T 3
e1
6 .. 7
6 . 7
6 T 7
6 ek 1 7
6 T 7
6 ei 7
6 T 7
6 ek+1 7
6 . 7
n n
7
Pk = 6
6 .. 7 2 R ;
6 eT 7
6 i 1 7
6 eT 7
6 k 7
6 eT 7
6 i+1 7
6 . 7
4 .. 5
eTn
and A(k 1) , b(k 1) denote the system data after step k 1.
Note that in row k we have eTi , and in row i we have eTk .
The complete pivoting corresponds the operation
Pk A(k
1)
Qk QTk x = Pk b(k
1)
where Qk is the permutation matrix that executes the column swap.
Theorem 32 If the n n type matrix A is nonsingular, then there exists a permutation matrix
P such that matrix P A has LU -decomposition.
Solution of linear systems
25
The LU-decomposition and the Gauss method
The …rst phase of the (plain) Gauss method produces the LU -decomposition of A, more precisely
the equivalent linear system
U x = L 1 b;
(2.15)
The lower triangular matrix L of
2
1
6
6 a21 =a11
6
6
6 a =a
L = 6 31 . 11
6
..
6
6
..
4
.
an1 =a11
the LU -decomposition can be obtained as
3
0
:::
0
7
..
7
.
.. 7
7
1
. 7
7;
.
7
ak+1;k =akk . .
7
7
..
.
1
0 5
: : : ank =akk : : : an;n 1 =an 1;n 1 1
where the entries of column k are computed in step k of phase I. It is common to write elements
ak+1;k =akk ; : : : ; ank =akk
into the coe¢ cient matrix A under the main diagonal.
This does not a¤ect the elimination phase. Also note that these numbers are the
elements
used in the elimination step.
Hence the Gauss method is based on the LU -decomposition and, in fact, it essentially
produces it.
For the partial pivoting case, the Gauss method gives the LU -decomposition of a matrix
P A, where P is a permutation matrix that can be obtained from the procedure.
For complete pivoting, we do the same, but row exchange must also be done on the entries
under the main diagonal.
The Gauss method on band matrices
De…nition 33 A matrix A 2 Rn n is called band matrix with lower bandwidth p and upper
bandwidth q, if
aij = 0; for i > j + p and i + q < j:
Those elements aij of A are (considered) nonzero for which
i
p
j
i + q;
Solution of linear systems
26
or equivalently
j
2
a11
q
i
a12 : : : : : : a1;1+q
6
6 a21 a22
6 .
..
6 ..
.
6
6
...
6 a1+p;1
6
6
..
A=6 0
.
6
.
...
6 .
6 .
6 .
..
6 ..
.
6
6 ..
4 .
0
::: ::: :::
j + p:
0
...
:::
..
:::
0
..
.
..
.
.
...
..
0
.
an
...
..
...
0
.
...
an;n
p
: : : an;n
1
..
.
..
.
q;n
an 1;n
ann
3
7
7
7
7
7
7
7
7
7
7:
7
7
7
7
7
7
7
5
The band matrices are important if p and q are much smaller than n.
On band matrices the Gauss method has a special form. Here the k-th step of phase 1 must
be executed on a matrix of the form
=
A
x
b
It is obvious that the zeroing of entries under akk must be performed only in the rows
i = k + 1; : : : ; k + p provided that k + p n. It is also clear that in these "active" rows the
new coe¢ cients aij must be computed until column k + q.
The second phase must also be modi…ed accordingly. Here the upper triangular matrix has
upper bandwidth.
It follows that the LU -decomposition A is also special.
Theorem 34 Let A 2 Rn n be a band matrix with lower bandwidth p and upper bandwidth
q. Assume that A has an LU -decomposition A = LU . Then L is a band matrix with lower
bandwidth p and U is a band matrix with upper bandwidth.
Solution of linear systems
27
It follows that we can store L and U in place of A.
If we have to make pivoting on a band matrix, then its bandwidth doubles.
Band matrices arise in discretization methods, spline interpolation, etc.
Consider the following linear system with tridiagonal matrix:
b1 x1 +c1 x2
a2 x1 +b2 x2 +c2 x3
a3 x2 +b3 x3 +c3 x4
..
..
.
.
...
..
.
...
an 1 x n
2
...
+bn 1 xn
an x n
1
1
+cn 1 xn
+bn xn
=
=
=
..
.
..
.
=
=
d1
d2
d3
dn
dn
1
Here we use four vectors to store data in order to save memory space.
Assume that after the …rst k 1 steps of phase 1 rows k 1 and k have the form
xk
ak x k
+ ek 1 xk
= gk
1 + bk xk + ck xk+1 = dk
1
1
Subtracting the multiple of the …rst equation from the second one obtains
(bk
ak ek 1 ) xk + ck xk+1 = dk
and
xk +
b
|k
dk
ck
xk+1 =
a e
b
{zk k 1}
|k
ek
Hence the tridiagonal Gauss method has the form
ak gk
1
ak gk 1
:
ak ek 1
{z
}
dk
c1
ck
; ek =
(k = 2; : : : ; n 1) ;
b1
bk ak ek 1
dk ak gk 1
d1
(k = 2; : : : ; n 1; n) ;
g1 = ; gk =
b1
bk ak ek 1
xn = gn ; xk = gk ek xk+1 (k = n 1; n 2; : : : ; 1) :
e1 =
Matlab programs: testlin6.m, gtri.m.
Sparse matrices
De…nition 35 (heuristic): A matrix A 2 Rn
nonzero elements is essentially smaller than n2 .
n
is said to be sparse, if the number nnz of
Solution of linear systems
28
Under the term "essentially smaller" we mean that nnz = O (n) or at most O (n1+" ) , where
0 < " < 1 and " 0.
Band matrices are also sparse but their nonzero elements follow a regular sparsity pattern.
Under sparsity pattern we mean the geometry of the distribution of nonzero elements in the
matrix.
Large sparse matrices with irregularly distributed nonzero elements are a source of big and
serious computational problems.
The …rst problem is the storing of a large sparse matrix e¢ ciently.
The second problem is that standard matrix operations on sparse matrices often result in
matrices whose sparsity patterns and numbers of nonzero elements di¤erent from the original
or the result itself becomes dense.
Example 36 Consider the …rst phase of the Gauss method on the matrix
2
6
6
6
6
6
A=6
6
6
6
6
4
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
that contains 28 nonzero elements denoted by
0
0
0
7
7
7
7
7
7
0 7
7
0 7
7
0 5
0
0
0
0
0
0
0
3
.
Assume that the Gauss method can be executed. The …rst step of phase 1 makes three
elements zero. The resulting matrix will have 31 nonzero elements.
2
6
6
6
6
6
6
6
6
6
6
4
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
The 6 new nonzero elements are denoted by .
0
0
0
0
0
0
0
0
3
7
7
0 7
7
7
7:
7
7
0 7
7
0 5
Solution of linear systems
29
The upper triangular matrix U obtained at
2
0 0
6 0
0 0
6
6 0 0
0
6
6 0 0 0
6
6 0 0 0 0
6
6 0 0 0 0
6
4 0 0 0 0
0 0 0 0
the end of phase 1 looks like
3
0 0
7
0
7
0 7
7
7
0 0
7
7
7
7
0
7
5
0 0
0 0 0
where we have 7 new nonzero elements (with respect to the original upper triangular part).
This phenomenon is called …ll in.
Matlab demo program: testlin7.m.
The three row or coordinate-wise method
It is a generally used method. We store the nonzero elements, their row and column indices in
three vectors of length nnz either in column-wise or row-wise.
Example 37 Consider
2
6
6
A=6
6
4
5
2
0
0
0
0
4
1
0
0
0
0
3
0
0
3
0
0
2
2
0
1
2
3
1
3
7
7
7
7
5
(n = 5; nnz = 12) :
The corresponding (column-wise) three row representation is the following
val
=
row_ind =
col_ind =
5 2 4 1 3 3 2 2 1 2 3 1
1 2 2 3 3 1 4 5 2 3 4 5
1 1 2 2 3 4 4 4 5 5 5 5
Storing nonzero data in column-wise of row-wise form is not a real restriction. It is common
to store the nonzero elements in "random" order, and consider this as a list data structure. In
fact, it is easy to insert new elements at the and of the list. The matrix-vector operations are
essentially list data structure operations.
Some other sparse matrix storage forms:
- CRS = Compressed Row Storage,
- CCS = Compressed Column Storage,
- SKS = Skyline Storage.
Solution of linear systems
30
Sparse matrices in MATLAB
Algorithms and functions handling sparse matrices can be found in the SPARFUN library.
MATLAB uses the three row representation for sparse matrices. The sparsity is an attribute
of data.
Sparse matrices can be de…ned by the command SPARSE that can be called in several
ways. The three most important ways of call:
1. S = SP ARSE(X)
Converts a dense matrix into sparse form.
2. S = SP ARSE(i; j; s; m; n; nzmax)
De…nes an m n sparse matrix in which the number of nonzero elements is maximum
nzmax and the nonzero elements are given by in three row form by rows of matrix [i; j; s],
where i, j and s are vectors of the same length.
3. S = SP DIAGS(A; d; m; n)
De…nes an m n type sparse matrix formed from diagonals.
Sparse matrices can be converted to dense form by the command FULL whose form is
A = F U LL(X).
Command F IN D gives the nonzero elements of a matrix.
Command N N Z gives then number of nonzero elements.
Command SP Y gives a graphic representation of the geometry of nonzero elements.
Useful commands are SP EY E and SP RAN D, which generate identity and random matrices stored in sparse form, respectively.
The MATLAB system executes sparse matrix operations automatically. The general rules
are the following:
- The results of operations with dense matrices are dense.
- The results of operations with sparse matrices are sparse.
- The results of operations with mixed sparse and dense matrices are dense, except for
the cases when the result is expected to be sparse.
The sparse technique applied in MATLAB makes it possible that we can apply our programs
to sparse matrices without any modi…cation or rewriting. For example, we can apply our Gauss
program(s) to sparse matrices. The only requirement is that A should be declared as a sparse
matrix. If A is sparse, that MATLAB makes the computations in sparse form.
Several built in function gives sparse output for sparse input, and the built in linear solver
also automatically handles the sparse matrices.
Chapter 3
Approximate solution of nonlinear equations
We consider equations of the form
f (x) = 0 ( f : Rn ! Rn ; n
(3.1)
1).
A vector x 2 Rn is the solution (root, zero) of equation, if
f (x ) = 0:
If n
2, we speak of system of equations.
We always assume that f (x) is continuous.
Each of the suggested methods for solving f (x) = 0 produces a sequence fxi g1
i=0 that
converges to a solution x .
n
De…nition 38 The sequence fxi g1
i=0 2 R (n
there exists a constant 0 q < 1 such that kxi
1) converges to x 2 Rn with linear speed, if
x k q kxi 1 x k (i 1).
In case of linear convergence we have
kxi
xk
q kxi
1
xk
q 2 kxi
2
xk
:::
q i kx0
x k:
(3.2)
Thus the errors of iterations xi can be estimated by the members of a geometric sequence that
tends to 0.
31
Approximate solution of nonlinear equations
32
n
De…nition 39 The sequence fxi g1
1) converges to x 2 Rn with speed order p
i=0 2 R (n
(p > 1) if there exists a constant > 0 such that kxi x k
kxi 1 x kp (i 1).
The convergence speed of order p is faster than the linear convergence speed, when p = 1.
We can prove by induction that
kxi
p
xk
1
p
p 1
(
kx0
p
p 1
i
x k)p
(i
1) :
(3.3)
x k n< 1,othan the errors of approximations xi can be estimated by the
p
i
members of the sequence cq p (c = 1= p 1 ) that tends to 0.
If q =
p 1
kx0
This clearly converges to 0 faster than cq i .
The bisection algorithm
Theorem 40 (Bolzano) Assume that f : R ! R is continuous on [a; b] and
(3.4)
f (a)f (b) < 0:
Then equation f (x) = 0 has at least one zero x 2 (a; b).
Bolzano proved the theorem by the following "algorithm":
[a1 ; b1 ] = [a; b],
ci = (ai + bi ) =2, [ai+1 ; bi+1 ] =
It is clear that for i
[ai ; ci ] ;
[ci ; bi ] ;
if f (ai )f (ci ) < 0
, (i = 1; 2; : : :) :
otherwise
1,
x 2 [ai+1 ; bi+1 ]
[ai ; bi ] :
We can approximate zero x by any point y of the interval [ai ; bi ]. For the error of the
approximate zero y, we have
jx
yj
maxfy
ai ; b i
yg:
(3.5)
i
.
The bound maxfy ai ; bi yg is the smallest if y = ai +b
2
Thus, in general, we use xi = (ai + bi ) =2 as an approximation of the solution. It is clear
that
b i ai
b a
jx
xi j
=
(i = 1; 2; : : :):
(3.6)
2
2i
We stop the algorithm if the error of approximation is less than a given error bound " > 0.
Bisection algorithm:
input [a; b] ; " > 0.
while b a > "
Approximate solution of nonlinear equations
33
x = (a + b) =2
if f (a)f (x) < 0
b=x
else
a=x
end
end
x = (a + b) =2
Sequence fxi g converges to a zero of f (x) only if f is continuous.
Example 41 Let f (x) = 4 (1
x2 )
ex = 0 and compute the zeros!
y
-2.0
-1.5
-1.0
2
-0.5
0.5
1.0
1.5
-2
2.0
x
-4
-6
-8
-10
-12
-14
-16
-18
4 (1
ex
x2 )
This f (x) has zeros in the interval [ 1; 0], and [0; 1] and in both intervals
f ( 1) f (0) =
3e
1
< 0;
f (0) f (1) =
3e < 0:
Hence we can apply the bisection method.
For a given error bound " > 0, the solution of inequality
jxi
xj
b
a
2i
=
1
2i
"
(3.7)
gives an upper bound for the number of iterations.
In the above case " = 10 6 , i
log "= log 2 19:93, that is i = 20 iterations are enough
(x1
0:950455, x2 = 0:703439).
Matlab programs: bisect.m, fveq1.m
There exist high dimensional generalizations of the bisection method. But ....
Fixed point iteration
We consider the system of equations
x = g (x) ;
Approximate solution of nonlinear equations
34
where g : Rn ! Rn is continuous. Assume that D Rn is a bounded, closed region such that
conditions
g (x) 2 D; x 2 D
(3.8)
and
kg (x)
g (y)k
q kx
x; y 2 D
yk ;
(0
(3.9)
q < 1)
hold.
Then for any x0 2 D, the following …xed point iteration algorithm converges to the unique
solution of equation x = g (x).
The …xed point iteration algorithm:
Input x0 , " > 0.
while exit condition=false
xi+1 = g (xi ) ;
i=i+1
end
The usable exit (or termination or stopping) conditions are
(B)
kxi+1
xi k
"2 ;
(3.10)
(C) i = imax :
It is common to use the combination of the above termination criteria. Note however that
none of them guarantees a true result.
Example 42 Solve equation f (x) = 4 (1
precision " = 10 5 !
ex = 0 in the interval [0:68; 0:72] with a
x2 )
Since the root is positive, we can write the equation in the form x = g (x) =
the interval [0:68; 0:72]
p
g 0 (x) = ex = 4 4 ex < 0:
p
1
ex =4. In
Hence g (x) is strictly monotone decreasing and
max
x2[0:68;0:72]
jg 0 (x)j = jg 0 (0:72)j
0:3682
implies that g is contraction. Since
g (0:68) =
and
hp
1
hp
g (0:72) =
1
i
ex =4
i
= 0:711 71
x=0:68
ex =4
= 0:697 42;
x=0:72
the conditions of convergence are satis…ed.
Set x0 = 0:70. Using tol = 1e 6, termination criterion (B) and (C) the result is
7:034394488411407e 001.
Matlab programs: fpit.m, fveq2.m
Approximate solution of nonlinear equations
35
Rewriting equations f (x) = 0 in the form x = g (x) is easy. However it is di¢ cult to
guarantee the contractivity of g. In many cases the equivalent form is
x=x
where
=
(3.11)
f (x) ;
(x) is chosen such that g (x) = x
(x) f (x) be a contraction.
In fact, the Newton method can be interpreted this way, as well.
The Newton method
Assume that f : R ! R is continuously di¤erentiable. The essential element of the method is
that we take the tangent line to the graph of the function at the point xi and the zero of this
tangent line gives approximation xi+1 (to the zero of f (x)).
The tangent line equation at the point (xi ; f (xi )) is given by
f (xi ) = f 0 (xi )(x
y
xi ):
(3.12)
The solution of equation y = 0 is
xi+1 = xi
f (xi )
;
f 0 (xi )
(3.13)
provided that f 0 (xi ) 6= 0.
We can obtain this formula with a di¤erent argument. We linearize f (x) at the point xi ,
that is
f (x) f (xi ) + f 0 (xi )(x xi ):
(3.14)
Then we replace equation f (x) = 0 by the approximate equation f (xi ) + f 0 (xi )(x
whose zero approximates the zero of f (x) = 0.
The Newton method:
xi ) = 0
Input an initial approximation x0 2 R,
for i = 0; 1; : : :
xi+1 = xi
f (xi )
:
f 0 (xi )
(3.15)
end
Note that Newton’method is a …xed point iteration applied to the equation
x=x
f (x) =f 0 (x) :
Theorem 43 Assume that f : (a; b) ! R is twice continuously di¤erentiable, jf 00 (x)j
and
jf 0 (x)j
> 0 (x 2 (a; b)). If equation f (x) = 0 has a zero x in (a; b), then there exists a
number > 0 such that for jx0 x j < , xi ! x (i ! +1) and
jxi+1
xj
2
jxi
x j2
(i = 0; 1; : : :):
(3.16)
Approximate solution of nonlinear equations
36
Set
1
= min fx
x g > 0. Consider the Taylor expansion
a; b
xi )2 ;
f (x ) = f (xi ) + f 0 (xi ) (x –xi ) + (1=2) f 00 ( i ) (x
from which f (x ) = 0 implies
f 0 (xi ) (x
f (xi ) =
xi )
(1=2) f 00 ( i ) (x
xi ) +
1 f 00 ( i )
(xi
2 f 0 (xi )
xi )2 :
Substituting to the recursion we get
xi+1 = xi + (x
that is
xi+1
Conditions jf 00 (x)j
x =
and jf 0 (x)j
jx1
implies x1 2 (x
jxi+1
Hence for jx0
xj
jx0
2
2
jxi
x j2 ;
= min f 1 ; 2 = g, then
x j<
xj
;x + )
x )2 :
> 0 (x 2 (a; b)) imply
jxi+1
provided that xi 2 (a; b). If jx0
1 f 00 ( i )
(xi
2 f 0 (xi )
x )2 ;
x j jx0
xj
jx0
x j<
(a; b). Similarly we can prove that xi 2 (a; b) and
xj
2i+1
2
2
jx0
xj
(i = 0; 1; : : :):
x j < , the sequence fxi g1
i=0 is convergent with order p = 2.
The convergence of the Newton method is local, because the initial approximation x0 must
be close to the zero x .
The second order convergence of the Newton method can be characterized by the following
fact.
Assume that x 6= 0. Then we have
jxi+1 x j
jx j
jx j
2
jxi x j
jx j
2
maxfjaj ; jbjg
2
jxi x j
jx j
2
:
(3.17)
This means that the relative error of iteration xi+1 is the square of the relative error of xi .
If the …rst two decimal digits are correct, then the iteration achieves the best available
precision in 3–4 iterations in the standard double precision ‡oating point arithmetic.
Approximate solution of nonlinear equations
37
Termination criteria: We would like to have an approximation subject to
jxi
xj
(3.18)
":
In general, we have only heuristic exit conditions:
(A)
jf (xi )j
"1 ;
jxi+1
(B)
xi j
"2 ;
(C) i = imax :
(3.19)
None of these conditions can guarantee the tolerance condition (3.18).
In practice a combination of the three conditions is preferred.
The square root algorithm
xk+1 =
1
2
xk +
a
xk
(3.20)
(k = 0; 1; : : :)
is an application of the Newton method to equation f (x) = x2
a = 0.
Theorem 44 (Fourier). Assume that f 2 C 2 [a; b], f 0 (x) 6= 0 and f 00 (x) 6= 0, if x 2 [a; b].
Assume that there exists a zero x 2 (a; b). If the point x0 2 [a; b] is such that f (x0 ) f 00 (x0 ) > 0,
then the Newton method monotonically converges to the solution x .
The interval
[a; b] can be in…nite. In the above case f (x) = x2
p
for x0 > a, f (x0 ) f 00 (x0 ) > 0.
For such x0 , the Newton method always convergent.
a and f 00 (x) = 2. Hence
The Newton method in high dimension
The coordinate-wise form of equation
f (x) = 0 (x 2 Rn )
is given by
f1 (x1 ; x2 ; : : : ; xn ) = 0
..
.
fn (x1 ; x2 ; : : : ; xn ) = 0:
(i)
(i)
(i)
We linearize component fk (x) at the point xi = [x1 ; x2 ; : : : ; xn ]T 2 Rn (k = 1; : : : ; n):
fk (x1 ; x2 ; : : : ; xn )
(i)
(i)
fk (x1 ; x2 ; : : : ; x(i)
n )
n
X @
(i)
(i)
fk (x1 ; x2 ; : : : ; x(i)
+
n ) xl
@x
l
l=1
(i)
xl
:
Approximate solution of nonlinear equations
38
In compact form
f1 (x)
f1 (xi ) + rf1 (xi )T (x
..
.
xi )
fn (x)
fn (xi ) + rfn (xi )T (x
xi ):
Instead of solving f (x) = 0 we seek for the solution of the linear(ized) system of equations
f1 (xi ) + rf1 (xi )T (x
..
.
xi ) = 0
(3.21)
T
fn (xi ) + rfn (xi ) (x
xi ) = 0
which de…nes the new approximation xi+1 .
Observe that equation y = fk (xi )+rfk (xi )T (x xi ) de…nes the tangential plane of function
y = fk (x) at the point xi . The new iteration xi+1 is the common point of tangential planes on
the hyperplane y = 0.
The matrix
@fi (x)
@xj
J(x) =
n
i;j=1
2
3
rf1 (x)T
6
7
..
=4
5
.
T
rfn (x)
(3.22)
is called the Jacobian matrix of f (x). Using J (x) we can write the linear system in the form
f (xi ) + J(xi )(x
xi ) = 0;
(3.23)
whose solution is given by
xi+1 = xi
[J(xi )]
1
f (xi ) (i = 0; 1; : : :):
(3.24)
This recursion is the general Newton method. In practice we never compute the inverse of the
Jacobian matrix J(xi ). Instead we solve the linear system form.
Newton method:
Input x0 2 Rn , " > 0.
for i = 0; 1; 2; : : :
Solve the linear system J(xi )
xi+1 = xi + i
end
i
=
f (xi )!
The termination conditions are those of (3.19) and their combinations. The convergence of the
procedure is locally quadratic.
Example 45 Consider the nonlinear system of two variables
f1 (x) = x21 + x22 1 = 0;
f2 (x) = x21 x2 = 0:
Approximate solution of nonlinear equations
39
The tangential plane S1 of function z = f1 (x) at the point x1 = [1; 1]T is
z = 2x1 + 2x2
3;
while the tangential plane S2 of z = f2 (x) at x1 is
z=
2x1
x2 + 1:
The intersection of the two planes is a line that crosses the plane z = 0 at the point x2 =
[ 1=2; 2]T .
The following …gure shows the above situation.
4
2
z
0
-2
-4
1 .5
1 .0
0 .0
0 .5
0 .5
0 .0
x
1 .0
-0 .5
1 .5
-1 .0
y
2 .0
Solve the system f = [(f1 (x); : : : ; fn (x)]T = 0; x = [x1 ; : : : ; xn ]T using Newton’s method,
if
f1 (x) = x1 ;
fi (x) = cos(xi 1 ) + xi
1 (i = 2; : : : ; n)!
Set x(0) = [ 1; 1; : : : ; 1; 1]T . The Jacobian matrix:
2
1
0
0
6
6 sin(x1 )
1
0
6
.
J(x) = 6
0
sin(x2 ) . .
6
6
..
..
..
4
.
.
.
0
0
For n = 4, the elements of the sequence
x(k+1) = x(k)
0
..
.
..
.
3
7
7
7
..
7
.
7
7
..
.
0 5
sin(xn 1 ) 1
[J(x(k) )] 1 f (x(k) )
Approximate solution of nonlinear equations
40
are
2
6
x(1) = 6
4
2
6
x(3) = 6
4
3
0
0:3818 7
7;
0:7030 5
0:2098
3
0
0 7
7;
0 5
0:0025
2
6
x(2) = 6
4
2
6
x(4) = 6
4
3
0
0 7
7;
0:7020 5
0:1720
0:5e
3
0
0 7
7:
0 5
16
It can be shown that for any n 1, x(0) , the sequence x(k) converges to the unique solution
x = [0; : : : ; 0]T .
Matlab programs: Newt1.m, fN1.m, fN1p.m
Chapter 4
Numerical solution of Ordinary Di¤erential Equations
Explicit one-step methods
We consider initial value problems (IVPs) of the form
y 0 = f (x; y) ;
y (x0 ) = y0
f :R
R` ! R` ;
(4.1)
where
f (x; y) = [f1 (x; y) ; : : : ; f` (x; y)]T
is continuous on the open domain
D = f(x; y) : kx
x
and
y
x0 k1 <
ky
x;
(x 2 R; y 2 R` )
y 0 k1 <
yg
R`+1 ;
(4.2)
are positive constants and there exists a constant L > 0 such that
kf (x; y)
f (x; z)k1
L ky
zk1
((x; y) ; (x; z) 2 D) :
(4.3)
Then for all (x0 ; y0 ) 2 D, there exists a unique solution y (x) = [y1 (x) ; : : : ; y` (x)]T of the
IVP on some interval, that is
y 0 (x) = f (x; y (x)) ;
x 2 [x0 ; B] :
(4.4)
We seek for the approximate solution at discrete points (nodes)
x0 = t0 < t1 < : : : < tj < : : : < tN = b
of the interval [x0 ; b] (b
(4.5)
B).
41
Numerical solution of Ordinary Di¤erential Equations
42
De…nition 46 The set fti gN
i=0 is said to be a mesh or grid of interval [x0 ; b]. The quantity
hi = ti+1 ti > 0 is said to be the ith stepsize. The mesh is called equidistant, if
ti = t0 + ih (i = 0; 1; : : : ; N );
h=
b
t0
N
:
(4.6)
An approximate solution of the exact solution y (x) at the point ti is denoted by yi . Clearly,
y (t0 ) = y0 .
The explicit Euler method
A characteristic property of the Cauchy problem is that if y (x) is known at a point x, then we
know its derivative y 0 (x) = f (x; y (x)) as well.
Using the component-wise linearizations (tangential lines)
yi (x + h)
yi (x) + hyi0 (x) = yi (x) + hi f (x; y (x))
we have the approximation
2
3
y1 (x + h)
..
6
7
.
6
7
6
7
y (x + h) = 6 yt (x + h) 7
6
7
..
4
5
.
y` (x + h)
If the approximate value y (x)
(i = 1; : : : ; `)
y (x) + hy 0 (x) = y (x) + hf (x; y (x)) :
y^ is known at a point x, then the above formula changes
to
y (x + h)
y^ + hf (x; y^) :
The basic idea of the Euler method is the following:
At the point t1 = t0 + h0 approximate y (t1 ) by the tangential "line" at the point (x0 ; y0 ):
y (t0 + h0 )
y0 + h0 y 0 (t0 ) = y1 = y0 + h0 f (t0 ; y0 ) :
Using the approximation y1 at the point t1 and the above argument we obtain the approximation
y (t2 ) y (t1 ) + h1 f (t1 ; y (t1 )) y2 = y1 + h1 f (t1 ; y1 ) :
So we obtain the recursion
yi+1 = yi + hi f (ti ; yi )
where yi
(i = 0; 1; : : : ; N
1);
(4.7)
y (ti ).
This is called the explicit Euler method.
For scalar ODE (y0 = y, y (1) = 3, y (x) = 3ex 1 ) we can obtain the following …gure
(test_euler1.m):
Numerical solution of Ordinary Di¤erential Equations
43
explic it Euler method
25
20
y
15
10
5
ex ac t s olution
Euler poly gon
0
1
1.5
2
t
2.5
3
The points marked by rhomboids are the approximate solutions yi (ti = 1 + 0:5i, i =
0; 1; 2; 3; 4), while the blue line shows the theoretical solution.
De…nition 47 The quantity
T (y (x) ; h) = y (x + h)
(y (x) + hf (x; y (x)))
is called the local error of the Euler method at the point x.
De…nition 48 Error ei = yi
y (ti ) is called the global error at the point ti (i = 0; 1; : : : ; N ).
Consider the magnitude of local error T (y (x) ; h). Assume that y (x) 2 C 2 [x0 ; b]. Then
y (x + h) = y (x) + hy 0 (x) + R1 (x; h) ;
where
kR1 (x; h)k1
h2 K2
2K2 = max max jyi00 (x)j :
1 i ` x2[x0 ;b]
(4.8)
Hence
y 0 (x)
T (y (x) ; h) = (y (x) + hy 0 (x) + R1 (x; h))
= R1 (x; h) ;
z }| {
(y (x) + hf (x; y (x)))
which implies
kT (y (x) ; h)k1
Since e0 = y0
K2 h2
(x; x + h 2 [x0 ; b]):
y (x0 ) = 0, the following result holds.
Theorem 49 If y (x) 2 C 2 [x0 ; b], then for the global error of the Euler method we have
max kei k1
1 i N
(b
x0 ) K2 eL(b
x0 )
max hi :
0 i N 1
(4.9)
Numerical solution of Ordinary Di¤erential Equations
44
The result simply means that the global error of the Euler method is proportional to the
oN
n
(N )
(N )
is such that max0 i N hi ! 0
length of the greatest stepsize. If the mesh sequence ti
i=0
for N ! 1, then
max kei k1 ! 0; N ! 1:
(4.10)
1 i N
Thus the convergence rate of the Euler method is 1.
The nature of convergence is indicated on the next …gure, where the theoretical (y (x) =
x + x2 ) and numerical solutions (by Euler’s method) of problem
4
y 0 = 2y=x + 2x3 ; y (1) = 2
fare shown for equidistant grids and stepsizes h = 0:5; 0:05 (test_euler2.m).
explicit Euler method
90
exact solution
h= 0.5
h= 0.05
80
70
60
y
50
40
30
20
10
0
1
1.5
2
t
2.5
3
Explicit one-step methods
There are several methods that are more e¢ cient than the Euler method. One of these is the
class of explicit one-step methods of the form
yi+1 = yi + hi (ti ; yi ; hi )
Function (x; y; h) ( : R R`
function. It is assumed that
(i = 0; 1; : : : ; N
1):
(4.11)
R ! R` ), which depends of f (x; y) is called the increment
is continuous in each variables,
satis…es the Lipschitz condition
k (x; y; h)
where K
(x; z; h)k1
0 is constant and jhj
b
h,
K ky
zk1
((x; y) ; (x; z) 2 D)
(4.12)
Numerical solution of Ordinary Di¤erential Equations
45
and
(4.13)
(x; y; 0) = f (x; y) :
The last condition is said to be the condition of consistency.
The Euler method corresponds to the special case
(x; y; h) = f (x; y).
De…nition 50 The local error of explicit one-step methods at the point x is de…ned by
T (y (x) ; h) = y (x + h)
(4.14)
(y (x) + h (x; y (x) ; h)) :
De…nition 51 The one-step method is said to be of order p, if there exists a constant Kp > 0
such that
kT (y (x) ; h)k1 Kp hp+1 ; x; x + h 2 [x0 ; b] :
(4.15)
The quantity
ei = yi
(4.16)
y (ti )
is called the global error at the point ti (i = 0; 1; : : : ; N ).
If the consistency condition
(x; y; 0) = f (x; y) holds, then T (y (x) ; h) = o (h), that is
(y (x + h)
T (y (x) ; h)
=
h
h
y (x))
(x; y (x) ; h) ! y 0 (x)
f (x; y (x)) = 0:
If, in addition, we assume that y 2 C 2 and
k (x; y; h1 )
(x; z; h2 )k
K ky
K; q
zk + K2 jh1
h2 j
0; (x; y) ; (x; z) 2 D; jh1 j ; jh1 j
then T (y (x) ; h) = O (h2 ) follows from
(4.17)
b
h ;
kT (y (x) ; h)k = k(y (x + h) y (x) h (x; y (x) ; 0))
+h [ (x; y (x) ; 0)
(x; y (x) ; h)]k
k(y (x + h) y (x) hy 0 (x))k
+ h k (x; y (x) ; 0)
(x; y (x) ; h)k
O h2 + K2 h2 = O h2 :
The convergence of explicit one-step methods
By de…nitions and condition (4.3) we have
kyi+1
y (ti+1 )k = kyi + hi (ti ; yi ; hi )
kyi
[y (ti ) + hi (ti ; y (ti ) ; hi )
+T (y (ti ) ; hi )]k
y (ti ) + hi [ (ti ; yi ; hi )
(ti ; y (ti ) ; hi )]k
+ kT (y (ti ) ; hi )k
(1 + hi K) kyi
y (ti )k + kT (y (ti ) ; hi )k ;
(4.18)
Numerical solution of Ordinary Di¤erential Equations
46
that is
kei+1 k
(1 + hi K) kei k + kT (y (ti ) ; hi )k
(i = 0; 1; : : : ; N
1):
(4.19)
Lemma 52 For the solution of inequality
i+1
+
i i
we have
n
Y
n+1
i;
(
0
n
X
!
j
j=0
+
(4.20)
0; i = 0; 1; : : : ; n)
i
n
Y
i=0
j
j=i+1
!
i;
n
(4.21)
0:
Proof. For n = 0, by the recursion
1
0 0
+
0;
Q
while the closed formula ( ij=l f (j) = 1, if i < l) gives
0
Y
1
j
j=0
!
0
0
0
X
Y
+
i=0
Assume the statement for some integer n
Qn
n+1
n+2
j=0
j
Qn+1
j=0
+
0
j
j
j=1
!
i
=
+
0 0
0:
0. For n + 1, we have
n+1 n+1 +
Pn
n+1
i=0
0
+
n+1
Qn
j=i+1
j
i
j=i+1
j
i;
Pn+1 Qn+1
i=0
+
n+1
which was to be proved.
If we apply the Lemma with j = 1 + hj K and j = kT (y (tj ) ; hj )k, then we obtain the
inequality
" n
#
" n
#
n
Y
X
Y
(4.22)
ken+1 k
(1 + hj K) ke0 k +
(1 + hj K) kT (y (ti ) ; hi )k :
j=0
i=0
Using the inequality 1 + x
n
Y
ex (x
(1 + hj K)
j=i+1
j=i+1
eKhj and
0) we obtain 1 + hj K
n
Y
eKhj = eK
j=i+1
Pn
j=i+1
hi
eK(b
x0 )
:
(4.23)
Substituting into the formula we obtain the estimate
ken+1 k
eK(b
x0 )
ke0 k +
n
X
i=0
!
kT (y (ti ) ; hi )k :
(4.24)
Numerical solution of Ordinary Di¤erential Equations
47
If the method is of order p, then
max kei k
1 i N
eK(b
eK(b
max kei k
1 i N
x0 )
a)
K(b a)
e
ke0 k +
n
X
dp hp+1
i
i=0
ke0 k + dp
ke0 k + (b
!
p
max hi
0 i N 1
x0 ) dp
n
X
!
hi ;
i=0
p
max hi
0 i N 1
:
(4.25)
Theorem 53 If one-step method (4.11) is of order p and y0 = y (t0 ), then the convergence is
also of order p, that is
p
max kei k
1 i N
(b
K(b a)
x0 ) dp e
max hi
0 i N 1
:
(4.26)
The result means that the global error is proportional with the pth power of the greatest
n
oN
(N )
(N )
stepsize. If the mesh sequence ti
is such that max0 i N hi ! 0 for N ! 1, then
i=0
p
max kei k = O
1 i N
max hi
0 i N 1
! 0;
N ! 1:
(4.27)
For the explicit Euler method kT (y (x) ; h)k
K2 h2 , if y 2 C 2 . Hence its convergence
order is 1.
In general, a p order ( p 1) method may have convergence order p, only if exact solution
of the IVP is continuously di¤erentiable at least (p + 1)-times. If the exact solution is less
smooth, we can expect smaller convergence order.
Key words and phrases:
consistency
convergence
stability (It will be seen later).
Runge-Kutta methods and implementation matters
The p order one-step method approximates the exact solution y (x) with an error of order p + 1.
This means that the Taylor expansions of functions y (x + h) and y (x) + h (x; y (x) ; h) around
x the …rst p + 1 coe¢ cients are identical.
The simplest of such methods is the Taylor (expansion) method, where
(x; y (x) ; h) =
p
X
i=1
y (i) (x)
hi 1
:
i!
(4.28)
The higher order derivatives y (i) (x) can be determined with the composition rule and the
relation y 0 (x) = f (x; y (x)).
Numerical solution of Ordinary Di¤erential Equations
48
Example 54
y 00 (x) = (f (x; y (x)))0 = fx0 (x; y (x)) + fy0 (x; y (x)) y 0 (x) =
= fx0 (x; y (x)) + fy0 (x; y (x)) f (x; y (x)) :
(4.29)
In case of complicated derivatives and several variables the use of the Taylor method is not
advised.
Quite popular are the Runge-Kutta methods developed …rst by K.D.T. Runge and M.W.
Kutta around 1900.
K.D.T. Runge,
1856-1927
M.W. Kutta,
1867-1944
The explicit Runge-Kutta methods avoid the Taylor expansion and use only the information
y and f (x; y). Their general form is
P
yn+1 = yn + hn m
i=1 ci ki ;
k1 = f (xn ; yn ) ;
Pi 1
bij kj
(i = 2; : : : ; m):
ki = f xn + ai hn ; yn + hn j=1
(4.30)
It can be proved that
m
X
ci = 1
(4.31)
i=1
implies the consistency condition
(x; y; 0) = f (x; y) :
(4.32)
We also assume that the parameters satisfy
ai =
i 1
X
j=1
bij
(i = 2; : : : ; m):
(4.33)
Numerical solution of Ordinary Di¤erential Equations
49
The explicit Runge-Kutta methods can also be given by the following (matrix) arrangement:
0
a2
..
.
am
b21
..
.
..
(4.34)
.
: : : bm;m 1
: : : cm 1 c m
bm1
c1
The best known method is the following fourth order Runge-Kutta method:
0
1=2
1=2
1
1=2
0 1=2
0
0
1
1=6 1=3 1=3 1=6
(4.35)
The Euler method corresponds to the scheme
0
1
For the numerical solutions we generally require the global error bound
max kei k1
(4.36)
:
1 i N
It is easily provable that if
kT (y (tn ) ; hn )k1
hn
(n = 0; : : : ; N
1)
(4.37)
holds then
b
c
max kei k1
1 n N 1
also holds with a suitable constant b
c > 0.
(4.38)
Unfortunately b
c depends on the solution and the method and so it is generally unknown.
In practice we try to satisfy condition (4.36) so that for the stepsizes hn , we require
kT (y (tn ) ; hn )k1
hn =q
(i = 0; : : : ; N
1)
(4.39)
with an experimental constant q > 0. If (4.39) holds, then we consider the computed solutions
satisfying (4.36).
Condition (4.39) raises the question of estimating the local error. There are several analytic
and numerical estimates. We just deal with the two most popular ones.
Stephalving estimate (Runge’s rule). Starting from tn we make two steps with stepsize hn =2.
Thus we obtain a second approximation z^n+1 at the pointtn + hn . It can be proved that
T (y (tn ) ; hn )
z^n+1
2p
yn+1
;
1
(4.40)
Numerical solution of Ordinary Di¤erential Equations
50
where p is the order of method.
Imbedded Runge-Kutta formulae. Such methods consist of two Runge-Kutta formulae:
one order p and one order p + 1 that are selected so that the ki components of the lower
order Runge-Kutta method appear in the higher order method. The local error of the lower
order method is estimated by the di¤erence of the approximations given by the two methods.
Schematically
0
a2
..
.
b21
..
.
am
bm1
c1
d1
..
.
: : : bm;m 1
: : : cm 1 c m
: : : dm 1 dm
(4.41)
The lower order Runge-Kutta formula is
yn+1 = yn + hn (c1 k1 + : : : + cm km ) :
(4.42)
y^n+1 = yn + hn (d1 k1 + : : : + dm km ) :
(4.43)
The higher order formula is
The estimate of the local error is
T (y (tn ) ; hn )
hn
m
X
(di
c i ) ki .
(4.44)
i=1
The advantage of such formulae over the stephalving is that they use only one point information. However for m 5, this estimates cannot be asymptotically correct.
One of the most known such formula is the so called 2(3) Runge-Kutta-Fehlberg method
(ode23.m function in MATLAB):
0
1
1=2
1
1=4 1=4
1=2 1=2 0
1=6 1=6 4=6
Here a second order formula is embedded into a third order formula. In practice they continue
the procedure with the higher order Runge-Kutta method.
The …rst asymptotically correct formula is due to England, where a 4th order formula is
Numerical solution of Ordinary Di¤erential Equations
51
embedded to a 5th order formula and m = 6.
0
1=2
1=2
1
2=3
1=5
1=2
1=4
0
7=27
28=625
1=6
14=336
1=4
1
2
10=27
0
1=27
125=625 546=625 54=625
0
4=6
1=6
0
0
35=336
378=625
0
0
162=336 125=336
Denote by EST the norm of the estimated local error. The scheme of adaptive Runge-Kutta
methods is described as follows.
Adaptive one-step methods:
Input t0 ; y0 ; b; tol; h0 ; set i = 0:
while ti < b
1. Compute yi+1 and estimate the local error (EST )
2. if EST tol
i=i+1
else
hi = hnew
goto 1
end
end
The new stepsize hnew comes from the estimate
c3 hp+1
i
kT (y (xi ) ; hnew )k1
EST
and the condition
c3 hp+1
new
According to this, c3
EST =hp+1
i
and
hp+1
uj
hnew = hi
tol:
(tol=EST ) hp+1
, which implies
i
tol=c3
tol
EST
1=(p+1)
(4.45)
:
If the estimated error is essentially smaller than tol, then we can increase the stepsize
similarly to the decrease.
In certain cases this strategy is optimal.
Example 55 Solve the ”orbital” di¤erential equation
y10 = y3 ;
y20 = y4 ;
y30 = 2 y21 3=2 ;
(y1 +y2 )
0
y4 = 2 y22 3=2 ;
(y1 +y2 )
y1 (0) = 1
y2 (0) = 0;
y3 (0) = 0;
;
y4 (0) = [(1 + ) = (1
)]1=2 ;
Numerical solution of Ordinary Di¤erential Equations
52
with Matlab’s ode23 program on the interval [0; 20] for the parameter
= 0:3.
Using the default parameters the program gives the following solution trajectories (test_Orbital1.m):
o d e 2 3 re s u l t s o n o rb i t a l DE
1 .5
y (t )
1
y (t )
2
y (t )
1
3
y (t )
4
y co mp o n e n ts
0 .5
0
-0 . 5
-1
-1 . 5
0
2
4
6
8
10
t
12
14
16
18
20
Solution trajectories (ode23)
These trajectories were obtained by ode23 in 114 steps.
The change of stepsizes is shown by the next …gure
s te p s iz e s o f o d e 2 3
0 .4
0 .3 5
0 .3
h
i
0 .2 5
0 .2
0 .1 5
0 .1
0 .0 5
0
0
2 0
4 0
6 0
8 0
1 0 0
1 2 0
i
The change of stepsize (ode23)
If we solve the problem with the explicit Euler method using 4000 equidistant points (h =
0:005) we get the following (very bad) approximate trajectories (test_Orbital2.m):
Numerical solution of Ordinary Di¤erential Equations
53
E u l e r re s u l t s o n o r b i t a l D E
1 .5
1
y co mp o n e n ts
0 .5
0
-0 . 5
-1
y (t )
1
-1 . 5
y (t )
2
y (t )
3
y (t )
4
-2
0
2
4
6
8
10
t
12
14
16
18
20
Explicit Euler eredmények
Using the fourth order Runge-Kutta method (4.35) and 400 equidistant points (h = 0:05)
we obtain the following approximate solutions (test_Orbital3.m):
RK4 re s u l ts o n o rb i ta l DE
1 .5
y (t)
1
y (t)
2
y (t)
3
y (t)
4
1
y co mp o n e n ts
0 .5
0
-0 .5
-1
-1 .5
0
2
4
6
8
10
t
12
14
16
18
20
4th order Runge-Kutta results
The results obtained by the program ode23 and the 4th order Runge-Kutta method give
us quite good approximations. However the approximations obtained by the Euler method are
not very precise. The comparison of the three algorithms show that in general the higher order
methods are the better.
Linear multistep methods
For simplicity we assume that
y 0 = f (t; y) ;
y (t0 ) = y0
(f : R
R ! R) :
(4.46)
Numerical solution of Ordinary Di¤erential Equations
54
We also assume that the nodes of the mesh are equidistant, that is
ti = t0 + ih (i = 0; 1; : : : ; N );
h=
b
t0
:
(4.47)
j fn+j
(4.48)
N
The general form of linear k-step methods is
k
X
j yn+j
=h
j=0
k
X
jf
(tn+j ; yn+j ) = h
j=0
k
X
j=0
provided that approximations yi at the points ti (i n + k 1) are known. Here the coe¢ cients
i and i are given constants and k = 1.
If k 6= 0, the method is called implicit (yn+k appears in both sides), and we have to solve
a nonlinear equation at each step.
If
k
= 0, the method is called explicit.
We always assume that approximations yn ; : : : ; yn+k
(n 0) are known.
1
at the corresponding points tn ; : : : ; tn+k
Example 56 Explicit Euler method
yn+1
yn = hf (xn ; yn )
(4.49)
Example 57 Trapezoid formula
yn+1
yn =
h
(f (tn ; yn ) + f (tn+1 ; yn+1 )) :
2
(4.50)
Example 58 Simpson formula
h
(fn + 4fn+1 + fn+2 ) :
3
Example 59 BDF (backward di¤erentiation) formula
yn+2
k
X
yn =
j yn+j
= h k f (tn+k ; yn+k ) :
(4.51)
(4.52)
j=0
Example 60 Newton-Cotes formula:
4h
(2fn+1 fn+2 + 2fn+3 ) :
3
De…nition 61 The local error of the k-step method (4.48) is given by
yn+4 = yn +
T (y (x) ; h) =
k
X
[ j y (x + jh)
h j y 0 (x + jh)] :
(4.53)
(4.54)
j=0
The method is of order p (p
The quantity
1), if T (y (x) ; h) = O (hp+1 ), that is kT (y (t) ; h)k
ei = yi
y (ti )
is called the global error at the point ti (i = 0; 1; : : : ; N ).
dp hp+1 .
(4.55)
1
Numerical solution of Ordinary Di¤erential Equations
55
Using Taylor expansion of T (y (x) ; h) we can check that
T (y (x) ; h) = O hp+1
(4.56)
holds if and only if
k
X
j
(4.57)
= 0;
j=0
k
X
jj
s
=s!
jj
s 1
= (s
1)! = 0
(4.58)
(s = 1; 2; : : : ; p) :
j=0
The greatest p for which this condition holds is called the order of the method. In such a case
Chp+1 y (p+1) (x)
T (y (x) ; h)
(h ! 0) ;
(4.59)
where C 6= 0 is a constant that is independent of y (x). This means that T (y (x) ; h)
for polynomials of degree at most p.
For the above examples the values of p and C are the following:
k
Euler
1
trapezoid
1
Simpson
2
Newton-Cotes 4
p
1
2
4
4
0 holds
C
1=2
1=12
1=90
14=45
We associate two polynomials to method (4.48) as follows
The method is of order p
( )=
k
( )=
k
k
+
k 1
k
+
k 1
k 1
+
+
0;
(4.60)
k 1
+
+
0:
(4.61)
1 if and only if (1) = 0 and
0
(1) =
(1).
De…nition 62 The method (4.48) is called zero-stable if the zeros zi of
and the zeros whose modulus is 1 are simple zeros.
(z) satisfy jzi j
1
Theorem 63 (Dahlquist, 1956). The maximal order of a zero-stable linear k-step method is
at most k + 2. If k is odd, the maximal order is at most k + 1.
Rewrite formula (4.48) in the form of a one-step matrix di¤erence equation:
Numerical solution of Ordinary Di¤erential Equations
56
2
6
6
6
6
6
4
yn+k
yn+k 1
..
.
..
.
yn+1
3
2
k 1
7 6
7 6
7 6
7=6
7 6
5 4
that is
1
0
0
..
.
1
0
j=0
0
..
.
...
..
0
2 P
k
6
6
6
+ h6
6
6
4
k 2
.
..
0
jf
(tn+j ; yn+j )
0
..
.
..
.
0
.
1
3
0
0
32
76
76
76
76
76
54
yn+k
yn+k
..
.
..
.
yn
1
2
3
7
7
7
7
7
5
7
7
7
7
7
7
5
(4.62)
Yn+1 = AYn + hF (Yn ; Yn+1 ) ;
where
and
2
6
6
6
Yn = 6
6
4
yn+k
yn+k
..
.
..
.
yn
1
2
3
7
7
7
7;
7
5
2
k 1
6
6
6
A=6
6
4
k 2
1
0
0
..
.
1
0
2 P
k
j=0
6
6
6
F (Yn ; Yn ) = 6
6
6
4
jf
0
0
..
.
..
.
...
..
0
1
(tn+j ; yn+j )
0
..
.
..
.
0
.
0
0
3
7
7
7
7:
7
7
5
3
7
7
7
7
7
5
(4.63)
(4.64)
The coe¢ cient matrix A is the Frobenius companion matrix whose characteristic polynomial
coincides with (z) (Hence its eigenvalues are identical with the zeros of (z)).
The implicit equation has a unique solution yn+k for h small enough. After a rearrangement
we obtain
c
}|
{
z
k 1
k 1
X
X
yn+k =
j yn+j + h
j fn+j + h k f (tn+k ; yn+k ) ;
j=0
j=0
which is a …xed point equation of the form
contraction for hL < 1, because
j(c + hf (tn+k ; ))
= c + h k f (tn+k ; ). The right hand side is
(c + hf (tn+k ; ))j
hL j
j:
Numerical solution of Ordinary Di¤erential Equations
De…ne the global error vector
2
57
en+k
en+k
..
.
..
.
en
6
6
6
En = 6
6
4
Then by using the relations y 0 (x) = f (x; y (x)) and
yn+k =
k 1
X
j yn+j + h
j=0
y (tn+k ) =
k
X
jf
1
2
3
7
7
7
7
7
5
(4.65)
(tn+j ; yn+j ) ;
j=0
k 1
X
jy
(tn+j ) + h
j=0
k
X
jf
(tn+j ; y (tn+j )) + T (y (tn ) ; h)
j=0
we obtain
en+k =
k 1
X
j en+j
+h
j=0
k
X
j
[f (tn+j ; yn+j )
f (tn+j ; y (tn+j ))]
j=0
+ T (y (tn ) ; h) ;
that is
(4.66)
En+1 = AEn + G (En ; En+1 ) ;
where
2
6
6
6
G (En ; En+1 ) = 6
6
6
4
h
Pk
j
j=0
[fn+j
f (tn+j ; y (tn+j ))] + T (y (tn ) ; h)
0
..
.
..
.
0
The Lipschitz property of f implies that
k
X
j
[f (tn+j ; yn+j )
f (tn+j ; y (tn+j ))]
j=0
where r = L
Pk
j=0
k
X
j=0
j j j L ken+j k
r (kEn k + kEn+1 k) ;
j j j. It follows that
kG (En ; En+1 )k
hr (kEn k + kEn+1 k) + kT (y (tn ) ; h)k :
Here we used notation kEn k = max0
i k 1
ken+i k.
3
7
7
7
7:
7
7
5
(4.67)
Numerical solution of Ordinary Di¤erential Equations
58
The solution of the recursion
xk+1 = Axk + bk ;
k = 0; 1; : : :
is
k
xk = A x0 +
k 1
X
Ak
1 j
(4.68)
bj :
j=0
Hence the solution of recursion En+1 = AEn + G (En ; En+1 ) is
En+1 = A
n+1
E0 +
n
X
An j G (Ej ; Ej+1 ) :
j=0
Theorem 64 Let A 2 Cn n . Then Ak ! 0 if and only if every eigenvalue of A has modulus
strictly less than one; Ak is uniformly bounded for all k, if every eigenvalue of A has modulus
at most one and every Jordan block associated with an eigenvalue of modulus one is 1-by-1.
This condition is equivalent to the zero stability of a linear k-step method, which we assume
now. Then A satis…es the requirement of the above theorem.
Assume that kAn k
(and
1) hold for all n 1. Then
kEn+1 k
kE0 k +
2h r
n
X
j=0
n
X
j=0
[hr (kEj k + kEj+1 k) + kT (y (tj ) ; h)k] :
kEj k
!
+ h r kEn+1 k +
kE0 k +
n
X
j=0
!
kT (y (tj ) ; h)k ;
which implies for hr < 1,
(1
h r) kEn+1 k
Using the bound (1
x)
kEn+1 k
1
2h r
n
X
j=0
2 (0
4h r
j=0
kE0 k +
j=0
kEj k + 2
(n = 0; 1; : : : ; N
k
kE0 k +
1) :
n
X
j=0
xi
+
j=0
xj
1=2 we obtain
!
kT (y (tj ) ; h)k
This is an inequality of the type
i 1
X
!
kT (y (tj ) ; h)k :
1=2) and assuming h r
x
n
X
kEj k +
n
X
( > 0; n = 0; 1; : : :);
Numerical solution of Ordinary Di¤erential Equations
59
the solution of which is
x0
x1
;
+ x0
x2
+ (x0 + x1 )
+
=
(1 + ) ;
+
+ ( + x0 ) =
+
+2
=
(1 + )2 ;
..
.
xn
(1 + )n :
We prove the claim by induction. Case n = 0; 1 are clearly true. Assume that for k
claim holds. Then
xn+1
+
n
X
xj
n
X
+
j=0
=
+
(1 + )j =
j=0
n+1
(1 + )
(1 + )
1
1
=
2 e4nh
2 e
r
"
kE0 k +
4(b x0 ) r
"
(1 + )j
(1 + )n+1 ;
N
Xk
j=0
kE0 k +
= 4h r we get
N
Xk
2 (1 + 4h r)n kE0 k +
"
n
X
j=0
which was to be proved.
PN k
kT (y (tj ) ; h)k and
Setting = 2 kE0 k + j=0
kEn k
+
n, the
j=0
#
kT (y (tj ) ; h)k
#
kT (y (tj ) ; h)k
N
Xk
j=0
#
kT (y (tj ) ; h)k :
(4.69)
Hence we proved the following.
Theorem 65 If the linear k-step method (4.48) is zero-stable, b > t0 is …xed, then there is
constant h0 > 0 such that for all 0 < h h0 , the estimate
!
N
Xk
max kej k
max kei k +
kT (y (tj ) ; h)k
(4.70)
0 j N
holds with a constant
0 i k 1
j=0
> 0 that is independent of h.
Proof. Assume that h0 > 0 is such that h0 r 1=2. Then inequality (4.69) can be written in
the above form with constant = 2 e4(b x0 ) r .
Numerical solution of Ordinary Di¤erential Equations
60
De…nition 66 The linear k-step method (4.48) is stable in the Dahlquist sense, if for all 0 <
" < "0 and
N
Xk
max kei k +
kT (y (tj ) ; h)k ";
(4.71)
0 i k 1
j=0
we have
max kej k
0 j N
(4.72)
K";
where K is independent of h and ", but it may depend on the interval and f .
The previous theorem says that a zero-stable linear k-step method is stable in the Dahlquist
sense.
If the method and the initial approximations ei (i = 0; 1; : : : ; k 1) are such that for h ! 0
max kei k +
0 i k 1
N
Xk
j=0
kT (y (tj ) ; h)k ! 0,
then the convergence of the method also follows
Theorem 67 If the linear k-step method (4.48) is zero-stable, has order p (p 1), b > t0 is
p
…xed and max0 i k 1 kei k
3 h ( 3 > 0 constant), then the method is convergent in the sense
that for h ! 0 (N ! 1),
max kyj y (tj )k ! 0
(4.73)
0 j N
and
max kyj
y (tj )k
0 j N
Khp
(4.74)
hold with a suitable constant K > 0.
dp hp+1 . Inequality (4.70) implies
!
N
Xk
p
hp+1
( 3 + dp (b t0 )) hp :
3 h + dp
Proof. By assumption kT (y (t) ; h)k
max kej k
0 j N
j=0
Dahlquist (1956) proved the following su¢ cient and necessary theorem under the assumption
p 1.
Theorem 68 (Dahlquist). Assume that method (4.48) has order p
Dahlquist stable (convergent) if and only if it is zero-stable.
1. The method is
We do not prove this result. However we can show that zero-stability is necessary.
Consider ODE y 0 = 0, y (0) = 0 on the interval [0; 1] (b = 1). Then y (x)
0 and recursion (4.66) has the form
En+1 = AEn = An+1 E0 :
0, T (y (x) ; h) =
(4.75)
The sequence En is bounded if and only if Ak is uniformly bounded for all k, which is equivalent
to the zero-stability of the method.
Numerical solution of Ordinary Di¤erential Equations
61
Di¤erence methods for boundary value problems
Existence and unicity results for boundary value problems
Consider the second order di¤erential equation
y 00 = f (x; y; y 0 )
(4.76)
a0 y (a) a1 y 0 (a) = ; ja0 j + ja1 j =
6 0;
b0 y (b) + b1 y 0 (b) = ; jb0 j + jb1 j =
6 0:
(4.77)
under the boundary conditions
The solution is sought on the interval [a; b].
In order to characterize the solution consider the related initial value problem
u00 = f (x; u; u0 ) ;
a0 u (a) a1 u0 (a) = ;
c0 u (a) c1 u0 (a) = s:
(4.78)
Here the second condition is independent of the …rst, if a1 c0
generality we can assume that
a1 c0 a0 c1 = 1:
a0 c1 6= 0. Without loss of
(4.79)
Denote by u = u (x; s) the solution of IVP (4.78). Parameter s must be selected so that u (x; s)
satis…es the boundary condition at x = b, that is
(s) = b0 u (b; s) + b1 u0 (b; s)
= 0.
(4.80)
For …xed b and , this is a nonlinear equation for s. If it has a solution s , then we expect that
y (x) = u (x; s ) is the solution of the boundary value problem (4.76)-(4.77).
Theorem 69 Assume that f (x; u1 ; u2 ) is continuous on the region
R:a
x
b;
u21 + u22 < 1
(4.81)
and it is uniformly Lipschitz in variables u1 and u2 . Then the boundary value problem (4.76)(4.77) has as many solutions as there are distinct roots, s = s( ) , of equation (4.80). The
solutions are of the form y = u x; s( ) , where u is the solution of IVP (4.78) for s = s( ) .
We do not prove the result here. We note however, that it can by done by the substitution
u1 = u, u2 = u0 , which transforms problem (4.78) into a …rst order system of di¤erential
equations.
In many cases we can prove that equation (4.80) has exactly one solution.
Numerical solution of Ordinary Di¤erential Equations
62
Theorem 70 (Keller). Assume that f (x; u1 ; u2 ) satis…es the conditions of the previous theorem and has continuous derivatives on R which satisfy, for some positive constant M > 0,
@f
> 0;
@u1
@f
@u2
(4.82)
M:
Furthermore assume that coe¢ cients a0 , a1 , b0 and b1 satisfy
a0 a1
0;
b 0 b1
0;
ja0 j + jb0 j =
6 0:
(4.83)
Then the boundary value problem (4.76)-(4.77) has a unique solution.
This theorem has the following important consequence.
Theorem 71 Let the functions p, q and r be continuous on [a; b] with q (x) > 0 (x 2 [a; b]).
Assume that coe¢ cients a0 , a1 , b0 and b1 satisfy
a0 a1 0; ja0 j + ja1 j =
6 0;
b0 b1 0; jb0 j + jb1 j =
6 0;
ja0 j + jb0 j =
6 0:
(4.84)
Then the linear boundary value problem
Ly =
y 00 + p (x) y 0 + q (x) y = r (x)
a0 y (a)
has a unique solution for each
a1 y 0 (a) = ;
(4.85)
(a < x < b)
b1 y (b) + b1 y 0 (b) =
(4.86)
and .
For the linear boundary value problems we have a stronger result as well.
Theorem 72 Let p, q and r be continuous functions on [a; b]. Then for any
following mutually-exclusive alternatives hold: either the boundary value problem
Ly =
y 00 + p (x) y 0 + q (x) y = r (x) ;
y (a) = ;
and
y (b) =
the
(4.87)
has a unique solution, or else the corresponding homogeneous boundary value problem
Ly =
y 00 + p (x) y 0 + q (x) y = 0;
y (a) = 0;
y (b) = 0
(4.88)
has a nontrivial solution.
This means that problem (4.87) has a unique solution if and only if problem (4.88) has only
the trivial solution, y 0.
A similar result holds for the linear algebraic system Ax = b as well.
Numerical solution of Ordinary Di¤erential Equations
63
Numerical di¤erentiation
Lemma 73 Assume that f 2 C 3 . The error of approximation
f (x + h)
f 0 (x)
f (x
h)
(4.89)
2h
is O(h2 ).
Proof. Let
f (x + h) = f (x) + hf 0 (x) +
h3
h2 00
f (x) + f (3) ( 1 );
2
6
h2 00
f (x)
2
By subtraction and rearrangement one obtains
f (x
h) = f (x)
hf 0 (x) +
f (x + h)
f (x
h)
2h
where x
h<
3
h3 (3)
f ( 2 );
6
x<
x
1
< x + h;
h<
2
< x:
h2 (3)
= f (x) + [f ( 1 ) + f (3) ( 2 )]
12
h2
= f 0 (x) + f (3) ( 3 );
6
0
< x + h.
Lemma 74 Assume that f 2 C 4 . The error of approximation
f 00 (x)
f (x + h)
2f (x) + f (x
h2
h)
(4.90)
is O(h2 ).
Proof. Let
h2 00
h3
h4
f (x) + f (3) (x) + f (4) ( 1 );
2
6
24
2
3
h
h (3)
h4
f (x) + f (4) ( 2 );
f (x h) = f (x) hf 0 (x) + f 00 (x)
2
6
24
< x + h and x h < 2 < x. By addition and rearrangement one obtains
f (x + h) = f (x) + hf 0 (x) +
where x <
1
f (x + h)
2f (x) + f (x
h2
h)
=
h2 f 00 (x) +
= f 00 (x) +
h4
24
f (4) ( 1 ) + f (4) ( 2 )
h2
h2 (4)
f ( 3) ;
12
where x h < 3 < x + h.
Approximations to higher order derivatives can be constructed similarly. In general, we seek
for approximations of the form
f (j) (x)
n
1 X
ci f (x + ih);
hj i= m
(4.91)
Numerical solution of Ordinary Di¤erential Equations
64
where n+m
Let
j. The unknown coe¢ cients ci can be determined from the precision requirement.
f (x + ih) =
j
X
f (l) (x)
l=0
Then
n
X
ci f (x + ih) =
i= m
ci f (x + ih) =
i= m
If the relations
ci
i= m
By arrangement one obtains
n
X
n
X
j
X
l=0
n
X
l
ci i = 0
(0
j
X
l=0
il hl
f (l) (x)
l!
hl
f (l) (x)
l!
l
j
il hl
+ O hj+1 :
l!
n
X
i= m
n
X
+ O hj+1 :
!
+ O hj+1 :
n
X
1) ;
i= m
hold, then
c i il
!
ci ij = j!;
i= m
ci f (x + ih) = f (j) (x) hj + O hj+1 :
i= m
This immediately implies formula (4.91) with precision O (h).
Solution of linear boundary value problems
We study the linear second order boundary value problem
y 00 + p (x) y 0 + q (x) y = r (x)
(a < x < b)
(4.92)
y (a) = ;
y (b) = ;
where y; p; q; r : R ! R, and assume that it has a unique solution.
We use a di¤erence method. De…ne the grid
xj = a + jh (j = 0; 1; : : : ; N + 1);
use notation uj
quotients
h=
b a
;
N +1
(4.93)
y (xj ) and approximate the derivatives at the point xi by the di¤erence
y 00 (xi )
y (xi+1 )
y 0 (xi )
2y (xi ) + y (xi 1 )
ui+1 2ui + ui
2
h
h2
y (xi+1 ) y (xi 1 )
ui+1 ui 1
2h
2h
1
;
(4.94)
(4.95)
Numerical solution of Ordinary Di¤erential Equations
65
of order O (h2 ).
Substitute these approximations into the di¤erential equation at the point x = xi . For
1 i N , we obtain
ui+1
2ui + ui
h2
1
+ p (xi )
ui+1
ui
1
2h
+ q (xi ) ui = r (xi )
(4.96)
where
u0 = ;
uN +1 = :
By arrangement we obtain the equation
1
h
p (xi ) ui
2
h
h2 q (xi ) ui + 1 + p (xi ) ui+1 = h2 r (xi ) :
2
2
1
Introducing the notations
ai = 1
h
p (xi ) ; bi =
2
2
h
1 + p (xi )
2
h2 q (xi ) ; ci =
we obtain the tridiagonal linear system
b1 u1 + c 1 u2
= h2 r (x1 ) a1
ai ui 1 + bi ui + ci ui+1 = h2 r (xi )
(2
aN uN 1 + bN uN
= h2 r (xN ) cN
whose matrix is
2
6
6
6
6
6
Lh = 6
6
6
6
4
b1 c 1
a2 b 2
0 a3
0
c2
b3 c 3
.. ..
.
.
...
i
N
0
aN
0
(4.97)
3
7
7
7
7
7
7:
7
0 7
7
5
1
..
.
...
1)
...
bN 1 c N
aN
bN
1
0
Theorem 75 Assume that p; q 2 C [a; b] and
jp (x)j
Furthermore let h
P;
0 < Q1
q (x)
Q2 ;
a
x
b:
(4.98)
2=P . If y 2 C 4 [a; b], then
juj
y (xj )j
M
h2
(M4 + 2P M3 ) ;
12
holds, where M = max f1; 1=Q1 g and Mi = maxa
x b
0
j
N + 1;
y (i) (x) (i = 3; 4):
(4.99)
Numerical solution of Ordinary Di¤erential Equations
66
If y 2 C 3 [a; b], then
y(x + h)
y(x
h)
h2
M3 :
6
y 0 (x)
2h
If y 2 C 4 [a; b], then
y(x + h)
2y (x) + y(x
h2
h)
h2
M4 :
12
y 00 (x)
Hence
y (xi+1 )
2y (xi ) + y (xi 1 )
y (xi+1 ) y (xi 1 )
+ q (xi ) y (xi )
+
p
(x
)
i
h2
2h
= r (xi ) + i ; (4.100)
y (x0 ) = ;
where
y (xN +1 ) = ;
h2
h2
h2
M4 + jp (xi )j M3
(M4 + 2P M3 ) (1 i N ) :
12
6
12
yi = ei . By subtraction of equations (4.96) and (4.100) one obtains
j ij
Let y (xi )
ei+1
2ei + ei
h2
1
+ p (xi )
ei+1
ei
1
2h
+ q (xi ) ei = i ;
e0 = eN +1 = 0:
An arrangement yields
z
for 1
i
a
1
}|i
{
h
p (xi ) ei
2
z
1
2
c
}|i
{
}|
{
h
h2 q (xi ) ei + 1 + p (xi ) ei+1 = h2
2
z
bi
N . It can be written in the form
2
e1
6 e2
6 .
6
Lh h = Lh 6 ..
6 .
4 ..
eN
which implies
h
3
2
1
7
6
7
6
7
6
7 = h2 6
7
6
5
4
2
..
.
..
.
N
3
7
7
7
7=
7
5
h;
= Lh 1 h :
For a diagonally dominant matrix B,
B
1
1
1
mini jbii j
P
j6=i jbij j
:
i
Numerical solution of Ordinary Di¤erential Equations
67
By assumption q (x) < 0 for all x 2 [a; b]. Hence for small enough h > 0,
h2 q (xi ) > 2 =
jbii j = 2
h
p (xi ) + 1 +
2
{z
} |
1
|
Thus we have
Lh 1
>0
1
1
mini jbii j
k
hk
Lh
k hk
h4
12
=
j6=i
which implies that
1
P
jbij j
h2
X
h
p (xi ) =
jbij j :
2
{z
} j6=i
>0
1
;
mini jq (xi )j
(M4 + 2P M3 )
M4 + 2P M3
= h2
mini jq (xi )j
12 mini jq (xi )j
h2
h2
M (M4 + 2P M3 ) :
12
Hence we proved the theorem.
Hence if y 2 C 4 [a; b] the approximate solutions fuj gN
j=1 satisfy
uj = y (xj ) + O h2 ;
1
j
N:
Assume that approximate solutions must have the precision > 0. The stepsize h necessary
to achieve this is determined by the extrapolation (or Runge) principle.
This consists of solving linear system (4.97) with stepsizes h and h=2. If the di¤erence of
approximate solutions at the common points of the two meshes is less than c , where c is an
experimental number (for example, c = 1=100), then h is accepted.
If not, we halve h=2 to h=4 and so on. Doing so we can use the approximate solutions
belonging to the mesh de…ned by h=2.
Example 76 Solve the problem y 00 + x1 y = 0, y(1) = 1, y(2) = 2!
The exact solution of y 00 + x1 y = 0 is
p
p
p
p
C3 x BesselJ1 2 x + C4 x BesselY1 2 x ;
where
BesselJv (z) = Jv (z) = p
z v
2
v+1
2
Z
cos (z cos t) sin t2v dt
0
and
BesselYv (z) = Yv (z) =
Jv (z) cos v
sin v
J
v
(z)
:
Numerical solution of Ordinary Di¤erential Equations
68
The constants C3 and C4 can be obtained from the boundary conditions:
p
p
p
p
C3 x BesselJ1 2 x + C4 x BesselY1 2 x
x=1
0:576 72C3
p
p
p
p
C3 x BesselJ1 2 x + C4 x BesselY1 2 x
0:107 03C4 = 1;
x=2
0:565 96C3 + 0:386 25C4 = 2;
which gives C3 = 2:1187 and C4 = 2:0735.
The error of the numerical solution for N = 9 is shown on the next …gure (test_odeivp1.m):
-5
-1
abs olute error
x 10
-2
-3
-4
y
-5
-6
-7
-8
-9
-1 0
-1 1
1
1.1
1.2
1.3
1.4
1.5
t
1.6
1.7
1.8
1.9
2
Approximation error for N = 9.
Solution of nonlinear boundary value problems
Consider the second order boundary value problem
y 00 = f (x; y; y 0 ) (a < x < b)
y (a) = ; y (b) =
(4.101)
using the di¤erence quotients (4.94)-(4.95). Then we obtains the nonlinear system of equations
uj+1
where u0 =
2uj + uj
h2
1
f
xj ; uj ;
uj+1
uj
2h
1
= 0;
(j = 1; : : : ; N );
(4.102)
and uN +1 = .
Theorem 77 Assume that f (x; y; z) is continuously di¤erentiable and
@f
@z
P;
0 < Q1
@f
@y
Q2 :
(4.103)
Numerical solution of Ordinary Di¤erential Equations
69
2=P . If y 2 C 4 [a; b], then
Furthermore let h
juj
y (xj )j
M
h2
(M4 + 2P M3 )
12
holds, where M = max f1; 1=Q1 g and Mi = maxa
Consider the local error
j
=
Since y 00 (xj )
y (xj+1 )
j
x b
(0
j
(4.104)
N + 1);
y (i) (x) (i = 3; 4):
which is de…ned by the formula
2y (xj ) + y (xj 1 )
h2
f
xj ; y (xj ) ;
y (xj+1 )
y (xj 1 )
2h
(4.105)
:
f (xj ; y (xj ) ; y 0 (xj )) = 0, we can write
j
y (xj+1 )
=
2y (xj ) + y (xj 1 )
y 00 (xj )
2
h
y (xj+1 ) y (xj 1 )
xj ; y (xj ) ;
f (xj ; y (xj ) ; y 0 (xj ))
2h
f
and so
j jj
y (xj+1 )
2y (xj ) + y (xj 1 )
h2
y 00 (xj ) +
y (xj+1 )
+P
Using the notation ej = uj
y (xj 1 )
2h
y 0 (xj ) = O h2 :
y (xj ) and subtraction we obtain
ej+1 2ej +ej
h2
1
f xj ; uj ;
uj+1 uj
2h
f xj ; y (xj ) ;
1
y(xj+1 ) y(xj
2h
1)
i
(4.106)
=
j:
We use the following result (Lagrange’s mean value theorem): Let f : Rm ! R, u; v 2 Rm and
assume that f is di¤erentiable on the line segment [u; v] Rm . Then there is a point c 2 (u; v)
such that
m
X
f (v) f (u) =
fx0 i (c) (vi ui ) :
(4.107)
i=1
Thus we have
f
xj ; uj ;
uj+1
uj
2h
1
f
xj ; y (xj ) ;
y (xj+1 )
y (xj 1 )
2h
= qj ej
+ pj
ej+1
ej
2h
1
;
Numerical solution of Ordinary Di¤erential Equations
70
where
qj =
@f
@y
xj ; y (xj ) +
j ej ;
pj =
@f
@z
xj ; y (xj ) +
j ej ;
and
with 0 <
j
y (xj+1 )
y (xj 1 )
2h
y (xj+1 )
y (xj 1 )
2h
+
j
ej+1 ej
2h
+
j
ej+1 ej
2h
< 1. By the initial assumptions
0 < Q1
qj
Q2 ;
jpj j
P:
ej+1
ej
Hence we can rewrite (4.106) in the forms
ej+1
and
2ej + ej
h2
h
1 + pj ej
2
1
qj ej + pj
=
2h
j
h
pj ej+1 =
2
2 + h2 qj ej + 1
1
1
h2
j
for j = 1; 2; : : : ; N , which is quite similar to the linear case. Observe that the coe¢ cients are
depending implicitly from the unknown values ej . However under the conditions matrix Lh is
diagonally dominant for small enough h, and so
Lh 1
1
h2
1
mini qi
holds for any value distribution of ej ’s . Thus we obtain the relation maxj jej j = O (h2 ), which
proves the theorem.
We usually apply the Newton method to solve the system of nonlinear equations. Then
the Jacobian matrix is tridiagonal. Stepsize h necessary for a given tolerance " > 0 is also
established using the Runge principle. For good initial values of the Newton iterations we have
to make guessing, in general.
The Jacobian matrix
The components of nonlinear equation F (u) = 0 are the following:
F1 (u) =
Fi (u) =
ui+1
FN (u) =
u2
2u1 +
h2
2ui + ui
h2
1
f
f
2uN + uN
h2
x1 ; u1 ;
u2
2h
ui+1
ui
xi ; ui ;
1
f
2h
xN ; uN ;
1
=0
(i = 1) ;
=0
(2
uN
2h
1
i
N
(i = N ) :
1) ;
Numerical solution of Ordinary Di¤erential Equations
The Jacobian matrix J(u) =
components:
h
@Fi (u)
@xj
in
71
is a tridiagonal matrix with the following "non-zero"
i;j=1
@F1 (u)
2
u2
0
=
f
x
;
u
;
1
1
y
@u1
h2
2h
@F1 (u)
1
u2
1 0
= 2
fz x1 ; u1 ;
@u2
h
2h
2h
@Fi (u)
@ui 1
=
@Fi (u)
@ui
=
@Fi (u)
@ui+1
=
1
h2
+
1 0
f
2h z
xi ; ui ; ui+12hui
fy0 xi ; ui ; ui+12hui
2
h2
1
h2
1 0
f
2h z
1
1
xi ; ui ; ui+12hui
1
;
;
;
(i = 2; : : : ; N
;
1)
;
@FN (u)
1
1
= 2 + fz0 xN ; uN ;
@uN 1
h
2h
2
@FN (u)
=
fy0 xN ; uN ;
@uN
h2
uN 1
;
2h
uN 1
:
2h
Using this we can produce a function routine.
Example 78 Solve the problem
y 00 = (y + x + 1)3 =2;
Here f (x; y; z) = 21 (1 + x + y)3 , fy0 =
components of the Jacobian matrix are
@F1 (u)
=
@u1
@Fi (u)
@ui 1
=
@Fi (u)
@ui
=
@Fi (u)
@ui+1
=
2
h2
3
2
y (0) = y (1) = 0:
(1 + x + y)2 and fz0 = 0.
3
(1 + x1 + u1 )2 ;
2
Thus the non-zero
@F1 (u)
1
= 2;
@u2
h
1
;
h2
2
h2
3
2
(1 + xi + ui )2 ;
(i = 2; : : : ; N
@FN (u)
=
@uN
3
(1 + xN + uN )2 :
2
1) ;
1
;
h2
@FN (u)
1
= 2;
@uN 1
h
2
h2
Starting from the initial value u(0) = [5; 5; : : : ; 5]T the Newton iteration gives the following
results for N = 10 and N = 20 (test_odeivp2.m):
Numerical solution of Ordinary Di¤erential Equations
72
0
N= 1 0
N= 2 0
-0 .0 2
-0 .0 4
-0 .0 6
y
-0 .0 8
-0 .1
-0 .1 2
-0 .1 4
-0 .1 6
-0 .1 8
0
0 .1
0 .2
0 .3
0 .4
0 .5
t
0 .6
0 .7
0 .8
0 .9
1
Example 79 Solve the boundary value problem
3yy 00 + (y 0 )2 = 0;
y(2) = 1;
y(6) = 5:
The exact solution is
3
2
y(x) =
5p
3
5+
2
5p
3
5
4
3=4
1
4
x
:
We can rewrite the equation in the form y 00 = (y 0 )2 = (3y) with f (x; y; z) = z 2 = (3y). After
discretization the Newton method starting from u(0) = [1; 1; : : : ; 1]T gives the following results
for N = 10 and N = 20 (test_odeivp3.m):
5
N= 1 0
N= 2 0
4 .5
4
y
3 .5
3
2 .5
2
1 .5
1
2
2 .5
3
3 .5
4
t
4 .5
5
5 .5
6
We can observe the close agreement of the two numerical solutions.
Example 80 Solve the problem
y 00 + y 2 = x2 + 1;
y (0) = y (1) = 0:
Here f (x; y; z) = 1 + x2 y 2 . We can obtain two di¤erent solutions (bifurcation phenomenon). Starting from u(0) = [1; 1; : : : ; 1]T the Newton iteration give the following approximate
solutions
Numerical solution of Ordinary Di¤erential Equations
73
0
N= 1 0
N= 2 0
-0 .0 2
-0 .0 4
y
-0 .0 6
-0 .0 8
-0 .1
-0 .1 2
-0 .1 4
-0 .1 6
0
0 .1
0 .2
0 .3
0 .4
0 .5
t
0 .6
0 .7
0 .8
0 .9
1
Starting from u(0) = [5; 5; : : : ; 5]T we obtain the following approximate solutions
12
N= 1 0
N= 2 0
10
y
8
6
4
2
0
0
0 .1
0 .2
0 .3
0 .4
0 .5
t
0 .6
0 .7
0 .8
0 .9
1
In order to understand the phenomenon we apply expansion technique. Approximate the
theoretical solution with the fourth degree polynomial y (x) ax+bx2 +cx3 +dx4 and substitute
it into the di¤erential equation:
@2
@x2
2
(ax + bx2 + cx3 + dx4 ) + (ax + bx2 + cx3 + dx4 ) =
= 2b + 6cx + x2 (a2 + 12d) + 2abx3 + x4 (b2 + 2ac) + O (x5 ) = 1 + x2 :
2
By comparison we obtain b = 1=2, c = 0 and d = 1 12a . Using the boundary condition y (1) = 0
we obtain
y (1) = a + b + c + d = 0;
2
which implies the quadratic equation a + 0:5 + 1 12a = 0 (equivalently a2 12a 7 = 0) the
p
solutions of which are a1;2 = 6
43. Thus we obtain two di¤erent approximate solutions
p
2
p
1
43 + 6
2
y1 (x)
6 + 43 x + 0:5x +
x4 ;
12
and
p
2
p
1
43 6
2
y2 (x)
6
43 x + 0:5x +
x4 :
12
Numerical solution of Ordinary Di¤erential Equations
74
The two approximate solutions are shown on the next …gure
y6
5
4
3
2
1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
x
It can be seen that the expansion approximations are not very precise. In order to increase the
precision we must add higher order terms in a very tricky way (we have to make element 2abx3
disappear).
The ODE programs of Matlab (A quick reminder)
The Matlab has several good quality programs for solving ordinary and partial di¤erential
equations.
For solving initial value problems of ODE’s we can use the routines
ode23, ode45, ode113, ode15s, ode23s, ode23t, ode23tb.
For solving boundary value problems of ODE’s we can use the programs
bvp4c, bvp5c.
For retarded ODE’s one can use the routine dd23.
For solving partial di¤erential equations we can use two programs:
- pdepe,
- Partial Di¤erential Equation Toolbox.
Symbolic solution of ODEs is possible in many cases with the program dsolve whose general
form is
dsolve(’eqn1’,’eqn2’, ...).
Chapter 5
Numerical methods for partial di¤erential equations
Partial di¤erential equations (PDEs) appear in several problems of physics, engineering, biology,
economics, etc. Their study started in the 18th century.
A partial di¤erential equation (PDE) is an equation in which at least one partial derivative
of an n-variable functions, where n 2.
The general form of the …rst order PDEs (n = 2) is
F x; y; u0x ; u0y = 0;
(5.1)
where F is a given function. In the n-variable case the common form is
F x1 ; x2 ; : : : ; xn ; u0x1 ; : : : ; u0xn = 0;
(5.2)
where u = u (x1 ; : : : ; xn ) and F is a given 2n-variable function.
The implicit form of second order PDEs (n = 2) is
F x; y; u; u0x ; u0y ; u00xx ; u00xy ; u00yy = 0;
u = u (x; y) :
(5.3)
Linear PDEs play an important role in many applications.
The …rst order linear inhomogeneous PDEs have the general form (n = 2)
f1 (x; y) u0x + f2 (x; y) u0y + f0 (x; y) u + f (x; y) = 0;
where f0 ; f1 ; f2 ; f are given functions. If f (x; y)
u = u (x; y) ;
(5.4)
0, then the PDE is homogeneous.
75
Numerical methods for partial di¤erential equations
76
The second order linear inhomogeneous PDEs have the general form
X
@u
@2u
+
+ C (x) u = f (x) ;
Bj (x)
Aij (x)
@xi @xj j=1
@xj
i;j=1
n
X
n
where Aij , Bj , C and f are the functions of vector x on a domain D
(5.5)
Rn .
We enlist some famous PDEs:
2D heat equation:
u = u00xx + u00yy =
1 @u
:
2 @t
The Laplace equation:
u = 0:
The Poisson equation:
u + f = 0:
The wave equation (D’Alembert equation):
u=
1 @2u
:
c2 @t2
For PDE (5.5), we de…ne the quadratic form
Q ( 1; : : : ;
n)
=
n
X
Aij (x)
(5.6)
i j:
i;j=1
In every point x 2 D the quadratic form Q can be transformed into the canonical form
Q=
n
X
2
i i
(5.7)
i=1
where
If
i
i
2 f 1; 0; 1g.
= 1 (i = 1; : : : ; n), or
i
=
1 (i = 1; : : : ; n), then PDE (5.5) is elliptic at the point x.
If exactly one i is negative [positive], while the other n
[negative], then PDE (5.5) is hyperbolic at the point x.
If one of the coe¢ cients
i
1 coe¢ cients
i
are positive
is zero, then PDE (5.5) is parabolic at the point x.
If the same condition holds in every point of region D, the PDE is called elliptic, hyperbolic
or parabolic.
The solution of PDE’s is much more di¢ cult than those of the ODEs because of their
essentially di¤erent properties.
For example, an order n linear homogeneous ODE may have n independent solutions and
the general solution has n independent constants.
Numerical methods for partial di¤erential equations
77
Example 81 Consider the homogeneous linear PDE
2u0x
u0y = 0:
If we take any di¤erentiable function f , then the composite function
(5.8)
u = f (x + 2y)
solves the PDE, because
u0x = f 0 (x + 2y) ;
u0y = 2f 0 (x + 2y) :
We can prove that function (5.8) is the most general solution. Introduce the new variables
t = x + 2y and s = x! Then
u0x = u0t t0x + u0s s0x = u0t + u0s ;
u0y = u0t t0y + u0s s0y = 2u0t
implies
2u0x
u0y = 2u0s = 0:
The general solution of equation 2u0s = 0 is u = f (t) = f (x + 2y).
Problem 82 Consider the solution of the homogeneous second order PDE
au00xx + bu00xy + cu00yy = 0;
(5.9)
where a; b; c are constants!
We seek for the general solution in the form
u = f (y + mx) ;
where f is twice di¤erentiable function. Substituting expressions
u00xx = m2 f 00 (y + mx) ;
u00xy = mf 00 (y + mx) ;
u00yy = f 00 (y + mx)
into the PDE we obtain
am2 + bm + c f 00 (y + mx) = 0:
This gives the condition am2 + bm + c = 0 for m.
If we have two di¤erent zeros (m1 6= m2 ), then by the linearity any function of the form
u = f (y + m1 x) + g (y + m2 x)
(5.10)
is a solution of the PDE provided that g is also twice di¤erentiable. Introducing the new
variables s = y + m1 x and t = y + m2 x the PDE (5.9) takes the form u00st = 0. This implies
that (5.10) is indeed the most general solution.
Numerical methods for partial di¤erential equations
78
If m1 = m2 , then we do the following. First we assume that m1 6= m2 . Then
h (y + m1 x) h (y + m2 x)
m2 m1
is a solution of (5.9). If m2 ! m1 , then this solution converges to
[h0m (y + mx)]m=m1 = xh0 (y + m1 x) :
Using g = h0 we obtain the general solution
(5.11)
u = f (y + m1 x) + xg (y + m1 x)
of equation (5.9) in case of multiple zero.
Example 83 For
u00xx
we have m2
3u00xy + 2u00yy = 0;
3m + 2 = 0 with the solutions m1 = 1, m2 = 2. Hence the general solution is
u = f (x + y) + g (2x + y) :
Example 84 For
u00xx
we have m2
2u00xy + u00yy = 0;
2m + 1 = 0, m1 = m2 = 1 and the general solution
u = f (x + y) + xg (x + y) :
Although a PDE may have in…nitely many solutions, the solution can be made unique with
supplementary (initial and boundary value) conditions.
Parabolic equations
We show the basic techniques on the 1D heat equation.
Denote by u = u (x; t) the temperature of a rod of length L at the point x and time t.
u(L,t)=0
u(0,t)=0
u(x,0)=f(x)
The change of temperature is described by the 1D heat equation
u0t = c2 u00xx
(0 < x < L; t > 0) ;
(5.12)
where c is a constant. The two ends of the rod have the temperature 0 C (insulated ends). The
initial temperature of the rod depends on the point x, and it is given by a function f (x) C.
Thus we have the boundary conditions
u (0; t) = 0;
u (L; t) = 0
(t > 0) ;
(5.13)
and the initial condition
u (x; 0) = f (x)
(0 < x < L) :
(5.14)
Numerical methods for partial di¤erential equations
79
The method of separation of variables
Its a classical method who goes back to Fourier.
We seek for the solution in the product form u (x; t) = X (x) T (t).
Then by substitution we obtain
u0t =
@ 2 X (x)
@T (t)
X (x) = c2 T (t)
= c2 u00xx :
2
@t
@x
If we divide both sides with the product X (x) T (t) (X (x) T (t) 6= 0) we obtain the equality
X 00 (x)
1 T 0 (t)
=
=
c2 T (t)
X (x)
2
;
where the left side is a function of t, while the right side is a function of x.
2
This is possible if both sides are equal to a (separation) constant
.
We can derive two ordinary di¤erential equations
T0 +
2 2
X 00 +
2
c T =0
and
X = 0:
The solutions of these di¤erential equations are
T (t) = e
2 c2 t
and
X (x) = a sin ( x) + b cos ( x) ;
respectively.
The boundary condition
u (0; t) = u (0; 0) = X (0) T (0) = b = 0
implies b = 0. Assumption T (L) 6= 0 implies that
u (L; t) = X (L) T (L) = 0
may hold if X (L) = a sin ( L) = 0.
For a 6= 0, the solution of a sin ( L) = 0 is = n =
Hence a particular solution of the heat equation is
an sin
n c 2
n x
n
e ( L ) t = an sin
x e
L
L
n
L
for n = 1; 2; : : :.
2t
n
(n = 1; 2; : : :) ;
where n = nL c .
We can consider the sum (superposition) of the particular solutions as the general solution
u (x; t) =
1
X
n=1
an sin
n
x e
L
2t
n
:
(5.15)
Numerical methods for partial di¤erential equations
80
We can determine the unknown coe¢ cients an using the initial condition:
f (x) = u (x; 0) =
1
X
n
x :
L
an sin
n=1
This is a sine Fourier series and so
Z
2 L
n
x dx
an =
f (x) sin
L 0
L
(5.16)
(n = 1; 2; : : :) :
Example 85 Solve the problem
u0t = u00xx (0 < x < ; t > 0)
u (0; t) = 0; u ( ; t) = 0 (t > 0) ;
u (x; 0) = 100 (0 < x < ) :
Using the general solution we have
Z
2
200
an =
100 sin (nx) dx =
(1
n
0
Since cos n = 1, if n is even and cos n =
u (x; t) =
=
400
n=0;
1
X
(n = 1; 2; : : :) :
1, if n is odd, the general solution is
1
X
400
cos n )
1
e
n
n=2k
e
n2 t
(2k+1)2 t
sin ((2k + 1) x) :
2k + 1
k=0
sin (nx)
If we take the …rst 11 members of the series of u (x; t), then we obtain the following heat
distribution in the function of t (0 t 5) (D1heat_demo1.m):
s o lu tio n o f 1 D h e a t e q u a tio n
120
100
u (x, t )
80
60
40
20
0
3 .5
3
5
2 .5
4 .5
4
2
3 .5
3
1 .5
2 .5
2
1
1 .5
0 .5
1
0
x
0 .5
0
t
We can observe the fast decrease of temperature in time t.
Numerical methods for partial di¤erential equations
81
Finite di¤erence methods
Consider the solution of the heat equation
u0t = u00xx (0 x L; 0 t T )
u (0; t) = u0 ; u (L; t) = uL ; u (x; 0) = f (x) ;
where
(5.17)
> 0 is a constant.
Discretize the space and time variables:
tk = (k
1) t;
1
k
M;
xj = (j
1) x;
1
j
where
t=
T
M
1
;
L
x=
N
1
:
Let
Ujk = u (xj ; tk ) :
Note that
U1k = u (0; tk ) = u0 ;
UNk = u (L; tk ) = uL :
The …nite di¤erence approximation of u0t at the points (xj ; tk ) is either
Ujk+1 Ujk
@U (xj ; tk )
=
+ O ( t) ;
@t
t
or
Ujk+1 Ujk
@U (xj ; tk )
=
@t
2 t
1
+ O ( t)2
Similarly, we obtain
k
Uj+1
2Ujk + Ujk
@ 2 U (xj ; tk )
=
@x2
( x)2
1
+ O ( x)2 :
Here we de…ned a discrete mesh shown in the next …gure
N;
Numerical methods for partial di¤erential equations
82
k+1
k
k-1
t=0
1
N
j-1 j j+1
x=0
x=L
where the encircled points are used in the …nite di¤erence approximations of the derivatives.
The explicit Euler method
The di¤erence scheme uses the green points of the next …gure to approximate the derivatives
k+1
k
t=0
1
N
j-1 j j+1
x=0
x=L
Hence the approximate equation has the form
Ujk+1
Ujk
t
Using substitution s =
=
t
( x)2
2
( x)
k
Uj+1
2Ujk + Ujk
1
(j = 2; : : : ; N
1) ;
and some arrangements we obtain the recursion
Ujk+1 = sUjk
1
+ (1
k
2s) Ujk + sUj+1
(j = 2; : : : ; N
1) :
Numerical methods for partial di¤erential equations
83
We can write this in the form
sUk1 + (1
U2k+1 =
..
.
Ujk+1 = sUjk
..
.
UNk+11 = sUNk
1
2s) U2k + sU3k
k
2s) Ujk + sUj+1
+ (1
2
2s) UNk
+ (1
1
+ sUkN
and also in the compact matrix form
2
3
z
2
U2k+1
6 .. 7 6
6 . 7 6
6 k+1 7 6
6 Uj
7=6
6 . 7 6
4 .. 5 4
UNk+11
A
1
s
0
0
}|
2s s
0
0
1 2s s
...
... ...
0
.. ..
. s
.
0 s
1
3{ 2
2s
3
U2k
7 6 .. 7
76 . 7
76
7
7 6 Ujk 7
76 . 7
5 4 .. 5
UNk
1
2
U1k
0
..
.
6
6
6
+ s6
6
4 0
UNk
3
7
7
7
7 (5.18)
7
5
where the matrix is a symmetric tridiagonal matrix, Uj0 = U (xj ; 0) = f (xj ), U1k = u0 and
UNk = uL .
It is easy to program this iteration. However the questions of stability and convergence
immediately occurs.
Denote by L = @t c2 @xx the PDE operator. The discrete PDE operator de…ned on the
mesh should be denoted by L t; x . The solution of the discrete problem (5.18) is denoted U .
Note that U 6= u.
The discrete solution U converges to the continuous (and exact) solution, if in some appropriate norm
lim ku U k = 0
t; x!0
holds.
The discretization is called consistent, if
lim
t; x!0
kLu
L
t; x uk
=0
holds in some norm.
The discretization is said to be (numerically) stable, if the perturbation of the initial conditions does not increase unbounded. If the scheme is of the form z n+1 = Az n ( A is a matrix) then
Numerical methods for partial di¤erential equations
84
it is commonly understood under stability that there is a constant C (A) such that Ak
holds for all k.
C (A)
The Lax-Richtmyer equivalence theorem: "Given a well-posed problem and a …nite di¤erence
approximation, which is consistent, the stability and consistency is the necessary and su¢ cient
condition of convergence."
De…nition 86 A matrix recursion of the form
x(k+1) = Ax(k) + bk
A 2 Rn
n
, bk 2 Rn , k = 0; 1; : : :
is said to be stable [asymptotically stable] if for any z (0) = x(0) +
z (k+1) = Az (k) + bk
the sequence
z (k)
x(k)
is bounded [ z (k)
(5.19)
x(0) and
(k = 0; 1; : : :)
(5.20)
x(k) ! 0 (k ! 1)].
For our problem the requirements of stability and asymptotic stability is obvious, since the
theoretical solution tends to zero for t ! 1.
Observe that
z (k+1) x(k+1) = A z (k) x(k) = Ak+1 x(0) :
(5.21)
is bounded if and only if Ak is bounded for all k, and z (k)
Hence z (k) x(k)
holds if and only if Ak ! 0.
The following result is of great importance.
x(k) ! 0
Theorem 87 Let A 2 Cn n . Then Ak ! 0 holds if and only if every eigenvalue of A has
modulus strictly less than one; Ak is bounded for every k, if every eigenvalue of A has modulus
at most one and every Jordan block associated with an eigenvalue of modulus one is 1-by-1.
We need the following result as well.
Theorem 88 Let
2
a b
6 b a
6
.
6
6 0 ..
6
4
0
be a n
0
b
.. ..
.
.
... ...
0
b
0
3
7
7
7
0 7
7
b 5
a
(5.22)
n matrix. Then the eigenvalues of the matrix are
k
= a + 2b cos
k
n+1
(k = 1; : : : ; n) :
(5.23)
Numerical methods for partial di¤erential equations
85
In our case, matrix A is such, a = 1 2s and b = s. Hence every eigenvalues of A is real. The
Gershgorin theorem implies that all eigenvalues belong to the interval [1 4s; 1]. Condition
1 1 4s implies s 21 . For the matrix (5.18), the exact eigenvalues are
=1
k
2s + 2s cos
k
N 1
(k = 1; : : : ; N
These are pairwise di¤erent, that is simple eigenvalues. For s
1>
k
=1
= cos
2s 1
|
k
>
N 1
k
cos
N 1
{z
}
1
1
2) :
1
,
2
cos
k
N 1
>0
1:
Consequently the condition of stability is
t
( x)2
s=
which is a bound for the relation of
t and
x( t
1
;
2
(5.24)
( =2) ( x)2 ).
The convergence of the explicit Euler method
Consider the order of the maximum global error
max Ujk
on the mesh!
Since
(5.25)
u (xj ; tk )
ij
u (xj ; tk+1 ) u (xj ; tk )
@u (xj ; tk )
=
+ O ( t)
@t
t
and
@ 2 u (xj ; tk )
u (xj+1 ; tk )
=
@x2
2u (xj ; tk ) + u (xj 1 ; tk )
+ O ( x)2
2
( x)
we have the relation
u (xj ; tk+1 )
u (xj ; tk )
t
=
u (xj+1 ; tk )
2u (xj ; tk ) + u (xj 1 ; tk )
( x)2
+ O ( t) + O ( x)2 :
The global error of the explicit Euler method at the point (xj ; tk ) is ekj = Ujk
Subtracting the previous equation from the equation
Ujk+1
Ujk
t
=
2
( x)
k
Uj+1
2Ujk + Ujk
1
u (xj ; tk ).
Numerical methods for partial di¤erential equations
86
we obtain the equation
ejk
ek+1
j
t
Using the notation s =
ek+1
= sekj
j
=
ekj+1
2
( x)
2ekj + ekj
1
+ O ( t) + O ( x)2 :
t= ( x)2 we can rewrite this as follows
1
2s) ekj + sekj+1 + O ( t)2 + O
+ (1
(j = 2; : : : ; N
t ( x)2
1) :
We can assume that on the bounded region [0; L] [0; T ] the big O members have a common
constant K > 0. We can rewrite the above recursion in the form
Ek+1
z }| 3{
z
2
2
ek+1
2
6 .. 7
6
6 . 7
6
6 k+1 7
6
6 ej
7 = A6
6 . 7
6
4 .. 5
4
ek+1
N 1
Ek
}| 3{
2
ek2
.. 7
6
. 7
6
6
7
ekj 7 + s 6
6
7
.. 5
4
.
ekN
1
ek1
0
..
.
3
7
7
7
7+
7
0 5
ekN
k
where k is a column vector of dimension N 2 the element of which are bounded by K t t + ( x)2 .
In our case ek1 = 0 = ekN , since the solution is known on the boundary. Hence the recursion
takes the form
Ek+1 = AEk + k :
Note that E0 = 0 also holds (t = 0).
The solution of the recursion
xk+1 = Axk + bk
is
x1 = Ax0 + b0 ;
x2 = A (Ax0 + b0 ) + b1 = A2 x0 + Ab0 + b1 ;
x3 = A A2 x0 + Ab0 + b1 + b2 = A3 x0 + A2 b0 + Ab1 + b2 ;
..
.
x k = Ak x 0 +
k 1
X
Ak
1 j
bj :
j=0
The last formula can be proved by induction:
xk+1 = Axk + bk = A
k+1
x0 +
k 1
X
j=0
A
k j
bj + bk = A
k+1
x0 +
k
X
j=0
Ak j b j :
Numerical methods for partial di¤erential equations
In our case
Ek+1 =
87
k
X
Ak
j
j:
j=0
If we have the stability, that is Ak
KA (k
kEk+1 k
Since k t
kKA K t
t + ( x)2 :
KA KT
t + ( x)2 :
T , we have
max kEk k
k
If
1), then
t ! 0 and
x ! 0 hold with condition s
1=2, then the process is convergent.
Theorem 89 The explicit Euler method is convergent to the solution of the 1D heat equation
(5.17) on the bounded region [0; L] [0; T ], if the parameters t, x satisfy t ! 0 and
x ! 0 hold with condition s 1=2.
Demo program of the explicit Euler method: D1Heat_explEuler.m.
Example 90 Solve the problem
u0t = u00xx (0 < x < ; t > 0)
u (0; t) = 0; u ( ; t) = 0 (t > 0)
u (x; 0) = 100 (0 < x < )
on the interval 0
t
5 using the explicit Euler method!
For N = 10 and M = 100, s = 0:4145 (< 1=2) and the plot of the solution
100
80
60
40
20
0
4
3
5
4
2
3
2
1
0
1
0
N = 10, M = 100, s = 0:4145
Numerical methods for partial di¤erential equations
88
This is similar to the solution obtained by the Fourier series, but not very precise.
If we increase N to N = 12, then s = 0:6192 > 1=2 (no stability) and the obtained numerical
solution has the following plot
x 10
11
3
2
1
0
-1
-2
-3
4
3
5
4
2
3
2
1
0
1
0
N = 12, M = 100, s = 0:6192
We can observe a big numerical instability.
Now select N = 20, M = 1000. Then c = 0:1831 and the plot of solution is
100
80
60
40
20
0
4
3
5
4
2
3
2
1
0
1
0
N = 20, M = 1000, s = 0:1831
This gives a better approximation but it is very expensive because of the great number of mesh
points.
Numerical methods for partial di¤erential equations
89
Hyperbolic equations
We solve the 1D wave equation with three di¤erent methods.
Consider the motion of a vibrating string
2
@2u
2@ u
=
c
@t2
@x2
under the boundary conditions
(5.26)
(0 < x < L; t > 0)
u (0; t) = 0;
u (L; t) = 0
(5.27)
(t > 0)
and the initial conditions
u (x; 0) = f (x) ;
u0t (x; 0) = g (x) ;
(5.28)
(0 < x < L) :
Here c > 0 is constant, f (x) denotes the initial form of the string, and g (x) denotes the initial
speed of the string.
D’Alembert’s method
D’Alembert showed that the general solution of the above PDE is
Z
1 x+ct
1
g( )d :
u (x; t) = [f (x + ct) + f (x ct)] +
2
2c x ct
(5.29)
Function f and g are the odd periodic extensions of function f and g de…ned on (0; L).
De…nition 91 Let f (x) be de…ned on the interval (0; p). The even periodic extension of f is
given by
8
< f (x) ; 0 < x < p
f ( x) ;
p<x<0 :
f1 (x) =
(5.30)
:
f (x + 2p) ; otherwise
The odd periodic extension of f is given by
8
< f (x) ; 0 < x < p
f ( x) ;
p<x<0 :
f2 (x) =
:
f (x + 2p) ; otherwise
(5.31)
The even and odd periodic extensions are shown on the next …gure:
y
y
y
f(x)
1
1
f1(x)
1
x
x
x
p
f2(x)
-2p
-p
p
2p
even periodic extension
-2p
-p
p
2p
odd periodic extension
Numerical methods for partial di¤erential equations
90
A simple derivation of the general solution
1
ct)] +
2c
1
u (x; t) = [f (x + ct) + f (x
2
Z
x+ct
g( )d
x ct
yields
f 0 (x
ct)] +
1
[g (x + ct) + g (x
2
c2 00
[f (x + ct) + f 00 (x
2
ct)] +
c 0
[g (x + ct)
2
g 0 (x
ct)] :
1 0
[f (x + ct) + f 0 (x
2
ct)] +
1
[g (x + ct)
2c
g (x
ct)]
u0t =
and
u00tt =
c 0
[f (x + ct)
2
ct)] ;
Similarly we obtain
u0x =
and
i
1 h 00
1
f (x + ct) + f 00 (x ct) + [g 0 (x + ct)
2
2c
It is now easy to see that
u00tt = c2 u00xx :
u00xx =
g 0 (x
ct)] :
So the above function gives a general solution. This clearly satis…es the initial conditions
u (x; 0) = f (x) ;
u0t (x; 0) = g (x) :
(5.32)
The odd periodic extension of f and g imply that u also satisfy the boundary conditions
1
1
u (0; t) = [f (ct) + f ( ct)] +
2
2c
Z
g( )d = 0
ct
and
1
u (L; t) = [f (L + ct) + f (L
2
ct
1
ct)] +
2c
Z
L+ct
g ( ) d = 0:
L ct
This is the solution of D’Alembert’s.
Example 92 Determine the solution of equation
@2u
@2u
=
@t2
@x2
u (0; t) = 0;
u (x; 0) = 0;
(0 < x < 1; t > 0)
u (1; t) = 0
u0t (x; 0) = x;
(t > 0)
(0 < x < 1) :
(5.33)
(5.34)
(5.35)
Numerical methods for partial di¤erential equations
91
In this case f = 0 and its odd periodic extension is also zero. Hence
1
u (x; t) =
2
Z
x+t
1
(G (x + t)
2
g (s) ds =
x t
G (x
t)) ;
where g (x) is the odd periodic extension of x, and G (x) is the corresponding primitive function.
Since g (x) = x ( 1 < x < 1), we have
G (x) =
Z
x
1
2
1
sds = s2
2
1
and
G (x) =
8 1 2
< 2x
:
(x 2 ( 1; 1))
1
;
2
1<x<1
otherwise
G (x + 2) ;
The extended functions g and G are shown on the next …gure:
y
1.0
g(x)
0.8
0.6
0.4
0.2
-3
-2
-1
1
2
3
x
-0.2
-0.4
-0.6
G(x)
-0.8
-1.0
The solution of the problem in the coordinate system (x; t; u) (D1Wave1.m):
0 .2 5
0 .2
0 .1 5
0 .1
0 .0 5
0
-0 .0 5
-0 .1
-0 .1 5
-0 .2
-0 .2 5
1
0 .9
0 .8
0 .7
0 .6
0 .5
0 .4
0 .3
0 .2
0 .1
0
0
0 .5
1
1 .5
2
2 .5
3
3 .5
4
4 .5
5
Numerical methods for partial di¤erential equations
92
The method of separation of variables
Solve the hyperbolic PDE
2
@2u
2@ u
=
c
@t2
@x2
u (0; t) = 0;
(0 < x < L; t > 0)
u (L; t) = 0
(t > 0)
u0t (x; 0) = g (x) ;
u (x; 0) = f (x) ;
(0 < x < L)
(5.36)
(5.37)
(5.38)
using the method of variable separation. Assume that the nonzero solution has the form
u (x; t) = X (x) T (t) :
Substituting into the equations we obtain that
X (x) T 00 (t) = c2 X 00 (x) T (t) ;
which immediately implies the equation
T 00 (t)
X 00 (x)
=
:
c2 T (t)
X (x)
This holds for all x and t, if
X 00 (x)
T 00 (t)
=
=k
c2 T (t)
X (x)
where k = 2 ( > 0) is the so called separation constant. Hence we obtain two second order
ordinary di¤erential equations
X 00
kX = 0;
T 00
kc2 T = 0:
It follows that
X (x) = d1 sin ( x) + d2 cos ( x) :
Substituting into the boundary value conditions we get
6=0
and
z}|{
u (0; t) = X (0) T (t) = d2 = 0
6=0
z}|{
u (L; t) = X (L) T (t) = d1 sin ( L) T (t) = 0 , sin ( L) = 0:
The latter can hold, if = n = nL (n = 1; 2; : : :). For simplicity we can set d1 = 1. Substitute
k = 2 into the second di¤erential equation:
T 00
c
n
L
2
T = 0:
Numerical methods for partial di¤erential equations
93
The solution of this is
Tn (t) = an sin (
n t)
+ bn cos (
(5.39)
n t) ;
where
n
(n = 1; 2; : : :) :
L
Hence a particular solution of the PDE is given by
n
un (x; t) = sin
x (an sin ( n t) + bn cos ( n t)) ; (n = 1; 2; : : :) :
L
Using the principle of superposition we obtain the general solution
n
u (x; t) =
1
X
=c
n
x (an sin (
L
sin
n=1
n t)
+ bn cos (
n t)) :
(5.40)
(5.41)
(5.42)
This must satisfy the initial conditions. Condition
u (x; 0) = f (x) =
1
X
bn sin
n=1
n
x
L
(0 < x < L)
implies that the right hand side is a sine Fourier series. Hence
Z
n
2 L
f (x) sin
x dx; n = 1; 2; : : : :
bn =
L 0
L
(5.43)
(5.44)
We can do similarly with the second initial value condition
u0t
(x; 0) = g (x) =
1
X
n an
sin
n=1
which implies that
2
n an =
L
Z
2
an =
cn
Z
and
L
n
x
L
(0 < x < L) ;
(5.45)
g (x) sin
n
x dx;
L
n = 1; 2; : : : ;
(5.46)
g (x) sin
n
x dx;
L
n = 1; 2; : : :
(5.47)
0
L
0
Example 93 Determine the solution of the PDE
@2u
1 @2u
=
(0 < x < 1; t > 0)
2 @x2
@t2
u (0; t) = 0; u (1; t) = 0 (t > 0)
u (x; 0) = f (x) ;
where
f (x) =
8
<
:
u0t
(x; 0) = 0;
3
x;
10
0
(0 < x < 1) ;
(5.48)
(5.49)
(5.50)
1
3
x
(5.51)
3(1 x)
;
20
1
3
x
1
Numerical methods for partial di¤erential equations
94
The plot of initial f (x) is
y
0.10
0.09
0.08
0.07
0.06
0.05
0.04
0.03
0.02
0.01
0.00
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
x
Since g (x) = 0 and an = 0 for all n, the solution has the form
u (x; t) =
1
X
bn sin
n=1
n
x cos (
L
Using the appropriate formula we obtain
Z 1
bn = 2
f (x) sin (n x) dx
0
Z
Z 1
3 1=3
3
=
x sin (n x) dx +
(1
5 0
10 1=3
n t) :
x) sin (n x) dx:
Using integration by parts we get
Z
1=3
x sin (n x) dx =
0
=
=
Z 1=3
1=3
x cos (n x)
1
+
cos (n x) dx
n
n 0
x=0
cos n3
1
1=3
+ 2 2 [sin (n x)]x=0
3n
n
1
n
1
n
cos
+ 2 2 sin
;
3n
3
n
3
and
Z
1
1=3
(1
x) sin (n x) dx =
(1
Z
x) cos (n x)
n
1
1
x=1=3
1
cos (n x) dx
n 1=3
2
n
1
=
cos
[sin (n x)]1x=1=3
2
2
3n
3
n
1
2
n
n
=
cos
+ 2 2 sin
:
3n
3
n
3
Numerical methods for partial di¤erential equations
95
Summing up these two expressions we obtain
bn =
3
5
1
n
cos
3n
3
+
+
1
n2 2
3
10
n
3
sin
2
n
cos
3n
3
+
1
n2 2
n
3
sin
=
Since c = 1= ,
n
= c nL =
1n
1
9
10
2 n2
sin
n
3
:
= n. Hence the general solution of the example is
u (x; t) =
9
10
2
1
X
sin
n
3
2
n
n=1
(5.52)
sin (n x) cos (nt) :
The solution is shown in the coordinate system (x; t; u) (n = 100, D1Wave2.m):
0 .1
0 .0 5
0
-0 .0 5
-0 .1
1
0 .9
0 .8
0 .7
0 .6
0 .5
10
9
0 .4
8
7
0 .3
6
5
0 .2
4
3
0 .1
2
1
0
0
Explicit di¤erence method
Consider the wave equation
2
@2u
2@ u
=
c
@t2
@x2
u (0; t) = 0;
(0 < x < L; 0 < t < T )
(5.53)
u (L; t) = 0
(5.54)
(t > 0)
u0t (x; 0) = g (x) ;
u (x; 0) = f (x) ;
(5.55)
(0 < x < L) :
Discretize the region as follows
tk = (k
1) t;
1
k
where
t=
M;
T
M
1
;
xj = (j
x=
1) x;
L
N
1
:
1
j
N;
Numerical methods for partial di¤erential equations
96
Let
Ujk = u (xj ; tk ) :
Note that
U1k = u (0; tk ) = 0;
UNk = u (L; tk ) = 0;
Uj1 = u (xj ; 0) = f (xj ) :
We use the central di¤erence quotient to approximate the second derivatives:
k+1
k
k-1
t=0
1
N
j-1 j j+1
x=0
x=L
that is
u00tt (xk ; tj )
Ujk+1
2Ujk + Ujk
2
( t)
1
;
u00xx (xk ; tj )
k
Uj+1
2Ujk + Ujk
2
( x)
1
:
Hence the discretized PDE is
Ujk+1
2Ujk + Ujk
( t)2
Setting s =
c t
x
1
=
c2
k
Uj+1
( x)2
2Ujk + Ujk
1
:
we can rewrite this equation in the form
Ujk+1 = s2 Ujk
1
+ 2
k
2s2 Ujk + s2 Uj+1
Ujk
1
(j = 2; : : : ; N
1) :
Numerical methods for partial di¤erential equations
97
Since U1k = 0 and UNk = 0, the recursion can be written in the form
2
3 2
U2k+1
6 .. 7 6
6 . 7 6
6 k+1 7 6
6 Uj
7=6
6 . 7 6
4 .. 5 4
UNk+11
|
2 2s2 s2
0
0
s2
2 2s2 s2
...
... ...
0
0
.. ..
.
. s2
2
0
0 s
2 2s2
{z
A
32
3
U2k
7 6 .. 7
76 . 7
7
76
7 6 Ujk 7
76 . 7
5 4 .. 5
}
UNk
1
2
U2k
6 ..
6 .
6 k
6 Uj
6 .
4 ..
UNk
1
1
1
1
3
7
7
7
7 ; (5.56)
7
5
which is second order linear matrix recursion.
This means that for the computation of Ujk+1 ’s we need to know the Ujk ’s (k-th time level)
and the Ujk 1 ’s ((k 1)-th time level).
In our case Uj1 = 0 for all j. In order to start with the recursion we also need the values Uj2 . These can be determined using the initial condition u0t (x; 0) = g (x). Consider the
approximations
u(xj ; t2 ) u (xj ; t1 )
+ O ( t) = g (xj ) ;
(5.57)
u0t (xj ; 0) =
t
u (xj ; t2 ) f (xj )
u (xj ; t2 ) u(xj ; t1 )
=
= g (xj ) + O (( t)) ;
t
t
u (xj ; t2 ) = f (xj ) + ( t) g (xj ) + O ( t)2
(5.58)
which give the approximate values Uj2 = f (xj ) + ( t) g (xj ).
Having known the values Uj1 and Uj2 we can compute the above recursion.
The di¤erence schemi is stable, if
and
k
s sin
< 1:
2 (N 1)
This is the famous CFL (Courant-Friedrichs-Lewy) condition. Since 0 < sin 2(Nk
can write this into the su¢ cient CFL condition
s=
c t
x
1:
(5.59)
1)
< 1, we
(5.60)
Example 94 Using the explicit di¤erence method solve the problem
@2u
1 @2u
=
2 @x2
@t2
(0 < x < 1; t > 0)
(5.61)
Numerical methods for partial di¤erential equations
98
u (0; t) = 0;
u (x; 0) = f (x) ;
with
f (x) =
8
<
:
u (1; t) = 0
(5.62)
(t > 0)
u0t (x; 0) = 0;
(0 < x < 1)
3
x;
10
1
3
0
x
(5.63)
(5.64)
3(1 x)
;
20
1
3
x
1
The Matlab program is D1Wave_expl.m.
For N = 100, M = 10000, s = 0:0315, which satis…es the CFL condition. The obtained
approximate solution is shown on the next …gure.
Elliptic equations
We solve the 2D Poisson problem
u00xx + u00yy = f (x; y)
u (x; y) = g (x; y)
((x; y) 2 (0; a)
(5.65)
(0; b) = )
((x; y) 2 @ )
with the …ve-point di¤erence method. Here @
Note that 2D Laplace equation
u = u00xx + u00yy = 0
(5.66)
denotes the boundary of the region
(0
x
a; 0
y
b)
.
(5.67)
with the boundary conditions
u (x; 0) = f1 (x) ;
u (x; b) = f2 (x) ;
(5.68)
u (0; y) = g1 (y) ;
u (a; y) = g2 (y) :
(5.69)
is special case of the Poisson. The PDE describes a heat transfer problem on a rectangular
with temperatures given on its four sides:
Numerical methods for partial di¤erential equations
99
y
u(x,b)=f2(x)
b
u(0,y)=g1(y)
u(a,y)=g2(y)
0
x
a
u(x,0)=f1(x)
Discretize the region as follows
y
y=b
(a,b)
N
j+1
j
j-1
x
y=0
1
M
i-1 i i+1
x=0
x=a
Let
xi = (i
1) x;
1
i
where
x=
M;
a
yj = (j
;
y=
1) y;
1
j
N;
b
:
M 1
N 1
Let Uij = u (xi ; yj ), fij = f (xi ; yj ) and gij = g (xi ; yi ) (wherever g is de…ned). Note that
U1j = u (x1 ; yj ) = g1j ; UM j = u (xM ; yj ) = gM j ;
Ui1 = u (xi ; y1 ) = gi1 ; UiN = u (xi ; yN ) = giN :
In the interior meshpoints (xi ; yj ) we discretize the equation by the central di¤erence schemes:
Ui
Let hx =
x and hy =
h2y Ui
1;j
hy
Ui
hx
1;j
2Uij + Ui+1;j Ui;j
+
( x)2
2Uij + Ui;j+1
= fij :
( y)2
y! Then the equation takes the forms
+ h2x Ui;j
1;j
1
+
2 h2x + h2y Uij + h2x Ui;j+1 + h2y Ui+1;j = h2x h2y fij ;
1
hx
Ui;j
hy
1
2
hx hy
+
hy hx
Uij +
hx
hy
Ui;j+1 +
= hx hy fij
hy
hx
(5.70)
Numerical methods for partial di¤erential equations
100
and
1
where =
following
hx
,
hy
Ui
1;j
+ Ui;j
2
1
+
= hx hy , i = 2; : : : ; M
1
1
Uij + Ui;j+1 + Ui+1;j = fij ;
1, j = 2; : : : ; N
Uij = gij ;
Uij = gij ;
(5.71)
1. The boundary conditions are the
(5.72)
(5.73)
i = 1; M ; j = 1; : : : ; N;
j = 1; N ; i = 1; : : : ; M:
The linear system for the unknowns Uij in the interior points has the size (M 2) (N 2).
Using a clever reindexing of the unknowns we can make an equivalent linear system whose
matrix is a block banded matrix.
We make the reindexing according to the following …gure (natural indexing):
y
MN
N
t=N-2
(i,j)n(s,t)
direction of
indexing
j
2d
1+d
1
t=1
2
d
3
x
1
i
s=1
Then (s; t) = (i
1; j
M
s=M-2=d
1) and the index of point (i; j) is
=d
k = (t
1) (M
2) + s = (j
z }| {
2) (M 2) + i
1:
(5.74)
Hence Uij corresponds to zk . The linear system (5.71) changes at the boundary meshpoints:
only those variables Uij (zk ) remain in the left hand side that are shown on the next …gure,
while the other "variables" - since they are known on the boundary - will be arranged to the
right hand side.
Numerical methods for partial di¤erential equations
101
y
j=N
MN
t=N-2
j
t=1
x
i,j=1
i
i=M
s=1
s=M-2=d
The indices and the coe¢ cients of the new variables ordered to the meshpoint (i; j) and its
neighbors are shown by the next …gure:
(λ) zk+d (i,j+1)
-2(λ+1/λ)
(1/λ) zk-1
(i,j)
(i-1,j)
(λ) zk-d
If we put together the (M
system, then it has the form
2) (N
2
(1/λ) zk+1
zk
2)-by-(M
(i+1,j)
(i,j-1)
2) (N
2) matrix of our reindexed linear
T
I
6 I T
I
6
6
I T
I
6
A=6
.
.
.
.
.
6
.
. ..
6
4
I T
I
I T
where I is the (M
2)
(M
3
7
7
7
7
7;
7
7
5
2) unit matrix and T is the (M
(5.75)
2)
(M
2) tridiagonal
Numerical methods for partial di¤erential equations
102
matrix
2
a b
6 b a b
6
.. .. ..
6
.
.
.
6
T =6
.
.
6
.. .. ...
6
4
b
a b
b
a
3
7
7
7
7
7
7
7
5
(5.76)
with a = 2 + 1 and b = 1= .
In the case M = 5, N = 7 the sparsity structure of matrix A is (laplace5p_matrix.m,
laplace5p_matrix_B.m):
0
2
4
6
8
10
12
14
16
0
5
10
15
nz = 59
Case M = 5, N = 7
There are other types of reindexing that results in other types of coe¢ cient matrices.
The obtained linear system has the general block tridiagonal form
2
C1 B1
6 A2 C2 B2
6
6
A3 C3 B3
6
6
... ...
6
6
4
An
32
...
1
Cn
An
1
Bn
Cn
1
3
X1
X2
X3
..
.
76
76
76
76
76
76
76
5 4 Xn
Xn
1
2
3
F1
F2
F3
..
.
7 6
7 6
7 6
7 6
7=6
7 6
7 6
5 4 Fn
Fn
1
7
7
7
7
7
7
7
5
(5.77)
with Ai , Bi ; Ci 2 Rm m and Xi ; Fi 2 Rm . This can be solved e¢ ciently with the block version
of the tridiagonal Gauss elimination method.
Numerical methods for partial di¤erential equations
103
Example 95 Solve the Poisson equation
u = 5 2 sin ( x) cos (2 y) ;
uj@ = sin ( x) cos (2 y)
= [0; a]
[0; b]
for a = 1 and b = 2!
The exact solution of the PDE is u (x; y) = sin ( x) cos (2 y).
For M = 10 and N = 20, the obtained numerical solution is (D2Poisson2.m):
approximate solution
The error of the approximate solution:
The error function
Numerical methods for partial di¤erential equations
104
Example 96 Solve the problem
u = 0;
= [0; 1] [0; 1]
2x
u (x; 0) = e ;
u (x; 1) = e 2x cos 2; 0 x 1;
u (0; y) = cos (2y) ;
u (1; y) = e 2 cos (2y) ; 0 y 1:
The exact solution is u (x; y) = e 2x cos (2y). Using the program D2Poisson2.m (M = N =
20) we obtain the following approximate solution and its error:
The approximate solution
The error function