Mechanical Engineering Department (power branch)
3rd Year
Linear system of algebraic equations
Course instructor
Dr. Mohamed Emam
2020
1
Introduction to Numerical Methods
Mathematical modeling
Mathematical modeling can be broadly defined as a formulation or equation that expresses the essential features of a physical
system or process in mathematical terms.
Physical system
Analysis
(mathematical modeling)
Equations
Types of equations
Mathematical equations can be generally classified into two main types:
1- Algebraic equations
2- Differential equations
2
1- Algebraic equations
o Linear and nonlinear systems of algebraic equations are associated with many problems in engineering and
science.
➢ General form of linear system of algebraic equations
where the a’s are constant coefficients,
the b’s are constants, and n is the
number of equations.
➢ Methods to solve linear system of algebraic equations
Gauss elimination
Jacobi Method
Indirect Methods
Iterative methods
Gauss-Seidel
SOR
&
Direct Methods
Gauss Jordan
Non iterative
methods
LU factorization
3
2- Differential equations
o A differential equation is a mathematical equation that consists of derivatives.
o Mathematical formulation of most of the physical and engineering problems lead to differential equations, so,
it is important for engineers and scientists to know how to set up differential equations and solve them.
For example:
or
➢ Differential equations are of two main types
(a) Ordinary differential equations (ODE)
Differential equations
(b) Partial differential equations (PDE)
4
(a) Ordinary differential equations (ODE)
o An Ordinary differential equation is that in which all the derivatives are with respect to a single independent
variable.
For example:
➢ Ordinary differential equation are classified in terms of order and degree
o Order of an ODE is the same as the highest order derivative, while degree of an ODE is the degree of the
highest order derivative.
1
For example:
5
➢ Numerical solution of Ordinary differential equations
o There are several methods to solve ODE such as:
1st order ODE
Higher order ODE
Consider nth order ODE, n > 1
Single equation
System of equations
• This kind of equations is firstly converted to
system of n equations of 1st order ODE.
1- Euler’s method
3- Heun’s method
2- Midpoint method
4- Rung-Kutta method
• Then we can use one of the previous four
methods to solve it.
6
(b) Partial differential equations (PDE)
o An equation involving partial derivatives of an unknown function of two or more independent variables is
called a partial differential equation, or PDE.
or
For example:
o The order of a PDE is that of the highest-order partial derivative appearing in the equation.
2nd order
3rd order
o A PDE is said to be linear if it is linear in the unknown function and all its derivatives, with coefficients
depending only on the independent variables.
Linear
Non-linear
Because of their widespread application in engineering, our treatment of PDEs will focus on linear, 2nd order
equations.
7
1- Algebraic equations
❑ Iterative methods to solve linear systems of algebraic equations (Indirect methods)
(a) The Jacobi method
o The first iterative technique is called the Jacobi method, after Carl Gustav Jacob Jacobi (1804-1851).
o To solve the following linear system of equations using Jacobi method, solve the 1st equation for x1, the 2nd equation
for x2 and so on, as follows.
1
𝑥1 =
(𝑏 − 𝑎12 𝑥2 − 𝑎13 𝑥3 − ⋯ − 𝑎1𝑛 𝑥𝑛)
𝑎11 1
1
𝑥2 =
(𝑏 − 𝑎21 𝑥1 − 𝑎23 𝑥3 − ⋯ − 𝑎2𝑛 𝑥𝑛)
𝑎22 2
.
.
.
1
𝑥𝑛 =
(𝑏𝑛 − 𝑎𝑛1 𝑥1 − 𝑎𝑛3 𝑥3 − ⋯ − 𝑎𝑛𝑛 𝑥𝑛 − 1)
𝑎𝑛𝑛
o Then make an initial approximation of the solution, (x1, x2, x3, …., xn), and substituting these values of xi into the righthand side of the rewritten equations to obtain the first approximation.
o After this procedure has been completed, one iteration has been performed.
o In the same way, the 2nd approximation is formed by substituting the 1st approximation’s x-values into the right-hand side
of the rewritten equations.
o By repeating iterations, you will form a sequence of approximations that often converges to the actual solution.
8
Example (1)
Use the Jacobi method to approximate the solution of the following system of linear equations. Continue
the iteration until two successive approximations are identical when rounded to three significant digits
𝟓𝒙𝟏 − 𝟐𝒙𝟐 + 𝟑𝒙𝟑 = −𝟏
−𝟑𝒙𝟏 + 𝟗𝒙𝟐 + 𝒙𝟑 = 𝟐
𝟐𝒙𝟏 − 𝒙𝟐 − 𝟕𝒙𝟑 = 𝟑
Solution
o To begin, write the system in the form
1 2
3
𝑥1 = − + 𝑥2 + 𝑥3
5 5
5
5𝑥1 − 2𝑥2 + 3𝑥3 = −1
2 3
1
𝑥2 =
+ 𝑥1 − 𝑥3
−3𝑥1 + 9𝑥2 + 𝑥3 = 2
9 9
9
3 2
1
2𝑥1 − 𝑥2 − 7𝑥3 = 3
𝑥3 = − + 𝑥1 − 𝑥2
7 7
7
o Because we do not know the actual solution, choose 𝑥1 = 0, 𝑥2 = 0, 𝑥3 = 0 (Initial approximation) as a convenient
initial approximation. So, the 1st approximation is:
1 2
3
𝑥1 = − + 0 + 0 = −0.2
5 5
5
2 3
1
𝑥2 =
+ 0 − 0 = 0.222
9 9
9
3 2
1
𝑥3 = − + 0 − 0 = −0.429
7 7
7
initial
Solution
n
0
1
2
3
4
5
6
7
x1
0
-0.2
0.146
0.192
0.181
0.185
0.186
0.186
x2
0
0.222
0.203
0.328
0.332
0.329
0.331
0.331
x3
0
-0.429 -0.517 -0.416 -0.421 -0.424 -0.423 -0.423
o Therefore, the solution is:
𝑥1 = 0.186, 𝑥2 = 0.331, 𝑥3 = −0.423
9
(b) The Gauss-Seidel method
o Gauss-Seidel method is a modification of Jacobi method after Carl Friedrich (1777-1855) and Philipp L. Seidel
(1821-1896).
o This modification is no more difficult to use than the Jacobi method, and it often requires fewer iterations to produce
the same degree of accuracy.
1
𝑥1 =
(𝑏 − 𝑎12 𝑥2 − 𝑎13 𝑥3 − ⋯ − 𝑎1𝑛 𝑥𝑛)
𝑎11 1
1
𝑥2 =
(𝑏 − 𝑎21 𝑥1 − 𝑎23 𝑥3 − ⋯ − 𝑎2𝑛 𝑥𝑛)
𝑎22 2
.
.
.
𝑥𝑛 =
1
(𝑏𝑛 − 𝑎𝑛1 𝑥1 − 𝑎𝑛3 𝑥3 − ⋯ − 𝑎𝑛𝑛 𝑥𝑛 − 1)
𝑎𝑛𝑛
o With the Jacobi method, the values of xi obtained in the nth approximation remain unchanged until the entire (n+1)th
approximation has been calculated.
o With the Gauss-Seidel method, on the other hand, you use the new values of each xi as soon as they are known.
o That is, once you have determined x1 from the first equation, its value is then used in the second equation to obtain the new
x2.
o Similarly, the new x1 and x2 are used in the third equation to obtain the new x3 and so on.
10
Example (2)
Use the Gauss seidel iteration method to approximate the solution of the following system of linear
equations. Continue the iteration until two successive approximations are identical when rounded to
three significant digits
𝟓𝒙𝟏 − 𝟐𝒙𝟐 + 𝟑𝒙𝟑 = −𝟏
−𝟑𝒙𝟏 + 𝟗𝒙𝟐 + 𝒙𝟑 = 𝟐
Solution
𝟐𝒙𝟏 − 𝒙𝟐 − 𝟕𝒙𝟑 = 𝟑
o To begin, write the system in the form
1 2
3
𝑥1 = − + 𝑥2 − 𝑥3
5 5
5
5𝑥1 − 2𝑥2 + 3𝑥3 = −1
2 3
1
𝑥2 =
+ 𝑥1 − 𝑥3
−3𝑥1 + 9𝑥2 + 𝑥3 = 2
9 9
9
3 2
1
2𝑥1 − 𝑥2 − 7𝑥3 = 3
𝑥3 = − + 𝑥1 − 𝑥2
7 7
7
o Because we do not know the actual solution, choose 𝑥1 = 0, 𝑥2 = 0, 𝑥3 = 0 (Initial approximation) as a convenient
initial approximation. So, the 1st approximation is:
Solution
initial
1 2
3
𝑥1 = − + 0 − 0 = −0.2
5 5
5
2
3
1
𝑥2 = 9 + 9 −0.2 − 9 0 = 0.156
n
0
1
2
3
4
5
x1
0
-0.2
0.167
0.191
0.186
0.186
3 2
1
𝑥3 = − + −0.2 − 0.156 = −0.429
7 7
7
x2
0
0.156
0.334
0.333
0.331
0.331
x3
0
-0.508 -0.429 -0.422 -0.423 -0.423
o Therefore, the solution is:
𝑥1 = 0.186, 𝑥2 = 0.331, 𝑥3 = −0.423
11
Note,
o For the previous iterative methods, we can say that they are converged when repeated iterations succeed in
producing an approximation that is correct to three significant digits.
o Also, this can be achieved (converged) by reaching an acceptable error value (its value is defined based on the
engineering problem) which is calculated using the following equation:
𝒆𝒓𝒓𝒐𝒓 =
𝒙𝒊,𝒏𝒆𝒘 − 𝒙𝒊,𝒐𝒍𝒅
𝒙𝒊,𝒐𝒍𝒅
≤ specified error
However, it is possible to apply the Jacobi method or Gauss-Seidel method to a system of linear equations and obtain a
divergent sequence of approximations and in such cases, it is said that the method diverges.
12
(c) The SOR method
o This approach is also called successive or simultaneous overrelaxation, or SOR method.
o This is another modification for the Jacobi method, to reduce the number of iterations required to produce the same degree
of accuracy and to make a nonconvergent system converge or to hasten convergence by dampening out oscillations.
o After each new value of xi is computed such as previous methods, that value is modified by a weighted average of the
results of the previous and the present iterations:
where
𝒙𝒊, 𝒏𝒆𝒘 = 𝟏 − 𝝎 𝒙𝒊, 𝒐𝒍𝒅 + 𝝎𝒙𝒊, 𝒏𝒆𝒘
𝝎 is a weighting factor that is assigned a value
between 0 and 2.
1
(𝑏 − 𝑎12 𝑥2 − ⋯ − 𝑎1𝑛 𝑥𝑛)
𝑎11 1
1
𝑥2 =
(𝑏 − 𝑎21 𝑥1 − ⋯ − 𝑎2𝑛 𝑥𝑛)
.
𝑎22 2
𝑥1 =
.
.
𝑥𝑛 =
1
(𝑏𝑛 − 𝑎𝑛1 𝑥1 − ⋯ − 𝑎𝑛𝑛 𝑥𝑛 − 1)
𝑎𝑛𝑛
1
𝑥1 = 1 − 𝜔 𝑥1 + 𝜔(
𝑏 − 𝑎12 𝑥2 − ⋯ − 𝑎1𝑛 𝑥𝑛 )
𝑎11 1
1
𝑥2 = 1 − 𝜔 𝑥2 + 𝜔(
𝑏 − 𝑎21 𝑥1 − ⋯ − 𝑎2𝑛 𝑥𝑛 )
.
𝑎22 2
.
.
𝑥𝑛 = 1 − 𝜔 𝑥𝑛 + 𝜔(
1
𝑏 − 𝑎𝑛1 𝑥1 − ⋯ − 𝑎𝑛𝑛 𝑥𝑛 − 1 )
𝑎𝑛𝑛 𝑛
o Then follow the same steps of Gauss-Seidel method to complete your solution.
Note,
o 𝝎 should be between (0 to 1) or (1 to 2) but not equal 0 or 1.
o If 𝝎 is set at a value between 0 and 1, the result is a weighted average of the present and the previous results.
o On the other hand, for values of 𝝎 from 1 to 2, extra weight is placed on the present value.
13
Example (3)
Use the SOR method to approximate the solution of the following system of linear equations. Continue
the iteration until two successive approximations are identical when rounded to three significant digits
Solution
o To begin, write the system in the form
𝟒𝒙𝟏 − 𝟑𝒙𝟐 = 𝟐𝟒
𝟑𝒙𝟏 + 𝟒𝒙𝟐 + 𝒙𝟑 = 𝟑𝟎
− 𝒙𝟐 − 𝟒𝒙𝟑 = −𝟐𝟒
3
3
𝑥1 = 1 − 𝜔 𝑥1 + 𝜔(6 + 𝑥2)
𝑥1 = 6 + 𝑥 2
4
4𝑥1 − 3𝑥2
= 24
4
3
1
3
1
𝑥2 = 1 − 𝜔 𝑥2 + 𝜔(7.5 − 𝑥1 − 𝑥3)
𝑥2 = 7.5 − 𝑥1 − 𝑥3
3𝑥1 + 4𝑥2 + 𝑥3 = 30
4
4
4
4
1
1
− 𝑥2 − 4𝑥3 = −24
𝑥3 = 6 − 𝑥2
𝑥3 = 1 − 𝜔 𝑥3 + 𝜔(6 − 𝑥2)
4
4
o Because we do not know the actual solution, choose 𝑥1 = 1, 𝑥2 = 1, 𝑥3 = 1 (Initial approximation) as a convenient
initial approximation and 𝜔 equals 1.25. So, the 1st approximation is:
initial
𝑥1 = −0.25 1 + (1.25)(6 + 3/4 (1)) = 8.1875
3
𝑥2 = −0.25 1 + (1.25)(7.5 − 8.1875 − 1/4 (1)) = 1.1367
4
1
𝑥3 = −0.25 1 + 1.25 6 − 1.1367 = 6.894
4
n
0
1
x1
0
8.1875
x2
0
1.1367
x3
0
6.894
2
3
4
….
14
3
1.1367 = 6.518
4
3
1
𝑥2 = −0.25 1.1367 + 1.25 7.5 − 6.518 − 6.894
4
4
1
𝑥3 = −0.25 6.894 + 1.25 6 − 0.8258 = 5.518
4
𝑥1 = −0.25 8.1875 + 1.25
6+
= 0.8258
n
0
1
2
x1
0
8.1875
6.518
x2
0
1.1367
0.8258
x3
0
6.894
5.518
3
4
5
15