Gauss-Siedel Method
Gauss-Seidel Method
An iterative method.
Basic Procedure:
-Algebraically solve each linear equation for xi
-Assume an initial guess solution array
-Solve for each xi and repeat
-Use absolute relative approximate error after
each iteration to check if error is within a prespecified tolerance.
Gauss-Seidel Method
Why?
The Gauss-Seidel Method allows the user to control
round-off error.
Elimination methods such as Gaussian Elimination
and LU Decomposition are prone to prone to roundoff error.
Also: If the physics of the problem are understood, a
close initial guess can be made, decreasing the
number of iterations needed.
Gauss-Seidel Method
Algorithm
A set of n equations and n unknowns:
a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1
a21 x1 + a22 x2 + a23 x3 + ... + a2n xn = b2
.
.
.
.
.
.
an1 x1 + an 2 x2 + an 3 x3 + ... + ann xn = bn
If: the diagonal elements are
non-zero
Rewrite each equation solving
for the corresponding unknown
ex:
First equation, solve for x1
Second equation, solve for x2
Gauss-Seidel Method
Algorithm
Rewriting each equation
x1 =
c1 − a12 x2 − a13 x3 − a1n xn
a11
From Equation 1
c2 − a21 x1 − a23 x3 − a2 n xn
a22
From equation 2
x2 =
xn −1 =
xn =
cn −1 − an −1,1 x1 − an −1, 2 x2 − an −1,n − 2 xn − 2 − an −1,n xn
From equation n-1
an −1,n −1
cn − an1 x1 − an 2 x2 − − an ,n −1 xn −1
ann
From equation n
Gauss-Seidel Method
Algorithm
General Form of each equation
n
cn −1 − an −1, j x j
n
c1 − a1 j x j
x1 =
j =1
j 1
a11
xn −1 =
j =1
j n −1
an −1,n −1
n
c n − a nj x j
n
c2 − a2 j x j
x2 =
j =1
j 2
a 22
xn =
j =1
jn
a nn
Gauss-Seidel Method
Algorithm
General Form for any row ‘i’
n
ci − a ij x j
xi =
j =1
j i
a ii
, i = 1,2, , n.
How or where can this equation be used?
Gauss-Seidel Method
Solve for the unknowns
Assume an initial guess for [X]
x1
x
2
xn -1
xn
Use rewritten equations to
solve for each value of xi.
Important: Remember to
use the most recent value
of xi. Which means to apply
values calculated to the
calculations remaining in
the current iteration.
Gauss-Seidel Method
Calculate the Absolute Relative Approximate Error
a i =
−x
new
i
old
i
x
new
i
x
100
So when has the answer been found?
The iterations are stopped when the absolute relative
approximate error is less than a prespecified tolerance for all
unknowns.
Gauss-Seidel Method: Example 1
The upward velocity of a rocket
is given at three different times
Table 1 Velocity vs. Time data.
Time, t (s )
Velocity v (m/s )
5
106.8
8
177.2
12
279.2
The velocity data is approximated by a polynomial as:
v(t ) = a1t 2 + a2t + a3 , 5 t 12.
Gauss-Seidel Method: Example 1
Using a Matrix template of the form
t12
2
t 2
t32
t1
t2
t3
1 a1 v1
1 a2 = v2
1 a3 v3
25 5 1 a1 106.8
64 8 1 a = 177.2
The system of equations becomes
2
144 12 1 a3 279.2
Initial Guess: Assume an initial guess of
a1 1
a = 2
2
a3 5
Gauss-Seidel Method:
Example 1
Rewriting each equation
25 5 1 a1 106.8
64 8 1 a = 177.2
2
144 12 1 a3 279.2
106.8 − 5a 2 − a3
a1 =
25
177.2 − 64a1 − a3
a2 =
8
279.2 − 144a1 − 12a 2
a3 =
1
Gauss-Seidel Method:
Example 1
Applying the initial guess and solving for ai
a1 1
a = 2
2
a3 5
Initial Guess
a1 =
106.8 − 5(2) − (5)
= 3.6720
25
177.2 − 64(3.6720 ) − (5)
a2 =
= −7.8510
8
279.2 − 144(3.6720 ) − 12(− 7.8510 )
a3 =
= −155.36
1
When solving for a2, how many of the initial guess values were used?
Gauss-Seidel Method:
Example 1
Finding the absolute relative approximate error
xinew − xiold
a i =
100
new
xi
a 1 =
3.6720 − 1.0000
x100 = 72.76%
3.6720
− 7.8510 − 2.0000
a 2 =
x100 = 125.47%
− 7.8510
a 3 =
− 155.36 − 5.0000
x100 = 103.22%
− 155.36
At the end of the first iteration
a1 3.6720
a = − 7.8510
2
a3 − 155.36
The maximum absolute
relative approximate error is
125.47%
Gauss-Seidel Method:
Example 1
Using
a1 3.6720
a = − 7.8510
2
a3 − 155.36
Iteration #2
the values of ai are found:
106.8 − 5(− 7.8510 ) − 155.36
a1 =
= 12.056
25
from iteration #1
a2 =
177.2 − 64(12.056 ) − 155.36
= −54.882
8
279.2 − 144(12.056 ) − 12(− 54.882 )
a3 =
= −798.34
1
Gauss-Seidel Method:
Example 1
Finding the absolute relative approximate error
12.056 − 3.6720
At the end of the second iteration
a 1 =
x100 = 69.543%
12.056
a 12.056
1
a 2 =
a 3 =
− 54.882 − (− 7.8510 )
x100 = 85.695%
− 54.882
− 798.34 − (− 155.36 )
x100 = 80.540%
− 798.34
a = − 54.882
2
a3 − 798.54
The maximum absolute
relative approximate error is
85.695%
Gauss-Seidel Method: Example 1
Repeating more iterations, the following values are obtained
Iteration
1
2
3
4
5
6
a1
a 1 %
a2
a 2 %
a3
a 3 %
3.6720
12.056
47.182
193.33
800.53
3322.6
72.767
69.543
74.447
75.595
75.850
75.906
−7.8510
−54.882
−255.51
−1093.4
−4577.2
−19049
125.47
85.695
78.521
76.632
76.112
75.972
−155.36
−798.34
−3448.9
−14440
−60072
−249580
103.22
80.540
76.852
76.116
75.963
75.931
Notice – The relative errors are not decreasing at any significant rate
a1 0.29048
Also, the solution is not converging to the true solution of
a
=
19
.
690
2
a 3 1.0857
Gauss-Seidel Method: Pitfall
What went wrong?
Even though done correctly, the answer is not converging to the
correct answer
This example illustrates a pitfall of the Gauss-Siedel method: not all
systems of equations will converge.
Is there a fix?
One class of system of equations always converges: One with a diagonally
dominant coefficient matrix.
Diagonally dominant: [A] in [A] [X] = [C] is diagonally dominant if:
n
n
aii aij
j =1
j i
for all ‘i’
and
aii aij for at least one ‘i’
j =1
j i
Gauss-Seidel Method: Pitfall
Diagonally dominant: The coefficient on the diagonal must be at least
equal to the sum of the other coefficients in that row and at least one row
with a diagonal coefficient greater than the sum of the other coefficients
in that row.
Which coefficient matrix is diagonally dominant?
2 5.81 34
A = 45 43 1
123 16
1
124 34 56
[B] = 23 53 5
96 34 129
Most physical systems do result in simultaneous linear equations that
have diagonally dominant coefficient matrices.
Gauss-Seidel Method:
Example 2
Given the system of equations
The coefficient matrix is:
12 x1 + 3 x2 - 5 x3 = 1
12 3 − 5
A = 1 5 3
3 7 13
x1 + 5 x2 + 3 x3 = 28
3 x1 + 7 x2 + 13 x3 = 76
With an initial guess of
x1 1
x = 0
2
x3 1
Will the solution converge using the
Gauss-Siedel method?
Gauss-Seidel Method:
Example 2
Checking if the coefficient matrix is diagonally dominant
12 3 − 5
A = 1 5 3
3 7 13
a11 = 12 = 12 a12 + a13 = 3 + − 5 = 8
a22 = 5 = 5 a21 + a23 = 1 + 3 = 4
a33 = 13 = 13 a31 + a32 = 3 + 7 = 10
The inequalities are all true and at least one row is strictly greater than:
Therefore: The solution should converge using the Gauss-Siedel Method
Gauss-Seidel Method:
Example 2
Rewriting each equation
With an initial guess of
12 3 − 5 a1 1
1 5 3 a = 28
2
3 7 13 a3 76
x1 1
x = 0
2
x3 1
1 − 3 x 2 + 5 x3
x1 =
12
28 − x1 − 3x3
x2 =
5
76 − 3x1 − 7 x 2
x3 =
13
x1 =
1 − 3(0) + 5(1)
= 0.50000
12
x2 =
28 − (0.5) − 3(1)
= 4.9000
5
76 − 3(0.50000 ) − 7(4.9000 )
x3 =
= 3.0923
13
Gauss-Seidel Method: Example 2
The absolute relative approximate error
0.50000 − 1.0000
a 1 =
100 = 100.00%
0.50000
4.9000 − 0
a 2 =
100 = 100.00%
4.9000
3.0923 − 1.0000
a 3 =
100 = 67.662%
3.0923
The maximum absolute relative error after the first iteration is 100%
Gauss-Seidel Method:
Example 2
After Iteration #1
x1 0.5000
x = 4.9000
2
x3 3.0923
Substituting the x values into the
equations
1 − 3(4.9000 ) + 5(3.0923)
x1 =
= 0.14679
12
x2 =
28 − (0.14679 ) − 3(3.0923)
= 3.7153
5
x3 =
76 − 3(0.14679 ) − 7(4.900 )
= 3.8118
13
After Iteration #2
x1 0.14679
x = 3.7153
2
x3 3.8118
Gauss-Seidel Method:
Example 2
Iteration #2 absolute relative approximate error
a 1 =
0.14679 − 0.50000
100 = 240.61%
0.14679
3.7153 − 4.9000
a 2 =
100 = 31.889%
3.7153
3.8118 − 3.0923
a 3 =
100 = 18.874%
3.8118
The maximum absolute relative error after the first iteration is 240.61%
This is much larger than the maximum absolute relative error obtained in
iteration #1. Is this a problem?
Gauss-Seidel Method: Example 2
Repeating more iterations, the following values are obtained
Iteration
a1
a 1 %
a2
a 2 %
a3
a 3 %
1
2
3
4
5
6
0.50000
0.14679
0.74275
0.94675
0.99177
0.99919
100.00
240.61
80.236
21.546
4.5391
0.74307
4.9000
3.7153
3.1644
3.0281
3.0034
3.0001
100.00
31.889
17.408
4.4996
0.82499
0.10856
3.0923
3.8118
3.9708
3.9971
4.0001
4.0001
67.662
18.876
4.0042
0.65772
0.074383
0.00101
x1 0.99919
x1 1
The solution obtained x = 3.0001 is close to the exact solution of x = 3 .
2
2
x3 4.0001
x3 4
Gauss-Seidel Method: Example 3
Given the system of equations
3 x1 + 7 x2 + 13 x3 = 76
Rewriting the equations
x1 + 5 x2 + 3 x3 = 28
76 − 7 x2 − 13 x3
x1 =
3
12 x1 + 3x2 − 5 x3 = 1
With an initial guess of
x1 1
x = 0
2
x3 1
28 − x1 − 3x3
x2 =
5
1 − 12 x1 − 3x 2
x3 =
−5
Gauss-Seidel Method:
Example 3
Conducting six iterations, the following values are obtained
Iteration
a1
a 1 %
A2
a 2 %
a3
a 3 %
1
2
3
4
5
6
21.000
−196.15
−1995.0
−20149
2.0364×105
−2.0579×105
95.238
110.71
109.83
109.90
109.89
109.89
0.80000
14.421
−116.02
1204.6
−12140
1.2272×105
100.00
94.453
112.43
109.63
109.92
109.89
50.680
−462.30
4718.1
−47636
4.8144×105
−4.8653×106
98.027
110.96
109.80
109.90
109.89
109.89
The values are not converging.
Does this mean that the Gauss-Seidel method cannot be used?
Gauss-Seidel Method
The Gauss-Seidel Method can still be used
3 7 13
The coefficient matrix is not
A = 1 5 3
diagonally dominant
12 3 − 5
12 3 − 5
But this is the same set of
equations used in example #2,
A = 1 5 3
which did converge.
3 7 13
If a system of linear equations is not diagonally dominant, check to see if
rearranging the equations can form a diagonally dominant matrix.
Gauss-Seidel Method
Not every system of equations can be rearranged to have a
diagonally dominant coefficient matrix.
Observe the set of equations
x1 + x 2 + x3 = 3
2 x1 + 3 x 2 + 4 x3 = 9
x1 + 7 x 2 + x3 = 9
Which equation(s) prevents this set of equation from having a
diagonally dominant coefficient matrix?
Gauss-Seidel Method
Summary
-Advantages of the Gauss-Seidel Method
-Algorithm for the Gauss-Seidel Method
-Pitfalls of the Gauss-Seidel Method
THE END