Gauss-Siedel Method
Major: All Engineering Majors
Authors: Autar Kaw
(Modified by P. Goel for IDC103)
http://numericalmethods.eng.usf.edu
Transforming Numerical Methods Education for STEM
Undergraduates
3/25/2010
http://numericalmethods.eng.usf.edu
1
Gauss-Seidel Method
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
An iterative method.
Basic Procedure :
-Algebraically solve each linear equation for xi
-Assume an initial guess solution array
-Solve for each xi and repeat
-Use absolute relative approximate error after each iteration
to check if error is within a pre-specified tolerance.
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Why?
Obviously it isn’t possible to obtain better than machine precision for
a solution of SLEs (using Gaussian Elimination and LU
Decomposition). Actually, the situation is worse for large systems: it
isn’t possible to get close to machine precision in direct methods.
The Gauss-Seidel Method allows the user to control round-off error.
In fact, iterative methods can be used to improve the solution
obtained by direct methods.
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Algorithm
A set of n equations and n unknowns:
a11 x1 + a12 x2 + a13 x3 + ... + a1n xn = b1
a21 x1 + a22 x2 + a23 x3 + ... + a2n xn = b2
.
.
.
.
.
.
an1 x1 + an 2 x2 + an 3 x3 + ... + ann xn = bn
If: the diagonal elements are
non-zero
Rewrite each equation solving
for the corresponding unknown
ex:
First equation, solve for x1
Second equation, solve for x2
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Algorithm
Rewriting each equation
x1 =
c1 - a12 x 2 - a13 x3 KK - a1n x n
a11
From Equation 1
c2 - a21 x1 - a23 x3 KK - a2 n xn
a22
From equation 2
x2 =
M
xn -1 =
xn =
M
M
cn -1 - an -1,1 x1 - an -1, 2 x2 KK - an -1,n -2 xn -2 - an -1, n xn
an -1,n -1
cn - an1 x1 - an 2 x2 - KK - an , n -1 xn -1
ann
From equation n-1
From equation n
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Algorithm
General Form of each equation
n
c1 - å a1 j x j
x1 =
j =1
j ¹1
a11
cn -1 xn -1 =
a 22
j =1
j ¹ n -1
n -1, j
xj
an -1,n -1
c n - å a nj x j
c2 - å a2 j x j
x2 =
åa
n
n
j =1
j¹2
n
xn =
j =1
j¹n
a nn
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Algorithm
General Form for any row ‘i’
n
ci - å aij x j
xi =
j =1
j ¹i
aii
, i = 1,2,K , n.
How or where can this equation be used?
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Solve for the unknowns
Assume an initial guess for [X]
é x1 ù
êx ú
ê 2ú
ê M ú
ê ú
ê xn -1 ú
êë xn úû
Use rewritten equati ons to solve for
each value of x i.
Important: Remember to use the
most recent value of xi. Which
means to apply values calculated to
the calculations remaining in the
current iteration.
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Calculate the Absolute Relative Approximate Error
Îa i =
-x
new
i
old
i
x
new
i
x
´ 100
So when has the answer been f ound?
The iterations are stopped when the absolute relative
approximate error is less than a prespecified tolerance for all
unknowns.
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 1
The system of equati ons
Initial Guess: Assume an initial guess of
é 25 5 1ù é a1 ù é106 .8 ù
ê 64 8 1ú êa ú = ê177 .2 ú
ê
ú ê 2ú ê
ú
êë144 12 1úû êë a3 úû êë 279 .2 úû
é a1 ù é1 ù
êa ú = ê2 ú
ê 2ú ê ú
êë a3 úû êë5úû
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 1
Rewriting each equation
é 25 5 1ù é a1 ù é106 .8 ù
ê 64 8 1ú ê a ú = ê177 .2 ú
ê
ú ê 2ú ê
ú
êë144 12 1úû êë a3 úû êë 279 .2 úû
106.8 - 5a 2 - a 3
a1 =
25
177.2 - 64a1 - a 3
a2 =
8
279.2 - 144a1 - 12a 2
a3 =
1
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 1
Applying the initial guess and solving for ai
é a1 ù é1ù
êa ú = ê2ú
ê 2ú ê ú
êë a3 úû êë5úû
Initial Guess
a1 =
106.8 - 5(2) - (5)
= 3.6720
25
177.2 - 64(3.6720 ) - (5)
a2 =
= -7.8510
8
a3 =
279.2 - 144(3.6720 ) - 12(- 7.8510 )
= -155.36
1
When solving for a2, how many of the initial guess values were used?
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 1
Finding the absolute relative approximate error
Îa i
xinew - xiold
=
´100
new
xi
Îa 1 =
Îa
2
3.6720 - 1.0000
x100 = 72.76%
3.6720
- 7.8510 - 2.0000
=
x100 = 125.47%
- 7.8510
Îa 3 =
At the end of the first iteration
é a1 ù é 3.6720 ù
ê a ú = ê - 7.8510 ú
ê 2ú ê
ú
êë a 3 úû êë - 155 .36 úû
The maximum absolute
relative approximate error is
125.47%
- 155.36 - 5.0000
x100 = 103.22%
- 155.36
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 1
Using
é a1 ù é 3.6720 ù
êa ú = ê- 7.8510ú
ê 2ú ê
ú
êë a 3 úû êë - 155.36 úû
Iteration #2
the values of ai are found:
a1 =
106.8 - 5(- 7.8510 ) - 155.36
= 12.056
25
a2 =
177.2 - 64(12.056 ) - 155.36
= -54.882
8
from iteration #1
a3 =
279.2 - 144(12.056 ) - 12(- 54.882 )
= -798.34
1
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 1
Finding the absolute relative approximate error
12.056 - 3.6720
At the end of the second iteration
Îa 1 =
x100 = 69.543%
12.056
é a ù é 12.056 ù
1
Îa 2 =
Îa 3 =
- 54.882 - (- 7.8510)
x100 = 85.695%
- 54.882
- 798.34 - (- 155.36 )
x100 = 80.540%
- 798.34
êa ú = ê- 54.882ú
ê 2ú ê
ú
êë a3 úû êë- 798.54úû
The maximum absolute
relative approximate error is
85.695%
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 1
Repeating more iterations, the following values are obtained
Iteration
1
2
3
4
5
6
a1
3.6720
12.056
47.182
193.33
800.53
3322.6
Îa 1 %
72.767
69.543
74.447
75.595
75.850
75.906
a2
Îa 2 %
a3
−7.8510
−54.882
−255.51
−1093.4
−4577.2
−19049
125.47
85.695
78.521
76.632
76.112
75.972
−155.36
−798.34
−3448.9
−14440
−60072
−249580
Îa 3 %
103.22
80.540
76.852
76.116
75.963
75.931
Notice – The relative errors are not dec reasing at any significant rate
é a 1 ù é0.29048ù
Also, the solution is not converging to the true s olution of ê ú ê
ú
a
=
19
.
690
ê 2ú ê
ú
êëa 3 úû êë 1.0857 úû
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Pitfall
What went wrong?
Even though done correctly, the answer is not converging to the
correct answer
This example illustrates a pitfall of the Gaus s-Siedel method: not all
systems of equations will converge.
Is there a fix?
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Pitfall
GAUSS-SEIDEL CONVERGENCE THEOREM: If A is
diagonally dominant, then the Gauss-Seidel method
converges for any starting vector x. A sufficient, but
not necessary condition.
Diagonally dominant: [A] in [A] [X] = [C] is diagonally dominant if:
n
n
aii ³ å aij
j =1
j ¹i
for all ‘i’
and
aii > å aij
for at least one ‘i’
j =1
j ¹i
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Pitfall
Diagonally dominant: The coefficient on the diagonal must be at least
equal to the sum of the other c oefficients in that row and at least one row
with a diagonal coefficient greater than the sum of the other coefficients
in that row.
Which coefficient matrix is diagonally dominant?
é 2 5.81 34 ù
[A ] = êê 45 43 1 úú
êë123 16
1 úû
é124 34 56 ù
[ B] = êê 23 53 5 úú
êë 96 34 129 úû
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 2
Given the system of equations
12 x1 + 3 x2- 5 x3 = 1
x1 + 5 x2 + 3x3 = 28
3x1 + 7 x2 + 13x3 = 76
With an initial guess of
é x1 ù é1ù
ê x ú = ê0 ú
ê 2ú ê ú
êë x3 úû êë1úû
The coefficient matrix is:
é12 3 - 5ù
[A] = êê 1 5 3 úú
êë 3 7 13 úû
Will the solution converge using the
Gauss-Siedel method?
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 2
Checking if the coefficient matrix is diagonally dominant
é12 3 - 5ù
[A] = êê 1 5 3 úú
êë 3 7 13 úû
a11 = 12 = 12 ³ a12 + a13 = 3 + - 5 = 8
a 22 = 5 = 5 ³ a 21 + a 23 = 1 + 3 = 4
a33 = 13 = 13 ³ a31 + a32 = 3 + 7 = 10
The inequalities are all true and at least one row is strictly greater than:
Therefore: The solution should converge using the Gauss-Siedel Method
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 2
Rewriting each equation
With an initial guess of
é12 3 - 5ù é a1 ù é 1 ù
ê 1 5 3 ú ê a ú = ê 28 ú
ê
ú ê 2ú ê ú
êë 3 7 13 úû êë a3 úû êë76 úû
1 - 3x 2 + 5 x3
x1 =
12
28 - x1 - 3x3
x2 =
5
76 - 3x1 - 7 x 2
x3 =
13
é x1 ù é1 ù
ê x ú = ê0 ú
ê 2ú ê ú
êë x 3 úû êë1 úû
x1 =
1 - 3(0) + 5(1)
= 0.50000
12
28 - (0.5) - 3(1)
x2 =
= 4.9000
5
76 - 3(0.50000 ) - 7(4.9000 )
x3 =
= 3.0923
13
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 2
The absolute relative approximate error
0.50000 - 1.0000
Îa 1 =
´ 100 = 100.00%
0.50000
Îa
Îa
2
3
4.9000 - 0
=
´ 100 = 100.00%
4.9000
3.0923 - 1.0000
=
´ 100 = 67.662%
3.0923
The maximum absolute relative error after the first iteration is 100%
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 2
After Iteration #1
é x1 ù é0.5000 ù
ê x ú = ê 4.9000 ú
ê 2ú ê
ú
êë x3 úû êë 3.0923 úû
Substituting the x values into the
equations
x1 =
1 - 3(4.9000 ) + 5(3.0923)
= 0.14679
12
28 - (0.14679 ) - 3(3.0923)
x2 =
= 3.7153
5
x3 =
After Iteration #2
é x1 ù é0.14679 ù
ê x ú = ê 3.7153 ú
ê 2ú ê
ú
êë x3 úû êë 3.8118 úû
76 - 3(0.14679 ) - 7(4.900 )
= 3.8118
13
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 2
Iteration #2 absolute relative approximate error
Îa 1 =
Îa
Îa
2
3
0.14679 - 0.50000
´ 100 = 240.61%
0.14679
3.7153 - 4.9000
=
´ 100 = 31.889%
3.7153
3.8118 - 3.0923
=
´ 100 = 18.874%
3.8118
The maximum absolute relative error after the fi rst iteration is 240.61%
This is much larger than the maxi mum absolute relative error obtai ned in
iteration #1. Is this a problem?
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 2
Repeating more iterations, the following values are obtained
Iteration
a1
Îa 1 %
a2
Îa 2 %
a3
1
2
3
4
5
6
0.50000
0.14679
0.74275
0.94675
0.99177
0.99919
100.00
240.61
80.236
21.546
4.5391
0.74307
4.9000
3.7153
3.1644
3.0281
3.0034
3.0001
100.00
31.889
17.408
4.4996
0.82499
0.10856
3.0923
3.8118
3.9708
3.9971
4.0001
4.0001
Îa 3 %
67.662
18.876
4.0042
0.65772
0.074383
0.00101
é x1 ù é0.99919ù
The solution obtained ê x ú = ê 3.0001 ú is close to the exact solution of
ê 2ú ê
ú
êë x3 úû êë 4.0001 úû
é x1 ù é1 ù
ê x ú = ê 3ú .
ê 2ú ê ú
êë x3 úû êë4úû
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 3
Given the system of equations
3 x1 + 7 x2 + 13 x3 = 76
Rewriting the equations
x1 + 5 x2 + 3x3 = 28
76 - 7 x2 - 13 x3
x1 =
3
12 x1 + 3x2 - 5 x3 = 1
With an initial guess of
é x1 ù é1 ù
ê x ú = ê0 ú
ê 2ú ê ú
êë x3 úû êë1 úû
28 - x1 - 3x3
x2 =
5
1 - 12 x1 - 3 x 2
x3 =
-5
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 3
Conducting six iterations, the following values are obtained
Iteration
a1
1
2
3
4
5
6
21.000
−196.15
−1995.0
−20149
2.0364×105
−2.0579×105
Îa 1 %
A2
95.238
0.80000
110.71
14.421
109.83
−116.02
109.90
1204.6
109.89
−12140
109.89 1.2272×105
Îa 2 %
a3
Îa 3 %
100.00
94.453
112.43
109.63
109.92
109.89
50.680
−462.30
4718.1
−47636
4.8144×105
−4.8653×106
98.027
110.96
109.80
109.90
109.89
109.89
The values are not converging.
Does this mean that the Gauss-Seidel method cannot be used?
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
The Gauss-Seidel Method can still be used
The coefficient matrix is not
diagonally dominant
But this is the same set of
equations used in example #2,
which did converge.
é 3 7 13 ù
[A] = êê 1 5 3 úú
êë12 3 - 5úû
é12 3 - 5ù
[A] = êê 1 5 3 úú
êë 3 7 13 úû
If a system of linear equations is not diagonally dominant, check to see if
rearranging the equati ons can form a di agonally dominant matrix.
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Not every system of equations can be rearranged to have a
diagonally dominant coefficient matrix.
Observe the set of equati ons
x1 + x 2 + x3 = 3
2 x1 + 3 x 2 + 4 x3 = 9
x1 + 7 x 2 + x3 = 9
Which equation(s) prevents this set of equation from having a
diagonally dominant coefficient matrix?
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Summary
-Advantages of the Gauss-Seidel Method
-Algorithm for the Gauss-Seidel Method
-Pitfalls of the Gauss -Seidel Method
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Questions?
http://numericalmethods.eng.usf.edu
Additional Resources
For all resources on this topic such as digital audiovisual
lectures, primers, textbook chapters, multiple-choice
tests, worksheets in MATLAB, MATHEMATICA, MathCad
and MAPLE, blogs, related physical problems, please
visit
http://numericalmethods.eng.usf.edu/topics/gauss_seid
el.html
THE END
http://numericalmethods.eng.usf.edu