Gauss-Siedel Method
Chemical Engineering Majors
Authors: Autar Kaw
http://numericalmethods.eng.usf.edu
Transforming Numerical Methods Education for STEM
Undergraduates
3/21/2016
http://numericalmethods.eng.usf.edu
1
Gauss-Seidel Method
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
An iterative method.
Basic Procedure:
-Algebraically solve each linear equation for xi
-Assume an initial guess solution array
-Solve for each xi and repeat
-Use absolute relative approximate error after each iteration
to check if error is within a pre-specified tolerance.
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Why?
The Gauss-Seidel Method allows the user to control round-off error.
Elimination methods such as Gaussian Elimination and LU
Decomposition are prone to prone to round-off error.
Also: If the physics of the problem are understood, a close initial
guess can be made, decreasing the number of iterations needed.
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Algorithm
A set of n equations and n unknowns:
a11x1  a12 x2  a13 x3  ...  a1n xn  b1
a21x1  a22 x2  a23 x3  ...  a2n xn  b2
.
.
.
.
.
.
an1x1  an 2 x2  an3 x3  ...  ann xn  bn
If: the diagonal elements are
non-zero
Rewrite each equation solving
for the corresponding unknown
ex:
First equation, solve for x1
Second equation, solve for x2
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Algorithm
Rewriting each equation
x1 
c1  a12 x2  a13 x3   a1n xn
a11
From Equation 1
c2  a21 x1  a23 x3   a2 n xn
a22
From equation 2
x2 
xn 1 
xn 



cn 1  an 1,1 x1  an 1, 2 x2   an 1,n  2 xn  2  an 1,n xn
From equation n-1
an 1,n 1
cn  an1 x1  an 2 x2    an ,n 1 xn 1
From equation n
ann
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Algorithm
General Form of each equation
n
c1   a1 j x j
x1 
j 1
j 1
cn 1 
xn 1 
a11
n
a
j 1
j  n 1
n 1, j
xj
an 1,n 1
n
c n   a nj x j
n
c2   a2 j x j
x2 
j 1
j 2
a 22
xn 
j 1
j n
a nn
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Algorithm
General Form for any row ‘i’
n
ci   aij x j
xi 
j 1
j i
aii
, i  1,2, , n.
How or where can this equation be used?
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Solve for the unknowns
Assume an initial guess for [X]
 x1 
x 
 2
  
 
 xn -1 
 xn 
Use rewritten equations to solve for
each value of xi.
Important: Remember to use the
most recent value of xi. Which
means to apply values calculated to
the calculations remaining in the
current iteration.
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Calculate the Absolute Relative Approximate Error
a i
xinew  xiold

100
new
xi
So when has the answer been found?
The iterations are stopped when the absolute relative
approximate error is less than a prespecified tolerance for all
unknowns.
http://numericalmethods.eng.usf.edu
Example: Liquid-Liquid Extraction
A liquid-liquid extraction process conducted in the Electrochemical
Materials Laboratory involved the extraction of nickel from the aqueous
phase into an organic phase. A typical experimental data from the
laboratory is given below:
Ni aqueous phase, a (g/l)
2
2.5
3
Ni organic phase, g (g/l)
8.57
10
12
Assuming g is the amount of Ni in organic phase and a is the amount of
Ni in the aqueous phase, the quadratic interpolant that estimates g is
given by
g  x1a 2  x2 a  x3 , 2  a  3.5
Example: Liquid-Liquid Extraction
The solution for the unknowns x1, x2, and x3 is given by
2 1  x1  8.57
 4
6.25 2.5 1  x    10 

  2 

 9
3 1  x3   12 
Find the values of x1, x2,and x3 using the Gauss Seidel method.
Estimate the amount of nickel in organic phase when 2.3 g/l is in
the aqueous phase using quadratic interpolation.
 x1  1
Initial Guess:  x   1 Conduct two iterations.
 2  
 x3  1
Example: Liquid-Liquid Extraction
Rewriting each equation
2 1  x1  8.57
 4
6.25 2.5 1  x    10 

  2 

 9
3 1  x3   12 
8.57  2 x2  x3
x1 
4
10  6.25 x1  x3
x2 
2.5
12  9 x1  3x2
x3 
1
Example: Liquid-Liquid Extraction
Iteration 1
Applying the initial guess and solving for each xi
8.57  2 1  1
x1 
 1.3925
4
 x1  1
  
Initial Guess  x2   1
 x3  1
x2 
10  6.25 1.3925  1
 0.11875
2.5
x3 
12  9 1.3925  3  0.11875
 0.88875
1
When solving for x2, how many of the initial guess values were used?
Example: Liquid-Liquid Extraction
Finding the absolute relative approximate error for Iteration 1.
a i
xinew  xiold

100
new
xi
a 1 
a
2
1.3925  1
100  28.187%
1.3925
0.11875  1

100  742.11%
0.11875
a 3 
 0.88875  1
100  212.52%
 0.88875
At the end of the Iteration 1
 x1   1.3925 
 x    0.11875 
 2 

 x3   0.88875
The maximum absolute
relative approximate error is
742.11%.
Example: Liquid-Liquid Extraction
Iteration 2
Using
 x1   1.3925 
 x    0.11875 
 2 

 x3   0.88875
from Iteration 1
the values of xi are found.
8.57  2  0.11875  (0.88875)
 2.3053
4
10  6.25  2.3053  (0.88875)
x2 
 1.4078
2.5
12  9  2.3053  3  (1.4078)
x3 
 4.5245
1
x1 
Example: Liquid-Liquid Extraction
Finding the absolute relative approximate error for Iteration 2.
2.3053  1.3925
a 1 
 100
2.3053
 39.596 %
a
2
 1.40778  0.11875

 100
 1.4078
 108.44 %
 4.5245  (0.88875)
 100
 4.5245
 80.357 %
a 3 
At the end of the Iteration 1
 x1   2.3053 
 x     1.4078 
 2 

 x 3   4.5245
The maximum absolute
relative approximate error is
108.44%
Example: Liquid-Liquid Extraction
Repeating more iterations, the following values are obtained
Iteration
x1
a 1 %
x2
1
2
3
4
5
6
1.3925
2.3053
3.9775
7.0584
12.752
23.291
28.1867
39.5960
42.041
43.649
44.649
45.249
0.11875
−1.4078
−4.1340
−9.0877
−18.175
−34.930
a 2 %
x3
742.1053 −0.88875
108.4353 −4.5245
65.946
−11.396
54.510
−24.262
49.999
−48.243
47.967
−92.827
a 3 %
212.52
80.357
60.296
53.032
49.708
48.030
Notice – The relative errors are not decreasing at any significant rate
 x1   1.14 
  

Also, the solution is not converging to the true solution of  x 2    2.27
 x 3   8.55 
Gauss-Seidel Method: Pitfall
What went wrong?
Even though done correctly, the answer is not converging to the
correct answer
This example illustrates a pitfall of the Gauss-Seidel method: not all
systems of equations will converge.
Is there a fix?
One class of system of equations always converges: One with a diagonally
dominant coefficient matrix.
Diagonally dominant: [A] in [A] [X] = [C] is diagonally dominant if:
n
aii   aij
j 1
j i
n
for all ‘i’
and
a ii   a ij
j 1
j i
for at least one ‘i’
Gauss-Siedel Method: Pitfall
Diagonally dominant: The coefficient on the diagonal must be at least
equal to the sum of the other coefficients in that row and at least one
row with a diagonal coefficient greater than the sum of the other
coefficients in that row.
Original (Non-Diagonally dominant)
2 1  x1  8.57
 4
6.25 2.5 1  x    10 

  2 

 9
3 1  x3   12 
Rewritten (Diagonally dominant)
3 1  x1   12 
 9
6.25 2.5 1  x    10 

  2 

 4
2 1  x3  8.57
Example: Liquid-Liquid Extraction
 x1  1
Iteration 1
  
With an initial guess of  x2   1
 x3 
1
Rewriting each equation
12  3 1  1
 0.88889
9
10  6.25  0.88889  1
x2 
 1.3778
2.5
8.57  4  0.88889  2 1.3778
x3 
 2.2589
1
x1 
Example: Liquid-Liquid Extraction
The absolute relative approximate error for
Iteration 1 is
0.88889  1
a 1 
 100  12.5%
0.88889
a
2
a 3
1.3778  1

100  27.419%
1.3778
2.2589  1

100  55.730%
2.2589
At the end of Iteration 1
 x1  0.8889
 x   1.3778 
 2 

 x3  2.2589
The maximum absolute relative error after the first
iteration is 55.730%
Example: Liquid-Liquid Extraction
Iteration 2
 x1  0.8889
  

Using  x2   1.3778 
 x3  2.2589
from Iteration 1
the values for xi are found
12  3 1.3778  1 2.2589
 0.62309
9
10  6.25  0.62309  1 2.2589
x2 
 1.5387
2.5
8.57  4  0.62309  2 1.5387
x3 
 3.0002
1
x1 
Example: Liquid-Liquid Extraction
The absolute relative approximate error for Iteration 2
a 1 
a
2
a 3
0.62309  0.88889
 100  42.659%
0.62309
1.5387  1.3778

100  10.460%
1.5387
3.0002  2.2589

 100  24.709%
3.0002
At the end of Iteration 2
 x1  0.62309
 x    1.5387 
 2 

 x 3   3.0002 
The maximum absolute relative error after the first
iteration is 42.659%
Example: Liquid-Liquid Extraction
Repeating more iterations, the following values are obtained
Iteration
x1
a 1 %
x2
1
2
3
4
5
6
0.88889
0.62309
0.48707
0.42178
0.39494
0.38890
12.5
42.659
27.926
15.479
6.7960
1.5521
1.3778
1.5387
1.5822
1.5627
1.5096
1.4393
a 2 %
27.419
10.456
2.7506
1.2537
3.5131
4.8828
x3
a 3 %
2.2589
3.0002
3.4572
3.7576
3.9710
4.1357
55.730
24.709
13.220
7.9928
5.3747
3.9826
After six iterations, the absolute relative approximate error
seems to be decreasing. Conducting more iterations allows
the absolute relative approximate error to decrease to an
acceptable level.
Example: Liquid-Liquid Extraction
Repeating more iterations, the following values are obtained
Iteration
x1
a 1 %
x2
a 2 %
x3
a 3 %
199
200
1.1335
1.1337
0.014412
0.014056
−2.2389
−2.2397
0.034871
0.034005
8.5139
8.5148
0.010666
0.010403
The value of
 x1   1.1337 
 x    2.2397 
 2 

 x3   8.5148 
closely approaches the true value of
 x1   1.14 
 x    2.27
 2 

 x 3   8.55 
Example: Liquid-Liquid Extraction
The polynomial that passes through the three data points is then
g a   x1 a   x2 a   x3
2
 1.1337a    2.2397 a   8.5148
2
Where g is grams of nickel in the organic phase and a is the
grams/liter in the aqueous phase.
When 2.3g/l is in the aqueous phase, using quadratic interpolation, the
estimated the amount of nickel in the organic phase
g 2.3  1.1337  2.3   2.2397 2.3  8.5148
2
 9.3608 g/l
Gauss-Seidel Method: Example 3
Given the system of equations
3x1  7 x2  13x3  76
Rewriting the equations
x1  5x2  3x3  28
76  7 x2  13 x3
x1 
3
12 x1  3x2  5x3  1
With an initial guess of
 x1  1
 x   0 
 2  
 x3  1
28  x1  3 x3
x2 
5
1  12 x1  3 x 2
x3 
5
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method: Example 3
Conducting six iterations, the following values are obtained
Iteration
a1
1
2
3
4
5
6
21.000
−196.15
−1995.0
−20149
2.0364×105
−2.0579×105
a 1 %
A2
95.238
0.80000
110.71
14.421
109.83
−116.02
109.90
1204.6
109.89
−12140
109.89 1.2272×105
a 2 %
a3
a 3 %
100.00
94.453
112.43
109.63
109.92
109.89
50.680
−462.30
4718.1
−47636
4.8144×105
−4.8653×106
98.027
110.96
109.80
109.90
109.89
109.89
The values are not converging.
Does this mean that the Gauss-Seidel method cannot be used?
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
The Gauss-Seidel Method can still be used
The coefficient matrix is not
diagonally dominant
 3 7 13 
A   1 5 3 
12 3  5
But this same set of equations will
converge.
12 3  5
A   1 5 3 
 3 7 13 
If a system of linear equations is not diagonally dominant, check to see if
rearranging the equations can form a diagonally dominant matrix.
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Not every system of equations can be rearranged to have a
diagonally dominant coefficient matrix.
Observe the set of equations
x1  x2  x3  3
2 x1  3x2  4 x3  9
x1  7 x2  x3  9
Which equation(s) prevents this set of equation from having a
diagonally dominant coefficient matrix?
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Summary
-Advantages of the Gauss-Seidel Method
-Algorithm for the Gauss-Seidel Method
-Pitfalls of the Gauss-Seidel Method
http://numericalmethods.eng.usf.edu
Gauss-Seidel Method
Questions?
http://numericalmethods.eng.usf.edu
Additional Resources
For all resources on this topic such as digital audiovisual
lectures, primers, textbook chapters, multiple-choice
tests, worksheets in MATLAB, MATHEMATICA, MathCad
and MAPLE, blogs, related physical problems, please
visit
http://numericalmethods.eng.usf.edu/topics/gauss_seid
el.html
THE END
http://numericalmethods.eng.usf.edu