GENERAL INFORMATION ABOUT ITERATIVE SOLVERS
There are methods that can be used to solve a set of linear equations that are based on
iteration. An initial estimate of the parameters is estimated and then the equations are solved,
yielding an updated version of the parameters. These new values are then inserted back into the
equations and process continues until the desired solution is reached. The two common methods
which are discussed here are the Jacobi method and the Gauss-Seidel method.
1.
Jacobi Method
Given a set of linear equations:
𝑎11 𝑥1 + 𝑎12 𝑥2 + 𝑎13 𝑥3 + ⋯ + 𝑎1𝑛 𝑥𝑛 = 𝑏1
𝑎21 𝑥1 + 𝑎22 𝑥2 + 𝑎23 𝑥3 + ⋯ + 𝑎2𝑛 𝑥𝑛 = 𝑏2
𝑎31 𝑥1 + 𝑎32 𝑥2 + 𝑎33 𝑥3 + ⋯ + 𝑎3𝑛 𝑥𝑛 = 𝑏2
⋮
𝑎𝑛1 𝑥1 + 𝑎𝑛2 𝑥2 + 𝑎𝑛3 𝑥3 … + 𝑎𝑛𝑛 𝑥𝑛 = 𝑏𝑛
Solve for 𝑥1 , 𝑥2 , … , 𝑥𝑛
We begin by rearranging these equations in the form of solving for the unknown parameters
one equation at a time:
𝑏1 𝑎12 (0) 𝑎13 (0)
𝑎1𝑛 (0)
−
𝑥2 −
𝑥3 − ⋯ −
𝑥
𝑎11 𝑎11
𝑎11
𝑎11 𝑛
𝑏2 𝑎21 (0) 𝑎23 (0)
𝑎2𝑛 (0)
𝑥2 =
−
𝑥1 −
𝑥3 − ⋯ −
𝑥
𝑎22 𝑎22
𝑎22
𝑎22 𝑛
⋮
𝑏𝑛
𝑎𝑛1 (0) 𝑎𝑛2 (0)
𝑎𝑛,𝑛−1 (0)
𝑥𝑛 =
−
𝑥1 −
𝑥2 − ⋯ −
𝑥
𝑎𝑛𝑛 𝑎𝑛𝑛
𝑎𝑛𝑛
𝑎𝑛𝑛 𝑛−1
𝑥1 =
The superscript (0) indicates the initial estimate of the parameters. For the first iteration, these
parameters are given the value zero. The equations are then solved which results in an updated
value of the parameters. These current estimates are then inserted back into the equations and a
newer set of parameters is arrived at by solving these equations. The process continues until the
solution converges.
Example:
7𝑥1 + 3𝑥2 + 𝑥3 = 18
{2𝑥1 − 9𝑥2 + 4𝑥3 = 12
𝑥1 − 4𝑥2 + 12𝑥3 = 6
Rearrange these equations:
18 3
1
− 𝑥2 − 𝑥3 ≈ 2.571 − 0.429𝑥2 − 0.143𝑥3
7 7
7
12 2
4
𝑥2 = − + 𝑥1 + 𝑥3 ≈ −1.333 + 0.222𝑥1 + 0.444𝑥3
9 9
9
6
1
4
𝑥3 =
− 𝑥 +
𝑥 ≈ 0.5000 − 0.083𝑥1 + 0.333𝑥2
12 12 1 12 2
𝑥1 =
(0)
(0)
(0)
Use initial estimates: 𝑥1 = 𝑥2 = 𝑥3 = 0
(1)
(0)
(0)
𝑥1 = 2.571 − 0.429𝑥2 − 0.143𝑥3 = 2.571 − 0.429(0) − 0.143(0) = 2.571
(1)
(0)
(0)
(0)
(0)
𝑥2 = −1.333 + 0.222𝑥1 + 0.444𝑥3 = −1.333 + 0.222(0) + 0.444(0) = −1.333
(1)
𝑥3 = 0.5000 − 0.083𝑥1 + 0.333𝑥2 = 0.5000 − 0.083(0) + 0.333(0) = 0.5000
Insert these updated estimates back into original equation again, yielding:
(2)
(1)
(1)
𝑥1 = 2.571 − 0.429𝑥2 − 0.143𝑥3 = 2.571 − 0.429(−1.333) − 0.143(0.5000) = 3.071
(2)
(1)
(1)
(2)
(1)
(1)
𝑥2 = −1.333 + 0.222𝑥1 + 0.444𝑥3 = −1.333 + 0.222(2.571) + 0.444(0.5000) = −0.540
𝑥3 = 0.5000 − 0.083𝑥1 + 0.333𝑥2 = 0.5000 − 0.083(2.571) + 0.333(−1.333) = −0.159
Continue this process until the desired results are obtained.
The table below shows the solutions arrived at after each iteration:
Iteration
1
2
3
4
5
6
X(1)
2.55143
3.07143
2.82540
2.87141
2.85811
2.85979
X(2)
-1.33333
-.53968
-.72134
-.67695
-.68453
-.68261
X(3)
.5000
-.15873
.06415
.2410
.03506
.03365
𝑥1
2.859
𝑥
The exact solution is [ 2 ] = [−0.683], as we can see, the solutions using the Jacobi iterative
𝑥3
0.034
method are very close.
(1)
(0)
(0)
𝑥1 = 2.571 − 0.429𝑥2 − 0.143𝑥3 = 2.571 − 0.429(0) − 0.143(0) = 2.571
(1)
(1)
(0)
(1)
(1)
(1)
𝑥2 = −1.333 + 0.222𝑥1 + 0.444𝑥3 = −1.333 + 0.222(2.571) + 0.444(0) = −0.762
𝑥3 = 0.5000 − 0.083𝑥1 + 0.333𝑥2 = 0.5000 − 0.083(2.571) + 0.333(−0.762) = 0.033
Next iteration:
(2)
(1)
(1)
𝑥1 = 2.571 − 0.429𝑥2 − 0.143𝑥3 = 2.571 − 0.429(−0.762) − 0.143(0.033) = 2.893
(2)
(2)
(1)
(2)
(2)
𝑥2 = −1.333 + 0.222𝑥1 + 0.444𝑥3 = −1.333 + 0.222(2.893) + 0.444(0.033) = −0.676
(2)
𝑥3 = 0.5000 − 0.083𝑥1 + 0.333𝑥2 = 0.5000 − 0.083(2.893) + 0.333(−0.676) = 0.034
The table below shows the solutions arrived at after each iteration:
Iteration
1
2
3
4
X(1)
2.55143
2.89342
2.85646
2.85957
X(2)
-.76190
-.67624
-.6869
-.68273
X(3)
.03175
.03347
.03406
.03412
2. Gauss-Seidel Iterative Method
The Gauss-Seidel iterative method of solving for a set of linear equations can be thought
of as just an extension of the Jacobi method. Start out using an initial value of zero for each
of the parameters. Then, solve for 𝑥1 as in the Jacobi method. When solving for 𝑥2 , insert the
just computed value for 𝑥1 , and so on. In other words, for each calculation, the most current
estimate of the parameter value is used.
Let’s use the Gauss-Seidel iterative method to solve the last example: