Iterative Methods for Solving
Linear Systems
Leo Magallon & Morgan Ulloa
What is an Iterative Method
- An alternative method for solving the linear
systems problem Ax=b where A is an nxn matrix
with n equations and n unknowns.
- These methods use a repetitive algorithm to
generate sequences of vectors that steadily
approach your solution.
We will discuss two types of iterative methods:
 Jacobi
 Gauss-Seidel
Convergence
• Convergence is when the outputs of your functions stop
changing significantly. In other words they converge
towards a solution.
• Iterative methods rely on the functions’ ability to
converge towards a solution, if they don’t converge then
you cannot find an answer.
• We can say that a system will converge if it is diagonally
dominant. When the absolute value of each diagonal
entry is greater than the absolute value of the sum of
every component in its row.
The linear systems problem
Ax=b
• Let’s look at the following
system
• From what we’ve learned
in class we know how to
solve this system by
taking rref(A) in an
augmented matrix.
• So lets solve our system
using elementary row
operations.
As we can see by
taking rref(A) the
solution to our
system is
x1 = 1
x2 = 2
Lets check our answer:
7(1) – 1(2)= 5
3(1) – 5(2)= -7
•☺Correct!
So why use iterative
methods?
• When using elementary row operations by hand
it is likely that calculating errors will be made
(especially when the matrix is full of fractions)
• Iterative methods can be quicker if the matrix is
simple or has many zero components.
• You can stop calculating when your answers
start to converge, whereas with row operations
you must work all the way to rref(A)
• Round off errors may actually accelerate
convergence because you are jumping more
quickly towards your solution.
Let’s look at Jacobi’s method
From our 2x2 matrix we can see that there are 2
equations with 2 unknowns, and the matrix is
diagonally dominant; therefore it will converge.
7x1 – 1x2 = 5
3x1 – 5x2 = -7
Step 1: solve the first equation for x1 and solve the
second equation for x2
x1 = 5 + x2
x2 = 7 + 3x1
7
5
Step 2: choose an initial approximation. We choose
x1 = 0
x2 = 0
Step 3: plug in your chosen values and solve for x1 and x2.
X1 = 5 + 1(0) = 5/7
X2 = 7 + 3(0) = 7/5
7
5
Step 4: use new values for x1 and x2 and input them into
your original functions…. continue this process until it
converges to your solution.
Here we can see the solution is converging to
x1=1 x2= 2 at around 6 iterations.
the same as our answer from rref(A).
Gauss-Seidel
Almost identical to Jacobi’s method except it
converges faster because we use our new
outputs as soon as we can.
Let’s see how it works…
Step1: from our two equations, solve for x1 and x2
7x1 – 1x2 = 5
3x1 – 5x2 = -7
x1 = 5 + x2
7
x2 = 7 + 3x1
5
Step 2: choose an initial approximation. we choose
x1 = 0
x2 = 0
Step 3: plug in your initial value and solve for x1
x1 = 5 + 1(0) = 5/7
7
Step 4: now use this new value for x1 to solve for x2
x2 = 7 + 3(5/7) = 64/35 ≈ 1.829
5
solve again for x1 x = 5 + 1(64/35)≈ 0.976
7
• Step 5: continue to input your newest
values for x1 and x2 into the functions until
they converge towards a solution.
• Here we can see that the solution is
converging to x1= 1 x2= 2 at around
4 iterations.
iteration shown graphically
• David Strong's applet for solving Ax=b
using iterative methods
Iterative methods rock!