Chapter 2 System of Equations
2.1 Gaussian Elimination
Let us consider a system of n linear equations in unknowns x 1 ,x 2 ,
,x n . It can be written
in the form
é
ê
ê
ê
ê
ê
ê
ê
ë
a11
a12
a13
a21
a22
a23
a31
a32
a33
a1n ù é
úê
a2n ú ê
úê
a3n ú ê
úê
úê
ann ú ê
ûë
an1 an 2 an 3
x1 ù é
ú ê
x2 ú ê
ú ê
x3 ú = ê
ú ê
ú ê
xn ú ê
û ë
b1 ù
ú
b2 ú
ú
b3 ú .
ú
ú
bn ú
û
The matrices in this equation are denoted by A, x and b . Thus, our system is simply
Ax  b
(1)
Easy-to-solve systems:
We begin by looking for special types of systems that can be easily solved. For example,
suppose that the n  n matrix A has a diagonal structure. This means that all the nonzero
elements of A are on the main diagonal, and system (1) is
é a
ê 11
ê
a22
ê
ê
ê
ë
ann
ùé x
úê 1
úê x2
úê
úê
úê x
ûë n
ù é b ù
ú ê 1 ú
ú ê b2 ú
ú=ê
ú.
ú ê
ú
ú ê b ú
û ë n û
In this case, our system collapses to n simple equations and the solution is
é b
ê 1
a11
ê
ê b
ê 2
a22
x=ê
ê
ê
ê bn
ê
ann
ë
ù
ú
ú
ú
ú
ú
ú
ú
ú
ú
û
If aii  0 for some index i , and if bi  0 also, then xi can be any real number. If
aii  0 and bi  0 , no solution of the system exists.
1
Continuing our search for easy solution of system (1), we assume a lower triangular structure
of A . This means that all the nonzero elements of A are situated on or below the main
diagonal, and system (1) is
é
ê
ê
ê
ê
ê
ê
ê
ë
a11
0
0
a21
a22
0
a31
a32
a33
0 ùé
úê
0 úê
úê
0 úê
úê
úê
ann ú ê
ûë
an1 an 2 an 3
x1 ù é
ú ê
x2 ú ê
ú ê
x3 ú = ê
ú ê
ú ê
xn ú ê
û ë
b1 ù
ú
b2 ú
ú
b3 ú
ú
ú
bn ú
û
To solve this, assume aii  0 for all i , and then obtain x1 from the first equation. With the
known value of x1 substituted into the second equation, solve the second equation for x2 .
We proceed in the same way, obtaining x 1 ,x 2 ,
,x n , one at a time and in this order. A
formal algorithm for the solution in this case is called forward substitution.
 
Input n, aij ,  bi 
for i  1 to n do

xi  bi   j 1 aij x j
i 1
a
ii
end do
Output
 xi 
As is customary, any sum of the type


i 
in which    is interpreted to be 0.
The same ideas can be exploited to solve a system having an upper triangular structure. Such
a matrix system has the form
é
ê
ê
ê
ê
ê
ê
ê
ë
a11 a12
0
a13
a22 a23
0
0
a33
0
0
0
a1n ù é
úê
a2n ú ê
úê
a3n ú ê
úê
úê
ann ú ê
ûë
x1 ù é
ú ê
x2 ú ê
ú ê
x3 ú = ê
ú ê
ú ê
xn ú ê
û ë
b1 ù
ú
b2 ú
ú
b3 ú .
ú
ú
bn ú
û
2
Again, it must be assumed that aii  0 for 1  i  n . The formal algorithm to solve for x is
as follows and is called back substitution:
 
input n, aij ,  bi 
for i  n to 1 step 1 do

xi  bi   j i 1 aij x j
n
a
ii
end do
output  xi 
Naive Gaussian Elimination
Here is a simple system of four equations in four unknowns that will be used to illustrate the
Gaussian elimination algorithm.
 6 2
12 8

 3 13

 6 4
4   x1   12 
6 10   x2   34 

9 3   x3   27 
  

1 18  x4   38
2
(1)
In the first step of the process, we subtract 2 times the first equation from the second. Then we
subtract 1 2 times the first equation from the third. Finally, we subtract 1 time the first
1
2
equation from the fourth. The numbers 2, , 1 are called the multipliers for the first step in
the elimination process. The number 6 , used as the divisor in forming each of these
multiplier is called the pivot element for this step. After the first step has been completed, the
system will look like this:
6 2
0 4

0 12

0 2
4   x1   12 
2 2   x2   10 

(2)
8 1   x3   21 
  

3 14   x4   26 
2
3
In the next step of process, row 2 is use. We subtract 3 times the second row from the third,

1
1
times the second row is subtracted from the fourth. The multipliers are 3 and  . The
2
2
result is
6 2
0 4

0 0

0 0
4   x1   12 
2 2   x2   10 

2 5   x3   9 
  

4 13  x4   21
2
(3)
The final step consists of subtracting 2 times the third row from the fourth. The resulting
system is
6 2
0 4

0 0

0 0
4   x1  12 
2 2   x2  10 

2 5  x3   9 
   
0 3  x4   3
2
(4)
This system is upper triangular and equivalent to the original system in the sense that the
solutions of the two systems are the same. The final system is easily solved by backward
substitution. The solution is
1
 3 
x 
 2 
 
1
4
Operation Counts
Following is the Gaussian Elimination algorithm:
for j = 1: n -1
for i = j +1: n
mult = a(i, j) / a( j, j);
for k = j +1: n
a(i, k) = a(i, k) - mult ´ a( j, k)
end
b(i) = b(i) - mult ´ b( j)
end
end
Operation Count for the elimination step of Guassian Elimination:
The elimination step for a system of
n equations in n variables can be completed in
n3 n 2 5n
+ multiplication / divisions.
3 2
6
5
The back-substitution step is
for i = n : -1:1
for
b(i) = b(i) - a(i, j) ´ x( j)
end
x(i) = b(i) / a(i,i)
end
Operation Count for the back subsitution step of Guassian Elimination:
The back subtitution step for a system of
n equations in n variables can be completed in
n2 n
+ multiplication / divisions.
2 2
If we ignore the lower order terms in the expressions, we find that elimination takes on the
order of
n3
n2
operations and that back substitution takes on the order of
. We will often
3
2
use the shorthand terminology of “big-O” to mean “on the order of” saying that elimination is
æ n3 ö
æ n2 ö
an O ç ÷ algorithm and that back substitution is O ç ÷ .
è 3ø
è 2ø
The operation count shows that direct solution of n equations in n unkonwns by Guassian
æ n3 ö
elimination is an O ç ÷ process.
è 3ø
6