Part II: Linear Algebra
Chapter 8 Systems of Linear Algebraic
Equations; Gauss Elimination
8.1 Introduction
There are many applications in science and engineering
where application of the relevant physical law immediately
produces a set of linear algebraic equations. For instance, to
find a particular solution to the differential equation.
y ''' y ''  3x 2  5sin x
(1)
y( x)  Ax 4  Bx3  Cx 2  D sin x  E cos x.
(2)
five linear algebraic equations on the unknown coefficients A,
B,…, E can be derived.
Five chapters (8-12) on linear algebra with an introduction to
the theory of systems of linear algebraic equations, and their
solution by the method of Gauss elimination are discussed. 1
8.2 Preliminary Ideas and Geometrical Approach
For the equation in the form of
f ( x)  0
(1)
is said to be algebraic, or polynomial, if f(x) is expressible in
the form, anxn ＋ an-1xn-1 ＋ … ＋ a1x ＋ a0, where an≠0 for
definiteness, and it is said to be transcendental otherwise.
Transcendental equation: An equation contains a
Example 1.
transcendental function.
The equations
Transcendental function: A function cannot be
expressed in terms of algebra. Examples of
(1) 6x – 5 = 0
transcendental functions include the exponential
(2) 3x4-x3+ 2x + 1 = 0 (2)
function, the trigonometric functions, and the
3
(3) x + 2sinx = 0
inverse function of both.
(4) ex – 3 = 0
(1)and (2):algebra； (3) and (4): (transcendental)
Besides the algebraic versus transcendental distinction,
we classify (1) as linear if f(x) is a first-degree polynomial,
2
a1x + a0 = 0 and the nonlinear otherwise.
A system of equations consisting of m equations in n
unknowns, where m≥1 and n≥1,
f1 ( x1 ,..., xn )  0,
f 2 ( x1 ,..., xn )  0,
(3)
f m ( x1 ,..., xn )  0
In general, however, m may be less than, equal to, or
greater than n so we allow for m≠n in this discussion even
though m=n is the most important case.
a11 x1  a12 x2 
 a1n xn  c1 ,
(eq.1)
a21 x1  a22 x2 
 a2 n xn  c2 ,
(eq.2)
am1 x1  am 2 x2 
 amn xn  cm ,
(eq.m)
(5)
and we restrict m and n to be finite, and the aij’s and cj’s to be
real numbers. If all the cj’s are zero then (5) is homogeneous;
3
If they are not all zero then (5) is nonhomogeneous.
We say that a sequence of numbers s1 ,s2 ,…,sn is a
solution of (5) if and only if each of the m equations is
satisfied numerically when we substitute s1 for x1, s2 for x2,
and so on.
If there exist one or more solutions to (5), we say that the
system is consistent; if there is precisely one solution, that
solution is unique; and if there is more than one, the solution
is nonunique. If, on the other hand, there are no solutions to
(5), the system is said to be inconsistent. The collection of all
solutions to (5) is called its solution set.
Consider the case where m＝n＝2:
a11 x1  a12 x2  c1 ,
(eq.1)
(7a)
a21 x1  a22 x2  c2 .
(eq.2)
(7b)
(eq.1) defines a straight line, say L1,
(eq.2) defines a straight line L2.
4
There exist three possibilities, as illustrated in Fig. 1.
(1) The lines may intersect at a point, say P as long as
a11a22  a12 a21  0;
(8)
(2) The lines may be parallel and nonintersecting (Fig.1b), in
which case there is no solution.
(3) The lines may coincide (Fig. 1c), in which case the
coordinate pair of each point on the line is a solution.
Fig. 1 Existence and uniqueness for the system
5
Consider the case where m = n = 3.
a11 x1  a12 x2  a13 x3  c1 ,
(eq.1)
(9a)
a21 x1  a22 x2  a23 x3  c2 ,
(eq.2)
(9b)
a31 x1  a32 x2  a33 x3  c3 .
(eq.3)
(9c)
If a11, a12, a13 are not all zero then (Eq.1) defines a plane,
say P1, in Cartesian x1, x2, x3 space, and similarly for (Eq.2)
and (Eq.3). In the generic case, P1 and P2 intersect along a
line L, and L pierces P3 at a point P.
There will be no solution if L is parallel to P3 and hence
fails to pierce it, or if any two of the planes are parallel
and not coincident. There will be an infinity of solutions if
L lies in P3.
6
8.3 Solution by Gauss Elimination
8.3.1. Motivation. In this section we continue to consider the
system of m linear algebraic equations
a11 x1  a12 x2 
 a1n xn  c1 ,
a21 x1  a22 x2 
 a2 n xn  c2 ,
am1 x1  am 2 x2 
 amn xn  cm ,
(1)
In the n unknowns x1,…, xn, and develop the solution technique
known as Gauss elimination.
7
Example 1 Determine the solution set of the system
x1  x2  x3  1,
3x1  x2  x3  9,
x1  x2  x3  1,
(2)
-2x2  4x3  6,
x1  x2  4 x3  8,
 2 x2  5 x3  7,
x1  x2  x3  1,
x1  x2  x3  1,
-2x2  4x3  6,
x3  1,
(4)
x2  2 x3  3,
(3)
(5)
x3  1,
Comments：
1. The original system is tangled because the
equations are coupled.
2. The system was treated into triangular form.
3. Solve the whole system by back substitution.
8
Two linear systems in n unknowns, x1 through xn, are said
to be equivalent if their solution sets are identical.
The following operations on linear systems are known as
elementary equation operations:
1. Addition of a multiple of one equation to another
Symbolically: (eq.j) → (eq.j)＋α (eq.k)
2. Multiplication of an equation by a nonzero constant
Symbolically: (eq.j) → α (eq.j)
3. Interchange of two equations
Symbolically: (eq.j) ↔(eq.k)
9
Theorem 8.3.1 Equivalent Systems
If one linear system is obtained from another by a finite
number of elementary equation operations, then the two
systems are equivalent.
Example 2. Inconsistent System
x1 - 2 x2  x3  3
7 x1
2 x1  3x2  2 x3  4
2 x1  3 x2  2 x3  4
2 x1  3 x2  2 x3  4
(7)
- x3  2
7
- x2  2x3  1
2
21
- x2  6x3  12
2
7
- x2  2x3  1
(9)
2
0  15
(8)
If we change the system (7) by changing the final 2 in (7) to 17,
then the final -12 in (8) become a 3.
2 x1  3x2  2 x3  4
2 x1  3x2  2 x3  4
x1 - 2 x2  x3  3
7 x1
- x3  17
and
7
- x2  2x3  1
2
00
17 1
 
7 7
2 4
x2     (12)
7 7
x3  
x1 
(10)
10
Example 3. Nonunique Solution. Consider the system of four
equations in six unknowns (m＝4, n＝6)
x1  3 x2
2 x2  x3  4 x4  3x5  x6  2,
x1  x2  x3
 2 x6  0, (13)
x1  x2  2 x3  4 x4  x5  2 x6  3,
x1  3 x2
 4 x4  2 x5  x6  0.
x1  3 x2
 4 x4  2 x5  x6  0,
2 x2  x3  4 x4  2 x5  x6  0,
4 x2  2 x3  8 x4  3 x5  x6  3,
x1  x2  x3
2 x2  x3  4 x4  2 x5  x6  0,
 x5  x6  3,
x1  3x2
2 x2  x3  4 x4  2 x5  x6  0,
 x5  x6  3,
 x6  5.
 4 x4  2 x5  x6  0,
x5
 4 x4  2 x5  x6  0,
x2 
(17)
(14)
2 x2  x3  4 x4  3x5  x6  2.
2 x2  x3  4 x4  3x5  x6  2.
x1  3 x2
 2 x6  0,
x1  x2  2 x3  4 x4  x5  2 x6  3,
x1  3 x2
(15)
 4 x4  2 x5  x6  0,
(16)
 2.
 4 x4  2 x5  x6  0,
1
1
x3  2 x4  x5  x6  0
2
2
x5  x6  3,
x6  5.
(18)
11
x6  5,
x2 
x5  2,
1
1
 21   2 ,
2
2
x4  1 ,
x2 
x3   2 ,
21
3
 21   2 ,
2
2
(19)
If a solution set contains p independent arbitrary parameters
(α1,…,αp), we call it (in this text) a p-parameter family of
solutions. Each choice of values for α1,…,αp yields a particular
solution.
8.3.2 Gauss elimination
The method of Guass elimination can be applied to any
linear system (1), whether or not the system is consistent,
and whether or not the solution is unique.
     = 
    = 
   = 
(a)
     = 
     = 
0= 
    = 
0= 
0= 
0= 
0= 
0= 
(d)
(b)
(c)
Fig. 1 The final pattern; m = 3, n = 5
12
Theorem 8.3.2 Existence/uniqueness for linear systems
If m < n, the system (1) can be consistent or inconsistent.
If it is consistent it cannot have a unique solution; it will have
a p-parameter family of solutions, where n-m ≦ p ≦n.
If m≧n, the system (1) can be consistence or inconsistent.
If it is consistent it can have a unique solution or a p-parameter
family of solutions, where 1≦ p ≦ n.
Theorem 8.3.3 Existence/uniqueness for linear systems
Every system (1) necessarily admits no solution, a unique
solution, or an infinity of solutions.
13
Theorem 8.3.4 Existence/uniqueness for homogeneous systems
Every homogeneous linear system of m equations in n
unknowns is consistent. Either it admits the unique trivial
solution or else it admits an infinity of nontrivial solutions in
addition to the trivial solution. If m < n, then there is an infinity
of nontrivial solutions in addition to the trivial solution.
8.3.3 Matrix notation
The augmented matrix of the system (1)
 a11 a12 a1n c1 
a a

 21 22 a2 n c2 




 am1 am 2 amn cm 
14
Coefficient matrix:
 a11 a12 a1n
a a
 21 22 a2 n


 am1 am 2 amn






Elements:
Row:
Column:
Thus, corresponding to the elementary equation operations
on members of a system of linear equations there are
elementary row operations on the augmented matrix, as
follows:
1. Addition of a multiple of one row to another:
2. Multiplication of a row by a nonzero constant:
3. Interchange of two rows
We say that two matrices are row equivalent if one can be
obtained from the other by finitely many elementary row
operations.
15
8.3.4 Guass-Jordan reduction
With the Gauss elimination completed, the remaining steps
consist of back substitution. In fact, those steps are elementary
row operations as well. The difference is that whereas in the
Gauss elimination we proceed from the top down, in the back
substitution we proceed from the bottom up.
1 1 -1 1 
0 1 -2 -3


0 0 1 1 
1 1 0 2 
0 1 0 -1


0 0 1 1
1 0 0 3 
0 1 0 -1


0 0 1 1
The entire process, of Gauss elimination plus back
substitution, is known as Gauss-Jordan reduction.
1. In each row not made up entirely of zeros, the first nonzero
element is a 1, a so-called leading 1.
2. In any two consecutive rows not made up entirely of zero,
the leading 1 in the lower row is to the right of the leading 1
16
in the upper row.
3. All rows made up entirely of zeros are grouped together
at the bottom of the matrix.
4. If a column contains a leading 1, every other element in
that column is a zero.
Row-echelon from:含 (1) 、(2) 、(3) 。
Reduced row-echelon from:含 (1) 、(2) 、(3) 、(4) 。
1 -3

0 1

0 0

0 0
0 -4 -2 1 0 

1
1
2 1
0 
2
2

0 0 1 1 -3 

0 0 0 1 -5 

1

0

0

0

1

0

0

0
3
3
2 0
2
2

3 

3 

-3 
-5 

1

0

0

0
0
1
1
2
2 0 -
1
2
0 0 0 1 1
0 0 0 0 1
Pivot equation
0
1
3
5
2 1
2
2
1
2
2 1
1
2
0 0 0 1 1
0 0 0 0 1

0

0 

-3 
-5 
3
21 
2 0 0
2
2 

1
1 
1
2 0 0
2
2 
0 0 0 1 0 2 
0 0 0 0 1 -5 
0
17
Problems for Chapter 8
Exercise 8.3
1.(g)、(j)、(k)、(q)
2.(q)
6.(a)
7.(b)、(e)
10.(b)
18