MATH-415 Numerical Analysis
SOLVING SYSTEMS OF LINEAR
EQUATIONS. INTRODUCTION.
CRAMER’S RULE
1
Linear Algebraic Equations
• An equation of the form ax+by+c=0 or ax+by=-c
is called a linear equation in unknowns x and y
• ax+by+cz=d is a linear equation in three unknowns x, y, and z
• Thus, a linear equation in n unknowns is
a1x1+a2x2+ … +anxn = b
a1 ,..., an , b  R or a1 ,..., an , b  C
• A solution of such an equation consists of numbers
c 1 , c2 , c3 , … , cn
• If we need to work more than one linear equations, a set
(system) of linear equations must be solved simultaneously
2
Systems of Linear Equations
a11 x1  a12 x2  a13 x3 
 b1
a21 x1  a22 x2  a23 x3 
 b2
a31 x1  a32 x2  a33 x3 
 b3
 In a matrix form:
 a11
a
 21
 a31


a12
a22
a32
a13
a23
a33
  x1   b1 
  x  b 
 2   2
  x3   b3 
   
   
Ax  b
Systems of Linear Equations
• Solve Ax=b, where A is an nn matrix and
b is an n1 column vector
• Can also talk about non-square systems where
A is mn, b is m1, and x is n1
 Overdetermined if m>n:
“more equations than unknowns”
 Underdetermined if n>m:
“more unknowns than equations”
Square Matrices
• A square matrix of order n contains the same
number of rows and columns
 a11
a
 21
A   a31


 an1
a12
a22
a32
a13
a23
a33
an 2
...
a1n 





ann 
• Square matrices are particularly important
when a system of equations to be solved
5
A Symmetric Matrix
• A square matrix is called a symmetric matrix
when the pairs of elements with the
symmetric indexes across the diagonal are
aij  a ji ; i, j  1,..., n
equal
 a11
x

 y
x
a22
z
y

z 
a33 
6
The Transpose of a Matrix
• The transpose of a matrix is a matrix obtained
by writing the rows as columns or (which is
the same) by writing the columns as rows
A   aij  ; AT   a ji 
 a11
a
A   21
 ...

 an1
a12
a22
...
an 2
... a1n 
 a11
... a2 n  T  a12
;A 

 ...
... ...


... ann 
 a1n
a21
a22
...
a2 n
... an1 
... an 2 
... ... 

... ann 
7
Diagonal Matrix
• If all elements in a square matrix except those on the
main diagonal are zero, the matrix is called a
diagonal matrix
 a11
0

A 0

0
 0
0
a22
...
...
...
...
0
...
...
...
0 an 1n 1
...
0
0
0 
0

0
ann 
8
Triangular Matrices
• A square matrix is called triangular if all the
elements above or below diagonal are zeros
0
...
0
a
• Lower-triangular


11
• Upper-triangular
 a21
 ...

 an1
a22
...
an 2
0...
 a11
0

0

0
a12
a22
0
0
... a1n 
... a2 n 
... 

0 ann 
...
0
0

ann 
9
Identity Matrix
• If the nonzero elements of a diagonal matrix of order
n all are equal to 1 , the matrix is called the identity
matrix of order n
1
0

I n  0

0
0
0 ... ... 0 
1 0 ... 0 
...
... 0 

... 0 1 0 
... ... 0 1 
• For any square matrix A of order n I n A  AI n  A
10
Transposition Matrix
• If two rows of an identity matrix are
interchanged, it is called a transposition
matrix
1
0
I4  
0

0
0
1
0
0
0
0
1
0
0
1
0
0 


0  2 nd and 4th rows are interchanged 0


1
0
0
0
0
1
0
0
1
0
0
1 
 P24  P42
0

0
11
Transposition Matrix
• If a transposition matrix is multiplied with a
square matrix A of the same size, the product
will be the A matrix, but with the same two
rows interchanged as in the transposition
matrix
9
4
A
0

3
6
2
7
2
2 13
1
0
8 11
; P24 A  
0
1 9


6 8
0
0
0
0
1
0
0
1
0
0  9
1   4
0  0

0 3
6
2
7
2
2 13 9
8 11  3

1 9  0
 
6 8  4
6
2
7
2
2 13
6 8 
1 9

8 11
12
Transposition Matrix
• If a square matrix A is multiplied with the
transposition matrix P of the same size, the
product will be the A matrix, but with the
columns of A interchanged according to the
rows of P interchanged
9
4
A
0

3
6
2
7
2
2 13
9
4
8 11
; AP24  
0
1 9


6 8
3
6
2
7
2
2 13 1 0 0
8 11 0 0 0
1 9  0 0 1

6 8  0 1 0
0  9 13 2 6 
1   4 11 8 2 

0  0 9 1 7 
 

0 3 8 6 2
13
Permutation Matrix
• A permutation matrix is obtained by
multiplying several transposition matrices
1
0
P24 P12  
0

0
9
4
A
0

3
6
2
7
2
1
0
0
0
0
0
1
0
0  0
0  0

0  0
 
1  1
1
0
0
0
0
0
1
0
2 13
0
0
8 11
; PA  P24  P12 A   
0
1 9


6 8
1
1
0
0
0
0
0
1
0
0  9
1   4
0  0

0 3
6
2
7
2
2 13  4
8 11  3

1 9  0
 
6 8  9
0
0
0
1
0
0
1
0
0  0
1  1
0  0

0  0
0
1 
P
0

0
2
2
7
6
8 11
6 8 
1 9

2 13
14
Permutation
• A permutation is a function that reorders an ordered set of
integers. This function can be determined by a table
containing two rows: the 1st row contains the integers in the
initial order, the 2nd row contains the corresponding reordered
set
 1 2 ... n 

 ;  i  1,..., n , i  1,..., n
 1  2 ...  n 
• Symbols r and s in a permutation create an inversion if r>s
and r precedes s :
1 2 3 4 
n 
 1 2 ...


2
1
4
3




 1 r ... s ...  n 
2,1 and 4,3 are inversions
• A permutation is even if it contains the even number of
inversions , and it is odd if it contains the odd number of
inversions
15
Permutation and Permutation
Matrix
• A permutation corresponding to a permutation
matrix determines a new order of rows:
1
0
P24 P12  
0

0
0
0
0
1
0
0
1
0
0  0
1  1
0  0

0  0
1
0
0
0
0
0
1
0
0  0
0  0

0  0
 
1  1
1
0
0
0
0
0
1
0
0
1 
P
0

0
1 2 3 4 


2
4
3
1


16
Determinant
• A determinant of order n corresponding to a given
square matrix of order n is a value, which equals to the
algebraic sum of n! additive terms such that every term
is a product of exactly the n elements from the matrix
taken by 1 from every row and every column
• The additive term appears in the sum with the “+” sign
if the permutation created from the numbers of rows
(the 1st row of the permutation) and columns (the
second row of the permutation) is even, and with the
“-” sign if this permutation is odd
• det(A) is a common notation for the determinant of A
17
Determinant
• 2x2 matrix
 a11
A
 a21
a12 
;

a22 
• 3x3 matrix
det( A)  a11a22  a12 a21
1 2  1 2 

 

1
2
2
1

 

0 invers.
1 invers.
 a11 a12 a13 
A   a21 a22 a23  ;
 a31 a32 a33 
det( A)  a11a22 a33  a12 a21a33  a13a21a32  a11a23a32  a12 a23a31  a13a12 a31
1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 







1
2
3
2
1
3
3
1
2
1
3
2
2
3
1
3
2
1







0 invers.
1 invers.
2 invers.
1 invers.
2 invers.
3 invers.
18
Singular Matrix
• A matrix is called singular if its determinant equals 0
• A singular matrix is not invertible!
• A matrix is always singular if some row (column) is a
linear combination of some other rows (columns)
 a11
A   ka11
 a31
a12
ka12
a32
a13 
a11
a12
a13


ka13  A   k1a11  k2 a31 k1a12  k2 a32 k1a13  k2 a33 


a31
a32
a33
a33 
• A system of n linear equations in n unknowns has a
single non-zero solution only if its matrix is not
singular
19
Singular Systems of Linear
Equations
• Singular systems can be underdetermined:
2 x1  3 x2  5
4 x1  6 x2  10
(the second equation is coefficient-wise
proportional to the first one)
• or inconsistent: 2 x1  3x2  5
4 x1  6 x2  11
(the system matrix
 2 3
4 6


is singular)
Minor
• Let us have an nxm matrix A. Let us choose in
this matrix arbitrarily k rows and k columns
k  p; p  min  m, n  .
• The elements located in the intersection
points of these k rows and k columns form a
kxk matrix.
• The determinant of this kxk matrix is called a
minor of order k of the matrix A.
21
Minor
• Any nxm matrix has more than 1 minor of
order k<n for any k.
• A rank of an nxm matrix is the highest order of
a non-zero minor of this matrix
a11 a12 a13
D  a21 a22 a23
a31 a32 a33
D11 
a22 a23
D12 
a21 a23
a32 a33
a31 a33
 a22 a33  a32 a23 ;
 a21 a33  a31 a23 ; D13 
a21 a22
a31 a32
 a21 a32  a31 a22
22
Systems of Linear Equations
• Solve Ax=b, where A is an nn matrix and
b is an n1 column vector
• Usually not a good idea to compute x=A-1b
 Inefficient (because finding an inverse matrix is a
special and computationally very costly
procedure)
 Prone to round-off errors
• So some numerical methods are needed…
Systems of Linear Equations
• Solve Ax=b, where A is an nm matrix and
b is an m1 column vector
• A is called a matrix of the system of linear
equations
b 

•  A ...  is an augmented n(m+1) matrix of the



b  system
1
m
• Rouché-Kronecker-Capelli Theorem: A system of
linear equations has a solution if and only if the
rank of its coefficient matrix is equal to the rank
of its augmented matrix
Methods for Solving Systems of
Equations
• For small number of equations and unknowns (n
≤ 3) linear equations can be solved by simple
techniques such as “method of elimination by
hand”
• On the other hand, Linear algebra provides the
tools to solve such systems of linear equations
• Nowadays, easy access to computers makes the
solution of large sets of linear algebraic equations
possible and practical
25
Methods for Solving Systems of
Equations
Solving Systems of Equations
• There are many ways to solve a system of
linear equations:





Graphical method
Cramer’s rule
Method of elimination
Directed elimination
Other Numerical methods
For small n
For large n
26
Example: Graphical Method
• For two equations:
a11 x1  a12 x2  b1
a21 x1  a22 x2  b2
• Solve both equations for x2:
 a11 
b1
x2   
 x2  (slope1 )x1  intercept1
 x1 
a12
 a12 
 a21 
b2
x2   
 x2  (slope 2 )x1  intercept 2
 x1 
a22
 a22 
by Lale Yurttas, Texas A&M
University
Part 3
27
• Plot x2 vs. x1
for both
equations.
• The
intersection
of the lines
present the
solution
by Lale Yurttas, Texas A&M
University
28
Graphical Method
No solution
by Lale Yurttas, Texas A&M
University
Infinite solutions
Part 3
Ill-conditioned
(Slopes are too close)
29
Trivial Elimination Method
• For two equations:
a11 x1  a12 x2  b1
a21 x1  a22 x2  b2
• Let us solve both equations for x2:
30
Trivial Elimination Method
From the
1st
eq.
Then
And now
 a11 
 a21 
b1
b2
x2   
 
 x1 
 x1 
a12
a22
 a12 
 a22 
 a21 a11 
b1 b2



0
 x1 
a12 a22
 a22 a12 
From the 2nd eq.
 b1 b2   b2 b1 



 

a12 a22   a22 a12 

 x1  

 a21 a11   a21 a11 



 

a
a
a
a
 22
12 
 22
12 
x2 can now easily be found from either of the two equations
31
Cramer’s Rule
• Cramer’s rule expresses the solution of a
system Ax=b (where A is an nn matrix) of
linear equations in terms of ratios of
determinants
• The unknown xi is equal to the ratio of the
determinant created from the one of the
matrix A by replacement of its ith column with
a column of “free” coefficients to the
determinant of the matrix A.
32
Cramer’s Rule
• For example:
Ax  b
 a11 a12 a13 
 x1 
 b1 






A   a21 a22 a23  ; x   x2  ; b  x  b2  ; D  A  det  A 
 a31 a32 a33 
 x3 
 b3 
x1 
b1 a12 a13
a11 b1 a13
a11 a12 b1
b2 a22 a23
a21 b2 a23
a21 a22 b2
b3 a32 a33
a31 b3 a33
x2 
D
a31 a32 b3
x3 
D
D
33
Cramer’s Rule
• Advantage of the Cramer’s rule is its formal
algebraic simplicity
• Disadvantage of the Cramer’s rule is a
computational complexity of determinants
computation for large values of n
34