Chapter 4 Numerical Methods in Linear Algebra
4-1 Eigenvalues and Eigenvectors of Matrices
Power method: Suppose λ1, λ2, …, λn are eigenvalues of A, and x1, x2, …, xn are
the corresponding eigenvectors; i.e., Axi=λixi, i=1, 2, …, n. Let v(0) be an arbitrary
vector, then
v ( 0)  c1 x1  c 2 x 2    c n x n , v (1)  Av ( 0)  c11 x1  c 2  2 x 2    c n  n x n
v ( 2)  Av (1)  A 2 v ( 0)  c112 x1  c 2 22 x 2    c n 2n x n

v ( n )  A n v ( 0)  c11n x1  c 2 n2 x 2    c n nn x n
If | 1 ||  2 |  |  n | , v
(n)
A v
n
(0)
n

 2 

  c1 x1  c2 x2      cn xn  n

 1 
 1
n
1



n



As n→∞, v ( n )  A n v ( 0)  1n c1 x1 is a multiple of x1.
With an appropriate normalization factor c1, this method can be used find the
dominant eigenvalue λ1 and its corresponding eigenvector.
4 1 0 
Eg. Find the dominant eigenvalue of A= 0 2 1  and its corresponding


0 0 1
eigenvector.
1
(Sol.) Select 1 , Ax 
1
1  5 
 1 
 1  4.6
 1 










A1   3   5 0.6  , A 0.6    1   4.60.2174
1  1
 0.2
 0.2 0.2
0.0435
 1   4.2134 
 1 
A0.2174   0.4783   4.2134 0.1134 

 



0.0435  0.0435
 0.0103
 1  4.1134
 1 




A 0.1134   0.2165  4.11340.0526
 0.0103 0.0103
0.0025
 1   4.0526 
 1

1 






A0.0526   0.1077   4.0526 0.0266  , ∴ λ→4 and x→ 0 
0.0025  0.0025
 0.0006
0 
～13～
A C++ Program (developed by K. –Y. Lee) of computing the multiplication of a
n×n matrix and a n-dimensional vector is listed as follows.
#include <stdio.h>
#include <math.h>
main()
{
int i,j,k,nn; float a[100][100],x[100],y[100];
printf("dimension of matrix=?\n");
scanf("%d",&nn);
for(i=0;i<=nn-1;i++)
{printf("x[i]=? %d \n",i+1); scanf("%f",&x[i]);}
for (i=0;i<=nn-1;i++)
{ for (j=0;j<=nn-1;j++)
{printf("a[i][j]=? %d %d \n",i+1,j+1); scanf("%f",&a[i][j]);}
}
for (i=0;i<=nn-1;i++)
{ y[i]=0.;
for (j=0;j<=nn-1;j++)
{y[i]=y[i]+a[i][j]*x[j];}
printf("y[i]= %f %d \n",y[i],i+1);}
}
A C++ Program (developed by K. –Y. Lee) of computing the multiplication of
two n×n matrices C=AB is listed as follows.
#include <stdio.h>
#include <math.h>
main()
{
int i,j,k,nn; float a[100][100],b[100][100],c[100][100];
～14～
printf("dimension of matrix(<=100)=?\n");
scanf("%d",&nn);
for (i=0;i<=nn-1;i++)
{ for (j=0;j<=nn-1;j++)
{printf("a[i][j]=? b[i][j]=?%d %d \n",i+1,j+1);
scanf("%f %f",&a[i][j],&b[i][j]);}
}
for (i=0;i<=nn-1;i++)
{
for (j=0;j<=nn-1;j++)
{
c[i][j]=0.;
for (k=0;k<=nn-1;k++)
{c[i][j]= c[i][j]+a[i][k]*b[k][j];}
printf("c[i][j]= %f %d %d \n",c[i][j],i+1,j+1);
}
}
}
～15～
Inverse Power method:
1. To find the eigenvalue of the least magnitude, we can apply the power method
to A-1 to obtain its eigenvalue of the largest magnitude λM. Then 1/λM is the
eigenvalue of the least magnitude for A. (Ax=λx  A-1x=λ-1x)
2. To find the eigenvalue near q, i.e., λ-q  0, (A-qI)x=(λ-q)x, λ-q is the eigenvalue
of the least magnitude, then apply power method to find the dominant eigenvalue
of (A-qI)-1, say  ( AqI ) 1 . Then q  1 ( AqI )1 is the eigenvalue of A near q. One
can guess x(0) to obtain q by q 
x t ( 0) Ax ( 0)
.
x t ( 0) x ( 0)
 4 14 0
Eg. For A=   5 13 0 , find its eigenvalue near 6.


  1 0 2
(Sol.) Guess x
(0)
1
 1 , q 
1
1
[111] A1
1
1
[111]1
1
 6.3333
 ( A  qI ) 1 x (0)      6.000005  x  [1 0.714286  0.2499 ]
 4 1 1 
Eg. Find all eigenvalues of A=  1
1
1  .
 2 0  6
(Sol.)
1
4
  0.5 
x ( 0)  1  Ax ( 0)   3   8 0.375

 


1
 8
 1 
  0.5 
 0.125 
 0.1157
,
 A 0.375  5 0.025    Ax(  )  5.76849 0.1306
 1 
 1 
 1

A1  
 1  5.76849
 6  6  2  3 13 3 13 1 13 
1 
4  22  3   2 13 11 13 3 26 
26 
 2
2
5    1 13 1 13  5 26
1
 0.4121 
1
,
x ( 0)  1    A1 x(  )  0.7689 1    2  0.7689  1.29923
1
 0.1129
 1 1 1 
Suppose it has an eigenvalue near q=3, A  qI   1  2 1 


 2 0  9
1
1


,
x( 0)  1    ( A  qI )1 x(  )  2.13174 0.31963 
1
 0.21121
～16～
∴ 3  3 
1
 3.4691
2.13174
In Matlab language, we can use the following instructions to obtain the eigenvalues
of a matrix:
>>A=[1,4,3;4,9,6;7,1,9]
A=
1
4
3
4
9
6
7
1
9
>>eig(A)
ans =
-1.0205
5.1344
14.8861
And we can use the following instructions to find out the ij-entry, the mth row, and the
nth column of a matrix:
>>A=[0,1,2;3,4,5]
A=
0
1
3
4
>>A(2,1)
ans =
3
>>A(2,3)
ans =
5
>>ROW1=A(1,:)
ROW1 =
0
1
>>COL2=A(:,2)
COL2 =
1
4
2
5
2
～17～
4-2 Matrix Inversions
1  1 2
Eg. A= 3 0 1 , A-1=?


1 0 2
1  1 2 1 0 0
1  1 2 1 0 0




(Sol.) 3 0 1 0 1 0  0 3  5  3 1 0
1 0 2 0 0 1
0 1
0  1 0 1

2
1  1 2 1 0 0 
1  1 0 1
5


 0 1
0  1 0 1   0 1 0  1 0

1
0 0  5 0 1  3
0 0 1 0 
5

6
 
5
1 
3 

5 
1 0 0 0
2 5  1 5
 0 2 5  1 5


1
 0 1 0  1
0
1  , A   1
0
1 

0 0 1 0  1 5 3 5 
 0  1 5 3 5 
In Matlab language, we can use the following instructions to obtain the inverse of a
matrix:
>>A=[2,1;4,3]
A=
2
4
>>inv(A)
ans =
1.5000
-2.0000
1
3
-0.5000
1.0000
～18～
4-3 Determinants of Matrices
A (
    triangular matrix  det(A)=Product of the diagonal elements
Gaussian E limination)
(Gaussian Elimination: Add a multiple of a column/row to another column/row.)
1
2
Eg. A= 
3

1
4 2
2 0
0 1
1
2
(Sol.) 
3

1
4 2
2 0
0 1
2
2
2
2


 , det(A)=?


 3
3
4
2
4
2 3 

1

0  6
4  2


0  12 5  7



 3
4  6
0  2
3
4
2
3  1 4  2 3 
1 4  2
0  6 4
 2  0  6 4  2



, det(A)=1．(-6)．(-3)．(-8)=-144
0 0  3
 3  0 0  3  3 

 

0  8
0 0 8 3  16 3 0 0
0
3
1 1
 2 1 1 1 
 , det(A)=?
Eg. A= 
 1 2 3  1


 3 1 1 2 
0
3
0
3 
0
3 
1 1
1 1
1 1
0  1  1  5 
0  1  1  5 
0  1  1  5 



(Sol.) A  
0 3
0 0 0  13
0 0
3
2
3
13 






0  13
0  4  1  7 
0 0 3 13 
0 0
det(A)=-[1．(-1)．3．(-13)]=-39
In Matlab language, we can use the following instructions to obtain the determinant
of a matrix:
>>A=[1,3,0;-1,5,2;1,2,1];
det(A)
ans =
10
～19～
4-4 Factorizations of Matrices
LU Factorization: A=LU, where L is a lower triangular matrix and U is an upper
triangular matrix. There are many ways.
Algorithm 1
 a11
a
A   21
a31

a 41
a12
a 22
a13
a 23
a32
a 42
a33
a 43
a14    11 0
a 24   21  22

a34   31  32
 
a 44   41  42
0
0
 33
 43
0  1 u12
0  0 1
0  0 0

 44  0 0
u13
u 23
1
0
u14 
u 24 
u 34 

1 
 i1  a i1 , 1  i  n

u1 j  a1 j  11  a1 j a11 , 1  j  n
j 1

  ij  aij    ik ukj , j  i
k 1

i 1



uij  aij    ik ukj   ii , i  j

k 1


3  1 2 
Eg. A= 1 2
3  = LU, L=? U=?
2  2  1
(Sol.)  11  3 ,  21  1 ,  31  2 , u12   1 3 , u13  2 3
4
 1
 1 7
 22  2  1      ,  32  2  2      
3
 3 3
 3
2  7 
2  4

u 23  3  1      1 ,  33  1  2       1  1
3  3
3  3 

0
0
1  1 3 2 3
3
L  1 7 3
0  , U  0
1
1 
0
2  4 3  1
0
1 
Algorithm 2
0
1

1
L   21
 31  32

 41  42
0
0
1
 43
0
u11 u12

0 u
0
22
, U 

0
0
0


1
0
0
u13
u 23
u 33
0
u14 
u 24 
u 34 

u 44 
 u1 j  a1 j , 1  j  n ,  i1  ai1 u11  ai1 a11 , 1  i  n
j 1


u ij  a ij    ik u kj , j  i ,  ij  aij    ik u kj  u jj , i  j
k 1
k 1


i 1
～20～
In Matlab language, we can use the following instructions to obtain the LU
factorization of a matrix. However, the answer may be incorrect.
>>A=[5,2,3;4,2,5;8,7,2]
A=
5
2
3
4
2
5
8
7
2
>>[L,U]=lu(A)
L=
0.6250
1.0000
0.5000
0.6316
1.0000
0
0
1.0000
0
U=
8.0000
0
0
7.0000
-2.3750
0
2.0000
1.7500
2.8947
Note: L is not an upper triangular matrix in this case.
～21～
QR Factorization: A=QR, where Q is an orthogonal matrix and R is an upper
triangular matrix.
Let the columns of A be A1, A2, A3, …, An. And then
A1 | v1 | w1 , A2  A2 , w1  w1  | v2 | w2 , A3  A3 , w1  w1   A3 , w2  w2  | v3 | w3
 A1t  
 t 

A
t
A  2 
   
 t 
 An  
| v1 |
A2 , w1 

0
0
0
| v2 |
0
 
A3 , w1   A3 , w2  | v3 |  



 
( Rt )
0
 
 

vn 
 w1t 
 t
 w2   A  QR
  
 t
 wn 
(Qt )
 1 2  1
Eg. Find the QR factorization of the matrix A=  1 1 2  .


 1 0 2 
(Sol.)
 1
 2
 1
,
,
A1   1 
A2  1 A3   2 
 1 
0
 2 
 1 3 
 1 3 
 1




A1   1   3  1 3   3w1  v1  3 , w1   1 3 
1 3 
1 3 
 1 




2  2  1 3    1 3 

 

| v2 | w2  A2   A2 , w1  w1  1   1   1 3    1 3 
   

0  0  1 3    1 3 
 1 3  2  1 3 5 3
 2
   
 1  


  
 1    
   1 3   1   1 3   4 3
3 

   
0
  
 1 3   0   1 3  1 3 
5
42 

4
3 
1
5
42 


42
42  | v 2 | w2  v 2 
, w2  4
3
1
42 

42 

42 
42 
| v3 | w3  A3   A3 , w1  w1   A3 , w2  w2
 1   1  1 3    1 3    1 5

 


  2     2    1 3    1 3     2   4



 2    2   1 3    1 3    2  1

 
 


 1 3 
5
 1

5 
5 
  2  
1
3



4
3
42 

 2 
1
3


1
 v3 
1
14
,
42   5
 
42   4

42   1

42 

42 
42 
 1  5 3 25 42
 1 14 
42 
 
 

   
42    2    5 3   20 42   2 14 
42   2   5 3   5 42   3 14 
 1 14 


w3   2 14 ,
 3 14 


 1 3 5

A  QR   1 3 4
1 3 1

～22～
42
42
42
 1 14 

1 
 2 14  | v3 | w3
14 

 3 14 
1 14   3  1 3 5 3 


 2 14   0
42 3 5 42 
3 14   0
0
1 14 
4-5 Gaussian Elimination
 x1  x2  2 x3  x4  8
2 x  2 x  3x  3x  20
 1
2
3
4
Eg. Solve 
.
x

x

x


2
1
2
3

 x1  x2  4 x3  3x4  4
1  1

2 2
(Sol.) 
1 1

1  1
1  1 2  1  8
2 1  8 



3  3  20
0 0  1  1  4


0 2  1 1 6 
1 0 2



4 3 4 
2 4 12 
0 0
1  1 2  1  8 1  1 2  1  8

 

0 2  1 1 6  0 2  1 1 6 



0 0  1  1  4  0 0  1  1  4 

 

0 0 2 4 12  0 0 0 2 4 
x4  4 2  2 , x3  [4  (1)  2] (1)  2
x2  [6  2  (1)  2] 2  3
x1  [8  (1)  2  2  2  (1)  3 1  7
0.003x1  59.14 x2  59.17
Eg. Solve 
.
5.291x1  6.13x2  46.78
 x  10 5.291
 1763.66  1764
The exact solution is  1
,
0
.
003
x

1
 2
 (6.13  59.14 1764) x2  46.78  1764  59.17  104300 x2  104400
 x2  1.001  1  x1  [59.17  59.14 1.001] 0.003  10  inaccurate
( 0)
( 0)
Pivot procedure: max{| a11
|, | a21
|}  max{| 0.003 |, | 5.291 |}  5.291 , E2  E1
5.291x1  6.130 x2  46.78
，

0.003x1  59.74 x2  59.17
0.003
 0.0005670  (59.74  (6.13)  0.000567) x 2
5.291
 59.17  0.000567  46.78
 59.14x2  59.14  x2  1  x1  [46.78  1 6.13] 5.291  10
～23～
Eg. An example of FORTRAN Program of solving a system of linear equations
 x  3 y  5z  2

2 x  4 y  6 z  1 by Gaussian elimination method
 x  2y  z  3

C+++++++++++++++++++ NUM=3 IN THIS EXAMPLE ++++++++++++++++++++++++
C+++ COEFFICIENTS: C(1:NUM,1:NUM), CONSTANT TERMS:C(1:NUM,NUM+1) +++
C+++++++++++++ DIMENSION OF INPUT: C(NUM+1,NUM+1) +++++++++++++++++++
C+++++++++++++++ DIMENSION OF OUTPUT: S(NUM) +++++++++++++++++++++++
DIMENSION C(4,4),S(3)
C(1,1)=1.
C(1,2)=3.
C(1,3)=5.
C(1,4)=2.
C(2,1)=2.
C(2,2)=4.
C(2,3)=6.
C(2,4)=1.
C(3,1)=1.
C(3,2)=2.
C(3,3)=1.
C(3,4)=3.
NUM=3
CALL SOLMATR (NUM,C,S)
WRITE (*,*) S(1),S(2),S(3)
STOP
END
230
250
220
270
310
290
SUBROUTINE SOLMATR (NUM,C,S)
DIMENSION C(4,4),S(3)
Z=0.
DO 200 I=1, NUM
IF (C(I,I).NE.Z) GOTO 220
DO 230 J=I+1,NUM
IF (C(J,I).NE.Z) GOTO 250
CONTINUE
CALL PIVOT(I,J,NUM,C)
DIV=C(I,I)
DO 270 J= 1,NUM+1
C(I,J)=C(I,J)/DIV
CONTINUE
DO 290 II=1,NUM
IF (II.EQ.I) GOTO 290
RR= C(II,I)
DO 310 K=1, NUM+1
C(II,K)=C(II,K)-RR*C(I,K)
CONTINUE
CONTINUE
～24～
200
330
400
CONTINUE
DO 330 I=1, NUM
S(I)=C(I,NUM+1)
CONTINUE
RETURN
END
SUBROUTINE PIVOT(I,J,NUM,C)
DIMENSION C(4,4)
DO 400 K=1, NUM+1
TRAN=C(I,K)
C(I,K)=C(J,K)
C(J,K)=TRAN
CONTINUE
RETURN
END
In Matlab language, we can use the following instructions to solve a linear system of
 x  3 y  5z  2

equations: 2 x  4 y  6 z  1
 x  2y  z  3

>>A=[1,3,5;2,4,6;1,2,1];
>>B=[2,1,3]'; % [2,1,3]’ is the transpose of [2,1,3]
>>rref([A,B])
ans =
1.0000
0
0
0
1.0000
0
0
0
1.0000
-3.7500
4.0000
-1.2500
～25～
4-6 Iterative Methods for Solving Systems of Linear Equations
Jacobi iterative method:
10 x1  x2  2 x3  6
 x  11x  x  3x  25
 1
2
3
4
Eg. Solve 
.
2 x1  x2  10 x3  x4  11
3x2  x3  8 x4  15
 ( k ) 1 ( k 1) 1 ( k 1) 3
 x1  10 x 2  5 x3  5

 x ( k )  1 x ( k 1)  1 x ( k 1)  3 x ( k 1)  25
1
3
4
 2
11
11
11
11
(Sol.) 
1
1
1
 x ( k )   x ( k 1)  x ( k 1)  x ( k 1)  11
1
2
4
 3
5
10
10
10

3
1
15
 x 4( k )   x 2( k 1)  x3( k 1) 
8
8
8

(0)
(0)
(0)
(0)
(x1 ,x2 , x3 ,x4 )=(0,0,0,0)  
 (x1(10),x2(10), x3(10),x4(10))=(1.0001,1.9998,-0.9998,0.9998)
Gauss-Seidel iterative method:
Eg. For the same systems as above, solve it by the Gauss-Seidel method.
 ( k ) 1 ( k 1) 1 ( k 1) 3
 x1  10 x 2  5 x3  5

 x ( k )  1 x ( k )  1 x ( k 1)  3 x ( k 1)  25
1
3
4
 2
11
11
11
11
(Sol) 
 x ( k )   1 x ( k )  1 x ( k )  1 x ( k 1)  11
1
2
4
 3
5
10
10
10

3
1
15
 x 4( k )   x 2( k )  x3( k ) 
8
8
8

(x1(0),x2(0), x3(0),x4(0))=(0,0,0,0)  
 (x1(5),x2(5), x3(5),x4(5))=(1.000,2.000,-1.000,1.000)
～26～