MODULE 6B | Iterative Methods in Linear Algebra
MODULE 6B
Iterative
Methods in
Linear Algebra
Eigenvalue
Algorithms
Power Method | Shifted-Inverse
Power Method
CE 25
Mathematical Methods in Civil Engineering II
22
CE 25 – Mathematical Methods in Civil Engineering II
MODULE 6B | Iterative Methods in Linear Algebra
Eigenvalue Algorithms
Matrix Eigenvalue Problem
▪ Matrix eigenvalue problems are concerned with vector
equations of the form:
𝑨 {𝒙} = 𝜆{𝒙}
EIGENVECTOR
(characteristic vector)
EIGENVALUE
(characteristic value)
𝑨: the given SQUARE matrix (in this case, all matrices to be
handled are square matrices)
CE 25: Mathematical Methods in CE II
23
Examples of eigenvalue problem encountered in civil engineering are: (a) buckling in
Mechanics of Materials, and (b) determining mode shapes in Structural Dynamics
(pertinent to vibration in structures)
23
CE 25 – Mathematical Methods in Civil Engineering II
MODULE 6B | Iterative Methods in Linear Algebra
EIGENVALUE ALGORITHMS
Matrix Eigenvalue Problem
RECALL
Theorem on Eigenvalues and Eigenvectors:
Theorem 1. The eigenvalues of a square matrix A are the roots of the
characteristic equation of A. Hence, an n x n matrix has at least one and
at most n numerically different eigenvalues.
CE 24: Mathematical Methods in CE II
24
Previously on CE 24
24
CE 25 – Mathematical Methods in Civil Engineering II
MODULE 6B | Iterative Methods in Linear Algebra
EIGENVALUE ALGORITHMS
Matrix Eigenvalue Problem
Analytical solution determines
eigenvalues 𝜆 first as the roots
of p 𝜆 = det 𝑨 − 𝜆𝑰 = 0.
The eigenvectors 𝒗 are then
computed for each eigenvalue
be getting the solution of
𝑨 − 𝜆𝑰 𝒗 = 0.
𝑨𝒙 = 𝜆𝒙
𝑨𝒙 = 𝜆𝑰𝒙
𝑨𝒙 − 𝜆𝑰𝒙 = 𝟎
𝑨 − 𝜆𝑰 𝒙 = 𝟎
CE 24: Mathematical Methods in CE II
25
Previously on CE 24
25
CE 25 – Mathematical Methods in Civil Engineering II
MODULE 6B | Iterative Methods in Linear Algebra
Eigenvalue Algorithms
Dominant Eigenpair
Assume that the 𝑛 × 𝑛 matrix 𝑨 has n distinct eigenvalues
𝜆1 , 𝜆2 , … , 𝜆𝑛 and they are ordered in decreasing magnitude; that is,
𝜆1 > 𝜆2 > 𝜆3 > ⋯ > |𝜆𝑛 |
Eigenvalue 𝜆1 is called the dominant eigenvalue.
Eigenvector 𝒗𝟏 corresponding to 𝜆1 is called a dominant
eigenvector.
CE 25: Mathematical Methods in CE II
26
The eigenvalue and eigenvector together is called an eigenpair.
26
CE 25 – Mathematical Methods in Civil Engineering II
MODULE 6B | Iterative Methods in Linear Algebra
Eigenvalue Algorithms
Power Method
If 𝑥0 is chosen appropriately, then the sequences 𝒙𝒌 = 𝑥1𝑘 𝑥2𝑘 … 𝑥𝑛𝑘
𝑇
and
{𝑐𝑘 } can be generated recursively by
𝒚𝒌 = 𝑨 𝒙𝒌
and
1
{𝒙𝒌+𝟏 } =
{𝒚𝒌 }
𝑐𝑘+1
where
𝒌
𝑐𝑘+1 = max 𝒚𝒊
will converge to the dominant eigenvector 𝒗𝟏 and dominant eigenvalue 𝜆1 ,
respectively.
CE 25: Mathematical Methods in CE II
27
Here, a variable in bold is a matrix.
27
CE 25 – Mathematical Methods in Civil Engineering II
MODULE 6B | Iterative Methods in Linear Algebra
Eigenvalue Algorithms
Limitation of Power Method
▪ The power method only determines the dominant eigenpair.
▪ Power method can be used to determine eigenpairs of matrices
with distinct eigenvalues. If two or more eigenvalues of a matrix
𝑨 have equal magnitude, the method may not converge.
CE 25: Mathematical Methods in CE II
28
These limitations will be addressed by the other method that we will be discuss in this
module.
28
CE 25 – Mathematical Methods in Civil Engineering II
MODULE 6B | Iterative Methods in Linear Algebra
Eigenvalue Algorithm
EXAMPLE 6B-3
Use the power method to find the dominant eigenpair for the
matrix
0 11 −5
𝑨 = −2 17 −7
−4 26 −10
CE 25: Mathematical Methods in CE II
29
29
CE 25 – Mathematical Methods in Civil Engineering II
MODULE 6B | Iterative Methods in Linear Algebra
Eigenvalue Algorithm
EXAMPLE 6B-3
Use the power method to find the dominant eigenpair for the matrix
0 11 −5
𝑨 = −2 17 −7
−4 26 −10
𝒚 𝒌 = 𝑨 𝒙𝒌
1
{𝒙𝒌+𝟏 } =
{𝒚𝒌 }
𝑐𝑘+1
𝑐𝑘+1 = max 𝒚𝒊
𝒌
FIRST ITERATION
Step 1. Select initial guess, say 𝒙𝟎 = 1
Step 2. Compute 𝑦0 .
0
𝒚𝟎 = −2
−4
11
17
26
1
1
𝑇
1ൗ
−5 1
6
2
−7 1 = 8 = 12 2ൗ
3
12
−10 1
1
𝒙𝟏
Step 3. Identify 𝑐1 . For this case, 𝑐1 = 12.
Step 4. Compute 𝒙𝟏 .
𝒙𝟏 =
1
12
1ൗ
6
2
8 = 2ൗ
3
12
1
𝑐1
CE 25: Mathematical Methods in CE II
30
As a numerical method, we also give an initial guess to {𝑥}.
One iteration consists of:
(a) Computing for {𝑦𝑘 }
(b) Determining absolute max element in {𝑦𝑘 } → this will be 𝑐𝑘+1
(c) Dividing {𝑦𝑘 } by 𝑐𝑘+1 to get {𝑥𝑘+1 }
30
CE 25 – Mathematical Methods in Civil Engineering II
MODULE 6B | Iterative Methods in Linear Algebra
Eigenvalue Algorithm
EXAMPLE 6B-3
Use the power method to find the dominant eigenpair for the matrix
0 11 −5
𝑨 = −2 17 −7
−4 26 −10
FIRST ITERATION
SECOND ITERATION
Step 1. Compute 𝑦0 .
1ൗ
2
𝑐1 {𝒙𝟏 } = 12 2ൗ
3
1
𝒚𝟏
𝑁
0
= −2
−4
11
17
26
−5
−7
−10
7ൗ
1ൗ
3
2
16
10
ൗ3 = 𝑐2 𝒙𝟐 =
2ൗ =
3
3
16ൗ
1
3
{𝑥𝑛 }𝑇
𝑐𝑛
0
Steps 2 and 3. Identify 𝑐2 and compute 𝑥2 .
𝜖
1.00000
1.00000
1.00000
1
12.0000
0.50000
0.66667
1.00000
2
5.33333
0.43750
0.62500
1.00000
6.66667
7ൗ
16
5ൗ
8
1
Step 4. Compute
error. Terminate
accordingly.
𝜖𝑛 = |𝑐𝑛 − 𝑐𝑛−1 |
CE 25: Mathematical Methods in CE II
31
Error can only be computed after the second iteration because we are comparing the 𝑐𝑘
values.
31
CE 25 – Mathematical Methods in Civil Engineering II
MODULE 6B | Iterative Methods in Linear Algebra
Eigenvalue Algorithm
EXAMPLE 6B-3
Use the power method to find the dominant eigenpair for the matrix
0 11 −5
𝑨 = −2 17 −7
−4 26 −10
1
2
𝑐1 {𝒙𝟏 } = 12 2
3
1
9
𝑐3 {𝒙𝟑 } =
2
5
12
11
18
1
𝑐2 {𝒙𝟐 } =
16
3
38
𝑐4 {𝒙𝟒 } =
9
7
16
5
8
1
𝑁
0
31
76
23
38
1
𝜖
1.00000
1.00000
1.00000
1
12.0000
0.50000
0.66667
1.00000
2
5.33333
0.43750
0.62500
1.00000
6.66667
3
4.50000
0.41667
0.61111
1.00000
0.83333
4
4.22222
0.40789
0.60526
1.00000
0.27778
5
4.10526
0.40385
0.60256
1.00000
0.11696
6
4.05128
0.40190
0.60127
1.00000
0.05398
4.00000
0.40000
0.60000
1.00000
0.00000
…
20
CE 25: Mathematical Methods in CE II
32
(𝑥𝑛 )𝑇
𝑐𝑛
32
CE 25 – Mathematical Methods in Civil Engineering II
MODULE 6B | Iterative Methods in Linear Algebra
Eigenvalue Algorithm
EXAMPLE 6B-3
Use the power method to find the dominant eigenpair for the matrix
0 11 −5
𝑨 = −2 17 −7
−4 26 −10
The dominant eigenpair of
matrix A is
𝑁
0
0.4
𝜆1 = 4, {𝒗𝟏 } = 0.6
1
(𝑥𝑛 )𝑇
𝑐𝑛
𝜖
1.00000
1.00000
1.00000
1
12.0000
0.50000
0.66667
1.00000
2
5.33333
0.43750
0.62500
1.00000
6.66667
3
4.50000
0.41667
0.61111
1.00000
0.83333
4
4.22222
0.40789
0.60526
1.00000
0.27778
5
4.10526
0.40385
0.60256
1.00000
0.11696
6
4.05128
0.40190
0.60127
1.00000
0.05398
4.00000
0.40000
0.60000
1.00000
0.00000
…
20
CE 25: Mathematical Methods in CE II
33
After 20 iterations, we arrive at this dominant eigenpair.
33
CE 25 – Mathematical Methods in Civil Engineering II