Eigenanalysis
• What’s Eigenanalysis?
• Fourier’s Eigenanalysis Model is a Replacement Process
• Powers and Fourier’s Model
• Differential Equations and Fourier’s Model
• Fourier’s Model Illustrated
• What is Eigenanalysis? What’s an Eigenvalue? What’s an Eigenvector?
• Data Conversion Example
• History of Fourier’s Model
• Determining Equations. How to Compute Eigenpairs.
• Independence of Eigenvectors.
What’s Eigenanalysis?
Matrix eigenanalysis is a computational theory for the matrix equation
y = Ax.
Fourier’s Eigenanalysis Model
For exposition purposes, we assume A is a 3 × 3 matrix.
(1)
x = c1v1 + c2v2 + c3v3 implies
y = Ax
= c1λ1v1 + c2λ2v2 + c3λ3v3.
Eigenanalysis Notation
The scale factors λ1 , λ2 , λ3 and independent vectors v1 , v2 , v3 depend only on A. Symbols c1 , c2 , c3 stand for arbitrary numbers. This implies variable x exhausts all possible
fixed vectors in R3 .
Fourier’s Model is a Replacement Process
A (c1v1 + c2v2 + c3v3) = c1λ1v1 + c2λ2v2 + c3λ3v3.
To compute Ax from x = c1 v1 + c2 v2 + c3 v3 , replace each vector vi by
its scaled version λi vi .
Fourier’s model is said to hold provided there exist scale factors and independent vectors
satisfying (1). Fourier’s model is known to fail for certain matrices A.
Powers and Fourier’s Model
Equation (1) applies to compute powers An of a matrix A using only the basic vector
space toolkit. To illustrate, only the vector toolkit for R3 is used in computing
A5x = x1λ51v1 + x2λ52v2 + x3λ53v3.
This calculation does not depend upon finding previous powers A2 , A3 , A4 as would be
the case by using matrix multiply.
Details for A3 (x)
Let x = x1 v1 + x2 v2 + x3 v3 . Then
A3(x) =
=
=
=
=
A2(A(x))
A2(x1λ1v1 + x2λ2v2 + x3λ3v3)
A(A(x1λ1v1 + x2λ2v2 + x3λ3v3))
A(x1λ21v1 + x2λ22v2 + x3λ23v3)
x1λ31v1 + x2λ32v2 + x3λ33v3
Differential Equations and Fourier’s Model
Systems of differential equations can be solved using Fourier’s model, giving a compact
and elegant formula for the general solution. An example:
x01 = x1 + 3x2,
x02 =
2x2 − x3,
0
x3 =
− 5x3.
The general solution is given by the formula [Fourier’s theorem, proved later]
 



 

3
1
1
x1
2t
−5t
t
 −2  ,
 x2  = c1 e  0  + c2 e  1  + c3 e
0
0
−14
x3
which is related to Fourier’s model by the symbolic formulas
x(0) = c1v1 + c2v2 + c3v3
undergoes replacements vi → eλi t vi to obtain
x(t) = c1eλ1tv1 + c2eλ2tv2 + c3eλ3tv3.
Fourier’s model illustrated
Let


1 3
0

0 2 −1 
A=
0 0 −5
λ1 = 
1,  λ2 = 
2,  λ3 = 
−5,

1
3
1
v1 =  0  , v2 =  1  , v3 =  −2  .
0
0
−14
Then Fourier’s model holds (details later) and
 




1
3
1





1
0
−2  implies
x = c1
+ c2
+ c3
0
0
−14
 
 


1
3
1
Ax = c1 (1)  0  + c2 (2)  1  + c3 (−5)  −2 
0
0
−14
Eigenanalysis might be called the method of simplifying coordinates. The nomenclature is justified, because
Fourier’s model computes y = Ax by scaling independent vectors v1 , v2 , v3 , which is a triad or coordinate
system.
What is Eigenanalysis?
The subject of eigenanalysis discovers a coordinate system v1 , v2 , v3 and scale
factors λ1 , λ2 , λ3 such that Fourier’s model holds. Fourier’s model simplifies the
matrix equation y = Ax, through the formula
A(c1v1 + c2v2 + c3v3) = c1λ1v1 + c2λ2v2 + c3λ3v3.
What’s an Eigenvalue?
It is a scale factor. An eigenvalue is also called a proper value or a hidden value or a
characteristic value. Symbols λ1 , λ2 , λ3 used in Fourier’s model are eigenvalues.
The eigenvalues of a model are scale factors. Think of them as a system of units
hidden in the matrix A.
What’s an Eigenvector?
Symbols v1 , v2 , v3 in Fourier’s model are called eigenvectors, or proper vectors or hidden
vectors or characteristic vectors. They are assumed independent.
The eigenvectors of a model are independent directions of application for the
scale factors (eigenvalues). Think of each eigenpair (λ, v) as a coordinate axis v
where the action of matrix A is to move λ units along v.
Data Conversion Example
Let x in R3 be a data set variable with coordinates x1 , x2 , x3 recorded respectively
in units of meters, millimeters and centimeters. We consider the problem of conversion
of the mixed-unit x-data into proper MKS units (meters-kilogram-second) y-data via the
equations
(2)
y1 = x1,
y2 = 0.001x2,
y3 = 0.01x3.
Equations (2) are a model for changing units. Scaling factors λ1 = 1, λ2 = 0.001,
λ3 = 0.01 are the eigenvalues of the model.
Data Conversion Example – Continued
Problem (2) can be represented as y = Ax, where the diagonal matrix A is given by


λ1 0 0
A =  0 λ2 0  ,
0 0 λ3
λ1 = 1, λ2 =
Fourier’s model for this matrix A is
  
 
 
1
1000
 
, λ3 =
1
100
.
 
 
1
0
0
1
0
0
A c1 0 + c2 1 + c3 0 = c1λ1 0 + c2λ2 1 + c3λ3 0
0
0
1
0
0
1
The eigenvectors v1 , v2 , v3 of the model are the columns of the identity matrix.
Summary
The eigenvalues of a model are scale factors, normally represented by symbols
λ1 ,
λ2,
λ3,
....
The eigenvectors of a model are independent directions of application for the
scale factors (eigenvalues). They are normally represented by symbols
v1,
v2,
v3,
....
History of Fourier’s Model
The subject of eigenanalysis was popularized by J. B. Fourier in his 1822 publication on
the theory of heat, Théorie analytique de la chaleur. His ideas can be summarized as
follows for the n × n matrix equation y = Ax.
The vector y = Ax is obtained from eigenvalues λ1, λ2, . . . , λn
and eigenvectors v1, v2, . . . , vn by replacing the eigenvectors by their
scaled versions λ1v1, λ2v2, . . . , λnvn:
x = c1v1
+ c2v2
+ · · · + cnvn implies
y = x1λ1v1 + x2λ2v2 + · · · + cnλnvn.
Determining Equations
The eigenvalues and eigenvectors are determined by homogeneous matrix–vector equations. In Fourier’s model
A(c1v1 + c2v2 + c3v3) = c1λ1v1 + c2λ2v2 + c3λ3v3
choose c1 = 1, c2 = c3 = 0. The equation reduces to Av1 = λ1 v1 . Similarly, taking
c1 = c2 = 0, c2 = 1 implies Av2 = λ2v2. Finally, taking c1 = c2 = 0, c3 = 1
implies Av3 = λ3 v3 . This proves the following fundamental result.
Theorem 1 (Determining Equations in Fourier’s Model)
Assume Fourier’s model holds. Then the eigenvalues and eigenvectors are determined by the three equations
Av1 = λ1v1,
Av2 = λ2v2,
Av3 = λ3v3.
Determining Equations and Linear Algebra
The three relations of the theorem can be distilled into one homogeneous matrix–vector
equation
Av = λv.
Write it as Ax − λx = 0, then replace λx by λIx to obtain the standard forma
(A − λI)v = 0,
v 6= 0.
Let B = A−λI . The equation Bv = 0 has a nonzero solution v if and only if there are
infinitely many solutions. Because the matrix is square, infinitely many solutions occurs
if and only if rref (B) has a row of zeros. Determinant theory gives a more concise
statement: det(B) = 0 if and only if Bv = 0 has infinitely many solutions. This
proves the following result.
a
Identity I is required to factor out the matrix A − λI. It is wrong to factor out A − λ, because A is 3 × 3 and λ
is 1 × 1, incompatible sizes for matrix addition.
College Algebra and Eigenanalysis
Theorem 2 (Characteristic Equation)
If Fourier’s model holds, then the eigenvalues λ1 , λ2 , λ3 are roots λ of the polynomial equation
det(A − λI) = 0.
The equation det(A − λI) = 0 is called the characteristic equation. The characteristic polynomial is the polynomial on the left, det(A − λI), normally obtained by
cofactor expansion or the triangular rule.
Eigenvectors and Frame Sequences
Theorem 3 (Finding Eigenvectors of A)
For each root λ of the characteristic equation, write the frame sequence for B =
A − λI with last frame rref (B), followed by solving for the general solution v
of the homogeneous equation Bv = 0. Solution v uses invented symbols t1 , t2 ,
. . . . The vector basis answers ∂t1 v, ∂t2 v, . . . are independent eigenvectors of A
paired to eigenvalue λ.
Proof: The equation Av = λv is equivalent to Bv = 0. Because det(B) = 0, then this system has infinitely
many solutions, which implies the frame sequence starting at B ends with rref (B) having at least one row of
zeros. The general solution then has one or more free variables which are assigned invented symbols t1 , t2 , . . . ,
and then the vector basis is obtained by from the corresponding list of partial derivatives. Each basis element
is a nonzero solution of Av = λv. By construction, the basis elements (eigenvectors for λ) are collectively
independent. The proof is complete.
Eigenpairs of a Matrix
Definition 1 (Eigenpair)
An eigenpair is an eigenvalue λ together with a matching eigenvector v 6= 0 satisfying
the equation Av = λv. The pairing implies that scale factor λ is applied to direction v.
An applied view of an eigenpair is a coordinate axis v and a unit system along this axis.
The action of the matrix A is to move λ units along this axis.
A 3 × 3 matrix A for which Fourier’s model holds has eigenvalues λ1 , λ2 , λ3 and
corresponding eigenvectors v1 , v2 , v3 . The eigenpairs of A are
(λ1, v1) , (λ2, v2) , (λ3, v3) .
Eigenvectors are Independent
Theorem 4 (Independence of Eigenvectors)
If (λ1 , v1 ) and (λ2 , v2 ) are two eigenpairs of A and λ1 6= λ2 , then v1 , v2 are
independent.
More generally, if (λ1 , v1 ), . . . , (λk , vk ) are eigenpairs of A corresponding to
distinct eigenvalues λ1 , . . . , λk , then v1 , . . . , vk are independent.
Theorem 5 (Distinct Eigenvalues)
If an n × n matrix A has n distinct eigenvalues, then its eigenpairs (λ1 , v1 ),
. . . , (λn , vn ) produce independent eigenvectors v1 , . . . , vn . Therefore, Fourier’s
model holds:
!
n
n
X
X
A
civi
i=1
=
ci(λivi).
i=1