Basic Topics
Objectives
Get you comfortable and familiar with tools of linear algebra and other applications.
Will be given 12-14 exercise sets, will choose the best 10 (must serve at least 10)
80% of the grade is based on homework!
20% is based on the exam
Topics Planned to be Covered
-
Vector Spaces over ℝ and ℂ.
Basic definitions connected with vector spaces
Matrices, Block of matrices
Gaussian elimination
Conservation of dimensions
Eigen values and Eigen vectors
Determinants
Jordan forms
Difference and differential equations
Normed linear spaces
Inner product spaces (orthogonality)
Symmetric, Hermitian, Normal matrices
Singular Value Decompositions (SVD)
Vecor Valued Functions of many variables
Fixed point theorems
Implicit function theorem
Extremal problems with constraints
General Idea of Linear Algebra
𝑎11 𝑥1 + 𝑎12 𝑥2 + ⋯ + 𝑎1𝑞 𝑥𝑞 = 𝑏1
𝑎21 𝑥1 + 𝑎22 𝑥2 + ⋯ + 𝑎2𝑞 𝑥𝑞 = 𝑏2
⋮
𝑎𝑝1 𝑥1 + 𝑎𝑝2 𝑥2 + ⋯ + 𝑎𝑝𝑞 𝑥𝑞 = 𝑏𝑝
Given 𝑎𝑖𝑗 , 𝑖 = 1, … , 𝑝, 𝑗 = 1, … , 𝑞; 𝑏1 , … , 𝑏𝑝
Find 𝑥1 , … , 𝑥𝑞
A choice of 𝑥1 , … , 𝑥𝑞 that satisfies these equations is called a solution.
1) When can you guarantee at least one solution?
2) When can you guarantee at most one solution?
3) How can you find solutions?
4) Are there approximate solutions?
Short notation:
𝐴𝑥 = 𝑏
𝑎11 … 𝑎1𝑞
⋮ ]
𝐴=[ ⋮
𝑎𝑝1 … 𝑎𝑝𝑞
𝑥1
𝑥=[⋮ ]
𝑥𝑞
A vector space 𝑉 over ℝ is a collection of objects called vectors on which two operations are
defined:
1) Vector addition – 𝑢 ∈ 𝑉, 𝑣 ∈ 𝑉 𝑢 + 𝑣 ∈ 𝑉
2) Scalar multiplication – 𝛼 ∈ ℝ, 𝑢 ∈ 𝑉 𝛼𝑢 ∈ 𝑉
Subjects to following constraints:
1) 𝑢 + 𝑣 = 𝑣 + 𝑢
2) (𝑢 + 𝑣) + 𝑤 = 𝑢 + (𝑣 + 𝑤)
3) There is a zero vector, 0 s.t. 𝑢 + 0 = 𝑢, 0 + 𝑢 = 𝑢
4) 𝑢 ∈ 𝑈, then there is a 𝑤 s.t. 𝑢 + 𝑤 = 0
5) 1 ∈ ℝ, 1𝑢 = 𝑢
6) 𝛼, 𝛽 ∈ ℝ, then 𝛼(𝛽𝑢) = 𝛼𝛽𝑢
7) 𝛼, 𝛽 ∈ ℝ, then (𝛼 + 𝛽)𝑢 = 𝛼𝑢 + 𝛽𝑢
8) 𝛼(𝑢 + 𝑣) = 𝛼𝑢 + 𝛼𝑣
Vector Space Example
ℝ𝑘
𝑢1
𝑢2
[⋮]
𝑢𝑘
𝑢𝑖 ∈ ℝ
𝑢1
𝑣1
𝑢1 + 𝑣1
⋮
⋮
[ ]+[ ]= [ ⋮
]
𝑢𝑘
𝑣𝑘
𝑢𝑘 + 𝑣𝑘
ℝ2𝑋3
𝑎11 𝑎12
𝐴 = [𝑎
21 𝑎22
𝑎13
𝑎23 ]
𝐵=[
𝑏11
𝑏21
𝑏12
𝑏22
𝑏13
]
𝑏23
Claim: if 𝑉 is a vector space over ℝ, then there is exactly one zero element.
Proof: Suppose that 0 and 0̃ are both zero elements.→
0+𝑢 =𝑢+0=𝑢
0̃ + 𝑢 = 𝑢 + 0̃ = 𝑢
Lets set 𝑢 = 0̃
0 + 0̃ = 0̃
Lets set 𝑣 = 0
0̃ + 0 = 0
Therefore 0 = 0̃
Claim: If 𝑢 ∈ 𝑉, there is exactly one additive inverse
Suppose 𝑤 and 𝑤
̃ are both additive inverses of 𝑢.
𝑢+𝑤 =0
𝑢+𝑤
̃ =0
𝑤 =𝑤+0
Lets mix things up:
𝑤 = 𝑤 + 0 = 𝑤 + (𝑢 + 𝑤
̃) = (𝑤 + 𝑢) + 𝑤
̃ =0+𝑤
̃ →𝑤=𝑤
̃
We denote the additive inverse of 𝑢 by – 𝑢.
𝑢 + (- − -𝑢) = 0
Since this is clumsy to write we abbreviate this as 𝑢 − 𝑢 = 0
Note: We can replace ℝ by ℂ!
Let 𝑉 be a vector space over 𝔽 Means we can replace 𝔽 by either ℝ or ℂ in all that follows this
statement.
Definitions
Let 𝑉 be a vector space over 𝔽.
A subset 𝑈 of 𝑉 is called a subspace of 𝑉 if:
1) 𝑢, 𝑤 ∈ 𝑈 → 𝑢 + 𝑤 ∈ 𝑈
2) 𝛼 ∈ 𝔽, 𝑢 ∈ 𝑈 → 𝛼𝑢 ∈ 𝑈
If (1) and (2) hold, then 𝑈 is a vector space of 𝑉 over 𝔽.
Example
𝑎
𝑉 = ℝ2 = {[ ] |𝑤𝑖𝑡ℎ 𝑎, 𝑐 ∈ ℝ}
𝑏
𝑎
𝑈 = {[ ] |𝑎 ∈ ℝ}
0
7
1
8
[ ]+[ ]= [ ]
0
0
0
𝑎
𝛼𝑎
𝛼𝑎
𝛼[ ] = [ ] = [ ]
0
𝛼0
0
As we can see, this is a subspace…
However…
𝑎
Let 𝑈 = {[ ] |𝑎 ∈ ℝ}
1
𝑎
𝑏
𝑎+𝑏
[ ]+[ ] = [
]
1
1
2
So this is not a subspace!
Span
A span {𝑣1 , … , 𝑣𝑘 } (𝑣1 , … , 𝑣𝑘 ∈ 𝑉)
𝑘
= {∑ 𝛼𝑖 : 𝛼1 , … , 𝛼𝑘 ∈ 𝔽}
𝑖=1
2𝑣1 + 7𝑣2 + 3𝑣3
2𝑣1 + 0𝑣2 + 0𝑣3
Check: span {𝑣1 , … , 𝑣𝑘 } is a subspace of 𝑉,
Clear
Span 𝑠𝑝𝑎𝑛{𝑣1 , 𝑣2 } ⊆ 𝑠𝑝𝑎𝑛{𝑣1 , 𝑣2 , 𝑣3 } ⊆ 𝑠𝑝𝑎𝑛{𝑣1 , 𝑣2 , 𝑣3 , 𝑣4 }
For example:
1
1 2
1 2 3
𝑠𝑝𝑎𝑛 {[ ]} ⊆ 𝑠𝑝𝑎𝑛 {[ ] , [ ]} ⊆ 𝑠𝑝𝑎𝑛 {[ ] , [ ] , [ ]}
1
1 2
1 2 3
But they are all equal(!!):
1
2
3
1
𝛼 [ ] + 𝛽 [ ] + 𝛾 [ ] = (𝛼 + 𝛽 + 𝛾) [ ]
1
2
3
1
Linear Dependency
𝑣1 , … , 𝑣𝑘 are said to be linearly dependent if there is a choice 𝛼1 , … , 𝛼𝑘 ∈ 𝔽 s.t.
𝛼1 𝑣1 + ⋯ + 𝛼𝑘 𝑣𝑘 = 0
Not all of which are zero!
𝛼
𝛼
If say 𝛼1 ≠ 0, 𝑣1 = (-𝛼2 ) 𝑣2 + ⋯ + (-𝛼𝑘 ) 𝑣𝑘
1
1
𝑣1 , … , 𝑣𝑘 are said to be linearly independent if the following statement is true:
𝛼1 𝑣1 +, … , 𝛼𝑘 𝑣𝑘 = 0 → 𝛼1 = 𝛼2 = ⋯ = 𝛼𝑘 = 0
If 𝑣1 , … , 𝑣𝑘 are linearly independent, then
𝑢 = 𝛼1 𝑣1 + ⋯ + 𝛼𝑘 𝑣𝑘 = 𝛽1 𝑣1 + ⋯ + 𝛽𝑘 𝑣𝑘 → 𝛼𝑖 = 𝛽𝑖 for all 𝑖.
A set of vectors 𝑢1 , … , 𝑢𝑙 is said to be a basis for a vector space 𝑈 over 𝔽, if 𝑢1 , … , 𝑢𝑙 ∈ 𝑈 and:
1) 𝑠𝑝𝑎𝑛{𝑢1 , … , 𝑢𝑙 } = 𝑈 (there’s enough vectors to span the entire space)
2) 𝑢1 , … , 𝑢𝑘 are linearly independent. (there are’t too many vectors)
Matrix Mutiplication
If 𝐴𝑝𝑋𝑞 , 𝐴𝑞𝑋𝑟 are matrices,
𝑞
𝐶 = 𝐴𝐵 is a 𝑏𝑋𝑟 matrix with 𝑐𝑖,𝑗 = ∑𝑠=1 𝑎𝑖,𝑠 𝑏𝑠,𝑗
𝑏1,𝑗
[𝑎𝑖,1 , … , 𝑎𝑖,𝑝 ] ∙ [ ⋮ ]
𝑏𝑞,𝑗
If 𝐴𝑝𝑋𝑞 , 𝐵𝑞𝑋𝑟 , 𝐷𝑟𝑋𝑠 are matrices, then (𝐴𝐵)𝐷 = 𝐴(𝐵𝐷)
In general – 𝐴𝐵 ≠ 𝐵𝐴!
0 1 1 1
0 0
1
][
]=[
]≠[
0 1 0 0
0 0
0
𝛼, 𝛽 𝛼𝛽 = 0 → 𝛼 = 0 or 𝛽 = 0
[
1 0 1
0
][
]=[
0 0 1
0
2
]
0
1
𝐴 ∈ 𝔽𝑝𝑋𝑟 Is said to be left invertible if there is 𝐵 ∈ 𝔽𝑞𝑋𝑝 s.t. 𝐵𝐴 = 𝐼 𝑞 = [0
0
𝑝𝑋𝑞
𝑞𝑋𝑝
𝐴∈𝔽
is right invertible if there’s a 𝐶 ∈ 𝔽
s.t. 𝐴𝐶 = 𝐼𝑝
If 𝐵𝐴 = 𝐼𝑞 and 𝐴𝐶 = 𝐼𝑝 then 𝐵 = 𝐶.
𝐵 = 𝐵𝐼𝑝 = 𝐵(𝐴𝐶) = (𝐵𝐴)𝐶 = 𝐼𝑞 𝐶 = 𝐶
Later we shall show that this also forces 𝑝 = 𝑞.
Triangular matrices
𝐴𝑝𝑋𝑝 ∈ 𝔽 is said to be upper triangular if 𝑎𝑖,𝑗 = 0 for 𝑖 > 0
𝑎11 …
𝑎1𝑝
0
⋱
⋮ ]
[
0
0 𝛼_𝑝𝑝 𝑝𝑋𝑝
𝐴𝑝𝑋𝑝 ∈ 𝔽 is said to be lower triangular if 𝑎𝑖,𝑗 = 0 for 𝑖 > 0
0 0
⋱ 0]
0 1 𝑞𝑋𝑞
𝑎11
[ ⋮
𝑎𝑝1
0
⋱
…
0
0 ]
𝛼_𝑝𝑝
𝑝𝑋𝑝
If both upper and lower than we denote it as simply triangular.
𝑎11 0
0
⋱
0 ]
[ 0
0 0 𝛼_𝑝𝑝 𝑝𝑋𝑝
Theorm: Let 𝐴 ∈ 𝔽𝑝𝑋𝑝 be a triangular matrix, then A is invertible if and only if all the diagonal
entries 𝑎𝑖𝑖 are nonzero.
Moreover, in this case, 𝐴 is upper triangular ↔ its inverse is upper triangular
And 𝐴 is lower triangular ↔ its inverse is lower triangular.
𝐴=[
𝑎
0
𝑏
]
𝑑
Investigate 𝐴 𝐵 = 𝐼2
want to:
𝑎
[
0
[
𝑏 𝛼
][
𝑑 𝛾
𝑎𝛼 + 𝑏𝛾
𝑑𝛾
𝛽
1
]=[
𝛿
0
𝑎𝛽 + 𝑏𝛿
]
𝑑𝛿
Want to choose 𝛼, 𝛽, 𝛾, 𝛿 so that:
𝑎𝛼 + 𝑏𝛾 = 1
𝑑𝛾 = 0
𝑎𝛽 + 𝑏𝛿 = 0
𝑑𝛿 = 1
1
(4)→ 𝑑 ≠ 0, 𝛿 ≠ 0, 𝛿 = 𝑑
(2)+(𝑑 ≠ 0) → 𝛾 = 0
(1)+𝛾 = 0 → 𝑎𝛼 = 1 → 𝑎 ≠ 0, 𝛼 ≠ 0,
1
1
𝛼
1
(3)𝛽 = − (𝑎) 𝑏 (𝑑)
If 𝐴𝐵 = 𝐼2 it is necessary that 𝑎 ≠ 0 and 𝑑 ≠ 0.
Also we have a formula for B
Can show 𝐵𝐴 = 𝐼2
0
]
1
Theorm: Let 𝐴 ∈ 𝔽𝑝𝑋𝑝 , 𝐴 is triangular, then A is invertible if and only if the diagonal entries of 𝐴
are all non-zero.
Moreover, if this condition is met, then A is upper triangular ↔ the inverse to 𝐴 is upper
triangular. (lower↔lower)
𝐴 invertible, there exists a matrix 𝐵 ∈ 𝔽𝑝𝑋𝑝
Such that, 𝐴𝐵 = 𝐵𝐴 = 𝐼𝑝
Let 𝐴(𝑘+1)𝑋(𝑘+1) upper triangular matrix.
𝑎=[
𝐴11
0
𝐴12
]
𝐴22
Plan: Suppose we know that if 𝐴𝑘𝑋𝑘 upper triangular, then 𝐴 is invertible if and only if it’s
diagonal entries are nonzero.
Objective: Extend this to (𝑘 + 1)𝑋(𝑘 + 1) upper triangular matrices.
Suppose first that 𝐴 is invertible.
𝐴11 𝐴12 𝐵11 (𝑘𝑋𝑘) 𝐵12 (𝑘𝑋1)
][
]=
𝐴21 𝐴22 𝐵21 (1𝑋𝑘) 𝐵22 (1𝑋1)
𝐴 𝐵 + 𝐴12 𝐵21 𝐴11 𝐵12 + 𝐴12 𝐵22
[ 11 11
]
𝐴22 𝐵21
𝐴22 𝐵22
𝐴𝐵 = [
TODO: Draw single vector multiplication
[𝑎𝑖1 𝑎𝑖2 𝑎𝑖3 ]
[ ]
(1)
(2)
(3)
(4)
𝐴11 𝐵11 + 𝐴12 𝐵21 = 𝐼𝑘
𝐴22 𝐵21 = 01𝑋𝑘
𝐴11 𝐵12 + 𝐴12 𝐵22 = 0
𝐴22 𝐵22 = 1
(4)→ 𝐴22 ≠ 0, 𝐵22 ≠ 0 𝑎𝑛𝑑 𝑏22 =
1
𝐴22
(2)+𝐴22 ≠ 0 → 𝐵21 = 0
(1)+𝐵21 = 0 → 𝐴11 𝐵11 = 𝐼𝑘
𝐵11 𝐴11
𝐵𝐴 = 𝐼𝑘+1
𝐵
𝐵12 𝐴11 𝐴12
𝐼 0
[ 11
][
]=[𝑘
]
0 𝐵22 0 𝐴22
0 1
= 𝐼𝑘 → 𝐴11 𝑖𝑠 𝑎 𝑘𝑋𝑘 𝑢𝑝𝑝𝑒𝑟 𝑡𝑟𝑖𝑎𝑛𝑔𝑢𝑙𝑎𝑟 𝑚𝑎𝑡𝑟𝑖𝑥 𝑡ℎ𝑎𝑡 𝑖𝑠 𝑖𝑛𝑣𝑒𝑟𝑡𝑖𝑏𝑙𝑒!
If the theorem is true for 𝑘𝑋𝑘 marices, then if 𝐴 ∈ 𝔽(𝑘+1)𝑋(𝑘+1) triangular upper matrix,
diagonal of 𝐴11 are nonzero and 𝐴22 ≠ 0!
Showed: 𝐴(𝑘+1)𝑋(𝑘+1) is upper triangular invertible.
Now take 𝐴(𝑘+1)𝑋(𝑘+1) upper triangular with non zero entries on its diagonal.
𝐴 (𝑘𝑋𝑘)
𝐴12
𝐴 = [ 11
]
0
𝐴22 (1𝑋1)
𝐴11 is 𝑘𝑋𝑘 upper triangular matrix with non-zero entries on its diagonal.
Theorm true for 𝑘𝑋𝑘 matrices, then this = there exists a 𝑘𝑋𝑘 𝐶 𝑠. 𝑡.
𝐴11 𝐶 = 𝐶𝐴11 = 𝐼𝑘
−1
𝐶 −𝐶𝐴12 𝐴−1
𝐴−1 −𝐴11
𝐴12 𝐴−1
22
22
Lets define B=[[
] = [ 11
]
−1
0
𝐴22
0
𝐴−1
22
Can show – 𝐴𝐵 = 𝐵𝐴 = 𝐼𝑘+1
[
𝐴11
0
−1
𝐴12 𝐴11
][
𝐴22 0
−1
−1
−𝐴11
𝐴12 𝐴−1
22
] = [𝐴11 𝐴11
−1
𝐴22
0
𝐼𝑘 - − 𝐼𝑘 + 𝐼𝑘
=[
]
0
∗
−1
−1
𝐴11 (−𝐴11
𝐴12 𝐴−1
22 ) + 𝐴12 𝐴22 ]
∗
𝐴𝑇 =transpose of 𝐴.
1 4
2 3
] = [2 5 ]
5 6
3 6
𝑖𝑗 entry of 𝐴𝑇 is 𝑗𝑖 entry of 𝐴
𝐴𝐻 =Hermitian transpose - ̅̅̅̅
𝐴𝑇 (take complex conjugate)
1
Example: 𝐴 = [
4
1−𝑖
3
𝐻
] → 𝐴 = [ −2𝑖
6 − 7𝑖
3
𝐴 ∈ 𝔽𝑝𝑋𝑞 , 𝒩𝐴 = {𝑥 ∈ 𝔽𝑞 : 𝐴𝑥 = 0} - subspace of 𝔽𝑞
Nullspace of 𝐴
ℛ𝐴 = {𝐴𝑥: 𝑥 ∈ 𝔽𝑞 } - subspace of 𝔽𝑞
1+𝑖
𝐴=[
4
2𝑖
5
𝐴𝑥 = 0
𝐴𝑦 = 0
→
𝐴(𝑥 + 𝑦) = 𝐴𝑥 + 𝐴𝑦 = 0 + 0 = 0
𝐴𝛼𝑥 = 𝛼(𝐴𝑥) = 𝛼0 = 0
Need to show all properties. But they exist…
4
5 ]
6 + 8𝑖
Matrix Multiplication
𝐴𝑝𝑋𝑞 , 𝐵𝑞𝑋𝑟
𝑎1
𝑎2
𝐴 = [𝑎1 𝑎2 … 𝑎𝑞 ] = [ ⋮ ]
𝑎𝑝
𝑏1
𝑏2
𝐵 = [𝑏1 𝑏2 … 𝑏𝑞 ] = [ ]
⋮
𝑏𝑝
[
1 2
4 5
3
] = [𝑢1 𝑢2 𝑢3 ]
6
1
𝑢1 = [ ]
4
𝑎1 𝐵
𝑎1
𝑎
𝐵
𝐴𝐵 = 𝐴[𝑏1 … 𝑏𝑟 ] = [𝐴𝑏1 𝐴𝑏2 … 𝐴𝑏𝑟 ] = [ ⋮ ] 𝐵 = [ 2⋮ ]
𝑎𝑝
𝑎𝑝 𝐵
𝑞
𝐴𝐵 = ∑ 𝑎𝑖 𝑏𝑖
𝑖=1
𝐴𝐵 = [𝑎1 𝑎2 … 𝑎1 ]𝐵 = [𝑎1 0 … 0]𝐵 + [0 𝑎2 0 … 0]𝐵 + ⋯ + [0 … 0 𝑎𝑞 ]𝐵
There are 𝑝𝑋𝑞 matrices that are right invertible but not left invertible.
Similarly - there are 𝑝𝑋𝑞 matrices that are left invertible and not right invertible.
This does not happen if 𝑝 = 𝑞.
1
1 0 0
[
] [0
0 1 0
𝑎
0
1] = 𝐼2
𝑏
Gaussian Elimination
𝐴
⏟
𝑔𝑖𝑣𝑒𝑛 𝑝𝑋𝑞
∙𝑥 =
⏟
𝑏
𝑔𝑖𝑣𝑒𝑛 𝑞𝑋1
Looking for solutions for 𝑥…
For instance:
3𝑥1 + 2𝑥2 = 6
2𝑥1 + 3𝑥2 = 4
A Gaussian Elimination is a method of passing to a new system of equations that is easier to
work with.
This new system must have the same solutions as the original one.
𝑈 𝑞𝑋𝑞 is called upper echelon if the first nonzero entry in row 𝑖, sits to the left of the first
nonzero entry in row 𝑖 + 1.
Example:
1
0
𝑈=[
0
0
3
6
0
0
4
0
0
0
2
0
4
0
1
2
]
1
0
The first nonzero entry in each row is called a pivot. In the example we have 3 pivots marked in
red.
Consider the equation:
𝐴𝑥 = 𝑏
0 2 3 1
1
𝐴 = [ 1 5 3 4] 𝑏 = [ 2]
2 6 3 2
3
Two operations:
1) Interchange rows
2) Substract a multiple of one row from another row
Each operation corresponds to multiplying on the left by an invertible matrix.
We can revert the process later:
→ 𝐶𝐴𝑥 = 𝐶𝑏 → 𝐶 −1 (𝐶𝐴𝑥) = 𝐶 −1 𝐶𝑏 → 𝐴𝑥 = 𝑏
Steps:
1)
2)
3)
4)
0
̃
[𝐴𝑏]
Construct the augmented matrix 𝐴 =
= [1
2
1 5 3 4 |
Interchange rows 1 and 2. 𝐴̃1 = [0 2 3 1 |
2 6 3 2 |
1 5
̃
Subtract 2 times row 1 from row 3 𝐴2 = [0 2
0 −4
1 5 3 4
Add 2 times row 2 to row 3 𝐴̃3 = [0 2 3 1
0 0 3 −4
2 3 1
5 3 4
6 3 2
2
1]
1
3
4
3
1
−3 −6
| 2
| 1]
| −1
| 1
| 2]
| 1
| 2
| 1]
| −3
2
5) Solve the system – 𝑈𝑥 = [ 1 ]
−1
𝑥1
1 5 3 4 𝑥
2
[0 2 3 1 ] [𝑥 ]
3
0 0 3 −4 𝑥
4
6) Work from bottom up
3𝑥3 − 4𝑥4 = −1
Solve the pivot variable: 𝑥3 = (−1 + 4𝑥4 )3
Note:
0 1 0
𝐴̃1 = 𝑃1 𝐴̃, 𝑃1 = [1 0 0]
0 0 1
0 1 0 𝑎
𝑏
[ 1 0 0] [ 𝑏 ] = [ 𝑎 ]
0 0 1 𝑐
𝑐
𝐴̃2 = 𝐸1𝐴̃1
1 0 0
𝐸1 = [ 0 1 0]
−2 0 1
𝑎
𝑎
𝐸1 [𝑏] = [ 𝑏
]
−2𝑎 + 𝑐
𝑐
𝐴̃3 = 𝐸2 𝐴̃2
1 0 0
𝐸2 = [0 1 0]
0 2 1
𝑎
𝑎
𝑏
𝐸2 [ ] = [ 𝑏 ]
2𝑏 + 𝑐
𝑐
------- End of lesson 2
Gauss Elimination
0 2 3
𝐴 = [1 5 3
2 6 3
1
𝑏 = [2]
1
Try to solve 𝐴𝑥
1
4]
2
=𝑏
𝐴̃ = [𝐴 𝑏]
Two operations:
1) To permute (interchange) rows
2) Subtract a multiple of one row from the other
(1)&(2) are implemented by multiplying 𝐴𝑥 = 𝑏 on the left by the appropriate invertible matrix.
𝐴𝑥 = 𝑏
𝑃1 𝐴𝑥 = 𝑃1 𝑏 where 𝑃1 is an appropriate permutation matrix
𝐸1 𝑃1 𝐴𝑥 = 𝐸1 𝑃1 𝑏 where 𝐸2 is a lower triangular matrix
⋮
Eventally you will have 𝑈𝑥 = 𝑏
Such that 𝑈 is an upper echelon matrix.
1 5 3 4
2
𝐴̃ → [0 2 3 1
1 ] = [𝑈 𝑏 ′ ]
0 0 3 −4 −1
Now we need to solve 𝑈𝑥 = 𝑏.
𝑥1 + 5𝑥2 + 3𝑥3 + 4𝑥4 = 2
2𝑥2 + 3𝑥3 + 𝑥4
3𝑥3 − 4𝑥4 = −1
Work from the bottom up, Solve the pivot variables in terms of the others.
𝑥3 =
−1+4𝑥4
3
2𝑥2 = −3𝑥3 − 𝑥4 + 1 = 1 − 4𝑥4 − 𝑥4 + 1 = −5𝑥4 + 2
𝑥1 = 2 − 5𝑥2 − 3𝑥3 − 4𝑥4 = 2 −
−1 +
25
𝑥4 − 5𝑥4
2
=-−1+
5(2−5𝑥4 )
—1 +
2
15
𝑥
2 4
4𝑥4 − 𝑥4 + 1 =
9
−2
𝑥1
2
5
1
𝑥2
−2
1 + 𝑥4
𝑥 = [𝑥 ] =
−3
3
4
𝑥4
[0]
3
[1]
9
2
0
5
𝐴 − 2 = [ 0]
4
0
3
[1]
Another example:
0
0
𝐴=[
0
0
0
1
2
0
3
0
3
6
𝑏1
7
𝑏
0
] , 𝑏 = [ 2]
𝑏3
8
𝑏4
14
4
0
6
8
Try to solve 𝐴𝑥 = 𝑏
0
0
𝐴̃ = [𝐴 𝑏] → [
0
0
1
0
0
0
0
3
0
0
0
4
2
0
0
7
1
0
𝑏2
𝑏1
]
𝑏3 + 2𝑏2 − 𝑏1
𝑏4 − 2𝑏1
𝐴𝑥 = 𝑏 ⇔
𝑥2 = 𝑏2
3𝑥3 + 4𝑥3 + 7𝑥4 = 𝑏1
2𝑥4 + 𝑥5 = 𝑏3 − 2𝑏2 − 𝑏1
0 = 𝑏4 − 2𝑏1
Solve for 𝑥2 , 𝑥3 , 𝑥4 and find that:
0
0
1
𝑏2
0
0
5
3𝑏1 +4𝑏2 −2𝑏3
𝑥=
+ 𝑥1 0 + 𝑥5 − 3
3
1
𝑏3 −2𝑏2 −𝑏1
0
−2
2
[0]
[1]
[
]
0
This is a solution of 𝐴𝑥 = 𝑏 for any choice of 𝑥1 and 𝑥5 provided 𝑏4 = 2𝑏1.
Let’s track the Gaussian elmnimation from a mathematical point of view:
𝑃𝐸𝑃𝐴 … = 𝑈
𝐴 ∈ 𝔽𝑝×𝑞 𝐴 ≠ 0𝑝×𝑞 , then there exists an invertible matrix 𝐺𝑝×𝑝 such that 𝐺𝐴 = 𝑈 upper
echelon.
Theorem: 𝑈 ∈ 𝔽𝑝×𝑞 be upper echelon with 𝑘 pivots, 𝑘 ≥ 1,
Then:
(1) 𝑘 ≤ min{𝑝, 𝑞}
(2) 𝑘 = 𝑞 ⇔ 𝑈 is left invertible⇔ 𝒩𝑈 = {0}
(3) 𝑘 = 𝑝 ⇔ 𝑈 is right invertible⇔ ℛ𝑈 = 𝔽𝑝
Proof of (2):
Suppose 𝑘 = 𝑞.
▭
If 𝑝 = 𝑞, 𝑈 = [
▭
] →no zero diagonal entries
⋱
0
Otherwise 𝑝 > 𝑞,
▭
⋱
▭
𝑞×𝑞
𝑈11
= [ 𝑝−𝑞×𝑞
]
− − − −
𝑈21
0 0 0 0
[0 0 0 0]
𝑞×𝑞
𝑝×𝑝−𝑞
The left invertible matrix 𝑉 = [𝑉11 , 𝑉12
]
𝑞×𝑞
𝑉11 𝑈11 = 𝐼
by definition
𝑈=
𝑉𝑈 = [𝑉11
𝑉12 ] [
𝑈11
] = 𝑉11 𝑈11 + 𝑉12 𝑈21 = 𝐼𝑞 + 0 = 𝐼𝑞
𝑈21
(2a) We shown that 𝑘 = 𝑞 ⇒ 𝑈 is left invertible.
(2b) 𝑈 left invertible ⇒ 𝒩𝑈 = {0}.
If 𝐴𝑝×𝑞 , 𝒩𝐴 = {𝑥|𝐴𝑥 = 0}
Let 𝑥 ∈ 𝒩𝐴 By definition, it means that 𝑈𝑥 = 0
By assumption, 𝑈 is left invertible⇒ there is a 𝑉 ∈ 𝔽𝑝×𝑞 𝑠. 𝑡. 𝑉𝑈 = 𝐼𝑞
0 = 𝑉0 = 𝑉(𝑈𝑥) = (𝑉𝑈)𝑥 = 𝐼𝑞 𝑥 = 𝑥
(2c) 𝒩𝑈 = {0} ⇒ 𝑈 has 𝑞 pivots.
𝒩𝑈 = {0} ⇒ 𝑈𝑥 = 0 ⇔ 𝑥 = 0
𝑥1
𝑥2
[𝑢1 , 𝑢2 , … , 𝑢𝑞 ] [ ⋮ ] = 0 ⇔ 𝑥1 = 𝑥2 = ⋯ = 𝑥𝑞 = 0
𝑥𝑞
𝒩𝑈 = {0} ⇒Columns of 𝑈 are independent.
That forces k=q
(3𝑎)𝑘 = 𝑝 ⇒ 𝑈 is right invertible
𝑞 = 𝑝 ⇒ 𝑈 upper triangular with nonzero diagonal entries ⇒ U is invertible
𝑞>𝑝
▭
| ∙ ∙
▭ ∙ ∙ ∙ ∙
𝑠𝑤𝑎𝑝 𝑐𝑜𝑙𝑢𝑚𝑛𝑠
[0 0 ∙ ∙ ∙ ]𝑃
=
▭
| ∙ ∙]
[
0 0 0 0 ▭
▭ | ∙ ∙
You can find such a 𝑃 such that 𝑈𝑃 = [𝑈11 𝑈12 ] here 𝑈11 is 𝑝 × 𝑝 upper triangular with
nonzero diagonal elements, and 𝑈12 = 0.
𝑉
𝑉
𝑉 = [ 11 ] so that [𝑈11 𝑈12 ] [ 11 ] = 𝐼𝑝 ⇒ Therefore 𝑈11 is invertible. By VP!
𝑉21
𝑉21
And it doesn’t matter what 𝑉21 we choose.
(3b) 𝑥 is the input, 𝐴𝑥 is the output.
The range is the set of outputs.
ℛ𝐴 = {𝐴𝑥|𝑥 ∈ 𝔽𝑝 }
Claim: Given any 𝑏 ∈ 𝔽𝑝 , can find an 𝑥 ∈ 𝔽𝑞 𝑠. 𝑡. 𝑈𝑥 = 𝑏
Let W be a right inverse to 𝑈, let 𝑥 = 𝑊𝑏.
𝑈𝑥 = 𝑈(𝑊𝑏) = (𝑈𝑊)𝑏 = 𝐼𝑝 𝑏 = 𝑏
(3c)ℛ𝑈 = 𝔽𝑝 ⇒ 𝑝 𝑝𝑖𝑣𝑜𝑡𝑠
If 𝑘 < 𝑝 then it must look something like:
▭
⋱
𝑈=
−
0
[0
−
0
0
−
0
0
▭
−
0
0
−
0
0]
We have 𝑝 − 𝑘 zero rows.
So all of our answers would always have 𝑝 − 𝑘 zeroes at the end! Therefore, we don’t cover all
𝔽𝑝
⋮
⋮
𝑈𝑥 = ⋮
⋮
[0]
Theorem: If 𝐴 ∈ 𝔽𝑝×𝑞 , 𝐵 ∈ 𝔽𝑝×𝑞 and 𝐵𝐴 = 𝐼𝑞 then 𝑝 ≥ 𝑞
Proof: Clear that 𝐴 ≠ 0𝑝×𝑞 .
If we apply Gaussian elimination, can find 𝐺𝑝×𝑝 invertible matrix such that 𝐺𝐴 = 𝑈 which is
upper echelon.
𝐼𝑞 = 𝐵𝐴 = 𝐵(𝐺 −1 𝑈) = (𝐵𝐺 −1 )𝑈 ⇒ 𝑈 is left invertible⇒ 𝑈 has 𝑞 pivots⇒ 𝑝 ≥ 𝑞.
Theorem: Let 𝒱 be a vector space over 𝔽 and let {𝑢1 , … , 𝑢𝑘 } be a basis for 𝒱.
And let {𝑣1 , … , 𝑣𝑙 } also be a basis for 𝒱. Then 𝑘 = 𝑙.
𝑙
∀𝑖 = 1, … , 𝑘: 𝑢𝑖 = ∑ 𝑏𝑠𝑖 𝑣𝑠
𝑠=1
Define 𝐵 as 𝑙 × 𝑘
𝑙
∀𝑗 = 1, … , 𝑘: 𝑣𝑗 = ∑ 𝑎𝑡𝑗 𝑢𝑡
𝑠=1
Define 𝐴 as 𝑘 × 𝑙
𝑙
𝑘
𝑠=1
𝑡=1
𝑘
𝑙
𝑙
0 𝑖𝑓 𝑡 ≠ 𝑖
𝑢𝑖 = ∑ 𝑏𝑠𝑖 ∑ 𝑎𝑡𝑠 𝑢𝑡 = ∑ ∑ 𝑎𝑡𝑠 𝑢𝑡 ⇒ ∑ 𝑎𝑡𝑠 𝑏𝑠𝑖 = {
⇒ 𝐴𝐵 = 𝐼𝑘 => 𝑙 ≥ 𝑘
1 𝑖𝑓 𝑡 = 𝑖
𝑡=1 𝑠=1
𝑠=1
But I can do this again symmetrically with u’s replaced by v’s, I would get that 𝑘 ≥ 𝑙
So 𝑘 = 𝑙.
-------End of lesson 3
Theorem:
Let 𝒱 be a vector space over 𝔽. Let {𝑢1 , … , 𝑢𝑘 } be a basis for 𝒱 and {𝑣1 , … , 𝑣𝑙 } also a basis for 𝒱,
then 𝑙 = 𝑘.
That number is called the dimension of 𝒱
The vector space 0 = {0}. Define its dimension to be 0.
A transformation (mapping, operator,…) from a vector space 𝒰 over 𝔽 into a vector space 𝒱
over 𝔽 is a rule (algorithm, formula) that assigns a a vector 𝑇𝑢 in 𝒱 for each 𝑢 ∈ 𝒰.
𝑎
Example 1: Let 𝒰 = ℝ2 = {[ ] |𝑎, 𝑏 ∈ ℝ} 𝒱 = ℝ3
𝑏
2
𝑎
𝑎
𝑇1 [ ] = [ 2𝑏 2 ]
𝑏
𝑎2 + 𝑏 2
𝑎
Example 2: Let 𝒰 = ℝ2 = {[ ] |𝑎, 𝑏 ∈ ℝ} 𝒱 = ℝ3
𝑏
1
3
𝑎
𝑎
𝑇2 [ ] = [ 2 0] [ ] (by matrix multiplication)
𝑏
𝑏
−4 6
Definition: A transformation 𝑇 from 𝒰 into 𝒱 is said to be linear if:
1) 𝑇(𝑢1 + 𝑢2 ) = 𝑇𝑢1 + 𝑇𝑢2
2) 𝑇(𝛼𝑢) = 𝛼𝑇𝑢
Is 𝑇1 linear?
No!
4
1
1
1
1
𝑇1 ([ ] + [ ]) = [8] ≠ 2 ∙ 𝑇1 [ ] = 2 ∙ [2]
1
1
1
8
2
Every linear transformation can be expressed in terms of matrix multiplication.
If 𝑇 is a linear transformation from a vector space 𝒰 over 𝔽 into the vector space 𝒱 over 𝔽. Then
define:
𝒩𝑇 = {𝑢 ∈ 𝒰|𝑇𝑢 = 0𝒱 } –null space of 𝑇.
ℛ 𝑇 = {𝑇𝑢|𝑢 ∈ 𝒰} - range space of 𝑇.
𝒩𝑇 is a subspace of 𝒰
ℛ 𝑇 is also a subspace of 𝒰
𝑢1 ∈ 𝒩𝑇 , 𝑢2 ∈ 𝒩𝑇 ⇒ 𝑇𝑢1 = 0𝒱 , 𝑇𝑢2 = 0_𝒱
𝑇(𝑢1 + 𝑢2 ) = 𝑇𝑢1 + 𝑇𝑢2 = 0𝒱 + 0𝒱 = 0𝒱
𝑢1 + 𝑢2 ∈ 𝒩𝑇
Conservation of dimension
Theorem: Let 𝑇 be a linear transformation from a finite dimension vector space 𝒰 over 𝔽 into a
vector space 𝒱 over 𝔽.
Then dim 𝒰 = dim 𝒩𝑇 + dim ℛ 𝑇
Proof:
Suppose 𝒩𝑇 ≠ {0} and ℛ 𝑇 ≠ {0}
Let 𝑢1 , … , 𝑢𝑘 be a basis for 𝒩𝑇 .
Claim ℛ 𝑇 is also a finite dimensional vector space. Let 𝑣1 , … , 𝑣𝑙 be a basis for ℛ 𝑇 .
Since 𝑣𝑖 ∈ ℛ 𝑇 , so we can find 𝑦𝑖 ∈ 𝒰 𝑠. 𝑡. 𝑇𝑦𝑖 = 𝑣𝑖 .
Claim: {𝑢1 , … , 𝑢𝑘 ; 𝑦1 , … , 𝑦𝑙 } are linearly independent.
To check: Suppose there are coefficients: 𝑎1 , … , 𝑎𝑘 , 𝑏1 , … , 𝑏𝑙 ∈ 𝔽 𝑠. 𝑡.
𝑎1 𝑢1 + ⋯ + 𝑎𝑘 𝑢𝑘 + 𝑏1 𝑦1 + ⋯ + 𝑏𝑙 𝑦𝑙 = 0𝒰
Let’s activate the transformation on both sides
𝑇(𝑎1 𝑢1 + ⋯ + 𝑎𝑘 𝑢𝑘 + 𝑏1 𝑦1 + ⋯ + 𝑏𝑙 𝑦𝑙 ) = 𝑇(0𝒰 )
𝑎1 𝑇𝑢1 + ⋯ + 𝑎𝑘 𝑇𝑢𝑘 + 𝑏1 𝑇𝑦1 + ⋯ + 𝑏𝑙 𝑇𝑦𝑙 = 𝑇𝑜𝒰 = 0𝒱
Why 𝑇0𝒰 = 0𝒱
𝑇0𝒰 = 𝑇(0𝒰 + 0𝒰 ) = 𝑇0𝒰 + 0𝒱 ⇒ 𝑇0𝒰 = 0𝒱
Since 𝑢1 , … , 𝑢𝑘 ∈ 𝒩𝑇 , their transformation is zero!
Linearly independent items must never be zero, otherwise you can multiply the zero one with
some value other than zero and get zeros.
{𝑣1 , … , 𝑣𝑙 } is a basis for ℛ 𝑇 ⇒ 𝑣1 , … , 𝑣𝑙 are linearly independent⇒ 𝑏1 = 𝑏2 = ⋯ = 𝑏𝑙 = 0.
⇒ 𝑎1 𝑢1 + ⋯ + 𝑎𝑘 𝑢𝑘 = 0𝒰
{𝑢1 , … , 𝑢𝑘 } is a basis (for 𝒩𝑇 ) ⇒ 𝑢1 , … , 𝑢𝑘 are linearly independent⇒ 𝑎1 = ⋯ = 𝑎𝑘 = 0.
Claim next: 𝑠𝑝𝑎𝑛 {𝑢1 , … , 𝑢𝑘 , 𝑣1 , … , 𝑣𝑙 } = 𝒰.
Let 𝑢 ∈ 𝒰 and consider 𝑇𝑢 ∈ ℛ 𝑇 .
𝑙
𝑙
𝑙
⇒ 𝑇𝑢 = ∑ 𝑑𝑗 𝑣𝑗 = ∑ 𝑑𝑗 𝑇𝑦𝑗 = 𝑇 (∑ 𝑑𝑗 𝑦𝑗 )
𝑗=1
𝑦−1
𝑙
𝑇 (𝑢 − ∑ 𝑑𝑗 𝑦𝑗 ) = 0𝒱
𝑦−1
𝑙
𝑢 − ∑ 𝑑𝑗 𝑦𝑗 ∈ 𝒩𝑇 ⇒
𝑦−1
𝑦−1
𝑙
𝑘
𝑢 − ∑ 𝑑𝑗 𝑦𝑗 ∈ 𝒩𝑇 = ∑ 𝑐𝑖 𝑢𝑖 ⇒ 𝑢 ∈ 𝑠𝑝𝑎𝑛{𝑢1 , … , 𝑢𝑘 , 𝑦1 , … , 𝑦𝑙 } ⇒
𝑦−1
𝑖=1
{𝑢1 , … , 𝑢𝑘 , 𝑦1 , … , 𝑦𝑙 } is a basis for 𝒰
⇒ dim 𝒰 = ⏟
𝑘 + ⏟𝑙
dim 𝒩𝑇
dim 𝒩𝑇
∎
If ℛ 𝑇 = {0} it forces 𝑇 to be the zero transformation
Last time we showed that:
If 𝑈 is an upper echelon 𝑝 × 𝑞 with 𝑘 pivots.
(1) 𝑘 ≤ min{𝑝, 𝑞}
(2) 𝑘 = 𝑞 ⇔ 𝑈 left invertible⇔ 𝒩𝑈 = {0}
(3) 𝑘 = 𝑝 ⇔ 𝑈 right invertible⇔ ℛ𝑈 = 𝔽𝑝
Objective: Develop analogous set of facts for 𝐴 ∈ 𝔽𝑝×𝑞 not necessarily upper echelon.
(1) 𝐴 ∈ 𝔽𝑝×𝑞 , 𝐴 ≠ 0𝑝×𝑞 , then there exists an invertible matrix 𝐺 𝑠. 𝑡. 𝐺𝐴 = 𝑈 is upper
echelon.
𝐵1 , 𝐵2 , 𝐵3 are invertible 𝑝 × 𝑝 matrices, then 𝐵1 𝐵2 𝐵3 is also invertible.
(2) Let 𝐴 ∈ 𝔽𝑝×𝑞 , 𝐵 ∈ 𝔽𝑞×𝑞 , 𝐶 ∈ 𝔽𝑝×𝑝 𝑠. 𝑡. 𝐵 and 𝐶 are invertible. Then:
a. ℛ𝐴𝐵 = ℛ𝐴 dim 𝒩𝐴𝐵 = dim 𝒩𝐴
b. dim ℛ𝐶𝐴 = dim ℛ𝐴 𝒩𝐶𝐴 = 𝒩𝐴
Suppose 𝑥 ∈ ℛ𝐴𝐵 ⇒ there is 𝑢 𝑠. 𝑡. 𝑥 = (𝐴𝐵)𝑢
But this also means 𝑥 = 𝐴(𝐵𝑢) ⇒ 𝑥 ∈ ℛ𝐴 .
i.e. ℛ𝐴𝐵 ⊆ ℛ𝐴 .
Suppose 𝑥 ∈ ℛ𝐴 ⇒ 𝑥 = 𝐴𝑣 = 𝐴(𝐵𝐵−1 )𝑥 = (𝐴𝐵)(𝐵−1 𝑣) ⇒ 𝑥 ∈ ℛ𝐴𝐵 i.e. ℛ𝐴 ⊆ ℛ𝐴𝐵
ℛ𝐴 ⊆ ℛ𝐴𝐵 ⊆ ℛ𝐴 ⇒ ℛ𝐴 = ℛ𝐴𝐵
Can we also show 𝒩𝐴𝐵 = 𝒩𝐴 ?
No. Or at least not always…
0 1
Let 𝐴 = [
]
0 0
𝑥1
𝑥
0
0
Let 𝑥 ∈ 𝒩𝐴 , 𝐴 [𝑥 ] = [ 2 ] = [ ] ⇒ 𝒩𝐴 = {[ ] |𝛽 ∈ 𝔽}
𝛽
0
0
2
0 1
1 0
2
Let 𝐵 = [
], 𝐵 = [
]. B is invertible.
1 0
0 1
𝛼
𝒩𝐴𝐵 = {[ ] |𝛼 ∈ 𝔽}
0
What we can show, is the dimensions of these spaces are equal.
Let {𝑢1 , … , 𝑢𝑘 } be a basis for 𝒩𝐴
{𝐵−1 𝑢1 , … , 𝐵−1 _𝑢𝑘 } ∈ 𝒩𝐴𝐵
Easy to see: 𝐴𝐵(𝐵−1 𝑢𝑖 ) = 𝐴𝑢𝑖 = 0.
Claim: they are linearly independent.
Proof:
𝛼1 𝐵−1 𝑢1 + ⋯ + 𝛼𝑘 𝐵−1 𝑢𝑘 = 0
𝐵−1 (𝛼1 𝑢1 + ⋯ + 𝛼𝑘 𝑢𝑘 ) = 0
𝛼1 𝑢1 , … , 𝛼𝑘 𝑢𝑘 = 𝐵0 = 0
But 𝑢1 , … , 𝑢𝑘 is a basis (therefore independent) so 𝛼1 = 𝛼2 = ⋯ = 𝛼𝑘 = 0.
dim 𝒩𝐴 = 𝑘
dim 𝒩𝐴𝐵 ≥ 𝑘
So we’ve shown that: dim 𝒩𝐴𝐵 ≥ dim 𝒩𝐴
Now take basis – {𝑣1 , … , 𝑣𝑙 } of 𝒩𝐴𝐵 ⇒ 𝐴𝐵𝑣𝑖 = 0 ⇒ 𝐵𝑣𝑖 ∈ 𝒩𝐴
Check {𝐵𝑣1 , … , 𝐵𝑣𝑙 } are linearly independent. Same as before…
⇒ dim 𝒩𝐴 ≥ 𝑙 = dim 𝒩𝐴𝐵
So now we have two inequalities resulting I dim 𝒩𝐴 = dim 𝒩𝐴𝐵
Definition: If 𝐴 ∈ ℝ𝑝×𝑞 𝑟𝑎𝑛𝑘𝐴 = dim ℛ𝐴
ℛ𝐴 = 𝑠𝑝𝑎𝑛 𝑜𝑓 𝑡ℎ𝑒 𝑐𝑜𝑙𝑢𝑚𝑛𝑠 𝑜𝑓 𝐴
𝐴 = [𝑎1 𝑎2 𝑎3 𝑎4 ]
𝑥1
𝑥2
𝐴 [𝑥 ] = 𝑥1 𝑎1 + 𝑥2 𝑎2 + 𝑥3 𝑎3 + 𝑥4 𝑎4
3
𝑥4
▭ 0 0 0
Say 𝑈 = [ 0 0 ▭ 0] = [𝑢1 𝑢2 𝑢3 𝑢4 ]
0 0 0 0
ℛ𝑈 = {𝑢1 , 𝑢3 }
𝑟𝑎𝑛𝑘𝑈 = number of pivots in 𝑈.
Theorem: Let 𝐴 ∈ 𝔽𝑝×𝑞 then
(1) 𝑟𝑎𝑛𝑘𝐴 ≤ min(𝑝, 𝑞)
(2) 𝑟𝑎𝑛𝑘𝐴 = 𝑞 ⇔ 𝐴 is left invertible ⇔ 𝒩𝐴 = {0}
(3) 𝑟𝑎𝑛𝑘𝐴 = 𝑝 ⇔ 𝐴 is right ivertible ⇔ ℛ𝐴 = 𝔽𝑝
Proof:
If 𝑟𝑎𝑛𝑘𝐴 = 0 (1) is clearly true.
If 𝑟𝑎𝑛𝑘𝐴 ≠ 0 ⇒ ∃𝐺 𝑝×𝑝 invertible such that 𝐺𝐴 = 𝑈 upper echelon.
This implies that 𝑅𝑎𝑛𝑘𝐺𝐴 = 𝑟𝑎𝑛𝑘𝑈 = number of pivots in 𝑈.
Suppose 𝑟𝑎𝑛𝑘𝐴 = 𝑞 ⇒ 𝑟𝑎𝑛𝑘𝐺𝐴 = 𝑞 ⇒ 𝑟𝑎𝑛𝑘𝑈 = 𝑞 ⇒ 𝑈 is left invertible ⇒ there is a
𝑉 𝑠. 𝑡. 𝑉𝑈 = 𝐼𝑞
Same as to say 𝑉(𝐺𝐴) = 𝐼𝑞 ⇒ (𝑉𝐺)𝐴 = 𝐼𝑞 ⇒ 𝐴 is left invertible.
(3)
------End of lesson 4
𝑇 linear transformation from a finite dimension vector spce 𝒰 over 𝔽 into a vector space 𝒱 over
𝔽, then
dim 𝒰 = dim 𝒩𝑇 + dim ℛ 𝑇
𝑝×𝑞
𝑞 |𝐴𝑥
{𝑥
If 𝐴 ∈ 𝔽 , 𝒩𝐴 =
∈𝔽
= 0} (also a subspace of 𝔽𝑞 )
𝑞 = dim 𝒩𝐴 + dim ℛ𝐴
Theorem: If 𝐴 ∈ 𝔽𝑝×𝑞 , then:
(1) rank𝐴 ≤ min{𝑝, 𝑞}
(2) rank𝐴 = 𝑞 ⇔ 𝐴 is left invertible⇔ 𝒩𝐴 = {0}
(3) rank𝐴 = 𝑝 ⇔ 𝐴 is right invertible⇔ ℛ𝐴 = 𝔽𝑝 =”everything”
Exploited fact: if 𝑈 upper echelon with 𝑘 pivots, then:
(1) 𝑘 ≤ min{𝑚, 𝑛}
(2) 𝑘 = 𝑞 ⇔ 𝑈 is left invertible⇔ 𝒩𝑈 = {0}
(3) 𝑘 = 𝑝 ⇔ 𝑈 is right invertible⇔ ℛ𝑈 = 𝔽𝑝
Gaussian elimination corresponds to finding and invertible 𝐺 ∈ 𝔽𝑝×𝑝 such that
𝐺𝐴 =
𝑈
⏟
𝑢𝑝𝑝𝑒𝑟 𝑒𝑐ℎ𝑒𝑙𝑜𝑛
𝐵1 , 𝐵2 , 𝐵3 invertible ⇒ 𝐵1 𝐵2 𝐵3 is invertible.
Rank𝐴=dim ℛ𝐴
If 𝑈 upper echelon with 𝑘 pivots ⇔rank𝑈 = 𝑘
Implications
1) System of equaeions:
𝑎11 …
𝐴𝑥 = 𝑏 𝐴 = [ ⋮
𝑎𝑝1 …
𝑎1𝑞
𝑏1
⋮ ] 𝑏=[⋮ ]
𝑏𝑝
𝑎𝑝𝑞
𝑥1
Looking or vectors 𝑥 = [ ⋮ ] 𝑠. 𝑡. 𝐴𝑥 = 𝑏 (if any).
𝑥𝑞
(a) As 𝐴 left invertible, guarantees at most one solution.
To chek: Suppose 𝐴𝑥 = 𝑏, 𝐴𝑦 = 𝑏 ⇒
𝐴𝑥 − 𝐴𝑦 = 𝑏 − 𝑏 = 0
𝐴(𝑥 − 𝑦) = 0
If 𝐵 is a left inverse of 𝐴
𝑥 − 𝑦 = 𝐼(𝑥 − 𝑦) = 𝐵𝐴(𝑥 − 𝑦) = 𝐵(𝐴(𝑥 − 𝑦)) = 𝐵0 = 0
(b) 𝐴 right invertible guarantees at least one solution.
Let 𝐶 be a right inverse of 𝐴 and choose 𝑥 = 𝐶𝑏
Then 𝐴(𝐶𝑏) = (𝐴𝐶)𝑏 = 𝐼𝑝 𝑏 = 𝑏
2) If 𝐴 ∈ 𝔽𝑝×𝑞 and 𝐴 is both right invertible and left invertible then 𝑝 = 𝑞.
Earlier we showed that 𝐵, 𝐶 ∈ 𝔽𝑝×𝑞 𝑠. 𝑡.
𝐵𝐴 = 𝐼𝑞 and 𝐴𝐶 = 𝐼𝑝 then 𝐵 = 𝐶.
Rank𝐴 = 𝑘.
𝐴 left invertible ⇒ 𝑘 = 𝑞
𝐴 right invertible ⇒ 𝑘 = 𝑝
So 𝑝 = 𝑞.
3) 𝐴 ∈ 𝔽𝑝×𝑞 , 𝐺 ∈ 𝔽𝑝×𝑝 invertible, 𝐺𝐴 = 𝑈 is upper echelon.
And lwr 𝑘 = number of pivots in 𝑈.
⇒rank𝐴 =rank𝑈 = 𝑘.
Claim: The pivot columns of 𝑈 are linearly independent and form a basis to ℛ𝑈
The corresponding columns in 𝐴 are linearly independent and form a basis for ℛ𝐴
1 𝑢12 𝑢13 𝑢14
𝑈 = [0 0
2 𝑢24 ] = [𝑢1 𝑢2 𝑢3 𝑢4 ]
0 0
0
0
𝑢1 , 𝑢3 are lin independent, and span{𝑢1 , 𝑢3 } = ℛ𝑈
If 𝐺𝐴 = 𝑈, 𝐴 = [𝑎1 𝑎2 𝑎3 𝑎4 ]
claim: 𝑎1 , 𝑎3 are linearly independent and span{𝑎1 , 𝑎3 } = ℛ𝐴
Suppose we can find coefficients such that 𝛼𝑎1 + 𝛽𝑎3 = 0
𝐴 = 𝐺 −1 𝑈
[𝑎1 𝑎2 𝑎3 𝑎4 ] = [𝐺 −1 𝑢1 𝐺 −1 𝑢2 𝐺 −1 𝑢3 𝐺 −1 𝑢4 ]
𝛼𝑎1 + 𝛽𝑎3 = 𝛼𝐺 −1 𝑢1 + 𝛽𝐺 −1 𝑢3 = 0 = 𝐺 −1 (𝛼𝑢1 + 𝛽𝑢3 ) = 0
𝛼𝑢1 + 𝛽𝑢3 = 𝐺0 = 0 ⇒ 𝛼 = 𝛽 = 0 since 𝑢1 and 𝑢3 are linearly independent.
4) Related application
Given 𝑥1 , … , 𝑥𝑛 ∈ 𝔽𝑝
Find a basis for span{𝑥1 , … , 𝑥𝑛 }
Define 𝐴 = [𝑥1 𝑥2 … 𝑥𝑛 ] 𝑝 × 𝑛
Bring to upper echelon form via Gaussian elimination.
The number of pivots in 𝑈 = dim 𝑠𝑝𝑎𝑛{𝑥1 , … , 𝑥𝑛 }
and the corresponding columns will be a basis
5) Calculating inverses
Let 𝐴 be 3 × 3 and it is known to be invertible.
How to calculate its inverse?
1
0
0
𝐴𝑥1 = [0] , 𝐴𝑥2 = [1] , 𝐴𝑥3 = [0]
0
0
1
1 0 0
𝐴[𝑥1 𝑥2 𝑥3 ] = [𝐴𝑥1 𝐴𝑥2 𝐴𝑥3 ] = [0 1 0]
0 0 1
Gauss-Seidel
Do all these calculations in one shot
𝐴̃ = [𝐴
𝐼3 ]
𝐺𝐴̃ = [𝐺𝐴
𝐺 ] 𝐺𝐴 = 𝑈 upper echelon
2 4
Suppose 𝑈 = [0 3
0 0
1
2
6
6]
4
0 0
1 2 3
0 , 𝐷𝑈 = [0 1 2]
0 0 1
1
[0 0 4 ]
𝑎
1 0 −3 𝑎
𝑎 − 3𝑐
𝐹1 [𝑏] = [0 1 −2] [𝑏 ] = [𝑏 − 2𝑐 ]
𝑐
0 0 1 𝑐
𝑐
3
3−3
0
[
] [2] = [2 − 2] = [0]
1
1
1
D= 0
1
3
1 0 −3 1 2
[0 1 −2] [0 1
⏟0 0 1 0 0
3
1
2] = [0
1
0
2 0
1 0]
0 1
𝐹1 𝐷𝑈
𝐴̃ = [𝐴 𝐼3 ]
(1) Manipulate by permutations and subtracting multiplies of rows from lower rows.
(2) Multiplying through by a diagonal matrix [ 𝐷𝐺𝐴
⏟
𝐷𝐺]
𝑢𝑝𝑝𝑒𝑟 𝑡𝑟
𝐹𝐷𝐺𝐴
⏟
𝐹𝐷𝐺
(3) Subtract multiplies of rows from higher rows [ 𝐼
] inverse of 𝐴 is 𝐹𝐷𝐺.
𝑝
𝐴 ∈ 𝔽𝑝×𝑞 , 𝐴𝑇
𝐴=[
𝑖, 𝑗 entry of 𝐴 is 𝑗𝑖 entry of 𝐴𝑇
1 2
4 5
3
],
6
1
𝐴𝑇 = [2
3
Claim: rank𝐴 =rank𝐴𝑇
𝐺𝐴 = 𝑈 upper echelon
rank𝐴 =rank𝑈
(𝐺𝐴)𝑇 = 𝐴𝑇 𝐺 𝑇 = 𝑈 𝑇
Rank 𝐴𝑇 =rank𝑈 𝑇
∗ ∙ ∙ ∙
𝑈 = [0 0 ∗ ∙ ]
0 0 0 0
dim ℛ𝑈 = 2
4
5]
6
∗ 0
∙ 0
𝑈𝑇 = [
∙ ∗
∙ ∙
0
0
] , 𝑟𝑎𝑛𝑘𝑈 𝑇 =number of pivots in 𝑈=rank𝑈.
0
0
If 𝑇 is a linear transformation from a vector space 𝒰 over 𝔽 into itself (𝑥 ∈ 𝒰 ⇒ 𝑇𝑥 ∈ 𝒰)
A subspace 𝒳 of 𝒰 is said to be invariant under 𝑇 if for every 𝑥 ∈ 𝒳, 𝑇𝑥 ∈ 𝒳.
The simplest non-zero invariant subspace (if they exist) would be one dimensional spaces.
Suppose 𝒳 is one dimensional and is invariant under 𝑇, then if you take any vector 𝑥 ∈ 𝒳, 𝑇𝑥 ∈
𝒳
𝒳 = {𝛼𝑦|𝛼 ∈ 𝐴̃}
𝑇𝑥 = ⏟
𝜆𝑥
∈𝔽
In this case, 𝜆 is said to be thean Eigen value of 𝑇 and 𝑥 is said to be an Eigen vector.
Important – 𝑥 ≠ 0!!!
In other words, a vector 𝑥 is called an Eigenvector of 𝑇 if:
(1) 𝑥 ≠ 0
(2) 𝑇𝑥 = 𝜆𝑥 some 𝜆 ∈ ℂ
That 𝜆 is called an Eigenvalue of 𝑇.
If 𝑇𝑥 = 𝜆𝑥, then 𝑇2𝑥 = 𝜆2𝑥.
So there’s a “flexibility” of stretching.
There isn’t the Eigenvector.
Theorem: Let 𝒱 be a vector space over ℂ, 𝑇 a linear transformation from 𝒱 into 𝒱 and let 𝒰 be a
non-zero finite dimension subspace of 𝒱 that is invariant under 𝑇. Then, there exists a vector
𝑢 ∈ 𝒰 and a 𝜆 ∈ ℂ 𝑠. 𝑡. 𝑇𝑢 = 𝜆𝑢.
Proof:
Take any 𝑤 ∈ 𝒰, 𝑤 ≠ 0. Suppose dim 𝒰 = 𝑛
Consider 𝑤, 𝑇𝑤, … , 𝑇 𝑛 𝑤. That’s a list of 𝑛 + 1 vectors. In an 𝑛 dimensional space. Therefore,
they are linearly dependent.
Can find 𝑐0 , … , 𝑐𝑛 not all of which are zero such that 𝑐0 𝑤 + 𝑐1 𝑇𝑤 + ⋯ + 𝑐𝑛 𝑇 𝑛 𝑤 = 0
Suppose that 𝑘 is the largest index that is non-zero.
So this expression reduces to 𝑐0 𝑤 + 𝑐1 𝑇𝑤 + ⋯ + 𝑐𝑘 𝑇 𝑘 𝑤 = 0
𝑐0 𝐼 + 𝑐1 𝑇 + ⋯ + 𝑐𝑘 𝑇 𝑘 = 0
Consider the following polynomial:
𝑐0 + 𝑐1 𝑥 + ⋯ + 𝑐𝑘 𝑥 𝑘
𝑐𝑎𝑛 𝑏𝑒 𝑓𝑎𝑐𝑡𝑜𝑟𝑒𝑑
=
(𝑥 − 𝜇1 )(𝑥 − 𝜇2 ) … (𝑥 − 𝜇𝑘 ) =
𝑐𝑘 (𝑇 − 𝜇𝑘 𝐼)(𝑇 − 𝜇2 𝐼) … (𝑇 − 𝜇1 𝐼)𝑤 = 0
Possibilities Either:
(1) (𝑇 − 𝜇1 𝐼)𝑤 = 0
(2) (𝑇 − 𝜇1 )𝑤 ≠ 0 but (𝑇 − 𝜇2 𝐼){(𝑇 − 𝜇1 𝐼)𝑤} = 0
(3) (𝑇 − 𝜇2 𝐼)(𝐼 − 𝜇3 𝐼) ≠ 0 but (𝑇 − 𝜇3 𝐼){(𝑇 − 𝜇2 𝐼)(𝑇 − 𝜇1 𝐼)𝑤} = 0
(4) ⋮
(5) (𝑇 − 𝜇𝑘 𝐼) … (𝑇 − 𝜇1 𝐼)𝑤 ≠ 0 but (𝑇 − 𝜇𝑘 𝐼){(𝑇 − 𝜇𝑘 𝐼) … (𝑇 − 𝜇1 𝐼)𝑤} = 0
----- End of lesson 5
Previously – on linear algebra
Thorem: Let 𝒱 be a vector space over ℂ, T a linear transformation from 𝒱 into 𝒱, and let 𝒰 be a
finite dimensional subspace in 𝒱 that is invariant of 𝑇. (i.e. 𝑢 ∈ 𝑈, then 𝑇𝑢 ∈ 𝑈). Then there
exists a non-zero vector 𝑤 ∈ 𝒰 and a 𝜆 ∈ ℂ 𝑠. 𝑡. 𝑇𝑤 = 𝜆𝑤.
We could have just worked just with 𝒰.
Note: 𝑇𝑤 = 𝜆𝑤 ⇔ (𝑇 − 𝜆𝐼)𝑤 = 0 ⇔ 𝒩𝑇−𝜆𝐼 ≠ {0}
Could have rephrased the conclusion as:
There is a finite dimension subspace 𝒰 of 𝒱 that is invariant under 𝑇 ⇔There is a one dimension
subspace 𝒳 of 𝒰 that is invariant under 𝑇.
Implication
Let 𝐴 ∈ ℂ𝑛×𝑛 , Then there exists a point 𝜆 ∈ ℂ 𝑠. 𝑡. 𝒩𝐴−𝜆𝐼𝑛 ≠ {0}
Because the transformation 𝑇 from ℂ𝑛 into ℂ𝑛 that is defined by the formula 𝑇𝑢 = 𝐴𝑢
Proof: Take any vecto 𝑤 ∈ ℂ𝑛 , 𝑤 ≠ 0.
Consider the set of vectors {𝑤, 𝐴𝑤, … , 𝐴𝑛 𝑤}. These are 𝑛 + 1 vectors in ℂ - an 𝑛 dimensional
space.
Since they are independent, there are coefficients:
𝑐0 𝑤 + 𝑐1 𝐴𝑤 + ⋯ + 𝑐𝑛 𝐴𝑛 𝑤 = 0 such that not all coefficients are zero.
Same as to say (𝑐0 𝐼𝑛 + 𝐶1 𝐴 + ⋯ + 𝑐𝑛 𝐴𝑛 )𝑤 = 0
Let 𝑘 = max{𝑗|𝑐𝑗 ≠ 0} Claim 𝑘 ≥ 1 (trivial)
(𝑐0 𝐼𝑛 + 𝑐1 𝐴 + ⋯ + 𝑐𝑘 𝐴𝑘 )𝑤 = 0, 𝑐𝑘 ≠ 0
Claim that if we look at the ordinary polynomial:
𝑐0 + 𝑐1 𝑥 + 𝑐2 𝑥 2 + ⋯ + 𝑐𝑘 𝑥 𝑘 = 𝑐𝑘 (𝑥 − 𝜇1 )(𝑥 − 𝜇2 ) … (𝑥 − 𝜇𝑘 ), 𝑐𝑘 ≠ 0
⇒ 𝑐0 𝐼𝑛 + 𝑐1 𝐴 + ⋯ 𝑐𝑘 𝐴𝑘 = 𝑐𝑘 (𝐴 − 𝜇1 𝐼𝑛 )(𝐴 − 𝜇2 𝐼𝑛 ) … (𝐴 − 𝜇𝑘 𝐼𝑛 )
𝑐𝑘 (𝐴 − 𝜇𝑘 𝐼𝑛 ) … (𝐴 − 𝜇1 𝐼𝑛 )𝑤 = 0
𝑟𝑒𝑣𝑒𝑟𝑠𝑒 𝑡ℎ𝑒 𝑜𝑑𝑒𝑟
=
Suppose 𝑘 = 3
Either:
(1) (𝐴 − 𝜇1 𝐼𝑛 )𝑤 = 0
(2) (𝐴 − 𝜇1 𝐼𝑛 )𝑤 ≠ 0 and (𝐴 − 𝜇2 𝐼𝑛 ){(𝐴 − 𝜇1 𝐼𝑛 )𝑤} = 0
(3) (𝐴 − 𝜇2 𝐼𝑛 )(𝐴 − 𝜇1 𝐼𝑛 )𝑤 ≠ 0 and (𝐴 − 𝜇3 𝐼𝑛 )(𝐴 − 𝜇2 𝐼𝑛 )(𝐴 − 𝜇1 𝐼𝑛 )𝑤 = 0
We are looking fo 𝜆 𝑠. 𝑡. (𝐴 − 𝜆𝐼𝑛 )𝑥 = 0, 𝑥 ≠ 0
Questions: Suppose 𝐴 ∈ ℝ𝑛×𝑛
Can one guarantee that 𝐴 has at least one real Eigen value?
No!
0 1
]
−1 0
𝑎
0 1 𝑎
Looking for [
] [ ] = 𝜆 [ ] ⇔ 𝑏 = 𝜆𝑎, −𝑎 = 𝜆𝑏 ⇔ 𝑏 = −𝜆2 𝑏 ⇔ (1 + 𝜆2 )𝑏 = 0
𝑏
−1 0 𝑏
If 𝑏 = 0 ⇒ 𝑎 = 0 ⇒The entire vector is zero! Not acceptable…
So 𝜆2 + 1 = 0 i.e. 𝜆 = ±𝒾
𝐴=[
Result for 𝐴 ∈ ℂ𝑛×𝑛 says that there is always a one dimensional subspace of ℂ𝑛 that is invariant
under A.
𝐴 ∈ ℝ𝑛×𝑛 - there always exists at least one two dimensional subspace of ℝ𝑛 that is invariant
under 𝐴.
Implication:
Let 𝐴 ∈ ℂ𝑛×𝑛 - then there exists a point 𝜆 ∈ ℂ such that 𝒩𝐴−𝜆𝐼𝑛 ≠ {0}
Suppose 𝒩𝐴−𝜆𝐼𝑛 ≠ {0} for some 𝑘 distinct points in ℂ 𝜆1 , … , 𝜆𝑘 .
That is to say, there are non-zero vectors 𝑢1 , … , 𝑢𝑘 of 𝐴𝑢𝑗 = 𝜆𝑗 𝑢𝑗 1 ≤ 𝑗 ≤ 𝑘
Claim 𝑢1 , … , 𝑢𝑘 are linearly independent.
Let’s check for 𝑘 = 3
Suppose 𝑐1 𝑢1 + 𝑐2 𝑢2 + 𝑐3 𝑢3 = 0
(𝐴 − 𝜆1 𝐼𝑛 ) (𝑐1 𝑢1 + 𝑐2 𝑢2 + 𝑐3 𝑢3 ) = (𝐴 − 𝜆𝐼𝑛 )0 = 0
Let’s break it up:
(𝐴 − 𝛼𝐼𝑛 )𝑢𝑗 = 𝐴𝑢𝑗 − 𝛼𝑢𝑗 = 𝜆𝑗 𝑢𝑗 − 𝛼𝑢𝑗 = (𝜆𝑗 − 𝛼)𝑢𝑗
Continue with our calculation:
𝑐1 (𝜆1 − 𝜆1 )𝑢1 + 𝑐2 (𝜆2 − 𝜆1 )𝑢2 + 𝑐3 (𝜆3 − 𝜆1 )𝑢3 = 𝑐2 (𝜆2 − 𝜆1 )𝑢2 + 𝑐3 (𝜆3 − 𝜆1 )𝑢3
We’ve reduced the problem!
(we know the lambda’s are different so the rest of the coefficients are not zero)
We can do it again to get
(𝐴 − 𝜆2 𝐼𝑛 )(𝑐2 (𝜆2 − 𝜆1 )𝑢2 + 𝑐3 (𝜆3 − 𝜆1 )𝑢3 ) =
𝑐2 (𝜆2 − 𝜆1 )(𝜆2 − 𝜆2 )𝑢2 + 𝑐3 (𝜆3 − 𝜆1 )(𝜆3 − 𝜆2 )𝑢3 = 𝑐3 (𝜆3 − 𝜆1 )(𝜆3 − 𝜆2 )𝑢3
Now 𝑐3 must be zero since the other scalars and vectors in the expression are non-zero.
We can in fact repeat the process to knock out 𝑐2 and also 𝑐1 .
This can also be generalized for 𝑛 other than 3.
𝐴𝑢𝑗 = 𝜆𝑗 𝑢𝑗 𝑗 = 1, … , 𝑘 𝜆𝑖 ≠ 𝜆𝑗 if 𝑖 ≠ 𝑗
𝜆1
𝐴[𝑢1 … 𝑢𝑘 ] = [𝐴𝑢1 … 𝐴𝑢𝑘 ] = [𝜆1 𝑢1 … 𝜆𝑘 𝑢𝑘 ] = [𝑢1 … 𝑢𝑘 ] [
𝜆2
]
⋱
𝜆𝑘
𝐴 ∈ ℝ𝑛×𝑛 , then there is at least one dimensional subspace of ℂ𝑛 that is invariant under 𝐴.
Translate this to: There is 𝑢 ∈ ℂ𝑛 , 𝜆 ∈ ℂ 𝑠. 𝑡. 𝐴𝑢 = 𝜆𝑢
𝑢 = 𝑥 + 𝒾𝑦, 𝜆 = 𝛼 + 𝒾𝛽
𝐴(𝑥 + 𝒾𝑦) = (𝛼 + 𝒾𝛽)(𝑥 + 𝒾𝑦) = (𝛼𝑥 − 𝛽𝑦) + 𝒾(𝛽𝑥 + 𝛼𝑦)
𝐴 ∈ 𝑅 𝑛×𝑛
𝐴𝑥 = 𝛼𝑥 − 𝛽𝑦 𝐴𝑦 = 𝛽𝑥 + 𝛼𝑦
𝒲 = 𝑠𝑝𝑎𝑛{𝑥, 𝑦} with real coefficients
𝑤 ∈ 𝒲 ⇒ 𝐴𝑤 ∈ 𝒲
𝑤 = 𝑎𝑥 + 𝑏𝑦
𝐴𝑤 = 𝑎𝐴𝑥 + 𝑏𝐴𝑦 = 𝑎(𝛼𝑥 + 𝛽𝑦) + 𝑏(𝛽𝑥 + 𝛼𝑦) = (𝑎𝛼 + 𝑏𝛽)𝑥 + (−𝑎𝛽 + 𝑏𝛼)𝑦
Example:
If 𝐴 ∈ ℂ𝑛×𝑛 with 𝑘 distinct Eigenvalues, then 𝑘 ≤ 𝑛.
Because, 𝑢1 , … , 𝑢𝑘 Eigenvectors are linearly independent. And they sit inside an 𝑛 dimensional
space ℂ𝑛 .
The matrix of the Eigenvectors:
[𝑢1 … 𝑢𝑘 ] is an 𝑛 × 𝑘 matrix with rank 𝑘.
𝜆1
If 𝑘 = 𝑛, then 𝐴[𝑢1 … 𝑢𝑛 ] = [𝑢1 … 𝑢𝑛 ] [
𝜆2
]
⋱
𝜆𝑘
𝐴𝑈 = 𝑈𝐷
𝑈 is invertible. (Rank k)
So we can rewrite this as: 𝐴 = 𝑈𝐷𝑈 −1
If you need to raise some matrix 𝐴 to the power of 100
Suppose: 𝐴 = 𝑈𝐷𝑈 −1 , 𝐴2 = 𝑈𝐷𝑈 −1 𝑈𝐷𝑈 −1 = 𝑈𝐷 2 𝑈 −1
So 𝐴100 = 𝑈𝐷100 𝑈 −1
𝛼 100
𝛽100
𝐷100 = [
⋱
]
𝛾100
Sums of Subspaces
Let 𝒰, 𝒱 be subspaces of a vector space 𝒲 over 𝔽.
𝒰 + 𝒱 = {𝑢 + 𝑣|𝑢 ∈ 𝒰, 𝑣 ∈ 𝒱}
Claim: 𝒰 + 𝒱 is a vector space.
(𝑢1 + 𝑣1 ) + (𝑢2 + 𝑣2 ) = ⏟
(𝑢1 + 𝑢2 ) + ⏟
(𝑣1 + 𝑣2 )
∈𝒰
∈𝒱
Lemma:
Let 𝒰, 𝒱 subspaces of 𝒲 a finite dimensional vector space
dim(𝒰 + 𝒱) = dim 𝒰 + dim 𝒱 − dim(𝒰 ∩ 𝒱)
Another claim: \𝒰 ∩ 𝒱 is a vector space.
Proof: Suppose {𝑤1 , … , 𝑤𝑘 } is a basis for 𝒰 ∩ 𝒱
𝒰∩𝒱 ⊆𝒰
if really 𝒰 ∩ 𝒱 ≠ 𝒰
{𝑤1 , … , 𝑤𝑘 , 𝑢1 , … , 𝑢𝑠 } basis for 𝒰
𝒰 ∩ 𝒱 ⊆ 𝒱. Suppose 𝒰 ∩ 𝒱 ≠ 𝒱
{𝑤1 , … , 𝑤𝑘 , 𝑣1 , … , 𝑣𝑡 } is a basis for 𝒱
Claim: {𝑤1 , … , 𝑤𝑘 , 𝑢1 , … , 𝑢𝑠 , 𝑣1 , … , 𝑣𝑡 } basis for 𝒰 + 𝒱
Need to show:
(a) 𝑢 + 𝑣 is a linear combination of these
𝑢 = 𝑐1 𝑤1 + ⋯ + 𝑐𝑘 𝑤𝑘 + 𝑑1 𝑢1 + ⋯ + 𝑑𝑠 𝑢𝑠
𝑣 = 𝑎1 𝑤1 + ⋯ + 𝑎𝑘 𝑤𝑘 + 𝑏1 𝑣1 + ⋯ + 𝑏𝑡 𝑣𝑡
𝑢+𝑣 =
(𝑎1 + 𝑐1 )𝑤1 + ⋯ + (𝑎𝑘 + 𝑐𝑘 )𝑤𝑘 + ⋯ 𝑑1 𝑢1 + ⋯ + 𝑑𝑠 𝑢𝑠 + ⋯ + 𝑏1 𝑣1 + ⋯ + 𝑏𝑡 𝑣𝑡
(b) Show {𝑤1 … 𝑤𝑘 , 𝑢1 , … , 𝑢𝑠 , 𝑣1 , … , 𝑣𝑡 } are linearly independent.
Claim: b is correct.
Denote 𝑣 = 𝑐1 𝑣1 + ⋯ + 𝑐𝑡 𝑣𝑡
Suppose:
𝑎1 𝑤1 + ⋯ + 𝑎𝑘 𝑤𝑘 + 𝑏1 𝑢1 + ⋯ + 𝑏𝑠 𝑢𝑠 + 𝑐1 𝑣 + ⋯ + 𝑐𝑡 𝑣𝑡 = 0
𝑎1 𝑤1 + ⋯ + 𝑎2 𝑤𝑘 + 𝑏1 𝑢1 + ⋯ + 𝑏𝑠 𝑢𝑠 = −𝑣
⏟
⏟
∈𝒰
∈𝒱
We don’t know who −𝑣 is, but it is definitely in 𝒱 ∩ 𝒰!
It has to be expressible this way:
(−𝑐1 )𝑣1 + ⋯ + (−𝑐𝑡 )𝑣𝑡 = 𝛼1 𝑤1 + ⋯ + 𝛼𝑘 𝑤𝑘 ⇒ 𝑐𝑖 = 0
dim(𝒰 + 𝒱) = 𝑘 + 𝑠 + 𝑡 = (𝑘 + 𝑠) + (𝑘 + 𝑡) − 𝑘 = dim 𝒰 + dim 𝒱 − dim 𝒰 ∩ 𝒱
----- End of lesson 6
𝒰, 𝒱 subspaces of a vector space 𝒲 over 𝔽.
𝒰 + 𝒱 = {𝑢 + 𝑣|𝑢 ∈ 𝒰, 𝑣 ∈ 𝒱}
𝒰 ∩ 𝒱 = {𝑥|𝑥 ∈ 𝒰, 𝑥 ∈ 𝒱}
Both of these are vector spaces.
We established that a dimension dim 𝒰 + 𝒱 = dim 𝒰 + dim 𝒱 − dim 𝒰 ∩ 𝒱
The sum 𝒰 + 𝒱 is said to be direct (called a direct sum) if dim 𝒰 + 𝒱 = dim 𝒱 + dim 𝒰
(consequence, the sum 𝒰 + 𝒱 is a direct sum ⇔ 𝒰 ∩ 𝒱 = {0})
In a direct sum, the decomposition is unique.
If 𝑥 = 𝑢1 + 𝑣1 , such that 𝑢1 ∈ 𝒰, 𝑣1 ∈ 𝒱 and also 𝑥 = 𝑢2 + 𝑣2 such that 𝑢2 ∈ 𝒰, 𝑣2 ∈ 𝒱 then
𝑢1 = 𝑢2 and 𝑣1 = 𝑣2 .
Why?
𝑢1 + 𝑣1 = 𝑢2 + 𝑣2
𝑢1 − 𝑢2 = ⏟
𝑣1 − 𝑣2
⏟
∈𝒰
∈𝒱
So 𝑢1 − 𝑢2 ∈ 𝒰 ∩ 𝒱. From our assumption, 𝑢1 − 𝑢2 = 0.
Similarly, 𝑣2 − 𝑣1 = 0.
This means 𝑢1 = 𝑢2 and 𝑣1 = 𝑣2 .
𝒱1 , … , 𝒱𝑘 subspaces of a vector space 𝒲 then
𝒰1 + ⋯ + 𝒰𝑘 = {𝑢1 + ⋯ + 𝑢𝑘 |𝑢𝑗 = 𝒰𝑗 , 𝑗 = 1 … 𝑘}
𝒰1 + ⋯ + 𝒰𝑘 is said to be direct if dim 𝒰1 + ⋯ + 𝒰𝑘 = dim 𝒰1 + ⋯ + dim 𝒰𝑘
It’s very tempting to jump to the conclusion (for example) that
𝒰1 + 𝒰2 + 𝒰3 is a direct sum ⇔ 𝒰1 ∩ 𝒰2 = {0}, 𝒰1 ∩ 𝒰3 = {0}, 𝒰2 ∩ 𝒰3 = {0}
However, this is not enough.
1
0
1
Consider 𝒰1 = 𝑠𝑝𝑎𝑛 {[0]} , 𝒰2 = 𝑠𝑝𝑎𝑛 {[1]} , 𝒰3 = 𝑠𝑝𝑎𝑛 {[1]}
0
0
0
𝒰1 ∩ 𝒰2 = {0}, 𝒰1 ∩ 𝒰3 = {0}, 𝒰2 ∩ 𝒰3 = {0}
dim 𝒰1 + dim 𝒰2 + dim 𝒰3 = 3
But dim 𝒰1 + 𝒰2 + 𝒰3 = 2
So forget the false conclusions, and just remember dim 𝒰1 + 𝒰2 = dim 𝒰1 + dim 𝒰2.
Generally we say 𝒰1 + ⋯ + 𝒰𝑘 is direct ⇔ every collection of non-zero vectors, at most one
from each subspace is linearly independent.
Suppose 𝑘 = 4. 𝒰, 𝒱, 𝒳, 𝒴. If the sum 𝒰 + 𝒱 + 𝒳 + 𝒴 is direct, then all non-zero 𝑢 ∈ 𝒰, 𝑣 ∈
𝒱, 𝑥 ∈ 𝒳, 𝑦 ∈ 𝒴 ⇒ {𝑢, 𝑣, 𝑥, 𝑦} are linearly independent.
Every 𝐴 ∈ ℂ𝑛×𝑛 has at least one Eigenvalue. i.e. there is a point 𝜆 ∈ ℂ and a vector 𝑥 ≠ 0 such
that 𝐴𝑥 = 𝜆𝑥 (same as saying (𝐴 − 𝜆𝐼)𝑥 = 0
That is equivalent to saying 𝒩(𝐴−𝜆𝐼) ≠ {0}
Suppose 𝐴 has 𝑘 distinct Eigenvalues 𝜆1 , … , 𝜆𝑘 ∈ ℂ. We shoed that if 𝐴𝑢𝑗 = 𝜆𝑗 𝑢𝑗 𝑗 = 1 … 𝑘 then
𝑢1 , … , 𝑢𝑘 are linearly independent,.
Same as to say 𝒩(𝐴−𝜆1 𝐼) + ⋯ + 𝒩(𝐴−𝜆𝑘 𝐼) is a direct sum.
Same as to say dim 𝒩(𝐴−𝜆1 𝐼) + ⋯ + 𝒩(𝐴−𝜆𝑘 𝐼) = dim 𝒩(𝐴−𝜆1 𝐼) + ⋯ + dim 𝒩(𝐴−𝜆𝑘 𝐼)
Criteria: A matrix 𝐴 ∈ ℂ𝑛×𝑛 is diagonizable ⇔ can find 𝑛 linearly independent Eigenvectors.
Same as to say dim 𝒩(𝐴−𝜆1 𝐼) + ⋯ + dim 𝒩(𝐴−𝜆𝑘𝐼) = 𝑛.
𝐴5×5 , 𝜆1 , … , 𝜆5 distinct Eigenvalues. 𝐴𝑢𝑗 = 𝜆𝑗 𝑢𝑗 , 𝑗 = 1, … ,5 𝑢𝑗
𝑈
𝜆1
⏞
𝐴 [𝑢1 , … , 𝑢_5 ] = [𝐴𝑢1 … 𝐴𝑢5 ] = [𝜆1 𝑢1 … 𝜆5 𝑢5 ] = [𝑢1 … 𝑢5 ] [ 0
0
≠0
0 0
⋱ 0]
0 𝜆5
𝐴𝑈 = 𝑈𝐷
Additional fact: 𝑢1 , … , 𝑢5 are linearly independent. Therefore, 𝒰 is invertible. dim ℛ𝑈 = 5. 𝑈 is
a 5 × 5 matrix.
So 𝐴 = 𝑈𝐷𝑈 −1
dim
⏟ 𝒩(𝐴−𝜆𝑗𝐼5 ) ≥ 1
𝛾𝑗
𝛾1 + 𝛾2 + ⋯ + 𝛾5 ≤ 5 ⇒ 𝛾1 = 𝛾2 = ⋯ = 𝛾5 = 1
𝐴 is a 5 × 5 matrix.
2 distinct Eigenvalues 𝜆1 , 𝜆2 . 𝜆1 ≠ 𝜆2
Crucial issue is the dimension of the null spaces: dim 𝒩(𝐴−𝜆1 𝐼) = 3 and dim 𝒩(𝐴−𝜆2 𝐼) = 2
⇒ 𝐴𝑢1 = 𝜆1 𝑢1 , 𝐴𝑢2 = 𝜆1 𝑢2 , 𝐴𝑢3 = 𝜆1 𝑢3 , 𝐴𝑢4 = 𝜆2 𝑢4 , 𝐴𝑢5 = 𝜆2 𝑢5
{𝑢1 , 𝑢2 , 𝑢3 } a basis for 𝒩(𝐴−𝜆1 𝐼)
{𝑢4 , 𝑢5 } a basis for 𝒩(𝐴−𝜆2 𝐼)
𝜆1 0 0 0 0
0 𝜆1 0 0 0
𝑢
𝑢
𝑢
𝑢
𝑢
𝑢
𝑢
𝑢
𝑢
𝑢
0 𝜆1 0 0
𝐴[ 1
2
3
4
5] = [ 1
2
3
4
5] 0
0 0 0 𝜆2 0
[ 0 0 0 0 𝜆2 ]
(1) Find Engenvalues 𝜆1 , … , 𝜆𝑘 of 𝐴 (distinct Eigenvalues)
(2) Find basis for each space 𝒩(𝐴−𝜆𝑖𝐼) , 𝑖 = 1 … 𝑘
(3) Stack resulting vectors 𝐴𝑈 = 𝑈𝐷
(columns of 𝑈 are taken from basis 𝒩(𝐴−𝜆𝑖𝐼) , 𝑖 = 1 … 𝑘 and always linearly
independent)
Question: Do you always have enough columns? No. 𝑈 could be non invertible, and then 𝐴 is not
diagonizable.
Example:
2 1 0
𝐴 = [0 2 1]
0 0 2
2−𝜆
1
0
(𝐴 − 𝜆𝐼3 ) = [ 0
2−𝜆
0 ]
0
0
2−𝜆
(𝐴 − 𝜆𝐼3 ) is invertible (nullspace is {0}) if 𝜆 ≠ 2, not invertible (nullspace is not {0}) if 𝜆 = 2.
0 1 0
𝐴 − 2𝐼3 = [0 0 1]
0 0 0
Let’s look for vectors in the null space:
0
[0
0
1 0 𝛼
𝛽
0 1] [𝛽 ] = [ 𝛾 ] = 0
0 0 𝛾
0
1
So 𝛽 = 𝛾 = 0 ⇒the nullspace is 𝑠𝑝𝑎𝑛 {[0]}
0
Only one Eigenvector!
1 0 0
𝑈 = [ 0 0 0]
0 0 0
2
0
Suppose 𝐴 = 0
0
[0
0
0
𝐴 − 2𝐼 = 0
[
1
2
0
0
0
1
0
0
0
1
2
0
0
0
0
0
3
0
0
2−𝜆
0
0
,
𝐴
−
𝜆𝐼
=
0
0
5
1
0
[ 0
3]
0
1
0
,
1
0
𝒩𝐴−2𝐼
1
1]
Remember: 𝐵𝑝×𝑞 : 𝑞 = dim 𝒩𝐵 + dim ℛ𝐵
1
2−𝜆
0
0
0
1
0
= 𝑠𝑝𝑎𝑛 0
0
{[0]}
0
1
2−𝜆
0
0
0
0
0
3−𝜆
0
0
0
0
1
3 − 𝜆]
−1 1
0 0 0
0 −1 1 0 0
𝐴 − 3𝐼 = 0
0 −1 0 0 ,
0
0
0 0 1
[0
0
0 0 0]
0
0
(𝐴 − 2𝐼)2 = 0
0
[0
0
0
0
0
0
1
0
0
0
0
0
0
0
1
0
0
0
0 ,
2
1]
𝒩𝐴−3𝐼
[
𝐴11
0
0
0
= 𝑠𝑝𝑎𝑛 0
1
[
{ 0]}
𝐴𝑘
0 𝑘
] = [ 11
𝐴22
0
0
]
𝐴𝑘22
𝒩(𝐴−2𝐼)2 = 𝑠𝑝𝑎𝑛{𝑒1 , 𝑒2 } – added eigenvector
0
0
(𝐴 − 2𝐼)3 = 0
0
[0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
0
0
0
∗
1]
𝒩(𝐴−2𝐼)2 = 𝑠𝑝𝑎𝑛{𝑒1 , 𝑒2 , 𝑒3 } - added eigenvector
0
0
(𝐴 − 2𝐼)𝑘 , 𝑘 ≥ 4 will still be something of the form 0
0
[0
eigenvectors in the following powers.
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
0
0
0 so we won’t get any new
∗
1]
𝒩𝐴−3𝐼 = 𝑠𝑝𝑎𝑛{𝑒4 }
𝒩(𝐴−3𝐼)2 = 𝑠𝑝𝑎𝑛{𝑒4 , 𝑒5 }
Summary: dim 𝒩(𝐴−2𝐼)3 + dim 𝒩(𝐴−3𝐼)2 = dim 𝒩(𝐴−2𝐼)5 + dim 𝒩(𝐴−3𝐼)5 = 5
1) 𝐵 ∈ ℂ𝑛×𝑛 , 𝒩𝐵 ⊆ 𝒩𝐵2 ⊆ ⋯
But there is a saturation! If 𝒩𝐵𝑘 = 𝒩𝑏𝑘+1 then 𝒩𝐵𝑘+1 = 𝒩𝑏𝑘+2 and so on.
Note: 𝒩𝐵𝑗 = 𝒩𝐵𝑘 ∀𝑗 ≥ 𝑘
If 𝑥 ∈ 𝒩𝐵𝑗 ⇒ 𝐵 𝑗+1 𝑥 = 𝐵(𝐵 𝑗 𝑥) = 𝐵0 = 0
Translation: 𝑥 ∈ 𝒩𝐵𝑗+1 i.e. 𝒩𝐵𝑗 ⊆ 𝒩𝐵𝑗+1
Suppose 𝒩𝐵𝑘 = 𝒩𝐵𝑘+1 . Take 𝑥 ∈ 𝒩𝐵𝑘+2 ⇒ 𝐵𝑘+2 𝑥 = 0 ⇒ 𝐵𝑘+1 (𝐵𝑥) = 0 ⇒
𝐵𝑥 ∈ 𝒩𝐵𝑘+1
But we assumed 𝒩𝐵𝑘+1 = 𝒩𝐵𝑘 ⇒ 𝐵𝑥 ∈ 𝒩𝐵𝑘 ⇒ 𝐵𝑘+1 𝑥 = 0 ⇒ 𝑥 ∈ 𝒩𝐵𝑘+1
2) 𝐵𝑘 𝑥 = 0 ⇒ 𝐵𝑛 𝑥 = 0 always.
suppose 𝑛 = 5, 𝐵 5 × 5, 𝐵3 𝑥 = 0 ⇒ 𝐵5 𝑥 = 0 (easy)
Interesting 𝐵6 𝑥 = 0 ⇒ 𝐵5 𝑥 = 0
Because if it is not true ⇒ 𝒩𝐵5 ⊊ 𝒩𝐵6 ⇒ Saturation after 5. ⇒ dim 𝒩𝐵6 ≥ 6. But 5 ×
5 matrices can’t have such degree!
Let 𝐴 ∈ ℂ𝑛×𝑛 with 𝑘 distinct eigenvalues.
i.e. 𝒩𝐴−𝜆𝐼𝑛 ≠ 0 ⇔ 𝜆 = 𝜆1 or 𝜆 = 𝜆2 or … 𝜆 = 𝜆𝑘
𝛾𝑗 = dim 𝒩(𝐴−𝜆𝑗𝐼𝑛 ) called the geometric multiplicity
𝛼𝑗 = dim 𝒩(𝐴−𝜆
𝑛
𝑗 𝐼𝑛 )
called the algebraic multiplicity
It’s clear that 𝛾𝑗 ≤ 𝛼𝑗 ⇒ 𝛾1 + ⋯ + 𝛾𝑘 ≤ 𝛼1 + ⋯ + 𝛼𝑘
3) 𝒩𝐵𝑛 ∩ ℛ𝐵𝑛 = {0}
Let 𝑥 ∈ 𝒩𝐵𝑛 ∩ ℛ𝐵𝑛
𝑛𝑜𝑡 𝑜𝑏𝑣𝑖𝑜𝑢𝑠 𝑎𝑛𝑑 𝑤𝑖𝑙𝑙
𝑏𝑒 𝑝𝑟𝑜𝑣𝑒𝑑 𝑙𝑎𝑡𝑒𝑟 𝑜𝑛
=
𝐵𝑦 2
𝐵𝑛 𝑥 = 0, 𝑥 = 𝐵𝑛 𝑦 ⇒ 𝐵𝑛 𝐵𝑛 𝑦 = 0 ⇒ 𝑦 ∈ 𝒩𝐵2𝑛 ⇒
4) ℂ𝑛 = 𝒩𝐵𝑛 + ℛ𝐵𝑛 and this sum is direct
3 implies that the sum is direct. That means:
dim(𝒩𝐵𝑛 + ℛ𝐵𝑛 ) = dim 𝒩𝐵𝑛 + dim ℛ𝐵𝑛
𝑛
𝑦 ∈ 𝒩𝐵 𝑛 ⇒ 𝐵 𝑛 𝑦 = 0 ⇒ 𝑥 = 0
𝑐𝑜𝑛𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛 𝑜𝑓 𝑑𝑖𝑚𝑒𝑛𝑠𝑖𝑜𝑛𝑠
=
𝑛
Remark: Is it true that the dim(𝒩𝐶 + ℛ𝐶 ) = dim 𝒩𝐶 + dim ℛ𝐶 for any square matrix 𝐶?
Answer: NO!
0 1
1
1
Consider: 𝐶 = [
], 𝒩𝐶 = 𝑠𝑝𝑎𝑛 {[ ]} , ℛ𝐶 = 𝑠𝑝𝑎𝑛 {[ ]}
0 0
0
0
dim 𝒩𝐶 + dim ℛ𝐶 = 2 ≠ dim 𝒩𝐶 + ℛ𝐶 = 1
---- end of lesson 7
𝐴 ∈ ℂ𝑛×𝑛
𝜆 Eigenvalue means 𝒩𝐴−𝜆𝐼𝑛 ≠ {0}
𝐴 has at least one Eigenvalue in ℂ.
If 𝐴 has 𝑘 distinct Eigenvalues, then
𝛾𝑗 = dim 𝒩𝐴−𝜆𝐼𝑛 - Geometric multiplicity
𝛼𝑗 = dim 𝒩(𝐴−𝜆𝐼𝑛 )𝑛 – Algebric multiplicity
𝑂𝐵𝐽𝐸𝐶𝑇𝐼𝑉𝐸
𝛾𝑗 ≤ 𝛼𝑗 always: 𝛾1 + ⋯ + 𝛾𝑘 ≤ 𝛼1 + ⋯ + 𝛼𝑘
=
𝑛
EX. 𝐴 5 × 5 2 distinct Eigenvalues 𝜆1 , 𝜆2
If 𝛾1 + 𝛾2 = 𝑛 (equivalent to statement 𝒩𝐴−𝜆1 𝐼5 + 𝒩𝐴−𝜆2 𝐼5 = ℂ5
Then 𝐴[𝑢1 … 𝑢5 ] = [𝑢1 … 𝑢5 ]𝐷
(5 linearly independent Eigenvectors and 𝐷 is diagonal and 𝑈 is invertible)
If 𝛾1 = 3 and 𝛾2 = 2
Then: 𝐴[𝑢1 … 𝑢5 ] = [𝜆1 𝑢1 𝜆1 𝑢2 𝜆1 𝑢3 𝜆2 𝑢4 𝜆2 𝑢5 ]
𝜆1
𝜆1
[𝑢1 𝑢2 𝑢3 𝑢4 𝑢5 ]
𝜆1
𝜆2
[
𝜆2 ]
𝑥 ∈ ℂ5
5
𝑥 = ∑ 𝑒𝑗 𝑢𝑗
1
𝐴𝑥 = 𝐴(𝑐1 𝑢1 + ⋯ + 𝑐5 𝑢5 ) = 𝑐1 𝜆1 𝑢1 + 𝑐2 𝜆1 𝑢2 + 𝑐3 𝜆1 𝑢3 + 𝑐4 𝜆2 𝑢4 + 𝑐5 𝜆2 𝑢5
Will show that 𝑎 ∈ 𝐶 𝑝×𝑝 with 𝑘 distinct Eigenvalues 𝜆1 , … , 𝜆𝑘 can find invertible 𝑈𝑛×𝑛 and
upper triangular 𝑛 × 𝑛 𝐽 of special form such that 𝐴𝑈 = 𝑈𝐽
𝐵𝜆1 |
−
+
−
+
| 𝐵𝜆2 |
𝐽=
+
−
+
⋱
[
𝐵𝜆𝑘 ]
𝛾𝑗 cells in 𝐵𝜆𝑗
Theorem: 𝐴 ∈ 𝐶 𝑛×𝑛 , 𝜆1 , … , 𝜆𝑘 distinct Eigenvalues, then:
𝐶 𝑛 = 𝒩(𝐴−𝜆1 𝐼𝑛)𝑛 + ⋯ + 𝒩(𝐴−𝜆𝑘 𝐼𝑛)𝑛 and the sum is direct.
Suppose 𝑘 = 3
If e.g. 𝑘 = 3
{𝑢1 , … , 𝑢𝑟 } basis for 𝒩(𝐴−𝜆1 𝐼𝑛)𝑛
{𝑣1 , … , 𝑣𝑟 } basis for 𝒩(𝐴−𝜆2 𝐼𝑛)𝑛
{𝑤1 , … , 𝑤𝑟 } basis for 𝒩(𝐴−𝜆3 𝐼𝑛)𝑛
Then {𝑢1 , … , 𝑢𝑟 , 𝑣1 , … , 𝑣𝑠 , … , 𝑤1 , … , 𝑤𝑡 } is a basis for ℂ𝑛 .
1) 𝐵 ∈ 𝐶 𝑛×𝑛 , ℛ𝐵𝑛 ∩ 𝒩𝐵𝑛 = {0}
2) 𝐶 𝑛 = 𝒩𝐵𝑛 + ℛ𝐵𝑛 and this sum is direct
3) If 𝛼 ≠ 𝛽, then 𝒩(𝐴−𝛼𝐼𝑛 )𝑛 ⊆ ℛ(𝐴−𝛽𝐼𝑛 )𝑛
Binomial theorem:
𝑛
𝑛
𝑎, 𝑏 ∈ ℂ ⇒ (𝑎 + 𝑏) = ∑ ( 𝑗 ) 𝑎 𝑗 𝑏 𝑛−𝑗
𝑛
𝑗=0
2
2
2
(𝑎 + 𝑏)2 = ( ) 𝑎0 𝑏 2 + ( ) 𝑎𝑏 + ( ) 𝑎2 𝑏 0 = 𝑏 2 + 2𝑎𝑏 + 𝑎2
0
1
2
Can we do something similar for matrices?
2
2
2
(𝐴 + 𝐵)𝑛 = ( ) 𝐴0 𝐵2 + ( ) 𝐴𝐵 + ( ) 𝐴2 𝐵0 = 𝐵2 + 2𝐴𝐵 + 𝐴2
0
1
2
(𝐴 + 𝐵)2 = (𝐴 + 𝐵)𝐴 + (𝐴 + 𝐵)𝐵 = 𝐴2 + 𝐵𝐴 + 𝐴𝐵 + 𝐵2
AB=BA
Correct only if 𝐴𝐵 = 𝐵𝐴
We accept it as correct for 𝑛 though we can verify it ourselves.
𝐴 − 𝛼𝐼𝑛 = 𝐴 − 𝛽𝐼𝑛 + (𝛽 − 𝛼)𝐼𝑛
𝐴 − 𝛽𝐼𝑛 commutative with (𝛽 − 𝛼)𝐼𝑛
𝑛
(𝐴 − 𝛼𝐼𝑛
)𝑛
𝑛
= ((𝐴 − 𝛽𝐼𝑛 ) + (𝛽 − 𝛼)𝐼𝑛 ) = ∑ ( 𝑗 ) (𝐴 − 𝛽𝐼𝑛 )𝑘 ((𝛽 − 𝛼 )𝐼𝑛 )𝑛−𝑗
𝑛
𝑗=0
𝑛
𝑛
𝑛−𝑗
𝑛
= ( ) (𝐴 − 𝛽𝐼𝑛 )0 (𝛽 − 𝛼)𝑛 𝐼𝑛 + ∑ ( 𝑗 ) (𝐴 − 𝛽𝐼𝑛 )𝑗 ((𝛽 − 𝛼)𝐼𝑛 )
0
𝑗=1
𝑥 ∈ 𝒩(𝐴−𝛼𝐼𝑛 )𝑛 ⇒ (𝐴 − 𝛼𝐼𝑛 )𝑛 𝑥 = 0
𝑛
𝑛
0 = (𝛽 − 𝛼) 𝑥 + ∑ ( 𝑗 ) (𝛽 − 𝛼)𝑛−𝑗 (𝐴 − 𝛽𝐼𝑛 )𝑗 𝑥
𝑛
𝑗=1
𝑛
𝑥=
−1
𝑛
∑ ( 𝑗 ) (𝛽 − 𝛼)𝑛−𝑗 (𝐴 − 𝛽𝐼𝑛 )𝑗 𝑥 =
𝑛
(𝛽 − 𝛼)
𝑗=1
𝑛
−1
𝑛
(𝐴 − 𝐵𝐼𝑛 ) {
∑ ( 𝑗 ) (𝛽 − 𝛼)𝑛−𝑗 (𝐴 − 𝛽𝐼𝑛 )𝑗 𝑥 } 𝑥
(𝛽 − 𝛼)𝑛
𝑗=1
𝑥 = (𝐴 − 𝛽𝐼𝑛 )𝑃(𝐴)𝑥
So we can replace 𝑥 with it’s polynomial:
(𝐴 − 𝛽𝐼𝑛 )𝑃(𝐴)2 𝑥 = (𝐴 − 𝛽𝐼𝑛 )𝑛 𝑃(𝐴)𝑛 𝑥 = (𝐴 − 𝛽𝐼𝑛 )𝑛 ⇒ 𝑥 ∈ ℛ(𝐴−𝛽𝐼𝑛 )𝑛
4) If 𝒲 = 𝒰 + 𝒱 and the sum is direct
Let 𝒳 be a subspace of 𝒲, then:
𝒳 ∩ 𝒲 = 𝒳 ∩ 𝒰 + 𝒳 ∩ 𝒱 this is not always true!
ℝ2 = 𝒲
1
𝒰 = 𝑠𝑝𝑎𝑛 {[ ]} ,
0
0
𝒱 = 𝑠𝑝𝑎𝑛 {[ ]}
1
1
𝒳 = 𝑠𝑝𝑎𝑛 {[ ]}
1
𝒳 ∩ 𝑈 = {0}
𝒳 ∩ 𝒱 = {0}
𝒳 ≠ 𝒳∩𝒰+𝒳∩𝒱
Not always true!
It is true if 𝒰 is a subset of 𝒳 or 𝒱 is a subset of 𝒳
Let 𝒰 ⊆ 𝒳, 𝑥 ∈ 𝒳, 𝒳 ⊆ 𝒲 ⇒
𝑥 ∈ 𝒲 ⇒ 𝑥 = 𝑢 + 𝑣 some 𝑢 ∈ 𝒰, 𝑣 ∈ 𝒱
If also 𝒰 ⊆ 𝒳 ⇒ 𝑢 ∈ 𝒳 ⇒ 𝑣 = 𝑥 − 𝑢 ∈ 𝒳
𝒳 = 𝒰∩𝒳+𝒱∩𝒳
5) ℂ𝑛 = 𝒩(𝐴−𝜆1 𝐼𝑛 )𝑛 + ⋯ + 𝒩(𝐴−𝜆𝑘 𝐼𝑛 )𝑛 sum is direct.
For simplicity fix 𝑘 = 3
To simplify the writing 𝒩𝑗 = 𝒩(𝐴−𝜆
𝑛
𝑗 𝐼𝑛 )
ℛ𝑗 = ℛ(𝐴−𝜆
𝑛
𝑗 𝐼𝑛 )
Wish to show ℂ𝑛 = 𝒩1 + 𝒩2 + 𝒩3 (sum is direct)
(2) ⇒
1- 𝐶 𝑛 = 𝒩1 ∔ ℛ1
ℂ𝑛 = 𝒩2 ∔ ℛ2
ℂ𝑛 = 𝒩3 ∔ ℛ3
∔ means the sum is direct
(3) ⇒ 𝒩2 ⊆ ℛ1
ℂ𝑛 = 𝒩2 ∔ ℛ2
2- [ℛ1 = ℛ1 ∩ ℂ𝑛 = ℛ1 ∩ 𝒩2 ∔ ℛ1 ∩ ℛ2 = 𝒩2 ∔ ℛ1 ∩ ℛ2 ]
3- [ℛ1 ∩ ℛ2 = ℛ1 ∩ ℛ2 ∩ 𝒩3 ∔ ℛ1 ∩ ℛ2 ∩ ℛ3 = 𝒩3 ∔ ℛ1 + ℛ2 + ℛ3 ]
ℂ𝑛 = 𝒩3 ∔ ℛ3
𝒩3 ⊆ ℛ2 , 𝒩3 ⊆ ℛ1 ⇒ 𝒩3 ⊆ ℛ1 ∩ ℛ2
ℂ𝑛 = 𝒩1 ∔ (𝒩2 ∔ ℛ1 ∩ ℛ2 ) = 𝒩1 ∔ (𝒩2 ∔ {𝒩3 ∔ ℛ1 ∩ ℛ2 ∩ ℛ3 })
ℂ𝑛 = 𝒩1 ∔ 𝒩2 ∔ 𝒩3 ∔ ℛ1 ∩ ℛ2 ∩ ℛ3
ℛ1 ∩ ℛ 2 ∩ ℛ 3
Claim: ℛ1 ∩ ℛ2 ∩ ℛ3 is invariant under A.
i.e. 𝑥 ∈ ℛ1 ∩ ℛ2 ∩ ℛ3 ⇒ 𝐴𝑥 ∈ ℛ1 ∩ ℛ2 ∩ ℛ3
ℛ𝑗 = ℛ(𝐴−𝜆
𝑛
𝑗 𝐼𝑛 )
𝑛
𝑥 ∈ ℛ𝑗 ⇒ 𝑥 = (𝐴 − 𝜆𝑗 𝐼𝑛 ) 𝑦
𝑛
𝐴𝑥 = 𝐴(𝐴 − 𝜆𝑗 𝐼𝑛 ) 𝑦
𝑇ℎ𝑒𝑠𝑒 𝑚𝑎𝑡𝑟𝑖𝑐𝑒𝑠 𝑎𝑟𝑒 𝑖𝑛𝑡𝑒𝑟𝑐ℎ𝑎𝑛𝑔𝑒𝑏𝑙𝑒
=
𝑛
(𝐴 − 𝜆𝑗 𝐼𝑛 ) (𝐴𝑦)
ℛ1 ∩ ℛ2 ∩ ℛ3 is a vector space, subspace of ℂ𝑛 . It’s invariant under 𝐴.
⇒Either ℛ1 ∩ ℛ2 ∩ ℛ3 = {0} , or ∃𝑢 ∈ ℛ1 ∩ ℛ2 ∩ ℛ3 and a 𝜆 ∈ ℂ such that 𝐴𝑢 = 𝜆𝑢.
𝜆 = 𝜆1 or 𝜆2 or 𝜆3 .
If 𝜆 = 𝜆1 , 𝑢 ∈ ℛ1 and (𝐴 − 𝜆1 𝐼)𝑢 = 0 ∈ 𝒩1
So the second possibility cannot happen.
So we conclude that ℛ1 ∩ ℛ2 ∩ ℛ3 = {0}
Therefore: ℂ𝑛 = 𝒩1 ∔ 𝒩2 ∔ 𝒩3
ℂ𝑛 = 𝒩(𝐴−𝜆1 𝐼𝑛 )𝑛 ∔ 𝒩(𝐴−𝜆2 𝐼𝑛 )𝑛 ∔ 𝒩(𝐴−𝜆3 𝐼𝑛 )𝑛
Let
{𝑢1 , … , 𝑢𝑟 } be any basis for 𝒩1
{𝑣1 , … , 𝑣𝑠 } be any basis for 𝒩2
{𝑤1 , … , 𝑤𝑡 } be any basis for 𝒩3
Then
{𝑢1 , … , 𝑢𝑟 , 𝑣1 , … , 𝑣𝑠 , … , 𝑤1 , … , 𝑤𝑡 } is a basis for ℂ𝑛
𝐴[𝑢1 … 𝑢𝑟 𝑣1 … 𝑣𝑠 𝑤1 … 𝑤𝑡 ]
𝐴𝒩𝑗 ⊆ 𝒩𝑗
𝑎𝑏𝑏𝑟𝑒𝑣𝑖𝑎𝑡𝑒
=
𝐴[𝑈 𝑉 𝑊]
𝑛
If 𝑥 ∈ 𝒩𝑗 then (𝐴 − 𝜆𝑗 𝐼𝑛 ) 𝑥 = 0
Is 𝐴𝑥 ∈ 𝒩𝑗 ?
Yes. We can interchange:
𝑛
𝑛
(𝐴 − 𝜆𝑗 𝐼𝑛 ) 𝐴𝑥 = 𝐴(𝐴 − 𝜆𝑗 𝐼𝑛 ) 𝑥 = 𝐴0 = 0
𝐴𝑢𝑗 ∈ 𝑠𝑝𝑎𝑛 {𝑢1 , … , 𝑢𝑟 } 𝑗 = 1, … , 𝑟
𝐴𝑣𝑗 ∈ 𝑠𝑝𝑎𝑛 {𝑣1 , … , 𝑣𝑠 } 𝑗 = 1, … , 𝑠
𝐴𝑤𝑗 ∈ 𝑠𝑝𝑎𝑛 {𝑤1 , … , 𝑤𝑡 } 𝑗 = 1, … , 𝑡
𝑥 ∈ 𝑠𝑝𝑎𝑛 {𝑢1 , … , 𝑢𝑟 } ⇒ 𝑥 = 𝑈𝑎 ⇒ 𝐴𝑥 = 𝐴𝑈𝑎
𝐺1
−
𝐴[𝑈 𝑉 𝑊] = [𝑈 𝑉 𝑊]
−
[
𝐺1
−
𝐴=𝑇
−
[
|
+
|
+
|
−
𝐺2
−
|
+ −
| 𝐺2
+ −
|
|
+ −
|
+ −
| 𝐺3 ]
|
+ −
|
𝑇 −1
+ −
| 𝐺3 ]
Next we will choose basis for each of the spaces of 𝒩𝑗 which is useful!
----- End of lesson 8
Theorem: If 𝐴 ∈ ℂ𝑛×𝑛 with 𝑘 distinct Eigenvalues 𝜆1 , … , 𝜆𝑘 , then
ℂ𝑛 = 𝒩𝐴−𝜆1 𝐼 + ⋯ + 𝒩𝐴−𝜆𝑘 𝐼 and this sum is direct.
Implication: (write for 𝑘 = 3)
If {𝑢1 , … , 𝑢𝑟 } is a basis for 𝒩𝐴−𝜆1 𝐼
And {𝑣1 , … , 𝑣𝑠 } is a basis for 𝒩𝐴−𝜆2 𝐼
And {𝑤1 , … , 𝑤𝑡 } is a basis for 𝒩𝐴−𝜆3 𝐼
Then {𝑢1 , … , 𝑢𝑟 , 𝑣1 , … , 𝑣𝑠 , 𝑤1 , … , 𝑤𝑡 } is a basis for ℂ𝑛
𝛾𝑗 = dim 𝒩𝐴−𝜆𝑗𝐼 – Geometric multiplicity.
𝛼𝑗 = dim 𝒩(𝐴−𝜆1 𝐼)𝑛 - Algebraic multiplicity.
𝑟 = 𝛼1 , 𝑠 = 𝛼2 , 𝑡 = 𝛼3
Recall also:
𝒩(𝐴−𝜆 𝐼)𝑛 is invariant under 𝐴.
𝑗
i.e. 𝑥 ∈ 𝒩(𝐴−𝜆
𝑛
𝑗 𝐼)
⇒ 𝐴𝑥 ∈ 𝒩(𝐴−𝜆
𝑛
𝑗 𝐼)
𝐴𝑢1 ∈ 𝒩(𝐴−𝜆1 𝐼)𝑛 ⇒ 𝐴𝑢1 ∈ 𝑠𝑝𝑎𝑛{𝑢1 , … , 𝑢𝑟 }
So
𝑥11
𝑥21
𝐴𝑢1 = [𝑢1 … 𝑢𝑟 ] [ ⋮ ]
𝑥𝑟1
Same as 𝑥11 𝑢1 + 𝑥21 𝑢2 + ⋯ + 𝑥𝑟1 𝑢𝑟
𝐴𝑢2 = [𝑢1
𝑥12
𝑥22
… 𝑢𝑟 ] [ ]
⋮
𝑥𝑟2
[𝐴𝑢1
𝐴𝑢2
… 𝐴𝑢𝑟 ] = [𝑢1
So
𝑢1
𝐴 [⏟
… 𝑢𝑟 ] = [𝑢1
… 𝑢𝑟 ]𝑋 𝑟×𝑟
𝑈
𝑣1
𝐴 [⏟
… 𝑣𝑠 ] = [𝑣1
… 𝑣𝑠 ]𝑌 𝑠×𝑠
𝑉
𝑤1
𝐴 [⏟
… 𝑤𝑡 ] = [𝑤1
𝑊
… 𝑤𝑡 ]𝑍 𝑡×𝑡
𝑥11
𝑥21
… 𝑢𝑟 ] [
⋮
𝑥𝑟1
𝑥12
𝑥22
⋮
𝑥𝑟2
… 𝑥1𝑟
…
⋮
⋮ ]
… 𝑥𝑟𝑟
𝐴[𝑈
𝑉
𝑊 ] = [𝑈𝑋
𝑉𝑌
𝑊𝑍] = [𝑈
𝑋
𝑊] [ 0
0
𝑉
0
𝑌
0
0
0]
𝑍
This much holds for any choice of basis for each of the spaces 𝒩(𝐴−𝜆
Next objective is to choose the basis in 𝒩(𝐴−𝜆
𝑛
𝑗 𝐼)
𝑛
𝑗 𝐼)
.
to give nice results.
If 𝑘 = 3, we would like 𝑋, 𝑌, 𝑍 to be matrices which are easy to work with.
Lemma: Let 𝐵 ∈ ℂ𝑛×𝑛 , then:
(1) dim 𝒩𝐵𝑘+1 dim 𝒩𝐵𝑘 ≤ dim 𝒩𝐵𝑘 − dim 𝒩𝐵𝑘−1 𝑘 = 2,3, …
(2) dim 𝒩𝐵2 − dim 𝒩𝐵 ≤ dim 𝒩𝐵
Proof (1): We know that always dim 𝒩𝐵 ≤ dim 𝒩𝐵2 ≤ dim 𝒩𝐵3 ≤ ⋯ and somewhere it
saturates.
If the left side is zero, it’s not an interesting statement, because the right side is always bigger.
Assume dim 𝒩𝐵𝑘+1 > dim 𝒩𝐵𝑘
But then dim 𝒩𝐵𝑘 > dim 𝒩𝐵𝑘−1
Let {𝑎1 , … , 𝑎𝑟 } be a basis for 𝒩𝐵𝑘−1
Let {𝑎1 , … , 𝑎𝑟 , 𝑏1 , … , 𝑏𝑠 } be a basis for 𝒩𝐵𝑘
Let {𝑎1 , … , 𝑎𝑟 , 𝑏1 , … , 𝑏𝑠 , 𝑐1 , … , 𝑐𝑡 } be any basis for 𝒩𝐵𝑘+1
Claim: (𝑟 + 𝑠 + 𝑡) − (𝑟 + 𝑠) ≤ (𝑟 + 𝑠) − 𝑟
i.e. wish to show that 𝑡 ≤ 𝑠
1) 𝐵𝑘 𝑐1 + ⋯ + 𝐵𝑘 𝑐𝑡 are linearly independent
Suppose 𝛾1 𝐵𝑘 𝑐1 + ⋯ + 𝛾𝑡 𝐵𝑘 𝑐𝑡 = 0
𝐵𝑘 (𝛾1 𝑐1 + ⋯ + 𝛾𝑡 𝑐𝑡 ) = 0
This means that 𝛾1 𝑐1 + ⋯ + 𝛾𝑡 𝑐𝑡 ∈ 𝒩𝐵𝑘
However, we know that
𝒩𝐵𝑘 = 𝒩𝐵𝑘−1 ∔ 𝑠𝑝𝑎𝑛{𝑏1 , … , 𝑏𝑠 }
𝒩𝐵𝑘+1 = 𝒩𝐵𝑘 ∔ 𝑠𝑝𝑎𝑛{𝑐1 , … , 𝑐𝑡 }
And we also know that 𝛾1 𝑐1 + ⋯ + 𝛾𝑡 𝑐𝑡 ∈ 𝒩𝐵𝑘+1
But we know that
𝑐𝑖 𝑠 𝑎𝑟𝑒 𝑖𝑛𝑑𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡!
𝒩𝐵𝑘 ∩ 𝑠𝑝𝑎𝑛{𝑐! , … , 𝑐𝑡 } = {0} ⇒ 𝛾1 𝑐1 + ⋯ + 𝛾𝑡 𝑐𝑡 = 0 ⇒
𝛾1 = 𝛾𝑡 = 0
2) 𝐵𝑘−1 𝑏1 , … , 𝐵𝑘−1 𝑏𝑆 are linearly independent. The proof goes similarly to the proof of 1.
3) 𝐵𝑘 𝑐𝑖 ∈ 𝑠𝑝𝑎𝑛{𝐵𝑘−1 𝑏1 , … , 𝐵𝑘−1 𝑏𝑠 }
𝑐𝑖 ∈ 𝒩𝐵𝑘+1
𝐵𝑘+1 𝑐𝑖 = 0 ⇒ 𝐵𝑘 (𝐵𝑐𝑖 ) = 0 ⇒ 𝐵𝑐𝑖 ∈ 𝒩𝐵𝑘
Due to this observation, we can write:
𝐵𝑐𝑖 = 𝑢 + 𝑣 such that 𝑢 ∈ 𝑠𝑝𝑎𝑛{𝑎1 , … , 𝑎𝑟 } = 𝒩𝐵𝑘−1 , 𝑣 = 𝑠𝑝𝑎𝑛{𝑏1 , … , 𝑏𝑠 } = 𝒩𝐵𝑘
𝑘
𝐵𝑐𝑖 = 𝑢 + ∑ 𝛽𝑗 𝑏𝑗
𝑗=1
𝑘
𝐵𝑘−1 (𝐵𝑐𝑖 )
=𝐵
𝑘−1
𝑢 + ∑ 𝛽𝑗 𝐵
𝑗=1
𝑠
𝑘−1
𝑏𝑗 = ∑ 𝛽𝑗 𝐵𝑘−1 𝑏𝑗
1
4) 𝑠𝑝𝑎𝑛{𝐵𝑘 𝑐1 , … , 𝐵𝑘 𝑐𝑡 } ⊆ 𝑠𝑝𝑎𝑛{𝐵𝑘+1 𝑏1 , … , 𝐵𝑘+1 𝑏𝑠 }
Since all 𝐵𝑘 𝑐1 are linearly independent, so 𝑠 ≥ 𝑡!
Lemma: Suppose 𝑠𝑝𝑎𝑛{𝑢1 , … , 𝑢𝑘 } ⊆ 𝑠𝑝𝑎𝑛{𝑣1 , … , 𝑣𝑙 } and 𝑢1 , … , 𝑢𝑘 are linearly independent
and 𝑣1 , … , 𝑣𝑙 are linearly independent. Then can choose find 𝑙 − 𝑘 of the vectors 𝑣1 , … , 𝑣𝑙 call
them 𝑤1 , … , 𝑤𝑙−𝑘 , then 𝑠𝑝𝑎𝑛{𝑢1 , … , 𝑢𝑘 , 𝑤1 , … , 𝑤𝑙−𝑘 } = 𝑠𝑝𝑎𝑛{𝑣1 , … , 𝑣𝑙 𝑒}
Let 𝑘 = 3, 𝑙 − 5
𝑠𝑝
TODO: Fill in!!!!
𝐴 ∈ ℂ𝑛×𝑛 with 𝜆1 , … , 𝜆𝑘 distinct Eigenvalues
Then ℂ𝑛 = 𝒩(𝐴−𝜆1 𝐼𝑛 ) ∔ … ∔ 𝒩(𝐴−𝜆𝑘 𝐼𝑛 )
In other words:
Take any basis of 𝒩(𝐴−𝜆1 𝐼𝑛 ) with 𝛼1 vectors
Take any basis of 𝒩(𝐴−𝜆2 𝐼𝑛 ) with 𝛼2 vectors
⋮
Take any basis of 𝒩(𝐴−𝜆𝑘 𝐼𝑛 ) with 𝛼𝑘 vectors
Take all those 𝛼1 + ⋯ + 𝛼𝑘 vectors together. It will form a basis for ℂ𝑛
Next step, is to choose the basis for each space in a good way.
Let 𝐵 = 𝐴 − 𝜆𝑗 𝐼𝑛 𝒩𝐵 ≠ {0}
𝒩𝐵 ⊊ 𝒩𝐵2 ⊊ 𝒩𝐵3 = 𝒩𝐵4
{𝑎1 , … , 𝑎𝑟 } basis for 𝒩𝐵
{𝑎1 , … , 𝑎𝑟 , 𝑏1 , … , 𝑏𝑠 } basis for 𝒩𝐵2
{𝑎1 , … , 𝑎𝑟 , 𝑏1 , … , 𝑏𝑠 , 𝑐1 , … , 𝑐𝑡 } basis for 𝒩𝐵3
𝐵2 𝑐1 , … , 𝐵2 𝑐𝑡 ⏟
𝐵𝑏1 , … , 𝐵𝑏𝑠
⏟
𝑖𝑛𝑑𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡
𝑖𝑛𝑑𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡
𝑎1 , … , 𝑎𝑟
⏟
𝑖𝑛𝑑𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡
All in the nullspace of 𝐵
𝑡 + 𝑟 + 𝑠 vecors in 𝒩𝐵 !
But we only need 𝑟 vectors (since the dimension of the nullspace).
And also
𝑠𝑝𝑎𝑛{𝐵2 𝑐1 , … , 𝐵2 𝑐𝑡 } ⊆ 𝑠𝑝𝑎𝑛{𝐵𝑏1 , … , 𝐵𝑏𝑠 }
𝑠𝑝𝑎𝑛{𝐵𝑏1 , … , 𝐵𝑏𝑠 } ⊆ 𝑠𝑝𝑎𝑛{𝑎1 , … , 𝑎𝑟 }
So we will keep:
all 𝐵2 𝑐1 , … , 𝐵2 𝑐𝑡
𝑠 − 𝑡 of 𝐵𝑏1 , … , 𝐵𝑏𝑠
𝑟 − 𝑠 of 𝑎1 , … , 𝑎𝑟
Leaves us with 𝑡 + (𝑠 − 𝑡) + (𝑟 − 𝑠) = 𝑟 vectors.
Can find 𝑠 − 𝑡 of 𝐵𝑏1 , … , 𝐵𝑏𝑠 such that 𝑠𝑝𝑎𝑛{𝐵2 𝑐1 , … , 𝐵2 𝑐𝑡 , 𝑡ℎ𝑜𝑠𝑒 𝑠 − 𝑡} = 𝑠𝑝𝑎𝑛{𝐵𝑏1 , … , 𝐵𝑏𝑠 }
Add 𝑟 − 𝑠 of the {𝑎1 , … , 𝑎𝑟 } vectors, so the whole collection will equal to 𝑠𝑝𝑎𝑛{𝑎1 , … , 𝑎𝑟 } = 𝒩𝐵
Let’s take an example with numbers:
𝑡 = 2, 𝑠 = 3, 𝑟 = 5
{𝑎1 , … , 𝑎5 } basis for 𝒩𝐵
{𝑎1 , … , 𝑎5 , 𝑏1 , 𝑏2 , 𝑏3 } basis for 𝑠𝑐𝑟𝑖𝑝𝑡𝐵2
{𝑎1 , … , 𝑎5 , 𝑏1 , 𝑏2 , 𝑏3 , 𝑐1 , 𝑐2 } basis for 𝑠𝑐𝑟𝑖𝑝𝑡𝐵3
𝐵2 𝑐1 𝐵2 𝑐2 𝐵𝑏1 𝐵𝑏2 𝐵𝑏3 𝑎1 𝑎2 𝑎3 𝑎4 𝑎5
Suppose 𝐵𝑏2 is linearly independent of 𝐵2 𝑐1 , 𝐵2 𝑐2
And 𝑎2 , 𝑎4 are linearly independent of 𝐵2 𝑐1 , 𝐵2 𝑐2 , 𝐵𝑏2
So keep
𝐵2 𝑐1 𝐵2 𝑐2 𝐵𝑏2 𝑎2 𝑎4
𝐵𝑐1 𝐵𝑐2
𝑏2
𝑐1
𝑐2
Claim: Those 10 vectors are Linearly independent.
To this point, we only know 𝐵2 𝑐1 , 𝐵2 𝑐2 , 𝐵𝑏2 , 𝑎2 , 𝑎4 are linearly independent.
𝛾1 𝐵2 𝑐1 + 𝛾2 𝐵𝑐1 + 𝛾3 𝑐1 + 𝛾4 𝐵2 𝑐2 + 𝛾5 𝐵𝑐2 + 𝛾6 𝑐2 + 𝛾7 𝐵𝑏2 + 𝛾8 𝑏2 + 𝛾9 𝑎2 + 𝛾10 𝑎4 = 0
Let’s apply 𝐵2 on both sides.
What’s remains of these two terms is:
𝛾3 𝐵2 𝑐1 + 𝛾6 𝐵2 𝑐2 = 0 ⇒ 𝛾3 = 𝛾6 = 0 Since they are linearly independent!
Let’s apply 𝐵!
𝛾2 𝐵2 𝑐1 + 𝛾5 𝐵2 𝑐2 + 𝛾8 𝐵𝑏2 = 0 ⇒ 𝛾2 = 𝛾5 = 𝛾8 because they are linearly independent!
𝛾1 𝐵2 𝑐2 + 𝛾4 𝐵2 𝑐2 + 𝛾7 𝐵𝑏2 + 𝛾9 𝑎2 + 𝛾10 𝑎4 = 0 ⇒ 𝛾1 = 𝛾4 = 𝛾7 = 𝛾9 = 𝛾10 = 0 because they
are linearly independent!
So all coefficients must be zero, therefore all coefficients are linearly independent.
Since dim 𝒩𝐵3 = 10, the vectors in this array form a basis f=r 𝒩𝐵3 .
Claim: 𝐴 [𝐵
⏟2 𝑐1
𝐵𝑐1
𝑐1
𝐵2 𝑐2
𝐵𝑐2
𝑐2
𝐵𝑏2
𝑏2
𝑎2
𝑎4 ]
𝑈𝑗
(3)
𝐶𝜆𝑗
𝐴𝑈𝑗 = 𝑈𝑗
0
0
0
0
(3)
𝐶𝜆𝑗
0
0
0
0
𝐶𝜆𝑗
(2)
0
0
0
0
𝐶𝜆𝑗
[ 0
0
0
0
(1)
𝜆𝑗
0
0
0
0
0
0
0 = 𝑈𝑗
0
0
0
0
(1)
𝐶𝜆𝑗 ]
0
[0
1
𝜆𝑗
0
0
0
0
0
0
0
0
0
1
𝜆𝑗
0
0
0
0
0
0
0
0
0
0
𝜆𝑗
0
0
0
0
0
0
0
0
0
1
𝜆𝑗
0
0
0
0
0
0
0
0
0
1
𝜆𝑗
0
0
0
0
0
0
0
0
0
0
𝜆𝑗
0
0
0
0
0
0
0
0
0
1
𝜆𝑗
0
0
0
0
0
0
0
0
0
0
𝜆𝑗
0
0
0
0
0
0
0
0
0
0
𝜆𝑗 ]
If 𝐴 has 3 Eigenvalues 𝜆1 , 𝜆2 , 𝜆3
𝐷𝜆1
0
0
0 ]
𝑈3 ] [ 0 𝐷𝜆2
0
0 𝐷𝜆3
𝐷𝜆𝑗 is an 𝛼𝑗 × 𝛼𝑗 with 𝜆𝑗 on the diagonal and 𝛾𝑗 Jordan cells in its “decomposition”.
𝐴[𝑈1
𝑈2
𝑈3 ] = [𝑈1
𝑈2
dim 𝒩(𝐴−𝜆𝑗𝐼) = number of Jordan cells.
Analyzing 𝒩
𝑘
(𝐴−𝜆𝑗 𝐼𝑛 )
𝑘 = 1,2, …
For simplicity, denote 𝐴 − 𝜆𝑗 𝐼𝑛 = 𝐵
dim 𝒩𝐵 = 5 = 𝑟
dim 𝒩𝐵2 = 8 = 𝑟 + 𝑠
dim 𝒩𝐵3 = 10 = 𝑟 + 𝑠 + 𝑡
Just by these numbers you can find out how 𝑈 should look like:
𝑋
𝑋
𝑋
𝑋
𝑋
𝑋
𝑋
𝑋
𝑋
𝑋
5
8−5=3
1−8=2
The Jordan form:
2 3 × 3 Jordan cells
1 2 × 2 Jordan cells
2 1 × 1 Jordan cells
1 2 0 0 1
0 1 2 0 0
𝐴= 0 0 1 0 0
0 0 0 2 0
[0 0 0 2 2]
The objective is to find 𝑈 5 × 5 such that 𝐴𝑈 = 𝑈𝐽 ←Jordan form
(1) Find the Eigenvalues – 𝐴 has 2 distinct Eigenvalues – 𝜆1 = 1, 𝜆2 = 2, 𝛼1 = 3, 𝛼2 = 2
(2) First set 𝐵 = 𝐴 − 𝜆1 𝐼5 = 𝐴 − 𝐼5 , calculate 𝒩𝐵 , 𝒩𝐵2 , …
0 2 0 0 1
0 0 2 0 0
𝐵= 0 0 0 0 0
0 0 0 1 0
[0 0 0 2 1]
𝒩𝐵 = 𝑠𝑝𝑎𝑛{𝑒1 } 𝑒𝑗 = 𝑗th column of 𝐼5
1
0
𝑒1 = 0 , …
0
[ 0]
𝒩𝐵2 = 𝑠𝑝𝑎𝑛{𝑒1 , 𝑒2 }
𝒩𝐵3 = 𝑠𝑝𝑎𝑛{𝑒1 , 𝑒2 , 𝑒3 }
0 2 0
0 0 2
0 0 0
𝐵2 =
− − −
0 0 0
[0 0 0
| 0 1 0
| 0 0 0
| 0 0 0
+ − − −
| 1 0 0
| 2 1] [ 0
(𝐵 )2
[ 11
0
2 0 | 0 1
0 2 | 0 0
𝐵
0 0 | 0 0
= [ 11
0
− − + − −
0 0 | 1 0
0 0 | 2 1]
𝐵11 𝐵12 + 𝐵12 𝐵22
]=
(𝐵22 )2
0 0 0 |
0 0 0 |
0 0 0 |
𝐵3 =
− − − +
0 0 0 |
[0 0 0 |
0 0 0 |
0 0 0 |
0 0 0 |
𝐵4 =
− − − +
0 0 0 |
[0 0 0 |
∗
∗
∗
−
@
@
∗
∗
∗
−
@
@
∗
∗
∗
−
@
@]
∗
∗
∗
−
@
@]
So we only keep {𝑒1 = 𝑐1 }
In this case dim 𝒩 − 𝜆1 𝐼5 = 1 ⇒ 1 Jordan cell for this Eigenvalue
Calculate 𝒩𝐵 , 𝒩𝐵2
𝑒3 , 𝐵𝑒3 , 𝐵2 𝑒3
𝐵𝑒3 = 2𝑒2
2
𝐵 𝑒3 = 𝐵(𝐵𝑒3 ) = 𝐵2 𝑒2 = 4𝑒1
A detour to old history:
0
𝐵[𝐵2 𝑐1 𝐵𝑐1 𝑐1 ] = [𝐵3 𝑐1 𝐵2 𝑐1 𝐵𝑐1 ] [0
0
So
𝜆1 1 0
𝐴[𝑢1 𝑢2 𝑢3 ] = [𝑢1 𝑢2 𝑢3 ] [ 0 𝜆1 1 ]
0 0 𝜆1
1 0
0 1]
0 0
𝐵12 𝐵11
][
𝐵22 0
𝐵12
]=
𝐵22
−1 2
0 0 1
0 −1 2 0 0
Now let 𝐵 = 𝐴 − 𝜆2 𝐼5 = 𝐴 − 2𝐼5 = 0
0 −1 0 0
0
0
0 0 0
[0
0
0 2 0]
1
0
𝐵 0 =0
0
[−1]
𝒩𝐵 = 𝑠𝑝𝑎𝑛{𝑒1 + 𝑒5 }
𝑥1 − 4𝑥2 − 4𝑥3 + 2𝑥4 − 𝑥5
−1 −4 −4 2 −1 𝑥1
0 −1 −4 0 0 𝑥2
−𝑥2 − 4𝑥3
𝐵2 𝑥 = 0
0
1 0 0 𝑥3 =
𝑥3
0
0
0 0 0 𝑥4
0
[0
]
𝑥
[
]
[
]
0
0 0 0
5
0
𝑥1
−2𝑥4 + 𝑥5
−2
1
𝑥2
0
0
0
𝑥3 =
0
= 𝑥4 0 + 𝑥5 0
𝑥4
𝑥4
1
0
[0]
[1]
[𝑥5 ] [
𝑥5
]
1
0
𝒩𝐵 = 𝑠𝑝𝑎𝑛 0
0
{[1]}
1 −2
0
0
2
𝒩𝐵 = 𝑠𝑝𝑎𝑛 0 , 0
0
1
{[1] [ 0 ]}
0
𝐵[𝐵𝑏1 𝑏1 ] = [𝐵2 𝑏1 𝐵𝑏1 ] = [0 𝐵𝑏1 ] = [𝐵𝑏1 𝑏1 ] [
0
2
1
So 𝐴[𝑢4 𝑢5 ] = [𝑢4 𝑢5 ] [
]
0 2
𝐴[𝑢1
---- end of lesson 9
𝑢2
𝑢3
𝑢4
𝑢5 ] = [𝑢1
𝑢2
𝑢3
1
]
0
𝑢4
1
0
𝑢5 ] 0
0
[0
1
1
0
0
0
0
1
1
0
0
0
0
0
2
0
0
0
0
1
2]
Determinants
𝐴 ∈ 𝐶 𝑛×𝑛 Special number called its determinant.
Show that there exists exactly one function from 𝐶 𝑛×𝑛 to ℂ
(1) 𝑓(𝐼𝑛 ) = 1
(2) If 𝑃 a simple permutation, then 𝑓(𝑃𝐴) = −𝑓(𝐴)
(3) 𝑓(𝐴) is linear in each row of 𝐴 separately.
Let 𝐴 ∈ ℂ3×2
𝑎1
𝐴 = [𝑎2 ], if say 𝑎2 = 𝛼𝑢 + 𝛽𝑣then
𝑎3
𝑎1
𝑎1
𝑎1
𝑎1
𝑎1
𝛼𝑢
𝑢
𝛼𝑢
+
𝛽𝑣
𝛽𝑣
𝑓 ([
]) = 𝑓 ([ ]) + 𝑓 ([ ]) = 𝛼𝑓 ([ ]) + 𝛽𝑓 ([ 𝑣 ])
𝑎3
𝑎3
𝑎3
𝑎3
𝑎3
Properties 1,2,3 imply automatically:
(4) If 2 rows of 𝐴 coincide, then 𝑓(𝐴) = 0
𝑎1
𝑛×𝑛
𝐴∈ℂ
𝑎𝑖 = 𝑎𝑗 ≠ 𝑗 𝐴 = [ ⋮ ]
𝑎𝑛
Let 𝑃 be the simple permutation that interchanges row 𝑖 with 𝑗.
But −𝑓(𝐴) = 𝑓(𝑃𝐴) = 𝑓(𝐴)
2𝑓(𝐴) = 0 ⇒ 𝑓(𝐴) = 0
Example:
1 2 3
0 0 1
𝐴 = [ 4 5 6] ,
𝑃 = [ 0 1 0]
1 2 3
1 0 0
1 2 3
𝑃𝐴 = [4 5 6] = 𝐴
1 2 3
𝑎11
𝐴 = [𝑎
𝑎12
[𝑎11 𝑎12 ] = 𝑎11 [1 0] + 𝑎12 [0 1]
𝑎22 ] ,
1
0
0
1
𝑓(𝐴) = 𝑎11 𝑓 ([
]) + 𝑎12 𝑓 ([
])
𝑎21 𝑎22
𝑎21 𝑎22
[𝑎21 𝑎22 ] = 𝑎21 [1 0] + 𝑎22 [0 1]
21
So
[1 0]
[1 0]
[0 1]
[0 1]
]) + 𝑎11 𝑓 ([
]) + 𝑎12 𝑓 ([
]) + 𝑎12 𝑓 ([
]) =
𝑎21 [1 0]
𝑎22 [0 1]
𝑎21 [1 0]
𝑎22 [0 1]
1 0
1 0
0 1
0 1
𝑎11 𝑎21 𝑓 ([
]) + 𝑎11 𝑎22 𝑓 ([
]) + 𝑎12 𝑎21 𝑓 ([
]) + 𝑎12 𝑎22 𝑓 ([
]) =
1 0
0 1
1 0
0 1
0 + 𝑎11 𝑎22 − 𝑎12 𝑎21 + 0 = 𝑎11 𝑎22 − 𝑎12 𝑎21
𝑓(𝐴) = 𝑎11 𝑓 ([
𝐴∈ℂ
3×3
𝑎1
𝑎11
𝑎
𝑎
, 𝐴 = [ 2 ] = [ 21
𝑎3
𝑎31
𝑎12
𝑎22
𝑎32
𝑎13
𝑎23 ] , 𝑒1 , 𝑒2 , 𝑒3 columns of 𝐼3 .
𝑎33
𝑎1 = 𝑎11 𝑒1 + 𝑎12 𝑒2 + 𝑎13 𝑒3
𝑎1
𝑒1𝑇
𝑒2𝑇
𝑒3𝑇
𝑓(𝐴) = 𝑓 ([𝑎2 ]) = 𝑎11 𝑓 ([𝑎2 ]) + 𝑎12 𝑓 ([𝑎2 ]) + 𝑎13 𝑓 ([𝑎2 ])
𝑎3
𝑎3
𝑎3
𝑎3
Let’s call them 1,2,3 accordingly.
So
1 = 𝑎2 = 𝑎21 𝑒1 + 𝑎22 𝑒2 + 𝑎23 + 𝑒3
𝑒1𝑇
𝑒1𝑇
𝑒1𝑇
𝑇
𝑇
1 = 𝑎11 𝑎21 𝑓 ([𝑒1 ]) + 𝑎11 𝑎22 𝑓 ([𝑒2 ]) + 𝑎11 𝑎23 𝑓 ([𝑒3𝑇 ])
𝑎3
𝑎3
𝑎3
The first value is zero! We only have to deal with the last two terms.
𝑎3 = 𝑎31 𝑒1𝑇 + 𝑎32 𝑒2𝑇 + 𝑎33 𝑒3𝑇
𝑒1𝑇
𝑒1𝑇
So 1 will be: 𝑎11 𝐴22 𝑎33 𝑓 ([𝑒2𝑇 ]) + 𝑎11 𝐴23 𝑎32 𝑓 ([𝑒3𝑇 ])
𝑒3𝑇
𝑒2𝑇
So
1 = 𝑎11 𝑎22 𝑎33 − 𝑎11 𝑎23 𝑎32
2 = Two more terms
3 = Two more terms
TODO: Draw diagonals
𝑎1
(5) If 𝐴 = [ ⋮ ] , 𝐵 = 𝐸𝐴, 𝐸 lower triangular, 1’s on the diagonal and exactly one non-zero
𝑎𝑛
entry below diagonal.
Then det(𝐸𝐴) = det 𝐴
1 0
𝐸 = [0 1
0 𝛼
𝑎1
𝐸𝐴 = [ 𝑎2 ]
𝛼𝑎2 + 𝑎3
𝐴 ∈ ℂ3×3 ,
0
0]
1
𝑎1
𝑎1
det 𝐸𝐴 = αdet [𝑎2 ] + det [𝑎2 ] = det 𝐴
𝑎3
𝑎2
Same as to say: adding a multiple of one row to another row does not change the determinant.
(6) If 𝐴 has a row which is all zeroes, then det(𝐴) = 0
(7) If two rows are linearly dependent, then det 𝐴 = 0
(8) If 𝐴 ∈ ℂ𝑛×𝑛 is triangular, then det 𝐴 = 𝑎11 𝑎22 … 𝑎𝑛𝑛
𝑎11
𝐴=[ 0
0
𝑎12
𝑎22
0
𝑎13
𝑎23 ]
𝑎33
By the linearity in row 3⇒ det 𝐴 =
𝑎11 𝑎12 𝑎13 𝑝𝑟𝑜𝑝𝑒𝑟𝑡𝑦 5
𝑎11 𝑎12 0
𝑎33 det [ 0 𝑎22 𝑎23 ]
=
𝑎33 det [ 0 𝑎22 0] =
0
0
1
0
0 1
𝑎11 𝑎12 0
𝑎11 0 0
1 0
𝑎33 𝑎22 det [ 0
1 0] = 𝑎33 𝑎22 det [ 0 1 0] = 𝑎33 𝑎22 𝑎11 det [0 1
0 0
0
0 1
0 0 1
(9) det 𝐴 ≠ 0 ⇔ 𝐴 is invertible
𝐴 nonzero … 𝐸3 𝑃3 𝐸2 𝑃2 𝐸1 𝑃1 𝐴 = 𝑈 upper echelon
𝑃1 - Permutation of the first row with one of others or just 𝐼
𝑃2 - Permutation of the second row with either 3rd, 4th … or 𝐼
So you can generalize it to:
𝐸𝑃𝐴 = 𝑈
𝐸-lower triangular, 𝑃-permutation.
1
𝑃2 𝐸1 = 𝑃2 [𝛼
𝛽
0
1
0
0
0]
1
𝑃2 interchanges 2nd with 3rd row
1 0 0
𝑃2 = [0 0 1]
0 1 0
𝑃2 𝐸1 ≠ 𝐸1 𝑃2 (not always!)
0 0
𝑃2 (𝐼3 + [𝛼 0
𝛽 0
0 0
This is the same as 𝑃2 + [𝛽 0
𝛼 0
However, it is equal to
𝐸1′ 𝑃2 𝑃1
0
0 0
0]) = 𝑃2 + 𝑃2 [𝛼 0
0
𝛽 0
0
0 0
0] 𝑃2 = (𝐼3 + [𝛽 0
0
𝛼 0
0
0
0] = 𝑃2 + [𝛽
0
𝛼
0
0]) 𝑃2 = 𝐸1′ 𝑃2
0
𝑃2 𝐸1 𝑃1 ≠ 𝐸1 𝑃2 𝑃1
where has the same form as 𝐸1
𝐸1′
0 0
0 0]
0 0
0
0]
1
If 𝐴 ∈ ℂ𝑛×𝑛 , 𝐴 ≠ 0, then there exists an 𝑛 × 𝑛 permutation matrix 𝑃, and an 𝑛 × 𝑛 lower
triangular 𝐸 with 1’s on the diagonal such that:
𝐸𝑃𝐴 = 𝑈 𝑛 × 𝑛 upper echelon.
In fact 𝑈 is upper triangular!
det 𝐸𝑃𝐴 = det 𝑈 = 𝑢11 𝑢22 … 𝑢𝑛𝑛
By iterating property 5, det 𝐸𝑃𝐴 = det 𝑃𝐴 = ± det 𝐴
So we know that |det 𝐴| = |𝑢11 𝑢22 … 𝑢𝑛𝑛 |
Claim: 𝐸𝑃𝐴 = 𝑈
Since 𝐸 and 𝑃 are invertible, 𝐴 is invertible ⇔ 𝑈 is invertible ⇔ all values on the diagonal are
non zero ⇔ |det 𝐴| ≠ 0 ⇔ det 𝐴 ≠ 0
(10)𝐴, 𝐵 ∈ ℂ𝑛×𝑛 , then det 𝐴𝐵 = det 𝐴 ∙ det 𝐵
Case (1) – 𝒩𝐵 ≠ {0} ⇒ 𝒩𝐴𝐵 ≠ {0}
So 𝐵 not invertible means 𝐴𝐵 not invertible. So by 9, det 𝐵 = 0 and det 𝐴𝐵 = 0. Which also
means that det 𝐴 ∙ det 𝐵 = 0 ∙ 𝑠𝑜𝑚𝑒𝑡ℎ𝑖𝑛𝑔 = 0 = det 𝐴𝐵
Case (2) – 𝐵 is invertible.
𝜑(𝐴) =
det(𝐴𝐵)
det 𝐵
What are the properties of this function?
𝜑(𝐼) = 1
if 𝑃 is a simple permutation
𝜑(𝑃𝐴) =
det 𝑃(𝐴𝐵)
det 𝐴𝐵
=−
= −𝜑(𝐴)
det 𝐵
det 𝐵
Claim: 𝜑 is linear in each row of 𝐴
In the 3 × 3 case
𝑎1
𝑎
𝐴 = [ 2]
𝑎3
𝑎1 𝐵
det [ 𝑎2 𝐵 ]
𝐴3 𝐵
𝜑(𝐴) =
det 𝐵
Say that 𝑎1 = 𝛼𝑢 + 𝛽𝑣
So 𝑎1 𝐵 = 𝛼(𝑢𝐵) + 𝛽(𝑣𝐵)
𝑢𝐵
𝑣𝐵
(𝛼 det [𝑎2 𝐵] + 𝛽 det [𝑎2 𝐵])
𝛼𝑢𝐵 + 𝛽𝑣𝐵
𝛼𝑢 + 𝛽𝑣
𝑎3 𝐵
𝑎3 𝐵
𝑎2 𝐵
𝜑 ([ 𝑎2 ]) = det [
]=
det
𝐵
𝑎
𝑎 𝐵
3
3
1
(11) If 𝐴 is invertible, then det 𝐴−1 = det 𝐴. Easily obtained by calculating det 𝐴 ∙ 𝐴−1 which
must be 1 and according to property 3 is also the multiplication of determinants.
(12)det 𝐴𝑇 = det 𝐴
𝐸𝑃𝐴 = 𝑈 upper echelon
So according to previous properties we know that det 𝑃 ∙ det 𝐴 = 𝑢11 𝑢22 … 𝑢𝑛𝑛
But we also know that 𝐴𝑇 𝑃𝑇 𝐸 𝑇 = 𝑈 𝑇
So:
det 𝐴𝑇 ∙ det 𝑃𝑇 ∙ det 𝐸 𝑇 = det 𝑈 𝑇
We know that det 𝐸 = det 𝐸 𝑇 = 1
Also when we flip 𝑈, we still have a triangular matrix, just instead of a upper triangular we have
a lower triangular. So its determinant stays the same. So, so far this is true:
det 𝑃 ∙ det 𝐴 = det 𝑃𝑇 ∙ det 𝐴𝑇
Let 𝑃 be some permutation.
Claim: 𝑃𝑇 𝑃 = 𝐼
Let’s multiply both sides by det 𝑃 !
(det 𝑃)2 ∙ det 𝐴 = det 𝑃 ∙ det 𝑃𝑇 ∙ det 𝐴𝑇 = det 𝑃𝑃𝑇 ∙ det 𝐴𝑇
(±1)2 ∙ det 𝐴 = 1 ∙ det 𝐴𝑇 ⇒ det 𝐴 = det 𝐴𝑇
𝐴 ∈ ℂ𝑛×𝑛 , 𝜆1 , … , 𝜆𝑘 distinct Eigenvalues.
Then 𝐴𝑈 = 𝑈𝐽 where 𝐽 is in Jordan form.
𝐴 with 𝜆1 , 𝜆2 algebraic multiplicities 𝛼1 , 𝛼2
𝛼𝑗 = dim 𝒩(𝐴−𝜆
𝑛
𝑗 𝐼𝑛 )
det 𝜆𝐼𝑛 − 𝐴 = det(𝜆𝐼𝑛 − 𝑈𝐽𝑈 −1 ) = det 𝑈(𝜆𝐼𝑛 − 𝐽)𝑈 −1 = det 𝑈 ∙ det 𝜆𝐼𝑛 − 𝐽 ∙ det 𝑈 −1
= det(𝜆𝐼𝑛 − 𝐽) = (𝜆 − 𝜆1 )𝛼1 (𝜆 − 𝜆2 )𝛼2
---- end of lesson 10
Main Properties of Determinants - Again
Let 𝐴 ∈ ℂ𝑛×𝑛
det 𝐼𝑛 = 1
det 𝑃𝐴 = − det 𝐴 if 𝑃 is a simple permutation
det 𝐴 is linear in each row of 𝐴
𝐴 in invertible ⇔ det 𝐴 ≠ 0
Let 𝐵 ∈ ℂ𝑛×𝑛
det 𝐴𝐵 = det 𝐴 ∙ det 𝐵 = det 𝐵𝐴
If 𝐴 triangular then det 𝐴 = 𝑎11 𝑎22 … 𝑎𝑛𝑛
Determinants in the aid of Eigenvalues
Recall 𝜆 is an eigenvalue of 𝐴 if 𝒩𝐴−𝜆𝐼𝑛 ≠ {0}
𝐵 matrix. It’s Left invertible ⇔ 𝒩𝐵 = {0}
If 𝐵 ∈ ℂ𝑛×𝑛 it’s invertible ⇔ 𝒩𝐵 = {0}
det(𝜆𝐼 − 𝐴) ≠ 0 ⇔ 𝜆𝐼 − 𝐴 invertible ⇔ 𝒩𝜆𝐼−𝐴 = {0}
So the opposite is:
det(𝜆𝐼 − 𝐴) = 0 ⇔ 𝒩𝜆𝐼−𝐴 ≠ {0}
𝐴 ∈ ℂ𝑛×𝑛 has 𝑘 distinct eigenvalues, then there exists an invertible matrix 𝑈 and an upper
triangular matrix 𝐽 of special form (cough cough, the Jordan form cough cough)
Such that:
𝐴𝑈 = 𝑈𝐽 equivalently 𝐴 = 𝑈𝐽𝑈 −1
If 𝑘 = 3:
𝜆1
0
0
−
0
0
0
0
0
0
0
⋱
⋱
0
−
0
0
0
0
0
0
0
0
⋱
𝜆1
−
0
0
0
0
0
0
0
| 0
| 0
| 0
+ −
| 𝜆2
| 0
| 0
+ −
0 0
0 0
0 0
0 0
0 0
0 0
− −
⋱ 0
⋱ ⋱
0 𝜆2
− −
0 0
0 0
0 0
0 0
0 0
0 0
+ 0
| 0
| 0
| 0
+ −
| 𝜆3
| 0
| 0
0 0
0 0
0 0
0 0
0 0
0 0
0 0
− −
⋱ 0
⋱ ⋱
0 𝜆3
det(𝜆𝐼 − 𝐴) = det(𝜆𝐼𝑛 − 𝑈𝐽𝑈 −1 ) = det(𝑈(𝜆𝐼𝑛 − 𝐽)𝑈 −1 ) = det 𝑈 ∙ det(𝜆𝐼𝑛 − 𝐽) ∙ det 𝑈 −1 =
det(𝜆𝐼𝑛 − 𝐽)
So if 𝑘 = 3 this would leave us with (𝜆 − 𝜆1 )𝛼1 (𝜆 − 𝜆2 )𝛼2 (𝜆 − 𝜆3 )𝛼3
If we set 𝜆 = 0 we get (−𝜆1 )𝛼1 ∙ (−𝜆2 )𝛼2 ∙ (−𝜆3 )𝛼3
So det −𝐴 = (−𝜆1 )𝛼1 ∙ (−𝜆2 )𝛼2 ∙ (−𝜆3 )𝛼3 ⇒ (−1)𝑛 det 𝐴 = (−1)𝑛 (𝜆1 )𝛼1 (𝜆2 )𝛼2 (𝜆3 )𝛼3
So det 𝐴 = (𝜆1 )𝛼1 (𝜆2 )𝛼2 (𝜆3 )𝛼3
𝐴 ∈ ℂ𝑛×𝑛
𝑛
𝑡𝑟𝑎𝑐𝑒𝐴 = ∑ 𝑎𝑖𝑖
𝑖=1
𝑛×𝑛
𝐴 = 𝑈𝐽𝑈 −1 ,
𝐴, 𝐵 ∈ ℂ
,
𝑡𝑟𝑎𝑐𝑒𝐴𝐵 = 𝑡𝑟𝑎𝑐𝑒𝐵𝐴
𝑡𝑟𝑎𝑐𝑒𝐴 = 𝑡𝑟𝑎𝑐𝑒𝑈(𝐽𝑈 −1 ) = 𝑡𝑟𝑎𝑐𝑒(𝐽𝑈 −1 )𝑈 = 𝑡𝑟𝑎𝑐𝑒𝐽 = 𝛼1 𝜆1 + ⋯ + 𝛼𝑘 𝜆𝑘
𝐴11
0
Claim: det 𝐴 = det 𝐴11 ∙ det 𝐴22
0
],
𝐴22
𝐴=[
𝐴11 ∈ ℂ𝑝×𝑝 ,
𝐴22 ∈ ℂ𝑞×𝑞
𝐴11 = 𝑈𝐽𝑈 −1 , 𝐽 upper triangular
𝐴22 = 𝑉𝐽̃𝑉 −1 , 𝐽̃ upper triangular
det 𝐴11 = det 𝐽 , det 𝐴22 = det 𝐽̃
𝑈𝐽𝑈 −1
0
𝑈
𝐴=[
]=[
0
0
𝑉𝐽̃𝑉 −1
𝐽 0
So det 𝐴 = det [
] = det 𝐽 det 𝐽̃
0 𝐽̃
0 𝐽 0 𝑈 −1
][
][
𝑉 0 𝐽̃
0
𝑥11
𝑥22
𝐽 0
[
]=
0 𝐽̃
𝑥33
𝑦11
𝑦22 ]
[
It is still upper triangular!
A more sophisticated claim:
𝐴=[
𝐴11 (𝑝 × 𝑝) 𝐴12 (𝑝 × 𝑞)
]
0(𝑞 × 𝑝) 𝐴22 (𝑞 × 𝑞)
Claim: det 𝐴 = det 𝐴11 det 𝐴22
𝐴=[
𝐼𝑝
0
0 𝐴11
][
𝐴22 0
𝐴22
𝐼𝑞 ]
0 ]
𝑉 −1
So we now know that det 𝐴 = det [
𝐼𝑝
0
𝑎11
𝑎21
𝑎31
0
[ 0
0
𝐴11
] det [ 0
𝐴22
𝑎12
𝑎22
𝑎32
0
0
𝑎13
𝑎23
𝑎33
0
0
|
|
|
|
|
𝐴22
𝐼𝑞 ]
𝑎14
𝑎24
𝑎34
1
0
𝑎15
𝑎25
𝑎35
0
1 ]
I can subtract rows from other rows and the determinant still stays the same. So I can zero out
all the rows in the right (since the rest of the values are zeroes)
So we get:
𝑎11 𝑎12 𝑎13 | 0 0
𝑎21 𝑎22 𝑎23 | 0 0
𝑎31 𝑎32 𝑎33 | 0 0
0
0
0 | 1 0
0
0
0 | 0 1]
[
But now it’s of the form as in the previous claim.
𝐼𝑝
0
𝐴11 𝐴22
𝐴11 0
So det 𝐴 = det [
] det [ 0
𝐼𝑞 ] = det 𝐴22 det [ 0 𝐼𝑞 ] = det 𝐴22 det 𝐴11
0 𝐴22
An even more complicated claim - Schur Complemnts:
𝐴
𝐴12
𝐴 = [ 11
]
𝐴21 𝐴22
(all blocks are squares)
If 𝐴22 is invertible, then:
−1
𝐼𝑝
𝐼 𝐴12 𝐴−1
0
22 𝐴11 − 𝐴12 𝐴22 𝐴21
𝐴=[𝑝
][
] [ −1
0
𝐼𝑞
0
𝐴22 𝐴22 𝐴21
−1
From this it follows that det 𝐴 = det(𝐴11 − 𝐴12 𝐴22 𝐴21 ) ∙ det 𝐴22
Suppose 𝐴11 is invertible
𝐼𝑝
0 𝐴11
0
𝐼𝑝
𝐴=[
][
][
−1
−1
𝐴12 0
𝐴21 𝐴11 𝐼𝑞 0 𝐴22 − 𝐴21 𝐴11
−1
det 𝐴 = det 𝐴11 det(𝐴22 − 𝐴21 𝐴11
𝐴12 )
1
0
𝐴= 0
0
[0
−1
𝐴11
𝐴22
]
𝐼𝑞
2 0 | 0
1 2 | 0
0 1 | 0
0 0 | 2
0 0 | 2
𝐴𝑈 = 𝑈 −1 𝐽
𝜆1 = 1,
𝜆2 = 2
𝛼1 = 3,
𝛼2 = 2
1
0
0
0
2]
0
]
𝐼𝑞
𝜆−1
2
0
|
0
1
0
𝜆−1
2
|
0
0
0
0
𝜆−1 |
0
0
det(𝜆𝐼 − 𝐴) = det
=
−
−
−
+
−
−
0
0
0
| 𝜆−2
0
0
0
|
2
𝜆 − 2]
[ 0
𝜆 − 1 −2
0
𝜆−2
0
det [ 0
] = (𝜆 − 1)3 ∙ (𝜆 − 2)2
𝜆 − 1 −2 ] ∙ det [
−2 𝜆 − 2
0
0
𝜆−1
Expansions of Minors
If 𝐴 ∈ ℂ𝑛×𝑛 let 𝐴{𝑖,𝑗} = determinant of (𝑛 − 1) × (𝑛 − 1) matrix which is obtained by erasing
row 𝑖 and column 𝑗
1 2 3
𝐴 = [4 5 6]
7 8 9
1 2
𝐴{2,3} = det [
] = 8 − 14 = −6
7 8
det 𝐴 = ∑𝑛𝑖=1(−1)𝑖+𝑗 ∙ 𝑎𝑖𝑗 𝐴{𝑖𝑗} for each 𝑗, 𝑗 = 1, … , 𝑛 (expansion along the j’th column)
det 𝐴 = ∑𝑛𝑗=1(−1)𝑖+𝑗 ∙ 𝑎𝑖𝑗 𝐴{𝑖𝑗} for each 𝑖, 𝑖 = 1, … , 𝑛 (expansion along the I’th row)
So if you have a row with a lot of zeroes for instance:
0 0 0 4
∙ ∙ ∙ ∙
[
]
∙ ∙ ∙ ∙
∙ ∙ ∙ ∙
Then
𝑛
det 𝐴 = ∑(−1)𝑖+𝑗 ∙ 𝑎𝑖𝑗 𝐴{𝑖𝑗}
𝑗=1
So if we choose 𝑖 = 1 we can get all zeroes but the 4th column.
Example:
𝑎11 𝑎12 𝑎13
𝑎1
Suppose that 𝐴 = [𝑎21 𝑎22 𝑎23 ] = [𝑎2 ]
𝑎31 𝑎32 𝑎33
𝑎3
𝑎1 = 𝑎11 [1 0 0] + 𝑎12 [0 1
So we can use linearity of the determinant:
1
0
0
0
det 𝐴 = 𝑎11 det [𝑎21 𝑎22 𝑎23 ] + 𝑎12 det [𝑎21
𝑎31 𝑎32 𝑎33
𝑎31
0] + 𝑎13 [0
1
𝑎22
𝑎32
0 1]
0
0
𝑎23 ] + 𝑎13 det [𝑎21
𝑎33
𝑎31
0
𝑎22
𝑎32
1
𝑎23 ]
𝑎33
1
0
0 𝑠𝑢𝑏𝑡𝑟𝑎𝑐𝑡 𝑡ℎ𝑒 𝑓𝑖𝑟𝑠𝑡 𝑟𝑜𝑤 𝑚𝑢𝑙𝑡𝑖𝑝𝑙𝑖𝑒𝑑 𝑓𝑟𝑜𝑚 𝑡ℎ𝑒 2𝑛𝑑 𝑎𝑛𝑑 3𝑟𝑑
1) 𝑎11 det [𝑎21 𝑎22 𝑎23 ]
=
𝑎31 𝑎32 𝑎33
1 0
0
𝑎22 𝑎23
𝑎11 det [0 𝑎22 𝑎23 ] = 𝑎11 det [𝑎
] = 𝑎11 𝐴{11}
32 𝑎33
0 𝑎32 𝑎33
0
1
0 𝑠𝑢𝑏𝑡𝑟𝑎𝑐𝑡 𝑡ℎ𝑒 𝑓𝑖𝑟𝑠𝑡 𝑟𝑜𝑤 𝑚𝑢𝑙𝑡𝑖𝑝𝑙𝑖𝑒𝑑 𝑓𝑟𝑜𝑚 𝑡ℎ𝑒 2𝑛𝑑 𝑎𝑛𝑑 3𝑟𝑑
2) 𝑎12 det [𝑎21 𝑎22 𝑎23 ]
=
𝑎31 𝑎32 𝑎33
0 1 0
0
0 1
𝑎21 𝑎23
𝑎
0
𝑎
𝑎
𝑎
𝑎12 det [ 21
]=
23 ] = −𝑎12 det [ 21
23 0] = −𝑎12 det [𝑎
31 𝑎33
𝑎31 0 𝑎33
𝑎31 𝑎33 0
−𝑎12 𝐴{12}
3) 𝑎13 𝐴{13}
Theorem: 𝐴 ∈ ℂ𝑛×𝑛 , 𝐶 is the 𝑛 × 𝑛 matrix with 𝑖𝑗 entries 𝑐𝑖𝑗 = (−1)𝑖+𝑗 𝐴{𝑗𝑖} , then
𝐴𝐶 = (det 𝐴)𝐼
𝐴 3×3
𝑥
𝑦
𝑧
𝑎
𝑎
𝑎
det [ 21
22
23 ] = 𝑥𝐴{11} − 𝑦𝐴{12} + 𝑧𝐴{13}
𝑎31 𝑎32 𝑎33
Suppose 𝑥 = 𝑎11 , 𝑦 = 𝑎12 , 𝑧 = 𝑎13 ⇒ det 𝐴 = 𝑎11 𝐴{11} − 𝑎12 𝐴{12} + 𝑎13 𝐴{13}
Suppose 𝑥 = 𝑎21 , 𝑦 = 𝑎22 , 𝑧 = 𝑎23 ⇒ 0 = 𝑎21 𝐴{11} − 𝑎22 𝐴{12} + 𝑎23 𝐴{13}
Suppose 𝑥 = 𝑎31 , 𝑦 = 𝑎32 , 𝑧 = 𝑎33 ⇒ 0 = 𝑎31 𝐴{11} − 𝑎32 𝐴{12} + 𝑎33 𝐴{13}
𝑎11
𝑎
[ 21
𝑎31
𝑎12
𝑎22
𝑎32
𝑎13 𝐴{11}
det 𝐴
𝑎23 ] [−𝐴{12} ] = [ 0 ]
𝑎33 𝐴{13}
0
(to be continued!)
We can do the same for the second row!
𝑎11 𝑎12 𝑎13
𝑦
𝑧 ] = 𝑥𝐴{21} − 𝑦𝐴{22} + 𝑧𝐴{23}
det [ 𝑥
𝑎31 𝑎32 𝑎33
[𝑥 𝑦 𝑧] = 𝑎1 ⇒ 0 = 𝑎11 𝐴{21} − 𝑎12 𝐴{22} + 𝑎13 𝐴{23}
[𝑥 𝑦 𝑧] = 𝑎2 ⇒ det 𝐴 = 𝑎21 𝐴{21} − 𝑎22 𝐴{22} + 𝑎23 𝐴{23}
[𝑥 𝑦 𝑧] = 𝑎2 ⇒ 0 = 𝑎31 𝐴{21} − 𝑎32 𝐴{22} + 𝑎33 𝐴{23}
So we can fill in the matrix a bit more:
𝐴{11} −𝐴{21}
𝑎11 𝑎12 𝑎13
det 𝐴
[𝑎21 𝑎22 𝑎23 ] [−𝐴{12} 𝐴{22} ] = [ 0
𝑎31 𝑎32 𝑎33
𝐴{13} −𝐴{23}
0
0
det 𝐴]
0
And if we do the same thing again for the third row we get:
𝑎11
𝑎
[ 21
𝑎31
𝑎12
𝑎22
𝑎32
𝐴{11}
𝑎13
𝑎23 ] [−𝐴{12}
𝑎33
𝐴{13}
−𝐴{21}
𝐴{22}
−𝐴{23}
𝐴{31}
det 𝐴
−𝐴{32} ] = [ 0
𝐴{33}
0
0
0
det 𝐴
0 ]
0
det 𝐴
𝑓 (𝑥) 𝑓12 (𝑥)
𝐹(𝑥) = [ 11
]
𝑓21 (𝑥) 𝑓22 (𝑥)
𝑓(𝑥) = det 𝐹(𝑥)
𝑓(𝑥 + 𝜖) − 𝑓(𝑥)
𝑓 ′ (𝑥) = lim
ϵ→0
𝜖
Shorthand notations: ⃗⃗⃗
𝑓1 = [𝑓11
⃗⃗⃗2 = [𝑓21
𝑓12 ], 𝑓
𝑓22 ]
𝑓11 (𝑥 + 𝜖) 𝑓12 (𝑥 + 𝜖)
𝑓 (𝑥) 𝑓12 (𝑥)
] − det [ 11
]
𝑓21 (𝑥 + 𝜖) 𝑓22 (𝑥 + 𝜖)
𝑓21 (𝑥) 𝑓22 (𝑥)
=
𝜖
⃗⃗⃗
⃗⃗⃗
⃗⃗⃗⃗ (𝑥 + 𝜖) − 𝑓
⃗⃗⃗⃗1 (𝑥) + ⃗⃗⃗
𝑓 (𝑥)
𝑓 (𝑥)
𝑓
𝑓1 (𝑥)
`
det [ 1 `
] + det [ 1
] − det [ 1 ]
⃗⃗⃗2 (𝑥 + 𝜖)
⃗⃗⃗
𝑓2 (𝑥 + 𝜖)
𝑓
𝑓2 (𝑥)
det [
But we know that:
⃗⃗⃗ (𝑥)
⃗⃗⃗1 (𝑥)
𝑓
𝑓
det [ 1
] = det det [
]
⃗⃗⃗
⃗⃗⃗2 (𝑥 + 𝜖) − ⃗⃗⃗
𝑓2 (𝑥 + 𝜖)
𝑓
𝑓2 (𝑥) + ⃗⃗⃗
𝑓2 (𝑥)
So the entire formula is:
⃗⃗⃗
⃗⃗⃗
𝑓 (𝑥 + 𝜖) − ⃗⃗⃗
𝑓1 (𝑥)
𝑓1 (𝑥)
det [ 1
] + det [
]
⃗⃗⃗
⃗⃗⃗2 (𝑥 + 𝜖) − ⃗⃗⃗
𝑓2 (𝑥 + 𝜖)
𝑓
𝑓2 (𝑥)
Or
1
= det [ 𝜖
1
⃗⃗⃗
𝑓1 (𝑥 + 𝜖) − ⃗⃗⃗
𝑓1 (𝑥)
] + det [
] det [
0
⃗⃗⃗
𝑓2 (𝑥)
0
⃗⃗⃗
𝑓1 (𝑥)
1] det [
]=
⃗⃗⃗
𝑓2 (𝑥 + 𝜖) − ⃗⃗⃗
𝑓2 (𝑥)
𝜖
1
⃗⃗⃗
⃗⃗⃗
𝑓1 (𝑥 + 𝜖) − ⃗⃗⃗
𝑓1 (𝑥)
𝑓1 (𝑥)
det [
+
det
]
[
⃗⃗⃗
⃗⃗⃗2 (𝑥)]
𝜖
𝑓2 (𝑥 + 𝜖) − 𝑓
⃗⃗⃗
𝑓2 (𝑥)
𝜖
′ (𝑥)
⃗⃗⃗
⃗⃗⃗
𝑓 (𝑥)
𝑓
So as 𝜖 goes to zero it equals det [ 1
] + det [ 1
]
⃗⃗⃗
⃗⃗⃗
𝑓 ′ (𝑥)
𝑓 (𝑥)
1
---- end of lesson 11
2
𝐴 ∈ ℂ𝑛×𝑛
𝐴{𝑖,𝑗} determinant of the (𝑛 − 1) × (𝑛 − 1) matrix that is obtained from 𝐴 by erasing the 𝑖’th
row and 𝑗’th column
Let 𝐶 ∈ ℂ𝑛×𝑛
𝑐𝑖𝑗 = (−1)𝑖+𝑗 𝐴{𝑗,𝑖}
𝐴𝐶 = det 𝐴 𝐼𝑛
𝐶
If det 𝐴 ≠ 0 then 𝐴 is invertible and 𝐴−1 = det 𝐴
Cramer’s rule
Let 𝐴 be an invertible 𝑛 × 𝑛 matrix, consider 𝐴𝑥 = 𝑏
𝑛
𝑛
𝑗=1
𝑗=1
𝐶𝑏
1
1
⇒𝑥=𝐴 𝑏=
⇒ 𝑥𝑖 =
∑ 𝑐𝑖𝑗 𝑏𝑗 =
∑(−1)𝑖+𝑗 𝐴{𝑗,𝑖} 𝑏𝑗 =
det 𝐴
det 𝐴
det 𝐴
−1
det[𝑎1 … 𝑎𝑖−1 𝑏 𝑎𝑖+1 … 𝑎𝑛 ]
det 𝐴
det 𝐴̃𝑖 where 𝑎𝑖 is replaced by 𝑏
𝑓
′ (𝑡)
⃗⃗⃗
⃗⃗⃗
⃗⃗⃗1 (𝑡)
𝑓1 (𝑡)
𝑓
𝑓1′ (𝑡)
⃗⃗⃗2 (𝑡)] =
= det [ ⃗⃗⃗
𝑓2′ (𝑡)] + det [𝑓
𝑓2 (𝑡) ] + det [⃗⃗⃗
⃗⃗⃗
⃗⃗⃗3 (𝑡)
⃗⃗⃗3 (𝑡)
𝑓3′ (𝑡)
𝑓
𝑓
∑ 𝑓1′ (𝑡)(−1)1+𝑗 𝐴{1,𝑗} + ∑ 𝑓2′ (𝑡)(−1)2+𝑗 𝐴{2,𝑗} + ∑ 𝑓3′ (𝑡)(−1)3+𝑗 𝐴{3,𝑗} =
∑ 𝑓1,𝑗 (𝑡)𝑐𝑗1 (𝑡) + ∑ 𝑓2,𝑗 (𝑡)𝑐𝑗2 (𝑡) + ∑ 𝑓3,𝑗 (𝑡)𝑐𝑗3 (𝑡) =
3
3 3
𝑐𝑗𝑖 (𝑡)
′
′
(𝑡)𝑔𝑗𝑖 (𝑡)
𝑓(𝑡) ∑ ∑ 𝑓𝑖,𝑗 (𝑡)
= 𝑓(𝑡) ∑ ∑ 𝑓𝑖,𝑗
𝑓(𝑡)
𝑖=1 𝑗=1
𝑖=1 𝑗=1
𝑓 ′ (𝑡)
′ (𝑡)
−1 }
𝑓(𝑡)
3
= 𝑡𝑟𝑎𝑐𝑒{𝐹
𝐹(𝑡)
A reminder: 𝑡𝑟𝑎𝑐𝑒𝐴 = ∑ 𝑎𝑖𝑖
𝑛
𝑛
𝑡𝑟𝑎𝑐𝑒𝐴𝐵 = ∑ (∑ 𝑎𝑖𝑗 𝑏𝑗𝑖 ) = 𝑡𝑟𝑎𝑐𝑒𝐵𝐴
𝑖=1
𝑗=1
0 0
]
0 1
To calculate eigenvalues find the root of the polynomial:
𝜆
0
det(𝜆𝐼2 − 𝐴) = det [
] = 𝜆(𝜆 − 1) 𝜆 = 0,1
0 𝜆−1
𝐴=[
0 0
]
0 1
𝑣2 ] = [𝑣1
𝐴𝑉 = 𝑉 [
𝐴[𝑣1
0
0
𝐴= 0
0
[−𝑎0
1
0
0
0
−𝑎1
𝑣2 ] [0 0] = [0𝑣1
0 1
0
1
0
0
−𝑎2
1
0
In general 𝐴 = [
0
−𝑎0
0
0
1
0
−𝑎3
1𝑣2 ]
0
0
0
1
−𝑎4 ]
0
0
⋱
0
]
0
1
… −𝑎𝑛−1
To calculate Eigenvalues, need to find roots of det(𝐴𝐼𝑛 − 𝐴)
0
1
𝑛 = 2,
𝐴=[
]
−𝑎0 −𝑎1
𝜆
−1
𝜆𝐼1 − 𝐴 = [
]
𝑎0 𝜆 + 𝑎1
det(𝜆𝐼3 − 𝐴) = 𝜆(𝜆 + 𝑎1 ) + 𝑎0 = 𝑎0 + 𝑎1 𝜆 + 𝜆2
det(𝜆𝐼3 − 𝐴) = 𝑎0 + 𝑎1 𝜆 + 𝑎2 𝜆2 + 𝜆3
𝑛=5
𝜆
0
𝜆𝐼5 − 𝐴 = 0
0
[−𝑎0
−1
𝜆
0
0
−𝑎1
0
−1
𝜆
0
−𝑎2
0
0
−1
𝜆
−𝑎3
0
0
0
−1
𝑎4 + 𝜆]
𝑢𝑠𝑖𝑛𝑔 𝑡ℎ𝑒 𝑟𝑖𝑔ℎ𝑚𝑜𝑠𝑡 𝑐𝑜𝑙𝑢𝑚𝑛
(−1)(−1)4+5 𝐴{45} + (𝑎4 + 𝜆)(−1)5×5 𝐴{55} =
det(𝜆𝐼5 − 𝐴)
=
𝜆 −1
𝜆 −1
det [
] + (𝜆 + 𝑎4 )𝜆4
𝜆
−1
𝑎0 𝑎1 𝑎2 (𝑎3 − 𝜆) + 𝜆
𝑎0 + 𝑎1 𝜆 + 𝑎2 𝜆2 + (𝑎3 − 𝜆)𝜆3 + 𝜆4 + 𝑎4 𝜆4 + 𝜆5
IF 𝑎 is an 𝑛 × 𝑛 companion matrix then:
1) det(𝜆𝐼𝑛 − 𝐴) = 𝑎0 + 𝑎1 𝜆 + ⋯ + 𝑎𝑛−1 𝜆𝑛−1 + 𝜆𝑛
1
1
0
𝜆
𝐴[
]=[
0
⋮
−𝑎0
𝜆𝑛−1
0
0
1
⋱
0
𝜆
][
]=
0
1
⋮
… −𝑎𝑛−1 𝜆𝑛−1
𝜆
𝜆2
⋮
𝜆𝑛−1
[−(𝑎0 + 𝑎1 𝜆+. . +𝑎𝑛−1 𝜆𝑛−1 )]
1
𝜆
Denote 𝑣(𝜆) = [
]
⋮
𝜆𝑛−1
𝑝(𝜆) = det(𝜆𝐼𝑛 − 𝐴)
0
⋮
𝐴𝑣(𝜆) = 𝜆𝑣(𝜆) − 𝑝(𝜆) [ ]
0
1
Suppose 𝑝(𝜆1 ) = 0
𝐴𝑣(𝜆1 ) = 𝜆1 𝑉(𝜆1 )
1
𝜆
So [
] is an eigenvector
⋮
𝜆𝑛−1
0
1
′ (𝜆)
2𝜆
Denote 𝑣
=
⋮
[(𝑛 − 1)𝜆𝑛−2 ]
0
⋮
𝐴𝑣 ′ (𝜆) = 𝑣(𝜆) + 𝜆𝑣 ′ (𝜆) − 𝑝′ (𝜆) [ ]
0
1
𝑝(𝜆) = (𝜆 − 𝜆1 )2 𝑞(𝜆)
𝑝′ (𝜆) = 2(𝜆 − 𝜆1 )𝑞(𝜆) + (𝜆 − 𝜆1 )2 𝑞′ (𝜆)
So 𝑝′ (𝜆1 ) = 0
1. 𝐴𝑣(𝜆) = 𝜆𝑣(𝜆) − 𝑝(𝜆)𝑒𝑛
2. 𝐴𝑣 ′ (𝜆) = 𝜆𝑣 ′ (𝜆) + 𝑣(𝜆) − 𝑝′ (𝜆)𝑒𝑛
3. 𝑎𝑣 ′′ (𝜆) = 2𝑣 ′ (𝜆) + 𝜆𝑣 ′′ (𝜆) − 𝑝′′ (𝜆)𝑒𝑛
4. 𝑎𝑣 ′′′ (𝜆) = 3𝑣 ′′ (𝜆) + 𝜆𝑣 ′′ (𝜆) − 𝑝′′′ (𝜆)𝑒𝑛
Suppose 𝑝(𝜆1 ) = 𝑝′ (𝜆1 ) = 𝑝′′ (𝜆1 ) = 𝑝′′′ (𝜆1 ) = 0
Denote:
𝑣1 = 𝑉(𝜆1 )
𝑣2 = 𝑣 ′ (𝜆1 )
𝑣 ′′ (𝜆1 )
2!
𝑣 ′′′ (𝜆1 )
𝑣4 =
3!
𝑣3 =
1⇒ 𝐴𝑣1 = 𝜆1 𝑣1
2⇒ 𝐴𝑣2 = 𝜆1 𝑣2 + 𝑣1
3⇒ 𝐴𝑣3 = 𝜆1 𝑣3 + 𝑣2
4⇒ 𝐴𝑣4 = 𝜆1 𝑣4 + 𝑣3
𝐴𝑣3 = 2𝑣2 + 𝜆1 𝑣 ′′ (𝜆1 )
𝐴[𝑣1
𝑣2
𝑣3
𝑣4 ] = [𝑣1
𝑣2
𝑣3
𝜆1
𝑣4 ] [ 0
0
0
1
𝜆1
0
0
0
1
𝜆1
0
0
0
]
1
𝜆1
2) 𝐴 is invertible⇔ 𝑎0 ≠ 0
3) The geometric multiplicity of each eigenvalue of 𝐴 is equal to 1
𝜆
−1
0
0
0
0
𝜆
−1
0
0
0
𝜆
−1
0
𝜆𝐼5 − 𝐴 = 0
0
0
0
𝜆
−1
[−𝑎0 −𝑎1 −𝑎2 −𝑎3 𝑎4 + 𝜆]
Claim that for every 𝜆 ∈ ℂ − dim ℛ𝜆𝐼5 −𝐴 ≥ 4
Or generally: dim ℛ𝜆𝐼𝑛 −𝐴 ≥ 𝑛 − 1
Since we have 4 independent columns!! Maybe the first column is dependent, and
maybe not.
𝜆 is an eigenvalue, then
𝛽 = dim 𝒩𝐴−𝜆𝐼 ≥ 1
A 5 × 5 companion matrix
det(𝜆𝐼5 − 𝐴) = (𝜆 − 𝜆1 )5 (𝜆 − 𝜆2 )2 ,
Find an invertible matrix 𝑈, 𝐽 in Jordan form such that:
𝐴𝑈 = 𝑈𝐽
Since the geometrical multiplicity is one, then:
𝜆1 1 0 | 0 0
0 𝜆1 1 | 0 0
0 0 𝜆1 | 0 0
𝐽=
− − − + − −
0 0 0 | 𝜆2 0
[ 0 0 0 | 0 𝜆2 ]
For sure!
𝜆1 ≠ 𝜆2
1
𝜆1
𝑈 = (𝜆1 )2
(𝜆1 )3
[(𝜆1 )4
0
1
2𝜆1
3(𝜆1 )2
4(𝜆1 )3
0
0
1
3𝜆1
6(𝜆1 )2
|
1
|
𝜆2
| (𝜆2 )2
| (𝜆2 )3
| (𝜆2 )4
0
1
2𝜆2
3(𝜆2 )2
4(𝜆2 )3 ]
𝑥2 = 𝑎𝑥1 + 𝑏𝑥0
𝑥3 = 𝑎𝑥2 + 𝑏𝑥1
𝑥4 = 𝑎𝑥3 + 𝑏𝑥2
⋮
𝑥𝑛 = 𝑎𝑥𝑛−1 + 𝑏𝑥𝑛−2
𝑥𝑛−2
𝑥𝑛 = [𝑏 𝑎] [𝑥 ]
𝑛−1
𝑥𝑛−2
𝑥𝑛−1 = [0 1] [𝑥 ]
𝑛−1
𝑥𝑛−1
0 1 𝑥𝑛−2
[ 𝑥 ]=[
][
]
𝑏 𝑎 𝑥𝑛−1
𝑛
Given 𝑥0 , 𝑥1 want a recipe for 𝑥𝑛
0 1
Denote 𝐴 = [
]
𝑏 𝑎
𝑥𝑗
𝑢𝑗 = [𝑥 + 1]
𝑗
𝑥0
𝑢0 = [ 𝑥 ]
1
𝑥1
𝑢1 = [𝑥 ]
2
𝑢2 = 𝐴𝑢1 = 𝐴2 𝑢0
𝑢3 = 𝐴𝑢2 = 𝐴3 𝑢0
𝑢𝑛 = 𝐴𝑛 𝑢0
So we take 𝐴, and we want to write it in a form such that 𝐴 = 𝑈𝐽𝑈 −1
𝑢𝑛 = 𝑈𝐽𝑈 −1 𝑢0
det(𝜆𝐼2 − 𝐴) =?
This is a companion matrix!
0 1
So let’s change the notation such that 𝐴 = [
]
𝑎0 𝑎1
So det(𝜆𝐼2 − 𝐴) = 𝑎0 + 𝑎1 𝜆 + 𝜆2 = −𝑏 − 𝑎𝜆 + 𝜆2
𝜆
0
𝜆
1
(𝜆 − 𝜆1 )2 ⇒ [ 1
] 𝑜𝑟 [ 1
]
0 𝜆1
0 𝜆1
det(𝜆𝐼2 − 𝐴) = {
𝜆
0
(𝜆 − 𝜆1 )(𝜆 − 𝜆2 ) ⇒ [ 1
]
0 𝜆2
𝜆
0
But the case [ 1
] cannot happen since the geometric multiplicity is 1!
0 𝜆1
In the first case:
𝑢𝑛 = [
1
𝜆1
0 𝜆1
][
1 0
1 1
][
𝜆1 𝜆1
0 −1
] 𝑢0
1
1 𝜆1
][
𝜆2 0
0 1
][
𝜆2 𝜆1
1 −1
] 𝑢0
𝜆2
Or in the second case:
𝑢𝑛 = [
--- end of lesson
1
𝜆1
0
0
𝐴= 0
0
[−𝑎0
1
0
0
0
−𝑎1
0
1
0
0
−𝑎2
0
0
1
0
−𝑎3
0
0
0 5 × 5 companion matrix
1
−𝑎4 ]
(1) det(𝜆𝐼5 − 𝐴) = 𝑎0 + 𝑎1 𝜆 + ⋯ + 𝑎4 𝜆4 + 𝜆5
(2) If 𝜆𝑗 is an eigenvalue of 𝐴, then 𝛾𝑗 =Geometrical multiplication= 1 ⇒
only one Jordan cell connected with 𝜆𝑗
det(𝜆𝐼5 − 𝐴) = (𝜆 − 𝜆1 )3 (𝜆 − 𝜆1 )2 ,
𝜆1 ≠ 𝜆2
Suppose we know that:
𝑥3 = 𝑎𝑥0 + 𝑏𝑥1 + 𝑐𝑥2
𝑥4 = 𝑎𝑥1 + 𝑏𝑥2 + 𝑐𝑥3
⋮
𝑥𝑛+3 = 𝑎𝑥𝑛 + 𝑏𝑥𝑛+1 + 𝑐𝑥𝑛+2
𝑥0
𝑥3 = [𝑎 𝑏 𝑐] [𝑥1 ]
𝑥2
𝑥1
0 1 0 𝑥0
[𝑥2 ] = [0 0 1] [𝑥1 ]
𝑥3
𝑎 𝑏 𝑐 𝑥2
𝑥𝑗
𝑣𝑗 = [𝑥𝑗+1 ]
𝑥𝑗+2
𝑣1 = 𝐴𝑣0
𝑣2 = 𝐴𝑣1 = 𝐴2 𝑣0
𝑣𝑛 = 𝐴𝑛 𝑣0
𝑥𝑛
𝑥0
𝑛 𝑥
𝑥
[ 𝑛+1 ] = 𝐴 [ 1 ]
𝑥𝑛+2
𝑥2
𝐴 is a companion matrix!
det(𝜆𝐼5 − 𝐴) = 𝑎0 + 𝑎1 𝜆 + 𝑎2 𝜆2 + 𝜆3 = −𝑎 − 𝑏𝜆 − 𝑐𝜆2 + 𝜆3
3 cases:
(𝜆 − 𝜆1 )(𝜆 − 𝜆2 )(𝜆 − 𝜆3 ) 3 𝑑𝑖𝑠𝑡𝑖𝑛𝑐𝑡 𝑒𝑖𝑔𝑒𝑛 𝑣𝑎𝑙𝑢𝑒𝑠
(𝜆 − 𝜆1 )2 (𝜆 − 𝜆2 ) 2 𝑑𝑖𝑠𝑡𝑖𝑛𝑐𝑡 𝑒𝑖𝑔𝑒𝑛 𝑣𝑎𝑙𝑢𝑒𝑠
det(𝜆𝐼5 − 𝐴) = {
(𝜆 − 𝜆1 ) 1 𝑑𝑖𝑠𝑡𝑖𝑛𝑐𝑡 𝑒𝑖𝑔𝑒𝑛 𝑣𝑎𝑙𝑢𝑒
(1) 𝐽𝑛 = [
𝜆1𝑛
𝜆𝑛2
] , 𝐴 = 𝑈𝐽𝑈 −1
𝜆𝑛3
𝑥𝑛
𝑥
[ 𝑛+1 ] = 𝑣𝑛 = 𝑈𝐽𝑛 𝑈 −1 𝑣0
𝑋𝑛+2
𝑥𝑛 = [1 0 0]𝑣𝑛 = [1 0 0
]𝑈𝐽𝑛
𝑈
−1
𝑣0 = [𝛼
𝛽
𝜆1𝑛
𝛾] [ 0
0
0
𝜆𝑛2
0
0 𝑑
0 ] [𝑒 ] =
𝜆𝑛3 𝑓
𝛼𝑑𝜆1𝑛 + 𝛽𝑒𝜆𝑛2 + 𝛾𝑓𝜆𝑛3
𝑥𝑛 = 𝛼̃𝜆𝑛1 + 𝛽̃ 𝜆𝑛2 + 𝛾̃𝛾3𝑛
We must be able to assume 𝑎 ≠ 0 otherwise, we are talking about second order and not
third order!
So assume 𝑎 ≠ 0
𝑥0 = 𝛼̃ + 𝛽̃ + 𝛾̃
𝑥1 = 𝛼̃𝜆1 + 𝛽̃ 𝜆2 + 𝛾̃𝜆3
𝑥1 = 𝛼̃𝜆21 + 𝛽̃ 𝜆22 + 𝛾̃𝜆23
1 1 1 𝛼̃
𝑥0
𝜆
𝑥
[ 1 ] = [ 1 𝜆2 𝜆3 ] [𝛽̃ ]
𝑥2
𝜆12 𝜆22 𝜆23 𝛾̃
𝜆1
(2) 𝐽 = [ 0
0
𝜆1
(3) 𝐽 = [ 0
0
1
𝜆1
0
1
𝜆1
0
0
0]
𝜆2
0
1]
𝜆1
Detour: Analyze 𝐽𝑛 for case (3)
0 1 0
𝐽 = 𝜆1 𝐼3 + 𝑁,
𝑁 = [ 0 0 1]
0 0 0
𝑘
𝑘
𝑘
𝑘
𝑘
𝐽 = (𝜆1 𝐼3 + 𝑁) = ∑ ( ) (𝜆𝐼3 )𝑘−𝑗 𝑁𝑗 = ( ) 𝜆𝑘 𝐼 + ( ) 𝜆𝑘−1 𝑁 + ( ) 𝜆𝑘−2 𝑁 2 =
𝑗
0
1
2
𝑘
𝑘
𝑗=0
𝜆1𝑘
𝑘
( ) 𝜆1𝑘−1
1
0
𝜆1𝑘
[0
0
And for case 2:
𝑘
( ) 𝜆1𝑘−2
2
𝑘 𝑘−1
( ) 𝜆1
1
𝜆1𝑘 ]
𝜆1𝑛 𝑛𝜆1𝑛−1 0
𝑥𝑛 = [1 0 0]𝑈 [ 0
𝜆1𝑛
0 ] = [𝛼
0
0
𝜆𝑛2
𝛼𝑑𝜆1𝑛 + 𝛼𝑒𝑛𝜆1𝑛−1 + 𝛽𝑒𝜆1𝑛 + 𝑓𝜆𝑛2
𝑥𝑛 = 𝛼̃𝜆𝑛1 + 𝛽̃ 𝑚𝜆1𝑛 + 𝛾̃𝜆𝑛2
𝑥𝑛 = 𝑔𝜆1𝑛 + ℎ𝑛𝜆1𝑛 + 𝑙𝜆𝑛2
𝛽
𝑑
𝛾][ ] [ 𝑒 ] = [𝛼
𝑓
𝛽
𝑑𝜆1𝑛 + 𝑒𝜆1𝑛 𝑑
𝛾] [ 𝑒𝜆1𝑛 ] [ 𝑒 ] =
𝑓
𝑓𝜆1𝑛
𝑥0 = 𝑔
𝑥1 = 𝑔𝜆1 + ℎ𝜆1 + 𝑙𝜆2
𝑥2 = 𝑔𝜆12 + ℎ𝜆12 + 𝑙𝜆22
1
𝑥0
[𝑥1 ] = [𝜆1
𝑥2
𝜆12
0
𝜆1
2𝜆12
1 𝑔
𝜆2 ] [ ℎ ]
𝜆22 𝑙
Go back to case 3:
𝑥𝑛 = [1 0 0]𝑈
𝜆1𝑛
𝑛𝜆1𝑛−1
0
𝜆1𝑛
0
[0
𝑛
𝑛
2 4
𝑥𝑛 = 𝑔𝜆1 + ℎ𝑛𝜆1 + ln 𝜆1
𝑛(𝑛 − 1) 𝑛−2
𝜆1
2
𝑈 −1 𝑣0
𝑛𝜆1𝑛−1
𝜆1𝑛
]
𝑥0 = 𝑔
𝑥1 = 𝑔𝜆1 + ℎ𝜆1 + 𝑙𝜆1
𝑥2 = 𝑔𝜆12 + ℎ2𝜆12 + 𝑙4𝜆12
1
𝑥0
𝑥
𝜆
[ 1] = [ 1
𝑥2
𝜆12
0
𝜆1
2𝜆12
0 𝑔
𝜆1 ] [ℎ ]
4𝜆12 𝑙
General Algorithm
𝑥5 = 𝑎𝑥0 + 𝑏𝑥1 + 𝑐𝑥2 + 𝑑𝑥3 + 𝑒𝑥4 𝑎 ≠ 0
𝑥𝑛+5 = 𝑎𝑥𝑛 + 𝑏𝑥𝑛+1 + 𝑐𝑥𝑛+2 + 𝑑𝑥𝑛+3 + 𝑒𝑥𝑛+4
𝑗
(1) Replace 𝑥𝑗 by 𝜆 - 𝜆𝑛+5 = 𝑎𝜆𝑛 + 𝑏𝜆𝑛+1 + ⋯ + 𝑒𝜆𝑛+4 ⇒
⏟5 − 𝑎 − 𝑏𝜆 − 𝑐𝜆2 − 𝑑𝜆3 − 𝑒𝜆4 = 0
𝜆
𝑝(𝜆)
(2) Calculate the roots of the polynomial.
This will give you the eigevalues of A.
(3) Suppose 𝑝(𝜆) = (𝜆 − 𝜆1 )3 (𝜆 − 𝜆2 )2
𝑥𝑛 = 𝛼𝜆1𝑛 + 𝛽𝑛𝜆1𝑛 + 𝛾𝑛2 𝜆𝑛1 + 𝛿𝜆𝑛2 + 𝜖𝑛𝜆𝑛2
(4) Obtain 𝛼, … , 𝜖 from the given 𝑥0 , 𝑥1 , 𝑥2 , 𝑥3 , 𝑥4
Discrete Dynamical Systems
𝑣1 = 𝐴𝑣0
𝑣2 = 𝐴𝑣1
⋮
𝑣𝑛 = 𝐴𝑛 𝑣0
𝐴 = 𝑈𝐽𝑈 −1
𝑣𝑛 = 𝑈𝐽𝑈 −1 𝑣0
In such a problem, it is often easier to calculate 𝑈 −1 𝑣0 in one step, as apposed to calculating
𝑈 −1 then 𝑈 −1 𝑣0
𝑤0 = 𝑈 −1 𝑣0
𝑈𝑤0 = 𝑣0
Solve this!
𝐵
6
[0
0
𝑤 = 𝐵−1 𝑢0
6 2 2
6
= [0 3 1] ,
𝑢0 = [ 0 ]
0 0 1
0
𝐵𝑤 = 𝑢0
2 2 𝑤1
6
5
3 1] [𝑤2 ] = [0] , ⇒ 𝑤 = [0]
0 1 𝑤3
0
0
𝜆1
𝐽 = [0
0
𝑣𝑛 = 𝑈
𝜆1𝑛
𝑛𝜆1𝑛−1
0
[0
𝜆1𝑛
0
1 0
𝜆1 1 ]
0 𝜆1
𝑛(𝑛 − 1) 𝑛−2
𝜆1
2
𝑈 −1 𝐽
𝑛𝜆1𝑛−1
𝜆1𝑛
]
Evaluate:
1
𝑛2
𝑣𝑛
𝑛 2
𝜆1 𝑛
=𝑈 0
[0
1
𝑛𝜆1
1
𝑛2
0
𝑛3 − 𝑛 1
∙
2𝑛2 𝜆12
1
𝑉 −1 𝑣0
𝑛𝜆1
1
]
𝑛2
Differential Equations
𝑥 ′′ (𝑡) = 𝑎𝑥 ′ (𝑡) + 𝑏𝑥(𝑡)
Let
𝑥1 (𝑡) = 𝑥(𝑡)
𝑥2 (𝑡) = 𝑥 ′ (𝑡)
[
𝑥1′ (𝑡)
]=
𝑥2′ (𝑡)
𝑥1 ′
0
[𝑥 ] = [
𝑏
2
1 𝑥1
][ ]
𝑎 𝑥2
𝑥 ′ (𝑡) = 𝐴𝑥(𝑡)
𝐴2 𝐴3
𝑒 =𝐼+𝐴+
+
+⋯
2! 3!
2 2
𝑡 𝐴
𝑒 𝑡𝐴 = 𝐼 + 𝑡𝐴 +
+⋯
2!
𝑒 𝐴 𝑒 𝐵 = 𝑒 𝐴+𝐵 if 𝐴𝐵 = 𝐵𝐴!
𝐴
∞
𝑒
𝐴+𝐵
=∑
𝑘=0
(𝐴 + 𝐵)𝑘
𝐴𝑗
𝐵𝑙
=∑ ∑ =⋯
𝑘!
𝑗!
𝑙!
(𝑒 𝑡𝐴 𝑒 𝜖𝐴 − 𝑒 𝑡𝐴 ) 𝑏𝑦 𝑒𝑥𝑝𝑙𝑒𝑛𝑎𝑡𝑖𝑜𝑛 𝑏𝑒𝑙𝑜𝑤
(𝑒 𝜖𝐴 − 𝐼)
(𝑒 (𝑡+𝜖)𝐴 − 𝑒 𝑡𝐴 )
= lim
=
lim 𝑒 𝑡𝐴
= 𝑒 𝑡𝐴 𝐴
ϵ→∞
ϵ→∞
ϵ→∞
𝜖
𝜖
𝜖
= 𝐴𝑒 𝑡𝐴
lim
Explanation:
𝜖 2 𝐴2
+⋯
2!
(𝑒 𝜖𝐴 − 𝐼)
𝜖 2 𝐴2
= 𝜖𝐴 +
+⋯
𝜖
2!
𝑥(𝑡) = 𝑒 𝑡𝐴 𝑢
𝑢 is constant
𝑒 𝜖𝐴 = 𝐼 + 𝜖𝐴 +
𝑥 ′ (𝑡) = 𝐴𝑒 𝑡𝐴 𝑢 = 𝐴𝑥(𝑡)
𝑥(𝑡) = 𝑒 𝑡𝐴 𝑥(0)
If 𝐴 = 𝑈𝐽𝑈 −1
Claim:
𝑒
𝑡𝐴
∞
∞
∞
𝑘=0
𝑘=0
𝑘=0
(𝑡𝐴)𝑘
(𝑡𝑈𝐽𝑈 −1 )𝑘
(𝑡𝐽)𝑘
𝑈(𝑡𝐽)𝑘 𝑈 −1
=∑
=∑
=∑
= 𝑈 (∑
) 𝑈 −1 = 𝑈𝑒 𝑡𝐽 𝑈 −1
𝑘!
𝑘!
𝑘!
𝑘!
𝑥1 (𝑡)
] = 𝑈𝑒 𝑡𝐽 𝑈 −1 𝑥(0)
𝑥2 (𝑡)
𝑥(𝑡) = [1 0]𝑥(𝑡) = [1 0]𝑈𝑒 𝑡𝐽 𝑈 −1 𝑥(0)
𝑥(𝑡) = [
Case 1: det(𝜆𝐼2 − 𝐴) = (𝜆 − 𝜆1 )(𝜆 − 𝜆2 ), 𝜆1 ≠ 𝜆2 , 𝐽 = [
𝜆1
0
0
]
𝜆2
𝑡𝜆1
0 ] ⇒ 𝑥(𝑡) = 𝛼𝑒 𝜆1 𝑡 + 𝛽𝑒 𝜆2 𝑡
𝑒 𝑡𝐽 = [𝑒
0
𝑒 𝑡𝜆2
Case 2: det(𝜆𝐼2 − 𝐴) = (𝜆 − 𝜆1 )2
𝜆
0
𝜆
1
𝐽=[ 1
] or [ 1
]
0 𝜆1
0 𝜆1
But the first form doesn’t exist!!! So we only have the second form
𝑒 𝑡𝐽 = 𝑒 𝑡(𝜆1 𝐼+𝑁) = 𝑒 𝜆1 𝑡𝐼 ∙ 𝑒 𝑡𝑁
But 𝑁 2 is zero! So 𝑒 𝑡𝑁 = 𝐼 + 𝑡𝑁
Therefore: 𝑒 𝑡𝐽 = 𝑒 𝜆1 𝑡𝐼 ∙ (𝐼 + 𝑡𝑁)
--- end of lesson
∞
𝐴
𝑒 =∑
𝑘=0
𝐴𝑘
𝑘!
𝐴 𝐵
𝑒 𝑒 = 𝑒 𝐴+𝐵 if 𝐴𝐵 = 𝐵𝐴
𝑓(𝑡) = 𝑒 𝑡𝐴 𝑣 𝑓 ′ (𝑡) = 𝐴𝑒 𝑡𝐴 𝑣 = 𝐴𝑓(𝑡)
Final solution of:
𝑎0 𝑥 + 𝑎1 𝑥 ′ (𝑡) + 𝑎2 𝑥 ′′ (𝑡) + 𝑎3 𝑥 ′′′ (𝑡) + 𝑥 𝐼𝑉 (𝑇) = 0,
𝑥(0) = 𝑏1
𝑥 ′ (0) = 𝑏2
𝑥 ′′ (0) = 𝑏3
𝑥 ′′′ (0) = 𝑏4
0≤𝑡≤𝑑
Strategy:
Set
𝑥1 (𝑡) = 𝑥(𝑡)
𝑥2 (𝑡) = 𝑥 ′ (𝑡)
𝑥3 (𝑡) = 𝑥 ′′ (𝑡)
𝑥4 (𝑡) = 𝑥 ′′′ (𝑡)
𝑥1′ (𝑡)
𝑥2
𝑥1
′ (𝑡)
𝑥3
𝑥2
𝑥
𝑥(𝑡) = [𝑥 ] ⇒ 𝑥 ′ (𝑡) = 2′
=[
]=
𝑥4
𝑥3 (𝑡)
3
𝑥4
−𝑎0 𝑥 − 𝑎1 𝑥 ′ − 𝑎2 𝑥 ′′ − 𝑎3 𝑥 ′′′
[𝑥4′ (𝑡)]
𝑥2
𝑥3
[
]
𝑥4
−𝑎0 𝑥1 − 𝑎1 𝑥2 − 𝑎2 𝑥3 − 𝑎3 𝑥4
𝑥1 = 𝑥,
𝑥1′ = 𝑥 ′ ,
𝑥2′ = 𝑥1′′ = 𝑥 ′′
So in matrix notation:
𝑥1
0
1
0
0
𝑥2
0
0
1
0
[
] [𝑥 ]
0
0
0
1
3
−𝑎0 −𝑎1 −𝑎2 −𝑎3 𝑥4
⃗⃗⃗
𝑥 ′ (𝑡) = 𝐴𝑥(𝑡)
𝑥 (𝑡) = 𝑒 𝑡𝐴 𝑥(𝑡)
𝑥(𝑡) = [1 0 0
0]𝑒 𝑡𝐴 𝑥(0)
𝐴 = 𝑈𝐽𝑈 −1 where 𝐽 is of Jordan form.
(1) Calculate Eigenvalues of 𝐴
These are the roots of the polynomial det(𝜆𝐼 − 𝐴) = 𝑎0 + 𝑎1 𝜆 + 𝑎2 𝜆2 + 𝑎3 𝜆3 + 𝜆4
If all are different, then 𝐽 is diagonal!
𝐴 = 𝑈𝐽𝑈 −1 ,
𝑒 𝑡𝐴 = 𝑈𝑒 𝑡𝐴 𝑈−1
𝑒 𝜆1 𝑡
⋱
𝑥(𝑡) = [1 0 0 0]𝑈 [
] 𝑈 −1 𝑥(0)
⋱
𝑒 𝜆4 𝑡
𝐴 = 𝑈𝐽𝑈
𝑒 𝜆1 𝑡
𝑈[
−1
,
𝐴𝑈 = 𝑈𝐽
𝑒 𝜆1 𝑡
⋱
⋱
] = [𝑢1
𝑢2
𝑢3
𝑢4 ] [
𝑒 𝜆4 𝑡
𝑒 𝑢1 + 𝑒 𝑢2 + 𝑒 𝜆3 𝑡 𝑢3 + 𝑒 𝜆4 𝑡 𝑢4
𝑥(𝑡) = 𝑎𝑒 𝜆1 𝑡 + 𝑏𝑒 𝜆2 𝑡 + 𝑐𝑒 𝜆3 𝑡 + 𝑑𝑒 𝜆4 𝑡
𝜆1 𝑡
⋱
]=
⋱
𝑒 𝜆4 𝑡
𝜆2 𝑡
1. Find the “eigenvalues” by substituting 𝑒 𝜆𝑡 into the given differential and then dividing
through by 𝑒 𝜆𝑡
𝑎0 𝑒 𝜆𝑡 + 𝑎1 𝜆𝑒 𝜆𝑡 + 𝑎2 𝜆2 𝑒 𝜆𝑡 + 𝑎3 𝜆3 𝑒 𝜆𝑡 + 𝜆4 𝑒 𝜆𝑡 = 0
After we cross out 𝑒 𝜆𝑡 we get the same polynomial as we got from the determinant!
2. Find the roots.
Case (1) – 4 distinct roots: 𝜆1 , 𝜆2 , 𝜆3 , 𝜆4
𝑥(𝑡) = 𝛼𝑒 𝜆1 𝑡 + 𝛽𝑒 𝜆2 𝑡 + 𝛾𝑒 𝜆3 𝑡 + 𝛿𝑒 𝜆4 𝑡
3. Calculate 𝛼, 𝛽, 𝛾, 𝛿 from the given initial conditions
2. Case (2) – 𝑎0 + 𝑎1 𝜆 + ⋯ + 𝜆4 = (𝜆 − 𝜆1 )4
𝜆1
0
𝐽=[
0
0
1
𝜆1
0
0
0
1
𝜆1
0
0
0
]
1
𝜆1
𝑒 𝑡𝐽 = 𝑒 𝑡(𝜆1 𝐼+𝑁)
0
0
Where 𝑁 = [
0
0
0 0
0 0
So 𝑁 2 = [
0 0
0 0
1
0
0
0
1
0
0
0
0 0
1 0
]
0 1
0 0
0
0 0
1
0 0
] , 𝑁3 = [
0
0 0
0
0 0
0
0
0
0
1
0
] , 𝑁4 = 0
0
0
So:
1
𝑡
𝑡2
2
𝑡2𝑁2 𝑡3𝑁3
+
) = 𝑒 𝑡𝜆1 0 1 𝑡
2!
3!
0 0 1
[0 0 0
0]𝑈𝐸𝑡𝐽 𝑈 −1 𝑥(0) = 𝛼𝑒 𝑡𝜆1 + 𝛽𝑡𝑒 𝑡𝜆2 + 𝛾𝑡 2 𝑒 𝑡𝜆3 + 𝛿𝑡 3 𝑒 𝑡𝜆4
𝑒 𝑡𝐽 = 𝑒 𝑡(𝜆1 𝐼+𝑁) = 𝑒 𝑡𝜆1 𝐼 𝑒 𝑡𝑁 = 𝑒 𝑡𝜆1 ∙ (𝐼 + 𝑡𝑁 +
𝑥(𝑡) = [1
0 0
2. case (3) 𝑎0 + ⋯ + 𝜆4 = (𝜆 − 𝜆1 )3 (𝜆 − 𝜆2 ), 𝜆1 ≠ 𝜆2
𝛼𝑒 𝑡𝜆1 + 𝛽𝑡𝑒 𝑡𝜆1 + 𝛾𝑡 2 𝑒 𝑡𝜆1 + 𝛿𝑡 3 𝑒 𝑡𝜆2
So far everything was homogeneous. But what happens if not?
𝑎0 𝑥(5) + 𝑎1 𝑥 ′ (𝑡) + 𝑥 ′′ (𝑡) = 𝑓(𝑡),
𝑥1 = 𝑥,
𝑥2 = 𝑥1′
𝑡≥0
𝑥2
𝑥1
0
[𝑥 ] = [𝑓(𝑡) − 𝑎 𝑥 − 𝑎 𝑥 ] = [
−𝑎
2
0 1
1 2
0
𝑥1
1
0
][ ] + [
]
𝑓(𝑡)
−𝑎1 𝑥2
𝑥 ′ (𝑡) = 𝐴𝑥(𝑡) + 𝐵𝑢(𝑡)
(𝑒 −𝑡𝐴 𝑥)′ = −𝐴𝑒 −𝑡𝐴 𝑥 + 𝑒 −𝑡𝐴 𝑥 ′ = 𝑒 −𝑡𝐴 (𝑥 ′ − 𝐴𝑥) = 𝑒 −𝑡𝐴 𝐵𝑢(𝑡)
𝑑 −𝑠𝐴
(𝑒 𝑥(𝑠)) = 𝑒 −𝑠𝐴 𝐵𝑢(𝑠)
𝑑𝑠
We can integrate both sides!
𝑡
𝑡
𝑑
∫ (𝑒 −𝑠𝐴 𝑥(𝑠)) ∙ 𝑑𝑠 = ∫ 𝑒 −𝑠𝐴 𝐵𝑢(𝑠) ∙ 𝑑𝑠
𝑑𝑠
0
0
𝑡
𝑡
𝑒 −𝑠𝐴 𝑥(𝑠)| = ∫ 𝑒 −𝑠𝐴 𝐵𝑢(𝑠) ∙ 𝑑𝑠
0
0
𝑡
𝑡
𝑒 −𝑡𝐴 𝑥(𝑡) − 𝑥(0) = ∫ 𝑒 −𝑠𝐴 𝐵𝑢(𝑠) ∙ 𝑑𝑠 ⇒ 𝑥(𝑡) = 𝑒 𝑡𝐴 𝑥(0) + 𝑒 𝑡𝐴 (∫ 𝑒 −𝑠𝐴 𝐵𝑢(𝑠) ∙ 𝑑𝑠)
0
0
Norm Linear Spaces
Before we actually go into norm linear spaces, we need to prove some inequalities.
𝑡3
3!
𝑡2
2
𝑡
1]
Inequalities
(1)
𝑎 > 0,
𝑏 > 0,
1 1
𝑠 > 1, + = 1
𝑡 𝑠
𝑡 > 1,
𝑎𝑠 𝑏𝑡
+
𝑠
𝑡
Proof: 𝑠 + 𝑡 = 𝑡𝑠 ⇒ 𝑡𝑠 − 𝑠 − 𝑡 + 1 = 1 ⇒ (𝑡 − 1)(𝑠 − 1) = 1
Following are equivalent for a pair of numbers 𝑡, 𝑠 with 𝑡 > 1, 𝑠 > 1
1 1
+ =1
𝑡 𝑠
(𝑡 − 1)(𝑠 − 1) = 1
(𝑡 − 1)𝑠 = 𝑡
(𝑠 − 1)𝑡 = 𝑠
⇒ 𝑎𝑏 <
TODO: Draw graph of 𝑦 = 𝑥 𝑠−1
2. (Holder’s Inequality)
1
1
If 𝑠 > 1, 𝑡 > 1 and 𝑠 + 𝑡 = 1
𝑘
1
𝑠
𝑘
1
𝑡
𝑘
𝑠
𝑡
∑|𝑎𝑗 𝑏𝑗 | ≤ (∑|𝑎𝑗 | ) ∙ (∑|𝑏𝑗 | )
𝑗=1
𝑗=1
𝑗=1
Inequality is obvious if right hand side is 0.
It suffices to focus on a case where the right hand side is strictly positive.
𝑎𝑗
𝛼𝑗 =
𝑘
1,
𝑠 𝑠
(∑𝑘𝑗=1|𝑎𝑗 | )
𝑘
𝛽𝑗 =
𝑠
𝑘
𝑏𝑗
1
𝑠 𝑡
𝑘
(∑𝑗=1|𝑏𝑗 | )
𝑡
|𝛼𝑗 |
|𝛽𝑗 |
1 1
∑|𝛼𝑗 ∙ 𝛽𝑗 | ≤ ∑
+∑
= + =1
𝑠
𝑡
𝑠 𝑡
𝑗=1
𝑘
𝑗=1
𝑗=1
𝑘
𝑠
𝑠
∑𝑘𝑗=1|𝑎𝑗 |
|𝛼𝑗 |
∑|𝛼𝑗 | = ∑ 𝑘
=
=1
∑𝑖=1|𝑎𝑖 |𝑠 ∑𝑘𝑖=1|𝑎𝑖 |𝑠
𝑠
𝑗=1
Similarly
𝑘
∑(
𝑗=1
𝑗=1
𝑡
∑|𝑏𝑗 |
=1
|𝑎𝑗 |
(∑𝑘𝑖=1|𝑎𝑖
1
|𝑠 )𝑠
∙
|𝑏𝑗 |
1)
𝑘 |𝑏 |𝑡 𝑡
(∑𝑙=1 𝑙 )
≤1⇒
𝑘
1
𝑠
𝑘
1
𝑡
𝑘
𝑠
𝑡
∑|𝑎𝑗 𝑏𝑗 | ≤ (∑|𝑎𝑗 | ) ∙ (∑|𝑏𝑗 | )
𝑗=1
𝑗=1
𝑗=1
So it’s proven.
Let’s prove something extra:
𝑘
𝑘
∑|𝑎𝑗 𝑏𝑗 | = (∑|𝑎𝑖 |) ∙ (∑|𝑏𝑖 |)
𝑖=1
𝑖=1
3. Shwartz inequality:
𝑘
𝑘
𝑘
2
2
∑|𝑎𝑗 𝑏𝑗 | ≤ √∑|𝑎𝑗 | ∙ √∑|𝑏𝑗 |
𝑗=1
𝑗=1
𝑗=1
4.(Minkowski inequality) If 𝑠 ≥ 1, then
1
𝑠
𝑘
1
𝑠
𝑘
𝑠
1
𝑡
𝑘
𝑠
𝑡
(∑|𝑎𝑗 + 𝑏𝑗 | ) ≤ (∑|𝑎𝑗 | ) + (∑|𝑏𝑗 | )
𝑗=1
𝑗=1
𝑗=1
If 𝑠 = 1, its easy.
Suppose 𝑠 > 1, consider:
𝑘
𝑘
𝑘
𝑠
∑|𝑎𝑗 + 𝑏𝑗 | = ∑|𝑎𝑗 + 𝑏𝑗 | ∙ |𝑎𝑗 + 𝑏𝑗 |
𝑗=1
𝑘
𝑠−1
= ∑(|𝑎𝑗 | + |𝑏𝑗 |) ∙ |𝑎𝑗 + 𝑏𝑗 |
𝑗=1
𝑠−1
=
𝑗=1
𝑘
∑|𝑎𝑗 | ∙ |𝑎𝑗 + 𝑏𝑗 |
𝑗=1
⏟
𝑠−1
+ ∑|𝑏𝑗 | ∙ |𝑎𝑗 + 𝑏𝑗 |
𝑗=1
⏟
(1)
𝑠−1
(2)
1
1
By Holder’s inequality: 𝑠 + 𝑡 = 1
1
𝑠 𝑠
1
𝑡
𝑘
(𝑠−1)𝑡
(1) ≤ (∑|𝑎𝑗 | ) ∙ (∑|𝑎𝑗 + 𝑏𝑗 |
1
𝑠 𝑠
𝑠
𝑗=1
𝑠
1
𝑡
(2) ≤ (∑|𝑏𝑗 | ) ∙ (∑𝑘𝑗=1|𝑎𝑗 + 𝑏𝑗 | )
𝑘
𝑠
1
𝑠 𝑠
1
𝑠 𝑠
1
𝑡
𝑘
𝑠
∑|𝑎𝑗 + 𝑏𝑗 | ≤ {(∑|𝑎𝑗 | ) + (∑|𝑏𝑗 | ) } (∑|𝑎𝑗 + 𝑏𝑗 | )
𝑗=1
--- end of lesson
𝑠
) = (∑|𝑎𝑗 | ) ∙ (∑|𝑎𝑗 + 𝑏𝑗 | )
𝑗=1
1
𝑠
1
𝑡
𝑘
𝑗=1
Normed Linear Spaces (NLS)
Holder’s inequality:
1
1
If 𝑠 > 1, 𝑡 > 1 and 𝑠 + 𝑡 = 1
𝑘
1
𝑠
𝑘
1
𝑡
𝑘
𝑠
𝑡
∑|𝑎𝑗 𝑏𝑗 | ≤ (∑|𝑎𝑗 | ) ∙ (∑|𝑏𝑗 | )
𝑗=1
𝑗=1
𝑗=1
Cauchy-Shwartz inequality:
𝑘
𝑘
𝑘
2
2
∑|𝑎𝑗 𝑏𝑗 | ≤ √∑|𝑎𝑗 | ∙ √∑|𝑏𝑗 |
𝑗=1
𝑗=1
𝑗=1
(Minkowski inequality) If 𝑠 ≥ 1, then
1
𝑠
𝑘
1
𝑠
𝑘
𝑠
1
𝑡
𝑘
𝑠
𝑡
(∑|𝑎𝑗 + 𝑏𝑗 | ) ≤ (∑|𝑎𝑗 | ) + (∑|𝑏𝑗 | )
𝑗=1
𝑗=1
𝑗=1
A vector space 𝒱 is said to be a normed-linear space over 𝔽 if for each vector 𝑣 ∈ 𝒱 there is
𝜑(𝑣) with the following properties:
1) 𝜑(𝑣) ≥ 0
2) 𝜑(𝑣) = 0 ⇔ 𝑣 = 0
3) 𝜑(𝛼𝑣) = |𝛼| ∙ 𝜑(𝑣)
4) 𝜑(𝑢 + 𝑣) ≤ 𝜑(𝑢) + 𝜑(𝑣) (triangle inequality)
Example 1:
𝒱 = ℂ𝑛 , fix 𝑠 ≥ 1
1
Define 𝜑(𝑥) = (∑𝑛𝑖=1|𝑥𝑖 |𝑠 )𝑠
Disclaimer: I don’t have the mental forces to copy the proof why 𝜑 has all properties.
Example 2: 𝒱 = ℂ𝑛
Define 𝜑(𝑥) = max{|𝑥1 |, … , |𝑥𝑛 |}
Disclaimer 2: See disclaimer 1.
A fundamental fact that on a finite dimension normed-linear space all norms are equivalent.
i.e. If 𝜑(𝑥) is a norm, and 𝜓(𝑥) is a norm, then can find constants 𝛼1 > 0, 𝛼2 > 0 such that:
𝛼1 𝜑(𝑥) ≤ 𝜓(𝑥) ≤ 𝛼2 𝜑(𝑥)
It might seem not symmetric. But in fact, the statement implies:
𝜓(𝑥)
𝜓(𝑥)
𝜑(𝑥) =
,
𝜑(𝑥) ≥
𝛼
𝛼2
So:
𝜓(𝑥)
𝛼2
𝜓(𝑥)
𝛼1
≤ 𝜑(𝑋) ≤
The three most important norms are: ‖𝑥‖1 , ‖𝑥‖2 , ‖𝑥‖∞
‖𝑥‖∞ ≤ ‖𝑥‖2 ≤ ‖𝑥‖1 ≤ 𝛾‖𝑥‖∞
𝑥1
𝑥=[ ⋮ ]
𝑥𝑛
𝑛
|𝑥𝑖 | =
√|𝑥|2
2
≤ √∑|𝑥𝑗 | ⇒ |𝑥𝑖 | ≤ ‖𝑥‖2 ⇒ max|𝑥𝑖 | ≤ ‖𝑥‖2
𝑗=1
(‖𝑥‖2 )2 = |𝑥1 |2 + ⋯ + |𝑥𝑛 |2 ≤ |𝑥1 |(|𝑥1 | + ⋯ + |𝑥𝑛 |) + ⋯ + |𝑥𝑛 |(|𝑥1 | + ⋯ + |𝑥𝑛 |) =
(|𝑥1 | + ⋯ + |𝑥𝑛 |)2
But since it was normed-negative we are allowed to take the root and therefore:
‖𝑥‖2 ≤ ‖𝑥‖1
𝑛
𝑛
‖𝑥‖1 = ∑|𝑥𝑖 | ≤ ∑‖𝑥𝑖 ‖∞ = 𝑛‖𝑥𝑖 ‖∞
𝑖=1
𝑖=1
(so in our case 𝑛 = 𝛾)
𝑛
1
𝑠
‖𝑥‖𝑠 = (∑|𝑥𝑖 |) ,
1≤𝑠<∞
𝑖=1
‖𝑥‖∞ = max|𝑥𝑖 |
If 𝒱 is a vector space of functions defined on a finite interval 𝑎 ≤ 𝑡 ≤ 𝑏
𝑏
1
𝑠
‖𝑓‖𝑠 = (∫|𝑓(𝑡)|𝑠 𝑑𝑡)
𝑎
‖𝑥‖∞ = max|𝑓(𝑡)| ,
𝑎≤𝑡≤𝑏
Matrix Norms (Transformation Norms)
Let 𝐴 ∈ ℂ𝑝×𝑞 , ‖𝐴‖ = max
‖𝐴𝑥 ‖2
‖𝑥‖2
, 𝑥 ∈ ℂ𝑞 , 𝑥 ≠ 0
0≤𝑥<1
What is the maximum value of 𝑥 in this interval? There’s no maximum value.
Lemma: Let 𝐴 ∈ ℂ𝑝×𝑞 and let
‖𝐴𝑥‖2
𝛼1 = max {
|𝑥 ∈ ℂ𝑞 , 𝑥 ≠ 0}
‖𝑥‖2
𝛼2 = max{‖𝐴𝑥‖2|𝑥 ∈ ℂ𝑞 , ‖𝑥‖2 = 1}
𝛼3 = max{‖𝐴𝑥‖2|𝑥 ∈ ℂ𝑞 , ‖𝑥‖2 ≤ 1}
Then 𝛼1 = 𝛼2 = 𝛼3
‖𝐴𝑥‖2
‖𝑥‖2
Take 𝑥 ∈ ℂ𝑞 , 𝑥 ≠ 0 and consider
‖𝐴𝑥‖2
𝑥
= ‖𝐴 ∙
‖ ≤ 𝛼2 ≤ 𝛼3
‖𝑥‖2
‖𝑥‖2 2
‖𝑥‖
𝑥
Now we know that ‖‖𝑥‖ ‖ = ‖𝑥‖2 = 1
2
2
So we get: 𝛼1 ≥ 𝛼2 ≥ 𝛼3
Take 𝑥 ∈ ℂ𝑞 , ‖𝑥‖2 ≤ 1, 𝑥 ≠ 0
‖𝐴𝑥‖2 =
‖𝐴𝑥‖2
,
‖𝑥‖2
‖𝑥‖2 ≤
‖𝐴𝑥‖2
≤ 𝛼1
‖𝑥‖2
And for 𝑥 = 0 it’s a special case where the inequality is true as well.
So we get equality.
‖𝐴𝑥‖
ℂ𝑝×𝑞 is a normed linear space in respect to the norm ‖𝐴‖ = max { ‖𝑥‖ 2 |𝑥 ∈ ℂ𝑞 , 𝑥 ≠ 0}
2
It’s clear 𝜑(𝐴) ≥ 0
Is 𝜑(𝐴) = 0 ⇒ every vector we put in there equals to zero.
𝜑(𝛼𝐴) = |𝛼|𝜑(𝐴)
Lemma: ‖𝐴𝑥‖2 ≤ 𝛼1 ‖𝑥‖2
Proof: 𝑥 ≠ 0,
‖𝐴𝑥‖2
‖𝑥‖2
≤ 𝛼 ⇒ ‖𝐴𝑥‖2 ≤ 𝛼1 ‖2‖2
If 𝑥 = 0 then it’s also true. So true for all 𝑥.
𝜑(𝐴 + 𝐵) ≤
‖𝐴 + 𝐵𝑥‖2 ‖𝐴𝑥‖2 ‖𝐵𝑥‖2
≤
+
≤ 𝜑(𝐴) + 𝜑(𝐵)
‖𝑥‖2
‖𝑥‖2
‖𝑥‖2
ℂ𝑝×𝑞 is a normed-linear space with respect to the norm:
‖𝐴𝑥‖2
‖𝐴‖ = max {
|𝑥 ≠ 0}
‖𝑥‖2
Can also show (in a very similar manner)
‖𝐴𝑥‖𝑠
‖𝐴‖𝑠,𝑠 = max {
|𝑥 ≠ 0}
‖𝑥‖𝑠
‖𝐴‖𝑠,𝑡 = max {
‖𝐴𝑥‖𝑡
|𝑥 ≠ 0}
‖𝑥‖𝑠
Lemma:
Let 𝐴 ∈ ℂ𝑝×𝑞 , 𝐵 ∈ ℂ ∈𝑞×𝑟
‖𝐴𝐵‖ ≤ ‖𝐴‖ ∙ ‖𝐵‖
Proof: ‖𝐴𝐵𝑥‖2 ≤ ‖𝐴‖‖𝐵𝑥‖2 ≤ ‖𝐴‖‖𝐵‖‖𝑥‖2
If 𝑥 ≠ 0
‖𝐴𝐵𝑥‖2
‖𝑥‖2
≤ ‖𝐴‖‖𝐵‖ ⇒ ‖𝐴𝐵‖ ≤ ‖𝐴‖‖𝐵‖
Theorem: 𝐴 ∈ ℂ𝑛×𝑛 suppose 𝐴 invertible.
If 𝐵 ∈ ℂ𝑛×𝑛 and 𝐵 is “close enough” to 𝐴 then 𝐵 is invertible.
Proof: Let 𝐵 = 𝐴 − (𝐴 − 𝐵) = 𝐴(𝐼 − 𝐴−1 (𝐴 − 𝐵)) = 𝐴 (𝐼 −
𝑋
⏟
)
𝐴−1 (𝐴−𝐵)
If ‖𝑋‖ < 1, then 𝐼 − 𝑋 is invertible.
Enough to show 𝒩𝐼−𝑋 = {0}
Let 𝑢 ∈ 𝒩𝐼−𝑋 ⇒ (𝐼 − 𝑋)𝑢 = 0 ⇒ 𝑢 = 𝑋𝑢 ⇒ ‖𝑢‖2 = ‖𝑋𝑢‖2 ≤ ‖𝑋‖‖𝑢‖2
(1 − ‖𝑋‖) ∙ ‖𝑢‖2 ≤ 0 ⇒ ‖𝑢‖2 ≤ 0
So ‖𝑢‖ = 0 ⇒ 𝑢 = 0.
--- end of lesson
NLS –normed linear spaces
𝐴 ∈ ℂ𝑝×𝑞 , 𝐵 ∈ ℂ𝑞×𝑟
‖𝐴𝐵‖2,2 ≤ ‖𝐴‖2,2 ∙ ‖𝐵‖2,2
‖𝐴𝑥‖2,2 ≤ ‖𝐴‖2,2 ∙ ‖𝑥‖2
‖𝐴𝑥‖𝑡
‖𝐴‖𝑠,𝑡 = max {
|𝑥 ≠ 0}
‖𝑥‖𝑠
‖𝐴‖2,2 = max{‖𝐴𝑥‖2,2 : ‖𝑥‖2 = 1} = max{‖𝐴‖2,2 : ‖𝑥‖2 ≤ 1}
Theorem: If 𝐴 ∈ ℂ𝑝×𝑝 that is invertible and 𝐵 ∈ ℂ𝑝×𝑝 that is close enough to 𝐴, then 𝐵 is also
invertible.
First idea: 𝑥 ∈ ℂ𝑝×𝑝 , ‖𝑥‖ < 1, then 𝐼𝑝 − 𝑋 is invertible.
Let 𝑢 ∈ 𝒩𝐼𝑝 −𝑋 ⇒ (𝐼𝑝 − 𝑋)𝑢 = 0 ⇒ 𝑢 = 𝑋𝑢
⇒ ‖𝑢‖ = ‖𝑋𝑢‖ ⇒ ‖𝑋𝑢‖ ≤ ‖𝑋‖ ∙ ‖𝑢‖ ⇒ (1 − ‖𝑋‖)‖𝑢‖ ≤ 0 ⇒ ‖𝑢‖ ≤ 0 ⇒ 𝑢 = 0
𝛼∈ℂ
𝑆𝑛 = 1 + 𝛼 + 𝛼 2 + ⋯ + 𝛼 𝑛
𝛼𝑆𝑛 = 𝛼 + 𝛼 2 + ⋯ + 𝛼 𝑛 + 𝛼 𝑛+1
(1 − 𝛼)𝑆𝑛 = 1 − 𝛼 𝑛+1
1 − 𝛼 𝑛+1
1
𝑆𝑛 = 1 + 𝛼 + ⋯ + 𝛼 𝑛 =
→
1−𝛼
1−𝛼
∞
1
|𝛼| < 1 ∑ 𝛼 𝑗 =
1−𝛼
𝑗=1
So as 𝑛 ↑ ∞
𝑥 𝑛+1 → 0
− 0‖ → 0
‖𝑥 𝑛+1
So ‖𝑥 𝑛+1 ‖ ≤ ‖𝑥‖𝑛+1
If ‖𝑥‖ < 1 ⇒ (𝐼 − 𝑋)𝑆𝑛 = 𝐼𝑝 − 𝑋 𝑛+1 → 𝐼𝑝 𝑎𝑠 𝑛 ↑ ∞
∞
(𝐼 − 𝑋)
−1
∞
−1
𝑗
= ∑ 𝑥 𝑖. 𝑒. ‖∑ 𝑥 𝑗 − (𝐼𝑝 − 𝑥) ‖ → 0
𝑗=0
𝑗=0
Theorem: If 𝐴 ∈ ℂ𝑝×𝑞 and 𝐴 is left invertible, 𝐵 ∈ ℂ𝑝×𝑞 and 𝐵is close enough to 𝐴, then 𝐵 is left
invertible.
Proof: 𝐴 is left invertible ⇒ there is a 𝐶 ∈ ℂ𝑞×𝑝 such that 𝐶𝐴 = 𝐼𝑞
𝐵 = 𝐴 − (𝐴 − 𝐵) ⇒ 𝐶𝐵 = 𝐶𝐴 − ⏟
𝐶(𝐴 − 𝐵) = 𝐼𝑞 − 𝑋
𝑋∈ℂ𝑞×𝑞
If ‖𝑋‖ < 1 ⇒ 𝐼𝑞 − 𝑋 is invertible.
Doing that, we find (𝐼𝑞 − 𝑋)
−1
𝐶𝐵 = 𝐼𝑞
⇒ (𝐼𝑞 − 𝑋)𝐶 is left inverse of 𝐵, i.e. 𝐵 is left invertible.
𝑋 = 𝐶(𝐴 − 𝐵) ⇒ ‖𝑋‖ ≤ ‖𝐶‖ ∙ ‖𝐴 − 𝐵‖
If ‖𝐴 − 𝐵‖ <
1
‖𝐶‖
⇒ ‖𝑋‖ < 1
We can then prove symmetrically that if it is right invertible then any close enough 𝐵 is also
right invertible.
𝐴 ∈ ℂ𝑝×𝑞
𝑟𝑎𝑛𝑘𝐴 = 𝑝 or 𝑟𝑎𝑛𝑘𝐴 = 𝑞
𝐵 close to 𝐴 it will have the same rank as 𝐴.
Question: 𝑟𝑎𝑛𝑘𝐴 = 𝑘, 1 ≤ 𝑘 < min{𝑝, 𝑞}
Can we prove that if 𝐵 is close enough to 𝐴, then rank 𝐵 = 𝑘.
2
Observation: ∈ ℂ𝑝×𝑞 , 𝐴 = [𝑎𝑖𝑗 ] , then |𝑎𝑖𝑗 | ≤ ‖𝐴‖2,2 ≤ √∑𝑖,𝑗|𝑎𝑖𝑗 | ⇒
|𝑎𝑖𝑗 − 𝑏𝑖𝑗 | ≤ ‖𝐴 − 𝐵‖2,2
Proof: ‖𝐴𝑥‖22
𝑝
=
2
∑𝑝𝑖=1|∑𝑞𝑗=1 𝑎𝑖𝑗 𝑥𝑗 |
1
2
𝑞
1
=
2
∑𝑝𝑖=1 {(∑𝑞𝑗=1|𝑎𝑖𝑗 | )2
1
∙
2 2
𝑞
(∑𝑗=1|𝑥𝑗 | ) }
2
≤
2
2
‖𝑥‖22 ∑ (∑|𝑎𝑖𝑗 | )
𝑖=1
𝑗=1
{
}
But ‖𝐴‖2,2 = max{‖𝐴𝑥‖2|‖𝑥‖2 = 1}
So we get what we wanted to prove…
‖𝐴𝑒𝑙 ‖2 ≤ ‖𝐴‖2,2 ∙ ‖𝑒
⏟𝑙 ‖2,2
=1
𝑞
√∑|𝑎𝑖𝑙 |2 ≤ ‖𝐴‖2,2 ∙ 1 ≤ 0
𝑖=1
Useful fact: Let 𝜑(𝑥) be a norm on a NLS 𝒳 over 𝔽. Then |𝜑(𝑋) − 𝜑(𝑦)| ≤ 𝜑(𝑥 − 𝑦)
Or in other notation: |‖𝑥‖ − ‖𝑦‖| ≤ ‖𝑥 − 𝑦‖
𝜑(𝑥) = 𝜑(𝑥 − 𝑦 + 𝑦) ≤ 𝜑(𝑥 − 𝑦) + 𝜑(𝑦) ⇒
𝜑(𝑥) − 𝜑(𝑦) ≤ 𝜑(𝑥 − 𝑦)
But symmetrical to 𝑦 − 𝑥 so we can also claim:
𝜑(𝑦) − 𝜑(𝑥) ≤ 𝜑(𝑦 − 𝑥)
At least one of them is non-negative and will be selected as the absolute value.
𝐴 ∈ ℂ𝑝×𝑞
‖𝐴‖2,2 = ⋯
There are other ways of making it a normed linear space.
1
𝑠 𝑠
For instance, can also define: ‖𝐴‖𝑠 = {∑𝑖,𝑗|𝑎𝑖𝑗 | }
This is a valid definition of norm!
But then we lose the fact that ‖𝐴𝐵‖𝑠 ≤ ‖𝐴‖𝑠 ∙ ‖𝐵‖𝑠 !
Inner Product Spaces
A vector space 𝒰 over 𝔽 is said to be an inner product space if there exists a number which we’ll
designate by: ⟨𝑢, 𝑣⟩𝑢 for every pair of vectors 𝑢, 𝑣 ∈ 𝒰 with the following properties:
1) ⟨𝑢, 𝑢⟩𝑢 ≥ 0
2) ⟨𝑢, 𝑢⟩𝑢 = 0 ⇒ 𝑢 = 0
3) ⟨𝑢, 𝑣⟩ is linear in the first entry – ⟨𝑢 + 𝑤, 𝑣⟩𝑢 = ⟨𝑢, 𝑣⟩ + ⟨𝑤, 𝑣⟩ and ⟨𝛼𝑢, 𝑣⟩ = 𝛼⟨𝑢, 𝑣⟩
⟨𝑢, 𝑣⟩𝑢
4) ⟨𝑣, 𝑢⟩ = ̅̅̅̅̅̅̅̅̅
Observations:
⟨0, 𝑢⟩ = 0 for any 𝑢 ∈ 𝒰
Proof: ⟨0, 𝑢⟩ = ⟨0 + 0, 𝑢⟩ = ⟨0, 𝑢⟩ + ⟨0, 𝑢⟩ ⇒ 0 = ⟨0, 𝑢⟩
⟨𝑢, 𝑣⟩ - “conjugate” linear in second entry
⟨𝑢, 𝑣 + 𝑤⟩ = ̅̅̅̅̅̅̅̅̅̅̅̅̅
⟨𝑣 + 𝑤, 𝑢⟩ = ̅̅̅̅̅̅̅
⟨𝑣, 𝑢⟩ + ̅̅̅̅̅̅̅̅
⟨𝑤, 𝑢⟩ = ⟨𝑢, 𝑣⟩ + ⟨𝑢, 𝑤⟩
̅̅̅̅̅̅̅
⟨𝑢, 𝛼𝑣⟩ = ̅̅̅̅̅̅̅̅̅
⟨𝛼𝑣, 𝑢⟩ = 𝛼̅⟨𝑣,
𝑢⟩ = 𝛼̅⟨𝑢, 𝑣⟩
Examples:
1) ℂ𝑛 ⟨𝑢, 𝑣⟩ = ∑𝑛𝑗=1 𝑢𝑗 𝑣̅𝑗
2) ℂ𝑛 𝜌1 , … , 𝜌𝑛 all > 0 ⟨𝑢, 𝑣⟩ = ∑𝑛𝑗=1 𝜌𝑗 𝑢𝑗 𝑣̅𝑗
3) 𝜌([𝑎, 𝑏]) = continuous function on [𝑎, 𝑏]
𝑏
̅̅̅̅𝑑𝑡
⟨𝑓, 𝑔⟩ = ∫ 𝑓(𝑡)𝑔(𝑡)
𝑎
Cauchy’s-Shwarz in inner product spaces
Let 𝒰 be an inner product space over ℂ. Then:
1
1
|⟨𝑢, 𝑣⟩| ≤ {⟨𝑢, 𝑢⟩}2 {⟨𝑣, 𝑣⟩2 } with equality if and only if dim 𝑠𝑝𝑎𝑛{𝑢, 𝑣} ≤ 1
Proof: Let 𝜆 ∈ ℂ and consider:
⟨𝑢 + 𝜆𝑣, 𝑢 + 𝜆𝑣⟩ = ⟨𝑢, 𝑢 + 𝜆𝑣⟩ + ⟨𝜆𝑣, 𝑢 + 𝜆𝑣⟩ = ⟨𝑢, 𝑢⟩ + 𝜆⟨𝑣, 𝑢⟩ + 𝜆̅⟨𝑣, 𝑢⟩ + |𝜆|2 ⟨𝑣, 𝑣⟩ +
The inequality is clearly valid if 𝑣 = 0
Can restrict attention to ease ⟨𝑣, 𝑣⟩ > 0 and ⟨𝑢, 𝑣⟩ ≠ 0.
Let’s write ⟨𝑢, 𝑣⟩ = 𝑟 ∙ 𝑒 𝒾𝜃 , 𝑟 ≥ 0, 0 ≤ 𝜃 < 2𝜋
Choose 𝜆 = 𝑥𝑒 𝒾𝜃 . 𝑥 ∈ ℝ
⟨𝑢 + 𝑥𝑒 𝒾𝜃 , 𝑢 + 𝑒 𝒾𝜃 ⟩ = ⟨𝑢, 𝑢⟩ + 𝑥𝑒 𝒾𝜃 𝑟𝑥𝑒 −𝒾𝜃 + 𝑥𝑒 −𝒾𝜃 𝑟𝑥𝑒 𝒾𝜃 + 𝑥 2 ⟨𝑣, 𝑣⟩ =
⟨𝑢, 𝑢⟩ + 2𝑥𝑟 + 𝑥 2 ⟨𝑣, 𝑣⟩
𝑓(𝑥) = ⟨𝑣, 𝑣⟩𝑥 2 + 2𝑟𝑥 + ⟨𝑢, 𝑢⟩
𝑓 ′ (𝑥) = ⟨𝑣, 𝑣⟩2𝑥 + 2𝑟
𝑟
𝑓 ′ (𝑥) = 0 ⇒ 𝑥 = −
= 𝑥0
⟨𝑣, 𝑣⟩
⟨𝑢 + 𝑥0 𝑒 𝒾𝜃 𝑣, − − −⟩ = ⟨𝑢, 𝑢⟩ −
2𝑟 2
𝑟2
𝑟2
⟨𝑣,
⟨𝑢,
+
𝑣⟩
=
𝑢⟩
−
⟨𝑣, 𝑣⟩ ⟨𝑣, 𝑣⟩2
⟨𝑣, 𝑣⟩
𝑟2
= ⟨𝑢, 𝑢⟩ − ⟨𝑢 + 𝑥0 𝑒 𝒾𝜃 𝑣, 𝑢 + 𝑥0 𝑒 𝒾𝜃 𝑢⟩ ≤ ⟨𝑢, 𝑢⟩
⟨𝑣, 𝑣⟩
𝑟 2 ≤ ⟨𝑢, 𝑢⟩⟨𝑣, 𝑣⟩
𝑟 2 = ⟨𝑣, 𝑣⟩2
And there is an equality if ⟨𝑢 + 𝑥0 𝑒 𝒾𝜃 𝑣, 𝑢 + 𝑥0 𝑒 𝒾𝜃 𝑢⟩ = 0
But this happens iff 𝑢 + 𝑥0 𝑒 𝒾𝜃 𝑣 = 0 ⇒ 𝑢 and 𝑣 are linearly dependent!
Claim: If 𝒰 is an inner product space, then its also a normed-linear-space with respect to 𝜑(𝑢) =
1
⟨𝑢, 𝑢⟩2
Check:
1)
2)
3)
4)
𝜑(𝑢) ≥ 0
𝜑(𝑢) = 0 ⇒ 𝑢 = 0
𝜑(𝛼𝑢) = |𝛼|𝜑(𝑢), 𝛼 ∈ ℂ
𝜑(𝑢 + 𝑣) ≤ 𝜑(𝑢)𝜑(𝑣) ??
𝜑(𝑢 + 𝑣)2 = ⟨𝑢 + 𝑣, 𝑢 + 𝑣⟩ = ⟨𝑢, 𝑢⟩ + ⟨𝑢, 𝑣⟩ + ⟨𝑣, 𝑢⟩ + ⟨𝑣, 𝑣⟩ ≤
1
1
⟨𝑢, 𝑢⟩ + 2⟨𝑢, 𝑢⟩2 ⟨𝑣, 𝑣⟩2 + ⟨𝑣, 𝑣⟩ = 𝜑(𝑢)2 + 2𝜑(𝑢)2 + 𝜑(𝑣)2 = {𝜑(𝑢) + 𝜑(𝑣)}2
𝜑(𝑢 + 𝑣) ≤ 𝜑(𝑢) + 𝜑(𝑣)
𝜑(𝑢 + 𝑣)2 = ⟨𝑢 + 𝑣, 𝑢 + 𝑣⟩ = ⟨𝑢, 𝑢⟩ + ⟨𝑢, 𝑣⟩ + ⟨𝑣, 𝑢⟩ + ⟨𝑣, 𝑣⟩
𝜑(𝑢 − 𝑣)2 = ⟨𝑢 − 𝑣, 𝑢 − 𝑣⟩ = ⟨𝑢, 𝑢⟩ − ⟨𝑢, 𝑣⟩ − ⟨𝑣, 𝑢⟩ − ⟨𝑣, 0⟩
𝜑(𝑢 + 𝑣)2 + 𝜑(𝑢 − 𝑣)2 = 2𝜑(𝑢)2 + 2𝜑(𝑣)2
So:
‖𝑢 + 𝑣‖2 + ‖𝑢 − 𝑣‖2 = 2‖𝑢‖2 + 2‖𝑣‖2
Parallelogram law!
If the norm in a NLS satisfies the parallelogram law, then we can define an inner product on that
space by SeHing:
4
1
2
⟨𝑢, 𝑣⟩ = ∑ 𝒾 𝑗 ‖𝑢 + 𝒾 𝑗 𝑣‖
4
𝑗=1
‖𝑥‖𝑠 satisfies parallelogram law ⇔ 𝑠 = 2
--- end of lesson
Inner Product Sace over 𝔽 is a vector spac over 𝔽 plus a rule that associates a number ⟨𝑢, 𝑣⟩ ∈ 𝔽
for each pair of vectors 𝑢, 𝑣 ∈ 𝒰
1) ⟨𝑢, 𝑢⟩ ≥ 0
2) If ⟨𝑢, 𝑢⟩ = 0 ⇒ 𝑢 = 0
3) Linear in first entry
4) ⟨𝑢, 𝑣⟩ = ⟨𝑣, 𝑢⟩
1
1
Checked that ⟨𝑢, 𝑢⟩2 defines a norm on 𝒰. i.e. 𝒰 is a normed-linear space with respect to ⟨𝑢, 𝑢⟩2
i.e. we shoed that:
1
1) ⟨𝑢, 𝑢⟩2 ≥ 0
1
2) ⟨𝑢, 𝑢⟩2 = 0 ⇒ 𝑢 = 0
1
1
3) ⟨𝛼𝑢, 𝛼𝑣⟩2 = |𝛼|⟨𝑢, 𝑢⟩2
1
1
1
4) ⟨𝑢 + 𝑣, 𝑢 + 𝑣⟩2 ≤ ⟨𝑢, 𝑢⟩2 + ⟨𝑣, 𝑣⟩2
Question: Is every NLS an inner product space?
No!
1
The norm that we introduced above ‖𝑢‖ = ⟨𝑢, 𝑢⟩2 satisfies the parallelogram law, i.e.:
‖𝑢 + 𝑣‖2 + ‖𝑢 − 𝑣‖2 = 2‖𝑢‖2 + 2‖𝑣‖2
FACT: If 𝒰 is a normed-linear space such that its norm satisfies this extra condition, then can
define an inner product on this space.
Orthogonality
Let 𝒰 be an inner product space over 𝔽, then a pair of vectors 𝑢 ∈ 𝒰 is said to be orthogonal to
a vector 𝑣 ∈ 𝒰 if ⟨𝑢, 𝑣⟩ = 0
TODO: Draw cosines
‖𝑎‖2 + ‖𝑏‖2 + ‖𝑏 − 𝑎‖2 ‖𝑎‖2 + ‖𝑏‖2 + ∑3𝑗=1 𝑏𝑗2 + 2𝑏𝑗 𝑎𝑗 + 𝑎2
cos 𝜃 =
=
2‖𝑎‖ ∙ ‖𝑏‖
2‖𝑎‖ ∙ ‖𝑏‖
Since:
3
‖𝑎‖2
= ∑ 𝑎𝑖2
𝑖=1
3
2
2
‖𝑏 − 𝑎‖ = ⟨𝑏 − 𝑎, 𝑏 − 𝑎⟩ = ∑(𝑏𝑗 − 𝑎𝑗 )
𝑗=1
So:
cos 𝜃 =
⟨𝑎, 𝑏⟩
‖𝑎‖ ∙ ‖𝑏‖
A family {𝑢1 , … , 𝑢𝑘 } of non-zero vectors is said to be an orthogonal family if ⟨𝑢𝑖 , 𝑢𝑗 ⟩ = 0 for 𝑖 ≠
𝑗.
A family {𝑢1 , … , 𝑢𝑘 } of nonzero vectors is said to be orthonormal family if
⟨𝑢𝑖 , 𝑢𝑗 ⟩ = 0 , 𝑖 ≠ 𝑗
{
⟨𝑢𝑖 , 𝑢𝑖 ⟩ = 1, 𝑖 = 1 … 𝑘
If 𝒱 is a subspace of 𝒰 then the orthogonal complement of 𝒱 which we write as 𝒱 ⊥ of 𝒱 (in 𝒰)
is equal to {𝑢 ∈ 𝒰|⟨𝑢, 𝑣⟩ = 0 ∀𝑣 ∈ 𝒱}
Claim: 𝒱 ⊥ is a vector space.
To check: Let 𝑢, 𝑤 ∈ 𝒱 ⊥
⟨𝑢 + 𝑤, 𝑣⟩ = ⟨𝑢, 𝑣⟩ + ⟨𝑤, 𝑣⟩ = 0 + 0 = 0
⟨𝛼𝑢, 𝑣⟩ = 𝛼⟨𝑢, 𝑣⟩ = 𝛼0 = 0
Claim: 𝒱 ∩ 𝑉 ⊥ = {0}
Let 𝑥 ∈ 𝒱 ∩ 𝒱 ⊥ , but 𝑥 ∈ 𝒱 ⊥ means it’s orthogonal to every vector in 𝒱 ⇒ ⟨𝑥, 𝑥⟩ = 0 ⇒ 𝑥 = 0.
Observation: If {𝑢1 , … , 𝑢𝑘 } is an orthogonal family, then it’s a linearly independent set of
vectors.
Proof: Suppose you could find coefficients 𝑐1 , … , 𝑐𝑘 such that:
𝑘
𝑘
∑ 𝑐𝑗 𝑢𝑗 = 0 ⇒ ⟨∑ 𝑐𝑗 𝑢𝑗 , 𝑢𝑖 ⟩ = ⟨0, 𝑢𝑖 ⟩ = 0
𝑗=1
1
It’s true for all 𝑖. So suppose 𝑖 = 1:
𝑘
∑ 𝑐𝑗 ⟨𝑢𝑗 , 𝑢1 ⟩
𝐼𝑡 ′ 𝑠 𝑎𝑛 𝑜𝑟𝑡ℎ𝑜𝑔𝑜𝑛𝑎𝑙
𝑓𝑎𝑚𝑖𝑙𝑦!
=
𝑢1 ≠0
𝑐1 ⟨𝑢1 , 𝑢1 ⟩ ⇒
𝑐1 = 0
1
We can do the same for 𝑢2 and so on…
Thus proving ∀𝑖. 𝑐𝑖 = 0 which means they are linearly independent.
Let 𝒰 be an inner product space over 𝔽 and let {𝑢1 , … , 𝑢𝑘 } be a set of 𝑘 vectors in 𝒰 then:
𝒢 = [𝑔𝑖𝑗 ], with 𝑔𝑖𝑗 = ⟨𝑢𝑖 , 𝑢𝑗 ⟩ called Gram matrix.
Lemma: {𝑢1 , … , 𝑢𝑘 } will be linearly independent ⇔ 𝒢 is invertible.
Suppose first 𝒢 is invertible, claim that if ∑𝑘𝑗=1 𝑐𝑗 𝑢𝑗 = 0 then 𝑐1 = ⋯ = 𝑐𝑘 = 0.
𝑐1
Consider 𝑐 = [ ⋮ ] (𝒢𝑐)𝑖 = ∑𝑘𝑗=1 𝑔𝑖𝑗 𝑐𝑗 = ∑𝑘𝑗=1 𝑐𝑗 ⟨𝑢𝑗 , 𝑢𝑖 ⟩ = ⟨∑𝑘𝑗=1 𝑐𝑗 𝑢𝑗 , 𝑢𝑖 ⟩ = ⟨0, 𝑢𝑖 ⟩ = 0
𝑐𝑘
But this is true for all 𝑖!
So 𝒢𝑐 = 0
𝒢 is invertible ⇒ 𝑐 = 0
Assume 𝒢 is invertible ⇒ 𝑐1 , … , 𝑐𝑘 are such that ∑ 𝑐𝑗 𝑢𝑗 = 0 ⇒ 𝑐1 = ⋯ = 𝑐𝑘 = 0 and linearly
independent.
Suppose now 𝑢1 , … , 𝑢𝑘 are linearly independent.
Claim: 𝒢 is invertible.
It’s enough to show that 𝒩𝒢 = {0}
Let 𝑐 ∈ 𝒩𝒢 ⇒ 𝒢𝑐 = 0 ⇒ ∑𝑘𝑗=1 𝑔𝑖𝑗 𝑐𝑗 = 0 for 𝑖 = 1, … , 𝑘 ⇒ ∑⟨𝑢𝑗 , 𝑢𝑖 ⟩𝑐𝑗 = 0 ⇒
𝑘
𝑘
𝑘
𝑘
𝑘
⟨∑ 𝑐𝑗 𝑢𝑗 , 𝑢𝑖 ⟩ = 0 𝑓𝑜𝑟 𝑖 = 1, … , 𝑘 ⇒ ∑ 𝑐̅𝑖 (⟨∑ 𝑐𝑗 𝑢𝑗 , 𝑢𝑖 ⟩) = 0 ⇒ ⟨∑ 𝑐𝑗 𝑢𝑗 , ∑ 𝑐𝑖 𝑢𝑖 ⟩
𝑗=1
𝑖=1
𝑗=1
𝑗=1
But these are the same vector! Denote it as 𝑤 so: ⟨𝑤, 𝑤⟩ = 0 ⇒ 𝑤 = 0 ⇒
{𝑢1 , … , 𝑢𝑘 } linearly independent ⇒ 𝑐1 = ⋯ = 𝑐𝑘 = 0
𝑖=1
𝑘
∑𝑗=1 𝑐𝑗 𝑢𝑗
=0
Thus 𝒢𝑐 = 0 ⇒ 𝑐 = 0 ⇒ 𝒩𝒢 = {0}. 𝒢 is invertible.
Adjoints
Let 𝑇 be a linear transformation from a finite dimensional inner product space 𝒰 into a finite
dimensional inner product space 𝒱. Then there exists exactly one linear transformation 𝑆 from
𝒱 to 𝒰 such that for 𝑢 ∈ 𝒰, 𝑣 ∈ 𝒱 - ⟨𝑇𝑢, 𝑣⟩𝒱 = ⟨𝑢, 𝑆𝑣⟩𝒰
𝑆 is called the adjoint of 𝑇. And the usual notation is 𝑇 ∗.
Proof: At the end of the lesson if we have time.
It’s easy to show that there is most one linear transformation 𝑆 from 𝒱 to 𝒰 such that
⟨𝑇, 𝑢, 𝑣⟩𝒱 = ⟨𝑢, 𝑆𝑣⟩𝒰 all 𝑢 ∈ 𝒰, 𝑣 ∈ 𝒱.
Suppose could find 𝑆1 , 𝑆2 that met this condition.
⟨𝑇𝑢, 𝑢⟩𝒱 = ⟨𝑢, 𝑆1 𝑣⟩𝒰 = ⟨𝑢, 𝑆2 𝑣⟩𝒰
⟨𝑢, 𝑆1 𝑣 − 𝑆2 𝑣⟩ = 0 for all 𝑢 ∈ 𝒰, 𝑣 ∈ 𝒱.
This is true for all 𝑢! We can choose 𝑢 = 𝑆1 𝑣 − 𝑆2 𝑣.
But then: ⟨𝑆1 𝑣 − 𝑆2 𝑣, 𝑆1 𝑣 − 𝑆2 𝑣⟩ = 0
So the vector 𝑆1 𝑣 − 𝑆2 𝑣 = 0 ⇒ 𝑆1 𝑣 = 𝑆2 𝑣 ⇒ 𝑆1 = 𝑆2 .
Let 𝑇 be a linear transformation from a finite dimensional inner product space 𝒰 into 𝒰.
Then 𝑇 is said to be:
Normal if 𝑇 ∗ 𝑇 = 𝑇𝑇 ∗
Selfadjoint if 𝑇 ∗ = 𝑇
Unitary if 𝑇 ∗ 𝑇 = 𝑇𝑇 ∗ = 𝐼
𝑇 = 𝑇 ∗ ⇒ 𝑇 is normal
𝑇𝑇 ∗ = 𝑇𝑇 ∗ ⇒ is normal
Theorem: Let 𝑇 be a linear transformation from an 𝑛 dimensional inner product space 𝒰 into
itself and suppose 𝑇 is normal. Then:
(1) There exists an orthonormal basis {𝑢1 , … , 𝑢𝑛 } of eigenvectors of 𝑇
(2) If 𝑇 = 𝑇 ∗ ⇒ all the eigenvalues are real.
(3) If 𝑇 is unitary⇒ all eigenvalues have magnitude 1.
1) 𝑇𝜆 = 𝜆𝐼 − 𝑇, (𝑇𝜆 )∗ = 𝜆̅𝐼 − 𝑇
⟨𝑇𝜆 𝑢, 𝑣⟩𝒰 = ⟨𝑢, (𝑇𝜆 )∗ 𝑣⟩
But the left side equals:
⟨(𝜆𝐼 − 𝑇)𝑢, 𝑣⟩ = 𝜆⟨𝑢, 𝑣⟩ − ⟨𝑇𝑢, 𝑣⟩ = ⟨𝑢, 𝜆̅𝑣⟩ − ⟨𝑢, 𝑇 ∗ 𝑣⟩ = ⟨𝑢, (𝜆̅𝐼 − 𝑇 ∗ )𝑣⟩
But there is only one adjoint matrix! So (𝑇𝜆 )∗ = (𝜆̅𝐼 − 𝑇 ∗ )
2) T normal ⇒ 𝑇𝜆 is normal.
𝑇𝜆 (𝑇𝜆 )∗ = (𝜆𝐼 − 𝑇)(𝜆̅𝐼 − 𝑇) = 𝜆𝐼(𝜆̅𝐼 − 𝑇 ∗ ) − 𝑇(𝜆̅𝐼 − 𝑇 ∗ ) = (𝜆̅𝐼 − 𝑇 ∗ )𝜆𝐼 − (𝜆̅𝐼 − 𝑇 ∗ )𝑇 =
(𝜆̅𝐼 − 𝑇)(𝜆𝐼 − 𝑇) = (𝑇𝜆 )∗ 𝑇𝜆
3) 𝑇𝑢 = 𝜆𝑢 ⇒ 𝑇 ∗ 𝑢 = 𝜆̅𝑢
Let 𝑢 ∈ 𝒩𝑇𝜆 ⇒ 𝑇𝜆 𝑢 = 0 ⇒ (𝑇𝜆 )∗ 𝑇𝜆 𝑢 = 0 ⇒ 𝑇𝜆 (𝑇𝜆 )∗ = 0 ⇒ ⟨𝑇𝜆 (𝑇𝜆 )∗ 𝑢, 𝑢⟩ = 0 ⇒
⟨(𝑇𝜆 )∗ 𝑢, (𝑇𝜆 )∗ 𝑢⟩ = 0 ⇒ (𝑇𝜆 )∗ 𝑢 = 0 ⇒ (𝜆̅𝐼 − 𝑇 ∗ )𝑢 = 0
4) Establish orthonormal basis of eigenvectors of 𝑇.
𝒰 is invariant under 𝑇 ⇒ 𝑇 has an eigenvector 𝑢1 ≠ 0 i.e. there is 𝑇𝑢1 = 𝜆1 𝑢1 , 𝑢1 ≠ 0
𝑢1
𝑢1
𝑇(
) = 𝜆1 (
)
‖𝑢1 ‖
‖𝑢1 ‖
So can assume 𝑢1 has norm 1.
Suppose we could find {𝑢1 , … , 𝑢𝑘 } such that 𝑇𝑢𝑗 = 𝜆𝑢𝑗 , 𝑗 = 1, … , 𝑘
⟨𝑢𝑖 , 𝑢𝑗 ⟩ = 0 if 𝑖 ≠ 𝑗 and 1 otherwise.
ℒ𝑘 = 𝑠𝑝𝑎𝑛{𝑢1 , … , 𝑢𝑘 }
ℒ𝑘⊥ = vectors in 𝒰 that are orthogonal to ℒ𝑘 .
𝑣 ∈ ℒ𝑘⊥ ⇔ ⟨𝑣, 𝑢𝑗 ⟩ = 0,
𝑗 = 1, … , 𝑘
ℒ𝑘⊥ is invariant under 𝑇, i.e. 𝑣 ∈ ℒ𝑘⊥ ⇒ 𝑇𝑣 ∈ ℒ𝑘⊥
⟨𝑇𝑣, 𝑢𝑗 ⟩ = ⟨𝑣, 𝑇 ∗ 𝑢𝑗 ⟩ = ⟨𝑣, 𝜆̅𝑗 𝑢𝑗 ⟩ = 𝜆𝑗 ⟨𝑣, 𝑢𝑗 ⟩ = 0,
𝑗 = 1, … , 𝑘
Because:
𝑇𝑢𝑗 = 𝜆𝑗 𝑢𝑗
𝑇 ∗ 𝑢𝑗 = 𝜆̅𝑗 𝑢𝑗
So can find an eigenvector of 𝑇 in ℒ𝑘⊥ . Call it 𝑢𝑘+1 = 𝑠𝑝𝑎𝑛{𝑢1 , … , 𝑢𝑘 }.
So we can increase our ℒ𝑘⊥ until it is big enough…
6) 𝑇 unitary ⇒ |𝜆𝑗 | = 1
If 𝑇 unitary:
2
𝑇𝑢𝑗 = 𝜆𝑗 𝑢𝑗 ⇒ 𝑇 ∗ 𝑇𝑢𝑗 = 𝜆𝑗 𝑇 ∗ 𝑢𝑗 = 𝜆𝑗 𝜆̅𝑗 𝑢𝑗 ⇒ 𝑢𝑗 = |𝜆𝑗 | 𝑢𝑗 .
--- end of lesson
Recall: IF 𝑇 is a linear transformation from a finite dimension inner product space 𝒰 over 𝔽 into
a finite dimension inner product space 𝒱, then there exists exactly one linear transformation 𝑆
from 𝒱 to 𝒰 such that ⟨𝑇𝑢, 𝑣⟩𝒱 = ⟨𝑢, 𝑆𝑣⟩𝒰 for every 𝑢 ∈ 𝒰, 𝑣 ∈ 𝒱
That 𝑆 is called the adjoint of 𝑇 and is denoted by 𝑇 ∗ .
So in the future we’ll write ⟨𝑇𝑢, 𝑣⟩𝒱 = ⟨𝑢, 𝑇 ∗ 𝑣⟩𝒰 .
Existence: Suppose {𝑢1 , … , 𝑢𝑘 } is a basis for 𝒰 and {𝑣1 , … , 𝑣𝑙 } is a basis of 𝒱.
Can reduce the existence to showing that exists 𝑆 such that
⟨𝑇𝑢𝑖 , 𝑣𝑗 ⟩𝒱 = ⟨𝑢𝑖 , 𝑆𝑣𝑗 ⟩𝒰 ,
𝑖 = 1, … , 𝑘, 𝑗 = 1, … , 𝑙
𝑙
𝑙
𝑙
𝑇𝑢𝑖 ∈ 𝒱 ⇒ 𝑇𝑢𝑖 = ∑ 𝑎𝑡𝑖 𝑣𝑡 𝑓𝑜𝑟 𝑖 = 1, … , 𝑘 ⇒ ⟨𝑇𝑢𝑖 , 𝑣𝑗 ⟩ = ⟨∑ 𝑎𝑡𝑖 𝑣𝑡 , 𝑣𝑗 ⟩ = ∑ 𝑎𝑡𝑖 ⟨𝑣𝑡 , 𝑣𝑗 ⟩𝒱 =
𝑡=1
𝑡=1
𝑡=1
𝑙
∑ 𝑎𝑡𝑖 (𝐺𝒱 )𝑗𝑡 = (𝐺𝒱 𝐴)𝑗𝑖
𝑡=1
𝑘
𝑆𝑣𝑗 = ∑ 𝑏𝑖𝑗 𝑢𝑟
𝑖=1
Want to choose 𝐵 = [𝑏𝑖𝑗 ] so that equiality (1) holds.
𝑘
𝑘
⟨𝑢𝑖 , 𝑆𝑣𝑗 ⟩𝒰 = ⟨𝑢𝑖 , ∑ 𝑏𝑖𝑗 𝑢𝑟 ⟩𝒰 = ∑ ̅̅̅
𝑏𝑖𝑗̅⟨𝑢𝑖 , 𝑢𝑟 ⟩𝒰 = (𝐺𝒰 𝐴)
𝑖=1
𝑖=1
𝑏̅𝑟𝑗 = (𝐵𝐻 )𝑗𝑟
𝐵𝐻 𝒢𝒰 = 𝒢𝒱 𝐴
𝑇: 𝒰 → 𝒰
𝑇 is said to be normal if 𝑇 ∗ 𝑇 = 𝑇𝑇 ∗
𝑇 is said to be self-adjoint if 𝑇 = 𝑇 ∗
𝑇 is said to be unitary if 𝑇 ∗ 𝑇 = 𝑇𝑇 ∗ = 𝐼
Let 𝐴 ∈ ℂ𝑝×𝑞 , 𝒰 = ℂ𝑝 , ⟨, ⟩𝒰 , 𝒱 = ℂ𝑝 , ⟨, ⟩𝒱
Then there exists exactly one matrix 𝐴 ∈ ℂ𝑞×𝑝 such that ⟨𝐴𝑥, 𝑦⟩𝒱 = ⟨𝑥, 𝐴∗ 𝑦⟩𝒰
Example: ⟨𝑢1 , 𝑢2 ⟩𝒰 = ⟨Δ1 𝑢1 , Δ1 𝑢2 ⟩𝑠𝑡 = (Δ1 𝑢2 )𝐻 Δ1 𝑢1
𝑞×𝑞
⟨𝑎, 𝑏⟩𝑠𝑡 = ∑𝑘𝑗=1 𝛼𝑗 𝑏̅𝑗 = 𝑏 𝐻 𝑎 = 𝑢2𝐻 Δ𝐻
invertible.
1 Δ1 𝑢1 , Δ1 ∈ ℂ
ℂ𝑝 , 𝐵 ∈ 𝐶 𝑞×𝑞 invertible.
𝑑𝑒𝑓
1)
2)
3)
4)
⟨𝑥, 𝑦⟩𝐵 = ⟨𝐵𝑥, 𝐵𝑦⟩𝑠𝑡 = (𝐵𝑦)𝐻 𝐵𝑥 = 𝑦 ℎ 𝐵𝐻 𝐵𝑥
⟨𝑥, 𝑥⟩𝐵 ≥ 0? ⟨𝑥, 𝑥⟩𝐵 = 𝑥 𝐻 𝐵𝐻 𝐵𝑥 = (𝐵𝑥)𝐻 𝐵𝑥 = ⟨𝐵𝑥, 𝐵𝑥⟩𝑠𝑡 = ∑|𝐵𝑥|2
⟨𝑥, 𝑥⟩𝐵 = 0 ⇒ 𝑥 = 0 …
…
…
Δ2 𝑝 × 𝑝 invertible.
𝑑𝑒𝑓
⟨𝑣1 , 𝑣2 ⟩𝒱 = ⟨Δ2 𝑣1 , Δ2 𝑣2 ⟩𝑠𝑡 = (Δ2 𝑣2 )𝐻 Δ2 𝑣1 = 𝑉2𝐻 (Δ𝐻
2 Δ2 )𝑣1
𝐻
𝐻
⟨𝐴𝑥, 𝑦⟩𝒱 = 𝑦 Δ2 Δ2 𝐴𝑥
𝐻
∗ 𝐻 𝐻
⟨𝑥, 𝐴∗ 𝑦⟩𝒰 = (𝐴∗ 𝑦)𝐻 Δ𝐻
1 Δ1 𝑥 = 𝑦 (𝐴 ) Δ1 Δ1 𝑥
Since it holds for all 𝑥 ∈ ℂ𝑞 and 𝑦 ∈ ℂ𝑝 :
𝐻
𝐻
∗ 𝐻 𝐻
∗ 𝐻
−1
Δ𝐻
2 Δ2 𝐴 = (𝐴 ) Δ1 Δ1 ⇒ (𝐴 ) = Δ2 Δ2 𝐴(Δ1 Δ1 )
−1 𝐻 𝐻
𝐴∗ = (Δ𝐻
1 Δ1 ) 𝐴 Δ2 Δ2
If Δ1 = 𝐼𝑞 , Δ2 = 𝐼𝑝
i.e. if using standard inner product, then 𝐴∗ = 𝐴𝐻
Theorem: Let 𝐴 ∈ ℂ𝑝×𝑝 , ℂ𝑝 have inner product ⟨, ⟩𝒰
Then:
(1) 𝐴∗ 𝐴 = 𝐴𝐴∗ ⇒ 𝐴 is diagonalizable and there exists an orthonormal set of eigen vectors i.e.
𝐴𝑈 = 𝑈𝐷, 𝐷 diagonal and columns of 𝑈 are orthonormal. 𝐷 ∈ ℝ𝑝×𝑝
(2) 𝐴 = 𝐴∗
(3) 𝐴∗ 𝐴 = 𝐴𝐴∗ = 𝐼𝑝 , |𝑑𝑖𝑗 | = 1
(1) If 𝐴∗ 𝐴 = 𝐴𝐴∗ , then 𝐴𝑢 = 𝜆𝑢 ⇒ 𝐴∗ 𝑢 = 𝜆̅𝑢
𝐴𝐴∗ =𝐴∗ 𝐴
(𝐴 − 𝜆𝐼)(𝐴∗ − 𝜆̅𝐼)𝑢 = 0 ⇒
𝐴𝑢 = 𝜆𝑢 ⇒ (𝐴 − 𝜆𝐼)𝑢 = 0 ⇒ (𝐴∗ − 𝜆̅𝐼)(𝐴 − 𝜆𝐼)𝑢 = 0 ⇒
⟨(𝐴 − 𝜆𝐼)(𝐴∗ − 𝜆̅𝐼)𝑢, 𝑢⟩ = ⟨(𝐴∗ − 𝜆̅𝐼)𝑢, (𝐴∗ − 𝜆̅𝐼)𝑢⟩ = 0 ⇒ (𝐴∗ − 𝜆̅𝐼)𝑢 = 0 ⇒ 𝑢 is an
eigenvector of 𝐴∗ with Eigenvalue 𝜆̅𝐼.
Let 𝜆 be an eigenvalue of 𝐴.
It’s enough to show 𝒩(𝐴−𝜆𝐼 )2 = 𝒩(𝐴−𝜆𝐼 )
𝑥 ∈ 𝒩(𝐴−𝜆𝐼)2 ⇒ (𝐴 − 𝜆𝐼)2 𝑥 = 0 ⇒ (𝐴 − 𝜆𝐼)((𝐴 − 𝜆𝐼)𝑥) = 0
Denote 𝑦 = (𝐴 − 𝜆𝐼)𝑥. Then 𝐴𝑦 = 𝜆𝑦
By (1), 𝐴∗ 𝑦 = 𝜆̅𝑦 ⇒ (𝐴∗ − 𝜆̅𝐼)(𝐴 − 𝜆𝐼)𝑥 = 0 ⇒ ⟨(𝐴∗ − 𝜆̅𝐼)(𝐴 − 𝜆𝐼)𝑥, 𝑥⟩ = 0 ⇒
∗
⟨(𝐴 − 𝜆𝐼)𝑥, (𝐴∗ − 𝜆̅𝐼) 𝑥⟩ = 0 ⇒ ⟨(𝐴 − 𝜆𝐼)𝑥, (𝐴 − 𝜆𝐼)𝑥⟩ = 0 ⇒ 𝑥 ∈ 𝒩𝐴−𝜆𝐼
∗
∗
Since: (𝐴∗ − 𝜆̅𝐼) = (𝐴∗ )∗ − (𝜆̅𝐼) = 𝐴 − 𝜆𝐼
Have shown:
𝒩(𝐴−𝜆𝐼)2 ⊆ 𝒩(𝐴−𝜆𝐼)
But this is always true:
𝒩(𝐴−𝜆𝐼) ⊆ 𝒩(𝐴−𝜆𝐼)2
So we must have equality: 𝒩(𝐴−𝜆𝐼) = 𝒩(𝐴−𝜆𝐼)2
(3) 𝐴𝑢𝑖 = 𝜆𝑖 𝑢2 , 𝐴𝑢𝑗 = 𝜆𝑗 𝑢𝑗 , 𝜆𝑖 ≠ 𝜆𝑗 ⇒ ⟨𝑢𝑖 , 𝑢𝑗 ⟩ = 0
𝜆𝑖 ⟨𝑢𝑖 , 𝑢𝑗 ⟩ = ⟨𝜆𝑖 𝑢𝑖 , 𝑢𝑗 ⟩ = ⟨𝐴𝑢𝑖 , 𝑢𝑗 ⟩ = ⟨𝑢𝑖 , 𝐴∗ 𝑢𝑗 ⟩ = ⟨𝑢𝑖 , 𝜆𝑗̅ 𝑢𝑗 ⟩ = 𝜆𝑗 ⟨𝑢𝑖 , 𝑢𝑗 ⟩ ⇒
(𝜆𝑖 − 𝜆𝑗 )⟨𝑢𝑖 , 𝑢𝑗 ⟩ = 0. So if 𝜆𝑖 ≠ 𝜆𝑗 then it must be that ⟨𝑢𝑖 , 𝑢𝑗 ⟩ = 0 and thus they are
orthogonal.
(4) 𝐴∗ 𝐴 = 𝐴𝐴∗ ⇒ can choose an orthonormal basis of eigenvectors of 𝐴.
det(𝜆𝐼3 − 𝐴) = (𝜆 − 𝜆1 )2 (𝜆 − 𝜆2 ) ⇒ dim 𝒩(𝐴−𝜆1 𝐼3 ) = 2, dim 𝒩(𝐴−𝜆2 𝐼3 ) = 1
⟨𝑢1 , 𝑢3 ⟩ = 0, ⟨𝑢2 , 𝑢3 ⟩ = 0
⟨𝑢1 , 𝑢2 ⟩ =?
We will later use the grahm shmidt method to construct an orthonormal basis for the same
eigenvalue.
𝜆1
But since 4 is true: 𝐴[𝑢1 , … , 𝑢𝑛 ] = [𝜆1 𝑢1 … 𝜆𝑛 𝑢𝑛 ] = [𝑢1 … 𝑢𝑛 ] [ 0
0
𝐴𝑈 = 𝑈𝐷
0
⋱
0
0
0]
𝜆𝑛
(5) If 𝐴 = 𝐴∗ ⇒ 𝜆𝑖 ∈ ℝ
𝜆𝑖 ⟨𝑢𝑖 , 𝑣𝑖 ⟩ = ⟨𝐴𝑢𝑖 , 𝑢𝑖 ⟩ = ⟨𝑢𝑖 , 𝐴∗ 𝑢𝑖 ⟩ = ⟨𝑢𝑖 , 𝐴𝑢𝑖 ⟩ = ⟨𝑢𝑖 , 𝜆𝑖 𝑢𝑖 ⟩ = 𝜆̅𝑖 ⟨𝑢𝑖 , 𝑢𝑗 ⟩
(𝜆𝑖 − 𝜆̅𝑖 )⟨𝑢𝑖 , 𝑢𝑖 ⟩ = 0 ⇒ 𝜆𝑖 = 𝜆̅𝑖
𝐴∗ 𝐴 = 𝐴𝐴∗ = 𝐼 ⇒ |𝜆𝑖 | = 1
⟨𝑢𝑖 , 𝑢𝑖 ⟩ = ⟨𝐴∗ 𝐴𝑢𝑖 , 𝑢𝑖 ⟩ = ⟨𝐴𝑢𝑖 , (𝐴∗ )∗ 𝑢𝑖 ⟩ = ⟨𝐴𝑢𝑖 , 𝐴𝑢𝑖 ⟩ = ⟨𝜆𝑖 𝑢𝑖 , 𝜆𝑖 𝑢𝑖 ⟩ = |𝜆𝑖 |2 ⟨𝑢𝑖 , 𝑢𝑖 ⟩ ⇒
(1 − |𝜆𝑖 |2 )⟨𝑢𝑖 , 𝑢𝑖 ⟩ = 0 ⇒ |𝜆𝑖 |2 = 1
If ⟨𝑥, 𝑦⟩𝒰 = ⟨𝑥, 𝑦⟩𝑠𝑡 = 𝑦 𝐻 𝑥
𝐴∗ = (… )𝐴𝐻 (… )
If Δ1 = 𝐼𝑞 , Δ2 = 𝐼𝑝 then 𝐴∗ = 𝐴𝐻
If 𝐴 ∈ ℂ𝑝×𝑝 and 𝐴 = 𝐴𝐻 , then:
1) 𝐴 is diagonalizable
2) It’s Eigenvalues are real
3) There exists an orthonormal basis (w.r.t. standard inner product) of eigenvectors
𝑢1 , … , 𝑢𝑛 of 𝐴.
𝐴𝑈 = 𝑈𝐷, where 𝐷 is diagonal with real entries
𝑈 = [𝑢1
𝑢2
𝑢3 ]
𝑢1𝐻
𝑈 𝑈 = [𝑢2𝐻 ] [𝑢1
𝑢3𝐻
𝐻
𝑢2
𝑢3 ]
1, 𝑖𝑓 𝑖 = 𝑗
(𝑈 𝐻 𝑈)𝑖𝑗 = 𝑢𝑖𝐻 𝑢𝑗 = ⟨𝑢𝑗 , 𝑢𝑖 ⟩𝑠𝑡 = {
0 𝑖𝑓 𝑖 ≠ 𝑗
Projections And Direct Sums
A linear transformation 𝒫 from a vector space 𝒰 into 𝒰 is called a projection if 𝒫 2 = 𝑃
If 𝒰 is an inner product space, 𝒫 is called an orthogonal projection if 𝒫 2 = 𝑃 and 𝑃 = 𝑃∗ or
𝒫 2 = 𝑝 and ⟨𝑃𝑥, 𝑦⟩𝒰 = ⟨𝑥, 𝑃𝑦⟩𝒰 for all 𝑥, 𝑦 ∈ 𝒰
𝒰 = ℂ2
1 𝑎
𝑃=[
]
0 0
𝑥 ∈ ℂ2 ,
𝑃𝑥 = ℂ2
𝑝2 = 𝑃
‖𝑃‖ can be large. ‖𝑃‖ = 1 will be denoted as an orthogonal projection later…
Let 𝒰 be a finite dimension vector space over 𝔽, let 𝑃 a projection. Then 𝒰 = 𝒩𝑃 ∔ ℛ𝑝
𝒩𝑝 = {𝑢|𝑃𝑢 = 0},
ℛ𝑝 = {𝑃𝑢|𝑢 ∈ 𝒰}
Let 𝑢 ∈ 𝒰. Then: 𝑢 = 𝑃𝑢
⏟ + (𝐼 − 𝑃)𝑢
∈ℛ𝑃
Claim: (𝐼 − 𝑃)𝑢 ∈ 𝒩𝑃
𝑃=𝑃2
𝑃(𝐼 − 𝑃)𝑢 = (𝑃 − 𝑃2 )𝑢 = 0
Remains to show that 𝒩𝑃 ∩ ℛ𝑃 = {0}
Let 𝑥 ∈ 𝒩𝑃 ∩ ℛ𝑃
𝑥 ∈ ℛ𝑝 ⇒ 𝑥 = 𝑃𝑦
𝑥 ∈ 𝒩𝑃 ⇒ 𝑃𝑥 = 0
But then:
0 = 𝑃𝑥 = 𝑃2 𝑦 = 𝑃𝑦 = 𝑥
Conversely, let 𝒰 = 𝒱 ∔ 𝒲 is a direct sum decomposition of vector space 𝒰 in terms of the two
subspaces 𝒱 and 𝒲.
Then each 𝑢 ∈ 𝒰 have a unique decomposition: 𝑢 = 𝑣 + 𝑤 where 𝑣 ∈ 𝒱 and 𝑤 ∈ 𝒲.
Claim: There’s only 1 such decomposition.
Proof: if 𝑣1 + 𝑤1 = 𝑣2 + 𝑤2 ⇒ (𝑣1 − 𝑣2 ) = (𝑤2 − 𝑤1 )
But the left hand side is in 𝒱 and the right hand side is in 𝒲. So in their intersection – which
means both sides are zero and therefore 𝑣1 = 𝑣2 and 𝑤1 = 𝑤2 .
Start with 𝑢. There is exactly one 𝑣 ∈ 𝒱 such that 𝑢 − 𝑣 ∈ 𝒲.
Call this 𝑣 𝑃𝒱 𝑢
Claim: 𝑃𝒱 2 = 𝑃𝒱
Proof: 𝑃𝒱 𝑢 = 𝑣 ∈ 𝒱
𝑣 =𝑣+0
𝑃𝒱 𝑣 = 𝑣
Example: ℂ2
1
𝒱 = 𝑠𝑝𝑎𝑛 {[ ]}
0
1
𝒲 = 𝑠𝑝𝑎𝑛 {[ ]}
2
𝑎
1
1
[ ] = 𝛼[ ]+𝛽[ ]
𝑏
0
2
𝑏 = 2𝛽
𝑎 =𝛼+𝛽
𝑏 1
𝑏 1
𝑎
[ ] = (𝑎 − ) [ ] + [ ]
𝑏
2 0
2 2
𝑏 1
𝑎
𝑃𝒱 [ ] = (𝑎 − ) [ ]
𝑏
2 0
1
𝒱 = 𝑠𝑝𝑎𝑛 {[ ]}
0
1
𝒲 = 𝑠𝑝𝑎𝑛 {[ ]}
3
𝑏 1
𝑎
𝑃𝒱 [ ] = (𝑎 − ) [ ]
𝑏
3 0
--- end of lesson
𝒰 = 𝒱 ∔ 𝒲,
𝒱 ∩ 𝒲 = {0}
𝑢 ∈ 𝒰 there is exactly 1 decomposition 𝑢 = 𝑣 + 𝑤 with 𝑣 ∈ 𝒱, 𝑤 ∈ 𝒲
Let 𝑣1 , … , 𝑣𝑘 be a basis for 𝒱, 𝑤1 , … , 𝑤𝑙 be a basis for 𝒲 then
{𝑣1 , … , 𝑣𝑘 , 𝑤1 , … , 𝑤𝑙 } is a basis for 𝒰 ⇒
𝑘
𝑙
𝑢 = ∑ 𝑎𝑖 𝑣𝑖 + ∑ 𝑏𝑗 𝑤𝑗 ,
𝑖=1
𝑎𝑖 , 𝑏𝑗 ∈ 𝔽
𝑗=1
𝑘
𝑃𝒱 𝑢 = ∑ 𝑎𝑖 𝑣𝑖
𝑖=1
This recipe depends upon 𝒲 as well as 𝒱.
(1) Start with direct sum decomposition, can define 𝑃𝒱 and observe that
𝑃𝒱2 𝑢 = 𝑃𝒱2 (𝑣 + 𝑤) = 𝑃𝒱 (𝑣 + 𝑤)
(2) Can start with 𝑃, a linear transformation from 𝒰 into 𝒰 such that 𝑃2 = 𝑃, then 𝒰 =
ℛ𝑃 ∔ 𝒩𝑃
If 𝒰 is an inner product space and 𝑃 is linear transformation from 𝒰 into 𝒰 such that:
1) 𝑃2 = 𝑃
2) 𝑃∗ = 𝑃
Then 𝒰 = ℛ𝑃 ∔ 𝒩𝑃
Now 𝒩𝑃 is orthogonal to ℛ𝑃 .
𝑣 ∈ ℛ𝑃 ,
𝑤 ∈ 𝒩𝑃
𝑣 ∈ ℛ𝑃 ⇒ 𝑣 = 𝑃𝑦
𝑎𝑙𝑤𝑎𝑦𝑠 𝑡𝑟𝑢𝑒
⟨𝑣, 𝑤⟩𝒰 = ⟨𝑃𝑦, 𝑤⟩𝒰
=
Now 𝒩𝑃 is orthogonal to ℛ𝑃
𝑃=𝑃∗
⟨𝑦, 𝑃∗ 𝑤⟩𝒰 = ⟨𝑦, 𝑃𝑤⟩𝒰 = ⟨𝑦, 0⟩𝒰 = 0
If 𝒱 is a subspace of 𝒰
𝒱 ⊥ - the orthogonal complement of 𝒱 in 𝒰 = {𝑢 ∈ 𝒰|⟨𝑢, 𝑣⟩𝒰 = 0 ∀𝑣 ∈ 𝒱}
Claim: ℛ𝑃⊥ ⊆ 𝒩𝑃
Proof: Suppose that 𝑤 ∈ ℛ𝑃⊥ ⇒ ⟨𝑃𝑦, 𝑤⟩ = 0 for all 𝑦 ∈ 𝒰
This is the same as: ⟨𝑦, 𝑃 ∗ 𝑤⟩ = ⟨𝑦, 𝑃𝑤⟩
𝑖𝑛
𝑎𝑟𝑡𝑖𝑐𝑢𝑙𝑎𝑟
So we know that 0 = ⟨𝑦, 𝑃𝑤⟩ ∀𝑦 ∈ 𝒰 ⇒
⟨𝑃𝑤, 𝑃𝑤⟩ = 0 ⇒ 𝑃𝑤 = 0
Orthogonal Decomposition
𝒰 =𝑉⊕𝒲
Means that 𝑢 = 𝑣 + 𝑤, 𝑣 ∈ 𝒱, 𝑤 ∈ 𝒲
Moreover, ⟨𝑣, 𝑤⟩𝒰 = 0 ∀𝑣 ∈ 𝒱, 𝑤 ∈ 𝒲
We really have 3 symbols!
𝒱 + 𝒲 = {𝑣 + 𝑤|𝑣 ∈ 𝒱, 𝑤 ∈ 𝒲}
𝒱 ∔ 𝒲 when 𝒱 ∩ 𝒲 = {0}
𝒱 ⊕ 𝒲 when 𝒱 ⊥ 𝒲 (𝒲 = 𝒱 ⊥ ) (only defined in an inner product space)
Let 𝒰 be an inner product space over 𝔽, let 𝒱 be a subspace of 𝒰 with basis {𝑣1 , … , 𝑣𝑘 }
Then 𝒰 = 𝒱 ⊕ 𝒲
Let 𝑢 = 𝑣 + 𝑤, 𝑣 ∈ 𝒱, 𝑤 ∈ 𝒲, ⟨𝑣, 𝑤⟩ = 0
Question: Given 𝑢 find 𝑣
𝑘
𝑘
𝑢 = ∑ 𝑐𝑗 𝑣𝑗 + 𝑢 − ∑ 𝑐𝑗 𝑣𝑗
Try to choose 𝑐𝑗 such that ⟨𝑢 −
𝑗=1
𝑘
∑𝑗=1 𝑐𝑗 𝑣𝑗 , 𝑣𝑖 ⟩𝒰
𝑗=1
= 0, 𝑖 = 1, … , 𝑘
𝑘
⟨𝑢, 𝑣𝑖 ⟩𝒰 = 𝑢 − ∑ 𝑐𝑗 ⟨𝑣
⏟𝑗 , 𝑣𝑖 ⟩ = 0
𝑗=1
𝑔𝑖𝑗
We would like to achieve the equality:
𝑘
∑ 𝑔𝑖𝑗 𝑐𝑗 = ⟨𝑢, 𝑣𝑖 ⟩𝒰 ,
𝑖 = 1, … , 𝑘
𝑗=1
𝑐1
𝑏1
⋮
⟨𝑢,
⟩
Let’s choose a vector 𝑏 = [ ] , 𝑏𝑖 =
𝑣𝑖 𝒰 , 𝑐 = [ ⋮ ]
𝑐𝑘
𝑏𝑘
So we got that: 𝒢𝑐 = 𝑏
Since all vectors are linearly independent, 𝒢 is invertible. So: 𝑐 = 𝒢 −1 𝑏
𝑃𝒱 𝑢 = ∑𝑘𝑗=1 𝑐𝑗 𝑣𝑗 where 𝑐 = 𝒢 −1 𝑏
Suppose 𝒰 = ℂ𝑛 with the standard inner product.
⟨𝑢, 𝑤⟩ = ∑ 𝑢𝑗 ̅̅̅
𝑤𝑗 = 𝑤 𝐻 𝑢
⟨𝑢, 𝑣1 ⟩
𝑣1𝐻 𝑢
𝑏 = [ ⋮ ] = [ ⋮ ],
⟨𝑢, 𝑣𝑘 ⟩
𝑣𝑘𝐻 𝑢
𝑉 = [𝑣1
𝑣1𝐻
… 𝑣𝑘 ] ⇒ 𝑉 = [ ⋮ ] ⇒ 𝑏 = 𝑉 𝐻 𝑢
𝑣𝑘𝐻
𝐻
𝑔𝑖𝑗 = ⟨𝑣𝑗 , 𝑣𝑖 ⟩ = 𝑣𝑖𝐻 𝑣𝑗 = 𝑒𝑖𝐻 𝑉 𝐻 𝑉𝑒𝑗
So 𝒢 = 𝑉 𝐻 𝑉
𝑘
𝑃𝒱 𝑢 = ∑ 𝑐𝑗 𝑣𝑗 = 𝑉𝑐
𝑗=1
Because 𝑐 = 𝒢 −1 𝑏 = (𝑉 𝐻 𝑉)−1 𝑉 𝐻 𝑢
So 𝑃𝒱 𝑢 = 𝑉(𝑉 𝐻 𝑉)−1 𝑉 𝐻 𝑢
𝑉 = [𝑣1 … 𝑣𝑘 ]
TODO: Draw area of triangle…
ℎ ∙ ‖𝑎‖ = area.
𝒱 = 𝑠𝑝𝑎𝑛{𝑎}
𝑃𝒱 𝑏 = 𝑎(𝑎𝑇 𝑎)−1 𝑎𝑇 𝑏
𝑏 = 𝑎(𝑎𝑇 𝑎)−1 𝑎𝑇 𝑏 + 𝑏 − 𝑎(𝑎𝑇 𝑎)−1 𝑎𝑇 𝑏
So area = ‖𝑏 − 𝑎(𝑎𝑇 𝑎)−1 𝑎𝑇 𝑏‖ ∙ ‖𝑏‖
𝑎𝑟𝑒𝑎2 = ⟨𝑏 − 𝑎(𝑎𝑇 𝑎)−1 𝑎𝑇 𝑏, 𝑏 − 𝑎(𝑎𝑇 𝑎)−1 𝑎𝑇 𝑏⟩ ∙ ‖𝑎‖2
(⟨𝑏, 𝑏⟩ − ⟨𝑃𝒱 𝑏, 𝑏⟩) ∙ ‖𝑎‖2 = (𝑏 𝑇 𝑏 − 𝑏 𝑇 𝑎(𝑎𝑇 𝑎)−1 𝑎𝑇 𝑏)𝑎𝑇 𝑎 = (𝑏 𝑇 𝑏)(𝑎𝑇 𝑎) − (𝑏 𝑇 𝑎)(𝑎𝑇 𝑏)
Since: ⟨𝑏 − 𝑃𝒱 𝑏, 𝑃𝒱 𝑏⟩ = ⟨𝑃𝒱 𝑏 − 𝑝𝒱2 𝑏, 𝑏⟩ = ⟨0, 𝑏⟩ = 0
𝑇
𝑇
(𝑎𝑟𝑒𝑎)2 = [𝑎𝑇 𝑎 𝑎𝑇 𝑏] = det 𝐶 ∙ 𝐶 𝑇
𝑏 𝑎 𝑏 𝑏
Where 𝐶 = [𝑎 𝑏]
𝑠𝑖𝑛𝑐𝑒 𝑖𝑛 ℝ2
=
𝐶=𝐶 𝑇
det 𝐶 ∙ det 𝐶 𝑇 = (det 𝐶)2
If 𝑎, 𝑏 ∈ ℝ2
Correction
When we said triangle area earlier, we meant the area of a parallelogram!
𝑎, 𝑏 ∈ ℝ2 , then the area is = det[𝑎
𝑏]
Back to the lesson
𝒰 inner product space, 𝒱 subspace of 𝒰 with basis 𝑣1 , … , 𝑣𝑘 . Let 𝑃𝒱 𝑢 denote the orthogonal
projection of 𝑢 onto 𝒱. Have a recipe.
In addition to what we’ve already done,
Fact (n): min{‖𝑢 − 𝑣‖|𝑣 ∈ 𝒱} is achieved by choosing 𝑣 = 𝑃𝒱 𝑢.
2
‖𝑢 − 𝑣‖2 = ‖
𝑃𝒱 𝑢 − 𝑣 ‖
‖𝑢 − 𝑃𝒱 𝑢 + ⏟
‖
∈𝒱
𝐷𝑒𝑛𝑜𝑡𝑒
𝑎𝑠 𝑤
Claim: it equals to ‖𝑢 − 𝑃𝒱 𝑢‖2 + ‖𝑃𝒱 𝑢 − 𝑣‖
𝑏𝑦 𝑑𝑒𝑓
Proof: ‖𝑢 − 𝑃𝒱 𝑢 + 𝑤‖2 = ⟨𝑢 − 𝑃𝒱 𝑢 + 𝑤, 𝑢 − 𝑃𝒱 𝑢 + 𝑤⟩ =
⟨𝑢 − 𝑃𝒱 𝑢, 𝑢 − 𝑃𝒱 𝑢⟩ + ⟨𝑢 − 𝑃𝒱 , 𝑤⟩ + ⟨𝑤, 𝑢 − 𝑃𝒱 𝑢⟩ + ⟨𝑤, 𝑤⟩
Harry claims the second and third piece is zero. Why?
If 𝑤 ∈ 𝒱 then
⟨𝑢 − 𝑃𝒱 𝑢, 𝑤⟩ = ⟨𝑢 − 𝑃𝒱 𝑢, 𝑃𝒱 𝑤⟩ = ⟨𝑃𝒱∗ (𝑢 − 𝑃𝒱 𝑢), 𝑤⟩ = ⟨𝑃𝒱 (𝑢 − 𝑃𝒱 ), 𝑤⟩ = ⟨0, 𝑤⟩ = 0
So we are left with ‖𝑢 − 𝑃𝒱 𝑢‖2 + ‖𝑃𝒱 𝑢 − 𝑣‖
Since we want the expression as small as possible, we can only play with the right hand part
(since the first hand part is determined!)
So we want to choose the 𝑣 such that ‖𝑃𝒱 𝑢 − 𝑣‖ = 0
We can do so if we choose 𝑣 = 𝑃𝒱 𝑢
So min‖𝑢 − 𝑣‖2 𝑣 ∈ 𝒱 is in fact
‖𝑢 − 𝑃𝒱 𝑢‖2 = ⟨𝑢 − 𝑃𝒱 𝑢, 𝑢 − 𝑃𝒱 𝑢⟩ = ⟨𝑢 − 𝑃𝒱 𝑢, 𝑢⟩ − ⟨𝑢 − 𝑃𝒱 𝑢, 𝑃𝒱 𝑢⟩
But ⟨𝑢 − 𝑃𝒱 𝑢, 𝑃𝒱 𝑢⟩ = 0!
So it actually equals ⟨𝑢 − 𝑃𝒱 𝑢, 𝑢⟩ = ⟨𝑢, 𝑢⟩ − ⟨𝑃𝒱 𝑢, 𝑢⟩ = ⟨𝑢, 𝑢⟩ − ⟨𝑃𝒱2 𝑢, 𝑢⟩
But 𝑃𝒱∗ = 𝑃𝒱 !
So equals: ⟨𝑢, 𝑢⟩ − ⟨𝑃𝒱 𝑢, 𝑃𝒱∗ 𝑢⟩ = ‖𝑢‖2 − ‖𝑃𝒱 𝑢‖2
𝑘
𝑃𝒱 𝑢 = ∑ 𝑐𝑗 𝑣𝑗 ,
𝑐 = 𝒢 −1 𝑏,
1
𝑏=[
⟨𝑢, 𝑣𝑖 ⟩
⋮ ]
⟨𝑢, 𝑣𝑘 ⟩
Another special case, when 𝑣1 , … , 𝑣𝑘 are orthonormal to the given inner product.
𝑔𝑖𝑗 = ⟨𝑣𝑗 , 𝑣𝑖 ⟩ = 1 if 𝑖 = 𝑗 and 0 if 𝑖 ≠ 𝑗
In this case, 𝒢 = 𝐼 ⇒ 𝑐 = 𝑏 ⇒
𝑘
𝑃𝒱 𝑢 = ∑⟨𝑢, 𝑣𝑗 ⟩𝒰 𝑣𝑗
𝑗=1
Suppose 𝒰 is an inner product space, and suppose {𝑢1 , … , 𝑢𝑙 } is an orthonormal basis for 𝒰.
Given 𝑢 ∈ 𝒰 ⇒ 𝑢 = ∑𝑙𝑗=1 𝑐𝑗 𝑢𝑗
𝑙
⟨𝑢, 𝑢𝑖 ⟩ = ∑ 𝑐𝑗 ⟨𝑢𝑗 , 𝑢𝑖 ⟩ = 𝑐𝑖
𝑗=1
2
So ⟨𝑢, 𝑢⟩ = ∑𝑙𝑗=1|𝑐𝑗 |
Gram-Schimdt
Given a linearly independent set of vectors 𝑢1 , … , 𝑢𝑘 in 𝒰.
Claim: There exists an orthonormal set 𝑣1 , … , 𝑣𝑘 such that
𝑠𝑝𝑎𝑛{𝑣1 , … , 𝑣𝑗 } = 𝑠𝑝𝑎𝑛{𝑢1 , … , 𝑢𝑗 } for 𝑗 = 1, … , 𝑘
Proof:
𝑢
Define 𝑣1 = ‖𝑢1 ‖ ⇒ ⟨𝑣1 , 𝑣1 ⟩ −
1
⟨𝑢1 ,𝑢1 ⟩
‖𝑢1 ‖2
=1
Introduce the notation: 𝒱𝑗 = 𝑠𝑝𝑎𝑛{𝑣1 , … , 𝑣𝑗 }
Define 𝑤2 = 𝑢2 − 𝑃𝒱1 𝑢2
We are doing it since this vector will be orthogonal to 𝑣1 .
Also claim 𝑤2 ≠ 0.
𝑤
And define: 𝑣2 = ‖𝑤2 ‖
2
Keep on doing that.
Define 𝑤3 = 𝑢3 − 𝑃𝒱2 𝑢3
So the vector is also orthogonal!
𝑤
And then 𝑣3 = ‖𝑤3 ‖
3
Now 𝑤4 = 𝑢4 − 𝑃𝒱3 𝑢4 = 𝑢4 − {⟨𝑢4 , 𝑣1 ⟩𝑣1 + ⟨𝑢4 , 𝑣2 ⟩𝑣2 + ⟨𝑢4 , 𝑣3 ⟩𝑣3 } such that
𝑤4
𝑣4 =
‖𝑤4 ‖