18.06.08: ‘Inverting
matrices’
Lecturer: Barwick
Monday 22 February 2016
18.06.08: ‘Inverting matrices’
Suppose
𝑎11 𝑎12
𝑎
𝑎22
𝐴 = ( 21
⋮ ⋮
𝑎𝑛1 𝑎𝑛2
⋯ 𝑎1𝑛
⋯ 𝑎2𝑛
)
⋱ ⋮
⋯ 𝑎𝑛𝑛
an 𝑛 × 𝑛 (square!!) matrix. We contemplate 𝐴 via the map 𝑇𝐴 ∶ R𝑛
R𝑛 .
18.06.08: ‘Inverting matrices’
Recall that 𝑇𝐴 is defined by the rule
𝑣1
𝐴⃗ 1 ⋅ 𝑣1⃗
∑𝑛𝑖=1 𝑎1𝑖 𝑣𝑖
𝑛
𝑛
𝑣
𝐴⃗ ⋅ 𝑣 ⃗
∑ 𝑎 𝑣
𝑇𝐴 (𝑣)⃗ = 𝐴 ( 2 ) = ∑ 𝑣𝑖 𝐴𝑖⃗ = ( 𝑖=1 2𝑖 𝑖 ) = ( 2 2 ) .
⋮
⋮
⋮
𝑖=1
𝑛
⃗
𝑣𝑛
𝐴 𝑛 ⋅ 𝑣𝑛⃗
∑𝑖=1 𝑎𝑛𝑖 𝑣𝑖
We’re interested in inverting 𝑇𝐴 ; that is, we’re interested in finding, for each
vector 𝑤⃗ ∈ R𝑛 , a vector 𝑥⃗ ∈ R𝑛 such that 𝑤⃗ = 𝐴𝑥.⃗ When we can do this, we
write
𝑥⃗ = 𝑇𝐴−1 (𝑤).
⃗
18.06.08: ‘Inverting matrices’
In other words, we want to have a way of solving uniquely any system of linear
equations that looks like
𝑛
𝑤1
=
∑ 𝑎1𝑖 𝑥𝑖 ;
𝑖=1
𝑛
𝑤2
=
∑ 𝑎2𝑖 𝑥𝑖 ;
𝑖=1
⋮
𝑛
𝑤𝑛
=
∑ 𝑎𝑛𝑖 𝑥𝑖 ,
𝑖=1
regardless of what numbers 𝑤1 , … , 𝑤𝑛 are!
Note that we can’t always do this …
18.06.08: ‘Inverting matrices’
Here’s a 5 × 5 matrix:
1 1 1 1 1
2 3 4 5 6
(
𝐴=
3 5 7 9 11 )
4 7 10 13 16
10
16 22 28 34 )
(
I know straightaway that I won’t be able to solve just any old equation 𝑤⃗ = 𝐴𝑥.⃗
(How?)
18.06.08: ‘Inverting matrices’
Here’s the bit that is genuinely surprising: suppose 𝐴 is invertible – that is,
suppose we can find, for each vector 𝑤⃗ ∈ R𝑛 , a vector 𝑥⃗ ∈ R𝑛 such that
𝑤⃗ = 𝐴𝑥.⃗ Then there’s a matrix 𝐴−1 such that
𝑥⃗ = 𝐴−1 𝑤.⃗
Let us reflect upon this miracle!
18.06.08: ‘Inverting matrices’
What we’re saying is that if 𝐴 is invertible, then any system of linear equations
𝑛
𝑤1
=
∑ 𝑎1𝑖 𝑥𝑖 ;
𝑖=1
𝑛
𝑤2
=
∑ 𝑎2𝑖 𝑥𝑖 ;
𝑖=1
⋮
𝑛
𝑤𝑛
=
∑ 𝑎𝑛𝑖 𝑥𝑖
𝑖=1
18.06.08: ‘Inverting matrices’
admits a unique solution, and moreover that there’s some matrix
𝐴−1
𝑏11 𝑏12
𝑏
𝑏
= ( 21 22
⋮ ⋮
𝑏𝑛1 𝑏𝑛2
such that for any 𝑖,
⋯ 𝑏1𝑛
⋯ 𝑏2𝑛
)
⋱ ⋮
⋯ 𝑏𝑛𝑛
𝑛
𝑥𝑖 = ∑ 𝑏𝑖𝑗 𝑤𝑗 .
𝑗=1
What are the columns of this matrix?
18.06.08: ‘Inverting matrices’
When 𝑛 = 1, this is familiar. A 1 × 1 matrix 𝐴 is just a number. When is 𝐴
invertible? What’s the inverse matrix 𝐴−1 ?
18.06.08: ‘Inverting matrices’
When 𝑛 = 2, the action gets a bit more interesting:
𝐴=(
𝑎 𝑏
)
𝑐 𝑑
In order for 𝐴 to be invertible, I need to know that any time I have a couple
of numbers 𝑢 and 𝑣, I can solve the system of linear equations
𝑢 = 𝑎𝑥 + 𝑏𝑦;
𝑣 = 𝑐𝑥 + 𝑑𝑦
for 𝑥 and 𝑦. Let’s work our way through the computation.
18.06.08: ‘Inverting matrices’
So our 2 × 2 matrix 𝐴 is invertible if and only if 𝑎𝑑 − 𝑏𝑐 ≠ 0, in which case we
have
𝑑
−𝑏
𝑑 −𝑏
1
𝑎𝑑−𝑏𝑐 ) .
𝐴−1 =
(
) = ( 𝑎𝑑−𝑏𝑐
−𝑐
𝑎
𝑎𝑑 − 𝑏𝑐 −𝑐 𝑎
𝑎𝑑−𝑏𝑐
𝑎𝑑−𝑏𝑐
The number 𝑎𝑑 − 𝑏𝑐 is called the determinant of 𝐴.
18.06.08: ‘Inverting matrices’
The 2 × 2 case is, sadly, a bit misleading. General formulæ are not always so
pleasant, or so useful. The problem is that the complexity of these formulæ
increases like 𝑛3 𝑛!. So the 3 × 3 case isn’t so bad: the matrix
𝑎 𝑏 𝑐
𝐴=( 𝑑 𝑒 𝑓 )
𝑔 ℎ 𝑖
is invertible if and only if
𝑎(𝑒𝑖 − 𝑓ℎ) − 𝑏(𝑑𝑖 − 𝑓𝑔) + 𝑐(𝑑ℎ − 𝑒𝑔) ≠ 0,
18.06.08: ‘Inverting matrices’
and in that case,
−1
𝐴
𝑒𝑖 − 𝑓ℎ 𝑐ℎ − 𝑏𝑖 𝑏𝑓 − 𝑐𝑒
1
=
( 𝑓𝑔 − 𝑑𝑖 𝑎𝑖 − 𝑐𝑔 𝑐𝑑 − 𝑎𝑓 ) .
𝑎(𝑒𝑖 − 𝑓ℎ) − 𝑏(𝑑𝑖 − 𝑓𝑔) + 𝑐(𝑑ℎ − 𝑒𝑔)
𝑑ℎ − 𝑒𝑔 𝑏𝑔 − 𝑎ℎ 𝑎𝑒 − 𝑏𝑑
18.06.08: ‘Inverting matrices’
However, the 4 × 4 case already pushes the utility of general formulæ to their
breaking point: consider the matrix
𝑎11
𝑎
𝐴 = ( 21
𝑎31
𝑎41
𝑎12
𝑎22
𝑎32
𝑎42
𝑎13
𝑎23
𝑎33
𝑎43
𝑎14
𝑎24
).
𝑎34
𝑎44
18.06.08: ‘Inverting matrices’
Now 𝐴 is invertible if and only if
det(𝐴) = 𝑎14 𝑎23 𝑎32 𝑎41 − 𝑎13 𝑎24 𝑎32 𝑎41 − 𝑎14 𝑎22 𝑎33 𝑎41 + 𝑎12 𝑎24 𝑎33 𝑎41
+ 𝑎13 𝑎22 𝑎34 𝑎41 − 𝑎12 𝑎23 𝑎34 𝑎41 − 𝑎14 𝑎23 𝑎31 𝑎42 + 𝑎13 𝑎24 𝑎31 𝑎42
+ 𝑎14 𝑎21 𝑎33 𝑎42 − 𝑎11 𝑎24 𝑎33 𝑎42 − 𝑎13 𝑎21 𝑎34 𝑎42 + 𝑎11 𝑎23 𝑎34 𝑎42
+ 𝑎14 𝑎22 𝑎31 𝑎43 − 𝑎12 𝑎24 𝑎31 𝑎43 − 𝑎14 𝑎21 𝑎32 𝑎43 + 𝑎11 𝑎24 𝑎32 𝑎43
+ 𝑎12 𝑎21 𝑎34 𝑎43 − 𝑎11 𝑎22 𝑎34 𝑎43 − 𝑎13 𝑎22 𝑎31 𝑎44 + 𝑎12 𝑎23 𝑎31 𝑎44
+ 𝑎13 𝑎21 𝑎32 𝑎44 − 𝑎11 𝑎23 𝑎32 𝑎44 − 𝑎12 𝑎21 𝑎33 𝑎44 + 𝑎11 𝑎22 𝑎33 𝑎44 ≠ 0.
And I just refuse to write down the formula for the inverse. Ain’t nobody got
time for that.
18.06.08: ‘Inverting matrices’
Still, the idea of inversion has conceptual value. Recall that the following are
logically equivalent for an 𝑛 × 𝑛 matrix 𝐴:
⃗ are linearly independent;
1. the column vectors 𝐴1⃗ , 𝐴2⃗ , … , 𝐴𝑚
2. the row vectors 𝐴⃗ 1 , 𝐴⃗ 2 , … , 𝐴⃗ 𝑚 are linearly independent;
18.06.08: ‘Inverting matrices’
3. the system of linear equations
0
=
𝑎11 𝑥1 + 𝑎12 𝑥2 + ⋯ + 𝑎1𝑛 𝑥𝑛 ;
0
=
𝑎21 𝑥1 + 𝑎22 𝑥2 + ⋯ + 𝑎2𝑛 𝑥𝑛 ;
⋮
0
=
𝑎𝑛1 𝑥1 + 𝑎𝑛2 𝑥2 + ⋯ + 𝑎𝑛𝑛 𝑥𝑛 ;
has exactly one solution (namely 0).
18.06.08: ‘Inverting matrices’
We have more:
4. for any real numbers 𝑤1 , … , 𝑤𝑛 , the system of linear equations
𝑤1
=
𝑎11 𝑥1 + 𝑎12 𝑥2 + ⋯ + 𝑎1𝑛 𝑥𝑛 ;
𝑤2
=
𝑎21 𝑥1 + 𝑎22 𝑥2 + ⋯ + 𝑎2𝑛 𝑥𝑛 ;
⋮
𝑤𝑛
=
𝑎𝑛1 𝑥1 + 𝑎𝑛2 𝑥2 + ⋯ + 𝑎𝑛𝑛 𝑥𝑛 ;
has exactly one solution;
5. the matrix 𝐴 in invertible;
6. there exists an 𝑛 × 𝑛 matrix 𝐴−1 such that 𝐴−1 𝐴 = 𝐴𝐴−1 = 𝐼.
18.06.08: ‘Inverting matrices’
There’s even one more:
7. the determinant of 𝐴 is nonzero.
(We’ll talk more about determinants later in the course.)
The point is, we have a notion that’s conceptually convenient, but not so very
computable. How do we manage?
18.06.08: ‘Inverting matrices’
Figure 1: Mother Mathematics. Artist’s rendition.
When Mother Mathematics gives you a problem that’s too hard to solve in general,
you solve it in an easy special case, and you try to reduce to that case.
18.06.08: ‘Inverting matrices’
So …which matrices can we invert easily?
18.06.08: ‘Inverting matrices’
Diagonal matrices are pretty easy: the matrix 𝐴 = diag(𝜆1 , … , 𝜆𝑛 ) is invertible
if and only if none of the 𝜆𝑖 s are zero, in which case
−1
𝐴−1 = diag(𝜆−1
1 , … , 𝜆𝑛 ).
18.06.08: ‘Inverting matrices’
More generally, if I’ve got myself an upper triangular matrix, we should be
able to work out whether it’s invertible:
𝑎11 𝑎12
0 𝑎22
𝐴=(
⋮ ⋮
0
0
When does that happen?
⋯ 𝑎1𝑛
⋯ 𝑎2𝑛
).
⋱ ⋮
⋯ 𝑎𝑛𝑛
18.06.08: ‘Inverting matrices’
So, what about something that isn’t quite upper triangluar? Is
2
0
𝐴=(
0
0
invertible?
1
3
0
0
8
8
8
8
5
2
)
5
1
18.06.08: ‘Inverting matrices’
Is
(
𝐴=(
(
(
2
1
0
0
0
0
1
3
0
0
0
0
𝑎
𝑒
8
8
0
0
𝑏
𝑓
5
1
0
0
𝑐 𝑑
𝑔 ℎ
𝑖 𝑗 )
)
𝑘 𝑙 )
5 −7
9 1 )
invertible? (Does it matter what 𝑎, 𝑏, 𝑐, 𝑑, 𝑒, 𝑓, 𝑔, ℎ, 𝑖, 𝑗, 𝑘, or 𝑙 are?)
18.06.08: ‘Inverting matrices’
Is
(
(
(
𝐴=(
(
(
(
(
(
invertible?
2
1
1
0
0
0
0
0
1
2
1
0
0
0
0
0
1 𝑎 𝑏 𝑐 𝑑 𝑒
1 𝑓 𝑔 ℎ 𝑖 𝑗
2 𝑘 𝑙 𝑚 𝑛 𝑜 )
)
0 5 6 𝑝 𝑞 𝑟 )
)
)
0 −4 9 𝑠 𝑡 𝑢 )
)
0 0 0 3 1 1 )
0 0 0 1 1 3
0 0 0 1 3 1 )