Lecture 6. Inverse of Matrix
Recall that any linear system can be written as a matrix equation
A~x = ~b:
In one dimension case, i.e., A is 1 £ 1; then
Ax = b
can be easily solved as
x=
b
1
= b = A¡1 b provided that A 6= 0:
A
A
In this lecture, we intend to extend this simple method to matrix equations.
De…nition 7.1. A square matrix An£n is said to be invertible if there exists a unique
matrix Cn£n of the same size such that
AC = CA = In :
The matrix C is called the inverse of A; and is denoted by
C = A¡1
Suppose now An£n is invertible and C = A¡1 is its inverse matrix. Then the matrix
equation
A~x = ~b
can be easily solved as follows. Left-multipling the matrix equation by the inverse matrix
C = A¡1 ; we have
CA~x = C~b:
By de…nition, CA = A¡1 A = In : It leads to
In ~x = C~b;
which is the same as
~x = A¡1~b:
(1)
We have just solve the matrix equation and obtained a unique solution (1). The above
discussion is summarized as
Theorem 7.1 Let A be an invertible matrix. Then matrix equation
A~x = ~b
has a unique solution
~x = A¡1~b:
1
Example 7.1 (a) Show A is invertible and A¡1 = C; where
·
¸
·
¸
2
5
¡7 ¡5
A=
; C=
:
¡3 ¡7
3
2
(b) Solve
2x1 + 5x2 = 1
¡3x1 ¡ 7x2 = 4:
(c) Show that the matrix
·
¸
0 2
B=
0 0
is NOT invertible.
Solution: (a) Direct calculations lead to
·
¸·
¸ ·
¸
2
5
¡7 ¡5
1 0
AC =
=
= I2
¡3 ¡7
3
2
0 1
·
¸·
¸ ·
¸
¡7 ¡5
2
5
1 0
CA =
=
= I2 :
3
2
¡3 ¡7
0 1
By de…nition,
C = A¡1 :
(b) The system has the coe¢cient matrix A; :i:e:; the matrix equation is
· ¸
1
A~x =
:
4
Therefore, According to Theorem 1, the solution is
· ¸
¡1 1
~x = A
4
·
¸· ¸
¡7 ¡5 1
=
3
2
4
·
¸
¡27
=
:
11
(c) By calculation, we …nd that
·
0 2
B =
0 0
2
¸·
¸ ·
¸
0 2
0 0
=
:
0 0
0 0
Now, suppose that B is invertible. Then there exists a matrix D such that
BD = I2 :
2
Multiplying the above equation by B from the left, we …nd
B (BC) = BI2 ;
B 2 C = B;
which implies
0 = B;
2
since B = 0: This is obviously a contradiction. Therefore, the assumption at the beginning
"suppose that B is invertible" is false, and consequently, B is NOT invertible.
Theorem 7.2 A 2 £ 2 matrix
·
¸
a b
A=
c d
is invertible i¤
def
det (A) = ad ¡ bc 6= 0:
When ad ¡ bc 6= 0; the inverse is
¡1
A
Example 7.2. (a) Find A¡1 if
·
¸
1
d ¡b
=
:
ad ¡ bc ¡c a
·
¸
2
5
A=
:
¡3 ¡6
(b) Solve
2x + 5y = 1
¡3x ¡ 6y = 2
Solution: (a) a = 2; b = 5; c = ¡3; d = ¡6: ad ¡ bc = ¡12 ¡ (¡15) = 3: So
·
¸
1
d ¡b
¡1
A =
ad ¡ bc ¡c a
·
¸ ·
¸
1 ¡6 ¡5
¡2 ¡5=3
=
=
:
2
1
2=3
3 3
We may verify the above solution as follows:
·
¸·
¸ ·
¸
2
5
¡2 ¡5=3
1 0
=
:
¡3 ¡6
1
2=3
0 1
(b) To solve the system, we write its matrix equation: A~x = ~b; where
·
¸
· ¸
2
5
1
~
A=
; b=
:
¡3 ¡6
2
The solution is
¸ · ¸ · 16 ¸
¡2 ¡5=3 1
¡
~x = A b =
= 73 :
1
2=3
2
3
¡1~
·
3
² Properties if Invertible Matrix:
Theorem 7.3 Suppose that A is an invertible square matrix. Then
¡1
1. A¡1 is also invertible and (A¡1 ) = A:
¡ ¢¡1
T
2. AT is also invertible and AT
= (A¡1 ) :
3. If B is another invertible matrix of the same size, then AB is also invertible, and
(AB)¡1 = B ¡1 A¡1 :
4. The reduced Echelon of A is the identity matrix I of the same size, i.e., A ¡! I.
Proof. (1) By de…nition, A¡1 if we can …nd a matrix C such that
¡ ¡1 ¢
¡
¢
A
C = C A¡1 = I:
The above is indeed true if C = A:
(2) Take transposes of all three sides of (A¡1 ) A = A (A¡1 ) = I;
¡¡
¢ ¢T ¡ ¡
¢¢T
¡
¢T ¡
¢T
A¡1 A = A A¡1
= I T =) AT A¡1 = A¡1 AT = I
¡
¢T
=) AT C = CAT = I; where C = A¡1
is the inverse of AT :
(3) Let C = B ¡1 A¡1 : Since
¡¡
¢ ¢
¡
¡
¢¢
¡
¢
¡
¢
C (AB) = (CA) B = B ¡1 A¡1 A B = B ¡1 A¡1 A B = B ¡1 I B = B ¡1 B = I
¡ ¡
¢¢
¡¡
¢
¢
¡
¢
(AB) C = A (BC) = A B B ¡1 A¡1 = A BB ¡1 A¡1 = A IA¡1 = AA¡1 = I:
it follows from the de…nition that this C is the inverse of AB; i.e.,
(AB)¡1 = C = B ¡1 A¡1 :
(4) Since A~x = ~0 has the only solution
~x = A¡1~0 = ~0;
there is no non-trivial solution. Consequently, A has no free variable. All columns are pivot
columns. Since A is a square matrix, this means that r (A) = number of columns = number
of rows. Therefore, the reduced Echelon form of A has a non-zero entry in each row and
thus has to be the identity matrix.
We next develop an algorithm to …nd inverse matrices.
De…nition 7.2 A matrix is called an elementary matrix if it is obtained by performing
one single elementary row operation on an identity matrix.
4
Example 7.3 Let us look at 3 £ 3 elementary matrices for corresponding row operations.
A type (1) elementary matrix E1 is obtained by performing one type (1) row operation. For
instance,
2
3
2
3
1 0 0
1 0 0
40 1 05 R2 + ¸R1 ! R2 4¸ 1 05 = E1 :
¡¡¡¡¡¡¡¡¡¡¡!
0 0 1
0 0 1
We call E1 is the elementary matrix associated with the row operation
R2 + ¸R1 ! R2 :
For any matrix A; performing the above row operation is the same as left multiplying by E1 :
For instance, we see that
2
3
2
3
1 3 ¡1
1 3 ¡1
42 1 0 5 R2 + (¡2) R1 ! R2 40 ¡5 2 5 :
4 0 1 ¡¡¡¡¡¡¡¡¡¡¡¡¡¡! 4 0
1
On the other hand, left multiplication by E1
2
3
1 3 ¡1
E1 A = E1 42 1 0 5
4 0 1
2
32
1 0 0
1
4
5
4
= ¡2 1 0
2
0 0 1
4
with ¸= ¡2 yields
3 2
3
3 ¡1
1 3 ¡1
1 0 5 = 40 ¡5 2 5 :
0 1
4 0
1
A type (2) elementary matrix E2 is obtained by performing one type (2) row operation. For
instance,
2
3
2
3
1 0 0
0 1 0
40 1 05 R2 ! R1 41 0 05 = E2 :
¡¡¡¡¡¡!
0 0 1
0 0 1
For any matrix A; performing the above row operation is the same as left multiplying by E2 :
For instance, we see that
2
3
2
3
1 3 ¡1
2 1 0
42 1 0 5 R2 ! R1 41 3 ¡15
¡¡¡¡¡¡!
4 0 1
4 0 1
On the other hand, left multiplication by E2 yield
2
3
1 3 ¡1
E2 A = E2 42 1 0 5
4 0 1
2
32
3 2
3
0 1 0
1 3 ¡1
2 1 0
= 41 0 05 42 1 0 5 = 41 3 ¡15 :
0 0 1
4 0 1
4 0 1
5
A type (3) elementary matrix E3
instance,
2
1 0
40 1
0 0
is obtained by performing one type (3) row operation. For
3
2
3
0
1 0 0
05 ¸R3 ! R3 40 1 0 5 = E3 :
¡¡¡¡¡¡¡!
1
0 0 ¸
Similarly, performing the above row operation is the
instance, we see that
2
3
2
1 3 ¡1
1
42 1 0 5 1 R3 ! R3 42
4¡¡¡¡¡¡! 1
4 0 1 ¡
On the other hand, left multiplication by E3
2
3
1 3 ¡1
E3 A = E3 42 1 0 5
4 0 1
2
32
1 0 0
1
4
5
4
2
= 0 1 0
0 0 1=4 4
same as left multiplying by E2 : For
3
3 ¡1
1 0 5:
0 1=4
with ¸= 1=4 leads to
3 2
3
3 ¡1
1 3 ¡1
1 0 5 = 42 1 0 5 :
0 1
1 0 1=4
Conclusion Performing an elementary row operation on an identity matrix produces an
elementary matrix corresponding to that elementary row operation. Any elementary row
operation is equivalent to left multiplying by the corresponding elementary matrix.
² Justi…cation of LU Decomposition Algorithm
Recall in Lecture 2, we introduced LU Decomposition as follows: Any matrix A may be
decomposed as the product
Am£n = Lm£m Um£n
of a lower triangle matrix L and a upper triangle matrix U by the following algorithm:
1. Reduce A to an echelon form from U by a sequence of type one row operations (row
replacement row operation)
2. Place entries in L such that the same sequence of row operations reduces L to the
identity matrix.
We can now justify the algorithm. We know that reducing A to U by a sequence of row
operations is equivalent to multiplying A from the left by a sequence of elementary matrices,
i.e.,
Ep ¢ ¢ ¢ E2 E1 A = U:
It follows that
A = (Ep ¢ ¢ ¢ E2 E1 )¡1 U:
6
This means that
L = (Ep ¢ ¢ ¢ E2 E1 )¡1
or equivalently
Ep ¢ ¢ ¢ E2 E1 L = I:
This shows that the same sequence of row operations reduces L to I
² Inverse matrix Algorithm:
Suppose now A is invertible. Then, its reduced Echelon form is the identity matrix. In
other words, by a series of successive row operations, the matrix A is reduced to I: Since
each row operation is equivalent to left multiplication by an elementary matrix, this means
that there exist elementary matrices E1 ; E2 ; :::; Ek ; such that
(Ek :::E2 E1 ) A = I:
By de…nition, this implies
A¡1 = Ek :::E2 E1 = Ek :::E2 E1 (I) :
The very last equation says that the exact same row operations that reduce A to the identity
I; in the same time, also transforms the identity matrix I to A¡1 :
We create a n £ (2n) matrix by adding the identity to the left of A :
2
3
a11 a12 ::: a1n
1 0 ::: 0
6 a21 a22 ::: a2n
0 1 ::: 0 7
6
7
[A I] = 6 ..
..
..
.. ..
.. 7
4 .
. :::
.
. . ::: . 5
am1 ::: ::: amn
0 ::: ::: 1
We then perform row operations till the …rst n columns form the identity matrix. When the
…rst n columns form the identity matrix, the remaining columns form the inverse A¡1 :
£
¤
[A I]
elementary row operations
I A¡1
¡¡¡¡¡¡¡¡¡¡¡¡¡¡¡¡¡¡¡¡¡¡!
Example 7.4 Find A¡1 if
2
3
1 3 ¡1
A = 4 2 7 0 5:
¡2 0 15
7
Solution:
2
3
1 3 ¡1 1 0 0
[A; I] = 4 2 7 0 0 1 05
¡2 0 15 0 0 1
2
1 3 ¡1 1
R2 ¡ 2R1 ! R2 4
0 1 2 ¡2
R3 + 2R1 ! R3
¡¡¡¡¡¡¡¡¡¡¡¡! 20 6 13 2
1 0 ¡7 7
R1 ¡ 3R2 ! R1 4
0 1 2 ¡2
R3 ¡ 6R2 ! R3
¡¡¡¡¡¡¡¡¡¡¡¡! 0 0 1 14
2
1 0 0 105
R1 + 7R3 ! R1 4
0 1 0 ¡30
R2 ¡ 2R3 ! R2
¡¡¡¡¡¡¡¡¡¡¡¡! 0 0 1 14
2
3
105 ¡45 7
A¡1 = 4¡30 13 ¡25
14 ¡6 1
3
0 0
1 05
0 1
3
¡3 0
1 05
¡6 1
3
¡45 7
13 ¡25
¡6 1
We may verify our answer directly:
2
32
3 2
3
1 3 ¡1
105 ¡45 7
1 0 0
4 2 7 0 5 4¡30 13 ¡25 = 40 1 05 :
¡2 0 15
14 ¡6 1
0 0 1
² Extended Inverse Matrix Algorithm
Using the same idea, the inverse matrix algorithm can be generalized to …nd A¡1 B: Let
An£n be a n £ n square invertible matrix and B be a n £ m matrix. We combine the columns
of A and columns of B to create a n £ (n + m) matrix [A B] ; and perform row operations
till the …rst n columns of the augmented matrix [A B] form the identity matrix of dimension
n: When the …rst n columns of the augmented matrix [A B] form the identity matrix, the
last m columns forms A¡1 B: In short,
£
¤
[A B] ! I A¡1 B :
² Characterization of Invertible Matrices
We summarize this entire lecture by the following theorem.
Theorem 7.4 (Invertible matrix theorem) Let An£n be a square matrix. The following
statements are equivalent:
1. The matrix A is invertible.
2. AT is invertible.
8
3. AB = AC =) B = C (i.e., cancellation law holds for A.
4. A is left-invertible, i.e., there exists C such that CA = I:
5. A is right-invertible, i.e., there exists D such that AD = I:
6. The reduced Echelon form of A is identity matrix I:
7. A is row-equivalent to identity matrix I:
8. A has n pivot positions.
9. A~x = ~0 has only trivial solution ~x = ~0
10. Column vectors of A are linearly independent.
11. Column vectors of A span Rn :
12. Row vectors of A span Rn :
13. A~x = ~b is consistent for all ~b in Rn :
14. The columns of A form a basis for Col (A) :
15. Col (A) = Rn :
16. Rank(A) = n:
17. Rank(Null (A)) = 0:
¡ ¢¡1
T
Proof. (Very brie‡y) (1) () (2) : because AT
= (A¡1 )
(1) =) (3) : AB = AC =) A (B ¡ C) = 0 =) A¡1 A (B ¡ C) = 0 =) (B ¡ C) = 0
(1) =) (4) and (1) =) (5) : obviously;
(1) =) (5) : Ep :::E2 E1 A = I =) Ep :::E2 E1 = A¡1
(6) () (7) : A » I
(7) () (8) : I has n pivots
(8) =) (9) : no free variable
(9) =) (10) : any linear relation ~x is a solution of A~x = ~0
(8) =) (11) : A~x = ~b is always consistent since by (10), the last column in the augmented
matrix is non-pivot
(2) =) (12) : columns of AT are exactly the rows of A
(11) () (13) : re-statements.
(11) () (15) : re-statements.
(16) () (17) : dimension theorem
9
² Homework #6.
·
¸
· ¸
· ¸
· ¸
· ¸
1 2 ~
¡1 ~
1 ~
2 ~
3
1. Let A =
; b1 =
; b2 =
; b3 =
; b4 =
:
5 12
3
5
5
1
(a) Find A¡1 ; and use it to solve A~x1 = ~b1 ; A~x2 = ~b2 ; A~x3 = ~b3 ; A~x4 = ~b4 :
(b) Consider the augmented matrix
h
i ·1 2 ¡1 1 2 3¸
~
~
~
~
A b1 b2 b3 b4 =
:
5 12 3 5 5 1
Perform row operations to reduced the the augmented matrix to a matrix whose
…rst two columns are identity. When the …rst two columns form identity matrix,
…nd the last four columns.
(c) Are the answers from part (a) and (b) above the same? Can you explain why?
2. Use Inverse Matrix Algorithm to …nd the inverse of
2
3
1
0 ¡1
(a) A = 4¡3 1
25
2 ¡3 2
2
3
1
0 ¡1 1
6¡1 1
2 27
7
(b) B = 6
4¡2 ¡3 2 05
0
0
1 2
3. Given
A¡1
2
3
2
3
1 ¡3 2
¡2 1 ¡1
=4 0
1
3 5 ; B ¡1 = 4 0 2 1 5 :
¡1 0 ¡1
3 1 0
Find (AB)¡1 and (BA)¡1 :
4. Find A¡1 B if
2
3
2
3
1 ¡2 1
3 4 ¡1
A = 44 ¡7 3 5 ; B = 42 3 6 5
0 6 ¡4
6 1 3
5. For each statement below, determine whether it is true or false. If it is false, provide
a counterexample. If true, explain why.
(a) Suppose A is a n £ n matrix, and B and C are n £ m matrices. If AB = AC;
then B = C:
10
(b) let A and B be n £ n matrices. Suppose that AB is invertible. Then both A and
B are invertible.
(c) (AB)¡1 = A¡1 B ¡1 :
(d) If a square matrix A has an echelon form whose diagonal entries are all pivot,
then A is invertible.
(e) If A is a square matrix and if A~x = ~b has a unique solution for some ~b; then A is
invertible.
(f) Let A be a n £ n matrix, and B a n £ m matrix. If AB = 0; then B = 0:
11