Matrices, transposes, and inverses
Math 40, Introduction to Linear Algebra
Wednesday, February 1, 2012
Matrix-vector multiplication: two views
•
1st perspective: A�
x is linear combination of columns of A
 
��
�� 44
� �
�� ��
�� ��
�� ��
−22 33  
−22
11 −
4
11
−
33
= 44
+ 33
+ 22
33 =
+
+
=
22 11 55
21
22
11
55
22
A
•
�x
2nd perspective: A�
x is computed as dot product of rows of A with vector �x
�
1
2
−2
1
A
  
4
� 4
3   
��
2
3 =
5
1
dot
product
of
2
5
�x
and

� �
4  =
3
2
�
4
21
�
Notice that # of columns of A = # of rows of �
x.
This is a requirement in order for matrix multiplication to be defined.
Matrix multiplication
What sizes of matrices can be multiplied together?
For m x n matrix A and n x p matrix B,
mxn nxp
the matrix product AB
is an m x p matrix.
“inner”
parameters
must match
“outer” parameters become
parameters of matrix AB
If A is a square matrix and k is a positive integer, we define
Ak = �A · A��· · · A�
k factors
Properties of matrix multiplication
Most of the properties that we expect to hold for matrix multiplication do....
A(B + C) = AB + AC
(AB)C = A(BC)
k(AB) = (kA)B = A(kB) for scalar k
.... except commutativity!!
In general, AB �= BA.
Matrix multiplication not commutative
Problems with hoping AB and BA are equal:
• BA may not be well-defined.
(e.g., A is 2 x 3 matrix, B is 3 x 5 matrix)
In general,
AB �= BA.
• Even if AB and BA are both defined, AB and BA may not be
the same size.
(e.g., A is 2 x 3 matrix, B is 3 x 2 matrix)
• Even if AB and BA are both defined and of the same size, they
still may not be equal.
�
��
� �
��
� �
� �
�
1 1 1 2
2 4
3 3
1 2 1 1
�
=
=
=
1 1 1 2
2 4
3 3
1 2 1 1
Truth or fiction?
Question 1
For n x n matrices A and B, is
A2 − B 2 = (A − B)(A + B) ?
No!!
(A − B)(A + B) = A2 + AB − BA − B 2
� �� �
�=0
Question 2
No!!
For n x n matrices A and B, is (AB)2 = A2 B 2 ?
(AB)2 = ABAB �= AABB = A2 B 2
Matrix transpose
Definition The transpose of an m x n matrix A is the n x m matrix
AT obtained by interchanging rows and columns of A,
i.e., (AT )ij = Aji
∀ i, j.
Example
A=
�
1 3 5
5 3 2
−2
1
�
Transpose operation can be viewed as
flipping entries about the diagonal.
Definition

1

3
AT = 
 5
−2

5
3

2
1
A square matrix A is symmetric if AT = A.
Properties of transpose
(1) (AT )T = A
apply twice -- get back
to where you started
(2) (A + B)T = AT + B T
(3) For a scalar c, (cA)T = cAT
(4) (AB)T = B T AT
To prove this, we show that
Exercise
Prove that for any matrix A, ATA is symmetric.
[(AB)T ]ij =
..
.
= [(B T AT )]ij
Special matrices
Definition A matrix with all zero entries is called
a zero matrix and is denoted 0.


0 0 0 0
A = 0 0 0 0
0 0 0 0

2
A = 0
0
Definition A square matrix is upper-triangular
if all entries below main diagonal are zero.
analogous definition for a lower-triangular matrix
 3
−8
 0
A=
 0
0
Definition A square matrix whose off-diagonal
entries are all zero is called a diagonal matrix.
Definition The identity matrix, denoted In, is the
n x n diagonal matrix with all ones on the diagonal.

5
0
−3
1
4
6
0

0
0 0
−2 0 0

0 −4 0
0
0 1

1
I3 = 0
0
0
1
0

0
0
1

1

I3 = 0
0
0
1
0

0
0
1
Identity matrix
Definition The identity matrix, denoted In, is the
n x n diagonal matrix with all ones on the diagonal.
If A is an m x n matrix, then
ImA = A and AIn = A.
If A is a square matrix, then
IA = A = AI.
Important property
of identity matrix
TheThe
inverse
of aofof
matrix
notion
inverse
The
inverse
a matrix
Exploration
Let’s think
inverses
first in
theincontext
of real
num-numExploration
Let’s about
think
first
the
of real
Consider
theabout
set ofinverses
real numbers,
andcontext
say that
we have
Exploration
bers. Say
we
have
equation
bers.equation
Say we have equation
the
=2
3x = 3x
23x =
2
1
weto
want
to solve
for
x. so, multiply both sides by
What
we do?
1obtain
and weand
want
solve
x. for
To x.
do
todo
and
we want
to for
solve
To do so, multiply both sides3 by
3 to obtain
1 sides
1 the 1equation by 12 to obtain
1
We multiply both
of
(3x) =(3x)(2)= (2)
=⇒ =⇒
x = 3x. = 2 .
3
3
31
3 2
13 3
1
1
(3x)
=
(2)
x1 (3)
=
For R, For
the13 multiplicative
1. = .1.
is the multiplicative
of =⇒
3 since
3 isR,
3 (3) =
3 inverseinverse
3of 3 since
3
3
multiplicative
inverse
of 3 since
1
(3) = 1
Now consider
the
systemsystem
of of
NoticeNotice
that that
we can
thesethese
3 following
Now consider
the following
we rewrite
can rewrite
equations
equations
as weascan rewrite equations as
equations
equations
The Now,
inverse
ofthea linear
matrix
consider
that
The
inverse
of
a system
matrix Notice
� � � ��� � � �
�
3 −5
6
3
x xof1 real
6 num3x
−
5x
=
6
3x
−
5x
=
6
Exploration
Let’s
think
about
inverses
first
in
the
1
2
1
2 Let’s think about inverses first−5
Exploration
incontext
the 1context
num= =of real
−2
3x2 x2 −1 −1
−2
3
bers. Say
we
have
equation
−2x
+
3x
=
−1
−2x
+
3x
=
−1
1
2
1 we have
2
� ����
bers. Say
equation
� ��� ��
� ����
� �� �� �� �
A �x
�x
�b
3x = 3x
2 =2
A
�b
which
wetowant
tofor
solve
for
x
and
x
.
1
2
which we
want
solve
x
and
x
.
1 isolate
2 the vector �
Howfordox.we
itselfby
on1 LHS?
x� bysides
and weand
want
solve
To x.
doTo
so,do
multiply
to 1obtain
�so, multiply
� � both
wetowant
to solve for
both
sides
3 by
x1
3 to obtain
x
1
How
do
we
isolate
the
vector
x
=
by
itself
on
the
How do we isolate
itself
the LHS?LHS?
1 the 1vector
1 x1= x by
2 on
x
2
2
(3x) =(3x)(2)= (2)
=⇒ 2 =⇒
x = x. = .
3
3 �3
3 3
3
� −3
�
�
1
1 −2 −5
both
sides
of matrix
equation
by
−3
−5
1sides
1 1.
For R, Multiply
is
the
multiplicative
inverse
of
3
since
=
Multiply
both
of
matrix
equation
by
: −3
For
R,
is
the
multiplicative
inverse
of
3
since
(3): = 1.
−2 −3
3
3 (3)
3
3
� � �� � �� � � ��
� � ��
� � � � �� �
�
�
−3
−5
3
−5
x
−3 −56 6
1
The notion
−3 −5 of inverse
3 −5 x1
−3
−5
=
=
Now consider
the following
of of
NoticeNotice
that−2
we −3
can
thesethese
Now consider
the following
system
that
we rewrite
can rewrite
−2
−3 system
−2 −3
−2 3−2 x32 x2 −2 −3
−1 −1
equations
equations
� equations
� � equations
� �as we
�ascan rewrite equations as
Now,
consider the linear system
� � �1 0� Notice
� that
�
x�
−13
1
��� � � �
1 0 x1
−13
=� 3��
x1 x1 6 6
3x1 − 3x
5x12 −
= 5x
62=6
0 1 =x2 3 −5
−9 −5
= =
0 1 � ��
x2 �
−9
−2 −2
3 3x2 x2 −1 −1
� �� � I
−2x
−2x1 +
3x12 +
= 3x
−12 = −1
� ����
� �����
� �� �� �� �
� �� ���� ��
I
� � x1� A�−13A �x �x �b �b
which
wetowant
to solve
x1xand
=
xx1 2.
−13
which we
want
solve
and
1 for
2 . the
x� by−9
How for
do x
we
isolate
vector
itself on LHS?
=x
2� �
�
�
x
−9
x
2
1
�
� ��the vector�xx�1= �� by� itself on the
�� �
we isolate
How doHow
we do
isolate
the vector
=
by
itself on the LHS?LHS?
3 x −5
x
6
1
x
2 =
x2
?
?
Thus, the solution to (�) is
and
−2x1 = 3−13 x
−1
2 = −9.
2 x
�−9.
�
Thus, the solution
to
(�)
is
x
=
−13
and
x
=
�
�
1
2
−3
−5
�both sides of��matrix equation
�
Multiply
by−5 −2 −3 :
Multiply
bothwant
sides
ofequal
matrix
equation
by I −3
−2 −3 :
this
to
identity
matrix,
� � �� � ��
� � ��
� � � � �� �
Lecture �
7
Math �
40,�Spring
Prof.
Page 1
� �Kindred
���’12,
−3
−5
3
−5
x
−3 −56 6
1
−3 −5 Math3 40, −5
x1 −3
−3
−5
=
Lecture 7
Spring ’12,
Prof. Kindred
Page 1
−5
=
−2 −3
−2 −3 −1
−2 −3
−2 3−2 x32 −2x2 −3
−2 −3
−1
� �� � �
�
� �� � �
�−13
� �
� 1 �0� 1x�01 �x1 −13
�
�� �
�
=
x1
1 0 x01 1 =x−3
−5
6
−13
−9
2
=
0 1 x2 =
� −2−9 −3 −1 = −9
x2
0 � 1�� ��x2��
I
� � �
�
I
� � x1�
�−13
=
x1
−13
−9
=x2
x2
−9
Matrix inverses
Definition A square matrix A is invertible (or nonsingular) if ∃ matrix
B such that AB = I and BA = I. (We say B is an inverse of A.)
Example
� �
�
�
2 7
4 −7
A=
is invertible because for B =
,
1 4
−1 2
� ��
� � �
2 7
4 −7
1 0
we have AB =
=
=I
1 4 −1 2
0 1
�
�� � � �
4 −7 2 7
1 0
and likewise BA =
=
= I.
−1 2
1 4
0 1
The notion of an inverse matrix only applies to square matrices.
- For rectangular matrices of full rank, there are one-sided inverses.
- For matrices in general, there are pseudoinverses, which are a generalization to matrix inverses.
�
1
Example Find the inverse of A =
1
� ��
� � �
�
1 1 a b
1 0
a+c
=
=⇒
1 1 c d
0 1
a+c
=⇒
�
1
. We have
1
� � �
b+d
1 0
=
b+d
0 1
a + c = 1 and a + c = 0
Impossible!
� �
1 1
Therefore, A =
is not invertible (or singular).
1 1
Take-home message:
Lecture 7
Not all square matrices are invertible.
Math 40, Spring ’12, Prof. Kindred
Page 1
Important questions:
• When does a square matrix have an inverse?
• If it does have an inverse, how do we compute it?
• Can a matrix have more than one inverse?
Theorem. If A is invertible, then its inverse is unique.
Proof. Assume A is invertible. Suppose, by way of contradiction, that the
inverse of A is not unique, i.e., let B and C be two distinct inverses of A.
Then, by def’n of inverse, we have
BA = I = AB (1)
and CA = I = AC. (2)
It follows that
B = BI
= B(AC)
= (BA)C
= IC
= C.
by
by
by
by
by
def’n of identity matrix
(2) above
associativity of matrix mult.
(1) above
def’n of identity matrix
Thus, B = C, which contradicts the previous assumption that B �= C.
⇒⇐ So it must be that case that the inverse of A is unique.
�
Take-home message:
The inverse of a matrix A is unique,
and we denote it A−1.
Theorem (Properties of matrix inverse).
(a) If A is invertible, then A−1 is itself invertible and (A−1)−1 = A.
Lecture 7
Math 40, Spring ’12, Prof. Kindred
Page 2
(b) If A is invertible and c �= 0 is a scalar, then cA is invertible and
(cA)−1 = 1c A−1.
(c) If A and B are both n × n invertible matrices, then AB is invertible
and (AB)−1 = B −1A−1.
“socks and shoes rule” – similar to transpose of AB
generalization to product of n matrices
(d) If A is invertible, then AT is invertible and (AT )−1 = (A−1)T .
To prove (d), we need to show that there is some matrix
AT = I
and
AT
such that
= I.
Proof of (d). Assume A is invertible. Then A−1 exists and we have
(A−1)T AT = (AA−1)T = I T = I
and
AT (A−1)T = (A−1A)T = I T = I.
So AT is invertible and (AT )−1 = (A−1)T .
Question:
�
If A and B are invertible n × n matrices,
what can we say about A + B?
There is no guarantee A + B is invertible even if A and B themselves are
invertible! In other words, we cannot say that (A + B)−1 = A−1 + B −1.
How do we compute the inverse of a matrix, if it exists?
Lecture 7
Math 40, Spring ’12, Prof. Kindred
Page 3
Inverse of a 2 × 2 matrix: Consider the special case where A is a 2 × 2
matrix with A = [ ac db ]. If ad − bc �= 0, then A is invertible and its inverse is
�
�
1
d −b
A−1 =
.
ad − bc −c a
� Exercise: Check that AA−1 = [ 10 01 ] = A−1A.
�
�
−2 1
Example For A =
, we have
3 −3
�
� �
�
1
1
−3
−1
−1
−
3 .
A−1 =
=
−1 − 23
3 −3 −2
We can easily check that
AA−1
and
�
��
� � �
1
−2 1
−1 − 3
1 0
=
2 =
3 −3 −1 − 3
0 1
�
��
� � �
1
−1
−
−2
1
1 0
3
A−1A =
=
.
−1 − 23
3 −3
0 1
How do we find inverses of matrices that are larger than 2 × 2 matrices?
Theorem. If some EROs reduce a square matrix A to the identity matrix
I, then the same EROs transform I to A−1.


A
I


EROs


I
-1
A


If we can transform A into I, then we will obtain A−1. If we cannot do so,
then A is not invertible.
Lecture 7
Math 40, Spring ’12, Prof. Kindred
Page 4
Example: Find the inverse of the matrix A =



� −1 −3 1 �
3
1
6 0
0 1
.

−1 −3 1 1 0 0
−1 −3 1 1 0 0
R2 +3R1
 3 6 0 0 1 0  −−−−→  0 −3 3 3 1 0 
R3 +R1
1 0 1 0 0 1
0 −3 2 1 0 1


1 3 −1 −1 0 0
−R1
−−−−
→  0 −3 3 3 1 0 
R3 −R2
0 0 −1 −2 −1 1


1 0 2 2 1 0
R +R2
 0 −3 3 3 1 0 
−−1−−→
−R3
0 0 1 2 1 −1


1
0
2
2
1
0
− 13 R2
−−−→  0 1 −1 −1 − 13 0 
0 0 1 2 1 −1


1 0 0 −2 −1 2
R1 −2R3
−−
−−→  0 1 0 1 23 −1 
R2 +R3
0 0 1 2 1 −1
Thus, A is invertible and its inverse is


−2 −1 2
A−1 =  1 23 −1 .
2 1 −1
Why does this work?
Lecture 7
=⇒
discussion next class
Math 40, Spring ’12, Prof. Kindred
Page 5