1.3.2 Multiplication of Matrices/Matrix Transpose
In section 1.3.1, we considered only square matrices, as these are of interest in solving
linear problems Ax = b.
The interpretation of a matrix as a linear transformation can be extended to non-square
matrix. If we consider a M x N real matrix A, then A maps every vector v ∈ RN into a
vector (now of dimensions m, not N) A v ∈ RN, according to the rule
a 11
a
Av =  21
:

a m1
a 12
a 22
:
a m2
a 1N   v1  a 11 v1 + a 12 v 2 + ... + a 1N v N 
... a 2N   v12  a 21 v1 + a 22 v 2 + ... + a 2N v N 
=
 :  :

:
  

... a mN   v N  a m1 v1 + a m2 v 2 + ... + a mN v N 
...
(1.3.2-1)
Note that the product Av is defined only if the number of columns of A equals the
dimensions (# of components) of v.
We give the M x N matrix A, with all aij real, the following pictorial interpretation:
RN
v
RN
A
Av
The interpretation of non-square matrices as linear transformations provides the
following rule for multiplying two real matrices:
Let A be a M x P real matrix, and let B be a P x N real matrix. We define C = AB to be
the M x N matrix (also real) that performs the same transformation to a vector v ∈ RN as
1st applying B, then A.
RN
v
Rp
B
RM
A(Bx)
A
Bv
C = AB
First, to v ∈ RN we apply B,
b 11
b
21
Bv = 
:

b p1
 N b v 
b 1N   v 1  ∑ j=1 1 j j 
N
... b 2N   v 12  ∑ j=1 b2 j v j 
 =

 :  :
:

  

... b pN   v N   N
b v 
∑ j=1 pj j 
b 12
...
b 22
:
b p2
(1.3.2-2)
We then apply A to Bv ∈ Rp,
a 11

a 21
A(Bv) = 
:

a m1
a 12
a 22
:
a m2
N
 N
  p

a 1p  ∑ j=1 b1 j v j  ∑k =1 a1k ∑ j=1 bkj v j 

N
p
N
... a 2p  ∑ j=1 b2 j v j  ∑k =1 a 2 k ∑ j=1 bkj v j 

=





:
:
 :
 

p
N
... a mp   N



b
v
a
b
v
∑ j=1 pj j  ∑k =1 mk ∑ j=1 kj j 
...
If we compare to
 N c v 
c11 c12 ... c1N   v1  ∑ j=1 1 j j 
N
c
c 22 ... c 2N   v12  ∑ j=1 c2 j v j 
21



Cv =
=
:

:
:  :  :

  

c m1 c m2 ... c mN   v N   N c v 
∑ j=1 mj j 
(1.3.2-4)
(1.3.2-3)
We see that rearranging (1.3.2-2) yields
∑ N
 j=1
 N
∑
A(Bv) =  j=1
:

∑ N
 j=1
(∑
(∑
)
)
N
a 1k b kj v j  ∑ j=1 c1j v j 
 

N
p


c v 
a b vj
k =1 2k kj
 = ∑ j=1 2j j 
 :

 

N
p


c
v
a
b
v
∑k =1 mk kj j  ∑ j=1 mj j 
p
k =1
(
(1.3.2-5)
)
The (i,j) element of the matrix C = AB is therefore
Cij =
∑
p
k =1
(1.3.2-6)
a ik b kj
We compute this element by summing the product of elements A along row #I from left
Æ right with those elements of B in column #j from top Æ bottom.
AB =
a 11

a 21

a i1

:
a m1

a 12
...
a 1p
a 22
...
a 2p
...
a ip
→
a i2
:
a m2









: :
... a mp
=
b11

b 21
:

b p1
b12
b 22
:
b p2
... b1j ... b1N 

... b 2j ... b 2N 

↓:

... b pj ... b pN 

:



:
.....cij ..... row #i



:

:


column # j
(1.3.2-7)
column # j
row # i
We note that the product of two matrices A and B, C = AB, is defined only if the number
of columns of A equals the number of rows of B.
Note also that in general AB ≠ BA (1.3.2-8). We define the commutator of A and B as
[A,B] ≡ A B – BA (1.3.2-9)
Note that we can interpret our rule for multiplying a vector v ∈ RN by an M x N matrix A
by considering v to be a matrix of dimension N x 1, i.e. a column vector.
a 11
a
 21
:

a m1
a 12
a 22
:
a m2
a 1N 
... a 2N 

:

... a mN 
...
 N

 v1  ∑ k =1 a1k vk 
v   N

 2  =  ∑ k =1 a 2 k vk 
:  :

  

N
v N  
a
v

∑
mk
k
 k =1
(1.3.2-10)
This is the convention that we will use. We can also write v as a row vector by taking the
transpose,
vT = [v1
v2 … vN]
(1.3.2-11)
We see that vT is a 1 x N matrix.
The dot product v • w can therefore be written for v, w ∈ RN
v • w = vTw = [v1
v2
w 1 
w 
… vN]  2  = v1w1 + v2w2 + … + vNwN
: 


w N 
(1.3.2-12)
We define a matrix transpose operation on a real matrix A of M rows and N columns
as
a 11
a
T
A =  21
:

a m1
a 12
a 13
a 22
a 23
:
a m2 a m3
a 11
T
a 1N 
a
 12
... a 2N 
= a 13

:


:
... a mN 
a 1N

...
a 21
...
a m1
a 22
...
a m2
a 23
...
a m3
...
a mN
:
a 2N
If A is an M x N matrix, AT is N x M and (AT)ij = aji
(1.3.2-14)








(1.3.2-13)