2E1: Linear Algebra | Lecture Notes
§1 Matrices and matrix algebra
1 Matrices and determinants
1.1
Matrices
Definition: An m×n matrix is a rectangular array of numbers
(m rows and n columns) enclosed in brackets. The numbers
are called the elements of the matrix.
Examples:
(i) A 2 × 3 matrix has 2 rows and 3 columns:
µ
¶
1 2 3
A=
5 6 7
(ii) Here’s a 3 × 3 square matrix:


1 2 3
A=5 6 7 
8 9 10
(iii) Column vectors are matrices with only one column:
 
1
b=5
8
(iv) Row vectors are matrices which only have one row:
b = (1 2 3) .
Unless specifically stated otherwise, we will assume that vectors are column vectors.
1.1
2E1: Linear Algebra | Lecture Notes
§1 Matrices and matrix algebra
A general real matrix, A ∈ Rm×n with m × n elements is of
the form


a11 a12 a13 . . . a1n
 a

 21 a22 a23 . . . a2n 


A =  a31 a32 a33 . . . a2n 
(1)
 ..

.
.
.
.
..
..
. . .. 
 .
am1 am2 am3 . . . amn
We refer to the elements via double indices as follows
(i) The first index represents the row.
(ii) The second index represents the column.
Example
a32 is the element in row 3, column 2 of the matrix A.
Notation/Conventions:
Use lowercase boldface (or underlined) letters for vectors
a b c
(or
a,
b,
c)
Use uppercase boldface (or underlined) letters for matrices
A
B C
(or
A,
B,
C)
Refer to the respective elements by lowercase letters with the
appropriate number of indices e.g.
bi
aij
is a vector element
is a matrix element
1.2
2E1: Linear Algebra | Lecture Notes
§1 Matrices and matrix algebra
1.2 Special matrices
The unit matrix, I, is a square matrix whose only non-zero
elements are on the diagonal and are equal to one, e.g.


1 0 0 ... 0 0
 0 1 0 ... 0 0 




1 0 0


0
0
1
.
.
.
0
0


I =  0 1 0  , I =  .. .. .. . . .. .. 
. . . . . .


0 0 1
 0 0 0 ... 1 0 
0 0 0 ... 0 1
All elements of the zero matrix, 0, are equal to zero, e.g.


0 0 0 ... 0 0
 0 0 0 ... 0 0 




0 0 0


0
0
0
.
.
.
0
0


0 =  0 0 0  , 0 =  .. .. .. . . .. .. 
. . . . . .


0 0 0
 0 0 0 ... 0 0 
0 0 0 ... 0 0
A diagonal matrix only has non-zero elements on the main
diagonal. These non-zero elements can have any value, e.g.


d11 0 . . .
0
0



0
0 
d11 0 0
 0 d22 . . .


.
.
.
.
.
..
..
..
.. 
D =  0 d22 0  , D =  ..



0 0 d33
 0 0 . . . dn−1,n−1 0 
0 0 ...
0
dnn
are square diagonal matrices.
1.3
2E1: Linear Algebra | Lecture Notes
§1 Matrices and matrix algebra
1.3 Matrix algebra
1.3.1
Matrix equality
Two matrices are equal if they have the same size and if their
corresponding elements are identical , i.e.
A=B
if and only if
aij = bij
for i = 1, ..., m; j = 1, ..., n
1.3.2 Matrix addition
Two matrices can only be added if they have the same size.
The result is another matrix of the same size.
We add matrices by adding their corresponding elements, i.e.
A=B+C
is obtained (element-wise) via
aij = bij + cij
Example
µ
A=
µ
A+B=
1 2 3
5 6 7
for i = 1, ..., m; j = 1, ..., n
¶
µ
B=
1 + 10 2 + 1 3 + 23
5 + 5 6 + 16 7 + 3
1.4
10 1 23
5 16 3
¶
µ
=
¶
11 3 26
10 22 10
¶
2E1: Linear Algebra | Lecture Notes
§1 Matrices and matrix algebra
1.3.3 Multiplication of a matrix by a scalar
A matrix is multiplied by a scalar (= a number) by multiplying
each element of the matrix by that scalar.
The result is a matrix of the same size.
Hence
A=αB
is given (element-wise) by
aij = α bij
for i = 1, ..., m; j = 1, ..., n
Example
µ
¶
1 2 3
A=
5 6 7
µ
¶ µ
¶
3×1 3×2 3×3
3 6 9
3A=
=
3×5 3×6 3×7
15 18 21
1.3.4 Matrix – Vector multiplication
Before defining the product of a matrix and a vector, let us
recall the notion of the dot product (or, scalar product) of two
vectors.
Let a ∈ Rn and b ∈ Rn be two column vectors with n real
elements each:


 
a1
b1
a 
b 
 2
 2


 
a =  a3  ,
b =  b3  .
 .. 
 .. 
 .
 .
an
bn
1.5
2E1: Linear Algebra | Lecture Notes
§1 Matrices and matrix algebra
Then the dot product (or, scalar product) of a and b is defined
as
a · b = a1b1 + a2b2 + a3b3 + · · · + anbn.
Note that the row vector corresponding to the (column) vector
a can be defined by the vector transpose operation:

T
a1
a 
 2
¡
¢


T
a =  a3  = a1 a2 a3 . . . an .
 .. 
 . 
an
Then the product of the row vector aT and the column vector
b can be defined as follows:
def
aT b = a · b = a1b1 + a2b2 + a3b3 + · · · + anbn
(2)
Example






2
5
5
¡
¢
T




a = −1 , b =
2 ; a b = 2 −1 3  2 
3
−4
−4

 

5
2
=  −1  ·  2  = 10 − 2 − 12 = −4.
−4
3
1.6
2E1: Linear Algebra | Lecture Notes
§1 Matrices and matrix algebra
Now we can define a matrix – vector multiplication.
The product of an m × n matrix A and a column vector x of
length n produces a column vector b of length m:
A |{z}
x = |{z}
b .
|{z}
m×n n×1
m×1
This implies that the number of columns of the matrix must
be equal to the number of rows in the column vector!
The elements of the vector b are calculated by taking
dot products of the rows of the matrix with the column vector:
if rows of A ∈ Rm×n are vectors aT1 , aT2 , . . ., aTm ∈ Rn then


aT1
 T 
 a2 
A =  .. 
 . 
aTm

⇒

aT1 x
 T 
 a2 x 
Ax = 

...
=b

aTm x
where aTi x are dot products defined in equation (2).
Element wise this is
n
X
aij xj = bi
for i = 1, ..., m.
j=1
1.7
(3)
2E1: Linear Algebra | Lecture Notes
§1 Matrices and matrix algebra
Example
µ
A=
1 2 3
5 6 7


3
x=2
4
¶
,
then A consists of the row vectors
aT1 = ( 1 2 3 )
aT2 = ( 5 6 7 )
and the matrix vector product



3

   ( 1 2 3 ) 2  

µ
¶ 3


1 2 3   
4 
  .
Ax =
2 =
5 6 7
3 



4
 ( 5 6 7 ) 2  
4
The vector products are simply the dot products of the respective row aTi with the column vector x
µ
¶
1×3+2×2+3×4
Ax =
5×3+6×2+7×4
and hence
µ
b=
3 + 4 + 12
15 + 12 + 28
1.8
¶
µ
=
19
55
¶
.
2E1: Linear Algebra | Lecture Notes
§1 Matrices and matrix algebra
1.3.5 Matrix – Vector multiplication using Falk’s scheme
For practical computations use Falk’s scheme and evaluate
Ax = b
in tabular form as follows:
x
A b
Example
µ
A=
1 2 3
5 6 7

¶
,

3
x=2
4
Evaluate Ax as follows:
3
2
4
1 2 3
5 6 7
Now place the elements of the resulting vector at the ‘intersection’ of the column vector and the rows of the matrix:
1 2 3
5 6 7
3
2
4
µ ¶
1×3+2×2+3×4
19
=
5×3+6×2+7×4
55
1.9
2E1: Linear Algebra | Lecture Notes
§1 Matrices and matrix algebra
1.3.6 Matrix – Matrix multiplication
The product of an m × n matrix A and an n × p matrix B
produces an m × p matrix C, i.e.
A |{z}
B = |{z}
C .
|{z}
m×n n×p
m×p
This implies that the number of columns of the first matrix
must be equal to the number of rows in the second matrix.
Here are some examples:
µ
A=
1 2 3
5 6 7


1 2 3
B=5 6 7 
8 9 10
¶
2×3
3×3



1 2


5 6
D=

8 9
3 7
3×4
4×2

1 2 3 2
C=5 6 7 2
8 9 10 7
We can form
AB – the result is a 2 × 3 matrix
AC – the result is a 2 × 4 matrix
CD – the result is a 3 × 2 matrix
It is not possible to form
BA
or
1.10
AD
2E1: Linear Algebra | Lecture Notes
§1 Matrices and matrix algebra
The product matrix of two matrices is obtained by taking dot
products of the rows of the left matrix with the columns of the
right matrix.
If the rows of A ∈ Rm×n are the vectors aT1 , aT2 , . . ., aTm ∈ Rn
and the columns of B ∈ Rn×p are b1, b2, . . ., bp ∈ Rn. Let


T
a1
 T 
¡
¢
 a2 
A =  ..  and B = b1 b2 . . . bp
 . 
aTm
then the matrix matrix product is

aT1 b1
 T
 a 2 b1
aT1 b2
aT2 b2

aT1 bp

aT2 bp 
...
...
AB =  ..
 ∈ Rm×p
.
.
.
.
.
.
.
.
. 
 .
aTm b1 aTm b2 . . . aTm bp
Element-wise this is
n
X
aij bjk = cik
for i = 1, ..., m; k = 1, ..., p.
j=1
1.11
(4)
2E1: Linear Algebra | Lecture Notes
Example
µ
A=
§1 Matrices and matrix algebra
1 2
3 4
¶
µ
B=
4 3 2
2 1 1
¶
The respective row and column matrices are
µ ¶
µ ¶
µ ¶
aT1 = (1 2)
4
3
2
b1 =
, b2 =
, b3 =
,
T
2
1
1
a2 = (3 4)
the matrix-matrix product is then
µ ¶
µ ¶

4
3
(1
2)
(1
2)

2
1

AB = 
µ ¶
µ ¶

4
3
(3 4)
(3 4)
2
1
µ
(1 2)
2
1
¶


µ ¶
2 
(3 4)
1
which can be evaluated by a series of dot products. That is


1×4+2×2 1×3+2×1 1×2+2×1

AB = 
3×4+4×2 3×3+4×1 3×2+4×1
Giving the final result

C=
8
5
4
20
13
10
1.12


2E1: Linear Algebra | Lecture Notes
§1 Matrices and matrix algebra
1.3.7 Matrix – Matrix multiplication using Falk’s scheme
For practical computation we again use Falk’s scheme and
evaluate AB = C in tabular form as follows:
B
A C
Example
µ
A=
1 2
3 4
¶
µ
,
B=
4 3 2
2 1 1
¶
Evaluate AB as follows
4
3
2
2
1
1
1 2 (1 × 4 + 2 × 2) (1 × 3 + 2 × 1) (1 × 2 + 2 × 1)
3 4 (3 × 4 + 4 × 2) (3 × 3 + 4 × 1) (3 × 2 + 4 × 1)
Place the elements at the intersection of the rows of the left
matrix and the columns of the right matrix
The result
4
3
2
2
1
1
1 2 (1 × 4 + 2 × 2) (1 × 3 + 2 × 1) (1 × 2 + 2 × 1)
3 4 (3 × 4 + 4 × 2) (3 × 3 + 4 × 1) (3 × 2 + 4 × 1)
therefore gives
µ
AB =
8 5 4
20 13 10
1.13
¶
2E1: Linear Algebra | Lecture Notes
§1 Matrices and matrix algebra
1.3.8 Differences from multiplication with numbers
(i) Matrix multiplication is not commutative
AB 6= BA
Example
µ
¶µ
¶ µ
¶
1 2
2 2
1×2+2×3 1×2+2×1
=
4 1
3 1
4×2+1×3 4×2+1×1
µ
¶
8 4
=
11 9
µ
2 2
3 1
¶µ
1 2
4 1
¶
µ
2×1+2×4 2×2+2×1
=
3×1+1×4 3×2+1×1
µ
¶
10 6
=
7 7
We must be careful how we multiply out!
(ii) AB = 0 does not imply A = 0, B = 0 or BA = 0
Example
µ
¶µ
¶ µ
¶
1 1
−1 1
0 0
=
2 2
1 −1
0 0
µ
−1 1
1 −1
¶µ
1 1
2 2
1.14
¶
µ
=
1 1
−1 −1
¶
¶
2E1: Linear Algebra | Lecture Notes
§1 Matrices and matrix algebra
(iii) AC = AD does not necessarily imply C = D
Example
µ
µ
1 1
2 2
1 1
2 2
¶µ
¶µ
2 1
2 2
3 0
1 3
¶
µ
=
¶
µ
=
4 3
8 6
4 3
8 6
¶
¶
(iv) BUT other properties are similar to numbers
A(B + C) = AB + AC
distributive law
A(BC) = (AB)C
associative law
1.15
2E1: Linear Algebra | Lecture Notes
§1 Matrices and matrix algebra
1.4 Transpose of a matrix
The transpose of a matrix is obtained by interchanging its
rows and columns
aTij = aji
for i = 1, ..., m; j = 1, ..., n
The transpose is denoted by a superscript T and the general
matrix given in equation (1) becomes


a11 a21 a31 . . . am1
 a a a ... a 
 12 22 32
m2 


T
A =  a13 a23 a33 . . . am3 
 ..
...
... . . . ... 
 .

a1n a2n a3n . . . amn
Example
µ
A=
1 2 3
5 6 7


1 5
AT =  2 6 
3 7
¶
If A = AT then A is a symmetric matrix, e.g.


3 2 −1
A= 2 7 0
−1 0 8
The matrix transpose also satisfies the following rules:
i) (AT )T = A for any matrix A;
ii) (A + B)T = AT + BT and (AB)T = BT AT ,
provided that matrices A and B have compatible dimensions.
1.16
2E1: Linear Algebra | Lecture Notes
§1 Matrices and matrix algebra
? < Proof of the identity (AB)T = BT AT ∗
If the rows of A ∈ Rm×n are the vectors aT1 , aT2 , . . ., aTm ∈ Rn
and the columns of B ∈ Rn×p are b1, b2, . . ., bp ∈ Rn.
Taking the transpose of equation (4)

aT1 b1
 T
 a 1 b2
aT2 b1
aT2 b2

aTm b1

aTm bp 
...
...
(AB)T =  ..
...
... 
...
 .

aT1 bp aT2 bp . . . aTm bp
For column vectors ai and bj vector multiplication aTi bj is
defined as the dot product between ai and bj . The dot product
is commutative so
aTi bj = ai · bj = bj · aj = bTj ai
which implies that

bT1 a1
 T
 b2 a1

bT1 a2
bT2 a2
bT1 am

bT2 am 
...
...
(AB)T =  ..
 = B T AT
.
.
.
.
.
.
.
.
. 
 .
bTp a1 bTp a2 . . . bTp am
since


bT1
 T 
 b2 
BT =  .. 
 . 
bTp
and AT =
¡
¢
a1 a2 . . . am .
>?
∗ Here and further in these notes, the material between the
markers ? < and > ? is for advanced reading.
1.17
2E1: Linear Algebra | Lecture Notes
§1 Matrices and matrix algebra
1.5 Determinant of a matrix
The determinant of a 2 × 2 matrix
µ
¶
a11 a12
A=
a21 a22
is written det A or |A| or
¯
¯
¯ a11 a12 ¯
¯
¯
¯ a21 a22 ¯ = a11a22 − a12a21
Example
A=
µ
1 2
4 −7
¶
,
det A = −7 − 8 = −15.
The determinant of a 3 × 3 matrix is written as
¯
¯
¯ a11 a12 a13 ¯
¯
¯
|A| = ¯¯ a21 a22 a23 ¯¯
¯ a31 a32 a33 ¯
¯
¯
¯
¯
¯
¯
¯ a21 a23 ¯
¯ a21 a22 ¯
¯ a22 a23 ¯
¯
¯ − a12 ¯
¯ + a13 ¯
= a11 ¯¯
¯
¯
¯
¯
a31 a32 ¯
a32 a33
a31 a33
= a11(a22a33 − a32a23) − a12(a21a33 − a31a23)
+ a13(a21a32 − a31a22)
That is the 3 × 3 determinant is defined in terms of determinants of 2 × 2 sub-matrices of A. These are called the minors
of A.
1.18
2E1: Linear Algebra | Lecture Notes
§1 Matrices and matrix algebra
Example
¯
¯
¯
¯ a11 a12 a13 ¯ ¯
¯
¯ ¯ a21 a23 ¯
¯
m12 = ¯¯ a21 a22 a23 ¯¯ = ¯¯
a31 a33 ¯
¯ a31 a32 a33 ¯
is obtained by suppressing the elements in row 1 and column
2 of matrix A.
Cofactors
The cofactor cij is defined as the coefficient of aij in the determinant A. If is given by the formula
cij = (−1)i+j mij
where the minor is the determinant of order (n − 1) × (n − 1)
formed by deleting the column and row containing aij .
Examples
¯
¯
¯ a11 a12 a13 ¯
¯
¯
c11 = (−1)1+1m11 = +1 · ¯¯ a21 a22 a23 ¯¯ = a22a33 − a32a23
¯ a31 a32 a33 ¯
¯
¯
¯ a11 a12 a13 ¯
¯
¯
2+3
¯
c23 = (−1) m23 = −1 · ¯ a21 a22 a23 ¯¯ = −a11a32 + a31a12
¯ a31 a32 a33 ¯
1.19
2E1: Linear Algebra | Lecture Notes
§1 Matrices and matrix algebra
General determinant
The value of an n × n determinant equals the sum of the products of the elements in any row (or column) and their cofactors, i.e.
n
X
|A| =
aij cij , for i = 1, . . . , n − 1, or n
j=1
or
|A| =
n
X
aij cij ,
for
j = 1, . . . , n − 1, or n
i=1
Example
For a 3 × 3 matrix
det A = a11c11 + a12c12 + a13c13
(1st row)
or
det A = a12c12 + a22c22 + a32c32
(2nd column)
Points to note:
• the determinant det A is equal to zero if
(i) rows or columns of A are multiples of each other,
(ii) rows or columns are linear combinations of each other,
(iii) entire rows or columns are zero;
if det A = 0 the matrix A is called a singular matrix;
• for any square matrices A and B there holds
det A = det(AT ),
det(AB) = det(A) det(B).
• for the unit matrix I one has det I = 1.
1.20
2E1: Linear Algebra | Lecture Notes
§1 Matrices and matrix algebra
1.6 The matrix inverse
The inverse a−1 of a scalar (=a number) a is defined by
a a−1 = 1.
For square matrices we use a similar definition: the inverse
A−1 of a n × n matrix A fulfils the relation
AA−1 = I
where I is the n × n unit matrix defined earlier.
Note: if A−1 exists then
det(A) det(A−1) = det(AA−1) = det I = 1.
Hence, det(A−1) = (det A)−1.
Example
The inverse of
µ
A=
is given by
3 2
7 5
µ
B = A−1 =
¶
5 −2
−7 3
¶
since
5
−2
−7
3
AB =
3 2 (3 × 5 − 2 × 7) (−3 × 2 + 2 × 3)
7 5 (7 × 5 − 5 × 7) (−7 × 2 + 5 × 3)
which gives
µ
AB =
1 0
0 1
as required.
1.21
¶
=I
2E1: Linear Algebra | Lecture Notes
§1 Matrices and matrix algebra
The matrix inverse can be computed as follows
1. Find the determinant det A
2. Find the cofactors of all elements in A and form a new
matrix C of cofactors, where each element is replaced by
its cofactor.
3. The inverse of A is now given as
A
−1
CT
=
det A
Note: the inverse A−1 exists if (and only if) det A 6= 0.
Example Find the inverse of


1 −1 2
A = −3 1 2  .
3 −2 −1
¯
¯
¯
¯
¯
¯
¯1 2¯
¯−3 2 ¯
¯−3 1 ¯
¯ − (−1) ¯
¯
¯
¯
det A = 1 ¯¯
¯ 3 −1¯ + 2 ¯ 3 −2¯
−2 −1¯
= 1 × 3 + 1 × (−3) + 2 × 3
=6
Since the determinant is nonzero an inverse exists.
1.22
2E1: Linear Algebra | Lecture Notes
§1 Matrices and matrix algebra
Calculate the matrix of minors
¯ ¯
¯ ¯
¯
¯¯
¯ ¯−3 2 ¯ ¯−3 1 ¯
1
2
¯
¯ ¯
¯ ¯
¯
¯−2 −1¯ ¯ 3 −1¯ ¯ 3 −2¯


¯
¯
¯ ¯
¯ ¯
¯−1 2 ¯ ¯1 2 ¯ ¯1 −1¯ 
¯
¯ ¯
¯ ¯
¯
M=
¯−2 −1¯ ¯3 −1¯ ¯3 −2¯ 


¯
¯ ¯
¯ ¯
¯
 ¯−1 2¯ ¯ 1 2¯ ¯ 1 −1¯
¯
¯ ¯
¯ ¯
¯
¯ 1 2¯ ¯−3 2¯ ¯−3 1 ¯


3 −3 3
=  5 −7 1 
−4 8 −2
Modify the signs according to whether i + j is even or odd to
calculate the matrix of cofactors


3 3 3
C = −5 −7 −1 .
−4 −8 −2
It follows that


3 −5 −4
1
1
A−1 = CT = 3 −7 −8 .
6
6
3 −1 −2
To check that we have made no mistake we can compute

 


1 0 0
1 −1 2
3 −5 −4
1
A−1A = 3 −7 −8 −3 1 2  = 0 1 0 .
6
0 0 1
3 −2 −1
3 −1 −2
This way of computing the inverse is only useful for hand
calculations in the cases of 2 × 2 or 3 × 3 matrices.
1.23