Matrix Algebra
by Salim
A-Preface
The modern approach to system analysis and design, including control systems, relies heavily on matrix
and matrix function manipulations. This handout reviews matrix addition, subtraction, multiplication,
determinant, inverse, simultaneous equations, eigenvalues, and eigenvectors. This handout serves as a
quick reference and it is not intended to be a primary source for matrix algebra basics.
B- What is a Matrix?
A matrix is generally defined as a rectangular array containing constants or variables. A matrix is
typically referenced by its name and size. The size is described by the number of rows and columns.
Examples:
7
−4
], and 𝑩 = [5] are matrixes of dimensions 2X3 and 3X1. The matrices are referred to
0
2
as A is a 2 by 3 matrix and B is a 3 by 1 matrix. As can also be seen, B is also referred to as a column
vector.
𝑨= [
1 5
3 1
Elements in a matrix are referenced according to their locations. In matrix A, the value -4 is element
A1,3.
A square matrix is one in which the number of rows equal the number of columns.
The identity matrix is a special matrix. It is a square matrix with 1s on the diagonal and zeros on the
off-diagonal (any non-diagonal element). The notation, I4 , implies a 4 X 4 identity matrix.
Symmetric matrix is another special square matrix. A matrix is said to be symmetric if An,m = Am,n
∈ 𝐼𝑛𝑡𝑒𝑔𝑒𝑟 𝑛𝑢𝑚𝑏𝑒𝑟𝑠 . Differently stated, Matrix A and its transpose are equal. A = AT
Example:
5 12 2
𝐴 = [12 7
6 ] = 𝐴𝑇
2
6 −4
Vector. A matrix with a single row or column is sometimes referred to as a vector. A matrix with a
single column is called a column vector and a matrix with a single row is called a row
vector.
Any 1X1 matrix, i.e. a matrix with only 1 entry, is called a scalar.
m,n
C- Matrix Operations
Transposition
The transpose of an m X n matrix is an n X m matrix constructed by switching rows and columns of the
original matrix. That is to say, elements that were on the first column of the original matrix now make
up the first row of the transposed matrix. The transpose of a given matrix A is denoted as AT.
Example:
Find the transpose of the following matrices.
𝑎
[𝑏
𝑐
sin(𝑡)
𝑑
𝑒 ], [ cos(𝑡) ]
𝑓
sin(𝑡) cos(𝑡)
Answer:
[
𝑎
𝑑
𝑏
𝑒
𝑐
], [sin(𝑡) cos(𝑡) sin(𝑡) cos(𝑡)]
𝑓
Matrix Addition and Subtraction
Matrix addition is a simple operation. The result of adding two matrices together is a third matrix whose
elements in any given row and column are obtained by adding elements in that same row and column in
the first matrix to the elements in the same row and column of the second matrix.
As a result, matrix addition is legal only when performed with two matrices of identical dimensions, i.e.
a 3X2 matrix can only be added to another 3X2 matrix.
Example:
4 8
9 14
5 6
[
]+ [
]=[
]
2 −9
−1 0
−3 9
Matrix subtraction is very similar. The result of A - B where A and B are both mXn matrices
is another m X n matrix C where cij = aij - bij . Where, cij, aij, and bij are corresponding matrix elements
I,j
∈ 𝐼𝑛𝑡𝑒𝑔𝑒𝑟 𝑛𝑢𝑚𝑏𝑒𝑟𝑠
Rules of Addition
A + B = B + A (Commutative)
(A + B) + C = A + (B + C) (Associative)
Scalar Multiplication
Any matrix can be multiplied by a scalar. If A is an mXn matrix and q is a scalar, the matrix that results
from multiplying A by the scalar q is a m X n matrix C, where each element is obtained by multiplying
the corresponding element A by q, i.e. cij = q aij .
Example:
7
5[
1
0 2
35 0 10
]=[
]
3 4
5 15 20
Matrix Multiplication
Matrix multiplication is more complicated than the basic operations mentioned above. When one
multiplies matrix A by matrix B (denoted AB) to obtain matrix C, a given element is obtained
by multiplying the ith row of A by the jth column of B. Therefore, multiplication of two
matrices is legal only when the ith row of the first matrix has the same number of elements
as the jth column of the second matrix: in other words the number of columns in A has
to equal the number of rows in B.
More generally an m X n matrix A can be multiplied by an n X p matrix B where p is any
positive integer. The resulting matrix C is of dimension m X p with elements denoted as
follows: Cij =∑𝑛𝑟=1 𝐴𝑖,𝑟 𝐵𝑟,𝑗 for i = 1,…m, j = 1, …p
A simple example illustrates the procedure much more clearly than the notation. Suppose we
have a 3X2 matrix, A, and a 2X3 matrix, B. Note that according to the rules of multiplication
we can do either C=AB (which will be a 3 X 3 matrix) or D=BA (which will be a 2 X2 matrix) in
this case. (Note: If B was a 2X5 matrix, for example, only AB would be defined. BA would
not be defined since the number of columns in B (5) is not equal to the number of rows in A
(3)).
Example
2 7
2 −4 5
𝑨 = [3 4], 𝑩 = [
]
1 2 7
1 5
11 6 59
−3 23
AB = [10 −4 43] and BA =[
]
15 50
7
6 40
Rules of Multiplication (where A is m X n, D is n X p and E is p X q)
1- Generally, AB ≠BA
2- (AD)E = A(DE)
3- AIn = A
4- ImA = A
5- (AD)T = DTAT
Students are encouraged to use examples for each of the rules mentioned above.
Rules of Vector Multiplication
If X is an nX1 vector (n rows , one column), then:
1- 𝑋 𝑇 𝑋 is a scalar.
2- 𝑋𝑋 𝑇 is a symmetric matrix of size n.
Example:
1
Let 𝑋 = [2]
3
Then: 𝑋 𝑇 𝑋 = 14 (verify)
and
1
𝑋𝑋 𝑇 = [2
3
2 3
4 6] notice, symmetric matrix. (Verify)
6 9
Determinant
The determinant of a square matrix is a unique number associated with that matrix. The determinant of
a matrix is usually denoted by writing the name of the matrix surrounded by two vertical bars: e.g. the
determinant of A is |𝐴|.
How is the determinant of a matrix calculated? For a 1 X1 matrix, a 2 X 2 matrix or a 3X 3 matrix, the
process is simple. However, for higher dimension matrices, the calculation is more complicated.
 The determinant of a 1X 1 matrix (a scalar) is the scalar itself.
 The determinant of a 2 X 2 matrix of the form:
𝑎 𝑐
𝐴=[
]
𝑏 𝑑
Then, the determinant of A, |𝐴| = ad-cb
𝑎 𝑏 𝑐
𝑑
𝑒 𝑓]
 The determinant of a 3 X 3 matrix of the form 𝐴 = [
𝑔 ℎ 𝑖
then: |𝐴| = (aei + bfg + cdh) - (ceg + bdi + afh)
 Higher order matrices, we will resort to advanced methods or to application software such as
Matlab®
Properties of the Determinant
There are several important properties that relate to determinants.
1- The determinant of A is equal to the determinant of its transpose |𝐴| =|𝐴𝑇 |
2- The determinant of a diagonal matrix (i.e. one whose off--diagonal elements are ALL
zero) is equal to the product of the element of the diagonals.
3- A is invertible if |𝐴| ≠0
4- If any row of a matrix is a linear combination of one or more of the other rows of the
matrix then the matrix has a determinant of zero. The same holds true for columns. So
matrices with linearly dependent rows or columns are not invertible.
5- If any row of a matrix consists only of zeros then the matrix has a determinant of zero.
Determining the Inverse of a Matrix
Any square matrix that has an inverse is said to be a non-singular matrix. A matrix that
has no inverse is said to be a singular matrix. The inverse of a n-dimensional square
matrix A is denoted A-1, if it exists. The inverse matrix has the property that AA-1 = In
and A-1A = In.
If a matrix has an inverse matrix, then the matrix must be a square matrix, i.e. only square
matrices have inverses. This does not mean that ALL square matrices have inverses, it just
means that matrices that are not square do not have an inverse matrix. (note, a pseudo inverse exists, it
is beyond the scope of this handout.)
1-
Calculating the inverse of a 2X2 matrix:
3 5
Let 𝐴 = [
], then,
4 8
2 −1.25
𝐴−1 = [
]
−1 0.75
The trick to finding the inverse of a 2X2 matrix is as follows:
a- swap the diagonal elements and change the sign od the off diagonal elements
8 −5
[
],
−4 3
b- calculate the determinant : (8)(3)-(5)(4) = 4
c- divide every element in the swapped matrix by the determinant, the result is then:
𝐴−1 = [
2 −1.25
]
−1 0.75
2- Calculating the inverse of a 3X3 matrix
This process is a little tedious. Suppose A is a 3X3 matrix and one is to find its inverse. Here are the
steps:
5 8 3
Let matrix A be, [ 2 1 7 ]
−5 2 −6
a- find the cofactors matrix, C.
The cofactors matrix is found as follows:
|𝐶11 | = |1 7 | = (1)(-6)-(7)(2) = -20
2 −6
7
|𝐶12 | = − | 2
| = -( (2)(-6)-(7)(-5) )= -23
−5 −6
|𝐶13 | = | 2 1| = (2)(2)-(1)(-5) = 9
−5 2
|𝐶21 | = − |8 3 | = -( (8)(-6)-(3)(2) )= 54
2 −6
5
3
|𝐶22 | = |
| = (5)(-6)-(3)(-5) = -15
−5 −6
|𝐶23 | = − | 5 8| = -50
−5 2
8
3
|𝐶31 | = |
| = 50
1 7
|𝐶32 | = − |5 3| = -29
2 7
5
8
|𝐶33 | = |
| = -11
2 1
Therefore, the cofactors matrix , C, is:
−20 −23
9
𝑪 = [ 54 −15 −50]
50 −29 −11
b- Find the adjoint matrix of A.
−𝟐𝟎 −𝟐𝟑
𝟗 𝑻
−𝟐𝟎 𝟓𝟒
𝟓𝟎
adj(A) = CT = [ 𝟓𝟒 −𝟏𝟓 −𝟓𝟎] = [−𝟐𝟑 −𝟏𝟓 −𝟐𝟗]
𝟓𝟎 −𝟐𝟗 −𝟏𝟏
𝟗
−𝟓𝟎 −𝟏𝟏
c- determine |𝐴|
|𝐴| = -257
d- The inverse of A is then given as:
𝐴−1 =
𝑎𝑑𝑗(𝐴)
|𝐴|
0.0778 −0.2101 −0.2000
𝐴−1 = [ 0.0895
0.0584
0.1128 ]
−0.0350 0.1946
0.0428
It is typically a good idea to verify that this is correct. AA-1 should yield the identity matrix. It is left to
the student to verify.
Properties of the Inverse
If A-1 is the inverse of a matrix A, then
−𝟏
(𝑨−𝟏 ) = A
If A-1 is the inverse of a matrix A, then (AT)-1 = (A-1)T
If A and B are both square matrices with inverse matrices A-1 and B-1. Then the
matrix C = AB has an inverse defined as C-1 = B-1A-1
D- Simultaneous Equations
Assume a set of equations is given by:
Ax = y
Where x is an unknown vector, y is the known vector, and A is the matrix of coefficients. ∃x : x is a
unique solution.
x = A-1y
Example:
−4 6
A=[
]
5 3
4
y=[ ]
1
then
−0.1429
x = A-1y = [
]
0.5714
It is left to the students to verify the correctness of the above.
E- Eigenvalues and Eigenvectors
Let A be an nXn matrix, then x is defined as the eigenvector and 𝝀 is defined as the eigenvalue. These
two operators satisfy the relationship:
𝝀𝒙 = 𝑨𝒙
The values of 𝜆 that satisfy the equation are those eigenvalues for which:
|𝝀𝑰 − 𝑨| = 𝟎
Associated with each eigenvalue, 𝝀, is an eigenvector ,x.
If A is an nXn matrix, then ∃ 𝑛 𝑒𝑖𝑔𝑛𝑒𝑣𝑎𝑙𝑢𝑒𝑠 (𝝀) and n eigenvectors.
Example:
−1 0
Let A = [
]
1 −2
then,
|𝜆𝐼 − 𝐴| = |𝜆 [1 0] − [−1 0 ]| = 0
0 1
1 −2
𝜆+1
0
|
|=0
−1 𝜆 + 2
Which yields,
(𝜆 + 1)(𝜆 + 2) = 0
𝜆2 + 3𝜆 + 2 = 0
Thus,
𝝀𝟏 = −𝟏,
𝝀𝟐 = −𝟐
The eigenvalues are therefore,
𝝀𝟏 = −𝟏, 𝝀𝟐 = −𝟐
In order to determine the eigenvectors, the following equation is applied:
(𝜆𝐼 − 𝐴)𝑥 = 0 (1)
Note, there are two eigenvectors x1 and x2.
The eigenvector, x1 ,corresponding to 𝜆1 = −1 is found as follows:
𝑥11
1 0
−1 0
0
(𝜆1 [
]−[
]) [ ] = [ ]
0 1
1 −2 𝑥12
0
0 0 𝑥11 0
[
] [ ]=[ ] (2)
−1 1 𝑥12 0
From equation (2), regardless of what the value of x11 is, the first row is satisfied.
Pick x11 =1
the second row of equation (2) results in:
- x11 + x12 = 0  x12 = x11 = 1
Therefore, for 𝜆1 = −1 the corresponding eigenvector is
1
x1 = [ ]
1
Now, the process is repeated for 𝜆2 = −2
The eigenvector, x2 ,corresponding to 𝜆2 = −2 is found as follows:
𝑥21
1 0
−1 0
0
(𝜆2 [
]−[
]) [𝑥 ] = [ ]
0 1
1 −2
0
22
−1 0 𝑥21 0
[
] [ ]=[ ] (2)
−1 0 𝑥22 0
the first row yields:
-𝑥21 =0  𝑥21 = 0
likewise, the second row yields:
-𝑥21 =0  𝑥21 = 0, same result
We notice that both equations are independent of x22. Thus, pick x22 =1.
Therefore,
0
The eigenvector associated with 𝜆2 = −2 is x2 = [ ]
1
Summary of results:
the eigenvalues are :
Λ=[
𝜆1
0
0
−1 0
]=[
]
𝜆2
0 −2
And the corresponding eigenvectors are:
1 0
]
1 1
More details on eigenvalues and eigenvectors are explored in future course
𝑥=[
Property:
If A is nXn matrix and x is the corresponding eigenvectors, then:
𝑥 −1 𝐴𝑥 = Λ
Students should apply the above equation on the results of this example to confirm correctness.
F- Rank of a matrix
Rank of a matrix is defined as the minimum number of linearly independent
vectors in the matrix.
Example
If A is a 3X2 matrix, then R(A) = 2.
If A is a 4X4 matrix with all vectors being linearly independent, then R(A) = 4.