Introduction to Matrix Algebra
George H Olson, Ph. D.
Doctoral Program in Educational Leadership
Appalachian State University
September 2010
What is a matrix?
A matrix is an ordered array of elements (usually numbers). A matrix can be a
single row of numbers, for instance:
Dimensions and order of a matrix. An r by c dimensioned matrix is a r (rows) by
c (columns) array of numbers, or symbols, and will itself be represented by an
upper-case, bold, italic symbol; e.g, A , β . For instance, a 2 by 3 matrix,
can be represented, symbolically, as a symbolic array
a
A
a
11
21
a
a
12
22
A,
a 
,
a 
13
23
or as an actual array of numbers
3 5 6
.
A

 2 4 3
In either case, the order of A is said to be 2  3. On the other hand, the matrix,
d
d
D
d

d
11
d
12
21
d
22
31
41
d
d
32
42
d 
d 
,
d 

d 
13
23
33
43
is a 4  3 matrix. That is, its order is 4  3. In general, a matrix having r rows and
c columns is an r  c matrix.
A matrix having only one column or one row is called a vector and is represented
by a lower-case, bold, italic symbol; e.g., a, . Hence the 1  4 and 3  1
matrices,
a  a
a
1
a
2

β  

 
1
2
3
3
a
4
 , and

,


are both vectors. a is a row vector;  is a column vector.
The numbers (or symbols) inside a matrix are called elements. Thus, in the first
matrix, A, given above, a11 and a23 are elements of the matrix A. Similarly, in the
second matrix, A, above, the numbers 4 and 6 are elements.
We index (or refer to) elements in a matrix by subscripts that give their row and
column locations. For, instance, in the first matrix, A, above, a23 is the element in
the second row of the third column. Similarly, in the matrix, D, above, element d41
can be found in the fourth row, first column. In general, we refer to aij as i,j’th
element of A. When referring to elements of vectors, however, the first row (or
column) is assumed. Hence, in the vector, a, above, a2 is its second element; in
, 3 is the third element. In general, when referring to elements of vectors, we
indicate the j’th element of row (i.e., 1 x c) vectors and the i’th element of
column (i.e., r x 1) vectors.
A matrix in which the number of rows (r) equals the number of columns (c) is
called a square matrix; otherwise, providing it is not a vector, it is called a
rectangular matrix.
The natural order of a rectangular matrix has more rows than columns (i.e., r >
c). Hence,
a
A  a

 a
11
12
13
a 
a ,

a 
21
22
23
a 3 x 2 matrix, is presented in its natural order.
The transpose of a matrix is obtained by interchanging its rows and columns.
Thus, the transpose of A, represented as A′ (called A prime), is
a
A  
a
a 
,

a 
a
a
11
13
12
21
23
22
a 3 x 2 matrix. As a concrete example, consider
2
5
X 
1

2
3
4
,
7

6
4
7
6
2
which, when transposed, becomes
2 5 1 2
X   4 7 6 2 .


 3 4 7 6 
The natural order of a vector is a r x 1 column vector. Here, the 4 x 1 column
vector,
a
a
a
a

a
1
2
3
4


,



Is shown in its natural order. The transpose of a,
a   a
1
a
2
a
3
a
4
,
a 1 x 4 row vector.
As mentioned earlier a matrix where r = c is a square matrix. At least two
particular types of square matrices are important, symmetric matrices and
diagonal matrices. A symmetric matrix is a square matrix in which the elements
above the main diagonal are mirror images of the elements below the main
diagonal (the main diagonal is that set of elements running from the upper lefthand corner of a square matrix to the lower right-hand corner of a square matrix.
In the symmetric matrix V, given below, the elements in bold represent the main
diagonal. Note that the elements above the main diagonal are mirror images of
the elements below the main diagonal.
56 52 38 
V   52 64 61  .


 38 61 59 
In other words, element vij = vji. Symmetric matrices have important properties.
For instance, a symmetric matrix is equal to its transpose. That is, for the matrix
just given, V = V′.
Diagonal matrices are square matrices in which all the elements except the main
diagonal are zero (0). For instance,
37 0 0 0 
 0 52 0 0 

D
 0 0 49 0 


 0 0 0 61
is a symmetric diagonal matrix.
An important diagonal matrix is the identity matrix. The identity matrix is a
diagonal matrix in which all the elements along the main diagonal are unity (1).
For example,
1 0 0 
I  0 1 0 


 0 0 1 
is a 3 x 3 identity matrix.
Matrix Operations
Addition
The addition of two matrices, A and B, is accomplished by adding corresponding
elements, e.g., aij + bij.; hence,
C  AB
a
a

a

a
11
21
31
41
a
a

a

a
11
21
31
41
c
c

c

c
11
21
31
41
a
a
12
22
a
a
32
42
a  b
a  b

a  b
 
a  b
13
11
23
21
33
31
43
41
b
a b
b
b
b
a b
a b
a b
11
21
31
41
c
c
12
22
c
c
32
42
12
12
22
22
32
32
42
42
b 
b 

b 

b 
b
b
12
13
22
23
b
b
32
33
42
43
a b 
a b 

a b 

a b 
13
13
23
23
33
33
43
43
c 
c 

c 

c 
13
23
33
43
It is obvious (or should be obvious) that the orders of the two matrices being
added need to be identical. Here, for instance, both matrices are of order 4 x 3.
When the orders of the two matrices are not identical, addition is impossible.
Addition of matrices is commutative. Hence A + B = B + A.
As a concrete example, let
7
3
A
4

1
5
1
5
3
8
1
 , and
8

4
2
1
B
4

3
0
3
1
1
3
3
.
2

3
Then, C = A + B = B + A
=
7  2
 3 1

4  4

 1 3
9
4
= 
8

4
5
4
6
4
50
1 3
5 1
3 1
8  3
1 3 

8  2

43
11
4
.
10 

7
Multiplication
Multiplication is somewhat more complicated. When multiplying matrices we
need to distinguish between multiplication by a scalar, vector multiplication, premultiplication and post-multiplication, inner-products and outer-products. But first,
lets define a scalar. A scalar is simply a single number, such as 2, -17.6, or
300.335. Symbolically, we typically use the symbol, c, to represent a scalar.
Scalar multiplication
Scalar multiplication is most easily shown by example. Given the 3 x 2matrix X
and the scalar, c, the product, cX, is given by
 x 11
cX  c  x 21

 x
cx 11
  cx 21

 cx
31
31
 x 11
  x 21

 x
 x 11 c
  x 21 c

 x c
31
31
x 
12



cx 12 
cx 

cx 
x
x
22
32
22
32
x 
12
c


x 12 c 
x c .

x c 
x
x
22
32
As a concrete example, let
4 7
X  3 0 


 1 2 
and c = 5. Then
22
32
4 7
cX  5 3 0 


 1 2 
4 7
 3 0  5


 1 2 
 20 35 
  15 0 


 5 10 
In the above expression, the product, cX, results from pre-multiplying X by the
scalar c; whereas the product Xc results from post-multiplying X by the scalar, c.
Since scalar multiplication is commutative, cX will always equal Xc. This is not
the case with matrix multiplication, however. Only in certain special cases will the
products formed by pre-multiplying and post-multiplying two matrices, X and Y be
equal. In general XY ≠ YX.
Matrix Multiplication
First, note that given two matrices, e.g., A of order n x m, and B of order p x q,
multiplication is only possible when the number of columns in the pre-multiplier is
equal to the number of rows in the post-multiplier. Hence the product, AB, is only
possible when m = p. Similarly, the product BA is only possible when q = n.
When m = p the product, C = AB will have order n x q. Similarly, when q = n, the
product, C = BA will have order p x m.
We can illustrate this by defining the two matrices,
x
x
X 
x

x
11
x
21
x
31
41
12
22
x
x
32
42
x 
x 
,
x 

x 
13
23
33
43
and
y
Y  
y
11
21
y
y
12
22
y 
.
y 
13
23
Note that X has order 4 x 3, and Y’ has order 2 x 3. Here, only the products, XY
(not XY′) and Y′ X′ (not YX) are possible. Post-multiplying X by Y, we have
x
x
XY  
x

x
x 
y y 
x x 
 y y 

x x 
  y y 
x x 
x y x y x y x y x y
x y  x y  x y x y  x y

x y  x y  x y x y  x y

x y  x y  x y x y  x y
x
11
12
13
21
22
23
31
32
33
41
42
43
11
21
12
22
13
23
11
11
12
12
13
13
11
21
12
22
21
11
22
12
23
13
21
21
22
22
31
11
32
12
33
13
31
21
32
22
41
11
42
12
43
13
41
21
42
22
x y 
x y 

x y 

x y 
13
23
23
23
33
23
43
23
where the product is of order 4 x 2. As a more concrete example, let
2
1
X 
2

1
0
3
1
3
2
1
XY  
2

1
0
3
1
3
1
1

0

2
and
Then,
1
 2 3
1 
 1 0

0 
1
2



2
 4  0 1
 2  3 1

4 1 0

2  3  2
5
6

5

7
9
5
,
6

7
 2 1 1 .

3 0 2 
Y 
6  0  3
3 0 2

6  0  0

3  0  4
a 4 x 2 matrix. The only other product possible between these two matrices is
Y′X′:
y
 
YX
y
x
y 
x
y  
 x
12
13
21
11
21
12
22
12
12
32
23
 y x y x y x

y x  y x  y x
2 1 2
 2 1 1 

 0 3 1
3 0 2  
1 1 0
11
31
22
23
22
x
x
x
21
13
12
21
x
x
x
11
y
y
11
13
13
23
13
33
x
x
x
41
42
43




y x y x y x
y x y x y x
11
21
12
22
13
23
21
21
22
22
23
23
y x y x y x
y x y x y x
11
31
12
43
21
31
22
43
13
23
33
33
y x y x y x 
y x  y x  y x 
11
21
41
41
12
22
42
42
13
23
43
43
1
3

2 
 2  0  1 2  3  1 4  1  0 2  3  2


6  0  2 3  0  2 6  0  0 3  0  4 
3 6 5 7 

,
8 5 6 7 
which is a 2 x 4 matrix.
When A and B are both square matrices of the same order, then all the products,
AB, BA, A′B, AB′, B′A, BA′, A′B′, and B′A′ are possible. Furthermore, except is
certain special cases, the matrices formed by these various products will be
different from each other. As an exercise, try computing all possible products of
the following two square matrices:
1 2 3
A   2 3 1


 0 2 1
and
 2 2 1
B  3 1 0 .


 2 1 1 
Vector Multiplication
Having learned how to compute matrix multiplication, vector multiplication is
easy. It follows the same rules of matrix multiplication except that here one of the
matrices is a vector. For instance, given the 4 x 1 column vector,
n 
n 
n   ,
n 
 
n 
1
2
3
4
y
y
and the 4 x 2 matrix, Y  
y

y
y 
y 
,
y 

y 
11
12
21
22
31
32
41
42
then the product n′Y is the only possible product. Hence,
y
y
n 
y

y
y
y
y
y
11
n Y   n
n
1
21
n
2
12
3
4
31
32
41
 n y  n y  n y  n y
1
11
2
  n y
i
i1
21
3
31
22
4
42
41






n y n y n y n y
1
12
2
22
3
32
4
42

 n y .
i
i2
To put this in concrete terms, let
n    3 1 2 5
and
3
2
Y 
2

4
1
3
.
2

4
Then
3
2
n Y  3 1 2 5 
2

4
.
1
3
  6  2  4  20 3  3  4  15  32 25
2

3
Some special cases of vector and matrix multiplication are particularly important.
For instance, let the vector, 1 be the unit vector, defined as a vector with all
elements equal to 1:
1
1

1  1

1
1
and
2
1

y   4

3
 4
Y y
1
2
3
2

0 .

1
2 
{Note, y1 and y2 are column vectors}.
Then,
1Y    y
 y   14 8 .
1
2
If we let
X x
1
1
1

x   1

1
1
2
1

3

2
2 
2
2
1

y   4

3
 4
Y y
and
1
2
3
2

0 .

1
3 
Then,
y

XY 
  xy
1
1
y
 xy
2
2
 14 9 
  31 16 .

 
Note also that
 x
X X  
 x x
1
2
1
xx
x
1
2
2
2
  n
   x
 
Inner-products and outer-products
2
x
x
2
2
2
  5 10 
  10 22 .

 
Given the two matrices,
a
A  a

 a
11
21
31
a 
a 

a 
12
and
22
b
B
b
11
21
b
b
12
22
b 
,

b 
13
23
32
Verify that the following two products are possible, AB and BA (of course, A′B′
and B′A′ also are possible). The orders of the two products are not identical. AB
has order, 3 x 3, while BA has order 2 x 2. The product AB is an outer-product
while the product BA is an inner-product. In general, whenever both an innerproduct and an outer-product exist for two matrices, the order of the innerproduct will be less than the order of the outer-product. Furthermore, in general
the inner-product will not be identical to the outer-product.
For example, let
 2 1
A  3 2 


 3 1 
and
1 2 4 
.
B

 3 2 1
Then
5 6 9 
AB  9 10 14 


6 8 13
and
 20 9 
.
BA  

15 8 
Inner- and outer-products are particularly important invector multiplication. Let
a  1 1 1 1 , b  1 1 1 ,
and
1
2
X 
4

2
2
3
3
1
3
4
.
2

1
Then
aX  9 9 10    x
i1
x
i2
x
i3
,
{Note the summation is over rows} and
6   x
9    x
Xb     
9    x
   x
4 
1j
2j
3j
4j


.



{Here the summation is over columns}.
Also, it is easy to verify that
aXb  28    x
ij
columns.
where the summation is over rows and