MTL101
Lecture #1 Matrix Algebra
Matrices are rectangular array of numbers. The numbers we deal with in this course are either
the set of real numbers (R) or complex numbers (C). For example,
a11 a12 · · · a1n
a21 a22 · · · a2n
A = (aij ) =
(1)
..
..
..
...
.
.
.
am1 am2 · · ·
amn
Definition 1.
(1) Size of a matrix A is (number of rows )×(number of columns)=m × n.
(2) If m = n then A is called a square matrix.
(3) The diagonal entries of a11 , . . . , ann are called the diagonal entries of a square matrix A.
Remark 2. Matrices with a single row (m = 1) or a single column (n = 1) are called row
vectors or column vectors.
1 0
is a 2 × 2 matrix.
Example 3.
(1)
π e
1
(2)
is a column vector and 1 0 is a row vector.
π
Definition 4. Two matrices A = (aij ) and B = (bij ) are equal if and only if
(1) they have the same size
(2) corresponding entries are equal, i.e., aij = bij for all i, j.
Definition 5. The sum of two matrices A = (aij ) and B = (bij ) of the same size m × n is a
matrix C = (cij ) of size m×n such that cij = aij +bij for all i, j. In this case we write C = A+B
Remark 6. Two matrices of different sizes cannot be added!
−4 6 2
1 2 3
Example 7. If A =
and B =
, then the sum
1 2 3
−4 6 2
−4 + 1 6 + 2 2 + 3
−3 8 5
A+B =
=
1−4 2+6 2+3
−3 8 5
Definition 8. The scalar multiplication of a matrix A with a number c ∈ R or C is a matrix
of the same size of that of A, denoted by cA, and is obtained by multiplying each entry of A by
the scalar c. That is
ca11 ca12 · · · ca1n
ca21 ca22 · · · ca2n
cA = (caij ) =
..
..
..
...
.
.
.
cam1 cam2 · · ·
camn
The negative of a matrix A, denoted by −A is defined as −A = (−1)A = (−aij )
√
√
2
2
1 2
√
√
√2
3
4
Example 9. If A =
and c = 2, then cA = 3√ 2 4 √2
e π
2e π 2
Theorem 10 (Properties of matrix addition and scalar multiplication). If A and B are m × n
matrix then
(1) A + B = B + A
(2) (A + B) + C = A + (B + C)
(3) A + 0m×n = A where 0m×n is the zero matrix all of whose entries are zero and is of size
m × n.
(4) A + (−A) = 0m×n where 0m×n is the zero matrix all of whose entries are zero and is of
size m × n.
(5) c(A + B) = cA + cB where c is a scalar.
(6) (c + d)A = cA + dA where c, d are scalars.
(7) c(dA) = (cd)A where c, d are scalars.
(8) 1A = A
Definition 11. Given a m × n matrix A = (aij ) and an r × p matrix B = (bij ) then the product
AB is defined if and only if r = n and
(1) the product
AB = (cij ) is an m × p matrix
P
(2) cij = nl=1 ail blj = ai1 b1j + · · · + ain bnj
Example 12.
a11
a11
a31
a41
a12
a12
a32
a42
a13
c11
b11 b12
a23
c21
b
b =
c31
a33 21 22
b31 b32
a43
c41
c12
c22
c32
c42
(4 × 3)(3 × 2) = (4 × 2)
Remark 13. Another way to think about matrix multiplication is to first study the multiplication of a matrix with a column vector. That is
a11 a12 · · · a1n
b1
a11 b1 + a12 b2 + · · · + a1n bn
a21 a22 · · · a2n b2 a21 b1 + a22 b2 + · · · + a2n bn
.=
.
..
..
..
..
..
.
.
. ..
.
am1 am2 · · ·
amn
bn
am1 b1 + am2 b2 + · · · + amn bn
Now we can define matrix multiplication of two matrices A and B
AB = A[b1 b2 · · · bn ] = [Ab1 Ab2 . . . Abn ]
b1i
bi2
where the matrix B is represented in terms of the column vectors bi =
...
bin
22 −2 43
42
2 −2 3 1
3
5 1
0 2 5 0 7 8 = 26 −16 14
6
Example 14. 4
−9 4 −37 −28
9 −4 1 1
−6 −3 2
Theorem 15 (Properties of Matrix multiplication).
(1) (dA)B = d(AB)A(dB) where d is a scalar(number)
(2) A(BC) = (AB)C
(3) (A + B)C = AC + BC
(4) C(A + B) = CA + CB
Sketch of Proof. Can be checked using definition Definition 11.
1
0
Definition 16. The identity matrix In×n is a square matrix
...
0
diagonals and 0 everywhere else.
Remark 17.
0 ···
1 ···
.. ..
. .
0 ···
□
0
0
with 1’s along the
..
.
1
(1) The order of multiplication of the matrices are extremely
In
important.
2 −1
4 −2
general, it is not true that AB = BA. FOr example if A =
,B =
,
1 3
−1 1
then
9 −5
6 −10
AB =
̸= BA =
1 1
−1 4
(2) In general, it is not true that AB = 0 implies that A = 0 or B = 0. (Here 0 represents
the zero matrix and the size depends on the
That
is if A = 0 then 0 has
context.
thesame
1 2
−4 6
0 0
size as that of A). FOr example if A =
,B =
, then AB =
= 0.
3 6
2 −3
0 0
(3) In general,
it is not true
that
C = 0. For example, if
AC =
BC implies A = B or −3 3
−1 2
1 3
3 9
A=
,B =
,C =
, then AC = BC =
11 −3
3 1
2 6
5 15
(4) In general, one has to be very careful using the algebraic laws for matrices. For example,
(a + b)2 = a2 + 2ab + b2 where a, b are numbers. The same may not be true for matrices,
i.e., (A + B)2 and A2 + 2AB + B 2 may not be equal.
Definition 18. The transpose of a m × n matrix A = (aij )is a n × m matrix, denoted by AT
that has the first row of A as its first column, second row of A as its second column and so on
i.e., AT = (aji ).
Theorem 19 (Properties of Matrix Transpose).
(1) (AT )T = A
(2) (A + B)T = AT + B T
(3) (cA)T = cAT where c is a scalar
(4) (AB)T = B T AT .
Sketch of Proof. These statemetns can be easily proved by using the definition of transposition.
□
Definition 20. An n × n matrix A is invertible, if there exists a matrix B such that AB =
BA = I3×3 .
Definition 21.
(1) Skew symmetric matrices are square matrices A whose transpose equals
the negative of A. That is AT = −A.
(2) Symmetric matrices are square matrices A whose transpose equals A. That is AT = A.
(3) Diagonal matrices are square matrices which can have nonzero entries only along the
main diagonal.
(4) Upper triangular matrices are square matrices which can have nonzero entries only on
and above the main diagonal.
(5) Lower triangular matrices are square matrices which can have nonzero entries only on
and below the main diagonal.
(6) If A is a square matrix, then we can define a power of A. That is An = AȦ · · · A(n times).
Definition 22. The trace of a square matrix A is the sum of the diagonal entries of A and is
denoted by tr(A).
Theorem 23. [Properties of Trace] If A is a square matrix, then
(1) tr(A + B) = tr(A) + tr(B)
(2) tr(A) = tr(AT )
(3) tr(AB) = tr(BA).
Sketch of proof. Statement (3) can be easily proved using the matrix multiplication definition
Definition 11. That is
n X
n
n X
n
X
X
tr(AB) =
ail bli =
bli ail = tr(BA)
i=1 l=1
l=1 i=1
□
Definition 24. Idempotent matrices are square matrices A which satisfy A2 = A.
Partitioned Matrices Suppose there exists two large matrices of the form
3
1 −2 1
8
2
0 −3 1
7
5
2
1
2
0
4
−1 2
−1 4
2 −2
8 −3
6 −6 7
6 −1 1
1 10 −9 2
2
7 −3 3
B= 0
A= 0
2 −5 9
0
0 −6 9
0
2
2
1 −1 7
1 −1 8
5 −1
5 −1
4
6
8
7
8
4
6
9
7
8
It can be quite cumbersome to add these two matrices directly and hence we use the method of
partitioning it into smaller pieces which aids in adding these matrices.
7
8
2
0 −3 1
3
1 −2 1
−1 4
−1 2
2
5
0
4
2
1
6 −1 1
8 −3
2 −2
A11 A12
B11 B12
6 −6 7
B= 0
2
7 −3 3
1 10 −9 2
A=
0
= A21 A22
= B21 B22
2
2
0
0
0 −6 9
2 −5 9
A31 A32
B31 B32
1 −1 8
5 −1
1 −1 7
5 −1
4
6
9
4
6
8
7
8
7
8
2
0 −3
2 and so on. Now adding these two matrices can be thought of
where A11 = −1 4
6 −1 1
adding two 3 × 2 matrices. That is
2
15
5
1 −5
7
2
5
−2 6
12 −7 8
10
−5
A11 A12
B11 B12
A11 + B11 A12 + B12
3
17 −12 5
A + B = A21 A22 + B21 B22 = A21 + B21 A22 + B22 = 0
4
2
−11
18
0
A31 A32
B31 B32
A31 + B31 A32 + B32
2 −2 15
10 −2
8 12 17
14 16
cA11 cA12
The scalar multiple cA can be done in a similar way cA = cA21 cA22 .
cA31 cA32
The multiplication of partitioned matrices mimics the usual matrix multiplication too with
certain constraints. For example, if
3
1 2
3 −1 2
4
0
4
0 3
0 2
1
−3
1
−1 7 0
A=
B
=
2 3 −4 0 −4
2
4 1
1 6
0
2 −2
0 −1 1
Since their dimensions are 4 × 5 and 5 × 3, they can be multiplied together. Now we can parition
these matrices to enable multiplication of smaller sized matrices at one time. But once we
partition the matrix A, then the column partition of A should match the row parition of B. For
example, suppose we partition A into
3 −1 2
4
0
0 2
1
−3
1
A
A
11
12
A=
2 3 −4 0 −4 = A21 A22
1 6
0
2 −2
Now we need to partition B as
1
2
3
4
0
3
B11 B12 B13
0 =
B = −1 7
B21 B22 B23
2
4
1
0 −1 −1
What we are attempting to do is multiply the paritioned A, B following the same rules for matrix
multiplication. That is we would like
(2)
A11 A12 B11 B12 B13
A11 B11 + A12 B21 A11 B12 + A12 B22 A11 B13 + A12 B23
AB =
=
A21 A22 B21 B22 B23
A21 B11 + A22 B21 A21 B12 + A22 B22 A21 B13 + A22 B23
For this above formula to work, we need to make sure that we can multiply A11 B11 , A12 B21 etc.
Thus the way we partition B is very important. The number of columns of A11 should match
the number of rows of B11 , the number of columns of A12 should match the umber of rows of
B21 and so on. Thus the only requirement is that when setting the size of the parittion is that
the column sizes of A much match thte row sizes for B.
Now using the formula in (2) we have
11 33 7
1 −4 4
AB =
22 30 9
31 11 24
MTL101
Lecture #2 System of Linear Equations
System of equations is a set of linear equations in n variables. On a plane, it represents the set
of lines in a plane. And in a 3 dimensional space it represents a set of planes.
For example, consider the set of equations
3x + 4y = 10
2x + 8y = 12
They can be represented in plane as
The solution is the point of intersection of these lines. For example, (2, 1) satisfy both the
equations, i.e., x = 2, y = 1 will satisfy both the equations above.
Definition 25. A system of equations is a collection of equations of the form
(3)
a11 x1 + a12 x2 + · · · a1n xn = b1
a21 x1 + a22 x2 + · · · a2n xn = b1
..
..
.
.
am1 x1 + am2 x2 + · · · amn xn = b1
where aij , 1 ≤ i ≤ m, 1 ≤ j ≤ n, b1 , . . . , bm are all constants (real or complex) and x1 , . . . , xn are
variables (unknowns).
Definition 26. A solution (s1 , . . . , sn ) is an ordered set of n numbers such that if we set
xi = si , 1 ≤ i ≤ n then the above system (3) is satisfied.
Definition 27. If b1 = b2 = · · · = bm = 0, then the system is called a homogeneous system of
equations. If the linear system is not homogeneous then it is called a non-homogeneous system.
0.1. Three types of solutions.
Example 28. Consider 3x + 4y = 10, 2x + 8y = 12. Mutiplying the first equation by (−2) and
adding the resulting equations with the first equation we get
−6x − 8y = −20
2x + 8y
= 12
−4x
−8
The last equations gives us the coordinate x = 2. Back substituting in either of the original
equations gives us the solution (2, 1) (the point of intersection in the plot above). This is an
example where the system has a unique solution.
Example 29. Consider 6x − 10y = 0, −3x + 5y = 8. Multiplying the second equation by 2 and
adding it to the first equation we get
6x − 10y = 0
−6x + 10y = 16
0
= 16
The eqauation 0 = 16 is false and hence our original set of equations cannot have any solution.
Thus this is an example where the system has no solutions.
Example 30. Consider 4x + 10y = 14, −6x − 15y = −21. Multiply the first equation by 3/2
and adding it to the second equation we get
6x + 15y
= 21
−6x − 15y = −21
0
=0
This tells us the second equation is the same as the first equations. In fact the second equation
is a multiple of the first equation. Thus the point of intersection of these two lines is the whole
line itself. Algebraically, we set a free parameter for one of the variables, say y = s (where s is
a real or complex number). Thus the complete set of solutions is x = 7/2 − 5/2s, y = s where s
can be any real or complex number. Notice that this is every point on the line.
Theorem 31. A solution to a system of linear equations can take the following forms:
(1) No solutions
(2) Unique solutions
(3) Infinitely many solutions.
Sketch of proof. Proof postponed to Lecture 3
□
Geometrically, the theorem can be represented in the following way. On the x − y plane, the
three types of solutions can be represented as
Here in the first figure, the lines do not intersect anywhere showing that it has no solutions. The
second figure is an example where the lines intersect at a unique point representing a unique
solution. The third figure represents a system where it intersects at every point, basically the
two lines are a multiple of one another (the lines lie on top of one another). Thus the system
has infinite number of solutions.
In a 3D space, the theorem can be represented as
The first figure represents the case where the planes do not intersect each other at all, showing
the case where the system has no solutions. The second figure represents the case where the
planes intersect at a line showing the case where the system has infinite number of solutions.
The third figure gives an example where the planes intersect at a unique point representing the
case where the system of equations have a unique solution.
0.2. Solving a system of linear equations algebraically. We use three elementary operations to simplify the system of equations which enables us to solve the system quickly. The
three main elementary operations are
(1) Interchange the position of two equations
(2) Multiply an equation by a nonzero constant c
(3) Replace an i-th equation by a sum of the i-th equation and the multiple of j-th equation.
These elementary operations do not change the solution set. This is primarily because each of
the operations can be reversed to get the original set of equations. Of course, we can easily
check that each P
elementary row operations do not change the solution.
For example let j aij xj = bj be an equation appearing in a linear system Ax = b. Multiplying
the i-th row of the augmented matrix bya nonzero constant c, the corresponding equation
s1
s2
P
P
is a solution to Ax = b, then we have
becomes j caij xj = cbj . Suppose s =
.
j aij sj = bj ,
..
sn
but this also implies j caij sj = cbj too!. Thus multiplying the i-th row by c does not change
the solution. Similarly we can show that the other elementary row operations do not change
the solution!
P
Example 32. Consider the system
2x1 − 3x2 + 10x3 = −2
x1 − 2x2 + 3x3 = −2
−x1 + 3x2 + x3 = 4
We first interchange the first two equations to get
x1 − 2x2 + 3x3 = −2
2x1 − 3x2 + 10x3 = −2
−x1 + 3x2 + x3 = 4
Next replace the second equations by the sum of the second equations and −2 multiple of the
first equation. We also replace the third equation by the sum of the third equations and the
first equation.
x1 − 2x2 + 3x3 = −2
x2 + 4x3 = 2
x2 + 4x3 = 2
Next we replace the third equation by subtracting the third equation from the second equation
to get
x1 − 2x2 + 3x3 = −2
x2 + 4x3 = 2
Thus we set a free parameter to the variable x3 to get x3 = s where s is any real or complex
number. We can now back substitute in the above equations to solve for x2 , x1 to get
x1 = 2 − 11s
x2 = 2 − 4s
x3 = s
We can now substitute this in to the original set of equations to see that they satisfy them.
2(2 − 11s) − 3(2 − 4s) + 10s = −2
(2 − 11s) − 2(2 − 4s) + 3s = −2
−(2 − 11s) + 3(2 − 4s) + s = 4
This example also shows that performing elementary operations do not change the solutions of
the linear system.
Example 33. Elementary column operations can change solution of the system!. For example
consider the solution
x + 2y + 3z = 10
4x + 5y + 6z = 11
7x + 8y + 9z = 12
Multiplying the first column by 2 we get the system
(4)
2x + 2y + 3z = 10
8x + 5y + 6z = 11
14x + 8y + 9z = 12
Using elementary operations we can show that the original system has the solution x = −(28/3)+
s, y = (29/3) − 2s, z = s where s is a free paramter. Clearly this set of solutions does not satisfy
above system (4).
We can also use the theory of matrices to perform these elementary operations. We first need
to represent the systme of equations in a matrix form.
The first form is the Ax = b where
x1
b1
a11 a12 · · · a1n
x2
b2
a21 a22 · · · a2n
A=
x=
b=
..
..
..
...
...
...
.
.
.
am1 am2 · · ·
amn
xn
bn
The second form is the augmented matrix of the system. It takes the form
a11 a12 · · · a1n b1
a21 a22 · · · a2n b2
.
..
..
..
..
..
.
.
.
.
am1 am2 · · ·
amn bm
For each linear system we can associate an augmented matrix. We can now perform the operations we mentioned in the previous example on the augmented matrix to make solving the
problem more simpler. The operations we perform on the augmented matrix is called elementary
row operations. The three elementary row operations are
(1) Interchange the position of two rows
(2) Multiply a row by a nonzero constant c
(3) Replace i-th row by the sum of the i-th row and a multiple of the j-th row.
Example 34. Consider the system
2x1 − 3x2 + 10x3 = −2
x1 − 2x2 + 3x3 = −2
−x1 + 3x2 + x3 = 4
The augmented matrix is of the form
2 −3 10 −2
1 −2 3 −2
−1 3 1 4
Now we represent each row as Ri ad use them to represent the operations we perform. For
example the first elementary row operation we perform is interchanging the first and second
row. We represent it as R1 ↔ R2 .
1 −2 3 −2
2 −3 10 −2
R1 ↔ R2
−1 3 1 4
1 −2 3 −2
0 1 4 2
R2 → R2 − 2R1 , R3 → R3 + R1
0 1 1 2
1 −2 3 −2
0 1 4 2
R3 → R3 − R2
0 0 1 0
The linear system of equations associated to the above augmented matrix is
x1 − 2x2 + 3x3 = −2
x2 + 4x3 = 2
Substituting x3 for a free parameter we get
x1 = 2 − 11s
x2 = 2 − 4s
x3 = s
We can now substitute this in to the original set of equations to see that they satisfy them.
2(2 − 11s) − 3(2 − 4s) + 10s = −2
(2 − 11s) − 2(2 − 4s) + 3s = −2
−(2 − 11s) + 3(2 − 4s) + s = 4
Remark 35. One should be careful in recogonizing elementary row operations. While it is
true that R2 → R2 − R1 is an elementary operation, but R2 → R1 − R2 is not one elementary
operation. In fact, R2 → R1 − R2 is a composite of two elementary operation R2 → (−1)R2 and
R2 → R2 + R1 .
Definition 36. Two matrices are (row) equivalent if one can be obtained from the other through
a sequence of elementary row operations. Thus (row) equivalent augmented matrices corresponds
to equivalent systems.
Equivalent matrices can also be obtained as a a conequence of sequence of elementary column
operations too. But since we are dealing with linear system of equations, we deal with only row
operations here.
Theorem 37. Two equivalent system of linear equations have the same solution.
Sketch of Proof. Equivalnet system have equivalent augmented matrices and hence have the
same set of solution. This is because the elementary row operations do not change the solution
set.
□
Definition 38. [Leading Term] For a system of equations a variable that appears in the first
term in at least one equation is called a leading variable. The coefficient of the leading variable
is called the leading term.
Definition 39. A matrix is said to be in echelon form if
(1) Every leading term is in a column to the left of the leading term of the row below it.
(2) Any zero row are are the bottom of the matrix.
1 2 3
Remark 40. The echelon form may not be unique! For example, the matrix 1 4 6 have
3 4 5
1 4
6
1 2 3
two echelon forms namely 0 2 3 and 0 −2 −3.
0 0 2
0 0 −1
Definition 41. For a matrix in echelon form the pivot positions are those that contain a leading
term. The pivot columns are those columns that contain pivot positions.
Example 42. Consider the linear system of equations
2x − 2y − 6z + w = 3
−x + y + 3z − w = −3
x − 2y − z + w = 2
The augmented matrix assocaited to this sytem is
3
2 −2 −6 1
−1 1
4 −1 −3
1 −2 −1 1
2
We now perform elementatry row operations to convert it the echelon form
2
1 −2 −1 1
−1 1
4 −1 −3
R1 ↔ R3
3
2 −2 −6 1
1 −2 −1 1
2
0 −1 2
0 −1
R2 → R2 + R1 , R3 → R3 − 2R1
0 2 −4 −1 −1
1 −2 −1 1
2
0 −1 2
0 −1
R3 → R3 + 2R1
0 0
0 −1 −3
The last matrix is in the echelon form as it satisfies both the conditions of the echelon form.
Now we can identify the pivot positions and the columns
2
1 −2 −1 1
0 −1 2
0 −1
0 0
0 −1 −3
The pivot positions are marked in red in the above system. Notice that the pivot columns here
are the first, second and the fourth column. The third column is not a pivot column. So we
assign the third variable z (associated to the non-pivot column) as a free parameter. The linear
system associated to the above echelon form is
x − 2y − z + w = 2
−y + 2z = −1
−w = −3
Since the free parameter is assigned to z, we can now back substitute to find all the solutions
w = 3, z = s, y = 1 + 2s, x = 1 + 5s.
Remark 43.
(1) The zero matrix is already in the echelon form.
(2) The free parameters are assigned to a variable corresponding to non-pivot columns.
MTL101
Lecture #3 Gaussian Elimination
The process of solving a system of linear equations by converting its augmented matrix into the
echelon form is the premise of Gaussian Elimination.
Example 44. Consider
x + 4y − 3z = 2
3x − 2y − z = −1
−x + 10y − 5z = 3
The associated augmented matrix is
1
4 −3 2
3 −2 −1 −1
−1 10 −5 3
Convert the augmented into the echelon form through the following steps
1 4 −3 2
0 −14 8 −7
R2 → R2 − 3R1 , R3 → R3 + R1
0 14 −8 5
1 4 −3 2
0 −14 8 −7
R3 → R2 + R3
0 0
0 −2
The associated linear system is of the form
x + 4y − 3z = 2
−14y + 8z = −7
0 = −2
The last equations makes no sense and hence the system of equations have no solutions.
Remark 45. If at any point of solving the augmented system into the echelon form there is a
row of the form [0 0 · · · 0|c] where c is nonzero, then the system will always have no solution.
0.3. Gauss-Jordan Elimination. The Gauss-Jordan Elimination requires one to reduce the
echelon form into a more simpler form called the reduced row echelon form using more elementary
row operations.
Definition 46. A matrix is in reduced echelon form (or reduced row echelon form) if
(1) It is in echelon form
(2) All the pivot positions are 1
(3) The only nonzero terms in a pivot columns are in the pivot positions.
In order to achieve a reduced echelon form, we first make the augmented matrix into an echelon
form and then use the following elementary row operations
(1) Multiply each nonzero row by the reciprocal of the pivot so that we end up with 1 as
the leading term in each nonzero row.
(2) Use elementary row operations to introduce zeroes in the entries abvoe the pivot positions.
Example 47. Consider the linear system of equations as in the previous lecture.
2x − 2y − 6z + w = 3
−x + y + 3z − w = −3
x − 2y − z + w = 2
The augmented matrix assocaited to this sytem is
3
2 −2 −6 1
−1 1
4 −1 −3
2
1 −2 −1 1
We can use the elementary row operations in the example to convert the above matrix into the
following echelon form
1 −2 −1 1
2
0 −1 2
0 −1
0 0
0 −1 −3
We now use more elementary row operations to satisfy the steps mentioned above. Since the
pivot position are not 1, we first use elementary row operations to produce a 1 in the pivot
positions
1 −2 −1 1 2
0 1 −2 0 1
R2 → (−1)R2 , R3 → (−1)R2
0 0
0 1 3
1 0 −5 1 4
0 1 −2 0 1
R1 → R1 + 2R2
0 0 0 1 3
1 0 −5 0 1
0 1 −2 0 1
R1 → R1 − R3
0 0 0 1 3
The associated linear system is of the form
x − 5z = 1
y − 2z = 1
w=3
Since the free parameters are assinged to non pivot columns, we have z = s and solutions
y = 1 + 2s, x = 1 + 5s, w = 3.
One of the main advantages of the reduced echelon form is the following theorem
Theorem 48. A given matrix is equivalent a UNIQUE matrix what is in reduced echelon form.
Sketch of Proof. Let A be a m × n matrix. We give a proof via induction. If n = 1, then there
is only one column and the proof is clear. Now suppose n > 1. Let A′ be the matrix obtained
bfrom A by deleting the nth column. The sequence of elementary row operations to make A into
reduced echelon form also makes A′ into reduced echelon form.
Now suppose there exists two forms of the reduced echleon forms of A, namely B, C such that
B ̸= C. Notice that B and C can differ only in the nth column (by induction hypothesis). Let
j be an integer that bjn ̸= cjn where bjn , cjn represents the j-th row and n-th column entry of
B, C respectively.
s1
s2
Consider a solution s =
... be a solution the homogeneous linear system Ax = 0. Since
sn
As = 0, we have Bs = 0 and Cs = 0. Thus (Bc )s = 0. Now the n − 1 columns of B and C
are the same, so the n − 1 columns of B − C are zero. Thus we have (bjn − cjn )sn = 0. Since
bjn ̸= cjn we have sn = 0. Thus a solution to the homogeneous linear system As = 0 as the n-th
component sn equal zero.
The n-th column of B and C must have a pivot position. For, otherwise, the n-th variable would
be a free variable which would contradict that xn = sn = 0. Now combining all these statements
together, we have B and C which have the n − 1 columns equal and the n-th column is a pivot
position in B and C. Since B and C are already in the reduced echelon form (zero rows appear
in the bottom), we see that B = C. This completes our proof.
□
Remark 49. Homogeneous linear system always have a solution. That is x1 = x2 = · · · = xn =
0 will always be a solution to a homogeneous linear system. This is called a trivial solution to the
homogeneous linear system. If there are additional solutions, then they are called a nontrivial
solution. Thus if a homogeneous linear system has a unique solution, then it has to be the trivial
solution.
Example 50. COnsider the system
2x − 6y − z + 8w = 0
x − 3y − z + 6w = 0
−x + 3y − z + 2w = 0
The augmented form of the above system is
2 −6 −1 8 0
1 −3 −1 6 0
−1 3 −1 2 0
We now compute the reduced echelon form of the above augmented system
1 −3 −1 6 0
2 −6 −1 8 0
R1 ↔ R2
−1 3 −1 2 0
1 −3 −1 6 0
0 0
1 −4 0
R2 → R2 − 2R1 , R3 → R3 + R1
0 0 −2 8 0
1 −3 −1 6 0
0 0
1 −4 0
R3 → R3 + 2R2
0 0
0
0 0
1 −3 0 2 0
0 0 1 −4 0
R1 → R1 + 2R2
0 0 0 0 0
The above is a reduced echelon form. The associated homogeneous system is
x − 3y + 2w = 0
z − 4w = 0
The free parameters are y = s1 , w = s2 and we get the complete solution z = 4s2 , x = 3s1 − 2s2 .
Sketch of proof of Theorem 31. Consider the augmented system corresponding to Ax = b and
conver it to the echelon form E. The echelon form E has three forms
(1) The system has a row of the form [0 0 · · · 0|c] where c is nonzero. The corresponding
equation is 0 = c which has no solution. If the above cannot occur, then either the
following should happen:
(2) The echelon form has no free parameter, then it has a unique solution.
(3) The echelon form has one or more free parameters and hence infinite many solutions.
□
Definition 51. The rank of a matrix M is the number of nonzero rows in the echelon form.
1 −3 0 2
Example 52. The rank of the identity matrix In×n is n. The rank of the matrix 0 0 1 −4
0 0 0 0
is 2 as the matrix is in the echelon form and has 2 nonzero rows.
Remark 53. The rank also equals the number of pivot columns. There is always one pivot
position per row in the row reduced echelon form of a matrix. Since the number of pivot
columns cannot exceed the number of variables, the rank of a matrix is also at most the number
of variables. Thus the rank A ≤ min{m, n}.
Theorem 54 (Rank-Nullity Theorem). If Ax = 0 is a homogeneous linear system of equations
with n variables , then the number of free variables equals n − rank(A).
Sketch of Proof. Consider the system Ax = 0 and its augmented matrix [A | b]. COnvert the
augmented matrix to the reduced echelon form E. The size of the augmented matrix and its
reduced echelon form are the same. Since E has only rank(A) many nonzero rows and n is the
numeber of variables in the system, then each of the nonzero row contribute to a pivot position.
The rest of the nonpivot positions contribute to the list of free variables. Thus n − rank(A) is
the number of free varibales.
□
Theorem 55. If Ax = 0 is a homogeneous linear system of equations with n variables , then
rank(A) < n if and only if Ax = 0 has a nontrivial solution.
Sketch of proof. The condition rank(A) < n is equivalent to the echelon form having free variables (Rank-Nullity Theorem). The presence of free variables give rise to nontrivial solution. □
Theorem 56. f Ax = 0 is a homogeneous linear system of equations with n variables and m
equations. If the number of equations m is less than that of n, then the system has at least one
free variable and hence the system has infinitely many solutions.
Sketch of proof. Of course, since m < n, then rank of the matrix is strictly less than n and hence
by the previous theorem, it has a free variable and hence nontrivial solution.
□
Remark 57. If a system of equations have a [0 0 · · · 0|0] in the echelon form, then it need
not always have
solutions. It can have no solutions or a unique solution. The
infinitely many
1 0 0 1
0 1 0 1
echelon form
0 0 1 3 of a system of equations have unique solution. The echelon form
0 0 0 0
1 0 0 1
0 1 0 1
0 0 0 3 has no solution.
0 0 0 0
MTL101
Lecture #4 Elementary Matrices
We first start with an example
1 1
Example 58. Suppose A = 6 −2
4 5
1
R1 → R2 +2R1 , we get the matrix 12
4
−1 −2
0
1 . If we perform an elementary row operation
1 −1
1 −1 −2
0 −2 5 . Now we perform elementary row operation
5 1 −1
1 0 0
1 0 0
R1 → R2 + 2R1 on the identity matrix 0 1 0, we the matrix E = 2 1 0. In fact the
0 0 1
0 0 1
matrix we get after performing the row operation on A is the same as the product EA!
1 0 0 1 1 −1 −2
1 1 −1 −2
1 = 12 0 −2 5
EA = 2 1 0 6 −2 0
0 0 1 4 5
1 −1
4 5 1 −1
Definition 59. If we perfrom a single elementary row operation on an identity matrix Im×m ,
then the result is called an elementary matrix.
Example 60. Some types of elementary matrices are
Row Operation
R3 → R3 − 4R1
R2 → 5R2
R2 ↔ R1
Elementary matrix
1 0 0
0 1 0
−4 0 1
1 0 0
0 5 0
0 0 1
0 1 0
1 0 0
0 0 1
The first example is codified in the following theorem
Theorem 61. Let E be an elementary matrix obtained by performing an elementary row
operation R on the identity matrix Im×m . If we perform the same elementary row operation R
on a m × n matrix A, then the resulting matrix is the product EA.
Sketch of proof. Supose E is an elementary matrix obtained by performing the interchange row
operation Ri ↔ Rj on the identity matrix. Now we perform the same operation on the marix
A. That is, the row i and j are interchanged. Then the matrix becomes
a11 · · · a1n
a11 · · · a1n
..
..
..
..
..
..
.
.
.
.
.
.
ai1 · · · ain
aj1 · · · ajn
.
.
..
..
..
..
A=
.
.
.
.
..
−→ ..
= EA
a
j1 · · · ajn
ai1 · · · ain
.
..
..
..
..
..
...
.
.
.
.
am1 · · · amn
am1 · · · amn
Similarly we can check that the other types of elementary matrices also have the same property.
□
−3 −2 3
Example 62. Let A =
. Consider the three elementary row operations being
1
0 −2
applied
(1) R1 ↔ R2
(2) R2 → R2 + 3R1
(3) R2 → −2R2 .
1 0 −2
If we perform all these row operations on A, then the resulting matrix is of the form
.
0 4 6
The elementary matrices which corresponds to the row operations above are
0 1
1 0
1 0
E1 =
E2 =
E3 =
1 0
3 1
0 −2
1 0 −2
Then we see that the matrix
can be recovered by multiplying A with the elementary
0 4 6
matrices E1 , E2 , E3 sequentially.
1 0 −2
1 0
1 0 0 1 1 0 −2
=
= E3 E2 E1 A
0 4 6
0 −2 3 1 1 0 0 4 6
Theorem 63. Equivalent matrices have the same rank.
Sketch of proof. If two matrices A, B are equivalent then there exists elementary matrices E1 , . . . , Ep
such that Ep · · · E1 A = B. Now suppose we convert B to the row reduced echelon form, then
we have elementary matrices E1′ , . . . , Eq′ such that Eq′ · · · E1′ B = C (where C is the unique row
reduced echelon form of B). Now Eq′ · · · E1′ Ep · · · E1 A = Eq′ · · · E1′ B = C. Since the row reduced
echelon form are unique, the row reduced echelon form of A is also C. Thus the number of
nonzero rows of C serves are the rank of A and B.
□
Theorem 64. Equivalent linear system of equations have the same set of solutions.
Sketch of proof. Consider two equivalent systems Ax = b and A′ x = b′ . Since they are equivalent, their respective augemented matrices are equivalent. Thus there exists elementary matrices
E1 , . . . , Ep such that Ep · · · E1 [A | b] = [A′ | b′ ]. In order to find the solutions of A′ x = b′ we
convert the augmented matrix [A′ | b′ ] into the row reduced echelon form using elementary
matrices E1 , . . . , Ep . Thus Ep · · · E1 [A′ | b′ ] = [A′′ | b′′ ] where the latter matrix is in the row
reduced echelon form. Since the system of equations are equivalent and the row reduced echelon
form is unique, it follows that [A′′ | b′′ ] is also the row reduced echelon form of [A | b]. Thus
the solutions of the equivalent system depends only on the augmented matrix [A | b] and hence
the solutions are the same.
□
Theorem 65. The system of linear equations Ax = b has a solution if and only if rank A =
rank [A | b]. Moreover the system has unique solution if and only if rank A equals the number
of variables in the system.
Sketch of proof. Since equivalent system have the same set of solutions, assume that the augmented matrix [A | b] is in the reduced echelon form.
Suppose the system has a solution but the ranks rank A ̸= rank [A | b]. Then the matrix [A | b]
has a row of the form [0 · · · 0 | bj ]. Then the j-th equation of this system is 0 = bj which doesnt
have any solutions. This contradiction forces rank A = rank [A | b].
Now suppose rank A = rank [A | b]. Then there exists no rows of the form [0 · · · 0 | bj ] where
bj ̸= 0. Since the system can have only three types of solutions and it cannot have no solution,
it has to have a unique solution or infinitely many solutions.
Having a unique solution is equivalent to the property of the system having no free variables. □
1 0 0
Example 66. Elementary matrices are invertible!. For example let E = 0 1 0 be an
−4 0 1
elementary matrix corresponding to the elementary row operation R3 → R3 − 4R1 . Now we
perform the exact “opposite”
operation
R3 → R3 + 4R1 on the identity matrix to construct the
1 0 0
elementary matrix E ′ = 0 1 0. Notice that
4 0 1
1 0 0
1 0 0
1 0 0
E ′ E = 0 1 0 0 1 0 = 0 1 0 = I3×3
4 0 1 −4 0 1
0 0 1
Similary we can show that EE ′ = I3×3 .
Theorem 67. Elementary matrices are invertible.
Sketch of proof. Suppose E is the elementary matrix obtained by interchanging row i and row
j of the identity matrix In×n . Then we can show that EE = In×n . This shows that E is its
inverse. Similarly we can show that the elementary matrix E corresponding to each type of
elementary row operation are invertible, by constructing a matrix E ′ obtained by performing
the exact “opposite” row opertaion on the identity matrux. In fact EE ′ = E ′ E = I showing
the elementary matrices are invertible.
□
In fact the inverse of elementary matrices are invertible too!
Theorem 68. Suppose A is a n × n matrix. Then the following statements are equivalent:
(1) A is invertible
(2) the linear system Ax = b has a unique solution for all vectors b
(3) the homogeneous linear system Ax = 0 has only the trivial solution
(4) A can be row reduced to an identity matrix.
(5) A can be written as a product of elementary matrices.
Sketch of proof. (1) ⇒ (2): Suppose A is invertible, then there exists a matrix B such that
AB = BA = I. Now multiply B on both sides of the equation Ax = b, we get BAx = Bb
giving a unique solution x = Bb.
(2) ⇒ (3) is clear.
(3) ⇒ (4): Suppose Ax = 0 has only the trivial solution. Let U be the reduced echelon form
of A. Since there is only one solution to the above system, there is only one solution to the
homogeneous system U x = 0. Since U is a n × n matrix and has a unique solution, each column
of U is a pivot column and has n leading variables. Thus U = In×n . Thus A can be reduced to
a identity matrix through a series of elementary row operations.
(4) ⇒ (5): Suppose A can be row reduced to an identity matrix. Let Ei be the elementary
matrix correspoding to each of the elementary row operation applied on A to get the identity
matrix. Then Ek · · · E2 E1 A = In×n . Since the elementary matrices are invertible, we see that
A = E1−1 · · · Ek−1 . Since the inverse of elementary matrix is also elementary, the result follows.
(5) ⇒ (1) Let B = Ek · · · E2 E1 , then clearly BA = In×n .
□
0.4. Computing the inverse of A. The above theorem enables us to construct inverses for a
square matrix. For example, suppose A is an invertible matrix, then there exists B such that
AB = BA = I. Let
a11 a12 · · · a1n
a1i
a21 a22 · · · a2n a2i
A=
= a1 · · · an
where ai =
..
..
..
...
...
.
.
.
an1 an2 · · ·
ann
ani
Similary, let
b1i
b2i
where bi =
...
B = b1 · · · bn
bni
0
..
.
0
where ei = 1
0
.
..
0
In×n = e1 · · · en
were 1 appears in the i-th row of ei . Now since AB = I, we have
Ab1 = first column of the matrix AB = e1
Ab2 = second column of the matrix AB = e2
....
..
Abn = nth column of the matrix AB = en
The above statements imply that
b1 is a solution to the linear systemAx = e1
b2 is a solution to the linear systemAx = e2
..
.
bn is a solution to the linear systemAx = en
The above can also be obtained by converting the augmented matrix [a1 · · · an | ei ] of each
of the system Axi = ei into the row reduced echelon form. Thus the solution to the system
corresponding to the reduced echelon form of the augmented matrix [a1 · · · an | ei ] is bi . This
means that the reduced echelon form of the augemented matrix will of the form [In×n | bi ]. We
can combined all these computations together by perform the row opertaions on one matrix
[a1 a2 · · · an | e1 e2 · · · en ] into a row reduced echelon form (row reduced echelon forms are
unique!). The matrix which appears on the right will be the inverse matrix B.
elementary row operations
[A | In×n ] −−−−−−−−−−−−−−→ [In×n | B]
This matrix B satisfies AB = BA = In×n .
1 3
Example 69. Let A =
. Then
2 5
[A | I2×2 ] =
1 3 1 0
2 5 0 1
Now we perform elementary row operations on the above matrix to make the left part of the
matrix into an identity matrix.
1 3 1 0
2 5 0 1
1 3
1 0
0 −1 −2 1
1 3 1 0
0 1 2 −1
1 0 −5 3
0 1 2 −1
R2 → R2 − 2R1
R2 → (−1)R2
R1 → R1 − 3R2
The matrix is of the form [I2×2 | B]. We can easily check that AB = BA = I2×2 .
1 −2 1
Example 70. Suppose A = −3 7 −6. Then we perform elementary row operations on
2 −3 0
the matrix [A | I3×3 ] to get
0 0
1 0
0 1
0 0
1 0
0 1
1
0 0
1 −2 1
0 1 −3 3
1 0
0 0
1 −5 −1 1
1 −2 0 6
1 −1
0 1 0 −12 −2 3
0 0 1 −5 −1 1
1 0 0 −18 −3 5
0 1 0 −12 −2 3
0 0 1 −5 −1 1
1 −2 1 1
−3 7 −6 0
2 −3 0 0
1 −2 1
1
0 1 −3 3
0 1 −2 −2
We can easily verify that AB = BA = I.
R2 → R2 + 3R1 , R3 → R3 − 2R1
R3 → R3 − R2
R2 → R2 + 3R3 , R1 → R1 − R3
R2 → R2 + 3R3 , R1 → R1 + 2R2
1
1 −2
1 −3. Then we perform elementary row operations on
Example 71. Suppose A = 2
−3 −1 4
1
1 −2 1 0 0
1 −3 0 1 0 to get
the matrix 2
−3 −1 4 0 0 1
1 1 −2 1 0 0
0 −1 1 −2 1 0
R2 → R2 − 2R1 , R3 → R3 + 3R1
0 2 −2 3 0 1
1 1 −2 1 0 0
0 −1 1 −2 1 0
R3 → R3 + 2R2
0 0
0 −1 2 1
But since the left part of the matrix can never be made into an identity matrix through any
elementary row operations (due to the presence of a zero row), Theorem 68 tells us that the
matrix A is NOT invertible!
MTL101
Lecture #5 Determinants and Cramer’s Rule
Definition 72. An n × n matrix A is invertible, if there exists a matrix B such that AB = In .
Theorem 73. Suppose A is invertible with AB = In . Then BA = In too.
Proof. Suppose AB = I. Now consider AB(Ax) = I(Ax) = Ax. Now AB(Ax) = A(BAx) =
Ax. Thus A(BAx − x) = 0. Now using Theorem 68, we see that BAx − x = 0 (as A is
invertible). Since BAx = x for all x, we get BA = I.
□
Remark 74. The matrix B is unique! Suppose there exists B and C such that AB = BA =
AC = CA = I, then consider B(AB) = B(AC) which implies (BA)B = (BA)C which gives us
the conclusion B = C.
Definition 75. If a n × n matrix A is invertible, then A−1 is called the inverse of A and denote
the unique n × n matrix by A−1 such that AA−1 = A−1 A = I.
Remark 76. An invertible matrix A is also called non-singular. If A is does not have an inverse,
then it is called singular.
Theorem 77 (Properties of Inverses). Suppose A is an n × n invertible matrix,
(1) (A−1 )−1 = A
(2) If B is also n × n invertible then AB is also invertible and (AB)−1 = B −1 A−1 .
(3) If C and D are n × m matrices, then AC = AD implies C = D.
(4) If AC = 0 then C = 0 where 0 represents the zero matrix.
Sketch of Proof. All the statements use the uniqueness of the inverse! To show (1), it is enough to
notice that A(A−1 ) = A and hence (A−1 )−1 = A. To show (2), AB(B −1 A−1 ) = A(BB −1 )A−1 =
AA−1 = I. Hence by the uniqueness of inverses, (AB)−1 = B −1 A−1 . To show (3), mutiply A−1
on both sides of AC = AD, to get C = D. (4) follows easily from (3).
□
0.5. Determinants. The main inspiration behind determinants is to associate a number to a
matrix which enables one to decide the invertible nature of the matrix.
We start with a 1 × 1 matrix [a] then it is invertible with inverse [a−1 ] as long as a is nonzero.
Thusthe nonzero
nature of a determines the invertibility
of [a]. Moving onto the 2 × 2 matrix
a b
a b
1
A=
, then the inverse is given by A−1 = ad−bc
. Thus the inverse makes sense if
c d
c d
the number ad − bc is nonzero. Thus the determinant of a 2 × 2 matrix is ad − bc.
For a 3 × 3 matrix A it is invertible, if A can be converted in to an identity matrix through a
series of reduced echelon form. So we take a 3 × 3 matrix and perform a row operation
a11 a12 a13
a11
a12
a13
a21 a22 a23 → 0 a11 a22 − a12 a21 a11 a23 − a13 a21
a31 a32 a33
0 a11 a32 − a12 a31 a11 a33 − a13 a31
assuming a11 is nonzero. The above system has a unique solution as long as the sub-system
a11 a22 − a12 a21 a11 a23 − a13 a21
a11 a32 − a12 a31 a11 a33 − a13 a31
has a unique solution. The 2×2 matrix determinant is already computed above. Thus the above
matrix is invertible, as long as the determinant (a11 a22 − a12 a21 )(a11 a33 − a13 a31 ) − (a11 a32 −
a12 a31 )(a11 a23 − a13 a21 ) is nonzero. Thus we can define the determinant of a 3 × 3 matrix
a11 [(a11 a22 − a12 a21 )(a11 a33 − a13 a31 ) − (a11 a32 − a12 a31 )(a11 a23 − a13 a21 )].
In general, the determinant of a n × n matrix can be reduced to a computation of a 2 × 2 matrix.
We use a more streamlined version for computing determinant.
Definition 78. Let A be a n × n matrix
a11 a12 · · ·
a21 a22 · · ·
A=
..
..
...
.
.
an1 an2 · · ·
a1n
a2n
..
.
ann
Then the determinant of A is given by
det(A) = a11 C11 + a12 C12 + · · · + a1n C1n
where C11 , . . . , C1n are the cofactor matrix of a11 , . . . , a1n respectively. The cofactor Cij =
(−1)i+j det Mij where Mij denote the (n − 1) × (n − 1) matrix obtained from A by deleting the
row i and column j. Mij is called the minor of aij .
2 −1 3
Example 79. Let A = 1 4 0 Then the determinant of A is give by the formula a11 C11 +
3 1 2
a12 C12 + a13 C13 . The cofactor are
1+1
C11 = (−1)
4 0
1 0
1 4
1+2
1+3
det
= 8 C12 = (−1) det
= −2 C13 = (−1) det
= −11
1 2
3 2
3 1
Thus det A = a11 C11 + a12 C12 + a13 C13 = 2(8) + (−1)(−2) + 3(−11) = 15.
Example 80. det In×n = 1
Theorem 81 (Determinant expansion through different row and column). Let A be a n × n
matrix
(1) det A = ai1 Ci1 + · · · + ain Cin (Expansion along row i).
(2) det A = a1j C1j + · · · + anj Cnj (Expansion along column j).
The original definition of the determinant is an expansion along row 1.
−2 1
4 −1
1
0 −1 2
. The presence of zeroes in the row 4 or column 2
Example 82. Let A =
5 −1 2
1
0
0
3 −1
will enable us compute the determinant in a simpler fashion. If we had expanded along the first
row, then the number of computations will increase.
det A = a12 C12 + a22 C22 + a32 C32 + a42 C42
Since a22 = a42 = 0, we have a reduced version of the determinant formula det A = a12 C12 +
a32 C32 . Now
1 −1 2
1 = −20
C12 = (−1)1+2 det 5 2
0 3 −1
Then det A = 1(−20) + (−1)(−11) = −9.
−2 4 −1
C32 = (−1)3+2 det 1 −1 2 = −11
0
3 −1
−1
0
Example 83. Let A =
0
0
0
column 1 repeatedly
2 7 −5 8
3 4
1 −9
0 −2 4 11
be a diagonal matrix. Then expanding along
0 0 −4 5
0 0
0
1
3 4
1 −9
−2 4 11
0 −2 4 11
−4
5
det A = (−1) det
0 0 −4 5 = (−1)3 det 0 −4 5 = (−1)(3)(4) det 0 1
0
0 1
0 0
0
1
= (−1)(3)(4)(−4)(1) = −24
The above method can be codified in the following theorem
Theorem 84. If A is diagonal, upper triangular or lower triangular, then the det A is the
product of the terms along the diagonal. (det A = a11 a22 · · · ann ).
Theorem 85. If A is a n × n matrix, then det AT = det A.
Sketch of proof. Expanding the matrix A along the first row is the same as the expansion the
matrix AT along the first column.
□
Theorem 86. Let A be a n × n matrix.
(1) If A has a row or column of zeroes then det A = 0.
(2) If A has two identical rows or columns then det A = 0.
Theorem 87 (Effect of elementary row opertaions on determinant function). Suppose A is a
n × n matrix. Then
(1) Suppose B is produced by interchanging any two rows of A, then det A = − det B.
(2) Suppose B is produced by multiplying a row of A by a nonzero constant c. Then
det A = 1c det B.
(3) Suppose B is produced by adding a multiple of one row of A to another, then det A =
det B.
Sketch of proof. We give a proof of (1) in 3 × 3 matrices. The proof then can be scaled to the
n × n case.
Supose we interchange the first two rows of the matrix A we get
a11 a12 a13
a21 a22 a23
A = a21 a22 a23 −→ A′ = a11 a12 a13
a31 a32 a33
a31 a32 a33
Using the determinant formula on A we get det A = a11 (−1)1+1 det M11 + a12 (−1)1+2 det M12 +
′
a13 (−1)1+3 det M13 . We can expand A′ along the second row to get det A′ = a11 (−1)2+1 det M21
+
2+2
′
2+3
′
a12 (−1) det M22 + a13 (−1) det M23 . Notice that in the computation of the minors, we have
′
′
′
M21
= M11 , M22
= M12 , M23
= M13 . Thus det A = − det A′ . Now we can extend this idea for
the interchange of row 1 and row 3 too. This is the composition of three row opertaions, first
interchanging row 2 and row 1, then row 1 and row 3 and then row 3 and row 2. Thus there are
three row opertations involved changing the sign.
The proof for the other types of elementary operations is similar.
□
−2 1
4 −1
1
0 −1 2
. To compute the determinant we use above theoExample 88. Let A =
5 −1 2
1
0
0
3 −1
rem and elementary row operations.
1
0 −1 2
−2 1
4 −1
R1 ↔ R2
A1 =
5 −1 2
1
0
0
3 −1
Then det A = − det A1 . Applying elementary row operations on A1 we get
1 0 −1 2
0 1 2
3
A2 =
R2 → R2 + 2R1 , R3 → R3 − 5R1 , R3 → R2 + R3
0 0 9 −6
0 0 3 −1
Since det A1 = det A2 , we have det A = − det A2 . Next we apply row operations on A2
1 0 −1 2
0 1 2
3
A3 =
R3 → −1/3R3
0 0 −3 2
0 0 3 −1
Thus det A2 = −3 det A3 and hence det A = 3 det A3 . Performing one more operatio on A3 we
get
1 0 −1 2
0 1 2 3
R4 → R4 + R3
A4 =
0 0 −3 2
0 0 0 1
Finally we have det A3 = det A4 which results in det A = 3 det A4 . Since A4 is triangular, we
get det A = 3(1)(1)(−3)(1) = −9.
Theorem 89. Let A be an n × n matrix. Then A is invertible if and only if det A ̸= 0.
Sketch of proof. Since the elementary row operations do not change the non-zero nature of determinant (Theorem 87), we have that det A is nonzero if A is invertible. Conversly suppose the
determinant is nonzero, then perform elementary row operations on A and reach the reduced
echelon form. Since the elementary row operations do not change the nonzero nature, the determinant of the reduced echelon form is also nonzero. Thus the reduced echelon form has no row
of zeroes and hence the reduced echelon form is the identity matrix. Thus A is invertible. □
Theorem 90. If A and B are n × n matrices. Then det(AB) = det A · det B.
Lemma 91. If B is a n×n matrix and E is an elementary matrix, then det(EB) = det E ·det B.
Sketch of the proof of the Lemma. Suppose E is a matrix obtained by interchanging two rows
of the identity matrix. Then from Theorem 87 we know that det E = − det In×n = −1. But
EB is the matrix obtained by performing the interchanging of the same rows on the matrix B
(Theorem 61). Thus det(EB) = − det B (Theorem 87). Thus we have det(EB) = − det B =
det E ·det B.Similarly we can show that result is true for other types of elementary matrices. □
Sketch of the proof of the theorem. If A is not invertible, then clearly AB is not invertible. For,
if A is not invertible, then after a series of elementary row operations A cannot be made into
an identity matrix. That is, the echelon form of A has a row of zeros. Thus after mutiplying a
series of elementary matrices Ek · · · E1 A has a row of zeroes. But Ek · · · E1 AB also has a row
of zeroes, whose determinant will also be zero. Thus AB is not invertible.
Now suppose A is invertible. Then after multiplying by a series of elementary matrices, we get
the identity matrix. That is Ek · · · E2 E1 A = In×n . Thus A = Ek−1 · · · E2−1 E1−1 . Thus we have
−1
det(AB) = det(Ek−1 · · · E2−1 E1−1 B) = det Ek−1 det(Ek−1
· · · E2−1 E1−1 B)
= det(Ek−1 · · · E2−1 E1−1 ) det B = det A · det B
where the decomposition in the above steps make repeated use the above lemma.
□
Remark 92. It is not in general true that det(A + B) = det A + det B.
Theorem 93. If A is an invertible n × n matrix, we get det A−1 = det1 A .
Sketch of proof. Since AA−1 = In×n , we get det(AA−1 ) = det A · det A−1 = 1 and hence the
result follows.
□
Definition 94. Let C = (Cij ) where Cij is the cofactor corresponding to aij . C is called the
Cofactor matrix. The transpose of the cofactor matrix is called the adjoint matrix adj(A).
1
Theorem 95. If A is an invertible matrix then A−1 = det(A)
adj(A).
Sketch of the proof.
a11 a12 · · ·
a21 a22 · · ·
A adj(A) =
..
..
...
.
.
an1 an2 · · ·
a1n
C11 C21 · · ·
a2n C12 C22 · · ·
.
..
..
..
. ..
.
.
ann
C1n C2n · · ·
Cn1
det A
0
···
Cn2 0
det A · · ·
= .
..
..
..
. ..
.
.
Cnn
0
0
···
0
0
..
.
det A
Notice that the product of the i-th row of A and the i-th column of adj(A) is the routine formla
of the determinant. But the product of i-th row of A and j-th column of adj(A) with i ̸= j
equals zero as the product is the determinant of the matrix obtained by replacing the j-row of
A with the i-th row. But the determinantof the matrix with
two equal rows are zero.
a11 a12 · · · a1n
a11 a12 · · · a1n
For example, a11 C21 + · · · + a1n C2n = det
=0
□
..
..
..
...
.
.
.
an1 an2 · · ·
ann
Definition 96. Let A be a m × n matrix. Then the number of rows/columns of the largest
square submatrix of A that has a nonzero determinant is called the determinantal rank.
Theorem 97. The rank of a m × n matrix A and the determinantal rank of A are equal.
Sketch of proof. Suppose rank A = r. Then in the row reduced echelon form, the matrix A
has r nonzero rows. Let B is the row reduced echelon form of A. Now there exists elementary
matrices E1 , . . . , Ep such that Ep · · · E1 A = B. Thus the submatrix of B corresponding to the
first r rows and the pivot columns will have nonzero determinant (as it is the identity matrix).
Let C be the submatrix of A with corresponding the rows and columsn chosen above. Then
E1 · · · Ep C = Ir×r . Thus the det C ̸= 0. Thus the determinantal rank if at least r.
Now suppose determinantal rank of A is r. Then there is a submatrix C of size r × r whose
determinant is nonzero. Without loss of generality assume that the rows of the matrix C are
from the first r rows of A and the columns be c1 , . . . , cr of A. Thus there exists elementary
matrix E1′ , . . . , Ep′ such that Ep′ · · · E1′ C = Ir×r . Now for 1 ≤ i ≤ p, define
0r×n−r
Ei
Ei =
0n−r×r In−r×n−r
Then Ep · · · E1 A will be a matrix which have 1 along the first r rows and the columns c1 , . . . , cr .
Now in the first r rows and columns c1 , . . . , cr it will have a 1 in the pivot positions and zero
everywhere else. Since this matrix has at least r nonzero rows, the row reduced echelon form
of A will have at least r nonzero rows (think!). Thus the rank is at least r. This finishes the
proof.
□
0.6. Cramers Rule.
Theorem 98. Suppose Ax = b is a linear system of equation and A is an n × n invertible
Ai
matrix, then the components of the unique solution to Ax = b is given by xi = det
where Ai
det A
is the matrix obtained from the matrix A after replacing i-th column by b.
Sketch of proof. Consider the system Ax = b. Let In,i denote the n × n identity matrix with
x1
the i-th column replaced by x = ... . Thus
xn
AIn,i = A[e1 · · · ei−1 x ei+1 · · · en ]
= [Ae1 · · · Aei−1 Ax Aei+1 · · · Aen ]
= [a1 · · · ai−1 b ai+1 · · · an ] = Ai
Ai
Thus det(AIn,i ) = det Ai . Since det(AIn,i ) = det A det An,i = det xi . Thus xi = det
.
det A
□
Example 99. Consider the system
3x + y = 5
−x + 2y + z = −2
−y + 2z = −1
5
3
1 0
Consider A = −1 2 1 and b = −2. We can check that the determinant det A ̸= 0.
1
0 −1 2
Now compute
5
1 0
3
5 0
3
1
5
A2 = −1 −2 1
A3 = −1 2 −2
A1 = −2 2 1
1 −1 2
0
1 2
0 −1 1
Using the Cramer’s rule, we get the unique solution
det A1
28
det A2
1
x1 =
=
x2 =
=
det A
17
det A
17
x3 =
det A3
−8
=
det A
17
MTL101
Lecture #6 Vector Spaces
Now that we have learnt to study the homogeneous system of equatoins Ax = 0, the question
now becomes if the solutions are related to each other. If the system has more than one solution
say x1 , x2 , one can immediately see that x1 + x2 is also a solution. Further if we multiply a
solution x1 by a constant c, cx1 is still a solution to the system Ax = b. Now is it possible to
get the set of solution to have a nice property. Further what does the set of solutions of the
system signify geometrically. For example take the case when two planes intersect at a line. The
intersection line becomes the set of solutions of the system of two planes! So in general, does
the set of solutions (assuming it has a solution) always be a line? a plane? or the whole space?
In the so called 3D space, it does seem intuitive that this should be true. But what about the
higher dimensional space? What is meant by dimension? As it happens in mathematics, we
abstract out the varied required notions to study objects more precisely. In this case, we require
the study of abstract objects with the required properties is called vector spaces.
Definition 100. Let V be a set whose members are called vectors. Let F be a field (assume for
the time being that it is R or C). We impose two structures on V , namely, the vector addition
and the scalar multiplication.
The vector addition is a binary operation, i.e., a map + : V × V → V . If v1 , v2 ∈ V , then
under the binary operation +(v1 , v2 ) = v3 . The vector v3 is what we called the vector addition
of v1 , v2 . We represent v3 as v1 + v2 .
The scalar multiplication is also a binary operation · : F × V → V . If c ∈ F, v ∈ V , then
·(c, v) = u ∈ V . The vector u is called the c- scalar multiple of v and we denote u = c · v. We
often supress the · and simply write cv to denote scalar multiplication.
The following properties are satisfied for a vector space
(1) v + w = w + v for vectors v, w ∈ V (commutative law of vector addition)
(2) u + (v + w) = (u + v) + w (associative law of vector addition)
(3) there is a unique vector 0 ∈ V such that 0 + v = v for all v ∈ V . The vector 0 is called
the zero vector (the additive identity)
(4) For every vector v ∈ V there is vector w ∈ V such that v + w = 0. This vector can be
proved to be unique (check!). Thus the vector w, denoted by −v is the additive inverse
of v.
(5) for v ∈ V, c, d ∈ F, (cd)v = c(dv) = (dc)v = d(cv) (associated and commutative law of
scalar multiplication rolled into one).
(6) v ∈ V, c, d ∈ F., (c + d)v = cv + dv
(7) v, w ∈ V, c ∈ F, c(v + w) = cv + cw
(8) 1 ∈ F, v ∈ V 1v = v
The first 4 properties show that V has a structure of a group with the binary operation +. The
rest of the properties show how both the operations +, · interact with each other. In (3), the
existence of the zero vector will automatically prove its uniqueness (Check!).
a1
Example 101.
(1) Let V be the set of column vectors ... where ai ∈ R. Let the vector
an
addition be defined as
a1
b1
a1 + b 1
... + ... = ...
bn
an + b n
an
and the scalar multiplication by
a1
ca1
.
c .. = ...
an
can
Then it can be easily checked that this forms a vector space. We usually represent this
space by Rn where n is the number of rows in the column vector. This also has a nice
2
geometric structure.
For example R can be taught of the plane with x and y axis. That
x
is an element
∈ R2 can be thought of as a point (x, y) in the plane. Similarly R3 is
y
the 3D space.
(2) Let A be a matrix. Consider the homogeneous system Ax = 0. A solution to this system
acan be viewed as a column vector. Let V be the set of solutions to this sytem. Then we
can show using the same vector addition, scalar multiplication as above, that V forms a
vector space. You will see later that this is called the nullspace of A.
(3) Let V be the set of all m × n matrices with entries in R. Then the vectors of V are
matrices. The vector addition is the matrix addition and the scalar multiplication is the
multilication of the matrix by a scalar. With these definitions we can se that V forms a
vector space. This space is usually denoted by Mm×n (R).
(4) Let V be the set of all continuous function f [0, 1] → R. Then The vector addition
corresponds to addition of two functions and the scalar multiplication corresponds to
multiplying the function by a nonzero real number. Then V also forms a vector space.
This space is usually denoted by C[0, 1]. One can improve this set by considering the set
of all differentiable or integrable functions too!
(5) Let V be the set of all functions f : N → R. Then we can easily show that V forms
a vector space. In fact, this is the vector space of all sequences. This is denoted by
Map(N, R).
(6) Let V be the set of all polyomials of degree at most n. Then V also forms a vector
spaces. This space is usually denoted by Pn .
(7) The set of all taylor series expansions around x = 0 also forms a vector space.
Remark 102.
(1) We will see later that the first four examples are of the same type. In
fact examples (2),(3),(4) are subset of example (1).
(2) In example (2), if the matrix A is invertible then V = {0}. This is called the trivial
space or the zero space.
(3) In each of these examples, we can replace R by C to get C-vector spaces.
(4) The field can play an important role. For example, it is clear that Rn is a R-vector
space (or a vector space over R). But it is not true that Rn is an C-vector space. For,
a1
the scalar multiplication is not closed. That is, for a vector ... ∈ Rn , and c ∈ C,
an
a1
ca1
c ... = ... ̸∈ Rn .
an
can
Definition 103. Let V be a vector space over F. Let W ⊆ V . Now W is subspace of V if
(1) 0 ∈ W
(2) u + v ∈ W for all u, v ∈ W
(3) cu ∈ W for all u ∈ W, c ∈ F
In fact W forms a vector space under the operations induced from V . That is W becomes a
vector space under the same vector addition, scalar multiplication operation in V .
Example 104.
(1) Any vector space V is a subspace of itself. And the zero space {0} is
a subspace of every vector space. These are called the trivial subspaces. In fact, this is
can be used as a test for checking a subset is not a subspace. For example if a subset W
does not contain the zero vector 0, then it cannot be a subspace!
(2) Example (2) above is a subspace of example (1). That is, the null space is the subspace
of its domain space. In fact, all subspaces are formed this way. In R2 , anynon trivial
x
subspaces are lines through the origin. For example, the subspace W =
|x∈R
x
is the line y = x through the origin. In R3 , any line or plane through the origin form
subspaces. In fact since the zero vector 0 is a subset of all vectorspaces (and hence
subspaces), any line which do not pass through the origin is NOT a subspace.
The first figure gives example of subspaces while the second figure shows an example
which is NOT a subspace.
Remark 105.
(1) Union of two subspaces are not subspaces. This is clear as the union of
two lines through the origin is not a line
(2) Intersection of two subspaces is indeed a subspace. This will become apparent after the
next result.
Theorem 106 (Subspace criterion). Let V be a F-vector space. Then a non empty subset
W ⊆ V is a subspace if and only if for every u, v ∈ W, c, d ∈ F, we have cu + dv ∈ W .
Sketch of proof. Clearly if W is a subspace of V , then for every u, v ∈ W, c, d ∈ F, we have
cu + dv ∈ W . Conversly, if for every u, v ∈ W, c, d ∈ F, we have cu + dv ∈ W . Since W ̸= ∅,
let u ∈ W then 0 = u + (−1)u ∈ W . Also for u, v ∈ W, c ∈ F, u + v = 1 · u + 1 · w ∈ W and
cu ∈ W . Thus W forms a subspace of V .
□
Remark 107. Intersection of two subspaces form a subspace. This becomes clear. SUppose
W1 , W2 are subspaces of V . Then for u, v ∈ W, c ∈ F, we have cu + dv ∈ W1 and cu + dv ∈ W2
(by above theorem). Thus cu + dv ∈ W1 ∩ W2 . Using the above theorem, we get W1 ∩ W2 is a
subspace. Of course, the intersection W1 ∩ W2 could very well be empty.
MTL101
Lecture #7 Span and Linear independence
Definition 108. Let S be any subset of a vector space V . Then the span of S, denoted by
span S or span (S), is the collection of all combinations a1 u1 + · · · am um where ai ∈ F, ui ∈ S.
If S is an empty set, then we set span (S) = {0}.
Remark 109. span (S) is P
a subspace of
Pthe vector
P space V . This can be easily checked. For, if
w, v ∈ spanP
(S), then w = P
ci ui , v =
di ui + ej u′i , ci , di , ei ∈ F, ui , u′i ∈ S. But for a, b ∈ F,
aw + bv = (aci + bdi )ui + bei u′i ∈ span (S). Thus by subspace criterion, we have span (S)
is a subspace.
Definition 110. Let S be a subset of V . Then ⟨S⟩ is the intersection of all subspaces containing
S.
Of course, if W is a subspace of V , then ⟨W ⟩ = W
Theorem 111. span (S) is the smallest subspace of the vector space V . In other words
span (S) = ⟨S⟩
T
Sketch of proof. Suppose W ′ = Wi where the intersection is over all the subspaces Wi which
contains S. Since S ⊆ span (S) and span (S) is a subspace, we can immediately see that
W ′ ⊆ span (S).
P
Conversly if u ∈ span (S), then u =
ci ui , ci ∈ F, ui ∈ S. ButP
ui ∈ S ⊆ Wj for all j. THus
repeatedly
using
the
subspace
criterion,
we
can
see
that
u
=
ci ui ∈ Wj for all j. Thus
T
u ∈ Wj = W .
□
Theorem 112. Let S = {u1 , . . . , um } be a subset of vectors in the vector space Rn (or Cn ).
A vector v ∈ V is in the subspace span (S) if and only if the linear system Ax = v where
A = [u1 u2 · · · um ] has a solution.
P
Proof. If v ∈ span (S), then v =
ci ui , ci ∈ F, ui ∈ S. BUt this equality can also be written
c1
c1
.
as A .. = v where A = [u1 u2 · · · um ]. Thus ... is a solution to the system Ax = v. The
cm
cm
converse statement is true by following the proof in the reverse.
□
Remark 113. The above theorem gives a practical way to check if the vector v is in the
span (S).
Definition 114. Let u1 , . . . , um ∈ V . Then the vectors u1 , . . . , un are said to be linearly
dependent if there exists a linear combination
c1 u1 + · · · + cm um = 0, ci ∈ F
such that not all ci ’s equal zero.
The vectors u1 , . . . , un are said to be linearly independent if they are not linearly dependent. Of
course, it means that if there is a linear combination c1 u1 + · · · + cm um = 0, ci ∈ F, then all
ci = 0.
Example 115.
(1) A set which contains the zero vector is always linearly dependent. For
example if the set S = {0, u1 , . . . , um }, then we can easily get a linear combination
1 · 0 + 0 · u1 + · · · 0 · um = 0
(2) A set with a single nonzero vector is always linearly independent. Of course, if S = {u}
where u ̸= 0, and suppose cu = 0. If c ̸= 0, then c−1 (cu) = c−1 0 = 0. Thus c−1 (cu) =
(c−1 c)u = u = 0, which is a contradiction. Thus c = 0.
0
..
.
n
(3) For the vector space R , let ei denote the column vector 1 which has a 1 in the
.
..
0
i-th row and zero everywhere else. Then we can see that e1 , . . . , en forms a linearly
independent set. This is not the only linearly independent set in Rn . In fact, you can
see in the following theorems that
the
columns of an invertible matrix A willforman
1
3
1 3
linearly independent set. Thus
,
form an linearly independent set as
is
2
4
2 4
an invertible matrix.
Theorem 116. Let S = {u1 , . . . , um }. Then S is linearly dependent if and only if one of the
vectors can be written as a linear combination of the others.
P
Sketch of proof. If S is linearly dependent, then there exists a linear combination
ci u i = 0
where not all ci ’s are zero. Suppose c1 ̸= 0, then we have c1 u1 = −c2 u2 − · · · − cn un . Since
2
u − · · · − ccn1 u1 . Thus u1 ∈ span (S). The converse statement can be
c1 ̸= 0, we have u1 = −c
c1 1
recovered by following the proof backward.
□
Theorem 117. Let S = {u1 , . . . , um } be a subset of vectors in the vector space Rn (or Cn ).
Then S is linearly independent if and only if Ax = 0 where A = [u1 u2 · · · um ] has a unique
solution.
P
Sketch of proof. SUppose S is linearly independent, then
ci ui = 0 implies that ci = 0 for
all i. Another way to see this statement is that the system of equations Ax = 0 where A =
[u1 u2 · · · um ] has a unique trivial solution. Now use Theorem 68 to see that A is invertible (an
equivalent criterion).
□
Corollary 118. Let S = {u1 , . . . , um } be a subset of vectors in the vector space Rn (or Cn ).
Then S is linearly independent if and only if Ax = b where A = [u1 u2 · · · um ] has a unique
solution for every b ∈ Rn
Definition 119. A subset B of a vector space V is called a basis for a subspace W of V if B
is linearly independent and spans W .
Example 120.
(1) For a vector space Rn , the vectors e1 , . . . , en forms a basis. This is called
the standardbasis.
1
3
(2) The vectors
,
form a basis in R2 as it is linearly independent and spans R2 . Use
2
4
2
Theorem 112 and Theorem 117to make
this conclusion. This is not the only basis for R .
1
1
We can choose something like
,
also is a basis for R2 by virtue of Theorem 112
0
−1
and Theorem 117.
Theorem 121. Let B = {u1 , . . . , um }. Then B is a basis for a subspace W . Then every vector
w ∈ W can be expressed uniquely as a linear combination
of elements in B. That is, there
P
exists, unique elements c1 , . . . , cm such that w = ci ui .
Sketch of proof. Clearly every element of V can be expressed as a linear combination of elements
of B (as B spans V ). The important part is proving uniqueness. FOr example, suppose v ∈ V
and
X
X
v=
ci u i =
c′i ui
where B = {u1 , . . . , un }. Then we get
X
X
X
ci u i −
c′i ui =
(ci − c′i )ui = 0
Since elements of B form a basis, the vectors u1 , . . . , um are linearly independent. So ci − c′i = 0
for all i. Thus ci = c′i for all i.
□
Theorem 122. Suppose V has a finite basis. Then any linearly independent set is part of a
basis for a vector space V
Sketch of proof. Let B = {u1 , . . . , um } be a linearly independent set. Now let S1 = span (B).
If S1 = B, then B is already a basis. If not, then there is a vector say um+1 ∈ V \S1 . Let
B1 = B ∪ {um+1 } and S2 = span (B1 ). By construction B ⊆ B1 and S1 ⊆ S2 . If S2 = V , then
B1 is a basis and the linearly independent set B is part of the basis B1 . If not keep repeating
the sets until you reach Bn , then basis. You should be cautious here as we are assuming that
the basis of the vector space is finite. If it were infinite, you have to use a different proof.
□
From the above theorem, we see that a linearly independent set S is already a basis or we can
add vectors to it to form a basis of the vector space( or subspace). Similarly if we start with a
a set S which spans the vector space V , then S already linearly independent or we can remove
vectors from it to form a basis of V .
Theorem 123. B is a basis for a vector space V if and only if it is a maximal linearly independent set (that is the set B ∪ {u} for u ̸= 0, u ∈ V \B is linearly dependent).
Proof. Suppose B = {u1 , . . . , um } is a basis and let
P u ̸= 0, u ∈ V \B.
P Then consider B ∪ {u}.
Since B is a basis for V , B spans V . Thus u =
ci ui . Now u − ci ui = 0 is a non trivial
linear combination of elements of B. Thus B ∪ {u} is linearly dependent.
□
Theorem 124. If V has two bases B1 , B2 with finitely many elements, then both B1 , B2 have
the same number of elements.
Sketch of proof. Suppose B1 = {u1 , . . . , um } and B2 = {v1 , . . . , vn } and assume m < n. Since
B1 is a basis, it spans V . Thus
X
(5)
vi =
cij uj , 1 ≤ i ≤ n, 1 ≤ j ≤ m
P
Consider the equation
bi vi = 0. Thus the above equality can be substituted here to get
X X
bi
cij uj = 0
i
j
!
X X
j
bi cij
uj = 0
i
Since B1 forms a basis, its linerly independent. Thus
X
bi cij = 0 for all j
i
Recall that bi , cij ∈ R (or the field F). THus we get a system of equations
b1 c11 + b2 c21 + · · · + bn cn1 = 0
b1 c12 + b2 c22 + · · · + bn cn2 = 0
..
.
b1 c1m + b2 c22 + · · · + bn cnm = 0
b1
Thus ... is a solution the homogeneous system Cx = 0 where C = (cji ). Since m < n, we
bn
have rank C < n, use Theorem 55 to conclude that Cx = 0 has a non trivial solution. But any
s1
P
solution ... to the system Cx = 0 gives a non trivial linear combination
si vi = 0. But B2
sn
is also a basis and hence linearly independent. Thus contradiction forces m = n.
□
Theorem 125. Every vector space has a basis
Proving this for vectorspaces with finite basis is obvious. You can use the technique of Theorem 122 to construct a proof. For a vector space with infinite basis, one has to use Zorn’s lemma
or transfinite induction to construct a basis.
Definition 126. If the vector space V has a finite basis (a basis with finite number of elements)
then we say that it is finite dimensional. The dimension of the vector space V is the number of
elements in a basis. It is denoted by dim V . If no finite basis exits, then we say that the vector
space is infinite dimensional.
Example 127.
(1) Rn is finite dimensional with dimension n. A basis is e1 , . . . , en .
(2) The Set of all sequences Map(N, R) is infinite dimensional. Recall that this is the space
of all sequences. It is tempting to define a basis ei = (0, · · · , 0, 1, 0, · · · ) to have 1 in
the i-th term of the sequence and zero everywhere else. But one can immediately see
that this does not span Map(N, R). In fact one can show that this vector space has an
uncountable basis. It is hard to come up with a construction of a basis for this vector
space without using more tools (even then it can be very taxing!). But one can intuitively
see that no finite set can actually span this whole space.
(3) Let P n be the space of all polnomyials of degree at most n. This space has a finite
basis 1, x, · · · , xn . This dim P n = n. If we assign P to be the space of al polynomials,
then it is infinitely dimensional and we can clearly see that the basis is 1, x, · · · , xn , · · · .
The advantage that this infinitely dimensional space has over other infinite dimensional
spaces is that any polynomial has only finitely many terms.
Theorem 128. Let V be a vector space of dimension n. Let S = {u1 , . . . , un }. If S is linearly
independent or spans V , then U forms a basis for V .
Sketch of proof. Suppose S is linearly independent but does not span V . Then there is a vector
v ∈ V such that v ̸∈ span (S). Using the same technique as in Theorem 122 we can see
that {u1 , . . . , un , v} is linearly independent. But any basis has only dimension (n) number of
elements. This contradiction forces S to span V . A similar argument shows that if S spans V
then it has to be linearly independent.
□
The above theorem along with Theorem 117 and Theorem 112 give the following equivalent
conditions
Theorem 129. Let Rn be the vector space and S = {u1 , . . . , un } ⊆ V . THen the following
conditions are equivalent
(1) S spans Rn
(2) S is a linearly independent set
(3) S is a basis for Rn
(4) The homogeneous system Ax = 0 where A = [u1 , . . . , un ] has a unique solution.
(5) The system of equations Ax = b where A = [u1 , . . . , un ] has a unique solution a solution
for all b ∈ Rn .
In the above theorem, one can replace Rn by any vector space of dimension n (check!)
Remark 130. Given a subset S = {u1 , . . . , um } of Rn . We are interested in finding the basis of
u1
u2
the subpace spanned by S. The way to go about doing this is to construct a matrix A =
... .
um
Take care here that we are arranging the vectors in a row form and not the column form. Now
convert the matrix A into reduced echelon form B. Then the ROWS of the matrix
Bwill
form
1
4
7
a basis for the subspace spanned by S. So suppose you are given three vectors 2 , 5 , 8
3
6
9
in R3 . Let W be
the
subspace
spanned
by
these
vectors.
We
go
about
arranging
the
vectors
in
1 2 3
1 0 −1
row form A = 4 5 6 and convert it to the reduced echelon form B = 0 1 2 . Then
7 8 9
0 0 0
1
0
the basis for W are the vectors 0 , 1.
−1
2
MTL101 Lecture #7 Coordinate vectors, row space, row
ranks
In this lecture we see that all finite dimensional V vector space can be made to “look like”
Rn where n = dim V . For example, consider Pn which consists of all polynomials of degree
at most n. A basis for Pn is B = {1, x, x2 , . . . , xn } and any element can be made to look like
a0 + a1 x + · · · + an xn (notice that since B is a basis, any polynomial can be written uniquely as
a0
a1
a linear combination of the basis elements). Thus we associate a vector
... to the polynomial
an
a0 + a1 x + · · · + an xn . Thus we see that any polynomial can be made to look like a vector in
Rn+1 . Even though you need more tools to actually show that Rn+1 and Pn are one and the
same, you can easily see that any polynomial can be thought of as a vector.
Definition 131. Suppose B is a basis for a vector space V . Then B with any ordering of the
elements is called an ordered basis
Definition 132. Let B =P
{u1 , . . . , un } be an ordered basis for a vector space V over a field F.
Now let v ∈ V . Thus v = i ci ui be a unique representation of v in terms of the ordered basis
B. Then the column vector
c1
[v]B = ... ∈ Fn
cn
is the coordinate vector of v with respect to B.
The coordinate vector depends both of the basis AND the ordering of the basis. THe followign
examples show why both points are important for a coordinate vector representation.
(1) Let V = Rn with an ordered basis B = {e1 , . . . , en } (the standard basis).
c1
Then any element of V is of the form v = ... is the coordinate vector of v with respect
cn
to the ordered basis B. If we change the ordering of the basis to B ′ = {e2 , e1 , e3 , . . . , en },
c2
c1
c
then the coordinate vector of [v]B ′ is given by
.3 .
..
cn
1
3
1
1
2
(2) In R , we have already seen that B1 =
,
, B2 =
,
form two bases
2
4
0
−1
c
for R2 . Let v = 1 ∈ R2 be an element written with respect to the basis {e1 , e2 }. Then
c2
we would like to know the coordinate vector of v with respect to B1 and B2 . To obtain
this, first write down the vector v interms of the basis vectors, i.e., we need to know the
coefficients a, b in
c1
1
3
v=
=a
+b
c2
2
4
Example 133.
But the coefficients a, b can easily be obtained by solving the system
c1 = a + 3b
c2 = 2a + 4b
Then we can easily get the solution for a = 3c1 − 3c2 , b = c2 − 2c1 . THus the coordinate
vector of v with respect to B1 is
3c1 − 3c2
[v]B1 =
c2 − 2c1
c1 + c2
Similarly we can show that [v]B2 =
−c2
Definition 134. SUppose B1 , B2 are two ordered basis for a vector space V of dimension n.
Then for any v ∈ V there are two coordinate vector representations [v]B1 , [v]B2 . There exists an
n × n invertible matrix P such that [v]B1 = P [v]B2
The existence of the inverse and its uniqueness can be obtained quite quickly. For example,
suppose
B1 = {u1 , . . . , un } and B2 = {v1 , . . . , vn }. Since B1 is a basis, we get that vj =
Pn
c
u
j=1 ij i . Now for any vector w ∈ V whose coordinate vector with respect to B2 is of the form
a1
[w]B2 = ... , then we have
an
!
n
X
X
X
(6)
w=
aj v j =
aj
cij ui
j=1
j
i
P
j c1j aj
Thus the coordinate vector of w with respect to B1 is [w]B1 =
P
j c1j aj
c11 c12 · · ·
..
..
..
.
.
[w]B1 =
= .
.
.
.
P
cn1 cn2 · · ·
j cnj aj
..
. But notice that
P .
j cnj aj
c1n
a1
.. .. = P [w]
B2
.
.
cnn
an
This gives us that there exists a matrix n × n matrix P such that [w]B1 = P [w]B2 . To see that
it is invertible, we show that there exists Q such that P Q =P
I. To construct the matrix Q, we
reverse the above process. Since B2 is a basis, we have uk = j bjk vj . Using the above equality
P
P
P P
we also have uk = j bjk ( i cij ui ) = i
c
b
j ij jk ui . Now all these vectors combinations
u1 , . . . , un as a row vector and you can see that
u1
b11 · · · b1n
c11 · · · c1n
u1
u1
.
.
.
.
.
.
.
.
.. = ..
..
.. ..
..
.. .. = QP ...
un
bn1 · · ·
bnn
cn1 · · ·
cnn
un
un
Thus we see that QP = I. We can similarly prove that P Q =
One can get the change of basis matrix P very easily by following the above steps.
1
1
1
3
Example 135. Consider the two bases B1 =
,
, B2 =
,
, in R2 . To
0
−1
2
4
construct the change of bases matrix P , we first write each of the basis elements of B2 in terms
of the basis elements of B1 . That is we get
1
1
1
=3
+ (−2)
2
0
−1
3
1
1
=7
+ (−4)
4
0
−1
3
7
Then the matrix P =
−2 −4
0
advertisement
Related documents
![Quiz #2 & Solutions Math 304 February 12, 2003 1. [10 points] Let](http://s2.studylib.net/store/data/010555391_1-eab6212264cdd44f54c9d1f524071fa5-300x300.png)
Download
advertisement
Add this document to collection(s)
You can add this document to your study collection(s)
Sign in Available only to authorized usersAdd this document to saved
You can add this document to your saved list
Sign in Available only to authorized users