De…nitions of Linear Algebra Terms
In order to learn and understand mathematics, it is necessary to understand
the meanings of the terms (vocabulary words) that are used. This document
contains de…nitions of some of the important terms used in linear algebra.
All of these de…nitions should be memorized (and not just memorized but
understood). In addition to the de…nitions that are given here, we also include
statements of some of the most important results (theorems) of linear algebra.
The proofs of these theorems are not included here but will be given in class.
There will be a question on each of the exams that asks you to state
some de…nitions. It will be required that you state these de…nitions very
precisely in order to receive credit for the exam question. (Seemingly very
small things such as writing the word “or” instead of the word “and”, for
example, can render a de…nition incorrect.) The de…nitions given here are
organized according to the section number in which they appear in David
Lay’s textbook.
Here is a suggestion for how to study linear algebra (and other subjects
in higher level mathematics):
1. Carefully read a section in the textbook and also read your class notes.
Pay attention to de…nitions of terms and examples and try to understand each concept along the way as you read it. You don’t have to
memorize all of the de…nitions upon …rst reading. You will memorize
them gradually as you work the homework problems.
2. Work on the homework problems. As you work on the homework problems, you should …nd that you will need to continuously refer back to
the de…nitions and to the examples given in the textbook. (This is
how you will naturally come to memorize the de…nitions.) You will
…nd that you have to spend a considerable amount of time working on
homework. When you are done, you will probably have read through
the section and your course notes several times and you will probably
have memorized the de…nitions. (There are some very good True–False
questions among the homework problems that are good tests of your
understanding of the basic terms and concepts. Such True–False questions will also appear on the exams given in this course.)
3. Test yourself. After you are satis…ed that you have a good understanding of the concepts and that can apply those concepts in solving
1
problems, try to write out the de…nitions without referring to your book
or notes. You might then also try to work some of the supplementary
problems that appear at the end of each chapter of the textbook.
4. One other tip is that it is a good idea to read a section in the textbook
and perhaps even get started on working on the homework problems
before we go over that section in class. This will better prepare you to
take notes and ask questions when we do go over the material in class.
It will alert you to certain aspects of the material that you might not
quite understand upon …rst reading and you can then ask questions
about those particular aspects during class.
1.1: Systems of Linear Equations
De…nition 1 A linear equation in the variables x1 ; x2 ; : : : ; xn is an equation of the form
a1 x 1 + a2 x 2 + : : : + an x n = b
where a1 ; a2 ; : : : ; an and b are real or complex numbers (that are often given
in advance).
Example 2
4x + 4y
7z + 8w = 8
is a linear equation in the variables x; y; z; and w.
3x + 5y 2 = 6
is not a linear equation (because of the appearance of y 2 ).
De…nition 3 Suppose that m and n are positive integers. A system of
linear equations in the variables x1 ; x2 ; : : : ; xn is a set of equations of the
form
a11 x1 + a12 x2 + : : : + a1n xn = b1
a21 x1 + a22 x2 + : : : + a2n xn = b2
..
.
am1 x1 + am2 x2 + : : : + amn xn = bm
2
where aij (i = 1; : : : ; m, j = 1; : : : n) and bi (i = 1; : : : ; m) are real or complex
numbers (that are often given in advance). This system of linear equations
is said to consist of m equations with n unknowns.
Example 4
2x1 + 2x2
3x1
5x3 + 4x4
4x1 + x2
2x2 + 4x4
2x5 = 4
5x4 = 1
2x5 = 4
is a system of three linear equations in …ve unknowns. (The “unknowns” or
“variables” of the system are x1 ; x2 ; x3 ; x4 ; and x5 .)
De…nition 5 A solution of the linear system
a11 x1 + a12 x2 + : : : + a1n xn = b1
a21 x1 + a22 x2 + : : : + a2n xn = b2
..
.
am1 x1 + am2 x2 + : : : + amn xn = bm
is an ordered n–tuple of numbers (s1 ; s2 ; : : : ; sn ) such that all of the equations
in the system are true when the substitution x1 = s1 ; x2 = s2 ; : : : ; xn = sn is
made.
Example 6 The 5–tuple 0; 1;
2x1 + 2x2
3x1
12
; 0; 3
5
5x3 + 4x4
4x1 + x2
2x2 + 4x4
is a solution of the linear system
2x5 = 4
5x4 = 1
2x5 = 4
because all equations in the system are true when we make the substitutions
x1 = 0
x2 = 1
12
x3 =
5
x4 = 0
x5 = 3.
The 5–tuple 1; 3; 53 ; 0; 21 is also a solution of the system (as the reader can
check).
3
De…nition 7 The solution set of a system of linear equations is the set of
all solutions of the system.
De…nition 8 Two linear systems are said to be equivalent to each other if
they have the same solution set.
De…nition 9 A linear system is said to be consistent if it has at least one
solution and is otherwise said to be inconsistent.
Example 10 The linear system
2x1 + 2x2
5x3 + 4x4
4x1 + x2
2x2 + 4x4
3x1
2x5 = 4
5x4 = 1
2x5 = 4
is consistent because it has at least one solution. (It in fact has in…nitely
many solutions.) However, the linear system
x + 2y = 5
x + 2y = 9
is obviously inconsistent.
De…nition 11 Suppose that m and n are positive integers. An m n matrix
is a rectangular array of numbers with m rows and n columns. Matrices are
usually denoted by capital letters. If m = 1 or n = 1, then the matrix is
referred to as a vector. In particular, if m = 1, then the matrix is a row
vector of dimension n, and if n = 1 then the matrix is a column vector of
dimension m. Vectors are usually denoted by lower case boldface letters or
(when hand–written) as lower case letters with arrows above them.
Example 12
is a 5
4 matrix.
2
6
6
A=6
6
4
8 10
5 7
0
5
1
3
4
8
r=!
r =
2 3
4
6
6
10
10
1
8
6
1
5
10
1
9 8
3
7
7
7
7
5
is a row vector of dimension 5.
2
3
4
c=!
c =4 5 5
10
is a column vector of dimension 3.
Remark 13 If A is an m n matrix whose columns are made up of the
column vectors c1 ; c2 ; : : : ; cn , then A can be written as
A = [c1
cn ] .
c2
Likewise, if A is made up of the row vectors r1 ; r2 ; : : : ; rm , then A can be
written as
2
3
r1
6 r2 7
6
7
A = 6 .. 7 .
4 . 5
rm
Example 14 The 5
can be written as
4 matrix
2
8 10
6 5 7
6
5
A=6
6 0
4 1
3
4
8
6
6
10
10
1
A = [c1
c3
c2
8
6
1
5
10
3
7
7
7
7
5
c4 ]
where
2
6
6
c1 = 6
6
4
8
5
0
1
4
3
2
7
6
7
6
7 , c2 = 6
7
6
5
4
10
7
5
3
8
3
2
7
6
7
6
7 , c3 = 6
7
6
5
4
5
6
6
10
10
1
3
2
7
6
7
6
7 , c4 = 6
7
6
5
4
8
6
1
5
10
3
7
7
7
7
5
or can be written as
2
r1
r2
r3
r4
r5
6
6
A=6
6
4
where
r1 =
8 10
7
7
7
7
5
6
8
5 7 6
r2 =
r3 =
0 5
r4 =
1
r5 =
3
10 1
3
4
6
10 5
8
1
10 .
De…nition 15 For a given linear system of equations
a11 x1 + a12 x2 + : : : + a1n xn = b1
a21 x1 + a22 x2 + : : : + a2n xn = b2
..
.
am1 x1 + am2 x2 + : : : + amn xn = bm ,
the m
n matrix
2
6
6
A=6
4
a11
a21
..
.
a12
a22
..
.
am1 am2
is called the coe¢ cient matrix of
2
a11
6 a21
6
[A b] = 6 ..
4 .
am1
...
a1n
a2n
..
.
amn
3
7
7
7
5
the system and the m
3
a12
a1n b1
a22
a2n b2 7
7
..
..
.. 7
..
.
.
.
. 5
am2
amn bm
is called the augmented matrix of the system.
6
(n + 1) matrix
Example 16 The coe¢ cient matrix of the linear system
2x1 + 2x2
5x3 + 4x4
4x1 + x2
2x2 + 4x4
3x1
is
2
2
A=4 4
3
2
1
2
and the augmented matrix of this
2
2
4
4
[A b] =
3
5
0
0
2x5 = 4
5x4 = 1
2x5 = 4
4
5
4
3
2
0 5
2
4
5
4
2
0
2
system is
2
1
2
5
0
0
3
4
1 5.
4
De…nition 17 An elementary row operation is performed on a matrix
by changing the matrix in one of the following ways:
1. Interchange: Interchange two rows of the matrix. If rows i and j are
interchanged, then we use the notation ri ! rj .
2. Scaling: Multiply all entries in a single row by a non–zero constant,
k. If the ith row is multiplied by k, then we use the notation k ri ! ri .
3. Replacement: Replace a row with the sum of itself and a non–zero
multiple of another row. If the ith row is replaced with the sum of itself
and k times the jth row, then we use the notation ri + k rj ! ri .
Example 18 Here are some examples
formed on the matrix
2
2 5
4
7 4
A=
8 7
1. Interchange rows
2
2
4 7
8
1 and 3:
5
4
7
7
8
8
of elementary row operations per7
8
8
3
7
2 5.
2
3
2
7
8 7
r1 !r3
5
4
2
7 4
!
2
2 5
7
8
8
7
3
2
2 5
7
2. Scale row 1 by a factor of 5.
2
3
2
2 5 7
7
10
5 r1 !r1
4 7 4
8 2 5
! 4 7
8 7 8
2
8
25
4
7
35
8
8
3
35
2 5
2
3. Replace row 2 with the sum of itself and 2 times row 3.
2
3
2
3
2 5 7
7
2 5 7 7
r +2 r3 !r2
4 7 4
4 9 18 8
8 2 5 2 !
2 5
8 7 8
2
8 7 8
2
De…nition 19 An m n matrix, A, is said to be row equivalent (or simply
equivalent) to an m n matrix, B, if A can be transformed into B by a
series of elementary row operations. If A is equivalent to B, then we write
A B.
Example 20 As can
2
2
4 7
8
2
2 5
4 7 4
8 7
and
2
be seen by studying the previous
3 2
5 7
7
8 7 8
5
4
4
8 2
7 4
8
7 8
2
2 5 7
3 2
7
7
10
25
35
5
4
8 2
7 4
8
8
2
8
7
8
2 5
4 7 4
8 7
7
8
8
3
7
2 5
2
2
2 5 7
4 9 18 8
8 7 8
Example 21 The sequence of row operations
2
3
2
3
2
3
6
6
12
6
1 !r1
3 !r3
4 1 6 5 2 r!
4 1 6 5 4 r!
4 1
7
1
7
1
28
shows that
2
3
4 1
7
3
6
6 5
1
2
28
4 1
6
8
3
4
6 5.
12
example
3
2
2 5,
7
3
35
2 5,
2
3
7
2 5.
2
3
2
12
28
r !r
6 5 1 !3 4 1
4
6
3
4
6 5
12
Remark 22 (optional) Row equivalence of matrices is an example of what
is called an equivalence relation. An equivalence relation on a non–empty
set of objects, X, is a relation, , that is re‡exive, symmetric, and transitive.
The re‡exive property is the property that x
x for all x 2 X. The
symmetric property is the property that if x 2 X and y 2 X with x
y,
then y x. The transitive property is the property that if x 2 X, y 2 X,
and z 2 X with x
y and y
z, then x
z. If we take X to be the set
of all m n matrices, then row equivalence is an equivalence relation on X
because
1. A
A for all A 2 X.
2. If A 2 X and B 2 X and A
B, then B
3. If A 2 X, B 2 X, and C 2 X and A
A.
B and B
C, then A
C.
1.2: Row Reduction and Echelon Forms
De…nition 23 The leading entry in a row of a matrix is the …rst (from
the left) non–zero entry in that row.
Example 24 For the matrix
2
6
6
6
6
4
0
0
5
0
9
1
0
0
0
7
3
4
8
4 3 7
7
0 10 7
7,
0
1 5
2
8
the leading entry in row 1 is 1, the leading entry in row 2 is 4, the leading
entry in row 3 is 5, the leading entry in row 4 is 1, and the leading entry
in row 5 is 9. (Note that if a matrix happens to have a row consisting of all
zeros, then that row does not have a leading entry.)
De…nition 25 A zero row of a matrix is a row consisting entirely of zeros.
A nonzero row is a row that is not a zero row.
De…nition 26 An echelon form matrix is a matrix for which
1. No zero row has a nonzero row below it.
9
2. Each leading entry is in a column to the right of the leading entry in
the row above it.
De…nition 27 A reduced echelon form matrix is an echelon form matrix
which also has the properties:
3. The leading entry in each nonzero row is 1.
4. Each leading 1 is the only nonzero entry in its column.
Example 28 The matrix
2
6 4
6 1 2
6
4 4 4
0 10
is not an echelon form matrix.
The matrix
3
10
10 7
7
2 5
1
0 0 0 4 8
0 0 0 0 3
is an echelon form matrix but not a reduced echelon form matrix.
The matrix
2
3
0 0 1 0
6 0 0 0 1 7
6
7
6 0 0 0 0 7
6
7
4 0 0 0 0 5
0 0 0 0
is a reduced echelon form matrix.
Theorem 29 (Uniqueness of the Reduced Echelon Form) Each matrix, A, is equivalent to one and only one reduced echelon form matrix. This
unique matrix is denoted by rref(A).
Example 30 For
2
6 4
6 1 2
A=6
4 4 4
0 10
10
3
10
10 7
7;
2 5
1
we can see (after some work) that
2
For
1
6 0
rref (A) = 6
4 0
0
A=
3
0
0 7
7.
1 5
0
0
1
0
0
0 0 0 4 8
0 0 0 0 3
,
we have
0 0 0 1 0
0 0 0 0 1
rref (A) =
For
we have
2
6
6
A=6
6
4
0
0
0
0
0
2
6
6
rref (A) = 6
6
4
0
0
0
0
0
1
0
0
0
0
0
1
0
0
0
0
0
0
0
0
0
0
0
0
0
1
0
0
0
0
.
3
7
7
7,
7
5
0
1
0
0
0
3
7
7
7.
7
5
De…nition 31 A pivot position in a matrix, A, is a position in A that
corresponds to a row–leading 1 in rref(A). A pivot column of A is column
of A that contains a pivot position.
Example 32 By studying the preceding example, we can see that the pivot
columns of
2
3
6 4
10
6 1 2
10 7
7
A=6
4 4 4
2 5
0 10
1
are columns 1, 2, and 3. (Thus all columns of A are pivot columns).
11
The pivot columns of
A=
0 0 0 4 8
0 0 0 0 3
are columns 4 and 5.
The pivot columns of
2
6
6
A=6
6
4
are columns 3 and 4.
0
0
0
0
0
0
0
0
0
0
1
0
0
0
0
3
0
1
0
0
0
7
7
7
7
5
De…nition 33 Let A be the coe¢ cient matrix of the linear system of equations
a11 x1 + a12 x2 + : : : + a1n xn = b1
a21 x1 + a22 x2 + : : : + a2n xn = b2
..
.
am1 x1 + am2 x2 + : : : + amn xn = bm .
Each variable that corresponds to a pivot column of A is called a basic variable and the other variables are called free variables.
Example 34 The coe¢ cient matrix of the linear system
2x1 + 2x2
3x1
is
Since
5x3 + 4x4
4x1 + x2
2x2 + 4x4
2
2
4
4
A=
3
2
1
2
5
0
0
2
1 0 0
rref (A) = 4 0 1 0
0 0 1
12
2x5 = 4
5x4 = 1
2x5 = 4
4
5
4
6
5
1
5
2
5
3
2
0 5.
2
2
5
8
5
6
5
3
5,
we see that the pivot columns of A are columns 1, 2, and 3. Thus the basic
variables of this linear system are x1 , x2 ; and x3 . The free variables are x4
and x5 .
Theorem 35 (Existence and Uniqueness Theorem) A linear system of
equations is consistent if and only if the rightmost column of the augmented
matrix for the system is not a pivot column. If a linear system is consistent,
then the solution set of the linear system contains either a unique solution
(when there are no free variables), or in…nitely many solutions (when
there is at least one free variable).
Example 36 The augmented matrix of the system
2x1 + 2x2
3x1
is
5x3 + 4x4
4x1 + x2
2x2 + 4x4
2
Since
2
[A b] = 4 4
3
2
1
2
5
0
0
2
1 0 0
rref ([A b]) = 4 0 1 0
0 0 1
2x5 = 4
5x4 = 1
2x5 = 4
4
5
4
2
0
2
6
5
2
5
8
5
6
5
1
5
2
5
3
4
1 5.
4
6
5
19
5
6
5
3
5,
we see that the rightmost column of [A b] is not a pivot column. This means
that the system is consistent. Furthermore, since the system does have free
variables, there are in…nitely many solutions to the system.
Here is a general procedure for determining whether a given linear system of equations is consistent or not and, if it is consistent, whether it has
in…nitely many solutions or a unique solution:
1. Compute rref([A b]) to determine whether or not the rightmost column of [A b] is a pivot column. If the rightmost of [A b] is a pivot
column, then the system is inconsistent.
13
2. Assuming that the rightmost column of [A b] is not a pivot column,
take a look at rref(A). (This requires no extra computation.) If all
columns of A are pivot columns, then the linear system has a unique
solution. If not all columns of A are pivot columns, then the system
has in…nitely many solutions.
1.3: Vector Equations
De…nition 37 Suppose that
2
6
6
u=6
4
u1
u2
..
.
un
3
7
7
7
5
2
6
6
and v = 6
4
v1
v2
..
.
vn
3
7
7
7
5
are column vectors of dimension n. We de…ne the sum of u and v to be the
vector
2
3
u1 + v1
6 u2 + v2 7
6
7
u+v =6
7.
..
4
5
.
un + vn
De…nition 38 Suppose that
2
6
6
u=6
4
u1
u2
..
.
un
3
7
7
7
5
is a column vectors of dimension n and suppose that c is a real number (also
called a scalar). We de…ne the scalar multiple cu to be
2
3
cu1
6 cu2 7
6
7
cu = 6 .. 7 .
4 . 5
cun
14
De…nition 39 The symbol Rn denotes the set of all ordered n–tuples of numbers. The symbol Vn denotes the set of all column vectors of dimension n.
(The textbook author does not make this distinction. He uses Rn to denote
both things. However, there really is a di¤erence. Elements of Rn should
be regarded as points whereas elements of Vn are matrices with one column.)
If x = (x1 ; x2 ; : : : ; xn ) is a point in Rn , then the position vector of x is the
vector
2
3
x1
6 x2 7
6
7
x = 6 .. 7 .
4 . 5
xn
Example 40 x = (1; 2) is a point in R2 . The position vector of this point
(which is an element of V2 ) is
x=
1
2
.
De…nition 41 Suppose that u1 ; u2 ; : : : ; un are vectors in Vm . (Note that
this is a set of n vectors, each of which has dimension m). Also suppose that
c1 ; c2 ; : : : ; cn are scalars. Then the vector
c 1 u1 + c 2 u2 +
+ c n un
is called a linear combination of the vectors u1 ; u2 ; : : : ; un .
Example 42 Suppose that
u1 =
1
2
, u2 =
0
5
, u3 =
1
4
.
2
0
5
1
4
=
Then the vector
4u1
2u2 + 3u3 = 4
1
2
+3
7
30
is a linear combination of u1 ,u2 , and u3 .
De…nition 43 Suppose that u1 ; u2 ; : : : ; un are vectors in Vm . The span of
this set of vectors, denoted by Spanfu1 ; u2 ; : : : ; un g, is the set of all vectors
that are linear combinations of the vectors u1 ; u2 ; : : : ; un . Formally,
Span fu1 ; u2 ; : : : ; un g
= fv 2 Vm j there exist scalars c1 ; c2 ; : : : ; cn such that v = c1 u1 + c2 u2 +
15
+ c n un g .
De…nition 44 The zero vector in Vm is the vector
2 3
0
6 0 7
6 7
0m = 6 .. 7 .
4 . 5
0
Remark 45 If u1 ; u2 ; : : : ; un is any set of vectors in Vm , then 0m 2 Spanfu1 ; u2 ; : : : ; un g
because
0m = 0u1 + 0u2 +
+ 0un .
De…nition 46 A vector equation in the variables x1 , x2 ,. . . ,xn is an equation of the form
x1 a1 + x2 a2 +
+ xn an = b
where a1 ,a2 ,. . . ,an and b are vectors in Vm (which are usually known beforehand). An ordered n–tuple (s1 ; s2 ; : : : ; sn ) is called a solution of this vector
equation if the equation is satis…ed when we set x1 = s1 , x2 = s2 ,. . . , xn = sn .
The solution set of the vector equation is the set of all of its solutions.
Example 47 An example of a vector equation is
x1
1
2
+ x2
0
5
+ x3
1
4
=
4
2
.
The ordered triple (x1 ; x2 ; x3 ) = ( 9; 4=5; 5) is a solution of this vector equation because
4
0
1
4
1
+5
=
.
9
+
5
4
2
2
5
Remark 48 The vector equation
x1 a1 + x2 a2 +
+ xn an = b
is consistent if and only if b 2 Spanfa1 ; a2 ; : : : ; an g.
Remark 49 The vector equation
x1 a1 + x2 a2 +
16
+ xn an = b
where
and
2
6
6
a1 = 6
4
a11
a21
..
.
am1
3
2
7
6
7
6
7 , a2 = 6
5
4
3
a12
a22
..
.
am2
2
6
6
b=6
4
2
a1n
a2n
..
.
7
6
7
6
7 , . . . ,an = 6
5
4
amn
3
b1
b2
..
.
bm
3
7
7
7
5
7
7
7
5
is equivalent to the system of linear equations
a11 x1 + a12 x2 + : : : + a1n xn = b1
a21 x1 + a22 x2 + : : : + a2n xn = b2
..
.
am1 x1 + am2 x2 + : : : + amn xn = bm .
This means that the vector equation and the system have the same solution
sets. Each can be solved (or determined to be inconsistent) by row–reducing
the augmented matrix
a1 a2
an b
.
Example 50 Let us solve the vector equation
x1
1
2
+ x2
0
5
+ x3
1
4
4
2
=
This equation is equivalent to the linear system
2x1
x1 + x3 =
5x2 + 4x3 =
4
2.
Since
[A b] =
1
2
0 1
5 4
4
2
and
rref ([A b]) =
1 0
0 1
17
1
2
5
4
6
5
,
.
we see that the vector equation is consistent (because the rightmost column of
[A b] is not a pivot column) and that it in fact has in…nitely many solutions
(because not all columns of A are pivot columns). We also see that the general
solution is
x1 =
4 x3
6 2
x2 =
+ x3
5 5
x3 = free.
Note that if we set x3 = 5, then we get the particular solution
x1 =
9
4
5
x3 = 5
x2 =
that was identi…ed in Example 47.
1.4: The Matrix Equation Ax = b
De…nition 51 For a matrix of size m n,
2
a11
6 a21
6
an = 6 ..
A = a1 a2
4 .
a12
a22
..
.
...
am1 am2
and a vector of dimension n,
2
6
6
x=6
4
x1
x2
..
.
xn
we de…ne the product Ax to be
3
7
7
7,
5
Ax = x1 a1 + x2 a2 +
18
+ xn an .
a1n
a2n
..
.
amn
3
7
7
7,
5
Example 52 Suppose
2
4
A=
and
3
3
4
3
2
3
4
x = 4 1 5.
0
Then
Ax = 4
2
4
3
3
1
+0
4
3
=
11
19
.
De…nition 53 Suppose that
A=
is an m
a1 a2
2
6
6
=6
4
an
n matrix and that
2
Then
6
6
b=6
4
b1
b2
..
.
bm
a11
a21
..
.
a12
a22
..
.
...
am1 am2
3
7
7
7 2 Vm .
5
Ax = b
is called a matrix equation.
Remark 54 The system of linear equations
a11 x1 + a12 x2 + : : : + a1n xn = b1
a21 x1 + a22 x2 + : : : + a2n xn = b2
..
.
am1 x1 + am2 x2 + : : : + amn xn = bm ,
19
a1n
a2n
..
.
amn
3
7
7
7
5
the vector equation
x1 a1 + x2 a2 +
+ xn an = b,
and the matrix equation
Ax = b
are all equivalent to each other! The only subtle di¤erence is that solutions
of the system and of the vector equation are ordered n–tuples
(x1 ; x2 ; : : : ; xn ) = (s1 ; s2 ; : : : ; sn ) 2 Rn ;
whereas solutions of the matrix
2
x1
6 x2
6
6 ..
4 .
xn
equation are the corresponding vectors
3 2
3
s1
7 6 s2 7
7 6
7
7 = 6 .. 7 2 Vn .
5 4 . 5
sn
1.5: Solution Sets of Linear Systems
De…nition 55 A homogeneous linear system is one of the form
Ax = 0m
where A is an m n matrix.
A non–homogeneous linear system is a linear system that is not homogeneous.
Remark 56 Homogenous systems are consistent! That’s because they have
the solution x = 0n .
De…nition 57 The solution x = 0n is called the trivial solution of the
homogeneous system Ax = 0m . A non–trivial solution of the homogeneous
system Ax = 0m is a solution x 2 Vn such that x 6= 0n .
Example 58 The system
5x + 2y 3z + w = 0
3x + y 5z + w = 0
x 3y 5z + 5w = 0
is homogeneous. A non–trivial solution of this system (as the reader can
check) is (x; y; z; w) = ( 18; 34; 1; 25).
20
Theorem 59 If every column of A is a pivot column, then the homogeneous
system Ax = 0m has only the trivial solution. If not every column of A is
a pivot column, then the homogeneous system Ax = 0m has in…nitely many
non–trivial solutions (in addition to the trivial solution).
Example 60 The system given in Example 58 has coe¢ cient matrix
2
3
5 2
3 1
5 1 5.
A=4 3 1
1
3
5 5
Without doing any computations, we can see that it is not possible that every
column of A can be a pivot column. (Why?) Therefore the homogeneous
system Ax = 0m has in…nitely many non–trivial solutions.
Theorem 61 Suppose that A is an m n matrix and suppose that b 2 Vm .
Also suppose that v 2 Vn is a solution of the homogeneous system Ax = 0m
and that w 2 Vn is a solution of the system Ax = b. Then v + w is also a
solution of Ax = b.
1.7: Linear Independence
De…nition 62 A set of vectors fv1 ; v2 ; : : : ; vn g in Vm is said to be linearly
independent if the vector equation
x1 v1 + x2 v2 +
+ xn vn = 0m
has only the trivial solution. Otherwise the set of vectors is said to be linearly
dependent.
Example 63 Let v1 and v2 be the vectors
2
3
2
4
v1 = 4 1 5 ,
v2 = 4
4
We solve the vector equation
x1 v1 + x2 v2 = 03
21
3
1
1 5.
4
by observing that
2
3 2
3
4
1
1 0
1 5 4 0 1 5.
A=4 1
4
4
0 0
Since each column of A is a pivot column, the vector equation has only the
trivial solution. Therefore the set of vectors fv1 ; v2 g is linearly independent.
De…nition 64 Suppose that fv1 ; v2 ; : : : ; vn g is a linearly dependent set of
vectors and suppose that (c1 ; c2 ; : : : ; cn ) is a non–trivial solution of the vector equation
x1 v1 + x2 v2 +
+ xn vn = 0m .
Then the equation
c1 v1 + c2 v2 +
+ cn vn = 0m
is called a linear dependence relation for the set of vectors fv1 ; v2 ; : : : ; vn g.
Example 65 Consider the set of vectors
2
3
2
3
2
3
4
1
2
v1 = 4 1 5 ,
v2 = 4 1 5 ,
v3 = 4 3 5 ,
4
4
4
2
3
4
v4 = 4 3 5 .
3
We solve the vector equation
x1 v1 + x2 v2 + x3 v3 + x4 v4 = 03
by observing that
2
4
4 1
4
1
1
4
2
3
4
3
4
3 5
3
2
1 0 0
4 0 1 0
0 0 1
11
40
13
20
9
8
3
5.
Our conclusion is that the vector equation has non–trivial solutions and that
the general solution is
x1
x2
x3
x4
= 11t
= 26t
= 45t
= 40t
where t can be any real number. By choosing t = 1, we see that a linear
dependence relation for the set of vectors fv1 ,v2 ,v3 ,v4 g is
11v1 + 26v2 + 45v3 + 40v4 = 03 .
22
1.8: Introduction to Linear Transformations
De…nition 66 A transformation (or function or mapping), T , from Vn
to Vm is a rule that assigns each vector x 2 Vn to a vector T (x) 2 Vm . The
set Vn is called the domain of T and the set Vm is called the codomain (or
target set) of T . If T is a transformation from Vn to Vm , then we write
T : Vn ! Vm .
De…nition 67 Suppose that T : Vn ! Vm . For each x 2 Vn , the vector
T (x) 2 Vm is called the image of x under the action of T . The set of all
images of vectors in Vn under the action of T is called the range of T:In set
notation, the range of T is
fy 2 Vm j y = T (x) for at least one x 2 Vn g .
Example 68 This example illustrates the terms de…ned in the preceding two
de…nitions: Consider the transformation T : V2 ! V3 de…ned by
2
3
x2
x
T
= 4 4 5.
y
y x
The domain of T is V2 and the codomain of T is V3 .
The image of the vector
4
x=
1
is
2
3
16
T (x) = 4 4 5 .
5
The range of T consists of all vectors in V3 that have the form
2 3
s
4 4 5
t
where s can be any non–negative real number and t can be any real number.
For example, the vector
2
3
6
y=4 4 5
9
23
is in the range of T because
y=T
p
6p
9+ 6
.
Note that it is also true that
y=T
p
9
6p
6
.
Are there any other vectors x 2 V2 such that T (x) = y? The answer to this
does not matter as far as determining whether or not y is in the range of
T is concerned. We know that y is in the range of T because we know that
there exists at least one vector x 2 V2 such that T (x) = y.
De…nition 69 A transformation T : Vn ! Vm is said to be a linear transformation if both of the following conditions are satis…ed:
1. T (u + v) = T (u) + T (v) for all vectors u and v in Vn .
2. T (cu) = cT (u) for all vectors u in Vn and for all scalars c.
Example 70 Let T : V3 ! V2 be de…ned by
02 31
x
x
@
4
y 5A =
T
x + y + 2z
z
.
We will show that T is a linear transformation: Let
3
2
2
3
x1
x2
u = 4 y1 5
and
v = 4 y2 5
z1
z2
24
be two vectors in V3 . Then
02
31
x1 + x2
T (u + v) = T @4 y1 + y2 5A
z1 + z2
=
x1 + x2
(x1 + x2 ) + (y1 + y2 ) + 2 (z1 + z2 )
=
x1 + x2
(x1 + y1 + 2z1 ) + (x2 + y2 + 2z2 )
=
x1
x1 + y1 + 2z1
+
x2
x2 + y2 + 2z2
= T (u) + T (v) :
Now let
2
3
x
u =4 y 5
z
be a vector in V3 and let c be a scalar. Then
02
31
cx
T (cu) = T @4 cy 5A
cz
=
=c
cx
cx + cy + 2 (cz)
x
x + y + 2z
cT (u) .
We have shown that T is a linear transformation.
De…nition 71 The zero transformation is the linear transformation T :
Vn ! Vm de…ned by T (u) = 0m for all u 2 Vn .
De…nition 72 The identity transformation is the linear transformation
T : Vn ! Vn de…ned by T (u) = u for all u 2 Vn .
We leave it to the reader to prove that the zero transformation and the
identity transformation are both linear transformations.
25
Example 73 The transformation T : V2 ! V3 de…ned by
2
3
x2
x
T
=4 4 5
y
y x
is not a linear transformation. We leave it to the reader to explain why not.
1.9: The Matrix of a Linear Transformation
De…nition 74 A transformation T : Vn ! Vm is said to be onto Vm if the
range of T is Vm .
De…nition 75 (Alternate De…nition) A transformation T : Vn ! Vm
is said to be onto Vm if each vector y 2 Vm is the image of at least one
vector x 2 Vn .
De…nition 76 A transformation T : Vn ! Vm is said to be one–to–one
if each vector y 2 Vm is the image of at most one vector x 2 Vn .
De…nition 77 A transformation T : Vn
one–to–one is called a bijection.
! Vm that is both onto Vm and
Remark 78 In reference to the “dart” analogy that we have been using in
class: Supposing that Vn is our box of darts and Vm is our dartboard, a
transformation T : Vn ! Vm is onto Vm if every point on the dartboard
gets hit by at least one dart; whereas T is one–to–one if every point on the
dartboard either does not get hit or gets hit by exactly one dart. If T is a
bijection, then every point on the dartboard gets hit by exactly one dart.
2.2: The Inverse of a Matrix
a b
is a 2 2 matrix. The quantity
c d
bc is called the determinant of A. It is denoted either by det (A) or by
De…nition 79 Suppose that A =
ad
jAj.
De…nition 80 The n n identity matrix, denoted by In , is the matrix
that has entries of 1 at all positions along its main diagonal and entries of 0
elsewhere.
26
Example 81 The 3
3 identity matrix is
2
3
1 0 0
I3 = 4 0 1 0 5 .
0 0 1
De…nition 82 An n n matrix, A, is said to be invertible if there exists
an n n matrix, B, such that BA = AB = In .
Proposition 83 If A is an invertible n n matrix and B and C are n n
matrices such that BA = AB = In and CA = AC = In , then B = C. In
other words, if A has an inverse, then this inverse is unique.
De…nition 84 Suppose that A is an invertible n n matrix. The inverse
of A is de…ned to be the unique matrix, A 1 , such that A 1 A = AA 1 = In .
Example 85 The inverse of the matrix
A=
is
A
1
=
3 0
1 3
1
3
0
1
9
1
3
.
(The reader should perform the computations to check that A 1 A = AA
I2 .)
1
=
De…nition 86 An elementary matrix is a matrix is a matrix that can
be obtained by performing a single elementary row operation on an identity
matrix.
Example 87 The matrix
2
3
1 0 0
E=4 0 1 0 5
9 0 1
is an elementary matrix because it is obtained from I3 by the elementary row
operation 9 R1 + R3 ! R3 .
27
2.3: Characterizations of Invertible Matrices
De…nition 88 A transformation T : Vn ! Vn is said to be invertible if
there exists a transformation S : Vn ! Vn such that S (T (x)) = x for all
x 2 Vn and T (S (x)) = x for all x 2 Vn .
Proposition 89 If T : Vn ! Vn is invertible, then there is a unique transformation S : Vn ! Vn such that S (T (x)) = x for all x 2 Vn and
T (S (x)) = x for all x 2 Vn . This unique transformation is called the
inverse of T and is denoted by T 1 .
Example 90 If T : V2 ! V2 is the linear transformation that rotates all
vectors in V2 counterclockwise about the origin through an angle of 90 , then
T is invertible and T 1 : V2 ! V2 is the linear transformation that rotates
all vectors in V2 clockwise about the origin through an angle of 90 .
Theorem 91 Suppose that A is an n n matrix and let T : Vn ! Vn be
the linear transformation de…ned by T (x) = Ax for all x 2 Vn . Then the
following statements are all equivalent to each other (meaning that all of the
statements are true or all of them are false).
1. A is invertible.
2. T is invertible.
3. T is onto Vn .
4. T is one–to–one.
5. A
In .
6. The columns of A span Vn .
7. The homogeneous equation Ax = 0n has only the trivial solution.
8. For every b 2 Vn , the equation Ax = b has a unique solution.
28
4.1: Vector Spaces and Subspaces
De…nition 92 A vector space is a non-empty set of objects, V , called vectors, on which are de…ned two operations, called addition and multiplication
by scalars (real numbers), subject to the ten axioms listed below. The axioms
must hold for all vectors u; v; and w in V and for all scalars c and d.
1. The sum of u and v, denoted by u + v is in V .
2. The scalar multiple of u by c, denoted by cu, is in V .
3. u + v = v + u.
4. (u + v) + w = u + (v + w).
5. There is a zero vector, 0, in V such that u + 0 = u.
6. There is a vector
u in V such that u + ( u) = 0.
7. c (u + v) = cu + cv.
8. (c + d) u = cu + du.
9. c (du) = (cd) u.
10. 1u = u.
De…nition 93 If V is a vector space and W is a non–empty subset of V ,
then W is said to be a subspace of V if W is itself a vector space under the
same operations of addition and multiplication by scalars that hold in V .
De…nition 94 Suppose that v1 ,v2 ,. . . ,vn are vectors in a vector space, V ,
and suppose that c1 ,c2 ,. . . ,cn are scalars. Then
c1 v1 + c2 v2 +
+ cn vn
is said to be a linear combination of the vectors v1 ,v2 ,. . . ,vn .
De…nition 95 Suppose that fv1 ; v2 ; : : : ; vn g is a set of vectors in a vector
space, V . This set of vectors is said to be linearly independent if the
equation
c1 v1 + c2 v2 +
+ cn vn = 0
has only the trivial solution (c1 = c2 =
= cn = 0). Otherwise the set of
vectors is said to be linearly dependent.
29
De…nition 96 Suppose that S = fv1 ; v2 ; : : : ; vn g is a set of vectors in a
vector space, V . The span of S, denoted by Span (S), is de…ned to be the
set of all possible linear combinations of the vectors in S.
4.2: Linear Transformations
De…nition 97 Suppose that V and W are vector spaces. A transformation
T : V ! W is said to be a linear transformation if
T (u + v) = T (u) + T (v)
for all u and v in V
and
T (cu) = cT (u)
for all u in V and all scalars c.
Remark 98 As in our previous discussions of transformations, the vector
space V is called the domain of T and the vector space W is called the
codomain (or target set) of T .
Remark 99 In the above de…nition, since V and W might be two di¤erent
vectors spaces and hence might have two di¤erent operations of addition and
multiplication by scalars, we might want to use di¤erent symbols to denote
these di¤erent operations. For example we might use + and to denote
addition and scalar multiplication in V and use and to denote addition
and scalar multiplication in W . With these notations, the conditions for T
to be a linear transformation would appear as
T (u + v) = T (u)
T (v)
for all u and v in V
and
T (c u) = c
T (u)
for all u in V and all scalars c.
In most cases it is not necessary to use di¤erent notations such as this since
the operations that are intended are clear from the context of whatever is
being discussed.
De…nition 100 The nullspace (also called the kernel) of a linear transformation T : V ! W is the set of all vectors in V that are transformed
by T into the zero vector in W . This space (which is a subspace of V ) is
denoted by ker (T ). Thus
ker (T ) = fv 2 V j T (v) = 0W g .
30
De…nition 101 The range of a linear transformation T : V ! W is the
set of all vectors in W that are images under T of at least one vector in V .
This space (which is a subspace of W ) is denoted by range(T ). Thus
range (T ) = fw 2 W j T (v) = w for at least one v 2 V g .
Another (shorter) way to write this is
range (T ) = fT (v) j v 2 V g .
De…nition 102 In the case of a linear transformation T : Vn ! Vm whose
standard matrix is the m n matrix A, the nullspace of A, denoted by
Nul(A), is de…ned to be the nullspace (or kernel) of T . Thus Nul(A) =
ker (T ).
Remark 103 Nul(A) is the solution set of the matrix equation Ax = 0m .
De…nition 104 In the case of a linear transformation T : Vn ! Vm whose
standard matrix is the m n matrix A, the column space of A, denoted by
Col(A), is de…ned to be the range of T . Thus Col(A) =range(T ).
Remark 105 If A = [a1 a2
an ], then Col(A) = Span (fa1 ; a2 ; : : : ; an g).
4.3: Bases
De…nition 106 A basis for a vector space, V , is a …nite indexed set of
vectors, B = fv1 ; v2 ; : : : ; vn g, in V that is linearly independent and also
spans V (meaning that Span (B) = V ).
Remark 107 Some vector spaces have “standard” bases – meaning bases
that are rather obvious without having to do to much checking. For example,
the standard basis for Vn is B = fe1 ; e2 ; : : : ; en g where
2 3
2 3
2 3
1
0
0
6 0 7
6 1 7
6 0 7
6 7
6 7
6 7
e1 = 6 .. 7 , e2 = 6 .. 7 ,
, en = 6 .. 7 .
4 . 5
4 . 5
4 . 5
1
0
0
31
The standard basis for Pn (R) (the set of all polynomial functions with domain
R and degree less than or equal to n) is B = fp0 ; p1 ; p2 ; : : : ; pn g where
p0 (x) = 1 for all x 2 R
p1 (x) = x for all x 2 R
p2 (x) = x2 for all x 2 R
..
.
pn (x) = xn for all x 2 R.
The vector space F (R) (the set of all functions with domain R) does not
have a basis because no …nite set of functions spans this space.
4.4: Coordinate Systems
De…nition 108 Suppose that V is a vector space, that B = fv1 ; v2 ; : : : ; vn g
is a basis for V , and that v is a vector in V . The coordinates of v relative
to the basis B are the unique scalars c1 ; c2 ;
; cn such that c1 v1 + c2 v2 +
+ cn vn = v. The coordinate vector of v relative to the basis B is the
vector
2
3
c1
6 c2 7
6
7
[v]B = 6 .. 7 2 Vn .
4 . 5
cn
The transformation T : V ! Vn de…ned by T (v) = [v]B is a linear transformation that is called the coordinate transformation (or coordinate
mapping) on V determined by the basis B.
De…nition 109 If V and W are vector spaces and T : V ! W is a linear
transformation that is one–to–one and maps V onto W , then T is said to be
an isomorphism of V and W .
De…nition 110 If V and W are vector spaces and if there exists an isomorphism T : V ! W , then V is said to be isomorphic to W .
Remark 111 Isomorphism is an equivalence relation on the set of all vector
spaces. This is because:
32
1. Every vector space is isomorphic to itself. (Why?)
2. If V is isomorphic to W , then W is isomorphic to V . (Why?)
3. If V is isomorphic to W and W is isomorphic to X, then V is isomorphic to X. (Why?)
4.5: The Dimension of a Vector Space
De…nition 112 If V is a vector space that is spanned by a …nite set of
vectors, then V is said to be …nite–dimensional and the dimension of V ,
denoted by dim (V ), is de…ned to be the number of vectors in a basis for V .
The dimension of the zero vector space, V = f0g, is de…ned to be 0. (Note
that the zero vector space has no basis.) If V 6= f0g and V is not spanned by
any …nite set of vectors, then we say that V is in…nite–dimensional.
4.6: The Rank of a Matrix
De…nition 113 If A is an m n matrix, then the rank of A, denoted by
rank (A), is de…ned to be the dimension of the column space of A. Thus
rank (A) = dim (Col (A)).
Remark 114 Since we know that the column space and the row space of any
matrix have the same dimension, it is also true that rank (A) = dim (row (A)).
Direct Sums of Subspaces
De…nition 115 If V is a vector space and H and K are subspaces of V such
that H \ K = f0g, then the direct sum of H and K, which is denoted by
H K and which is also a subspace of V , is de…ned to be
H
K = fu + v j u 2 H and v 2 Kg .
Remark 116 In order for the direct sum, H K, to be de…ned, it is necessary that H \ K = f0g. (This is part of the de…nition of direct sum.)
Whether or not H \ K = f0g, the set
H + K = fu + v j u 2 H and v 2 Kg
33
is called the sum of H and K. H + K is a subspace of V whether or not
H \ K = f0g. The di¤erence is that when H \ K = f0g (and hence H + K =
H K), each vector, w, in H K can be written uniquely in the form
w = u + v where u 2 H and v 2 K; whereas this unique representation
might not be possible for all vectors in H + K if H \ K 6= f0g.
De…nition 117 Two subspaces, H and K, of a vector space V are said to
be complementary to each other if H K = V .
De…nition 118 Two vectors, u and v, in Vn are said to be orthogonal to
each other if uT v = vT u = [0].
Remark 119 If u and v are any two vectors in Vn , then uT v = vT u. Thus
we could simply state the above de…nition by saying that u and v are orthogonal to each other if uT v = [0].
Remark 120 If [a] is a 1 1 matrix, then we often adopt the convention
to just view [a] as a scalar and hence just write a instead of [a]. With this
convention, we can say that u and v are orthogonal to each other if uT v = 0
(without writing the brackets around 0). Also, with this convention, uT v (for
any two vectors, u and v, in Vn ) is called the dot product of u and v and is
denoted by u v. For example, if
2
3
2
3
4
1
u = 4 1 5 and v = 4 1 5 ,
4
4
then
u v = uT v =
4
1
2
4 4
3
1
1 5 = (4) ( 1)+( 1) ( 1)+( 4) ( 4) = 13.
4
De…nition 121 Two subspaces, H and K, of Vn are said to be orthogonal
to each other if uT v = 0 for all u 2 H and v 2 K.
De…nition 122 Two subspaces, H and K, of Vn are said to be orthogonal
complements of each other if H and K are orthogonal to each other and
H K = Vn .
34
Fundamental Subspaces of a Matrix
Let
A = [a1
2
6
6
an ] = 6
4
a2
b1
b2
..
.
bm
3
7
7
7
5
be an m n matrix (where a1 , a2 , . . . , an are the column vectors of A (each
of which has size m 1) and b1 , b2 , . . . , bm are the row vectors of A (each
of which has size 1 n). The following four subspaces (two of which are
subspaces of Vn and the other two of which are subspaces of Vm are called
the fundamental subspaces of the matrix A:
1. The row space of A,
row (A) = Span bT1 ; bT2 ; : : : ; bTm
is a subspace of Vn .
2. The null space of A,
N ul (A) = fx 2 Vn j Ax = 0m g
is a subspace of Vn .
3. The column space of A,
Col (A) = Span (a1 ; a2 ; : : : ; an )
is a subspace of Vm .
4. The null space of AT ,
N ul AT = x 2 Vm j AT x = 0n
is a subspace of Vm .
35