J
Chapter
LINEAR FUNCTIONS
You
probably recall from calculus that a function is a rule which associates
particular values of one variable quantity to particular values of another
variable quantity. Analysis is that branch of mathematics devoted to the
study, or analysis, of functions. The main kind of analysis that goes on is
this: for small changes in the first variable, we try to determine an approxi
mate value to the corresponding change in the second.
Now, we ask, for
in
the
first
variable
to
what
extent
can
we
large changes
predict, from such
the
in
the
second?
The primary
approximations,
corresponding change
involved
in
this
kind
of
is
of
the problem.
technique
analysis
simplification
That is, we replace the given function by a suitable very simple and more
easily calculable function and work with this simple function instead (making
sure to keep in mind the effect of that replacement).
The simplest possible functions are those which behave linearly. This
means that they have a straight line as graph.
Such a function has the
in
the
value
of
the function correspond
The
increment
following property.
to
an
in
is
a
constant
increment
the
variable
ing
multiple of that increment :
f(x
for
some
+
C.
t)-f(x)
=
Ct
Now, when
(1.1)
one moves
to the consideration of
functions of
simplest functions becomes
study
quantities
mathematical
it
forms
a
that
discipline, called linear
special
complex enough
with
the concepts and
of
one
calculus
The
variable,
coupled
algebra.
several variable
the
of
even
these
1
2
1.
Linear Functions
algebra constitute the basic tools of analysis of functions
of several variables.
It is our purpose in this text to study this subject.
First then, we must study the notions and methods of linear algebra.
We begin our study in a familiar context: that of the solution of a system
of simultaneous equations in more than one unknown. We shall develop
a standard technique for discovering solutions (if they exist), called row
reduction.
This is the foundation pillar of the theory of linear algebra.
After a brief section on notational conventions, we look at the system of
equations from another point of view. Instead of seeking particular solu
tions of a system, we analyze the system itself. This leads us to consider the
fundamental concept of linear algebra: that of a linear transformation. In
this larger context we can resolve the question of existence of solutions and
effectively describe the totality of all solutions to a given linear problem.
We proceed then to analyze the totality of linear transformations as an
object of interest in itself. This chapter ends with the study of several
important topics allied with linear algebra. We study the plane as the system
of complex numbers and the inner and vector products in three space.
techniques
1.1
Let
of linear
Simultaneous
us
Equations
begin by considering
a
well-known
to systems of simultaneous linear equations.
is that of two equations in two unknowns.
problem : that of finding solutions
The simplest nontrivial example
Examples
The
technique
ables.
for solution is that of elimination of
This is
one
of the vari
accomplished by multiplying the equations by appro
priate nonzero numbers and adding or subtracting the resulting
equations. This is quite legitimate, for the set of solutions of the
system will not be changed by any such operations. It is our intention
to select such operations so that we eventually obtain as
equations:
x
something, y
something. In the present case this is quite
easy: if we add five times the second equation to the first, y will
conveniently disappear:
=
=
8* +
35x
43x
-
5y
5y
=
3
=
40
^43
and
obtain the
we
equation gives
solution. Let
2.
5y
try
us
a
3,
=
few
x
=
y
=
or
more
1
that in the first
Substituting
.
Then
1.
illustrative
x
=
1,
y
=
1 is the
examples.
3x-2y= 9
11
+ 3y
-x
We
equation
8 +
3
Simultaneous Equations
1.1
=
eliminate
can
as
x
follows:
multiply
the second
equation by
3
and add :
3x-2y= 9
33
9y
-3x +
=
7y
We obtain
3.
v
3x +
6x +
If
we
42
=
6 and
=
7.
x
1
4y=
8y
n~
'
v
'
15
=
subtract twice the first
equation
from the second,
we
obtain
a mess:
6x + Sy= 15
14
-6x-8y
^ 4x
=
0=
Thus there
because
can
be
they imply
if the second
6x +
Sy
then
our
but
1
=
equation
x
+
3x-2y
2x +
y
z
=
equation
information.
We
causes
0
can
0 which is true,
conclude that our
=
always produce results.
for this in Section 1.3.
z
=
more
5
+ 5z= -I
-
new
generalize our technique to systems involving
Consider, for example, the system
us now
y +
would lead to the
of elimination does not
We shall go into the
variables.
were
offers much
simple technique
4. Let
Equation
and y satisfying Equations (1.3),
(1.4) which is patently false. Notice,
x
14
technique
hardly
numbers
no
the
0
(1.5)
4
1.
Linear Functions
equation expresses x in terms of y and z; if the second and
equation were free of x we could solve as above for the two
The first
third
unknowns y, z and then use the first to find x. But now it is easy to
replace those last two equations by another two which must also be
satisfied and which are free of the variable x. We use the first to
eliminate
from the latter two.
x
Namely,
subtract three times the
first from the second:
3x
-
3x +
2y
3y
+ 5z
=
+ 3z
=
1
-
15
5v + 2z= -16
-
and twice the first from the third :
2x +
2x +
=
0
+ 2z=
10
z
y
2y
y-3z
-
The system
x
+
y +
-5y
-
y
and
-10
=
has been
(1.5)
z
replaced by
this
new
5
=
+ 2z=-16
-
3z
=
(1.6)
10
-
way clear to the end.
in
two
unknowns
:
system
we can now see our
as a
system:
We solve the last two
-5y+ 2z= -16
50
5y+15z
=
34~
17z=
2
z=
Then, substituting this value in the last equation,
10
first
z
=
5.
or
y
=
4.
equation,
Finally, substituting
we
find
x
=
2.
x
y
2x + y
-
-4x-y+
z
=
5
3z
=
0
z=
10
-
1.
we
obtain
these values for y and
Thus the solution is
x
=
6
y
-
z
1,
=
in the
y
=
4,
1.1
Eliminate
x
multiples
of the
2x +
y
from the second and third
3z
-
-4x
4x
4y
5y
-
The
x
-
-
-
3y5y
-
z
=
10
=
20
3z
=
30
has been
=
5
-10
3z
10
-
z=
z
We solve the last two
Of course,
we
replaced by these equations :
30
=
into the first
equations by adding appropriate
10
4z
given system
y
5
0
=
=
y +
-
-
Equations
first;
2x-2y-2z=
z
3y
-
Simultaneous
easily: y
equation completes
can
run
=
-30/7,
z
=
the solution:
into difficulties
as we
-20/7. Substitutions
15/7.
x
=
did in the two unknown
equations of Example 3. We should be prepared for such occurrences
and perhaps even more mysterious ones.
Nevertheless, our technique is
productive : if there is a solution to be had we can locate it by this process
of successive eliminations. Furthermore, it easily generalizes to systems
with more unknowns. This is the technique stated for the case of n un
knowns. Eliminate the first variable from all the equations except the first
by adding appropriate multiples of the first. Then, we handle the resulting
1 unknowns.
That is, using the second equation
equations as a system in n
we can eliminate the second variable from all but the second equation, using
the new third equation we can eliminate the third variable from the remain
ing equations, and so forth. Eventually we run out of equations and we
ought to be able to find the desired solution by a succession of substitutions.
We shall want to do
to be able to
systems, and
predict
we
more
than discover solutions if
the existence of solutions ;
want to know in
some sense
we want
they exist.
We want
to be able to compare
how many solutions there
we should come to understand the nature of
are.
given system
words,
equations. In order to do that we have to analyze this technique and
develop a notation and theory which do so. That is where linear algebra
begins. Before going into this, we study another pair of more complicated
examples.
In other
of
a
6
1.
Linear Functions
Examples
6.
x
5x
2y
+
-
-
y
y +
3x +
According
13
z
3w
=
z
+ 2w
=
z
+
=
4
=
-7
w
2y-2z
technique,
to our
14
-
adding the suitable multiple
(-5)
(first)
(first)
(first)
x
0
x
(-3)
x
Now
we
+ second:
+ third
+ 4z + 17w
+ fourth:
4y
-
solve this set
+
=
-79
+
w
=
4
z+
9w
=
-46
z
by applying the
equations by a
by
We do this
x
y +
:
the last three
does not appear.
of the first equation:
lly
-
replace
we
set in which the variable
new
same
procedure:
we
now
eliminate y.
Of course the order of the equations is not relevant;
we could have listed them some other way.
Since we can avoid
fractions by adding multiples of the second equation to the first and
third, let's do it that
(11)
+ first:
(second)
(second)
x
4
x
way.
15z + 28 w
=
-35
5z + 13w
=
-30
+ third:
Finally, of this set, (-3) x (second) + first gives
the original set of four equations is replaced by this
x
+
2y
y +
z
3w
=
13
+
w
=
4
z
-
live
=
55.
Thus
set:
5z+ 13w= -30
-llw
The solutions
w
5
=
7.
x
z
2y
easily found,
are now
=
Now, let
+
55
=
7
us
+ 3z +
y
=
u-
v
4u + 3v
3z
x=l
consider this set:
-5x+ y + lz
2y+
2
u
v
=
2
=
-5
=
18
=
5
t1,7)
1.1
x
is
eliminated from the last two
already
to eliminate
from the second,
of the last three above,
place
1
x
22z + 5m
\y +
2y+
Simultaneous Equations
5v
=
5
4u + 3v=
18
3z
-
u
v
we
7
Using the first
equations in
equations.
obtain these three
(1.8)
5
=
Now y is already eliminated from the last. We eliminate it from the
second (without getting involved with fractions) in this way :
(-2)
x
(first)
Now this
+
(11)
(second):
x
-44z + 34u + 43y
188
=
with the last of the set
equation together
(1.8) gives this
system
-44z + 34u + 43v= 188
3z
We
can
eliminate
the system
x
2y
lly
+
v
u
+
(1.7)
5
=
v
from the first to obtain 129z
3z +
v
=
+ 22z + 5w-5i>
=
3z-
u
v=
u-
we can
=
27.
Thus
2
5
K
5
-
'
=-27
\29z-9u
Now
9u
has been transformed into this:
solve for
by
x
the first
equation
once we
know y, z, u, v;
solve for y by the second once we know z, u, v; we can solve
in the third once we know z and u; and we can use any z, u which
we can
for
v
make the last
For
true.
equation
example,
if
z
=
0,
we
must have
1
Notice that for
0, x
2, y
up the line : v
that
make
these equations
find
can
of
z
we
value
u,
v,
x,
y
always
any
all hold. Thus in this case there is more than one solution.
u
=
3, and
Formulation
Now, let
=
so on
of the Procedure:
us
=
=
.
Row Reduction
turn to the abstract
formulation of this procedure.
In the
general case we will have some, say m, equations in n unknowns. Let
x". These m equations may be written
refer to the unknowns as x1,
.
.
.
,
+
a21x2
a22x2
+
+
a1x"
an2x"
+
a2mx2
+
+
a,"*"
a^x1
a^x1
+
aimxl
+
---
+
=
b1
=
b2
=bn
us
as
(1 10)
8
1.
We
Linear Functions
proceed
to solve this
a2\a^
x
al3/a1 and
system
multiply the first equation
equation; multiply the first equation by
follows:
as
and add it to the second
by
which
a^x1
We
now
=
+
a21x2
+
+
<x2x
+
+ aV
tx2mx2
+
+
continue with the
equations
equation).
computed
a/
add it to the third and
The result will be
so on.
in
n
-
ajxr
ocnmx"
=
=
p2
(1.11)
P"
technique applied
same
to the
This is
now
an
introduce
equation
must be
a
x" are known.
x2,
modified.
We just
slightly
once
1
Of course, if
...,
formalism which allows
It is clear that the
Equations (1.10)
m
given by the system (1.11) (except for the first
effective reduction of the problem, because x1 can be
from the first
procedure.
system of
1 unknowns
0, this technique
us
to
renumber the
nonzero
in
no
keep
and then
equation
so
track of this
of the left side of the system of
is embodied in the array of numbers.
/V 021
a
A
system,
b1
=
equations so that the coefficient of x1 in the first one is
proceed as above. If that is impossible then x1 appears
we can disregard it and work with x2 instead.
We
a new
may write this way:
we
essence
2\
=
(1.12)
amJ
This array is called a matrix: the
upper index of the general term is the
index and the lower index is the column index.
Thus, a53 is the number
in the third row and fifth column,
a\2 is in the seventh row and
row
column, akJ is the number in the jth row
(1.12) is an m x n matrix: it has m rows and
b
is
forty-second
and the kth column.
n
columns.
Symbol
The matrix
=
an m x
Ax
1 matrix.
=
b
Equations (1.10)
can now
be written
symbolically
as
(1.13)
Simultaneous Equations
1.1
Now the
technique
for
solving
the
equation
described above consists of
sequence of such equations with new matrices A and b, ending with
solution is obvious. The step from one equation to the next is
by
operation (remember, the
of these particular steps :
a row
one
rows are
Step 1 Multiply a row by a nonzero
Step 2. Add one row to another.
Step 3. Interchange two rows.
.
It is clear
change
not
(and
a
/l 2*
0
0
\
a
whose
performed
equations) ; that is,
1.3) that any such operation does
Finally, the end result desired is a
will be verified in Section
matrix of this form, called
1
the separate
one
constant.
the collection of solutions.
0
9
row-reduced matrix:
^
(1.14)
/
:
Descriptively : the first nonzero entry of any row is a 1 and this 1 in any
is to the right of the 1 in any previous row. This is the kind of matrix
the above procedure leads to ; and it is most desirable because the system it
row
immediately solved. In order to see this, we shall distin
guish between two cases by resolving the dotted ambiguity in the lower right
corner of (1.14).
represents
can
be
Let
Ax
be
a
=
b
system of linear
where A is
Let d be the number of
(1.14)).
Case 1
equations
.
d
=
In this
n.
xl+a21x2
x2
case
a32x3
the system of
+
+
x"-1
row-reduced matrix
nonzero rows
+--- +
+
a
+
an1x
a2x
cr-1xn
equations
=
bl
=
b2
=
bn-1
xn
=
b"
0
=
bn+1
0
=
Z>'
(of the
of A.
has this form :
form
10
Linear Functions
1.
solution if and only if b"+l
is found by successive substitutions: In this
Thus there is
=
In this
d<n.
Case 2.
xl
a
+
a^x2
+ +
x2
xd
case our
+
=
case
bm
=
0, and the solution
the solution is
unique.
system has the form :
=
b1
=
b2
=
bd
0
=
bd+1
0
=
bm
aV
+ a3 2x3 +
+
a2x"
add+1 'xd+l---
+
andxn
(We may have to reindex the variables in order to get all the leading l's in a
bm
0, and all the
line.) There is a solution if and only if bd+1
solutions are obtained by giving xd+1, ...,xn arbitrary values, and finding
the values of the remaining variables by substitutions.
We now summarize the factual (rather than the procedural) content of this
discussion in a theorem, the proof of which will appear in Section 1.3.
=
=
A matrix A is called
Definition 1.
a
=
row-reduced matrix if
(i) the first nonzero entry in any row is 1,
(ii) in any row this first 1 appears to the right of the first
1 in any
preceding
row.
The number of
of A is called its index.
nonzero rows
Theorem 1.1.
reduced form A'
Let A be
an
succession
m x n
matrix.
A
by
precisely the same solutions as
by the same sequence of row operations that led from
a
can
be
brought
=
=
EXERCISES
1
.
Find solutions for these systems
(a) 2x 3y 23
-
(b)
=
3x+ y=-4
10
hx + \y
=
-fx+8y= 0
(c)
x+ y+
z=
y+
z=
x-
2x-3y-5z
=
into
row-
b has
of
operations. The equation Ax
the equation A'x
b' ifV is obtained from b
row
15
3
-7
A to A'.
1.1
(d)
x+
y+ z+
x+ y+
y+
x
+
-x
(e)
w
=
w=2
z
h>=0
=
11
4
z
2y-3z + 4w
Simultaneous Equations
2
z=
-x+ y+
0
2x + 2y+10z
28
=
x+
y+
z
(f )
3x +
(g)
x+
y
=
x
y
=
\
=
0
x
=
3x
(h)
22
=
12
6y + 9z
+ 2y + 3z= 4
4y
x+ y
x + 2y
7
=
7
=
9
x+3y =11
(i)
jc
+ y + z+v
4
=
z w
6
+y
x + 2y+
z=
0
x
(j)
=
x
4
3y 6z
ll
4x+8y + 4z
2. A homogeneous system of linear equations is a system of the form
Ax
0; that is, the right-hand side is zero. Find nonzero solutions (if
possible) to these homogeneous systems.
(a) x+ y + z 0
=
=
=
=
y +
X-
z
0
+ 2y + z
0
x+y + z
x
(b)
=
=
x
(c)
z
y
=
0
y+ z+ w =0
2y + z 2w=0
x+
x
2x
3.
0
=
y + 2z
Suppose (x1,
...,
w=0
x") is
a
solution for
Show that for every real number /, (tx1,
...,
given homogeneous system.
tx") is also a solution.
a
x"), (y1, ...,y) are solutions for a given homogeneous
x" + y).
(xl + y1,
5. Find the row-reduced matrix which corresponds to the given matrix
according to Theorem 1.1.
4. If
(x\
system, then
.
.
so
,
is
.
0
7
1\
3
2
2
\-l
6
/
A=
B
.
.
.
,
4/
'l
0
0
6
5^
2
3
0
0
0
=
1
i0
0-1-1
0
0
2
1
0y
12
Linear Functions
1.
1\
2
6
-2
-4
0
2
0
0
8
8
3
6
9
12/
(1
i)
I
6. Let
^
i)
equations:
Ax=b, (b)
Solve these
(a)
(e)
Cx
=
(c)
Bx=a,
c, where A, B, C
are
Bx
given
=
(d)
c,
Cx
=
a,
in Exercise 5.
PROBLEMS
1. Show that the system
ax
+
by
=
cx
+ dy
=
has
a
a
fi
solution
no
matter what a,
jS are
if adbc^ 0, and there is only
one
such solution.
2. Can you suggest
by Example 3 ?
an
explanation of the ugly phenomenon illustrated
only one row-reduced matrix to which a given matrix may be
operations? If A' and A" are two such row-reduced
matrices, coming from a given matrix A show that they must have the same
3. Is there
reduced by
row
index.
4. Suppose we have a system of n equations in n unknowns, Ax
b.
row reduction the index of the row-reduced matrix is also n.
Show
=
After
that in this case the equation Ax
b always has one and only one solution
for every b.
5. Suppose that you have a system of m equations in n unknowns to
solve. What should you expect in the way of existence and uniqueness
of solutions in the cases m<n,m>nl
=
6.
Suppose we are given the n x n system Ax b, and all the rows of A
s" such that a/
multiples of the first row; that is, there are s1,
s'af
for all j and i
n.
Under what conditions will the given system
1,
=
are
.
=
have
a
.
.
.
.
.
=
,
,
solution ?
7. Suppose instead that the columns of A are multiples of the first column.
Can you make any assertions?
8. Verify that if the columns of a 3 x 3 matrix are multiples of the first
column, then the rows are multiples of one of the rows.
1.2
1.2
Numbers, Notation, and Geometry
13
Numbers, Notation, and Geometry
We
now
interrupt
our
discussion of simultaneous
equations
in order to
introduce certain facts and notational conventions which shall be in
use
throughout this text. We shall also describe the geometry of the plane from
the linear, or vector point of view as an alternative introduction to linear
algebra.
First of all, the collection {1, 2, 3,
.} of positive integers (the "counting
numbers ") will be denoted by P. Every integer n has an immediate successor,
If a fact is true for the integer 1 and also holds for the
denoted by n + 1
successor of every integer for which it is true, then it is true for all integers.
This is the Principle of Mathematical Induction, which we shall take as an
axiom, or defining property of the integers. We shall formulate it this way.
.
.
.
Principle of
properties:
(i)
(ii)
1 is
a
Mathematical Induction.
Let S be
a
subset of P with these
member of S,
whenever
a
particular integer
n
is in S,
so
also is its
successor n
+ 1
inS.
Then S must be the set P of all
This assertion is
intuitively
positive integers.
clear.
You
can
1 + 1
see, for
example
that 2 is in 5.
2 is also in 5.
Continuing,
(ii)
By applying (ii) another time 4 is in S.
Applying (ii) another 32 times we see that all the integers up to 36 are also
in S. No positive integer can escape : since 1 is in 5" we need only apply (ii) n
times to verify that the integer n is in S. In fact, the assertion of the principle
of mathematical induction is that there are no integers other than those that
can be captured in this way, and in this sense the principle is a defining
For by (i) 1 is in S, and thus by
3
2+1 is in S, again by (ii).
=
=
property of the integers.
of mathematical induction provides us with a tool for writing
for all positive integers which avoids the phrases:
assertions
proofs of
this
in
way," and so forth," "...,".... We shall find this a
continuing
in
helpful device verifying assertions concerning problems with an unspecified
number, n, of unknowns. Let us illustrate this method by proving a few
The
"
principle
"
propositions about integers.
14
Linear Functions
1.
The
1.
Proposition
of the first
sum
Let S be the set of
Proof.
n
(l/2)(
is
integers
integers for which Proposition
1).
+
1 is true.
Certainly
1 is in S:
1=*- 1(1 + 1)
Now, assuming the assertion of Proposition 1 for
holds for
+ 1
n
l +
+ 7j+l=l +
---
=
+
---
M
(n+ l)(in
Proposition 2.
The
l2
surely.
1
Proof.
=
+ 1
n
1 + 3 +
sum
1)
=
show that it also
we
n,
We
Thus
now assume
n
+ 1
K + \)(n + 2)
by
the
the
of mathematical in
principle
of the first n odd integers
is
n2.
proposition for
any n, and show that
:
+ 2(n + 1)
-
1
=
=
Proposition 3.
we can
integer
+ +l= in(n + 1) +
+
which is the appropriate conclusion.
duction, Proposition 1 is proven.
it follows for
any
.
Let K be
a
1 + 3 +
1 + In + 1
+ 2n
n2 + 2n+l=(n+l)2
-
given positive integer.
Then
for
integer
any
n=QK+R
with 0
<
R
<
K in
one
(U5)
and
only
one
way.
Proof. We may of course immediately discard the case K
1 for in that
(1 1 5) is just the trivial comment that n
n
1 for all n.
Thus take K > 1 and
proceed by mathematical induction. The proposition is true for n
1 :
=
case
=
.
,
=
\=0K+\
Now
we assume
n
for
n
write
some
n
=
that the proposition is true for
any
given integer
n.
Thus
QK+R
Q and R,0^R<K.
+ 1
=
QK+ (R + 1)
ThcnR+l^K.
If R + 1<
K,
we
have
now
1.2
with 0
< R
+ 1
n+l
as
=
<
K,
as
desired.
QK+K
=
Numbers, Notation, and Geometry
Otherwise,
R + 1
=
K, in which
case
(Q+ \)K+ 0
desired.
This
Thus, by mathematical induction, (1.15) is possible for
representation is unique, for if
=
n
is also
15
every
integer
n.
Q'K+R'
possible
with 0 <, R' < K, then
Q'K+R'
=
we
have
QK+R
or
(Q'-Q)K
R-R'
R' is between K and K.
and R
K is 0,
=
so
(Q!
-
Q)K
=
Now the
only multiple of
R-R'=0 from which
we
conclude R
K between
=
R', Q
=
K and
Q.
Set Notation
The set of
positive integers forms a subset of a larger number system, the
integers. Z consists of all positive integers, their negatives and 0.
The collection of all quotients of members of Z is the set of rational numbers,
denotedy by Q. Q is a very large subset of the set of all real numbers R.
For the purposes of geometric interpretation we will conceive of the real
number system R as being in one-to-one correspondence with the points
on a straight line.
That is, given a straight fine, we fix two points on it,
All
one is the origin O, and the other denotes the unit of measurement.
:
it
is
the
a
numerical
value
other points P on the line are given
displacement
from O as measured on the given scale (negative if O is between P and 1
and positive otherwise). (See Figure 1.1.)
set Z of all
There
are
certain ideas and notations in connection with sets which
Figure
1.1
we
16
Linear Functions
/.
shall standardize before proceeding. Customarily, in any given context
there is one largest set of objects under consideration, called the universe
(it may be the positive integers P, or the rabbits in Texas, or the people on
the
x
is
moon) and
an object
that
means
x
all sets
we
is not
The set with
no
specific
are
sets
actually subsets
are
shall write xelto
member of X.
a
is
x
a
set and
member of X.
a
Thus, for example,
-
7
e
x$X
Z, but -7 $P.
elements is called the empty set, and is designated 0. Most
defined by a property : the set in question is the set of all
We
elements of the universe that have that property.
hand form to represent that phrase
{xeU:
If X is
of this universe.
mean :
has that
x
use
the
following short
property}
example, the set of all positive real numbers is {x e R: x > 0}. The set
Englishmen who drink coffee is {x e England: x drinks coffee}. The
This is the same
set of all integers between 8 and 18 is {x e Z: 8 < x < 18}.
13 1 < 5}.
as {xeP: 8 <x< 18} and {xeZ: \x
For
of all
-
If X and Y are two sets, and every element of X is an element of Y we shall
Notice that 0 c X for every
say that X is contained in Y, written X <= Y.
set X.
We shall consider also these operations on sets :
X: the set of all
x
not in X
X
u
Y: the set of all
X
n
Y: the set of all
x
in both X and Y
Y: the set of all
x
in X, and not in Y
X
1.2 for
(Consult Figure
the
same
-X
u
Y
a
Xn-Y.
as
-(X
Y),
x
in either X
or
Y (or
both)
pictorial interpretation.)
There
X
are
(Y
many
Z)
(X
other
Notice that X
identities:
Y is
-
X
=
X,
(X n Z), and so on,
so don't be surprised when two different collections of
symbols identify the
same set.
A final operation is that of forming the Cartesian
product. If U
is a given universe, then U x U is the set of all ordered
of
elements in U.
pairs
U x U is often denoted by U2.
By extension we can define U3 as the set of
all ordered triples (x1, x2, x3) of elements of
U; and more generally U" is the
set of all ordered -tuples of elements of U.
-
X1,
If
=
X"
n
n
u
=
n
Y)
u
subsets of U, the set of all ordered n-tuples (x1,
...,x")
e Xn is denoted X1 x
x X".
Not every subset of
V is of the form X1 x--- xX", those which are of this form
are called
with x1
...
e
,
X1,
.
.
are
.
,
xn
rectangles.
Thus the space of n-tuples of real numbers is denoted R". If
I"
I1,
intervals in R, then I1 x--- xl" is indeed a
rectangle. A point
(x
x") in R" will be denoted, when specific reference to its elements is
...,
are
,
.
.
.
,
1.2
Numbers, Notation, and Geometry
17
^^^
X U Y
required, by a single boldfaced letter x (xl,
(ft1,
b") are two points in R" with b' > a',
notation [a, b] to denote the rectangle.
not
b
=
...,
.
.
.
,
{(x1,
If the
.
.
.
,
x"): a1
inequalities
(a, b)
=
A function from
a
x1
strict
are
{(x\
<
.
.
.
,
b\
<
we
.
.
.
,
a"
<
xh
shall denote the
x"): a1
<
x1
<
b1,
.
.
.
If a
x").
I <i<n
=
=
we
(a1,
shall
...
a"),
,
use
the
b"}
<
rectangle by (a, b) :
,
a"
<
x"
<
b"}
set X to another set Y is a rule which associates to each
point y in Y. It is customary to avoid the
by defining a function as a certain kind of subset
of X x Y.
Namely, a function is a set of ordered pairs (x, y) with xe X,
If
y e 7, with each x e X appearing precisely once as a first member.
(x, y) is such a pair we denote y by f(x) : y =f(x). We shall use the notation
X is called
Y to indicate that /is to mean a function from X to Y.
/: X
If every
the domain of/; the range of /is the set {f(x): x e X} of values off.
point of y appears as a value off we say that/maps X onto Y. If every point
More
of y is the value off at at most one x in X, we say that/is one-to-one.
precisely,/is one-to-one if x # x' implies /(x) #/(x')- Now, if/is a one-toone function from X onto Y, then for each y e 7, there is one and only one
point
use
of
x a
of the
uniquely
new
determined
word rule
-
xeX such
that/(x)
=y.
This defines
one-to-one and onto and has this
In this
case we
shall say that
/
a
function #: Y-+X which is also
x if and only \ff(x)
y.
property: g(y)
=
is invertible and g is its
=
inverse, denoted
18
1.
g=f~l.
Linear Functions
Y, f2 : Y-^Zare two functions, we can compose
Finally, iffx : X
a third function, denoted /2 of^.X^Z defined by
->
them to form
fi /iW =/2(/i(*))
Plane
Geometry
geometric study of the plane, as an alternative intro
duction to linear algebra. According to the notion of the Cartesian co
ordinate system we can make a correspondence between a plane, supposed
to be of infinite extent in all directions, and the collection R2 of ordered
pairs of real numbers. This is done in the following way: first a point on the
plane is chosen, to be called the origin and denoted O (Figure 1.3). Then
two distinct lines intersecting at O are drawn (it is ordinarily supposed that
These lines are
these lines are perpendicular, but it is hardly necessary).
called the coordinate axes; they are sometimes referred to more specifically
as the x and y axes, ordered counterclockwise (Figure 1.4).
Now a point
is chosen on each of these axes; we call these Et and E2 (Figure 1.5).
These
are the "unit lengths" in each of the directions of the coordinate axes.
Having chosen a unit on these lines, we can put each of them in one-to-one
correspondence with the real numbers. Now, letting P be any point in the
plane, we associate a pair of real numbers to P in this way. Draw the lines
through P which are parallel to the coordinate axes and let x be the inter
section with the line through EL and y the intersection with the line through
We
now
turn to the
o
Figure
1.3
Figure
1.4
1.2
Numbers, Notation, and Geometry
19
Figure 1.5
E2
.
Then
we
identify
P with the
pair of real numbers (x, y) (Figure 1.6.)
In this way to every point in the plane there corresponds a point in R2
(called its coordinates relative to the choice O, Elt E2). Clearly, for any pair
of real numbers
the
fourth vertex of the
co
ordinates
axes
(x, y) we have a point with those coordinates, namely
parallelogram of side lengths x and y along the
with one vertex at O (Figure 1.6).
P(x.y)
Figure 1.6
20
Linear Functions
1.
Figure 1.7
particular point in the plane is fixed as the origin, there can be
defined
operations on the points of the plane, and these operations form
of
linear
tools
the
algebra. Since they cannot be defined on the points until
an origin is chosen, we are forced to distinguish between the point set the
plane and the plane with chosen point. This distinction gives rise to the
notion of vector : a vector is a point in the plane-with-origin. The vector
can be physically realized as the directed line segment from the origin to the
given point; such a visualization is nothing more than a heuristic aid. It is
important to realize that as sets, the set of vectors in the plane is the same
The difference is that the set of vectors has
as the set of points in the plane.
additional structure : a particular point has been designated the origin. We
shall denote vectors by boldface letters; thus the point P becomes the vector
P, the origin O becomes the vector 0. We shall now describe these two
operations geometrically and then compute them in coordinates.
1.
Scalar Multiplication. Let P be a vector in the plane, and r a real
number. Consider the line through 0 and P.
Considering P now as a unit
length, we can put that line into one-to-one correspondence with R. Using
this scale, rP is the point corresponding to the real number r.
Said differently,
rP is one of the points on that line whose distance from 0 is \r\ times the
distance of P from 0 (Figure 1.7). Now, if P has the coordinates (x, y) we
shall see that rP has coordinates (rx, ry). First, suppose r > 0. Draw the
Once
a
two
1.2
Numbers, Notation, and Geometry
21
triangle formed by the fine through 0, P and rP, and the E^ axis and the lines
parallel to the E2 axis (Figure 1.8). Triangles I and II are similar. Thus,
referring to the lengths as denoted in Figure 1.8,
Jpl
=
|rP|
i
s
By definition |P|/|rP|
is
coordinates
2.
of
a
=
l/r,
thus the first coordinate of rP (here denoted
The second coordinate is
rx.
(rx, ry).
The
case r
by s),
similarly seen to be ry. Thus rP has the
< 0 is only slightly more complicated.
Let P, Q be vectors in R2. Then 0, P, Q are three vertices
determined parallelogram. We define P + Q to be the fourth
Addition.
uniquely
description of this operation in terms of coordinates is extremely
(x, y), and Q has coordinates (s, t), then P + Q
simple:
has (x + s, y + t) as coordinates. There is nothing profound to be learned
from the verification of this fact, so we shall not go through it in detail.
After all, it is not our purpose here to logically incorporate plane geometry
vertex.
The
if P has coordinates
Figure
1.8
22
1.
Linear Functions
Figure 1.9
to use it as an intuitive tool for comprehen
suspicious of our assurances we include the verifica
tion of a special case.
Consider Figure 1.9 and include the data relating
to the coordinates (Figure 1.10): To show that the length of the line segment
OB is s + x, we must verify that the length of AB is s.
Draw the line through
P and parallel to OE^ and let C be the intersection of that line with the line
through P + Q and B. The quadrilateral ABCP has parallel sides and thus
is a parallelogram.
Hence, AB and PC have the same length. Now triangles
and
+
O^Q
PC(P
Q) have pairwise parallel sides (as shown in Figure 1.10)
and further 0Q and P(P + Q) have the same length.
Thus these triangles
are congruent so the length of PC is the same as the
length of 0s, namely s.
Thus the length of AB is also s, and so OB has length s + x.
Notice that this
is a special case since it refers to an explicit picture which does not cover all
possibilities.
The operation inverse to addition is subtraction : P
Q is that vector
which must be added to Q in order to obtain P (Figure
1.11). The best way
to visualize P
Q is as the directed line segment running from the head of
Q to the head of P (denoted L in Figure 1.11). In actuality, P
Q is the
vector obtained by translating L to the origin; in
practice, it is customary not
to do this but to systematically confuse a vector with
any of its translates.
We shall do this only for purposes of pictorial
representation.
Notice that, having chosen the vectors E1 and
E2 we can express any
vector in the plane uniquely in terms of them and the
operations of addition
into
sion.
our
mathematics, but rather
For those who
are
-
-
-
,
1.2
Numbers, Notation, and Geometry
P +
Figure 1.11
Q
23
24
Linear Functions
/.
and scalar
multiplication:
(x, y)
=
(x, 0)
no
0, Ej, E2 do
not lie
Thus
Proposition
Then
we can
Q
=
(0, y)
matter how
This is true
collinear).
+
we
x(l, 0)
y(0, 1)
+
Ex and E2
same
have this
Let
4.
the
on
=
line
are
(we
=
xEt
chosen,
yE2
+
so
say the vectors
long as the points
E1 and E2 are not
fact.
important
EY and E2 be any two noncollinear
Q uniquely as
vectors in
the
plane.
write any vector
*%
+
x2E2
x1 and x2
are the coordinates of Q relative to the choice of origin 0 and the
Et and E2
If we state this geometric fact purely as a fact about R2, it turns out to be
a theoretical assertion about the solvability of a pair of linear equations.
Thus, let us suppose Ex
(ax1, a,2), E2 (a2x, a2) relative to some standard
coordinate system (for example, the usual rectangular coordinates).
First
of all, how do we express algebraically the assertion that Et and E2 do not
lie on the same line? We need an algebraic description of a straight line
through the origin.
vectors
.
=
=
Proposition 5.
(i) A set L is
a
straight
line
0
through
if and only if there
exists
(a, b)
e
R2
such that
L
=
{(x, y) :
ax
+
by
=
0}
(ii) The points (x, y), (x', y')
only if
-
=
x'
lie
on
the
same
line
through the origin if and
L
y'
You certainly recall these facts from
analytic geometry we leave the
verification to the exercises. Returning to the vectors
Ex and E2, these
geometric facts become the following algebraic fact.
Proposition 6.
Let
la,1 aA
be
a
2
x
2 matrix with
nonzero
columns.
Numbers, Notation, and Geometry
1.2
(i)
If at *a22
-
a2a2
for every be R2.
(ii) The equation Ax
0, then the equation Ax
=
0 has
a nonzero
=
b has
solution if and
a
unique
25
solution
only if a1 a2
=
afa1.
a^a2
al2b1.
If
(iii)
ifal1b2
=
=
a2a2
,
the
equation
Ax
=
b has
a
solution if and
only
Proof.
(i) This condition is according to Proposition 5 (ii) precisely the assertion that
the vectors (ai\ ai2), (a2l, a22) are not collinear.
Then, according to Proposition 4,
for any (b1, b2) there is a unique pair (x1, x2) such that
(b\ b2)
This is the
=
x'iai1, a,2) + x2(a2\ a22)
same as
the
pair of equations
b
=
Ax.
(ii) By the above, if aila22 ai2a2, then the only solution of Ax 0 is x 0.
ai2) or ( a2\ a22) is a non
On the other hand, if a?a22 =ai2a2\ then either (a22,
in which case everything
A
are
entries
of
all
the
Ax
or
of
zero,
zero solution
0,
=
=
=
=
solves Ax
=
0.
(iii) If a,la22 =ai2o21, then (a/, ai2), (a2l, a22) lie on the same line through the
origin by Proposition 5(ii). Any combination x1(a1\ ai2) + x2(a2\ a22) willhaveto
lie on that line, and conversely, any point on that line must be such a combination.
b has a solution if and only if (b\ b2) lies on the line through 0 deter
Thus Ax
mined by (at\ at2). The equation for this is, by Proposition 5(ii),
=
h1
h2
a,1
a,2
_=f_
or
bW=b1a12
Examples
the line
Lx be the line through (0, 0) and (3, 2) and L2
of
and
L2
Lv
through (1, 1) and (0, 6). Find the point of intersection
6.
the equation 5x + y
and
2x
0,
the
L2
has
3y
equation
Lt
8.
Let
.
-
point of intersection
(x, y) solving
The
3y
=
0
y
=
6
We find
x
=
2x
-
5x+
9.
18/17,
y
=
Find the line L
line L': 8x + 2y
=
17.
=
=
must lie
on
both lines, and thus is the
pair
12/17.
through
the
L will be
point (7, 3) that is parallel to the
given by an equation of the form
26
/.
Linear Functions
+
by
point
of
ax
8x +
ax
=
intersection,
=
have
can
so
the
L and L' must have
to
L',
we
must have
parallel
equations
no
17
2y=
by
+
In order to be
c.
c
no common
solution.
Thus
8_2
a~b
Furthermore, since (7, 3) is
la + 3b
This
6
c
4x + y
?
=
=
we
must also have
c
of
pair
1,
=
=
L,
on
equations
31.
in three unknowns has for
Thus Z, is
given by
the
a
solution
a
=
4,
equation
31
EXERCISES
7. Show that for every
l2 + 22 +
+ n2
=
9. Show that
=
n,
in(n + 1)(2 + 1)
8. Show that for every
2 + 22+--- + 2"
integer
integer
n,
2"+1-2
Xn(YvZ)
=
(Xn Y)u(XnZ)
and
Xu(YnZ)
=
(XuY)n(XvZ).
10. Give
an
example
of
a
subset of
a
Cartesian product which is not
a
rectangle.
11
.
Find the point of intersection of these
pairs of lines in R2 :
(a)
3x+
(b)
\
x~\ly
4
x-2y
0
2x+ y
y
=
l
(c)
=
2x+
2y=-l
at+ 12^
(d)
=
=
y=2x+
1
jc
3y + 18
through P which is parallel
(2, -l),Z.:3* + 7y 4
(8, \),L:x-y
-\
=
12. Find the line
(a)
(b)
(c)
P
=
P
=
V
=
14
=
to L-
=
=
(0,-l),L:y-2x
=
3
PROBLEMS
9. We
can
the vector X
-
define the line through P and
Q
P is parallel to the vector P
parallel if and only if one is
the set of all X such that
Show that two vectors are
of the other.
Conclude that the line
-
a
multiple
Q.
as
1.2
through
P and
Q is the
Numbers, Notation, and Geometry
27
set
{P + t(P~Q):teR}
10.
is, in
Using the definition in Exercise 9
coordinates, given as
show that
a
straight line in
the
plane
terms of
{(x,y)eR2:ax + by
+
c
=
0}
for suitable a, b, c.
1 1 Suppose Z, is
a line given by the
equation bx + ay + c 0
Show that the tangent of the angle this line makes with the
horizontal is b/a.
=
.
(a)
(b)
(c)
Show that the vector
(a, b) is perpendicular to L.
Find the point on L which is closest to the origin.
12. Find the line through the point P and perpendicular to L:
(a) P
(3, l),L:x-3y 2.
(b) P (-l, l),Z.:2x + 3y 0.
(c) P=(0, 2),L:5x 2y.
13. Suppose coordinates have been chosen in the plane. Let Ei E2 be
two vectors in the plane which are not collinear.
(That is, 0, Ei, E2 do not
lie on a straight line.) Then we can recoordinatize the plane relative to this
choice of principal directions. Give formulas which relate these two
coordinatizations in terms of the given coordinates of Ei E2 (see Figure 1.12).
=
=
=
=
=
,
,
p
/
<E, (x*
/
/
/
y
Uj)
(u.v)
)
"~
^-'
vE. r
/
uE,
Figure
1.12
E.U'.y1)
1.
28
Linear Functions
14. In the text, the equation 1 + 2 -\
hn
There is another way of doing this.
=
entries.
There
2 +
+
-
We
now
equations,
1) +
=
n
by
on the diagonal and 1+2+
diagonal. Thus
\-
n
1
n2
return to the
with
problem of analyzing systems of simultaneous linear
question in mind: given the m xn matrix A, for
solution of the equation Ax
b? In order to study this,
broader
a
which b is there
a
=
associate to A the function from Rm to R":
fA(x\ ...,*")
=
(V*1
+
Thus the set of b, such that Ax
a^x",
+
a,mx\
...,
...,
anmx")
b, is precisely the range offA
begin by introducing the two fundamental operations
the case n
2 studied in the previous section) :
Let
in
verified
matrix has n2
Linear Transformations
1.3
we
n
was
n x n
of these entries
are n
entries both above and below that
2(1 +
$n(n + 1)
An
induction.
(1.16)
=
.
us
on
R"
(just
as
=
1. Scalar
rx
=
(rx1,
...,
2. Addition: for
x
+
for
Multiplication:
x
R,
x
=
(x1,
.
.
.
,
xT),
+
(y1,
=
y
.
.
.
,
x")
R", define
e
.
.
.
,
y")
e
R", define
y")
A function
preserves these two
(x1,
=
rx")
y^x1 +y1,...,xn
Definition 2.
r e
/ from R"
operations, that is,
to Rm is a linear transformation if it
if
/(rx)
rf(x)
/(x + y) =/(*) +/(y)
=
The function fA defined above for the
fA(rx)
=
=
=
(a, 'rx1 +
(rfa1*1 +
rfA(x)
m x n
a1",
an V),
+
.
.
.
.
.
,
matrix A is linear:
afrx1 +
rfa"*1 +
.
,
a^rx")
anmxn))
+
+
1.3
/*(
+
y)
(xl
=
+
=
=
y1)
+
+
+<*."(*
(a11a:1
a^x1
//.*)
+
+
+
an\xr
y"),
+
Transformations
.
.
.
,
a^x1
+
29
yl)
/))
+
---+aV
+
Linear
+
a"V
+
a11y1 + ...+,,//,...,
+ anmy")
^V +
+ /,(y)
The
significance of the introduction of linear transformations, from the
of view of systems of equations, is that it provides a context in which to
consistently interpret the technique of row reduction. For, the application
point
of
a row
operation
to a
system of equations
amounts to composition of the
particular linear transformation
corresponding to the row operation. Once we have seen that we can analyze
the given system by studying these successive compositions.
Looking ahead,
it is even more important to recognize row reduction as a tool for analyzing
linear transformations. Let us now interpret the row operations as linear
associated linear transformation with
a
transformations.
Type
I.
of the rth
Multiply a row by a nonzero constant. Consider
row by c # 0.
Let Pt be the transformation on
Pl(b1,...,bm)
=
the
Rm
multiplication
:
(bl,...,cbr,...,bm)
(multiplication of the rth entry by c). The effect of this row operation is that
of composing the transformation/,: R"
Rm with Pu and changing the
Ax
b
into
the
equation PtAx Pib. These two equations have
equation
->
=
the
same
=
set of solutions since the transformation
Px
can
be reversed
its inverse is
invertible). Precisely,
Type II. Add one row
corresponds
this transformation
to
P2(b\
.
.
.
,
bT)
=
(it is
1/c.
given by multiplying the rth entry by
to another.
Adding the rth row to the 5th
(by
,
.
.
.
,
Rm
on
V,
.
.
.
,
row
:
b* + b',
.
.
.
,
bm)
Again, this step in the solution of the equations amounts to transforming the
equation Ax b to P2Ax P2b. Since P2 is invertible (what is its inverse?)
=
=
we
cannot have affected the solutions.
Type
III.
corresponds
Interchange
to
two
P3(b1,...,br,...,P,...,m
The
Interchanging
rows.
the rth and sth
rows
the transformation
importance
=
(bl,...,V,...,br,...,bm)
of these observations is this : the
to linear transformations which in turn
are
operations correspond
representable by matrices. The
row
30
1.
Linear Functions
solution of the system of equations Ax
b thus can be accomplished com
pletely in terms of manipulations with the matrix corresponding to the system.
=
It is
our
purpose now to study the representation of linear transformations
the representation of composition of transformations.
by matrices and
In Rn the
n
vectors
fundamental role.
Thus
(1,0,..., 0), (0, 1, 0,
We
Ef has all entries
.
.
.
zero, but the
a
.
as
rth, which is
in R" has
Proposition 7. Any vector
ofEu ...,En.
0),
E]
,
shall refer to them
1
.
.
,
(0,
E
.
,
.
.
.
,
.
.
,
,
0, 1) play a
respectively.
.
unique representation
as a
linear
combination
Proof.
Obviously,
b") =*! + +b'E
(b1
We shall refer to the set of vectors Ei,
, E as the standard basis for R".
Proposition 7 comes this more illuminating fact.
.
of
.
.
Proposition 8. Corresponding to any linear transformation
a unique m xn matrix
(a/) such that
L: R"
Out
->
Rm
there is
L(x'
x")
=
(
V/=l
Proof.
define
a
It is
clear, by
a/xi,
a
that, given the matrix (a,1), Equation (1.17) does
Now, given L, since it is linear, we can write
(E,)
Then
+
linear transformation is
standard basis.
=
(1 .17)
j=l
the way,
linear transformation.
L((x\ ...,x")) =L(x1E1
Thus
..., a/v)I
+
xE)
=
x'KEO + + x?L(En)
completely determined by
what it does to the
Let
(at\
...,
fll-),
.
.
.
,
Z,(E)
=
(an\
...,
a)
Equation (1.18) becomes
L((x\ ...,x"))= x\ai\
....
(xW,
...,
=
=
which is just Equation
(xW +
(1.17).
---
ai") + + x"(al
am)
x1*,"1) +
+ (x"a\
x"am)
+ x"a\..., x'ar + + x-V)
...,
(1.18)
1.3
31
Transformations
Multiplication
Matrix
Now
we
must discover how to
transformations by
make this
Rm ->RP
(b/),
Linear
an
operation
matrices.
There is
only
then that T: R"
discovery: compute.
are
represent the composition of
on
two linear
way to
Rm and S:
one
Suppose
represented by the matrices (a/) and
compute the composition ST as follows:
->
linear transformations
Then,
respectively.
we can
T(x1,...,xn)=(tcij1xj,...,
2>/v)
V/=i
j
ST(x\ ...,x")
=
=
Thus ST is
=
/
i
( Jx( ,f>,V)>
(]tl(jty*j)*i
the p
represented by
x n
-
*>*'( ,f>;V))
i(iy^
matrix
(jtw)
Definition 3.
matrix.
(U9>
Let A
=
(a/)
be
an
m x n
product BA is defined
Then the
as
matrix and B
the p
x n
=
(bf)
a
matrix whose
p
x m
(i,j)th
entry is given by (1 19).
.
The
preceding discussion thus provides
the verification of
Proposition 9. If T: R" -> Rm, S: Rm -? R" are represented by the
A, B, respectively, then ST is represented by the product BA.
The
product operation
may
seem a
bit obscure at first
sight;
matrices
but it is
easily
described in this way: the (i,j)th entry of BA is found by multiplying entry
by entry the rth row of B to they'th column of A, and adding.
Examples
10.
A=
/5
3
7\
6
5
1
\8
11
-4/
B=
/
\
0\
6
1
-3
2
5
4
4
4/
Linear Functions
/.
32
LetAB
c11
Cl2
Cl3
cj2
=
=
=
=
Then
(c/).
=
49
3(-3)+ 7.4
25
1.4
6.6+ 5(-3)+
+
8.6+ll(-3) (-4)4=-l
5.6+
=
=
1.4
5.5 +
6.0 +
=
29
'
AB
/ 49
39
43\
25
20
29
\-l
14
39/
=
11.
(2
5
(
0)(2
\
12.
0
1)
\2
0\/
/-I
0
1
7
lj\-l
4
1
+ 0-5 +
5-0
1
1
1
5
0\_/-l
-2) \-\
-
l-l)
=
1/
-7
0\
4
-2/
-2\
[l 0J\-1 4-2/^17
Of
17
(I 1\(
1
7
1
4
recapitulate the
0\/-l
0\
-v
0
11
-2\
l-i
4
-2)
/
(-l)
01
4
(0 1\(
[0 iA
us
l)=[2-0
5
1/
\
Now, let
=(-l-2
discussion of this section
so
far.
The
problem
of systems of m linear equations in n unknowns amounts to describing the
The technique of row reduc
range of a linear transformation T: R" -* Rm.
tion
on
we
be
corresponds to composing Tby a
Rm.
These transformations
shall call them
represented by
of the
solve
a
are
succession of invertible transformations
those which
elementary transformations.
provide
the
row
operations;
Linear transformations
can
of the standard basis
by matrices, and composition
transformations corresponds to matrix multiplication. Thus, we
system of linear equations as follows: Multiply the matrix on the left
means
succession of
elementary matrices in order to obtain a row-reduced
easily read off the solutions. Since multiplication
by an elementary matrix is the same as applying the corresponding row
operation to the matrix it is easy to keep track of this process.
by
a
matrix.
Then
we can
1.3
Linear
Transformations
33
Examples
1 3.
Let
consider the system of four equations in three unknowns
corresponding to the matrix
A
us
=
We shall record the process of row reduction in two columns. In
we shall list the succession of transformations which A under
the first
goes and in the second
shall accumulate the
we
products
of the
matrices.
corresponding elementary
(a) Multiply the third
first,
by
row
>\
0
-1\
/0
0
-1
3
2
2
1/0
1
0
0
4
0
1
II
1
0
0
0
\0
1
2/ \0
0
0
ly
1 and
interchange
0^
(b) Multiply the first row by 3 and subtract it from
multiply the first row by 4 and subtract it from the third.
'I
0
0
2
0
V0
0
-1\ /0
5 \ / 0
5 II 1
1
2/ \0
(c)
0
-1\
0
1
5/2 1
0
0
-1
1
3
0
0
4
0
0
0
1,
/o
0
1
1
2/ \o
(d)
row
0
1/5
row
by
0
-1
0'
1/2
3/2
4/5
0
0
0
0
1
Subtract the second
2 and the third
-1\
1
0
1
5/2
0
0
1
0
0
0/ \1/10
\
/
1
1/5
second;
row
0
-1
0'
3/2
4/5
-11/10
0
-1/2
by
5.
from and add one-half the third
1/2
0
row
0
to the fourth.
0
the
0\
Divide the second
1
it with the
0
1,
34
1.
Linear Functions
Let
us
denote the
is the last matrix
product of the elementary matrices by P; thus P
the right and the matrix on the left is PA. Now,
on
it is easy to see that if PAx
y has a solution, the fourth entry of y
must be zero.
Now our original problem
=
Ax
b
=
has
a solution if and only if PAx
Pb is solvable (since P is invertible).
Thus b is in the range of A if and only if the fourth entry of Pb is zero :
Ax
b can be solved if and only if
=
=
A*1-i*2-A*3
If b satisfies that
it
by solving
=
=
o
condition, there is
PAx
x1-x3= -b3
x2 + \x3
\b2
x3
ib1 + fb3
6*
+
=
an x
such that Ax
=
b;
we
find
Pb:
\ b3
+
=
14.
We
row
Consider
row
1
reduce
now
as
the system in three unknowns
above.
(a) Multiply row 1 by 4 and subtract
by 3 and subtract it from row 3.
'1
3
0
-10
,o
-5
~2\,/
1
3
0
1
\0
0
10
3/1
-3
0
-2 \ / 1
-3/5
0/\-l
+
2-b2
+
=
b3
of row 2 from
0
2; multiply
row
3 ; divide
row
2
by
-
10
0\
-1/10
1/2
b thus has
=
row
1,
0
2/5
it from
0"
-4
The system Ax
-b1
0
6
(b) Subtract 1/2
/I
given by
a
0
1/
solution if and
only if
(120)
1.3
In that
x1
3x2
x*
+
the solution is
case
2x3
3V3
-Lx*
TX
-
_
=
_
=
b1
21,
2^1
rO
T q-
ft2
of x3 will
Any arbitrarily chosen value
condition (1.20) is satisfied).
15.
A=
/2
0
0
2\
3
-1
1
0
\2
2
0
0/
(a) Divide
/I
0
3
-1
\2
2
(b)
from
row
0
0/
\
2.
0
0\
1
0
0
0
1/
Subtract 3 times
row
1/2
o
o\
1
-3-3
1
0
2
0
-2/\-2
0
1/
\0
provide
1 from
row
a
solution
(granted
2; subtract twice
the
row
1
3.
i\ /
oo
(c) Multiply
row
by
0
1W1/2
10
0
1
-1
n
(0
row
35
Transformations
given by
1
_
Linear
2
row
by -1, and subtract twice the result from
3.
condition for the equation PAx
y to be
solution
The
solvable.
is
also
b
Ax
thus
every problem
solvable,
Pb:
is found by writing the system PAx
Here there is
=
no
=
=
x1
+
x2- x3
+
x4
3xA
=
=
ii1
3b1
-
b2
2x3-8x*= -8Z>1+262
Clearly, the value of x* can
easily found by the equations.
be
+
&3
freely chosen,
and
x1, x2, x3
are
36
1.
Linear Functions
Validity of Row Reduction
The basic
point behind the present discussion
equations in n unknowns is the
taneous linear
study of m simul
study of linear
the study of m x n
is that the
same as
the
transformations of R" into Rm, which is the same as
matrices under multiplication by m x m matrices. The matrix version of
this story is the easiest to work, if only because it imposes the minimum
However, the linear transformation interpretation is
amount of notation.
the most
first, let
But
and in the next section
significant,
record
us
a
we
will follow that line of thought.
of the main result of Section 1 1 in terms of
proof
.
matrices.
Theorem 1.2.
E0
,
.
is in
Let A be
an
x n
m
matrix.
There is
a
finite collection
E0A
Es of elementary m xm matrices such that the product Es
row-reduced form. Let P
Es
E0 and let d be the index ofPA.
.
.
,
=
The system Ax
solution.
(i)
=
(ii) The system Ax
of Pb vanish.
(iii) n d unknowns
b has
=
Proof.
First of all,
matrix whose first
we
b has
can
a
solution
if
and
only if
by
sequence of
a
PAx
if and only if the last
be freely chosen in any solution
may,
nonzero
solution
a
row
=
Pb has
d entries
m
of Ax
a
=
operations, replace
b.
A with
a
column is
(!)
namely, supposing the y'th column is the first nonzero column.
in that column, say a/, is nonzero. Interchange the first and
Thus
yth
some
rows.
entry
This is
accomplished by multiplication on the left by an elementary matrix of Type III, call
Now, E0 A
(a/) with a/ #0. Multiply the first rowby (a/)"1; thismakes
the (l,y) entry 1 and is accomplished by means of an
elementary matrix, say Ei.
Now, let Et be the elementary matrix representing the operation of adding
-1
-<*/(/) times the first row to the y'th row (this makes the (/,;) entry zero). Then
Em
E0 A has its first nonzero column (1.21).
The proof now proceeds by induction on m. If m
1 the proof is concluded:
the 1 x n matrix (0,
0, 1, a}+1
al) is in row-reduced form. For m > 1,
itE0.
=
=
.
the matrix Em
.
.
,
,
E0A has the form
.
.
.
,
1.3
where B is
that Fs
Transformations
37
(m
1) x (nj) matrix. The induction assumption thus applies to
collection F0
F, of (m
1) x (m
1) elementary matrices such
F0B is in row-reduced form. Now let
an
There is
B.
Linear
Em+J
a
-
.
,
-\0
.
.
-
,
)
Fj
Then, it is easy to compute that further multiplication of (1.22) by these matrices
does not affect the first row, and in fact,
E0A:
/0
1
a}+1
\0
0
Fs
aA
F0B
which is in row-reduced form.
(i) Suppose there are x and b such that Ax
preserves the
equality,
such that PAx
so
Pb.
PAx
Let
/
=
Then
b.
multiplication by
P
On the other hand, supopse x, b are given
be the transformation on Rm corresponding to P.
=
Pb.
fP
fr
composition
operations which are invertible, thus/j. is invertible. In
particular, fP is one-to-one, so since fp(Ax) =fr(b) we must have Ax b.
(ii) If d is the index of the mx n matrix PA, its last m d rows vanish. Thus for
d rows of Pb must vanish. By (i)
PAx
Pb to hold for some x, the last m
is
=
of row
a
=
=
this is also the condition for Ax
(iii)
The solutions of Ax
=
b
=
b to have
are
the
a
solution.
same as
those of PAx
Pb.
=
This latter
system has the form
xl + a^x2 +
x2 + a32x3
+
+
x' +
Clearly, x1,
.
.
.
,
x"
are
ai+1xd+l +
are
+ andx"
uniquely determined once xd+1,
m
d equations of
restricted by the last
free to take any values.
The b's
an1x"=ZpJ1bJ
a2x"=2pj2bJ
=
.
.
.
Pb
,
2 p/bJ
b" are known.
x", b\
0, but x'*1, ...,x" are
.
.
.
,
=
EXERCISES
13.
Compute the products
AB:
3^
(b)
A=
j
3;
38
Linear Functions
1.
4\
2
(c)
A
B
(6, 6, 3, 2,1,)
=
-1
8
\ V
(d)
A
4
2
8
6
1
2
-3
0
0
1
1
0
1
0
-1
-1
B
=
14. Compute the products BA for the matrices A, B of Exercise 13.
1 5. Compute the matrix corresponding to the sequence of row operations
which
row
reduce the matrices of Exercise 5.
given m x n matrix A find conditions
equation Ax b has a solution.
as given in Exercise 13(a).
as given in Exercise 13(b).
16. For the
under which the
(a)
(b)
(c)
A
A
on
the vector b in Rm
=
/
2\
3
-1
0
2
-1
0
1
4
(d)
V
'2
6
0
4
2
0
2
1
0
1
^8
6
0
-1
0
17. Show that if A is
an m x n matrix with m>n, then there are always
equation Ax b has no solution.
18. Verify that the composition of two linear transformations is again
b for which the
=
linear.
19.
Suppose that
T: Rm
-*
Rm and has this property:
r(E1)=0,...,T(Em)=0.
Show that T(x)
0 for every x e Rm.
20. Show that there is only one linear function
=
/(E,)
=
E2 ,/(E2)
=
E3
,
.
.
.
,
/(E)
=
on
R" with this
property:
Ei
PROBLEMS
1 5. Let
is
a
/:
R"
-+
R be denned
linear function.
by f(x\
Is the function
g(x\...,x")= ]>7=i(*02
.
.
.
,
x)
=
2?=
i
x*
.
Show that
/
1.3
linear ?
Is the function
ftfpc1, x2)
=
Linear
jc1*2 linear ?
16. Suppose that S, Tare linear transformations of R" to Rm5 +T, defined by
(S+T)(x)
=
entry
Show that the matrix
of the matrices
A, B, C be
17. Let
Show that
S(x)+T(x)
is also linear.
sum
39
Transformations
representing S +
representing S, T, respectively.
n x n
matrices.
T is the entry
by
Show that
(AB)C A(BC)
(A + B)C=AC + BC
A(B+C)=AB + AC
=
Show that AB
=
BA need not be true.
18. Write down the
products of the elementary matrices which
row
reduce these matrices:
11
4
7
3
l\
0
-5
6
2
0
1
0
0
3
0
0
5
1
2
o
-2
-1
1
o
/ ~26
\
3
2
-5
4
3
2
1
3
6
3
3
0
possible to apply further operations to the matrices of Exercise 1 8
bring them to the identity? Notice that when this is possible
for a given matrix A, the product P of the elementary matrices corre
sponding to these operations has the property PA I. That is, P is an
inverse to A. Using this suggestion compute inverses to these matrices
19. Is it
in order to
=
also:
'3
2
8
6
0
0
0
1
A
0
0
1
0
,1
0
0
0
1
0
0
\2
0
-1
-1
!\
20. Find
Find
a
2
x
2 matrix A, different from the
I.
2 matrix such that A2
a
2
x
identity
such that A2
=
I.
=
21. Is the
equation (I + A)(I + B)=I
+A+B
possible (with
nonzero
AandB)?
22. An
n X n
Show that
diagonal
matrices, AB
=
matrix to have
(a/) is said to be diagonal if a/ 0 for i =j.
matrices commute; that is, if A and B are diagonal
Give necessary and sufficient conditions for a diagonal
matrix A
BA.
an
inverse.
=
=
Linear Functions
1.
40
Linear
1.4
In the last section
Ax
we saw
that the
equation
b
=
be solved
can
of R"
Subspaces
for b's restricted
just
by certain linear equations and that the
equation might have some degrees of freedom. In both
determined by some linear equations ; such sets are called
set of solutions of that
cases
these sets
are
subspaces of R". We will begin with an intrinsic definition of linear
subspace and the notion of its dimension. In the next section we shall find
a simple relation between the dimensions of the sets related to the equation
linear
Ax
=
b.
Definition 4.
(i)
A set V in R" is
addition and scalar
a
linear
subspace if it is closed under the operations of
That is, these conditions must be
multiplication.
satisfied :
(1)
(2)
v1; v2
(ii)
If S is
V implies v, + v2 e V.
rv 6 V.
e
reR,\eV implies
a
set of vectors in
R", the linear span of S, denoted [5] is the
set
of all vectors of the form
c'-Vi
-\
1-
ckvk
with v,
VjeS.
(iii) The dimension of
a
linear
subspace
V of R" is the minimum number
of vectors whose linear span is V.
Linear
Span
Having
we
now
had better
just
given the intuitively loaded word "dimension" a definition,
hope that it suits our preconception of that notion. It does
that in R3 :
line is
one dimensional since it is the linear
span of but one
is
two
dimensional because we need that many vectors
plane
to span it.
In fact, it is precisely those observations which have
motivated
the above definition. We should also ask that the above
definition makes
vector; and
a
a
1.4
this assertion true: R" has dimension
that this is not
Linear
You may need
n.
41
Subspaces ofR"
a
little
convincing
since you do know of n vectors (the
standard basis) whose linear span is R". But how can we be sure that we
cannot find less than n vectors with the same properties ?
Consider this
immediately obvious,
restatement of the notion of
If the vectors vl5
"spanning":
...,
yk span
R", then the system of n simultaneous linear equations
ZA
=
b
j=l
has
solution for every b
a
that this cannot be if k
We
n.
now
.
.
,
We
already know from the preceding section
gives us a proof that R" has dimension
and that
// the
set S
The
is
proof
R",
v* span
by induction
on
R", then S has
least
at
n
n.
n
of them must have
one
context.
in R" spans
of vectors
of R" is
Thus, the dimension
Proof.
.
R".
repeat the arguments in the present
Theorem 1.3.
members.
Vi,
e
< n,
Supposing that
Subtracting an
and goes like this.
first entry.
a nonzero
appropriate multiple of that from each of the others, we may suppose that the
1 vectors all have first entry equal to zero. Then they are the same
remaining k
v* spanned R" we can show that
as vectors in -R"-1, and since the original vx,
1 and we have it. (Notice
1 >n
these must span R"-1. Now, by induction k
Theorem
of
in
the
the
first step
1.2.) Here now is a
that this is the same as
proof
more precise argument.
0) could
If none of the Vi,
(1, 0,
vt has a nonzero first entry then Ei
hardly be in their linear span. Letting as be the first entry of v,, we may suppose
2,
Vi and Wj
v,
as ar v2 for /
(by reordering) that at = 0. Now let wx
v* (see
k. The vectors wi,
w* have the same linear span as the vectors Vi,
(ai bi),
Problem 1 8) ; the difference is that only Wi has a nonzero first entry. Let Wi
bt span
b are in Rn~\ Now, b2
wk
v/2
(0, b), where bi,
(0, b2),
v/k span R", there are
R"-1. For let c 6 R"-\ Then (0, c) e R", and since wi,
.
.
.
,
,
=
.
.
.
.
.
.
.
,
.
.
'
=
=
.
.
,
=
-
.
,
.
.
.
,
,
=
,
=
=
.
.
.
.
,
.
.
,
,
.
x1
.
.
.
.
.
,
,
xke R such that
i>'W,=(0,c)
1=1
Thus xlai +x2-0-\
equation implies xx
Thus, b2
dim R"
,
.
.
.
,
> n.
fact dim R"
=
h*"-0
=
0,
so
=
h xkb
0, *% H
the second
equation
=
1 >n
by induction k
bt span R"-\
On the other hand, the standard basis Ei,
so
n.
-
Since
c.
1 ; that
-
.
.
.
,
a,
# 0, the first
c.
h x*b*
is, k > n. Thus,
E clearly spans, so in
becomes x2b2 H
=
42
Linear Functions
1.
Examples
16.
Vi
=
v2
=
v3
=
Let
(0, 1, 0, 3)
(2, 2, 2, 2)
(3, 3, 3, 3)
R4, and let S be their linear span. Then clearly
But it is also clear that v3 is superfluous, since v3
Thus S is also the linear span of vx, v2 : if
be three vectors in
dim S
<
3.
3/2(v2).
v
a1v1
=
then
v
a2v2
+
+
(a2
Thus, dim S
were a
a3\3
+
also write
we can
a\
=
=
2.
<
vector
3/2(a3))v2
+
w
In
=
fact,
S has
(a1, a2, a3, a4)
precisely dim 2. For suppose there
which spanned S. Then we would
have numbers cu c2 such that vt
=
qw, v2
=
Explicitly
c2w.
this
becomes
0
=
1
=
0
=
3
=
CiO1
cta2
cta3
cta4
But this is
0,
ct /
so
2
=
2
=
2
=
2
=
clearly impossible. By the second equation
0.
But 2
by the first we must have a1
=
could not be.
17.
V=
c2a}
c2a2
c2a3
c2a4
Thus, dim 5
=
+
v3 -v4
=
c2
must have
a1,
which
2.
Let V be the subset of R4
{y.v1 +v2
we
=
given by
0}
V is certainly a linear subspace of R4.
We will shortly have the
theoretical tools to deduce that V has dimension
3; with a little
work we can show it now.
First of all, let Ax
(1, 0, 0, 1), A2
(0, 1, 0, 1), A3 (0, 0, 1, 1). Then A1; A2 A3 are all in V, and if
v
(v1, v2, v3, v4), since v4 vl + v2 + v3 we have
=
=
,
=
v
=
=
v1A1
+
v
2A2
+
v3A3
=
1.4
Linear
43
Subspaces ofR"
Thus V is the linear span of Al5 A2 A3 so dim V < 3. On the other
hand, if dim V < 3, then Al5 A2 A3 can all be expanded in terms of
,
,
,
of vectors
pair
some
Bl7 B2.
If
we
delete the fourth entry in all
these vectors this amounts to
saying
in R3
of vectors.
can
be
spanned by
a
Thus dim V
impossible.
pair
that the standard basis vectors
But dim
R3
=
3,
so
this is
3 also.
=
Independence
Repeating
the definition
once
again, dimension
is the minimum number of
linear space. There is another closely allied
intuitive concept: that of" degrees of freedom" or "independent directions."
In such phrases as "there is a four parameter family of curves," "two
vectors it takes to span
a
are involved," allusion is being made to a
dimension-like notion. Now, if we try to pin down this notion mathematic
ally and specify the concept of independence in the linear space context, it
turns out to be precisely the requirement for a spanning set of vectors to be
minimal. In other words, the dimension of a linear space is also the maxi
independently varying quantities
number of
mum
degrees of freedom,
Let S be
Definition 5.
vectors if the
independent
x1y1
x1,
with
0,
.
.
.
,
xk
.
.
.
=
,
x*vk
+
+
x*
e
indpendent
vectors in the space.
We say that S is
set of vectors in R".
a
set of
equation
=
and
R
a
or
0
\u...,vk distinct elements of S
implies x1
=
0.
The standard basis of R" is an
now verify that R" has in fact
We
independent set,
no more
than
n
as
is very easy to verify.
of freedom in this
degrees
sense.
Proposition
Proof.
since
n
> 1
The
Let \u...,\k be
10.
set in R".
Then k<n.
is
=
Now let
always.
the first entry of v,\
are zero, then bi,
.
independent
1 is automatically true,
by induction on k. The case k
Let as be
us proceed to the induction step (k > 1).
we can thus write v,
(at b,\ where b, e R-1. If all the a,
bk are an independent set in R"'\ By the induction assump
proof
.
an
.
=
,
,
We
tion then, k <n l,so k <n. Now suppose instead that some as is nonzero.
Then
for
i > 2.
Let
'v,
0.
Vi
a,
aT
that
w,
so
d
=
vectors
the
can reorder
given
P* are an
(0, (3,) with p, e R-1. p2
the first entry of w, is 0, so w,
=
=
-
44
/.
Linear Functions
set in R"'1.
independent
For if
|-ic'aiWIVi+
\
J
Since vi,
.
1
=
.
.
2
,
vk are an
1
2
=
2*=2 c'/J,
0, then also
2J=2
c'w(
0,
p2
0,
=
so
c'v,=0
2
c2
independent set,
Thus, by induction,
and the proposition is proved.
independent.
k <, n,
=
once
=
=
again
k
c*
=
1 <
n
so
1
.
are
also
Thus in every
case
,
.
.
.
,
pt
Examples
18.
vt
=
v2
=
v3
=
Let
(0, 3, 0, 2)
(5,l,l,2)
(1, 0, 2, 2)
In order to show that these vectors
are
independent
we
must show
that the system of equations
x1vl+x2\2+x3\3
=
0
(1.23)
has only the zero solution. But this system is the same as
corresponding to the matrix whose columns are vx, v2 v3 :
the system
,
V2
If
we row
y0
2
2)
reduce this matrix
0
x2 + *3
x2 + 2x3
x3
+
0
obtain
0y
Now the system PAx
x1
we
=
0
=
0
=
0
=
0
=
0
obviously
has
only the
zero
solution: if
O-24)
1.4
find, reading upward that x3
we
=
Linear
0, x2
=
Subspaces ofR"
0, x1
0.
=
45
Since P is
0 has only the zero solution. What is
invertible then the system Ax
x1
0. Thus
x2
the same, if (1.23) holds, so must (1.24), so x3
=
=
the vectors \t, v2
=
v2
=
v3
=
A
=
independent.
(3, 2, 1, 0)
(1, 2, 3, 1)
(2, 0,-2,-1)
Again,
A=
=
Now let
19.
Vl
v3
,
are
I
let A be the matrix whose columns
1
3
\0
1
row
are
vt, v2
,
v3:
-2]
-1/
reduces to
~4 :fi
rA
lo
\0
ol
0/
o
0
The system PAx
x1
x2
=
=
-3x2
x3
+
=
0 has the solutions
2x3
0 has the same solutions.
The system Ax
the particular solution (-1, 1, 1). Thus
=
Taking x3
=
-Vi + v2 + v3 =0
20.
Four vectors in R3 cannot be
?,=(2,1,2)
(0,3,0)
v2
(1,0,4)
v3
=
=
v4
=
(0,l,2)
independent.
Let
1
we
have
Linear Functions
/.
46
Find
a
linear relation which these vectors must
reduce the matrix whose columns
A=
/l
3
(0
1
\0
If
satisfy.
we row
obtain the matrix
we
1/3
-1/
-1/6
1
Now, corresponding
0, and thus of
Ax
the v's,
1 \
0
0
are
to any value of
=
xVj
x4
Take x4
0.
=
we
obtain
1
Then
=
.
a
solution of
1
x4
x3
x2
fr3-3-x4 \
x1= -3x2-x4=-i
=
=
=
=
Thus
Vl
2
~
iV2
+ V3 + V4
=
0
Now, the equivalent form of these
two propositions about R", that any
members, and any independent set has
spanning
set of vectors has at least
at most
members, holds for any linear subspace of R"
n
Proposition 11.
n
Let Vbea linear
well.
as
subspace ofR" of dimensions d.
(i) A spanning set has no less than d elements.
(ii) An independent set has no more than d elements.
Proof. Part (i) is of course just the definition, so we need only consider part (ii).
proof amounts to a reduction to the case where V is R", and an application of
The
Proposition
10.
Let Wi,...,wd span V; since V has dimension d there exists such vectors.
V. Then we can write
Suppose, as in (ii), that vt,
, vk are independent vectors in
.
each Vj
as a
vj
=
.
.
linear combination of Wi,
2 flj'w,
.
.
.
,
wd;
1 <, j <, k
j=i
for suitable numbers
a/.
corresponding to the
independent. For if,
R"
The vectors
vectors
io(fl/,...,a/)=0
j=i
{v,};
(/,
we
.
.
.,
shall
a/)
for j
1,
show that
now
=
.
.
.
,
k
are
they
vectors in
are
likewise
1.4
then also
25
cJ\j
i
=
=
ilk
i
%cJ\j= J=l
2cJ2/w,= 2
J=l
1
=
l=l\j=l
1
Vi,
independent in R",
a/)
Definition 6.
are
.
.
.
,
vt
hO-w
/
the k vectors
...,
,
we
must have c1
so
c*
0. Thus
0,
by Proposition 10 d> k.
v s
V
can
=
=
.
a linear space.
A basis of V is
be written in the form
Let Fbe
such that each
47
\
2cV|Wi=0-WiH
Thus, by the independence of
(a/,
Subspaces ofR"
by this computation:
0
k
k
Linear
a
.
.
,
set 5 of vectors
k
v
in
one
2! c%
;=i
=
and
only
cl
witn
one
R,\te S
e
way.
Another way of putting this is: a basis for
independent and spanning vectors in V.
Proposition
12.
S is
a
a
linear
basis for the linear space V
subspace
V is
a
if and only if both
set of
these
conditions hold:
S is
(i)
(ii)
an
independent set,
of S is V.
the linear span
Suppose that S is a basis of V. Since every vector in V can be written
linear combination of vectors in S, certainly (ii) is true: Kis the linear span of 5.
Since 0 can be written in only one way as a linear combination of vectors in V, any
Proof.
as a
time
we
have
c1vi +
with
0,
.
.
c1,
.
,
c*
.
.
.
=
---
+ c'vt
=
0
& in R and vi;
,
0
(since
0
=
0
Vi
.
.
.
,
distinct members of S, we must have c1
Thus (i) holds : S is an indepen
v* also).
=
vk
\- 0
-I
dent set.
Conversely,
any vector
with c'
In
v
v
=
e
R,
suppose now that
(i)
and
(ii)
are
true for the set S.
Then
(by (ii))
in V can be written
ciVl +
v, e
S.
...
(1.25)
+ c*Vt
This
can
be done in only
one
way because of the
independence.
fact, suppose (1.25) holds, and also
Y=aiVl +
is true, with (c1
..-
0-26)
+ akvk
a^vi -I
h
(ck
ak)yk
=
0,
so
c'
=
a' since the v,
are
independent.
48
Linear Functions
/.
Dimension and Basis
The
are
facts to know about dimension of linear
important
these : such
a
has
basis with
a
That number is the
greater than
We summarize this
n.
Theorem 1.4.
Let V be
a
linear
of R"
subspaces
finite number of elements.
space V always
same for all bases and is the dimension of
a
V, and is
not
follows :
as
subspace ofR".
(i) There is an integer d<n such that V has dimension
(ii) Any basis of V has precisely d elements.
(iii) d independent vectors in V form a basis.
(iv) d spanning vectors in V form a basis.
d.
The proof of this part of the theorem is by mathematical induction
1, either V= {0} or V has a nonzero vector, in which case V=R.
Now we proceed to the induction step.
Thus either dim V
0 or 1 so dim V < 1
(i)
Proof,
If
on n.
n
=
=
.
,
describe how it goes. We assume the assertion (i) for
R"'1 as the set of w-tuples in J?" with zero first entry. If Kis
1, and consider
of R", it
intersects this space in some subspace of R"~* which is, by induction spanned by
some S vectors, with 8<n 1.
Now, choosing any other vector in V with a
nonzero first entry, this together with the vectors referred to above will span V.
Let
us
Now
we
make this argument
a subspace of R".
=
coordinates
W
Wis
a
(a1,
...,
assume
=
If V
{0}, then dim V
=
0 ; if not, V has
=
a"). One of the entries is nonzero;
that a1 ^ 0. Let now
we may,
a nonzero
by reordering
the
{weR-1:(0,w)e V}
linear
subspace of R"-1.
c\0, wi) + c2(0, w2)
in V,
8 ^n
subspace
precise.
Let V be
vector v0
n
a
c'wi + c2w2
=
For if
Wi
,
W and
w2 e
c\ c2
e
R,
we
also have
(0 c'wi + c2w2)
,
W.
Now, by the induction hypothesis, W has dimension
span W.
By definition of W, Vi
\6
(0, Wi),
we need only show that v0
(0, wa) are
Let v e V, and
v span V.
Vi,
let c be its first entry. Then v
c(a*) 'v0 is also in V and its first entry is 0. Thus
this vector is of the form (0, w) with weI^. Then there are c1
c" such that
so
1.
-
e
Let wi,
in V. Now
.
.
.
,
ws
=
.
.
.
,
"
-
W
=
CJWi H
h cVa
Thus
v
-
c(a1)"1Vo
=
(0, w)
=
.
,
=
c'CO, w,) +
+
cs(0, w5)
.
.
,
1.4
Linear
49
Subspaces ofR"
or,
v
=
c(a1)~1v0
+ c'vi H
Thus, there are 8 + 1
h c'va
vectors which span
(n-l)+l=n.
(ii) This follows easily
from
V,
so
Khas dimension d with d < 8 + 1 <
Proposition 12. If 5 is a basis for V (dim V d),
elements, and since 5 is independent it has at
=
then since S spans, it has at least d
most d elements.
vd are independent vectors
(iii) Suppose that Vi
they span. Let y0eV. By Proposition 12(ii), since
c") # 0 such that
dependent, so there exist (c
cv0 H
h c-v,,
in
V;
we
dim V
=
must show that
d,
v0,
.
..,
vd are
0
=
cd
vd are independent we must also have c1
0, a
0, since Vi,
h cdvd) as desired.
contradiction. Thus c # 0, so v0
( c0)'1^! ^
If they are dependent, then the equation
vd span V.
(iv) Suppose that vi7
If c
=
=
.
.
.
=
=
,
=
.
1- Cvd
c'vi H
holds with at least
vr
=
one
.
=
.
,
0
c' = 0.
(-^"'(c'vi
If say c' # 0, then
+ C-'vr-i + c'+1vr+1 +
+
+
Cvd)
Hence, Khas dimension at most
Vi,...,vd, with vr excluded, also span V.
d 1, a contradiction, so we must have had Vi, ..., v independent and thus a
so
basis.
proposition, whose proof is
(theoretical) ease in finding bases.
This final
the
Proposition
13.
Let V be
a
linear
left
as an
exercise, is
an
indication of
subspace of R" of dimension
d.
(i) Any set of vectors whose span is V contains d vectors which form
(ii) Any set of independent vectors in Vis part of a basis for V.
Examples
21
.
Find
Vl
=
v2
=
v3
=
v4
=
(4,
(5,
(0,
(1,
a
3,
2,
1,
0,
basis for the linear span V of the vectors
2,
2,
0,
0,
1)
1)
1)
1)
and express V by
a
linear
equation.
a
basis.
50
Linear Functions
/.
We want to find all vectors b of the form
x\
and
we
(1-27)
b
=
want to find
system corresponding
Now
basis for such vectors.
a
(1.27)
is the
to the matrix A whose columns are the vectors
Trie sPan f these vectors is Just trie range of A. If
of
elementary matrices row reducing A, then any vector
product
Thus by
b is in the range of A if and only if Pb is in the range of PA.
row reduction we should easily be able to solve our problem.
v3
?i, v2
P is a
A
v4
.
=
The end result of
row
'11
1
PA=|
'
0
l
~!
0
yO
0
reduction
V
0
0
0
1
0
0
-4
1
|
1
1
0
0
0
Oy
,1
1
-1/2
-3/2
-4
rA
Thus, the
to
V
produces
P
r
=
range of A is obtained
by setting
the fourth entry of Pb
zero :
range of A
=
{(b1, b2, b3, b4) -.b1
=
+
b2
-
f b3
dim V
<
3.
22. Find
a
the
Thus, dim V
=
-
1/4).
4b4
-
V has dimension at least three since it contains the
(4, 0, 0, 1), (0, 4, 0, 1), (0, 0, 2/3,
so
1
+
8x2
x2
We
A=
+
-
independent vectors
hand, V # R4,
3 and these three vectors
basis for the linear subspace
are
(5
1
\0
3x3
x3
8
x5
x5
=
0
=
0
3
1
0
2
1\
-1
0/
=
are a
V of R5
=0
the solution space of Ax
0-10
1
+
2x4
+
seeking
x4
+
-
0}
On the other
equations
5*1
x1
=
0, where
basis.
given by
1.4
Linear
Subspaces of R"
51
Row reduction leads to
PA=
/l
0
-1
0
0
1
-1\
0
2
0
\0
0
-2
-15
6/
and V is the set of x such that PAx
0.
are to be freely chosen and
choice. Thus, dimF=2. Choosing
According to these equations
jc1, x2, x3 determined by this
(x4, x5) (1, 0), (0,1), re
=
x4 and x5
spectively,
we
obtain
as a
=
basis
(-. -2, -Y.1.0)
(4,0,3,0,1)
EXERCISES
21. What is the dimension of the linear span of these vectors?
(a)
=(-1,2, -1,0)
v2=(2,5,7,2)
?,=(0,2,1,1)
t4
(3, 5, 7,1)
Vl =(-1,0,2, 1)
Vl
=
(b)
?a
=(2, 2, -2,2)
=(1,1, 1,1)
(0,2,1, 1)
?a
(1,7, 3, 3)
v3=(0,0, 0,1)
v=(l,3, 1,2)
v5=(l,5,2,2)
Tl= (0,0, 1,1,1)
v2= (1,0, 0,1,1)
?a =(0,1, 0,1,0)
Ta
(C)
Vi=
=
(d)
22. What is the dimension of the space S given by these
(a) S {x Rs : x1 + x2 x3 x* 0, x1 + x3 0}
(b) 5 {x e Rs: x2 + x* + xs =0, x1 x3 + x* =0
=
-
-
=
=
equations:
=
-
x1
-
x2
-
x3
-
xs
=
0}
R*: x1 + x2 + x3
x3
x2
x1 + x*}
23. Determine the linear span of these vectors by a system of equations
(c)
S
(a)
^=(1,0,0,1)
?a=(0, 1,1,0)
?,= (0,1,0,1)
vx=(2,2,6,2)
v2
(l,2,3,0)
(b)
=
{x
e
=
=(0,1,0,-1)
?!=(!, 0,1)
?a
(-1,1,1)
vx= (1,0, 0,0,0)
v2=(2,0, 1,0, 1)
v3
(c)
=
(d)
=
-
-
52
/.
Linear Functions
independent?
given in Exercise 21(c).
Vi, v2 v3 as given in Exercise 23(a).
^=(0,2,0,2,0,6)
?a =0,1, -1,-1, 1,1)
v3=(2,4,6,8,10,12)
V4
(0,0, -2, -2,0,0)
v3= (0,1, 0,0, 1,0)
v6= (1,1, 1,1,1,1)
25. Find all linear relations involving these sets of vectors.
(a) Vl =(0,1,1)
v2
(5,3, 1)
v3= (0,2,0)
v4
(1,-1,1)
(b) v1=(0,2,0,2)
v2
(0,l,0,0)
v3= (0,1,0,1)
v*
(0, 0, 0, 1)
Ts=(l, 0,-1,0)
(c) ^=(0,0,0,0)
?a
(1,1, 1,1)
V3
(1,1,0,0)
V4
(0,0, -2, -2)
26. Find a basis for the linear subspace of Rs spanned by (0, 0, 0, 1, 1),
(0, 1, 0, 0, 0), (1, 0, 0, 0, 1), (1, 1, 0, 0, 1), (2, 1, 0, 1, 2)
24. Are these vectors
(a)
(b)
(c)
Vi,
.
.
.
v5 as
,
,
=
=
=
=
=
=
=
=
27. Find
(a)
a
basis for these linear spaces:
+ 2x2 + x3 =0, x1 + 2x* + x5 =0,
{(x1, ...,x5)eR5:x1
x' + x'^O}
(b)
{(x1, ...,xi)eR*:x1-x2 + x3-x*=0,x1-x3=0}
given vectors on Rs are independent, extend them to a basis:
(a) (0, 0, 0, 0, 1), (0, 0, 0, 1, 1), (0, 0, 1, 1, 1)
(b) (1, 5, 2, 0, -3), (6, 7, 0, 2, 1), (1, 0, -1, -2, 0), (1, 1, 1, 1, 1)
(c) (4, 4, 3, 2, 1), (3, 3, 3, 2, 1), (2, 2, 2, 2, 1)
28. If the
PROBLEMS
23.
Suppose
w2 =v2
given
we are
/J2Vi,...,
w*=v*
k vectors Vi, ...,v* in R". Let wi=Vi,
for some numbers /32, ...,/?*. Show
/3tvi
that the sets {vi,
vj and {wi,
24. The proof of Theorem 1.3
.
.
.
,
sists of the vectors Vi,
many elements?
25. Prove
.
.
.
,
vt.
w*} have the same linear span.
proceeds by assuming that the set S con
What of the case where S has infinitely
.
.
.
,
Proposition 13.
V, W are subspaces
26. Show that if
of R",
so
is Vn W.
1.5
Rank +
Nullity
=
Dimension
53
27. Show that if A is obtained from B
by a row operation, the linear span
of A is the linear span of the rows of B.
28. Show that if A is a row-reduced matrix the dimension of the linear
span of the rows of A is the same as its index.
of the
1.5
rows
Rank +
Now let
Nullity
us
spaces, and in
There
certain obvious linear spaces to be associated
are
given transformation.
Definition 7.
Let T: R"
Rm be
->
a
linear transformation.
The set
(i)
K(T)
a
Dimension
apply the propositions of the preceding section about linear
particular the notion of dimension, to the subject of linear
transformations.
to a
is
=
linear
=
{veR":T(\)
subspace
of T, denoted
(ii) The set
0}
=
of R", called the kernel of T.
Its dimension is the
nullity
v(T).
R(T)= {T(v):veRn}
subspace of R", called the
p(T).
is
a
of
linear
T, denoted
Let T: R"
Theorem 1.5.
n
v(T)
=
+
that is, dimension
For
Proof.
Let v+i,
R".
Let y/j
.
wv+i,
p(j)
(i)
.
=
.
.
.
,
-
a
linear
We have
transformation.
p(T)
=
nullity
+ rank.
short, write v(T)
v
Rm be
Its dimension is the rank
range of T.
be the rest of
as v.
a
Let Vi,
.
.
.
,
be
v,
basis for R": vi,
.
.
.
a
,
basis for the kernel of T.
v,
,
vv+i,
.
.
.
,
v
thus span
Now the crux of the matter is this:
n.
v+ 1,
of
T.
Once this is shown, we will have
the
range
w form a basis for
desired
the
is
which
equation.
n
v,
Then there is a v e R" such that w
T(v). Expand v in the
Let w e R(T).
.
=
T(\j) for j
...,
,
=
54
Linear Functions
1.
basis vi,
.
.
.
v: v
,
vi=T(v)
=
=
=
=
c'vi +
\- c"\
T(c1Yi +
c'^vi) +
+ c"\)
---
+
cT(v)
+ c"w
justified since T is linear
T(vv+i) wv+i,
The second line is
=
and the third follows since Vi,
vv are
Thus these last vectors
w.
, T(v)
.
.
.
,
=
R(T).
(ii)
wv+1,
w are
...,
cv+1wv+1 +
r(cv+1vv+1 +
cv+1vv+i -I
independent.
Suppose
+ c"w=0
-"
We must show that the
so
c"T(vv) + cv+1r(vv+1) +
+
cv+1wv+i +
in the kernel of T and
span
Then
.
{c1}
+
h c"v
e
are
c"v)
(1.28)
all
=
.(7").
In any event, from
zero.
cv+17(vv+1) +
vi,
.
.
.
,
+
vv span
cT(v)
^(J)
so
=
(1 .28)
we
have
0
there
are
c\
.
.
.
,
c" such
that
cv+1vv+1 H
h c"v
=
c'vi H
h cvvv
or
(-c^Vi +
Since vi,
.
.
.
,
+
v are
(-cv)vv + cv+1vv+1
independent,
+
all the cJ
are
+ c"v
zero,
=
as
0
required.
The theorem is
proven.
Examples
23. Let T: R4
A=
/l
3
2
(0
0
1
1
1
0
0/
\0
We
A
/l
0
\0
3
R3 be given by the matrix
7\
completely analyze
can
easily
->
row
2
this transformation
by
row
reduction.
reduces to
7\
10
0
0
1/
1
(1.29)
(1.30)
1.5
Rank +
merely by interchanging the last
transformation corresponding to
Nullity
two rows.
=
Dimension
Thus, letting
55
P be the
f. 2 !)
0/
1
\o
know that PT is the linear transformation
to (1.30).
R3, and the range of
1
3. The
T is P~ (range of PT), which is again all of R3, so p(T)
kernel of T is the same as the kernel of PT, which has the equations
given by (1.30):
we
Now, the range of PT is easily
seen
corresponding
to be all of
=
x1
+
3x2
+
2x3
x3
+
+
lx4
x2
x4
=
0
=
0
=
0
(1.31)
by letting x4 take on all real
remaining coordinates by (1.31). Thus
R}, which is one dimensional.
The set of all such solutions is found
values and
K(T)
=
{(
solving for the
St, 0, -t,t): te
24. Let T: R4
A
->
R4 be given by the matrix
=
Let
us row
reduce this matrix,
10
keeping
0
10
0
track of
our row
operations :
0^
0
-3010
-200L
P
0
0
0
=
Oy
Now, the kernel of T is easy
the transformation S
to
find; it is the
corresponding
to the last
same as
the kernel of
matrix PA (because
56
Linear Functions
/.
S
the
=
of T
composition
by
an
invertible transformation). Now the
corresponding to the matrix PA,
kernel of S, and thus also of T, has,
the form:
x1
or
x2
x2
+
x2
+
x3
+
+
x4, x1
=
K(T) ={(-(
so
2x4
x4
v(T)
=
+
0
=
0
0
=
0
0
=
0
=
v),
x3
x4.
-v, u,
Thus,
v): (u, v)
The range of T is
2.
transformation
matrices
=
corresponding
a
to
e
R2}
little harder to find.
If R is the
the
elementary
product
of the
the left, then S
R
T, so the range of T is R_1 of the
x4
0.
range of S, which has the equations x3
(That is, the vector
b4) is in the range of S if and only if there exist (x1,
(b1,
x4)
on
=
?
=
...
=
,
...,
such that
x1
+
x2
x3
x2
+
+
+
2x4
x4
=
=
0
=
0
=
b1
b2
b3
b4
The necessary and sufficient condition is b3
b4
0.) Thus the
necessary and sufficient condition for v to be in the range of T is that
Pv be in the range of S; that is, the third and fourth coordinates of
=
=
Pv must vanish :
-3*1 +4x2 +x3 =0
-2X1 -5x2 + x4
0
=
Thus, p(T)
25. Let
=
us
corresponds
/l2
0
1\
1
3
0
1
1
1
1
2
\4
3
7/
2.
do
one more
to the matrix
example briefly.
Suppose
that T: R3
->
Rs
1.5
This matrix
can
/I
0
1\
0
1
1
0
0
0
0
0
0
\0
0
0/
A
=
be
by multiplication
P
=
V
row
Nullity
=
Dimension
57
reduced to
the left
on
Rank +
by this matrix
1
0
0
0
0\
2
1
0
0
0
2
-1
1
0
0
1
-1
0
1
0
1
-1
-1
-1
1
The kernel of T can be found
by looking at the row-reduced form A;
0.
(x1, x2, x3)
Precisely, we
must have x1 + x3
x2
x3
0.
Thus
a
in K(T) if
vector
is
+
0,
its first and second coordinates are the negative of the third ; that is,
1.
The range of T is
K(T)= {(-t, -t,t):teR}. Thus v(T)
such
that
Px
is
in
the
the set of x
x5)
range of A (since
(x1,
of
A
are
A
in
the
The
precisely those with
range
PT).
5-tuples
zero.
Thus
the
third through fifth
fifth
coordinates
and
fourth,
third,
coordinates of Px must be zero for x to be in the range of T. Specifically
R(T) is the set of simultaneous solutions of
it is the set of
x
in R3 such that Ax
=
=
=
=
=
=
.
.
.
,
=
2X1 -x2 + x3
X1 -x2
+ x4
-x1 -x2 -x3 -x4
We
can
take
x3, x4, x5 ;
R(T)
so
These
purely
=
p(T)
x1, x2
+
as
x5
=
0
=
0
=
0
free variables and
these
equations
to define
thus
{(u, v-2u,v-u,3v- 2u) : (u, v)
=
use
e
R2}
2.
examples
illustrate the fact that Theorem 1.5
in terms of matrices.
We
now
do
just
that.
can
be formulated
58
/.
Linear Functions
Proposition 14. Let A be
formation T: R" ->Km. Then
p(T)
an m x n
matrix, representing the linear
of independent columns of A
of independent rows of A
of the row-reduced matrix to which
=
number
=
number
=
index
A
can
trans
be reduced.
Finally, we can also reformulate Theorem 1.5 as a conclusion for systems
equations, thus bringing us to the ultimate version of Theorems 1.1
of linear
and 1.2.
Suppose given a system of m linear equations in n unknowns,
or rank, of the corresponding matrix A.
Then
Theorem 1.6.
and suppose d is the index,
(i) d <m,d < n.
(ii) {x: Ax 0} is a vector space of dimension n d.
(iii) {b : there exists a solution of Ax b} is a vector
=
=
space
of dimension
d.
EXERCISES
29. Describe
by linear equations the
range and kernel of the linear trans
formations given by these matrices in terms of the standard basis :
(a)
(b)
(c)
(d)
/8
0
30. Find bases for
0
1
6\
K(T), R(T) for each
T
given by the matrices (a)-(d) of
Exercise 29.
31. Let
of /is
/:
R"->R be
a nonzero
linear function.
Show that the kernel
linear
1.
subspace of R" of dimension n
32. Let f(x\ ...,*")
*'
Find
a
i
spanning
2"=
kernel of/.
a
=
set of vectors for the
1.6
Invertible Matrices
59
PROBLEMS
29. Let T: R"
linear
->
Rm be
a
linear transformation.
of R", Rm, respectively.
30. Let T be the transformation represented
are
Show that R(T) is spanned by the columns of A.
K(T)={(x\...,x):
fCjxJ=0}
where d,
31. Let
the columns of A.
.
.
.
32. Let S
1.6
w e
C
,
are
Define
R".
w e
Show that for
all
Then K(T) and J?(D
subspaces
as
the set of
v
m x n
matrix A.
Show that
such that
2"=i
v'w'=0.
^ 0, J_(w) is a subspace of Rn of dimension n\.
R\ Define (S) as the set of v such that 2?=i v'w'
w
=
Show that
S.
_|_(w)
by the
(S) is a subspace of R",
and dim
(S)
+
=
dim[S]
0 for
=
.
Invertible Matrices
In this section
we
shall pay
particular attention
to the collection of linear
transformations of R" into R"
From the
the
case
or, what is the same, the n x n matrices.
of view of linear equations this is reasonable ; for it is usually
point
a given problem
that
will have
as
many
equations
as
unknowns.
First of all ; it is clear that there are certain operations which are defined
on the collection of all linear transformations of R", thus making of this set an
algebraic object of some
following definition.
Definition 8.
The
sort.
We collect
algebra of linear operators
collection of linear transformations
(i) if/is in ", and
(cf)(x)
(ii) if/
g
=
are
c
is
E",f+g
(f+g)(x)=f(x)
(iii) fgis
a
provided
real number,
cf(x)
in
defined
together all
by
(fff)(x)=f(9(x))
+
is defined
g(x)
by
on
these notions in the
R" denoted
with these
by E",
operations :
is the
60
Linear Functions
1.
important to think of the elements of E" as functions taking n-tuples
ra-tuples of numbers ; but in working with them it is convenient
Thus, we are
to represent them in terms of the standard basis by matrices.
with
the
led to consider also the algebra M" of real n xn matrices
operations
of scalar multiplication, addition and multiplication, the definitions of which
we now recapitulate.
It is
of numbers into
The
Definition 9.
with these
If A
(i)
cA
(ii)
operations
=
=
If A
algebra
M" is the collection of
n
xn
matrices
provided
:
(a/) is in M", and c is a real number,
(ca/)
=
A + B
(a/)
=
and B
=
(b/)
are
in M", then
(a/ + b/)
AB=(ak%k])
The two
algebras E", M" are completely interchangeable, for M" is just
explicit representation of E" relative to the standard basis.
Now the operations on M" obey certain laws, some of which we have
already observed in previous sections. Let us list some important ones.
the
Proposition
These equations hold for all
15.
n
xn
A, B, C and all
matrices
real numbers k.
(i)
(ii)
(iii)
(iv)
k(A + B) A:A + A:B
C(A + B)=CA + CB
(A + B)C AC + BC
A(BC) (AB)C
=
=
=
If A is
a given matrix, we shall let A2 denote A
A A A, and
A, A3
general A" is the -fold product of A with itself. Since we may also add
matrices, and multiply by real numbers, we may consider polynomials in a
given matrix. That is, A2 + 3A + A, A7 + 3ttA3 + A6,
In fact, if
we adopt the usual convention that A0
then
for
J,
any polynomial
PVQ X"=o ctX' in the indeterminate X, we may consider the matrix p(A)
Z"=o c;A'. A most remarkable observation can now be made, by noticing
=
in
.
.
.
.
=
=
=
that the collection M" of
2-tuples
of real numbers.
n
x n
matrices is the
same as
the collection R"2 of
1.6
p
Proposition 16. Given any n x n
of degree at most n2 such that p(A)
A, there
matrix
=
Invertible Matrices
is
a nonzero
61
polynomial
0.
Proof. An element of M" is a rectangular array of n2 real numbers, thus corre
We may make this correspondence explicit, by, say,
sponds to an element of R"
placing the rows one after another. That is, the matrix (a/) corresponds to the
vector (ai1,
al- 1, a") in R"2. In any event, the notions
a,1, ai2,
an2, d3,
of sum and scalar multiplication is the same in the two interpretations. Now con
A"2. These n2 + 1 vectors in R"2 cannot be inde
sider the matrices I, A, A2,
.
.
.
.
,
...
pendent
there
so
Thus the
+
c2
.
,
,
real numbers c0
are
c2A"2 +
.
.
...,
,
cu
.
.
.
,
c2
,
not
all zero, such that
A2 + CiA + coI=0
proposition is verified with p the polynomial
p(X)
=
c2
X"2 +
+
c2
X2 + ClX+
Co
rephrase this proposition in this way: Every matrix is a root of some
polynomial equation with real coefficients. From the purely
algebraic point of view this formulation is of some interest and raises the
converse speculation: given a polynomial with real coefficients, does it have
We shall verify this fact, and with n no greater
some n xn matrix as a root?
we
than two.
More precisely,
shall, in a later section, introduce the system
a
certain
collection of 2 x 2 matrices, and later verify
of complex numbers as
a
root in the system of complex numbers.
has
that every real polynomial
of algebra.
theorem
the
fundamental
This is known as
We may
nonzero
linear transformation in E" is invertible if it has an inverse as a
For this it must be one-to-one and onto; that is,
function from R" to R".
rank + nullity)
We have seen (n
it must have zero nullity and rank n.
Now,
a
=
that either of these assertions
assertions must be
The
Definition 10.
that BA
I
=
=
Proposition
expressible
AB.
17.
case
BA
I
=
BI
=
=
B is said to be
An invertible matrix has
This is clear : if B, C
Proof.
in terms of matrices;
AB
CA
=
=
B(AC)
=
a
an
I
AC
(BA)C
=
IC
=
C
a
matrix B such
unique inverse.
inverses to A, then all these
=
do that.
inverse for A.
are
Then
B
we now
matrix A is invertible if there is
nxn
In this
implies
Now it is clear that these
the other.
equations hold :
62
1.
Linear Functions
We shall denote the inverse of
ship
a
Let A be
Proposition 18.
(i)
(ii)
(iii)
(iv)
(v)
by A 1. The relation
gives us this propostion:
matrix A, if it exists,
between matrices and linear transformations
an n x n
matrix.
These assertions
are
equivalent:
A is invertible.
A represents an invertible transformation.
I.
There is a matrix B such that BA
=
There is
a
matrix B such that AB
A has index
=
I.
n.
Proof. We have already seen (in discussing systems of linear equations) that (ii)
(v) are equivalent. By definition (i) implies both (iii) and (iv). Thus we have
left to prove that (i) and (ii) are equivalent, (iii) implies (i), and (iv) implies (i).
and
(i) implies (ii).
it represents.
Since A A-1
Let A be the
given invertible matrix, and T the transformation
represented by the inverse, A-1, of A.
Let S be the transformation
I
A_1A, we have T- S I S T. Thus S is inverse to T, so
(ii) holds.
(ii) implies (i) by the same kind of reasoning with the roles of matrix and trans
formation interchanged.
(iii) implies (i). If T is the linear transformation represented by A, then by (iii),
there is a transformation S such that S T
I. Thus, if T(x)
0 we must also
have x
S(T(x)) S(0) 0, so T has nullity zero and is thus invertible. Thus
(iii) implies (ii), so also implies (i).
(iv) implies (i). If again, T is the transformation represented by A, by (iv) there
is a transformation S such that T S
I. This implies that T has rank n and thus
=
=
=
=
=
=
=
=
=
=
is invertible.
Computing
the Inverse
Now, it is clear that the question of invertibility for a given matrix is
important and that the problems arise of effectively deciding this question
and of effectively computing the inverse, if it exists. To ask that the rows
(or columns) be independent, or span R", while responsive to this question
hardly provides a procedure for determining invertibility. We shall now
introduce two such procedures: one is a continuation of row reduction and
the second is based on the notion of the determinant. The determinant is a
real-valued function defined on the algebra M" of n x n
its basic
matrices;
property is that it is
nonzero
only
on
the invertible matrices.
We shall
the determinant in the study of
eigenvectors (Section 1.7).
In Section 1.9 we shall explore the connection between the
determinant and
the notion of volume in R3.
depend heavily
on
1.6
In order to
Invertible Matrices
63
the critical
properties of the determinant function it is
elementary matrices, for they provide a technique
for decomposing an invertible matrix into a product of simple ones, and as
a result, a technique for computing inverses.
We recall these facts: the
are
matrices
the
matrices
which
elementary
represent the row operations.
Since the row operations are invertible, so are the elementary matrices
invertible. For any matrix A there is a sequence Ps
P0 of elementary
is
in
row-reduced
form.
The index
matrices such that B
PjPj-j
P0A
of A is the number of nonzero rows of B. We augment these facts by this
verify
necessary to return to the
.
,
.
.
,
=
further observation:
Proposition 19. Suppose that A is an invertible n x
P0o/ elementary n xn matrices such
sequence P,
matrix:
identity
P0A
P,
Proof.
The
Theorem 1.2.
n
matrix.
that
There is
a
P0 A is the
P,
I
=
proof
will be by induction
The first column of A is
As
independent (A is invertible).
exist elementary matrices P0
,
.
.
.
we
on n.
It is
a
slight modification of
since the columns of A must be
in the proof of Theorem 1.2, there
nonzero
have
seen
P* such that the first column of P*
,
P0A is Ei.
Thus,
1
n-1
/l
Al\
1
P*---P0A-|o A22j
Since P*
applies
to
P0A is invertible so is A22 (see Problem 37). Thus the proposition
1)
1) x (n
Q*+i of elementary (n
A22. There is a sequence Qs
-
,
matrices such that Qs
Pj
n-1
=
Q*+iA22
( q)
(J q'
P.-PA
*>r7
=
=
I.
.
.
.
-
,
Let
* + l,...,J
Qt+lA22) (o 1)
=
=
...
Now the matrix
(I -f)
is the product Ps+_,
Ps+i of elementary matrices corresponding to these
row
Linear Functions
1.
64
operations:
a/
subtract
times
they'th
from the first row, j
row
=
2,
...,n.
Finally,
p.+-i-poa=(j _f)(0 f)=(j ;)=i
as
required.
This
we
proposition provides
just continue
with
us
the process of
corresponding product
Then the
an
row
effective way for
reduction until
we
computing inverses;
obtain the identity.
of elementary matrices is the inverse.
Examples
26.
1
2
2\
-3
4
2
1
0
8/
/
A=
\
Product of elementary matrices
Row reduction
/I
2
2\
0
10
8)
2
6/
\o
-
/
1
0
3
l
0
\-l
0
1/
A
0
0
1
A\
*
/
\o
0
w
\-A
/i0
\o
-
*
/
1
\
0
13 7
190
1
190
0
1/
\
0
/
o\
-iV
iV
A
o
1/
21
190
1 5
190
2
76
Thus
A_1=Tfo
0\
/ 137
-21
65
15
\-10
5
-10\
-20
25/
27.
1\
(212
10
1
-1
-1
0
-1
4
1
2
1
-24/
76
-iV\
-A
w
1.6
T
0
1
0
0'
1
2
/
2\
1
1
2
I
0
0
0
0
-1
1
3
0
1
1
0
0
4
2
\o
1
0
1,
'10
-22/
1
2\
0
-31/
0
1
Oil
0
2
'10
0
1
0
^0
/
0
10
0^
1
-1
1
0
\-4
9
0
1;
l-\
0
2\
1
0
-31/
0
0
1
oil
0)
0
0
n
o
o
o\
o
i
o
oW
0
0
1
ON
^0
0
0
1/ \
65
0-200
-10/
0
Invertible Matrices
2
-1
1-2
0
1
-1
10
-10/\-6
11
/-u
if
ft
-I*
1-1
-4
10
10
0^
0
-2
1,
-u
a
1
10
Thus,
The Determinant Function
The determinant of a matrix is
into
of it and its
pretty complicated concept; before going
properties, we shall first see how to compute it.
a
study
Looking ahead, the method of computation comes from Equations (1.35)
and (1 .36), but we shall not use those equations to derive it. Instead we shall
simply describe the technique for finding determinants.
a
The determinant of
det I
,)
=
a
ad
The determinant of
a row.
2
x
2 matrix is defined
be
a
3x3 matrix is found
The determinant will be
a sum
of the
withy numbers, called their cofactors.
is (- 1)'+-' times the determinant of the 2 x
row and/th column are deleted.
row
by
follows.
as
products
First, select
of the elements of that
The cofactor of the
(i,j)th entry
2 matrix
when the rth
remaining
66
Linear Functions
1.
Examples
Compute the determinant of
28.
1
3
2\
-1
4
0
7
-2
/
A=
\
If
we
det A
select the first
=
1[4(1)
(-2)0]
the second
Selecting
=
-
=
7[3(0)
way.
=
(-2)2]
-
+
4[1(1)
-
2(7)]
-
4(2)]
-
(-2)[1(0)
2(- 1)]
-
For
a
example, selecting
sum
-3[(- 1)1
same
way.
of the
0(7)]
-
Select
products
+
4[1(1)
andy'th
-
7(0)]
(or column).
of the entries in that
(n
column
0\
(431
6
10
0-1
0
-
are
29. Let
2
a row
The cofactor of the
determinant of the
2
-
7(4)]
+
0[. .]
.
+
[1(4)
-
3(- 1)]
2[1(0)
-
-
2(- 1)]
-45
cofactors.
row
2[(- 1)(-2)
column first, and proceeded in the
the second column:
Now, in general, the determinant of the
the
+
row:
could also have selected
we
same
=
7(0)]
-
-45
Now,
det A
3[(- 1)0)
-45
Selecting the third
=
-
find
row:
-(- 1)[3(1)
=
det A
row we
-45
=
det A
1/
4l
11-1/
1)
x
(n
deleted.
-
x n
n
matrix is found in
The determinant is the
row
(or column)
with their
(i,j)ih entry is (-1)'+J' times the
1) matrix remaining when the rth
1.6
Select the first
det A
We
4 det
=
/6
0
j
0
0
\1
1
(2
12
0
1
0
4
\2
1
-l)
3 det
12
6
0\
0
Odet 1
0
0
\2
1
\2
1
1/
compute the determinants of the 3
3 matrices
by taking
since its factor is 0:
4(-l)[6(4) 0(-l)] 3(-l)[2(4) 1(-1)]
[2(4)
1(-1)]
+(-6)[l(-l) 4(2)]
-
-
-
-
-
-
-24.
=
30.
A
=
/6
2
1
0
4
3
8
1
~l\
0
0
0
2
0
0
8
1
4
0
1
\2
1
4
-1
Select the third
=
x
Select the second column in the
advantage
first three, and don't bother with the last
detA
-1\
6
of the location of the O's.
=
67
row
+ det 1
now
det A
Invertible Matrices
1/
row:
/6
(-l)3+3(2)det I48
2
0
-1
3
1
0
1
0
1
\2
1
-1
1
-
Select the third column:
/6
detA
=
2(-l)2+3det
8
\2
=
2
2
1
1
+
(-l)4+3(-l)det
-1\
4
3
0
\8
1
1/
96
theory of determinants. We begin with a definition
which is appropriate to the theoretical dis
function
of the determinant
the multiplicative property:
it
has
that
then
cussion and
verify
We turn
now
to the
det(AB)=
det A- det B
Linear Functions
1.
68
and
(1.36) below which form the basis for the preceding
a rewriting of the formula for the determinant.
The formulas
(1.35)
computations
will result from
The determinant of
the
sum
of all
matrix
an n x n
products
of
precisely
one
can
be described in this way: it is
row and column,
element from each
Our first business is to determine this appropriate
of
precisely one element from each row and column is
sign. A selection
In
the first row we select a certain element, say in the
follows:
as
described
with
appropriate signs.
In the second
7t(l) column.
column, say n(2).
ment
a\{i)
in the rth
row
...,
or
'
'
an
element, coming from
and
so
sure
These numbers then form
numbers 1,
.
.
.
,
n.
a
different
We select the ele
forth.
n(2) n(\),
7i(i)th column, making
^
that the numbers
a
rearrangement,
To form the determinant
then,
we
'
ait(n)
2(1)
as 7t
and
all distinct.
n(n)
permutation of the
consider all products
n(\),
are
select
row we
We have
ranges
over
all
n.
A
of the numbers 1,
,
of
two
successive
integers:
interchange
permutations
kind of permutation is
an
.
.
.
particular
1-1
2-2
+ 1
i-
i+1h
n
(We consider the integers as arranged in a circle, so that 1 is the successor to
n.) Now it is a fact about permutations, that any permutation consists of a
succession of such interchanges. There may be many ways to build up a
given permutation by these simple interchanges, but the parity of the number
involved is always the same. That is, if we can write a given permutation as a
succession of an even number of interchanges, then every way of writing that
permutation as a succession of interchanges will involve an even number.
For example, consider the permutation on four integers
1 2 3 4-3 1 4 2
This is obtained
12 3 4
2 13 4
2 3 14
3 2 14
3 12 4
3 14 2
by this succession of interchanges:
1.6
Here is
a
better way of
doing
Invertible Matrices
69
it:
12 3 4
13 2 4
3 12 4
3 14 2
Either way, there is an odd number of interchanges involved. We shall not
verify these facts about permutations; the verification would be tangential
to our present study.
However, we shall use these facts. We shall say that
a given permutation is even if it can be formed by an even numbered succession
interchanges ; the permutation is odd if an odd numbered
interchanges is required. For any permutation n, its sign,
of
succession of
denoted
e(n)
1 if n is odd. There is another way of defining
will be + 1 if n is even, and
the sign function on permutations which is described in Problem 36. This
does not involve the notion of
description
If A
Definition 11.
=
(a/)
detA=
all
a
an n x n
matrix its determinant is
e(X>fl<o
(1-32)
1=1
n
show that det A # 0 if and only if A is invertible, by
det A det B.
stronger statement : det (AB)
We shall
in fact
permutations
is
interchange.
now
showing
=
Lemma 1.
(i)
(ii)
det 1=1.
If A
has
a zero
row, det A
=
0.
Proof.
(i) Writing I (a/), we have ai<0 =0, unless ttQ) i. Thus, the sum (1.32)
has only one nonzero term, that corresponding to the identity permutation. Since
1
1.
1 1
each a,'
1, det I
(ii) If the /th row of A is zero, each term of the sum (1.32) has a factor a{u) 0,
=
=
=
=
=
=
so
is
zero.
Then det A
Lemma 2.
IfP
det(PA)
Proof.
Type
I.
=
Let A
If P
det PA
is
=
an
(a/),
PA
multiplies
=
elementary matrix, and A
any matrix,
(1 -33)
det A
det P
=
0.
=
(b/).
the rth
2 e(n) fl #<o
=
row
by
c, then
2 e(w)J<'>
'
caSw
aSoo
1=1
n
=
2 s(tt)c 1II1 alw
=
=
c
det A
Linear Functions
1.
70
In the
special
holds in this
Type II.
the
A
case
Suppose
det PA
=
det(PI)
P
now
c
=
det I
=
c.
Thus
(1.33)
interchanges the rth and sth rows. Let 77 represent
r and s.
Thus, b/
af'\ Now we compute:
which interchanges
2 W ii ,>
=
1
we
have det P
we
case.
permutation
Now
I,
=
=
1
=
=
2 eWll <l
change the index of summation.
(T V) 11 <%)
=2
det PA
Let
w
=
t
17, and
sum over t.
-2 to 11 <('.,)
=
1=1
sign changes since r; is an interchange; thus, if r is even, t 17 is odd. Now the
product FT" 1 a?(o> is tne same as tne product fl" 1 {(o (another change of index)
The
=
=
so
det PA
In
II {<o
2 *(t) j=i
=
particular, det
=
We
a/ + aaf.
bf
det(PI)
det A
-
=
1,
P adds
Suppose that
Type III.
and
P
=
now
a
(1.33) holds in this
so
times
row r
to row
s.
case.
Then
b/
=
a/
if i =
s
compute
det(PA)=2e("-)n#(o
(
=
1
n
=
n
2 EW II aid) +
2
1=1
(7r) 11
(1 .34)
ct*u> aUr) aUsy
<=i
(*r
The first term
the
on
right
s.
of the form
17, where
77
2
n even
=
2
n even
It is
11 aid)
important
-
n
is
The second term is
even.
a'n(r) a'Ms)
11 fl(D <&(D fl(s)
t^r,s
II i(i)(o(o a'Ms)
zero.
We
can see
that
by
Let 77 represent the inter
to note that the odd permutations are just those
l&r,s
Z
rt even
=
sum
is det A.
into odd and
splitting up the
change of r and
permutations.
even
Thus the last term in
2
nodd
EI i<o
'
Equation (1.34) is
a'*w *<>
'
t$r,s
2
n even
a;(s) a'Mr>)
11 flioKD>0ii(J|(r <(<)>
l<trts
=
0
l^r,s
det A. In particular, detP
Thus, det PA
Type III elementary matrices.
=
=
l,
so
(1.33) is verified also
for
1.6
Invertible Matrices
71
Now, lemma is a word denoting a logical particle of no particular intrinsic
interest, but of crucial importance in the verification of a theorem. Here
now
is the main theorem
Theorem 1 .7.
concerning determinants.
A matrix M is invertible
if and only if det M #
0. det AB
=
det B for any two matrices.
det A
Proof. Suppose M is an n x n matrix which is not invertible. Then there are
P0 such that P5
elementary matrices P,
P0M is row reduced and has zero
rows.
Thus, by the above lemma
.
,
0
=
det(P3
.
.
,
P0M)
det P3
=
det P^i
det P0
det M
Since the determinant of an
On the other
that I
=
elementary matrix is nonzero, we must have det M 0.
hand, if M is invertible, there are elementary matrices P,
P0 such
,
Ps
1
=
det I
=
det AB
so
det AB
is true.
=
=
det P,
det P5_,
det P
matrices.
n x n
0 and either det A
.
,
If
=
0
det M
one
or
of A
det B
=
or
B is not
0.
In any
invertible, neither
case
det B
det A
If A and B
.
Then
P0M.
Thus det M = 0.
Now let A, B be two
is AB,
=
.
invertible, there are elementary matrices P,
are
P0 Q
,
Qo
such that
Ps
PA
=
I
=
QB
Q
Then
Q
PoAB
QoP,
PoA)B
Q0(P5
0,
=
=
Q
Q0B
=
I
Thus
det Q
det Q
det Ps
Thus
det Q0
det Q0
det P0
again det(AB)
=
det P
det P.
det B
det A
det A
=
det(AB)
=
1
1
=
1
det B.
Notice that the formula det AB
the basis of the definition above.
=
det A
det B is far from transparent
on
fact, it is not at all derivable without
of n x n matrices. We have a
the
structure
some information regarding
matrix A; namely, the process
invertible
for
a
means of computing A-1
given
not
have
of row reduction. But we
given explicitly any formula for the
In
72
1.
Linear Functions
determinant.
This
formula is of theoretical interest, but not of any great
As far as computations are concerned, the surest and
computational
quickest route
value.
inverse is the process of row reduction.
Let A be an n x n matrix. The adjoint matrix of
an
Such is
inverse.
provided by
the cofactor
expansion
of
a
to the
entry of A is the
(n 1) x (n 1) matrix obtained by deleting the row and column of the
given entry (see Figure 1.13). Let A/ be the adjoint matrix of the entry a/.
Then the inverse to the matrix A (if it is invertible) is easily given by the
determinants of the adjoints: the (i,j)th entry of A-1 is
(-ir^
det A
More
A-1 A
precisely we have these formulas (the explicit version
I) known as Cramer's rule:
of AA_1
=
=
detA=
(-l)'+'a/detV
for alii
(1.35)
J(-l)l+'a/detA/
for
(1.36)
;=i
detA=
i
0
=
0
=
=
all;
l
f (- 1),+ V det V
for all i # k
I (- l)i+ V det V
for
all;
I
A;'
Figure 1.13
= k
(1.37)
(1.38)
1.6
The verifications of these formulas
be based
fix
directly
index i.
a row
formula
on
(1 .32).
are
Invertible Matrices
73
simpler than it may seem; they can
example, let us verify (1.35). First
the sum in (1.32) into n parts: those
For
We shall break up
permutations taking / 1, i 2, ...,/ n. Consider, for a fixed column
index the permutations taking i-*j. (That is, those n for which n(i) =j.)
These are precisely the same as all permutations on the indices of the
matrix adjoint to a/ (those permutations which take the integers 1, ...,,
Thus the terms appear
n, except for;').
except for i, into the integers 1,
ing in the sum (1.32) which have a/ as a factor, are the same as those in
(1.35): we must now verify that the signs agree. Let % be a permutation on
the indices of the adjoint to a/. The corresponding permutation n of
n) does the same as t and takes i into j. The number of interchanges
(1,
.
.
.
.
building
required
to send i to
i+j.
Thus, e(n)
(1 .32)
.
,
,
involved in
as
.
(1.35) also
and
that for t, with the
this
permutation is just
j.
The last number is j
interchanges
same parity
signs of corresponding terms in
( 1)'+J(r), so the
Thus (1.35) is true.
=
agree.
tions of the other formulas to the exercises.
require
a
small
i, which has the
We shall leave the verifica
(Equations (1.37)
and
(1.38)
trick.)
Cramer's rule allows for a simple description for solving the equation
Let A(i) be the matrix obtained
b when A is an invertible n xn matrix.
Ax
column
b. Then the equation
by replacing the rth column of A with the
turns
Ax
out, according to Cramer's rule
A_1b,
b, which is the same as x
=
=
=
to read
detA(i)
,
xl
.
t
1
=
< i < n
det A
This is checked out
by unraveling all the definitions and applying the formulas
of Cramer's rule: since
x
=
A_1b,
x-^A-^-^^-iy-detA^
But the summation is
just
the determinant of the matrix obtained by replacing
Thus we can solve by taking
the rth column of A with the column vector b!
quotients
of determinants.
Example
31. Solve the
equations
2x2- x3
**+ jt2 + 3x3
2x1 + 2x2+ x3
x1
+
=
2
=
0
=
l
74
1.
Linear Functions
The determinant of the matrix
is
easily found by cofactor expansion along the first
det A
=
1(1
-
6)
-
2(1
6)
-
-
1(2
-
2)
=
row :
5
By Cramer's rule
x1
=
|
(2
det 0
2
-1
-1\
1
=
|[2(-5)
+
1(6
+
1)]=-|
2
2
x2
x3
=
=
i
detj
1
idet(l
\2
-r
0
=
l[-2(-5)-(l+2)]=l
1
1
2
(the determinants
0)=i[2(0)+l(l-2)]=-i
1/
are
computed by
column cofactor
EXERCISES
33. Find the inverse of these matrices
(a)
(b)
(c)
(d)
expansion).
1.6
34. Solve the
Hi)
where A is
equation
the matrix in Exercise
the matrix in Exercise
(c)
A
'I
0
-1'
2
1
2
?
-1
ly
'4
6
1(
3
1
2
,0
4
0,
=
(d)
A
=
Suppose that the
if
=0
a/
75
given by
(a)
(b)
35.
Invertible Matrices
i
Show that A"
36. If A is
n x n
33(a)
33(b)
matrix A
=
(a/)
has this property:
<y
=
a
0.
matrix such that A"
=
0 show that I + A is invertible.
PROBLEMS
33. Show that if
Show that if there is
a
a
linear transformation T has rank n, it is invertible.
transformation S such that T S
I, T is invertible.
=
Equations (1.35)(1 .38) using the definition of the determinant.
35. Assume this fact about polynomials: A polynomial of degree d has
no more than a" roots.
Prove the following assertions:
(a) Let A be an n x n matrix. There are at most n numbers s such
34. Derive
that A + si is not invertible.
(b) The m x n matrix
/l
V=(l
\l
has
n2
r22
rV'X
r2
r
r2
rrl)
n
rrl)
determinant if and only if the r( are all distinct. (Hint:
0, there is a nonvanishing linear relation among the columns.)
a nonzero
If det V
=
36. Let
Rx1
*9
where
even
x\
.
.
=
.
,
n(*'-*0
x" axe,
distinct numbers.
if and only if
f(x'w
x")=f(x1,...,x")
Show that the
permutation
7t
is
76
1.
Linear Functions
and
similarly
f(x"w,
...,
w
**<">)
37. Let A be
x
is odd if and
-f(x\ ...,x")
=
an
only if
invertible
n x n
m) matrix formed from
(n
Show that B is also invertible.
A
matrix.
For m<n, let B be
by deleting any m rows and
(Hint: You need only take
an
(n
m)
m
columns.
m
=
\, and
proceed by induction.)
38. Let
H t)
Verify that
A2
-
(a + d)A + (ad-bc)I=0
that is, that A is a zero of a polynomial of degree 2.
39. The same fact is true for all n, that is an n x n matrix is the
zero of a
This is part of a famous theorem of algebra, which
goes like this: If A is any matrix, the polynomial
polynomial of degree
PA(x)
=
det(A
-
n.
xl)
is the characteristic polynomial of A. A is
PA(x) 0 (Cayley-Hamilton). That is,
a
root of the
polynomial equation
=
i^(A)=0
Verify the Cayley-Hamilton theorem for (i) a diagonal matrix, (ii) a triangular
matrix.
1.7
Eigenvectors
and
Change
of Basis
One fruitful way of studying linear transformations on R" is to find direc
along which they act merely by stretching the vector. For example,
if a transformation Tis represented by a diagonal matrix
tions
(1.39)
1.7
then
Eigenvectors and Change of Basis
T(Ej) ^E; where Ex
EB are the
a
factor
by stretching by
dt along the rth
standard basis vectors.
=
,
.
,
.
.
,
Tacts
T(x\
More
.
.
.
,
xn)
<*!*%
=
+
d2 x2E2
+
---
+
dx"En
generally, suppose we can find a basis \lt
by stretching along the direction of v;
Then, if
v
=
.
v of vectors in R"
for each i:
.
.,
d,yt
is any vector, the action of T is easily computed by referring v to
v
s'yt then T(v) Y,disiyi. T is represented
the basis ylt ...,y: if
by the diagonal matrix
a
Thus
direction:
such that T acts
T(yt)
77
basis of vectors
=
=
,
relative to this basis.
(1.39)
The process of
finding
which T acts
along
by stretching is called diagonalization.
not all transformations can be so diagonalized and this
Unfortunately,
a major difficulty in this line of investigation.
For example,
rotation in the plane clearly does not have any such directions in which
acts as a stretch.
More precisely, let T be represented by the matrix
presents
a
it
M-! i)
Then
If
v
=
T(x, y)
(a, b) is
(y, x). (T is
such that T(a, b)
=
da
=
b
Then d2a
=
db
db
=
(except 0).
Nevertheless, there
clockwise rotation
d(a, b),
we
through
a
right angle.)
must have
a
=
-a,
a
=
and there
are no
real numbers
d,
a
making this equation
true
this way, and it is
doing
many transformations which can be analyzed in
for
purpose in this section to study the techniques
are
our
so.
Definition 12.
R" be
Let T: R"
a
linear transformation.
number d for which there exists a nonzero vector
An eigenvector of T with eigenvalue d is a nonzero vector
of T is
a
An
eigenvalue
v
such that Tv
=
dy.
v
such that 7v
=
dy.
which there is
Proposition 20. // T: R" R" is a linear transformation for
with
a basis of eigenvectors y1,...,yn
eigenvalues du...,d, respectively,
then for any vector v
2! *S T(v) Z dis'y>
=
=
.
78
Linear Functions
1.
Compute T(y) using the fact that
Proof.
T is linear.
eigenvalues of a linear transformation T by making use
di is singular (not
an eigenvalue of T if and only if T
invertible). If A is the matrix representing T in terms of the standard basis,
[this condition is verified precisely when det(A dX) 0. Thus the eigen
Notice that when Tis rotation
values of T are just the roots of this equation.
by a right angle
Now
we
find the
of this remark: d is
-
u i)
det
which has
tion has
d2
=
+ l
explaining in
real roots, thus
no
no
d\
We shall
eigenvectors.
=
see
another way why this transforma
we extend the real number
that when
system in which every polynomial has
a root (the complex
represented in terms of (complex) eigenvectors.
This is one of the important reasons (particularly in the study of differential
equations, as we shall see) for so extending the number system. Let us now
to a
system
numbers),
then T
be
can
collect these observations.
Let T be
Proposition 21.
d is
A.
an
eigenvalue
det(A
-
fl)
=
a
transformation on R" represented by the
ifd is a root of the equation
matrix
ofT if and only
0
If d is an eigenvalue,
ofT-dl.
the
of eigenvectors corresponding
set
to
d is the kernel
Suppose d is an eigenvalue of T. Then there is a v ^ 0 such that
(T- dl)y 0. Thus the nullity of T di is positive, so T di is not
invertible. Thus, det(A
di) 0. On the other hand, if det(A dl) 0, then
Tdl is not invertible, so has a positive dimensional kernel. If v ^ 0 is in the
kernel, (T dl)(y)
0, or Ty
dy; thus d is an eigenvector of T.
7v
Proof.
dy,
=
or
=
-
=
=
=
Examples
32.
Let T be
-G D
represented by the matrix
-
=
1.7
Eigenvectors and Change of Basis
79
Then
*-H27' 1i,)
det(A fl) t2 3t + 2. The roots are r 2,
eigenvectors corresponding to t 2 is the kernel of
and
of
=
-
-
=
1.
The space
=
*--(! -?)
that is, the space of all vectors (x, y) such that x
0.
y
(1, 1) is an eigenvector with eigenvalue 2. The eigenvectors
=
-
sponding
to t
=
Thus
corre
1 lie in the kernel of
C o)
that
is, in the space of vectors (x, y) such that x 0. (0, 1) is such
eigenvector. Since (1, 1) and (0, 1) are a basis for R2, we have
diagonalized T. Relative to this basis T is represented by the matrix
=
an
33. Consider the transformation
basis
by
given,
relative to the standard
the matrix
GD
Then
fl)
det(A
eigenvalue of T.
-
=
r2
-
8r + 16
=
(t
-
4)2.
Thus
4
is
the
only
0}, which is one dimensional. Thus
there cannot be a basis of eigenvectors for the only eigenvectors he
on the line x
2y.
Notice that this example differs from that of a rotation, for there
is no problem with the roots; the difficulty lies with the transformation
has
as
kernel
{(x, y):
=
itself.
x
+ 2v
=
80
Linear Functions
1.
R3 be given by the matrix
34. Let T: R3
0
2
2
4
3
0
8/
\
Then
det(A
det(A
are
-18\
1-1
A=
tl)
-
fl)
-
=
=
1
-
3
+
3t2
-
0
2, -1.
2:
Eigenvalue
-18\
1-9
0
2
0
4
3
0
6/
A -21=
\
The kernel is the set of vectors
(x,
space is two dimensional,
with eigenvalue 2; for example, vx
so we can
1
Eigenvalue
A
+ 3z
=
1
4
3
0
9/
0
which is
or
=
one
0
or
[2
0
0
2
0
\0
0
-1/
a
(x,
x
=
3z
y
=
2z
such that
x
+ 2z
=
0.
This
=
independent eigenvectors
(0, 1, 0), v2 (2, 0, 1).
=
y,
z)
such that
( 3, 2, 1). These
eigenvector is v3
basis of eigenvectors, and T is represented by
An
dimensional.
v1; v2 v3 thus form
the matrix
,
z)
-18\
0
2
The kernel is the set of vectors
2x + y + 4z
y,
find two
:
1-6
-(-1)1=
\
x
The roots of
4.
0\
relative to the basis yu v2
=
(1.40)
,
v3
.
1.7
Eigenvectors and Change of Basis
81
Jordan Canonical Form
Notice that in general there are two difficulties with the procedure de
The polynomial det(A
fl) may not have many real roots,
scribed above.
and it may have multiple roots. As we shall see in the next section the first
difficulty can be overcome by transferring to the complex number system.
34 above demonstrates that the second
Example
possibility,
that of
multiple
roots, may not be severe, whereas Example 33 shows that it can seriously
handicap the diagonalization procedure. Continued study of this situation
becomes
the
quite difficult and we shall not enter into it. The conclusion is
typical matrix which cannot be diagonalized is of this form
0\
Id
1
0
0
0
0
d
1
0
0
0
0
0
d
1
0
0
0
0
0
d
0
0
[o
that
d)
0
the transformation
representing
T(x\
.
.
.
,
a")
=
(dx1
+
x2, dx2
+
x3,..., dx")
Given any matrix, we can find a basis of vectors (which includes all possible
eigenspaces) relative to which T decomposes into pieces, each of which has
the form (1.41). This is called the Jordan canonical form.
Change of Basis
Before
allow
x
us
in R"
leaving this subject, let
to change bases in Rn.
can
x
uniquely.
=
us
If
compute explicitly the formulas which
E} is a basis for R", then any
{E1;
.
.
.
,
be written
*%
+
+
x"En
w-tuple (x1,
E} denoted xE
We shall refer to the
relative to the basisE:
{Eu
.
.
.
,
.
.
.
,
x")
as
the coordinate
of x
.
of
F} be another basis for R". Let xF be the coordinates
{Fu
associate
we
can
To each set of E coordinates x
x relative to this new basis.
In this way we can
the F coordinates xF of the point corresponding to xE
The precise relation is this.
write xF as a function of xE
Let F:
.
.
.
,
.
.
82
Linear Functions
/.
E,.=
Proof.
=
x
E},
of the
^
=
yl
(x\
.
.
.
,
x"),
xF
=
(y\
...,
be two
F}
different
is denoted
AFE.
Then
y").
i>'F<
=
1
=
^
which is the
(1.42)
(AFE)~
that
=
*
=
AEF.
For, given any xf
AF AF XF
I.
relative to any basis E:
T(x)E
(1.42).
same as
Now, if T is any linear transformation
{Et
,
.
.
on
.
,
R", it
can
Let
E}.
be
us
represented by
a
denote that matrix
Tx
Proposition 23.
R" is
=
.,
2(1^/^^,
=2"=i aj'xJ>
AF XE
A/A/
Tf
.
(1.42)
Notice that it follows from
and T: R"
.
2ZxjVj=2Zx4Z"j'F<)
Thus for each i,
matrix,
byTE:
{F1;
A/x
1
a
F:
E's:
have this relation between its E andE coordinates:
we
=
Thus
.,
change of basis matrix, and
is called the
in R"
=
XF
.
aj%
Let xE
x=
.
;=i
For any point
xF
{E1;
Write the E's in terms
(a/)
The matrix
Let E:
22.
Proposition
bases for R".
IfE:{Elt..., E}, F: {Ft
,.F}
transformation, we have
,
a
linear
(A/)-1TA/
.
.
.
are
two
bases
of R",
1.7
Eigenvectors and Change of Basis
83
Proof.
T(x)F
On the other
r(x)F
Thus TF
=
=
AFET(x)E
=
A/Tx
=
AF*TA/xF
hand, by definition
=
TfxF
ArTEA/
=
(A/r^A/
Examples
35. Let T: R2
R2 be represented, relative
to the standard basis E
by
*-G J)
Let F:
{(1, 1), (2, -1)}
be another basis.
Find the matrix
Tf
Now,
*-G -?)
A/=(A/)-i=i(; _2)
Thus
*-G -96 K -?)
-i\
/7/3
\4/3
2/3,/
36. Let T be
given, relative
0
2
2
4
3
0
8/
\
and let F:
that F is
{(0, 1, 0), (-2, 0, 1), (-3, 2, 1)}.
a
respectively.
by
-18\
1-1
T=
to the standard basis E
basis of
Thus
we
We have
already
seen
for T, with eigenvalues 2,2, 1,
may conclude that Tf is given by (1 .40).
eigenvectors
.
84
Linear Functions
1.
EXERCISES
37. Find
represented
(a)
a
basis of
eigenvectors,
if
possible, for the transformation
by the matrix A:
in terms of the standard basis
1
eigenvalues: 2, 3,
(b)
eigenvalues: 1, 1, 0,
2
(c)
4
eigenvalues: 1,
(d)
/
1
1^
-1
0/
38. Show that for F: {Fi,
F} a basis for R", and E the standard basis,
the matrix AEF is just the matrix whose columns are Fi,
F
.
.
.
,
.
.
39. Find the matrix A/ for these
.
,
.
of bases in R".
pairs
(1, 0, 1), (0, 1, 1), (1, 0, 0)
E: (0,1, 2), (2, 0,1), (1,2,0).
(b) F: (1,0,0), (2, 0,1), (0,1,0)
E: (3, 1,5), (0,2, 3), (-1,-1,0).
(c) F: (1, 0, 1, 0), (0, 1, 1, 0), (0, 0, 2, 0), (0, 0, 1, 1)
E: (0, 2, 0, 2), (2, 0, 0, 0), (2, 0, 2, 0), (0, 2, 2, 0).
40. Let T: R3
R3 be a linear transformation represented by
(a)
(b)
/ 2 0 0\
/l 0 -1\
F:
(a)
TE=
\
-1
0
1
0
relative to the standard basis E.
(F
in Exercises
39(a)
3
TE=
1/
Find TE
,
0
1
\2
0
where F is
one
one
4
-1/
of these bases
and
39(b)).
independent eigenvectors with the
eigenvalue, then T is represented by a diagonal matrix in any basis.
as
of
41. If T:R2-rR2 has two
same
PROBLEMS
40. Prove
Proposition
41. If T is
which has
Pa(x)
and
=
n
a
=
on
R"
represented by the matrix A
distinct eigenvalues di,...,d, then
(- l)"(x
PA(A)
20.
linear transformation
0
-
dt)(x -d2)---(x- d)
(PA is defined in Problem 39).
1.8
Complex Numbers
42. Let T be a linear transformation on R". Let
E(r)
Show that E(r) is a linear subspace of R" (called the
Show that if r # s, then (r) n (5)
{0}.
=
r
{v
e
R" : Tv
eigenspace
85
=
of
rv}.
T).
=
43.
if ru
.
Pa(x)
Suppose
.
.
=
,
r are
A represents
the
(- l)"(x
-
a
linear transformation
eigenvalues of A, then
r,)
d'm
<ri>
(x
-
rk)
Verify the Cayley-Hamilton theorem for
44. Find
a
matrix with
no
n
d'm
=
2*=i
on
R" with this property :
dim
Then
(">
A.
nontrivial
eigenspaces.
expect to prove the Cayley-Hamilton theorem for such
1.8
E(rj).
How would you
a
matrix ?
Complex Numbers
Pythagoras' discovery, that yjl is not the quotient of two integers, was
day to be a geometric mystery. His conception of numbers
was limited to rational numbers and his desire to measure
lengths (to associate
numbers to line segments) led to this unhappy realization: there are some
lengths which are not measurable ! (as the hypotenuse of an isosceles right
triangle of leg length 1). It took a long time for mathematicians to realize
that the solution to this situation was to expand the notion of number. The
general liberation of thought that was the Renaissance led in mathematics to
the possibility of expressing the value of certain lengths by never-ending
decimals, or continued fractions, or other types of infinite expressions. It
was during those days that mathematicians formulated the view that such
expressions represented numbers and served to determine all lengths. Earlier,
Middle Eastern mathematicians were led from certain algebraic problems
considered in his
concept in another direction. As they
1 has no square root; some bold adventurer then
observed, quite clearly
suggested that we contemplate, in our minds, some purely imaginary quantity
to envision extension of the number
As
1 and treat it as if it were another number.
whose square would be
this supposition did not contradict any of the known facts concerning the
number system, it could do
(at least in our minds).
no
harm
and
might
do
a
great deal of good
Today we need not be so mysterious or cunning in our ways. We need
only recall that there is a 2 x 2 matrix (see Problem 20) whose square is the
negative of the identity. We can thus say quite factually that in the set of
1 does indeed have a square root. Well, there is also a
2x2 matrices,
5x5 matrix, and an n x n matrix for any n whose square is I, so we should
1 has a square root. The
ask for the smallest algebraic system in which
-
Linear Functions
1.
86
complex number system is this system and we shall later derive the remarkable
fact (the fundamental theorem of algebra) : Every polynomial has a root in
the complex number system.
Now,
to be
explicit,
the matrix
-G "D
(1.43)
-I. The complex number system is the collection
has the property that i2
the
form al + bi, where a, b are real numbers.
of
of all 2 x 2 matrices
=
Definition 13.
C,
the set of
complex
numbers is the collection of all 2
x
2
matrices of the form
Proposition 24.
(i) The operations of addition and multiplication are defined on
(ii) Every nonzero complex number has an inverse.
(iii) C is in one-to-one correspondence with R2.
Proof.
(i) (a
-b\
\b
(ii)
a)
+
(c -d\_(a +
c)
\d
la
-b\lc
\b
a)\d
\b
c
c)
-(b + d)\
+ d
-d\^/ac-bd
C.
a
+
c
J
-(ad+bc)\
ac-bdj
\ad+bc
If
r)
is nonzero, then one of a or A is nonzero,
an inverse.
By Cramer's rule
M
-,_
1
a2 +
/
a
so
det M
=
a2 + b2 = 0, and thus M has
b\
b2\-b a)
(iii) is obvious, since every complex number is given by
conversely every pair (a, b) of real numbers gives rise
and
a
pair of real numbers
a complex number.
to
1.8
Cartesian Form
We need
of a Complex
now a
Complex Numbers
87
Number
notation which is
more
convenient than the matrix notation,
from
(iii) above. The matrices I and i correspond to the
points (1, 0), (0, 1) of the plane and thus form a basis for C. More explicitly
and
we
get
our cue
eD-c.;K-i)
al + bi
=
(1.44)
identify the real number 1 with the identity matrix I; and more
generally the real number r with the complex number rl + Oi, then we can
In fact, the complex
say that every real number is also a complex number.
with
a
root
of
1 tacked on.
the
real
number
is
system
square
just
system
of
that
Arabian
circle
back
to
the
us
full
takes
original
conception
(This
adventurer. The difference here is that we now know what we mean by this
procedure and that it produces no inconsistencies.)
Thus, we can suppress the identity matrix in the expression and write a
complex number in the form a + bi. We now recapitulate the relevant facts.
If
we
C is the set of all 2
x
2 matrices
c
=
a
+ bi with a, b real numbers,
a
is
Re c, and b is the imaginary part, written
the real part of c, written a
lmc. And these following rules hold:
b
=
=
i2=-\
(a
+
bi)
(a
+
(c
+
di)
bi)(c
+
di)
(a
bi)-1^
+
=
=
(a
ac
+
Polar Form
(b
+
+
(bd
'
d)i
+
ac)i
whena2
b2
+
Z>2/0
of a Complex Number
Since C is in one-to-one
numbers
a2
+
bd +
-
U~
+
c)
by points
plex
numbers is the
seek
a
correspondence with R2, we can represent complex
plane (see Figure 1.14). Addition of com
in the
same
addition of vectors in the plane. We
of multiplication of complex numbers.
as
geometric description
this purpose it is convenient to
Let
Definition 14.
distance from the
\z\=(x2
+
z
origin:
y2y
=
x
move
+
yi.
to
now
For
polar coordinates.
The modulus of z, written
\z\, is
its
Linear Functions
1.
88
Z
=
x
+
iy
u
\argz
s'
i
x
0
Figure 1.J4
The
z, written arg z, is defined for
argument of
the ray
which
on
z
arg
=
z
tan
angle defining
x
write
can
=
z
= 0; it is the
-
complex numbers
r
coordinates (r, 9) then, since x
We
z
lies :
r(cos
=
9 +
sin
j
in
polar
cos
9,
y
form : If
=
r
sin 9,
a
=
we
x
+
yi has the polar
have
9)
have moved the i in front of sin 9 for the obvious notational con
venience which results.) The set of points of modulus 1 is the unit circle
(We
It is the set of all points of the form cos 9 + i sin 9.
this to cis 9. Precisely, cis 9 is the point of
abbreviate
We shall sometimes
on
the
unit
circle
the
Now, let z, w be two complex
ray of angle 9.
lying
centered at the
origin.
numbers,
z
=
r
cis 9
w
=
p cis (b
Then
zw
=
=
=
=
Thus
we
modulii and
r cis(9)p cis((j>)
(r cos 9 + ir sin 9)(p cos <b + ip sin <b)
rp(cos 9 cos (f> sin 9 sin <b) + irp(cos
rp cis(0 + <b)
form the
product
of two
complex
9 sin <b +
numbers
adding the arguments. (This does not make
numbers is zero, but that case is trivial anyway.)
cos
$
sin
9)
by multiplying
sense
if
one
the
of the
1.8
Notice
z"
then, if z
=
=
p cis
9, then z2
p2
=
Complex Numbers
cis 29 and
more
generally
p" cis n9
(1.45)
This observation leads to the fact that it is easy to extract roots.
converse of (1.45) is
zllk
p1* cis
=
k
and k
=
Write
rcisO
=
in
c
c
=
polar form:
zk=pk
Thus the modulus of
z
For the
j
Proposition 25. Let c be a complex number,
precisely k distinct solutions to the equation Xk
Proof.
89
cis
c
r
=
cis fl.
If
z
an
integer.
There
are
c.
=
p cis
is
</>
a
solution, then
<
is the kth root of the modulus of c, and the argument of a
an angle, but so is
angle such that k times it is fl. Well, (l/)fl is such
(l//fc)(fl + 2tt). In fact, each of the angles
is
an
1 (fl), i (fl + 2tt), 1 (fl + 4tt),
.
.
.
have the property that fc times it is fl.
has precisely these k roots :
r1/k
cis
fl, r1'*
cis
.
,
.
.
,
r1'*
,
i (fl + 2(A
All these
1)tt)
-
angles
are
distinct,
so c
=
r
cis fl
cis
k
k
Complex Eigenvalues
We shall work
with the
extensively
complex
number system in this text.
shall discover many situations besides the algebraic one above
where study within the system of complex numbers is beneficial. In par
In
fact,
we
ticular, let
us
return to the
eigenvalue problems
of the
preceding
section.
We can define
space of -tuples of complex numbers.
linear transformations on C just as we did on R". In fact, the entire theory
We consider
of linear
Ej
C, the
algebra through
=
(0, 0,
.
.
.
,
Section 1.7 holds
0, 1, 0,
.
.
.
,
0)
(1
over
C
as
in the rth
well
place)
as
R".
Let
90
/.
Linear Functions
be the standard basis vectors for C.
C" is
given by
a
matrix A
=
(a/)
Again,
any linear transformation
on
of complex numbers relative to the standard
basis :
T(zi,...,zn)=(i:Jv)...,i^v)
for all
(z1,
.
.
.
,
z")
C.
e
Examples
37. Consider the matrix
"(-? J)
as
representing
dard basis.
Its
det(A fl) f2
Eigenvalue i:
=
a
transformation T
eigenvalues
+
1,
so
are
the roots
on
C2 relative
the roots of
are
det(A
to
fl)
the stan
0.
But
fl is
given
=
i, i.
-(:: ')
The second
row
is
-
i times the
the
first,
so
the kernel of A
by
single equation ix + y 0. An eigenvector is (1, i).
The eigenvalue i has the eigenvector (1, 0- Now, F:
(1,
0}
=
are a
basis of
eigenvectors for C2,
relative to this basis :
Tf
"(i -)
if
-CM-,-)
then
CM-,)
so
T becomes
{(1,0,
diagonalized
1.8
Complex Numbers
91
38. Consider the matrix
H? i 1)
representing a transformation T
f + 1.
det(A- fl)= -f3 + f2
-
1 i,
Since the roots
i.
,
ing eigenvector, there is
a
are
on
to the standard basis.
has
polynomial
distinct and each must have
basis of
a
C3 relative
This
We
eigenvectors.
a
the
roots
correspond
now
find such
basis.
1
Eigenvalue
A-I=(\
:
i
-1
2
1
|)
0/
I is found
The kernel of A
Such
(recall Example 18).
C!
2C2
-
3C3
-
=
(1, -2, -3)
Eigenvalue /:
A-iI=U
\ 2
[o\0 i
a
row
with
eigenvalue
1.
//
-
relation among the columns
reduction is
we
must
row
reduce-
-i + iM
0
0
/
A solution of the
taking z3
(_3 + 4/,
we
linear relation among the columns
I
1
1
In order to find
eigenvector
an
-i-
The result of
as a
relation is
0
is
Thus
a
=
l
find the
corresponding homogeneous system
5, then
-
we
obtain z2
=
1
-
3i,
zt
=
is found
-3 + 4i.
by
Thus,
eigenvector with eigenvalue i. Similarly,
to the
eigenvector (-3 4i, 1 + 3i, 5) corresponding
3/, 5) is
an
-
92
/.
Linear Functions
i.
eigenvalue
Thus,
T is
represented by
relative to the basis
(1, -2, -3), (-3
+
4i, 1
3i, 5), (-3
-
-
4i, 1
+
3/, 5).
EXERCISES
42. Find the inverse of these
(a)
(b)
(c)
5-3/
(d)
(e)
(1-0/2
complex numbers:
4cis(2/3)
cis 7
3 + /
43. Show that z_1
44. Show that the
z if and only if z is on the unit circle.
complex number cis fl represents rotation in the plane
=
through the angle fl, when considered
45. Find all Mi roots of
as a
2
x
2 matrix.
z:
(a) k 2,z=-i.
(d) k 3,z i.
-l
(b) k 5,z
(e) k 2,z 3i-4
15 + 5/
(c) k=4,z l+i
(f) yt 3,z
46. Find, if possible, a basis of possibly complex eigenvectors for the
transformations represented by these matrices
(a)
=
=
=
=
=
=
=
=
=
=
G
O)
I
i
-i
0
0
PROBLEMS
45.
Compute that the matrix
(-!!)
G"J)
has square
so
equal
to I.
We have chosen i to be the 2
x
2 matrix
that the correspondence between complex numbers and
operations on
More precisely, we conceive a complex number in two
R2 will be correct.
ways: as a certain transformation on the plane, and as a vector on the plane.
Given two complex numbers z, w we may interpret their
product in two
1.9
Space Geometry
93
composition of the transformations, or the application of the transfor
corresponding to z to the vector vv. We would like these two
interpretations to have the same result. If z a + ib, w
c + id, show
ways :
mation
=
=
that zw,
c -% 1
and
all the
are
same
under that correspondence.
46. Show that the complex numbers z, z, when considered
R2 are independent (unless they are real or pure imaginary).
Why do the complex eigenvalues of
47.
real matrix
a
as
vectors in
in
conjugate
come
pairs?
1.9
Space Geometry
In this section
shall introduce the basic notions of three-dimensional
we
vector notation.
geometry, using
First of all,
as
in the
plane,
we
select
a
in space, called the origin and denoted 0. That being done
we may refer to the points of space as vectors and think heuristically of the
directed line segment from the origin to the point as a vector. The operations
particular point
and addition
be defined
of scalar
multiplication
expressed
in terms of coordinates in much the
can
same
as on
the
plane
and
way:
(i) If P is a vector and r a real number, rP is the vector lying on the line
through 0 and P and of distance from 0 equal to |r| times the length of the
If
segment OP.
r >
0, rP lies
on
the
same
opposite side.
(ii) if P, Q are two vectors in space,
in the plane determined by P and Q,
side of 0
as
P; if
r
<
0,
rP lies
on
the
We define P +
Q
there is
a
unique parallelogram lying
three of whose vertices
are
0, P, Q.
to be the fourth vertex.
the coordinatization of space.
Having chosen a point
new points with the property that 0, Eu E2
three
be
origin,
El5 E2 E3
E3 do not all lie on the same plane (we say the vectors E1; E2 E3 are not
coplanar). The three lines determined by the vectors E1; E2 E3 are called
Now,
we
turn to
let
as
,
,
,
,
the coordinate
Ej, E2 E3
,
axes.
enables
us
Just
to
in two dimensions the choices of the vectors
each line in one-to-one correspondence with the
as
put
real numbers.
In three dimensions two lines determine
a
plane.
We shall call the
planes
94
1.
Linear Functions
Figure
1.15
through 0 determined by Et and E2 the 1-2 plane, by Et and E3 the 1-3 plane,
and the plane determined by E2 and E3 is the 2-3 plane (Figure 1.15). These
three planes are called the coordinate planes. Each of these planes can be put
into one-to-one correspondence with R2 just as in the case of two dimensions.
Now, to each point in space we can associate a triple of numbers relative to
these choices in the following way. Let P be any such point. There is a
unique plane through P which is parallel to the 2-3 plane; and this plane
intersects the 1 axis in a unique point. This point has the coordinate x1
relative to the scale determined by E^ We shall call x1 the first coordinate
of P. The second, x2, is found in the same way : by intersecting the plane
through P and parallel to the 1-3 plane with the 2 axis. Finally we find
the third coordinate x3 similarly, and associate the triple (x1, x2, x3) to P.
In this way we put all of space into one-to-one correspondence with R3,
dependent upon the choice of vectors Els E2 E3 called a basis for space.
The expression in terms of coordinates of the operations of addition and
scalar multiplication are precisely the same as in R2 (no matter what basis
is chosen) :
,
r(x\ x2, x3)
(x1, x2, x3)
+
(y1, y2, y3)
=
=
,
(rx1, rx2, rx3)
(x1
+
y\ x2
+
v2, x3
+
y3)
1.9
Space Geometry
95
need to check that these formulas correspond to the geometric
descriptions given above; we need only refer to the computation in the plane.
When we are interested in the pictorial representation of problems of
There is
no
three-dimensional Eculidean geometry it is best if we consistently use a
particular coordinatization. For this purpose we select the "right-handed
coordinate system"; where the coordinate axes are mutually
perpendicular and the order 12 3 is that of a right-handed screw (see
rectangular
Figure 1.16). It is common in particular problems to refer to the coordinates
by the letters (x, y, z) rather than (x1, x2, x3). We shall use the numbered
coordinates when it is
more
convenient to do
so.
Inner Product
Now, the basic notions of Euclidean geometry
will be of
ordinates.
and the
importance
to us to derive
expressions
are
length
and
angle.
for these in terms of
It
co
Consider first the length of the line segment OP between the origin
P with coordinates (x, y, z). This can be easily computed
point
by use of the Pythagorean theorem (consult Figure 1.17).
point of intersection with the xz plane of the line through
Then OPP' is a right triangle, so
to the v axis.
|0P|2= |0P'|2+ |P'P|2
Figure
1.16
Let P' be the
P and
parallel
96
Linear Functions
/.
>
P{x,y,z)
.si,-
Figure 1.17
Letting P" be the point of intersection with
and parallel to the z axis, we obtain
|0P|2
But
now
|0P"|2
=
|0P"|2
+
|P"P'|2
x2, |P"P'|2
=
v2
+
Now, suppose P(x,
y,
|0P|
[x2
=
+
+
=
the
x
axis of the line
|P'P|2
z2, |P'P|2
=
y2,
so
z2]1/2
z), Q(a, b, c)
any two
are
definition of addition, P is the fourth vertex of the
whose vertices are 0, P
Q and Q. Thus, the side
-
the
same
length
IPQI
=
the side
as
l(P
-
Q)0|
through
=
[(*
-
Q and 0,
P
a)2
+
(y
b)2
-
Finally, we can compute the angle between
(consult Figure 1.18); if 9 is that angle, then
|PQ|2
In
=
|0P|2
+
through P'
|0Q|2
-
2|0P| |0Q|
+
so
(z- c)2]1/2
P and
cos
points in space. By
parallelogram three of
through P and Q has
Q by the law of cosines
9
coordinates,
(x
-
a)2
+
(y- b)2
+
(z- c)2
=
x2
(1.46)
+ y2 + z2 + a2 + b2 + c2
-2(x2 + y2+z2)112
x(a2 + b2 + c2)1/2cos9
1.9
97
Space Geometry
which reduces to
xa
COS0
+
=
(x2
y2
+
+
yb
+
z2)ll2(a2
cz
+
The form in the numerator thus has
with the notion of length determines
b2
+
(1.47)
c2)112
some special importance: it together
angles. It is called the inner product
of the two vectors P, Q.
Let P,
Definition 15.
Q be
two vectors in space.
Their Euclidean inner
product, denoted <P, Q>, is defined as |P| |Q| cos 9, where 9 is the angle
between P and Q. In coordinates, P
(xu yu Zj), Q (x2 y2 z2),
=
=
,
(P,Q}
Propositions
i/<p,Q> o.
=
,
x1x2 + y^2 + zxz2
26.
The
nonzero vectors
P and
Q
are
perpendicular if and only
=
Proof. P and Q are perpendicular if and only if the angle fl between them is a
right angle, fl is a right angle if and only if cos fl 0, and this holds precisely
0.
When <P, Q>
=
=
A
plane through
vector
the
perpendicular
origin
to such a
n=<x.N>=o
is the linear span of two vectors. If N is
plane n> then 11 is iven by the ecluat'on
a
98
1.
More
Linear Functions
generally, if p is a point on a plane (not necessarily through
orthogonal to FJ, F] is given by the equation
the
origin)
and N is
<x
-
N>
p,
0
=
through the origin is the linear span of a single vector, and can be
expressed by two linear equations (since a line is the intersection of two
planes).
A line
Examples
39. Find the
equation of the plane through (1, 2, 0) spanned by the
(1, 0, 1), (3, 1, 2). If N (n1, n2, n3) perpendicular to
two vectors
this
plane
=
must have
we
<N,(1,0, l)>
<N, (3, 0, 2)>
A
(1
=
of
1,
,
=
=
solution
N
n1+3
3k1 +n2
=
this
0
2n3=0
+
is (1,-1,-1), so
equation of the plane is
system
Then the
1).
<x- (1,2,0), (1, -1, -1)>
=
0
or
<N,
P
-
Letting
R
N
=
0
=
<N, Q
(m1, n2, n3),
=
take
equation of the plane through P (1, 0,
1), Q
(3, 1, 1). If N is perpendicular to the plane, we have
=
R>
may
x-y-z+l=0
40. Find the
(2, 2, 2),
we
-
we
R>
=
-
=
0
obtain this system of
equations:
-2n1-n2-2n3=0
n1
-
+
n2
which has
<x
-
Ix +
N, R>
4y
+
n3 =0
a
solution N
=
+ 3z
0,
=
41. Find the
or
l(x
=
-
(7, 4. 3).
3) + 4(y
Thus the
-
1)
+
3(z
-
equation we
1) 0 or
28
equations of the line L through (4, 0, 0) and perpen
Example 40. If x is on L we must have
dicular to the plane in
<x
-
(4, 0, 0),
P
seek is
=
-
R>
=
0
<x
-
(4, 0, 0), Q
-
R>
=
0
1.9
may take these
so we
2x + y + 2z
+ y +
-x
z
as
the
99
Space Geometry
equations :
8
=
-4
=
Vector Product
Given two noncollinear vectors vx, v2 in space, the set of vectors perpen
a line.
We shall now develop a useful formula for select
dicular to v1; v2 is
ing
a
If N is
vector on that
particular
on
that line, and
<x, N>
=
x
for all
x e
Vl
uniquely
a
R3.
=
vector
,
product
we
vt
have
0
On the other hand, since x, vx, v2
Now there is
line, called the
is in the linear span of vt and v2
This is
are
coplanar we
have
determined vector N such that
easily
(V, vt2, vt3),
seen
y2
=
using coordinates.
(v2\ v22, v23),
x
=
Write
(x1, x2, x3)
Then
det(
Vj
)
=
W
=
2
vt3
v^
.,2
.,3
vx2
22
detj V
1
V2/
x3
x2
Ix1
(x \
1
XKV.W V3V22) *W23 lW)
+ x3(v11v22-vl2v21)
(ix1, X2, X3), ((Vl2V23 1-! V), (V.W V^W),
-
~
-
-
-
=
(v.W-v.W))}
x
v2
.
100
1.
Linear Functions
Definition 16.
The vector
V x w
Proposition
(i)
<x,
(ii)
(iii)
(iv)
Let
product
=
v
=
v x w
(v2w3
(v1, v2, v3),
is defined
v3w2, v3w1
-
w
(w1, w2, w3)
=
be two vectors in R3.
by
-
v1w3, v1w2
-
v2wx)
27.
v x
v x w
w>
det
=
vl
forallxeR3.
w x v.
=
orthogonal to v and w.
The equation of the plane through
the equation <x, v x w>
0.
is
v x w
the origin
spanned by
v
and
w
has
=
The
proof
discussion.
for
example,
<u,
of this proposition is completely contained in the
preceding
The basic property of the vector product is the first; it follows,
that for any three vectors u, v, w
w>
v x
Notice that if v,
ordered basis u
proposition gives
of
=
<u
x
v,
w>
=
collinear,
<v,
w x
u>
=
<v
x
w,
u>
0. If they are not collinear, the
right handed (see Figure 1.19). The following
important geometric interpretation of the magnitude
w are
v x w
=
vxwis
v
an
v x w.
Proposition
(i)
The
28.
area
Let u, v,
w
be three noncollinear vectors.
of the parallelogram spanned by
wx v
Figure
1.19
u
and
v
is
||u
x
v||.
1.9
(ii)
The volume of the
|detQ
parallelepiped spanned by
J
v
This follows
u, v, and
w
101
is
'
Proof.
Step (a). The first step is to verify (ii) in case
perpendicular. In that case we must show that
det/
Space Geometry
||u||
=
the vectors u, v,
w are
mutually
||v||- ||w||
-
easily from the multiplicative property of the determinant.
First
we
note that
/u\
/<u,u>
(u,
v
v,
w)
=
\w/
\
0
0
0
<v, v>
0
0
0
\
<w,w>/
(i,j)th entry is the inner product of the rth
of the second (see Problem 49). Thus,
since the
;th
row
detjv j =detjv )(u,
Step (b). In particular,
perpendicular, so
v,
if u,
w)
=
v are
llu
so
||u
x
v||
=
||u||
x
of the first matrix with the
||u||2||v||2||w||2
perpendicular,
(u
=
row
x
v\
u
J
then u, v,
u x v are
mutually
||u||- ||v||
vll
||v||, when
is
perpendicular
part (i) in general.
u
to
v.
angle between
prove
the
of
by u and
area
the
spanned
Then
parallelogram
1.20).
(see Figure
is the product of the base and the height of the base:
Step (c).
and
Now
we
v
area
=
||u||a
=
||u||
||v||sinfl
Let fl be the the
u
v
102
/.
Linear Functions
Now the vector
by
u
and
u
<v,
=cosg-fl)
sin
Since
(u x v) is orthogonal to u and
orthogonal to u. We have
u x
and is
v
and
u x
HTll
l|U
(u
X
X
orthogonal, by Step (b)
u x v are
v,
so
lies in the plane spanned
v)>
x
(U
u x
T)||
we
have ||u
x
(u
x
v) ||
=
||u ||
||u
x
v||
Thus
area
=
<v,
||u
Nu
||r||
=
<U
(u
u x
X V, U X
v>
l|u|iVrrruxy ^-
v)>
x
(u
x
x
v)||
lluxv||
=
u
The volume of the parallele
Step (d). To prove part (ii) we refer to Figure 1 .21
piped spanned by u, v, w is the product of the area of the base and the altitude:
.
volume
Since
u X t
is
,
sin
=
||u
x
y\\b
orthogonal
<6=C0S
(tt
-
\2
=
||u
x
to the u,
\
4>\
7
||w||sin <j>
v||
v
|<w,
=--!
plane,
u>
xv>|
_
llwll-Hu xvll
u
X(u
Figure 1.20
X
v)
1.9
Space Geometry
x V
Figure
1.21
Thus
..
volume
A final
=
|lu
equality
x
v|
<w,uxv>
l|w||-n-T7]
w
uxt
-=
det
/w\
u
/
which will prove useful is this :
||uxv||2=||u||2||v||2-<u,v>2
This follows
easily
from the above arguments :
||uxv||2=||u||2||v||2sin24>
||u||2||v||2(l-cos20)
=
=
since the
angle
llu||2|
between
<u, v>2
u
and
v
is (b.
EXERCISES
47. Which
vx=(2, 1,2),
t5= (1,3,0),
pairs of the following
vectors
are
orthogonal?
t,=(3, -1,4), v3=(7,0,5),
v6= (0,0,1),
tt=(-15,5,21)
v*
=
(6, -2,5)
104
1.
Linear Functions
48. Find the vector
products
v, x V/
for all
pairs of
vectors
given in
Exercise 47.
49. Find
vector
a
<v,v,>=2
<v,v2>
where vi, v2 v3
50. Find the
such that
=
-l
<v,v3>=7
given in Exercise 47.
equation of the plane spanned by the
(c) t5, v6, (d) v2, t4, where the v,
,
(b)
v
v2, v5,
are
(a) Vi, y6
given in Exer
vectors
are
,
cise 47.
51. Find the
of the line
equation
spanned by the
vectors
given in
Exercise 47.
52. Find the equation of the plane
through (3, 2, 1) and orthogonal
(7, 1, 2).
53. Find the equation of the line through (0, 2, 0) and
orthogonal to the
plane spanned by (1, -1, 1) and (0, 3, 1).
54. Find the equation of the line through the origin and
perpendicular
to the plane through the points
(a) E,,E2,E3
(b) (1,1, 5), (0,0, 2), (-1,-1,0)
(c) (0, 0, 0), (0, 0, 1), (0, 1, 0)
55. Find the equation of the line of intersection of the two
planes,
(a) determined by (a), (b) of Exercise 54.
(b) determined by (a), (c) of Exercise 54.
(c) determined by (b), (c) of Exercise 54.
56. Find the plane of vectors perpendicular to each of the lines deter
to the vector
mined in Exercise 55.
57. Let A be
(a)
if the
tions of Ax
a
a
3
=
3 matrix.
x
rows
of A lie
b forms
a
line,
(b) if the rows of A lie
plane, or is empty.
58. Show that ||v x w||
the two vectors v and w.
59. Is the vector product
=
(u
x
v)
x w
=
u x
(v
x
Show that
on a
or
on a
||v||
plane (but
not on a line), the set of solu
is empty.
line, the set of solutions of Ax b forms
=
||w|| sin fl, where fl is the angle between
associative; that is, is
w)
always true?
60. If v,
v, v x w,
w are
v x
(v
X
two noncollinear vectors show that the three vectors
w)
are
pairwise orthogonal.
PROBLEMS
48. Prove the identities of Proposition 27.
49. Let vj, v2, v3 be three vectors in R3.
Let A be the matrix whose
1.10
rows are
Vi, v2
(a) the (i,/)th entry
(b) det
A
50. Let P be
=
of Linearity
and B the matrix with columns Vi,
of AB is <v, Vj>.
v3
,
Abstract Notions
v2
v3
,
.
105
Show that
,
det B.
given by
the coordinates
(x, y, z) relative to a choice Ei, E2
Show that the point of intersection of the line
through P parallel to the Ei axis with the 2-3 plane has coordinates (0, y, z).
,
E3 of basis for space.
Abstract Notions of
1.10
There
with
a
are
Linearity
many collections of mathematical objects which are endowed
algebraic structure which is very reminiscent of R". To be less
natural
vague, there is defined, within these collections, the operations of addition
and multiplication by real numbers. Furthermore, the problems that
naturally arise in these other contexts are reminiscent of the problems on R"
which we have been studying. The question to ask then, is this: does the
same theory hold, and will the same techniques work in this more general
We shall see in this section that for a large class of such objects
context ?
(the finite-dimensional vector spaces) the theory is the same. We shall see
later on that in many other cases, the techniques we have developed can be
modified to provide solutions to problems in the more general context.
First, let us consider some examples.
Examples
42.
[0, 1],
If/ and
then
g
are
we can
continuous real-valued functions on the interval
cf as follows:
define the functions /+ g,
(f+g)(x)=f(x) + g(x)
(cf)(x) cf(x)
=
Clearly,/ +
g and
c/are
of addition and scalar
C([0, 1])
also continuous.
multiplication
of all continuous functions
Thus
are
on
that
we see
defined
on
the interval
operations
the collection
[0, 1].
example, if/ and g are differentiable, so are/+ g
and cf. Thus the space ^([O, 1]) of functions on the interval [0, 1]
with continuous derivatives also has the operations of addition and
scalar multiplication. Notice that the operation of differentiation
takes functions in CHC0, 1]) into C([0, 1]): if /is in CH[0, 1]) it
has a continuous derivative, so /' is in C([0, 1]). Furthermore,
43. In the above
106
Linear Functions
/.
differentiation could be described
as a
linear transformation:
if+g)'=f'+3'
(cf)' cf
=
is, by the
So
way,
integration
a
linear transformation:
\if+g)=lf+U
l(cf)
ctf
=
The fundamental theorem of calculus says that differentiation is the
inverse operation for integration:
(J/)' =/
merely a curious way of describing the
implied point of view has led to a wide
The subject of functional analysis which
range of mathematical discoveries.
20th
in
the
was developed early
century came out of this geometric-algebraic
to
long standing problems of analysis.
approach
These remarks may strike you
well-known phenomena, but the
as
Examples
44. If S and T
are
linear transformations of R" to Rm, then
so
is
the function S + T defined by:
(S
+
We
T)(x)
also
can
(cS)(x)
=
=
S(x)
+
T(x)
multiply
a
linear transformation
by
a
scalar:
cS(x)
Thus the space L(R", Rm) of linear transformations of R" to Rm has
defined on it operations of addition and scalar multiplication.
45. We have
of
n x n
In
fact,
we
used, in
M" this way it
These
already observed (Section 1.6) that the collection M"
on it these two important operations.
matrices has defined
was
an
just
essential way, the fact that when
the
same as
examples, together with R", lead
we
viewed
R"2.
to the notion of
an
space : a set together with the operations of addition and scalar
We include in the definition the algebraic laws governing these
abstract vector
multiplication.
operations.
1.10
Definition 17.
Abstract Notions of Linearity
107
An abstract vector space is a set V with a
distinguished
on which are defined two
operations:
and vv are elements of V, then v + vv is a well-defined element
element, 0, called the origin,
Addition.
If v
of V.
Scalar
multiplication. If v is in V and c is a real number, cv is a well-defined
These operations must behave in accordance with these laws :
element of V.
(i)
(ii)
(iii)
(iv)
(v)
(vi)
The
of the
v
+
(vv
v
+
vv
t>
+ 0
+
=
=
c(v w)
ct(c2 vv)
+
\w
x)
(v
=
vv)
+
+ x,
+ v,
vv
v,
cv
=
=
+ cw,
(Ci c2)w,
vv.
=
preceding examples are all abstract
required laws are easily performed.
vector spaces; the verifications
We
now
want to
investigate
the
extent to which the ideas and facts discussed in the case of R" carry over to
abstract vector spaces. First of all, all the definitions carry over sensibly
to the abstract
case
vector space V.
case :
if
Thus
we
just replace
we
the word R"
take these notions
as
by
the words
an
abstract
defined also in the abstract
linear transformation, linear subspace, span, independent, basis, dimension.
one bit of amplification necessary in the case of dimension.
Now there is
We have until
now
encountered spaces of only finite dimension.
Example
46. Let R
an
be the collection of all sequences of real numbers.
an ordered oo-tuple,
Thus
element of I?00 is
(x1,*2,. ..,*",...)
R00 is
an
abstract vector space with these
operations:
(x1,x2,...,x,...) + (y1,y2,...,yn,...)
x" + y",
.)
(x1 + y1, x2 + y2,
c(x\ x2,...,xn,...) (ex1, ex2, ...,cx",...)
=
.
.
.
,
.
.
=
be the
has an infinite set of independent vectors. Let
This
are zero but for the nth, which is 1
entries
whose
all
of
sequence
entire collection {Eu ...,,.. .} is an independent set. For if
there is a relation among some finite subset of these, it must be of the
Now R
.
form
c1^
+
+
ckEk
=
0
108
Linear Functions
1.
(of course,
many of the c's may be
c1El
+
so
+
ckEk
if this vector is
indeed the set
We
=
must have
{Eu ...,,...}
make the
now
(c1, c2,..., ck, 0, 0,
zero we
following
shall
vector space; and
we
holds also in this
more
is
an
general
.
.
c1
.)
=
c2
infinite
=
=
ck
independent
=
0.
Thus
set on/?.
restriction to the so-called finite-dimensional
that all of the
see
But
zero).
preceding
information about R"
case.
Definition 18. A vector space V is finite dimensional if there is a finite set
of vectors vl,...,vk which span V. That Rx is not finite dimensional
follows from some of the observations to be made below. It can also be
verified in the terms of the above definition
(see Problem 53). The important
they are no different from
result about finite-dimensional vector spaces is that
the spaces R".
Proposition
Let V be
29.
a
finite-dimensional
basis for V, every vector in V
Ifvt,...,vdisa
combination of vu
v
=
x1v1
(x1, ...,xd)
The
.
.
.
,
+
xdvd
is called the coordinate of
correspondence
be
vd:
+
v
of dimension d.
expressed uniquely as a
vector space
can
(x1,
.
..,
xd)
is
v
a
relative to the basis vlt...,vd.
one-to-one linear transformation
of V onto R*.
Proof.
The definition of basis
(Definition 6) makes this proposition quite clear.
(Problem 54).
We leave the verifications to the reader
What is not
so clear is that every finite-dimensional vector
space has a
and
that
basis,
every basis has the same number of elements.
However,
once these facts are established the above
proposition serves to reduce the
general finite-dimensional space
1.3 through 1.6 carry over.
Proposition 30.
to one of the
R", and the results of Section
Every finite-dimensional vector space V has a finite basis,
same number of elements, the dimension
of V.
and every basis has the
Proof. Suppose V is finite dimensional. Then V has a finite spanning set. Let
{vi,
vd} be a spanning set with the minimal number of vectors; by definition V
has dimension d. We shall show that {vi,
vd} is a basis.
...,
...,
Abstract Notions of Linearity
1.10
Since {vu
vd}
span, every vector in V can be written as a linear combination
We have to show that there is only one way in which this can be
...,
of these vectors.
done.
Suppose for
some
x'Vi H
=
109
vector
h x*vd
=
have two different such ways :
v we
yhi
+ +
y"vd
(1 .48)
Then
(x>
y>i +
-
Vj-i, fJ+i,
equation
.
.
.
yd)vd
.
.
.
=
these elements
y' for
some
Thus
j.
Thus it must be
d.
That any two bases have the
same
Hence
{vL,
.
.
.
...,
But this contra
to span all of V also.
serve
in two different ways.
vd
,
0
must have xJ #
we
assumption about
dicts the minimal
in terms of Vi,
-
says that vj is in the linear span of the a- 1 elements vit
so
vd,
,
(xd
expressions differ
Since these two
Now this
+
impossible to express
vd} is a basis.
v
,
number of elements follows
easily
from
Proposition 28 (see also Problem 55). Let T: V^>Rd be the linear transfor
mation associating to each vector its coordinate relative to the above basis
{vu...,vd}. If {wu...,wd} is another basis, let S: K1R be the same
S T~ is a one-to-one
coordinate mapping relative to this basis. Then L
0. Thus (rank + nullity
linear mapping of Rd onto R3, so p(L)
5, v(L)
=
dimension) :
5
=
=
=
=
d.
PROBLEMS
a
{vi, ...,vk] in
51. Show that for any finite set of vectors S
vector weR which does not lie in their linear span [S].
v'
represent the first (k + l)-tuple of entries in
R", there is
=
span
there is
Rk+1,
a
.
.
Since v/,
.
vector w' in Rk+l which cannot be written
nation of v/,
vt'. Let
52. Are the vectors E,,
.
v.
,
.
w
.
.
=
,
(w', 0,
E
,
.
.
.
.
.
.
(Hint:
.
v/
,
Let
cannot
combin-
as a
.).)
in R described in
Example
43
a
basis
for Ra ?
53. Let Ro" be the collection of those sequences of real numbers
x",
.) such that x" 0 for all but finitely many n. Then R0^
(x1, x2,
are a
E
is a linear subspace of R". Show that the vectors Ei
=
.
.
.
.
,
.
,
.
.
.
,
,
.
.
.
basis for i?0.
54. Prove
of
a
Proposition
29.
that any two bases
by following the arguments in Section 1.4,
finite-dimensional vector space have the same number of elements.
55. Prove,
110
1.
Linear Functions
Show that the collection L(V,
56. Let V, W be two vector spaces.
linear transformations from V to W is
a
W) of
vector space under the two opera
tions :
(a)
(b)
if
e L( V, W), (cL)(x)
cL(x),
L(V, W), (L + L')(x) L(x) + L'(x).
R, L
c g
if L, L'
6
=
=
57. What is the dimension of L(R",
58. Show that
a
Rm)l
vector space V is finite dimensional if there is
a
one-to-one
linear transformation of F0 into J?" for some n.
59. Show that a vector space F is finite dimensional if there is
a
linear
transformation T of R" onto V for some n.
60. Verify that the collection P of polynomials is an abstract vector space.
For a positive integer n, let P be the collection of polynomials of degree not
Show that P is a linear subspace of P. Show that P is not
more than n.
finite dimensional, whereas P is. What is the dimension of P1
61 Let x0
c another collec
x be distinct real numbers and c0
.
,
.
.
.
,
,
Show that there is
tion of real numbers.
one
.
.
.
and only
,
one
polynomial p in
P such that
p(xt)
=
(Hint:
ct
0<Li<,n
Let L:P
that L has rank
62. Let g be
G(p)
n
J?"+1 be defined
(p(x0)
a
polynomial, and define the function
a
linear function.
63. Define Dk:P^-P:
of A?
64. Let x0
and
only
\
p(x0) =d0
(Hint:
e
Dk(p)
R, and let
one
=
p(x)).
G: P
Show
P:
Describe the range and kernel of G.
What are the range and kernel
dfy/dx!'.
c0
ck be given numbers.
polynomial p in P such that
,
dp
.
.
,
Use the
same
Show that there is
dkp
d~t(-X)=Cl'""dxk ^
idea
as
=
C"
in Exercise
65. Does Dk:P-+P have any
66. Show that C([0, 1]) is not
1.11
=
=pg
Show that G is
one
by L(p)
+ 1 .)
61.)
eigenvalues?
a
finite-dimensional vector space.
Inner Products
The notion of
length, or distance, is important in the geometric study of
spatial configurations. In Section 1.3 we studied these concepts
and related them to an algebraic concept, the inner product. From the
point of view of analysis also it is true that these concepts are significant:
planar
and
1.11
it is in terms of distance that
"convergence."
(vv1,
.
.
.
By analogy
express "closeness" and in particular
with R3 we define the inner product in R",
,
vv"),
<v, w>
product of
by <v, w> is defined
We shall say that v is orthogonal to
between v and w is defined by
\y\
=
v
than the
v
=
(v1,
...,
v"),
w
=
as
w
if <v,
w>
=
The distance
0.
d(v, w)
v
and 0,
d(y,0)=V(vt)2-l1<2
d(y, w)2
<v
two vectors
is the distance between
Distance in R" behaves much
Pythagorean theorem holds:
when
shall, in this section,
we
{X(vi-wi)2-\1'2
=
|v| of a vector
The modulus
here
v'w'
=
d(y,yv)
we are
The inner
denoted
111
we can
and in terms of it, distance. While
introduce some topological terms.
Definition 19.
Inner Products
w,
sum
=
d(y, x)2
+
as
it does in R2 and
d(y, x)
+
points
are no
further apart
(1 .50)
d(x,yv)
These facts will be verified in the
the
particular,
(1 .49)
=
<
in
d(x, w)2
w
x> 0. In any event, two
of the distances from a third,
d(v, w)
R3;
problems.
Topological Notions
The ball in R" of radius R > 0 and center c, denoted B(c,
points whose distance from c is less than R:
Definition 20.
is the set of all
R),
B(c,R)= {xeR":d(x,c)<R}
A set S is said to be
centered at
each of its
Thus,
a
c.
a
neighborhood
of
a
point
c
if it contains
A set V is said to be open if it contains
a
points.
set S is
a
neighborhood
of
c
if there is
some
R
some
ball
neighborhood
(presumably
of
very
112
Linear Functions
1.
small)
such that
d(x, c)
R
<
implies
x e
S
A set U is open if for every c e U, there is an R such that U => B(c, R).
For suppose xeB(c,R). Then d(x, c)
that any ball is open.
R
d(x, c) > 0. Now B(c, R) contains the ball of radius R
Notice
<
-
-
centered at
For if y is
x.
d(y, c)
Here is
a
d(y, x)
^
+
point in
a
that ball, then
d(x, c)<R- d(x, c)
collection of formal
properties
+
R,
so
d(x, c)
by (1 .50),
d(y, c)
=
R
of the collection of open sets.
Proposition 31.
(i) R" is open.
n U.
(ii) IfUu..., U are open, so is Ut n
(iii) // C is any collection of open sets, then the set of all points belonging
to any of the sets in C is open.
(This set is denoted [j U).
Proof.
(i) Clearly, R" contains a ball centered at every one of its points.
Then there are Ri
(ii) Suppose Ui,..., / are open, and x is in every U,
U => B(x, if). Let R
R such that Ui => B(x, Ri),
min[.Ri,
Rn]. Then if
In
is
in
each
Thus
is
in
n
n U
is
in
each
so
Ut
Ui
y
B(x,
Ri)
d(y, x)<R,y
n U => B(x, R).
Thus Ui n
n U is a neighborhood of
particular, Ui n
any one of its points x, and is thus open.
(iii) Suppose C is a collection of open sets. If x is in any one of them, say U,
then since U is open there is an R such that U => B(x, R). Thus, \JV ec U => B(x, R).
Thus [Jvee Uisa neighborhood of any one of its points, so is open.
.
=
.
.
.
.
,
.
.
,
.
.
Many of the concepts a mathematician studies are so-called local concepts:
They happen in a neighborhood of a point, or are determined by what goes
on near a point; far behavior being irrelevant.
Differentiation is thus local,
whereas
integration is
The
importance of open sets is that it is precisely
study these local concepts, since their definition
at a point depends on behavior in some neighborhood of the point.
If a set is open its complement, the set of all points not in the given set, is
said to be closed. Thus, S is a closed subset of R" if R"
S
{x e R" : x S}
is open. Corresponding to Proposition 31 we have this proposition about
on
such sets that
we
not.
should
-
closed sets.
Proposition 32.
(i) R" is closed.
(ii) If Su
Sn
.
.
.
,
are
closed,
so
is
S^v
u
S.
=
1.11
Inner Products
(iii) IfC is a collection of closed sets, then the set of all points
the sets of C is closed. (This set is denoted [)s c S).
113
common
to
all
6
Problem 67.
Proof.
Notice that there
are
sets which are both open and closed.
There
are
R" and 0 are the only ones. There are also sets which
many of them.
neither open nor closed, and there are many of them. For example,
not
are
an
interval is open in R1 if it contains neither end point, closed if it contains
both, and neither open nor closed if it contains only one end point.
We
are
acquainted
That
with the notion of
"dropping
a
perpendicular"
in the
line and p is a point not on the line, then we can drop
from p to / as in Figure 1 .22. The point p0 of intersection
is, if / is
plane.
a perpendicular
of the perpendicular with / is the point on / which is closest to p. A more
sophisticated way of describing this situation is to say that p0 is the orthogonal
projection of p on /. The concept of orthogonal projection generalizes to R"
and will prove quite useful there. In order to discuss this problem, we shall
generalize even further.
a
A Euclidean vector space is an abstract vector space V on
which is defined a real-valued function of pairs of vectors, called the inner
product, and denoted <, >. The inner product must obey these laws:
Definition 21.
(i)
(ii)
(iii)
(iv)
(v, v} > 0. If (v, v}
<v, w>
<w, v}.
a(v, vv>.
(av, vv>
=
0, then
v
=
0.
=
=
<>!
+ v2
,
w>
=
<!>!, w>
+
<v2 w}.
,
\
\
*p
\
\
\
\
\
\
\
\
\
\
\
^
\
\
Figure 1.22
114
1.
Linear Functions
Euclidean vector space when endowed with its inner
C[0, 1 ] of continuous functions on the unit interval is a
It is clear that R" is
The space
product.
a
Euclidean vector space with this inner
</, 9>
=
(f(t)g(t)
product:
dt
We leave it to the reader to
verify that the laws (i)-(iv) are obeyed. It is
(i)-(iv) are all that is essential to the notion of inner
interesting
that
such
function behaving in accordance with those laws
;
is,
any
product
will have all the properties of an inner product. Despite the inherent interest
in this
metamathematical
point, we shall not pursue it further, but take
it for granted that the above definition has indeed abstracted the essence of
that the laws
"
"
this notion.
In terms of an inner product
length and orthogonality:
llll
v
1
on a
vector space
we can
define the notions of
[<, t>>V'2
=
if and
vv
only
if
<y, vv>
=
0
The
important bases in a Euclidean vector space are those bases whose
are mutually orthogonal.
More specifically, we shall call a set
{Elt ...,} in a Euclidean vector space V an orthonormal set if
vectors
||,||
Et
1
for all
1
=
i
for all i # /
Ej
If the vectors
Eu
E span V we shall call them an orthonormal basis.
(Any orthonormal set of vectors is independent Problem 68.) The basic
geometric fact concerning orthonormal sets is the following:
Proposition
...,
33.
Let V be
orthonormal set in V.
is
orthogonal
to
a
Euclidean vector space and
v in V, the vector
v
v0
For any vector
the linear span
=
-
{l5 ...,}
?= <t>, jE;> Et
x
Sof{E1,...,En}.
n
Proof.
Let
w
=
i
<v, w>
=
<y
2 Ci E,
=
-
be in S.
Then
i
2
'=i
<v, ,> <, w>
,
=
iv, w>
-
J
i=i
an
<t>, E,> <, w>
,
1.11
Inner Products
115
Now
< w>
<,
=
,
2 CjEj>
,
=
,
=
c,
n
2 ctE,y
<y,
<, ,>
j=i
ft
<t>, w>
2 c./
=
j=i
2
=
c,
<y, (>
1=1
1=1
Thus
<t>, w>
=
1
2
Theorem 1.8.
be
an
c,
<v, E,y
-
1
=
1
Let V be
orthonormal set in V.
v0=
2
=
a
<y, -Ei)
c,
=
o
1
Euclidean vector space, and let
{Eu ...,}
For any vector v, let
<<;,,>,
i=l
Then
\\v\\2=\\v-v0\\2+\\vQ\\2;
(i)
(ii)
for any
vv
in the linear span of
{Eu ...,},
\\v-v0\\2<\\v-yv\\2
Proof.
(i)
II" II2
=
<(y v0) + v0,(v- v0) + v0>
\\v-v0\\2+ \M\2+ <.V0,V-Vo> + <V-V0,Vo>
<v v~>
The last two terms
span of
{Ei,
-
=
,
=
...,
(ii) ||y w||2
are zero
by the preceding proposition, since
=
=
<y
<,v
llw
vv,
vv>
v
wy
Vo + v0
w,v
foil2 + \\v0-w\\2+<v-v0,Vo-w) + <Vo-w,v-v0y
v0
+
v0
Again, the last two terms are zero for both
Thus
span of {,,...,}.
v0
,
w
and thus also
||t,_vv||2= ||_0||* + ||t;o-w||2> Hf-Uoll2
(ii) is
proven.
is in the linear
E}.
=
so
v0
v0
w
is in the linear
116
1.
Linear Functions
Gram-Schmidt Process
v' is orthogonal to the linear span S of {Eu ..., }.
v0
v0 is the vector in S which is closest to v; it is called the orthogonal projection
of v into S. It seems, by Theorem 1.8 that one needs an orthonormal basis
Notice that
v
=
orthogonal projections; the following proposition gives a
obtaining orthonormal basis for finite-dimensional vector
thus with it, orthogonal projections.
in order to find
for
procedure
spaces, and
Proposition 34. Let Fu ...,F be a basis for a Euclidean
can find an orthonormal basis Eu ...,Enso that the linear
Fjfor allj.
Ej is the same as the linear span ofF1,
We
.
.
The
Proof.
NPiir'pi.
Now in
proof is by induction
general, let Fi,
.
.
.
the linear span W of P\,
apply the proposition to W
.
.
orthonormal basis with the
F be
,
.
a
on
ofEu
...,
.,
n.
If
basis for
F-L is
vector space V.
span
a
n
=
1,
we
need
only take Ei
Euclidean vector space V.
=
Then
Euclidean vector space also, and we can
the inductive hypothesis. Let Eu ..., E-i be an
,
a
by
required properties.
Now,
we must
find
a
vector
En
such that
1
11^11
(E,E,)=0
=
all/^K
Fn is in the linear span of E,
If E is
Thus
a
we
.
.
.
,
E
vector that fulfills the last two
need
En
=
only find
F-
a
vector
conditions, then we can take E
||^ \\~1E
filling the last two conditions. That is easy ; take
=
.
2(Fn,E3)Ej
ji
Then, for
i < n,
( E,)
=
,
(F Ed
-
,
2 (Fn Ej)(Ej E,)
,
,
J<n
=
(Fn E,)
,
-
(F E,)(E, ,,)= 0
,
Furthermore,
F=+
so
>Z(F-,Ej)Ej
the last two conditions
The
are
fulfilled and the proposition is proven.
proof of this proposition provides a procedure for finding orthonormal
1.11
bases in
an
Euclidean vector space, known
as
Inner Products
the Gram-Schmidt process.
111
It
goes like this:
any basis
First, pick
1
=
Fu
=
.
,
F of
V.
Take
=
Fj+i
.
.,
Ej
be the vector of
J+1
are
by
the
length
to find
found, take
(Fj+i, i)i
~
and divide
(F2, E^)Elf
-
.
and let
.
11^111-^1
Then choose 2
F2
and so forth. If Eu
E+i
.
length
(FJ+l, E2)E2
-
one
-
collinear with
-
(FJ+l, ,);
+i.
Examples
47.
F1
F2
F3
=
=
=
Apply
the Gram-Schmidt process to this basis ofR3:
(1,0,1)
(3, -1,2)
(0, 0, 1)
Take
Fx
/
J_
_1_\
El"lFj'V2*0,>/2/
E2
a_1)2)_(3.^
=
-(3. -1.2)-
+
(-i).o
+
2.ii)(-L,o,ii)
(|,0.|) (|. -1.^)
-
Then
E2
=
E3
{i) [r~1'^r)
=
(0, 0, 1)
-
=
\Wr2'~\v
'(up)
-^ (-^,0, -^) -^3 (^75.
"(7) '(W71)
+
2,
118
1.
Linear Functions
and
finally
~2
I
-
~3
2
\
\(17)1/2'(17)1/2'(17)1/2j
3
48. Find
an
X(x\ x2, x3, x4)
orthonormal basis for the kernel of X: R4
=
of all,
First
(1, 0, 0,
x1
let
+
us
1/2), (0, 1, 0,
-
Schmidt process,
we
x2
-
+
x3
+
-*
R,
2x4.
pick a suitable basis for K(X); that is,
1/2), (0, 0, 1, 1/2). Applying the Gram-
obtain
2
E1
=
=
7^ (i, o,o,^i)
\(3V)m'\6) '^(mr^J
E:
(30)1
E3
=
(1094)1/2
49. Find the
T:
orthogonal projection of (3, 1, 2) into the kernel of
R3^R:
T(x,
y,
z)
=
x
+
2y
+
z
Now the kernel of T is
Applying
Ei
=
(l)1/2(2,
Thus the
spanned by Ft
the Gram-Schmidt process,
-
1, 0)
E2
=
orthogonal projection
(l)1/25(l)1/2(2, -1,0)
+
=
we
(2,
-
=
(0,
-
1, 2).
obtain the orthonormal basis
(fWj, -1,
of (3, 1,
1, 0), F2
1)
2) into this plane is
(f)i/2l(f)^(_i
_
|, 1}
=
m,
_az
f)
1.11
50. Find the
L:
x
+
y
3y
+
point
z
=
0
z
=
0
on
which is closest to
(
line
(the
^(7 1
'
'
closest
0)J'
(7, 1,0).
(-*. -L3)\
/
(26)1'2
L is
the
linear span of the vector
of (7, 1,0) on this
orthogonal projection
is
point)
(-4,-1,3)^27 l
(26)1'2
26
'
'
'
EXERCISES
61. Which of the
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
following sets are open; closed; or neither.
{xeR:2< |x-5|<13}.
{xeR:0<x<4}.
{xeR:x>32}.
{X6fi":<x,x>=4}.
{x6A3:<x,(0,2,l))=0}.
{xe#3:2<||x-(3,0,3)||<14}.
{xeR":x1>0,...,x">0}.
The set of integers (considered as a subset of R).
(i)
{xeR":
2 *''<)
1=1
x
|>'a'^l}.
(j)
{xeR-.
(k)
{xe/?":2(^)3<2^')2}.
62. Find the
+ 3y + 2z
on
the plane
4
=
closest to the
point
point (1, 0, 1).
point on the line
63. Find the
x
+ 7v +
x
z
=
2
z
=
0
closest to the
64. Find
(a)
(b)
(c)
(d)
point (7, 1, 0).
an
orthonormal basis for the linear span of
Vl
=
Vi
=
v,
=
vi
=
(0,
(0,
(0,
(1,
2,
1,
3,
2,
(1, 0, 2), t, (1, 2, 4).
(1, 1, 2, 3).
(1, 0, 1, 0), v3
0, 1), v2
(0, 0, 2,
(0, 6, 0, 3, 0), y3
0, 0, 0), y2
(2, 1, 4, 3).
(4, 3, 2, 1), v3
3, 4), y2
2),
119
the line
Thus the
4, -1,3).
Inner Products
y2
=
=
=
=
=
=
=
=
-
1, 1).
120
/.
Linear Functions
65. Find orthonormal bases for the linear span and kernel of these
transformations
(a)
R*:
on
0>)
'8
/8
6
1
0
0\
1
2
1
2
1
2
0
2
2
1
2
1
1
-1
1
1
0
1
0
3
3
0
1
7
4
1
-2
0
PROBLEMS
67. Prove
Proposition
69. Give
an
32.
orthogonal set of vectors is independent.
example of a sequence {U} of open sets such that f)"=i U
68. Show that
an
is not open.
70. Give an example of
is open.
71. Find
an
vector space
sequence
{C} of closed
sets such that
orthonormal basis for the linear span of 1,
C([0, 1])
In the next four
inner
a
with the inner
problems
product, denoted <
,
V
product </, gy
represents
a
=
C
x2, x3 in the
\fg.
vector space endowed with
an
>.
72. Let v, w, x be three points in V such that v
x.
Show that the Pythagorean theorem is valid :
w
x,
(J"=i
x
is orthogonal to
\\v-W\\2=\\v-x\\2+\\x-w\\2
of
73. Let v, w be two vectors in V.
w which is closest to v is
Show that the vector in the linear span
(1.51)
w
v0
(You can verify this by minimizing the function f(t)
calculus.)
74. Prove Schwarz's inequality:
\\v
tw\\2 by
\<v,wy\^\\v\\-\\w\\
for any two vectors in
V.
(Hint: [|f
o
II2
=> 0 where v0 is
(1.50).)
75. Prove the
triangle inequality:
ll-x||^||y-w||+||w-x|l
for any three vectors in V
(use Schwarz's inequality).
given by
1.11
76. Let V be
is
vector space with
a
an
Inner Products
inner product.
Suppose that
subspace of V. Let J_( W) {v: <u, w> 0 for all w e IF}.
called the orthogonal complement of W. Show that L(W) is
a
=
subspace of
V and
F is finite
(if
121
dimensional) that
W and
W
This is
=
linear
a
1_(W) together
span V.
11. Let T: R"
A.
-*
Show that the
78. Show that
/?m be
a
linear transformation represented by the matrix
of A span (K(T)).
linear transformation is one-to-one
rows
a
the
on
orthogonal
complement of its kernel.
FURTHER READING
R. E. Johnson, Linear Algebra, Prindle, Weber & Schmidt, Boston, 1968.
covers the same material and includes a derivation of the Jordan
This book
canonical form.
K. Hoffman and R. Kunze, Linear Algebra, Prentice-Hall, Englewood
This book is more thorough and abstract, and has a full
Cliffs, N.J., 1961.
discussion of canonical forms.
L. Fox, An Introduction to Numerical Linear Algebra, Oxford University
Press, 1965. This is a detailed treatment of computational problems in
matrix
theory.
Nickerson, D. C. Spencer, and N. Steenrod, Advanced Calculus,
Van Nostrand, Princeton, N.J., 1957. This set of notes has a full treatment
of all the abstract linear algebra required in modern analysis.
H. K.
MISCELLANEOUS PROBLEMS
79. Show that if A' is obtained from A
by
a
sequence of
row
operations
0.
0, A'x
80. Show that every nonempty set of positive integers has a least element.
81. Show that a set with n elements has precisely 2" subsets.
then these
equations have the
same
solutions: Ax
82. Show that the -fold Cartesian
product of
=
a
=
set with k elements has
k" elements.
83. Can you
interpret the
case
k
=
2in Problem 82
so as
to deduce the
assertion of Exercise 3 ?
84. Let A
r
the
as
=
(a/)
be
an n x n
Show that A"~r
> 0.
=
0.
i>r for
assumption j
matrix such that
Show that the
some r
> 0.
a/
same
Will the
=
0 if
i
j>
r
for
some
conclusion follows from
hypothesis \i
j\
>
r
do
well?
85. Let T: R"
there
->
Rm be
a
linear transformation of rank
r.
Show that
linear transformations S,: Rm-^Rm-', S2: R"-'^Rn such that
(a) Si has rank m r and b e R(T) if and only if Sib 0.
are
=
(b) S2 has rank
86. Suppose that T:
-
R"
r
->
and
x 6
K(T) if and only if
R" and Tk
=
/.
x e
Show that T is invertible.
Show that the linear span
intersection of all linear subspaces of R" containing S.
87. Let S be
a
subset of R".
R(S2).
[S] of
S is the
Linear Functions
1.
Show that
88. Let S, T be subsets of R".
dim([S
r])
u
<
dim([5]) + dim([r]),
equality holds if and only if [S] n [T]
89. Let V and W be subspaces of Rn.
=
and
{0}.
Let X be the set of all
sums
subspace of R". The
+
V,
V + W. If
relationship between X and V and W is indicated by writing X
the form
in
written
be
xeJcan
in addition V r\ W={0}, then every
we say
n
W
V
W
with
X
this
0,
In
V+
one
case,
v + w in only
way.
v
w
with
w 6
v e
W.
Show that X is
linear
a
=
=
=
of V and IF and write X=V@W.
90. Suppose X= V W. Then dim X= dim F + dim IF.
91. Show that if A: R" -> .R is a linear function, there exists a
that X is the direct
sum
w e
.R" such
that A(v)
<v, w> for all v e /?".
92. If S is a subset of R" define
=
1(5)
=
{v
e
R":
<v, s>
=
0 for all
s e
S}.
{0}.
(a) Show that (S) is a subspace of .R" and that S n _L(S)
that
Show
=(.(S)).
[S]
(b)
F J_(F).
(c) If V is a linear subspace of .R", R"
93. Suppose that T: V-> Wis a linear transformation and Fis not finite
dimensional. Show that either the rank or the nullity of T must be infinite.
=
=
94. Let V be
an
A bilinear function p
abstract vector space.
function of two variables in V with these
on
V is
a
properties:
cp(v, w)
p(v, cw)
p(cv, w) cp(v, w)
p(v,
p(vi + v2,w) =p(vu w) + p(v2 w)
=
=
,
Wi
+
w2) =p(v, Wi) + p(v, w2)
In fact, the space
sum of two bilinear functions is bilinear.
Bv of all bilinear functions is an abstract vector space. If V is finite dimen
sional, what is the dimension of Bv ? (Hint: See the next problem.)
Show that the
95. Let p be
a
bilinear function
on
R".
Let
a,;j=piE,,Ej)
Show
that/? is completely determined by
96. Let V be
(a)
the matrix
(at;j).
abstract vector space.
Show that the space V* of linear functions
an
under addition and scalar
on
Fis
a
vector space
multiplication.
d also.
(b) If dim V=d, show that dim V*
(c) Show that to every A e R"* there isaweff such that A(v)
<v, w> for all v e Rn. (Recall Problem 91.)
97. Suppose that V is a linear subspace of W. We define the annihilator
of V, denoted ann(F), to be the set of A e W* such that X(v)
0 if v e V.
=
=
=
1.11
Show that ann(F) is a linear subspace of IF*.
show that ann(F) has dimension n
d.
98. Let V be
linear
a
subspace
linear transformation.
R"
-=?
J?m defined
on
If dim W
=
123
dim V
n,
=
d,
of .R", and suppose that T: V->Rm is a
a linear transformation 7":
Show that there is
all of R" which extends T.
99. The closure of
every
Inner Products
a
neighborhood of
set
x
S, denoted 5, is the
contains
points of
S.
set of all
points
x
such that
Find the closure of all the
sets in Problem 61.
1 00. Show that the closure of a set S is the smallest closed set containing S.
101. The boundary of
a
set
S, denoted dS, is the
set of all
points x such
complement
that every neighborhood of x contains points of both S and the
of S. Find the boundary of all the sets in Problem 61.
102. Show that the
103. Show that the
boundary of a
boundary of a
set is
a
closed set.
set S is also the
S n (Rn
plement
fact, show that 8S
104. Let T: V-> W be a linear transformation of
R"
inner
S.
-
In
=
-
boundary of its
com
S).
vector space with
a
an
The adjoint of Tis the transformation T*: W-+ V defined
product.
in this way
<T*(w), vy
=
for all
<w, Tv>
(a) Show
(b) If T:
that T* is
a
v e
V
well-defined linear transformation.
represented by the matrix A (a/),
represented by the matrix A* (a*j), where a*j
(This matrix is called the adjoint or transpose of A.)
(c) Show that R(T*) is complementary to K(T).
v(T), v(T*) p(T).
(A) In fact, P(T*)
R"
--
Rm is
=
T* : Rm^ R" is
=
then
=
atJ.
=
=
105. A bilinear form pona vector space V is called symmetric if it obeys
the law: p(v, w) =/>(w, v) for all v and w. An inner product is a symmetric
manipulations with inner products
symmetric bilinear forms. For example, the GramSchmidt process (Proposition 32) gives rise to this fact (see if you can work
the proof of Proposition 32 to give it):
Proposition. Let p be a symmetric bilinear form on V. Suppose Fi,
We can find another basis, Eu...,EofV such that the
F is a basis for V.
Fj for all j, and
linear span of Eu...,Ej is the same as that of Fu
p(E,,Ej) 0ifi^j.
We shall call such a basis Ei,..., E p-orthogonal.
106. Let/? be a symmetric bilinear form on a vector space V, and suppose
E is a p-orthogonal basis.
Ei,
(a) Show that p(v, w) can be computed in terms of this basis as
bilinear form and much of the formal
remains valid for
...,
.
.
.
,
=
...,
follows : if
p(p, w)
=
v
=
2
v'Et
2 v'w'p(E,
1=1
,
,
w
E,)
=
2
W'E> then
>
(1.52)
/.
Linear Functions
(b) Show that
p is an inner
product
on
the linear span of the Et
such that p(E,, E,) >0.
(c) Similarly,
p is an
inner
product
on
the linear span of the Et
such that p(E, E,) < 0.
107. Prove this fact: Let p be a symmetric bilinear form on a finitedimensional vector space V. There is a basis Eu ...,E , integers r, s such
that r + s <; n and such that if v
2 v'-Ei then
,
=
,
Xm)=2M!fr
2 (f')2
0-53)
rlr+s
(Hint: Modify the basis {,} in Problem 106 so that (1.52) becomes (1.53).)
108. The integers r, s of Problem 107 are determined by p alone, and are
independent of the basis. Here is a sketch of how a proof would go.
Suppose Fi, ...,F is another p-orthogonal basis and p is the number of
r.
Let W be the linear
Ft's such that p(Ft Ft) > 0. We have to show p
E
span of these F's. Expressing points of W in terms of the basis Ei
we may consider the transformation T: W->R" given by
=
,
TQv'Et)=(v\...,tf)
T is one-to-one
on
W, for if
0<p(w,w)=2 ("')21ST
so we
w e
W, and
w
= 0,
2 (')2
rir+i
must have
2(')2>o
tsr
on
W.
Since T is one-to-one, it follows that r ;> p. The inequality p >; r
same argument with the roles of Eu
E and FU...,F
follows from the
.
.
.
,
interchanged.
109. Let A
=
(aci) be
Then A determines
PA(y, w)=
a
a symmetric n x n matrix, that is, a,;
j
symmetric bilinear form on R" as follows:
=
a,;
,
2 aiww'w-'
i.j
If P is the matrix
corresponding to the change of basis from the standard
basis to that described in Problem 105, then P*AP is diagonal. Verify that
assertion.
1.11
Inner Products
125
110. Find the p-orthogonal basis and the
107 for the symmetric bilinear forms
[4
(a)
3
p(y, y)
=
p(y, v) >0, =0,
(v1)2 + (i;2)2 + (v3)2
transformation
112. A
T:
v,
w e
Show that if T is
V).
self-
v, w are eigenvectors of a self-adjoint transformation T
eigenvalues. Show that <v, w> 0.
self-adjoint transformation on R", and v0 e Rn is such that
Suppose that
V with different
114. If T is
2(V)2
=
a
=
land
<Ty0, v0> =max{<rv, v>;
then v0 is
an
2 0>')2
eigenvector for
=
1}
T.
115. Use Problems 113 and 114 to prove the
adjoint operators
Theorem.
T
a
K(T)R(T)
113.
on
given by
self-adjoint if it is self-
V-> V is called
=
=
<0 in R* where p is
(v*)2
-
<v, 7w> for all
adjoint 7v, w>
on R", then
transformation
adjoint
R?
(b)
l\
0
111. Describe the sets
representation (1 .53) of Problem
given by these matrices:
can
be
on
There is
computed
r(2*'E,)
=
Spectral
theorem for self-
R":
an
orthonormal basis Ei,
in terms
.
.
.
,
E of eigenvectors of T.
of this basis by
2*'c.E,
1 1 6. Find a basis of eigenvectors in R* for the self-adjoint transformations
given by the matrices (a), (b) of Problem 110.
117. Orthonormalize these bases of R*:
(a)
(b)
(1, 0, 0, 0), (0, 1, 1, 1), (0, 0, 2, 2), (3, 0, 0, 3).
(-1, -1, -1, -D,(0, -1, -1, -1), (0,0, -1, -1),
(0,0,0,-1).
(c) (0, 1, 0, 1), (1, 0, 1, 0), (1, 0, 0, 1), (0, 1, 1, 0).
118. Find the orthogonal projection of R5 onto these spaces:
(a) The span of (0,1,0,0,1).
(b) The
(c) The
(d) The
(1, 1, 0, 0, 0), (1, 0, 1, 0, 0).
(1, 0, 0, 0, 1), (0, 1, 0, 0, 1), (0, 0, 1, 0, 1).
of the vectors given in (c) and the vector (0, 0, 0, 1, 1).
span of
span of
span