AM ITS APPLICATIONS S522
Mw. oe
9TUDY
Se
GUIDE
SECOND
aan
ALGEBRA
APPLICATIONS
DAvID GC. LAY
UNIVERSITY
OF
MARYLAND
An imprint of Addison Wesley Longman, Inc.
praca
neeae etts
era a pase e New York ¢ Harlow, England
aii Oni: e Sydney * Mexico City * Madrid
« Amsterdam
MATLAB
is a registered trademark of the MathWorks, Inc.
MAPLE is a registered trademark of Waterloo Maple, Inc.
Mathematica is a registered trademark of Wolfram Research, Inc.
Both TI-85 and TI-86 are registered trademarks of Texas Instruments Incorporated.
HP-48G is a registered trademark of Hewlett-Packard Company.
Reproduced by Addison Wesley Longman from camera-ready copy supplied by the author.
Copyright © 2000 Addison Wesley Longman.
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or
transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise,
without the prior written permission of the publisher. Printed in the United States of America.
ISBN: 0-201-64847-4
3456789
10VG 0201
TABLE OF CONTENTS
TECHNOLOGY
SUPPORT
NEW
REVIEW
MATERIALS
HOW
TO STUDY
1
LINEAR
LINEAR
EQUATIONS
FOR
THE
UPDATED
SECOND
EDITION
ON THE
WEB
CD
viii
AND
ALGEBRA
ix
IN LINEAR
ALGEBRA
vi
Systems of Linear Equations
A=1
Row Reduction and Echelon Forms
lot
Vector Equations
pa
The Matrix Equation Ax = b
1-14
Solution Sets of Linear Systems
1-18
Linear Independence
1-23
Introduction to Linear Transformations
rteA
The Matrix of a Linear Transformation
d=s0
EPP
Ppp
PPOONOD’U
PWNHRE
Linear Models in Business,
Science,
and Engineering
Chapter 1 Supplementary Exercises
1=38
Glossary Checklist
1-40
Cie
tom
ey
Lee
Qe
De
Sm
i|
2
MATRIX
1-34
ALGEBRA
1 Matrix Operations
oot
2 The Inverse of a Matrix
2-5
a3 Characterizations of Invertible Matrices
279
2-14
ND
NNN
4 Partitioned Matrices
Appendix:
The Principle of Induction
2-19
Matrix Factorizations
2-20
Appendix:
Permuted LU Factorizations
27-23
Iterative Solutions of Linear Systems
2-27
The Leontief Input-Output Model
2aa8
Applications to Computer Graphics
(Tee)
Subspaces of R"
2535
ND
NNN
oanonn
Chapter 2 Supplementary Exercises
2-39
Glossary Checklist
2-41
nN 16)
iii
3
DETERMINANTS
3.1
3.2
3.3
4
Introduction to Determinants
Sire!
Properties of Determinants
3m
Cramer’s Rule, Volume,
and Linear Transformations
3-8
Appendix:
A Geometric Proof of a Determinant Property
Chapter 3 Supplementary Exercises
Soa
Glossary Checklist
3-44
VECTOR
SPACES
Vector Spaces and Subspaces
4-1
Null Spaces, Column Spaces, and Linear Transformations
Linearly Independent Sets; Bases
4-8
Coordinate Systems
4-12
The Dimension of a Vector Space
4-17
Rank
4-20
Change of Basis
4-25
ONQOUPWNRE
PP
PPP
Applications to Difference Equations
4-28
Appendix:
The Casorati Test
4-31
F~) ie) Applications
to Markov Chains
4-32
Chapter 4 Supplementary Exercises
4=35
Glossary Checklist
4-37
S
BSjo 2
EIGENVALUES
AND
4-4
EIGENVECTORS
5.1
5.2
Eigenvectors and Eigenvalues
ord
The Characteristic Equation
5-5
Appendix:
Factoring a Polynomial
5-3
3 Diagonalization
5-9
-4 Eigenvectors and Linear Transformations
5-15
Appendix:
The Characteristic Polynomial of a 2x2
5.5 Complex Eigenvalues
5-419
5.6 Discrete Dynamical Systems
ore
5.7 Applications to Differential Equations
a-26
5.8 Iterative Estimates for Eigenvalues
5Saz29
Chapter 5 Supplementary Exercises
> a's
Glossary Checklist
5-35
Matrix
S718
6
ORTHOGONALITY
AND
LEAST-SQUARES
Inner Product, Length, and Orthogonality
6-1
Orthogonal Sets
6-2
Orthogonal Projections
6=5
The Gram-Schmidt Process
Siate|
Least-Squares Problems
6-14
0
000
QOlhtwndr
Applications to Linear Models
6-14
Appendix:
The Geometry of a Linear Model
6-17
Inner Product Spaces
6-18
O)onApplications
of Inner Product Spaces
Gel
Appendix:
The Linearity of an Orthogonal Projection
Chapter 6 Supplementary Exercises
6-25
Glossary Checklist
6-27
7
SYMMETRIC
7.1
7.2
7.3
7.4
7.5
MATRICES
AND
QUADRATIC
FORMS
Diagonalization of Symmetric Matrices
tae
Quadratic Forms
W
bats)
Constrained Optimization
eats:
The Singular Value Decomposition
t=O
Applications to Image Processing and Statistics
Chapter 7 Supplementary Exercises
Cus)
Glossary Checklist
Teed
7-16
APPENDICES
GETTING STARTED WITH MATLAB
Index of MATLAB Commands
ML-1
ML-3
NOTES FOR
Index
THE MAPLE COMPUTER
of Maple Commands
ALGEBRA
MP-18
NOTES FOR
Index
THE MATHEMATICA COMPUTER
of Mathematica Commands
NOTES FOR
Index
THE TI-85/86 CALCULATORS ' TI-1
of TI-85/86 Commands and Programs
NOTES FOR THE HP-48G CALCULATOR
Index of HP-48G Commands and
SYSTEM
ALGEBRA
MM-21
HP-1
Programs
MP-1
SYSTEM
P18
Heaps
MM-1
6-24
fe
< i
infred.
ona
Pr
rey
wo
3.2
(racer
‘
ge
Sagetok al Detepop0 ani2
re
ic. >
tute,
Aime: “a° yy".
Cj
é apter
148
cideaite
‘Votes (et,
and
Met” rie
Pryor
wOO
Lingar
of
Irs
ae
a
ied
ise
kI-3
;
Tiv@
7 TOR i.
- Ati
oy
‘s-té
iat
Che
:
ah
Dxery
ta
Suve megniarcy
>
Glecaaty
4
tor
att
faba vost a
Bis
>
ey
ee Varye(ge. gees 181 58m,“a hanged
steps
140Pi
'nei “toya
4.2 Mui’
4.3
Syecom, Gobir
Lineesr?s
‘6.ataendte
.
ite
i:
Magee. “rl
Underendss!
Saris
a
Tre
baat
s
Rete:
>
&
\
(Ape)
etiam
(pean ia
Lives
Ths
sa
2
*shiov Sitio aKa 2
—~t'
4.7 Change of tosis . 42)
4.6
Joeere
gaatqel- oind
pees
arinfe)
.
Prnawe® eens tl
vat
ty
ou
osalorexs
pects
4.3 Apes tevdlmes +o «
SMT aot
i¥teq
i Guepter™ grtthe ray?’ ery sa 8hteaetaClieary
*
oA,
bt
a
.
ee
,
>a
ae
o Siege tals
i
;liars.
aT hy
|
y
slzy
io
S-
Le arg
rom aS¢—
f-JM
ar
“ay
eee
|
a
=
-
:
ie
6.4. Tig
Psigs
figetra ting “<4
Riygareetnw
3},
:
AG
t
Pi
2
»
Ane
ee
tS-¥
/
|
Elan
A
ho aA gTAK ari 413 ue
Bhasamod.: SRITAM. ret
fretefomms tongs
Bele e
is
tere”
ames kiran
;
ee
RAMEE
“idle dabdenaddiomel SoRvcnrean rr a
Re a
<
ee
| Ntinaatie” wot temas to:
Pat
:-Tt
ras
eo
:
: Sa
-
BROTAOAS Perey
Bit: eicuaigD: BONER-1T to xsbal
———
ste
at
|
TECHNOLOGY SUPPORT FOR THE UPDATED SECOND EDITION
This
Updated
Study
Guide
contains
valuable
support
for
three
computer
programs and three graphing calculators.
Everything you need to know about
using this technology with your text
is here.
Also,
data for over 800
numerical
exercises
from
the text
are stored
in files
for
the computer
programs
and the calculators.
You’1l
save hours of time and avoid errors
in typing.
The files also contain special
programs
that reinforce
basic
concepts in the course.
You will find them on the CD in the back of your
text and at the Web site:
http:
The files on the
files available.
Web
will
//www. laylinalgebra.
com
always
reflect
the
latest
versions
of
the
data
MATLAB
Special
MATLAB boxes
in many Study Guide sections discuss MATLAB commands
as they are needed for the exercises.
An appendix to this Study Guide contains
a brief
introduction,
Getting Started
With Matlab,
and an index of
MATLAB commands.
I encourage you to try MATLAB—it
is easy to learn.
MAPLE,
MATHEMATICA,
AND
THE
TI-85/86
AND
HP-48G
GRAPHING
CALCULATORS
Notes for the Maple and Mathematica computer algebra systems and for the
TI-85/86
and HP-48G
graphing
calculators
are
included
in the
last
four
appendices to this Study Guide.
The notes correspond to the MATLAB boxes
and translate MATLAB commands
into syntax appropriate for the other technologies.
Each set of notes has its own index.
I am indebted to the following faculty members who wrote the notes and
developed the appropriate special programs.
They have been using technology effectively for several
years
in their
linear algebra courses,
and I
greatly appreciate their contributions to this Study Guide.
Maple
Mathematica
TI-85/86
HP-48G
Douglas Meade, University of South Carolina,
Columbia,
South Carolina
Lyle Cochrane,
Whitworth College,
Spokane, Washington
Luz Maria DeAlba,
Drake University,
Des Moines,
Iowa
Thomas W. Polaski, Winthrop University,
Rock
Hill,
South
Carolina
vii
NEW REVIEW MATERIALS ON THE WEB AND CD
I have provided some new
In addtion to all the help in this Study Guide,
really appreciate —review
material on the Web and the CD that my students
using
because
below,
the advice
heed
Please
exams.
and practice
sheets
I
these study aids in the wrong way can lead to a disaster at exam time.
suggest four steps to prepare for each exam.
1.
Assemble a review sheet.
The Web/CD review sheets reflect what I emphasize in my course.
If possible, you need to find out what your instructor expects of you.
The three courses on the Web/CD vary somewhat
in
their content and approach,
and they organize the material differently.
To construct a review sheet for one of your exams,
you may need to combine parts of two sheets from the Web/CD.
2.
Try
to
complete
an
to
do,
but
worth
it’s
initial
the
review
for
effort.
Don’t
an
exam
a day
read
the
or
sample
two
early.
exams!
Hard
Studying
from an old exam is a big mistake.
Instead,
study your lecture notes,
looking for items that were emphasized in class.
Read over your homework and old quizzes.
I insist
that my students
learn key definitions,
practicallly word for word.
If they cannot write a definition properly,
they don’t know what they are
talking
about.
3.
After
the
review,
take
the
sample
exam
whose
subject
material
most
closely fits what your exam will cover.
Find a quiet place and time
when you can work the entire exam without stopping,
and without
looking
at the text
or your notes.
(Of course
skip any questions
that
are
inappropriate for your course.)
Write your answers.
Time yourself.
4.
Finally,
after
Identify
the
you
have
completed
areas
that
need
example in the text
the solution to the
related
example
Peek,
Don’t
if
necessary.
the
further
test,
look
review.
If
the
solutions.
possible,
find
an
to an area in which you are weak.
Cover up
and try to write out the solution yourself.
be
reluctant
to
ask
The section on the next two pages shows how to
minimize the time you need for an exam review.
viii
at
for
lay
help.
a
foundation
that
will
HOW TO STUDY LINEAR ALGEBRA
A first course in linear algebra is dramatically different from most mathematics courses
that precede
it.
The focus shifts from learning computational procedures to digesting and mastering basic concepts that underlie
the computations.
To survive,
you may need to learn a new way to study
mathematics.
That’s
why I wrote
this
Study Guide—to
show you how to
succeed in the course and to give you tools to do this.
Because you are likely to use linear algebra later in your career,
you
need to learn the material
at a level that will carry you far beyond the
final exam.
I believe that the strategies below are crucial to success.
STRATEGIES
1.
FOR SUCCESS
IN LINEAR
ALGEBRA
Read each section thoroughly before you begin the exercises.
Most students are not used to doing this in courses that precede linear algebra.
They could survive by looking at the examples only when they were unable
to work an exercise.
That simply will not work in linear algebra.
Te
you "copy" an example (with necessary modifications),
you may think you
are “understanding" the problem, but very little true learning will take
place.
(You’1l
find
that
out
on
your
first
exam.)
For
example,
in
addition to knowing how to carry out a certain procedure,
you must learn
when that procedure is appropriate and (most importantly) why it works.
After reading the text,
use the "Key Ideas"
and "Study Notes"
in
this Guide to help you work through the text again.
Then start on the
exercises!
In the long run, this strategy will improve your performance
and save you time.
2.
Prepare for each class as you would for a language class.
Mastery of
the subject requires that you learn a rich vocabulary.
Your goal now is
to become so familiar with concepts that you can use them easily
(and
correctly)
in conversation and in writing.
In homework,
try to write
complete sentences,
such as you’ll find in the Study Guide solutions.
Pay attention,
too, to the warnings here about misuse of terminology.
This course resembles a language class because of the preparation
needed between class meetings to avoid falling behind.
Most sections in
the text build on preceding sections,
and once you are behind,
catching
up ‘with’ the class is often difficult.”
The” fact” that= concepts may seem
"simple" does not mean you can afford to postpone your study until the
weekend.
The homework may be harder than you expect.
The most valuable
advice I can give you is to keep up with the course.
ix
3.
Concentrate
practicing
concepts.
more
on
learning
definitions,
facts,
and
concepts,
on
than
Seek connections between
routine computations or algorithms.
describe such connections.
Many theorems and boxed "facts"
see Theorem 2 in Section 1.2, and Theorems 3 and 4 in SecFor instance,
to imagine typical
terms,
Your goal is to think in general
tion 1.4.
on the
to focus
and
any arithmetic,
performing
without
computations
principles behind the computations.
4.
Review and reflection are key ingredients for sucReview frequently.
cess
in learning the material.
At strategic
points
in this Guide,
I
have inserted special subsections labeled "Mastering Linear Algebra Concepts," to provide specific help for your review of each main concept.
I
urge you to prepare the sheets as you reach each review box,
thinking
carefully about the material as you write.
Later, you may choose to add
further
notes and,
of course,
use
the sheets
to review for exams.
A
Glossary
Checklist
at
the
end
of
each
chapter
may
help
you
learn
important definitions.
CAUTION
Because you can find complete solutions
here to many exercises, you will be tempted to read the explanations before you really try
to write out the solutions yourself.
Don’t do it!
If you merely think
a bit about a problem and then check to see if your idea is basically
correct, you are likely to overestimate your understanding.
Some of my
students have done this and miserably failed the first exam.
By then
the damage was done, and they had great difficulty catching up with the
class.
Proper use of the Study Guide,
however,
will help you to succeed and enjoy the course at the same time.
A PERSONAL
NOTE
Students who have used this material have told me how much it helped them
learn linear algebra and prepare for tests.
The first
time my students
used these notes, they had already taken one exam.
Grades on the next exam
were
substantially
higher.
For some
students,
the
improvement
was dramatic.
I hope this Study Guide will encourage you to master linear algebra
and to perform at a level higher than you ever dreamed possible.
David
C.
Lay
LINEAR
EQUATIONS
IN LINEAR
ALGEBRA
As you work through this chapter and the next, your experience may resemble
several walks through a village at different seasons of the year.
The surroundings will be familiar,
but the landscape will change. You will examine
various
mathematical
concepts
from several
points
of view,
and a major
problem will be to learn all the new terminology and the many connections
between the concepts.
In Chapter 4, you will see these ideas in a more
abstract setting.
Diligent work now will make the trip through Chapter 4
just another walk through the same village.
1.1 SYSTEMS OF LINEAR EQUATIONS
The fundamental
concepts presented
in this section and
mastered,
for they will be used throughout the course.
the
next
must
be
STUDY NOTES
—- 4) ia eae
Please read How to Study Linear Algebra, on the preceding two pages, before
you continue.
The text uses boldface type to identify important terms the first time
they
appear.
3x5
cards,
You
for
need
review.
to
learn
At
the
them;
end
some
of
students
each
write
chapter
in
selected
this
Study
terms
on
Guide,
a glossary checklist may help you learn definitions.
The text defines the size of a matrix.
Don’t use the term dimension,
even though that appears in some computer programming languages, because in
linear algebra, dimension refers to another concept (in Section 4.5).
A=1
1-2
Chapter
1
~¢
Linear
Equations
in Linear
Algebra
The first few examples are so simple that they could be solved by a
But it is important to learn the systematic method
variety of techniques.
presented here, because it easily handles more complicated linear systems,
and it works in all cases.
important
in this section are based on the following
The calculations
fact:
When elementary row operations are applied to a linear system,
the new system has exactly the same solution set.
Summary
1.
of
in
the
The
steps
the
Elimination
(See the text.)
in Section 1.2.
below
summary
Method
(for
This
be
will
modified
slightly
Section)
The first
equation must contain an xj.
Interchange
equations,
if necessary.
This will create
a nonzero
entry in
the first row, first column, of the augmented matrix.
Eliminate x, terms in the other equations.
That is, use replacement operations to create zeros in the first column of
the matrix below the first row.
Obtain
an
xp
term
in
second equation with
the first equation.)
desired,
to create a
of the matrix.
the
second
Eliminate
xp terms
in equations
using replacement operations.
Continue
equation,
equation.
(Interchange
below
the
second
with xg in the third equation,
x,
etc.,
eliminating
these
variables
tions
below.
least
for
This
systems
the
one below,
if needed,
but don’t
touch
You may scale the second equation,
if
1 in the second column and second row
will
in this
produce
a
equation,
in the fourth
in the
equa-
"triangular"
system
(at
section).
Check if the system in triangular form is consistent.
If it
is, a solution
is found by starting with the last nonzero
equation and working back up to the first equation.
Each
variable on the "diagonal" is used to eliminate the terms in
that variable above it.
The solution to the system becomes
apparent when
nal" form.
the
system
Check any solutions
original system.
you
is finally
find
by
transformed
substituting
into
them
"diagointo
The solution set of a system of linear equations either
contains one solution,
or contains
infinitely many solutions.
is
the
empty,
or
When asked
1.1
to “solve"
solution.
a
system,
you
may
write
°¢
Systems
of Linear
"inconsistent"
if
Equations
the
system
p
ae |
has
no
As you will see later, determining the number of solutions in the solution set is sometimes more important than actually computing the solution
or solutions.
For that reason,
pay close attention to the subsection on
existence and uniqueness questions.
Key Exercises:
21—26.
SOLUTIONS
TO EXERCISES
eee
Get into the habit now of working the Practice Problems before you start
the exercises.
Probably,
you should
attempt
all
the Practice
Problems
before checking the solutions at the end of the exercise set, because once
you start reading the first solution,
other solutions and spoil your chance
1
:
X,
+
7Xo
=
-2X4
-
9xo
=2
Replace
Scale
1
row
2
1 by row
solution
is
6)
=6
2
Sipe
g
7
6)
6)
1
(3h
£308)
0
1\°
Interchange
the
augmented
rows
1 and
i
X,
1:
:
2:
+
7Xo
=
A
1
4
fe
xX, + 7Xg = 4
pases
1
EF
tT
1
x4
1
is
Ont=10
1
5
RE
=
-10
the
4
Sxstouo
k
bere
(-7)-row
1 +
to
interchange
possibility is
with
-
2:row
by 1/5):
902)
"> -Ol2))
3
iP
0
Work
2 plus
(-10,2).
1
6)
0
either
Another
13.
4
2 by row
(10)
=20-10)
a
te
Pe AS tS)
:
2 (multiply
row
Replace
The
row
4
you might tend to read on through
to benefit from those problems.
5
10
24
5
Check:
eae 10ers a =o
=ac0r- elo e= 2
To
:
:
simplify
hand
:
computation,
the
next
step
rows
3 and 4 or to multiply
row
3 by
to replace row 4 by row 4 + (-1/2)-row 3.
6)
1
ou
matrix:
2:
6)
i)
1
4
if
5
So 4
SIT PA |e
eat |
=4
5
is
1/72.
1-4
Chapter
Add
1
«+
(-2)-row
Add
row
The
equation
Linear
Equations
1 to row 3:
2 to
row
3:
Tepe
Onin
Ade
oie
ae
OF.=
Fano
s
1
4
3
=2
0
f
S
nore
SP
Ox, = -1 or
in Linear
(Ox,
Algebra
X,
+
Axo
6 An ope |
+ Ox
=
—2
Xo + 5x3 =
+
3x3
-4
Ox3 = -1
+ Ox3 = -1)
has
no
solution set of this triangular system is empty.
system has the same solution set, the system has no
solution,
Since the
solution.
so
the
original
Study Tip:
When writing a coefficient
matrix or augmented
matrix for a
system of linear equations,
be sure that the variables appear in all equations in the same order.
Arrange the variables
in columns,
as in the text;
place zeros in the matrix whenever a variable is missing from an equation.
19.
Work
with
the
Interchange
Add
Add
Add
The
2:row
rows
1 to
(-3/2)-row
(-3/72)-
1: row
system
augmented
1 and
row
is
2:
4:
2 to row 4 :
hae o row
6)
4
0
2
8)
0
2
0
1
-2
3
2
ey
matrix:
4:
inconsistent,
O
6)
S=2o07-3
eee |
1
1
6)
0
2
6-20
0
2
1
-2
3
2
1
6)
0
6)
z
0
(6 ieee oe |
2
0)
6)
1
a", an
6)
3
2
(=8.)=1
1
6)
6)
2
Oe
2
=2
6)
8)
O*=1
1
0
6)
2
0
2
-2
6)
aetna
eee
8)
0
0
0
-5
Ox, =
-5
has
OUR
Oty
Oto
O:.
because
toe
5
= 2a 3
6)
6)
sy 24
1
3)
*-=3
(6)
see
oto + ik
=-3
8)
no
solution.
1.1
¢
Systems
of Linear
Equations
1-5
Study Tip:
Pay attention to how a problem is worded.
If you are instructed
only to determine existence or uniqueness of a solution, as in Exercise 19,
stop row operations when you reach a "triangular" form.
25.
This is an important problem.
Try to work it
before you read further
in this Study Guide.
treating g, h, and k as unspecified numbers:
1, sApeey.
Onis 5S
-2
§ -9
Olly
1P <02
74
lp
“CnPeSool
4. 8k
k
0 -3
5 k+2g
out completely yourself
Row reduce the matrix,
(ir tt4 OR 27
~|o
3 -5
Ome Oe 0 ne
The original system has a solution if and only if k + 2g + h=0.
is,\2g +h + k.=-0.
To see
why
this
conclusion
is true,
think
of k+2g+h
as
number b.
Then the third equation has the form 0x, + Oxo + Oxg
which has no solution when b # O. But if b is zero, the system
X, -
has
eae
31.10...
On
4X0 + 7X3
= g
3X2
5x3
=
0
=0
a solution
-
no
matter
some
= b,
h
what
veal, a
1herman Sric=2yWadi
1o-4...2],..10....1.-4..
2)...
Garo ond
OS O
ero)
the
values
of
g and
Only.
the
third
row
h.
changes.
of the left matrix by its sum with -2 times row 2.
To
cess, replace row 3 of the right matrix by its sum with
33.
That
Replace
row
reverse the pro+2 times row 2.
My own students
have recommended
that
I never
give you the complete
answers to the true/false questions.
They felt that the temptation to
read the answers
is too great.
After working both with and without
answers,
they realized how much doing the true/false work on their own
helped them. So all you will see here are the places where you can find
the answers.
a.
See
the
remarks
b.
See
page
5.
c.
See
the
middle
d.
See
the
box
following
of
before
page
the
3.
Example
2.
box
Elementary
Row
3
Operations.
A Mathematical
Note:
in Linear
Equations
Linear
+
1
Chapter
1-6
"If
...,
then
Algebra
...."
as
written
are
text
the
in
theorems
and
facts
important
Many
Q
P and
where
Q",
then
"If P,
in the form
statements,
implication
For instance the statement in the box at
represent complete sentences.
the end of page 7 has the form
the augmented matrices
<of two linear systems
are row equivalent
If
An
implication
statement
Q
terminology,
statement
is
we
"If
P,
},
the two systems
have the same
solution set
then
then
Q"
true whenever
statement
say that "P implies Q,"
is
itself
true
P is true.
and we write
G1)
provided
that
In mathematical
P > Q.
Be careful
to distinguish
between
an
implication
statement
'"P
implies Q" and the converse or "opposite" implication,
"Q implies P".
The converse may or may not be true when the original
implication is
true.
For
instance,
the
converse
of
(1)
above
is
not
true,
there exist two linear systems with the same solution set
augmented matrices are not row equivalent.
For example:
eo
2X—
MATLAB
I
+ 2X0
Row
kt
2
Xone—
2x,
+ 2x0
3X4
Ge 3X2
=
because
but
whose
1
2
3
Operations
To obtain
the data for the exercises
in this
section,
install
the
files from Lay’s MATLAB Toolbox,
which you download from the Web/CD.
Then, while you are running MATLAB, type
cis1
(chapter 1 section 1),
and press <Enter>.
You should see a list of exercises for which data
are available.
Type the number of the appropriate exercise
(and press
<Enter>).
In this
matrix M.
(from
the
exercise set,
Row operations
the data
on M are
for each
performed
exercise
are stored
in a
by the following commands
Toolbox):
replace(M,r,m,s)
Replaces
swap(M,r,s)
scale(M,r,c)
Interchanges rows r and s of M
Multiplies row r of M by a nonzero
(Press
<Enter>
after
each
MATLAB
row
r
of
command,
M
by
row
r
displayed
+
m*row
s
scalar
c
in boldface
type. )
The name
of any matrix
in your MATLAB
workspace
can be inserted
place of M; the letters r,m,s, and c stand for numbers you choose.
in
1.2
If
you
matrix,
enter
produced
If, instead,
a new matrix
you
M1.
one
of
from
these
M,
*
Row
Reduction
commands,
is stored
say,
in the
type
M1 = swap(M,1,3),
If the next operation is
and
Echelon
swap(M,1,3),
matrix
"ans"
Forms
then
(for
the
1-7
new
"answer").
then the answer is stored in
M2 = replace(M1,2,5,1), then
the
result of changing M1 is placed in M2, and so on.
The advantage of giving a new name to each new matrix is that you
can easily go back a step if you don’t
like what you just did to a
matrix.
If, instead, you type
M = replace(M,2,5,1)
, then the result
is placed back in M and the "old" M is lost.
Of course,
the "reverse"
operation,
M = replace(M,2,-5,1)
will bring back the old M.
Note:
For the simple problems
in this section
and the next,
the
multiple m you need in the command
replace(M,r,m,s)
will usually be a
small integer or fraction that you can compute in your head.
In general, m may not be so easy to compute mentally.
The next two paragraphs
describe how to handle such a case.
The
entry
in row
If the number
the displayed
case,
use the
tions.
For
to
0,
instance,
enter
mn
=
M=
r and
column
c of a matrix
M is denoted
by M(r,c).
stored in M(r,c) is displayed with a decimal point,
then
value may be accurate to only about five digits.
In this
symbol M(r,c) instead of the displayed value in calcula-
the
if you
want
to
use
the
entry
M(s,c)
to
change
M(r,c)
commands
-M(r,c)/M(s,c)
The
replace(M, |Sars m, s)
Adds
multiple
m
times
of
row
s
to
be
row
s
to
row
r
added
to
row
r
Or,
you can use just one command:
M = replace(M,r,-M(r,c)/M(s,c),s).
Finally,
the command
format compact
will eliminate extra space between displays,
so you can see more data on the screen.
The command
format
will return the screen to the normal display.
Warning:
Using a matrix program such as MATLAB is fun and will save you
time,
but make sure you can perform row operations rapidly and accurately
with pencil
and paper.
Probably,
you should work all the exercises
in
Section 1.1 by hand and only use your matrix program to check your work.
1.2 ROW REDUCTION AND ECHELON FORMS
Our
interest
in the row reduction
algorithm
lies mostly
in the echelon
forms that are created by the algorithm.
For practical
work,
a computer
should perform the calculations.
However, you need to understand the algorithm so you can learn how to use it for various tasks.
Also, unless you
take your exams at a computer or with a matrix programmable calculator,
you
‘hand.
by
accurately
and
quickly
reduction
must be able to perform row
1-8
Chapter
STUDY
NOTES
1
-°
Systems
of Linear
Equations
“th Shahaem
not just an augmented
The row reduction algorithm applies to any matrix,
In many cases, all you need is an echelon form.
matrix for a linear system.
The reduced echelon form is mainly used when it comes from an augmented
matrix and you have to find all the solutions of a linear system.
Strategies
for
faster
and
more
in
> Avoid
subtraction
arithmetic.
Instead,
Always
enclose
each
accurate
a negative
matrix
reduction:
It
replacement.
row
a
add
row
with
multiple
brackets
or
leads
to
one
row
of
large
in
mistakes
to
another.
parentheses.
To save
time,
combine
all row replacement
operations
that
use
the
same pivot
position,
and write
just one new matrix.
Never
"clean
out" more than one column at a time.
(You can combine several scaling operations,
or combine several
interchanges,
if you are careful.
But that
seldom
will
be necessary.)
Never combine an interchange with a replacement.
In general,
don’t
combine different types of row operations.
This will be particularly
important when you evaluate determinants,
in Chapters 3 and 5.
How
to
avoid
copying
> Practice
work
errors:
neat writing, not
your work on tests
so
too small.
Develop proper habits
will be accurate and complete.
in home-
Write a matrix row by row.
Your eye may less likely to read from the
wrong row if you place the new matrix beside the old one.
Arrange
your sequence of matrices across the page, rather than down the page.
> Don’t
let
Study Tips:
procedure in
of equations
your
work
flow
from
one
side
of
a paper
Theorem 2 is a key result for future
the box following Theorem 2.
Failure
(step 4) is a common source of errors.
to
the
reverse
side.
work.
Also,
study the
to write out the system
SOLUTIONS TO EXERCISES
ee
1.
To
check
(i)
(ii)
whether
Is every
a matrix
nonzero
The
matrix
Are
the
each
leading
(d)
leading
is
in echelon
row
above
the
fails
this
test,
entries
entry?
form
ask
all-zero
so
it
the
rows
is not
in a stair-step
questions:
(if
any)?
in echelon
pattern,
with
forn.
zeros
below
1.2
¢
Row
Reduction
(b),
(c)
all
pass
tests
To check whether a matrix in echelon
lon form, ask two more questions:
form
is
actually
this
test,
so
in
column?
The
matrices
are
in echelon
(iii)
(iv)
7
Is
there
The
matrices
1
a 1 in every
1 the
only
nonzero
entry
Matrices
(a)
and
(b)
pass
all
four
tests,
echelon;
but
matrix
not
in reduced
S}-.
6
{1
OO
corresponding
The
and
basic variables are
23, in..the.matnix.
is
(c)
system
x
-3
{xo
x3
is free.
= 4
of
all
is
OSTS2°
*Oieee
The
solution
(i)
and
Forms
(ii),
1-9
so
they
in
reduced
pass
1954
Tee
911
On
equations
x, and x3,
Any other
6)
50
is:
its
so
ask:
they
are
echelon
form.
2
1
Spee
4
*1
in reduced
QO
6)
6)
scQ? .<S
(1) 4
y
ab 5
= i.
to the pivot
corresponding
variables
are
free,
so
the
columns 1
general
2G
Two
eche-
position?
leading
2
3
(a),
pivot
(c)
0
6)
Echelon
forn.
(b),
Is each
eke
(a),
and
incorrect
answers
are
Xo
x3
-3
=
=
O
A
and
be
= ~34 (with nothing said about x).
ioe —
Warning:
The remarks for Exercise 7 are important,
have difficulty with a problem in which a variable
equation.
because students often
does not appear
in the
13.
step:
Combine
two
replacement
2. vtes
= Gen
4 -8
Thus
x;
Study Tip:
you later.
ig.
E
i
2
=
3/2
3
Serio
6
aan
+ 2xo,
operations
2
le
di
OT 7 Olle
Obnge O8 beO
and
the
general
in the
first
@ -2 3/2)
Om Caer
One,One
0
;
solution
1) - 2x2 = 3/2
O-¢=-0
Ore =ra0
‘
is
Nee S/o
ae wo
fe! Prdle gers
The experience
Be sure to work on Exercises 17—22.
too.
These exercises make nice quiz questions,
a oe lo ae
alt
There
are
only
two
will
possibilities:
f
help
1-10
Chapter
*
2h
=0.'°
system.
So
8" =
(1)
1
Systems
case
ki
system
has
no
this
case
In
A2Znets~0;
(2)¥S"=
Equations
this.
In
the
of Linear
:
solution
See
Theorem
See
the
Basic
2h = 8,
when
free
an
inconsistent
and
h =
4.
matrix
corresponds
variables.
So
augmented
the
and there are no
system,
consistent
has a unique solution when h # 4.
23.
bes represents
the
to
a
system
1.
beginning
variables
of
are
the
section.
defined
after
equation
(4).
A See
as
the subsection Parametric Descriptions of Solution Sets.
Actually, this question does not take the case of an inconsistent system
into account.
A better true/false statement would be:
"If a linear
system is consistent,
then finding a parametric description of the
solution set is the same as solving the system."
e.
29.
The
row
shown
corresponds
to
the
equation
5x,
= 0.
If a linear system is consistent,
then the solution is unique if and
only if every column in the coefficient matrix is a pivot column;
otherwise,
there
are
infinitely
many
solutions.
This statement is true because the free variables correspond to nonpivot columns of the coefficient
matrix.
The columns are all pivot
columns if and only if there are no free variables.
And there are no
free variables if and only if the solution is unique, by Theorem 2.
Study Tip:
Notice from Exercise 29 that the question of uniqueness
solution of a linear system is not influenced by the numbers in the
most column of the augmented matrix.
31.
The
full
solution
A Mathematical
You
need
to
is
Note:
know
what
in
the
text.
"If
and
only
the
phrase
of the
right-
if"
"if
and
only
if"
means.
It
was
used
above in Exercise 29, and you will see it again in theorems and boxed
facts.
The phrase
"if and only if" always appears
between
two complete statements.
Look at Theorem 2, for instance:
A specific
linear
system|
is consistent
the
if and only
if
rightmost
|augmented
system
column
matrix
is not
of
of the
a pivot
the
linear
column.
“i9)
1.3
The entire
sentence
either both true or
means
that
both false.
(1) has
the general
Sentence
Po
iftiand
only
if
where
P
denotes
the
first
ment.
The
sentence
(2)
If
If
statement
statement
P is
Q is
A mathematical
shorthand
the
two
-¢.Vector
statements
in
Equations
parentheses
are
form
Q
(2)
statement
says
true,
true,
for
14-3
two
then
then
(2)
and
Q
denotes
the
second
state-
things:
is
statement
statement
"P <=>
Q is
P is
true,
too.
also true.
(3)
(4)
Q".
1.3 VECTOR EQUATIONS
Do not be deceived by the rather simple beginning of Section
1.3.
The
important material on Span{vi,...,vp} will take time to digest.
Exercises
11—14
and
21-26
are
important.
Each
exercise
involves
an
existence
question
about
whether
a certain
vector
equation
has a solution.
(You
don’t have to find the solution.)
Note how the same basic question can be
asked in several different ways.
STUDY NOTES
ier coms ol
Develop
the habit
of reading the section carefully once or twice
before
looking at the Study Guide and before starting the exercises.
(Don’t just
look at the pictures and examples!
Important comments lurk in between.)
In nearly all of the text, a scalar is just a real number.
By convention, scalars usually are written to the left of vectors,
such as 5v or cv,
rather than v5 or ve.
To identify vectors in your lecture notes and homework, you can write underlined letters for vectors.
(Some students write
arrows above the letters,
but that takes longer.)
Vectors must be the same size to be added or used in a linear combination.
For instance,
a vector in R? cannot be added to a vector in R?.
SOLUTIONS
a
TO EXERCISES
vue [ve Gere Bi ao) fl
1-12
Chapter
1
«+
Linear
Equations
in Linear
Algebra
e- ave [3] -2[2])-[3)- [4] <b ecal= Pil
usually
not
written
To write a as a linear combination of u and v, imagine walking from the
of how many
and keep track
"streets"
the grid
to a along
origin
"blocks" you travel in the u-direction and how many in the v-direction.
For instance,
starting at the origin,
you can reach a if
you travel 2
units
in
the
a=
u-direction
2-u
+
(-1):v
and
=
then
-1
units
in
the
v-direction.
Hence,
2u-v
Similarly,
b = 2u - 2v. For c, it might be easier to visualize -2 units
in the v-direction and then 3.5 units
in the u-direction,
so that ec =
=2Vetessburonic = 895uU =iacvi.
To get to d by a path that stays in the figure,
note that to get
from b to d you could travel 2 units in the u-direction and -1 unit in
the
13.
‘ap
(2u - 2v)
foe Oct
“b]= [=2°°°S
ag
Zot-5
“O° 41|
1
lo
0
~
+ 2u - v = 4u -
O
Sel
2 -5
aa
i:
By
combination
av;
just
v2
=
a multiple
+ bv2 = av,
So
Span{v;,v2}
The
vector
y =
corresponding
(-3/2)v,.
is
augmented
linear
For
instance,
v,.
+ b(-3/2)v,
=
set
of
Diy,
4 is
aes
in
Span{v,,v2}
to augmented
column
so
linear
reduce
2 -5
ar
Ba
Saad
b
is
not
a
v,
and
vo
is
through
v, and
0.
if
vector
equation
of
(a - 3b/2)v,
the
Bat, th.
rAliguekaack
The
of
Any
0
noe
Deel?
pag
inconsistent,
inspection,
3v.
determine
if b is a
fact on page 34. Row
The system for this augmented
matrix
is
linear combination of the columns of A.
actually
23.
So d = b + 2u - v =
Denote the columns of A by a,, ao, ag.
To
combination of these columns,
use the boxed
the augmented matrix:
[a;
19.
v-direction.
points
matrix
on
the
;
if
[v,
and
vo
line
only
yl! has
the
a solution.
Compute:
2
0” Qk}hfe
cannot
be
a
pivot
column,
so
this
matrix
corre-
1.3
sponds
to
a
consistent
in Span{v,,v2}.
25.
are
for
words,
three
vectors
only
all
h and
Vector
k.
Hence
Span{v,,vo}
is all
in the
{a,,a9,a3},
each
There
b.
@
a
ag
pe
[Oo
Or
in
R° is
and
b is not
To
one
deter-
ese
eee
0 -4
4
a82ia2idmdd usl0751G) ee
eel
Gi Hit
Ove EDrad2
bd
The
system
Cc.
a, =
la;
+ Oamp + Oag.
a.
See
the
remarks
b.
See
the
statement
c.
See
the
line
d.
See
the
box that
e.
Read
the
geometric
for
this
augmented
matrix
See
last
following
Constructing
the
Example
preceding
displayed
Study Tip:
I urge my
questions and then meet
discuss their answers.
MATLAB
y
1-13
of R*.
of them.
There are infinitely many vectors in W = Span{a;,,ag,a3}.
mine if b is in W, use the method of Exercise 13.
le t
set
Equations
a.
to On gate a A
0
“te3 arr2ilool
2
6
3 -4
MOAT,
alas
31.
system
In other
¢
just
discusses
line
of
3:
before
Example
the matrix
of
page
so
b is
in W.
34.
1.
Example
description
students
together
is consistent,
4.
in (5).
Span{u,v}
carefully.
to work by themselves
on
in groups of two or three,
the true/false
to compare and
a Matrix
To access the data for Section 1.3, give the command
cis3.
The data
for Exercise 25, for example,
consists of a matrix A, its columns al,
a2, a3, and the vector b.
The command
M = [ai a2 a3 b]
creates a
matrix using the vectors as its columns.
The same matrix is created by
the
command
M=
[A
bl.
Each time you want data for a new exercise
in Section
1.3,
you
need the command
cis3.
After the first exercise, you can use the uparrow (t). This will make MATLAB scroll back through your old commands.
You may be able to find "c2si1" faster than you can retype it.
Press
<Enter> to reuse the command.
Exercises
11—14 and 25—28 can be solved using the commands replace, swap, and (occasionally) scale, described on page 1-6.
Chapter
1-14
1
~+
Equations
Linear
Algebra
in Linear
1.4 THE MATRIX EQUATION Ax = b
The
boxed
ideas,
statements,
rest
for the
fundamental
extremely carefully.
KEY
IDEAS
ine
cae
of
in
theorems
and
the
you
so
text,
this
should
absolutely
are
section
section
the
read
The definition of Ax as a linear combination of the columns of A will be.
used often.
You should learn the definition in words as well as symbols.
Note:
It is not wrong to write a scalar on the right side of a vector and
write Ax as a,x; + °°: + a,xX,, but the text follows the usual practice of
writing a scalar on the left side of a vector.
You need to understand why Theorem 4 is true.
That may take some time
and effort.
Example
3 should
help,
along with
the proof.
Theorem
4(c)
can
be
restated as "The reduced echelon form of A has no row of zeros."
The phrase logically equivalent is explained in the statement of Theorem 4.
This phrase is used with several statements in the same way that if
and only if, or the symbol
¢,
is used between
two statements.
(See the
Mathematical Note on page 1-11 in this Study Guide.) Saying that statements
(a),
(b),
that
(a) <=>
and
(c)
are
(b) and
logically
(a) <=>
equivalent
means
the
same
thing
as
saying
(c).
Key exercises are 17-28,
37, 38.
Think about 37 and 38, even
are not assigned,
because they introduce ideas you will need soon.
check the solution of 37 until you have written your own answer.)
if they
(Don’t
Checkpoint
1: True or
position in every row,
a
False?
If an augmented matrix
[A
b]
then the equation Ax = b is consistent.
has
pivot
Note:
You should work a checkpoint
problem when you see it, provided
that
you have already read the text
at least
once.
Always
write
your
answer
before comparing
it with the one
I have written.
The checkpoint
answer will be at the end of the solutions to the exercises.
SOLUTIONS
TO EXERCISES
eee
1.
The
the
7.
The solution is in the text.
The goal of Exercises 7-16 is to help you
learn Theorem 3.
If a problem involves vectors,
say vj, Vo, V3, you can
place the vectors into a matrix [v,
v2
v3], if that is helpful.
Ifa
problem involves a matrix A, you can give names to the columns of 4,
Say @,82,a3,
and reformulate a matrix equation as a vector equation.
text has the solution.
definition of Ax.
Exercises
1-6
are
designed
to
help
you
learn
1.4
If a problem
as either a
useful.
¢
leads to a system of
vector equation or a
The
Matrix
linear
matrix
Equation
equations,
equation,
Ax = b
1-15
you may regard it
is most
whichever
Warning:
Be careful to distinguish between the matrix equation Ax = b and
the augmented matrix [a, °°: a,
b], which is used in Theorem 3 to refer to
a system
of linear
equations
having
this
augmented
matrix.
Thus,
the
answer
to
Exercise
7 is not
the
augmented
Ce |
|-4
5
0
ic
matrix
4
Gs)
3"
1
Ol
6
13.
The given equation
is in the form Ax = b.
The equivalent
vector
equation is in the text’s answer.
Note how the entries in the vector x
are used as the weights in the linear combination of the columns of A.
17.
To justify
a sentence
19.
The
the answer,
make
tnat mentions an
augmented
P foddcés
matrix
UAT
@o*
for
an appropriate calculation,
and
important fact described in this
Ax
=
b
is
BA
|: which
0wacn Botte,
shows
not consistent when 2b; + bo #0.
is consistent
is a line
through
(by,b2) satisfying bo = -2b,.
25.
Oy
(0s
AL=7
4
“40
2
=5
116
6s
Pers)
6,
16 €5)
One
-3
1],
by
The
the
2
As|:
2
that
the
=
One
row
equation
then write
section.
operation
Ax
=
b
is
set of b for which the equation
origin—the
set
of all
points
so the matrix
By Theorem 4,
has
the
a pivot
columns
in each row,
of A spanR .
Study Tip:
The answer shown here for Exercise 25 is the sort of answer
expected for Exercises 23-28.
A simple calculation
is not enough.
The
phrase "so the matrix has a pivot in each row" is needed because
it explains how Theorem 4 is used.
Ona test, you probably would not have to
know
the
theorem
number.
It might
be enough
to say
"By a_ theorem",
instead
of
Checkpoint
R
2g.
that
"By Theorem
2:
Given
4".
(Check
v,,V2,v3
as
with
in
your
Exercise
instructor.)
27,
is not
in Span{v;,v2,v3}.
(If necessary,
See
the
paragraph
following
equation
See
the
box
Example
3.
See
the
warning
ae
SeeSee
Example
before
4.
following
Theorem
4.
e.
Theorem
4.
See
find
a
reread
specific
(3).
f.
See
vector
Example 3. )
Theorem
4.
in
Chapter
1-16
31.
Equations
in Linear
Algebra
a matrix-vector product Ax, as in this exercise,
recognize it as a linear combination of the colthan not, you will find this point of view helpin
entries
the
are
the
If A is mxn with more rows than columns,
then A cannot possibly have
a pivot
in every row,
so the three statements
in Theorem
4 are
all
false.
Thus, the equation Ax = b cannot be consistent for all binR.
to try Exercise
38.
It contains
a valuable
idea.)
[M] Use a matrix program to obtain an echelon form of the matrix.
Try
deleting (or simply covering) various columns of this echelon form, one
column at a time, and ask yourself
if the columns
of the resulting
matrix span R.
If you can delete one column,
can you delete a second
column?
Why or why not?
A good answer would include a discussion of
your reasoning.
Answers
if
Linear
you need
the scalars
this exercise,
c; = -1, co = 4, and cg = 2.
namely,
In
x,
(Be sure
43.
«¢
Whenever you encounter
you should immediately
More often
umns of A.
fuly
vector
37.
1
to Checkpoints:
False.
See page 42.
If you missed this, you are not studying the text
properly.
You should read the text thoroughly before you look at the
Study Guide and before you work on the exercises.
i
Let
A=
[v,
vo
v3)
0
a)
by
=
;
g
and b =
=
0
0
Op eae
eet
Ax = b to determine values
ba
Row reduce
b3
ba
of b,,...,bq4 that
mented matrix for
equation inconsistent.
ok
Orel
cls
Ae:
iv
30;
home
wa!
6)
0
b3
0
6)
OF
Siar
bg
8
beeshe? 0a
1
SO:
6)
6)
1
0
OO.
—1
choice
of
the
aug-
make
the
Shi Bee
oF Teds “bai V1 eae io mpab tell Sito aiiaegu
ante
1 “SDs
+2 bz
ba
lene0 od 1
ri JOrt
Take
b =
sum
is
(1,1,0,0),
not
for
TD,
rde
b,
Oi.
suey
Oe
0
6)
6)
6)
example,
bg
+ by
be
es 6 what
ba +
or
any
bo +
b3+ by
other
bi,. ..,bq
Whose
zero.
———————
eee
Mastering
Please
of the
Linear
begin
Study
Algebra
by reviewing
Guide.
Concepts:
Span
"How
Study
to
Linear
Algebra",
at
the
beginning
1.4
¢
The
Matrix
Equation
Ax = b
1-17
To really understand a key concept,
you need to form an image in your
mind that consists
of the basic definition(s)
together with many related
ideas.
Your goal at this point is to collect various ideas associated with
the set Span{vi,...,vp} and the concept of a set that "spans" R".
Here are
specific things to do now to prepare a sheet
(or sheets)
for review and
reference.
> Write
the definition
> Write
42.)
the
Here
> Add
the
definition of the phrase:
span is a verb rather than
equivalent
Span{v;,...,Vp}.
Copy
your
description
(See page
of
some
Figs
geometric
:n1l0>
Identify
tiv
special
it word-for-word.)
"{v1,...,Vp}
spans R"." (See
a noun as in Span{w,...,Vp}.
what
is
meant
for
a
vector
(If you try to rephrase or
to change the meaning.)
interpretations
i2siniSecs
cases.
(Learn
b
page
to
be
in
summarize
it
in
34.)
Theorem 4 word-for-word.
own words, you are likely
Sketch
of
of Span{v;,...,vp}.
tks
of
ort Figo
(Describe
Span{v;,...,vp}.
lisimt
Span{u}
Sec:
and
(Select
some
744.)
Span{u,v}
in words.)
Summarize algorithms or typical computations (such as Example 6 and Exercises
11—14
and 21-26
in Sec. 1.3,
or Example
3 and Exercises
17—20
and
23—26
> Describe
in Sec. 1.4.)
connections
with
other
concepts.
(See
pages
41—42. )
Whenever you encounter new examples or situations that help you
the concept of a spanning set, add them to this review sheet.
MATLAB
To
gauss
solve
(5; 3; -7]
Ax
and
=
b,
creates
understand
bgauss
row
reduce
the
a
column
vector
matrix
x
M =
[A
with
entries
b].
The
command
5,3,-7.
x =
Matrix-
vector multiplication is
A*x.
To speed up row reduction of M = [A
b], the command
gauss(M,r)
will use the leading entry in row r of M as a pivot,
and use row replacements to create zeros in the pivot column below this pivot entry.
The result
is stored in the default
matrix
"ans",
unless you assign
the result to some other variable,
such as M itself.
For the backward phase of row reduction,
use
bgauss(M,r),
which
selects the leading entry in row r of M as the pivot, and creates zeros
in the column above the pivot.
Use
scale
to create leading 1’s in
the pivot positions.
The commands
gauss , swap , bgauss , and_
scale
are
in
the
Toolbox,
which
you
download
from
the
Web/CD.
Chapter
1-18
Equations
Linear
+
1
in Linear
Algebra
1.5 SOLUTION SETS OF LINEAR SYSTEMS
sets of
involve
linear algebra
The most
as lines and planes.
in
Many of the concepts and computations
y
geometricall
vectors which are visualized
examples
important
of such
sets
are
solution
the
sets
of
linear
Ax =
0 as:
systems.
KEY IDEAS
Sune
Har |
Visualize
the
> the
> a
solution
single
line
point
through
set
of
0,
when
0,
when
> a plane through 0,
(For more than two
For
# 0,
visualize
the
when
free
a homogeneous
Ax = O has
Ax
=
O has
solution
set
only the
one
Ax = O has
variables,
of
equation
free
two free
also use
Ax
=
trivial
solution,
variable,
variables.
a plane through 0. )
b as:
empty,
if b is not a linear combination of the columns of A,
one nonzero point (vector), when Ax = b has a unique solution,
APN
VA
lee a line
not through 0, when Ax = b is consistent and has one
free variable,
Vv a plane
not through 0, when Ax = b is consistent and has two
or more free variables.
The solution set of Ax = b is said to be described implicitly,
because
equation
is a condition an x must satisfy in order to be in the set,
yet the equation does not show how to find such an x.
When the solution
set of Ax = 0 is written as Span{v,,...,v,+,
the set
is said to be described explicitly; each element in the set is produced by forming a linear
the
combination
A common
vector form.
x
x
X
xX
of
vj,...,Vp.
explicit
Examples
= tv,
=p t+ tv,
= XoU + X3V,
= p+ Xou + X3V,
description
are:
of
a
set
is
an
equation
in
parametric
a line through O in the direction
a line through p in the direction
a plane through 0, u and v,
a plane through p parallel to the
whose equation is x = xou + xv.
of
of
v,
v,
plane
An equation
in parametric vector form describes a set explicitly because
the equation shows how to produce each x in the set.
Solving an equation Ax = b means to find an explicit description of the
solution set.
This description can be written in parametric vector form in
Which the parameters are the free variables from the system.
Important: The
number of free variables in Ax = b depends only on A, not on b.
Theorem 6 and the paragraph following it are important.
They describe
how the solutions of Ax = 0 and Ax = b are related.
See Figs. 5 and 6.
1.5
*
Solution
Sets
of
until
presence
Linear
Systems
1-19
SOLUTIONS
TO EXERCISES
eee
1.
7.
Row reduce the augmented
variable is evident:
matrix
125 y Dive
=-1
“4=-36
2 3e="9*
0
1-5
9
-0| ~10 71
6
0
Oo r2"29-
There
free
are
no
variables;
Always use
the reduced
solutions of a system.
on pages 22—23.
Wer
as)
0
Oe
Oe
2
O
-4
0e,
eC
2002
lO
acy
Onsel e iS
10.
Q,
8)
8)
0
6)
6)
8)
or
absence
of a free
the
trivial
solution.
O
1), <5,
298 0
O| #fi0
6ie-0
on"Or.(6)%
0
the
echelon
See the
6)
the
system
has
only
form of an augmented
matrix
to find
text’s discussion of back substitution
=o)
6)
10,
OnesOs
6)
6)
sun oe
dd 2S
2
6)
0
6)
6)
6)
07,
Oust
Xq = Oxo
Xe
=
2
== -3
Xx = 5
If you wrote something
like the system above,
then you made a common
mistake.
The matrix
in the problem
is a coefficient
matrix,
not an
augmented matrix.
You should row reduce
[A
0].
The correct system
of equations is
6)
@)
BS
of
cueU1]
6)
6)
i 6)
“eu
Some students are not sure what to do with x3.
Some ignore it, others
set it equal to zero.
In fact, x3 is free;
there is no constraint on
Xz at all. - Soox;*="5x="— 2x6, Xq = “3X6, “-Xg = —SXePuwithexo, X35, X%_— free.
The general solution is
5x4
%
x,
=12%6
5X2
0
-2X6
3)
6)
Xo
Xo
6)
0
J
6)
X3
O
X3
6)
0
i
3x6
J
10)
aol
6)
z
6)
SNOle.
FO
\esealen
-5x6
6)
6)
-5X56
6)
6)
=e
X6
O
6)
X6
6)
0
1
+
u
The
set
set
—
solution.
set is the same as Span{u,v,w}.
Originally,
was described
implicitly,
by a set of equations.
Now
is described explicitly,
in parametric vector form.
+
Vv
the
the
@eectas
im
Ww
solution
solution
identify
When solving a system,
All other variables are free.
Study Tip:
variables.
13.
Row
reduce
augmented
1
-3
-2
<2
13.)
tame
x, =
7 + 5x3,
O
So
the
solution
line)
set
is
the
determined
4 + x3,
line
by the
perhaps
circle)
¥
2)
- 5x5
=7
O
=
-1
41,
O07
O-
20"
<9
with
xg free,
4[~---~]0
Xg =
(and
@
7
4},
@)
through
homogeneous
~ @-
p=
ea
q =
ei
q.=
basic
5
1]..
1
The
and
x =
parallel
to
system
x=4
0
7
4}.
6)
the
in Exercise
=0
+ xsi
solution
ip =
Br
The
line
through
set
(a
5.
Checkpoint:
Let A be a 2x2 matrix.
Answer True or False:
If
tion set of Ax = 0 is a line through the origin in R and if b
the solution set of Ax = b is a line not through the origin.
19.
the
matrix:
-5
1-1
Algebra
in Linear
Equations
Linear
+
1
Chapter
1-20
p and
q
the solu# O, then
is
parallel
to the line through q - p and the origin.
A parametric representation
of the line through p and q has the form x = a + tb, where a is some
point on the line (such as p or q) and b is any nonzero
multiple of
q- p.
So one parametric equation of the line is x = p + t(q - p), or
fl >[a] + [3]
>
Sh |
Another
first
choice
answer
t varies
21.
a.
=k
is
1
x =
p +
is often
0 to
1,
from
the
first
paragraph
See
the
first
See
the
box
See
the
paragraph
Theorem
q).
Other
written
in the
x varies
See
eS.See
oS
s(p
given
-
from
the
point
of
the
subsection
two
sentences
of
the
before
Example
1.
6.
that
precedes
answers
form
p to
5.
are
possible.
(1 the
t)p
point
Homogeneous
subsection
Fig.
x =
+
The
As
q.
Linear
Parametric
tq.
Systems.
Vector
Form.
1.5
25.
*
Solution
Suppose p satisfies
Ax = b.
solution set of Ax = b equals
such that Av, = 0}.
S satisfies Ax = b,
a.
Let
Then Ap
the set
Sets
of
Linear
the
form
w = p + vy,
Aw =
A(p + vy)
= Ap + Avy
1-21
= b.
Theorem 6 says that the
S = {w:w = p + v, for some vp
There are two things to prove:
(b) every vector that satisfies
w have
Systems
where
Av; = 0.
By Theorem
(a) every vector
Ax = b is in S.
in
Then
S(a)
in Section
1.4
=b+0=b
b.
So
every
Now
let
vector
of
w be
solution
any
the
form
p + v, satisfies
Ax = b.
Ax =
set
vz
solution
of
of
b,
and
Av, = A(w - p) = Aw - Ap =bSo v, satisfies
w= p + Vp.
31.
A
is
a
3x2
Ax
matrix
position
in
each
So there
are
no
=
0.
with
two
column,
free
Thus
in
R ,
because
vectors
37.
a.
are
A has
possible
in
(a)
Ax
=
and Ax = 0 has
position in each
have
a solution
here
means
The
candidates
that
context
for
Ax
O
Distribution
of Output
Chemicals
Fuel
Denote
the
a
of
of
course
the
we
problem
Po
.
dy,
=
only
solution.
so the
b, by
consider
b
what
Purchased
by:
4
input.
Chemicals
el
.4
ed
Fuels
Als
ay!
2
—_——
Machinery
-2Pc + .8pr + .4pu
pivot
The entries ina
decimal fractions
8
expenses
a
variable.
b.
From:
output
form
determines
eS
annual
the
A has
basic
no nontrivial
(in
dollars)
of
Pr, and py.
From the first row of the table, the
Chemical & Metals sector is .2pc + .8pr + .4py.
prices must satisfy
income
b has
ie
total
Then
of its three rows,
for every possible
Machinery
aly
=
Since
is
Fill in the exchange table one column at a time.
column describe where a sector’s output goes.
The
in each column sum to 1.
output)
b.
3 rows.
p.
0
positions.
variable
variables,
(b) A cannot have a pivot
equation
Ax = b does
not
Theorem 4 in Section 1.4.
Note: The term possible
every
pivot
each
b=
w-
the
sectors
by pg,
total input to the
So the equilibrium
Equations
Linear
*
1
Chapter
1-22
Pr = .3pc
+
income/expense
the
table,
of the
rows
& Power sector and the Machinery sector
third
Fuels
and
the second
From
the
for
s
requirement
are, respectively,
.1pp
+
Algebra
in Linear
.4py
Pu = -5pc + .1pp + .2Pm
Move
all
variables
to
the
c.
+
.Opr
—
. 4px
=e
Dea
. 1 pr
+
.8py
side
and
combine
like
terms:
0
6)
6)
-8pc - .8pr — .4Py
-.3pc
left
[M] You can obtain the reduced echelon form with a matrix
Actually,
hand calculations are not too messy.
To simplify
culations,
first scale each row of the augmented matrix by
continue as usual.
8
-3
hast
-8
9
whe!
-4
-4
8
T=
0)
1
6)
0
0
OO]
0
LS)
-.917
18)
Peal.
{-3
9
-—5
-1
~
8)
OO}
0
~
1
|O
0
=.5
-4
8
(Ogg
1
6)
0
O|
0
T
10
0”
~
coaleevee
ayp
—. 917
8)
=f
=.5
Si=S. 5
=6
ovo
O
0)
0
program.
the cal10, then
6)
Ol
6)
~
The number of decimal
places displayed is
somewhat arbitrary.
The general solution is pc = 1.417py, Dr = .917py, with py free.
If
Pu is assigned the value 100, then pc.= 141.7 and pr = 91.7.
Note
that
only the ratios
of the prices
are
determined.
This
makes
sense, for if the prices were converted from, say, dollars to yen or
Deutschmarks,
the inputs and outputs of each sector would still balance.
The economic
equilibrium
is not affected
bya proportional
change in prices.
Answer
to
paragraph
and
b =
Checkpoint:
in the
al
False.
text
Then
preceding
the
if h # 5, the solution
furnish such an example
general
The
solution
Theorem
6.)
solution
of
set
could
Suppose
Ax
=
0
be
A =
empty.
i
(See
ch v=
the
Rae
is
the
line
x =
tv,
but
set of Ax = b is empty.
(You
at this point in the course.)
are
not
expected
to
1.6
MATLAB
solving
Then
The command
an equation
Linear
Independence
zeros(m,n)
creates an mxn matrix
Ax = 0, create an augmented matrix:
M =
[A _ zeros(m,1) ]
use
gauss
,
¢
bgauss
m is the
,
number
and
scale
This section is as important as
carefully.
Full
understanding
started on the section now.
Section
of the
to
of
rows
row
of
zeros.
i=23
When
in A.
reduce
M completely.
1.6 LINEAR INDEPENDENCE
KEY
1.4 and should
concepts
will
be studied
take
time,
just as
so get
IDEAS
2
ay
|
Figures 1 and 2, along with Theorem 7, will help you understand the nature
of a linearly dependent set.
(Fig. 2 applies only when u and v are independent.)
But you must also learn the definitions of linear dependence and
linear independence,
word for word!
Many theoretical problems involving a
linearly dependent set are treated by the definition,
because it provides
an equation (the dependence equation) with which to work.
(See the proof
of Theorem 7. )
The box before Example 2 contains
a very useful
fact.
need to study the linear independence of a set of p vectors
always form an nxp matrix A with those vectors as columns
the matrix equation Ax = 0.
This is not the only method,
alert for three special situations:
Any time you
in R’, you can
and then study
however.
Stay
>» A set of two vectors.
Always check this by inspection;
don’t waste
time on row reduction of [A
0].
The set is linearly independent
if
neither of the vectors is a multiple of the other.
(For brevity,
I
sometimes say that "the vectors are not multiples.")
See Example 3.
> A set
tries
that contains too many vectors,
that is, more vectors than enin the vectors;
the columns of a short, fat matrix.
Theorem 8.
> A set
that
contains
the
zero
vector.
Theorem
Q.
The most common mistake students make when checking a set of three or
more vectors for independence is to think they only have to verify that no
vector
is a multiple of one of the other vectors.
That’s wrong!
Study
Example 5 and Figure 4.
in Linear
Equations
Linear
+
1
Chapter
1-24
Algebra
SOLUTIONS
TO EXERCISES
aay
1.
3
-3
6
independence
To test the linear
| 4].
21, w=
|O}], v=
@)
a
0
use an augmented matrix to study the solution set of
=
u
Let
{u,v,w},
Ga
x,u + Xov + xXgw = 0
3
bles,
vectors
you
tr
free
are
6
0
maya
ee
{0
O|~
4
2
Cpe
no
Warning:
omit the
ations.
your own
.=3
|0
Since
0
variables,
linearly
and
=
6
0
0 @
so
(*)
o
has
only
the
trivial
solution.
might
independent.
oF panes
Ow
2
A)
Ores?)
40
Soy
vutiOw
O
conclude
that
=3
lie
Osi
"the
6
4
+6
system
is
inconsistent"
and
might
Oraa4
ames Totes.
1
1
= 1
3
conclude
that
4
O
GQ;
0
eaoljosy
then
go on
"the
system
Study
the
that
equation
would
be
Ax = 0.
a waste
You
of
to make
Don’t laugh.
in a problem
1
Ont
ad.
1
Sy
3
sO
arty
7
~O°'s,0574,.0
has
a unique
solution"
are
linearly
independent.
However,
the four columns
actually linearly dependent.
In both cases,
the error
your matrix as an augmented matrix.
but
The
Whenever you study a homogeneous equation,
you may be tempted to
augmented column of zeros because it never changes under row operI urge you to keep the zeros,
to avoid possibly misinterpreting
calculations.
In Exercise 1, if you wrote
1
1
=A
©
Oeics
1
OF
7.
varia-—
OJ, there are three basic
4.
(@
some crazy statement about linear dependence or independence.
I have seen this happen on exams.
A more common error occurs
like Exercise 11.
In that exercise,
if you write
you
of
could
time.
row
There
of
is
and
reduce
1
<
3
Hh
eae
are
three
only
the
vectors
the matrix
are
to misinterpret
e
art
S|
0
cf
7
6)
O},
8)
rows,
so
there
are at most three pivot positions.
At least one of the four variables
must be free.
So the equation Ax = 0 has a nontrivial solution and the
columns of A are linearly dependent.
(If you know Theorem 8, you can
even omit most of this discussion. )
1.6
Checkpoint:
What
The
vectors
there
is
a
is wrong
3
eal
free
with
ale
the
a
variable",
¢
following
aI: EA
or
Independence
1-25
statement?
are
"because
Linear
linearly
there
dependent
are
more
"because
variables
than
equations".
1
13.
VALS
3
2
5
NS
-2
a.
For
5
\Aey
SS
4
v3
to
be
considered
as
1 le
AES}
a,
4
No further
impossible,
2
h
in
consistent.
b.
1
=6
Span{v,,v2},
To
find
an
augmented
1
Zila
h
work is
so the
out
if
the
this
system
is
x,;v,
+
row
reduce
true,
XoVve
ie
6)
Oe
2
4
must
vo
be
v3i,
1
al
h
in Span {v,, v2} for
For
be
to
v3
matrix.
needed.
The second equation,
Ox;
original system is inconsistent.
any value of h.
{vi,v2,v3}
=
[v,;
linearly
X3V3 = 0 should
have
only
associated augmented matrix
independent,
the trivial
[v,
vo
v3
the
+ Oxe =
Thus v3
system
solution.
OO]:
x1v,
Row
-1, is
is not
+ XoVo
reduce
oe eens te)
Te
eee ee)
Orem A@)
ge:Te) ¢
(OA
he,
ODO = a Ommea)
+
the
DeyeSbed crap OlaralsOnre> pele cOlece| CO m. Onde emeee
i
ee,
ES.
For every value of h, x2 is a free variable,
and so the homogeneous
system
has
a nontrivial
solution.
Thus
{vj,V2,v3}
is
linearly
dependent for all h.
(Note:
You can avoid all calculation
if you
happen to notice that vo = -2v,, and then mention Theorem 7.
In
general,
however,
you should not spend much time looking for special
cases
such
as
this.)
Warning: Exercise 13 and Practice Problem 3 emphasize that to check whether
a set such as {v;,Vo2,v3}
is linearly independent,
it is not wise to check
instead whether v3 is a linear combination of v, and vo.
19.
The
set
ick Be el |
is obviously
5
1
4
6
because there are more vectors (4)
orem 8,
(On a test,
you
would
probably
not
have
linearly
than
to know
dependent,
by
The-
entries in the vectors.
the theorem number.)
25.
27.
33.
1
A = |0
6)
the
box
See
the
warning
See
Fig.
3,
ae
aot
Eh See
the
remark
2
es
2
ee
before
Example
after
after
1
1
1
2.
Theorem
Theorem
following
Example
+ Ov, = 0,
The
the
printing
"If
v,; and
of
vp are
text
in R
4.
conclude
that
{vj1,v2,v3,va}
is
is to rewrite the equation vz =
as 2v, + lvo + (-1)v3
first
7.
8.
The
text
uses
Theorem
7 to
dependent.
Another argument
which
has
and
is a linear
a misprint.
vo is not
dependence
The
a scalar
first
linearly
2v, + Vo
relation.
phrase
multiple
should
of v,,".
If one of the five columns of the
then that column would correspond
=
0,
in
which
case
for the columns
pivot columns.
39.
6)
Ag
1
See
read,
37.
Algebra
in Linear
Equations
The answer in the text is correct because Ax = 0 has only the trivial
independent.
linearly
of A are
if and only if the columns
solution
if and
independent
linearly
are
they
If A has only two nonzero columns,
is a
column
neither
precisely,
(more
only if they are not multiples
multiple of the other).
Examples:
31.
Linear
«+
1
Chapter
1-26
to
the
be
7x5 matrix A were not a pivot column,
to a free variable in the equation Ax
columns of A would be linearly dependent.
So,
linearly independent,
all five columns
must be
If for all b, the equation Ax = b has at most one solution,
then take
b = 0, and conclude that the equation Ax = O has at most one solution.
Of course the trivial
solution
is a solution,
so it is the only solu-
tion.
Answer
to
Thus
the
columns
Checkpoint:
The
of
A are
set
of
linearly
four
independent.
vectors
contains
only
vectors,
no
variables of any kind, and no equations.
It makes no sense to talk about
the variables in a set of vectors.
Variables appear in an equation.
One
cannot assume that the writer of the statement has any idea of the appropriate equation.
If you want to give an explanation involving variables,
then you must specify the equation.
One correct answer is: the vectors are
linearly
a
dependent
necessarily
has
because
a free
the
equation
variable.
x1| S| +
xal| Ft xl
3| ¥
xa 1| zi
1.7
Mastering
Linear
Algebra
¢
Introduction
Concepts:
Linear
to
Linear
Transformations
1-27
Independence
In Section
1.4 of this Guide,
I described how to begin forming a mental
image of the concept
of a spanning
set.
The same
technique
works
for
linear independence.
The goal is to merge all the ideas you find regarding
linear independence into a single mental
image, with each part immediately
available
in your mind for use as needed.
Start now to organize on paper
your understanding of linear
independence/dependence,
using the following
list as a guide.
In each case,
write information that you think will be
helpful.
(Definitions
and
theorems
> definitions of linear independence
> equivalent descriptions
> geometric
>
special
interpretations,
cases,
> examples and "counterexamples",
> algorithms or typical computations,
>» connections with other concepts.
should
be copied
word-for-word.)
and dependence
Theorem 7
Figs.
1,2,4
Theorems
8,9,
box
on
p. 61,
Examples
Figs. 1,2,3,4, Exercises 13-18, 31-36
Examples 1,2, Exercises 1-12
Box on p. 60, Examples 2,4, Exercises
3,5,6
37-39
As you work on your notes,
be careful
to use terminology correctly.
For
instance,
the term “linearly independent" may be applied to a set of vectors,
but it never is applied to a matrix or to an equation.
The columns
of a matrix may be linearly independent,
but it is meaningless to refer to
a linearly independent matrix.
Similarly,
solutions of a system of linear
equation may be linearly
independent,
but the term
"linearly
independent
equations"
has never been defined.
Finally,
a set of vectors or a matrix
cannot have a "nontrivial solution".
Only equations have solutions.
1.7 INTRODUCTION TO LINEAR TRANSFORMATIONS
Linear transformations are important for both the theory and the applications of linear algebra.
You will see both uses in a variety of settings
throughout
the text.
The graphical
descriptions
in this section will
be
graphics.
computer
on
section
later
a
in
and
1.8
Section
in
augmented
STUDY
NOTES
— oe ee
from a vector x to a vector Ax as a mapping
Viewing the correspondence
provides a dynamic interpretation of matrix-vector multiplication and a new
of computer
Using the language
the equation Ax = b.
way to understand
1-28
Chapter
1
°
Linear
Equations
in Linear
Algebra
(a recscience, we can describe a matrix in two ways—as a data structure
transformfor
prescription
(a
program
a
as
and
tangular array of numbers)
the actual linear transformation
Strictly speaking, however,
ing vectors).
is the function or mapping xt> Ax rather than just A itself.
Here is a way to visualize a matrix acting as a linear transformation.
The entries in the input vector x are assigned as weights that multiply the
corresponding columns of A, then the resulting weighted columns are added
together to produce the output vector b.
by
bo
*
*
*
*
*
*
:
b3
ar eet a
*
*
X2
ba
*
*
*
X3
oe
As
you
learn
the
1
*
>
definition
of
a
linear
transformation
T,
don’t
forget
the crucial phrases "for all u and v in the domain of T" and "for all u and
all scalars c."
The mapping T defined by T(x1,xe) = (|xel, |x,1) is not a
linear
mapping,
and yet T satisfies
the
linearity
properties
for
some
vectors in its domain and some scalars.
SOLUTIONS
TO EXERCISES
we ee
cee
Can
you
omeBMDE) 6 IE [3)
show
that
T is a dilation
transformation?
v.
If A is 7x5S,
then x must’ have exactly 5 entries
‘(that
“is, “be vin R®)
for Ax to be defined.
Since Ax is a linear combination of the columns
of A and each column has 7 entries,
Ax is in R’. If T(x) = Ax, then T
maps R* into R’, where a = 5 and b = 7.
13.
a.
oS
b.
A reflection through the origin;
two other good answers are: a
a rotation of + m radians about
the origin.
1.7
19.
By
linearity,
wise,
since
v
T(2u + 3v)
23.
25.
¢
Introduction
since
u maps
maps
onto
onto
EAE
a
3v
to
Linear
2u maps
maps
onto
A function
b.
See
the
paragraph
before
c.
See
the
paragraph
following
the
box
that
d.
See
the paragraph
following
equation
(5).
is another
word
for
T(x)
= T(p + tv) = T(p)
=
then
0,
T(x)
= T(p)
2|: =
1
Be
transformation
Example
A point x on the line through p
parametric equation
x = p + tv.
fies the parametric equation
If T(v)
onto
is the sum of the images of 2u and 3v,
a.
original
equation
Transformations
=
or
|a
BS:
namely,
1-29
Like=
Finally,
ee
mapping.
1.
contains
equation
(4).
in the direction of v satisfies
the
By linearity,
the image T(x) satis-
+ tT(v)
for
all
line is just a single point.
of a line through T(p) in the
651)
values
of
t,
and
the
image
Otherwise,
(*) is the
direction of T(v).
of
the
parametric
Study Tip:
Exercise 31 is important,
because
it will help you to connect
the concepts of linear dependence and linear transformation.
Be* sure to
try the exercise first,
before
looking in the answer section of the text.
Don’t
feel
badly
if you need to peek at the hint
there.
Only my best
students can do this problem unaided.
Once ;you, have seen the hint;
try
hard to construct the desired explanation without consulting the solution I
have written below.
Don’t
give up too soon.
Reread
the definitions
of
linear dependence and linear transformation,
if necessary.
After you have written your best attempt
at an explanation,
check
it
against the Study Guide solution.
Also, study the strategy there of how I
found the solution.
Even if your attempt is quite unsatisfactory,
the time
spent on this problem is worthwhile,
because you will learn more from the
solution here.
31.
To help you use this Study Guide properly,
I have hidden the solution
at the end of the solutions for Section 1.8.
Do not look there until
you have followed the instructions above.
(I will not "hide" a solution
again, but I wanted this one time to emphasize the importance of working seriously on a problem before checking the solution. )
Linear
+
1
Chapter
1-30
in Linear
Equations
Algebra
8
Algebra
Linear
Mastering
ee
Transformation
Linear
Concepts:
by preparing
Start to form a robust mental image of a linear transformation
a review sheet that covers the following categories:
>» definition
> equivalent descriptions
> geometric interpretations
Page 70
Equations (4)
Figs.
1 or 2
> special
cases
Matrix
> examples
and
> connections
"counterexamples"
with
other
concepts
add
Exercise
a note
does not
material
31
should
about
enrich
it to your
list
(5)
;
transformation:
Superposition,
page
Examples
Paragraph
before
Exercises
29,30,33
Existence
and
Linear
Note:
and
1,
uniqueness:
Exercise
linear
mental
image
"linear
independence".
of
in
page
dependence:
for
emphasize the next section,
turn
for Section 1.8 and read the box
2-6
Exercise
your
70
this
Guide
69
31
dependence,
If your
so
course
now to the end of the Study Guide
on Existence and Uniqueness.
1.8 THE MATRIX OF A LINEAR TRANSFORMATION
Every matrix transformation is a linear transformation.
This section shows
that every linear transformation from R" to R" is a matrix transformation.
Chapters 4 and 5 will discuss other examples of linear transformations.
KEY IDEAS
i tan Bolt
A linear transformation T:R"—>R" is completely
to the columns of the identity matrix I,. The
matrix
for
There
T is T(e;),
are
two
T(e;),...,T(e,),
in Exercises
to
ways
which
where
to
e; is the jth column
compute
is easy
the
standard
to do when
determined
jth column
by
of
what it does
the standard
of I).
matrix
T is described
A.
Either
compute
geometrically,
1-14, or fill in the entries of A by inspection,
which
as in Exercises 15-22.
is described by a formula,
is
as
easy
do when T
Fxistence and uniqueness questions about the mapping xr>Ax are determined by properties of A.
You should know how this works.
The proof of
Theorem
11 also applies
to linear
transformations
on the general
vector
spaces in Chapter 4.
Here is a shorter proof that applies only to matrix
transformations.
1.8
¢
The
Matrix
of
a Linear
Transformation
1-31
Let A be the standard matrix of T.
Then T is one-to-one if and only
if the equation Ax = b has at most one solution for each b.
This
happens if and only if every column of A is a pivot column which,
in
turn, happens if and only if Ax = O has only the trivial solution.
The
"if
discussed
and
only
if"
phrase
in a Mathematical
in Theorem
Note,
11
in Section
(and
1.2
in the
of
this
proof
above)
was
e,
eo.
Guide.
SOLUTIONS
TO EXERCISES
Se
1.
The
columns
Since
T(e,)
of
the
4
standard
matrix
aS)
=
|-1]
and T(ez)
= | 3],
2
7.
Tle,)
A of
T are
we have
the
A=
images
Ay ss
|{-1_
3}.
2
«6
1
6)
=
1
= e, + 2e9 =
al T(eg)
= eg =
6)
Gi so A=
E
of
and
Ale
Checkpoint:
Use an idea of this section'to explain why the linear transformation
T that
reflects
points
in R
through
the
origin,
T(x1,xo)
=
(-x1,-X2),
is the same as the linear transformation
R that rotates points
about the origin in R through m radians.
13.
19.
The
vector
e,
the
xp-axis
rotates
ae into
standard
matrix
reflects
into
:
into
-1/V2
1/v2
A for
a/v.
wat
ee Ls =
sin 1/4
|°
The
The matrix
A that changes
be found by inspection when
the equation
OY
?edat ih
Banbeater
must
hold
for
Wisc
all
bin
tod,
of
e,
vector
eg
rotates
Pwl-i/v2"—
|1/V2
Thus,
and
ee image
This
T.
aeA
A =
;
is
then
the
reflects
;
first
into
column
hee
of
and
the
then
.1/7ve
ie |:
(x1,X2,x3)
into
(3x2g- x3,
x,+4x2+x3)
can
vectors are written in column format. Since
Bxpi
EKG
xinsedxomthes
(x1, X2,x3),
through
you
can
see
that
A =
On2—3
E
A
"
11°
«*
Linear
When
T
is
described
by
of
Exercise
19
Chapter
Study
Tip:
can
method
the
use
Equations
1
1-32
in Linear
Algebra
formula
as
a
such
A
an
find
to
Exercises
in
that
15—22,
you
=
Ax,
T(x)
(Finding A proves that T is
provided that T is a linear transformation.
T is probably not a linear transIf you can’t find the matrix,
linear.)
you have to find either
To show that such a T is not linear,
formation.
two vectors u and v such that T(u
scalar c such that T(cu) # cT(u).
+ v)
# T(u)
+ T(v)
or
a vector
a
u and
Note: The text does not give you practice determining whether a transformation
is linear because
the time needed to develop this skill
would
have to be taken away from some other topic.
If you are expected to have
this skill, you will need some exercises.
Check with your instructor.
23.
25.
a.
See
Theorem
10.
b.
See
Example
3.
c.
See
the
Row
reduce
definition
the
of
standard
"onto"
matrix
on page
A of
81.
the
transformation
T
in
Exercise
3:
1 SAA) -245-Shw- 9 [iiqne2ss—a8
lAteiQe
-8
@)
17
-20
Since the third column of A is not a pivot column,
any equation of the
form Ax = b will have x3 as a free variable.
So T is not one-to-one.
Another way to show that T is not one-to-one is to look at A and see
that its columns
Theorem 12(b).
31.
are
obviously
T is one-to-one if and only
follows by combining Theorem
Section 1.6.
A Mathematical
Note:
linearly
dependent
(why?),
and
then
use
if A has n pivot columns.
This statement
12(b) with the statement in Exercise 38 of
One-to-one
Many students have difficulty with the concept of a one-to-one mapping.
Figure 4 should help.
The transformation T on the left appears to map
three (or even more) points to one image point.
In contrast,
the transformation T on the right maps three points to three points.
You could
say that T is three-to-three
(or six-to-six),
but the standard terminology is one-to-one.
33.
Define T:R"—>R"
standard matrix
is
the
Tle;)
jth
by T(x)
for T.
column
= Be; = bj,
of
the
= Bx for some
By definition,
I[,.
However,
jth column
of B.
mxn
A =
by
matrix
[T(e,)
B, and let
-++ T(e,)],
matrix-vector
“So A=
1b,
=~
A be the
where e;
multiplication,
b)
= 7.
1.8
35.
The
Matrix
of
a Linear
Transformation
If T:R" > R™ maps R" onto R", then its standard matrix
each row,
by Theorem 12 and by Theorem 4 in Section
have at least as many columns as rows, so m < n.
When
2,
37.
¢
T
-‘SOnm
is
one-to-one,
A must
have
a pivot
in each
1-33
A has
1.4.
a pivot in
So A must
column,
by Theorem
Sin.
[M] There
is no pivot
in the fourth column,
so the columns of the
matrix are not linearly independent and hence the linear transformation
is not one-to-one (Theorem 12).
39.
[M]
Row
pivots,
matrix
onto.
reduction
of
that
columns
but
ig. no
pivot
in
the
fifth
row,
R.
By Theorem
12,
the
linear
do
there
not
span
the
matrix
shows
1,2,3,
and
the
columns
of
the
transformation
is
not
so
Siis(Thissolution
isSieforsSeection,
Pav)
Towconstruct
write in mathematical
terms what is given.
Since
{v,,Vo2,v3}
not
zero,
all
C1V¥1
is
linearly
such
that
CoVeo
+
+
C3V3
=
dependent,
there
5 contain
&thesmproofsatrizst
exist
scalars
c1,Co,C3,
0
Ce)
Next,
think about what you must prove.
the image points are linearly dependent,
[In this problem,
to prove that
you need a dependence relation
among
T(v1),
suggests
Apply
T to both
T(vo),
Tl civ
and
sides
+
T(v3).
of
CoV2
+
(*)
C3V3)
That
and
=
fact
use
linearity
the next
of T,
step.
obtaining
T(0)
and
C1T (vi)
not
all
dependent
Since
set.
Study
1.7).
Tip:
This
Answer
to
+ CeT(v2)
the
+ c3T(v3)
weights
a
linearly
Analyze the strategy above for solving Exercise 31 (in
approach will work later in a variety of situations.
Section
This
Checkpoint:
are
= 0
completes
The
zero,
the
reflection
{T(v,),T(vo2),T(v3)}
is
proof.
T
has
the
property
that
T(e,)
=
-e
and T(e2) = -ez, while the rotation R has the property that R(e,) = -e, and
R(ez) = -eg.
Since a linear transformation is completely determined by
what it does to the columns
be the same transformation.
T and
R have
the
same
e; and ez of the identity matrix,
T and R must
(You could also explain this by observing that
standard
matrix,
namely,
[-e,
—eg]l.)
1
Chapter
1-34
+
Linear
Equations
in Linear
i
i
Existence
Mastering Linear Algebra Concepts:
Algebra
and
Uniqueness
It’s time to review and organize what you have learned about existence and
The review will help
if you have not already done so.
uniqueness concepts,
to prepare you for an exam on the chapter material.
Search through the chapter and collect all the various ways to express
in boxes
Most of them can be found
existence and uniqueness statements.
theorems)
(and
with
an
"if
and
only
Also,
statement.
if"
check
the
the equation
exercises.
make two lists—one
that concerns
For existence,
and.
Ax = b for some fixed b (but not always phrased as a matrix equation),
one that concerns the existence of solutions of Ax = b for all b.
1.9 LINEAR MODELS IN BUSINESS, SCIENCE, AND ENGINEERING
This is the first of eleven sections devoted to uses of linear algebra.
The
applications
in the text were selected
to give you an impression
of the
power of linear algebra.
You are likely to encounter some of these topics
again—in school or in your career—and
the discussions
in your text will
be valuable references.
The main point of this first section is to present several
interesting
applications
STUDY
a
in which
"linearity"
arises
naturally.
NOTES
aa ar] |
Nutrition Problem:
In some applied problems such as the nutrition problem
considered here,
the data are already organized naturally in a manner that
leads to a vector equation of the type we have discussed.
The steps to the
solution in this case may be diagrammed as follows:
Applied
Problem
Vector
Equation
Augmented
Matrix
=
Solution
Set
as
Answer to
Problem
The nutrition model
is linear because
the nutrients
supplied by
foodstuff are proportional to the amount of the foodstuff added to the
mixture,
and each nutrient
in the mixture
is the sum of the amounts
each
foodstuff.
Study
equations
(1)
and
(2)
on page
each
diet
from
87.
The nutrition problem leads naturally into linear programming,
a subject that uses linear algebra and has applications
in agriculture,
busi-
1.9
+*
Linear
Models
in Business,
Science,
and
Engineering
ness,
engineering,
and other areas.
In the 1950’s and 1960’s,
most common applications of linear algebra (measured in millions
1-35
one of the
of dollars
per year for computer time) was to linear programming problems.
Such problems are still of great importance in operations research and management
science.
The following
reference
gives an entertaining
introduction
to
linear programming.
Matrix notation is used in the appendix (pp. 127-152).
Saul I. Gass, An Illustrated Guide to Linear Programming,
New
McGraw-Hill,
1970.
Republished by Dover Publications,
1990.
Electrical
Matrix
Networks:
equation
Ri
hoff’s
voltage
law.
needed
when
The
linearity
v,
comes
=
of
from
(Kirchhoff’s
studying
another
this
the
model,
which
is
evident
linearity
of
Ohm’s
law
current
model
that
also
York:
from
and
law,
which
is
involves
branch
currents.)
the
Kirch-
linear,
is
Population Movement:
The entries
in each column of the migration matrix
must sum to one because the (decimal) fractions in a column account for the
entire population in one region.
A certain fraction of the population ina
region remains
in (or moves within)
that region,
and other fractions move
elsewhere.
Warning:
The
order
of
the
entries
match the order of the columns.
the population in the city, then
fraction
of a population
that
in
For
the
a column
of
a migration
matrix
must
instance,
if the first column concerns
first entry in each column must be the
moves
to
(or remains
in)
the
city.
SOLUTIONS TO EXERCISES
eee
aaa aaaanaeaaey
1.
a.
If x, is the number of servings
servings of 100% Natural Cereal,
nutrients
X,|per serving]
of Cheerios
That
*11 20
0
2
The
nutrients
+ x2]/per serving of}
100% Natural
=
of
quantities
|jof nutrients
required
is,
110
4
b.
of Cheerios and xg is the number
then x, and xg should satisfy
equivalent
130
oF
295
2
5
8
2! 18
‘
matrix
48
:
equation
:
is
110
4
20
2
130
eH ihre
18 bal
5
=
295
)
ag |:
8
To
solve
Linear
+
1
Chapter
1-36
this,
row
110
4
20
2
130
3
pki
5
augmented
the
reduce
Algebra
for
matrix
this
8
5
2
= gee
3
foA
20>" | {1899048
110
211901 76295
le
Pere
10
110
4
Tee
ee
kelp
7 [30a
tee
en
-16
On oa On
aeO
Pa
-145
0
eia0, to 6
Oft,
il
ee
295
OB} ie
tsI143
8
lugee-9
pote Ke
panera
0 -16
0 -145
in Linear
Equations
The
desired
nutrients
together with 1 serving
equation.
aoa
taeeS
9
3
24
9
2130q6x295
pence’
Oay.O
are
provided
by 1.5
servings
of 100% Natural Cereal.
of
Cheerios
Study Tip: Be sure to distinguish between (i) the vector equation,
(ii) the
matrix equation (which has the form Ax = b), and (iii) the augmented matrix
(which has the form [A b]) that represents a system of linear equations.
7.
Loop
1:
The
11]
Total
r,=
-5]
Voltage
drop
O]|
Current
I33 does
-5]
Voltage
drop for
Iq in loop 1
=5
Voltage
drop
Ij, is
12]
Total
-5]
OQ}
Voltage
Current
Loop
ie
2=
Ps
of three
vector
RI voltage
for
Id is
not
is
drops
for current
negative,
flow
in
loop
Ip flows
Ij
in opposite
direction
1
2
Constructing
v,
resistance
we
obtain
=a
0
ei
93
“5
"4
Note
that
loop
current
the
for
negative
of three RI voltage drops for current Io
drop for Ig in loop 2
Iq does not flow in loop 2
rg and
ap
rg similarly,
5
a)
|
14
(For each loop j,
in the direction
a
z.
eat
off-diagonal
directions
are
and
listing
the
four
11
Sees
S} -Sinel2
ral =i]
ole--5
alors
Ont
0)
-5
13le
Petpet
entries
of
all
the
this forces
opposite
to
in
R
are
all
same
direction
loop
voltages
in
es
40
0
30
-5I? % “olen
h
10
negative
on
because
the
the
figure.
the currents
in the other loops to flow
current
Ey.)
Also,
remember
to check
1.9
+¢
Linear
Models
in Business,
Science,
and
Engineering
1-37
whether
the voltages
are positive
or negative.
(See the statement
before the box on Kirchhoff’s voltage law.)
The loop current vector i satisfies Ri = v.
Row reduction of [R v]
11.85, 9.46, 8.73), to two decimal places.
produces i = (12.99,
29S
MaI03
[M] Set
M ==
a.
of
the
ee
523, 293
13.
Some
sn
S
600, 000
|
|:
bee as and Xo
population
A7T6.067|7°
ah
400, 000
vectors
are
472,737
°°
oe
527,203).
72°
445,018
554,982
|”
EA
425,161
2°
574, 839
The data here shows that the city population
is declining and the
suburban
populations
is increasing,
but the changes
in population
each year seem to grow smaller.
_ [350,000
PEMMnS nex yt fee ood |: the situation
ai
S
ae
3S ,O23
B41, 477')|
Diates
me30
is
364, 140
different.
aes
635;86o0}’
«15
Now
367, 843
er
G32;157
370, 283
es
629,717
The city population is increasing slowly and the suburban population
is decreasing.
No other conclusions are expected.
If you find the
results
somewhat
interesting,
you
might
explore
further.
(This
example will be analyzed in greater detail later in the text. )
MATLAB
The
Generating
m-file
initial
a Sequence
(in
the
Toolbox)
vectors
in
x0.
Then use the command
Xo,....
You only type
(*) key to recall
for
Set
x =
the
x
Exercises
=
x0
to
9—13
put
the
in Section
initial
M*x
repeatedly to generate
command once.
After that,
the command,
and press
the
use
1.9
data
stores
into
x.
sequence xj,
the up-arrow
<Enter>.
In Exercise
11, you need 6 decimal
places to get four significant
figures in M(1,2).
Use the command
format long
and then
M_ to see
more decimal places in M. The command
format short will return MATLAB
| to the standard four decimal place display.
(The display format does
not affect MATLAB’s accuracy in computations.)
Numbers are entered in MATLAB without commas.
The number 600,000 in
MATLAB scientific notation is 6e5.
A small number such as .00000012
is; i>2e-7,
Chapter
1-38
1
Equations
Linear
+
in Linear
Algebra
CHAPTER 1 SUPPLEMENTARY EXERCISES
are given below.
11 is also sup-
Justifications for the answers to the True/False questions
A solution to Exercise
Other justifications are possible.
plied, because the text’s answer is only a hint.
1.
a.
False.
Counterexample:
Let
sof feeb Jpe-b 7
Then
B and
and
form,
in echelon
C are
equivalent
A is row
B and
C.
b.
False.
Counterexample:
pivot columns.
Then the
c.
True.
If a linear system has more than one solution,
it is a consistent
system
and
has
a free
variable.
By the
Existence
and
Uniqueness
Theorem in Section 1.2, the system has infinitely many solutions.
d.
False.
has no
Counterexample:
solution
X,
X,
+
+
Let A be any nxn
equation Ax = 0 has
to
The
Xo
following
=
system
matrix with fewer
than n
infinitely many solutions.
has
no
free
variables
b]
the
is transformed
two
augmented
1
Gris
5
Xp
2
=
e.
True.
See the box at the bottom of page 7.
into
[C
ad] by elementary
row
operations,
matrices are row equivalent.
If [A
then
f.
True.
says
Theorem
consistent,
homogeneous
two
equations
6
in
Section
the solutions
equation are
have
the
1.5
essentially
that
when
same
number
of
=
b
h.
False.
reduced
i.
False.
Every equation Ax
some variables are free.
transformed
=
0 has
the
is
the
the
solutions.
False.
For the columns of A to span R', the equation Ax =
consistent for all b in R", not for just one vector b in R".
be
Ax
sets of the nonhomogeneous
equation and
translates
of each other;
in this case,
g.
Any matrix can
echelon form.
and
by elementary
trivial
row
solution
b must
be
operations
into
whether
not
or
Chapter
+
Supplementary
Exercises
1-39
j.
True, by Theorem 4 in Section 1.4.
If the equation Ax = b is consistent for every b in R", then A must have a pivot position in every one
of its m rows.
If A has m pivot positions,
A must have at least m
columns.
k.
False.
Counterexample.
Let A be any matrix with m pivot columns but
more than m columns altogether.
Then the equation Ax = O has m basic
variables
not
have
1.
False.
m.
False.
[v, ++:
is
pi
1
and
at
least
a unique
See
one
free
variable,
so
the
equation
Ax
=
O
does
solution.
Example
5 in Section
1.6.
If a set
{vi,...,Vva}
were
to span
R®, then
va] would have a pivot position in each of its
impossible since A has only four columns.
True.
dent,
Any set of three vectors in R® would
by Theorem 8, in Section 1.6.
co.
True,
by Theorem
p.
True.
mation
q.
True.
For the transformation
xt» Ax to map R* onto R°, the matrix A
would have
to have a pivot
in every row and hence
have
five pivot
columns.
This is impossible,
because A has only four columns.
in Section
1.6,
because
A transformation is a function (page
is a special type of transformation.
u
to
is
nonzero.
67),
and
be
matrix
A =
rows, which
n.
7
have
the
five
a
linearly
linear
depen-
transfor-
Let M be the line through the origin that is parallel to the line through
Vi, V2, and v3.
Then vo - vy, and vg - vw, are both on M.
So one of these
two vectors
is a multiple of the other,
say vo - wv, = k(v3 - vi).
This
equation produces the linear dependence relation (k - 1)v, + v2 - kvg = 0.
A second
t(v2 - vi).
to(v2 V2, and
solution:
A,parametric
equation of the line .is,
Since v3 is on the line, there is some to such that
= (1 - to)v, + tove.
So vg is a linear combination
vi)
{vj,V2,v3}
is linearly
dependent.
.x. =. vi.+
v3 = v;, +
of v, and
1-40
Chapter
1
°¢
Systems
of Linear
Equations
CHAPTER 1 GLOSSARY CHECKLIST
Check your knowledge by attempting to write definitions of the terms below.
Then compare your work with the definitions given in the text’s Glossary.
if any, might appear on a test.
Ask your instructor which definitions,
transformation:
affine
augmented
matrix:
A matrix
back-substitution
(with
ObBan
basic
variable:
T:R°"—>R" of the form T(x)
A mapping
made
matrix
up of
a....
notation):
The
...
A variable
coefficient
matrix:
consistent
linear
in a
A matrix
system:
A mapping
linear
system
that
entries
are
..
A linear
system
with
....
(or linear recurrence
whose solution is ....
dilation:
A mapping
xp
domain
(of
a
echelon
form
echelon
matrix
row
(or
equivalent
T):
echelon
row
Vectors
....
relation):
An
equation
of
echelon
set
of
matrix
that
matrix):
(1)...
in R’ whose
A
2).
An echelon
rectangular
...
(2)...
matrix
(3)
....
the
property
existence
question:
Asks, "Does ... exist?"
Also, "Does ,.. exist. forshwace
or
"Is
...?"
floating
point
Arithmetic
with
arithmetic
operation
....
variable:
Any
Gaussian
elimination:
general
solution
set
Linear
arithmetic:
One
in a linear
See
reduction
algorithm.
system):
A...
(of a linear
that
numbers
variable
row
expresses
..
that
tay
with
flop:
the
...
a matrix):
~ (fy...
systems:
of
systems
free
(linear)
The
form,
properties.
row operations:
vectors:
reduction
....
transformation
three
equal
row
xpP....
equation
form ...
elementary
of
....
whose
difference
(or
phase
svt.
codomain (of T:R" »R"): The set ...that contains
contraction:
=....
system
represented
that
that
as
....
....
..
description
of a solution
has
Chapter
homogeneous
equation:
identity
matrix
image
a vector
(of
inconsistent
leading
An equation
(denoted
entry:
...
ina
of
linear
combination:
A sum
linear
dependence
linear
equation
(in the variables
written in the form ...
relation:
linearly
dependent
linearly
independent
linear
transformation:
line
p
matrix
equation:
matrix
transformation:
migration
mxn
one
v:
A matrix
set
that
..
A mapping
xP
....
An
...
with
the
with
the
(Use
for
with
gives
solution
one-to-one
(mapping):
A mapping
T:R"—>R"
A mapping
the
...
T:R"—>R"
such
of
such
that
property
....
involving
....
which
(i)
...,
be
....
and
symbols)
A system
of
parallelogram
rule
addition:
A geometric
for
movement
between
that
....
equations
with
....
interpretation
parametric
equation
of a line:
An equation
of
the
form...
parametric
equation
of a plane:
An equation
of
the
form....
pivot:
A...
number
that
either
is used
...
or
is
A
column
that
..
A position
different
.
system:
position:
that
..
overdetermined
pivot
can
....
A nonzero
column:
that
property
equations
T:R"—»R"
....
equation
....
solution:
pivot
symbols)
....
An equation
(mapping):
(Use
....
where
more
nontrivial
onto
....
{v1,...,Vp}
or
The
matrix:
A matrix that
locations, 2.fr0me......
matrix:
vector
{v1,...,Vp}
|
A rectangular
The
x,,...,X,):
A set
of
to
....
a matrix.
A transformation
parallel
with
with
equation
A set
A collection
(ii)
matrix:
A...
(vectors):
system:
1-41
....
(vectors):
linear
through
of
Checklist
matrix
T):
system
row
Glossary
....
A square
A linear
entry
¢
form
a transformation
system:
The
the
by I or I,):
x under
linear
of
1
in a matrix
that
corresponds
..
....
of
....
1-42
Chapter
1
plane
through
u,
°¢
Systems
Linear
the origin:
and
v,
of
Equations
A set
whose
set
is....
matrix):
A
Ax:
product
transformation
T):
The
reduced
(or reduced
echelon form
matrix thateiste..:
row
echelon
reduced
echelon matrix:
A rectangular
these additional properties:
matrix
roundoff
error:
arithmetic
caused
matrices
for
there
that
been
range
equation
parametric
(of a linear
Error
in floating
row
equivalent
(matrices):
row
reduced
row
reduction
row
replacement:
An elementary
rule
for
Ax:
(matrix):
Two
A matrix
algorithm:
computing
point
has
A systematic
row
of
....
in
which
a
of
form,
echelon
transformed
method
operation
using
....
that
....
form
that
when
....
exists
....
has
....
scalar:
scalar
set
multiple
spanned
size
of
u by c:
The
Two
numbers
solution
(of a linear
system):
solution
set:
set
of
The
set...
The
Span'{vi,
c= <,;Vpt:
system
matrix
of linear
trivial
equations
underdetermined
uniqueness
The
system:
question:
transformation
(or a linear
weights:
equation:
The
system):
p):
The
operation
solution...
A system
matrix
A collection
of
....
equations
with
R® to R®:
Notation:
of a....
of
Asks,
"If a solution
An equation
involving
vector:
vector
T):
function or mapping)
T from
to each vector x inR™ a....
(by a vector
solution:
....
..
(for a linear
transformation
(or
assigns
translation
....
by {vi,...,Vp}:
(of a matrix):
standard
vector
....
..
of a system
...?"
....
of
....
A rule that
T:R°—>R®.
MATRIX
ALGEBRA
2.1 MATRIX OPERATIONS
Most of this chapter is an outgrowth of the idea in Section 1.7 that a matrix can transform data.
This dynamic role of matrices
suggests
that we
study the combined effect of several matrices on data (that is, on a vector
or a set of vectors).
Sections 2.1 to 2.5 describe this matrix algebra.
KEY IDEA
Sy” aged
Matrix multiplication corresponds to composition of linear transformations.
The definition of AB, using the columns of B, is critical for the development of both the theory and some of the applications in the text.
STUDY NOTES
‘5a (ee |
Double-subscript notation:
The subscripts tell the location of an entry in
the matrix—the first subscript identifies the row and the second subscript
the column.
(Remember:
Row is shorter than column, so row goes first. )
In the product AB, left multiplication
(that is, multiplication on the
left) by A acts on the columns of B, by definition,
while right multiplica-
tion
by B acts
on the
column j}
of AB
To compute
A is mxn,
rows
of A (see
,}/column j
_
= al Rep
page
104).
That
is,
and
[row
.
i of AB]
4
[row
i of A]B
a specific matrix product by hand, use the Row-Column Rule.
then the (i, j)-entry of AB is written with sigma notation as
If
2-1
Chapter
2-2
2
°¢
Matrix
Algebra
n
(AB); ;
=
> anny
k=1
in a
of the factors
Remember that if you change the order (position)
be
even
not
may
it
or
the new product may be different,
matrix product,
Also,
equal!
not
(A + C)B and AB + BC are probably
For instance,
defined.
the warning box on page 106.
see
SOLUTIONS
ee
TO EXERCISES
ee
Vare2Ane
=
((ie 2)|7
i/
=
O07
5
eink
cate
1
=14
3 =
4
sa aici fs -3
The
not
=]
> 6
0
2
60
Hall 2 -10
eb
Next,
U=ZAye
+. (-2A)
use B =- 2A == B +
Al
Fed
4
end
ae | 7 -13
product AC is not defined because
match the number of rows of C.
sf
il
CD = be
rule
is
4
1
alle
OR
4 =
use
a=%
ea
than
4
of
faster
to
the
has
5
Since
columns,
B must
AB has 7 columns,
For
e
et
the
number
of
columns
hand
;
computation,
of
A does
ad
row-column
the
definition.
7.
Since
A
defined.
13.
If you had difficulty with this problem, read the definition of AB from
right to left.
Here is the definition,
written in reverse order:
[Ab,
°°:
Ab,]
Thus. [Qr)-..°°2.30rp],
15.
19.
= A[bi
S.QR,.
<-->
have
5 rows.
Otherwise,
AB
so does B.
Thus B is 5x7.
bp]
= AB,
See
the
definition
of
b.
See
the
box
after
Example
c.
See
Theorem
2(b),
read
right
to
left.
d.
See
Theorem
3(b),
read
right
to
left.
e.
See
the
after
Theorem
3.
text.
The
A solution
is
in the
columns
AB
are
of
Ab;,...,
B=
[by
:::
bp]
exercise
is
that
not
when /Ro= [ra | yt,
a.
box
when
is
AB.
3.
Abp.
main
point
of
the
the
2.1
21.
Let
bp be
the
last
column
of
B.
Then
Abp
¢
=
0,
Operations
but
is
bp
the
zero
and
Suppose that Ax = 0 for some x.
Then C(Ax) = CO = 0, and so (CA)x
by associativity.
But CA = I, which implies that x must be zero.
the equation Ax = 0 has only the trivial
(zero) solution.
= 0,
So
dependence
not
2-3
relation,
vector.
Thus the equation Ab, = 0 is a linear
the columns of A are linearly dependent.
23.
Matrix
Supplementary Exercises:
The following
two problems will
help you review
concepts from Chapter 1.
Try them now.
The answers are at the end of this
section.
In each problem, assume that the product AB is defined.
a.
Show that if y is a linear combination
a linear combination of the columns of
b.
Show that if the
columns of AB.
25.
columns
of
B are
of
A.
linearly
the
columns
of
dependent,
AB,
then
then
so
are
y
is
the
The product
uv is a 1x1 matrix,
which usually
is identified
with a
real number and. is written without the matrix brackets.
The text shows
that uv and vu both equal = 2a be 3D aac,
Since u is 3x1 and v
is
1x3,
the outer. product, uv
is 3x3,
and so is wu.
Look at the\text’s
answers for vu
and uv
to see how they are related.
Study Tip:
Inner products
(u'v and viu) have the transpose
middle.
Outer
products
(uv
and vu)
have
the transpose
symbol
symbol
in
on
the
the
outside.
27.
The
The
(i, j)-entry
of A(B + C) equals
the
n
n
n
k=1
k=1
k=1
(i, j)-entry
of
(B + C)A equals
n
the
(i, j)-entry
of AB + AC,
because
(i, j)-entry
of BA + CA,
because
n
n
>, (ix + Ci) a_j = > dina + yor
k=1
k=1
k=1
31.
The
(i, j)-entry
Bibi
The
the
teh
in
entries
ith column
in (AB)" is the
(j,i)-entry
in AB,
which
is
ajnPni
row i
of B.
of
B’ are
Likewise,
bi, °**, Dny,
the entries
they
because
in column j
come_ from
are
of A
2-4
Chapter
2
«+
Matrix
jth
the
from
they come
is byjyaj, + *** + bpiajn.
because
in BA
aj1, +--+, @jn,
(i,j)-entry
Algebra
Solutions
to Supplementary
and
(AB)"
of
(i, j)-entries
other sum displayed, so the
This proves Theorem 3(d).
Thus
of A.
row
sum_equals
This
are
BA
the
the
equal.
Exercises:
a.
If d is a linear combination of the columns of AB, then the equation
If x is
(See the box on page 41 in the text.)
ABx = d@ has a solution.
that
shows
multiplication
then associativity of matrix
such a solution,
a
is
d
So
solution.
a
has
d
=
Au
That is, the equation
A(Bx) = da.
linear combination of the columns of A.
b.
If the columns
such that Bx =
ity,
which
x is not
MATLAB
of
0.
shows
B are
Hence
linearly dependent,
then there
A(Bx) = AO = O, and (AB)x = 0,
that
the
columns
of
Notation
and
Operations
AB
are
linearly
Matrix
=
[1
2 3;4
5 -6]
data
rows.
Use
brackets
around
the
data.
a 2x3 matrix A. If A is mxn,
then size(A)
nly
The (i, j)-entry”in A is~ ACI, 3.” ‘lf i or
a colon,
the
result
A(:,3)
A(2,:)
specify
symbols
MATLAB uses
multiplication,
3,
4 and
4 5])
3:5
works
is a column
is column 3 of
is row 2 of A
columns
A(:,[3
notation
because
row-by-row,
with a space
between
For instance,
the command
creates
tor [m
The
dependent,
zero.
To create
a matrix,
enter
the
entries and a semicolon between
To
is a nonzero x
by associativ-—-
for
of
A,
respectively.
you
can
use
for
the
rows
+, -, and * to
respectively.
A,
stand
of
vector
Similar
denote matrix addition,
subtraction,
and
If A is square and k is a positive inte-
ger,
A*k
apostrophe
the
(i, j)-entry of A’ is the complex conjugate of the
Use a single column (or row) matrix for a vector.
for
[3 4 5].
A.
an
denotes
Examples:
A(:,3:5)
"3 to 5")
selected
row
A
5 of
or
(read
or
is the row vecJtis replaced-vy
the
kth
power
the
prime
symbol).
of A.
The
transpose
Note:
when
of A is
A has
complex
A’
(with
entries,
(j,i)-entry of A.
If u and v are
their inner product,
column vectors of the same size,
then
u’*v
is
and u¥v’
is an outer product.
Type
help randint
to learn about the Toolbox command
a matrix with random integer entries.
that
produces
2.2
°¢
The
Inverse
of
a Matrix
2-5
2.2 THE INVERSE OF A MATRIX
Matrix inverses
section and the
are essential
next describe
for
the
many
main
discussions in linear algebra.
This
properties of invertible matrices.
STUDY NOTES
oe
a
|
The inverse formula for a 2x2 matrix will be used frequently in exercises
later in the text.
(See Theorem 4.)
To invert a 2x2 matrix,
interchange
the diagonal
entries,
reverse
the signs of the off-diagonal
entries,
and
divide each entry by the determinant
(assuming ad - bc # 0).
Theorem 5 and its proof are important.
The phrase "has a unique solution"
includes the assertion that a solution exists,
so the proof has two
parts.
The equation AA’
= I is used to prove that a solution exists,
and
the equation A A= I is used to show that the solution is unique.
Except
when A is 2x2,
Theorem 5 is practically never
used to solve
Ax =
b.
Row
reduction
of
[A _ b]
is, faster.
Actually,
in practical
work,
you will
seldom need to compute A.
. (However,
Example
3 illustrates
a
case in which the entries of A’
could be useful.)
When using an inverse in matrix algebra, remember that matrix multiplication
is not commutative.
The phrase
"left-multiply B by A"
means to
multiply
Bon
its
stand for AB
left
side
by Aiea
Never
write
2 (or
A
B/A)
because
it
could
or BA.
Elementary matrices are used in this text mainly to link row reduction
to matrix multiplication.
Each elementary row operation amounts to leftmultiplication by an elementary matrix.
So, if A can be row reduced to U,
then there is a product F of elementary matrices such that FA = U.
Theorem 7 includes an if and only if statement,
which was discussed
in
the Appendix to Section 1.2 in this Study Guide.
The proof of this state-
ment
in Theorem
7 has
two
parts:
that
A ~
(2)
assume
that
I,;
and
(1) assume
A ~
I, and
that
A is invertible
prove
that
A is
SOLUTIONS
TO EXERCISES
SSS
(eh
ep\iS)
anes
Seles
26),*
1
6 -(-5)])-
= TOG
5 (e5)es oS
ae
ip Se
aiyo4
[5s
5
=5i--4
8 -2] _if8 -2
aie
|-aLs j so ioe
and
invertible.
—1
i
prove
2-6
Chapter
2
+
Matrix
Algebra
-1
T
-1
=
Ab,
= par
= Abs
-6
Similar calculations give A - bz =
b.
ts
f
wiley
Sen
[A
bi
_f
@ So -2 3° "160s
gnte2 Hpi ned. ice, oT 4
eT
ite
OM
beg
bg
bal] =
Gubig
4
The solutions
are
“Sst
4
4
“2 P desvenel
OI
FHS
4
eee
t
=
ome
tae ter stag
Hee Bie Re
eae ele |a} Par the same
-4
2
“2
as
2
in part
(a).
Note:
This exercise
was designed to make the arithmetic
simple
for
both methods, but (a) requires more arithmetic than (b).
In fact,
(a)
requires 22 multiplications
or divisions and 9 additions
or subtrac-—
tions,
but
(b) only takes 14 multiplications
or divisions
and
tions or subtractions.
In general,
the arithmetic for method
be unpleasant
for hand calculation.
2x2, method (b) is much faster than
Study
Tip:
However,
(a).
when
A
is
9 addi(b) can
larger
than
Notice
in Exercise 7(a) how the 1/2 in the formula for A’ was
8°
“=2
kept outside
the matrix
when computing
A’ x.
This
trick often
=3
1
simplifies hand calculation by postponing the arithmetic with fractions (or
decimals)
until
the end.
9.
the
definition
11..
a.
See
ce.
SGA
E
ofsinvert ible.
ai then
that
this
matrix
d.
See
Theorem
a.
Given:
A .is nxn
X, satisfies AX; =
5.
is
not
e.
See
cd =
1:1
invertible,
the
box
See
Theorem
- 0:0
=
1 #0,
because
just
before
ad
-
6(b).
but
bc
=
Example
.and_invertible,.
and..B.
is =n <p.
B. Then,
left-multiplying each side
A*(AX,) = A*B,
If there is
To show that
ab -
b.
IX, = A’B,
and
Theorem
4 shows
0.
6.
Suppose
by Aas
that
“Xx, = AB
a_solution of AX = B, then the solution must be AB.
A B is a solution, substitute it for X and compute
ACA 'B) =\4A0B = TR = oB
2.2
b.
°¢
The
Inverse
Write B = [b,; +++ bp] and X = [Ww --:
multiplication,
AX = [Au, --- Au].
up].
So
By definition of matrix
the equation AX = B is
equivalent
Au,
to
the
= bi,
of
a Matrix
p systems:
...
,
Alp = bp
Since A is the coefficient
be solved
simultaneously,
C1)
matrix in each system,
placing
the augmented
these systems may
columns
of these
systems next to A to form [A
b,
bp] = [A
BJ].
solutions
u,...,u,
in (1) are uniquely determined,
we
[A by
“++
“b,] reduces to [fF uw,
+--+
u,], which is [I
X is
to
the
solution
of
solution:
Since
A
Thus
[A _ B] can
be row
reduced
for some
X.
Let
this row reduction.
(Ex: *°E,)A
=
E£,,...,E,x,
Then
I
Right-multiplying
that
The
second
and
each
(E,:*-E,)AA
be
is
invertible,
side
of
=
the
be
row
of the
matrices
reduced
form
that
[I
X]
implement
(2)
first
shows
can
X
(Ey-°-E,)I
in (2) then
it
to a matrix
elementary
(Ex: + *E,)B
= IA’,
equation
Since the
know that
X],/*where
AX = B.
Another
I.
2-7
equation
= A nb
brandis
that A B=
X.
by
A’,
we
find
Eye seep leki Al
Study Tip:
Whenever you are told "A is invertible",
you know that as
exists,
and you may use A
to solve an equation or to make appropriate
calculations.
13.
If AB = AC and A ve invertible, then we can left-multiply each side of
the equation by A:
A ‘AB = AW TAC,
IB = IC,
and
B= C._
To show
that the conclusion B = C does not always follow when A is singular,
find a counterexample:
If A is the zero matrix,
then AB = AC for any
nxp matrices B and C.
For another example,
let
a= She B deerG J
Then
Warning:
(AB)?
=
AB
=
AC
=
0
E
at
because
=
A common mistake
By a hes
But this
that A and B are
advance,
that A is invertible.
19.
Suppose
that
X satisfies
B and
C have
the
same
first
row.
16 is to try to use the formula
in Exercise
formula
is available only when you know,
in
invertible.
the
equation
In
Exercise’
cA
16,
+ X)B
“you
=i,
must
4 then
"prove
Chapter
2-8
2
°¢
Matrix
G WAL HEKIBSD
Algebra
(A+
xX)I
=CB
2»
At+tX=CB
2»
Left-multiplication
by C
Right-multiplication by B
«(A S9X)RB ebee
So T(A"? XIB’ = CS
>
= CI
ccol(A + X)B
Om
SAT
X=CB-A
The right side must be CB,
careful above when you multiply by B.
BC.) To show that CB - A really is a solution, substitute it for xX:
(Be
not
Gl
After
A;# (CBI-=A)IB =-C°IGBIBR \=eGoCBBy = dI sub
this
details
argument
Gi
may
permit
in your
calculations
(check
above could be shortened to:
section,
your
on
this).
(Ast) XB
_ Leremutbilpl tcatioambye Cues
sila
instructor
erapGActeX
WB
e
(A +X)
e
X=
ac= OC
= CB
you
to
include
For
fewer
instance,
the
C
Right-multiplication by B or B
CB-A
21.
Suppose A is invertible.
By Theorem 5, the equation Ax = O has only one
solution,
namely,
the zero solution.
This means that the columns of A
are linearly independent,
by a remark in Section 1.6.
23.
Suppose A is nxn and the equation Ax = 0 has only the trivial solution.
Then there are no free variables in the equation Ax = 0, and so A has n
pivot columns.
Since A is square and the n pivot positions must be in
different rows,
the pivots in an echelon form of A must be on the main
diagonal.
Hence A is row equivalent to the nxn identity matrix.
25.
Suppose that ad - bc = 0.
matrix and the equation Ax
one
Ax
of
a
=
0.
the
vectors
For
= 0.
When Ax
by Theorem 5.
31.
pe and
:
instance,
=
If a,b,c,d are all zero,
then A is
= 0 is true for all x.
Otherwise,
O has
@
ee is
nonzero,
and
both
1.0
E
All 4 =
[Diab
1
fh: a ie =
-|-abi tebat
2
more
than
one
solution,
A cannot
a
of
them
because
be
the zero
at least
satisfy
ad
-
bc
invertible,
100°
(8 tf Goeeco
1p SO Aho ee te One
ret
Leh OMI 3 1 POLO Be
Les
aieubolee
Qwenk
§nG AGie Oleal
Ose 279e9ls95000
eid
(7
O°
0
Howser
© oF 6
1 8-1
1. Of
son? Bierce
Gy
10 4
O
4.0!
ge ew
se
1G 60 On
eis
sedpe-ias sea
6etlO
eee
shancae
425
ee
ft
2.3
Study
Characterizations
arithmetic
Invertible
Exercise
40,
to
multiply
the inverse of 1000D.
D by
But (1000D)"' = 1000'D°.
So multiply the inverse
matrix
you
find
by’ 1000
to obtain
using
first.
Diet
Of
to
invert
row
course,
hand
is
operations
D by
will
2-93
In
1000
Then
Matrices
help
MATLAB
the
of
may
It
Tip:
°¢
MATLAB
not
bad.
produce
is easier.
The Identity Matrix and A’
The nxn identity matrix is denoted by eye(n).
If A is a 5x5 matrix,
then the command
M = [A eye(5)]
creates the augmented matrix [A I].
Use
gauss, swap, bgauss, and
scale
to reduce [A I]. See page 1-17.
There are other MATLAB commands
that row reduce
matrices,
invert
matrices,
and solve equations Ax = b, but they are not discussed
because they will not help you learn the concepts in this section.
here
2.3 CHARACTERIZATIONS OF INVERTIBLE MATRICES
In many linear algebra texts,
the equivalent of Chapter 4 is the "killer"
chapter.
But you won’t have difficulties if you prepare well now.
Review
the major concepts
in the previous sections,
and spend more study time on
this short section than you ordinarily spend on one section.
KEY
a
IDEAS
we
The Invertible Matrix
ever,
some groups of
rectangular matrices.
Theorem (IMT) only applies to square matrices.
Howthese statements
in the IMT are also equivalent
for
See Theorem 4 in Section 1.4, for example.
STATEMENTS
FROM
THE
INVERTIBLE
MATRIX
THEOREM
Equivalent statements,
for any nxp matrix A.
Equivalent only for an
nxn square matrix A.
Equivalent statements,
for any nxp matrix A.
k.
There is a matrix
D such that AD = I.
a.
Jj. There is a matrix
C such that CA =I.
*.
A has
c.
a pivot
in every
h.
posi-
row.
The columns
span R™.
A is an
matrix.
A has
invertible
n pivot
*.
positions.
of
A
b.
A is row equivalent
to the nxn
identity matrix.
e.
A has
a pivot
posi-
tion
in every
column.
The columns of A
form a linearly
independent set.
2-10
Chapter
for
solution
°¢
Matrix
Algebra
The
has
*.
Ax = b
one
equation
at least
The
has
g.
2
solution
b
each
*.
The linear transformation x b Ax
maps R? onto R*.
*.
1.
seven
statements
equation Ax = b
at most one
The linear transformation xb Ax
is invertible.
f.
A is a product
d.
The linear transformation x b Ax
one-to-one.
is
The
equation
of elementary
has
only
matrices.
trivial
A’ is an
denoted
b
each
for
in R’.
invertible
*.
matrix.
The
The
has
solution
b
each
for
in R’.
in R’.
i.
*.
Ax = b
equation
a unique
The
Ax = 0
the
solution.
equation
Ax = 0
has no free
variables.
by (*)
were
not
listed
in the
text
as
part
of the IMT, mainly to avoid intimidating you with so many statements in one
theorem.
As part of your review,
make sure you understand why these extra
statements can be added to the IMT.
The text did not actually prove that for a rectangular matrix,
statements (j) and (k) are equivalent to the other statements in their respective columns,
but I listed them here anyway so you could see how they fit
into
and
the
table.
a matrix
A matrix
D such
Checkpoint:
What
A has
more
rows
third
column
that
C such
=
say
about
can
you
columns?
A has
CA
I is called
than
when
that
AD =
more
the
is
called
statements
(Why?)
columns
I
than
a
left-inverse
a right-inverse
What
in the
about
rows?
of
the
of
A,
A.
first
column
statements
when
in
the
(Why?)
A question such as the one in the box below is one way I test whether
my students know the IMT.
Test yourself.
Cover up the IMT, write your
answer,
and then check your work.
The answer is given at the end of this
section.
Test
Question:
Let
A be
an
nxn
Theorem,
each
the following
(ii)
the
(v)
linear
matrix.
Write
equivalent
concepts,
equation
AD =
I,
transformation.
6 statements
from
the
Invertible
Matrix
to the
one
in
statement
that A
each
statement:
is invertible.
Use
(i)
row
equivalent,
(iii)
columns,
equation
(iv)
the
Ax = 0,
and
2.3
¢
Characterizations
of
Invertible
Matrices
2-11
SOLUTIONS
TO EXERCISES
Se
1.
To show that A is invertible,
you have to find only one statement
in
the Invertible Matrix Theorem that is true.
For instance,
the text’s
answer used the obvious fact that the columns of A are linearly indeThe text also showed how to use det A and Theorem 4 to show
pendent.
Until Chapter 3, this approach is available only
that A is invertible.
for 2x2 matrices.
1
6)
Boe)
3
QO
Pree2
¢
3
3
4x4
ts
-I
eas
2
1
0
0
3
148
0
0
= 2
3
—I
5
SSeS
Oo
-4
8
matrix
has
4 pivot
positions
0
1
6)
0
3
TO
0
One
Zi
3
1
O21
ol:
6)
6)
O
O
=4
is
invertible,
and
so
The
by the
rae
original
IMT.
Study Tip:
For theoretical
purposes,
the most useful characterization of
an invertible matrix A is that Ax = 0 has only the trivial solution.
For
most computational
work by hand,
the fastest way to determine
if A is in-
vertible
13.
is to
(find and)
count
the pivot
positions.
the
Invertible
Matrix
Theorem.
The
Study
statements
there
are
true
only for an invertible matrix.
Also,
if one of the statements
is true
about
a square matrix A, then all statements
in the theorem are true;
if one of the statements is false, then all are false.
19.
a.
See
statements
(d) and
(b) of the
IMT.
b.
See
statements
(h)
and
(e).
c.
See
statement
(g).
d.
See
statements
(d)
and
(c).
e.
See
statement
(1).
is
in
answer
section.
The
full
solution
the
text’s
Study Tip:
Learn how to recognize when a square matrix is not invertible.
See Exercises
17 and 21.
Each of the following statements was constructed
by negating one of the statements in the IMT.
If A is an nxn matrix,
then
each statement
is true if and only if A is not invertible:
> The matrix A has fewer than n pivot positions.
The equation Ax = O has a nontrivial
(nonzero) solution.
:
The equation Ax = b has more than one solution for some b inR.
The equation Ax = b has no solution (is inconsistent) for some b in R™.
The columns of A are linearly dependent.
The columns of A do not span R".
vvvvyv
23.
By Theorem
is
6(b)
invertible,
in Section
because
both
2.2
(with
A and
A in
A are
place
of
invertible.
B),
the
product
AA
2-12
25.
Chapter
2
Algebra
Study
Tip:
A(BW)
Study
Tip:
Just
remember
a good
test
then
there
is
solution
that
to
statements
Whenever you use
you should point
A is
invertible,
Exercise
27
shows
(j)
(k)
of
one
out
a
the
by the
common
IMT
use
apply
of these statements
in your argument that
I and
=
(AB)W
that
W such
matrix
a
and
the matrix B (= A ) is
question.
A is square,
Since
I.
=
The
matrices.
invertible,
25 makes
invertible,
is
AB
If
hence
the
b
A_ AB =
A is_invertible,
by A , we have
= A.
is (A)
and its inverse
Exercise
By Theorem 6(a),
B= A.
and
IB =A’,
invertible
29.
Matrix
Then
Suppose that A is square and AB = I.
= I
AB
the equation
Left-multiplying
IMT.
A‘I,
27.
«+
IMT.
of
the
only
to
IMT.
square
to deduce
that
A is square.
A
is
To show that T is one-to-one, suppose that T(u) = T(v) for some vectors
u and v in R’.
Then S(T(u)) = S(T(v)),
where S is the inverse of T.
By Equation (1), u = S(T(u)) and S(T(v)) =v, so u= vv.
Thus T is oneto-one.
To
vector
in
T(S(y))
= y,
show
T
is
and
define
x
which
shows
that
R'
that
onto,
=
suppose
Sy.
y
Then,
T maps
represents
using
R" onto
an
Equation
arbitrary
(2),
T(x)
=
R’.
Second
proof:
By Theorem
9,
the
standard
matrix
A of
T
is
invertible.
By the IMT, the columns of A are linearly independent and
span R.
Finally,
by Theorem
12 in Section
1.8,
T is one-to-one
and
maps R" onto R’.
33.
The
standard
matrix
of
T
det A # O.
By Theorem
standard matrix of T°
A
35.
38.
oo Nae
(i
=
f
9
4s
SOn7
-1
is
9,
is
=5
A=
the
A.
(x4, Xo)
pate
invertible,
because
transformation T is invertible
From the formula for a 2x2
and the
inverse,
6
=
i
Given any v in R", we may write
onto mapping.
Then, the assumed
S(T(x))
= x and
U(v)
each
That
S and
v.
is,
To perform
the
instructions can
= U(T(x))
U are
9
A
le
ST
x
4|
which
=
is
(7x,
+
QOX5,
4x,
+
5X2).
v = T(x) for
properties of
some x, because T is an
S and U show that S(v) =
= x.
and
U(v)
from
R” into
the
same
So
S(v)
function
experiment
described,
be repeated easily:
the
x = rand(4,1);
x1
b = A®x;
Use format
long.
If you repeat
times, you will probably find that
this
x and
following
= A\b;
are
equal
for
R”.
MATLAB
line
of
x - xi
instruction
x1 agree to
line
ten
11 digits
or more
twice as
2.3
often as when they
even 13 digits.
¢
Characterizations
agree
to
12 digits.
of
Invertible
Occasionally,
Matrices
they
may
2-13
agree
to
Answers to Checkpoint:
If A has more rows than columns,
then all statements
in the first
column of the table must
be false,
because
they are
equivalent and the statement about a pivot position in each row cannot be
true.
If A has more columns than rows,
then all statements
in the third
column of the table must be false, because A cannot have a pivot in each of
its columns.
Answers
to
(i) A is
that AD=
only
onto
Test
row
I.
Question:
equivalent to Ip.
(ii) There exists an nxn matrix D such
(iii) The columns of A span R".
(iv) The equation Ax = 0 has
the trivial
solution.
R.
Another answer for (iii)
Similarly,
ceptable
as
A is
The
linear
is:
The
columns
of
But
following
(v)
has
another
one
of
the
answers
to
invertible
if and
only
This statement
statement that
Mastering
(v)
answer.
the
the
test
if the
transformation
A are
Algebra:
Reviewing
and
linearly
Ax
maps
R"
independent.
statement
is unac-
question:
columns
of
is itself a (true)
theorem
(assuming
is true precisely when A is invertible.
Linear
xh
A span
A
is
R".
square),
not
a
Reflecting
Two
important
steps
to mastery of linear algebra are periodic
review of
earlier material and reflection on its relation to new material.
When you
reread the basic conceptual
material
from Chapter
1, you may be surprised
to discover new insights that you missed earlier.
Your broader experience
now should give you a better framework within which to understand concepts
such as spanning and linear independence.
Compare the review you conducted
in Section 1.8 (see the Study Guide
appendix to that section)
with the three-part
table at the beginning of
this Study Guide section.
(You did carry out that review,
didn’t you?)
The left and right columns of the table should match some of your "existence" and "uniqueness" statements, respectively.
If your review
in Section
1.8 was thorough,
you probably anticipated
some of the content of the Invertible Matrix Theorem. Existence and uniqueness threads run through the fabric of linear algebra,
and they intertwine
when related to square matrices (the middle column of the table).
A good
review procedure now is to expand the table to include references to theorems,
examples,
and counterexamples.
This will occupy several pages.
The
process of constructing this table is what will help you most.
2-14
Chapter
2
inv
and
MATLAB:
°¢
Matrix
Algebra
rank
Determining whether a matrix is invertible is not always a simple matinv(A) ,
A fast and fairly reliable method is to use the command
ter.
A warning is given if the matrix is
which computes the inverse of A.
singular (noninvertible) or close to singular.
rank
and
shows
if rank(A)
= n.
The
is
discusses
and only
culations
singular.
than
inv
but
nxn
matrix
MATLAB
rank
command
more
reliable
when
an
that
rank(A)
is to enter
invertibility
to test
method
Another
4.6
A
is
requires
a
.
Section
invertible
matrix
more
is
if
cal-
nearly
2.4 PARTITIONED MATRICES
The ideas in this section are fairly simple.
However, mark them for future
reference,
because
you are
likely to use this
notation
after
you
leave
school.
Partitioned matrices arise
in theoretical
discussions
in essentially every field that makes use of matrices.
Here are two examples.
1.
The modern state space approach to control
systems engineering
depends
on matrix calculations.!
The problem of determining whether a system is
controllable amounts to calculating the number of pivot positions
ina
controllability matrix
[5
where
of
2.
the
A
n-1
AB
ATE
ee ae
is
nxn,
B
form
(8)
in the
has
n
rows,
discussion
and
the
matrices
preceding
come
Exercise
from
an
equation
19.
Discussions of modern algorithms and computer software design for scientific computing naturally use the "language"
of partitioned
matrices.
For instance,
common techniques for parallel processing of large matrix
calculations,
such as slicing and crinkling,
are described with parti-
Tan understanding of control systems is important
in the design of filtering circuitry,
robots,
process
control
systems,
and spacecraft.
Thus
a
control
systems course
is often part of the undergraduate
curriculum for
electrical,
mechanical,
chemical,
and aerospace engineering.
See Control
Systems Engineering,
by Norman S. Nise,
Benjamin/Cummings
Publishing Co.,
Redwood City, CA, 1992.
2.4
tioned
matrix
¢
Partitioned
matrices.2
Also,
the
standard
computer
calculations relies heavily on partitioned
Matrices
2-15
science
reference
matrices.?
on
KEY IDEAS
eer wraa |
The column-row evaluation of AB is the last of five different
"views" of
matrix
multiplication.
All five are special
cases
of the block matrix
version of the row-column rule for matrix multiplication.
Here they are:
(1)
The definition of
only one column:
Ax =
Ax
amounts
[ar| +++ |an] [xi]
to
block
=
[xray
and
one
multiplication
ware
ners
column.
Then
of
AB
where
B has
Xn
(2) Partition
usual
A as
product
one
AB
row
is a row-column
block
the
definition
of
the
is
partitioned
as
one
product:
AB = A[b,
|b2|--- |bp] = [Abi
|Abe | --- | Abp]
(3)
Likewise,
we observed
in
row and one column,
then
row, (A)
ABcE
row, (A)
2.1
that
if
B
row, (A)B
B=
row,(A)B
row (A)
(4)
Section
row (A)B
The next display can be viewed either as just the standard
rule in which each entry of AB is computed as the product of
and
a
having
column
only
of
one
or
as
column
B,
of
a
multiplication
blocks
and
B having
of
block
only
one
row-column
a row of A
matrices
row
of
(with
A
blocks)
?Parallel
Algorithms
and Matrix Computations,
by Jagdish J. Modi,
Oxford
Applied Mathematics and Computing Science Series,
Clarendon Press, Oxford,
1988," pp. .73-75.
SMatrix
The
Computations,
Johns
Hopkins
2nd
Press,
ed.,
by
Baltimore,
Gene
1989.
H.
Golub
and
Charles
F.
Van
Loan,
2-16
Chapter
2
°¢
Matrix
Algebra
row, (A)
row, (A) [col (B)
:
col _(B) °:+ col (B)]
1
:
re
row (A)
row, (A)col
=
(5)
(B)
cee
Coy
a
see
sicbeleceRAg
lrow
(A)col (B)
Loe
1
-::
row
(A)col (B)
cee
j
-°*:
row
row
(A)col, (B)
toe
row, (A)col ,(B)
cee
row (A)col (B)
i
(A)col (B)
.
p
The final display is the column-row expansion of AB (Theorem 10 in this
section).
In this view, AB is expressed_as a sum of outer products of
the form uv,
with u a column of A and v
arrow of B.
But the display
can also be viewed as the block version of the row-column product
in
which
A has
one
row
(of blocks)
and
B has
row
AB = [col,(A)
col,(A)
one
1
column
(of blocks):
(B)
+++ col (A)] ary
row (b)
= col
(A) -row, (B) +
see
+ col (A) -row (B)
You might say that the row-column rule computes AB as an array of inner
products
(view 4 above),
while the column-row expansion displays AB as a
sum
of arrays
(view 5).
SOLUTIONS
TO EXERCISES
we eee
1.
Apply the row-column rule as if the matrix entries were numbers,
for each product (such as EA below), always write the entry of the
block-matrix on the left.
POP
fet
ge
AS
LO
Bt
ome
EAL OC
EA+IC
This
eID ODT. &
EB+ID|
must
be
EA,
not
A
|EA+C
AE.
B
EB+D
but
left
2.4
Checkpoint:
Notice
in
Exercises
1
and
3
¢
that
Partitioned
Matrices
é
k
| and
2-17
| act
block-matrix
generalizations
of elementary
matrices.
What sort
of
block matrix is the appropriate generalization of an elementary matrix
acts as a scaling operation?
(Answer this carefully. )
7.
Compute
the
left
side
of
the
xX
O
0
Beg
F,
|
XASea
YAre
this
equal
to
the
Set
XA
Since
the
the
the
ON
OBIS
Dick TBA
side
DBE
6)
Sarda
(1,1)-blocks
of
AZataOgs 0
IY
Zi
seek
the
equation:
XA
=
r| Slat or Wide HiBas
are
equal,
XA
IMT implies that A and
(1,2)-entries,
XZ = 0.
X are
Since
invertible,
and
X is invertible,
=
fore,
the
(2,2)-entries
give no new
(2,1)-entries,
YA + B = 0 and YA =
shows
that
Y =
-BA
.
The
2x2
that
equation:
right
XZ
ine
as
order
of
I.
Since
[
X and
XB
w=SH0
A are.
square,
Vz-+ cle =a.8
hence X =
Z must be
A.
0.
From
Therefrom the
information.
Finally,
-B.
Right-multiplication
the
factors
for
Y is
by
A
crucial.
Study Tip:
Problems such as 5—10 make good exam questions.
Remember to
mention the IMI when appropriate,
and remember that matrix multiplication
is generally not commutative.
11.
13.
a.
See
the
subsection
Addition
b.
See
the
paragraph
before
You
are
asked
to
“(BD
Scalar
Example
establish
pose that A is invertible,
and
an
"if
3.
and
B
O||D
BE}
CATE»
GIVIS\\CFSeieG
BE
i
I
6)
OMT
|
Since B is square,
the equation BD =
the IMT.
Similarly,
CG = I implies
BE = O implies
Gigi (BoduO |B ehD
= [ | -[P
A
.
-1
only
and let A” = E
Ore
equation
Multiplication
that
statement.
First,
sup-
ab Then
I implies that B is invertible,
by
that C is invertible.
Also,
the
E= B
0=0.
etek] tt, [B-y1
Abul
BOK
o|
-1
if"
Similarly
F = 0.
Thus
:
™
2-18
Chapter
2
«¢
Matrix
Algebra
For
This proves that A is invertible only if B and C are invertible.
suppose that B and C are invertible.
the "if" part of the statement,
Then (*) provides a likely candidate for A’ which can be used to show
Compute:
that A is invertible.
Hae | areal
ol
this calculation and the IMT imply that A is inverSince A is square,
Without it, the argument is
this final sentence.
forget
(Don’t
tible.
incomplete.)
Instead of that sentence, you could add the equation:
Ee y-Eed-E ¢
19.
The
matrix
equation
(A - sI,)x
Rewrite
the
+ Bu=0O
first
invertible,
x
=
formula
x
into
for
(8)
in the
and
text
*(-Bu)
wIh)xe=
-
sI,)
the
second
equation
+ u = y,
so that
Thus y = (Ip - C(A - sIp) B)u.
21.
to
Cx+u=y
equation | asa(Ao=
(A
C(-(A - sI,)"'Bu)
W(s)u.
in the
is equivalent
=
-(A
-
~Bu.
Sin)
WhenniA
Bu.
-esi,—is
Substitute
this
above:
I,u - C(A - sI,) Bu = y
If W(s) = Im - C(A - sIp)B, then y =
The matrix W(s) is the Schur
system matrix in equation (8).
complement
of
the
matrix
A - sI,
To prove a statement by induction,
a good first step is to write
statement
that depends on n but exclude the phrase
"for all n",
label the statement for reference:
The product of
gular matrices
Second,
verify
that
two nxn
is lower
the
the
and
lower triantriangular.
statement
is
C*)
true
for
n
=
1.
In
this
parti-
cular case,
(*) is obviously true, because every 1x1 matrix is lower
The "induction step" is next.
triangular.
Suppose that (*) is true when n is some positive
integer k, and
consider two (k+1)
x (k+1) matrices By and C,.
Partition B, and C, as
where
B and
C are
kxk
matrices,
x
and
y are
in
R",
and
b
and
c
are
2.4
scalars.
Since
Partitioned
Matrices
2-19
B, and
C; are
pic, = [2
©
oO] _[bero'y
bor+o'c] _ [ be
o7
X85
Bid
nak
x0 + BC
BC
Assuming
|fe
(*)
is true
of
B,C,
shows
that
(*)
for
any
positive
to Checkpoint:
lower
triangular,
xc + By
for
it,
n = k,
too,
is
integer
BC must
lower
be
so
lower
are
B and
C.
Now,
cx+ By
triangular.
The
Thus,
the
truth
of
triangular.
form
n =
k implies
the
truth
of
(*)
for
the
k+1.
By the
principle
of
induction,
(*)
must
be
The
block
diagonal
of
Induction
next positive integer
truestor,-alion- 2.1.
Answer
°¢
E
1 and ki A are
0
ii
6)
E
obvious choices.
Less obvious is the requirement that E be invertible,
in
order to make these block matrices invertible.
(Recall that the invertibility of elementary matrices was essential for the theory in Section 2.2.)
Principle
matrices
Appendix:
The
Consider
Exercise
a statement
"(*)"
21.
To prove "by
that depends on
induction"
that
integers,
you
two
things:
is true
for n=
must
(a) Statement
(b)
(The
then
prove
(*)
a positive
integer n, as in
(*) is true for all positive
1.
induction step) If (*) is true for any positive integer
(*) is also true for the next integer n = k + 1.
n = k,
A property or axiom of the real number system,
called
the principle
of
mathematical induction, says that if (a) and (b) are true, then (*) is true
for all integers n 21.
This is reasonable,
because if (*) is true for n =
1, then (b) shows that (*) is true for n= 2.
Applying (b) again with n =
2, we see that (*) is true for n= 3.
Applying (b) repeatedly, we see that
C{jtsetrTuerfor*2; 3a,
"5;
MATLAB
Partitioned
MATLAB
E, and
uses partitioned matrix notation.
For example,
if A,
F are matrices of appropriate sizes, then the command
M=[ABC;
creates
a
there
is
instance,
M(1,2)
larger
DE
same
of
the
form
M =
A=
D
of the partition that
B was the
(1,2)-block
as
B,
C,
D,
F]
matrix
no record
although
is the
Matrices
the
(1,2)-entry
of A.
Be
E
C
eh
Once
M is
formed,
was used to create M.
For
used to form M, the number
2-20
Chapter
2
«¢
Matrix
Algebra
2.5 MATRIX FACTORIZATIONS
In a sense, Section 2.5 is the most up-to-date section in the text, because
matrix factorizations
lie at the heart of modern uses of matrix algebra.
For
instance,
they are
indispensable
for the analysis
of computational
algorithms and research in parallel processing.
The text focuses here on
triangular factorizations,
but the exercises introduce you to other important factorizations that you may encounter later.
KEY IDEAS
“ete Beng!
When a matrix A is factored as A = LU, the data in A are preprocessed
in a way that makes the equation Ax = b easier to solve.
Write LUx = b, or
L(Ux) = b and let y = Ux.
Solve Ly = b for y and then solve Ux = y for x.
The two-step process is fast when L and U are triangular.
Finding L and U requires the same number of multiplications
and divisions as row reducing A to an echelon form U (about n/3 operations when A
is nxn).
After that,
L and U are available for solving other equations
involving A.
The key to finding L is to place entries in L in such a way
that the sequence of row operations reducing A to U also reduces L to the
identity.
The
In this
text
case,
discusses
LU must
how
to
equal
build
A.
(See
L when
the
top of page
row
interchanges
no
136.)
are
needed
to reduce
A to JU.
In this
case,
L can
be unit
lower
triangular.
appendix below describes
how to build L in permuted unit triangular
when row interchanges are needed (or desired,
for numerical
reasons).
SOLUTIONS
TO EXERCISES
eae
1.
L=
|-1
1
OS
iE
b]
=
O},
1
U=
-7
10
(@)
-2
-1],
(9 ee |
1
0
0”
=7
=]
1
fe)
5
1
2
Stas
~
1
10
@)
0
1
0
Se
-2
3
O;-,/0
1
OTe?
Ow 42,1
1
6
~
b=
5}.
2
First,
solve
Ly =
1
6)
9
Wf
10
1
0
-2
The
only
On
esd
1
16
is
in column
b.
arithmetic
4
27
1450 Y.=al=c
6
Next,
solve
Ux = y,
using
[U
yl =
Srbal
|0 -2
8 eer «Pee
Shel
(Sl
-1
=2)¢~s.10.
222.
|
6
6)
0
back-substitution
(with
matrix
Paes.
3 i,
40
p=2ieeulGecoo
1!
51
fe)
6)
notation).
i =19
0, =3
1 =o
An
form
She
10
6)
=
So x =
T/
1
¢)
0. -19
0
4|
ly? 26
~
3
10
0
2.5
¢
0
1
6)
0
6)
t
Matrix
9
4;
=6
Factorizations
1
|0
8)
~
6)
1
6)
6)
6)
hs
2-21
3
4
=G
(3,4,-6).
Checkpoint:
Exercise
11 in Section 2.2 shows how to compute
reduction.
Describe how you could speed up this calculation
an LU factorization of A available (and A is invertible).
A'B by row
if you have
7.
dividing
Place
the
column
by 2 (the
2
first
pivot
column
of
-3e
°
=
into
L,
pivot),
then
add
3/2
times
row
1 to row
a-ba-
|
19.
2
Worse
0B MisOs
-2.-4
5
4
-9
=10
3
+1
NY
+¢+-2
NY
1
i.
1
4a8b073
TE
Staten
reves
7
.5
10
Bete
0-10
15
5
Oy.
b2 «c=
Sarl
Use the
to make
yielding
1
=3/2,
9)
4
U.
ee 2
NY
NY
=3/2
fea
re
GQ)
2,
the
Ye
Bt
sqieae
after
MRE
ES
CEES
|?
SOP
(0% a2
0
6—O
Olsa0
Sigectalran gee
Ome
Olew
eee
0
606—O
Aas
a aa
last two columns of I4
:
A
L unit lower triangular.
|
fis
4
5
1
ae
=a!
6)
;
f
1
zt
0
1
6)
8)
0
6)
uh
5
1
8)
=Z
=
6)
1
A good answer will require a written paragraph or two.
If you have not
tried to write your answer,
do so now,
without
reading
the solution
below.
Explain how you would row reduce
[A
I], knowing
that
A is
lower triangular.
Your answer to this question should contain some of
the ideas shown below, although your wording might be quite different.
2-22
Chapter
A
Let
¢
Matrix
Algebra
and
consider
augmented
the
with
matrix
nxn
lower-triangular
a
be
diagonal,
a.
2
matrix
nonzero
on
entries
the
FAT}
The (1,1)-entry can be scaled to 1 and the entries below it can be
This
changed to O by adding multiples of row 1 to the rows below.
So
affects only the first column of A and the first column of I.
the (2,2)-entry in the new matrix is still nonzero and now is the
only nonzero entry of row 2 in the first n columns
(because A was
lower
triangular).
The (2,2)-entry can be scaled to 1, and the entries below it
can be changed to O by adding multiples of row 2 to the rows below.
This affects only columns 2 and n+2 of the augmented matrix.
Now
the (3,3) entry in A is the only nonzero entry of the third row in
the first n columns,
so it can be scaled to 1 and then used as a
pivot to zero out entries below it.
Continuing
in this way,
A is
eventually reduced to I, by scaling each row with a pivot and then
using only row operations
that add multiples
of the pivot row to
rows below.
b.
21.
25.
The row operations just described only add rows
the I on the right in [A
I] changes into a lower
By Theorem 7 in Section 2.2, that matrix is A.
to rows below,
so
triangular matrix.
Suppose A = BC, with B invertible.
Then there exist elementary matrices £;,...,E, corresponding to row operations
that reduce B to I, in
the sense that E,:::E,B = I.
Applying the same sequence of row operations to A amounts to left-multiplying A by the product | eS ae
By
associativity of matrix multiplication,
Ep:
°° E,A = Ep°>-E,yBC
=
so
the
row
A=
T
UDV’.
imply
T\-1
(V)~
same
that
=V..
sequence
of
;
Since
U and
U and
ve are
:
Since
the
IC =C
operations
Vv’ are
square,
invertible,
:
diagonal
reduces
A to
the
equations
U'U =
I and
by the
IMT,
hence
ut
entries
and
o,,...,¢,
is, invertible,
with the inverse of D being the
01 ,---,0,
On the diagonal.
Thus A is a product
ces.
C.
in D are
VV
=
=
[
ie and
nonzero,
D
diagonal
matrix with
of invertible matri-
By Theorem 6, A is invertible and A’ = (UDV')~ = (y')“p%y”™ =
Vor:
Answer to Checkpoint:
If A is an invertible nxn matrix, with an LU factorization A = LU, and if B is nxp,
then A'B can be computed
by first
row
reducing
[L
B] to a matrix
[I
Y] for
some
Y and
then
reducing
[UY]
2.5
to [I
A J)
[A by -*:A
of, equations
A
B (after
One
way
to see
that
Matrix
algorithm
Factorizations
works
2-23
is to view
AB
as
bp] and use the LU algorithm to solve simultaneously the set
Ax = bi, ...,
AX = bp.
MATLAB uses this approach to compute
first
finding
L and U).
Appendix:
Permuted
LU Factorizations
Any
matrix
admits
mxn
this
°¢
A
a
factorization
A
=
LU,
with
U
in
echelon
form
and L a permuted unit lower triangular matrix.
That is, L is a matrix such
that a permutation
(rearrangement)
of its rows
(using row interchanges)
will produce a lower triangular matrix with 1’s on the diagonal.
The construction of L and U, illustrated below,
depends on first using
row replacements
to reduce A to a permuted echelon form V and then using
row interchanges to reduce V to an echelon form U.
By watching the reduction of A to V, we can easily construct a permuted unit lower triangular
matrix L with the property that the sequence of operations changing A into
U also changes L into I.
This property will guarantee that A = LU.
(See
the
paragraph
before
Example
The following algorithm
In the algorithm when a row
2 in the
text. )
reduces any matrix to a permuted
is covered,
we ignore it in later
1.
Begin with
the pivot.
the leftmost nonzero column.
Designate the corresponding
2.
Use
all
3.
Repeat steps 1 and 2 on the
nonzero entries are covered.
row replacements
uncovered rows.
to create
Then cover
Choose any nonzero
row as a pivot row.
zeros above and
that pivot row.
uncovered
echelon form.
calculations.
entry
as
below
the
pivot
(in
if
any,
until
all
submatrix,
This
algorithm
forces
each pivot
to be to the right
of the preceding
pivots; when the rows are rearranged with the pivots in stair-step fashion,
all entries below each pivot will be zero.
Thus, the algorithm produces a
permuted echelon matrix.
Whenever a pivot is selected, the column containing the pivot will be used to construct a column of L, as we shall see.
As
matrix
an example,
as the first
choose any entry in the
pivot, and use the pivot
choose the (3,1)-entry.
column
1.
We
call
Row
3
this
column
Rte jpene=
~|@
8
tt
=1
Say
-4
5
-8
-7
2
v3
O:
°=5
3
is
the
ist
pivot
first column of the following
to create zeros in the rest of
a
Ore
O -8
row.
Choose
the
(2,2)-entry
€
ist
as
the
pivot
row
second
pivot,
2-24
and
Chapter
2
«¢
fa
Cover
row
2,
choose
Algebra
call
the
this
column
excluding
2,
of column
rest
in the
zeros
create
Matrix
b
(4,4)-entry
as
the
pivot is relative to the original matrix.)
(in the pivot column), excluding the first
fark call
row.
pivot
first
the
this
column
pivot.
€
2nd
pivot
row
€
ist
pivot
row
(The
row
Create zeros in
two pivot rows.
c
ia
index
the
column
other
of
the
rows
d
€
4th
pivot
row
€
2nd
pivot
row
€
1st
pivot
row
€
3rd
pivot
row
Let V denote
this permuted echelon form,
and permute
the rows
of V to
create an echelon form.
The first pivot row goes to the top, the second
pivot row goes next, and so on.
The resulting echelon matrix U is
4
So =a
6)
JHeS
2 ala Oiad 9
ee
ee
Oo
Su Sorie. Uy
Ae)”:
6
ss
The last step is to create L.
Go back and watch the reduction of A to V.
As each pivot is selected,
take the pivot column, and divide the pivot into
each entry in the column that is not yet in a pivot row.
Place the resulting column into L.
At the end, fill the holes in L with zeros.
Column:
a
b
Co}
1
-2
=:
al
d
2}
la}
+4
+3
+-2
+-5
NT
NH
NZ
v
We
1
1/74
-—1/2
1
1/2
|
|e
1
i
:
<-1/3
1
1/4
=1/2
Ie
1/2
=)
1
8)
-Tt/3
=1/2
0
0
1
1
0
0
0
2.5
¢
Matrix
Factorizations
2-25
You can check that LU = A.
To see why this is so, observe that L is
constructed so the operations that reduce A to V also reduce L to a permuSince the pivots in L are in exactly the same rows as
ted identity matrix.
in V, the sequence of row interchanges that reduces V to U also reduces the
Thus, the full sequence of operations that
permuted identity matrix to I.
A to
reduces
135
of the
U also
L to
reduces
I,
so
that
A =
LU.
(See
the
box
on
page
text.)
The next example illustrates what to do when V has one or more rows of
zeros.
The matrix
is from the Practice
Problem for Section 2.5.
For
the reduction of A to V, pivots were chosen to have the largest possible
magnitude (the choice used for “partial pivoting"). Of course, other pivots
could have been selected.
pime
4-AS2hrg
8}
ff
4
-1
0
A=]2
6
-7
-3
-2
-2
3
first
three
g9|~]0
3 y4
Opts
Oien0 S- 10
-5
0
0
0
0
=2
6?
Sti2
columns
of
L come
0
cele
157.3
Z
4
=6
-4
4
€6)
=573
Ny
NY
1/3
1/6
-1
2/3
=-2/3
1
-1
1
needs
two
4
The
matrix
L
identity matrix
remaining holes
:
more
to place 1’s
with zeros.
~
4/3 -19/3
-2
12
.
€ 3rd
€ 2nd
row
row
the
three
pivot
€ ist
pivot
row
€ 3rd
€ 2nd
pivot
pivot
row
row
columns.
in the
Use
-9
10°
0-0
3
and
1
1 and
3.
8
-2
8)
0
columns
rows
-5
-6
0
8)
columns
"nonpivot"
6
0
aU =
pivot
pivot
from
1
1/3
2/3
-1
19/3]
8 | € ist pivot row
0O
Ps
-4/3
0 €
© -9
~v=1|10
The
i
@ -9 -5
12
573
0
0
0
8)
above.
of
5x5
the
Fill
in
the
2-26
Chapter
2
«+
Matrix
Algebra
ae
qhOeiey
Oe
eOihet
te}
oo
pe
2740s
plurotdonagO
{0
uae
1xtaaOe
1 |A@ Bicevig|8/3 s2/32.
ret
1/3 1/6 -1
G
¥$Oinab!
1/3: 2/Bii sly
Tn40
1/Siqel/6. (Ga lade wlan
1033
Qin
0
OSIN
Y
5=%o
92/31
173
Ohet
a
e
()
03
2/300¢273
9201
Oi
-1
a permuted
produces
of L using only row replacements
Row reduction
into rows
4
and
5,
2,
rows"
"pivot
the
in
1’s
the
Moving
matrix.
identity
1, 2, and 3 of the identity requires the same row swaps as reducing V to U.
If a further row interchange on the permuted identity is required,
it will
which came from the "nonpivot"
involve the bottom two rows,
rows 1 and 3.°
A corresponding interchange of the bottom two rows of U has no effect on U
As a result, L is reduced to I by the
(and the product LU is unaffected).
same operations that reduce A to V and then to U.
Check that A = LU.
MATLAB
LU Factorization
and
the
Backslash
Operator
\
Row reduction of A using the command
gauss
will produce the intermediate matrices needed for an LU factorization of A.
You can try this
on the matrix
in Example 2, stored as Exercise 33 in the Toolbox.
The
matrices in (5S) on page 136 in the text are produced by the commands
gauss(A,
1)
gauss(U, 2)
gauss(U, 3)
U has
Now
The
the
first
pivot
U has 0’s below
echelon form
0’s
below
pivots
1 and
2
You can copy the
information
from
the screen
onto
your
paper,
and
divide by the pivot entries to produce L as in the text.
For most text
exercises,
the pivots are integers and so are displayed accurately.
To construct
a permuted LU factorization,
use
U = gauss(U,r,v)
,
where r is the row index of the pivot and v is a row vector that lists
the rows to be changed by replacement operations.
For example,
if A has
5 rows and the first pivot is in row 4, use
U = gauss(A,4,[1
If the next pivot is in row 2, use
U = gauss(U,2,[1 3 5]) .
the permuted matrix L, use full columns from A or
U, divided by the pivots.
Then change entries to
row already selected as a "pivot row."
The
MATLAB
command
[L U] = lu(A)
produces
2 3 5])
To build
the partially reduced
zero if they are ina
a permuted
LU factoriza-
tion for any square matrix A, but it does not handle the general case.
When A is invertible,
the best way to solve Ax = b with MATLAB is to
use the backslash command
x = A\b .
MATLAB proceeds to compute a permuted LU factorization of A and then use L and U to compute x.
The
alternative command
x = inv(A)*b
is less efficient and can be less
accurate.
In the
The _command
form
Ui
+L.
.
inv(A)
uses
the
LU factorization
to compute
re
2.6
°¢
Iterative
Solutions
of Linear
Systems
2-27
2.6 ITERATIVE SOLUTIONS OF LINEAR SYSTEMS
Some knowledge of iterative methods for solving linear systems is important
for anyone who uses linear algebra.
If your course does not have time to
cover this material, you might at least glance at the discussion.
Then, if
you need
to solve
a large
sparse
system
later
in your
career,
you may
remember (perhaps vaguely) the ideas here, and you can review this section
in your text before setting out to use appropriate computer software.
KEY IDEAS
——
7}
Both
form
iterative
(M - N)x
Nx + b.
Given
algorithms described here are based on writing Ax = b in the
= b, where M is readily invertible,
and then writing Mx =
an
initial
MxD = wx
estimate
+ bd
x
for
the
solution,
the
recursion
(k= 0,1,...)
(*)
defines a sequence of vectors that may converge to the desired solution.
For Jacobi’s method,
M is the diagonal matrix formed from the diagonal
entries
in A; for the Gauss-Seidel
method,
M is the lower triangular part
of A.
In both cases,
of course,
N=M - A.
SOLUTIONS
“SR
TO EXERCISES
SE
peara aig app
If working
by hand,
the
recursion
abe”aloesl si waleglt ol
(*)
above
0
6)
can
be simplified
ce mme oo = FESIE ae” + LD
(k+1)_
Then,
with
x
Ce
x
1
x
(k)
=
0,
(1)
(2iead
(3)
OG collet. 75
TNOGY
OL et 40
Pre
25
=1) > ¢k)
(4
to
2-28
Chapter
Using
2
«¢
Matrix
Algebra
MATLAB,
To
four
*
decimal
(Sr
ani ¥ick OGRO|
ValayrL tis: 0044
% <n Ta
S¥00GS | ia:
(ey ror) SBD.
0009
in the sequence
its predecessor.
{x}
oO =
1
-5
Sim
0
"21,
6
b=
el.
115\..
17
For*Gauss-Seidel,
Near
3
tat.
6)
0
p25
3
O
Ouse
f
3
eels
6)
li
a5
3
0
2| =
if.
9
|0
6)
If working by hand, write the recursion
This
will
-1.0001
. 0009
3
1-1
6)
A=
| 2.0003
places,
Thus x ®) is the first vector
within .001 of the entries in
7.
(6) _
1.9994
(5) 8
1.19350
(4)
* 52 “pela 9075 4aen) 1
avoid
solving
a 3x3
system
whose
3
|[-1
6)
6)
-s
os
entries
6)
O|,
vi
are.
and
at
O
Omts2
0
0
(k+1)
(*) as x“*=
for
each
mMtwx"
iteration.
The
+ mb.
fastest
way to compute
M N and M b is to reduce
the augmented
matrix
[M N b] to a matrix of the form [I M°N
Mb].
See Exercise 11 in
Section
2.2,
with
the matrix
B replaced
by [NWN
3480
160.0) GRis LO) On! pit
aie! =SiqnOeieO
aeO «-Oc.45]
Omeiet
Te icOrmnQed
OM iy,
1 ae OW
eotloluss5S
O4eSy.
1
Oy
10
1
Oe ~ 0
Oe
asl/3
Cir
ells
1
Ou- 1735
13/3
-56/15
6141735
¥
MoO N=
0-173
6)
1 O°=35
N40) 1/4
S/S le = Tos O
MYTLAR
0 3=1738:76735
0, 4-3
s
11/3
|[-56/15]|
141/735
%9
M
b =
6)
i275
jpi-5/35if
1
385
= 105 =392)
423
2)
6)
42
(=18
3.66667
|=34/3230
4.02857
bl.
Ont
(Oi
eiOno
iia
eG
*
= 178
0
¢)
0 = 133333
10 .O06667
0) =202857'™
Octet
-2
0)
173
15
17
6)
.40000
217143
2.6
The
first
two
¢
Iterative
iterations,
by hand
Solutions
(with
0
385
0
423
x=
zc MN(Mb)
x
Mb
és (MN
<
0),
Linear
1
T05
105
0
6)
=-35
LA
— $8:
1)
1
42 Ea
87
Your
answers
may vary
slightly
throughout your work.
Next,
= os -392
423
1)Mb
4.9111
-2. 3708
3.4446
To
four
decimal
x) ~ x
4.4569
ano td
3.5058
385
-392|]
423
if
=
you
Using
matrix
54145
-26138]
3ISTL
used
decimal
4.49953
=2 0001S
3.50006
to
calculations
4.9111
|-2.3708
3.4446
~
approximations
4.50005
-2: 49998
3.49999
places,
=
O00)
= |- oois],
x
(a)
- x"
=
. 0005
= | 0002
=S000T
Thus x®) is the first vector in the sequence
within .001 of the entries in its predecessor.
3 4-[5 abel e-bs ab*-f
-2
algebra
the
1
ie
105
4.5045
-2. 4986
3.4994
. 0007
4
2-29
are
simplify
=
Systems
385
x? = w'no + wb = |ol} + aa -392]
x!”
of
10
Beye
ee
efor
{x}
whose
entries
are
2
Some matrix program is essential here.
To help you check your work, if
necessary,
the results of the first few computations are shown below.
(See
the
text
for
the
final
answer.)
2-30
17.
Chapter
c.
2
«+
Matrix
Algebra
x) S99 ca 4 ss.
specific
recursion
x
k =
(k)
+
=
k =
for
d,
(cS pe
(*)
recursion relation,
for k = 2,
holds
1.
Substitute
the
+d
=Cld
+
that
suppose
Now
+ Cd.
d =a
cx’) +
=
x‘?
because
CMa)
Cx
(*) holds for k = 1, because by the
(*)
Also,
= d.
+ d = CO +d
observe that
= Cx
x
some
1
aes _
«6°? = 0 and x) satisfies
prove by induction that
Suppose that
1,2,....
To
(*), holds
for
into
the
x
for
(*)
formula
relation:
kth)
be Cx!”
+
Ca
-*
+
Crd)
+d
Cd + Cd+ +--+ +Catd
Thus the truth of (*) for k implies the truth of (*) for
the principle of induction,
(*) is true for all k = 1.
MATLAB
Jacobi
and
Gauss-Seidel
k +
1.
By
Methods
The command
M = diag(diag(A))
creates a diagonal matrix from the diagonal entries of a matrix A.
(The command
diag(A)
produces a column
vector that lists the diagonal entries of A.)
To create the lower triangular part of A for the Gauss-Seidel Method, use
M = tril(A).
The
except
Jacobi
and Gauss-Seidel
methods
use
the same
MATLAB
commands,
for the construction of M.
Enter
N= M- A_ and make sure the
initial
solve
this
data
the
is
the
vector
Mx
=
To
M\(N®x
+ b);
x’
"backslash"
the
ifor
xe.
"next"
The
correct
x,
you
operation:
way
must
the
the
is,
b,
the
instead of assigning the solution to x.
you should put
back into x, so you can repeat the operation with exactly
That
Nxct
produce
do
symbols.
x =
x:
to
is to use MATLAB’s
However,
solution
same
in
equation
x‘)? = M\(N*x + b).
command
To
save
space,
x
is displayed
horizontally
is used over and over,
creating the sequence xk
(The semicolon suppresses
the display of the column vector x. )
These vectors
can be
recorded
on paper as they appear.
You only type the command
once.
After that, use the up-arrow to recall the command, and press <Enter>.
If you wish to monitor
how close
the approximations
are
to each
other, use the following sequence of commands (repeatedly):
t = M\(N¥x + b);
change = t’ - x’
x
ts
t’
t is a temporary storage for
Compare t with the old x
Update
x
with
new
values;
the
don’t
new
x
display
them
2.7
¢
The
Leontief
Input-Output
Model
2-31
the
in
sec-
2.7 THE LEONTIEF INPUT-OUTPUT MODEL
If
you
are
in
economics,
you
definitely
will
need
material
this
tion for later work.
Although most of the discussion concerns economics,
the formula for the inverse of I - C is used in a variety of applications.
STUDY
NOTES
er See |
The power of Leontief’s model of the economy is that it compresses hundreds
of equations
in hundreds
of variables
into
the
simple
matrix
equation
(I - C)x = d.
You should know how to construct
the consumption matrix C
and know the algebra that leads from the matrix equation x = Cx + d to its
solution
less
than
x =
(I -
C)
d,
under
the
assumption
the
formula
that
the
column
sums
of
C are
one.
You may need
to know
(8)
for
(I - Cl
with your instructor.)
The formula is analogous
of a geometric series of positive numbers:
peg
pee
tp ae
an
to
the
on page
152.
formula
for
(Check
the
sum
fe) POMhener | <et.
SOLUTIONS
TO EXERCISES
a
1.
Fill in C one column at a time, since each column is a unit consumption
vector for one sector.
Make sure that the order of the sectors
is the
same for the rows and columns of C. From the way the data are presented,
we use
the
order:
manufacturing,
agriculture,
and services.
Read
the
sentences carefully,
to get the data arranged correctly.
Purchased
from:
Unit consumption
Manuf.
Agric.
Manufacturing
Agriculture
Services
10
230
630
. 60
. 20
+10
vectors
Serv.
.60
. 00
50
The intermediate demands created by a production vector x are given by
Cx.
If agriculture plans to produce 100 units
(and the other sectors
plan to produce nothing), then the intermediate demand is
Cx =
710.
|.30
a0"
~. 60)
.20
(10)
460
6)
.00]|100;}
=
710
0
60
/|20
10
2-32
7.
Chapter
C==
|-O
te
of
sector
a.
The
2
°¢
Matrix
Algebra
<5
ea
d =moe
Sab
dy =asl
Let
lo}: the
demand
the
demand
1 unit
for
of
output
p
1.
to
required
production
satisfy
Ls
De 17m |S4 =| i MA
Ne
Pv
is
4d,
the
x;
vector
5,
From Exercise
d,.
x; = (J - C)
namely,
(I - C)x, = d,,
such that
iPas Ti oe,
= [he 5
so
oie eee
b.
For
the
given
final
demand
do
=
BA
the
corresponding
-1,.
_
[1.6ocpfsxtsifen
production
x2
is
the
d
is
by
ie
c.
eae
e
From
Exercise
5,
given
by x =
20 i
6
ee ies ¢ fea = veep
the
production
110
Observe
x
from
corresponding
(a)
and
(b)
to
that
demand
xo = x + x,.
Also,
as pointed out in the text, dg = d + dy.
The sum of the production
vectors x and x; gives the production needed to satisfy the sum of
the demands d and d;.
This is expressing the linearity between final
demand
and
production.
This
relation
is
true
in
general,
because
Xo = (I - C)"*de = (I - C) (a + dy)
(T= GC) td etl
=
11.
xX +
Cds
X,
Following the hint
in the text,
you should obtain
p x = p Cx
(from the price equation).
Then,
from the production equation,
p,.(Cx a
= pCx + pd.
Equate the two expressions for px to
pd=vx.
Another solution:
Take transposes in the price equation,
p = (Cp)
and
tv
right-multiply
©
T
ey
= pc +v,
by x to
-
so
v=
p
-pc
From
the
obtain
a
vx=px-pCx=p(I-C)x=pd
production
equation
+ vx
px =
yield
2.8
°¢
Applications
to Computer
Graphics
2233
Warning:
In Exercise 9, don’t multiply the consumption matrix C by 10, to
That changes the equation Cx = x + d into 10Cx =
get rid of the decimals.
However, you may multiply the augmented
x + d, whose solution is different.
matrix
[(I
C)
0]
affected
when
both
sides
13.
data
for
The
-
this
by
10,
are
the
because
multiplied
exercise
are
in
solution
of
an
equation
by a nonzero
number.
the
To
Toolbox.
solve
the
is
not
equation
(I - C)x = d, row reduce the augmented matrix
[(I - C)
4d] rather than
compute (I - C) .
(Another reasonable solution method is suggested in
Exercise
15.)
The
numerical
solution
is given
in the
text.
2.8 APPLICATIONS TO COMPUTER GRAPHICS
According to my students over the past few years,
this section
is one of
the most interesting application sections in the text, because it shows how
matrix calculations,
performed millions of times per second, can create the
illusion
of 3D-motion
on a computer screen or in a movie
theater.
Of
course,
one short section cannot begin to indicate the vast scope of computer graphics.
I encourage
you to look at the book by Foley,
et al.,
referenced in your text.
Chapters 5, 6, and 11 are filled with matrices!
The rest of the 1100 pages in the book contains
lots of interesting mathematics,
detailed discussions
of computer algorithms,
and scores
of spectacular (in some cases,
almost unbelievable) color plates.
KEY IDEAS
thie Bact
A graphical
object, represented by a number of key points, can be stored in
a data matrix D, one column for each key point.
When a linear transformation,
determined by a matrix A, acts on the object,
the transformed object
is determined
by the data matrix AD, because
A acts separately on each
column of D.
Homogeneous coordinates are needed to make translation act as a linear
transformation and to provide perspective projections
that give the illusion
of
three
dimensions.
iT
q
i
je
|
The
matrix
for
these
transformations
has
the
form
A is a 2x2
For 2D-graphics,
A
For 3D-graphics,
vector.
matrix,
is 3x3,
q: = [0 0], and
q is associated
p is
with
a
a
translation
perspective
2-34
Chapter
2
°¢
Matrix
have
fact,
considered
the only perspective transformations
In
7dix
0-4 Ocul
and q. =»[
jection at (0,0,d),
and
Exercises
SOLUTIONS
Se
1.
From
their center of prod = 10 in Example 8
20.
TO EXERCISES
Example
neous
5,
the
coordinates
ordinary
is true.
the
and
19
simplicity,
For
vector.
translation
a
is
p
and
any),
(if
transformation
Algebra
for
vectors
Let A
action
of
ellie
|0
8)
matrix
in
be
8)
1
6)
R® that
the
6)
O|
1
matrix
R*.
Partitioned
a 2x2
matrix.
Ki 1 on
has
E
7] corresponds
to
eee
ae tLOsl
el
Ax
*
1
Ox
1)
a
1)
1
Leip
X)
Agee OUT
+
same
effect
e| of
Example
1-1
the
action
of
Coordinates
13
Ov
0
so
-Oeet61d
oles 8
2420
1403/2
8)
1
the
0
Then,
translate
for
1]
}|O
x.
coordinates
Omogeneous
°
about
:
fo)
cos
60°
ae
ls
| is
-6
-8
6)
the
1
translate
by
6)
6
1/2
-vV3/2
=|0
-3+4Vv3
1
£8I{|v¥872
1/2
-4-3vV3]
0
0
1
8)
1
The answer
formations
the matrix
this
that
First,
origin
0
on
Fe
‘
rotation
Ghitha®d
O}]]O
1
rotate
about
back
13.
matrix
-V3420e
1/2
0
Finally,
at
3x3
2 has
Re
sin60°
al
homoge-
A on
in
7. A 60% rotation about the origin in R® is given by er a
Varo
on
matrix notation explains why
The following diagram shows
Sa
s
St ee
Ssh s
Rie
KS
the
-p
=
1/2.
-v3/2
3+ 4v3
|v3/2
1/2
4-3v3
6)
0
1
is given in the text.
Notice that the order of the
is important.
If the translation is done first (that
for the translation is on the right),
then
transph A
2.9
A*
Oe
O][f
wth |10
Here, 0
matrix.
19.
s Eid
is
a
p) Sf
et
zero
AT+00°
0'1+10°
row
vector,
and
product
00°
P for
the
perspective
at
(0,0,10)
and
the
data
are
shown
below.
The
data
matrix
Oid4a.24m
O} il
O;|
4
1
1
R"
[ap
Olea
matrix
0
0
i
6)
O
26.0
eee |
of
apy |
temaut
tion
1
6)
6)
6)
Subspaces
Ap +081] Sopa
=
O'p+i-1
O;
The
PD =
¢
so
the
outer
transformation
matrix
for
with
center
D using
homogeneous
the
of
Bigeye
2tyiaiot2iad
oe
aS)
iL
1
image
4.2
dae
6)
<6
the
2-35
is
a
zero
of
pro jec-
coordinates
triangle
is PD:
a6
2
ee
0,
0
8
.4
3
:
:
:
The R coordinates of the image points come from the top three entries
in each column, divided by the corresponding entries in the fourth row.
Od bol)
1276
6)
67.8
4/.8
0
2/74
2/.4|
0
7
2
6)
=
eo,
5
0
So
5
6)
2.9 SUBSPACES OF RN
This section is a condensed version of the ideas from Chapter 4 needed for
Chapters 5—7.
You need to study this section only if you plan to skip most
or all of ,Chapter..4..
If you reviewed carefully.after Sections.1.8,and. 2.3,
you should be well prepared for the new material here.
STUDY
NOTES
Sa or a
There
are
seven
fundamental
concepts
in this
section:
subspace,
column
space,
null
space,
basis,
coordinate
vector,
dimension,
and rank.
(The
term
isomorphism
is also
used
briefly,
to explain
why a _ p-dimensional
subspace "acts like" RP.)
Actually,
there is really not so much to learn,
because
you have
already
been working
with most
of these
concepts
for
several weeks,
without the terminology.
But you do need to know the precise
definitions
of these
than
just
at
the
end
mechanical
of
this
seven
terms,
computations.
Study
Guide
See
section
and
you
need
"Mastering
for
help.
to
understand
Linear
Algebra
much
more
Concepts"
2-36
Chapter
2
«¢
Matrix
Algebra
not to
you have to be especially careful
With many terms to learn,
corsound
may
they
though
even
undefined,
are
that
ways
in
ideas
combine
you
section
this
on
work
your
finish
you
after
example,
For
you.
to
rect
should be able to recognize that the following phrases (which have appeared
"the basis of a matrix," "the dimension
on student papers) are meaningless:
of a basis,"
and
"the
rank
of a basis."
You should pay particular attention to the notion of a subspace.
The
best mental image for a subspace is a plane in R through the origin.
The
distinguishing feature of such a plane is that the sum of any two vectors
in the plane is another vector
in the same plane
(by the parallelogram
rule),
and any scalar multiple of any vector in the plane is also in the
plane.
The main examples of subspaces are column spaces
(defined explicitly) and null spaces (defined implicitly).
SOLUTIONS
TO EXERCISES
eee
1.
The
vector
7.
a.
{v1,Vo,Vv3}
b.
Col A, which
vectors: all
c.
p is a linear combination of the columns of A if and only if the
equation Ax = p has a solution.
The following row reduction shows
that p is indeed in Col A:
[A
13.
u =
p] =
For Col A,
columns of
(1,2)
is
is a set
in the
that
set
H,
contains
but
(-1)-u
three
vectors.
is not
is the same as Span{vji,vo2,v3},
contains
possible linear combinations of vj, vo,
eat a |
|-8
S$
6
Bo Sto
a7
select
A.
To
any
find
a
10
6)
Note 8
6
—4 =10.
14)"
a
Dead,
C4
~210fe)
infinitely
and v3.
ee 4
7-410)
0
6)
2
Ses,
haan:|
1
Ts ee OC
MR NSS)
i!
0
3
8)
0
2
i
0
0
ee?
40
3
1 Oe
OF
aia
ee
4:
le
4
o-8
5
=8
ib
6)
Ole
0
3
120
0
1
iO
0
Oj
a
04
yet
PATE
970
|
YS
0
0
8
6)
X4
ee
2
0
many
6
514
6)
column of A, or any
linear
combination
vectors in NulA, solve the equation Ax =
3
as
Se
eS
Cee
OM!
1039
6
=10|.~
11
in H.
:
A=
6)
6)
of
0:
the
8)
Oi
6)
=
Eko te Xe
Xo. +. 2Xq-— 4X, = 0
0 =0
The general
solution
is x; = x3 - X4,
Xg = -2x3 + 4X4,
With x3,
X4
free.
Choose any values of x3 and xq (not both zero)
to produce
a
specific nonzero vector.
For instance,
if x3 = 1 and x, = 0, the vec-
toreGh,
122,
(1, ,O)6is
sin: NulJA;
19.
No.
Be
careful
not
are
not
in
RR’,
because
to
say
they
vectors do form
is not the same
a basis
as R’.
for
that
the
each
a
2.9
¢
vectors
are
have
R® is not a subspace of R®.
The
numbers with exactly two entries.
numbers)
with
Warning for
columns
of
in general,
2S.
Exercises 21-24:
an mxn matrix
A
even when m = n.
A common error is
form a basis
for
1-2
2 -3
10. ecOienB tally
CoM EOP Osa
Oj.o0
<6 =11
pivot columns are 1, 3, and 4.
by the four vectors consists of
iz
aa.
LN ae)
-4
So
31.
[xlp are
x1,
To
find
x; and
tte
2D aaa |
=3
5
So x, = 7/4,
37.
2-37
for
R?.
They
R°,
The
but
that
explicitly to
all lists (of
the nonpivot
is not
true
space
spanned
subspace
genera-
1-2
2 -3
$0440
Ao38po8
Capen Opes
ieee
Og 0~.10
4Q0
for
the
=
&
joa
4
25
of
in
to think that
Nul A.
This
a basis
mistake
in Exercise
when the definitions
entries
of
notation R? refers
R is the set of
Warning:
A standard
This easily happens
Cisly.
The
subspace
R"
entries.
the four vectors.
Then ColA is the
Look for the pivot columns of A:
1-2
2 -3
A
8-5
Al.
2 -4
4 -1
-5 10 -4
4
1
-4
roe
=O
two)
of
entries.
Make a matrix A from
by the four vectors.
The
ted
a basis
(not
two-dimensional
Warning:
lists of
three
three
Subspaces
xp,
21-2
Oias
fi
row
the
reduce
1 éaSijage
Ore
8
S10'
Ofest
eS
numbers
is to confuse ColA with Nul A.
these spaces are not known pre-
the
See
the
definition
at
b.
See
the
paragraph
before
c.
See
Theorem
12.
such
that
x,b,
+
x]:
the
augmented
matrix
[b;
ba
~
1”
10°
Gry
=
1
10
0
0
1
4a:0
tl44
5/4
0
=3etee
78 at0
aDtwn0
Xq = 5/4 and the coordinate
a.
Xa
beginning
Example
4.
vector
of
the
:
of x is
section.
[xlo =
Xobo
74
=
ale
x.
2-38
43.
Chapter
2
°¢
Matrix
d.
See
the
Warning
e.
See
Example
Algebra
after
Theorem
13.
5.
Suppose W = Span{b;,...,b,} and let {a;,...,ag} be any set in W with
To
ag].
°°:
bp] and A= [a,;
<*:
Let B= [b,;
more than p vectors.
the
that
show
we will
is linearly dependent,
{a,,...,ag}
show that
equation
Ax = 0 has
a nontrivial
solution.
a.
the columns of B span W, so for each vector aj; in W,
By hypothesis,
Note that~
there exists a vector c, of weights such that aj = Be;.
cj; is in R? because B has p columns,
and there are q vectors cj, one
for each of the a;.
b.
Let
C =
[c,
+++
the
equation
Cx
By definition
of
matrix
A=
oo ane
aq]
c.
[a,
cg].
=
Since
0 must
have
C has
q columns
a nontrivial
Linear
Algebra
p rows,
solution;
call
and
q > p,
it
u.
multiplication,
=
[ Be,
ont
Beg]
Hence, Au
(BC)u = B(Cu) = BO = 0.
tion Au = O shows that the columns of
Mastering
and
Concepts:
Seven
="
BC
Since
A are
Basic
u is nonzero,
the equalinearly dependent.
Ideas
For each of the seven key concepts
in this section,
assemble
notes that
will help you form a complete image of the idea.
For each concept,
include
as many of the following categories as possible:
> definitions
>» equivalent
Always
descriptions
> geometric interpretations
>» special cases
> examples and counterexamples
> algorithms and computations
> connections with other concepts
Finally,
basis,
strengthen
dimension,
and
could be added to the
A is nxn,
write what
your
rank,
the
definitions
example,
For
The
example, Fig. 1
zero subspace, the
Theorem
12
standard basis
Fig?2
Check the examples and exercises
For instance, Theorems 13 and 14
understanding
by using
write
For
each
of
of
column
these
Invertible Matrix Theorem.
must be true about ColA if
space,
terms
in
null
space,
a sentence
that
(For instance,
assuming
and only if A is inver-
tible.) Write as many new statements for the IMT as possible.
When you are
finished, check the table that appears on page 4-21 in this Study Guide.
eee
Chapter
MATLAB
ref
The
command
you
can
and
describe
produces
a basis
Nul A.
for
(Don’t
forget
You can use
ref(A)
extremely small pivot
A more
reliable
command
the
Supplementary
reduced
ColA or
augmented
matrix. )
MATLAB
basically the same as
ref
for rational entries in the
an
¢
Exercises
2-39
rank
ref(A)
write
2
write
that
echelon
the
form
of
homogeneous
A is a coefficient
has another
but is often
matrix.
From
that
equations
A.
that
matrix,
not
an
command,
rref
, that
works
much slower because it checks
to check the rank of A, but roundoff error or
entry can produce an incorrect echelon form.
is’
rank(A)
CHAPTER 2 SUPPLEMENTARY EXERCISES
1.
Justifications
for the answers
below.
Other justifications are
to the True/False
possible.
questions
are
given
a.
True. . If A and B are mxn,
then BE has
so AB
is defined;
also AB is defined
has m rows.
as many
because
rows as A has columns,
A has m columns and B
b.
False.
as
columns
c.
True.
of AB
d.
False.
Take the zero matrix for B.
Or, construct
a matrix B such
that the equation Bx = O has nontrivial
solutions,
and construct
C
and D so that C # D and the columns of C - D satisfy the equation Bx
= 0.
Then B(C - D) = 0 and BC = BD.
e.
False.
Praises
and
g.
B must
2 columns.
The ith row of A has
is: (0,2s...,d;,:..,0)B,
Counterexample:
cal Ace Bi
only
True.
scale
have
A has
ee
A =
with
as
B has
rows.
the form (0,...,d;,...,0).
So the ith
which istd, times ithe ith row of B.
ip!
E
0
4 and
Asc Bice Amen ABO + BAB
if A commutes
many
6510.5
C = k
ia
row
0
ne
This equals As = oso 16
B.
An nxn replacement matrix has n+1 nonzero
and swap matrices have n nonzero entries.
entries.
The
nxn
2-40
Chapter
2
°¢
Matrix
Algebra
True.
of the
The transpose
same type.
of an
True.
An nxn
True.
If A is
equivalent to I[3.
3x3
with
A must be square
A is invertible.
False.
not
elementary
matrix
by a row
operation
on
invertible,
are
Elementary matrices
But not every square
is invertible.
False.
matrices
False.
I that
is obtained
matrix
elementary
is an
matrix
elementary
three
in order
AB is invertible,
always
equal
to
A
but
pivot
to
(AB)
positions,
False.
The, correct equation
= (rr-)(AA’) = 1-1 = 1.
are
If
no
three
the
equation
Ax
=
free
variables
in
pivot
positions
(since
al;
Ba
;| =
on
from
= BA,
then
the
A
is
row
equation
AB
and this product
=
is
B
True.
Given AB = BA, left-“multiply byy, 4° to
right-multiply by A.
to have BA’
= A *B.
True.
of such
product
is invertible.
so a
matrix
conclude
Ip.
;
1
this
is
1
|0O]}
¢)
(rA)"*
has
equation,
A is 3x3).
a
get
B=
A’’BA,
and
then
= ra",
because
(rA)(r
‘A*)
unique
solution,
then
there
A must
have
which
means
By the
-1
4 :
11.
The solution is in the text.
However,
note that in practice,
interpolating polynomials are usually found from special formulas,
because the
equation Ve = y is ill-conditioned for large values of n, in the sense
that minor errors
in the entries
of V and y can greatly affect
the
solution.
18.
If
Ay
=
J
-
the
Ip,
nxn
and
hia
A’
of
1
ones,
then
Th et? Ser ton
3
A is invertible.
A satisfies
J denotes
by
that
8.
a ag ape ed
“ik Right-multiply
1
IMT,
Chapter
2
«¢
Glossary
Checklist
2-41
CHAPTER 2 GLOSSARY CHECKLIST
Check your knowledge by attempting to write definitions of the terms below.
Then compare your work with the definitions given in the text’s Glossary.
Ask
your
instructor
associative
law
basis
a
(for
which
definitions,
if any,
might
appear
A set
8 =
CVG
multiplication
of
...
of
on
a test.
of multiplication:
subspace
H of
R",
§2.9):
ots pr
unl R” such
that:
block
matrix:
block
matrix
column
space
(of
an
mxn
matrix
column
sum:
The
sum
of
....
Two
matrices
commuting
See
partitioned
multiplication:
matrices:
linear
matrix.
The
...
A,
§2.9):
A and
composition
of
transformations:
conformable
for block multiplication:
such that...
consumption
matrix:
coordinate
vector
of
vector
[xlp whose
A matrix
x
in the
relative
to
The
set
ColA
B such
that
....
A mapping
Two
...
a
model
whose
B
¢C,,...,
=
entries
(in a matrix):
Entries
having
diagonal
matrix:
A square
whose
...
entries
are
dimension
(of
....
determinant
of
distributive
elementary
final
§2.9):
The
number
A =
al
The
number
...,
denoted
by
....
that
results
by
....
(left)
An
vector
model
(or
that
...
B
.
(§2.9):
The
....
V,
laws:
and
..
space
2
A
....
..
a vector
matrix:
demand
matrix
are
{b;,...,bp},
diagonal
....
matrices
columns
cp satisfy
if
by applying
partitioned
basis
entries
produced
as
(right)
invertible
bill
of
listS
©...
matrix
final
The
demands):
vector
The
d-can
vector
d
represent
flexibility
matrix:
A matrix whose jth column gives ...
at specified points when ... is applied at .
Gauss-Seidel
An iterative method that
algorithm:
tain cases converges to a ...; based
Nie Ne WiC ie. cise
of
an
in
the
...
~...
elastic
beam
produces ... that in ceron the decomposition A =
2-42
CHAPTER
2
input-output
matrix:
See
consumption
input-output
model:
See
Leontief
intermediate
demands:
inverse
(of an
invertible
nxn
linear
that
invertible
Jacobi’s
ladder
left
Demands
matrix
matrix:
exists
or
An nxn
model.
services
matrix
A
linear
A
that
such
....
that
....
transformation
T:Ro
network:
inverse
A square
matrix
An electrical
(of A):
that
Vion:
network
Any rectangular
input-output
model
(or
.4 > where?
..8:
triangular
matrix:
A matrix
lower
triangular
part
A):
factorization:
where.
matrix
(of a matrix):
The
null
space
an
A,
outer
product:
mxn
matrix
A matrix
matrix:
A matrix
C such
of
whose
a
location
The
rank
(of a matrix
right
inverse
row-column
whose
entries
are
....
Sometimes
A matrix
such
that
..
The
rule
for
matrix
computing
=
LU,
..
of a matrix
model
A
all
v are
Any rectangular
equa-
..
u and
...
form
that
lists
C such
that
called
A in the
....
A, §2.9):
(of A):
rule:
the
where
in the
The
on....
in
Nul A of
LU factorization:
The representation
LU where: 1 is, ..j¢ andtUhis..
vector
A
set
permuted
The
....
equation):
...
matrix:
in certain
A = M - N,
....
entries
of the
lower
vector:
that
matrix
permuted
production
by connecting
production
matrix
§2.9):
product
triangular
such
....
The representation
L ‘is? ..%* and?U- is 4
diagonal
partitioned
assembled
with
A...
main
(of
R”
....
Leontief
lower
(of
Ss
....
method:
An iterative
method that produces
...
that
cases converges to a ...; based on the decomposition
With My ux
Leontief
LU
goods
transformation:
there
....
v are
matrix.
input-output
for
A):
....
u and
where
...
Q=....
where
xt+>Qx,
I, with
I or
matrix
product
A matrix
product:
Algebra
A transformation
nxn
The
matrix:
identity
Matrix
reflection:
Householder
inner
«
a product
AB
.
in which
....
form
....
A =
Schur
complement:
tioned
A certain
matrix
matrix
=
complement
complement
A
Chapter
2
formed
from
the
blocks
a 2x2
parti-
Ay,
is
invertible,
its
Schur
is
invertible,
its
Schur
[Ajj].
If
is given by ....
is given by ....
¢
If
Glossary
Ago
Checklist
of
2-43
stiffness
matrix:
The inverse of a...
matrix.
The jth column of a stiffness matrix gives ... at specified points on an elastic beam in
order to produce: ..
subspace
(of
transfer
matrix:
input
transpose
R”,
consumption
unit
lower
upper
triangular
A subset
H of
R” with
the
properties:
A matrix A associated with an electrical
and output terminals,
such that ....
(of A):
Offic
unit
Vandermonde
§2.9):
An
nxm
vector:
triangular
matrix:
An
A column
matrix:
matrix:
matrix
A...
A matrix
nxn
matrix
A’ whose
vector
...
are
the
in the
...
model
with
..
matrix
U with
V or
circuit
having
corresponding
that
lists
....
its
transpose,
of
the
form
....
...
ike
tabbinedts vieepahe iw A
LNG
mutts
a?
Kedvesieeh! 5m
231
.efdisteval
eee
\Yper
ee
freducs:
@f
ok
wi
Tey
we | Vie oS
Paphiomaieinscgz:.
Raat Bie2 aauelsaett wastbonehettam
Cputoukpalband ae 4es
mi}
interaad ate SRE
BAd SK
gol
pervs metemic ole tt mit areas|mae
Invert ibis Lineer tranaRord
fogotee
ayeal
dt
coy pethooge
iad St SET
ancl er
:
invertibie watria: 94 equare wag 4 phat.Peat
cor be 0uae
Janeyot PA,ats Loboayw,
eased
with #
ladder hetwuet:
famer
‘to pacar?
ers diverge (e “a
-
z)
:
bias
7
-a
oral
See
>=
ibe
Pastor tization:
The
pet
weiter
epade.
ale
am
{of 4 wotriu):-
lef an mn
protuct:
OK. e. ‘wait
>
7
-
ra
=.
arine
*:
U is rages
~~
Tin ooattan >
eal
a
vateie A, $2.aj... thessét LOE
A q@eirl=
prodtet
¢..
Meeee Band 7.are
*
weal
partitioned
@atrie:
perwuted
Jower
pesvasted
UT
A auttz
{riangvler
fetort
Prato
sacdh
wetria:
mation
Udtere
eretinticn
—_
hires
shot Sr
KX oalriz ouch’ | hat:
oo
~ 7 See
eo) 7
:
The rapramentt ian. oR Am ru eA,
te. .</jan
ar
entt weeare
rector
7
im
tes...
the
\
on
Ps
qs vt ten)) The
<
wer
repeewentakion. =
wore [ is
miin diagenst
ras
oh
A: matetie wth are sie eS
tof
lower telonguber part
hie
:
model [or ‘Lea
demets Crtacguler wmiede
1
et) 10 4
Any oot ang
ne aa
tequt-eutput
:
vai
pf ejectriass aes
eee Ll
Lesutief
nua
ida
a
te -
eee a
a
Sad
_
kA,
et
DETERMINANTS
3.1 INTRODUCTION TO DETERMINANTS
This section is relatively short and easy.
Some exercises provide computational practice and others allow you to discover properties of determinants
to be studied in the next section.
You will enjoy Section 3.2 more if you
finish your work on this section first.
KEY
IDEAS
anh
ae a |
The second paragraph of the section sets the stage for what follows.
Read
it quickly,
without worrying about the details of the row operations.
The
main idea is that the determinant of A is a number that appears along the
diagonal of an echelon form of A, and this number is nonzero if and only if
the matrix is invertible.
Later (on pages 189 and 199) you will understand
why this idea is important.
For now, this 3x3 case is only used to motivate the definition of det A.
Determinants are defined here via a cofactor expansion along the first
row.
Since
the cofactors
involve
determinants
of smaller
matrices,
the
definition is said to be recursive.
For each n 2 2, the determinant of an
nxn
matrix
is based on the definition
of the determinant
of an (n-1)x
(n-1)
matrix.
but
shall
we
There
not
are
digress
other
equivalent
to discuss
definitions
of
the
determinant,
them.
Study Tip:
Watch how parentheses are used in Example 2 to avoid a common
mistake.
The cofactor expansion puts a minus sign in front of ago because
Ci) = = -1.
Since ago happens to be negative,
the correct term in the
expansion
is -(-2)det Ag2,
not -2 det Ago.
3-2
CHAPTER
3
«
Determinants
SOLUTIONS
TO EXERCISES
easy
1.
By definition,
first row:
3
2
0
O
a
_ sere
Set
- als
Alea
Soi
3(-3
For
-
10)
2
OF
- 0 + 4(10
expansion
down
Zan
Pe
Ae
a
ese art
2
Os
(-1)2*2.3
|
SS
Sr Oar
Or
et
4
ee)
ory
eS
21,
1
(6s
the
along
expansion
the
3
sa
- 0) = -39
a cofactor
+ 40=1
second
column
Re
Ors
ie (-1)3t2.5
So Jise oer
yields
3
aw
4
we
Oe
To save time, omit the zero terms in a cofactor expansion,
but
to use the proper plus or minus signs with the nonzero terms.
By definition,
Using
4
6
Sia
3
Ss
0
a) =4lo
ae
the
second
2] = 3/8 4h as
Cees
column
of
re barat
analog
Ar ee = -3|5
Oey
ed
2
= =-2(18513.
ate
=1
comparison,
Study Tip:
be careful
7.
0
cofactor
a
via
computed
det A is
A instead,
18)
+.5(12
Row 2 or column 2 are the best
zeros.
We’ll use row 2.
Since
2,
the determinant
is
(-1)”~
dott
Apmalot)
— 0)
ease" Wise
= 7(8.—
0) = Oa
GO
=. 56 sud
choices because they contain the most
the only nonzero entry in that row is
~-2:-Ag,.
On 5 ee
oe
0
2 -3
ZO
= aera
pe
3
5
ee
4.
-g]
= (-2)
Sane
ae
gee
3.1
The
choice
for
Notice
is the
that the cofactor associated with
(2,2)-cofactor of the 4x4 matrix.
the
"3"
in the
5x5
determinant
matrix
det A = (-2)-(-1)7*7(3)-|5
2+2
is
use
column
to
Determinants
expand
down
the
3-3
second
the 3 in the (2,2)The original loca-
irrelevant.
2 -3
.
Beil
ae
OFe—t
Finally,
is
to
best
of
4x4
Introduction
column.
position
tion
this
¢
1 (although
(at
row
3 would
work
as
well).
>
>
(ay 6-1) eeaye lales1 as
: al1 al]
det A
-5[aca —8395
=95(6u=
5)| =U=
6(4
S15)
=76
Checkpoint:
Try to complete the following statement:
"If the kth column of
the nxn
identity matrix
is replaced by a column vector
x whose entries
are X1,...,Xpn,
then the determinant of the resulting matrix is
sm
«LO
discover the answer, compute the determinants of the following matrices:
a.
1
6)
8)
Oe
f9-det|*c
<3y)
Aw
5
ea)
Ae
iO . 0
i!
6)
wai
1
The
matrix
is
1
|0O
6)
6)
1
k
det
31.
b.
(8 ile =e
ieee
1
ye)
Oe
OG
1
l=
ad - ba,-and det|°xa
d
Interchanging two
for the 2x2 case.
25.
1
)
Ouse
On
A 3x3
row
1
6)
O;
1
use
=1°1-1=1
replacement
0.0
so
i
matrix
Oa_n0
*therserapcisertaye:
1
Até?
Os
neO
BOCStSa
we
aff
oe
OF
*OMeH7:
“l=
cb=da=—
b
rows reverses
Perhaps this
triangular,
Ci)
:
6)
(OS
:
SEO
detlec
lk
d
the sign of the determinant,
is true for larger matrices.
Theorem
Product
2.
of
has
one
of
1
Om
0
the
diagonal
the
following
.
1
entries
forms:
at
least
3-4
CHAPTER
3
«
1
0
Oy!
kin.
1
20
£0
O},
1
Determinants
rh
|0
Orr
QO:
«&
d
LletsOioe4qO
s¢O
1
Or
0,
1
30
20
k
1
so its
In each case the matrix is triangular with 1’s on the diagonal,
ie We
matrix
replacement
row
a
of
determinant
The
1.
determinant equals
matrices.
larger
for
true
is
this
Perhaps
case.
3x3
at least for the
37.
2
det A = act[7
tie
4 = 3(2)
det 5A =
41.
matrix?
a
=6
1(4)
(523)(5*2)
So det 5A # 5:detA.
and det A really is,
39.
-
-
=
-42=2.
(S-1)(524)
2x2
det
rA,
where
a.
See
the
paragraph
preceding
b.
See
the
definition
of
Try
r is any
3
6)
1
2
to
guess
(and
scalar
and
A is any
(base)(altitude)
eee)
Be
5<1
ale
verify)
a
Theorem
1.
x] = det
a
6)
= 6,
= 6,
(1,2)
det[u
and
The
det A.
bs
a
and
the
areas
(x,2):
is determined by u
3 and its altitude
base.
equals
for
v] and [ux] both equal 6.
the parallelograms determined
by u and x =
parallelogram
(x,2),
has the same
x, so the area again
formula
matrix.
preceding
on the left
Its base is
= 6.
nxn
cofactor,
parallelograms determined by [u
why the areas are equal, consider
and v =
perhaps
of
of the
To see
by u =
and x =
value of
= 50
100
definition
v] = det
parallelogram
ut+v and 0).
-
the
det[u
The
ces
me
SA =
Can you see what the true relation between det 5A
at least for this example?
What about det 5A for
any
(3,0)
150
Since
on
the
Also,
the
3:2 = 6.
and v (and the vertiis 2, so the area is
right,
altitude
determined
is
2
for
by u
any
Answer to Checkpoint:
a. 4
|
oR)
Cu go
"If the kth column of the nxn
identity matrix is replaced by a column vector x whose entries are XG
Sy
Xn, then the determinant of the resulting matrix is x,y."
Can you explain
why this is true?
You’1l learn the answer when you begin Section 3.3.
3.2
¢
Properties
of Determinants
3-5
3.2 PROPERTIES OF DETERMINANTS
This section presents the main properties of determinants,
gives an efficient method of computation,
and proves that a matrix is invertible if and
only if its determinant is nonzero.
KEY IDEAS
a
a
It is not surprising
that
row operations
relate
nicely to determinants.
After all, we found the definition of a 3x3 determinant by row reducing a
3x3 matrix.
Theorem 3 can be rephrased informally as follows:
a.
b.
Adding
change
a multiple of one
the determinant.
Interchanging
row
two
rows
(or
be
factored
(or
column)
columns)
of
of
A
to
another
A reverses
the
does
sign
of
not
the
determinant.
c.
A constant may
minant of A.
out
of
one
row
(or
column)
of
the
deter-
The other properties to learn are stated in Theorems 4, 5, and 6, together
with the boxed formula for det A on page 188. Theorem 4 is sometimes stated
as: A square matrix is nonsingular if and only if its determinant
is nonzero.
Theorems 4 and 6 will be used extensively in Chapter 5.
Your
instructor
may
or
may
not
want
property on page 191.
This property
but is not used later
in the text.
unequal to det A + det B.
SOLUTIONS
ae
1.
Rows
is
you
to
1 and
2 are
interchanged,
so
0
#2
4 igeiou F2
dl cal
Liege
>, Wien JOr—4) .2°-5Te
+
2 <3
0-4
2-5
the
determinant
1970;
a2
(Om
pte
O ls
10 20 W330" 27
O 6-30.27
that
shows
already
array
the second
Note,
because two rows are equal, as in Example 3.
9
(multi-)
linearity
courses
+ B) is
changes
sign.
1.995"
2.0%
22
eC
ec
ORIG Pee 8 aeyg
O00.
Omen
the
determinant
is
zero,
In general, computation of a 3x3 determinant by row reduction
Tip:
10 multiplications (and divisions), but cofactor expansion only takes
multiplications.
which
the
TO EXERCISES
aaa
1ue37
lc
Riva
:
TUG
feel
Study
takes
know
important
in more advanced
Warning:
in general,
det(A
requires
23
At
n
=
4,
the
multiplications,
advantage
cofactor
to
switches
expansion
40
row
(9 for
reduction,
each
3x3
3-6
CHAPTER
3
«+
Determinants
times det A;;).
as in Exercises
of a;,;
plus four multiplications
determinant,
best strategy is to combine the two techniques,
13.
2
S
4
1
2
5
4
1
4
tg
6
Apol
O}srS8i
22
6)
Zero
62-2)
—s:
U
5-4
ile
Olan
8)
6IS=—25
-6
7
»-4
U6
0
0)
in
deen
FOE
a
a
rs
sit
Gee
GO,
=3 > =2
eos)
a
b
|2d+a
2etb
gZ
h
c
2f+c|
=
oat ad
Zero
in
3
=J3~2
=
b
g
h
i
a
|
g
need
ef
=2:7
By Theorem 4 and the
and only if det[vi;
[vi
of
vo
the
v3],
third
you
=
cofactor
down
column
4
1
cofactor
down
column
1
F
ie pare ae
o
oes
2
row
factored
ou t
of
row
2
14
IMT, the set {v,,v2,v3}
is linearly independent
vo
v3] # O.
Rather than use row operations
might
choose
to
expand
the
determinant
Yah 5
vee
oe
eee @)
=
7
27 Nae Sea:
ie +
M2A)<=
determinant
+
( - 3)
StS)
Seat
is nonzero,
so
the
lik =8
=,
i iaig = I=28cr
vectors
are
>
230) == Sido =
aes
linearly
if
on
by cofactors
column:
-4
The
c
2f
2Id
of
expansion
2e
4
created
column
Result
3
|2d
i
of
expansion
5
a
created
column
Result
a
5
25.
the
Use row or column operations
whenever convenient
to create a row or
column that has only one nonzero entry.
(I recommend using only row
operations, because you already have experience with them.)
Then use a
cofactor expansion to reduce the size of the matrix.
0
19.
Often,
11—14.
independent.
3.2
¢
Properties
of
Determinants
3-7
Study Tip:
For 3x3
matrices,
some students
tend to prefer the special
trick suggested for Exercises 15—18 in Section 3.1, even though in general
there are
12 multiplications
instead of the 9 multiplications
needed for
cofactor expansion.
Note, however,
that numbers in the special method can
sometimes be large.
For comparison,
here are those computations for the
matrix studied above in Exercise 25:
7
-4
-6
-8
5
7
det [v;
id
O
%-5
vo
7
-4
-6
-8
5
7,
vg)
TS MS)" 4948)
(0) (-6) tt 7 (4) G72
=(=6)(5)(7)
175+
27.
31.
See
Theorem
See
the
paragraph
See
the
remark
following
8
See
2
On.
See the
warning
after
Since
determinant
the
O34
HO7 COOn)
(196)
7— “(ES NeA(es)
©—3(=210) ) 1 Or 160
eae
3.
following
Example
Theorem
Example
2.
4.
5.
is multiplicative
(Theorem
6),
(det A)(det A’) = det(AA") = det I = 1. So det A” = 1/det A.
Study
Tip:
The
result
of
Exercise
31
(det A)(det
33.
By Theorem
6
(twice),
det
35.
By Theorem
5,
det U' =
det U.
T
det
UU =
If. u'U =
37.
T
(detU )(det U) =
then
(det U)® =
Tip:
check
Exercises
your
computation.
rectly.
What
15—26,
knowledge
Exercise
would
be
of
39(b)
the
So,
39
come
I =
in handy
B) =
by Theorem
(det U)
det
The solution is in the text.
is the product of the entries
Study
they
I,
AB =
might
on
a test.
(det B)(det
A) = det BA.
6,
2
1,
which
implies
that
detU
=
+ 1.
(The determinant of a
on the main diagonal.)
triangular
matrix
and
questions
because
40
make
good
test
determinant
properties
without
requiring
is the
most
to be
answered
answer
one
to 39(b)
likely
if A were
4x4?
much
incor-
3-8
43.
CHAPTER
3
Determinants
Compute
det A by a cofactor
det A =
(u,+v,)-det
Aig
expansion
down
(ug + Vo) -det Ao3
-
column
+
3:
(ug + v3) -det A33
u;, ‘det A1;3 -
Uug:det Agg
+
ug’det
Ag3g +
vi:det
Ay3
-
Vo:det
Ag3
+
v3:det
A33
-
Uug:det Bog
+
u3'det
B33
+
vi °det
Cig
-
V2:det
Coz
+
v3:det
C33
u, ‘det
By3
det B + det C
45.
Suppose A is mxn with more columns than rows.
Then AA is
must be singular.
If A is generated
with random entries,
will
be nonsingular
(invertible)
practically
all
the
time.
explain
MATLAB
why these
Computing
statements
should
be true.
(Use
the
nxn and
then AA
Try «to
IMT.)
Determinants
To compute detA, set
U = A
and then repeatedly use the commands
U =
gauss(U,r)
and
U = swap(U,r,s)
as needed to reduce A to an echelon
form U.
Then,
except for a + or - sign (depending on how many times
you swap rows), the determinant of A is given by the command
prod(diag(U) )
The command
diag(U)
extracts
the diagonal
entries
of U and places
them
in a column
vector,
and_
prod
computes
the product
of those
entries.
You can also use
det(A)
to check your work, but the longer
sequence
of commands
helps you think about
the process
of computing
det A.
3.3 CRAMER’S RULE, VOLUME, AND LINEAR TRANSFORMATIONS
This section will be a valuable reference for students who plan to take a
course in multivariable calculus.
Mathematics and statistics majors probably will encounter the material here several times.
Also, economics students and engineers
(particularly electrical
engineers)
are likely to need
Cramer’s rule and some of the supplementary exercises in later courses.
3.3
*¢
Cramer’s
Rule,
Volume,
and
Linear
Transformations
3-9
KEY IDEAS
eS
BS
The main results of the section are stated in Theorems 7, 8, 9, and 10.
The proof of Theorem 7 is simple and yet involves three important
ideas:
the definition
of a matrix product,
the multiplicative
property of the
determinant,
and the evaluation of a determinant by cofactors.
Check with
your instructor about whether you should be able to reproduce the proof of
Theorem 7.
A heuristic proof of Theorem 9 for 2x2 matrices is given in an appendix at the end of this section.
Theorem 10 provides a key idea in calculus
and physics
matrix used
STUDY
NOTE
ee
an.
needed
for
the study of
there is called a Jacobian.
double
and
triple
integrals.
The
2
In Exercise 25, you are asked to use Theorem 9 to explain why the determinant of a 3x3
matrix A is zero
if and only if A is not invertible.
(A
similar explanation holds for the 2x2 case.)
The answer
is in the text,
so be sure to work on this before looking at the answer section.
Work on
Exercise 25, even if it is not-assizgned?Remember,
learning
does
take
place
when
you
think
hard
about
an
exercise,
even when you are unsuccessful,
if you try to look at the problem
from different
angles,
browse back through the text,
and perhaps
look at
earlier exercises.
Write your solution,
don’t just talk to yourself about
what you would write if you had to.
SOLUTIONS
ea
1.
The
TO EXERCISES
eae
system
is equivalent
to
Ax = b,
where
A =
Ee
2
4 and
b =
al
1
Write
a0
47
Ade), = F
At
ALBEEelsE ee|
aN
b
and
nN
b
compute
det A = 20 -
aude
Xi 2
14 = 6,
As(b)
“det A
5
a
detA,(b)
2
=
det Agtb) ol
det A
det An(b)
12 - 7 = 5,
6
og
6
= 5 - 6 = -1
3-10
CHAPTER
The
3
Determinants
A =
to Ax = b, where
is equivalent
system
6s
E
4
o and
b =
5
Esp
Write
A,(b)
and
=
5
4
3
sal:
"Tes"
A2(b)
=
§
ie
aH
compute
det A = 12s - 36 = 12(s* - 3) = 12(s - V3)(s + V3)
det A,(b)
The
For
=
10s
+ 8,
det Ap(b)
= -12s - 45
resedetitAy(
ae
brie AOS
Sr
ee
debut
12 (sh
toart
Ssat
13.
First,
amdet. Aw.
find
5
rf)
arrange
=
- [2
$}
4
| =e-1;
C22
oe
2
| =
4
if=
C32. [05
3
1
4
| =iL5
the
-1
adjA=|1
1
check,
1
cw
4
st
Were
need
al
-1,
5
=
Then,
of A.
5
8)
1
| =
Cot.
as23)
3
}1
2
1
-[f
2
A=
1
=
ACs
of
0
ca
‘a
12(Csyaces)
cofactors
Ci
4
-4s - 15
2
the
5,
s # # v3.
Bh) WE Gs 8S)
_ det Ag(b) _ -12s - 45
a2)
when
is,
system has a unique solution when det A # 0, that
such a system, the solution is x = (x,,x2), where
transpose
of
the
4
Teh
1
i|=
array
1
8)
1,
CIBte in
2
a|=
3
5
=o;
ca
[2
‘|=
7
=
-5
=
Cag) =>
of cofactors
-
3
1
into
the
2
adjugate
-1
5
-5
1
7-5
you to compute det A now, you could write i
but you would still
to check whether your calculations are correct.
To build in this
compute
3.3
¢
Cramer’s
Rule,
Volume,
and
Bray
Sah ieiera1——
5
|
on0 SThl) 1) esis
ee
TOE)
87 Ba
Acadj A=
Linear
aye
leSnl-e"e
a)
Transformations
3-11
S07 ape
Bend0
Gy
If any
off-diagonal
entries
in the
product
are
nonzero,
or
if the
diagonal entries are not all the same,
then some errors have been made,
and you can recheck your cofactor calculations.
(One possible mistake
is to
error
forget the + signs
is to not transpose
calculations
-1
19.
The
above
are
in front of the 2x2 determinants.
the array of cofactors.)
In this
correct
-1
1
1
—adjA= =| 1
det A
6
1
=
parallelogram
with
and
-1
-5
7
det A must
be
6.
Another
case,
the
So
5
1
25
vertices
(0,0),
(5,2),
(6,4),
(11,6)
is
shown
below.
If no vertex were zero,
we would have to translate the parallelogram to the origin by subtracting one vertex from all four vertices.
Since one vertex already is zero,
use the two vertices adjacent
to the origin to construct the columns of A, and compute
|det A|.
aad
te
fs:
6
2
At
the
-
eee
area
of
2
een
25.
The
answer
is
in
the
text.
a
1
Ce
©
ee
ion
GleieS
10
I hope
you
took
ees
X41
the
advice
at
the
beginning
of this Study Guide section and worked the problem (or at least tried
hard to work the problem) before checking the answer section.
If you
were successful,
you should be proud of yourself;
you are mastering the
material —not only determinants but also linear dependence!
x4
31.
Let
x =
uy
|xXo/,
u =
Jug],
X3
in
ball
u;
+
ub
Au.
Ug
A =
U3
R®, whose
+
and
us =
bounding
surface
and
S’
1,
let
a
6)
@)
|0
b
O}.
0)
0)
Cc
be
consists
the
of
image
Also,
let
all
of
S
S$ be
the
unit
vectors
u
such
that
under
the
mapping
3-12
CHAPTER
3
Determinants
If
in
S’,
x
is
then
x
=
Au
for
some
u
in
shows
that
B
ball
S,
and
=
2
(F2] =
2
The
2s
condition
Since
the
Appendix:
u;,,u2g,u3
volume
of
the
unit
of
is
abc,
Theorem
determinant
region
on
bounded
A Geometric
A
bounded
by
shows
that
10
u
=
A’x
=
2
[22] =
1.
S
is
the
by S’ is 4nabc/3.
Proof
of
a Determinant
Property
a
d
b
a
|
a
2]
det 2
bint
A =
ad
&
a
b
(a + b)(e
-
2
+ d)
Cc
=
=
ac
+ ad + be + bd
-
2bc
b
(Area
of the Parallelogram)
=
ad - be
-
bd
-
be
Xo/b].
4mn/3
and
the
volume
of
the
Chapter
3
*
Supplementary
Exercises
3-13
CHAPTER 3 SUPPLEMENTARY EXERCISES
1.
Justifications
Other
below.
a.
for
the
answers
justifications
True.
If det A = 0,
Since there are only
the other.
of
zeros,
in
False.
If
A is
False.
:
Consider
False.
If
A
which
3x3,
is
then
and
False,
by Theorem
3(b).
True,
by Theorem
3(c).
True,
by using
False,
by Theorem
False.
is an
The
even
Theorem
statement
True,
because
5.
True/False
are
given
of A are linearly dependent.
column must be a multiple of
Section
reduced
3.2.)
If the rows
of A are
echelon form of A will have a
det
A = 0, by Theorem 4.
det 5A =
5° det A,
6)
ein
Ae B=
k
detA =
2,
¢)
a
then
by Theorem
and
3(c).
A+
wa
B=
t3
k
oe.
by
Theorem
det A° =
¢)
at
6.
|
3(a)
several
times.
false
for
by Theorem
is
3(c).
nxn
invertible
matrices
when
n
det A’A = (det A')(det A) = (det A)* = 0, by Theorems
False.
Cramer’s
rule
applies
invertible coefficient matrix.
only
False.
The
area
of
mentioned
True.
By Theorem
6,
False.
The
True.
questions
5.
integer,
6eand™
case
tale
A = E
nxn
the
possible.
then the columns
two columns,
one
True.
(See Exercise
30 in
linearly dependent,
then the
row
to
are
correct
the
triangle
to
an
nxn
is
given
with
is 5.
(det A)° = detA’ = det 0 = 0.
formula
system
in Exercise
Hence
31
of
det A = 0.
Section
By Theorem 6, (det A)(det A+) = det(AA*) = det I = 1.
3.2.
an
3-14
CHAPTER
3
¢-
Determinants
CHAPTER 3 GLOSSARY CHECKLIST
Check your knowledge by attempting to write definitions of the terms below.
Then compare your work with the definitions given in the text’s Glossary.
Ask your instructor which definitions,
if any, might appear on a test.
adjugate
(or
classical
matrix
cofactor:
A _number,C;;
is
cofactor
Rule:
interchange
row
the
=
row
A
or
A formula
(matrix):
An
by r (matrix):
for
each
the
as
entry
matrix
elementary
An elementary
for
in
matrix
formed
from
a
square
of A by ....
(Ci, j)-cofactor
lor “A,
where
Ar,
....
det A using
such
elementary
An
adjA
by deleting
for
column,
matrix
(i, j)-entry
called
formed
formula
one
replacement (matrix):
Matrix iby. i).
scale
The
the
.....,
submatrix
expansion:
one
Cramer’s
adjoint):
A by replacing
:
cofactors
row
associated
with
1:
..
obtained
matrix
by
obtained
obtained
interchanging
from
the
by multiplying
.
identity
....
VECTOR
SPACES
4.1 VECTOR SPACES AND SUBSPACES
The main focus of the chapter is on R" and its subspaces.
However, Section
4.1 builds a framework within which the theory for R" rests.
Most of the
exercises
in the chapter concern subspaces of R", but some are designed to
help you learn gradually about other important vector spaces.
KEY
IDEAS
eae
|
A vector space is any collection of objects that behave as vectors do in
R’.
(The precise meaning of "behave"
is described by the axioms on page
Zit)
A vector
is simply
any object
that
belongs
to a vector
space.
Lists of numbers,
arrows,
and polynomials are all examples of vectors,
in
different vector spaces.
The most
important
vector
spaces
in this
text
are
subspaces
of R™.
Visualize
subspaces as lines or planes
through the origin,
the origin by
itself, or the entire space R".
To show that a set is a vector space, use
Theorem 1.
To show that a set is not a subspace, show that one of the pro-
perties
in the subspace
definition
is violated.
(See Exercises
1-4.)
STUDY NOTES
Seen
= 4s
Parts
of
this
chapter
are
somewhat
more
theoretical
than
the
earlier
chap-
ters,
but that
is necessary
in order to give a solid foundation for the
rest of the course.
Learn the key definitions and theorems as they appear
in the text (rather than waiting until just before an exam).
You need this
knowledge
later
to
get
through
the
conceptual
exercises
and
to
be
prepared
for
sections.
4-1
4-2
CHAPTER
4
«+
Vector
Spaces
ina
"vector"
of a function as a single
In Example 5, the concept
not
should
you
and
reading,
first
a
on
absorb
to
difficult
is
space
vector
expect to master it in a few days.
The word subspace should usually be accompanied by a phrase such as
"of V", or “of R'". Without such a phrase, the nature of the elements in
the subspace is unknown. For instance, the statement "H is a subspace" does
not
tell
us
the
whether
in H are
elements
pairs
of
or
numbers,
polynomials,
or something else. But the phrase "H is a subspace of P3" includes the information that each vector in H is a polynomial of degree three or less.
Set Notation:
The notation introduced
in Example 8 is sometimes
used in.
this chapter
as an efficient
way to describe
a set.
The symbols
and
phrases inside the set brackets describe the set.
The part to the left of
the colon displays the basic nature of the elements
in the set
(such as
vectors in R ) while the part to the right adds any qualifying conditions
that must be met in order for an element to belong to the set (such as all
vector entries must be positive).
For instance,
the set in Example 8 can
also be written as
H =
As
another
W =
t}:
u
s and
Tip:
real,
in
example,
the
set
a
b|
5b
+ 2c,
: a =
Here, and elsewhere,
that the scalars can
Study
t are
Review
and
Exercise
where
u =0
11
b and
can
be
c are
arbitrary
reference to the scalars
be any real numbers.
Sections
1.3
to
1.6
before
written
b and
you
c as
reach
as
"arbitrary"
Section
means
4.3.
SOLUTIONS
TO EXERCISES
ee]
1.
a.
V is a subset of
of every vector
b.
The text’s solution gives a specific u and c.
One specific “counterexample" suffices to show that V is not a vector space.
However,
you could also simply say,
"If any nonzero vector v in V is multiplied by a negative scalar c, then cv is not in V because at least
one of its entries is negative."
R?.
The defining property of V is that the entries
in V are nonnegative.
50,
1f UW and ‘Vo are
qin Vv.
their entries are nonnegative.
Since a sum of nonnegative numbers
is nonnegative,
the
vector
u + v satisfies
the
condition
that
defines V.
That is, u + v is in V.
4.1
7.
Most
examples
in
ple,
and
can
closed
one
under
Checkpoint:
condition
13.
integers,
to
the
fact
that
by all
real
numbers.
the
z
=0.
certainly
not
wis
b.
Span{v,,v2,v3}
w
involve
overlook
Let, H be
-
is
in
the
set
There
2
1
3
is
ar
fee
Ofn
no
of
three
vectors
and
infinitely
(of
many
R®) spanned
i
48)
OL
2
ih
See
gtPete i)
a
SW
Onna
augmented
4-3
calculations
sim-
in
must
be
satisfy
the
subspace
in R® that
{Vv},
Vo, V3}.
by vi,V2,v3
is
Thinalion
a
Subspaces
vectors.
w
in the
w is
the
and
make
(x,y,z)
of R?
+ xX3vV3 =
matrix:
*3
th
ue
pivot
points
subspace
contains
subspace
consistent,
of all
Is Ha
one
equation xX;Vv, + XoVe2
reduce the augmented
1
0
=a
Spaces
easily
+ y
x
Vector
text
multiplication
a.
c.
the
¢
consistent
hpea ion|
column,
if and
(has
a
if
the
solution).
only
Row
1
2
KC)
1
00
4
2
0
so
vector
the
3
L,
0
equation
is
in Span{v;
vo, v3}.
19.
Let H be the set of all functions described in (5).
Then H is a subset
of the vector space V of all real-valued functions,
and H consists of
all linear combinations of the functions coswt and sinwt.
By Theorem
1, H is a subspace of V, and hence is a vector space.
23.
a.
See Example 5.
This is
and justification before
this section.
b.
See
the
See
Exercises
See
the
See
Example
e.
definition
1,
answer
end of
of a vector.
2,
paragraph
an important problem,
so write your
you read a note I have added at the
or
3.
before
Example
6.
3.
25.
Axiom 4 (plus Axiom 2) shows that 0 + w=w.
Exercises 25—30 show how
facts that we take for granted in R™ depend only on a few basic properties of R’, properties that are now axioms for a general vector space.
31.
Let
is
H be
contains
subspace
under
closed
multiples
all
a
of u and
all
vectors
sums
of
V that
multiplication
contains
by
the
scalars,
v.
Since
H is also
closed
of
scalar
multiples
of
in Span{u,v}.
vectors
H
u and
must
under
v.
u
and
v.
vector
That
Since
all
contain
addition,
is,
H
scalar
it
H contains
4-4
CHAPTER
32.
(Comments,
4
°¢«
not
Vector
Spaces
If
solution)
the
subspaces
K are
H and
of
a vector
space
V, then H n K is the largest subspace of V that is contained in both H
is the smallest
not mentioned in the text,
A related subspace,
and K.
is denoted by
subspace
This
K.
and
H
subspace of V that contains both
in the
written
be
can
that
V
in
H + K, and it consists of all vectors v
in K.
vector
some
is
k
and
H
in
v = h +k, where h is some vector
form
In set
notation,
H + K = {h + k:h
If Exercise
where along
is
in HW and
32 is assigned as
the way you might
k is
in K}
homework,
encounter
there
is a chance
the subspace
H +
that someK.
(Guess
where! )
23.
Note for part (a):
I included this True/False question to point out a
mistake that many students make in their writing.
The statement in (a)
is false because the equation f(t) = 0 is not accompanied by the phrase
“for-allmt’ sor “for all t "in R."
The Zero vector in a) space lof weal
valued
that
functions
has
An
the
is
value
equation
not
the
number
O but
zero
for
all
t in
as
f(t)
=
O must
such
its
is
instead
a
accompanied
by
be
specific phrase that explains what the equation "f(t) =
For instance,
t might represent a specific known number
f(t) = 0, or t might
by solving f(t) = 0.
Answer
to
Checkpoint:
H
represent
is
not
an unknown
a
subspace.
function
f(t)
domain.
number
a
or numbers
Counterexample:
context
or
O" represents.
that satisfies
to be found
(1,0,1)
and
(9,1, 1) are in H and yet their sum,
(1,1,2),
is not in H (because 1°
1° - 2 #0).
Many counterexamples are possible.
Only one is needed.
+
4.2 NULL SPACES, COLUMN SPACES, AND LINEAR TRANSFORMATIONS
Many problems in linear algebra involve a subspace in one way or
This section provides an opportunity to become comfortable
with
cept.
The foundation for this section was laid in Sections
1.3
Have you reviewed those sections yet?
KEY
another.
the conand 1.5.
IDEAS
Se aR
ooh
Theorems 2 and 3 describe the main
rem 2 makes a good exam question,
Nul A and
the definition
types of subspaces.
(The proof of Theobecause it tests both the definition of
of a subspace.)
4.2
¢
Theorem
The
first
Null
Spaces,
3 actually
phrase
remain in Col A.
entries (because
Column
has
tells
two
us
Spaces,
and
conclusions:
that
linear
Linear
Transformations
4-5
Col A is a subspace of R™.
st
ee es
combinations
of
vectors
The phrase "of R"" reminds us that each
A similar remark applies to
A has mrows).
in
ColA
vector has m
the statement
Theorem 2 that Nul A is a subspace of R".
"Col A = R"" can
The box after Example 4 shows that the statement
added to the list of equivalent statements in Theorem 4 of Section 1.4.
from
be
STUDY NOTES
OE ees |
In Example 3, the statement
"Nul A = Span{u,v,w}" would be an explicit
description of Nul A (provided you specify what u, v, and w are).
In some applications,
it is important
to know that for a given mxn
matrix A, every equation Ax = b has a solution (assuming b is in Ro)
vet
it may require some effort to determine whether this is the case.
If not
every equation Ax = b has a solution,
then not every b belongs
to Col A,
and hence Col A is a proper subspace of R". One of the goals of the next
few sections is to obtain a method for determining when Col A = R™.
Checkpoint
1:
Col A = R"?
How
many
pivot
positions
does
an
mxn
matrix
A
have
if
Study Tip: Theorems 1, 2, and 3 are the main tools for showing that a set
is a vector space (that is, a subspace of some known vector space).
Review
these theorems now, before starting the exercises.
SOLUTIONS
TO EXERCISES
SS
1.
Now is the time to learn the definition of NulA.
Nul A precisely when the product Ax is defined and
simply compute Ax to determine whether Ax is zero.
Ax
=
oC i
6
-2
-8
4
Jet|
0
1)
1
0
3-="1-0) |
|-4
8)
«so
il
3]
-4
is
in
A vector x is
Ax = 0.
Given
in
x,
Nul A.
In Exercises 3—6, writing an equation x = cu + dv is not the same
Warning:
The approas listing, say, the vectors u and v that span the null space.
is a list of a small finite number of
priate answer for these exercises
vectors
that
Study Tip:
you can’t
span
NulA, not
a description
of
all
the
vectors
in
Try Practice Problem 1 before you work on Exercises
reread
find two ways to work the practice problem,
Nul A.
If
7-14.
the first
4-6
CHAPTER
4
«+
Vector
4.2
Section
paragraph
of
practice
problem).
Spaces
until
it
at
look
don’t
(but
the
attempted
have
you
7.
the
(under
space
If W were a vector
of R®.
The set W is a subset
fails
W
But
R.
standard operations in R’), it would be a subspace of
For
space.
a vector
it is not
so
of a subspace,
property
every
a+b+c#=
the vector (0,0,0) does not satisfy the condition
instance,
2, and so the zero vector is not in W.
13.
A typical
1-2-8
element
be written
as
u
Vv
B ae
follows:
Since c and d,are any real numbers,
this
subspace of R
(and hence a vector space)
calculation shows that Wis a
by Theorem 3.
Alternatively,
this calculation shows that
space of R_, by Theorem 1.
as
Checkpoint
19.
of W can
2:
Why
is Wa
W
is
the
subspace
same
Exercises
25.
the
Check
17—20
definition
may
seem
Example
Example
Theorem
See
the
remark
just
before
See
the
table
that
contrasts
see
Fi
gre:
NulA
Thus
simple,
before
See
so
W
is
a
sub-
must
NulA
5 entries
and
subspace of R
"of R°"?
The matrix
A is 2x5,
so vectors
in
vectors
in ColA must have 2 entries.
and Col A is a subspace of R.
Study Tip:
Section 4.5.
Span{u,v},
but
have
is a
they
will
help
you
in
1.
2.
4.
Nul A and
Col A.
so
aoe
p See the remark after Theorem 3.
31.
The
solution
in
the
text
shows
that
T
is
a
linear
transformation.
1a:
T(p) is the zero vector, then p(0) = 0 and p(1) = 0, by definition of T.
One such polynomial is p(t) = t(t - 1).
Any other quadratic polynomial
that vanishes
Of r.
at
O and
1 must
be
a multiple
of
p,
so
p spans
the
kernel
4.2
*
For
;
tion
Null
the
Spaces,
range
1
HE and
is
of
Column
T,
the
Spaces,
and
Linear
that
the
image
observe
image
of
the
polynomial
Transformations
of
t is
the
constant
i|: Denote
4-7
1 func-
these
two
images by u and v, respectively.
Since the range of T is a subspace of
R that contains u and v, the range must contain all linear combinations
of u and
span RR’.
37.
2.
so
they
The
[A
not
vector w is in Col A because row reduction of the augmented matrix
wl] shows that the equation Ax = w is consistent.
The vector w is
in Nul A because Ax is not the zero vector.
(The first printing of
the
text
Answers
1.
v.
By inspection,
u and v are linearly independent,
Thus the range of T must contain all of R’*.
to
an error:
An
mxn
of
(1,1)-entry
have
m
in A should
be 7,
not
-7.)
matrix
A must
pivot
positions
in
order
for
Col A to
three
entries.
be
R".
a subspace
Mastering
the
Checkpoints:
all
W is
has
Linear
"of
Algebra
R® "because
each
vector
Concepts:
Subspace
in W has
You will need a clear mental image of a subspace throughout the rest of the
text.
Organize what you have learned
in Sections
4.1 and 4.2
(together
with Sections
1.3—1.5)
on a review sheet that covers the following categories:
> definition
> equivalent
> geometric
> subspaces
subspaces
>
description
interpretations
defined explicitly
defined implicitly
special
cases
> examples
and
counterexamples
> algorithms
and
> connections
with
The
references
in
computations
other
the
more facts and examples
eigenen
Ran!
concepts
right
column
Page 214
Paragraph
before Example 6 in Sec. 4.1
Figssp- 6,709 AngSeomm 44
Theorems 1, 3; Exercises 15-18 in Sec. 4.1
Theorem 2; Exercise 20(b), 22 in Sec. 4.1
Theorems
1,
Examples
and
Exercises
7-14
Fig. 2 and
above
are
not
2;
Fig.
2
exercises
in Sec.
Example
in
Sec.
4.2
in Sec.
4.1,
4.2
4.2
8 in Sec.
exhaustive.
4.2
You
in the exercises for Sections 4.1 and 4.2.
eS
Eee
Se
et a ee
can
find
eS
4-8
CHAPTER
4
«+
Vector
Spaces
4.3 LINEARLY INDEPENDENT SETS; BASES
The definition of linear independence carries over from R" to any vector
And the geometric interpretations in Chapter 1 of linearly indepenspace.
dent and dependent sets should help you visualize these concepts here.
KEY
IDEAS
:
In general, you cannot use an ordinary matrix equation Ax = 0 to study linear dependence of {v1,...,Vp+.
You have to work with the vector equation
Civ, + *** + CpVp = 0, or with Theorem 4, unless the vectors happen to be
n-tuples
of
A set
tion
x;v
4.1.)
numbers.
{v}
=
O
The
with
has
set
v # 0 is
only
{0}
is
the
linearly
trivial
linearly
independent
solution.
dependent,
because
(See
because
the
Exercise
the
vector
30
equation
in
equa-
Section
x,0 = O has
many
nontrivial solutions.
Theorem 6 is important for later work, but its proof is rather subtle.
Study Examples 8 and 9 carefully, as well as the proof of the theoren.
A basis for a vector space V is a set in V that is large enough to span
V and small enough to be linearly independent.
See also the subsection Two
Views of a Basis.
The plural of basis is bases.
Warning:
dependent.
The
Theorem,
If a set
table
at
in V does
below
the
bottom
adds
not
span
V,
the
three
new
statements
of
the
list
from
STATEMENTS
FROM
THE
INVERTIBLE
set
Section
may
to
or
may
the
not
be
linearly
Invertible
2.3
in this
MATRIX
THEOREM
Study
Matrix
Guide.
Equivalent statements,
for any nxp matrix A.
Equivalent only for an
nxn square matrix A.
Equivalent statements,
for any nxp matrix A.
k.
There is a matrix
D such that AD = I.
a.
A is an
matrix.
Jj. There is a matrix
C such that CA = I.
*.
A has a pivot
tion in every
posirow.
c.
A has n pivot
positions.
*.
A has a pivot
tion in every
h.
The columns
span R.
A
b.
A is row equivalent
to the nxn
identity matrix.
e.
The columns of A
form a linearly
independent set.
of
invertible
posicolumn
g.The
equation
Ax = b
has at least
solution for
one
each
*.
4.3
¢
The
equation
b
The
linear
R'
trans-
*.
each
AX
The linear transformation xh Ax
onto
R .
is
b
f.
d.
elementary
is an
The
equation
has
only
trivial
invertible
*.
matrix.
SOLUTIONS
A
b
is one-to-one.
matrices.
*. Col A = R"
Ax = b
The linear transformation xb Ax
invertible.
A
equation
4-9
has at most one
solution for each
*,. A is a product
it
Bases
in R’.
xr
of
Sets;
*,. The
in R”.
SAS
maps
Independent
Ax = b
has a unique
solution for
in R’.
i.
Linearly
The
Ax = 0
the
solution.
equation
Ax = 0
has no free
variables.
*. The columns of A
*.
Nul A = {0}
TO EXERCISES
1.
The complete solution is in the text.
in R', row operations on a matrix will
if the matrix has n pivot positions.
7.
Again,
the solution is in the text.
Any set in R" with fewer than n
vectors cannot span R° and therefore cannot be a basis for R'.
Such a
set may or may not be linearly independent.
What similar statement can
you make about a set in R" with more than n vectors?
See Exercise 8
for ideas.
Study Tip:
Theorem
vectors spans R®.
4
13.
in echelon
The
for
the
matrix
B is
Col A consists
only
choice
For
correct
would be
the null
[AmeOle-
[BU
in
of
Section
form
columns
choice,
1.4
may
and
it
is
columns 1 and 2 of B.
space, solve Ax = 0:
Ol-=
1
10.
0
ve
oe
help
you
displays
1 and
but
For a general set
usually be needed
-3
decide
the
pivot
ar,
Kol
the
=o
"standard"
See
0
0)
6)
the
Warning
1
16
OF
0
1
a
set
columns.
4
A:
n vectors
determine
whether
=.
2 of
of
to
A basis
- Ths*1
Seno
8
choice.
A
after
Theorem
6
DA 4
0
S
poses
0
of
6)
U
0
wrong
6.
4-10
CHAPTER
4
°¢
=
-
6x3
x,
Then
The
general
Vector
5X4,
-
solution
X41
Spaces
|xp]
_
aie:
XB
ii
=
=6
describes
avoids
a
basis
=
6)
pe
1
a
1
0
X4
equation
xq
-3/2
_ , |-5/2
X3
NulA,
in
vectors
all
"standard"
basis,
Another ‘choice is
a
For
Nul A.
(-5; =372; 0,1).
which
5X4
|-(5/2)x3 - (3/2)xa]
X4
This
free.
and
is
-6x3
_
xg
with
(3/2)x4,
-
-(5/2)xg
=
Xq
just
not
for
1, 0O) and
-5/2,
(-6,
is
choice
(-12, -5, 2,0) and (-10,"-3,00,,2),0
fractions.
Warning:
You really need to know the definition of Nul A and the definition of Col A, not just the procedures for finding bases for these spaces.
The definitions will help you avoid the fairly common mistake of attempting
to use the null space procedure
to find a basis for a column space,
or
vice-versa.
19.
21.
23.
We can solve the equation 4v, + 5vo - 3v3 = O for any one of the three
vectors in terms of the others.
By the Spanning Set Theorem,
the set
spanned by all three vectors is the same as the set spanned by any two
of the vectors—any one of the three vectors can be discarded.
If we
discard v3, then v,; and vj span H and are obviously linearly independent.
Hence {v;,v2} is a basis for H.
The same reasoning applies to
{vi, V3} and
{vo, v3}.
a.
See
the
paragraph
preceding
b.
See
the
definition
of a basis.
d.
See
the
subsection
Two
e.
See
the
box
Example
Let
the
A=
[v,;
v2
vg
columns of A must
Theorem.
Checkpoint:
before
Views
Explain
{vi,...,Vn}
why
4.
c.
See
Example
3.
of a Basis.
Q.
va].
Since
be linearly
So {vj,vVo2,v3,v4}
Suppose
Theorem
A is square
independent,
is a basis
is
for R.
a basis
{Av;,...,Av,}
and its columns span R*,
by the Invertible Matrix
nxn
matrix.
25.
The displayed equation shows only that
fact,
the vectors vj,V2,v3 are not all
for
R" and
is a basis
for
A is
an
invertible
R”.
Span {v,,v2,v3}
contains
in H, so Span{vj,v2,v3}
be H.
Therefore {v1,v2,v3} cannot bea basis
check that {v,,v2,v3} is a basis for R°.)
for
H.
(It
is
H.
In
cannot
easy
to
4.3
31.
(This
generalizes
linearly
C1V1
Then,
Exercise
dependent.
+
***
since
Di Cava
+
Covp
T is
gt
Then
=
°
31
Linearly
Independent
in Section
there
exist
= T(O)
= 0
1.7.)
c),... »Cp,
Sets;
Suppose
not
all
Bases
4-11
{vi,...,Vpt
zero,
such
is
that
O
linear,
prt
CpVp)
+
-:-
+ epT (vp) = 0
all
the
cy are
See
the
boxed
statement
on
page
71.
and
C1T (vy)
Since
not
zero,
{T(v,),...,T(vp)}
is
linearly
dependent.
Study Tip:
The solution of Exercise 31 illustrates how to use linear
pendence:
|
SP arta »Vp} is known to be linearly dependent,
then you
write civ; + °°: + Cpvp = 0, assume that not all the cy, are zero,
and
this equation in some way.
decan
use
Answer to Checkpoint:
Let B= [v,
-:: v,].
Then B is invertible because
{v,,...,Vp} is a basis for R’.
If A is an invertible nxn matrix, then so
is AB; hence the columns of AB, namely Avj,..., Avy, form a basis for R™.
Mastering
Linear
Algebra
Concepts:
Basis
To the review sheet(s) you have on linear independence,
add Examples 1, 2,
and 6 from this section.
The definition and geometric interpretations are
unchanged.
Add a note about not using the matrix equation Ax = O in the
general case.
Start
other
a separate
concepts
(span
review
and
sheet
linear
for
"basis",
independence)
> definition
> geometric interpretation
Page 232
Fig.1
> special
Standard
cases
> examples and counterexamples
> algorithms and computations
> connections with other concepts
even
though
already
being
bases
involves
two
reviewed.
for R” and Pp
Example 10, Exercise 25
Examples 7 and 9; Example
Invertible Matrix Theorem
Unique
it
Representation
3 in Sec.
Theorem
(in
Sec.
4.2
4.4)
4-12
CHAPTER
MATLAB
The
you
ref
4
°+
and
cos
command
can
write
describe
Vector
Spaces
produces
ref(A)
a basis
for
NulA. (Don’t
the
reduced
Col A or
forget
that
A
echelon
the
write
is
form
of
homogeneous
a coefficient
From
that
equations
that
A.
matrix,
not
an
works
, that
rref
command,
has another
MATLAB
matrix.)
augmented
2-39.
page
See
slower.
much
is
often
but
ref
as
same
basically the
roundoff error or an extremely small pivot entry can
In some cases,
The more reliable
an incorrect echelon form.
produce
to
ref
cause
for
bases
produce
can
7.4)
Section
(see
decomposition
value
singular
Col A and Nul A, but
ref
is satisfactory for our purposes.
For Exercise 34: If t is a vector and k is a positive integer,
then
cos(t).*k
is a vector
the same
size
as t, formed
entrywise
from
t
using
the
function
cos*(
).
4.4 COORDINATE SYSTEMS
This section contains a variety of geometric and algebraic
the idea of a coordinate system for a vector space.
KEY
explanations
of
IDEAS
i ea
ae
The coordinate mapping from a vector space V (with a basis of n elements)
onto R" is a rule for giving "R'-names" to vectors in V in such a way that
the vector space structure of V is still visible in R’.
Every vector space
calculation in V is precisely mirrored by the same calculation in R’.
An important special case is when V is itself R", and each vector x and
its coordinate vector [xlp are related by a matrix equation x = Palxla Everything in the section depends on the Unique Representation Theorem.
The
proof
of
that
theorem
could
appear
on
an
exam
because
it
shows
precisely why the two properties of a basis B are important,
and it illustrates how linear independence can be used in an argument:
Any vector in V has "coordinates" because
nates are uniquely determined because B is
B spans V, and the coordilinearly independent.
If you are asked to prove the theorem,
make sure your proof shows exactly
where each property of a basis is needed in the proof.
Also,
be careful
not to use a matrix in the proof.
The vectors vj,...,V, cannot be arranged
as the columns of an ordinary matrix when the vectors are in some abstract
vector space.
4.4
+
Coordinate
Systems
4-13
Checkpoint:
Let B = {b,,...,b,} be a basis for R’.
Apply the Invertible
Matrix Theorem to the matrix A = [b,
--+
b,] and deduce the Unique Representation
STUDY
Theorem
for
the
case
when
V = R”.
NOTES
=
a
oe
|
Be
to
careful
ral,
even
distinguish
if x itself
between
x and
is in R® (unless
[x]o.
They
8 is the
are
not
standard
equal
basis
for
in gene-
R’).
Theorem
8 and Exercises
25 and 26 show that
the coordinate
mapping
translates
vector
space statements
or calculations
in V into equivalent
(and familiar) calculations
in R".
The table below lists some examples of
typical linear algebra statements.
CORRESPONDING
STATEMENTS
IN ISOMORPHIC
VECTOR
Linear
Algebra
in V
Matrix
Algebra
a.
u,
and
in V
lula,
[v]o,
b.
wis
v,
ware
in Span{u,v},
w is in the
spanned
Cc.
w=
or
subspace
by u and
(wle is
of V
[w]e are
in R™
in Span{[ul 9, [vlo},
spanned
cu + dv
in R"
and
(wl, is in the
v
SPACES
subspace
of R™
by [ula and
[vlo
or
[w]e = clul, ch divle
Get
Vi,. +, Vor 1S: Lins
indep.
e.
{v1,...,Vp}
spans
f.
{v,,...,Vnt
is a basis
V
for
V
{Cvilg,.-->[Vpla}
is
lin.
indep.
{[vilg,.+-»[vpla}
spans
{[vilg,---»[vnlg}
is a basis
R"
for R™
SOLUTIONS
EXERCISES
a—a—eewerer*ereTO =<
[x], =
5
a
we
1.
Since
7.
The B-coordinates of
c3b3 = X.
To solve
matrix:
io
a
a
Bena
BO
wa
CO
2 ea
HOI EG
bo
bg
ta
b;
te
x
have
x
are
this
x = 5b,
+ 3bo = 5|5] +
scalars c;,Co,cg3
vector equation,
pase
tO
Onb=
1
O10
BOS sTOFT30
OF
hises
that
row
1a Sa cl
OO
1
10
|
0
ues
satisfy c,b; + Cobo +
reduce
the augmented
tt)
ete
3
1
10
O#
OEMNO™
OF
1
I
290
=i
=]
3
4-14
CHAPTER
4
°¢
Vector
Spaces
-1
So’
13.
[x]2°="7=11"
8
3
terms
on
obtain
the
system
c3
=
1
2c3
=
4
t4.CoetnsCaae
7
and
or tle 3
Co
Ch
+
reduction
1
0
1
0
1
1
of
the
1
1
dla)
1
7
augmented
1
6)
3,
1
OO
Waited
matrix
1
2
EZ
y,
4)
ee
for
look
such
c1,C2,¢3
oD
2
of
powers
like
of
that
nye
Wee opis ortien ate
coefficients
equate
left,
the
out
Multiply
Row
2,
Problem
(lets tnt oCol toot). trc4llok 2t chet
Cy
t,
Practice
of
method
the
Using
produces
1
6)
10
Z
0 een
~-
0
O*
dL
2
Git
oracle
and
2
6
aa
[Pl =|]
A slightly faster solution uses Theorem 8 and the fact that a calculation
in Po can be done
instead
with coordinate
vectors
of all
the
given polynomials
relative
to the standard
basis
Pe oe ae
Equation
(*)
in such coordinate
i
cy
vectors
6)
pol’
real
fi
1
at es]
1
becomes
i!
1
2i2=
4
1
%
This vector equation is, of course,
tions above, and it is solved by the
15.
19.
a.
See
the
b.
See
equation
c.
See
Example
The
set
(unique)
dence,
S
definition
suppose
a B-coordinate
system of
process.
equa-
vector.
(4).
5.
spans
linear
of
equivalent
to the
same row reduction
V
because
combination
that
every
of
x
in
elements
S = {vj,...,vp}
and
V
has
a
representation
of
S.
To
show
-**
+ cnv,
civ,
+
linear
as
a
indepen-
= O for
some
scalars Cj,...,Cn.
The case when c, = ::* = c, = 0 is one possibility.
By hypothesis,
this
is the only possible
representation
of the zero
vector as a linear combination of the elements of S.
So S is linearly
independent and hence is a basis for IV.
4.4
21.
We
want
A to
Ax =
[xl
of-coordinates
matrix
satisfies
two
we
that
equations,
sop
ER
23.
satisfy
lex
ape
Suppose
in
that
this
dinate
see
for
each
x,
coordinate
(wl, for
vector
some
by
Since
ye
the
the
tO
+ CAD
[cyu,
coordinate
same
Ci U4
Since
we
Palxle = x for
u and
know
each
Systems
that
x.
the
+
and
mapping
solutions,
OI ONG
t+ ve°
SS
these
w in V,
and
denote
By
definition
°°:
+ cpb,
the
of,
entries
the.
coor-
w=
c,by
+ CpUpl a =
coordinate
alu),
tosce
is
+
w were arbitrary
is one-to-one.
one-to-one,
the
elements
following
if
It
equations
(the
zero
vector
in V)
(1)
[0],
(the
zero
vector
in R")
(2)
+ cpluple =
that
{[ulo,...,lupla}
immediate
consequence
V,
0
mapping
follows
of
Ci, ..., Cp:
CpUp
the
solution.
an
change-
Comparing
is linear,
(2)
is equivalent
to
0
;
(3)
Hence"c;,...,¢e, satisfy (1) if and only if they satisfy (3).
has only the trivial solution if and only if (3) has only the
only
4-15
ee
A
cj,...,Cp.
which shows that u = w.
Since u and
this shows that the coordinate mapping
have
and
sii.
[tly salar =
lS
ea 9 agg ie s| r [
[ule =
Coordinate
vectors,
eC
25.
¢
{u,,...,up}
is
linearly
of Exercises
is
iinearly
independent.
31
and
32
©"So_ (1)
trivial
independent
if
and
(This
is
also
in Section
fact
4.3.)
Warning: The standard mistake in Exercises 27-32 is to write the coordinate
vectors as the rows of a matrix and then to turn around and check the linear independence or dependence of the columns.
Since we mainly work with
column vectors,
it is wise to write the coordinate vectors first
(as columns) and afterwards write a matrix that can be row reduced to check the
linear independence of its columns.
4-16
27.
CHAPTER
The
4
°¢
Vector
vectors
coordinate
Spaces
;
polynomials
the
of
are
1
6)
ol’
4
1
|-2}?
6)
al!
3
1
6)
Fd,
wae
(rela
are
One can verify that these vectors
the corresponding
that
show
will
This
This same type of analysis
independent.
The details are omitted.
tive to the standard basis).
in R.
independent
linearly
polynomials in P3 are linearly
is needed for Exercises 28-32.
Study Tip:
Exercises 27-32 are easily changed into a question whether the
given polynomials form a basis for P3.
What else would you have to write
or calculate in this case?
Remember that although you may look at vectors
in R,, your answer must
include a discussion about
the polynomials
themselves.
That is, you must explain why the given polynomials form,
or do
not form, a basis for the space of polynomials.
Answer to Checkpoint:
The columns of the matrix A = [bi
<:-°:
b,] forma
basis for R", so A is invertible,
by the Invertible
Matrix Theorem.
By
Theorem 5 in Section 2.2, for each x in R" there exists a unique vector c =
(Ci, ..:,0n)
MATLAB
such
The
thatetx
Backslash
=LAc,
thatels,ox
Operator
= cob;
®
-- = +ucup.
\
If an equation Ax = b has a unique
produce x if you use the command
solution,
MATLAB
will
automatically
x = A\b
In this section,
the equation
probably
with u the B-coordinate vector of x, and
The
square,
"backslash"
the
A (see
Section
to
produce
the
gives
Ax = b has
ities that
works
A\b
causes
command
of
MATLAB
command
the
2.5);
if A is
in
will
have
the command
two
MATLAB
different
to
invertible,
unique
solution
to
error
message
"matrix
Ax
=
the form
will be
b;
create
the
and
an
LU
P\x.
When
A
has
6.
A
is
not
(even
is
factorization
factorization
if
is singular"
a solution).
The backslash operator
will be introduced later,
in Chapter
ways.
Pu
u =
is
used
invertible,
if the
additional
system
capabil-
4.5
¢
The
Dimension
of
a Vector
Space
4-17
4.5 THE DIMENSION OF A VECTOR SPACE
This
ious
short section provides
subspaces of a vector
a convenient way to compare
space without having first
the "sizes" of varto choose specific
bases
for
and
then
basis
the
dimension
the
subspaces
to count
the
number
of
elements.
KEY IDEAS
Pics, I
Theorem
10
shows
that
of
a
finite-dimensional
vector
space
does not depend on the particular basis for the space.
Example 4 shows
to visualize subspaces of various dimensions.
Theorems 9 and 12 are important for later theory and applications.
might remember Theorem 9 more easily in this form:
In an n-dimensional
vector
must be linearly dependent.
You
already
know
this,
of
space,
course,
for
R”.
phisms in Section 4.4, then you can see
because
any n-dimensional
vector
space
Theorem
(Theorem
12)
may be restated
any
set
And
why
is
if
of
more
you
the result
isomorphic
than
read
how
You
n
vectors
about
isomor-
is true
to R”.
in general,
The
Basis
as follows:
If dim V = p = 1 and if S is a subset of V that contains exactly
elements,
then S is linearly independent if and only if S spans /.
p
Warning:
Suppose H is a subspace of a finite-dimensional space V.
Theorem
11 shows that any basis of H can be extended to a basis of V.
But it is
not true that any basis of V can be cut down to a basis for H.
That is, if
S is a basis for V, it is not likely that a subset of S is a basis for H.
For instance,
consider the standard basis for R and any plane H that contains the origin but none of the coordinate axes.
SOLUTIONS TO EXERCISES
aaa
Sorat
1.
Since
1
8
spans
the
3.
The
the
for
ict
)
1
ted
|tfor’al
it
the
vectors
Hence,
dim
H =
call
it H,
given subspace,
vectors
the
Also,
H.
call
(because
H.
lis,tw thevset
3
subspace,
independent
basis
ne
Ss tahineesiit!,
set
is
are
not
multiples),
is the
set
of all
=2
Tai,
et
0
3
obviously
so
the
certainly
linearly
set
is
a
2.
linear
combinations
of
CHAPTER
4-18
4
¢+
Vector
Spaces
0
1
2
Via
8)
i |
Sse
O|’
eh
1|°
1
2
0
Rea
3
=
2
0
One way to do
is linearly independent.
if {v1,V2,Vv3}
First determine
A faster
0].
v3
v2
[v;
matrix
augmented
the
reduce
row
this is to
is not
v2
and
0,
#
vi
Clearly,
4.3.
Section
in
4
Theorem
use
to
way is
vectors
the
of
combination
linear
a
not
is
v3
and
vi,
of
multiple
a
V1,V2 that
precede
it,
because
the first
entry
in v3 is not
zero.
Hence {v1,V2,V3}
is linearly independent
and thus is a basis for the
space H it spans.
Thus dim H = 3.
Standard calculations show that the set of solutions of the homogeneous
system consists of only the trivial solution.
So the subspace is {0},
and it has no basis.
(The vector 0 spans the space,
but {0} is a
linearly dependent set.)
By definition,
the dimension is zero.
[Note:
Persons who want every subspace to have a basis often define the empty
set
to
zero,
be
so
a
the
basis
for
dimension
{0}.
The
of
is still
{0}
13.
A has three pivot columns,
without pivot positions, so
and dim NulA = 2.
19.
a.
See
the
c.
See
Example
1.
e.
See
Practice
Problem
before
el.
Form
the
box
before
you start
matrix
polynomials,
Example
2.
of
vectors
in
this
basis
is
zero. ]
dim ColA = 3.
There are two columns
equation Ax = O has two free variables,
5.
(You
b.
Read
Example
4 carefully.
dad.
See
Theorem
10.
should
be
working
the
practice
problems
the exercises. )
whose
relative
tte
so
the
number
columns
to
the
are
the
standard
coordinate
basis
vectors
of
the
Hermite
{1,t,t°,t}:
1
Ose
2
8)
O
Tae
HOrkei12
OnnOad
4
0)
One
40o-40
8
The matrix has four pivots and hence is invertible.
So its columns,
the coordinate vectors,
are linearly independent.
Hence the Hermite
polynomials themselves are linearly independent in P3.
Since there are
four Hermite polynomials,
and dim Pz = 4, we conclude from The Basis
Theorem that the Hermite polynomials form a basis for P3.
4.5
Note:
You
could,
of
-¢
course,
The
Dimension
of
say
that
columns
the
a Vector
of
Space
the
4-19
matrix
A
span R.
But you cannot stop with that assertion,
because you need the
polynomials to span P3.
You have to go on and point out that because
of the isomorphism between P3 and R*, a set of vectors spans P3 if and
only if the set of coordinate vectors
(the columns of A) spans R.
So
the solution is shorter if you appeal to the Basis Theorem.
25.
Note
that
n =
1,
because
S cannot
have
fewer
=n2
1, then V # {0}.
for V, by the Spanning
If S spans
Set Theorem.
then S’ also has fewer
10, because dim V = n.
than n vectors.
This
So S cannot span V.
than
O vectors.
If
dim
V
V, then a subset S’ of S is a basis
But if S has fewer than n vectors,
is
impossible,
by Theorem
27.
If P were finite-dimensional,
then Theorem 11 would imply that
dim P, = dim P for each n, because each P, is a subspace of P.
impossible,
so P must be infinite-dimensional.
29.
a.
True.
Apply the Spanning Set Theorem to the set
produce a basis for V.
This basis will have no more
in it, so dim V must be no more than p.
b.
True.
By Theorem 11, {vj,...,Vp} can be expanded to a basis for
The basis will have at least p elements
in it, so dim V must be
least p.
c.
True.
Take any basis
(of p vectors)
for V and adjoin the zero
vector.
Spanning sets can be arbitrarily large. The dimension of V
being p only keeps spanning sets from having fewer than p elements.
31.
Since
H is a nonzero
finite-dimensional
and
subspace
of
has a basis,
has
some
the
exist
form
T(y)
scalars
for
c;,...,Cp
y
such
n+
1 =
This is
{vj,...,vp)}
and
than p elements
V.
at
a finite-dimensional
space,
dH is
say, vi,...,Vp.
Amy vector in T(H)
in
H.
that
y
Since
=
civj
{vi,...,vp}
+
+**
+
spans
CpVp.
H,
there
Since
T
is
linear,’
T(y) = cyT(v;) + +++ + cpf(vp).
This shows that {T(vz),...,
T(vp)} spans T(H).
By Exercise 29(a), dim T(H) = p = dim d.
Second
wise,
T(v,)
proof:
T(H) has
for some
Let
k = dim
T(H).
a basis,
which can
vectors
vj,....,VK
If k = 0,
then
linearly independent,
so is {vj,...,Vx},
by Exercise
Since vi,...,V,x are in H, the dimension of H must be
Hint
for
Exercise
32:
Use
an
exercise
in
k < dim
H.
Other-
be written in the form T(v,),...,
in H.
Since
{T(v,),...,T(vy)}
is
Section
31 in Section
at least k.
4.3.
4.3.
Study Tip:
The next section is quite important.
Do your best to
up now.
Otherwise,
you may have difficulty relating the various
and facts about matrices that will be reviewed in Section 4.6.
get caught
concepts
4-20
4.6:
CHAPTER
4
«+
Vector
Spaces
RANK of io crmusloo jsit Jat)
ee |. Seon) 30) eee
This section gives you a chance to put together most of the
chapter in the same way that Section 2.3 collected the main
sections that preceded it.
of
of
ideas
ideas
the
the
KEY IDEAS
Se
ee aa
rank A = dim Col A.
By definition,
The Rank Theorem is the main result.
But because rankA
is also the dimension of RowA,
the displayed equation
in the theorem leads to the equation:
dim RowA + dim NulA = n.
Equivalent
Descriptions
The
an
rank
of
mxn
of
of Rank
matrix
the
A may
in several
of
definition)
dimension
>the
>the
number of pivot positions in A,
(from Theorem 6)
maximum number of linearly independent columns in A,
>the
dimension
>the
>the
>the
maximum number of
number of nonzero
maximum number of
(Supplementary
the
row
space
described
>the
of
column
be
space
of
A,
A,
(our
(from
the
Rank Theorem)
linearly independent rows in A,
rows in an echelon form of A,
columns in an invertible submatrix
Exercise
12
at
the
end
of
the
ways:
of
A.
chapter)
Pay attention to how Theorem 13 differs from the results in Section 4.3
about Col A: If you are interested in rows of A, use the nonzero rows of an
echelon form B as a basis for RowA;
if you are interested
in the columns
of A, only use B to obtain information about A (namely,
to identify the
pivot columns),
and use the pivot columns of A as a basis for Col A.
For
NulA, it is important to use the reduced echelon form of A.
When a matrix A is changed into a matrix B by one or more elementary
row operations,
the row space, null space, and column space of A may or may
not be the same as the corresponding subspaces for B.
The following table
summarizes what can happen in this situation.
Effects
of Elementary
>Row operations do
among the columns.
same
linear
Row
Operations
not affect the linear dependence relations
(That is, the columns of A have exactly the
dependence
relations
as
the
columns
row-equivalent
to A.)
>Row
>Row
bRow
usually change the column space.
never change the row space.
never change the null space.
operations
operations
operations
of
any
matrix
4.6
¢
Rank
4-21
The four subspaces shown in Figure 1 in the text are called the fundamental subspaces associated with A.
(See Exercises 27-239.) The main difficulty here
is to avoid confusion
between
RowA,
Nul A, and Col A.
The
fourth
for
subspace
The
the
will
appear
table below adds
Invertible Matrix
STATEMENTS
again
in Sections
6.1
FROM
THE
INVERTIBLE
MATRIX
Equivalent statements,
for any nxp matrix A.
Equivalent only for an
nxn square matrix A.
k.
a.
There is a matrix
D such that AD = I.
A has a pivot
tion in every
The
columns
span
R®.
The
equation
posirow.
of
has at least
solution for
b.
A
Ax = b
one
each
b
A is an
matrix.
THEOREM
Equivalent statements,
for any nxp matrix A.
ae There
invertible
list
C such
is a matrix
that CA = I.
A has n pivot
positions.
A has a pivot
tion in every
A is row equivalent
to. the nxn
identity matrix.
The columns of A
form a linearly
independent set.
The equation
has a unique
solution for
Ax = b
The
each
has at most one
solution for each
b
equation
posicolumn
Ax = b
in R’.
in R’.
The linear transformation xt> Ax
The linear transformation xb Ax
is invertible.
The linear transformation xb Ax
is one-to-one.
A is a product
of elementary
matrices.
Ta6.
P
The equation Ax =
has only the
trivial solution.
onto
n
R .
A
is an
matrix.
a
invertible
dim
rank
ColA=n
SOLUTIONS
TO EXERCISES
2
eee
Jil
1me=4%6
or
2
ge =o"
98h
4
10"
=7
1
*7
A =
The
equation
has no free
variables.
The columns of A
form a basis of R”.
be
7.4.
in R’.
maps
Oo.
and
three more statements
to the bottom of the
Theorem, Section 4.3 in this Study Guide.
n.
b
0
Ax = 0
4-22
CHAPTER
4
«+
Vector
Spaces
Look at B, and conclude that A has two pivot columns and the equation
In
So rank A = 2 and dim NulA = 2.
Ax = 0 has two free variables.
so
columns,
pivot
are
A
of
fact, the first two columns
saintereed Sat
For
the
Basis
For
for
{(1,0,-1,5),
RowA:
the
null
space,
use
yp WR
2 icon)
1
19;
0
SH
572
0
Thus
x;
solution
=
Xg3 -
x3
5X4,
of
Ax
_
(S/2) x3
X4
Xo]
0
1
6)
=
¥5
{u,v}
Xo
B.
is,
That
(0,-2,5,-6)}
reduced
5
6 ie
0
=
form
X41
(5/2)x3
-
echelon
form
of
A to
solve
=
Xo -
X3 + 5x, = O
(5/2)xg + 3x, = O
O0.6= 0
3xq,
with
x3,Xq
free.
Ax
The
=
O:
general
is
>
x3
X4
Thus
0
the
echelon
in the
rows
the
use
space,
row
5X
=
1
JXa
tet
Te ect
X4
is a basis
mie)
Oe
ele
leet
a
Teo
6)
for
ib
~
+
u
Vv
Nul A.
Study Tip: Because rankA = 2 in Exercise
1, any two linearly
independent
columns of A form a basis for ColA, and any two linearly independent rows
of A form a basis for RowA.
When the rank of a matrix exceeds 2, selecting bases in this way is not so easy.
(That is why you examine an echelon
form of A.)
On an exam, you should always choose the pivot columns of A as
the basis for Col A and the nonzero rows of an echelon form of A as the
basis
for
7. Yes,
hence
RowA.
This
will
show
you
can
handle
matrices
with
Col A = R*, because
ColA
is a 4-dimensional
subspace
coincides
No,
Nul A cannot
because
with
R.
in Nul A have 7 entries.
the Rank Theorem.
13.
that
NulA
If A is either a 7x5 matrix
pivot positions.
So 5 is the
is
a
be
R,
3-dimensional
subspace
any
rank.
of R* and
the
of
or a 5x7 matrix,
then A has at
largest possible value for rank A.
vectors
R,
by
most
5
4.6
17.
19.
a.
See
the
paragraph
b.
See
the
warning
c.
See
the
Rank
e.
See
the
Numerical
before
after
Example
Example
Theorem.
d.
Note
¢
5.
2.
See
before
the
the
But Col A is a subspace of R°.
Study
Tip:
for
all
Exercises
4-23
1.
Rank
Theorem.
Practice
Problem.
Visualize
the
system
as
Ax = O where
A is a 5x6
information
in the problem
implies
that
the solution
dimensional.
By the Rank Theorem,
rank A = 6- 1= 5.
solution
Rank
matrix.
The
space
is oneSo dim ColA =
Hence Col A = R°. Thus Ax = b has a
b.
19—25
make
good
exam
questions.
21.
Visualize the system as Ax = b, where A is 9x10 matrix.
You are told
that the system has a solution for all b, so A must have a pivot position in each row.
(That is, rankA = 9.)
Since A has 10 columns,
the
Rank Theorem implies that dim Nul A= 1.
So it is not possible to find
two linearly independent vectors in Nul A.
23.
The set of interest
is the null space of a 12x8
matrix A.
The description of this set implies that dim Nul A = 2.
By the Rank Theoren,
rank A= 8 - 2 = 6.
So the equation Ax = 0 is equivalent to Bx = 0,
where B is an echelon form of A with 6 nonzero rows.
The answer to the
question is that six homogeneous equations are sufficient.
25.
Let A be the 10x12 coefficient matrix.
By hypothesis,
there are three
free variables
in the system Ax = b, so dim NulA = 3. By the Rank
Theorem,
dim ColA = 12 - 3 = Q.
Since
-ColA
is, 2 subspace
of R
(because A has 10 rows), Col A cannot be all of R , so some nonhomogeneous equations Ax = b will not have solutions.
31.
The
solution
Let
A= [u
33.
is
us
in
the
us].
text.
If u # 0,
then
Col A is one-dimensional.
Hence
U2 = ru and ug = su, so that
A={[u
ru
su)
=uli
r
u must
there
s]=
uv’,
be a basis
exist
where
scalars
r
for
Col A,
and
s
such
since
that
v =
Oe
If the first column of
then A = [0
nonzero,
[0 0.
Stas
(Onl or
call
A is zero and the second column,
case,
this
In
r.
ru] for some
u_
ul. take vy = (00,1).
it u, is
take v =
4-24
CHAPTER
a.
35.
4
°
Spaces
the
matrices
N be
C and
Let
Vector
RowA
4 and
=
rank
A =
you understand
these
statements.)
because
(Make
sure
RowA
dim
b. The matrix S = [R’
N] is 7x7,
R), and N should
R.
of
subspace
28(b).
Since
be
then R should
a subspace of R.
is
because R’ is 7x4 and N is 7x3.
If M is a matrix
whose
columns
should be 5x1,
because NulA
is
by Exercise
bases
A
matrix
5x7
For the specific
So ¢ should be 5x4
NulA = 3.
subspace of
(because Col A is a four-dimensional
is a three-dimensional
NulA
(because
7x3
be
if the rows of R form a basis for RowA,
Also,
4x7,
are
columns
whose
construct
you
for Col A and Nul A, respectively.
rank A = 4 and dim
in this problem,
form a basis
for NulA,
a one-dimensional
subspace
C is 5x4,
the
matrix
T =
[C
M]
then M
of R ,
should
be
sate D
In general,
the matrix
S is nxn
because
the
dimensions
of
Row A (spanned by the columns of R’) and Nul A add up to n, the number of columns of A, and both RowA and NulA are subspaces
of R”.
The matrix T is mxm because the dimensions of the column space and
Nul A_. add up to m,
the number
of rows
of A, and both
ColA
and
Nul A are subspaces of R™.
37.
Giving
Mastering
an
answer
Linear
here
Algebra
would
spoil
Concepts:
the
Eight
problem
Basic
for
you.
Ideas
Sometime between now and when you finish the chapter,
you should do a major
review of the eight key concepts introduced in this chapter:
vector space,
subspace,
column space,
null space,
basis,
coordinate
vector,
dimension,
and rank.
(The row space of A is not really a separate concept;
it is just
the column space of A.)
Study your old review sheets,
and prepare
new
summary sheets for coordinate vector, dimension,
and rank.
Use as many of
the standard
categories
(special
cases,
examples,
algorithms,
etc.)
as
possible.
The tables in this section will be helpful.
Also,
add crossreferences
about
dimension
and
rank
to other
sheets
(subspace,
column
space,
etc.)
and
update
your
summary
sheet
for
the
Invertible
Matrix
Theorem.
ese
MATLAB
ref,
rank,
In this
course,
rank
A.
of
and
you
can
In practical
rank(A),
based
The
Toolbox
on
randint
use
either
work,
you
the
singular
command
randint
ref(A)
should
value
use
or
rank(A)
the
more
decomposition
produces
a matrix
(Section
with
to
check
reliable
the
command
7.4).
integer
entries.
4.7
¢
Change
of
Basis
4-25
4.7 CHANGE OF BASIS
This section will help you better understand
of Section 4.4 now is strongly recommended.
coordinate
systems.
A review
KEY IDEAS
ea
Figure 1 and the accompanying discussion will help you visualize the main
idea of the section.
Imagine superimposing the 6-graph paper (Figure 1-b)
on the B-graph paper (Figure 1-a).
Can you see where b, will lie on the
G-coordinate system?
Four units in the c,-direction and one unit in the
Co-direction.
That is the geometric interpretation of the equation [bil, 7
A\ ,
at
;
A
=6
ii in
Example
1.
Similarly,
since
[ba], =
Bt
bo lies
six
units
in the
negative ec,-direction and one unit in the co-direction.
In general,
the locations of b, and bo on the 6-graph paper are precisely what you must find in order to build the columns of the change-ofcoordinates matrix:
efs
The
for
=
notation
changing
[(bil,.
[bo]
e]
for this matrix should help you
B-coordinates into G-coordinates:
remember
the
basic
equation
[xlo = pPalxle
The calculations
are simple when B and
after
Example
2 illustrates
the algorithm
coordinates matrix.
In general,
[e,
Oo
Equivalently,
[Pe
where
«ce
using
the
and
2.2,
Section
[xl
as
bn]
~
notation
[I
eg
|
of Section
4.4,
Pal
~C1 @bgl
EB )onn changes
from
bi
Pp is the matrix
coordinates,
of
oe
© are bases for R”.
The box
for computing
the change-of-
[b;
°°:
Pe is similarly
you
will
standard
follows:
3
see
bp] that
changes
B-coordinates
defined.
» If you
refer
that
coordinates
:
as
(Pe)
G-coordinates,
we
can
the
11
toExercise
same
els is
to
back
to standard
faa
“
Pa.
Since
obtain
[x],
4-26
This
CHAPTER
4
*+
Vector
Spaces
-1
=
Palxle ——>
(Pe)
B-coordinates
standard
coordinates
C-coordinates
provides
diagram
viewing
of
another
way
2¢2 and
bg = Sc;
-
Ms
Palxle = efglxle =
[x], ——
the
coordinates.
of
change
[x],
SOLUTIONS
TO EXERCISES
cea
1.
a.
From
by = 6c;
-
-
4c2,
bekBa 2): Su |< yes letSn
b.
Since
x =
-3b,
7.
Unlike
can
Exercise
write
vectors
1,
[bil
and
separately,
PGiieGar
lta} > Le]
peel
=
[x]. =
you
do
[ba].
use
the
not have
Rather
_ | dagen
Pigebali + E:
(73
z
2
11.
7s teal
15.
a.
See
Theorem
b.
See
the
Fac’ Spi a
-1
= fr |
first
13. Let b; represent
let
The
paragraph
=
6
1
|-2],
1
i
thar
5 eT
Example
Sid.
take
[bol
=
e
aie
in the
2:
metres
th)
Osi
thi: chapter
estes.
yee
Sale
Places
subsection
on
yr
40
peests
at)
,
from © to 8 is
matrix
at
3]
Change
of Basis
in R”.
1 - 2t + t*, let bo be 3 - St + 4t?,
© be the standard basis
the vectors b,,b2,bg3 are
3
|-5],
4
from which you
two coordinate
di,
lo -8
_ =(5 4 ce i
the polynomial
bg be 2t + 3t', and let
6-coordinate vectors of
(biJ..
from
iit
arent
5
aI: The change-of-coordinates
es
ate ~ “efs)
direct
information
than compute these
algorithm
sowilptses ni Fode
O
apdce 5
J
Thus es =
2-4]
+ 2bo,
ae3] and
[x] B
write
[bs].
=
6
0
2
3
{1,
t,
t?}
for
Po.
4.7
¢
Change
of
Basis
ee 2tle =
5
|-2
i
4-27
So
=
1
3
@)
|-2
-5
2
1
4
3
eee
The
coordinate
vector
[-1
+ 2tle satisfies
-1
epg l-1
This
19.
[M]
+ 2tle =
[-1
equation
can
1
3
ote)
1
4
Orr =}
AS ON a |
ais
6)
a.
Let
© be the
[ual pe, and
uj
where
You
=
V
matrix
be
solved
MATLAB
The
forn.
by row
reduction:
1
ea 8)
0,
OU
1
Ope
40
i
{v,,v2,v3}.
[v1
is
Vo
the
ges5
2 ee
i
Then
ile
the
columns
of 6-coordinate
v3]
uj],
=
Viujle
matrix
V =
[v,
v2
v3].
Then,
of P are
(uly,
vectors,
by
the
definition
of
multiplication,
ug] = V[ [ui],
[u,
Ug
know
V and
P,
P,
so
to part
compute
so
you
row
has
data
reduces
can
[ugly ] = VP
[ual
compute
uj,
Us,
=7]/[
1-2-1
-6
a2{|-3 -5
oO] =/-5
eA
eel
21)
(a),
W from
Change-of-Coordinates
Toolbox
ref(M)
;
[us]. . By definition
By analogy
and
Ea
basis
“2-8
vP=/]2
5
922
b.
+ atl, =
for
V
W
WP, , where
us.
-6 -5
-9
O| =[u,
320043
W =
[W,
Wo
up
Ws].
us]
You
know
VP
Matrix
Exercises
a matrix
and
such
as
7—10
and
[cy
co
17-19.
by ~ be]
to
The
command
the
desired
V
4-28
CHAPTER
4
«
Vector
Spaces
4.8 APPLICATIONS TO DIFFERENCE EQUATIONS
equations.
Difference equations are the discrete analogues of differential
inuous
cont
and
discrete
The
engineering.
and
science
in
Both are important
similar
in
applied
is
algebra
linear
and
parallel,
remarkably
theories are
equafor difference
easier
are somewhat
the calculations
although
ways,
difference
some
illustrate
here
exercises
and
examples
of
variety
A
tions.
equations you may encounter later in your work.
KEY IDEAS
[ee |
Each signal in $ is an infinite list of numbers.
Linear independence of a
set of signals can often be demonstrated by looking at short segments of
the signals,
that
is, by showing that a Casorati
matrix
is invertible.
A
Casorati matrix cannot be used in general to demonstrate
linear dependence
of a set. However, see the appendix at the end of this Study Guide section.
The main focus of the section is on difference equations.
Given a homogeneous difference equation,
you should be able to:
> determine
> find
whether
solutions
of
> give the general
ple roots and no
a specified
the
signal
equation,
solution (when
complex roots).
using
the
is
a solution
the
auxiliary
auxiliary
of
the
equation;
equation;
equation
has
no
multi-
Theorem 18 is the key result
that enables you to write the general
solution.
Just finding some specific solutions
is not enough;
you must show
that they span the set of all solutions.
But Theorem 18 and the Basis Theorem in Section 4.5 together show that for an nth order equation,
you only
need to find n linearly independent solutions.
(See Example 5.)
The same
principle apples to an nth order differential
equation (discussed later in
Section 5.7.)
This principle is one of the most powerful applications of
linear algebra in the text.
The subsection
on nonhomogenous
equations
is optional.
If “this7is
covered, you should be able to work Exercises 25—28.
The general principle
is illustrated in Figure 4, page 278:
General solution of
nonhomogeneous eqn.
_
JParticular solution of
nonhomogeneous eqn.
s
General solution of
homogeneous eqn.
The final subsection shows the modern way to study an nth order linear
difference equation, rewriting it as a first order system
x,4, = Ax,
(k=
eee
Such systems
were
introduced
in Section
1.9 and they will
be
discussed further in Sections 4.9 and 5.6.
4.8
°
Applications
to Difference
Equations
4-29
Substitute
these
SOLUTIONS
TO EXERCISES
eee
1. If yy =
formulas
(-4)*,
then year =
into
the
left
side
(-4)
k+1
and yeio =
of
the
equation:
Vise * 2yusi — 8¥y-= (24)
2s)
k+2
(-4)
ee)
(-ay" (24)
@F (74)
81
(=A) Lieeo
8 ecOl eeGemtor alle k
Since the difference
equation holds for all k,
The text answer displays the similar calculations
7.
Compute the Casorati matrix
k = O for convenience:
2 2tO
Cee
1
2 xn (52)5|
(oreo
(5)
This
Casorati
IMT.
Hence
set
+e
|1y.
leg
=p
matrix
the
for
has
the
24
230-28)
Weel
three
signals
ees
|
4u |One iek-3l
Op aes
pivots
of signals
es
and
hence
(ir eae (<208)
is
(-4)* is a
for y,=2.
solution.
ox.
and
(=2)me
>
eee
10
0
ee
Se ds ge
oO 12
is
invertible,
linearly
setting
by
the
independent
in
Ss.
We know
(from the text)
that
these
signals
are
in the solution
space H of a third-order difference equation.
By Theorem
17, dimdH =
3.
Since the three signals are linearly independent,
they form a basis
for H, by the Basis Theorem in Section 4.5.
Warning:
sion,
Many
often
student
papers
for
confusing
linear
independence
Exercise
7 suffer
of
the
from
a
columns
lack
of
of
the
preci-
Casorati
matrix with linear independence of the signals in S.
There is no need to
discuss the columns of the Casorati matrix— just observe that the matrix is
invertible.
But you must point out that the three
signals are
linearly
independent,
in order to apply the Basis Theorem
(Theorem
12) in Section
4.5 to the vector space H of solutions to the difference equation.
13.
The
auxiliary
equation
the
quadratic
formula,
for
Yxso
-
Yer
+
2
g Vk
=
:
Oeis*
Te
2
rot
2
Ga
G..
By.
1
et, VilF. 8/9 aa 25173ic82
Voliuloe 2ieluolinad 2.0) slay ube
Two
solutions
of
the
nals are obviously
of the other. Since
the
two
signals
form
difference
equation
are
(=)* and
ie
because neither
linearly independent
the solution space is two-dimensional
a basis
for
the
solution
space
These
sig-
is a multiple
(Theorem 17),
(Theorem
12).
4-30
CHAPTER
Study
Tip:
4
+
I think
Vector
Spaces
questions,
exam
good
are
25—28)
(and
7—19
Exercises
Probbecause they illustrate how important Theorems 17 and 12 really are.
discussion
your
but
numbers,
theorem
memorize
to
you do not have
ably,
should show that you have two theorems in mind and that you know how to use
them.
19.
(Check
The
with your
auxiliary
By the
Two
instructor. )
equation
quadratic
solutions
for
Yxso
+
4¥n+1
+
Ye
=
O
is
r° MN
ate: SP
formula,
of the
difference
equation
are
(-2 + V3)" and
(-2 - ¥3)*.
They are obviously linearly independent
because neither
is a
of the other.
Since
the solution space
is two-dimensional
17),
the
two
signals
and the general
25.
To
prove
that
9 FF
form
solution
y, =
a
fundamental
set
has
the form
c,(-2
k? is a solution
of
solutions
multiple
(Theorem
(Theorem
+ v3) + Caan
12),
v3).
of
Yux2 + S¥us1'~ 4¥y = 10k + 7
C1)
show that when k*, (k + 1)*, and (k + 2)* are substituted for yx, Yxet,
and
yx+2,
respectively,
the
(hot Q)7 4 B6kEnod
resulting
Sr Ae
Yuen) +) Byes
of
oo Ay
(1).
= 0:
is
true
for
all
k:
aidkete4)la Bhi seek 4o0e-.1ki)
as
;
So k is a solution
difference equation
equation
The
(1S
aye
10k
for
+ 7
wx
auxiliary
(ek
all
4)
k.
equation
for
the
homogeneous
Vfor) aldick
(2)
is r° + 3r - 4 = 0, which factors as (r - 1)(r + 4) = 0, sor =1, -4.
Thus
1°
and
dependent
basis for
(2)
is
(-4)"
are
cy:1°
+
ce(-4)°.
obtain the general
31.
The
full
solutions
(for neither
is a
the two-dimensional
Add
of
(2).
multiple
solution
this
to
The
is
in the
text’s
answer
are
linearly
in-
other),
so they form a
The general solution of
a particular
solution k? + cy + co(-4)*
explanation
signals
of the
space.
solution
of (1).
section.
of
(1)
and
4.8
35.
For
{y,}
and
T(Lyy}
+
{z,}
°*
Applications
in S, the
{2y})
=
(Yuso
kth term of {y,}
+ Zea)
+ BYurr
(Y+2 + AYne1
For
any
scalar
r,
T{ Yui
+
kth
term
the
Thus
T has
Appendix:
The
the
Let
< ys,....4.
¥nr_ be
let
y;(k)
denote
C(k)
a.
If C(k)
is
b.
If yi,...,Yn
=
Hence
r{y,}
is ry,,
+ DZ)
TI Zy}
of
and
define
a linear
that
so
+ b(ryy)
properties
transformation.
Test
kth
“of Signals
entry
in
in the
S.
For
signal
yi (k)
pe
Yn(k)
yi (k+1)
ie
Yn(k+1)
yi (ktn-1)
sss
y,(ktn-1)
for
satisfy
Yuen + 21Yxen-1
4-31
DU Ye + 2x)
+ (Ze+2 + AZK41
= rT{yx}
invertible
all
Equations
is YR + Z.
+
+ DY~)
a ‘set
the
+ Zee1)
= rlYus2 + AYxe1
two
Casorati
+ {z,}
+ DY)
TYx+2 + alryysi)
T(r{ yy} )
to Difference
+ °°*
some
k,
(with a, # 0), and if the
some k, then {y1,...,ynt
is
is not invertible.
= O
Casorati
linearly
1,...,n
and
for
any
kK,
let
The
Casorati
is
linearly
independent.
difference
equation
of order
{y;,...,yn}
a homogeneous
+ 2nYk
Jj =
y, and
for all k
matrix
n,
(#)
matrix C(k)
is not invertible
for
dependent in S, and for all k, C(k)
Proof.
(a) The argument given in the text (page 273) for a set of three
(b) Suppose that yj,...,Yn
signals generalizes immediately to n signals.
not invertible for some
is
C(ko)
and
(*)
of
are in the set H of solutions
kp.
It
is readily
verified
that
if T:H
y(ko)
eer
y(ko+1)
y(korn=1)
> R" is defined
by
4-32
CHAPTER
4
«¢
Vector
Spaces
The proof of Theorem 16 is easily modithen T is a linear transformation.
...,
y(ko),
y whenever
solution
unique
a
has
(*)
that
show
to
fied
H
of
mapping
one-to-one
a
is
T
that
means
This
specified.
y(kotn-1) are
onto R".
Casorati
Furthermore,
matrix C(ko)
not invertible.
by Exercise 32
in
the
and
images T(y,), ..., T(yn) form the columns of the
because C(ko) is
hence are linearly dependent,
Since T is one-to-one,
In this
Section 4.3.
and
., T(yn) are linearly dependent,
Exercise 31 in Section 4.3.)
MATLAB
is linearly dependent,
{y;,...,yp}
case,
for any k the images T(y,),
so
C(k)
not
is
invertible.
(See
roots
In Exercises 7—16 and 25—28,
the polynomial in the auxiliary equation
is stored
in a row vector p, with coefficients
in descending
order.
For instance,
if the auxiliary equation is r
+ 6r + 9 = 0, then p =
(-i--6++9).
The MATLAB
are the roots
command
roots(p)
of the polynomial
produces
described
a column
by p.
vector
whose
entries
4.9 APPLICATIONS TO MARKOV CHAINS
This section builds on the
should review that example
population movement example in Section 1.9.
You
now.
Markov chains are widely used in applica-
tions and there is a rich theory connected with them.
The simple
examples
and exercises
in this section provide a basic foundation on which you can
build later as needed.
Two of the examples here will be analyzed from a
different point of view in Section 5.2.
KEY IDEAS
~~...ee
A probability
vector is a list of nonnegative numbers that sum to one.
A
Markov chain is a sequence of probability vectors {x,} that satisfy a difference equation
Xx4, = Px,
(k = 0,1,...)
for some stochastic matrix P
(whose columns are themselves probability vectors).
The theory of this chapter can be used to show that P - I always has a
nontrivial
null
space when P is a stochastic
matrix
(Exercise
17).
In
advanced texts it is shown that the null space of P - I always includes at
least one probability vector,
which then is a steady-state
vector for Ee
because the equation
Exercise 18.)
(P
-
I)q
=
0
is
equivalent
to
Pq
=
q.
(Also,
see
4.9
¢
Applications
Our main interest
is in a regular stochastic
according to
is unique,
vector
the steady-state
predicting the distant future for a Markov chain
is to find
the
q no
what
matter
the
Chains
4-33
matrix P.
In this case
18.
The key to
Theorem
associated with such a P
sequence
{x,}
to
converges
q,
since
the
matrix
P,
label
the
columns
N
(for
news)
the
same
order
for
the
rows.
vector
steady-state
to Markov
state.
initial
SOLUTIONS
TO EXERCISES
oe
1.
a.
To
set
up
the
and
M
(for
music)
stochastic
in some
use
order;
(Failure to keep the same order is a common source of error in this
The data should be arranged so you read down a
type of problem.)
column and then to the right along a row.
From:
b.
c.
You
N
M
To:
.7
.3
.6]
.4]
News
Music
are
told
that
the
listeners
Markov
chain
a.m.,
so
start
the
There
are
two
breaks
between
The entries
audience is
Study Tip:
compute all
ifying that
of mine!)
find
8:15
and
are
listening
then,
with
xo =
9:25,
so
you
to
the
news
at
a
need
Xp.
Po =[3 ‘alLol = [3]
Pap ehisle |
Xo
= Px,
To
of
8:15
xy
7.
100%
in xm show that after two station breaks,
67% of
listening to the news and 33% is listening to music.
the
When you compute a typical probability vector Px, be sure to
Then check your work by verof the entries in the product Px.
the entries sum to 1.
(This trick has caught numerous errors
the
state
vector
a basis
vector
steady
for
(i)
(ii)
set up the matrix P - I;
find the general solution
(iii)
choose
a regular
stochastic
matrix
of (P - I)x = 0;
for Nul(P - I) whose entries
sum
P:
to
1.
4-34
CHAPTER
4
«+
Vector
at?
442...
Lith
4d
8ib..21
00P! —
Pre.
52
e= 22
oe
a.
OES TE
ag
io
2a
a
oly =a
cif tee 8
<a
=o
Spaces
~
1
4
=3'F
10
5th
=5
2
rf)
uD
O|
3
(6)
=3
X41
~
1
OvG=32
6)
TTLO
Bre
Or
e)
0
0
iwd
2
ma
1
Xo
6)
Six
=
-
=
X3
entries
1
in | 2]
sum
to
4,
so
13>
Exercise
Solve
(P -
=
10
3
1
2x3|
=
X3
2
1
ao
ore
oours
1/4
.25
(AS
50)
PAs}
meses
1.25).
enigsl®,
So
k=
.25
0
50
AAs:
O|
padected
SEK
5
=
1
0
Scale
rows
by 4
1
)
Interchange
rows
eee
1
~
1
“a
@)
= OO
~~
w
ie)
1
1
6)
0
0
=
-1
OgerO ee Oso
for
Nul(P
-
I)
is
:
AAS.
=. 50
225
we
BASH
=. 50
lc
will
be preferred
X4
Xp
=
=
1 & 3
xX3
X3
x3 is free
1/3
1}}+;
the
4
food
-.50
725)
.20
[=
I
a
= 0:
.25
(20
w25
So each
an d
Xx3
|I/AiF
1
qsl0e
.Pe=in254)nasi
3S;
I)x
-.50
A basis
=
1/4
=
1
by
Notice that the column sums are all zero for the matrix P 7.
This always happens
(see Exercise
17), and so you have
check your arithmetic for the entries in P - I.
Krom
oO
t
rows
row
X3
Xo]
X3
q = aq ay
1
Study Tip:
of Exercise
fast way to
XX
Ong
is free
1
The
0
1
2x3
Interchange
Scale every
steady-state
vector
is q =
|1/3].
1/3
equally.
Warning:
You may have noticed that in Exercises 7 and 13, I scaled rows by
10, to avoid decimals.
A common mistake
is to do this only to P, before
forming I - P.
That changes P drastically.
The scaling I did was permissible because it was applied to all the coefficients in an equation
Chapter
19.
a.
+
Supplementary
The product Sx equals the sum of the entries
tion, x is a probability vector if and only
negative and Sx = 1.
Let
P=[pi
matrix
multiplication
SP
=
By part
ly
MATLAB
po
(Sp;
(b),
°°:
the
S(Px)
The
MATLAB
here.
box
where
and part
-°:
=
Spy]
(SP)x
because
condition
homework
Pn],
Spo
nonnegative,
(a),
for
4
on
=[1
x
1-37
Thus, by definientries are non-
p; are
probability
1
1)
csr
1.
The
have
only
= 1 shows
page
in x.
if its
4-35
vectors.
By
(a),
= Sx =
P and
S(Px)
the
Exercises
that
contains
=S
entries
in Px are
nonnegative
Px is a probability
information
obvious-
entries.
that
By
vector.
is useful
CHAPTER 4 SUPPLEMENTARY EXERCISES
Justifications for the answers to the
Other justifications are possible.
included.
1.0
4..
[rus.
itself
Span{v;,...,Vp}.
a vector space.
b.
True.
nation
Any linear combination of vj,...,Vp-1 is also
of v1,...,Vp-1,
Vp, using a zero weight on vp.
c.
False.
Counterexample:
dependent.
d.
False.
Counterexample:
Let {e1,,e2,e3}
Then {e,,e2} is a linearly independent
e.
True,
f.
By the Basis Theorem,
True.
exactly p vectors.
So S must
False.
by the
The
Spanning
plane
must
is
a
True/False questions are given below.
A solution of Exercise 9(a)
is also
Take
Set
go
subspace
v, =
Theorem
2v;,
of
V,
then
and
every
a
subspace
linear
combi-
is
linearly
{vi,.°.,Vp}
be the standard basis for R®.
set but not a basis for R.
(Section
4.3).
S is a basis for V because
be linearly independent.
through
the
is
origin
S contains
to be a subspace.
Chapter
4
«¢
Vector
Spaces
False.
Counterexample:
2
|0
0
So
6)
8)
ae
W/
6)
6)
3].
6)
Try
Theorem 13 in Section 4.6.
operation on some 3x3 matrix.
This statement appears before
True.
to make an example of a specific row
False.
Row
operations
False.
Counterexample:
on
A do
A =
not
3
change
ai two
the
solutions
nonzero
rows
False.
n- k,
If U has k nonzero
by the Rank Theorem.
rows,
then
True.
Row
have
the
same
number
:
:
Hh
If
False.
of
equivalent
matrices
Counterexample:
columns
of
True.
If
Theorem
12(a)
A),
xty>Ax
A =
then
the
is
onto,
in Section
transformation
then
See
the
second
paragraph
False.
The
jth column
of ef,
y
in
Col AB
has
the
ColA
=
xh
=
k
but
and
of
rank A =
Ax
is
R” and
Ax = 0.
rank
dim
pivot
n
is
1.
NulA
=
columns.
(the
number
one-to-one.
rankA
=
m._
See
1.8.
True.
Any
rankA
of
form
after
is
Theorem
15
in Section
4.7.
[bj]l_.
(AB)x
for
some
x.
Then
y =
A(Bx),
which shows
that y is a linear combination
of the columns
of A.
Thus Col AB is contained in ColA; that is, Col AB is a subspace of
Col A.
By Theorem
11 in Section 4.5,
dim Col AB = dim Col A.;:* that
is, rank AB = rank A.
The
solution
is
in
the
text.
Chapter
4
«+
Glossary
Checklist
4-37
CHAPTER 4 GLOSSARY CHECKLIST
Check your knowledge by attempting to write definitions of the terms below.
Then compare your work with the definitions given in the text’s Glossary.
Ask your instructor which definitions,
if any, might appear on a test.
equation:
basis
a subspace
H):
A set
of x:
See
coordinates
(for
B-coordinates
change-of-coordinates
that
column
space
matrix
transforms
(of an mxn
Coan
<a:
controllable
(pair
has
n
of
B=
that:
to
the
basis
(from
a basis
B to a basis
Namely,
(equation)
matrix
ie
and
such
of x relative
A):
The
set
A matrix
ColA
pair
of x relative
dimension
(of a vector
explicit
full
rank
x relative
space
(vector
(matrix):
The
The
number
V):
vector
subspace
of ...
W of
R"):
space):
A vector
for
the
The
of A, and..
A ...
(of A):
...
(of a subspace
(vector
mapping
<P
notation,
is
nxn,
set
from
vector
(of a linear
transformation
linear
combination:
Any
linear
dependence
relation:
linear
filter:
that
equation
used
vector
space
B
JV):
...
representation
....
or
space
of
...
more
....
V that
....
....
T:V > W):
The set
equation
where
....
transform
....
to
is
of one
sum....
A ...
space
entries
of solutions
A set
A nonzero
one
vector
is ....
space):
kernel
A...
W of R'):
a
[xlp whose
space
A...
infinite-dimensional
set
A
A parametric
of solutions:
description
matrix
....
rank
subspaces
isomorphism:
to 8:
whose
fundamental
In
where
in
B = {b;,...,b,}:
matrix
set
B
basis
An mxn
fundamental
implicit
to the
description
(of a
of W as the set
finite-dimensional
....
(A,B)
coordinates
of
A
....
mapping (determined by an ordered basis
a mapping that associates to each ....
vector
©):
8.
i
of
coordinate
coordinate
from...
created
WVi;.>.,VpasineV
...,
matrices):
rows,
r,
in a variable
equation
A polynomial
auxiliary
of
...
such
that
.
4-38
CHAPTER
4
°
Vector
dependent
linearly
independent
T(x)
Markov
chain:
maximal
linearly
minimal
in W, such
independent
set
that
spanning
set
(for
a subspace
property
[hat
(lf <..,
(of
an
probability
-lt. 2. «4
mxn
vector:
Chen
matrix
NulA
The
Py
of
(of a linear
transformation
rank
(of a matrix
A):
signal
(of
a matrix
(or discrete-time
Span (Vj5..2, Vp:
H and
NulA
of
....
In
entries
a vector
T:V
—
matrix
The
set
space
W):
all
has
the
set
nota-
V .
The
set
of all
P such
that
....
RowA
of
all
...;
also
....
spanned
vectors
denoted
the.
....
by
....
(for a subspace
H):
Any set
{vj,...,vp}
basis
...
for
R” consisting
of
sans
...
such
that
...,
or
the
A...
vector.
In general,
a vector that
nection with a difference equation ....
...
vector
steady-state
3 in V
....
Also,
basis:
The
Fors, 3
(for a stochastic
stochastic
matrix:
A...
submatrix
(of A):
Any matrix
subspace:
A subset
zero
spans
.....
standard
vector
B that
of
set
a
signal):
The.set,
spanning
vector:
A...
A):
set
A set
set
subspace
range
space
H):
in R” whose
matrix:
independent
with
:
:
subspace:
stochastic
together
chen.
proper
regular
V2,-.-.-,
A linearly
A vector
Any
Vi,
ets.
A):
= {
Vo,
(in V):
such
space
state
A
W):
vector
that:
A sequence of ...
vectors
hatrix_P such that...
tion,
row
....
transformation T (from a vector space V into a vector space
rule T:V > W that to each vector x in V assigns a unique
linear
....
that
property
the
with
{v1,...,Vp}+
A set
(vectors):
property
the
with
{v,,...,Vp}
A set
(vectors):
linearly
null
Spaces
matrix
,
P):
A...
vector
V such
that
..
matrix.
H of some
space:
A set
of
subspace:
The
subspace
obtained
by ....
vector
space
objects,
...
called
vectors,
consisting
of
on
....
which
...
....
basis
often
in con-
v such
..
EIGENVALUES
AND
EIGENVECTORS
9.1 EIGENVECTORS AND EIGENVALUES
This section
introduces
eigenvectors
and eigenvalues.
connection with dynamical systems appears
at the end of
A hint
about
the section.
the
KEY IDEAS
Sih cae Sans Set |
In
words,
a nonzero
vector
v
is
an
eigenvector
of
a matrix
A
if
and
only
if
the
transformed vector Ax points in the same or opposite direction of v.
Notice
that
while
an eigenvalue
A might
be zero,
an eigenvector
is
never zero
(by definition).
An eigenspace contains
eigenvectors
together
with the zero vector.
The two equations Ax = Ax and (A - AI)x = O are equivalent.
See Example 3.
The first equation is useful for understanding what eigenvalues and
eigenvectors are, and it shows the geometric effect of the linear transformation xt> Ax on an eigenvector.
The second equation shows that the eigenspace is a subspace
(because it is the null space of the matrix A - AI),
and the equation
is used to find a basis for the eigenspace,
when A is a
known eigenvalue.
The
for another purpose.
second
equation
will
be
used
if
only
again
in
Section
5.2
SOLUTIONS
TO EXERCISES
oer
_ ess
4.
The
number
has
0.
a nontrivial
Compute
2
is
an
eigenvalue
solution.
of
This
A
and
equation
suf i-B J-b 2
if
the
is equivalent
equation
to
(A -
Ax
=
2x
2I)x
=
5-2
CHAPTER
5
«
of A are
columns
The
a nontrivial
7.
Proceed
as
A-
Eigenvalues
in
4I =]
and
and
Exercise
1:
so
3.
cOagoe
2
3
1iSES
HO
eigenvalue
an
2 is
4
Oe
|O
<a
O|
Ovo
&
=
matrix
for
it
si
1
1
6)
Ob
6)
is clear
that
ale
Oe
6)
x =
es |
1
1
This could be checked in
eigenvector,
in the event
row reduce
the
augmented
Oe a=
ee
6)
of A [because
(A - 4I)x
= 0 has
The
coordinates
of
an
eigenvector
satisfy
= O.
The general solution is not requested,
for x3 to produce an eigenvector.
If x3 = 1,
(-1,-1,1).
Checkpoint
1: The answer
Why is it also correct?
Helpful
4
6)
OSS
0
be)
4
4 is an eigenvalue
a nontrivial
solution].
-X; - X3 = O and -xp - x3
so take any nonzero value
then
= 0 has
(A - 4I)x = 0:
all!
0
2eecs
=3
4
Now
2I)x
A.
of
=i
(3
25k
=3
4
We need to know whether A - 4I is invertible.
several ways,
but since we are asked for an
that
one
exists,
the best
strategy
is to
(A -
so
dependent,
linearly
obviously
solution,
Eigenvectors
Hint:
Suppose
you
in
think
the
that
text
is
different,
4
an
eigenvalue
is
namely,
of
a
(1,1,-1).
matrix,
as
in
Exercise 7, and you row reduce the augmented matrix for (A - 4I)x = 0.
If
you discover that there are no free variables, then there are only two possibilities:
(1) 4 is not an eigenvalue of A, or (2) you have made an error.
RS a oy ey
The
wee
ake
Ax=
TRA
equations
4
01n2
=2
0
1
£0
1
Ot
1
for
(A -
I)x = 0 are
Row operations hardly seem
X3 is also zero. There are
1
f=240
Ot
OsdtO
1
Ors
240
i
easy
3
0
pe2s0e@)
+2
6)
to solve:
1
)
{
-2x,
f Satea
necessary.
Obviously x, is zero,
and hence
three variables,
so xg is free. The general
6)
solution of (A - I)x = 0 is xpe9, where eg = | 1], and so @€2 provides a
6)
basis for the eigenspace.
5.1
For
°¢
Eigenvectors
and
Eigenvalues
5-3
A = 2:
So
A - 2I =
4
|-2
Se
[(A - 21)
0] =
2
|-2
—2
-(1/2)x3,
x2
xX,
=
0
1
6)
4
0
i)
2
19v0T
<0
Oe)
(2 caer ©
ens
6)
1
-1
OS
On]
=
x3,
0
Ole
0
with
2c
me
Bl
1Q
1
|
6)
piri gn
aa
Zen
Oe
6)
OF"
1
1
0S
0
1
Oa
0
6)
x,
free.
The
general
Qazl/2.
0
teat
0
6)
Oro
solution
=172
(A -
2I1)x
=
0 is
xs
1
1
A nice
basis
for
the
eigenspace
is
2|
1
EorsA
=
ACs
of
>.
2
3:
4
ts2
ae
Slax
GCA SS1379}20))
0
1
8)
1
O}
1
1
a)-21
ee
°=
346-O0A6
Sar
3
OS3104
6)
-2
OF
it
Of)
=
10
510]
6-3
¢)
.O162s
2)
1
Mie
=2
1
O
\Oai=2)
Oe
0
6)
1
a2
FO
OFZ
1
6)
eZ
Ok
(Oye)
1
10
0.
~A
@)
1
0.
il
st
SOF
0
6)
0
+h
SO
X,
=
-X3,
Xo =
X3,
With
x3 free.
Basis
for
the
eigenspace
is
1
1
Study Tip:
The text’s answer
to Exercise
15 is likely to be the same as
yours,
but there are many answers.
What
should you do if your vectors
differ from those in the answer key?
Example 2 gives the answer.
Whenever
you compute an x that you think is an eigenvector of A, you can check
simply by computing Ax.
There is a little more to do in Exercise 15,
ever.
The
answer
shows
a
basis
eigenspace is two-dimensional.
independent eigenvectors.
You
and
then
their
linear
19.
The
matrix
1
j1
1
S
3/
3
is
two
eigenvectors,
which
means
that
the
So your answer must consist of two linearly
can check that they are indeed eigenvectors,
independence
2
2
2
of
this
how-
not
can
be
checked
invertible
early dependent.
So the number O is
the discussion following Example 5.
an
by
inspection.
because
its
eigenvalue
of
columns
the
are
lin-
matrix.
See
5-4
CHAPTER
5
°
Pa Ne
Carefully
Eigenvalues
read
the
®
ve
See
the
c.
See
the discussion
=
See Example
ical Note.
e.
See
the
Warning
Eigenvectors
definition
of an
of equation
after
Example
box.
Theorem
Matrix
(3).
the
see
Also,
it.
preceding
paragraph
the
eigenvalue.
Invertible
the
before
paragraph
2 and
and
Numer-
3.
23.
then by Theorem 2
If a 2x2 matrix A had three distinct eigenvalues,
there would correspond three linearly independent eigenvectors (one for.
each eigenvalue).
This is impossible because the vectors all belong to
a two-dimensional
vector space
in which any set of three vectors
is
linearly dependent.
See Theorem 8 in Section 1.6.
In general,
if an
nxn matrix has p distinct
eigenvalues,
then by Theorem 2 there would
be a linearly
independent
set of p eigenvectors
(one for each eigenvalue).
Since these vectors belong to an n-dimensional vector space, p
cannot exceed n.
25.
Let x be_a nonzero vector
x = A(A
x). | Since x #
zero.
Then
Note: The
so-called
A
x,
which
shows
that
A
is
an
A (Ax), and
A cannot
be
eigenvalue
of
Ans
5.8.)
fy
Ht
For any A, (A - AD)? era
= (nr)tvswa
sdapet
rSinéer(At--al} Mis inver=
tible if and only if A =XE is invertible (by Theorem 6(c) in Section
2.2),
is
is
33.
x =
Then Al Ax =
invertible),
relation between the eigenvalues of A and Wotis important in the
inverse power method for estimating an eigenvalue
of a matrix.
(See Section
at,
A
such that Ax = Ax.
O (and since A is
we
conclude
that
A
not invertible.
That
an eigenvalue of A.
-
AI
is
not
is,
A
is
an
invertible
eigenvalue
if and
of
A
if
only
if
and
only
A -
AI
if
A
The solution is given in the text.
This exercise is important because
it sets the stage for work later in the chapter.
You ought to spend at
least a little time on Exercise 34, too, even if that is not assigned.
Observe
that the set of all solutions
of the difference
equation
X41
=
AX,
(k =
0,1,...)
sequences
of
vectors
sequence)
is
in H.
in
So,
is
R™.
a subset
H of
The
vector
Exercise
zero
33(b)
shows
the
vector
of
that
space
V of
all
the
zero
H is a subspace
of V.
V
(that
is,
Answer to Checkpoint:
Of course,
because the answer in the text is a nonzero multiple of the eigenvector found in the solution to Exercise 7, and
any nonzero multiple of an eigenvector is another eigenvector.
(The eigenspace is a vector space and so is closed under scalar multiplication.)
5.2
MATLAB
Finding
¢
The
Characteristic
Equation
5-5
Eigenvectors
The linear algebra Toolbox has a command that will simplify your homework by automatically producing a basis for an eigenspace.
For example,
if A is a 3x3 matrix with an eigenvalue 7, then the commands
C
=A - Teye(3)
B = nulbasis(C)
or,
simply,
B = nulbasis(A
- 7*eye(3)),
will
produce
a matrix
B whose
columns form a basis for the eigenspace for A corresponding to A = 7.
In general,
eye(k)
is a kxk
identity matrix,
and nulbasis(C)
is a
matrix whose columns form a basis for NulC
(the same basis you would
get if you started with
ref(C)
and made the calculations by hand).
Remarks:
1.
If the numbers in the basis matrix B are messy,
Which will display the answer to the
nulbasis
with rational entries,
if possible.
try
format
command as
rat; B ,
a matrix
Although MATLAB has more powerful commands for eigenvector calculations,
you should not use them yet, because you need to learn basic
concepts about eigenvectors and eigenvalues.
9.2 THE CHARACTERISTIC EQUATION
a
matrix.
The definition here in terms of the pivots in an echelon form has
vantage that it is easy to state and understand,
and in most cases
vides the most efficient way to compute a determinant.
There
are
several
equivalent
definitions
of
the
determinant
of
the adit pro-
KEY IDEAS
ct
|
When A is 3x3,
the
why det A = O if and
geometric interpretation of
only if A is not invertible:
<=>
The
The
determinant of A is zero.
parallelepiped determined
<=>
One
column
<=>
<=>
The
The
columns of A are linearly dependent.
matrix A is not invertible.
of
A is
in
the
by the
subspace
det A as
columns
spanned
by
of
a volume
A has
the
zero
other
explains
volume.
columns.
5-6
CHAPTER
«+
Eigenvalues
-
det(A
then
nxn,
is
A
If
5
and
Eigenvectors
AI).
=.0
if
only
and
if
now
Al_is
A.-
vertible, and this happens if and only if A is an eigenvalue of A.
familiarity
basic
some
are designed only to provide
1—14
Exercises
as a tool
is
AI)
det(A
of
use
main
The
with characteristic polynomials.
for studying eigenvalues rather than computing them.
A
is defined as det (AI - A).
Sometimes the characteristic polynomial
when
A
even)
or
to make
is
implies
determinants
of
property
are
they
hand
the
so
nxn,
negatives
calculations
det (AI - A)
that
of
one
easier
=
(-1)"det (A -
either
the
same
(when
n
The
of
det (A
- AI)
tends
are
polynomials
two
another.
use
to
copying
and
less
prone
Ca ry
if
|
©). s Sehe aa.
=
Al),
is
errors.
SOLUTIONS
TO EXERCISES
a
cea
1.
ake
A=
E
tic
id,
SF
polynomial
Laka
Aiv= k
A
In factored form, the characteristic
the eigenvalues of A are 9 and -5.
equation
Warning:
Don’t row reduce a matrix A to find
tion preserves the null space of A but not the
”
A=
5
we
ar a
characteris
is
det(A = AI) = (2-A)" = 7 =ehetotiet Ve
7.
"4
Soak the
6
=
rib A
AI
SS
=
TLSeAY
iy
hes
ol:
the
0. =
eee
is
9)(A
(A -
+ 5)
its eigenvalues.
eigenvalues of A.
gal
gs
characteristic
=
Row
;
polynomial
0,
so
reduc-
:
is
det(A - AI) = (5-A)(4-A) - (3)(-4) = 20 - 9A + A* + 12
=A =9A-+ 32
The characteristic polynomial does not
formula provides the solutions of A’ -
factor easily,
9A + 32 = 0.
but
the
quadratic
+9 + V81 - 4(32)
STeenee
These values for A are not real numbers,
so A has no
There
is no nonzero
vector x in R® such that Ax =
(For any x # O in R’, the vector
Ax has only real
could not equal a complex multiple of x.)
Study Tip:
If
tested on them.
real eigenvalues.
Ax for such a AJA.
entries
and thus
you are asked to work some of Exercises 9—14,
you may be
This is one way of finding out if you know what the char-
5.2
acteristic
you
can
LSPA
polynomial
show
ern
is
and
that
you
6-A
|= 2
5
-2
0
9-0
Se
-3=A
e
know
how
some
it
¢
The
is
Characteristic
connected
elementary
with
properties
Equation
o-@
eigenvalues.
Also,
of determinants.
The method using the special 3x3 formula will produce the characteristic polynomial -A
+ 18d"
- 95A + 150.
Factoring such a polynomial
to find the eigenvalues
requires
a little
experience.
(See the appendix at the end of the exercise solutions.)
However,
if you use a
cofactor expansion down the third column (see Section 3.1), you immediately obtain
det(A
-
AI)
6-A
(3 -
-2
a) det [25
ed
(B>="A) P(62A)€9=00tr8]
(Bie aye eS a6 eis)
The characteristic
polynomial
is already partially
factored,
remaining
quadratic
factor
is itself
easily factored.
The
characteristic
-(A
-
3)(A
-
polynomial
5)(A
-
is
(3 -
A)(A
-
10)(A
-
5)
or,
and the
factored
equivalently,
10).
Note: The solutions of Exercises 11—14 are similar to that of Exercise 13.
These matrices
have the property that
if a cofactor expansion
is chosen
along a column or row that
contains
two zeros,
then the characteristic
polynomial appears in a partially factored form.
19.
21.
23.
Since
the
equation
all
A,
set
A = O and
a.
See
Example
1.
b.
See
Theorem
c.
See
Theorem
3.
d.
See
the
If
A =
which
det(A
-
AI)
conclude
=
that
(Ay -
4.
A, =
RQ,
then
A;
=
Q invertible,
and
if
that
A, is similar
to
A.
29.
The
Toolbox
solution
command
is
in the
text.
gauss
will
produce
A)
holds
Q”*QRO
=
for
3.
Example
with
complete
A)s+* (Aq -
of
QR,
The
-
A4Ag°°'Ap.
solution
shows
25.
A)(Ae
det A =
the
echelon
form
See the MATLAB box on p. 1-17.
without row scaling.
see the notes at the back of this Guide.
programs,
Q”* AQ,
in
three
steps,
For
other
matrix
5-8
30.
CHAPTER
5
«
Eigenvalues
In MATLAB, the following
teristic polynomial:
and
Eigenvectors
commands
x =
linspace(0,
3)
Choose
grid
hold
on
on
Include a grid on the graph
Keep all graphs on the same
A(3, 2)=32;
p=poly(A);
100
v=polyval(p,x);
Compute
at
Edit
Appendix:
the
last
line
Factoring
points
graph
the
produce
will
to change
the
the
between
O and
value
3
axes.
plot(x,v,’b’)
characteristic
points
charac-—
one
of
in
of
x,
polynomial,
plot
a and
evaluate
the
curve
in
blue.
the
color
of
the
curve.
a Polynomial
In general
it is difficult
to factor a polynomial
of degree 3 or higher
(unless
you
have
one
of several
powerful
computer
programs
available).
Fortunately,
textbook
examples
and exercises
tend to have small
integer
solutions.
The following observation is helpful.
Let p(A) be a polynomial with integer coefficients.
for some.integer'c,then A‘i- c is a factor of #p(A)=
divisor of the constant term of p(A).
EXAMPLE
Find
mial
Solution
By the observation above, any
divisor of the constant
term
the eigenvalues
is p(A) = -A +
of the matrix A whose
18A° - SBA + 160.
integer
160 in
If p(c) = 0
and «cO isma
characteristic
polyno-
eigenvalue of A
must be a
the characteristic
polyno-
mial.
There (are twenty-four” such divisors:
"fr 2," 4775,
5. 10.
16, 20, 32, 40, 80, and 160, together with the negatives of these
numbers.
We let c be one of these
numbers
and try to divide
A - c into the polynomial
A + 2 don’t work, and the
by long division.
The terms
first successful division is
A + 1 and
Rae
L AG
Re APE + 18A° — S6A + 160
=A ee
Thus
the
A AM
AAS =. SGA
14x = SGA
-A0A.+ 160
-40A + 160
characteristic
polynomial
is
(A -
4)(-A?
+
14a
-
40).
it
5.3
The
quadratic
polynomial
factors
equation is (A - 4)(A - 4)(10
4 (with multiplicity two) and
¢
easily,
- A) = 0.
10.
Diagonalization
and
The
the
5-9
characteristic
eigenvalues
of A are
o
MATLAB:
You can use the MATLAB command
poly(A)
to check your answers
in Exercises 9—14.
Note that if Ais nxn,
poly(A)
lists the coef—
ficients of the characteristic polynomial of A, in order of decreasing
powers of A, beginning with A". If the polynomial is of odd degree, the
coefficients are multiplied by -1, to make +1 the coefficient
of A’.
This
corresponds
to using
det(AI
- A)
instead
of det(A
- AI).
9.3 DIAGONALIZATION
The factorization A = PDP is used
namical
system in Section 5.6, and
forms in Chapter 7.
to compute powers of A, decouple a dystudy symmetric matrices and quadratic
KEY IDEAS
Ct Gee |
Example 3 gives
struct P and D,
the algorithm for diagonalizing
check your calculations:
1.
Compute
AP and
2.
Make sure the
to save time.
corresponding
the dimension
PD,
and
check
that
a matrix
A.
After
you
con-
AP = PD.
columns of P are linearly independent.
You only have to verify that for each
eigenvectors are linearly independent.
of the eigenspace is 2 or 1.
Use Theorem 7
eigenvalue,
the
That’s easy if
The key equation in this section is AP = PD.
It will help you to keep the
order of the factors correct
when you write A = PDP
, and it also
leads
immediately to P AP = D, which you may need occasionally.
Possible test
question:
If AP = PD, explain why the first column of P is an eigenvector
of A (if the column is nonzero).
(Study the proof of Theorem 5. )
Warning:
Do
perty
of
being
5
diagonalizable,
The
is
matrix
eigenspace
not
tn
E
for
confuse
the
invertible.
but
it
eZ
1
is
A =
1 is only
property
of
being
They
not
connected.
is
are
not
invertible,
diagonalizable
invertible
but
it
is
one-dimensional.
not
because
The
O
with
matrix
is
an
diagonalizable
the
pro-
in Example
eigenvalue.
because
the
SOLUTIONS
ee
TO EXERCISES
ees
k
0
-1
Ie
Vilsise
Saitt|-2
37
4
“i D =
A
39/5
As D=
o
711/16
A=
f
For
A°=1t:
mreKe
sip
=
A -“ilT
xe|"7°].
So”
A =
=-1:
2x,
=
with
as
a basis
0,
From
u, and
where
the
bes
bSO
1s
3h
i,
Siler
432
0-SLietel
S
mort
=
E ah
The
equation
(A -
“="(173) xo,
“with
for
ther,
together
226
-525
90
-209
(since
“free:
A is
triangular).
I)x = 0 amounts
The
general
to
6x,
solution
is
vector
for
the
eigenspace
is
uw, =
ee
E
a
The
equation
(A +
I)x
=
The
general
=
the
solution
is
xo|9].
6)
11°
Then
0 amounts
Take
Us
=
to
ey
eigenspace.
Us|
=
1
3
in D correspond
to
wu, and
construct
eigenvalues
x5
;
this
T:
+1
xp free.
ug,
A
Putting
obviously
(=1)r
vector
@)
a
are
basis
A =
16
|0
eigenvalues
x,
A nice
For
St
(@)
=
‘|
Atl-2+
The
OF
4
2
k
6)
121stBiltiG@oo
Next, compute
A= Ppp’, and A* = PD’P’.
+f
1
3
E
2
1. P=
7.
Eigenvectors
and
Eigenvalues
«
5
CHAPTER
5-10
P =
ju,
us,
set
D =
1
0
Osh
respectively.
Warning:
The 3x3 matrices
in Exercises
12—18 may be diagonalizable
even
though each matrix has only two distinct
eigenvalues.
(Theorem 6 gives
only a sufficient condition for diagonalizability.)
You have to check for
three linearly independent eigenvectors.
13.
A=
eZ
1
he
PM
TN
3-1}.
eee
2
The
3x3,
you
need
linearly
For
A = 5:
three
eigenvalues
5
and
independent
1 are
given.
(A - 5I)x
= 0.
Form
A -
A
is
eigenvectors.
-3
Solve
Because
SI
=
1
Soe
Ail
-2
-1
te
and
compute
5.3
=3
P
=-1
So
X,
=
ou
=2%
=1
-=2 ° =3
-Xg3,
X2
6)
OWS
0
=
-X3,
a basis vector for
is diagonalizable.
1? veep
73
25 <4
21 2 F<2
2849
~o
with
x3
0
OV
6)
free.
HISe
Take
¢
es
Diagonalization
1
FO
6)
6)
1
6)
1
1
0
=I
|-1],
1
for
instance,
xx
v,; =
5-11
6)
6)
6)
as
At this point, you don’t know if A
is to find two linearly independent
eigenvectors
inside
the eigenspace
for
A = 1, because
there
are
no
other eigenspaces in which to look.
For
A=
1:
Form
the eigenspace.
Your only hope
inecee
|1
2
=
lose
A- I=
-1].
1
The
equation
(A - I)x = O reduces
to xX, + 2Xp - X3 = O.
So xX, = -2xo + X3, with xo and x3 free.
At this
point you know that the eigenspace
is two-dimensional
(because
there
are two free variables).
So there are the necessary two linearly independent
eigenvectors,
and hence A is diagonalizable.
To produce
the
eigenvectors, write the solution of (A - I)x = 0 in the form
Xi)
2
|-2xXatx3
>:GM |
Xo
XB
XB
2,
Set
vo = |] 1],
=2
=
Xo|
1]
1
+
x3]
0
0
ar
and
P= [v,
nl
vg =
¢)
|O],
1
ed)
vo
vg] =
{-1
1
2
1
1dncOis
6)
1
The columns of P are linearly independent,
by Theorem 7, because
eigenvectors
form bases
for their respective
eigenspaces.
SOs
invertible.
Since
the first column of P corresponds
to A = 5,
first diagonal entry in D must be 5, which means that
D=
Warning:
the usual
S
0
|0
1
OF — 0
the
sis
the
r50
0
1
A common mistake
way and then take
13 is to build
in Exercise
D to be a 2x2 matrix, such
a 3x3
as
matrix
P
in
5-12
Of
CHAPTER
this
course
5
°
Eigenvalues
doesn’t
make
and
Eigenvectors
PD
because
sense
isn’t
Another
defined.
even
is to make a 2x2 matrix D as above and build
mistake
say one from each eigenspace.
the three eigenvectors,
is a 3x2 matrix and is not invertible.
P with
Now AP
only two of
= PD, but P
Study Tip: Exercise 18 is a good test question.
Try it.
Note:
the eigenvector (-2,1,2) does not correspond to the eigenvalue A = 5.
If you have
trouble here, review Exercises 5—8 in Section 5.1.
10:
7A
For
3
aa
0
9
(@)
3
gt
1
fo
-2
0
0
0
2
A = 2:
aol
To solve
ee
:
eigenvalues
Xs
a
31/0
0.
0
+0
1
6.0
6304
_
mia
°- "Xa
|~X3
Pee
X4
x5
| ee
=
X3/
-1
=1
=p
2
X4
4
ree
0
A = 3:
To solve
Caio
10
8)
6)
vg
0?
0-1.
Of.
OO
Choosing
(A - 3I)x
xg = 2 produces
= 0,
[0
the
reduce
=
[(A-2I)
1
jH0
6)
0
6)
1
O
6)
xq, free.
(Why?)
0]:
1
1
Lee
0
6)
0
0
The
6)
6)
6)
6)
usual
calcu-
at!
Hee
Basis:
me
=f
ee |
vy =
a
v2"
0
completely
reduce
v3 =
(3,2,0,0).
2
0
1
as
in this
to determine
[(A-3I)
1393/2000
3 Omen
0
1
6)
0
8)
OT
0
1
(oh ie
8)
0 . (O°
S00
eigenvector
a:
and 2.
to choose
A = 2 first,
information at this point
Nee
O
eh
740470
met
0
0
5,3,
1
Checkpoint:
If you happened
solution,
would you have enough
whether A is diagonalizable?
For
3
Seu
Lrenga
Oi
gin
0)
6)
Osn60
0
-X3 - Xq4, X2q = —-X3 + 2X4, With xg and
produce a basis for the eigenspace:
X4
X2|
obviously
(A - 2I)x = 0, completely
38033 sx ObnaSan
8
0
1
dio Geren
Oyo
0
0: op 0.
-eOnend
6)
O'
OsfdO<!t
fO
So X, =
lations
are
X,
Xq
X3
X,
0]:
= (3/2) xo
is free
= 0
= O
5.3
For
A = 5:
To solve
@53
6)
g
0
0
OP
8)
are:
OSE
0
3
8)
6)
vector
for
the
A basis
Poe ynbivar
This
(A - 3I)x = 0, completely
0.
Ones2vtalian®
answer
¢
6)
o| |
differs
thes
Omme®
8)
6)
x2 = 0
0
6)
6)
0
0
0...
1
{0
Cie
6)
x3 =
X, =
er
=a
ad
2
3
PZ
1
0
6)
1
6)
0
Oo.
that
SOM
in the
The
symbol
See
the
Check
wc See
Se
ee
D does
remark
out
the
Example
not
after
ey
0
6)
2
@)
8)
6)
6)
6)
O
0)
5
P =
[vg
= ocean
ah
text.
There,
matrix
the new
Theorem,
v3
denote
a diagonal
the
of
Diagonalization
in Example
the
vi
order
both
automatically
statement
0
O
Set
and the entries
in D are rearranged
to match
eigenvectors.
According to the Diagonalization
are correct.
el.
xX, is free
(1,0,0,0).
Ol
Sz12
0]:
6)
is vg =
from
[(A-5I)
1
eigenspace
1
reduce
6)
qalOs0c0d
Syst val -
Diagonalization
Vol,
of the
answers
matrix.
Theorem.
4.
4.
23.
A is diagonalizable
because
you know that
five
linearly
independent
eigenvectors
exist:
three
in the three-dimensional
eigenspace and two
in the two-dimensional
eigenspace.
Theorem 7 guarantees that the set
of all five eigenvectors is linearly independent.
25.
Let
{v,}
be
a basis
for
the
one-dimensional
eigenspace,
let
v2 and
v3
form a basis
for
the
two-dimensional
eigenspace,
and
let
v4 be any
eigenvector
in the remaining eigenspace.
By Theorem 7, {v1,
V2, V3, Va}
is linearly independent.
Since A is 4x4,
the Diagonalization Theorem
shows that A is diagonalizable.
27.
If A is diagonalizable,
then
A = PDP
for
some
invertible
nal D.
Since A is invertible,
O is not an eigenvalue
of
diagonal entries
in D (which are eigenvalues of A) are not
By the theorem on the inverse of a product,
is invertible.
Ae
Since
ppp)
= (Pe
D’ is obviously
ede
diagonal,
= PDP
A" is diagonalizable.
P and
diago-
A.
So the
zero,
and D
A Second
A
If
Proof:
is
29.
Diagonalization
by the
1941
[3 a
Although the first
the eigenvalue 3,
multiple
three
of
Theorem.
So. inter=in D.
from those
make them correspond properly
different
gg
[5 s
abe n=
column of P must
there
is nothing
bel: say
eigen-
3
OfacA
vi,...,V, are also eigenvectors
diagonaliis
A
Hence
5.1.
in Section
in D, are reversed
entries
The diagonal
change the (eigenvector) columns of P to
In this case,
to the eigenvalues in D,.
a=
independent
linearly
n
has
it
nxn,
Then
«3 45 Vni
eVi5.
Say,
veetors,i
by the solution of Exercise 25
zable,
Eigenvectors
and
Eigenvalues
«+
5
CHAPTER
5-14
igh
and
factorizations
or
be
to
an eigenvector corresponding to
prevent
us from selecting some
letting
Po
=
i
eh
"diagonalizations"
of
We
now
have
A:
A = PDP = P,D,P;_ = PoD,Pa°
Answer to Checkpoint:
Yes.
In Exercise
19, the fact that the eigenspace
for A = 2 is two-dimensional
guarantees that A is diagonalizable,
because
each of the other two eigenvalues
will
produce at least
one eigenvector,
and the resulting set of four eigenvectors will be linearly independent,
by
Theorem 7.
So A is diagonalizable,
by the Diagonalization Theorem.
(Note
that since one eigenspace is two-dimensional,
the other eigenspaces must be
one-dimensional,
because
there could not possibly be more than four
lin-
early
independent
Mastering
Linear
eigenvectors
Algebra
I suggest
that you
terms listed above.
inR.)
Concepts:
Eigenvalue,
Eigenvector,
take time now to prepare
review
Begin with their definitions,
of
Eigenspace
sheets
course.
for
the
three
page
307
Eigenvalue:
> equivalent
description
Sec.
5.1:
Equation
(3);
Box
on
> geometric interpretation
>» special cases
> typical computations
> connections with other concepts
Sec."S.1) Fig. 2
Theorems 1, 6
Sec. 5.1:.Exer. 7, 19; Sec. S.2: Exer. 7, 15
The IMT; Theorem 4; Sec. 5.2: Exer. 19
Eigenvector:
> special cases
> typical computations
> connections with other
Theorem
concepts
2
Sec.
5.1:
Examples
Sec.
5.1:
Exer.
Sec.
5.3:
Theorem
2, 4,
Exer.
5, 15
Sec.
5.2:
Example
33;
5,
Exer.
5, 18
5
5.4
>
>
>
>
°
Eigenvalues
Eigenspace:
equivalent description
geometric interpretation
typical computations
connections with other concepts
MATLAB
and
Linear
Transformations
Sec. 5.1: Equation (3)
Sec. 5.4: Fig.22,.3
Sec. 5.1: Example 4
Theorem 7
eig
The command
ev = eig(A)
produces a vector ev that
ues of A.
To learn the diagonalization procedure,
method of Section 5.1 to produce the eigenvectors.
is 5x5, the command
V = nulbasis(A
lists the eigenvalyou should use the
For instance,
if A
- ev(1)*eye(5))
creates a matrix whose column(s)
give a basis for the eigenspace
responding to the first eigenvalue of A listed in ev.
In later work,
you can automate
the diagonalization process.
command
[P
D] = eig(A)
produces
other
names
you choose)
is invertible,
then A is
is
the
zero
matrix.
you
construct
for
"unit"
5-15
vectors
In
the
two
square
such that
AP =
diagonalizable.
any
case,
homework.
(studied
in
P
is
P and
The
D (or any
PD,
with D diagonal.
LiAve
Check whether
A - P*D*inv(P)
likely
The
columns
Chapter
6).
entries may make some of the new entries
blems in the text usually involve simple
matrices
cor-
of
to
be
different
P are
scaled
so
they
P
one
of
its
because
the
pro-
Dividing
recognizable,
numbers.
by
from
what
are
5.4 EIGENVALUES AND LINEAR TRANSFORMATIONS
This section introduces
the matrix of a linear transformation relative to
specified bases for vector spaces, and then uses this concept to give a new
interpretation of the matrix factorization A = PDP
.
STUDY NOTES
oar
ge poe
The
exercises
will
help
you
learn
the
definition
of
the
matrix
representa-
tion of a transformation relative to specified bases, say B and ©.
Compare
this definition with the standard matrix of a transformation from R" into
R". What are B and © in this case?
After Example 1 (and Exercises
1-6 and 9-10),
the section focuses on
The following algorithm and the solution
the case when T:V:— V and B= &.
to Exercise 7 describe two different ways to construct [T]..
5-16
CHAPTER
6
Algorithm
1.
Compute
«
Eigenvalues
and
for
Finding
the
B-matrix
of T:V
the
images
of
the
vectors:
T(by),
2.
3.
Convert
Place
these
the
Eigenvectors
basis
—
V
. ¢-endobba)
images
into
B-coordinate
[T(bi)]g,
--->
[T(bn) lp
B-coordinate
vectors
into
vectors:
the
columns
of
[T],.
The sentence before Theorem 8 summarizes the main idea of the theorem.
Studying the proof should help you understand the theorem and review important
concepts.
The
subsection
on Similarity
of Matrix
Representations
could have contained more ideas.
Take notes carefully if your instructor
decides to expand this material somewhat.
SOLUTIONS
TO EXERCISES
SE
1Pe
T (b,)
=
3d,
-
5do,
[T(bylp = Ey
=
-d,
3°
=-5
D:
6do,
T (b3)
+
agt
—— ——>
3a
=
Ado
[T(ba)ly = A
C= 1
6
The method of Example 2 works.
But,
that the information given provides
in the figure below:
2
T
ao + a,t
+
[T(ba)Ip = ha
Matrix forT
relative to B and
7.
T (bo)
0
4
perhaps a faster way is to realize
the general form of T(p) as shown
+
(Sap
=
2a,)t
coordinate
+
(4a,
3
ao)t
2
coordinate
mapping
mapping
a
0
a4
ao
multiplication
a ipedbia(
is
B
a
ep
ee
5ao
ze 2a,
4a,
+
The matrix that implements the multiplication
figure is easily filled in by inspection:
ao
along
the
bottom
of
the
5.4
te
?
?
?
?
?
?
?i]la:|
leo
¢
ao
Eigenvalues
and
Linear
Transformations
3a
3
implies
[Sap - 2a,]
Ma, + ao
=
O
|5
Of
ITl,.=
that
5-17
O
25-0
<Aiey4
Study Tip:
See the Study Guide
Exercise 19).
This method allows
notes for Section
1.8 (particularly
you to find a matrix that implements
mapping
linear.
without
assuming
that
T
is
In
fact,
this
as
a matrix
that T is linear, because it is now represented
Why not try this method on Example 2?
13.
Start
2
by diagonalizing
FODlA.
=
A
- 4A + 3 =
120A
=
;.
1)(A - 3),
so
the
le
The
[=
Xo,
with
xp free.
FORMA
=)3:%ArN=S3I
=
cs
:.
Xo
x;
=
vector.
0.
The
So
vectors
X29/3,
u,,U2
AS
The
with
can
characteristic
eigenvalues
(A -
a basis
equation
xp free.
form
the
are
proves
transformation.
polynomial
1 and
3.
I)x = O amounts
vector,
take
(A -
=
Take
I)x
up
columns
=
of
u,
to
=
-x,
El
O amounts
EF as
a
is
a
to
basis
matrix
P
that
diagonalizes A.
By Theorem 8, the basis B =
that the B-matrix of the transformation xt Ax
{u,,us} has the property
is a diagonal matrix.
If
invertible
A
is
similar
to
B,
then
that P AP = B.
Thus B is
vertible
matrices.
By a
Pta'cp')!
eke
The
equation
=
=
x,
:.
0.
+
So
es
+ Xp =
-3X1
19.
(A -
A =
derivation
for
the
By
= p'a'p,
hypothesis,
Q@ co
=
A.
there
Then
there
exists
an
matrix
P
invertible because it is the product
theorem
about
inverses
of products,
such
of inB
=
which shows that A’ is similar to B?.
exist
P' BP
invertible
=
QO CO.
multiplying by Q-°, we have
P and
Q such
that
P BP
Left-multiplying
by
Q,
QP BPQ” = QQ CQQ°.
and
=
A and
right-
So C = QP BPQ
=
(Pq *)"*B(PQ”'), which shows that B is similar to C.
23.
If Ax = Ax,
x # 0,
then
p tax = aPuix.
If (B=
PAP,
then
(*)
B(P'x) = PUAP(P‘x) = PAx = APx
Note
that Px
#0,
because
x # 0 and P” is
by the first
calculation.
invertible.
sponding to
(*) shows that P x is an eigenvector of B corre.Hence
A is an eigenvalue of both A and B because
(Of course,
A.
the
matrices
are
similar,
by Theorem
4
in Section
5.2.)
5-18
CHAPTER
6
25.
If A = PBP,
tr(A)
«+
Eigenvalues
and
Eigenvectors
then
tr((PB)P)
=
tr(P"'(PB))
By the
trace
property
tr(P 'PB) = tr(IB) = tr(B)
If B is diagonal,
then the diagonal entries of B must
ues of A, by the Diagonalization Theorem
(Theorem 5
So trA = trB = {sum of the eigenvalues of 4A}.
be the eigenvalin Section 5.3).
Study Tip:
It can be shown that for any square matrix A, the trace of A is
the sum of. the eigenvalues of A, counted according to multiplicities.
You
can use this fact to provide a quick check on your eigenvalue calculations.
The sum of the eigenvalues must match the sum of the diagonal entries in A
(even
29.
if A is not
diagonalizable).
If B= {b,,...,b,},
then the B-coordinate
dard basis vector for R".
For instance,
by, =
Thus
1:by
+
O-bo
[I(b)],
=
[bj]
Lilo
Appendix:
The
+
*-:+
=
ej,
+
vector
of
b; is
eal
B
ej,
the
stan-
Orb,
and
LUMPLe tt
lal
Characteristic
Polynomial
ae len
of
a 2x2
Matrix
By Exercise 19 in Section 5.2, det A is the product of the eigenvalues of
A.
From the Study Tip above, trA is the sum of the eigenvalues.
Thus,
if
a 2x2 matrix A has eigenvalues
A; and Ag, the characteristic
polynomial
of A has the form
UAgt =a Caste ON) me A
cus |Semen ef
(As) + An eee
= r* - (tr A)A + (det A)
5.5
9.9
¢
Complex
Eigenvalues
COMPLEX EIGENVALUES
If the characteristic
equation of an nxn
real
matrix
A
eigenvalue
A and if v is a nonzero vector in C" such that
both A and v provide useful information about A. |
KEY
5-19
has a complex
Av = Av, then
IDEAS
Fae
Only matrices
with real entries
are considered
here.
If A is a complex
eigenvalue
of A, with v a corresponding
eigenvector,
then A is also an
eigenvalue of A, with v an eigenvector.
Find out if you should know how to
prove this fact.
Example 6 describes the prototype for all 2x2 matrices with a complex
eigenvalue A.
Only A is needed if you want to know the angle 9g of rotation
and the scale factor
|A|, but an associated eigenvector v is also needed if
you want
to factor
A as PCP
SOLUTIONS
wa
TO EXERCISES
ee
, as in Example
7.
fi -2sh A ye ar eegull
ae
1 a= _ [i
= |"che a
det(A - AI) = (1 - A)(3 - A) - (-2) = A?- 4A+ 5
Use
the
Vag Aig
Example
For
quadratic
the
and
= ey
eigenvalues:
oul
A =
=
Example 2 gives a shortcut
for finding one eigenvector,
5 shows how to write the other eigenvector with no effort.
(2+i)I
find
VY
=
A-
to
+
ave
A= 2+i:
formula
The equation
(A - AI)x = 0
gives
(adn)
Xe
2X2 = O
Kae tele =
As in Example
same relation
i) Xo
2, the two
between x;
0
equations are equivalent —each determines the
and xp. So use the second equation to obtain
The
xg free.
-
i)xg,
with
the
vector
vi =
iF 4 ‘|provides
For
A = 2-i:
X, = -(1
shows
that
vo
Let
vo
=
vi =
is
automatically
general
a basis
k i af
an
solution
for
The
the
+
7
eigenspace.
remark
eigenvector
-
is xa ; 1 ‘le and
for
prior
to
2 + i.
Example
in,
tac,
5
Eigenvectors
and
Eigenvalues
«+
5
CHAPTER
5-20
calculations similar to those above would show that {vg} is a basis for
(In general, for a real matrix A, it can be shown that
the eigenspace.
in a basis of the eigenthe set of complex conjugates of the vectors
space for A is a basis of the eigenspace for A.)
a
are
permitted
if you
The
5 |
pas
are
to
down
the
eigenvalues
to find
them
via
write
expected
that
the eigenvectors
are
easy
to
Problem.
The scale factor associated with
r=
|All =
point
( (v3)? 5ai
(a,b)
=
secs
(V3,1)
= 2.
in the
the
characteristic
remember,
For
too.
equation.
Note
the
Practice
the transformation
xt Ax
the angle of the rotation,
is simply
plot the
xy-plane
and
use
See
or
memory,
Ei iA from
of
if you
instructor
Ask your
V3 + i.
are
eigenvalues
trigonometry.
2
13.
From
to
Exercise
A=
Then
2
-
1,
i.
i
A =
2 + i,
Since
Rev
g =
mt/6
and
the
=
~4
A
radians
eigenvector
and
Imv
=
af
O|’
ae
take
P
corresponds
=
~1
71
i
oO]
compute
Sonstar
mol callt95|[ool
Bie? |
Beal
be?
eee
*fede Samah alae be
le te
eae
a
Actually,
Theorem 9 gives the formula for C.
v corresponds to a - bi instead of a + bi.
the
v =
eigenvector
of the eigenvalue
for
2 +
is the
i,
your
C will
(1,2)-entry
be
Ne
vO Wy ese
Stat
ys
[Pea
cy |
Note that the eigenvector
If, for instance,
you use
=|
The
imaginary
part
in C.
So there are two possible choices
for C (depending on the vector
used to produce P).
On an exam,
if you are not sure of the form of C,
you can always compute it quickly from the formula
C = P AP, as in the
solution above.
Note:
Because there are two possibilities for C in the factorization of a
2x2
matrix as in Exercise
13, the measure of rotation g associated
with
the transformation x+> Ax is determined only up to a change of Sign.
The
"orientation" of the angle is determined by the change of variable x = Pu.
See Fig. 4 in the text.
ioe
. 56
“a7
4 |: det (A ---AI)
=
A
2
-
5.5
¢
Complex
1.92A
+
1.
Eigenvalues
5-21
Use the quadratic formula to solve A“ - 1.92A + 1 = 0:
A=
To
find
1.92 + v-.
owe
So. 9Oee
+21
the
for
A =
.96
(.4
=
(1.52
eigenvector
-
.96-+
is
a nonzero
you can use
equation,
Ohl
Since
This
25.
+
solve
-.7X2q
=
0
+.28i)xo
=
0
(because
.96
equation
to
the
find
-
.28i
ahh
.5i) x9
xa = 2 produces
the
(complex)
eigenvector
=
bial take
P =
=
Hl and
Imv
tee
aoc
(CO)
5|-2
eel |i a
All . 56
final
matrix,
which
is the
has
Complex
Eigenvalues
If Ais
a 2x2
matrix
values,
you can
vector v. Then,
the
and
if
part
eigenvalue),
From
v =
|: “El
the
real
and
imaginary
same
size
as
parts
of
so
second
f : Ak
Finally,
compute
eig(A)
form,
is C.
Ax = A(Rex) + iA(Imx).
Thus A(Rex)
is the real
Since A
part of
of Ax.
returns
a pair
of complex
eigen-
use
nulbasis
to find a corresponding complex eigenbuild a 2x2 real matrix P from the real and imaginary
parts of v and compute C = P AP.
For any matrix V, the commands
/.
proper
we have
A(Imx).
imaginary
MATLAB
an
ar i See
ee oS
mle
| = EE
e4
Writing x = Rex + i(Imx),
is real, so are A(Rex) and
A(Imx)
is
solution.
c,
Rev
Ax and
.96
solution
either
re
Setting
.28i,
.28i) x4
. 56x,
There
-
the
real(V)
entries
in V,
and
imag(V)
displayed
as
produce
the
matrices
the
5-22
CHAPTER
5
«+
Eigenvalues
and
Eigenvectors
5.6 DISCRETE DYNAMICAL SYSTEMS
in
began
that
of ideas
the climax to a crescendo
through parts of Chapters 4 and 5. You need not have
in this
the interesting applications
to appreciate
profit from a review of page 302 and Example 5 in
This section presents
Section 1.9 and flowed
read all the material
but you will
section,
Section 5.2.
KEY IDEAS
a
Bh
A solution
of
Xx+1
is
a
a first
=
sequence
order
AX,
612605
{x,}
that
homogeneous
r1),
difference
equation
23980)
satisfies
(1)
(1)
and
is
described
by
a
formula
for
each x, that does not depend on the preceding terms in the sequence other
than the initial
term Xp.
In Section 5.1, you saw how a solution can be
constructed when Xp is an eigenvector.
When Xo is not an eigenvector,
look
for an eigenvector decomposition of Xo:
XQe =
C1
Ae
tenn
Each
Vy
is
an
eigenvector.
(2)
To make (2) possible for any xp in R", the section assumes that the
matrix A has n linearly independent eigenvectors.
If x, = A Xo, then
XxX, =
When
{x,}
C1(Aq)
"vy 5
describes
the
MOTOS SC
Cn (An) “Vn
"state"
of a system
= 0,1,2,...),
the long-term behavior of
tion of what happens to x, as k > o.
important situation.
Let
A be
an
nxn
matrix
nxn
with
n
at
this
The
linearly
discrete
times
dynamical system
text focuses
on
independent
(denoted
by k
is a descripthe following
eigenvectors,
cor-
responding to eigenvalues such that
|A;| 2 1 > |A;| for j = 2,...,n.
If Xo is given by (2) with c, #0, then for all sufficiently large k,
Xx+1
xX,
* AX,
*
C1(A,)
Each
k
Vi
X,
entry
is
ratio
the
in X, grows
approximately
between
same
as
any
the
two
a
by a factor
multiple
entries
corresponding
of
in
of
Aj.
V,,
and
so
X,
is
nearly
ratio
for
Vi.
the
5.6
¢
Discrete
Dynamical
Systems
5-23
STUDY NOTES
‘Gri
ie, |
When the two approximations
above are true in an application,
the eigenvalue A; and the eigenvector v; have interesting physical interpretations.
Make sure you can describe these on an exam.
(See the last four sentences
in the solution of Example 1, for instance. )
The
predator-prey
model
is
rather
primitive
and
provides
only
a
starting point for more refined models.
Still, you might enjoy considering
what the model in Example 1 predicts if xg happens to be a multiple of vo =
(5,1),
or if initially there are more than 5 owls for every 1 thousand
rats, assuming p = .104.
The graphical descriptions of solutions to difference equations should
help you understand what can happen to x, as k > ».
I hope you enjoy studying the figures even if your class does not have time to cover this part
of the section.
Only the simplest cases are shown,
but these cases form
the foundation
for studying nonlinear dynamical
systems
which are widely
used
(but require
calculus
techniques
not covered here).
Even for nonlinear systems,
eigenvalues
and eigenvectors
of certain matrices
play an
important role.
SOLUTIONS
TO EXERCISES
SE
1.
a.
The
eigenvectors
find
the
action
That
is,
find
Rt.
Gia!
Since
b.
Each
ci,
v,
of
=
[| and
A on
Xo
co such
=
v2
=
bey form
Fak express
that
Xp =
civ;
<O] Coldte
Oe 95i]t a
tlie Of elieked| are
eigenvectors
xX, =
AXo
5Av,
-
po
15
(EY
e463
4/3
Pg
on
a
linear
combination
time
A acts
(for
the
= 5:3v,
-
eigenvalue
in
for
terms
of
eigenvalues
3 and
X, = 5(3)*v,
To
and
Vo.
v,
term
1/3):
4:(1/73)vo
(ne 4)
41/3
3 and
the
of
v,
v2
and
term
vo,
is
the
multiplied
Xo = AX, = ALS(3)v1 - 4(1/3)vel = 5(3)°v, - 4(1/3)%v2
In general,
v,
R*.
ie ened
ae Ptah 8e
v2 are
is multiplied by the
the eigenvalue 1/3.
basis
+ CoVo:
vi,
4Avo
Xo
a
- 4(1/3)*vo
for
k=
0.
by
5-24
7.
CHAPTER
5
«¢
Eigenvalues
and
Eigenvectors
a.
The
and
matrix A in Exercise 1 has eigenvalues 3 and
|1/3| < 1, the origin is a saddle point.
b.
The
direction
attraction
greatest
of
1/3.
;
determined
is
|3|
Since
by
mil
hii '
=
v2
> 1
the eigenvector corresponding to the eigenvalue with absolute value
The direction of greatest repulsion is determined by
less than 1.
c.
v=
Bt
than
1.
The
drawing
origin,
(2)
corresponding
eigenvector
the
below
shows:
arrows
(1)
toward
lines
the
through
origin
greater
eigenvalue
the
to
the
eigenvectors
and
the -
(showing
attraction)
on
the
line through vz and arrows away from the origin (showing repulsion)
on the line through v1,
(3) several
typical
trajectories
(with arrows)
that show the general
flow of points.
No specific
points
other than v; and ve were computed.
This type of drawing is about
all that one can make without using a computer to plot points.
X2
V2
V1
X4
Remark:
Sketching trajectories for a dynamical system in which the origin
is an attractor or a repellor is more difficult than the sketch in Exercise
7.
There has been no discussion of the direction in which the trajectories
“bend" as they move toward or away from the origin.
For instance,
if you
rotate Figure 1 of Section 5.6 through a quarter-turn and relabel the axes
so that x, is on the horizontal
axis,
then the new figure corresponds
to
the matrix A with the diagonal entries .8 and .64 interchanged.
In general,
if A is a diagonal matrix, with positive diagonal entries a and d, unequal
to 1, then the trajectories
lie on the axes or on curves whose equations
have
the
initial
form
point
McGraw-Hill,
xp =
r(x,)°,
Xp.
(See
1992,
pp.
where
Encounters
s
=
with
(Ind)/(lna)
Chaos,
by
and
Denny
r
depends
Gulick,
New
on
the
York:
147-150.)
Study Tip:
If your
instructor
wants you to graph
origin is an attractor or repellor,
there will need
cussion of exactly how to do this.
trajectories
when the
to be some class dis-
5.6
13.
8
.A
A=
:
1 a
First
find
¢
the
Discrete
Dynamical
1.2)(A
-
5-25
eigenvalues:
det(A - AI) = (.8 - A)(E5 - a) - (.3)(-24)
(A -
Systems
1.1)
Use
the
2a” = 2.3 $432
quadratic
formula,
if
needed.
0
Since both eigenvalues,
1.2 and 1.1, are greater than
a repellor.
For the direction of greatest repulsion,
vector for the larger eigenvalue,
1.2:
[(A
25)
Any
multiple
est
repulsion.
MATLAB
Plotting
0)
of
over
and
8
eS
3
such
as
Bae
Discrete
Given a vector x,
on the trajectory.
mand,
a
&
x
This
A*®x;
in
0
4;
determines
the
Bhs
3/4
ape
direction
of
great-
will compute the "next" point
and <Enter> to repeat the com-
x]
Store
the
line
"trajectory" matrix T whose columns are
(Change 15 to any integer you wish.)
the
loop
Compute
End
type
374
0
over.
Put
you
,
OO
m
the command
x = A*x
Use the up-arrow (*)
=1:15
After
a=
E
Trajectories
The following steps create a
the points x, Ax, Ax,...,A
x.
{T
0)
3
1, the origin is
find the eigen-
the
the
of
the
first
repeats
next
new
column
of
T.
the
next
two
point
on
the
point
lines
times.
in T.
loop.
with
"for",
MATLAB
calculations
(while you type additional
If you want MATLAB itself to plot the
beginning
lines)
points
until
in T,
plot(T(1,:),T(2,:),’ow’),
15
trajectory.
will
you
use
suspend
all
type "end".
the commands:
grid
The ’o’ produces a small circle at each point on the trajectory.
The
’w’ makes the circle white.
If you have the data for another trajectory
stored in a matrix S, you can plot both trajectories on the same graph:
plot(T(1,
Each
new
"plot"
-JyT(28:95
command
ow’ 5 SG1s2),
erases
the
S(2,:),
#2),
previous
graph.
grid
g
is
for
green.
5-26
CHAPTER
5
«
Eigenvalues
and
Eigenvectors
5.7 APPLICATIONS TO DIFFERENTIAL EQUATIONS
in
the material
equations,
in differential
If you plan to take a course
studied
already
have
you
If
reference.
valuable
a
be
will
section
this
you may gain new understanding as you work through
differential equations,
this section.
KEY
IDEAS
ee iad |
A basic
solution
x(t)
yet,
=
of
the
differential
A
is
where
an
equation
eigenvalue
of
A
x’ = Ax
is
an
eigenfunction
and
is
a
corresponding
v
eigenvector.
In all examples and exercises in this section,
every solution
of x’ = Ax is a linear combination of eigenfunctions.
(This is because A
is diagonalizable.
The general case is usually handled in a full course in
differential
equations.)
An
initial
condition,
x(0)
=
xo,
determines
the
weights for the linear combination of eigenfunctions.
The
eigenvalues
of
A determine
the
nature
of
the
origin
for
the
dynamical system described by x’ = Ax.
Most of the discussion involves the
case when A is 2x2.
If one eigenvalue
is negative and one is positive,
the origin is a saddle point.
If the real parts of the eigenvalues
are
negative
(this includes the case when both eigenvalues
are real and negative),
the origin is an attractor of the dynamical
system.
If the real
parts are positive,
the origin is a repellor.
If an eigenvalue is complex,
then the trajectory of a corresponding eigenfunction forms a spiral —either
toward
the origin,
away
from
the
origin,
or
on an ellipse
around
the
origin,
depending on the real part of the eigenvalue.
Study Tip:
Section 5.6.
Note that conditions on eigenvalues here differ from those in
For differential equations,
the real parts of the eigenvalues
determine
the nature
of the
trajectories;
for
difference
equations,
the
absolute values of the eigenvalues are important.
You can remember this if
you
note
if
the
real
Xk+1
=
AX,
shan
that
a basic
part
of
tends
to
solution
A
0
vet
of
x’ = Ax
is
negative.
In
(as
k > »)
only
if
for
x’ = Ax
are
vier"
tends
contrast,
the
a
to
0 (as
basic
absolute
t > »)
solution
value
of
The
general
only
atv of
A is
less
ts
SOLUTIONS
TO EXERCISES
eS
1.
The
eigenfunctions
of x’ = Ax has
bee
iy
(Be
in
form
the
rae
ieee
are
and
Sones
solution
5.7
The
initial
°
condition
Applications
x(0)
=
(-6,1)
Oly
Thus c, = 5/2,
Checkpoint:
eckpoint:
Let
tality foe tle ater
=3 Fit .Fe
cs = -3/2,
A
be
and
[le
On
c, and
Equations
co:
s0en ore
“4-872
J
372
ee
ei
|: obtained
by
matrix in Exercise 3 by 2.
Now the eigenvalues of A are
the origin an attractor or a saddle point for the equation
7.
Use
the
create
P =
[v1
in the
v,
vo].
aI: The
lo
given
eigenvectors
Match
details
answer
=
BE and
v2
=
re (found
in
the
.5 and -.5.
x’ = Ax?
Is
Exercise
eigenvectors
with
of the
substitution
of x = Py into
of
the
the
dividing
the
section
5-27
x(t) = S|alee" - S|ile
i
matrix
the
Differential
determines
ath? afte a hd
& ~Lie
Hh ag lie
to
eigenvalues
5)
to
in
D
x’ = Ax are
text.
Helpful
Hint:
The
idea of changing variables
to uncouple
a differential
equation
is fairly common
in engineering texts.
Exercises
7 and 8 test
your understanding of the value of a diagonalization.
(You might see such
a question on an exam.)
13.
An
eigenvalue
complex
space
of
A is
eigenfunctions
of
solution
all
complex
A =
vet
1 + 3i,
with
and
ve!
provide
solutions
of
x’
the
solution.
=
a
Ax.
v =
(1+i,
basis
for
The
general
2).
the
The
solution
(complex)
is
c1|’ E plement F; ca’ : se see
Use
eigenvector
imaginary parts
1+3i)t
as
i)
Rewrite vee
real
’ ’ gee
and
*
’ Z ‘|(cos
of
vet
(Sates
ade
to
of + i sin 3t)e!
boa rat,
build
the
general
real
5-28
CHAPTER
5
¢
Eigenvalues
»
general
real
ecos=3f
|
2 cos 3t
solution
-Psin
Shy
2 cos 3t
Eigenvectors
(eoseStiersinyst?
:
The
and
has
the
t
sin
-
SbatecesiStyit
2 sin 3t
form
sin
Je ‘ ca
St{f"
cos
2 sin 3t
The trajectories are spirals because
spirals
tend
away
from
the
origin
eigenvalues are positive.
19.
LO};
_
sty
eal
le (oy eng d ee eee
the eigenvalues are
because
the
real
complex.
parts
of
The
the
Substitute R, = 5, Ro = 3, C, = 4, and Cog = 3 into the formula for A
given in Example 1, and use a matrix program to find the eigenvalues
and eigenvectors:
A=
General
The
ea
Lee
pi
ran |;
solution:
condition
program,
x(t)
=
=
-.5:
1
VALS
EI
A2
=
=2. 5:
Bilalanue
2
ay
2
reltba =
ee
ot
x(0)
=
H
implies
5/2
and
cog =
-1/72,
evelt
ie
5} 1 apeoe
PA Ors
c, =
v(t)
v2(t)
Ai
that
so
,
Von
=
BY
Zo
i
By a
matrix
that
igs
Zee
Helpful Hint:
(for 21 and 22)
Find the general real solution before you
use the initial condition to find the constants c, and cpg.
Otherwise,
your
Cc, and cz will probably be complex,
and you will
have to do unnecessary
complex arithmetic to write the solution using only real scalars.
Answer to Checkpoint:
One eigenvalue
the origin is a saddle point.
If you
is positive and one is
consider the difference
=
the
Ax,
(with
the
same
matrix
A),
then
origin
both eigenvalues are less than 1 in absolute value.
a test that covers both Sections 5.6 and 5.7.
is
an
negative,
so
equation x,41
attractor,
Be careful
because
if you
have
5.8
¢
Iterative
Estimates
for
Eigenvalues
5-29
0.8 ITERATIVE ESTIMATES FOR EIGENVALUES
The algorithms
in this
decomposition described
eigenvalue
estimation
STUDY
NOTES
at
ee
section illustrate another use
in Section 5.6 (on page 337).
were
mentioned
in Section
5.2
of the
Other
(on
page
eigenvector
methods for
311).
Throughout the section, we suppose that the initial vector x9 can
ten aS Xo = Civ; + +++ + CaVp, where vi,...,Vpn are eigenvectors of
+
QO.
(In
composition
practice,
you
will
is used
only
to explain
not
know
C1Vi1,-.-,CnVn-
why
the
power
This
method
be writA and c,
eigenvector
de-
works.)
The Power Method:
Assume the eigenvalue A, for vi is a strictly dominant eigenvalue (so that |A,;| > |A;| for j = 2,...,n).
Then, for large k,
the line through Axo and O nearly coincides with the line through v, and
0.
The vector Axo itself may never approach a multiple of v, (see Exercise 21), but if each Axo is scaled so its largest entry is 1, then the
scaled
vectors
approach
an
eigenvector
(a multiple
of
v,)
as
k > o.
The Inverse Power Method:
You must start with an initial
estimate
a
for a particular eigenvalue,
say Az, and a must be closer to A» than to any
other eigenvalue
of A.
In, this, case,.«1/lAe - «),is ~a_strickly.dominant
eigenvalue of the matrix B = (A - aI).
The inverse power method avoids
computing B.
Instead of multiplying x, by B to get x4, (suitably scaled),
you
solve
the
equation
(A -
ol)y,
=
x, for
y, and
then
scale
y,
to
produce
Xk+1-
SOLUTIONS
1.
The
TO EXERCISES
vectors
i
in
the
i
1
dl: ae
sequence
1
Bel
1
EL
1
|
approach an eigenvector
vj.
Of pthesewvectonrs,.thelastsone
exe,
is
probably the best estimate of v,;.
To compute an estimate of A,, multiply one of the vectors by A and examine its entries.
Again,
the best
information
probably
comes
from
Ax,
=
bigete ,
Whose
entries
are
he
Fromeatheytinst..entry,
in xX.
approximately A, times the entries
estimate of A, is 4.9978.
The computed value of Ax, can be used as an estimate of the direcis not necessarily a better
but this vector
tion of the eigenspace,
For that, you should use xs, the
estimate for an eigenvector than x,.
the distance from Ax, to
With the given data,
scaled version of Ax,g.
the
eigenspace
eigenspace
is
about
is less than
7.0
x
10
1.4 x 10.
,
but
the
distance
from
xg
to
the
5-30
CHAPTER
5
°
Eigenvalues
and
Eigenvectors
questions
requiring
Study Tip: Exercises 1-6 make good exam
understanding of the power method without
7.
because they test your
extensive calculation.
produced
were
19
The data in the table below and the tables in Exercise
by MATLAB,
which carried more decimal places than shown
here.
Vi hesaiahea
11.5
12470
12.78
12.959
12.946
12.9927
12.9948
12.9990
12.9987
12.78
12.959
12.9948
12.9390
The exact eigenvalues are 13 and -2.
The subspaces
are
lines
whose
slopes
alternate
above
and below
eigenspace (the eigenspace is the line xp = x;).
determined
the
slope
by A‘x
of the
13.
If the eigenvalues close to 4 and -4 have different
absolute values,
then one of these eigenvalues is a strictly dominant eigenvalue,
so the
power method will work.
But the power method depends on powers of the
quotients A2/A; and A3/A, going to zero.
If |Ae/A;| is close to 1, its
powers will go to zero slowly.
15.
Suppose
Ax
=
(A - aIl)x =
invertible
aim:
This
19.
a.
and
with
A -
(APH oR)
last
sponding
Ax,
(A - a)x.
#
0.
is not
0;
(Al + 28)x
endl
equation
a
x
For
If w is
shows
to the eigenvalue
that
not
any
an
a,
Ax -
eigenvalue
alx
of
=
A,
(A -
then
a)x,
A -
and
al
is
hence
ic -ew)aix teeth e er ale
x
is
(A - a).
an eigenvector
of
(A -
aI)~*
corre-
The data in the table on the next page show that ug = 30.2887 = U7
to four decimal places.
Actually,
to six places,
the largest eigenvalue is 30.288685, with eigenvector (.957629, .688937, 1, .943782).
SUSI
Os
Iterative
. 961
7691
1
. 942
Estimates
. 9581
. 6893
1
. 9436
30.2892
The
inverse
for
Eigenvalues
5-31
957637
. 688942
1
. 943778
. 957630
. 688938
1
. 943781
29.0054
20. 8671
30. 2887
28.5859
29.0053
20.8670
30. 2887
28. 5859
30. 2887
30. 2887
power method (with a = 0) produces vy, =
“O10 18012
which seems
to be accurate
to
py = .010141,
at least
four
Actually,
V2 is accurate to six places,
v3 is accurate
to
eight places , and vg is accurate to ten places.
The convergence is
so rapid because the next-to-smallest eigenvalue is near .85, which
is much farther away from O than .0101501.
The vector xq gives an
estimate
for the eigenvector
that
is accurate
to seven
places
in
each entry.
-. 6098
1
-. 2439
. 1463
-. 60401
1
=) 25105
. 14890
=.603973
1
-.251134
. 148953
-. 6039723
1
mag Aah efor
. 1489534
=59; 56
98.61
-24.76
14.68
=59. 5041
98.5211
-24. 7420
14.6750
-59. 5044
98.52.1057.
-24. 7423
14.6751
-59. 50438
98.52170
-24. 74226
14.67515
.010141
0101501
.010150049
.0101500484
5-32
CHAPTER
MATLAB
5
Power
~«
Eigenvalues
Method
and
and
Inverse
Eigenvectors
Power
in your data.
digits
decimal
a strictly dominant eigenvalue,
to display 15
long
format
Use
algorithms below assume that A has
the
initial
initial
is x,
with
called
x0,
vector
vector
is
Method
rename
it
(If your
1 (in magnitude).
entry
largest
x =
by entering
x0
.)
is perof commands
an
(in many cases)
When the following sequence
The Power Method
of x approach
the values
and over,
over
formed
eigenvector for a strictly dominant eigenvalue:
y = A*x
[t
x =
The
and
hy
r] = max(abs(y));
y/y(r)
mu
=
y(r)
mu = estimate for eigenvalue
Estimate for the eigenvector
(2)
(3)
In (2), t is the absolute value of the largest entry in y and r is the
index of that entry.
As these commands are repeated,
the numbers that
appear in y(r) are the ,, that approach the dominant eigenvalue.
Recall that
After entering
MATLAB commands can be recalled by the
(1)-(3), your keystrokes can be
“Tt T<Enter>
|)
t<Enters
"©
<Enter>
could
enclose
and so on.
Alternatively,
(see page 5-25).
you
The Inverse Power
value in the variable
Store the initial
enter the command
n
is
of
Method
a, and
the
number
columns
y =
C\x
[t
r] = max(abs(y));
of
A.
=
As these
commands
produce
(3)
are
in
a
the
Solves
the
equation
=
repeated
sequences
and
loop
commands
(A
-
al)y
=
x
(1)
(2)
estimated
for
eigenvalue
the
(using 7T? <Enter>
{vy}
(7).
estimate of the eigenC = A - a*eye(n), where
enter
Estimate
the
(1)-(3)
key
nu = a + 1/y(r)
y/y(r)
and
lines
Then
nu
x
up-arrow
{x,}
eigenvector
each
described
time),
in the
(3)
lines
(2)
text.
Displaying
Data
If your computer screen displays
only 24 or 25
lines, vectors in the sequence {x,} tend to scroll off the screen soon
after you compute them.
To see more vectors at once,
and to compare
their entries more easily, you can display them as row vectors.
Change
(1)
and
to
y = A®x; y’
(power method) or
y = C\x; y’
(inverse power),
for both methods, change (3) to
x = y/y(r); x’
For even more data on your screen, use the command
format compact ,
which removes extra lines between data displays.
The simple command
format
returns everything to normal.
Chapter
5
«¢
Supplementary
Exercises
5-33
CHAPTER 5 SUPPLEMENTARY EXERCISES
Justifications
Other
ie
a.
for
the
justifications
True. _| If
A
are
is
as
340 (1-x),
for
A.
False.
Take
to
the
True/False
and
if
questions
are
given
below.
possible.
invertible
and
so
A =
[
553.
=ection:
invertible and
True.
tible,
answers
A
x =
1-x,
: ,
or
Ax
=
1°x
which
let
for
shows
A be
the
In both cases
A is not
hence is row equivalent to
some
that
nonzero
1 is
matrix
an
in
x,
Example
4
of
but
diagonalizable,
an identity matrix.
A
is
If A contains a row or column of zeros,
then A is not
and hence O is an eigenvalue of A, by the Invertible
Theorem
(stated
in Section
True.
The rotation
eigenvectors inR.
is an
from
eigenvalue
of
Zero
True.
By definition,
an
Example
every
eigenvector
1
in
singular
must
False.
Let v be an eigenvector for A.
eigenvectors,
but v and 2v are linearly
be
Section
square
This
False.
Let
follows
A be
from
Then v and
dependent.
the
Theorem
4
in Section
matrix
in
Example
3x3
5.5
has
no
matrix.
nonzero.
False.
Let A be a matrix with a two-dimensional
there exist two linearly independent eigenvectors
True.
inverMatrix
5.1).
matrix
False.
then
eigenvalue
2v
are
distinct
eigenspace.
of A.
Then
Section
Then
5.2.
3 of
5.3.
A is similar to a diagonal matrix D.
The eigenvectors of D are
columns’ of I3;, "but the eigenvectors of A are entirely different:
False.
tors
of
Let
A=
A,
but
two
eigenvectors
sum
cannot
False.
AllI
E
e,
+
of
ak
Then
eg
not.
is
e,
A correspond
=
is and
(Actually,
to
e€@2 =
it
distinct
can
*
are
be
shown
the
eigenvec-
that
if
eigenvalues,
then
their
triangular
matrix
are
be an eigenvector. )
the
the eigenvalues
diagonal
entries
of the matrix.
of
an
(Theorem
upper
1 in Section
5.1.)
Chapter
5-34
m.
5
°¢
Eigenvalues
and
A’
have
and
A
because
True,
By the determinant
transpose
Eigenvectors
Wedd beste 1
characteristic
same
the
det(A’
property,
-
AI) = det(A
-
AI)
det (A - AI).
n.
False.
Counterexample:
o.
False.
columns
If A is a diagonal matrix with O on
of A are not linearly independent.
p.
True.
If Au = A,u and
If u # 0,
q.
then
A, must
Let
A be
the
Au = Agu,
equal
5x5
then
identity
Au
the
matrix.
diagonal,
= Agu and
(A; -
then
the
Ag)u = O.
Ag.
False.
Let A be a singular matrix that
is diagonalizable.
(For
instance,
let A be a diagonal matrix with O on the diagonal.)
Then,
by Theorem 8 in Section 5.4, the transformation xr Ax is represented by a diagonal matrix relative to a coordinate system determined
by eigenvectors of A.
To show that, A" tends to the zero matrix,
it suffices to show that each
column of A
can be made as close to the zero vector
as desired
by
taking k sufficiently large.
The jth column of A is Ae;, where e; is
the jth column of the identity matrix.
Since
A is diagonalizable,
there
is
a
basis
for
R™
consisting
of
eigenvectors
vj,...,Vn,
corresponding
to
eigenvalues
aA1,...,A,.
So
there
exist
scalars
Cprormtlegr*Ssuchsthat
@y = C1Vq
Then,,
for
+
*°°
+ CyVn
(an eigenvector
decomposition
of e,;)
K =ale2ssoa;
k
Ae;
=
c1(A1) “vy pL
ISRe*
Cn(An) “Vp
Cf)
If the eigenvalues are all less than 1 in absolute value,
then their
kth powers all tend to zero.
So (*) shows that A* e; tends to the zero
vector, as desired.
Chapter
5
+
Glossary
to write
definitions
Checklist
Sé35
CHAPTER 5 GLOSSARY CHECKLIST
Check
your
Then
compare
Ask
your
knowledge
your
by attempting
work
instructor
with
which
the
definitions,
algebraic
multiplicity:
attractor
(of a dynamical system
tories tend ....
B-matrix
(for
T):
tive
The
a basis
characteristic
equation
characteristic
polynomial
companion
matrix:
in
difference
discrete
of
the
property
of
matrix
whose
that
below.
Glossary.
a test.
....
R? when
with
all
T:V
trajec-
> V rela-
....
(of A):
form
A nonzero
of
the
x
in
vector
:
©
difference
equation
in which Ais
....
(of a square
COVERT
(matrix):
matrix
A):
A matrix
that
equation
(or
form
whose
linear
characteristic
is
characteristic
equation
such
n
yxy,
A number
that
...,
where
Ay,,
or
a
=
det A computed
may be written
recurrence
...
an
differential
A;
equal
formas
....
equation
of
linear dynamical
system (or briefly,
a dynamical
ference equation of the form ... that describes
system):
..
A dif-
solution
x’(t)
eigenspace
(of
A corresponding
to
eigenvalue
(of A):
eigenvector
(of
eigenvector
basis:
eigenvector
decomposition
fundamental
set
of solutions
The
vector
A):
A scalar
is
set
....
x such
that
vector
A basis
consisting
in
The
that
A...
(of x):
A function
of
...
of
An equation
x=.
from
of the
solutions
...
entirely
(for*x-=FAx)>
R” formed
the
....
= Ax(t)):
A):
A such
relation):
of
....
from
in...
....
An
...
(of the equation
x:
origin
V,
eigenfunction
Im
The
as
B for
eigenvector:
diagonalizable
on
transformation
complex
determinant
appear
of an eigenvalue
R®):
terms
text’s
a linear
eigenvalue:
A nonreal
root
nxn matrix A, when ....
system:
A
equation,
might
the
the
[Tle for
complex
decoupled
of
in
(of A):
A special
°
given
if any,
multiplicity
A matrix
to
definitions
A basis
....
....
for
2...
form
of
....
....
5-36
CHAPTER
invariant
5
subspace
inverse
power
matrix
for
(of
the
Re
saddle
The
point
similar
spiral
stage-matrix
strictly
trace
for
M is called
H such
that
..
estimating
...
and
for
estimating
...
system
in R?)
THe
sa
lend...
R(x)
=
....
Matrices
(of a dynamical
model:
(of a square
The
graph
©
An estimate
in R" formed
(of a dynamical
dominant
perty
trajectory:
matrix
vector
(matrices):
point
Eigenvectors
A subspace
An algorithm
a dynamical
quotient:
x:
A):
An algorithm
tonies
Rayleigh
(for
and
.
T relative to bases B and ©: A matrix
mation T:V > W with the property that
method:
repellor
Eigenvalues
method:
B,
power
¢
from
system
A difference
in R*):
equation
The
of
...,
a
solution
for
t = 0,
The
x,4;
matrix
of x(t)
of
that
An eigenvalue
graph
OL lei.
in R*): The
eigenvalue:
that»; :.
A):
a linear
When W =
is denoted by ....
all
trajec—
....
origin
origin
....
in R°® when
....
= Ax, where
a matrix
x,
lists
..
A
with
the
pro-
Also,
the
by tr A.
{xo,x,,xXo,...}
...
in R® when
....
A; of
denoted
when
ii) R® when
transforV and 6 =
....
A and B such
system
M for
....
of
a
....
ORTHOGONALITY
AND
LEAST-SQUARES
6.1 INNER PRODUCT,
LENGTH, AND ORTHOGONALITY
The concepts of length,
distance,
and
tion are essential for many geometric
orthogonality introduced
descriptions in the rest
in this secof the text.
STUDY NOTES
OF isrvacal
The first half of the section is computational and easily learned.
The second half,
however,
requires more attention.
Read it carefully.
The concepts of orthogonality and orthogonal
complements are the foundation for
the rest of the chapter.
In fact, Theorem 3 is sometimes called the Fundamental Theorem of Linear Algebra.
In Figure 8, when a subspace such as ColA is referred to as a fundamental subspace of A, the phrase "of A" points to the important connection
between Col A and the matrix;
the phrase does not imply that Col A in some
sense lies inside the matrix.
SOLUTIONS
TO EXERCISES
eS
1.
u=
Tak v=
Ak
ueu
=
(aay e + oe = 5,
veu = 4(-1) + 6(2) = 8,
7. w=
(3,-1,-5),
13.
(10,-3),
x*=,
. 8
VWU=
u‘u
5
llwl? = wew = 3% + (-1)? + (-5)? = 35.
yi=
(=1;=5)
ix - yl = 110 ~ (-1)1 * [-2 — (c5)) = 121+ 4 S725
dist(x,y)
= |x - yll = V125,
or 5v5
So |lwil = V35.
6-2
19.
CHAPTER
a.
6
«
Orthogonality
and
the
definition
of
See
the
discussion
of Fig.
21.
See
the
box
following
Example
and
from
Theorem 1(c):
3(c) and 2(d)
Section
and
Theorem
see
2.1.
(cu)*v = (cu) 'v = (cu")v = c(u'v
= c(urv),
in Section 2.1.
Also, us(cv) = u (cv) = cuv
If y is orthogonal to u and v,
property of the inner product,
is orthogonal to u + v.
then
ys(u
29.
Take
+
typical
vector
3 for
6.
27.
a
ones,
T
T
(u + v) w= (u" +v)w=uw
+ vw = uw +
equalities used Theorems 3(b) and 2(c), re-
Theorem 1(b):
(u + v)ew =
vew.
The second and third
spectively,
1(c).
5.
Think about a 2x2 matrix of zeros
statements that are always true.
e.
Theorem
See
b.
|lv|l.
See
Least-Squares
w =
civ,
by Theorems
= c(lurv).
yeu = O and yev = O, and hence
+ v) = yeu+ yev =O+0=0.
°°*
+
Cpvp
in
W.
If
x
is
by a
Soy
orthogonal
to each vj, then using the linearity of the inner product (Theorem 1(b)
and 1(c)), wex = (cyvy + *** + CpVvp)*x = cyviex + *** + CpVpex = O..So
x
31.
is orthogonal
to
each
w in W.
Suppose x is in W and Ww.
Then, since x is in Ww’, x is orthogonal to
every vector in W, including x itself.
So xx = 0.
This is true only
if x = 0.
This problem shows that WnwW
is the zero subspace.
(See
Exercise
MATLAB
v’¥u
The
); the
32
in Section
4.1.)
inner
product
length
of v is
6.2 ORTHOGONAL SETS
of
real
norm(v)
column
. See
vectors
u and
the MATLAB
v
note
is
u’*v
on page
(and
2-4.
SS
Orthogonal sets and orthogonal bases are used throughout the chapter.
The
“orthogonal projection"
discussed in this section
is an important special
case of the orthogonal projections studied in Section 6.3.
STUDY NOTES
prt a ee
The proofs of Theorems 4 and 5 are worth studying
calculation you will see and use several times.
because
they
involve
a
6.2
¢
Orthogonal
Sets
6-3
The subsection entitled An Orthogonal Projection is simple but extremely important.
Also, the geometric interpretation of Theorem 5 on page 382
will be helpful when you study Theorem 8 in the next section.
The attention paid to Theorems 6 and 7 will depend on what your instructor plans to do later in the course.
In some cases, an instructor may
discuss
Theorems
6 and 7 only for square
matrices.
The mxn
case
is
needed
matrix
matrix
later,
for Theorems
10,
applies only to a square
must be orthonormal,
not
12 and 15.
Remember:
the term
matrix.
Also, the columns of an
simply orthogonal.
orthogonal
orthogonal
SOLUTIONS
TO EXERCISES
a
1.
~1
4|,
=3
u=
uew =
is no
oo pe
5
2i,
1
v=
-3 need
16 + 21 = 2 #0.
to check vew.
Al
U2 =
a
Theorem
basis
y=
for
u =
-7]
So
The
set
+8
-3=0,
{u,v,w}
is
not
orthogonal.
There
is an
{u,,uUg}
12 = 0, so
uj"ug = 12 -
{uj,us2}
is
SiUls
wenlSahely
The
orthogonal
uUsets =
you... 1.8f 121 5
The
component
TOS
Y Svar
u =
R’.
ine)
Yi4tueu
19.
-5
Since the vectors are nonzero,
u, and Us are linearly
Theorem
4. But two such vectors
in Rautomatically
KU
A
a
=
an
orthogonal
basis
for
R.
By
5,
dj
13.
uev
[cep
Pal x =
orthogonal set.
independent,
by
form
6
|-41,
=7
w=
pat
3
set.
Also,
hake = v-v =
des490. 5
of
e=1375
ope
y orthogonal
Cy ey
v=
499
to
'
yo smnee
36446°-
projection
_ 3-7 =
—5/47
a
uis
y -
-3[ 2] +3
of
2 hehe
y onto
u
4]
bar
is
ea
7/5
A
y=
4 -
-4/5
es =
eek
ae eae % Boalt
BAL uev
= -.48
+
.48
=
0,
so
{u,v}
|u| Surdeer(-.6) 111.8), =e.G6 +.
Thus {u,v} is an orthonormal set.
1.
64
is
an
orthogonal
=
1.
Similarly,
23.
25.
«+
6
CHAPTER
6-4
and
Orthogonality
Least-Squares
instance.
a.
See
Example3, for
c.
See
the
paragraph
after
Example
5.
d.
See
the
paragraph
before
Example
7.
(Ux)*(Uy)
=
b.
See
Theorem
5.
e.
See
Example
4.
T
U Unaey-i-)s
(because
= x 'U'Uy = x y = xy
(Ux) " (Uy)
(a).
part
If y = x, Theorem 7(b) says that ||Ux||~ = ||xl|°, which implies
Part (c) of Theorem 7 follows immediately from part (b).
Study Tip:
If your
instructor
emphasizes
orthogonal
matrices,
work Exercises
27—29.
(They make
good test
questions.)
In each
case,
mention
explicitly how you use the fact that the matrices are square.
Don’t read
the solutions below until you have first written down your own solution.
27.
If U has orthonormal columns,
square,
then the equation UU
Invertible Matrix Theorem.
29.
Since U and V are orthogonal,
each is invertible.
By Theorem 6 in
Section 2.2,
UV is invertible
and (UV)
= VLU
= Vel
srs. (UY) oa (UbY
Theorem 3 in Section 2.1).
Thus UV is an orthogonal matrix.
Mastering
To the
Linear
review
Algebra
sheet(s)
then u'U = I, by Theorem 6.
If U
= I implies that U is invertible,
Concepts:
you
have
Orthogonal
on
"basis",
nal basis and an orthonormal
basis
special properties they posses.
>» basic
for
definitions
(counting
Orthogonality
the
keystrokes)
In
Basis
add
the
concepts
of an
You
to
subspace.
Pages
> equivalent descriptions
>» geometric interpretation
>» special cases
> examples and counterexamples
> algorithms and computations
> connections with other concepts
MATLAB
a
379
and
need
Exercises
1—9
to
MATLAB
and
test
17—22,
a
orthogoknow
what
383
Theorems 4 and 6
Figs. 1 and 6
Matrix with orthonormal
Examples 2 and 5
Example 2
Orthogonal matrix
in
is also
by the
set
columns
the
such
fastest
as
way
{uj,Us,u3}
for orthogonality is to use a matrix
U = [ul
u2
u3]
whose columns
are the vectors from the set. See the proof of Theorem 6.
For column vectors y and u, the orthogonal projection of y onto u is
(y’ *u) /(u’ *u) *u
The parentheses
scalar quotient
(and the final
(y u)/(uu) and
*) are essential.
MATLAB computes
then multiplies u by this scalar.
the
6.3.
¢
Orthogonal
Projections
6-5
6.3 ORTHOGONAL PROJECTIONS
A familiar idea in Euclidean geometry is to construct a line segment perpendicular to a line or plane.
This section treats an analogous situation
in R, namely, the orthogonal projection of a vector (a point in R") onto a
subspace.
The case when the subspace
is a line through the origin was
already examined in Section 6.2.
KEY IDEAS
aes |
Lys
..1n R" and if W is a subspace
of y onto W, denoted by y or proj.,y:
(i)
yand
(ii)
y is the closest
Properties
(i)
of R”, then the orthogonal
projection
has two important properties:
y is orthogonal to W (so y is
a vector y - y in W), and
and
(ii)
are
point
the
sum
of
a vector
y in
W
in W to y.
described
in the Orthogonal
Decomposition
Theo-
rem and the Best Approximation Theorem.
You should learn the statements of
both theorems.
(By now you probably know that whenever a theorem has an
officiai name,
an instructor has an easy time asking test questions about
it.)
When you need one of these theorems in a discussion (homework or test
question),
you should mention the theorem by name.
If your class covers Theorem 10, then the paragraph following the theo-
rem will help you understand the difference between an orthogonal
matrix
(which must be square) and a rectangular matrix with orthonormal columns.
SOLUTIONS
2
TO EXERCISES
eee
0
1.
uw =
S|
a 7
: Ugh
Ug. =
=t
late
all
=
1
the
x°u
inner
products
ty, + U2x°u°UoSy
Uy °uy
1
5
5 ,» Ug =
pu Hi
-4
{
in the
once
you
Sine
3 -
Yourcould*caleu=
0
decomposition:
x°u
+ Sy,
+Ugx°u
Sy
U3 °U3
°Ug
in Span{uj,
uo, us}
However,
10
know
Span{u;,
uz, Ug} is determined
the
(*)
in Span{u,}
vector
completely
in
Span{uy},
by (*).
So all
the
you
vector
need
is
in
Least-Squares
and
Orthogonality
°«
6
CHAPTER
6-6
KitLgirinsere Sint 2A natal
eee
app aS eo Be
10
yaaebent
ed
2
10
; Span{u;,uU9,u3}
in
vector
The
-87
is x - 2u,g =
10
aed
6)
Study
verify
y
that
to
way
One
Tip:
-
check
whether
projyy
is
orthogonal
to
proj. y
is
each
-2te
oes
a
ee
correctly
computed
is
to
orthogonal
the
in
vector
A faster check that will
W.
that y - projy,y is orthogonal
basis {u,,...,up} for
not all) is to verify
6)
-6
os
catch most errors
to proj, y:-
(but
1
1
5
pI u; = EI U2 = Hr First,
make sure that {u,,us}
is an
S
fe
4
orthogonal
basis for Span{uj, ud}.
This is easy,
since u, and ug are
nonzero and u,*Us = 0.
Next, by the Orthogonal Decomposition Theorem,
y is the sum of proj,,y and y - projy y, where W = Span{uj, uo}.
Lanluns
—
yeu
y°uo
Prod Y = goon,ot
= Ou,
Geta
3
10/3
+ RU2 =
Z/s
“CH
79
Peg
7.10
Oe
tT 2
4 OD Teas Sarees
Ue
8/3
and
Ys
As
a
check,
vector
as
Warning:
1
ot
S
projy,y =
scale
is obviously
it should
The
y
=
-
10/3
2/3|
8/3
projy,y
orthogonal
73
ize
(f45}
=
el
1},
1
to
to
u,
and
and
us.
observe
that
Thus
projiy
y -
the
scaled
is:
in te
be.
formula
for
proj,,y applies
only
if
{u,,...,u,}
is
an
orthogo-
nal basis for W.
That’s why you should check orthogonality,
as in Exercise
7, if you are not sure
that the basis
is orthogonal.
If an orthogonal
basis is not available,
then other methods can be used to compute y.
(See
Exercise 23 in Section 6.5, for example.)
6.3
13.
3
al
Bie
z=
2
Bar
3]
med
Vic =
3
1
1
ol:
Z
V2
1
Note
¢
Orthogonal
that
v, and
Z°vy
Z°Vo
+
viv
1
ae
Va°V>
=
poeta
15 “2 © aa
4
Check:
z-Z=
a .
By the Orthogonal
= Span{u,,u2} and
proj, Us =
orthogonal.
By
point
in
Span{vj,vo}
2
7H
Neo
|
==
Need
anes
=!
=
1
=f
3
2s"=3)e0F
1
isle
The
vector
z
is orthogonal
to
z
49
both
v, and
vo.
0
Decomposition Theorem,
us is the sum
a vector v orthogonal to W.
First,
-2
su
to
1
6)
19.
vo are
6-7
=1
the Best Approximation Theorem,
the closest
is the orthogonal projection z, where
a
Projections
2
+ 39 U2 =
-2/6
|-2/6]
4/6
an
Rovere
|-2/30]
4/30
+
=
of
a vector
in W
0
|-2/5
4/5
Then
Vv =
Not
U3 -
only
Study Tip:
sult with
is v orthogonal
It would
Exercise
that
{u,,u2}
an orthogonal
basis of
with your instructor.
a.
92o
=
6)
1-275
4/5
=
0
1275
I/5
to
W,
but
also
any
is an
be a good
19.
Then
find
this
21.
0
Of
a
proJj,, Us =
orthogonal
See the calculations
in Section 6.1.
idea to try
think about
set
for
Za
of
v is
in wo
Exercise 20 and compare
the following problem:
of nonzero
R? that
multiple
vectors
contains
u,; and
in Example
1 or
in R°.
ue?
How
You
box
after
statement
of
Theorem
the
Orthogonal
Decomposition
See
the
the
second
paragraph
after
See
the
box
before
The
Best
Approximation
See
the
paragraph
after
Theorem
would
might
the
See
the reSuppose
Example
Theorem.
10.
the
Theorem.
you
discuss
Q.
6
CHAPTER
6-8
23.
6
Orthogonality
«
and
Least-Squares
each x in R" can be written
Theorem,
Decomposition
By the Orthogonal
By Theorem 3
u in (Row A)”.
and
RowA
in
p
with
u,
uniquely as x = p +
in Section 6.1, u is in Nul A.
suppose
Next,
x = p + u,
write
b is
=
that
Ax
as
above.
Then
Let
consistent.
x
be
Ap = A(x - u) = Ax -
a
and
solution,
Au = b-
the equation Ax = b has at least one solution p in RowA.
suppose that p and p,; are both in RowA and satisfy
Finally,
Then p - p; is in Nul A because
O =
b.
=
b.
So
A(p - pi) = Ap - Ap, =
Ax
b- b=0
The equations p = p, + (p - p,) and p = p + O both decompose p as the
sum of a vector in RowA and a vector
in (RowA)’.
By the uniqueness
of the orthogonal decomposition (Theorem 8), p; = p, so p is unique.
Warning:
I had to work hard to make the arithmetic simple in the exercises
for this section,
to avoid distractions for you and to save you time.
You
might not be so lucky on an exam.
Even if a problem
is designed to be
numerically simple,
there is always a chance that a minor error will make
the calculations
messy.
In such a case,
don’t
despair.
Carry out
the
arithmetic
as best you can,
showing the details
of your work
(patterned
after the solutions
in this Study Guide).
Chances are that you will get
substantial credit for showing that you understand the concepts.
MATLAB
Orthogonal
Projections
The orthogonal
projection of y onto a single vector was described
in
the MATLAB note for Section 6.2.
The orthogonal
projection onto the
set spanned by an orthogonal
set of vectors
is the sum of the onedimensional
projections.
Another way to construct
this projection
is
to normalize
the orthogonal
vectors,
place them in the columns
of a
matrix
onal
U,
set
=
has
and
of
use
nonzero
Theorem
[y1/norm(y1)
orthonormal
10.
vectors,
columns,
For
then
instance,
the
y2/norm(y2)
and
U*(U’*y)
tion of y onto the subspace spanned
around U’*y speeds up the computation
if {y1, ye,y3}
is an
orthog-
orthogonal
pro jec-
matrix
y3/norm(y3) ]
produces
the
by {yj},
yo, ys}.
of UU’ xy. )
(The
parentheses
6.4
6.4 THE GRAM-SCHMIDT
¢
The
Gram-Schmidt
Process
6-9
PROCESS
This section has a nice geometric
appeal.
well-liked by students and faculty because it
the process is seldom used in practical work,
tions to spaces other than R” (to be discussed
The Gram-Schmidt
process
is
is easily learned.
Although
it has important generalizabriefly in Section 6.7).
KEY IDEAS
| FS=,
|
When
the
Gram-Schmidt
process
is applied
to
{x,,...,x,p},
the
first
step
is
to set v; = x,.
For k = 2,...,n,
the kth step consists of subtracting from
x, its projection onto the subspace spanned by the previous x’s.
At each
step the projection is easy to compute because an orthogonal basis for the
appropriate subspace has already been constructed.
The QR factorization of a matrix A encapsulates the result of applying
the Gram-Schmidt process to the columns of A, just as the LU factorization
of a matrix encodes
the row operations
that reduce
a matrix
to echelon
form.
Also,
just as the LU factorization can be implemented via multiplication by elementary matrices,
so can the QR factorization be constructed
via multiplication by orthogonal matrices.
SOLUTIONS
we
1.
TO EXERCISES
3
O|,
ae
x, =
8
5
=6
Xo =
Set
v, =
x; and
8
Vo
= Xo
Check:
vo*°v,
wei vy =
va°vt
=
-3
+ 0+
2
Ci
Xpe=
3
5
==
-6
Ui
O|
=
an
orthogonal
-1
3 = 0.
So
Foom
Exercise.
3)
-3
basis
4
aXou=.
| Dn
3,.
use
svi_.=
| =5))
23
1
¥30|
1
2/V30
=
|-5/V30
1/V¥30
|],
Span{x,,x2}.
Scale
1 2
2/V6
up
= —| 1]
= | 1/6
1
1/V6
v6}
vg
to
=I
5
SS)
3
and
.Voe=
th
basis for W =
then obtain
2
= ——|-5|
=
Ol,
=i
is
2
l . e,
io
1
as an orthogonal
normalizing,
and
u;
compute
1372
3/2
(2,1,1)
before
6
CHAPTER
6-10
Least-Squares
and
Orthogonality
«¢
If you need to normalize a vector by hand, first consider scalStudy Tip:
if possible.
ing the entries in the vector to make them small integers,
13.
5;
9
Dn
aE
APM
TERS ei |
PT
-1/6
5/6
1/6
3/6
1ORG 1S
Ae
ELSG
5/6.
tag
a
=
ee
2 COA
As
5/6
7ore
a check,
oats
5
9g
7
|91
oes
:
aes
4.176]
«3/6;
A/G ee de
41/6
5/6
compute
rag |
5/6
© LL I4e
-1/6
576)
1/6
3/6
QR =
-3/6
We
1/6
\[>2
pesto
;
eee
6
9)
7276]
36/6
[86/68
)
5
54/6
en |e eeee
| aha
|e
sea Loa oe A.
1
3076
Remark:
The reason the R in Exercise 13 works is that the columns of Q form
an orthonormal
basis
for _ColA
(since
they were
obtained
by the
GramSchmidt process).
Thus QQ y_= y for all y in Col A, by Theorem 10 in Sec-
tion
6.3.
In particular,
QQA=
nh
a.
See the remark after
there to Exercise 32.
b.
See
(1)
in the
See
the
solution
A.
So
if R is QA,
then
QR = Q(Q A) = A.
Example
5
in
6.2,
and
statement
of
of Theorem
Example
Section
the
reference
11.
4.
19.
The
21.
The solution in the text is complete, except for the details of extending an orthonormal
basis for Span{q;,...,qn,}
to an orthonormal
basis
for R
Here is one algorithm.
Let e,,...,e, be the standard basis
for R.
Let f£,; be the first vector in this basis that is not in W, =
full
solution
Span{q;,...,Gn},
is
and
in
the
let
text.
uw, = f, - proj,£1.
Then
{qi,-°.,@n,
W)} fs an
n
orthogonal
basis
for
Whi
=
Span{q;,...,qn,,u4}.
vector in {e1,...,em} that is not
than f; in the list e,,...,em.)
Let
fo
be
the
first
in Wyi:.
(Of course fz occurs later
Form up = fo - proj,
fo and Wrsro =
n+i
Span{qi,...,Qn,U,,Ua}.
have been added to the
This process will continue until m - n vectors
original n vectors.
Normalizing the new vectors
produces
basis
an
orthonormal
for
R".
6.5
¢
Least-Squares
MATLAB
The
Suppose
columns,
that A has linearly independent columns.
then the Gram-Schmidt process is
Gram-Schmidt
Process,
proj,
and
Problems
6-11
has
two
gs
If
A
only
v1 = A(:,1)
If
v2
= A(:,2)
A has
three
-
columns,
v3 = A(:,3)
You
should
cedure.
columns
add
the
command
(AC:,3)’*v1)/(
v1’ *v1)#v1
these
that,
projection
for
a while,
can
the
Toolbox
of
use
a vector
(or vector)
V.
= A(:,2)
-
proj(A(:,2),
v1)
v3 = A(:,3)
-
proj(A(:,3),[v1i
The
columns
of
V
in
work,
but
if
they
to
-
commands
you
of a matrix
v2
mand
the
-
use
After
computes
(A(:,2)’*v1)/(v1’
*v1) *v1
proj(x,V)
are,
the
x onto
For
(A(:,3)’*v2)/(
v2’ *v2)*v2
to
learn
command
the
the
general
proj(x,V)
subspace
pro-
, which
spanned
by
the
example,
Vv = vi
v2])
need
V =
not
entries
[vi
v2]
be
orthogonal
for
the
in
proj(x,V)
will
usually
com-
agree with those computed via Theorem 10 in Section 6.3, to twelve or
more decimal places.
Enter
help proj
to learn more about
proj.
To check your work or to save time, enter
Q = gs(A) , which uses
the Gram-Schmidt process to construct the columns of Q.
See
help gs.
Although you should construct the QR factorization of a matrix using
the approach in the text, you might like to see what MATLAB does.
The
command
[Q1
R1] = qr(A)
creates a modified QR factorization of an
mxn matrix A as described in Exercise 21 in the text.
If rankA = r,
then the first r rows of R, are nonzero and the first r columns of Q,
form an orthonormal basis for Col A.
6.5 LEAST-SQUARES
The
all
basic geometric
the applications
PROBLEMS
principles in this section
in Sections 6.6—6.8.
provide
the
foundation
for
KEY IDEAS
aes |
material up to and including Figure 2 needs to be read carefully sevetimes, so you understand what the term "least-squares solution" means.
Be careful to distinguish between x and 6.
The vector 6 is unique Since it
is the closest point in Col A to b. The least-squares solution x is unique
if and only if A has linearly independent columns (see Theorems 13 and 14).
The
ral
6-12
CHAPTER
6
°¢
Orthogonality
and
Least-Squares
x itself somehow has the leastThe vecLook at Fig. 2 again.
A common mistake is to think that
Warning:
squares norm or is the closest point to b.
tor closest to b is Ax, not x.
Theorem 13 provides a common way to find least-squares solutions.
One way to remember the normal equations
is to observe
that they are the
same as Ax = b, with A left-multiplied on each side of the equation.
It is
completely wrong,
however, to try to derive the normal equations from Ax =
b by multiplying by A.
If the equation Ax = b has no solution,
then the
equation itself is a false statement about every x.
Matrix algebra on such
a false statement is meaningless.
SOLUTIONS
a
1.-A
=
TO nana
EXERCISES
=
[ 2
a
2
=3);
3
4
1
2
bs
me
oe
“Binet
T, at s[ey
5 4
(ites bee -3
a.
The
b.
Since
|
;
:
equations:
normal
A’A is only
Ge
Be
2x2,
22
(ATA)
11
h[AOeret
ta”
_ [-4
pb a ee -3
_ ; [ al
MATT
hee |
os |e =
is easy
x yp ext
ea]ig 54
elie!)
4
ton
Beh -“11
zak
d&: ee
FA
to
compute,
and
f2 ec14) [esp inossfeay & fis
11}11
G|:Allie
=atl22 | Jay192
Warning:
It is important to distinguish between the normal equations A Ax =
Ab and the formula x = (AA)
Ab.
Both equations describe x (implicitly
or explicitly),
but the formula for x holds only when A has linearly independent columns.
Note that the expression
(A A)SaA* cannot be simplified
when A is not invertible.
1 -2
T
A=
cath|
2
Z
5
Ouges
3
:
jo) —
i
Ba
2
1 -2
.
T
ae
=
1
i=
0
be eBawitc
Ot
ak
2
2
3)
| O
aes
=
6
E
6
rt
The
normal
mal
6
[
equations :
6.5
¢
=
6
:I
6] |x 1)
1 |
The particular numbers
in AA suggest
be solved easily via row operations:
Thus
6
Gwi
She
GYRLGI6
4206-6
O'
x =
ak
The
1
a
6
736
Oo)
= 12
=2
maslat a3 2/64/73
tie O azt|
Study Tip:
thogonal to
equations
t
gard
1
byZe
ey
1
0
0
4/73
ie pin VAS.
||Ax -
bl],
so
error
is
3
2]
3
-a| ~ | 3
-a| ~
|-1
2
2
1
=
2
1 = 20, and
your
work
|/Ax - bil = V20 = 2v5
is
to
verify
3
4)rg
11
11
iN
13... Au)=-[-21 Ei = |-11], b-- Au-=-[-9]°--"]-11}
=
oo
4
a
5
11
3
Ales
%
11
7
Av=
|-2
1 ed ='|=12], b - Av = |=9) - |-12] =
Bint
wi
5
ri
Obviously,
Au
closer.
Hence
17.
is not the closest
point
of ColA
u is not the least-squares solution
a.
See
the beginning
of
b.
See
the
comments
about
c.
Read
the
definition
ad.
See
Theorem
19.
The
full
21.
a.
13.
solution
is
the
of
that
-
Ax -
b is
or-
fo)
|-2], ‘lb - “Aull = V40
4
/"3],-|b_- Aull = v29
=2
to
of
b,
Ax
because
= b.
Av
is
section.
equation
a
compute
il
(Owsi3 bees
A good way to check
each column of A.
might
normal
|e
Ax - bi? = 1+9+9+
6-13
the
1| d/A=2hiuahod |.ob
5
Problems
that
eet
0
least-squares
Least-Squares
(1).
least-squares
e.
See
Theorem
in
the
text.
solution.
14.
then the equation Ax = 0 has
If A has linearly independent columns,
only the trivial solution.
By_Exercise 17, A Ax = 0 also has only
the trivial solution.
Since AA is square,
it must be invertible,
by the Invertible Matrix Theorem.
CHAPTER
6-14
6
«+
Orthogonality
b.
Since the n linearly
not be less than n.
c.
The
the
MATLAB
independent
n linearly independent
rank of A is n.
The
Backslash
and
Least
Squares
columns
columns
of
of
A belong
A form
a
basis
to
R™,
for
m could
Col A,
so
Command
Note on page 410 in. the
15 and 16, see the Numerical
For Exercises
text. The MATLAB "backslash" command
R\(Q’*b)
will solve Rx = Qb by
row reduction.
When A is nonsquare with linearly independent columns,
this procedure of first forming Q and R and then solving Rx = Qb is
what MATLAB itself does (internally)
when the MATLAB command
A\b_
is
used to solve Ax = b. Compute
A\bfor the data in Exercises
15 and
16, and compare the results with your calculations involving Q and R.
For
Exercise
26,
the
command
A =
[A1;A2]
creates
a
matrix whose top block is Al and bottom block is A2. This
as long as Al and A2 have the same number of columns.
(partitioned)
command
works
6.6 APPLICATIONS TO LINEAR MODELS
This section of the text will be a valuable reference for any person who
works with data that requires statistical
analysis.
Many graduate fields
require such work, often in connection with doctoral
research.
Even most
undergraduates will take a course where least-squares lines are used.
KEY IDEA
ee areae
Linear algebra unifies
the study of many problems
in statistics
and data
analysis.
All the examples in this section,
from ordinary linear regression (using a least-squares line) to multiple regression,
concern just one
idea:
find a least-squares
solution of XB = y. Only the design matrix X
varies.
The exercises help you practice choosing X. The least-squares solution B always satisfies the normal equations xXXR =Xy
STUDY
NOTES
Sari
oe sil
Don’t confuse the least-squares line in Fig.1 with the
Section 6.5 onto which we projected various vectors b.
more than a special case of the curves in Figures 2—5.
lines and planes in
The line is nothing
In each case,
the
6.6
¢
Applications
to Linear
Models
6-15
"linearity" of the model lies not in the curve, but rather in the fact that
the unknown parameters
(or weights) Bo, 8,,...
occur linearly in the formula for the curve,
just as the variables x,, xX2,... occur in an ordinary
linear equation.
Any 4x4 submatrix of the design matrix in Example 3 is called a Vandermonde matrix.
Using Exercise 11 on page 178, one can show that if at
least
four of the values
x;,...,x,
are distinct,
then the
least-squares
solution B will be unique, by Theorem 14 in Section 6.5.
FURTHER
2a
READING
ee
An important
generalization of the discussion
analysis,
which involves several y vectors rather
case the basic equation
one dependent variable,
determined.
That
is,
is XB = Y, where
and each column
X[B:
see
the classic
text
Statistical
Analysis,
-::
Bp]
=
here
than
is to multivariate
just one.
In this
each column of Y is a data set for
of B is a set of parameters to be
[y1
°::
by T. W. Anderson,
John Wiley & Sons,
yp].
For
more
An Introduction
New York,
1984
information,
to Multivariate
(and 1958).
The
preface of the text says,
"A knowledge of matrix algebra is a prerequisite
[for understanding the text]."
Most modern multivariate
statistics
texts
rely heavily on matrix notation and matrix algebra.
SOLUTIONS TO EXERCISES
We
1.
Place
the
x-coordinates
y-coordinates
1
0
1
1
in the
x
The
matrix
4
6
data
y.
6)
1}
2
3
_
So
{4
6..
X =
6
142
The
the
second
if
O
;
5 and
1
3
1
6)
_
column
and
equation
y =
;.
1
2
1
8
heed
Sililsa
a
y
are:
[6&6
11
reuiGstA
11}
least-squares
line,
20|-6
y =
6) eit fiel: «plo
4)}{11
Bo + Bix,
is
4
8
20|
y=
X
and
Compute
2
1
1
solution
its
of
1
ye
Boleteil4eucd |. ceGli—nurd (d4encSh[
Byler
in
Xx
normal
6][6o]
14]1Bi
the
vector
1
Hila
=e fo Bil
1
1
2
of
.9
+
.4x
eee
6
11
the
6-16
7.
CHAPTER
6
«+
XB +
Squares
18
1
a
et f
2
4
(X’*X)\(X’*y)
X =
[3
9],
Cm ass
Sate
the
equation
B = X\y_
immediately
In any case,
computes
the
fe =
decimal
places)
B =
hes
Bo
desired
solves
B=
| € =
Bo
|&3
&4
Es
(to
=e eU
two
least-squares
equation
is
y =
and
kth
for k = 2 and 3, let
powers of the entries
Study Guide
section.)
Keeipyld
wat:
The
observation
a.
Numerical
Br)
The
b.
The
t*.k denote
in t.
(See
Then
1.76x
From
the design
matrix
y lists
the
measured
of
the
normal
equations
(-.8558,
4;7025)
5.5554.
=, 0274)
least-squares
polynomial
y = —.8558
+ 4.7025t
velocity
the
is
(position
+ 5.5554t"
derivative
seconds,
v(4.5)
equation
(1) on page
415,
x'x =
ae
axe
i
X4
Koy ee
eee
1
Xn
Page
Xp?
eee
the
of
20x".
let t = (0, ...,
12),
is
positions
solution
:
-
and
solution
aetie2uarts 33]
vector
t = 4.5
X y,
the vector whose entries are the
the MATLAB box at the end of this
of
of
-
the
AIM
= 53.0
lee E
.
ap
ES
ele
oxy
n
of
the
the
position
ft/sec.
|
plane.
at
time
yields
plane
.o274t°
v(t) = 4.7025 + 11.1108t - .og22t?
When
(X X)B =
least-squares
13. Let 1 be the vector in R** with 1 in each entry,
15.
Eo
In this problem,
x'Xx is invertible,
and you can use
your matrix
program to, solve the normal equations via row reduction or the inverse of X X.
MATLAB offers two additional possibilities:
the com-
The
|3.4],
mS yes
3.9
€4
B
b.
B =
y =
Least
y =
command
XB = y.
where
and
a.
mand
©,
Orthogonality
function:
t)
is
6.6
16.
normal
equation
Note:
The
X'XB =
formulas
¢
n=x
you
Some
statistics
-
should
(Xx)
texts
to Linear
derive
Appendix:
the
Pythagorean
The
Geometry
6-17
R
2
ae nzxy
pea
>
present
are
other
=
(=x) (Ly)
n=x’ -
(=x)?
equivalent
formulas
for
19. The equation to be proved is llyll* = ||xpll? + lly - x@ll?.
from
Models
X'y.
es (Ex*) (Zy) - (Ex) (Exy)
2
Applications
Theorem
(in Section
of a Linear
6.1)
and
the
Bo and
Ai.
This follows
figure
below.
Model
The column space of the design matrix X is sometimes
subspace.
If B is the least-squares solution of y = XB,
called the design
then the residual
vector € = y - XB is orthogonal to the design subspace,
and the equation y
= XB + €& is an orthogonal decomposition of the observed y into the sum of
the least-squares predicted y and the residual vector e.
y
&
The
MATLAB
Functions
If x is a vector
the same size as
x.
The
design
function
of
The
tion,
log(x)
,
Col
and k is a positive
x whose entries are
act
integer,
then
powers
kth
the
was
mentioned
function,
,
cos(x).*k
also
X
Vectors
exponential
4-12.
subspace,
on
each
exp(x)
entry
in the vector exp(-.02*x) are computed
to the corresponding entries in x.
in x.
natural
For
note
MATLAB
in the
and
x.*k
is a vector
of the entries in
instance,
by applying
the
on
logarithm
the
function
page
func-
entries
e--ae
6-18
CHAPTER
6
«+
Orthogonality
and
Least-Squares
6.7 INNER PRODUCT SPACES
1,
in Examples
Three examples of inner product spaces are described here,
Material
section.
next
the
in
appear
applications
Corresponding
7.
2, and
particularly
careers,
for many
be useful
will
6.7 and 6.8
in Sections
cover this
not
does
course
your
If
mathematics.
and
engineering,
science,
now, the text and Study Guide can help you learn it later on your own.
KEY
IDEAS
et
ae SY
The concepts of length and orthogonality
in R" have analogues
in a number
of other vector spaces.
The definition of an inner product
identifies the
basic properties needed for a theory that parallels the familiar theory for
R™.
Two
useful
facts,
the
Cauchy-Schwarz
inequality
and
the
triangle
inequality, were not developed earlier, but they are important for applications both in R" and in other inner product spaces.
Every mathematics major
will need to know these facts in other undergraduate courses.
The
inner product
in Example
1 is used
in Section
6.8
to describe
weighted least-squares problems.
The inner product
in Examples 2—6 provides
a more
sophisticated
approach
to the
least-squares
curve
fitting
discussed in Section 6.6.
See the "trend analysis" in Section 6.8.
Be sure to read the paragraph preceding Example 6.
The idea of
"best
approximation"
to a function is of fundamental
importance
in mathematics.
The most common applications of best approximation (such as Fourier series,
introduced in Section 6.8) involve the inner product in Example 7.
SOLUTIONS
TO EXERCISES
Sa
a ee
1. The
inner product
is (x,y) = 4xiyio+ ‘Sxeye.
“Let x = (1,1),
y = (5,-1).
a. |ixl° = 4-1-1 + 5-1-1 = 9, xl) = 3
lly? = 4-5-5 + 5(-1)(-1)
2
I(x, y) |
b. A vector
that
Thus
=
|4°-1-5
= 108,
+ Beit)
z = (z1,Z2)
|e =
|lyl| = vTO5
115|° =
is orthogonal
225
to y if and only if (z,y) = 0,
is,
4°24°9
+5 *2Zo°(-1)"
(2;,Z2)
is orthogonal
= 0, * 202,
to
= 525 = 0, eyona
y if and
only
if
Zo =
ee = eT
42,.
6.7
7.
Given
p(t)
q onto
the
Example
5 organizes
= 4 + t and
subspace
q(t)
= 5 - 4t®.
spanned
by p is
the
calculations
¢
Inner
Product
The
orthogonal
[(q,p)/(p,p)]p.
Spaces
6-19
projection
q of
The
notation
of
product
of the
two
nicely:
Polynomial:
Vector
of
values:
DU
OPW
The
inner
product
corresponding
(q, p) equals the
yectors
any) AR ke:
(pP,p)
= 3° + 4° + 5° =.50.
a
-*2e
q(t)
13.
=
Suppose A is
that (u, v) is
i”
(u, v)
FP
Or
Q
g5(4 +
t)
=
at
-1
€ value
at
0
€ value
at
i
(standard)
(q, P) = 301
Thus
:
=
56
inner
+ 4°65
.7 9°)
(u, v) = (Au)-(Av),
for u,v in R’.
each axiom in the definition on page
(Av)°(Au)
.
Property
(cu,v)
of dot
Matrix
Note
422:
product
multiplication
(Au)-(Aw)
+
= (uw) + (vw)
(Av)
(Aw)
Property
=
[A(cu)]+(Av)
=
[c(Au)]+(Av)
Matrix
=
c(Au) °(Av)
=
olin lary.
14
(v,u)[A(u + v)]*(Aw) = [Au + Av]*(Aw)
ii. (u + v,w)
iii.
= 26.
56 + set
invertible and
in R, and check
(Au) *(Av)
€ value
of
dot
product
multiplication
Property
of
dot
product
= c(u,
v)
iv.
(u,u)
=
(Au)+(Au)
=
||Au 7 = 0,
only if the vector Au is
because A is invertible.
0.
and
But
this
Au
=
quantity
O
if
and
is
zero
if
only
if
u
and
=
0,
Another
method for verifying the axioms
is to use properties
of the
transpose operation.
The calculations are similar.
However,
for (i),
you need to use the fact that the transpose of a scalar (which is
1x1_matrix)
is
the
scalar
itself:
(Av)"(Au)™? = (Av)"(Au) = (v,u).
17.
lu + vil"
(u,v)
=
(u,v)
=
(u + v,u + v) = (u,u + v) + (v,U + Vv)
Axiom 11
{u + v,u) + (u + v,v)
Axiomi
{u,u) + (v,u) + (u,v) + (v,v)
Axiom ii
(u,u) + 2(u,v) + (v,v)
Axiom i
[(Au)
(Av)]
Ta)
6-20
replace
Next,
lu - vil
2
and
Least-Squares
v by -v and use
the
fact
(u, u) +
2(u, -v) +
Gian
2(u,v)
Subtracting,
sion
Orthogonality
«
6
CHAPTER
|lu + vi? -
by 4 gives
the
25.
In the space C[-1,1] with the
and 1 are orthogonal,
because
is
in
the
integral
inner
product,
the
1
2
= 501°
L
=i
1 and
= A{u,v).
Divi-
text.
(t.1)=[ : t-1dt=5e7)Es
So
(-2(u,v))
15
identity.
The
solution
iii and Exercise
-
ju - v\|° = 2(u,v)
19.
full
Axiom
-v.
by
v above
Replacing
Cy. -v)
+ (-1) Xv, v)
desired
+ (-1)v.
u - v =u
that
polynomials
t
1
2
3-1)"
= 0
-1
t can
be
in
compute
proj, t”,
the
subspace
W spanned
by
an
orthogonal
orthogonal
1 and
basis
projection
for
of
Span{1,
the
t, ra
vector
Next,
t? onto
the
t.
1 2
1 _wI SER,
Ivits
terdt=-5t°|
= 3(1)° - 3(-1)9= ons he2]5
~1
“1
1
1
(1,1) = | 1-1at = ¢| =1-(-1)=2
-1
-1
2
a
Cian
yea
2
SSS)
There
is no
A polynomial
is
this
i
“1
need
2
1,4
= Gt")
t'-tdt
-1
to compute
orthogonal
polynomial
to
scaled
1
4
1
=0
= 5(1)" - Z{-1)"
Cit)
W is
by
3,
because
t? >
t* is orthogonal
proj, t° =yt?
namely,
Sto
2
T
<
=:
to
t.
Another
Thus,
the
Thus
choice
polyno-
miale1,
ee and 3t* - 1 form an orthogonal basis for Span{1, ty Ba).
Can you find the next Legendre polynomial,
a cubic polynomial
that
is orthogonal to each of the first three Legendre polynomials?
6.8
¢
Applications
of
Inner
Product
Spaces
' 6-21
6.8 APPLICATIONS OF INNER PRODUCT SPACES
Of the three applications in this section,
the discussion of Fourier series
is by far the most important.
Such series have great practical value, particularly in mathematics,
engineering and the physical sciences.
Calculations with Fourier series are simple because sine and cosine functions are
orthogonal.
This fact is often overlooked
not assume a linear algebra background.
in undergraduate
courses
that
do
KEY IDEAS
ee
are
The text gives the normal equations for the weighted least-squares solution
of Ax = y.
When applied to a least-squares
line problem,
the most common
situation,
the normal equations are usually written as
(WX) WXB = (WX)"y
where
W is
the
(diagonal)
weighting
matrix,
X is
the
design
matrix,
B is
the
least-squares parameter vector, and y is the observation vector.
Trend analysis is really a least-squares regression problem of the type
described in Section 6.6, with data points (x1,y1),...,(%n,
yn) fitted by a
curve of the form
Y =
where
the
respect to
Bofe(x)
Birfil)
eS
Bt
(x)
functions
fo,...,f,
are
polynomials
an inner product on P,-; defined by
that
are
arranged to be evenly spaced
are of degree 3 or 4 or less.
and
orthogonal
with
(p,q) = plx1)q( x1) + +++ + plxn)q(xn)
Usually,
x1,...,X, are
the functions f;,...,f,
In C[0O,2n]
£1htcosit
is
the
the
The
nth
orthogonal
polynomials
are the
onto W.
the
integral
cos 2tnuniey
orthogonal.
simply
with
spanned
weights
in
COS
order
the
product,
the
set
Sit. tasinel,
ie.,
sinnt;
Fourier
projection
by the
inner
of
approximation
f onto
functions
usual
formula
set
is obtained
by replacing
the
in (*).
for
If an application involves an interval
inner product requires an integral over
thogonal
sum
to
some
subspace
to
zero,
(*)
f in C[0,2m]
W of
coefficients
orthogonal
projection
[0,T]
[0,T],
t in each
is
trigonometric
The Fourier
the
and
of f
of
f
instead of [0,2m], then
and the appropriate or-
function
in
(*)
with
2nt/T.
6-22
CHAPTER
SOLUTIONS
a
1.
For
6
Orthogonality
and
Least-Squares
TO EXERCISES
the
data
are
(-2,0),
do fF2
trot
1
1
Since
other
«+
1
ve
(=1,0),
Design
matrix
(0,2),
Pine
Bo|
By
(1,4),
(2,4),
Parameter
vector
Fx
W =
6)
2
6)
@)
6)
6)
6)
2
0
6)
0
Owr?
0
2
6)
8)
O
O|,
fe)
1
so
WX =
0
6)
|2
Observation
vector
A
&
the first and last data points are about
points, a suitable weighting matrix is
1
0
|0
0
8)
‘construct
dine
22-2
{2
6)
2
2
1
2
half
as
reliable
as
the
.
and
Wy =
6)
6)
|4
8
4
The remaining calculations are the same as in ordinary
least-squares,
except that the weighted design matrix WX and the weighted observation
vector Wy appear in place of X and y, respectively.
The
2 aes eee
(Wx) T wx) a= |[ie e areas
1 -2
2
at (2 -20 ita
6
2)
- ("0 fa
ee
is. 2
T
1 2a
She (wy) = | =o.
2 Ree
Qe
0
DCGIAO
28
| 3 = [er
4
normal
solution
are
equations
and
[fo sells) = Bel [el = [oe] a] - [ore]
The
equation
of
the
least-squares
line
is
y = 2 +
(3/2)x
5.8
°¢
Applications
T
7. \Icos kt\|? = f CORKE “Cos KAR
fe!
sin 2kt
on
[5t+
Ak
|,
=
f(t)
=
below,
2m
-
9-0-8
2
re)
t.
we 9
ak
The
computed
ou
= (5-20 - 0) -O=n
(if k #0)
0
definite
integrals
in Example
4.
The
of
tcoskt
Fourier
Tl
and
tsinkt,
coefficients
ou
1
4u
of
shown
f are:
i
= 0 + -—(2n)"
0)
/lemS>
k # O)
a
‘h
G2e+
=
aoe ml, Cn ~
: Hat p
BB
= jaell
pel-p)Addn
(2m ep 2) andes
aiayel aioe
(if
6-23
Tl
(lesan
2
were
Spaces
2
0
0
9.
Product
on
m
a
Inner
a if 1 + cos 2kt 4,
6)
fone
of
=t1
(0)s
il
GA (2x. -~t)cos kt dt
ag eS
1
or 2ncos kt dt - 1a
=
0
0
t cos kt dt
6)
=0O0-0=0
ue
by = Ji (2n -
"nO
=O
The
-
2
Uae?
third-order
an+
2sint
+
t)sinktdt
am
= =| 2nsinktdt
0
2
k
Fourier
approximation
sin2t
+ fsin 3t
to
f is
-
Tl
ts tsinktdt
0
6-24
13.
CHAPTER
6
«+
Orthogonality
and
Least-Squares
((f + g),cosmt)
+
the
(g,cos mt)
by
equality
this
in
term
each
Dividing
= (f,cosmt)
Then,
integer.
Take f and g in C[0,2m] and let m be a nonnegative
linearity of the inner product shows that
conclude
we
(cosmt,cosmt),
that the Fourier coefficient a, of f + g is the sum of the corresponding Fourier coefficients of f and of g.
Similarly,
the Fourier coefficient b, of f + g is the sum of the corresponding Fourier coefficients
Of -f--and. Of «2.
Appendix:
The
The
argument
any
inner
u.
To
Linearity
for
space,
the
take
any
x and
y in the
space
and
x,uU
Sto
u,U
4)
u
ot,
u, U
and
cx, u
{yu
u,U
=
this,
=
y, Wy
Uu,, U4
is
are
any
4a be Stel
ob
a:
is a linear transformation.
for a subspace W, then the
nonzero
vectors,
then
any
scalar
c(x,u
—{-u
u,U
the
c.
Then
x,U
= c)}—
u
u, U
mapping
y,u
Up, Up
Thus,
mapping
In particular,
if W is the vector
order at most n, and if f and g are
proj
Projection
a special case of a general principle.
In
‘
y,u
;
“
mapping yo eu
is linear,
for any nonzero
if uwj,...,U,)
vr
Orthogonal
13
x+y,u
77D
u, U
Similarly,
an
Exercise
product
verify
of
mP
if {u,,...,uUp}
is an orthogonal
basis
yoproj y is a linear transformation.
space of trigonometric
in C[0,2n], then
polynomials
of
(f + g) = proj, f + proj.
That is, the nth order Fourier approximation to f + g is the sum of the nth
order Fourier approximations to f and to g.
Can you use the linearity of
the mapping froproj f and the final result of Example 4, to produce (with
practically no work) the answer to Exercise 9?
[Hint: The nth order Fourier approximation to a constant function is the function itself. ]
oe
es i he 3S
eee eee
ee
Chapter
6
¢
Supplementary
Exercises
6-25
CHAPTER 6 SUPPLEMENTARY EXERCISES
1.
Justifications
below.
Other
the
answers
to
the
True/False
justifications
are
possible.
length
zero
vector
False.
The
True.
See
True,
by definition
the
of
box
the
before
of
Example
questions
are
2 in Section
6.1.
distance.
||(-1)v||
False.
Orthogonal
True.
If xu
True.
2uev.
From the calculations on pages 373,
|lu + vl" = lull +
Thus |ju + vil = jlull” + |lvil° if and only if uev = O.
True.
The
False.
u, not
The orthogonal
y (except when
True,
because
the
True.
Given
w,
let
in
then
for
any
scalars
c(uew)
+ d(vew)
= 0.
So cu
False.
The
vector
True.
If vyevy
H,
# (-1)|lvl|
same
xev
argument
zero
for nonzero
nonzero
= 0 and
are
= O,
xe(u
as
H =
is
27
False.
A least-squares
vector
Ax
False.
AA
is
closest
By definition,
invertible,
Many students
+ dv
Certainly
(cu
is
both
of y (onto
+
dv)ew
in H.
W and
Thus
and
only
for
is square
28
solution
point
and
and
has
in H.
=
(cu)ew
the
multiple
of
in W.
If u,v are
+
(dv)ew
=
of R’.
= cyo;(O)
matrix.
= O.
See
orthonormal
Theorem
columns.
Decomposition
x (not
Ax) such
equations
are
A'Ax = A'b.
of the
normal
equations.
the normal
(r) and
by -v.
H is a subspace
Orthogonal
is a vector
b in Col A.
parts
+
6.2.
to
solution
is
Iv?
ae
in Section
Theorem
W)
0 is
a square
x is the
miss
c,d,
= O.
v replaced
true
Pythagorean
the
with
= cyey(vievy)
matrix
An orthogonal
is
applies,
(civ) s(eyvy)
False.
the
(g)
independent.
- v) = x*eu - xev
projection
in
is
True,
by
Theorem.
in
(Span{w})*.
then
Exercises
then
linearly
projection of y onto u is a scalar
y itself is a multiple of wu).
False.
The statement
107 in Section 6.3:
See
v.
vectors
orthogonal
= 0,
given
is zero.
False.
True.
Warning:
for
(s) of Exercise
1.
that
the
When
6-26
2.
CHAPTER
For
p =
6
«+
Orthogonality
Least-Squares
arc
2
the stated equality holds
that
shows
for p = k, with k 2
holds
equality
orthonormal set.
Suppose
KH = Cq1Vq +
888
+ CyaerVae1
for p = 2.
let
2, and
;
in
the
text
the
that
Suppose
be an
{v1,..-,Vkei}
== U + CKeiVe+1
O for
Vyasievy =
theorem,
Since
*°* + CyVy.
By the Pythagorean
Civ, +
to u.
where u =
orthogonal
;
Hint
The
|e,|~.
=
IIx =~{c,|"IIxl“
x = cv,
1 and
and
kg
j =
1 =
Vig
DOTS
x? = ful? + Wcxsavierl = Nu? + beysa!lvcell” = Wall? + IeperI?
and
by the
assumed
equality
for
p =
k
ul? = fol? + +++ + dexl?
Hence
III = le, |? fs “exes
leuer lee which shows that the
p= k + 1 follows from the equality for p = k.
By the
induction,
the equality is true for all integers p 2 2.
If Q=1 - 2uu’, then Q is square and Q' = Q, because
(uu')’ = uu’ =
uu .
for
By
Exercise
Q = 1. Thus,
by Theorem 6
10.
equality for
principle of
13(c)
in
the
QQ = QQ = I, which
in Section 6.2.
x
a
Letx = |yl, b= |b|, v=
Supplementary
shows
that
Exercises
Q is
1
ve
-2|, and A= |v} =
Z
c
a
given set of equations
is Ax =
solutions coincides with the set
T
T
A Ax = Ab.
Now
an
Chapter
orthogonal
1% -2yq5 5
1 -2
5|.
2,
matrix,
Then the
Vv
1
=2
5
b, and the set of all
least-squares
of solutions of the normal equations
Vv
A’ Ax
=Ivvv
v, x
=
(vv
+ vw
+
vv')x
Vv
tr
Column-row
expansion
in
2.4
Section
T
= 3vv x = 3v(vx) = 3(x - 2y + 5z)v
and
Thus
f
Ab=
|v
the
normal
v
vil
a
b|
c
= av + bv+ev=(a+b+te)v
equations
reduce
to
3(x
-
2y +
5z)v
=
(a +
b +
c)vy,
Which is equivalent to x - 2y + 5z = (a + b + c)/3, since v # 0.
The
points on this plane are the least-squares
solutions
of the original
system
(which itself describes
a set of planes,
all parallel
to the
least-squares
solution
plane).
Chapter
6
«¢
Glossary
to write
definitions
Checklist
6-27
CHAPTER 6 GLOSSARY CHECKLIST
Check
your
knowledge
by attempting
of
the
terms
below.
Then compare your work with the definitions
given in the text’s Glossary.
Ask your instructor which definitions,
if any, might appear on a test.
angle
best
(between nonzero vectors u and v in
the ....
Related to the scalar
approximation:
Cauchy-Schwarz
component
design
of
The
closest
inequality:
...
y orthogonal
to
u
point
...
for
all
u,
(for
u # 0):
matrix:
The matrix X in the
are determined in some
distance
between
u and
...,
denoted
n):
approximation
(of
order
Fourier
coefficients:
The
weights
The
vector
...,
....
where
the
columns
of
X
by dist(u,v).
The
used
between
v.
linear model
way by ....
v:
Fourier
R@ or R®): The angle @
product by: u-v=....
closest
point
to
....
make
in...
to....
Fourier
series:
An infinite series that ... in C[0,2m], with the inner product given by a definite integral.
fundamental subspaces
(of A): The - + Space and Space ofA,
2aNnG RENE 44.6.
space and ... space of A, with Col A commonly called the ....
general
least-squares
problem:
Given
Reentindic. 7 -such=thate.-.
Gram-Schmidt
inner
process:
product:
The
vectors
In
An
for
general,
a
function
on
-vectors,.u.and
product
space:
A vector
space
least-squares
line:
The
...
least-squares
solution
length
(of v):
linear
model
The
line
form
producing
(of
scalar
(of
a
on
b):
A vector
|x| =
...;
also
a vector):
space
that
vector
b
in
...
such
that
called
the
...
whose
assigns
in the
equation of the form
be chosen to minimize
entries
u and v are
of u and v.
to
to
each
certain
....
...
A vector
a
..........Ssubject,
is defined
minimizes
Ax =
and
as u*v,
where
called the ...
number
which
A
....
vector
~y..a
that
(in statistics):
Any
are known and B is to
mean-deviation
matrix
sess
scalar
...,
usually written
in R’ viewed as ....
Also
pair —of.
axioms.
inner
algorithm
an mxn
for oll
equation
....
....
of v.
...,
....
....
where
X and
y
b.
of creating
a
of
... solutions
....
tion yields all
mon notation is
process
The
v):
=
Ax
u that
vector
normalizing
(a vector
observation
y = XB
in the linear model
...
The vector
vector:
the entries in...
are the observed values of ....
where
the
ori-
orthogonal
complement
orthogonal
matrix:
orthogonal
projection of y onto u (or onto the line through
gin, for u # 0): The vector y defined by ....
u and
orthogonal
projection
Notation:
of y onto W:
y = proj. y-
The
unique
UChaAL jc as
orthogonal
set:
S of
such
that
orthogonal
to
W:
Orthogonal
to
every
orthonormal
basis:
A basis
that
is
orthonormal
set:
An...
set
of
....
parameter
vector:
The
unknown
regression
coefficients:
A set
The
the
QR factorization:
same
direction
matrix
...
.
vector
y SUCH
...
....
for
in the
linear
...
the
coefficients
in
model
....
.
quantity ... that appears in the general linear model:
difference: between’ .a-jandithens.
-svaiues (ofay)z
columns,
A
=
of
an
QR,
where
v):
Multiply
a vector
The
that
....
vector
a vector
analysis:
aliain
....
(as
trend
use
of
to
A
linear
matrix
an...
is
....
with
the
inner
combination
of
...
by
fit
Q
A with
is
A vector
...
mxn
that
linearly
matrix
whose
indea;
....
data,
product
....
inequality:
trigonometric
polynomial:
such as ...
vector:
weighted
vectors
The
gat
U such
matrix.
(a vector):
unit
set
R is an...
scale
triangle
The
A factorization
pendent
and
....
(of W):
A...
....
e&,
basis:
vector:
.,
that
+
orthogonal
residual
A basis
a com-
In statistics,
...
solu-
whose
...,
by
represented
equations
of
system
The
equations:
normal
a «
Bnd).
variables
...
involving
model
A linear
regression:
multiple
Least-Squares
and
Orthogonality
«
6
CHAPTER
6-28
A vector
least
alla
iy
v such
that
...
functions
....
Least-squares
dh) it a
and
problems
with
a
...inner
product
such
SYMMETRIC
MATRICES
AND
QUADRATIC
FORMS
7.1 DIAGONALIZATION OF SYMMETRIC MATRICES
To prepare
Theorem of
KEY
for this section,
Section 5.3, along
review Section 6.2 and the Diagonalization
with Example 3 and Theorem 6 of Section 5.3.
IDEAS
ae a
If a symmetric matrix has distinct eigenvalues,
as in Example 2, then the
ordinary diagonalization process produces a matrix P with orthogonal columns,
because
eigenvectors
from
different
eigenspaces
are
automatically
orthogonal.
However,
the P you need here must have orthonormal
columns.
Forgetting to normalize the columns of P is the main error students make in
this section.
The statements
in the spectral
theorem,
together with the general approach used in Example 3, lead to the following outline for orthogonally
diagonalizing any symmetric matrix.
Procedure
for
the
Orthogonally
characteristic
Diagonalizing
Matrix
A
1,
eigenvector
1.
Find
2.
For
and
3.
For each eigenvalue
with multiplicity k = 2, find a
of
k
vectors
for
the
eigenspace.
Then
use
the
Schmidt process to construct an orthonormal basis.
A.
The union of the bases for all the eigenspaces is an orthonormal basis for R"® (if A is nxn).
Use these vectors as
the columns of an orthogonal matrix P.
each eigenvalue
normalize it.
with
equation
a Symmetric
of A.
multiplicity
find
an
basis
Gram-
in an order corresponding
Construct D from the eigenvalues,
an eigenvalue
of times
The number
of P.
to the columns
to the dimension of
of D is equal
appears on the diagonal
the corresponding eigenspace.
5.
6. Finally,
SOLUTIONS
“noOONeer
1)
lt.
A = PDP” = PDP’.
TOOwes
EXERCISES
7| = A
E
Sieeart
e
entries on the main
because
the
diagonal
of
MAS =
f=
(‘é ax =
[p,;
pel.
columns
are
orthonormal:
=
(892
+8)
P is an
orthogonal
§
1
F
1 to
polynomial:
tl
1,
=
get
xg =
get
a unit
For
For
A = 2:
A= 2:
Take
xg =
1 to get
a basis
get
,
a unit
vector:
_
up =
|-1/V2
|7B |:
Set
i
P =
me
wus] =
MANS
ee
wal /v2
War
[u,
[
a basis
u
-1
Take
vector:
A-2I'0)
for
the
k/V2,
uw, =
by hand
calcu-
pi*pe
=
lone
I
Ipoll”
.48
-
.48
Since
=
0,
P
is
matrix.
So the eigenvalues
=
;
A=
ee
The
values.
P is orthogonal
similarly,
and
1,
Se
“lb Characteristic
3
(A - 4)(A - 2).
any
that
that
A=
have
show
show
square,
its
A can
match.
entries
(2,1)
and
(1,2)
To
lations,
pill
13.
Forms
Quadratic
and
Matrices
Symmetric
°
7
CHAPTER
7-2
1 eal
=
are
0
~
o|
(3
ryebe =e
1
sel
eigenspace:
(Don’t
forget
1
|
6)
1
4
the
eigenspace:
The
~
i=
ep ~ +
>
C0 rT
4 and 2.
f
for
-
0
X, 1 = X 2
|. Xo is free
sl
Then
this
step.)
1
t
6)
é
0
A
xX,
normalize
=
to
-x.
i
2
Xo is free
FRE
Then
normalize
;
corresponding
;
D is
4
i
to
0
3.
Study Tip: The fact that eigenvectors for distinct eigenvalues are orthogonal gives you a check on your work.
After you find ug in Exercise 13, verify that ug-u,; = 0.
When uw, and up are in R’, you can easily guess what uo
7.1
¢
Diagonalization
of Symmetric
Matrices
1-3
must
sure
be, once you know uw.
If you do
that ug is indeed an eigenvector.
19.
Be sure to work this problem before reading the solution. Use Exercises
13—24 to sharpen your skills.
They are critical for the rest of the
chapter.
Here,
Fornra:
= 7:
X, =
(1/2)x3,
A basis
FOr
=
A
basis
for
-X3,
Xq
=
for
the
to
FAcl@
O} =)
jana
for
the
(Also,
check
that
Sete Ps= [ujeatsueug]»=
the
you
eigenvalues
are
Take
xg = 2 to avoid
1
2|;
2
a unit.
2
6)
a
6)
O
2-90
i)
0
aw
is
free.
Take
ae
|-1];
2
a
x3
ORL
1
¢)
2
—1
6)
4,
1.
6)
6)
0
fractions.
1/3
Be :
273
1
it
1/2
6)
8)
0
O
2 to
avoid
fractions.
us
=
=273
|-1/73].
2/3
=
272
|-2/3].
173
unit
6)
6)
1
6)
x3
=
eigenvector:
2
12
6)
2
3
2
6)
2
4
O
Oi
0
1
WO
O
Xo
ws
siree.
fake
x,
a
unit
eigenvector:
ug*ue
= 0.)
= 0.)
|2/3
-1/3 9 -2/3|,
e273
2/3
2/3
273
173
b/3
7,
uj, =
up*u,
= O and
to make
eigenvector:
that
2
|-2];
i!
Aus,
given:
x3 is free.
eigenspace:
ug*-u,
compute
1
sO
¢)
eigenspace:
mentally
the
should
8)
Ohe~n
6)
and
check
you
0
2
=2
=
2
6)
OS
Nor=g—-2xXsy
and
-4
2
ena-c
6)
2
and
-(1/2)x3,
A basis
Because
freedom
¢)
2},
5
eigenspace:
PA
—-ay
Aasiiey
SueXay
the
zZ
4
2
60) =
Xo = x3,
Xo
(Remember
For
whA=7I
eke sea
Xy
3
[2
0
A =
this,
On
d
8)
=)
t=2
2
6)
6)
6)
0
1ug
hh
10)
0
@)
0
d
Vand D =.|0
».4.<,0).
the only
A in this exercise has three distinct eigenvalues,
have in choosing P is to rearrange its columns (and change
Forms
Quadratic
and
Matrices
Symmetric
«+
7
CHAPTER
7-4
the
(because
of P by -1
and to multiply any column
D accordingly)
above
P
Notice how the matrix
length).
must have unit
eigenvectors
differs from the P in the text’s answer section.
21 and 22 have only two distinct
in Exercises
The matrices
Study Tips:
two.
least
be at
must
eigenspace
of one
so the dimension
eigenvalues,
You will have to construct an orthonormal basis for the eigenspace.
(Why?)
Exercises 23 and 24 are good models for exam questions because they give
you information from which you can quickly diagonalize A (orthogonally).
25.
29.
a.
See
Theorem
2 and
the
paragraph
preceding
b.
See
Theorem
1.
c.
See
3.
d.
See
the
paragraph
Example
following
formula
the
theorem.
(2) and exercise
35.
By hypothesis,
A = PDP’,
where
P is orthogonal
and D is diagonal.
Since A is invertible,
O is not an eigenvalue and D is invertible.
Then
es
es es es es es
A 1vee CPRP
) 3 ="
) eb Pe FDP |
Since
D
A
is diagonal,
second
invertible,
so A’
A
argument:
a property
is symmetric.
is orthogonally
By
Theorem
of
2,
transposes
By Theorem
diagonalizable.
A
is
symmetric.
Since
shows
that
(A y7 =
again,
A
is orthogonally
product,
or
2,
A
(A")*
=
is
Agee
diago-
nalizable.
31.
The
35.
a.
solution
is
in
the
text.
is
The
matrix
B =
uu’
in
R,
=
(uu )x
;
n
Bx
an
=
outer
u(ux)
=
(ux)u,
rank
because
i matrix.
E
C
ux
is
Given
a
Using dot products,
Bx = (xeu)u.
Since u is a unit vector,
the orthogonal projection of x onto u.
See Section 6.2.
b.
B
is
symmetric,
(uu )(uu
MATLAB
because
) = u(uu)u
Orthogonal
B’ =
= uu
(uu’)?
= B,
=
because
uu
=
uu
="
Bp.
this
Also,
with
Diagonalization
v2 = v2 -
a basis
{v,,vo},
use
(v2’*v1)/(
v1’ *v1) *v1
the command
is
Be =
uu = 1.
The command
[P
DJ] = eig(A)
orthogonally diagonalizes
any symmetric
matrix A, but you miss the opportunity to learn the procedure of this
section.
Instead, use
eig(A)
for eigenvalues and
nulbasis
to obtain
eigenvectors,
as in Section 5.3.
If you encounter a two-dimensional
eigenspace,
x
scalar.
7.2
¢
Quadratic
Forms
7-5
v2 = v2 - proj(v2,
vi)
to make the
The command
new eigenvector vz orthogonal to v;.
proj
was introduced in the MATLAB
After you normalize the eigenvectors and create
to verify that P is indeed an orthogonal matrix.
(Review Section 6.2.)
note for Section 6.4.
P,
check
that
PP
=
I
7.2 QUADRATIC FORMS
This
of
section,
the
together
with
Section
7.1,
forms
the
foundation
for
the
rest
chapter.
KEY IDEAS
i aE.
The main point here is to learn how a change of variable,
x = Pu, with P an
orthogonal matrix, can transform a quadratic form into a new quadratic form
with no cross-product terns.
If you study the various classes of quadratic forms (or, equivalently,
classes of symmetric matrices),
you should learn both the definitions and
the characterizations
(in Theorem 5) of these classes.
Exercise
“24 “describes
another
useful
way to characterize
quadratic
forms.
Crhe *2°x2
case can be generalized to nxn matrices.)
SOLUTIONS
TO EXERCISES
ae
aay
a ae:
ix
x! Ax
5
xa) |
195
1/3
1 ed eee
= 5x; + (2/3)x4X2
7.
5x, + (1/3)
xal| (1/3) + a
+ x5
bua Wheres =: (6,1) box
Axn=.5(6) >+ (273)(6)
01), ee (14
Beynon
Ace S(1) + 4)
The
matrix
Sul 123).
of
the
x
quadratic
form
yay
is
A =
tes
iave+ats) += 16
ale 1/°
5
The
eigenvalues
polynomial
is rn? - 2a - 24 =
(A - 6)(A + 4);
For A = 6:
an eigenvector
‘
is
1
:
ik normalized:
Be
characteristic
are
il ha:
1|
uw, = z3|
6 and -4.
+
Symmetric
For
io
A = -4:
an
‘
eigenvector
Thus
A =
The
desired
CHAPTER
PDP’
-1
Pd SE
Pa
change
of variable
is x =
Use
3
oP Ashes
P =
when
P AP
=
AP
P
D =
and
Weed:
normalized:
ra
1\?
:
is
Forms
Quadratic
and
Matrices
7
7-6
Py,
ali
so
Ti=t
1
y2|
d6sa9
i
D
i and
lo -al:
that
x'Ax = (Py)"A(Py) = y PAPy = y Dy
2
(*)
2
= 6yi - 4y2
Study
Tip:
To
make
should:
(1)
write
algebra
in
(*)
new
the
the
that
"change
of
variable"
equation
x =
Py and
produces
the
new
requested
ply
13.
The
of
the
quadratic
form
7,
(2)
show
the
matrix
form;
and
(3)
include
the
information
on an exam.
you
should
sup-
quadratic
quadratic form.
Find out how much of this
if a problem like Exercise 7 were to appear
matrix
Exercise
P;
specify
is
A =
ss
‘sh
polynomial
is ae - 10A = A(A - 10); the eigenvalues
the quadratic
form is positive
semidefinite.
To
variable,
proceed as in Exercise 7:
For
A ==
j
10: . an eigen
vector
For
A
O:
Take
A,
P =
the
an
?
eigenvector
1
hoe
715 |-3
desired
3
: and
change
i
is
3
:
lI: normalized:
10...
k
D =
of
0
,
af Since
variable
is
x =
Py,
The
you
characteristic
are 10 and O.
Thus
find the change
of
1
l
i
es
I normalized
::
:
is
in
:
u, == i0|-3|
us
os f eases
= 715|
if
P orthogonally
A
F
diagonalizes
and
T
x'Ax
= (Py)"A(Py) = y'P'APy = y'Dy = 10y*
The
19.
new
quadratic
Because 8 is
possible.
The
Xx, = O and
a
c.
form
is
10y%.
larger than 5, you
constraint x; + xg
xg =
1, the
value
of the
Example
See
the
definition
before
See
the
paragraph
following
See
the
Principal
Axes
Section
5.3).
should make the xs term as
= 1 keeps xg from exceeding
quadratic
is 5(0)
+ 8(1)
= 8.
1.
Example
Theorem
form
large as
1.
When
3.
and
the
Diagonalization
Theorem
(in
7.2
25.
dg.
Check
f.
See
the
the
definition.
Numerical
The
text’s
answer
positive
definite,
||Bx||/" = O and Bx =
this
27.
case,
the
e.
Note
after
See
Theorem
Example
¢
Quadratic
Forms
7-7
5.
6.
showed
that
xB’ Bx = 0. “for abi
x.
To show
B'B is
suppose
that x B' Bx = 0.
Then
(Bx) "Bx = 0, so that
0. If B is invertible,
then x = 0, which shows that, in
form
x B Bx
is positive
definite.
The
quadratic
forms
x Ax and
x'Bx are
both
positive
definite,
by
Theorem 5, because all eigenvalues of A and B are_positive.
Then for
any nonzero x
x (A + B)x = x (Ax + Bx) = x Ax + x Bx > 0, so the quadratic
form
x (A + B)x
symmetric,
because
positive
eigenvalues.
Mastering
Linear
is
positive
(A + B)
Algebra
= A
Concepts:
definite.
+ B
Also,
= A+B.
the
matrix
By Theorem
Diagonalization
and
5,
Quadratic
A +
B is
A + B has
Forms
Since
the end of the course
draws
near,
I recommend
that
you prepare
a
review sheet that contrasts the Diagonalization Theorem in Section 5.3 with
Theorem 2 in Section 7.1.
You might begin by copying the statements
of
these theorems,
and then use the following questions to guide your review:
>» What special
not
present
eigenspaces,
properties does an orthogonal diagonalization have that are
in all
diagonalizations?
Consider
the eigenvalues,
the
the eigenvectors,
and the matrix P in PDP
.
> Suppose A is symmetric,
and all eigenspaces are one-dimensional.
What
differences are there between a general diagonalization and an orthogonal
diagonalization of A?
What differences are there when A is symmetric and
one eigenspace is two-dimensional?
> Why is an orthogonal diagonalization needed to simplify
Make the change of variable x = Py in x Ax and show the
a quadratic form?
algebra involved.
>» If
you
Figure
studied
3
differ?
in Section
in
Section
Section
they
Why do
5.7
be
5.6
5.7.
or
5.7,
How
do
differ?
associated
compare
4
in
Section
shapes
of
the
trajectories
Figure
5 in Section
5.6
or
a symmetric
matrix?
the
Could
with
Figure
general
5.6
with
Figure
5
CHAPTER
7-8
+
7
Matrices
Symmetric
and
Forms
Quadratic
7.3 CONSTRAINED OPTIMIZATION
This section is important in its own right, since constrained optimization
The main
problems and applications.
problems arise in many mathematical
sections.
two
following
the
in
used
also
are
section
the
of
results
KEY IDEAS
a ae
The maximum value of a quadratic form x’ AX
Theorem 6 gives the main idea.
over the set of all unit vectors can be computed by finding the greatest
eigenvalue of A; this maximum value is attained at a corresponding eigenvector.
The key step in the proof is to diagonalize A by P, substitute x =
Py, and use the fact that in this case, x and y have the same norm.
Example 6 presents a topic that is widely discussed in elementary economics texts.
SOLUTIONS
aaa
aa TO
ena EXERCISES
cane
1.
We
are
given
an
equality
of
5x4 ne 6x5 + 7x5 + 4xX4Xq
two
-
quadratic
forms:
4Xx0x3 = Qys ies 6ys a 3y5
)
2
6)
|2
6
4c21:
Ogr=2
Z
The equality between the two quadratic forms indicates that the eigenvalues of A are 9,6,3.
(Proof: The diagonal matrix D of the quadratic
form Sy; + 6y2 + 3y3 obviously has eigenvalues 9,6,3.
Since A is similar to D, A has the same eigenvalues as D.)
The standard calculations
produce a unit eigenvector for each eigenvalue.
Don’t forget to norThe
matrix
malize
A=
of
the
left
each
eigenvector.
3)
1/3
Us = -b 2/3/35
-273
quadratic
A=
6:
form
Up =
obviously
2/3
|173];
27/3
is
A = 3:
A
=
Ug =
=273
2/3
1/3
These eigenvectors are mutually orthogonal
because they correspond
distinct eigenvalues.
So the desired change of variable is
x = Py,
where
P =
T7Se
273
Sess
to
es Se ers
173
2/3
ors
i723
Study Tip: Review the matrix algebra that leads
be sure you can show that ||x|| = |lyll when P is an
from x’ Ax
orthogonal
to y Dy.
matrix.
Also,
7.3
1%
The
matrix
of
Q(x)
=
~2x4 =
¢
x5 + 4x1Xp
Constrained
+ 4xoxg3
is
Optimization
7-9
ee
|2
0
Bilis
A=
2
-1
6)
2
0
The hint in the exercise lists 2, -1, and -4 as the eigenvalues.
The
greatest eigenvalue is 2, not -4, because "greatest" here refers to the
eigenvalue that is farthest to the right on the real line.
The maximum
is attained at a unit eigenvector
value of Q(x)
(for x a unit vector)
for A= 2.
Standard calculations produce the eigenvector:
vi
=
1/2
i
1
|, scaled
1
| 2],
2
to
and
normalized
Warning: Exercise 7 illustrates the
of 2 as the greatest eigenvalue.
potential
12.
using
This
exercise
direct
13.
can
be
computation,
done
using
by
the
hint
to
error
of
a
theorem,
in the
text.
1/3
|2/3}].
2/3
w=
selecting
but
try
-4
to
instead
do
it
by
If m = M and x is the unit eigenvector u,,
=m.
Otherwise,
set a = (t - m)/(M - m).
then x Ax = uy Au; = uy (mu; )
The hint in the text shows
that
a,
O = a = 1 when
m = t = M.
For such
let
x = Vi-au,
+ Vow.
Then
the
vectors
Vi-au,
and
Vau,
are
orthogonal
because
they
eigenvectors
for different
eigenvalues
(or one of the vectors
is
By the Pythagorean Theoren,
ole
xx = |/xil°
2
= |VWi-aull~
2
+ ||Vvou;||
11 - a|ilunll”
=(1-a)
2
+
are
0).
2
Jol lus
2
t+a=1
because u, and u; are unit vectors and
that u, and u, are orthogonal,
we have
0 = a=
1.
Also,
using
the
fact
m and
M for
x'Ax = (Vi-au, + Vau,) A(Vi-au, + Vau;)
(Vi-au, + Vou) (mVi-au, + MVau;)
j1 -
o.| musu, +
Thus the quadratic form
suitable unit vector x.
la |Muju, =(1-a)m+
x Ax assumes
every
oM=t
value
between
a
Symmetric
°«
7
CHAPTER
7-10
Forms
Quadratic
and
Matrices
7.4 THE SINGULAR VALUE DECOMPOSITION
It completes the story of the
This section is the capstone of the text.
in so doing,
xt» Ax for a general mxn matrix A and,
linear transformation
gives you an opportunity to review many basic concepts from Chapters a—7.
this section opens the door into the modern world of applied
In addition,
is
An understanding of the singular value decomposition
linear algebra.
essential for advanced work in science and engineering that requires matrix
computations.
KEY IDEAS
i
eee L
The
first
singular
value
o,
of
an
mxn
matrix
A
is
the
maximum
of
||Ax|l
over all unit vectors.
This maximum value is attained at a unit eigenvector
v, of AA corresponding
to the greatest
eigenvalue
A, of AA.
The second
singular value
is the maximum of ||Ax|| over all unit vectors orthogonal
to
vi.
The following algorithm produces the singular value decomposition for
A.
(As mentioned in the text, other more reliable methods are used in pro-
fessional
software. )
Procedure
1.
for
Find
an
Computing
a Singular
{v1,...,Vp}
for
R' consisting
the
and)
associated
«AW 4s
Construct
V =
[v,
the
oj = VA;
matrix
nxn
basis
Decomposition
eigenvectors of AA,
arranged so that
values
satisfy
Ay = *°"
2 A, > *Os
where r = rank A.
Let
orthonormal
Value
orthogonal
matrix
(1 = j <n),
and construct
(j,j)-entry
is o;
= whose
the mxn
(1 = j = n)
of
eigen-
diagonal
and
has
zeros
elsewhere.
The
uy
set
=
Extend
R.
{Av,,...,Av,;}
(1/0) Av;
a
{U,,...,Up}
Write
the mxm
is
orthogonal
and
o; =
||Avj||.
Compute
< Defoe a
‘tow an’
orthonormaly
orthogonal
matrix
basis .{uy,..
U =
[u,
<--
Aigh. for
ul.
A = usy',
The diagram on the next page illustrates how the SVD splits the action
of YATInNtG first multiplication by ve (which amounts to an orthogonal change
of basis
in R'), then a scaling by = in the directions
of the standard
basis
vectors
€1,---5€n,
and finally
multiplication
by U (an orthogonal
change
of basis
inR
).
7.4
V
Note
that
v1 ————>
The
Value
the
U;,..-.,U,
appear
SOLUTIONS
TO
E
OF
Singular
so-called
U
At=
appears
on
left
on
the
the
right
oe,
of
Decomposition
———>_
Decomposition:
in
side
Value
7-11
U
singular
left
Singular
vectors
A =
of
A
the
factorization
the
diagram
o1
US"
are
of
the
columns
of
A),
even
though
above.
EXERCISES
iI. AA
=3
(Remember
=
to arrange
:
2 fa
compute
a
Eigenvalues
8 eee XS
the
The singular values are
same shape as A, and
Next,
The
=
Co
e, ——!—»
(because
1..
°¢
eigenvalues
o,
=
v9 =
and
eigenvectors
in decreasing
3 and
op
=
1.
of
order.)
The
matrix
AA
are:
Thus
©
is
the
U
CHAPTER
7-12
and
Symmetric
+
7
we}
sald
normalize:
u, = ree
=
5|_3
the
so
R?,
for
basis
a
already
is
{u,,ue}
Finally,
Forms
Quadratic
and
Matrices
Con ee gse
Gol
Rieti
sa LA
7.
A=
E
seit
ins
mas en
Ik
Find
af siecarnas
wfod oh a
#2)
AA
characteristic
=
E
NM
o1
A,
=
9
for
This
happens
qual
EB
st
aH
the
eigenvalues
2
Ris
to
V.
0
of
AA
from
omitted)
are
the
equation.
O.= AY - 890. + 86. =nblseOhorond4
Corresponding
| Vig)
basis
unit
vi
i
=
eigenvectors
27¥5.
[Ve |)
for
wid,
Ao
=
A:
A, = 9,
AA
ve
Ag=4
(calculations
_
=
f-17v5
[2/8 |
Take
yx
2/VS5
WS
The singular
the
same
-1/V5
OM2//5
values
shape
Moeieide
Sieahiow,
as
are
A and
aay fe/Vente
2)117vsi
To
check
your
work
at
orthogonal.
(They are.)
o; = V9 = 3 and op = V4 = 2.
The
&£ =
compute
ins 0
eae
le7vs |”
this
Then
.
Since
oust
| SI6/V5
{u,,u2}
is a basis
5
Azusyt = [17V8
2/V5
2
for R°,
-2/v8][3
1/VS||0
g .
Next,
[2 SBT [Sia
lo" oll o/yel
point,
verify
normalize:
2/v5
,
é
Mae
V2
& -Wtipe Get fens orifice
ST
=
that
matrix
|
Av,
2.05
take
U =
o]f
a5
_
2ll-1/445
62|¢27v5.1
and
[ivv5
Ee
15
2v5
=)
Avo
Iv
-2/Vv5
AB |: Thus
t
is
ees
eye
ie 1s Lenyta7vS
ee eve
|’
=
VeRE
Ya? abe its
are
7.4
Study Tip:
Your answer
that given in the text.
then A = UZV
and your
thogonal matrices).
13.
The
T
matrix
AA
is
¢
The
Singular
Value
Decomposition
for a singular value decomposition may differ from
To check your work, compute AV and UZ.
If AV = Uz,
(provided U and V truly are oranswer is correct
3x3.
Because
the
text
has
not
given
you
computing and solving a cubic characteristic
equation,
the
gests that you consider A
instead of A.
(You are free to
itself,
if you prefer.)
Using
Et
ea
i
lowe
(ATTA = ast = |?
The
characteristic
ee
ee
ee?
equation
A,
compute
3
2
me
eb
a]
= [
corresponding
er.
A, = 25:
Mane A
vy =
unit
eigenvectors
/V2>)| 2
singular
values
are
size
as
To
u,
Ae:
get
P
and
3
2
2
-2
9)
and
Nee
the
matrix
ne
5 and
op = 3.
Thus
us,
V are
Oar eloay von»
Vo =
compute
1/V2
Av;
-1/v2
and
|
LO
WA
WAC
Vem
ee
5
6)
WB
= is
|0
6)
3],
6)
the
|
same
A'vo,
5/v2.—s-1/V2
0)
1/v2
-4/V2
-1/V18
|1/v2],
ue = | 1/V18
6)
need_one
tions
-4/V18
vector,
more
ux = 0 and
+
v2u;x
=
Vi8u3x
The
Peed:
|
A295 93
o, =
and
sugon A
normalize:
u, =
We
eM
lie |
The
practice
Hint
work
is
Ose N= 5230) 40905. = (1-295) (A
The
7-13
=
ux
Ae
0
solution-is
to
u,
and
solve
for
x.
Simpler
x, = -2x3,
+ Xp -
write
the
equations
are
So
equa-
=
X1 + Xo
-X,
ug.
orthogonal
= O and
=0
4x3 = 0
X2 = 2x3,
X3 free.
A suitable
unit
vector
is
7-14
Forms
Quadratic
and
Matrices
Symmetric
+
7
CHAPTER
-2/3
u3
=
2/3
V3
Thus an SVD of A’ is
A
So
an
T
=
[uy
SVD
A=
of
Us
us}
5
0)
6)
3
O
6)
A appears
Ke
[v3
vo]
by taking
“rele
transposes:
Wye
ive
)
4 -1/V18
2075
1/V18
273
-4/vi8
1/3
This is an SVD because the outside matrices
are orthogonal
matrices,
and the center matrix is a diagonal matrix of the proper type.
Another
way to find uz is to realize that u, and us form an orthonormal
basis
for
ColA
=
RowA.
Let A = UxvV'.
Then
The
remaining
uz
must
be
a basis
for
(Row
A)”
=
Nul A.
19.
A'A = (UEV")"UZV" = Vz'U'UEV"
=
viz'z)v
If
o,,...,0,
nal,
with
are
the
diagonal
Because
nonzero
entries
U and
V are
diagonal
o1,...,0;,
orthogonal
entries
and
in_z,
possibly
then
some
yah > is
zeros.
diagoThus
V
diagonalizes AA.
By the Diagonalization Theorem in Section 5.3,
the
columns
of V are eigenvectors
of AA,
and O7,...,00 are
the nonzero
eigenvalues of AA.
Hence o;,...,o0, are the nonzero singular values of
A?
K Similar calculation of AA shows that the columns of U are eigenvectors of AA.
23.
From the proof of Theorem
10, UZ = [o,u,
°::
o,uU,
column-row expansion of a matrix product shows that
0
= ssn
042
The
T
A =
(UZ)V’
T = (UZ) V1
=
T + *¢*
= 04U,V,)
T
+ O,UpV;
Vn
This
expansion
generalizes
the
spectral
decomposition
in Section
7.1.
7.4
¢
The
Singular
Value
Decomposition
Study Tip:
In Exercise
23,
the left Singular
vectors
are
U1,...,Um, Of the left factor U in UZV, but the right singular
the columns of V, not V.
25.
Consider
the
{v1,...,Vn}
SVD
and
of V and U,
orthonormal,
tity matrix.
T(v;)
So
=
for
©
the
=
Av;
=
USV'v,
=
The
command
Singular
[P
D]
Uze;
Formula
Section 5.4 shows
that
relative to B and 6&.
The
matrix
be
of
bases
T,
say,
A =
constructed
the columns
vectors are
usV’.
from
the
Let
the
Value
=
=
Uc je;
(4)
=
oe;
=
o ju;
in the discussion
"diagonal"
matrix
=
at the beginning
of
is
T
the
matrix
V = P(:,[1
of
Decomposition
eig(A’*A)
produces
an
orthogonal
matrix
eigenvectors
and a diagonal
matrix D of
eigenvalues
of AA,
but
eigenvalues
in D may not be in decreasing order.
In such a case,
will have to rearrange things to form V and = (denoted below by S).
For instance,
if P is 3x3,
the command
interchanges
B=
columns
respectively.
Observe that,
since the columns of V are
Vv; = e;, where e; is the jth column of the nxn
idenTo find the matrix of T relative to B and 6, compute
[T(v;)], = ojej.
MATLAB
standard
{u,,...,u,}
7-15
P of
the
you
3 2])
columns
2 and
S = zeros(size(A));
3 of
S(2,2)
P to
form
V.
The
commands
= sqrt(D(3,3))
produce a zero matrix for "ZX" the same size as A and place the square
root of the (3,3)-entry of D into the (2,2)-entry of S.
Other diagonal
entries for S can be entered similarly.
To form U for the SVD, normalize the nonzero columns of A*V. If U needs more columns, use the method
of Example 4.
This construction helps you to think about properties of the factorIn practical work, however, you should use the much faster and
ization.
[U,S,V] = svd(A)
more numerically reliable command
CHAPTER
7-16
«
7
Symmetric
Matrices
and
Quadratic
Forms
7.5 APPLICATIONS TO IMAGE PROCESSING AND STATISTICS
or if you plan
If you find remote sensing or image processing interesting,
then you will want
later in your career,
statistics
to use multivariate
an
finding
difficulty
have
may
You
thoroughly.
section
this
to study
The idea for the applielementary explanation of this material elsewhere.
in my
algebra
a student
linear
from
came
processing
cation
image
to
in remote
class—a geography major who was taking an undergraduate course
sensing.
The book by Lillesand and Kiefer, referenced in the text, was one
of the texts for her course.
KEY IDEAS
St
ae
The first principal component of the data
a unit eigenvector u, corresponding to the
in the matrix of observations
is
largest eigenvalue of the covar-
iance
the
matrix
in u, are
weights
in
a linear combination of the original variables,
x;,...,Xp,
that
new variable y, (sometimes called a composite score or index):
creates
a
¥1
=
—_
S.
uUyX
uy
If uw, =
=
—
CyXy
+
(c,,...,cp),
eee
+
then
entries
CpXp
The variance of the values of this index is the largest possible among all
indices whose coefficients c,,...,c,) form a unit vector.
(The variance of
yi is the largest eigenvalue of S.)
The second principal component
is the
unit eigenvector corresponding to the second largest eigenvalue of S.
The
entries in the second principal component determine the index with greatest
variance among all possible
indices
(from a unit vector)
that are uncorrelated
(in a statistical
sense) with y,.
Additional
principal components
are defined similarly.
Checkpoints:
(1) If the variables x, and xg are uncorrelated,
what can you
say about the covariance matrix S?
(2) What is the covariance matrix of
the new variables yi,...,yp formed from the principal components of S?
SOLUTIONS
1. The e
TO EXERCISES
matr
mat ix
sample
mean
Bo
ft,
2
of
obse
b rvations
M is
12
Fab
10. 6
-4 -1
.
is
xX
Subtract
oe ome
5
3 -5
ig
2e>
E
Oo
6
9
M from
each
column
rs
4548
eee
es) |: and
5
of X to obtain
the
7.5
The
sample
¢
Applications
covariance
matrix
division
Processing
and
Statistics
7-17
is
-6
_
860
=27
SO lt sa
=27
16
-135|
430
§ 1-135
Study Tip:
N,
but
for
Image
if
a
10
-6
-4
-1
Care-9 pe-10ON 8 | aca
10
= 5 [2 CO
yer BB Tey.
s Peet!
es x
to
Note that the formula
statistical
reasons,
for
the
-10
3
8
=
ra,
ae,
contains
sually,
decimals.
the sample mean involves division by
covariance
matrix
formula
involves
by N-1.
Let x1,,X2g denote the variables for the two-dimensional data in Exercise
1. The characteristic equation of the covariance matrix S from Exercise
1 is A - 102A + 647 = 0.
By the quadratic formula,
the roots of this
equation
are
A, = 95.20
and
Az = 6.80
(to
two
decimal
places).
The
first
principal
component
of. the
data
is a unit
eigenvector
corresponding
to
A,,
which
turns
out
to be
(-.95,
.32),
or
(.95, -.32).
The two possible choices for the new variable are y; =
-.95x, + .32xXo
and
yy, = .95x; - .32x5.
The variance of y, is 95.20,
while the total
variance
is 95.20
+ 6.80
=
102.
Since
95.20/102
=
.933,
the
new
variable y, explains about 93.3% of the variance in the data.
i
a.
The solution
in
form, where Y, =
the
T
P X
b.
By part
text shows
that
the Y,
F
for some pxp matrix P.
(a),
the
covariance
wei /%1
ae
YnI TY,
matrix
1
=
of Y,,...,¥y
T
b geP im
wai
1
- Per
okt:
ms
ae
Xx
Samcen®
Yn]
ae[X,
[P
-c
Marl he
ee
‘
xi) |
x T ii
= P'SP
because
X;,...,Xy
are
in mean-deviation
form.
are
is
in
mean-deviation
CHAPTER
7-18
13.
Let
+
7
Let B = [%,
xX, 6- M.
By the
deviation form.
ance matrix is
the
of
mean
sample
the
M be
Quadratic
and
Matrices
Symmetric
and
data,
k =
for
Forms
1s raps
2
ivondte
the matrix of observations
expansion of BB, the sample
Rn,
mel
column-row
for
in mean
covari-
PH
N
=
N
oe
XR
1
—
ea)
the
are
M) (X,
ae M)"
1
1
Answers
(2) The
=
to Checkpoints:
(1) The (1,3)-entry and (3,1)-entry of S are zero.
covariance matrix of yi,...,Yp is the diagonal matrix formed from
eigenvalues of S.
This
pairwise uncorrelated.
MATLAB
Computing
matrix
Principal
is
diagonal
because
the
new
variables
Components
The command
mean(X’)
produces a row vector whose
average of the jth row of X, and
diag(mean(X’))
jth entry lists the
creates a diagonal
matrix
of
whose
diagonal
entries
not to use
mean(X) ,
Finally,
the command
the
are
the
row
averages
which lists the averages of
diag(mean(X’))*ones(size(X))
size
of
X,
whose
averages
of
X.
To convert
columns
are
the
all
data
the
same,
in X into
each
X.
(Be
careful
the columns of X.)
creates a matrix
one
listing
mean-deviation
the
form,
row
use
B = X - diag(mean(X’ ))*ones(size(X) )
The
sample
covariance
matrix
is produced
by
data
is produced
by
S = B*B’ /(N-1)
The
principal
[U,D,V]
component
= svd(B’ /sqrt(N-1))
The columns
of V are
the principal
components
of the data,
and
diagonal entries of
D*2
list the variances of the new variates.
the
Chapter
7
*
Supplementary
Exercises
7-19
CHAPTER 7 SUPPLEMENTARY EXERCISES
1.
Justifications
below.
Other
for
the
answers
justifications
to
are
the
True/False
questions
are
given
possible.
a.
True,
by Theorem
2 in Section
b.
False.
Counterexample:
A =
ec.
True.
\|Ax ||° =
= x'AlAx = x'x =
d.
False.
The principal axes of x Ax are the columns of any orthogonal
matrix P that diagonalizes A.
Note: When A has an eigenvalue whose
eigenspace has dimension greater than 1 (for example,
when A = I),
then the principal axes are not uniquely determined.
e.
False.
(Ax)'(Ax)
Counterexample:
7.1.
a i
P =
;
Se
orthonormal
columns,
then P’ = P.
f.
False.
See
Exercise
11
g.
False.
Counterexample:
ce > 0,
put
x’ Ax
is
h.
True, by the
tic form can
i.
False.
j.
False. The maximum
otherwise x Ax can
k.
False.
See
Any
an
If
A
Example
=
indefinite
Principal Axes
be taken to be
If
P
is
and
x
value must
be made as
E
|
quadratic
of
an
|[xil, all
nxn
matrix
x.
with
=
als
then
x Ax
=
form.
the
matrix
of any
quadra-
set
unit
vectors;
7.3.
computed
large as
change
|/Ax|| =
7.2.
Theorem, since
symmetric.
3 in Section
orthogonal
definite form
the matrix of
and therefore
over the
desired.
variable
x
=
Py
of
changes
a
positive
x Ax into another positive definite form,
because the
the new quadratic form is P AP,
which is similar to A
has the same eigenvalues as A.
term
“definite
1.
False.
The
m.
True.
If x = Py,
n.
False.
Counterexample:
orthonormal
in Section
IIxil*, so
eigenvalue"
then x Ax =
to make
Let
UU'x the
is undefined.
(Py) A( Py) = x P' APy = xP
U =
3
a
orthogonal
The
columns
projection
of
APy.
U must
of x onto
be
Col U.
6.
Symmetric
CHAPTER
7
+
o.
True.
See
Examples
p.
True.
See
Theorem
q.
False.
The
gular
values
7-20
1 and
10
singular
of
AA
=
symmetric,
A is
Since
{ttg,
. fxpuah efor R®.
Let
because
otherwise
2 in Section
in Section
values
E
Quadratic
and
Matrices
of
Forms
7.4.
7.4.
A =
+ are
E
i] are
4 and
1.
1,
~but“the’sin-
basis
eigenvector
orthonormal
We may assume
that O < r < n,
an
is
there
r = rank A.
the decomposition
“2 and
of Exercise
4(b)
is either
y = 0 +
y or y = y + O, and we are finished.
Note that dim Nul A =n-r,
so O is an eigenvalue with multiplicity
n- r, and we can assume that u,,...,u, are the unit eigenvectors corresponding to the remaining r nonzero eigenvalues.
(See the solution
to, Exercise,.3.)...:.By, Exercise
5. uj#.tiUn
aren,
Gots:
Also,
Up41,
.,U, are in NulA, because
these vectors
are eigenvectors
for A = 0.
For any y in R", there are scalars cj; such that
Y =
CyUy
a
+ cee
era
+ CphUp
tarmac
+ Cpr4iUp41 F 88% F CyUy
[aie amine Sevee =warns ed 2 sen
a
y
The
vector
y,
shown
Zz
above,
is
in Col A and
the
vector
z
is
in
Nul A.
Teele
A=
R'R,
Exercise
25
where
R is invertible,
then
A is positive
definite,
by
in Section 7.2.
Conversely,
suppose
that
A is positive
definite.
Then by Exercise 26 in Section 7.2, A= BB for some positive definite matrix B.
Since the eigenvalues of B are positive,
O is
not an eigenvalue and so B is invertible.
In particular,
the columns
of B are linearly independent.
By Theorem 12 in Section 6.4,
B = QR
for some nxn matrix Q with orthonormal
columns and some
angular matrix R with positive elements on its diagonal.
square,
QQ= I.
So
upper
Since
triQ is
A = BB = (QR) (QR) = RQ'OR=RR
and
8.
R has
the
required
properties.
Suppose that Ais positive definite, and let A have a Cholesky factorization,
A = RR,
with R upper triangular and positive entries on its
diagonal.
Let D be the diagonal matrix whose diagonal entries are the
entries on the diagonal of R.
Since right-multiplication by a diagonal
matnix scales the columns of the matrix on its left,
the matrix L =
RD
7S, lower triangular with 1’s on its diagonal.
If U = DR, then
DR = LU.
A= RD
Chapter
11.
Start
with
an
SVD
decomposition,
A =
USV'.
Since
7 Glossary
7-21
orthogonal
U'U =
U is
I, and so A = USU'UV' = PQ, where P = UsU' = UsU and Q = UW".
The
matrix P is symmetric,
because
£ is symmetric,
and P has nonnegative
eigenvalues because it is similar to © (which is diagonal with nonnegative
entries).
Thus P is positive
semidefinite.
The matrix
Q is
orthogonal
because
it
is the
product
of orthogonal
matrices.
CHAPTER 7 GLOSSARY CHECKLIST
Check your knowledge by attempting to write definitions of the terms below.
Then compare your work with the definitions given in the text’s Glossary.
Ask your instructor which definitions,
if any, might appear on a test.
condition
number
covariance
(of
(of A):
variables
iance
The quotient
where
x;
and
x;,
for
i # j):
S
for
a
matrix
of
matrix
vary over
vectors.
o,/o,,
the
...
coordinates,
The
respectively
matrix (or sample covariance matrix):
by S =...,
where B is a pxN matrix
indefinite
matrix:
indefinite
quadratic
left
form:
matrix
of
A such
that
A quadratic
form
Q such
columns
of
observations:
A pxWN matrix whose
listing p measurements made on ..
mean-deviation
form
Lors
Moore-Penrose
(of a matrix
are.
inverse:
See
the
covar-
x;
and
the
Xj
observation
that
Q(x)
in
the
...
columns
of observations):
are
singular
...,
A matrix
each
whose
value
column
...
vec-
pseudoinverse.
matrix
A such
that
A quadratic
form
Q such
A symmetric
definite
matrix:
negative
definite
quadratic
negative
semidefinite
matrix:
negative
semidefinite
quadratic
diagonalizable:
POP
of
in
where
.
negative
orthogonally
...
The pxp matrix S defined
of observations
.
matrix
singular vectors
(of A): The
decomposition A=....
entry
observations,
covariance
A symmetric
..
form:
A symmetric
form:
A matrix
withers 2, ahd)
Di.
matrix
A quadratic
A
that
that
A such
that
form
Q such
admits
a
Q(x)
that
factorization,
A
=
CHAPTER
T=22
definite
matrix:
positive
definite
quadratic
positive
semidefinite
matrix:
positive
semidefinite
quadratic
a
quadratic
axes
principal
(of
orthogonal
principal
quadratic
row
that
A quadratic
form
Q such
A such
that
form:
A quadratic
form
Q such
form
x’ Ax):
P such
that
matrix
of A.
...,
....
that
of
columns
an
....
matrix
(or orthogonal
projection
such that ....
A simple example
when
matrix):
is B=.
UDV"
is
A
B): The
B, with
symmetric
a reduced
the
matrix
singular
B
value
form: A function Q defined for x in R" by Q(x) = ...,
where A is
annxn...
matrix A (called the matrix of the quadratic form).
singular value decomposition:
A factorization A=
...,
for an mxn
matrix A of rank r, where U is _x__ with orthonormal
columns,
Dis _x
with ..., and V is . x —. with orthonormal columns.
singular vectors
(of A):
The
decomposition A=....
sum:
sample
mean:
The
sum
of
the
The
average
entries
columns
of
M of a set
of
vectors,
singular
values
(of A):
The
...
spectral
decomposition
(of
A):
symmetric
matrix:
variance:
The
...
of
of the
the
eigenvalues
covariance
matrix
variables:
Any two variables x;
over the ith and jth coordinates
an observation matrix, such that
variance
a variable
matrix
of
coordinates
x,;:
The
singular
value
given
by M=....
of
...,
A =...
,
arranged
...
where....
that
uncorrelated
of
the
A): A=
..., where U is an
Amabeixgmondere. 1s) ane see
A representation
A such
in
X;,...,Xy,
value decomposition (of an mxn matrix
Xe
MACPLR
Vis anv Coo
Se
matrix with .
A matrix
...
..
singular
total
....
orthonormal
The
Q(x)
that
matrix
A symmetric
matrix
pseudoinverse (of A): The
decomposition
right
A such
components
(of the data in a matrix of observations
eigenvectors of a sample covariance matrix S for
eigenvectors arranged so that the corresponding ..
projection
reduced
form:
....
matrix
A symmetric
positive
Forms
Quadratic
and
Matrices
Symmetric
+
7
diagonal
observations,
of the ....
entry
where
S of
a matrix
of
....
and x; (with i * j) that range
of the observation vectors in
...
x;
in the
varYes*
...
matrix
“over™
S fora
the’
th
APPENDICES
~~
i-22
arin.
°
"
mete
le
rostiive
definite
matriz:
a2
ooditive
definite
,cnadratse. toe r
positive emaidestiaLe amen ba, Aevnbentd
powtlive
semidefinite
pelocipal
aveq
Lg
(gf's
ha
quadratte..
oMthogomni
” rndi pal
q
quadratic" fortw:!va
rosgonenta
fof
elgenvectaty
the
data
of
a canple
ae
“maven
elwwovectors arrandid so ‘thak
Projection
metrix
(or orthogonal prajedtic
siveh tnt
yewudol svoree
ie
ee
Gtetsetis
rediced
farm
eA
sw rr on
bon. of
A simiiacehin
“i
0 1, ven
A:
k
|. cat ned for. * eo
0 > > wiithe A (oalied the rahe af the
slygular
«alu decomposition & sectoral
ratrin 2 of rerd
© here) U ie
ae
»a
Dis
elgn®
es
ie
deoowncartt
ie.
. Astepte sme”
ulegvier
-@
with
weetorn
,
(cf
fpr
pe)
I ud)
Tw
‘ie
of
@44
ber
ay
Alr
A
-The
aA.
A
yen
te =e Cath
oolunne
af 7
_
|
“ae
eas
Sn)
@
aig
| aj
yg
emma)
1s
we»
iat
int
(te
Ue
kT
#
ex
riag
Gx
a
| cton
anteie
©
ast
hor
%«
Af
wri
>
is
is eh
Pik) ie
one
Siena
ry
4):
wa!) Hips
wy) | eaten ‘
—-
aing
ar
*
+ — at ae
e
ig
welucw
io
fi
A) + The
Pos)
(On
aS
Loy
ips
ny
/
‘1 }
tr
A
rT
v4
af
Pp.
ive
eats
bh
tbe
opted
ore
oh
‘
.
dae
the
. cs
ti
Yet
=
igonValosa. oP
esentation
(
A
1
; ha, 7
w
:
;
.
Ks ;
coveartenie
gdtrix S oF a eat fs
het i atv ‘ “u
x
ooordipales
ated rix
S
-
iTUrr)
aa
pes.
et
earn
ot
ul
kiwi
‘
th
be Ji
4
“Al
eotry
.
\
“wre
my
hehe
Cares
Mat.
:
;
¥
2
~
i
vs
~*
Bi aa
GETTING STARTED WITH MATLAB
MATLAB stands for MATrix LABoratory. Originally written by Cleve Moler for college linear
algebra courses, MATLAB has evolved into the premier software for linear algebra computations in
science and industry, all over the world. The purpose of using MATLAB in this linear algebra course
is to save you time on homework, help you learn linear algebra, and give you an impression of how
linear algebra is actually used in practical applications.
At first, you will learn simple commands that merely eliminate the drudgery of hand calculations.
Then, you will move to commands that semi-automate more complex tasks but still require you to
understand the mathematics behind the computations. Finally, you will learn to use some of the stateof-the-art MATLAB programs available. For instance, given a 32x32 matrix A (which is small by
today's standards), MATLAB’s eigenvalue program provides important and extremely accurate
information about A in a fraction of a second, after performing over a million arithmetic calculations.
WHAT YOU NEED
You need access to the MATLAB program, either on a personal computer (PC or Mac) or on a shared
computer network system at your school. A Student Version of MATLAB is available at modest cost.
You also need special MATLAB programs written for this course, along with data for over 800
exercises in the text (so you will not have to type in the exercise data). These are on the CD that
accompanies the Updated Second Edition of Linear Applications and Its Applications. They are also
on the Web, at http://www.laylinalgebra.com.
A ReadMe file with the data explains how to copy the data into the MATLAB folder on your
computer and how to adjust the "MATLABPATH" that tells the program where to look for the data.
Some schools will already have the special programs and data on their computer network system.
STARTING,
STOPPING,
AND RUNNING MATLAB
Once you initiate MATLAB (by clicking on a menu or an icon), you will see the MATLAB logo as the
program loads, followed by a prompt symbol », or EDU». You type a MATLAB command, press
<Enter>, and wait until you see MATLAB's response. Some simple commands are:
help
1
X=2
A=[123; 4-56]
Provides a long list of topics for which help is available.
Sets up a variable x in your "workspace" and stores 1 in it.
Sets up another variable. MATLAB is case-sensitive.
Extra spaces are ignored. They were inserted for readability.
Creates a matrix with 1 2 3 in the 1st row, 4 -5 6 in the 2nd row.
The spaces are important here, to separate entries in each row.
what
Lists the names of all the variables in your workspace.
Lists the current directory and the files in it.
help what
Tells you about the what command.
who
ML-1
ML-2
clear
Removes all variables (and their contents) from the workspace.
quit
Immediately clears the workspace and stops MATLAB.
Typing "exit" works, too.
Other commands could be listed, of course, but they will be discussed as you need them. For example,
at this point you don't need to know how to enter a matrix, because at first you will be working with
matrices that are already loaded in the laydata Toolbox for you. The MATLAB note for Section 1.1
tells how to load exercise data into your MATLAB workspace.
KEEPING A RECORD OF YOUR WORK
If your instructor wants you to submit a typed report of your MATLAB work, you can use the diary
command. Entering the command diary project1 (for example) tums on a "recorder" that captures
everything you type and MATLAB puts on the screen (except for graphics). When you enter diary
Off or end your MATLAB session, the contents of the diary are saved to a text file named project1.
Entering diary on starts the recorder again, whose contents will be appended to project1. You can
edit and print this file with any text editor. For more information, type help diary.
If you are working on a computer network, you may need to get special instructions from your
instructor about how to print text files.
INDEX OF MATLAB COMMANDS
\ (backslash), 2-26, 4-16, 6-14
: (colon), 2-4
*“ (power), 2-4
; (semicolon), 2-4, 2-30
abs, 5-32
accurate calculations with
displayed numbers, 1-17, 1-37
“ans”, 1-7
bgauss, 1-17
column of A, 2-4
constructing a matrix, 1-13, 2-4,
augmented, 1-13, 1-23, 2-9
vector, 1-17
cos, 4-12
det, (determinant) 3-8
diag, 2-30, 3-8, 7-18
eig, 5-15, 5-21
entry (in a matrix), 1-7, 2-4
exercise data, 1-6, 1-13
exp, 6-17
eye, 2-9, 5-5
for..end (loop), 5-25
format, 1-7
compact, 1-7, 5-32
long, 1-37, 5-32
rat, 5-5
short, 1-37
functions of vectors, 6-17
gauss, 1-17, 2-26
generating a sequence, 1-37, 5-25
grid, 5-25
gs, 6-1]
hold, 5-8
horizontal display, 2-30, 5-32
identity matrix, 2-9
imag, 5-21
inner product, 2-4, 6-2
interchange columns, 7-15
inv, 2-14, 2-26
inverting a matrix, 2-9
linspace, 5-8
log, 6-17
loop, 5-25
lu (factorization), 2-26
matrix notation and operations, 2-4
matrix-vector multiplication, 1-17
max, 5-32
mean, 7-18
norm, 6-2
nulbasis, 5-5, 5-15
numbers in MATLAB, 1-37
ones, 7-18
orthogonal projections, 6-4, 6-8
outer product, 2-4
partitioned matrices, 2-19, 6-14
plot, 5-25
poly, 5-8, 5-9
polyval, 5-8
power of A, 2-4
prod, 3-18
proj, 6-11
qr, 6-11
randint,
rank, 2-14, 2-39, 4-24
real, 5-21
ref, 2-39, 4-12, 4-24
replace, 1-6
roots, 4-32
row of A, 2-4
row reduction, 1-17
rref, 2-39, 4-12
scale, 1-6
scientific notation, 1-37
singular value decomposition, 7-15
size, 2-4
sqrt, 7-15
svd, 7-15
swap, 1-6
transpose, 2-4
tril, 2-30
zeros, 1-23, 7-15
ML-3
—
—
-
a
=
=
.
ai
Der
in,
we
*
“
é
|
tb
ae
ee
sate
i
ft.mT
A
cctthenie *S
; igs
tA IRR MIA there woth tiedowel
iver ¢ it[14 PTE
yoe dom? heed to know" Roky Meetora mat, hoomtiga’
vance: thet ure wiredy loaded atthal WARE ‘Toothan for you “hE “LAT
tells baw toload mtorr dats ike yO? OAR workeete
ef-2
/
oc
;
22 eheedlan
se ={.Bh.Te mapadigi
if your
“A-wie
-
_
7
insuct Mae, fee: *yHnt99 (wT leoyoriiedl report. of pair:
ccrumenl,
;
of nau: i
et
cos SRNR <a sare
amoiimeqe bie ease
it aie bi,oBd
os
Externy thoy condeiysubéaqngtngiast?
(f
apte)
<3
<i
Yur mi:
evew hourx ae Wc each eet howd inp sores. (encopL for praphins).
off or eal your
MATLAD ceanion, Setqsataiis af the. ‘ary ane seed to at
Fter toy Pag On stariy Ge MRE
Oogeg4howe contents ‘will be ay
whit ad prac this fils were ety SerBeblervieg more & Jommation, type daly
if yc are working
Om « ‘hilt Ae seme’. yow inny need Lp pet epee
mstractor about Lew ta erent
tnt 1UR-E borg
7
iid org
1}-0 4p
Bt
Ambsins
OF.6 Bi-t dine
(2%
:
dass
—s
Geb Sif DE-S Jay
ol soalgey
Seb eter
~¢
to wor
maaubet wor
i+!
-
Stab OFAC Fore
d-I ,ainse™
7 £-[ .norsdon sRitaoins
Qt-\ naoitieogracet attlev sedinguie
‘
r-\
tye
sala
C2-0 Supe
l-\ Tha
Ol
qaw
“f scott
O£.f°
4
,
re
|
Dey
o fro
<4
som,
=
.
Notes for the Maple
Computer Algebra System
GETTING STARTED WITH MAPLE
Using Maple with Linear Algebra and Its Applications
The Maple commands for manipulating vectors and matrices and most linear algebra operations can
be found in the linalg package. The laylinalg package contains data for many problems in the
text and Maple implementations of the special MATLAB commands presented in the text. These
files can be downloaded via the world-wide web at the address:
http: //www.laylinalgebra.com/
A package is loaded into a Maple session with the with command.
> with(
laylinalg
For example, the command
);
loads the laylinalg package into the current Maple session. If you experience any difficulties loading
the laylinalg package, check with your instructor for special instructions for your installation of
Maple.
Note that Maple is case-sensitive but ignores all whitespace (blanks) around numbers, names,
keywords, and operators in commands. For example, the above command could have been entered as
with(laylinalg) ; but not as with( lay lin alg ); or with( LayLinAlg );.
Entering Matrices in Maple
There are many ways to create matrices in Maple. In general, a matrix is created with the matrix
command. The first two arguments of this command are the number of rows and columns in the
matrix, respectively. If there are only two arguments, then the elements of the matrix are unassigned
(see matrix A, below). If the third argument is a list or vector, these quantities are used to fill the
matrix row-by-row from left-to-right (see matrices B and C). The third argument can also be a Maple
function of two variables. Each element of the matrix is assigned value by evaluating the function
with the corresponding row and column indices (see matrix £).
Sahtec= matrix
2,0
)5
AC = array elisced elms alui)
> evalm(
A );
bay Ai2
Agi A22
>B
——
matrix
(
DD.
af
(1.23745
56)
ma
A23
es
iDe P45
ee
|
4
O |
MP-1
MP-2
Notes for Maple
SC
2= matrix(
23°93,
102,354).
1
as
Ml matrix(.25
|
3
C23 |
oe:
1
3
L
Ble
Ore
The entermatrix command, from the linalg badltaee provides a somewhat more user-friendly
interface for entering the entries of a matrix:
>A := matrix(2,3): # define the matrix A
V 3)
.3,. (is,
2
|
4 Co2
jde=>51/ Catt)
> entermatrix(A);
enter element 1,1 > 1; # note that the semicolon is required
enter element 1,2 > 2;
enter element 1,3 > 3;
enter element 2,1 > 4;
enter element 2,2 > 5;
enter element 2,3 > 6;
The final method for defining a matrix is new to Maple V, Release 5. A template for a matrix
with up to four (4) rows and four (4) columns is available on the Matrix Palette.
To access this
palette, activate (left mouse click) on the View pull-down menu, click on Palettes, then select Matrix
Palette. When you click on the palette entry corresponding to the dimensions of the matrix to be
entered, the template for the corresponding matrix command is inserted in the current worksheet at
the current cursor location. Here is how the input region appears after typing F := and clicking on
the 2 x 3 matrix icon in the Matrix Palette:
SF
on MatrixG
iG [ton hkig bd mols
Potetrretd
onk vibe
Each occurrence of the string %? should be replaced with the value of the corresponding matrix
element. Use the <Tab> key to move to the next occurrence of the string %7.
Note:
Observe that the matrix commands produced via the palette do not explicitly specify the
number of rows or columns. While this syntax is permissible, it is not recommended in general usage.
Using Maple’s Online Help
These notes are intended to be self-contained. If, however, you are interested in additional information, you should consult Maple’s online help. The command help( laylinalg ); or, equivalently,
?laylinalg accesses the help worksheet for the laylinalg package. Help worksheets exist for all
Maple commands, including all commands in the linalg and laylinalg packages.
The Help menu on the user interface can also be used to search for help on a specific keyword.
A Full Text Search lists all help worksheets containing keywords that you specify. A Topic Search
interactively shows all help worksheets whose topic matches the search string. For example, there is
a long list of help topics that begin with the string 1a. When the search string is expanded to lay
the list of matching help topics reduces to all of the help worksheets for the laylinalg package.
1.1
STUDY
MAPLE
e
Systems of Linear Equations
MP-3
GUIDE NOTES
Initial Commands
At the beginning of each Maple session for this linear algebra course, enter the following commands:
>with( linalg ): with( laylinalg ):
>with( share ): with( pvac ): pvac:=true:
The commands on the second line tell Maple to print vectors as columns.
Without these commands,
vectors print as horizontally (as rows) even though Maple treats them as n x 1 column matrices.
Note:
e As illustrated above, one line can contain more than one Maple command.
key is pressed, all commands on the current line are executed.
e All Maple commands
When the <Return>
are terminated with either a semicolon (;) or colon (:). A semicolon
instructs Maple to display the result of the command; a colon suppresses the result.
1.1 Systems of Linear Equations
MAPLE
Row
Operations
The data for the exercises in this section are contained in the laylinalg package. The list of exercises
within chapter 1 section 1 for which Maple data is available is displayed with the command:
c1s1();.
The data for a specific problem, e.g., Exercise 13 in Section 1.1, can be loaded with the command:
cis1( 13 );. The output from this command displays each assignment that Maple has made. In
this section the data for each exercise are stored in a matrix M. To see the current contents of the
matrix M, use the command:
evalm(
M );.
Row operations on M can be performed using commands from either the laylinalg or the linalg
package. The following table summarizes the commands for the three elementary row operations.
Taylinalg Command Syntax with( linalg );
/replace( M, r, m, s ); _| addrow(
M,
s,
r,
load package
m);
| rowr<—rowr+™m™*
row s
As the previous summary illustrates, the syntax and functionality of the commands in the two packages are almost identical. The names in the laylinalg package more closely match those used elsewhere in the textbook; those used in linalg are used throughout Maple. The laylinalg commands
will be used in these notes.
Note:
e The name of any matrix in your current Maple session can be inserted in place of M in the
commands
above.
e The letters r and s are positive integers corresponding to row numbers. The names m and c
are scalar expressions (often numbers, but sometimes symbolic expressions) that you choose.
MP-4
Notes for Maple
If you enter one of these commands, say,
> swap (eMpais (32);
then the new matrix produced from M is not given a name.
history operator.
> Mi
You can refer to this result using 4, the
(Note that prior to Release 5, the history operator was ".) If, instead, you type
:= swap(
M,
1,3
);
then the answer is stored in the matrix M1. If the next operation is
SIM2:
s=areplace&
M1
9:2s) Sieckh)e
then the result of changing M1 is placed in M2, and so on.
symbol = is used to create an equation.
Note the use of := in assignments; the
One advantage of giving a new name to each new matrix is that you can easily go back a step if
you do not like what you just did to a matrix. If you type
>M
:= replace(
M,
2, 5,
1 );
then the result is placed back in M and the “old” M is lost. Of course, the “reverse” operation
SM
v= replace
Me
*2,-=5,
1 5
will recreate the original matrix M.
Note: For the problems in this section and the next, the multiple m needed in the replace or addrow
command will usually be a small integer or fraction that you can compute in your head. The next
two paragraphs describe how to use Maple in situations where m is not easily computed mentally.
The entry in row r and column c of a Maple matrix M is denoted by M[r,c]. If the number
stored in M[r,c] is displayed with a decimal point, then the displayed number may be accurate to
only about eight (8) digits. It is generally more accurate — and simpler — to use M[r,c] instead of
re-typing the displayed values in computations.
For instance, if you want to use the entry M[s,c] to change M[r,c] to 0, enter the commands:
Sn
>M1
Mir .cl._/_
:= replace(
Misch:
M, r, m,
# the multiple of row s to be added to row r
s ); # adds m™* rows to rowr
Or, you could use the single command:
> M1 := replace( M, r, -M(r,c]/M[s,c],
s );
The environment variable Digits controls the number of significant digits Maple uses in floatingpoint computations. The default is 10. While this should be more than sufficient for the problems in
this text, you can change this by assigning a new value to Digits. In general, you should expect a
floating-point computation to be accurate to approximately Digits-2 digits.
MAPLE
Symbolic Row
Operations
A powerful feature of Maple is its ability to work with matrices and vectors that involve unspecified
parameters. For example, Exercise 25 in Section 1.1 can be solved using Maple and the laylinalg
package in the following steps
1.3
PICIShCA25e
e
Vector Equations
MP-5
de
> evalm(
M );
# display the augmented matrix
> Mics
replace(M,.
oM2,3=
replace
> M2(3,4]
= 0;
3,0 251 .)sa% add-2exaowd-to rowsd
M1... 3): 1,.\2.)
s.4-add rows? tor0owed
# equation that makes system consistent
Symbolic multipliers can be used in replace and scale, for example:
replace(
A, 3, a,
1 );,
but all row numbers must be explicit integers.
1.3 Vector Equations
MAPLE
Constructing a Matrix
To see the problems with data for Section 1.3, enter the command:
SaCr esa):
The data for Exercise 25, for example, consists of the matrix A, the right-hand side vector b, and
the columns of the matrix, a, a2, a3. The corresponding Maple names, A, b, al, a2, and a3, can be
assigned using the command:
PICA
SIK 20);
The command
>M
:= augment(
al,
a2,
a3,
b );
creates a matrix with the given vectors as its columns. The same matrix could also be created by
>M
:= augment(
A, b );
Note that even though the columns of A (and b) appear as row vectors they are actually column
vectors (see the note in the Maple Note for Section 1.4).
Exercises 11-14 and 24-28 can be solved using the commands replace, swap, and (occasionally)
scale (or the corresponding linalg commands: addrow, swaprow, and mulrow), described previously.
1.4 The Matrix Equation Ax = b
MAPLE
gauss and bgauss
To solve Ax = b, row reduce the augmented matrix
>M
:= augment( A, b );
The command
Sx =
vector(
(5,63 -()
0%
creates a column vector, x, with entries 5, 3, and -7. Matrix-vector multiplication is
> evalm(
Ak*x
);
The laylinalg package contains two commands, gauss and bgauss, that can be used to speed up
row reduction of a matrix. Use the command with( laylinalg ); to load the laylinalg package.
To make row reduction of
M=[A_
bj] more efficient, the command
gauss(
M, r ); chooses the
leading entry in row r of M as a pivot and uses row replacements to create zeros in the pivot column
below this pivot entry. The result can be assigned to a name, otherwise the only way to access the
result is with the history operator (%). In either case, the result is displayed to the screen.
MP-6
Notes for Maple
For the backward phase of row reduction, use bgauss( M, r ); which selects the leading entry
in row r of M as the pivot, and creates zeros in the column above the pivot. Use scale to create
leading 1’s in the pivot positions.
Note: Exercise 19 of this section can be solved using symbolic row operations as discussed in the
Maple Note for Section 1.1.
1.5 Solution Sets of Linear Systems
MAPLE
Augmented
The command
vector(
m,
Matrix
0 ); creates the zero vector in R™ and matrix(
m,
n,
O ); creates an
m x n matrix of zeros. To solve an equation Ax = 0, use the command
>M
:= augment(
A, vector(
m, 0 ) );
to augment A with the column vector containing m zeros, then use gauss, swap, bgauss, and
scale to complete the row-reduction of M.
1.9 Linear Models in Business, Science, and Engineering
MAPLE
Generating a Sequence
The data for Exercises 9-13 in Section 1.9 consists of the migration matrix, M, and the initial
population vector x9. The following set of commands generates the first ten (10) terms in the
sequence defined by x44, = Mxx:
>x.0 := evalm(x0);
# display initial population
> for k from 0 to 9 do
# create loop to be executed 10 times
>
x.(k+1) := evalm( M &* x.k ); # compute x;4; from x,
|
>o0d:
Note that the first command reassigns x0 to itself. It can be omitted if you do not wish to include
the initial population in the output.
Numbers are entered in Maple without commas. Maple uses scientific notation for numbers larger
|
than 1,000,000 (10°) or smaller than 10~°. In Maple, numbers in scientific notation can be entered as
4.0 * 10°5 or 2.46 * 10°(-3). Note that the parentheses around a negative exponent are required.
When the mantissa is entered as a floating point number, the number is displayed (and computed) with
Digits digits. The default setting of Digits, 10, should be more than sufficient for all computations
in this text.
2.1 Matrix Operations
MAPLE
|
Matrix Notation and Operations
Several methods for defining a matrix in Maple have been described previously in this appendix.
Refer to these discussions for information about creating your own matrices.
Recall that, in Maple, the (7, j)-entry of A is ALi,j]. One or more columns or rows of A can be —
obtained with the col or row command.
For example, col(A,3)
is column 3 of A and row(A,2)
row 2 of A. To extract the first two columns of A, use col(A,1..2).
is
(The expression a. .b represents
the integers from a to b.) Be aware that both row and col return a sequence of one or more (column)
vectors.
2.2
e
The Inverse of a Matrix
MP-7
Multiple vectors can be assembled as the columns of a new matrix with augment, as described in
Section 1.4. For example, augment ( col(A,1..2) );. To create a matrix from the rows of an existing
matrix, either take the transpose of the result from augment or use the stackmatrix command.
The command
submatrix(
A, a..b,
c..d
); returns a matrix whose entries come from rows a
through b and columns c through d of A. The number of rows in A is given by rowdim(A) and the
number of columns by coldim(A).
Maple uses + and - to denote matrix addition and subtraction, respectively. Multiplication of a
matrix or vector by a scalar can be obtained with *, but &* must be used for matrix multiplication,
including matrix-vector products. If A is square and k is an integer, A~k denotes the xth power of
A. The evalm command is needed to force Maple to display the result of commands that produce
vectors or matrices.
The transpose of A is transpose(A). The outer product of u and v can be obtained with the
command evalm( u &* transpose(v) );. The command innerprod(u,v) ; gives the inner product
of u and v, and this scalar is displayed without using evalm. (If the entries of u and/or v are complex,
then you probably want to use dotprod instead of innerprod; see also the Maple box for Section 6.1.)
2.2 The Inverse of a Matrix
MAPLE
The Identity Matrix
The n x n identity matrix can be defined with diag( 1$n ); where n is an explicit positive integer. If
Ais5 x5, the command M := augment( A, diag( 1$5 ) ); creates the augmented matrix [A J].
Use gauss, swap, bgauss, and scale to reduce the augmented matrix to [I AT if possible. See
the Maple Note for Section 1.4.
There are other Maple commands that row reduce matrices, invert matrices, and solve systems
Ax = b. These are not discussed here because they do not help you learn the concepts in this section.
2.3 Characteristics of Invertible Matrices
MAPLE
inverse
and rank
It is not always a simple matter to determine whether a matrix is invertible, particularly if the matrix
involves floating-point numbers.
Usually, the command
inverse(
A ); will produce the inverse when
A is invertible. In this case evalm( inverse(A)&*b ); gives the solution of Ax = b.
When A is not invertible, Maple responds with an error message saying that the matrix is singular.
A method to test for invertibility without computing the inverse is presented in Section 4.6.
2.4 Partitioned Matrices
MAPLE
Partitioned
Matrices
Partitioned matrices are created in Maple with the blockmatrix command.
E, F, and G are matrices of appropriate sizes, then the command
> M := blockmatrix(
2, 3,
creates a larger matrix of the form
(A,B,C,
E,F,G]
);
For example, if A, B, C,
MP-8
Notes for Maple
Once M is formed, there is no record of the partition that was used to create M. For instance, although
B is the (1,2)-block used to create M, the value of M[1,2] is the same as the (1,2)-entry of A. You
should be aware, however, that if A[1,2] is an unevaluated name at the time the blockmatrix
command is executed and is subsequently assigned a value, this assignment to A(1,2] changes all
instances of A[1,2] in the current Maple session, viz., the (1,2)-entry of M.
Other commands that can be useful in the creation of new matrices from existing matrices include:
stackmatrix,
augment, concat, delrows, delcols, diag, extend, minor, and submatrix.
2.5 Matrix Factorizations
MAPLE
LU
Factorization
Row reduction of A using the command gauss will produce the intermediate matrices needed for an
LU factorization of A. You can try this on the matrix in Example 2, stored as Exercise 33 in the
laylinalg package. The matrices in (5) for Example 2 are produced by the commands:
> U = gauss(A,1); # U has 0’s below the first pivot
> U = guass(U,2);
> U = gauss(U,3);
# now Uhas 0’s below pivots 1 and 2
# the echelon form
You can copy the information from the screen onto your paper, and divide by the pivot entries to
produce L as in the text.
The Maple command U := LUdecomp( A, L=’L’,
tion for any square matrix A, i.e., A = PLU.
P=’P’
); produces a permuted LU factoriza-
2.6 Iterative Solutions of Linear Systems
MAPLE
Jacobi and Gauss-Seidel
Assume A is an n xX n matrix.
Methods
The Jacobi and Gauss-Seidel methods use the same set of Maple
commands except for the construction of M. For Jacobi’s method, the command
> Mo ss.diagGAlioalys
aiians
ys
creates the diagonal matrix from the diagonal entries of A. For the Gauss-Seidel method,
>
MerscmatrixOnyenpn(
if
prercit! as=9 then tit velsetOram:
extracts the lower triangular part of A. Once M is defined, set
>N := evalm( M-A );
To construct the sequence defined by Mx(*+)) = Nx(*) +b, use the following template of Maple
commands:
> x.0 := vector(n,0);
# change initial vector as needed
>x.1 := evalm( inverse(M) &* (N&*x.0+b) );
>x.2 := evalm( inverse(M) &* (N&*x.1+b) );
>x.3 := evalm( inverse(M) &* (N&*x.2+b) );
To have the iterations continue until the maximum difference between successive iterates does not
exceed a specified tolerance, say, 0.001, use the following template:
2.9
:= false:
> x.0 := vector(n,0);
pfor tk from/0sdo
>
x.(k+1) := convert(evalf(evalm(
e
Subspaces of R”
MP-9
> pvac
..,.if norm(x. (k+1)—-x.k)
> od; > pvac := true:
< 0.001
inverse(M) &*
then break £1;
(N&*x.k
+ b)
)),
vector);
Note:
e The first command, pvac := false:, instructs Maple to display vectors as rows (horizontally).
In this case this is preferable so more vectors can be viewed at one time (on the screen or on
paper). The final command, pvac := true:, causes vectors to be displayed as columns.
e The convert command forces the iterates to be vectors — and hence displayed horizontally.
(Without this command, the iterates are n x 1 matrices, which Maple displays vertically.)
e The name D is a reserved word in Maple — it is the “differentiation operator”. If you really
want to use the name D for a matrix (or any of object), you must first issue the command
unprotect(D);. But, be warned: if you do this, you have redefined one of Maple’s differentiation
commands.
2.9 Subspaces of R”
MAPLE
rref and rank
The Maple command
rref (A); produces the reduced row echelon form of the matrix A. From this
matrix you can write a basis for Col A or write the homogeneous equations that describe Nul A. (Do
not forget that rref (A); is not an augmented matrix.) For the rank of A, use rank(A) ;.
3.2 Properties of Determinants
MAPLE
Computing Determinants
To compute det A, define U:=A; and repeatedly use U:=gauss(U,r); and U:=swap(U,r,s) ; as needed
to reduce A to an echelon form U (see Section 1.4). Then the determinant of A is given by
SadetAtae.
(—Lckak
intl
U bist |), etme
where A is an n X n matrix and k is the number of row swaps made while reducing A to U.
The command det( A ); can be used to check your work, but the longer sequence of commands
is preferred — at this time — because it emphasizes the process of computing determinants.
4.3 Linearly Independent Sets; Bases
MAPLE
rref and genmatrix
The Maple command
rref (A) produces the reduced row echelon form of the matrix A.
From this
matrix you can write a basis for Col A or write the homogeneous equations that describe Nul A. For
the rank of A, use rank(A) ;.
For Exercises such as 33 and 34 the genmatrix command can be used to generate the coefficient
matrix A from a list or set of linear equations involving a set of unknowns, say c[1], c[2],.... For
MP-10
Notes for Maple
example, the coefficient matrix for Exercise 34 can be obtained by the following commands:
> EQN
:= add( c[j]
>eq[0]
:= eval(
* cos(t)*j,
EQN,
j=0..6
> eq([6] := eval( EQN, t=Pi );
>A := genmatrix( [ eq[k] $ k=0..6
>
C c{k)
# create first equation in system
);
t=0
) = 0; # define general form of equation
],
# create last equation in system
# generate coefficient matrix
$ k=0..6J;
4.4 Coordinate Systems
MAPLE
linsolve
The Maple command linsolve( A, b ); produces the solution of Ax = b when A is invertible.
The linsolve command has additional capabilities that will be introduced later, after you have the
appropriate background.
;
4.6 Rank
MAPLE
rank and randint
The rank of a matrix A can be found with the Maple command rank(
randint(
m,
n,
A );. The laylinalg command
r ); creates an m X n matrix of random integers with rank r.
4.7 Change of Basis
MAPLE
rref and blockmatrix
The Maple command rref can be used to row reduce a matrix such as [c}
C2
by
bgj. The
command blockmatrix (see Section 2.4) can be used to create a matrix from a collection of column
vectors.
4.8 Applications to Difference Equations
MAPLE
solve and polyroots
In Exercises 7-16 and 25-28, the coefficients of the polynomial in the auxiliary equation are stored
in a vector p. The coefficients are stored in descending order, e.g., the polynomial x? + 6z + 9 would
be represented with p := vector(3,[1,6,9]);.
Given the coefficient vector p the corresponding
polynomial can be created as follows:
>n := vectdim(p);
# degree of the polynomial
> P := add( p[k]*x*(n-k), k=1..n ); # polynomial with coefficients from p
The roots of the polynomial can now be found using solve( P=0, x ); Or, if you are only interested
in floating-point solutions, the fsolve command can be used. Note, however, that finding all floatingpoint solutions with fsolve can be somewhat tricky. (By default, fsolve attempts to locate all
real-valued roots of a polynomial.)
The laylinalg package contains a command that streamlines this process. Floating-point approximations to all real and complex roots of the characteristic polynomial whose coefficients are
stored in the vector p can be found with the command polyroots(
MAPLE
p );.
4.9 Applications to Markov Chains
See Section 1.9 for information about how to use Maple for these exercises.
5.1
e
Eigenvectors and Eigenvalues
MP-11
5.1 Eigenvectors and Eigenvalues
MAPLE
Finding Eigenvectors
The laylinalg package contains a command that will simplify your homework by automatically
producing a basis for an eigenspace when you know the eigenvalue. For example, if A is a 3 x 3
matrix with eigenvalue \ = 7, then the command nulbasis( A-7*diag(1$3) ); returns a matrix
whose columns form a basis for the corresponding eigenspace. In general, nulbasis( C ); is a matrix
whose columns form a basis for Nul C (the same basis you would get if you started with rref (C) ;
and computed the basis by hand).
Maple’s nullspace command is similar to nulbasis. While the bases returned by the two commands are equivalent, the basis returned by nullspace may not be the same as you would obtain by
hand from the reduced row echelon form of the matrix.
Maple contains additional commands for computing eigenvalues and eigenvectors that will be
discussed later. You should use nulbasis now, to reinforce the basic concepts of this chapter.
5.2 The Characteristic Equation
MAPLE
charpoly
The Maple command charpoly can be used to check your answers in Exercises 9-14. Note that if A
is n X n, charpoly (A, lambda)
is det(AJ— A). If n is even, this is the characteristic polynomial of A.
When n is odd, this is the negative of the characteristic polynomial of A (as defined in the text).
0.3 Diagonalization
MAPLE
eigenvalues and eigenvectors
The eigenvalues command returns the eigenvalues of a square matrix. The results from eigenvalues
can be combined with the method introduced in Section 5.1 to determine a basis for an eigenspace.
For example, if A is 5 x 5,
> lambda
> Bi
:= eigenvalues(
:= nulbasis(
A );
A - lambda[1]*diag(1$5)
# determine eigenvalues
); # basis for eigenspace for A
Once you become familiar with diagonalization, you can use other Maple commands to simplify
the process. For example, the command eigenvectors(A); returns, for each eigenvalue of A, a
list containing three pieces of information: (i) the eigenvalue, (i) the (algebraic) multiplicity of the
eigenvalue, and (iii) a basis for the corresponding eigenspace. The data structure is a little complicated, but the following sequence of commands illustrates how to find the matrices P and D such that
AP/= PD:
> EV := [ eigenvectors( A ) ];
# find e-values and corresponding e-spaces
> lambda := seq( op(1,e)$op(2,e), e=EV ); # extract e-values (w/repetitions)
> espace := seq( op(op(3,e)), e=EV );
# extract e-space basis
> DD := diag( lambda );
# construct diag e-value matrix
> P := augment( espace );
# construct e-vector matrix
Note that the matrix P may not be the same as the one you find by hand. A quick check that your
answer is correct is: iszero( A&*P - P&*D );. The result will be true if the argument is a zero
matrix. When the matrices involve floating-point entries, Maple may return false when one entry is
very small (but not exactly zero). In any case, a visual inspection should be performed.
MP-12
Notes for Maple
5.5 Complex Eigenvalues
MAPLE
Complex
Eigenvalues, Re, and Im
If A is a 2x2 real matrix and if eigenvalues( A ); returns a pair of complex conjugate eigenvalues,
you can use nulbasis to find a corresponding complex cipeyecton v. Then, build a 2 x 2 real matrix
P from the real and imaginary parts of v and compute C= P-lAP.
The real and imaginary parts of a complex-valued vector v are obtained with
> evalm( Re(v) ); # real part
> evalm( Im(v) ); # imaginary part
5.6 Discrete Dynamical Systems
MAPLE
Plotting Discrete Trajectories
The following sequence of commands
given initial vector x:
>x
:= vector(
[1,0]
> T:=sconvert(ax,
creates the sequence of vectors x, Ax, A2x, le
);
igst.
# initial vector (change vector as needed)
.):
SJsfot jutrom 1. to215.do
>
>
x := evalm( A&*x
TT :=T, convert(
A se forte
# initialize list of points
# begin loop (change 15 to number of iterations)
);
x, list
# compute the “next” point
); # add new point to list
> od;
These points can be plotted with the command plot( [T], style=POINT );. Omitting the second
argument of the plot command will “connect the dots” between the points. Other optional arguments
can be used to further customize the plot. The display command — in the plots package — can
be used to combine multiple trajectories into a single plot. (Use the command with(plots) ; to load
the plots package.)
5.7 Applications to Differential Equations
MAPLE
Plotting Trajectories for Differential Equations
The following template of Maple commands can be used to create a plot of the direction field and a
solution curve for the 2 x 2 system of differential equations in Example 1.
> with( DEtools );
# load DEtools package
SA Y=? patrizx©@ 2 6-2) oC (-Pe6 F707 Ss) Sipser yey
# enter coef matrix
> SYS
=
:= [ diff(y1(t),t) = A[1,1]*y1(t)
diff (y2(t) ,t) = A(2,1] *y1(t)
+ A[1,2]*y2(t),
#DO NOT CHANGE
+ A(2,2]*y2(t) 1; # DO NOT CHANGE
> Ic := [ yi(0)=5, y2(0)=4 J];
> RANGE := t = 0..10;
> WINDOW := yl = -5..5, y2 = -4..4;
> DEplot( SYS, [y1(t),y2(t)], RANGE,
[IC], WINDOW );
# enter init cond
# enter time interval
# enter plot window
# DO NOT CHANGE
To create plots for other systems, simply modify the coefficient matrix and the initial condition
then modify the time interval and window to create the desired plot.
Note that the with(
first use of DEplot.
DEtools
); command
needs to be entered only once — anytime before the
5.8
e
Iterative Estimates for Eigenvalues
MP-13
5.8 Iterative Estimates for Eigenvalues
MAPLE
Power
Method
and Inverse Power
Method
A Maple implementation of the Power Method (p. 359 of the text) assumes that A is an n x n matrix
with a strictly dominant eigenvalue and an initial vector xo:
>x := evalm( x0 );
# initial vector
2 torskhirom 1utopt's ido
# change 15 to number of iterations
> ©. mits=rnorm@x: )&
# largest element in x
>
x := evalm( A &* x/mu ); # next iterate in power method
Sod;
The value of 44 should approach the dominant eigenvalue and the vector x should approach a corresponding eigenvector.
One possible implementation of the Inverse Power Method is:
> alpha := 3.3;
# initial guess at eigenvalue
prion avecton¢s (1,132) 2s
# initial guess at eigenvector
> for k from 1 to 15 do
# change bounds as needed
>
y := linsolve(
>
>
>
mu := norm( y );
nu := alpha + 1/mu;
x := evalm( y/mu );
A-alpha*diag(1$3),
x );
# solve (A—al)y, = xx
# largest element in absolute value
# updated approx to eigenvalue
# updated approx to corr eigenvector
= od;
Note that the initial value of a should be reasonably close to the desired eigenvalue and x is a vector
whose largest element (in absolute value) is 1.
Note:
Recall that norm(x)
returns the maximum
of the absolute values of the entries in x.
6.1 Inner Product, Length, and Orthogonality
MAPLE
innerprod and norm
The inner product of two real-valued column vectors u and v is innerprod(u,v).
length of a vector v is norm(v,2).
The associated
6.2 Orthogonal Sets
MAPLE
Orthogonality
A fast way to test the orthogonality of a set such as {u1, U2, U3} is to use Theorem 6. Set
>U
:= augment(
ul,
u2,
u3 );
and then check if evalm( transpose(U) &* U); yields the identity matrix. The orthog command
could also be used, but this does not emphasize the essential properties of orthogonality.
The Maple command to obtain the orthogonal projection of y onto x is:
> evalm( innerprod(y,u)/innerprod(u,u) * u );
MP-14
Notes for Maple
6.3 Orthogonal Projections
MAPLE
Orthogonal Projections
The orthogonal projection of a vector onto a single vector was described in the Maple Note for
Section 6.2. The orthogonal projection onto the set spanned by an orthogonal set of vectors is the
sum of the one-dimensional projections. Another way to construct this projection is to normalize the
orthogonal vectors, place these orthonormal vectors in the columns of a matrix U, and apply Theorem
10. For instance, if {y,,y2,y3} is an orthogonal set of nonzero vectors, then the matrix
y2/norm(y2,2),
>U := augment( y1/norm(y1,2),
has orthonormal columns and
y3/norm(y3,2)
);
> evalm( U &* (transpose(U) &* y) );
produces the orthogonal projection of y onto Span{yi,
y2, y3}.Note: The parentheses around transpose(U)
matrix product.
&* y speeds up the computation by avoiding a matrix—
6.4 The Gram—Schmidt
MAPLE
The Gram-Schmidt
Process
Process, proj, and gs
If A has only two linearly independent columns, then the Gram-Schmidt process is:
> Vie om, colCA. 1);
> v2
:= evalm(
col(A,2)
-
(innerprod(col(A,2),v1)/innerprod(vi,v1i))
* v1
);
* v1
* v2
);
If A has three columns, add the command:
> v3
=
:= evalm(
col(A,3)
-
(innerprod(col(A,3),v1)/innerprod(vi,v1))
(innerprod(col(A,3) ,v2)/innerprod(v2,v2))
The generalization to larger matrices should now be apparent.
The Maple command to obtain the orthogonal projection of y onto u is:
> evalm(
innerprod(y,u)/innerprod(u,u)
* u );
A simpler, but less illustrative, alternative is to use the proj command from the laylinalg package.
The appropriate syntax is:
>
proj( y, convert(u,matrix)
);
After you are familiar with the Gram-Schmidt process, the command proj( x, V ); can be used
to compute the projection of a vector x onto the subspace spanned by the columns of a matrix V.
For example:
>v2 := i] evalm( col(A,2) - proj( col(A,2), augment(v1) ) );
>v3
:=
evalm(
col(A,3)
- proj(
col(A,3),
augment(vi,v2)
) );
Although you should construct the QR decomposition of a matrix A using the approach in the text,
you can check your construction of the matrix Q with the command Q := gs(A);. The gs command
from the laylinalg package provides an even faster method of applying the Gram-Schmidt process.
The syntax is simply: gs (A) ;. This command can be used to check your work or, after you are very
familiar with the Gram-Schmidt process, to save time.
6.5
e
Least-Squares Problems
MP-15
6.5 Least-Squares Problems
MAPLE
linsolve and leastsqrs
The least-squares solution to the problem Ax = b can be found using the command:
> linsolve(
transpose(A)&*A,
transpose(A)&*b
);
For Exercises 15 and 16, see the Numerical Note near the end of Section 6.5 of the text. A solution
of RX = Q'b can be found from rref( augment( R, transpose(Q)&*b ) ); . For Exercise 26,
the command A := stackmatrix( A1, A2 ); creates the (partitioned) matrix whose top block is
A, and bottom block is Az. This command works as long as each block has the same number of
columns.
(See also the Maple box for Section 2.4 for more information on partitioned matrices.)
To solve least-squares problems that arise in another course or at work, you might prefer to use
the command leastsqrs( A, b ); to obtain the least-squares solution to Ax = b. If more than one
least-squares solution exists, this command returns all solutions in a parametric form. To obtain the
solution whose norm is as small as possible, include ’ optimize’ as the third argument to leastsqrs:
> leastsqrs(
A, b,
’optimize’
);
6.6 Applications to Linear Models
MAPLE
Computing Regression Coefficients
Maple can be used to construct the design and observation vectors from a list of data points.
example, for Exercise 1:
>, c6s6(
1);
# load data from laylinalg
>n
Sx
:= coldim( DATA );
a= Vows DATA, et. )3
> y
:= row(
DATA,
# number of data points
# x-coordinates of data points
2 );
> X := augment( vector(n,1),
# observation vector
x ); # design matrix
The least squares solution to XG = y can be found by using the linsolve
normal equations.
MAPLE
For
command
to solve the
Functions of Vectors and map
If x is a vector and f is a Maple function, then map( f, x ); is a vector the same size as x whose
entries are the result of applying f to the entries in x. For example, in Exercise 7 the x vector can
be created as above and the vector of the square of each element is x2 := map( u->u*2, x );. The
design matrix is now easily constructed. When f is a built-in Maple function, say the exponential
function exp, the syntax is somewhat simpler: map( exp, x );.
6.8 Applications of Inner Product Spaces
MAPLE
Graphing Functions
The plot command can be used to plot one or more functions. For example, the two plots in Figure 3
for Example 4 can be created as follows:
MP-16
Notes for Maple
Since ts
S F3 := Pi - 2*sin(t)
t=0.>2*P1
= plot C [fihel,
> F4
- 2/3*sin(3*t);
- sin(2*t)
# Figure 3 (a)
7;
#- fourth-order Fourier approx of f
:= F3 - 2/4*sin(4*t) ;
t=0..2*Pi
[£,F4],
> plot(
# define function f
# third-order Fourier approx of f
# Figure 3 (b)
);
7.1 Diagonalization of Symmetric Matrices
MAPLE
Orthogonal Diagonalization
Use the eigenvalues and nulbasis commands to find the eigenvalues and corresponding eigenvectors
as in Section 5.3. If you encounter a two-dimensional eigenspace, with a basis {vi,v2}, use the
command
= IVZi
evalm(
v2 -
innerprod(v2,vi)/innerprod(vi,v1)
Saye s
evalm(
v2 - proj(v2,augment(v1))
* v1
);
or
to make the two eigenvectors orthogonal.
);
(Review Section 6.2 and the Maple Note for Section 6.4.)
After you normalize the vectors and use augment to create P, check whether P' P= I to verify that
P is indeed an orthogonal matrix.
7.4 The Singular Value Decomposition
MAPLE
The Singular Value Decomposition
The singular value decomposition of a matrix A is constructed from the orthogonal diagonalization
of A'A. See Section 7.1 for a discussion of orthogonal diagonalization. Note that the eigenvalues
of A'A may not be sorted in decreasing order. You may need to reorder some of the information
from the orthogonal diagonalization when creating the diagonal matrix D and matrix of right singular
vectors V. For example, if A' A is a 3 x 3 matrix and lambda is a list of the eigenvalues of A' A with,
say, lambda[3] > lambda[1] > lambda[2], then the command
> Sigma
:= diag(
seq(
sqrt(lambda[i]),
i=[3,1,2]
) );
creates the diagonal matrix of singular values of A. The matrix V would then be created by:
> V := augment(
seq( col(P,i),
i=(3,1,2]
) );
The matrix of left singular vectors, U, can be formed by normalizing the nonzero columns of AV.
The method of Example 4 can be used to fill the remaining columns of U if necessary.
This construction helps you to think about the basic properties of the singular value decomposition. There are two additional Maple commands that can be used to directly obtain information
about the singular values of a matrix. The singularvals command returns the singular values of
A as a list but does not provide any information about the singular vectors. The Svd command
computes floating-point approximations to the singular values and vectors of a numeric matrix A.
The command
> SV"
:= evalf(
Svd(
Ay UY
oe F
returns the singular values in the table SV. The matrices U and V
contain the right and left singular
vectors, respectively.
To create the diagonal matrix © from the table of singular values, use the
7.5
Applications to Image Processing and Statistics
MP-17
command:
> Sigma
:= diag(
seq( SV[i],
i=map(op, [indices(SV)])
) );
7.5 Applications to Image Processing and Statistics
MAPLE
Computing Principal Components
The p-dimensional vector containing the mean of each row of a p x N matrix X can be created with
the command:
> Rmean
:= vector([{
>
seq(
add(
x,
x=convert(row(X,i),list)
i=1..rowdim(X)
) / coldim(X),
) J);
The p x N matrix containing N copies of the row averages is obtained with:
> augment(
Rmean
$ coldim(X)
);
Thus, to convert the data in X into mean-deviation form, use
>B
:= evalm(
X - augment(
Rmean
$ coldim(X)
) );
The sample covariance matrix is produced by:
>S
:= evalm( B &* transpose(B)
/ (coldim(X)-1)
);
The principal component analysis is completed by performing a singular value decomposition on the
matrix A = iB:
(See the Numerical Note at the end of Section 7.5 of the text.)
INDEX OF MAPLE COMMANDS
general commands
* (multiplication, scalar), MP-7
+ (plus), MP-7
- (minus), MP-7
.. (range), MP-6
: (colon), MP-3
:= (assignment), MP-4
; (semicolon), MP-3
= (equality), MP-4
?, see help
%, (history operator), MP-4, MP-5
47, MP-2
&* (multiplication, matrix), MP-5, MP-7
~ (power), MP-7
add, MP-10, MP-17
break, MP-9
convert, MP-9, MP-14, MP-17
DEplot (DEtools package), MP-12
diff, MP-12
Digits, MP-4, MP-6
display (plots package), MP-12
evalf, MP-9, MP-16
evalm, MP-3, MP-5 to MP-7
for..do..od (loop), MP-6, MP-9, MP-12
fsolve, MP-10
help, MP-2
if..then..fi (conditional), MP-8, MP-9
Im (imaginary part), MP-12
indices, MP-16
map, MP-15, MP-16
mul, MP-9
op, MP-11, MP-16
plot, MP-12, MP-15, MP-16
Re (real part), MP-12
seq, MP-11, MP-16, MP-17
solve, MP-10
sqrt, MP-16
Svd, MP-16
with, MP-1
laylinalg package, MP-1 to MP-3
bgauss, MP-5 to MP-7
gauss, MP-5 to MP-9
gs, MP-14
MP-18
nulbasis, MP-11, MP-12, MP-16
polyroots, MP-10
proj, MP-14, MP-16
randint, MP-10
replace, MP-3, MP-4
scale, MP-3, MP-6, MP-7
swap, MP-3, MP-4, MP-6, MP-7, MP-9
linalg package, MP-1 to MP-3
addrow, MP-3, MP-4
augment, MP-5 to MP-7, MP-11,
to MP-17
:
blockmatrix, MP-7, MP-10
charpoly, MP-11
col, MP-6, MP-14, MP-16
coldim, MP-7, MP-15, MP-17
MP-13
det, MP-9
diag, MP-7, MP-8, MP-11, MP-16
eigenvalues, MP-11, MP-12, MP-16
eigenvectors, MP-11
entermatrix, MP-2
genmatrix, MP-9
innerprod, MP-7, MP-13, MP-14, MP-16
inverse, MP-7
iszero, MP-11
leastsqrs, MP-15
linsolve, MP-10, MP-13, MP-15
LUdecomp, MP-8
matrix, MP-1, MP-2, MP-6, MP-8
mulrow, MP-3
norm, MP-9, MP-13, MP-14
nullspace, MP-11
orthog, MP-13
rank, MP-9, MP-10
row, MP-6, MP-15
rowdim, MP-7, MP-17
rref, MP-9 to MP-11, MP-15
singularvals, MP-16
stackmatrix, MP-7, MP-15
submatrix, MP-7
swaprow, MP-3
transpose, MP-7, MP-13 to MP-15
vectdim, MP-10
vector, MP-5, MP-6, MP-13, MP-17
Notes for the Mathematica
Computer Algebra System
GETTING STARTED WITH MATHEMATICA
Using Mathematica with Linear Algebra and Its Applications
All of the Mathematica commands contained in these notes are either standard built-in functions or they are commands that originate from an add-on package. The standard add-on
packages, which were created by the makers of Mathematica, are divided into several folders
according to different mathematical topics. A folder which is frequently referred to in these
Mathematica notes is called LinearAlgebra. This folder contains five packages called Cholesky,
GaussianElimination, MatrixManipulation, Orthogonalization, and Tridiagonal. To load functions
contained in a particular package, the command Needs[ “folder* package’” ] must be executed.
For example, executing Needs[ “LinearAlgebra* MatrixManipulation’” | loads the commands in
the MatrixManipulation package found in the LinearAlgebra folder.
Several additional add-on packages, contained in a folder named LayData, were created
specifically for this course. The names of these packages are LayFunctions, Chapterl, Chapter2,
Chapter3, Chapter4, Chapter5, Chapter6, Chapter7, and Misc. LayFunctions contains several commands to be discussed in these notes while the remaining packages contain homework data corresponding to exercises in the textbook. By executing Needs[ “LayData* LayFunctions*” ], commands in the LayFunctions package are loaded into memory. Homework data can be obtained
by first loading the appropriate chapter package. For example, Needs[ “LayData* Chapterl*” ]
loads the data for selected exercises in chapter one. Once the appropriate chapter package has
been loaded, execute a command of the form C(chapter#)S(section#)[ exercise# ] to obtain
data to particular exercises . For example, the command C1S1[ 13 ] loads the data found in
Exercise 13 in Section 1.1.
Text-based and Notebook Interfaces
When
working with Mathematica,
you are either using a text-based interface or a notebook
interface. In a text-based interface, users interact with the Mathematica program by typing in
commands from the keyboard. In a notebook interface, not only can commands can be typed
in from the keyboard, but shortcuts to commands can be entered using palettes contained in
the Palettes submenu of the File menu.
In a text-based interface, a prompt of the form In[1]:= appears when the program is ready
for input. Once the input has been entered, pressing Shift+Enter results in the execution of
the input (on some computer platforms, pressing Return or Enter is sufficient). Mathematica
processes the input, generates a solution and displays the result on a line labeled Out[1]=. The
MM-1
MM-2
Notes for Mathematica
second input and output commands are labeled !In{2]:= and Out[2]=, and so forth. For example,
in the following, 10 — 2 is entered after the In[1]:= prompt and upon execution, the output of
8 is displayed after Out(1].
In{1]:=10 — 2
Out[1]=8
The input and output displays for a notebook interface are similar except that the In[n]:=
label does not appear until after a command has been executed and unlike a text-based interface,
commands on an In[n] line can be edited and reexecuted. Throughout these notes, dialogs with
Mathematica are shown with the In[{n]:= and Out[n]= labels.
Important Rules of Mathematica
A brief overview of some of the important features and rules of Mathematica are given in this
section.
1. The Mathematica arithmetic operators used for addition, subtraction, multiplication, division
and powers are +, —, *, / and~, respectively. Instead of an asterisk (*), it is more common to
use a space key for multiplication. For example,
In[1]:=3*4
can also be expressed as__In{1]:=3 4.
2. Mathematica performs exact calculations on numbers entered in rational form. A number is
in rational form if it is expressed as a fraction of two integers. (To save space in these notes,
the output Out[n] is frequently displayed on the same line as the input In[n], as is the case in
the following example.)
In[2]:=56/77 — 32/91
Out[2)=3%
Approximate answers can be obtained by inserting a decimal point (.) in at least one of the
numbers appearing in the calculation.
In[3]:=56./77 — 32/91
Out[3]=0.375624
The same task can be accomplished using the N command (N is short for numerical).
In[4]:=N[ 56/77 — 32/91 ]
Out[4]=0.375624
To obtain n digits of accuracy, use the command N[ 0, n J.
In[5):=N[ 56/77 — 32/91, 10 ]
Out[5]=0.3756243756
3. Mathematica is case-sensitive and all Mathematica functions begin with a capital letter.
An error and/or an undesired result is obtained from an input command where capital letters
are not appropriately used. For example, the command RowReduce[ A ] (to be defined later)
row reduces a given matrix A, but typing and executing rowreduce[ A ], rowReduce[ A ] or
Rowreduce[ A | leads to undesired output.
4. Parenthesis ( ...
) are used to group terms in an arithmetic or algebraic expression.
e
5. Curly brackets { ...
Creating Matrices with Mathematica
} are used to represent lists.
A list { a1,@2,...,@n
MM-3
} is just a finite
sequence of elements. As you will see shortly, vectors and matrices are created using lists.
6. Square brackets [ ... | are used exclusively for defining and executing functions.
The com-
mand RowReduce[ A ] is an example of a built-in Mathematica function which uses square
brackets [ ... ] for its argument A. As you read through these notes, note that spaces are occasionally placed before and/or after brackets. These spaces are added to enhance the readability
of this document and you are not required to include the spaces when typing in commands.
7. The semicolon ( ; ) is used to place several commands on a line, and the output of any
command immediately preceding a semicolon is not displayed.
8. An equals sign ( = ) is used to assign values to symbols. (Comments appearing in (* ...*)
are used to help you better understand the input and output statements.)
In{6]:=x = 2;3x
Out[6]=6
(* x is assigned the value of 3 and the product is computed *)
(* Output for 3 x in In[6] is displayed *)
Creating Matrices with Mathematica
Q@11
412
«.-'
Gin
;
Q21
a22
Dilys
Ain
The Mathematica representation of the matrix
:
‘
;
;
Qm1
Am2
---
Amn
'
;
;
.
,
is a list of n lists:
L{ 011,019,
.1, 41m },-{ Gor, 022,.--;
02m }, «675 (Gm, Oia
1ins2)
Suppose we wish to define A to be the matrix
|5
9
6
medizigd
7
8
(AOQoedley
12
One way to do this is
to enter the following assignment statement.
1h aed ci ad ths Sas amath ale br
GT bljan el sD es a ae ng eth a Go =
pla btbon! ea NE
ak A eo
The entry in the ith row and jth column of matrix A is accessed by entering A|[#, j]].
(* Display the entry in row 2 and column 3 of A *)
In[2]:=A[[ 2, 3 ]]
(* The corresponding entry in A is displayed *)
Outl2|=7
For those using a notebook interface, a more intuitive method for entering a matrix is used.
From the Palettes submenu of the File menu, open the palette called BasicInput. Type in
tex
i aE
A = and then click on the button in BasicInput containing
(i a
i Press Ctrl+Enter to
create another row in the matrix and press Ctrl and the comma key simultaneously two times
to create two more columns. Then click on the box in row 1 and column 1 and begin entering
MM-4
Notes for Mathematica
the numbers contained in the following matrix, using the Tab key to move from one entry to
the next.
Inftj=A =
peek
|5 6
G10,
eee
7 8
Adiga?
Out[1J={ { 1,2,3,4}, {5,6,7,8 }, { 9,10, 11,12 } }
Notice that the output is not in matrix form. The command MatrixForm| A ], or equivalently
A//MatrixForm is used to display a matrix in a more familiar form.
In[2]:= A//MatrixForm
Out[2]//MatrixForm=|
DGD
Bae
A
5
7
8
6
92. 10srhheyl2
To make Mathematica automatically display matrix output in matrix form, execute the
following command. (This command involves the advanced concept of pure functions and it is
not necessary to understand how this command works.)
In{3]:= $Post:=If[ Or[ MatrixQ[#], VectorQ[#] ], MatrixForm[#], # ]&
Mathematica commands are reserved words which cannot be used in assignment statements.
For example, executing the following input statement results in an error message stating that
N is a protected symbol.
Yar]
&lap 2
mas
hee
Set::wrsym: Symbol N is Protected.
In[4]:=N
Out[4]//MatrixForm =
(r ‘)
One way to avoid this problem is to use a lower case letter n instead of a capital N in the
assignment statement. Since all Mathematica commands begin with a capital letter, a lower
case letter never results in a protected symbol error. Another way around this problem is to
use an entire word (starting with a lower case letter) for the name of a matrix in an assignment
statement. For example, we let matN represent the matrix in the following assignment statement
(the output is not displayed here). Any such word created by the user may end with a digit
(O—9), but cannot begin with a digit and no spaces can occur in the word.
In[5]:=matN = (. :
It is not always necessary to assign a name to a matrix since Mathematica automatically
attaches the name Out[n] to the matrix entered immediately after the In[n]:= prompt. For
example, the name Out[6] is assigned to the matrix in the output of the following assignment
statement.
teh =(; ;
Out[6]//MatrixForm =
(herid
3.4
1.1
e
Systems of Linear Equations
MM-5
The matrix is retrieved simply by typing and executing %6, which is short for Out[6].
In[7]:=%6
Out[7]//MatrixForm =
(; j)
Online Help
The question mark (?) is used to obtain a precise definition of a given Mathematica command.
For example, by executing ?MatrixForm, a brief description of the MatrixForm command will be
displayed. For more information about Mathematica commands, consult the Help menu.
STUDY GUIDE NOTES
MATHEMATICA
Initial Commands
At the beginning of each Mathematica session for this linear algebra course, execute the following commands. (The extra spaces in the commands are optional.)
In{1]:=Needs[ “LinearAlgebra* MatrixManipulation~” |
in[2]:=Needs[ “LayData* LayFunctions~” ]
In[3]:=$Post:=If[ Or[ MatrixQ[#], VectorQ[#]], MatrixForm[#], # ]&
In addition, if you plan to work on exercises from the text, execute
a command of the form
In[4]:=Needs[ “LayData* ChapterN*” ]
in which N is replaced by the appropriate chapter number.
1.1 Systems of Linear Equations
MATHEMATICA
Accessing and Using Textbook Data
Begin with the initial commands (In[1] through In[4]) above, replacing N with 1 in input statement In[4]. The following command In[5] asks which exercises in Chapter 1, Section 1, have
data for use with Mathematica.
In[5]:=?C1S1
The response Out[5] supplies the answer.
Out[5]:= C1S1[ Exercise Number] for Exercises: 13—20, 36.
Omit In[5] if you already know what exercise you want.
for instance, enter
Ov
If you want the data for Exercise 13,
Clo 0(.13)]
Ot le oc.—4
Out[6]//MatrixForm = | 1 4 3 —2
Jee (s olla
The output suggests (but does not require) that the matrix could be called M and then the
matrix is displayed on Out[6]. Note that the matrix is the augmented matrix for the system of
equations listed in Exercise 13.
in[7)—=Me=_C1S1[ 135]
(* M is assigned to represent the matrix contained in C1S1{ 13 ] *)
MM-6
Notes for Mathematica
MATHEMATICA
Row
Operations
The LayFunctions package contains the following commands that perform elementary row operations on a matrix, denoted here by M:
ReplaceRow| M, r, m, s |
(* Replaces row r of M by row r plus m times row s. *)
Swap[ M, 1, s |
Scale[ M, r, c ]
(* Interchanges rows r and s of the matrix M. *)
(* Multiplies row r of M by a scalar c. *)
The letters r and s, are integers while m and c can be any scalar expressions. Any of these
variables can be a symbolic expression as well.
A
Suppose you want to apply row reduction to a given matrix M. Enter the matrix on the next
input line, say In[1], and execute the input statement. The matrix you entered now appears on
line Out[1] and is assigned the name %1. Now suppose you wish to switch rows 1 and 3. Then
execute the command Swap[%1, 1, 3] and note that matrix, whose rows have been switched,
appears in Out[2]. Then the next operation, say ReplaceRow[ %2, 2, 5, 1], uses %2 to refer to
the matrix from the previous output and so on.
Note: For simple problems, the multiple m you need in ReplaceRow[%n, r, m, s] is usually a
small integer or fraction that can be computed in your head. In general, m may not be so easy
to compute mentally. The next two paragraphs describe how to handle such cases.
For a given matrix appearing in Out[n], Yon|[s, c]] is the entry in row s and column c of this
matrix. Now suppose you want to use the entry in row s and column c of the matrix appearing
in Out[7] to change %7[[r, c]] to 0. Enter the commands
In[8]:=m = —%7[[ r, c ]] / %7[[s, ¢ ]];
(* The multiple of row s to be added to row r *)
In[9]:=ReplaceRow[ %7, r, m, s |
(* Add m times row s to row r *)
Or, just use one command: M=ReplaceRow|[ %7, r, —%7[[ r, ¢ ]]/%7[[ s, ¢ ]], s ].
MATHEMATICA
Symbolic Elementary
Row
Operations
Mathematica works with variables and unspecified parameters as in the following example.
tuo: =(4 ;
Ous[.0)//Matrixform=|
In{11]:=ReplaceRow[ %10, 2, —c/a, 1 ]
;
Out[a}//MatrixForm=| ‘ ay? q
1.3 Vector Equations
MATHEMATICA
Transpose
The transpose of a given matrix A is a matrix whose ith column is the ith row of A. The
heir B.ncd
command Transpose[A] creates this matrix. For example, if A =
708
, then executing
1.3
1
Transpose[A] results in an output of | 3
Vector Equations
MM-7
at
0
5
MATHEMATICA
e
8
Representation of a Vector
V1
;
aL,
A vector v which is either represented as ( v1, v2,
represented by a list { v1, v2,
..., Un ) or as
v2
;
in the textbook, is
Un
..., Un } in Mathematica.
We also refer to this vector as being
in row form. The command Transpose[ {v} ] turns v into an n x 1 (column) matrix.
MATHEMATICA
Forming a Matrix From Vectors
To determine how to form the matrix [ a;
ag
ag
b ] in Exercise 11 using the Chapterl
package, you must first execute the command C1S3[11] to see how the data is arranged. Observe
that C1S3[11] represents a matrix whose rows are the vectors aj, a2, a3, and b, respectively.
Therefore the Transpose command creates a matrix M representing |a; ag a3 b ].
In{1]:=M = Transpose[ C1S3[11] ];
In Exercise 13, you will use
a new command,
AppendRows.
For matrices A and B with
the same number of rows, AppendRows[A, B] attaches each row of B onto the end of the
corresponding row of A, when the matrices are considered as lists of rows. Consider this
example:
nz te bc
4d, ef tb =
gent, (egies
In[3]:=AppendRows| A, B ];
Eton.
(inate
Out[3] //MatrixForm= (
ag
Ph
rae
Notice how AppendRows| A, B | places the columns of B to the right of the columns of A when
these matrices are displayed in matrix form.
To answer the question in Exercise 13, you must form the matrix [ a, a, a3 b] and
row reduce it. Instead of breaking apart the matrix A into its three columns and then putting
them along with b into the columns of this matrix, you can accomplish the same thing by just
augmenting the matrix A with the vector b, which can be visualized as [A b]. When you
execute C1$3[13], you will note that a list is returned containing a matrix followed by a vector.
Use the following statement to assign the first part of the list to A and the second part to b.
[Aled A, bate=C1S3[ 13a.
The command Transpose[{b}] transforms b into a (column) matrix so it can be appended to A:
In[5]:=M = AppendRows[ A, Transpose[{b}] ];
MM-8 _ Notes for Mathematica
The command C1S3[25] represents a list containing A followed by vectors b, a1, ag, and ag.
Execute the assignment statement {A, b, al, a2, a3} = C1S3[25] and then use the AppendRows
command to form the appropriate augmented matrix.
1.4 The Matrix Equation Ax=b
MATHEMATICA
Gauss and BGauss
To solve Ax = b, row reduce the matrix M = [A b]. For example, the assignment statement.
b={ {0}, {-5}, {7} } creates a column vector b with entries 0, —5 and 7 and the command
AppendRows[A bj can be used to form M provided that A contains exactly three rows. The
commands Gauss, BGauss and Scale speed up the row reduction process. Gauss[A, r] uses the
leading entry in row r of A as a pivot, and uses row replacements to create zeros in the pivot
column below this pivot entry. In the backward phase of row reduction, use BGauss[A, r], which
selects the leading entry in row r of A as a pivot, and creates zeros in the pivot column above
the pivot. Use Scale to create leading 1’s in the pivot positions.
MATHEMATICA
Multiplying
a Matrix and a Vector
A period (.) is used to multiply a matrix and a vector.
Pye
fl
Acs=
pes
ae
she ah
ply
fs 5 8 7 jem
er)
14 68 O:A10) HA.
Out[1]=//MatrixForm= (Es
The somewhat strange calculations in In[{1] and Out[1] illustrate the unfortunate decision of
Mathematica to store vectors as rows but to treat them as columns when they are left-multiplied
by a matrix. For instance, if you wanted to append the result shown in Out[1] to a matrix C,
you would have to convert A.x to a n X 1 matrix.
1.5 Solutions Sets of Linear Systems
MATHEMATICA
ZeroMatrix
The command ZeroMatrix[m, n] creates a m x n zero matrix.
strates how to set up an augmented matrix
command.
L
lalilsA = (es
2
The following example demon-
M corresponding to Ax = 0 using the AppendRows
iM = AppendRows| A, ZeroMatrix[2, 1] ];
Then use Gauss, Swap, BGauss, and Scale to row reduce M completely.
1.9
e
Linear Models in Business, Science, and Engineering
MM-9
1.9 Linear Models in Business, Science, and Engineering
MATHEMATICA
Generating a Sequence
The data for Exercise 13, Section 1.9, is used here to demonstrate how to generate a sequence.
By executing C1S9[13], you will see that the output is a list containing a matrix followed by
two vectors and therefore In[1] is an appropriate assignment statement for the data:
In[{1]:={ M, x0, yO } = C1S9f 13 J;
(* Assign M to the matrix and x0 and y0 to the vectors*)
Suppose you want to create the first 20 terms of the sequence {x), x2, x3, ...} using the linear
difference equation x,4; = Mx, fork =0, 1, 2, ..., where x9 = x0. You can do this by first
creating a function x satisfying x[k] = x, fork =0, 1, 2, ... . When defining a new function
with Mathematica, it is good practice to first clear out any previously defined values of the
function using the Clear command as demonstrated in the first part of In[2], below. Next, the
assignment statement x[(0]:=x0 defines the value of x[0] to be x0 and then z[1], z[2], x[3], ... are
recursively defined by letting x[k_] := M.x([k — 1]. The underscore immediately following k on
the left hand side of the assignment statement is necessary in order to define k to be a variable
and the colon-equals sign(:=), which is called a delayed equals sign, must be used instead of an
equals sign ( = ), when defining a function recursively
Inf2t'—Clearp-x |5"x(0"] =O?
x ke] SPR;
Now that the function z is defined, the value of x, is retrieved by simply executing z[n].
In[3]:=x[ 100 ]
(* For example, determine the value of xj99 *)
375054.
624946.
Out[3]=//MatrixForm= (
(* The value of x09 is displayed *)
Notice that Mathematica displays 6 significant digits for each entry in Out[3]. To see n digits
of precision, replace x[100] with N[ x[{100], n ] in In{3}.
Executing Table[ x[k], {k, 1, n} ]generates and displays the values of [1], z[2],...,z[n] (the
output is not displayed here).
In[4]:=Table[ x[k], { k, 1,20 }]
| (* Generate
the values x1, x2, ..., X20 *)
2.1 Matrix Operations
MATHEMATICA
Matrix Operations
Matrix addition and subtraction are performed using the plus (+) and minus (—) signs, respectively. Matrix multiplication is performed using a period (.). To compute A* for ann x n
matrix A, as defined on page 106 in the textbook, execute MatrixPower[ A, k ]. Computing
A*k returns a matrix whose entries are the kth powers of the corresponding entries in A.
Multiplying a scalar c and matrix A is performed by entering c A, leaving a space between the
cand the A. To compute A’F, enter Transpose[ A ].F . Here are a few other examples (the
output is not displayed):
MM-10
Notes for Mathematica
Weed
5 0 -3
Jnl Wh
Bip area * hh Hi ale 7 sete ek
In[3]:=A—B
In(2]:=A+B
In[4]:=F.A
If u and v are vectors of the same size, then
In[5]:=5 A
u.v_
In[6]:=MatrixPower|F,25]
represents their inner product, and
Outer[Times, u, v] is their outer product.
The command Randon| Integer,{a,b} ]produces a random integer between a and 8; replacing
Integer by Real produces a real number between a and b. For positive integers m and n, the
command Table[ Random[Integer,{a,b}], {m}, {n} ] creates an m x n matrix whose entries are.
random integers between a and 0.
2.2 The Inverse of a Matrix
MATHEMATICA
IdentityMatrix
The n x n identity matrix is denoted by IdentityMatrix[n]. If A is 5 x 5, then the command
M=AppendRows| A, IdentityMatrix[ n ] ]creates the augmented matrix [A I]. Use Gauss, BGauss
and Scale (introduced in Section 1.4) to reduce [A J].
There are other Mathematica commands that row reduce matrices, invert matrices, and
solve equations Ax = b. These are not discussed here because they will not help you learn the
concepts of this section.
2.3 Characterizations of Invertible Matrices
MATHEMATICA
Inverse
Determining whether a matrix is invertible is not always a simple matter.
A fast and fairly
reliable method is to use the command Inverse[A], which computes the inverse of A. If all the
entries in A are in rational form and if A is singular (noninvertible), then a warning message
is displayed. If at least one of the entries in A contains a decimal point, a warning message is
given if A is singular or almost singular. In the latter case, Mathematica might not be able to
find the inverse. But if you first replace each entry in A with is rational equivalent, then the
inverse is computed and displayed, even if it is almost singular.
2.4 Partitioned Matrices
MATHEMATICA
Partitioned
Matrices
The command A|[ {t:, 72, ..., tr}, {J1, Jo,---,; js} ]] creates a r x s submatrix of A whose
entries consist of the numbers contained in rows %;, i2, ..., 2; and in columns jj, jo,..., js.
The BlockMatrix command creates partitioned matrices. For instance, if A, B, F, G, H, and
J are matrices having appropriate sizes, the command BlockMatrix[{{A,B,F},{G,H,J}}] creates
the larger matrix
(as
Bias
inesia
2.5
e
Matrix Factorizations
MM-11
2.5 Matrix Factorizations
MATHEMATICA
LU
Factorization
The Gauss command introduced in Section 1.4 is used to find the LU factorization of a given
matrix. Consider the LU factorization for Exercise 1 in Section 2.5. Suppose we have created
an assignment statement for matrix A in statement in[1]. Then the following two commands
are used to create the LU decomposition of A (the output statements are not shown).
In[2]:=Gauss[A, 1]
In[3]:=Gauss[%2, 2]
Then Out[3] contains the matrix U (note that accurate results for U are found using Gauss
whenever the entries of A and the arguments in Gauss are in rational form ).
The add-on package GaussianElimination in the LinearAlgebra folder contains a command
LUFactor that produces a permuted LU factorization of any square matrix A. Another command
in the package, LUSolve, uses LUFactor to solve Ax = b. These commands become useful when
you need to solve Ax = b repeatedly with a different b each time. For details, consult the
Mathematica documentation.
2.6 Iterative Solutions of Linear Systems
MATHEMATICA
Jacobi and Gauss Seidel Method
The Jacobi and Gauss-Seidel methods for solving Ax = b use the same Mathematica commands,
except for the construction of the matrix M in the algorithm described on page 143 in the text.
Assuming that M has been defined in In[2] as discussed later, the remaining Mathematica
commands are
nish
d Oa 10s, 04002):
(* Set up the initial approximate solution. *)
In[4]:=x[i_] = LinearSolve[ M, (M—A).x[i—1]+b]
In[5]:=Table[ x[ i], {i, 1, 6} ]
(* List the first six successive approximations *)
The statement in In[4] is a recursive definition of x[{ 7] in terms of z[ 1 — 1]. The command
LinearSolve[ C, d ] produces a solution to Cx = d.
To construct
M, we need several new functions.
The expression
Length[A] denotes the
number of rows of the matrix A. The expression Table[ Al[i, i]], {i, 1, Length[A]} ] is a list of
the diagonal entries in A, and if list is a list of n numbers, then DiagonalMatrix{ list ] is an n x n
matrix whose diagonal entries are the elements of list. Therefore, a diagonal matrix containing
the diagonal entries of A, is obtained using DiagonalMatrix[ Table{A([i, iJ], {i, 1, Length[A]} ].
This is the matrix M needed in Jacobi’s method:
In[2]:=M = DiagonalMatrix[ Table[A|[i, iJ], {i, 1, Length[A]} ]; | (*Jacobi*)
For the Gauss-Seidel method, we use the fact that if f is a function of two variables, the
command LowerDiagonalMatrix[ f, n ]produces an n x n lower-triangular matrix whose entries
above the main diagonal are all zeros and whose entry in the ith row and jth column below the
main diagonal is f{i, j]. When A is 3 x 3, the definition of M for the Gauss-Seidel method is
In[2]:=f[ i, j-] -= Alli, i]; M = LowerDiagonalMatrix[ f, 3];
| (* Gauss-Seidel *)
MM-12
Notes for Mathematica
2.9 Subspaces of R”
MATHEMATICA
RowReduce and Rank
The command RowReduce[A] produces the reduced row echelon form of A. From this matrix,
a basis for the column space of A is obtained and the homogeneous equations describing the
nullspace of A are formed by appending an extra column of zeros to the matrix.
Mathematica does not contain a Rank command. If the entries of A are in rational form,
RowReduce[A] reveals the rank of A, since Mathematica performs exact arithmetic in finding
the reduced row echelon form of A. If at least one of the entries of A contains a decimal point,
then roundoff error or an extremely small pivot entry can produce an incorrect echelon form.
3.1 Introduction to Determinants
MATHEMATICA
Random- Valued
Matrices
For positive integers m and n, the command Table[ Random[Integer, a, b], {m}, {n} ] creates a
m X n matrix whose entries are random integers between a and b.
3.2 Properties of Determinants
MATHEMATICA
Computing Determinants
To compute det A, define A and then repeatedly use Gauss[%n, r] and Swap[%n, r, s] (where n
is chosen so that %n refers to the output of the previous step of row reduction) as needed to
reduce A to an echelon form.
Now suppose the echelon form is obtained on line Out[{10]. On line In[11], let VU= %10. The
command Product f[i], {i, 1, n} ]computes the product f[1]f[2]---f[n]. Thus, if k represents
the number of row swaps used in the reduction of A to U, then the determinant of A is given
by
In[{12]:=(—1)*k Product[ Uj[i, iJ], { i, 1, Length[U]} ]
Notice the space between (—1)* and Product, which indicates multiplication of two numbers.
You can use the command Det[A] to check your work, but the longer sequence of commands
is preferred - at this time - because it emphasizes the process of computing determinants.
4.3 Linearly Independent Sets; Basis
MATHEMATICA
RowReduce and LinearEquations ToMatrices
The command RowReduce[A] produces the reduced row echelon form of A. From this matrix,
a basis for the column space of A is obtained and the homogeneous equations describing the
nullspace of A are formed by appending an extra column of zeros to the matrix.
Two consecutive equal signs, ==, represent an equal sign appearing in an equation. For
Exercise 33 in Section 4.3, an assignment statement is used to let eq[t] represent equation (5).
In{1}:=eq[t_] = cl t + c2 Sin[t] + c3 Cos[2 t] + c4 Sin[t] Cos[t] == 0
4.4
To generate a system of four equations and 4 unknowns,
equation (5) with the values of
In[2]:=Table[ eq[ t], { t, 0,
e
Coordinate Systems
MM-13
use the Table command
to replace
t= 0, 1, 2, and 3 (Out[2] is not displayed).
3} ]
Suppose eqlist is a list of linear equations containing unknowns 71, Z2,..., Zn. The command
LinearEquationsToMatrices| eqlist, { x1, x2, ..., xn } ] returns the list { A, b }, where A and b
form the matrix equation Ax = b equivalent to eqlist. The coefficient matrix A and vector b,
corresponding to set of equations given in Out[2], are found using In[3] (Out[3] is not displayed).
In[3]:={ A, b } = LinearEquationsToMatrices[ %2, { cl, c2, c3, c4 } ]
4.4 Coordinate Systems
MATHEMATICA
LinearSolve
The command LinearSolve[A, b] solves Ax = b where A is m x n. If Ax = b is inconsistent,
then a warning message is returned stating that a solution does not exist. If more than one
solution exists, one possible solution is returned. LinearSolve has additional capabilities that
will be introduced later, after you have the appropriate background.
4.6 Rank
MATHEMATICA
Mathematica
Rank and Random
Matrices
does not contain a Rank command.
If the entries of A are in rational form,
RowReduce[A] reveals the rank of A, since Mathematica performs exact arithmetic in finding
the reduced row echelon form of A. If at least one of the entries of A contains a decimal point,
then roundoff error or an extremely small pivot entry can produce an incorrect echelon form.
To create a matrix whose entries are random integers, see the Mathematica note for Section
Del’
4.7 Change of Basis
MATHEMATICA
Change-of-Coordinates
Matrix
Given vectors c1, C2, b;, and bz in R?, the command
In{1]:=M = Transpose[ {cl, c2, b1, b2} ]
produces a 2 x 4 matrix whose columns are cj, C2, bi, and be, respectively. (See the note for
Section 1.3.) Use RowReduce[M] to find a change-of-coordinates matrix in columns 3 and 4.
4.8 Applications to Difference Equations
MATHEMATICA
Solve
In Exercises 7-16 and 25—28, the coefficients of the polynomial in the auxiliary equation are
stored in a (row) vector form, in descending order. For instance, the polynomial r* — 25 from
Exercise 15 is represented by {1, 0, — 25}. The polynomial itself is recreated by the product
MM-14
~-Notes for Mathematica
{r?, r, 1}.p, where p = {1, 0, — 25} (notice the period before the p). The list {2 jen) is
produced by the command aoe ry {260 el}. The numbers 2, 0, —1 specify that the list
should start with r? and go to r°, with the exponent changing by —1 each time. The following
commands illustrate how to moive the auxiliary equation for the difference equation in Exercise
Cs
In{1]:= Clear[r];
In{2]:= p=C4S8[7]; Table[ r', { i, Length[p]—1, 0, -1 } ].p
Out[2]=4 — 4r -— 1? + °°
In[3]:=Solve[ %2 == 0, r ]
Out[3]:= { {r + —2}, {r > 1}, {r— 2} }
Note that Mathematica displays all polynomials from lowest to highest power as in Out[2]. The -
command Solve[ f[r] == 0, r], as above, attempts to find the roots of the equation f(r) = 0.
When Solve is unsuccessful, try NSolve instead, which attempts to find numerical approxima-
tions to the roots of f(r) = 0. The Clear[r] statement in the first line above makes sure that
any previously defined functions assigned to r are removed before a new assignment statement
is given.
4.9 Applications to Markov Chains
MATHEMATICA
exercises.
See Section 1.9 for information on how to use Mathematica in these
5.1 Eigenvectors and Eigenvalues
MATHEMATICA
NulBasis
The LayFunctions package contains a command that simplifies your homework by automatically
producing a basis for an eigenspace when you know the eigenvalue. For example, if A is a 3 x 3
matrix with eigenvalue 7, the command
In{1]:=NulBasis[
A — 7 IdentityMatrix[3] ]
produces a matrix whose rows form a basis for the eigenspace corresponding to \ = 7. Note
the space before the identity matrix, which indicates multiplication by the number 7.
In general, NulBasis[A] produces a basis for Nul A, the same basis you would get if you
started with the reduced echelon form of A and computed the basis by hand. Mathematica
has other eigenvalue and eigenvector commands that will be discussed later. You should use
NulBasis now, to reinforce the basic concepts in this section.
5.2 The Characteristic Equation
MATHEMATICA
The Characteristic Polynomial
Executing the command Det[A-A IdentityMatrix[5]] produces the characteristic polynomial of a
n Xn matrix A. For example, if A is 5 x 5,
In[1]:=Det[A-A IdentityMatrix[5]]
5.3
returns the corresponding characteristic polynomial.
e@
Diagonalization
MM-15
To plot the characteristic polynomial,
represented by %1 (the output to In[1]), execute
In[2]:=Plot[ %1, { A, a,b } ]
and the graph will be displayed for values of \ ranging from a to b. If you do not have access
to the BasicInput palette, substitute lambda for 2.
5.3 Diagonalization
MATHEMATICA
Finding Eigenvalues and Eigenvectors
In{1] produces a vector evals listing the eigenvalues of A where evals|[i]] represents the ith
eigenvalue in the list. Executing In[2] produces a list representing a basis for the eigenspace
corresponding to first eigenvalue evals|[1]]. The command NulBasis is part of the LayFunctions
package and can be replaced with the built-in Mathematica command
NullSpace.
In[1]:=evals = Eigenvalues[A]
In[2]:=v = NulBasis[ A — evals[[1]] IdentityMatrix[2] ]
Later on, the commands Eigenvectors[A] and Eigenvalues[A] are used to find the eigenvectors
and eigenvalues, of an n x n matrix A, respectively. The command Eigensystem[A] finds both
simultaneously, returning a list of eigenvalues followed by corresponding eigenvectors. Assuming
that A has been defined in In[1], the assignment statement in In[2] below assigns the names
evals and evecs to the list of eigenvalues and eigenvectors, respectively. In this way evecs|[i]] is
an eigenvector corresponding to the eigenvalue evals|[i]] for i = 1,2,...,n. Line In[3] creates
the diagonal matrix D (we use the name matD since D is a reserved Mathematica symbol).
Since evecs lists the eigenvectors in row form, the transpose of this list produces P in In[4]. The
command In[5] verifies that the diagonalization is correct. The //Simplify command is needed
to simplify the products.
A warning message will occur if matrix P in not invertible.
If A is
4 x 4 or larger and the entries of A are rational numbers, then Mathematica may not be able
to quickly compute the eigenvalues and eigenvectors of A. In this case, the entries of A should
be entered in decimal form.
In[2]:={evals, evecs} = Eigensystem|[A];
In[3]:= matD = DiagonalMatrix[evals];
In[4]:=P = Transpose[evecs];
In[5]:=A—P.matD.Inverse[P] //Simplify
5.5 Complex Eigenvalues
Complex Eigenvalues, Re and Im
MATHEMATICA
The letter | is a protected Mathematica symbol representing the imaginary number 2. If A is a
2x2 real matrix and if Eigenvalues[A] contains a complex eigenvalue, then you can use NulBasis
to find a corresponding complex eigenvector v. To build a 2 x 2 real matrix P from the real
and imaginary parts of v, use the Mathematica expressions Re[v] and Im|v], respectively. Then
compute C = P-1AP.
MM-16
Notes for Mathematica
5.6 Discrete Dynamical Systems
MATHEMATICA
Plotting Discrete Trajectories
For a given 2 x 2 matrix A, if x,4; = Ax, then the following commands are used to create x;
for k = 1,2,3,.... As an example, the matrix A defined in In[1] is found in Example 6 starting
on page 343 in your textbook. The purpose of statement In[2] is to define a function x where x[k]
represents x,. Statement In[3] uses the Table command to define a list called pts representing
x(0] through x[50]. Then the command ListPlot[pts] points is used plot the points in the zyplane and the trajectory is assigned the name trajl. The optional PlotStyle command is added
to increase the size of the displayed points on the screen (without this optional command, the
plotted points are small and barely visible).
In{1]:=A =
2
bilgi
be
rg
In{2]:==Clear[x]; x[0] = { 50, 60}; x{i_] := A.x[i— 1];
In[3]:=pts = Table[ x[i], { i, 0, 50 } J; trajl = ListPlot[ pts, PlotStyle—PointSize[.02] ]
To create another trajectory, repeat the commands on line In[2] with a different value of x[0]
and then repeat In{3], replacing trajl with traj2 to represent the new trajectory.
If necessary,
repeat this process to obtain traj3, traj4, ..., trajn and then execute Show|trajl, traj2,...,trajn]
to display all the trajectories on a single plot.
5.8 Iterative Estimates For Eigenvalues
MATHEMATICA
The Power Method
The steps for estimating the dominant eigenvalue appear on page 359 in your textbook. There
is no Mathematica function to determine yu, described in part b of step 2. The commands
Max{lis] and Min|lis] below compute the largest and smallest value in a list lis, respectively.
In statement In{1] below, a function called maxmag is created which determines the number
in a list whose absolute value is as large as possible.
The If command will return the second
statement, Max[x], if Max[x]>Abs[ Min[x] is true and it will return Min|x] otherwise.
In{1]:=Clear[ maxmag ];
maxmag|x_] := If[ Max[x]>Abs[ Min[x] ], Max[x], Min[x] ]
On the top of the following page, In[{2] is an application of the power method algorithm to
Example 2, beginning on the bottom of page 359, in your textbook. The goal of this string
of commands is to create a list called lis which contains the sequence of numbers and vectors
converging to the eigenvalue and a corresponding eigenvector, respectively. The command
For[k=0, k<=5, k++, body] evaluates the body of commands for k = 0,1,...,5. The value of y
stores the current value of Ax,, mu stores the current value of j4, and then Append[lis, {mu, x}]
adds the values of mu and x to lis. Finally x is overwritten with (1/mu) y, representing x,4 in
the power method algorithm. After the For loop, the lis statement causes the list to appear in
an output statement (which is not displayed here).
6.1
lnl2C=A- =
e
(; ;)Nees eee
Inner Product, Length, and Orthogonality
MM-17
1oOe lay.
Pom k=O ka=5, kot.
ye
Ao:
mu = maxmag| y ];
lis = Append{[ lis, { mu, x } ];
< = (1/mu) y; J;
is
MATHEMATICA
The Inverse Power
Method
The inverse power method is a modification of the power method.
sponds to Example 3 starting on page 361 in the textbook.
10 -8 =4
Ini2( =A
Onl.
Ao
—4
4
5
lis =)
x
ee eee
The following code corre-
eal De—mlecs
For[k=0, k<=5, k++,
y = LinearSolve[ A—alpha IdentityMatrix[3], x ];
mu = maxmag| y |;
v = alpha+1/mu;
lis = Append| lis,{ mu, x } ];
x = (1/mu) y];
lis
6.1 Inner Product, Length, and Orthogonality
MATHEMATICA
Inner product and Norm
If u and v are vectors, then u.v is their inner product.
The norm of v is Sqrt[v.v]. On the
BasicInput palette, there is a ,/O) button you can use to compute //V.v.
6.2 Orthogonal Sets
MATHEMATICA
Orthogonality
Executing U = Transpose[ { ui, U2, ..., Un } ] followed by Transpose[U].U is a quick way to
determine the orthogonality of { ui, uz, ..., Un }. Note: U is constructed using the Transpose
command in order to produce a matrix whose columns are { uj, U2, ..., Un }.
To compute the orthogonal projection of y onto u, evaluate (y.u)/(u.u) u.
MM-18
Notes for Mathematica
6.3 Orthogonal Projections
MATHEMATICA
Orthogonal Projections
The orthogonal projection of a single vector onto a single vector was described in the Mathematica note for Section 6.2. The orthogonal projection onto the set spanned by an orthogonal
set of vectors is the sum of the one-dimensional projections. Another way to construct this
projection is to normalize the orthogonal vectors, place them in the columns of a matrix U, and
use Theorem 10. For instance, if {y1, y2, y3} is an orthogonal set of nonzero vectors, then the
command
In[1]:=U=Transpose[ { y1/V/Y1.-y1, y2//y2.y2, y3/V/y3-y3 } |
creates U, containing orthonormal columns, and
In[2]:=U.( Transpose[ U ].y )
produces the orthogonal projection of y onto Span{y1,
y2,
ys}.
(The parentheses around
Transpose[U].y speeds up the computation by avoiding a matrix-matrix product.)
6.4 The Gram-Schmidt Process
MATHEMATICA
The Gram-Schmidt
Process
Let A be a matrix with n linearly independent columns.
In Mathematica,
the columns are
Transpose[A][[k]] for k = 1,...,n. If nm = 2, the Gram-Schmidt process is
In{1]:= xl = Transpose[A]|[[1]]; x2 = Transpose[A]|[[2]];
Inf2hevl = exh
ln[3}iv2 = x2. —(x2.V1l. a vival
If A has three columns, add x3 = Transpose[A]|[[3]]; to the end of In[1] and then compute
In[4]:=v3 = x3 — (x3.vl ) / ( v1.v1) v1 — (x3.v2 ) / ( v2:v2) v2
To check your work, use the command GS[A], which uses the Gram-Schmidt process on
the columns of A to produce a matrix whose columns are orthogonal. When the columns are
normalized, they form the columns of Q in the QR factorization of A.
6.5 Least-Squares Problems
MATHEMATICA
Finding a Least Squares Solution
To find a least squares solution to Ax = b, execute the following LinearSolve and Transpose
commands.
In[{1]:=LinearSolve[ Transpose[A].A, Transpose[A].b ]
Exercises 15 and 16 should be worked by hand, to review the QR factorization.
See
the Numerical Note near the end of Section 6.5 in the text. For Exercise 26, the command
A=BlockMatrix[ { {Al}, {A2} } ] creates a partitioned matrix whose top block is A, and whose
bottom block is Ag.
6.6
e
Applications to Linear Models
MM-19
6.6 Applications to Linear Models
MATHEMATICA
Functions of Vectors
If x is a vector and k is a real number, then the Mathematica expression x°k is a vector whose
entries are the kth powers of the corresponding entries in x. Similarly, the expression Sin[x]*k
is the vector produced by applying the function (sinz)* to each entry of x. The exponential
function E*x or Exp|x] and the natural logarithm function Log{x] also act on each entry of
x. For example, Log] {1, 2, 3} ] is the same as { 0, Log[1], Log[2] }.
6.8 Applications of Inner Product Spaces
MATHEMATICA
Graphing Functions
The Plot function is used to display the graph of one or more functions. In[1] contains three
assignment statements for the functions appearing in Figure 3 on the top of page 437. The
remaining plot commands display the graphs of the functions. The constant 7 is entered using
the BasicInput palette or by typing in the expression Pi.
lft
=F iats |=:
f3[ t_] := a — 2 Sin[ t ]—Sin[ 2 t ] —2/3 Sin[ 3 t J;
f4[ t_] := f3[ t ]—2/4 Sin[ 4 t J;
In[2]:=Plot[ {f[t], f3[t]}, {t, 0, 27} ]
In{3]:=Plot[ {f[t], f4[t]}, {t, 0, 27} ]
7.1 Diagonalization of Symmetric Matrices
MATHEMATICA
Orthogonal Diagonalization
Use the Eigenvalues and NulBasis commands to find the eigenvalues and corresponding eigenvectors as in Section 5.3. If you encounter a two-dimensional eigenspace, with a basis {v1, v2},
use the command
In{1]:= v2 = v2 — (v2.v1)/(v1.v1) v1;
to produce an eigenvector orthogonal to v,;. (Review Section 6.2 in your textbook and the
Mathematica note for Section 6.4). After you normalize the vectors and create a matrix P,
compute P? P — J to verify that P is indeed an orthogonal matrix.
7.4 The Singular Value Decomposition
MATHEMATICA
The Singular Value Decomposition
The command { evals, evecs } = Eigensystem[ Transpose[A].A | produces a list of eigenvalues
evals, in decreasing order, and a matrix evecs whose rows are the eigenvectors corresponding to
the eigenvalues in evals. Therefore, assigning sigma to DiagonalMatrix[ Vevals | produces © and
letting V equal Transpose[evecs] produces the matrix V. To form U for the SVD, normalize the
nonzero columns of A.V. If U needs more columns, use the method of Example 4.
MM-20
Notes for Mathematica
This construction helps you think about properties of the factorization.
In practical work,
however, you should use the command { U, sigma, V }=SingularValues[ Transpose[A].A] ] which
produces a list of singular values sigma along with U and V.
7.5 Applications to Image Processing
MATHEMATICA
The command
Computing
Principal Components
rmean=MapThread[ Plus, Transpose[X] ] / Length[ Transpose[X] ] produces a
vector whose jth entry lists the mean average of the jth row of X and DiagonalMatrix[rmean]
creates a diagonal matrix whose diagonal entries are the row averages of X. Finally, the
command
rmean.Table[ 1, {Length[X]},{Length[X]} ] creates a matrix the size of X, whose
columns are the same, each one listing the row averages of
X. To compute the data in X into
mean-deviation form, use
In{1]:=B = X—rmean.Table[ 1, {Length[X]}, {Length[X]} ]
The sample covariance matrix is produced by
In[2]:=S = B.Transpose[B]/(Length[X]—1)
and the principle component data is produced by
In{3]:= { U, sigma, V } = SingularValues[ Transpose[B] / Sqrt[Length[X]—1] ]
See the Numerical Note at the end of Section 7.5 of the text.
INDEX OF MATHEMATICA COMMANDS
General Commands
:= (delayed equals), MM-9
= (equals), MM-3
== (equation equality), MM-12
? (help), MM-5
. (inner product), MM-17
. (Matrix multiplication), MM-9
. (Matrix-vector multiplication), MM-8
* (multiplication), MM-2
%n (Out[n]), MM-5, 6, 15
“ (power), MM-2, 9, 19
Append, MM-17
AppendRows, MM-7, 8, 10
BlockMatrix, MM-10, 18
Clear, MM-9, 14
Det, MM-12, 14
DiagaonalMatrix, MM-11, 15, 19
Eigensystem, MM-15, 19
Eigenvalues, MM-15, 19
Eigenvectors, MM-15
Exp, MM-19
For, MM-16, 17
IdentityMatrix, MM-10
If, MM-16
Inverse, MM-10
Length, MM-11
LinearEquations ToMatrices, MM-13
LinearSolve, MM-11, 13, 18
ListPlot, MM-16
Log, MM-19
LUFactor, MM-11
LUSolve, MM-11
MapThread, MM-20
MatrixForm, MM-4
MatrixPower, MM-9
Max, MM-16
Min, MM-16
Needs, MM-1
NullSpace, MM-15
Outer, MM-10
Plot, MM-15, 19
PlotStyle, MM-16
Post, MM-5
Product, MM-12
Random, MM-10
RowReduce, MM-2, 3, 11, 12, 13
Show, MM-16
Simplify, MM-15
Sin, MM-19
SingularValues, MM-20
Sqrt, MM-17, 18
Table, MM-9, 10, 11, 12, 13, 14, 16, 20
Transpose, MM-6, 7, 9, 15, 18, 20
ZeroMatrix, MM-8
LayData Folder, MA-1
ChapterN package, MM-1, 5, 7
LayFunctions Package, MM-1, 5, 6, 14
BGauss, MM-8, 10
Gauss, MM-8, 10, 11, 12
GS, MM-18
NulBasis, MM-14, 15, 19
ReplaceRow, MM-6
Scale, MM-6, 8, 10
Swap, MM-6, 8, 12
Misc Package, MM-1
LinearAlgebra Folder, MM-1, 11
Cholesky package, MM-1
GaussianElimination package, MM-1, 11
MatrixManipulation package, MM-1, 5
Orthogonalization package, MM-1
Tridiagonal package, MM-1
MM-21
hie cotworudinin ip pall ea a
it, YOU Noe ee th leh SY esa
7
be
ay
Prod)
;
On
i
SEShf
Ly i ie Ray
;
pA
» a ny;
withte a d
Ae,ae
af a-MaM sold
SIM
MATHEMATICA
rhe
os
hf nT
Abbiat
G3)
i p
Principal Cam portal a:
SaidLi sdoubatthe,... Tra napaseley PX Lenght
u
one 10) enti’
eraphe
a
Saran
und. rem
seis
ca.
shaziol4
nMiNiwnebaerhverage of the bP Wvedgy (xx .
cat
1 MM aba
skanvisiiyd ronan Table REMENWedR { GuuhOQ) =e gue
éntarach ase
th
a Ms aCe 0we DOW: averagesGP 7)
oven hdeviolion
bitei
ferns, we
nail
DG
BY STM
Oh PE SP Sr yy ORAM
il dhe
mle
btn
eT
weer
oot
9
*| 1)
coche Mi wametdee Sie cad
ahha
VycM, Penre {86}, Oe# Tyce.
Lf ‘M
AOBADA q
Ui
~
try
‘eepeicea
Is
:
e
2tuede
M1
oi
Mi
0;
DIMM
“lL WM
.2o
,aeetlluy
Cf
.
OI
aA
“Lid:
Ry akelthing
reiterate
y of :hit te ty
_
én
ae)
.yenrl4
oi
i MM
tt I-A
ee
as
Riv
ee
Scat
OLRMA
>_>
= DRM (peioval aa
“TEMM
Jail
:
nie
sitgneJ
%
Lia Priel orvrineieenr
«Sf BE LMM sviodimonid a
geeGED
qj hoe
cae
.
st BLMM pete |
“OE MEM:atnse Maing
|
EMM cohol degli Aesni)
WM novE
datinNIIO
g, heaton)
Syeelad.
a Ate) J Syl:
wor: fi
nigel? ;
o*2
segs
a
: ‘ eos
FE?
fried.
O1MM soa’
Ale
a)
;
1G
or JAM gol
i1-MM
Spent ame
i
a
- er fut & MM panies
Sk an oa
peared ides
Li
efjar
trwyyo raty éJ
BoM
ers
pe. nt
fi caged AMS rohaiga
ate ele! Trasspinel
Mike
ai ic
io
aig
ome idan
polviple
gedaan
Tiled
OT-MM'
angie ashyNee ih,
/
MG Bt APF
war: rialonto
Arr
Gen
OSM, bead TgeMe
;
b-MM olintsM.
|”
id
“4
O-MM sawo%aiteM
\
'
aie
a)
VM
6M
01 -EM. nie
-MiM zhao
,
Notes for the TI-85 and TI-86 Graphing Calculators
GETTING STARTED WITH A TI-85 OR TI-86 CALCULATOR
Entering Matrices
There are several ways to enter matrices and vectors into the TI-85/86
calculators. One method consists of entering the matrix directly as an expression, another method
is through the use of the matrix (vector, respectively) editor found in the [MATRX] ([VECTR],
respectively) menu. Below are examples that illustrate the creation of matrices and vectors using
these methods. The entries in a matrix or vector can be either real or complex (pairs of real
numbers). Some special matrices and vectors are constructed with additional commands found in
the [MATRX] ({[VECTR], respectively) menu, we illustrate these in the appropriate notes. For
other entry methods and additional information consult the TI-85/86 Graphing Calculator
Guidebooks.
Direct Entry
Begin a matrix with a square bracket, and each row of the matrix with a
matching pair of square brackets; separate the entries in the rows with commas. Figure 1 below
Lanes 3
shows how you can enter the matrix
|4
5
tes
6] directly and assign to it the name MATI1
(case
oho,
sensitive)—the symbol » appears after pressing the STO») key, (the closing ']]' is not necessary at
the end of the command)—press
to display the matrix MAT1 on the screen. For
complex numbers, follow the same procedure, only enter the entries as pairs of real numbers (i.e.
(real, imaginary)).
Figure 1 MATI entered directly.
The Matrix Editor
— Press2ndl (yellow key), [MATRX] (7 key), to see
in the menu. The
editor allows you to create a matrix by specifying a variable name, a size—row size and column
size—and the entries of the matrix. The editor, Figure 2, is accessed by pressing the [F2] menu key.
Select from the matrix names shown or enter a new name for the matrix. For example to enter the
matrix Q = i
a type Q and press |ENTER;; your screen should be similar to Figure 3.
TI-1
TI-2
Notes for the TI-85 and TI-86 Graphing Calculators
MATRX:Q
171=0
c
Tmatil
x >
Figure 2 The matrix editor.
4cOL 7 COLD E INSy IT DELr f INScD
Figure 3 TI-85/86 matrix editor, ready to receive Q.
The cursor is blinking at the top right, ready for you to enter the number of rows, press |2} and
ENTER); for the number of columns, press |3}and |ENTER|. Type in the entries as prompted by
rows. With the TI-86 use the cursor arrows to move up or down in a column, or left and ile ina
row. (With the TI-85 use the cursor arrows to move up or down in a column and use the FL
|F1} or
menu keys to move left or right in a row.) Press the
key to exit the editor. To view the
matrix Q on the screen, press [ALPHA] (blue key), [Q] ({4] key) and [ENTER\.
You can create vectors in a similar fashion. If using direct entry, a vector is entered as a
row vector: [1, 2, 3, 4]. If using the vector editor, the procedure
described above.
is similar to the matrix editor
WARNING: Vectors and matrices are different objects; operations in the [MATRX] menu are
only for matrices, and operations in the [VECTR] menu are only for vectors. Vectors can also be
entered as nx1 matrices; it will be impossible to use the operations in the [VECTR] menu on
these. In the following notes the term vector is never used for an nX1 matrix. The programs
and
convert an nxl matrix into a vector, and a vector into an nx1
matrix,
respectively. For TI-85/86 calculators row vectors do not exist.
STUDY GUIDE NOTES
1.1 Systems of Linear Equations
TI-85/86
Row Operations
In this exercise set, the data for each exercise are stored in a matrix M. Row operations on M are
performed by the following commands:
mRAdd(m,M;>s>r2
Replaces row r ofM by (rowr
rSwaPp(M-r;s)
Interchanges rows r and s of M
multR¢m>Msr>
Multiplies row r of M by a nonzero scalar m
rAdd¢M;s-+r)
Replaces rowr
of M by rowr+rows
) + m x (row s)
Vector Equations
These operations are found on the second page of the [MATRX]
(yellow) key followed by
[7], to bring up the [MATRX]
menu.
°
TI-3
(Press the 2nd)
menu, Figure 4. The row operations are
found when you press the [F4] menu key followed by the [MORE] key, Figure 5.)
To use the row operations efficiently it is convenient to store matrices in memory.
The name of
any matrix in your calculator memory can be inserted in place of M; the letters r, m, and s stand
after each command.
for numbers you choose. Press
INAMEST
EDIT PMaTHT
ops
| cPLx |
Figure 4 The matrix menu
aug
ne
rAdd ae
|
Figure 5 The matrix menu showing row operations
If you enter one of these commands, say, rSwap¢M; 13>, then the new matrix, produced from
M, is stored in the matrix "ANS" (for "answer"). If instead you enter rSwap¢M;12329M1,
the answer is stored in a new matrix M1. If the new operation is mRAdd¢5,M1;,1,2399M2,
then
then
the result of changing M1 is placed in M2, and so on.
The advantage of giving a new name to each new matrix is that you can easily go back a
step if you don't like what you just did to a matrix. If, instead you enter MRAdd(5,M,1,23M,
then the result is placed back in M and the "old" M is lost. Of course the "reverse" operation
mRAdd¢ -52M21223M, will bring back the old M.
Note: For the simple problems in this section and the next, the multiple m you need in
the command mRAdd‘m;:M:s-r) will usually be a small integer or fraction that you can
compute in your head. In general, m may not be so easy to compute mentally. The next two
paragraphs describe how to handle such a case.
The entry in rowr
and column c of a matrix
M is denoted
by M(r, c). If the number
stored in M(r, c) is displayed with a decimal point, then the displayed values may be accurate to
only about five digits. In this case, use the symbol M(r, c) instead of the displayed value in
calculations.
For instance, if you want to use the entry (pivot) M(s, c) to change M(r, c) to 0, enter the
commands
-M¢rsco37MCs2C3%m
mRAdd<m:M:s+2r22M
The multiple
of row s to be added
to rowr
Adds m times rows torowr
or you can just use the command mRAdd< -M¢rsco/7MCs2cdsMsssrdoM.
1.3 Vector Equations
TI-85/86
Constructing a Matrix
The aug operator concatenates two matrices or a matrix and a vector. (see the second page of the
[MATRX]
menu. Press 2nd] [MATRX] [F4]
Figure 6. Fill in with the two desired arguments
to view the command, then [Fi] to select it,
and press [ENTER|.)
The command
TI-4
Notes for the TI-85 and TI-86 Graphing Calculators
au9(au9ai;a2>;au9g¢a3;sb> 22M creates the matrix [a, a,
but, if you enter al» a2,a3 and b as vectors, you must convert al
a, b] and saves it as M;
and a3 ton Xl matrices
program in the
before using the aug command. (Use the
is created by the command aug¢A+b)>M—assuming
menu.) The same matrix
that the matrix A and the vector b, have
been created.
aug
tee
rAdd eee
Figure 6 The au9¢ command.
Exercises
11-14
and
25-28
can
be
solved
using
the
commands
mRAdd,
rSwap,
and
(occasionally) multR, described in Section 1.1.
1.4 The Matrix Equation Ax = b
TI-85/86
[GAUS| AND
To solve Ax = b, row reduce
the matrix
[A
b]*M.
The command
[5;3:
-71%x
creates a
vector x with entries 5, 3, -7. Matrix-vector multiplication is A¥x.
To speed up row reduction of [A
b]*M, the program
(in the [PRGM] menu) will
use the leading entry in row r of M as a pivot, and use row replacements to create zeros in the
pivot column below this pivot entry. The output of (GAUS| is stored in the matrix U. If you wish,
you can assign U to some other variable, such as M itself.
For the backward phase of row reduction, use the |BGAUS| program, which selects the
leading entry in row r as the pivot, and creates zeros in the column above the pivot. Use multR
to create leading 1's in the pivot positions. The output of
is stored in the matrix U. If you
wish, you can assign U to some other variable, such as M itself.
To run a program, enter [PRGM]
and use the [MORE] key as many times as
necessary until the name of the program appears in the menu. Press the desired menu
through [F5}) then
to begin execution of the program. Both [JGAUS] and
key (Fi]
will
prompt the user for the matrix and the row number.
1.5 Solution Sets of Linear Systems
TI-85/86
The Zero Matrix
The command <msn>%dim Z creates an mxn matrix of zeros and stores it in Z, if Z does not
already exist (if Z already exists, the command will reformat Z and fill it with zeros). The same
procedure applies to vectors. The '{' and '}' symbols are found in the [LIST] menu, and the dim
command is in the [MATRX]
menu (the > symbol appears when pressing the STO*|key.)
When solving the equation Ax = 0 create an augmented
matrix aug¢A»Z29M,
and assign it to
Linear Models in Business, Science, and Engineering
TI-5
the variable M, or execute the commands
A>M
{msntlixdim
M
m and n are the number of rows and columns of A
to reformat the matrix A as an mx(n + 1) matrix with a column of zeros appended and save it as
M. Then use [GAUS],
and multR to row reduce M completely.
1.9 Linear Models in Business, Science, and Engineering
TI-85/86
Generating a Sequence
Enter and store the matrix M and the vector x, as x. Then use the command M*x>x repeatedly to
generate the sequence x, , X, , ... . You only enter the command once. After that, press
[ENTRY] to recall M*x+x and press ENTER] .
In Exercise 11 you need 6 decimal places to
TI-85/86 calculators display up to 12 digits; press
get four significant figures in M(1, 2). The
[MODE], use the cursor keys to select
Normal and Float, then [EXTT].
Numbers are entered into the TI-85/86 without commas. The number 600,000 in TI85/86 scientific notation is 6ES. A small number such as .00000012 is 1.2E~-7.
2.1 Matrix Operations
TI-85/86
Matrix Notation and Operations
To create a matrix, begin with a square bracket, enter the data row-by-row, with a comma between
entries, each row of the matrix must begin and end with square brackets. For instance, the
command
[f[12:2:31[4:5,:6]1]?A
Usebrackets
around the data
creates a 2x3 matrix A. If A is mxn, thendim
A (found in the [MATRX]
menu) is the list
{m n}. The (i, j)-entry in the matrix A is A(i, j). A(i) is the ith row of the matrix A, given as a
vector. To extract the jth column of A, use the program (cou, the program will prompt for a
matrix, and column number, it will return the jth column as a vector. The command
A<(i, j, r, s)
extracts, from A, the submatrix that begins in row i, column j, and ends in row r, column s.
Examples:
AC1is;3:m:; 33
AC23
column 3 of A, as a matrix
row 2 of A, as a vector
To specify columns 3, 4, and 5 of A, you can use
AC1:3,m;:59
m is the number of rows of A
TI-6
Notes for the TI-85 and TI-86 Graphing Calculators
The TI-85/86 use the Ht
, and [4 keys to denote matrix addition, subtraction
and
multiplication, respectively. (Multiplication shows on the screen and in these notes as *.) If A is
square and k is a positive integer, A al k denotes the kth power of A (A ke], is equivalent to
Af)
2). The transpose of A is AT (the T operator is in the [MATRX]
menu). Note: when
A has complex entries, the (i, j)-entry of AT is the complex conjugate of the (i, j)-entry of A.
If u and y are two vectors of the same size, then dot¢u;v> is their inner product (the
dot operator is in the [VECTR]
menu). If the vectors are entered as nx1 matrices the
inner product is v™*u. The outer product of two real vectors u and v is computed as u*vT and
you must enter the vectors as nx1 matrices. In this section you can experiment with random
matrices. The command randM returns a random matrix with integer entries in the interval
[-9, 9]. Go to the [MATRX]
two arguments—row
menu to activate the command;
it requires that you enter the
size and column size—then close the parenthesis and press [ENTER]. (To
save the random matrix into memory press the [STO»] key and enter the name of the matrix
before you press |ENTER..)
2.2 The Inverse of a Matrix
TI-85/86
The Identity Matrix
The nxn identity matrix is ident
A is nxn, then aug¢As ident
and
n (the ident
command is in the [MATRX]
n>M creates the augmented matrix M=[A_
menu). If
I]. Use the
programs, and the multR operation to reduce [A_ I].
2 ere are other commands
that can be used to row reduce matrices, invert matrices, and
solve equations Ax = b, but they are not discussed here because they will not help you to learn
the concepts in this section.
2.3 Characterizations of Invertible Matrices
TI-85/86
Inverse
Determining whether a specific numerical matrix is invertible is not always a simple matter. A fast
and fairly reliable method is to enter the matrix A and press 2nd) A [x-1], which computes the
inverse of A. If the matrix in question is not invertible, the TI-85/86 will display the message
"SINGULAR
MAT".
For exercises 38-41,
MATH] menu.
the condition
number,
cond,
is in the [MATRX]
2.4 Partitioned Matrices
TI-85/86
Partitioned Matrices
The TI-85/86 use partitioned matrix notation. For example, if A,.B, C, D, E, and F are matrices of
appropriate sizes, then the command
augCaugCaugtA;BI;CIT,
augCaug(D,E,FITITOM
Matrix Factorizations
creates a larger matrix of the form
M
=
ASE BEG
Dick
TI-7
|Once M is formed, there is no record of
the partition that was used to create M. For instance, although B was the (1, 2)-block used to form
M, the number M(1, 2) is the same as the (1, 2)-entry of A.
2.5 Matrix Factorizations
TI-85/86
LU Factorization
Row reduction of A using the [GAUS] program will produce the intermediate matrices needed for
an LU factorization of A. You can try this on the matrix in Example 2. The matrices in (5) on
page 136 in the text are produced by running the commands
AUS
AUS
Atatd
Returns a matrix U with 0's below the first pivot
Ur 2
Urs
Returns a matrix U with 0's below pivots 1 and 2
Returns an echelon form, U
You can copy the information from your screen onto your paper, and divide by the pivot
entries to produce L as in the text. For most text exercises, the pivots are integers and so are
displayed accurately.
The TI-85/86 command LUCA,L2U;P) produces a permuted LU factorization for some
square matrices A, but it does not handle the general case.
2.6 Iterative Solutions of Linear Systems
TI-85/86
Jacobi and Gauss-Seidel Methods
The [MdiaV| program produces a vector (not an nx1 matrix) that lists the diagonal entries of a
given matrix. To create an nxn diagonal matrix that has the same diagonal entries as the matrix A
use the
program. Both programs prompt you for the matrix A. The TI-85/86 calculators
do not have commands for creating triangular matrices. To create lower triangular part of A first
copy A into the matrix M, then edit the matrix M so that it contains
zeros above the main
diagonal.
The
Jacobi
and
Gauss-Seidel
methods
use
the
same
commands,
except
for
the
construction of M. Enter M - A*N, and make sure the initial data are in the vector x. To produce
the "next" x, you must solve the equation M* x” = N*x +b for x”. The correct way to do this is
to use the TI-85/86 inverse key (2nd) [x-1]) to compute
x” = M-1*(N*x
+b). However, instead of assigning the solution to x”, you should put the
solution back into x, so you can repeat the operation with exactly the same symbols, that is, the
command
M-tk CN*X+b
99x
is used over and over, creating the sequence x’. These vectors can be recorded on paper as they
appear; you may need to scroll down to view all the entries. You only enter the command once.
After that, use the 2nd) [ENTRY]
keys to recall and execute the command.
The
TI-8
Notes for the TI-85 and TI-86 Graphing Calculators
numerical results may vary from the answers in the text depending on the number of digits of
:
accuracy set in the calculator.
If you wish to monitor how close the approximations are to each other, use the following
sequence of commands (repeatedly):
M-lkCN*x+b>>t
t-x?change
t>x
t is temporary storage for the new x
Compare
t with the oldx
Update x with new value
2.9 Subspaces of R"
TI-85/86
rref and ref
The command rref A, in the [MATRX]
menu, produces the reduced row echelon form of
A. This form gives you enough information to write down a basis for Col A, and the
homogeneous equations that describe Nul A. (Don't forget that A is a coefficient matrix, not an
augmented matrix.) The command rref does not work if the number of columns in A is less
than the number of rows in A. If this is the case, you can augment A with enough columns of
zeros, and work with a square matrix. Use the command
as usual, then ignore
the columns
of
zeros when you interpret the output.
Your calculator also has the command ref A in the [MATRX]
menu, that produces
a row echelon form of A (the number of columns in A must be greater than the number
in A). This form gives you the pivot positions, but you
homogeneous equations that describe Nul A.
You can use ref
of rows
will not be able to write down
A to check the rank of A, but roundoff
error of an extremely
the
small
pivot entry can produce an incorrect echelon form.
3.2 Properties of Determinants
TI-85/86
Computing Determinants
To compute det A, use the
program and rSwap¢(matrix,rowl,row2),
repeatedly, to
reduce A to a matrix U, which 1s an echelon form of A. Keep track of how many times you swap
rows. Then except for a +1, the determinant is obtained by using the program
to extract
the diagonal of U, followed by the [PRDCT] program to compute the product of the diagonal. You
can, of course, use the det] key (on the first page of the [MATRX]
menu) to check your
work, but the longer sequence of commands helps you to think about the process of computing
det A. For Exercise 46, the condition number, cond, is in the [|MATRX] MATH menu.
4.3 Linearly Independent Sets; Bases
TI-85/86
rref
With a TI-85/86, you should use an echelon form of A to determine the rank of A.
You can row
reduce A using |GAUS}, or you can use one of the commands rref or ref from the [MATRX]
[OPS] menu.
The ref
A command
produces a row echelon form of A.
Bothrref
and ref
Coordinate Systems
only work on matrices with at least as many
columns
as rows.
»°
TI-9
However, you can augment
A with
enough columns
of zeros to make a square matrix, if necessary, and then ignore the columns of
zeros when you interpret the output. With each of these three methods, roundoff error from an
extremely small pivot entry can sometimes produce an incorrect echelon form. An advanced
method for rank determination is discussed in Section 7.4.
4.4 Coordinate Systems
TI-85/86
Finding a Coordinate Vector
If the equation
Ax = b has a unique
solution and A is a square matrix, the TI-85/86
will
automatically produce x if you use the command
A-i*xb>~x
In this section, the equation will probably have the form Pu = x, with u the B-coordinate vector of
x, and the command will be P-1*xu.
4.6 Rank
TI-85/86
ref and rank
With a TI-85/86, you should use an echelon form of A to determine the rank of A.
You can row
reduce A using [GAUS], or you can use one of the commands rref or ref from the [MATRX]
[OPS] menu.
The
ref
A command
produces
a row echelon form of A.
Both rref
and ref
only work on matrices with at least as many columns as rows. However, you can augment A with
enough columns of zeros to make a square matrix, if necessary, and then ignore the columns of
zeros when you interpret the output. With each of these three methods, roundoff error from an
extremely small pivot entry can sometimes produce an incorrect echelon form. An advanced
method for rank determination is discussed in Section 7.4.
In this section you can experiment with random matrices. The command randM returns a
random
matrix with integer entries in the interval [-9, 9]. Go to the [MATRX]
activate the command;
it requires
that you
enter the two
arguments—row
size and
menu
to
column
size—then close the parenthesis and press[ENTER\. (To save the random matrix into memory press
the STO) key and enter the name of the matrix before you press ENTER.)
4.7 Change of Basis
TI-85/86
Producing a Change of Basis Matrix
A command will completely reduce a matrix ic;
The rref
See the note for Section 1.3 to on how to construct this matrix.
Coe,
b,| to the desired form.
TI-10
Notes for the TI-85 and TI-86 Graphing Calculators
4.8 Applications to Difference Equations
TI-85/86
POLY and SOLVE
To find the roots of a polynomial, use the 2nd] [POLY] sequence. Input the order (degree) of the
polynomial, and the coefficients as prompted (press
after each entry), then press the
menu key. The roots of the polynomial are displayed on the screen. Complex roots
appear as pairs of real numbers. Refer to your TI-85/86 Graphing Calculator Guidebook.
4.9 Applications to Markov Chains
TI-85/86
The TI-85/86 note for Section 1.9 contains useful information.
5.1 Eigenvectors and Eigenvalues
TI-85/86
Finding Eigenvectors
The program
(in the [PRGM] menu) will simplify your homework
by automatically
producing a basis for an eigenspace. For example, if A is a 3x3 matrix with an eigenvalue 7, first
store A, then use the keystrokes
A-7*2nd) [MATRX]
ident] 3c
to produce the matrix C = A — 7I. Then run the JNULB| program; at the prompt for a matrix, enter
C. The output is a matrix B whose columns form a basis for the eigenspace of A corresponding to
X = 7. In general ident
k (in the [MATRX]
menu) produces the kxk identity matrix and
produces a matrix whose columns form a basis for Nul C (the same basis you would get if
you started withrref
C and calculated by hand).
Remarks:
|S
The program JNULB) uses the command rref and requires that the number of columns
of the input matrix be greater than or equal to the number of rows.
Pas
If the numbers in the basis matrix B are messy, try the sequence
[MATH]
3
bFracl
of keystrokes 2nd)
which shows the last answer in fractions, if possible.
Although your TI-85/86 has more powerful commands for eigenvector calculations, you
should not use them yet, because you need to learn basic concepts about eigenvalues and
eigenvectors.
The Characteristic Equation
¢
TI-11
5.2 The Characteristic Equation
TI-85/86
CHAR
You can use the [CHAR] program to check your answers in Exercises 9-14 Note that if A is nxn,
running this program produces a vector listing the coefficients of the characteristic polynomial
of
A, in order of decreasing powers of A, beginning with A”. If the polynomial is of odd degree, the
coefficients are multiplied by -1, to make +1 the coefficient of A’; this corresponds to finding
the determinant of AI — A. (The program works for matrices up to size 3x3.)
5.3 Diagonalization
TI-85/86
eigVI and eigVc
The command eigV1
A>ev (eigV]is in the [MATRX]
menu) produces a list, of the
eigenvalues of the matrix A. (The > sign sets the calculator to ALPHA mode, to produce lower
case press 2nd] [ALPHA].) To learn the diagonalization procedure, you should use the method of
Section 5.1 to produce the eigenvectors. For instance, A-ev¢1>*ident
A — Al, then use B as input for the
5B
is the matrix B =
program to create a matrix whose column(s) give a
basis for the eigenspace corresponding to the first eigenvalue of A listed in ev.
In later work you can automate the diagonalization process. The command
(kigV¢| is in the [MATRX}
eigVc»P
menu) produces a matrix P such that AP = PD, where D is the
diagonal matrix you can create from the list of eigenvalues, with the eigenvalues in the diagonal.
(Use the command 1i*ve, in the
menu, to change ev into a vector, then the program
[VdiaM], with ev as input, to produce D.) If A happens to be diagonalizable, then P will be
invertible. In any
homework.
ENTER]
case, P is likely to be quite different
from
what you
construct
The columns of P may be scaled. (The sequence 2nd) [MATH]
for your
bFracl
shows the last answer in fractions, if possible.)
5.5 Complex Eigenvalues
TI-85/86
The
Complex Eigenvalues and Eigenvectors
and
keys (mentioned
in Section 5.3) also work for matrices with complex
eigenvalues. In this case the resulting list of eigenvalues or the matrix containing the eigenvectors
as columns have some complex entries.
For any matrix V thefreal] and
keys in the [MATRX]
menu produce the real and
imaginary parts of the entries in V, displayed as matrices of the same size as V.
5.6 Discrete Dynamical Systems
TI-85/86
Plotting Discrete Trajectories
Given a vector x, the command A*xx
will compute the "next" point on the trajectory. Use the
2nd [ENTRY] keys to repeat the command over and over.
TI-12
Notes for the TI-85 and TI-86 Graphing Calculators
The following program creates a "trajectory" matrix whose columns are the points x, Ax, AAO,
A'*x. (Change 15 to any number you wish.) The program [TRAJT] below, assumes that the matrix
A and the vector x have been stored, then use the program [VtoM] to convert the vector x into an
nx1 matrix. The program creates a trajectory matrix T.
TRAJT
x>T
For¢K:1,15,1)
A*xx>x
augtTsxdI>T
End
x is the first column of T
Repeat the next two lines 15 times
Compute the next point on the trajectory
Store the new point inT
If you want to plot the points of the matrix T then enter the commands
vekFli
verli
The command vc?1i
T¢1)3>x
T¢2)3>¥
Convert the first row of T into a list X
Convert the second row of T into a list Y
is on the second page of the [VECTR]
To plot the data on the TI-85 go to the
menu.
menu and select [EDIT]. Make sure that the
lists that appear in the editor are the X and Y lists that you created, select the
menu
(2nd)
[F3}), select the [CLDRW] option (this clears other plots that may have been created before), and
then select the
option. You may want to change the viewing window so that your plot is
visible. If you have data from another trajectory stored in two other lists, you can plot both
trajectories on the same graph, by plotting first one and then the other. Do not select CLDRW
between your plots. See the TI-85 Graphing Calculator Guidebook for more information about
programming and plotting with the TI-85.
To plot the data on the TI-86 go to the
menu (2nd) f}), and select [PLOT] (LOTT).
Press the
key to activate the plot, using the menu keys, scroll down to Type = and select
[SCAT], continue to scroll down and type in X for Xlist, and Y for Ylist , there are three options for
the Mark, the square is the best one for visibility on the screen. Press the
to return to the
home screen. Press the GRAPH] key and then Gea from the menu. You may want to change
the viewing window so that your plot is visible. If you have data from another trajectory stored in
two other lists, you can plot both trajectories on the same graph, by plotting one in Plot 1 and the
other in Plot 2. See the TI-86 Graphing
programming and plotting with the TI-86.
Calculator
Guidebook
for more
information
about
5.8 Iterative Estimates for Eigenvalues
TI-85/86
Power Method and Inverse Power Method
Set your calculator to display as many decimal places as possible. The algorithms below assume
that A has a strictly dominant eigenvalue, and the initial vector is x, with largest entry 1 (in
magnitude). (If your initial vector is called x9, rename it by entering x®>x.)
Inner Product, Length, and Orthogonality
¢
TI-13
The Power Method When the following steps are executed over and over, the values of x
approach (in many cases) an eigenvector for a strictly dominant eigenvalue (the program
prompts for a vector and returns the entry in the vector with the maximum
absolute value and
stores it in mu.)
A*x?>y
(1)
y7mu>x
Returns mu = estimate for the eigenvalue
(2)
Estimate for the eigenvector
(3)
As these commands are repeated, the numbers that appear are the {, that approach the dominant
eigenvalue. You can program your TI-85/86 to perform this sequence a certain number of times
by using a loop structure. See the TI-85/86 Graphing Calculator Guidebooks for more
information about programming with the TI-85/86.
The Inverse Power Method Store the initial estimate for the eigenvalue in the variable a,
and enter the command
the commands
of columns
of A. Then
C-lkx>y
Solves the equation (A — al)y = x
(1)
[ANS] 2ndx-]+a>nu
y7nuUur?x
nu = estimate for the eigenvalue
Estimate for the eigenvector
(2)
(3)
As these commands
described in the text.
A-~a*ident
are repeated,
nC, where n is the number
lines (2) and
(3) produce
the sequences
{v,} and
enter
{x,}
6.1 Inner Product, Length, and Orthogonality
TI-85/86
dot and norm
The inner product of two vectors is found with the [do] command in the [VECTR]
menu.
The
command in the same menu produces the length of a vector. See the TI-85/86 note for
Section 2.1.
6.2 Orthogonal Sets
TI-85/86
Orthogonality and PROJ
In Exercises 1—9 and 17-22, the fastest way (counting keystrokes) with the TI-85/86 to test a set
such as {u,, u,, u,} for orthogonality is to use a matrix U = [u,
:
~
Theorem 6.
For vectors y and u, the orthogonal projection of y onto u, is:
Cdotty,;uIvdotCuzur
*uU
u,
u,]. See the proof of
TI-14
Notes for the TI-85 and TI-86 Graphing Calculators
that prompts for the vectors and will produce the orthogonal
There is a program
projection of y onto u. It is important that you understand
shortcut to compute the projection.
the formula
before
use the
you
6.3 Orthogonal Projections
TI-85/86
Orthogonal Projections
The orthogonal projection of y onto a single vector was described in the TI-85/86 note for
Section 6.2. The orthogonal projection onto the set spanned by an orthogonal set of nonzero —
vectors is the sum of the one-dimensional projections. Another way to construct this projection is
to normalize the orthogonal vectors, (use the
key in the [VECTR]
menu) place
them in the columns of a matrix U, and use Theorem 10. For instance if {y,, y5} y;} is an
orthogonal set of nonzero
vectors, then the matrix
U =
bins tomy ia
|
norm(y,)
norm(y,)
norm(y,)
can
be created with the sequence:
unitV
yirul
ul
ul is a unit vector in the direction of yl
Convert ul to an nx1 matrix
2nd) [ANS] >U
Store the resulting matrix in U
unitV
y¥2?u2
aug¢€U,u2)9U
unitVU
y3%us
aug¢U,;u3)2U
u2 is a unit vector in the direction of y2
Append
u2 to U
u3 is a unit vector in the direction of y3
Append u3 to U.
The resulting matrix U has orthonormal columns, and U*UT*y
projection of y onto the subspace spanned by {y,, y,, y;}.
produces
the orthogonal
6.4 The Gram-Schmidt Process
TI-85/86
The Gram-Schmidt Process, PROJV, GS, and GSO
If A has only two columns,
following commands:
then the Gram-Schmidt
process
can be implemented
using the
AT>B
B is the transpose of A
BCidrvil
BC2)3x2
x2—-Cdotcx2,vidvdotivisvildd*V1l%eV2
V1 is the first vector
x2 is the second column of A (a vector)
v2 is the second vector
The program
prompts for the vectors v2 and v1; it produces
projection of v2 onto vi. If A has three columns, then add the commands:
the orthogonal
BC3)%x3
xS—CdotCxSsvildr7dottvisvild*eVUI1—Cdot(x3,
v2) 7dotCu2; V2) *U229”uU3
Least-Squares Problems
¢
TI-15
You can continue to use the program [PROJ], although in this situation the Eee
program—which computes the projection of a vector x onto the subspace spanned by a set 0
vectors given as columns of a matrix V—will prove to be more useful. For example,
AT?B
B is the transpose of A
AC1i,1:.m,13%v1
BC2)>x2
v1 is the first column of A (as a matrix)
x2 is the second column of A (as a vector)
Compute the projection of x2 onto V = [v1]
x2
V
x2- [2nd] [ANS ]>v2
augeVU;Vv239U
v2 is the second vector in the process
V = [vl v2]
x3 is the third column of A (as a vector)
Compute the projection of x3 onto
V=[vl_
BC33%x3
PROJV| x3
x3
-
V
2nd [ANS]>v3
Note:
v2]
v3 is the third vector in the process
in this example, v1 is an nx1 matrix, and v2 and v3 are vectors, this is because the
aug operator requires at least one matrix, and the
program takes a vector and a matrix as
data. Recall that if B is a matrix, then B(j) is row j of B, given as a vector, not as a 1xn matrix.
To check your work or save time, you can use the [GS] and
programs to perform the
Gram-Schmidt process on the columns of a given matrix. The
program produces a matrix
whose columns are orthogonal, while the
program produces a matrix with orthonormal
columns. Thus to find Q for a matrix A, use the
program.
6.5 Least-Squares Problems
TI-85/86
Power Method and Inverse Power Method
For Exercises 15 and 16, see the Numerical Note on page 410 in the text. For Exercise 26, the
command
augcALT,A2TIT
creates a (partitioned) matrix whose top block is Al and the bottom block is A2. This command
works as long as Al and A2 have the same number of columns.
7.1 Diagonalization of Symmetric Matrices
TI-85/86
The
Orthogonal Diagonalization
and|eigVq keys orthogonally diagonalize any symmetric matrix A, but you miss the
opportunity to learn the procedure of this section. Use
to find the eigenvalues of the matrix
and the
program to obtain eigenvectors, as in Section 5.3. If you encounter a twodimensional eigenspace with a basis {v,, v,}, use the command nd] [ANS1V to save the
matrix of eigenvectors in V. Then use the sequence
VUcilsz1sm-21>>v1
VC1>22m22)9V2
V2—-CCu2TeulLIsCultevidoeuleu2
v1 is the first eigenvector (as a matrix)
v2 is the second eigenvector (as a matrix)
TI-16
Notes for the TI-85 and TI-86 Graphing Calculators
or the sequence
UT2U
U is a matrix that has the eigenvectors as rows
UC1ld%v1
v1 is the first eigenvector (as a vector)
UC23%v2
V2-Cdottu2,v1ld
v2 is the second eigenvector (as a vector)
7dotivulsvlII*V1IxV2
or the sequence
UT2U
Ucidrvl
UC257%V2
v2 vi
U is a matrix that has the eigenvectors as rows
v1 is the first eigenvector (as a vector)
v2 is the second eigenvector (as a vector)
Returns the projection of v2 onto v1 (v1 and v2 vectors)
2nd) [ANS]>x
Store the answer in x
V2-x>V2
v2 is the new eigenvector
to make the new eigenvector v2 orthogonal to v1. (Review Section 6.2.) After you normalize the
eigenvectors and create P, check that P’P = I to verify that P is indeed an orthogonal matrix.
7.4 The Singular Value Decomposition
TI-85/86
The Singular Value Decomposition
The commands ei9V1
ATAXEV and eigVc
ATAP produce the list of eigenvalues and an
orthogonal matrix P of eigenvectors of the matrix A‘A, but the eigenvalues may not be in
decreasing order. In such a case, you will have to rearrange things to form V and > (denoted
below by S).
For instance, if P is 3x3 the commands
PT >M
rSwaptM,>;2:32°9M
MT >P
interchange columns 2 and 3 of P to form V. The commands
Bee,
Fill
¢0;S)
[YIEVC399S¢2,25
produce a zero matrix for "i" the same size as A, and place the square root of the 3rd element in
the list of eigenvalues into the (2, 2)-entry of S. (The
key is in the [MATRX]
menu, its
purpose is to fill a matrix with a specified value). Other diagonal entries for S can be entered
similarly. To form U for the SVD, normalize the nonzero columns of A*V. If U needs more
columns, use the method of Example 4.
Applications to Image Processing and Statistics
°¢
TI-17
7.5 Applications to Image Processing and Statistics
TI-85/86
Computing Principal Components
The
program takes a stored matrix X as input, and produces a vector whose entries list
the averages of the rows of X. Use the
program with this vector as input to create a
diagonal matrix with diagonal entries equal to the row averages of X. Multiply this diagonal
matrix on the right by a matrix of all ones to construct a matrix A whose size is the same as that
of X, and whose colunins are all the same. Each column of A lists the row averages of X. To
create a matrix of all ones of the appropriate size, make a copy of the matrix X, call it OS, and use
the [Fill] key in the [MATRX]
menu. (Fill¢1,0S>
fills the matrix OS with ones.)
To convert the data in X into mean-deviation form, use
X-A2B
The sample covariance matrix S is produced by
C17 CN-1)99*B*BT9S
INDEX of TI-85 and TI-86 Commands and Operations
STO TI-4
A TI-6
x7! TI-6
' TI-6
Built-in TI85/86 Commands and Menus
aug TI-3, 6, 14-15
CLDRAW TI-12
cond TI-6, 8
CPLX TI-11
det TI-8
dim TI-5
dot TI-6, 13, 14, 16
DRAW TI-12
EDIT TI-1, 12
eigVc TI-11, 15-16
eigV1 TI-11, 15-16
Fill TI-16, 17
*Frac TI-10-11
GRAPH TI-12
ident TI-6, 10, 11, 13
imag TI-11
litve TI-11
LU TI-7
MATH TI-6, 8, 10, 11, 13-14
MATRX TI-1- 6, 8-11, 16-17
mRAdd TI-2
multR TI-2
NAMES TI-4
norm TI-13
POLY TI-10
OPS TI-3- 6, 8- 10, 12 , 16-17
TI-18
PLOT TI-12
PRGM TI-4, 10
rAdd TI-2
randM TI-6, 9
real TI-11
ref TI-8-10
rref TI-8-10
rSwap TI-2-4, 8
SCAT TI-12
SOLVE TI-10
unitV TI-14
verli TI-12
VECTR TI-1-2, 6, 12-14
Programs Downloaded from Web/CD
BGAUS TI-4-6
CHAR TI-11
COL TI-5
GAUS TI-4-9
GS TI-14-15
GSO TI-14-15
MdiaM TI-7
MdiaV TI-7
MtoV TI-2
NULB TI-10-11, 15
PRDCT TI-8
PROJ TI-13-16
PROJV TI-14-15
RMEAN TI-17
TRAJT TI-12
VAMX TI-13
VdiaM TI-11, 17
VtoM TI-2, 4, 12, 14
Notes for the HP-48G Graphics Calculator
Section 1.1 — Systems of Linear Equations
HP-48G
Row Operations
Row operations on a matrix A are performed by the following keys, which are found in the
MATH | |MATR
menu:
Swaps rows,
scales a row by a non-zero constant,
and |RCIJ|
performs a row replacement operation. The [RSWP key is on the second page of
the menu.
helpful.
These keys are used as follows; you may also find the User’s Guide (p. 14-19)
RSWP: With the matrix on level 1, enter the numbers of the rows you want swapped.
example, 1
2
For
will swap rows 1 and 2.
RCI: With the matrix on level 1, enter the constant c by which you want to multiply row
2. Next enter the row number 7. For example, 5 |ENTER| 1 |ENTER
will multiply
row 1 by 5.
RCIJ: With the matrix on level 1, first enter the constant c by which you want to multiply
row number 7. Next enter the row number 27 of the row you wish to multiply, then
enter the row number j of the row to which you want to add c times row 7. For
example, 5
1
2
will add 5 times row 1 to row 2.
The new matrix will now be on level 1 of the stack; if you wish to keep a copy of it for
later use, simply press [ENTER |;a copy of it will then appear on level 2. If you then perform
a row operation that you don’t like for some reason, simply use
(the purple backspace
key) to remove it from the stack. The old matrix on level 2 will now move down to level
1, ready for your next operation. Make sure to recopy it using
before proceeding.
More permanent storage can be achieved using the
key; see the User’s Guide (p. 5-11)
for more information.
Note:
need in the
For the simple problems in this section and the next, the multiple c you will
and
can compute in your head.
commands will usually be a small integer or fraction that you
In general, c may not be so easy to compute mentally.
The
paragraphs that follow describe a simple way to write c in terms of the entries in A.
The (2,7) entry in A is denoted by the algebraic expression ‘A(i,j)’. To use this
expression in calculations, it must be surrounded by single quotes, and the matrix A must
be stored as a variable in your current directory. You can use this expression to help you
row reduce matrices.
For instance, if you want to scale row 7 of A to change the value of A(i,k) to 1, you
and |EVAL|. The proper scaling factor should
can enter A, then ‘A(i,k)’, then press
now be on level 1. Finally, enter the row number i and press |RCI|.
If you want to use a pivot entry A(i,j) to change A(k,j) to 0, you can enter A, then
S(O
ay p aaa
VGlop Bg [=]
EVAL | to produce the proper factor. Then enter 7, then k,
then press |RCIJ |.
HP-1
HP-2
Notes for the HP-48G Graphics Calculator
Section 1.3 — Vector Equations
HP-48G
Constructing a Matrix
To create the matrix A =
|a
ag
a3
b |;do the following steps.
First enter ay as a
vector. This can be confusing — you must enter it as a row. Open one pair of brackets
with the
(purple
key, then enter the entries of the vector from top to bottom as
the cursor proceeds from left to right. Separate the entries with a|SPC|; when you have
completed typing the entries, press
|ENTER|. Enter ag, a3, and b onto the stack in similar
fashion. You then enter the number of columns (4 in this case), and press |
COL |,which is
found in the
menu.
To append the column vector b to the matrix A, thus forming
A b IFfirst place A
on the stack, then enter b as described above. You then enter the number of the column in
A which you wish b to become, and press the
key in the
Consult your User’s Guide (pp.14-3, 14-5) for more details.
Exercises 11-14 and 25-28 can be solved using the [RSWP |,[RCT |, and
menu.
|RCIJ|
keys de-
scribed in the HP-48G Note for Section 1.1.
Section 1.4 —- The Matrix Equation Ax = b
HP-48G
and
To solve Ax = b, row reduce the matrix |A b ikwhich you can create by the method
outlined in the HP-48G Note for Section 1.3. Recall that you enter column vectors as rows;
thus the vector
1
x= | 2 | is enteredas [1 2
3
Eb
To multiply a matrix A by a vector x place A on the stack, then place x on the stack and
press [x]. The number of entries in x must match the number of columns in A. You should
interpret the result as a column vector.
To speed up row reduction of the augmented matrix M = [ A | b ], the
key
in your
directory may be used. Place the matrix M on the stack, then enter the
number of the row you wish to use. The
program will now use the leading entry
in the given row of M as a pivot, and use row replacements to create zeroes in the pivot
column below this pivot entry. The result is returned to the stack. For the backward phase
of row reduction, use the
key which is also in your
directory. The key works
exactly as the
key, except that the program creates zeroes in the pivot column above
the pivot entry. You may then use the
key to create 1’s in the pivot positions. The
directory which contains the
and
programs is a subdirectory of the
directory [LALG]. The
directory is available from your instructor.
15
©
Solution Sets of Linear Systems
HP-3
Section 1.5 — Solution Sets of Linear Systems
HP-48G
To create an m x n matrix of zeroes, enter the list { m n }, then 0, then press
the
key in the
menu. To create a vector containing m zeroes,
proceed as above, except use the list { m }. When solving the equation Ax = 0, where A is
an m Xn matrix, you can create the matrix
A 0 by entering A, then creating a vector
of m zeroes by the above method.
Finally use the
key in the
menu (described in the HP-48G Note for Section 1.3) to append the vector onto the matrix.
You can then use the [Gaus],
and
keys to row reduce
A
0
completely.
Section 1.9 — Linear Models in Business, Science, and Engineering
HP-48G
Generating a Sequence
To generate the sequence x1, X2,..., enter and store the matrix M. You can then enter the
vector Xg onto the stack, and press
to copy it. Press
if necessary to produce a
list of your variables, then press the menu key labelled [m]. The series of commands
will compute x, and copy it onto the stack. Repeating this process will yield
the sequence of vectors in order on your stack; the final vector in the sequence will appear
twice. The stack will extend upwards as long as the calculator has memory to hold it;
you shouldn’t worry about exhausting your calculator’s memory with the exercises in this
section.
Numbers are entered into the HP-48G without commas. The number 6,000,000,000,000
in HP-48G scientific notation is 6.E12. A small number such as .0000000000012 is 1.2E-12.
Section 2.1 — Matrix Operations
HP-48G
Matrix Notation and Operations
To create a matrix, you may use the MatrixWriter; see Chapter 8 of your User’s Guide for
more information. You may also use the command line to enter a matrix. For example, the
keystrokes
[ta] [3] 1 [Sec] 2 [Sec] 3 [B] 4 [Src] 5 [SPC] 6 [EnTER]
will create the 2 x 3 matrix
eas
4
FisGel
|
If A is an m X n matrix, you can find its size by placing A on level 1 of the stack and
pressing the
key in the
menu. The list { m n } will be returned
to the stack. As was noted in Section 1.5, the (7,7) element in the matrix A is ‘A(i,j)’.
keys to denote matrix addition, subtraction, and
The HP-48G uses the [+], [=] and
key will not operate on matrices, but the
multiplication, respectively. Note that the
HP-4
Notes for the HP-48G Graphics Calculator
key will. You can produce the transpose of a matrix by using the
MATR|
|MAKE| menu.
key in the
To compute the inner product of two vectors, place them at levels 1
and 2 of the stack and use the
key in
VECTR| menu. In order to take the outer
product uv? of two vectors, you must enter them as n x 1 matrices, and use the matrix
and [x]. For complex vectors or matrices, consult your User’s Guide.
commands
Section 2.2 — The Inverse of a Matrix
HP-48G ___ To produce the n x n identity matrix, enter n and press the
MATR||MAKE|
means of the
menu.
key in the
You may augment the matrix A with an identity matrix by
command mentioned in Section 1.3. Use the |GAUS |, |BGAU|
and
keys to row reduce
A I |.
There are other keys that can be used to row reduce matrices, invert matrices, and solve
equations Ax = b, but they are not discussed here because they will not help you learn the
concepts in this section.
Section 2.3 — Characterizations of Invertible Matrices
HP-48G
and
Determining whether a specific numerical matrix is invertible is not always a simple matter.
A fast and fairly reliable method is to enter the matrix onto the stack and press }1/a', which
computes the inverse of the matrix. An error message is given if the calculator finds that
the matrix is not invertible.
Another method to test invertibility is to enter the matrix and press the |RANK| key
located on the second page of the
MATR
menu. Section 4.6 discusses rank
and shows that an n x n matrix is invertible if and only if rank A = n. The HP-48G
program requires more calculations than 1 /a|, but is more reliable when the matrix is
nearly singular.
Section 2.4 — Partitioned Matrices
HP-48G _ Partitioned
You may use the
MATR
Matrices
key in the
MATR
menu and the
key in the
menu to append matrices to each other, thus creating partitioned matrices.
Consult your User’s Guide (p. 14-5) for more details.
Section 2.5 — Matrix Factorizations
HP-48G
LU Factorization and the [=] Key
Row reduction of A using the
key described in the HP-48G Note for Section 2.2 will
produce the intermediate matrices needed for an LU factorization of A. You can try this
on the matrix in Example 2 of Section 2.5. The matrices in equation (5) on page 136 of the
text are produced by placing A on the stack and keying
2:5in<0
Matrix Factorizations
HP-5
This produces a matrix with 0’s below the first pivot
This produces a matrix with 0’s below pivots 1 and 2
This produces the echelon form of the matrix
You can copy the
information
from your screen onto your paper, and divide by the
pivot entries to produce L as in the text. (For most text exercises, the pivots are integers
and so are displayed accurately.) The
key in the
menu produces
the ingredients for a permuted LU factorization of a square matrix A, but does not handle
the general case. The calculator produces three matrices on the stack. On level 1 you will
find a matrix P, on level 2 an upper triangular matrix U, and on level 3 a lower triangular
matrix L. Notice that in this case the ones lie on the diagonal of U, not L. These three
matrices satisfy the identity PA = LU, or
A= P-!LU. The matrix P~!L is a permuted
lower triangular matrix, so the factorization A = (P~!L)U is a permuted LU factorization
of A. As the algorithm used by the calculator differs from that in your text, you should not
expect your permuted LU factorization to agree with that of the calculator.
When A is invertible, the best way to solve Ax = b is to use the [=] key. Enter b onto
the stack (as a vector), then enter A. Pressing [=] now will cause x to be produced, again
as a vector. The HP-48G performs a permuted LU factorization on A and uses the matrices
Py) LeandjUstorinds As \=Us! Lal, then.xj=sA 4baThe [=| operation uses 15-digit
internal precision, which provides for a more accurate answer than would be obtained by
calculating A~! by using the
operation. The
operation also uses a permuted LU
decomposition, but does not carry as great an internal precision.
Section 2.6 — Iterative Solutions of Linear Systems
HP-48G
Jacobi and Gauss-Seidel Methods
The |DIAG | key in the second page of the
MATR| menu produces a vector that lists
the diagonal entries of a given matrix. To create an n x n diagonal matrix that has the
same diagonal entries as the matrix A, place A on the stack and press
|—>DIAG|, then enter
the number n, and press the |DIAG— | key, which is also on the second page of the
iy
The Jacobi and the Gauss-Seidel methods use the same commands, except for the
construction of M. Store M, N, and b, and place your initial data vector xo on the stack.
To produce the “next” x, you must solve the equation
Mx = Nxo + b for x. The correct
way to do this is to use the HP-48G’s division key: compute Nxo + b and place it on the
M and press [=} This operation will produce x on level 1 of the stack. If
you repeat this process over and over, you will create the sequence x(4). The vectors may
be recorded on paper as you calculate them.
If you wish to monitor how close the approximations are to each other, you can store
stack, then enter
the approximations on the stack and subtract them when appropriate.
HP-6
Notes for the HP-48G Graphics Calculator
Section 2.9 — Vectors in R”
HP-48G
and
By now, the row reduction algorithm should be second nature, so now it’s time to cut to the
menu to a matrix A produces
key in the
chase. Applying the
be able to write a basis
immediately
will
you
that
From
A.
of
form
echelon
row
the reduced
for Col A and to write the homogeneous equations that describe Nul A. Don’t forget that
A is a coefficient matrix, not an augmented matrix.
key to check the rank of A, but roundoff error or small pivot
You can use the
entries can produce an incorrect reduced row echelon form. A more reliable strategy is to
menu.
key is located on the second page of the
use [RANK |. The
problems
avoid
helps
This
0.
to
matrix
a
in
elements
By default the HP-48G sets all “tiny”
with roundoff error in calculations, but can also generate unexpected results. See the User’s
Guide (pp. 14-9, 14-20, D-5) for more information.
Section 3.2 — Properties of Determinants
HP-48G
Computing Determinants
To compute det A, place A on the stack and then repeatedly use the
and
keys
as needed to reduce A to a matrix U which is in echelon form. (See the HP-48G Note for
Section 2.2.) Keep track of how many times you swap rows. Then except for a +1, the
determinant of A can be found by placing U on the stack and executing the keystrokes
the
IPROD|.The
key is found on the second page of the
menu;
key is found in the
directory. The
key extracts the diagonal
entries from U and places them in a vector, and the
those entries. You can, of course, use the
key computes the product of
key (found on the second page of the
MATR
menu) to check your work, but the longer sequence of commands helps you
to think about the process of computing det A.
Section 4.3 — Linearly Independent Sets; Bases
HP-48G
and
Applying the
key in the
menu to a matrix A produces the
reduced row echelon form of A. From that you will immediately be able to write a basis for
ColA and to write the homogeneous equations that describe NulA. Don’t forget that A is
a coefficient matrix, not an augmented matrix.
For Exercise 34: To form the necessary coefficient matrix in this case, you can first
produce each column then use the
key (See the HP-48G Note for Section 1.3 or
the User’s Guide p. 14-3). To produce each column, you will want to apply the function
cos*(t) to each element in a vector. To do this, enter the vector on the stack enclosed not
by brackets, but by set braces ({ and }). This is an example of what the HP-48G
calls a list. You may operate on lists just as you do on numbers, so pressing
k
will apply the function cos*(t) to each element in the list. To change this list into a vector,
4.3
press
e
Linearly Independent Sets; Bases
[—-arR |. These keys are found in the
HP-7
menu. For more on lists,
see Chapter 17 of the User’s Guide.
Section 4.4 — Coordinate Systems
HP-48G
The Divsion Key [=]
If the equation Ax = b has a unique solution and A is a square matrix, you may calculate
the solution x by entering first b then A and pressing [=} In this section, the equation will
probably have the form Pu = x.
This command actually computes A~'b, but uses 15-digit internal precision. This
provides a more precise result than computing A~'b by inverting A and multiplying. If A
is not invertible, the HP-48G will give you an “Infinite Result” error. When A is either not
invertible or not square, but Ax = b still has a solution, you may find a solution by entering
b and A onto the stack, then pressing the
key in the
MATR| menu. This action
produces a least-squares solution to the system (see Section 6.5).
Section 4.6 — Rank
HP-48G
and
In this course you may use either the |RREF| or the |RANK| key to check the rank of a
matrix. In practical work the |RANK| key should be used, since this key uses a more reliable
algorithm. This algorithm is based on the singular value decomposition (see Section 7.4).
Section 4.7 — Change of Basis
HP-48G
= The
key will completely row reduce the matrix
Ci
Co
by
be
to
the desired form.
Section 4.8 — Applications to Difference Equations
HP-48G __ To solve a polynomial, use the Solve poly option in the |SOLVE}
application.
A vector of roots will be returned to the stack. Refer to your User’s Guide (p. 18-10) for
more information.
Section 4.9 — Applications to Markov Chains
HP-48G
The HP-48G
homework here.
Note for Section
1.9 contains information
that is useful for
HP-8
Notes for the HP-48G Graphics Calculator
Section 5.2 — Eigenvectors and Bigenvalues
HP-48G __—sw Finding Ejigenvectors
Your |TBOX |directory has a program
that will simplify your homework by automatFor example, if A is a 3 x 3 matrix with an
eigenspace.
an
for
basis
a
producing
ically
eigenvalue 7, first place A on level 1 of the stack, then use the keystrokes
3 iw] 7 KE]
to produce the matrix
A—7J. Then pressing the
key will produce a set of vectors on
the stack which forms a basis for the eigenspace for A corresponding to A = 7. In general,
Ki
produces the k x k identity matrix, and C
produces a set of vectors which
is a basis for NulC.
Although your HP-48G has more powerful commands for eigenvalue calculations, you
should not use them yet, because you need to learn basic concepts about eigenvalues and
eigenvectors.
Section 5.2 —- The Characteristic Equation
HP-48G
—_ You can use the |CHAR|
key in the |TBOX|
directory to check your answers in
Exercises 9-14. Note that if A is n x n, pressing this key with A at level 1 produces a vector
listing the coefficients of the characteristic polynomial of A, in order of decreasing powers
of A, beginning with A”. If the polynomial is of odd degree, the coefficients are multiplied
by —1, to make +1 the coefficient of A”. This corresponds to finding the determinant of
AI — A.
Section 5.3 — Diagonalization
HP-48G
The
and
key on the second page of the [MTH |
menu produces a vector containing
the eigenvalues of the matrix on level 1 of the stack, which we will call A. To learn the
diagonalization procedure, you should use the method of Section 6.1 to produce the eigenvectors. For example, if A is 5 x 5 with eigenvalue A, and is at level 1 of the stack, the series
of operations
» Gi]
KE
will produce a basis for the eigenspace corresponding to the eigenvalue A.
In later work, you can automate the diagonalization process. The
key in the
menu also produces a vector containing the eigenvalues of A; in addition, it produces
a matrix P on level 2 of the stack such that AP = PD, where D is a diagonal matrix
created from the vector of eigenvalues. If A happens to be diagonalizable, then P will be
invertible. In any case P is likely to be quite different from what you construct for your
homework.
5.5
Complex Eigenvalues
HP-9
Section 5.5 — Complex Eigenvalues
HP-48G
Complex Eigenvalues
The
and
keys mentioned in the HP-48G Note for Section 5.3 also work for
matrices with complex eigenvalues.
For a matrix on level 1 of the stack, the [RE |and [1] keys in the
menu produce
the real and the imaginary parts of the entries in a matrix, displayed as matrices the same
size as the given matrix. The
menu is located on the second page of the
menu.
Section 5.6 — Applications to Dynamical Systems
HP-48G
Plotting Trajectories
Given a vector x, you may compute the product Ax by placing x on the stack, then placing
A on the stack, pressing [SWAP |,then 2! The product vector will be left on the stack, and
you may repeat the above procedure over and over if you wish.
The following program creates a “trajectory” matrix whose rows are the points x, Ax,
A*x, ..., Al®x. (Change 15 to any number you wish.) This program asssumes that the
matrix A is on level 2 of the stack and the vector v is on level 3 of the stack when the
program is run from level 1 of the stack.
<3
VA
Inputs data.
< V DUP
Places v on stack.
1,15 STARL
This loop repeats the next line 15 times.
A SWAP
* DUP
DEPTH
ROW->
NEXT
DROP
Computes the next point on the trajectory.
End of the loop.
Assembles points into the matrix.
>
>
If you place this program on level 1 of the stack and press [ENTER |, the result is the
trajectory matrix. If you intend to use the program more than once, store it as a variable.
If you want to plot the points in the output matrix, store the matrix under some name
and enter the PLOT utility. Choose the plot type “Scatter” in the window labelled TYPE
and choose your matrix in the window labelled SDAT. You can also elect to change the
corners of your viewing window at this point, if you so desire. Pressing the ERASE and
DRAW menu keys will produce a graph of the trajectory. If you have the data for another
trajectory stored in another matrix, you can plot both trajectories on the same graph by
plotting first.one and then the other. Do not press ERASE in between your plots. See
Chapters 22, 23, and 29 in your User’s Guide for more information about programming and
plotting with the HP-48G.
HP-10
Notes for the HP-48G Graphics Calculator
Section 5.8 — Iterative Estimates for Eigenvalues
HP-48G
Power Method
and Inverse Power Method
The algorithms below assume that A has a strictly dominant eigenvalue, and the initial
vector is x, with largest entry 1 (in magnitude). Store A, then place A and your initial
vector x on the stack. You proceed as follows, using the |VAMX |key in your |TBOX |directory.
The Power Method When the following keystrokes are repeated over and over, the
resulting vectors approach (in many cases) an eigenvector for a strictly dominant eigenvalue:
(1)
[=]
estimate for eigenvalue at level 1
(2)
estimate for the eigenvector
(3)
In (2), the program |VAMX |finds the entry of largest absolute value in the vector Ax. To
repeat the process, recall A to the stack and press
these commands are repeated,
the numbers that appear at level 1 after you press |VAMX| are the yp, that approach the
dominant eigenvalue. You could program your HP-48G to perform this algorithm a certain
number of times by using a loop structure (see the HP-48G Note for Section 5.6).
The Inverse Power Method Store the initial estimate of the eigenvalue in the variable
B, then perform the following keystrokes, where n is the number of columns in A:
[4][B]» [apn] [x] [E}
Store the resulting matrix as C. Place your initial vector x on the stack, followed by C.
You then enter the keystrokes
a
(1)
(2)
[=]
estimate for eigenvalue at level 1
(3)
estimate for the eigenvector
(4)
You may now enter C’ onto the stack and repeat the keystrokes. As you repeat these
keystrokes, the numbers at level 1 after step 3 form the sequence referred to as {v,} in the
text; the vectors at level 1 after step 4 form the sequence {x}.
(os
HP-48G
ee
Inner Product, Length, and Orthogonality
HP-11
Section 6.1 — Inner Product, Length, and Orthogonality
The inner product of two vectors may be found using the
key in the
VECTR| menu. The
key in the same menu produces the length of a vector. See
the HP-48G Note for Section 2.1.
Section 6.2 — Orthogonal Sets
HP-48G
In Exercises 1-9 and 17-22, the quickest way to test a set such as {uj, uz, us}
for orthogonality is to use the matrix
Uz, U2 Us |:See the proof of Theorem 6.
To find the orthogonal projection of y onto u, compute the inner product of y and u
and divide by the inner product of u with itself. Take this number and multiply it by u.
This process is easily done using the stack; you should enter 3 copies of u and one copy of
y, then use the following keystrokes:
[par] [swap] [aps] [2?] [=][x]
There is a key (labelled
in your
directory) which will produce the orthog-
onal projection of y onto u when given y on level 2 of the stack and u on level 1. It is
important that you understand the formula before using this shortcut.
Section 6.3 — Orthogonal Projections
HP-48G
Orthogonal Projections
The orthogonal projection of y onto a single vector was described in the HP-48G Note for
Section 6.2. The orthogenal projection onto the set spanned by an orthogonal set of vectors
is the sum of the one-dimensional projections. Another way to construct this projection
is to normalize the orthogonal vectors, place them in the columns of a matrix U, and use
Theorem 10. That is, the desired projection is UU7y.
Section 6.4 — The Gram-Schmidt Process
HP-48G
The Gram-Schmidt
Process
If A has only two columns, then the Gram-Schmidt process can be implemented using the
following keystrokes. The
key in your
directory will be used to get columns
from A. Store the matrix under the variable A.
[a] 1
‘vi?
[sTO
[a] 2 [ecou] [enTeR] [vi] [por] [vi] [Ewrer] [por] [=] [v1] [xk] []
HD?
STO
If A has three columns, add the keystrokes
HP-12
Notes for the HP-48G Graphics Calculator
Ce] 3 (coc) [EwTeR) [EWTER) [Vi] [DOT] [va] [ewe] or] EE] Wa]
—] Sar] (2) bor] [2] rm] bor] EF] we) &] A
‘V3’
|STO
You should use these keystrokes for awhile, to learn the general procedure. After that,
you can use the |PROJ| key in your |TBOX| directory, which computes the projection of a
vector x onto the subspace spanned by a set of vectors. To use the |PROJ| program, place x
on the stack, then enter the set of vectors one by one onto the stack. If your set of vectors
is the set of columns of a matrix, you may enter the matrix then press
DROP | to
enter the vectors. Pressing the |PROJ| key will produce the projection of the first vector
entered onto the span of the remaining vectors.
To implement the Gram-Schmidt process on a matrix A with three columns using |PROJ |,
you would use the following keystrokes:
[a] 1 [ocoL]
‘v1’
[sto
[a] 2 [ccou] [vi] [Pros] [a] 2 [con] [swap] [=] ‘v2’ [sto]
[&] 3 (Gcoc] [vs] [v2] [PROS] [a] 3 [Bcou] (Sma) ] ve [St]
The set of vectors need not be orthogonal for the |PROJ| program to work, but if they
are, the resulting vector will usually agree with those computed via Theorem 10 in Section
6.3 to ten or more decimal places.
To check your work or save time, you can use the
and
keys in the
directory to perform the Gram-Schmidt process on the columns of a given matrix. The IGS.
key will produce a matrix whose columns are orthogonal, while the
key will produce
a matrix with orthonormal columns. Thus to find Q for a matrix A, use the
key.
Your calculator computes a permuted QR factorization of a matrix A with the |QR|key
in the
menu. This command will produce matrices Q, R, and P at
levels 3, 2, and 1 respectively. The matrix @ is orthogonal, R is upper triangular, and P is
again a permutation matrix (See the HP-48G Note for Section 2.5) such that AP = QR.
Section 6.5 — Least-Squares Problems
HP-48G
The [=] and
Keys
For Exercises 15 and 16, see the Numerical Note in Section 6.5 of the text. You can solve
the equation Ax = b using the QR factorization by solving R& = QTb. Entering Q7b and
R onto the stack and then pressing the [=] key will solve the system. You could also enter b
and A onto the stack directly and press |LSQ|. This key is located in the
MATR |menu.
Compute the solutions to Exercises 15 and 16 using |LSQ|, and compare your answers to
those which you obtained using the QR factorization and [=].
GS
te
Least-Squares Problems
HP-13
For Exercise 26, you can create a (partitioned) matrix whose top block is Al and bottom
block is A2 by placing Al and A2 on the stack, then entering the number of rows in Al plus
1. Now pressing the
key in the
menu will produce the desired
matrix.
Section 7.1 — Diagonalization of Symmetric Matrices
HP-48G
Orthogonal Diagonalization
You can use the |EGVL|
and
keys to orthogonally diagonalize a matrix by the pro-
cedure of this section. You can obtain eigenvectors as in Section 5.3; if you encounter a
two-dimensional eigenspace with a basis {u,, ug}, replace ug with a new eigenvector v2
orthogonal to u;. (Review Section 6.2.) You can use the
key introduced in the HP-
48G Note for Section 6.4 to help with this calculation. After you normalize these vectors
and create P, you can check that P is orthogonal by confirming that P? P= J.
HP-48G
Applying the
Section 7.4 — The Singular Value Decomposition
The Singular Value Decomposition
operation on the matrix A‘A will produce a matrix of eigenvectors and a
vector of eigenvalues for A’A, but there are two problems. First, the matrix of eigenvectors
may not be orthogonal. The procedure in the HP-48G Note for Section 7.1 can help you to
produce an orthogonal matrix of eigenvectors. Second, the eigenvalues in the vector may
not be in decreasing order. In such a case you will have to rearrange things things to form V
and &. The
key in the
menu allows you to swap columns just as
allows you to swap rows. To form U for the singular value decomposition, normalize
the nonzero columns of AV. If U needs more columns, use the method of Example 4.
After you thoroughly understand the singular value decomposition, you will want to
use the much faster and more numerically reliable
key. This key is found in the
MATR| |[FACTR| menu. If you place the matrix A on level 1 of the stack and press |SVD|, the
result will be the matrix U on level 3, the matrix V on level 2, and a vector of singular
values on level 1. You can create the matrix 4 from this vector by using the
key
(described in the HP-48G Note for Section 2.6), then appending rows and/or columns of
zeros if necessary.
Section 7.5 — Applications to Image Processing and Statistics
Computing Principal Components
HP-48G
The RMEAN menu key (abbreviated |RMEA|) in the
directory takes a matrix X on
level 1 of the stack and produces a vector whose entries list the averages of the rows of
X. With this vector you may create a diagonal matrix whose diagonal entries are the row
MATR}| menu. See the HP-48G Note
averages of X by using the |DIAG—| key in the
for Section 2.6 for more information. Finally multiplying this diagonal matrix on the right
HP-14
Notes for the HP-48G Graphics Calculator
by a matrix of all ones creates a matrix A which is the size of X , whose columns are all the
same: each column of A lists the row averages of X. To create a matrix of all ones of the
appropriate size, place a copy of X on the stack, enter a 1, then press the
key in the
menu.
To convert the data in X into mean-deviation form, find the matrix A above, then use
B=X-
A.
The sample covariance matrix S is produced by the formula
$=
1
T
BB",
where WN is the number of columns in B.
The principal component data you need is produced by using the
MATR]|
|FACTR|
key in the
menu. If you place the matrix
BT
N-1
on level 1 of the stack and press |SVD|,the result will be the matrix U on level 3, the matrix
V on level 2, and a vector of singular values on level 1. The columns of V are the principal
components of the data, and the squares of the singular values list the variances of the new
variates.
INDEX OF HP-48G COMMANDS
Symbols
Brackets ([]), HP-2, HP-3
Division Sign (+), HP-5, HP-7, HP-12
Minus Sign (—), HP-3
Multiplication Sign (x), HP-3
Plus Sign (+), HP-3
Quotes (‘’), HP-1, HP-3
Set Braces ({}), HP-6
Built-In HP-48G
Commands and Menus
ABS, HP-11
— ARR, HP-7
COL—, HP-2, HP-6
COL+, HP-2, HP-3, HP-4
COL, HP-12
CON, HP-3, HP-14
CSWP, HP-13
DET, HP-6
DIAG—>, HP-5, HP-13
—sDIAG, HP-5, HP-6
DOT, HP-4, HP-11, HP-12
DROP, HP-1, HP-12
EGV, HP-8, HP-13
EGVL, HP-8, HP-13
EVAL, HP-1
IDN, HP-4, HP-8
IM, HP-9
LSQ, HP-7, HP-12
LU, HP-5
MTH CMPL menu, HP-9
MTH MATR menu, HP-5, HP-6,
HP-7, HP-8, HP-12
MTH
MATR
COL menu, HP-2, HP-3,
HP-4, HP-13
MTH MATR FACTR menu, HP-5, HP-6,
HP-12, HP-13, HP-14
MTH MATR MAKE menu, HP-3, HP-4,
HP-14
MTH MATR NORM menu, HP-4, HP-6
MTH MATR ROW menu, HP-1, HP-4,
HP-13
MTH VECTR menu, HP-4, HP-11
OBJ-—>, HP-7
PLOT application, HP-9
PRG LIST ELEM menu, HP-3
PRG TYPE menu, HP-7
RANK, HP-4, HP-6, HP-7
RCI, HP-1, HP-2, HP-3, HP=4
RCIJ, HP-1, HP-2
RE, HP-9
Reciprocal key (1/2), HP-1, HP-4
ROW+, HP-4, HP-13
RREF, HP-6, HP-7
RSWP, HP-1, HP-2, HP-6
SIZE, HP-3
SOLVE application, HP-7
STO, HP-1
SVD, HP-13, HP-14
SWAP, HP-3
TRN, HP-4
VAR, HP-3
z*, HP=4
Programs Downloaded from Web/CD
BGAU, HP-2, HP-3, HP-4
CHAR, HP-8
GAUS, HP-2, HP-3, HP-4, HP-6
GCOL, HP-11, HP-12
GS, HP-12
GS.0, HP-12
LALG directory, HP-2
NULB, HP-8, HP-13
PROD, HP-6
PROJ, HP-11, HP-12, HP-13
RMEA, HP-13
TBOX directory, HP-2, HP-8, HP-10,
HP-11, HP-13
VAMX, HP-10
.
‘ &:
bye
ie
Bettie
by ng
af .eb
' m
4 a;
igen
ry
f
G Oe ~
A
hk atin
meos “th |+ eee int7 _oe
¥ eres
i
mit
bys!
,
enw
Oy i
whan JAMA w
dre ofmen
tiseey apiogsps
ayafaH“ious TDA ery
om
it) 2
ty fee
SCGH det
Heviatk
ouorn,
te
oats Becities TOMA. x Ay 7 ae |
§-4H wooo MAIS Teld Dat
|
ETH
The akople thy
a
avpraAgs| by Beeerrs
1TH .O-TH 21H AMAR r
|
b-TH 2TH ea ASH FTO
See
VG the number of¢lange tha.
fat Dep etet
;
aN ba abrsecnuoc? §
SH STH ATOR
wher
fy on
Sad
dad
SS
_-
TSH,
ss
SES
AN ey Yaar IoFq REAPS is (Htoclucet by sing
the ep deme 8
TR) {RAGIN coring Pepe wheres caapygtin |
TH
TH8
OSH aaa.
6-48 ASE WOR seep :
TH
AEA
)
eH asia!”
am level Lof the}ll hy.vl
Y ou leee)'2. abd @ vector o
chdiGomerts
IDE anit wiltbeihe it
SDE Abs, on tovel 1, ‘The colnet ett
«¢ Clann, ‘bok Sid ryeShin QV!
arate,
|
£SH
Awe
wT
RAY
thw saciias walvesr tes, Oeil
.
DTH MAT
ah, qth UTE,
bFEsy
STH AAHD
TT
iM
=
ae
|
“Tht SLR
OTH
tt
Sh
x
,apRa
LR
opnd |!
CEQA AMMA
olvetih 2OF
LAD 1-9
qi
TAH pay
es
eS:
om
-*
Cee
OH
net I BTM
OTH 3-4 \onste RYAN HOM
|
“oll
4H Sah ator JOD ATARETM
. Sy
|
es
aa
ee
ec: |) ee
LEE
et
|
.o- “uyhanes SDA RAM, HTM
AN-TE RTH Sa
»-9H EU
soon BEN ARAM STH,\\
on
YMAY
AVS
RAH
2-9
S-FA wnotosilh DIAL
£f-4H BIH aun
hr
a, Ds
ba mw 24M war
- OAH ETH 8-H STH 20a
Ct-SH 11-4 Jodo
|
Sf-4 2p
S14 O18
a
fl-4TH 8-4 WOd = =
|
t-TH fH .S-TH Wana
y
toe “a
£19
MT \doW noon bahsolewod snmrgord
A
WAS
a4
~ 1H
wrote MA
pil
OH A
|
SS
i
a
Ge
a
oS
“LINEAR ALGEBRA
~ANDATS APPLICATIONS
Study Guide
Second Edition
With this Study ORE you can dramatically improve performance and understanding in your linear
algebra course. This Guide can help you learn new material, review and_organize old material, and
prepare effectively for tests. Careful explanations and tips lead you through hundreds of exercises, at
your own pace; difficult concepts are anticipated, and help is provided where you need it most.
e Key Ideas and Study Notes highlight important facts and guide you through each section with
summaries and tables that connect related ideas.
See pages 1-23, 2-1, 2-9, 4-20, 6-14.
e Detailed solutions to every third, odd exercise allow you to check your work or help you get started
on a difficult problem. Also, complete explanations are provided for each writing exercise whose
answer in the text is only a "Hint". See pages 1-4, 2-8, 3-6, 4-19.
e Study Tips point out key exercises, give hints about how and what to study, and sometimes highlight
potential exam questions. See pages 1-20, 2-12, 3-7.
° Frequent Warnings identify common errors. See pages 2-7, 5-11, 6-12.
e Updated support for technology: MATLAB boxes give details on MATLAB commands used to
solve the exercises. These boxes are strategically placed to show you all you need to know, as you
need to know it. po vesbanelne notes
2
are at the back of the Guide for:
Maple, Mathematica, and the TI-85/86 and HP-48G calculators.
e Free Matrix Program files contain custom programs that implement algorithms in the text, along
with data for over 800 of the text's exercises. Available on the CD-ROM in the back of your text or
via the Web site www. laylinalg. com.
Addison
Wesley
Longman
ee
m ‘iN vin
|
ISBN
O-201-bY¥a84?-4