Math 103, Summer 2006 Geometry of Vectors, Matrix Algebra & Linear Operators
July 5, 2006
GEOMETRY OF VECTORS, MATRIX ALGEBRA & LINEAR OPERATORS
1. Announcements
We have a quiz tomorrow (Thursday) on the material covered up to today’s class. I should also have
Homework 2 posted by the end of the day, together with solutions to Homework 1. Finally, we have a midterm
at the end of next week.
2. Recap
Last class period we covered a lot of ground. We formally introduced vectors and matrices (even though we
had been using the latter for a while) and introduced some basic matrix operations. For instance, we talked
about what it meant to add two matrices or to scale a vector by a constant. We said that matrix addition and
scaling satisfied some ‘nice’ properties like distributivity (is that even a word?) and associativity. We also said
that vectors have a nice geometric interpretation as arrows centered at the origin.
We also discussed some more advanced operations on vectors which are special cases of matrix multiplication
(which we’ll see later). First we introduced the dot product of two vectors and said that dot product can be
used to compute the length of a vector. We then said that one can multiply an n × m matrix by a m × 1 column
vector, the result being a n × 1 column vector.
In today’s class I want to revisit some of these concepts and emphasize the geometric aspects of vectors.
We’ll also talk about some algebraic properties of dot product and matrix multiplication. At the end of class
we will talk about linear operators.
3. The Geometry of Vectors
3.1. Sums and scalars in graphical depictions. In class last period we said we can add two vectors together.
Today I want to show what addition ‘looks like,’ which is to say I want to give a geometric interpretation of
addition. We’ll do the same for scaling.
−
→
→
−
Example. Suppose we are given vectors →
u and −
v in Figure 1. What is −
u +→
v?
u
v
→
→
Figure 1. Vectors −
u and −
v
aschultz@stanford.edu
http://math.stanford.edu/~aschultz/summer06/math103
Page 1 of 6
Math 103, Summer 2006 Geometry of Vectors, Matrix Algebra & Linear Operators
July 5, 2006
Solution. To add two vectors geometrically, we simply translate one vector onto the other. More precisely, we
move the base of the second vector onto the tip of the first vector. The sum of the two vectors is then the tip
of the translated second vector.
→
−
u
→
−
u
→
−
−
u +→
v
→
−
−
u +→
v
→
−
v
→
−
v
→
→
The sum −
u +−
v
You can add in either order!
Figure 2.
→
→
−
→
Notice that since −
u +−
v =→
v +−
u , we should be able to translate either vector onto the other to perform
addition. Indeed, both translations produce the same sum, as we see in the second illustration above.
¤
−
→
→
Example. What is 2→
u or −−
u , where −
u is as above?
→
→
→
→
Solution. We can compute 2−
u in two ways. The first would be to recognize that 2−
u =−
u +−
u , after which we
−
→
→
−
could apply the reasoning above. The second is to notice that 2 u is just u stretched by a factor of 2. Hence
→
→
we can just draw a vector twice the length of −
u , but in the same direction. The vector −−
u is drawn as a vector
−
→
in the opposite direction as u but with the same magnitude.
¤
−
2→
u
→
−
u
→
−
u
−
−→
u
→
The vector 2−
u
→
The vector − −
u
Figure 3.
aschultz@stanford.edu
http://math.stanford.edu/~aschultz/summer06/math103
Page 2 of 6
Math 103, Summer 2006 Geometry of Vectors, Matrix Algebra & Linear Operators
July 5, 2006
3.2. Dot Product, again. It turns out that dot product can be defined in a different way than how we defined
it last period. Both are equivalent, but this second one—while harder to use in practice—has very nice geometric
consequences.
→
→
Theorem 3.1. The dot product of two vectors −
u,−
v ∈ Rm is given by
−
→
→
→
→
u ·−
v = k−
u k k−
v k cos (θ) ,
−
→
where θ is the angle between →
u and −
v.
This new definition can be used to verify that the length of a vector is given by the square root of the dot
product of the vector with itself. Indeed this new definition shows us that
−
→
→
→
→
→
v ·−
v = k−
v kk−
v k cos(0) = k−
v k2 ,
and taking square roots gives the result
√
−
→
→
→
v ·−
v = k−
v k.
Dot products can also tell us when two vectors are perpendicular or, in linear algebra speak, orthogonal.
→
−
→
−
Corollary 3.2. Two nonzero vectors −
u and →
v are orthogonal if and only if −
u ·→
v = 0.
Proof. We need to show two things: that orthogonal vectors have dot product zero, and that two vectors with
−
→
0 dot product are orthogonal. For the first statement, if →
u and −
v are orthogonal then the angle between them
◦
is 90 . Hence we have
−
→
→
→
→
u ·−
v = k−
u kk−
v k cos(90◦ ) = 0.
For the other statement, notice that a nonzero vector has nonzero length. Hence if
−
→
→
→
→
u ·−
v = k−
u kk−
v k cos(θ) = 0
→
−
we must have cos(θ) = 0. But cos(θ) = 0 if and only if θ = 90◦ (or 270◦ or −90◦ , etc), and so −
u and →
v are
orthogonal.
¤
What’s important about this result is that it gives you a quick way to determine when two vectors are
perpendicular without having to draw anything. When we move into 18 dimensional spaces (or even in 3
dimensions), this is going to be really handy.
3.3. Span. We want to define what the span of a collections of vectors is, but first we need some preliminary
definitions.
−
→
→
→
Definition 3.1. A vector b ∈ Rm is said to be a linear combination of vectors −
v 1, · · · , −
v s ∈ Rm if there exist
real numbers c1 , · · · , cs so that
−
→
→
→
→
b = c1 −
v 1 + c2 −
v 2 + · · · + cs −
v s.
Here’s a silly example of a vector which is a linear combination.
−
→
→
→
Example. The zero vector 0 ∈ Rm is a linear combination of −
v 1, · · · , −
v s for any collection of vectors
−
→
−
→
v 1 , · · · , v s ∈ Rm . To see this, notice that
→
−
−
→
0 = 0→
v 1 + · · · + 0−
v s.
To see a less silly example, we provide the following
aschultz@stanford.edu
http://math.stanford.edu/~aschultz/summer06/math103
Page 3 of 6
Math 103, Summer 2006 Geometry of Vectors, Matrix Algebra & Linear Operators
July 5, 2006
→
Definition 3.2. The vector −
e i ∈ Rm , called the ith standard basis vector, is that vector whose only nonzero
entry is a 1 in the ith position
 
0
 0 
 
 .. 
 . 
 
 0 
→
−

e i := 
 1  ← ith position .
 
 0 
 
 . 
 .. 
0
→
→
The collection {−
e ,··· ,−
e } ⊂ Rm is called the standard basis of Rm .
1
m
→
Theorem 3.3. Any vectors −
x ∈ Rm is a linear combination of the standard basis of Rm .
→
Proof. For a random −
x ∈ Rm , write



→
−
x =

x1
x2
..
.



.

xm
Then we have



−
→
x = x1 

1
0
..
.
0






 + x2 


0
1
..
.






 + · · · + xm 


0
0
0
..
.





1
→
→
−
= x1 −
e 1 + x2 −
e 2 + · · · + xm →
e m.
−
→
−
→
−
→
Hence we have that x is a linear combination of e 1 , · · · , e m as desired.
¤
−
→
−
→
Definition 3.3. The span of a collection of vectors { v 1 , · · · , v s } is the set of all vectors which are a linear
→
→
combination of −
v ,··· ,−
v .
1
s
−
→
Example. Our previous example tells us that 0 is in the span of any collection of vectors. The previous
−
→
m
theorem tells us that any vectors v ∈ R is in the span of the standard basis of Rm .
4. Matrix Algebra
There are some properties of the kind of matrix multiplication we’ve done so far which are particularly nice.
You’ll use them all the time when you’re computing, even though you won’t realize you’re using them.
4.1. Linear Properties of Dot Product.
−
→
→
Theorem 4.1. Let →
u,−
v , and −
w be vectors in Rm , and suppose k is a scalar. Then
→
−
→
→
−
→
→
u · (→
v +−
w) = −
u ·→
v +−
u ·−
w and
• −
−
→
−
→
−
→
−
→
→
−
→
• (k u ) · v = k( u · v ) = u · (k −
v ).
aschultz@stanford.edu
http://math.stanford.edu/~aschultz/summer06/math103
Page 4 of 6
Math 103, Summer 2006 Geometry of Vectors, Matrix Algebra & Linear Operators
July 5, 2006
These properties, together with the alternate definition of dot product we talked about at the beginning of
class, allow us to introduct the following
→
→
Definition 4.1. A unit vector is a vector with length 1. The unit vector in the direction of −
u (−
u 6= 0) is the
vector
1 →
−
u.
→
k−
uk
The second definition implies that the stated vector is unit length. To see this, notice that
s
°
° sµ
¶ µ
¶ s
2
° 1 →
°
1
1
kuk
1
−
→
−
−
→
−
→
−
→
°
°=
u
u
·
u
=
(
u
·
u
)
=
→
→
−
2
2 = 1.
° k−
uk °
k−
uk
k→
uk
kuk
kuk
4.2. Linear Properties of Matrix/vector multiplication. The properties above for dot product have analogues for multiplication of a matrix with a vector.
→
−
Theorem 4.2. Let A be an n × m matrix, −
u and →
v be vectors in Rm , and k be a scalar. Then
→
−
→
→
• A(−
u +→
v ) = A−
u + A−
v and
→
−
→
−
→
• (kA) u = k(A u ) = A(k −
u ).
→
4.3. Reconstructing a matrix from A−
e i . Suppose that I have a n × m matrix A, but I refuse to tell you
→
what the matrix is. I will, however, tell you the product A−
x for any vector x ∈ Rm that you like. Can you use
this information to reconstruct my secret matrix A? It turns out that you can.
Theorem 4.3. If A is an n × m matrix, say

→
 −
c1

−
→
c2
···
→
−c
.
m
→
→
Then A−
ei=−
c i.


x1


−
→
Proof. Recall that for a random vector →
x =  ...  ∈ Rm , we compute A−
x as
xm
−
−c + · · · + x −
→
A→
x = x1 →
1
m c m.
−
Since the only nonzero entry of →
e i is a 1 in the ith position, this means
→
−c + · · · + 0−
→
→
→
→
−c .
A−
e i = 0→
c i−1 + 1−
c i + 0−
c i+1 + · · · + 0−
em=→
1
i
¤
This fact will come in very handy later, so don’t forget it!
aschultz@stanford.edu
http://math.stanford.edu/~aschultz/summer06/math103
Page 5 of 6
Math 103, Summer 2006 Geometry of Vectors, Matrix Algebra & Linear Operators
July 5, 2006
5. Linear Operators
We’re now going to introduce linear operators. Linear operators will occupy our discussions for the rest of
the term, but happily they are a concept we have already been studying for a while. First, let’s review what a
function is.
Definition 5.1. A function f : D → C is a rule that assigns to each input d ∈ D an output c ∈ C. We write
f (d) = c or d 7→ c. The set of all inputs for f , which we’ve called D above, is the domain of f . The set in which
outputs take values, written C above, is the codomain (or target). The range (or image) of D is the collection
of all outputs.
Notice that the range of a function is not necessarily all of the codomain. Indeed, it’s always an interesting
question to ask exactly what the range of a function is.
→
Now recall that an n × m matrix A can be multiplied on the left of a vector −
x ∈ Rm to produce a vector
−
→
n
m
n
m
y ∈ R . Hence A defines a function from R to R (after all, a function from R to Rn is nothing more than a
rule for assigning an input from Rm to an output in Rn ). We write A : Rm → Rn to signify that multiplication
on the left by A provides a function from Rm to Rn . Functions which arise from multiplication by a matrix are
a very special class of functions from Rm → Rn , so we make the following
Definition 5.2. A linear operator is a function T : Rm → Rn which is given by multiplication on the left by
an n × m matrix A.
One is often interested in finding the matrix A associated to a linear operator T , and we can use the last
theorem from the previous section to answer this question.
Theorem 5.1. Suppose that T is a linear operator. Then the matrix A associated to T is given by


→
→
 T (−
e 1 ) T (−
e 2) · · ·
→
T (−
e m)  .
Proof. We know that secretly the action of T is given by a matrix A. How can we find out what A is? We
→
answered just this question earlier in the day: to find the ith column of A we just multiply on the right by −
ei
−
→
−
→
→
−
(ie, we compute A e i ). But A e i = T ( e i ) since A is the matrix associated to T .
¤
Remark. It is worth noting that our definition of a linear operator is not the only one out there. Some books
−
−
will define a linear operator to be a map from Rm → Rn so that for vectors →
u and →
v in Rm and k a random
scalar,
−
−
• T (k →
u ) = k(T (→
u )) and
→
−
−
→
→
→
• T ( u + v ) = T (−
u ) + T (−
v ).
Rest assured: this definition is the same as our definition. In fact, we know that ‘our’ linear operators satisfy
these conditions since matrix multiplication satisfies these conditions (see the previous section). Conversely,
an operator which satisfies these conditions gives a matrix in exactly the way we’ve described in the previous
theorem.
aschultz@stanford.edu
http://math.stanford.edu/~aschultz/summer06/math103
Page 6 of 6