Some Linear Algebra Problems
SOLUTIONS
Isabel Vogt
Last Edited: May 24, 2013
Most of these problems were written for my students in Math 23a/b at Harvard in 2011/2012
and 2012/2013.
1. Consider a parallelogram spanned by vectors ~v and w.
~
~v
w
~
(a) Prove the “parallelogram law,” which says that the sum of the squares of the lengths
of the diagonals of the parallelogram is equal to 2(|~v |2 + |w|
~ 2 ).
——–
Solution:
We label the diagonals as:
~v
~v − w
~
~v + w
~
w
~
Then it is clear that
|~v + w|
~ 2 + |~v − w|
~ 2 = (~v + w)
~ · (~v + w)
~ + (~v − w)
~ · (~v − w)
~ = 2|~v |2 + 2|w|
~ 2 + 2(~v · w)
~ − 2(~v · w)
~
(b) Use vectors to prove that the diagonals of a rhombus are perpendicular.
——–
Solution:
The diagonals are ~v + w
~ and ~v − w
~ - so let’s use the dot product to test for orthogonality:
(~v + w)
~ · (~v − w)
~ = |~v |2 − |w|
~ 2=0
As ~v and w
~ have the same length!
2. T/F:
(a) For all ~v ∈ R3 , the set of vectors ~u ∈ R3 such that ~u × ~v = ~0 forms a subspace of R3 .
——–
Solution:
1
TRUE
We know that for a given ~v ∈ R3 , the set of ~u such that ~u × ~v = 0 are α~v where α ∈ R.
Furthermore V = {α~v |α ∈ R} is a subspace of R3 as α1~v + α2~v = (α1 + α2 )~v where you
will note that (α1 + α2 ) ∈ R so (α1 + α2 )~v ∈ V.
2
(b) The set of all invertible n × n real matrices forms a subspace of Rn .
——–
Solution:
FALSE
2
Consider [In ] ∈ M (invertible real n × n matrices). If M were a subspace of Rn then
2
0 · [In ] = [0] ∈ M. But [0] is not an invertible matrix in Rn . Thus M is not closed and
thus not a subspace.
3. A linear transformation T : R3 7→ R3 sends the first standard basis vector ~e1 to the vector
~a1 . Similarly T (~e2 ) = ~a2 and T (~e3 ) = ~a3 . Furthermore ~a1 + ~a2 + ~a3 = 0. Does there exist a
unique S = T −1 such that S ◦ T = T ◦ S = I. Provide a proof.
——–
Solution:
We can write T as a 3×3 matrix in the form [~a1 ~a2 ~a3 ]. We know that T can not be invertible
if det T = 0. To examine this: det T = ~a1 · ~a2 × ~a3 . But we can write ~a1 = −~a2 − ~a3 by the
relation above. So
det T = (−~a2 − ~a3 ) · (~a2 × ~a3 )
= −~a2 · ~a2 × ~a3 − ~a3 · ~a2 × ~a3
= −~a2 × ~a2 · ~a3 − ~a2 · ~a3 × ~a3 = 0
Thus T is not invertible and no such S exists.
4. T/F: The linear transformation Rθ : R2 → R2 given by rotation through angle θ about the
origin has at least one real eigenvalue.
————————
Solution: FALSE
For a general θ this is false. We can either think of this geometrically or algebraically.
Geometrically, we know that Rθ rotates the entire plane so only the origin is preserved
(unless θ = π or 0). Thus no subspace is brought into itself except trivially the entire
plane, which can’t be decomposed into the cartesian product of two preserved 1-dimensional
subspaces. Thus Rθ does not have a real eigenvalue. Algebraically, we can compute the
determinant of λI − Rθ as
λ − cos θ − sin θ
λI − Rθ =
sin θ
λ − cos θ
So p(λ) = det(λI − Rθ ) = λ2 − 2λ cos θ + cos2 θ + sin2 θ = λ2 − 2λ cos θ + 1. To see if this
has real roots, we take the discriminant ∆(p) = 4 cos2 θ − 4. As | cos θ| ≤ 1, ∆(p) < 0 unless
θ = 0, π in which case ∆(p) = 0 and any vector in R2 is an eigenvector with eigenvalue −1.
Thus, in general, no real roots exist.
2
5. T/F: If T : Rn → Rn has n linearly independent eigenvectors, then T is invertible.
————————
Solution: FALSE
Consider T the zero matrix [0]n . The n standard basis vectors form an eigenbasis for T with
eigenvalue uniformly 0, however [0]n is not invertible.
6. T/F: Given an orthonormal basis {~v1 , ..., ~vn } for a vector space V , if w
~ ∈ V is in terms of
the standard basis, then w
~ = c1~v1 + ... + cn~vn where ci = w · vi .
————————
Solution: TRUE
Well, let’s test it. We know that as {~v1 , ..., ~vn } forms a basis for V we can write w
~ =
c1~v1 + ... + cn~vn for some scalers c1 , ..., cn . Now
w · ~vi = ci~vi · ~v1 + ... + cn~vn · ~vi
As the set {~v1 , .., ~vn } are orthonormal, ~vi · ~vj = δij , where δij is the Kronecker delta function:
δij = 0 for i 6= j and δij = 1 for i = j. Thus in the above sum all the dot products are 0
except ci~vi · ~vi = ci . So ci = w · vi .
7. T/F: If A : Rn → Rn and B : Rn → Rn are two linear transformations such that A =
C −1 ◦ B ◦ C for some invertible C, then dim ker(A) = dim ker(B)
————————
Solution: TRUE
The short answer is true as C is just a change of basis, and thus A is isomorphic to B.
Isomorphic transformations have the same nullity. The long answer is think of kerB as a
subspace of Rn . Then by Hubbard theorem 2.4.19, the dimension of kerB is well-defined, i.e.
it is the same in the basis for A. The very very short (intuitive) answer is that if this were
false the world would be pretty messed up (change your basis and change the dimension of
the kernel?! Blasphemy!).
8. T/F: Each of the follow forms a vector space over R (evaluate them seperately):
(a) D[π, 2π], the space of discontinuous real-valued functions
d : [π, 2π] → R
under the composition law (d1 + d2 )(x) = d1 (x) + d2 (x) and scaling law r(d1 (x)) =
r ∗ d1 (x) ∀r ∈ R
————————
Solution: FALSE
For D[π, 2π] to be a vector space,
∀v ∈ D[π, 2π], av ∈ D[π, 2π], a ∈ R
By that logic 0v ∈ D[π, 2π], but the zero function is continuous, and so not in D[π, 2π].
Thus D[π, 2π] is not a vector space.
3
(b) The space of invertible linear transformations T : Rn → Rn under the composition law
(T1 + T2 )(x) = T1 (x) + T2 (x) and obvious scaling law by elements of R
————————
Solution: FALSE
Again, 0 would have to be in this space, but 0 is not invertible.
(c) The set of symmetric matrices A ∈ Mat(3 × 3) with trace(A) = 0.
————————
Solution: TRUE
Checking scaler multiplication and vector addition, let’s calculate
X
tr(cA1 + A2 ) = ctr(A1 ) + tr(A2 ) = 0
X
(cA1 + A2 )T = cAT1 + AT2 = cA1 + A2
So it once again a symmetric traceless matrix.
(d) Functions from a set S = {1, 2, ..., n} to R under the obvious composition and scaling
rules.
————————
Solution: TRUE
This is quite obviously isomorphic to Rn . We can send the first element of the set 1 to
anything in R, the same for the second element, etc up to the last nth element. The
vector of coefficients in Rn then represents the real numbers where each element of the
set is sent. We then extend the function by linearity.
(e) The set of real polynomials of degree less than or equal to 3 with a root at +3.
————————
Solution: TRUE
Let’s see, the 0 polynomial satisfies this because if p(x) = 0, then surely p(3) = 0.
Furthermore, it is closed under addition and scaler multiplication as if r(x) and q(x) ∈
V , then cr(3) + q(3) = 0 and thus this sum is also in V . So we’re good! All that we
are doing is putting one further constraint on our system which is that the evaluation
on +3 gives 0. This is isomorphic to just a 3D subspace of R4 though the origin. As a
side note, a basis could be x − 3, x2 − 9, x3 − 27.
9. Does row reduction preserve the kernel and image of a linear transformation A : Rn → Rm ?
If yes, why? If no, what does it preserve?
————————
Solution:
For the kernel, this first statement is actually true - if A is the original transformation and
à the row reduced version, then I claim that A and à have the same kernel. This is a very
cool result! Why is it true. Well we have a theorem that tells us that the solutions ~x to
A~x = ~b are unchanged by row reduction. That is, Ã~x = ~b0 , where ~b0 is the vector post the row
operations that reduced A. Now, vectors in the kernel are just solutions to A~v = 0. So these
vectors ~v are also solutions to Ã~v = 00 . But what is 00 ? Row operations only involve taking
sums of elements within a single column, or multiplying by a scaler, or switching elements
within a column. In any case, none of these actions are going to change a columns of zeros
4
into anything else, this 00 = 0. And thus Ã~v = 0, for the same ~v . This A and à have the
same kernel (to be perfectly rigorous, you should show that they are subsets of each other,
i.e. imagine w
~ in the kernel of A, then row reduce and w
~ ∈ ker(Ã), thus ker(A) ⊂ ker(Ã).
How would you show the other direction?)
Now, this is not true of the image, as that would mean every image was spanned by some
set of k standard basis vectors... no! What is preserved is the rank, i.e. the number of
independent vectors in the image, the dimension of the image. This must be preserved as
row reduction preserves the dimensions of A, and the dimension of the kernel, because we
have the much stronger claim that it actually preserves the kernel. Thus by rank nullity, it
must preserve the rank.
10. Given that a linear transformation T acting on a real vector space V has at least one eigenvalue, what is the codomain of T ? If T has only one eigenvalue what can you say? If T
has an eigenbasis without distinct eigenvalues what can you say about geometry of T ? Say
T : R3 → R3 , |T | =
6 0, but trace(T ) = 0. If T does not have distinct eigenvalues but has an
eigenbasis, what can you say about the eigenvalues of T ?
————————
Solution:
We know that for T to have an eigenvalue (and thus an eigenvector) T : V → V , thus the
codomain must also be V . If T has only 1 eigenvalue (but you don’t know whether it has an
eigenbasis) then you can’t see that much except that it preserves some 1-dim’l subspace (i.e.
it takes a line into itself if we think of Rn ) through the operation of scaling. However, if we
know that it has an eigenbasis, then we know that if it has only one eigenvalue, it preserves
all of V by scaling. Thus T = λI. If T has an eigenbasis and 2 repeated eigenvalues, it
preserves a 2-dim’l space through scaling, etc. For R3 this looks like preserving a line and
a plane if it has 2 distinct e-values, preserving three lines if it has 3 distinct e-values, and
preserving all of R3 (ie acting as scaling) if it has only 2 distinct e-value.
Now, for this last part, we need the theorem that trace(T ) is the sum of the eigenvalues of T .
So if T : R3 → R3 and tr(T ) = 0, but |T | =
6 0, then if T has degeneracy in the eigenvalues, it
must just be that 2 are the same, as if all 3 were the same, they would have to be all 0, and
thus T would be the zero transformation, which has length 0. So from our trace condition,
we know that the eigenvalues must be {λ, λ, −2λ}.
11. Let V be an n-dimensional complex vector space. Consider two linear transformation A :
V → V and B : V → V . Prove that if A and B do no commute (i.e. if A ◦ B 6= B ◦ A) then
there cannot exist a basis {vi } for V which is simultaneously an eigenbasis for both A and
B. (Hint: proceed by contradiction.)
————————
Solution:
Assume that there exists a basis {~v1 , ..., ~vn } for V which is an eigenbasis of both A and B,
with A~vi = ai~v1 and B~vi = bi~vi . Then any w
~ ∈ V can be written as a linear combination of
these basis vectors as
w
~ = c1~v1 + ... + cn~vn
5
Now consider the linear function AB − BA. We act on an arbitrary vector w
~ as
(AB − BA)w
~ = AB(c1~v1 + ... + cn~vn ) + BA(c1~v1 + ... + cn~vn )
= A(b1 c1~v1 + ... + bn cn~vn ) − B(a1 c1~v1 + ... + an cn~vn )
= (a1 b1 c1~v1 + ... + an bn cn~vn ) − (b1 a1 c1~v1 + ... + bn an cn~vn ) = 0
As scaler multiplication is commutative, these two sums are equal and thus the entire action
of (AB − BA) on w
~ is 0. However, w
~ was an arbitrary vector in V , so this must be true for
all vectors in V . So (AB − BA) is the 0-tranformation, or AB − BA = 0, ⇒ AB = BA. But
this contradicts our assumption that they don’t commute. Thus a simultaneous eigenbasis
can not exist.
12. This problem will lead you through a proof and application of the Buckingham π Theorem,
one of the most fundamental results in dimensional analysis. Given a system of n physical
variables ui (say the gravitational constant g, mass of an object m, the length of a string l,
etc.) in k independent physical dimensions vi (for example T time, M mas, L length, etc),
(a) Show that the space of fundamental and derived “units” forms a vector space over
Q. (Hint: if L, T , M are fundamental units, then M ∗ L ∗ T −2 is a derived unit, as
is (M ∗ L ∗ T −2 )3/2 . How might we represent these derived units in terms of fundamental units? What might we call the “fundamental units”? Note that multiplying a
derived/fundamental unit by a scaler is considered an equivalent unit.)
————————
Solution:
This is a vector space as follows: basis vectors are the fundamental units of the system
U1 , ..., Un . All derived units are then products of fundamental units as U1a1 U2a2 ...Unan
where ai ∈ Q. This is closed under ”scaler multiplication” (which is just multiplying the exponents by an element of Q) and ”vector addition” (which is addition in
the exponents) as Q is closed. Similarly the distributive law follows from Q, so it is
an honest-to-goodness vector space. Note that ”0” in this vector space is the unit
U10 ...Un0 = 1 - i.e. the dimensionless quantity.
(b) Now consider a matrix M which we will call the dimension matrix for obvious reasons.
Each column of M tells how to form the n variables ui out of the k physical dimensions
vi , i.e. the (s, t)th entry of M is the power of the unit vs in the constant ut .
i. What are the dimensions of M ?
————————
Solution:
M represents the n variables ui in terms of the k physical dimensions, it is thus
ci ci
ci
k × n. If the ith variable is u11 u22 ...unn then the matrix M is represented as:


c11 c21 c31 · · · cn1
 c1 c2 c3 · · · cn 
2
 2 2 2
M =  ..
.. 
.
.
1
2
3
ck ck ck · · · cnk
6
 
a1
 a2 
 
ii. Describe in words what the result of a matrix multiplication M  ..  is.
.
an
————————
Solution:
this in
If we form some new
physical variable v1a1 v2a2 ...vnan , werepresent
 derived


a1
a1
 a2 
 a2 
 
 
terms of the vector  .. . Then the matrix multiplication M  ..  gives a new
.
.
an
an
k × 1 vector which represents the units of this derived physical variable in terms of
the fundamental units - i.e. the first entry will be the number of copies of u1 in the
derived variable v1a1 v2a2 ...vnan .
iii. Apply a theorem to find a formula for the number of independent *dimensionless*
parameters πi (combinations of the original n physical variables) in terms of n and
some characteristic of the matrix M . What are the bounds on the maximum and
minimum number of independent dimensionless parameters?
————————
Solution:
A dimensionless parameter is a combination of physical variables which does not
depend
  on any fundamental (or derived for that matter) units - i.e. it is a vector
a1
 a2 
 
 ..  which is in the kernel of the matrix M . We want to know how many inde.
an
pendent of these such dimensionless parameters we can find. That is, we want to
know the nullity of the transformation M .
To do this, we use the rank-nullity theorem. This tells us that for the matrix M ,
whose domain is n-dimensional, the nullity of M is n− rank(M ). In particular, the
rank of M is at most k and at least 1 (assuming our physical variable aren’t nonsense like all dimensionless). So there are at least n − k independent dimensionless
parameters, and at most n − 1.
As a “historic” note, the buckingham pi theorem is normally quoted as the second
to last result - for a physical system of n physical variables in k physical dimensions,
there are (at least) n − k dimensionless parameters. Note that even the “at least”
is occasionally dropped, but remember to parse this as “at least”!
(c) A standard application of dimensional analysis is to determine a relation for the period
of a pendulum. The obvious list of physical quantities here are
7
Description
Variable Units
length of string
l
L
arc of displacement
s
L
gravitational constant
g
L ∗ T −2
mass at end of string
m
M
period of swing
τ
T
Where L is length, M is mass, and T is time. Use the methods developed in part
(b) and in class to find an expression for the period τ of the pendulum in terms of a
*dimensionless constant* times some relation in the above physical variables.
————————
Solution:
We form the matrix described above as M by decomposing the various physical variables
in terms of the fundamental units. Using the implicit ordering in the table above:


1 1 1 0 0
M = 0 0 0 1 0
0 0 −2 0 1
To find vectors in the kernel, we will find a basis for the kernel using the methods we
derived in class:




1 1 1 0 0
1 1 0 0 1/2
row reduce
M = 0 0 0 1 0 −−−−−−→ 0 0 1 0 −1/2
0 0 −2 0 1
0 0 0 1
0
Without having to be fancy we see that (in this ordering) the physical variables s and
τ are non-pivotal. To find a basis for the kernel, we can note that the row reduction
(and inspection, really) tells us that

  
1/2
1
 0  −1

  
−1/2 ,  0 

  
 0  0
−1
0
are in the kernel. As they are independent, we have found a basis!
1/2
Let’s look a little more closely, we see that the first vector means gl1/2 τ is dimensionless.
Let’s call this constant 1c . Then
1/2
1
l
= 1/2 ⇒ τ = c
c
g τ
s
l
g
So we have found an expression for τ in terms of a dimensionless constant times a
combination of our physical variables.
13. Suppose A : R3 → R3 is a linear transformation such that for three linearly independent
~v1 , ~v2 , ~v3 we have
8
A~v1 = λ~v1 , A~v2 = λ~v2 , A~v3 = λ~v3
for some λ ∈ R.
(a) In the eigenbasis, what does A look like? Justify your answer.
————————
Solution:
In terms of the eigenbasis, we know that A acting of each of the eigenvectors extracts
the corresponding column of the matrix presentation. So, quite obviously,


λ 0 0
A = 0 λ 0
0 0 λ
(b) In the standard basis, what does A look like? Justify your answer.
————————
Solution:
Now, we can do this in several different ways: the most algorithmic is to think about
changing our basis. First we change from the standard to the eigenbasis. This is achieved
by the matrix P −1 = [~v1 ~v2 ~v3 ]−1 . Then we transform in the eigenbasis, and then we
need to change the basis back to the standard basis which is achieved by P = [~v1 ~v2 ~v3 ].
So the entire transformation is given by Aei = P Avi P −1 . But we know that Avi = λI.
So we manipulate as:
Aei = P Avi P −1 = Aei = λP IP −1 = λI
So again, in terms of the standard basis, A looks like:


λ 0 0
A = 0 λ 0
0 0 λ
The more intuitive reason is that if there is an eigenbasis with only one eigenvalue, then
a 3-dimensional subspace of R3 is the preserved subspace multiplied by a scaler. The
only 3-dimensional subspace of R3 is R3 itself, so every vector in R3 is scaled by λ - in
particular the standard basis vectors.
(c) Give a condition on λ that determines when A is an isomorphism.
————————
Solution:
This is already linear, but it will be invertible if and only if λ 6= 0.
14. Let T be the linear transformation given by
 


x
3x
+
y
y 
  2z 
T : R4 → R3 , T 
z  =
y+w
w
9
Furthermore, let B1 be an alternative basis for R4 given by
       
0
1
2 

 1


1 1 1 2








B1 =   ,   ,   ,  
2
0
0
2 





1
3
0
5
And let B2 be an alternative basis for R3 given by
     
1
4 
 3





0 , 1 , 0
B2 =


0
2
2
(a) Find the matrix representation for T with respect to the standard basis. Is this unique?
Find a basis for the image and kernel of T .
————————
Solution:
To find the matrix representation for T with respect to the standard basis vectors we
observe how T acts on each of the standard basis vectors. The matrix is:


3 1 0 0
T = 0 0 2 0
0 1 0 1
We must determine which variables are pivotal and which are nonpivotal. To do this,
we row reduce:




3 1 0 0
1 1/3 0 0
row reduce
T = 0 0 2 0 −−−−−−→ 0 0 1 0
0 1 0 1
0 1 0 1
So we see that x, z, and w are pivotal, whereas y is nonpivotal. So the basis for the
image is given by the columns corresponding to action by ~e1 , ~e2 , ~e4 . Namely:
     
0
0 
 3
0 , 2 , 0


0
0
1
Row reduction also tells us that T (~e2 ) = T ((1/3)~e1 + ~e4 ). (I got this from the second
column of the row reduced matrix).
Thus T (~e2 − (1/3)~e1 − ~e4 ) = 0 - meaning that ~e2 − (1/3)~e1 − ~e4 is a basis for the kernel.
Let’s check:

  
 −1/3
0
3 1 0 0 


X 0

0 0 2 0  1  =
 0  0
0 1 0 1
−1
0

(b) Determine the matrix representation for T with respect to B1 in the domain and B2 in
the codomain in two ways:
10
i. Determine how T acts on the basis vectors B1 in terms of B2
————————
Solution:
Let’s call the ordered vectors in B1 ~v1 , ~v2 , ~v3 , ~v4, and
~ 1, w
~ 2, w
~ 3, w
~ 4.
 the vectors in B2 w
4
Now we know that T (~v1 ) gives us the vector 4. Similarly:
2
 
 
 
1
4
8





T (~v2 ) = 0 , T (~v3 ) = 0 , T (~v4 ) = 4
4
1
7
Now we want to write
reduction in tandem:

3 1 4 4 1
0 1 0 4 0
0 2 2 2 4
these in terms of the basis B2 . We will do this by row



4 8
1 0 0 4 −7/3 2/3
2
row reduce
0 4 −−−−−−→ 0 1 0 4
0
0
4 
1 7
0 0 1 −3
2
1/2 −1/2
The second half of this matrix should give us the vectors T (~vi ) in terms of the {w
~ i}
- i.e. this is our matrix!!


4 −7/3 2/3
2
0
0
4 
T̃ =  4
−3
2
1/2 −1/2
ii. Write down an expression in terms of change of basis matrices and evaluate it.
————————
Solution:
We want to start in the basis for R4 given by B1 , then change to the standard basis.
Then we transform in the standard basis. Finally, we need to change back from the
standard basis for R3 to the basis B2 . Let’s call the first transformation P , and the
second Q−1 . We can easily write down




1 0 1 2
3 1 4
1 1 1 2



P =
2 0 0 2 , Q = 0 1 0
0 2 2
1 3 0 5
So we invert and find:
Q−1


1/3 1 −2/3
1
0 
= 0
0 −1 1/2
So all together:


 1
1/3 1 −2/3 3 1 0 0 
1
1
0  0 0 2 0 
T̃ = Q−1 T P =  0
2
0 −1 1/2
0 1 0 1
1

11
0
1
0
3
1
1
0
0

2
2

2
5
Sure enough, we find again that:


4 −7/3 2/3
2
X
0
0
4 
T̃ =  4
−3
2
1/2 −1/2
(c) Does T admit an eigenbasis?
————————
Solution:
Nope, its domain and codomain are not the same space.
15. Let V be a finite dimensional vector space. Prove that any ordered spanning set of vectors can
be reduced to a basis by removing vectors from the set, and any ordered linearly independent
set of vectors can be expanded to a basis by adding vectors in V .
————————
Solution:
Theorem 0.1 Every set of vectors ~v1 , ..., ~vm that spans V can be reduced to a basis for V by
removing a subset of the vectors.
Proof. We begin with a set B̃ = (~v1 , ..., ~vm ). Now we carry out an algorithm to pare down
this spanning set to a basis:
Step 1: If ~v1 = 0 then remove ~v1 , if not leave B̃ unchanged
Step 2: If ~v2 ∈ Spanṽ1 then remove ~v2 , if not leave B̃ unchanged
Step i: If ~vi ∈ Spanṽ1 , ..., ṽi−1 then remove ~vi , if not leave B̃ unchanged
If we continue the algorithm through m steps, SpanB̃ = Spanṽ1 , ..., ṽm by construction.
Furthermore no vector is in the span of the previous vectors, thus it is linearly independent.
So B̃ is a basis for V .
Theorem 0.2 Every linearly independent set of vectors ~v1 , ..., ~vn in a finite dimensional
vector space V can be extended to a basis for V by addition a set of vectors.
Proof. By definition of V being finite dimensional, there exists a spanning set of a finite
number of vectors w
~ 1 , ..., w
~ m for V . We will construct a basis for V using the following
algorithm. First, let B̃ = (~v1 , ~v2 , ..., ~vn ):
Step 1: If w
~ 1 6∈ Spanṽ1 , ..., ṽn , then add w
~ 1 to B̃; if not leave B̃ unchanged
Step 2: If w
~2 ∈
/ Spanṽ1 , ..., ṽn , w̃1 , then add w
~ 2 to B̃; if not leave B̃ unchanged
Step i: If w
~i ∈
/ Spanṽ1 , ..., ṽn , w̃1 ...w̃i−1 , then add w
~ i to B̃; if not leave B̃ unchanged
After m steps of the algorithm we have a set which is still linearly independent as no vector
is in the span of the previous vectors. Further more it is a spanning set as it contains
Spanw̃1 , ..., w̃, which by definition spans V . Thus B̃ is a basis for V .
16. T/F: If A is a 3 × 3 matrix with an eigenbasis but only 2 distinct eigenvalues, then for any
w
~ ∈ R3 , {w,
~ Aw,
~ A2 w}
~ are linearly dependent.
——————–
Solution: TRUE
12
Call the basis of eigenvectors {~v1 , ~v2 , ~v3 }
Without loss of generality let A~v1 = λ1 , A~v2 = λ1 , A~v3 = λ2
Any w
~ can be represented as a linear combo of these basis vectors:
w
~ = a~v1 + b~v2 + c~v3
Then:
Aw
~ = aA~v1 + bA~v2 + cA~v3
= aλ1~v1 + bλ1~v2 + cλ2~v3
And:
A2 w
~ = A(aλ1~v1 + bλ1~v2 + cλ2~v3 )
= aλ1 A~v1 + bλ1 A~v2 + cλ2 A~v3
= aλ21~v1 + bλ21~v2 + cλ22~v3
To check independence we put these into a matrix (in the eigenbasis) and see if we could row
reduce to the identity:

a aλ1 aλ21
 b bλ1 bλ21 
c cλ2 bλ22

Clearly, b(row 1) - a(row 2) yields a row of 0s. Thus this matrix can not row reduce to the
identity and A2 w,
~ Aw,
~ w
~ must be dependent.
17. The operation × (the cross product) is a linear operator from (R3 , R3 ) → R3 .
——————–
Solution: FALSE
The operator is multi linear. That means it is linear in each component, if you fix the other
components. To see why this is wrong as stated (a linear transformation) consider:
(~v , w)
~ 7→ ~v × w
~
c(~v , w)
~ = (c~v , cw)
~ 7→ c2 (~v × w)
~ 6= c(~v × w)
~
18. Approximations
13
(a) True or False: for every n × n real matrix A, if A has an eigenvalue, then there is an
explicit formula for this eigenvalue in terms of the coefficients of A (i.e. something along
the lines of the quadratic formula)
——————–
Solution: FALSE
For 5 × 5 matrices and above, there is the possibility that the polynomials won’t have
a closed-form solution for the roots, as quadratics do.
(b) As you may have discovered, it is something of a pain to determine the eigenvalues of
an n × n matrix of dimension greater than 3 using the formulas we derived in class; as n
becomes even larger, you can imagine that this becomes unwieldy even for a computer.
As such there are many eigenvalue-computing algorithms that approximate the eigenvalues of a matrix.
Assume that A is diagonalizable (i.e. an eigenbasis exists) with λ1 the unique eigenvalue
of greatest magnitude. For a random (you can assume “nice”, but see part D) vector
b0 , we define the recursive relation
Abk
|Abk |
in terms of the original b0 .
bk+1 =
i. First, find an expression for bk+1
——————–
Solution:
Ak+1 b0
|Ak+1 b0 |
ii. As k → ∞, what can you hypothesis about bk+1 ? Use what you know about A to
prove your claim. In the process, if you haven’t approximated λ1 and an associated
unit eigenvector, you might want to reconsider.
bk+1 =
We hypothesize that bk+1 will be a unit eigenvector with eigenvalue λ1 . We can
prove this as follows:
First write b0 in the eigenbasis. Thus
b0 = c1~v1 + c2~v2 + ... + cn~vn
We order the eigenvectors so that λ1 > λ2 ≥ λ2 .... Then
v1 + c2 λk+1
v2 + ... + cn λk+1
vn
Ak+1 b0 = c1 λk+1
1 ~
2 ~
n ~
k+1
k+1 !
λ
c
λn
c
2
2
n
Ak+1 b0 = c1 λk+1
~v1 +
~v2 + ... +
~vn
1
c1 λ1
c1 λ1
So this converges to a multiple of ~v1 as all of the eigenvalue fractions are less than
1. After normalization this is a unit.
We get λ1 by the ratio of components of bk and Abk .
14
iii. Bonus: What is the (approximate) rate of convergence of the above algorithm?
How does this compare to other algorithms we have seen in this class. (Note: your
answer will depend on some characteristic of A).
——————–
Solution:
This converges geometrically with ratio λλ21 . This is fast if λ2 << λ1 and slow if
they are similar.
iv. Where could the above algorithm go wrong (think about b0 ).
——————–
Solution:
If b0 did not have a component along ~v1 . Luckily this happens with measure 0 in
the space of vectors and thus probability 0.
19. The Trace
(a) Working from the definition of matrix multiplication, prove that tr(AB) = tr(BA).
——————–
Solution:
Call C = AB, then
!
tr(C) =
X X
i
ai,k bk,i
k
Call D = BA, then
!
tr(D) =
X X
i
bi,k ak,i
k
These two are the same as the elements of R commute and so they can both be written
as
X
ai,k bk,i
i,k
(b) Use this to prove that the trace of a matrix A is preserved under change of basis. This
is very good because it says that the trace is an algebraic invariant of the operator A
that is independent of coordinate system.
——————–
Solution:
Let A0 = P −1 AP (so A0 is related to A through a change of basis). Then
tr(A0 ) = tr (P −1 A)P = tr P (P −1 A) = tr(A)
This means trace is an algebraic invariant. It joins the ranks of the likes of the kernel,
the rank, the nullity, the determinant, etc.
15
(c) Use the above analysis to conclude that if A has an eigenbasis with eigenvalues {λi },
then
X
tr(A) =
λi
i
——————–
Solution:
This is true in the eigenbasis, so it is true in any basis.
20. Symmetric Matrices
(a) For A a symmetric matrix which admits an eigenbasis with unique eigenvalues. Prove
that for vi and vj eigenvectors,
vi · vj = 0 for i 6= j
This says that the eigenvectors are pairwise orthogonal.
——————–
Solution:
Consider ~vi and ~vj eigenvectors
λi~vi · ~vj = A~vi · ~vj = (A~vi )T ~vj = ~viT AT ~vj = ~viT A~vj = ~vi · A~vj = ~vi · λj ~vj
If λi 6= λj , then ~vi · ~vj = 0.
16