Applied Linear Algebra , Final Exam
1. (20pts) Let A be a real symmetric square matrix. Show the followings.
(a) Every eigenvalue of A is real.
(b) Any two eigenvectors of A corresponding to distinct eigenvalues are orthogonal.
(c) If A is positive definite, then every eigenvalue of A is positive.
(d) If every eigenvalue of A is positive, then A is positive definite.
Solution
(a) Every eigenvalue of A is real .
pf) Let v be an unit eigenvector corresponding to λ. 0 ≤ (Av)H Av = v H AH Av =
|λ|2 v H v = |λ|2
(b) Any two eigenvectors of A corresponding to distinct eigenvalues are orthogonal.
pf) Let v1 and v2 be two eigenvectors corresponding to λ1 and λ2 respectively. then
λ1 v1T v2 = (Av1 )T v2 = v1T AT v2 = v1T Av2 = λ2 v1T v2 so v1 and v2 are orthogonal .
(c) If A is positive definite, then every eigenvalue of A is positive.
pf) Let v be an unit eigenvector of A corresponding to λ. then v T Av = λ is positive
(d) If every eigenvalue of A is positive, then A is positive definite.
pf) Since A can be diagonalizable, we can decompose A as A = QT ∆Q where Q is
an orthogonal matrix and ∆ is a diagonal matrix whose entries on the main diagonal.
for any x ∈ Rn xT Ax = xT QT ∆Qx = (Qx)T ∆Qx which is positive definite.
1
2. (10pts) Let A be a real skew-symmetric square matrix, i.e., AT = −A.
P
tn n
(a) Show that the matrix etA = I + ∞
n=1 n! A is orthogonal for all t ∈ R.
(b) Show that
d
dt ||u(t)||
= 0 if
d
dt u(t)
= Au(t).
Solution
From
e
tA
=I+
∞ n
X
t
n=1
we have
tA T
(e ) = I +
∞ n
X
t
n=1
n!
T n
n!
(A ) = I +
An ,
∞
X
(−t)n
n=1
n!
An = e−tA
Therefore, we have (etA )T etA = e−tA etA = I.
The solution of the differential equation is u(t) = etA u(0). It follows that ||u(t)||2 =
d
u(0)T e−tA etA u(0) = u(0)T u(0) = ||u(0)||2 . Hence dt
||u(t)|| = 0.
2
3. (10pts) Let A, B be two 3 × 3 matrices given by




0 1 2
0 0 1
A = 0 0 1  ,
B = 0 0 0  .
0 0 0
0 0 0
Determine whether A and B are similar.
Solution
Please do it yourself.
3
4. (10pts) Let F0 , F1 , F2 , . . . be a sequence of real numbers defined by the recurrence
relation:
F0 = 0,
F1 = a,
Fk+2 = aFk+1 + Fk for k = 0, 1, 2, 3, . . . ,
where a ̸= 0. Find the general formula for Fk .
Solution:
a 1
Set A =
to find that
1 0
Fk+2
a 1
Fk+1
aFk+1 + Fk
=
=
Fk+1
1 0
Fk
Fk+1
From
Fk+1
k a
=A
0
Fk
⇒
a−z 1
det(A − zI2 ) = det
= z 2 − az − 1,
1
−z
the eigenvalues of A are
√
a2 + 4
,
λ1 =
2
with the corresponding eigenvectors
λ1
v1 =
,
1
a+
λ2 =
a−
√
a2 + 4
,
2
λ2
v2 =
1
Diagonalize A
A = SΛS
−1
=
λ1 λ2
1 1
1
λ1 0
1 −λ2
0 λ2 λ1 − λ2 −1 λ1
√
a2 + 4 ̸= 0. Compute
k
1
λ1 λ2
λ1 0
1 −λ2
k
−1
A = SΛS =
1 1
0 λk2 λ1 − λ2 −1 λ1
k+1
1
1 −λ2
λk+1
λ1
2
=
−1 λ1
λk1
λk2
λ1 − λ2
k+1
1
λ1 − λk+1
λ1 λk2 − λ2 λk1
2
=
λk1 − λk2
λ1 λk2 − λ2 λk1
λ1 − λ2
where λ1 − λ2 =
Finally
k+1
1
a
Fk+2
a(λk+1
− λk+1
)
λ1 − λk+1
k a
1
2
2
=A
=√
=√
Fk
0
a(λk1 − λk2 )
λk1 − λk2
a2 + 4
a2 + 4
to obtain that

Fk = √
√
a2
a
+4
 a+
2
+4
a2
4
!k
−
a−
√
a2
2
+4
!k 

5. (5pts) Use an appropriate method to show

1
A = 1
1
that the following matrix

1 1
5 3
3 11
is positive definite.
Solution:
The characteristic polynomial of A is


1−z
1
1
5−z
3  = −(z 3 − 17z 2 + 72z − 42)
det(A − zI3 ) =  1
1
3
11 − z
But it is not easy to find roots.
You can try Cholesky factorization A = CC T

1 0

C= 1 2
1 1
to find that

0
0
3
and notice that C has positive diagonals so that A is positive definite.
You can try A = LU = LDLT factorization to find that all pivots are positive.
You can use Syvester’s criterion and find
det 1 = 1 > 0,
1 1
det
= 3 > 0,
1 5

1 1 1
det 1 5 3  = 1(55 − 9) − 1(11 − 3) + 1(3 − 5) = 36 > 0
1 3 11

to conclude that A is positive definite. Etc.
5
6. (20pts) Let A be a m × 3 real matrix, where m is a positive integer, satisfying


3
a −1
3 −1 and ab = 1, a > 0.
AT A =  b
−1 −1 5
Answer the following questions and justify your answers.
(a) What are the possible values of a and b? (3pts)
(b) What is the rank of A? (3pts)
(c) What are the possible values of m? (4pts)
(d) What is the maximum value of ||Ax|| for all x ∈ R3 with ||x|| =
√
2. (10pts)
Solution
AT A should be a symmetric matrix since (AT A)T = AT (AT )T = AT A. Therefore
a = b. From a2 = 1 and a > 0, we have a = 1.
Then the eigenvalues of AT A are λ1 = 2 λ2 = 3 and λ3 = 6 with corresponding
orthonormal eigenvectors are
 
 
 
1
1
−1
1  
1  
1  
−1 ,
1 , v3 = √
−1 ,
v2 = √
v1 = √
2
3
6
0
1
2
It follows that A has non-zero singular values σ1 =
the rank of A is 3.
√
2, σ2 =
√
3 and σ3 =
√
6 so that
Since A is an m × 3 matrix with rank 3, m must be larger or equal to 3.
We can decompose any x ∈ R3 as x = c1 v1 + c2 v2 + c3 v3 such√that ||x||2 = c21 + c22 + c23 ,
since {v1 , v2 , v3 } is an orthonormal basis of R3 . From ||x|| = 2, we have c21 +c22 +c23 =
2. From
AT Ax = c1 λ1 v1 + c2 λ2 v2 + c3 λ3 v3
we obtain that
||Ax||2 = c21 λ1 + c22 λ2 + c23 λ3 ≤ λ3 (c21 + c22 + c23 ) = 2λ3 = 12
√
√
where we have used 0 < λ1 < λ2 < λ3 . Therefore 12 = 2 3 is an upper bound of
||Ax||. On the other hand
√
√
√
√
√
A 2v3 = 2(Av3 ) = 2(σ3 u3 ) = 12u3 ,
||A 2v3 ||2 = 12
T
where u3 is the third column of U
√ obtained from A = U ΣV . Therefore the maximum
of ||Ax|| is 12 attained for x = 2v3 .
6
7. (25pts) Let A be a 2 × 3 real matrix given by
1 0 1
A=
.
−1 1 0
(a) Find a singular value decomposition A = U ΣV T . (10pts)
(b) Find orthonormal bases of four fundamental subspaces associated to A and justify
your answers. (5pts)
(c) Find the pseudo (Moore-Penrose) inverse A+ of A and check that A(I3 −A+ A) =
0 and AT AA+ = AT . (5pts)
1
3
(d) Find every linear least square solution x̂ ∈ R minimizing ||Ax − b|| for b =
1
and justify your answer. (5pts).
Solution.
We have


2 −1 1
AT A = −1 1 0
1
0 1
and


2 − z −1
1
0  = (2 − z)(1 − z)(1 − z) − (1 − z) − (1 − z)
det(AT A − zI3 ) = det  −1 1 − z
1
0
1−z
= (3 − z)(1 − z)z
Therefore the eigenvalues of AT A are λ1 = 3, λ2 = 1 and λ3 = 0, all distinct, with
the corresponding orthonormal eigenvectors
 
 
 
2
0
−1
1  
1  
1  
−1 , v2 = √
1 , v3 = √
−1
v1 = √
6
2 1
3
1
1
So that we have the following 3 × 3 orthogonal matrix
√ 



2
−1 √1
2 √0 −√2
√
1
1

0
V = √ −1 √3 −√ 2 ⇒ V T = √  √
√3 √3
6
6
− 2 − 2
2
1
3
2
The singular values of A are σ1 =
√
3, σ2 = 1 and σ3 = 0 so that
√
3 0 0
Σ=
0 1 0
7
From
u1 =
1
1
Av1 = √
σ1
18
1
1
u2 =
Av2 = √
σ2
2
1 0
−1 1
1 0
−1 1

2
1
1
1  
3
1
−1 = √
=√
,
0
−3
−1
18
2
1
 
0
1 1
1  
√
1 =
0
2 1
1

we obtain the following 2 × 2 matrix
1
U=√
2
1 1
−1 1
Finally we have a singular value decomposition
1
A = U ΣV T = √
2
1 1
−1 1
√
3 0 0
0 1 0


2
−1 √1
√
1
√  √
0
3
3
6 − 2 −√2 √2
Or in the compact form
A=
2
X
j=1
σj uj vjT
√ 1
3 1
1 1 1
√ 2 −1 1 + √
√ 0 1 1
=√
2 −1
6
2 1
2
Check if the student has correctly identified the column space of A, column space of
AT , the null space of A and the null space of AT from the above.
8
The pseudo inverse A+ of A is
 
 
2
2
0
X
1 1   1
1   1
1
+
T
−1 √ 1 −1 + √
1 √ 1 1
A =
vj uj = √ √
σ
3 6
2
2 1
2
j=1 j
1



 1

1
2 −2
0 0
3 −3
1
1
2 
−1 1  + 1 1 =  13
=
3
6
2
2
1
1 −1
1 1
3
3


1 −1
1
= 1 2 
3
2 1
so that




1 −1 2 −1 1
1
1
1 0 1
= −1 2 1
A+ A =  1 2 
−1 1 0
3
3
2 1
1
1 2
Therefore we have the projector R3 → Null (A)

1
1
I3 − A+ A =  1
3
−1
given by

1 −1
1 −1
−1 1
and check
A(I3 − A+ A) =
1
3

1
1 −1
1 0 0 0
1 0 1 
1
1 −1 =
−1 1 0
3 0 0 0
−1 −1 1





1 −1
2 −1 1
3 −3
1
1
AT AA+ = −1 1 0 1 2  = 0 3  = AT
3
3
2 1
1
0 1
3 0

A least square x̂ is a solution to the normal equation
AT Ax = AT b
and A+ b is a particular solution AT AA+ b = AT b and AT A(I3 − A+ A) = 0. Therefore
every least square solution x̂ is given by
x̂ = A+ b − (I3 − A+ A)w,
9
∀w ∈ R3 .