1
CARNEGIE MELLON UNIVERSITY
DEPARTMENT OF ELECTRICAL AND COMPUTER ENGINEERING
18-771
LINEAR SYSTEMS
SPRING 2003
PROBLEM SET III
300 POINTS
DUE 6 MARCH 2003
in LECTURE
NUMERICAL LINEAR ALGEBRA
LINEAR ALGEBRAIC EQUATIONS
GAUSSIAN ELIMINATION
L U FACTORIZATION
CHOLESKY FACTORIZATION
ORTHOGONAL FACTORIZATIONS
LEAST-SQUARES
STATE-SPACE COMPUTATION
MATLAB COMPUTATION IN ENGINEERING
READING ASSIGNMENT
WEEK
B
Week of 17 February:
Week of 24 February:
6
Week of 3 March:
6
2
MIDTERM EXAMINATION
OPEN TEXTBOOK AND COURSE NOTES
TUESDAY 25 FEBRUARY 2003
2:30 PM – 4:30 PM
PH A 18 C
PROBLEM
XI *
25 POINTS
B
PAGE
5.6
223
PROBLEM XII *
100 POINTS
Let A(nxn) be a real symmetric (and diagonalizable) matrix with the positive real
eigenvalues 1, 2, …, n, and let P be the matrix of eigenvectors. Define the square root
of A as
Where D  Diagonal [ 1
1/2
Α1/2 :  P D1/2 P T
2 ... n ]
(1)
Show that:
(A) Α1/2 is symmetric;
(B) Α1/2 Α1/2  Α ;
(C) If Α -1/2 :  P D-1/2 PT , then Α1/2 Α-1/2  Α-1/2 Α1/2  I ; and
(D) Α-1/2 Α-1/2  Α-1 .
(E) Apply the foregoing development to compute the square root of the matrix
1
0
Α
0

0
0 0 0
2 0 0
0 6 2

0 2 6
According to (1).
(F) Apply the function of a square matrix concept and methodology to compute the
square root of the matrix A.
3
(G) Apply the MATLAB function chol to compute the Cholesky square-root factors of
the matrix A.
(H) Apply the MATLAB function sqrtm to compute the square-root of the matrix A.
(I) Highlight and interpret your findings.
PROBLEM XIII *
75 POINTS
A rather straightforward application of the Cayley-Hamilton Theorem reveals that
the transition matrix exp[Ft] of the constant state-space matrix vector differential
equation
dx
 Fx
dt
is a matrix polynomial in F; i.e.,
n 1
exp[Ft ]    k ( t )F k with the initial condition I(nxn).
k 0
(A) Find the differential equations
k . for k  0,1,2 ..., (n -1)  satisfy.
which
the
n
scalar
time
functions
(B) Specify the initial/boundary conditions.
(C) Find an explicit representation for the  k 
. for k  0,1,2 ..., (n -1)  .
(D) Formulate and test a MATLAB - script file to compute the resulting explicit
matrix polynomial representation for the transition matrix exp[Ft].
PROBLEM XIV *
100 POINTS
I.
INTRODUCTION
Every rank r matrix A(mxn) has a unique reduced row echelon form R = rref (A).
For example, the reduced row echelon form [1] of the matrix
 1 4 2 3 10 


A   1 2 0 1 6 
 2 5 3 0 16 


(1)
4
is
1 0


R  rref ( A )   0 1
0 0


2 0 -1 
3
3
1 0 10 
3
3 
0 1 -1 

(2)
The reduced row echelon form R = rref (A) indicates that:
(1)
The rank of A is three; A contains three linearly independent columns (the pivot
columns). The rank of A equals the number of pivot columns.
(2)
The first, second and fourth columns are the pivot columns of A. The pivot columns
1
 
 -1
 2
 
 4
 
 2
 5
 
and
 3
 
1
 0
 
of A lay a basis for the image of A.
(3)
Columns three and five are linear combinations of the pivot columns of A.
Specifically,
c(3) 
2
1
c (1)  c ( 2 )
3
3
c(5) 
-1
10
c (1) 
c ( 2 )  c (4 )
3
3
and
where c(i) is the i-th column of A.
(4)
The independent solution vectors
 1 
 
  3
  1
 
 1 
 1 
 
and
2
 
1
 -3 
 
0
0
 
of Rx=0 lay a basis for the kernel of A.
(5)
Rank(A) = Dimension [ colspace(A)]
= Dimension [ Image(A)]
= Number of basis vectors in the image of A = 3
(6)
Nullity of A = Dimension [ nullspace(A)]
5
= Dimension [ kernel ( A ) ]
= Number of basis vectors in the kernel of A = 2
And R = rref (A) tells much more.
The objectives of this problem are:
(1)
To delve into the structural characteristics and fundamental subspace information
rref returns about the rank r matrix A; and
(2)
To relate the rref computed response with the structural characteristics and
fundamental subspace information contained in the singular value decomposition
of A.
The singular value decomposition of the (mxn) matrix A is
A  UΣ V *
(3)
i.e.,
[U,S,V] = svd(A)
where U(mxn) and V(nxn) are unitary and Σ(mxn) is a real diagonal (staircase)
matrix. The (i, i) element of Σ is the i-th singular value  of A. The rank r of A equals
i
the number of non-zero singular values.
II.
REDUCED ROW ECHELON FORM
We begin in this section by defining the reduced row echelon form R = rref(A).
Then, in Section III, we outline the MATLAB computation and interpretation of R.
The reduced row echelon form of a rectangular matrix satisfies the following five
requirements:
(1)
All entries in a column below a leading entry (the first non-zero entry in a row) are
zero.
(2)
The leading entry in row i lies to the right of the leading entry in row (i-1).
(3)
Rows of all zeros (if any) are grouped at the bottom.
(4)
The leading entry in each row is one.
(5)
All entries above each leading entry are zero.
An equivalent set of three requirements for the reduced row echelon form of a
rectangular matrix is:
6
(1)
The first non-zero element in each row is a one. These elements of the matrix are
called the pivot elements or pivots. The column in which a pivot element appears is
called a pivot column. The rank of the matrix equals the number of pivots.
(2)
The pivot elements are the only non-zero elements in the pivot columns.
(3)
Rows of all zeros (if any) are grouped at the bottom of the matrix.
For example,
 1 1 2 3


U   0 0 2 2
 0 0 0 0


is an echelon (staircase) matrix (but not a reduced row echelon matrix) whereas
 1 1 0 1


R   0 0 1 1
 0 0 0 0


is a reduced row echelon matrix.
III.
rref IN MATLAB®
The function R = rref(A) in MATLAB® and Maple® with the linear algebra
package (>with(linalg);) applies Gauss-Jordan elimination with partial pivoting to
compute the reduced row echelon form of A. Gaussian elimination strikes again. The
function R = sym(rref(A)) returns the infinitely precise reduced row echelon form
of the matrix A. The function [R, pivcol] = rref(A) produces (in pivcol) the
list of the numbers of the pivot columns in A and R.
For the rank three matrix A(3x5) in (1), R = sym(rref(A)) returns the
reduced row echelon form in (2) and [R, pivocol] = rref(A) returns the list
[1 2 4] of the numbers of the pivot columns in A and R. The function rref leads to the
following information:
(i)
r = length (pivocol) returns 3
(ii)
A(:, pivocol) returns the matrix
 1 4 3


  1 2 1
 2 5 0


containing the pivot columns of A and
7
(iii) R(1:r, pivcol) returns the matrix
1

0
0

0
1
0
0

0
1 
containing the pivot columns of R.
IV.
FOUR FUNDAMENTAL SUBSPACES OF A
The reduced row echelon form R = rref(A) and singular value
decomposition svd(A) in (3) reveal the four fundamental subspaces of A.
(1)
Column space C(A) of A – The first r columns of U, where r = rank(A) = number
of non-zero singular values of A. The r pivot columns of A returned by pivot lay
a basis for C(A).
(2)
Row space C(AT) of A – The first r columns of V. The first r rows of R lay a basis
for C(AT).
(3)
Null space N(A) of A – The last (n - r) columns of V. The (n - r) solutions of Rx = 0
lay a basis for N(A).
(4)
Null space N(AT) of AT – The last (m - r) columns of U. The last (m - r) rows of E [1]
and [2] lay a basis for N(AT).
V.
TASKS
(A) Highlight in matrix format the four fundamental subspaces reviewed in Section IV.
Columns of V
–––––––––
eigenvectors of ATA
Columns of U
–––––––––
eigenvectors of AAT
First r columns of V
––––––
row space of A
Last (n-r) columns of V
––––––
null space of A
First r columns of U
––––––
column space of A
Last (m-r) columns of U
––––––
null space of AT
8
Av = σu
Column space Null space 

U  
of A
of A


r
  m  r 

Pivot columns
 
of A

Last (m  r) 
rows of E 
 orth(A) null(AT)
 

 r   m  r  
(B) Apply the MATLAB® functions rref and sym to compute in detail the examples
in [2]. Extract the four fundamental subspaces of
2
A  
1
6

3 
Compute the svd of A. Relate algorithmically and computationally the
fundamental subspace information returned by rref and svd.
(C) Repeat Task (B) for the rank three matrix A(3x5) in (1).
(D) Highlight your findings. Apply MATLAB® to substantiate your findings. Draw your
conclusions.
REFERENCES
[1]
G. Strang, Introduction to Linear Algebra. Second Edition. Wellesley, MA:
Wellesley-Cambridge, 1998.
[2]
S. L. Lee and G. Strang, “Row Reduction of a Matrix and A = CaB,” American
Mathematical Monthly, Vol. 107, No. 8, October 2000, pp. 681 – 688.
[3]
http://web.mit.edu/18.06