Chapter 4. General Vector Spaces
Lecturer KIM NAMHYUNG
4.6. Change of Basis
Solution. (Change-of-Basis Problem)
If we change the basis for a vector space 𝑉 from an old basis 𝐵 = {𝑢1 , 𝑢2 , … , 𝑢𝑛 }
to a new basis 𝐵′ = {𝑢′1 , 𝑢′ 2 , … , 𝑢′ 𝑛 } , then for each vector 𝑣 ∈ 𝑉 , the old
coordinate vector [𝑣]𝐵 is related to the new coordinate vector [𝑣]𝐵′ by the
equation
[𝑣]𝐵 = 𝑃𝐵′ →𝐵 [𝑣]𝐵′
where 𝑃𝐵′ →𝐵 is called the transition matrix and the column vectors of 𝑃 are
[𝑢′1 ]𝐵 , [𝑢′ 2 ]𝐵 , … , [𝑢′ 𝑛 ]𝐵
Theorem 4.6.1.
If 𝑃 is the transition matrix from a basis 𝐵′ to a basis 𝐵 for a finitedimensional vector space 𝑉, then 𝑃 is invertible and 𝑃 −1 is the transition
matrix from 𝐵 to 𝐵′ .
Procedure. (Computing Transition Matrices)
[new basis | old basis] → [𝐼 | transition from old to new]
Theorem 4.6.2.
Let 𝐵′ = {𝑢1 , 𝑢2 , … , 𝑢𝑛 } be any basis for the vector space 𝑅𝑛 and let 𝑆 = {𝑒1 , 𝑒2 , … , 𝑒𝑛 }
be the standard basis for 𝑅𝑛 . Then
𝑃𝐵′ →𝑆 = [𝑢1
𝑢2
⋯
𝑢𝑛 ]
Chapter 4. General Vector Spaces
Lecturer KIM NAMHYUNG
4.7. Row Space, Column Space, and Null Space
Definition 1.
For an 𝑚 × 𝑛 matrix
𝑎11
𝐴=[ ⋮
𝑎𝑚1
⋯
⋱
⋯
𝑎1𝑛
⋮ ]
𝑎𝑚𝑛
the vectors
𝑟1
⋮
𝑟𝑛
=
=
=
[𝑎11
[𝑎𝑚1
⋯
⋮
⋯
𝑎1𝑛 ]
𝑎𝑚𝑛 ]
in 𝑅𝑛 that are formed from the rows of 𝐴 are called the row vectors of 𝐴,
and the vectors
𝑎11
𝑎1𝑛
⋮
𝑐1 = [
] , … , 𝑐𝑛 = [ ⋮ ]
𝑎𝑚1
𝑎𝑚𝑛
in 𝑅𝑚 formed from the columns of 𝐴 are called the column vectors of 𝐴.
Definition 2.
If 𝐴 is 𝑚 × 𝑛 matrix, then the subspace of 𝑅𝑛 spanned by the row vectors of
𝐴 is called the row space of 𝐴, and the subspace of 𝑅𝑚 spanned by the column
vectors 𝐴 is called the column space of 𝐴. The solution space of the
homogeneous system of equations 𝐴𝑥 = 0, which is a subspace of 𝑅𝑛 , is called
the null space of 𝐴.
Theorem 4.7.1.
A system of linear equations 𝐴𝑥 = 𝑏 is consistent if and only if 𝑏 is in the
column space of 𝐴.
Theorem 4.7.2.
if 𝑥0 is any solution of a consistent linear system 𝐴𝑥 = 𝑏, and if 𝑆 =
{𝑣1 , 𝑣2 , … , 𝑣𝑘 } is a basis for the null space of 𝐴, then every solution of 𝐴𝑥 = 𝑏
can be expressed in the form
𝑥 = 𝑥0 + 𝑐1 𝑣1 + 𝑐2 𝑣2 + ⋯ + 𝑐𝑘 𝑣𝑘
Conversely, for all choices of scalars 𝑐1 , 𝑐2 , … , 𝑐𝑘 , the vector 𝑥 in this
formula is a solution of 𝐴𝑥 = 𝑏. The vector 𝑥0 in that formula is called a
particular solution of 𝐴𝑥 = 𝑏, and the remaining part of the formula is
called the general solution of 𝐴𝑥 = 0.
Theorem 4.7.3.
Elementary row operations do not change the null space of a matrix.
Theorem 4.7.4.
Elementary row operations do not change the row space of a matrix.
Proof.
Let 𝐴 = [𝑟1 , 𝑟2 , … , 𝑟𝑚 ]. The permutating operation do not affect to the row
Chapter 4. General Vector Spaces
Lecturer KIM NAMHYUNG
space. The multiplying scalar operation on 𝑟𝑖 creates the new row vectors
𝑟1 , 𝑟2 , … , λ𝑟𝑖 , … , 𝑟𝑚 also span the same row space of 𝐴. The last kind of
elementary row operation, replacing 𝑟𝑗 with 𝑟𝑘 + λ𝑟𝑙 , we can write
α1 𝑣1 + ⋯ + α𝑛 𝑣 = α1 𝑣1 + ⋯ + α𝑘−1 𝑣𝑘−1 + α𝑘 (𝑣𝑘 + λ𝑣𝑙 ) + α𝑘+1 𝑣𝑘+1 + ⋯ + (α𝑙 − α𝑘 λ)𝑣𝑙 + ⋯ + α𝑛 𝑣𝑛
Therefore, the row space not changed.
Theorem 4.7.5.
If a matrix 𝑅 is in row echelon form, then the row vectors with the leading
1’s (the nonzero row vectors) form a basis for the row space of 𝑅, and the
column vectors with the leading 1’s of the row vectors form a basis for
the column space of 𝑅.
Proof.
Row space: a zero row does not contribute the span of row vectors.
Column space: the column not containing the leading 1 can be expressed
as linear combination of the other column that contains the leading 1.
Theorem 4.7.6.
If 𝐴 and 𝐵 are row equivalent matrices, then:
(a)
A given set of column vectors of 𝐴 is linearly independent if and only
if the corresponding column vectors of 𝐵 are linearly independent.
(b)
A given set of column vectors of 𝐴 forms a basis for the column space
of 𝐴 if and only if the corresponding column vectors of 𝐵 form a basis
for the column space of 𝐵.
Proof.
(a)
(b)
Let an invertible matrix 𝑅 such that
𝐴 = 𝑅𝐵
since 𝐴~𝐵.
Let 𝐴 = [𝑎1 𝑎2 ⋯ 𝑎𝑛 ] and 𝐵 = [𝑏1 𝑏2 ⋯ 𝑏𝑛 ] , then 𝑎𝑖 = 𝑅𝑏𝑖 for 𝑖 = 1,2, … , 𝑛 .
Let {𝑎𝑡1 , 𝑎𝑡2 , … , 𝑎𝑡𝑘 } be a basis of col(𝐴). It is sufficient to show that
{𝑏𝑡1 , 𝑏𝑡2 , … , 𝑏𝑡𝑘 } is a basis of col(𝐵).
1) {𝑏𝑡1 , 𝑏𝑡2 , … , 𝑏𝑡𝑘 } is linearly independent.
𝑐1 𝑏𝑡1 + 𝑐2 𝑏𝑡2 + ⋯ + 𝑐𝑘 𝑏𝑡𝑘 = 0
⇒ 𝑐1 𝑅𝑏𝑡1 + 𝑐2 𝑅𝑏𝑡2 + ⋯ + 𝑐𝑘 𝑅𝑏𝑡𝑘 = 0
⇒ 𝑐1 𝑎𝑡1 + 𝑐2 𝑎𝑡2 + ⋯ + 𝑐𝑘 𝑎𝑡𝑘 = 0
⇒ 𝑐1 = 𝑐2 = ⋯ = 𝑐𝑘 = 0
2) {𝑏𝑡1 , 𝑏𝑡2 , … , 𝑏𝑡𝑘 } spans col(𝐵).
Chapter 4. General Vector Spaces
Lecturer KIM NAMHYUNG
It is sufficient to show that all 𝑏𝑖 can be expressed as a linear
combination of {𝑏𝑡1 , 𝑏𝑡2 , … , 𝑏𝑡𝑘 }.
For all 𝑎𝑖 , there exists (𝑐1 , 𝑐2 , … , 𝑐𝑘 ) such that 𝑎𝑖 = 𝑐1 𝑎𝑡1 + 𝑐2 𝑎𝑡2 + ⋯ + 𝑐𝑘 𝑎𝑡𝑘
because {𝑎𝑡1 , 𝑎𝑡2 , … , 𝑎𝑡𝑘 } is a basis of col(𝐴). Thus there exist (𝑐1 , 𝑐2 , … , 𝑐𝑘 )
such that
𝑅𝑎𝑖 = 𝑐1 𝑅𝑎𝑡1 + 𝑐2 𝑅𝑎𝑡2 + ⋯ + 𝑐𝑘 𝑅𝑎𝑡𝑘 ⟺ 𝑏𝑖 = 𝑐1 𝑏𝑡1 + 𝑐2 𝑏𝑡2 + ⋯ + 𝑐𝑘 𝑏𝑡𝑘
Therefore {𝑏𝑡1 , 𝑏𝑡2 , … , 𝑏𝑡𝑘 } is a basis of col(𝐵).
Procedure. Basis for span(𝑆)
(i)
Form the matrix 𝐴 whose columns are the vectors in the set
𝑆 = {𝑣1 , 𝑣2 , … , 𝑣𝑘 }
(ii) Reduce the matrix 𝐴 to reduced row echelon form 𝑅.
(iii)
Denote the column vectors of 𝑅 by 𝑤1 , 𝑤2 , … , 𝑤𝑘 .
(iv) Identify the columns of 𝑅 that contain the leading
corresponding column vectors of 𝐴 form a basis for span(𝑆).
(v)
1’s.
The
Obtain a set of dependency equations by expressing each column vector
of 𝑅 that does not contain a leading 1 a s a linear combination of
preceding column vectors that do contain leading 1’s.
(vi) Replace the column vectors of 𝑅 that appear
equations by the correspond column vectors of 𝐴.
in
the
dependency
Chapter 4. General Vector Spaces
Lecturer KIM NAMHYUNG
4.8. Rank, Nullity, and the Fundamental Matrix Spaces
Theorem 4.8.1.
The row space and column space of a matrix 𝐴 have the same dimension.
Proof.
Let 𝑅 be any row echelon form of 𝐴. 𝑑𝑖𝑚(row space of 𝑅) is the number of
nonzero rows, and 𝑑𝑖𝑚(column space of 𝑅) is the number of leading 1’s, and
these two are the same. The dimension maintains for elementary row
operation, therefore 𝑑𝑖𝑚(row(𝐴)) = 𝑑𝑖𝑚(col(𝐴)).
Definition 1.
The common dimension of the row space and column space of a matrix 𝐴 is
called the rank of 𝐴 and is denoted by rank(𝐴); the dimension of the null
space of 𝐴 is called the nullity of 𝐴 and is denoted by nullity(𝐴).
Theorem 4.8.2. (Dimension Theorem for Matrices)
If 𝐴 is a matrix with 𝑛 columns, then
rank(𝐴) + nullity(𝐴) = 𝑛
Proof.
rank(𝐴) is the number of nonzero rows in 𝐴, and nullity(𝐴) is the number of
free variables, which is the number of zero rows in 𝐴. This yields the
sum of them is the number of columns of 𝐴, which is 𝑛.
Theorem 4.8.3
If 𝐴 is an 𝑚 × 𝑛 matrix, then
(a) rank(𝐴) is the number of leading variables in the general solution of
𝐴𝑥 = 0.
(b) nullity(𝐴) is the number of parameters in the general solution of 𝐴𝑥 = 0.
Theorem 4.8.4. (Equivalent Statements)
If 𝐴 is an 𝑛 × 𝑛 matrix, then the following statements are equivalent.
(a) 𝐴 is invertible.
(b) 𝐴𝑥 = 0 has only the trivial solution.
(c) The reduced row echelon form of 𝐴 is 𝐼𝑛 .
(d) 𝐴 is expressible as a product of elementary matrices.
(e) 𝐴𝑥 = 𝑏 is consistent for every 𝑛 × 1 matrix 𝑏.
(f) 𝐴𝑥 = 𝑏 has exactly one solution for every 𝑛 × 1 matrix 𝑏.
(g) 𝑑𝑒𝑡(𝐴) ≠ 0.
(h) The column vectors of 𝐴 are linearly independent.
(i) The row vectors of 𝐴 are linearly independent.
Chapter 4. General Vector Spaces
Lecturer KIM NAMHYUNG
(j) The column vectors of 𝐴 span 𝑅𝑛 .
(k) The row vectors of 𝐴 spans 𝑅𝑛 .
(l) The column vectors of 𝐴 form a basis for 𝑅𝑛 .
(m) The row vectors of 𝐴 form a basis for 𝑅𝑛 .
(n) 𝐴 has rank 𝑛.
(o) 𝐴 has nullity 0.
Proof.
(h) ⟹ (m): omit the details.
(b) ⟹ (o): There are no parameters in that solution, so 𝐴 has nullity
0.
(o) ⟹ (n): By the Dimension Theorem for Matrices, 𝐴 has rank 𝑛.
(n) ⟹ (b)
There are 𝑛 leading variables, hence no free variables in the general
solution of 𝐴𝑥 = 0. This leaves the trivial solution as the only
possibility.
Theorem 4.8.5.
If 𝐴𝑥 = 𝑏 is a consistent linear system of 𝑚 equations in 𝑛 unknowns, and if
𝐴 has rank 𝑟 , then the general solution of the system contains 𝑛 − 𝑟
parameters.
Theorem 4.8.6.
Let 𝐴 be an 𝑚 × 𝑛 matrix.
(a)
(Overdetermined Case). If 𝑚 > 𝑛 , then the linear
inconsistent for at least one vector 𝑏 in 𝑅𝑛 .
system 𝐴𝑥 = 𝑏 is
(b)
(Underdetermined Case). If 𝑚 < 𝑛, then for each vector 𝑏 is 𝑅𝑚 the
linear system 𝐴𝑥 = 𝑏 is either inconsistent or has infinitely many
solutions.
Proof.
(a) The column vectors for 𝐴 cannot span 𝑅𝑚 , thus ∃𝑏 ∉ col(A), and for that
𝑏 the system is inconsistent.
(b) If it is inconsistent, the proof is complete. If it is consistent,
then the general solution has 𝑛 − 𝑟 parameters, where 𝑟 = rank(𝐴) .
Therefore 𝑛 − 𝑟 = 𝑛 − 𝑚 > 0, and it implies that the general solution has
at least one parameter and hence there are infinitely many solutions.
Theorem 4.8.7.
If 𝐴 is any matrix, then rank(𝐴) = rank(𝐴𝑇 ).
Proof.
rank(𝐴) = 𝑑𝑖𝑚(row space of 𝐴) = 𝑑𝑖𝑚(column space of 𝐴𝑇 ) = rank(𝐴𝑇 )
Chapter 4. General Vector Spaces
Lecturer KIM NAMHYUNG
Definition 2.
If 𝑊 is a subspace of 𝑅𝑛 , then the set of all vectors in 𝑅𝑛 that are
orthogonal to every vector in 𝑊 is called the orthogonal complement of 𝑊
and is denoted by the symbol 𝑊 ⊥ .
Theorem 4.8.8.
If 𝑊 is a subspace of 𝑅𝑛 , then:
(a)
𝑊 ⊥ is a subspace of 𝑅𝑛 .
(b)
The only vector common to 𝑊 and 𝑊 ⊥ is 0.
(c)
The orthogonal complement of 𝑊 ⊥ is 𝑊.
Proof.
(a) 0 ∈ 𝑊 ⊥ and for an arbitrary 𝑣 ∈ 𝑊 ⊥ , ∀𝑢 ∙ 𝑣 = 0 such that 𝑢 ∈ 𝑊 ⇒ 𝑢 ∙ (−𝑣) = 0
therefore 𝑊 ⊥ is a subspace of 𝑅𝑛 .
(b) For an arbitrary 𝑢 ∈ 𝑊 ∩ 𝑊 ⊥ , 𝑢 ∙ 𝑢 = 0 ⇒ 𝑢 = 0.
(c)
∀𝑢 ∈ 𝑊, 𝑢 ∙ 𝑣 = 0 such that ∀𝑣 ∈ 𝑊 ⊥ by its definition, i.e. ∀𝑣 ∈ 𝑊 ⊥ , 𝑣 ∙ 𝑢 = 0
for ∀𝑢 ∈ 𝑊.
∴ ∀𝑢 ∈ 𝑊, 𝑢 ∈ (𝑊 ⊥ )⊥ ⟺ 𝑊 ⊂ (𝑊 ⊥ )⊥
∀𝑥 ∈ (𝑊 ⊥ )⊥ , 𝑥 = 𝑢 + 𝑣 such
(𝑊 ⊥ )⊥ , 𝑣 ∈ (𝑊 ⊥ )⊥
that 𝑢 ∈ 𝑊 and 𝑣 ∈ 𝑊 ⊥ ⇒ 𝑣 = 𝑥 − 𝑢 :
since 𝑊 ⊂
∴ 𝑣 ∈ (𝑊 ⊥ )⊥ ∩ 𝑊 ⊥ = {0}, 𝑣 = 0
Therefore, 𝑥 = 𝑢 and 𝑥 ∈ 𝑊 ∴ (𝑊 ⊥ )⊥ ⊂ 𝑊, hence 𝑊 = (𝑊 ⊥ )⊥ .
Theorem 4.8.9.
If 𝐴 is an 𝑚 × 𝑛 matrix, then:
(a) The null space of 𝐴 and the row space of 𝐴 are orthogonal complements
in 𝑅𝑛 .
(b) The null space
complements in 𝑅𝑚 .
of 𝐴𝑇 and
the
column
space
of 𝐴 are
orthogonal
Theorem 4.8.10. (Equivalent Statements)
If 𝐴 is an 𝑛 × 𝑛 matrix, then the following statements are equivalent.
(a) 𝐴 is invertible.
(b) 𝐴𝑥 = 0 has only the trivial solution.
(c) The reduced row echelon form of 𝐴 is 𝐼𝑛 .
(d) 𝐴 is expressible as a product of elementary matrices.
(e) 𝐴𝑥 = 𝑏 is consistent for every 𝑛 × 1 matrix 𝑏.
(f) 𝐴𝑥 = 𝑏 has exactly one solution for every 𝑛 × 1 matrix 𝑏.
Chapter 4. General Vector Spaces
Lecturer KIM NAMHYUNG
(g) 𝑑𝑒𝑡(𝐴) ≠ 0.
(h) The column vectors of 𝐴 are linearly independent.
(i) The row vectors of 𝐴 are linearly independent.
(j) The column vectors of 𝐴 span 𝑅𝑛 .
(k) The row vectors of 𝐴 spans 𝑅𝑛 .
(l) The column vectors of 𝐴 form a basis for 𝑅𝑛 .
(m) The row vectors of 𝐴 form a basis for 𝑅𝑛 .
(n) 𝐴 has rank 𝑛.
(o) 𝐴 has nullity 0.
(p) The orthogonal complement of the null space of 𝐴 is 𝑅𝑛 .
(q) The orthogonal complement of the row space of 𝐴 is {0}.