Lecture 8: Subspaces
De…nition 8.1 A set H of Rn is called a subspace of Rn if H is closed under linear
operations. More precisely, H is a subspace if it meets the following three criteria:
1. H contains ~0:
2. For any two vectors ~u and ~v in H; ~u+ ~v is in H; (i.e., H is "closed" under additions;
we could also say that additions within H would not expand H or produce anything
more.)
3. For any vector ~u 2 H =) ¸~u 2 H for any number ¸: (i.e., H is closed under scalar
multiplication. By taking ¸= 0; we see that this includes the …rst criterion.)
n o
Example 8.1 (1) ~0 and Rn itself are always subspaces of Rn , and are called the trivial
spaces.
(2) Graphically, a subspace is a hyperplane passing through the origin. In R2 ; a hyperplane is a straight. In R3 ; a hyperplane is either a straight (with co-dimension two) or a
plane (with co-dimension one).
(3) In R2 ; the only non-trivial subspaces are lines passing through the origin.
(3) In R3 ; the only non-trivial subspaces are either lines passing through the origin or
planes passing through the origin.
(4)Let A be a matrix. The solution set M of A~x = ~0 is a subspace.
Proof: We verify that all three criteria hold as follows.
(i) A~0 = ~0 =) ~0 2 M
(ii) If ~u; ~v 2 M; then A (~u + ~v) = A~u + A~v = ~0 =) (~u + ~v) 2 M
(iii) If ~u 2 M; then A (¸~u) = ¸A~u = ¸~0 = ~0 =) ¸~u 2 M:
(5) Span f~v1 ; ~v2 ; :::; ~vp g is a subspace.
Proof: Recall that Span f~v1 ; ~v2 ; :::; ~vp g = all possible linear combinations of ~v1 ; ~v2 ; :::; ~vp :
(i) Obvious, ~0 = 0~v1 2 Span f~v1 ; ~v2 ; :::; ~vpg
(ii) If ~u; ~v 2 Span f~v1 ; ~v2 ; :::; ~vp g ; i.e.,
~u = a1~v1 + a2~v2 + ::: + ap~vp
~v = b1~v1 + b2~v2 + ::: + bp~vp ;
then
~u + ~v = (a1~v1 + a2~v2 + ::: + ap~vp ) + (b1~v1 + b2~v2 + ::: + bp~vp )
= (a1 + b1 ) ~v1 + (a2 + b2 ) ~v2 + ::: + (ap + bp) ~vp :
This means
(iii)
~u + ~v 2 Span f~v1 ; ~v2 ; :::; ~vpg :
¸~u = ¸a1~v1 + ¸a2~v2 + ::: + ¸ap~vp 2 Span f~v1 ; ~v2 ; :::; ~vp g :
1
(6) Any subspace in Rn is a subspace spanned by no more than n vectors.
De…nition 8.2. (a) Let A = [~a1 ; ~a2 ; :::; ~ap ]m£p be matrix that has p column vectors
~a1 ; ~a2 ; :::; ~ap : We call the space spanned by columns
Col (A) = Span f~a1 ; ~a2 ; :::; ~ap g
the column space.
(b) The null space Null (A) of A is a subspace of all solution A~x = ~0 :
n
o
~
Null (A) = ~x j A~x = 0 :
(c) If a subspace H is spanned by p vectors ~v1 ; ~v2 ; :::; ~vp ; i.e.,
H = Span f~v1 ; ~v2 ; :::; ~vp g ;
then we said that H is generated by ~v1 ; ~v2 ; :::; ~vp ; and these ~v1 ; ~v2 ; :::; ~vp are called a set of
generators for H.
(d) If in addition, generators ~v1 ; ~v2 ; :::; ~vp of a subspace
H = Span f~v1 ; ~v2 ; :::; ~vp g
are the linearly independent, then we call the set f~v1 ; ~v2 ; :::; ~vp g a basis for H: The number
of vectors in basis is called the dimension of H: In particular,
dim (Null (A)) = number of free variables.
Example 8.2. In Rn ; the columns of In form the standard basis:
2 3
2 3
2 3
0
1
0
617
607
6 7
6 .. 7
6 7
6 7
6 7
~e1 = 6 .. 7 ; ~e2 = 607 ; :::; ~en = 6 . 7 :
6 .. 7
4.5
405
4.5
0
1
0
If ~x = [x1 ; x2 ; :::; xn ]T ; then ~x = x1~e1 + x2~e2 +; :::; xn~en :
Example 8.3. Find a basis for (a) Null(A) and (b) Col (A) if
2
3
¡3 6 ¡1 ¡1 1 ¡7
2 3 ¡15 :
A = 4 1 ¡2 2
2 ¡4 5
5 8 ¡4
Solution: (a) We need to solve A~x = ~0: We …rst perform row operations:
2
2
3
2
¡3 6 ¡1 ¡1 1 ¡7
1 ¡2 2
2 3
4
5
4
A = 1 ¡2 2
2 3 ¡1 R1 ! R2 ¡3 6 ¡1 ¡1 1
¡¡¡¡¡¡!
2 ¡4 5
5 8 ¡4
2 ¡4 5
5 8
2
3
2
1 ¡2 2 2 3 ¡1
1
R2 + 3R1 ! R2 4
R ¡ 5R3 ! R2 4
0 0 5 5 10 ¡105 2
0
R3 ¡ 2R1 ! R3
R1 ¡ 2R3 ! R1
0 0 1 1 2 ¡2 ¡¡¡¡¡¡¡¡¡¡¡¡! 0
¡¡¡¡¡¡¡¡¡¡¡¡!
2
3
1 ¡2 0 0 1 1
R2 ¡ 5R3 ! R2 40 0 1 1 2 ¡25 :
¡¡¡¡¡¡¡¡¡¡¡!
0 0 0 0 0 0
3
¡1
¡75
¡4
3
¡2 0 0 1 1
0 0 0 0 05
0 1 1 2 ¡2
The corresponding system equivalent to A~x = ~0 becomes
x1 ¡ 2x2 + x5 + x6 = 0
x3 + x4 + 2x5 ¡ 2x6 = 0:
Since #2, #4, #5, #6 are non-pivot columns, x2 ; x4; x5 ; x6 are free variables, and they can
be solved as
x1 = 2x2 ¡ x5 ¡ x6
x3 = ¡x4 ¡ 2x5 + 2x6 ;
and solution has the parametric form
2 3 2
3
2 3
2 3
2 3
2 3
x1
2x2 ¡ x5 ¡ x6
2
0
¡1
¡1
6x2 7 6 x2
7
617
607
607
607
6 7 6
7
6 7
6 7
6 7
6 7
6x3 7 6¡x4 ¡ 2x5 + 2x6 7
607
6¡17
6¡27
627
6
7
6
7
6
7
6
7
6
7
6 7:
~x = 6 7 = 6
=
x
+
x
+
x
+
x
2
4
5
6
7
607
617
607
607
x
x
4
4
6 7 6
7
6 7
6 7
6 7
6 7
4x5 5 4
5
405
405
415
405
x5
x6
x6
0
0
0
1
These 4 vectors are linearly independent and form a basis, and
82 3 2 3 2 3 2 39
2
0
¡1
¡1 >
>
>
>
>
>617 6 0 7 6 0 7 6 0 7>
>
>
>
>
6
7
6
7
6
7
6
7
<6 7 6 7 6 7 6 7>
=
07 6¡17 6¡27 6 2 7
6
Null (A) = Span 6 7 ; 6 7 ; 6 7 ; 6 7 ;
>
>607 6 1 7 6 0 7 6 0 7>
>
>
>
>
405 4 0 5 4 1 5 4 0 5>
>
>
>
>
:
;
0
0
0
1
and dim (N ull (A)) = 4:
(b) From (a)
2
3
2
3
¡3 6 ¡1 ¡1 1 ¡7
1 ¡2 0 0 1 1
2 3 ¡15 ¡! 40 0 1 1 2 ¡25 = B
A = 4 1 ¡2 2
2 ¡4 5
5 8 ¡4
0 0 0 0 0 0
3
Since in B; ~b1 and ~b3 , the pivot columns of B; are linearly independent, and since ~b2 ; ~b4 ; ~b5 ; ~b6
are all linear combination of ~b1 and ~b3 ; we see that ~b1 and ~b3 form a basis and
n
o
Col (B) = Span ~b1 ; ~b3 :
Note that row operations produce equivalent systems (i.e., systems have the same solutions),
row operations will not change linear relations. More precisely, if x1~a1 + x2~a2 + x3~a3 + x4~a4 +
x5~a5 + x6~a6 = ~0; then x1~b1 + x2~b2 + x3~b3 + x4~b4 + x5~b5 + x6~b6 = ~0: We hence conclude
(a) ~a1 and ~a3 are linearly independent, since ~b1 and ~b3 are linearly independent, and
(b) ~a2 ; ~a4 ; ~a5 ; ~a6 are linear combination of ~a1 and ~a3 (e.g. ~a2 = 2~a1 +3~a3 ), since ~b2 ; ~b4 ; ~b5 ; ~b6
are linear combination of ~b1 and ~b3 (e.g., ~b2 = 2~b1 + 3~b3 ):
and consequently
Col (A) = Span f~a1 ; ~a3 g
and
2 3
2 3
¡3
¡1
4 1 5 and 4 2 5 form a basis.
2
5
² Summary: Basis of Col (A) = all pivot columns of A:Basis of Null (A) =
vectors associated with free variables in a parametric vector form.
Example 8.4. Suppose that
2
3
2
3
1 ¡2 0 0 1 1
1 ¡2 0 0 1 1
6 2 ¡4 1 1 4 0 7
6
7
7 ¡! 60 0 1 1 2 ¡27 :
A=6
4¡1 2 0 1 9 195
40 0 0 1 10 20 5
0
0 0 0 0 0
0 0 0 0 0 0
82 3 2 3 2 39
1
0
0 >
>
>
<6 7 6 7 6 7>
=
0
1
7 ; 6 7 ; 617 ???
Then Col (A) = Span 6
405 405 415>
>
>
>
:
;
0
0
0
Correct answer
82 3 2 3 2 39
1
0
0 >
>
>
<6 7 6 7 6 7>
=
2
1
7 ; 6 7 ; 617 :
Col (A) = Span 6
4¡15 405 415>
>
>
>
:
;
0
0
0
² Dimensions.
As we know, any subspace has the form H = Span f~v1 ; ~v2; :::; ~vp g : When the generators ~v1 ; ~v2; :::; ~vp are linearly independent, they form a basis for H. A subspace may
4
have in…nite many sets of basis, for instance, suppose that ~v1 ; ~v2; :::; ~vp form a basis of
H = Span f~v1 ; ~v2;:::; ~vp g ; then all of the following sets are also bases:
2~v1 ; 2~v2; :::; 2~vp
2~v1 ; ~v2; :::; ~vp
(~v1 + 2~v2 + 5~v3 ) ; ~v2;:::; ~vp :
However, the number p of vectors in a basis remain the same. Furthermore, if ~v1 ; ~v2; :::; ~vp
form a basis, then for any ~x 2 H; we have the unique linear relation
~x = c1~v1 + c2~v2 + ::: + cp~vp:
(1)
In fact, if there is another expression for the same ~x :
~x = d1~v1 + d2~v2 + ::: + dp~vp ;
then
c1~v1 + c2~v2 + ::: + cp~vp = ~x = d1~v1 + d2~v2 + ::: + dp~vp;
and thus
(c1 ¡ d1 ) ~v1 + (c2 ¡ d2 ) ~v2 + ::: + (cp ¡ dp) ~vp = 0:
Since ~v1 ; ~v2; :::; ~vp are linearly independent,
c1 ¡ d1 = 0; (c2 ¡ d2 ) = 0; :::; cp ¡ dp = 0;
=)
c1 = d1 ; c2 = d2 ; :::; cp = dp
i.e., the linear relation is unique
De…nition 8.3. Let H be a subspace of Rn with a basis B = f~v1 ; ~v2; :::; ~vp g of p linearly
independent vectors. We call p; the number of vectors in a basis, the dimension of H, and
denote it by
p = dim (H) :
For any vector ~x in H, there is a unique expression
2 3
c1
6c2 7
6 7
~x = c1~v1 + c2~v2 + ::: + cp~vp = [~v1 ; ~v2; :::; ~vp] 6 .. 7 :
4.5
cp
This unique column vector of the linear relation is called the coordinate of ~x relative to the
basis B (or B¡coordinate, in short), and is denoted by
2 3
c1
6c2 7
6 7
[~x]B = 6 .. 7 ; (B¡coordinate of ~x);
4.5
cp
5
i.e.,
~x = [~v1 ; ~v2; :::; ~vp ] [~x]B
Example 8.5. In Rn ; the standard basis B = f~e1 ; ~e2 ; :::; ~en g consists of n vectors. So,
dim(Rn ) = n: For any vector
2 3
2 3
x1
x1
6 x2 7
6 x2 7
6 7
6 7
~x = 6 .. 7 2 Rn ; [~x]B = 6 .. 7 :
4.5
4.5
xn
xn
Example 8.6. Consider R2 : Let
· ¸
· ¸
· ¸
· ¸
1
¡1
1
0
~v1 =
; ~v2 =
; ~e1 =
; ~e2 =
:
2
1
0
1
Find (a) [ ~e1 ]f~v1 ;~v2 g and [ ~e2 ]f~v1 ;~v2 g ; (b) [ ~x]f~v1 ;~v2 g for any ~x:
Solution: To …nd [ ~e1 ]f~v1 ;~v2 g ; we need to solve ~e1 = x~v1 + y ~v2 ; or
· ¸
· ¸
· ¸
1
1
¡1
=x
+y
:
0
2
1
We proceed by reducing the augmented matrix:
·
¸
·
¸
·
¸
1 ¡1 1
1 ¡1 1
1 0 1=3
!
!
:
2 1 0
0 3 ¡2
0 1 ¡2=3
Therefore,
µ ¶
1
2
~e1 = ~v1 + ¡
~v2 ;
3
3
[ ~e1 ]f~v1 ;~v2 g
·
¸
1=3
=
:
¡2=3
This relation can be written as
µ ¶
·
¸
1
2
1=3
~e1 = ~v1 + ¡
~v2 = [~v1 ; ~v2 ]
= [~v1 ; ~v2 ] [ ~e1 ]f~v1 ;~v2 g :
¡2=3
3
3
Similarly, by solving ~e2 = x~v1 + y ~v2 ; or by row operations
·
¸
·
¸
·
¸
1 ¡1 0
1 ¡1 0
1 0 1=3
!
!
;
2 1 1
0 3 1
0 1 1=3
we found
[ ~e2 ]f~v1 ;~v2 g
We may also write the above relations as
· ¸ · ¸
x
1=3
=
=
:
y
1=3
· ¸
1
1
1=3
~e2 = x~v1 + y ~v2 = ~v1 + ~v2 = [~v1 ; ~v2 ]
= [~v1 ; ~v2 ] [ ~e2 ]f~v1 ;~v2 g :
1=3
3
3
6
From this, we see that the 2 £ 2 matrix with columns ~e1 ; ~e2 ;
h
i
h
i
[~e1 ; ~e2 ] = [~v1 ; ~v2 ] [ ~e1 ]f~v1 ;~v2 g ; [~v1 ; ~v2 ] [ ~e2 ]f~v1 ;~v2 g = [~v1 ; ~v2 ] [ ~e1 ]f~v1 ;~v2 g ; [ ~e2 ]f~v1 ;~v2 g = [~v1 ; ~v2 ] V;
where the matrix
2
3
1
1
h
i
6
7
V = [ ~e1 ]f~v1 ;~v2 g ; [ ~e2 ]f~v1 ;~v2 g = 4 32 31 5
¡
3 3
represents
· ¸ changes of coordinates from the standard basis to basis f~v1 ; ~v2 g : In fact, for any
x
~x = 1 ; we have
x2
µ
µ ¶ ¶
µ
¶
1
2
1
1
~x = x1~e1 + x2~e2 = x1
~v1 + ¡
~v2 + x2
~v1 + ~v2
3
3
3
3
µ
¶
µµ ¶
¶
1
1
2
1
=
x1 + x2 ~v1 +
¡
x1 + x2 ~v2 :
3
3
3
3
Using matrix notation,
~x = [~e1 ; ~e2 ] ~x = [~v1 ; ~v2 ] V ~x:
Hence,
[~x]f~v1 ;~v2 g
2 1
3
1
¸· ¸
x1 + x2
1=3 1=3 x1
6
7
= V ~x =
= 4µ 32 ¶ 3 1 5 :
¡2=3 1=3 x2
¡
x1 + x2
3
3
·
² Basis Changing Matrix
Note that in the above example, to …nd the coordinate, we need to solve
· ¸
· ¸
1
¡1
x1
+ x2
= ~e1
2
1
· ¸
· ¸
1
¡1
y1
+ y2
= ~e2 ;
2
1
or in matrix equation form
·
¸· ¸ · ¸
1 ¡1 x1
1
=
2 1
x2
0
·
¸· ¸ · ¸
1 ¡1 y1
0
=
:
2 1
y2
1
7
These two equations may be combined into
·
¸·
¸ ·
¸
1 ¡1 x1 y1
1 0
=
:
2 1
x2 y2
0 1
Therefore, the basis change matrix can be found as
·
¸¡1
1 ¡1
V =
:
2 1
² Dimension Theorem
Recall the following summary discussed early in this lecture:
Summary: Basis of Col (A) = all pivot columns of A:Basis of Null (A) = vectors
associated with free variables in a parametric vector form.
Recall also that the rank r (A) of a matrix A is the number of pivot columns in A: Thus
dim (Col (A)) = r (A) = number of pivots
dim (Null (A)) = number of non-pivot columns.
It follows that
r (A) + dim (N ull (A))
= (number of pivots) + (number of non-pivot columns)
= total number of columns.
This leads to
Dimension Theorem Let Am£n be a m £ n matrix. Then
r (A) + dim (Null (A))
= dim (Col (A)) + dim (Null (A))
= n:
Example 8.7. Find r (A) and dim (Null (A))
2 ¤
2
5 ¡3
6 0 ¡3¤ 2
As6
40
0
0
0
0
0
if
3
¡4 8
5 ¡77
7:
4¤ ¡65
0
0
Solution: We see that column#1, #2, #4 are pivot. Therefore,
r (A) = 3;
dim (N ull (A)) = 5 ¡ 3 = 2:
8
² Homework #8
2 3
2 3
2 3
2 3
1
4
5
¡4
6¡27
6¡77
6¡87
667
7
6 7
6 7
7
1. Let ~v1 = 6
~ =6
4 4 5 ; ~v2 = 4 9 5 ; ~v3 = 4 6 5 ; and w
4¡25.
3
7
5
¡2
(a) Determine whether w
~ is in the subspace spanned ~v1 ; ~v2 ; ~v3 :
(b) Find a basis and dimension of span f~v1 ; ~v2 ; ~v3 g. What is the dimension of span f~v1 ; ~v2 ; ~v3 ; wg?
~
(c) Find the coordinate of w
~ relative to the basis you found in part (b).
· ¸
· ¸
¡2
1
2. Let ~u1 =
; ~u2 =
:
3
2
· ¸
· ¸
1
0
(a) Find the coordinate of ~e1 =
and ~e1 =
relative to the basis B = f~u1 ; ~u2 g :
0
1
(b) Find the matrix V that convert the standard basis to B; i:e:;
· ¸
· ¸
· ¸
x
x
x
=V
for any vector
:
y B
y
y
3. Following exercises display a matrix A and an echelons form of matrix A. Find a basis
and dimension of Col (A) and a basis of dimension of N ull (A) :
2
3 2
3
4 5 9 ¡2
1 1 1 1
(a) A = 46 5 1 12 5 » 40 1 5 ¡65 :
3 4 8 ¡3
0 0 0 0
2
3 2
3
1 4 8 ¡3 ¡7
1 4 8 0 5
6¡1 2 7 3
6
7
47
7 » 60 2 5 0 ¡17 :
(b) A = 6
4¡2 2 9 5
5 5 40 0 0 1 4 5
3 6 9 ¡5 ¡2
0 0 0 0 0
4. For each statement below, determine whether it is true or false. If it is false, provide
a counterexample. If true, explain why.
(a) The dimension of Span f~v1 ; ~v2 ; ¢ ¢ ¢ ; ~v10 g equals to 10:
(b) The columns of an n £ n invertible matrix form a basis for Rn :
(c) Row operations do not a¤ect linear dependence relations among the columns of
a matrix.
(d) In R3 ; every plane is a subspace.
(e) The null space of a m £ n matrix is a subspace of Rm :
(f) The dimension of the column space of a matrix equals to its rank.
(g) The dimension of Null (A) is the number of pivots of A:
9