Orthogonal Complements
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
1 / 22
The orthogonal complement of a vector space S ⊆ Rn , denoted
S ⊥ , is
S ⊥ = {x ∈ Rn : x0 y = 0
∀ y ∈ S}.
Is S ⊥ is a vector space?
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
2 / 22
∀ c1 , c2 ∈ R, x1 , x2 ∈ S ⊥ ,
(c1 x1 + c2 x2 )0 y = c1 x10 y + c2 x02 y = 0
whenever y ∈ S.
Thus
x1 , x2 ∈ S ⊥ , c1 , c2 ∈ R =⇒ c1 x1 + c2 x2 ∈ S ⊥ .
It follows that S ⊥ is a vector space.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
3 / 22
Suppose a vector space S is a subset of Rn . Show the following:
(a) S ∩ S ⊥ = {0}
(b) dim(S) + dim(S ⊥ ) = n.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
4 / 22
Proof of (a)
Suppose x ∈ S ∩ S ⊥ .
Then x0 x = 0, which implies that
n
X
xi2 = 0 =⇒ xi = 0 ∀ i = 1, . . . , n
i=1
=⇒ x = 0.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
5 / 22
Proof of (b)
Let r = dim(S) and let a1 , . . . , ar be a basis for S.
If we define
A = [a1 , . . . , ar ],
then C(A) = S.
Now note that S ⊥ = N (A0 ) as follows:
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
6 / 22
x ∈ S ⊥ =⇒ x0 ai = 0
∀ i = 1, . . . , r
=⇒ x0 [a1 , . . . , ar ] = 00
=⇒ x0 A = 00 =⇒ A0 x = 0
=⇒ x ∈ N (A0 ) ∴ S ⊥ ⊆ N (A0 ).
Conversely,
x ∈ N (A0 ) =⇒ A0 x = 0
=⇒ x0 A = 00
=⇒ x0 Ay = 0 ∀ y ∈ Rr
=⇒ x0 z = 0 ∀ z ∈ C(A) = S
=⇒ x ∈ S ⊥ .
∴ N (A0 ) ⊆ S ⊥ and we have N (A0 ) = S ⊥ .
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
7 / 22
From Theorem A.1,
dim(N (A0 )) = n − rank(A0 )
= n − rank(A)
= n − r.
Therefore,
dim(S) + dim(S ⊥ ) = dim(C(A)) + dim(N (A0 ))
= r+n−r
= n.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
8 / 22
Result A.4:
Suppose a vector space S ⊆ Rn . Then any x ∈ Rn can be written as
x = s + t,
where s ∈ S, t ∈ S ⊥ . Furthermore, the decomposition is unique.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
9 / 22
Proof of Result A.4:
Let a1 , . . . , ar be a basis for S. Let b1 , . . . , bk be a basis for S ⊥ . We
know that r + k = n.
We now show that a1 , . . . , ar , b1 , . . . , bk is a set of LI vectors and is
thus a basis for Rn by V4.
Suppose
c1 a1 + · · · + cr ar + cr+1 b1 + · · · + cn bk = 0.
Then
c1 a1 + · · · + cr ar = −(cr+1 b1 + · · · + cn bk ).
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
10 / 22
Moreover,
c1 a1 + · · · + cr ar ∈ S and − (cr+1 b1 + · · · + cn bk ) ∈ S ⊥ .
Because these two vectors are equal each other, they are in S
and S ⊥ and must be 0 ∵ S ∩ S ⊥ = {0}.
By LI of a1 , . . . , ar , we have c1 = · · · = cr = 0.
By LI of b1 , . . . , bk , we have cr+1 = · · · = cn = 0.
Thus, a1 , . . . , ar , b1 , . . . , bk are LI and a basis for Rn .
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
11 / 22
This implies that any x ∈ Rn may be written as
x = d1 a1 + · · · + dr ar + dr+1 b1 + · · · + dn bk
for some d1 , . . . , dn ∈ R.
If we take
s = d1 a1 + · · · + dr ar and t = dr+1 b1 + · · · + dn bk ,
we have s ∈ S, t ∈ S ⊥ and x = s + t.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
12 / 22
To prove the uniqueness, suppose
x = s1 + t1 = s2 + t2 ,
where s1 , s2 ∈ S and t1 , t2 ∈ S ⊥ .
Now show that s1 = s2 and t1 = t2 to complete the proof.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
13 / 22
s1 + t1 = s2 + t2 =⇒ s1 − s2 = t2 − t1 .
Because S and S ⊥ are vectors spaces, s1 − s2 ∈ S,
t2 − t1 ∈ S ⊥ .
The equality of these vectors implies that
s1 − s2 = t2 − t1 ∈ S ∩ S ⊥ =⇒ s1 − s2 = t2 − t1 = 0.
Therefore, s1 = s2 ,
t2 = t1 .
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
14 / 22
Suppose A =
Suppose A =
" #
1
0
" #
1
1
. Find and sketch C(A) and N (A0 ).
. Find and sketch C(A) and N (A0 ).
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
15 / 22
If A =
" #
1
0
, then C(A) = {x ∈ R2 : x2 = 0}.
N (A0 ) = {x ∈ R2 : [1, 0]x = 0}
= {x ∈ R2 : x1 = 0}.
c
Copyright 2012
Dan Nettleton (Iowa State University)
X2
N(A')
s+t
t
C(A)
s
X1
Statistics 611
16 / 22
If A =
" #
1
1
, then C(A) = {x ∈ R2 : x1 = x2 }.
N (A0 ) = {x ∈ R2 : [1, 1]x = 0}
= {x ∈ R2 : x1 + x2 = 0}.
c
Copyright 2012
Dan Nettleton (Iowa State University)
X2
C(A)
s
s+t
X1
t
N(A')
Statistics 611
17 / 22
Result A.5:
If A is an m × n matrix, then C(A)⊥ = N (A0 ).
Proof of Result A.5:
We essentially made this argument in our proof that
dim(S) + dim(S ⊥ ) = n
if S ∈ Rn is a vector space.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
18 / 22
Result A.6:
Suppose S1 , S2 are vector spaces in Rn such that S1 ⊆ S2 . Then
S2⊥ ⊆ S1⊥ .
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
19 / 22
Proof of Result A.6:
Suppose x ∈ S2⊥ .
Then
x0 y = 0
∀ y ∈ S2
=⇒ x0 y = 0
∀ y ∈ S1
=⇒ x ∈ S1⊥ .
We have S2⊥ ⊆ S1⊥ .
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
20 / 22
Suppose S is a vector space in Rn . Prove that S = (S ⊥ )⊥ .
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
21 / 22
Proof
x ∈ S =⇒ x0 y = 0 ∀ y ∈ S ⊥ =⇒ x ∈ (S ⊥ )⊥ . ∴ S ⊆ (S ⊥ )⊥ .
Now suppose x ∈ (S ⊥ )⊥ . Then x0 y = 0 ∀ y ∈ S ⊥ .
By Result A.4, x = s + t for some s ∈ S and some t ∈ S ⊥ .
By the previous two points, 0 = x0 t = (s + t)0 t = t0 t. ∴ t = 0 and
x = s ∈ S so that (S ⊥ )⊥ ⊆ S.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
22 / 22