Orthogonal complement
Aim lecture: Inner products give a special way of constructing vector space
complements.
As usual, in this lecture F = R or C. We also let V be an F-space equipped with
an inner product (·|·).
Defn
Let S ⊆ V . We define the orthogonal complement to S to be
S ⊥ = {v ∈ V |v ⊥ S}
= ∩w∈S ker(w|·)
Hence S ⊥ is a subspace orthogonal to S & in particular, is closed under addition.
Proof. Clear.
E.g. This concept is easily understood in R3
Daniel Chan (UNSW)
Lecture 33: Orthogonal complements & projections
Semester 2 2012
1 / 10
Orthogonal complements of spans
Lemma
⊥
Let S ⊆ V . Then Span(S) = S ⊥
Proof.
w ∈ S ⊥ ⇐⇒ v ⊥ w for all v ∈ S
⇐⇒ w ⊥ v for all v ∈ S
⇐⇒ S ⊆ ker(w|·)
⇐⇒ Span(S) ⊆ ker(w|·)
⇐⇒ w ∈ Span(S)⊥
This completes the proof.
E.g. The orthogonal complement to S = Span((1, 1, 0)T , (0, 1, 1)T ) is
Daniel Chan (UNSW)
Lecture 33: Orthogonal complements & projections
Semester 2 2012
2 / 10
Orthogonal (internal) direct sums
Prop-Defn
Let W1 , . . . , Wr ≤ V P
be mutually orthogonal subspaces i.e. Wi ⊥ Wj whenever
r
i 6= j. Then the sum i=1 Wi is direct & we say the internal direct sum ⊕i Wi is
orthogonal.
Pr −1
Proof. The lemma ensures that Wr is orthogonal to W<r = i=1 Wi so by
induction, it suffices to show that any w ∈ Wr ∩ W<r ⊆ Wr ∩ Wr⊥ must be 0.
But w ⊥ w so (w|w) = 0 & w = 0. This completes the proof.
E.g.
Daniel Chan (UNSW)
Lecture 33: Orthogonal complements & projections
Semester 2 2012
3 / 10
Vector space complement
Prop
Let W ≤ V & dim W < ∞ then W + W ⊥ = V so V = W ⊕ W ⊥ .
Proof. We prove this here only in the case where dim V < ∞. The propn on
orthog direct sums ensures the sum W + W ⊥ is direct. Pick a basis
w1 , . . . , wr ∈ W for W . For i = 1, . . . , r we have li = (wi |·) ∈ L(V , F) so we may
form the r × 1-matrix T whose i-th entry is li . Note
T : V −→ Fr : v 7→ (l1 (v), . . . , lr (v))T .
By the lemma
W ⊥ = ∩i ker li = ker T .
Since im T ≤ Fr , rank-nullity ensures that
dim W ⊥ = dim V − dim im T ≥ dim V − r = dim V − dim W .
However, the sum W + W ⊥ is direct, so we must have
dim W + W ⊥ = dim W + dim W ⊥ = dim V which ensures V = W + W ⊥ as
desired.
Daniel Chan (UNSW)
Lecture 33: Orthogonal complements & projections
Semester 2 2012
4 / 10
Examples
E.g. Let V = C[x]≤1 with inner product (f |g ) =
orthogonal complement to W = C(1 + ix).
Daniel Chan (UNSW)
R1
0
f (t)g (t)dt. Find the
Lecture 33: Orthogonal complements & projections
Semester 2 2012
5 / 10
Orthogonal projections, bases
Cor-Defn
1
2
3
4
Suppose that V = W ⊕ W ⊥ (e.g. when dim W < ∞). The
orthogonal
w
projection onto W is the linear map projW : V −→ V : ww0 →
7
0 .
For v ∈ V we have v − projW v ∈ W ⊥ .
If V = W1 ⊕ . . . ⊕ Wr is an orthogonal direct sum then Wi⊥ =
particular, if V = W ⊕ W ⊥ then (W ⊥ )⊥ = W .
P
j6=i
Wj . In
We say a set S = {w1P
, . . . , wr } ⊆ V is orthogonal if wi ⊥ wj for i 6= j.
Equivalently, the sum i F wi is an orthogonal direct sum. In particular, S is
lin indep in this case.
An orthogonal set S ⊆ V is orthonormal if furthermore, kwi k = 1 for all i.
P
Proof. 2),4) follow from propns. We prove 3) first noting that W6=i = j6=i Wj is
orthogonal to Wi . It thus suffices to show Wi⊥ ⊆ W6=i so suppose w ⊥ Wi . We
may write w = wi0 + wi with wi ∈ Wi , wi0 ∈ W6=i . Then
5
0 = (wi |w) = (wi |wi0 + wi ) = (wi |wi0 ) + (wi |wi ) = (wi |wi )
so wi = 0 & w = wi0 ∈ W6=i .
Daniel Chan (UNSW)
Lecture 33: Orthogonal complements & projections
Semester 2 2012
6 / 10
Existence of orthonormal bases
Theorem
Let V be fin dim. Then
1
V is the orthogonal direct sum of 1-dimensional vector spaces.
2
V has an orthonormal basis.
Proof. 1) We argue by induction on d = dim V , the cases d = 0, 1 being clear so
suppose that d > 1. We may thus pick a non-zero subspace W 6= V e.g. F w for
any non-zero w ∈ W . Now V = W ⊕ W ⊥ & dim W , dim W ⊥ < d. By induction,
each of W & W ⊥ are orthogonal direct sums of 1-dimensional F-spaces, say
W = ⊕i Wi , W ⊥ = ⊕j Vj . Clearly, the subspaces {Wi , Vj } are still mutually orthog
so V is the orthogonal direct sum of them.
2) By 1), it suffices to find an orthonormal basis for a 1-dim F-space F v. Just
v
pick kvk
.
Daniel Chan (UNSW)
Lecture 33: Orthogonal complements & projections
Semester 2 2012
7 / 10
Orthogonal projection formula
Prop
1
2
3
Given a 1-dim F-space W = F w we have V = W ⊕ W ⊥ & projW v =
for any v ∈ V .
(w|v)
kwk2 w
Suppose that V = Wi ⊕ Wi⊥ for i = 1, . . . , r so we have orthog projn maps
projWi . Suppose further that the Wi are mutually orthog so we may consider
the orthogonal
direct sum W = ⊕Wi . Then V = W ⊕ W ⊥ &
P
projW = i projWi .
(“Fourier decomposition”) In particular, if W is spanned by the orthog set
{w1 , . . . , wr }, then
r
X
(wi |v)
wi .
projW v =
kwi k2
i=1
Rem This gives a proof of the propn on vector space complements in general.
Proof. Note 3) follows immediately from 1) & 2)
Daniel Chan (UNSW)
Lecture 33: Orthogonal complements & projections
Semester 2 2012
8 / 10
Proof propn
1) We need only show v0 = v −
v=
(w|v)
kwk2 w
(w|v)
kwk2 w
∈ W ⊥ for then we see
+ v0 ∈ W + W ⊥ . But
(w|v0 ) = (w|v −
(w|v)
(w|v)
w) = (w|v) −
(w|w) = 0
kwk2
kwk2
P
2) As above V = W ⊕ W ⊥ follows from showing v − i projWi v ⊥ W . In this
case we may write V = W1 ⊕ . . . Wr ⊕ W ⊥ & writing v = (w1 , . . . , wr , w0 )T for
wi ∈ Wi , w0 ∈ W ⊥ we see
   
 
w1
w1
0
 ..   
 .. 
 .  0
 .  X
   . 

..  =  ..  + . . . + 
projW v = 
projWi v.
0=
 .   
 
   
i


0
wr
wr 
0
0
0
Daniel Chan (UNSW)
Lecture 33: Orthogonal complements & projections
Semester 2 2012
9 / 10
Example
√
√
E.g. Consider the orthonormal basis f1 (x) = 1, f2 (x) = 2 3x − 3 for
R1
W = R[x]≤1 (wrt (f |g ) = 0 f (t)g (t)dt). Find projW x 2 .
Daniel Chan (UNSW)
Lecture 33: Orthogonal complements & projections
Semester 2 2012
10 / 10