Orthogonal projection
•
Unit II-2
Orthogonal projection
Unit II-2 Orthogonal projection
•
1
the scalar
along v
is called the component of u
–
in real ips this may be a positive or negative value
–
in complex ips this may have any complex value
two vectors u,v are
–
orthogonal if and only if proj(u,v) = 0
–
linearly dependent if and only if proj(u,v) = u
Unit II-2 Orthogonal projection
3
Example. For u=(1,-3,4), v=(3,4,7) find proj(u,v) and proj(v,u).
Orthogonal projection
•
the orthogonal projection of u on v is the vector
•
C-S says that ||proj(u,v)|| ≤ ||u||
–
just evaluate it and apply C-S at the critical moment
Unit II-2 Orthogonal projection
2
Unit II-2 Orthogonal projection
4
Orthogonal complement
•
V is an ips, S is a subset of vectors
•
the orthogonal complement of S [ S perp ] is
•
S⊥ is a subspace of V:
–
w∈S, u,v∈S⊥, and a,b scalars then
–
〈au+bv,w〉 = a〈u,w〉 + b〈v,w〉 =a(0) + b(0) = 0
–
so au+bv∈S⊥
•
if S = {w} write w⊥ instead of S⊥ for simplicity
•
if W is subspace of V then
–
W∩W⊥ = {0}
–
(W⊥)⊥ = W
Unit II-2 Orthogonal projection
Rowspace and nullspace again
The row space R and null space N of a matrix are
orthogonal complements.
–
example on slide 6 illustrates this
–
given a basis for R you can find the nullspace N by
finding the orthogonal complement R⊥
–
given a basis for the nullspace N you can find R by
finding the orthogonal complement N⊥
5
Unit II-2 Orthogonal projection
Example. Find a basis for W⊥ if W is the subspace of R5
spanned by u=(1,2,3,-1,2), v=(2,4,7,2,-1).
7
A geometric view of planes
•
the vector equation of a plane in R3 is (r - r0)n = 0
•
or rn = r0n
•
or proj(r,n) = proj(r0,n)
n
r
Unit II-2 Orthogonal projection
6
r0
Unit II-2 Orthogonal projection
8
Orthogonal projection onto a subspace
•
have projection onto a line
•
what about onto a plane?
•
v is decomposed
into a sum of two
orthogonal vectors:
For any subspace W how do we calculate proj(v,W)?
n
v2 = v - proj(v,W)
v = v 1 + v2
•
v1∈W and v2∈W⊥
•
geometrically the
decomposition is
obviously unique
Orthogonal projection onto a subspace
v
v1 = proj(v,W)
Unit II-2 Orthogonal projection
9
Unit II-2 Orthogonal projection
Orthogonal projection onto a subspace
•
take any subspace W of an ips V
•
V can be decomposed as V = W⊕W⊥
•
•
11
Orthogonal sets
•
a set of non-zero vectors {u1, u2, ..., un} is an
orthogonal set if 〈ui,uj〉 = 0 for i ≠ j
any vector v∈V can be decomposed uniquely
as v = v1 + v2 with v1∈W and v2∈W⊥
•
an orthogonal set is an orthonormal set if
||ui || = 1 for all i
for instance for TA:V→U the decomposition is
•
orthogonal sets are linearly independent
V = rowspace A ⊕ nullspace A
Unit II-2 Orthogonal projection
10
Unit II-2 Orthogonal projection
12
Examples: orthogonal sets
•
the standard basis vectors are an orthonormal
basis for Rn
•
the basis {(1,2), (1,-1)} for R2 is not an
orthogonal basis
•
the vectors {1, cos t, cos 2t, ..., sin t, sin 2t, ...}
are an orthogonal set in C [-π,π]
–
Orthogonal bases
•
the decomposition of v is
•
or
this is the foundation for Fourier analysis
Unit II-2 Orthogonal projection
13
choose an orthogonal basis {u1, u2, ..., un} for V
•
express a vector v in terms of this basis:
15
Example. S={u1,u2,u3,u4} with u1=(1,1,0,-1), u2=(1,2,1,3),
u3=(1,1,-9,2), u4=(16,-13,1,3) is an orthogonal basis. Find the
coordinates of a general vector (a,b,c,d) with respect to S.
Orthogonal bases
•
Unit II-2 Orthogonal projection
v = a1u1 + a2u2 + ... + anun
•
now calculate 〈v,ui〉 = 〈aiui,ui〉 = ai〈ui,ui〉
•
and solve for the scalars
•
these are called the Fourier coefficients of v
with respect to the orthogonal basis chosen
Unit II-2 Orthogonal projection
14
Unit II-2 Orthogonal projection
16
Projection onto a subspace
•
illustration for a plane in R3
v
proj(v,u2)
proj(v,u1)
u2
v1 = proj(v,W)
u1
Unit II-2 Orthogonal projection
17
now we can answer the question about how to
define and calculate proj(v,W)
•
choose an orthogonal basis {u1, u2,..., ur} for W
•
the projection of v on W is
•
this o n l y works if the basis is orthogonal
Unit II-2 Orthogonal projection
19
Example. Find the proj(v,W) for v = (1,3,5,7) and W < R4 spanned by
{u1,u2} where u1=(1,1,1,1), u2=(1,-3,4,-2).
Orthogonal projection onto a subspace
•
Unit II-2 Orthogonal projection
18
Unit II-2 Orthogonal projection
20
Orthonormal bases
•
Gramm-Schmidt algorithm
if
is an orthonormal basis things
can be written more neatly....
•
this procedure converts any old basis {u1,u2,...,un} of
V into an orthogonal basis
•
define
w 1= u 1
–
w2= u2 - proj(u2,w1)
this gives exactly the previous formula when written
as projections
Unit II-2 Orthogonal projection
w3= u3 - proj(u3,w1) - proj(u3,w2)
.....and so on
•
then {w1,w2,...,wn} is an orthogonal basis of V
•
normalize the wi to get an orthonormal basis
21
Projection onto a subspace
Unit II-2 Orthogonal projection
23
Why Gramm-Schmidt works: geometrically
w3
What do you do if your basis for W isn t orthogonal?
u3
proj(u3,w2)
proj(u3,w1)
w2
w1
Unit II-2 Orthogonal projection
22
Unit II-2 Orthogonal projection
24
Why Gramm-Schmidt works: algebraically
•
say you have the orthogonal set {w1,w2,...,wr}
•
wr+1 = ur+1 – a1w1 – a2w2 – ... – arwr
•
the Fourier coefficients are
•
wr+1 is orthogonal to all w1, ..., wr vectors
Unit II-2 Orthogonal projection
25
Unit II-2 Orthogonal projection
Example. Find an orthonormal basis for the subspace W of R4
spanned by {u1,u2,u3} where u1=(1,1,1,1), u2=(1,2,4,5) u3=(1,-3,-4,-2).
Gramm-Schmidt in practice
•
G-S is not a very robust numerical algorithm, but it s
valuable for important theoretical reasons
–
•
•
•
•
any multiple of each wi will do the job just as well
normalize all the wi vectors at the very last step once
you have the orthogonal basis
the procedure works on any linearly independent set
–
26
e.g. construction of the Legendre polynomials in the next
example
you won t usually have more than three basis
vectors because the calculations can be tedious
to avoid messy arithmetic clear all fractions as you
make your choice for each wi vector
–
Unit II-2 Orthogonal projection
27
it gives an orthogonal basis for the subspace it spans
Unit II-2 Orthogonal projection
28
Example. P3(t) with 〈f,g〉 defined on the interval [-1,1]. Apply G-S to the
mononomial basis to find an orthogonal basis called Legendre polynomials.
Example. Find the projection of v = (1,3,5,7) onto the subspace W
spanned by {u1,u2} where u1=(1,1,1,1), u2=(1,2,3,2).
Unit II-2 Orthogonal projection
29
Unit II-2 Orthogonal projection
31
Unit II-2 Orthogonal projection
30
Unit II-2 Orthogonal projection
32
Example. Find an orthogonal basis for w⊥ with w=(1,2,3,1) without
using the G-S algorithm.
Projection onto a subspace
•  {u1, ..., up} an orthonormal basis for W<Rn
•  U = [ u1| ...|up ] is n×p
•  proj (y,W) = UUTy for all y∈Rn
Unit II-2 Orthogonal projection
33
Unit II-2 Orthogonal projection
35
Orthogonal projection onto a subspace
•
•
•
is the closest point to y in W
for any v∈W,
is the best approximation to y by vectors in W
y
d
W
proj(y,W)
Unit II-2 Orthogonal projection
34
Unit II-2 Orthogonal projection
36