3.V. Projection
3.V.1. Orthogonal Projection Into a Line
3.V.2. Gram-Schmidt Orthogonalization
3.V.3. Projection Into a Subspace
3.V.1. & 2. deal only with inner product spaces.
3.V.3 is applicable to any direct sum space.
3.V.1. Orthogonal Projection Into a Line
Definition 1.1:
Orthogonal Projection
The orthogonal projection of v into the line spanned by a nonzero s is the vector.
projs v v sˆ sˆ
sˆ
s
s
Example 1.3:
x
s
2x
vs
s
ss
s
ss
Orthogonal projection of the vector ( 2 3 )T into the line y = 2x.
s x 4x 5 x
2
2
2 1
1 1 8 1
projs
2 6
5
5 2 5 2
3
1
ˆs
5
1
2
Example 1.4: Orthogonal projection of a general vector in R3 into the y-axis
0
e 2 1
0
x x 0
proje2 y y 1
z z 0
0
0
y 1
1
0
0
0
y
0
Example 1.5:
Project = Discard orthogonal components
A railroad car left on an east-west track without its brake is pushed by a wind
blowing toward the northeast at fifteen miles per hour; what speed will the car reach?
15
w
2
1
1
15 1
v proje1 w
2 0
speed v
15
2
Example 1.6:
Nearest Point
A submarine is tracking a ship moving along the line y = 3x + 2.
Torpedo range is one-half mile.
Can the sub stay where it is, at the origin on the chart, or must it
move to reach a place where the ship will pass within range?
1
Ship’s path is parallel to vector sˆ
10
1
3
Point p of closest approach is
0
0
p projs
2
2
0
6
1
10
10
2
p
1
10
5
0.632
1
0 3 1
1 3
3 2 5 3 5 1
(Out of range)
Exercises 3.V.1.
1. Consider the function mapping a plane to itself that takes a vector to its
projection into the line y = x.
(a) Produce a matrix that describes the function’s action.
(b) Show also that this map can be obtained by first rotating everything in
the plane π/4 radians clockwise, then projecting into the x-axis, and then
rotating π/4 radians counterclockwise.
3.V.2. Gram-Schmidt Orthogonalization
Given a vector s, any vector v in an inner product space can be decomposed as
v projs v v projs v v // v
where
v // v 0
Definition 2.1:
Mutually Orthogonal Vectors
Vectors v1, …, vk Rn are mutually orthogonal if
vi · vj = 0
ij
Theorem 2.2:
A set of mutually orthogonal non-zero vectors is linearly independent.
Proof:
c v
i
i
0
→
v j ci vi c j v j v j 0
i
i
→
cj = 0 j
Corollary 2.3:
A set of k mutually orthogonal nonzero vectors in V k is a basis for the space.
Definition 2.5:
Orthogonal Basis
An orthogonal basis for a vector space is a basis of mutually orthogonal vectors.
Example 2.6:
1 0 1
1 , 2 , 0
1 0 3
Turn
1
κ1 1
1
1
1 1
1
projκ1 0 1 0
3 3 1 3
0
1 0
1
projκ1 2 1 2
0 3 1 0
κ1 κ1 3
0
0
κ 2 2 projκ1 2
0
0
1
1
1 2 2
3
1
1
0
2
2
0 3
1
1
4
1 1
3
1
1
1
1
1
κ 3 0 projκ1 0 projκ2 0
3
3
3
into an orthogonal basis for R3.
1
4
0
3 3
1
1
1
κ2 κ2
1
2
1
3
1
24 8
9 3
1
1 1 1
3
2
2
2
projκ2 0 2 0 2
3
3 8 3 1 3 3 1
1
1 2
3
1
1
1
2 0
1
1
K
1
2
1
1 1 1
1 , 2 2 , 0
3
1 1 1
Theorem 2.7: Gram-Schmidt Orthogonalization
If β1 , …, βk is a basis for a subspace of Rn then the κj s calculated from the
following scheme is an orthogonal basis for the same subspace.
κ1 β1
κ1 κ1
β 2 κ1
β2
κ1 I
β2
κ
κ
κ1 κ1
1
1
κ 2 β2 projκ1 β2
κ 3 β3 projκ1 β3 projκ2 β3 β3
k 1
κ k βk projκ j βk
j 1
Proof:
k 1
βk κ j
j 1
κj κj
βk
κj
κ1 κ1
κ2 κ2
I
β2
κ
κ
κ
κ
1
1
2
2
k 1 κ
κj
j
I
j 1 κ j κ j
βk
For m 2 , Let β1 , …, βm be mutually orthogonal, and
m
βm1 κ j
j 1
κj κj
κ m1 βm1
Then
β 3 κ1
β κ
κ1 3 2 κ 2
κ1 κ1
κ2 κ2
κj
m
βm1 κ j
j 1
κ j κj
κ i κ m1 κ i βm 1
κ i κ j κi βm1 βm1 κi 0
i 1,
,m
QED
If each κj is furthermore normalized, the basis is called orthonormal.
The Gram-Schmidt scheme simplifies to:
κ1 β1
e1
κ1
κ1
κ 2 β2 projκ1 β2 β2 β2 e1 e1
I e1 e1
κ 3 β3 projκ1 β3 projκ2 β3 β3 β3 e1 e1 β3 e2 e2
k 1
κ k βk projκ j βk
j 1
k 1
βk βk e j e j
j 1
e2
β2
I e1 e1 e 2 e 2
k 1
I e j e j βk
j 1
κ2
κ2
β
e3
2
ek
κk
κk
κ3
κ3
Exercises 3.V.2.
1. Perform the Gram-Schmidt process on this basis for R3,
2 1 0
2 , 0 , 3
2 1 1
2. Show that the columns of an nn matrix form an orthonormal set if and
only if the inverse of the matrix is its transpose. Produce such a matrix.
3.V.3. Projection Into a Subspace
Definition 3.1:
For any direct sum V = M N and any v V such that
v=m+n
with m M and n N
The projection of v into M along N is defined as
projM, N (v) = m
Reminder:
• M & N need not be orthogonal.
• There need not even be an inner product defined.
Example 3.2:
M
The space M22 of 22 matrices is the direct sum of
N
a, b R
a b
0 0
Task: Find projM , N (A), where
0 0
c d
c, d R
3 1
A
0 4
Solution:
Let the bases for M & N be
→
∴
B BM º BM
BM
1 0 0 1
0 0 , 0 0
1 0 0 1 0 0 0 0
0 0 , 0 0 , 1 0 , 0 1
1 0
0 1
0 0
0 0
A 3
1
0
4
0 0
1 0
0 1
0 0
1 0
0 1
projM , N A 3
1
0 0
0 0
3 1
0 0
BN
0 0 0 0
1 0 , 0 1
is a basis for M22 .
Example 3.3:
Consider
and subspaces
Both subscripts on projM , N (v) are significant.
M
x
y
z
y 2z 0
0
N k 0
1
It’s straightforward to verify
k R
with basis
&
1 0
0 , 2
0 1
BM
0
L k 1
2
k R
R3 M N M L
Task: Find projM , N (v) and projM , L (v)
where
2
v 2
5
Solution:
For
BM N BM º BN
1 0 0
0 , 2 , 0
0 1 1
→
2
v 1
4
B M N
2
2
proj M , N v 1
2
0
B M N 1
For
BM L BM º BL
1 0 0
0 , 2 , 1
0 1 2
→
2
2
v 2 9 / 5
8 / 5
5
B M L
2
2
proj M , L v 9 / 5
18 / 5
0
B M N 9 / 5
Note: BML is orthogonal but BMN is not.
2a b 2
a 2b 5
Definition 3.4:
Orthogonal Complement
The orthogonal complement of a subspace M of Rn is
M = { v Rn | v is perpendicular to all vectors in M }
( read “M perp” ).
The orthogonal projection projM (v ) of a vector is its projection into M along M .
Example 3.5:
In R3, find the orthogonal complement of the plane
Solution:
→
P v
Natural basis for P is
v β1 0, v β2 0
v1
v
2
v
3
B
P
x
y
z
1 0
0 , 1
3 2
v1
1
0
3
0
0 1 2 v2 0
v
3
3x 2 y z 0
( parameter = z)
3
k 2
1
k R
Lemma 3.7:
Let M be a subspace of Rn.
Then M is also a subspace and Rn = M M .
Hence, v Rn, v projM (v) is perpendicular to all vectors in M.
Proof: Construct bases using G-S orthogonalization.
Theorem 3.8:
Let v be a vector in Rn and let M be a subspace of Rn with basis β1 , …, βk .
If A is the matrix whose columns are the β’s then
projM (v ) = c1β1 + …+ ck βk
where the coefficients ci are the entries of the vector (AT A) AT v.
That is,
projM (v ) = A (AT A)1 AT v.
Proof:
projM v M
By lemma 3.7,
→
0 AT v Ac
projM v A c
→
AT A c AT v
where c is a column vector
→
c AT A
1
AT v
Interpretation of Theorem 3.8:
If B = β1 , …, βk is an orthonormal basis, then ATA = I.
In which case, projM (v ) = A (AT A)1 AT v = A AT v.
β1T
βk
v β1
βk T
projM v β1
β1T v
βk
βk T v
v1
βk
v
k B
β1
k
v j β j
vj βj v
with
j 1
In particular, if B = Ek , then A = AT = I.
In case B is not orthonormal, the task is to find C s.t. B = AC and BTB = I.
I AC
Hence
T
AC
C A AC
T
T
→
AT A CT C1 CCT
1
projM v BBT v AC AC v
T
1
→
ACCT A T v A AT A AT v
1
CCT AT A
1
Example 3.9:
To orthogonally project
P
From
0
y 1
0
into subspace
x, y R
1 0
2 0
1
0
1
T
A A
0 1 0 1
0
1
0
1 0
→
A AT A
1
x 0
1
1
v 1
1
1
we get
P
x
y
z
xz0
1 0
A 0 1
1 0
1
1/ 2 0
T
A
A
0 1
1 0
1 0
1/ 2 0 1/ 2
1/
2
0
1
0
1
1/
2
0
1/
2
AT 0 1
0 1
0
1
0
0
1 0 0 1 0 1 0 1 0 0 1
1/ 2 0 1/ 2
1/ 2 0 1/ 2 1
0
projP v 0
1
0 1 1
1/ 2 0 1/ 2 1
0
Exercises 3.V.3.
1. Project v into M along N, where
3
v 0
1
2. Find M for
M
M
x
y x y 0
z
x
y x 3y z 0
z
N
1
c 0 c R
1
3. Define a projection to be a linear transformation t : V → V with the property
that repeating the projection does nothing more than does the projection alone:
( t t )(v) = t (v) for all v V.
(a) Show that for any such t there is a basis B = β1 , …, βn for V such that
β
t βi i
0
for
i 1, 2, , r
i r 1, r 2,
,n
where r is the rank of t.
(b) Conclude that every projection has a block partial-identity representation:
I 0
RepB B t
0
0
0
