The Projection Matrix
David Arnold
Fall 1996
Abstract
In this activity you will use Matlab to project a set of vectors onto a
single vector.
Prerequisites. Inner product (dot product) and orthogonal vectors.
1 The Inner Product
We begin with the de…nition of the inner product.
De…nition 1 Let K and L be vectors from 4n. The inner product (dot prod-
uct) of vectors K and L is de…ned by
K ¢ L = KT L
· ¸
·
¸
Example 2 If K = 17 and L = 42 , then
¡
K¢L
Check this result in Matlab.
= KT L
£
= 1 7
= 10
>> u=[1;7]
u =
1
7
>> v=[-4;2]
v =
-4
1
¤
·
4
2
¡
¸
2
>> u’*v
ans =
10
If K, L and M are vectors from 4n and c is any real number, then it is not
di¢cult to show that each of the following properties are true:
²
²
²
²
K¢L =L¢K
K ¢ (L + M) = K ¢ L + K ¢ M
(cK) L = K (cL) = c(K L)
K K 0 and K K = 0 if and only if K = .
¢
¢
¸
¢
¢
¢
Finally, orthogonal (perpendicular) vectors are de…ned as follows.
De…nition 3 Let K and L be vectors from 4n. Vectors K and L are orthogonal
if and only if K ¢ L = 0.
1.1 The Projection of One Vector Onto Another
Figure 1 shows the projection of vector K onto vector L.
Figure
1. The projection of K onto L.
In Figure 2, it is clear that the projection of K onto L is some scalar multiple
of L; that is, projLK = cL.
L
2
Figure 2. The vector K cL is orthogonal to L.
Note that the di¤erence vector K cL is orthogonal (perpendicular) to L.
Consequently,
L (K cL) = 0
and we can use the previously mentioned properties to arrive at the following
result:
¡
¡
¢
¡
L ¢ K ¡ L¢(cL)
L ¢ K ¡ c(L ¢ L)
c(L ¢ L)
=
=
=
=
c
Finally, substitute result (1) into projL K = cL.
projL K
0
0
L ¢K
L ¢K
L¢L
(1)
(2)
= LL KL L
¢
¢
· ¸
Example 4 Use formula (2) to …nd the projection of K = 17 onto L =
·
4
2
¡
¸
:
projL K
= LL KL L
·
4¸·1¸
= · 42 ¸ · 74 ¸
2
2
·
¸
4
= 10
2
20
·
¸
2
=
1
¢
¢
¡
¢
¡
¢
¡
¡
Check this result in Matlab.
3
¡
·
4
2
¡
¸
>> (v’*u)/(v’*v)*v
ans =
-2
1
1.2 The Projection Matrix
Let’s rework formula (2). First, when you multiply a vector by a scalar, it
doesn’t matter whether you position the scalar before or after the vector.
projL K
= LL KL L
= L LL KL
¢
¢
¢
¢
Next, use the transpose de…nition of the inner product followed by the associative property of multiplication. Remember, when performing the dot product,
a scalar multiplier may be placed anywhere you wish.
projL K
= LT1 L L(LT K)
= LT1 L (LLT )K
T
= LLLT L K
The expression LLT is called an outer product (the transpose operator is
outside the
product versus its inside position in the inner product). If we de…ne
LLT
P = T , then the projection formula becomes
L L
LLT
projL K = P K, where P = T :
L L
The matrix P is called the projection matrix. You can project any vector onto
P.
the vector L by multiplying
by the matrix
· ¸
¸
·
Example 5 Let K = 17
onto L = ¡42 and …nd P , the matrix that will
project any matrix onto the vector L. Use the result to …nd projL K.
First, …nd the projection matrix.
P
T
= LLLT L
·
4
= · 2
4
2
¡
¡
4
¸·
4
2
¸T
¡
¸T ·
4
2
¡
¸
·
=
£
=
Check this result in Matlab.
¡
¡
¡
·
=
4 ¸£ 4 2 ¤
2
·
¸
¤
4
4 2
2
¸
16 8
8 4
20
¸
:8
:4
:4
:2
¡
¡
¡
·
¡
¡
>> P=(v*v’)/(v’*v)
P =
0.8000
-0.4000
-0.4000
0.2000
Use this result to …nd the projection of K onto L.
projL K
=
=
=
Check this last result in Matlab.
PK
·
·
8
4¸
2
1
:
¡:
4
:2
¡:
¸·
1
7
¸
¡
>> P*u
ans =
-2
1
Note that this is identical to the result in Example 4.
1.3 Projecting a Lot of Vectors onto a Single Vector
First, use Matlab to generate 100 random vectors and plot them. The following
commands will produce an image similar to that in Figure 3.
>>
>>
>>
>>
U=8*rand(2,100)-4;
x=U(1,:);
y=U(2,:);
plot(x,y,’o’)
5
4
2
0
-2
-4
-4
-2
0
2
4
Figure 3.
The image in Figure 3 bears some explanation. Each circle in Figure 3
represents the tip of a vector whose tail begins at the origin.
Next, use the projection matrix P to project each of the 100 2 1 vectors in
matrix U onto the vector L, then plot the results on the graph in the color red.
£
>>
>>
>>
>>
>>
PU=P*U;
x=PU(1,:);
y=PU(2,:);
hold on
plot(x,y,’ro’)
4
2
0
-2
-4
-4
-2
0
2
4
Figure 4.
In Figuure 4, note that each vector
in¸ the matrix U is projected onto a line
·
4
in the direction of the vector L = 2 .
¡
2 Homework
1. Create 100 random 2 1 vectors in a matrix U with components between
2 and 2. Plot them.
6
£
¡
(a) Create a matrix P which will·project
¸ each of the 2 1 vectors in
1
matrix U onto the vector L = 1 .
(b) Use the matrix P to project each 2 1 vector in matrix U onto the
vector L. Plot the results in a second color.
(c) Obtain a printout of your result.
2. The column space of the projection matrix P is a line in the direction of
the vector L.
(a) What is the dimension of the column space of P ? Check your answer
with the ATLAST distribution’s colbasis command.
(b) What is the rank of P ? Check your answer with Matlab’s rank
command.
3. If matrix P projects all vectors onto the vector L, what should the matrix
P P do? Can you explain geometrically why the matrices P P and P are
equal? Check this in Matlab.
£
£
7