Section 7.2: Projections and the Gram-Schmidt Procedure
The essence of mathematics is not to make simple things complicated, but to
make complicated things simple.
(Stanley P. Gudder)
1 / 10
Orthogonal Projections
Recall that the matrix for the orthogonal projection of vectors onto the column
space of a matrix A is given by
P = A(AT A)−1 AT
This formula simplifies significantly when the columns of A form an orthonormal
set.
If the columns of an m × k matrix M form an orthonormal basis for a subspace
W ⊆ Rm , then M T M = Ik .
The projection formula becomes:
projW ⃗x = M(M T M)−1 M T ⃗x = M(Ik )M T ⃗x = MM T ⃗x
The matrix P = MM T is the projection matrix for the subspace W .
2 / 10
Theorem (Orthogonal Projection Formula)
Let W be a subspace of Rn and let ⃗x ∈ Rn .
(a) If {⃗v1 , . . . ,⃗vk } is an orthogonal basis for W , then
projW ⃗x =
⃗x · ⃗v1
⃗x · ⃗v2
⃗x · ⃗vk
⃗v1 +
⃗v2 + · · · +
⃗vk
2
2
∥⃗v1 ∥
∥⃗v2 ∥
∥⃗vk ∥2
(b) If {⃗u1 , . . . ,⃗uk } is an orthonormal basis for W , the formula simplifies to
projW ⃗x = (⃗x · ⃗u1 )⃗u1 + (⃗x · ⃗u2 )⃗u2 + · · · + (⃗x · ⃗uk )⃗uk
This is just a restatement of the coordinate formula from the previous section, as
the projection of ⃗x onto W is simply the component of ⃗x that lies in W .
3 / 10
⃗ 1 = (0, 1, −4, −1) and w
⃗ 2 = (3, 5, 1, 1) form an
Example 7.6: The vectors w
orthogonal basis for a plane W in R4 . Find the orthogonal projection of
⃗x = (1, 2, 0, −2) onto W .
4 / 10
Example 7.7: Suppose {⃗u1 , . . . ,⃗uk } is an orthonormal basis for a subspace W of
⃗ is in W , what is projW w
⃗ ? Use the projection formula to express
Rn . If a vector w
⃗ as a linear combination of the basis vectors.
w
5 / 10
The Gram-Schmidt Procedure
We often start with a standard basis for a subspace and need to convert it into an
orthogonal or orthonormal one.
Theorem
Every non-zero subspace of Rn has an orthonormal basis.
The constructive proof for this theorem is an algorithm known as the
⃗ k } for a subspace W , we can
Gram-Schmidt Process. Given a basis {⃗
w1 , . . . , w
generate an orthogonal basis {⃗v1 , . . . ,⃗vk } as follows:
⃗ 1.
Step 1: Set the first vector: ⃗v1 = w
⃗ 2 subtract proj⃗v1 w
⃗ 2:
Step 2: To get the second vector, from w
⃗v2 = w
⃗ 2 − proj⃗v1 w
⃗2 = w
⃗2 −
⃗ 2 · ⃗v1
w
⃗v1
∥⃗v1 ∥2
6 / 10
⃗ 3 subtract the projection of w
⃗ 3 onto the
Step 3: To get the third vector, from w
subspace spanned by ⃗v1 and ⃗v2 :
⃗v3 = w
⃗ 3 − projSpan{⃗v1 ,⃗v2 } w
⃗3 = w
⃗3 −
⃗ 3 · ⃗v1
⃗ 3 · ⃗v2
w
w
⃗v1 −
⃗v2
2
∥⃗v1 ∥
∥⃗v2 ∥2
General Step: For the j-th vector:
⃗vj = w
⃗j −
j−1
X
⃗ j · ⃗vi
w
i=1
∥⃗vi ∥2
⃗vi
The resulting set {⃗v1 , . . . ,⃗vk } is an orthogonal basis for W . To get an
orthonormal basis, simply normalize each vector ⃗vi .
7 / 10
Theorem (Gram-Schmidt Process)
⃗ 2 , ..., w
⃗ k } is a basis for a non-zero subspace of Rn and {⃗v1 ,⃗v2 , ...,⃗vk } is
If {⃗
w1 , w
the corresponding orthogonal basis by the Gram-Schmidt process, then for each
j ≥ 2:
⃗ j }.
(a) {⃗v1 ,⃗v2 , ...,⃗vj } is an orthogonal basis for Span{⃗
w1 , ..., w
⃗ j−1 }.
(b) ⃗vj is orthogonal to Span{⃗
w1 , ..., w
8 / 10
⃗ 2, w
⃗ 3 }, where
w1 , w
Example 7.8: The subspace W ⊆ R4 has a basis {⃗
⃗ 1 = (1, 2, 1, 0), w
⃗ 2 = (0, 1, 1, −2), and w
⃗ 3 = (1, 0, 3, 1). Use the Gram-Schmidt
w
process to find an orthogonal basis for W , and then find an orthonormal basis.
9 / 10
10 / 10