EECS 275 Matrix Computation
Ming-Hsuan Yang
Electrical Engineering and Computer Science
University of California at Merced
Merced, CA 95344
http://faculty.ucmerced.edu/mhyang
Lecture 12
1 / 18
Overview
QR decomposition by Householder transformation
QR decomposition by Givens rotation
2 / 18
Reading
Chapter 10 of Numerical Linear Algebra by Llyod Trefethen and
David Bau
Chapter 5 of Matrix Computations by Gene Golub and Charles Van
Loan
Chapter 5 of Matrix Analysis and Applied Linear Algebra by Carl
Meyer
3 / 18
Householder triangularization
In Gram-Schmidt orthogonalization
A R1 R2 · · · Rn = Q
| {z }
R −1
has orthonormal columns. The product R = Rn−1 · · · R2−1 R1−1 is upper
triangular
In Householder triangularization, a series of elementary orthogonal
matrices Qk is applied to the left of A, so that the resulting matrix
Qn · · · Q2 Q1 A = R
|
{z
}
Q>
is upper triangular. The product Q = Q1> Q2> · · · Qn> is orthogonal,
and therefore A = QR is a QR factorization of A
Two methods for QR factorization
I
I
Gram-Schmidt: triangular orthogonalization
Householder: orthogonal triangularization
4 / 18
Geometry of elementary projectors
For u, x ∈ IRn , s.t. kuk = 1
Orthogonal projectors onto span{u} and u⊥ are
Pu =
uu>
, and
u> u
Pu⊥ = I −
uu>
u> u
For u 6= 0, the Householder transformation or the elementary reflector
about u⊥ is
uu>
R =I −2 >
u u
or
R = I − 2uu>
when kuk = 1, and
R = R > = R −1
5 / 18
Triangularization by introducing zeros
The matrix Qk is chosen to introduce zeros below the diagonal in the
k-th column while preserving all the zeros previously introduced








× × ×
× × ×
× × ×




× × ×
× ×



 Q3 
 Q1  0  Q2 


 0 −→
× × ×−→

0 −→








 0 
× × ×
0
0 
0
0 0 × × ×
A
Q1 A
Q2 Q1 A
Q3 Q2 Q1 A
Qk operates on row k, . . . , m (changed entries are denoted by
boldface or and blank entries are zero)
At beginning of step k, there is a block of zeros in the first k − 1
columns of these rows
The application of Qk forms linear combinations of these rows, and
the linear combination of the zero entries remain zero
After n steps, all the entries below the diagonal have been eliminated
and Qn · · · Q2 Q1 A = R is upper triangular
6 / 18
Householder reflectors
At beginning of step k, there is a block of zeros in the first k − 1
columns of these rows
Each Qk is chosen to be
I 0
Qk =
0 F
where I is the (k − 1) × (k − 1) identity matrix and F is an
(m − k + 1) × (m − k + 1) orthogonal matrix
Multiplication by F has to introduce zeros into the k-th column
The Householder algorithm chooses F to be a particular matrix called
Householder reflector
At step k, the entries k, . . . , m of the k-th column are given by vector
x ∈ IRm−k+1
7 / 18
Householder transformation (cont’d)
To introduce zeros into k-th column (x ∈ IRm−k+1 ), the Householder
transformation F should
 
 
×
kxk
×
 0 
  F
 
x =  .  −→ F x =  .  = kxke1 = αe1
 .. 
 .. 
×
0
The reflector F will reflect the space IRm−k+1 across the hyperplane
H orthogonal to u = kxke1 − x
A hyperplane is characterized by a vector u = kxke1 − x
8 / 18
Householder transformation (cont’d)
Every point x ∈ IRm is mapped to a mirror point
F x = (I − 2
uu>
u> x
)x
=
x
−
2u(
)
u> u
u> u
and hence
F = (I − 2
uu>
)
u> u
Will fix the +/- sign in the next slide
9 / 18
The better of two Householder reflectors
Two Householder reflectors (transformations)
For numerical stability pick the one that moves reflect x to the vector
kxke1 that is not to close to x itself, i.e., −kxke1 − x in this case
In other words, the better of the two reflectors is
u = sign(x1 )kxke1 + x
where x1 is the first element of x (sign(x1 ) = 1 if x1 = 0)
10 / 18
Householder QR factorization
Algorithm:
for k = 1 to n do
x = Ak:m,k
uk = sign(x1 )kxk2 e1 + x
uk = kuukkk2
Ak:m,k:n = (I − 2uk u>
k )Ak:m,k:n
end for
Recall
Qk =
I 0
0 F
Upon completion, A has been reduced to upper triangular form, i.e.,
R in A = QR
Q > = Qn · · · Q2 Q1 or Q = Q1> Q2> · · · Qn>
11 / 18
QR decomposition with Householder transformation
Want to compute QR decomposition A with Householder
transformation


12 −51
4
A =  6 167 −68
−4 24 −41
Need to find a reflector for first column of A, x = [12, 6, −4]> to
kxke1 = [14, 0, 0]>
u=kxke1 − x =[2, −6, 4]> = 2[1, −3,2]>


6/7
3/7 −2/7
14 21 −14
>
F1 =I − 2 uuu> u =  3/7 −2/7 6/7  , F1 A =  0 −49 −14
−2/7 6/7
3/7
0 168 −77
Next need to zero out A32 and apply the same process to
−49 −14
0
A =
168 −77
12 / 18
QR decomposition with Householder (cont’d)
With the same process


1
0
0
F2 = 0 −7/25 24/25
0 24/25 7/25
Thus, we have


6/7 −69/175 58/175
Q = Q1 Q2 =  3/7 158/175 −6/175
−2/7 6/35
33/35

14 21 −14
R = Q2 Q1 A = Q > A =  0 175 −70
0
0 −35
The matrix Q is orthogonal and R is upper triangular
13 / 18
Givens rotations
Givens rotation: orthogonal transform to zero out elements selectively


1 ··· 0 ··· 0 ··· 0
 .. . .
.
..
.. 
.
. ..
.
.


0 · · · c · · · −s · · · 0

 i
 ..
..
..
.. 

. ···
.
.
G (i, k, θ) =  .

0 · · · s · · · c · · · 0 k


 ..
..
..
.. 
..
.
.
.
.
.
0 ··· 0 ··· 0 ··· 1
i
k
where c = cos(θ) and s = sin(θ) for some θ
Apply G on A, only rows i and j of A are affected,
c −s a
r
=
s c
b
0
thus
r←
p
a2 + b 2 , c ← a/r , s ← −b/r
14 / 18
Givens rotations (cont’d)
Pre-multiply G (i, k, θ) amounts to a counterclockwise rotation θ in
the (i, k) coordinate plane, y = G (i, k, θ)x


cxi − sxk j = i
yj = sxi + cxk j = k


xj
j 6= i, k
Can zero out yk = sxi + cxk = 0 by setting
−xk
xi
, s=q
, θ = arctan(xk /xi )
c=q
xi2 + xk2
xi2 + xk2
QR decomposition can be computed by a series of Givens rotations
Each rotation zeros an element in the subdiagonal of the matrix,
forming R matrix, Q = G1 . . . Gn forms the orthogonal Q matrix
Useful for zero out few elements off diagonal
(e.g., sparse matrix)
√
√
>
Example If x = [1, 2, √
3, 4] , cos(θ) = 1/ 5, and sin(θ) = −2/ 5,
then G (2, 4, θ) = [1, 20, 3, 0]> .
15 / 18
QR factorization with Givens rotation
Given A


6 5 0
A = 5 1 4
0 4 3
with G1
Want to zero out A21
With θ we have G1


c −s 0
G1 =  s c 0
0 0 1
√
r = 62 + 52 = 7.8102, c = 6/r

7.8102
G1 A =  0
0



c −s 0 6 5 0
 s c 0 5 1 4
0 0 1
0 4 3
= 0.7682, s = −5/r = −0.6402

4.4813 2.5607
−2.4327 3.0729
4
3
Continue to zero out A32 and form a triangular matrix R
The orthogonal matrix Q > = G2 G1 , and G2 G1 A = Q > A = R for QR
decomposition
16 / 18
Gram-Schmidt, Householder and Givens
Householder QR is numerically more stable
Gram-Schmidt computes orthonormal basis incrementally
Givens rotation is more useful for zero out few selective elements
17 / 18