ROTATION MATRICES
Note that
A
1
0
A=
cos θ − sin θ
sin θ
cos θ
=
cos θ
sin θ
,
A
0
1
=
− sin θ
cos θ
This shows the vectors e(1) and e(2) are rotated counterclockwise thru an angle of θ radians. In particular,
(2)
Ae
=
− sin θ
cos θ

=
cos θ +
sin θ +

π
2 
π
2
Thus in general, the transformation x → Ax corresponds to a rotation of x counter-clockwise thru an
angle of θ radians.
A−1 should correspond to a clockwise rotation thru
−θ radians; or replacing θ by −θ in the original formula for A, we have
A−1 =
cos (−θ ) − sin (−θ )
sin (−θ )
cos (−θ )
=
cos θ sin θ
− sin θ cos θ
We generalize this in a very simple way to Rn. Let
1 ≤ k < l ≤ n, and define the matrix Rk,l to the
following matrix:

1 0
···
0
 0 ... ...


... 1

0
0


0 cos θ 0 · · · 0 − sin θ 0


0
1
0

 .
..
..
..
..
...
 .


0
1
0


0 sin θ 0 · · · 0 cos θ
0


0
0
1 ...


..
..
... ... 0

0
···
0 1





















In it, we have modified the identity matrix I , changing
it in the four elements in positions (k, k), (k, l) , (l, k) ,
and (l, l).
The matrix Rk,l will rotate the (k, l)-plane thru an
angle of θ , while leaving unchanged the remainder of
Rn, that part perpendicular to (k, l)-plane.
HOUSEHOLDER MATRICES
Let w ∈ Cn be a vector of Euclidean length 1,
w∗w
wT w
=1
= 1 for
w ∈ Rn
Define the matrix
H = I − 2ww
∗
I − 2ww
T
n
for w ∈ R
Then H is a Hermitian unitary matrix (orthogonal if
w ∈ Rn ).
First,
H ∗ = (I − 2ww∗)∗
= I − 2 (ww∗)∗
= I − 2 (w∗)∗ w∗ = H
Also,
H ∗H = H 2
= (I − 2ww∗)2
= I − 4ww∗ + 4 (ww∗ ) (ww∗ )
Note that
(ww∗) (ww∗) = w(w∗w)w∗ = ww∗
=1
Thus
H ∗H = I − 4ww∗ + 4 (ww∗) (ww∗) = I
showing H is unitary.
What does H do in a geometric sense? First, note
that
Hw = (I − 2ww∗) w = w − 2ww∗w = w − 2w = −w
Also, let v be any vector orthogonal to w. Then
(ww∗) v = w (w∗v ) = w (0) = 0
Hv = (I − 2ww∗) v = v − 2 (ww∗) v = v
Thus H is a reflection of space thru the (n − 1)dimensional hyperplane perpendicular to w.
The rotation matrices Rk,l and the Householder matrices H are the most commonly used orthogonal (or
unitary) matrices used in numerical analysis.
EXAMPLES
For the n = 3 case, with w ∈ R3,

H=

1 − 2w12 −2w1w2

2
 − 2w w
1
−
2
w
1
2

2
− 2 w1 w3

− 2 w2 w3 

2
− 2 w1 w3 − 2 w2 w3 1 − 2 w3
Then H T = H and H 2 = I .
For w =
1, 2, 2 T ,
3 3 3

H=
For w =
T
3
4
0, 5 , 5 ,
7
 9
 −4
 9
− 49

− 49
1
9
− 89

4
−9

8
−9 

1
9

1
0
0

7 − 24 

0
H=

25
25 
7
0 − 24
−
25
25
REDUCTION OF A VECTOR
Given a vector d ∈ Rm, we want to find v ∈ Rm with
v2 = 1 and



T
I − 2vv d = 

α
0
..
0





(1)
Since I − 2vv T is orthogonal, the length of d is preserved in the tranformation of d:
|α| = d2 ≡ S
and
α = ±S = ± sqrt
d21
+ · · · d2m
with the sign to be determined later.
Introduce
p = vT d
From (1),
d − 2pv = [α, 0, ..., 0]T
(2)
Multiply on the left by v T to get
p − 2pv T v = v1α
p = −αv1
Substitute this back into (2). Then look at the individual components, obtaining
d1 + 2αv12 = α
di − 2pvi = 0,
i = 2, ..., m
(3)
From the first equation,
α − d1
1
d
v12 =
=
1− 1
(4)
2α
2
α
Recall that we have not yet chosen the sign of α. Now
choose the sign of α according to
sign (α) = − sign (d1)
The subtraction in (4) is now actually an addition,
so as to avoid a ‘loss of significance’ error. We can
take the square root in (4) to find v1, and there is no
obvious choice of the sign here, although most people
would choose v1 > 0.
Return to p = −αv1 to obtain p. Then return to (3)
to find v2, ..., vm:
d
vi = i , i = 2, ..., m
2p
This completes the construction of v and therefore the
Householder matrix I − 2vv T based upon it. Again,
we now have

I − 2vv T


d=

α
0
..
0





EXAMPLE
Let
d = [2, 2, 1]T
Then
v1 = sqrt 56 ,
α = − d2 = −3,
2
v2 =
,
sqrt (30)
Then

H=
− 23
 2
 −
 3
− 13
p = sqrt
1
v3 =
sqrt (30)
− 23
11
15
2
− 15

1
−3

2
− 15 

14
15
In practice, one does not need H explicitly.
15
2
QR FACTORIZATION
Let A be a matrix that is n × n. We want to factor
it into the form
A = QR
with Q orthogonal and R upper triangular. We do
this by working on each of the columns in succession.
For the first step, let
P1 = I − 2w
(1) (1)T
w
,
(1) w = 1
2
Let A = A∗,1, ..., A∗,n . Then
P1A = P1A∗,1, ..., P1A∗,n
We choose P1 so that
P1A∗,1 = [α, 0, ..., 0]T
We can of course do this by the construction already
described above, with d = A∗,1.
With this choice of P1, the matrix P1A will have zeroes in the first column below the diagonal position.
Next, we wish to do the same to the second column,
but without changing the first column.
Define
P2 = I − 2w
(2) (2)T
w
,
(2) w = 1
2
But now require
w(2) =
0
v
,
Then
P2 =
Calculate P2P1A:
v T v = 1,
1
0
0 I − 2vv T
v ∈ Rn−1
P2P1A = P2P1A∗,1, ..., P2P1A∗,n
We know that
P1A∗,1 = [α, 0, ..., 0]T
and therefore
P2P1A∗,1 =
1
0
0 I − 2vv T
α
0
=
α
0
Thus the first column of P2P1A retains its zero structure below the diagonal. Now choose v so as to force
the second column of P2P1A to have zeroes below
the diagonal position. Writing
P1A∗,2 =
c
d
,
d ∈ Rn−1
we choose v by forcing
I − 2vv
T
d = [α2, 0, ..., 0]T
for a suitable value of α2.
Continue in this manner, defining
Pi = I − 2w
with
w(i) =
0
v
(i) (i)T
w
,
(i)
w = 1
2
,
v T v = 1,
v ∈ Rn−i+1
and choosing v to force Pi · · · P1A to have zeros below
the diagonal in column i. After n− 1 steps, the matrix
Pn−1 · · · P2P1A ≡ R
is upper triangular. Note that the matrix Pn−1 · · · P2P1
is orthogonal,
(Pn−1 · · · P2P1)T (Pn−1 · · · P2P1)
T P
= P1T · · · Pn−1
n−1 · · · P2P1
=I
We define
QT = Pn−1 · · · P2P1
Then
QT A = R
A = QR
EXAMPLE
An example for the matrix


4 1 1


A= 1 4 1 
1 1 4
is given on page 613 in the text. The result is A = QR
with


R=

−4.24264 −2.12132 −2.12132

−3.67423 −1.22475 
0
0
0
3.46410

−0.94281
0.27217 −0.19245



Q = P1P2 =  −0.23570 −0.95258 −0.19245 
−0.23570 −0.13608
0.96225
This factorization can also be accomplished using the
Matlab instruction qr:
[Q
R] = qr(A)
HOUSEHOLDER MULTIPLICATIONS
Consider multiplying a Householder matrix H times
another matrix:
HA =
I − 2wwT
A =
=
T
A − 2ww A
T
A − 2w w A
The quantity wT A is a vector, and it can be produced with approximately 2n2 operations if w has
T
all nonzero components. Then calculating 2w w A
will cost a further n2 multiplications, approximately,
and A−2w wT A will cost n2 subtractions. Thus the
total operations cost to produce HA will be around
4n2 operations, rather than the 2n3 one would usually
expect with multiplying two n × n matrices.
This also shows why we generally do not produce Q =
P1 · · · Pn−1 explicitly, as it is cheaper to carry out
matrix multiplications when Q is in factored form.
REDUCTION OF SYMMETRIC MATRICES
Let A be a symmetric matrix. We want to use a similarity transformation to reduce it to symmetric tridiagonal form.
Assume A is n × n. We will perform a sequence of
similarity transformations, using Householder matrices, to reduce A to tridiagonal form. We begin by
using a similarity transformation to put zeros into the
first column of the matrix, in the positions below the
(2,1) position.

a
 1,1
 a
2,1

P1T AP1 = A2 = 
 0
 ..

0
2,1 0 · · ·
a
2,2
a
···
..
n,2
a
We use the Householder matrix
P1 = I − 2w
(2) (2)T
w
,
...
0

n,2 
a

..
n,n
··· a
(2) w = 1
2
with
w(2) = [0, v1, ..., vm]T ,

m=n−1




Then



P1 = 

1
0
0 1 − 2v12
..
..
0 − 2v 1 v m
···
0
· · · − 2v 1 v m
..
...
2
· · · 1 − 2v m





Let A = A∗,1, ..., A∗,n . Then
P1A = P1A∗,1, ..., P1A∗,n
The first column looks like





1
0
···
0
a1,1
a1,1
 0 1 − 2v 2 · · · − 2v v   a

 a






m
2,1
1
2,1 
1
=
 .





..
..
...
 .
  .. 
 0 
..
2
an,1
0 − 2v 1 v m · · · 1 − 2v m
With reference to our earlier work on reducing a vector d to a simpler form with only a single nonzero
component, we use that algorithm with
d = a2,1, ..., an,1
T
With that algorithm, we can find v ∈ Rm with
I − 2vv
T
2,1 , 0, ..., 0
d= a
T
In fact,
2,1 = − sign a2,1 sqrt
a
a22,1
+ · · · + a2n,1
Now look at P1A and P1AP1:

a
∗ ···
 1,1
 a
 2,1 ∗ · · ·
P1A = 
∗
 0
 ..
..

0
P1AP1 is given by








a1,1
2,1
a
0
..
0
∗ ···
∗ ···
∗
..
∗ ···
∗
∗
∗
..
∗










∗
∗
∗
..
∗ ··· ∗
1
0
0 1 − 2v12
..
..
0 − 2v 1 v m
a
∗ ···
 1,1
 a
 2,1 ∗ · · ·
=
∗
 0
 ..
..

0
∗
∗
∗
..
∗ ··· ∗








···
0
· · · − 2v 1 v m
..
...
2
· · · 1 − 2v m













In addition, the matrix P1AP1 is symmetric:
(P1AP1)T = P1T AT P1T = P1AP1
Therefore, P1AP1 must look like

2,1 0 · · ·
a
a
 1,1
 a
∗
···
 2,1
A2 = P1AP1 = 
∗
 0
 ..
..
...

0
∗
∗
..
··· ∗
∗
0








We continue this process by now reducing the second
column by a similar operation. We use the Householder matrix
P2 = I − 2w
(3) (3)T
w
(3) w = 1
2
,
w(3) = [0, 0, v1, ..., vm]T ,
Then




P2 = 



1
0
0
..
0
0
0
1
0
0 1 − 2v12
..
0 − 2v 1 v m
···
m=n−2
0
0
· · · − 2v 1 v m
..
...
2
· · · 1 − 2v m








We choose v ∈ Rn−2 such that
I − 2vv
T
3,2 , 0, ..., 0
d= a
T
with d the elements in positions 3 thru n of column
2 of the matrix A2. Note that the form of P2 is such
that P2A2P2 will have the same first column and row
as in A2. For example, consider calculating first P2A2:








1
0
0
..
0
0
0
1
0
0 1 − 2v12
..
0 − 2v 1 v m
···
0
0
· · · − 2v 1 v m
..
...
2
· · · 1 − 2v m

2,1 0 · · ·
a
a
  1,1
 a
∗
···
  2,1
 0
∗

  ..
..
...

0
∗
We continue in this way, obtaining finally
Pn−2 · · · P1AP1 · · · Pn−2 = T
with T a symmetric tridiagonal matrix. Define
Q = P1 · · · Pn−2
It is orthogonal, and
QT AQ = T
0
∗
∗
..
··· ∗








Therefore the eigenvalues of A and T are the same.
For the eigenvectors, let
x=
0
T x = λx,
Then
QT AQx = λx
A (Qx) = λ (Qx)
Thus Qx is the eigenvector of A corresponding to the
eigenvector x for T .
EXAMPLE: (From page 617)


1 3 4


A =  3 1 2 ,
4 2 1
w
(2)
2
1
= 0,
,
sqrt(5) sqrt(5)


1
0
0

3 −4 

−
0
P1 = I − 2w(2) w(2)T = 

5
5 
3
0 − 45
5


1 −5
0

73 − 14 

−
5
T = A2 = 

25
25 
23
0 − 14
−
25
25
When the rounding errors inherent in producing T
from A are taken into account, how do the eigenvalues of T and A compare. This is given in the text
as Theorem 9.4 (page 617). It is assume that the
arithmetic being used is t-digit binary arithmetic with
rounding; and moreover, it is assumed that all inner
products
m
aj bj
j=1
are accumulated in a higher precision and then rounded
back to t digits at the completion of the summation
process. With this, we obtain for the eigenvalues λj
and τj
of A and T respectively, that

n 
τj − λj

 j=1


n


λ2j
j=1
cn = 25 (n − 1)
2
1
2



 ≤ cn2−t



2n−4
.
−t
1 + (12.36) 2
=
25 (n − 1)