Lecture 17
18.086
Krylov Subspaces
The linear combinations of b, Ab, . . . , AJplb form the j t h Krylov subspace. This space
depends on A and b. Following convention, I will write K j for that subspace and K j
for the matrix with those basis vectors in its columns:
Krylov subspaces
...
Krylov matrix
Krylov subspace
K j = [ b Ab A2b
Aj-lb].
1Cj = a l l combinations of b, Ab,
. . .,Aj-l b.
(3)
• There
⇢ Kn the
=K
= Kn+2 =as...
is column
an integer
n such
Thus
ICj is the
space of
K j . Wethat
wantK
toj choose
best
combination
n+1
our improved x,. Various definitions of "best" will give various x,. Here are four
• n is the maximum dimension that the Krylov subspaces can have.
is the important decision:
different approaches to choosing a good x j in &-this
•
If A is NxN, then n  N
=b )
Knis orthogonal to ICj (Conjugate Gradients).
1. • Ax
The⇤residual
r, =xb⇤-2Axj
2. • Krylov:
The residual
has minimum norm
for x, in
K jbe
( Gfound
M R E Sinand
M I N Rj<n
ES).
Goodr, approximations
to Ax=b
can
spaces
Find optimal
in j-th Krylov
space, then
iterativelyGradients).
go to
3. • Idea:
r, is orthogonal
to axdifferent
space ICj(AT)
(BiConjugate
higher Krylov spaces and improve x
The error e, has minimum norm in ICj (SYMMLQ).
4.
•
Optimal x can be defined in terms of residual, e.g.
•
Optimize x such that
•
Optimize x such that
rk ? K j
|rk |
(conjugate gradient, A sym., pos. def.)
is minimal (GMRES, MINRES algorithm)
Arnoldi orthogonalization
=> Need a good basis of Krylov spaces:
7.4 Krylov Subspaces and Conjugate Gra
1. orthonormal
2. built iteratively
see lecture / book
Arnoldi's
orthogonalization
=>
Arnoldi orthogonalization
ql = blllbll;
f o r j = 1,..., n - 1
1
t = Aqj;
f o r i = 1,. . . , j
h23. .= q.2Tt;
2
3
t = t - hijqi;
0
end
hj+l,j = Iltll;
qj+l = t/hj+l,j;
4
5
end
of b, Ab, .. . ,A"-1 b:
% Normalize b to 11 ql 11 = 1
% Start computation of qj+l
% one matrix multiplication
% t is in the space K j + l
% hijqi = projection o f t on qi
% Subtract that projection
% t is orthogonal to ql, . . . , qj
% Compute the length of t
% Normalize t to llqj+lll = 1
% ql, . . . , qn are orthonormal
hll = 512
t = Aqi - (5/
t = [ - 3 -1
h21 = &/2
q2 = [-3 - 1
basis for Kryl
CG algorithm
•
•
Basic idea: Iterative method with finds optimal xk at each step k
with xk 2 Kk
CG: Optimal <=> rk ? Kk
rk ? Kk ) rk> ri for i < k
•
Again:
•
In particular: rk = ±|rk |qk+1
with q’s an orthonormal basis,
Kj = span(q1 , . . . , qj )
•
Important requirement: A symmetric, positive definite!