A Very Short Introduction on Multigrid Methods
Qinghai Zhang
Motivation of multigrids:
2
The residual equation
• The convergence rates of multigrid methods does
not depend on the grid spacing or problem size,
Definition 2. For an approximate solution ũj ≈ uj , the
error is e = u − ũ, the residual is r = f − Aũ. Then
Ae = r
• The complexity is O(n) and thus optimal.
(8)
holds and it is called the residual equation.
1
The model problem
Question 1. For inexact arithmetic, does a small residual imply a small error?
On the unit 1D domain x ∈ [0, 1], we numerically solve
Poisson equation with homogeneous boundary condition Definition 3. The condition number of a matrix A is
cond(A) = kAk2 kA−1 k2 . It indicates how well the resid∆u = f,
x(0) = x(1) = 0.
(1) ual measures the error.
Discretize the domain with n cells and locate the Theorem 4.
knowns fj and unknowns uj at nodes xj = j/n = jh,
krk2
kek2
krk2
1
≤
≤ cond(A)
j = 0, 1, . . . , n. We would like to approximate the seccond(A) kf k2
kuk2
kf k2
ond derivative of u using the discrete values at the nodes.
Using Taylor expansion, we have
3
(9)
Fourier modes and aliasing
∂ 2 u uj+1 + uj−1 − 2uj
+ O(h2 ).
=
∂x2 jh
h2
(2) Hereafter Ωh denote both the uniform grid with n intervals and the corresponding vector space.
Wavelength refers to the distance of one sinusoidal
Definition 1. The one-dimensional second-order disperiod.
crete Laplacian is a Toeplitz matrix A ∈ R(n−1)×(n−1)
as

Proposition 5. The kth Fourier mode wk,j = sin(xj kπ)
i=k
 2,
has wavelength Lw = k2 .
−1, i − k = ±1
(3)
aij =

0,
otherwise
The wavenumber k is the total number of crests and
troughs in the unit domain.
Then we are going to solve the linear system
Question 2. What is the range of representable
Au = f ,
(4) wavenumbers on Ωh ?
Proposition 6. On Ωh , a Fourier mode wk = sin(xj kπ)
with n < k < 2n is actually represented as the mode wk0
where k 0 = 2n − k.
where fj = h2 f (xj ).
Proposition 1.
Definition 4. On Ωh , the Fourier modes with wavenum∀j = 1, . . . , n − 1. (5) bers k ∈ [1, n/2) are called low-frenquency (LF) or
smooth modes, those with k ∈ [n/2, n − 1) highProposition 2. The eigenvalues λk and eigenvectors wk frenquency (HF) or oscillatory modes.
of A are
kπ
λk (A) = 4 sin2
,
(6) 4
Weighted Jacobi
2n
1
(Au)j − (∆u)|xj = O(h2 ),
h2
wk,j = sin
where j, k = 1, 2, . . . , n − 1.
jkπ
,
n
Classical iterative methods split A as A = M − N
(7) and convert (4) to a fixed point (FP) problem u =
M −1 N u + M −1 f . Let T = M −1 N , c = M −1 f . Then
fixed point iteration yields
Remark 3. The 2D counterpart of A is A ⊗ I + I ⊗ A.
u(`+1) = T u(`) + c.
1
(10)
A Very Short Introduction on Multigrid Methods
Qinghai Zhang
5
After ` iterations
Two-grid correction
h
(11) Proposition 8. The kth mode on Ω becomes the kth
2h
mode on Ω :
h
2h
wk,2j
= wk,j
.
(19)
Obviously, the FP iteration will converge iff ρ(T ) =
n n
|λ(T )|max < 1.
h
However, the LF modes k ∈ [ 4 , 2 ) of Ω will become HF
Decompose A as A = D + L + U . Consider a gener- modes on Ω2h .
alization of the Jacobi iteration.
Definition 7. The restriction operator Ih2h : Rn−1 →
n/2−1
maps a vector on the fine grid Ωh to its counterDefinition 5. The weighted Jacobi is given by the fol- R
part on the coarse grid Ω2h :
lowing.
e(`) = T ` e(0) .
u∗ = −D−1 (L + U )u(`) + D−1 f,
u(`+1) = (1 − ω)u(`) + ωu∗ ,
Ih2h v h = v 2h .
(12)
(20)
A common restriction operator is the full-weighting operator
1 h
h
h
+ 2v2j
+ v2j+1
),
(21)
vj2h = (v2j−1
4
where j = 1, 2, . . . , n2 − 1.
(13)
Setting ω = 1 yields Jacobi.
Proposition 7. The weighted Jacobi has the iteration
Definition 8. The prolongation or interpolation opermatrix
h
ator I2h
: Rn/2−1 → Rn−1 maps a vector on the coarse
ω
2h
h
(14) grid Ω to its counterpart on the fine grid Ω :
Tω = (1 − ω)I − ωD−1 (L + U ) = I − A,
2
h 2h
I2h
v = vh .
(22)
whose eigenvectors are the same as those of A, with the
A common prolongation is the linear interpolation opercorresponding eigenvalues as
ator
h
v2j
= vj2h ,
kπ
(23)
h
2h
λk (Tω ) = 1 − 2ω sin2
,
(15)
v2j+1 = 12 (vj2h + vj+1
).
2n
Definition 9. For Au = f , the two grid correction
where k = 1, 2, . . . , n − 1.
scheme
P
v h ← MG(v h , f h , ν1 , ν2 )
(24)
Write e(0) = k ck wk , then
consists of the following steps:
X
(`)
` (0)
`
e = Tω e =
ck λk (Tω )wk .
(16) 1) Relax Ah uh = f h ν1 times on Ωh with initial guess
k
v h : v h ← Tων1 v h + c0 (f ),
Having accepted that no value of ω damps the smooth 2) compute the fine-grid residual rh = f h − Ah v h and
restrict it to the coarse grid by r2h = Ih2h rh : r2h ←
components satisfactorily, we ask what value of ω proIh2h (f h − Ah v h ),
vides the best damping of the oscillatory modes.
2h 2h
2h
2h
2h
2h −1 2h
Definition 6. The smoothing factor µ is the smallest 3) solve A e = r on Ω : e ← (A ) r ,
factor by which the HF modes are damped per iteration. 4) interpolate the coarse-grid error to the fine grid by
h 2h
An iterative method is said to have the smoothing propeh = I2h
e and correct the fine-grid approximation:
h
h 2h
erty if µ is small and independent of the grid size.
v ← v h + I2h
e ,
h h
h
h
For weighted Jacobi, this leads to an optimization 5) Relax A u = f ν2 times on Ω with initial guess
h
h
ν1 h
0
v : v ← Tω v + c (f ).
problem
µ=
min
k∈[n/2,n)
λk (Tω ),
for ω ∈ (0, 1].
Exercise:
ω=
2
1
⇒ |λk | ≤ µ =
3
3
Proposition 9. Let T G denote the iteration matrix of
(17) the two-grid correction scheme. Then
h
T G = Tων2 I − I2h
(A2h )−1 Ih2h Ah Tων1 .
(25)
Our objective is to show that T G ≈ 0.1 for small ν1
and ν2 .
(18)
2
A Very Short Introduction on Multigrid Methods
5.1
The spectral picture
Definition 10. wkh (k ∈ [1, n/2)) and wkh0
are called complementary modes on Ωh .
Qinghai Zhang
Proposition 15. A basis for the range of the interpolah
tion operator R(I2h
) is given by its columns, hence dim
(k 0 = n − k)
n
h
h
R(I2h ) = 2 − 1. N (I2h
) = {0}.
Proposition 10. For a pair of complementary modes on Corollary 16. For the full-weighting operator,
Ωh , we have
n
n
h
dim R(Ih2h ) = − 1,
dim N (Ih2h ) = .
wkh0 ,j = (−1)j+1 wk,j
(26)
2
2
(32)
Lemma 11. The action of the full-weighting operator Proposition 17. A basis for the null space of the fullon a pair of complementary modes is
weighting operator is given by
kπ
w2h = ck wk2h ,
Ih2h wkh = cos2
(27a)
N (Ih2h ) = span{Ah ehj : j is odd},
(33)
2n k
kπ 2h
h
h
w = −sk wk2h ,
(27b) where ej is the jth unit vector on Ω .
2n k
h
The basis vector of N (Ih2h ) are thus of the form
where k ∈ [1, n/2), k 0 = n−k. In addition, Ih2h wn/2
= 0.
Ih2h wkh0 = − sin2
Lemma 12. The action of the interpolation operator on
(0, 0, . . . , −1, 2, −1, . . . , 0, 0)T .
Ω2h is
h
I2h
wk2h = ck wkh − sk wkh0 ,
(28) Hence N (Ih2h ) consists of both smooth and oscillatory
modes.
where k 0 = n − k.
Hence, the range of the interpolation operator con- Theorem 18. The null space of the two-grid correction
tains both smooth and oscillatory modes. In other words, operator is the range of interpolation:
it excites oscillatory modes on the fine grid. However, if
2
h
N (T G) = R(I2h
).
(34)
k n, the amplitudes of these HF modes sk ∼ O( nk 2 ).
Theorem 13. The two-grid correction operator is inNow that we have both the spectral decomposition
variant on the subspaces Wkh = span{wkh , wkh0 }.
Ωh = L ⊕ H and the subspace decomposition Ωh =
h
h
) ⊕ N (I2h
), the combination of relaxation with TG
R(I2h
T Gwk = λνk1 +ν2 sk wk + λνk1 λνk20 sk wk0
(29a)
correction is equivalent to projecting the initial error vecT Gwk0 = λνk10 λνk2 ck wk + λνk10 +ν2 ck wk0 ,
(29b) tor to the L axis and then to the N axis.
where λk is the eigenvalue of Tω .
(29) can be rewritten as
wk
c
TG
= 1
wk 0
c3
6
c2
c4
wk
.
wk0
Multigrid cycles
(30) Definition 11. The V-cycle scheme is an algorithm
vh ← V h (vh , f h , ν1 , ν2 )
(35)
Consider the three cases of (1) ν1 = 0, ν2 = 0, (2)
ν1 > 0, (3) ν1 = 0, ν2 > 0. For the second case of ν1 > 0, with the following steps.
2
if k n, although λνk1 +ν2 ≈ 1, sk ∼ nk 2 , hence c1 1.
h h
h
Also, λνk10 1, hence c2 , c3 , c4 1. See Figure 1 for four 1) relax ν1 times on A u = f with a given initial guess
h
v ,
examples.
5.2
2) if Ωh is the coarsest grid, go to step 4), otherwise
The algebraic picture
f 2h ← Ih2h (f h − Avh ),
Lemma 14. The full-weighting operator and the linearinterpolation operator satisfy the variational properties
h
I2h
= c(Ih2h )T , c ∈ R.
h
Ih2h Ah I2h
= A2h .
(31b) is also called the Galerkin condition.
v2h ← 0,
v2h ← V h (v2h , f 2h ).
(31a)
(31b) 3) interpolate error back and correct the solution: vh ←
h 2h
vh + I2h
v .
3
A Very Short Introduction on Multigrid Methods
Qinghai Zhang
1
1
c1
c2
c3
c4
0.9
c1
c2
c3
c4
0.9
0.8
0.8
0.7
0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0
0
0
10
20
30
40
50
60
70
0
10
20
30
(a) ν1 = 0, ν2 = 0
40
50
60
70
(b) ν1 = 0, ν2 = 2
0.12
0.12
c1
c2
c3
c4
c1
c2
c3
c4
0.1
0.1
0.08
0.08
0.06
0.06
0.04
0.04
0.02
0.02
0
0
0
10
20
30
40
50
60
70
0
10
20
(c) ν1 = 2, ν2 = 0
30
40
50
60
70
(d) ν1 = 2, ν2 = 2
Figure 1: The damping coefficients of two-grid correction with weighted Jacobi for n = 64. The x-axis is k.
4) relax ν2 times on Ah uh = f h with the initial guess
vh .
3), otherwise
f 2h ← Ih2h f h ,
Definition 12. The Full Multigrid V-cycle is an algorithm
vh ← F M Gh (f h , ν1 , ν2 )
(36)
v2h ← F M G2h (f 2h , ν1 , ν2 ).
h 2h
2) correct vh ← I2h
v ,
with the following steps.
3) perform a V-cycle with initial guess vh :
V h (vh , f h , ν1 , ν2 ).
1) If Ω is the coarsest grid, set v ← 0 and go to step
h
h
4
vh ←