ETNA
Electronic Transactions on Numerical Analysis.
Volume 41, pp. 93-108, 2014.
Copyright  2014, Kent State University.
ISSN 1068-9613.
Kent State University
http://etna.math.kent.edu
THE BLOCK PRECONDITIONED STEEPEST DESCENT ITERATION FOR
ELLIPTIC OPERATOR EIGENVALUE PROBLEMS
KLAUS NEYMEYR AND MING ZHOU
Abstract. The block preconditioned steepest descent iteration is an iterative eigensolver for subspace eigenvalue
and eigenvector computations. An important area of application of the method is the approximate solution of mesh
eigenproblems for self-adjoint elliptic partial differential operators. The subspace iteration allows to compute some
of the smallest eigenvalues together with the associated invariant subspaces simultaneously. The building blocks of
the iteration are the computation of the preconditioned residual subspace for the current iteration subspace and the
application of the Rayleigh-Ritz method in order to extract an improved subspace iterate.
The convergence analysis of this iteration provides new sharp estimates for the Ritz values. It is based on
the analysis of the vectorial preconditioned steepest descent iteration which appeared in [SIAM J. Numer. Anal.,
50 (2012), pp. 3188–3207]. Numerical experiments using a finite element discretization of the Laplacian with up
degrees of freedom and with multigrid preconditioning demonstrate the near-optimal complexity of the
to
method.
Key words. subspace iteration, steepest descent/ascent, Rayleigh-Ritz procedure, elliptic eigenvalue problem,
multigrid, preconditioning
AMS subject classifications. 65N12, 65N22, 65N25, 65N30
1. Introduction. Let be a bounded polygonal region in , often or ,
and let the boundary be the disjoint union of and . We consider the self-adjoint
elliptic eigenvalue problem
(1.1)
"!$#%!'&)(+*(-,/.0!1&2(3* 5
4 *
* 9
:
8
;#<!'&2(=+* 98
in 76
on 6
%>
on Here #<!'&2( is a: symmetric and positive definite matrix valued function, .0!1&2( a nonnegative real
function and denotes the normal vector on the boundary . The coefficient functions are
assumed piecewise continuous. We are interested in the numerical approximation of some of
the smallest eigenvalues 4 and the corresponding eigenfunctions * .
The finite element discretization of (1.1) yields the generalized matrix eigenvalue problem
? 2& @
4 @CBD&E@ >
A
:GFH:
?
B
The discretization matrix and the mass matrix
are symmetric, positive definite
(1.2)
matrices. Typically, these matrices are very large and sparse.
Our aim is to compute a fixed number of the smallest eigenvalues of (1.2) together with
:
the associated eigenvectors. The numerical algorithm should exploit the structure of the mesh
eigenproblem, and its computational costs should increase almost linearly in dimension .
The demand for a near-optimal-complexity method rules out all eigensolvers which are usually used for small and dense matrices. See [2, 4, 21] for getting an overview on the wide
variety of numerical eigensolvers primarily for small and dense matrices.
I
Received January 23, 2014. Accepted February 24, 2014. Published online on May 23, 2014. Recommended
by M. Hochstenbach.
Universität Rostock, Institut für Mathematik, Ulmenstraße 69, 18055 Rostock, Germany
( klaus.neymeyr, ming.zhou @uni-rostock.de).
J
K
93
ETNA
Kent State University
http://etna.math.kent.edu
94
K. NEYMEYR AND M. ZHOU
A conceptually easy approach to the desired near-optimal-complexity eigensolvers is
based on gradient iterations for the Rayleigh quotient; cf. Dyakonov’s monograph on optimization for elliptic problems [3] and its Chapter 9 on the Rayleigh-Ritz method for spectral
problems. The starting point is the generalized Rayleigh quotient for (1.2)
(1.3)
L !1&2( '! & 6 ? &)(=MN!'& 6 BD&2( 6
&P5
O 8 >
As L !3Q( attains its minimum 4 at an associated eigenvector, the minimization of (1.3) can be
implemented by means of a gradient iteration. The negative gradient of (1.3) reads
L !'&)( ! ? R
& L !'&)(BD&2(SM0!1& 6 BD&)(
and allows us to construct a new iterate &)T &UHV L !'&2( with L !1&ETW(YX L !1&2( , whenever & is
not an eigenvector and V5Z is a proper step length. The optimal step length V minimizes
L !'&ETW( with respect to V[Z . These optimal &)T and L !'&2TW( can be computed by applying the
Rayleigh-Ritz procedure
to the 2D space spanned by & and L !'&2( . The gradient iteration
?
does not change
and B and does not need these matrices explicitly. It is a so-called
?
matrix-free method in the? sense that its implementation only requires routines \^]_
\ and
\`]_ B \ . The sparsity of and B allows us to implement these matrix-vector products with
computational cost which only increases linearly in the number of unknowns.
1.1. Preconditioning of gradient iterations. Gradient iterations for the Rayleigh quotient which use the? Euclidean gradient L !'&)( are well known to converge poorly if the condition number of is large [5, 6, 7, 8, 12, 23, 28]. This is particularly
the case for a finite
?
element discretization of (1.1) since the condition number of increases as a !$bdc ( in the
b
discretization parameter .
The key ingredient to make a gradient iteration an efficient solver for the operator eigenvalue
problem (1.1) is multigrid
If a symmetric and positive
definite matrix
? preconditioning.
?
e
e
is an approximate inverse of , then is called a preconditioner for . The preconditioned
e
gradient iteration uses L !1&2( as the descent direction for the Rayleigh quotient. The use
age of the -gradient can be interpreted as a change of the underlying geometry which makes
ellipsoidal level sets of L !3Q( more spherical [3, 25]. Proper preconditioning, for instance by
multigrid or multilevel preconditioning, can accelerate the convergence considerably. In the
best case this can result in grid-independent convergence behavior [10, 11].
e
The -gradient steepest descent iteration optimizes the step length V
& T &fV e L !1&2( &U V e ! ? &R L !'&)(BD&2(SM0!1& 6 BD&)(
e ?
in a way that & T attains the smallest possible Rayleigh quotient for all VgZ . If ! &h
L !'&2(=BD&2( is a Ritz vector, then V may be infinite. Computationally &T is determined by the
?
Rayleigh-Ritz
procedure since &)T is a Ritz vector of ! 6 B ( in the two-dimensional subspace
?
e
i=jlk%mn & 6 ! &o L !1&2(BD&)(qp corresponding to the smaller Ritz value. The resulting itera(1.4)
tion has some similarities with the Davidson method [20, 24] if the iteration is restarted after
each step so that the dimension of the truncated search subspace always equals 2. Stathopoulos [27] calls such an iteration with constant memory requirement a generalized Davidson
method.
The eigenpairs of (1.2) are denoted by ! 4 @ 6 &E@r( . The enumeration is 8 X 4) X 4l X
X
>;>s> 4lt . We assume simple eigenvalues. The case of multiple eigenvalues can simply be
reduced to that of only simple eigenvalues by a proper projection of the eigenvalue problem.
Alternatively, a continuity argument can be used to show that colliding eigenvalues do not
change the structure of the convergence estimates; cf. [17, Lemma 2.1].
ETNA
Kent State University
http://etna.math.kent.edu
BLOCK PRECONDITIONED STEEPEST DESCENT
95
For the vectorial Preconditioned Steepest Descent (PSD) iteration (1.4) sharp convergence estimates have recently been proved in [17]. The central theorem reads as follows:
T HEOREM 1.1. Let &Z : t be such that the Rayleigh quotient L !1&2( satisfies 4 @X L !1&2(
X 4 @vu for some w Z nyx 6 >;>;> 6 x p . Let & T be the Ritz vector of ! ? 6 B ( in i=jlk%m)n & 6 e ! ? &+
L !1&2(=BD&2(qp which corresponds to the smaller Ritz value. The Ritz value is L !'&TW( . The preconditioner satisfies
z '! & 6 e c &2(Y{|!'& 6 ? &2(}{ z !'& 6 e c &2(~€&oZ t
with constants z 6 z ƒ‚„8 . Let zH… ! z z (SM0! z , z ( .
Then either L !'&2TW(Y{ 4 @ or L !1&ETW(Z†! 4 @ 6S4 @‡u ( is bounded from above as follows
, z ! ( @1Š @vu ! L !1&ETW((
4 @ ! 4 t 4 @vu (
(1.6)
with Œ 8 X‰ˆ @'Š @vu ! L !'&2(=( {‹Œ ! ( , z Œ
4 @vu ! 4Et 4 @( >
ˆ
Œ
ŒŽ
Here ˆƒ Š ‘’!1“"( … !1“” 4  (=MN! 4 ‘Y“"( .
The estimate (1.6) cannot be improved in terms of the eigenvalues 4 @ , 4 @‡u and 4 t . The
bound can be attained for 4•_–4 @ in the three-dimensional invariant subspace — @'Š @vu Š t , which
is spanned by the eigenvectors corresponding to 4 @ , 4 @vu and 4 t .
Theorem 1.1 guarantees monotone convergence of the Ritz values L !1&TW( towards an
eigenvalue. If the Rayleigh quotient L !'&)( has reached the final interval ˜ 4-’6q4l ( , then (1.6)
proves linear convergence of the Ritz values towards the smallest eigenvalue 4d .
(1.5)
1.2. Aim of this paper. This paper aims at generalizing the preconditioned gradient
iteration (1.4) to a subspace iteration for the simultaneous computation of the ™ ? smallest
eigenvalues of (1.2). New and sharp convergence estimates for the Ritz values of ! 6 B ( in
the iteration subspaces are presented, which generalize Theorem 1.1.
The effectiveness of the method is demonstrated for the Laplacian eigenvalue problem
on the sliced unit circle with mixed homogeneous Dirichlet and Neumann boundary conditions. The operator eigenvalue problem is discretized with linear and quadratic (only for error
estimation) finite elements. Our finite element code AMPE, see Section 4, is demonstrated
to work with up to š x 8"› degrees of freedom. AMPE uses a multigrid preconditioner with
Jacobi smoothing.
1.3. Notation and overview. Subspaces are denoted by capital calligraphic letters, e.g.,
the column space of a matrix œ is ž ijlk<mn œ p . Similarly, ijlk<m)n œŸ6= p is the smallest
vector space which contains the column spaces of œ and of . All eigenvalues and Ritz
values are enumerated
in order of increasing magnitude. Further, — ? index-set denotes the invari?
ant subspace of ! 6 B ( which is spanned by the eigenvectors of ! 6 B ( whose indexes are
contained in the index set.
This paper is structured as follows: The preconditioned gradient subspace iteration is
introduced in Section 2. The new convergence estimates are presented in Section 3; Theorem 3.4 is the central new estimate on the convergence of the Ritz values. In Section 4
numerical experiments illustrate the sharpness of the estimates and the cluster-robustness of
the preconditioned gradient subspace iteration.
2. The block preconditioned iteration with Rayleigh-Ritz projections. The reconditioned subspace/block iteration is a straightforward generalization of vector iteration (1.4).
The iteration (1.4) is just applied to each column of the matrix which columnwise contains
the Ritz vectors of the current subspace. Each subspace correction step is followed by the
Rayleigh-Ritz procedure in order to extract the Ritz approximations.
ETNA
Kent State University
http://etna.math.kent.edu
96
K. NEYMEYR AND M. ZHOU
t
which is given by the column
The starting point is an ™ -dimensional subspace ¡ of Z tE£l¤ . The columns of ¢ are assumedF to be the B -normalized Ritz vectors of
¢
space
of
! ? 6 B ( in ¡ . Further, ¥AA¦0§ k%¨ !'“ %6 >s>;> 6 “ ¤ ( is the ™ ™ diagonal matrix of the corresponding
Ritz values. The matrix residual
?
¢ †B ¢€¥ Z tE£N¤
contains e
columnwise
with the ? precon© ? c the residuals of the Ritz vectors. Left multiplication
™
coincides with the approximate
solution
of
linear
systems
in . This
ditioner
e ?
yields the correction-subspace i=jlk<mn ! ¢ 5B ¢+¥ (ªp of the preconditioned subspace iteration. The Rayleigh-Ritz? procedure is usede to? extract the ™ smallest Ritz values and the
associated Ritz vectors of ! 6 B ( in ijEk%m)n ¢6 ! ¢ PB ¢+¥ (qp ; see Algorithm 1.
Algorithm 1 Block preconditioned steepest descent iteration.
?
e
Require: Symmetric and positive definite matrices 6 B«Z tE£Nt , a preconditioner satisfying (1.5) and an initial (random) ™ -dimensional
subspace ¡7¬®­S¯ with °± ! ¡¬®­q¯q6—l=² ¤ (X
³ M where —E=² ¤ is the invariant subspace of ! ? 6 B ( associated with the ™ smallest eigen:´F
values.
1. Initialization: Apply the Rayleigh-Ritz procedure to ¡7¬‡­S¯ . The
™ matrix
¬‡­S¯ ¸ contains the Ritz vectors of ! ? 6 B ( corresponding to the ™
¢µ¬®­q¯¶[˜ · ¬‡­S¯ 6 >;>;> · ¤|
?
¢µ¬®­q¯ Ritz values “ ¬®­q¯ 6 >s>;> 6 “ ¤ ¬®­q¯ . Let ¥`¬®­q¯Y ¦0§ k<¨ !1“ ¬®­q¯ 6 >s>;> 6 “ ¤ ¬®­q¯ ( and ¹+¬®­q¯Y
B ¢ ¬®­q¯ ¥ ¬®­q¯ .
i=jlk%mn ¢f¬ @ ¯
6 e ¹+¬ @ ¯ p .
2. Iteration, w‚|8 : The Rayleigh-Ritz
@‡u procedure
@vu is applied to
@vu ?
¯º»˜ · ¬ ¯ 6 >s>;> · ¤ ¬ ¯ ¸ @‡are
The columns of ¢R¬
the Ritz@‡u vectors of ! 6 B (
u
“ ¬ ¯ 6 >;>s> 6 “ ¤ ¬ ¯ . Let ¥`¬ @vu ¯
corresponding
to the
™ smallest@vu Ritz values
v
@
u
v
@
u
‡
@
u
@vu
@‡u
?
¦0§ k<¨ !1“ ¬ ¯ 6 >s>;> 6 “ ¤ ¬ ¯ ( and ¹+¬ ¯¼ ¢R¬ @‡u ¯ PB ¢µ¬ ¯¥`¬ ¯ .
¯
½ drops below a specified accu3. Termination: The iteration is stopped if ½
¹€¬
racy.
The block steepest
? c iteration as given in Algorithm 1 has already been analyzed
e descent
for the ? special case
(preconditioning with the exact inverse of the discretization
matrix ) in [19]. If in the central convergence estimates of this paper, see Corollary 3.3 and
Theorem 3.4, z A8 is inserted, then the convergence factor
z
Œ ! @ , @r! ( , z Œ @ ( @ with Œ @ 42¾q¿ u ! 4 t ! 42 ¾q¿ u ((
Œ
4 ¾ ¿ 4Et 4 ¾ ¿
Œ
simplifies to Π@ M0! Π@ ( . This bound has been proved in case 1b of [19, Theorem 1.1].
However, the convergence analysis for the block steepest descent iteration with a general
preconditioner does not rest upon the analysis in [19]. In fact, the analysis of the current
paper uses the convergence analysis for the vectorial preconditioned steepest descent iteration
from [17]. By starting with the result from [17] some of the proof techniques from [19] can
be adapted to the block iteration with general preconditioning. One important modification
is that the argument using the subspace dimensions from Theorem 3.2 of [19] cannot be
transferred to the case of general preconditioning. Instead, we use here Sion’s theorem for
the proof of Theorem 3.2. The proof structure of the present paper has also some similarities
with the chain of arguments in [14] wherein a convergence analysis of the subspace variant
of the preconditioned inverse iteration is contained. References to similar results and similar
proof techniques are given prior to Lemma 3.1, Theorem 3.2 and Theorem 3.4.
ETNA
Kent State University
http://etna.math.kent.edu
97
À`Å5ƆÀ
BLOCK PRECONDITIONED STEEPEST DESCENT
ÁƒÂÃ
Ä
PSfrag replacements
F IG . 3.1.
È
ÁƒÇ ÂÃsÄ
À€Å
À
-orthogonal decomposition in the
É-ÊÌË<Í .
3. Convergence
analysis. The convergence estimates on the convergence of the Ritz
“ Î ¬ @vu ¯ with Ï5 x 6 >s>;> 6q™ and wh‚Ð8 , see Algorithm 1, towards the eigenvalues of
values
! ? 6 B ( are proved in this section. In Theorem 3.2 together with Corollary 3.3 the conver@‡u ¯ , is analyzed. It is shown that the
gence of the largest
of these Ritz values,? namely “̤T … “ ¤ ¬
?
e
Ritz value “"¤T of ! 6 B ( in ijEk%m)n ¢6 ! ¢ /B ¢`¥ (ªp is bounded from above by the largest
Ritz value which can be attained by the vector iteration PSD (1.4) if applied elementwise to
the column space of ¢ . Theorem 3.4 proves sharp estimates for the remaining Ritz values by
induction.
following analysis we assume that the preconditioner satisfies the inequality ½;Ñ e ? ½;InÒ the
{ z X x . This assumption is equivalent to (1.5) with z9… ! z z (SM0! z , z ( if
the preconditioner is optimally scaled. The PSD iteration implicitly determines this optimal
scaling by the Rayleigh-Ritz procedure.
The next lemma generalizes Lemma 3.1 from [14].
tE£N¤ form a Ritz basis of i=jlk%mn ¢ p , and let e be
L EMMA 3.1. Let the columns of ¢ Z e ? ½sÒ { z X x .
a symmetric and positive definite matrix with ½
Ñ ¤ŸÔ n 8 p and any #7Z ¤ it holds that
i) For any Ó Z ?
?
½ c B ¢`¥7Ó `
¢ Ó½ Ò { ½ c B ¢€¥7Ó ¢ # ½ Ò>
e ? B
The matrix V €
¢ ¥ ,ÖÕ ¢ ! ¢ G
¢`¥ (=× preserves for all VAZ ™ of the matrix ¢ .
Ø Z t satisfy
Let & be given as in Theorem 1.1, and let &h
?
?
(3.2)
½ c BD&R &Ø ½;Ò { z ½ c BD&R&M L !'&)( ½
Ò >
(3.1)
ii)
iii)
the full rank
If
(3.3)
& T Z k<Ù=¨ ÛÜ%Ý'ÞsÚ ß=à’§ m á=⠊ âã ä L ! · ( 6
then the PSD estimate (1.6) applies to the Ritz vector
Rayleigh quotient L !1&2TW( .
?
Proof. i) The -orthogonality, see Fig. 3.1,
&2T
given by (3.3) and its
! ¢ƒ\E6 ! ? c B ¢`¥ ¢ ( Ó ( Ò 9\yåd¢`å !$B ¢€¥ ? ¢ ( Óµ5\yå ! ¥ ¥ ( Óµ98 ~ \ Z ¤
?
? ¢ ¥7Ó on ijlk<mn ¢ p . This guarantees
shows that ¢”Ó is the -orthogonal projection of c B `
that
?
?
½ c B ¢€¥7Ó ¢ƒÓ½
Ò { ½ c B ¢`¥7Ó ¢ # ½
Ò
ETNA
Kent State University
http://etna.math.kent.edu
98
K. NEYMEYR AND M. ZHOU
¤
for all #7Z .
ii) Direct computation shows that for Ó
Z ¤ŸÔ n 8 p
?
½ V ¢`¥7Ó ,A! ¢ e ! ¢ PB ¢`¥ (( Ó½ Ò
?
?
?
+½ V ¢`¥7Ó , c B ¢+¥7Ó ,A! Ñ e (
! ¢ c B ¢+¥ ( Ó½ Ò
?
?
?
‚”½ V ¢`¥7Ó , c B ¢+¥7Ó)½ Ò ½ ! Ñ e (
! ¢ c B ¢`¥ ( Ó½ Ò
æ ½ ? c B ¢+¥7Ó ¢ !V ¥7Ó ( ½ Ò ½ ! ¢ ? c B ¢+¥ ( Ó)½ Ò ‚º8 >
ç è
é ê
ë ²ì
e !? ¢ The last inequality holds with # V ¥7Ó due to (3.1). The fact ½ V ¢`e ¥7Ó ? ,D! ¢ B ¢`¥ (=( Ó½ Ò æ 8 for all Ó í
O 8 proves that the matrix V ¢`¥ ,|! ¢ ! ¢ „B ¢`¥ (( has
the full rank.
? ? iii) Inequality (3.2) is equivalent to ½ L !'&2( c BD&† L !'&2( &Ø ½ Ò { z ? ½ L !'&)( c BD&î
& ½ Ò . This means that L !1&2( &Ø is contained in a ball with the center L !'&2( c BD& and the
? radius z ½ L !1&2( c BD&}€& ½ Ò . The geometry of the fixed-step-length preconditioned gradient
iteration (see [10, 17]) shows that L !'&)( &Ø is representable by a symmetric and positive definite
e Ø with ½
Ñ e Ø ? ½;Ò { z in the form
L !'&)( &Ø &f e Ø ! ? &• L !'&)(BD&)( >
The preconditioned steepest descent iteration accelerates the convergence of this fixed-steplength iteration by applying the Rayleigh-Ritz procedure to
ijEk%mn & 6 e Ø ! ? f
& L !'&2(=BD&2(ªp i=jlk%mn & 6S&Ø p >
Thus the smallest Ritz value L !1&2TW( in this space is bounded from above by the estimate (1.6).
The next theorem proves the desired convergence behavior for the largest Ritz value
Comparable estimates for the largest Ritz values, but with respect to different spaces,
have been used in [14, Theorem 3.2] for the subspace analysis of the preconditioned inverse
iteration and in [19, Theorem 3.2] for the block steepest descent
? iteration.
T HEOREM 3.2. Let ijEk%m)n ¢ p contain no eigenvector of , which otherwise could
? easily
be split off within ? the Rayleigh-Ritz procedure. Then the ™ -th Ritz value “̤T of ! 6 B ( in
ï … ijlk<m)n ¢6 e ! ¢ PB ¢+¥ (qp , which is given by
“<¤T .
“ ¤T (3.4)
satisfies
(3.5)
e ?
k’ö Ú m L
y… þ
“ ¤T { úÜÚR
ûü ø á ­ ä ýE܁§ û Õ V ¢€¥7Ó , Õ ¢ ! ¢ PB ¢+¥ ( × Ó × “ ¤ >
Proof. Let Ó
attained
(3.6)
F !
Hence
Úµðùk%ø ö L ! \ ( 6
ð¶Ú ñ2§ ò m
ó;ô õ+ð ë ¤ ÷sÜ á ­ ä
Z ¤Ô n 8 p
and V
Z be given in a way so that the max-min in (3.5) is
“þ ¤ L Õ V ç ¢€èªé¥ ê Ó , Õ ¢ e ! ? ¢ PB ¢+¥ ( × Ó × >
ç
ªè é
ê
ë ² ÿ
ë ² ÿ
Ó 6 V ( is a saddle point of the Rayleigh quotient. In a sufficiently small neighborhood
of ! Ó 6 V ( the Rayleigh quotient L ! ( is a smooth function which is concave in Ó and
ETNA
Kent State University
http://etna.math.kent.edu
BLOCK PRECONDITIONED STEEPEST DESCENT
99
convex in V . Sion’s-minimax theorem [22, 26] provides the justification to swap the order of
minimization and maximization, i.e.,
“þ ¤ L !WV Ó , Ó ( Úµú Ük% ö Úý2ܧ m L !'V Ó , Ó ( ÚýEܧ m ÚRúÜk’ ö L !WV Ó , Ó ( >
( is the Rayleigh quotient of Ó with respect to the projected matrix-pair
L Thus
! ? !' V 6 ªÓ ,B ;( Ó with
V , . The local maximum in Ó Z coincides with
å
å
(because the maximum is taken in the
the global maximum of L ! ( in the column space of
interior of and because the Hessian of the Rayleigh quotient shows that there is only one
local extremum which is a maximum). We conclude that
Ú §m R
Ú k’ö L Ú m
ý2Ü ú Ü !'V Ó , Ó ( Eý ܧ “ õ ß
!WV , ( >
(.
We denote the minimum point with respect to VDZ by V so that “ þ ¤ “ õ ß
! V ,
?
Therefore “ þ ¤ is the largest Ritz value of ! 6 B ( in the column space  of V , Z ï tE£N¤ .
one
Lemma 3.1 guarantees that the dimension of  equals ™ . Since  is a subspace of
gets with (3.4) that “"¤T { ÚRk%ö ÷sÜ ðdø á ä L ! \ ( “ þ ¤ .
?
­
:
C OROLLARY
3.3. If the ™ -th Ritz value “ ¤ of ! 6 ? B ( in ¡ is contained? in ! 4 ‘ 6S4 ‘3u ( ,
.RZ nyx 6 >s>;> 6 x p , then for the ™ -th Ritz value “"¤T of ! 6 B ( in i=jlk%m)n ¢¶6 e ! ¢ /B ¢+¥ (qp
it holds that either “"¤T { 4 ‘ or
, z ! ( ‘ªŠ ‘3u !'“<T (
4 ‘ ! 4 t 4 ‘u (
(3.7)
with Œ 8 Xgˆ ‘ªŠ ‘3u !'“ ¤¤ ( { ‹Œ ! (, z Œ
4 ‘3u ! 4lt 4 ‘s( >
ˆ :
Œ
Œ¶Ž
x , then the first alternative “"¤T { 4Et c applies since the ™ -th Ritz value
If . ? Proof.
e ?
!
B
(
in the at least ! ™ , x ( -dimensional subspace ijEk%m)n ¢6 ! ¢ oB ¢€¥ (qp is smaller
of 6
than or equal to 4E: t c (due to the min-max principles).
x and assume 4 ‘7X “ ¤T for which (3.7) is to be proved. For 4 ‘7{„“µX 4
Next let .+X
'
!
"
“
q
‘
Š
the function ˆ ( !'“ƒ 4 ‘ (SM0! 4 †“"( is monotone increasing in “ . Theorem 3.2 shows
that “<¤T { “ þ ¤ so that
þ
T
(3.8)
ˆ ‘ªŠ ‘3u !'“ ¤ (}{ ˆ ‘qŠ ‘u ! “ ¤ ( >
e ! ? ¢ ^B ¢`¥ (=× Ó
In order to apply Lemma 3.1, Case iii, to & ´¢+¥7Ó and &Ø Õ ¢ we show that
?
½ c BD&f &Ø ½ Ò í½
í½
{ z
{ z
? c !Ñ ?
½ c
?
½ c
B +
¢ ¥7Ó
e ? (
! ? c
B ¢+¥7Ó
B ¢+¥7Ó
?
`
¢ Ó , e ! ¢
B ¢+¥7Ó ¢ƒÓ
¢ƒÓ ½
Ò
¢/ç ¥7Ó èªM é L !1&2ê (
ì
/B ¢`¥ ( Ó ½ Ò
( ½;Ò
?
½;Òî z ½ c BD&f&M L '! &)( ½ Ò >
The last inequality is proved with (3.1). Thus Case iii in Lemma
the PSD
? 3.1 guarantees that
estimate (1.6) can be applied to the smallest Ritz value of ! 6 B ( in ijEk%mn & 6S&Ø p . By (3.6)
this Ritz vector is V &•, &Ø with the Ritz value “ þ ¤ . Hence the vectorial PSD estimate (1.6)
results in
(3.9)
, z ! Œ ( ‘ªŠ ‘3u Õ L !'&)(×
þ
q
‘
Š
‘
u
!
“
(
{
‹
Œ
¤
>
! Œ (, z Œ Ž ˆ
ˆ
ETNA
Kent State University
http://etna.math.kent.edu
100
K. NEYMEYR AND M. ZHOU
This? concludes the proof since L
of ! 6 B ( in ijlk<mn ¢ p , i.e.,
!'&)( , &/Z ijlk<m)n ¢ p
(3.10)
, is bounded by the largest Ritz value “
¤
L
ˆ ‘ªŠ ‘3u Õ 1! &2(=ה{ ˆ ‘ªŠ ‘3u !'“ ¤ ( >
The chain of inequalities (3.8)–(3.10) proves the proposition.
The convergence estimates for the remaining Ritz values “Ì@T , w x 6 >;>;> 6S™ x , follow from
Corollary 3.3 by induction. Comparable estimates for the remaining ™ x Ritz values, but
with respect to different spaces, have been used in [14, Theorem 3.3] for the subspace analysis
of the preconditioned inverse iteration and in [19, Theorem 3.1] for the block steepest descent
iteration.
3.4. Let ¡ be an ™ -dimensional subspace of the t . The Ritz vectors of
?! B T HEOREM
6 ( in ¡ are denoted by ·y6 >;>;> 6·<¤ and let ¢ … ˜ ·y6 >;>;> 6·<¤ ¸ Z tE£N¤ . The associated
“ @ Z†! 42¾q¿
6q4E¾q¿ u ( .
Ritz values are “ @ L ! · @ ( with “ { ? >;>;> {G“ ¤ . Indices @ are
e ? given so that
The ™ smallest Ritz values of ! 6 B ( in i=jlk%mn ¢¶6 ! ¢ B ¢+¥ (ªp are denoted by “y@ T
with “"T { >;>s> {G“"¤T . Then for each w Z nyx 6 >;>;> 6S™ p it holds that “y@ T {„“ @ and either “"@T { 42¾q¿ or
Š ¾q¿ u !'“ @T ( { ‹ Œ @ , z ! Œ @ ( q
¾
¿
42¾q¿ ! 4 t 42¾q¿ u ( >
X
@
ˆ
8
(3.11)
with
z
! Œ @$(-, Œ @ Ž
Œ
4 ¾ ¿ u ! 4Et 4 ¾ ¿ (
ˆ ¾ ¿ Š ¾ ¿ u !'“’@$(
Here ˆ  Š ‘ !'“"( … !'“” 4  (=MN! 4 ‘ î“"( .
?
The bound (3.11) cannot be improved in terms of the eigenvalues of ! 6 B ( as for each
w the upper bound is attained for “%@ _ 4 ¾ ¿ in the 3D invariant subspace associated with the
eigenvalues 4 ¾ ¿ , 4 ¾ ¿ u and 4lt .
Proof. The proof is given by induction over the subspace dimension ™ . For a onedimensional subspace ¡ ijEk%mn &p , Theorem 1.1 proves the Ritz value estimate (3.11)
with L !'& T ( “ T .
?
For an ™ -dimensional subspace the Ritz vectors · @ of ! 6 B ( are arranged in the columns
of ¢ ? ˜ ·"%6 >s>;> 6=·<¤ ¸ … ˜ ¢µ¬ ¤ c ¯q6·<¤ ¸ Z tE£l¤ . Let “"@ T ! ¢R¬ ¤ c ¯ ( be the ™ x smallest Ritz values
of ! 6 B ( in
i=jlk%mn ¢ ¬ ¤ c ¯ 6 e Õ ? ¢ ¬ ¤ c ¯ †B ¢ ¬ ¤ c ¯ ¥ ¬ ¤ c ¯ × p 6 ¥ ¬ ¤ c ¯ … A¦0§ k<¨ !1“ ’6 >s>;> 6 “ ¤ c ( 6
with “<T ! ¢R¬ ¤ c ¯ (Y{ >;>s> {G“"¤T c ! ¢µ¬ ¤ c ¯ ( . The induction hypothesis with respect to the ! ™ x ( dimensional space reads
Š u !'“ @T ! ¤ c ((
, z ! ( 8 { ˆ ¾ ¿ ¾ ¿ ¾q¿ Š ¾q¿ u ¢ !1“ ¬ @ ( ¯ { ‹ Œ ! ( , z Œ
ˆ
Œ
Œ Ž
4 ¾ ¿ ! 4lt 4 ¾ ¿ u (
Œ E4 ¾q¿ u ! 4 t E4 ¾q¿ ( >
¤ The Courant-Fischer variational principles show that these ™ x smallest Ritz values “0@ T ! ¢µ¬ c ¯ (
¤
i
l
j
<
k
m
n
c
p
decrease when
¢µ¬ ¯ is enlarged to ¡ , i.e.,
Ú m
ÚRk%ö L ! N(
“ @T ! ¢ ¬ ¤ c ¯ ( ñ Ý'Þsß=àá ü Š Õ Ò ü§ c ± ü ü × ä Ü ø á ­ ä
å ósô õ ë @
¬ ¯
‚ ñ Ý'Þsß=àá Š Ú § Ò m ± ä ’Ü ÚRk’ø öá ä L ! N( “ @T >
ó;ô õ å ¬ ë c @
¯
­
¬ ¯
Finally, ˆ ¾q¿ Š ¾q¿ u !1“"( is a monotone increasing function in “ so that
¤ z
ˆ ¾ ¿ ŠŠ ¾ ¿ uu !'!'“<“ @@T (( { ˆ ¾ ¿ Š ¾ ¿ u Š !1“<u @T ! ¢f!'“ ¬ @ ( c ¯ (=( {‹ Œ ! , ! ( , z Œ ( 6
Œ
ˆ ¾q¿ ¾q¿ ˆ ¾q¿ ¾q¿ Œ¶Ž
with
ETNA
Kent State University
http://etna.math.kent.edu
BLOCK PRECONDITIONED STEEPEST DESCENT
101
which proves the proposition for the first ™ x smallest Ritz values. For the Ritz value “ ¤
Corollary 3.3 completes the proof.
The sharpness of the estimate (3.11) is a consequence of Theorem 1.1 since for “ @ Z
! 42¾q¿ª6S42¾q¿ u ( the columns of ¢ can be formed by the vector of poorest convergence in —)¾q¿ Š ¾q¿"!# Š t
: all other columns are taken as eigenvectors with indexes different form @ , @ , x and
and
. Then the subspace iteration behaves like the vectorial preconditioned steepest descent
iteration due to stationarity of the iteration in the eigenvectors.
C OROLLARY 3.5. The Ritz value convergence estimate (3.11) cannot be improved in
terms of the eigenvalues 4)¾q¿ , 42¾q¿ u and 4 t and is attainable for “ @ _–42¾q¿ in —N¾q¿ Š ¾q¿ u Š t .
Proof. For “ @ Z†! 4E¾q¿;6S42¾q¿ u ( let · @ be an B -normalized vector in the invariant subspace
Š
—N¾q¿ ¾q¿ u Š t with L ? ! · @ ( “ @ . The remaining columns of ¢ are filled with pairwise different
eigenvectors of ! 6 B ( which are B -orthogonal to —E¾q¿ Š ¾q¿ u Š t . Then a step of the block
steepest descent iteration, Algorithm 1, leaves all the eigenvectors invariant. The convergence
of the w th column · @ is exactly that of the vectorial preconditioned steepest descent iteration
as treated in Theorem 1.1 because the iteration is stationary in all eigenvectors. The nonimprovability of the convergence estimate in terms of the eigenvalues has already been treated
in Theorem 1.1.
Corollary 3.5 shows that the convergence estimate (3.11) cannot be improved in terms
of the eigenvalues without further assumptions on the subspace ¡ . Hence cluster robust convergence estimates, which should depend in some way on the ratio 4 @rM 4E¤ u , are not provable
in terms of the convergence measure ˆ ‘ªŠ ‘3u as used in Theorem 3.4. The numerical experiments in Section 4 illustrate that the block preconditioned steepest descent iteration behaves
as a cluster robust iteration. In order to derive cluster robust convergence estimates, additional
assumptions have to be made: For instance the angle between the iteration subspace and the
invariant subspace has to be bounded.
4. Numerical experiments. The block preconditioned steepest descent iteration is applied to the Laplacian eigenvalue problem
(4.1)
ˆ * !'&2( 54 *ù!1&2( 6
&oZ … %$ Õ&('*) i !+¼( 6 & i § m !+ ( × … & Z ˜ 8l6 x ¸ 6 +GZ-, 8N6l ³/.10
on the unit circle with a slit along the positive abscissa. Homogeneous Dirichlet boundary
conditions are given on the boundary for & x and on the upper side of the slit n ! & 6 +¼(
& Z ˜ 8N6 x ¸ 6 + Ð8 p . Homogeneous Neumann boundary conditions are used on the lower
side of the slit. The numerical approximations of the eigenvalues and eigenfunctions can be
compared with the exact solutions of (4.1). The eigenfunctions are i § m Õ32 ¾ +¼×14#576Ì!8 ¾ Š 9 & ( ; see
Fig. 4.1. Here 4:5:6 is a Bessel function of first kind of fractional order [1] and 2 ¾ … , ; .
The eigenvalues are the squares of the positive zeros 8 ¾ Š 9 of 4#576 .
The operator eigenvalue problem is discretized by linear finite elements on a triangle
mesh. Our program code AMPE (Adaptive Multigrid Preconditioned Eigensolver) in FORTRAN is an adaptive multigrid finite element code with an edge oriented error estimator
which uses linear and quadratic finite elements. All test computations have been executed on
a personal computer with an Intel Xeon 3.2GHz CPU and with a RAM of 31.4GiB. Our finite
element code on this computer can solve eigenvalue problems that exceed š<8 x 8=< degrees
of freedom. The program includes multigrid preconditioning with Jacobi smoothing. The
FORTRAN code is embedded in a Matlab GUI which allows easy and convenient usage of
the program and has graphical presentation of its output.
Experiment I. In this first experiment the block preconditioned steepest descent iteration is applied in a -dimensional subspace; we refer to this algorithm as PSD( ™75 ). Further,
the eigenvalue problem is discretized on a series of uniform triangle meshes to which nested
ETNA
Kent State University
http://etna.math.kent.edu
102
K. NEYMEYR AND M. ZHOU
F IG . 4.1. Contour lines of the three eigenfunctions of (4.1) corresponding to the three smallest eigenvalues.
iteration is applied. The first grid level comprises x nodes from which 15 are located on
the Dirichlet boundary. This gives > initial degrees of freedom. On this coarsest level the
eigenvalue problem is solved exactly (aside from rounding errors). Piecewise linear interpolation is used to prolongate the approximate eigenfunctions from one grid level to the next
refined level. The multigrid preconditioner is a V-cycle multigrid solver with damped Jacobismoothing; the damping constant is V M and two pre-smoothing and post-smoothing
iterations are applied on each level.
:
The quality of the preconditioner is controlled
? e by a stopping condition for the linear
system solver. For each Ritz vector the bound ½ ! & ( & ½ X x 8 M : is tested. Here &
e
denotes the residual of a Ritz vector, & is the preconditioned residual and is the dimension
of the discrete problem on the current level. In most cases only one V-cycle is required
to reach this accuracy since the initial solution on a refined grid is the prolongation of the
solution from the coarser grid.
™o ) is ½ & ½ å ! & å e & ( ?S X x 8 c ­ where & runs
The stopping criterion
? @ forL ! PSD(
· @C(B · @ for the Ritz vectors ·" , ·% and ·1@ . This stopping
through the residuals ·
criterion is justified by the generalized Temple inequality (see [3, Lemma 3 in Chapter 11]),
which is the first inequality in
L !'&)(%Õ L !'&2(¼ 4 @×EÕ 4 v@ u L !'&2(=×
x
{ ½ & ½ Ò { x z ½ & ½ if L !1&2(Z ˜ 4 @ 6S4 @vu ¸ >
å
4 @ 4 @vu e ? ½sÒ { z X x . Hence ½ & ½ is an upper bound for
The second inequality follows with ½
Ñ å
the product of the relative distances of L !'&)( to the enclosing eigenvalues 4 @ and 4 @vu .
x
The nested iteration is stopped on the level A¶
with š%8< B7C"8<< nodes and š%8" xD >"8
degrees of freedom. Figure 4.2 (left) shows that the computational costs increase more or
less linearly in the dimension of the problem which indicates the near optimal complexity of
¾
the PSD( ™€ ) solver. Figure 4.2 also shows the errors “ @ ¬ ¯ 4 @ , w} x 6q06S , for the three
smallest eigenvalues
4 8 ­ Š ©FE > E "<< E 6 4 8 Š © x > x C E x BE6 4 @ 8 Š © x E > "š<8 E C >
¾
The error “ ¬ ¯ 4 is relatively large since the associated eigenfunction has an unbounded
derivative at the origin. The next experiment shows that the approximation of this eigenfunction clearly profits from an adaptively generated grid.
Experiment II. Next we show that adaptive mesh generation with a posteriori edge
oriented error estimation similar to that in [15] results in much better approximations. The
ETNA
Kent State University
http://etna.math.kent.edu
103
BLOCK PRECONDITIONED STEEPEST DESCENT
computational costs [sec]
1
PSfrag replacements N K
2
10
M
JLHI K
d.o.f. G
0
10
PSfrag replacements
−1
10
d.o.f.
−3
computational costs [sec]
10
computational costs [sec]
−2
10
PSfrag replacements
10
−5
10
1
10
3
5
10
10
7
10
Level index
0
3
d.o.f.
6
9
12
Level index
O(PRQ
F IG . 4.2. PSD(
) nested iteration on uniform triangle meshes. Left: Computational costs until a level is
reached and finished (solid line); computation time on the current level (markers). Center: Error of the eigenvalue
by a line with markers,
by a broken line and
by a solid line. Right: The initial
approximations,
triangulation.
STP/
STPRU
S/PRQ
error estimator computes the eigenvector residuals with respect to quadratic finite elements
for Ritz vectors which are represented by linear finite elements. The largest (in modulus)
components of the residual are used to select those edges which belong to triangles that are
to be refined.
Once again, the PSD( ™`| ) solver is used. In order to compute a grid which allows an
optimal approximation of the eigenfunction associated with the smallest eigenvalue, the error
estimation and grid refinement aim at an optimal approximation of just this eigenfunction.
Figure 4.3 shows a relatively coarse triangulation of the unit circle and sectional enlargements
of finer triangulations around the origin, where the depth of the triangulations takes its largest
values due to the unbounded derivative ! M & (4 5:6 around & –8 . The adaptive process
generates highly non-uniform triangulations. Further details of the adaptive process and its
more or less linearly increasing costs (as a function of the d.o.f) are shown in Figure 4.4. The
©-E E "<< E
resulting smallest Ritz values “ , which approximate the smallest eigenvalue 4 ,
>
are as follows.
Depth of
triang.
1
30
43
57
68
73
Nodes
21
10709
108693
1185777 10961756 34157092
D.o.f.
“
6
10409
107630
1182184
10951337
34137627
12.95561
7.738704
7.733789
7.733379
7.733341
7.733338
In contrast to this, a uniform refinement yields a final depth 12 with 50348033 nodes only in
E > EE "%" ; a comparable quality of approximation can be gained
a poor approximation “ E7E nodes and “ } E > E >"C1B x .
by the adaptive process on level 19 with %
Finally, the results of the PSD iteration to approximate the 15 smallest eigenvalues in a
20-dimensional subspace are listed in Table 4.1.
Experiment III. Next the case of the poorest convergence of the block preconditioned
steepest descent iteration is explored. Therefore, we use the final grid from Experiment II with
about > B x x 8"› d.o.f. and apply the PSD( ™U‰ ) iteration. According to Theorem 1.1, the
poorest convergence of the vectorial iteration PSD( ™ x ) is attained in the invariant subspace
— @'Š @vu Š t . The subspace iteration behaves similarly. To show this we consider subspaces which
are spanned by a? single nonzero vector from — @'Š @vu Š t whereas all: the other basis vectors are
eigenvectors of ! 6 B ( with indexes different from w , w , x and . Then block-PSD method
behaves like a vectorial iteration due to the stationarity in the eigenvectors. Theorem 1.1
ETNA
Kent State University
http://etna.math.kent.edu
104
K. NEYMEYR AND M. ZHOU
t ë › < , in V c Š X W t ëY ; ­Z< ; , in V c ­ 7 [qŠ ­ 7 [ W PSfrag replacements
PSfrag replacements
PSfrag replacements
t ë < Y ­ @ ; , in V c ­ Š ­ W , in
, in
, in
F IG . 4.3. Triangle meshes and enlargements around the origin with 1716, 54064 and 165034 nodes (with
1628, 53387 and 163772 inner nodes). The associated depths of the triangulations are ,
and . The positive
and
belongs to the boundary.
axis
acbf
deP \ Q^]
_*`
25
50
PSfrag replacements
PSfrag replacements
PSfrag replacements N K
10
M
JHI K
G
0
10
d.o.f.
computational costs [sec]
−2
10
1
10
−1
10
computational costs [sec]
−3
Depth of triangulation
10
−5
10
1
10
3
10
5
10
7
10
d.o.f.
10
d.o.f.
Residual estimates
computational costs [sec]
1
2
0
25
50
−1
10
−3
10
−5
10
75
0
Depth of triangulation
75
Depth of triangulation
OfPgQ
F IG . 4.4. Result of PSD(
) adaptive eigensolver. Left: Computational costs: Total computation time until
# d.o.f. has been reached by solid line. Line with markers shows the computation time on the current level. Center:
Errors
for
by line with markers,
by broken line and
by solid line. Right: Estimated
residual norm for
by solid line, estimate used for the stopping criterion by broken line and actual residual norm
w.r.t. linear elements by line with markers.
hjoq
ilknm psr o SfP„
y a y {uz | yX} a y~ xh w
SfPtU
SuPvQ
provides the convergence estimate for the single vector from — @1Š @‡u Š t . Figure 4.5 shows in
the intervals ! 4 @ 6q4 @vu ( the upper bounds Õ Œ , z ! Œ ( × M Õ ! Œ ( , z Œ × (dashed lines)
and the largest ratios ˆ @'Š @vu !'“ @T (=M ˆ @1Š @‡u !1“ @ ( for 1000 equispaced normalized test vectors in
— @'Š @vu Š t whose Rayleigh quotients equal “ @ . All this is done for equidistant “ @ Zº! 4 @ 6S4 @vu ( .
In each interval ˜ 4 @ 6q4 @vu ( the estimate (3.11) is sharp, cf. Theorem 3.4, and can be attained
for “’@ _ž4 @ .
Experiment IV. In this experiment the sharp single-step estimates (3.11) are compared
æ x the
with multi-step estimates
for the PSD( ™Rí ) iteration. For each grid level A with A
Š9
x
initial subspace ¡¬‡­ ¯ , which is the prolongation of the ? final subspace Š from
9 the level A ,
!
9
t
B
1
9
(
is of sufficient quality so that the three Ritz values of
6
in ¡¬®­ ¯ have reached their
“destination interval”, i.e.,
“’@=! ¡ ¬®­ Š 9 ¯ (Zî! 4 @¬ 9 ¯ S6 4 @v¬ u9 ¯ ( 6
¾ Š9
wù x 6SN6=N6
so9 that the Ritz values “ @ ¬ ¯ ( is the iteration index on level A ) do not leave this interval. Here,
! ? 9 6 Bt91( with respect to the grid level A . Theorem 3.4 guarantees
4 @¬ ¯ are the eigenvalues
of
Š
9
9
¾
that ‡§ Ú ¾*€‚ “ @ ¬ ¯ A4 @ ¬ ¯ .
ETNA
Kent State University
http://etna.math.kent.edu
105
BLOCK PRECONDITIONED STEEPEST DESCENT
ƒ „xz … k of (4.1). Left: Ritz values in a _x\q^Q_ -dimensional linear finite element
ªª^U*\Qª -dimensional linear finite element space. Right: “Exact” eigenvalues
TABLE 4.1
The 15 smallest eigenvalues
space. Center: Ritz values in a
of (4.1).
†
k l
0
1
2
3
4
5
6
7
8
9
1
7.733389
12.18725
17.35102
23.19983
29.71530
36.88311
44.69164
53.13167
62.19503
71.87493
2
34.88339
44.25893
54.36164
65.17971
76.70204
†
k l
0
1
2
3
4
5
6
7
8
9
1
7.733342
12.18715
17.35080
23.19943
29.71460
36.88199
44.69009
53.12939
62.19189
71.87073
2
34.88260
44.25768
54.35978
65.17704
76.69829
†
k l
0
1
2
3
4
5
6
7
8
9
1
7.733337
12.18714
17.35078
23.19939
29.71453
36.88189
44.68994
53.12918
62.19159
71.87033
2
34.88252
44.25756
54.35960
65.17677
76.69790
1
0.8
‡ o … o‰ˆ ^w Š ‹xoŒ 
‡ o … o‰ˆ w^Š ‹ o 
0.6
0.4
PSfrag replacements
0.2
0
7.733
12.187
17.351
“’@
23.199
Ž r o  r o"ˆ w
ho
29.715
F IG . 4.5. Poorest convergence of block preconditioned steepest descent iteration. Abscissa: Five smallest
are the upper bounds
eigenvalues according to Table 4.1. The dashed lines in the intervals
for
. The curves are the largest ratios
over
1000 equispaced test vectors in
whose Rayleigh quotients equal .
z|
’  — ‘ Ž˜U p ’  “™•’ —
z
•šPœ› o … SJo‰ˆ  ž… Q  ž ];K
wÊ
‘‰’”“–•=Ž‰U p
Ÿ o … o‰ˆ w Ž‰h oŒ  | Ÿ o … o"ˆ w Ž˜h o 
All this allows us to apply the x -step estimates
(4.2)
“ @ ¬ ¾ u Š 9 ¯ 4 @¬ 9 ¯ { ‹ Œ ¬ 9 ¯ , z ! Œ ¬ 9 ¯ ( “ @ ¬ ¾ Š 9 ¯ 4 @¬ 9 ¯ …¡ !1“ ¾ u Š 9 (
¯ (6 Ÿo8l6 x 6 >;>s> 6
@¬
9
u Š9
! Œ ¬ 9 ¯ -( , z Œ ¬ 9 ¯ Ž 4 @v¬ u9 ¯ î
“ @¬ ¾ Š 9 ¯ 4 @‡¬ u ¯ “ @ ¬ ¾ ¯
recursively, which yields the multistep estimate
“ @ ¬ ¾ Š 9 ¯ 4 @¬ 9 ¯ { ‹ Œ ¬ 9 ¯ , z ! Œ ¬ 9 ¯ ( ¾ “ @ ¬®­ Š 9 ¯ 4 @¬ 9 ¯ …£¢ !'“ ¾ Š 9 (
@ ¬ ¯ 6¤} x 6SN6 >s>;> 6
9
Š9
(4.3)
! ¬ 9 ¯ (d, z ¬ 9 ¯ Ž 4 @v¬ u9 ¯ î“ @ ¬®­ Š 9 ¯ ¬4 @vu ¯ î“ @ ¬ ¾ ¯
Œ
Œ
9
@ by 4 @¬ 9 ¯ for the relevant indexes w . The
4
where Œ ¬ ¯ is given bye (3.11)
after
substitution
of
?
½ Ò for the quality of the preconditioner
parameter z –½;Ñ is approximated on each
e ?
grid level by computing the spectral radius of Ñ with the power method. For this
experiment we use again two steps of pre-/post-smoothing on each level with9 damped (V M ) Jacobi-smoothing. This results in z © 8 > E C . The discrete eigenvalues 4 @¬ ¯ are estimated
by extrapolation from the computed Ritz values.
ETNA
Kent State University
http://etna.math.kent.edu
106
K. NEYMEYR AND M. ZHOU
Figure 4.6 shows the multistep bound (4.3) as a bold line, the x -step bound as a dotted
line and the numerical result as a thin solid line. The x -step estimate (4.2) is a very good
upper estimate. In all cases the multistep estimate (4.3) is a relatively rough estimate. It
accumulates the over-estimation
of the error from step to step and suffers from its inability to
¾ Š9
use the current “ @ ¬ ¯ on the right-hand side of the estimate in order to improve the quality of
the upper bound.
9
The ˆ !1“"( -ratio depends on the discrete eigenvalues 4d¬ ¯ and decreases monotonically
for the iteration on each grid level; the ratio may increase after prolongation to a refined grid
level. The somewhat oscillating behavior of the ˆ !1“"( -ratio for 4) in contrast to the smoother
behavior for 4) reflects the fact that the error estimation and grid generation is controlled by
error estimates for the first eigenfunction. The second eigenfunction also profits from the grid
refinement (cf. Figure 4.4) but the ˆ !1“"( -ratio shows a stronger variation for changing level
index A .
5. Conclusion. This paper concludes our efforts of analyzing preconditioned gradient
iterations and their subspace variants with either fixed step length (case of inverse iteration
and preconditioned inverse iteration) or with optimal step-length (case of steepest descent and
preconditioned steepest descent). Within the hierarchy of preconditioned gradient iterations,
as suggested in [13], these solvers are denoted as INVIT( , ™ ) and PINVIT( , ™ ) with µ x 6S
and subspace dimensions ™ Zg¥ .
For all these iterative eigensolvers sharp convergence estimates have been derived which
have the common form
L T
¡ L
ˆ @1Š @vu ! !1& ((}{ ˆ @1Š @vu ! !1&2(=(
8<( and convergence factors ¡
with ˆ @1Š @vu ! 8<( ! 8U 4 @ (=MN! 4 @‡u ¦
convergence estimates have been gained:
Iterative Eigensolver
Convergence factor
Inverse iteration
¡ Preconditioned inverse iteration
Block preconditioned inverse iteration
¡ z ,A! x z ( §¿"!#¿
§
Steepest descent
Precond. steepest descent
Block precond. steepest descent
. The following sharp
§¿
§ "¿ !#
¡ ¨ 6
c ¨ Œ §§
¡ ¨ uuª ¬ u«c ª ¨ ¯ 6
¬ c ¨¯ ¨ Œ
Ref.
[16]
¿ ¬ §ž© c § ¿‰!: ¯
¿‰!: ¬ §ž© c § ¿ ¯
§ ¿¿‰!:¬ § © c § c ¿"!# ¿ ¯
§ ¬§ © § ¯
[10]
[14]
[18]
[17]
here
Scientific efforts for the future are aimed at a convergence analysis of the important
locally optimal preconditioned conjugate gradient (LOPCG) iteration [9]. As the convergence
behavior of the LOPCG eigensolver has been observed to behave similarly to the conjugate
gradient iteration for linear systems, sharp convergence estimates are highly desired.
Acknowledgement. The authors would like to express their gratitude to the anonymous
referee for his very positive feedback and constructive remarks!
REFERENCES
[1] G. A RFKEN , Mathematical Methods for Physicists, Academic Press, San Diego, 1985.
ETNA
Kent State University
http://etna.math.kent.edu
107
BLOCK PRECONDITIONED STEEPEST DESCENT
Approximation of ±
Ã
1
²”³˜´µ
¶ ³‰´Zµ
· ³‰´Zµ
10
−2
10
−5
10
PSfrag replacements
−8
10
0
50
100
Convergence history on levels ¬®­x¯¯x¯*­¬=°
Approximation of ±T¸
1
²”³˜´µ
¶ ³‰´Zµ
· ³‰´Zµ
10
−2
10
−5
10
PSfrag replacements
−8
10
STPG  U
0
50
Convergence history on levels ¬®­x¯¯x¯*­¬=°
100
„x
„x
„x
Ÿ oinkn… o‰m ˆ w Ž‰h o i knm  P-Ž˜h o i knm p¹r oinknm  | Ž r o‰inˆknm w p h o i knm 
F IG . 4.6. Convergence history of the error ratios
for
and PSD( ). The -step estimate (4.2) (broken line) is good estimate for the numerical worst-case results
(thin solid line); the multistep estimate (4.3) (bold line) accumulates the over-estimation from step to step.
Q
ETNA
Kent State University
http://etna.math.kent.edu
108
K. NEYMEYR AND M. ZHOU
[2] Z. B AI , J. D EMMEL , J. D ONGARRA , A. R UHE , AND H. VAN DER V ORST (eds.), Templates for the Solution
of Algebraic Aigenvalue Problems: A Practical Guide, SIAM, Philadelphia, 2000.
[3] E. G. D’ YAKONOV , Optimization in Solving Elliptic Problems, CRC Press, Boca Raton, 1996.
[4] G. H. G OLUB AND C. F. VAN L OAN , Matrix Computations, 3rd ed., Johns Hopkins University Press, Baltimore, 1996.
[5] M. R. H ESTENES AND W. K ARUSH , A method of gradients for the calculation of the characteristic roots and
vectors of a real symmetric matrix, J. Research Nat. Bur. Standards, 47 (1951), pp. 45–61.
[6] L. V. K ANTOROVICH , Functional Analysis and Applied Mathematics, National Bureau of Standards, Los
Angeles, 1952.
[7] L. V. K ANTOROVICH AND G. P. A KILOV , Functional Analysis in Normed Spaces, Pergamon, Oxford, 1964.
[8] A. V. K NYAZEV , Convergence rate estimates for iterative methods for a mesh symmetric eigenvalue problem,
Soviet J. Numer. Anal. Math. Modelling, 2 (1987), pp. 371–396.
, Preconditioned eigensolvers—an oxymoron?, Electron. Trans. Numer. Anal., 7 (1998), pp. 104–123.
[9]
http://etna.math.kent.edu/vol.7.1998/pp104-123.dir
[10] A. V. K NYAZEV AND K. N EYMEYR , A geometric theory for preconditioned inverse iteration. III: A short
and sharp convergence estimate for generalized eigenvalue problems, Linear Algebra Appl., 358 (2003),
pp. 95–114.
[11]
, Gradient flow approach to geometric convergence analysis of preconditioned eigensolvers, SIAM J.
Matrix Anal. Appl., 31 (2009), pp. 621–628.
[12] A. V. K NYAZEV AND A. L. S KOROKHODOV , On exact estimates of the convergence rate of the steepest
ascent method in the symmetric eigenvalue problem, Linear Algebra Appl., 154/156 (1991), pp. 245–
257.
[13] K. N EYMEYR , A Hierarchy of Preconditioned Eigensolvers for Elliptic Differential Operators, Habilitation
Thesis, Mathematische Fakultät, Universität Tübingen, 2001.
, A geometric theory for preconditioned inverse iteration applied to a subspace, Math. Comp., 71
[14]
(2002), pp. 197–216.
[15]
, A posteriori error estimation for elliptic eigenproblems, Numer. Linear Algebra Appl., 9 (2002),
pp. 263–279.
, A note on inverse iteration, Numer. Linear Algebra Appl., 12 (2005), pp. 1–8.
[16]
[17]
, A geometric convergence theory for the preconditioned steepest descent iteration, SIAM J. Numer.
Anal., 50 (2012), pp. 3188–3207.
[18] K. N EYMEYR , E. OVTCHINNIKOV, AND M. Z HOU , Convergence analysis of gradient iterations for the
symmetric eigenvalue problem, SIAM J. Matrix Anal. Appl., 32 (2011), pp. 443–456.
[19] K. N EYMEYR AND M. Z HOU , Iterative minimization of the Rayleigh quotient by the block steepest descent
iteration, Numer. Linear Algebra Appl., in press, doi:10.1002/nla.1915.
[20] E. OVTCHINNIKOV , Convergence estimates for the generalized Davidson method for symmetric eigenvalue
problems II: The subspace acceleration, SIAM J. Numer. Anal., 41 (2003), pp. 272–286.
[21] B. N. PARLETT , The Symmetric Eigenvalue Problem, Prentice Hall, Englewood Cliffs, 1980.
[22] J. P ONSTEIN , An extension of the min-max theorem, SIAM Rev., 7 (1965), pp. 181–188.
[23] V. G. P RIKAZCHIKOV , Strict estimates of the rate of convergence of an iterative method of computing eigenvalues, U.S.S.R. Comput. Math. Math. Phys., 15 (1975), pp. 235–239.
[24] Y. S AAD , Numerical Methods for Large Eigenvalue Problems, Manchester University Press, Manchester,
1992.
[25] B. A. S AMOKIS , The steepest descent method for an eigenvalue problem with semi-bounded operators,
Izv. Vyssh. Uchebn. Zaved. Mat., 5 (1958), pp. 105–114.
[26] M. S ION , On general minimax theorems, Pacific J. Math., 8 (1958), pp. 171–176.
[27] A. S TATHOPOULOS , Nearly optimal preconditioned methods for Hermitian eigenproblems under limited
memory. I. Seeking one eigenvalue, SIAM J. Sci. Comput., 29 (2007), pp. 481–514.
[28] P. F. Z HUK AND L. N. B ONDARENKO , Sharp estimates for the rate of convergence of the -step method of
steepest descent in eigenvalue problems, Ukraı̈n. Mat. Zh., 49 (1998), pp. 1694–1699.
O