ETNA Electronic Transactions on Numerical Analysis. Volume 41, pp. 93-108, 2014. Copyright 2014, Kent State University. ISSN 1068-9613. Kent State University http://etna.math.kent.edu THE BLOCK PRECONDITIONED STEEPEST DESCENT ITERATION FOR ELLIPTIC OPERATOR EIGENVALUE PROBLEMS KLAUS NEYMEYR AND MING ZHOU Abstract. The block preconditioned steepest descent iteration is an iterative eigensolver for subspace eigenvalue and eigenvector computations. An important area of application of the method is the approximate solution of mesh eigenproblems for self-adjoint elliptic partial differential operators. The subspace iteration allows to compute some of the smallest eigenvalues together with the associated invariant subspaces simultaneously. The building blocks of the iteration are the computation of the preconditioned residual subspace for the current iteration subspace and the application of the Rayleigh-Ritz method in order to extract an improved subspace iterate. The convergence analysis of this iteration provides new sharp estimates for the Ritz values. It is based on the analysis of the vectorial preconditioned steepest descent iteration which appeared in [SIAM J. Numer. Anal., 50 (2012), pp. 3188–3207]. Numerical experiments using a finite element discretization of the Laplacian with up degrees of freedom and with multigrid preconditioning demonstrate the near-optimal complexity of the to method. Key words. subspace iteration, steepest descent/ascent, Rayleigh-Ritz procedure, elliptic eigenvalue problem, multigrid, preconditioning AMS subject classifications. 65N12, 65N22, 65N25, 65N30 1. Introduction. Let be a bounded polygonal region in , often or , and let the boundary be the disjoint union of and . We consider the self-adjoint elliptic eigenvalue problem (1.1) "!$#%!'&)(+*(-,/.0!1&2(3* 5 4 * * 9 : 8 ;#<!'&2(=+* 98 in 76 on 6 %> on Here #<!'&2( is a: symmetric and positive definite matrix valued function, .0!1&2( a nonnegative real function and denotes the normal vector on the boundary . The coefficient functions are assumed piecewise continuous. We are interested in the numerical approximation of some of the smallest eigenvalues 4 and the corresponding eigenfunctions * . The finite element discretization of (1.1) yields the generalized matrix eigenvalue problem ? 2& @ 4 @CBD&E@ > A :GFH: ? B The discretization matrix and the mass matrix are symmetric, positive definite (1.2) matrices. Typically, these matrices are very large and sparse. Our aim is to compute a fixed number of the smallest eigenvalues of (1.2) together with : the associated eigenvectors. The numerical algorithm should exploit the structure of the mesh eigenproblem, and its computational costs should increase almost linearly in dimension . The demand for a near-optimal-complexity method rules out all eigensolvers which are usually used for small and dense matrices. See [2, 4, 21] for getting an overview on the wide variety of numerical eigensolvers primarily for small and dense matrices. I Received January 23, 2014. Accepted February 24, 2014. Published online on May 23, 2014. Recommended by M. Hochstenbach. Universität Rostock, Institut für Mathematik, Ulmenstraße 69, 18055 Rostock, Germany ( klaus.neymeyr, ming.zhou @uni-rostock.de). J K 93 ETNA Kent State University http://etna.math.kent.edu 94 K. NEYMEYR AND M. ZHOU A conceptually easy approach to the desired near-optimal-complexity eigensolvers is based on gradient iterations for the Rayleigh quotient; cf. Dyakonov’s monograph on optimization for elliptic problems [3] and its Chapter 9 on the Rayleigh-Ritz method for spectral problems. The starting point is the generalized Rayleigh quotient for (1.2) (1.3) L !1&2( '! & 6 ? &)(=MN!'& 6 BD&2( 6 &P5 O 8 > As L !3Q( attains its minimum 4 at an associated eigenvector, the minimization of (1.3) can be implemented by means of a gradient iteration. The negative gradient of (1.3) reads L !'&)( ! ? R & L !'&)(BD&2(SM0!1& 6 BD&)( and allows us to construct a new iterate &)T &UHV L !'&2( with L !1&ETW(YX L !1&2( , whenever & is not an eigenvector and V5Z is a proper step length. The optimal step length V minimizes L !'&ETW( with respect to V[Z . These optimal &)T and L !'&2TW( can be computed by applying the Rayleigh-Ritz procedure to the 2D space spanned by & and L !'&2( . The gradient iteration ? does not change and B and does not need these matrices explicitly. It is a so-called ? matrix-free method in the? sense that its implementation only requires routines \^]_ \ and \`]_ B \ . The sparsity of and B allows us to implement these matrix-vector products with computational cost which only increases linearly in the number of unknowns. 1.1. Preconditioning of gradient iterations. Gradient iterations for the Rayleigh quotient which use the? Euclidean gradient L !'&)( are well known to converge poorly if the condition number of is large [5, 6, 7, 8, 12, 23, 28]. This is particularly the case for a finite ? element discretization of (1.1) since the condition number of increases as a !$bdc ( in the b discretization parameter . The key ingredient to make a gradient iteration an efficient solver for the operator eigenvalue problem (1.1) is multigrid If a symmetric and positive definite matrix ? preconditioning. ? e e is an approximate inverse of , then is called a preconditioner for . The preconditioned e gradient iteration uses L !1&2( as the descent direction for the Rayleigh quotient. The use age of the -gradient can be interpreted as a change of the underlying geometry which makes ellipsoidal level sets of L !3Q( more spherical [3, 25]. Proper preconditioning, for instance by multigrid or multilevel preconditioning, can accelerate the convergence considerably. In the best case this can result in grid-independent convergence behavior [10, 11]. e The -gradient steepest descent iteration optimizes the step length V & T &fV e L !1&2( &U V e ! ? &R L !'&)(BD&2(SM0!1& 6 BD&)( e ? in a way that & T attains the smallest possible Rayleigh quotient for all VgZ . If ! &h L !'&2(=BD&2( is a Ritz vector, then V may be infinite. Computationally &T is determined by the ? Rayleigh-Ritz procedure since &)T is a Ritz vector of ! 6 B ( in the two-dimensional subspace ? e i=jlk%mn & 6 ! &o L !1&2(BD&)(qp corresponding to the smaller Ritz value. The resulting itera(1.4) tion has some similarities with the Davidson method [20, 24] if the iteration is restarted after each step so that the dimension of the truncated search subspace always equals 2. Stathopoulos [27] calls such an iteration with constant memory requirement a generalized Davidson method. The eigenpairs of (1.2) are denoted by ! 4 @ 6 &E@r( . The enumeration is 8 X 4) X 4l X X >;>s> 4lt . We assume simple eigenvalues. The case of multiple eigenvalues can simply be reduced to that of only simple eigenvalues by a proper projection of the eigenvalue problem. Alternatively, a continuity argument can be used to show that colliding eigenvalues do not change the structure of the convergence estimates; cf. [17, Lemma 2.1]. ETNA Kent State University http://etna.math.kent.edu BLOCK PRECONDITIONED STEEPEST DESCENT 95 For the vectorial Preconditioned Steepest Descent (PSD) iteration (1.4) sharp convergence estimates have recently been proved in [17]. The central theorem reads as follows: T HEOREM 1.1. Let &Z : t be such that the Rayleigh quotient L !1&2( satisfies 4 @X L !1&2( X 4 @vu for some w Z nyx 6 >;>;> 6 x p . Let & T be the Ritz vector of ! ? 6 B ( in i=jlk%m)n & 6 e ! ? &+ L !1&2(=BD&2(qp which corresponds to the smaller Ritz value. The Ritz value is L !'&TW( . The preconditioner satisfies z '! & 6 e c &2(Y{|!'& 6 ? &2(}{ z !'& 6 e c &2(~&oZ t with constants z 6 z 8 . Let zH ! z z (SM0! z , z ( . Then either L !'&2TW(Y{ 4 @ or L !1&ETW(Z! 4 @ 6S4 @u ( is bounded from above as follows , z ! ( @1 @vu ! L !1&ETW(( 4 @ ! 4 t 4 @vu ( (1.6) with 8 X @' @vu ! L !'&2(=( { ! ( , z 4 @vu ! 4Et 4 @( > Here !1"( !1 4 (=MN! 4 Y"( . The estimate (1.6) cannot be improved in terms of the eigenvalues 4 @ , 4 @u and 4 t . The bound can be attained for 4_4 @ in the three-dimensional invariant subspace @' @vu t , which is spanned by the eigenvectors corresponding to 4 @ , 4 @vu and 4 t . Theorem 1.1 guarantees monotone convergence of the Ritz values L !1&TW( towards an eigenvalue. If the Rayleigh quotient L !'&)( has reached the final interval 4-6q4l ( , then (1.6) proves linear convergence of the Ritz values towards the smallest eigenvalue 4d . (1.5) 1.2. Aim of this paper. This paper aims at generalizing the preconditioned gradient iteration (1.4) to a subspace iteration for the simultaneous computation of the ? smallest eigenvalues of (1.2). New and sharp convergence estimates for the Ritz values of ! 6 B ( in the iteration subspaces are presented, which generalize Theorem 1.1. The effectiveness of the method is demonstrated for the Laplacian eigenvalue problem on the sliced unit circle with mixed homogeneous Dirichlet and Neumann boundary conditions. The operator eigenvalue problem is discretized with linear and quadratic (only for error estimation) finite elements. Our finite element code AMPE, see Section 4, is demonstrated to work with up to x 8" degrees of freedom. AMPE uses a multigrid preconditioner with Jacobi smoothing. 1.3. Notation and overview. Subspaces are denoted by capital calligraphic letters, e.g., the column space of a matrix is ijlk<mn p . Similarly, ijlk<m)n 6= p is the smallest vector space which contains the column spaces of and of . All eigenvalues and Ritz values are enumerated in order of increasing magnitude. Further, ? index-set denotes the invari? ant subspace of ! 6 B ( which is spanned by the eigenvectors of ! 6 B ( whose indexes are contained in the index set. This paper is structured as follows: The preconditioned gradient subspace iteration is introduced in Section 2. The new convergence estimates are presented in Section 3; Theorem 3.4 is the central new estimate on the convergence of the Ritz values. In Section 4 numerical experiments illustrate the sharpness of the estimates and the cluster-robustness of the preconditioned gradient subspace iteration. 2. The block preconditioned iteration with Rayleigh-Ritz projections. The reconditioned subspace/block iteration is a straightforward generalization of vector iteration (1.4). The iteration (1.4) is just applied to each column of the matrix which columnwise contains the Ritz vectors of the current subspace. Each subspace correction step is followed by the Rayleigh-Ritz procedure in order to extract the Ritz approximations. ETNA Kent State University http://etna.math.kent.edu 96 K. NEYMEYR AND M. ZHOU t which is given by the column The starting point is an -dimensional subspace ¡ of Z tE£l¤ . The columns of ¢ are assumedF to be the B -normalized Ritz vectors of ¢ space of ! ? 6 B ( in ¡ . Further, ¥AA¦0§ k%¨ !' %6 >s>;> 6 ¤ ( is the diagonal matrix of the corresponding Ritz values. The matrix residual ? ¢ B ¢¥ Z tE£N¤ contains e columnwise with the ? precon© ? c the residuals of the Ritz vectors. Left multiplication coincides with the approximate solution of linear systems in . This ditioner e ? yields the correction-subspace i=jlk<mn ! ¢ 5B ¢+¥ (ªp of the preconditioned subspace iteration. The Rayleigh-Ritz? procedure is usede to? extract the smallest Ritz values and the associated Ritz vectors of ! 6 B ( in ijEk%m)n ¢6 ! ¢ PB ¢+¥ (qp ; see Algorithm 1. Algorithm 1 Block preconditioned steepest descent iteration. ? e Require: Symmetric and positive definite matrices 6 B«Z tE£Nt , a preconditioner satisfying (1.5) and an initial (random) -dimensional subspace ¡7¬®­S¯ with °± ! ¡¬®­q¯q6l=² ¤ (X ³ M where E=² ¤ is the invariant subspace of ! ? 6 B ( associated with the smallest eigen:´F values. 1. Initialization: Apply the Rayleigh-Ritz procedure to ¡7¬­S¯ . The matrix ¬­S¯ ¸ contains the Ritz vectors of ! ? 6 B ( corresponding to the ¢µ¬®­q¯¶[ · ¬­S¯ 6 >;>;> · ¤| ? ¢µ¬®­q¯ Ritz values ¬®­q¯ 6 >s>;> 6 ¤ ¬®­q¯ . Let ¥`¬®­q¯Y ¦0§ k<¨ !1 ¬®­q¯ 6 >s>;> 6 ¤ ¬®­q¯ ( and ¹+¬®­q¯Y B ¢ ¬®­q¯ ¥ ¬®­q¯ . i=jlk%mn ¢f¬ @ ¯ 6 e ¹+¬ @ ¯ p . 2. Iteration, w|8 : The Rayleigh-Ritz @u procedure @vu is applied to @vu ? ¯º» · ¬ ¯ 6 >s>;> · ¤ ¬ ¯ ¸ @are The columns of ¢R¬ the Ritz@u vectors of ! 6 B ( u ¬ ¯ 6 >;>s> 6 ¤ ¬ ¯ . Let ¥`¬ @vu ¯ corresponding to the smallest@vu Ritz values v @ u v @ u @ u @vu @u ? ¦0§ k<¨ !1 ¬ ¯ 6 >s>;> 6 ¤ ¬ ¯ ( and ¹+¬ ¯¼ ¢R¬ @u ¯ PB ¢µ¬ ¯¥`¬ ¯ . ¯ ½ drops below a specified accu3. Termination: The iteration is stopped if ½ ¹¬ racy. The block steepest ? c iteration as given in Algorithm 1 has already been analyzed e descent for the ? special case (preconditioning with the exact inverse of the discretization matrix ) in [19]. If in the central convergence estimates of this paper, see Corollary 3.3 and Theorem 3.4, z A8 is inserted, then the convergence factor z ! @ , @r! ( , z @ ( @ with @ 42¾q¿ u ! 4 t ! 42 ¾q¿ u (( 4 ¾ ¿ 4Et 4 ¾ ¿ simplifies to @ M0! @ ( . This bound has been proved in case 1b of [19, Theorem 1.1]. However, the convergence analysis for the block steepest descent iteration with a general preconditioner does not rest upon the analysis in [19]. In fact, the analysis of the current paper uses the convergence analysis for the vectorial preconditioned steepest descent iteration from [17]. By starting with the result from [17] some of the proof techniques from [19] can be adapted to the block iteration with general preconditioning. One important modification is that the argument using the subspace dimensions from Theorem 3.2 of [19] cannot be transferred to the case of general preconditioning. Instead, we use here Sion’s theorem for the proof of Theorem 3.2. The proof structure of the present paper has also some similarities with the chain of arguments in [14] wherein a convergence analysis of the subspace variant of the preconditioned inverse iteration is contained. References to similar results and similar proof techniques are given prior to Lemma 3.1, Theorem 3.2 and Theorem 3.4. ETNA Kent State University http://etna.math.kent.edu 97 À`Å5ÆÀ BLOCK PRECONDITIONED STEEPEST DESCENT ÁÂÃ Ä PSfrag replacements F IG . 3.1. È ÁÇ ÂÃsÄ ÀÅ À -orthogonal decomposition in the É-ÊÌË<Í . 3. Convergence analysis. The convergence estimates on the convergence of the Ritz Î ¬ @vu ¯ with Ï5 x 6 >s>;> 6q and whÐ8 , see Algorithm 1, towards the eigenvalues of values ! ? 6 B ( are proved in this section. In Theorem 3.2 together with Corollary 3.3 the conver@u ¯ , is analyzed. It is shown that the gence of the largest of these Ritz values,? namely ̤T ¤ ¬ ? e Ritz value "¤T of ! 6 B ( in ijEk%m)n ¢6 ! ¢ /B ¢`¥ (ªp is bounded from above by the largest Ritz value which can be attained by the vector iteration PSD (1.4) if applied elementwise to the column space of ¢ . Theorem 3.4 proves sharp estimates for the remaining Ritz values by induction. following analysis we assume that the preconditioner satisfies the inequality ½;Ñ e ? ½;InÒ the { z X x . This assumption is equivalent to (1.5) with z9 ! z z (SM0! z , z ( if the preconditioner is optimally scaled. The PSD iteration implicitly determines this optimal scaling by the Rayleigh-Ritz procedure. The next lemma generalizes Lemma 3.1 from [14]. tE£N¤ form a Ritz basis of i=jlk%mn ¢ p , and let e be L EMMA 3.1. Let the columns of ¢ Z e ? ½sÒ { z X x . a symmetric and positive definite matrix with ½ Ñ ¤Ô n 8 p and any #7Z ¤ it holds that i) For any Ó Z ? ? ½ c B ¢`¥7Ó ` ¢ Ó½ Ò { ½ c B ¢¥7Ó ¢ # ½ Ò> e ? B The matrix V ¢ ¥ ,ÖÕ ¢ ! ¢ G ¢`¥ (=× preserves for all VAZ of the matrix ¢ . Ø Z t satisfy Let & be given as in Theorem 1.1, and let &h ? ? (3.2) ½ c BD&R &Ø ½;Ò { z ½ c BD&R&M L !'&)( ½ Ò > (3.1) ii) iii) the full rank If (3.3) & T Z k<Ù=¨ ÛÜ%Ý'ÞsÚ ß=ৠm á=â âã ä L ! · ( 6 then the PSD estimate (1.6) applies to the Ritz vector Rayleigh quotient L !1&2TW( . ? Proof. i) The -orthogonality, see Fig. 3.1, &2T given by (3.3) and its ! ¢\E6 ! ? c B ¢`¥ ¢ ( Ó ( Ò 9\yåd¢`å !$B ¢¥ ? ¢ ( Óµ5\yå ! ¥ ¥ ( Óµ98 ~ \ Z ¤ ? ? ¢ ¥7Ó on ijlk<mn ¢ p . This guarantees shows that ¢Ó is the -orthogonal projection of c B ` that ? ? ½ c B ¢¥7Ó ¢Ó½ Ò { ½ c B ¢`¥7Ó ¢ # ½ Ò ETNA Kent State University http://etna.math.kent.edu 98 K. NEYMEYR AND M. ZHOU ¤ for all #7Z . ii) Direct computation shows that for Ó Z ¤Ô n 8 p ? ½ V ¢`¥7Ó ,A! ¢ e ! ¢ PB ¢`¥ (( Ó½ Ò ? ? ? +½ V ¢`¥7Ó , c B ¢+¥7Ó ,A! Ñ e ( ! ¢ c B ¢+¥ ( Ó½ Ò ? ? ? ½ V ¢`¥7Ó , c B ¢+¥7Ó)½ Ò ½ ! Ñ e ( ! ¢ c B ¢`¥ ( Ó½ Ò æ ½ ? c B ¢+¥7Ó ¢ !V ¥7Ó ( ½ Ò ½ ! ¢ ? c B ¢+¥ ( Ó)½ Ò º8 > ç è é ê ë ²ì e !? ¢ The last inequality holds with # V ¥7Ó due to (3.1). The fact ½ V ¢`e ¥7Ó ? ,D! ¢ B ¢`¥ (=( Ó½ Ò æ 8 for all Ó í O 8 proves that the matrix V ¢`¥ ,|! ¢ ! ¢ B ¢`¥ (( has the full rank. ? ? iii) Inequality (3.2) is equivalent to ½ L !'&2( c BD& L !'&2( &Ø ½ Ò { z ? ½ L !'&)( c BD&î & ½ Ò . This means that L !1&2( &Ø is contained in a ball with the center L !'&2( c BD& and the ? radius z ½ L !1&2( c BD&}& ½ Ò . The geometry of the fixed-step-length preconditioned gradient iteration (see [10, 17]) shows that L !'&)( &Ø is representable by a symmetric and positive definite e Ø with ½ Ñ e Ø ? ½;Ò { z in the form L !'&)( &Ø &f e Ø ! ? & L !'&)(BD&)( > The preconditioned steepest descent iteration accelerates the convergence of this fixed-steplength iteration by applying the Rayleigh-Ritz procedure to ijEk%mn & 6 e Ø ! ? f & L !'&2(=BD&2(ªp i=jlk%mn & 6S&Ø p > Thus the smallest Ritz value L !1&2TW( in this space is bounded from above by the estimate (1.6). The next theorem proves the desired convergence behavior for the largest Ritz value Comparable estimates for the largest Ritz values, but with respect to different spaces, have been used in [14, Theorem 3.2] for the subspace analysis of the preconditioned inverse iteration and in [19, Theorem 3.2] for the block steepest descent ? iteration. T HEOREM 3.2. Let ijEk%m)n ¢ p contain no eigenvector of , which otherwise could ? easily be split off within ? the Rayleigh-Ritz procedure. Then the -th Ritz value ̤T of ! 6 B ( in ï ijlk<m)n ¢6 e ! ¢ PB ¢+¥ (qp , which is given by <¤T . ¤T (3.4) satisfies (3.5) e ? kö Ú m L y þ ¤T { úÜÚR ûü ø á ­ ä ýEܧ û Õ V ¢¥7Ó , Õ ¢ ! ¢ PB ¢+¥ ( × Ó × ¤ > Proof. Let Ó attained (3.6) F ! Hence Úµðùk%ø ö L ! \ ( 6 ð¶Ú ñ2§ ò m ó;ô õ+ð ë ¤ ÷sÜ á ­ ä Z ¤Ô n 8 p and V Z be given in a way so that the max-min in (3.5) is þ ¤ L Õ V ç ¢èªé¥ ê Ó , Õ ¢ e ! ? ¢ PB ¢+¥ ( × Ó × > ç ªè é ê ë ² ÿ ë ² ÿ Ó 6 V ( is a saddle point of the Rayleigh quotient. In a sufficiently small neighborhood of ! Ó 6 V ( the Rayleigh quotient L ! ( is a smooth function which is concave in Ó and ETNA Kent State University http://etna.math.kent.edu BLOCK PRECONDITIONED STEEPEST DESCENT 99 convex in V . Sion’s-minimax theorem [22, 26] provides the justification to swap the order of minimization and maximization, i.e., þ ¤ L !WV Ó , Ó ( Úµú Ük% ö Úý2ܧ m L !'V Ó , Ó ( ÚýEܧ m ÚRúÜk ö L !WV Ó , Ó ( > ( is the Rayleigh quotient of Ó with respect to the projected matrix-pair L Thus ! ? !' V 6 ªÓ ,B ;( Ó with V , . The local maximum in Ó Z coincides with å å (because the maximum is taken in the the global maximum of L ! ( in the column space of interior of and because the Hessian of the Rayleigh quotient shows that there is only one local extremum which is a maximum). We conclude that Ú §m R Ú kö L Ú m ý2Ü ú Ü !'V Ó , Ó ( Eý ܧ õ ß !WV , ( > (. We denote the minimum point with respect to VDZ by V so that þ ¤ õ ß ! V , ? Therefore þ ¤ is the largest Ritz value of ! 6 B ( in the column space of V , Z ï tE£N¤ . one Lemma 3.1 guarantees that the dimension of equals . Since is a subspace of gets with (3.4) that "¤T { ÚRk%ö ÷sÜ ðdø á ä L ! \ ( þ ¤ . ? ­ : C OROLLARY 3.3. If the -th Ritz value ¤ of ! 6 ? B ( in ¡ is contained? in ! 4 6S4 3u ( , .RZ nyx 6 >s>;> 6 x p , then for the -th Ritz value "¤T of ! 6 B ( in i=jlk%m)n ¢¶6 e ! ¢ /B ¢+¥ (qp it holds that either "¤T { 4 or , z ! ( ª 3u !'<T ( 4 ! 4 t 4 u ( (3.7) with 8 Xg ª 3u !' ¤¤ ( { ! (, z 4 3u ! 4lt 4 s( > : ¶ x , then the first alternative "¤T { 4Et c applies since the -th Ritz value If . ? Proof. e ? ! B ( in the at least ! , x ( -dimensional subspace ijEk%m)n ¢6 ! ¢ oB ¢¥ (qp is smaller of 6 than or equal to 4E: t c (due to the min-max principles). x and assume 4 7X ¤T for which (3.7) is to be proved. For 4 7{µX 4 Next let .+X ' ! " q the function ( !' 4 (SM0! 4 "( is monotone increasing in . Theorem 3.2 shows that <¤T { þ ¤ so that þ T (3.8) ª 3u !' ¤ (}{ q u ! ¤ ( > e ! ? ¢ ^B ¢`¥ (=× Ó In order to apply Lemma 3.1, Case iii, to & ´¢+¥7Ó and &Ø Õ ¢ we show that ? ½ c BD&f &Ø ½ Ò í½ í½ { z { z ? c !Ñ ? ½ c ? ½ c B + ¢ ¥7Ó e ? ( ! ? c B ¢+¥7Ó B ¢+¥7Ó ? ` ¢ Ó , e ! ¢ B ¢+¥7Ó ¢Ó ¢Ó ½ Ò ¢/ç ¥7Ó èªM é L !1&2ê ( ì /B ¢`¥ ( Ó ½ Ò ( ½;Ò ? ½;Òî z ½ c BD&f&M L '! &)( ½ Ò > The last inequality is proved with (3.1). Thus Case iii in Lemma the PSD ? 3.1 guarantees that estimate (1.6) can be applied to the smallest Ritz value of ! 6 B ( in ijEk%mn & 6S&Ø p . By (3.6) this Ritz vector is V &, &Ø with the Ritz value þ ¤ . Hence the vectorial PSD estimate (1.6) results in (3.9) , z ! ( ª 3u Õ L !'&)(× þ q u ! ( { ¤ > ! (, z ETNA Kent State University http://etna.math.kent.edu 100 K. NEYMEYR AND M. ZHOU This? concludes the proof since L of ! 6 B ( in ijlk<mn ¢ p , i.e., !'&)( , &/Z ijlk<m)n ¢ p (3.10) , is bounded by the largest Ritz value ¤ L ª 3u Õ 1! &2(=×{ ª 3u !' ¤ ( > The chain of inequalities (3.8)–(3.10) proves the proposition. The convergence estimates for the remaining Ritz values Ì@T , w x 6 >;>;> 6S x , follow from Corollary 3.3 by induction. Comparable estimates for the remaining x Ritz values, but with respect to different spaces, have been used in [14, Theorem 3.3] for the subspace analysis of the preconditioned inverse iteration and in [19, Theorem 3.1] for the block steepest descent iteration. 3.4. Let ¡ be an -dimensional subspace of the t . The Ritz vectors of ?! B T HEOREM 6 ( in ¡ are denoted by ·y6 >;>;> 6·<¤ and let ¢ ·y6 >;>;> 6·<¤ ¸ Z tE£N¤ . The associated @ Z! 42¾q¿ 6q4E¾q¿ u ( . Ritz values are @ L ! · @ ( with { ? >;>;> {G ¤ . Indices @ are e ? given so that The smallest Ritz values of ! 6 B ( in i=jlk%mn ¢¶6 ! ¢ B ¢+¥ (ªp are denoted by y@ T with "T { >;>s> {G"¤T . Then for each w Z nyx 6 >;>;> 6S p it holds that y@ T { @ and either "@T { 42¾q¿ or ¾q¿ u !' @T ( { @ , z ! @ ( q ¾ ¿ 42¾q¿ ! 4 t 42¾q¿ u ( > X @ 8 (3.11) with z ! @$(-, @ 4 ¾ ¿ u ! 4Et 4 ¾ ¿ ( ¾ ¿ ¾ ¿ u !'@$( Here !'"( !' 4 (=MN! 4 î"( . ? The bound (3.11) cannot be improved in terms of the eigenvalues of ! 6 B ( as for each w the upper bound is attained for %@ _ 4 ¾ ¿ in the 3D invariant subspace associated with the eigenvalues 4 ¾ ¿ , 4 ¾ ¿ u and 4lt . Proof. The proof is given by induction over the subspace dimension . For a onedimensional subspace ¡ ijEk%mn &p , Theorem 1.1 proves the Ritz value estimate (3.11) with L !'& T ( T . ? For an -dimensional subspace the Ritz vectors · @ of ! 6 B ( are arranged in the columns of ¢ ? ·"%6 >s>;> 6=·<¤ ¸ ¢µ¬ ¤ c ¯q6·<¤ ¸ Z tE£l¤ . Let "@ T ! ¢R¬ ¤ c ¯ ( be the x smallest Ritz values of ! 6 B ( in i=jlk%mn ¢ ¬ ¤ c ¯ 6 e Õ ? ¢ ¬ ¤ c ¯ B ¢ ¬ ¤ c ¯ ¥ ¬ ¤ c ¯ × p 6 ¥ ¬ ¤ c ¯ A¦0§ k<¨ !1 6 >s>;> 6 ¤ c ( 6 with <T ! ¢R¬ ¤ c ¯ (Y{ >;>s> {G"¤T c ! ¢µ¬ ¤ c ¯ ( . The induction hypothesis with respect to the ! x ( dimensional space reads u !' @T ! ¤ c (( , z ! ( 8 { ¾ ¿ ¾ ¿ ¾q¿ ¾q¿ u ¢ !1 ¬ @ ( ¯ { ! ( , z 4 ¾ ¿ ! 4lt 4 ¾ ¿ u ( E4 ¾q¿ u ! 4 t E4 ¾q¿ ( > ¤ The Courant-Fischer variational principles show that these x smallest Ritz values 0@ T ! ¢µ¬ c ¯ ( ¤ i l j < k m n c p decrease when ¢µ¬ ¯ is enlarged to ¡ , i.e., Ú m ÚRk%ö L ! N( @T ! ¢ ¬ ¤ c ¯ ( ñ Ý'Þsß=àá ü Õ Ò ü§ c ± ü ü × ä Ü ø á ­ ä å ósô õ ë @ ¬ ¯ ñ Ý'Þsß=àá Ú § Ò m ± ä Ü ÚRkø öá ä L ! N( @T > ó;ô õ å ¬ ë c @ ¯ ­ ¬ ¯ Finally, ¾q¿ ¾q¿ u !1"( is a monotone increasing function in so that ¤ z ¾ ¿ ¾ ¿ uu !'!'< @@T (( { ¾ ¿ ¾ ¿ u !1<u @T ! ¢f!' ¬ @ ( c ¯ (=( { ! , ! ( , z ( 6 ¾q¿ ¾q¿ ¾q¿ ¾q¿ ¶ with ETNA Kent State University http://etna.math.kent.edu BLOCK PRECONDITIONED STEEPEST DESCENT 101 which proves the proposition for the first x smallest Ritz values. For the Ritz value ¤ Corollary 3.3 completes the proof. The sharpness of the estimate (3.11) is a consequence of Theorem 1.1 since for @ Z ! 42¾q¿ª6S42¾q¿ u ( the columns of ¢ can be formed by the vector of poorest convergence in )¾q¿ ¾q¿"!# t : all other columns are taken as eigenvectors with indexes different form @ , @ , x and and . Then the subspace iteration behaves like the vectorial preconditioned steepest descent iteration due to stationarity of the iteration in the eigenvectors. C OROLLARY 3.5. The Ritz value convergence estimate (3.11) cannot be improved in terms of the eigenvalues 4)¾q¿ , 42¾q¿ u and 4 t and is attainable for @ _42¾q¿ in N¾q¿ ¾q¿ u t . Proof. For @ Z! 4E¾q¿;6S42¾q¿ u ( let · @ be an B -normalized vector in the invariant subspace N¾q¿ ¾q¿ u t with L ? ! · @ ( @ . The remaining columns of ¢ are filled with pairwise different eigenvectors of ! 6 B ( which are B -orthogonal to E¾q¿ ¾q¿ u t . Then a step of the block steepest descent iteration, Algorithm 1, leaves all the eigenvectors invariant. The convergence of the w th column · @ is exactly that of the vectorial preconditioned steepest descent iteration as treated in Theorem 1.1 because the iteration is stationary in all eigenvectors. The nonimprovability of the convergence estimate in terms of the eigenvalues has already been treated in Theorem 1.1. Corollary 3.5 shows that the convergence estimate (3.11) cannot be improved in terms of the eigenvalues without further assumptions on the subspace ¡ . Hence cluster robust convergence estimates, which should depend in some way on the ratio 4 @rM 4E¤ u , are not provable in terms of the convergence measure ª 3u as used in Theorem 3.4. The numerical experiments in Section 4 illustrate that the block preconditioned steepest descent iteration behaves as a cluster robust iteration. In order to derive cluster robust convergence estimates, additional assumptions have to be made: For instance the angle between the iteration subspace and the invariant subspace has to be bounded. 4. Numerical experiments. The block preconditioned steepest descent iteration is applied to the Laplacian eigenvalue problem (4.1) * !'&2( 54 *ù!1&2( 6 &oZ %$ Õ&('*) i !+¼( 6 & i § m !+ ( × & Z 8l6 x ¸ 6 +GZ-, 8N6l ³/.10 on the unit circle with a slit along the positive abscissa. Homogeneous Dirichlet boundary conditions are given on the boundary for & x and on the upper side of the slit n ! & 6 +¼( & Z 8N6 x ¸ 6 + Ð8 p . Homogeneous Neumann boundary conditions are used on the lower side of the slit. The numerical approximations of the eigenvalues and eigenfunctions can be compared with the exact solutions of (4.1). The eigenfunctions are i § m Õ32 ¾ +¼×14#576Ì!8 ¾ 9 & ( ; see Fig. 4.1. Here 4:5:6 is a Bessel function of first kind of fractional order [1] and 2 ¾ , ; . The eigenvalues are the squares of the positive zeros 8 ¾ 9 of 4#576 . The operator eigenvalue problem is discretized by linear finite elements on a triangle mesh. Our program code AMPE (Adaptive Multigrid Preconditioned Eigensolver) in FORTRAN is an adaptive multigrid finite element code with an edge oriented error estimator which uses linear and quadratic finite elements. All test computations have been executed on a personal computer with an Intel Xeon 3.2GHz CPU and with a RAM of 31.4GiB. Our finite element code on this computer can solve eigenvalue problems that exceed <8 x 8=< degrees of freedom. The program includes multigrid preconditioning with Jacobi smoothing. The FORTRAN code is embedded in a Matlab GUI which allows easy and convenient usage of the program and has graphical presentation of its output. Experiment I. In this first experiment the block preconditioned steepest descent iteration is applied in a -dimensional subspace; we refer to this algorithm as PSD( 75 ). Further, the eigenvalue problem is discretized on a series of uniform triangle meshes to which nested ETNA Kent State University http://etna.math.kent.edu 102 K. NEYMEYR AND M. ZHOU F IG . 4.1. Contour lines of the three eigenfunctions of (4.1) corresponding to the three smallest eigenvalues. iteration is applied. The first grid level comprises x nodes from which 15 are located on the Dirichlet boundary. This gives > initial degrees of freedom. On this coarsest level the eigenvalue problem is solved exactly (aside from rounding errors). Piecewise linear interpolation is used to prolongate the approximate eigenfunctions from one grid level to the next refined level. The multigrid preconditioner is a V-cycle multigrid solver with damped Jacobismoothing; the damping constant is V M and two pre-smoothing and post-smoothing iterations are applied on each level. : The quality of the preconditioner is controlled ? e by a stopping condition for the linear system solver. For each Ritz vector the bound ½ ! & ( & ½ X x 8 M : is tested. Here & e denotes the residual of a Ritz vector, & is the preconditioned residual and is the dimension of the discrete problem on the current level. In most cases only one V-cycle is required to reach this accuracy since the initial solution on a refined grid is the prolongation of the solution from the coarser grid. o ) is ½ & ½ å ! & å e & ( ?S X x 8 c ­ where & runs The stopping criterion ? @ forL ! PSD( · @C(B · @ for the Ritz vectors ·" , ·% and ·1@ . This stopping through the residuals · criterion is justified by the generalized Temple inequality (see [3, Lemma 3 in Chapter 11]), which is the first inequality in L !'&)(%Õ L !'&2(¼ 4 @×EÕ 4 v@ u L !'&2(=× x { ½ & ½ Ò { x z ½ & ½ if L !1&2(Z 4 @ 6S4 @vu ¸ > å 4 @ 4 @vu e ? ½sÒ { z X x . Hence ½ & ½ is an upper bound for The second inequality follows with ½ Ñ å the product of the relative distances of L !'&)( to the enclosing eigenvalues 4 @ and 4 @vu . x The nested iteration is stopped on the level A¶ with %8< B7C"8<< nodes and %8" xD >"8 degrees of freedom. Figure 4.2 (left) shows that the computational costs increase more or less linearly in the dimension of the problem which indicates the near optimal complexity of ¾ the PSD( ) solver. Figure 4.2 also shows the errors @ ¬ ¯ 4 @ , w} x 6q06S , for the three smallest eigenvalues 4 8 ­ ©FE > E "<< E 6 4 8 © x > x C E x BE6 4 @ 8 © x E > "<8 E C > ¾ The error ¬ ¯ 4 is relatively large since the associated eigenfunction has an unbounded derivative at the origin. The next experiment shows that the approximation of this eigenfunction clearly profits from an adaptively generated grid. Experiment II. Next we show that adaptive mesh generation with a posteriori edge oriented error estimation similar to that in [15] results in much better approximations. The ETNA Kent State University http://etna.math.kent.edu 103 BLOCK PRECONDITIONED STEEPEST DESCENT computational costs [sec] 1 PSfrag replacements N K 2 10 M JLHI K d.o.f. G 0 10 PSfrag replacements −1 10 d.o.f. −3 computational costs [sec] 10 computational costs [sec] −2 10 PSfrag replacements 10 −5 10 1 10 3 5 10 10 7 10 Level index 0 3 d.o.f. 6 9 12 Level index O(PRQ F IG . 4.2. PSD( ) nested iteration on uniform triangle meshes. Left: Computational costs until a level is reached and finished (solid line); computation time on the current level (markers). Center: Error of the eigenvalue by a line with markers, by a broken line and by a solid line. Right: The initial approximations, triangulation. STP/ STPRU S/PRQ error estimator computes the eigenvector residuals with respect to quadratic finite elements for Ritz vectors which are represented by linear finite elements. The largest (in modulus) components of the residual are used to select those edges which belong to triangles that are to be refined. Once again, the PSD( `| ) solver is used. In order to compute a grid which allows an optimal approximation of the eigenfunction associated with the smallest eigenvalue, the error estimation and grid refinement aim at an optimal approximation of just this eigenfunction. Figure 4.3 shows a relatively coarse triangulation of the unit circle and sectional enlargements of finer triangulations around the origin, where the depth of the triangulations takes its largest values due to the unbounded derivative ! M & (4 5:6 around & 8 . The adaptive process generates highly non-uniform triangulations. Further details of the adaptive process and its more or less linearly increasing costs (as a function of the d.o.f) are shown in Figure 4.4. The ©-E E "<< E resulting smallest Ritz values , which approximate the smallest eigenvalue 4 , > are as follows. Depth of triang. 1 30 43 57 68 73 Nodes 21 10709 108693 1185777 10961756 34157092 D.o.f. 6 10409 107630 1182184 10951337 34137627 12.95561 7.738704 7.733789 7.733379 7.733341 7.733338 In contrast to this, a uniform refinement yields a final depth 12 with 50348033 nodes only in E > EE "%" ; a comparable quality of approximation can be gained a poor approximation E7E nodes and } E > E >"C1B x . by the adaptive process on level 19 with % Finally, the results of the PSD iteration to approximate the 15 smallest eigenvalues in a 20-dimensional subspace are listed in Table 4.1. Experiment III. Next the case of the poorest convergence of the block preconditioned steepest descent iteration is explored. Therefore, we use the final grid from Experiment II with about > B x x 8" d.o.f. and apply the PSD( U ) iteration. According to Theorem 1.1, the poorest convergence of the vectorial iteration PSD( x ) is attained in the invariant subspace @' @vu t . The subspace iteration behaves similarly. To show this we consider subspaces which are spanned by a? single nonzero vector from @' @vu t whereas all: the other basis vectors are eigenvectors of ! 6 B ( with indexes different from w , w , x and . Then block-PSD method behaves like a vectorial iteration due to the stationarity in the eigenvectors. Theorem 1.1 ETNA Kent State University http://etna.math.kent.edu 104 K. NEYMEYR AND M. ZHOU t ë < , in V c X W t ëY ; ­Z< ; , in V c ­ 7 [q ­ 7 [ W PSfrag replacements PSfrag replacements PSfrag replacements t ë < Y ­ @ ; , in V c ­ ­ W , in , in , in F IG . 4.3. Triangle meshes and enlargements around the origin with 1716, 54064 and 165034 nodes (with 1628, 53387 and 163772 inner nodes). The associated depths of the triangulations are , and . The positive and belongs to the boundary. axis acbf deP \ Q^] _*` 25 50 PSfrag replacements PSfrag replacements PSfrag replacements N K 10 M JHI K G 0 10 d.o.f. computational costs [sec] −2 10 1 10 −1 10 computational costs [sec] −3 Depth of triangulation 10 −5 10 1 10 3 10 5 10 7 10 d.o.f. 10 d.o.f. Residual estimates computational costs [sec] 1 2 0 25 50 −1 10 −3 10 −5 10 75 0 Depth of triangulation 75 Depth of triangulation OfPgQ F IG . 4.4. Result of PSD( ) adaptive eigensolver. Left: Computational costs: Total computation time until # d.o.f. has been reached by solid line. Line with markers shows the computation time on the current level. Center: Errors for by line with markers, by broken line and by solid line. Right: Estimated residual norm for by solid line, estimate used for the stopping criterion by broken line and actual residual norm w.r.t. linear elements by line with markers. hjoq ilknm psr o SfP y a y {uz | yX} a y~ xh w SfPtU SuPvQ provides the convergence estimate for the single vector from @1 @u t . Figure 4.5 shows in the intervals ! 4 @ 6q4 @vu ( the upper bounds Õ , z ! ( × M Õ ! ( , z × (dashed lines) and the largest ratios @' @vu !' @T (=M @1 @u !1 @ ( for 1000 equispaced normalized test vectors in @' @vu t whose Rayleigh quotients equal @ . All this is done for equidistant @ Zº! 4 @ 6S4 @vu ( . In each interval 4 @ 6q4 @vu ( the estimate (3.11) is sharp, cf. Theorem 3.4, and can be attained for @ _4 @ . Experiment IV. In this experiment the sharp single-step estimates (3.11) are compared æ x the with multi-step estimates for the PSD( Rí ) iteration. For each grid level A with A 9 x initial subspace ¡¬­ ¯ , which is the prolongation of the ? final subspace from 9 the level A , ! 9 t B 1 9 ( is of sufficient quality so that the three Ritz values of 6 in ¡¬®­ ¯ have reached their “destination interval”, i.e., @=! ¡ ¬®­ 9 ¯ (Zî! 4 @¬ 9 ¯ S6 4 @v¬ u9 ¯ ( 6 ¾ 9 wù x 6SN6=N6 so9 that the Ritz values @ ¬ ¯ ( is the iteration index on level A ) do not leave this interval. Here, ! ? 9 6 Bt91( with respect to the grid level A . Theorem 3.4 guarantees 4 @¬ ¯ are the eigenvalues of 9 9 ¾ that § Ú ¾* @ ¬ ¯ A4 @ ¬ ¯ . ETNA Kent State University http://etna.math.kent.edu 105 BLOCK PRECONDITIONED STEEPEST DESCENT xz k of (4.1). Left: Ritz values in a _x\q^Q_ -dimensional linear finite element ªª^U*\Qª -dimensional linear finite element space. Right: “Exact” eigenvalues TABLE 4.1 The 15 smallest eigenvalues space. Center: Ritz values in a of (4.1). k l 0 1 2 3 4 5 6 7 8 9 1 7.733389 12.18725 17.35102 23.19983 29.71530 36.88311 44.69164 53.13167 62.19503 71.87493 2 34.88339 44.25893 54.36164 65.17971 76.70204 k l 0 1 2 3 4 5 6 7 8 9 1 7.733342 12.18715 17.35080 23.19943 29.71460 36.88199 44.69009 53.12939 62.19189 71.87073 2 34.88260 44.25768 54.35978 65.17704 76.69829 k l 0 1 2 3 4 5 6 7 8 9 1 7.733337 12.18714 17.35078 23.19939 29.71453 36.88189 44.68994 53.12918 62.19159 71.87033 2 34.88252 44.25756 54.35960 65.17677 76.69790 1 0.8 o o ^w xo o o w^ o 0.6 0.4 PSfrag replacements 0.2 0 7.733 12.187 17.351 @ 23.199 r o r o" w ho 29.715 F IG . 4.5. Poorest convergence of block preconditioned steepest descent iteration. Abscissa: Five smallest are the upper bounds eigenvalues according to Table 4.1. The dashed lines in the intervals for . The curves are the largest ratios over 1000 equispaced test vectors in whose Rayleigh quotients equal . z| U p z P o SJo Q ];K wÊ =U p o o w h o | o o" w h o All this allows us to apply the x -step estimates (4.2) @ ¬ ¾ u 9 ¯ 4 @¬ 9 ¯ { ¬ 9 ¯ , z ! ¬ 9 ¯ ( @ ¬ ¾ 9 ¯ 4 @¬ 9 ¯ ¡ !1 ¾ u 9 ( ¯ (6 o8l6 x 6 >;>s> 6 @¬ 9 u 9 ! ¬ 9 ¯ -( , z ¬ 9 ¯ 4 @v¬ u9 ¯ î @¬ ¾ 9 ¯ 4 @¬ u ¯ @ ¬ ¾ ¯ recursively, which yields the multistep estimate @ ¬ ¾ 9 ¯ 4 @¬ 9 ¯ { ¬ 9 ¯ , z ! ¬ 9 ¯ ( ¾ @ ¬®­ 9 ¯ 4 @¬ 9 ¯ £¢ !' ¾ 9 ( @ ¬ ¯ 6¤} x 6SN6 >s>;> 6 9 9 (4.3) ! ¬ 9 ¯ (d, z ¬ 9 ¯ 4 @v¬ u9 ¯ î @ ¬®­ 9 ¯ ¬4 @vu ¯ î @ ¬ ¾ ¯ 9 @ by 4 @¬ 9 ¯ for the relevant indexes w . The 4 where ¬ ¯ is given bye (3.11) after substitution of ? ½ Ò for the quality of the preconditioner parameter z ½;Ñ is approximated on each e ? grid level by computing the spectral radius of Ñ with the power method. For this experiment we use again two steps of pre-/post-smoothing on each level with9 damped (V M ) Jacobi-smoothing. This results in z © 8 > E C . The discrete eigenvalues 4 @¬ ¯ are estimated by extrapolation from the computed Ritz values. ETNA Kent State University http://etna.math.kent.edu 106 K. NEYMEYR AND M. ZHOU Figure 4.6 shows the multistep bound (4.3) as a bold line, the x -step bound as a dotted line and the numerical result as a thin solid line. The x -step estimate (4.2) is a very good upper estimate. In all cases the multistep estimate (4.3) is a relatively rough estimate. It accumulates the over-estimation of the error from step to step and suffers from its inability to ¾ 9 use the current @ ¬ ¯ on the right-hand side of the estimate in order to improve the quality of the upper bound. 9 The !1"( -ratio depends on the discrete eigenvalues 4d¬ ¯ and decreases monotonically for the iteration on each grid level; the ratio may increase after prolongation to a refined grid level. The somewhat oscillating behavior of the !1"( -ratio for 4) in contrast to the smoother behavior for 4) reflects the fact that the error estimation and grid generation is controlled by error estimates for the first eigenfunction. The second eigenfunction also profits from the grid refinement (cf. Figure 4.4) but the !1"( -ratio shows a stronger variation for changing level index A . 5. Conclusion. This paper concludes our efforts of analyzing preconditioned gradient iterations and their subspace variants with either fixed step length (case of inverse iteration and preconditioned inverse iteration) or with optimal step-length (case of steepest descent and preconditioned steepest descent). Within the hierarchy of preconditioned gradient iterations, as suggested in [13], these solvers are denoted as INVIT( , ) and PINVIT( , ) with µ x 6S and subspace dimensions Zg¥ . For all these iterative eigensolvers sharp convergence estimates have been derived which have the common form L T ¡ L @1 @vu ! !1& ((}{ @1 @vu ! !1&2(=( 8<( and convergence factors ¡ with @1 @vu ! 8<( ! 8U 4 @ (=MN! 4 @u ¦ convergence estimates have been gained: Iterative Eigensolver Convergence factor Inverse iteration ¡ Preconditioned inverse iteration Block preconditioned inverse iteration ¡ z ,A! x z ( §¿"!#¿ § Steepest descent Precond. steepest descent Block precond. steepest descent . The following sharp §¿ § "¿ !# ¡ ¨ 6 c ¨ §§ ¡ ¨ uuª ¬ u«c ª ¨ ¯ 6 ¬ c ¨¯ ¨ Ref. [16] ¿ ¬ §© c § ¿!: ¯ ¿!: ¬ §© c § ¿ ¯ § ¿¿!:¬ § © c § c ¿"!# ¿ ¯ § ¬§ © § ¯ [10] [14] [18] [17] here Scientific efforts for the future are aimed at a convergence analysis of the important locally optimal preconditioned conjugate gradient (LOPCG) iteration [9]. As the convergence behavior of the LOPCG eigensolver has been observed to behave similarly to the conjugate gradient iteration for linear systems, sharp convergence estimates are highly desired. Acknowledgement. The authors would like to express their gratitude to the anonymous referee for his very positive feedback and constructive remarks! REFERENCES [1] G. A RFKEN , Mathematical Methods for Physicists, Academic Press, San Diego, 1985. ETNA Kent State University http://etna.math.kent.edu 107 BLOCK PRECONDITIONED STEEPEST DESCENT Approximation of ± à 1 ²³´µ ¶ ³´Zµ · ³´Zµ 10 −2 10 −5 10 PSfrag replacements −8 10 0 50 100 Convergence history on levels ¬®­x¯¯x¯*­¬=° Approximation of ±T¸ 1 ²³´µ ¶ ³´Zµ · ³´Zµ 10 −2 10 −5 10 PSfrag replacements −8 10 STPG U 0 50 Convergence history on levels ¬®­x¯¯x¯*­¬=° 100 x x x oinkn om w h o i knm P-h o i knm p¹r oinknm | r oinknm w p h o i knm F IG . 4.6. Convergence history of the error ratios for and PSD( ). The -step estimate (4.2) (broken line) is good estimate for the numerical worst-case results (thin solid line); the multistep estimate (4.3) (bold line) accumulates the over-estimation from step to step. Q ETNA Kent State University http://etna.math.kent.edu 108 K. NEYMEYR AND M. ZHOU [2] Z. B AI , J. D EMMEL , J. D ONGARRA , A. R UHE , AND H. VAN DER V ORST (eds.), Templates for the Solution of Algebraic Aigenvalue Problems: A Practical Guide, SIAM, Philadelphia, 2000. [3] E. G. D’ YAKONOV , Optimization in Solving Elliptic Problems, CRC Press, Boca Raton, 1996. [4] G. H. G OLUB AND C. F. VAN L OAN , Matrix Computations, 3rd ed., Johns Hopkins University Press, Baltimore, 1996. [5] M. R. H ESTENES AND W. K ARUSH , A method of gradients for the calculation of the characteristic roots and vectors of a real symmetric matrix, J. Research Nat. Bur. Standards, 47 (1951), pp. 45–61. [6] L. V. K ANTOROVICH , Functional Analysis and Applied Mathematics, National Bureau of Standards, Los Angeles, 1952. [7] L. V. K ANTOROVICH AND G. P. A KILOV , Functional Analysis in Normed Spaces, Pergamon, Oxford, 1964. [8] A. V. K NYAZEV , Convergence rate estimates for iterative methods for a mesh symmetric eigenvalue problem, Soviet J. Numer. Anal. Math. Modelling, 2 (1987), pp. 371–396. , Preconditioned eigensolvers—an oxymoron?, Electron. Trans. Numer. Anal., 7 (1998), pp. 104–123. [9] http://etna.math.kent.edu/vol.7.1998/pp104-123.dir [10] A. V. K NYAZEV AND K. N EYMEYR , A geometric theory for preconditioned inverse iteration. III: A short and sharp convergence estimate for generalized eigenvalue problems, Linear Algebra Appl., 358 (2003), pp. 95–114. [11] , Gradient flow approach to geometric convergence analysis of preconditioned eigensolvers, SIAM J. Matrix Anal. Appl., 31 (2009), pp. 621–628. [12] A. V. K NYAZEV AND A. L. S KOROKHODOV , On exact estimates of the convergence rate of the steepest ascent method in the symmetric eigenvalue problem, Linear Algebra Appl., 154/156 (1991), pp. 245– 257. [13] K. N EYMEYR , A Hierarchy of Preconditioned Eigensolvers for Elliptic Differential Operators, Habilitation Thesis, Mathematische Fakultät, Universität Tübingen, 2001. , A geometric theory for preconditioned inverse iteration applied to a subspace, Math. Comp., 71 [14] (2002), pp. 197–216. [15] , A posteriori error estimation for elliptic eigenproblems, Numer. Linear Algebra Appl., 9 (2002), pp. 263–279. , A note on inverse iteration, Numer. Linear Algebra Appl., 12 (2005), pp. 1–8. [16] [17] , A geometric convergence theory for the preconditioned steepest descent iteration, SIAM J. Numer. Anal., 50 (2012), pp. 3188–3207. [18] K. N EYMEYR , E. OVTCHINNIKOV, AND M. Z HOU , Convergence analysis of gradient iterations for the symmetric eigenvalue problem, SIAM J. Matrix Anal. Appl., 32 (2011), pp. 443–456. [19] K. N EYMEYR AND M. Z HOU , Iterative minimization of the Rayleigh quotient by the block steepest descent iteration, Numer. Linear Algebra Appl., in press, doi:10.1002/nla.1915. [20] E. OVTCHINNIKOV , Convergence estimates for the generalized Davidson method for symmetric eigenvalue problems II: The subspace acceleration, SIAM J. Numer. Anal., 41 (2003), pp. 272–286. [21] B. N. PARLETT , The Symmetric Eigenvalue Problem, Prentice Hall, Englewood Cliffs, 1980. [22] J. P ONSTEIN , An extension of the min-max theorem, SIAM Rev., 7 (1965), pp. 181–188. [23] V. G. P RIKAZCHIKOV , Strict estimates of the rate of convergence of an iterative method of computing eigenvalues, U.S.S.R. Comput. Math. Math. Phys., 15 (1975), pp. 235–239. [24] Y. S AAD , Numerical Methods for Large Eigenvalue Problems, Manchester University Press, Manchester, 1992. [25] B. A. S AMOKIS , The steepest descent method for an eigenvalue problem with semi-bounded operators, Izv. Vyssh. Uchebn. Zaved. Mat., 5 (1958), pp. 105–114. [26] M. S ION , On general minimax theorems, Pacific J. Math., 8 (1958), pp. 171–176. [27] A. S TATHOPOULOS , Nearly optimal preconditioned methods for Hermitian eigenproblems under limited memory. I. Seeking one eigenvalue, SIAM J. Sci. Comput., 29 (2007), pp. 481–514. [28] P. F. Z HUK AND L. N. B ONDARENKO , Sharp estimates for the rate of convergence of the -step method of steepest descent in eigenvalue problems, Ukraı̈n. Mat. Zh., 49 (1998), pp. 1694–1699. O