ETNA
Electronic Transactions on Numerical Analysis.
Volume 25, pp. 67-100, 2006.
Copyright  2006, Kent State University.
ISSN 1068-9613.
Kent State University
etna@mcs.kent.edu
ERROR CONTROLLED REGULARIZATION BY PROJECTION
WOLFGANG DAHMEN AND MARKUS JÜRGENS
Dedicated to Ed Saff on the occasion of his 60th birthday
Abstract. The paper is concerned with regularization concepts for the inversion of diffusion processes. The
application of the involved evolution operators is based on Dunford integral representations combined with the
adaptive application of resolvents using recent wavelet methods. In particular, this allows us to develop and realize
numerically, to our knowledge for the first time in this context, an SVD projection method which is compared to
several versions of Tikhonov-type schemes. The theoretical findings are complemented by numerical tests shedding
some light on the quantitative performance of the schemes.
Key words. inverse problems, Dunford integrals, quadrature, Tikhonov method, projection methods, truncated
SVD expansion, adaptive wavelet methods
AMS subject classifications. 47A52, 65J20, 65J22
1. Introduction. The notion of “inverse problem” serves as an important conceptual
link between experimental and theoretical sciences. For example, unknown system parameters appearing in a model suggested by theory need to be retrieved from typically indirect
measurements. “Given an answer one has to look for the question” perhaps best reflects
the typical ill-posedness of inverse problems. An important class of such even “severely”
ill-posed inverse problems arises in connection with evolution equations of the form
(1.1)
),+-/.
where is a function in time taking values in some Hilbert space and is a positive definite
operator on
. When
, the negative Laplacian,
, where
is an open domain, this is the classical heat equation describing heat diffusion. The
“solution” operator that assigns to any initial data
the solution
at later time
is given by the operator exponential
and is known to define an analytic semigroup.
When
has an unbounded spectrum will be compact. A classical inverse
problem consists in finding the initial data
from the solution
at some
fixed time
. Thus one has to “undo diffusion” which is expected to be a delicate task.
A similar situation arises with the inhomogeneous problem
0
,= > @? 2D5G
!#"$
0 8:9;3<
A
0
&%(')*
1324
2657
0 3ABC24D789E;3<F
HI
KJL/
where the solution operator is given by
(1.2)
0 JMN324@KOHP 8 9RQTSU9E;3V3< J13WX4YUWXZ
;
0 = ?\[
These are special instances of the following general situation. Let
and compact operator between two Hilbert spaces and . We aim at solving
(1.3)
01] _^
[
be a linear
` Received August 12, 2005. Accepted for publication February 1, 2006. Recommended by D. Lubinsky. This
research was supported in part by the EEC Human Potential Programme under contract HPRN-CT-2002-00286,
“Breaking Complexity”, and the SFB 401, “Flow Modulation and Fluid-Structure Interaction at Airplane Wings”,
funded by the German Research Foundation.
RWTH Aachen, Institut für Geometrie und Praktische Mathematik, Templergraben 55, D-52056 Aachen, Germany (dahmen@igpm.rwth-aachen.de).
67
ETNA
Kent State University
etna@mcs.kent.edu
68
W. DAHMEN AND M. JURGENS
^Ha_[
^Ga_bIcUdeUfU 0 ^ga
^
D0 h = 0h ij[k? . This equation has no solution unless
. Even if
for some given
, the solution is in general neither unique nor continuously depending on . A way
to restore uniqueness is to impose some additional selection criterion. A theoretical tool to
solve (1.3) in this sense is the Moore-Penrose generalized inverse
,
, defined by
bIcUdeUfU 0 0 h bcdefU 0 FlbIcUdeUf 0 nm
y v wz{wU|E
] h 0 h ^KO oqposr u t Yvxw 0 ^A
bIcdXeUfU 0 0 0
where
denotes the spectral family of the self-adjoint, compact operator
. The
integral converges if and only if belongs to
. Thus when
is not closed
may fail to belong to
. Equivalently,
can be defined as the least-squares
solution of (1.3) with minimal norm, i.e.
^}a~[
^
0@h [
0 h n[
]Mh
€ 01] h 
" ^ N€ ‚ „
Bˆ ƒ…‰Ud‡Š † € 0/‹ "^ €N‚
and A
] h has minimal norm among all ‹ a for which the infimum is attained.
Still the unboundedness of the (generalized) inverse prevents a continuous dependence
of the solution on the data so that regularization is necessary. Of course, this subject has been
treated in numerous studies documented in the literature. The two most common regularization strategies are Tikhonov type schemes and projection methods. The numerical implementation of Tikhonov schemes requires the repeated approximate application of the solution
operator and its dual which is typically done with the aid of time stepping schemes like the
method of lines for the solution of the forward problem. This is computationally rather demanding while the so called “qualification” (limiting the attainable order of accuracy in terms
of a decreasing noise level, see [9]) is only
. Among the class of projection methods
a prominent role is played by truncated singular value expansions (SVD-expansions) which
yield in some sense “optimal” projection methods, see e.g. [9]. Unfortunately the required
system of singular basis functions is usually not numerically accessible at affordable cost.
The objective of this paper is to develop and analyze certain new variants of both strategies, Tikhonov and projection methods, for the inversion of diffusion processes. Since the
essential algorithmic ingredients for the treatment of operators of the form
can
be carried over to those given by (1.2) (see [15]) we shall exemplify matters for
.
In Tikhonov schemes the explicit regularization parameter affects the whole spectrum of the
operator. Therefore one might expect improvements from penalizing only the small part of
the spectrum which will lead to one of the variants. This will be facilitated through certain
projections based on representations as contour integrals in the complex plane similar to analogous representations of the solution operator itself. A central issue in this paper is therefore
the development and analysis of error controlled numerical schemes for the evaluation of such
integral representations which will allow us to completely avoid costly time stepping schemes
in either regularization method. Perhaps more importantly, this will allow us to realize, to our
knowledge for the first time in an efficient way an (approximate) SVD-projection method
for the inversion of (the full infinite dimensional) diffusion processes without ever having to
determine the singular basis functions.
Our schemes are based on a suitable quadrature rule for the contour integrals combined
with an error controlled application of the involved resolvents at the quadrature points. A
computationally attractive feature is that the different resolvent calculations are completely
independent which leads to a trivial but highly efficient parallelization of the schemes. In
principle, the resolvent equations (which are well-posed) can be solved by any discretization
that guarantees a given accuracy tolerance. Here we shall employ recent adaptive wavelet
Πi
t
8‡9;3<
0 
8:9E;3<
0 Ž
ETNA
Kent State University
etna@mcs.kent.edu
69
ERROR CONTROLLED REGULARIZATION BY PROJECTION
schemes for the following reasons: Due to the induced norm equivalences, the wavelet setting
allows us to realize conveniently the relevant topologies arising in this context and to control
errors in the right norms. For instance, involved applications of resolvents can be performed
with optimal complexity in the sense of [2] within desired error tolerances. Consequently, the
application of the true operator exponential (not of
for some given discretization
of
and its inherited inaccuracy) can be given with rigorous accuracy bounds. In this sense the
approach allows us to disentangle regularization and discretization of in favor of transparent
and accurate error bounds. Finally, the complexity results for the adaptive schemes from [2]
will be seen to offer a first rough assessment of the computational complexity of the SVDprojection regularization. In this context one encounters a number of interesting questions
that remain open but indicate in our opinion a promising potential of this approach.
8 9;3<
’‘
It should be mentioned that a direct approximate application of the operator exponential
can be also based on the hierarchical matrix concept, see e.g. [11]. However, this is an
entirely discrete approach approximating
for a given matrix approximation
of .
Some comments on the resulting conceptual distinctions can be found at the end of 2.
8“9;3<
”
‘
”
The layout of the paper is as follows. In 2 we collect some basics from regularization
theory and formulate the principal strategies. 3 is devoted to the concept of separating the
spectrum of closed linear operators which our projection method will be based upon. We
identify a contour integral representation of the boundedly invertible projection of onto the
eigenspace of corresponding to a bounded part of its spectrum. This will serve later as the
regularized approximation to .
”
0
0
”
In 4 we present the main regularization algorithm in terms of several basic routines. At
this stage these routines will be formulated merely in terms of abstract building blocks each
of which describes a numerical task (such as the solution of certain operator equations or
the application of the exponential of an operator) to be performed within a given accuracy
tolerance. As an important ingredient in this context we analyze in addition the quadrature
error arising from the discretization of the integrals and explain how to control this error when
using the discrepancy principle.
In principle one could realize the above routines through any of the standard discretization concepts. Here we employ recent adaptive wavelet methods discussed in 5. More
precisely, our approach is based on an adaptive paradigm developed in [2] for operators with
sparse wavelet representations. The main reasons for this choice can be summarized as follows. The current understanding of adaptive wavelet concepts allows us to draw first conclusions on the complexity of such regularization tasks. Moreover, the norm equivalences
characterizing relevant function spaces in terms of weighted sequence norms for wavelet coefficient arrays suggests, as a byproduct, a new regularization scheme employing different
smoothness norms. The resulting computable algorithms for the above building blocks hinge
on suitable wavelet representations of the involved operators and corresponding adaptive evaluation schemes for their application within any desired accuracy tolerance. These ingredients
are also derived in this section.
”
”
Finally, in 6 the theoretical findings are quantified and illustrated by numerical experiments.
2. Regularization. We begin with briefly recalling some facts from regularization theory that will guide the subsequent developments, see [9] for details and proofs of the quoted
results.
:– — u ˜ 5_
0@h
u
One way to approximate
by a bounded function is to replace
functions
,
, which are bounded and tend to
pointwise if
t™•
t{˜ •
u
by a family of
approaches zero.
ETNA
Kent State University
etna@mcs.kent.edu
70
W. DAHMEN AND M. JURGENS
€ 0 € =š € 0 N€ › ŠDœ ‚ , the regularized solution is then defined as
Q V
]— –— 0 0 0 ^ O ožpožr –— u Yv w 0 ^
€ € '¢¡ . In particular,
u
where the integral converges for every ^Ÿa
[ as –“— is bounded on 0
the operator –:— 0 0 0 =U[£? is continuous with
€ –— 0 0 0 €6¤G¥n¦§ y¨ uª© –— u © = u a € 0 € ' ¡ z Z
Abbreviating
The residual can be explicitly calculated as
] h " ] — ]
7O
h " – — 0 0 0 ^j«¬" –— 0 0 0 0 ] h
u
ožpožr ­ — /Y:vw ] h
with
u
u u
­ — @ t " – — ®Z
The approximation properties of this method are stated
€ ' ¡ ?¯- theorem.
u in the€ following
T HEOREM 2.1. ([9], Theorem 4.1) Let – — ~= 0
fulfill the following
assumptions for all ˜ 5„ : –:— is piecewise continuous, there exists a constant °(± 5„
independent of ˜ such that
© u – — u © ¤ ° ± and ² ƒ…³ – — u @ u¶µ u a~X € 0 € ' ¡ Z
(2.1)
t
—:´
Then
ƒ…³ ^
:— ² ´ –— 0 0 0
(2.2)
and
0 h ^A
ƒ…³ € ^ € k¸
:— ² ´ – — 0 0 0 Š
(2.3)
if
^·a 0 h ®
if
^}a ¹ 0 h ®Z
]Mh
Concrete rates of convergence in (2.2) can only be expected when
belongs to some
compact subset of
usually described in terms of “regularity” properties. A typical assumption leading to convergence rates is that
belongs to the range of some power of
.
Knowing such a convergence rate would be important for dealing with noisy data
which
are usually encountered in realistic applications. In fact, suppose that the noise level is
,
i.e.
]h
€ ^i"¼^ º € ‚ ¤ » Z
Accordingly, one obtains perturbed results
If a rate for
€ ]Ah " ] º € Š
—
] º— –— 0 0 0 ^ º Z
was known, in view of
€] h " ] º €Š ,
¤ € ] h " A] — € Š € ] — " ] º € Š —
—
^º 0 0
» 5g
ETNA
Kent State University
etna@mcs.kent.edu
71
ERROR CONTROLLED REGULARIZATION BY PROJECTION
€] — " ] º €Š
—
in order to balance both
it would be sufficient to bound the ‘propagated error’
terms and thereby obtain the right regularization.
T HEOREM 2.2. ([9], Theorem 4.2) Let the assumptions of Theorem 2.1 be fulfilled and
set for
˜ 5½
¾
' ¡qÀ
— ¥4¦X§¿ © – — u © = u a € 0 € Z
Then the estimates
°Â±
€ 0/] — " 10 ] º €N‚g¤ ° ± » —
and
¾
€ ] — " ] º € Š ¤7Á ° ± — » —
€] h "
hold true with
from (2.1).
Thus, the propagated error stays proportional to the noise level and a bound for
would tell us how to choose .
In practice, the necessary information is typically not available and a reasonable parameter value
has to be chosen for given data
in a different way. A widely
used method is the discrepancy principle which defines
] º— € Š
˜ ˜ » n^‡ºND5½
˜
»
^º
W»z
˜ ˜ » n^ º @ 4¥ ¦X§ By à 5½i= € 01] ĺ "^ º € _
(2.4)
with some fixed parameter
WÅ5 ¥n¦§ ¿ © ­ — u © = ˜ 5½ u a X € 0 € ' ¡ À Z
u
Under fairly general conditions on – — convergence
] º— QTº œ ÆNÇ V ? ] h if » ?\X
of optimal order can be shown ([9], Theorem 4.17).
The strict condition (2.4) can be relaxed to finding the largest
€ 01] º "¼^ º € ‚ ¤ W » ,
¤ € 10 ] 'º "^ º € ‚
—
—
˜
such that
holds which eases the use of the discrepancy principle.
Clearly,
is generally again the solution of an operator equation. Thus, in practice,
neither
nor the residual
can be calculated exactly. Aiming at a rigorous error
control we have to analyze the behavior of the discrepancy principle when both quantities are
replaced by approximations
and
.
L EMMA 2.3. Let
be a numerical approximation to the regularized solution
satisfying
] º—
(2.5)
] º—
€ 01] º i" ^“º € ‚
—
º—
Ib f ¥ —
]
È
] È º—
€ ]È º " ] º € Š ¤ ÊW É € 0 € 9 É » Z
— —
Moreover, suppose that the approximation to the true residual
satisfying
] º—
0]È º— "^“º
ËË b*f ¥ — " € 0]È º— "¼^ º € ‚ ËË ¤ WN' » where the positive constants W É 4W ' are chosen such that
W"WÊÉD"WN'¬5GÌ}=š ¥n¦§ ¿ © ­ — u © = ˜ 5G u a X € 0 € ' ¡ À Z
(2.7)
Now let ˜ ˜ » 4^‡ºB be chosen such that
b*f ¥ — ¤ W » ¤ bIf ¥ ' — Z
(2.8)
(2.6)
has norm
b*f ¥ —
ETNA
Kent State University
etna@mcs.kent.edu
72
W. DAHMEN AND M. JURGENS
]È º—
] aHº— bIcdXeUfU ?
Í
Î
^
]Ah »
¡ 0
H
a
I
b
U
c
d
U
e
f
Ó
Ò
a
"
A] h
0 0
Œ
Œ
Œ
t{•Ô
€ 01] º "g^‡º € ‚
] º—
—
to the regularized solutions
corresponding to the
Then the numerical approximations
method
converges in
to
for
for all
. In fact, it
has optimal order
when
for
, where
is the
“qualification” of the regularization method, see [9, Chapter 4.1].
Proof. We refer to the proof of Theorem 4.17 in [9] were the analogous statement is
shown for the discrepancy principle with exact values of
and
. A careful
inspection reveals that the claim is reduced to verifying the two estimates
– — 0 0 0 ˜ » rÏCrsÐ“Ï Ñ
€ 01]A— "¼^ € ‚ g
¤ ÕÉ»
Õ Õ
for some constants ə '’5G .
(2.9)
and
€ 01] ' — "^ € ‚ 5 Õ ' »
bIf ¥ — ¤ W » as follows. One has
€ 01] — "¼^ €B‚g¤7€ 0/] º "^ º €N‚ ÎÌ » ¤ª€ 0]È º "¼^ º €B‚ € 0 €U€ ] º " ]È º € Š ÎÌ »
¤ b*f ¥ — — WN' » WÊÉ » ÎÌ » ¤ 3— WiWÊÉWN'ÂÎÌM » — —
u
where the first inequality results from the properties of ­ — and the definition of Ì .
We infer the first part of (2.9) from the assumption
To see the second estimate of (2.9) we observe that
€ 01] º' "^ º €N‚gÖª€ 0]È º' "^ º €N‚ " € 0 €U€ ] º' " È] º' € Š ¤ *b f ¥ ' — "½3W É W ' »
—
—
—
—
5,3W"¼W É "¼W ' » Z
€
€N‚½Ö W "W É "~W ' "Ì » where W "~W É "W ' "Ì is positive due to
This implies 01] ' — "~^
the choice of W , W É , and W ' .
Loosely speaking, it is sufficient to keep the discretization errors proportional to the noise
level. It should be noticed that this result is purely asymptotic for » ?×‡Ø . In particular, the
constants WX4W{ə and WN' which heavily influence the outcome of numerical experiments can be
chosen freely as long as (2.7) is satisfied.
u
u ¢9 É yields
It remains to specify the regularization method –“— . Setting –— ’# ˜ the well-known Tikhonov regularization which reads
˜ «6 0 0 ] º— 0 ^ º Z
(2.10)
] º—
can equivalently be defined as solution of the minimization problem
? UÚ³‰Uƒ…Š d ˜ € ] € Š € 10 ] "^ º €N‚ Ù
€ 0/] "¼^‡º € ‚
€] €Š
Πwhich can be interpreted as compromise between the size of
and the residual error
. It is well-known that in this case the qualification is
.
The penalization imposed by the Tikhonov scheme affects the whole spectrum of
.
This raises the question whether the resulting bias could perhaps be diminished by making
the penalization dependent on the spectrum. In fact, it should be active only on the small
eigenvalues of
. Thus, a first alternative is
0 0
(2.11)
t
0 0
É
–:— u ( =šj ˜FÛÜ œ —Ý u F u 9 Z
This gives rise to what may be called “modified” Tikhonov regularization.
The second alternative deviates in a more essential way from the Tikhonov concept by
taking
(2.12)
w É uuàÖß ˜ u
*
š
=
Þ
:– —
˜ ETNA
Kent State University
etna@mcs.kent.edu
73
ERROR CONTROLLED REGULARIZATION BY PROJECTION
0 0
. This corresponds to setting the small singular
simply clipping the small eigenvalues of
values of to zero and thus to a truncated singular value expansion which is known to be in
some sense an “optimal” regularization method by projection. Moreover, in this case one has
, see e.g. [9, Example 4.8]. We refer to this regularization scheme as SVD-projection.
But of course, in practice the issue is to realize this projection method at affordable cost.
Both strategies (2.11), (2.12) hinge on the ability to realize projections
taking
into an eigenspace corresponding to
. Then (2.11) gives rise to
solving
0
Œ K¸
á — = ? â 0 0 Âã X ˜ ˜á — 0 0 ] º— 0 ^ º (2.13)
while (2.12) leads to
ä — 0 0/ä6—] º— ä — 0 ^ º ä6— =š«>" á>— Z
8:9;3< , as in ” 1, such projections can be efficiently
We shall show below that for 0 £
realized numerically which to our knowledge is new. Before addressing this in a systematic
fashion, a few comments are in order. Of course, discretizations of lead to approximations
x‘ of defined on a finite dimensional subspace of being now bounded with a norm
depending on the discretization parameter å . Hence the spectrum of 0 ‘8U9;3< is bounded
(2.14)
away from zero so that solving
0 ‘ 0 ‘ ] º— 0 ‘ ^ º
(2.15)
å
has a similar effect as (2.14) where e.g. the mesh size of the underlying discretization acts
as a regularization parameter. Solving (2.15) (which becomes increasingly ill-conditioned
for
) iteratively, the termination of the iteration adds another bias. Each application
of
or
(either by approximately solving evolution equations or by employing -matrix
techniques, [11]) is inaccurate. For -matrix concepts the problem is posed now in Euclidean
metric which hampers somewhat taking the topological requirements of the original problem
into proper account. So several sources of inaccuracies are mixed. We shall therefore pursue
here a different line separating regularization and discretization. In particular, this will allow
us to apply
for the “true” within any desired tolerance. This will be based
on two conceptual pillars: (a) the actual numerical realization of projections of the form
as
well as of
within any desired target accuracy; (b) the involved application of resolvents
appearing in the Dunford integral is based on adaptive wavelet schemes again
realizing proper target accuracies without ever fixing any discretization of beforehand.
We shall now fix our assumptions on the operator which, of course, relates to the
topologies of and the image space under . will always be assumed to be sectorial
and normal with spectrum in the right complex half plane. In particular, this covers selfadjoint
positive definite operators. Moreover, it will be important to specify the mapping properties
of . To this end, we shall always assume to be given a Gelfand triple of Hilbert spaces
satisfying
å ‘ ?æ
0
0‘
èÌM«"G6¢9
ä — 0 0/ä6— ¢9 É
8É 9;3<
é ê@ éìë
éìë
0
[
(2.16)
ç
ç
é
á —
0 éîí ?ïê í ? é ë ð4ñ ñ ò ó= 4ð ñ ñ ò®ô(õXô Õ < ° <
where
is the dual of
represented through the inner product
of the
pivot space . In these terms we require that there exist bounded positive constants
such that
(2.17)
ê
Õ < €®öM€C÷,¤ª€ öA€N÷Âø1¤ ° < C€ öA€N÷ àa é ETNA
Kent State University
etna@mcs.kent.edu
74
W. DAHMEN AND M. JURGENS
é
é¼ë
£= é ? é}ë
from onto . A typical case is ê£% ' )* and
É9 )*
é é É s)*® éìë is a é norm-isomorphism
which fits to the heat equation with &î"$ , say. This will
i.e.
serve as a guiding example in the selfadjoint positive definite case.
In most part of the subsequent developments we shall have
ê£ [@
(2.18)
é ë
> x é Ž[
which accounts for the fact that noise should not be measured in too strong a norm. Nevertheless, an alternative would be
, so that
, and later in 5.5 we
shall encounter intermediate cases.
R EMARK 1. An analogous mapping property to (2.17) does not hold for
,
i.e. we cannot find a pair of spaces for which an analogous estimate like (2.17) holds. Therefore, the concepts used e.g. in [3] do not apply in the present situation.
”
8:9E;3<
0 ù
3. Separation of Spectrum. We return to problems of the form (1.1). When
is a
Hilbert space and is self adjoint and positive definite there exists an orthonormal basis of
eigenvectors
of with associated eigenvalues
, sorted by decreasing size.
Then, the operator exponential can be written as
yB]Aú z ú ‰™û
uú Ž
5 8 9;3< ýþ ü 8 9; w ð ] ú ò Š õ Š ] ú úCÿ É
2 Ö Z
From this representation, it is immediately clear that
€ 8 9;3< € Š ¤ 8 9E; w € € Š and â 8 9;3< y z 8 9E;:QT<MV 2D5GZ
Ñ
w
:
8
9
Moreover, the eigenvector of ;3< to the eigenvalue 89; coincides with the eigenvector of
to the eigenvalue u ú . This close relationship can be carried over to a more general situation
as stated in the following theorem. To make that more precise, let us recall the following facts
relating the spectrum of to that of 8:9E;3< even for the wider class of sectorial operators.
T HEOREM 3.1. (Spectral mapping theorem, [10, 14]) Let be a sectorial operator on
a Banach space . For every 2D5G
w
â 8 9;3< y z 8 9; :QT<MV =ó y 8 9E; = u a â 6 z and
â 8 9E;3< 8 9;
™Q…<MV â 8:9E;3<
where denotes the point spectrum, i.e. the set of eigenvalues. Moreover, the eigenspaces
of and
are related by
8 9; w >« "8 9E;3< 2D5½
u a
ú‰
lin u «¬"~
Ô2
for each
and .
A representation of projections needed in (2.13), (2.14) can be based on the following
principle, known as separation of the spectrum, that projects as well as
onto the
eigenvector spaces of associated with a bounded subset of
.
To this end, assume that
is split into two disjoint, nonempty sets
and
such
that
lies in the interior of some Jordan curve and
in the exterior part. Then,
âÉ
(3.1)
â 6
=š
Ñ
=š
É
â x
â'
6(=ó t O 3ÌA«"6 9 É Y Ì
Ñ
Ô Ñ
89E;3<
âÉ
â'
ETNA
Kent State University
etna@mcs.kent.edu
ERROR CONTROLLED REGULARIZATION BY PROJECTION
75
defines a bounded linear operator. Its basic properties can be summarized as follows.
T HEOREM 3.2. (Theorem III.6.17 of [12]; 2.3 of [14]) The operator is a projection
with
for every . Moreover, setting
bcdefU D ú [AÉ*
”
a
C['j«>"
É =[ É ?\[ É = ] ?\
]
C ®
for all
] aÅ[ É '’= '{@ > Mã
['x?\['’= ] ?\
] for all ] a '{®
? [ É is linear and bounded, and
the operator É =U[ É \
â 6É ÉÊ â É{ â É '™ â '
3ÌA«>"’ÉC 9 jèÌM«¬"6 9 © ‚ Ììa !x®
3ÌA«>"x'B 9 É jèÌM«¬"6 9 É © ‚ Ñ Ììa !x®Z
r
If, in addition, is a Hilbert space and is normal, then is the orthogonal projection
onto [ É .
Thus, is split into a bounded part É and an unbounded one ' . As the projection is defined by a curve integral similar to the definition of the operator exponential 8‡9;3< , both
are compatible in the following sense.
P ROPOSITION 3.3. ([14], Prop. 2.3.3) The projection commutes with 8“9E;3< , which
implies 89E;3<([AÉN &[MÉ and 89E;3<(['{ &[' , i.e. [MÉ and [' are invariant subspaces of with respect to the semigroup. ¬É and x' generate semigroups in [MÉ and [' , respectively,
with
8 9;3< 8 9E;3< © ‚ 8 9E;3< © ‚ 8 9;3< Ñ 8 9E;3< © ‚ Ñ 8 9E;3< «" Ñ © ‚ Z
r
r
r
As É is bounded, y 89;3< z ; | extends to a group y 8:9E;3< z ; ‰" . In particular, 8:9E;3< ,
Ñ inverse
Ñ
Ñ
2Â5½ , is boundedly invertible with
8 9;3< 9 É _8 ;3< t O 8 ;$# 3ÌA«"6 9 É YÌ1Z
Ñ
Ñ
Ô
4
¥
¦
§
y % è̐Â=™Ììa â É z thereÑ exists a constant & Ö such that
Moreover, for every â 5
t
€ 8 ;3< € Š ¤ & 8 {; 2 Ö XZ
(3.2)
Ñ
To see how this relates to an SVD-projection for 0 ù8“9E;3< , suppose that the Jordan
curve
eigenvalues of the operator ' _ } that are smaller than
É e É É encloses all (positive)
ËË ; ²)( — ËË . Recall that when is normal the product 0 0 takes the form
8 9E;3< * 8 9E;3< _8 9E; + ' GH (3.3)
w u
a single exponential. Since by Theorem 3.1, any , a â 0 0 has the form , #8:9E; ,
an eigenvalue of ' , we infer from Proposition 3.3 and (2.14) that with the projection 䬗 ' with respect to ' one has
Ñ
Ñ
(3.4)
] º— ª 0 0- h 0 ^ º 8 ; + Ñ 8 9;3< * ^ º Z
Ñ
Ñ
ETNA
Kent State University
etna@mcs.kent.edu
76
W. DAHMEN AND M. JURGENS
Hence the SVD-projection reduces here to the application of an exponential and a projected
exponential.
Things simplify when is selfadjoint and positive definite, so that singular values and
eigenvalues coincide. Now one needs a curve enclosing all eigenvalues of up to the size
and a projection to obtain from (3.4), again in view of the properties stated
.(
in Proposition 3.3,
É
É
ËË ' ; ² e — ËË
Ñ
6
] º— 8 ;3< Ñ ^ º Z
(3.5)
Thus regularization along this line amounts to just applying the inverse of a projected semigroup (and in the general case of an exponential) as opposed to solving operator equations of
the type (2.10) or (2.13).
Of course, (3.5) can always be used in this form as long as the separation of spectrum is
feasible, even when is not selfadjoint. The regularization scheme (3.5) does then, however,
not necessarily agree with an SVD-projection. For simplicity, we shall concentrate in what
follows on (3.5), remarking that with the tools developed below, we could realize (3.4) as
well.
We close this section with the following observation.
R EMARK 2. The projected semigroup provides also an efficient way for long time integration, i.e. to apply
for -/
. Due to Proposition 3.3, one has
:8 9E;3< 2 ×
€ 8 9;3< "8 9;3< € Š € «¬" Ó8 9;3< € Š ¤ 8 9;0 123Q) V € € Š Z
r
ƒ
‡
d
†
Ê
'
·
!
5
2
?
¸
:
8
9
3
;
<
Thus, for
and
, the semigroup
is closely approximated by the
% â
projected semigroup and it is sufficient to apply 8“9E;3< instead of 89E;3< . Moreover, when is a suitable open curve separating the spectrum of from the rest of the complex plane we
have formally the same Dunford integral representation
8 9E;3< ö 8 9E;3< ö (3.6)
É
Ö
t O 8 9;$# 3ÌA«¬"~x 9 YÌ1L2 Ô
of the full exponential which is valid for sectorial operators.
4. Algorithmic Concepts. In this section we shall formulate the main algorithm, which
realizes an appropriate regularization scheme based on a version of the discrepancy principle
that accounts for numerical errors in a theoretically sound way.
To this end, we shall first identify the main building blocks of this scheme which we
formulate at this point as abstract routines that perform certain tasks within any prescribed
accuracy tolerance.
The first ingredient is the solution of an operator equation
(4.1)
'
Ÿ ö = é ? éìë or ' =Xê&? ê is invertible and where in either case accuracy is
ê , recall (2.18). Thus we require a routine
ö
€
9 É öM€ ô ¤ 7AZ
465 S OLVE ' 8 7A ® such that 4 " '
where either '
to be controlled in
(4.2)
In view of (2.10) and (2.13), we will have to use this routine for instance with
(4.3)
'
˜ «6 0 0
or '
˜1á>— 0 0 ö 0 ^ º 0 8 9;3< 2Â5½Z
ETNA
Kent State University
etna@mcs.kent.edu
77
ERROR CONTROLLED REGULARIZATION BY PROJECTION
0
while the
Note that in both cases the right hand side is the result of the application of
regularization (3.5) requires the application of
. In both cases the integrals (3.6) and
(3.2) involve the application of resolvents which corresponds to (4.1) with
8{;3<
(4.4)
Ñ
gÌM«¬" ö _ Z
'
Thus the second main building block (which in turn will invoke the S OLVE-routine) is the
approximate application of operators of the form
,
, or
,
. We shall
formalize this as a routine
8“9;3< 2Å5î
(4.5)
465
Ñ
2Åaª-
€ 4 "~8 9;3< Mö € Š ¤ 7AZ
89E;3<
separates the complete spectrum we have 8“9;3< K
A PPLY
32® 87 ö 8Ê;3<
such that
2
Recall that when and must be
positive.
To make our approach as transparent as possible, we shall indicate first how to realize
(4.5) given (4.2) and how to realize the discrepancy principle given (4.5) postponing the
concrete realization of (4.2) to the next section, see 5.
”
4.1. Applying Projections. We are now prepared to formulate algorithms for computing the application of operators of the form
8 9;3<
or as the special case
ö a ö t O
8 E9 ;$# 3 ÌA«>"6 9 É ö YUÌ1
Ñ Ô Ñ
2 , of the
projection
Éö
ö t O èÌM«>"~x 9 Y Ì1
Ñ Ô Ñ
2Da
-D
2
to some fixed
. We emphasize that in connection with projected semigroups may
attain negative values as well. Aside from being able to apply resolvents within a given
accuracy tolerance (see (4.4)) we shall need a suitable quadrature scheme.
Our convention will be that when writing we assume that the Jordan curve encloses some bounded subset of
, as described in 3, and that an analytic and 9
9
9
periodic parameterization
â x
? Ì1 ®
É
”
Ô
É
a X C Ô
9
9
is given. Then, the 9integrand 9
: £" Ì ë Ó 8 9E;$#:Q.;qV 3Ì1 Ó«>"~x 9 É ö a ®
Ô
: =@so
is also -periodic and analytic in the topology of
that the following theorem applies.
-#? be analytic and -periodic.
Ô
T HEOREM
4.1. ([13], Theorem 12.6) Let
:
5
Ô func
Then, there exists <
such that extends
toß an analytic, bounded, and -periodic
©

©
z
·
^
=
Å
a
Â
^
Ô
tion on the strip >
= yÊ]
]
< . Moreover, the quadrature error of the
trapezoidal rule
'? 9 É
þ : : @
a D
á ?
@
t úCÿ A
ÔA
A
is bounded by
9
9
C' B
€
¤ª€ : E€ DGF œ Š '
:
: €
Z
(4.6)
IQ HKJ V 8 =L? t "
t O ‡Y " @á ? Š
Ô
t
ETNA
Kent State University
etna@mcs.kent.edu
78
W. DAHMEN AND M. JURGENS
É
Analogous statements hold when replacing by an open curve to represent the full
exponential (for
). For corresponding details on suitable analytic parameterizations we
refer e.g. to [16] and will make use of these results without further elaboration.
: MD F
Since we are lacking any estimate for < and
it seems reasonable to use
IH J
2I5K
€ €
Q œŠ V
€ @á ? : 1 "~8 9;3<
€ á@? : 1 " á ' ? : € 6
Š N
¤ Ñ
öA€ Š
as error estimator. This is certainly true when (4.6) holds with replaced by O . To ensure
convergence of the upcoming algorithm we therefore make the following
EP
P
A SSUMPTION 1. There exist constants
and
such that
Õ ° G
Ö
5 ÕP6€ @á ? : 1 " á ' ? : € Š ¤7€ á@? : 1"8 9A ;3< t Aö € Š
Ñ
and
Ö
€ á@? : 1"8 9;3<
Ñ
öA€ Š ¤ ° Px€ @á ? : 1 " á ' ? : € Š
for all
.
9
A
ToA formulate
the following algorithm,
we 9 define
, S T . The quadrature
9 B
Q
ARQ
A . Obviously, the sequences
points associated with
are
?VU ,
9
:
Q
WAXQ
Q i.e.
of quadrature points areA>nested,
. Therefore,
can be improved to
? U
:
:
Q
by evaluating the integrand Q at the nodes
,
.
? U
Y
7
A
Q
Q
A LGORITHM 1. w 5 A PPLY á ' Z
Z
Z
Z
a
N ZBZNZ® " Ô
Ô
t Ø É œ ' ú œ út
NZBZNZC
ÉØ œ ' ú Ø É á 2® Én A ö t
œú ú
"
ß [ ¤ ]C^`É _
\
Fix some 9" For X NZNZBZC
É
t
W3A Ì1 t Ó«>"~>
Ô
b 0 1c œed
ö 9œú
]Mi # ø Qj;lkLaG
4 œ ú 5 S OLVE
m
E
V
p
n
o
™
q
Q
E
9
$fh#:g Q.;lkYm V3V i
;
'
É
r
9 r ? k 9 Ì ë œ ú 48 9E;$#:Q.; kYm V 4 œ ú
s
4 5 ' ? kut ú®ÿ NZBZNZ
For S 8 vBZNZBZ®
t
9"
– for Ô Éœd
t
Ô
Aè>
Q
t
4 œX
ú 5 SOLVEÉ Ì/ 9 œ ú 4«>"~> ]Mi # ø Qj; U aGm b Vnp0 1ocq™Q9Er ;$fh#:g Q.; U m V3V i ö ? U 9 Ì ë Q '
Q 9 r
œ ú Ø É 489E$; #:Q.; U m V 4 œ ' ú Ø É
– 5 É' s
? U t ú®ÿ
r Ð“Ñ Q
Q
– Y Q 5 ' É 4 9 É "
– 4 Q € 5 € ' 4 Q ¤ 9 [ É Q Q 7 thenQ return 4 4 9 É .
– If Q Y Š
Q
Q
9É
t
R EMARK 3. To remain consistent with the theory the algorithm returns 4
. In pracQ
tice, 4 should be returned as it is the more accurate approximation.
87
Q
L EMMA
4.2. If Assumption 1 holds the algorithm A PPLY terminates after
7
a finite number of steps for every
and its output 4 satisfies
Ÿ5g
€ 4 "8 9;3<
Proof. Let 4
È œú
Q
32® Én> A ö Ñ
öM€ Š ¤ A7 Z
denote the exact solution
of
9
3Ì1 œ ú Ó«"6 4 È œ ú ö Q
Q
ETNA
Kent State University
etna@mcs.kent.edu
79
ERROR CONTROLLED REGULARIZATION BY PROJECTION
È Y È
9
È
èÌ/ œ ú ® A ö ' ?VU)w 9 Q É 9
þ Ñ © Ì ë É œ 48 9;$#:Q.; U)w m V © € É œ "
€
ú
ñ
4 9 ú
4È 9 É œ ú Š
9
Ñ
Q
Q
' ? ú®U)ÿw 9 É 7 Q v
þ Ñ
³ƒ…d y [ z ¤ v t 7³ƒ…d y v [ z Z
v
t Ô
t Ô
ú®ÿ and define , and 4 as the corresponding quantities from the algorithm. According to the
87
Q by S OLVE
Q
error boundsQ guaranteed
we have
€ 4 9 É " 4È 9 É € Š ¤
Q
Q
¤
ÔA
t9É
Q
t9É
ÔA>Q
The same argument yields
€ " È € Š ¤ x D7 ³ƒTd y v [ z € Y " Y È € Š ¤ 7D³ƒTd y v [ z Z
vt
t
t Ô
t Ô
Q
Q
Q
Q
First, the termination criterion is shown to be met for sufficiently large S ay
Theorem 4.1 and Assumption 1 the estimate
. From
€Y €Š ,
¤ € Y È € Š € Y " Y È €{z
Q.|/V
Q
¤ Õ P Q 9 É € : €ED Q F Q œ Š r '
QIH J V 8 . ?Vt U " t [ 7
t Ô
follows. Since the first
summand
tends to zero for S ?æ¸ there exist S Åa} such that the
€
€
¤
[
7 is indeed met.
termination criterion Y k Š
In a second step, theQ error bound is shown to hold for 4 4 k 9 É . Obviously, one has
€ 4 k®9 É("8 9E;3< öM€ Š ¤7€ 4 kN9 ÉÂ" 4 È kN9 É € Š € 4 È kN9 Q ÉÂ"~8 9;3< öA€ Š
Q
Ñ
Ñ
¤ 7¬Q € 4 È k 9 Q É "8 9;3< öAQ € Š Z
vt
Q
Ñ
The second summand is bounded with the aid of Assumption 1,
€ 4 È Ck 9 ÉD"~8 9E;3<
Q
öM€ Š ,
¤ € @á ? U w : 1"~8 9E;3< öA€ Š
¤ ° P€ @á k ? Ñ U w : /" á@? U : € Š
¤ ° P€ 4 È kC9 k ÉÂÑ " 4 È k € Š k
¤ ° P € Q Y k € Š Q € Y È k "Y k € Š Z
(4.7)
Q
Q
Q
Since the termination criterion is met the first summand in the parenthss is bounded by
€Y k€Š ¤
!~7
¤
v
Q
°
7
Z
P
The second one can be estimated by
€ Y È ®k 9 ÉÂ"~Y Ck 9 É € Š ¤
t
Q
Q
Ô
[ 7
¤

°
7
Inserting both estimates in (4.7) yields
€ 4 È k 9 É "8 9E;3<
Q
öA€ Š ¤
Thus, the overall error of the output 4 is bounded by 7 .
v
Ô A7 Z
P
Z
ETNA
Kent State University
etna@mcs.kent.edu
80
W. DAHMEN AND M. JURGENS
ÕP
is not needed in the
It should be noted that the explicit knowledge of the constant
algorithm. Its existence is merely needed to show that the algorithm terminates (i.e. it is
actually an algorithm).
87
R EMARK 4. An analogous scheme A PPLY can be constructed when is a
suitable open curve separating the spectrum
from the rest of the complex plane and is
positive, see [16]. We shall use the same notation for this version as well.
Since the constants in the quadrature rule are not numerically accessible, one might think
of extrapolation techniques to overcome this handicap. But extrapolation is useless for analytic periodic functions since all constants in the asymptotic error expansion of the trapezoidal
rule vanish.
2® n> A ö â x
8“9E;3<
2
4.2. Approximate Regularized Inversion of
. We shall use next the tools developed in the preceding section in order to regularize the inversion of the operator
for some fixed time
. Since is one-to-one and compact the operator fulfills the assumptions made in 2. We shall work in the setting (2.18).
The following algorithm comes in three variants each of which is based on Lemma 2.3.
We assume that the three parameters
are given such that
holds
(cf. (2.7)). Moreover, we fix
and test the stopping criterion
”
25L
0 89E;3<
0
WnW{ə4WN'5_
W "W{É("WN'5
Õ a t
€ 01] — t™•"Ô ^ tº € ô ¤ W » ¤,€ 01] º "¼^ º € ô
—
:
Ð
Ñ
Õ
˜ ú ú ˜ . Recall that we have here ê# [ . The choice of Õ permits a certain
for 1
tuning of the algorithm,
as the search for the optimal ˜ becomes more thorough and more
Õ
expensive for closer to one.
The three ways of determining a sequence of regularized solutions ] º— are based on the
two variants (2.10), (2.13) of the Tikhonov method or on the projection method (3.5). As a
core ingredient of the main scheme we shall provide for each variant a routine
]€5
R EG
˜ Y7A4^ º satisfying
€ ] " ] º € ô ¤ A7 Z
—
In the cases (2.10), (2.13) the realization of R EG is given by the scheme S OLVE for
solving the operator equations (4.1) corresponding to the choices (4.3).
The third method is quite different and uses the fact that, by Proposition 3.3, the projection of
onto the eigenspaces associated with a bounded subset of
becomes
boundedly invertible.

To fix ideas we briefly specify the discussion in 3 and assume that
with %
%
and %
for
. Then, by the spectral mapping
theorem,
has the eigenvalues
which tend
to zero if
. Thus, we discard
‚ and set
the eigenvalues for large by projection. Fix a
. According to the notation
of 3, the projection
onto the eigenspaces associated with
is denoted by . The projected semigroup is boundedly
invertible with inverse
0 8 9E;3<
â x
”? ¸
â 6Â y u ú = a z
u ú ¤ u ú Ø ÉÊ
u ú }? ¸
89E;3<
:8 9E; w
? ¸
a
Ɂ y u ú = ¤ ¤
â
z
t
”
Éâ
89E;3<_89E;3< =Mê¶? ê
Ñ
Ñ
Ñ
8 ;3< t O 8 ;$# 3ÌA«¬"~x 9 É YÌ
Ñ
Ô Ñ
and
€ 8 ;3< €C› Š ¤ ° 8 e; 3Q w k V Ñ Q V
Ö
™
°
for some constant °
. Here we take again êŽ ª[ . Thus, the regularized
t
solution defined as
] ºú k 8 ;3< Ñ ^ º ETNA
Kent State University
etna@mcs.kent.edu
ERROR CONTROLLED REGULARIZATION BY PROJECTION
^º
˜ ? ^
˜ t™•
81
plays the role of the regularization parameter, see
depends continuously on , and
) and is given exactly the regularized solution k
(3.5). Clearly, if
(i.e.
will tend to if has a complete system of eigenvectors.
When dealing with noisy data, has to be chosen properly. This can be done again with
the aid of the discrepancy principle, i.e. we pick
]Ah Ÿ?æ¸
]Mú
½ƒ…d‡† y a = € 8 9;3< ] ºú "¼^ º € ô ¤ W » z
Ö . A
for some fixed parameter W
t of the main algorithm arise from the choice of the correIn summary, the three variants
sponding auxiliary routine R EG ˜ Y 7A4^XºB which can be any one of the following three:
1. S OLVE ˜ «’ 0 0 8 7 0 ^“ºB ,
2. S OLVE ˜1á>— 0 0 8 7 0 ^ º , c.f. (4.3),
3. A PPLY Ó"*2® É ˜ ®nY 7A4^‡º{ .
Each of these routines returns a regularized solution of the ill-posed equation 0/] ½^º .
Of course, embedded in the routines S OLVE is a way for evaluating the given data ^º
(depending on the format in which they are given). This is explicitly used when computing
residuals. We refer to a corresponding routine
ƒ
465
A E VAL 7 4
satisfying
€ 4ƒ "
4
€ ô ¤ A7 Z
The realization of the discrepancy principle for either of these three choices reads now
as follows.
A LGORITHM 2. x 5 D ISCREPANCY P RINCIPLE
, R EG
For k = 0, 1, . . .
^ºU » ˜ =š Õ ú ˜ ˜
ú
É
Z ]ú„5 R EG ˜1ú 4W É € 0 € 9 » n^‡ºC
É
Z 4Iú\5 A PPLY 32® G ' W ' » ]ú É
Z
' W ' » n^‡ºC
ú € 5…€hz 4Iú " ¤ EVAL
­
Z If
W
­ ú jQ |/V » then return ]5„]Aú .
r
T HEOREM 4.3. ß Algorithm
D ISCREPANCY P RINCIPLE defines a regularization scheme of
Œ ¤
optimal order for for the Tikhonov-type schemes (cases Z and Z ) and for any
•Ô
t
Ô
positive Œ for the projection t™scheme.
More precisely, the output ] of D ISCREPANCY
P RINCI 3^ º » ˜ Ê satisfies
PLE
€ ] " ] h € ô ¤ °’» !
(4.8)
rsÏCrsÏÐ:Ñ rsÏCÑ Ð“Ñ € € ¤
€
€
¤
for all ^‡º6a
[ with ^‡º@"¼^ ô
» and all ]Ah £ 0 0 4Ò 4 with 4 ô ! , under the above
restrictions on Π.
Proof. From the target accuracy in R EG S OLVE, A PPLY, and E VAL R HS
¥ it is€ immediate
€
that the conditions (2.5) and (2.6) of Lemma 2.3 are satisfied for ]Aú and b*f — ­ ú Š . The
algorithm terminates if and only if (2.8) is satisfied. Therefore we infer from Lemma 2.3 the
validity of (4.8), recalling that Π6
for the Tikhonov scheme.
As pointed out at the end of ” 3 t the version 3. becomes an SVD-projection when is
selfadjoint and positive definite. In this case the qualification is Œ 6¸ , i.e. the range of the
smoothness index Πthat can be exploited is now unbounded.
Z
Of course, in principle, the projection method is a familiar concept in regularization
theory. But, to our knowledge, it has not been implemented yet in the present context with
a rigorous error control since the eigenvalues and eigenspaces are in general not numerically
ETNA
Kent State University
etna@mcs.kent.edu
82
W. DAHMEN AND M. JURGENS
”
accessible. By the techniques developed in [15] and 4.1 the scheme becomes for the first
time realizable in a fairly general setting.
A rough comparison already indicates at this point that, aside from the fact that higher
order regularity can be exploited better, the projection method is computationally more efficient than the Tikhonov scheme. In our setting, for some fixed positive
the algorithm
Y7
A PPLY(
) will be called for some 7‡†
until the discrepancy principle is met.
Each call will need elliptic subproblems to be solved within a prescribed tolerance where
we observed to beA well below ˆ in our experiments. In contrast, the Tikhonov method re87
quires severalA calls of S OLVE(
). Any linear solver applied to this problem
Y7 4 ) once per step and will need several
will perform an application of A PPLY( steps to solve the linear problem within the desired tolerance. In fact, it will need more and
Y7
Y7 4 need
more steps as decreases. As A PPLY(
) and A PPLY( at least for small roughly the same amount of work the projection method is significantly
faster than the Tikhonov scheme.
Of course, it is crucial to come up with suitable curves separating the spectrum according to the needs of the discrepancy principle. Moreover, Assumption 1 needs to be validated.
These issues will be addressed in 6 for a simple test case.
"*2® ÉÊn A4^ º
2’5,
»
«˜ x 0 0 0 ^“º
®2 n¼ A
"*2® É > An^“º
˜ 2
2® HH A É
”
5. Realization of Computational Routines. So far we have left open how to realize the
algorithmic building blocks S OLVE, A PPLY as well as E VAL numerically. We shall address
this issue now for the canonical choice
(e.g.
). The main tool will be a suitably
chosen wavelet basis. This leads to computable routines with guaranteed error bounds and
some information on the computational complexity of S OLVE for (4.3) and (4.4).
Moreover, as an interesting byproduct, the fact that
9 wavelet bases allow us to characterize
by simple rescaling, a new regularization method which measures the
the spaces ;
residual
in the norm of ;
,
, suggests itself that will later be
compared with the above versions.
Kê
é ™s)*
01] º— "½^‡º
é {)* "
t
% ' )*
¤
¤
t
5.1. Wavelets for Operator Equations. We shall not give any details on specific realizations of wavelet bases but collect only their relevant properties. How to realize them
has been discussed in the literature and we refer to references given e.g. in [7, 4]. As for
Œ‹
some notational conventions, a wavelet basis will be denoted by ‰
Š
‹
where is a countable index set whose elements typically encode the scale, the spatial location and the type of the i wavelets
Š . The wavelets will always be assumed to be local,
i
i.e. 
, where
denotes the scale, i.e. the dyadic refinement level on
Š
O
which the wavelet lives. Our first requirement is that ‰ be a Riesz basis for . This means
VŠ Ž Š
that every
has a unique expansion
which is stable in the fol|
t

lowing sense. Denoting
by
the
array
of
wavelet
coefficients
of , there exists
|
E

bounded constants
such that
ƒ…cU³7 ¥4¦§X§ w ö a½ê
Ô
y w = u a
w
© uF©
9 w
ö w ‰ ðö w ò w
ê
z
ö
! ö w“4w ‰
Õ ° 5g
Õ  €  €{z ¤7€CöA€ ô ¤ °  €  €hz ö a}ê@Z
(5.1)
Q.|V
Q.|V
r
r
u
It is well known thatø this implies
the existence
ofö a dual Riesz basis ‘‰ y Š Ž w = a‹ z _ê
ö
ø
ê
Ž
E
w
w
w
w
w
w
Š
ò » œ , so that
ð Š ò , and one has the alternate expansion
for , i.e. ð Š
t w ‰ | ðeŠ w ö ò Š Ž w , see e.g. [5]. Viewing ‰ ‘‰ as column vectors with respect to some
ö ‘‰ ò , 𑉠ö ò are the arrays of wavelet coefficients in the first
fixed ordering of the
indices, ð Ÿ
ö
representation of viewed as row and column vectors, respectively. The above expansions
can then conveniently be rewritten as
ö ð ö ő‰ òC‰ ð ö Å
‰ ò ‘‰ ETNA
Kent State University
etna@mcs.kent.edu
83
ERROR CONTROLLED REGULARIZATION BY PROJECTION
where we formally view the expansions as “inner products” of the coefficient arrays with the
bases. As a default we shall view sequences as column vectors.
Moreover, a duality argument shows that in these terms
€ ö €hz ¤7€®öM€ ô ¤ Õ  € ðp‰ ö ò €hz ö aàê@Z
Qj|V
t
° t  pð ‰ ò r Qj|V
r
For ê£_%Â'Us)* and a wide range of domains one can construct ‰ and ‘‰
simultaneously
in a way that the primal as well as dual wavelets are compactly supported. Moreover, these
constructions actually offer more, namely that properly scaled versions of ‰ and ‘‰ form
Riesz bases for certain smoothness spaces ,
, respectively, appearing in (2.16). More
precisely, for
from (2.16), we shall assume (in agreement with all known constructions)
from now on that for the positive diagonal matrix
é é¼ë
é
(5.2)
one has
’
diag
ö a 镔—–
’€
a ‹ z Y w =š_³ c “ y € Š w C€ ÷ z yYw =u 
t
a™˜ ' š‹Ÿ®
where 
𑉠ö ò i.e.
ö 
P ‰ and that the norm equivalence
Õ ÷ € ’™ €hz ¤ª€®öM€ ÷ ¤ ° ÷ € ’™ €{z ö  ‰ a é Q.|V
Q.|/V
P
r ÷ 5G
r
Õ
÷
°
holds with constants
. Again it follows from biorthogonality and duality that
É÷9 € 9 É €hz ¤ª€®öM€C÷Dø1¤GÕ ÷9 É € 9 É €hz ö é ëZ
° ’ ‘ Qj|/V
’
‘
‘ P ‘‰ a
Q.|/V
r
r
É
é
éìë
In these terms the rescaled bases ’ 9 ‰ and ’›‘‰ are Riesz bases of and , respectively.
ß
é
é
é
®
;
s
*
)
;
*
)
2
v
for Sobolev spaces
ori w i ,
. In particular,
é This
éhas É been
)* therealized
U
•
Ô
for
scaling weights are of the order Y w . This suggests
with
é and é~working
ë . However,
Ô spaces
the rescaled bases exclusively, if one is only interested in the
é
é ë
we are going to employ wavelet representations of operators from onto
even from
é è̐Â? é 3ÌM ë for certain ê é 3ÌMI+ é ) as well as of endomorphisms of(and
ê . Recording
(5.3)
the dependence on ’ (and further scalings) explicitly will enable us to adapt to the relevant
topologies. Therefore, expansion coefficients will always relate to the -normalized bases.
With these preparations we can transform now any variational problem of the form: Find
' such that for
ê
àa
(5.4)
Jàa éìë
ð ö ' ò ð ö  J ò ö a é into an equivalent one in the wavelet coordinate domain. To this end, it will be convenient
to define for any two countable collections œ 8 of functions, ordered in an unspecified but
fixed way, the corresponding generalized Gramian by
ðpœ 8  ò =šŸž ð , òY¡¢ ‰W£1œ ¤C‰W¥ i.e. the first entry is formally viewed as a column vector while the second one acts as a row
vector which explains the subsequent formal manipulations. Using the scaled basis functions
as test functions in (5.4), shows that the original variational problem (5.4) is equivalent to the
infinite-dimensional system
É
É
ðe’ 9 ‰ ' ò ðe’ 9 ‰ ¢ J ò ETNA
Kent State University
etna@mcs.kent.edu
84
9É
‰
as ’
yields
is complete in
(5.5)
±
é
W. DAHMEN AND M. JURGENS
’
ðp² '—²’ò
§¦ P ‰ into the previous equation,
9 É ¨ð ‰ '©‰Åò ’ 9 É ’ ¦1@ ’ 9 É ¨ð ‰ ¢J ò Z
ª
«{¬
­
ª «{¬ ­
ÿ®
¯
ÿ°
¯
. Inserting the representation
is called the (standard) wavelet representation of ' with respect to ² . It is
well known that, when ' satisfies a relation like (2.17) and the basis satisfies (5.3), this latter
problem is now well-posed in ˜ š‹ , see e.g. [2]. The relevant properties may be summarized
as follows.
R EMARK 5. For every bounded linear operator '
we have
' Ÿ
€’ 9 ɱ
=é ? é ë
'
° ÷ € ' €C› Q ÷ œ ÷Dø V Z
a€³ì é}ë é exists, we have
÷ ' €  h€ z Q.|V  a ˜N'U¨‹·CZ
+ °
9Ér é
9 É €N› Q z Q.|V œ z Q.|/V3V ¤
É
Õ r r
If (2.17) holds for ' with constants + ° + , i.e. ' 9
Õ + Õ '÷ €  €hz ¤ª€ ’ 9 É ± ’ 9 É  €{z ¤ °
(5.6)
Q.|/V
Q.|V
r
r
‰ for
i.e. the representation of ' with respect to the scaled basis ’
continuous endomorphism ' =ê£?Ùê the estimates
€ 𨉠'©‰
ò €C› z œ z
¤ ' € C€ ›
Q Q.|/V Qj|/V3V °  ' Q ô œ ô V
r r
and
€ 𠑉 ' ő‰ ò €C› z œ z
¤gÕ ' € €C›
Q .Q |V jQ |/V3V 9 ' Q ô œ ô V
r r
’
. Likewise, for every
are valid.
The property (5.6) provides one of the main foundations of the recent adaptive paradigm
developed in [2]. In fact, it suggests that one can find simple preconditioners ´©µ such that
the idealized iteration
ú Ø É ’ ¦ ú ´Rµ¶’ 9 É š·I" ± ¦ ú C
NZBZNZC
t
converges to the ß solution GŸ¦ P ‰ of (5.5) for any initial guess ¦ with some fixed error
reduction rate !
per step. In fact, for symmetric positive definite ' (implying that also
±
t definite) the choice ´ µ K˜ ¸ would work forÉ a± suitable
is symmetric positive
É damping
parameter depending on the constants in (5.6). In general, ´ µ =š ˜ ’ 9
’ 9 P would in
(5.7)
’ ¦
principle do the job although the corresponding squaring of condition numbers might degrade
the quantitative performance.
For a numerical realization of (5.7), one has to approximate for any finitely supported
±
input vector  the infinite-dimensional
vector ’
 by some finitely supported ¹ that
±
˜
š
‹
 in
approximates the exact sequence ’
within suitable dynamically updated tol±
’
erances. In fact, one actually applies compressed versions of ’
to scaled vectors
’€ . These perturbations of the ideal iteration (5.7) can be shown to provide an adaptive
solution scheme
9É
Solve( ’
± 87C· ) ?º¦
9É
9É
a
5G an approximate solution ¦
€ ’ š ¦à"»¦ €hz ¤ 7A
Qj|/V
a
r
that outputs for any target accuracy 7
(5.8)
9É
' ·
a of
›·
± ¦
such that
ETNA
Kent State University
etna@mcs.kent.edu
85
ERROR CONTROLLED REGULARIZATION BY PROJECTION
€ 
" €÷ ¤ ° ÷
9É 9É
é ö
w
7 .
which, in view of (5.3), means that
a
Moreover, for operators ' considered here one can contrive efficient schemes for the
±
’
approximate application of ’
so that the algorithm Solve can be shown to exhibit
an overall asymptotically optimal work/accuracy balance in the following sense. To this end,
let us denote for
by ¼>½
¼X½
‰
the class of those functions
such that
the error G?
of best -term approximation of in , i.e., the accuracy obtainable by
½ . It is not hard to see that
a linear
combination of at A most
wavelets Š , decays like
¾¿
A
A
defines
a quasi-norm for ¼>½ . Moreover,
when
for
½ À?
?
A
some
, the space
, where
¼>½ is almost characterized by the Besov space '
½
, see [2] for more details.
an element
¼R½ can be recovered within
8Á
YÁ Thus,
¾ ¿½
accuracy 7 using only the order of 7
terms. It is important to note that, on account
½
of (5.1), an element
belongs to ¼X½
if and only if its scaled wavelet
 ‰
‰
½ by the order
coefficient array ’™ can also be approximated in ˜ š‹
within accuracy
˜ š‹
A 7
of
terms.YÁ Equivalently,
approximating
in
within
a
tolerance
, takes the
€
’

8Á
¾ ¿½
7
A
order of
terms. Thus, in brief, (near-)best -term approximation in is directly
½
linked to (near-)best -term approximation in ˜ š‹ . A Clearly, best -term approximation in
˜ ¨‹
A very simple since it boils down to retaining the
A
is conceptually
largest (in absolute
A
value) terms of ’€ .
Now the main complexity estimate from [2] states that whenever the solution of (5.4)
± Y7 8·
belongs to ¼>½ for some
, then the computational work required by Solve ’
to
output an 7 -accurate approximation ¹ a of ¦ and hence, in particular,
the
number
of
nonzero
8Á
YÁ
¾ ¿½
½
terms in ¹ a , is bounded uniformly by a constant multiple of 7
. Moreover, when
it has been shown in [17] that the optimality
using spline wavelets of exactness order Â
Â
range covers the highest possible order, i.e.
. For this optimal complexity
estimate it is crucial to have a relation like (5.6) which requires controlling the constants in
(5.3) and those in a mapping property like (2.17).
We shall apply these concepts next to equations (5.4) for the choices (4.3) and (4.4).
5&
­
÷
ö
⠜
© ö © =š ¥4¦§ ‰™û B⠜ ÷ ö 2*5
WÅ ­ A9 É © ö © É
t{•
t™•Ô
ö é
P a
9 É © ö © É
­
9
é é ;®s)*
S3; Ø . % S s)*4
ö a
é {'U Ÿ
Ê' Ÿ
9
H5j
é
Ê' ·
N'U ·
ß
ö a é
é
­
a
M9 É © ö © É
­ 5
A {
"_2 Y
•
5.2. Solution of the Resolvent Equations. To apply the wavelet technique to the resolvent equation
(5.9)
èÌM«¬"6 ö J
Ì}a!x®
we will start from the following weak formulation: For any given
that
(5.10)
where
É
ð 3ÌMÓF ö ò ð Ì 9 J ö ò
for all
JÎa éë , find ~a é
such
ö a é 3ÌM(=š_«"ìÌ 9 É >Z
éë , equation (5.10) has a unique
Here and in the sequel we assume Ì}a!A6 , i.e. for any Jìa
solution continuously depending on J . In particular, the norm equivalence
Õ €®öM€C÷,¤ª€ 3ÌM öA€N÷Âø1¤ °
€®öM€C÷ ìa é Z
(5.11)
<QI#™V
<QI#™V
Õ
holds for some constants <QI#™V ° <QI#™V 5½ that may depend on Ì . The corresponding wavelet
representations is denoted by à è̐ ¨ð ‰ è̐ ‰}ò . In principle, the comments from the
previous section imply that, because of (5.11), the scheme Solve with the scaling ’ would
indeed work for each fixed Ì in the resolvent set with optimal complexity. However, we
ETNA
Kent State University
etna@mcs.kent.edu
86
©Ì ©
°3ÌM< Q ™V Õ <Q ™V
•
3ÌM
W. DAHMEN AND M. JURGENS
Ì
expect the ratios
I#
I# to become large when gets close to the spectrum. Moreover,
when
grows
acts more and more like the identity on lower scales so that the mapping
properties of
agree less and less with the topology of . Thus the complexity bounds
deteriorate due to the increasing ratio
I#
I# .
This problem can be ameliorated by introducing appropriate -dependent norms on .
The lower constant I# will always depend on the distance of from the spectrum. To
control this will be a matter of the choice of the curve . So all one can hope for in addition
is to reduce (or even avoid) the dependence of such stability constants on the size . To this
end, we introduce next different norms on and set
Õ
(5.12)
(with
Yw
’™Ä
é
Õ
° <Q ™V < Q ™V
•
<Q ™V
É
Ì
é
Ì
©Ì ©
é
 ƒ ceY:w3ÌMn4w ‰ | &
Y“wX3ÌM(=š½³ c “ y © Ì © 9 YÉ Á' “Y w z t
é
from (5.2)) and endow
with the norm
€®öM€ ÷
€
{€ z
ö
QI#™V =š ’ÅÄs .Q |/V  P ‰ Z
r
Clearly, by duality
€ 4 € ÷ ø € ’ 9 É ð¨‰ 4’ò €hz Z
IQ #™V
Qj|/V
Ä
r
Õ÷
÷
é
The analog to (5.3) holds for è̐ trivially with constants QI#™V ° QI#™V addition a mapping property
ÕUƒ €CöA€ ÷
t
. If we had in
¤7€ 3ÌM öA€ ÷ ø ¤ ° ƒ €®öM€ ÷ ö a é è̐®
QI#™V
QI#™V
©
©
with constants independent of Ì , we could conclude according to Remark 5 that
9É
9É
(5.13)
ÆE( d  'U ’ Ä Ã è̐ ’ Ä O
t
which is needed for Solve
to perform with asymptotically optimal complexity. To this end, it
ö
is easy to see that for  P ‰
€ 3ÌM öA€ ÷ ø € ’ 9 É ð¨‰ niè̐ ö ò {€ z ¤ ° € ’ Ä  h€ z ° €®öM€ ÷ QI#™V
Q.|/V
Qj|V
QI#™V
Ä
r
r
where the constant ° depends only on the constants in (5.1) and (5.3) and is thus independent
é
of Ì . Hence è̐ is uniformly bounded in É è̐ for Ì É in the resolvent set. This means
9
9
that the upper estimate in (5.6) holds for ’ Ä © à © è̐ ’ Ä with a constant independent of Ì .
Ì
The validity of a lower bound independent of
seems less clear. However, computational
Q)#{V
evidence shows clearly that scaling by ’€Ä is by far superior to scaling by ’ .
Thus we shall employ Solve with respect to the scaling ’ Ä . To bound the error with
respect to
independent of we shall make frequent use of the following simple facts.
R EMARK 6. Note that for all the above scalings one has ’
, ’™Ä
, for
all ‹ . This implies that for 4
¹
‰
u a
€ ñ€ô
Ì
P
w œw Ö
t
€ ’ 9 É ¹ h€ z ¤ª€ ¹ €hz ¹ a€˜N'U¨‹·C
.Q |/V
Q.|V
r
r
which are the discrete analogues of the embedding inequalities
€ 4 € ô ¤,€ 4 €C÷ € 4 € ô ¤,€ 4 € ÷ 4 a é Z
IQ #™V
We are now prepared to formulate a realization of S OLVE(ÌM«’"Y7A4M ).
A LGORITHM 3. w 5 S OLVE 3ÌA«" , 7 , AÊ
ZǦ 5 ¨ð ‰ 4 ò
(5.14)
4w œ w Ö
t
ETNA
Kent State University
etna@mcs.kent.edu
87
ERROR CONTROLLED REGULARIZATION BY PROJECTION
Z
¹È5 Solve( ’ #
465…¹
‰
Z
P
à è̐® °  9 É 7C¦R )
€ 3ÌA«"½x®9 É "
€ô ¤
7 is met thanks to the accuracy of Solve,
The requirement
4
c.f. (5.8), the discrete embedding inequality (5.14) and the right inequality in (5.1).
Concerning the specification of the scheme Solve in the above Algorithm 3, we can
refer directly to [2] since the underlying equivalent formulation of the resolvent equation
in wavelet coordinates is straightforward as given above. Moreover, the fast approximate
application of Ã
as well as the approximation of the data ¦ follow here standard lines.
One such ingredient is the approximation of given data in terms of wavelet coefficient series
by sequences with shorter support. These approximations can be obtained (after appropriate
7 
preprocessing of the data) with the aid of a coarsening routine
 a that
{z Coarse
7
approximates É by a finitely supported vector such that   a
and  a has (nearly)
.|
minimal support, see [2] for details concerning this routine. The routine E VAL appearing in
Algorithm 2 looks as follows:
A LGORITHM 4. w 5 E VAL 7 ,
è̐
€ "
Z

Z
¹
Z
4
ö
𑉠ö ò , 7A 5 Coarse  ,
5º¹ P ‰ .
€
r
Q /V
A }?
¤
5
The realization of S OLVE also for (4.3), i.e. in the context of the Tikhonov-type schemes
is more involved since the application of the concepts from [2] require
(a) the wavelet representation of the underlying operators (in this case '
or '
),
(b) their efficient application in wavelet coordinates as well as
(c) the less straightforward approximate evaluation of the right hand side in wavelet
coordinates.
Therefore we turn now to providing these necessary ingredients in the subsequent subsections.
˜ «¬ 0 0
˜1á>— 0 0
5.3. Application of (Projected) Semigroups in Wavelet Coordinates. The adaptive
wavelet schemes hinge on the wavelet representation of the involved operators and data.
Therefore we we shall derive next the wavelet representation of the projected semigroup and
of the semigroup itself. Since the resolvents (5.9) apply to elements in
which are naturally
expanded in terms of the dual basis ‘‰ we choose to represent
with respect to ‘‰ ,
i.e. we set
8:9;3<
(5.15)
Ê@Ë
324 𑉠n8 9E;3<
‘‰ ò thus
Ñ
ÌM«I"Å> An 324C A
Õ  7A  ‘‰ P
8 9E;3<
Ñ
éìë
Ñ
ö ª Ê@Ë 24  ‰ Z
P
To apply this operator to a given vector  we invoke Algorithm 1. As we have already
Y7
provided a realization of S OLVE(
), Algorithm 1 and consequently the following
algorithm is a computable scheme.
Y7  )
A LGORITHM 5. w 5 Apply(Ê@Ë
Z
Z
465
¹ 5
È
A PPLY
ðC² ‘ 4’ò
89;3<
Ñ
The formulation of Apply is in principle only a formality which could be dispensed with.
However, on one hand, it will be convenient in what follows and, on the other hand, reflects
how the A PPLY–routine actually works. In fact, since the resolvent equations are solved in the
ETNA
Kent State University
etna@mcs.kent.edu
88
W. DAHMEN AND M. JURGENS
œ ú 4 nY 4 generated in Algorithm
wavelet coordinate domain, the intermediate quantities 4
Q
Q Q
Q
1 are actually given in terms of their wavelet coefficient arrays.
Similarly to (5.15), the representation of
in wavelet coordinates is given by
8:9;3<
0 324D=š 𑉠n8 9E;3< ‘‰ ò Z
Ë
We note that approximate evaluation schemes
24®Í Ì Y7®
Apply Ë
(5.16)
can be constructed analogously based on the corresponding version of A PPLY. For a detailed
discussion and analysis of these evaluation schemes we refer to [15, 16, 11]. Alternatively, in
͏ Ì Y7 with the aid of Algorithm
view of Remark 2, we could realize the scheme Apply Ë
5 combined with a proper choice of the curve . Therefore, we shall assume that the scheme
in (5.16) is given without further elaborating on its specific features.
324C É
Schemes. We shall now provide computable schemes for S OLVE
˜ «>5.4.
0 Tikhonov-Type
To this end, we will apply
0 Y7A 0 ^ and SOLVE ˜1ᗠ0 0 87 0 ^‡ , see «>(4.3).
˜ê 0 0 and ' ˜1á — 0 0 ,
the general solution concept of ” 5.1 to the operator '
respectively. Both are norm-automorphisms of the space such that it is sufficient to choose
’ ¸ , i.e. preconditioning is not necessary.
Recalling that is assumed to be normal, the (modified) Tikhonov regularization requires, in view of (3.3), solving the operator equation
˜á — H
8 9;+ ] _8 9E;3<* ^A
(5.17)
'
=ó Concerning an equivalent formulation in wavelet coordinates, recall that the resolvent application suggested to represent the evolution operators in the dual basis ‘‰ , i.e.
— 𠑉 á>— ‘‰ ò Î
á—
Ë
+
324 𠑉 n8 9 ;+ ő‰ ò ˜
where
either denotes the identity or a projection, depending on the size of . Here we use
the superscript ' to distinguish Ë +
from the representation Ë
of the original semigroup
. If we were content with representing the solution in dual coordinates
‘ ‘‰ ,
(5.17) is equivalent to
89E;3<
324
ž
(5.18)
˜ Î — Ë
+
324 ¡ ‘Ï ðp‰ 8 9E;3<* ^ ò ]
324
Ë
] ]P
324 ðp‰ n ^ ò Z
But representing the solution in dual coordinates has several drawbacks. In practical constructions the primal basis ‰ is typically piecewise polynomial and comfortable routines are
available for further manipulating or processing the approximate solutions. The dual basis
‘‰ is typically less regular and less convenient to handle. Therefore, we indicate next briefly
how to switch representations with the aid of numerically executable Riesz maps. Clearly,
expanding the Š in terms of the ‘Š , yields
w
w
𨉠‰Åò ‘‰ i.e. ¸ ðp‰ Å
‘‰ ò Z
‰ òC𠑉 Thus, defining the Gramian Р𨉠‰
ò , we have
 Ì P ‰ ª Ð  Ì P ‘‰ ‰
i.e. application of the Gramian to primal coordinates yields the dual coordinates. Likewise,
p‰
‘‰ , we obtain for any endomorphism ' of
since ¸
ð Åò
(5.19)
ê
‘‰ òN𨉠Å
‘‰ ò8Ð Z
𨉠'©‰Ÿò 𨉠Å
‰ òC𠑉 ' Ÿ
‰ ò 
Р𠑉 ' ETNA
Kent State University
etna@mcs.kent.edu
89
ERROR CONTROLLED REGULARIZATION BY PROJECTION
] aÅê
P
in primal coordinates we obtain,
If, as argued above, we wish to represent
Ï ‰
on account of (5.19), the alternate equivalent formulation
(5.20)
Ð
ž
˜ Î — +
Ë
324 ¡Ð
Ï
324 ðp‰ 4 ^ ò Z
Ð Ë
(5.18) would save two applications of Ð per iteration step in comparison with (5.20). Of
course, one could transform at the end ϑ into primal coordinates by applying the inverse
‘‰
‘‰ . But the latter one is usually less compressible rendering its application less
Ð
efficient. Therefore we concentrate in what follows on (5.20).
We wish to solve the system (5.20) within any desired target accuracy with the aid of
the scheme Solve following the lines of [2]. To invoke the concepts developed there we need
algorithms to apply the involved operators up to any prescribed accuracy. We provide these
ingredients in the following subsections. The above representations involve, in particular, the
(infinite) matrix Ð . The vanishing moment property of wavelets allows one to apply Ð to any
finitely supported vector within any prescribed accuracy tolerance by the schemes developed
in [2, 1]. Moreover, within a certain basis dependent range such an application has asymptotically optimal complexity in the sense described earlier. Such a multiplication scheme is also
¹ a
a central building block in the scheme Solve and will be denoted
by Ñ ¦KÒÓ Ð Y7 
{z
7 .
¹ a
providing an approximation ¹ a to Ѐ such that Ѐ
.|
It should be noted that the following building blocks are all acting in the wavelet domain.
± Y7
 ) itself operates in the wavelet domain.
This is necessary since the scheme Solve(’
Once all ingredients are gathered we shall formulate an algorithm acting in the continuous
domain, see Algorithm 9 at the end of 5.4.3.
9 É ð ò
€
€
"
A
r
A I?
Q /V
¤
”
5.4.1. The Right Hand Side Ô . Then the right hand side of (5.20) has the representation
Ô
9;3<* ‘‰ òÉ ðp‰ 8 9E;3<* ^ ò ¨ð ‰ Å
‰ òM𠑉 8
24 É Z
Ð Ë
Again since É has possibly infinitely many non-vanishing entries the following scheme starts
formally with a coarsening step.
A LGORITHM 6. w 5 EvalRhs(7 , É )
Z
¹
'
Z
Z
¹
¹
É
“É É ,
Apply Ë 24®Y7'U ¹ ÉN ,
Mult Ð Y 7 ] ¹ ' ,
:ÉI
7U'x
Coarse 7
5
5
5
L EMMA 5.1. The output ¹
7
7 ]
of EvalRhs e7 É
€ 𨉠n8 9E;3<* ‰
òlÉ "
¹
]8^
ro
]8^ a Ö l;×
a] . Ö
r
r
w Õ ØjÖ Ù *
W
opÚWÛ)ÜÝ
7 ;
satisfies
€hz
r
Qj|/V
¤ A7 Z
Proof. Repeated application of the triangle inequality yields
€ ¨ð ‰ n8 9 ;3<* ^ ò " ¹ €hz
Q.|@V
¤7€ Ð C€ › z
ž € Ë r 324 €C› z
7]Z
Q Q.|/V3V
Q Q.|V3V ñ 7“ÉÞ7' ¡ ß
r ¤ '
€ €C› z
€ r €C› z
¤ªÕ 9 ' € 8 9E;3<* C€ › ô , the choice
As, by Remark 5, Ð
and Ë 324 Q Q.|/V3V
°
Q
jQ |/V3V
Q V
of 7:É , 7U' , and 7 ] ensures that
r the error is bound by 7 . r
ETNA
Kent State University
etna@mcs.kent.edu
90
W. DAHMEN AND M. JURGENS
˜ 8‡57
9E;3< 8U9;3<½Ž8Ê;
gH
¡
˜
uy ú z ú | É
w
¡
¡
â 8 E9 ;+ ã X ˜ ž y záà y 8 9E; = Ö z ¡ ã X ˜
y záà y 8 9E; w = u ú Ö t t2 ².( e t z Z
˜
By the results of ” 3 we are capable to evaluate projections onto the eigenspaces associated
á — ß É «àe" É — z is used where
to any bounded subset of â ' . Thus, the representation
u
u
=
É
—
'
ú
and the Jordan curve encloses y ú
; .² ( — (cf. ” 3). The
Ñ of á — with respect to ‰ becomes
representation
¨ð ‰ á>G
— ‰
ò Ð " Ð Ê>G— Ð where Ê>— Ê@Ë is the projection in wavelet coordinates and can be evaluated by Apply
:®87 the orthogonal projec5.4.2. The Projection. Given a regularization parameter
*
+ , in
tion
onto the eigenspace associated with the eigenvalues of
is to be evaluated. Denote by
the eigenvalues of '
sorted by increasing
size,
. Due to the spectral mapping theorem 3.1 we have
á —
u ú ¤ u úØ É
Ê@Ë
 , see Algorithm 5. We do not define a Apply–routine for this operator, but use
this representation in the upcoming Algorithm 8.
8:9E;3< N89E;3<Žk89E;
”
ð Ê ˜ «6H8 9E; Åò ê ˜ 3244
ð B ˜ «’8:9E; D Åò A
Z ¹ É 5 Mult Ð 87:ə  ,
7:ÉI ]8^
7 ;
wWÕ rÖ Øjã
É
Z ¹ ' 5 Apply Ë +24®Y 7'U ¹ ÉN ,
7U'x ]8^ Ö r ol7 × ; opÚWÛIÜÝ
É
Z ¹
7 ] ] 7 Ö .r
5 Mult Ð 8 7 ] ˜  ¹ 'Ê ,
Clearly, in the third step we invoke the scheme Apply Ë +324C  Ì Y 7 from (5.16) for the
< *nV ‘‰ ò . An analogous reasoning as in the proof of Lemma 5.1
operator Ë +x324@ 𠑉 89E;žQT<Mؐ
yields the following estimate.
L EMMA 5.2. The output ¹ of Apply ¨ð ‰ Ê ˜ «’H8:9E; +D ‰
ò 8 7A  satisfies
€ ¨ð ‰ B ˜ «’8 9; + ‰Åò " ¹ h€ z ¤ 7AZ
jQ |V
r
If the identity is replaced by a projection ái— «" — we use an algorithm based on the
*
+ . The observations in 5.4, suggest the fol5.4.3. The Operator
+
lowing algorithm to evaluate ¨‰
‰ l
Ð K¸
Ë +
Ѐ , where  are here
the expansion coefficients with respect to the basis ‰ of .
+ ‰ 87  )
A LGORITHM 7. ¹â5 Apply( ¨‰
decomposition
Ê
8 9E; + ‰
ò Œä ˜ «6 Ð ž Ë + 241" ˜Ê — ¡{åÐ Z
ðp‰ B 1˜ á — H
in the previous section.
— is chosen depending on ˜ as discussed
A LGORITHM 8. ¹â5 Apply( ¨ð ‰ B ˜á — H8 9E;+ ‰Åò 87A  )
Õ
Z ¹ É 5 Mult Ð 8 7 É  ,
7 É æ
^ — Ø rÖ wWØjã
É
o ÚWÛ.ÜÝ
Z ¹ ' 5 Apply Ë +24®Y 7 ' ¹ É ,
7 ' æ ^ Ö r 7 ; ol×
É
Z ¹ ] 5 Apply 𠑉 — 7 ] æ ^ Ör 7 ;
‘‰ ò 8 7 ] ¹ É ,
—É
Z ¹
7 æ æ 7 . Ör
5 Mult Ð 8 7 æ ˜  " ¹ ] R ¹ ' ,
A similar proof as in Lemma 5.1 yields the following estimate.
+ ‰ 87 
L EMMA 5.3. The output ¹ of Apply ¨‰
ð
€ 𨉠B ˜á— 8 9;+
Ê ˜1ᗠ89E; D ò A ‰Åò " ¹ h€ z jQ |V ¤ 7AZ
r
satisfies
7 ;
ETNA
Kent State University
etna@mcs.kent.edu
91
ERROR CONTROLLED REGULARIZATION BY PROJECTION
We have prepared now all the algorithmic ingredients needed to apply the scheme Solve
from [2] to the operator equation (5.17) which will return an approximate solution a that
approximates with accuracy 7 . Thus we have the following implementation of the abstract
87
routine S OLVE
.
Y7
A LGORITHM 9. w 5 S OLVE
]
]
Z
Z
Z
Z
˜1ᗠ0 0 A 0 ^ ˜á—
n
^
Ɍ5 ðp‰
ò É Õ '  9 É 7A °
É
Ôç5 EvalRhs ' ˜
¹Œ5 Solve(¸ Ð ˜ Q Ë +244 Ð
4Œ5…¹ P ‰
0 0 A 0 ^
'É °  9 É 7 Ô )
5.5. Regularization of the Transformed Problem. So far we have first regularized
the continuous problem and then transformed it into a well-posed problem in ˜ š‹ (whose
condition, of course, depends on the regularization parameter). In this subsection, we will
exchange the order of regularization and transformation. We shall exemplify this briefly only
for the Tikhonov method, the modified version being similar. This gives rise to a scheme
where norms are easily changed by scaling offering an additional way of influencing the
regularization compromise. Accordingly and are now different smoothness spaces and
no longer coincide with .
We consider the problem
' Ÿ
' posed in ˜ ¨‹
[
ê
Ë ;
324 Ï É
with the evolution operator given in wavelet9 coordinates as
a " ¡
t t
where now ] Ï P ’ 9s; ‘‰ . The power of the diagonal elements of ’ steers the involved
norms. For instance, when is a second order elliptic operator, Ë 24i Ë 324 is a repre èU'Us)*é
? è':)*xj[ . In
sentation of the operator 8:9;3< regarded as a mapping ê¶
é
é
;
*
)
;
general, for
, it is well-known that the scaling by ’
induces equivalent norms
é
é
on the interpolation spaces between 9;¢s)* and ; s)* , see e.g. [5]. Let us assume here the
standard case 2
.
t
As 89;3< is compact
the transformed operator has the same property as endomorphism
of ˜N'Uš ‹· and the resulting sequence equation is ill-posed, as well. Applying Tikhonovregularization in ˜B'U¨ ‹· yields the equation
˜ «6 Ë ; 324 Ë ; 244 Ï º— Ë ; 324 É º where ¸ denotes again the identity in ˜Ê'U¨ ‹ and not the Gramian matrix. É º and Ï º— are given
Ë ;
by
É
24
º ;
’
’
;
𑉠8 9 9 ;3< Ÿ‘‰ òY’ ; 𨉠4 ^ º ò Ï
º— ’
9s; ðp‰ ] º— ò Z
The regularized equation is equivalent to the solution of the minimization problem
ê
;
˜ É º > ˜ € Ï º— € z' Q.|/V € Ë ; 24 Ï º— " É º € z' Q.|V
˜ € ’ 9 ; ¨ðr ‰ ] º— ò € z' jQ |/V € ’ ; pð ‰ r8 9E;3< ] º— "¼^ º ò € z' .Q |/V
? ³z ƒ…d Z
r
r
ë ìÇ ‰ .Q |V
r
ETNA
Kent State University
etna@mcs.kent.edu
92
W. DAHMEN AND M. JURGENS
By the norm equivalences of the wavelet basis one sees that
€ º €{z O € ] º € ÷ wÍí and € É º €{z O € ^ 9 º € ÷ í Q.|/V
Q)îEV
Q.|/V
Q)îEV
—
Ï —
r
r
é
é
9s;{s)* , [î ;{s)* . Thus, the smaller is the stronger the norm
i.e. we take here 9
9
becomes in which smoothness of ] º— is measured. In turn, the norm in which
¡ the residual is
measured becomes weaker for smaller . In practice, the range a "  is of interest as
t
the solution ] º— should be smooth.
?
‡
Ø , the properties of
Concerning the convergence of the regularization scheme for »
Tikhonov regularization read as follows. Suppose that
^8 9;3< ] h with ] h a é 9s; s)*
and
€ ^>"^ º € ÷ í ¤ » Z
)Q îEV
Then,
if
˜ ˜ » n^‡ºN
€] º " ] h€÷
—
Q XV ?\
wÍí )î
for
» ?\ Ø is chosen properly, e.g. according to the discrepancy principle.
5.6. Some Comments on Computational Complexity. Not much seems to be known
about the actual computational cost of determining the numerical approximation
for the
regularized solution
depending on the given noise level as tends to zero. We address
this question now in a somewhat sketchy and partly heuristic way for the simplest scenario
that is a symmetric self adjoint second order elliptic operator so that
,
. Hence the singular values of
are for fixed
given by
,
where
are the eigenvalues of . A detailed rigorous study would be beyond the scope of
the present paper and will be given elsewhere. Here we merely wish to bring out what the
approach might offer at best and to identify some of the questions along that way.
Let ï denote a suitable Jordan curve whose interior contains the first ð eigenvalues
ð , and let Íñ be again the projector defined in (3.1). As pointed out
ï
with
, corresponds to a truncated SVD
before,
ï
ï
ï
expansion. Since by assumption we can invoke [9, Theorem 3.26] to
ï
conclude that
é 9 É )*
uú
] º—
É
u ú N ZBZNZC
] t º =šj8 ;3< Ñ ^ º
»
âú
0 _89E;3<
»
] È º—
é é É )* é}ë w
2D5½
â ú 89E;
š= 8 9;3< Ñ 8 9E;3< £8 9E;3<
€ Ñ 3 ^"~^“ºB € ô ¤ °’»
€ ] h " ] º € ô ¤ ° ë â Ø É »
ï
ï
âï u
ë
for some constant ° independent of » ð . Having some estimates for the ú and hence for
the âEú , a reasonable parameter choice is to balance both terms on the right hand side of (5.21).
This suggests choosing
Ö »zZ
ð ð » _³ c“ y = â ú Ø É â ú
(5.21)
In order to retain the accuracy offered by this projection method we compute
] ƒ ºï ϐP ‰ Ï Ã™òò Ò É Ê@Ë Ó"*24® É º 87® with 7·=š » â ï Z
•
Recall that the latter routine is given in Algorithm 1. As before, we do not specify in which
form the noisy data ^‡º>aìê are actually given but assume for simplicity that we can observe
its wavelet coefficients É º for the expansion ^ºIª É ºÊ P ‘‰ . Note that by (5.1), we have then
€ É " É º h€ z ¤½Õ 9 É » Z
(5.22)
.Q |@V
r
ETNA
Kent State University
etna@mcs.kent.edu
93
ERROR CONTROLLED REGULARIZATION BY PROJECTION
9
In this scheme the computational cost is then dominated by the approximate applications
,
,
, on the curve
of resolvents at quadrature points
Q
A Q by the A quadrature at the
Q overall
Q
ï . By a geometric series argument the
work isA dominated
D
D
terminating
level of quadrature resolution. For that the schemeD 9 Solve is called
$# .;Ló m
times with target accuracies O 7
,
. Thus, theA most
D
A
stringent
accuracy constraint is encountered when the real part of A
is of the order of
. Hence, a rough bound for the computational cost ô
of Ùòò Ò É Ê@Ë
É Y7 with
ï
7 ï
is given by
9
9
É
Ì/ œ ú %
Ôu
·=š » â
•
kXNZNZBZC
© 8 ; :Q
•
ï
V©
%
"
t
ÙXNZBZNZC
=ó
Ô
"
Ì/ œ ú t Ó"*24® ºU ’™ø í ó m ù ó w à 3Ì1 D œ ? ó 9 É 4C Õ » 4Ì1 D œ ? ó 9 É 9 É É º ¡ ï O
Û
Ô A
wñ ÑÝ
E
9
;
8
where we have used that â
(recall the scaling ’ Ä given
(5.12)). Here,
ß bymeans
ï
¨õöGÒ denotes the computational
cost of the routine Solve and
that can
(5.23)
ô
ô
ß
D
ô
žpõ¯ö`Ò `÷
G÷
ú
ú
Oüû
be bounded by a fixed multiple of û which is independent of the parameters on which ú and û
may depend.
It remains now to estimate and ô ¨õöGÒ G÷ for a target accuracy of the order . We
shall address this latter issue first. Recall that, according to (5.7), Solve endowed with the
’ Ä É which are the expansion
scaling ’™Ä operates on right hand side data of the form
coefficients with respect to the
-scaled basis ’€Äý‘‰ . Thus, the noisy input data É are
damped in two ways, first by
when
is large, and second, the entries
for
.(
are attenuated by the weights ’€Ä
, see (5.12). By (5.22) and (5.14) we also
know that
%
é è ̐ ë
1Ì 9 É
'É ² e ' © Ì ©
©Ì ©
»
/Ì 9 É 9 É º
º
© uF© Ö
^wº
w9 œ É w
€ Ì 9 É ’ 9 É É " É º €hz ¤gÕ 9 É »
Ä
.Q |@V
r
Ö
©
©
uniformly in Ì
.
t
To obtain an impression
of ô p õ¯ö`Ò `÷ in this context, suppose now that the exact wavelet
4
2
½
^ É P ‘‰ is sparse in the sense that
coefficient array É of
^·a ¼ ½ sê ‘‰ © © ¿
for some ­ 5g , i.e. the error of best -term approximation in ˜ ' ¨ ‹ decays as 9 ½ ^ ¾ Q ô œ$ÿ þ V .
é 3ÌM -basis ’ ø í ‘‰ A are given by
A ^
Since
Û Ý
9 É the expansion coefficients of with respect to the
(5.24)
’ ø í É , (5.14) implies that also
Û Ý
­ 5
9
Ì1 ^·a
É
¼ ½
é è̐ ë ‘‰ 9
for that
and any9 point
on Kï , uniformly in and ð . We shall explain later what
this means in terms of the regularity of . Now suppose further that the operator and its
shifted versions
permit the following “regularity” result:
9
A9 SSUMPTION 2. Given Ì
¼>½ 9
‘‰
for some
(with the fixed compressibility limit
depending on the wavelet bases (see [2, 17])) then the solution
of
9
Ì belongs to ¼ ½
‰ , i.e. sparseness is preserved under the inversion
of
.
Then, if (5.13) was true, the results
in [2] would assert that
9
èÌ/ n
3Ì1 n ö 3Ì1 n
^
àa é è̐ ë é 3Ì1 4® ­
ß
­
­
ö a é èÌ/ n
± 3Ì1 D œ ? ó 9 É 4C Õ » É Ì ¡ ß » 9 É8Á ½ © ö © ¾ÉYÁ ¿ ½ Z
O
9É
However, Solve “sees” only the perturbed data É Ì º}=š ’ ø í É º . We dispense here with in(5.25)
ô ž õ¯ö`Ò `÷
Û Ý
troducing a statistical model and try to construct an estimator
reflecting the sparseness of
ETNA
Kent State University
etna@mcs.kent.edu
94
W. DAHMEN AND M. JURGENS
the exact data with high probability or in expectation. Instead we resort to a more heuristic
reasoning why under the above assumptions one may still obtain a work count like (5.25).
This requires a somewhat closer look at the way Solve works, see [2]. First of all the data
É Ì can be considered as a finite array, since every coefficient É Ì with ’ ø í
can be
Û Ý
discarded which, in view of (5.22), introduces only an error of the order at most in ˜ š‹
for the scaled right hand side (which is what mattershz for Solve). Let us
hz call this finite array
again É Ì . Thus, by (5.24), we still have that É Ì
for some
ÉÌ
É
É
j|
.|
finite constant . Now, given some target accuracy for Solve, the right hand side data are
approximated first by a possibly small array within a tolerance ú where ú is typically smaller
than one. This approximation is done by the routine Coarse described e.g. in [2], see also
5.4.1. If we relax our target accuracy in Solve by some constant factor larger than one,
the coarsening of the right hand side data É Ì would have to be done with a target accuracy
where û is now any fixed number larger than one. But then the Coarsening Lemma from
û
[2] states that the resulting approximation É to É Ì now satisfies
8Á
YÁ
{z
ÿ
½ Ì ¾ ¿½
ÉÌ
ÉÌ
ÉÌ
º
ºw
º
€ " º€
¤ª€ " º €
Q /V
Q V
r
r
°
Õ»
”
°’»
w9 œ É w ¤ »
» Ê' Ÿ
¤ °’»
º
€ " €
¤
Q.|/V ß û °’» © ^ Ì © ¾¿ Q ÷ r QI#™V3V © ^ Ì t © ¾K¿ Q ÷ QI#™V3V Z
O
º
¥4¦X§§
O
ß
É É
» 9 © ^ © Q ÷ IQ #™V œ V »
Thus, using a fixed but sufficiently large multiple of as target accuracy in Solve the corresponding coarsening of the data that “peels” off at this level of accuracy the sparsity of the
exact array É Ì in the neighborhood. In that sense the routine Solve still “sees” É Ì and exploits
its compressibility giving the work count (5.25).
Let us briefly comment now on the above assumptions. We wish to recover the initial
data
‰ from the data . Suppose that
'R ½
where
Ï
½
. The space 'R½
is just embedded in
and signifies in some sense
the largest space of smoothness contained in
. Note that the larger the smaller and the weaker the measure for smoothness. It is well known that
'R ½
implies
X
¼
½
,
see
e.g.
[8].
Moreover,
it
is
known
that
then
belongs
Ï
9
also to '> ½
, see [16]. Assumption9 2 then says that the assumed gain in regularity
through the application of
suffices to9 ensure that the solution array with respect
to ’ ø í ‰ is still in ¼ ½ . For moderate
this is simply a regularity theorem for
O
Ý
itself. Û For
as above we would have
. Note that already
'R ½
9
9
would
suffice
to
ensure
that
for
the
array
'R ½
 ‰
’ ø í  belongs to
Û Ý
.
On
the
other
hand,
for
large
the
operator
gets closer to the
¼X½
‰
identity (at least on low scales) and the norm becomes closer to the
-norm so that
Assumption 2 becomes less demanding. Nevertheless, the validity of (5.13) as well as of
Assumption 2 is crucial for the above reasoning.
D
Finally, let us quickly address the estimation of
. A qualitative estimate on the
:
-norm of the integrand
is at best of the order A ñ . Since the required quadrature
accuracy in Apply is of the order 7
we have to ensure, in view of (4.6), that
ï
=Y?
O
. Although we do not know < we see that
O
.(
. YThus,
by
Á
ï
ï
A
(5.23), the overall work count can at best be expected to be of the order
.(
½ which
would almost correspond to the sparseness of
.
É }Í h P
^‡º
a
% )*n
É ]h ×
%
*
)
4
Â
%
U
'
s
*
)
'
.
%('U)*
­
a % ­ s)*4
R
]
h
^ª\1324}ï8:9;3<F1:
h a
% s)*4
èÌ/ n®9 É
É9
© 1Ì ©
ö
3Ì1 t 4C9 É ^ìa Ø ' % s)*4
ª
É
ö a Ø % s)*4
ö P 3Ì1 4
© Ì/ ©
é 3ÌMC % ' )*
%
ü
» â
•
t{•
â 8 ' » â
•
8™; w
]h _1:
© e
© ² ²e »
©
©» » 9 É
Experiments. For the numerical experiments, we choose the operator
£6."$ Numerical
with homogeneous boundary conditions on the unit interval )½£ . The eigent
values and corresponding eigenvectors of this operator are given by
u ú ' ' ö ú ¨ ¥ ƒ…dR ] C
NZBZNZ
Ô
t Ô
We will exploit this orthonormal basis of %D')* to obtain a reference solution in the following
experiment. Furthermore, we use the wavelet basis constructed in [6] of order 3 on the primal
ETNA
Kent State University
etna@mcs.kent.edu
95
ERROR CONTROLLED REGULARIZATION BY PROJECTION
and dual side for the spatial discretization.. In all computations, a highest level of
fixed.
ê
t
is
1
0.8
0.6
0.4
0.2
0
-0.2
0
0.2
0.4
0.6
0.8
1
F IG . 6.1. Initial value
a%Â'Us)*
into the
6.1. Application of Projections. First, the projection of a vector
joint eigenspaces of and
is tested. The main concern is to justify Assumption 1 which
the adaptive projection algorithm is based upon.
v
v , see Figure 6.1. Its
For the value
we choose the characteristic function of
expansion into the eigenvector basis of can easily be calculated and the Fourier coefficients
with respect to this basis decay sufficiently slowly to pose a meaningful test.
To apply the projection Ê@Ë
and the projected semigroup Ê@Ë
we choose the path
of integration as an ellipse
9
9
9
9
8:9E;3<
¡
t™• ԕ
324
:
Ì/ Õ úuÆ( ¥ R ¥ ƒ…d C
a C
û
u u
u for some fixed ð 5g . TheÔmost
enclosing the eigenvalues É 'BZNZBZN
satisfactory results
ï
are obtained by the choice
Õ j u É u ï
•Ô
ª u " u É ï
•Ô
ú
To justify Assumption 1 we compare the true error
û
ú
•
ð
Z
ò {€ z .Q |/V O € á@? : q "8 9;3< €D Q)îXV
A
r
r
with the error estimator
¡
err =š € 𑉠á@? : 1" á ' ? : ò {€ z .Q |/V O € á@? : q " á ' ? : Ó €D Q)îXV
A
r
 x ˆ . The arisingr elliptic
for the three time steps 2ì "IZ nX and XZ and ð t latter accuracy
Ô is chosen only in order to
equations are solved up to machine t accuracy. This
err
(=š € 𑉠á@? : /"8 9E;3<
ˆWð
validate Assumption 1 and should not be confused with the adapted tolerances in Solve as
part of the numerical solution process.
In Figures 6.2, 6.3, and 6.4 we have plotted the true error and the error estimator as
functions of . Since the true error and the error estimator are too close in the graphical
A
displays the ratio
between true error and error estimator is additionally depicted as dotted
line.
We observe an exponential convergence with respect to as predicted by Theorem 4.1.
The oscillating behavior of the error seems to be caused byA the symmetry of the integrand
ETNA
Kent State University
etna@mcs.kent.edu
96
W. DAHMEN AND M. JURGENS
K =2
2
10
K =4
2
10
0
10
0
10
−2
10
−2
10
−4
L2−error
−4
−6
10
2
L −error
10
−8
10
−6
10
10
−8
10
−10
10
−14
10
−10
err(N)
[err(N)]
err(N) / [err(N)]
−12
10
0
10
err(N)
[err(N)]
err(N) / [err(N)]
10
−12
20
30
N
40
50
10
60
K =6
2
10
0
20
40
N
60
80
100
60
80
100
K =8
1
10
0
10
0
10
−1
10
−2
10
−2
−3
10
2
L −error
2
L −error
10
−4
10
−4
10
−5
10
−6
10
err(N)
[err(N)]
err(N) / [err(N)]
−8
10
0
20
err(N)
[err(N)]
err(N) / [err(N)]
−6
10
−7
40
N
60
80
10
100
0
20
40
F IG . 6.2. True error compared to error estimator for projection
N
with respect to the real axis which was not taken into account by the analysis. It should be
noted that the error estimator shows the same oscillatory behavior.
The speed of convergence becomes slower for increasing values of ð . This indicates
that <
, the width of the strip R= to which extends analytically, decreases for increasing
ð . As for large ð , the parameterization introduces a strong distortion of the complex
plane, this effect is not surprising. It could be avoided by splitting into several curves each
enclosing one or a few neighboring eigenvalues.
When solving the backward problem, i.e.
, for ð
ˆ the absolute error is substantially larger than for the other test cases. This effect is caused by the ill-posedness of the
backwardproblem.
For
the projected semigroup
has the highest eigen
v x
value
. Thus, some of the Fourier modes of
are amplified substantially
which results in a large norm of
and an increased absolute error.
Concerning the quality of the error estimator we obtain a ratio of almost one between the
true error and the estimated one. This close relation only breaks down for large when the
A in
quadrature error drops below which corresponds to the discretization error
ê
when the wavelet basis is truncated at
. Thus, the breakdown of the error estimator for
large values of seems to be caused by discretization errors. Thus, Assumption 1 is justified
in our test case.A
As the error estimator is almost equal to the true error already for small , we choose
[
] and test itsA reliability. In
the constants from the adaptive algorithm to be
and
this experiment we observe that the actual error A stays well below the target accuracy 7
.
Moreover, the actual error is roughly proportional to 7 which indicates that the analysis of
5g
Ì
Ì
2£"IZ 89E; w 2x!"IZ Z ZBZNZ
t
8 9E;3<
t
9
t
8“9;3<
é É )*
t
É
ì57
ETNA
Kent State University
etna@mcs.kent.edu
97
ERROR CONTROLLED REGULARIZATION BY PROJECTION
K =2
5
10
K =4
4
10
2
10
0
0
10
10
−2
L2−error
−5
10
2
L −error
10
−4
10
−6
10
−10
−8
10
10
err(N)
[err(N)]
err(N) / [err(N)]
−15
10
0
10
−12
20
30
N
40
50
10
60
K =6
2
10
0
20
40
N
0
10
−2
10
60
80
100
60
80
100
K =8
2
10
0
10
−2
2
2
L −error
10
L −error
err(N)
[err(N)]
err(N) / [err(N)]
−10
10
−4
10
−6
−6
10
10
err(N)
[err(N)]
err(N) / [err(N)]
−8
10
−4
10
0
20
−8
err(N)
[err(N)]
err(N) / [err(N)]
40
N
60
80
100
10
0
20
40
N
F IG . 6.3. True error compared to error estimator for projected semigroup
Lemma 4.2 is not too pessimistic. We do not provide a graphical display of this results
because there is no further insight to be gained from the plot.
]h  6.2. Comparison of the four schemes. We strive to recover
from approximations
to
,
, where the initial value is given by
“
,
. The perturbation
is created by adding white noise to
coefficients (normalized in
the wavelet
) such that error in the discrete
norm
becomes exactly .
The four algorithms from 4.2 will be compared by measuring the difference
of the
D
regularized solution and the true one in the discrete
-norm, i.e.
)î when
the perturbation
tends to zero. The regularization parameter will be chosen according
to the discrepancy principle.
The results are reported in Figure 6.5. First, we observe that the conventional Tikhonov
scheme and the modified one yield the same errors. This is somewhat disappointing as the
modified Tikhonov scheme seems to be superior to the usual one by its construction. Nevertheless, the experiments indicate that the additional effort
for applying the projection does
9
not pay off.
9
9
Second, the regularization in the discrete setting with
produce an error close to the
error of the Tikhonov regularization, but the methods with
and
yield more
accurate approximations to
for large data errors . This effectDwas by no means clear in
advance, as these methods regularize in norms different from
, in which the error
was measured.
^‡º
^ 1 324i&89E;3< 2 &XZ f § ¥ ƒ…dR ] n" ] G
a t ^ º*"~^
%
' Ô
t
t
t
»
”
(% ')*
5» G
]Mh
»
˜
A: ] % ' X t
€ ] º " ]Ah €
Q XV
—
r
ª"
,"
t
€ ñ t{€ •UÔ œ É
Q V
r
ETNA
Kent State University
etna@mcs.kent.edu
98
W. DAHMEN AND M. JURGENS
K =2
5
10
K =4
4
10
2
10
0
0
10
10
−2
L2−error
−5
10
2
L −error
10
−4
10
−6
10
−10
−8
10
−15
10
0
10
err(N)
[err(N)]
err(N) / [err(N)]
10
−10
10
−12
20
30
N
40
50
10
60
K =6
2
10
0
err(N)
[err(N)]
err(N) / [err(N)]
20
40
N
60
80
100
60
80
100
K =8
2
10
1
10
0
10
0
10
−2
2
L −error
−1
2
L −error
10
−4
10
10
−2
10
−3
10
−6
10
−8
10
0
−4
err(N)
[err(N)]
err(N) / [err(N)]
20
10
−5
40
N
60
80
100
10
0
err(N)
[err(N)]
err(N) / [err(N)]
20
40
F IG . 6.4. True error compared to error estimator for projected semigroup
N
€ ] º " ] E€ D œ É
Q V
—
r
Third, we notice that the projection method yields errors
which are
much smaller than those of the other three methods. Since this method requires also less
computational effort than the other methods as discussed in 4.2 it seems to be preferable
over the other three methods.
As far as can be judged from Figure 6.5 none of the four methods converges with a
substantially higher rate than the other ones. As the inversion of the operator exponential is a
severely ill-posed problem (see [9] for a definition) there is perhaps no chance to improve on
the quality of the standard methods but merely on their efficiency.
”
7. Conclusions. We have developed and implemented regularization schemes for the
severely ill-posed inversion of diffusion processes. These schemes, including Tikhonov-type
as well as projection methods, avoid any time stepping PDE solvers for the forward problems
but rely on the error controlled approximate evaluation of certain Dunford integrals which
are based on appropriate quadrature techniques and the error controlled solution of resolvent
systems for well-posed operators. This latter task is trivially parallelized. In particular, this
allows us to realize, to our knowledge for the first time in this context, approximate SVDprojections without actually having to compute or manipulate corresponding singular basis
functions. Of course, the computational complexity is not proportional to the number of
recovered modes but the scheme works also in situations where the SVD basis functions
are not known and difficult to obtain. Moreover, the regularization is separated from the
discretization of the diffusion operator yielding error bounds for the true infinite dimensional
problem. Finally, we present a first discussion of the complexity of the projection scheme.
ETNA
Kent State University
etna@mcs.kent.edu
99
ERROR CONTROLLED REGULARIZATION BY PROJECTION
Tikhonov
Modified Tikhonov
Projection Method
Transf. Tikhonov s = 0
Transf. Tikhonov s = −1/2
Transf. Tikhonov s = −1
0
α
|| xδ − x ||
L
2
10
−1
10
−2
10
0
10
−1
δ || y ||
10
−2
10
L
2
F IG . 6.5. Comparison of Different Methods for Approximate Inversion of the Operator Exponential
This together with the numerical experiments support its superiority over the Tikhonov-type
schemes. This superiority is expected to grow with an increasing order of the wavelet basis
since the qualification of the underlying projection scheme is (in contrast to the Tikhonovtype schemes) unbounded.
REFERENCES
[1] A. B ARINKA , W. D AHMEN , AND R. S CHNEIDER , Fast Computation of Adaptive Wavelet Expansions, IGPM
Report, RWTH Aachen, August 2004.
[2] A. C OHEN , W. D AHMEN , AND R. D E V ORE , Adaptive wavelet methods II – Beyond the elliptic case, Found.
Comput. Math., 2 (2002), pp. 203–245.
[3] A. C OHEN , M. H OFFMANN , AND M. R EISS , Adaptive wavelet-galerkin methods for inverse problems, SIAM
J. Numer. Anal., 42 (2004), 1479-1501.
[4] W. D AHMEN , Wavelet and multiscale methods for operator equations, Acta Numer., vol. 6, Cambridge University Press, 1997, pp. 55–228.
[5] W. D AHMEN , Multiscale and wavelet methods for operator equations, in Multiscale Problems and Methods
in Numerical Simulation, C.I.M.E. Lecture Notes in Mathematics, vol. 1825, Springer, Heidelberg, 2003,
pp. 31–96.
[6] W. D AHMEN , A. K UNOTH , AND K. U RBAN , Biorthogonal spline wavelets on the interval – stability and
moment conditions, Appl. Comput. Harmon. Anal., 6 (1999), pp. 132–196.
[7] W. D AHMEN AND R. S CHNEIDER , Composite wavelet bases for operator equations, Math. Comput., 68
(1999), pp. 1533–1567.
[8] R. D E V ORE , Nonlinear approximation, Acta Numer., vol. 7, Cambridge University Press, 1998, pp. 51–150.
[9] H.W. E NGL , M. H ANKE , AND A. N EUBAUER , Regularization of Inverse Problems, Kluwer Academic Publishers, 1996.
[10] K.-J. E NGEL AND R. N AGEL , One-parameter Semigroups for Linear Evolution Equations, Springer, 2000.
[11] I. P. G AVRILYUK , W. H ACKBUSCH , AND B. N. K HOROMSKIJ , -matrix approximation for the operator
exponential with applications, Numer. Math., 92 (2002), pp. 83–111 (2002).
[12] T. K ATO , Perturbation Theory for Linear Operators, Corr. printing of the 2nd ed., Springer, 1980.
[13] R. K RESS , Linear Integral Equations, Springer, Berlin, 1999.
[14] A. L UNARDY , Analytic Semigroups and Optimal Regularity in Parabolic Problems, Birkhäuser, 1995.
ETNA
Kent State University
etna@mcs.kent.edu
100
W. DAHMEN AND M. JURGENS
[15] M. J ÜRGENS , A Semigroup Approach to the Numerical Solution of Parabolic Differential Equations, PhD
thesis, RWTH Aachen, 2005.
[16] M. J ÜRGENS , Adaptive application of the operator exponential, submitted to J. Numer. Math., special issue
on Breaking Complexity: Multiscale Methods for Efficient PDE Solvers.
[17] R. S TEVENSON , On the compressibility of operators in wavelet coordinates, SIAM J. Math. Anal., 35 (2004),
pp. 1110–1132.