Adaptive regularization strategies for nonlinear PDE
Coarse mesh solvers for quasilinear PDE
Sara Pollock
Texas A&M Department of Mathematics
Finite Element Circus, UMass Dartmouth, October 16-17, 2015
S. Pollock
Texas A&M Mathematics
1/18
Adaptive nonlinear solver
FE Circus October 16-17, 2015
MOTIVATION: Convergence problem!!
(FE Circus, Oct. 2014)
Z
κ(u)∇u · ∇v =
Z
f v,
κ(u) = 1 +
1
10−2 + (u − a)2
The standard Newton iterations for solving the nonlinear problem: hF(U), vh i = 0 :
Starting with initial guess U = U0 , iterate:
Find: w such that hDF(U)w, vi = −hF(U), vi
Update: U = U + w
Check: if kF(U)k < tolerance
Sensitivity to initial data: This method did not converge for a ∈ (0.155, 1.35). In fact ,
starting the Newton method with the solution U0 = sin(πx) sin(πy) converged in 3
iterations, but failed to converge with a perturbation of more than 2 × 10−2 .
For the results here a damped Newton method is implemented where the step size is
decreased by a factor of 1/2 until the residual decreases (line search method).
A better method by means of Tikhonov regularization with appropriate penalty
parameters is currently under investigation.
S. Pollock
Texas A&M Mathematics
1/18
Adaptive nonlinear solver
FE Circus October 16-17, 2015
Recent results: −div(κ(u, |∇u|)∇u) + b(u) · ∇u = f
Recent papers:
SP, Stabilized and inexact adaptive methods for capturing internal
layers in quasilinear PDE.
Available as arXiv:1507.06965 [math.NA], 2015
SP, An improved method for solving quasilinear convection
diffusion problems on a coarse mesh.
Available as arXiv:1502.02629 [math.NA], 2015
SP, A regularized Newton-like method for nonlinear PDE.
Numerical Functional Analysis and Optimization, Accepted.
Available as arXiv:1412.6487 [math.NA], 2014
Right: Log-scale visualization of diffusion coefficient, meshsize: 1/48.
κ(u) = 1 + {ε + (u − 0.5)2 }−1 ,
S. Pollock
Texas A&M Mathematics
ε = 10−2 , 10−3 , 10−4 , 10−5 .
2/18
Adaptive nonlinear solver
FE Circus October 16-17, 2015
Recent results: −div(κ(u, |∇u|)∇u) + b(u) · ∇u = f
Recent papers:
SP, Stabilized and inexact adaptive methods for capturing internal
layers in quasilinear PDE.
Available as arXiv:1507.06965 [math.NA], 2015
SP, An improved method for solving quasilinear convection
diffusion problems on a coarse mesh.
Available as arXiv:1502.02629 [math.NA], 2015
SP, A regularized Newton-like method for nonlinear PDE.
Numerical Functional Analysis and Optimization, Accepted.
Available as arXiv:1412.6487 [math.NA], 2014
Right: Log-scale visualization of diffusion coefficient, meshsize: 1/24.
κ(u) = 1 + {ε + (u − 0.5)2 }−1 ,
S. Pollock
Texas A&M Mathematics
ε = 10−2 , 10−3 , 10−4 , 10−5 .
2/18
Adaptive nonlinear solver
FE Circus October 16-17, 2015
Recent results: −div(κ(u, |∇u|)∇u) + b(u) · ∇u = f
Recent papers:
SP, Stabilized and inexact adaptive methods for capturing internal
layers in quasilinear PDE.
Available as arXiv:1507.06965 [math.NA], 2015
SP, An improved method for solving quasilinear convection
diffusion problems on a coarse mesh.
Available as arXiv:1502.02629 [math.NA], 2015
SP, A regularized Newton-like method for nonlinear PDE.
Numerical Functional Analysis and Optimization, Accepted.
Available as arXiv:1412.6487 [math.NA], 2014
Right: Log-scale visualization of diffusion coefficient, meshsize: 1/12.
κ(u) = 1 + {ε + (u − 0.5)2 }−1 ,
S. Pollock
Texas A&M Mathematics
ε = 10−2 , 10−3 , 10−4 , 10−5 .
2/18
Adaptive nonlinear solver
FE Circus October 16-17, 2015
Main idea: inexact solves of regularized problems
Coarse mesh regime: Extract information from regularized
problem for mesh refinement.
I
I
Partial solves of inexact, regularized problems as the internal layers
are uncovered.
Goal of regularization: stability not accuracy
Preasymptotic regime: Build sequence of initial guesses to
regularized problems, and continue to refine the mesh adaptively.
I
I
I
I
As the mesh is refined, reduce the regularization to increase the
accuracy of each solve, constructing a better initial guess for the
subsequent refinement.
Update the regularization parameters adaptively to increase both
accuracy and efficiency through the preasymptotic regime.
For convergence of the method:
Stopping criteria for inexact nonlinear solves.
Update criteria for regularization parameters.
Asymptotic regime: Solve the discrete problem, and refine mesh
adaptively.
I
Reduction to a standard Newton method.
S. Pollock
Texas A&M Mathematics
3/18
Adaptive nonlinear solver
FE Circus October 16-17, 2015
Target problem class
Problem 1: quasilinear diffusion and convection-diffusion:
F(x, u, ∇u) = −div(κ(x, u, |∇u|)∇u) + b(u) · ∇u − f = 0 in Ω,
u = 0 on ∂Ω.
Diffusion: b(u) = 0. Convection-diffusion: b(u) 6= 0.
1
Existence and uniqueness results for F(x, u, ∇u) = 0 carry over to the discrete
problem assuming the mesh is fine enough.
2
Local uniqueness and approximation properties of the finite element solution hold
assuming the mesh is fine enough.
3
Existence an (local) uniqueness of F(x, u, ∇u) = 0 does not necessarily carry
over to the coarse mesh problem.
4
The coarse mesh problem may be considered noisy and ill-posed.
5
Coarse mesh Jacobians are ill-conditioned and sometimes indefinite.
Adaptive method: Assume the initial mesh is not fine enough.
(1) Construct a sequence of efficient adaptive partitions over which the problem is
well-resolved and stable.
(2) Construct sequence of transitional solutions to determine mesh refinement
and to start a Newton-like iteration on the next partition.
S. Pollock
Texas A&M Mathematics
4/18
Adaptive nonlinear solver
FE Circus October 16-17, 2015
Newton-like udpates: Tikhonov and Ψtc
Common idea: stabilize the Jacobian by penalizing the update.
Newton update: linearize F(u, x) = 0
⇒ F 0 (x, un )wn = −F(x, un ),
un+1 = un + wn .
Pseudo-transient continuation: Discretize ∂u/∂t + F(x, u) = 0 with backward
Euler:
(∆tn−1 I + F 0 (x, un ))wn = −F(x, un ),
un+1 = un + wn .
Tikhonov regularization of the linerization with positive semidefinite R:
minimize Gα (w) = kF 0 (x, un )w + F(x, un )k2 + αkRwk2 , yields iteration
(αn RT R + F 0 (x, un )T F 0 (x, un ))wn = −F 0 (x, un )T F(x, un ),
un+1 = un + wn ,
Without the normal equations: Larentiev regularization
(αn R + F 0 (x, un ))wn = −F(x, un ),
un+1 = un + wn .
Coincides with ψtc stabilization of ∂(Ru)/∂t + F(x, u) = 0 with α = 1/∆t .
Important idea from regularization: Generality : positive semidefinite R
chosen to regularize only bad dof. Set R as a selectively-nullified Laplacian
stiffness matrix:
I dof in regions of low curvature in null (R)
I
Predict curvature with the jump-term in residual-based indicator.
S. Pollock
Texas A&M Mathematics
5/18
Adaptive nonlinear solver
FE Circus October 16-17, 2015
Improved method: continuation point of view
1
Replace backward Euler discretization with Newmark update
un+1 − un = ∆tn {(1 − γ)u̇n + γ u̇n+1 },
u̇ = ∂u/∂t.
Reason: for γ > 1, controllable increase in numerical dissipation.
Problem: Resulting iteration still requires use of normal equations to stabilize
ill-conditioned indefinite Jacobians.
2
Stabilize the Newmark update by σ-splitting
un+1 − un = ∆tn {σ(1 − γ)u̇n + (1 − σ)(1 − γ)u̇n + σγ u̇n+1 + (1 − σ)γ u̇n+1 },
and linearize the (1 − σ) fraction by
R(u̇n+1 − u̇n ) = F(x, un+1 ) − F(x, un ) = F 0 (x, ū)wn for fixed ū.
Resulting iteration:
αn R + γ (1 − σ)F 0 (x, ū) + σF 0 (un , x) wn = −F(x, un ),
un+1 = un + wn ,
Good news: Stabilizes iteration without use of normal equations.
Convergence rate: Newmark update decreases rate of convergence from quadratic
to q-linear with rate q = 1 − 1/γ.
S. Pollock
Texas A&M Mathematics
6/18
Adaptive nonlinear solver
FE Circus October 16-17, 2015
Inexact iteration for quasilinear diffusion
Previous iteration:
αn R + γ (1 − σ)F 0 (x, ū) + σF 0 (x, un ) wn = −F(x, un ),
un+1 = un + wn ,
Observation 1: γ introduces numerical dissipation by scaling down residual.
The residual: F(x, u) = −g(u) + f (x) .
Observation 2: g(u) is not the problem!!
⇒ The oscillations are caused by the source f (x).
⇒ Solution: Scale down f (x) until problem stabilizes.
New inexact iteration:
αn R + γ (1 − σ)F 0 (x, ū) + σF 0 (x, un ) wn = −g(un ) + δ f (x),
un+1 = un + wn ,
Now require:
δ → 1 from below: consistency.
And want:
γ → 1 from above: efficiency.
Easier but important: σ → 1 From below, α → 0 with kF(x, u)k
S. Pollock
Texas A&M Mathematics
7/18
Adaptive nonlinear solver
FE Circus October 16-17, 2015
γ → 1 from above: assumptions and update criteria
Assumption: Upper bound on γ. Given a rate tolerance εT , γnk is bounded above for
all n, k by γMAX , satisfying
γnk ≤ γMAX < 1/εT .
Update criteria for γ
Given a user set tolerance εT , and γn < γMAX , and a minimum number of iterations
between updates Imin , the dissipation parameter γn is updated after each iteration n on
which all the following criteria hold.
1
γn > 1.
2
The observed rate of convergence is within tolerance of the predicted value
n+1
kr k
1 krn k − 1 − γn < εT .
3
The observed rate of convergence is sufficiently stable.
n+1
kr k
krn k −
< εT .
krn k
krn−1 k 4
At least Imin ≥ 2 iterations have passed since the last update of γ.
S. Pollock
Texas A&M Mathematics
8/18
Adaptive nonlinear solver
FE Circus October 16-17, 2015
γ → 1 from above: definition and monotonicity
Definition
Given an initial γ0 > 1, and decrease parameter 0 < q < 1, for refinement k > 0
γ̃n+1 = q ·
hrn , rn i
hrn , gn+1 − gn i
,
γn+1 = max 1 , γ̃n+1 .
Stability:
q · γn
q · γn
n+1
<
γ̃
<
.
γn (2 + εT ) − 1
1 − εT γn
Monotonic decrease of γ: Assuming the stronger upper bound:
γnk
1
≤ γMONO =
εT
q
1−
,
q̄
with 1 < q̄ < q, then for γn+1 = γ̃n+1 > 1, the sequence satisfies γn+1 < q̄γn .
Key estimate:
n 2
kr k
S. Pollock
Texas A&M Mathematics
1
− εT
γn
n
n+1
< hr , g
1
− g i < kr k 2 − n + εT .
γ
n
n 2
9/18
Adaptive nonlinear solver
FE Circus October 16-17, 2015
δ → 1 from below: assumptions and update criteria
Update criteria for δ: Given a user set rate tolerance εT and a convergence tolerance
εCON , the iterations are terminated and δ is updated after iteration n of refinement k,
when conditions (1-4) are all met, or when conditions (1) and (5) are satisfied.
(1) δk < 1. If δk = 1, it is not updated.
(2) Sufficient decrease in the sequence of residuals krk k
krkn k < min{krk0 k, krk−1 k}.
(3) The observed rate of convergence is beneath the accepted rate (1 − 1/2γ)
krn+1 k
1
< 1− n.
n
kr k
2γ
(4) The observed rate is sufficiently stable
krn+1 k εT
krn k
+
>
.
krn k
2
krn−1 k
(5) krn k < εCON . The iteration reached convergence.
S. Pollock
Texas A&M Mathematics
10/18
Adaptive nonlinear solver
FE Circus October 16-17, 2015
δ → 1 from below: definition
Definition
Given an initial δ0 < 1, for k > 0
h f , γσ(gn+1 − gn ) + (γ(1 − σ)g0 (un ) + αR)wn + gn i
δ˜ k+1 =
,
qk k f k2
δk+1 = min{1 , δ˜ k+1 },
with R = Rk , α = αn , γ = γn and σ = σn .
In terms of the one-step linearization error
L e (un ) := gn+1 − gn − g0 (un )wn , gn+1 = g(un + wn ), the recursive relation for δ:
1
δk+1 =
qk
γσ
e n
δk +
h f , L (u )i .
k f k2
Effective bound: δk+1 > (1/q̄)δk , 1 < q̄ ≤ qk whenever
qk k f k
h fˆ, L e (un )i > −δk 1 −
.
q̄ γnk σnk
Convergence of δ to 1 is also established.
S. Pollock
Texas A&M Mathematics
11/18
Adaptive nonlinear solver
FE Circus October 16-17, 2015
Adaptive algorithm for nonlinear diffusion
Set the parameters qγ , εT , γMAX , εCON , σ0 and δMIN . Let ū = 0. Start with initial u0 , γ0 ,
δ0 . On partition Tk , k = 0, 1, 2, . . .
1
Compute Rk , and g0 (ū).
2
Set r0 = −g(u0 ) + δ0 f0 , and set α0 = kr0 k.
3
While the Exit Criteria are not met on iteration n − 1 :
n
krn k
o
.
(a) Set σ = max σ0 , 1 − K
0
n
n
n
0
n
(b) Solve (α Rk + γ {σ g (u ) + (1 − σ)g0 (ū)})wn = rn , for wn .
(c) Update un+1 = un + wn , and rn+1 = −g(un+1 ) + δk fk .
(d) If Criteria to update γ are satisfied, update γn+1 .
(e) Update αn ≤ krn k.
4
5
If Criteria to update δ are satisfied, update δk+1 for partition Tk+1 , with
qδ = (qγ )P+1 , where P counts the number of times γ was updated on refinement
k.
Compute the error indicators to determine the next mesh refinement.
S. Pollock
Texas A&M Mathematics
12/18
Adaptive nonlinear solver
FE Circus October 16-17, 2015
Exit strategy
Given a user set tolerance εCON , and an accepted rate of convergence given by
qACC = 1 − 1/(2γ), and a baseline number of allowed iterations IBASE ,
set the maximum number of iterations IMAX by
&
'
log (krk−1 k) − log krk0 k
+ 1, IMAX = max{IACC , IBASE }.
IACC =
log(qACC )
Exit criteria: Exit the solver on partition Tk after calculating residual krn+1 k when one
of the following conditions holds.
1
The iteration reached convergence.
2
Conditions (2-4) of δ-update Criteria: the iteration converges an at acceptable
rate and is stopped before convergence.
3
Maximum number of iterations IMAX completed: reset γ0k+1 , δk+1 , and u0k+1 .
S. Pollock
Texas A&M Mathematics
13/18
Adaptive nonlinear solver
FE Circus October 16-17, 2015
Model problem with steep layers
Quasilinear diffusion problem on Ω = (0, 1)2 .
−div(κ(u)∇u) = f in Ω ,
u = 0 on ∂Ω.
1
ε = 6 × 10−5 , a = 0.5, and k = 1.
κ(s) = k +
((ε + (s − a)2 )
f (x, y) chosen so the exact solution u(x, y) = sin(πx) sin(πy).
Parameter a controls the position of the internal layers.
The initial mesh has 144 elements. εT = 5 × 10−3 , q = 0.9.
106
103
γ 105
δ
4
102
101
100
10-1
10-2
10-3
5
10
15
20
25
Refinements
30
10
103
102
101
100
10-1
10-2
10-3
-4
3510
|u−uk |1
|u−uk |0
ηk
n−1/2
103
104
Degrees of freedom, n
Figure: Left: Parameters γ, δ → 1. Center: error. Right: mesh from Level 20.
S. Pollock
Texas A&M Mathematics
14/18
Adaptive nonlinear solver
FE Circus October 16-17, 2015
Figure: Level 18 updates.
S. Pollock
Texas A&M Mathematics
15/18
Adaptive nonlinear solver
FE Circus October 16-17, 2015
Model problem with unstable linearization
Quasilinear diffusion problem on Ω = (0, 1)2 .
−div(κ(u)∇u) = f in Ω ,
u = 0 on ∂Ω.
κ(s) = 10 + sin(s/ε) + arctan(s/ε),
2
ε = 6 × 10−3 .
2
f (x, y) = (1 − x)(1 − y)(e8x − 1)(e8y − 1).
102
γ
δ
101
100
10-1
10-2
5
10
15
Refinements
20
Figure: Left: Parameters γ, δ → 1. Right: Solution, Level 20.
S. Pollock
Texas A&M Mathematics
16/18
Adaptive nonlinear solver
FE Circus October 16-17, 2015
Updates from Levels 5, 10 and 20
···
···
Figure: Level 5 updates: 1, 11 and 22.
···
···
Figure: Level 10 updates: 1, 9 and 18.
Figure: Level 20 updates.
S. Pollock
Texas A&M Mathematics
17/18
Adaptive nonlinear solver
FE Circus October 16-17, 2015
Current directions
Finite element solution
Current directions
Analysis of parameter choice for convection-diffusion problems (M.
Ebrahimi, SP) and more general boundary conditions (SP).
Generalization to inner iteration with multigrid method for inexact
linear solves (Y. Zhu, SP).
Finite element solution
Accelerated explicit pseudo-time method for low-order
discretization of Monge-Ampère type equations (G. Awanou, SP).
Goal-oriented adaptivity for multiscale methods (E. Chung, Y.
Efendiev, W.T. Leung, SP).
S. Pollock
Texas A&M Mathematics
18/18
Finite element solution
Adaptive nonlinear solver
FE Circus October 16-17, 2015
Main references
Engl, H.W., Hanke, M. and Neubauer, A., Regularization of Inverse Problems,
Springer, 1996.
Holst, M., Adaptive Numerical treatment of elliptic systems on manifolds,
Adv. Comput Math., 15, 1-4, 139-191, 2001.
Kelley, C.T, and Keyes, D, Convergence Analysis of Pseudo-Transient
Continuation,
SIAM J. Numer. Anal., 35(2), 508-523, 1998.
N. M. Newmark, A method of computation for structural dynamics,
J. Eng. Mech.-ASCE, 85, EM3, 67-94, 1959.
SP, A regularized Newton-like method for nonlinear PDE,
Accepted: Numer. Func. Anal. Opt., arXiv:1412.6487 [math.NA], 2014.
SP, An improved method for solving quasilinear convection diffusion problems on
a coarse mesh,
Available as arXiv:1502.02629 [math.NA], 2015.
SP, Stabilized and inexact adaptive methods for capturing internal layers in
quasilinear PDE,
Available as arXiv:1507.06965 [math.NA], 2015.
S. Pollock
Texas A&M Mathematics
18/18
Adaptive nonlinear solver
FE Circus October 16-17, 2015