MATH 676
–
Finite element methods in
scientific computing
Wolfgang Bangerth, Texas A&M University
http://www.dealii.org/
Wolfgang Bangerth
Lecture 31.55:
Nonlinear problems
Part 2: Newton's method for PDEs
http://www.dealii.org/
Wolfgang Bangerth
The minimal surface equation
Goal: Solve
(
−∇⋅ A
∇u
= f
2
√ 1+|∇ u|
)
⇔
∇u
= 0
2
√ 1+|∇ u|
⏟
(
)
f + ∇⋅ A
=: R (u)
Newton's method: Iterate
[R ' (uk )] δ u k = −R (u k ),
uk +1 = u k +δ uk
Questions:
●
What is the variational formulation of this?
●
What is R'(u) anyway?
http://www.dealii.org/
Wolfgang Bangerth
Derivatives for operators
Question: What is R'(u)?
Answer, part 1: Start with R(u):
(
R (u) := f + ∇⋅ A
∇u
√ 1+|∇ u|2
)
This is an operator that maps
1
R : u∈H 0 → H
http://www.dealii.org/
−1
Wolfgang Bangerth
Derivatives for operators
Question: What is R'(u)?
Answer, part 2: Start with R(u):
(
R (u) := f + ∇⋅ A
∇u
√ 1+|∇ u|2
)
Then use the definition of a directional derivative:
R ' (u)(δ u) := lim ϵ→ 0
http://www.dealii.org/
R (u+ϵ δ u)−R(u)
ϵ
Wolfgang Bangerth
Derivatives for operators
Question: What is R'(u)?
Answer, part 3: Now we need to do some scary calculus:
R ' (u)(δ u) := lim ϵ→ 0
R (u+ϵ δ u)−R(u)
ϵ
[(
= lim ϵ→0
1
ϵ f +∇⋅ A
= lim ϵ→0
1
ϵ ∇⋅A
http://www.dealii.org/
[ (
(
∇ (u+ϵ δ u)
√ 1+|∇ (u+ϵ δ u)|2
∇ (u+ϵ δ u)
)) ( (
∇u
− f +∇⋅ A
√ 1+|∇ u|2
∇u
−
√ 1+|∇ (u+ϵ δ u)|2 √ 1+|∇ u|2
))]
)]
Wolfgang Bangerth
Derivatives for operators
Question: What is R'(u)?
Answer, part 3: Now we need to do some scary calculus:
[ (
1
R ' (u)(δ u) := lim ϵ→ 0 ϵ ∇⋅A
∇ (u+ϵ δ u)
∇u
−
2
2
1+|∇
(u+ϵ
δ
u)|
1+|∇
u|
√
√
)]
We need to do a Taylor expansion on a term of the form
x +ϵ y
f (ϵ)=
=
2
√ 1+( x +ϵ y )
This yields:
[ (
1
R ' (u)(δ u) := lim ϵ→ 0 ϵ ∇⋅A
http://www.dealii.org/
(
x
+
2
√ 1+ x
2
)
y
x y
2
−
ϵ+O(ϵ
)
2
2 3 /2
√ 1+ x ( 1+x )
( ∇ u⋅∇ ϵ δ u) ∇ u
∇ ϵδu
2
−
+O(ϵ
)
2
2 3 /2
( 1+|∇ u| )
√ 1+|∇ u|
)
]
Wolfgang Bangerth
Derivatives for operators
Question: What is R'(u)?
Answer, part 3: After this step...
[ (
1
R ' (u)(δ u) := lim ϵ→ 0 ϵ ∇⋅A
( ∇ u⋅∇ ϵ δ u) ∇ u
∇ ϵδu
2
−
+O(ϵ
)
2
2 3 /2
( 1+|∇ u| )
√ 1+|∇ u|
)
]
… all we need to do is take the limit:
(
R ' (u)(δ u) := ∇⋅ A
( ∇ u⋅∇ δ u) ∇ u
∇ δu
−
A
2
2 3 /2
1+|∇
u|
( 1+|∇ u| )
√
)
This operator is linear in the direction δu in which we take
the derivative!
http://www.dealii.org/
Wolfgang Bangerth
Derivatives for operators
Question: What is R'(u)?
Short answer: Really, taking derivatives of things like
(
R (u) := f + ∇⋅ A
∇u
√ 1+|∇ u|2
)
… works almost like taking normal derivatives. You just
always have to provide the direction δu !
http://www.dealii.org/
Wolfgang Bangerth
Derivatives for operators
Examples:
●
F (u) = au
●
F (u) = u
●
F (u) = (∇ u)
●
F (u) =
2
⇒
F ' (u)(δ u) = a δ u
⇒
F ' (u)(δ u) = 2 u δ u
2
⇒
1
2
1+( ∇ u)
http://www.dealii.org/
F ' (u)(δ u) = 2 (∇ u)⋅(∇ δ u)
⇒
F ' (u)(δ u) = −
1
2 2
( 1+(∇ u) )
( 2 ∇ u⋅∇ δ u )
Wolfgang Bangerth
Derivatives for operators
Question: What is R'(u)?
Short answer: Really, taking derivatives of things like
(
R (u) := f + ∇⋅ A
∇u
√ 1+|∇ u|2
)
… works almost like taking normal derivatives:
( ∇ u⋅∇ δ u) ∇ u
∇ δu
R ' (u)(δ u) := ∇⋅ A
−A
3 /2
2
( 1+|∇ u|2 )
√ 1+|∇ u|
(
http://www.dealii.org/
)
Wolfgang Bangerth
Derivatives for operators
Question: What is R'(u)?
Some more theory:
●
●
●
We call R'(u)(δu) the Gateaux differential of R at u in
direction δu .
If R'(u)(δu) exists for all δu then we say that R is
Gateaux differentiable at u.
Under certain conditions (linearity, continuity, …) we can
define a linear operator R'(u) and then we can write
R'(u)(δu) = R'(u) δu
http://www.dealii.org/
Wolfgang Bangerth
The minimal surface equation
Goal: Solve
(
−∇⋅ A
∇u
= f
2
√ 1+|∇ u|
)
⇔
∇u
= 0
2
√ 1+|∇ u|
⏟
(
)
f + ∇⋅ A
=: R (u)
Newton's method: Iterate
[R ' (uk )] δ u k = −R (u k ),
uk +1 = u k +δ uk
Questions:
●
What is the variational formulation of this?
●
What is R'(u) anyway?
http://www.dealii.org/
Wolfgang Bangerth
Newton's method for PDEs
Question: What is the variational formulation of
[R ' (uk )] δ u k = −R (u k ),
uk +1 = u k +δ uk
Answer: With...
(
R (u) := f + ∇⋅ A
∇u
√ 1+|∇ u|2
)
...we get:
(
R ' (u)(δ u) := ∇⋅ A
http://www.dealii.org/
( ∇ u⋅∇ δ u) ∇ u
∇ δu
−
A
2
2 3 /2
1+|∇
u|
( 1+|∇ u| )
√
)
Wolfgang Bangerth
Newton's method for PDEs
Question: What is the variational formulation of
[R ' (uk )] δ u k = −R (u k ),
Answer: With...
uk +1 = u k +δ uk
A (∇ uk⋅∇ δ u k )
A
∇ δ uk − ∇ φ ,
∇ uk
( φ , R ' (u k )(δ uk )) := ∇ φ ,
2
2 3 /2
√ 1+|∇ uk|
( 1+|∇ uk| )
)(
(
(
( φ , R(uk )) := ( φ , f ) + ∇ φ ,
A
∇ uk
2
√ 1+|∇ uk|
)
)
...we arrive at this in each Newton step:
(
A (∇ uk⋅∇ δ uk )
A
∇φ,
∇ δ uk − ∇ φ ,
∇ uk
2
2 3 /2
√ 1+|∇ uk|
( 1+|∇ u k| )
)(
(
= −(φ , f )− ∇ φ ,
http://www.dealii.org/
)
A
∇ uk
2
√ 1+|∇ uk|
)
∀ φ∈H 10
Wolfgang Bangerth
Practical considerations
Question: What if R(u) is already really complicated?
Answer 1: Then your R'(u) will be even more complicated.
You will probably make mistakes calculating it, or
implementing it.
http://www.dealii.org/
Wolfgang Bangerth
Practical considerations
Question: What if R(u) is already really complicated?
Answer 2: You could use a symbolic math package to
compute the derivative (e.g., Maple, Mathematics, …)
http://www.dealii.org/
Wolfgang Bangerth
Practical considerations
Question: What if R(u) is already really complicated?
Answer 3: Or you could use automatic differentiation from
within the code:
– in the code you only implement R(u)
– which you need anyway
– and get R'(u) for free (and correct!) if you do it right
Step-33 shows an example of how to do this.
http://www.dealii.org/
Wolfgang Bangerth
Practical considerations
Question: How accurate do we have to be?
Observation: In the first few Newton steps, we are still far
away from the solution!
– We could compute Newton updates δuk on a coarse mesh
– We could solve the linear system inaccurately
In practice, this is exactly what is done.
http://www.dealii.org/
Wolfgang Bangerth
Practical considerations
Question: But Newton's method does not always converge?
Answer: Yes. In many cases on needs a “globalization”
strategy such as
– line search
– a trust region method
http://www.dealii.org/
Wolfgang Bangerth
Newton's method for PDEs
Summary: For Newton's method on PDEs, we need to think
about what the derivative of the residual R(u) should mean.
In practice: The derivative can be computed almost as
normal.
We can then define each Newton iteration in weak form as
( φ ,[ R ' (u k )] δ uk ) = −(φ , R (uk )),
uk +1 = u k +δ u k
We then solve for the Newton updates δuk on finer and
finer meshes with more and more accuracy.
http://www.dealii.org/
Wolfgang Bangerth
MATH 676
–
Finite element methods in
scientific computing
Wolfgang Bangerth, Texas A&M University
http://www.dealii.org/
Wolfgang Bangerth