Lecture 22
18.086
dF
dF
After integrating
Inside the integral
F ( u + v , u ' + v f ) = F ( u , u f )+ v - + v f - + - - The best Chapter
u defeats8every
other candidate
that satisfies
these du
boundaryduf
conditions.
628
Optimization
andu+v
Minimum
Principles
Then ( u v)(O)= a and ( u v ) ( l )= b require that v(0) = v(1) = 0. For small v
That
integrated
term is the "first variation" of P. We have already reached 6P/6u:
After
integrating
and v', the correction terms come from dF/du and dF/du'. They don't involve v2:
+
+
Variational calculus & finite
+--elements
1
<
One-dimensional Problerns
dF
dF
That
integrated
term
is
the
"first
variation"
of
P.
We
have
already
reached
First
variation
v
+
v
dx
=
0
f
o
r
v . 6P/6u:
Inside the integral
F ( u + v , u ' v f ) = F ( u , u f )+ v - + v f -every
du with
dua boundary
du condition
duf at each end:
The basic problem is 6u
to minimize P(u)
After
integrating
First
+v
0 f odirection
r every vv. must be zero.
This
isvariation
the
equation for u. Thev -derivative
of Pdxin= each
6u
du
du
•Otherwise
Energy functional:
P(u)
=
F ( U , uwould
') dx with
( 0=)a andP(u)
u(1)
= good.
b.
we can make 6P/6u
negative,
which
mean~P(u+v)
: no
Thatisintegrated
term for
is the
"first
variation"
ofofP.PWe
have
already
reached
6P/6u:
This
the
equation
u.
The
derivative
in
each
direction
v
must
The weak form comes from integrating v (dF/du') by parts to get - v (bed ~zero.
/ d u '':
)
Otherwise
can make
6P/6u
negative,
which
would
P(u+v)
P(u): noconditions.
good.
The best we
u defeats
every
other
candidate
u+v
that mean
satisfies
these <
boundary
First variation
v -v+) (v l )= b require
dx = 0 f o rv(0)
every v .
u v)(O)
= a6uand
( uintegrating
Then
The (weak
form comes
from
v (dF/du')that
by parts =
to v(1)
get -=
v (0.
d ~For
/ d usmall
'':
) v
du
du
Weak
formcorrection terms come-from
( dF/du and ddF/du'.
[ v & ] l They
= 0 .don't involve v2:
and
v', the
'
+
+
L ' ~ (( ~ )
'
This is the equation for u. The derivative of P in each direction v0 must be zero.
Weak form
Otherwise
we can make 6P/6u negative,
would mean
P(u+v)
<
good.
dF
- ( which
d= [0.
v&
] lguarantee
=P(u)
0 . : nodF
The
boundary
term
vanishes
because
v(0)
=
v(1)
To
zero
for every
Inside the integral
F ( u + v , u ' + v f ) = F ( u , u f )+ v0- + v f weak
form comes
integrating
v (dF/du
') by parts
to get
/ d u '':
)
du- v ( d ~duf
v ( x )The
in the
integral,
the from
function
multiplying
v must
be zero
(strong
form):
L ' ~ (( ~ )
+---
'
The boundary term vanishes because v(0) = v(1) = 0. To guarantee zero for every
integrating
v ( xAfter
)Weak
in the
integral, the function multiplying v must be zero (strong form):
form
L ' ~ ( for
~ -u) (
d [v&]l = 0 .
Euler-Lagrange equation
(
0
That
integrated
term
is thebecause
"first
P. We have already reached 6P/6u:
Euler-Lagrange
equation
for
u variation"
The
boundary
term
vanishes
v(0) = v(1)of
= 0. To guarantee zero for every
v ( x ) in the
the
function
multiplying
v must(0,
bea )zero
form):
Example
1 integral,
Find the
shortest
path
u ( x )between
and(strong
( 1 ,b): u(0)
= a and u(1)= b.
variation
+ vstep on(0,
dx =
0,b):Pf u(0)
o( ru fevery
vu(1)
. = b.
By First
Pythagoras,
, / the
( d ~shortest
) ~6u( dpath
~is )ua (short
~vx-)du
the
path.
So
)
=
Example
1 Find
between
a
)
and
(
1
=
a
and
d
m d x
du
Euler-Lagrange equation for u
is
the length of the path between the
This square root ~ ( u 'depends
)
only on u
By Pythagoras, , / ( d ~ ) ~ ( d ~is )a short
~ points.
step on the path. So P ( u f )= d m d x
This
is the=equation
for u. The
derivative
in each
v must be zero.
and
dF/du
0.
The derivative
dF/du
brings of
thePsquare
rootdirection
into the denominator:
is the length of the path between the points. This square root ~ ( u 'depends
)
only on u
Otherwise
make
6P/6u
negative,
which
P(u+v)
<
P(u)
Example
Find
the
shortest
path
u ( x )between
(0,
)square
and ( 1mean
,root
b): u(0)
=the
a and
u(1)
= b.: no good.
and
dF/du1=we
0.can
The
derivative
dF/du
brings
thea would
into
denominator:
First variation
6P) +
u' step on the path. So P ( u f )= U'
By Pythagoras,
/ ( d ~comes
~ ( d from
~is )a short
~
The weak ,form
integrating
by) parts
get)-ddvx(=
)
d xv =(dF/du
- I 1 v ')( ~dx
d ( dto m
0 /. (d4u)'':
xd ~
gives
weak form 6P
du
First variation
u'
U'
+
+
'
'
'
'
d
m
'
such as airplanes or turbines.
From the weak form to finite
e element method has big advantages over finite di↵erences
egular mesh discretization. elements
We will discuss the finite element
g geometries such as moving pistons.
• Electrostatics: The electrostatic potential p and the charge
ensional
Poisson equation
distribution r fulfill the Poisson equation (1D):
00
(x) =
4⇡⇢(x)
on interval Ω=[a,b]
(3.29)
•
Alternative system: r the mass density, p the gravitational potential
•
Dirichlet
the analysis (but not necessary):
(0) = BCs
(1)to=simplify
0.
(3.30)
(a) = a , (b) = b
•
We have seen that this can be expressed in its weak form as
pand theZsolution (x) in terms of basis functions {vi }, i =
b
pace:
for all test functions v
( 00 v + 4⇡⇢v) dx = 0
a
or(x) =
Z
b
1
X
ai vi (x).
(3.31)
i=1
( 0 v 0 + 4⇡⇢v) dx + 0 v|ba = 0
for all testafunctions
v
ion the infinite
basis
set
needs
to
be
truncated,
choosing
fia
0 (BCs) orthogonal,
N of N linearly independent, but not necessarily
ot the unknown of the system, but something that only exists virtually to
problem. The weak formulation is, in that context, a principle of virtual
(principle of virtual work, etc).
What’s a test function?
miting spaces
d a point where we should be a little more specific on where we are looking
• From the weak form to make sense, the test functions have to be integrable. A
re v belongs. The first space we need is the space of square–integrable
sufficient condition for that is the space of functions
ØZ
Ω
æ
Ø
L2 (≠) = f : ≠ ! R ØØ
|f |2 < 1 .
≠
• Weofwill
see
soonrequires
that it makes
moreintroduction
sense to work
withLebesgue
the weak form where
se definition
this
space
either
the
of
the
7
derivatives
are
“balanced”
integration
byabout,
parts,good
i.e. for you!
plying someZlimiting ideas. If you knowbywhat
this is all
b
go on: for most functions
you know you will
0 0
0 b always be able to check whether
(
v + 4⇡⇢v) dx + v|a = 0the integral and seeing if it
o this space aor not by computing or estimating
t.
•
need to be integrable, so we require the space of test
nd space is Thus,
one ofalso
the derivatives
wide familyofofvSobolev
spaces:
functions to be (there is much more theory behind the requirements, actually):
Ø
n
o
Ø @u @u
2
H 1 (≠) = u 2 L2 (≠) Ø @x
,
2
L
(≠) .
1 @x2
• An
rm related
to important
this space subspace of H1 are those test functions that satisfy the
Dirichlet BCs:
!1/2
Ø2 Z Ø
Ø2 Z
µZ
∂1/2 √Z Ø
Z
Ø @u Ø
Ø @u Ø
21
2
Ø + Ø
Ø + |u|2
1Ø
=
|ru|
+
|u|
=
.
H
(⌦)
=
v
2
H
(⌦)|v(x)
=
0
for
x
=
a,
b
Ø
Ø
Ø
Ø
D ≠
≠
≠ @x1
≠ @x2
≠
his norm is called the energy norm and functions that have this norm finite
Final weak formulation
•
Precise statement of the weak formulation: Find ϕ such that for all
v in H1, the following holds:
1
Find u 2 H D (⌦), such that
Z b
( 0 v 0 + 4⇡⇢v) dx = 0, for all v 2 H 1D (⌦)
a
1
•
Of course there are infinitely many functions in H
Then we can write:
(x) =
1
X
i vi (x)
N
X
i vi (x)
𝛤D.
But suppose we know a basis vi .
i=1
•
Suppose we truncate the series after N terms. Then we obtain an
approximation (this is the finite element approximation)
(x) ⇡
i=1
FE approximation
•
(x) =
Plugging
1
X
i vi (x)
into the weak form we get:
i=1
Find coefficients i such that
!
!
Z b
N
X
0
0
i vi (x) v (x) + 4⇡⇢(x)v(x) dx = 0,
a
•
i=1
for all v 2 H 1D (⌦)
What about those v’s? Since we have a basis and the problem is linear in v and
v’, we can just write the equivalent problem:
Find coefficients i such that
!
!
Z b
N
X
0
0
v
(x)
v
i i
j (x) + 4⇡⇢(x)vj (x) dx = 0,
a
i=1
for j = 1, .., N
FE approximation
Find coefficients i such that
!
!
Z b
N
X
0
0
v
(x)
v
i i
j (x) + 4⇡⇢(x)vj (x) dx = 0,
a
•
for j = 1, .., N
i=1
Rearranging the terms a bit, the equation can be written as: Solve
A ~ = ~b
with
~=(
•
1,
2 , ...,
N),
Aij =
Z
b
a
vi0 (x)vj0 (x)dx , bi = 4⇡
Z
b
⇢(x)vi (x)dx
a
Since we need to invert a matrix, it makes sense to choose the basis such that most
entries in A are zero! This is the case if different vi have only very small support (hence
the name “finite elements”)
Finite elements
•
The choice of basis functions depends on the problem (continuity of basis functions
=> continuity of solution)
•
For our problem, partition the interval [a,b] into arbitrarily spaced mesh
points xk. A good choice are the hat functions
Aij =
Z
b
a
vi0 (x)vj0 (x)dx =
8
>
>
>
<
xi
1
xi
1
1
xi+1 xi
1
xi xi 1
>
>
>
: 0
+
1
xi+1 xi
for i = j
for i = j 1
for i = j + 1
otherwise
Finite elements: remarks
•
With this choice, A is a tridiagonal matrix and is easily inverted!
•
Vector b depends on the problem (charge distribution), but can easily be (analytically)
calculated.
•
In this example, we assumed Dirichlet BCs with
We can have arbitrary Dirichlet BCs (a) =
if we choose
N
X
(x) = a (1 x) + b x +
i vi (x)
i=1
(a) = (b) = 0
a,
(b) =
b
ago with theoretical more than numerical intentions. In the jargon
them triangular Lagrange finite elements of order one, or simply
or for short (because using initials and short names helps speaking
e dynamic) P1 elements.
Higher dimensions
ctions
on a triangle
• In principle the same: Start from weak form, then choose a finite
function space
k for a moment about linear functions. A linear function1 of twop
• Simplest choice
in 2D:of
(linear)
polynomials.
s a polynomial
function
degree
at mostEach
one
choice of coefficients ai denotes a different function:
p3
T
• Instead of “indexing” all such functions by their coefficients, we
p1
that each p(x1,x2)
is uniquely defined
by its value
at three
tions isnote
denoted
P1 . Everybody
knows
that
a linear function is
p(x1 , x2 ) = a0 + a1 x1 + a2 x2 .
points of a triangle T
y its values
on three diÆerent non–aligned points, that is, on the
• We can thus “store” the values p1,p2,p3 of p at each
enerate)
triangle.
vertex 1,2,3.
n arbitrary
non–degenerate
triangle,
that
we
call
K.
You
might
• If we split our domain into a triangulation, and define p on
ngle T each
, astriangle
manybypeople
However,
onwe(in Lesson 3) the
the value do.
it has on
the triangle later
vertices,
obtain aand
continuous
function! something else, maybe a quadrilateral,
g a triangle
will become
of the
initial isT uniquely
will bedefined
lost. We
drawp1,….,pN
it as inatFigure
1.2, marking
• Function
by values
N
vertices
h this we
mean that a function
p2
8
17
4
13
Higher dimensions
9
Figure 1.7: Global numbering of nodes
vi (x1 , x2 )
•
•
The i-th
basis1.8:
function
vi withoflocal
support
obtained
setting
pj =tent.
0 for
Figure
The graph
a nodal
basis is
function:
it by
looks
like a camping
As in the 1D case, we can now approximate a
restricted to each triangle it is a polynomial (or smooth) function. Then
function Φ of 2 variables as
1
N
u
2
H
(≠)
h
X
()
j 6= i
uh is continuous.
(x1is, certain
x2 ) =intuition toi vbei (x
, xwhy
2 ) this result is true. If you take a derivative of a
There
had1on
piecewise smooth function,
i=1 you obtain Dirac distributions along the lines where there are
discontinuities. Dirac distributions are not functions and it does not make sense to see if
the are square–integrable or not. Therefore, if there are discontinuities, the function fails
to have a square–integrable gradient.
2.4
Dirichlet nodes
Quadrature/Numerical
integration
•
Already in the 1D case we had to integrate “element-wise” integrals of the form
Aij =
Z
b
a
vi0 (x)vj0 (x)dx
bi = 4⇡
Z
b
⇢(x)vi (x)dx
a
•
Although we did this analytically for the 1D case, one typically integrates numerically
(allowing, e.g., more complicated basis functions than linear ones, and arbitrary 𝜌(x).
•
Fundamental theorem (Gauss quadrature): If f(x) is (close to) a polynomial of
degree 2n-1, then there exist optimal weights wi and locations xi such that
•
I.e. if the basis functions are polynomials, the entries Aij can be exactly evaluated
as a finite sum!
Elements of finite elements
•
1. Weak form (you!)
•
2. Domain “meshing”: Split domain into elements (arbitrary
domains possible, unstructured mesh, mix triangles with
quadrilaterals etc.
•
3. Choice of basis functions: Typically low order
polynomials
•
4. Numerical integration / Gauss quadrature
implementation
•
5. Linear/nonlinear solvers (matrix inversion/root finding)
Finite element advantages
•
Can easily adapt mesh (=> introduce new/remove unneeded basis
functions (“elements”))
Finite element advantages
•
Can solve higher order Zproblems byZ reducing order:
2
4 u=f
⌦
42 uvi dA )
⌦
4u4vi dA
I.e. lower continuity requirements on test functions!
•
If you can find basis functions for curved topologies =>
can solve PDEs on arbitrarily curved surfaces
a
a
b
10
c
R/h=50
ods
nlinear partial di↵erential equations
Nonlinear finite elements
ment method can also be applied to non-linear partial di↵erential equations
ig changes. Let us consider a simple example
•
d2
(x) 2 (x) =
dx
Solve
4⇡⇢(x)
(3.41)
• Eq.
Equivalent
statement:
Find Φthe
such
that for all vi:
e ansatz,
(3.32) asweak
before
and minimizing
residuals
Z
b
g
ai
(
=
Z1 00
[
(x)
+
4⇡⇢(x))
v
(x)dx
=
0
i
00
(x) + 4⇡⇢(x)] w (x)dx
i
(3.42)
0
The same derivation as before now results in nonlinear coupled equations
ow end upX
with a nonlinear equation instead of a linear equation:
•
i,j
Aijk X
i
j
= bk
Aijk ai aj = bk
i,j
with bk as before but
•
Z1
Aijk =
(1)
Z
(3.43)
b
a
vi (x)vj00 (x)vk (x)dx
Aijk =
ui (x)u00j (x)wk (x)dx
(3.44)
We now need a nonlinear root-finding algorithm to solve (1) for ɸi.
0
as before.
di↵erence between the case of linear and nonlinear partial di↵erential
Nonlinear FE
•
How to solve
n problems
•
X
i j
= bk ?
i,j
Write as
rk ( ~ ) =
ation problems
n
et f : R • →
(−∞, ∞],
Minimize
|~r( ~ )|22
find
Aijk
X
Aijk
i j
bk
i,j
!
minn {f (x)}
x∈R
We
thus need methods to solve nonlinear problems. Quite general:
n
m: Let f : R → (−∞, ∞],
find given
x∗ s.t.
f (x∗ ) =f, we
minnneed
{f (x)}
a function
to
•
find
minn {f (x)}
x∈R
x∈R
al, but some cases,
like f convex, are fairly solvable.
find x∗ s.t. f (xn∗ ) = minn {f (x)}
blem: •How
about f : R → R,
differentiable?
x∈R
Let’s assume f is differentiable…
Then the problem becomes a root finding problem:
eneral, butfind
somexcases,
like f convex, are fairly solvable.
∗ s.t. ∇f (x∗ ) = 0
s problem: How about f : Rn → R, differentiable?
easonable shot at this, especially if f is twice
lest we can get
The simplest we can get
Nonlinear root-finding
c optimization: f (x) = 12 x t Ax − x t b + c.
common (actually universal)
or expansion
• Let’s first consider
Quadratic
optimization:
a particular
form of f: f (x) = 12 x t Ax − x t b + c.
t ∇∇f (x)∆x + · · ·
+ ∆x) = f (x) + (∆x)t ∇f (x) + 21very
(∆x)common
(actually universal)
• Interpretation: Energy, quadratic in x. We have seen these kinds of
∇f (x) = 0
energies in the contextTaylor
of CG.expansion
Here:
f (x + ∆x) = f (x) + (∆x)t ∇f (x) + 21 (∆x)t ∇∇f (x)∆x + ·
∇f (x) = Ax − b = 0
Finding
∇f (x) = 0
−1
x∗ = A b
∇f (x) = Ax − b = 0
mean •A In
has
to be
invertible?
Is this
all we need?
other
words,
quadratic
optimization
amounts to a single matrix
x∗ = A−1 b
inversion (using direct methods, iterative methods (CG etc.)).
Does this mean A has to be invertible? Is this all we need?
•
But A has to be more than invertible!
R. A. Lippert
Non-linear optimization
R. A. Lippert
Non-linear optimization