Algorithms for Uncertainty Quantification
Lecture 7: Polynomial Chaos Approximation 2. The Stochastic
Galerkin Approach
Dr. Tobias Neckel
Scientific Computing in Computer Science
TUM
ST 2024
Repetition of Previous Lecture
Polynomial chaos methods
• polynomial chaos expansion
• approximate quantity of interest by polynomial series
P
• f (t, ω) ≈ N−1
n=0 f̂n (t) ϕn (ω)
• orthogonal polynomials and polynomial chaos
• inner product 0 for orthogonal polynomials
• < ϕi (ω), ϕj (ω) >ρ = δij
• choose polynomial type according to input distribution
• the pseudo-spectral approach
• use quadrature rule to compute coefficients
P −1
• f̂n ≈ Kk=0
f (t, xk ) ϕn (xk ) wk
• model problem: damped linear oscillator
• multivariate polynomial chaos expansion
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
2
Concept of Building Block:
• Time: ≈ 90 minutes
• Content
• Stochastic Galerkin method
• Application to example of damped linear oscillator
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
3
Concept of Building Block:
• Time: ≈ 90 minutes
• Content
• Stochastic Galerkin method
• Application to example of damped linear oscillator
• Expected Learning Outcomes
• The participants can describe the basic concept of the Stochastic
Galerkin method and its individual steps.
• They are able to apply it to simple model problems similar to the
oscillator example. In particular, they can represent gPC expansions
of one-dimensional uniform and normal input parameters and can
derive the modified model problem for the stochastic Galerkin
approach for new applications.
• They can list and explain the advantages and drawbacks of
stochastic Galerkin compared to the pseudo-spectral approach.
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
3
Agenda
Topic
Stochastic Galerkin method
Content
• forward propagation of uncertainty
• idea of stochastic Galerkin method
• Galerkin projection
• example: damped linear oscillator
• comparison with non-intrusive methods
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
4
Forward Propagation of Uncertainty
deterministic
model f (t, ω)
stochastic input Ω
stochastic output Y
What we have
• deterministic model with solution f (t, ω)
• random input variable Ω ∼ ρ(ω)
• corresponding orthogonal polynomials ϕi (ω)
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
5
Forward Propagation of Uncertainty
deterministic
model f (t, ω)
stochastic input Ω
stochastic output Y
What we have
• deterministic model with solution f (t, ω)
• random input variable Ω ∼ ρ(ω)
• corresponding orthogonal polynomials ϕi (ω)
What we want
• stochastic output f (t, ω) = Y ∼ p(Y )
• quantities of interest: e.g. E[Y ], Var [Y ]
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
5
Forward Propagation of Uncertainty (2)
deterministic
model f (t, ω)
stochastic input Ω
stochastic output Y
Which method to use?
• remember: pseudo-spectral approach
• write f (t, ω) as gPC expansion
• use quadrature rule to compute coefficients
• quadrature introduces error
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
6
Stochastic Galerkin Method
remember: polynomial chaos expansion
f (t, ω) ≈
N−1
X
f̂n (t) ϕn (ω)
n=0
Idea
• do not rely on quadrature
• requires the polynomial chaos expansion of the uncertain inputs
• modify solver implementation to compute coefficients f̂n (t)
Properties
• faster convergence than the pseudo-spectral approach
• requires access to model/equations/code
• time-consuming modifications necessary
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
7
Galerkin Projection
Analogy: Finite Elements
• formulate problem in weak form + discretize in space
• assumption: solution u is weighted sum of base of shape functions Nn
u(x) =
X
ûn Nn (x)
n
• find best approximation to real solution
→ solve for coefficients ûn
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
8
Galerkin Projection
Analogy: Finite Elements
• formulate problem in weak form + discretize in space
• assumption: solution u is weighted sum of base of shape functions Nn
u(x) =
X
ûn Nn (x)
n
• find best approximation to real solution
→ solve for coefficients ûn
Stochastic Galerkin method
• solution: displacement u(x) → stochastic model output f (t, ω)
• local shape functions Nn (x) → global orthogonal polynomials ϕn (ω)
• coefficients ûn → coefficients f̂n (t)
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
8
Stochastic Galerkin Method – Steps
Steps
1. determine the polynomial chaos expansion of the uncertain inputs (this
expansion is exact!)
2. write the underlying model’s solution as an N th order polynomial chaos
expansion
Ω=
PM−1
f (t, ω) ≈
m=0 ĉn ϕm (ω)
PN−1
n=0
f̂n (t) ϕn (ω)
Stochastic Galerkin Method – Steps
Steps
1. determine the polynomial chaos expansion of the uncertain inputs (this
expansion is exact!)
2. write the underlying model’s solution as an N th order polynomial chaos
expansion
3. insert both expansions into model equations
Ω=
PM−1
f (t, ω) ≈
m=0 ĉn ϕm (ω)
PN−1
n=0
f̂n (t) ϕn (ω)
mathem.
model
Stochastic Galerkin Method – Steps
Steps
1. determine the polynomial chaos expansion of the uncertain inputs (this
expansion is exact!)
2. write the underlying model’s solution as an N th order polynomial chaos
expansion
3. insert both expansions into model equations
4. use orthogonality to get a system of equations with N unknown
coefficients
Ω=
PM−1
f (t, ω) ≈
m=0 ĉn ϕm (ω)
PN−1
n=0
f̂n (t) ϕn (ω)
mathem.
modified
model < ϕn , ϕj >= model
δnj
Stochastic Galerkin Method – Steps
Steps
1. determine the polynomial chaos expansion of the uncertain inputs (this
expansion is exact!)
2. write the underlying model’s solution as an N th order polynomial chaos
expansion
3. insert both expansions into model equations
4. use orthogonality to get a system of equations with N unknown
coefficients
5. modify solver to solve new (coupled) system of equations
Ω=
PM−1
f (t, ω) ≈
m=0 ĉn ϕm (ω)
PN−1
n=0
f̂n (t) ϕn (ω)
mathem.
modified
model < ϕn , ϕj >= model
δnj
modified
solver
Stochastic Galerkin Method – Steps
Steps
1. determine the polynomial chaos expansion of the uncertain inputs (this
expansion is exact!)
2. write the underlying model’s solution as an N th order polynomial chaos
expansion
3. insert both expansions into model equations
4. use orthogonality to get a system of equations with N unknown
coefficients
5. modify solver to solve new (coupled) system of equations
6. compute statistical properties from coefficients
Ω=
PM−1
f (t, ω) ≈
m=0 ĉn ϕm (ω)
PN−1
n=0 f̂n (t) ϕn (ω)
mathem.
modified
model < ϕn , ϕj >= model
δnj
modified
solver
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
f̂n (t)
9
Model Problem: Damped Linear Oscillator
System of first order ODEs
 dx

dt (t) = v (t)


 dv (t) = f cos(ω t) − cv (t) − kx(t)
O
dt

x(0)
=
x
0



v (0) = v0
• x(t): position, x0 : initial position
• v (t): velocity, v0 : initial velocity
• c – damping coefficient
• k – spring constant
• f – forcing amplitude
• ωO – forcing frequency
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
10
Model Problem – Uncertainty in Input Parameters
Uncertain parameter: damping constant c
• assume c now as RV C ∼ U(a, b)
• linear transformation with Ω ∼ U(−1, 1)
c(ω) =
a+b b−a
+
ω
2 } | {z
2 }
| {z
cµ
cσ
• polynomial chaos basis: legendre polynomials ϕi (ω) (orthogonal w.r.t
Uniform distribution)
• polynomial chaos expansion:
c = cµ + cσ ω
= cµ ϕ0 (ω) + cσ ϕ1 (ω)
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
11
Model Problem – Polynomial Chaos Expansion
Polynomial chaos expansions
x(t, ω) =
N−1
X
x̂n (t) ϕn (ω)
n=0
v (t, ω) =
N−1
X
v̂n (t) ϕn (ω)
n=0
• note: coefficients depend on t, polynomials on ω
• notation from now on: ϕn (ω) → ϕn ,
x̂n (t) → x̂n ,
v̂n (t) → v̂n
• 2 steps:
1. insert expansions into ODEs and IC
2. transform system of equations via Galerkin ansatz and orthogonality
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
12
Model Problem – Initial Conditions
1. insert expansions into IC (analoguously for v0 )
x(0) = x0
N−1
X
x̂n (0) ϕn = x0
n=0
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
13
Model Problem – Initial Conditions
1. insert expansions into IC (analoguously for v0 )
x(0) = x0
N−1
X
x̂n (0) ϕn = x0
n=0
2. use Galerkin + orthogonality: inner product with < ·, ϕj >
<
N−1
X
x̂n (0) ϕn , ϕj > = < x0 , ϕj >
n=0
N−1
X
n=0
x̂n (0) < ϕn , ϕj > = x0 < ϕ0 , ϕj >
| {z }
| {z }
δnj γj
x̂j (0) = δ0j x0
δ0j
∀ j = 0, . . . , N − 1
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
13
Model Problem – 1st ODE Component (x)
1. insert expansions into ODE
d
x =v
dt
N−1
N−1
X
d X
x̂n ϕn =
v̂n ϕn
dt
n=0
n=0
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
14
Model Problem – 1st ODE Component (x)
1. insert expansions into ODE
d
x =v
dt
N−1
N−1
X
d X
x̂n ϕn =
v̂n ϕn
dt
n=0
n=0
2. use Galerkin + orthogonality: inner product with < . . . , ϕj >
N−1
N−1
X
d X
<
x̂n ϕn , ϕj > = <
v̂n ϕn , ϕj >
dt
n=0
d
dt
N−1
X
n=0
n=0
N−1
X
x̂n < ϕn , ϕj > =
v̂n < ϕn , ϕj >
| {z } n=0
| {z }
δnj γj
d
x̂j = v̂j
dt
δnj γj
∀ j = 0, . . . , N − 1
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
14
Model Problem – 2nd ODE Component (v )
1. insert expansions into ODE
d
v =f cos(ωO t) − c v − k x
dt
N−1
N−1
N−1
X
X
d X
v̂n ϕn =f cos(ωO t) − (cµ ϕ0 + cσ ϕ1 )
v̂n ϕn − k
x̂n ϕn
dt
n=0
n=0
=f cos(ωO t) − cµ ϕ0
|{z}
=1
N−1
X
n=0
v̂n ϕn − cσ
n=0
N−1
X
v̂n ϕ1 ϕn − k
n=0
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
N−1
X
x̂n ϕn
n=0
15
Model Problem – 2nd ODE Component (v ) (cont’d)
2. use orthogonality: inner product with < ·, ϕj >
N−1
<
N−1
X
d X
v̂n ϕn , ϕj > = < f cos(ωO t) , ϕj > − < cµ
v̂n ϕn , ϕj >
dt
n=0
n=0
− < cσ
N−1
X
n=0
v̂n ϕ1 ϕn , ϕj > − < k
N−1
X
x̂n ϕn , ϕj >
n=0
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
16
Model Problem – 2nd ODE Component (v ) (cont’d)
2. use orthogonality: inner product with < ·, ϕj >
N−1
<
N−1
X
d X
v̂n ϕn , ϕj > = < f cos(ωO t) , ϕj > − < cµ
v̂n ϕn , ϕj >
dt
n=0
n=0
− < cσ
N−1
X
v̂n ϕ1 ϕn , ϕj > − < k
n=0
N−1
N−1
X
x̂n ϕn , ϕj >
n=0
N−1
X
d X
v̂n < ϕn , ϕj > = f cos(ωO t) < ϕ0 , ϕj > −cµ
v̂n < ϕn , ϕj >
dt
n=0
n=0
− cσ
N−1
X
n=0
v̂n < ϕ1 ϕn , ϕj > −k
N−1
X
x̂n < ϕn , ϕj >
n=0
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
16
Model Problem – 2nd ODE Component (v ) (cont’d)
2. use orthogonality: inner product with < ·, ϕj >
N−1
N−1
X
d X
v̂n < ϕn , ϕj >
v̂n < ϕn , ϕj > = f cos(ωO t) < ϕ0 , ϕj > −cµ
dt
| {z }
| {z }
| {z }
n=0
δnj γj
δ0j
− cσ
N−1
X
n=0
n=0
v̂n < ϕ1 ϕn , ϕj > −k
δnj γj
N−1
X
n=0
x̂n < ϕn , ϕj >
| {z }
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
δnj γj
17
Model Problem – 2nd ODE Component (v ) (cont’d)
2. use orthogonality: inner product with < ·, ϕj >
N−1
N−1
X
d X
v̂n < ϕn , ϕj >
v̂n < ϕn , ϕj > = f cos(ωO t) < ϕ0 , ϕj > −cµ
dt
| {z }
| {z }
| {z }
n=0
δnj γj
δ0j
− cσ
N−1
X
n=0
v̂n < ϕ1 ϕn , ϕj > −k
n=0
δnj γj
N−1
X
n=0
x̂n < ϕn , ϕj >
| {z }
δnj γj
⇒
N−1
X
d
v̂j γj = f cos(ωO t) δ0j − cµ v̂j γj − cσ
v̂n < ϕ1 ϕn , ϕj > −k x̂j γj
dt
n=0
∀j = 0, . . . , N − 1
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
17
Model Problem – Stochastic Galerkin System
Final IVP
• modifications leads to new IVP
• similar to original IVP
• 2 coupled ODEs → 2N coupled ODEs
• modified model solver can solve for x̂j , v̂j

d


dt x̂j = v̂j


 d v̂ = δ 1 f cos(ω t) − c v̂ − k x̂ − c PN−1 v̂ <ϕ1 ϕn , ϕj >
µ j
σ
0j γj
j
O
n=0 n
dt j
γj

x̂
(0)
=
δ
x
 j
0j 0


v̂ (0) = δ v
∀ j = 0, . . . , N − 1
j
0j
0
• expectation and variance computed as in pseudo-spectral approach
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
18
Model Problem – Stochastic Galerkin Results
Results
• C ∼ U(0.08, 0.12)
• T = 15
• deterministic result: x(T ) = −1.51e − 01
• stochastic Galerkin method, 3 coefficients:
E[x(T )] = −1.52e − 01, Var[x(T )] = 7.80e − 04
• pseudo-spectral approach 5 nodes:
E[x(T )] = −1.52e − 01, Var[x(T )] = 7.80e − 04
• Monte Carlo sampling, 100000 samples:
E[x(T )] = −1.53e − 01, Var[x(T )] = 7.83e − 04
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
19
Model Problem – Stochastic Galerkin Results
Results
• C ∼ U(0.08, 0.12)
• T = 15
• deterministic result: x(T ) = −1.51e − 01
• stochastic Galerkin method, 3 coefficients:
E[x(T )] = −1.52e − 01, Var[x(T )] = 7.80e − 04
• pseudo-spectral approach 5 nodes:
E[x(T )] = −1.52e − 01, Var[x(T )] = 7.80e − 04
• Monte Carlo sampling, 100000 samples:
E[x(T )] = −1.53e − 01, Var[x(T )] = 7.83e − 04
Comparison with pseudo-spectral approach
• difference in E[x(T )]: 2e − 10
• difference in Var[x(T )]: 1e − 9
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
19
Comparison with Pseudo-spectral Approach
stochastic Galerkin
pseudo-spectral approach
• intrusive: need to modify model
→ model access required
→ redo for each model
• coefficients computed from a
system of (coupled) ODEs
/PDEs, no quadrature error
• non-intrusive: model treated as
black box
→ only model output required
→ can reuse code
• coefficients approximated
numerically via quadrature
• modeling error:
• series truncation
• modeling error
• series truncation
• quadrature
⇒ more accurate
⇒ easier to use
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
20
Comparison with Pseudo-spectral Approach
stochastic Galerkin
pseudo-spectral approach
• intrusive: need to modify model
→ model access required
→ redo for each model
• coefficients computed from a
system of (coupled) ODEs
/PDEs, no quadrature error
• non-intrusive: model treated as
black box
→ only model output required
→ can reuse code
• coefficients approximated
numerically via quadrature
• modeling error:
• series truncation
• modeling error
• series truncation
• quadrature
⇒ more accurate
⇒ easier to use
Conclusion
• stochastic Galerkin method requires much more work
• accuracy gain must be “worth it”
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
20
Literature
• R. Ghanem, P. Spanos: Stochastic Finite Elements: A Spectral
Approach, Springer New York, 1991
• Chapter 10 of R. C. Smith: Uncertainty Quantification – Theory,
Implementation, and Applications, SIAM, 2014
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
21
Summary
Stochastic Galerkin method
• idea
• insert polynomial expansions into model
• modify model to compute coefficients simultaneously
• Galerkin projection like in FEM
• comparison with non-intrusive methods
• needs model modifications
• good convergence properties
• example: damped linear oscillator
Dr. Tobias Neckel | Algorithms for UQ | L7: PC approx. 2: stoch. Galerkin approach | ST 2024
22