Algorithms for Uncertainty Quantification

Lecture 7: Polynomial Chaos Approximation 2: The Stochastic

Galerkin Approach

ST 2018

Tobias Neckel
Scientific Computing in Computer Science
TUM
Repetition of Previous Lecture

Polynomial chaos methods

• polynomial chaos expansion
• approximate quantity of interest by polynomial series
PN−1
• f (t, ω) ≈ n=0 f̂n (t) φn (ω)
• orthogonal polynomials and polynomial chaos
• inner product 0 for orthogonal polynomials
• < φi (ω), φj (ω) >ρ = δij
• choose polynomial type according to input distribution
• the pseudo-spectral approach
• use quadrature rule to compute coefficients
PK −1
• f̂n ≈ k =0 f (t, xk ) φn (xk ) wk
• model problem: damped linear oscillator
• multivariate polynomial chaos expansion

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 2
Concept of Building Block:

• Time: ≈ 90 minutes
• Content
• Stochastic Galerkin method
• Application to example of damped linear oscillator

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 3
Concept of Building Block:

• Time: ≈ 90 minutes
• Content
• Stochastic Galerkin method
• Application to example of damped linear oscillator
• Expected Learning Outcomes
• The participants can describe the basic concept of the Stochastic
Galerkin method and its individual steps.
• They are able to apply it to simple model problems similar to the
oscillator example. In particular, they can represent gPC expansions
of one-dimensional uniform and normal input parameters and can
derive the modified model problem for the stochastic Galerkin
approach for new applications.
• They can list and explain the advantages and drawbacks of
stochastic Galerkin compared to the pseudo-spectral approach.

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 3
Agenda

Topic
Stochastic Galerkin method

Content
• forward propagation of uncertainty
• idea of stochastic Galerkin method
• Galerkin projection
• example: damped linear oscillator
• comparison with non-intrusive methods

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 4
Forward Propagation of Uncertainty

deterministic
model f (t, ω)

stochastic input Ω stochastic output Y

What we have
• deterministic model with solution f (t, ω)
• random input variable Ω ∼ ρ(ω)
• corresponding orthogonal polynomials φi (ω)

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 5
Forward Propagation of Uncertainty

deterministic
model f (t, ω)

stochastic input Ω stochastic output Y

What we have
• deterministic model with solution f (t, ω)
• random input variable Ω ∼ ρ(ω)
• corresponding orthogonal polynomials φi (ω)

What we want
• stochastic output f (t, ω) = Y ∼ p(Y )
• quantities of interest: e.g. E[Y ], Var [Y ]
T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 5
Forward Propagation of Uncertainty (2)

deterministic
model f (t, ω)

stochastic input Ω stochastic output Y

Which method to use?

• remember: pseudo-spectral approach
• write f (t, ω) as gPC expansion
• use quadrature rule to compute coefficients
• quadrature introduces error

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 6
Stochastic Galerkin Method
remember: polynomial chaos expansion
N−1
X
f (t, ω) ≈ f̂n (t) φn (ω)
n=0

Idea
• do not rely on quadrature
• requires the polynomial chaos expansion of the uncertain inputs
• modify solver implementation to compute coefficients f̂n (t)

Properties
• faster convergence than the pseudo-spectral approach
• requires access to model/equations/code
• time-consuming modifications necessary

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 7
Galerkin Projection
Analogy: Finite Elements
• formulate problem in weak form + discretize in space
• assumption: solution u is weighted sum of base of shape functions Nn
X
u(x) = ûn Nn (x)
n

• find best approximation to real solution

→ solve for coefficients ûn

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 8
Galerkin Projection
Analogy: Finite Elements
• formulate problem in weak form + discretize in space
• assumption: solution u is weighted sum of base of shape functions Nn
X
u(x) = ûn Nn (x)
n

• find best approximation to real solution

→ solve for coefficients ûn

Stochastic Galerkin method

• solution: displacement u(x) → stochastic model output f (t, ω)
• local shape functions Nn (x) → global orthogonal polynomials φn (ω)
• coefficients ûn → coefficients f̂n (t)

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 8
Stochastic Galerkin Method – Steps
Steps
1. determine the polynomial chaos expansion of the uncertain inputs (this
expansion is exact!)
2. write the underlying model’s solution as an N th order polynomial chaos
expansion

PM−1
Ω= m=0 ĉn φm (ω)
PN−1
f (t, ω) = n=0 f̂n (t) φn (ω)
Stochastic Galerkin Method – Steps
Steps
1. determine the polynomial chaos expansion of the uncertain inputs (this
expansion is exact!)
2. write the underlying model’s solution as an N th order polynomial chaos
expansion
3. insert both expansions into model equations

PM−1
Ω= m=0 ĉn φm (ω)
mathem.
PN−1 model
f (t, ω) = n=0 f̂n (t) φn (ω)
Stochastic Galerkin Method – Steps
Steps
1. determine the polynomial chaos expansion of the uncertain inputs (this
expansion is exact!)
2. write the underlying model’s solution as an N th order polynomial chaos
expansion
3. insert both expansions into model equations
4. use orthogonality to get a system of equations with N unknown
coefficients

PM−1
Ω= m=0 ĉn φm (ω)
mathem. modified
PN−1 model < φn , φj >= model
f (t, ω) = n=0 f̂n (t) φn (ω)
δnj
Stochastic Galerkin Method – Steps
Steps
1. determine the polynomial chaos expansion of the uncertain inputs (this
expansion is exact!)
2. write the underlying model’s solution as an N th order polynomial chaos
expansion
3. insert both expansions into model equations
4. use orthogonality to get a system of equations with N unknown
coefficients
5. modify solver to solve new (coupled) system of equations

PM−1
Ω= m=0 ĉn φm (ω)
mathem. modified modified
PN−1 model < φn , φj >= model solver
f (t, ω) = n=0 f̂n (t) φn (ω)
δnj
Stochastic Galerkin Method – Steps
Steps
1. determine the polynomial chaos expansion of the uncertain inputs (this
expansion is exact!)
2. write the underlying model’s solution as an N th order polynomial chaos
expansion
3. insert both expansions into model equations
4. use orthogonality to get a system of equations with N unknown
coefficients
5. modify solver to solve new (coupled) system of equations
6. compute statistical properties from coefficients
PM−1
Ω= m=0 ĉn φm (ω)
mathem. modified modified
f̂n (t)
PN−1 model < φn , φj >= model solver
f (t, ω) = n=0 f̂n (t) φn (ω)
δnj

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 9
Model Problem: Damped Linear Oscillator

System of first order ODEs

 dx

 dt (t) = v (t)
 dv (t) = f cos(ω t) − cv (t) − kx(t)

dt O


 x(0) = x 0
v (0) = v0


• x(t): position, x0 : initial position

• v (t): velocity, v0 : initial velocity
• c – damping coefficient
• k – spring constant
• f – forcing amplitude
• ωO – forcing frequency

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 10
Model Problem – Uncertainty Input Parameters
Uncertain parameter: damping constant c
• assume c now as RV C ∼ U(a, b)
• linear transformation with Ω ∼ U(−1, 1)

a+b b−a
c(ω) = + ω
2 } | {z
| {z 2 }
cµ cσ

• polynomial chaos basis: legendre polynomials φi (ω)

• polynomial chaos expansion:

c = cµ + cσ ω
= cµ φ0 (ω) + cσ φ1 (ω)

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 11
Model Problem – Polynomial Chaos Expansion
Polynomial chaos expansions

N−1
X
x(t, ω) = x̂n (t) φn (ω)
n=0
N−1
X
v (t, ω) = v̂n (t) φn (ω)
n=0

• note: coefficients depend on t, polynomials on ω

• notation from now on: φn (ω) → φn , x̂n (t) → x̂n , v̂n (t) → v̂n
• 2 steps:
1. insert expansions into ODEs and IC
2. transform system of equations via Galerkin ansatz and orthogonality

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 12
Model Problem – Initial Conditions
1. insert expansions into IC
x(0) = x0
N−1
X
x̂n (0) φn = x0
n=0

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 13
Model Problem – Initial Conditions
1. insert expansions into IC
x(0) = x0
N−1
X
x̂n (0) φn = x0
n=0

2. use Galerkin + orthogonality: inner product with < ·, φj >

N−1
X
< x̂n (0) φn , φj > = < x0 , φj >
n=0
N−1
X
x̂n (0) < φn , φj > = x0 < φ0 , φj >
n=0
| {z } | {z }
δnj γj δ0j

x̂j (0) = δ0j x0 ∀ j = 0, . . . , N − 1

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 13
Model Problem – 1st ODE Component (x)
1. insert expansions into ODE
d
x =v
dt
N−1 N−1
d X X
x̂n φn = v̂n φn
dt
n=0 n=0

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 14
Model Problem – 1st ODE Component (x)
1. insert expansions into ODE
d
x =v
dt
N−1 N−1
d X X
x̂n φn = v̂n φn
dt
n=0 n=0

2. use Galerkin + orthogonality: inner product with < . . . , φj >

N−1 N−1
d X X
< x̂n φn , φj > = < v̂n φn , φj >
dt
n=0 n=0
N−1 N−1
d X X
x̂n < φn , φj > = v̂n < φn , φj >
dt | {z } | {z }
n=0 n=0
δnj γj δnj γj

d
x̂j = v̂j ∀ j = 0, . . . , N − 1
dt
T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 14
Model Problem – 2nd ODE Component (v )

1. insert expansions into ODE

d
v =f cos(ωO t) − c v − k x
dt
N−1 N−1 N−1
d X X X
v̂n φn =f cos(ωO t) − (cµ φ0 + cσ φ1 ) v̂n φn − k x̂n φn
dt
n=0 n=0 n=0
N−1
X N−1
X N−1
X
=f cos(ωO t) − cµ φ0 v̂n φn − cσ v̂n φ1 φn − k x̂n φn
|{z}
=1 n=0 n=0 n=0

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 15
Model Problem – 2nd ODE Component (v ) (cont’d)
2. use orthogonality: inner product with < ·, φj >
N−1 N−1
d X X
< v̂n φn , φj > = < f cos(ωO t) , φj > − < cµ v̂n φn , φj >
dt
n=0 n=0
N−1
X N−1
X
− < cσ v̂n φ1 φn , φj > − < k x̂n φn , φj >
n=0 n=0

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 16
Model Problem – 2nd ODE Component (v ) (cont’d)
2. use orthogonality: inner product with < ·, φj >
N−1 N−1
d X X
< v̂n φn , φj > = < f cos(ωO t) , φj > − < cµ v̂n φn , φj >
dt
n=0 n=0
N−1
X N−1
X
− < cσ v̂n φ1 φn , φj > − < k x̂n φn , φj >
n=0 n=0

N−1 N−1
d X X
v̂n < φn , φj > = f cos(ωO t) < φ0 , φj > −cµ v̂n < φn , φj >
dt
n=0 n=0
N−1
X N−1
X
− cσ v̂n < φ1 φn , φj > −k x̂n < φn , φj >
n=0 n=0

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 16
Model Problem – 2nd ODE Component (v ) (cont’d)
2. use orthogonality: inner product with < ·, φj >
N−1 N−1
d X X
v̂n < φn , φj > = f cos(ωO t) < φ0 , φj > −cµ v̂n < φn , φj >
dt | {z } | {z } | {z }
n=0 n=0
δnj γj δ0j δnj γj
N−1
X N−1
X
− cσ v̂n < φ1 φn , φj > −k x̂n < φn , φj >
n=0 n=0
| {z }
δnj γj

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 17
Model Problem – 2nd ODE Component (v ) (cont’d)
2. use orthogonality: inner product with < ·, φj >
N−1 N−1
d X X
v̂n < φn , φj > = f cos(ωO t) < φ0 , φj > −cµ v̂n < φn , φj >
dt | {z } | {z } | {z }
n=0 n=0
δnj γj δ0j δnj γj
N−1
X N−1
X
− cσ v̂n < φ1 φn , φj > −k x̂n < φn , φj >
n=0 n=0
| {z }
δnj γj

⇒
N−1
d X
v̂j γj = f cos(ωO t) δ0j − cµ v̂j γj − cσ v̂n < φ1 φn , φj > −k x̂j γj
dt
n=0
∀j = 0, . . . , N − 1

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 17
Model Problem – Stochastic Galerkin System

Final IVP
• modifications leads to new IVP
• similar to original IVP
• 2 coupled ODEs → 2N coupled ODEs
• modified model solver can solve for x̂j , v̂j

d
 dt x̂j = v̂j


 d v̂ = δ 1 f cos(ω t) − c v̂ − k x̂ − c PN−1 v̂
 <φ1 φn , φj >
dt j 0j γj O µ j j σ n=0 n γj


x̂j (0) = δ0j x0

v̂ (0) = δ v
j 0j ∀ j = 0, . . . , N − 1
0

• expectation and variance computed as in pseudo-spectral approach

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 18
Model Problem – Stochastic Galerkin Results
Results
• C ∼ U(0.08, 0.12)
• T = 15
• deterministic result: x(T ) = −1.51e − 01
• stochastic Galerkin method, 3 coefficients:
E[x(T )] = −1.52e − 01, Var[x(T )] = 7.80e − 04
• pseudo-spectral approach 5 nodes:
E[x(T )] = −1.52e − 01, Var[x(T )] = 7.80e − 04
• Monte Carlo sampling, 100000 samples:
E[x(T )] = −1.53e − 01, Var[x(T )] = 7.83e − 04

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 19
Model Problem – Stochastic Galerkin Results
Results
• C ∼ U(0.08, 0.12)
• T = 15
• deterministic result: x(T ) = −1.51e − 01
• stochastic Galerkin method, 3 coefficients:
E[x(T )] = −1.52e − 01, Var[x(T )] = 7.80e − 04
• pseudo-spectral approach 5 nodes:
E[x(T )] = −1.52e − 01, Var[x(T )] = 7.80e − 04
• Monte Carlo sampling, 100000 samples:
E[x(T )] = −1.53e − 01, Var[x(T )] = 7.83e − 04

Comparison with pseudo-spectral approach

• difference in E[x(T )]: 2e − 10
• difference in Var[x(T )]: 1e − 9

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 19
Comparison with Pseudo-spectral Approach
stochastic Galerkin pseudo-spectral approach

• intrusive: need to modify model • non-intrusive: model treated as

→ model access required black box
→ redo for each model → only model output required
• coefficients computed from a → can reuse code
system of (coupled) ODEs • coefficients approximated
/PDEs, no quadrature error numerically via quadrature
• modeling error: • modeling error
• series truncation • series truncation
• quadrature
⇒ more accurate ⇒ easier to use

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 20
Comparison with Pseudo-spectral Approach
stochastic Galerkin pseudo-spectral approach

• intrusive: need to modify model • non-intrusive: model treated as

→ model access required black box
→ redo for each model → only model output required
• coefficients computed from a → can reuse code
system of (coupled) ODEs • coefficients approximated
/PDEs, no quadrature error numerically via quadrature
• modeling error: • modeling error
• series truncation • series truncation
• quadrature
⇒ more accurate ⇒ easier to use
Conclusion
• stochastic Galerkin method requires much more work
• accuracy gain must be “worth it”
T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 20
Literature

• R. Ghanem, P. Spanos: Stochastic Finite Elements: A Spectral

Approach, Springer New York, 1991
• Chapter 10 of R. C. Smith: Uncertainty Quantification – Theory,
Implementation, and Applications, SIAM, 2014

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 21
Summary

Stochastic Galerkin method

• idea
• insert polynomial expansions into model
• modify model to compute coefficients
• Galerkin projection like in FEM
• comparison with non intrusive methods
• needs model modifications
• good convergence properties
• example: damped linear oscillator

T. Neckel | Algorithms for Uncertainty Quantification | L7: PC approx. 2: stoch. Galerkin approach | ST 2018 22