STOCHASTIC MODELING OF
MULTISCALE SYSTEMS
NICHOLAS ZABARAS
Materials Process Design and Control Laboratory
Sibley School of Mechanical & Aerospace Engineering
188 Frank H T Rhodes Hall
Cornell University
Ithaca, NY 14853
Email: zabaras@cornell.edu
URL: http://mpdc.mae.cornell.edu
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
OUTLINE
 Motivation: coupling multiscaling and uncertainty analysis
 Mathematical representation of uncertainty
 Variational multiscale method (VMS)
 Stochastic support method
 Stochastic convection-diffusion equations
 Computing PDFs of microstructures (Maximum Entropy)
 Sparse grid collocation methods (Smolyak quadrature)
 Natural convection on rough surfaces
 Diffusion in stochastic heterogeneous media
 Future research directions
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
NEED FOR UNCERTAINTY ANALYSIS
 Uncertainty is everywhere
From NIST
Porous
media
From Intel website
Silicon
wafer
From GE-AE website
Aircraft
engines
From DOE
Material
process
 Variation in properties, constitutive relations
 Imprecise knowledge of governing physics, surroundings
 Simulation based uncertainties (irreducible)
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
WHY UNCERTAINTY AND MULTISCALING ?
 Uncertainties introduced across various length scales have
a non-trivial interaction
 Current sophistications – resolve macro uncertainties
Micro
 Physical
properties, structure
follow a statistical
description
CORNELL
U N I V E R S I T Y
Meso
Macro
 Use micro
averaged
models for
resolving
physical scales
 Imprecise
boundary conditions
 Initial
perturbations
Materials Process Design and Control Laboratory
UNCERTAINTY ANALYSIS TECHNIQUES
 Monte-Carlo : Simple to implement, computationally
expensive
 Perturbation, Neumann expansions : Limited to small
fluctuations, tedious for higher order statistics
 Sensitivity analysis, method of moments : Probabilistic
information is indirect, small fluctuations
 Spectral stochastic uncertainty representation
 Basis in probability and functional analysis
 Can address second order stochastic processes
 Can handle large fluctuations, derivations are general
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
RANDOM VARIABLES = FUNCTIONS ?
 Math: Probability space (W, F, P)
Sample space
Probability measure
Sigma-algebra
 Random variable
F
W
W

W : Random variable
W : (W)
 A stochastic process is a random field with variations
across space and time
X : ( x, t , W)
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
SPECTRAL STOCHASTIC REPRESENTATION
 A stochastic process = spatially, temporally varying random
function
X : ( x, t , W)
CHOOSE APPROPRIATE
BASIS FOR THE
PROBABILITY SPACE
HYPERGEOMETRIC ASKEY POLYNOMIALS
GENERALIZED POLYNOMIAL
CHAOS EXPANSION
SUPPORT-SPACE
REPRESENTATION
PIECEWISE POLYNOMIALS (FE TYPE)
SPECTRAL DECOMPOSITION
KARHUNEN-LOÈVE
EXPANSION
COLLOCATION, MC (DELTA FUNCTIONS)
SMOLYAK QUADRATURE,
CUBATURE, LH
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
KARHUNEN-LOEVE EXPANSION

X ( x, t ,  )  X ( x, t )   X i ( x, t )i ( )
i 1
ON random variables
Deterministic functions
Stochastic Mean
process function
 Deterministic functions ~ eigen-values , eigenvectors of
the covariance function
 Orthonormal random variables ~ type of stochastic
process
 In practice, we truncate (KL) to first N terms
X ( x, t ,  )  fn( x, t , 1 ,
CORNELL
U N I V E R S I T Y
,N )
Materials Process Design and Control Laboratory
GENERALIZED POLYNOMIAL CHAOS
 Generalized polynomial chaos expansion is used to
represent the stochastic output in terms of the input
X ( x, t ,  )  fn( x, t , 1 ,

,N )
Stochastic input
Z ( x, t ,  )   Z i ( x, t ) i (ξ( ))
i 0
Stochastic
output
Askey polynomials in input
Deterministic functions
 Askey polynomials ~ type of input stochastic process
 Usually, Hermite, Legendre, Jacobi etc.
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
SUPPORT-SPACE REPRESENTATION
 Any function of the inputs, thus can be represented as a
function defined over the support-space
A  ξ  (1 , ,  N ) : f (ξ)  0
FINITE ELEMENT GRID REFINED
IN HIGH-DENSITY REGIONS
X  Xˆ
L2



2
ˆ
  ( X (ξ )  X (ξ )) f (ξ )dξ 
A

 Ch q 1
JOINT PDF OF A
TWO RANDOM
VARIABLE INPUT
CORNELL
U N I V E R S I T Y
– SMOLYAK
QUADRATURE
OUTPUT REPRESENTED ALONG
SPECIAL COLLOCATION POINTS
– IMPORTANCE
MONTE CARLO
Materials Process Design and Control Laboratory
VARIATIONAL MULTISCALE METHOD WITH ALGEBRAIC
SUBGRID MODELLING
 Application : deriving stabilized finite element formulations
for advection dominant problems
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
VARIATIONAL MULTISCALE HYPOTHESIS
EXACT SOLUTION
COARSE SOLUTION
INTRINSICALLY
COUPLED
SUBGRID SOLUTION
H
COARSE GRID
RESOLUTION CANNOT
CAPTURE FINE SCALE
VARIATIONS
THE FUNCTION SPACES
FOR THE EXACT SOLUTION
ALSO SHOW A SIMILAIR
DECOMPOSITION
 In the presence of uncertainty, the statistics of the solution
are also coupled for the coarse and fine scales
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
VARIATIONAL MULTISCALE BASICS
DERIVE THE WEAK
FORMULATION FOR
THE GOVERNING
EQUATIONS
PROJECT THE WEAK
FORMULATION ON
COARSE AND FINE
SCALES
COARSE WEAK
FORM
SOLUTION FUNCTION
SPACES ARE NOW
STOCHASTIC FUNCTION
SPACES
FINE (SUBGRID)
WEAK FORM
COMPUTATIONAL
SUBGRID MODELS
REMOVE SUBGRID
EFFECTS IN THE COARSE
WEAK FORM USING
STATIC CONDENSATION
MODIFIED MULTISCALE COARSE WEAK
FORM INCLUDING SUBGRID EFFECTS
CORNELL
U N I V E R S I T Y
ALGEBRAIC
SUBGRID MODELS
APPROXIMATE
SUBGRID
SOLUTION
NEED TECHNIQUES TO
SOLVE STOCHASTIC PDEs
Materials Process Design and Control Laboratory
VMS – ILLUSTRATION [NATURAL CONVECTION]
Mass conservation
Momentum conservation
Energy conservation
Constitutive laws
v  vg
 v0
v
 v v   Ra( ) Pr( ) eg   
t

 v    2
t
   pI  2 Pr( ) (v)
1
 (v)  [v  (v)T ]
2

D
VMS
DERIVE
DERIVE
DEFINE
PROBLEM WEAK FORM HYPOTHESIS SUBGRID
CORNELL
U N I V E R S I T Y
  g
OBTAIN FINAL COARSE
ASGS FORMULATION
Materials Process Design and Control Laboratory
WEAK FORM OF EQUATIONS
 Energy function space
 Test
 Trial
E   :   L2 (W; L2 (T ; H 1 ( D))),   g on 
E0  w : w  L2 (W; H 1 ( D)), w  0 on 
 Energy equation – Find
the following holds
  E such that, for all w  E0 ,
(t , w)  (v. , w)  ( , w) v  0
 VMS hypothesis: Exact solution = coarse scale solution +
fine scale (subgrid) solution
    '
E  E  E ', E0  E0  E0 '
VMS
DERIVE
DERIVE
DEFINE
PROBLEM WEAK FORM HYPOTHESIS SUBGRID
CORNELL
U N I V E R S I T Y
OBTAIN FINAL COARSE
ASGS FORMULATION
Materials Process Design and Control Laboratory
ENERGY EQUATION – SCALE DECOMPOSITION
 Energy equation – Find   E and w  E0 such that, for all
   E and w  E0 , the following holds
 Coarse scale variational formulation
( t   t ', w)  (v.  v. ', w)  (   ', w) v  0
 Subgrid scale variational formulation
( t   t ', w ')  (v.  v. ', w ')  (   ', w ') v  0
 These equations can be re-written in the strong form with
assumption on regularity as follows
 t ' v. '  2 '  ( t  v.   2 )  R
VMS
DERIVE
DERIVE
DEFINE
PROBLEM WEAK FORM HYPOTHESIS SUBGRID
CORNELL
U N I V E R S I T Y
OBTAIN FINAL COARSE
ASGS FORMULATION
Materials Process Design and Control Laboratory
ELEMENT FOURIER TRANSFORM
 Element Fourier transform
D
(e)
gˆ (k ,  ) 

exp(i
D( e )
SPATIAL MESH
RANDOM FIELD
DEFINED IN
WAVENUMBER SPACE
k x
) g ( x,  )dx
h
RANDOM FIELD
DEFINED OVER
THE DOMAIN
 Addressing spatial derivatives
kj
kj
g
k x
  n j exp(i
) g ( x,  )d  i gˆ (k ,  )  i gˆ (k ,  )
x D( e )
h
h
h
NEGLIGIBLE FOR LARGE
WAVENUMBERS  SUBGRID
VMS
DERIVE
DERIVE
DEFINE
PROBLEM WEAK FORM HYPOTHESIS SUBGRID
CORNELL
U N I V E R S I T Y
APPROXIMATION
OF DERIVATIVE
OBTAIN FINAL COARSE
ASGS FORMULATION
Materials Process Design and Control Laboratory
ASGS [ALGEBRAIC SUBGRID SCALE] MODEL
 t ' v. '  2 '  ( t  v.   2 )  R
STRONG FORM OF EQUATIONS FOR SUBGRID
1
 t f n  ( f n 1  f n ),
t
f n    f n 1  (1   ) f n
CHOOSE AND APPROPRIATE TIME INTEGRATION ALGORITHM
t n'  L( n'  )  Rn
TIME DISCRETIZED SUBGRID EQUATION
TAKE ELEMENT
FOURIER
TRANSFORM
2
 1
 '
k
vk
1 ˆ'

i
 2 ˆn  Rˆn 
n
  t
h
h 
 t

2
 1

1 '
1   v 
  t  Rn  
 n  ,  t    c1 2 
  c2  

  h  t   h  
 t 



2
 n' 
CORNELL
U N I V E R S I T Y

1
2
Materials Process Design and Control Laboratory
MODIFIED COARSE FORMULATION
 Assume the solution obeys the following regularity conditions
(v  ', w)  ( ', v w), ( ', w) v  ( ',  2 w) v
 By substituting ASGS model in the coarse scale weak form
( t n , w)  (v  n  , w)  ( n  , w) v  ( q0 , w) ht
Nel
  ( t n  v  n    2 n  , t ( w  v w   2 w) 
e 1
Nel
  ( n' , t w /( t )   t v w)   2 w  w)   0
e 1
w  w /( t )
 A similar derivation ensues for stochastic Navier-Stokes
VMS
DERIVE
DERIVE
DEFINE
PROBLEM WEAK FORM HYPOTHESIS SUBGRID
CORNELL
U N I V E R S I T Y
OBTAIN FINAL COARSE
ASGS FORMULATION
Materials Process Design and Control Laboratory
FLOW PAST A CIRCULAR CYLINDER
NO-SLIP
TRACTION FREE
6
Y
RANDOM UINLET
8
4
2
0
0
5
10
15
20
X
NO-SLIP
INLET VELOCITY ASSUMED TO BE A UNIFORM RANDOM VARIABLE
KARHUNEN-LOEVE EXPANSION YIELD A SINGLE RANDOM VARAIBLE
THUS, GENERALIZED POLYNOMIAL CHAOS  LEGENDRE
POLYNOMIALS (ORDER 3 USED)
 Investigations: Vortex shedding, wake characteristics
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
FULLY DEVELOPED VORTEX SHEDDING
 Mean pressure
 Second LCE coefficient
 First LCE coefficient
 Wake region in the mean
pressure is diffusive in
nature
 Also, the vortices do not
occur at regular intervals
[Karniadakis J. Fluids.
Engrg]
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
VELOCITIES AND FFT
0.6
0.15
0.4
0.2
0.09
V
Amplitude
0.12
0
0.06
-0.2
0.03
0
0.1
-0.4
-0.6
0.2
0.3
Frequency
0.4
0.5
FFT YIELDS A MEAN SHEDDING
FREQUENCY OF 0.162
FFT SHOWS A DIFFUSE
BEHAVIOR IMPLYING CHANGING
SHEDDING FREQUENCIES
CORNELL
U N I V E R S I T Y
Deterministic
Mean
5
8
11
X
14
17
20
MEAN VELOCITY AT NEAR WAKE
REGION EXHIBITS
SUPERIMPOSED FREQUENCIES
Materials Process Design and Control Laboratory
VARIATIONAL MULTISCALE METHOD WITH EXPLICIT
SUBGRID MODELLING FOR MULTISCALE DIFFUSION IN
HETEROGENEOUS RANDOM MEDIA
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
MODEL MULTISCALE HEAT EQUATION

D
u
  ( K u )  f in D
t
u ( x, 0)  u0 ( x) in D
u  u g on 
THE DIFFUSION COEFFICIENT K IS HETEROGENEOUS AND POSSESSES RAPID
RANDOM VARIATIONS IN SPACE
OTHER APPLICATIONS
– DIFFUSION IN
COMPOSITES
– FUNCTIONALLY
GRADED MATERIALS
FLOW IN HETEROGENEOUS
POROUS MEDIA 
INHERENTLY STATISTICAL
DIFFUSION IN
MICROSTRUCTURES
FINAL COARSE
VMS
AFFINE
DERIVE
COARSE-TODEFINE
PROBLEM WEAK FORM HYPOTHESIS SUBGRID MAP CORRECTION FORMULATION
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
STOCHASTIC WEAK FORM
U  u : u  L2 (W; H 1 ( D)), u   u g 
V  v : v  L2 (W; H 1 ( D)), v   0
 Weak formulation : Find
u U such that, for all v V
(u,t , v)  a(u, v)  ( f , v); a(u, v) : ( K u, v)
 VMS hypothesis
Exact solution
u  u u
C
F
Coarse solution
U  U C U F
Subgrid solution
V  V C V F
FINAL COARSE
VMS
AFFINE
DERIVE
COARSE-TODEFINE
PROBLEM WEAK FORM HYPOTHESIS SUBGRID MAP CORRECTION FORMULATION
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
EXPLICIT SUBGRID MODELLING: IDEA
DERIVE THE WEAK
FORMULATION FOR
THE GOVERNING
EQUATIONS
COARSE WEAK
FORM
PROJECT THE WEAK
FORMULATION ON
COARSE AND FINE
SCALES
FINE (SUBGRID)
WEAK FORM
COARSE-TO-SUBGRID
MAP  EFFECT OF
COARSE SOLUTION ON
SUBGRID SOLUTION
AFFINE CORRECTION 
SUBGRID DYNAMICS THAT
ARE INDEPENDENT OF
THE COARSE SCALE
LOCALIZATION, SOLUTION OF SUBGRID
EQUATIONS NUMERICALLY
FINAL COARSE WEAK FORMULATION THAT
ACCOUNTS FOR THE SUBGRID SCALE EFFECTS
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
SCALE PROJECTION OF WEAK FORM
Find u  U and u  U such that, for all
F
F
and v  V
 Projection of weak form on coarse scale
C
C
F
F
v V
C
C
(u,Ct , vC )  (u,Ft , vC )  a(uC , vC )  a(u F , vC )  ( f , vC )
 Projection of weak form on subgrid scale
(u,Ct , v F )  (u,Ft , v F )  a(uC , v F )  a(u F , v F )  ( f , v F )
u F  uˆ F  u F 0
EXACT SUBGRID
SOLUTION
COARSE-TO-SUBGRID
MAP
SUBGRID AFFINE
CORRECTION
FINAL COARSE
VMS
AFFINE
DERIVE
COARSE-TODEFINE
PROBLEM WEAK FORM HYPOTHESIS SUBGRID MAP CORRECTION FORMULATION
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
SPLITTING THE SUBGRID SCALE WEAK FORM
 Subgrid scale weak form
(u , v )  (u , v )  a(u , v )  a(u , v )  ( f , v )
C
,t
F
F
,t
F
C
F
F
F
F
 Coarse-to-subgrid map
F
F
C
F
F
F
ˆ
ˆ
(u , v )  (u,t , v )  a(u , v )  a(u , v )  0
C
,t
F
 Subgrid affine correction
(u , v )  a(u , v )  ( f , v )
F0
,t
CORNELL
U N I V E R S I T Y
F
F0
F
F
Materials Process Design and Control Laboratory
NATURE OF MULTISCALE DYNAMICS
ASSUMPTIONS:
1
1
NUMERICAL ALGORITHM
FOR SOLUTION OF THE
MULTISCALE PDE
t
SUBGRID TIME STEP t
COARSE TIME STEP
A(t )
B (t )
ũC
ūC
Coarse
solution field
at start of
time step
ûF
t
CORNELL
U N I V E R S I T Y
Coarse
solution field
at end of
time step
t
Materials Process Design and Control Laboratory
REPRESENTING COARSE SOLUTION
COARSE MESH
ELEMENT
D( e )
RANDOM FIELD DEFINED OVER THE
ELEMENT
u C ( x, t ,  )
FINITE ELEMENT PIECEWISE
POLYNOMIAL REPRESENTATION
C
u
  1  (t, )  ( x)
USE GPCE TO REPRESENT THE
RANDOM COEFFICIENTS
C
u
  1  s0  s (t) s ( )  ( x)
nbf
nbf
PC
 Given the coefficients u s (t ) , the coarse scale
solution is completely defined
C
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
COARSE-TO-SUBGRID MAP
COARSE MESH
ELEMENT
D( e )
ANY INFORMATION FROM COARSE
TO SUBGRID SOLUTION CAN BE
PASSED ONLY THROUGH
uCs (t )
uˆ ( x, t ,  )    1  s 0 uCs (t )Fs ( x, t ,  )
F
COARSE-TOSUBGRID MAP
nbf
PC
INFORMATION BASIS FUNCTIONS
THAT ACCOUNT
FROM COARSE
FOR FINE SCALE
SCALE
EFFECTS
FINAL COARSE
VMS
AFFINE
DERIVE
COARSE-TODEFINE
PROBLEM WEAK FORM HYPOTHESIS SUBGRID MAP CORRECTION FORMULATION
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
SOLVING FOR THE COARSE-TO-SUBGRID MAP
START WITH THE WEAK FORM
(u,Ct , v F )  (uˆ,Ft , v F )  a(uC , v F )  a(uˆ F , v F )  0
APPLY THE MODELS FOR COARSE SOLUTION AND THE C2S MAP
u
C
s
 

( s    Fs ) ,t , v F  K  uCs ( s    Fs ) , v F  0
AFTER SOME ASSUMPTIONS ON TIME STEPPING
 ( 
s

 Fs ),t , v F    K ( s    Fs ), v F   0
THIS IS DEFINED OVER EACH ELEMENT,
IN EACH COARSE TIME STEP
0
CORNELL
U N I V E R S I T Y
t
t
Materials Process Design and Control Laboratory
BCs FOR THE COARSE-TO-SUBGRID MAP
 ( 
s

 Fs ),t , v F    K ( s    Fs ), v F   0
INTRODUCE A SUBSTITUTION
  s   s    Fs

F
F
,
,
v

K

,

v


0
s t
s
CONSIDER AN ELEMENT
x3
x4

F

F
,
v

K

,

v


0
 s ,t
 s ,

  s ,t ( x , t ',  )     s ( )
x1
  s   s
CORNELL
U N I V E R S I T Y
x2
  s ,t ( x, 0,  )    s ,t ( x, tn ,  )
Materials Process Design and Control Laboratory
SOLVING FOR SUBGRID AFFINE CORRECTION
START WITH THE WEAK FORM
(u,Ft 0 , v F )  a(u F 0 , v F )  ( f , v F )
CONSIDER AN ELEMENT
x4
x3
WHAT DOES AFFINE CORRECTION MODEL?
– EFFECTS OF SOURCES ON SUBGRID SCALE
– EFFECTS OF INITIAL CONDITIONS
x1
uF0  0
x2
IN A DIFFUSIVE EQUATION, THE EFFECT OF
INITIAL CONDITIONS DECAY WITH TIME. WE
CHOOSE A CUT-OFF
 To reduce cut-off effects and to increase efficiency, we can
use the quasistatic subgrid assumption
  s ,t  u,Ft 0  0
FINAL COARSE
VMS
AFFINE
DERIVE
COARSE-TODEFINE
PROBLEM WEAK FORM HYPOTHESIS SUBGRID MAP CORRECTION FORMULATION
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
MODIFIED COARSE SCALE FORMULATION
 We can substitute the subgrid results in the coarse scale
variational formulation to obtain the following
C
C
C
C
C
C
u
,

,
v

u

,
,
v

u
K

,

v
 s t s   s s t   s

s
 ( f , v C )   K u F 0 , v C    u F 0 , t , v C 
 We notice that the affine correction term appears as an antidiffusive correction
 Often, the last term involves computations at fine scale time
steps and hence is ignored
FINAL COARSE
VMS
AFFINE
DERIVE
COARSE-TODEFINE
PROBLEM WEAK FORM HYPOTHESIS SUBGRID MAP CORRECTION FORMULATION
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
DIFFUSION IN A RANDOM MICROSTRUCTURE
– A MIXTURE MODEL IS USED AS AN
EXAMPLE OF GENERATING A
HETEROGENEOUS DISTRIBUTION OF
CONDUCTIVITY
– WE ASSUME THAT THE DIFFUSION
COEFFICIENTS OF INDIVIDUAL
CONSTITUENTS ARE NOT KNOWN
EXACTLY
k* ( )  k*0  k*1 ( )
THE INTENSITY OF THE GRAY-SCALE
IMAGE IS MAPPED TO THE
CONCENTRATIONS
DARKEST DENOTES  PHASE
LIGHTEST DENOTES  PHASE
k ( x,  )  (k  ( )  k ( )) I ( x)  k ( )
u (x, 0,  )  0
CORNELL
U N I V E R S I T Y
u |( x 1)  0, u |( x 0)  1, k
u
|( y 0,1)  0
n
Materials Process Design and Control Laboratory
RESULTS AT TIME = 0.05
FIRST ORDER GPCE
COEFF
SECOND ORDER GPCE
COEFF
FULLY RESOLVED
GPCE SIMULATION
RECONSTRUCTED FINE
SCALE SOLUTION (VMS)
MEAN
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
RESULTS AT TIME = 0.2
FIRST ORDER GPCE
COEFF
SECOND ORDER GPCE
COEFF
FULLY RESOLVED
GPCE SIMULATION
RECONSTRUCTED FINE
SCALE SOLUTION (VMS)
MEAN
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
HIGHER ORDER TERMS AT TIME = 0.2
FOURTH ORDER GPCE
COEFF
FIFTH ORDER GPCE
COEFF
FULLY RESOLVED
GPCE SIMULATION
RECONSTRUCTED FINE
SCALE SOLUTION (VMS)
THIRD ORDER GPCE
COEFF
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
SUPPORT-SPACE – STOCHASTIC GALERKIN
f ( ( ))
- Joint probability density function of the inputs
A  { ( ) : f ( ( ))  0}
- The input support-space denotes the regions where input
joint PDF is strictly positive
Triangulation
of the supportspace
Any function X ( ( )) can be represented as a
piecewise polynomial on the triangulated support-space
h
X ( ( )) - Function to be approximated
X h ( ( )) - Piecewise polynomial approximation
over support-space
L2 convergence – (mean-square)
 (X
Error in approximation is penalized
severely in high input joint PDF regions.
We use importance based refinement of
grid to avoid this
CORNELL
U N I V E R S I T Y
h
( ( ))  X ( ( )))2 f ( ( ))d  Chq 1
A
h = mesh diameter for the support-space
discretization
q = Order of interpolation
Materials Process Design and Control Laboratory
IMPLEMENTATION OF SUPPORT-SPACE
A stochastic process W ( x, t ,  ) can be interpreted as a random variable at each spatial
point
Two-level grid
approach
A
D
Support-space grid
Spatial domain
• Mesh dense in
regions of high
input joint PDF
Spatial grid
• There is finite element interpolation at
both spatial and random levels
D( e )
( x,  )
A( e ')
Element
• Each spatial location handles an
underlying support-space grid
• Highly OOP structure
nbf
nbf ' nbf
i 1
j 1 i 1
f ( x,  )   fi ( ) Ni ( x)   fi j N 'j ( ) Ni ( x)
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
CAPTURING UNSTABLE EQUILIBRIUM
• Computational details – 1600 bilinear elements for spatial grid
• Time of simulation – 1.5 nondimensional units
Cold wall  c  0
1
0.75
Insulated
Y
Insulated
0.5
• Rayleigh number – uniformly
distributed random variable
between 1530 and 1870 (10%
fluctuation about 1700)
• Prandtl number – 6.95
• Time stepping – 0.002 nondimensional units
0.25
0
0
0.25
0.5
0.75
1
X
Hot wall  h  1
• Support-space grid – Onedimensional with ten linear
elements
• Simulation about the critical Rayleigh number – conduction below, convection above
• Both GPCE and support-space methods are used separately for addressing the problem
• Failure of Generalized polynomial chaos approach
• Support-space method – evaluation and results against a deterministic simulation
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
FAILURE OF THE GPCE
9.2E-07 5.7E-06 1.0E-05 1.5E-05 2.0E-05 2.5E-05 3.0E-05 3.4E-05
1
-6.4E-08 4.3E-07
1
1.4E-06
1.9E-06
2.4E-06
2.9E-06
3.4E-06
Y-vel
X-vel
0.75
0.75
0.5
Y
Y
Mean X- and Yvelocities
determined by
GPCE yields
extremely low
values !! (Gibbs
effect)
0.25
0
0.5
0.25
0
0.25
0.5
0.75
0
1
X
-5.0E-03 -3.6E-03 -2.1E-03 -7.1E-04 7.1E-04
1
2.1E-03
3.6E-03
0.5
0.75
1
1.6E-03
2.8E-03
4.0E-03
5.2E-03
Y-vel
0.75
Y
Y
0.5
0.5
0.25
0.25
0
0
0.25
0.5
X
CORNELL
0.25
-3.2E-03 -2.0E-03 -8.0E-04 3.9E-04
1
5.0E-03
0.75
X- and Yvelocities
obtained from a
deterministic
simulation with
Ra = 1870 (the
upper limit)
0
X
X-vel
U N I V E R S I T Y
9.3E-07
0.75
1
0
0
0.25
0.5
0.75
1
X
Materials Process Design and Control Laboratory
PREDICTION BY SUPPORT-SPACE METHOD
-5.0E-03 -3.6E-03 -2.1E-03 -7.1E-04 7.1E-04
1
2.1E-03
3.6E-03
5.0E-03
-3.2E-03 -2.0E-03 -7.4E-04 4.9E-04
1
2.9E-03
4.2E-03
5.4E-03
Y-vel
X-vel
0.75
Y
Mean X- and Yvelocities
determined by
support-space
method at a
realization
Ra=1870
Y
0.75
0.5
0.25
0.25
0
0.5
0
0
0.25
0.5
0.75
1
0
0.25
-5.0E-03 -3.6E-03 -2.1E-03 -7.1E-04 7.1E-04
1
2.1E-03
3.6E-03
1
1.6E-03
2.8E-03
4.0E-03
5.2E-03
0.75
Y
Y
0.5
0.5
0.25
0.25
0
0
0.25
0.5
X
CORNELL
0.75
Y-vel
0.75
X- and Yvelocities
obtained from a
deterministic
simulation with
Ra = 1870 (the
upper limit)
-3.2E-03 -2.0E-03 -8.0E-04 3.9E-04
1
5.0E-03
X-vel
0.5
X
X
U N I V E R S I T Y
1.7E-03
0.75
1
0
0
0.25
0.5
0.75
1
X
Materials Process Design and Control Laboratory
SPARSE GRID COLLOCATION
 If the number of random inputs is large (dimension D ~ 10
or higher), the number of grid points to represent an output
on the support-space mesh increases exponentially
 GPCE for very high dimensions yields highly coupled
equations and ill-conditioned systems (relative magnitude of
coefficients can be drastically different)
 Instead of relying on piecewise interpolation, series
representations, can we choose collocation points that still
ensure accurate interpolations of the output (solution)
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
SMOLYAK ALGORITHM: SPARSE GRIDS
Full tensor product grid:
289 points
Example of using sparse grids to build interpolating functions:
Discontinuous functions
Sparse Grid: 65 points
Left to right: Improving interpolation depth
Number of points required to construct
For an error around 2x10-2:
interpolating functions reduces combinatorially.
Required number of points using sparse grids 3300
Reduction more significant as the number of
Required number of points using full tensor products: 32769
dimensions increases
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
SMOLYAK ALGORITHM
LET OUR BASIC 1D INTERPOLATION SCHEME BE SUMMARIZED AS
Ui( f ) 
a
xi  X i
x
i
f ( xi )
IN MULTIPLE DIMENSIONS, THIS CAN BE WRITTEN AS
(U i1 
 U id )( f ) 

xi1 X i1

xid X id
(axi1 
 axid ) f (x i1 ,
, x id )
TO REDUCE THE NUMBER OF SUPPORT NODES WHILE MAINTAINING
ACCURACY WITHIN A LOGARITHMIC FACTOR, WE USE SMOLYAK METHOD
U 0  0, i  U i  U i 1 ,
i  i1 
Aq ,d ( f )  Aq 1,d ( f )   (i1 
 id
id )( f )
i q
IDEA IS TO CONSTRUCT AN EXPANDING SUBSPACE OF COLLOCATION
POINTS THAT CAN REPRESENT PROGRESSIVELY HIGHER ORDER
POLYNOMIALS IN MULTIPLE DIMENSIONS
A FEW FAMOUS SPARSE QUADRATURE SCHEMES ARE AS FOLLOWS:
CLENSHAW CURTIS SCHEME, MAXIMUM-NORM BASED SPARSE GRID AND
CHEBYSHEV-GAUSS SCHEME
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
SMOLYAK ALGORITHM
Extensively used in statistical mechanics
Uni-variate interpolation
Provides a way to construct interpolation
functions based on minimal number of
points
Ui( f ) 
a
xi  X i
x
i
f ( xi )
Multi-variate interpolation
(U i1 
Univariate interpolations to multivariate
(U i1   U id )( f )  
interpolations
xi1 X i1
 U id )( f ) 

xid X id
( axi1 


(axi1 
 axid ) f (x ,
, x id )
xi1 X i1
xid iX id
1
Smolyak interpolation
U 0  0, i  U i  U i 1 ,
Some degradation in accuracy
i  i1 
Aq ,d ( f )  Aq 1,d ( f )   (i1 
 id
id )( f )
i q
Maximal reduction when the
function is assumed to be smooth
CORNELL
U N I V E R S I T Y
D = 10
ORDER
CC
FE
3
1581
1000
4
8801
10000
5
41625
100000
Materials Process Design and Control Laboratory

SPARSE GRID COLLOCATION METHOD
Solution Methodology
PREPROCESSING
Compute list of collocation points based on number of
stochastic dimensions, N and level of interpolation, q
Compute the weighted integrals of all the interpolations
functions across the stochastic space (wi)
Solve the deterministic problem defined by each set of
collocated points
Use any validated deterministic
solution procedure.
Completely non intrusive
POSTPROCESSING
0.301
0.260
0.301
0.260
0.220
0.220
0.180
0.180
0.140
0.140
0.100
0.100
0.060
0.060
0.020
0.020
Compute moments and other statistics with simple
operations of the deterministic data at the collocated
points and the preprocessed list of weights
Std deviation of temperature:
Natural convection
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
USING THE COLLOCATION METHOD FOR HIGHER DIMENSIONS
1. Flow through heterogeneous random media
Alloy solidification, thermal insulation, petroleum prospecting
Look at natural convection through a realistic sample of heterogeneous material
Square cavity with free fluid in the middle
part of the domain. The porosity of the
material is taken from experimental data1
Left wall kept heated, right wall cooled
Numerical solution procedure for the
deterministic procedure is a fractional time
stepping method
1. Reconstruction of random media using Monte Carlo methods, Manwat and Hilfer, Physical Review E. 59 (1999)
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
FLOW THROUGH HETEROGENEOUS RANDOM MEDIA
Experimental correlation for the porosity of the
sandstone.
Material: Sandstone
Eigen spectrum is peaked. Requires large
dimensions to accurately represent the stochastic
space
Simulated with N= 8
Number of collocation points is 11561 (level 4
interpolation)
15
1
0.9
0.8
0.7
0.6
0.5
Eigen
spectrum
0.4
0.3
Eigenvalue
10
Numerically
computed
5
0.2
0.1
0
0
10
20
30
40
50
60
0
5
10
15
20
Index
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
FLOW THROUGH HETEROGENEOUS RANDOM MEDIA
Temperature
Snapshots at a few collocation points
Temperature
9.1
6.4
3.7
1.0
-1.7
-4.4
-7.1
-9.8
0.9
0.8
0.7
0.6
0.4
0.3
0.2
0.1
14.2
10.1
6.1
2.0
-2.0
-6.1
-10.1
-14.1
CORNELL
U N I V E R S I T Y
12.1
8.6
5.1
1.7
-1.8
-5.3
-8.7
-12.2
7.0
4.4
1.8
-0.8
-3.4
-6.0
-8.6
-11.2
Streamlines
Temperature
0.9
0.8
0.7
0.6
0.4
0.3
0.2
0.1
Y velocity
0.94
0.81
0.69
0.56
0.44
0.31
0.19
0.06
y-Velocity
0.9
0.8
0.7
0.6
0.4
0.3
0.2
0.1
FIRST MOMENT
SECOND MOMENT
0.097
0.084
0.071
0.058
0.045
0.032
0.019
0.006
Y velocity
5.056
4.382
3.708
3.034
2.359
1.685
1.011
0.337
Materials Process Design and Control Laboratory
USING THE COLLOCATION METHOD FOR HIGHER DIMENSIONS
2. Flow over rough surfaces
Thermal transport across rough surfaces, heat exchangers
Look at natural convection through a realistic roughness profile
Rectangular cavity filled with fluid.
Lower surface is rough. Roughness auto
correlation function from experimental data2
T (y) = -0.5
Lower surface maintained at a higher
temperature
Rayleigh-Benard instability causes
convection
T (y) = 0.5
y = f(x,ω)
Numerical solution procedure for the
deterministic procedure is a fractional time
stepping method
2. H. Li, K. E. Torrance, An experimental study of the correlation between surface roughness and light scattering for rough
metallic surfaces, Advanced Characterization Techniques for Optics, Semiconductors, and Nanotechnologies II,
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
NATURAL CONVECTION ON ROUGH SURFACES
Experimental ACF
Experimental correlation for the surface roughness
1
Eigen spectrum is peaked. Requires large
dimensions to accurately represent the stochastic
space
0.8
0.6
V2
Simulated with N= 20 (Represents 94% of the
spectrum)
0.4
0.2
Number of collocation points is 11561 (level 4
interpolation)
0.44
0.44
0.31
0.31
0.19
0.19
0.06
0.06
-0.06
-0.06
-0.19
-0.19
-0.31
-0.31
-0.44
-0.44
0.44
0.31
0.19
0.06
-0.06
-0.19
-0.31
-0.44
CORNELL
U N I V E R S I T Y
0
0.5
1
1.5
2
V1
Numerically computed
Eigen spectrum
Sample realizations
of temperature at
collocation points
16
12
Eigenvalue
0.44
0.31
0.19
0.06
-0.06
-0.19
-0.31
-0.44
0
8
4
0
5
10
15
20
Index
Materials Process Design and Control Laboratory
NATURAL CONVECTION ON ROUGH SURFACES
FIRST MOMENT
SECOND MOMENT
Temperature
Temperature
0.17
0.14
0.12
0.10
0.08
0.06
0.03
0.01
0.44
0.31
0.19
0.06
-0.06
-0.19
-0.31
-0.44
Streamlines
Y Velocity
7.63
6.62
5.60
4.58
3.56
2.54
1.53
0.51
Roughness causes improved thermal transport due to enhanced nonlinearities
Results in thermal plumes
Can look to tailor material surfaces to achieve specific thermal transport
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Statistical characterization of microstructures
Can we compute statistical response
of a class of microstructures
subjected to applied loads based on
limited experimental information?
Features of a microstructure
Grain size (in 3D,
grain volume)
When a specimen is
manufactured, the
microstructures at a
sample point will not
be the same always.
Orientation
Distribution
Function
Rodrigues’
representation
FCC fundamental
region
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Technique employed
Maximum entropy (MAXENT): The probability distribution that maximizes entropy and
satisfies the given (experimental/simulation-based) information is the least-biased
estimate that can be made.
Problem formulation
Numerical implementation
Limited microstructures computed
using phase field simulations
We employ the voronoi cell
tessellation technique for
representing microstructures.
Extract features of the microstructure
Geometrical: grain size
Texture: ODFs
Conjugate
Compute a PDF of microstructures
Entropy
gradient
Compute bounds on macroscopic properties
microstructure
feature constraints
Meshing a statistical class of microstructures
using CUBIT
CORNELL
U N I V E R S I T Y
features of microstructure, I
Given information about
microstructures. We use grain
size and texture features
Materials Process Design and Control Laboratory
Statistical class of 3D Aluminium polycrystals
Three statistical
Aluminium polycrystal
samples generated using
phase field simulations
Comparison of grain size distributions
between a phase field simulation from the
representative class and a MaxEnt sample
First four
statistical
moments of
grain sizes
(volumes)
Probability mass function
0.25
Grain volume distribution
using phase field simulations
pmf reconstructed using MaxEnt
0.2
0.15
0.1
0.05
0
0
CORNELL
U N I V E R S I T Y
2000 4000 6000 8000 100001200014000160001800020000
Grain volume (voxels)
Materials Process Design and Control Laboratory
ODF reconstruction using MAXENT
Input ODF
0.35
Grain size distribution of a
microstructural sample.
Comparison with the MaxEnt
distribution
Probability mass function
0.3
0.25
Reconstructed
samples using
MAXENT
0.2
0.15
Represent
ation in
FrankRodrigues
space
Rcorr=0.9644
KL=0.0383
0.1
0.05
0
0
5000
10000
15000
Grain volume (voxels)
20000
25000
A microstructural specimen computed from the
MaxEnt distribution
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Statistical variation of properties
Homogenization scheme:
First order stress averaging
Scheme employing Hill’s criterion
60
How to mesh a
microstructure?
We employ hexahedral
elements using Cubit
software
Mean std
Equivalent stress (MPa)
50
Mean stress-strain
curve
40
30
Aluminium polycrystal
with rate-independent
strain hardening. Pure
tensile test.
20
10
0
0
1
2
Equivalent strain
CORNELL
U N I V E R S I T Y
3
Statistical variation of
homogenized stressstrain curves.
-4
x 10
Materials Process Design and Control Laboratory
HETEROGENEOUS DIFFUSION
a) Two phase materials
b) Micro-emulsions,
c) porous media,
d) ceramics
e) Polycrystals
f) Foams, blends
- To apply physical processes on these
heterogeneous systems
- worst case scenarios
- variations on physical properties
Different morphology, anisotropy
Aim: To develop a procedure to
predict statistics of properties of
heterogeneous materials
undergoing certain phenomena
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
OVERVIEW OF METHODOLOGY
Given certain properties P1, P2, ..
Pn, that the structure satisfies.
STEP 1
These properties are usually
statistical: Volume fraction, 2 Point
correlation, auto correlation
Reconstruct realizations of the
structure satisfying the properties.
Monte Carlo, Gaussian Random
Fields, Stochastic optimization ect
STEP 2
Solve the heterogeneous property
problem in the reduced stochastic
space for computing property
variations.
Collocation schemes
CORNELL
U N I V E R S I T Y
STEP 3
Construct a reduced stochastic
model from the data. This model
must be able to approximate the
class of structures.
KL expansions, FFT and other
transforms, Autoregressive models,
ARMA models
Materials Process Design and Control Laboratory
EXAMPLE: THERMAL DIFFUSION THROUGH TUNGSTEN—SILVER MATRIX
MC-Potts model, generate
microstructures database.
Apply the KL transform
Z
First 9 eigen values are
enough
X
Tungsten-silver composite image1
15
Z
10
5
0
0
Y
1.5807E-02
1.4753E-02
1.3699E-02
1.2645E-02
1.1592E-02
1.0538E-02
9.4840E-03
8.4303E-03
7.3765E-03
6.3227E-03
5.2689E-03
4.2151E-03
3.1613E-03
2.1076E-03
1.0538E-03
0
5
5
Y 10
10
15
X
15
1. S. Umekawa, R. Kotfila and O.D. Sherby, Elastic properties of a
tungsten-silver composite above and below the melting point of silver, J.
Mech. Phys. Solids 13 (1965)
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
REDUCED MODEL FOR THE STRUCTURE
Represent any microstructure as a linear combination of the eigen-images
I = Iavg + I1a1 + I2a2+ I3a3 + … + Inan
=a1
+ ..+ an
+a2
Image I belongs to the class of structures?
It must satisfy certain conditions
a) Its volume fraction must equal the specified volume fraction
b) Volume fraction at every pixel must be between 0 and 1
c) It should satisfy higher order statistics
Thus the n tuple (a1,a2,..,an) must further satisfy some constraints.
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
REDUCED MODEL FOR THE STRUCTURE
Constraints on the coefficients
15
a v
10
i i
v
i
5
 a I ( j )  1, j  1: NPixels
 a I ( j )  0, j  1: NPixels
i i
0
i
-5
i i
i
-10
10
15
15
10
20
-15
-10
-5
0
5
10
15
5
Construct the Convex Hull of the set of linear inequalities.
This is the allowable set of coefficients.
This represents the space of allowable microstructures
0
-5
In this space all the structures are equiprobable.
This represents a stochastic space in (n-1) dimensions.
Actually a plane in n dimensions, Call this the ‘material
plane’
CORNELL
U N I V E R S I T Y
-10
-15
11
11.5
12
12.5
13
13.5
14
14.5
15
15.5
16
Materials Process Design and Control Laboratory
PHYSICAL PROBLEM UNDER CONSIDERATION
Structure size 20x20x20 μm
Tungsten Silver Matrix
T= -0.5
T= 0.5
Heterogeneous property is the
thermal diffusivity.
Tungsten: ρ 19250 kg/m3
k 174 W/mK
c 130 J/kgK
Left wall maintained at -0.5
Silver:
ρ 10490 kg/m3
Right wall maintained at +0.5
k 430 W/mK
All other surfaces insulated
c 235 J/kgK
Diffusivity ratio αAg/αW = 2.5
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
COLLOCATION SCHEME: SAMPLE REALIZATIONS
First column: conductivity
Second column: Temperature
15
10
5
0
-5
-10
-15
11
CORNELL
U N I V E R S I T Y
11.5
12
12.5
13
13.5
14
14.5
15
15.5
16
Materials Process Design and Control Laboratory
MEAN STATISTICS
Temperature isosurfaces
Mean temperature: No variations closer
to the surfaces, significant variations
inside
Mean distribution of silver
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
SECOND ORDER STATISTICS
Temperature slice
Property slice
Left, isosurface of
temperature deviation
0
0
5
5
y
10
10
15
15
CORNELL
U N I V E R S I T Y
x
0
0
5
5
y
10
10
15
Right, isosurface of
properties
x
15
Materials Process Design and Control Laboratory
HIGHER ORDER STATISTICS
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory
Future research directions
 Algorithms to address the curse-of-dimensionality
 Adaptivity in the support space, adaptive sparse-grid
quadrature rules, SPDE model reduction, etc.
 Stochastic multiscale advection-diffusion-reaction
 Stochastic multiscale modeling in materials
 Information-theoretic algorithms for
coupling statistics across length scales
 Robust design techniques
 Interface stochastic and statistical
(Bayesian) computation
CORNELL
U N I V E R S I T Y
Materials Process Design and Control Laboratory