Properties of Covariance and
Variogram Functions
CWR 6536
Stochastic Subsurface Hydrology
The Covariance Function
• The covariance function must be positive definite which
requires that:
1. 𝑃𝑘𝑘 0 = 𝑉𝑎𝑟 𝐾(𝑥) ≥ 0
2. 𝑃𝑘𝑘 (ℎ) = 𝑃𝑘𝑘 (−ℎ)
3. 𝑃𝑘𝑘 (ℎ) ≤ 𝑃𝑘𝑘 0
Variance cannot be negative
Covariance function is even
Covariance decreases from
a maximum at h=0
• positive definiteness guarantees that all linear
combinations of the random variable will have nonnegative variances. This implies: 𝑖𝑓 𝑌 = 𝑖 𝜆𝑖𝐾 𝑥𝑖
𝑉𝑎𝑟 𝑌 =
𝜆𝑖𝜆𝑗𝑃𝑌𝑌(𝑥𝑖 − 𝑥𝑗) ≥ 0 for any λi
𝑖
𝑗
Mathematical condition
for positive definiteness
The Variogram Function
• The negative semivariogram function must be conditionally
positive definite which requires that:
1. 𝛾𝑘𝑘 0 = 0
2. 𝛾𝑘𝑘 (ℎ) = 𝛾𝑘𝑘 (−ℎ) ≥ 0
Variogram function is even
3. γkk(h) ≥ γkk 0
Degree of difference increases
from a minimum at h=0
• conditional positive definiteness guarantees that all linear
combinations of the random variable will have non-negative
variances. This implies:
𝑖𝑓 𝑌 =
𝜆𝑖𝐾 𝑥𝑖 𝑎𝑛𝑑
𝑖
𝜆𝑖 = 0 𝑡ℎ𝑒𝑛 𝑉𝑎𝑟 𝑌 = −
𝑖
Condition required in the case where
P(0) does not exist
𝜆𝑖𝜆𝑗𝛾𝑌𝑌(𝑥𝑖 − 𝑥𝑗) ≥ 0 for any λ𝑖
𝑖
𝑗
Mathematical condition for
conditional positive definiteness
• Positive-definiteness is related to the
number of dimensions in space over which
the function is defined.
• Positive definiteness in higher order
dimensional space guarantees positive
definiteness in lower order dimensional
space, but not vice-versa
• Must fit functions to sample covariances/
variograms which are positive definite in
the appropriate dimensional space
Behavior of Covariance/Variogram
functions near the origin
• Parabolic behavior
Variability increases very slowly with separation,
highly regular random field
Example: Gaussian model
−ℎ2 2
2
𝑎 )
𝛾 ℎ = 𝜎 (1 − 𝑒
2
−ℎ
2
P ℎ = 𝜎2𝑒 𝑎
• Linear behavior
More rapid increase in variability with separation,
Example:
Exponential
model
Example:
Spherical
model
ℎ
𝛾3ℎℎ 1=ℎ2𝜎2(1 − 𝑒 − 𝑎 )
𝛾 ℎ = 𝜎2 2𝑎− 2𝑎P2 0ℎ ≤=ℎ 𝜎≤2𝑒𝑎−ℎ 𝑎
= 𝜎2 h ≥ 𝑎
or exponential model
Behavior of Covariance/Variogram
functions near the origin
• The nugget effect
• Pure nugget effect
Discontinuity at origin
Total absence of correlation
Behavior of Covariance/Variogram
functions near the infinity
• The presence of a sill on the variogram indicates second-order
stationarity, i.e. the variance and covariance exist
• If the variogram increases more slowly than h2 at infinity, this
indicates the process may be intrinsically stationary
𝜎𝑘 2, Pkk ℎ
Do not exist
for this case
𝛾(ℎ)
→0
ℎ →∞ ℎ 2
if lim
Intrinsically
stationary
i.e. 𝜃 < 2 for 𝛾 ℎ = ℎ𝜃
• If the variogram increases faster than h2 this suggests the
presence of higher order non-stationarity
e.g. fractal behavior
The hole effect
• A variogram (covariance) exhibits the hole effect if
its growth (decay) is non-monotonic
Ex. 𝛾 ℎ = 𝜎2 1 − sinℎ ℎ
P(h)= 𝜎2
sin ℎ
ℎ
or
2
P(h)= 𝜎2 1 − ℎ3 𝑒 − ℎ a2
• The hole effect is often the result of some ordered
periodicity in the data. If possible take care of this
deterministically
e.g. layered media
Example of the hole effect
Nested Structures
• Nested structures are the result of observation of
different scales of variability, i.e.
- measurement error
- pore-to-core scale variability
- core-to-lens scale variability
- lens-to-aquifer scale variability
• Variogram of total random field is represented by
the sum of variograms at each scale
Might observe
nugget effect if
meas. Grid is
not fine enough
Nugget effect closely related
to scale of observation and
measurement grid
The Cross-Covariance & CrossVariogram Functions
• In general the cross covariance can be an odd function, i.e.
E[K(xi),f(xj)] 𝑃𝑘𝜙 𝑥𝑖 , 𝑥𝑗 ≠ 𝑃𝑘𝜙 𝑥𝑗, 𝑥𝑖 but 𝑃𝑘𝜙 𝑥𝑖 , 𝑥𝑗 = 𝑃𝜙𝑘 𝑥𝑗, 𝑥𝑖
𝑃𝑘𝜙 ℎ ≠ 𝑃𝑘𝜙 −ℎ but 𝑃𝑘𝜙 ℎ = 𝑃𝜙𝑘 −ℎ
Examples?
h=(xi-xj)
• The cross variogram is always a symmetric even function because
interchanging xi and xj (or substituting –h for h) makes no difference,
i.e.
1
𝛾𝑘𝜙 ℎ = 𝐸 𝐾 𝑥 − 𝐾(𝑥 + ℎ) 𝜙 𝑥 − 𝜙(𝑥 + ℎ)
2
1
= 𝑃𝑘𝜙 0 − 𝑃𝑘𝜙 ℎ + 𝑃𝑘𝜙 −ℎ
2
“averages out” potentially odd terms
The Cross-Covariance & CrossVariogram Functions
• In empirical modeling the asymmetry of the crosscovariance function is often neglected because:
– Geostatistical applications generally use the crossvariogram which is symmetric
– Lack of data typically prevents discovering the physical
reality of the asymmetry
– Fitting valid models to asymmetric cross-covariances is
difficult
• However in physically-based stochastic modeling
asymmetric cross-covariances often arise.
Cross-covariance and Crossvariogram models
• Use of N multivariate random fields
requires modeling N*(N+1)/2 auto and
cross covariance (or variogram) models
• These models cannot be fit independently
from one another because entire covariance
matrix must be positive definite (positive
semi-definite for variograms)
Cross-covariance and Crossvariogram models
• Ensuring that the cross-covariance (variogram)
matrices for multivariate random fields are
positive (semi) definite can be tedious when fitting
models to data. Goovaerts (p. 108-123) outlines
one technique (linear co-realization) for doing so
• Stochastic modeling techniques ensure that the
resulting matrices are positive definite
Rules for Linear Model of Coregionalization (from Goovaerts)
• Every structure appearing in the cross semi-variogram
must be present in all auto- semivariograms
• If a structure is absent on an auto-semivariogram it must
be absent on all cross semivariograms involving this
variable
• Each auto- or cross-semi variogram need not include all
structures
• Structures appearing in all auto-semivariograms need not
be present in all cross semivariograms
• There are constraints on the coefficients of the structures to
ensure overall positive definiteness