Calibration and Reliability in Groundwater Modelling (Proceedings of the ModelCARE 96 Conference
held at Golden, Colorado, September 1996). IAHS Publ. no. 237, 1996.
523
Accuracy of uncertainty estimation using inverse firstorder reliability analysis
M. A. BAILEY
Cooperative Research Centre for Catchment Hydrology, Department of Civil
Engineering, Monash University, Clayton, 3168, Australia
Goulburn-Murray Water, Tatura, 3616, Australia
L. D. CONNELL
Cooperative Research Centre for Catchment Hydrology, Department of Civil
Engineering, Monash University, Clayton, 3168, Australia
R. J. NATHAN
Sinclair Knight Men Pty Ltd, 590 Orrong Road, Armadale, 3143, Australia
Abstract An inverse reliability procedure based on the first-order
reliability method (FORM) relevant to hydrological modelling is
presented. FORM and the inverse reliability procedure are compared to
Monte Carlo and mean-value first-order methods for reliability analysis
of two-dimensional contaminant transport in porous media. Both FORM
and the inverse procedure provide relatively accurate predictions of
reliability at parameter coefficients of variation up to 1.0 with significantly less computational effort than Monte Carlo simulation. Increased
parameter variation introduced prediction overestimation in the two
procedures, and created some instability in both the FORM and inverse
procedure algorithms. The inverse reliability procedure was observed to
offer greater stability than the FORM algorithm in this application.
INTRODUCTION
While hydrological models are useful tools for prediction and decisionmaking, even the
most sophisticated models are based on significant simplifications of the complex
interactions and spatial and temporal behaviour that occurs in the natural environment
(Beven, 1989). Inaccuracies in processes conceptualization, incomplete knowledge of
parameter behaviour and initial and boundary conditions, and numerical truncation
combine to introduce significant error in model output (Guymon, 1994).
In order to properly interpret model results, an indication of the uncertainty
associated with the output is desirable. Monte Carlo-based procedures are popular for
the analysis of prediction uncertainty under uncertain parameter information (e.g.
Freeze, 1975; Binley et al., 1991). While these procedures are generally robust, they
are computationally intensive, especially for low probability outcomes (McLaughlin &
Wood, 1988). As a result, there has been considerable interest in alternatives which
could offer computational savings. Dettinger & Wilson (1981), Townley & Wilson
(1985) and Connell (1995) have described computationally efficient techniques based on
second moment statistical information. Other procedures used include perturbation (e.g.
Tang & Pinder, 1977), and two-point estimation (e.g. Yen & Guymon, 1990).
524
M. A. Bailey et al.
More recently, reliability methods have attracted interest for use in uncertainty
analysis of hydrological modelling (e.g. Sitar et ai, 1987; Melching, 1992; Cawlfield
& Wu, 1993; Jang et al., 1994). These methods, originally derived for structural
engineering problems, are used to estimate the probability of exceedance of a target
value or of some other failure state. The primary benefit of these approaches is believed
to lie in accuracy comparable with other rigorous techniques such as Monte Carlo
analysis while requiring significantly less computational effort.
However, the approach taken for reliability analysis is not appropriate for many
hydrological applications. Most hydrological applications require the model result
associated with a given probability. The standard reliability approach applied to
hydrological problems to date has been to calculate the probability for a given outcome
(Melching et al., 1990). This paper presents and applies an inverse reliability procedure
based on the first-order reliability method (FORM). The performance and stability of
both FORM and its inverse procedure with respect to increasing parameter variability
is also examined.
RELIABILITY METHODS
Analysis of prediction uncertainty considers model output to be a function of
deterministic parameters and a set of random variables with known statistical properties.
Uncertainty or error in the modelling procedure is generated by these random
parameters.
Reliability methods are based on defining the probability that a target level is
exceeded:
PF = P[g(X) < 0]
(1)
where PF denotes the probability of failure, g(X) is known as the performance function
and involves the model of interest, and X is a vector of random model parameters.
The probability of failure defined by equation (1) is defined exactly as the integral
of the j oint probability distribution function (PDF) of X in the region where g(X) is < 0
(Jang et al., 1994). This integral is usually impractical to evaluate due to its complexity,
or more commonly, limited knowledge of the behaviour of the uncertain parameters
(Melching, 1992). In practical implementations of reliability methods, the integral is
approximated.
First-order reliability method
One approach to approximating the probability integral is the first-order reliability
method, FORM (Madsen et al. ,1986; Sitar et al., 1987). The first step in the procedure
is the transformation of the random variables, X, of the model to uncorrelated standard
normal deviates, U, with zero mean and unit variance. With second moment statistical
information available, the transformation is summarized by (Sitar et al., 1987):
U = L~lD-\X-M)
(2)
where D = diag[o-,] is a diagonal matrix of the standard deviations, L = lower triangular
Accuracy of uncertainty estimation using inverse first-order reliability analysis
525
(Cholesky) decomposition of the correlation matrix R = \py\ such that R = LU and M
= mean vector of the variables of X. The performance function is then expressed in
standard space by:
G{U) = g{DLP+M) = g{X)
(3)
Der Kiureghian & Liu (1986) and Sitar et al. (1987) discuss incorporation of marginal
distribution information for the parameters into the transformation.
Hasofer & Lind (1974) introduced the concept of the reliability index (/?) and
demonstrated its equivalence to the distance from the origin to the nearest point on the
limit state surface defined by the performance function in standard space. The reliability
index is obtained by linearizing the performance function at the point on the limit state
surface nearest the origin (called the design point, denoted by u* in standard space and
JC* in original space), and defined by:
p = a*u*
(4)
where a* is a line vector describing the unit normal at the design point on the failure
region directed towards region of failure (Sitar et al., 1987).
The first-order approximation of the probability of failure is found by assuming the
performance function in standard space to be normally distributed, and is given by:
PF = *(-P)
(5)
where $(2) is the standard normal cumulative probability evaluated at i.
Algorithm for determining the design point The design point and hence reliability
index is found from solution of a constrained optimization problem to minimize the
distance from the origin subject to the point being on the limit state surface (Zhang &
Der Kiureghian, 1994):
p = mm{\U\\G{U) = 0}
(6)
Various algorithms are available for the solution of this problem (e.g. Liu & Der
Kiureghian, 1991). The Hasofer & Lind-Rackwitz & Fiessler (HL-RF) algorithm
(Hasofer & Lind, 1974; Rackwitz & Fiessler, 1978) is used in both structural and
hydrological applications (e.g. Madsen et al., 1986; Sitar et al, 1987; Jang et al.,
1994). The algorithm locates the design point with an iterative procedure:
»*•! = -^^[^(uk)uk-(uk)fuG(uky
(7)
8G(.uk)
du1
(8)
where:
\G(uk)
-
dG(uk)
3K„
defines the gradient vector of the limit state surface in the standard normal space.
Liu & Der Kiureghian (1991) introduced a merit function to improve the
convergence properties of the HL-RF algorithm. A simpler form of the merit functions:
m(u) = 1 / 2 |M|+C|G(M)|
(9)
M. A. Bailey et al.
526
where c ( > 0) is a penalty parameter, is described by Zhang & Der Kiureghian (1994),
and has been implemented for the FORM analyses conducted in this paper.
Inverse reliability - modified FORM
Inverse reliability has been investigated in structural engineering applications (Der
Kiureghian et al., 1994) and groundwater transport (Schanz & Salhotra, 1992). The
procedure described by Schanz & Salhotra (1992) is a modification of the HL-RF
algorithm (equation (7)), and was used to locate the target concentration and design point
for several transport models.
Zhang & Der Kiureghian (1994) have proposed additional modifications to the
HL-RF algorithm to permit general inverse reliability investigations. The performance
function is modified to include an unknown deterministic parameter 6, such that G(U)
= G(U,B). This parameter may be considered to be the target value of the performance
function (as implicitly done by Schanz & Salhotra (1992)), or a component of the model
function. Inverse reliability identifies 6 such that the reliability index for the
performance function equals a known target /3r:
e:min{|£/||G(C/,e)} = P7
(10)
The minimization problem is defined by the set of equations (Zhang & Der Kiureghian,
1994):
(Ha)
0
u+
KG(u,d)\
VG(u,Q) = 0
G{u,Q) = 0
(lib)
(He)
These equations are then linearized and solved. With the iteration algorithm described
by xi+1 = Xj + dt, the search direction di is given by the solution of the linearized
equations:
"ft
v„G(M,)
(12)
^ =
dGiu^ldO
Zhang & Der Kiureghian (1994) use a merit function to control the value of dh to
improve convergence of the algorithm. The function is given by:
m(u) = V2\u\2 + c\G(u,Q)\
( 13 )
Implementation of the algorithm requires c > PT\u\/ô, where <5 is the required tolerance
in satisfying equation (lie).
Both the techniques of Schanz & Salhotra (1992) and Zhang & Der Kiureghian
(1994) require initial normalization of variables, although Schanz & Salhotra do not
Accuracy of uncertainty estimation using inverse first-order reliability analysis
527
specify that the variables are to be uncorrelated in standard space. The solution for the
input vector Zx of Schanz & Salhotra is similar to that used by Zhang & Der Kiureghian
for iteration.
RESULTS AND DISCUSSION
The behaviour of the first-order reliability method and inverse reliability were examined
by application to a simple analytical solution of a two-dimensional contaminant transport
problem. The predictions were then compared with an equivalent Monte Carlo analysis,
and the mean-centred first-order method of Ang & Cornell (1974). The Monte Carlo
simulation used 20 000 iterations to ensure accuracy at low probabilities of failure.
The solution for transport through porous media was given by Kinzelbach (1986) as:
C(x,y,y) _
C,
o
1
4TI./C
(
„
1
\
: exp
V2KL;
w
r2R
4aLvt
ry ^
2a L
(14)
where C is the concentration at point (x,y) and time t, C0 is the concentration of the
incoming contaminant, v is the steady-state velocity, aL is the longitudinal dispersivity,
aT is the transverse dispersivity, X is the degradation rate of the contaminant, and R is
the retardation factor. The function W(al,a2) is an integral term called the Hantushfunction (Hantush, 1956), r is the weighted radial distance defined by r2 = x2 +
(aL/aT)y2, and y = (1 + 4aL\R/u)0-5.
The performance function was defined for an ideal contaminant (R = 1, X = 0) at
x = 10 m, y = 0.5 m, and t — 10 years. The steady-state velocity, longitudinal
dispersivity and transverse dispersivity were treated as uncertain variables, with /xv= 1.0
m day"1, p,aL = 4.5 m, and/x„r = 1.125 m respectively. The standard deviations of the
parameters were varied during the analyses.
As it was desired to examine the effects of increasing uncertainty, all variables were
assumed to be independent and lognormally distributed. It is believed important to
establish the relative accuracy of FORM and inverse reliability in comparison with the
other procedures at high levels of uncertainty, and to identify any problems in the
implementation of the methods.
The initial investigations focused on the accuracy and stability of the FORM
procedure. The standard deviation of all three uncertain parameters was fixed at 10%
of their mean value (a coefficient of variation = 0.1 ). A cumulative distribution function
(CDF) of the probability of failure of the performance function was established by
incrementally varying the target concentration ratio (i.e. C/CQ).
For the initial analysis run, both the mean-value approach and FORM were found
to provide very good matches with the probability of failure calculated form the more
rigorous Monte Carlo procedure. This was expected, as techniques based on first-order
linearization of Taylor series expansions have been found to provide reasonable accuracy
for low variances (Melching, 1992).
Increasing the standard deviation of all values to 50% of the mean or a coefficient
of variation of 0.5 (Fig. 1) was found to contribute to significant divergence of the
mean-value approach from the Monte Carlo data. However, the FORM data was found
to provide good agreement, while requiring significantly reduced computational effort.
528
M. A. Bailey et al.
0.15
0.2
0.25
0.3
0.35
0.45
TARGET CONCENTRATION RATIO
Fig. 1 Comparison of FORM with Monte Carlo (population 20 000) and mean-value
first-order methods at coefficient of variation = 0.50.
It was expected that further increases in the skewness of the lognormal distribution
would contribute to increased divergence of the mean-value first-order technique
(Melching, 1992). Cawlfield & Wu (1993) and Jang et al. (1994) have previously
considered the effects of increased variability on the behaviour of FORM, and noted
divergence when compared with Monte Carlo data. Our study was also concerned with
the accuracy of the procedure, but expanded to include Considerations on the effects of
uncertainty on the stability on the solution.
Further runs were conducted with standard deviations set to 100%, 150% and 200%
of the mean values (Figs 2-3). The increased parameter variability was found to
0.15
0.2
0.25
0.3
0.35
TARGET CONCENTRATION RATIO
Fig. 2 Comparison of FORM with Monte Carlo (population 20 000) and mean-value
first-order methods at coefficient of variation = 1.00.
Accuracy of uncertainty estimation using inverse first-order reliability analysis
1
•
A
0.9
111
E
0.8
3 0.7-
\\
IV,
\
V
\\
\
0.5
\
ID
<
g
MC
MVFO
FORM
\Y
£ " •
o
j*.
529
\
0.3
DC
°- 0.2
^
0.10
^
1
1
1
1
0.0S
0.1
0.15
0.2
1—~-
0.2S
i
1
1
0.3
0.35
0.4
mi
|
0.45
TARGET CONCENTRATION RATIO
Fig. 3 Comparison of FORM with Monte Carlo (population 20 000) and mean-value
first-order methods at coefficient of variation = 2.00.
ultimately contribute to widespread disagreement in probability estimation. The
probability of failure estimated by FORM was found to consistently overestimate that
from Monte Carlo simulation for coefficients of variation in excess of 1.0.
It was also noted that the ability of FORM to identify an appropriate design point
progressively deteriorated, resulting in local (minor) instabilities. Similar behaviour has
been observed in other applications of FORM (Melching, 1995). Significantly poorer
stability was observed for several simulations conducted with a large standard deviation
in a single variable. Further investigation suggested that the choice of initial parameters
for the FORM iteration largely contributed to this instability.
The inverse reliability procedure was applied and tested in a similar manner to that
used for FORM. Simulations were again conducted at increasing levels of standard
deviation, and are presented in Fig;T 2, where they are compared against equivalent
Monte Carlo analysis.
The inverse reliability procedure .is in close agreement with the Monte Carlo results
for probabilities of failure corresponding to standard deviations up to 100% of mean
values. Beyond this level of uncertainty,, the procedure overestimated the probability of
failure associated with a given target level. With standard deviations set at 150% of the
magnitude of the mean, significant divergence was observed for probabilities of failure
between approximately 0.05 and 0.9. Reasonable agreement occurred at lower
probabilities. This agreement deteriorates markedly as the standard deviation is
increased further (Fig. 4). This behaviour mirrors that found by Schanz & Salhotra
(1992), who found that their procedure calculated accurate estimates of 90th and 95th
percentile targets, but poorer estimates of the 50th percentile targets.
A feature of the inverse procedure was greater stability in comparison with FORM.
Some instability in the algorithm was observed at lower probabilities of failure, as
occurred with FORM. However, thèse did not cause excessive inaccuracy in the
algorithm, and did not contribute to catastrophic instability.
530
M. A. Bailey et al.
PROBABILITY OF FAILURE
Fig. 4 Comparison of inverse reliability with Monte Carlo method (population 20 000).
CONCLUSIONS
The objective of this paper was to provide a perspective into the accuracy and stability
of reliability methods applied to hydrological engineering. Emphasis was placed on the
ability of the first-order reliability method (FORM) and an inverse reliability procedure
to predict probabilities of failure associated with a particular performance function.
The analysis confirmed that use of mean-value first-order techniques for uncertainty
evaluation is subject to error for applications involving high parameter variances, such
as those commonly encountered in practical problems. When compared to the meanvalue procedure, FORM offered greater accuracy up to levels of parameter uncertainty
above a coefficient of variation of 1.0. It was also found at these high variances that the
robustness of the FORM approach was at times questionable for this particular
application, particularly when the search algorithm appeared to identify local minima,
leading to numerical instability and ultimately failure of the algorithm.
Comparison of the inverse reliability procedure with the results of Monte Carlo
simulation was also favourable to coefficients of variation in the order of 1.0. The
inverse technique, which is thought to be more applicable to common hydrological
problems than the FORM approach on which it is based, was also found to have greater
numerical stability for the example considered. However, this stability cannot be
guaranteed for other applications.
The results just described were applicable for a relatively simple case of contaminant
transport. Requiring further investigation is the behaviour of reliability methods for
more complex situations such as those encountered with multi-process (i.e. catchment)
models. In such models, inaccuracies in individual processes must be combined to
evaluate overall prediction uncertainty. Problems associated with locating the global
minima become more difficult as the performance function becomes more complex. In
Accuracy of uncertainty estimation using inverse first-order reliability analysis
531
multi-process models, these problems are exacerbated by difficulties in defining
performance functions appropriate to the components of the model. The definition of the
inverse reliability procedure implies application to a whole system rather than individual
processes. Error propagation in multi-process models, and incorporation of techniques
such as FORM and inverse reliability, is the subject of ongoing research.
REFERENCES
Ang, A. H.-S. & Cornell, A. C. (1974) Reliability bases of structural safety and design. J. Struct. Div. ASCE 100(ST9),
1755-1769.
Beven, K. (1989) Changing ideas in hydrology — the case of physically-based models. J. Hydrol. 105, 157-172.
Binley, A. M., Beven, K. I., Calver, A. & Watts, L. G. (1991) Changing responses in hydrology: assessing the uncertainty
in physically based model predictions. Wat. Resour. Res. 27(6), 1253-1261.
Cawlfield, J. D. & Wu, M.-C. (1993) Probabilistic sensitivity analysis for one-dimensional reactive transport in porous
media. Wat. Resour. Res. 29(3), 661-672.
Connell, L. D. (1995) An analysis of perturbation based methods for the treatment of parameter uncertainty in numerical
groundwater models. Transport in Porous Media 21, 225-240.
Der Kiureghian, A. &Liu,P.-L. (1986) Structural reliability under incompleteprobability information. J. EngngMech.Div.
ASCE 117(1), 85-104.
DerKiureghian.A., Zhang, Y. &&Li,C.-C. (1994) Inverse reliability problem./. EngngMech. Di v. ASCE 120(5), 11541159.
Dettinger.M. D. & Wilson, J. L. (1981) First order analysisof uncertainty in numerical models of groundwater flow. Part
1: Mathematical development. Wat. Resour. Res. 17(1), 149-161.
Freeze, R. A. (1975) A stochastic-conceptualanalysis of one-dimensional groundwater flow in nonuniform homogenous
media. Wat. Resour. Res. 11(5), 725-741.
Guymon, G. L. (1994) Unsaturated Zone Hydrology. Prentice Hall, Englewood Cliffs, New Jersey.
Hantush, M. S. (1956) Analysis of data from pumping tests in leaky aquifers. Trans. AGU 37(6), 702-714.
Hasofer, A. M. & Lind, N. C. (1974) Exact and invariant second-moment code format. /. Engrs Mech. Div. ACSE
100(EM1), 111-121.
Jang, Y.-S., Sitar, N. & Der Kiureghian, A. (1994) Reliability analysis of contaminant transport in saturated porous media.
Wat. Resour. Res. 30(8), 2435-2448.
Kinzelbach, W. (1986) Groundwater Modelling. An Introduction with Sample Programs in BASIC. Elsevier Science
Publishers, Amsterdam.
Liu, P.-L. & Der Kiureghian, A. (1991) Optimization algorithms for structural reliability. Structural Safety 9, 161-177.
Madsen, H. O., Krenk, S. &Lind, N. C. (1986) Methods ofStructural Safely. Prentice-Hall, Englewood Cliffs, New Jersey.
McLaughlin, D. &Wood, E. F. (1988) Distributed parameter approach for evaluating the accuracy of groundwater model
predictions. 1. Theory. Wat. Resour. Res. 24(7), 1037-1047.
Melching, C. S. (1992) An improved first-order reliability approach for assessing uncertainties in hydrologie modeling. /.
Hydrol. 132, 157-177.
Melching, C. S. (1995) Reliability estimation. In: Computer Models of Watershed Hydrology (ed. by V. P. Singh), 69-118.
Water Resources Publications, Highlands Ranch.
Melching, C. S., Yen, B. C. & Wenzel, H. G. (1990) A reliability estimation in modeling watershed runoff with
uncertainties. Wat. Resour. Res. 26(10), 2275-2286.
Rackwitz, R. & Fiessler, B. (1978) Structural reliability under combined load sequences. Comp. and Struct. 9, 489-494.
Schanz, R. W. & Salhotra, A. (1992) Evaluation of the Rackwitz-Fiessler uncertainty analysis method for environmental
fate and transport methods. Wat. Resour. Res. 28(4), 1071-1079.
Sitar, N., Cawlfield, J. D. & Der Kiureghian, A. (1987) First-order reliability approach to stochastic analysis of subsurface
flow and contaminant transport. Wat. Resour. Res. 23(5), 794-804.
Tang, D. H. &Pinder, G. F. (1977) Simulation of groundwater flow and mass transport. Adv. Wat. Resour. 1(1), 25-30.
Townley, L. R. & Wilson, J. L. (1985) Computationally efficient algorithms for parameter estimation and uncertainty
propagation in numerical models of groundwater flow. Wat. Resour. Res. 21(12), 1851-1860.
Yen, C.-C. & Guymon, G. L. (1990) An efficient deterministic-probabilisticapproach to modeling regional groundwater
flow. 1. Theory. Wat. Resour. Res. 26(7), 1559-1567.
Zhang, Y. & Der Kiureghian, A. (1994) Two improved algorithms for reliability analysis. In: Reliability and Optimization
of Structural Systems (ed. by R. Rackwitz, G. Augusti & A. Borri) (Proc. Sixth IFIP WG7.5 Working Conf,
September 1994), 297-304. Chapman & Hall, London.