Stoch Environ Res Risk Assess (2014) 28:491–504
DOI 10.1007/s00477-013-0767-1
ORIGINAL PAPER
The comparison of sensitivity analysis of hydrological uncertainty
estimates by GLUE and Bayesian method under the impact
of precipitation errors
Lu Li • Chong-Yu Xu
Published online: 15 August 2013
Springer-Verlag Berlin Heidelberg 2013
Abstract The input uncertainty is as significant as model
error, which affects the parameter estimation, yields bias
and misleading results. This study performed a comprehensive comparison and evaluation of uncertainty estimates according to the impact of precipitation errors by
GLUE and Bayesian methods using the Metropolis Hasting
algorithm in a validated conceptual hydrological model
(WASMOD). It aims to explain the sensitivity and differences between the GLUE and Bayesian method applied to
hydrological model under precipitation errors with constant
multiplier parameter and random multiplier parameter. The
95 % confidence interval of monthly discharge in low flow,
medium flow and high flow were selected for comparison.
Four indices, i.e. the average relative interval length, the
percentage of observations bracketed by the confidence
interval, the percentage of observations bracketed by the
unit confidence interval and the continuous rank probability score (CRPS) were used in this study for sensitivity
analysis under model input error via GLUE and Bayesian
methods. It was found that (1) the posterior distributions
derived by the Bayesian method are narrower and sharper
than those obtained by the GLUE under precipitation
errors, but the differences are quite small; (2) Bayesian
L. Li (&)
Uni Climate, Uni Research, Bergen, Norway
e-mail: lu.li@uni.no
L. Li
Bjerknes Centre for Climate Research, Bergen, Norway
C.-Y. Xu
Department of Geosciences, University of Oslo, Oslo, Norway
C.-Y. Xu
Department of Earth Sciences, Uppsala University, Uppsala,
Sweden
method performs more sensitive in uncertainty estimates of
discharge than GLUE according to the impact of precipitation errors; (3) GLUE and Bayesian methods are more
sensitive in uncertainty estimate of high flow than the other
flows by the impact of precipitation errors; and (4) under
the impact of precipitation, the results of CRPS for low and
medium flows are quite stable from both GLUE and
Bayesian method while it is sensitive for high flow by
Bayesian method.
Keywords GLUE Bayesian Precipitation error Uncertainty Sensitivity Hydrological model
1 Introduction
Hydrological models are important tools for improving our
understanding of catchment dynamics, supporting water
resources management, and predicting hydrologic impacts
produced by future environmental changes. However,
model calibrations and subsequent predictions will be
subject to uncertainty, which arises in that no rainfallrunoff model is a true reflection of the processes involved,
and is impossible to specify the initial and boundary conditions required by the model with complete accuracy.
Hydrologists have been exploring many methodologies
to better treat uncertainty and applied to various models
and catchments. According to Montanari (2007), all of
these methods can be broadly classified into four types: (1)
approximate analytical methods; (2) techniques based on
the statistical analysis of model errors; (3) approximate
numerical methods/sensitivity analyses; (4) non-probabilistic methods. Numerous approaches for quantifying the
validation uncertainty for the model output have been
proposed. In which, two kinds of method have been widely
123
492
used, i.e. the Generalized Likelihood Uncertainty Estimation (GLUE) method (Beven and Binley 1992; Beven and
Freer 2001) and the Bayesian method (Engeland et al.
2005; Krzysztofowicz 1999; Thiemann et al. 2001; Vrugt
et al. 2009; Li et al. 2013a). The generalized likelihood
uncertainty estimation (GLUE) method was first proposed
by Beven and Binley (1992) in order to quantify the
parameters uncertainty. The Bayesian method which needs
to understanding the mathematics and statistics is difficult
to implement since it is hard to find the proper statistic
model which fits the data. However, numerous approaches
for quantifying the uncertainty in hydrological modelling
based on Bayesian method have been proposed (Blasone
et al. 2008a, b; Vogel et al. 2008; Vrugt et al. 2003a, 2009;
Yang et al. 2008).
Generally speaking, there are four important sources of
uncertainties in hydrological modelling, i.e. uncertainties in
input data, uncertainties in output data used for calibration,
uncertainties in model parameters and uncertainties in model
structure (Refsgaard and Storm 1996). However, it will be
difficult to study the modeling uncertainty considering all of
the four error sources simultaneously, partly because they
are not all independent of each other. The problem can be
simplified by considering these uncertainties separately
based on certain assumptions. Various methods have been
developed to analyze uncertainty sources in hydrological
modelling (Beven and Binley 1992; Refsgaard et al. 2006;
Vrugt et al. 2003b; Alvisi et al. 2012; Shen et al. 2013).
Recently, input uncertainty has been considered more and
more important in hydrological modelling. Kavetski et al.
introduced the Bayesian total error analysis (BATEA)
framework to estimate the input uncertainty and parameters
(Kavetski et al. 2002, 2006a, b). Bayesian model averaging
is put forward to analyse the structural deficiencies of the
specific model (Duan et al. 2007; Marshall et al. 2007; Tsai
2010). And it was developed to a new framework called
integrated Bayesian uncertainty estimator (IBUNE) which is
aimed to distinguish the various sources of uncertainty
including parameters, input and model structure uncertainty
(Ajami et al. 2007). Renard et al. (2009) used a set of
probabilistic calibration methods to analyse input and
structure errors quantitatively.
Some criteria have been proposed to evaluate the performance of uncertainty analysis methods: (1) indices for
compare the efficiency of the posterior distribution based
on the Nash–Sutcliffe (NS) coefficient or other objective
functions (Jin et al. 2010; Yang et al. 2008); (2) indices
used to compare the sharpness of the model uncertainty
intervals, which include the width of 95 % confidence
interval (95CI), and the average distance between the upper
and the lower limits of 95 % confidence intervals (95PPU)
(Li et al. 2009; Xiong et al. 2009; Yang et al. 2008); and (3)
indices to compare the reliability of the uncertainty
123
Stoch Environ Res Risk Assess (2014) 28:491–504
estimates, i.e. the percentage of observations bracketed by
the 95CI (PCI) and the continuous rank probability score
(CRPS) (Engeland et al. 2010; Li et al. 2013a, b).
However, none of the above applications included the
sensitivity analysis of the indices in evaluation of uncertainty assessment method, especially for different level of
flows, i.e. low flow, medium flow and high flow. It is
common knowledge that the uncertainty of hydrological
simulation is dependent on the input accuracy. However,
the question remains how the precipitation errors impact on
the performance of the uncertainty estimate methods and
how indices of uncertainty evaluation responses to different precipitation errors via GLUE and Bayesian methods.
The objective of this paper is to attempt to partly fill this
gap.
This study performed a comprehensive comparison and
evaluation of uncertainty estimates according to the impact
of precipitation errors by GLUE and Bayesian methods
using the Metropolis Hasting (MH) algorithm in a validated conceptual hydrological model (WASMOD). It seeks
to explain the sensitivity and differences between the
GLUE and Bayesian method applied to hydrological model
under precipitation errors with constant multiplier parameter and random multiplier parameter. In water and snow
balance modelling system (WASMOD) (Xu 2002; Kizza
et al. 2013; Li et al. 2010, 2011, 2013b), the 95CI of
monthly discharge in low flow, medium flow and high flow
were selected for comparison. Four indices, i.e. the average
relative interval length (ARIL), the percentage of observations bracketed by the confidence interval (PCI), the percentage of observations bracketed by the unit confidence
interval (PUCI) and the CRPS were used in this study for
sensitivity analysis under model input error via GLUE and
Bayesian methods. Finally, the paper tries to give some
suggestions on the choice of uncertainty estimate method
and evaluation indices with input uncertainty in hydrological model applications.
2 Method
2.1 Perturbing error
This paper considered the impact of precipitation uncertainty by perturbing method which is assessed by using
observed precipitation, which is assumed as ‘‘perfect’’, then
perturb that precipitation by multiplying parameter (e.g.
Monte Carlo method to create a range of different values of
precipitation within a selected interval) and see what
impact it has on. These multipliers are considered to be
systematic error and stochastic error in precipitation.
Rainfall depth multipliers which are considered to be latent
variables to the system are introduced by Kavetski et al.
Stoch Environ Res Risk Assess (2014) 28:491–504
(2002). They put forward an explicit term to the likelihood
function to estimate both variables and model parameters
by the probabilistic calibration procedure which is called
the BATEA. Besides, Ajami et al. (2007) developed a
random multiplier ut which is normally distributed with
mean equals to l and variance equals to r2l at time step t
which is expressed as follows:
r~t ¼ ut rt u N l; r2l ;
ð1Þ
493
subjectively determined and this was discussed a lot before
(Freer et al. 1996; Li et al. 2010; Smith et al. 2008). In this
study, the standardize NS value was chosen as the likelihood function:
T P
Robs;t Rsim;t
Lðhi jY Þ ¼ 1 t¼1
T P
Robs;t Robs
2
2
¼1
r2i
r2obs
r2i \r2obs
t¼1
ð3Þ
where r~t represents true precipitation depth at time step t; rt
is observed precipitation depth at time step t. Compared
with BATEA, it contains less variables, i.e. l and r2l ,
which are needed to be estimated. However, it still added
two more parameters in calibration procedure. In this
paper, it assumed that the observed precipitation depth is
true and which will be perturbed by adding either a
systematic multiplier or a random multiplier by following
form:
0
ut rt u N ðl; r2 Þ
rt ¼
ð2Þ
ut rt u cons tan t
where Lðhi jY Þ is the likelihood measure; Robs;t is the
observed discharge; Rsim;t is the simulated discharge, which
is depending on the model parameter hi; Robs is the average
value of Robs;t ; r2i is the variance of errors for the given
parameter set hi and the observed discharge data set Y; and
r2obs is the variance of the observed data set.
2.3 Bayesian method
2.3.1 Bayesian inference
GLUE and Bayesian method are used to estimate the
model and parameter uncertainty under the impact of
precipitation errors.
The Bayesian theorem is expressed as follows:
2.2 GLUE
where / ¼ fh; xg, in which h represents the hydrological
model parameters and x represents statistical parameters.
The posterior density pð/jgÞ can be derived from the prior
density f(/) and the likelihood function f ðgj/Þ. Variable g
is a transformed variable from the original space.
The GLUE methodology was proposed by Beven and
Binley (1992), in which hundreds and thousands of model
runs are made with randomly chosen parameter values
from a priori probability distribution. The acceptability of
each run is evaluated against observed values. The run is
considered to be ‘‘non-behavioral’’ and to be removed from
further analysis if the acceptability is below a certain
subjective threshold. A threshold is either defined in terms
of a certain allowable deviation of the highest likelihood
value in the sample, or sometimes as a fixed percentage of
the total number of simulations (Blasone et al. 2008a). The
fixed percentage method was used in the study, which is
named as the acceptable sample rate (ASR) (Li et al. 2010).
According to the research results of Li et al. (2010), there is
a good linear relation between threshold values and ASR
for hydrological models and ASR has been tested as a main
subjective factor for the uncertainty estimation result of
GLUE. Li et al. (2010) found that for monthly WASMOD,
when ASR is bigger than 1 %, the 95CI of discharge is
much less sensitive to the change in ASR. So in this study,
the ASR is chosen to be 1 % of the sampling size that is
200,000.
In the GLUE method, the likelihood values serve as
relative weights of each parameter set or simulated value. It
is noted that the likelihood function and the threshold are
pð/jgÞ ¼ R
f ðgj/Þ f ð/Þ
;
f ðgj/Þ f ð/Þd/
ð4Þ
2.3.2 Likelihood function
In this study, Box_Cox transformation was used to transform the observed and simulated discharges and the
resulting residuals are normally distributed, which has been
used a lot in hydrological uncertainty analyses before
(Engeland and Gottschalk 2002; Engeland et al. 2005; Jin
et al. 2010; Wang et al. 2009; Yang et al. 2007a). It can be
expressed as:
k
y 1
k 6¼ 0 :
k
gðy; kÞ ¼
ð5Þ
lnðyÞ k ¼ 0
In the study of Li et al. (2011), the Lilliefors test
(Lilliefors 1967) was used to verify the normality of the
distribution of the residuals of the Box–Cox transformed
variable for the monthly WASMOD in Chao River basin.
They found the Box–Cox transformation yields residuals
closest to normal for values of k = 0.2 in monthly
WASMOD. So in this study, the parameter k is chosen to
be 0.2. And the likelihood function is the key issue in the
Bayesian method, which is defined by the distribution of
123
494
Stoch Environ Res Risk Assess (2014) 28:491–504
M
residuals. nt represents the residual of gt ðyt Þ and gM
as
t yt
follows:
M
ð6Þ
nt ¼ gt ðyt Þ gM
t yt ;
where yM
t and yt represent the simulated and observed
variables in original space, respectively; gM
t and gt are
transformed variables. It is needed to check whether nt is
independent. If not, an AR (1) model is used to make the
residuals independent (Eq. 7).
nt ¼ ant1 þ b þ et :
ð7Þ
The Jarque–Bera test (Carlos and Bera 1980) was used
to check the independence of residuals, and the results
showed that an AR (1) model was needed to make the
transformed residuals independent. The resulting
likelihood function is:
Y
T
1 n
1 nt ant1 b
f ðnj/Þ ¼ q 0 q
:
ð8Þ
r
r
r
r
t¼1
In which / represents all parameters; T represents the
time; q represents the normal density operator. Assuming
that there is no systematic error in monthly residuals,
which results in the following likelihood function:
Y
T
1 n
1 nt ant1
f ðnj/Þ ¼ q 0 q
:
ð9Þ
r
r
r
r
t¼1
All the notations are as defined above.
2.3.3 Posterior density
There is no information about the distribution of parameters. Parameter r is an unknown model standard deviation
and with Jeffreys’ uninformative prior it is proportional to
r-1 (Bernardo and Smith 1994; Yang et al. 2007b).
Besides, the prior probability densities of other parameters
are generally taken as non-informative multi-uniform distribution in hydrological applications (Engeland et al.
2005; Jin et al. 2010; Liu et al. 2005; Yang et al. 2008).
The prior densities and intervals of all parameters in the
hydrological models are shown in Table 1. With the priori
density f ð/Þ considered to be uniform, the posterior density pð/jnÞ is given as follows:
f ðfjuÞ f ðuÞ
:
f ðfjuÞ f ðuÞdu
Substituting Eq. 9 into Eq. 10 results in:
Q
T
n0
nt ant1
1
q
q
f ð/ Þ
r
r
r
t¼1
pð/jnÞ ¼
T
R 1 n0 Q
nt ant1
q
q
f ð/Þd/
r
r
r
pðujfÞ ¼ R
ð10Þ
ð11Þ
Table 1 The prior distributions of all parameters in WASMOD
Parameter
a1
a2
a3
rb
Prior distributionsa
U[0, 1]
U[0, 1]
U[0, 1]
r-1
a
U ½a; b means the prior distribution of the parameter is uniform over
the interval½a; b
b
/ r1 means the prior density of the parameter at value r is proportional to r1
2.4 Metropolis hasting algorithm
A Markov Chain Monte Carlo (MCMC) methodology,
called MH algorithm (Hastings 1970), was used to get the
posterior distributions and estimate the parameters (Chib
and Greenberg 1995; Engeland and Gottschalk 2002; Kuczera and Parent 1998; Li et al. 2010). Furthermore, the
pffiffiffi
scale reduction score R (Gelman and Rubin 1992) was
used to check the convergence of Monte Carlo chains.
2.5 Ninety-five percentage confidence intervals
of discharge
Discharge values for each month were obtained by running
the hydrological model with all parameter sets from the
MH samples. The 95 % confidence intervals for discharge
due to parameter uncertainty are estimated by these discharge samples. The 2.5 % percentile and 97.5 % percentile of discharge will be derived by sorting ascending of all
discharge samples at each month. The intervals in GLUE
method consider weighted values, while is not in the
Bayesian method (Engeland et al. 2005). The detail steps of
calculation can be seen in the studies of Li et al. (2010,
2011).
2.6 Criteria for comparison
In this study, four indices were used to compare the derived
95CI of monthly discharge: the ARIL (Jin et al. 2010; Li
et al. 2010) is used for measuring sharpness, PCI (Li et al.
2009) is used for reliability, the percentage of observations
bracketed by the PUCI (Li et al. 2011) and the CRPS
(Hersbach 2000) are used for measuring efficiency which
combine sharpness and reliability.
1 X LimitUpper;t;p LimitLower;t;p
ARIL ¼
:
ð12Þ
T
Robs;t
LimitUpper;t;p and LimitLower;t;p are the upper and lower
boundary values of the p confidence interval, T is the
number of time steps, Robs;t is the observed discharge.
t¼1
in which r [ 0, a 2 ½0; 1Þ:
123
PCI ¼
NQin;p
:
T
ð13Þ
Stoch Environ Res Risk Assess (2014) 28:491–504
495
NQin;p is the number of observations which are
contained within the p confidence interval. PCI was
plotted as a function of p and should be close to the 45
diagonal line.
PUCI ¼ ð1:0 AbsðPCI 0:95ÞÞ=ARIL:
ð14Þ
Abs means absolute value. The PUCI is only used in
95CI evaluation, ranges from zero to infinity, and the upper
boundary is not clear (Li et al. 2011).
The crps for one time step for an ensemble with m
members is calculated as:
m m X
m X
1X
yi y
1
yi y j crpsðF; yÞ ¼
ð15Þ
2
m i¼1
2m i¼1 j¼1
where y is the observed value and yi is a sample member
and the || indicates the absolute distance. The calculations
above can be done faster by approximating the last double
sum with a single sum:
crpsðF; yÞ ¼
m m X
1X
1
yi y
yi yiþ1 :
m i¼1
2ðm 1Þ i¼1
CRPS ¼
T
1X
crpsðFt ; yt Þ:
T t¼1
ð17Þ
The CRPS is averaged over the whole time series. The
minimal value zero of CRPS is only achieved when the
empirical distribution is identical to the predicted
distribution, that is, in the case of a perfect deterministic
forecast (Hersbach 2000; Yang et al. 2008).
An ideal uncertainty analysis technique would lead to a
95 % probability band that is as narrow as possible while
still being a correct estimate under the statistical assumptions of the technique. The ARIL and CRPS should be as
small as possible while the PCI should be as close to 0.95
as possible. The larger the PUCI the lower the uncertainty
of 95CI of discharge is.
3 Study area and hydrological models
3.1 Study area
ð16Þ
To get the CRPS for a time series, take the average of all
crps:
The GLUE and Bayesian methods have been applied to
the Chao River basin upstream of the Miyun reservoir
with a drainage area of 5,300 km2 (Fig. 1). Available
Fig. 1 Main river and
meteorological stations and
discharge station of Chao River
basin in North China
Table 2 The main equations in WASMOD
Actual evapotranspiration
et ¼ minfðsmt1 þ pt Þð1 expða1 ept Þ; ept Þg
2
0 a1 1
Slow flow
st ¼ a2 ðsmt1 Þ
o a2 1
Fast flow
ft ¼ a3 ðsmt1 Þðpt ept ð1 expðpt =maxðept ; 1ÞÞÞÞ
o a3 1
Total flow
d t ¼ s t þ ft
Water balance
smt ¼ smt1 þ pt et dt
In which, t is t th month, pt is precipitation, ept is potential evaporation, smt is t th monthly soil moisture, SCt–1 is channel storage at time step t–1
123
496
Table 3 The scenarios of
uncertainty estimates by GLUE
and Bayesian method for
WASMOD with different
precipitation errors
Stoch Environ Res Risk Assess (2014) 28:491–504
The describe of scenarios
Abbreviation of scenarios
GLUE under systematic error with ut a equals to 1.1
gsa
GLUE under systematic error with ut equals to 1.05
gsb
GLUE under systematic error with ut equals to 1.0
gsc
GLUE under systematic error with ut equals to 0.95
gsd
GLUE under systematic error with ut equals to 0.9
gse
b
gra
GLUE under random error with ut from N (1.05, 0.005)
grb
GLUE under random error with ut from N (1.0, 0.005)
grc
GLUE under random error with ut from N (1.1, 0.005)
GLUE under random error with ut from N (0.95, 0.005)
grd
GLUE under random error with ut from N (0.9, 0.005)
gre
Bayesian under systematic error with ut equals to 1.1
bsa
Bayesian under systematic error with ut equals to 1.05
bsb
Bayesian under systematic error with ut equals to 1.0
bsc
Bayesian under systematic error with ut equals to 0.95
bsd
Bayesian under systematic error with ut equals to 0.9
Bayesian under random error with ut from N (1.1, 0.005)
bse
bra
ut is multiplier parameter
Bayesian under random error with ut from N (1.05, 0.005)
brb
N (1.1, 0.005) means the
normal distribution with
variance equals to 0.005 and
mean equals to 1.1
Bayesian under random error with ut from N (1.0, 0.005)
brc
Bayesian under random error with ut from N (0.95, 0.005)
brd
Bayesian under random error with ut from N (0.9, 0.005)
bre
Fig. 2 The box-plots of
posterior samples for parameter
a1 estimated by GLUE and
Bayesian methods under
systematic errors and random
errors. The subscripts S and R in
subtitle represent the systematic
error and random error,
respectively; the error scenatios
of 1, 2, 3, 4 and 5 means that the
multiplier parameter is 1.1,
1.05, 1.0, 0.95 and 0.9 for
systematic error and is normally
distributed with variance value
is 0.005 and mean values are
1.1, 1.05, 1.0,0.95 and 0.9 for
random error, respectively
GLUES
GLUER
0.8
0.8
Value
b
Value
a
0.6
0.4
0.4
1
2
3
4
5
1
Error scenarios
2
MHS
4
5
MHR
0.8
Value
Value
3
Error scenarios
0.8
0.6
0.4
0.6
0.4
1
2
3
4
Error scenarios
monthly hydrological data were between 01/1976 and
12/1983, which have undergone serious and strict quality
control measures in previous studies (Wang 2005; Li
et al. 2010). The models were calibrated against observed
discharges at the watershed outlet of Xiahui station during
123
0.6
5
1
2
3
4
5
Error scenarios
the same period. The mean annual precipitation is
494 mm, of which 80 % occurs in the rainy season from
June to September. The mean runoff coefficient of the
catchment is 0.19. There is rarely little precipitation in
the winter.
Stoch Environ Res Risk Assess (2014) 28:491–504
GLUER
0.8
0.8
0.6
0.6
Value
Value
GLUES
0.4
0.2
0
0.2
1
2
3
4
Error scenarios
0
5
1
2
3
4
Error scenarios
0.8
0.6
0.6
Value
0.8
0.4
0.4
0.2
0
0
1
2
3
4
Error scenarios
5
1
2
3
4
Error scenarios
GLUE S
0.4
Value
Value
5
GLUE R
0.4
0.2
0.2
0
0
1
2
3
4
5
1
Error scenarios
2
3
4
5
Error scenarios
MHS
MHR
0.4
Value
0.4
Value
5
MHR
0.2
Fig. 4 The box-plots of
posterior samples for parameter
a3 estimated by GLUE and
Bayesian methods under
systematic errors and random
errors. The subscripts S and R in
subtitle represent the systematic
error and random error,
respectively; the error scenarios
of 1, 2, 3, 4 and 5 means that the
multiplier parameter is 1.1,
1.05, 1.0, 0.95 and 0.9 for
systematic error and is normal
distributed with variance value
is 0.005 and mean values are
1.1, 1.05, 1.0,0.95 and 0.9 for
random error, respectively
0.4
MHS
Value
Fig. 3 The box-plots of
posterior samples for parameter
a2 estimated by GLUE and
Bayesian methods under
systematic errors and random
errors. The subscripts S and R in
subtitle represent the systematic
error and random error,
respectively; the error scenarios
of 1, 2, 3, 4 and 5 means that the
multiplier parameter is 1.1,
1.05, 1.0, 0.95 and 0.9 for
systematic error and is normal
distributed with variance value
is 0.005 and mean values are
1.1, 1.05, 1.0,0.95 and 0.9 for
random error, respectively
497
0.2
0.2
0
0
1
2
3
4
Error scenarios
3.2 WASMOD
A simple conceptual water balance model, monthly WASMOD was used (Xu 2002; Jin et al. 2010). The input data to
5
1
2
3
4
5
Error scenarios
the model include precipitation, potential evapotranspiration,
while the output data include fast flow, slow flow, actual
evapotranspiration, and soil moisture. The main equations are
shown in Table 2 (Xu 2002; Li et al. 2010).
123
498
Stoch Environ Res Risk Assess (2014) 28:491–504
4 Results
The error of observing precipitation come from many
sources, i.e. wind effect and evaporation (Wolff et al. 2013).
According to the technical standard for observations of
precipitation the systematic error of observing precipitation
is around 4–15 % in general (Ren et al. 2003), while the
random error is very hard to quantify. Ajami et al. (2007)
have studied the input uncertainty by adding a random
multiplier which is under normal distribution with mean
equal to [0.9,1.1] and variance equal to [1e–5, 1e–3].
According to pervious knowledge, in this study the Perturbing Error method was used and the multiplier parameters under system input errors are fixed to be 1.0, 0.9, 0.95,
1.05 and 1.1, while the multiplier parameters under random
input errors are normal distribution with variance equals to
0.005 and mean equals to 1.0, 0.9, 0.95, 1.05 and 1.1. GLUE
and Bayesian method are used to estimate the parameter and
model uncertainty under these system errors and random
errors. So there are 20 scenarios used in the study for
uncertainty estimates, which are shown in Table 3.
4.1 Parameter estimate by GLUE and Bayesian method
under precipitation errors
The box-plots of posterior distributions of parameters a1,
a2 and a3 that estimated by GLUE and Bayesian methods
are shown in Figs. 2, 3 and 4, respectively. It can been seen
from the figures that (1) the posterior distributions estimated by GLUE under input system errors and random
errors are nearly the same, while by Bayesian method, the
interval of posterior distributions of parameters under
system errors are slightly narrower than those under random error; and (2) the posterior distributions derived by the
Table 4 The character of parameters estimated by GLUE and Bayesian method for WASMOD under precipitation errors with constant
multiplier parameter
Method
Parameter
Multiplier parameter
Min
Max
Mean
Variance
Skewness
GLUE
a1
1.1
0.472
0.857
0.609
6.42E-03
0.512
1.05
0.491
0.853
0.622
6.17E-03
0.551
1.0
0.506
0.855
0.633
5.65E-03
0.530
0.95
0.522
0.853
0.644
4.94E-03
0.466
0.9
0.543
0.854
0.661
4.44E-03
0.451
1.1
0.038
0.737
0.362
2.55E-02
0.108
1.05
0.048
0.754
0.381
2.58E-02
0.112
1.0
0.057
0.766
0.397
2.58E-02
0.070
0.95
0.073
0.786
0.415
2.59E-02
0.061
0.9
1.1
0.099
0.018
0.793
0.262
0.426
0.131
2.56E-02
2.76E-03
0.134
0.200
1.05
0.024
0.295
0.149
3.56E-03
0.175
1.0
0.029
0.337
0.173
4.63E-03
0.149
0.95
0.043
0.381
0.203
5.82E-03
0.145
0.9
0.049
0.434
0.231
6.92E-03
0.122
1.1
0.697
0.807
0.749
3.32E-04
0.258
1.05
0.683
0.805
0.755
3.61E-04
-0.288
1.0
0.704
0.814
0.76
3.33E-04
0.012
0.95
0.719
0.825
0.764
2.47E-04
0.217
0.9
0.72
0.828
0.774
3.07E-04
0.192
1.1
0.083
0.23
0.15
4.87E-04
0.178
1.05
0.093
0.281
0.157
6.47E-04
0.616
1.0
0.099
0.264
0.166
6.42E-04
0.433
0.95
0.104
0.251
0.174
5.35E-04
0.049
0.9
1.1
0.101
0.026
0.242
0.105
0.176
0.061
5.61E-04
1.37E-04
-0.186
0.292
1.05
0.032
0.127
0.071
1.99E-04
0.582
1.0
0.037
0.163
0.084
2.97E-04
0.416
0.95
0.025
0.202
0.101
3.68E-04
0.374
0.9
0.048
0.204
0.118
5.62E-04
0.283
a2
a3
Bayesian
a1
a2
a3
123
Stoch Environ Res Risk Assess (2014) 28:491–504
499
Bayesian method are slightly narrower and sharper than
those obtained by the GLUE method, which means less
uncertainty in parameters. This is confirmed by the variances of parameters samples given in Tables 4 and 5,
which show the posterior summary statistics for all
parameters estimated by GLUE and Bayesian methods
with precipitation errors, including system errors and random errors, respectively. The tables show that (1) the
variances obtained by the Bayesian method are the smaller
compared with those obtained by the GLUE method; (2)
the skewness coefficients of the posterior distribution of
parameters are strongly impacted by the precipitation
errors in Bayesian method while which is not seen in
GLUE; (3) the differences of the variance and skewness of
posterior samples between system errors and random errors
are quite small in GLUE method; (4) the medium value of
posterior distribution for parameters a1, a2 and a3
estimated by GLUE and Bayesian methods are decreasing
with the precipitation errors increasing. It might because
that the evapotranspiration, fast flow and slow flow will
increase with the precipitation increasing, which results in
all parameters decrease in order to maintain original water
balance; and (5) in Bayesian method, the random error with
the N(1.0, 0.005) distributed multiplier parameter will
slightly increase the variances of parameters.
4.2 Ninety-five percentage confidence interval
of discharge by GLUE and Bayesian methods
under precipitation errors
Figure 5 shows the 95 % confidence intervals of monthly
discharge (1976/1–1983/12) due to parameter and model
uncertainty and observed discharge estimated by GLUE
and Bayesian method. It is seen that the 95CI estimated
Table 5 The character of parameters estimated by GLUE and Bayesian method for WASMOD under precipitation errors with normally
distributed multiplier parameter
Method
Parameter
Multiplier parameter
Min
Max
Mean
Variance
Skewness
GLUE
a1
N (1.1, 0.005)
0.472
0.836
0.608
6.38E-03
0.512
N (1.05, 0.005)
0.491
0.850
0.622
6.09E-03
0.546
N (1.0, 0.005)
0.508
0.855
0.633
5.59E-03
0.528
N (0.95, 0.005)
0.522
0.842
0.643
4.92E-03
0.471
N (0.9, 0.005)
0.543
0.846
0.660
4.44E-03
0.442
N (1.1, 0.005)
0.038
0.737
0.362
2.53E-02
0.113
N (1.05, 0.005)
0.048
0.754
0.380
2.55E-02
0.110
N (1.0, 0.005)
0.068
0.766
0.396
2.54E-02
0.081
N (0.95, 0.005)
0.073
0.786
0.416
2.61E-02
0.064
N (0.9, 0.005)
N (1.1, 0.005)
0.099
0.023
0.793
0.263
0.427
0.133
2.55E-02
2.85E-03
0.122
0.200
N (1.05, 0.005)
0.028
0.295
0.151
3.62E-03
0.169
N (1.0, 0.005)
0.029
0.338
0.175
4.73E-03
0.153
N (0.95, 0.005)
0.043
0.386
0.205
5.91E-03
0.139
N (0.9, 0.005)
0.055
0.440
0.235
7.18E-03
0.143
N (1.1, 0.005)
0.684
0.798
0.747
3.00E-04
N (1.05, 0.005)
0.697
0.805
0.756
3.15E-04
N (1.0, 0.005)
0.710
0.821
0.761
3.59E-04
0.183
N (0.95, 0.005)
0.718
0.824
0.765
2.73E-04
0.203
N (0.9, 0.005)
0.73
0.83
0.775
3.01E-04
0.291
N (1.1, 0.005)
0.094
0.228
0.151
4.73E-04
0.154
N (1.05, 0.005)
0.09
0.248
0.157
5.72E-04
0.484
N (1.0, 0.005)
0.092
0.251
0.165
6.48E-04
0.167
N (0.95, 0.005)
0.103
0.25
0.174
6.32E-04
0.184
N (0.9, 0.005)
N (1.1, 0.005)
0.11
0.025
0.243
0.12
0.176
0.062
7.24E-04
1.34E-04
0.01
0.385
N (1.05, 0.005)
0.029
0.136
0.071
1.99E-04
0.522
N (1.0, 0.005)
0.031
0.162
0.084
3.06E-04
0.293
N (0.95, 0.005)
0.038
0.186
0.101
3.87E-04
0.386
N (0.9, 0.005)
0.049
0.203
0.118
5.64E-04
0.247
a2
a3
Bayesian
a1
a2
a3
0.069
-0.17
123
500
Stoch Environ Res Risk Assess (2014) 28:491–504
80
(a)
95% confidence interval
observed
Simulated
Q(mm)
60
40
20
0
1976-01
1977-01
1978-01
1979-01
1980-01
1981-01
1982-01
1983-01
Month (Y-M)
80
95% confidence interval
observed
Simulated
(b)
Q(mm)
60
40
20
0
1976-01
1977-01
1978-01
1979-01
1980-01
1981-01
1982-01
1983-01
Month (Y-M)
Fig. 5 The 95 % confidence intervals of monthly discharge (1976/1–1983/12) due to parameter uncertainty and model uncertainty (grey bands)
and observed discharge (red spots). a GLUE (ARIL is 1.445 and P-95CI is 69.8 %); b Bayesian (ARIL is 2.487 and P-95CI is 96.9 %)
1
(b)
gsa
0.6
gsd
gse
0.4
1
gra
gsb
gsc
0.8
PCI of all flow
(a)
PCI of all flow
Fig. 6 Proportion of
observation inside confidence
intervals (PCI) for WASMOD
of monthly runoffs in Chao
River basin by a GLUE with
systematic error, b GLUE with
random error, c Bayesian with
systematic error and d Bayesian
with random error. Legend
details can be seen in Table 3
0.2
0.8
grb
0.6
grc
grd
gre
0.4
0.2
0
0
0
0.2
0.4
0.6
0.8
1
0
Confidence Interval
PCI of all flow
(d)
1
bsa
0.6
bse
0.2
1
brb
brc
0.6
brd
bre
0.4
0.2
0
0
0
0.2
0.4
0.6
0.8
Confidence Interval
123
0.8
1
0.8
bsd
0.4
0.6
bra
bsb
bsc
0.8
0.4
Confidence Interval
PCI of all flow
(c)
0.2
1
0
0.2
0.4
0.6
0.8
Confidence Interval
1
Stoch Environ Res Risk Assess (2014) 28:491–504
501
from GLUE is slightly narrower than that from Bayesian
method, which can be seen more clearly in the comparison
of PCI in Figure 6, which shows the PCI for WASMOD of
monthly runoffs in Chao river basin by GLUE and
Bayesian method with systematic or random error. It can
be seen that the uncertainty interval derived from Bayesian
method is more reliable than that from GLUE since the
points in Fig. 6c and d are more closer to the 45 diagonal
lines. Figure 7 shows the ARIL for WASMOD of monthly
runoffs in Chao river basin by GLUE and Bayesian
methods with precipitation errors. It can be seen the same
results as Fig. 5 that the uncertainty interval derived by
Bayesian is slightly wider than that from GLUE. Besides,
the impact of precipitation errors are more obvious in
Bayesian method than GLUE method since the lines of
ARIL are more various in Fig. 7c and d than Fig. 7a and b.
4.3 Sensitivity analysis of uncertainty estimates
by GLUE and Bayesian methods in response
to precipitation errors
The sensitivity analysis will be applied by a quantitatively
comparison of 95CI of WASMOD monthly runoff for three
3
(b)
ARIL of all flow
2.5
gsb
2
gsc
1.5
gsd
1
3
2.5
gsa
ARIL of all flow
(a)
gse
0.5
gra
grb
grc
grd
gre
2
1.5
1
0.5
0
0
0
0.2
0.4
0.6
0.8
1
0
Confidence Interval
(d)
3
2.5
bsa
bsb
2
bsc
1.5
bsd
1
0.2
0.4
0.6
0.8
1
Confidence Interval
ARIL of all flow
(c)
ARIL of all flow
Fig. 7 Average relative interval
length (ARIL) for WASMOD of
monthly runoffs in Chao River
basin by a GLUE with
systematic error, b GLUE with
random error, c Bayesian with
systematic error and d Bayesian
with random error. Legend
details can be seen in Table 3
levels of flows, i.e. low flow, medium flow, high flow,
which are accounting for 10, 70 and 20 % of total runoff,
respectively and all flows, obtained by GLUE and Bayesian
methods under different precipitation errors. There are four
indices, i.e. ARIL, PCI, PUCI and CRPS, used for evaluation of 95 % confident intervals of runoffs estimated by
GLUE and Bayesian methods. Figure 8 shows the variation
of four indices for uncertainty results of four level flows
according to 20 scenarios which are a combination of
different uncertainty estimate methods with different precipitation errors. It can be seen that (1) for the ARIL,
Bayesian method is obviously more sensitive than GLUE
in response to different precipitation errors, especially for
low flows, which means that there is more risk for Bayesian
method in uncertainty estimate of sharpness, especially for
the low flows; (2) according to PCI, the 95CI of discharge
from GLUE is much more sensitive than those from
Bayesian method, especially for low and medium flows,
which indicates that there is more risk in reliability for
uncertainty estimate by GLUE method than Bayesian
method; (3) the PCI is not sensitive in high flows according
to the precipitation errors by both GLUE and Bayesian
methods, which means that the risk of reliability in
bse
0.5
3
2.5
bra
2
brb
brc
1.5
brd
bre
1
0.5
0
0
0
0.2
0.4
0.6
0.8
Confidence Interval
1
0
0.2
0.4
0.6
0.8
1
Confidence Interval
123
gsa
gsb
gsc
gsd
gse
gra
grb
grc
grd
gre
bsa
bsb
bsc
bsd
bse
bra
brb
brc
brd
bre
PUCI
gsa
gsb
gsc
gsd
gse
gra
grb
grc
grd
gre
bsa
bsb
bsc
bsd
bse
bra
brb
brc
brd
bre
PCI
gsa
gsb
gsc
gsd
gse
gra
grb
grc
grd
gre
bsa
bsb
bsc
bsd
bse
bra
brb
brc
brd
bre
ARIL
(a)
(b)
(c)
(d)
6
5
Low
High
3.0
123
Medium
All
4
3
2
1
0
Scenarios
1.2
1.0
0.8
0.6
0.4
Low
Medium
0.2
High
0.0
All
Scenarios
1.8
1.6
1.4
1.2
1.0
0.8
0.6
0.4
0.2
0.0
Low
Medium
High
All
Scenarios
6.0
5.0
4.0
Low
Medium
High
2.0
All
1.0
0.0
Scenarios
Fig. 8 The uncertainty evaluation of 95 % confident intervals of
monthly discharge in Chao River basin in 20 scenarios by four
indices, including a ARIL, b PCI, c PUCI and d CRPS. The definition
of scenarios can be seen in Table 3
uncertainty estimates for high flows is not as large as the
other flows by both GLUE and Bayesian method under the
impact of precipitation errors; (4) the PUCI shows that
both GLUE and Bayesian method are more sensitive in
uncertainty estimate of high flow than other flows
according to precipitation errors; and (5) the CRPS of low,
medium flows are quite stable under impact of
bsb
0.55
1.44
bsa
PUCI
CRPS
2.16
0.95
0.46
1.16
ARIL
PCI
PUCI
CRPS
1.17
0.35
0.98
2.80
0.56
0.79
1.19
0.40
0.96
2.49
bsc
1.45
0.56
0.75
1.42
gsc
1.21
0.40
0.96
2.48
bsd
1.47
0.56
0.70
1.35
gsd
1.19
0.38
0.97
2.59
bse
1.49
0.57
0.68
1.28
gse
1.16
0.48
0.95
2.09
bra
1.43
0.55
0.79
1.53
gra
1.15
0.42
0.96
2.38
brb
1.43
0.57
0.79
1.49
grb
1.18
0.39
0.97
2.51
brc
1.44
0.57
0.76
1.41
grc
1.22
0.45
0.94
2.18
brd
1.47
0.55
0.69
1.34
grd
1.22
0.45
0.95
2.24
bre
1.48
0.57
0.68
1.27
gre
ARIL the average relative interval length, PCI the percent of observations bracketed by the 95 % confident interval, PUCI the percentage of observations bracketed by the unit confidence
interval, CRPS the continuous rank probability score
Bayesian
1.44
0.80
PCI
1.50
1.55
ARIL
gsb
GLUE
gsa
Scenarios
Indices
Method
Table 6 The evaluation results of 95 % confident interval of discharge for WASMOD in Chao River basin by GLUE and Bayesian method under different precipitation errors (20 scenarios)
gsa
gsb
gsc
gsd
gse
gra
grb
grc
grd
gre
bsa
bsb
bsc
bsd
bse
bra
brb
brc
brd
bre
CRPS
502
Stoch Environ Res Risk Assess (2014) 28:491–504
Stoch Environ Res Risk Assess (2014) 28:491–504
precipitation errors and it only changes in uncertainty
estimate of high flow in Bayesian method by the impact of
precipitation errors. The quantitative results of ARIL, PCI,
PUCI and CPRS estimated by GLUE and Bayesian method
due to different model precipitation errors are also summarized in Table 6.
503
Acknowledgments This study was supported by the Research
Council of Norway, Research Project-JOINTINDNOR 203867,
Department of Science and Technology, Govt. of India, Project
190159/V10 (SoCoCA) and Project NORINDIA 806793. The second
author was also supported by Program of Introducing Talents of
Discipline to Universities—the 111 Project of Hohai University.
References
5 Conclusion
In this study, a comprehensive evaluation and quantification of the effect of precipitation errors, i.e. multiplier
parameter is constant or multiplier parameter is a random
number with normal distribution, in GLUE on the parameter and model uncertainty was performed, and the results
were compared with a formal Bayesian method using the
MH algorithm for a conceptual hydrological model. The
95 % confidence intervals of monthly discharge for low
flow, medium flow and high flow were selected for comparison by four indices, i.e. ARIL, PCI, PUCI and CRPS.
The following conclusions are drawn from this study.
•
•
•
•
•
•
The posterior distributions of parameters derived by the
Bayesian method are slightly narrower and sharper than
those obtained by the GLUE method for both with and
without precipitation errors, though the differences are
quite small.
Bayesian method performs more sensitive in uncertainty estimates of discharge than GLUE according to
the impact of precipitation errors.
In terms of the PCI, the impact of precipitation errors
on the 95CI of WASMOD monthly runoffs in Chao
river basin obtained by Bayesian is very small,
especially for high flows, which indicates that the
Bayesian method is a more reliable approach for
application especially for regions which might be with
large uncertainty, e.g. ungauged basins.
In terms of the ARIL, the Bayesian method is more
sensitivity than GLUE method according to precipitation errors, especially for low flows.
The values of PUCI show that the GLUE and Bayesian
methods are more sensitive in uncertainty estimate of high
flow than the other flows by the impact of precipitation errors.
Under the impact of precipitation errors, the results of
CRPS for low and medium flows are quite stable from
both GLUE and Bayesian method while it is sensitive
for high flow by Bayesian method.
It is worth noting that the index of CRPS is very sensitive
for high flows, which is strongly recommended to use in
uncertainty evaluation for extreme events, i.e. flood. It
should be noted that although the study yielded interesting
findings, the generality of the results requires testing of the
method in other areas using other models in future research.
Ajami NK, Duan Q, Sorooshian S (2007) An integrated hydrologic
Bayesian multimodel combination framework: confronting
input, parameter, and model structural uncertainty in hydrologic
prediction. Water Resour Res 43(1):W01403
Alvisi S, Creaco E, Franchini M (2012) Crisp discharge forecasts and
grey uncertainty bands using data-driven models. Hydrol Res
43(5):589–602
Bernardo JM, Smith AF (1994) Bayesian theory. Wiley, Chichester
Beven KJ, Binley A (1992) Future of distributed models: model
calibration and uncertainty prediction. Hydrol Process 6(3):
279–298
Beven KJ, Freer J (2001) Equifinality, data assimilation, and
uncertainty estimation in mechanistic modelling of complex
environmental systems using the GLUE methodology. J Hydrol
249(1–4):11–29
Blasone RS, Madsen H, Rosbjerg D (2008a) Uncertainty assessment
of integrated distributed hydrological models using GLUE with
Markov chain Monte Carlo sampling. J Hydrol 353(1–2):18–32
Blasone RS et al (2008b) Generalized likelihood uncertainty estimation (GLUE) using adaptive Markov Chain Monte Carlo
sampling. Adv Water Resour 31(4):630–648
Carlos MJ, Bera AK (1980) Efficient tests for normality, homoscedasticity and serial independence of regression residuals. Econ
Lett 6(3):255–259
Chib S, Greenberg E (1995) Understanding the Metropolis–Hastings
algorithm. Am Stat 49:327–335
Duan Q, Ajami NK, Gao X, Sorooshian S (2007) Multi-model
ensemble hydrologic prediction using Bayesian model averaging. Adv Water Resour 30(5):1371–1386
Engeland K, Gottschalk L (2002) Bayesian estimation of parameters
in a regional hydrological model. Hydrol Earth Syst Sci 6(5):
883–898
Engeland K, Xu CY, Gottschalk L (2005) Assessing uncertainties in a
conceptual water balance model using Bayesian methodology.
Hydrol Sci J 50(1):45–63
Engeland K, Renard B, Steinsland I, Kolberg S (2010) Evaluation of
statistical models for forecast errors from the HBV model.
J Hydrol 384(1–2):142–155
Freer J, Beven K, Ambroise B (1996) Bayesian estimation of
uncertainty in runoff prediction and the value of data: an
application of the GLUE approach. Water Resour Res
32(7):2161–2174
Gelman A, Rubin DB (1992) Inference from iterative simulation
using multiple sequences. Stat Sci 7:457–472
Hastings WK (1970) Monte Carlo sampling methods using Markov
chains and their applications. Biometrika 57(1):97–109
Hersbach H (2000) Decomposition of the continuous ranked probability score for ensemble prediction systems. Weather Forecast
15(5):559–570
Jin X, Xu C-Y, Zhang Q, Singh VP (2010) Parameter and modeling
uncertainty simulated by GLUE and a formal Bayesian method
for a conceptual hydrological model. J Hydrol 383(3–4):147–155
Kavetski D, Franks S, Kuczera G (2002) Confronting input uncertainty in environmental modelling. In: Gupta HV, Sorooshian S,
Rousseau AN, Turcotte R (eds) Calibration of watershed models.
123
504
AGU Water Science and Applications Series. AGU, Washington, pp 49–68
Kavetski D, Kuczera G, Franks SW (2006a) Bayesian analysis of
input uncertainty in hydrological modeling: 1. Theory. Water
Resour Res 42(3):W03407
Kavetski D, Kuczera G, Franks SW (2006b) Bayesian analysis of
input uncertainty in hydrological modeling: 2.Application.
Water Resour Res 42(3):W03407
Kizza M, Guerrero J-L, Rodhe A, Xu C-Y, Ntale HK (2013)
Modelling catchment inflows into Lake Victoria: regionalisation
of the parameters of a conceptual water balance model. Hydrol
Res. doi:10.2166/nh.2012.152
Krzysztofowicz R (1999) Bayesian theory of probabilistic forecasting
via deterministic hydrologic model. Water Resour Res 35(9):
2739–2750
Kuczera G, Parent E (1998) Monte Carlo assessment of parameter
uncertainty in conceptual catchment models: the Metropolis
algorithm. J Hydrol 211(1–4):69–85
Li L, Xia J, Xu C-Y, Chu J, Wang R (2009) Analyse the sources of
equifinality in hydrological model using GLUE methodology.
Hydroinformatics in Hydrology, Hydrogeology and Water
Resources. In: Proceedings of Symposium JS.4 at the Joint
IAHS & IAH Convention, Hyderabad, India, September 2009.
IAHS Publications, vol. 331, pp. 130–138
Li L, Xia J, Xu C-Y, Singh VP (2010) Evaluation of the subjective
factors of the GLUE method and comparison with the formal
Bayesian method in uncertainty assessment of hydrological
models. J Hydrol 390(3–4):210–221
Li L, Xu C-Y, Xia J, Engeland K, Reggiani P (2011) Uncertainty
estimates by Bayesian method with likelihood of AR (1) plus
Normal model and AR (1) plus multi-normal model in different
time-scales hydrological models. J Hydrol 406(1–2):54–65
Li L, Xu C-Y, Engeland K (2013a) Development and comparison in
uncertainty assessment based Bayesian modularization method
in hydrological modeling. J Hydrol 486:384–394
Li L, Ngongondo CS, Xu C-Y, Gong L (2013b) Comparison of the
global TRMM and WFD precipitation datasets in driving a largescale hydrological model in Southern Africa. Hydrol Res. doi:10.
2166/nh.2012.175
Lilliefors HW (1967) On the Kolmogorov–Smirnov test for normality
with mean and variance unknown. J Am Stat Assoc 62(318):
399–402
Liu Z, Martina MLV, Todini E (2005) Flood forecasting using a fully
distributed model: application of the TOPKAPI model to the
Upper Xixian Catchment. Hydrol Earth Syst Sci 9(4):347–364
Marshall L, Nott D, Sharma A (2007) Towards dynamic catchment
modelling: a Bayesian hierarchical mixtures of experts framework. Hydrol Process 21(7):847–861
Montanari A (2007) What do we mean by ‘uncertainty’? The need for
a consistent wording about uncertainty assessment in hydrology.
Hydrol Process 21:841–845
Refsgaard JC, Storm B (1996) Construction, calibration and validation of hydrological models. Distrib Hydrol Model 22:41–54
Refsgaard JC, Van der Sluijs JP, Brown J, Van der Keur P (2006) A
framework for dealing with uncertainty due to model structure
error. Adv Water Resour 29(11):1586–1597
Ren ZH, Wang GL, Zou FL, Zhang HZ (2003) The research of
precipitation of precipitation measurement errors in China. Acta
Meteorol Sin 61(5):621–627
Renard B, Kavetski D, Thyer M, Kuczera G, Franks S (2009)
Understanding predictive uncertainty in hydrologic modeling:
123
Stoch Environ Res Risk Assess (2014) 28:491–504
the challenge of identifying input and structural errors. Water
Resour Res 46:W05521
Shen Z, Chen L, Chen T (2013) The influence of parameter
distribution uncertainty on hydrological and sediment modeling:
a case study of SWAT model applied to the Daning watershed of
the Three Gorges Reservoir Region, China. Stoch Environ Res
Risk Assess 27(1):235–251
Smith P, Beven KJ, Tawn JA (2008) Informal likelihood measures in
model assessment: theoretic development and investigation. Adv
Water Resour 31(8):1087–1100
Thiemann M, Trosset M, Gupta H, Sorooshian S (2001) Bayesian
recursive parameter estimation for hydrologic models. Water
Resour Res 37(10):2521–2535
Tsai Frank T-C (2010) Bayesian model averaging assessment on
groundwater management under model structure uncertainty.
Stoch Environ Res Risk Assess 24(6):845–861
Vogel RM, Stedinger JR, Batchelder R, Lee SU (2008) Appraisal of
the generalized likelihood uncertainty estimation (GLUE)
Method. American Society of Civil Engineers, Reston
Vrugt JA, Gupta HV, Bouten W, Sorooshian S (2003a) A shuffled
complex evolution metropolis algorithm for optimization and
uncertainty assessment of hydrologic model parameters. Water
Resour Res 39(8):1201
Vrugt JA, Gupta HV, Bouten W, Sorooshian S (2003b) A shuffled
complex evolution metropolis algorithm for optimization and
uncertainty assessment of hydrologic model parameters. Water
Resour Res 39(8):1201
Vrugt JA, ter Braak CJF, Gupta HV, Robinson BA (2009) Equifinality
of formal (DREAM) and informal (GLUE) bayesian approaches
in hydrologic modeling? Stoch Environ Res Risk Assess 23(7):
1011–1026
Wang G (2005) Theory and method of distributed time-variant gain
model. Institute of Geographic Sciences and Natural Resources
Research, CAS. Beijing, Ph.D (in Chinese)
Wang G, Xia J, Chen J (2009) Quantification of effects of climate
variations and human activities on runoff by a monthly water
balance model: a case study of the Chaobai River basin in
northern China. Water Resour Res 45(7):W00A11
Wolff M, Isaksen K, Brækkan R, Alfnes E, Petersen-Øverleir A, Ruud
E (2013) Measurements of wind-induced loss of solid precipitation: description of a Norwegian field study. Hydrol Res
44(1):35–43. doi:10.2166/nh.2012.166
Xiong L, Wan M, Wei X, O’Connor KM (2009) Indices for assessing
the prediction bounds of hydrological models and application by
generalized likelihood uncertainty estimation. Hydrol Sci J
54(5):852–871
Xu C-Y (2002) WASMOD—the water and snow balance MODelling
system. In: Singh VP, Frevert DK (eds) Mathematical models of
small watershed hydrology and applications. Water Resources
Publications, LLC, Chelsea, pp 555–590
Yang J, Reichert P, Abbaspour KC (2007a) Bayesian uncertainty
analysis in distributed hydrologic modeling: a case study in the
Thur River basin (Switzerland). Water Resour Res 43(10):
W10401
Yang J, Reichert P, Abbaspour KC, Yang H (2007b) Hydrological
modelling of the Chaohe Basin in China: statistical model
formulation and Bayesian inference. J Hydrol 340(3–4):167–182
Yang J, Reichert P, Abbaspour KC, Xia J, Yang H (2008) Comparing
uncertainty analysis techniques for a SWAT application to the
Chaohe Basin in China. J Hydrol 358(1–2):1–23