Analyzing possible causes of bias of
hydrological models with stochastic,
time-dependent parameters
Peter Reichert
Eawag Dübendorf, ETH Zürich, and SAMSI
Eawag: Swiss Federal Institute of Aquatic Science and Technology
Contents
Motivation
 Motivation
Approach
Implementation
Application
Discussion
 Approach
 Implementation
 Application
 Discussion
SAMSI Transition Workshop
May 14-16, 2007
Motivation
Motivation
Approach
Implementation
Application
Discussion
SAMSI Transition Workshop
May 14-16, 2007
Motivation
Motivation
Typical results of a hydrological model
0
200
Motivation
0
50
150
Discussion
850
900
950
1000
1050
time [days]
 Overall quality of fit demonstrates that the model
describes the most relevant mechanisms in the system
adequately.
SAMSI Transition Workshop
May 14-16, 2007
 However, remaining systematic deviations of model
results from data make uncertainty analysis difficult.
qrain [mm/d]
100
100
Application
Q [m3/s]
Implementation
50
150
Approach
Motivation
Residuals of Box-Cox transformed results
6
Motivation
2
-6
-4
Discussion
0
Application
-2
Implementation
nondim. resid.
4
Approach
500
1000
1500
time
Problems
 Heteroscedasticity of residuals (even after Box-Cox
transformation).
SAMSI Transition Workshop
May 14-16, 2007
 Autocorrelation of residuals.
Motivation
Motivation
Approach
These problems are typical for any kind of deterministic
dynamic environmental modelling.
Implementation
Application
Discussion
SAMSI Transition Workshop
May 14-16, 2007
They make uncertainty analysis difficult as this can
only be done if the statistical model assumptions are
not seriously violated.
Motivation
Motivation
Suggested solution (Kennedy and O‘Hagan, etc.):
Approach
Extend the model by a discrepancy or bias term.
Replace:
Implementation
YM (x, θ)  y D (x, θ)  Ε y (θ)
Application
Discussion
by:
YM (x, θ)  y D (x, θ)  B(x, θ)  Ε y (θ)
where yD = deterministic model, x = model inputs, q = model
parameters, Ey = observation error, B = bias or model
discrepancy, YM = random variable representing model
results.
SAMSI Transition Workshop
May 14-16, 2007
The bias term is usually formulated as a non-parametric
statistical description of the model deficits (often as a
Gaussian Stoachastic Process).
Motivation
Motivation
Advantage of this approach:
Approach
 The statistical description of the model discrepancy
allows for improved uncertainty analysis.
Implementation
Application
Discussion
Disadvantage:
 Lack of understanding of the cause of the discrepancy
makes it difficult to extrapolate.
We are interested in a technique that supports
identification of the causes of model discrepancies.
This can lead to an improved model formulation that
reduces the discrepancies.
SAMSI Transition Workshop
May 14-16, 2007
This cannot be done by a purely statistical approach,
but statistics can be supportive.
Motivation
Motivation
Approach
Implementation
Causes of deficits of deterministic models:
 Errors in parameter values.
Application
 Errors in model structure.
Discussion
 Errors in model input.
 Inadequateness of a deterministic description of
systems that contain intrinsic non-deterministic
behaviour due to
 influence factors not considered in the model,
 model simplifications (e.g. aggregation, adaptation,
etc.),
SAMSI Transition Workshop
May 14-16, 2007
 chaotic behaviour.
Motivation
Motivation
Approach
Because of these deficits we cannot expect a
deterministic model to describe nature appropriately.
Implementation
Application
Discussion
Pathway for improving models:
1. Reduce errors in deterministic model structure to
improve average behaviour.
2. Add adequate stochasticity to the model structure to
account for random influences.
This requires the combination of statistical analyses
with scientific judgment.
SAMSI Transition Workshop
May 14-16, 2007
This talk is about support of this process by statistical
techniques.
Approach
Motivation
Approach
Implementation
Application
Discussion
SAMSI Transition Workshop
May 14-16, 2007
Approach
Approach
Motivation
Approach
Implementation
Application
Discussion
Questions:
1. How to make a deterministic, continuous-time model
stochastic?
2. How to distinguish between deterministic and
stochastic model deficits?
 Replacement of differential equations (representing
conservation laws) by stochastic differential equations can
violate conservation laws and does not address the cause
of stochasticity directly.
SAMSI Transition Workshop
May 14-16, 2007
 It seems to be conceptually more satisfying to replace
model parameters (such as rate coefficients, etc.) by
stochastic processes, as stochastic external influence
factors usually affect rates and fluxes rather than states
directly.
Approach
Motivation
Approach
Implementation
Application
Discussion
SAMSI Transition Workshop
May 14-16, 2007
Approach
Motivation
Note that the basic idea of this approach is very old.
Approach
Implementation
Application
The original formulation was, however, limited to
discrete-time systems with slowly varying driving
forces (e.g. Beck 1987).
Discussion
Our suggestion is to
 extend this original approach to continuous-time
systems;
 allow for rapidly varying external forces;
 embed the procedure into statistical „biasmodelling“ techniques.
SAMSI Transition Workshop
May 14-16, 2007
This requires more complicated numerical techniques
and more extensive analyses of the results.
Implementation
Motivation
Approach
Implementation
Application
Discussion
SAMSI Transition Workshop
May 14-16, 2007
Implementation
Model
Motivation
Approach
Deterministc model:
y D (x, θD )
Implementation
Application
Discussion
Consideration of observation error:
YM (x, θ M )  y D (x, θ D )  E y (θ O )
qD
x
yD
SAMSI Transition Workshop
May 14-16, 2007
 θD 
θ M   
 θO 
yM
qO
qM
x
yM
f M (y x, θM )
Model
Motivation
Approach
Implementation
Model with parameter i time-dependent:

YM(i) x, θM(i)   y M x, θM,-i ,q
Application
t
M,i
(θ )
i
P

θ M(i)
 θ M,-i 
  i 
 θP 
qPi
Discussion
qM,-i
x
t
qM,i
yM
SAMSI Transition Workshop
May 14-16, 2007



 

f M(i) y x, θ M(i)   f M y x, θ M , i ,q Mt ,i f P q Mt ,i θiP dq Mt ,i
Time Dependent Parameter
Motivation
Approach
The time dependent parameter is modelled by a
mean-reverting Ornstein Uhlenbeck process:


dq t ( )   q t ( )  q   W dWt ( )
Implementation
Application
Discussion
This has the advantage that we can use the analytical
solution:
2


q t q s ~ Nq  (q s   )e  (t s ) , W 1  e 2 (t  s )
2


or, after reparameterization:
t s
t s



2


t
s
s
2
q q ~ Nq  (q  q )e  ,  1  e   





 W2
 , 

2
1
SAMSI Transition Workshop
May 14-16, 2007
2
t
θ P  (qini
, q ,  , )




Inference
Motivation
Approach
Implementation
Application
Discussion
We combine the estimation of
 constant model parameters, θ M,-i , with
 state estimation of the time-dependent parameter(s),
q Mt ,i , and with
 the estimation of (constant) parameters of the
Ornstein-Uhlenbeck process(es) of the time
dependent parameter(s), θ iP .
SAMSI Transition Workshop
May 14-16, 2007
Inference
Motivation
Approach
Implementation
Gibbs sampling for the three different types of
parameters. Conditional distributions:

 

simulation model (expensive)

f θ M , i x, q Mt ,i , θiP , y  f θ M ,  i x, q Mt ,i , y  f (θ M , i ) f M ( i ) y x, θ M ,  i , q Mt ,i
Application
Discussion
Ornstein-Uhlenbeck process (cheap)

 


f θiP x, θ M , i ,q Mt ,i , y  f θiP q Mt ,i  f (θiP ) f P q Mt ,i θiP

Ornstein-Uhlenbeck process (cheap)

fq
t
M ,i


x, θ M , i , θ , y  f P q
i
P
t
M ,i
θ
i
P
 f y x, θ
M(i )
M , i
simulation model (expensive)
x
SAMSI Transition Workshop
May 14-16, 2007
Tomassini et al. 2007
,q
t
M ,i

qPi
qM,-i
yM
t
qM,i

Inference
Metropolis-Hastings sampling for each type of parameter:
Motivation
Approach
Implementation
Application
Discussion

f θ
 
, y   f θ


f θ M , i x, q Mt ,i , θiP , y  f θ M ,  i x, q Mt ,i , y  f (θ M , i ) f M ( i ) y x, θ M ,  i , q Mt ,i
i
P
x, θ M , i ,q Mt ,i
i
P


q Mt ,i  f (θiP ) f P q Mt ,i θiP

Multivariate normal jump distributions for the
parameters qM and qP. This requires one simulation to
be performed per suggested new value of qM.





f q Mt ,i x, θ M , i , θiP , y  f P q Mt ,i θiP f M ( i ) y x, θ M , i , q Mt ,i
SAMSI Transition Workshop
May 14-16, 2007

The discretized Ornstein-Uhlenbeck parameter, q Mt ,i , is
split into subintervals for which OU-process
realizations conditional on initial and end points are
sampled. This requires the number of subintervals
simulations per complete new time series of q Mt ,i.
Tomassini et al. 2007

Estimation of Hyperparameters
by Cross - Validation
Motivation
Approach
Implementation
Application
Discussion
SAMSI Transition Workshop
May 14-16, 2007
Due to identifiability problems we select the two
hyperparameters (,) by cross-validation:
( , ) : psl   log  f ( yi y i ,  , )   max ( , )
i
Tomassini et al. 2007
Estimation of Hyperparameters
by Cross - Validation
Motivation
Approach
Implementation
For a state-space model of the form
dξ
ξ (t , x, θ D ) :
 F (ξ, x, θ D )
dt
Application
Yi   (ti , x, q D )  Ey ,i (q O )
Discussion
E y  N (0, Σ)
we can estimate the pseudo-likelihood from the sample:
f ( yi y  i ,  ,  ) 

1
f ( y,  , ) d
f ( yi y  i ,  )
1

(2 )
1/ 2
SAMSI Transition Workshop
May 14-16, 2007
1
 ( Σ 1 )
1
1 1 / 2

ii
(
Σ
)
exp

ii
 2
n k


( Σ 1 )ij
(k )
(k )
 yi   i   1 y j   j
j  i ( Σ ) ii






2




Tomassini et al. 2007
Application
Motivation
Approach
Implementation
Application
Discussion
SAMSI Transition Workshop
May 14-16, 2007
Application
Hydrological Model
Motivation
Simple Hydrological Watershed Model (1):
Approach
Application
dhs
 (qrain  qrunoff )  qet  qlat  qgw
dt
Discussion
dhgw
Implementation
dt
 qgw  qbf  qdp
dhr
 qrunoff  qlat  qbf  qr
dt
SAMSI Transition Workshop
May 14-16, 2007
Kuczera et al. 2006
Hydrological Model
Simple Hydrological Watershed Model (2):
Motivation
Approach
Implementation
Application
Discussion
qrain   f rain rain (t ) 
qgw  f sat qgw,max
A
4
qrunoff  f sat  f rain rain (t )
qbf  k bf hgw
qet  1  exp( ket hs )  f pet pet (t ) 
qlat  f sat qlat,max
1
qdp  kdp hgw
B
qr  kr hr
2
f sat
3
5
6
1 7
1


1  sF exp( ks hs ) sF  1
Qr  f Q Aw qr
C
8
SAMSI Transition Workshop
May 14-16, 2007
8
3
1
3
Kuczera et al. 2006
model parameters
initial conditions
standard dev. of obs. err.
„modification parameters“
Hydrological Model
Motivation
Simple Hydrological Watershed Model (3):
1-exp(
0.0
ks=0.02/mm, sF=400
ks=0.01/mm, sF=400
ks=0.02/mm, sF=100
0
SAMSI Transition Workshop
May 14-16, 2007
200 400 600 800
hs [mm]
0.8
0.6
0.4
0.6
0.4
0.2
fs at [-]
Discussion
0.2
k e t h s ) [-]
0.8
Application
ket=0.01/mm
ket=0.02/mm
ket=0.005/mm
0.0
Implementation
1.0
1.0
Approach
0
200 400 600 800
hs [mm]
Model Application
Motivation
Approach
Implementation
Application
Discussion
 Data set of Abercrombie watershed, New South
Wales, Australia (2770 km2), kindly provided by
George Kuczera (Kuczera et al. 2006).
 Box-Cox transformation applied to model and
data to decrease heteroscedasticity of residuals.
 Step function input to account for input data in
the form of daily sums of precipitation and
potential evapotranspiration.
 Daily averaged output to account for output data
in the form of daily averaged discharge.
SAMSI Transition Workshop
May 14-16, 2007
Analyses and Prior Distributions
Motivation
Approach
Implementation
Application
Discussion
A) Estimation of constant parameters:
Independent lognormal distributions for all
parameters (8+3+1=11) with the exception of the
measurement standard deviation (1/), keeping
correction factors (frain, fpet, fQ) equal to unity.
B) Estimation of time-dependent parameters:
SAMSI Transition Workshop
May 14-16, 2007
Ornstein-Uhlenbeck process applied to the log of
the parameter.
Hyperparameters:  =1d,  =0.2 (22%) fixed, only
estimation of initial value and mean (0 for log frain,
fpet, fQ). Constant parameters as above.
Estimation of Constant Parameters
Motivation
A) Estimation of Constant Parameters:
Approach
Implementation
Application
Discussion
Try to find a reasonably good fit in which the
deterministic model with constant parameters
reproduces the major features of the data.
The goal of the second analysis with timedependent parameters will then be to support
finding causes of remaining model deficiencies.
SAMSI Transition Workshop
May 14-16, 2007
Estimation of Constant Parameters
Prior and Posterior Marginals:
0 2 4 6 8
0.010
0
50
Application
100
150
200
0
100
150
200
2.0
6
8
10
2.0
0.00
8
10
12
4
0.0
0.4
0.8
1.2
s_F
0.008
0.012
0.004
0
1.5
6
0.000
100
5 10
1.0
4
0
0.008
250
20
0.004
k_s
0
0.5
2
8
800
0.000
sd_Q_trans
0.0
0
k_r
0
0.012
0.6
0.0
4
400
800
400
0.008
0.4
1.0
2.0
0.0
2
k_dp
0
0.004
0.2
q_gw_max
1.0
500
0
k_bf
0.000
0.0
q_lat_max
0.000 0.004 0.008 0.012
SAMSI Transition Workshop
May 14-16, 2007
50
k_et
0 200
Discussion
h_r_ini
0.000
0.010
Implementation
h_gw_ini
0.020
h_s_ini
0.020
Approach
0.000
Motivation
0.02
0.04
0
500
1000 1500 2000
Estimation of Constant Parameters
0 50
1000
0 50
150
Q
150
time
Discussion
80 120
1000
1500
1000
1500
1000
1500
time
100
0
40
0
hgw
500
250
Application
0
500
25
15
400
0 5
200
0
Qrunoff
time
500
2
6
tsresval$Time
-6 -2
nondim. resid.
0
SAMSI Transition Workshop
May 14-16, 2007
qrain
1050
50 0
950
qrain
900
hs
850
Implementation
500
1000
1500
Qlat Qbf Qdp
Approach
150
Q
150
Motivation
50 0
Max. post. simulation with constant parameters:
Estimation of Constant Parameters
Results of Constant Parameter Fit:
Motivation
Approach
Implementation
Application
Discussion
SAMSI Transition Workshop
May 14-16, 2007
 The hydrological model with constant
parameters leads to
 a fit that reasonably well reproduces the
features shown by the data;
 a simulation with physically meaningful behaviour of state variables with respect to their
values and to their time scales of variation;
 identifiable model parameters (with the
exception of the initial condition of hr).
 Despite this basic agreement, the remaining
systematic deviations violate simple statistical
assumptions and make uncertainty analysis
difficult.
Estimation of Time-Dependent Parameters
B) Estimation of time-dependent parameters:
Motivation
Approach
Implementation
Application
Discussion
Sequentially replace constant parameters by timedependent parameters.
Try to learn from the results about deficits of the
deterministic model structure as well as about the
need for stochastic model extensions.
How to learn from the results of the analysis?
SAMSI Transition Workshop
May 14-16, 2007
1.
2.
3.
4.
5.
Analysis of temporal behaviour of parameters.
Analysis of posterior distributions of const. parameters.
Analysis of behaviour of model results.
Analysis of indicators of the quality of the fit.
Explorative analysis of the relationships between timedependent parameters and system variables.
1. Temporal Behaviour of Parameters
Time dependent parameter k_s
Implementation
ks
Approach
0.005
Application
0.020
Motivation
Discussion
0
500
1000
1500
SAMSI Transition Workshop
May 14-16, 2007
0.010 0.020 0.030
ks
time
0
500
1000
time
1500
1. Temporal Behaviour of Parameters
Time dependent parameter f_rain
0.5
Application
1.0
Implementation
frain
Approach
1.5
2.0
Motivation
Discussion
0
500
1000
1500
1.4
1.0
0.6
frain
1.8
time
0
SAMSI Transition Workshop
May 14-16, 2007
500
1000
time
1500
1. Temporal Behaviour of Parameters
Motivation
Implementation
fQ
Approach
Application
0.6 1.0 1.4 1.8
Time dependent parameter f_Q
Discussion
0
500
1000
1500
1.0
0.6
fQ
1.4
1.8
time
0
SAMSI Transition Workshop
May 14-16, 2007
500
1000
time
1500
1. Temporal Behaviour of Parameters
Time dependent parameter s_F
Implementation
sF
Approach
100
Application
300
500
Motivation
Discussion
0
500
1000
1500
200
100
sF
time
0
SAMSI Transition Workshop
May 14-16, 2007
500
1000
time
1500
1. Temporal Behaviour of Parameters
Assessment:
Motivation
Approach
Implementation
Application
Discussion
SAMSI Transition Workshop
May 14-16, 2007
 In cases with highly dynamic external forcing,
identified parameter time series are difficult to
interpret directly.
 The variation of width measures of the posterior
time-dependent parameter allows us to
distinguish time periods during which we can
gain information about variations in the
parameter from periods during which we cannot.
 In our example, this varies somewhat from one
parameter to the other, with a general tendency
that we can learn more during periods with rain
events than during dry weather periods.
2. Posterior of Constant Parameters
Results for time dependent parameter k_s
h_gw_ini
h_r_ini
0
50
100
150
200
0
0.000
Implementation
0.000
4
8
0.020
Approach
h_s_ini
0.020
Motivation
0
50
100
150
200
0.0
0.2
0.4
0.6
Application
1.5
1.5
0.0
0.000 0.004 0.008 0.012
q_gw_max
0.0
400
Discussion
q_lat_max
0
k_et
0
2
4
8
10
12
0.008
0.012
0.000
0.004
0.008
0.012
0.008
s_F
0.000
10
20
sd_Q_trans
0
SAMSI Transition Workshop
May 14-16, 2007
0.0
0.5
1.0
4
1.5
2.0
6
8
10
0 4 8
600
0.004
2
k_r
0
0
0.000
0
k_dp
600
k_bf
6
0
500
1000
1500 2000
0.0
0.4
0.8
1.2
12
2. Posterior of Constant Parameters
Results for time dependent parameter f_rain
h_gw_ini
h_r_ini
0
50
100
150
200
0
0.000
Implementation
0.000
4
8
0.020
Approach
h_s_ini
0.020
Motivation
0
50
100
150
200
0.0
0.2
0.4
0.6
Application
1.5
1.5
0.0
0.000 0.004 0.008 0.012
0
2
4
6
8
10
12
0.012
0.000
0.008
k_s
0.5
1.0
1.5
2.0
0.00
6
8
10
12
0.012
0.0
0.4
0.8
1.2
s_F
0.000
0
10
0 150
20
0.004
0.008
0.008
sd_Q_trans
0.0
4
0 4 8
600
0.004
2
k_r
0
0
0.000
0
k_dp
600
k_bf
SAMSI Transition Workshop
May 14-16, 2007
q_gw_max
0.0
400
Discussion
q_lat_max
0
k_et
0.02
0.04
0
500
1000 1500
2000
2. Posterior of Constant Parameters
Results for time dependent parameter f_Q
h_gw_ini
h_r_ini
0
50
Application
150
200
0
100
150
200
4
6
8
10
12
1.5
0.0
0.000
0.008
1.5
2.0
0.00
6
8
10
12
0.012
0.0
0.4
0.8
1.2
s_F
0.000
10
1.0
4
0.008
k_s
0
0.5
2
0 4 8
0.004
0 150
20
sd_Q_trans
0.0
0
k_r
600
0.012
0.6
1.5
2
0
600
0.008
0.4
q_gw_max
k_dp
0
0.004
0.2
0.0
400
0
k_bf
0.000
0.0
q_lat_max
0.000 0.004 0.008 0.012
SAMSI Transition Workshop
May 14-16, 2007
50
k_et
0
Discussion
100
0
0.000
Implementation
0.000
4
8
0.020
Approach
h_s_ini
0.020
Motivation
0.02
0.04
0
500
1000 1500
2000
2. Posterior of Constant Parameters
Results for time dependent parameter s_F
h_gw_ini
h_r_ini
0
50
Application
150
200
0
100
150
200
4
6
8
10
12
1.5
0.0
0.000
0.008
k_s
10
0
0.5
1.0
2
4
1.5
2.0
0.00
6
8
10
0 4 8
0.004
0 150
20
sd_Q_trans
0.0
0
k_r
600
0.012
0.6
1.5
2
0
600
0.008
0.4
q_gw_max
k_dp
0
0.004
0.2
0.0
400
0
k_bf
0.000
0.0
q_lat_max
0.000 0.004 0.008 0.012
SAMSI Transition Workshop
May 14-16, 2007
50
k_et
0
Discussion
100
0
0.000
Implementation
0.000
4
8
0.020
Approach
h_s_ini
0.020
Motivation
0.02
0.04
0.012
0.0
0.4
0.8
1.2
12
2. Posterior of Constant Parameters
Assessment:
Motivation
Approach
Implementation
Application
Discussion
 The marginal posterior distributions of some
parameters depend significantly on which of the
parameters was made time-dependent.
 In particular, making the modification factor for
rain intensity time dependent, changes the
posterior distributions of the other parameters
strongly.
 This demonstrates the importance of addressing
input (rainfall) intensity carefully.
SAMSI Transition Workshop
May 14-16, 2007
3. Behaviour of Model Results
1000
50
time
0
time
500
1000
1500
40
0
0
20
0 400
Qrunoff
time
500
1000
1500
5
tsresval$Time
-5 0
nondim. resid.
0
500
1000
hs
0 150 350
1500
1500
Qlat Qbf Qdp
1000
0 40
hgw
100
500
SAMSI Transition Workshop
May 14-16, 2007
qrain
1050
qrain
950
150
Q
Discussion
900
100 200
850
Implementation
Application
50
0
Approach
150
Q
Motivation
100 200
Results for time dependent parameter k_s
3. Behaviour of Model Results
50
950
1000
1050
200
50
time
0
1000
1500
1000
1500
1000
1500
0
60
100
time
0
hgw
140
500
0
500
20 40
0
300
0
Qrunoff
time
500
5
tsresval$Time
-5
nondim. resid.
0
SAMSI Transition Workshop
May 14-16, 2007
hs
Discussion
900
100 200
Q
Application
850
qrain
Implementation
qrain
200
0
Approach
500
1000
1500
Qlat Qbf Qdp
Q
Motivation
100 200
Results for time dependent parameter f_rain
3. Behaviour of Model Results
qrain
50
0
Approach
150
Q
Motivation
100 200
Results for time dependent parameter f_Q
Implementation
950
1000
1050
0
40 80
time
500
1000
1500
10 20
0
0 200
Qrunoff
time
0
500
1000
1500
-6
0 4
tsresval$Time
500
1000
1500
Qlat Qbf Qdp
0
nondim. resid.
hs
1500
150 300
1000
0
hgw
500
SAMSI Transition Workshop
May 14-16, 2007
qrain
150
50
time
0
Discussion
900
100 200
Q
Application
850
3. Behaviour of Model Results
950
1000
1000
1500
1000
1500
150 300
1500
100
time
0
500
25
0 10
300
0
Qrunoff
time
500
0 4
tsresval$Time
-6
nondim. resid.
0
500
1000
1500
Qlat Qbf Qdp
0
0 40
hgw
1000
hs
0
qrain
50
time
500
SAMSI Transition Workshop
May 14-16, 2007
qrain
1050
150
Q
Discussion
900
100 200
850
Implementation
Application
50
0
Approach
150
Q
Motivation
100 200
Results for time dependent parameter s_F
3. Behaviour of Model Results
Assessment:
Motivation
Approach
Implementation
Application
Discussion
SAMSI Transition Workshop
May 14-16, 2007
 The basic features of the solutions are not
changed by introducing a time-dependent
parameter.
 For some of the parameters, making them timedependent significantly reduces the bias in
model output, for others this is not the case.
4. Quality of Fit
Improvement with time-dependent parameters:
Motivation
Nash-Sutcliffe indices:
Approach
Implementation
Application
Discussion
SAMSI Transition Workshop
May 14-16, 2007
ks
frain
fQ
sF
kr
fpet
qgw,max
qlat,max
kdp
kbf
base
0.83
0.78
0.68
0.64
0.59
0.57
0.54
0.53
0.53
0.53
0.53
NS  1 
M 2
(
y

y
 i i )
i
2
(
y

y
)
 i
i
Assessment:
 Input (frain) and output (fQ) corrections.
 Potential for soil /
runoff model (ks, SF)
improvements.
 Some potential for
river and evaporation
improvements.
Random or deterministic?
5. Relationsship with Model Variables
Scatter plot of k_s vs. model variables
SAMSI Transition Workshop
May 14-16, 2007
q_dp
0.020
0.5
1.5
q_r
0.020
ks
0.012
0.3
0.1
4
f_sat
2.5
0.020
0.012
ks
0.020
0.012
ks
0.012
0.1 0.3 0.5 0.7
0.020
0.012
ks
0.020
0.012
0.020
0.020
0.020
0.012
ks
0.020
0.012
ks
q_bf
q_runoff
q_et
q_rain
0.01 0.03 0.05
0.0 1.0 2.0 3.0
1.0 2.0 3.0 4.0
10 15
0.012
5
2 3
h_r
h_gw
0.012
ks
0.020
0.012
ks
0
1
20 40 60 80
250
h_s
Q
Discussion
ks
150
60
ks
20
Application
ks
Implementation
0.020
0.012
ks
0.020
0.012
Approach
ks
Motivation
0.1 0.3 0.5 0.7
q_lat
0.5
q_gw
1.5
5. Relationsship with Model Variables
Scatter plot of f_rain vs. model variables
20
Application
60
20
Q
60 100
160
20
h_s
60
100
1.2
0.8
frain
1.2
0.8
frain
1.2
frain
1.2
100
0.8
Implementation
0.8
frain
1.2
0.8
Approach
frain
Motivation
0.5
h_gw
1.5
2.5
0.05
h_r
0.15
f_sat
5
10 15 20
1.0 2.0
0.06
q_bf
SAMSI Transition Workshop
May 14-16, 2007
2.0
0.8
frain
1.2
0.2
0.6
q_dp
1.0
0.5 1.5
q_r
2.5
1.2
0.8
frain
1.2
0.8
frain
1.2
1.0
q_runoff
0.8
0.02
0.0
q_et
frain
1.2
0.8
frain
q_rain
3.0
1.2
0
0.8
frain
1.2
0.8
frain
1.2
0.8
frain
Discussion
0.2
0.6
q_lat
0.5
1.5
q_gw
0.25
5. Relationsship with Model Variables
Scatter plot of f_Q vs. model variables
20
Application
50
Q
150
20
40
h_s
60
80
1.2
0.8
fQ
1.2
0.8
fQ
1.2
fQ
1.2
60
0.8
Implementation
0.8
fQ
1.2
0.8
Approach
fQ
Motivation
0.5 1.5 2.5 3.5
h_gw
0.05
h_r
0.15
0.25
f_sat
10 15
0.015
q_bf
2.0
0.6
q_dp
0.5
1.5
q_r
2.5
1.2
0.8
fQ
0.1
0.3
q_lat
1.2
1.2
0.2 0.4
1.2
0.8
fQ
1.2
1.0
q_runoff
0.8
0.8
0.005
0.0
q_et
fQ
1.2
q_rain
fQ
0.8
1.0 2.0 3.0 4.0
0.8
5
fQ
0
SAMSI Transition Workshop
May 14-16, 2007
fQ
1.2
0.8
fQ
1.2
0.8
fQ
Discussion
0.2
0.6
q_gw
1.0
5. Relationsship with Model Variables
Scatter plot of s_F vs. model variables
20
Application
60
100
Q
200
20 40 60 80
h_s
250
150
sF
250
150
sF
250
150
sF
150
150
Implementation
sF
sF
250
Approach
250
Motivation
1
h_gw
2
3
4
0.05
h_r
0.15
0.25
f_sat
5
10 15
0.015
q_bf
2.0
150
q_dp
0.5
1.5
q_r
250
150
0.1
0.3
q_lat
sF
0.2 0.4 0.6 0.8
sF
250
150
1.0
q_runoff
250
150
sF
150
sF
0.005
sF
0.0
q_et
250
q_rain
SAMSI Transition Workshop
May 14-16, 2007
250
1.0 2.0 3.0
250
0
150
sF
250
150
sF
150
sF
250
Discussion
2.5
0.2
0.6
q_gw
1.0
5. Relationsship with Model Variables
Assessment:
Motivation
Approach
Implementation
Application
Discussion
SAMSI Transition Workshop
May 14-16, 2007
 Most of the time dependent parameters do not
show deterministic variation with any of the
system variables.
 The only exception is the parameter ks of the soil
submodel that varies significantly with the
saturated area (which it parameterizes).
Conclusions
Assessment:
Motivation
Approach
Implementation
Application
Discussion
 Stochasticity seems to be the dominating cause
of deviations of model results from
measurements.
 This is likeli to be dominated by input (rainfall)
uncertainty.
 The highest chance to find an improvement of
the deterministic model is for the soil/runoff
submodel of the hydrological model.
 It seems difficult to significantly improve the
model by changes to the groundwater and river
sub-models.
SAMSI Transition Workshop
May 14-16, 2007
Hydrological Model
Model extensions:
Motivation
Approach
Extension 1: Modification of runoff flux:
Implementation
Application
qrunoff
Discussion
e


f rain rain (t )  rain
  f sat   (1  f sat ) e
erain
rain


K

f
rain
(
t
)
rain
rain


 f rain rain (t )

Extension 2: Modification of sat. area funct.:
f sat
SAMSI Transition Workshop
May 14-16, 2007
hse1
hse 2
  e1
 (1   ) e 2
e1
K1  hs
K 2  hse 2
Both extensions lead to three more model parameters.
Hydrological Model
Motivation
Previous results:
Extended models:
Nash-Sutcliffe indices:
Nash-Sutcliffe indices:
Approach
Implementation
Application
Discussion
SAMSI Transition Workshop
May 14-16, 2007
ks
frain
fQ
sF
kr
fpet
qgw,max
qlat,max
kdp
kbf
base
0.83
0.78
0.68
0.64
0.59
0.57
0.54
0.53
0.53
0.53
0.53
ext. 1
ext. 2
0.72
0.54
Assessment:
Model extension 1
significantly improves
the description of the
system.
Approach
Motivation
Approach
Implementation
Application
Discussion
SAMSI Transition Workshop
May 14-16, 2007
Hydrological Model
Motivation
Approach
Implementation
Application
Discussion
Next Steps
 Redo analysis with model extension 1.
 Compare remaining stochastic uncertainty
with knowledge on input uncertainty.
 Do uncertainty analysis for model with
extensions 1 and input uncertainty.
 Investigate alternative ways of describing
rainfall input uncertainty
SAMSI Transition Workshop
May 14-16, 2007
Conclusions
The application of the technique led to the discovery of improvements of the deterministic model
structure as well as to the inclusion of stochasticity.
Discussion
Motivation
Approach
Implementation
Application
Discussion
SAMSI Transition Workshop
May 14-16, 2007
Discussion
Discussion
Motivation
Approach
Implementation
Application
Discussion
• The suggested procedure seems to fulfill the
expectations of supporting the identification of
model deficits and of introducing stochasticity
into a deterministic model.
• There is need for future research in the following
areas:
• Explore alternative ways of learning from the
identified parameter time series.
• Different formulation of time-dependent parameter
(for some applications smoother behaviour).
• Improve efficiency (linearization, emulation).
SAMSI Transition Workshop
May 14-16, 2007
• Learn from more applications.
Transition
On-going projects in various fields:
Motivation
Approach
Implementation
• Applications for gaining more experience:
• Reichert et al.:
• Cintron et al.:
hydrological model
epidemiological model
Application
Discussion
• Emulation of dynamic models:
•
•
•
•
Reichert et al.:
White et al.:
Liu et al.:
Gosling et al.:
simple physical-based prior
extended physical-based prior
statistical prior
emulation of time step
• Linearization for improving efficiency:
• Paulo et al.:
• Liu et al.:
SAMSI Transition Workshop
May 14-16, 2007
emulation of linearized model
direct use of linearized model
Other persons: Bayarri, Santner, Pitman, O‘Hagan, Wolpert
More ideas on the way. Post-program workshop next year?
Acknowledgements
Motivation
Approach
Implementation
Application
Discussion
 Development of the technique:
Hans-Rudolf Künsch, Roland Brun, Lorenzo
Tomassini, Mark Borsuk, Christoph Buser.
• Hydrological example
Johanna Mieleitner, George Kuczera.
• Interactions at SAMSI:
Susie Bayarri, Tom Santner, Gentry White, Ariel
Cintron, Fei Liu, Rui Paulo, Robert Wolpert, John
Paul Gosling, Tony O‘Hagan, Bruce Pitman, Jim
Berger, and many more.
SAMSI Transition Workshop
May 14-16, 2007
I would like to thank in particular to Jim Berger and
Susie Bayarri for setting up this program that lead to a
very stimulating and fruitful stay for me at SAMSI.