STOCHASTIC HYDROLOGY
Stochastic Simulation (I)
Univariate simulation
Professor Ke-Sheng Cheng
Department of Bioenvironmental Systems Engineering
National Taiwan University
Stochastic Hydrology
• Hydrological processes exhibit variations in
both space and time. As hydrological models
are simplified versions of reality, they
produce predictions or estimates of
hydrological variables (e.g. runoff, hydraulic
head, concentration) that are inherently
erroneous.
• Stochastic hydrology is mainly concerned
with the assessment of uncertainty in
hydrological analysis, modeling and
forecasting.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
2
• In stochastic hydrology, the assessment of
uncertainty is an integral part of hydrological
analysis and modeling, being as important as
the predictions themselves.
• Assessment of uncertainties is achieved by
using “stochastic models”, which are models
consisting of random components. These
random components characterize the part of
reality that is not explained by the
deterministic components in our model.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
3
• Errors in hydrological model prediction can
occur because
– the model concept is wrong, or Model uncertainty
– due to errors in parameters, boundary and
initial conditions.
Parameter uncertainty
• We may choose to ignore these errors and
accept our model predictions at face value.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
4
• However, in operational hydrology, when
actual decisions have to be made based on
hydrological model predictions (sometimes
involving human life such as in flood control),
it is imperative that uncertainty is taken into
account in the decision-making process.
– Government policy decision-making is a
complicated process. (Holistic decision making)
– An inappropriate decision may result in
significant losses of life or over/underinvestments in public infrastructures.
Unintended consequences of government policy
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
5
What does a prediction really represent?
• Example of a linear regression model
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
6
What does a prediction really represent?
• Example of a linear regression model.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
7
• Predictions of data-driven models.
• Predictions of deterministic models.
• Properties of the predictand variable
– Unbiased ?
– Confidence interval ?
– Efficient ?
For parameter estimation, we are concerned about the
above properties of our estimators, but why not the
predictions?
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
8
• The objectives of this course are
– To demonstrate the stochastic nature of many
hydrological processes,
– To facilitate students with a stochastic
perspective of hydrological modeling and
forecasting, and
– To introduce techniques of stochastic simulation
which will enable students to explore a wide
range of applications.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
9
Univariate Simulation
•
•
•
•
•
Pseudo random number generation
Probability integral transformation
Rejection method
Frequency-factor based generation
Random number generation using R
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
10
Pseudo Random Number
Generator (PRNG)
• Computer simulation of random variables is
the task of using computers to generate many
random numbers that are independent and
identically distributed (IID). It is also known
as random number generation (RNG).
• In fact, these computer-generated random
numbers form a deterministic sequence, and
the same list of numbers will be cycled over
and over again. This cycle can be made to be
so long that the lack of true independence is
unimportant.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
11
• Therefore, such computer codes are often
termed pseudo-random number generators
(PRNG).
• There exist mathematical transformation
methods to obtain other distributions from
uniform variates. For this reason, most
PRNGs found in software libraries produce
uniform random numbers in the unit interval
(0, 1).
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
12
Linear Congruential Generator
• Generation of random samples of various
probability densities is based random
samples of the uniform density U[0,1).
Therefore, the algorithm of generating
random numbers of U[0,1) is essential. This
can be achieved by the Linear Congruential
Generator (LCG) described below.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
13
• Let a sequence of numbers xn be defined by
xn  [axn 1  c] modulo m
where a, c, and m are given positive integers.
The above equation means that axn 1  c is
divided by m and the remainder is taken as the
value of xn . The quantity xn / m is then taken as an
approximation to the value of a U[0,1) random
variable. When c = 0, the algorithm is also called a
pure multiplicative generator.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
14
• A guideline for selection of a and m (c =0) is that m
be chosen to be a large prime number that can be
fitted to the computer word size. For a 32-bit word
5
31
computer, m = 2  1 and a = 7 result in desired
properties.
• For small computers without a random number
generator, the following a, c, and m are found to be
satisfactory when the LCG algorithm is used:
a  25173, c  13849 and m  65536.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
15
PROBABILITY INTEGRAL
TRANSFORMATION
• The PIT method is based on the property that a
random variable X with CDF FX () can be
transformed into a random variable U with uniform
distribution over the interval (0,1) by defining
U  FX (X )
• Conversely, if U is uniformly distributed over the
interval (0,1), then X  FX1 (U ) has cumulative
distribution function FX () .
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
16
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
17
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
18
• For random variables whose cumulative
distribution function cannot be expressed by
a close form the probability integral
transformation technique cannot be used for
generating random numbers of these random
variables.
• The normal distribution is one such random
variable.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
19
The Acceptance/Rejection Method
• This method uses an auxiliary density for
generation of random quantities from
another distribution. This method is
particularly useful for generating random
numbers of random variables whose
cumulative distribution functions cannot be
expressed in closed form.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
20
• Suppose that we want to generate random
numbers of a random variable X with density
f(X).
• An auxiliary density g(X) which we know
how to generate random samples is identified
and cg(X) is everywhere no less than f(X) for
some constant c, i.e.,
f ( x)  cg ( x)
3/14/2016
x
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
21
cg(X)
f(X)
X
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
22
•
•
•
•
Generate a random number x of density g(X),
Generate a random number u from the density
U[0,cg(x)),
Reject x if u > f(x); otherwise, x is accepted as a
random number form f(X),
Repeat the above steps until the desired number
of random numbers are obtained.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
23
Frequency-factor-based generation
• An advantage of the method is that it does
not require CDF inversion and frequency
factors of the five commonly used
distributions involves only the standard
normal and the uniform deviates.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
24
GENERAL EQUATION FOR
HYDROLOGICAL FREQUENCY ANALYSIS
A random variable X has cumulative distribution
function FX () with mean  and standard
deviation  . The magnitude of X corresponding to
return period T, denoted by xT , is defined as
P X  xT   1
T
Chow (1951) proposed the following general
equation for hydrologic frequency analysis:
xT    KT
where KT , the frequency factor, is a function of T
and is distribution-specific.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
25
• Apparently, if X is normally distributed, the
frequency factor KT corresponds to the
standard normal deviate with exceedence
probability 1/T.
• Frequency factors of distributions commonly
used in hydrologic frequency analysis have
been developed (Kite, 1988).
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
26
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
27
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
28
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
29
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
30
TEST AND VALIDATION
• In order to demonstrate the applicability of
the FQFT approach, random numbers of
normal, log-normal, extreme value type I
(EV1), Pearson type III (PT3) and LogPearson type III (LPT3) distributions were
generated and tested.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
31
• For each type of distribution N random
samples, each of size n, were generated and
used in subsequent analysis.
• In this study the sample size n was set to vary
from 50 to 500 at increment of 50 and
number of random samples N was set to
1,000 and 10,000.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
32
• Three means were adopted to test the
validity of the generated random
numbers:
– Graphical comparison of CDF and
empirical CDF (ECDF) derived from
generated data,
– Properties of estimated parameters, and
– Type-I-error of goodness-of-fit (GOF) test.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
33
Graphical comparison of CDF and
ECDF
• Figure 2 graphically illustrates the closeness of CDF
and ECDF with regard to sample size of 50 and 500.
Each ECDF in Figure 2 is based on one single
random sample of size 50 or 500 and it may change
when another random sample is used.
• It can be seen that even at sample size of 50 the
ECDF is fairly close to CDF of the designated
distribution. At sample size of 500, all ECDFs
become almost indistinguishable from their
corresponding CDFs.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
34
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
35
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
36
Properties of parameter estimators
• From each of the N generated random
samples, distribution parameters including
mean, standard deviation and coefficient of
skewness can be estimated.
• Furthermore, from a total of N random
samples, the sample mean and standard
deviation of the above estimated parameters
were calculated, with respect to sample size n
ranging from 50 to 500.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
37
Uncertainty in estimation
of mean reduces as
sample size increases.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
38
Uncertainty in estimation
of standard deviation
reduces as sample size n
increases.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
39
Uncertainty in estimation
of skewness coefficient
reduces as sample size n
increases.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
40
• With 1,000 and 10,000 random samples
(N=1000 or 10,000), sample means (the center
line) of the estimated parameters (including
mean, standard deviation and coefficient of
skewness) are very close to the theoretical
values designated for random number
generation.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
41
• It is also seen clearly that standard deviations
of all parameter estimators decrease with
increase of the sample size n, indicating the
unbiasedness of the estimator and reduction
of uncertainty in parameter estimation. Such
characteristics of parameter estimators
suggest the generated random samples are
indeed from the desired distributions.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
42
Type-I-Error of goodness-of-fit test
• Each random sample of size n is generated
from a theoretical distribution with
designated parameters and GOF test can be
applied to test whether the random sample is
drawn from the theoretical distribution. The
widely applied Chi-square GOF test is
adopted in this study.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
43
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
44
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
45
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
46
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
47
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
48
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
49
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
50
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
51
Generating random samples of
normal distribution
• The Box-Muller method
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
52
• The Central Limit Theorem
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
53
• Acceptance/Rejection Method
– The cumulative distribution function of the
exponential density and its inverse function can
be easily derived, and therefore, random samples
of the exponential distribution can be generated
with the probability integral transformation
method.
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
54
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
55
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
56
3/14/2016
Lab for Remote Sensing Hydrology and Spatial Modeling
Dept of Bioenvironmental Systems Engineering, NTU
57
Random number generation in R
• R commands for stochastic simulation (for
normal distribution
– pnorm – cumulative probability
– qnorm – quantile function
– rnorm – generating a random sample of a specific
sample size
– dnorm – probability density function
For other distributions, simply change the distribution names.
For examples, (punif, qunif, runif, and dunif) for uniform
distribution and (ppois, qpois, rpois, and dpois) for Poisson
distribution.
3/14/2016
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
58
Generating random numbers of
discrete distribution using R
• Discrete uniform distribution
– R does not provide default functions for random
number generation for the discrete uniform
distribution.
– However, the following functions can be used for
discrete uniform distribution between 1 and k.
•
•
•
•
3/14/2016
rdu<-function(n,k) sample(1:k,n,replace=T) # random number
ddu<-function(x,k) ifelse(x>=1 & x<=k & round(x)==x,1/k,0) # density
pdu<-function(x,k) ifelse(x<1,0,ifelse(x<=k,floor(x)/k,1))
# CDF
qdu <- function(p, k) ifelse(p <= 0 | p > 1, return("undefined"),
ceiling(p*k))
# quantile
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
59
– Similar, yet more flexible, functions are defined as
follows
• dunifdisc<-function(x, min=0, max=1) ifelse(x>=min & x<=max &
round(x)==x, 1/(max-min+1), 0)
>dunifdisc(23,21,40)
>dunifdisc(c(0,1))
• punifdisc<-function(q, min=0, max=1) ifelse(q<min, 0, ifelse(q>max, 1,
floor(q-min+1)/(max-min+1)))
>punifdisc(0.2)
>punifdisc(5,2,19)
• qunifdisc<-function(p, min=0, max=1) floor(p*(max-min+1))+min
>qunifdisc(0.2222222,2,19)
>qunifdisc(0.2)
• runifdisc<-function(n, min=0, max=1) sample(min:max, n, replace=T)
>runifdisc(30,2,19)
>runifdisc(30)
3/14/2016
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
60
• Binomial distribution
3/14/2016
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
61
• Poisson distribution
3/14/2016
Laboratory for Remote Sensing Hydrology and Spatial Modeling,
Dept of Bioenvironmental Systems Engineering, National Taiwan Univ.
62