ISSH - Stochastic Hydraulics 2005 - 23 and 24 May 2005 - Nijmegen - The Netherlands Implications of uncertainties on flood defence policy J.K. Vrijling, P.H.A.J.M. van Gelder Delft University of Technology, Faculty of Civil Engineering and Geosciences, P.O. Box 5048, NL-2600 GA Delft, The Netherlands 1 ABSTRACT This paper is concerned with the implications on flood defence policy connected with the lack of information in hydraulic engineering models. The paper focusses on hydraulic engineering models which result in a probability of failure. The different types of uncertainty in such models will be discussed. Future research might give information and reduce some of the uncertainties. It will be discussed how to present the current and the uncertain future probability of failure. A simplified model which calculates the probability of failure of a dike along the Dutch lake IJssel (1500 km2) will be used as an example. 2 INTRODUCTION The present safety standards of the Dutch dikes and flood defences date to the report of the "Staatscommissie voor den Waterweg" (1920) and are expressed as an exceedance frequency of the design water level (see Fig. 1). Much research on safetyand risk analysis has been carried out since 1920, especially in the area of extreme value statistics. In civil engineering practice many parameter estimation methods for probability distribution functions are in circulation. Well known methods are for example: the method of moments (Johan Bernoulli, 16671748), - the method of maximum likelihood (Daniel Bernoulli, 1700-1782), the method of least squares (on the original or on the linearized data), (Gauss, 1777-1855), the method of Bayesian estimation (Bayes, 1763), the method of minimum cross entropy (Shannon, 1949), - the method of probability weighted moments (Greenwood et al., 1979), - the method of L-moments (Hosking, 1990). Many attempts (for instance, Goda and Kobune, (1990), Burcharth and Liu, (1994), Yamaguchi, (1997), and Van Gelder and Vrijling, (1997)), have been made to find out which estimation method is preferable for the parameter estimation of a particular probability distribution in order to obtain a reliable estimate of the p-quantiles. It is desirable that the quantile estimate be unbiased, that is, its expected value should be equal to the true value. It is also desirable that an unbiased estimate be efficient, i.e., its variance should be as small as possible. The problem of unbiased and efficient estimation of extreme quantiles from small samples is commonly encountered in the civil engineering practice. For example, wave height data may be available for past 10 to 20 years and on that basis one may have to estimate a design flood level corresponding to a 100 years return period. The engineer who has to design a breakwater will hesitate a lot, or rather reject, the extrapolation of a wave height to a 10-2 quantile, if it is lower than the highest observation in the collected data (see Fig. 2). The engineer will prefer a method which takes physical reasoning into consideration. Physical reasoning to judge the homogeneity of a data set are for instance: 1. Type of spectrum (swell, wind, single or double peaked wave height spectrum); 2. Season, calendar period of the data set (sea level data in the winter or summer); 3. Physical characteristics of the phenomena (breaking, non-breaking waves). Combining statistical models with physical arguments leads to more homogeneous data and realistic outcomes than applying statistical methods only. The research of the design of the Eastern Scheldt barrier (Vrijling and Bruinsma, 1980 and Mulder and Vrijling, 1980) shows how wave height, sea level, and wave period data should be analysed with help of physical models. Also the case study of wave height and peak period data measured in the Bay of Bengal near the city of Madras by Van Gelder and Vrijling, 1998, demonstrates this importence. Klemes (1993) proposes to find the statistical distributions of river discharges by using orographic and Fig. 1: Frequency analysis 'avant-la-lettre'; 1920 publication with number of storm surges above water level at Hoek van Holland. POT data, exponential fit, and quantile uncertainty (1 std) 0 10 -1 Frequency 10 -2 10 -3 10 -4 10 2 3 4 5 6 X 7 8 9 10 Fig. 2: Extrapolation of peaks-over-threshold data with a ML method to a 10-2 quantile which is lower than the highest observation (6.0 m). ISSH - Stochastic Hydraulics 2005 - 23 and 24 May 2005 - Nijmegen - The Netherlands synoptic considerations, rather than fitting the annual maxima or peaks over threshold values of discharges to distributions. The Dutch Ministry of Public Works wants to change the policy of estimating failure probabilities to bring it in line with the approach in areas like planning and transport, where failure probabilities are given in a framework of acceptable risk. The transition from water level criteria towards flooding probabilities -and finally to a flood risk approachrequires a model to calculate the probability of a failure of a dike system with several boundary conditions: 1 The model should be widely accepted by the flood defence community and in some sense by the general public. 2 The results of the model should be sufficiently robust. The answers should not vary substantially with slight moderations of the input. 3 The results of the model should not actuate the decision to alter the dike systems. The dike systems are now in full compliance with the Deltacommittee standard and are generally considered to be sufficiently safe. In the past few years models have been developed in order to determine the probability of failure of a dike system, including several failure mechanisms and their statistical dependencies. The mentioned boundary conditions to the model were approached by carrying out case studies of typical, slightly schematised polders in the Netherlands. In the case studies a comparison between the old and the new safety approach has been made, to show that the differences are minor (condition 3). Subsequently these case studies are a good first step on the road to acception of the model (condition 1). A problem that still has to be investigated in accordance with the second condition is whether a future change in the uncertainties in the model could cause a significant change in the resulting probability of failure. To investigate this an uncertainty analysis of the model has to be carried out. An uncertainty analysis can give insight into the contributions and the effects that various types of uncertainty have on the resulting probability of flooding. The discussion of this paper will be the problems connected with an uncertainty analysis of a model with a probability of failure as a result. The paper is organized as follows. First the different types of uncertainty will be discussed followed by the available options to present the results of an uncertainty analysis. A model for the effects of the uncertainty due to lack of information will be presented and applied to a case study. Finally the conclusions are given. 3 TYPES OF UNCERTAINTY Uncertainties in decision and risk analysis can primarily be divided into two categories: uncertainties that stem from variability in known (or observable) populations and therefore represent randomness in samples (inherent uncertainty), and uncertainties that come from basic lack of knowledge of fundamental phenomena (epistemic uncertainty). Inherent uncertainties represent randomness or the variations in nature. For example, even with a long history of data, one cannot predict the maximum water level that will occur in, for instance, the coming year at the North Sea. It is not possible to reduce inherent uncertainties. Epistemic uncertainties are caused by lack of knowledge of all the causes and effects in physical systems, or by lack of sufficient data. It might be possible to obtain the type of the distribution, or the exact model of a physical system, when enough research could and would be done. Epistemic uncertainties may change as knowledge increases. 3.1 Five types of uncertainty The inherent uncertainty and epistemic uncertainty can be subdivided in the following five types of uncertainty (Paté-Cornell 1996): inherent uncertainty in time and in space, parameter uncertainty and distribution type uncertainty (together also known as statistical uncertainty) and finally model uncertainty. Fig. 3: Types of uncertainty. i) Inherent uncertainty in time Stochastic processes running in time such as the occurrence of water levels and wave heights are examples of the class of inherent uncertainty in time. Unlimited data will not reduce this uncertainty because the realizations of the process in the future stay uncertain. ii) Inherent uncertainty in space Random variables that represent the fluctuation in space, such as the dike height. Just as for inherent uncertainty in time it holds that unlimited data (e.g. if the height would be known every centimeter) will not reduce this uncertainty. There will always still be a fluctuation in space. iii) Parameter uncertainty This uncertainty occurs when the parameters of a distribution are determined with a limited number of data. The smaller the number of data, the larger the parameter uncertainty. iv) Distribution type uncertainty This type represents the uncertainty of the distribution type of the variable. It is for example not clear whether the occurrence of the water level of the North Sea is exponentially or Gumbel distributed or whether it has a completely different distribution. Remark: a choice was made to divide statistical uncertainty into parameter- and distribution type uncertainty although it is not always possible to draw the line; in case of unknown parameters (because of lack of observations), the distribution type will be uncertain as well. Later in this paper we will try to distinguish what type of uncertainty a stochastic parameter represents. Since parameter uncertainty and distribution type uncertainty can not be discerned, another practical less scientific- division has been chosen. The statistical uncertainty is divided in two parts: ‘statistical uncertainty of variations in time’ and ‘statistical uncertainty of variations in space’. Fig. 4: Statistical uncertainty. v) Statistical uncertainty of variations in time When determining the probability distribution of random variable that represents the variation in time of a process (like the occurrence of a water level), there essentially is a problem of information scarcity. Records are usually too short to ensure reliable estimates of low-exceedance probability quantiles in many practical problems. The uncertainty caused by this shortage of information is the statistical uncertainty of variations in time. This uncertainty can theoretically be reduced by keeping record of the process for the coming centuries. vi) Statistical uncertainty of variations in space When determining the probability distribution of random variable that represents the variation in space of a process (like the fluctuation in the height of a dike), there essentially is a problem of shortage of measurements. It is usually too expensive to measure the height or width of a dike in great detail. This statistical uncertainty of variations in space can be reduced by taking more measurements (see also Vrijling and Van Gelder, 1998). vii) Model uncertainty Many of the engineering models that describe the natural phenomena like wind and waves are imperfect. They may be imperfect because the physical phenomena are not known (for example when regression models without underlying theory are used), or they can be imperfect because some variables of lesser importance are omitted in the engineering model for reasons of efficiency. 3.2 Reduction of uncertainty In Sec. 3.1, it was mentioned that inherent uncertainties represent randomness or the variations in nature. Inherent uncertainties cannot be reduced. Epistemic uncertainties, on the other hand, are caused by lack of knowledge. Epistemic uncertainties may change as knowledge increases. In general there are three ways to increase knowledge: - Gathering data - Research - Expert-judgment Data can be gathered by taking measurements or by keeping record of a process in time. Research can for instance be done into the physical model of a phenomenon or into the better use of existing data. By using expert opinions it is possible to acquire the probability distributions of variables that are too expensive or practically impossible to measure. The goal of all this research obviously is to reduce the uncertainty in the model. Nevertheless it is also thinkable that uncertainty will increase. Research might show that an originally flawless model actually contains a lot of uncertainties. Or after taking ISSH - Stochastic Hydraulics 2005 - 23 and 24 May 2005 - Nijmegen - The Netherlands some measurements the variations of the dike height can be a lot larger. It is also thinkable that the average value of the variable will change because of the research that has been done. The consequence is that the calculated probability of failure will be influenced by future research. In order to guarantee a stable and convincing flood defence policy after the transition, it is important to understand the extent of this effect. In the next section will be discussed what influence the future reduction of uncertainty can have on the probability of failure and how to present this influence. It is assumed that in practice research will rarely increase uncertainty or change the average value of a variable. 4 THE EFFECT OF UNCERTAINTY DUE TO LACK OF INFORMATION The problem of lack of information in hydraulic engineering models is studied in detail in this section. Data records are usually too short to ensure reliable estimates of low-exceedance probability quantiles in many practical problems . The information about a random variable can be updated by the help of expert judgements. Cooke (1991) describes various methods to do so. Expert judgements can also be used to reduce the uncertainty of the quantiles. In order to include expert judgements in the quantile estimation of a certain quantity, the following approach is proposed in this paper. The approach will be applied to a case study of quantile estimation of water levels at Lake IJssel. Consider S the random variable describing the loads on the structure. Consider R the random variable describing the resistance of the structure. The effect of the value of information on the random variables S and R may be modelled by correcting its original mean value by subtracting U in which U is the distribution which represents the uncertainty. Furthermore the standard deviation of U will be added in (S2+R2) to take account for the uncertainty. Summarized in a table: The implications of the changes in and the related flooding frequency of the case study Lake IJssel will be discussed in Sec. 5. As discussed in Sec. 3.2 it is possible that some uncertainties might be reduced in the future. One could say that those uncertainties add to the ‘uncertainty’ of the probability of failure. The obvious problem is that it can not be predicted which uncertainties will be reduced in the future. There are several philosophies thinkable about what uncertainty will remain part of the future probability of failure. It is important to realise that, for every philosophy, the difference between the ‘true’ probability of failure and the ‘uncertainty’ of the probability of failure, is an artificial difference. If all uncertainties are integrated this will result in the same, current probability of failure. Let us consider four different philosophies (options) about the uncertainties that might be reduced and form the ‘uncertainty’ of the probability of flooding, and the uncertainties that cause the ‘true’ probability of flooding, as given in table 2: Inherent uncertainty (in time) Inherent uncertainty (in space) Statistical uncertainty (of variations in time) Statistical uncertainty (of variations in space) Model uncertainty Opt 1 Pf Opt 2 Pf Opt 3 Pf Opt 4 Pf Pf Pf Pf - Pf Pf - - Pf - - - Pf - - - Table 2: Four different options Option 1 No incertainties will be reduced. All uncertainties in the model will be integrated to determine the probability of failure. This probability of failure also represents the current, actual probability of failure. Option 2 A practical division is made between the uncertainties that might be influenced by taking measurements or doing research and the remaining uncertainties. Statistical uncertainty of variations in time will be a part of the probability of failure. This uncertainty can only theoretically be reduced by keeping record of the underlying process for the coming centuries. Table 1: The effect of information U on the random variables R and S. Option 3 A theoretical division is made between inherent uncertainty and epistemic uncertainty. The assumption is that through research all epistemic uncertainty disappears. This would result in a probability of failure that is solely caused by inherent uncertainties. Option 4 Assumed is that only inherent uncertainty in time causes the probability of failure. This is also the philosohy used in the present safety standards. wind surge, wave height, wave steepness, wave runup, and lake oscillations were taken into account. Two locations along the Lake IJssel were considered in detail: Enkhuizen and Rotterdamsche Hoek (Fig.1). A short summary of the uncertainty analysis study of Vrijling et al. (1999) will be given in Sec. 5.1. The goal of Sec. 5.2 is to analyze the influences of the uncertainties in the water levels on the reliability-based optimal design of the dikes at the locations of Enkhuizen and Rott.Hoek. 1.6 Option 1 1.4 Option 2 1.2 Option 3 1 Option 4 0.8 0.6 0.4 0.2 0 2 2.5 3 3.5 4 4.5 5 5.5 6 Reliability index Fig. 5: Probability Distributions of In Fig. 5 the probability of failure of a hypothetical example is shown in four different ways. Option 1 gives the probability with no uncertainty. With every next option, more uncertainty is removed from the ‘true’ probability of failure and added to the ‘uncertainty’ of the probability of failure. It is clear that the more uncertainty is removed from the resulting probability of failure; the higher the mean and the larger the standard deviation of the distribution of the reliability index will be. 5 CASE STUDY Lake IJssel is situated in the northern part of the Netherlands (Fig. 6). It has an area of approximately 1500km2. The lake is surrounded by dikes in order to protect the low-lying polders from flooding. The required safety against inundation of the polders is 1/4000 yr-1. In Westphal et al. (1997), a physical model has been developed for the water levels of Lake IJssel.It is based on WAQUA (a twodimensional water flow model) and HISWA (a wave model). The model of Westphal has been analyzed in an uncertainty study by Vrijling et al. (1999). Uncertainties with respect to water level, wind speed, Fig. 6: Lake IJssel and the two locations of inteest (Enkhuizen and Rott.Hoek). 5.1 Uncertainty analysis of the physical model The probability of Z < 0 (overtopping of the Lake IJssel dikes) was calculated by a first order reliability method (FORM) in Vrijling et al. (1999). Given the uncertainties in water level, wind speed, wind surge, wave height, wave steepness, wave run-up, and lake oscillations, the uncertainties in the probability of overtopping at the two locations were determined. The results of the uncertainty calculations are summarized graphically in Figs. 7 and 7 for Rotterdamsche Hoek and Enkhuizen respectively. Notice the differences between the required crest heights for the three cases: intrinsic uncertainty, instrinsic + statistical uncertainty, and intrinsic + statistical + model uncertainty. These differences can be up to 1 metre. There has also been made a distinction between additive and multiplicative uncertainty modelling (Vrijling et al., 1999), but this distinction is not important for the next part of this paper. ISSH - Stochastic Hydraulics 2005 - 23 and 24 May 2005 - Nijmegen - The Netherlands Table 3. Distribution parameters and required crest heights. Notice that the probabilities of overtopping are given by straight lines in the above semi-logarithmic figures. This means that they can be described by exponential distribution functions: P(K<k) = 1-exp(-(k-A)/B). When the following notation is adopted: P = probability of overtopping [1/yr] i.u. = intrinsic uncertainty s.u. = statistical uncertainty add. = additive model mult. = multiplicative model m.u. = model uncertainty h = required crest height table 3 can be derived. The results of this table will serve as input for the analysis of the economic optimal dike height in the next section. 5.2 Reliability-based optimal design In the reliability-based design of hydraulic structures, the idea is to determine the total costs function. By assuming the exponential distribution (with parameters A and B) for the probabilities of overtopping, we can write: The optimal dike height follows from the minimization of the total costs function and can be expressed by the formula: and the optimal probability of failure is given by: It is interesting to notice that the optimal probability of failure is independent of the A parameter of the exponential distribution. The A values from table 1 are therefore neglected in the determination of the optimal failure probability. An increase in the slope of the exponential distribution (i.e. B>0), results in an increase in popt. From table 1, it was seen that more uncertainty results in a higher B-value. Consequently this leads to a higher optimal probability of failure. The change in the optimal probability of failure (from popt to popt’) caused by the increase in uncertainty (from B to B+B) can also be expressed as follows: Given an optimal probability of failure of 1/4000 yr-1 for Rott.Hoek, the inclusion of all uncertainties (B=0.0852) leads to a new optimal probability of failure of Instead of an economic optimal dike height of 4.70 m, a height of 4.60 m is the result. For the location of Enkhuizen, the proposed approach leads to a decrease in the dike height from 3.89 m to 3.77 m. 6 CONCLUSIONS It can be concluded that the method described in this paper is a very practical and simple method to get insight in the problems concerned with the lack of information in hydraulic engineering models. It shows that the more uncertainty is expected to be reduced in the future, the higher the mean and the larger the standard deviation of the distribution of the reliability index will be. The method is demonstrated for a simple reliability function, but the advantage is that the method also works for a very complex reliability function with a large number of variables. The difference is that the result of the model is equal to the mean of the reliability index. This creates a seeming difference between the probability of flooding computed with the ‘old’ and the ‘new’ model. This causes a problem for the transition from the water level criteria towards flooding probabilities. It is still not decided how to deal with this seeming difference. There are three possible reactions: 1 Accept the difference and do nothing. 2 Heighten the dikes in order to lower the ‘new’ probability of flooding to the ‘old’ value. 3 Do research before the transition takes place in order to reduce some uncertainties and close the gap between the ‘old‘ and the ‘new’ probability of flooding. The reliability-based decision-making procedure which has been applied in this paper can succesfully be used in the analysis of the optimal failure probabilities for the dikes along the Lake IJssel. The influence of the uncertainties lead to an increase in the probability of exceedance lines. When the hydraulic boundary conditions are modelled in an exponential way, analytical considerations can be given for the optimal probabilities of failure and the optimal dike heights. Summarised, the following conclusions can be drawn: ISSH - Stochastic Hydraulics 2005 - 23 and 24 May 2005 - Nijmegen - The Netherlands • • • • • • • • • • • • • • Extreme value theory in the most refined form is less fruitful The limited amount of data and the various sources of uncertainty have to be seen in the context of the design decision All uncertainties have to be taken into account in the design decision A method to get insight in the effect of uncertainty in hydraulic engineering problems is described. The most influential random variables are generally the ones with inherent uncertainty (this uncertainty cannot be reduced). In case of exponential distribution + normal uncertainties, a simple expression for the economic optimal probability of failure can be derived Larger location parameter leads to higher optimal design, but has no influence on the optimal probability of failure Larger scale parameter leads to smaller optimal design and higher probability of failure More options than structural reduce uncertainty by data collection or research decrease loads (μ down or σ down) increase resistance (μ up or σ down) reduce damage in case of failure Economic optimal decisions should be proposed for the height as well as the timing of the improvement 7 REFERENCES Bayes, T., 1763. An essay towards solving a probleme in the doctrine of chances, Reprinted from Philos. Trans. Roy. Soc. London. Vol. 53: 370-418. Bernoulli Family, 1999. The MacTutor History of Mathematics archive, http://www-history.mcs.stand.ac.uk/history/ Burcharth, H.F., and Liu, Z., 1994. On the extreme wave height analysis, In Port and Harbour Research Institute, Ministry of Transport, editor, International Conference on Hydro-Technical Engineering for Port and Harbor Construction (Hydro-Port), Yokosuka, Japan, 1994, pages 123-142, Yokosuka: Coastal Development Institute of Technology. Cooke, Roger M.. Experts in uncertainty; opinion and subjective probability in science. Publisher New York : Oxford University Press, 1991 ISBN 019-506465-8. Delta Committee, (1960) Considerations on storm surges and tidal waves (in Dutch), Rapport Deltacommissie, Deel 3, Staatsdrukkerij, Den Haag. Gauss, C.F., 1901. Sechs Beweise des Fundamentaltheorems über quadratische Reste (Six proofs of the fundamental theorems on quadratic residuals; in German), Publisher Leipzig : Engelmann, HRSG von E. Netto. 1901, 111 pages, Series Ostwald's Klassiker der exakten Wissenschaften Nr. 122. Goda, Y., and Kobune, K., 1990. Distribution Function Fitting for Storm Wave Data, Proceedings of the International Conference on Coastal Engineering, Delft, The Netherlands. Greenwood, J.A., Landwehr, J.M., Matalas, N.C., and Wallis, J.R., 1979. Probability weighted moments: Definition and relation to parameters of several distributions expressable in inverse form. Water Resources Research, 15(5), 1049-1054. Hosking, J.R.M., 1990. L-moments: Analysis and estimation of distributions using linear combinations of order statistics. J. R. Stat. Soc. Ser. B., vol. 52, 105-124. Jorissen, R.E., Vrijling, J.K., (1998), The transition to a risk-based concept for dike design in the Netherlands, ICCE, Copenhagen, Denmark. Klemes, Vit, 1993, Personal Communication. Mulder and Vrijling, Probabilistic determination of the load for the Oosterschelde storm surge barrier. In: Proceedings on HYDRAULIC ASPECTS OF COASTAL STRUCTURES; developments in hydraulic engineering related to the design of the Oosterschelde storm surge barrier in the Netherlands. Editors: A. Paape, J. Stuip, W.A. Venis, Delft University Press 1980, pp. 109 - 133, ISBN 906275-0540. Paté-Cornell, M.E. 1996. Uncertainties in risk analysis; Six levels of treatment. Reliability Engineering and System Safety. Nr. 54. Northern Ireland: Elsevier Science Limited. Shannon, C.E., 1949. The Mathematical Theory of Communication. The University of Illinois Press, Urbana, IL, USA. Slijkhuis, K.A.H. , M.P.C. Frijters, R.M. Cooke, A.C.W.M. Vrouwenvelder, Probability of Flooding: An uncertainty analysis, ESREL'98: European Safety and Reliability Conference 1998, pp.1419-1425, 16 - 19 June 1998, Trondheim, Norway. Van Gelder, P.H.A.J.M., and Vrijling, J.K., 1998. Homogeneity aspects in statistical analysis of coastal engineering data, In: Coastal Engineering, Vol. 26, pp.3215-3223. Van Gelder, P.H.A.J.M., J.K. Vrijling, and K.A.H. Slijkhuis, Coping with uncertainty in the economical optimization of a dike design, Proceedings of the 27th IAHR Congress, Water for a Changing Global Community, pp.554-559, San Fransisco, 1997. Van Gelder, P. and J.K. Vrijling, Risk-Averse Reliability-Based optimization of Sea-Defences, Proceedings of the 8th Engineering Foundation Conference on Risk Based Decision Making in Water Resources, pp. 61-76, October 12-17, 1997, Santa Barbara, California. Van Gelder, P.H.A.J.M., and Vrijling, J.K., 1997. A comparative study of different parameter estimation methods for statistical distribution functions in civil engineering applications, Structural Safety and Reliability, Vol. 1, pp.665-668. Van Noortwijk, J.M., P.H.A.J.M. van Gelder, Bayesian estimation of quantiles for the purpose of flood prevention, ICCE 98: 26th International Conference on Coastal Engineering, 22 - 26 June 1998, Copenhagen, Denmark. Vrijling, J.K., P.H.A.J.M. van Gelder, L. van Asperen, M. van de Paverd, R. Westphal, H. Berger, and H.G. Voortman, Uncertainty Analysis of Water Levels on Lake IJssel in the Netherlands, ICASP8*1999,Sydney, Australia, 12-15 December 1999. Vrijling J.K. and P.H.A.J.M. van Gelder, The effect of inherent uncertainty in time and space on the reliability of flood protection, ESREL'98: European Safety and Reliability Conference 1998, pp.451-456, 16 - 19 June 1998, Trondheim, Norway. Vrijling J.K. and Bruinsma J., 1980. Hydraulic Boundary Conditions. In: Proceedings on HYDRAULIC ASPECTS OF COASTAL STRUCTURES; developments in hydraulic engineering related to the design of the Oosterschelde storm surge barrier in the Netherlands. Editors: A. Paape, J. Stuip, W.A. Venis, Delft University Press 1980, pp. 109 - 133, ISBN 90-6275-0540. Vrijling, J.K. and P.H.A.J.M. van Gelder, UNCERTAINTY ANALYSIS OF WATER LEVELS ON LAKE IJSSEL IN THE NETHERLANDS: A DECISION-MAKING ANALYSIS, The 9th Annual Conference Risk Analysis: Facing the New Millen- nium, p.178-181, Rotterdam, The Netherlands, October 10 - 13, 1999. Vrouwenvelder, A.C.W.M. 1997. Project Onzekerheidsanalyse Mechanisme Overslag/ Overloop (Project Uncertainty-analysis (in Dutch). Delft: TNO 97-CON-R0935. Westphal, R., Beyer, D., Blaakman, E., 1997, Backgrounds on hydraulic loads on the dikes of Lake IJssel; Part 8: Reproduction functions (in Dutch), Ministry of Water Management, RIZA, Lelystad. Yamaguchi, M., 1997. Intercomparison of parameter estimation methods in extremal wave analysis. In B.L. Edge, editor, 25th International Conference on Coastal Engineering, Orlando, Florida, U.S.A., 1996, pages 900-913, New York: American Society of Civil Engineers (ASCE).