Uncertainty Analysis in Integrated Assessment Models:
A Review of Recent Approaches
Silke Gabbert1, Carolien Kroeze1, Leen Hordijk1, John van Aardenne2
1
Environmental Systems Analysis Group, Wageningen University, P.O. Box 9100,
6700 HB Wageningen, The Netherlands
2
Max Planck Institute for Chemistry, Atmospheric Chemistry Division, P.O. Box 3060,
55020 Mainz, Germany
Abstract
The paper reviews the current state of research with regard to uncertainty analysis in
Integrated Assessment Models. The critique concerning previous treatment of uncertainty is
summarised and the three most important frameworks that have been developed for a more
systematic uncertainty analysis are outlined. A comparison of these frameworks with regard
to their applicability to Integrated Assessment Models for acidification is the focus of the
paper.
Keywords: Integrated Assessment models, uncertainty analysis, acidification
1 Introduction
Integrated Assessment Models (IAMs) aim at providing tools to analyse complex real world
problems and to portray their social, economic, environmental and institutional dimensions.
IAMs can have several functions. They can serve as early warning systems, policy evaluation
tools, and as tools for public negotiations and decision-making. According to the multidimensionality of the problems addressed, IAMs are subject to various uncertainties due to,
for example, variability of parameters, inaccuracy of model specification or lack of
knowledge with regard to model boundaries. Although the existence of uncertainties has been
recognised early in the process of developing IAMs, in many of them uncertainty analysis is
included only partially or still not at all.1 This has caused increasing dissatisfaction both
among scientists and policy-makers. Motivated by the awareness of the inadequacy of
deterministic models for problems such as climate change or air pollution, and on the wish to
improve confidence in model results it has repeatedly been pointed out that frameworks for a
systematic analysis of uncertainties in IAMs are necessary. Responding to this need, several
approaches have been proposed during the past five years in order to fill this gap. Among
these, the most important are the approach based on the NUSAP method, developed by van
der Sluijs (1997), the PRIMA approach by van Asselt (2000) and the FRAULEIN approach
by van Aardenne (2002). However, each of these frameworks has been developed for and
applied to a specific problem context. Given the heterogeneity of IAMs the question arises if,
and to what extent they can be generalised and applied integrated assessment problems other
than those for which they have been constructed.
1
In this paper the term “uncertainty analysis” includes all methodologies for qualitatively or quantitatively
assessing the uncertainty in model output induced by uncertainty in inputs.
The aim of this paper is therefore to conduct a comparative review of NUSAP, PRIMA
and FRAULEIN and to critically discuss strengths and weaknesses of these frameworks from
a research economics perspective.
The paper is organised as follows: In section 2, we present a classification of IAMs.
Then, we summarise different strands of critique with regard to the past treatment of
uncertainties in IAMs and we survey the three most important approaches that have been
developed in recent years in order to ensure a more systematic treatment of uncertainty
(section 3). Section 4 concludes with a discussion of the applicability of these approaches to
IAMs for acidification.
2 Classifying IA Models
IAMs can be considered tools for the evaluation of policies to solve complex problems in the
human society. Environmental problems have been subject to IA modeling for several
decades. In particular, IAMs have been developed for air pollution problems with an
international focus, including climate change and acidification. These environmental
problems have an international impact, and therefore require international solutions. Ideally,
IAMs for air pollution control cover the whole range of the problem from emissions of
pollutants to their environmental effects. They typically answer questions of the "what if-"
type and are used to evaluate policy strategies. By defining different scenarios for human
activities, the models explore a variety of possible future developments, thus assisting
decision-makers in choosing between alternative strategies. In scenario analysis, models can
be used to calculate the environmental impact and control costs for a given set of assumptions
on future trends in human activities. In optimisation analysis, models are typically used to
analyse how certain environmental targets can be met at minimum costs.
Here we focus on IAMs for acidification. In an earlier study Hordijk and Kroeze (1997)
reviewed 20 IAMs for acid rain. In their overview they only include models that at least cover
three of the following five components (i) sources of emissions, (ii) atmospheric transport and
transformations, (iii) environmental effects, (iv) abatement and mitigation options and (v)
monetary evaluation. Hordijk and Kroeze (1997) concluded that thorough sensitivity and
uncertainty analyses have seldomly been performed, which is surprising, given the still
existing uncertainties in many model components.
Among the IAMs for acidification, only a few (three models for Europe, i.e. ASAM,
CASM and RAINS; and two for North America, i.e. Canada IAMS-RAISON and Ellis) have
been used by policy makers and are still under development. These models are shown in
Table 1. In the table the models are characterised according to the research domain, the model
components selected, the methodology on which the model is based, the spatial and the
temporal resolution of the model, the area for which the model has been used in practise and
whether or not the model includes some kind of uncertainty analysis. Of the models selected,
only RAINS is used on a regular basis for policy purposes at present.
The models shown in Table 1 all include information on emissions, atmospheric
processes, environmental impacts and costs of emission control.2 They can therefore be used
to evaluate possible strategies to reduce acidification in North America or Europe. Of these
models, only three (Ellis, IAM-Raison and RAINS) include some uncertainty analysis. The
objectives of the uncertainty analysis and the methods applied are explained in more detail in
the following section (Table 2).
2
The tables will be included in the full version of the paper.
3 Uncertainty analysis in IAMs for acidification: an overview
According to Gardner et al. (1988) the usefulness of any given model depends on the
accuracy and reliability of the prediction. Accuracy and reliability are dependent on
knowledge, information on the process being studied and the translation of processes in
mathematical formulation. Because of the complexity of IAMs for acidification the
knowledge and information on the processes that are being studied is, in general, limited and
to a large extent uncertain. Following Alcamo and Bartnicki (1987), uncertainties in IAMs
occur because of
 uncertainty in model structure, resulting from a specified collection of model terms and
their interrelations;
 uncertainty in model parameters, resulting from the specification of coefficients which can
either be constant or change in time or space;
 uncertainty arising from lack of knowledge of model boundaries.
In order to determine the usefulness of IAMs as assessment and decision-making tools,
an assessment of these uncertainties is inevitable. This requires firstly defining a consistent
classification system by means of which uncertainties in IAMS can be identified. Secondly,
qualitative and quantitative methods for analysing the degree of uncertainty and their
influence on model output can be applied. There are different ways to deal with uncertainties
in IAMs. In general, a distinction can be made between deterministic and stochastic modelling
approaches. Stochastic models explicitly account for uncertainties in model components,
usually in terms of probability frequencies, while deterministic models do not. The
uncertainties in deterministic models may be assessed by sensitivity analyses or quantitative
uncertainty analysis for which different techniques are available. However, among the six
IAMs for acidification that we considere to be policy relevant, only the one developed by
Ellis (1990) and Watanabe and Ellis (1993, 1994) has been constructed as a stochastic model
(Table 2). The remaining five models are deterministic and only two of them (IAM-RAISON
and RAINS) have been subject to partial quantitative or qualitative uncertainty analysis. The
techniques used for this purpose are summarised in Table 2.
As the table demonstrates, uncertainty analysis in IAMs for acidification has been
constrained to a few quantitative methods. In recent years this has repeatedly been criticised
(Hordijk and Kroeze 1997, 413; Cocks et al. 1998, 639; Rotmans 1998, 168; Potting et al.,
2001, 4). One of the key arguments raised is that methods such as Monte Carlo techniques
and sensitivity analyses capture only parts of a multidimensional problem. Thus, these
techniques can provide only limited information about uncertainties in IAMs. Furthermore,
quantitative techniques can be of limited applicability. For example, a Monte Carlo based
analysis of uncertainty for each input parameter in IAMs is often not possible because of
constraints in time and computational resources. In order to conduct such analysis, a number
of assumptions, for example with regard to the distribution of parameters, have to be made
which can introduce additional uncertainties. Looking at the studies mentioned in Table 2, the
choice of a certain methodology for uncertainty analysis is often poorly motivated. For the
RAINS model, for example, Suutari et al. (2001) estimated coefficients of variation for
individual model parameters. They considered previous attempts to assess uncertainty in
RAINS “to provide only partial answers” to uncertainty. However, it remains unclear what
comparative advantages their method has in comparison to the previous ones.
Summing up, we can conclude that uncertainties have not only been insufficiently
addressed in the past but also that existing work on this topic seems to be conducted
unsystematically. Given the fact that uncertainty has become a key issue in IAMs, scientists
pointed out that developing a framework for a more systematic and comprehensive analysis of
uncertainty is necessary.
In recent years, three different approaches have been proposed for uncertainty analysis
in environmental modelling in order to respond to this need. These are
 a framework for uncertainty management based on the NUSAP method, developed by van
der Sluijs (1997);
 the PRIMA approach developed by van Asselt (2000); and
 the FRAULEIN approach developed by van Aardenne (2002).
A common element of these studies is that they are based on a two-dimensional
classification system of uncertainties where (a) sources and (b) types of uncertainties are
distinguished. However, the studies differ with respect to the definition of sub-categories, the
objectives stated for uncertainty assessment and the frameworks developed in order to reach
these objectives.
Van der Sluijs’ aim is to provide tools for a systematic assessment of uncertainties in
IAMs of anthropogenic climate change. By combining the sources and types of uncertainty
(Table 3) he defines different areas for uncertainty assessment (van der Sluijs 1997, 179ff.
and 1995, 33ff.). Looking at the tools that have in fact been applied for uncertainty analysis in
IAMs, he concludes that research activities have concentrated mainly on uncertainty due to
inexactness in input data and model parameters whereas all the other areas have received
much less attention so far (van der Sluijs 1997, 206).
In order to reduce this mismatch, van der Sluijs states that uncertainty assessment
should be embedded in a more comprehensive analysis of the quality of model output. For
this purpose he applied the NUSAP method developed by Funtowicz and Ravetz (1990, 183).3
The core element of this method is to assess the quality of any model parameter by means of a
diagnostic diagram. Such diagram allows for attaching a certain quality level (ranging from
poor to high) to each model parameter. The quality level depends (i) on the parameter’s
relative contribution to the spread in model output (Y-axis score, determined by means of
sensitivity and Monte-Carlo analysis) and (ii) its strength, i.e. its qualitative underpinning (Xaxis score). The latter can be obtained by asking experts in a panel to attach scores to defined
evaluation criteria of model parameters.
For the construction of a diagnostic diagram van der Sluijs (1997, 207ff) developed a
10-step iterative framework. The procedure starts with the selection of experts to form a
panel. Further steps include the selection of model constituents and the choice of appropriate
evaluation criteria. As a final step, the Y- and X-axis scores obtained for each parameter are
mapped in the diagnostic diagram.
Apart from evaluating overall model performance, the framework is also considered to
allow for an identification of the most uncertain parts in an IAM which should therefore be
given priority when examining possibilities for model improvements. It is also assumed to be
a useful tool in order to distinguish between solvable and unsolvable uncertainties.
With some modifications the NUSAP method has been applied to identify and assess
key uncertainties in the so-called TIMER model, which is part of the IMAGE model, an IAM
3
For a more detailed description of the approaches presented in this section cf. the long version of the paper.
for analysing climate change (van der Sluijs et al.2002).4 In another application NUSAP was
used for the assessment of the quality and uncertainty of Volatile Organic Compounds
emission figures (van der Sluijs et al. 2001).
Similarly to van der Sluijs, van Asselt and Rotmans (2001, 50) argue that uncertainty
analysis in integrated assessment modelling has failed to cover the whole range of
uncertainties. Since research has been focusing mainly on technical uncertainties, no methods
have been developed for examining certain aspects of variability (e.g. technological surprise)
and uncertainties in model structure. Furthermore, the use of qualitative methods for
uncertainty analysis has largely been neglected. Thus, methods currently available provide
only very limited knowledge about uncertainty in IAMs and even ignore those which the
authors consider to be most fundamental. Given the multidimensionality of uncertainties they
propose a “pluralistic framework for integrated uncertainty management and risk analysis”
(PRIMA). PRIMA is defined as “a participatory assessment process in which the salient
uncertainties are interpreted according to different perspectives and structured into multiple
risk judgements in such a way that robust insights can be deducted” (van Asselt 2000, 228).
The objective of PRIMA is to provide tools that allow for an examination of uncertainties
from a number of different directions. Thus, the authors assume the flexibility of PRIMA with
respect to future projections of IAMs, the focus of PRIMA on key uncertainties and the
inclusion of formerly neglected uncertainties to be the main comparative advantages of this
approach (Rotmans and van Asselt 2000, 53).
The choice of key uncertainties in an IAMs depends on from which view the world
(including nature and society) is interpreted (i.e. the choice of a “perspective”). Based on
Cultural Theory, Rotmans and van Asselt (2001, 52) suggest to use four different types of
perspectives (hierarchist, egalitarian, individualist and fatalist). The selection of uncertainties
will differ according to the choice of one of these perspectives. The next step within the
PRIMA framework is to specify “model routes”, i.e. to create model formulations for each of
these perspectives and the perspective-based selection of uncertainties (Rotmans and van
Asselt 1996, 124ff). Experimenting with the model routes is assumed to lead to multiple
model scenarios, which can then be assessed by, for example, comparing them with previous
IAM results. In order to test the PRIMA framework in a practical context, it has first been
applied to examine how uncertainties were treated in the “Environmental Outlooks”, a
scientific series published by RIVM. In addition, the first two steps of PRIMA (definition of a
“perspective on pluralism” and definition of a “perspective on uncertainties”) have been
applied in a RIVM workgroup (van Asselt 2000, 354ff).
A third framework that should be discussed has been developed by van Aardenne
(2002). In contrast to van der Sluijs and van Asselt, he focuses on a very specific topic, i.e.
uncertainties in large-scale emission inventories. Such inventories provide information about
the quantities of certain pollutants emitted in the atmosphere. They are used both as
instruments for monitoring progress of environmental policies as well as inputs into
atmospheric dispersion models, which themselves are part of several IAMs for air pollution
(for example the RAINS model). In emission inventories, uncertainties arise mainly because
of inaccurate representation of “true” emissions of pollutants and because of unreliability,
which is defined as “lack of knowledge of the degree to which the inventory is meeting user
specific quality criteria” (van Aardenne 2002, 45; Table 3). Inaccuracy and unreliability of
4
The IMAGE model (Integrated Model to Assess the Global Environment) has been developed by the
National Institute of Public Health and the Environment (RIVM, the Netherlands). The model has mainly
been used for the development of emission scenarios for policy decision-making, for example within the
Intergovernmental Panel on Climate Change (IPCC) (van der Sluijs 2002, 137).
emission inventories depend to each other in the sense that lack of reliability can be caused by
inaccuracy.
Based on this typology, the aim of van Aardenne’s study is to develop an approach for
systematically assessing uncertainties in emission inventories in order to evaluate potential
improvements with respect to its reliability and accuracy. This includes first identifying
sources of inaccuracy and unreliability. The sources of unreliability depend on the specific
quality criteria set by the users of emission inventories. Since they can differ across
inventories they have to be identified for each case of an inventory. Sources of inaccuracy, on
the contrary, can be generalised better and grouped into structural inaccuracy and input value
inaccuracy.
The framework suggested to assess the different sources of uncertainties (FRAULEIN –
Framework for the Assessment of Uncertainties in Large-scale Emission Inventories) consists
of two paths. Path A focuses on the analysis of uncertainties due to unreliability which is
checking whether or not the quality criteria specified for a certain inventory have in fact been
met. Path B refers to the analysis of uncertainties due to inaccuracy. Here, van Aardenne
(2002, 55ff) presents an overview of tools available for external (i.e. by using other
independent information) and internal assessment of uncertainties. For the latter a large
number of techniques is available ranging from qualitative discussion and expert judgement to
error propagation (including, for example, Monte Carlo analysis). In contrast to NUSAP and
PRIMA the FRAULEIN approach includes a complete overview of the tools and techniques
available for the analysis of uncertainties. Based on the purpose of the uncertainty analysis at
stake, researchers may select one or more of these tools to analyse uncertainties in emission
inventories. It should be noted that FRAULEIN does not indicate priorities within the group
of methodologies described for uncertainty analysis.
4 Conclusions and discussion of needs for further research
The treatment of uncertainty in IAMs has gained increasing attention during the past years.
Motivated by sustaining dissatisfaction concerning the way uncertainties have been addressed
in IAMs, substantial progress has been made in developing frameworks in order to identify
and analyse uncertainties more systematically. Looking at the frameworks proposed, we can
conclude the following:
 The three approaches described in the previous section have been developed from different
research perspectives. NUSAP and PRIMA both have a philosophical foundation, which
van der Sluijs (1997, 20) describes as “post-normal science”. The main characteristics of
this way of doing research are the multi-perspectivity of approaches used, the treatment of
uncertainties as endogenous elements and the involvement of stakeholders at any level of
the analysis. In contrast to this, van Aardenne’s work is rather technocratic, focusing
exclusively on the technical and statistical dimension of uncertainty assessment.
 According to the research perspectives taken, the (expected) outcomes of the frameworks
differ considerably:
 The framework suggested by van der Sluijs allows for a quality assessment of model
parameters. The aim is to identify most uncertain parameters and to set priorities among
them guiding further work for improving the model of concern.
 PRIMA is focusing on uncertainty management rather than on assessment. The main
innovative element is to filter key uncertainties according to an initial choice of a
“perspective” on uncertainties. Given the selection of uncertainties a range of different
model outcomes can be produced, which have then to be further interpreted and
analysed.
 Van Aardenne systemises the range of possibilities for uncertainty analysis in emission
inventories. This can facilitate the choice of an appropriate methodology for uncertainty
analysis, given an initial decision on what type and source of uncertainty should be
examined.
Especially NUSAP and FRAULEIN have been developed for specific problem contexts (i.e.
integrated assessment of anthropogenic climate change and large-scale emission inventories,
respectively). Although PRIMA does not explicitly focus on a certain type of integrated
assessment activity, its application has so far been constrained on an IAMs of global change
and sustainable development (TARGETS). We can therefore conclude that an approach for
the type of IAMs we are focusing on in this paper, i.e. policy relevant IAMs for acidification,
still does not exist.
However, particularly the NUSAP and the PRIMA approach are not restricted to the problem
context for which they initially have been developed and applied. To the contrary, the authors
underline the general usefulness of their frameworks for any type of integrated assessment
problem (van der Sluijs et al. 2001, 10; van Asselt 2000, 414). Although FRAULEIN is
explicitly focused on emission inventories, the structure of this approach might also be
transferable to other integrated assessment problems. It is therefore worth discussing if, and to
what extent, these frameworks could be applied to IAMs of acidification. Here we would like
to stimulate the discussion about uncertainty analysis in IAMs through the following principal
considerations:

The key motivation for the design of IAMs of acidification such as RAINS, ASAM and
CASM was to provide scientific backing to the political negotiation process for emission
reduction under the Convention on Long Range Transboundary Air Pollution (CLRTAP).5
One of the main functions of such models is therefore to provide decision support to
policy-makers in order to identify the “best” abatement strategy subject to, for example,
national budget constraints. Thus, making uncertainty analysis be an integral part of such
models requires policy-makers to participate in this process. The need for a better
involvement of policy actors in the design and use of IAMs has repeatedly been pointed
out in the literature (Schneider 1997, 235). One way how this could be done is to make
policy-makers become integrated assessors by asking them to express their preferences for
uncertainty analysis in IAMs.6
Among the frameworks for uncertainty analysis considered in this paper, NUSAP and
PRIMA implicitly account for this need. However, looking at the case studies where these
approaches have been applied, policy-actors have in fact not been included into the
analysis. Since FRAULEIN is designed as a purely technical approach, it does not
consider the inclusion of stakeholders.

The IAMs for acidification shown in Table 2 are fairly complex and technical models.
Uncertainty analysis can help to improve the quality of and the confidence in the models,
but these benefits have to be weighed against the costs of uncertainty analysis, arising
5
The CLRTAP was signed 1979 in Geneva by 34 Governments and the European Community.
6
One step in this direction is a study conducted by Gough (1999, 470) where representatives from several
Directorates of the European Commission were interviewed. One of the results stated is that the most
frequently noted uncertainties recognised by policy-makers are lack of data, lack of knowledge regarding
cause-effect links in IAMs, and institutional uncertainties (incomplete or inappropriate understanding of the
institutional context in which a policy will be applied).
mainly through increased research time needed. Thus, uncertainty analysis has clear
economic dimensions. We therefore propose to focus attention to the question how
uncertainty analysis can be done efficiently. In this context none of the three approaches
described seems to be directly applicable. Van der Sluijs and van Asselt explicitly
mention the high requirement of time for application as one of the main shortcomings of
their approaches (van der Sluijs et al. 2001, 3; van Asselt 2000, 260 and 413). Applying
PRIMA to IAMs for acidification would mean to define for each uncertain parameter (and
for each combination of uncertain parameters) different values and to compute the model
for each of these specifications. Keeping in mind that of the models mentioned in Table 2
many (for example the RAINS model) include several hundreds of parameters, this
procedure can easily cause work for years. Such approach might be very informative, but
it is questionable whether it is in fact supportive for decision-making purposes. Similar
doubts can be stated with regard to FRAULEIN: Since IAMs for acidification are much
more complex than emission inventories, a complete statistical analysis of uncertainties in
IAMs based on the methodologies suggested by van Aardenne would require immensely
time-consuming.

Because of the resource constraints explained above, one of the main challenges for an
efficient uncertainty analysis in IAMs for acidification is to define reasonable criteria in
order to select the most relevant (types and sources of) uncertainties. NUSAP and PRIMA
both suggest methodologies how this problem can be solved. Within PRIMA, the
selection of uncertainties depends on the initial choice of a perspective on uncertainty.
However, it remains unclear how priorities among uncertainties chosen within a given
perspective can be set. By means of the diagnostic diagram, the NUSAP approach
provides - at least for uncertainties in model parameters - a clear guidance for
prioritisation of uncertainties: The parameters shown in a diagnostic diagram can not only
be ranked according to their quality, it is also possible to identify which quality aspect of a
parameter has to be improved in order to strengthen its overall quality. The FRAULEIN
approach, on the contrary, does not intend to give a solution. Before this framework can
be used, the analyst has to decide what type of uncertainty (unreliability or inaccuracy, see
table 3) shall be examined. Given this decision, the further selection of methodologies
depends again on the judgement of the analyst. Thus, listing methodologies available for
the analysis of different types of uncertainties, FRAULEIN addresses the analytical step
following to the selection of uncertainties.
From a research economics perspective (costs and benefits of uncertainty analysis in IAMs)
the NUSAP approach seems to be most promising since it allows for the inclusion of
stakeholders and attempts a prioritisation of parameters. We regard both aspects to be
essential requirements for a more systematic analysis of uncertainty in IAMs for acidification
and for improving their practical usefulness.
5 References
ApSimon, Helen M.; Warren, Rachel F.; Wilson, J.J.N. (1994): The Abatement Strategies
Assessment Model – ASAM: Applications to Reductions of Sulphur Dioxide Emissions
Across Europe. Atmospheric Environment 28 (4), 649-663.
ApSimon, Helen M.; Warren, Rachel F. (1996): Transboundary Air Pollution in Europe.
Energy Policy 24 (7), 631-640.
Bailey, Peter D.; Gough, Clair M.; Millock, Katrin; Chadwick, Michael J. (1996): Prospects
for the joint implementation of sulphur emission reductions in Europe. Energy Policy 24 (6),
507-516.
Bailey, Peter D. (1995): Modelling Future Vehicle Exhaust Emissions in Europe. Water, Air
and Soil Pollution 85, 1879-1884.
Cocks, A. T.; Rodgers, I. R.; Skeffington, R. A.; Webb, A. H. (1998): The limitations of
integrated assessment modelling in developing air pollution control policies. Environmental
Pollution 102 (S1), 635-639.
Ellis, J. Hugh (1990): Integrating multiple-long-range transport models into optimisation
methodologies for acid rain policy analysis. European Journal of Operational Research 46,
313-321.
Funtowicz, Silvio O.; Ravetz, Jerome R. (1990): Uncertainty and Quality in Science for
Policy. Kluwer Academic Publishers, Dordrecht, 229 pp.
Gardner, R. H., Dale, V .H., O’Neill, R. V. (1988): Error propagation and uncertainty in
process modelling (1988). In: Proceedings of Conference on Forest Growth: Process
modelling of response to environmental stress, gulf shores conference, April 19-22, 1988.
Timber Press, Auburn University, Alabama.
Gough, Clair A.; Bailey, Peter D.; Biewald, B.; Kuylenstierna, J. C. I.; Chadwick, M. J.
(1994): Environmentally targeted objectives for reducing acidification in Europe. Energy
Policy 22 (12), 1055-1066.
Gough, Clair A.; Chadwick, M. J.; Biewald, B.; Bailey, P. D.; Kuylenstierna, J. C. I.; Bailey,
Peter D.; Cinderby, S. (1995): Developing Optimal Abatement Strategies for the Effects of
Sulphur and Nitrogen Deposition at European Scale. Water, Air and Soil Pollution 85, 26012606.
Gough, Clair A. (1999): Quality and practice in integrated assessment: some users’
perspective. International Journal of Environment and Pollution 11 (4), 462-478.
Hordijk, Leen; Kroeze, Carolien (1997): Integrated assessment models for acid rain. European
Journal of Operational Research 102, 405-417.
Lam, David C. L.; Howell, G.; Swayne, D. A.; Fraser, A. S.; Storey, J.; Wong, I. (1989): A
Knowledge-Based Approach to Regional Acidification Modelling: A Case Study of New
Brunswick, Canada. In: Maemari et al. (eds.): Regional Acidification Models. SpringerVerlag, Berlin, 213-225.
Lam, David C. L.; Puckett, K. J.; Womg, J.; Moran, M. D.; Fenech, G.; Jeffries, D. S.; Olson,
M. P.; Whelpdale, D. M.; McNicol, D.; Mariams, Y. K. G.; Minns, K. (1996): An Integrated
Acid Rain Assessment Model for Canada using the RAISON for Windows System. Paper
presented at the ASAAQ Conference 1996, Seattle.
Suutari, Riku; Amann, Markus; Cofalfa, Janusz; Klimont, Zbigniew; Posch, Maximilian;
Schoepp, Wolfgang (2001): From Economic Activities to Ecosystem Protection in Europe.
An Uncertainty Analysis of Two Scenarios of the RAINS Integrated Assessment Model.
CIAMS/CCE Report 1/2001, International Institute for Applied Systems Analysis,
Laxenburg, Austria.
Rotmans, Jan (1998): Methods for IA: The challenges and the opportunities ahead.
Environmental Modeling and Assessment 3, 155-179.
Rotmans, Jan; van Asselt, Marjolein B. A. (2001): Uncertainty in assessment modelling: A
labyrinthic path. Integrated Assessment 2, 34-55.
Schneider, Stephen H. (1997): Integrated assessment modeling of global climate change:
Transparent rational tool for policy making or opaque screen hiding value-laden assumptions?
Environmental Modeling and Assessment 2, 229-249.
Van Aardenne, John A. (2002): Uncertainties in emission inventories. PhD thesis,
Wageningen University, 100 pp.
Van Asselt, Marjolein B. A. (2000): Perspectives on Uncertainty and Risk. The PRIMA
Approach to Decision Support. Kluwer Academic Publishers, Boston, 434 pp.
Van Asselt, Marjolein B. A.; Rotmans, Jan (1996): Uncertainty in perspective. Global
Environmental Change 6 (2), 121-157.
Van der Sluijs, Jeroen P. (1995): Integrated Assessment Models and the Management of
Uncertainties. WP-96-119, International Institute for Applied Systems Analysis, Laxenburg,
Austria, 80 pp.
Van der Sluijs, Jeroen P. (1997): Anchoring amid uncertainty. On the management of
uncertainties in risk assessment of anthropogenic climate change. CIF-Gegevens Koninklijke
Bibliotheek, Den Haag, The Netherlands, 260 pp.
Van der Sluijs, Jeroen P.; Risbey, James S.; Ravetz, Jerry (2001): Uncertainty Assessment of
VOC Emissions from Paint in the Netherlands. Copernicus Institute for Sustainable
Development and Innovation, Department of Science Technology and Society, Utrecht
University, the Netherlands, 27 pp.
Van der Sluijs, Jeroen P. (2002): A way out of the credibility crisis of models used in
integrated environmental assessment. Futures 34 (1), 133-146.
Van der Sluijs, Jeroen P.; Potting, Jose; Risbey, James; van Vuuren, Detlev; de Vries, Bert;
Beusen, Arthur; Heuberger, Peter; Quintana, Serafin Corral; Funtowicz, Silvio; Kloprogge,
Penny; Nuijen, David; Petersen, Arthur; Ravetz, Jerry (2002): Uncertainty assessment of the
IMAGE/TIMER B1 CO2 emission scenario, using the NUSAP method. Report no. 410 200
104 (2002), Dutch National Research Programme on Global Air Pollution and Climate
Change, 225 pp.
Watanabe, Tsunemi; Ellis, J. Hugh (1993): Robustness in stochastic programming models.
Applied Mathematical Modelling 17, 547-554.
Watanabe, Tsunemi; Ellis, J. Hugh (1994): A joint chance-constrained programming model
with row dependence. European Journal of Operational Research 77, 325-343.