27th September 2012
EU River Basin Network for Agriculture
Working Group on Cost-Effectiveness
1. Background
The Water Framework Directive (WFD) requires the production of a River Basin Management
Plan (RBMP) for each River Basin District (RBD), including the compilation of a cost-effective
programme of measures.
The document entitled “Guidance for administrations on making WFD agricultural measures
clear and transparent at farm level”1, approved at the end of May 2011 by the EU Water
Directors, mentioned that information on the cost-effectiveness of suggested/required
measures was important information to share with farmers.
The PRB-AGRI report 20102 highlighting that CEA should be based on environmental costs,
resource costs and on scientific evidence. In addition, cost effectiveness should be
considered from the social point of view as well (e.g. what are the public ready to pay for
reducing leaching?).
It is well known that CEA is a complex task where the methodology and the definition of the
variables (e.g. cost) could vary significantly. The comparison of relative expenditure (costs)
and predicted outcomes (effects) often requires dealing with uncertainties. The net-effect of
an individual measure may be complex to estimate even though the direct costs may be
relatively easy to quantify.
Many approaches could be used to cope with this uncertainty, such as using
knowledge/experience from other places, expert judgement, or conducting different
simulations to understand the impact of a range of uncertain variables. Understanding how
different RBDs have tackled this complex work will help identify how they may improve
approaches in future.
2. Introduction
This article summarises the experiences and approaches used by members of the River
Basin Network for Agriculture to undertake cost-effectiveness analysis (CEA) for the first
round of Water Framework Directive (WFD) River Basin Management (RBM) Plans (2009 to
2015).
The article is intended to help share information among River Basin District (RBD) managers
and Expert Group (for Agriculture) policy makers to help improve future approaches e.g.
second cycle of RBM (2015 to 2021).
The views expressed in the article are those of the individual practitioners and do not
necessarily represent the official views or positions of individual organisations, competent
authorities or Members States for which the practitioners work. These views are shared in
good faith to help improve approaches to RBM in the future.

1
2
Why is cost-effectiveness analysis needed?
Hyperlink
http://agrienv.jrc.ec.europa.eu/publications/pdfs/EUR_24481_EN_2010.pdf
27th September 2012
Article 13 of the WFD sets out requirements for what River Basin Management Plans
(RBMPs) should contain. Specific details of what should be included in the plans is given in
Annex VII of the directive. Annex VII makes specific reference to Annex III which outlines
economic analysis shall contain enough information to make judgements about the most costeffective combination of measures to be included in the programme of measures. Further
specific references to cost-effectiveness are also made in relation to measures to control
priority substances (Article 16). Additional guidance has been provided through the EU WFD
Common Implementation Strategy (CIS) Working Group 2.6 (WATECO) Guidance Document
No. 1 ‘Economics and the Environment – The implementation challenge of the Water
Framework Directive’ (October 2003).
In addition to the fact that the WFD requires us to undertake CEA, it is incumbent upon
organisations tasked with implementing the WFD that the measures required to deliver the
objectives of the Directive achieve this in the most cost-efficient way possible.

Definition of Cost-Effectiveness Analysis
Cost-Effectiveness Analysis (CEA) can be defined as a way of comparing the costs and
effects (i.e. environmental improvement) of two or more options to determine the lowest cost
way of achieving the desired outcome.

How does it differ to Cost-Benefit Analysis?
CEA differs from Cost-Benefit Analysis in that monetary benefits for the effects (i.e.
environmental improvements) are not included in CEA. So there is no assessment of whether
benefits outweigh costs, for example as is needed in assessing whether a measure is
disproportionately costly or not.
The RBN Working Group are specifically looking at how Cost-Effectiveness was assessed in
the first round of RBM and therefore will not be looking at CBA or disproportionate cost
assessment.
3. River Basin Network Experiences
The following parts of the article have been drawn from responses from the working group.
3.1 Cost-effectiveness analysis - Methods employed by RBN members
The following text is based on survey responses from England & Wales, Scotland, Denmark,
International Commission for the Protection of the Danube River, Sweden, Italy (Arno River
Basin), Finland (SW) and the Netherlands. Additional information has been sourced from the
ACTeon Report.
3.1.1 Overview of Basic method
The majority of methods used combinations of qualitative analyses (largely “Expert
Judgement”) with some quantitative elements (i.e. nutrient load modelling). For example,
effects of measures have often been expressed quantitatively, like reductions in nutrient
loads, to help support decisions, but these were largely underpinned by expert judgement.
The benefits of specific measures, like the costs, effects, extent of measures (i.e. hectares of
land) have in most cases been deduced using expert judgement. To ensure consistency, a
number of countries have implemented mechanisms to share this information, such as
national databases. A good example of this is the “Knowledgesystem Measures” database
used in the Netherlands. This is a central system with details of costs and effects of
measures, which has been quality assured by a panel of experts. Similarly, the RAINS
database is used in the Danube to share costs data from previous projects.
27th September 2012
Quantitative approaches to assess the costs and effects of specific measures have been
adopted in some cases, particularly in relation to agri-environmental measures at the national
level. There are good examples of this used in the Danube River Basin, where the MONERIS
model has been applied to assess nutrient loads and emissions. A Strategic Planning tool has
also been used to undertake scenario analyses. In England and Wales, a quantitative
approach has been applied at the national scale to assess cost effectiveness of common
measures, known as Principal Cost Effectiveness Analysis (pCEA). A mathematical water
quality modelling programme (Simcat) has also been used to model the effects of measures,
to help inform expert judgement.
Table 1 below presents some examples of the quantitative methods applied.
Country
England & Wales
Quantitative Method
pCEA
Simcat
Danube River
Basin
MONERIS
Strategic Planning
Tool
CAPRI
Sweden
Nutrient load
reductions
Netherlands
Quantitative method
applied at national
level
Modelling Project
Scotland
Description (where available)
National database used to predict cost of
specific measures
Mathematical nutrient modelling
programme used to predict effects of
measures
Modelling programme used to assess
current nutrient emissions and loads – can
assess potential of measures to reduce
nutrient emissions.
River basin District Level – used to predict
costs for specific measures and can run
scenarios. Shows cost of measures against
reduction in load etc.
Agricultural sector model system used to
assess effects of agricultural policy
Benefits of measures expressed as nutrient
load reductions in kg/ha/yr using
quantitative methodology.
Semi quantitative programme of measures
with scale +++ ++ + - - - - - -
Running modelling project to predict
effectiveness & assessing realistic costs
of implementation.
Another common approach to identify cost effective measures has been to assume that
compliance with statutory good practice is cost effective. Handbooks on best practice
measures have been drawn up to contribute to a uniform approach nationally in some
countries (for example in the Netherlands, Denmark & Scotland).
When assessing the impacts of measures, there are differences in whether countries have
assessed the benefits beyond the objective of a specific measure. It is apparent that there are
advantages to a more integrated method, which considers a suite of options to address Good
Ecological Status, rather than addressing individual problems in isolation.
A few countries approaches have also considered the effects of measures beyond the
waterbody/ environment, for example on employment and supplying and processing
industries (e.g. Netherlands). In doing this, they have involved stakeholders as a source of
information which has served to improve their buy-in to any proposed suite of measures.
27th September 2012
In the Danube River Basin, it is reported that cultural differences between countries have
influenced stakeholder responses to measures. To address this, it is suggested that the EU
CAPRI model (agricultural sector model system to assess effects of policy) might been
adapted to potentially consider these impacts.
3.2 The geographic scale (level) of analysis
In terms of methodological changes at different scales, the majority of countries only applied
CEA at the national scale (where it is often seen that a common, consistent approach should
be adopted). Qualitative and some quantitative analyses were used, with many countries
focussing on the most commonly applied measures. This is because CEA has been seen as
an efficient way to agree cost effectiveness of common measures at the national level. It has,
however led to a perception that this is a “top down” approach by some stakeholder groups.
Those that did extend the analysis regionally / locally tended to use qualitative analyses at
this level (largely expert judgement), particularly as the measures applied were often more
bespoke. Some limited quantitative modelling was used to support this where available. This
is the case in both England & Wales and in the Netherlands.
3.3 Data considerations
Data availability and/or consistency were highlighted as a particular concern in nearly all
countries. Information gaps on the costs and effectiveness of measures beyond the local
scale are an issue. Impacts of diffuse pollution measures are also cited as an issue, as are
impacts of measures in “typical conditions” i.e. not laboratory. To address this, a number of
countries have developed or are developing central databases on the cost and effectiveness
of measures. This is intended to improve national consistency and ultimately the reliability of
CEA.
For example, England and Wales are building a central database of costs and effectiveness
of measures, costs data for the Danube River Basin have been drawn from previous projects
undertaken and/ or from European databases (i.e. RAINS project) and the Netherlands has a
central database on the costs effects of measures, known as Knowledgesystem Measures.
Teams with excellent knowledge of particular measures can share this more easily, thus
improving consistency. Quality assurance is provided through workshops and expert review.
The Netherlands has also drawn up a handbook on cost effectiveness for WFD to ensure that
the methods and details in various regions link up well, and that comparisons can be made.
Confidence in the data has been cited as a concern by many countries. To mitigate this, a risk
based approach to data quality has been applied when implementing measures in many
cases. For instance, where expensive measures are being required, significant research and
investigations have often been undertaken to improve the quality and certainty of the data, to
mitigate the risk. Where lower cost measures have been implemented, lesser data quality has
been accepted.
All countries agree that the confidence of any data underpinning CEA must be made clear, so
that decisions/ measures can better consider risk.
3.4 Tools to help undertake Cost-Effectiveness Analysis
Where quantitative modelling approaches have been adopted to support CEA, a number of
tools have been used to help with this. These include mathematical water quality modelling
tools and national databases. Please see Table 1 for more details.
27th September 2012
3.5 Communicating the results to support implementation of measures
The approach to stakeholder communication varies. One approach has been to inform
stakeholders of the results of the CEA and to inform them of the necessary measures and
why. This has had the negative result of reducing stakeholder acceptance of the required
measures.
However, expert judgement is acknowledged to be dependant on good social engagement in
some countries. It is agreed that this can be time consuming and resource intensive. There
has been a focus on specific sector groups, who are most impacted by measures, particularly
in the agricultural sector. Problems experienced include difficulty to get buy in, lack of
understanding and criticism of approach.
It has also been noted that some countries, who adopted a national approach were seen to
be implementing measures using a “top down” philosophy, using complicated and difficult to
understand methodologies. This significantly reduced transparency and thus stakeholder buyin.
As a lesson learnt from the first round of CEA, early engagement at all levels (particularly
participatory) is acknowledged as a necessity, using tools such as workshops and visits by a
number of countries. In England and Wales, a series of catchment pilot projects have been
initiated to test how best to undertake stakeholder engagement.
3.6 Common difficulties and opportunities/priorities for future action
Common difficulties, as outlined previously centre around stakeholder buy-in to proposed
measures as well as availability of data on cost effectiveness of measures. A number of
solutions have been proposed, including national databases and improved data transparency.
3.7 ACTeon Report
The aim of the ACTeon report was to “establish evaluation and overview of application of cost
effectiveness analysis in implementation of WFD in France and in Europe”. The paper
focuses on CEA use, its role in decision making and the main methods used.
It suggests that a NW/ SE divide exists, with countries in the NW more advanced in their use
of CEA, with more developed databases and methods etc. However, this survey has
suggested that some countries in the SE in fact have reasonably developed methods and
databases, for example within the Danube River Basin.
Lack of data for CEA is highlighted as an issue, which is supported by this present survey.
The report also proclaims that “CEAs do not often refer to stakeholder participation”. Whilst
this is the case for some countries, we have in fact uncovered a reasonable degree of actual
stakeholder participation or at least the recognition it will be an important factor for the next
round of River Basin Management Plans.
From 48 studies, the ACTeon report suggests that at least 2/3 of CEA is addressed at the
agricultural sector. This is supported by this present survey, in which the majority of measures
have been seen to be addressing nutrient levels.
In summary, this present survey largely supports the findings of the ACTeon report, but has
unearthed an extra level of detail from which we can get a better understanding of different
countries’ approaches to WFD CEA.
4.0 Recommendations
27th September 2012
It has been acknowledged by some countries that tools developed for local practitioners to
use to undertake CEA were too technical and required economic knowledge. For the next
CEA, it has been advised that a less detailed and more qualitative approach would be more
easily applied at the local level and would improve stakeholder understanding and buy-in.
All countries agree on the need for improved stakeholder buy-in. A variety of methods have
been proposed, but early engagement and simple approaches which can be easily
understood are commonly accepted as needed to increase acceptance.
Risk based analyses are also commonly acknowledged as a worthy solution, particularly in
relation to data quality as outlined previously.