26 June 2015
Ex post Evaluation of State Aid Measures
Phedon Nicolaides
Professor at the College of Europe
and at the University of Maastricht
Decentralisation and ex post state aid control
One of the pillars of the Commission’s State Aid Modernisation initiative is the
decentralisation of responsibility for the implementation of state aid measures. The
decentralisation works by allowing Member States to adopt and implement measures
without first having to notify them to the Commission. Fewer notifications mean that
Commission services can concentrate their efforts in identifying and assessing measures
which are more likely to cause a serious distortion of competition.
Decentralisation, then, should improve the effectiveness of state aid control and, by
implication, the functioning of the internal market.
But, every policy measure has its costs which cannot be ignored. While decentralisation
allows the Commission to focus its resources and assessment, it may also tempt Member
States to grant aid in amounts, forms and for purposes not permitted by state aid rules. For
this reason, the SAM has also introduced safeguards through more extensive ex post
control.
This ex post control is based on three types of action:
 increased transparency,
 wider monitoring of implemented measures, and
 evaluation of large or novel measures.
The procedures for increasing transparency will eventually be put in place only as of 2016.
At that time, Member States will have to publish all individual awards of aid that exceed
EUR 0.5 million. The Commission hopes that the publication of awards and the names of
beneficiaries will make it easier for competitors to spot incompatible aid and either initiate
domestic legal proceedings or inform the Commission.
Wider monitoring will take the form of checks by the Commission of a larger sample of
national aid measures. As Member States are submitting information sheets to the
Commission concerning the measures that are adopted on the basis of the GBER, what is
known is that the Commission is already earmarking the measures that it intends to subject
to ex post monitoring. The Commission has not revealed the criteria it uses to select
measures for monitoring and probably it will never make them public. Auditors, for
example, in order to prevent audited companies from hiding irregularities in their accounts
choose a random sample of transactions for individual checks. It is unlikely that the
1
Commission will choose randomly samples of national measures. It is more likely that they
will concentrate on larger and more complex measures.
The third safeguard is ex post evaluation. Strictly speaking this is not about potential misuse
of aid. It is more about the potential waste of public resources through ineffective but
legally compatible measures. The Commission has been criticised by the European Court of
Auditors for not examining the effectiveness of the measures which it approves. By not
enquiring whether state aid achieves its objectives, the Commission cannot know whether
the aid it considers to be compatible with the internal market is in fact capable of achieving
the public policy objectives which are defined in the EU Treaty.
It should be recalled that although the Treaty prohibits in principle state aid, it exceptionally
allows certain types of aid for the purpose of achieving certain public policy objectives.
These policy objectives correspond to economic conditions that cannot be brought about by
the normal functioning of the market. But if Member States or the Commission do not
measure the effects of the state aid measures they implement, how can anyone know
whether approved aid or legally compliant aid supports public policy, as the Treaty and the
case law require?
Ex post evaluation
Therefore, the Commission, belatedly in my view, has begun to request Member States to
carry out ex post evaluations of three types of measures:
 measures with large budgets,
 measures with novel features, and
 measures that can impact on competition because market conditions or technology
change fast.
In order to help Member States, the Commission has also adopted guidelines on the precise
aims and timeline of evaluation. The guidelines also provide a review of various evaluation
methodologies.
It appears that so far the measures that have been are approved on condition that they are
evaluated ex post are those for which the GBER makes the evaluation unavoidable. The
GBER provides that schemes with average annual budgets that exceed EUR 150 million can
be implemented for longer than six months only if Member States submit an evaluation plan
to the Commission for approval.
At this stage the evaluations of the following six GBER-based schemes have been approved
by the Commission:
SA.40324: Spain: development of industrial technology R&D
SA.40098: Finland: R&D projects
SA.39669: Hungary: regional development
SA.39273: UK: regional aid, broadband and risk finance
SA.38830: Poland: regional development
SA.38751: Czech Republic: regional investment incentives
2
Although Member States have not exactly welcomed evaluation with open arms, they
appear to concede that they can benefit from improving the effectiveness of their
measures. Most criticism that is still aired concerns the following:
 Evaluation should not apply to smaller measures with a limited impact on competition.
 Evaluation itself is a costly process.
 The Commission guidance on evaluation mentions only certain methodologies and that
appears to be limiting the options of Member States. Other, equally credible,
methodologies are also available.
An assessment of the criticism
Evaluation is a complex process that has to be adjusted to the specific facts, targets and
circumstances of each measure. The fact that the Commission guidance paper lists only
certain methodologies does not mean that it seeks to impose a standardised approach. It is
fair to say that the Commission has an open mind as to how evaluation could be carried out.
Undoubtedly, evaluation itself is not a costless process. Evaluation experts can be expensive
and the process itself can take a considerable time. Even worse, in the end it may not
produce unambiguous results. However, the evaluation process generates new knowledge.
As public authorities gain a better understanding of the impact of state aid, there will be
progressively less need to evaluate it or at least evaluate it in the same detail. Therefore, the
cost of the evaluation should in principle decline as we learn more how state aid influences
commercial decisions.
National officials often argue that the Commission should concentrate in preventing crossborder distortions rather than the waste of small amounts of public money that may have a
negligible effect on trade. But even for smaller state aid measures it still makes sense, from
a public policy perspective, to carry out what may be called an “evaluation light” exercise.
Just because the amount of public money that is committed to a state aid measure is small,
it does not necessarily follow that public authorities should be at liberty to waste it.
It is true, of course, that state aid rules and procedures apply only to situations that have a
cross-border effect. In fact, this is the principle on which all of the internal market rules are
predicated. If the effect is purely local, the rest of the European Union need not be
concerned.
There are at least two counter-arguments to the view that small aid measures should not be
evaluated. First, is it safe to assume that for small amounts of money national procedures
allow for wastage while for larger amounts all of a sudden public authorities become
prudent? I do not think so. It is more likely to believe that public officials have the same
attitude on how they manage public resources at all levels of expenditure. But it also makes
more economic sense to support lighter evaluations for smaller amounts and smaller
schemes. Therefore, an “evaluation light” approach should have a place in the ex post
control of state aid.
Second, there is a non-negligible risk that distortions from small measures may cumulatively
have a bigger impact. As the same firms can be eligible for different kinds of aid for different
3
underlying eligible costs, and given the fact that in certain market configurations a small
cost advantage may be capable of tipping the balance of competition, it cannot be excluded
that smaller amounts of aid will not produce large distortions. Hence, a certain degree of
discretion should be left to the Commission which should not be bound by budgetary
thresholds such as that laid down in the GBER.
Improvements
The first evaluation plans that have been approved by the Commission indicate that the
biggest difficulty concerns the issue which is at the heart of the evaluation logic. To evaluate
properly, one needs to know what would have happened without the state aid. In the jargon
of any evaluation methodology, this is called the definition of the “counterfactual”. Since it
cannot be known how the aid recipients would have fared without the aid, normally the
counterfactual in ex post statistical studies is taken to be the performance of a group of
unaided firms that are similar to the aided firms.
The problem with establishing the counterfactual through comparison or benchmarking is
that no two firms are identical. Even if they start from identical positions and with identical
resources, over time their performance will diverge as their managers make different
decisions. It is the nature of the competitive process.
I expect that most evaluation plans will encounter problems in establishing a credible
counterfactual. But even if it were possible to benchmark the performance of the aid
recipients against that of their cohorts, experience suggests that empirical studies have to
filter out many unexpected changes in the surrounding economic environment. Some
evaluation techniques are expressly designed to filter out these perturbations. But not every
eventuality can be thought of beforehand and taken into account. The point is that ex post
evaluations may not succeed to produce clear and unambiguous results.
This unsatisfactory outcome is something that the Commission guidance paper should have
taken into account. If the results are not decisive Member States will not know what to
make of them and what to expect of the Commission. Remember, the Commission will
authorise extension of the measures subject to evaluation only after Member States submit
to it the results of the evaluation studies. Should Member States abandon measures whose
evaluation will not lead to decisive outcomes?
The Commission should have defined standards of proof of increasing level of rigour. There
are at least four such levels.
At the lowest level, evaluation studies should produce evidence to confirm that no aid is
wasted in the sense of being granted without any incentive effect. This is the minimum that
should be demanded. Several special reports of the European Court of Auditors have found
that sometimes up to a third of granted aid has no incentive effect.
At a higher level of analysis, the aid measure that is evaluated should demonstrate that it
has been successful in increasing the values of its immediate targets, be they employment,
4
output, R&D, training, etc. Member States should be able to collect data on whether the
targets have been reached.
At a yet higher level, the aid recipients should do better than any similar firm that does not
receive aid. If they do not perform better, the aid has to be regarded as ineffective even if it
is not wasted in the strict sense of the word. It is at this level that the counterfactual has to
be established and a proper comparator group identified.
And, at the highest level, the beneficial effects of aid should be shown to outweigh the
detrimental effects from distortions of competition. Evaluation at this level would not only
take into account the harm from competition distortions but also the costs borne by public
authorities for running aid measures and the costs incurred by beneficiaries for complying
with the requirements imposed on them by granting authorities. The Commission guidance
paper does not request Member States to carry out a full blown costs-benefit analysis of all
the aspects of their state aid measures.
Conclusions
Ex post evaluation has been presented as one of the safeguards in the SAM initiative. It is
formally intended to protect the integrity of the decentralisation of state aid
implementation. But it can also stand on its own. With or without decentralisation, ex post
evaluation is necessary so that the Commission can know whether the aid it declares
compatible with the internal market is truly capable of achieving those public policy
objectives for which aid is exceptionally allowed by the Treaty and the case law.
Proper ex post evaluation is an intrinsically complex task. Given its complexity, cost and time
length, at least two recommendations for improvement of the current process can be made.
First, evaluation should ideally be carried out in a light form for smaller measures. More
extensive and robust means should be confined to larger and more distortionary measures.
Second, the Commission should define standards of proof of evaluation results. All
evaluation studies should provide information on whether the aid was necessary and
whether the immediate policy targets have been achieved. The Commission should also
consider the margin by which the performance of the aided firms should exceed that of the
unaided comparator group. Member States need to know what may constitute success or
failure of a state aid measure.
5