Urszula Romańska
Department for Coordination of Development Strategies and Policies
Ministry of Infrastructure and Development

Result-orientation in the partnership agreement
1. Participatory model – the process went hand in hand with programming of the PA and OPs.
2. Intervention logic was at the centre of our attention – correct identification and definition of desired results taking into account the Europe 2020 strategy.
3. Standardised approach to defining the desired results – started from the specific objectives of the PA within thematic objectives, defining the internal and external factors influence the final result: a. how should a TO be used to achieve the results?  specific objectives of the PA b. linking the EU targets and priorities with the PL strategic documents c. creating a matrix of logical intervention for each thematic objective d. strengthening the links between the diagnosis and the specific objectives  intervention logic  defining the input of cohesion policy to the desired results

THEMATIC OBJECTIVE 8 IN THE PA
4.8 bln €
(4 778,7mln € ESF + 200,8mln € ERDF)
NEEDS ACTIONS (IPs) RESULTS OBJECTIVES
EDUCATION
SYSTEM DOES
NOT MEET THE
NEEDS OF THE
LABOUR
MARKET
INCOMPLETE
USE OF
LABOUR
MARKET
RESOURCES
8.2
8.5.
8.6.
8.7
8.8.
8.11
3%
28%
25%
22%
1%
6%
8.9
8.10
.
10%
6%
DEVELOPMENT OF
ENDOGENOUS
POTENTIAL
IMPROVEMENT OF
CHANCES FOR
EMPLOYMENT OF THE
UNEMPLOYED AND JOB-
SEEKERS
INCREASE IN
ADAPTABILITY OF
PROFESSIONALLY
ACTIVE PEOPLE AND
ENTERPRISES
IMPROVEMENT IN THE
HEALTH CONDITION OF
LABOUR MARKET
RESOURCES
INCREASE IN
EMPLOYMENT
INDICATOR:
1) AGE 20-64 W / M:
57,6%/72,1%

65,2%/82,6%
2) AGE 55-64 W / M
31,0%/51,3% 
32,5%/65,2%
3) RURAL AREAS:
50,3%  57,9%
MORE EFFICIENT
USE OF LABOUR
MARKET
RESOURCES

Result-orientation in the partnership agreement
A three-step system of indicators:
1. 10 development indicators in the PA – macroeconomic indicators (GDP p.c., debt), human development index, etc. + E2020 targets relating to employment, higher education and drop-outs; without baseline values, only milestones for 2020 and target for 2023
2. 19 strategic result indicators in the PA – the other E2020 targets and other strategic results indicators stemming from the national strategies, a. 12 per TO (e.g. World Bank’s ease of doing business for TO11) b. with baseline values (2010, 2012 2013 depending on the data availability) and targets for 2020 and 2030 c. an estimated input of ESIF (ranging from 0,46% up to 93%) d.
„STRATEG” – an open digital system of strategic result indicators, based on public statistics as well as some data from the national and regional OPs
3. Strategic result indicators in the operational programmes – usually taken from the PA, in the regional OPs not limited to the impact of that particular OP.
4. COM forced us to regionalise the Europe 2020 targets – to be finalised in 2015

TO 1
2,5
2,0
1,5
1,0
0,5
0,0
Baseline value
Milestone v.
Target value
ESIF input into the change
2012
2020
2023
0,90%
1,70%
1,94%
Europe 2020 target
23% Direct impact of TO1 expenditures
ESI Funds’ intervention in 2014-2020 should result in an increase in growth rate of expenditure on R&D
(esp. In the private sector)
GERD/PKB - dane historyczne GERD/PKB - cele UP

TO 1
the ROPs’ input to the PA target is only 1,36% PKB instead the 1,96% set in the PA
REGION dolnośląskie kujawsko-pomorskie lubelskie lubuskie łódzkie małopolskie mazowieckie opolskie podkarpackie podlaskie pomorskie śląskie świętokrzyskie warmińsko-mazurskie wielkopolskie zachodniopomorskie
BASELINE VALUE
0,78
1,06
1,37
0,26
-
0,55
0,43
0,65
-
0,44
0,73
0,52
0,38
0,49
0,64
0,37
TARGET
VALUE
1,10
1,45
2,00
-
-
-
0,89
0,95
-
0,97
1,30
1,30
0,48
1,14
0,70
-
BENCHMARK
COHERENT WITH
THE PA
1,43
1,02
0,90
2,42
1,44
0,71
1,14
1,97
2,45
0,46
1,73
2,99
2,73
0,47
2,41
0,93
% BENCHMARK n/a
88%
39% n/a
64%
48%
73% n/a n/a
108%
54%
90%
67%
100%
35% n/a

Result-orientation in the partnership agreement
1. Participation of national and regional authorities in the process of ex ante evaluation of the PA
2. Regular information on the progress in negotiations of the EU legislative package and about, their implications, actions to be taken by the ministry, workshops, etc.
3. Analysis of coherence between the strategic result indicators in the PA and regional
OPs (as a part of the PA’s ex ante evaluation process)
4. Analysis of coherence and consistency of performance frameworks across all OPs
5. Explanatory and guiding materials: a. a handbook on programming and implementation of cohesion policy 2014-2020; b.
the ministry’s commentary to the OP template – guidance how to fill it in; c. a common list of key indicators;

Common list of key indicators
(CLKI)
1. 2007-2013 – lack of coherence across the OPs : a. the same actions/changes measured by different indicators b. indicators divergent in terms of definition, names, estimation methods, lack of historical data etc.:
• limited scope for aggregation of data
• limited process of institutional learning
• limited possibilities to present the evidence of policy’s effectiveness c. too many of them and insufficient link with the actions taken
2. 2014-2020 – new challenges: a. programming documents more result-oriented b. the EU list of common indicators c. performance framework and reserve
= a more rigorous approach to indicators much needed.

Common list of key indicators
(CLKI)
Common list of key indicators – a coherent set of indicators, complementary to the strategic results indicators of the PA and OPs:
1.
Indicators which are gathered from projects and aggregated at the OPs’ level and the
PA level using the system of electronic data exchange
2.All EU common indicators for the ERDF, the CF and the ESF + other commonly used indicators: a. direct result indicators – to be used in tables 3 & 4 of the OP template b. output indicators – to be used in table 5 of the OP template
3.Each indicator with an extensive fiche (name, unit, type, other linked indicators,
IP/IPs, definition and measurement method, interpretative notes, category of intervention)
= ca. 350 indicators,
4.A template for each OP on choice and methodology for estimation of all indicators used in it – for internal use only, not to be presented to COM.

Result and output indicators in the OPs
1. Its description should clarify how the envisaged types of actions (+ main target groups & areas) will lead to achieving the specific objectives and result indicators.
2. result indicators (tables 3 & 4) – quantifying specific objectives, coherent with the
PA, max 1-2 per priority axis. Two types: a. strategic result indicators – from public statistics/evaluations, baseline 0 b. direct result indicators – from the CLKI, OP’s monitoring system, baseline can’t be 0
+ for the ESF – from the CLKI,either a direct result indicator or a long-term result indicator
3. output indicators (table 5) – quantifying the actions to be taken and the expected change/needs: a. a majority of projects covered by a given IP should generate the output indicators included in that table. b. derived only from the CLKI, obligatory use of the EU common indicators.
4. COM forced us monitor the number of jobs created by each project.

Performance framework
1. Implications of performance framework and reserve should be taken into account when designing the internal structure of an OP  limiting the number of thematic objectives on a priority axis
2. Three elements in the performance framework – an ideal solution
 adding an extra (controllable!) KIS to have the three elements
• Strike the right balance between performance and minimising the risk
• 50% of financial allocation : on the IP level or types of projects within IP?
• How to use of KIS?  the link with the output indicator is not compulsory
• A moving target – the draft national OPs were sent to COM in January 2014; changes in the final Partnership Agreement = changes in the national and regional
OPs

1. Calculating the minimum amount of the EU co-financing in payment claims that must be send to COM by the end of 2018 to avoid the automatic decommitment
= (OP 2014 EU allocation
– the OP 2014 reserve ) + (OP 2015 EU allocation
– the OP 2015 reserve)
– initial pre-financing (i.e.: 3% * 94% of the total OP EU allocation) – annual prefinancing for 2018 (i.e.: 2,75% * 94% of the total OP EU allocation)
2. Calculating the total eligible expenditure to be certified by the end of 2018
= the product from point 1 / 0,85 for the lessdeveloped regions
= the product from point 1 / 0,80 for the
Mazovia region
3. Deducting the technical assistance
= the product from point 2 – the estimated total eligible expenditure for the TA
4. Ensuring that the product from point 3 = the sum of milestones for each regular (non-TA) priority axes

Performance framework
Using the method proposed for setting milestones but slightly modified:
• In point 1 we take simply 100% EU allocation for a given OP instead of the long formula
• Next we go through points 2-4
• MAs are advised to decrease the target value on each priority axes by few % because in 2024 expenditures can still be certified. The decrease below 100% has to be based on historical data regarding absorption level at the end of programming period for a given type of investments.
• Making ends meet – programming teams and financial management teams;
• Lack of historical data to estimate milestones and targets;
• IPs with one or few projects – what should be done with the reserve?

Performance framework
1. Only output indicators from the CLKI and only those presented in the table 5
2. Ensuring a causal effect : a. between milestones and targets of a given output indicators b. between output indicators/KIS and financial indicator (?)
Ensuring consistency across programmes:
1. which level is appropriate? – IP or type of projects? The financial indicator relates to the whole allocation of a priority axis BUT the output indicators and KIS relate to more than 50% of the allocation
2. which elements should be consistent? – the type & name of the indicator, EUR exchange rate, estimation methods (unit costs), price dynamics, financial instruments, state aid intensity:
• ERDF/CF fully implemented operations vs. partially implemented in the ESF
• Lack of data at regional level (esp. ESF indicators)

Performance framework
1. A template to present the methodology in a standardised manner: a. the necessary data required by the CPR and the IR 215/2014 b. the additional information required by the ministry:
• on practical implementation mechanisms which will ensure that the implemented projects will lead to achievement of milestones and targets;
• on possible external factors that may influence the estimation of milestone and target values  Article 5(6) of the IR 215/2014 c. it will become an official annex to the OPs.
2. Additional annual milestones for 2016, 2017, 2019-2022 a. schedule for the OP implementation: b. monitored on a regular basis c. underperformance leads to an explanation and an action plan
3. On-going evaluation for reallocation of the performance reserve – to be completed in a coordinated manner by all MAs in the 1st quarter of 2019