OECD World Forum on Key Indicators
Statistics, Knowledge and Policy
Palermo, 10-13 November 2004
“Indicators and benchmarking as a support to the
decision-making process: the Italian experience
in active employment policies ”
Aviana Bulgarelli
Director General for Vocational Training and Guidance Policies,
Ministry of Labour and Social Policies, Italy.
Benchmarking in the public decision-making process
Benchmarking can be used for two types of tasks:
for a coordinated, consistent and systematic analysis
of performance, possibly tied to a system of
targets/objectives, with the definition of a set of
significant and feasible indicators;
for an identification of the most effective intervention
policies, models and instruments for achieving the
targets and pinpointing good/best practices to be
transferred in various contexts.
Benchmarking in the public decision-making process
Benchmarking process
1.
Identification of macro-objectives and corresponding
performances to measure the success of implemented policies.
2.
Identification of a system of elementary variables/indicators
related to the macro-objectives.
3.
Comparison between the national/regional situations examined
and the reference benchmark.
4.
Understanding and explaining reasons for the differences observed
in performance and identifying actions to be undertaken.
5.
Monitoring policy implementation, analysing benchmarking results
and, possibly, correcting the entire process.
Benchmarking in the public decision-making process
Key problems in public policies
multiple objectives and, at the same time, specific intervention
instruments and policies;
alternative objectives and problems to identify the corresponding
trade-offs;
difficulty finding connections between specific policies/instruments
available and the different objectives identified;
plurality of stakeholders contributing to the identification of
objectives and to the implementation and monitoring of policies.
European Employment Strategy (EES) and benchmarking
Benchmarking has often been used as a “surrogate” for evaluation,
affecting therefore the decision-making process.
The EES is an example of multi-level strategic programming and
represents the reference framework of a complex benchmarking
exercise based on the open method of coordination.
Periodic
monitoring,
evaluation, peer
review as a mutual
learning process
Fixing
objectives and
guidelines for
the EU with
clear time-tables
Translating the
guidelines into
national and regional
policies
The open method of
coordination
Establishing
indicators and
benchmarks to
compare bestpractices
European Employment Strategy (EES) and benchmarking
Example from the guidelines for the employment policies of the
Member States in the EES (Council Decision of 22 July 2003)
Overarching objective: full employment
Overall employment rate: 67 % in 2005 and 70 % in 2010
Employment rate for women: 57 % in 2005 and 60 % in 2010
Employment rate for older workers (55 to 64): 50 % in 2010
Specific guideline: 4. Promote development of human capital
and lifelong learning
By 2010:
at least 85 % of 22-year olds in the EU should have completed
upper secondary education
the EU average level of participation in lifelong learning should
be at least 12,5 % of the adult working-age population (25 to 64
age group).
European Employment Strategy (EES) and benchmarking: the
Italian experience
EES process
 Maastricht Stability Pact process
but with a greater degree of freedom:
Social and labour policies are more complex in terms of
objectives, target population and effectiveness of intervention
tools
Complex decision-making mechanism involving a large number
of stakeholders at different decision-making levels with high
political, and managerial responsibility
However, the EES has had an influence on the action of policy-makers:
at a national level
at a regional level
at a local/provincial level in some cases
European Employment Strategy (EES) and benchmarking: the
Italian experience
Reform process of
labour market in Italy
Implementation of
labour monitoring system
Influence of EES and
EU benchmarking model
stronger emphasis on specific targets (female and older
population) even when the priority does not fully correspond to
local labour market conditions
multiplicity of stakeholders at many decision-making levels with
strong vertical and horizontal subsidiarity (the statistical
information system reflects this complexity)
the objective of complying with the Lisbon benchmark has
required the creation of many technical units involving a large
number of stakeholders
European Employment Strategy (EES) and benchmarking: the
Italian experience
Redirecting policies: the case of lifelong learning (I)
Indicator for lifelong learning = participation rate of population aged 2465 in permanent and continuing training
Lisbon benchmark: 12.5% by 2010
Italy: 4.7% participation rate in 2003
EU average: 9.7% participation rate in 2003
Italian context: strong disparities among generations and among
regions (especially in southern Italy) + time and budget constraints.
Problem:
Significant impact on policy-makers, but little room for manoeuvre
 shifting resources towards lifelong learning policies to the detriment
of alternative intervention
organisational consequences.
policies
would
have
political
and
European Employment Strategy (EES) and benchmarking: the
Italian experience
Redirecting policies: the case of lifelong learning (II)
How to make the most of available resources?
According to Eurostat’s continuing vocational training survey (CVTS2),
24% of Italian enterprises carry out training activities compared to an
EU average of 62%.
Key factor: majority of small and medium-sized enterprises and
traditional sectors, with little interest to offer training to employees:
Investing in human capital has no tangible effects in the short term
Greater incidence of training costs (both direct and indirect)
Smaller organisational and logistical capacity
Uncertainty regarding returns of training
Introduce new forms of interventions rather than
financial incentives
European Employment Strategy (EES) and benchmarking: the
Italian experience
Redirecting policies: the case of lifelong learning (III)
Creation of the Bilateral funds for continuing vocational
training – Fondi paritetici interprofessionali per la formazione
continua
Direct involvement of social partners
(managing of financial resources, planning and
directing interventions, organising and integrating
local and sector needs, monitoring the outcomes of
activities)
Aims:
encouraging a greater dissemination of the training culture and
activities (especially with regards to smaller-sized enterprises)
steering financing towards initiatives more in line with the
actual requirements of companies
European Employment Strategy (EES) and benchmarking: the
Italian experience
Influence of benchmarking at a local level
The Italian Provinces are required to play a leading role in active
labour policies and have also been involved in the benchmarking
system proposed with the EES.
The experience of Local Action Plans for employment is a
typical example, where many local authorities have set up integrated
strategic programming tools for training, education and labour,
within the framework of EES guidelines.
Indicators, benchmarking and evaluation
Main methodological and technical problems
Coherence and comparability: better no comparison than a
“wrong” one (e.g.: LLL indicator of Continuous Labour Force Survey  poor
statistical visibility of short training courses)
Relevance: in a multi-level programming system, important to
understand “who has to decide what” and to provide suitable cognitive
support (e.g.: Local Action Plans in Italy)
Timeliness 1: indicators reflect an “out-of-date” vision of the
economy  always ex-post and often available after a long delay
(e.g.: monitoring process of EES implementation)
Timeliness 2: acceleration of economic processes requires a switch
from reactive to proactive approach  Can we construct a common
system (methodologies and/or parameters) for estimating indicators?
Transferability: differences between contexts in which policies are
implemented can have a great impact on their effectiveness
Indicators, benchmarking and evaluation
Relationship between benchmarking and policy evaluation
How can we obtain information actually enabling to steer policy
choices?
How can we overcome the ex-post nature of benchmarking?
How can we have a system consistent with the specific level of the
decision-making process?
Debate:
Can we raise benchmarking from a tool for comparing and analysing
policy performance to an input for the evaluation process?