MEASURING FOR DECISION MAKING

advertisement
Statistics, Knowledge and Policy
OECD World Forum on Key Indicators
Palermo, 10-13 November 2004
MEASURING FOR DECISION MAKING
OFFICIAL STATISTICS FOR DECISION MAKING AND
EVALUATION: TERRITORIAL INDICATORS
LUIGI BIGGERI1
1.
Foreward
Nobody has ever doubted that statistical information and statistical methods are indispensable for taking
rational decisions. However for a long time even policy makers have rarely and badly used the statistical
information available and, at the same time, producers of official statistics have not always supplied the
necessary statistical data, often requested in a not well organized and articulated information demand.
Recently things have changed and governments at different levels are more and more interested in
formalizing their decision processes and in evaluating their programmes, activities and especially
intervention policies in economic and social areas. There are several reasons for such a renewed interest.
One of them is the continuous process of decisional and institutional decentralization, with a particular
attention to territory. Obviously, this implies adequate information and, above all, specific statistical
information systems and indicator sets that official statistics should implement, in the interest not only of
public decision-makers but also of citizens, so that they can exercize a documented democratic control.
Istat, as the major expression of official statistics in Italy, is ever more often called on to play a
considerable role in broadening statistical knowledge of social and economic phenomena for decisionmaking and evaluation purposes at national level. Special emphasis should be placed on deepening
territorial dimension, which in Italy is characterized by highly articulated elements and significant gaps in
development.
1
President, National Statistical Institute, Italy
Rev del 1/11/2004
It should be noted that today, as never before, the sinergies in progress have been so intense between
those who, for institutional reasons, demand for qualified information to orient, direct, monitor and
evaluate policy-making and those who provide statistical information, in particular official statistics. In
this overall framework, the active push coming from institutional bodies interested in a greater and more
detailed statistical information helps us better investigate the fields they operate in and, above all, acts as
a driving force to strengthening the vicious circle “new demand–new supply”. On the other hand, Istat has
always positively reacted to such a push and has more and more often started significant experiences of
institutional cooperation in the respect of the principles of reliability, transparency and impartiality
required by public statistics.
In this report, a statistical theoretical approach as well as the current policy-makers demand for statistical
information will be illustrated with particular focus on the production of sectoral and territorial statistics.
In this context, the experience made in the construction of territorial indicators within the Operative
Project will be presented. The title of the project is “Territorial and sectoral statistical information for
2001-2008 structural policies”, and it is carried out by Istat in collaboration with the Department for
development and cohesion policies (DPS) of the Ministry of Economy and Finance. The aim is to
highlight how information demand arises, how it is defined and specified, which are the answers as
regards the construction of a set of indicators, which are the advantages and which problems are met both
in the construction of the indicator system and in their interpretation. To give a reference framework and
a logic order to the presentation, we will start recalling the steps that must be implemented to define,
carry out and evaluate intervention policies; whereas the development and the principal uses of indicators
for the development of regional policies are illustrated by Fabrizio Barca, director of DPS.
2.
Policy definition and evaluation: a statistical theoretical approach
For the reasons mentioned above, the design of policy interventions and their evaluation have become, as
it is used to say, a strategic activity at any level since it is an irremissible means:
(i)
Of Scientific support for decisional processes to make more adequate choices and to rationalize
interventions, programmes and actions.
(ii)
Of Control of the feasibility and effective implementation of the programmed objectives with
interventions and activities, in terms of quantity, costs, efficiency, efficacy, process and product
quality and impact of implemented programmes;
(iii)
For ensuring a guarantee function with reference to users, to the investments made and to the
society as a whole.
To specify how policy intervention design and the ensuing impact evaluation should be organized, we can
refer to the simplified framework illustrated in Picture 1 (Biggeri, 2000):
Framework analysis highlights some important and specific aspects that must be taken into consideration
to organize policy design and evaluation. As a matter of fact, it is necessary:
(a)
To analyze the context and real situation (a good knowledge is required of how the phenomenon
works and how the involved units behave), and the problem that we have to face, to correctly plan
the policy, the intervention and the connected information system;
Page 2 of 12
(b)
(c)
(d)
(e)
To simulate the actions on which intervention is based with macro or micro models, in order to
highlight and evaluate their possibile consequences, the possibile need of modifing the intervention
and, in any case, to choose among the various alternative actions;
To evaluate each phase of implementation of the actions and the obtained results;
To use result and evaluation analyses for learning aims (that within policy evaluation is a very
important aspect) and, if necessary, to change the plan and the operative characteristics of the
intervention and/or to plan or to improve the information system already available;
To disseminate evaluation results also as a means of social control by general public and by
interested bodies.
Pict. 1. Simplified framework of the design, the implementation and the evaluation of an
intervention policy
This kind of research is different from others both because of its aims, that are precise and definite, and
because of its operative conditions (that involve decisions). Both these elements influence the theoretical
design of the research and its practical implementation that, as usual, will have to find an appropriate
“compromise” among the three criteria of representativeness, randomization and realism (the latter in
particular for explanatory variable choice).
Fundamentally for these reasons, even considering the experimental approach, the statistical design that
better suits the evaluation process is the one called “controlled investigations” design that comprises the
Page 3 of 12
almost-experimental designs and the designs effective for observational studies or, in particular, to make
comparisons; all these designs often represent ingenious adaptations of experimental designs (obviously
releasing some of the criteria and of the hypotheses that are at their basis).
As everyone knows, to project these statistical designs it is essential to deeply know the nature and
characteristics of the programme or of the process working model, etc that must be evaluated together
with its objectives. For example, it is necessary to know which are the elements that influence programme
results and how these elements presumably interact among themselves. Obviously the effects of the
different characteristics of the programme and/or of the various elements must be measured using
response variables strictly connected to the objectives. Undoubtedly, the analysis must be done
considering the real operative conditions and consequently the characteristics of the decisional process (as
highlighted in pict. 1).
It is therefore strategic to define a reference framework of the situation which is the subject of the study
and/or a working model, which can be more or less detailed depending on the evaluation aims, the field of
interest of the programme and considering the existing economical and sociological theories. If these
theories do not exist or are not convincing, it is necessary to use empirical evidence, through a
sociological, economic, managerial, engineering study of the organizations and the processes involved in
the programme or in the intervention (in this case, the importance of the interdisciplinary cooperation is
evident).
Two further considerations derive from the framework.
Firstly, it is necessary to have adequate quantitative and qualitative information presented, if necessary,
in an appropriate and pertinent statistical information system for definition and management (of the
intervention, etc) and for evaluation; there is no doubt that to plan and evaluate in an objective way, it is
necessary first “to measure”, then to have adequate measures of the variables of interest and plan the
opportune quantitative and qualitative indicators. It is therefore necessary to deeply know the
characteristics of data and indicators and also their possible interpretative limitations (expecially if we use
proxies of the theoretical indicator, without taking for granted that the supposed associative link between
the proxy and the related indicator is unchangeable both in time and space); considering that generally
many elements act interdependently to cause results, and it is therefore difficult to isolate the effects
caused by each element; taking into account that often there is the need to“standardize” the indicators, to
eliminate the effect of the structural characteristics of inputs and/or of the reference framework.
Secondly, for interpretation purposes, we always have to remember that evaluation aims at understanding
if the policy (the intervention) implemented has reached its planned objective, modifing the phenomenon
or, rather, the conditions and/or the behaviours of individuals belonging to the reference population
observed in the real situation. As various authors underline, the fundamental methodological problem of
an impact evaluation is that generally it cannot be reduced to measure only changes of a phenomenon or
of behaviours between the previous situation and the situation after the intervention. Unless the coeteribus
paribus condition is respected, this difference is not only attributable to the implemented policy or to the
realized intervention, but it also depends on a variety of elements, among which the typical development
of the phenomenon, apart from the intervention itself. What we have to measure is the net contribution of
the intervention, in other words the difference between what happened after the implemented policy
(factual event) and what would have happened without that policy (counterfactual event).
Page 4 of 12
3.
Policy-makers’ demand for statistical information: Istat efforts to satisfy the requests
As already mentioned, after a long period characterized by a substantial lack of quantitative and in
particular evaluation culture, in Italy a new approach is now being asserted, more oriented towards the
measurement of phenomena and its use for correct policy-making.
Istat has tried and tries to meet this demand by fostering the overall growth of the National Statistical
System (SISTAN) through the National Statistical Programme and the work of sectoral “quality groups”.
In addition, Istat2 organises every year the National Statistical Conference, a significant moment where
producers and users of official statistical information convene to discuss as well as to share methods,
experiences and perspectives for a more effective meeting between demand and supply of public
statistics. Moreover for over ten years Istat has drafted an Annual Report that is an occasion for an
evidence-based reflection on Italy’s economic and social situation and for a comparison with other
countries’ situations and trends.
The Report does not only deal with Italy’s macroeconomic evolution but it also contains detailed analyses
on business competitiveness, labour market effectiveness and productivity, features of the welfare system
and so on. The Report preparation is also an opportunity to check the quality and consistency of different
data on the same phenomenon as well as data on different phenomena arising from different statistical
sources. In the end, the Report highlights the most important results obtained from statistical surveys
(thus increasing the value added in the presentation of the results of each single survey) and provides
possible solutions to problems of economic and social policy.
Today, however, to meet policy-makers’ demand for statistical information Istat more and more often
participates in ad-hoc conventions with Ministries, Regions etc, that envisage the implementation of
statistical information systems for decision-making and for evaluating the impact that action policies have
on specific fields.
However, the context shows the consequences of some important shortcomings. First of all, little
importance is given to the planning of information needs that will have to be met subsequently. This is
especially true when planning complex evaluation actions, which often suffer from an overlapping of
various disciplinary approaches (economic, sociological, political, financial etc.). Secondly, there is a
tendency to underestimate the financial resources and the technical-scientific competence necessary to
carry out the planned process of measurement/evaluation; furthermore it is on territory that the most
important - as well as specific – information needs are expressed even if financial investments and human
resources are inadequate in this field.
However, there are also cases of conventions and cooperation relationship that are gaining excellent
results, both in the production of regular analysis and evaluation reports and in benchmarking exercises.
No doubt, one of them is the cooperation relationship with the DPS of the Ministry for Economy and
Finance aimed at building up indicators that can be used for the development of Regional Policies.
2
The organization of the National Statistical Conference is among ISTAT institutional duties as established by paragr.15 of the
Legislative Decree 6 September 1989, no. 322
Page 5 of 12
This kind of activity was requested in 1989 when regional policies had become a community goal, thus
emphasising the need for all the actors involved in the implementation and programming processes to be
provided with a harmonised statistical information basis. Later, this activity widely developed in the
context of the problems connected to measurement and evaluation in coincidence with the start of 20002006 cycle of structural funds3, both to identify areas eligible for funding and to check the criteria for
breaking down financial resources.
The main results are contained in the "Programme of Southern Italy Development" (PSM), where the
European pushes to the programming of development policies have been implemented in Italy in an
original way, through a marked theoretical and quantitative approach in relation to the past. In this
document there is a macroeconomic model containing the main instrument-objective relations through
which to develop a new policy of public investments based on:

renunciation of past welfarism

enhancement of the existing strengths

a system of indicators for the measurement of the planned objectives

the relation between the impact of priority actions and the overall objective
Moreover an attempt was made to clarify the link between the choices made in regional and national
programmes (POR) and the overall objectives established for the entire economy of Southern Italy.
The first experiences have then enabled us to launch a new phase of activities and studies aimed at
supporting official statistics for decision-making through a more organic programme for the production of
statistics, indicators and adequate technical tools. This has led to the already mentioned project
"Territorial and Sectoral Statistical Information for the 2001-2008 structural policies” whose basic
characteristics will be presented below in this report.
Such a project, that develops in a wide time lag (September 2001-December 2008) long enough to fulfil
the objectives set, represents only one of the numerous experiences that Istat has been carrying out in this
field. But perhaps the most representative characteristics of this project are its being an in-depth study of
statistical information on territory and its implementation according to the logical framework of Pict. 1, as
illustrated above.
4.
The project “Territorial and sectorial indicators for structural policies”
The project mentioned above is co-financed by Istat (30%) and Structural Funds for Objective 1 areas
(70%) and aims to contribute to the overall objective of increasing and improving statistical information,
especially at territorial level, so that the effects of structural programmes can be better measured.
Although this project represents one of Istat’s strategic activities in partnership, it is not the only one.
During the last few years many other relevant and qualified activities have been settled such as designing
the information system on Handicap and Child labour within the social policy. Furthermore it has to be
mentioned how relevant the surveys on Health and Social Services supply at territorial level is, which will
allow the development of other two information subsystems.
3
In programming Structural Funds for 1994-99, Istat launched a project entitled “Information systems for local/territorial
policies” aiming at producing detailed statistical information at local level. The project has not been developed much both for
the reduced funds and the limited time horizon.
Page 6 of 12
In the case of the project on Territorial and Sectoral Statistical Information, the overall objectives of
action policies are set at the community level and then adjusted to more specific national objectives.
Wider actions at territorial level have emphasised the need for enhancing information and cognitive bases.
The increased demand for statistical information arises - above all - from the need to complete and
implement the set of “key context” regional indicators and the “breach variables” indicated in the
Community Framework of Support for Objective 1 Regions (QCS). These are the primary tools to
measure the initial socio-economic situation and the existing structures as well as to monitor progress and
results of public interventions towards local development. It is therefore obvious that the main focus of
this activity was - and is - what statistical information and what set of indicators are adequate for this kind
of measurement and analysis with the aim of continuously improving them. It was thus necessary to study
in-depth the methodological design underlying regional indicators for decision-making and evaluation.
Moreover it should be noted that each action of the project focuses on territorial variables and indicators
that have been defined in close cooperation with the Department for development and cohesion policies
(DPS):
a) Elaboration - for the purpose of programming development policies - of a system of regional
economic accounts concerning employment within specific territorial areas (territorial sections,
regions, labour local systems), even through the design of estimation models and the connection
between preliminary and final series;
b) Updating, control, improvement and implementation of “key context regional indicators” and of
“breach variables” since they represent the system of indicators that are at the basis of a process
of macro evaluation of the effects of the actions carried out through the Structural Funds;
c) New production of “key context regional indicators” and “breach variables” through the inception
of activities linked to the production of basic statistical information in those specific areas that
represent community and national priorities (such as poverty and social exclusion, water system,
ICT equipment and use by enterprises, etc.);
d) Specific territorial elaborations and production of integrative indicators with a more detailed
territorial articulation than that available at the moment (territorial sections, provinces, local
labour systems);
The results of the activities implemented by the project until now are a lot. Without going into details4, it
can be said that the work begun at the end of 2001 resulted in the:

4
Production of regional statistical indicators. This is probably the most relevant activity of the
whole project, considering both available resources and the importance the indicators have for the
effect evaluation on Objective 1 regions. There are 111 regional indicators (14 “breach variables”
and 97 “key context indicators” articulated in six line of intervention provided for by QCS)
organized following the Italian regional division and the deriving territorial aggregations; they are
available in time series from 1995 until the most recent available datum. This data base is
For a more detailed description of the project contents please visit the website http://www.istat.it/Banche-dat/index.htm or
refer to the text presented at the VI National Statistical Conference that is available at
http://www.istat.it/Eventi/sestaconf/interventi/cuciani.pdf . Moreover on February 2, 2004 a Seminar on the results of the project
was held at Istat, Aula Magna. It was also an opportunity to deepen topics which will be dealt with in the next few years. The
related documents can be found at http://www.istat.it/Eventi/seminarioterritorio.htm.
Page 7 of 12
integrated with meta-information that explains the objectives, even to non institutional users.
Considering the system of indicators, Istat has the duty of:
- assuring transparent definitions, classifications, elaboration techniques and evaluation that
are agreed at national and international level.
- harmonizing the indicators considering the Italian context as regards availability and
utility in providing information;

Estimate of socio-economic aggregates with a greater breakdown than the traditional one. It is
worth noting that big efforts were made to increase the information available for the 784 local
employment systems in Italy. In this connection, it is possible to have accurate information about
the value added produced and employment by macro sectors of activity, resident population
employed, unemployment; these quantities are also available in time series.

Enhancement of the existing territorial statistical information. The most relevant activities regard
the production of two territorial data banks containing variables and indicators on infrastructures
in Italian provinces (NUTS3) and on the most important socio-economic aggregates available for
municipalities (level LAU2).
The potential of the indicators is enormous: they allow us to make temporal and spatial comparisons,
more specific sectoral analyses as well as the analysis of regions’ convergence towards common or
optimum values. On the other hand, when evaluating the effects of intervention programmes, “key
context indicators” do not allow us to distinguish the so-called exogenous factors; during a given time lag,
changes in economy can be caused by factors that are not linked to the actions carried out through the
programmes: the increase in active population may cause an increase in unemployment notwithstanding
the employment policies implemented by the Structural Funds.
5.
Other direct and indirect advantages linked to the implementation of the informative
system for structural policies
The attention towards territory arises from a general need of phenomenon knowledge in the articulated
Italian context still subject to north-south dualism and where often economic and social deviance coexists
with situations of excellence. Moreover, territory has become even more important for the
implementation of the programmes that are co-financed by Structural Indicators since it is just within
territory, and in particular through Regions, that objectives (even though in coherence with national and
communitarian levels), policies, actions and result evaluation are determined.
From the point of view of statistical information, territory represents an active variable since it is often
correlated with many other phenomena; but also from the point of view of action evaluation it represents
an active variable since it can influence success or failure of an action (for instance, the level of
infrastructures, human and cultural resources of an area, etc).
Two important elements stem from this situation:
Page 8 of 12

In the phase of intervention planning, the information available with an adequate territorial detail
must be generally wider than the information used for evaluation (for proper and further
information about the contextual situation, see pict.1);

Due to the implicit correlations between territory and social and economic phenomena, it is
necessary to study the interrelations between base conditions, tools and the actions to be
implemented.
It is therefore clear that official statistics has to supply more statistical information that must be
compatible with action government levels. However policy areas do not often coincide with the usual
administrative sections (regions, provinces and municipalities) that represent the territorial dominions
usually utilized by statistical production. Negotiated programming instruments (territorial pacts, area
contracts, programme contracts etc.) and territorial integrated Projects - instruments planned for by
Operative Regional Programmes (POR) - focus on local realities sometimes at a sub-municipal level,
often at a sub-provincial level and more often they represent sections of territory that overlap
administrative divisions.
Therefore information demand that more and more often is required is addressed to "micro" knowledge of
territory and on these dominions the possibility to carry out temporal comparisons is also called for, in
order to estimate the effects of actions.
Since surveys at territorial level are not always available, this means that it is often necessary to improve
and increase the availability of statistical information testing the use of administrative archives (data
mining, record linkage and statistical matching), the use of statistical models (estimation models for small
areas), the enhancement and the widening of current surveys are only some of the tools that official
statistics has to widen knowledge on territory. It deals with the activity of learning and improvement
highlighted in Pict. 1.
An interesting example in this respect derives from Istat participation in a working group aimed at
identifying territorial areas suffering from industrial crises. After a preliminary experimental phase based
on the analysis of statistical provincial indicators, a more detailed territorial dimension (local labour
systems) was examined. The strength that allowed us to apply an original methodology for the
identification of critical areas has been the availability of new data and indicators based on local labour
systems, a territorial entity more suitable to underline critical situations than the provincial dimension
which often, due to its administrative nature, hides phenomena that occur in circumscribed areas.
From the point of view of learning and improvement, another result (output) which is not explicitly
represented, but not less important, regards the possibility of systematizing the boost coming from the
new demand expressed by the Institutional actors. As a matter of fact, it is clear that improvements in
methodology and production (also in the improvement of production timeliness of territorial statistics) both obtained and to be obtained - will become a part of the ordinary supply in order to enhance official
statistics as a whole. In this context we have to remember that many of the activities provided for by the
project are already an integral part of the National Statistical Programme (NSP).
It is clear that a new demand for territorial statistical information cannot be met only through its
production and/or dissemination but goes together with an intense activity of organization, analysis,
elaboration in the kind of “what happens if” and therefore with the need to supply clear decision-making
instruments. Therefore Istat has to manage technical assistance activities and accompanying measures,
studies and enhancement of detailed territorial data bases (municipalities and local labour systems).
Page 9 of 12
Finally, the publication of statistical information, reports and analyses has - and will have - considerable
consequences for local actors and all the other actors involved in local development; in the future, through
a wide dissemination of the obtained results, they will have a great availability of social and economic
statistics at a regional or – in many cases - sub-regional levels and the citizens can have a social control
over the interventions. Increased territorial information has both a direct and indirect impact. If it is true
that those who have demanded for specific information are the first users of it, it is also true that the
indirect effects on the whole population are far from being irrelevant, also for the scientific world or for
their use in various contexts.
In particular, the demand for territorial statistics has generated at least three types of impacts:
(a) a relational impact, meaning an intense cooperation activity and an exchange of territorial data and
methodologies among the various competent Sections of Istat; which has surely contributed to
increase sensitiveness to the topics of territorial data and indicators.
(b) An institutional impact, through the involvement and sensitization of other administrations or bodies
to the provision and production of territorial data; which has contributed to disseminate a “good
practice” model of institutional relationships within Istat and Sistan for data production and
information sharing.
(c) An external impact to meet the demand for information on territorial indicators coming from other
actors not directly involved in policy-making (local authorities, universities, students, enterprises,
etc.). Moreover, a contribution to the achievement of this goal has been the improvement of data
accessibility (internet, paper publishing) for a wider range of users.
The attention that Istat is giving to territory in the production of official statistics proves how urging the
problem is. Much can - and will - be done in the next years to improve the information available for
evaluators, policy makers as well as the academic and scientific world.
6.
Open issues and final conclusions
The activities and products of this project testify the pivotal role that official statistics plays to support
evaluation and policy. Besides, it confers an important appreciation to Istat professionalism. This
experience represents a change of course in the relationship between official statistics and the subjects
involved in policy choices: in the past these two worlds were nearly detached and the dialogue between
them took place in the distance; nowadays the need for correct, transparent, impartial and public statistics
represents a crucial element for a more conscious implementation of the activities whose aim is the
development of our country.
Many are, however, the issues to be solved before fully reaching the goal of production and dissemination
of territorial statistical indicators. First of all, it should be stressed that the functions of those who design
indicators able to measure or to evaluate policy effects need to be better integrated with the functions of
those who are (then) called on to implement them. Such an exchange of competence can prevent, on the
one hand, the risk of a bad or distorted reading of the indicators; on the other hand, it is functional to
identify the best result among the possible ones.
It is, therefore, necessary to point at a more systematic demand from institutional bodies not only through
coordinating the needs of the subjects that express it (central administrations, regions, local authorities),
Page 10 of 12
but also through setting up common rules and schemes to be applied in the various contexts of evaluation
and monitoring. All this aimed at optimising the limited resources available.
Another possibility is to improve basic surveys in general. Looking carefully at the territorial dimension,
while programming, can affect future production costs as well as the enhancement of quality in basic
information. Following this view, awareness in Sistan bodies needs to be increasingly raised, especially in
the Regions which are often a not-yet explored information source, and it may also be obtained by
strengthening the functions of the “quality groups” envisaged for the drawing up of the National
Statistical Programme (NSP).
Last, but not least, the issue of the resources available needs to be mentioned. In Italy, considerably few
resources are devoted to the production of official statistics and to research activity in general, which of
course is a very critical point.
The prevailing idea is that information is a good and not a cost; it is rather something all the people have
or need to have. Unfortunately, this is increasingly less true and it is even less true when information is
required at a detailed territorial level. Efforts should be made to increase the resources available. It is
therefore indispensable that those responsible for development actions, especially Regions, include in
their budget the necessary means to increase the information basis that can be used for more correct and
punctual evaluation and monitoring. This is even truer in view of the new programming cycle of the
Structural Funds, a large share of which has currently been allocated to the Regions through the POR.
The growing complexity of development policies - in terms of variety of areas involved, interventions at
detailed territorial levels, cognitive needs for programming, evaluation and monitoring – challenges
official statistics and the National Statistical System in terms of efforts, projects, survey methodologies,
coordination and resources to be used. In this changing context, we cannot ignore the implications that
will arise from the reform process of EU structural policies after the enlargement to 25 Member States
and the mounting request for subsidiarity coming from Regions in programming their structural policies.
The above mentioned project is considerably improving data and territorial statistical indicator
availability while keeping a high-degree reliability, typical of official statistics. But to improve the culture
of measurement, in particular about territory - as everyone is expecting - all the actors involved have to
give their indispensable contribution.
If from one side we ask for systematic, well projected and adequately financed territorial statistical
information, from the other side it is the duty of official statistics and of all the Sistan bodies to direct and
to manage this demand, using the highest number of available resources. From the confluence of these
two urges, we expect a better knowledge of territory and the availability of adequate indicators for
implementation and control of development policies. In other words, a better service for society and our
country in the next few years.
Page 11 of 12
References
Biggeri L., Problemi riguardanti la misura statistica per le valutazioni delle politiche, in Rivista Italiana di
Economia, Demografia e Statistica, Roma, Ottobre-Dicembre 2000.
Cipe, “Orientamenti per la programmazione degli investimenti nel periodo 2000-2006 per lo sviluppo del
Mezzogiorno”, Delibera n.71/99 del 14/5/99;
Cipe, "Programma Statistico Nazionale", Delibera n.97/2001 del 15/11/2001
ISTAT, I sistemi locali del lavoro 1991, Roma 1997;
ISTAT, Programma Statistico Nazionale, triennio 2002/2004, Roma 2001;
Ministero del Tesoro, del Bilancio e della Programmazione Economica , Programma di Sviluppo del
Mezzogiorno (PSM), Roma, 30 settembre 1999;
Ministero dell’Economia e delle Finanze, “Quadro Comunitario di Sostegno per le regioni italiane
Obiettivo 1 2000-2006”;
Ministero dell’Economia e delle Finanze, Programma Operativo Nazionale “Assistenza tecnica e Azioni
di sistema”, Roma, gennaio 2001;
Ministero dell’Economia e delle Finanze, Programma Operativo Nazionale "Assistenza tecnica e Azioni
di sistema" per le Regioni Obiettivo 1 2000-2006- Complemento di programmazione, Roma, 16 luglio
2002;
Pellegrini G., Fondi Strutturali 2000-2006: il contributo del gruppo di contatto, Giornale del Sistan n.13
2000, Roma.
Page 12 of 12
Download