WYE CITY GROUP MEETING
ON STATISTICS ON RURAL DEVELOPMENT AND
AGRICULTURE HOUSEHOLD INCOME ,
Rome 11-12 June 2009
Session 3 Topic 3
Developing countries’ perspective: Selecting a core set of Indicators for Monitoring and Evaluation in
Agriculture and Rural Development in Less-than-Ideal Conditions and implications for countries
statistical system
Authors: Naman Keita (FAO), Nwanze Okidegbe, Sanjiva Cooke (World Bank), Tim Marchant, Consultant
Presented by N. Keita, Senior Statistician, FAO
World Bank
1
PRESENTATION OUTLINE
1.
Agriculture and Rural Development Policy Issues in
Developing Countries and M&E framework for
tracking Results
2.
The analytical framework
3.
The Indicators
4.
The data framework
5.
Capacity of National Statistical Systems
1. ARD POLICY ISSUES AND M&E
FRAMEWORK
Developed countries: agriculture is less and less the economic
base of rural areas.
Developing countries: agricultural base of the economy with:
o 40% of the workers
o over 20% of their GDP
o around 75% of the poor still live in rural areas and
o the proportion of rural population to total population is comprised
between 59.5% in less developed regions in 2000 (estimate of 56.8
% in 2005) and 74.8% in least developed countries (72.3 % in 2005)
The major policy issues are related to:
• sustainable agriculture and rural development and
• long term improvement of the people’s living standard, particularly
the rural population including food security
1. ARD POLICY ISSUES AND M&E
FRAMEWORK
• Sector-wide approach (SWAP) to ARD Programmes  growing demand for
verifiable evidence of the results and impacts of development programs.
• Most indicators are focused on performance and relate mainly to inputs and outputs
(used to populate management information systems).
• Results indicators have become increasingly prominent in the wake of recent
international resolutions such as the Paris Declaration on Aid Effectiveness in 2005
and the Monterrey Consensus on Financing for Development in 2002.
• Emphasis on aid effectiveness and results-based development  need to
demonstrate the impacts of their projects and programs  shifted the focus of M&E
from a concentration on inputs and outputs to a concentration on outcomes and
impacts.
• To measure outcomes and impacts imply the use of indicators that are based on
reliable data, and on the capacity to systematically collect and analyze that
information.
• In most developing countries conditions are “less-than-ideal.” Information is irregular
and often lacking altogether .
• Strengthening capacity for M&E begins at the national and sub-national levels, where
addressing the weaknesses of national statistical systems is a common priority.
1. ARD POLICY ISSUES AND M&E
FRAMEWORK
The ideal environment for
establishing a good M&E system
is where:
The less-than-ideal condition,
on the other hand, is where:
1. there is a strong and
consistent demand for
information.
1. demand for information is
weak.
2. the concept of "management
by results" is widely practised.
2. evidence is not used to
inform decision-making.
3. timely and relevant information
is being systematically used to
improve decision-making and
to advance the process of
development.
3. the stock and flow of timely
information are irregular
and unreliable and
statistical capacity is weak.
4. systems are in place to ensure
that reliable and relevant data
and information are available
when needed.
2. ANALYTICAL FRAMEWORK
 Logframe
Impact
 Tracking inputs and outputs
– Public Expenditure Tracking System (PETS)
and Quantitative Service Delivery Survey
(QSDS)
 Focus of this Sourcebook on:
Measuring results (outcomes and impact)
– Early outcomes
– Later outcomes
Defining a core set of priority indicators for ARD
programs
Outco
mes
Outp
uts
Input
s
3. INDICATORS
Difficulties with the measurement of agricultural output
De te cting a tre nd in maize yie lds
20000
Ethiopia maize yields:1996-2005
19500
18500
18000
17500
T a rge t
A c t ua l
17000
16500
16000
15500
20
03
20
04
20
05
20
01
20
02
19
98
19
99
20
00
15000
19
96
19
97
Yields (Kg/ha)
19000
7
3. INDICATORS
Frequency of Monitoring various Indicators
Impact Indicators
(Ultimate goal)
Outcome Indicators
(behavioral change)
Output Indicators
(Goods and Services)
Input Indicators
(Material, financial, human)
Medium to Long Run (maybe 5 years
by the time surveys are carried out)
Medium Term - Ideally annually –
maybe every 2 – 3 years
Short – Medium Term - Ideally more
than once a year or annually
Short Term - Ideally every three
months or annually
Issues – Cost and Capacity
3. INDICATORS
What makes a Good Indicator?
S SPECIFIC and SENSITIVE to the changes induced as a result of actions
taken
M MEASURABLE progress can be shown and is not easily manipulated
A ATTAINABLE and APPLICABLE to the policy action taken
R RELEVANT to the areas in question
T TIME BOUND and TRACKABLE by showing changes over time
Could also be RAVES
•
•
•
•
•
Reliable
Appropriate
Valid
Easy to collect
Sensitive and specific
3. INDICATORS
COUNTRY VALIDATIONS
 Countries: Cambodia, Nicaragua, Nigeria, Senegal, Tanania
 Purpose: Test the conceptual framework and a preliminary list of
indicators against country capacity (M&E and Statistics), practice
and learn lessons
 Findings
 All countries are engaged in strengthening and rationalising the
national M&E System in parallel with Statistical reform (NSDS)
 Countries are at different stages regarding M&E and statistical
development (Examples of Senegal and Tanzania)
 Disconnect between M&E and Statistical System
 Outcome: Revised list of indicators, framework and good practices
 Emerging Issues and challenges:
 Decentralization and devolution=>implication for M7E and Stat
system
 Linkage and articulation between M&E and Statistics system
 Linkages with international agencies
10
3. INDICATORS
Results of the country validation studies
No. of generic indicators currently available
Subsector
Total indicators
Cambodia
A. Core ARD sector indicators
28
8
7
9
8
3
B. Agribusiness and market development
13
2
4
4
3
3
C. Community-based rural development
9
2
4
D. Fisheries (aquaculture)
6
3
3
1
1
E. Forestry
13
5
3
3
5
3
F. Livestock
8
5
5
7
6
2
G. Policies and institutions
18
6
11
11
7
6
H. Research and extension
7
4
3
4
I. Rural Finance
7
5
5
J. Sustainable land and crop management
9
6
6
5
2
13
1
7
3
6
4
131
40
56
56
38
27
K. Water resource management
Nicaragua
Nigeria
Senegal
The
United
Republic
of
Tanzania
2
4
Total
11
3. INDICATORS
MENU of 86 indicators
Arranged by:
*Sub-sectors and thematic areas
A. Sector-Wide Indicators for Agriculture and Rural Development
B. Specific Indicators for Sub-sectors of Agriculture and Rural
Development
(1-Crops, 2-Livestock, 3-Fisheries and Aquaculture, 4-Forestry, 5-Rural Micro and
SME Finance, 6-Agriculture Research and Extension, 7-Irrigation and Drainage, 8Agri-Business)
C. Indicators for Thematic Areas related to Agriculture and Rural
Development
(1-Community-based rural development, 2-Natural Resources Management, 3-Land
Policy and Administration)
*Early outcome and long-term indicators
12
3. INDICATORS
Nineteen priority indicators
(MAIN CRITERIA USED: RELEVANCE, COMPARABILITY, AVAILABILITY)
1
Public spending on agriculture as a %
of GDP from agriculture sector
11
% of the rural population using financial
services of formal banking institutions
2
Public spending on agricultural input
subsidies as a percentage of total
public spending on agriculture
12
Public investment in agricultural
research as a % of GDP from
agriculture sector
3
Prevalence (%) of underweight children
under five years of age in rural areas
13
Irrigated land as a % of crop land
4
Food production index
14
% change in sales/ turnovers of agroenterprises
5
% annual growth in agricultural value
added
15
% of farmers who are members of
producer organizations
6
Rural poor as a proportion of total poor
population
16
Agricultural withdrawal as % of total
freshwater withdrawal
7
% change in yields of major crops of
the country
17
Proportion (%) of land area formally
establishes as protected area
8
% annual growth in value added in the
livestock sub-sector
18
% change in soil loss from
watersheds
9
Capture fish production as % of
existing stock ( or a rating of state of
major capture fish stocks relevant to
exports and local food)
19
% land area for which there exists a
legally recognized form of land tenure
10
Proportion of land area covered by
forest (%)
(see: menu of 86 indicators.doc)
4. DATA FRAMEWORK : Sources of data
Inputs
Administrative
information systems
Survey Programme
of the CSO or MoA
(Population census,
Agricultural census,
LSMS, DHS..)
Participatory Poverty
Assessments
Financial
Management Tools
Outputs
Outcomes
Impacts
4. DATA FRAMEWORK : TOOLS
Surveys vs. non-formal appraisal methods
Direct measurement
Household budget
survey
Censuses
Questionnaire
(quantitative)
P.P.A
Case
study
Sentinel site
surveillance
Purposive
selection
Participant
observation
Questionnaire
(Qualitative)
LSMS
Structured
interview Small prob.
Quota
sampling
Beneficiary
assessment
sample
Large prob.
sample
Open
meetings
Conversations
Windscreen
survey
CWIQ
Subjective
assessments
Community
Surveys
Census
4. DATA FRAMEWORK
• Applying the tools for M&E analysis
– Comparisons over time
• Baseline surveys
• Panel surveys
– Comparisons over space
– Counterfactual comparisons (with and without)
• Does the National Statistical System have the
capacity to deliver?
4. DATA FRAMEWORK
Comparison of key features of different surveys
1
2
Sample size
Duration
3
4
Visits to
household
5
Questionnaire size
Best used for:
Time
series
Cost ($m)
Sub- nat'l
Counterfactual
Population census
Full coverage
3-6 months
1
4-8
15-25



Agricultural census
20 000-50 000
1-1.5 years
2-4
5-12
5-10



LSMS/integrated survey
5 000-10 000
1-1.5 years
2
40+
1-2



Household budget survey
4 000-10 000
1-1.5 years
15-25
15-20
1-2



100-500
4-6 months
1
4-6
0.2-0.4



10 000-15 000
2-3 months
1
8
0.2-0.4



Focus group interviews
40-50
2-3 months
1-3
-
0.05-0.1



Windscreen survey
10-20
2-3 weeks
0
0.01


Community survey
Service delivery survey (CWIQ)
=not suitable
=adequate
=good

17
5. CAPACITY OF NATIONAL STATISTICAL
SYSTEMS
Main problems common to many developing countries:
• limited staff and capacity of the units that are responsible of for collection,
compilation, analysis and dissemination of agricultural statistics;
•
lack of adequate technical tools, packages and framework to support countries data
production efforts;
•
insufficient funding allocated of agricultural statistics from development partners and
national budget;
•
lack of institutional coordination which results in the co-existence of not harmonised
and integrated data sources;
•
lack of capacity to analyse data in a policy perspective which results in a significant
waste of resources as large amounts of raw data are not properly used;
•
difficult access to existing data by users with no metadata and indication of quality
18
5. CAPACITY OF NATIONAL STATISTICAL SYSTEMS
Opportunities
 growing interest in the monitoring and evaluation of national development
programmes=>growing interest in the rehabilitation of the NSS
NSDS process
Global strategy will provide:
• the framework to integrate a core set of agricultural and rural statistics into
the national and international statistical systems,
• identify a suite of methodologies for the data collection, provide a framework
for integrating agricultural and rural statistics with the overlapping data
requirements of other sectors, and address the need to improve statistical
capacity.
• propose a governance structure for coordination not only between the
national statistical organisations and other country ministries, but also
between national statistical organisations of other countries, donors, and
regional and international organisations.
Global Strategy to be discussed by senior experts during the upcoming
International Statistical Institute Satellite meeting to be held 13-14 August
2009 in Maputo, Mozambique
TRACKING RESULTS IN
AGRICULTURE AND RURAL
DEVELOPMENT
IN LESS-THAN-IDEAL CONDITIONS
A Sourcebook of indicators for monitoring and evaluation
World Bank
20
Thank You!
21