Designing a Monitoring and
Evaluation System for a Rural
Travel and Transport Project
Michael Bamberger
Gender and Development Group
The World Bank
RTTP Conference
Pretoria, South Africa
December 2 1999
1
A good M/E system helps
ensure that a project
• Is being efficiently implemented
• Is reaching the intended target groups
• Is achieving its intended objectives
2
A good M/E system will
 Monitor the use of project inputs
 Monitor the effectiveness of the project
implementation process
 Monitor the production of project outputs
 Assess project impacts on the target communities
 Assess the effectiveness of project outputs in
producing the intended short-term and long-term
impacts.
 Assess the extent to which these impacts can be
attributed to the effects of the project.
3
An RTT M/E system should
• Extract lessons and best practices for the
design of future projects
• Provide guidelines for the modification of
project design
• Provide adequate data for the evaluation
of program impact
4
The Basic
Monitoring and
Evaluation Model
5
Figure 1 Monitoring and Evaluation Model of the Project Implementation Process
Economic and Political
context within which the
project is implemented
Project
planning
and Inputs
Project
Implementation
process
Institutional context
within which the project
is implemented
Project outputs
Project impacts
Project
sustainability
Socio-economic characteristics of the communities
affected by the project
6
Stages of the project cycle
covered by the M/E system
•
•
•
•
•
Project planning and inputs
Implementation process
Output
Short and long term impacts
Project sustainability
7
Project implementation and
impacts are also affected by
• The organizational and institutional
context
• The economic and political context
• The social and economic characteristics
of the communities
8
Figure 3 Apply the Monitoring and Evaluation Model to the Tanzania Village Travel and Transport Project
Economic and Political
context within which the
project is implemented
Project
Inputs
Road
construction
, donkey,
bicycles,
construction
of water
pumps,
technical
assistance
Project
Implementatio
n process
Focus groups,
community
management
groups including
women, training,
female extension
workers
Institutional context
within which the project
is implemented
Project outputs
Improved and
new roads and
footpaths, women
use bicycles and
donkeys,
handpumps,
grinding mills.
Short and long
term impacts
travel time
reduced, easier
access to markets, ,
more children
attend school,
income increases,
women enter labor
market, women’s
status in family
enhanced
Sustainability
Infrastructure
maintained
Stronger
community
groups
Networks created
Cost-recovery
Agencies more
gender sensitive
Socio-economic characteristics of the communities affected by
the project
9
Key questions for designing a
M/E system
• What indicators should be used to
measure and monitor each stage of the
project?
• How should the information be collected?
• How to measure quality and
effectiveness?
• How and when to communicate findings
to managers and stakeholders?
10
Key questions continued….
• How do we assess the benefits and
impacts produced by the project?
• How to assess whether pilot projects have
been successful.
11
Developing and using indicators
Good indicators should
• Cover all stages of the project cycle
• Provide essential information needed to
monitor progress, identify problems and
assess results
• Provide information which can easily be
used by managers and policy makers
• Be objective and easily measurable
• Include the minimum number possible
12
Examples of indicators for a
feeder road project
• Planning and inputs
• Implementation
process
• Outputs
•
•
•
•
Stakeholder groups consulted
Funds approved and available
Equipment available
All community included in
planning and implementation
• Targets set for women
representatives
• Local organizations
established at district level
• Km. Roads constructed/
upgraded
13
Feeder road indicators page 2
• Impacts
• Increased household
income
• Higher proportion of
girls attending school
• Increased agricultural
production
• Wider range of goods
available in villages
14
Feeder road indicators page 3
• Sustainability
• Cash payments for
road maintenance
• Community labor for
road maintenance
• Condition of roads 1,
2 and 5 years after
construction
15
M/E systems must identify all
stakeholders and ensure their
information needs are covered.
• All community groups affected by the
project
• Local government agencies
• National government agencies
• NGOs
• International development agencies
16
Evaluation
studies
and
evaluation systems
17
Individual M/E studies can be
conducted
each using the methods best
suited to the particular issue
being studied
18
But to assess the overall
progress of a program
involving many different
activities ….
An evaluation system is
required
Logical framework analysis
[LOGFRAME]
Is one of the best methods for
setting up a monitoring and
evaluation system
LOGFRAME
helps define the basic
assumptions on which the project
design and implementation
systems are based.
21
LOGFRAME
monitors the following logical
sequence of activities……
22
If
certain inputs are used effectively
then
certain outputs will be produced
if
the outputs are produced
then
certain impacts will be achieved
23
Example
Using LOGFRAME to monitor
and evaluate the impacts of a
feeder road on women
24
If
feeder roads are upgraded
then
women will take more
agricultural produce to market
then
household income will increase
and women’s economic status
will improve
25
Main kinds of M/E studies
See Handout Annex I for details
Question
1.
2.
3.
Are RTTP activities
being implemented on
schedule?
Are women fully
involved in all stages?
Are the activities
producing their intended
benefits?
Study
• Monitoring study
• Gender responsiveness
assessment study
• Impact evaluation
26
Main kinds of studies page 2
Question
4. Are the activities
sustainable?
5. Were the pilot
interventions
successful and could
they be replicated?
Study
• Sustainability
assessment
• Replicability
assessment
27