Building National Systems for
Monitoring and Evaluation (M&E):
Lessons from Africa and Other
Regions
Presentation to the World Bank’s PRS Implementation
– Learning Series
Washington D.C., 12 March 2008
Keith Mackay
Independent Evaluation Group
The World Bank
(kmckay@worldbank.org)
1
Structure of Presentation
1. Five Main Uses of M&E
2. What Do We Mean by “M&E”?
3. International Case Study: Colombia
4. International Lessons
5. Some Strengths in African Countries
6. Some Difficulties in Africa
7. A Way Ahead
8. Useful Resource Materials
2
1. Five Main Uses of M&E
a) To support budget decision-making – i.e.,
performance-based budgeting
 3 main types: direct, indirect,
presentational
b) To support national, sectoral and sub-national
planning
c) To design new policies and programs
d) To assist ministries, entities and sub-national
governments in their management
e) To strengthen accountability relationships
3
2. What Do We Mean by M&E?
 Performance indicators inputs, processes,
outputs, outcomes, impacts
 Clarify program objectives, program logic /
results chains
 Service delivery surveys
 Participatory M&E
 Rapid evaluations
 Rigorous impact evaluations
 Cost-benefit analysis
 Etc – many tools, methods, approaches
 Each M&E tool has strengths, costs, limits
4
3. International Case Study: Colombia
 M&E system managed by the Department of
National Planning (DNP), with strong support
from the President’s Office
 On-line monitoring sub-system for monitoring
and reporting government progress vis-à-vis
Presidential Goals
 Ambitious agenda of impact and other
evaluations (∑15 underway)
 DNP provides technical assistance to a few
ministries/agencies to develop M&E, and to
municipalities to pilot monitoring sub-system
and performance-based budgeting
5
Colombia’s M&E System
Strengths
 Very high utilization of monitoring sub-system
by President for oversight of ministers and
ministries – via performance targets – and for
accountability, i.e. “social control”
 Rigorous impact evaluations conducted
externally, and have high credibility
 Collaborative approach between DNP and
sector ministries/ agencies, and with
municipalities
6
Colombia’s M&E System – Weaknesses
 Insufficient reliance on M&E information to
support national planning and budget decisionmaking – this might soon change
 Weak coordination of M&E roles / functions
within DNP and with central and sector
ministries
 Monitoring sub-system data quality perceived
as low
 Too high reliance on donor funding for the M&E
system – low level of government funding
support
7
Colombia
Next Steps
1. Gov’t to strengthen institutional framework
 committee will plan, oversee evaluations
and ensure recommendations implemented
2. Efforts to link M&E to core budget, planning
work
3. Broader range of evaluations to be done
4. Less reliance on donors to fund evaluations
5. Support for pilot regions, municipalities to
strengthen existing M&E systems
6. Stronger regulation of data quality; data
harmonization; external data audits
7. World Bank project ($10m over 5 years)
8
4. International Lessons (i)
1. Demand and supply sides both need careful
attention – a mistake to view M&E, and M&E
systems, as mainly technical issues
2. On supply side, (i) caveat multiple, competing,
uncoordinated systems;
(ii) ensure data
reliability, credibility;
(iii) rationalize data
collections, harmonize
3. Avoid common mistake of over-engineering the
M&E system
9
International Lessons (ii)
4. Monitoring information – performance
indicators – are useful but it helps to have
evaluations to understand reasons why
performance is good or bad
5. Importance of training – to be able to conduct
M&E and to use M&E information
10
International Lessons (iii)
6. On demand side, a powerful champion of M&E
plays key role – ideally centrally-driven, by
capable ministry (or ministries)
7. Home-grown demand for M&E likely to be most
sustainable (Tanzania, Uganda, South Africa)
8. Need incentives to ensure M&E is taken
seriously – on both the supply and demand
sides
9. Limitations of relying on government laws,
decrees, regulations
11
International Lessons (iv)
10.Diagnosis of existing M&E – supply and demand
aspects – is an important first step
11. Need continuous review and periodic
modification of a government M&E system
 via M&E of the M&E system
 successful countries developed their M&E
system opportunistically
 building a government M&E system is longhaul effort, requiring patience
12. Keep supply and demand in balance –
utilization of M&E is the metric of success
12
5. Some Strengths in African Countries
 Active national and regional evaluation
associations – e.g., RenSE, SAMEA, AfrEA
 Emphasis on Poverty Reduction Strategies
increases the focus on M&E
 Shift to programmatic lending increases
reliance on governments’ M&E information
 Some NGOs are active in M&E
 Many governments have some key staff who
understand and are committed to M&E
 High level of donor enthusiasm and support
13
6. Some Difficulties in Africa
 Disconnect between planning and budgeting
 Multiple, uncoordinated data systems
 Under-used data (even when good quality)
 Donor emphasis on national statistics agencies
 Most countries have poor systems for tracking
government spending
 Problem of the “missing middle”
 M&E often supply-driven – thin extent of
country demand for M&E
 Danger if M&E efforts are largely donor-driven
14
7. A Way Ahead (i)
 Start with a diagnosis of current M&E
–
strengths, weaknesses, options for the future
 Endeavor to build a coalition of support within
government, and with donors
– explain
to officials “what is M&E?”, and tell them “what
is in it for you?”. Thus raise awareness of, and
demand for, M&E
 Work with a sector ministry or agency to
develop a demonstration pilot
15
A Way Ahead (ii)
 Push donors to support the government’s M&E
agenda, and to fund evaluations
 Consider some or all of the following as
possible key elements of an M&E system:




national and sectoral statistical collections –
especially relating to national priorities such as
the MDGs
sector ministries’ administrative data
financial management information systems –
for better tracking of government spending
public expenditure tracking surveys to identify
“leakage” of funds and possible corruption
16
A Way Ahead (iii)
 Possible key elements of an M&E system
(continued):


service delivery surveys of client satisfaction and
perceptions of government service quality
rapid appraisals – e.g. of “problem” projects or
programs
 But caveat: “less is more”, so it is important to
monitor the extent to which each type of M&E
information is actually used

and, where utilization is low, identify the reasons
and try to address them
17
8. Useful Resource Materials

Keith Mackay, Comment mettre en place des systèmes de
S&E pour améliorer les performances du secteur public,
Banque mondial, 2007.
En anglais: http://www.worldbank.org/ieg/ecd/better_government.html
To be published in French shortly; Word version now available

Arild Hauge, Développement des capacités de suivi et d’
évaluation au service d’un renforcement de l’action
gouvernamentale en Ouganda, Documents de travail DCE
No 10, Octobre 2003
http://lnweb18.worldbank.org/oed/oeddoclib.nsf/24cc3bb1f94ae11c85256808006a0046
/2217928e3494955885256f320067fe87/$FILE/ecd_uganda_10_french.pdf

World Bank, A Diagnosis of Colombia’s National M&E System,
SINERGIA, World Bank, 2007.
http://www.worldbank.org/ieg/ecd/sinergia.html
18
Useful Resource Materials

Keith Mackay, Guide analytique et cadre d’action, Documents
de travail DCE No 6, Janvier 1999
http://lnweb18.worldbank.org/oed/oeddoclib.nsf/a4dd58e444f7c61185256808006a0008
/19c25e9a28f3871f852567fc00558815/$FILE/ECD_WP6_Fr.pdf

Groupe indépendant d’évaluation, Suivi & Évaluation:
Quelques outils, méthodes et approches, Banque mondiale,
2004
http://lnweb18.worldbank.org/oed/oeddoclib.nsf/24cc3bb1f94ae11c85256808006a0046
/662670dfe668188485256b9600500cc3/$FILE/french_me.pdf

Groupe indépendant d’évaluation, Évaluations marquantes:
des évaluations qui ont contribué à améliorer les résultats et
l’impact des programmes de développement, Banque
mondial, 2004.
http://lnweb18.worldbank.org/oed/oeddoclib.nsf/24cc3bb1f94ae11c85256808006a0046
/a5c2b725b9a7727c85256f320067befa/$FILE/influential_evaluations_ecd_french.pdf

World Bank website on Building Government M&E Systems:
En anglais: www.worldbank.org/ieg/ecd/
19