1
A Framework for Developing an Effective Monitoring and
Evaluation System in the Public Sector – Key Considerations
from International Experience
Robert Lahey
President
REL Solutions Inc., Canada
RELahey@rogers.com
Abstract
The introduction of a Monitoring and Evaluation (M&E) system in a public institution
or on a country-wide basis is a significant undertaking that generally requires both
organizational changes as well as cultural changes within the public sector. For
example, establishing clarity of ‘purpose’ for the M&E system, and communicating
the ‘vision’ are important elements at the outset.
To be effective, M&E needs to be positioned as far more than a technical instrument
for change. It is not enough to simply create a highly trained evaluation capacity and
expect that organizations and systems will eventually become more effective. This is
particularly true in situations where a broad-based and systematic approach to M&E
had not previously existed, as in the case of many developing countries. There is a
need to also address the institutional capacity and indeed the drivers that are
generating the demand for Evaluation.
The author will present a framework that identifies four broad building blocks needed
for an effective M&E system: (i) vision; (ii) an enabling environment; (iii) the
infrastructure to supply M&E information; and, (iv) the infrastructure to demand and
use M&E information.
A set of twelve ‘Critical Success Factors’ are identified that recognize the importance
of positioning M&E within a broader context than simply a technical one; that is, also
recognizing the political support factors needed to launch and sustain an effective
M&E system. And, for each element of the framework examined, comment will be
provided (drawing on standard practices from other countries) to advise on an
appropriate strategy for M&E capacity building.
While cautioning that ‘one size does not fit all’, the framework can indeed serve as a
useful guide or ‘diagnostic tool’ to international M&E system development in many
ways: in planning; in assessing progress and identifying gaps; and, as a
communication vehicle and springboard to inform and educate technical and nontechnical personnel alike on the various ways that knowledge-building and innovation
are being introduced into public sector organizations.
Key words: international, M&E capacity building, M&E system
REL Solutions Inc.
RELahey@rogers.com
2
1. Introduction
The experience of those countries that have often been considered as leaders in
public sector evaluation and performance measurement reveals that monitoring and
evaluation (M&E) capacity building is a long-term and iterative process. It has
further been observed that over time, there may be great swings in the use and
perceived usefulness of evaluation in the public sector. These shifts can occur as a
result of a government policy change, broader public sector reforms, a changing
political agenda and/or the introduction of a new government. This raises the
importance of recognizing the broad set of players that need to be involved in
developing an M&E system for it to be effective. And, for it to be sustainable, M&E
capacity building needs to recognize the importance of positioning M&E within a
broader context than simply a technical one; that is, also recognizing the political
support factors needed to launch and sustain an effective M&E system.
To be effective, M&E needs to be positioned as far more than a technical instrument
for change. It is not enough to simply create a highly trained evaluation capacity and
expect that organizations and systems will eventually become more effective. This is
particularly true in situations where a broad-based and systematic approach to M&E
had not previously existed, as in the case of many developing countries. There is a
need to also address the institutional capacity and indeed the drivers that are
generating the demand for Evaluation.
This paper presents a framework that identifies a comprehensive set of factors that
need to be considered in the process of developing and implementing an M&E system
in the public sector. Inherent in the framework is recognition of the broad set of
players that need to be involved in developing an M&E system for it to be both
effective and sustainable. Development of the framework has drawn on three
sources: the author’s extensive background and experience in evaluation capacity
building from first-hand experience in the Canadian federal government1;
comparative analysis on the evolution of Monitoring and Evaluation systems within
four countries historically viewed as leaders in M&E (Canada, Australia, the United
States and the United Kingdom)2; and, first-hand experience in advising on M&E
capacity building in developing countries.
While each country is unique, and the challenges faced in ‘growing’ an M&E system
may be daunting (especially if there is little or no infrastructure in place at the
outset), the framework can serve as a useful diagnostic aid, and a mechanism to
draw on ‘lessons learned’ from the experience of other countries.
2. Clarifying the Terms and Concepts of M&E
The term ‘monitoring and evaluation’ or ‘M&E’ might suggest that the concept and
practice of ‘evaluation’ is synonymous with ‘performance monitoring’. While the latter
two terms both refer to approaches to measuring ‘performance’, it is important to
note the distinctions between these two tools of accountability and governance.
The author was the founding head of Canada’s Centre of Excellence for Evaluation,
the Evaluation policy centre and the body that guides the implementation and use of
evaluation in the Canadian federal government.
2
See Lahey, Robert (2005), “A Comparative Analysis of Monitoring and Evaluation in
Four Selected Countries: Canada, United States, Australia and United Kingdom”,
unpublished manuscript.
1
REL Solutions Inc.
RELahey@rogers.com
3
It is also useful to recognize the ambiguity around each of these terms. They often
mean different things to technical and non-technical people…and not infrequently,
among technical people.
Generally speaking, ‘performance monitoring’ refers to the continuous collection of
information and data in a systematic fashion so as to provide an ongoing indication
of how well an organization, program, etc. is performing.
‘Evaluation’ typically refers to a disciplined assessment of government programs,
policies and activities based on systematic and objective measurement and analysis,
carried out to meet expectations set in policy and standards, and publicly reported.
The critical distinction between ‘performance monitoring’ and ‘evaluation’ is that the
former will only provide trend information, while the latter can explore performance
issues deeper, identifying underlying factors of why a particular trend is (or is not)
occurring; i.e. Evaluation provides more explanatory information to managers and
decision-makers to better understand, not only how well programs and policies are
performing, but also the factors to attribute this performance.
It is important to note that the notion of a Monitoring and Evaluation ‘system’ implies
that a capability within government is being created to both generate (i.e. supply)
performance information as well as to ‘use’ performance information in decisionmaking by government managers. This is particularly important to note for two
reasons. First, the goal of such an exercise is not simply the creation of an M&E
capability in itself, but the use of performance information to assist in improving
public sector management and governance. The use of (or demand for) M&E
information will function if there exists effective ‘incentives’ built within the system.
Second, while the collection and analysis of performance information generally
requires the assistance of technically trained analysts and data specialists, the
eventual ‘use’ of this performance information is generally carried out by nontechnical managers in government. Though not requiring a technical comprehension
of M&E methods, it is still important though for these managers to understand and
appreciate how and where M&E information can help them in the management of
their programs and policies.
3. A Framework for Developing a Monitoring and Evaluation (M&E)
System
The successful development, implementation and sustainability of a Monitoring and
Evaluation (M&E) System in the public sector requires four essential building blocks:

Vision: An understanding of how M&E information can assist public sector
managers and decision makers. This of course requires strategic leadership as
well as a clear understanding of the basic concepts and potential uses of M&E.

Enabling Environment: A fundamental requirement to ensuring a
commitment to not only launch an M&E exercise but to sustaining it over the
long-term. This translates into a commitment to resource such an exercise as
well as providing an enabling environment to allow it to develop and mature.
Given that the introduction of an M&E system might challenge the current
REL Solutions Inc.
RELahey@rogers.com
4
culture and the way of doing things within government organizations, political
will and leadership are essential to support the values and ethics that underlie
a successful M&E system; that is, transparency, objectivity, accountability and
a commitment to a ‘results’-orientation and good governance.

Technical Capacity and Infrastructure to Supply M&E Information:
Technical capacity includes both the existence of credible and relevant data
and information-gathering systems as well as the skilled personnel to gather,
analyze and report on the performance of government policies and programs.
Additionally, infrastructure is needed to help ensure a systematic,
comprehensive and credible approach to M&E. This would include policies and
standards that would clarify roles, responsibilities and accountabilities for
performance monitoring and evaluation; establish expectations across the
system re timing and level of reporting; and, set out quality standards for
M&E conduct. The infrastructure would also include the organizational units
that would serve to conduct or manage M&E exercises; as well, the Policy
Centre that provides the policy direction, oversight and assistance, needed
particularly for new emerging M&E systems.

Infrastructure to Demand and Use M&E Information: The capacity to
‘use’ M&E information requires both a clarity of expectations re where and
how M&E information is intended to be used within government organizations
(e.g. planning, policy or program development; decision-making; budgeting),
as well as the capacity within government institutions to actually incorporate
and use the M&E information as part of the normal process of business. The
latter is based on the assumptions that non-technical personnel (e.g. program
managers) have a suitable appreciation of M&E concepts and that there are
adequate ‘incentives’ within the organization to ensure that managers will
actually use M&E information, reporting credible and unbiased results
information in a timely fashion. Further, this reinforces the need within
organizations for formal or informal vehicles and fora for reporting and
sharing M&E information.
In general terms, the discussion of these four ‘building blocks’ really points to two
prime foundation pieces for an M&E system launching: political will for change
and development of M&E infrastructure.
Political support is needed as an essential ‘driver’ to launch and resource the M&E
exercise; lead the change in organizational culture that may be needed; provide the
champion(s); ensure an enabling environment; deflect resistance to the introduction
of M&E and the changes that this might imply; and, provide the basis to help ensure
that the M&E system is sustainable over the long term. The experience of other
countries has shown that such support for M&E often comes as a result of broadbased public sector reform initiatives that may incorporate M&E as useful tools for
purposes of monitoring, reporting and accountability.
The successful development and implementation of an M&E system takes more than
political will though. Even with a resource commitment to invest in M&E
development, the technical hurdles may require a lengthy process to put in place and
develop credible data systems; train needed M&E specialists; and educate managers
throughout the system on how and where M&E information will be used. This is
generally a lengthy and iterative process, as the experience of most countries using
M&E systems would attest to; and, one where allowance for continuous learning and
REL Solutions Inc.
RELahey@rogers.com
5
improvement through oversight mechanisms is particularly beneficial to the
improvement of the M&E system.
Box 1 below illustrates the link between these various components that together,
represent the foundation pieces of an effective and sustainable public sector M&E
system.
Box 1
Essential Foundation Pieces and Building Blocks for an Effective and
Sustainable M&E System
Enabling Environment
Culture, incentives
Vision
Capacity to
supply M&E
information
Capacity to
use M&E
information
Skills, resources
Skills, resources
& purpose
Development of M&E
Infrastructure
IT equip, offices, policies,
linkages
Political Will for Change:
Drivers, winners and Losers
Table 1 below provides a more detailed elaboration of the political and technical
factors inherent in the M&E building blocks. This listing could be considered as a
‘checklist’ of the Critical Success Factors for developing and implementing an M&E
system. It must be stressed though that this is intended as a ‘guide’ rather than a
‘one size fits all’ prescriptive approach to M&E development.
This can serve as a useful guide to M&E system development in many ways: at the
outset, to help in planning for all the key considerations and requirements underlying
a successful M&E system; over time, to help in the assessment of progress and
identification of gaps in M&E system development; and, throughout the long process,
as a communication vehicle to inform and educate technical and non-technical
personnel alike on the direction and pace of the M&E project.
Table 1
A Framework for Developing an M & E System
Critical Success
Key considerations for success
Factors
Drivers


What is driving the demand for M&E?
What are the broad goals of the exercise?

How will M&E information be used? By whom? And,
for what audience(s)?
Are there real needs for information that are
currently not being met?
Uses

REL Solutions Inc.
RELahey@rogers.com
6
Leadership
Commitment


Is leadership supportive? Leading the way?
Is there a ‘champion’ for the M&E exercise?

Is there commitment to not only launch an M&E
exercise, but to also sustain it?
What will serve as the basis to ensure that an M&E
system gets implemented? Is sustained?


Resourcing

Accountability



Technical Capacity



Infrastructure to
Supply M&E
Information



Infrastructure to
Use M&E
Information




Oversight

Where will the resources ($) to develop systems
and hire & train skilled personnel come from?
Will organizations be required to internally
reallocate? Or, be given new money?
Who will be accountable for ensuring that an M&E
system is a functioning part of the public sector?
Have roles & responsibilities been firmly
established?
Is there a capacity (data systems & infrastructure)
to collect reliable data & report credible
information?
Is there an adequate analytical capacity (skilled
personnel)?
Are there existing institutions that could serve as
credible ‘partners’ (e.g. National Statistical Office;
Research institutes)?
Is there a Policy and a set of Standards in place that
describe roles, responsibilities & expectations re the
operation of the M&E system and the use of M&E
information?
Are the organizations & Units that collect & analyze
M&E information structured & adequately resourced
($ & HR)?
Is the M&E information that gets reported credible,
timely & responding to the priority issues?
Are there formal policies or requirements on how
performance monitoring & evaluation information
gets ‘used’ by organizations?
What are the ‘incentives’ within an organization for
using M&E information (rewards and/or sanctions)?
Are there formal or informal
vehicles/mechanisms/fora for reporting, sharing or
tabling M&E information?
Is results-based performance factored into
personnel assessments?
How will the system be monitored over time to
ensure that it is functioning as expected? And, to
the level expected?
Does the National Audit Office play any role in
monitoring the M&E system & the use of
REL Solutions Inc.
RELahey@rogers.com
7



Values & Ethics


Sustainability

performance information across government?
Is there a Policy Centre (e.g. Centre of Excellence
for Evaluation) to monitor implementation?
Will the ‘performance’ of the M&E system itself be
measured? Adjusted as necessary?
Is there a formal code of conduct describing
accountabilities & expected behaviour for public
servants (eg. transparency; access to information;
fair & balanced reporting; accountability)?
Is this code well understood and adhered to by all?
Is ‘speaking truth to power’ considered appropriate
within a public service organization?
What requirements/safeguards are there to ensure
that an M&E system will be made sustainable (i.e.
allowed to continue over time)?
REL Solutions Inc.
RELahey@rogers.com