Mango’s Guide to Financial Management for NGOs
Further Reading – Accountability & Measuring Performance
November 2005
Introduction
This note on “Further Reading – Accountability & Measuring Performance” is part of Mango’s
guide to financial management for NGOs (available at www.mango.org.uk).
1
Ebrahim, Alnoor, NGOs and Organizational Change, Cambridge 2003
2
Edwards, Michael and Hulme, David, Non-Govermental Organisations – Performance
and Accountability, Earthscan 1995
3
Hofmann, Charles-Antoine et al, Measuring the Impact of Humanitarian Aid - A Review
of Current Practice, ODI, Humanitarian Practice Group Report #17, June 2004
4
Patta Scott-Villiers, The struggle for organisational change: how the ActionAid
Accountability, Learning and Planning System emerged, Development in Practice, Vol
12, Numbers 3 & 4, 2002
5
Wallace, Tina and Chapman, Jennifer, Some Realities Behind the Rhetoric of
Downward Accountability, INTRAC 2003
Note: This is a very short selection of a few books from a large literature. Many other books
and papers discuss the same issues.
1
Ebrahim, Alnoor, NGOs and Organizational Change, Cambridge 2003
Ebrahim provides a realistic, well-informed analysis of the relationships between two NGOs in
rural western India and donors in North America. Based on extensive field research, he
recognises how NGOs work in a complex web of relationships, and how they present
information in different ways (and sometimes withhold it) to different audiences.
Ebrahim demonstrates how NGOs and donors depend on each other: NGOs gain funds by
providing information on “successful” projects, thereby conveying a positive reputation on
their donors. NGOs have incentives not to write anything negative in their reports, because it
may damage the flow of funds. Donors have incentives not to press for further information,
because it may damage their reputation.
This exchange leads to structured interactions between NGOs and donors that favour short
term and easily measurable activities at the expense of longer-term processes of social and
political change. Unfortunately, it is a stable system, that reinforces itself without promoting
high quality work in the field. This is a critical, system-wide weakness.
Ebrahim also discusses the negative effects of results-based monitoring systems, such as
Logical Framework Analysis. He notes that they take a large amount of time, have a negative
influence on how NGO staff think about their work, and do not provide NGO staff with
information that is practically useful in their work with local communities.
2
Edwards, Michael and Hulme, David, Non-Governmental Organisations –
Performance and Accountability, Earthscan 1995
This comprehensive book provided a state-of-the-art review of the issues of performance and
accountability in 1995, based on a wide range of case studies and the substantial field
experience of the authors. It is still highly relevant today.
The book recognises the complexity of what NGOs are trying to achieve, and how difficult it is
to measure their performance, which is almost always a matter of judgement and
© Mango 2005
Charity No.: 1081406
Page 1 of 3
www.mango.org.uk
Mango’s Guide to Financial Management for NGOs
interpretation. But, written largely by NGO insiders, it is also highly critical of NGOs. It points
out that there is increasing evidence that NGOs do not perform as effectively as had been
assumed – or as effectively as they claim.
It describes how NGOs operate with very little accountability, and how “Northern NGO
survival has been almost completely delinked from performance”, because they appear to be
“under little obligation to tell the truth to their supporters”.
The book provides evidence that relationships with donors can distort accountability upwards
and overemphasise linear approaches to performance-measurement, with damaging effects
on the ability of NGOs to be effective catalysts for social change. It also provides evidence
that downward accountability is generally weak.
The editors emphasise the importance of the quality of the relationships between NGOs,
donors and governments – and of negotiation as a necessary step in improving accountability
between different actors.
The editors conclude: “Improving performance-assessment and accountability is not an
‘optional extra’ for NGOs; it is central to their continued existence as independent
organisations with a mission to pursue. However, judged by the meagre level of resources
devoted to organisational development in these areas and the reluctance of NGOs to
innovate, an independent observer would have to conclude that most were not seriously
concerned about accountability at all.”
“NGOs have spent too much time criticising others and not enough time ‘putting their own
house in order’.”
There is little evidence that the sector as a whole has changed much since this conclusion
was written.
3
Hofmann, Charles-Antoine et al, Measuring the Impact of Humanitarian Aid - A
Review of Current Practice, ODI, Humanitarian Practice Group Report #17, June
2004
Available from: www.odi.org.uk/publications/hpgreports.html
This report investigates current practice in measuring and analysing the impact of
humanitarian assistance. It notes that the level of scrutiny over how money is spent has
increased in recent years, but that this has yet to “translate into clear improvements in the
measurement of impact. … Assessment of impact is, in fact, consistently poor.”
The report recognises inherent reasons why measuring impact is difficult, including the
difficulty of establishing causality, challenging operating environments, a lack of shared
objectives and the fact that a great deal of humanitarian activity is preventative (for instance,
trying to reduce excess mortality). In addition, external data (such as population figures) may
not be available, and agencies may be working under pressure, with limited resources.
It continues to discuss different approaches to defining and measuring impact before, during
and after interventions. The report recognises that agencies use a mixture of process
indicators and outcome indicators.
The report comments that “it is too early to say whether the introduction of results-based
management systems in humanitarian organisations will significantly improve the
measurement and analysis of impact” – but that this has not happened yet. It calls for greater
investment in skills and capacities to improve impact assessment.
The study concludes with a series of recommendations for improving impact analysis,
including relating narrow focused evaluations to broader processes of change and
© Mango 2005
Charity No.: 1081406
Page 2 of 3
www.mango.org.uk
Mango’s Guide to Financial Management for NGOs
engagement with affected populations and a call for greater clarity about the objectives of
humanitarian assistance.
4
Patta Scott-Villiers, The struggle for organisational change: how the ActionAid
Accountability, Learning and Planning System emerged, Development in Practice,
Vol 12, Numbers 3 & 4, 2002
Scott-Villiers describes how ActionAid, one of the UK’s largest international NGOs, has
attempted to revolutionise its internal systems in recognition of the failure of bureaucratic
approaches to accountability.
The paper honestly recognises weaknesses over a period of 10 years of growth, when the
organisation grew from spending £20m a year to £50m. ActionAid’s previous internal system
was “an upward accountability system” that requested “regular information from the field up
through the hierarchy to the Trustees”. This came to be at odds to ActionAid’s communitycentred approach to development: “Since the sponsors and other donors, on whose money
everyone relied, were the ones to be kept happy first, the poor would never have real
influence.”
The old system led to a situation where “there was a tendency for much to be written by
fieldworkers that was not used, many decisions to be made by management that were
avoided in the field and much energy expended which might have been better spent. People
said they felt they were being pulled in two directions. Participation was increasing stress and
mendacity rather than creating influence among the poor and marginalised. Bureaucracy was
winning.”
Alongside this frustration, a number of factors drove fundamental change: a new strategy, a
highly critical external review of the entire organisation, and visionary leadership.
The new system, introduced in 2000 and known as the “Accountability, Learning and Planning
System” (ALPS), is a set of principles rather than rules or rigid procedures. It puts downward
accountability first, requiring staff at all levels to spend time in face-to-face discussion with
partners, the poor and one another – reflecting on the issues they are facing together. It sets
out a highly decentralised way of working, explicitly cutting away unnecessary bureaucracy
and requiring staff to take more responsibility for their own work.
ALPS is unproven, and may not succeed. But it is a radically new approach to issues that
almost all large NGOs and donors face.
5
Wallace, Tina and Chapman, Jennifer, Some Realities Behind the Rhetoric of
Downward Accountability, INTRAC 2003
Now published in: “Earle, Lucy (ed), Creativity and Constraint, INTRAC 2004”.
This is a preliminary report from a study on the policies and procedures surrounding aid
disbursement. Findings show that there are strong incentives to focus on upward
accountability at the expense of downward accountability. NGOs also have an incentive to
over-promise what they can do.
Tightening reporting requirements by donors (focused on measurable impact) and in
particular the increased use of logframes hinder meaningful development practice. They are
passed on to local partners and squeeze the development of different forms of civil society.
These practices are corrosive within NGOs. They take a very large amount of time and
energy – and they encourage secrecy. Senior managers and board members from the private
sector tend to share donors’ focus on measurable indicators.
© Mango 2005
Charity No.: 1081406
Page 3 of 3
www.mango.org.uk