An Analysis of Sector Level Quality
Initiatives:
Identifying Common Lessons to Inform
Bond’s Effectiveness Framework
Robert Lloyd
Purpose of the study
•
Identify the individual and collective lessons learnt by eleven initiatives
that have developed sector wide quality / effectiveness frameworks
•
Explored five common themes:
–
–
–
–
–
•
Design process
Framework structure
Incentives for Implementation
Compliance systems
Support infrastructure
Based on the lessons:
– Recommendations to Bond for moving the Effectiveness Program forward
– Propose a possible approach for an effectiveness framework and tentative
suggestions about its content.
Table 1: Quality / effectiveness frameworks included in the study
Host Organisation
Framework name
Brief Description
Australian Council for
International
Development (ACFID)
Canadian Council for
International
Development (CCIC)
Charities Eva luation
Services (CES)
Code of Conduct
The ACFID Code of Conduct aims to enhance standards of nongovern mental development organisations in Australia. Compliance
with the Code is mandatory to access AusAID funding.
The CCIC Code of Ethics provides a series of principles and
operational standards for Canadian international development
organisations.
PQASSSO is a quality assurance system, which provides
guidance on a how to run an effective and efficient organisation.
Disasters Emergency
Committee (DEC)
Code of Ethics
Practical Quality
Assurance System for
Small Organisations
(PQASSO)
Accountability
Framework
European Foundation
for Quality
Management (EFQM)
Excellence Model
Humanitarian
Accountability
Partnership HAP)
Humanitarian
Accountability and
Quality Management
Standard
ISO 9001: 2008
International
Org anisation for
standardisation (ISO)
International Social and
Environmental
Accreditation and
Labeling (ISEAL)
Alliance
National Council for
Voluntary
Org anisations (NCVO)
People in Aid
Code of Good Practice
for Setting Social and
Environmental
Standards
The Sphere Project
Humanitarian Charter
and Minimum
Standards in Disaster
Response
Campaigning
Effectiveness
Programme
Code of Good Practice
The DEC Accountability Framework is intended to ensure that
DECÕsmember agencies operate to a high leve l of standards
when providing assistance during humanitarian emergencies.
The EFQM Excellence Model is a framework to help an
organisation understand the connections between what it does,
and the results that it is capable of achieving. It is designed to be
used by any size or type of organisation.
The HAP Standard aims to improve the accountability and quality
of humanitarian aid, particularly in terms of its beneficiaries.
ISO 9001:2008 is a standard for setting up a quality management
system, and specifically looks at organisational processes, and
how these are managed. It is intended to be used by any type of
organisation.
The ISEAL Code is a framework, which sets standards on how
social and environmental standards are deve loped. It aims to
encourage good practice and protect the credibility of social and
environmental standard setters.
The NCVO Campaigning Effectiveness programme offers tools
approaches, and training to organisations in improving the quality
and effective ness of their campaigning.
The People in Aid Code of Good Practice addresses human
resource management in humanitarian and international
deve lopment agencies.
The Sphere Minimum Standards are an entirely voluntary set of
guidelines on the minimum acceptable standards for a
humanitarian operation.
Common themes that emerged from
the research: design process
• Clarity about why a framework is being developed and for whom?
– AFCID’s Code of Conduct was developed in response to external pressure
and as a result took the form of verifiable standards
– NCVO’s CAP emerged in response to a capacity gap in the sector and
therefore adopted a flexible framework of tools that organisations pick from
• A framework needs to have a niche among similar initiatives
– PQASSO - no quality assurance mechanism for small NGOs
– ISEAL - growing interest in standard setters by government
• Consultation with users helps generate by in
– SPHERE - consultation was just as important as the outcome
– HAP & ACFID - consult widely, but have a core group that leads
• Caution that consensus decision making doesn’t push a framework
to its lowest common denominator
– ACFID - need to balance listening and leading
Common themes that emerged from the
research: framework structure
• Taking an integrated view of effectiveness
– PQASSO & EFQM - pillars of effectiveness cover all aspects of an
organisation and each is equally important
– Makes a framework more daunting, but provides a complete and
realistic picture of what needs to be done
– People in Aid - support in prioritizing areas for improvement is key
• Identifying high level principles
– ISEAL - provide the foundations upon which the rest of the framework is
built
– EFQM - tool for engaging senior management
• A common approach which enables organisations to ‘see
themselves’ in the framework
– EFQM - general questions and then work with organisations to adapt
language and concepts
– SPHERE & SGS NGO benchmark - common / thematic standards
Common themes that emerged from the
research: framework implementation
• Challenge with donors backing a framework
– While it can lead to rapid up take of a framework, it should not be
the primary driver
– Can lead to an approach that emphasises minimum standards
rather than continuous improvement
• Importance of ongoing support, advise and guidance
– PQASSO & HAP - development of a framework is only first step
along the path towards effectiveness. Support in using the
framework is key
– Type of support will depend on how advanced an organisation’s
engagement with effectiveness is
• ISEAL - this can put a strain on the host organisation outsourcing capacity building
• Creating spaces for peer learning and sharing
– HAP & DEC - Agencies that apply the framework and benefit from it
can become the best advocates of the approach among peers
A proposed approach for an effectiveness
framework: key considerations
1.
There are already a wide range of existing tools, approaches
and frameworks that address the issue of quality /
effectiveness
The framework should not replicate what exists, but
rather help bring coherence, show linkages, sign post
and identify gaps
2.
To navigate the framework there will need to be a set of basic
questions to help organisations identify gaps and direct them
towards tools and approaches which are most appropriate to
their needs
The framework should have an element of selfassessment build into to it
A proposed approach for an effectiveness
framework: key considerations
3.
Bond members are involved in diverse activities and are at
different stages of their conversation on effectiveness
The framework should be flexible enough to allow
organisations to see themselves in the tool and find
elements that are useful to improving effectiveness
within their own agencies
4.
There is growing pressure on the UK NGO sector to
demonstrate effectiveness, but it is not facing a crisis of
confidence or public trust, the primary driver for the tool
should therefore be to support organisations in developing
their own approaches to effectiveness
The framework should be an ‘enabling’ rather than a
‘compliance based’ tool
Principles of development effectiveness
•Continually learning & improving
•Demonstrating & managing by
results
•Respect for diversity & human
rights
•Accountability to communities
•Financial & political independence •Local ownership & partnership
Principles of development effectiveness
•Continually learning & improving
•Demonstrating & managing by
results
•Accountability to communities
•Financial & political independence •Local ownership & partnership
Common levers of organisational
effectiveness
Leadership and management
Governance and strategy
Financial management
People management
Communications
Relationships and alliances
Learning & knowledge management
User / client focus
•Respect for diversity & human
rights
Principles of development effectiveness
•Continually learning & improving
•Demonstrating & managing by
results
•Accountability to communities
•Financial & political independence •Local ownership & partnership
Common levers of organisational
effectiveness
Leadership and management
Self-assessment questions:
•Managers identify and respond effectively to
changes in the external environment
•Managers provide clear leadership motivating
and inspiring people by their vision and
commitment
•Managers drive change throuhgout the
organisation
Leadership strengthening tools
•=MC Senior Management Team Competency
Assessment, Management Center
•INTRAC (2006) NGO Leadership Development,
Praxis Paper 10
Existing frameworks
•People in Aid, HAP, PQASSO
•Respect for diversity & human
rights
Principles of development effectiveness
•Continually learning & improving
•Demonstrating & managing by
results
•Accountability to communities
•Financial & political independence •Local ownership & partnership
Common levers of organisational
effectiveness
Leadership and management
Governance and strategy
Financial management
People management
Communications
Relationships and alliances
Learning & knowledge management
User / client focus
•Respect for diversity & human
rights
Mission specific levers of
effectiveness
Possible thematic areas:
Microfinance
Capacity building
Development education
Service delivery
Advocacy/campaigning
Policy research
Humanitarian relief
Principles of development effectiveness
•Continually learning & improving
•Demonstrating & managing by
results
•Accountability to communities
•Financial & political independence •Local ownership & partnership
Common levers of organisational
effectiveness
Mission specific levers of
effectiveness
Leadership and management
Governance and strategy
Financial management
People management
Communications
Relationships and alliances
Learning & knowledge management
User / client focus
•Respect for diversity & human
rights
Advocacy
Possible and
thematic
campaigning
areas:
Self-assessment Microfinance
questions
•Are primary stakeholder
in the design of
Capacityinvolved
building
your campaigns?
Development education
delivery
•Is the evidence Service
to substantiate
policy positions
publicly available?
Advocacy/campaigning
Tools
Policy research
Humanitarian relief
•Understanding power for social change, Just
associates
•Tools for Policy impact: a toolkit for researchers,
ODI
Existing frameworks
•NCVO campaign effectiveness program
•ODI - Research and Policy in Development,
Principles of development effectiveness
•Continually learning & improving
•Demonstrating & managing by
results
•Accountability to communities
•Financial & political independence •Local ownership & partnership
Common levers of organisational
effectiveness
Leadership and management
Governance and strategy
Financial management
People management
Communications
Relationships and alliances
•Respect for diversity & human
rights
Mission specific levers of
effectiveness
Possible thematic areas:
Microfinance
Capacity building
Development education
Service delivery
Advocacy/campaigning
Policy research
Humanitarian relief
Learning & knowledge management
User / client focus
Approaches to demonstrating your effectiveness
Tools and approaches for measuring
effectiveness of different thematic activities
Types of evidence for demonstrating
change in specific areas of work
Principles of development effectiveness
•Continually learning & improving
•Demonstrating & managing by
results
•Accountability to communities
•Financial & political independence •Local ownership & partnership
Common levers of organisational
effectiveness
Leadership and management
Governance and strategy
Financial management
People management
Communications
Relationships and alliances
•Respect for diversity & human
rights
Mission specific levers of
effectiveness
Possible thematic areas:
Microfinance
Capacity building
Development education
Service delivery
Advocacy/campaigning
Policy research
Humanitarian relief
Learning & knowledge management
User
User
/ beneficiary
/ client focus
focus
Example: advocacy and campaigning
Tools: Outcome mapping, composite logic model,
spider diagram for capacity building for advocacy,
theory of change, policy maker ratings,
bellweather methodology
Types of evidence: changes in awareness of
issue, improved capacity to communicate and
promote advocacy messages, increased visibility
of campaign messages
Comprehensive effectiveness review
Principles of development effectiveness
•Continually learning & improving
•Demonstrating & managing by
results
•Accountability to communities
•Financial & political independence •Local ownership & partnership
Common levers of organisational
effectiveness
Leadership and management
Governance and strategy
Financial management
People management
Communications
Relationships and alliances
•Respect for diversity & human
rights
Mission specific levers of
effectiveness
Possible thematic areas:
Microfinance
Capacity building
Development education
Service delivery
Advocacy/campaigning
Policy research
Humanitarian relief
Learning & knowledge management
User / beneficiary focus
Approaches to demonstrating your effectiveness
Tools and approaches for measuring
effectiveness of different thematic activities
Types of evidence for demonstrating
change in specific areas of work