COLLABORATIVE EFFORTS IN EVALUATION
Increasingly, international organisations and agencies are collaborating on evaluations, This trend
corresponds with a general shifting from a narrow project approach to programming around common highlevel goals (outcome/impact) whether for development assistance or humanitarian response. This trend is
demonstrated in:



Increasing focus on national development programmes
Support to sector-wide assistance programmes
Introduction and strengthening of the United Nations Development Assistance Framework (UNDAF)
In the humanitarian sector there is a similar shift:



The Consolidate Appeal Process is increasingly becoming a strategic planning framework akin to the
UNDAF, but broader in scope, involving non-UN organisations wherever possible
There is increasing recognition of the complexity and inter-relatedness of the humanitarian response
system (OECD-DAC, 1999)
There is recognition that the effectiveness of any humanitarian response is entwined in issues that are of
a broad system-wide nature; for example, co-ordination, protection issues, the potential to exacerbate
national tensions, and in complex emergencies actually prolong conflict and foster a war economy.
As there is increasing recognition of the importance of involving national stakeholders to strengthen their
capacities, for ownership and for greater use of evaluation results, the costs to national stakeholders
(including primary stakeholders) of participating in evaluation become more visible. The cost to national
stakeholders further justifies collaboration among international agencies and organisations.
ADVANTAGES AND CHALLENGES OF JOINT EVALUATION
Advantages







Reducing the burden of multiple, separate efforts on national stakeholders, including primary
stakeholders (e.g. consider overlapping or successive team visits and duplicative data collection efforts)
Providing an opportunity for agencies to learn and share their evaluation processes and techniques
among themselves
Enabling a greater diversity of perspectives and talents on the joint evaluation team
Permitting cost-sharing of the evaluation’s expenses
Enabling a broader scope and number of evaluation questions to be addressed, given extra (jointlyshared) resources
Promoting higher quality, more credible evaluations with broader ownership of findings and greater
influence on decisions by the participating agencies
Fostering a greater consensus on programme and country priorities and needs, thus stimulating
improved co-ordination of programs
Challenges







The need for greater lead-time and effort to organise, schedule, and otherwise co-ordinate among the
participating agencies
Given the wider scope and complexity, the costs of joint evaluations in terms of overall resources,
personnel, and management time are often considerable
Identifying other agencies interested in the same evaluation topic
Developing comprehensive, yet manageable, terms of reference that accommodate participating
agencies’ particular issues and interests
Developing joint management structures and communication processes that work effectively
Selecting teams that are credible and mutually acceptable to all participants
Co-ordinating schedules and travel logistics
UNICEF M&E Training Resource
Collaborative efforts in evaluation 1/3



Agreeing on cost-sharing and contracting arrangements
Agreeing on a process for finalising and clearing evaluation results; e.g. whether evaluation teams are
independent and how collaborating agencies will handle disputed conclusions and recommendations
Reaching agreement on reporting formats, findings/conclusions and recommendations
Many of the challenges stem from basic differences among donor agencies’ operational processes and
organisational cultures. For example, differences among donor agencies’:



Evaluation policies and procedures, methods, and reporting requirements
Administrative policies and procedures, especially financial and contracting mechanisms
Evaluation priorities, agendas, schedules, and issues.
Source: OECD/DAC (2000). Effective practices in conducting a multi-donor evaluation.
http://www1.oecd.org/dac/Evaluation/pdf/ev_multi_e.pdf
UNICEF M&E Training Resource
Collaborative efforts in evaluation 2/3
WHEN IS COLLABORATION APPROPRIATE?
The scope of country-level evaluations may vary from narrow single-agency, single-project evaluations to
broad system-wide evaluations. When is a joint evaluation approach is more appropriate and useful than the
more traditional single-agency evaluation approach?
System-Wide Evaluation
Evaluation of the response by the whole
system to a particular issue or event, e.g.
Joint Evaluation of Emergency
Assistance to Rwanda

Scope. The broader the scope, the more
agencies/organisations involved in the issue or
intervention evaluated, the more natural it will be to
collaborate.

Purpose. When focused on drawing lessons and
implications for operational decisions, collaborative
approaches tend to be especially useful. Lessons
based on the collective wisdom of numerous partners
are likely to be more valid and reliable than those
based on a single experience. When assessing
accountability for performance focusing on an
organisation’s individual contributions, collaboration is
less workable.

Commonality or complexity of the evaluation questions:
Certain complex questions benefit from analysis from
different perspectives within a system; for example,
questions related to national capacity building, reaching
the most vulnerable or integration of protection in
humanitarian response. Similarly, agencies and
organisations with common policies, such as gendersensitive programming, may share common questions
about strategies. Some evaluation criteria are
particularly targeted at system-level questions. (See
core content sheet Linking evaluation criteria to
evaluation questions: additional criteria for
humanitarian assistance.)

Practical considerations (e.g. cost, timing). Joint
evaluations are not recommended where the timeline
for producing results are short. Where the cost is high,
undertaking a joint evaluation provides opportunities for
cost sharing.

Involving national partners. The more international
agencies seek to involve national stakeholders as
partners – institutions, civil society, primary
stakeholders -- the more they will be driven to
collaborate.
Partial System Evaluation
Evaluation of part of the system, such as
a thematic or sectoral study
Single-Agency Evaluation
Evaluation of the overall response to a
particular disaster event or complex
emergency by a particular agency
(funding, channelling, or implementing
agency)
Single-Agency, Single-Project
Evaluation
Evaluation of a single project undertaken
by a single agency
Source: DAC/OECD (1999). Guidance for evaluating humanitarian Assistance in Complex Emergencies. On the web:
http://www.oecd.org/dac/evaluation/pdf/human_en.pdf
UNICEF M&E Training Resource
Collaborative efforts in evaluation 3/3