Humanitarian Program
Terms of Reference for Evaluative Research
On
Emergency Response and Recovery Programs 2008/09-2011/13
1. BACKGROUND
Over the last two decades many pastoral and agro-pastoral communities living in the vast arid
and semi-arid areas of the horn have been prone to great risk. This segment of the population is
confronted by complex natural economic, political, institutional, environmental factors leaving
it vulnerable to extreme and recurrent food insecurity. Successive failures of rain have
contributed to major humanitarian emergencies across the Horn of Africa affecting millions of
people. In the last five years alone (2008-2013), many parts of pastoral and agro-pastoral
regions of Ethiopia have been suffering from massive humanitarian crises resulted from
recurrent drought conditions.
During this period, Oxfam America has responded to emergencies in Borena, West Arsi, Tigray
and Bale in partnership with the national organizations GPDI, CDI, ADCS, and RCWDO. The
major focus of the humanitarian response was/is Emergency Food Security and Livelihoods
(EFSL)--cash transfer (to meet immediate food needs) through cash for work (CFW) activities
(pond rehabilitation, bush clearing, rehabilitation of degraded land through tree plantations,
terracing, check dams, and road construction), livestock health, agriculture seed support and
supplementary feeding. However, since 2009/2010, Oxfam incorporated WASH activities
(water trucking, water point rehabilitation, and public health promotion) into it’s humanitarian
response work.
In order to rebuild the capacity of disaster-affected community and reduce the risk of
vulnerability to any future disaster conditions, Oxfam America had designed a strategy of
recovery/rehabilitation program after every emergency response since 2008/2009. The major
activities undertaken during recovery interventions are EFSL (restocking, different income
generation activities, community-led grain bank initiatives and building the capacity of various
committees) and WaSH (existing water scheme maintenance and public health promotion
activities).
In order to assess the impact of its interventions on the lives and livelihoods of the people
benefiting from both emergency response and recovery programs since 2008 on longitudinal
bases, and also, to guide future programming, Oxfam America would like to invite experienced
and competent independent consultants with international and national experience to conduct
an evaluation. In this process, the evaluation is expected to pay due attention to 2008/09
1
recovery grants/works and 2011/13 emergency response activities on sample bases through
retrospective information gathering and referring baseline survey report of 2011.
2. OBJECTIVE of EVALUATION.
The aim of the evaluation is to provide an independent assessment of the achievements, gaps,
best practices, and lessons learnt and provide recommendations for improving the current
initiatives, and future programming (focusing on key program/projects components).
The key objectives of this evaluation include:
1. To assess the change/impact in the lives and livelihoods of affected communities as a
result of Oxfam’s humanitarian interventions.
2. To assess change in vulnerability and the shock absorbing capacity of communities.
3. To assess the quality of EFSL and WaSH programming in meeting international
humanitarian standards and the level of program integration in these sectors.
4. To assess the design, value added and key challenges in linking emergency with recovery
interventions.
5. To document key lessons learned and the way forward for future programming, how to
improve current operations.
3. EVALUATION METHODOLOGY
The methodology will include a combination of quantitative and qualitative methods for data
collection. Highly participatory methods should be employed to ensure stakeholder
participation. For interventions that have no baseline data, evaluators will use program needsassessment reports as baselines, review key documentation such as previous evaluation
reports, progress reports, programme monitoring data, and situation reports as necessary.
The evaluation should be rigorous, with quantitative data/findings being triangulated by
qualitative findings to ensure accuracy, richness and comprehensiveness.
In order to address the evaluation questions stated in this TOR, the evaluation team is expected
to propose detailed methodology for the evaluation, which should include but not be limited
to:



Review of secondary data for project documents, review reports, project completion
reports, previous evaluation reports, RTE reports, Oxfam America local implementing
partner reports, relevant government policy documents, and donor specific reports
other materials relevant to the objectives of the evaluation.
Discussion with Oxfam America (Horn of Africa Regional Office) humanitarian team and
management, implementing local NGO partner management.
Qualitative data collection such as individual / focus group interviews and /or
discussions using structured or semi-structured questionnaire with relevant
stakeholders stated above, and the use of other appropriate tools for data collections
and analysis key informant interviews with community leaders, relevant government
representatives at all levels (specially at district and zone levels), partners staff, and
2


focus group discussions with selected beneficiary groups at field level,
meetings/consultations with community level committee in the village,
Cross-sectional sample surveys/household interviews with beneficiaries both men and
women to capture more of quantitative information.
Regular consultation with evaluation committee and focal person coordinating the
process.
4. SCOPE AND EVALUATION CRITERIA
The evaluation is expected to cover Oxfam America and implementing local partners’
humanitarian works (emergency, recovery and DRR intervention) since 2008. Geographically
the study will cover samples of target districts and villages of Oromia (West-Arsi, Borena, Bale)
and Tigray (south and east zones) regions. However, the kind of work Oxfam has been doing is
different from place to place in terms of the communities served, thematic area and time of
intervention.

In some areas, there are consistent and repeated interventions in same district over the
last five year in the form of emergency and recovery.

The second category is the district/community where there has been intervention on
interruption bases and not happening year after year continuously.
From each of the above stated regions and zones, representative sample districts and kebeles
to be covered would be proposed by the evaluating team. These will be mutually agreed with
Oxfam and the rationale or criteria for making this choice will be included in the evaluation
report.
In terms of thematic areas, the scope evaluation is limited to EFSL (Emergency Food Security
and Livelihood) and WaSH (Water, Sanitation and Hygiene), which are the two competencies of
Oxfam in humanitarian operations. Within the context of these thematic areas, issues related
to gender in program design will be examined.
4.1 Evaluation Criteria
The evaluator will use DAC/OECD evaluation criteria (specifically the one for humanitarian
program evaluation) so as to analyze the effectiveness, efficiency, relevance, impact and
sustainability of the program’s interventions and also employ HEA principles where applicable.
4.1.1 Relevance of the emergency response and recovery intervention projects to the
immediate food security and WaSH needs of communities and the livelihood assets
built/protected by them. The consultant will assess the relevance, appropriateness and
timeliness of the programme to beneficiaries, government, partners, and Oxfam global
and regional priorities. The consultants will specifically evaluate:
 What were the critical factors affecting the speed and appropriateness of our
response at country level (and with respect to partners, and other affiliates)?
 What is the scale of the response in comparison to the size of the disaster?
3


To what extent emergency response and recovery project’s design have taken into
account the needs of targeted communities (whether they are involved in the
assessment, design and implementation of the response) and how have the projects
adapted to meet those differing needs.
The extent to which short-term emergency response are linked to the long-term
development in addressing the root causes of vulnerability. What were the benefits
and drawbacks of including recovery component to follow just after emergency
response?
4.1.2 Effectiveness of Programme Design, Implementation and Results
The consultant will assess how the immediate and long-term objectives of the
interventions/program were realized/achieved. This will be carried out through assessing:
 Have the EFSL and WaSH interventions achieved satisfactory results in relation to
the stated objects of the program?
 Were the assessments carried out jointly for WaSH and EFSL and were decisions
based on the findings? To what extent interventions were designed based on
evidence (comprehensive data collection and analysis)?
 To what extent have minimum quality requirements and global/OI EFSVL and WASH
standards been met? To what extent were these two core competencies integrated
in the project design in order to bring about a larger impact?
 How effective have emergency and recovery approaches been in reducing
vulnerabilities over time and is there evidence of this?
 How effective have the monitoring, evaluation, learning and accountability
mechanisms been in the program.
 How well did the response mainstream/integrate gender in its emergency response
and recovery activities
 How well disaster risk reduction (DRR), the environment, capacity building and
conflict/cultural sensitivities integrated in emergency and recovery works?
4.1.3 Efficiency:
The consultant/evaluation team will assess
 Have the essential program support functions of Oxfam and partners (including
finance, human resources, logistics, media and communications) been quickly and
effectively set up and resourced, and performing to an appropriate standard? How
efficient were procurement procedures and what could have been done better?
 How effective are coordination with other Oxfam affiliates (e.g. Humanitarian
Country Team), government, UN agencies, NGOs, and taskforces at different levels?
 Whether the programme strategies (including implementation strategy) was clearly
articulated following Oxfam international standards and followed-up; whether a
work plan was prepared and followed-up; if the programme have effective
4

monitoring mechanisms with clear indicators in place and the timeliness of the
programme inputs deployed in relation to the annual work plans;
How adequate and useful have emergency preparedness plans (Contingency plan)
were in place and updated regularly? What facilitated/impeded early action and
preparedness initiatives
4.1.4 Impact
 What were intended/unintended, positive/negative impacts of the emergency and
recovery projects had on men and women beneficiary community?
 What was the impact on the community’s shock absorbing capacity?
 What was the impact on creating communal assets and contribution in enhancing
their resilience capacity?
 What was the impact of the programme on target groups or institutions including on
partners, government and other actors in the sector?
4.1.5 Sustainability/Connectedness

What sustainability mechanisms are in place to ensure the resilience and
preparedness continue to be maintained and contributed to Disaster Risk Reduction
efforts in the future (after the intervention)

What is the likelihood that the benefits from the programme will be maintained for
a reasonable period after the program/interventions cease?
Do local community and government institutions support the programme? Do these
institutions demonstrate ownership commitment and technical capacity to continue
to work with the programme or replicate it?
What was the program’s contribution in influencing national, regional, local
government policies and programs on emergency and recovery interventions?
What are the program effects on local environmental resources and sustainability?
To what extent were Oxfam America’s programs coordinated with other Oxfam
affiliates and stakeholders?




5. Evaluation Outputs/Deliverables
Inception report: the inception report will detail the approach to the study, the study
methodology, tools to be utilized and evaluation framework. The Inception report will be
used to confirm a common understanding of the purpose, objectives, scope, timeframes,
and methodology for the evaluation between Oxfam America evaluation Steering
Committee and the evaluation team. The report is expected within a week period after
signing the contract. It will involve:

Conduct desk review of available documents and information
5





Review program theory of change, program logic/results chain and overview of the
evaluation purpose and objectives, Conduct brief interviews with partners and
other stakeholders so as to refine the evaluation scope and methodology
Develop evaluation framework and methodology including a tailored evaluation
questions matrix that includes evaluation criteria, questions, indicators, and sources
of information, information collection and analysis
Work Scheduling – including field visits and reporting timelines
Team - Roles and Responsibilities
Inputs to the outline of the evaluation reports
The consultant/evaluation team will submit the inception report (that includes all the above
bullet points) to OA-HARO (its evaluation steering committee) and should get approval from
OA-HARO before embarking on further activities and field work.

Draft Evaluation Report: At the end of the evaluation, the team leader will submit the draft
evaluation report to Oxfam as per the agreed outline. The evaluation report (with its
methodologies, and findings) should be completed and handed over to Oxfam for
comments (as per the agreed time schedule). It will include an executive summary of major
findings and recommendations (including references, peoples and institutions contacted
etc. as per the outline). After incorporating the comments from OA, and partners (within 2
days after receiving comments), the initial de-briefing sessions will be made with evaluation
committee and OA program staff (including other stakeholders if needed) so as to get
further inputs.
o Final Report: Based on the discussions and written comments of the stakeholders and
inputs during the final de-briefings, the evaluation team will finalize and submit the final
version of the report to Oxfam America-Horn of Africa Regional Office (Addis Ababa) within
two days of receipt of comments. The final report should be submitted in both hard and
soft copies including the data sets. The hard copy should be signed and submitted. On the
other hand, the soft/electronic copy plus the data set must be saved on a separate CD with
sample photos of the evaluation exercise. The maximum length of the final report should
not be more than 40 pages (excluding annexes). The consultant is responsible for editing
and quality control of the languages and technical issues and the final report should be
presented in a way that enables publications of the evaluation report ( Reporting outline
attached at the end of the TOR)
o Two Power Point presentations: At the time of the Inception Report, the evaluation team
will submit a Power Point detailing the features of the evaluation, which the team will be
sharing with key stakeholders during the evaluation. The second Power Point is expected
from the team during the presentation of the Draft Final Report at the end of the Evaluation
Period.
o Good Practices Document: The team is also expected to submit a separate document on
good practices and learning gleaned during the evaluation, which emphasize the
6
conclusions drawn in the Final Report. This document should include at least 3 case studies
(listing successes and failures - not more than one page for each case study) after
interacting with the community, beneficiaries/ beneficiary organizations that can be useful
for the future programming and humanitarian advocacy purposes.
6. Evaluation Implementation Arrangement

Oxfam America is sponsoring the evaluation and the final responsibility for the exercise
rests with the Ethiopia Associate Country Director (CD). The ACD will be assisted by an
evaluation steering committee consisting of HARO and HQ staff members.

The evaluating team will prepare an evaluation plan to operationalize the process. The
plan should clearly describe the different stages of evaluation, related deliverables,
method of data collecting and its analysis. It is important that the evaluation plan
conform/match with the ToR, but the team of evaluators/consultants should also
provide any refinement to the TOR explain the proposed approach to evaluation.
7. EVALUATION TEAM
The evaluation will be conducted by an independent and registered national or
international consultancy firm. The evaluation team will consist of 2-3 experts with
practical experience in handling humanitarian programme evaluation (EFSL and WASH) in
Ethiopia. The team leader will have the overall responsibility for designing and coordinating
the entire evaluation process being supported by other evaluation team members. All the
evaluation team members should have a mix of relevant experience in evaluating
humanitarian program (EFSL and WASH, recovery & DRR and also have experience in
evaluating gender dimension of the program. International experience on evaluating similar
programs and thematic areas especially in the horn of Africa is an added advantage.
8. Team Responsibilities
Under the guidance of the Oxfam America Horn of Africa Regional Office, the Evaluation
Team will be responsible for:
1. The evaluating team will be responsible for the overall coordination of the
evaluation to be carried out as detailed in the terms of reference as well as ensuring
the quality, consistency and appropriate contents of the reports and documents to
be delivered in the set timeline of the evaluation
2. The evaluating team would also design a detailed evaluation approach and
methodology, including the methodology for data collection and analysis.
3. Further developing and agreeing on the final TOR and methodology;
4. Implementing the Evaluation with adequate attention to building ownership of
common analysis and recommendations;
5. Development and testing of data collection tools, including questionnaires, interview
questions and focus group protocols;
6. Development of any databases needed for processing quantitative and qualitative
data;
7
7. Systematic data collection and data processing;
8. Design and facilitation of workshops or meetings;
9. Preparation and delivery of draft and final reports and presentations;
10. Completing the evaluation on time and within the agreed budget
9. Profile of the evaluation team:
The consultants should have different and complementary profiles:












Post-graduate degree in Social Sciences, Development Studies, Rural Development ,
Agriculture and Disaster Risk Reduction
At least 10 years of demonstrated experience in conducting evaluation on humanitarian
programmes, disaster risk reduction interventions focussed on food security, livelihood
development and WASH
Good understanding of the emergency response in the horn of Africa, mainly in Ethiopia
pastoral and agro-pastoral environment and livelihood preferably in the horn of Africa
are desirable.
Significant knowledge and experience of evaluation concepts and approaches in general
and of humanitarian programmes in particular;
Recent evaluation experience in Ethiopia and Horn countries
Facilitation skills, particularly design of stakeholder consultations exercises;
Strong quantitative and qualitative data collection, analysis and reporting skills;
Excellent analytical skills and communication skills as well as demonstrated writing skills
in English;
Computer literacy with specific regard to Word, Excel and Power Point;
An understanding of local languages and culture of the communities of the targeted
areas is a must
In-depth knowledge of quantitative and qualitative research methods
S/he should have extensive experience in humanitarian action with good record of
accomplishment in humanitarian evaluation, and possess proven communication,
facilitation and writing skills.
10. EVALUATION Schedule
The duration of the evaluation will not exceed a period of two months, including the fieldwork,
local travel time, consultations, research, briefing and debriefing at the partner field and head
office and Oxfam America (HARO) level. The draft report of the evaluation and presentation of
the finding will have to be delivered within ten days after the field work activities completed.
The entire evaluation work is expected to take a total of 45 actual consultancy days in the two
months period from July –August 2013.
Oxfam affiliates, partners, donor and relevant government representatives from national and
regional level would be invited to participate in the final presentation of the finding of the
evaluation.
8
The following timetable is suggested for the evaluation process subject to confirmation from
the evaluation team.
Ser.No
1
Activities to be undertaken
Expected number days
Literature Review and Submission of Inception 5 days
Report (not more than 20 pages) containing:

2
Overview of the evaluation purpose and
objectives
 Team - Roles and Responsibilities
 Evaluation Framework and Methodology
 Information Collection and Analysis
 Work Scheduling – including Reporting
timelines

Team - Roles and Responsibilities

Inputs to the outline of the evaluation
reports
Inception Workshop – presentation of the
2 days
evaluation team on:


3
Evaluation Framework
PowerPoint presentation on salient features
of the evaluation that will be shared with
key stakeholders
Interview with Oxfam staff/partners at head office
3 days
level
4
Travel to project locations
4 days
5
Training of field data collectors, pilot testing the
tools for data collection
3 days
6
Field visit/data collection
12 days
7
Data entry/Coding
4 days
8
Data analysis and report writing
5 days
9
Production and submission of draft report
1 days
10
Comment on the draft report (comment
management) and debriefing presentation to the
evaluation committee
2 days
9
11
Presentation of the draft final report (after
incorporating all comments from all stakeholders)
on validation workshop
2 days
10
Final report submission after incorporating inputs
on the validation workshop
2 days
Total
45 days
11. Reporting
Evaluation report with a maximum of 40 pages will be submitted to Oxfam America Horn of
Africa Regional Office (HARO). The report should be prepared according to the agreed outline
i.e., using Oxfam America (OA) standard reporting outline for Final Evaluation). See annex 1
below for OA standard reporting outline.
In the Final Report, the Mission is expected to provide detailed info with respect to:






Documents reviewed;
Interviews;
Field visits;
Questionnaires
Participatory techniques and other approaches for gathering and analysis of data; and
Participation of stakeholders and/or partners.
The report should be written in standard font size (12) and Arial font type with Microsoft
window 2007 version which is applicable to computers currently in use.
In addition, the final report should contain the following annexes:
-
Terms of Reference for the evaluation
Itinerary (actual)
List of meetings attended
List of persons interviewed
List of documents reviewed
Any other relevant material/s
12. Logistics
Oxfam America will cover costs related to transport (both vehicle and air ticket as required).
13. Expressions of Interest
Interested national and international consultancy firm with the stated profile and experience
can send their application documents which should include
1) Evaluation proposal (maximum up to 5 pages) – outlining the approach in undertaking the
evaluation
10
2) Latest curriculum vitae of the consultants that highlight their national and international
experience
3) Evidence of relevant previous experience and / or names and contact details of references
with good knowledge of your previous work.
4) Sample of previous evaluation done by the team members
5) Schedule of consulting fees
6) Copy of Renewed License, TIN& VAT registration certificate of the consultancy firm
14. Application procedure
Technical and financial proposals shall be submitted in separate sealed envelopes & submitted
/ sent to the following address in person or via P.O. Box:
Oxfam America Horn of Africa Regional Office
DH Geda Tower, 5th floor
Bole Sub-city, kebele 02
P.O.Box 25779/1000 Addis Ababa, Ethiopia
Tel.251-116-628422
Deadline for the submission of the proposals is on June 17 th, 2013 C.O.B ( 5:00PM)
15. Annex
Annex I: Evaluation Report Template
As per Oxfam Evaluation Report Template (to be provided later to the consultant) but will have
at minimum preface, executive summary, introduction, evaluation framework (which includes
evaluation purposes, key evaluation questions, evaluation team, methodology and its
limitations), core analysis and findings, conclusions and recommendations, which must be clear,
relevant and actionable and should follow the conclusions.
Annex II
 List of documents reviewed
 List of field sites visited
 Questionnaires and checklist used to gather data
 Terms of reference for the evaluation
 Itinerary( actual)
 List of meetings attended
 List of persons interviewed
 Evaluation Tools
 Bibliography
 Other relevant documents
11