GENERAL DISTRIBUTION
TEARFUND’S DARFUR EMERGENCY
PROGRAMME, 2003-2013:
EVALUATION OF HUMANITARIAN ACTION
Dr. Niaz Murtaza, University of California, Berkeley, February 2013
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
TABLE OF CONTENTS
Executive Summary
2
Context Analysis
6
Introduction to the Tearfund Darfur Programme
7
Methodology
12
Findings
13
Conclusions
26
Recommendations
28
Annexes
Terms of Reference for the Evaluation
Profile of the Evaluation Team
Evaluation Schedule
Documents consulted during the Evaluation
Persons participating in the Evaluation
List of Tearfund projects in Darfur, 2003-13
31
31
37
38
38
39
40
1
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
EXECUTIVE SUMMARY
The Darfur crisis erupted in 2003 as a result of fighting between rebel groups and the Sudanese
armed forces, resulting in between 178,258 and 461,520 deaths, with 80% of these due to
diseases and the rest due to violence. It was estimated in 2007 that around 2.4 million people1
had been displaced from their homes and were living in camps or host communities in Darfur
and Chad, creating an enormous work load for NGOs. The humanitarian context for the Darfur
population has deteriorated significantly since the conflict began in 2003.
Even before the eruption of the Darfur crisis, Tearfund had been supporting operational partners
in Sudan, including FAR (Fellowship for African Relief) Sudan and Medair. After the eruption of
the Darfur crisis in mid-2003, it became operational in Darfur by June 2004 given the huge scale
of the crisis and the limited capacity of in-country partners. Tearfund has focused on work in
Nutrition, Public Health Education, including Child-Focused Health Education, Water and
Sanitation, Food Security/Livelihoods, Education and Community Development.
Tearfund has adjusted its geographical focus in line with changing needs. Starting from the
emergency phase where life-saving services such as water, sanitation and nutrition were
needed, Tearfund has generally stayed in an area until the needs stabilized to a point where the
area was ready for long-term development. Tearfund initially started with three programme
areas in 2004—Beida, and Garsilla in West Darfur and Ed Daein in South Darfur (now part of
East Darfur). While Ed Daein still remains an operational area, Garsilla and Beida were phased
out in 2009 and 2010 as the situation was considered stable by then while Kass in South Darfur
was started around that time. In 2011 Tearfund added Um Dukhun and in 2012 added Nertiti in
Central Darfur to its programming mix. Tearfund has worked with both IDPs (internally displaced
persons) and local communities in these areas servicing between 200,000-500,000
beneficiaries annually over 10 years. Between 2004 and 2012, Tearfund spent more than GBP
39 million, with an average of more than GBP 4.5 million per year, with funding from more than
a dozen donors.
Tearfund’s programmes have not only ensured access to basic needs and reduced mortality
and morbidity, but have also enhanced people’s self-reliance, a sense of normality among
children and have enhanced the capacities of communities and government departments.
Finally, they have also enhanced peace and reconciliation at the local level and increased
people’s protection against physical dangers. The programmes have been delivered with a high
degree of community participation and in close coordination with government agencies. Thus,
Tearfund’s Darfur programme is a strong example of providing emergency and long-term
support in a precarious, complex and prolonged crisis. Tearfund’s programmes have
consistently scored more than 3 out of 5 on average in evaluations over the years, with
relevance and coverage as the strongest points and connectedness being the weakest point.
The evaluator’s own rating of the 10-year programme is as follows:
1
Internally Displaced Persons in Darfur: Taking Stock, Brookings Institute, USA
2
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
Effectiveness
The programme makes a substantial contribution to the aspect
Impact
The programme makes an acceptable contribution to the aspect; there are
shortcomings that could be addressed
Relevance
The programme makes a substantial contribution to the aspect
Efficiency
The programme makes an acceptable contribution to the aspect; there are
shortcomings that could be addressed
Coverage
The programme makes a substantial contribution to the aspect
Connectedness
The programme makes an acceptable contribution to the aspect; there are
shortcomings that could be addressed
Coherence
The programme makes an acceptable contribution to the aspect; there are
shortcomings that could be addressed
The main recommendations are as follows:
Coherence: 1) Enhance collaboration with UN agencies to leverage more inputs and expertise
from them for Tearfund and partners; 2) Enhance collaboration and sharing with other INGOs to
influence industry-wide programme quality and standards in different sectors. 3) Improve
coordination with partners. In particular, Tearfund must clearly spell out its added value in
working with partners and clearly demonstrate its added value over working through local NGOs
where partnership is not adopted. Tearfund is also advised to develop a more comprehensive
local partner strategy for Darfur setting clear selection criteria, introducing formal assessments
and critically evaluating partner projects. 4) Enhance follow-up and capacity-building of partner
ministries to ensure that programme quality standards are met after Tearfund’s withdrawal.
Connectedness: 1) Increase focus on livelihoods work by increasing its own capacity in such
work, advocating with donors about the importance of such work and by exploring synergies
between livelihoods work and traditional Tearfund sectors. 2) Review the sustainability of the
current model of Children’s Clubs based on inputs by Tearfund and instead explore the
possibility of making them more sustainable through greater community mobilization and
empowerment of women’s committees. 3) Undertake an analysis of the factors which are
constraining the return of people to their villages even where the security situation is stable and
aim to address the factors through programming, policy and networking. 4) Develop a more
explicit focus on Community Based Disaster Risk Reduction to enhance the future resilience of
communities in the face of climatic variability. 5) Strengthen Community Based Organisations
and build linkages between them and relevant Ministries to create ‘demand’ for government
services to address basic and recovery needs of communities once Tearfund withdraws. 6)
Enhance the sustainability of nutrition programmes by educating beneficiary families on
nutritional education and by developing better hand-over plans with MOH (Ministry of Health)
and UNICEF (United Nations Children Fund). 7) Increase Tearfund understanding of people’s
own strategies to rebuild their food and livelihood systems.
Coverage: 1) Improve focus on newly displaced people by striking a better balance between
their needs and those of longer-term displaced people and having larger emergency response
3
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
programmes. 2) Be more sensitive towards the specific needs of women in project design and
implementation, pro-actively involve women to that purpose and develop stronger focus on the
needs of groups of women that are at higher risk such as female headed households. 3)
Clarify/operationalize the concept of vulnerability (for staff and beneficiary community) and
make sure that staff operates from a shared understanding of what vulnerability constitutes and
how the project is going to address it.
Effectiveness: 1) Restructure the HPCA (health promotion and children activities) project to
enhance its effectiveness by integrating HP with Watsan into a new WASH project to strengthen
synergies between hard- and soft-ware and increase community ownership. 2) Incorporate
Children Activities as part of a new educational project strengthening the educational system
(focus on permanent structures), and school/community based Child Friendly Spaces and
children educational activities (Child Health Clubs and Environmental Clubs) to enhance their
effectiveness. 3) Develop a more flexible and longer-focused funding base given that the
situation in many areas is moving towards developmental needs. 4) Place greater attention and
investment in policy in UK / HQ.
Efficiency: 1) Review the possibility of reducing the number of work sites to enhance efficiency
while ensuring that this does not reduce Tearfund’s footprint and ability to influence the
humanitarian situation. 2) Reduce expenditures on expatriate staff by developing a
comprehensive capacity-building programme for national staff. 3) Review the possibility of
working with local partners as a way of increasing efficiency. 4) Increase synergy and
coordination across sectors as a way of increasing efficiency, e.g., between hygiene promotion
and watsan
Impact: 1) Enhance the ability of Tearfund to demonstrate impact through improving
documentation. In particular consider setting up a global user-friendly data system under which
each emergency programme regularly inputs information about outcomes and impact indicators
under each of the DAC (Development Assistance Committee) criterion. 2) Enhance the capacity
of the M&E function within the country programme by recruiting a senior M&E country-wide
manager.
Relevance: 1) Assessments and programme designs should include a stronger developmental
focus since communities now expected more livelihoods interventions. 2) Develop a more
comprehensive strategy for Kass operations, based on a proper needs assessment that focuses
on hand-over of selected project activities in Kass to relevant line Ministries, local NGOs and the
community. 3) Enhance the thoroughness of accountability processes across geographies,
sectors and programme phases. 4) Reintroduce the system of having the Sudan team provide
written feedback on evaluations and developing a detailed feedback incorporation plan.
4
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
TEARFUND COMMENT ON CURRENT THINKING
“Tearfund has made a switch over the years to more sustainable interventions. An example of
this is creating cost sharing mechanisms for water which enable handing over to communities
and the department of Water, Environment and Sanitation (WES), by moving to construction of
permanent structures instead of temporary ones. Although Tearfund has arrived at this
approach late, the approach taken now allows Tearfund to work for a limited time span with a
group of IDPs (2-4 years) and then hand over a sustainable intervention to them.
While working in this way, Tearfund maintains a two pronged approach to its programmes in
Darfur. First and foremost, Tearfund remains ready to work with newly displaced communities
and provides life saving emergency assistance to them. Secondly, Tearfund works on building
resilience in those communities which have been displaced longer, as well as host communities,
in order to reduce their dependency on food aid and other forms of assistance. Tearfund does
not believe there is a linear line from relief to early recovery to development, but that elements
of all these, embraced in a Disaster Management approach, are necessary at the same time in
its projects in Darfur, while at times one element will play a greater role.”
5
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
CONTEXT ANALYSIS
In 2003, in Darfur, western Sudan fighting between rebel movements, and Sudanese armed
forces developed into a major humanitarian crisis. Repeated ceasefires and peace agreements
between 2004 and now have failed to halt the fighting, leading to high levels of displacement,
mortality and morbidity. The Centre for Research on the Epidemiology of Disasters in Brussels
estimated, with 95% confidence, that the excess number of deaths in Darfur between 2004 and
2010 is between 178,258 and 461,520 (the mean being 298,271), with 80% of these due to
diseases and the rest due to violence. It was estimated in 2007 that around 2.4 million people2
have been displaced from their homes and are living in camps or host communities in Darfur
and Chad, creating an enormous work load for NGOs.
The socio-economic status of the Darfuri population has deteriorated since the beginning of the
conflict though a systematic evidence-based quantitative analysis is not possible due to the poor
status of official statistics separately for Darfur. However, the qualitative information suggests
that more than half the population lived below the poverty line even before the Darfur crisis.
Even before the conflict in 2003, the socio-economic status of the Darfur population was
deteriorating further due to drought and the increasing encroachment from outside pastoral
groups. The escalation of the conflict in 2003 has meant that more than half the population has
lost access to its two main sources of income, i.e., agriculture and animal-herding, due to
displacement as well as access to traditional sources of water. In addition, the availability of
government services in the areas of health, education and social services has reduced
significantly due to the conflict. Thus, the humanitarian context for the Darfur population has
deteriorated significantly since the conflict began in 2003.
Even before the eruption of the Darfur crisis, Tearfund had been supporting operational partners
in Sudan, including FAR Sudan and Medair. After the eruption of the Darfur crisis in mid-2003, it
became operational in Darfur by June 2004 given the huge scale of the crisis and the limited
capacity of in-country partners. Tearfund has focused on the following sectors since then:
Tearfund Darfur Sectoral Focus, 2004-2012
•
•
•
•
•
•
Nutrition
Public Health Education, including Child-Focused Health Education
Water and Sanitation
Food Security
Education
Community Development
Tearfund engages constructively with government line ministries (Ministry of Health; Department
of Water, Environment and Sanitation and Ministry of Education) and to provide capacity
building support and contribute to the shaping of appropriate policies. Tearfund has also laid
emphasis on ensuring adherence to international humanitarian conventions such as the Sphere
2
Internally Displaced Persons in Darfur: Taking Stock, Brookings Institute, USA
6
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
standards, the Red Cross NGO Code of conduct, humanitarian accountability, IASC’s code on
sexual exploitation and People in Aid principles.
Tearfund’s Main Operational Areas in Darfur, 20042004-2013
Tearfund has adjusted its geographical focus in line with changing needs. Starting from the
emergency phase where life-saving services such as water, sanitation and nutrition were
needed, Tearfund has generally stayed in an area until the needs stabilized to a point where the
area was ready for long-term development. Tearfund initially started with three programme
areas in 2004—Beida, and Garsilla in West Darfur and Ed Daein in South Darfur (now part of
East Darfur). While Ed Daein still remains an operational area, Garsilla and Beida were phased
out in 2009 and 2010 as the situation was considered stable by then while Kass in South Darfur
was started around that time. In 2011 Tearfund added Um Dukhun and in 2012 added Nertiti in
Central Darfur to its programming mix.
7
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
Tearfund Geographical Focus in Darfur, 20042004-12
2004
2005
2006
2007
2008
2009
2010
2011
2012
West Darfur
Beida
Garsilla
Nertiti
Um Dukhun
South Darfur
Ed Daein
Kass
The choice of these geographical areas reflects Tearfund’s desire to work in the most
vulnerable areas even if they happen to be in isolated, insecure environments. Thus, instead of
working in El Fasher and Nyala, the two more easily accessible operational areas which
represent the biggest hubs of NGO activities in Darfur with dozens of INGOs present, Tearfund
selected more isolated areas where needs were more intense while the number of NGOs was
very low, with Tearfund usually being one of 3-5 agencies present in the area. Tearfund has
worked with both IDPs and local communities in these areas servicing between 200,000500,000 beneficiaries annually and almost 3 million project beneficiaries over 10 years (the
actual number of people reached may be lower since multiple projects may have served the
same persons). Between 2004 and 2012, Tearfund spent more than GBP 39 million, with an
average of more than GBP 4.5 million per year.
Budget Details of Tearfund Darfur Programme, 20042004-2012
Year
04/05
05/06
06/07
07/08
08/09
09/10
10/11
11/12
Total
Budget
(GBP mln)
1.7
3
3.7
3.6
5.6
7.4
6.7
4.7
39.7
Beneficiaries (000)
56
258
423
481
246
418
478
498
2,860
8
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
Sectoral Details of Tearfund Darfur Programme, 2007-2012 (%)
In line with the priorities identified in the various programme frameworks and the 2008 six-year
strategy, Tearfund has mainly focused on the sectors of nutrition, health promotion, food
security, water and sanitation, community development and education. In line with Tearfund’s
global emergency focus and capacities, the core emergency sectors of water/sanitation (45%),
health promotion (30%) and nutrition (17%) were the three biggest sectors in terms of budget as
well as number of beneficiaries. However, Tearfund’s desire to enhance long-term self-reliance
is reflected in the fact that food security, education and community development sectors
comprised around 20% of the five-year budget and almost a quarter in the last year, reflecting a
clear evolution of the programme towards longer-term needs. In most cases, Tearfund adopted
a multi-sectoral approach where several sectors were implemented together in communities
depending on their needs, leading to broader needs coverage as well as synergies across the
sectors. The following table reflect the wide range of specific activities that Tearfund engaged in
under each of these sectors:
9
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
Details of Tearfund Sectoral Activities in Darfur, 20042004-2013
Nutrition
All 6 locations
Blanket feeding for children under 5;
supplementary feeding for moderately
malnourished
children;
outpatient
therapeutic
centre
for
severely
malnourished children
Health promotion
Ed Daein, Garsilla, and Kass,
Women’s clubs, children centres,
household visits, group sessions and
mass campaigns for health promotion
and psychosocial support
Food security
Beida, Ed Daein, Garsilla, Vegetable and crop production inputs;
Kass, and Um Dukhun
tree plantation; grinding mills; poultry;
seed banks; extension and training
Water & sanitation
Beida, Ed Daein, Garsilla, Kass
Hand dug wells, hand pumps, pumped
water systems, rainwater harvesting
schemes; camp and institutional latrine
construction;
household
latrine
construction; solid waste management
Community
development
Um Dukhun and Garsilla
Community committee formation and
training; peace-building; youth and
children activities; women’s activities
Education
Beida, Ed Daein, Garsilla, School infrastructure
Kass, Nertiti and Um Dukhun
construction
repair
and
This wide range of activities across the large geographical area was made possible via funding
from a wide range of donors. Of the seventeen donors that funded Tearfund, DFID (22.4%),
ECHO (19.3%) and OFDA (16.1%) were the three largest donors. These three donors provided
the core funding for Tearfund’s main operational areas while the other donors usually provided
supplementary funding to sectorally or geographically expand the scope of the work further. A
key feature of the Tearfund funding was the fact that the donors kept on funding large sums
consistently for several years.
10
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
Funding Details of Tearfund Darfur Programme, 2007-2012
Donor
Sectors
UN CHF Fund
Nutrition, Water, Sanitation,
Health
Promotion,
Food
Security and Emergency
Nutrition, Water, Sanitation,
Health
Promotion,
Food
Security and Emergency
Education, WASH and Nutrition
TEARFUND
family
Nutrition, Water, Sanitation,
Health
Promotion,
Food
Security,
Community
Development and Emergency
ECHO
World Relief
SGID
OFDA
DFID
DEC
BUZA
CIDA
AIDCO
CFGB
DCPSF
TOTALGBPmln
Nutrition, Water, Sanitation,
Health
Promotion,
Food
Security and Emergency
Nutrition, Water, Sanitation and
Health Promotion
Nutrition, Water, Sanitation,
Environment , Education, Food
Security and Health Promotion
Water,
Sanitation,
Health
Promotion and Nutrition
Community Development, Food
Security, Water, Sanitation and
Health Promotion
Water,
Sanitation,
Health
Promotion, Nutrition Support,
Emergency and Education
Food Security
Nutrition, Water, Sanitation
and Health Promotion
Community
Development,
Peace-building and Youth
07/08
08/09
09/10
10/11
11/12
Total
13%
20%
22%
25%
15%
19.3%
9%
6%
5%
0%
0%
4.0%
5%
4%
11%
2%
13%
6.8%
12%
2%
5%
5%
6%
6.7%
0%
7%
7%
7%
11%
6.2%
14%
20%
17%
20%
8%
16.1%
14%
18%
20%
32%
29%
22.4%
0%
0%
1%
0%
0%
0.3%
34%
23%
9%
0%
0%
12.9%
0%
0%
3%
9%
11%
4.5%
0%
0%
0%
1%
5%
1.0%
0%
0%
0%
0%
3%
0.4%
0%
0%
0%
0%
1%
0.1%
3.9
4.8
6.8
6.5
4.6
26.6
11
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
METHODOLOGY
Having worked in Darfur since 2004, in 2013 Tearfund decided to take stock of what it has
achieved but also to critically assess the 9 years of its programming in Darfur. Previously,
evaluations had been carried out of specific projects or sectors almost annually, but no
evaluation had been carried out of the programme as a whole covering the duration of its
existence. The time period covered by this evaluation encompasses evolutions in context and
strategy (in-country and corporately within Tearfund), multiple changes of personnel, and
changes in how Tearfund conducts operational humanitarian responses. Given Tearfund’s
operational interventions in a number of protracted humanitarian crises (DRC, South Sudan,
Afghanistan), the agency hoped that this type of learning will be applicable beyond the confines
of Sudan. It is intended that this evaluation will be useful in formulating practical suggestions for
how Tearfund can continue to improve the quality and appropriateness of its humanitarian
interventions. Such learning would also be applicable to the wider NGO and donor community.
The aim of this assignment is to critically assess Tearfund’s operations in Darfur since 2004
through an evaluation of humanitarian action (EHA). The Active Learning Network for
Accountability and Performance in Humanitarian Action (ALNAP) defines an EHA as ‘a
systematic and impartial examination of humanitarian action intended to draw lessons to
improve policy and enhance accountability.’ An evaluation therefore has two main functions: to
strengthen accountability and to increase learning. This EHA evaluates Tearfund’s operational
programme in Darfur since 2004 according to EACD-DAC’s seven criteria for complex
emergencies: relevance/appropriateness, connectedness, coherence/coordination, coverage,
efficiency, effectiveness, and impact. For each of the criteria, the evaluation makes a clear
statement about the programme’s performance using the following four-point scale:
The programme makes no contribution to the aspect
The programme makes a minimal contribution to the aspect; there are major shortcomings
that must be addressed
The programme makes an acceptable contribution to the aspect; there are shortcomings
that could be addressed
The programme makes a substantial contribution to the aspect.
The evaluation did not involve new data collection within Sudan but was undertaken primarily
through phone interviews with five past and present Tearfund Sudan staff and desk based
review of previous evaluations, project reports, strategy documents and other relevant
documents. The most important source of information were the 13 external evaluations that
Tearfund had conducted over the years which covered more than 80% of its expenditure over
the years in Darfur as well as all Tearfund geographical areas and sectors.
12
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
FINDINGS
Relevance/Appropriateness
Was need adequately assessed? Did the programme’s design show understanding of
and support for the capacities of the affected population? What other alternatives were
available and why were they rejected? Did the operational intervention address
communities’ own relief and recovery priorities? Were strategies appropriate and
relevant?
Among the strongest points about Tearfund’s programmes which emerges immediately from a
review of the proposals is the wide range of assessment activities that Tearfund engaged in to
help design its programmes, ranging from nutritional surveys for nutrition programmes; KAP
surveys for hygiene promotion activities; and focus group discussions for food security and
community development projects. All major Tearfund proposals were built on multiple
assessments information, including that from Tearfund’s own assessments as well as those
undertaken by other agencies. These observations are particularly true for Tearfund’s core
emergency programmes in the areas of nutrition WASH and health promotion where
programme designs clearly reflected the priorities and needs of communities identified during
assessments.
The one area where several evaluations3 point out some weaknesses in assessments relates to
food security and livelihoods. Evaluators felt that assessments and programme designs should
have included a stronger developmental focus. This is especially true of recent evaluations, e.g.,
the 2012 Ed Daein and Kass evaluations, which covered recent programmes which were
implemented in the backdrop of some degree of stabilization in these areas which Tearfund’s
own strategy documents also acknowledge. Both evaluations felt that communities expected
more developmental interventions and identify a number of alternatives programmes, e.g.,
community owned / managed wood lots. Such developmental programmes were rejected by
Tearfund partially due to a lack of staff skilled in recovery and development programmes as well
as the reluctance of donors to fund broader recovery and development programmes in the
context of a protracted emergency. Tearfund was also advised to develop a more
comprehensive strategy for its Kass operation, based on a proper needs assessment that
focuses on hand-over of selected project activities in Kass to relevant line Ministries, local
NGOs and the community. Such strategies should include a clear timeline and well worked out
roles and responsibilities of the different partners to sustain positive project impact.
Were appropriate systems for participation, information sharing and feedback, put in
place and used by project participants? Did the feedback received shape programme
design and implementation? How do beneficiaries and other stakeholders describe the
quality of relationships with programme staff? Were recommendations from evaluations
and key learning points implemented? If not, why not?
3
Evaluation of Tearfund 2009-2011 Kass Project, Sudan; Evaluation of Tearfund DMT projects
2009-2011 in Ed Daein, Sudan; Evaluation of DMT Projects in Ed Daien, Beida, & Garsilla North Sudan
13
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
Tearfund set up comprehensive systems for incorporating feedback from both communities and
evaluations. With respect to community feedback, ensuring accountability to beneficiaries is one
of Tearfund’s 12 Quality Standards. Accountability work in Darfur has included interactive notice
boards as communication tools; beneficiary participation and informed consent through MoUs
and public meetings, multi-party beneficiary accountability teams involving field staff, community
committees and leadership, as well as representatives of village based institutions for inspecting
projects together and a complaints-handling procedure. These activities were led by a dedicated
Beneficiary Accountability Officer who ensured that community consultation had taken place
and feedback received and built into future planning. As a result, both IDPs and host
communities developed a good understanding of Tearfund’s objectives and were involved in
beneficiary selection and in mobilizing communities. Clear beneficiary selection criteria were
developed and shared with the communities which ensured that humanitarian aid was usually
targeted at the most vulnerable (please see next paragraph for some exceptions to this trend).
Tearfund senior staff also facilitated periodic meetings with community leadership in project areas to
discuss the appropriateness of the intervention and quality standards of the projects. The
Beneficiary Accountability Officers played an important role in providing project information to the
communities, bringing back critical community feedback about programmes for programme staff
and relaying Tearfund responses back to the communities. Such feedback frequently led to
changes in programme designs. For example, in Um Dukhun, beneficiary feedback led to the
appointment of horse-drawn mobile extension teams to widen coverage especially in far strung
nomadic villages thereby increasing children admissions to nutrition OTP/SFP programmes. The
feedback system increased contact with isolated nomadic communities. Feedback also positively
informed the hand-over process leading to revisions of the handover agreements with partners
based on the inputs from the community.
However, several evaluations did identify unaddressed community issues and complaints. For
example, evaluations 2009 in Ed Daein, Garsilla and Beida revealed that targeting mechanisms
which were appropriate in acute phases no longer were targeting the poor, but largely the better
off poor or middle wealth groups. With respect to its food security projects, it was also advised
to critically reflect on the sorghum variety selected and determine if this is the priority choice of
farmers. Thus, there does seem to be enhancing of the thoroughness of accountability
processes across geographies, sectors and programme phases. With respect to incorporating
feedback from evaluations, Tearfund had been following a thorough process of having the
Sudan team provide written feedback on each evaluation and developing a detailed feedback
incorporation plan. However, this system seems to have been suspended later on as no such
plans were found for the post-2009 evaluations. The earlier feedback incorporation documents
reveal that in most cases remedial action was taken by Tearfund based on evaluation feedback.
Connectedness
Has the response reduced the vulnerability of beneficiaries to future disasters? To what
extent has local capacity been supported and developed? Has/will the benefits of the
intervention continue beyond the intervention’s end?
14
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
Tearfund aimed to reduce the future vulnerability of communities and enhance the sustainability of
its activities through the following mechanisms:
Setting up of a range of committees and contact groups for self-help/ support initiatives.
Training and empowerment of the village based and school-based Parents & Teachers’
Committees in running the community projects.
Capacity building of partners such as the GoS line ministries.
Construction of permanent critical infrastructure such as schools and health centers supported
by maintenance committees.
Provision of knowledge and information to communities on health and hygiene issues to
permanently alter their health behaviors in the future.
The use of children as change agents in the areas of health, peace-building and community
development, hence ensuring that the next generation of Darfur people have a solid foundation
in these issues.
Acknowledging community traditions and knowledge and wherever possible reinforcing this
through collaboration, e.g. by using traditional healers in community health activities.
Involvement in peace-building and community development activities whose benefits would last
beyond the term of the projects.
Involvement in agricultural and environmental interventions with a developmental focus, e.g. by
gradually shifting the seedlings production in the nurseries into small scale and manageable
community and group nurseries with proper training to interested people and enhancing direct
close corporation and collaboration with the Department of Forestry (DoF).
Development of clear exit plans communicated well in advance to communities.
Integration of health and hygiene promotion into a standard school curriculum in Kass.
Setting up of cost recovery measures through beneficiary committees and involving line
ministries in the process.
Supporting the return of some of the IDPs to their villages of origin.
However, several evaluations4 did identify gaps in ensuring connectedness in Tearfund
programming. Tearfund’s greatest challenge in 2012 evaluations was identified as the need to
maintain the sustainability of the project impacts and for the organisation to realise that the
continuation of a relief oriented modality will not achieve this. Tearfund was therefore advised to
enhance community ownership and take a more developmental approach in supporting the
recovery of people’s livelihood systems. In this regard, Tearfund was advised to be more
creative and exploit potential synergy between its sectoral areas, e.g., by making available
recreational materials for the children and providing equipment and materials for handicrafts and
income generating activities for the older students in children’s clubs. Evaluators also felt that
Tearfund had started income activities with Women- and Youth-Clubs but work has been
implemented with a ‘relief’ mindset. Such activities need to be based on a proper market
assessment, involve a technical and organisational training component and an agreement
between Women- or Youth-Clubs/Tearfund/HAC. Finally, it was also felt that the sustainable
4
Evaluation of Tearfund 2009-2011 Kass Project, Sudan; Evaluation of Tearfund DMT projects
2009-2011 in Ed Daein, Sudan; Evaluation of DMT Projects in Ed Daien, Beida, & Garsilla North Sudan
15
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
impact of seeds distribution appears to have been limited as most farmers have consumed the
harvest of their improved seed varieties. However, Tearfund was applauded too as it had later
adopted a more sustainable approach such as the establishment of seed banks.
Tearfund must also continue to follow up activities handed over to ministries, e.g., on the handover of water points to WES as communities complain about a lack of water from water points in
all Tearfund IDP camps with IDPs resorting to collect water at hand dug wells and from
unprotected wells. The maintenance of the wells, particularly the well heads, was a concern.
Although water committees have been trained, little evidence was observed of appropriate well
head care. For instance there were very few examples of the wells being fenced, animals were
encroaching right up to the water outlets, and the sites were usually poorly drained. It was
suggested that Tearfund review the effectiveness of the current water committee structure and
training.5
The Child Health Clubs were not seen as sustainable in its current form where continuation
depends on Tearfund making available incentives and free soap distributions. Tearfund was
advised to take a community based approach with an important role for traditional leaders and
CBOs, in particular the Women Committee. Tearfund should seriously consider investing in
more permanent structures for enhancing the sustainability of the clubs after Tearfund’s phase
out and seriously consider upgrading the clubs to meet the new 2011 UNICEF criteria. Tearfund
was also advised to have a more explicit focus on Community Based Disaster Risk Reduction to
enhance the resilience of communities in the face of climatic variability (e.g. by introducing
drought tolerant seeds), on-going volatility and displacement. It was also suggested to
strengthen Community Based Organisations and build linkages between them and relevant
Ministries to create ‘demand’ for government services to address basic and recovery needs of
communities.
Tearfund was advised to undertake a needs assessment into barriers and
drivers for return with programme implications for integrating host~IDP communities and
supporting voluntary return. Finally, it was advised to ensure that those families assisted
through the nutrition programme receive training in the nutritional properties of locally available
foods and their appropriate use so that their long-term nutritional habits change. The
sustainability of the nutritional interventions when Tearfund phases out was a main issue of
concern. Tearfund was advised to develop a comprehensive strategy for phasing out and
handing over to MoH by making sure that critical supply chains (Plumpy Nut and Unimix/CSB)
are in place. Tearfund was advised to consult with UNICEF in sustaining the nutritional projects
to discuss and inform its phase out strategy.
Coherence (coordination)
How was the programme integrated with the activities and priorities of other agencies
and organisations? To what extent did the programme and strategies cohere with local
and national policy? What facilitated/constrained coordination? To what extent did the
5
Evaluation of Tearfund 2009-2011 Kass Project, Sudan
16
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
programme cohere with Tearfund’s corporate disaster response strategy? Was the
programme consistent with the vision, values, strategy and resources of Tearfund?
Evaluators felt that Tearfund demonstrated competence in winning the respect of other actors,
both Governmental and Non Governmental. Tearfund was found to be very active within the
different interagency forums and has gained particular influence within specific sectors.
However, evaluators6 did advise Tearfund to use this positive status strategically to enhance the
value of its programmes through inter-agency collaboration by taking the following steps:
1. Gaining greater inputs from UN system; e.g., by i) requesting WFP to review the
reduction in food rations which likely has had a bigger impact on IDPs in Tearfund supported
camps in Kass as they have less opportunity to expand on agricultural production to make
up for the reduction in food rations; ii) Re-entering into an agreement with UNICEF on
WASH to gain free access to a wide variety of training materials, spare parts for pumps and
awareness raising materials; iii) Facilitate the MoE to enter into annual agreements with
UNICEF for the donation of visual aids and recreational kits.
2. Enhance coordination and sharing with other NGOs, e.g. i) Promote a common and
coherent approach by NGOs in establishing and strengthening CBOs and creating functional
linkages between CBOs and relevant government authorities; ii) Develop a Multi-agency
emergency preparedness and response protocol for swift action in the face of sudden IDP
influxes, iii) Share its approach with other INGOs/NGOs and decide on a shared common
strategy to promote seed banks in close consultation and co-ordination with MoA; iii) Not to
continue high capacity borehole and water yard rehabilitation but first sort out poor site
management and maintenance by the Drinking Water Corporation (DWC). Tearfund
together with other INGOs was advised to document persistent problems with the way DWC
manages the rehabilitated boreholes and water yards and advocate for improvements at Ed
Daein, Nyala and Khartoum level.
Evaluators7 advised that the relatively weak communication and co-ordination between
Tearfund and its operational partners within the field should be addressed since possibilities for
a more strategic, joint approach using the capabilities of the different partners have not been
utilized to the best advantage to date. As part of its partner policy Tearfund must clearly spell
out its added value in working with partners in particular sectors and geographic areas. In case
Tearfund does not opt for partnering Tearfund should clearly demonstrate its added value over
working through local NGOs. Tearfund was also advised to develop a more comprehensive
local partner strategy for Darfur setting clear selection criteria, introducing formal assessments
and critically evaluating partner projects.
With respect to coherence with national government policies, the main challenge during the
early years of the crisis was the government’s absence of a clear policy for humanitarian work
6
Evaluation of Tearfund 2009-2011 Kass Project, Sudan; Evaluation of Tearfund DMT projects 2009-2011 in Ed Daein, Sudan;
Evaluation of DMT Projects in Ed Daien, Beida, & Garsilla North Sudan; TEARFUND NORTH SUDAN EMERGENCY RESPONSE
June 2007 until September 2008
7
Evaluation of Tearfund 2009-2011 Kass Project, Sudan
17
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
and early recovery in Darfur. Presently, strategic documents such as the Darfur Development
Strategy have been generated by the government. However, Tearfund did collaborate
significantly with government ministries which approved of its approach and participated in its
programmes. Evaluations generally found highly positive views among government departments
about the positive impact of Tearfund’s work not only on communities but also the capacities of
government department themselves given that Tearfund worked in close collaboration with
them. In several cases, Tearfund was able to get government departments to contribute to
Tearfund’s work and to assume responsibilities for some of Tearfund’s work after its withdrawal.
For example, Ministry of Agriculture experts teach Tearfund beneficiaries on how to produce
pure seed and how to store them for the next season. However, evaluations8 did recommend
greater follow-up with and more capacity-building of line ministry partners. Finally, with respect
to coherence with Tearfund global strategies and visions, the Darfur programme seems largely
in line with Tearfund’s 12 global quality principles. Among these, it has been particularly
successful in meeting the standards about accountability, children/s development, technical
standards and conflict sensitivity though less so in the areas of gender transformation, advocacy
and DRR.
Coverage
Were the most vulnerable reached? Was the targeting appropriate? Who received
support and why? Was beneficiary selection impartial using clearly defined criteria?
There seems to be clear evidence that Teafund focused on the most vulnerable areas and
communities and adopted clear and impartial criteria for targeting. Firstly, Tearfund, focused on
highly isolated, off-the-beaten-track areas to work in. Thus, instead of working in El Fasher and
Nyala, the two more easily accessible operational areas which represent the biggest hubs of
NGO activities in Darfur with dozens of INGOs present, Tearfund selected more isolated areas
where needs were more intense while the number of NGOs was very low, with Tearfund usually
being one of 3-5 agencies present in the area. Secondly, Tearfund undertook in depth
quantitative surveys in the areas of nutrition and hygiene practices which further helped
Tearfund focus on the most vulnerable communities within these already isolated areas.
Although evaluations were generally positive about Tearfund’s targeting they did highlight some
areas for improvement. It was felt that the broader focus of the programme on long-term
displaced has limited the ability to respond effectively to the newly disenfranchised within the
specific areas of operation. Tearfund was also to be more sensitive towards the specific needs
of women in project design and implementation and pro-actively involve women to that purpose.
Tearfund was found to be ‘generalistic’ in its work with women; i.e., the organisation does not
focus on the needs of groups of women that are at higher risk such as female headed
households.
Tearfund was also advised to clarify/operationalize the concept of vulnerability (for staff and
beneficiary community) and make sure that staff operates from a shared understanding of what
vulnerability constitutes and how the project is going to address it. For example, evaluators
8
Evaluation of Tearfund 2009-2011 Kass Project, Sudan; Evaluation of Tearfund DMT projects 2009-2011 in Ed Daein, Sudan;
18
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
advised Tearfund to decide whether or not better-off farmers should benefit from its
interventions, who could function as lead farmers (referents) for enhancing food production and
income generating capacity. If selection criteria are widened Tearfund was advised to make
available more seeds (seen as an investment to establish community owned and managed
seed banks). Finally, for maximising programming impact Tearfund was advised to further
increase its understanding of people’s strategies to rebuild their food and livelihood systems and
improve targeting of beneficiaries to make sure that most vulnerable are included.
Effectiveness
Were outputs achieved? What factors influenced/constrained achievement? Were
purposes achieved? Were outputs of an appropriate technical quality? Was timely
provision of support, goods and services, achieved according to the perceptions of key
stakeholders?
By any account, Tearfund’s programme in Darfur represents a large and comprehensive
one with outputs spread over a wide range of critical sectors. Examples of Tearfund’s
cumulative outputs over the 10 years in major sectors were as follows:
Major Sectoral Outputs of Tearfund Programmes, 2004-2013
Nutrition
148,517 children and pregnant women provided
supplementary and therapeutic nutritional interventions
with
blanket,
Water
1,451 water points built or repaired
Sanitation
69,974 individual, communal and institutional latrines constructed
Food security
82,000 households provided with vegetable, crop and other income inputs
Education
6 schools and 125 classrooms constructed or repaired
Evaluations and project reports reveal that outputs and project targets were generally met and
often exceeded. Over the last 10 years, Tearfund projects have had nearly 3 million
beneficiaries (the actual number of people reached my be lower as multiple projects may have
served the same people). Discussions and interviews indicate that Tearfund were amongst the
earliest responders with an appropriate, well considered and planned response. Evaluators
found that interlocutors at various levels referred time and time again to Tearfund, as ‘the lead
agency’ or the ‘number one agency’ in a particular area. It may be technically incorrect to refer
to Tearfund as ‘The Lead Agency’ but this was an expression of appreciation rather than of
official appointment. The main factors that reduced effectiveness according to the evaluations
and project reports were as follows:
-Security situation and travel restrictions
-Government bureaucracy
-Insufficient capacity among staff, partners and line ministries
-Insufficient Tearfund capacity in developmental work due to the overall Darfur emergency focus
19
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
-Short-term project durations and donor horizons
In terms of appropriate technical standards, Tearfund was strongly advised by evaluators to
restructure the HPCA project to enhance its effectiveness. It was recommended to integrate HP
with Watsan into a new WASH project to strengthen synergies between hard- and soft-ware and
increase community ownership. Evaluators felt that Tearfund should consider incorporating
Children Activities as part of a new educational project strengthening the educational system
(focus on permanent structures), and school/community based Child Friendly Spaces and
children educational activities (Child Health Clubs and Environmental Clubs) to enhance their
effectiveness. Among the constraining factors was the fact that Tearfund’s interventions were
heavily dependent on short-term institutional funding, which has tended to reduce the ability to
respond to new critical needs as a consequence of continuing conflict and displacement. To
remain relevant in the current context, agencies need to maintain a degree of responsive
flexibility. Evaluators felt that greater attention and investment needs to be placed on the role of
policy change, with respect to the Darfur and wider Sudanese contexts, and the right to live in
safety and dignity for the displaced. Although a significant effort has been made at the field
level, it is felt that greater emphasis needs to be placed on this role at the UK / HQ on issues
like protection of women from combatants of all warring sides, adequate funding for the crisis
and children’s issues. Evaluators also felt that the impact of the food security initiatives has
been reduced, due to spreading the aid too thinly.
Efficiency
How cost-effective was the programme? Were objectives achieved on time and on
budget? How did Tearfund’s programmatic choices compare to other alternatives? What
were the major factors influencing the efficiency of the programme?
Darfur represents a difficult work environment for NGOs from the point of view of efficiency due
to several reasons which increase the cost of operations significantly. Firstly, Darfur, especially
the areas selected by Tearfund, is a very isolated working area with poor logistical and
communication infrastructure. Secondly, the insecure working environment also necessitates
significant investment in security protocols and equipment. Thirdly, the lack of local capacity due
to the war and the intense competition among a large number of agencies for skilled national
staff also necessitates the hiring of additional expatriate staff. Finally, a lot of the programme
inputs are not available within Darfur or even Sudan and have to be flown in. Even so, Tearfund
adopted a variety of strategies to enhance efficiency. Firstly, it emphasized cost-effective and
cheaper technology and inputs without sacrificing quality. For example, it used new technologies
such as soil stabilised blocks (SSBs) in the construction of permanent infrastructures (classrooms,
latrines, etc). This technology is not only cheaper compared to ordinary burnt bricks but is of a
higher quality and contributes to environmental sustainability, which resulted in significant cost
reductions in the construction of classrooms and latrines. Secondly, it designed and constructed
sustainable permanent structures such as water and sanitation infrastructure for longer term usage,
which enhanced the life of the infrastructure, thus reducing original construction cost per year of use,
and also reduced the cost of subsequent maintenance. Thirdly, enhancing community participation
in terms of the provision of labour as community input had dual benefits of reducing cost and
20
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
enhancing community ownership and sustainability. Fourthly, the partnership with OFDA, with its
annual financial audits, greatly benefitted the internal cost management and the policy and
procedures of Tearfund. Throughout the years many improvements were made regarding allocation
of in-country overhead, tender procedures, procurement planning and local market surveys etc, all
of which enhanced efficiency. Fifthly, over time Tearfund also reduced expatriates and enhanced
the capacity building of senior national staff and partners, mostly from 2010 onwards, reflecting the
changing situation in Darfur and creating the right structure for sustainability of the projects, as
well as enhancing efficiency. Sixthly, significant austerity measures were also put in place after the
2008 international economic crises and in-country management consistently worked towards the
reduction of operational costs of support hubs in Nyala and Khartoum. Finally, the use of
community cost-recovery measures and the use of beneficiary management committees also
reduced operational and monitoring costs. However, indirect costs remained high throughout the
project period due to the isolated locality of the project areas and the exceptionally high cost of
transportation, security, living conditions, basic services such as fuel, communication and other
support services which are the essential backbone of the programmatic implementation.
Thus, it is advised that Tearfund should look for other ways of improving cost efficiency. In some
cases economies of scale could be achieved in future by maintaining the number of targeted
interventions but in fewer sites. However, where coverage by other INGOs and UN is low‘,
Tearfund’s prescence it could be a justification of the current cost. Furthermore, Tearfund is
advised to reduce expatriate costs by developing a comprehensive national staff development
policy in particular with respect to its younger staff enhance the role of expat staff in coaching
and mentoring of national staff and local partner staff and optimise staff conditions in seeking
senior expat staff to serve longer with Tearfund.
Impact
What were the attributable intended and unintended effects (social, physical,
environmental, economic, spiritual), both positive and negative, of the programme on
beneficiaries and non-beneficiaries?
Tearfund has invested a high degree of effort in thorough needs assessments, independent
external programme evaluations and comprehensive project donor reports. These assessments,
reports and evaluations provide objective and verifiable information to help analyze the
outcomes and impact Tearfund’s programmes were able to produce in Darfur between 2003
and 2013 qualitatively but less so quantitatively. The following qualitative outcomes and impacts
emerge repeatedly in Tearfund’s programme evaluations and project reports:
1. Providing easier access to basic needs
The basic essentials of life were generally not available to the internally displaced people and
even the host communities in the highly isolated and harsh areas served by Tearfund. Thus, the
agency played a critical role in providing easy access to essentials such as water, sanitation
and household items, to these communities. The majority of households earlier collected water
from water points drilled over 10 years ago. Those in areas more directly affected by the
conflict were in dilapidated condition with broken down pumps and non-existent distribution
21
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
systems. In most cases the sanitary condition of the earlier water points was very poor, with
inadequate surface water drainage and no segregation of animals from drinking water collection
areas. Where water points were not functioning people were forced to use unprotected surface
water sources (ponds and puddles) or transport water over great distances by donkey (a six
hour one-way journey being typical). The evaluations repeatedly highlight the high degree of
gratitude expressed by communities towards Tearfund for the provision of these basic
necessities of life. Annually, Tearfund’s projects usually had between 200,000-500,000
beneficiaries. Tearfund’s surveys revealed that community knowledge about the benefits of
simple health practices, such as hand washing, ORS preparation and breast-feeding, increased
significantly. For example, in an ECHO-funded project in 2006 in South Darfur, a follow up KAP
survey found that there has been a greater than 30% increase in health knowledge among
children and women beneficiaries across indicators, such as hand-washing and soap use,
compared to the baseline situation.
2. Reducing
Reducing mortality and morbidity
Tearfund’s emergency programmes in several sectors played a critical role in reducing morbidity
and mortality within communities, especially its nutritional programmes which were implemented
in all locations. Among Tearfund’s wide range of activities, the nutritional inputs for children
played the most critical role in this regard. Tearfund’s assessments in Ed Daein from 2005 show
that global acute malnutrition (GAM) rates of nearly 25% and severe acute malnutrition (SAM)
rates in excess of 4% (both above critical levels) were common in the areas where Tearfund
worked. As a result of Tearfund’s project, it subsequently reduced to a GAM of 13.9% and SAM
of 1.4% (Ed Daein ECHO final report (2006)). Thus, Tearfund’s nutritional work controlled the
further spread of malnutrition and possible mortality. Furthermore, the vast majority of the
malnourished children also suffered from diarrhoea, malaria and other life-threatening
communicable diseases, which further enhanced their vulnerability. Tearfund’s water and
sanitation work further reduced the vulnerability of such children by reducing the chances of
diarrhoea. Finally, the hygiene promotion work complemented the water and sanitation work by
improving people’s hygiene practices and by reducing their vulnerability to diseases.
3. Enhancing selfself-reliance and livelihoods opportunities
Even though Tearfund was operating in an emergency environment, it focused increasingly over
time on enhancing people’s self-reliance by providing livelihoods opportunities in line with
people’s desires. These opportunities included crop and vegetable production, seed banks, tree
plantation, grinding mills, poultry and extension and training services. Beneficiaries’ own
accounts during evaluations and Tearfund surveys indicate that such interventions have
significantly improved people’s livelihood conditions. Seed deliveries and agriculture support
have made significant increases in local food production, income generation and the recovery of
the local rural economy. Besides providing practical assistance to communities in providing
fruits and holding soil together, trees plantation acted as a symbol of hope and peace as they
were planted with the knowledge that the real benefit of the trees will accrue years in the future
to all communities and different generations. Beneficiaries also reported satisfaction with the
extension support that was provided, particularly the messages about the importance of crop
22
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
spacing, natural fertiliser options and mulch. Tearfund’s projects have also contributed to
people’s return to their villages. For example, under a Tearfund project in Ed Daein between
2009-11, vegetable seeds and tools were distributed to 11,448 IDPs and 1,900 host population
households in 6 locations. In 2010, post distribution monitoring and follow-up covered a sample
of 600 sample households and confirmed 98% beneficiaries have reported receiving and
planting the vegetables. A total of 30,000 different tree seedlings were distributed in 6 locations.
Subsequent tree survival survey revealed a 70% survival rate. A 2009 project in Garsilla
focused on agricultural inputs and training provision. A post-harvest survey, conducted in
January 2010, indicated a 23% increase in yield over earlier levels.
4. Strengthening community participation, capacity and organization
Evaluations9 generally found that all aspects of Tearfund’s programme showed appropriate
levels of community participation, including identifying the location of hardware (latrines,
boreholes, etc.) activities; the selection of food security programmes at the request of the
community and community contributions (provision of labour and materials) in various project
components. Tearfund also set up committees for maintaining and repairing the hardware
components, and collecting user charges wherever appropriate. The training of individuals in
hardware skills, e.g. in maintenance of water infrastructure and construction of concrete latrine
slabs, increased the capacity of communities with a potential multiplier effect through
employment in the future. The software skills (organisational, management, revenue collection,
book-keeping, etc.) of water committees were also enhanced through trainings. The high levels
of participation correspondingly ensured high levels of accountability to the community given
that the former is a key dimension of the latter. The high level of participation ensured by
Tearfund further strengthened the social capital present in communities.
5. Enhancing mental health and normality among children
The Darfur conflict exposed children to a high degree of psychological trauma. Tearfund’s
programmes were successful in reducing the trauma through child-focused activities. Tearfund
set up numerous children’s clubs and friendly spaces, providing a much needed routine and
stable and stimulating environment that helped children establish some degree of normality.
Tearfund also had children carry messages to their home to bring about health practice
improvements. School teachers reported that children who had attended a Tearfund children
club made much better students compared with other children. Such children were better
disciplined in class, had better skills in basic reading and writing, more familiar with hygiene
routines and more willing to assuming responsibilities.
6. Enhancing
Enhancing peace and reconciliation
Tearfund’s programme was also commended in evaluations for the high level of sensitivity,
flexibility and appropriateness exhibited given the complex and fragile conflict situation.
Evaluations found that the inter-community relationships developed through Tearfund’s water
and sanitation and children programmes had provided a foundation for future peace activities.
For example, as Arab nomads and Masalit children attended the same children’s clubs in Beida,
9
Evaluation of Tearfund 2009-2011 Kass Project, Sudan; Evaluation of Tearfund DMT projects 2009-2011 in Ed Daein, Sudan
23
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
initial tensions were replaced by acceptance, in the process establishing mutual community
relationships which could positively influence the current and future interaction. The equitable
provision of services and benefits to all communities without distinction in Tearfund programmes
also helped reduce the threat of further violence. The presence of children’s clubs was seen by
beneficiaries as potentially reducing the likelihood of older children joining militias and
perpetuating the conflict. Tearfund also provided openings for inter-tribal dialogue for localised
conflict resolution. This was an unexpected outcome from the establishment of village project
committees, which eventually led in some cases to dialogue across different tribes. For
example, in a 2007-08 project in Garsilla 30 community leaders from a range of ethnic groups
across the region met 6 times to interact and discuss common issues, including decision making
and problem solving for common resource management, and motivation and encouragement for
relationship-building among the tribes,
7. Increasing sense of security
Tearfund’s continuing presence has increased the communities’ sense of security and has. By
providing water points and latrines within camps, Tearfund’s sanitation component has removed
the need for beneficiaries to expose themselves to the danger of travel beyond the camp
perimeter. Tearfund has also been sensitive to the suggestions of communities in locating its
water and sanitation facilities in line with their security concerns. There is a widespread
perception amongst communities that Tearfund has persisted in rendering services to
communities in areas that other agencies have considered too risky. This has especially led to a
deep sense of appreciation amongst the targeted communities along the Chad border.
8. Enhancing capacities of government departments
Multiple evaluations10 found highly positive views among government departments about the
positive impact of Tearfund’s work not only on communities but also the capacities of
government department themselves given that Tearfund worked in close collaboration with
them. In several cases, Tearfund was able to get government departments to contribute to
Tearfund’s work and to assume responsibilities for some of Tearfund’s work after its withdrawal.
The Ministry of Education is reviewing the possibility of adopting the Tearfund children’s clubs
as pre-schools in order to improve their sustainability. Tearfund water committees report that the
Ministry of Water and Sanitation (WES) provides spare parts free of charge to hand pump
technicians and water yard operators for the water points constructed by Tearfund. In
September 2012, Tearfund started the process of handing over IDP camp water supply
systems, motorised boreholes and hand pumps to the Ministry and the community. Coordination with WES on introducing partial cost recovery from communities through a monthly
fee for the generator fuel has also been good, with WES playing a leading and central role.
Finally, Ministry of Agriculture experts teach Tearfund beneficiaries on how to produce pure
seed and how best to store them for the next agricultural season.
9. Enhancing environmental protection
10
Evaluation of Tearfund 2009-2011 Kass Project, Sudan; Evaluation of Tearfund DMT projects 2009-2011 in Ed Daein, Sudan
24
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
The Tearfund projects contained intentional components of environmental rehabilitation and
mitigation of the environmental degradation. Tearfund facilitated the planting of saplings in the Beida
localities, and is largely credited with the greening of Beida Town. It promoted the planting of trees in
all infrastructure facilities constructed in Beida and Um Dukhun localities not only for the sake of
providing a shade but also for protection from effects of storms and strong winds. Tearfund also
promoted the use of environmentally-friendly construction blocks in Beida & Um Dukhun thereby
reducing damage to an otherwise larger volume of trees that would be required for firing the bricks.
Tearfund undertook joint ground water monitoring with WES and agreed on appropriate garbage
disposal points with health authorities. At Beida, Tearfund encouraged the transportation of garbage
out of the main towns/populated centers. The projects have also taken care of environmental
mitigation during the design and construction of different facilities e.g. water systems, sanitation and
school classrooms etc. For example: a) proper drainage systems were designed and constructed
around the water points, boreholes hand dug wells in order to control excess waste water filtering
back into the boreholes/wells, negatively affecting water quality; b) engineering safety and control
was implemented during the construction of different structures, i.e. not to cut or damage existing
trees around construction sites due to excavation, and also backfilling of the trenches after
completion of the construction; c) raising community awareness regarding proper utilisation of
facilities (separating clothes washing, animal drinking and drinking water collection points).
However, several evaluations11 felt that it is difficult to verify the actual impact on meeting basic
needs and the restoration of livelihood options quantitatively as there are no immediately
quantifiable monitoring systems for measuring impact since KAP and nutritional surveys could
not be done regularly due to government lack of approval and other reasons internal to
Tearfund. Where such information was available in some cases, nutrition survey findings
indicated that the mortality in <5shad reduced and the incidence of diarrhoea had also dropped
by over 20%. KAP survey reported significant positive changes in preventative health
knowledge. However, one evaluation team questioned the reliability of the KAP survey findings
given the inadequate sampling procedure and recommended that Tearfund should improve the
rigour of their KAP survey process so as to insure the findings are representative and can
reliably be generalised. It was also suggested for Tearfund to seriously consider appointing a
senior and well experienced M&E Officer at Khartoum level to provide neutral and authoritative
feedback.
11
Evaluation of Tearfund’s Water, Sanitation and Health Education Project in SW corridor of West Darfur, North Sudan, May 2005;
Evaluation of Tearfund DMT projects 2009-2011 in Ed Daein, Sudan.
25
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
CONCLUSIONS
Since 2004, Tearfund’s projects have provided critical services to almost 3 million beneficiaries
in some of the most isolated, insecure and badly affected parts of Darfur. Tearfund’s
programmes have combined the delivery of critical life-saving emergency inputs in the sectors
of nutrition, water, sanitation and health promotion with longer-impact work in the areas of
education, food security and community development. The programmes have been delivered
with a high degree of community participation and in close coordination with government
agencies. Tearfund’s ability to provide these critical and diverse services has dependent on its
success in garnering continued funding from a wide range of donors. Tearfund’s programmes
have not only ensured access to basic needs and reduced mortality and morbidity, but have
also enhanced people’s self-reliance, a sense of normality among children and have enhanced
the capacities of communities and government departments. Finally, they have also enhanced
peace and reconciliation at the local level and increased people’s protection against physical
dangers. Thus, Tearfund’s Darfur programme is a strong example of providing emergency and
long-term support in a precarious, complex and prolonged crisis.
Evaluation Ratings
Ratings of Tearfund Programmes, 20042004-2013
1
2
3
4
5
6
Average
Effectiveness
3.25
3.00
3.50
3.53
3.75
3.60
3.44
Impact
3.63
3.17
3.50
4.23
3.25
3.80
3.59
Relevance
4.00
3.33
3.50
4.48
4.75
4.20
4.04
Efficiency
3.00
3.67
3.50
3.45
3.50
3.40
3.42
Coverage
3.50
3.00
3.50
Connectedness
2.75
2.50
3.50
Coherence
2.50
3.17
3.50
Average
3.23
3.12
3.50
3.33
3.20
3.75
2.40
3.02
3.06
3.78
3.80
3.48
3.48
The table above presents the quantitative scores given by six different evaluations to Tearfund
programmes over the years on the DAC criteria. It reveals that Tearfund’s best performance
consistently over the years was in terms of relevance (average score 4.04) while the lowest score
was on connectedness (average score 3.02) though even this lowest score was over 3 out of 5.
Performance along most criteria is also improving gradually over the years, except for a slight dip in
the last evaluation in 2012.
At the same time, there are several issues which were raised by the two evaluations conducted for
2005 programmes and which continued to be raised even in evaluations in 2011 and 2012,
suggesting that TEARFUND was not able to find appropriate solutions for them over the years.
These issues include:
•
Greater focus on livelihoods and developmental programmes as desired by communities
26
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
•
•
High turnover and short duration of contract among expatriate staff
Lack of adequate quantitative monitoring information available for evaluations
These conclusions also match with this evaluator’s own subjective assessments in reviewing a wide
range of project documents and talking to key staff. Based on these factors, this evaluation assigns
the following ratings to the 10-year Tearfund programme:
Overall Rating
Rating of Tearfund 1010-year Programme,
Programme, 20042004-2013
Effectiveness
The programme makes a substantial contribution to the aspect
Impact
The programme makes an acceptable contribution to the aspect; there are
shortcomings that could be addressed
Relevance
The programme makes a substantial contribution to the aspect
Efficiency
The programme makes an acceptable contribution to the aspect; there are
shortcomings that could be addressed
Coverage
The programme makes a substantial contribution to the aspect
Connectedness
The programme makes an acceptable contribution to the aspect; there are
shortcomings that could be addressed
Coherence
The programme makes an acceptable contribution to the aspect; there are
shortcomings that could be addressed
27
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
RECOMMENDATIONS
Based on this rating of Tearfund’s programmes along different DAC criteria, the following
recommendations are provided. All recommendations here reflect the current evaluator’s own
assessment but since the current evaluator did not undertake field visits are also based on more
than one earlier evaluation suggestion, except for the first recommendation under “impact”
which is based solely on the current evaluator’s own judgment and experience:
Coherence
1) Enhance collaboration with UN agencies to leverage more inputs and expertise from
them for Tearfund and partners
2) Enhance collaboration and sharing with other INGOs to influence industry-wide
programme quality and standards in different sectors.
3) Improve coordination with partners. In particular, Tearfund must clearly spell out its
added value in working with partners and clearly demonstrate its added value over
working through local NGOs where partnership is not adopted. Tearfund is also advised
to develop a more comprehensive local partner strategy for Darfur setting clear selection
criteria, introducing formal assessments and critically evaluating partner projects.
4) Enhance follow-up and capacity-building of partner ministries to ensure that programme
quality standards are met after Tearfund’s withdrawal.
Connectedness
1) Increase focus on developmental livelihoods work to increase people’s self-reliance by
increasing its own capacity in such work, advocating with donors about the importance
of such work and by exploring synergies between livelihoods work and traditional
Tearfund sectors.
2) Review the sustainability of the current model of Children’s Clubs based on inputs by
Tearfund and instead explore the possibility of making them more sustainable through
greater community mobilization and empowerment of women’s committees.
3) Undertake an analysis of the factors which are constraining the return of people to their
villages even where the security situation is stable and aim to address the factors
through programming, policy and networking.
4) Develop a more explicit focus on Community Based Disaster Risk Reduction to enhance
the future resilience of communities in the face of climatic variability (e.g. by introducing
drought tolerant seeds), on-going volatility and displacement.
28
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
5) Strengthen Community Based Organisations and build linkages between them and
relevant Ministries to create ‘demand’ for government services to address basic and
recovery needs of communities once Tearfund withdraws.
6) Enhance the sustainability of nutrition programmes by educating beneficiary families on
nutritional education and by developing better hand-over plans with MOH and UNICEF.
7) For maximising programming impact, further increase Tearfund understanding of
people’s own strategies to rebuild their food and livelihood systems.
Coverage
1) Improve focus on newly displaced people by striking a better balance between their
needs and those of longer-term displaced people and having larger emergency
response programmes
2) Be more sensitive towards the specific needs of women in project design and
implementation, pro-actively involve women to that purpose and develop stronger focus
on the needs of groups of women that are at higher risk such as female headed
households.
3) Clarify/operationalize the concept of vulnerability (for staff and beneficiary community)
and make sure that staff operates from a shared understanding of what vulnerability
constitutes and how the project is going to address it.
Effectiveness
1) Restructure the HPCA project to enhance its effectiveness by integrating HP with
Watsan into a new WASH project to strengthen synergies between hard- and soft-ware
and increase community ownership.
2) Incorporate Children Activities as part of a new educational project strengthening the
educational system (focus on permanent structures), and school/community based Child
Friendly Spaces and children educational activities (Child Health Clubs and
Environmental Clubs) to enhance their effectiveness.
3) Develop a more flexible and longer-focused funding base given that the situation in
many areas is moving towards developmental needs.
4) Place greater attention and investment on policy in UK / HQ.
Efficiency
1) Review the possibility of reducing the number of work sites to enhance efficiency while
ensuring that this does not reduce the ability to hear the people’s voices by maintaining
links with the committees set up earlier
29
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
2) Reduce expenditures on expatriate staff by developing a comprehensive capacitybuilding programme for national staff
3) Review the possibility of working with local partners as a way of increasing efficiency.
4) Increase synergy and coordination across sectors as a way of increasing efficiency, e.g.,
between hygiene promotion and watsan
Impact
1) Enhance the ability of Tearfund to demonstrate impact through improving
documentation. In particular consider setting up a global user-friendly data system under
which each emergency programme regularly inputs information about outcomes and
impact indicators under each of the DAC criterion which are increasingly being used by
many donors and by negotiating with those which do not use them to accept the DAC
criteria
2) Enhance the capacity of the M&E function within the country programme by recruiting a
senior M&E country-wide manager.
Relevance
1) Assessments and programme designs should include a stronger developmental focus
since communities now expected more livelihoods interventions
2) Develop a more comprehensive strategy for its Kass operation, based on a proper
needs assessment that focuses on hand-over of selected project activities in Kass to
relevant line Ministries, local NGOs and the community.
3) Enhance the thoroughness of accountability processes across geographies, sectors and
programme phases.
4) Reintroduce the system of having the Sudan team provide written feedback on each
evaluation
and
developing
a
detailed
feedback
incorporation
plan.
30
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
ANNEXES
Terms of Reference for the Evaluation
Title:
Darfur 2013 Programme Evaluation
Consultancy Reference:
SUDAN/EVAL/12/12
Location of Consultancy:
Desk-based evaluation of operational programme in Sudan
Expected Start Date:
Early January 2013
Expected End Date:
Mid February 2013
Duration:
25-30 working days
BACKGROUND
Tearfund is a Christian relief and development agency with 40 years’ experience working in over
70 countries with over 290 local partners to eradicate poverty. Tearfund has responded through
direct operational responses to disasters in Mozambique, Latin America, Albania, Burundi, South
Sudan, Liberia, Sierra Leone, Afghanistan, northern Kenya, Kosovo, Serbia, Democratic Republic
of Congo, Sri Lanka, India, Pakistan and Indonesia in addition to Sudan. Further details on
Tearfund’s international work can be found at its website: www.tearfund.org.
·
Programme: Sudan (Darfur) operational programme
·
Background to programme:
Tearfund has been active in Darfur since 2004 and seeks to enable conflict-affected and
vulnerable people (IDP, host, rural populations, nomads and returnees) to meet their basic needs:
to live in safety and access nutritional support, safe water, sanitation and health/hygiene
education. In addition, Tearfund seeks to help target populations improve their food security,
livelihoods, environment and access to education. The total number of current beneficiaries of
Tearfund’s programme in Sudan is approx. 300,000 people per year.
·
Current Activities:
Tearfund currently has four project locations: Ed Daein in East Darfur, Um Dukhun and Nertiti in
Central Darfur and Kass in South Darfur. Project locations are supported by two hub offices in
state capitals (Zalingei, Central Darfur and Nyala, South Darfur) and a programme HQ in
Khartoum. Previous project locations in Garsila/Wadi Salih and Beida were closed in 2010 and
2011 respectively.
Donors: Tearfund has a strong donor support base. Recent and current donors include DFID,
ECHO, OFDA, CIDA, SGID (Scottish government), EU Devco, CHF (UN pooled funding), DCPSF
and various Tearfund partners as well as Gift in Kind (GIK) donations from UNJLC (UN Joint
logistics centre), WFP and UNICEF. Expected turnover for 2012/13 is approx £4,7 million.
·
31
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
Previous reports and evaluations: (the list provided below may not be exhaustive).
2005
● Wadi Salih Integrated Public Health Programme 29 Nov-12 Dec 2005 by Jean
MacCluskey
●
2006
●
2007
●
Evaluation of Tearfund’s Water, Sanitation and Health Education Project in SW corridor of
West Darfur, North Sudan, May 2005 by James Webster
External appraisal report, ECHO/SDN/BUD/2006/01020, Field appraisal by Frederic
Deparis 13-14, June 2006
Wadi Salih, BUZA, Watsan/PHE/ HHFS/CD by Cutter, McKemey and van Uffelen
●
Ed Daein, Darfur, ECHO. Watsan/Nutrition/PHE by McKemey
●
Darfur Chad 04-07 appeal by Cutter, McKemey and van Uffelen
●
West Darfur, BUZA, 2006 by Cutter, McKemey and van Uffelen
●
Ed Daein, Darfur, ECHO 2007 by Cutter, McKemey and van Uffelen
●
Darfur: Relief in a Vulnerable Environment by Brendan Bromwich, Dr Abuelgasim, Abdalla
Adam, Dr Abduljabbar Abudulla Fadul, Florence Chege, Jim Sweet, Victor Tanner & Geoff
Wright (Tearfund, 2007)
●
Darfur: Water supply in a vulnerable environment, (Tearfund, 25 October 2007)
2008
●
●
2009
●
2011
●
Tearfund North Sudan Emergency Response (June 2007-September 2008), Evaluation of
the ECHO funded programme by Sam Rutherford and Stephen McDowell
North Sudan Programme Community Based Therapeutic Feeding Programs Analysis &
Recommendations, June 2008 by Henry Reitzug
Evaluation of DMT Projects in Ed Daien, Beida & Garsilla North Sudan by Sam Rutherford
and Stephen McDowell, February 2009.
Evaluation of Tearfund DMT Projects 2009-2011 in Ed Daein, Sudan by van Uffelen.
Summary of programme strategy: Tearfund’s vision is:
To see 300,000 IDPs, returnees and host community members transformed into integrated,
peacefully coexisting, resilient and empowered communities. To be achieved through
efficient direct operational programmes and partnerships with local NGOs and line
ministries,demonstrating sustainability and the ability to phase out.
Tearfund’s programme in Sudan works through two approaches:
32
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
1.
Relief: provision of basic services to new IDPs and longer-displaced IDPs who have proved
inaccessible until now. Immediate basic needs will be met through water, sanitation, and nutrition
interventions in existing IDP camps and surrounding communities. Trained staff and emergency
stocks will be used whenever new displacements occur. Relief programming will place greater
emphasis on sustainability including hand over of facilities and/or activities to CBOs, local NGOs
and government bodies.
2.
Recovery: Building stable communities, (re-)integrating returnees and/or IDPs with host
communities to develop peacefully co-existing and resilient communities. Basic service delivery
will be complemented by work on disaster management and community peace building activities
through a strong community development focus.
In Darfur, Tearfund focuses on Water, Sanitation and Hygiene (WASH), Food Security,
Nutrition, Education, Environmental Sustainability, Disaster Risk Reduction, and
Emergency Preparedness and Response.
·
How the need for the requested assignment arose:
Tearfund has worked in Darfur since 2004. It is seeking to take stock of what it has
achieved but also to critically assess the 9 years of its programming in Darfur. Previously,
evaluations have been carried out of specific projects or sectors at one point in time, but no
evaluation has been carried out of the programme as a whole covering the duration of its
existence. The time period covered by this evaluation will encompass evolutions in context and
strategy (in-country and corporately within Tearfund), multiple changes of personnel, and changes
in how Tearfund’s conducts operational humanitarian responses. Given Tearfund’s operational
interventions in a number of protracted humanitarian crises (DRC, South Sudan, Afghanistan), it
is hoped that this type of learning will be applicable beyond the confines of Sudan. It is intended
that this evaluation will be useful in formulating practical suggestions for how Tearfund can
continue to improve the quality and appropriateness of its humanitarian interventions. Such
learning would also be applicable to the wider NGO and donor community.
PURPOSE
The aim of this assignment is twofold: firstly, to establish a summary record of Tearfund’s
achievements in Darfur, and secondly, to critically assess Tearfund’s operations in Darfur
since 2004 through an evaluation of humanitarian action (EHA).
1. Summary of achievement
The summary of achievement would seek to ennumerate and describe what Tearfund has done in
Darfur over the last 9 years. This would include an account of the context of Tearfund’s operations
and highlight instances where Tearfund has contributed to significant change. While Tearfund has
repeatedly recorded achievement at project-level, no efforts have yet been made to bring all this
information together into a single narrative.
2. Evaluation of humanitarian action
The Active Learning Network for Accountability and Performance in Humanitarian Action (ALNAP)
defines an EHA as ‘a systematic and impartial examination of humanitarian action intended to
draw lessons to improve policy and enhance accountability.’ An evaluation therefore has two main
functions: to strengthen accountability and to increase learning. The EHA will evaluate Tearfund’s
operational programme in Darfur since 2004 according to EACD-DAC seven criteria for complex
emergencies: relevance/appropriateness, connectedness, coherence (coordination[1]), coverage,
efficiency, effectiveness, and impact. Questions in italics below indicate suggested lines of
enquiry to explore each criterion.
33
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
●
Relevance/Appropriateness – ‘Relevance is concerned with assessing whether the
project is in line with local needs and priorities’. ‘Appropriateness is the tailoring of
humanitarian activities to local needs, increasing ownership, accountability and costeffectiveness accordingly.’
○ Was need adequately assessed?
○ Did the programme’s design show understanding of and support for the capacities
of the affected population?
○ What other alternatives were available and why were they rejected?
○ Did the operational intervention address communities’ own relief and recovery
priorities? Were strategies appropriate and relevant?
○ Were recommendations from evaluations and key learning points implemented?If
not, why not?
○ Were appropriate systems for participation, information sharing and feedback, put
in place and used by project participants?
○ Did the feedback received shape programme design and implementation?
○ Was the programme consistent with the vision, values, strategy and resources of
Tearfund?
○ How do beneficiaries and other stakeholders describe the quality of relationships
with programme staff?
●
Connectedness – ‘refers to the need to ensure that activities of a short-term emergency
nature are carried out in a context that takes longer term and interconnected problems into
account.’
○ Has the response reduced the vulnerability of beneficiaries to future disasters?
○ To what extent has local capacity been supported and developed?
○ Has/will the benefits of the intervention continue beyond the intervention’s end?
●
Coherence (coordination) [2] – ‘[t]he need to assess security, development, trade and
military policies as well as humanitarian polices to ensure that there is consistency and, in
particular, that all policies take into account humanitarian and human-rights
considerations.’
○ How was the programme integrated with the activities and priorities of other
agencies and organisations?
○ To what extent did the programme and strategies cohere with local and national
policy?
○ What facilitated/constrained coordination?
○ To what extent did the programme cohere with Tearfund’s corporate disaster
response strategy?
●
Coverage – the need to reach major population groups facing life-threatening suffering
wherever they are.
○ Were the most vulnerable reached?
○ Was the targeting appropriate?
○ Who received support and why?
○ Was beneficiary selection impartial and according to clearly defined criteria?
●
Efficiency – measures the outputs – qualitative and quantitative – achieved as a result of
inputs. This generally requires comparing alternative approaches to achieving an output,
to see whether the most efficient approach has been used.
○ How cost-effective was the programme?
34
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
○
○
○
●
●
Were objectives achieved on time and on budget?
How did Tearfund’s programmatic choices compare to other alternatives?
What were the major factors influencing the efficiency of the programme?
Effectiveness – measures the extent to which an activity
whether this can be expected to happen on the basis of the
criterion of effectiveness is timeliness.
○ Were outputs achieved?
○ What factors influenced/constrained achievement?
○ Were purposes achieved?
○ Were outputs of an appropriate technical quality?
○ How did beneficiaries and other stakeholders describe
with project staff?
○ Was timely provision of support, goods and services,
perceptions of key stakeholders?
achieves its purposes or
outputs. Implicit within the
the quality of relationships
achieved according to the
Impact – looks at the wider effects of the project – social, economic, technical,
environmental – on individuals, gender, and age-groups, communities and institutions,
Impacts can be intended and unintended, positive and negative, macro (sector) and micro
(household).
○ What were the attributable intended and unintended effects (social, physical,
environmental, economic, spiritual), both positive and negative, of the programme
on beneficiaries and non-beneficiaries?
METHODOLOGY
Phone interviews (past and present Tearfund Sudan staff, other stakeholders as relevant to
the evaluation)
Desk based review of previous evaluations, project reports, strategy documents and other
relevant documents.
WORKPLAN & TIMETABLE
Workplan to be arranged with the consultant but will include briefing and de-briefing.
Given the nature of this consultancy it is expected that the design of the workplan will be
formally reviewed within a week of the evaluation starting.
MANAGEMENT OF VISIT
The Sudan Programme Officer (Patrick Crowley) is commissioning this review on behalf of
Tearfund.The consultant should refer to the Sudan Programme Officer to resolve any issues
related to the review. The Sudan Programme Officer is expected to provide regular assistance
Tearfund will provide a briefing of the programme, documentation, sensitivity guidelines and
sign off procedures at the start of the consultancy.Tearfund will provide a desk, phone access
and IT whenever the consultant works at Tearfund’s headquarters in Teddington. Tearfund will
arrange remote access to Tearfund’s server to enable the consultant to access relevant
documents on the T-drive when working away from Teddington. When working away from
Teddington the consultant, will be responsible for providing their own computer. The consultant
will be responsible for any travel and accommodation arrangements connected to the
review.The consultant will not need to travel to Sudan. The consultant will be expected to use
their knowledge and prior experience of Darfur to inform their analysis.
EXPECTED OUTPUT
1. Summary of achievement:
35
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
An 8-10 page summary document suitable for external dissemination recording Tearfund’s main
achievements in Darfur since the inception of its programme. Such a document would include:
A brief account of the context which precipitated the programme’s intervention and an
account of how this context has evolved.
Breakdown of achievement by location and sector.
Conclusion.
2. EHA
An evaluation of humanitarian action report (in Tearfund standard reporting format: please refer to
the Consultants’ Briefing Pack) with the following sections:
● Executive Summary (no more than four A4 sides)
● Introduction / Background
● Methodology
● Context Analysis
● For each criteria:
○ Findings
○ Conclusions
○ Assessment
● Specific Actionable and Prioritised Recommendations
● Annexes (indicative)
○ Terms of Reference for the Evaluation
○ Profile of the Evaluation Team
○ Evaluation Schedule
○ Protocols for the Evaluation
○ Documents consulted during the Evaluation
○ Persons participating in the Evaluation
○ Field data used during the Evaluation, including baselines
○ Bibliography
For each of the criteria outlined under ‘Purpose’, the consultant is required to make a clear
statement of the consultant’s assessment of the programme’s performance. The consultant may
wish to consider using the following four-point scale to score the programme’s achievements for
each of the criteria:
the programme makes no contribution to the aspect;
the programme makes a minimal contribution to the aspect; there are major shortcomings that
must be addressed;
the programme makes an acceptable contribution to the aspect; there are shortcomings that
could be addressed; or
the programme makes a substantial contribution to the aspect.
INTENDED USE OF THE EXPECTED OUTPUT
An evaluation is not useful if the recommendations and lessons learnt are left on the shelf, not
being read or utilized. To ensure that these recommendations and lessons are not ‘lost’, the
programme staff and UK Team are required to respond to each evaluation. It is the responsibility
of the programme to address those recommendations that relate directly to the programme, by
drawing up a management response and action plan. It is the responsibility of the UK team to
ensure transferrable lessons learnt and recommendations are captured and disseminated as part
of Tearfund’s evaluation and learning system.
REQUIRED INPUTS
36
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
·
People to be involved: Tearfund staff (current and former), partners, community leaders,
NGOs, government officials.
EVALUATION OF CONSULTANCY
·
Who in Tearfund will review and comment on the report:
Sign off: Deputy Head of West and Central Africa and Sudan Country Director.
PROFILE OF THE EVALUATOR
ATTRIBUTE/SKILL
EDUCATION/
QUALIFICATIONS
EXPERIENCE
SKILLS/ABILITIES
PERSONAL QUALITIES
ESSENTIAL
DESIRABLE
Degree level qualification
Relevant Masters
level qualification
Experience with an international
NGO in a relief/rehabilitation setting
Darfur experience
Previous consultancy experience
Previous report writing
experience (will be required to
provide examples)
Consultancy design
Good communicator
Excellent interpersonal skills
Excellent report writing
Proven evaluation skills
Independent thought
Excellent time management
Flexible team member
Able to use initiative
Team player
Organized
Trustworthy
Senior
programme
management
experience
[1] Coordination was considered, but ultimately rejected by DAC as a formal criterion. It is
accepted as complementary to coherence.
[2] In this instance, coherence is to be analysed solely within the humanitarian sphere.
Profile of the Evaluator
The evaluation was conducted by Dr. Niaz Murtaza at the University of California, Berkeley who
has more than 15 years’ experience in emergency response, DRR activities and implementation
of international technical standards in more than 40 countries in Asia, Africa and Americas with
reputed agencies such as IRC, Oxfam and Actionaid. In his job as International Program
Manager, Emergencies for ActionAid, Niaz was the agency’s international lead person for the
implementation of all emergency response and DRR work. He was also responsible for
mainstreaming the Hyogo Framework, NGO code of conduct, Sphere guidelines and Red Cross
Code of Conduct within all Actionaid emergency programs. During a career spanning fifteen
37
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
years, he has gained invaluable experience in implementing DRR projects at the grass-roots
level in more than 15 countries. He is well versed with the Hyogo Framework and its application
in disaster-prone regions. Niaz has a Ph.D. from the University of California at Berkeley in
community-level sustainable development issues among disaster-prone communities and is
currently working there with a research focus on accountability and impact assessment issues.
(niaz@berkeley.edu)
Evaluation Schedule
Dates
Work
January 10-26
Review of documents and staff interviews
January 28-February 2
Preparation of Summary of achievements and EHA
February 4-8
Review of Summary of achievements by Tearfund
Preparation of EHA (contd)
February 11-15
Revision of Summary of achievements by consultant
Review of EHA by Tearfund
February 18-22
Revision of EHA by consultant
Documents consulted during the Evaluation
Internal project proposals, reports, and strategy documents
External reports and evaluations:
2005
● Wadi Salih Integrated Public Health Programme 29 Nov-12 Dec 2005 by Jean
MacCluskey
●
2006
●
2007
●
Evaluation of Tearfund’s Water, Sanitation and Health Education Project in SW corridor of
West Darfur, North Sudan, May 2005 by James Webster
External appraisal report, ECHO/SDN/BUD/2006/01020, Field appraisal by Frederic
Deparis 13-14, June 2006
Wadi Salih, BUZA, Watsan/PHE/ HHFS/CD by Cutter, McKemey and van Uffelen
●
Ed Daein, Darfur, ECHO. Watsan/Nutrition/PHE by McKemey
●
Darfur Chad 04-07 appeal by Cutter, McKemey and van Uffelen
●
West Darfur, BUZA, 2006 by Cutter, McKemey and van Uffelen
●
Ed Daein, Darfur, ECHO 2007 by Cutter, McKemey and van Uffelen
38
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
●
Darfur: Relief in a Vulnerable Environment by Brendan Bromwich, Dr Abuelgasim, Abdalla
Adam, Dr Abduljabbar Abudulla Fadul, Florence Chege, Jim Sweet, Victor Tanner & Geoff
Wright (Tearfund, 2007)
●
Darfur: Water supply in a vulnerable environment, (Tearfund, 25 October 2007)
2008
●
●
2009
●
2011
●
Tearfund North Sudan Emergency Response (June 2007-September 2008), Evaluation of
the ECHO funded programme by Sam Rutherford and Stephen McDowell
North Sudan Programme Community Based Therapeutic Feeding Programs Analysis &
Recommendations, June 2008 by Henry Reitzug
Evaluation of DMT Projects in Ed Daien, Beida & Garsilla North Sudan by Sam Rutherford
and Stephen McDowell, February 2009.
Evaluation of Tearfund DMT Projects 2009-2011 in Ed Daein, Sudan by van Uffelen.
Persons participating in the Evaluation
Bethan Gilbert – former Operations Officer 10-11
Cressida Thompson – former Operations Manager 08-09
Eleanor Tuck – original Programme Director and former Operations Manager
Tim Holmes former Deputy Country Director (06-08) and Country Director (09-11)
Jane Petty – former Deputy Country Director 07-09
39
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
40
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
Title
Tearfund Integrated Emergency
Nutrition, Water, Sanitation and
Health Promotion Project
-do-
Summary of Tearfund Darfur Projects, 2004-2013
Donor
Sector
Location
Period
Benefi
Amount
ECHO
Nutrition, Water, Sanitation and Health
Promotion
South Darfur, Ed
Daein
8/05 to
02/06
65,500
E460,000
ECHO
-do-
-do-
60,200
E460,000
-do-
-do-
49,200
$374,498
-do
WRC/
CFGB
ECHO
4/06 to
12/06
-do-
-do-
-do-
102,136
€ 621,156
-do-
ECHO
-do-
South Darfur, Ed
Daein
1/07 to
12/07
1/08 to
12/08
169,084
E1,050,00
0
-do
WRC/
CFGB
ECHO
-do-
-do-
-do-
Nutrition, Water, Sanitation, Food Security
and Health Promotion
-do-
1/09 to
1/10
WRC/
CFGB
Nutrition, Water, Sanitation and Health
Promotion
-do-
-do-
E496,339
SG
Nutrition, Water, Sanitation, Health
Promotion, Food Security and Emergency
Nutrition, Water, Sanitation, Health
Promotion child protection, mainstreaming
of gender, HIV awareness and
environmental sustainability
Nutrition, Water, Sanitation, Health
Promotion, Food Security and Emergency
South Darfur, Ed
Daein
South Darfur, Ed
Daein
1/09 to
3/11
11/09 to
4/11
GBP
1,000,000
C$1,500,0
00
South Darfur, Ed
Daein
2/10 to
1/11
Integrated Emergency Nutrition,
Water, Sanitation, Food
Security and Health Promotion
Project
Tearfund Integrated Emergency
Nutrition, Water, Sanitation and
Health Promotion Project
Darfur Environmental Health
Ed Daein Emergency Response
Pr
No proposal
WRC/
CIDA
Integrated Humanitarian
Response: Nutrition, Water,
Sanitation, Health Promotion,
Food Security and Emergency
Pro
ECHO
GENERAL DISTRIBUTION
E496,339
121,705
159,000
151,760
E1,469,53
0
E1,100,00
0
GENERAL DISTRIBUTION
Integrated Humanitarian
Response for conflict-affected
communities in South Darfur
through Nutrition, WASH and
Emergency Response &
Preparedness
Ed Daein Food and Nutrition
Security Project
South Darfur Humanitarian
Response Programme
Integrated Humanitarian
Response for Conflict-affected
Communities
ECHO
Nutrition, WASH and Emergency
Response & Preparedness
South Darfur, Ed
Daein
2/11 to
1/12
94,280
E900,000
WRC/
CFGB
WRC/
CIDA
ECHO
Nutrition and food security
-do-
110,900
Nutrition, Water, Sanitation, Health
Promotion, Food Security and Emergency
-do-
E530,520
+E365765
C$1,000,0
00
E850,000
South Darfur Humanitarian
Response Programme
Tearfund Darfur Humanitarian
Response Programme
WRC/
CIDA
WRC/
CIDA
Water, Sanitation, Health Promotion,
Psycho-social support and Emergency
Water, Sanitation, Health, shelter, NFI,
livelihoods and education
5/12 to
4/13
5/12 to
4/13
42,690
Integrated Humanitarian
Response for Conflict-affected
Communities
ECHO
Nutrition, shelter and NFI
Integrated relief and early
recovery project, Darfur
TEARF
UND
NZ
OFDA
Nutrition, Water, Sanitation, Health
Promotion, education food security
South Darfur, Ed
Daein and Kass
Central Darfur,
Nertiti & Um
Dukhun and East
Darfur, Ed Daein
South Darfur, Ed
Daein and Kass
South Darfur,
Kass and Central
Darfur, Nertiti
Central Darfur:
Nertiti & Um
Dukhun. East
Darfur: Ed Daein
KASS, DARFUR
3/11 to
2/13
5/11 to
4/12
4/12 to
3/13
Nutrition, Water, Sanitation, Health
Promotion
El Geneina, West
Darfur
OFDA
Nutrition, Water, Sanitation, Health
Promotion
Beida , West
Darfur
OFDA
Nutrition, Water, Sanitation, Health
Promotion
OFDA
Nutrition, Water, Sanitation, Health
Promotion
Community led multi-sectoral,
relief and rehabilitation rural
peace building programme
Community led multi-sectoral,
relief and rehabilitation rural
peace building programme
Community led multi-sectoral,
relief and rehabilitation rural
peace building programme
Community led multi-sectoral,
relief and rehabilitation rural
peace building programme
4/13 to
3/14
Oct 2009
to Aug
2011
1/06 t0
4/07
116,210
76,332
106,910
34,623
C$1,200,0
00
C$1,500,0
00
E1312,00
0
£64,219
115,735
$1,466,15
9
5/07 t0
4/08
114,780
$
1,599,970
Beida , West
Darfur
5/08 t0
4/09
124,350
$
1,800,000
Beida, West
Darfur
5/09 to
4/10
62,350
$2,081,87
6
1
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
Community led multi-sectoral,
relief and rehabilitation rural
peace building programme
Tearfund Emergency Nutrition
and Sanitation Project, West
Darfur
OFDA
Nutrition, Water, Sanitation, Health
Promotion
Beida, West
Darfur
5/10 to
7/11
62,350
$
1,818,369
DFID
Nutrition, Sanitation, Health Promotion
Beida, West
Darfur
08/04 to
7/05
56,215
£447,692
Tearfund Integrated Emergency
Nutrition, Water, Sanitation and
Health Promotion Project
TEARFUND Community-led,
multi-sectoral, relief and
rehabilitation programme, West
Darfur
Integrated relief and early
recovery project, Darfur
Tearfund – Darfur Integrated
Relief and Development
Program 2010-2011
Tearfund Integrated Emergency
Nutrition, Water, Sanitation and
Health Promotion Project
Wadi Salih Integrated Relief
Project
Wadi Salih Integrated Relief
Project
Wadi Salih Integrated Relief
Project
Wadi Salih Integrated Relief
Project
CHF
DFID
Nutrition, Water, Sanitation, and Health
Promotion
El Geneina, West
Darfur;
4/06 to
3/07
25,000
£ 401,590
DFID
Nutrition, Water, Sanitation, Environment ,
education, Food security , and Health
Promotion
Beida, West
Darfur; Kass, S.
Darfur
4/07 to
8/12
76,150
£6,786,99
0
ECHO
Water, Emergency response and Health
Promotion
Water, Sanitation, education, and Health
Kass, S. Darfur
10/09 to
06/10
04/10 to
06/11
43,825
E715,000
Nutrition, Water, Sanitation and Health
Promotion
South Darfur, Ed
Daein
01/09 to
01/10
UN
Community Development, food security,
water, sanitation and health promotion
Community Development, food security,
water, sanitation and health promotion
Community Development, food security,
water, sanitation and health promotion
Community Development, food security,
water, sanitation and health promotion
Livelihoods
1/06 to
12/06
1/06 to
12/06
1/07 to
3/08
4/08 to
3/10
2009
CHF
UN
Education, WASH, Nutrition
Wadi Saleh, W.
Darfur
Wadi Saleh, W.
Darfur
Wadi Saleh, W.
Darfur
Wadi Saleh, W.
Darfur
Wadi Saleh, W.
Darfur
Wadi Saleh, W.
Darfur
TEAR
AUSTR
ALIA
TEARF
UND
NZ
Tear
CH
Buza
Buza
Buza
Kass, S. Darfur
2011
AD200,00
0
E100,000
N$100,00
0
26,000
26,000
53,000
CHF
E522,462
145,000
€ 923,992
80,500
€
1,707,524
$421,225
18,925
42,850
7,153
32350
???
$ 591,757
$ 791,747
$ 435,365
$290,000
$ 639,059
2
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
DFID WASH - NSP Year 2
DFID
Wash
Ed Daein and
Beida, South
Darfur
Ed Daein and el
Geneina
Apr-09 Mar-10
DEC-DRP and ERP
DEC
Watsan, health and food security
Wadi Salih Integrated Relief
Project
Water
Wadi Salih
Community-led, multi-sectoral
relief programme
Darfur Integrated Relief and
Development Program
Dioces
e of
Brad
ford
WRC/C
IDA
Tear
Aust
water, sanitation, health promotion,
nutrition support and education
Water, sanitation, education, livelihoods,
health promotion, relief
Beida; West
Darfur
Kass and
Geneina
8/09 to
4/10
4/10 to
7/11
DEC Project
DEC
Strengthening Food Security in
South Darfur
Integrated Emergency Nutrition,
Water, Sanitation and Health
Promotion Project
Integrated Humanitarian
Response: Nutrition, Water,
Sanitation, Health Promotion,
Food Security and Emergency
Project
Promoting sustainable change
through conflict transformation
in rural communities in West
Darfur.
Integrated Humanitarian
Response: Nutrition, Water,
Sanitation, Health Promotion,
Food Security and Emergency
Project
UN CHF
EC/AID
CO
Tear
NZ
water, sanitation, health promotion,
nutrition
Food security
Emergency Nutrition, Water, Sanitation
and Health Promotion
Beida, West
Darfur
South Darfur, Ed
Daein
Kass, South
Dafur
4/10 to
9/10
2/11 to
4/13
6/10 to
5/11
Tear
CH
Nutrition, Water, Sanitation, Health
Promotion, Food Security and Emergency
Ed Daein, South
Darfur, Sudan
2/10 to
1/11
DCPSF
Community development, peace-building,
youth
UmDukhun, W.
Darfur
01/2012
to:
12/2013
Tear
Aust
Nutrition, Water, Sanitation, Health
Promotion, Food Security and Emergency
Kass, S. Darfur
2011-12
UN
CHF
Education, health promotion, psychosocial, Nutrition
Ed Daein and
Kass, South
2012-13
2007-08
128,553
135,160
50,000
100,000
2008
2,000
52,100
213,916
150,000
56,000
GBP
10,000
922,769
CAD
AD200,00
0
E100,000
GBP
80,000
€800,000
N$100,00
0
E36,000
60,000
$681,072
GBP
85,000
18,000
15,000
$527,286
$500,000
3
GENERAL DISTRIBUTION
GENERAL DISTRIBUTION
Food Security and Livelihoods ,
WASH
Strengthening food & livelihood
security in South Darfur
Building the capacity and
increasing the resilience of
households to combat
morbidity, food and livelihood
insecurity, and educational
attainment
Humanitarian Response in
Central Darfur
Improved Livelihood and Water,
Sanitation and Hygiene for
targeted communities in rural
villages in Kass locality, Darfur.
Strengthening food & livelihood
security in Kass Locality
Strengthening food and
livelihood security in Kass
Locality
EU
Food security
DFID
Food security, Wash, food security
OFDA
Nutrition, Water, Sanitation, Health
Promotion, and Emergency
Livelihood and Water, Sanitation and
Hygiene
Tear
Aust
Belgiu
m
France
Darfur, Um
Dukhun Locality,
Central Darfur
Kass, South
Darfur
West Darfur,
Beida; Central
Darfur
UmDukhun;
South Darfur
(Kass)
Nertiti, Central
Darfur
Kass, S. Darfur
food & livelihood security
Kass, S. Darfur
Food security
Kass, S. Darfur
18,000
$500,000
350,000
2013-15
43,240
2013-15
350,000
E1,170,00
0
£3,478,97
5
8/12 –
1/14
8/12
69,000
$1,500,00
0
GBP
124,814
8/12 to
1/14
8/12 to
1/14
40,000
E1,000,00
0
E495,000
16,770
4
GENERAL DISTRIBUTION