UNITED NATIONS RELIEF AND WORKS AGENCY FOR PALESTINE REFUGEES
ENSURING THE PROTECTION OF CONFLICT-AFFECTED PALESTINE REFUGEES FROM SYRIA
EXTERNAL FINAL PROJECT EVALUATION – TERMS OF REFERENCE
OCTOBER 2014
1. Background and context
UNRWA is a United Nations agency established by the General Assembly in 1949 and is mandated to
provide assistance and protection to a population of some 5.3 million registered Palestine refugees.
Its mission is to help Palestine refugees in Jordan, Lebanon, Syria, the West Bank and the Gaza Strip
to achieve their full potential in human development, pending a just solution to their plight.
UNRWA’s services encompass education, health care, relief and social services, camp infrastructure
and improvement, microfinance and emergency assistance. UNRWA is funded almost entirely by
voluntary contributions.
The European Commission – Instrument for Stability - supported UNRWA with a EUR 7.5 million
project “Building the Resilience of Conflict-affected Palestine Refugees in Syria and Lebanon”. The
project started on 1 October 2012, it was for an initial duration of 18 months and was subsequently
extended to 24 months, finishing on 30 September 2014. The project was part of UNRWA’s Regional
Syria Crisis Response and it was the first intervention funded by the EU Instrument for Stability. It
was one of the first UNRWA regional Syria Crisis interventions and one of the largest and first
regional projects implemented by the Agency in three countries of Syria, Lebanon and Jordan.
The project was initiated in response to the conflict and subsequent crisis in Syria which started in
early 2011 and which increasingly affected Palestine Refugees registered with UNRWA in Syria.
Much conflict took place within or in the vicinity of Palestine refugee camps and in particular
Yarmouk which was home to approximately 79% of Palestine refugees in Syria. This combined with a
seriously declining economy increased Palestine refugee humanitarian needs and resulted in high
levels of vulnerability with many not able to afford their basic needs. Even before the conflict,
UNRWA assessed 27% of Palestine refugees lived under the national poverty line and in mid-2012
UNRWA assessed over 300,000 out of 500,000 Palestine refugees in Syria were in need of
emergency assistance. Palestine refugees then began to displace within Syria and to neighbouring
countries and in mid-2012 already 10,000 had fled to Lebanon.
The project was designed to assist approximately 80,000 of the most vulnerable Palestine refugees
through interventions in Syria in the protection of children through alternative education modalities,
microfinance, and enhancement of security; and cash assistance for Palestine refugees displaced to
Lebanon. The overall objective was to build Palestine refugee’s socio-economic resilience, mitigate
the long-term impacts of the armed conflict and to meet their most pressing protection needs.
The project is part of UNRWA’s Regional Syria Crisis Response which from 2012-14 received USD
410,921,167 from 50 donors, of which USD 47,732,728 was from EU1. In Syria, UNRWA adapted its
services to ensure they are sustainable and effective in conditions of armed conflict. These services
include primary education, primary health care, social services, infrastructure services in refugee
1
USD 28,807,955 was received in 2012, USD 211,213,265 was received in 2013 and USD 170,899,948 has been
received so far in 2014 until 30 October.
Page | 1
UNITED NATIONS RELIEF AND WORKS AGENCY FOR PALESTINE REFUGEES
camps, programmes for women and youth, and microfinance. Alongside humanitarian response
activities, UNRWA’s regional network of existing services, directly offered by its 13,000 area staff in
Syria, Lebanon and Jordan, continues to provide a critical framework of stability for families and
communities, improving prospects for post-conflict recovery. Amidst increasing insecurity and
uncertainty, UNRWA’s emergency and regular programming combine to make a vital contribution to
the resilience of Palestine refugees and their communities.
The project encompassed following four components, targeting the most vulnerable:

Component 1: Alternative Education Modalities for Conflict-affected Children: EUR 2.5 million,
implemented in Syria, Lebanon and Jordan
The provision of education services was impeded as armed conflicts prevented teachers and
students from safely attending school. As a result alternative education modalities were
required to enable a measure of teaching and learning to take place. The outputs intended to be
achieved were: alternative learning modalities, child friendly spaces, self-learning material,
satellite distance learning programme and non-therapeutic psychosocial support for students.

Component 2: Maintaining Inclusive Microfinance Outreach in Syria: EUR 1.7 million,
implemented in Syria
UNRWA managed one of the largest microfinance operations in Syria open to all people in Syria
and 10% of its clients were Palestine refugees. Established in 2003 it had five branch offices and
by 2011 financed 17,717 loans valued at USD 12.36 million. The objective was to keep the
valuable programme operational through: resilience-building of microfinance services,
maintenance of operations and provision of working capital loans to micro-enterprises.

Enhanced Security of UNRWA Frontline Staff to Improve Delivery of Emergency Services: EUR
799,000, implemented in Syria
Continued programme and emergency delivery depends on managing a secure framework for
staff to operate. The objective was to strengthen UNRWA’s security capacity in Syria through: an
additional safety officer, security upgrades for facilities and emergency assessment and safety
training for staff. The beneficiary population targeted was over 3,600 Agency staff in Syria.

Emergency Shelter Assistance to Protect Palestine Refugees from Syria (PRS) in Lebanon: EUR 2.6
million, implemented in Lebanon
Without sources of livelihood, the 10,000 Palestine refugees who fled Syria for Lebanon were
increasingly vulnerable and in need of cash assistance to support costs of high rent. The
objective was to provide cash assistance for rent to over 2,000 Palestine refugee families and
ensure good relations between Palestine refugees from Syria and the host community.
The project was directly implemented by UNRWA through its staff and facilities in Syria, Lebanon,
Jordan and Headquarters Amman, with the assistance of staff in Gaza. It was coordinated with
relevant UN Country Teams, UNOCHA and UN Security Management.
A number of challenges emerged during implementation to which changes had to be made to the
project. The largest challenge was a dramatic escalation of conflict affecting Palestine refugees in
December 2012 which created an ongoing humanitarian crisis in Syria with widespread displacement
of Palestine refugees within Syria and to neighbouring countries. Most components had to be
Page | 2
UNITED NATIONS RELIEF AND WORKS AGENCY FOR PALESTINE REFUGEES
adapted to address this and many challenges were faced related to working in Syria including:
movement and access restrictions for trainers to Syria; restrictions on Syrian staff travelling to
Lebanon and Jordan; damage, destruction, looting and closure of UNRWA installations and access to
installations in areas of conflict; UNRWA staff were killed, the victims of violence, traumatised,
displaced within Syria and some left the Agency; severe economic downturn; large-scale
displacement of Syrian Palestine refugees (PRS) including students within and outside Syria; the
impact of this on UNRWA services in Lebanon and Jordan; out of school students; psychological
impact of the conflict on students; lack of qualified candidates for positions; materials not available
on the local market or at increased cost; and authorities not granting import licenses for equipment.
As a result the project did not complete in 18 months and a no-cost 6 month extension was granted,
the budget was revised three times, some activities were changed and some funding was not spent.
2. Purpose of the Evaluation
A final external evaluation of the project has to be conducted as outlined in the project agreement
and with the donor’s agreement. The evaluation will contribute to accountability and learning. The
evaluation should consider the overall project and integration of the four components, and then
consider each component separately. The evaluation should provide lessons learned and
recommendations for UNRWA to improve planning, implementation, management, monitoring and
evaluation of future similar interventions.
The key stakeholders for the evaluation are UNRWA Senior Management, Syria Regional Response
team, Syria and Lebanon Field Offices, Departments of Microfinance, Education and Security,
External Relations and Communications Division and the Department of Internal Oversight; and EU.
The results of the evaluation will be circulated internally within UNRWA to these stakeholders and
externally to EU who funded the intervention along with the final project report. The evaluation will
likely be used by EU to inform future funding decisions related to UNRWA and the Syria crisis.
3. Scope and objectives
The evaluation will cover the whole project which was implemented from October 2012 to end
September 2014, in all fields of implementation in Syria, Lebanon and Jordan. It will analyse the
project overall including its strategic aspects and evaluate each component separately in detail as
each component was quite distinct. The evaluation will follow standard OECD-DAC criteria for
evaluating development assistance of relevance, efficiency, effectiveness, impact and sustainability.
The primary issues that must be addressed are:






How relevant were the objectives of the project when it was designed? How well was it adapted
to the changing context?
To what extent were the overall and component-specific objectives achieved? Did the security
context in Syria impact on this and how?
What are the short-term impacts of the project? Are there likely long-term impacts and how can
these be maximized? Are there any unintended impacts?
Are aspects of education, microfinance and security components sustainable in the long term?
Was the project cost-effective and how can it be ensured all funding is spent in future projects?
Were there examples of innovation and best practice that can benefit other UNRWA projects?
Page | 3
UNITED NATIONS RELIEF AND WORKS AGENCY FOR PALESTINE REFUGEES


Are there lessons for integrating sustainability into other emergency projects?
What are the key lessons learned and recommendations for UNRWA in planning, implementing,
managing, monitoring and evaluating emergency projects in the Syria crisis?
4. Evaluation questions
The evaluation questions that must be answered are:
(i)
Relevance
 When it was designed to what extent were the overall objectives of the project and those of
each component consistent with the beneficiaries’ requirements and needs?
 As the context changed in December 2012 was the programme and each component still
relevant? Were the changes made in response sufficient?
 Was it appropriate to have a regional project of four components being implemented in three
countries? Were the four components integrated or not and if not, should they have been?
 What was the role of the project overall with regard to UNRWA’s Syria Crisis Response?
 How did component 1 and 3 fit into UNRWA’s overall education and security programmes
respectively?
 How did component 4 fit into UNRWA’s overall response for Palestine refugees from Syria
displaced in Lebanon?
(ii)
Effectiveness
 How well did UNRWA plan, manage and coordinate the project overall?
 To what extent were the overall objectives and those of each component achieved? What major
factors influenced the achievement or non-achievement of objectives?
 Did the security context in Syria affect delivery of the project? Did UNRWA plan well for
potential changes in the security situation?
 Did the work in each component contribute to emergency response capability in that area?
Were there examples of innovation and best practice?
 To what extent did the activities in Component 1 enable teaching and learning to continue in
Syria and continue for Palestine refugees from Syria displaced to Lebanon and Jordan?
 To what extent did the activities in Component 2 keep the microfinance programme
operational?
 To what extent did the activities in Component 3 strengthen UNRWA’s security capacity in Syria?
 To what extent did Component 4 support the costs of rent for Palestine refugees displaced to
Lebanon from Syria and did this reduce their vulnerability?
(iii)
Efficiency
 Was the project overall, and each component, cost-efficient (funding, personnel, installations
and resources)?
 Why was the budget revised a number of times and funding not spent by the end of the project?
How can this be avoided in future similar projects?
(iv)
Impact
Page | 4
UNITED NATIONS RELIEF AND WORKS AGENCY FOR PALESTINE REFUGEES





What were the positive and negative, primary and secondary short-term effects produced by the
project, directly or indirectly, intended or unintended?
What were the impacts of each component for the beneficiaries?
What were the impacts of the project for Syrian Palestine refugee woman and girls?
What were the socio-economic benefits of Component 4 to the beneficiaries?
What are possible long-term positive/ negative impacts of the project? How can positive aspects
be maximized or replicated/ scaled up in the future?
(v)
Sustainability
 Although this was an emergency response project in multiple-conflict setting, will benefits from
the project continue into the future and what is the probability of continued long term benefits?
 What aspects of the project are likely to be most sustainable into the future?
Following from this, the key lessons learned and recommendations should be outlined, especially
related to planning, implementation, management, monitoring and evaluation of Syria emergency
projects and large regional projects.
5. Methodology
The methodology that should be used should be mixed-methods due to the nature of the
intervention and implementation context, gender-sensitive and participatory to the extent possible
given the security and operational environment and restrictions in Syria and Lebanon. The
evaluation should be both quantitative and qualitative and include qualitative data collection and
analysis with beneficiaries.
Evaluation methods to be used can include, but are not restricted to:







Review of secondary data (project documentation including progress and financial reports,
and UNRWA strategic documentation);
Interviews with key project stakeholders and beneficiaries in Syria, Lebanon and Jordan;
Field visits to project implementation sites in Syria, Lebanon and Jordan;
Observation;
Focus groups with beneficiaries in Syria, Lebanon and Jordan;
Questionnaires (if feasible within the timeframe); and
Outcome mapping.
The Evaluation Consultant shall provide a detailed evaluation methodology to undertake the
evaluation in the inception report with methods of data collection and analysis. This will take into
consideration access and security restrictions in Syria which can be discussed. The Evaluation
Consultant will be required to travel to Syria, Lebanon and Jordan to collect data.
6. Deliverables and timeframe
The deliverables for the evaluation are:
(i) Inception Report
Page | 5
UNITED NATIONS RELIEF AND WORKS AGENCY FOR PALESTINE REFUGEES
The Evaluation Consultant should write and submit this following their initial desk review of
documentary information and discussion with the programme managers. The Inception Report
should contain an evaluation plan and list of reviewed documents. The evaluation plan should
contain a proposed methodology and an evaluation matrix showing how each evaluation question
will be answered by way of proposed data collection sources, data collection methods and data
analysis techniques. The report should contain a proposed schedule of tasks, activities and
deliverables. The Inception Report will be sent to and discussed with the Planning and Evaluation
Consultant who will manage the evaluation for UNRWA. It will be reviewed by UNRWA and the
Planning and Evaluation Consultant will approve it and the next steps.
(ii) Debriefing with key stakeholders
The Evaluation Consultant will debrief key UNRWA stakeholders after data collection has been
completed where they will present emerging key findings and recommendations for the Draft
Evaluation Report. The stakeholders will review and feedback on these to prepare for writing the
Draft Evaluation Report.
(iii) Draft Evaluation Report
A Draft Evaluation Report, no longer than 30 pages, shall be submitted as per the timeline below
containing an overall project chapter and separate chapters for each component. UNRWA will
review the report and the Planning and Evaluation Consultant will meet with the Evaluation
Consultant to provide feedback to produce the Final Report.
(iv) Final Evaluation Report
A Final Evaluation Report of no longer than 25 pages, not including annexes, shall be submitted as
per the timeline below. This shall include a one page executive summary, findings, lessons learned
and recommendations, pictures and annexes explaining data sources, data collection methods and
analysis used. A five page summary report should also be submitted containing a summary of the
main findings and recommendations.
(v) Presentation of final report
The Evaluation Consultant will deliver a presentation of the final report including key findings and
recommendations to the key UNRWA stakeholders after the Final Report is approved.
The Planning and Evaluation Consultant will provide a template for the Inception and Final Reports
to the Evaluation Consultant.
Timeframe: the starting date for the evaluation shall be in late-November and 60 working days will
be allowed for it. The evaluation and all deliverables have to be completed by the second week of
March 2015.
The timeframe for the evaluation is:
Task
Responsibility
Date
Meeting and briefing of Evaluation Consultant with Planning
and Evaluation Consultant and provision of all documents to
Evaluation Consultant
Planning and
Evaluation Consultant
30/11/2014
Desk review and write Inception Report
Evaluation Consultant
02-04/12/2014
Page | 6
UNITED NATIONS RELIEF AND WORKS AGENCY FOR PALESTINE REFUGEES
Submit Inception Report
Evaluation Consultant
07/12/2014
Meeting of Evaluation Consultant and Planning and
Evaluation Consultant and Approve Inception Report
Planning and
Evaluation Consultant
10/12/2014
Data collection field visits
Evaluation Consultant
14/12/2014 –
18/12/2014
(2 weeks break due
to holidays)
04/01 – 05/02/2015
Debriefing with key UNRWA stakeholders to present key
findings and recommendations
Evaluation consultant
08/02/2015
Write and prepare Draft Report
Evaluation Consultant
09/01/2014 –
17/02/2015
Submit Draft Report
Evaluation Consultant
17/02/2015
Meeting to discuss draft report
Evaluation Consultant
& Planning and
Evaluation Consultant
24/02/2015
Approve Draft Report
Planning and
Evaluation Consultant
24/02/2015
Submit Final Report, summary report incorporating feedback
Evaluation Consultant
03/03/2015
Final Report approved
Planning and
Evaluation Consultant
08/03/2015
Presentation of findings to UNRWA
Evaluation Consultant
15/03/2015
7. Qualifications and experience
The Evaluation Consultant should have an advanced university degree (Masters or equivalent) in
social sciences or a related field and have completed training on evaluation, research and analysis.
Practical experience may be substituted for formal education and training. The Evaluation
Consultant should have over 5 years of experience conducting external evaluations including
conducting field work and qualitative data collection and analysis and have participated in at least six
evaluation processes. Experience evaluating emergency projects including field work is valued, as is
experience evaluating EU-funded projects in the region and a good understanding of EU funding and
work in Syria. Experience working for UNRWA in particular or other UN Agencies on the Syria Crisis
would be a distinct advantage. The Consultant should have experience living and working in the
Middle East and be based in the region or willing to travel to the region for the evaluation including
in Syria, Lebanon and Jordan. The Consultant should be fluent in English and ideally with good skills
in Arabic. Excellent oral and written communications skills, high level analytical and report writing
skills and experience writing clear and concise reports for a range of audiences is also required.
Applicants are expected to be authorized to work in Syria and be able to travel to Syria.
Page | 7
UNITED NATIONS RELIEF AND WORKS AGENCY FOR PALESTINE REFUGEES
8. Ethics
The Evaluation Consultant shall undertake the evaluation in consultation with UNRWA and in full
accordance with the terms of reference. The Evaluation Consultant must conduct the evaluation in
accordance with the principles outlines in the UNEG “Ethical Guidelines for Evaluation”2 and
UNRWA’s Ethics and Standards of Conduct. The Evaluation Consultant must immediately bring any
critical issues to the attention of UNRWA that could jeopardise or impede the evaluation process.
The Evaluation Consultant must ensure procedures to safeguard the rights and confidentiality of
information providers and fully adhere to UNRWA procedures to interview and collect data from
beneficiaries. The Evaluation Consultant must ensure all information related to or collected during
the evaluation is securely and safely stored whether electronically or hard copy. They must adhere
to the workplan to be mutually agreed with UNRWA and ensure the deliverables are delivered on
time, following the highest professional standards.
9. Implementation arrangements
The Evaluation Consultant will report to the Planning and Evaluation Consultant who will manage
the evaluation for UNRWA. All documents for approval, queries and issues will be sent to the
Planning and Evaluation Consultant. Deliverables will be approved by the Planning and Evaluation
Consultant following internal consultation with relevant staff including the Evaluation Division and
component project managers. UNRWA will organize an internal meeting for the Evaluation
Consultant to present the findings and it will send the report to the donor. Provision of office space
will be clarified with the Evaluation Consultant but it is envisaged they will primarily work from their
home or the field. A lump sum will be paid to the Evaluation Consultant to cover all professional fees
and other costs related to the evaluation and no materials or equipment will be provided. The
Planning and Evaluation Consultant will provide all documents and information required to the
Evaluation Consultant, facilitate access to staff and visits to UNRWA offices in Syria, Lebanon and
Jordan.
10. Application process
Applicants should submit a proposal no longer than one page, cover letter, CV with three references
and their expected lump sum fee rate including professional fees and all evaluation costs to
consultancy@unrwa.org clearly indicating “IfS Evaluation Consultant” in the subject line of the
message. Please note the Evaluation Consultant will be paid a lump sum daily rate to include the
consultant fees and all costs related to the evaluation.
Due to the time constraint for the evaluation, the deadline for submission of applications is
22/11/2014 (late applications will not be considered).
UNRWA is an equal opportunity employer and welcomes applications from both women and men.
UNRWA encourages applications from qualified and experienced female candidates. Only those
applicants short-listed for interview will be contacted. UNRWA is a non-smoking environment.
2
See http://www.unevaluation.org/document/detail/102
Page | 8