18 March 2015
UNICEF Office of Research—Innocenti
Terms of Reference
Consultancy for International Academic Partner (IAP) to support design and implementation of
mixed-methods impact evaluation of Ghana LEAP 1000
1.
Background:
The UNICEF Office of Research—Innocenti (OoR) undertakes research- and study- related activities to ensure
that high quality research contributes to evidence-informed policy making within UNICEF activity
programming as well as to evidence informed policy design by its partners. To this end, the Office of Research
is carrying out research on the impact of national government social cash transfers in Africa under the Transfer
Project (http://www.cpc.unc.edu/projects/transfer). Areas of interest include impacts on household
consumption and food security, resilience, young child health and nutrition.
In collaboration with UNICEF Ghana Country Office, the Office of Research—Innocenti is managing the impact
evaluation (IE) activities around Ghana’s Livelihood Empowerment Against Poverty 1000 program (LEAP 1000).
The LEAP programme is Ghana’s flagship social protection program and is implemented by the Ministry of
Gender, Children and Social Protection (MGCSP). Developed in 2007, the LEAP is a cash transfer programme
currently reaching approximately 75,000 households and designed to empower extremely poor populations
in the country to exit poverty. Initial evaluations have demonstrated considerable impacts on educational
outcomes and access to health care. In its commitment to poverty reduction and social protection, MGCSP
aims to expand LEAP programming to include a new set of beneficiaries previously missed out on by existing
targeting schemes, namely households with small children and pregnant women (LEAP 1000). A motivation
for expanding eligibility to this group is to prevent stunting and malnutrition, which are largely determined in
the critical period of birth to two years. The LEAP 1000 pilot intervention will run for three years, and the
evaluation will serve as a learning tool to understand the determinants of child health and nutrition in Ghana,
and the role of combination interventions such as cash, health insurance and other public health interventions
in reducing stunting
2.
Objective(s)
This consultancy is to support the impact evaluation of the Ghana LEAP 1000 social protection programme
described above.
The aim of this consultancy is twofold.
1) The selected International Academic Partner (IAP) will design and implement the qualitative component
of the impact evaluation in collaboration with the Ghanaian research partner. The proposed qualitative
evaluation design will be an embedded longitudinal qualitative study. This entails selecting a set of families
(between 10-16, exact numbers to be proposed by IAP from the baseline quantitative sample (note: the
quantitative impact evaluation is being carried out by OoR) and conduct in-depth interviews with caregivers
1
at baseline, and then following these same families three more times over the next two years to see how
they respond to the cash transfer. Since the sample is ‘embedded’ the full range of information from the
household survey would also be available for these households and their responses and challenges as
reported in the in-depth interviews can be compared to the quantitative data collected from them to
understand the relationship between the two. OoR will participate in the selection of the local partner who
will conduct the in-depth interviews.
2) The IAP will also provide technical support to the quantitative component of the evaluation which will be
led by OoR. This will include technical advice on design and sampling, support to questionnaire design,
training, data analysis and report writing.
At the end of the consultancy, UNICEF and the Government of Ghana should have an improved
understanding of how LEAP 1000 can impact target households and young children.
3.
Specific activities to be completed to achieve the objectives:
The assignment is divided in five phases.
Phase I: The objective of the first phase is for the contractor, in consultation with UNICEF, to work together to
craft research questions informed by the overall impact evaluation’s conceptual model and hypothesized
pathways through which social cash transfers impact child health and nutrition.
Phase II: Propose a study protocol and develop study instruments.
Phase III: The objective of the third phase is to train interviewers, implement baseline and two rounds of
qualitative follow-up in-depth interviews (IDIs) data collection, and report on findings (one report per wave of
data collection). After the protocol is agreed in Phase II, the contractor will travel to the field to help train the
local firm to conduct interviews (for each wave of data collection), including overseeing pilot interviews if
necessary. Once data are collected and translated into English where needed by the local firm, the contractor
will analyse the data and write reports for each wave. For the quantitative work, the IAP investigator will
provide inputs as described below for baseline and one 24-month follow-up survey. The PI and co-Investigator
will contribute to inputs (largely impact reports) for the dissemination workshops. The PI will attend the
baseline dissemination workshop, and both the PI and co-Investigator will attend the 24 month follow-up
workshop.
Phase IV: The contractor in consultation with UNICEF will work on publications for refereed journals.
Manuscripts will include at least one mixed-method paper combining evidence from qualitative and
quantitative surveys, and possibly one or more qualitative papers. Early results based only on the qualitative
study are envisioned. The contractor will respond to all queries and recommendations from UNICEF and from
the peer reviewers.
Phase V: Based on the findings, the contractor in consultation with UNICEF will produce a short (3-5 pages)
policy note on program impacts on child health and well-being (Qualitative objective 5). The note will outline
the main findings of the evaluation and make recommendations for the design of scale up to achieve maximum
results related to child health and nutrition.
2
Below is a description of activities by component.
Qualitative
1. Develop research questions for qualitative impact evaluation related to pathways through which
LEAP 1000 impacts child nutrition, including but not limited to, economic well-being, food security,
caring practices, women’s empowerment and household bargaining, access to health care, improved
social networks, as well as possible unintended effects (Phase I).
2. Design qualitative study methodology, including semi-structured questionnaire design, methods for
household selection and data collection, and analytic plan (Phase II).
3. Train local interviewers and oversee data collection in collaboration with local partner (Phase III).
4. Analysis of qualitative data, working closely with OoR to ensure coordination of quantitative and
qualitative impact evaluation components and feedback loops for informing mutual analysis and
conclusions (Phase III).
5. To provide evidence and recommendations for policy-makers and programme managers to draw on
when designing next stages of program scale-up, if applicable (Phase IV).
6. To provide recommendations for the next steps in research (Phase IV).
Quantitative (support to OoR)
1. Develop research questions based on a theory of change on how LEAP1000 can affect young child
health and nutrition, including potential pathways. Based on these research questions, assist in
designing survey instruments (Phase I).
2. Provide technical guidance on overarching design (identification strategy) and sampling for household
survey (Phase II).
3. Attend training and/or field work to ensure quality control in data collection (baseline and follow-up)
(Phase III).
4. In collaboration with OoR, develop analysis plan and estimation strategy, and conduct statistical
analysis (baseline and follow-up) (Phase III).
5. Provide written inputs (analysis) to baseline and follow-up evaluation reports; attend national
dissemination workshops (Phase III).
Travel
A number of trips are foreseen to be carried out by the Principal Investigator and/or the co-investigator
for the field work and to attend to the dissemination workshops.
Proposed Trips
Purpose
Design of qualitative study (PI)
Field work – qualitative (PI)
Field work – quantitative (Co-I)
Baseline workshop – (PI)
1st follow-up field work – qualitative (PI)
Qualitative impacts workshop (PI)
2nd follow-up – qualitative (PI)
2nd follow-up quantitative (Co-I)
Final evaluation workshop (PI & Co-I)
Approximate Date and duration
May 2015 (6 days)
July (14 days)
July (14 days)
November 2015 (4 days)
January or July 2016 TBD (14 days)
May or October 2016 TBD (4 days)
July 2017 (14 days)
July 2017 (14 days)
November 2017 (4 days x 2 persons)
3
The workshops will be organized by UNICEF Ghana Country Office.
4.
Specific outputs/products/results with specific delivery dates:
The contractor is to deliver:
1. An inception report that details the timeline, evaluation strategy, major activities and dates and
key intermediate deliverables (i.e. survey instruments, ethics approvals) for the qualitative study:
31 May 2015
2. A study protocol for the evaluation (qualitative component only) including survey instruments
and other relevant documentation; additional documentation required for ethical approval and
eventual ethical approval certification (outlining the research questions, sampling and analysis
methodology, timelines, etc.; see Phase II above): draft 15 June 2015, final 25 June 2015
3. A baseline report summarizing methodology and baseline results. This report should include a
comprehensive description of the research questions, study design, concepts and items covered
in the field guides, and findings from baseline interviews (see Phase III above): draft 6 November
2015, final 15 January 2016
4. Baseline report (quantitative): The IAP will provide statistical analysis and written inputs into the
baseline quantitative evaluation report: draft 6 November 2015, final 15 January 2016
5. First follow-up report. This report will describe results from the first follow-up IDIs on the impact
of the program on recipient families: draft 1 April 2016, final 30 May 2016
6. Second follow-up report. This report will describe results from the second follow-up wave of
qualitative data collection. This report will report the results from the final qualitative field work,
and ideally be incorporated into the quantitative evaluation report to provide a holistic picture of
the impact of LEAP 1000 on women and children: draft Sept. 2017, Final December 2017
7. Comprehensive report. This report will synthesize the impacts of LEAP 1000 based on all
qualitative surveys, pulling out trends over time and other aspects of the dynamics of LEAP 1000
impacts: draft 15 January 2018, final 31 January 2018.
8. 24-month follow-up report (quantitative). The IAP will provide statistical analysis and written
inputs into the follow-up quantitative evaluation report: draft 15 November 2017, final 15
December 2017
9. A 4-5 page policy note on pathways through which social cash transfers impacted child health and
nutrition and other relevant outcomes, outlining the main findings of the evaluation (qualitative
and quantitative)and making recommendations on how to scale up effective interventions to
achieve improved health and well-being (see Phase V above): 15 February 2018, final 28 February
2018.
The IAP will report to the OoR, who has oversight of the overall evaluation activities. The IAP will work with
OoR to adjust and sharpen the research question, objectives and parameters of the impact evaluation in
order to conduct a holistic impact evaluation of LEAP 1000 on child health and well-being.
All deliverables are to be produced in English.
4
Project timeline*
Planned activities
Date
Contractor contracted
1 May 2015
Consultation with UNICEF to develop the research questions
and approach to deliverable 1, inception report, study
protocol
10 May 2015
Travel for design of qualitative study
May 2015
Inception report (deliverable 1)
31 May 2015
Draft 1 of qualitative study protocol (deliverable 2)
15 June 2015
Final qualitative study protocol (deliverable 2 finalised)
25 June 2015
Invoice 1
25 June 2015
Travel for baseline field work (qualitative)
July 2015
Draft qualitative baseline report (deliverable 3)
6 November 2015
Draft quantitative baseline report (deliverable 4)
6 November 2015
Dissemination workshop – Baseline
November 2015
Final qualitative baseline report (deliverable 3 finalised)
15 January 2016
Final quantitative baseline report (deliverable 4 finalised)
15 January 2016
Invoice 2
15 January 2016
Travel for first follow-up field work - qualitative
January or July 2016 TBD
First follow-up qualitative report draft (deliverable 5)
April or September 2016 TBD
Dissemination workshop qualitative
May or October 2016 TBD
First follow-up qualitative report finalised (deliverable 5
finalised)
June or November 2016 TBD
Invoice 3
June or November 2016 TBD
Travel for second follow-up qualitative
July 2017
Travel for second follow-up quantitative
June 2017
Second follow-up qualitative report draft (deliverable 6)
September 2017
5
Planned activities
Date
24-month quantitative report draft (deliverable 8)
15 November 2017
Final dissemination workshop
November 2017
24-month quantitative report draft finalised (deliverable 8
finalised)
15 December 2017
Second follow-up qualitative report finalised (deliverable 6
finalised)
15 December 2017
Invoice 4
16 December 2017
Comprehensive qualitative report draft (deliverable 7)
15 January 2018
Comprehensive qualitative report draft finalised (deliverable 7
finalised)
31 January 2018
4-5 page policy note draft finalised (deliverable 9)
15 February 2018
4-5 page policy note draft (deliverable 9 finalised)
28 February 2018
Invoice 5
28 February 2018
Contract finalized
28 February 2018
*Project timelines may change upon agreement between the contractor and UNICEF.
The contractor is responsible for meeting all deliverables outlined in section 4.2 on time. All work must be
original and follow the highest research and ethics standards.
The contractor must meet all deliverable deadlines unless otherwise agreed with UNICEF. Should the
contractor not be able to meet a particular deadline they must advise UNICEF in writing at least two weeks
in advance.
The contractor must respond to all requests made by UNICEF during the drafting process and ensure that all
work is of a high standard.
5.
Qualifications and/or specialized knowledge/experience required and desirable for undertaking the
assignment
The institution must be a University with an international reputation for research excellence, particularly in
the areas of health, nutrition and public policy, hereafter referred to as ‘the contractor’.
6
Requirements for the contracted team:
The contractor team should be made up of a lead PI for the qualitative component and a co-investigator for
the quantitative component, a data manager programmer (initial handling of data, quality and consistency
checks and validation/cleaning) and a research assistant.
1) Principal investigator (PI), qualitative
- Advanced degree (PhD) in relevant disciplines such as Public Health, Economics, Public Policy,
Sociology or related disciplines.
- Seven to ten years of worldwide experience on qualitative research (impact evaluation)
including experience in the African context (this was in the agreement signed with Ghana CO), a
publication record commensurate with her/his years of experience and field of expertise, with
significant work in low and middle income countries.
- Sound knowledge of qualitative and quantitative methods.
- Substantive knowledge of health, nutrition and poverty in Africa, and ideally have first-hand
experience evaluating large-scale social programs in developing countries.
- Strong English report writing skills and a track record of producing high quality publications.
The team leader will be the PI for the qualitative component.
2) Co-investigator (quantitative)
- Advanced degree (PhD) in relevant disciplines such as Public Health, Economics, Public Policy,
Sociology or related disciplines.
- Seven to ten years of experience and a publication record commensurate with her/his years of
experience and field of expertise, with significant work in low and middle income countries.
- Sound knowledge of quantitative methods.
- Sound experience in designing and implementing evaluation studies.
- Substantive knowledge of health, nutrition and poverty in Africa, and ideally have first-hand
experience evaluating large-scale social programs in developing countries.
- Expertise in child health and nutrition and interventions to improve these outcomes, particularly
in Africa.
- Strong English report writing skills and a track record of producing high quality publications.
3) Data management programmer
- Master’s degree in relevant disciplines such as Public Health, Economics, Public Policy, Sociology
or related disciplines.
- Sound experience managing large datasets
- Sound knowledge of Stata and quantitative data analysis methods
4) Research Assistant (RA)
- Graduate student (Master’s or PhD level) in relevant disciplines such as Public Health,
Economics, Public Policy, Sociology or related disciplines.
- Background in theory of qualitative analysis
- Sound experience coding and analysing qualitative data
7
6. Duration of the consultancy
The consultancy will last for the duration of the evaluation. Start date is approximately May 1, 2015 for 34
months, until 28 February 2018.
7.
Supervision and work arrangements:
The assignment will be supervised by Tia Palermo, Social Policy Specialist, UNICEF Office of Research –
Innocenti, with indirect reporting to the Chief of Social and Economic Policy. The contractor will be required
to respond to all questions and suggestions made by the consultancy supervisor and advisory board. The
contractor is to direct all queries, ideas, drafts and other deliverables to Dr. Palermo, who will share them
with relevant stakeholders and provide feedback to the contractor.
The contractor will work from its own location with field visits to Ghana as necessary related to data
collection efforts, and team members are not expected to visit the Office of Research – Innocenti in
Florence, Italy. All submissions will be made electronically unless other means are needed (e.g. voluminous
documents, technical problems). Communication with the Office of Research – Innocenti is to take place
over email, the phone or Skype.
8
The Technical Proposal will be evaluated against the criteria in the table below.
The Technical Proposals will be evaluated against the following:
REF
CATEGORY
POINTS
1.
The proposed team:
30
a. Research team holds advanced university degrees (PhD) in the Social
Sciences, Economics, Public Health, Research Methods and/or related
disciplines.
b. A team leader with a minimum of 5 years of established research credibility
and publication record in maternal and child health, nutrition, women’s
empowerment, and/or social protection in low and middle income
countries.
c. Sound experience in designing and implementing qualitative studies.
d. Good understanding of child health and nutrition issues in Africa.
e. Strong English report writing skills and a track record of producing high
quality reports.
f. Strong publication record on maternal and child health, gender, and
program intervention (at least one team member).
g. Communication and writing skills in English.
2.
Expertise in child health and nutrition and interventions to improve these outcomes. 10
a. Publication record in peer-reviewed journals on relevant topics and recognition
in the field.
3.
Expertise in impact evaluations more specifically
10
a. Experience and knowledge of health and social protection
interventions.
4.
Proposed approach to meeting the deliverables:
10
a. Description of potential pathways through which LEAP 1000 can affect
families and children, and implications for research questions
(qualitative and quantitative).
b. Description of specific survey questions (topics or questions) given
conceptual framework in (a) for qualitative and quantitative
components.
c. Description of approach to field work, quality assurance and data
analysis.
d. Descriptions are clear, detailed, and realistic and proposed work plan
and staffing is sufficient to meet Request for Proposal requirements.
9
5.
Corporate profile:
e. Financial administration capacity
f. Experience in conducting similar types of research projects.
6.
5
Risk assessment:
g. Specification of an approach or any measures the research team will
be taking to minimize negative unintended consequences and to
maximize positive outcomes.
h. Specification of quality assurance processes for research human 2.5
subjects risks and oversight.
7.
Overall response:
i. Understanding of, and responsiveness to, UNICEF requirements.
j. Understanding of scope, objectives and completeness of response.
k. Overall concord between UNICEF requirements and the proposals.
Total Technical
2.5
70
Only proposals which receive a minimum of 50 points will be considered further.
10