Design and
implementation of
Rikolto’s 2022-2026
evaluation framework
Technical & Financial Offer
Louvain-la-Neuve,
3rd August 2022
Proposed Methodology
1. Methodology & approach
The design and implementation of Rikolto’s 2022-2026 evaluation framework will rest on a rigorous
participatory theory-based and mixed-methods evaluation approach that is impact-orientated in
studying the outcomes and impacts Rikolto’s operations achieve from a food systems perspective. This
evaluation design and implementation approach will build on and complement Rikolto’s existing M&E
framework, data collection tools and sources. Rikolto's current M&E framework will serve as the
foundation and together with Rikolto the proposed evaluation design will refine, complement streamline,
and improve the current monitoring, evaluation, accountability and learning (MEAL) framework within
Rikolto, striving to strike the right balance between various evaluation, accountability and continuous
learning needs for the organization and its stakeholders.
1.1. Design phase
1.1.1. Fine-tune programme- and outcome-level ToCs
In the design phase, the core team will work in close collaboration with Rikolto to fine-tune its
programme-level Theory of Changes (ToC) to articulate the causal pathways to achieve the intended
outcomes and impacts. This will be done based on a thorough review of its new 2022-2026 strategies,
programmes and activities. The causal pathways to outcomes and impacts will be defined at three levels:
1) at the farmer level; 2) at the food organization (FO) level and; 3) at the institutional and systemic level.
Interlinkages between the 3 levels will also be articulated to capture the systems level and food systems
perspective of Rikolto’s operations and intended impacts.
The operationalization of fine-tuning the ToC will be done in collaboration with Rikolto’s M&E team
through a three-days in-person workshop. This includes (remote) interviews with focal points of Rikolto’s
3 programmes (Good Food 4 Cities, Rice, and Cocoa & Coffee) and other relevant Rikolto stakeholders to
delineate the monitoring, accountability and strategic learning needs that will frame the evaluation
framework. These ToCs will serve as the founding blocks in the design of the 2022-2026 evaluation
framework.
The programme-level ToCs will then be adapted to each of the 17 country contexts to capture their
specificities. The Rikolto country team members will adapt these ToCs to their local contexts, with
consultation and advice of the evaluation team before submitting it to Rikolto’s M&E and programme
focal points for feedback and co-validation.
1.1.2. Matching SMART Indicators
Once developed, the programme-level ToCs will serve as the foundation for the development of
Specific, Measurable, Achievable, Relevant, and Time-Bound (SMART) indicators to measure the
intended outputs, outcomes and impacts on the three levels. The development of these indicators will
mainly be based on existing indicators already collected in Rikolto’s existing M&E framework. Some
dimensions of the ToCs may however not have pre-established existing indicators collected by Rikolto or
may require refined indicators considering changes in intended impacts. Some additional SMART
indicators may thus be formulated for these.
To define these SMART indicators, the evaluation team will conduct a scoping of Rikolto’s existing M&E
framework including its data collection tools, system and information/data already collected through
its farmer surveys, SCOPE insight, global common indicators, internal documentation, evidence for
impact toolkit and management toolbox etc. This will allow the team to identify data and information
gaps existing in the current M&E framework. It is important to note that the evaluation team is familiar
with Rikolto’s existing data collection system thanks to a previous endline evaluation and will therefore
already have a good understanding of the current data and information needs.
The evaluation team will further assess how the gaps and needs can be addressed through Rikolto’s
existing data collection instruments by revising and complementing the existing farmer survey
questionnaires, SCOPE insight and current focus group discussion as well as key informant guides to
capture all relevant dimensions of analyses needed. In strong collaboration with Rikolto’s M&E team,
the evaluation team will make sure to match accountability requirements of the NGO’s main donors
with the defined indicators.
1
1.1.3. Standardized KIIs and FGD guides
The evaluation team will also develop standardized Key Informant Interviews (KIIs) and Focus Group
Discussion (FGD) guides for the three levels of analysis to collect qualitative information. This will
inform on the dynamics and mechanisms of change and nuance, complement and triangulate the
quantitative data collected. These qualitative tools will be based partly on the Qualitative Impact
Assessment Protocol (QuIP)1 and Rural Appraisal (RA) techniques.2
These standardized data collection tools will be developed in a participatory manner with Rikolto
teams and stakeholders. Once finalized, they will also be sent to country teams for inputs on how to finetune these data collection instruments to local contexts and to specific programmes. This participatory
approach will ensure appropriation by local Rikolto teams and ensure that the data collection tools are
harmonized and capture context and cluster-specific dynamics and realities.
These standardized tools will be adapted during the evaluation phase according to the precise
evaluation questions formulated at design phase and finetuned before each phase of evaluation
(baseline-midterm-endline). This will ensure that changing implementing and contextual realities as well
as strategic evaluation needs of Rikolto and its stakeholders/donors are considered.
1.1.4. Validation of evaluation design and capacity transfer
The evaluation team will closely work with Rikolto team members to discuss and validate the overall
approach to analysis for the three levels. This is an important part of the evaluation framework design
as it will guide the type of data and information collected, as well as the manner and means in which they
are collected across the various data collection tools. The approach proposed is one of synthesis of
information and data – both quantitative and qualitative - through systematic triangulation of evidence.
Based on the mixed-methods and theory-based evaluation approach, the data analyses will seek to
ascertain attribution where possible and feasible using the quantitative data collected and where control
groups are utilized. To achieve this, various methodologies of analysis will be applied which are described
in Table 1 below.
The close collaboration between Rikolto team and the evaluation team will ensure capacity transfer
from one team to the other. The evaluation team will be available for consultation to guide Rikolto team
from the start of the design phase to the end of the evaluation phase. This includes sending
methodological notes to local parties, which will share key points concerning the standardized data
collection instruments (farmer questionnaire, focus group and KII guides etc.), the manner in which they
are to be efficiently collected, and introductions on the methodologies on how and what information and
data is to be collected. Moreover, there will be capacity transfer in terms of how to digitize efficiently
both quantitative and qualitative data collected as well as more efficiently manage data systems and their
integration (e.g., through the need of contacting the same persons across time, having unique identifiers
to merge and append data/information, digitized coding of information, triangulation matrices, efficient
use of KoBoToolbox for survey data collection etc.).
The evaluation team, collaborating closely with Rikolto, will work towards establishing a ‘learning
journey’ framework which will underlie the whole evaluation phase. This involves a core group of
stakeholders to provide a central guiding and learning thread throughout the period of support. The
Learning Journey will last for the whole project phase and will get the teams together at key points during
the project lifecycle at baseline, midterm and end-line evaluation. Stakeholder meetings at key stages
will form part of the learning journey to include reflections of processes. The learning journey will follow
a participatory, iterative process where we will revisit the ToC periodically. Adaptive approaches do not
only improve decision making in complex environments, but also raise the quality of programming in the
face of long and uncertain pathways to achieving change. This will allow the development of
organizational learning guidelines to enhance the systems, which deliver robust reporting whilst looking
1
2
QuIP is a qualitative impact evaluation tool that collects evidence of programme/intervention impacts through narrative causal statements
from the beneficiaries and stakeholders of the programmes. Respondents are asked about to describe, over a pre-defined recall period, the
main changes in their lives and what the main driving factors of these changes were and to what or whom they attribute any changes. These
are used to triangulate quantitative data and offer rich and in-depth evidence of potential mechanisms to impacts. See: Qualitative Impact
Assessment Protocol (QUIP). Better Evaluation. Retrieved from: http://betterevaluation.org/en/plan/approach/QUIP
RA includes methods allowing sharing, mutual enrichment and assessment of local life and development issues (Chambers (1994)17;
Cornwall & Jewkes (1995)18). Facilitators help local people express their own thoughts and opinions by being actors in the analysis of their
current situation and such tools allow some quantitative ranking of the importance of certain mechanisms of change.
2
across operational processes and global programmes that promote a culture of learning across the
organization and align incentives across the various stakeholders in the MEAL processes.
1.2. Evaluation phase
Hereafter, the data collection instruments, and methods of analyses developed during the design phase
will be employed and implemented for each of the three analysis levels for all programmes selected and
agreed upon in each country and at each stage of the evaluation (baseline3-midterm-endline):
Table 1: Data collection instruments & methods of analysis
Farmer level
Data collection instruments:
•
•
•
•
Farmer questionnaire: the questionnaire will be refined and developed during the design phase according to
the dimensions of analyses and intended outcomes and impacts identified in the refined ToCs. These will include
economic, social, environmental dimensions of change and potential mechanisms and drivers of these change
(including Rikolto’s activities) to assess attribution and contribution of Rikolto activities on these dimensions.
▪
Control groups: the evaluation team will discuss and decide with Rikolto on the use of control groups and
the composition of these control groups (e.g., other similar farmers or a phase-in or pipeline design where
future similar farmer beneficiaries are used as a control group). The same farmer questionnaire would be
applied for both beneficiary and control groups.
Focus group discussions (FGD): focus group discussion sessions will be organized with a subset (or all, depending
on resources) of the survey respondents where the focus group guides using QuIP and RA developed during the
design phase will be applied to ascertain changes and drivers and mechanisms of these changes.
Key informant interviews (KIIs): KIIs with Rikolto field staff to ascertain certain perceived changes and
mechanisms of change for beneficiary farmers given their field experience.
Documentation and reports: documentation and reports of Rikolto activities and their potential contribution to
observed changes in outcomes for farmers.
(The same persons must be followed across the three phases of the evaluation to capture changes over time).
Analyses methods:
•
•
3
Rigorous impact evaluation methods: impact evaluation methods such as difference-in-differences and
matching can be applied if control groups are used, and sample sizes are large enough. The same people must
be surveyed at different stages of the evaluation. Other econometric and statistical methods (such as
multivariate regression analyses) can be used to determine attribution or at least correlations between the
programme and dimensions of change under study.
Triangulation with FGDs, KIIs & documentation: the statistical/econometric results and qualitative results and
drivers of change will be triangulated to synthesize key results and findings. Analyses will also be systematically
disaggregated by gender, age, and other characteristics to assess heterogenous effects.
Taking into account limited available resources, only selected sources of information will be analyzed at baseline.
3
Data collection instruments:
FO level
•
•
•
SCOPE insight data: the refined SCOPEinsight tool will be deployed to measure change in business and
organisational capacity of FOs (and Rikolto’s contribution to this change).
FGDs & KIIs: these guides may be developed during the design phase to fill any data and information gaps and
complement and nuance any data collected in SCOPEinsight data according to the dimensions of analysis and
evaluation needs.
Documentation and reports: documentation and reports of Rikolto activities related to FOs and their potential
contribution to observed changes in outcomes for FOs.
Analysis methods:
•
Institutional
& systemic
level
Triangulation: statistical analysis of SCOPEinsight and triangulation with KIIs and/or FGDs.
Data collection instruments:
•
•
•
•
Stakeholder mapping & analyses: this exercise will be done by Rikolto to ensure that the most relevant and
pertinent key stakeholders and institutions are taken into account in the systemic analyses of the food systems
in which Rikolto operates.
Farmer questionnaire & SCOPEinsight: the farmer questionnaires developed and SCOPEinsight will be refined
to include change dynamics in dimensions beyond the direct farmer and FOs and explore potential systemic
spillover effects on food systems and ecosystems in which they operate with questions on the drivers of these
changes (including Rikolto activities or drivers that are likely related to Rikolto activities).
KIIs & FGDs: guides developed during the design phase will incorporate QuIP and RA tools alongside traditional
qualitative questions to explore how Rikolto’s activities engaged with institutions and wider food systems and
how these may have contributed to observed changes, if any. These will be directed to institutions and
stakeholder actors in food systems that Rikolto operate in.
Documentation and reports: documentation and reports of Rikolto activities regarding institutions and food
systems.
Analysis methods:
•
Triangulation and use of Adopt-Adapt-Expand-Respond (AAER) framework: descriptive statistics of the
quantitative data sources will be triangulated with KII and FGD data collected. These will be analyzed in light of
the AAER framework4 to assess potential systemic effects of Rikolto activities in the food systems and
institutional ecosystems it operates in.
In light of resource constraints, the evaluation team proposes to decide with Rikolto and its
stakeholders at the design phase and before the commencement of the baseline to select one main
programme of interest in each country as well as certain countries of focus (which are similar to other
regional countries in terms of Rikolto operations and/or contexts) to serve as deep-dive case studies
where the evaluation team and Rikolto teams can then focus more resources and attention throughout
the next evaluation phases. This will help ensure that the evaluation phases make most efficient use of
resources to yield information and data for all countries whilst also having more in-depth data from
certain countries of interest where richer analyses can be made. These choices will be discussed and
agreed with Rikolto and stakeholders during the design phase.
1.2.1. Baseline
At baseline, the ToCs as well as the EQ developed in the design phase will be reviewed at outcome level
based on the analysis of relevant data and selected documentation. A short PPT presentation will then
be prepared at outcome level structured around the ToC and the EQ. Available sources of information,
main objectives, and a description of each outcome will be presented to Rikolto’s local staff in a
sensemaking workshop for feedback and validation. Selected baseline data and documentation will be
processed to inform various dimensions of analysis and measure relevant indicators developed in the
design phase. This will serve as the pre-intervention situation by which changes over time can be assessed
throughout the 2022-2026 period and evaluate the contribution of Rikolto and its activities to these
changes.
4
The AAER model is a framework to assess the systemic change process. See: Ton, G. & Vellema, S. (ed.). (2022). Theory-based Evaluation of
Inclusive Business Programmes. IDS Bulletin, 53(1).
4
Ideally, local partners should be involved since the beginning of the evaluation process. However, as
resources do not allow to engage with local partners at design and baseline phases, the core team5 will
conduct the baseline in close collaboration with Rikolto local teams. Each member of the core team will
be responsible for one region (see Section 2). Nevertheless, we suggest that local partners participate to
the baseline sensemaking workshop in-person, to become familiar with the programme’s ToC and
objectives and to meet Rikolto’s local teams.
If relevant, Rikolto local teams can make use of the PPT presentation to draft a baseline report at outcome
level under the guidance of ADE core team and Rikolto’s M&E team.
1.2.2. Mid-term & endline
More resources will be placed on the midterm and endline phases, where the evaluation team will hold
working sessions with Rikolto and stakeholders to review the ToC and EQ taking into account the
realities of implementation, as well as the evaluation, accountability and learning needs of Rikolto and
its stakeholders (including donors). Based on the EQ, the evaluation team will develop evaluation
matrices to define the dimensions of analyses related to the questions as well as the data sources and
methods needed to answer these. This will help frame which data collection instruments to adapt and to
implement to meet the EQ needs. These data collection instruments (Table 1) will be implemented by
the local evaluation and Rikolto teams with close supervision and guidance of the core team, and
accounting for the lessons learned at the baseline phase.
Once data and information are collected, the evaluation team will apply the selected methods of analyses
(Table 1) and develop triangulation matrices, which will then be used to triangulate information from
different sources for each level of analysis. This will allow synthesis information to derive relevant findings
in answering the evaluation questions.
The evaluation team proposes 4 field visits by the core team in the midterm and endline phases to be
decided with Rikolto. These countries should be selected in terms of: 1) their similarities with other
countries in the region in terms of Rikolto programme characteristics where findings can also be to some
extent generalized to the other similar countries) and, 2) the need for extra capacity transfers and
trainings of data collection teams to ensure that data collection is correctly implemented. These will serve
as the deep-dive case studies described above and ensure that richer analyses can be developed for these
country contexts and should preferably be the same four countries in the two phases.
The proposed methodology and budget allocation for the midterm and endline phases will be rediscussed
and agreed with Rikolto in due time.
See Figure 2 for a visual representation of the timeline of activities across the evaluation period.
2. Team, organization & timeline
2.1. Roles & responsibilities
We propose a highly qualified team of consultants and academics that possess rich and complementing
experiences in the field of impact evaluations, thematic expertise and experience in different
geographical locations. The core team will oversee the overall assignment. It will consist of Tatiana
Goetghebuer (Team Leader), Elena Serfilippi (impact evaluation and resilience expert), Luisa Van der
Ploeg (impact evaluation expert and project manager), Kevin Henkens (impact evaluation expert) and
Romain Fourmy (impact evaluation expert). The core team will be advised by Hubert Cathala given his
long experience in the thematic areas of Rikolto (thematic advisor and quality assurance). Each member
of the core team will be responsible to conduct the baseline phase as well as to supervise midterm and
endline phases for a specific region (see Figure 1).
ADE will lead and coordinate the evaluation in collaboration with other experts of the core team and
local consultants. Given ADE’s long experience in coordinating, designing, and implementing complex
impact and strategic evaluations, Tatiana Goetghebuer (Team Leader) and ADE colleagues will ensure
efficient coordination and communication amongst the core and local evaluation team as well as with
Rikolto and its stakeholders. The evaluation team will be implicated in every stage of the evaluation
5
Except for Belgium which will be handled by the local partner Tessa Avermaete.
5
process (see Figure 2) and closely coordinate and supervise the local experts and Rikolto and data
collection teams throughout the evaluation phases.
Overall, the selected members in both ADE core and local teams are open-minded, team players,
flexible, and solution oriented: four essential skills in such a challenging, complex, and large impact
assessment.
The local team will collaborate with the core team by providing an in-country coordination and
supervising role in terms of the data collection phases and contribute to drafting reports and
deliverables given their country and thematic expertise in the different phases of the assignment. This
team will consist of 18 local experts, one for each country6. They will work with the core team and Rikolto
to train in-country data collection teams (particularly at the baseline phase). These data collection teams
will then be used at each stage of the evaluation under the supervision and coordination of the country
experts.
Thanks to our extensive network, ADE has already identified and contacted 12 country experts that are
suitable for carrying out this critical function (see CVs in Annex). They were pre-selected due to having
worked with ADE or other experts in the evaluation team and whom the team trusts to fulfil their incountry coordination roles effectively and efficiently and providing analyses and deliverables of quality
(including strong critical thinking, analytical, and drafting skills). For the remaining 7 countries, ADE will
work with Rikolto to find reliable and competent local/regional experts that both the evaluation team
and Rikolto can rely upon to adequately supervise in-country activities and produce deliverables (and
analysis and drafting) of high quality.
Overall, the core team will design the evaluation framework (evaluation questions, cluster-level ToCs,
SMART indicators, and methodological set-up). In addition, it will be responsible for the revision and
analysis of the 2022 baseline and will lead the implementation of both the mid-term and the end-line
evaluation. The local experts will be actively involved in the within-country data collection process and in
writing the country and programme reports to support the development at midterm and endline. This
will be done under the supervision and coordination of ADE and the core team and in very close
collaboration and communication with Rikolto and local Rikolto teams.
Figure 1: Proposed evaluation team
6
ADE is also willing to engage with one or more regional local experts. This possibility can be discussed with Rikolto at the design phase of the
evaluation framework.
6
2.2. Team coordination
To coordinate and balance the workload, dedicated core team members will be responsible for specific
geographic areas and the core team will ensure regular communication with local consultants and local
Rikolto teams responsible for data collection. This will ensure a smooth and consistent process across
geographical zones as well as fostering a learning path. To do so, ADE suggests using WhatsApp groups,
e-meetings and a collaborative and knowledge-sharing e-platform (e.g., Microsoft Teams) where
tutorials, help desks and agendas will be easily accessible.
At the design and baseline stages, the core team will ensure clear communications and capacity
transfers to the local Rikolto teams in terms of the data collection instruments, means of efficiently
implementing them and for methodologies of analyses. Back-and-forth communication will ensure
alignment on the designed tools and framework to ensure that everyone has understood the framework
and the means to implement it.
For the mid-term and endline phases:
• Initial e-meetings at country level will bring together the ADE core team, the Rikolto evaluation
team and the ADE local team to clearly define the scope of the outcome evaluation and reflect
on the methodology, data collection tools and its practical implementation (agenda, sharing
existing documents/data, list of key stakeholders to consult...).
• Further e-meetings will be organized between the Rikolto, in-country data collection teams and
external evaluators to ensure common understanding of the approach, the deliverables (and
their expectations), and the communication process.
• Once data collection commences, the core team will be in communication with the country
experts, data collection teams and local Rikolto teams to supervise, coordinate and offer support
to the data collection process. This will ensure that if issues arise, that they are promptly resolved,
and that the data collected will thus be of high quality.
• Once the assessments are finalized, a final e-meeting will be organized in each country to present
respective provisional findings and discuss possible limitations with Rikolto.
The goal of these meetings is to refine and validate the different findings while following a rigorous
participatory sense-making approach. Once the preliminary findings are validated, recommendations will
be formulated following collective intelligence processes.
2.3. Core team
The core team integrates a group of competent and highly regarded professionals with extensive
experience in policy-relevant impact evaluations and MEAL systems. The members of the team come
from diverse backgrounds and have complementary expertise and interests in food systems, agricultural
value chains, resilience, farmer organizations, inclusive business development, market systems
development and sustainability. Also, they possess substantial knowledge of Rikolto’s work and mindset.
Most core team members also have substantial knowledge of Rikolto’s work, vision and overall MEAL
framework due to past evaluations and work experience with Rikolto.
Tatiana Goetghebuer – Team Leader
Tatiana is a Doctor of Development Economics and Director of ADE and Head of the Impact evaluation
department of ADE. She has extensive expertise and experience in designing rigorous impact
evaluations in relation to field reality. She has been working in the field of development for 20 years
and is responsible for managing multiple large teams in methodological design (quasi-experimental
designs combined with more qualitative approaches), writing and reporting, data collection, data
analysis and interpretation of results, to meet the objectives of evaluations, using rigorous approaches
adapted to the reality of the field and taking into account all the stakeholders, while remaining
transparent about the limitations. In addition to her excellent analytical competence, technical
expertise and solid field experience, she has strong didactic, communication and management skills,
having managed large teams in numerous, long-term complex impact evaluations in various contexts.
She also has strong knowledge of Rikolto and its MEAL system due to her role as an evaluator in the
7
endline evaluation of Rikolto 2017-2021. She has extensive experience in the field, particularly in Africa
and Latin America. Tatiana will bring in the following skills:
✓ her team leader and management experience of large teams in long-term complex, mixedmethods evaluations: she is familiar with fieldwork and knows how to manage multicultural
teams bringing together various expertises,
✓ her excellent knowledge of evaluation (and impact evaluation) methods and techniques,
✓ her extensive experience in designing and implementing surveys (in agriculture, food security,
gender, and inclusive business domains etc.), online and telephone surveys, focus groups and
individual interviews with various stakeholders,
✓ her leading analytical skills,
✓ her extensive field experience in multiple African and Latin American countries,
✓ her natural talent for communication, facilitation, coordination and knowledge sharing,
✓ her knowledge of Rikolto’s work, mindset and MEAL system,
✓ her approach to restitution and validation of results by stakeholders and participatory
approaches to evaluations,
✓ her ability to communicate in English, French, and Spanish,
✓ her pedagogical and training skills: she participates in various capacity transfer events and
courses in various contexts (including academic teaching and professional training in regional
conference on impact assessment in Francophone Africa etc.).
Elena Serfilippi – Agroeconomics, Sustainability and Resilience Expert (East Africa)
Elena Serfilippi, PhD, has extensive experience in agricultural livelihoods and resilience. She is a
Monitoring and Evaluation (M&E) expert with extensive experience in quantitative research and impact
evaluations of various sustainability and resilience related initiatives in the field of Agro and Behavioural
Economics, Sustainability and Resilience. She is currently the coordinator of resilience research at the
Committee On Sustainability Assessment (COSA) where she has been leading a working group of leaders
in the field of resilience to help filter and test practices so that resilience can be more readily measured
and understood and a common set of metrics, indicators and indices based on what is working becomes
widely recognized. She also has strong knowledge of Rikolto and its MEAL system due to her role as an evaluator
in the endline evaluation of Rikolto 2017-2021. Elena will bring the following skills:
✓ her excellent knowledge of farmers’ resilience and related literature,
✓ her flair for transforming complex systems into simple, effective tools,
✓ her experience in creating, developing, and managing impact assessments systems in Africa
and Latin America,
✓ her persistence and ability to get things done even in complex environments,
✓ her knowledge of Rikolto and its MEAL system,
✓ her ability to communicate in English, French, and Spanish.
Hubert Cathala - Agroeconomic Expert Advisor
Hubert holds the title of Engineer in Tropical Agronomy and has 25 years of experience in the planning,
implementation and evaluation of rural development projects in the field of agricultural value chains
development. He has extensive expertise in the field of agronomy, agro-economics, production systems
analysis, support to local authorities and farmers' organisations, natural resources management,
amongst others, in which he has conducted numerous evaluations. He also has a strong understanding
of Rikolto, its activities and its MEAL system given his role as Team Leader in the endline evaluation of
Rikolto’s 2017-2021 activities/programmes. Hubert will bring in the following skills:
✓ his extensive expertise and experience in thematics that Rikolto work in,
✓ his ability to successfully manage complex and large studies,
8
✓ his excellent knowledge of food systems, agricultural value chains, and FO capacity
development,
✓ his field experience in Belgium, Burkina, DRC, Ecuador, Mali, Peru, Senegal, Tanzania, Uganda,
and Vietnam,
✓ he is perfectly trilingual, speaking English, French, and Spanish.
Luisa Van Der Ploeg – Impact Evaluation Expert (West Africa & DRC) & Project Manager/Coordinator
Luisa is an ADE in-house evaluation expert with strong quantitative and didactic skills and specialized
in development and public policy economics. Through her training and professional experiences, she
acquired a solid knowledge of economic theory and empirical analysis tools. Luisa is a quantitative
analyst mastering a variety of econometric, statistical and experimental methods, and has hands-on
experience with quantitative and qualitative data collection methods (surveys, focus groups,
interviews). She has experience in coordinating and supporting large cross-country impact and
programme evaluations. At ADE she is involved in a range of evaluations covering a wide range of
sectors, including forestry, environment, resilience, and food security. Luisa is currently research
analyst and project manager of the endline impact assessment of the programme implemented by
Rikolto, a Belgian NGO working with farmers and farmers organizations (FOs) worldwide, with the
objective of tackling the interrelated challenges of food insecurity, climate change, and economic
inequality. She therefore has strong knowledge on Rikolto’s activities and existing MEAL system. Luisa
will bring the following skills:
✓
✓
✓
✓
✓
her solid knowledge of economic theory,
her experience in collecting quantitative & qualitative data
her strong command of empirical analysis tools,
her project management and coordination skills,
her extensive knowledge of Rikolto and its MEAL system.
Kevin Henkens –Impact Evaluation Expert (Asia)
Kevin is an in-house impact evaluation expert at ADE. He is specialized in public policy and human
development evaluations and has strong expertise in mixed-methods impact evaluations. He has
managed rigorous impact evaluations of inclusive business development programmes and in
developing MEAL systems. Prior to joining ADE, Kevin worked on numerous socioeconomic impact
assessments of COVID-19 in various African and MENA contexts as well as numerous evaluations which
harnessed big data and state-of-the-art data science methods to evaluate policies and programmes to
inform evidence-based decision-making. Kevin will bring the following skills:
✓
✓
✓
✓
✓
His strong data analysis & data presentation skills,
His experience in collecting quantitative & qualitative data,
His strong experience in developing and managing data systems for MEAL
His knowledge of multiple empirical analysis tools,
His experience in managing and supervising complex impact evaluations in various contexts.
Romain Fourmy – Impact Evaluation Expert (Latin America)
Romain is an impact evaluation expert with strong experience in collecting, processing, analysing, and
visualising both qualitative and quantitative data from different sources. Moreover, he participated in
strategic evaluations in the field of digitalisation in Africa, for DG INTPA (2020) and WFP (2021). He has
a strong interest and is familiar with the digital ecosystem in terms of incentives and initiatives led by
the different actors. Besides, he participated to other evaluations on various topics, such as on food
security, well-being, and natural resource management. Prior to joining ADE, Romain worked as a
9
research assistant at the UNU-Merit and contributed to research related to diverse topics in Africa, such
as on the contribution of mobile money on development.
✓
✓
✓
✓
✓
His strong data analysis & data presentation skills,
His experience in collecting quantitative & qualitative data,
His strong experience in developing and managing data systems for MEAL
His ability to communicate in English, French, and Spanish
His experience in managing and supervising complex impact evaluations in various contexts.
Table 2: Core team competencies
Elena Serfilippi
Romain Fourmy
Luisa Van der Ploeg
Kevin Henkens
Hubert Cathala
Advisors/QA
Tatiana Goetghebuer
Core Team
Evaluation methods and techniques
✔✔✔
✔✔✔
✔✔✔
✔✔✔
✔✔✔
✔✔
Experience in coordinating teams
✔✔✔
✔✔
✔✔
✔✔✔
✔
✔✔✔
Experience in impact assessment
✔✔✔
✔✔✔
✔✔
✔✔
✔✔
✔✔✔
Writing and editing skills
✔✔✔
✔✔✔
✔✔✔
✔✔✔
✔✔✔
✔✔✔
Ability to work remotely
✔✔✔
✔✔✔
✔✔✔
✔✔✔
✔✔✔
✔✔✔
Capacity to provide critical-constructive feedback
✔✔✔
✔✔✔
✔✔
✔✔
✔✔
✔✔✔
Capacity to transmit methodological improvements
✔✔✔
✔✔✔
✔
✔
✔
✔✔
Methodological affinity with Rikolto's impact assessment framework
✔✔✔
✔✔✔
✔✔✔
✔✔✔
Knowledge of Rikolto and its MEAL system
✔✔✔
✔✔✔
✔✔✔
✔✔✔
✔✔
✔✔✔
✔
✔✔
✔✔✔
✔✔✔
✔✔
✔✔
Agricultural value chains
✔✔
✔✔
✔
✔✔
✔✔✔
FO capacity development
✔
✔
✔
✔✔✔
English
✔✔✔
✔✔✔
✔✔✔
✔✔✔
✔✔✔
✔✔✔
French
✔✔✔
✔✔✔
✔✔✔
✔✔✔
✔✔✔
✔✔✔
Spanish
✔✔
✔✔
✔✔
✔
Impact assessment in food system
Change and upscaling dynamics in food systems
Policy environment
✔
✔✔
✔
✔
✔✔
✔✔
✔✔
✔✔✔
✔✔✔
Some experience ✔
Multiple experiences ✔✔
Extensive experience ✔✔✔
2.4. Local team
The local evaluation team will be involved at the four stages of the evaluation process. Thanks to its broad
network of local experts, ADE has already successfully pre-identified and contacted available, motivated,
and experienced local evaluators for 12 of the 18 countries of interest (Belgium, Benin, Burkina Faso,
Cote d’Ivoire, DRC, Ghana, Mali, Rwanda, Senegal, Tanzania, Vietnam, and Uganda). A second rigorous
round of selection will be put in place to ensure the evaluators possess the required coordinating,
supervising, writing and data collection and analysis skills, and are proactive and trustworthy. ADE, if
selected, will collaborate with Rikolto to find suitable local/regional consultants for the remaining
countries (Indonesia, Ecuador, Peru, Guatemala, Honduras, Nicaragua).
10
2.5. Quality assurance mechanisms
Quality is a key priority for ADE. This relates notably to the transparency of the process, the credibility of
the exercise, the utility of conclusions and recommendations, and the efficacy in meeting the evaluation’s
primary objective(s).
ADE’s Quality Assurance System (QAS) is organised to conduct
complex and challenging evaluations such as this one. Our quality
assurance system aims at ensuring that high-quality studies are not
only delivered but delivered in a timely manner and through a smooth
process both within the team and in interaction with the client. This
implies high-quality design and set-up of process and outputs. It also
implies that potential flaws are detected sufficiently early to ensure
adequate corrective action. We distinguish several dimensions in a
high-quality process, each of which being essential for the overall
product and final result. ADE’s experience of delivering complex
evaluations shows that ensuring quality along those four dimensions
(Design, Inputs, Process and Outputs) requires the following four
levels.
QUALITY LEVELS
1
Quality of the TEAM
2
Quality within the TEAM
3
EXTERNAL quality control
4
BACKSTOPPING
Level 1. Quality of the team. The composition of the proposed
evaluation team is centred on delivering quality, given the specific
characteristics and constraints of the project. The Team Leader
(Tatiana Goetghebuer) and team members have been selected from
within our network of high-level experts based on their (individual
and global) experience and skills. In addition, the inclusion of ADE
in-house researchers favours integration of ADE’s methods and
standards, and eases access to ADE’s internal resources and staff.
This includes interactions with senior staff and with administrative,
financial, logistical, secretarial, interpreting, IT support, editing, and
graphic design staff.
Source: ADE
Level 2. Quality within the team. Each team member is responsible
for the quality of his or her outputs throughout the evaluation with the Team Leader taking ultimate
responsibility for ensuring that deliverables are drafted in line with the client’s expectations and
standards before they are sent to the quality controller. The Team Leader and the Quality Controller will
ensure that all team members are briefed on ADE’s methodological approaches and quality standards, as
well as on the scope and approach for the evaluation mentioned in the ToR and intermediary reports.
Level 3. External Quality Control. The quality assurance (QA) expert will be Hubert Cathala. He will ensure
quality control. Hubert has extensive experience as a team leader, project manager and quality controller
and has worked on the previous endline evaluation (2017-2021) of Rikolto and is therefore well aware of
Rikolto’s activities but also of the lessons learned both in terms of evaluation process and results of the
previous evaluation. He will guarantee that the different levels of quality assurance are fully completed.
He will review all main (intermediary) deliverables and check in particular that each of them meets (i) the
requirements of the ToR, (ii) the proposed approach in this offer, (iii) the approach to be fine-tuned in
the intermediate reports, (iv) good ADE practices in thematic evaluations, and (vi) ADE’s editing standards
(incl. proofreading, formatting, translation, visuals and reader-friendliness review). The evaluation team
will have to take account of all his written comments or provide a justification for not doing so. They
should provide their green light before submission of (draft and final) reports to Rikolto.
Level 4. Backstopping. ADE will ensure continuity of services in the event of any major issue. Edwin
Clerckx, ADE Managing Director, will be available any time to Rikolto for discussing any specific concern
on implementation of the contract, and to propose solutions to any problem that may arise. He may
hereby rely on support from ADE’s 40+ staff, including more than six senior experts with a track record
as evaluation team leaders.
11
Effectiveness of the overall ADE Quality Assurance SystemWhat better
proof of the quality of our evaluations than clients’ quality assessments?
We are proud of ADE’s outstanding rate of high-quality evaluations. We
may evidence it in three main ways:
1/ Client Quality Judgment Grids: Not least than half of ADE’s 18 strategic
evaluations conducted for EuropeAid’s Evaluation Unit until recently had
received a ‘Very Good’ or even ‘Excellent’ rating based on their judgment
quality grid, which is an unequalled performance by any other firm.
2/ Client feedback assessments: Out of 282 studies for which ADE has
received feedback from clients, 89% were rated of “Excellent / Very Good”
or “Good” quality, with not least than 63% “Excellent / Very Good”.
3/ ISO 9001:2015: ADE is certified ISO 9001, for two decades (1999). This certification has been renewed
in 2018 according to the latest ISO 9001:2015 procedures.
2.6. Timeline of activities
Figure 2: Timeline of activities
12
PROPOSED CALENDAR
Activities and Deliverables
Aug-22
Sep-22
Oct-22
Nov-22
Dec-22
Jan-23
2024-25
2026-27
DESIGN PHASE
Desk review
Desk review of relevant documents
ToC finetuning
Preparation of the ToC workshop at programme level
Workshop with Rikolto on ToC at programme level
Adaptation of the ToC at outcome level incorporating context specificities
Revision of the M&E system
Review existing indicators & existing data/information
Matching & Adaptation/refining of existing data collection tools & defining new indicators
Integrate accountability requirements of major donors
Consultations to Rikolto's team for the digitalization of their M&E system
Inception PPT
Preparation of methodologies of analysis
Brainstorming with Rikolto staff on learning EQ
KII & FGD guides
Methodological notes (PPT) summarizing the methology and next steps
EVALUATION PHASE
Baseline
Desk review of relevant documentation
Interviews with Rikolto staff
Data analysis at baseline
Short PPT at outcome level
Presentation of the PPT to RT and LT (hybrid)
Revision of the PPT based on the workshop
Midterm
Desk review
Inception meetings & revision of the evaluation questions (EQs)
Short inception PPT & development of evaluation matrices
Adaptation of ToC, data collection instruments & processes based on EQ needs
Data collection
Data analysis & visualization
Sensemaking of the results
Drafting and revising mid-term reports at outcome level
Debriefing with relevant teams
Endline
Desk review
Inception meetings & refining evaluation questions (EQs)
Short inception PPT & development of evaluation matrices
Adaptation of ToC, data collection instruments & processes based on EQ needs
Data collection
Data analysis and visualization
Sensemaking of the results
Drafting and revising endline reports at outcome level
Debriefing with relevant teams
13