World Vision’s Approach to Local
Ownership in Evaluation
Korça, Albania Development Program –a case study
Prepared by: Holta Trandafili and Albana Dino, World Vision US
March 2015
Summary
• Intro to the Development Program Approach (DPA) and
how that informs our evaluation process
• Background of Korça, Albania Development Program
• Role of local actors in the evaluation process
• Why invest in local ownership in evaluations
• When to use this approach to evaluations
• What does it take? Who and timing?
• Testimonials
2
Development Program Approach
Sustained well-being of children within families and communities –
especially the most vulnerable
Includes:
• Understanding of
partnering
• Types of partners
• The critical path
Includes:
• Child well-being outcomes
• Child participation
• Defined project models
• Measurements
Working
with
partners
Includes:
• Roles of WV local -level staff
• Key competencies
• Management support
Equipping
local-level
staff
Contributing
towards child
well-being
Basic
programme
parameters
Includes:
• Geographic size
• Target populations
• Life span and cycle
• Resourcing
• Governance
• Disaster management
• Sponsorship programming
(where applicable)
3
The Critical Path for Working with
Communities and Local Partners
Baseline data
collection &
analysis
Programme/Project
Descriptions, incl.
Indicators
(All Partners)
World Vision
Logframe and
Design Document
Assessment Plan
Assessment Report
Korça Area Development Program (ADP)
Program Phase
1 October 2010- 30 September 2013 (1st implementation phase)
Evaluation Type
Mid-term Programme Evaluation (end of 1st implementation phase); Formative;
Evaluation Purpose
To generate learning from the program’s first cycle design and operation to inform
the redesign process and product.
Primary
Methodologies
Evaluation methodology was based on participatory approaches, based on
naturalistic inquiry (qualitative); Details were developed by the Lead Evaluator
Implementer
World Vision Albania
Funder
World Vision Hong-Kong
Evaluator
External evaluator hired by World Vision International
Korça ADP local partners and World Vision Albania (redesign for next phase)
Intended users of
World Vision Hong Kong (inform donors on progress)
Evaluation Product
World Vision International (organizational learning)
Duration
4 months (February – May 2013)
Program target
population
The ADP area has 50,100 inhabitants (8,432 children aged 0-18 years old) and
operates in three rural areas and one small municipality (Pojan, Vreshtas, Libonik,
and Maliq). 3500 registered children in Sponsorship project (aged 1-11)
Projects in the
program
Two projects implemented: Education and Sponsorship
Who were these local actors in involved in
Korça Program Evaluation?
• 48 local partners with solid foundational relationships
established from program inceptions and who are still
actively sustained were initially contacted; 20 of them
participated in workshops and 10 of them were
selected by the network to represent the local actors
voice through-out the process.
• The types of partners range from informal groups to
established CBOs, FBOs and local NGOs or local
government and regional directorates.
The nature of the participatory approach of Korça evaluation
What it was NOT:
• A capacity building process that
trains community members to
collect information for World
Vision
• Simply a process to get
information from the
community for World Vision
reporting needs
• A one time off participatory
event (actors / partners called
on purpose for only this step –
evaluation)
What it was:
• A capacity development and
learning process that provides
opportunity for all actors to
meaningfully participate in all
the steps of Evaluation.
• A process that seeks to meet the
needs of all partners (existing
community groups, individual
volunteers, local government
reps, local NGOs, etc.,)
• A continuation of working in
partnership to jointly design,
baseline, monitor and measure
the program progress.
How were participants involved? Overview of process
Local staff and
partners agree to
commence the
evaluation process
Articulating biases of
involving local
partners (informant
vs. part of eval. team)
Chart partners in
informants vs. eval.
team participants.
Determine follow-up
and next steps
Report drafting
(different formats)
Share draft ToR with
local partners
Articulating
evaluation limitations
(what it can say and
what not)
Local partners shape
ToR
(link Theory of Change,
purpose of Eval and
Objectives)
Develop sampling
framework
Agreed ToR for review
of National Office,
Funding Office and
WVI
Recruitment of the
Evaluation Leader to
deliver on the ToR
Evaluation Design
Workshop
Finalize indicators
Mapping local social
activities, interventions,
beneficiaries.
(articulation of success
in local terminology)
Finalize tools
Build capacity of eval.
Team members on
tools.
Form data collection
teams
Data analysis
Data Collection
Local eval. team
members determine
level of engagement
and in what step(s).
Presentation of
findings, reflection &
recommendations
(from Eval. Team to
network and wider)
Validation of findings
(link Theory of Change,
purpose of Eval and
Objectives)
Final report shared with
Funding Office
Legend
Community involvement
WV facilitates/ builds capacity
WV process
Topics covered by the Evaluation
Questionnaire
• Impact
• Partnership
• Partner Learning and
Capacity
•
•
•
•
Sponsorship
Most Vulnerable
Sustainability
Future
Field Work
• 17 Focus Groups
• 12 Key Informant
Interviews
• 5 Korça ADP Staff
Interviews
• 3 Partner Interviews
Initial Interpretation
• Step One: Type up focus group and key
informant interview responses – 5 teams –
70+ pages of notes
• Step Two: Describe the data
• Step Three: Present findings
• Step Four: Formulate Recommendations
Participatory Evaluation Process
Group Work -Analysis
Evaluation
Findings
Evaluation Discussion & Debate
EVALUATION Celebration
Evaluation Team
Participants’ Evaluation of the Locally
Owned Evaluation in Korca, Albania
Positive
– Partners included
– Discussion and debate
– Feeling of equality – partners, ADP staff, national
office
– Qualitative evaluation process modeled and included
full participation at every step
– External party facilitating evaluation
– Work in groups
– Engagement and commitment of WV staff
– Translation
Participants’ Evaluation (2)
Needs Improvement
– Not enough time for discussion and debate
– No accreditation or certification for participants who
are not staff members
– Not enough time for data collection and analysis
– Broader sample of focus groups required
– More in depth and better review of sponsorship
project required
– More review between evaluation teams and lead
evaluator (more direction/feedback required)
– More / deeper questions required of interviewees
Why use Local Ownership
in Evaluations?
It puts information in the hands
of the main user: the
community!
It can save money
Local Ownership to Evaluations
–when to use it?
Enabling factors:
• Substantive in-depth time invested “up front” for the initial
community engagement process
• Acknowledgment and joining paths with community actors –their
vision, plans, articulation of success indicators
• Staff commitment to Community Empowerment Approach (local
ownership, meaningful participation, capacity development)
• Rules of engagement of effective partnership are continually
adhered to:
a) Concern for equity, b) Commitment to
transparency, and c) Emphasis on mutual benefit
19
Challenges to consider:
• If local ownership and participation was not build in the design
phase then applying local ownership in evaluation is difficult.
• Difficult to apply in communities with little to no community
organization
• The involvement of technical specialists, while necessary in an
evaluation requires their buy-in a participatory process and
working with the groups to present the process and the
findings in palatable way (layman terms and using local
language).
• The big assumption that:
– we have the capacity to lead these highly participatory processes and
– the organization is ready for meaningful shared learning and sound
recommendations
• Challenges with time...
20
Summary
• A shared project/program suggests shared ownership and in the
context of a shared ownership we can have locally owned
evaluations.
• Locally owned evaluations are not a coincidence –they are the
product of hard work to build ownership over the project during the
whole program/project cycle.
• Local ownership is the highest level of participation where “owners”
are conscientious about what is happening... And they can shape
the process and implement it and above all they learn from it.
• The level of conscience of the local groups / partners should lead to
understanding where they are with the competences and when to
seek assistance from technical specialist to guide for rigor…
21
Korça Testimonials
Partners unanimously praised World Vision Korça ADP for
their outstanding commitment to “genuine partnership.”
One partner said, “I have worked with other organizations
in other places, and in these other places, I have felt that
when World Vision talked to us about partnership, they
were really trying to see how much milk they could get
out of the cow. I have not felt that way at all in Korça, and
have profited immensely from the relationship with World
Vision. They truly have put everyone on an equal plane
and are trying to serve others, not just themselves and
their plans.”
END