Demistyfing
equity-focused evaluations
[EFE]
Marco Segone, UNICEF Evaluation Office
Anne-Claire Luzot, M&E Regional Advisor, UNICEF CEECIS
EO is developing a resource package on EFE.
This presentation is therefore a work in progress
Outline
1.
2.
3.
4.
What does Equity mean?
Why does Equity matter?
What is an equity-focused evaluation?
How to manage an Equity-focused evaluation?
What does equity means?
Defining Equity
• Equity means that all children have an
opportunity to survive, develop, and reach
their full potential, without discrimination,
bias or favoritism
• The disparities between population groups
that are not driven by biology, are
avoidable and unfair are termed inequities.
• Equity is therefore based on notions of
fairness and social justice
All rights for all children everywhere, by
prioritizing the most deprived
Graph 1: % of children (7-14 years old) out of school, by race/ethnicity
2000 Baseline and goals for national average, black and white children
2010 Goal: reduce by 50% the national average and the equity ratio between black and white children
Equity Ratio : 1,8
6.9
5.5
Black children
National average
3.1
2.8
2.2
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
1,4
White children
Equity ratio:
3.8
Drivers: Income
Under-five mortality rate in Central Asian countries,
by wealth of households
110
per 1000
Poorest 60%
Richest 40%
55
0
Kazakhstan
Kyrgyzstan
Uzbekistan
Tajikistan
UNICEF
Source: DHS data. From NAGA pg. 27
Drivers: locations
Proportion of children aged 0-59 months living in households that
do not provide adequate support for early learning and development, by
worst-off and best-off regions
60
%
Best-off region
Worst-off region
40
Eastern Europe
UNICEF
Tajikistan
Kyrgyzstan
Uzbekistan
Albania
BiH
Georgia
Belarus
Serbia
Macedonia
Montenegro
0
Kazakhstan
20
District level Data
- Nyanza
Province
Central
Asia
Drivers: education
50
%
Proportion of children aged 0-59 months living in households that
do not provide adequate support for early learning and development, by level of
education of mother/caretaker
Basic or less
Higher than basic
Eastern Europe
UNICEF
Central Asia
Tajikistan
Kyrgyzstan
Uzbekistan
Albania
Bosnia and
Herzegovina
Georgia
Belarus
Serbia
Macedonia
Montenegro
0
Kazakhstan
25
Drivers: Gender
The majority of out-of-school children
continue to be girls (53%); achieving
gender parity would mean 3.6 million
more girls in primary school
Drivers: Disabilities
90% of children with disabilities in
developing countries do not attend
school
Multiple Dimensions: Who is not learning? Disparity
in learning
How is equity different from or similar to the Human
Rights Based Approach and CRC?
• UNICEF’s foundation strategy of a human rights based
approach underpins the equity agenda.
• Equity cannot be effectively pursued outside of a human
rights framework, while human rights cannot be realised
while inequity persists.
• Reaching the marginalized and excluded has always
been integral to UNICEF’s work (Progress for Children
2010). We are doing it. But are we doing it sufficiently
and everywhere?
12
Why does equity matter?
Why equity
Equity has a significant positive impact in reducing poverty
Poverty is very sensitive to income distribution changes, and small
improvements in income distribution can have an important positive
effect in reducing poverty.
Equity has a positive impact on economic growth
Equity and economic growth rates are directly related, with higher
equity having a positive impact not only on poverty reduction but also
on economic growth itself. Higher equity can therefore benefit the
poor in two ways: by increasing the overall growth rate and average
income, and by letting them share in that growth, since growth with
less inequity benefits the poor more.
Why equity
Equity has a positive impact in the construction of a socially fair
and democratic society
Unequal opportunities of social groups in society is often a significant
factor behind social unrest, which may lead to crime or even violent
conflict, with negative effects for the social cohesion of a society.
Prolonged inequity may lead to the “naturalization” of inequity
The long “stability” of inequity in several countries makes it
something “natural”. When inequity is perceived as a natural
phenomenon, society develops theoretical, political and ideological
resistances to identify and fight inequity as a priority in public
policies.
Inequity constitutes a violation of human rights and hampers
the equitable achievements on Human Development and MDGs
What is
an equity-focused
evaluation?
What is an equity-focused evaluation?
A judgment made of the relevance, effectiveness,
efficiency, impact and sustainability of
interventions on equitable development results.
Purposes of equity-focused evaluation





Accountability
Organizational learning and improvement
Evidence-based policy advocacy
Contribute to Knowledge Management
National Capacity development for equityfocused M&E systems
 Empowerment of worst-off groups
Empowering worst-off groups, including children,
through Equity-focused evaluation processes
• Involve worst-off groups:
– To ensure relevance, effectiveness, ownership of
evaluation
– Focus on Evaluation process use to empower
worst-off groups through the evaluation process
– Including children
• Understanding the cultural, political economic
and social context that influence inequity,
including perception of, and attitudes to,
equity
Why are equity-focused evaluation required?
•
Because “conventional” evaluations have some
potential limitations in evaluating equityfocused interventions:
–
–
going beyond conventional quantitative data to
analyze behavioral change, complex social
processes and attitudes, and collecting information
on difficult-to-reach socially marginalized groups.
In addition, conventional evaluation are often weak
on process and contextual analysis.
Is equity-focused evaluation a
completely new approach?
No. While some new analytical tools are
introduced, most of the EFE data collection and
analysis techniques are built on existing
approaches
The emphasis is on refining and refocusing existing
technics – and enhancing national capacities to use those
technics
HOW to manage
Equity-focused
evaluations?
Key Steps
2. Preparing
the ToR of
an EFE
1
1. Preparing
for the EFE
2
4.
Collecting/
analysing
evidence
3
3. Designing
the
evaluation
4
6. Utilizing the
evaluation
5
6
5. Findings,
conclusion and
reccs
MANAGE RISKS * MANAGE RISKS * MANAGE RISKS * MANAGE RISKS
Key Steps
1
2
3
4
5
6
1. Preparing
for the EFE
MANAGE RISKS * MANAGE RISKS * MANAGE RISKS * MANAGE RISKS
. Preparing
for
the Equity-focused evaluation
• Determining the evaluability of the intervention’s
equity dimensions
• Identifying evaluation stakeholders, including
worst-off
• Identifying intended use by intended users
• Identifying potential challenges in promoting and
implementing Equity-focused evaluations
Key Steps
2. Preparing
the ToR of
an EFE
1
1. Preparing
for the EFE
2
3
4
5
6
Preparing the ToR of an EFE
• Defining the scope and purpose of the evaluation
• Framing evaluation questions focusing on equity
• Selecting technically-strong and culturally-sensitive
evaluation team
• Ensuring appropriate budget
Framing evaluation questions focusing on equity
DAC criteria adapted for EFE
•
•
•
•
•
Relevance
Impact
Effectiveness
Efficiency
Sustainability
Key Steps
2. Preparing
the ToR of
an EFE
1
1. Preparing
for the EFE
2
3
3. Designing
the
evaluation
4
5
6
Designing the evaluation
• Selecting the appropriate evaluation framework
– Theory of Change
– Bottleneck analysis
Bottleneck supply and demand framework: factors
affecting use of services by vulnerable populations
Contextual factors
• Economic
• Political
• Institutional
• Legal and administrative
• Environmental
Supply side
factors
• Coverage
•Budgets and
available resources
•Culturally
acceptable
services
• Culturally
sensitive staff
Use of services by
vulnerable population
Demand side factors
• Knowledge,
Attitudes and Practice of
vulnerable groups
• Community ownership
factors affecting access
• distance
• cost of travel and fees
• available transport
• cultural constraints
• time poverty
31
Designing the evaluation
• Selecting the appropriate evaluation framework
• Selecting the appropriate evaluation design
Selecting the appropriate
evaluation design and tools
Contextual factors
• Economic
• Political
• Institutional
• Legal and administrative
• Environmental
Supply side
factors
• Coverage
•Budgets and
available resources
•Culturally
acceptable
services
• Culturally
sensitive staff
Use of services by
vulnerable population
Mix-Methods
Demand side factors
• Knowledge,
Attitudes and Practice of
vulnerable groups
• Community ownership
factors affecting access
• distance
• cost of travel and fees
• available transport
• cultural constraints
• time poverty
Designing the evaluation
• Selecting the appropriate evaluation framework
• Selecting the appropriate evaluation design
• Selecting the appropriate evaluation methods
Key Steps
2. Preparing
the ToR of
an EFE
1
1. Preparing
for the EFE
2
4.
Collecting/
analysing
evidence
3
4
5
6
3. Designing
the
evaluation
MANAGE RISKS * MANAGE RISKS * MANAGE RISKS * MANAGE RISKS
Selecting the appropriate evaluation methods
Approaches to be used
to evaluate complex equity-focused policies and
programmes
• Systems approach to evaluation
• Unpacking complex policies into components
that can more easily be evaluated
• Pipeline designs
• Policy gap analysis
• Using other countries or sectors as the
comparison group
• Concept mapping
• Portfolio analysis
Key Steps
2. Preparing
the ToR of
an EFE
1
1. Preparing
for the EFE
2
4.
Collecting/
analysing
evidence
3
3. Designing
the
evaluation
4
5
6
5. Findings,
conclusion and
reccs
MANAGE RISKS * MANAGE RISKS * MANAGE RISKS * MANAGE RISKS
Key Steps
2. Preparing
the ToR of
an EFE
1
1. Preparing
for the EFE
2
4.
Collecting/
analysing
evidence
3
3. Designing
the
evaluation
4
6. Utilizing the
evaluation
5
6
5. Findings,
conclusion and
reccs
MANAGE RISKS * MANAGE RISKS * MANAGE RISKS * MANAGE RISKS
Utilizing the evaluation
•
Preparing the evaluation report and alternative
forms of reporting
• Disseminating the evaluation and preparing a
Management Response
Do you want more?
Visit MyM&E
Remember
This is NOT what I was expecting!!!!!
Whose fault is it?
Invest time/energy in designing a good quality ToR with all
main intended users, and make sure consultants
understand it and are comfortable with it
Thank you
. Group
work
• Identifying potential challenges in promoting and
implementing Equity-focused evaluations
• Identify possible way to address the challenges
• What support would you need?