A Critical Analysis of
Current M&E Frameworks
Insights from AWID’s research on M&E frameworks
By Srilatha Batliwala, Scholar Associate, AWID
SBatliwala, AWID
First, let’s
remember that:
“When you work for women’s interests, it’s two
steps forward - and at least one step back. And
those steps back are… often evidence of your
effectiveness; they represent the threat you have
posed to the power structure, and its attempt to
push you back. Sadly, even our ‘success stories’
are sometimes nothing more than ways the
power structure is trying to accommodate and
contain the threat of more fundamental change
by making small concessions to us.”
Sheela Patel
SBatliwala, AWID
Assumptions in M&E
1. Everything should be measured
2. Everything can be measured – even
complex processes of social change
3. Measurement will enhance our ability
to accelerate, deepen, replicate
positive change, or improve
achievement of desired goals
4. Change is predictable – we know what
it will look like, where it will occur, and
how to assess it
SBatliwala, AWID
Assumptions in
M&E
5. The macro-political environment is stable,
democratic, upholds basic rights, and
protects change agents and change
processes – i.e., law and order, an impartial
judiciary and police, due process, rights of
association, civil liberties, an independent
media, etc. are inevitably present in every
context
6. The organizational environment is stable
and unchanging – staff don’t leave, core
operational costs are secure, and you
SBatliwala,
AWID
haven’t been shut down or
harassed
Why M&E?
In theory, to:
•Learn how change happens, make it
happen faster, better, etc.
•Analyze and sharpen our role in the
change process
•Empower our constituencies
•Practice accountability – to donors,
constituencies, peers, public, etc.
•To advance our advocacy agenda
SBatliwala, AWID
Why M&E?
In practice, to:
•Satisfy donor demands;
•Prove we’re doing what they funded
us to do
•Leverage more funding
•Protect ourselves - from backlash,
attacks, slander campaigns, etc.
SBatliwala, AWID
What isn’t working
•
•
Very few M&E frameworks actually
enable us to understand whether
change has happened, or gender
power altered – they measure
performance (program inputs and
outputs), not change
What they measure may not be
indicative of the change they’re
measuring at all – (e.g., GDP!)
SBatliwala, AWID
What isn’t working
•
•
Many current frameworks are very
linear, and look for simplistic causeeffect relationships (x intervention =
y effect = z change)
One such, the logical framework, is
often focused on performance - “[log
frames] focus on the expected
achievements laid out in the matrix –
rather than the work itself.”
,
SBatliwala, AWID
What isn’t working
•
•
Most frameworks do not provide for
tracking negative change, reversals,
backlash, unexpected events, etc., that
push back or shift the direction of the
change process. In women’s rights work,
this is vital, because the most effective
work seriously challenges patriarchal /
other social power structures, creating
negative reactions.
Reactions / backlash / negative change is
often evidence of positive impact!
SBatliwala, AWID
What isn’t working
•
•
Most current frameworks are
completely inappropriate for certain
kinds of social change organizations
and strategies: E.g., advocacy, training
/ capacity building, knowledge
production, challenging discourse, etc.
Consequently, these organizations are
forced to measure their processes,
outreach and outputs (number programs
held, number of participants, publications,
attendance at rallies, etc.), rather than
their impact.
SBatliwala, AWID
What isn’t working
•
•
Several false binaries and dichotomies
are embedded within or underlie many
M&E approaches – e.g., “quantitativequalitative”, “subjective-objective”,
“macro-micro”, “success-failure”, and so
forth.
These create problematic hierarchies
rather than approaches that can
integrate and transcend such dualities.
SBatliwala, AWID
What isn’t working
•
•
Disjuncture between change
measures and our time frames.
The changes we are trying to track
may not be visible within the time
frame in which we are seeking it
Most current M&E frameworks are
neither gendered nor feminist in
their principles or methodology.
SBatliwala, AWID
“It takes ten years to
build an organization,
twenty years to build a
movement, and thirty
years before you see
lasting impact.”
Ela Bhat
Measuring Social
Abstractions – the key
questions
1. What is it? Definition
2. Where is it? Location (geographic,
social, political, institutional, sectoral)
3. What are its boundaries? Spatial,
demographic, and conceptual
4. What does it look like? Characteristics
& situational analysis
5. What can be measured? Measurable
dimensions, sub-units, indicators
6. When do we measure? Baseline +
frequency of assessment
SBatliwala, AWID
Elements of
Feminist M&E
•
•
•
•
•
•
Feminist answers to the social abstraction
questions
Informed by / embedded in feminist
values and principles
Based on complexity
Use the best available tools (both
quantitative and qualitative)
Appropriate and differentiated time
frames
Designed for the level, nature of work
and strategies of each organization
SBatliwala, AWID
Principles of
Feminist M&E
1. The right of our constituency to inform, codesign and participate in the monitoring
and evaluation of change processes
2. Respect for the voice and perspective of all
key stakeholders
3. Prioritizing learning in our M&E goals
4. Positioning M&E as a political activity
5. Integrating political and social forces into
our analysis & frameworks
SBatliwala, AWID
Principles of
Feminist M&E
6. Avoiding attribution, assessing
contribution
7. Eschewing false binaries / dichotomies
8. Not using M&E for punitive purposes
9. Capturing, analyzing and addressing
negative changes, reversals
10. Willingness to abandon, revise, recast
our frameworks
SBatliwala, AWID
First steps in doing
it differently:
1. Problem definition & situational analysis
2. Enunciation of our M&E principles
3. Development of customized M&E
framework & indicators (see forthcoming
AWID guideline)
4. Establish a baseline
5. Track and map
• Our processes
• Our performance and
• External change
SBatliwala, AWID
Social Watch Gender
Equity Index
Education Gap
• literacy rate
• primary school enrolment
• secondary school enrolment
• tertiary education enrolment Empowerment Gap
% of women in technical positions
% of women in management and
Economic Gap
government positions
• economic activity rate
% of women in parliaments
• estimated earned income
% of women in ministerial posts
SBatliwala, AWID
SBatliwala, AWID