Monitoring, evaluation, and learning
(MEL) framework for the Millennium
Water Program, Kenya
May 8, 2012
MWP-K Learning Event
Background
 Collaboration between:
 Millennium Water Alliance
 MWP-Kenya partners (CARE, CRS, FH, Living Water,
Lifewater, World Vision)
 Center for Global Safe Water, Emory University
 Objectives:


Improve effectiveness and increase sustainability through
more data-driven decision making
Build foundation of learning and collaboration through MEL
to support future continuation of MWP-Kenya
Challenges Addressed by MEL
Framework


Monitoring usually limited to project
activities and outputs (e.g. # latrines built,
# trainings completed)
Final evaluation is only time seized to assess
achievement of outcomes (e.g. behavior
change)
Want to use monitoring to track progress
toward intended behavior change outcomes
to enable a change of course if needed
Challenges Addressed by MEL
Framework
 Minimal information sharing & reflection on
monitoring data or other lessons throughout
project life cycle.
Want to intentionally budget time and persons
responsible for guiding data sharing and
reflection to maximize lessons learned to
improve practice
Challenges Addressed by MEL
Framework
 Sustainability seldom evaluated after project
life cycle due to close of project, budget
limitations, etc.
Want to incorporate M&E indicators into tools
used during project life cycle to assess
likelihood of future sustainability (are factors
in place to improve longevity of benefits after
project is finished?)
Components of MEL for MWP Kenya
1. Key program indicators
2. Evaluation: baseline and follow-up
3. Monitoring: systematic, regular data
collection


Includes process and tools to guide “Data for decisionmaking”: Sharing, feedback, and action;
Monitoring tools to assess progress towards outcomes
4. Focused topical learning
 Includes partners’ leadership in special learning topics;
 Emory support for some sub-studies (e.g. sanitation
assessment)
1. Key Program Indicators
 Emory assisted partners in developing logical
framework in May 2010
 Indicators include:



Outputs (activities completed by partners)
Outcomes = central focus (changes that beneficiaries adopt
in response to program)
Includes indicators that suggest future sustainability

E.g. School has specific WASH budget; school management
committee is involved in WASH program
2. Evaluation
 Emory facilitated baseline assessment planning and
implementation:

Led development of evaluation tools and protocols


Household, school, and health facility surveys
Analyzed data
 Each organization collected data prior to
implementation
3. Monitoring
 Output-level indicators
 Responsibility of each partner
 Tracked through internal monitoring system
 Outcome-level indicators
 Measured throughout course of project


To engage implementers to critically reflect on and
address programmatic challenges in real-time
Monitoring activities conducted quarterly or once per
school term
Data for Decision-Making
 Sharing of experiences and lessons learned
between partners
 Data Reflection Tool (DaRT)
 Summarizes findings from each monitoring and
learning activity
Project expectations
 Progress toward achieving indicators
 Plans to address lack of progress
 Potential threats to sustainability


Shared and discussed within partnership on a quarterly
basis before Program Management Group meeting
Data for Decision-Making Quarterly Work Flow
= Responsibility of each organization
= Responsibility of Coordinator
= Responsibility of all partners together
Decide:
Complete
monitoring
activities
quarterly
Meeting:
Internal
Learning/
Reflection
Summarize
findings into
Data
Reflection
Tool (DART)
Send
MWP
Coordinator
reviews
DARTs
1.
Discussion of:
1.What’s going
well/ not as
well as
expected
2.Possible
course changes
3.Additional
info needs
2.
3.
Flag issues
for followup
Use to
guide
agenda for
Pre-PMG
meeting
Distribute
copies at
pre-PMG
meeting
Meeting:
Pre-PMG
Learning/
Reflection
Discussion of:
1. Highlights
from DARTs
2. Lessons
learned,
best
practices
3. Group
feedback
4. Issues to
raise at
PMG
Meeting:
PMG
Needed
course
changes
Additional
learning
needs
Discussion of:
1. Issues
flagged at
Pre-PMG
meeting
2. Lessons
learned &
best
practices
Summary of Monitoring Tools
Tool
Objective
Rapid household observations
Observe and inquire about WASH practices in households in the
intervention area to track changes toward achievement of intended
outcomes and understand the trends in behavior change.
Water point observations and
management committee key
informants interview
Observe repair or maintenance needs; observe practice of integrated
water resources management (IWRM); understand patterns of use for
the water point, financial management practices, maintenance
practices, and management challenges.
Water point user interviews
Determine functionality, use patterns, and perceptions of water point
and management committee from community perspective. Provides a
double check on some information gathered in the management
committee interview.
School monitoring visit
Determine whether and how the school enacts the intervention on a
day-to-day basis and assess threats to sustainability that might need to
be addressed.
School management committee
key informant interview
Understand the degree to which the SMC is involved in supporting
school WASH activities, and encourage their support by inviting them
to join you in your monitoring visit observations.
Pupil interviews
Gain pupil perspective on WASH provision and education at school
and preferences. Provides a double check on some information
gathered in the school assessment.
4. Focused Topical Learning
 Partners expressed interest in specific topics for
further investigation
 As learning leader, each org. should organize one
meeting with partners to discuss lessons on topic
Proposed
Learning Leader
CARE
Topic
Improving pupil handwashing behavior change in schools
CRS
Improving utilization of household latrines
FH
Improving community handwashing behavior in ASAL, nomadic areas
LWI
Improving financial management of water sources
WVI
Improving water supply sustainability
CARE
Improving pupil handwashing behavior change in schools
Summary
 Thoughtful, timely collection of relevant data
 Data used to drive decision-making:
 Guide targeting, technology choice, and program approaches to
improve effectiveness and sustainability
 Feed back findings into continued program activities in a timely
and meaningful way
 Engage donors, beneficiaries, and government stakeholders in
the water, sanitation, and hygiene (WASH) sector
 Embrace benefits of working in partnership
 Collective learning and reflection on best practices & lessons
learned