Wellness and Agriculture for Life Advancement (WALA)
A GUIDE TO WALA M&E SYSTEM
(Abridged Version)
Prepared By
Monitoring, Evaluation and Knowledge Management Unit
Consortium Administration and Technical Capacity Hub (CATCH)
JANUARY, 2012
TABLE OF CONTENTS
TABLE OF CONTENTS ................................................................................................................2
LIST OF ABBREVIATIONS .........................................................................................................3
LIST OF FIGURES ........................................................................................................................4
1.0. INTRODUCTION ...............................................................................................................5
1.1. The WALA Program ........................................................................................................5
2.0. WALA M&E SYSTEM .......................................................................................................5
2.1. WALA SMILER IN BRIEF .............................................................................................6
2.2. GETTING ORGANIZED ................................................................................................7
2.2.1. WALA M&E Purpose Statement..............................................................................7
2.2.2. CRS M&E standards .................................................................................................7
2.2.3. Stakeholder Analysis ................................................................................................8
2.2.4. M&E Technical Working Group ..............................................................................8
2.3. THE SET UP ....................................................................................................................8
2.3.1. Results Framework ...................................................................................................8
2.3.2. Indicator and Performance Tracking Table (IPTT) ................................................10
2.3.3. Indicator Tracking Table (ITT) ...............................................................................10
2.3.4. Performance Management Plan ..............................................................................10
2.3.5. Detailed Implementation Plan.................................................................................11
2.4. REPORTING FORMS AND REPORTING RELATIONSHIPS ..................................11
2.4.1. Data Flow Maps ......................................................................................................11
2.4.2. Communication and Reporting Maps .....................................................................13
2.4.3. Data Gathering Forms .............................................................................................15
2.4.4. Reports (monthly, quarterly, annual) ......................................................................15
2.4.5. Consortium Management Information System (C-MIS) ........................................15
2.4.6. Evaluations (Baseline, Midterm, Final) ..................................................................18
2.4.7. Knowledge management and learning ....................................................................18
2.4.8. Data Quality Assessments (DQAs) .........................................................................18
2.4.9. Quality Improvement Verification Checklist (QIVC) ............................................18
3.0. FURTHER READING ......................................................................................................19
M&E System Abridged Guide: January 2012
Page 2
LIST OF ABBREVIATIONS
ACDI/VOCA Agricultural Cooperative Development International/Volunteers in Overseas
Cooperative Assistance
ARR
Annual Results Report
CATCH
Consortium Administration and Technical Capacity Hub
C-MIS
Consortium Management Information System
CRS
Catholic Relief Services
DEC
District Executive Committee
DIP
Detailed Implementation Plan
DQA
Data Quality Assessment
EI
Emmanuel International
FFP
Food for Peace
I-LIFE
Increased Livelihoods through Increased Food Security Program
IPTT
Indicator Performance Tracking Table
IR
Intermediate Result
ITT
Indicator Tracking Table
M&E
Monitoring and Evaluation
MCHN
Maternal and Child Health and Nutrition
NGOs
Non-Governmental Organizations
PCI
Project Concern International
PMP
Performance Management Plan
PVO
Private Voluntary Organizations
QIVC
Quality Improvement Verification Checklist
RF
Results Framework
SAPQ
Standardized Annual Performance Questionnaires
SMILER
Simple Measurement of Indicators and Learning for Evidence Based Reporting
SO
Strategic Objective
SSI
Small Scale Irrigation
TA
Traditional Authority
TLC
Total Land Care
ToRs
Terms of Reference
TWG
Technical Working Group
USAID
United States Agency for International Development
VS&L
Village Savings and Loans
WALA
Wellness and Agriculture for Life Advancement
WVI
World Vision International
M&E System Abridged Guide: January 2012
Page 3
LIST OF FIGURES
Figure 1: Diagrammatic presentation of SMILER set up ...............................................................7
Figure 2: WALA results framework ...............................................................................................9
Figure 3: Results framework for SO2 .............................................................................................9
Figure 4: WALA indicator performance tracking table ................................................................10
Figure 5: WALA performance monitoring plan (PMP) ...............................................................11
Figure 6: Data flow map for SO1 .................................................................................................12
Figure 7: VS&L data flow map at PVO level ...............................................................................13
Figure 8: Communication and reporting flow map for CATCH ..................................................14
Figure 9: Example of communication and reporting map-PCI.....................................................14
Figure 10: C-MIS log in screen ....................................................................................................16
Figure 11: WALA/IMPACT log in window .................................................................................16
Figure 12: WALA switchboard in CMIS......................................................................................17
Figure 13: Services and functions under WALA in the CMIS .....................................................17
M&E System Abridged Guide: January 2012
Page 4
1.0.
INTRODUCTION
1.1. The WALA Program
Wellness and Agriculture for Life Advancement (WALA) is a Title II five year (2009-2014)
food security program, funded by Food for Peace (FFP) of USAID. The program targets the
eight most food insecure districts of southern Malawi namely Nsanje, Chikwawa, Thyolo,
Mulanje, Zomba, Machinga, Chiradzulu and Balaka. WALA is implemented by a consortium of
eight Private Volunteer Organizations (PVOs) alongside Catholic Relief Services/Malawi (CRS)
as the grant holder. CRS/Malawi through the Consortium Administration and Technical
Capacity Hub (CATCH) leads the management and implementation of the program. The eight
PVOs are ACDI/VOCA, Africare, Chikwawa Diocese, Emmanuel International (EI), Project
Concern International (PCI), Save the Children, Total Land Care (TLC) and World Vision
International (WVI).
The goal of WALA is to improve the food security of 214, 974 chronically food insecure
households in 39 Traditional Authorities (TAs) in 8 districts in Southern Malawi by 2014.
This will be achieved through 3 strategic objectives (SOs) namely
 SO1: 170,724 vulnerable households have improved maternal, child health and nutrition
status.
 SO2: 147,500 smallholder farming households have improved livelihood status.
 SO3: 273 targeted communities have improved capacity to withstand shocks and stresses.
Alongside these SOs, the program also addresses cross-cutting issues namely, HIV and AIDs,
gender, governance and environmental management.
In order to track progress and provide benchmarks for program performance evaluation, WALA
program has a comprehensive M&E system that outlines the programs monitoring tools,
resources, procedures and documents necessary for providing programmatic direction towards
achievement of its objectives. The system has developed appropriate data collection and
compilation tools with attention to data quality and how to achieve the set quality standards
including sound data analysis and reporting procedures. The M&E system is set up is such a way
that it serves the program managers with administrative reporting needs as well as helping
stakeholders to follow implementation progress, unearth bottlenecks and provide diagnosis of
the constraints and how to resolve them. At the same time, the system ensures accountability of
resource utilization while enhancing learning and knowledge management. The system also
ensures uniformity in approaches of program implementation among all PVOs in the 8 districts.
2.0.
WALA M&E SYSTEM
WALA M&E system adopted the CRS wide monitoring and evaluation standards while meeting
the donor requirements and allowing the PVOs to incorporate some reporting requirements that
are specific to their organizations. This way, the system is acceptable to all partners and
stakeholders.
M&E System Abridged Guide: January 2012
Page 5
WALA adopted the Simple Measurement of Indicators and Learning for Evidence-based
Reporting (SMILER) approach in developing its M&E System. SMILER is a comprehensive
approach to developing a project monitoring and evaluation system that supports learning and
decision-making based on evidence.
2.1. WALA SMILER IN BRIEF
WALA SMILER consists of a number of components that when put together, form a step by
step and systematic guide that constitute the entire M&E system. The components provide
guidelines, instructions documents that have been put together with wider consultation with the
program stakeholders and using experience and learning from previous but similar programs in
order to have a comprehensive system. SMILER has three main components as follows;
1. Getting Organized consists of documents that guided formulation of the M&E system
like the CRS wide M&E standards, CRS M&E standards support tool, Term of
References (ToRs) for M&E Technical Working Group (TWG), purpose of M&E and
stakeholders’ analysis. It also consists of the SMILER’s table of contents which
highlights all documents and necessary processes that constitute the M&E system.
2. Setting up consists of the Results Framework (RF), indicator performance tracking table
(IPTT), performance management plan (PMP), detailed implementation plan (DIP) and
trigger indicators and threshold table.
3. Designing Forms and Reports comprises of processes involved in designing different
standard reporting forms and procedures for enhancing efficient data flows report and
reporting relationships (data flow maps and communication maps) and data gathering
forms, data management and analysis, report (monthly, quarterly, annual) and
Evaluations (baseline, mid-term and final). This is where data quality assessments
(DQAs) and systems to verify quality of service delivery are integral part of.
Figure 1 is a diagrammatic presentation of the SMILER approach as it was applied in
developing WALA M&E system.
M&E System Abridged Guide: January 2012
Page 6
Figure 1: Diagrammatic presentation of SMILER set up
2.2. GETTING ORGANIZED
2.2.1. WALA M&E Purpose Statement
WALA M&E system is designed and implemented following CRS M&E Standards and is
compatible with program’s data and data quality requirements and provides other technical areas
with tools and guidance on how best they can collect and manage data that is useful for program
improvement and reporting. The WALA M&E has the following purposes;
1. Strengthen M&E capacity of consortium members
2. Consortium-wide uniform M&E system with minimum flexibility to allow for other
partners’ reporting requirements
3. Meet stakeholder (within WALA and beyond) information needs
4. Pilot and learn from M&E innovations
5. Monitor service delivery and quality
6. Establish knowledge management system and share lessons learned
7. Data are used by consortium partner staff and volunteers for decision-making and action
8. Optimize linkages with other program
2.2.2. CRS M&E standards
CRS’ M&E Standards are critically formulated to enhance program or project quality. They are
critically formulated to provide quality standards that need to be adhered to in order to improve
effectiveness service quality and delivery at each stage of program formulation, implementation
and evaluation. The Standards clearly define the key elements of project Design, Monitoring
M&E System Abridged Guide: January 2012
Page 7
and Evaluation (M&E) and an organizational environment in which M&E can improve
program quality and positively impact the people it serves. These standards reflect key
characteristics of high quality programs and agency culture that promote better learning and
strengthen accountability to stakeholders. The M&E systems inform decisions at the field level
regarding progress and success of projects and programs. These standards have been formulated
through learning and experience that CRS has had over many years of successful
implementation of complex programs in multicultural environments which have had immerse
positive impact on people’s lives.
2.2.3. Stakeholder Analysis
WALA stakeholder’s analysis was done. These are stakeholders who have information needs
from the WALA program. WALA M&E system sets out a framework for communicating with,
and reporting to these stakeholders and ensuring that they have the information they need to
review progress and support ongoing efforts. These stakeholders range from government
ministries and departments, other NGOs, faith based organizations, district level structures and
community based organizations among others.
2.2.4. M&E Technical Working Group
The WALA M&E System has an M&E technical working group (TWG) with well defined terms
ToRs. Among others the, M&E TWG aims at reviewing PVO progress against targets,
developing data collection tools, periodic review of data collection tools and the M&E system
and providing relevant feedback to other sector TWGs within the consortium. The M&E TWG
comprises the Head of M&E and Knowledge Management, CATCH based M&E staff, one
M&E Officer from each PVO. At any given seating of the TWG, other technical staff may be
co-opted into the meeting to if need arises. The M&E TWG meets regularly (quarterly) to
discuss M&E and knowledge management issues with reference to its ToRs.
2.3.
THE SET UP
2.3.1. Results Framework
The WALA Results Framework (RF) is a logic model which sets out a logical sequence of activities
and outputs that will lead to short, medium and long term outcomes and objectives. It states how the
program’s activities will cause the desired outcomes. It is drawn based on a solid and evidence based
theory of change which underlies the choice of the activities
The overall WALA RF is diagrammatically presented in Figure 2;
M&E System Abridged Guide: January 2012
Page 8
Figure 2: WALA results framework
Each intermediate result (IR) under the three SOs including the cross cutting issues has its
activities that if implemented will lead to the realization of that specific IR. For example, the
activities under SO2’s intermediate results are presented as follows in the RF;
Figure 3: Results framework for SO2
M&E System Abridged Guide: January 2012
Page 9
2.3.2. Indicator and Performance Tracking Table (IPTT)
WALA program has an indicator performance tracking table (IPTT) which comprises performance
indicators at impact, outcome and output levels. These indicators are linked to the program’s
objectives. The direction of desired change (positive or negative) is indicated for each indicator.
Estimated baseline values for each indicator are added in the IPTT. Depending on the direction of
desired change for each indicator, a target is set out for each implementation year. Figure 4 is a
section of the IPTT.
Figure 4: WALA indicator performance tracking table
The indicators in the IPPT have been carefully formulated in order to conform to national and
international definitions and universal measurement standards. Changes to the IPTT during the
implementation period can only be effected after thorough consultations have been made with
various stakeholders and concurrence has been sought from the donor of the program.
2.3.3. Indicator Tracking Table (ITT)
While the IPTT condenses most of the indicator into composite formulations, the indicator
tracking table (ITT) has been designed to capture minor and more detailed process and input
indicators. Depending on the nature of the indicators, some are exactly captured in the ITT are as
they are in the IPTT. The ITT is accompanied by and definition of each and every indicator in
order to enhance uniformity in data capturing across the consortium.
2.3.4. Performance Management Plan
The Performance Management Plan (PMP) contains definitions for each indicator listed in the IPTT
including the data collection methodology and sources, methods/approaches and instruments for data
collection, frequency and responsible personnel for data collection and how the data is going to be
analyzed and used by stakeholders. The PMP sets out roles and responsibilities on collection and
who needs the data. Figure 5 is a formulation and and example of one of the indicators in the IPTT.
M&E System Abridged Guide: January 2012
Page 10
Figure 5: WALA performance monitoring plan (PMP)
2.3.5. Detailed Implementation Plan
At the beginning of each program implementation year, the M&E team prepares a detailed
implementation plan (DIP) which describes activities that correspond to the resources requested in
the program proposal in order to achieve the proposed objectives and targets. The DIP describes
each activity to be carried out during an implementation year including the general location of the
activity; the intermediate result(s) (IR) that the activity supports and the timing of the activity by
month. This is done in order to estimate M&E activity requirement during the year.
2.4.
REPORTING FORMS AND REPORTING RELATIONSHIPS
This section briefly explains how data flows from one level to the other beginning from the field
level program staff and village or community based volunteers. It also highlights the reporting
relationships among different levels in the hierarchy of the program. These standard reporting
and data flow maps enhance easy communication and define responsibilities among program
staff and stakeholders.
2.4.1. Data Flow Maps
Each SO has a set of data collection tools and reporting forms at various levels. In order to
facilitate easy flow of information from one level to another and provided evidence based
reporting, each SO has a data flow map that outlines how information and data is collected and
flow and who is responsible for collecting the data at each level. The map also provides the steps
and tools (forms) through which the data is collected and/or consolidated into one major report
for the program. Figure 6 is a data flow map for SO1. There are similar data flow maps for other
SOs.
M&E System Abridged Guide: January 2012
Page 11
Figure 6: Data flow map for SO1
SO2 comprises of several components namely, agriculture, irrigation and village savings and
loans. There are separate data flow maps for each component. Figure 7 is a data flow map for
VS&L.
M&E System Abridged Guide: January 2012
Page 12
Figure 7: VS&L data flow map at PVO level
2.4.2. Communication and Reporting Maps
In order to provide for a structured and disciplined communication and reporting system, the
WALA M&E system has defined communication and reporting map at CATCH and PVO level.
The map provides focal persons at each reporting level so that data and information users and
stakeholder are guided on where and from whom they can get relevant data. The communication
and reporting map also reminds the responsible staff at each level their responsibility and
deadlines for data collection and reporting. Figure 8 is a CATCH communication and reporting
map.
M&E System Abridged Guide: January 2012
Page 13
Figure 8: Communication and reporting flow map for CATCH
Each PVO has a similar communication and reporting map. For instance, Figure 9 is a
communication and reporting map for PCI.
Figure 9: Example of communication and reporting map-PCI
M&E System Abridged Guide: January 2012
Page 14
2.4.3. Data Gathering Forms
Within each SO and within each IR, there are data gathering forms designed to collect adequate
data for assessing whether there is progress towards achieving the specific results. These data
collection forms are distributed to relevant field staff. Once completed, they are returned to
appropriate sector coordinators and forwarded to M&E Unit within each PVO for capturing into
the Consortium Management Information System (C-MIS). However, not all data collection
forms are entered into the MIS. Some are filed as hard copies and kept within their sector
offices.
2.4.4. Reports (monthly, quarterly, annual)
There are also specific standard reporting forms for monthly, quarterly, annual reports at
different levels. These are standardized across the consortium in order to ensure uniformity in
reporting of key performance areas and allow easy consolidation.
2.4.5. Consortium Management Information System (C-MIS)
The M&E system has a comprehensive MS Access based Consortium Management Information
System (C-MIS). The C-MIS has been carefully designed to provide a consolidated data
management platform where routine data from all areas of the program is stored and basic
reports generated. Within the C-MIS, program staff and stakeholders can generate key reports
highlighting some of the program’s achievements at any point in time. The C-MIS is an
integration of WALA and IMPACT program both of which are running concurrently and share
many impact areas and beneficiaries. The system is also designed to show synergy between
WALA and IMPACT programs and also among components within WALA. Therefore the CMIS captures data from both WALA and IMPACT programs.
The C-MIS comprises of the data collection tools section where data capturing takes place, the
report generation section where the operator can generate various reports depending on their
needs and the administrative tools section which is restricted to the MIS manager for
modification of the system. The following section provides a brief overview of getting into the
MIS.
2.4.5.1. CMIS Log in and Navigation
On any computer where the system is installed, it is advisable that there is a CMIS Shortcut on
the desktop. When you want to log in into the CMIS, double click on the shortcut located on the
desktop of the computer. The computer will present a log in screen as in Figure 10.
M&E System Abridged Guide: January 2012
Page 15
Figure 10: C-MIS log in screen
The operator will be required click on “Login Partner Name” combo box and select the partner
name for which they want to log in as. Upon entering an appropriate password and clicking
“Login” the operator will be prompted into a home page as presented in Figure 11. This page
will allow the operator to choose which program (IMPACT or WALA) they would like to use
the MIS for.
Figure 11: WALA/IMPACT log in window
The operator can then click on the logo associated with the program of their interest. For
example if they want to use the MIS for WALA, they will be required to click on the WALA
M&E System Abridged Guide: January 2012
Page 16
logo. To exit the MIS, click on the “EXIT” command button and the system will exit. If the
operator chooses WALA, they will be presented with window that looks like Figure 12, the
WALA switchboard.
Figure 12: WALA switchboard in CMIS
The switchboard will take you further into services and functions available under the WALA
program section in the MIS. Click on the tabs (i.e. Data collection tools, report generation and
administrative tools) depending on the functions or services that you want to perform with the
MIS. The data collection tools section is divided into different WALA program areas and
depending on the data that you want to enter, you will be required to click on the appropriate
form. Similarly, if you want to generate reports or perform administrative functions, you will be
required to click on the appropriate tab and you will be prompted to navigation windows as in
Figure 13. You can choose the appropriate WALA program area and select different reports as
required.
Figure 13: Services and functions under WALA in the CMIS
M&E System Abridged Guide: January 2012
Page 17
Beside predefined reports in the CMIS, if the operator has basic MS Access skills, they can
generate various reports depending on their needs.
2.4.6. Evaluations (Baseline, Midterm, Final)
WALA M&E systems complies with reporting standards for baseline, midterm and final
evaluations as prescribed by FFP. For example, WALA provides annual results reports (ARR) in
a standardized format and using the standardized annual performance questionnaire (SAPQ).
Similarly, WALA intends to apply standard procedures and practices as provided for by FFP
when conducting its mid-term and final evaluation.
2.4.7. Knowledge management and learning
With increasing realization of the importance of knowledge management and sharing as a key
element in success of programs, WALA has taken a deliberate step in incorporating knowledge
management into its M&E system. Knowledge management was never considered as an
important aspect of program and projects before. However, knowledge is now a treasured
resource and an important factor contributing to the success and effective performance of any
program or organization.
WALA’s knowledge management provides a platform for learning and sharing experiences
while at the same time prevents repetition of mistakes and loss of valuable resources. Thus the
success and magnitude of impact of WALA’s development initiatives depend on the constant
renewal of the program’s knowledge assets to match the ever changing global environment and
community needs. Thus WALA has a knowledge management strategy which is envisioned to
guide and strengthen the process of capturing, documenting, sharing and applying knowledge in
the program. This is expected to help avoid repeating past mistakes; highlight good practices;
make WALA’s work more relevant, effective and accessible; compare experiences and draw out
common issues and challenges; improve the documentation of lesson-learned; and help build
strong networks among people.
2.4.8. Data Quality Assessments (DQAs)
Data Quality Assessment (DQA) is an integral part of WALA M&E system. It is done to
improve quality of data to enhance effective decision making at program level. The assessment
process facilitates learning and improves on M&E systems that CATCH and PVOs have in place
for collecting, reporting and using quality data. While CATCH M&E team conduct DQAs in
PVOs, the PVO M&E Officers and every staff who are involved in data collection, compilation,
reporting and use takes responsibility in conducting DQA. However, it is the PVO M&E
Officers’ primary responsibility and who is accountable for DQA at PVO level. In order to be
systematic there is a document which clearly stipulates how and who should conduct DQA.
Similarly, at CATCH level, there is a clear agenda for conducting DQA and a standard reporting
form for the same.
2.4.9. Quality Improvement Verification Checklist (QIVC)
Another critical component of WALA M&E system is the Quality Improvement Verification
M&E System Abridged Guide: January 2012
Page 18
Checklist (QIVC). This is a system that is in place to check whether services in each WALA
sector or component are delivered to program beneficiaries as per the preset quality standards. It
is expected that if service delivery is up to the prescribed standards, the program is more likely
to have the intended project outcomes and/or impacts efficiently and effectively. This system
periodically (six-monthly) monitors if program services in key WALA program areas are
delivered in the program communities. Separate checklists with separate instruction sheets to
monitor service delivery and their quality standards of the following WALA program areas have
already been developed:
1. Maternal and Child Health and Nutrition (MCHN)
2. Agriculture
3. Small Scale Irrigation (SSI)
4. Village Savings and Loans
5. Agribusiness
3.0.
FURTHER READING
For detailed explanation and understanding of the whole WALA M&E system the following
reading materials are recommended:
1. WALA M&E operating manual
2. WALA program proposal
3. Sector specific QIVCs
4. DQA guidelines
5. CMIS User Guide (under development)
M&E System Abridged Guide: January 2012
Page 19