Monitoring & Evaluation Plans and Understanding data & information flow Guides for Demonstrating Program Progress and Success Learning Objectives Describe basic elements of M&E Plans Understand basic function of Plan components Identify opportunities for data demand and use Apply the Information Use Map Session Overview Definition and function of M&E Plan Describe components of a Plan Basic rules to guide M&E Plan development Introduce Tool: Information Use Map Introduction to workshop group work: Developing components of an M&E plan M&E Plans – Definition1 Document describing all M&E activities in a program Program objectives, interventions developed to achieve them, & procedures to be implemented to determine whether or not objectives are met Expected results of the program and how they relate to goals and objectives Data needed, how it will be collected & analyzed Information use, including resources needed to do so How the program will be accountable to stakeholders 1MEASURE Evaluation website, https://www.cpc.unc.edu/measure/training/mentor M&E Plans - Function Guides implementation of program M&E Enhances coordination, standardization States how program will measure achievements Accountability Documents stakeholder consensus Transparency & responsibility Helps achieve program results Ensures good use of data Preserves institutional memory A living document, adjusted for program modification M&E Plan Components Introduction Program Description Goals and objectives M&E Frameworks Conceptual, Logic, Results Indicators Presented in a both a Matrix & Indicator Reference Sheets Data sources, collection & reporting systems Plans for data use & dissemination Information Use Mapping Tool as an option M&E Plan Components Capacity needs for Plan implementation Funding, TA, staff, equipment (computers, GPS) Analysis of constraints & potential solutions Plans for demonstrating program impact Mechanism for Plan updates M&E Plan Components - Introduction Program Context National, community-based Nature of problem within the target population Purpose of the Plan Description of development process Stakeholders involved Consensus process M&E Plan Components – Program Description Problem Statement What is the nature of the HIV-related issue being addressed? Goal and Objectives What is the ultimate outcome of the program (goal) What are the shorter-term aims (objectives) Program Description Intervention(s), geographic scope, target population, duration M&E Plan Components – M&E Frameworks Conceptual: shows how program fits into causal pathway to desired health outcome Logic: shows program components that will contribute to the long and short-term outcomes Programmatic-specific frameworks Results for USAID/PEPFAR M&E Plan Components – Indicators Selection based on Logic models Donor requirements Presented in 2 ways Indicator Matrix (pp. 40-45, VN National Plan) Table presenting indicators including information on data source, frequency, who is responsible, etc. Indicator Reference Sheets (pp. 46-107) Detailed sheet describing each indicator, how to measure it, underlying assumptions & interpretation considerations M&E Plan Components – Data Sources used for inputs to indicators Existing/routine data Planned studies Special systems/studies for program Collection tools for program Patient records, supplies, clinic records for use of services Reporting systems Upward and downward flow of data M&E Plan Components – Data Use Specify plans for data use Identify Databases for storage Who the users are Dissemination methods including Reports, media, speaking events Apply Information Use Map tool to understand data flow and enhance use Understanding Data Flow Helps us better understand our role in health information system and importance of collecting data Identify opportunities for improving data collection, analysis, increase availability, and ensure its use Information Flow Feedback Program Clinical histories, service statistics Compiled data Service Delivery Point Reports Managers, Government , Donors Analysts, evaluators Higher levels: district, province, national Reasons to Assess Information Flow Local data not used locally Higher-level information does not return back to local level Local data not assessed in broad context Reports may not reflect what is being collected & is needed (e.g., sex differentiation) Little incentive to produce high-quality data 16 INFORMATION USE MAP Information Use Mapping Purpose Describe existing flow of health information to identify opportunities for improving its use Description Identifies gaps and opportunities for using information Identifies opportunities for additional feedback mechanisms Identifies points where analysis & data could support programmatic decision making Information Use Map: Swaziland National HIV/AIDS Program May 2005 NGOs Client data collected in registers Regional Client data collected in electronic patient record systems SNAP/MOHSW NERCHA Compilation Storage Analysis Reporting Client data collected in electronic patient record systems Government Facilities Private Clinics Data Collection Client data stored in electronic patient record systems Staff compiles into monthly summary sheets Regional facilities’ monthly summary sheets compiled Data entered into Access at MOH Health Statistics or NERCHA Reporting to WHO or GFATM Use Information Use Map: Swaziland National HIV/AIDS Program May 2005 NGOs Government Facilities Client data collected in registers Regional Compilation Storage Analysis Reporting Client data collected in electronic patient record systems Client data collected in electronic patient record systems SNAP/MOHSW NERCHA Feedback Loops Private Clinics Data Collection Use Opportunities for Use Client data stored in electronic patient record systems Opportunities for Use Staff compiles into monthly summary sheets Opportunities for Use Regional facilities’ monthly summary sheets compiled Opportunities for Use Data entered into Access at MOH Health Statistics or NERCHA Reporting to WHO or GFATM Opportunities for Use Key Messages Actual flow of data and information can reveal barriers to improving data quality and use Information Use Map can highlight intervention points 21 M&E Plan Components Capacity needs for plan implementation Identify resources needed to implement plan Funding, technical capacity, equipment etc. Analysis of constraints Be realistic; may be connected to above Plans for demonstrating program impact Evaluation probably not covered by monitoring indicators Mechanism for Plan updates Depending on length of program - annual, semi-annual An M&E Plan should be Practical Accessible to intended users Feasible, realistic, & diplomatic Legal & ethical Accurate Reveal technically correct information M&E Plan Development Advocate for the need for M&E Assess program information needs Assess existing system capabilities to address these needs Achieve consensus and commitment among stakeholders Indicators & reporting structure Prepare document for final approval M&E plan should be written during the initial stages of program development Workshop Group Project Developing components of an M&E Plan Workshop Group Project: Developing Components of an M&E Plan Goal Put workshop learning to practical use with real life case studies All case studies for projects provided by you In total, 5-6 groups Minimum of 3, max of 4 people Sign up for top 2 choices after this session Workshop Group Project: Assignment Total of 5-6 groups Look to author for missing information, or create plausible conditions As part of workshop sessions, all groups Logic model Some indicator selection Some tool utilization Remaining work is your choice: Focus on some M&E Plan components & use tools presented Develop real components – detailed Nature of program description may dictate some of what needs to be done Information Use Map Activity for Group Work: Complete the Information Use Map for your organization as data flows now Review the map and discuss among your group how the flow of information could be improved: How else could data be analyzed? Are there opportunities for feedback mechanisms? Is data being used by all stakeholders? Note potential interventions based on your discussion. Make a 2nd version of the map to illustrate an improved flow of information 28 Small Group Activity: Report Back 1. One group member remain at the table to present the map to visitors 2. Each table moves to the right to visit the table next to them. 3. The table presenter spends 5 minutes explaining how they improved information flow in their map and how this would facilitate use of data. 4. Repeat steps 2-3 5. Everyone returns to original tables 6. Consider improving your map based on what you learned from your neighbors. MEASURE Evaluation is a MEASURE project funded by the U.S. Agency for International Development and implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill in partnership with Futures Group International, ICF Macro, John Snow, Inc., Management Sciences for Health, and Tulane University. Views expressed in this presentation do not necessarily reflect the views of USAID or the U.S. Government. MEASURE Evaluation is the USAID Global Health Bureau's primary vehicle for supporting improvements in monitoring and evaluation in population, health and nutrition worldwide.