Towards a Participatory Monitoring & Evaluation System for the Eastern Cape Province ECSECC Action Plan Introduction Background & context Key lessons learnt Key objectives Impetus to the ECM&E System Key projects for development & funding Financing Key Stakeholders Constraints & solutions Action plan monitoring mechanisms Key resources Conclusion Eastern Cape M&E System 1 Do we need a Composite M&E? 3 What are the broad indicators? 2 What to Monitor & evaluate? October 2003 Herbert@ecsecc.org 5 What are the indicators & targets? 4 What is the Baseline? 7 Reporting 6 What M&E system should we have? 9 Information feedback 8 Evaluations 10 Sustaining M&E Background & context • ECSECC’s role & responsibilities • Deficiencies in current M&E systems • Demand-supply equation • Coordination structuresECMERG, TTT Project Cycle Phases Evaluation Appraisal Monitoring Planning Implementation Design M&E and Project Cycle [IFAD Guide] Identification, preparation & preappraisal Perform baseline studies, analyze sectoral data Identify risk and enabling factors Preparation/ Appraisal Identify risk, input, output, outcome, & Impact indicators Implementation/ Supervision Use indicators to monitor implementation Supervision/ Completion Evaluate output and outcome indicators Completion Onwards Evaluate outcome and impact indicators of project success Key lessons learnt • Structured involvement of relevant stakeholders at different levels of programme cycle. • Stakeholder analysis and participation • PLA principles, methodologies and tools: e.g. RAAKS, PRA, LFA- log frame, rich pictures, visioning, problem /objective tree, role plays, matrix analysis. • Use of log frame and other participatory tools in situation analysis • Use of decision matrix /matrix rankings • Delineating process issues from necessary supporting conditions. Key lessons learnt 2 Experiential learning cycle Joint planning and shared learning, public reflection and coordinated action. Cyclical and spiral nature of action learning. Organizational learning: barriers to learning & stimulating organization/social learning Processes and sequenced learning events and their relationship Key Objectives Efficient cluster programs tracking; Contribute to the attainment of quantified objectives of the PGDP; Improve programme/project management; Improve planning and projections; Improve information gathering, management, analysis and dissemination; Key Objectives 2 Increase efficiency and capability of M&E staff; Increase participatory and consensual development process; Contribute to the development of accurate, relevant & easily retrievable streamlined data; Increase coordination and efficiency; and Reduce duplication in data collection and management. Impetus to the ECM&E System Most stakeholders are excluded not only from planning but from reflective action as well; Baseline data is often not generated; Ill-defined indicators and objectives; There are budget over/ under-runs by departments with no proper explanation; Strategic planning cycle is not in unison with other planning cycles; Funds and programmes are not properly accounted for; Impetus to the ECM&E System 2 Impact of most programmes not known; There are no design and implementation guidelines; Often M&E function is not integrated in project management; There is lack of commitment and incentives; Irrelevant and poor quality information is generated; Limited staff capacity; There is no active engagement with generated data. Key projects for development & funding Stakeholder analysis and participation M&E information needs assessment in provincial departments & stakeholders Baseline studies for departmental programs Planning and indicator generation [strategic – implementation plans] IT hardware and software engineering [incl. database development] M&E Course design and development [phase one] Capacity development Publications and communication [incl. reflective learning] M&E Pillars (SMART) Objectives Data Communication & Reporting Capacity Development & Institutional arrangements Logical Indicators M&E System Data Collection & Management Responsibilities Financing ACTIVITY 1. Sta keh older a n a lys is & pa rticipa tion 2. M&E in form a tion n eeds a s s es s m en ts in provin cia l *depa rtm en ts 3. Ba s elin e s tu dies for depa rtm en ta l progra m m es 4. Pla n n in g a n d in dica tor gen era tion s u pport 5. IT h a rdwa re a n d s oftwa re en gin eerin g 6. M&E cou rs e des ign a n d developm en t ph a s e on e 7. M&E ca pa city developm en t ph a s e on e 8. Pu blica tion s a n d com m u n ica tion 9. Works h ops a n d lea rn in g n etworks SUB-TOTAL EXTERNAL FUNDS 150 000 220 000 ECSECC FUNDS 12 000 20 000 TARGETED DEPT. FUNDS 100 000 110 000 1 200 000 140 000 460 000 250 000 30 000 200 000 745 000 100 000 300 000 1 050 000 120 000 120 000 880 000 120 000 380 000 1 100 000 130 000 250 000 650 000 110 000 60 000 6 245 000 782 000 1 980 000 Communities of practice ECSECC /Office of the Premier (OTP) ECMERG M&E Technical Task Team Government departments (planning and M&E units) ECYC ECNGOCO IMT Organised business Treasury SALGA Legislature COSATU (labour) RIDA IPSP IGF Development Bank of Southern Africa (DBSA) Universities (Fort Hare, Natal etc.) M&E networks. DIMS Constraints & solutions Fragmented provincial M&E environment < > level the fragmented M&E environment Uncoordinated M&E environment in South Africa < > communicate, consolidate vertically Poor national and district level M&E linkages < > participate in integrated forums Changing information requirements < > review & incorporate current & future trends. Political agenda and outputs not factored in planning process < > consider in all planning cycle phases Constraints & solutions 2 Stakeholder mobilization & participation not comprehensive < > identify and engage with other stakeholders. Personnel shortages in provincial government < > identify and reorient existing personnel and make use of secondments from departments. Time for participation is short & clashes with other activities < > consolidate diaries & plan in advance. Financial constraints < > mobilize from existing budgets & write proposals to already known & interest donors. Magnitude of stakeholders and coverage area too big < > package participation into digestible groups with adequate representation and roll out programme in phases. Monitoring & Evaluation EX ANTE EVALUATION PROGRAM/ PROJ E CT APPRAIS AL Proje c t de s ign EX POS T EVALUATION IN ITINERE IN ITINERE MONITORING In pu t s Im ple m e n t at io n EVALUATION Out pu t s Im pac t s Mitiga tin g Fa ctors Replica tion , Red es ign , New projects on bas is of efficien cy, effectiven es s , releva n ce & s u s ta in a bilit y Action plan monitoring mechanisms Bi-monthly meetings and reports to Technical Task Team. Monthly meetings and reports to Eastern Cape monitoring and evaluation. Stakeholder workshops to provide feedback and reflect critically. Office of the premier who are the overseers of all provincial government programmes. Network partners such as UNDP, Free State Province and Universities of Fort Hare & Natal. Feedback loops by the core technical task team through the intra-net system within the province. Group discussions and feedback through other tools. Monitoring Action Plan +++ ++ X ** * ** * + ** * Jan Feb Mar ** * April ** * + ** * ** * May June july ** * + ** * ** * ** * + ** * Aug Sep Oct Nov Dec ** Bi-monthly meetings and reports to Technical Task Team. *Monthly meetings and reports to ECMERG; Treasury in-year quarterly meetings with departments; cluster meetings + Stakeholder workshops to provide feedback and reflect critically. +++ Departmental strategic planning X National provincial reports Risks Monitoring levels Direct Efficacy Relevance Sustainability OTP Impacts Effectiveness Departments Outcomes Inputs Efficiency Departments/ Implementers Key resources Secondments from departments Additional personnel at ECSECC New institutional arrangements for planning, implementation, management and monitoring & evaluation Departmental contributions DBSA IPSP ECSECC Key resources PIGITO Council OTP UNDP Free State province RIDA DIMS Universities DBSA Conclusion Integration of key lessons learnt Situation analysis Stakeholder identification & participation Multiple processes Vertical-horizontal linkages Methodologies & tools to all departments Project proposals Adequate resources What I will do Reconnaissance work Assume overall responsibility for the development of the M&E System; Prepare work schedule; Advance review of departmental plans with clear indicators; Meetings with key officials in departments & other stakeholders; What I will do • Provide inception reports to cluster groups, ECMETT & TTT; • Present draft PM&E design to ECMETT; • Organize consultative workshop(s) to review proposed design; • Provide project management support to M&E processes; • Develop operational guidelines and work plan for M&E; • Piloting the M&E system; and • Preparation of final report for critical reflection.