The UNDAF: Linking Analysis with Results Session 6 Session objectives • To introduce the concepts of RBM and to show how HRBA brings depth and legitimacy to our practice of RBM • To formulate results and indicators that responds to the analysis we conducted in session 5 What is a Result ? A describable or measurable change resulting from a cause and effect relationship - UNDG agreed RBM terminology What is results based management? A management strategy aimed at ensuring that activities achieve desired results Performance monitoring is a critical element • How well are results being achieved • What measures are needed to improve the process In plain language… RBM helps us to connect what we do to what we want to achieve RBM also tells us how we’ll know if we’ve achieved it Elements of RBM (1) • Problem analysis to understand causes Linkage RBM does not tell us the right questions to ask A HRBA does… HRBA and Problem Assessment & Analysis A HRBA helps the UN and partners to answer 4 critical questions: Who has been left behind Why? Which rights are at stake? Who has to do something about it? What do they need, to take action? Process and outcome are equally important Elements of RBM (2) • Structuring of programmes around a chain of SMART results – responding to analysis • Causality in the chain of results (if… then logic) • Use of ‘change language’ (future conditional) Linkage RBM does not tell us what kinds of changes we should be aiming for A HRBA does… HRBA and Change Process is guided by human rights principles … performance (behaviours of duty bearers and/or right holders and their institutions) Output: change in … …the capacity of duty bearers and right holders Conclusions and recommendations from UPR, Treaty Bodies, and Special Procedures help to identify specific behaviours and capacities Capacity Gap Analysis Outcome: change in… Role Analysis HRBA …quality of life (the realization of human rights) Causal Analysis RBM Impact: change in… Elements of RBM (3) • Indicators to measure performance • Costing of results (RBB) rather than isolated activity budgeting • Monitoring and reporting on progress against planned results Linkage RBM does not tell us how to monitor and report on change A HRBA does… HRBA and performance monitoring & measurement Human rights principles and standards guide… •Selection of indicators •Mechanisms for monitoring and reporting on development processes with stakeholders In sum… A HRBA brings depth and credibility to our practice of RBM by telling us… •the right questions to ask •the kinds of changes we should be aiming for •how to measure, monitor, and report on change with stakeholders. Advantages • Improved focus on results instead of activities • Improved transparency • Improved accountability • Improved measurement of programme achievements (performance rather than utilization) • Enhanced strategic focus • No choice, it is an industry standard • To get more funds!! Challenges • Difficult to apply causal logic • Difficult to learn • Difficult to integrate • Difficult to revise (... or reluctance to revise? ) • Difficult to measure • Difficult to ‘attribute’ (at outcome level, the UN is accountable but not wholly responsible) Go to typology Results Matrix Group Activity Using the set of cards provided, develop Results Framework Template a results framework… National priority/ goal Outcome Outcome Output Output Output Output Output Reduce the level of poverty and income inequality New businesses and jobs are created in targeted, poor rural and urban areas Market-based vocational training programmes are developed Disadvantaged groups including youth and PLWHA have access to improved employment services and preferential credits for business development. Local business development funds piloted in selected areas Private public partnerships (PPP)s are created in poor rural and urban areas for infrastructure development and service provision Local public administrations operate in a more effective and transparent manner Decentralization framework is strengthened, emphasising (1) delegation of authority, (2) decentralization of services, (3) costing of local services and budget needs of LPAs Administrative procedures and systems are simplified and streamlined to provide better services. Targeted departments of LPAs plan, implement, and monitor in a participatory manner Typical pitfalls • Wordy (..and no change language) To promote equitable economic development and democratic governance in accordance with international norms by strengthening national capacities at all levels and empowering citizens and increasing their participation in decision-making processes • Too ambitious Strengthened rule of law, equal access to justice and the promotion of rights • Containing multiple results The state improves its delivery of services and its protection of rights—with the involvement of civil society and in compliance with its international commitments Typical pitfalls • Wishy-washy (ie. Support provided to improve..) Support to institutional capacity building for improved governance • So general, it could mean anything To promote sustainable development and increase capacity at municipal level • Overlapping with National goals/ MDGs (impacts) Substantially reduce the level of poverty and income inequality in accordance with the MDGs and PRSP • Confusing means and ends Strengthen the protection of natural resources through the creation of an enabling environment that promotes sound resources management Country Examples Note to facilitation team: •Insert 1-3 slides with results chains from current UNDAF •With their new knowledge, ask participants to reflect on the ‘SMART’ness of their results •Examples from Lao PDR follow… Good example By 2013, women of reproductive age in Benguela province enjoy high quality maternal health care - % pregnant women completing 4 ANC visits - % births supervised by trained attendant - No. District Health Centres staffed and equipped to provide EMOC services - % women/ families reporting they are satisfied with obstetric care services (in-service survey completed in all health centres) It meets a standard related to the right to health It reflects recommendations from CEDAW, CRC, and CESCR It embodies the principle of equality and non-discrimination It contains a subject and a clear description of change (a “who” and a “what”) It is understandable by most readers It reflects a clear choice about policy or strategy It is specific, measurable, and time-bound But… • Is it achievable? • Is it relevant to National development priorities? • Is it a good bet? These are contextual factors Group Work In groups… • Reflect on the 3-step analysis • Develop… – 1 outcome A change in the performance of the rights holder or duty bearer…What are they doing differently? May involve legal, policy, and institutional reforms Hint: Combine the RH or DB and the claim for a more ambitious outcome Combine the DB and the obligation or RH and higher level capacity gaps (behaviours) for a less ambitious outcome – 2 contributing outputs New capacities that contribute to increased performance Hint: Look to the key capacity gaps of the DBs and RHs – particularly those related to skills, abilities, products and services Group Work (con’t) Build a results framework on the wall… Outcome Output Output UNDAF Process HRBA guides the process to measure, monitor, and report on progress with all stakeholders Road Map - Preparation process of the UNDAF Monitoring & Evaluation Country Analysis - Review existing analysis (assessment) - UNCT supported analysis - Identify UNCT comparative advantages Programme Planning & Implementation (Agency or Multi-Agency Joint Programmes) HRBA shows the kinds of results we should be aiming for HRBA offers the right questions to ask Strategic Planning - Strategic priorities for UNDAF/ UNDAF Action Plan A Human Rights-Based Approach to Monitoring Monitoring vs. Evaluation Monitoring Evaluation Systematic, ongoing Systematic, periodic During programme implementation During and after programme implementation Tracking of activities and progress Judgement of merit, value or worth of a programme/project According to AWP Compared to evaluation criteria (relevance, effectiveness, impact) For short term corrective action For decision-making about future programmes Accountability for implementation Accountability for results Contributes to evaluation For office & organizational learning Conducted by insiders Conducted by impartial outsiders Are we doing things right? Did we do the rights things? A Human Rights Based Approach to Monitoring • A comprehensive M&E plan sensitive to human rights concerns addresses: – Results and indicators – Processes (both development processes and processes for for monitoring and reporting) • Process monitoring – Checks that vulnerable groups are involved in programme implementation and benefit equally from programme results – Ensures that intended beneficiaries are able to participate freely in monitoring and reporting processes Some Fundamental Questions… • What to measure? • Who to involve? • How to measure? AND • How is M&E process addressing participation, nondiscrimination and accountability? (see checklist) What to Measure? • Ideally: – Results Outputs Outcomes Impacts (often tied to National monitoring systems for the National Plans, PRSs, and MDGs) – Mechanisms and processes for programme implementation, monitoring and reporting Who to Involve? • Another way of thinking about how to apply principles of ‘participation and inclusion’ to M&E process • A HRBA – Ensures that both rights-holders and duty-bearers are involved in M&E – Will pay special attention to measures that include vulnerable groups How to Measure? • Human rights principles and standards guide the selection of indicators and the development of monitoring and reporting systems • Indicators should be chosen that: – Capture the extent to which human rights principles have been incorporated into all stages of the programme – Demonstrate how incorporating human rights standards has contributed to overall programme effectiveness • Processes must be non-discriminatory, participatory and accountable – Be wary of “elite capture” What is an indicator? A tool to measure evidence of progress towards a result or that a result has been achieved Indicators “ Tools to measure evidence of progress towards a result or that a result has been achieved” Technical Qualities: Indicators must be able to be reliably replicated and objective Good and clear outcomes and outputs are necessary to generate good indicators Indicators must have a direct relationship to the outcomes or outputs they are measuring Indicators must be accompanied by baselines and targets Indicators Indicators describe how the intended results will be measured - accountability Objectively verifiable measures of a particular condition They force clarification of what is meant by the result like the “fine print” of an agreement Must be accompanied by baselines and targets Baseline, Target and Achievement Performance Commitment Planned Level of Achievement Current Level of Achievement Achievement At end of period Types of Indicators Quantitative statistical measures: • Number of • Frequency of • % of • Ratio of • Variance with Qualitative judgments or perceptions: • Alignment with • Presence of • Quality of • Extent of • Level of Example: Indicators, Baseline, Target and Source of Data Outcome: By 2007, more girls in Belem Province enjoy a quality, basic education Output: 800 teachers in Belem Province can deliver the new curriculum effectively Indicator: Net enrolment ratio (M; F) Baseline: F:45% Target: F: 75% - Improvement in school test scores Indicator: # Teachers with new certification Baseline: 0 Target: 800 -Teacher proficiency reports -Improvement of school satisfaction ratings Source of Data -MICS (survey) -Annual school test scores report Source of Data: - MECYS EMIS -School satisfaction surveys HRBA and indicator development • Inclusiveness: Do your indicators capture the experience of vulnerable and marginalised groups? Can your indicators be disaggregated? • Ownership: Have RHs and DBs contributed to the development of the indicators? Do they have confidence in the indicators chosen? • Clarity: Are they clear and understandable to all audiences, including vulnerable and marginalised groups? Group Work (30 m) In Groups… • Develop 2 indicators for each outcome and contributing output • Add to your results framework… Outcome Output Indicators Indicators Output Indicators Group Work (15 m) In Groups… • Choose 1 principle from… – Equality and non-discrimination – Participation and inclusion – Accountability and rule of law • Make up to 3 changes in your results framework (results and indicators) to reflect the principle • Be ready to present in plenary… Human Rights Principles • Universality and inalienability • Indivisibility • Interdependence and Interrelatedness • Equality and nondiscrimination • Participation and inclusion • Accountability and rule of law Gallery Your opportunity to “visit” other groups and give feedback Reflect on… – The logical flow of results and use of change language – The SMART-ness of the framework, particularly the formulation of indicators, assumptions and risks – The linkage between the results framework and the human rights-based analysis. Is it intuitive? Do the results respond to the underlying and root problems revealed by the analysis? Do the outcomes show changes in the performance of rights-holder to claim their rights, duty-bearers to meet their obligations? Do the outputs address identified capacity gaps? !! Remember - leave comments behind on post-it notes. UNDAF Process HRBA guides the process to measure, monitor, and report on progress with all stakeholders Road Map - Preparation process of the UNDAF Monitoring & Evaluation Country Analysis - Review existing analysis (assessment) - UNCT supported analysis - Identify UNCT comparative advantages Programme Planning & Implementation (Agency or Multi-Agency Joint Programmes) HRBA shows the kinds of results we should be aiming for HRBA offers the right questions to ask Strategic Planning - Strategic priorities for UNDAF/ UNDAF Action Plan