Business Case and Intervention Summary Intervention Summary Title: J-PAL's Governance Initiative (GI) What support will the UK provide? How much funding does the UK expect to provide? £904,000 in the form of an Accountable Grant. Period of funding? 3 years (Jan 2013 – Dec 2015) This is the second Accountable Grant to the J-PAL Governance Initiative. DFID provided £500,000 for the period January 2011 to March 2012 (see Quest No 3394371). Why is UK support required? What need are we trying to address? The increasing emphasis on the measurement of results has brought to the fore two related but distinct issues. First, what are the appropriate indicators of success of interventions designed to achieve social, political and institutional change? Second, what are the appropriate methodologies that can be used to monitor change and assess impact over the short and long-term, including at national scales? Primary data and high quality analysis is needed to assess who benefits, how, and in what contexts from different governance interventions. As the DFID Governance Portfolio Review (2010) concluded, there are serious limitations to the existing evaluation evidence on governance.1 Addressing this problem is urgent given DFID investment in this area (approximately £4 billion in 2004-9). Required also are methods that in addition to quantifying outputs and outcomes, can help to explain why different interventions are more or less successful in different contexts. The dearth of evidence on ‘what works’ in governance interventions is due, in part, to a number of methodological challenges. These include the difficulties of measuring complex outcomes such as “empowerment and accountability”2; the lack of standardized indicators; and difficulties in capturing impact of programmes (e.g. budget support) at national scales. Interest has grown in the use of experimental and quasi-experimental methods in this area3. However, given the importance of context in shaping social and political outcomes, the degree to which such methods can yield generalisable findings remains an issue of contention4. There are also limits to the questions such methods can answer; non-experimental methods are also vital. The Governance, Conflict and Social Development Research Team has targeted investment in the design and implementation of experimental impact evaluations on governance as one of its five priority areas.5 The purpose of this support is to begin to address this gap in evidence and to better understand the strengths and limitations of applying randomised methods in the governance sector. Under this thematic area, a £500,000 accountable grant was agreed with the Abdul Latif Jameel Poverty Action Lab (J-PAL) Governance Initiative at the Massachusetts Institute of Technology (MIT) which ended in March 2012. The Governance Initiative supports field experiments in governance, which currently comprise a small but growing area of research in development economics. Co-funding 1 DFID Governance Portfolio Review (July 2011), page 3. Cf E&A evidence review 3 Garcia (2011), Micro-methods in Evaluating Governance Interventions, BMZ, Bonn; Moehler (20xx), XXXXX 4 Moehler, as above, and Chris Blattman presentation to DFID 2011?? 5 Governance, Conflict and Social Development Research Priorities (November 2011). 2 1 is provided by the Hewlett Foundation and an anonymous donor (£2.6m, 2011-14). DFID funding was used to support a cluster of experimental impact evaluations in the areas of anti-corruption and citizen participation. The Project Completion Report at the end of this phase of DFID support concluded: “It is recommended that DFID funds a second phase of the J-PAL Governance Initiative but that any future support includes an independent evaluation of our support to Phase One. This should address question on the vfm of RCTs, generalisability as well as the significance of the evidence base generated through the Governance Initiative.” What will we do to tackle this problem? DFID funds will support: 1. 2. 3. 4. 5. 6. Additional randomised impact evaluations of innovative programmes designed to improve participation in the political and policy process, and reduce leakages in public programmes; A comprehensive update of the J-PAL Spring 2011 Governance Review Paper (this sets out what is currently known or not known about the drivers of governance and identifies an agenda for future research); A roundtable in fall 2013 for donors, researchers, and J-PAL staff to discuss preliminary results from GI funded projects, the updated review paper, mapping of funded proposals onto the existing priorities, and objectives for future funding rounds; A policy-research conference in 2015 to disseminate results, assess any remaining gaps in the evidence, and set the future research agenda; Strengthening capacity of DFID in randomised evaluation through 2 members of the DFID attending J-PAL’s Executive Education training on randomized evaluations per year (JPAL will cover the tuition cost of $4000 per person) and provision of advice to DFID country offices on RCT design on a case by case basis; and An evaluation of J-PAL’s Governance Initiative by an independent and professional evaluator Who will be implementing the support we provide? The GCSD team will provide an Accountable Grant to the Abdul Latif Jameel Poverty Action Lab (JPAL) at the Massachusetts Institute of Technology (MIT) to implement the above. An evaluation of JPAL’s Governance Initiative (GI) will be conducted by an independent and professional evaluator. What are the expected results? What will change as a result of our support? Outputs 3 grants awarded for high-quality, promising research proposals designed to improve participation in the political and policy process 4 grants awarded for high-quality, promising research proposals designed to reduce leakages in public programmes 7 policy or stakeholder meetings to promote use of evidence, and establish the priorities for the next round of the initiative 1 roundtable for donors, researchers, and J-PAL staff to discuss preliminary results from GI funded projects, the updated review paper, mapping of funded proposals onto the existing priorities, and objectives for future funding rounds 1 policy-research conference for 40-65 attendees to disseminate research results and assess the remaining gaps in the literature. The results of the conference will be incorporated into the updated review paper. An evaluation of the Governance Initiative in 2015. Outcome: policymakers, donors, practitioners, and experts have access to new research results from 2 rigorous, high-quality evaluations on the causes and consequences of poor governance in low-income countries, and how to effectively improve governance Impact: policymakers, donors, practitioners, and experts use evidence on the causes and consequences of poor governance in low-income countries, and what works (and what doesn't) to improve governance How will we determine whether the expected results have been achieved? Impact, outcome and output indicators will be monitored throughout the lifetime of the programme. Data will be obtained through routine programme monitoring tools and processes. An evaluation will be conducted by an independent and professional evaluator. S/He will assess the overall impact effectiveness, relevance and cost efficiency achieved, and test assumptions underpinning the Theory of Change. 3 Business Case Strategic Case A. Context and need for a DFID intervention Despite its crucial importance for development, most questions about how to effectively improve governance remain unanswered. While rigorous evaluations of programmes in areas such as health and education are well established, they are far more limited in the governance sector. Empirical research on the causes and consequences of poor governance in low-income countries faces several challenges. First, the illegal nature of many corrupt activities means that data on the extent of ‘misgovernance’ is often hard to obtain. Second, obtaining causal evidence on why poor governance persists requires the researcher to disentangle the channels of influence. It is often difficult to determine the specific causes of observed behaviour. For example, how do we know if observed voter choices reflect people’s true electoral preferences or choices limited by poor information about politician performance? The J-PAL Governance Initiative focuses on field experiments in governance, which currently comprise a small but growing area of research in development economics. Randomised controlled trials (RCTs) have potential to advance our understanding of the impact of governance interventions. Their value lies in the ability to attribute the impact of specific interventions applied experimentally to the beneficiary and control groups. In the case of governance, RCTs have focused on community participation, service delivery, anti-corruption, gender, and political participation. They demonstrate the value of financial incentives and oversight mechanisms in shaping the behaviour of civil servants and reducing absenteeism, provide mixed evidence on the efficacy of community monitoring, and show how electoral quotas can advance women’s political participation and development efficacy. The number of randomised control trials in governance remains small and it is difficult to generalise from a limited set of studies. There is an ongoing need for replication of field experiments in similar projects in different contexts which DFID support for the Governance Initiative will help to address. B. Impact and Outcome that we expect to achieve Expected Outcome: Policymakers, donors, practitioners, and experts have access to new research results from rigorous, high-quality evaluations on the causes and consequences of poor governance in low-income countries, and how to effectively improve governance Expected Impact: Policymakers, donors, practitioners, and experts use evidence on the causes and consequences of poor governance in low-income countries, and what works (and what doesn't) to improve governance Appraisal Case A. What are the feasible options that address the need set out in the Strategic case? Option 1: The GCSD team will provide an Accountable Grant to the Abdul Latif Jameel Poverty Action Lab (JPAL) at the Massachusetts Institute of Technology (MIT) to: (1) carry out a discrete number of additional randomised impact evaluations of innovative programmes designed to improve participation in the political and policy process, and reduce leakages in public programmes; (2) conduct a comprehensive update of its Spring 2011 Governance Review Paper (this sets out 4 what is currently known or not known about the drivers of governance and identifies an agenda for future research); (3) host a roundtable in fall 2013 for donors, researchers, and J-PAL staff to discuss preliminary results from GI funded projects, the updated review paper, mapping of funded proposals onto the existing priorities, and objectives for future funding rounds; and (4) host a policy-research conference in 2015 to disseminate results, assess any remaining gaps in evidence, and set the future research agenda. An evaluation of J-PAL’s GI will be conducted by an independent and professional evaluator in 2015. Appraisal The Governance Review Paper identified two areas where randomised methods might add particular value: i) how to improve the provision of public goods and services by improving citizen oversight and participation; ii) how to reduce leakages, exploring a wide range of mechanisms from innovative technologies to the provision of incentives. Selection of these areas is credible within the wide domain of governance activities. These themes also align closely with DFID priorities. Specifically, anti-corruption, and empowerment and accountability are key areas of concern to the current Administration. J-PAL researchers have significant previous experience undertaking work in these areas. J-PAL is a network of 75+ researchers who have completed over 350 evaluations in 52 countries. Under GI, JPAL has already launched four calls for proposals to undertake work against the selected thematic headings in governance. Funding process: The GI funding process is structured in three steps: (a) call for research proposals for evaluations of programs aimed at improving governance; (b) evaluate proposals based on a set of criteria; and (c) fund randomized impact evaluations of governance programs in developing countries. A Request for Proposals is issued every six months to fund both full evaluations and smallscale pilots. First, all proposals are peer reviewed. Finally, application proposals and comments from referees are received by the Review Board who makes the final decision on how to allocate funds. Selection Criteria: All proposals are evaluated based on the following criteria: Relevance; contribution; value of research; technical design, viability of the project, policy relevance and publishing data. Ethics: Because J-PAL studies involve human participants, J-PAL’s affiliates and their staff ensure that studies meet the guidelines of ethical research methods. This includes: Receiving MIT institutional review board (IRB) approvals for each study before it begins at the PI’s home institution. Providing IRB training course to all personnel involved in the study. Adhering to the IRB approved research protocol and guidelines throughout the course of the study. MIT provides final review of any projects awarded under GI and by policy does not issue a sub-award until all necessary IRB approvals are in place and on file. Environment: As work under GI funds only evaluations of policies and interventions, and not the interventions themselves, there is rarely if ever an environmental dimension. However, MIT committees that review projects as needed for environmental concerns include: Animal Usage, Biohazard Materials, and Radioactive Isotopes, among others. There is not a required environmental review of social science projects, but there is a standing Environmental Health and Safety office at MIT, which advises on these matters as needed if requested by GI or during MIT’s review. 5 Value for Money: Being hosted at J-PAL ensures that GI funding represents good value for money: by directing GI’s funding towards research that is “demand driven,” based on the priorities identified by the review paper; through GI’s institutional continuity, in the form of an underlying review paper agenda, cochairs, and reviewers. This means that new projects build upon the learning from one year to the other and complement existing projects so that the body of knowledge on governance is built in the most effective way possible; and by highest calibre peer review and advice during all stages of the process from the proposal review process all the through to day-to-day implementation J-PAL's dedicated policy group will add value through their work to compare the relative costs and impacts of various alternative interventions; disseminate the results from GI funded evaluations through policy publications, networking, and conferences; and facilitate the scale up of the underlying programs, if found successful, by identifying and guiding local partners. The lack of this type of support is one of the key reasons why many promising research results never get translated into meaningful policy changes or new programs on the ground. J-PAL has four other initiatives and thus has significant experience managing competitions of this type. Increasing the number of studies in this relatively small area has the benefit of allowing for further comparative analysis of findings and so potentially increasing generalizability and impact J-PAL staff will synthesise findings once a sufficient number of studies have been completed (expected early to mid-2015) in order to highlight any generalisable lessons. The results of the evaluations will be publicly accessible through J-PAL’s website. J-PAL will make presentations to DFID staff, and more broadly to key stakeholders in governance debates. Findings will also be disseminated at the Policy Conference planned for 2015 and drawn out in the subsequent Evaluation. Over the past year, J-PAL has made great efforts to be responsive to opportunities suggested by DFID to establish disseminate learning around impact evaluation and develop potential partnership opportunities around upcoming evaluations, for example: - At the DFID Malawi, J-PAL organized a 2-day evidence and training workshop on impact evaluation of governance programmes in December 2011 in Lilongwe, Malawi. J-PAL provided affiliates, staff and practitioners (See http://www.povertyactionlab.org/africa/malawi-governanceworkshop); - Rohini Pande, a co-chair of GI, was a speaker at the DFID Governance and Conflict Professional Development Conference 2011; - J-PAL affiliate participated in DFID’s "Workshop for Specialists" London, 2012; - J-PAL South Asia developed in collaboration with GCSD/RED a training session conducted in late 2011 for DFID India on impact evaluations; and - J-PAL Africa, sister-organization, Innovations for Poverty Action (IPA) and DFID Nigeria are discussing a range of potential partnerships opportunities around upcoming evaluations and other evaluation interests and identifying academic partners for future DFID evaluations. The priority evaluations identified by DFID Nigeria focused around governance and elections, and more recently around education in northern Nigeria. In May 2012, IPA staff travelled to Nigeria to meet with DFID Nigeria teams and led three tailored presentations on health, education, youth, conflict, employment and elections. The presentations focused on evaluation results to date, the structure of randomised control trials, key lessons to incorporate into upcoming programming and how to integrate rigorous evaluation into program planning and design. Moving forward IPA aims to identify interested researchers to collaborate on future evaluations with DFID Nigeria and 6 explore a more formal collaboration on one or more evaluations in 2013. A workflow graphic of J-PALs Theory of Change is shown below. Option 2 (the “Do Nothing” counterfactual): DFID does not provide further funding for J-PAL’s GI. An evaluation of J-PAL’s GI is not conducted. Appraisal This Option is discounted due to the significant opportunity lost. J-PAL holds a number of high quality and promising research proposals which is does not have sufficient budget to fund. Under this option, these evaluations would not take place due to the GI’s insufficient budget, and would not add to the existing knowledge on what works (and what doesn’t work) when trying to improve governance. Furthermore, this Option represents a waste of the learning accumulated by MIT’s JPAL, now an already tested (process-wise) model. The Governance Review Paper is already out of date, and has not been disseminated in a forum of policymakers, donors, practitioners and experts to encourage uptake and impact. Unless it is updated and enhanced, it will not become a fresh repository of learning and guidance for these target audiences. B. Assessing the strength of the evidence base for each feasible option In the table below the quality of evidence for each option is rated as either Strong, Medium or Limited Option Evidence rating 1 Medium 2 Limited 7 What is the likely impact (positive and negative) on climate change and environment for each feasible option? Categorise as A, high potential risk / opportunity; B, medium / manageable potential risk / opportunity; C, low / no risk / opportunity; or D, core contribution to a multilateral organisation. Option 1 2 Climate change and environment risks Climate change and environment and impacts, Category (A, B, C, D) opportunities, Category (A, B, C, D) C C C C There is likely to be minimal direct climate or environmental risks from this programme. Direct impacts are likely to be associated with any travel that may be incurred as part of the management and implementation of programme activities. Partners should be advised that the carbon footprint of all travel supported by this programme should be minimised and offset where possible; this should include the use of economy flights whenever possible. Recent evidence suggests that, for long haul flights, business class and first class seats account for GHGs emissions that are respectively 2.9 times and 4 times greater than those for economy seats (DEFRA, 2009. See http://www.defra.gov.uk/environment/business/reporting/pdf/091013-guidelines-ghg-conversionfactors-method-paper.pdf; A. Kollmuss & Crimmins A. M; Carbon Offsetting & Air Travel Part 2: Non-CO2 Emissions Calculations, SEI discussion paper, 2009). Furthermore all documents, reports and paper outputs funded by the programme should be printed on paper made from sustainably managed forests and/or recycled paper in order to mitigate the environmental impact of running the project operations. This project presents also some indirect opportunities to improve environmental stewardship and management, as well as to improve the response of beneficiary countries/institutions to climate change. Improved political participation and governance leading to reduced corruption can result in better management and conservation of natural resources and the environment (Tacconi, L. 2011. Developing environmental governance research: The example of forest cover change studies. Environmental Conservation, 38(2): 234-246.; HARMAN, 2005; Costanza, R. et al., 1997 ‘The value of the world's ecosystem services and natural capital’, Nature 387: pp. 255). Therefore an intervention such as this, which aims to measure the impact of governance interventions could contribute to improving the effectiveness of these and consequentially -albeit indirectly- improve environmental and climate policies. However, these impacts are likely to be generated in the medium to longer term and difficultly may arise around attributing them directly to the project activities. Nevertheless it would be advisable to keep track and monitor the number of environmental and climate related policies which would be directly affected from the activities of this intervention. These level of activities could also be compounded by a ‘qualitative’ study to assess or evaluate how such policies have been impacted by the supposed increase in the uptake of evidence and whether any beneficial impacts or outcomes were derived in terms of improved climate and environmental policies and/or action plans C. What are the costs and benefits of each feasible option? Cost Research - £707,000 Research uptake - £90,000 Review - £25,000 Indirect costs - £82,000 Benefit Risk 84% of funding goes direct to Risk is in one place research/uptake DFID resource cost low as research projects already selected and of suitable quality Input is centred around contributions to the Steering Committee and general project 8 management Direct day to day management is through J-PAL so quality assurance is anticipated D. What measures can be used to assess Value for Money for the intervention? The Steering Committee will be advised by information from the quarterly progress meetings (see Management Case) and other relevant information will assess the value for money as part of their remit. In doing so, the Steering Committee will be responsible for VfM of the quality of the research stemming from the investments made. This will include, as outlined in the indicative budget, a ‘value for money’ approach to the cost of researchers, travel, publications, conference etc. E. Summary Value for Money Statement for the preferred option The project follows from the earlier round of funding. For this type of research, a key aim is J-PAL’s use of ‘an informed, efficient and transparent' process that provides value for money in delivering its objectives. These high quality proposals were selected through an open call and evaluated against published criteria ensuring transparency of process and the aim of taking forward only the highest quality of research proposals. Having already been through this stringent selection process, there is no direct cost to DFID in issuing a new call to obtain research proposals in this area. We also have the benefit of time savings as the research is ready to be picked up through J-PAL. We are confident that this continued involvement with J-PAL provides this and meets these objectives. By developing this partnership we have maximised efficiency by reducing DFID’s administrative and transaction costs. We have used JPAL’s systems and working methods in the past and are satisfied that their activities, management structure, track record on a) managing research; b) commitment to knowledge sharing and research uptake; c) strengthening their M&E systems; and d) delivery of results which represent good value for money Commercial Case Direct procurement A. Clearly state the procurement/commercial requirements for intervention DFID intends to fund this 3-year research programme through the Accountable Grant mechanism. This project has been generated from an earlier DFID funded project with J-PAL who held an open call for proposals to who could deliver this programme. There was a high response with a high quality of proposals received. All bids received were evaluated against published criteria to determine which supplier’s technical and commercial proposal would best meet the call’s requirements to deliver high quality research required and suitably address the value for money B. How does the intervention design use competition to drive commercial advantage for DFID? 9 DFID, together with J-PAL, has sought to obtain the best value for money possible through selecting a suitable supplier through a competitive 2-stage process. The first stage involved an open, global call for proposals. The open and competitive process was chosen to aim for a wide response from the market and to ensure that the costs would be proposed on a competitive basis. C. How do we expect the market place will respond to this opportunity? The market provided a high response to the initial call run by J-PAL. The call was widely publicised among the research community and appropriate websites. There was a high calibre of proposals received and were assessed based on the strength of the research team’s expertise and experience, project management skills, as well as their ability to address the broader research questions. D. What are the key cost elements that affect overall price? How is value added and how will we measure and improve this? The majority of the funding will go direct to the research - Research - £707,000 Outreach and engagement with policymakers and key stakeholders - £90,000 Indirect costs - £82,000 Evaluation - £25,000 The major costs drivers for the evaluation will be professional services. Additions of value to the programme will be ensured by the use of accredited evaluation contractors with sufficient capacity to perform the tasks required to a high standard. Terms of Reference will be used to define the framework (including key performance indicators) within which contractors will be expected to work; these will provide the benchmark against which satisfactory performance is measured and payment will be made upon their fulfilment. E. What is the intended Procurement Process to support contract award? Accountable Grant with J-PAL F. How will contract & supplier performance be managed through the life of the intervention? A Steering Committee comprising representatives from DFID and J-PAL will be used to manage supplier’s performance and delivery against the Logical Framework Analysis. The supplier will provide periodic progress reports on the achievement of outputs against the agreed budget, which will be used to feed into the quarterly progress meetings and the Annual Review process. This should ensure that the supplier is clear on the output to be achieved to the agreed quality within the agreed timescale, and reports any deviation, acts on it early in the process, and presents plans to bring the outputs back in line Indirect procurement A. Why is the proposed funding mechanism/form of arrangement the right one for this intervention, with this development partner? An Accountable Grant allows DFID to manage results while allowing academic teams at J-PAL (part of MIT) to use their extensive knowledge and expertise to select the best projects to fund under GI, which is now entering its third year. 10 B. Value for money through procurement The great bulk of funding will go to direct research project costs; MIT accepts 10% overhead rate on this particular grant and sub-awardees under the Governance Initiative are limited to 9%. Financial Case A. What are the costs, how are they profiled and how will you ensure accurate forecasting? The costs are largely for direct research project costs in developing countries to evaluate interventions. Field costs are made up of field survey costs, data analysis effort, and research infrastructure costs in areas such as data security, data checking, etc. Under the Initiative, J-PAL’s affiliated researchers and financial team evaluate proposed project budgets for cost realism, based on their extensive experience on similar projects in similar countries. B. How will it be funded: capital/programme/admin? Programme C. How will funds be paid out? Payments will be made under the schedule to be agreed in the Accountable Grant. The funding will be based against a rolling forecast with spend linked to agreed payment schedules with J-PAL to ensure funding is provided on an appropriate basis against the actual spending?. D. What is the assessment of financial risk and fraud? Low – MIT is an established organisation with a good track record and strong financial and administrative systems to monitor spending against the approved items in the budget. E. How will expenditure be monitored, reported, and accounted for? Expenditures will be monitored through MIT’s US Government-approved financial and grants management system, with detailed financial reporting done on an annual basis by J-PAL, using MIT’s accounting data. Management Case A. What are the Management Arrangements for implementing the intervention? A Steering Committee will be established comprising representatives from DFID and J-PAL. This Committee will come together for quarterly progress meetings (virtual or actual) meetings and for the Annual Review process. J-PAL will be accountable to DFID for managing the implementation of the research programmes. JPAL will be responsible for overseeing all aspects of the projects and will provide DFID with regular updates through the quarterly progress meetings of the Steering Committee. In J-PAL, three experienced co-chairs will have overall responsibility for successful implementation: Rohini Pande is a member of J-PAL’s Board of Directors and the Mohammed Kamal Professor of Public Policy at Harvard Kennedy School of Government Benjamin Olken is a director of J-PAL and a professor in the Department of Economics at the 11 Massachusetts Institute of Technology (MIT). Iqbal Dhaliwal is the director of policy for J-PAL Global, a member of J-PAL’s Board of Directors, and the Scientific Director for J-PAL South Asia. The GI will also draw upon many of J-PAL’s skilled staff, including Conner Brannen, who will be the dedicated programme administrator for GI, and David Sears, J-PAL’s head of Finance, who will provide all financial support both in terms of reporting requirements to GI’s funders as well as with grantees In DFID, a Senior Responsible Officer (SRO) will have overall responsibility for the quality and the strategic direction of the programme. The SRO will work closely with an assigned project officer in DFID to ensure the programme is well managed according to the DFID Blue Book rules and principles. The SRO and the Project Officer will form the core DFID team. The SRO will establish a virtual team to ensure this programme is well managed and benefits from a multi-disciplinary approach. The Senior Governance Adviser or Governance Adviser in the GCSD team will be the SRO for this programme. B. What are the risks and how these will be managed? (a) Not enough demand for funding for governance-related projects (b) Not enough high-quality proposals are submitted for funding Analysis of proposals received during the previous three rounds of GI suggests that both of these scenarios are unlikely. GI has actually seen a higher level of demand for funding than was originally expected, and estimates that the demand will continue to increase over time. GI has also seen a steady stream of high quality research proposals across rounds of funding and it seems unlikely that this will change in the near future, especially given that the number of J-PAL affiliates is growing, J-PAL is adding a Southeast Asia regional office, and more of J-PAL’s researchers are working on governance issues. While it is too soon to assess for the 1st phase projects funded through DFID, given the high quality of proposals received by J-PAL, DFID is taking the considered risk of funding other high quality proposals which cover areas in which DFID has a firm interest. (c) J-PAL being over-stretched The pace of predictable implementation, timeliness of financial and narrative reporting, and strong financial management evidenced by accurate forecasting and timely invoices will be monitored throughout the lifetime of the programme. (d) Round-table and policy conference has little impact This risk will be managed and mitigated by ensuring that dissemination activities (roundtables, conferences etc.) have a clear agenda/purpose, and participants are aware of their role in each activity. Risks (a) Not enough demand for funding for governance-related projects Indicators to Be Measured Ratio of total amount requested to total funding available in each round (b) Not enough high-quality proposals are submitted for funding Ratio of total amount requested for proposals deemed to be of high quality by the Review Board to the total amount available in each round (c) J-PAL becomes over-stretched - Regularity and predictability of RFP rounds Timeliness of invoices 12 (d) Roundtable and policy conference has little impact A clear agenda is developed Relevant stakeholders are invited, and are aware of their role in each activity C. What conditions apply (for financial aid only)? None D. How will progress and results be monitored, measured and evaluated? Impact, outcome and output indicators will be monitored throughout the lifetime of the programme. Data will be obtained through routine programme monitoring tools and processes. An evaluation of JPAL’s Governance Initiative (GI) will be conducted by an independent and professional evaluator. S/He will assess the overall impact effectiveness, relevance and cost efficiency achieved by J-PAL, and test assumptions underpinning the programme. Logframe Quest No of logframe for this intervention: 13