Aid Program Performance Report 2013-14 Australian Non-Government Organisation Cooperation Program (ANCP) September 2014 Key Messages This Aid Program Performance Report (APPR) outlines progress made in the program management and the development activities of Australian Non-Government Organisations (ANGOs) funded under the Australian NGO Cooperation Program (ANCP). The report provides an evidence-based assessment of the progress of the ANCP against Australia’s aid objectives and outlines proposals for management action. The report assesses the ANCP program during 2013-14, while development activity results and expenditure are drawn from 2012-13.1 The ANCP recognises the unique strengths that ANGOs bring to development activities and both complements and strengthens Australia’s overall efforts to alleviate poverty. For 40 years the Australian Government has provided funding through the ANCP to support development activities of accredited ANGOs that directly alleviate poverty in developing countries. The ANCP is well placed to support the Government’s new aid policy and effectively contribute to the investment priorities of Australia’s aid program.2 Key findings of this report are: 1) The ANCP continues to deliver tangible results against its overall objective Funding provided under the ANCP is flexible and predictable, and has enabled 43 ANGOs and hundreds of their in-country partners to deliver more than 600 projects in over 50 countries in 2012-13. With $106.4 million in funding, the ANCP benefitted the lives of 19.7 million people (58 per cent women and girls) of whom 89 per cent were located in rural areas. Over 71 per cent of ANCP programming directly contributed to the ability of poor communities to work with the private sector to achieve economic development, improve local and national governance services and empower women and people with disability. The top three investment priorities were health (28% of programming), disaster risk reduction and social protection (15%) and rural development and governance (14%). 2) Engaging with ANGOs as partners is integral to the success of the ANCP The ANCP is an effective tool for high-level development policy dialogue between the ANGO sector and DFAT. The top ten recipients of ANCP funding (ANCP Partners) play a particularly important role in the scope and scale of the Australian aid program, drawing on their significant experience in implementing development activities, and their established global networks. Under a partnership agreement, the ten largest Australian NGO partners were awarded multiyear grants providing funding certainty and the ability for long term program planning to enhance sustainability of ANCP activities. In 2012-13 these ANCP Partner NGOs matched over $22.7 million in funds, reaching 17.5 million people. ANCP Partners include World Vision Australia, Oxfam Australia, CBM Australia, CARE Australia, ChildFund Australia, Plan International Australia, The Fed Hollows Foundation, Save the Children Australia, TEAR Australia, and Caritas Australia. 3) There is scope to improve the ANCP There are challenges in drawing a line of sight from the overall objective of the ANCP to impact on the ground and to articulate results and outcomes against a set of program objectives. A monitoring and evaluation system is used at the activity level, and data is then aggregated to highlight the program’s results against common indicators. However due to the size and scope of the ANCP, there has not been a framework in place to measure the impact of the ANCP program as a whole. In 2014-15 key stakeholders will be consulted on ANCP’s program logic to 1 Results from 2013-14 will be available in November 2014. 2 Australian aid: promoting prosperity, reducing poverty, enhancing stability http://aid.dfat.gov.au/Publications/Pages/australian-aid-promoting-prosperity-increasing-stability-reducing-poverty.aspx Aid Program Performance Report 2013-14 1 inform the development of a performance assessment framework. This framework will ensure the program remains relevant, can align with the new aid policy and measure progress against objectives. Findings from an independent evaluation by the Office of Development Effectiveness (ODE) in 2014-15 will also inform ANCP’s program policy. Program context The ANCP supports ANGOs that have passed a rigorous accreditation process with the Department of Foreign Affairs and Trade (DFAT). The ANCP extends the reach of the Australian aid program because it is a flexible and responsive funding model, and because it supports ANGOs’ strong focus on community development activities. The ANCP also offers a partnership model for policy dialogue on development issues that makes policy formulation more robust. For example, Christian Blind Mission Australia (CBM) worked closely with DFAT on the Development for All policy3 including providing advisory support and training to DFAT and delivery partners on disability inclusive development. Many ANGOs funded under the ANCP have been working in international aid and development for more than 60 years and often operate in areas where the government cannot. The ANCP celebrated its 40th anniversary in 2014 which is testament of the program’s long standing relevance. The Government is committed to supporting ANGOs because they are fundamental delivery partners for our aid program. There is significant Australian public support for ANGOs with more than 2 million Australians donating over $1 billion to ANGOs each year4. Through ANCP, the Government builds on this public support to ANGOs to maximise results in poverty reduction. The ANCP represents value for money. ANGOs leverage funding they receive from the Australian community to meet a required 20 per cent match under the ANCP, committing $1 for every $5 in ANCP funding. In reality, from acquittals received by DFAT, ANGOs provide greater match funding, including funding from other donors, where every dollar invested by DFAT translates into $1.50 on the ground. ANCP funds equalled only 2.1 per cent of total overseas development expenditure managed by DFAT in 2012-13, however these ANCP activities contributed to more than 50 per cent of many of the aid program’s individual aggregate development results.5 For example, over 960,000 women and men in poverty increased their access to financial services, representing 62 per cent of the aggregate results for the aid program in 2012-13. Furthermore, for every one person who directly benefits from ANCP activities, a further 1.5 people are reached indirectly. On average, for every ANGO which is funded under the ANCP, 36 in-country partners directly participate in projects, including the private sector, governments, local community groups and organisations for people with disability. The ANCP has undergone significant growth in recent years. Funding under the ANCP was $44 million in 2008-09, and tripled to $131 million in 2013-14. The number of ANGOs participating in the ANCP has also increased from 41 in 2009-10 to 44 in 2013-146. During 2013-14, the policy context in which the ANCP operates has been impacted by a change in government, integration of AusAID into DFAT, staff turnover and the introduction of a new aid 3 Development for All policy http://aid.dfat.gov.au/Publications/Pages/8131_1629_9578_8310_297.aspx 4 Australian Council for International Development (ACFID) http://www.acfid.asn.au/about-acfid/what-is-acfid 5 In 2012-13, 1,983 civil society organisations supported to track service provision (54% of total); 121,494 people provided with disability services like prostheses and assistive devices (89% of total); 250,886 poor women and men who gain access to agricultural technologies (50% of total); 966,322 poor women and men who increase their access to financial services (62% of total). Source: Department of Foreign Affairs and Trade, Aggregate Development Results for 2014. 6 A list of DFAT accredited ANCP NGOs is available at http://aid.dfat.gov.au/ngos/Pages/accredited.aspx Aid Program Performance Report 2013-14 2 policy and performance framework7. This year has also seen access to an unprecedented amount of program data through the introduction of a new online grant management system. Maintaining program operations was constrained by the factors noted above resulting in delayed payments to ANGOs, postponement of some overseas monitoring visits, and slower advancement on the ANCP Partnerships. However, the ANCP was able to progress some important pieces of work, namely: constructive engagement between the new government and ANGOs; transitioning the accreditation application system online8; and completing an independent review of ANCP NGO evaluations of activities in Cambodia.9 Expenditure in 2012-13 ANGOs received $106.4 million in funding from ANCP in 2012-13, which is an increase of $8.4 million from 2011-12 ($98 million). In 2012-13 total Overseas Development Assistance (ODA) funding to NGOs (Australian and non-Australian) was $564.5 million, or 11.0 per cent of ODA. ANCP funding represents 19 per cent of total funding to NGOs. Table 1: Total ODA expenditure10 to NGOs, 2012-13 A$ million % of ODA All Australian NGOs (102 NGOs) 299.1 5.8 Accredited Australian NGOs (44 NGOs) 246.9 4.8 ANCP funding (43 NGOs)* 106.4 2.1 Other funding (28 NGOs)^ 140.5 2.7 Non-accredited Australian NGOs (62 NGOs) 52.2 1.0 Non-Australian NGOs (224 NGOs) 265.4 5.2 All NGOs (326 NGOs) 564.5 11.0 *Australian Red Cross is accredited but has not received ANCP funding since 2010-11 ^28 out of 43 accredited NGOs received other sources of DFAT funding Source: DFAT ODA Statistics and Reporting Section5 August 2014 Diagram 1 shows the ANCP contribution to Australia’s aid program by the new investment priorities11. Diagram 1: ANCP expenditure by investment priorities 2012–13 Health 28% General Development Support 14% Infrastructure, Trade Facilitation and International Competitiveness 5% Agriculture, Fisheries and Water 13% Effective governance: policies, institutions and Building Resilience: functioning humanitarian 7 Making Performance Count: enhancing the accountability and effectiveness of Australian aid economies assistance, disaster http://aid.dfat.gov.au/Publications/Pages/framework-making-performance-count.aspx 14% risk reduction and 8 Accreditation acts as a front-end risk management process for DFAT. Further information on the ANCP accreditation process social protection Education is available at http://aid.dfat.gov.au/ngos/pages/accreditation.aspx 9 15% 11% The ANCP Meta-Evaluation http://aid.dfat.gov.au/Publications/Pages/ancp-meta-evaluation-2013.aspx by Debra Hartley 10 ODA managed by the Department of Foreign Affairs and Trade. 11 Investment priorities for the Australian aid were introduced with the new aid policy framework, Australian aid: promoting prosperity, reducing poverty, enhancing stability, June 2014. Last year this information was presented by strategic goals. Aid Program Performance Report 2013-14 3 With over 60 per cent of the ANCP investment, Asia and the Pacific remained the highest-priority regions for Australian aid in 2012-13. ANCP investment increased in Africa and Middle East from 18 percent in 2011-12 to 23 percent in 2012-13. There were minimal changes in the percentages for the other regions from 2011–12 to 2012–13. Diagram 2: ANCP expenditure by region 2012–13 Rest of the World and Unallocated 15% South Asia 14% Africa and Middle East 23% PNG & Pacific Island Countries 12% North, Central and South America 1% East Asia 35% Progress towards objective of the ANCP The objective of the ANCP is ‘to support accredited ANGOs to implement their own programs and strategic directions consistent with the Australian aid program’s strategic goals and objectives of poverty alleviation and sustainable development’. This is achieved by: promoting progress against the Australian aid program’s strategic goals targeting the most vulnerable and marginalised communities providing grant funding to Australian NGOs with strong community support and effective systems. The evidence base to demonstrate ANCP effectiveness is drawn from a variety of sources (table below). The evidence indicates progress in achieving results in poverty reduction. It also provides assurance that the ANCP continues to fund high-performing ANGOs. Table 2: ANCP Source Documentation Type Date ANCP NGO annual performance reports (2012-13) DFAT ANCP NGO survey results (April 2014) DFAT monitoring and evaluation visit findings (2013-14) ANCP meta-evaluation report (2013) Aid Program Performance Report 2013-14 4 Monitoring Evaluation and Learning Framework (MELF) review12 (2013) Recognised Development Expenditure (RDE) examination (2013) 13 Accreditation assessments and RDE spot checks (2013-14) With the ability to use up to 10 per cent of their ANCP grant towards design, monitoring and evaluation (DME) activities, ANCP NGOs invested over $8 million in DME activities and building the capacity of local partners in 2012-13. This same year 32 ANGOs submitted evaluations of 125 activities funded under ANCP, demonstrating their commitment to improving the effectiveness of aid delivery. Currently, DFAT has not been able to fully review and verify the wealth of performance information submitted by ANGOs. This will be a feature of the performance assessment framework in the future. Given the above, the ANCP was assessed against its overall objective (and sub-objectives) and found to be delivering results, maintaining its ‘effective’ rating from 2012-13 to 2013-14. Table 3: Rating of the ANCP's progress towards its objective Objective Previous Rating Support accredited ANGOs to implement their own programs and strategic directions consistent with the Australian aid program’s strategic goals and objectives of poverty alleviation and sustainable development - Green Current Rating Green Note: Reporting period: 2013-14, but results are drawn from 2012-13. Green. Progress is as expected for this point in time and it is likely that the objective will be achieved. Standard program management practices are sufficient. Amber. Progress is somewhat less than expected for this point in time and restorative action will be necessary if the objective is to be achieved. Close performance monitoring is recommended. Red. Progress is significantly less than expected for this point in time and the objective is not likely to be met given available resources and priorities. Recasting the objective may be required. Analysis of data derived from ANGO Performance Reports in 2012-13 indicates the ANCP is beginning to address new targets set for the aid program and delivering tangible results. For example: Promoting prosperity – at least 24% of ANCP funding was invested in projects that promote aidfor-trade Reducing poverty – 35% of ANCP funding was invested in projects that promote economic growth Ensuring value for money – NGOs exceeded their match obligations of 1:5 by contributing 30 cents for every dollar provided by DFAT. Each priority area below provides a snapshot of the ANCP’s aggregate results and examples of good practice by individual ANGOs. However there are gaps in beneficiary data as disaggregation of data by sex, age and disability remains difficult for ANGOs to provide systematically. DFAT has stipulated that for 2013-14 performance reports and 2014-15 annual 14 development plans , data disaggregation is compulsory. 12 Review of ANCP MELF Dr Linda Kelly, Praxis Consultants (March 2013) http://aid.dfat.gov.au/Publications/Pages/ancpmelf-report.aspx 13 Recognised Development Expenditure (RDE) Assessment under ANCP by Geoff Simkiss, Peekay Consolidated P/L. RDE is a measure of Australian’s support for NGOs’ overseas development assistance, and is applied to ANGOs that participate in the ANCP. 14 ANCP annual development plans are ANGOs proposed projects for the coming financial year. Aid Program Performance Report 2013-14 5 Aid priorities Supporting gender equality and the empowerment of women and girls 19,746 women survivors of violence received services such as counselling, compared to 9,243 in 2011-12 and 966 in 2010-11. 48,918 women were able to access training and other support services to enable them to participate in governance and other decision making processes at the community and/or sub national level. The ANCP accreditation criteria assess the ability of ANGOs to demonstrate policies and practices of sound development principles including gender equality. The ANCP Monitoring Evaluation and Learning Framework (MELF) include report templates that contain up to 90 indicators of which 16 relate to gender equality. In 2012-13, there were 4 million beneficiaries across the 16 gender equality indicators. However weaknesses were identified with the MELF report templates as they did not capture the full extent of ANCP projects that addressed gender equality. Only 37 per cent of projects specifically referenced gender equality and women’s empowerment. While a number of ANCP NGOs specifically focus their programming on the empowerment of women and girls including CARE Australia, International Women’s Development Agency, Marie Stopes International Australia, and Plan International Australia, there are many other ANGOs that ensure gender equality is addressed across their programs. Working with the People’s Council in Hai Duong province in Vietnam, Australian People for Health and Education & Development Abroad and its partners were able to incorporate gender equality into all council deliberations and activities, empowering women and giving them a greater voice in government. In Solomon Islands a small solar lights activity implemented by Anglican Overseas Aid improved the livelihoods of women’s groups in remote communities. Women’s groups sold solar lights and used their profits to pay for household needs. Other benefits also emerged, such as a decrease in household tension: there was no longer a need to buy kerosene, household tasks could be continued into the night, and there was added safety from carrying a light. Investing in infrastructure that enables private sector and human development, and working with the private sector 770,000 people (350,646 male; 394,448 female) accessed new or refurbished water supply facilities, compared to 450,000 people in 2011-12. Of the water and sanitation management committees supported, 76 per cent had at least 50 per cent female members. The ANCP supports infrastructure investment and private sector growth, facilitates trade and promotes international competitiveness. In Timor-Leste, WaterAid Australia’s ANCP project supported 954 people in seven rural communities to access new and sustainable water, sanitation and hygiene facilities, with six of these communities declared free of open defecation. CARE Australia used ANCP funds in Papua New Guinea (PNG) in 2012-13 to pilot a Coffee Industry Support Project that is now fully funded by DFAT’s Pacific Women Shaping Pacific Development program15. Increasing coffee production has improved the economic and social security of women involved in coffee production through an innovative partnership developed by CARE with the Coffee Industry Corporation and other private sector stakeholders 15 Pacific Women Shaping Pacific Development: http://aid.dfat.gov.au/countries/pacific/rp/Pages/initiative-pacific-womenshaping-pacific-development.aspx Aid Program Performance Report 2013-14 6 In Cambodia, World Vision Australia established agricultural cooperatives that provided small loans to members. One farmer reported that he was able to invest in cucumber cultivation using new farming techniques which he learned through training at his cooperative. The investment resulted in his family’s income quadrupling, providing the income for his children to attend school. Assisting those who are most vulnerable including ethnic minorities and people with disabilities 121,000 people (68,988 male; 52,281 female) provided with disability services such as access to prostheses and assistive devices, compared to 114,500 in 2011-12. The ANCP delivers programs that support the most vulnerable to become participants in, and contributors to, economic development. There were 48 activities totalling an estimated $8.3 million in 2012-13 that had a primary focus on disability. Field visits by DFAT to a select group of ANCP activities in 2013-14 revealed that ANCP NGOs were actively supporting one another to better identify people with disability in their programming. In 2013 CBM Australia, an ANGO, implemented 28 disability-inclusive development projects in 15 countries reaching a total of 9.9 million direct beneficiaries. Projects were successful in both immediate rehabilitation support and longer systemic change in the lives of people living with disabilities. CARE Australia’s ethnic minority project in Vietnam worked with the Ministry of Agriculture and Rural Development to revise the National Poverty Reduction Program, resulting in 34 ethnic minority representatives (including nine women) actively participating with government policy makers through different workshops and meetings. Supporting the agricultural, fisheries and water sectors 251,000 farmers in poverty (103,263 male; 136,793 female) gained access to and use of agricultural technologies, compared to 195,000 in 2011-12. The ANCP supports small-scale farmers, fishers and entrepreneurs to meet their livelihood and food security needs and improve agricultural productivity. In PNG and Solomon Islands, the World Wide Fund for Nature Australia supported their Western Melanesia Program Office to initiate three-year Inshore Fish Aggregating Device (IFAD) projects to create more accessible fishing sites and secure a good bait supply. The projects included access to micro-finance, business plan development and training opportunities (especially for women), including for the management and maintenance of the IFAD device, which contributed to improving sustainable livelihoods and poverty alleviation in these communities. Providing opportunities for economic development 966,000 people gained access to financial services and the incomes of 172,000 people increased, compared to 110,000 and 116,000, respectively in 2011-12. 40,700 adults with disability successfully obtained a livelihood as a result of skills training or capacity building; and 212,000 people participated in training and vocational training to assist them with access to increased and sustainable livelihoods ANCP supports community economic development and empowerment programs. Assisi Aid Projects facilitated close to 400 micro-loans across Cambodia, India and Timor-Leste to support families to increase their household income and meet basic needs by establishing a small business, or increasing their agricultural activities. World Education Australia Limited partnered with local microfinance institutions in the IndoPacific to improve financial services and consumer livelihoods by growing the capacity of both the institutions serving the market and the clients they serve. More than 5,900 clients attended workshops on: financial literacy, debt management, small business management, savings, and sustainable agriculture. One example is a woman in Nepal who received a micro-loan to Aid Program Performance Report 2013-14 7 purchase a sewing machine and establish a tailoring business. Enough money was saved to repay the loan and apply for a larger loan to establish a small grocery business. The woman has saved enough money to install a solar home system giving her family affordable light year round. Focusing on education and health 285,000 children (146,087 boys; 138,882 girls) in poor and disadvantaged communities accessed child protection services and child friendly spaces, compared to 92,000 in 201112. 1 million people (353,592 male; 493,935 female) benefitted from integrated community approaches to health and well-being, compared to 780,000 in 2011-12. 690,000 people in poverty (286,631 male; 345,633 female) had increased knowledge of hygiene practices, and 1.3 million additional people had increased access to essential medicines and health commodities including HIV treatments. Many ANCP programs are founded on education and health activities for communities to access basic services that will support skill building and improve living standards. Save the Children Australia’s ANCP activities resulted in nearly 2,500 additional children being enrolled in school, 84 teachers being trained, and 14 classrooms being built or upgraded. World Vision Australia supported the enrolment of 11,649 children in school, trained 580 teachers, and provided 4,464 textbooks. The program also resulted in 690 children with disabilities accessing schools, and 318 teachers being trained in disability inclusion. In Nepal, Transform Aid (formerly Baptist World Aid Australia) worked with marginalised communities to raise awareness on a range of public health issues to ensure people had access to accurate health information and were aware of the services available. The activity supported 858 people to receive training in better sanitation and in the construction of their own toilets. The Fred Hollows Foundation’s program in Kenya screened a total of 461,801 people, provided 10,489 health staff medical training, and delivered sight saving antibiotic treatment to 886,270 people to reduce the prevalence of active trachoma. Reducing disaster risks and promoting preparedness 685 communities were assisted through the ANCP to develop community disaster risk reduction plans. ANCP supports the provision of training to communities to raise their awareness of disaster preparedness and promote sustainable practices to overcome challenges such as water scarcity. Habitat for Humanity trained 1,712 people across Asia in disaster resilient construction skills and community based disaster risk management to reduce their vulnerability to the impacts of natural disaster. In Indonesia, Australian Lutheran World Service, focussed on capacity building in 29 communities, with emphasis on disaster preparedness and participatory village development. Communities were supported to build resilience and food security through agricultural training, the establishment of farmer groups and working with communities to safeguard their livelihoods from the impact of disasters and climate change. During the project 5,180 people participated in emergency simulations and 3,761 people participated in climate change and adaptation awareness-raising activities. Aid Program Performance Report 2013-14 8 Mutual Obligations - Partnerships Engaging with ANGOs as partners is integral to the success of the ANCP. DFAT and participating ANCP NGOs work with a variety of partners including other NGOs and community groups, other Australian Government departments, partner governments, academia, professional organisations, and the private sector. The ANCP and Australia’s aid program benefit from partner buy-in through their experience and knowledge to contribute to the sustainability of an activity. DFAT and ANCP NGOs also benefit from engagement with the Australian Council for International Development (ACFID) and several of its working groups, including the Development Practice Committee. DFAT’s partnership with ACFID16 supports policy dialogue, consultation, learning and development and standard setting. All ANCP funded ANGOs are members of ACFID and signatories to ACFID’s Code of Conduct. DFAT and ANGOs ANGOs participating in the ANCP must meet a rigorous set of accreditation criteria. DFAT organises a team of independent reviewers to assess an ANGO’s governance and organisational structures, financial systems, program management capacity, partner management, links with and support from the Australian public, and risk management. To maintain accreditation, ANGOs must be re-accredited every five years, and be available for DFAT spot checks and a rolling program of audits in the intervening period. They must also adhere to DFAT’s policies and guidelines including child protection, fraud and gender equality. Accredited ANGOs enter into head agreements and contracts that outline terms and conditions of the funding they receive under the ANCP. ANGOs are required to submit an annual development plan in June to DFAT outlining their proposed ANCP activities for the following financial year. ANGOs are also required to provide annual performance reports in September covering the previous financial year. DFAT and ANGOs participate in several committees under the ANCP. The Committee for Development Cooperation (CDC) is a joint DFAT/NGO advisory body that considers the accreditation process as well as broader ANCP policy and program matters and has been in operation since the ANCP was established 40 years ago. The Monitoring Evaluation and Learning reference group, established in 2013, provides technical advice on activities under the MELF. The ANCP Partner Agency Collaboration (APAC), also established in 2013, facilitates collaboration and shared learning among Partner ANGOs and complements the work of other organisations such as ACFID. There are strategic partnerships between DFAT and ten of Australia’s largest development NGOs, with the most recent Memorandum of Understanding signed in 201317. In 2013-14 the Partnerships received a total value of 75 per cent of ANCP funds (totalling $98.1 million). These strategic Partnerships promote policy dialogue, enable the Government and ANGOs to share lessons, and jointly identify the most effective ways to help people overcome poverty. The Foreign Minister and Parliamentary Secretary have met with several partner Chief Executive Officers highlighting the importance of the ANCP program to the Government. ANCP NGOs were included in the Parliamentary Secretary’s benchmark consultations in January 2014, from which an informal working group was established meeting with DFAT Executive in advance of the aid policy launch on 18 June 2014. DFAT- ACFID Partnership Memorandum of Understanding 2013 -2016 Partners NGOs include: World Vision Australia, Oxfam Australia, Caritas Australia, ChildFund Australia, Plan International Australia, CBM Australia, CARE Australia, TEAR Australia, The Fred Hollows Foundation and Save the Children Australia. 16 17 ANCP Aid Program Performance Report 2013-14 9 ANGOs and Partner Governments ANCP NGOs have constructive engagement with Partner Governments at all levels. This has been beneficial for the partnerships and community outcomes. For example, the Nepalese Ministry of Education cited the Australian Himalayan Foundation’s flagship Teacher Training Quality Education Program as an example of national best practice, ; and Burnet Institute was awarded the Labour Medal by the Prime Minister of Laos in 2013 for its work under ANCP in Savanakhet Province, Laos. Longer-term partnerships may result in ownership by partner governments of activities originally funded through ANCP. In Bangladesh, the government volunteered for the first time to fund the replication of The Fred Hollows Foundation’s (FHF) eye health model in a new district hospital. While in Vietnam, the government agreed to fund and expand a provincial eye hospital started by FHF to ensure sustainability of the ANCP activity. Program Quality and Management Major reforms have been introduced over the last two years to the ANCP to improve client service delivery and stakeholder relations, program effectiveness and demonstration of results, and program and risk management. This included updates to accreditation criteria and the MELF; and the transition of both functions to an online management system. DFAT actively consulted ANCP NGOs during this process. The ANCP program cycle, as seen in the diagram 3, has a robust entry and planning stage. It also has a prominent system to capture ANGO program data through annual development plans, performance reports and evaluations. The quality assurance stage of ANGO performance reports requires strengthening. The capacity to generate, verify and use evidence-based performance information could be enhanced with the introduction of a performance assessment framework. Diagram 3: ANCP program cycle Accreditation The ANCP accreditation process is known for being both rigorous and cumbersome, often making it difficult for some ANGOs to access funding. During the past 12 months, DFAT has Aid Program Performance Report 2013-14 10 updated the accreditation criteria to align with DFAT’s due diligence framework and incorporate effectiveness principles drawn from DFAT’s effectiveness assessment tool. Accreditation has also been put online to streamline the application and assessment process, removing duplication and providing clearer guidance for ANGOs. Four ANCP NGOs due for re-accreditation volunteered to pilot the new online system at the end of the 2013-14 before plans to go live in July 2014. The ANGOs found the new application system clear and easy to use. Monitoring Evaluation and Learning Framework Developed in 2012, the ANCP MELF enables consistent and transparent reporting on the development impacts of Australian aid program funding of ANCP NGOs as they are now able to report against up to 20 core performance indicators and 70 non-core performance indicators18. DFAT now has access to aggregated program data which was not available two years ago. The improved quality of data and analysis, gathered through the MELF has provided the Minister, the Parliamentary Secretary, DFAT Executive and external stakeholders such as ACFID, with robust and measurable achievements of the ANCP. The MELF includes a learning component with biennial thematic reviews and meta-evaluations. Thematic reviews provide an opportunity to explore particular topics on development effectiveness in greater depth and demonstrate impact of ANCP against sectoral themes. Metaevaluations focus on the quality and range of outcomes in the implementation of ANCP activities as well as lessons learned and impact. In 2013, a meta-evaluation was carried out to assess evaluations of ANCP programs in Cambodia over the past three to five years. Findings revealed that while there was adequate evaluation and reporting of ANCP activities there were some gaps in quality. Learning events held in December 2013 with members of ANCP’s Monitoring Evaluation and Learning Reference Group19 determined that ANGOs can do more to systematically capture gender equality in all aspects of the program cycle – from design to evaluation. In line with the new aid policy, there was agreement to re-focus on gender equality, particularly women’s leadership, and women’s roles in decision-making, stopping violence against women and girls, and women’s economic empowerment. The evaluation findings were also presented by DFAT and the ANCP NGO, International Women’s Development Agency, at the 2013 ACFID Universities Linkages conference. Monitoring and evaluation visits ANCP has an annual program of in-country visits to monitor effectiveness, compliance and alignment with government priorities. In 2013 DFAT made five in-country monitoring visits and found that ANGO projects were reaching the most vulnerable in communities, often through innovative public and private partnerships and demonstrating strong results20. The visits noted, however, that ANGOs could improve Australian aid branding, enhance risk analysis, strengthen the evidence base to support outcomes and increase coordination among ANCP-funded ANGOs in-country. Internal analysis in 2013 on the utility of these visits noted that while terms of reference were developed and shared with the relevant ANCP NGOs and DFAT Posts to provide a framework for the visit, there was not a uniform process for how these visits were prioritised or undertaken. The visits did provide opportunities for increased engagement with DFAT Posts on the ANCP and with NGOs in-country, enabling discussions on the operating environment and issues of mutual interest. Although there have been a number of reforms to the ANCP, some of the management consequences from the 2012-13 APPR were not completed including the development of a 18 NGOs report against the indicators that are relevant to their programming. The Monitoring, Evaluation and Learning Reference Group comprises of representatives from 20 of the 43 ANCP NGOs. 20 2013-14 In-country ANCP project visits included: Thailand, Ethiopia, Peru, Nepal, and Zimbabwe. 19 Aid Program Performance Report 2013-14 11 performance assessment framework. The delay in developing a performance framework was due to the policy environment highlighted in the context section of this report. However there is scope to improve the ANCP program to meet the Government’s new aid policy objectives, in particular working with effective partners. Despite not addressing some of the activities planned for 2013-14, DFAT was able to maintain a reasonable level of engagement with and provide guidance to the ANGO sector. According to findings from an ANCP survey conducted in April 2014, over 96 per cent of ANGO respondents found the quality of the engagement with DFAT overall in 2013-14 to be good or very good; over 65 per cent found the ANCP Online Manual to be useful or highly useful; and over 73 per cent found that the training sessions introducing the online system to be useful or highly useful. ANGO respondents did provide suggested improvements to the report templates, online manual and webinars such as introducing automatic calculation fields, providing examples of good practice and sorting out technical ‘bugs’ to increase efficiencies. ANGOs also suggested DFAT provide a rolling training program on reporting templates, Recognised Development Expenditure (RDE)21, accreditation and how to use the online system in light of the staff turnover in both DFAT and ANGOs. Analysis of Quality at Implementation Reports The ANCP continues to deliver results against its overall objective and is well placed to support the Government’s new aid policy and effectively contribute towards the development priorities of Australia’s aid program. The ANCP reflects Australia’s commitment to being a responsible member of the international community . An internal Quality at Implementation (QAI) review of the ANCP is undertaken annually to assess how well the program is performing. QAI ratings remained consistent from 2012 to 2013 with the efficiency criterion increasing from 4 (adequate quality) to 5 (good quality). The introduction of an online reporting system has directly translated into real gains for ANCP NGOs and DFAT and has been a key success for the ANCP in 2013-14. It has streamlined ANGO reporting, improved access to data, and provided greater capacity for data analysis. The transition enabled ANGOs to be paid two to three months earlier than in 2012-13 and, with significant savings in time, allowed DFAT staff to improve the quality of ANCP programming and progress other reforms. The large increase in staff efficiency has been coupled with positive feedback from ANGOs who have appreciated the shift to the online system. According to findings from the ANCP survey, approximately 70 per cent of respondents found ANCP Online easy or very easy to use. More than 90 per cent of ANGOs submitted their reports by the submission deadline, compared with 58 per cent the previous year. The QAI results indicated that while the program was largely of adequate to good quality, there were areas for improvement: The accreditation system demonstrates a rigorous selection process for ANGOs to participate in the Australian aid program. An independent examination of recognised development expenditure in 2013 confirmed that it is still a relevant method to measure Australian community support; however, its funding allocation methodology raised issues of transparency and equity. ANGO performance reports capture the lessons learnt but do not articulate how ANGOs will take these lessons, evaluation findings and results forward for more effective programming. 21 RDE is a measure of Australian’s support for NGOs’ overseas development assistance, and is applied to ANGOs that participate in the ANCP. Aid Program Performance Report 2013-14 12 The ANCP considers gender equality through the accreditation process and to an extent in ANGO reporting but there is no coherent gender equality strategy for the program. Risks Table 3: Management of key risks to achieving objectives Key risks What actions were taken to manage the risks over the past year? What further actions will be taken to manage the risks in the coming year? The absence of a program logic and performance assessment framework for the ANCP makes it difficult to articulate and measure outcomes of the program. ANCP NGOs must pass a rigorous accreditation process and meet contractual obligations: e.g. annual development plans and performance reports including a financial acquittal. A program logic and performance assessment framework will be developed in 2014-15 to align with the aid policy. A fluid staffing environment impacts the management of the program and engagement with ANCP NGOs. ANGOs are regularly notified through existing committees22 and email bulletin of changes to staffing and program arrangements. DFAT will continue to conduct regular updates for ANCP NGOs on program and contract requirements via webinars, email and in person. DFAT will reconsider certain activities if resourcing remains a challenge. Budget reductions to the aid program present challenges for ANGOs receiving funding from and seeking entry to the ANCP. Budget updates and new accreditation application guidelines shared with the ANGO sector. The program logic and ODE evaluation of ANCP will review mechanisms for entry to the ANCP and its funding model. Risks are reviewed between DFAT and ANCP NGOs on an ad-hoc basis. DFAT conducted in-country monitoring visits to ANCP projects. A Child Protection stakeholder survey in November 2013 highlighted that many ANGOs were still having difficulty implementing DFAT’s Child Protection Policy. DFAT responded by conducting a Child Protection policy webinar in December 2013 to address questions from ANGOs. A webinar on fraud was also conducted. DFAT to continue in-country monitoring visits to ANCP projects. Also, in response to ANCP NGO interest in receiving updates on a range of DFAT policies, DFAT will coordinate a schedule of webinars with ACFID. Upcoming webinars will include disability, environment and anti-terrorism. Refresher webinars and regular meetings will assist ANGOs to comply with policy and contractual obligations. Management Responses The ANCP continues to be a relevant, effective program that supports ANGO community development programs. The remaining program management challenges identified in the 201213 APPR include development of a program logic and strengthening systems to link performance to funding. Recognising that adequate resources will be needed to continue to improve the program, some activities may be prioritised over others. In 2014-15 we will aim to: 1. increase visibility of the ANCP across DFAT and linkages with Posts. The ANCP provides an excellent public diplomacy opportunity for DFAT and Heads of Missions to engage with NGOs in-country and for our overseas missions to draw on NGO expertise to develop or refine policy and country strategies. DFAT will develop an engagement strategy to raise the profile of the ANCP across the department and strengthen linkages with Posts. Interested Posts may consider identifying an ANCP focal point among staff to support project monitoring and NGO engagement. ANCP can also be considered an important complement to our Direct Aid Program (DAP)23, providing best practices in community Existing committees include: Committee for Development Cooperation (CDC), Monitoring Evaluation and Learning Reference Group and ANCP Partner Agency Collaboration (APAC). 23 The Direct Aid Program (DAP) is a flexible small grants program funded from Australia’s aid budget and managed through 66 of the department’s overseas posts. For further information on DAP visit DFAT website: http://www.dfat.gov.au/direct_aid_program/ 22 Aid Program Performance Report 2013-14 13 development and sharing effectiveness assessment tools for use during the project management cycle; 2. strengthen the MELF to reflect the priorities of the Government’s aid policy and performance benchmarks, with particular emphasis on gender equality and private sector engagement; and quality assurance of annual ANCP performance data. To improve data quality DFAT has updated the ANCP report templates for 2014-15 to include a mandatory marker for ANGOs to explicitly state if and how a project addresses gender equality and private sector engagement. 3. develop a program logic for the ANCP and associated performance assessment framework which will be informed by the 2014-15 Office of Development Effectiveness (ODE) evaluation of the ANCP24; and 4. continue a program of in-country monitoring and engagement visits and develop standard terms of reference for field visits to verify ANCP NGO performance and effectiveness. 24 http://www.ode.dfat.gov.au/publications/evaluations-plan.html Aid Program Performance Report 2013-14 14 Annex A Progress in addressing 2012-13 management responses Management consequences identified in 2012-13 APPR Rating Continue to reform the ongoing monitoring and evaluation of ANGO/CSO activities by finalising the Effectiveness Assessment Methodology (EAM). Improve the accreditation process for ANGOs by: implementing the Due Diligence Standards and reviewing the ANCP accreditation framework to align with them including the accreditation process in ANCP Online to improve efficiency and complete the program management cycle (application, planning and reporting). Progress made in 2013-14 Partly Achieved Achieved Progress the development of a Performance Assessment Framework (PAF) based on the findings of the ODE Evaluation of the ANCP. Not achieved Work with ANCP Partner ANGOs to develop a supplementary performance monitoring tool to demonstrate their development effectiveness under the Partnerships MoU. Partly achieved Provide ongoing opportunities to collaborate with ANCP NGOs on shared learning and innovation, particularly on gender equality. Achieved Work on the development of an effectiveness methodology was slower than expected. Solomon Islands program piloted the effectiveness criteria in a new competitive grants program. Work to date will be included in the development of the Department’s definition and methodology for identifying and assessing effective partners as outlined in Performance Benchmark 7: working with the most effective partners. Accreditation Online went live in July 2014 which will improve the efficiency of the application and assessment process; The ODE Evaluation and program logic development will consider entry into the ANCP. Work on the performance assessment framework was interrupted due to the launch of the new government’s aid policy and DFAT integration but will be considered in the context of work in 2014-15 on the ODE Evaluation and program logic development. Internal staffing changes have meant work was slower in this area than expected with minimal progress made. A learning event was held with Partner ANGOs in June 2013; and there were three learning events associated with the 2013 ANCP meta-evaluation. Note: Achieved. Significant progress has been made in addressing the issue Partly achieved. Some progress has been made in addressing the issue, but the issue has not been resolved Not achieved. Progress in addressing the issue has been significantly below expectations Aid Program Performance Report 2013-14 15 Annex B Quality at Implementation ratings Gender Equality Sustainability Monitoring and Evaluation Efficiency QAI year Relevance Effectiveness A Quality at Implementation (QAI) review of the ANCP is undertaken annually to assess how well the program is performing. This section compares the results of the 2012 and 2013 QAI reports. 2012 5 4 4 4 4 4 2013 5 4 5 4 4 4 Definitions of rating scale: Satisfactory (4, 5 and 6) = 6 = Very high quality = 5 = Good quality = 4 = Adequate quality, needs some work Less than satisfactory (1, 2 and 3) = 3 = Less than adequate quality; needs significant work = 2 = Poor quality; needs major work to improve = 1 = Very poor quality; needs major overhaul Aid Program Performance Report 2013-14 16 Annex C Evaluation and Review Pipeline Planning List of evaluations completed in the reporting period Name of Investment Aidworks number Name of evaluation Date finalised Date Evaluation report Uploaded into Aidworks Date Management response uploaded into Aidworks Published on website 2013 ANCP MetaEvaluation 13B270 2013 Meta-evaluation of 9 evaluations of ANCP activities in Cambodia March 2014 March 2014 4 July 2014 4 July 2014 List of evaluations planned in the next 12 months Name of Investment Aidworks number Type of evaluation Purpose of evaluation Expected completion date The Office of Development Effectiveness (ODE) Evaluation of ANCP. N/A ODE To evaluate the ANCP as a mode to assist ANGOs to reduce poverty and support sustainable development in developing countries. May 2015 2014 ANCP Thematic Review: Gender equality and women’s empowerment (Timor-Leste and Vanuatu). 13B270 Thematic Review Provide an insight into the way in which ANGOs are addressing gender equality and women’s empowerment in ANCP supported projects in TimorLeste and Vanuatu. Identify opportunities for shared learning on how ANCP activities can continue to better target gender equality and women’s empowerment. March 2015 Aid Program Performance Report 2013-14 17 Annex D Interim Performance Benchmarks 2014 -1525 (to be finalised in consultation with ANCP NGOs and in line with development of ANCP program logic) Aid objective 2014-15 benchmark Rationale for selecting this performance benchmark ANCP program and activities consider gender equality and women’s empowerment. 60 per cent of activities funded under ANCP address gender equality and women’s empowerment. 2014 ANCP Thematic Review on Gender Equality and Women’s Empowerment completed and documented. Gender equality is central to inclusive development and is a high priority for the Minister. This benchmark was chosen because improving attention to gender has been identified in previous program reviews and in the 2013 ANCP meta-evaluation. As the 2012-13 performance data indicated 37 per cent of activities addressed gender equality, and noted limitations in the MELF report templates, 60 per cent was chosen as an achievable target for 2013-14 performance data. Improvements to ANCP reporting templates and findings from the 2014 ANCP Thematic Review will enable this target to be achieved. Improved policy dialogue with ANCP Partner NGOs to shape program priorities. Partner NGOs via APAC engaged in at least three policy discussions across the aid program. The Partnerships MOU with ten ANCP NGOs commits DFAT to work with the Partners on policy development and engagement at various levels. The Partners receive the large majority of ANCP funding and are well placed to engage at a strategic level across the aid program. Similar to the previous year, it is also anticipated that there will be events for ongoing learning and innovation open to all ANGOs funded under the ANCP. Reorient the ANCP to ensure the program can measure progress on the Australian government’s new development priorities of sustainable economic growth and poverty reduction. ANCP program logic and performance assessment framework documented; DFAT management response to ODE Evaluation of ANCP documented. The Government’s policy on working with the most effective partners makes clear the link between ensuring partners are aligned with Australia's strategic objectives and delivering results. The ODE Evaluation and the development of a program logic and performance assessment framework will ensure the ANCP is well placed to fulfil these commitments. ANCP program and activities promote prosperity Maintain 20 per cent of ANCP focus on promoting prosperity In 2012-13 at least 24 per cent of ANCP funding was invested in projects that promote aid-fortrade. It is anticipated that a minimum of 20 per cent of activities promoting prosperity will be an achievable target for 2013-14 performance data. ANCP program and activities reduce poverty Maintain economic growth/poverty reduction objectives under the ANCP In 2012-13 35% of ANCP funding was invested in projects that promote economic growth through economic development activities. It is anticipated that 2013-14 performance data will maintain this type of activity. 25 Technical notes for Performance Benchmarks are currently under development. Aid Program Performance Report 2013-14 18 Aid objective 2014-15 benchmark Rationale for selecting this performance benchmark ANCP program and activities engage the private sector26 Strengthen the MELF to reflect a minimum of 20 per cent in private sector engagement in the ANCP. This is a new focus under ANCP reporting; however, ANGOs engage widely with the private sector in their projects. To improve data quality DFAT has updated the ANCP annual development plan to include a mandatory marker for ANGOs to explicitly state if they work with or partner with the private sector. Indicative planning for 2014-15 notes that 29 per cent of investments will promote private sector growth or engage the private sector (by number of projects). It is anticipated that 2013-14 performance data will reflect a minimum 20 per cent in private sector engagement. ANCP program and activities ensure value for money Maintain ANCP policy of a minimum of 1:5 match contributions to leverage additional funding for ANCP activities. Total number of projects funded under the ANCP reduces by 5%. In 2012-13, ANCP NGOs contributed 30 cents for every dollar provided by DFAT. It is expected that ANGOs will be able to confirm this match contribution in their 2013-14 acquittals. ANGOs have been encouraged to undertake strategic analysis of their ANCP-funded activities, reduce the number of projects and shift to programs for greater cohesion and to reduce administrative costs. 26 Private Sector Partnership – ANCP Reporting: The Private Sector comprises all privately owned commercial enterprises and includes individual farmers and street traders, small and medium enterprises, large locally-owned firms and multinational corporations. Also includes enterprises in developing countries, developed countries, and multinationals. Partnering with the private sector means working more to support private sector growth and working with the private sector to achieve broader development outcomes. In some cases this will mean directly partnering with businesses. In many cases it will mean supporting the development of laws, regulations, institutions, infrastructure, education and governance that support the growth of a vibrant and diverse private sector. Aid Program Performance Report 2013-14 19