Annual Review - Summary Sheet
This Summary Sheet captures the headlines on programme performance, agreed actions and learning over the
course of the review period. It should be attached to all subsequent reviews to build a complete picture of actions
and learning throughout the life of the programme.
Title: Promotion of Basic Services Programme Phase III (PBS)
Programme Value: £388.5m
Programme Code: 202991
Review Date: 20.10.14-07.11.14
Start Date: 12/2012
End Date: 12/2017
Summary of Programme Performance
Year
2013
2014
Programme Score
A+
A
Risk Rating
Medium
High
Acronyms
AfDB
AMIS
BoFED
CA
CDP
COPCU
COST
CSA
DA
DAG
DFID
DHS
DP
EFY
EIO
EMIS
ERA
ERAMIS
ESAP
ESIA
EU
FBG
FPPPA
FTA
GDP
GoE
GRM
GSEE
GTP
HEW
HMIS
HR
African Development Bank
Agriculture Management Information System
Bureau of Finance and Economic Development
Continuous Audit
Commune Development Programme
Channel One Programmes Coordination Unit
Construction Sector Transparency Initiative
Central Statistics Agency
Development Agent
Development Assistance Group
Department for International Development
Demographic and Health Survey
Development Partners
Ethiopian Financial Year
Ethiopian Institution of the Ombudsman
Education Management Information System
Ethiopian Roads Authority
Ethiopian Roads Authority Management Information System
Ethiopian Social Accountability Programme
Environmental and Social Impact Assessment
European Union
Federal Block Grant
Federal Public Procurement and Property Administration Agency
Financial Transparency and accountability
Gross Domestic Product
Government of Ethiopia
Grievance Redress Mechanism
Gender and Social Exclusion in Ethiopia
Growth and Transformation Plan
Health Extension Worker
Health Management Information System
Human Resource
1
IBEX
IEG
IFR
IT
JBAR
JPA
JRIS
M4R
MDGs
MDTF
MEFF
MoFED
MTR
NER
OFAG
ORAG
P4R
PAC
PAD
PBS
PDO
PEFA
PFM
PGHO
PP
PSIA
REF
SAC
SAIP
SBA
SDI
SNNPR
SPA
TWG
UK
VfM
WaSH
WMIS
Integrated Budget and Expenditure
Independent Evaluation Group
Interim Financial Report
Information Technology
Joint Budget and Aid Review
Joint Plan of Action
Joint Review and Implementation Support
Managing for Results
Millennium Development Goals
Multi Donor Trust Fund
Medium term Economic and Fiscal Framework
Ministry of Finance and Economic Development
Mid-term Review
Net Enrolment Rate
Office of Federal Auditor General
Office of Regional Auditor General
Programme for Results
Public Accounts Committee
Project Appraisal Document
Promotion of Basic Services
Project Development Objective
Public Expenditure and Financial Accountability
Public Financial Management
Public Grievance Hearing Office
Partnership Principles
Poverty and Social Impact Assessment
Result Enhancement Fund
Social Accountability Committee
Social Accountability Implementing Partners
Skilled Birth Attendant
Service Delivery Indicators
South Nations Nationalities and Peoples Region
Service Provision Assessment
Technical Working Group
United Kingdom
Value for Money
Water Hygiene and Sanitation
WaSH Management Information system
Summary of progress and lessons learnt since last review.
Results
Overall, this programme has met expectations on most of its results and there is evidence that
the Promotion of Basic Services (PBS) programme is helping Ethiopia make progress towards
its development goals. In 2014 successes include increasing the number of qualified teachers to
258,376 against a target of 215,000; posting budget information in 96% of Ethiopia’s districts
against a target of 90%; and expanding social accountability to 223 districts, exceeding the target
of 150. Over the last year the Promotion of Basic Services (PBS) programme has contributed to
increased utilisation of basic services across the country. For instance the proportion of women
2
delivering with a skilled birth attendant has increased from 23% in 2013 to 40% in 2014 and
exceeding the target of 28%. Net enrolment in upper primary schools has increased from 47.3%
to 48.5%, and just passing DFID’s target of 48%. However it falls short of the government’s
ambitious goal of 80%.
Value for Money: Equity, Effectiveness, Economy and Efficiency
PBS has been found to be pro poor, with the majority of its benefits felt by the poorer sections of
society in the historically disadvantaged areas of Ethiopia. Emerging evidence indicates that
education, health and agricultural extension services are effective. However it also highlights the
uneven distribution of services in the country and the need to review policies to ensure
communities receive the appropriate type and level of support they need to improve their lives.
The programme continues to represent good value for money. Investment in public sector
salaries is considered good value both in terms of economy and efficiency as well as being
highly effective in increasing access to basic services.
Financial Management of PBS
The World Bank’s management of DFID’s funds for PBS is satisfactory with the timely provision
of quarterly interim financial reports and a 2013 external audit report. PBS is audited separately
from the overall government resources by the Office of the Federal Auditor General and was
unqualified. The report confirmed that DFID funds had been used for its intended purpose of staff
salaries and operational costs in the health, education, agriculture, water and rural roads sectors.
However it did identify some financial management weaknesses such as incidences of no bank
reconciliations and mis-postings of expenditures at the district level. Action has been taken on
the findings and regions now refuse to release PBS funds unless bank reconciliations have been
conducted. Additional guidance on expenditure posting has been circulated.
Public Financial Management Reforms
The Public Financial Management Reform component of PBS aims to strengthen government’s
capacity to manage all public funds effectively. This has progressed well with an increased level
of scrutiny at the federal and regional level and the setting up of district audit committees. But
weaknesses persist at the district level with the low level of follow up on audit issues being a
concern. The Ministry of Finance and Economic Development are developing a strategy with the
regions to strengthen internal audit capacity and improve audit follow up. This will be launched
in April 2015.
Management of Risk
DFID continues to monitor and actively manage the risks of PBS including those posed by the
context in which it operates. This includes the possible risk posed by the government’s
Commune Development Programme (CDP) in Ethiopia’s four emerging regions. Actions taken by
DFID include participating in the Development Agency Group’s monitoring visits to CDP sites,
conducting a social assessment and engaging with government to increase compliance with
international good practice in relation to resettlement. In addition DFID is pushing for the fast
tracking of grievance redress mechanism in Developing Regional States and South Omo and for
the scaling up of social accountability activities across the country. However these two services
are still new and require further support to strengthen their capacity, functionality and
effectiveness.
Due these issues, together with the poor level of district follow up on audits, the forthcoming
Ethiopian elections, the uncertain future of the social accountability activities and the nascent
nature of the grievance redress mechanisms it is recommended that the risk level for this
programme is increased from medium to high.
Background
The Promotion of Basic Services programme aims to increase access to and quality of decentralised
services in health, education, water and sanitation, agriculture and rural roads throughout Ethiopia. It
does this through sub programme A which funds public workers’ salaries of the five sectors and through
sub programme B which supports system strengthening activities such as public financial management,
social accountability and grievance redress mechanisms. The programme is implemented and funded by
the Government of Ethiopia with additional funding from the UK, World Bank, European Union, African
Development Bank, Austrian and Italian Governments. DFID’s funding is channelled to the Ministry of
3
Finance and Economic Development through a World Bank Multi Donor Trust Fund, who manages our
money on our behalf. DFID has disbursed £234,712,422 to date out of a total planned allocation of
£388.5m over the programme’s lifetime which ends in March 2018. This is a reduction from its original
commitment of £510m. Along with DFID’s programmes of support in the health, education and water
sectors, this funding is expected to deliver the following results:





Support 2 million children in primary school (of which at least 50% are girls);
Provide 7.5 million people with access to basic health care;
Help half a million women to give birth safely;
Provide an additional three quarters of a million women with access to family planning services;
and
Provide an additional 1.4 million people with clean drinking water.
Key achievements in 2014
PBS is monitored biannually through the Joint Review and Implementation Support (JRIS) missions and
the Joint Budget and Aid Review (JBAR) with progress and deliverables captured in an aide memoire
agreed between government and development partners. These missions took place in May and
November in 2014 and were combined with a Mid-Term Review of PBS.
Key achievements in 2014 include:









Public sector workers providing basic services across the country including 258,376 qualified
teachers, 35,907 Health Extension Workers, and 10,000 agricultural Development Agents; 2,811
of which were trained on high yielding crops;
A Poverty and Social Impact Assessment which shows that PBS is pro poor and benefiting some
of the most disadvantaged areas of the country. It concludes that PBS is making a significant
contribution to Ethiopia’s progress towards its Growth and Transformation Plan and the
Millennium Development Goals;
Satisfactory project financial management of PBS by the World Bank with the timely receipt of
interim financial reports and the 2013 external audit. The audit was unqualified and reported that
DFID’s funds had been used for the intended purpose. It did report some incidences of bank
reconciliations not being carried out and of mis-postings at the district level but these have since
been addressed by government;
Achieving progress for the majority of the deliverables identified in the May 2014 Joint Review
Implementation Support mission as well as passing the tests of Additionality and Fairness and
upholding the PBS core principles of effectiveness, sustainability, equity, transparency, fiduciary
probity and predictability;
Implementing a Mid-Term Review whose initial findings report that PBS is making an important
contribution to good governance by significantly improving citizens’ engagement, financial
transparency and accountability, and public financial management;Improving grassroots
transparency and accountability by training 66,125 people to understand the budget, supporting
96% of districts to post simplified budget information, strengthening grievance redress
mechanisms in 7 regions and helping communities in 223 districts to hold local government to
account for the delivery of basic services;
Progress on public financial management reform with increased scrutiny of federal bodies and
regional governments by the auditor general and parliament and the establishment of audit
committees in 400 districts. The commitment of government to address the low levels of audit
follow up at district level by developing an internal audit strengthening strategy by April 2015;
Improved levels of supervision and a ‘new spirit’ to the Joint Review Implementation Support
meetings. This includes better representation from the regions with their more active participation
in discussions; a greater level of inter government challenge on the rate of progress and agreeing
a shorter, more strategic list of deliverables;
Undertaking a range of analytical studies including a mini Demographic Health Survey; a Service
Delivery Indicator study on education; a Service Provision Assessment on health; a costing
exercise of basic services to inform a sustainability strategy for PBS; and a comprehensive value
for money assessment; and
Assessing, mitigating and monitoring the risks of PBS and the context in which it works. This
included commissioning a social assessment; undertaking a study to identify how GRM can be
strengthened and the expansion of social accountability activities. In addition DFID has
4
participated in the DAG monitoring missions to resettlement sites in Somali region and SNNPR
and plans to participate in a mission to Gambella in late December.
A key development this year has been the reduction in the UK’s contribution to PBS including the
cancellation of the Results Enhancement Fund and changes to the evaluation plans for the programme.
These changes were made to adopt a more sustainable approach to PBS that encourages the
Government of Ethiopia to take responsibility for an increasing proportion of recurrent costs as their own
domestic budget and tax revenues increase. It also recognised the progress already being made in
terms of the Government of Ethiopia being able to take on more of this kind of expenditure, and it
allowed DFID to increase its economic development programming.
Summary of recommendations for 2015
The November 2014 Joint Review and Implementation Support meeting, confirmed that all of the PBS
tests and core principles had been passed and that overall there was good progress. Development
partners highlighted a number of areas for government attention. These were: (i) slow progress in
establishing management information systems in the sectors, especially water and agriculture; (iii)
the possibility of reallocating unutilised funds from areas of slow progress to other activities under
sub programme B; (iv) strengthening procurement capacity at district level and establishing the
Procurement Training Certification Programme; (v) strengthening citizens’ engagement in budget
discussions with local government; (vi) developing a roadmap for scaling up social accountability
activities; (vii) opening of additional branch offices of the Ethiopian Institute of the Ombudsman in
Afar and Benishangul Gomez to ensure all regions have independent grievance redress
mechanisms; and (viii) strengthen district’s audit capacity and follow up on audit recommendations.
Government and development partners agreed a total of 40 new priority actions to be delivered by the
April 2015 Joint Review and Implementation Support meeting. These are recorded in the November
2014 Joint Review and Implementation Support mission aide memoire. They cover a range of topics
including:









Continuing to uphold PBS’s core principles – undertaking a domestic resource mobilisation study
and a nationwide cost of service delivery estimation exercise to inform the sustainability of PBS;
Social accountability – agreeing on the future of the social accountability programme including
approaches to sustainability and scaling up;
Financial Transparency and Accountability – disseminating audit and procurement templates to
regions and districts; providing Budget Literacy Training; and providing evidence for financial
transparency and social accountability linkage activities;
Grievance Redress Mechanism – delivering the final report of the grievance redress mechanism
study; providing training for Public Grievance Hearing Officers at regional, district and village
level; identifying areas of support for the Ethiopian Institute of the Ombudsman and regional
Public Grievance Hearing Offices; and enhancing accessibility of the grievance redress system;
Public Financial Management reform– strengthening dialogue on public financial management of
all of government’s resources; strengthening the internal audit system and audit follow up at
district level; implementing plans for financial training; rolling out a new version of the Integrated
Budget and Expenditure system to districts; and supporting the professionalization of the
accounting and auditing professions;
Procurement - finalising procurement performance indicators and an implementation plan;
resolving technical problems with the Federal Public Procurement and Property Administration
Agency’s website; following up the study to identify gaps in district procurement systems; hiring
an independent procurement auditor; and reporting on the status of agreed audit action plan;
Managing for Results – finalising a work plan until 2017 to prioritise activities; strengthening
management information systems in education, water and agriculture; progressing with Ethiopia
Roads Authority’s baseline data collection; and preparing an analytical report on gender
disaggregation of results;
Effectiveness of PBS – follow up with government on how the findings from the education, health
and DA studies will be used to maximise the effectiveness of investments made through PBS
and sector programmes. In particular ensuring that basic services such as DA advice is available
to resettled communities;
Environmental and Social Impact Assessment– training regional and district staff on how to
conduct these assessments and finalising the PBS Social Assessment; and
5

Project financial management of the PBS programme- reporting on progress made on financial
management action plans of PBS supervision reports; improving the budget utilisation of PBS’s
sub-programme B; and continuing with PBS’s external/continuous audit reporting arrangements.
6
A. Introduction and Context (1 page)
DevTracker Link to Business Case:
DevTracker Link to Log frame:
http://iati.dfid.gov.uk/iati_documents/3855818.doc
http://iati.dfid.gov.uk/iati_documents/4491779.xls
Outline of the programme
Originally, DFID agreed to provide up to £510 million over five years to ensure continued access and
improvement of decentralised basic services for education, health, water and sanitation (WaSH),
agriculture and rural roads in Ethiopia as part of the Promotion of Basic Services phase III Programme
(PBS). However, the budget was reduced in 2014 to £388.5m in order to make funds available to
support Ethiopia’s economic development
and to facilitate a
transition to the Table 1: DFID contributions to PBS: Original and revised
Government of Ethiopia (GoE) having
Original
Revised
responsibility for its recurrent costs, as
£m
%
£m
%
highlighted in DFID’s 2014 Country Poverty Sub programme A:
425.0
83
353.9
91
Reduction Diagnostic. The reduction was Federal Block Grant
Sub programme B:
25.0
5
25.0
6
achieved by reducing DFID’s contribution to Systems Strengthening
sub programme A from £425m to £353.9m Result Enhancement
50.4
10
0
0
and by not activating the £50.4m allocation Fund
Management Fee
9.6
2
9.6
3
to the Results Enhancement Fund (REF). Total
510.0
100
388.5
100
Contributions to sub programme B of £25m
for systems strengthening and management
costs of £9.6m remained unchanged (see Table 1).
The expected impact of PBS is faster progress towards the health, education, water and poverty MDGs
through improving access to and quality of basic services. This will be achieved through two sub
programmes (A&B). Sub programme A increases the availability of qualified staff and operational
resources for delivery of basic services by contributing to the salaries of public workers in the five sectors
through the Federal Blocked Grant (FBG) which is allocated to regions and districts. Sub programme B
is focused on systems strengthening and has three outputs which are improved transparency and
fiduciary assurance regarding the use of public funds; improved citizen engagement on the quality of
basic service delivery; and strengthening M&E system at federal and sub national levels. Activities
include Ethiopia’s social accountability programme (ESAP); financial transparency and accountability
(FTA); grievance redress mechanisms (GRM); public financial management (PFM) and managing for
results (M4R).
Approximately 80% of PBS is funded by the GoE with
the remainder provided by the UK, World Bank, African
Development Bank (AfDB), European Union (EU) and
the Governments of Austria and Italy. The World Bank’s
support to PBS ends in 2015 but they are currently
developing plans to continue their support to PBS with
a Program for Results mechanism (P4R) where funding
will be released when the GoE achieves targets of
Disbursement Linked Indicators (see Box 1).
PBS funding is provided to Ministry of Financial and
Economic Development (MoFED) through a World
Bank Multi Donor Trust Fund (MDTF) that is managed
by a PBS Secretariat. The social accountability element
of PBS (ESAP) is implemented by a consortium of civil
society organisations, headed by a management agent,
VNG International.
Box 1: A Program for Results (P4R) instrument:
World Bank Plans
The World Bank presented plans for a new P4R
instrument to fund PBS to the JRIS in November 2014.
The program will replace the current Specific
Investment Loan that currently supports PBS. The
objective of P4R is to address disparities in human
development indicators between regions and wealth
quintiles, improve the efficiency of programme
expenditure and to achieve lasting results by
strengthening institutions and building capacity. This
development is based on a rationale drawn from the
Project Performance Assessment Report of the World
Bank’s Independent Evaluation Group. Disbursements
will be linked to defined results through a set of
Disbursement Linked Indicators (DLIs).
Discussions in the JRIS reflected interest and support
from regions especially regarding (i) prioritisation of
DLIs; (ii) elaboration of the context of equitable access;
and (iii) assurances that P4R would ensure continuity
of overall support to basic services.
The results expected of PBS are outlined in a PBS
results framework, a sub set of which are used to form
DFID’s log frame. The indicators in the PBS results
framework are set at a number of levels including
higher level results, project development objectives (PDOs) and intermediate results. Progress is
7
monitored throughout the year through a range of forums including Development Partners (DPs) monthly
meetings, a ministerial-level steering group and a number of technical working groups for sub
programme B activities which bring together representatives of MoFED, sector ministries and DPs.
Progress against results is reviewed and updated twice per year through Joint Review and
Implementation Support (JRIS) field missions and meetings, from which an aide memiore is produced.
The aide memoire is agreed by GoE and DPs and contains a list of deliverables or priority actions to be
undertaken in the next 6 months, the progress of which is then reported in the following JRIS.
PBS is a heavily scrutinised programme which has been subject to extensive reviews and analysis of its
various components. As documented in the 2013 Annual Review, there is increased scrutiny of DFID’s
support to PBS arising from allegations that public workers, whose salaries are in part paid for by PBS,
might be involved in activities related to the Government of Ethiopia’s Commune Development
Programme (CDP) that may not conform to international best practice. The World Bank’s Inspection
Panel is investigating whether Bank Management should have triggered and implemented safeguard
policies on involuntary resettlement and indigenous people in PBS. We understand the World Bank’s
Board will meet in early 2015 to consider the report and a management response. Any recommendations
and actions proposed by the World Bank’s Management in response to the Panel’s findings will need to
be approved by the World Bank’s Board and would apply to the whole of PBS, including DFID funds.
This matter is also subject to an ongoing judicial review against DFID brought by the claimant “Mr O”
who, it is stated, is a Gambellan refugee residing in a Kenyan refugee camp. Mr O claims that he, along
with his family and other villagers, was forcibly and violently evicted from his village by the Ethiopian
National Defence Force which, he alleges, oversaw the CDP. He further alleges that DFID, through its
support to the PBS programme, may be contributing directly and/or indirectly to the implementation of
the programme, and therefore that DFID’s development support is potentially being used in a way that is
incompatible with human rights.
DFID regularly assesses, manages and monitors the risks of PBS and the context in which it operates.
Since January 2011, DFID, alongside other Development Assistance Group (DAG) partners, has visited
all four regions where the CDP is operating, to monitor implementation against International Good
Practice Guidelines and Principles Regarding Resettlement. There was no evidence during these visits
that human rights abuses were widespread or systematic. DFID, with the DAG, continues to monitor the
implementation of the CDP against the guidelines and principles. DFID participated in the DAG
supervision missions to SNNPR and Somali region in 2014 and will attend the mission to Gambella in
late December .
B: PERFORMANCE AND CONCLUSIONS (1-2 pages)
The performance of PBS is measured against the PBS results framework agreed in the World Bank
Project Appraisal Document (PAD). The results framework’s targets and milestones are taken from the
government’s Growth and Transformation Plan and sectors’ national plans. These targets are highly
ambitious and based on an assumption of a fully funded programme. The government reports against
the results framework biannually at the JRIS using the most up to date government data. This is then
captured in the aide memoire.
The effectiveness of DFID’s support to PBS is monitored against its log frame that was developed with
the business case. The log frame has taken a sub-set of indicators from the PBS results framework
which has been modified in order to enable DFID to track progress against a more realistic trajectory
than the one set by the government.
Since the finalisation of the PAD, new baseline data has become available and it became clear early in
the year that some indicators need to be revised to account for higher than expected achievement (i.e.
lower primary school enrolment) and that new indicators should be used to monitor areas where
challenges persisted (i.e. upper primary school enrolment). This led to the revision of the results
framework and subsequent revision of the DFID log frame in early 2014. This annual review reports
progress against the DFID log frame milestones. However it also quotes performance against the results
framework’s milestones where relevant.
Annual impact and outcome assessment
8
Overall good progress has been made against PBS’s results at the impact and outcome level of
the logframe. However, some of the results are not directly comparable due to different data
sources and methodologies used for their calculation.
At the impact level maternal mortality per 100,000 live births reportedly fell from 676 in 2011 to 420 in
2014, exceeding the target of 594. However these figures are not directly comparable as the 2011
baseline was measured through a Demographic and Health Survey (DHS) whilst the 2014 value was
estimated by the World Health Organisation based on proxy indicators such as Skilled Birth Attendance
(SBA) and contraceptive rates. Ethiopia conducted a mini DHS in 2014 but due to its sample size could
only measure output level indicators and not calculate infant or maternal mortality rates. A full
comprehensive DHS will be conducted in 2016/17. The primary school completion rate for girls rose
slightly from 52.2% in 2013 to 52.8% in 2014 and was above the target of 51%. Although primary school
completion rate for boys fell slightly from 53.3% in 2013 to 52.8% in 2014, it exceeded its target of 48%.
The percentage of the population with access to potable water rose in both rural and urban areas from
2013 to 2014 and was above target levels. For example, the percentage of the population in rural areas
within 1500 meters of potable water rose from 66.5% in 2013 to 75.5% in 2014 which exceeded its target
of 68%. There is no new data available for the proportion of the population below the poverty line.
There is evidence that PBS is contributing to the expected outcomes being met. Net enrolment
rate in upper primary (grade 5-8) rose from 47.3% in 2013 to 48.5% in 2014. This was slightly above the
DFID log frame target of 48% but well below the target of 80% in the PBS results framework. This overall
trend was seen for boys with a rise in enrolment from 46.5% in 2013 to 49.1% in 2014. However, the
situation worsened slightly for girls from 48.1% in 2013 to 47.9% in 2014. The JRIS noted that regionally
disaggregated data was not available for this indicator, although this should be available soon when the
Ministry of Education finalise their 2014 report. According to data from the Health Management
Information System (HMIS), the percentage of deliveries attended by a skilled birth attendant rose from
23.1% in 2013 to 40.9% in 2014, exceeding the target of 28%. However the mini DHS 2014 reported that
the percentage of deliveries attended by a SBA rose from 10% in 2011 to 14.5% in 2014. The reason for
the discrepancy being that the HMIS includes deliveries performed by Health Extension Workers (HEWs)
whereas the DHS only recognises midwives, nurses and doctors as skilled attendants. HMIS data also
showed that the percentage of children vaccinated with Pentavalent 3 rose from 87.6% in 2013 to 91.1%
but this was below the log frame target of 94%. Performance varies across regions with a dramatic rise
in Afar from 37.4% in 2011 to 99% in 2014, but some regions such as Somali, Gambella and Dire Dawa
lagging behind and missing their targets.
Due to the scale and complexity of the PBS programme and the fact that it is being implemented by 6
government ministries, 10 regional governments and over 900 districts, it is difficult to give a generalised
statement of progress across all sectors at the operational level. Progress against DFID’s logframe
outputs is mixed and describe in detail in the next section. PBS is meeting or exceeding the DFID and
government milestones in many areas such as the number of teachers and district road officials; posting
budget and service delivery information; establishing procurement capacity at district level; introducing
social accountability and grievance redress mechanisms. However other areas are lagging behind in
relation to training development agents in high yielding crops, districts acting on audit recommendations,
and rolling out management information systems in the
Box 2: Examples of effects of social
WaSH, agriculture and rural road sectors.
accountability element of PBS
Data from other sources. There are some other
important data sources on outcomes of PBS. The
Poverty and Social Impact Assessment (PSIA) provides
evidence that district level investment in salaries for
health, education and agriculture workers resulted in
improved uptake of services and this was particularly
seen among the poorest wealth quintiles for health and
education. An increase of per capita district spending of
$1 was associated with an increase in contraceptive
prevalence rate of 6.4%, an 11.3% increase in SBA, a
3.6% increase in antenatal care coverage, and a 3.6%
increase in net primary enrolment rates. Increased
district level spending also increased agricultural outputs
and the greater uptake of improved agricultural methods.
9
In Dawe district in Afar, students and parents raised
concerns about absenteeism of teachers when
collecting their salaries. The district education office
assigned a cashier to pay salaries to teachers on the
spot – reducing absenteeism.
In Mao Komo district in Benshangul Gumuz, dialogue
between citizens and district officials resulted in joint
action to construct more than 13km of road. Village
communities contributed money and the district
administration provided equipment and staff. The
improved road network is reported to have made it
easier for women to deliver at health centres.
In Awash town in Afar, dialogue and site visits have
helped citizens to better understand the problems
faced by the water authority related to water points.
Although the issues are not yet fully resolved, there is
better mutual understanding and regional financial
and water bureau representatives have been invited
to attend the next interface meeting.
The study also showed positive benefits for women of district level spending in health. Although men
benefited more than women from agricultural extension services. Further studies on the quality of
services and their effectiveness are emerging and reported under output 1.
Reports from the management agent show the social accountability element of PBS beginning to have
demonstrable effects in particular districts (see Box 2). Social Accountability activities were taking place
in a quarter of Ethiopia’s districts and 150 districts have developed action plans between the
communities and government to improve services. This exceeded both performance in 2013 (70) and
the 2014 target of 120.
As noted in the last annual report, AfDB’s Innovation in Service Delivery study currently underway is
expected to produce data on the effects of different innovations on a range of outcomes. Data from the
Education Service Delivery Indicators (SDI) and Health Service Provision Assessment (SPA+) studies
will also provide insight into achievement of outcomes. The reports for these studies are not yet available
but the initial findings of the Education SDI study and the methodology for the Health SPA+ study were
presented in the November JRIS. Their reports will be shared by the April 2015 JRIS.
Overall output score and description
A – Met expectations
Key lessons
The PSIA study has provided evidence of the pro-poor value of district level spending from sub
programme A through the Federal Blocked Grant and concludes that PBS is making a significant
contribution to Ethiopia achieving its MDGs - particularly in health and education. This annual review has
highlighted the importance of the sub programme B activities which are important elements of the
programme for increasing the efficiency and effectiveness of the FBG resources.
Equity is one of the core PBS principles. The social appraisal in DFID’s business case suggests that
federal block grants would help improve the access of socially excluded groups to basic services. This
has been supported by the findings of the PSIA that assessed the correlation between increased
regional spending, expansion of basic services and development outcomes. It found that district-level
health and education spending provides proportionately greater returns to marginal groups, with 58% of
spending accruing to the two bottom income quintiles of the population. It also reports that on a regional
level, PBS financially favours Gambella and Benishangul Gumuz – two of the historically most
disadvantaged regions of the country. Similarly it reports that ethnic groups who are likely to be at a
relative historical disadvantage in terms of development receive a significantly higher proportion of
recurrent expenditure. This is especially true for the Anyiwak ethnic group from Gambella who are
allocated Ethiopian Birr 755 (approximately $15) more per capita than larger ethnic groups in Ethiopia.
However, there are differences in how equity is understood within PBS with different definitions being
used in DFID’s business case, the World Bank’s PAD and the recent PSIA. The PAD limits
understanding of equity to regions and gender while the DFID business case identified additional groups
that suffered from unequal access to basic services, e.g. child labourers, disabled people, female victims
of domestic violence, people living with HIV, older people and pastoralists. The PSIA study defined
equity as, income quintile, gender, ethnicity and geography.
In recognition of this, a social assessment is being conducted which will hold in depth consultations with
‘vulnerable and disadvantaged groups’ to ensure the programme is responsive to social development
concerns, including seeking to enhance benefits for poor and vulnerable peoples while minimising or
mitigating risk and adverse impacts. This study will be available in early 2015 and will allow partners to
use a broader and more consistent definition of equity and incorporate actions to mitigate any identified
negative effects into the programme. In addition, PBS is training government officials on Environmental
and Social Appraisal techniques so that district staff can design service delivery and interventions in
ways that meets the needs of their communities.
Key actions
To act on the social assessment’s recommendations and agree on a more comprehensive definition of
equity so that it can be tracked consistently across the programme and through the different analytical
pieces. This should go beyond the narrow PAD definition to include the range of vulnerable groups in the
business case’s social appraisal and the Leave No One Behind post-2015 agenda. There will be a good
opportunity to do this once the findings of the social assessment are available in early 2015 and during
the pre-appraisal design phase of the World Bank’s P4R mechanism for PBS. This joint understanding
10
will then help inform the development of government’s next Growth and Transformation Plan (GTP),
sector policies, risk management and the setting of mitigation measures and disbursement triggers for
future resources.
Has the log frame been updated since the last review?
Yes. The overall PBS results framework was updated in December 2013 and the DFID log frame was
revised in May 2014 to harmonise with this. Changes were extensive and are documented in a formal
World Bank restructuring paper. At the level of education outcome, the indicator was changed away from
net enrolment rate (NER) for grades 1-8 to grades 5-8 as this is where most dropouts occurs. Several
outcome targets were lowered (e.g. for NER and SBA) on the grounds that previous targets were aligned
to national targets which were considered ambitious and based on the assumption of a fully funded
national budget. For example, the 2014 target for NER grades 5-8 is 80% in the PBS results framework
(reflecting the GoE’s GTP target) but only 48% in the DFID log frame as 80% is seen as unobtainable
with the time and resources available.
The outcome indicator for social accountability was changed (from number of people assisted in holding
decision-makers to account to the number of districts that have developed joint action plans for service
quality improvement) on the basis that the former was not measurable given the stage of implementation
of the social accountability work and that the impact evaluation of ESAP will consider this question.
Extensive changes were made to the output indicators of the DFID log frame in May 2014. The wording
and order of the outputs themselves have been changed. Of the 12 original output indicators, only three
remain (related to action on audit recommendations; posting of district budget and expenditure
information; and posting of standardised service delivery templates by basic service delivery units). 18
new output indicators have been added to the log frame.
There are a number of outstanding issues with the log frame:
 For health results, the log frame continues to rely on data from HMIS rather than DHS to measure
progress at the output and outcome level. This is because the HMIS data, which is reported by
service delivery staff from each health facility, is available annually, whereas the population based
DHS surveys are only conducted every 3-5 years. However there is a divergence between health
results reported from HMIS and DHS, with HMIS results being much higher. This discrepancy can be
explained by the different methodologies used, making them not directly comparable and the recall
bias inherent in the DHS. However there are weaknesses in the HMIS as it is still primarily a paper
based system and staff lack understanding and capacity to record the data accurately. Whilst action is
being taken to improve the HMIS through PBS’s Data Quality Assurance Framework and through our
support to the health sector, it is likely that results from HMIS could be overstated;
 Milestones have been changed from a specific date (e.g. 1 July 2014) to just a year (e.g. 2014). While
this maps to the PBS results framework, it does make it less clear as to precisely when particular
targets refer to;
 DFID outputs do not fully match the overall PBS programme components, particularly outputs 2 and
3. In particular, FTA is grouped with PFM in the DFID log frame but with citizen engagement in the
PBS programme overall;
 Some of the indicators have not been documented accurately. For example, output indicator 2.5
refers to number of districts but it should be percentage;
 Some of the 2014 targets in the log frame are lower than the targets for 2013. This is because the
PBS results framework targets have been revised downwards over the last year and subsequently
adjusted in the log frame; and
 The log frame assumptions do not always match the risks that DFID has identified for the programme.
Suggestions as to how this might be addressed are presented in the section of this review on risk.
There should also be an assumption/risk about accuracy of annual administrative data.
C: DETAILED OUTPUT SCORING
Output Title
Improved availability of qualified staff and operational resources for delivery of basic
services
Output number per LF
Output Score
1
11
A
Risk:
Risk revised since last AR?
Medium
Impact weighting (%):
70%
No
Impact weighting % revised
since last AR?
No
Indicator(s)
Milestones
Number of qualified primary school teachers
Number of Health Extension Workers
Number of district road desks with appropriate staff
Number of Development Agents with specialisation on Agro ecological zones and high value crops
Progress
Score
215,000
36,000
470
258,376
35,9071
500
A+
B
A
3,800
2,8112
C
Key Points
Overall, this output met expectations; however progress across the different sectors was mixed. The
number of qualified primary teachers increased from 216,562 in 2013 to 258,376 in 2014, well above the
target of 215,000. The number of HEWs has just undershot its target of 36,000. This target is based on
an estimate of the number of HEWs needed for Ethiopia’s current population figures and will rise to
40,000 in 2017/18. The target for 2014 is below the 2011 baseline of 38,298 when the Ministry of Health
trained more HEWs than required in anticipation of staff attrition. The number of specialised agricultural
Development Agents (DAs) that have been trained in agro ecological zones and high value crops fell
from 2013 and missed the log frame target. It is even further below the PBS results framework target of
9,900. This issue was raised as a great concern by the State Minister of Finance with the State Minister
for Agriculture at the JRIS. The State Minister for Agriculture responded that his Ministry was prioritising
the general upskilling of all of Ethiopia’s 10,000 Development Agents rather than on specialised agroecological zones and high value crops. Whilst this continues to be an area of concern the number of
beneficiaries of agricultural extension services was more than 13 million in 2014 which exceeded the
target of 12.8m.
PBS is also beginning to examine issues relating to quality of services and the effectiveness of its
investment. For example, a PBS funded Education SDI study assessed a small but representative
sample of 222 rural schools and 125 urban schools. Teacher absenteeism was found to be 12% (10.1%
rural; 14.1% urban). This compares favourably with documented absence rates in other countries, e.g.
Kenya, Senegal, Tanzania and Uganda. Known reasons for absence included going to receive salary,
funerals, field trips, training, maternity leave and sick leave. However, not all teachers in school are in
class. More than a quarter of all teachers (27%) were in school but not in class, with the figure being
higher in rural areas, which is an issue of concern. This study also assessed teachers’ knowledge of
English, mathematics and teaching techniques. Overall, knowledge of mathematics met standards but
attention was needed in respect of English and teaching skills. Improving the quality of education,
including teacher’s knowledge and skills is being addressed under the DFID funded General Education
Improvement Programme (GEQIP) with the Ministry of Education. This study provides a useful baseline
to assess the effectiveness of PBS and GEQIP against in the future. It is planned to be repeated every
two years. A similar study for health is underway that is mapping the location of health facilities,
assessing their functionality, the quality of care provided and availability of 24 hour care. This will report
in early 2015 and be presented at the April 2015 JRIS.
New evidence on the effectiveness of Development Agents is emerging from some DFID funded
research. There appears to be a positive correlation between advice provided on animal husbandry,
marketing, land management and fertilisers and farmer’s crop yields and income. However advice on
agricultural practices seemed to have no effect and advice on credit appears to be having a negative
impact on poorer farmers’ income. The distribution of DAs is inconsistent across the country. Farmers in
Oromia were found to be the most likely to know of a DA in their village/ward and those in Gambella the
least, although the study was unable to say if this difference was significant. It also reported that there
was no difference in the DAs’ impact between male and female farmers. This contradicts the PSIA that
finds that men more than women benefit from the agricultural extensions services. However it did
1
This number is slightly below target and also below reported number (36,336) in 2013
This number is below the revised target (3,800) and the reported 2013 performance of 3,201. Reports to JRIS are still referring to the much
higher target of 9,900. There appear to be two versions of the results framework. The one presented to JRIS (Annex 3 of the aide memoire)
differs from the updated results framework presented in the World Bank’s Restructuring Paper dated 16 th December 2013. It is unclear why
different versions of the results framework are circulating.
2
12
confirm the PSIA’s findings that agricultural support yields its best results when targeted at those with the
ability to use it, i.e. the better educated and older farmer who will have learnt from experience3 . This
evidence will help inform the policies of the Ministry of Agriculture’s and regional agricultural bureaus
when considering how best to deploy PBS resources for the maximum gain.
DPs need to follow up with MoFED and the sector ministries on how they will use this emerging evidence
to improve the availability, quality and effectiveness of services funded by PBS and the sector support
programmes. Current policies and future plans must be reviewed to ensure they are compatible with this
new evidence and respond to the different context and needs across the country. In particular, DFID and
other DPs need to follow up on the availability of DAs to ensure that resettled communities in the
emerging regions are able to access the support and advice they need to be successful in their new
settings. These surveys provide a useful baseline and should be repeated in the future to assess the
effectiveness of government’s policies and investments under the next GTP.
Summary of responses to issues raised in previous annual reviews (where relevant)
Issue
DFID to continue its dialogue in the PBS sectors on the availability of
recurrent budget at district level to properly equip staff to deliver
quality services.
Support service quality monitoring in the health sector through the
Service Provision Assessment (SPA) survey in 2014 and the roll out of
the balanced score cards to monitor health workers’ performance.
The causes of high staff turnover in the health sector should be
identified as part of a broader effort to strengthen the monitoring and
evaluation of the human resources funded by PBS.
The causes of the major shortfall in appropriately trained DAs needs
to be understood and addressed.
Response
The issue of non-salary recurrent budget was a key topic of discussion
in the May 2014 JRIS/Joint Budget and Aid Review (JBAR). The
ability to increase sector’s operational budget at the sub national level
is very limited and will require an increase in the overall fiscal envelop
through increased domestic revenue mobilisation. While GoE did not
commit to increase non-salary recurrent budget, they did report at the
November 2014 JBAR that it had increased by 12% over the last year.
One of the expected deliverables for the April 2015 JRIS is for GoE to
agree a terms of reference and obtain expressions of interest for a
study focused on domestic resource mobilisation to increase
operational budgets.
The SPA is underway and will be shared at the April 2015 JRIS. No
information was available on the rollout of the balanced score cards.
Tracking surveys are planned in both agriculture and education
sectors.
The issue of the ‘shortfall’ in the number of DAs is reported to have
been largely as a result of an unreliable and unrealistic estimate of
need. There are particular concerns about the quality and reliability of
personnel data in the agriculture sector and steps are being taken to
address this. The findings of the DA tracking survey will be presented
at the April 2015 JRIS.
Recommendations



Focus on increasing the effectiveness of PBS – follow up with government on how the findings
from the education, health and DA studies will be used to maximise the effectiveness of
investments made through PBS and sector programmes. In particular ensuring that basic
services such as appropriate DA advice is available to resettled communities;
Due to the level of risk associated with supporting the salaries of public sector workers in Ethiopia
it is proposed that the risk rating for this output should be raised from medium to high; and
In order to emphasise the importance of the three elements of sub-programme B, it is proposed
to reduce the weighting of this element from 70% to 60% and increase the weightings of other
outputs accordingly.
Output Title
3
Improved transparency and fiduciary assurance regarding the use of public funds
Output number per LF
2
Output Score
A+
Risk:
Medium
Impact weighting (%):
10%
Risk revised since last AR?
No
Impact weighting % revised
since last AR?
No
Hamilton A and Hudson J, 2015, “The return to agricultural advice in Ethiopia: A rationale for a success story?”
13
Indicator(s)
% of districts that post refined and simplified budget and
expenditure template to the public
% of basic service units that post the standardized service delivery
template
% district sectoral offices audited by offices of regional auditor
general (ORAG) annually
% federal agencies and ministries audited by the Office of Federal
Auditor General (OFAG) annually
% of visited districts that implement prior period audit
recommendations
Number of districts staffed with procurement officers with regional
procurement training and with clear career path
Number of districts that post all procurement award decisions
Milestones
Progress
Score
90%
95.9%4
A+
40%
63%5
A++
40%
40%
B6
95%
100%
A
25%
5.1%
C
100
3337
A++
Revised
template drafted
and field tested
Completed 254 districts
posting
procurement
award
information
A+
Key Points
Overall, this output moderately exceeded expectations; however progress across the different indicators
was mixed with high performance in some areas. Elements of PBS sub-programme B covered under this
output include Financial Transparency and Accountability, Public Financial Management reform and
procurement.
FTA: The FTA initiative is well-integrated into the government PFM system and planned activities are
progressing well. The 2014 targets in DFID’s log frame and the PBS results framework have been
exceeded and other related deliverables agreed in the May 2014 JRIS achieved. Almost all districts and
facilities are now posting budget and expenditure information using the agreed template. An increasing
number of facilities are posting service delivery information. Regions are using a range of means to
share budget information, including TV broadcasting and SMS messaging. From May 2014, a total of
66,125 people received budget literacy training. The proportion of women trained varies depending on
who they are with 31% of council members and 37% of citizens trained being female. Extensive
discussions have taken place on how FTA and social accountability could be better linked, including at
the FTA annual review meeting held in August 2014 in Adama. Guidelines have been developed and
shared with the regions. Good examples are being reported from some regions including SNNPR and
Tigray. Ongoing challenges affecting FTA include staff turnover and limited female involvement in budget
literacy training. Through PBS, DFID is providing around £100,000 to the Construction Sector
Transparency Initiative (COST) through a MoU with the Ethiopian Ethics and Anticorruption Commission.
However, this support is not reflected in the DFID log frame and DFID is considering whether this could
be funded outside of PBS in the future.
PFM Reforms : Reforming the overall public financial management of government resources is a core
component of PBS. There has been good progress on federal level audit coverage and increased
scrutiny of regions by OFAG and the Parliamentary Accounts Committee (PAC). The federal and
regional level targets in DFID’s log frame have been met although the proportion of sectoral offices
audited by ORAG has been marked down as this year’s performance is lower than the 45% achieved in
2013 (against a target of 35%). The federal level PFM deliverables from the May 2014 JRIS were
achieved. Highlights include draft Public Expenditure and Financial Accountability (PEFA) reports for the
federal government and five regions; progress made on establishing professional accountancy bodies in
the country; the recruitment of regional internal auditors; 400 internal audit committees established in
districts and Ethio Telecom contracted to improve internet connectivity to districts.
Districts’ capacity to manage public funds is still weak. In May 2014, only 14.6% of districts visited
reported implementing audit findings as compared to the 20% target. District audit follow up was made a
priority deliverable, and raised repeatedly by DFID in the PFM TWG and DP/GoE meetings. To assist in
4
This figure was reported in November 2014 JRIS PowerPoint for posting templates at woreda level
This figure was reported in November 2014 JRIS PowerPoint for posting templates at service facility level
6 Whilst this indicator has met its 2014 target the rate of achievement is actually below the 2013 level of 45% and so is scored a B.
7 This includes all woredas in SNNPR, Amhara and Dire Dawa
5
14
this area PBS paid for internal auditors to be recruited for regions and established 400 districts audit
committees. These committees are responsible for ensuring action is taken against audit findings and
will help improve audit follow up at the sub national level. In addition MoFED has established an Internal
Audit task team to support the committees conduct repeat audits where necessary and address any
audit backlog. All 900 districts plan to establish audit committees but will need support to make them
functional and effective.
Supervision missions were conducted by MoFED and BoFED in Tigray, Oromia and SNNPR to follow up
on audit findings and to districts where less than 10% of previous audit issues had been rectified.
Despite these efforts only 5.1% of districts reported audit follow up at the November JRIS. In response
the November 2014 JRIS agreed that MoFED in consultation with regions will develop a roadmap for
strengthening Internal Audit by the April 2015 JRIS and that repeat visits to districts will be made to
check whether audit findings have been actioned. Progress on this deliverable will be measured by: (i)
Quarter 1 and Quarter 2 2014/15 audit reports including from repeat districts to be delivered within 60
days of end of a quarter; (ii) Action plans delivered to address any findings of the Quarter 1 and Quarter
2 of 2014/15 auditreports agreed by April 2015; (iii) Feedback from regions on actions taken on findings
of the audit reports delivered to DPs at the next JRIS; and (iv) The target is set that sample re-visited
districts by the auditors to show that more than 50% of past findings were effectively addressed. Audit
follow up has also been proposed as a disbursement linked indicator to trigger funding disbursements
under the World Bank’s future Program for Results support to PBS.
There is a need to better understand whether the decline in this indicator is due to more accurate
reporting from the districts’ audit committees and regional internal auditors or because of an actual
decline in performance. Either way it is of great concern to the MoFED, DFID and other PBS partners
and has been made a top priority for the coming period.
At the Federal level the draft 2014 PEFA report notes that ‘internal and external audit continue to report
cases of non-compliance with government’s financial guidelines, but these are a small minority of all
transactions’. It reports improvements in OFAG following up on its recommendations through the PAC
hearings of those who have had adverse or qualified reports, or disclaimers. Heads of audited bodies
attend and have to submit action plans on the measures that will be taken. OFAG follows up at the
following year’s audit to establish whether action has been taken. The OFAG reported that most heads
of budget institutions do respond. The Council of Ministers is strongly supporting this process but no
database is kept of outstanding recommendations or queries.
The PEFA’s summary scores are listed below:
Indicator
Score
2010
Score
2014
Justification
Performance
change
PI-26
C+
B+
M1
i
C
A
All entities of central government
are audited annually, covering
revenue, expenditure, and
assets/liabilities, financial and
performance audits, using ISSAIs.
Improvement
ii
A
A
Audit reports are submitted to
Parliament within 4 months of
receipt of the financial statements.
No change
iii
C
B
A formal response is made in a
timely manner, but follow up is not
systematic.
Improvement
performance
in
Procurement: Overall progress on procurement reform activities under sub-programme B was
considered satisfactory in the November 2014 JRIS. The Federal Public Procurement and Property
Administration Agency (FPPPA) reported that work on the revision of Procurement Proclamation and
Directives had been finalised at agency level and these would be submitted for approval. The process to
identify gaps in district procurement systems is advancing and should be concluded by December 2014.
A stakeholder workshop will then be held to discuss findings. A consultant was hired to review the
15
procurement competency framework, develop a certification programme and prepare materials for
institutionalised training. The World Bank has provided support to carrying out analytical work on
framework contracts and value chain analysis during this period which was delivered to government. All
regions were reported to have started carrying out procurement audits. MoFED’s Channel One
Programs Coordinating Unit (COPCU) reported that about 86% of PBS procurement activities have been
started against an agreed target of 90%. Slower progress than expected was reported in procurement
activities under M4R, which are discussed under output 4.
Summary of responses to issues raised in previous annual reviews (where relevant)



Issue
The lack of means to track several indicators under this output,
notably related to procurement and audit follow up, is a concern and
makes a meaningful assessment difficult. It is recommended that
special attention is given to ensuring that relevant data is collected
and reported to enable a proper review of progress next year. The
Public Financial Management and Procurement TWG are well placed
to do this.
Response
The indicators for this output have been revised extensively and data
made available.
Whilst federal audit performance and follow up continued to perform
well the lack of district audit follow up is of great concern and should
be prioritised for action. MoFED has provided guidance to BoFEDs on
audit but needs to ensure that guidance has been translated into local
languages and that district staff have been orientated. The PBS
secretariat should work with MoFED and BoFEDs to ensure audit
findings are communicated back to the district offices and that action
plans for follow up are agreed and implemented. DFID can monitor the
extent to which this is being done through the monthly PFM meetings
and DP meetings as well as requesting a report at the JRIS.
There is a need to expedite planned procurement systemstrengthening activities under PBS so that regions are able to take
action to achieve their respective share of agreed procurement
performance targets for the current fiscal year.
See PFM section above
The Procurement TWG has been much more active in the last year
and has made satisfactory progress against procurement activities
under sub programme B. Work on the revision of the Procurement
Proclamation and Directives has advanced and been finalised at
agency level. They will now be submitted to the Council of Ministers
for approval. All regions are conducting procurement audits and are
developing action plans.
District level procurement system gap identification and preparation of
simplified directives has advanced and is expected to be concluded by
December 2014. Work is also underway to review the procurement
competency framework, develop a certification programme and
prepare materials for institutionalizing training of district procurement
officers to strengthen capacity at the sub national level.
Recommendations
A number of the agreed deliverables for the next JRIS relate to these areas:

FTA – disseminating audit and procurement templates to regions and districts; providing budget
literacy training; and providing evidence for FTA/social accountability linkage activities;

PFM reform – continuing high level dialogue on strengthening PFM and audit follow up;
developing a road map for strengthening the internal audit system to improve the rate of audit
follow up; implementing plans for financial training; rolling out a new version of the Integrated
Budget and Expenditure (IBEX) system to districts; and supporting the professionalization of the
accounting/auditing staff; and

Procurement - finalising discussion with stakeholders on procurement performance indicators
and an implementation plan; resolving technical problems with the FPPPA’s website to enable
the publication of procurement rewards made; following up the study to identify gaps in district
procurement systems; hiring an independent procurement auditor; and reporting on the status of
agreed audit action plan.
Whilst DFID should monitor progress on these important deliverables the key recommendation is for:

DFID to continue to exert pressure to ensure an improved performance on district level follow up
of audit issues. DFID should secure agreement with the incoming PBS DP co-chairs that this is a
top priority for the programme; continue to raise the issue with the highest level of GoE and
through the PFM technical working group; and request monthly updates from COPCU on
progress towards developing an Internal Audit strengthening strategy. If useful DFID could also
16
provide additional short term technical assistance to COPCU to help them understand the gaps
and develop an effective strategy to address them. Due to the low performance in this area DFID
should consider escalating the risk rating of PBS from medium to high. If performance does not
improve by the November 2015 JRIS DFID will need to consider this carefully before disbursing
its next tranche of funds; and

Review the logframe to ensure that the future targets demand a year on year improvement and
are not already achieved. In order to emphasise the importance of this element of PBS, it is
proposed to raise the weighting from 10 to 15%.
Output Title
Improved citizens engagement on quality of basic service delivery
Output number per LF
3
Output Score
A+
Risk:
Medium
Impact weighting (%):
10%
Risk revised since last AR?
No
Impact weighting % revised
since last AR?
No
Indicator(s)
Number of districts where PBS Social Accountability implementing
partners (SAIP) facilitate the use of social accountability tools.
Number of regions with functioning Grievance Redress
Mechanisms (GRM) consistent with best international practice.
Number of regions preparing and adopting standardized manual
and guidelines for GRMs
Number of regions submit the PBS quarterly and annual interim
financial reports to COPCU without major inconsistencies
Milestones
Progress
Score
150
223
A++
7
7
A
8
8
A
8
11
A+
Key Points
Overall, this output exceeded expectations. Elements of PBS sub-programme B covered under this
output include the Ethiopian Social Accountability Programme and Grievance Redress Mechanism.
ESAP: Since the last JRIS, social accountability committees (SACs), assisted by civil society social
accountability implementing partners (SAIPs), undertook interface meetings and drew up joint plans of
action in a further 70 districts. As a result, 150 districts out
Box 3: Examples of results of use of social
of 223 in which ESAP operates have these plans in place
accountability tools
with many having started implementing and monitoring
them. A total of 178,267 citizen representatives have been
In the education sector, it was reported that social
trained on social accountability and 109,084 are directly accountability activities had resulted in
engaged in evaluating basic public sectors using social construction of 85 additional classrooms, 39
separate toilets for boys and girls, 17 school water
accountability tools. Positive experiences (see Box 3)
sources and 22 compound fences. Also, an
were heard during JRIS field visits to three districts in
additional 113 teachers were recruited. Costs were
Tigray, including evidence of linkages between FTA and covered by local government (56%), community
social accountability initiatives such as pre budget (15%) and others, e.g. NGOs and the private
sector (29%).
discussions forums being held. Feedback from people
interviewed provided insight into their perceived benefits Benefits reported in the health sector included
of ESAP including: (i) identifying special needs of the poor
improved supply of drugs, recruitment of additional
staff, construction and maintenance of water and
and vulnerable; (ii) providing citizens with information on
sanitation facilities at health units and improved
sector policies and service standards; (iii) promoting
equipment at laboratories.
dialogue between citizens, service providers and local
officials; (iv) promoting ‘bottom–up’ planning; and (v) In the water and sanitation sector, 56 new water
promoting local ownership of public and community points were constructed, 54 water points were
assets. Whilst feedback from the SAIPs is informative a repaired, 60 water points were fenced and 753
community dry-latrines were constructed.
more systematic way of measuring citizen’s satisfaction of
basic services is required. PBS plans to conduct a
In the agriculture sector, additional extension
beneficiary satisfaction survey as part of its overall evaluation starting
workers and
in 2015.
veterinary staff were recruited and
essential agricultural inputs were distributed.
17
ESAP phase 2 (started under PBS 2) will soon come to an end and DFID is actively engaging on the
future of this initiative and the possibility of a ESAP phase 3 programme. There are concerns about how
social accountability can be sustained as it requires ongoing facilitation, training and financial resources.
In response to this ‘road map thinking’ on the sustainability and scalability of ESAP has taken place
between GoE, DFID, DPs and the management agent. This has included study tours to Rwanda and
Indonesia to learn from their experiences and the review of possible strategies by a technical committee.
GoE has agreed to present a strategy for the scale up, institutionalization and sustainability of ESAP at
the April 2015 JRIS.
GRM: Ethiopia has two GRM structures. The first one is the Ethiopian Institution of the Ombudsman
(EIO) which is a federal independent entity which currently has branch offices in seven regions. It is
accountable to and funded by the federal parliament. The second one is the Public Grievance Handling
Office (PGHO), which are present at the regional, district and village level. These are accountable to the
chief administrator at the regional, zonal, and district levels under the regional government. At the lowest
level the village manager is in charge of hearing and handling grievances and referring issues upwards if
they cannot be resolved locally. Unlike the EIO, the PGHOs are not independent but are funded and
used by the regional government as a self-correcting mechanism. The EIO is still a relatively new
institution in Ethiopia and plans to open branch offices in every region. DFID has actively engaged with
the new Technical Working Group ( TWG) to identify ways GRM can be strengthened and made more
effective.
All the GRM related deliverables set in the May 2014 JRIS were achieved and the EIO is exploring ways
to further support the PGHOs. A study on the grievance hearing and redress system at federal and
regional levels has started to identify capacity gaps and areas for future support. Training has been
provided to 349 GRM officers in Amhara, Dire Dawa, Somali and Harari and dissemination on GRM has
been done through the media to raise public awareness. A technical working group has been established
with the Chief Ombudswomen and DPs to explore ways to support the regional grievance hearing
offices. Deliverables agreed for the April 2015 JRIS include the delivery of the GRM study report with
identified areas of support for regional offices/mechanisms; capacity building training for regional and
district grievance handling offices – including some village managers; and the opening of EIO branches
in the last two remaining regions of Afar and Benishangul Gumuz.
Achievements made to improve citizens’ engagement through this output adds significant value to PBS.
ESAP under PBS is of critical importance given its potential to strengthen incentives for continued and
improved service delivery in the long term. However its achievements need to be assessed over a
realistic time frame. The current time frame appears too short and this has implications for achieving
nationwide coverage and sustainability. The PBS results framework monitors the programme against
quantitative indicators (i.e. number of joint action plans) rather than the quality of engagement and
change as a result of activities. But, monitoring by the managing agent is more comprehensive and can
be used to better understand the effectiveness of the programme. There are concerns that ESAP’s
SAIPs are choosing the sector in which to work rather than this being identified by the community. This
top down approach is a missed opportunity for building generic social accountability capacity in districts.
This approach may have been adopted due to the limited number of implementing partners available in
the region. Never the less the need to empower and capacitate communities to hold local government to
account for all services must be considered when designing a sustainable social accountability model for
the whole country.
Project Financial Reporting: All regions submit interim financial reports (IFRs) on time and in
accordance with the agreed template. However the quality of reports still varies between regions and
sometimes lack explanation for variance in budget utilisation.
Summary of responses to issues raised in previous annual reviews (where relevant)
Issue
There is a need for a higher level indicator both in DFID’s logframe
and the PBS results framework to capture the impact of social
accountability activities on people’s understanding and the net result
of their raising issues with public officials. Including the DFID global
accountability indicator both in the log frame and the PBS results
Response
Although the indicators for this output have been revised, they remain
largely quantitative and procedural. Nevertheless, the ESAP
management agent does have many examples of activities resulting in
actions, and DFID’s role in the ESAP Steering Committee means it has
access to these more in-depth reports and is able to learn from them.
18
Issue
framework would help address this.

The nascent GRM mechanisms require continued active support to
help them become operational as quickly as possible. This is likely
to include the provision of their own transport - to enable
independence of mobility and hence investigative independence and other basic materials to enable operationalization of the new
forums.
DFID should continue to work to ensure proper linkages between
the new GRMs and the Social Accountability and Financial
Transparency and Accountability work streams so that issues raised
in the latter can be addressed.
Whilst continuing to report on activities and expenditure against
budget, the MA should adjust reporting away from process to report
on programme content, achievements and challenges. Stability in
leadership of the MA is now very important to ensure the effective
delivery of this component.
Response
The beneficiary survey planned as part of PBS’s evaluation will provide
additional information on satisfaction levels. This review recommends a
further revision of the log frame with respect to measuring the outcome
of social accountability.
GRM mechanisms are beginning to operate. For example, in three
districts in Tigray, the Grievance Hearing Offices considered 978 cases
over two years. Main topics relate to land and inclusion in safety nets
programme. A TWG has been established between DPs and the Chief
Ombudswomen to consider the needs of regional GRM and how they
can be strengthened. This has already been successful in mobilising
additional funding from the Austrian Government to strengthen GRMs
functionality through UNDP.
Progress is being made to link the FTA and SA elements of PBS as
seen with the holding of pre-budget discussion forums at district level.
However continued efforts are needed to ensure these links are
institutionalized and fully utilised. A deliverable has been included for the
April 2015 JRIS to provide evidence of the FTA/SA linkage activities.
The link with GRM is less obvious and requires further consideration.
The Management Agent is now more stable and their reporting and
performance has improved.
Recommendations
A number of the agreed deliverables for the next JRIS relate to these areas. These include:

Social accountability – agreeing on the future of ESAP including approaches to sustainability and
scaling up; and

GRM – delivering the final report of the GRM study; providing training for PGHOs at regional,
district and village level; identifying areas of support for the EIO and regional PGHOs; and
enhancing accessibility of the GRM system by opening EIO branch offices in Afar and
Benishangul Gumuz.
In addition, specific recommendations for DFID to focus on are:

To consider how support to this output could be strengthened through extending the timeframe
for ESAP 2, reviewing the overall approach/strategy (including adopting a generic rather than
sector based social accountability approach) and increased DP engagement. Any future phase of
supportto ESAP should consider how sustainability should be addressed before commencing
support;

Consider introducing an indicator into the log frame that measures the outcome of social
accountability work and is in line with DFID’s Departmental Result Framework indicator of ‘ The
number of people supported to have choice and control over their own development and to hold
decision-makers to account’;

Consider whether there is scope to promote better linkages between the SA component of PBS,
and the separate, DFID supported Civil Society Support Programme, managed by the British
Council; and

In order to emphasise the importance of this element of PBS, it is proposed to raise the impact
weighting from 10 to 15%.
Output Title
Strengthened M&E system at central, federal and sub national levels.
Output number per LF
4
Output Score
B
Risk:
Medium
Impact weighting (%):
10%
Risk revised since last AR?
No
Impact weighting % revised
since last AR?
No
19
Indicator(s)
Number of PBS sectors with their indicators assessed through
implementation of Ethiopia Data Quality Assurance Framework
(EDQAF)
Number of districts that rolled out the new education
management information system (EMIS)
Proportion of health centres implementing the new Health
Management Information System (HMIS)
Number of districts that rolled out the new WaSH management
information system (WMIS)
Number of regions implementing Agriculture Management
Information System (AMIS)
Development of Ethiopian Road Authority Management
Information System (ERAMIS)
Milestones
Progress
Score
2
2
A
600
8178
A
90%
90%
A
600
330
C
8
0
C
4
0
C
Key Points
Overall, this output has moderately not met expectations, with mixed progress across the indicators.
The Managing for Results (M4R) component under sub programme B aims to improve results
monitoring by strengthening sectors’ management information systems (MIS), conducting data quality
assurance checks and by undertaking demand driven analytical work and surveys. Strengthening
sector’s information systems is regarded as the most crucial for improving evidence based decision
making and management across basic services and its importance is reflected in the indicators tracked
in DFID’s log frame. Progress in other areas of this component are tracked through the PBS results
framework, JRIS deliverables and the M4R working group.
There have been significant and frustrating delays with the introduction of the new MIS in the agriculture,
roads and WaSH sectors. This has been due to procurement delays in the purchasing of IT equipment
for federal, regional and district offices. The May 2014 JRIS flagged this as a major priority that if not
addressed may lead to the return or reallocation of DP funding. Significant efforts have been made by
MoFED, the PBS secretariat and the World Bank (who provides no objections to all procurement) in the
last 6 months to address the bottlenecks. Whilst the targets have still not been met and budget utilisation
remains low (11%) over 85% of procurements are now in the contracting phase and the IT equipment
will be installed soon. It is recommended to reset the targets for the WMIS, AMIS and ERAMIS based on
2014 being year zero.
Due to slow progress on many of the recommendations within M4R’s work plan DPs urged GoE to
review and consolidate the numerous analytical pieces that were planned under this component. This
has been done and 6 activities were cancelled as they were either redundant or overlapped other
studies. The rest of the plan has been prioritised into 15 discrete activities. Analytical outputs since the
last JRIS include the education SDI survey, a mini DHS, gender disaggregated development data, and
the M&E system assessment reports for education and health sectors.
Overall, this is the component of PBS where there has been slowest and most limited progress.
However, significant management efforts have resulted in bottlenecks being addressed, and a more
focused prioritised work plan. Good progress is expected over the next year. Log frame targets should
be reset and DFID should continue its strong engagement in the M4R TWG as well as requesting
monthly updates in the GoE/DP monthly PBS meetings.
Summary of responses to issues raised in previous annual reviews (where relevant)
Issue
It is recommended that continued delays in the procurement of ICT
and other equipment needed for the roll out of MIS systems in the
roads and agriculture sectors are urgently addressed.
Response
There has been considerable effort made to address the bottlenecks
and improve performance of this component. These efforts are now
paying off with 85% of procurement in the contractual phase. This
8
MoE confirmed that districts in all regions except in Afar, Somali, Gambella, and Harari are inputting their consolidated data on the new EMIS
system at zonal level who then collates and passes data on to regional level for aggregation using the new system. In 2014 all 1000 districts
were using the newly revised EMIS questionnaire but were filling them in manually rather than on a computerised system. The correct definition
of this indicator is the number of districts that rolled out the new computerised EMIS. This has now been done in 817 districts and hence
appears lower than the 2013 achievement.
20
Issue
DFID should explore with other DPs what more can be done to ensure
that human resource management is addressed at regional and subregional levels. This has become more urgent since the Public Sector
Capacity Building Programme ended and would include understanding
recruitment, training, performance management, staff management
and payroll issues.
Response
should results in the importation of the ICT equipment needed to set
up the MIS in the coming year.
This is being considered by GoE and DPs.
Recommendations
A number of the agreed deliverables for the next JRIS relate to these areas. These include:

M4R – finalising the revised work plan until 2017; sharing the Education SDI and Health SPA+
reports; strengthening MIS in education and agriculture; progressing with Ethiopia Roads
Authority’s baseline data collection; preparing an analytical report on gender disaggregation of
results; conducting training in results-based monitoring and progressing on a number of studies.
In addition specific recommendations for DFID are:

To continue its intensive but effective engagement in the M4R technical working group to closely
track progress going forward and ensure improved performance;

To reset DFID’s log frame targets for the indicators on WMIS, AMIS and ERAMIS as 2014 will be
year zero; and for EMIS as it has already exceeded its target. Keep the impact weighting at 10%.

Escalate the level of risk from medium to high considering the substantial delays experienced to
date.
D: VALUE FOR MONEY & FINANCIAL PERFORMANCE
Based on Ethiopia’s low staffing Table 2: Allocation of Regional and District Recurrent Expenditure in PBS
costs, in comparison to other focus sectors in 2012/13 and 2013/14: Percentage share of total recurrent
countries; the low administrative cost expenditure
charged by the World Bank and the
2012/13
2013/14
cost-benefit
analysis
of
the
Sector
All
All
Salaries
Salaries
recurrent
recurrent
programme, PBS delivers very good
Education
50.6%
58.4%
48.5%
57.1%
value for money. As a high value and Health
13.6%
20.2%
13.3%
20.9%
complex programme, the 2013 Annual Agriculture
12.5%
16.3%
11.8%
17.7%
1.8%
4.1%
2.1%
3.3%
Review recommended a more in-depth Water
Roads
0.6%
1.0%
0.6%
1.0%
VfM review be done to understand the Total
79.2%
100.0%
76.4%
100.0%
different cost drivers under PBS’s
components. This is currently taking place using the REEES (Relevance, Economy, Efficiency, and
Effectiveness and Sustainability) framework and its initial findings are summarised below. The final
report will be available in early 2015.
Key cost drivers and performance.
91% of DFID support for PBS provides funding for Federal Block Grant expenditure, covering recurrent
costs in the five basic priority sectors; 6% for the strengthening of local transparency and accountability
systems; and 3% for the costs of programme implementation support9. Staff salaries for the 5 PBS
sectors are by far the largest cost driver of the programme. Table 2 shows the allocation of sub-national
recurrent expenditure in education, health, agriculture, water and roads over the last two years. It
highlights that PBS funding remains concentrated on salary payments which in 2013/14 accounted for
76.4% of recurrent expenditure in focus sectors. Among the sectors, education was the largest recipient
of recurrent funds with teachers’ salaries accounting for 48.5% of total recurrent expenditure across the
five sectors. The table also shows that the share of salaries in recurrent expenditure in the five priority
sectors declined slightly from 79.2% in 2012/13 to 76.4% in 2013/14. This decline is matched by a 12%
21
nominal increase in non-salary operating spend indicating a positive trend in the composition of the
budget.
The cost drivers for the programme’s expenditure to strengthen local transparency and accountability
systems remain charges for services such as training and consultancy. There is little expenditure in this
part of the programme on goods. The costs of programme implementation are driven by World Bank
charges for the secretariat and for administering DFID’s funds.
VfM performance compared to the original VfM proposition in the business case.
The business case sets out a number of VfM indicators under the headings of economy, efficiency, and
effectiveness. In practice a large number of these cannot be assessed because of the absence of data,
but it is still possible to assess PBS in the context of the core dimensions of value for money as follows:
 Relevance. To analyse the relevance of a programme requires it to be assessed in terms of the
extent to which it is suited to the priorities and policies of the recipient, donor and target group. PBS is
explicitly designed to support the GoE’s Growth and Transformation Plan (and the five sectors which it
prioritizes). It is also provided in the context of the DFID’s operational plan for its programme in Ethiopia
and the UK’s Partnership Principles. The biannual government led JRIS provides the basis for ensuring
PBS maintains its relevance to government policy and priorities over time. All this essentially means that
PBS passes the basic tests of relevance.
 Economy. Public sector salaries in Ethiopia, when benchmarked against GDP per capita, are not
high compared to other countries in East Africa. Ethiopia’s teachers and health workers are paid slightly
more than those in comparable public sector grades in other sectors, but less than teachers and health
workers in the private sector, except in the case of health workers at the start of their careers. A large
public sector salary increase that took effect in 2014/15 has increased salaries by around 33-37% and in
some cases more. However this followed a four year pay freeze and is still not close to private sector
levels. During this period, it is estimated consumer prices increased by over 90%, implying significant
falls in the real costs of public sector employment. In the case of support for local transparency and
accountability systems, procurement takes place within government procurement systems that still have
weaknesses but economy is protected by oversight from the World Bank who have helped put in place
important risk mitigation measures. The World Bank’s charges of 1.1% for administering DFID’S funds
compare favourably with charges by other international agencies, such as the 7-12% charged by other
UN organisations.
 Efficiency. Block grant financing of salaries has been associated with increases in basic service
staffing and improved efficiency by reductions in the ratio of staff to the population they serve. Primary
school pupil teacher ratios (grades 1-8) have fallen from 51:1 in 2010/11 to 49:1 in 2013/14. Although
the picture in the health sector is complicated by the government targeting a small increase in the ratio of
the population to HEWs to a level viewed to be sustainable. Staffing increases have enabled the ratio of
the population to nurses to fall from an estimated 2745:1 to 1864:1 over the same period. Significant
regional variations in these ratios, however, persist. It is worth noting that these developments in terms
of staff to population ratio are more or less in line with the GoE service delivery standards. For example,
in Amhara region, the regional standard in primary education is 1:50 and secondary 1:40. The standard
in health at regional level is 1 HEW to 2500 population. The recent Education Service Delivery Indicators
(SDI) study indicated that teacher absenteeism was around 12% for selected woredas (10.1% rural;
14.1% urban). This compares favourably with documented absence rates in other countries, e.g. Kenya,
Senegal, Tanzania and Uganda.
More information is required on a range of workforce issues to confirm that rising payroll numbers are
translated into increased availability of staff on the ground. Judgements on the efficiency of support to
strengthen systems for local transparency and accountability through sub programme B are affected by
the failure in 2013/14 to meet some of the targets under DFID’s log frame outputs 2 and 4. Slow
utilisation of committed funds in this context, particularly on M4R, raises additional value for money
concerns.
 Effectiveness. The PSIA, conducted in 2013/14, provides particularly strong confirmation of the cost
effectiveness of PBS. This shows that an increase of US$1 per capita in district level education spending
could be associated with a 3.6% increase in the net primary enrolment rate within that district. It also
shows that US$1 per capita increases in district level health expenditure per capita could be associated
22
with increases in the contraceptive prevalence rate of 6.4%, and in the proportion of births delivered by
skilled birth attendants of 11.3%, both of which have a significant impact on maternal mortality. Similarly,
an increase of US$1 per capita of district level health spending is associated with an increase in
coverage of antenatal care of 3.6%, which can significantly help reduce infant and child mortality.
Separately the assumptions used in the cost benefit analysis for DFID’s business case which estimated
an internal rate of return of 23.1%, excluding the now discontinued Results Enhancement Fund, have
been tested, and appear to be still largely realistic and this will be confirmed in the full VfM report. There
will be scope for further assessment of the assumption on PBS induced improvements in public
expenditure efficiency when the final results of the 2014 PEFA are available.
 Sustainability. Sustainability has been included as one of the Core Principles of the Programme and,
in order to make the principle practical, PBS DPs and the GoE have agreed to undertake a sustainability
review. The purpose of this review is to give all stakeholders a better understanding of the current and
future costs of basic service delivery as well as the current and potential sources of domestic financing
for basic services. A pilot data collection exercise has recently been carried out in two districts in Amhara
region and analysis of the data is underway. The outcome will enable us to understand the actual current
costs and to what extent basic service delivery at the district level is being financed as per the
government’s service delivery standards. In the coming months this exercise will be scaled up nationally.
In parallel a study on domestic resource mobilisation is agreed to be commissioned by the April 2015
JRIS.
Assessment of whether the programme continues to represent value for money.
Overall, the programme is considered good value for money. As highlighted above, investment in public
sector salaries in the five sectors represents good value for money in terms of both economy and
efficiency. The provision of FBG resources has been associated with significant increases in basic
service staffing and improved efficiency by reductions in the ratio of staff to the population they serve.
The programme is on track with respect to most of the intended outputs. Number of qualified teachers
has increased, grassroots transparency and accountability have been improving; and progress is being
made on public financial management. Key concerns relates to the observed slight reduction in HEW
and DA’s with specialized training; the poor follow-up by districts of prior audit recommendations, and the
delayed rollout of MIS in WaSH, agriculture and rural roads. The most powerful arguments concerning
value for money come from the effectiveness data from the PSIA study. The study highlighted that PBS
spending could be associated with service delivery outcomes and favours historically disadvantaged
areas of the country.
There are a number of tests built into the JRIS assessment of PBS which are relevant to VfM. First, the
JRIS conducts an Additionality Test to see if the government’s spending reflects its stated commitments
and policy direction to decentralized basic service delivery by examining actual fiscal out-turns against
proclaimed budget. This assesses three criteria: (i) whether Federal Block Grant allocation in the
Medium-Term Economic and Fiscal Framework (MEFF) is reflected in the budget proclamation; (ii)
whether at least 95% of the stated budget was disbursed; and (iii) whether the share of basic services
from sub-national governments’ budgets is not declining. The November 2014 JRIS considered that all
these criteria were fulfilled. Second, the JRIS conducts a Fairness Test to see whether the allocated
block grants amount in the preceding year had been fully (>90%) disbursed. Except for a very few cases,
regional governments transferred 100% of the block grant allocations to their respective districts. In all
cases, more than 94% of allocations were disbursed to districts. On this basis, the JRIS concluded that
the fairness test had been met.
Quality of project financial management.
A revised DFID budget for PBS was agreed earlier this year when commitments were reduced from
£510m to £388.5m. DFID’s administration agreement with the World Bank was amended to reflect this
and MoFED was notified. The financial reporting of PBS is done through quarterly unaudited interim
financial reports (IFRs), quarterly continuous audit reports provided to the PBS secretariat from the
regions and an annual external audit of PBS. PBS funds are audited separately from other public
resources in the regions’ consolidated government account. The audit is conducted by the Office of the
Federal Auditor General and checks that funds have been used for their intended purpose in line with the
agreed government and programme financial management regulations.
23
The external PBS audit report for 2012/13 was submitted on time in January 2014 and the opinion
expressed by the auditor was unqualified. However, the management letter noted some findings related
to incidences of bank reconciliations not being conducted and some mis-postings of expenditures. These
issues have since been addressed with regions’ now refusing to release PBS funds unless bank
reconciliations have taken place at district level and with the circulation of guidance on expenditure
classification. The next external audit report for the financial year that ended on 7 July 2014 is due in
January 2015. During 2013/14, all PBS Interim Financial Reports were delivered, reviewed and accepted
on time although there is still scope to improve the quality of reporting. Continuous and financial audit
reports covering regional expenditure were also received on time by MoFED and action plans were
prepared and followed up to address key issues. All regions are up to date on their continuous PBS
audits. The status of expenditure of DFID’s budget for the PBS programme is as follows based on the
adjusted programme budget of £388.5m.
Component Description
Total
Budget in
GBP
202991101
Design fund for PBS III
31,251
202991102
Contribution to the PBS II
MDTF
( Sub prog A, B & REF)
387,806,765
202991103
202991104
202991105
Total
Design fund for Results
Enhancement Fund
To provide additional
(DFID commissioned)
support for monitoring,
evaluation and
accountability (i.e. VfM
study, workforce study)
Support to the
Construction Sector
Transparency Initiative
Spent in GBP
2012
2013
31,250.
2014
-
60,500,000
92,000,000
-
82,000,000
Remaining
Balance in GBP
31,250
0.33
234,500,000
153,306,765
19,077.
-127.
18,950.
-
19,077.
542,905
-
18,343
78,750
97,094
445,811
100,000
-
25,000
40,000
65,000
35,000
92,062,421
82,118,750
234,712,422
153,787,449
388,499,872
60,531,250
-
Total
Spent in GBP
The JRIS in November 2014 considered a comprehensive presentation on PBS financial management
arrangements. Almost all deliverables identified in the May 2014 JRIS have been met. In particular:

IFRs were delivered on time and their quality has been improving;

Quarterly regional continuous audit reports were prepared in a timely manner. Action plans were
prepared and followed up in a timely manner with regions;

Timely disbursement of funds from donors for sub-programme A was achieved; and

Repeated field visits were conducted by MoFED and regions to districts to provide support and
follow up.
Three key challenges were noted:

Low budget utilisation of Ethiopian Financial Year 2006 ( July 2013 – June 2014) of funds for sub
programme B. Of the budget of Ethiopian Birr 741m, only Ethiopian Birr 478m (64.6%) had been
released and only Ethiopian Birr 306m (64%) had been utilised. This was only 41% of total available
budget. As detailed under output 4 the main reason for this low utilisation rate was slow
procurement of ICT equipment for sector’s management information systems. However the
November 2014 JRIS confirmed that procurement processes involving 85% of the budget has been
triggered and significant proportion of committed funds are expected to be spent by the next JRIS;
and

Quality of IFRs – despite improvements, some reports lack explanations of variance in budget
utilisation.
24
Date of last narrative financial report
Date of last audited annual statement
September 2014 for 4th quarter of Ethiopian Financial
Year 2006 (April to July 2014).
January 2014 for Ethiopian Financial Year 2005 (
July 2012 – June 2013),
E: RISK
Overall risk rating: High
Overview of programme risk
The World Bank PBS PAD and DFID’s PBS business case contain risk matrices for the programme.
DFID’s risk matrix is a live document which is updated to take account of ongoing assessment and
management of risk.
In 2013/14 the risk matrix was updated to include information on the way DFID is monitoring and
managing risk, at the portfolio level, related to the Government of Ethiopia’s implementation of the
Commune Development Programme (CDP) in the Developing Regional States and resettlement in South
Omo. This particularly dealt with the risks specific to achievement of PBS results, notably concerns that
in some cases services were not been fully in place before people moved and concerns about the
independence of grievance redress mechanisms.
In terms of mitigation, DFID has conducted regular, systematic monitoring and risk management in
relation to CDP and resettlement in South Omo. This has taken place at ‘portfolio level’, providing risk
assessment and management for all DFID programmes in affected areas, to assess and seek to mitigate
risks to successful implementation of our programmes and risks from our programmes. This includes:
 Monitoring of the implementation of CDP and resettlement in South Omo: DFID took part in
DAG visits to Somali region and SNNPR (South Omo) in 2014, monitoring these Government of
Ethiopia programmes against international good practice on resettlement. These visits found
continued concerns, although in most cases service delivery was seen to have improved
compared to visits in 2012-13, in some cases being reported as being as good or better than the
services communities were previously receiving.
 Reporting findings, raising concerns and pressing Government of Ethiopia to follow international
good practice on resettlement: Including through Ministerial engagement, bilateral human rigths
dialogues, and DFID, together with the World Bank, representing the DAG in meetings on this
issue with the State Ministers for Finance and Economic Deveopment and Federal Affairs.
 Improving the ability of communities affected by CDP and resettlement in South Omo to express
their concerns and have them redressed:
o Supporting the increased coverage and effectiveness of the Grievance Redress
Mechanism, with fast-tracked attention to the Developing Regional States. The Gambella
regional office of the independent EIO was opened in December 2013 and a study to
identify areas to improve GRM’s accessibility and effectiveness is underway.
o Supporting the scale up of social accountability activities across the country to help
communities hold local government to account for the provision and quality of basic
services.
 Assuring ourselves that no DFID resources (including PBS) were being used to fund CDP:
Through continued management of fiduciary risk in our programmes, including PBS, to assure
ourselves that money was being used for the intended purposes.
 Applying appropriate safeguards to all DFID social sector programmes which operate in affected
areas, where relevant.
o An enhanced social assessment was carried out for PBS in 2014.
o A comprehensive social assessment was conducted for the education sector and its
recommendations have been incorporated into the programme (GEQIP).
o In health, a formula for the equitable distribution of medicines and other inputs has been
agreed with DRS receiving a greater share of support. DFID is currently undertaking a
health equity study to see how the emerging regions can be further supported to address
their poor health indices.
o In WaSH, DFID has agreed strict protocols for the government to follow to ensure that
any relocation is done through prior consultation and follows international good practice.
World Bank safeguards on indigenous people and resettlement apply to all donors
25
funding in the sector, including that of DFID. The World Bank will supervise and report
on the implementation of these safeguards to DFID during the programme’s lifetime. In
addition mechanism are in place to ensure that donors will only fund the construction of
WaSH facilities in CDP villages if there is evidence that the movement of people has
compiled with international good practice on resettlement.
In 2013, DFID considered whether that it would be appropriate to withhold its PBS funding from areas
where CDP is operational. DFID judged that it would be harmful and inequitable to do so, as these are
often the poorest areas of Ethiopia that are most in need of basic services. In addition, readiness and
quality of services has been one of DFID’s concerns in relation to the implementation of CDP and
something the Government of Ethiopia is seeking to redress. Withholding PBS funding from CDP areas
was therefore judged as undermining DFID’s efforts to ensure CDP is implemented in line with
international Good Practice and Guidelines on Resettlement.
It contines to be DFID’s view that it is better to try to minimise any risks from CDP (and resettlement in
South Omo) rather than seek to withhold PBS (or other DFID programmes) from these areas.
In terms of assessment of other risks over the last year, and their mitigation, key risks of note include:
Political risks


The risk that PBS is allocated based on political factors has continued to be monitored, verifying
that resources are allocated and disbursed according to the Federal Block Grant allocation
formula.
Political risks are likely to increase in 2015, given the elections in May. It is recommended that
this is reflected in the updated risk matrix.
Macro and policy risks

Macro risks have reduced in 2014, as inflation (which reduces the value of PBS) has been
remained in single digits, after hitting a peak in 2012.
Delivery risks

The main delivery risk to the FBG is that increased resources for recurrent expenditure in the five
sectors does not translate into improved services. Assessments during the year (particulary the
PSIA) provide reassurance that these resources are translating into impact. The distribution,
appropriateness, quality and effectiveness of basic services has also been assessed by
monitoring service availability, utilisation and outcomes through the PBS results framework,
sectors’ annual performance reports and facility mapping exercises, DHS and other analytical
studies (see below) and DFID’sparticipatation in the sectors’ field missions and talks with
beneficiaries at the community levels. Overall, this reinforces the positive picture of recurrent
expenditure supporting improving outcomes. However, concerns were highlighted in terms of the
effectiveness of Development Agents (see above). The risk matrix should be updated to note the
higher risk in this area.

Investing in analytical studies (i.e. Education SDI, Health SPA, DA research) informs our
assessment of risk in this area and seeks to improve the effectiveness of government policy, PBS
and our sector programmes. Further analytical studies of the PBS programme which are planned
or underway include a beneficiary incidence analysis, a comprehensive value for money analysis,
and an independent evaluation.

Delivery risks in relation to Component B relate in particular to the early stage of
institutionalisation of the GRM and social accountability approaches.

A further delivery risk has emerged in 2014, linked to uncertainty over whether donors will contine
to fund PBS. This should be included in the updated risk matrix.
Environmental and social risks

The main social risk that PBS might exacerbate existing inequalities was assessed throught the
PSIA, which found that, in the case of gender, wealth quintile and regional inequalities, PBS was
helping marginalised groups disproportionately. An enhanced social assessment is being carried
out and attention will be given to acting on its recommendations.
26
Fiduciary risks



The risk of PBS funds being used for purposes other than those intended has continued to be
low. The audited account for PBS funds in 2014 was unqualified. However some financial
management weaknesses were highlighted such as incidences of no bank reconciliations and
mis-postings of expenditures at the district level. Action has been taken on the findings and
regions now refuse to release PBS funds unless bank reconciliations have been conducted.
Additional guidance on expenditure posting has been circulated.
It should be noted that this is distinct from the concern that fiduciary risk is higher for other, nonPBS government funds, given concerns about weaknesses at the district level, particularly in the
DRS, with the the low level of follow up on audit issues being a concern. The Ministry of Finance
and Economic Development are developing a strategy with the regions to strengthen internal
audit capacity and improve audit follow up. This will be launched in April 2015.
In addition to the internal and external audits of PBS, further checks and controls have been
applied to ensure PBS funds are not being used for anything other than the programme’s
intended purpose. This includes: (i) biannual assessments of whether PBS is supporting the
principles of additionality, fairness, fiduciary probity and transparency; (ii) examination of reports
and beneficiary feedback from the social accountability programme; and (iii) DFID’s participation
in field supervision missions at the regional, district and community level. Consideration of the
audit findings, government’s performance against the PBS principles, social accountability
reports and observations from the field have given DFID a reasonable level of assurance. An ongoing workforce survey, part of which will examine payroll controls, will provide an important input
to risk assessment and management in this area.
Reputational risks

Reputational risk in relation to PBS has been high in 2014, given the allegations by individuals
from Gambella, with an Inspection Panel investigation in relation to World Bank PBS processes,
and a Judicial Review in relation to DFID’s ability to assess the Government of Ethiopia’s
commitment to civil and political rights. DFID is arguing that it has a robust and rational approach
to this assessment.
Outstanding actions from risk assessment
To further strengthen the management of risk in DFID’s support to PBS the following should be
considered (in addition to the measures already underway):
First revisions to DFID’s risk matrix:



Considering escalating the programme’s level of risk to high for the range of reasons set out above.
Considering adding an additional risk in relation to the changing nature of DP support to PBS.
Specifically the World Bank’s transition to P4R, which may have a different set of targets and
disbursement linked indicators than the current PBS results framework;
Considering adding a risk related to the apparent ad hoc nature of the various analytical pieces and
reviews across PBS, which may risk duplication, gaps and that findings of particular studies might
be overlooked. There needs to be a clear mechanism to assess and meet evidence needs. This
should cover how different analytical pieces are commissioned and used coherently. It should be
clear how and by whom the findings will be mapped out to help identify progress and issues across
PBS; and
Second, reviewing and updating the log frame and assumptions to better reflect both the
programme’s underlying theory of change, the revised risk matrix, the recommendations of the social
assessment and the level of risk ( i.e. the risk that the assumption does not hold and therefore the next
level in the log frame will not be achieved). Log frame assumptions should reflect key risks identified in
the risk matrix and then positioned at the appropriate level in the log frame. Some specific assumptions
that should be considered for inclusion in the logframe are:

Output 1: The UK and other DPs can continue to provide support to PBS until the end of the
programme;
27

Output 1 or outcome: GoE will continue to increase domestic revenue generation and expenditure
on basic services;

Output 3: Increased social accountability creates incentives for sustainable service delivery and
helps to lock in benefits of the programme for the long term;

Output 3: Sufficient space exists for citizens and civil society to engage with and influence
government over key service delivery issues;

Output 3: Citizens have confidence to engage in the space that does exist, particularly on budget
and service delivery issues; and

Output 3: GRM has the will and authority to support citizens to address issues that are raised.
F: COMMERCIAL CONSIDERATIONS (½ page)
Delivery against planned timeframe
In general, most elements of PBS are delivering as expected regarding its timeframe. The main
exception is M4R which has experienced delays. However, steps have been taken and progress is being
made. This area of the programme is expected to catch up with its schedule over the next year.
Performance of partnership (s)
Overall, partners are considering to be performing well, particularly the GoE and the World Bank. The
performance of the managing agent for ESAP has improved since the last review.
Asset monitoring and control
All PBS assets are governed by Ethiopia’s public financial management systems.
G: CONDITIONALITY
Update on partnership principles.
The Partnership Principles were assessed for Ethiopia and approved by the Secretary of State in
September 2014. The headline was that: “ Overall, we assess that the Government of Ethiopia remains
committed to the underlying principles of our partnership sufficient to continue financial aid. Our
concerns prevent us from considering a return to general budget support.”
A programmatic level Partnership Principles Assessment was then conducted in October 2014 to inform
the Secretary of State’s decision on the 2014/15 funding disbursement to PBS. This assessment
headline was that: “Overall, we assess that within the context of the PBS programme, the government’s
level of committed to the underlying principles of our partnership is sufficient for our 2014/15
disbursement to PBS.” The Secretary of State approved this assessment in October which allowed the
disbursement of £82m for PBS in November 2014.
Other conditions
Disbursement of PBS funds is also conditional on a satisfactory JRIS. The six monthly JRIS takes an
overall judgement made by DPs on the basis of a review of a) the two ex-ante tests of Additionality and
Fairness and b) a review of the core principles of effectiveness, sustainability, equity, transparency,
fiduciary probity and predictability. The May and November 2014 JRIS concluded that the GoE has met
these tests and upheld the principles satisfactorily.
The November 2014 JRIS concluded that the Additionality Test has been passed as follows:
1) Confirm that the Federal Block Grant allocation for 2014/15 (EFY 2007) in the Macroeconomic and
Fiscal Framework is reflected in the Budget Proclamation.
Result: The FBG allocation for the current fiscal year of Ethiopian Birr 51 billion is reflected in the
Federal Government Budget Proclamation.
2) Confirm that at least 95 percent of the Federal Block Grant budgeted in the 2013/14 (EFY 2006) was
disbursed to regional governments.
Result: FBG allocations to all regions in EFY 2006 were fully transferred.
28
3) Confirm that basic service spending as a share of sub-national government budgets in the current
fiscal year is not declining compared to last year.
Result: Regions’ allocation to basic services in EFY 2007 budget increased compared to EFY 2006.
The November 2014 JRIS also concluded that the Fairness Test had been passed. This verifies that: (i)
allocation rules are transparent; and (ii) disbursements from the federal to the regional governments and
then from the regional to district governments are at least 90% of budgeted allocations. Any discrepancy
greater than 10% must be explained. The Federal government fully transferred block grant allocations to
regional governments in EFY 2006. Except for a very few cases, all regional governments also
transferred 100% of the block grant allocations and all districts received at least 94% of their allocations.
H: MONITORING & EVALUATION
Evidence and evaluation
A large number of analytical studies are available in relation to PBS. Many have been conducted since
the last annual review or are at different stages:

Studies across the whole PBS programme. These include an EU verification mission; the World
Bank IEG report on PBS; and AfDB’s study of innovation in the delivery of basic services. In
addition, the World Bank’s PSIA covered many of PBS’ achievements, as did the PBS mid-term
review (MTR). Preliminary results of the MTR were presented at the November 2014 JRIS. The
original ambitious plans to conduct an impact evaluation of PBS in year 2, 4 and 5 have now been
scaled back as its primary aim of measuring the impact of the REF is no longer necessary. Instead it
has been agreed to conduct studies on PBS’s operational and policy effectiveness; demand side
utilisation and a beneficiary perceptions and satisfaction levels. These are currently being
commissioned to start in 2015. Plans of how these studies and the other analytical pieces can be
consolidation or synthesised to provide an overall picture of PBS’s impact are being discussed;

Studies of particular elements of PBS. These include a MTR and a proposed randomised control
trial of particular aspects of the ESAP programme; a study on links between FTA and SA; and a
SDI study on education, a SPA+ for health and DFID research on DA effectiveness; and

Studies which go beyond the confines of PBS. These include an analysis of the context of Payment
by Results aid modalities in Ethiopia; a comprehensive VfM analysis of PBS; PEFA and
regional/sector fiduciary risk assessments and an ongoing ICAI review which is looking at how DFID
measures impact of its programmes including PBS.
There is a risk that a ‘tsunami’ of separate studies will overwhelm GoE and not be fully utilised. It is
important in the future that the findings of these reports are synthesised in order to bring together their
findings and assess what gaps, if any, exist. Potentially, this could be built into the
consolidation/synthesis element of the final evaluation.
Monitoring progress throughout the review period.
This review was conducted by a three person team consisting of one consultant (Roger Drew) and two
DFID staff members (Scott Caldwell, Senior Governance Adviser in DFID Rwanda and Rachel Grant,
Policy Adviser in International Finance). It drew on a wide range of other review processes, including
particularly the JRIS/JBAR meeting in Addis Ababa from 10-13 November 2014 and the JRIS field
missions preceding it to Harari, Southern Nations, Nationalities and People’s Region (SNNPR) and
Tigray. In addition, the VfM section of this review is based on information supplied by the Itad team who
are conducting a VfM review of DFID Ethiopia’s Human Development portfolio and a comprehensive
analysis of PBS. The review was based primarily on analysis of existing documentation, supplemented
by meetings with key programme staff and stakeholders including DFID, MoFED, the World Bank, the
PBS secretariat, DPs, ESAP and COST.
29
Smart Guide
The Annual Review is part of a continuous process of review and improvement throughout the programme cycle. At
each formal review, the performance and ongoing relevance of the programme are assessed with decisions taken
by the spending team as to whether the programme should continue, be reset or stopped.
The Annual Review includes specific, time-bound recommendations for action, consistent with the key findings.
These actions – which in the case of poor performance will include improvement measures – are elaborated in
further detail in delivery plans. Teams should refer to the Smart Rules quality standards for annual reviews.
The Annual Review assesses and rates outputs using the following rating scale. ARIES and the separate
programme scoring calculation sheet will calculate the overall output score taking account of the weightings and
individual outputs scores
Description
Outputs substantially exceeded expectation
Outputs moderately exceeded expectation
Outputs met expectation
Outputs moderately did not meet expectation
Outputs substantially did not meet expectation
Scale
A++
A+
A
B
C
Teams should refer to the considerations below as a guide to completing the annual review template.
Summary Sheet
Complete the summary sheet with highlights of progress, lessons learnt and action on previous recommendations
Introduction and Context
Briefly outline the programme, expected results and contribution to the overall Operational Plan and DFID’s
international development objectives (including corporate results targets). Where the context supporting the
intervention has changed from that outlined in the original programme documents explain what this will mean for
UK support
B: Performance and conclusions
Annual Outcome Assessment
Brief assessment of whether we expect to achieve the outcome by the end of the programme
Overall Output Score and Description
Progress against the milestones and results achieved that were expected as at the time of this review.
Key lessons
Any key lessons you and your partners have learned from this programme
Have assumptions changed since design? Would you do differently if re-designing this programme?
How will you and your partners share the lessons learned more widely in your team, across DFID and externally
Key actions
Any further information on actions (not covered in Summary Sheet) including timelines for completion and team
member responsible
Has the logframe been updated since the last review? What/if any are the key changes and what does this
mean for the programme?
C: Detailed Output Scoring
Output
Set out the Output, Output Score
Score
Smart Guide
i
Enter a rating using the rating scale A++ to C.
Impact Weighting (%)
Enter the %age number which cannot be less than 10%.
The figure here should match the Impact Weight currently shown on the logframe (and which will need to be
entered on ARIES as part of loading the Annual Review for approval).
Revised since last Annual Review (Y/N).
Risk Rating
Risk Rating: Low/Medium/High
Enter Low, Medium or High
The Risk Rating here should match the Risk currently shown on the logframe (and which will need to be entered on
ARIES as part of loading the Annual Review for approval).
Where the Risk for this Output been revised since the last review (or since inception, if this is the first review) or if
the review identifies that it needs revision explain why, referring to section B Risk Assessment
Key points
Summary of response to programme issues raised in previous annual reviews (where relevant)
Recommendations
Repeat above for each Output.
D Value for Money and Financial Performance
Key cost drivers and performance
Consider the specific costs and cost drivers identified in the Business Case
Have there been changes from those identified in previous reviews or at programme approval. If so, why?
VfM performance compared to the original VfM proposition in the business case? Performance against VfM
measures and any trigger points that were identified to track through the programme
Assessment of whether the programme continues to represent value for money?
Overall view on whether the programme is good value for money. If not, why, and what actions need to be taken?
Quality of Financial Management
Consider our best estimate of future costs against the current approved budget and forecasting profile
Have narrative and financial reporting requirements been adhered to. Include details of last report
Have auditing requirements been met. Include details of last report
E Risk
Output Risk Rating: L/M/H
Enter Low, Medium or High, taken from the overall Output risk score calculated in ARIES
Overview of Programme Risk
What are the changes to the overall risk environment/ context and why?
Review the key risks that affect the successful delivery of the expected results.
Are there any different or new mitigating actions that will be required to address these risks and whether the
existing mitigating actions are directly addressing the identifiable risks?
Any additional checks and controls are required to ensure that UK funds are not lost, for example to fraud or
corruption.
Outstanding actions from risk assessment
Describe outstanding actions from Due Diligence/ Fiduciary Risk Assessment/ Programme risk matrix
Describe follow up actions from departmental anti-corruption strategies to which Business Case assumptions and
risk tolerances stand
F: Commercial Considerations
Delivery against planned timeframe. Y/N
Compare actual progress against the approved timescales in the Business Case. If timescales are off track provide
an explanation including what this means for the cost of the programme and any remedial action.
Performance of partnership
How well are formal partnerships/ contracts working
Are we learning and applying lessons from partner experience
How could DFID be a more effective partner
Smart Guide
ii
Asset monitoring and control
Level of confidence in the management of programme assets, including information any monitoring or spot checks
G: Conditionality
Update on Partnership Principles and specific conditions.
For programmes for where it has been decided (when the programme was approved or at the last Annual Review)
to use the PPs for management and monitoring, provide details on:
a. Were there any concerns about the four Partnership Principles over the past year, including on human
rights?
b. If yes, what were they?
c. Did you notify the government of our concerns?
d. If Yes, what was the government response? Did it take remedial actions? If yes, explain how.
e. If No, was disbursement suspended during the review period? Date suspended (dd/mm/yyyy)
f. What were the consequences?
For all programmes, you should make a judgement on what role, if any, the Partnership Principles should play in
the management and monitoring of the programme going forward. This applies even if when the BC was approved
for this programme the PPs were not intended to play a role. Your decision may depend on the extent to which the
delivery mechanism used by the programme works with the partner government and uses their systems.
H: Monitoring and Evaluation
Evidence and evaluation
Changes in evidence and implications for the programme
Where an evaluation is planned what progress has been made
How is the Theory of Change and the assumptions used in the programme design working out in practice in this
programme? Are modifications to the programme design required?
Is there any new evidence available which challenges the programme design or rationale? How does the evidence
from the implementation of this programme contribute to the wider evidence base? How is evidence disaggregated
by sex and age, and by other variables?
Where an evaluation is planned set out what progress has been made.
Monitoring process throughout the review period.
Direct feedback you have had from stakeholders, including beneficiaries
Monitoring activities throughout review period (field visits, reviews, engagement etc)
The Annual Review process
Smart Guide
iii