Annual Review - Summary Sheet
Title: Access to health care in the Democratic Republic of Congo (ASSP)
Programme Value: £185.2m
Programme Code: 202732
Review Date: July 2014
Start Date: 01/08/2012 End Date: 31/03/2018
Summary of Programme Performance
Year
2013/14 2014/15
(design
phase)
Programme Score
Risk Rating
A
M
A+
M
Summary of progress and lessons learnt since last review
Overall the programme has been scored as an A+ as it has already exceeded expectations set out in the
business case particularly on health system strengthening, is delivering on the major log-frame
milestones, and is on track to exceed log-frame outcomes.
The project has exceeded expectations on the degree of concrete engagement with the Ministry of
Health on national health systems strengthening, depth of partnership with the Ministry of Health
centrally, extent of success on policy engagement with the Ministry, the extent to which the project has
worked through government delivery mechanisms, the commitment of the project to identify and tackle
underlying systemic issues/failures, the extent to which a multi-sectoral integrated approach to achieving
health results has been taken, the emphasis on learning and piloting new approaches (building on
experience in previous projects) and the value for money achieved (in halving the cost per capita from 14
USD per person per year in the previous project to 7 USD and supporting 8.6 million people in 56 health
zones compared to the 30 zones expected in the business case)
The project has made good progress on the outputs that were planned for the first year of
implementation. Based on performance in the first year of implementation, it is likely that the expected
outcomes of the project (as measured by the key outcome log-frame indicators) will be substantially
exceeded by the end of the project. The quality of the programme was recognised by a recent ICAI
team visiting as part of their review of DFID’s work in FCAS, who, in initial verbal feedback following their
field visit, commented that this was one of the best health programmes the team had reviewed,
highlighting in particular its focus on health systems strengthening, its multi-sectoral/integrated approach
and well thought-through design.
Key challenges for ASSP in year two will be to ensure that improvements in drug and commodity supply
chain are sustained, provide system reform support and advocacy to assist the government to increase
the proportion of health facility staff receiving salaries and to improve quality and completeness of
information coming through routine government systems.
Key lessons learnt are that it is much more difficult to change project approach than to start support in a
previously unsupported area, there are trade-offs in terms of utilisation and quality when trying to make
existing government service delivery systems work, that DFID delays in issuing contracts have had an
on-going negative effect on value for money and results, that there is a trade-off between specialist
knowledge brought in through sub-contracting against the complexity/overheads added and there are
benefits from engaging directly with faith based organisation system and with a lead contractor who has
longer term perspective .
1
Summary of recommendations for the next year
IMA should by the end of year two:
i.
Update the action plan on pharmaceuticals supply chain. Going forward, the action plan should
include both an operational and a strategic plan including starting some procurement through
FEDECAM (the national drug procurement agency).
ii.
Develop a plan on how health zones will be transitioned through different phases of support
within the project (e.g. investment, development, and maintenance phases).
iii.
Build on year one’s successes in policy engagement with the Ministry of Health on faith based
providers to strengthen coordination of corels (faith based organisations) and facilitate further
discussions on the relationship between faith based providers and government.
iv.
Improve asset management systems within IMA and used by implementing partners (IPs) and
ensure that IPs have adequate security arrangements and communications equipment in place.
DFID should by the end of year two:
i.
Strengthen horizontal links between DFID programmes within geographical areas e.g. links
between the SGBV (Sexual and Gender Based Violence) work within DFID’s SSAPR police
programme and ASSP. (Responsible: DFID Health Programme Officer)
ii.
Continue to engage on donor coordination/harmonisation of support within Directions
Provinciales de Santé and explore other new co-funding opportunities for ASSP going forward.
(DFID Programme Coordinator)
iii.
Contract some third party monitoring using health programme funds if a cross-office contract has
not been tendered by the end of 2014. (DFID Programme Officer)
iv.
Put in place longer term advisory expertise to oversee construction (either through 10% cadre
input or as cross office- consultancy). (Programme Coordinator)
v.
DFID DRC should explore the feasibility of cross-office support for IMA to further develop their
existing conflict analysis and resilience planning (for example through the EACP contract). (Lead
Adviser)
vi.
Ensure that more of the health systems strengthening work that has been successfully taken on
by ASSP and which extends their terms of reference is now funded as a cost extension [to IMA’s
contract] within the existing envelope of the Business Case (now that the anticipated RCI
institutional strengthening project will not be contracted as a stand-alone contract).
2
A. Introduction and Context
DevTracker Link to
Business Case:
DevTracker Link to
Log frame:
http://devtracker.dfid.gov.uk/projects/GB-1-202732/documents/
http://devtracker.dfid.gov.uk/projects/GB-1-202732/documents/
Outline of the programme
Following an allocation of £300,000 for design work, the UK is providing £184.9 GBP million over five
years to strengthen basic health service provision in the Democratic Republic of Congo (DRC) in order to
improve reproductive, maternal, neonatal and child health (£179.7m), and to strengthen the capacity of
the central Ministry of Health to support service delivery (£5.2m). The figure below outlines the key
results expected from this programme as outlined in the business case together with progress to date.
Figure 1: Results expected from the programme (as outlined in the business case).
Expected results
Reduce U5 mortality by 50% in
target areas:
Vaccinate 64,600 one year olds
against measles each year.
Provide contraception - 155,000
CYPs (couple years protection)
cumulatively by 2014/15 and
355,000 CYPs by the end of the five
year programme.
Ensure that 75,000 births per year
are attended by skilled health
personnel.
Make sure 100% of health facilities in
target areas offer appropriate
emergency obstetric care.
Provide 75,000 pregnant women with
Intermittent Presumptive Treatment
(IPT) for malaria during ante-natal
visits.
Provide 600,000 people with access
to clean water and sanitation.
Progress after
year one of
implementation
Results awaiting
end-point household
survey.
232,781 in year one
Comment
65,473 in year one
Expected to exceed target in year two as
commodities were not in place until the
middle of year one following DFID’s delay
issuing the implementation contract.
213,716
Exceeded expectations
Exceeded expectations
Data awaited from
health facility survey
138, 698 in year
one
Exceeded expectations
Not planned for year
one
Final target has been revised to 374,400 as
cost estimates in business case were based
on Village Assaini phase I planning
estimates which UNICEF found to be
significantly underestimated subsequent to
the business case.
The Access to Healthcare business case funds have been contracted as follows;
a) £179.7m for the ASSP (Accès aux soins de santé primaires) project – this was awarded to IMA
World Health following an OJEU international competitive tender. SIDA additionally co-fund this
project, so the total value of the contract is now £182,899,146.
b) £1.3m for the DRC Demographic and Health Survey through a memorandum of understanding
with UNICEF (from February 2013 to March 2015).
c) £0.4m for Public Financial Management support. This has been contracted through OPM (under
DFID’s HEART PEAKS framework agreement) and provides embedded technical assistance in
the Ministry of Health. The 6 month initial support ends in August 2014 and we are now seeking
to extend support to the Ministry for a further 12 months.
3
d) Funds which are currently un-contracted (originally allocated for the RCI institutional
strengthening project in the business case): £3.4m. Prior to the RAR budget review in July 2014
we had planned to contract £2.8m of these for additional systems strengthening work in ASSP.
This is now under review pending clarification of the 2015/16 and outer years allocations.
The ASSP project is implemented by IMA World Health with a consortium of NGOs and supports 56
health zones (out of 515 nationally) in five of the eleven provinces in DRC, providing an estimated 8.6
million people with access to essential primary and secondary healthcare services. The design phase
began in October 2012 with implementation planned to run for five years from April 2013 to March 2018.
ASSP builds on DFID’s proven track record of improving access to health care and delivering health
results in the DRC as a result of its previous Access to Healthcare programme (A2H2), which ended in
March 2013. It differs by covering a significantly larger area (increasing coverage from 20 to 56 health
zones), by supporting government delivery systems rather than operating in parallel, focusing more on
health systems strengthening and adopting a more sustainable approach (by removing the direct
payment of project incentives to front-line health facility staff and shifting to subsidised user fees with
some exemptions rather than free health care).
IMA subcontract four NGOs (SANRU, World Vision, IRC and Caritas) as Implementing Partners (IPs) to
support the Ministry of Health deliver services through a mixture of government, faith-based and, in
some areas, private health facilities and build the capacity of the Ministry at provincial, district and health
zone levels. The consortium also includes Pathfinder International who provides technical assistance for
family planning and reproductive health across the project and Tulane University who lead a £4m
Operations Research and Impact Evaluation (ORIE) component.
In addition to supporting service delivery, the project has developed a strong focus on governance and
health system strengthening as a result of better engagement on both the part of the Ministry and
consortium lead than was predicted during business case design;
At community level, ASSP aims to increase awareness on expectations of levels of service, pilot
community score cards, build the capacity of CODESAs (Community Health committees) and is piloting
a “Health Village” scheme which motivates communities to achieve health outcomes on a range of key
targets e.g. on vaccination, malaria control, family planning and WASH (through Village Ecole Assaini).
ASSP is piloting a hybrid cash/agricultural Community Health Endowment scheme to improve local
revenue collection and pool risk/subsidise user-fees. The project has an integrated approach, containing
a large WASH (Water, Sanitation and Hygiene) component based on Village Assaini plus additional
activities on water quality/cisterns and a nutrition package including both prevention work (home
gardening and education) and treatment of malnutrition.
At health facility level, ASSP has a large construction programme (including 200 new health centres, two
hospitals and over 300 rehabilitations) and provides facilities with essential equipment, drugs/supplies,
training and an operational allowance. ASSP is also supporting the piloting of a national MoH system of
financial management for HGR (general hospitals).
At Health Zone management team level, ASSP is strengthening government capacity to manage health
zones through capacity building Equipes Cadres de Zones de Santé to improve supervision of health
facilities and use planning tools for the development of the health zone and supporting Conseils
d’Administration de Zones de Santé.
At Provincial/District level, ASSP is supporting the setting up of the new DPS (Directions Provinciales de
Santé) management teams and supports MIPs (Medeçins Inspecteurs de Province), Comités
Provinciaux de Pilotage), and is improving capacity and minor rehabilitation of CDRs (regional drug
depots). ASSP intends to start procurement through FEDECAM next year (the national drug
procurement agency).
At central level, work on the set-up and roll out of the new national health management information
system HMIS (based on the DHIS2 platform) and HR information system pilot (I-HRIS) are key building
4
blocks of improvements to management. The HR work is an entry point to negotiating norms and
standards of staffing levels and also potentially rationalisation and cleaning of the pay-roll. ASSP is
facilitating policy dialogue between faith-based providers and the Ministry of Health following negotiation
of a new MoU which outlines government commitment on salary provision, tax exemptions etc for faith
based providers. Faith based providers (mostly the Catholic and Protestant churches) provide over 40%
of care in DRC.
5
B: PERFORMANCE AND CONCLUSIONS
This annual review covers the first year of project implementation from April 2013 to March 2014.
Support to new health zones did not start until quarter two in year one (pending DFID issuing the full
implementation contract). This review was conducted in June/July 2014. Indicator data in the report is
given until the end of March 2014 and is collected from quarterly narrative reports. The annual review
process consisted of the following components;
- Half day workshop between DFID DRC (including cross cutting advisers) and IMA
- Two day partners workshop chaired by central MoH including SIDA, IPs, MoH provincial/district
medicin chefs and technical partners (including feedback session with implementing partners in
absence of IMA).
- WASH meeting with IMA and DFID WASH adviser
- ORIE oversight committee meeting with DFID and Tulane University
- Field visit to three health zones (Ilebo, Mweka and Katende) in Kasai Occidental (See reports
Quest numbers 4568786 and 4565653)
- Review of year one quarter four financial and narrative reports
- Feedback meeting with IMA and planned feedback to DFID DRC in programme board meeting.
Annual outcome assessment
The outcome for this programme is “increased coverage with essential reproductive, maternal and child
health services in DFID-supported health zones” and as measured by the following indicators:
Outcome indicator(s)
Milestone
(March 2014)
17/18 target
only
Progress
(March 2014)
Survey
planned at
end of project
Number of births attended by skilled health
personnel
Number of 1 year old children vaccinated
against measles
Number of sexual and gender-based
violence (SGBV) survivors who have
received PEP Kits within 72 hours
218,691
213,716
230,475
232,781
385
236
Number of pregnant women who receive
two doses of intermittent presumptive
treatment for malaria (IPTp)
Number of long-lasting insecticide-treated
bed nets (LLINs) routinely distributed to
pregnant women and children under one
year old (18 HZs)
Number of children under five and
pregnant women reached through DFID’s
nutrition-relevant programmes
137,948
138,698
58,092
33,362
Not applicable
year one
Not applicable
year one
59,625
65,473
Contraceptive Prevalence Rate
Number of Couple Years Protection
6
Comment
Couple Years Protection
target (see below) is already
exceeding expectations and
is proxy indicator for
contraceptive prevalence rate
Meets expectations
(98% of target).
Exceeds expectations
Lack of PEP kits in initial
quarters were the result of
delay DFID issuing the
implementation contract
required for procurement.
These are now in place and
training completed and on
track for year two. Treatment
of fistulae is also in the
project.
Exceeds expectations
Routine distribution
suspended by MoH in
Maniema during mass
distribution.
This component was added as
cost neutral budget
amendment in year one as
nutrition was not originally
planned within ASSP
Exceeds expectations
Based on performance in the first year of the project it is likely that the expected outcomes of the project
measured by the outcome indicators above will be exceeded by the end of the project.
Overall output score and description
The project has made good progress on the outputs that were planned for year one (as outlined in
section C). Overall, the programme has been scored as an A+ as it has exceeded expectations set out
in the business case in the following ways:
1. The degree of concrete engagement on national health systems strengthening. The ASSP
project was predominantly set up to support service delivery in a sustainable way. Some
institutional capacity building (predominantly though technical assistance to the Ministry of Health
centrally) was expected to be contracted separately (through the RCI project). ICAI reviewed the
project in June 2014 and in initial verbal feedback following their field visit, commented that this
was one of the best health programmes the team had reviewed, highlighting in particular its focus
on health systems strengthening, its multi-sectoral/integrated approach and well thought-through
design. Examples of valuable health systems strengthening (HSS) work that were not originally
anticipated in the ASSP design include the set-up and roll-out of the new national Health
Management Information System (HMIS) using the DHIS-2 platform and the work on Human
Resources for Health, including supporting the MoH to review staffing norms and standards
nationally, and then at provincial level reviewing staffing/payroll lists in a validation exercise and
setting up an HR Information System to better manage staff/trainings and certification records.
2. Depth of partnership with the Ministry of Health centrally. Previous DFID health projects, whilst
liaising with the MoH centrally, in general did not manage to engage with the Ministry centrally to
a significant level. In contrast, most components of ASSP are developed with the Ministry - i.e.
the project assists the Ministry to review its national policy/guidance - and then the project assists
the MoH in rolling this out across ASSP health zones. Examples of this include ASSP’s
approach to reviewing MoH family planning/reproductive health fiches techniques, reviewing
CODESA (health committee) guidance and the way in which ASSP construction designs for
health facilities have been adopted by the Ministry as template national plans.
3. Extent of success on policy engagement. IMA has used the credibility of ASSP’s large scale
support to government service delivery systems on the ground, and its long-standing relationship
with the Ministry, to good effect in leveraging policy discussions. Most notably IMA managed to
facilitate discussions between faith based providers and the Ministry of Health which led to the
negotiation of a new Memorandum of Understanding between the major faith based institutions
and the Government (which outlines commitments in terms of salaries, tax exemptions etc). This
is a major step forward in policy engagement in terms of a future model of provision in terms of
the relationship between state and providers. This is strategically important as it could eventually
lead to more formalised commissioning of services by the state in the future.
4. The extent to which the project has worked through government delivery mechanisms. This
means for example that the project ensures that all trainings are delivered by the Ministry and
drugs are distributed through government supply chains. (DFID does not pass funds directly
through the Ministry of Health centrally). DFID’s previous projects tended to provide some
support in parallel (as this is often faster and ensures quality in the short term) rather than using
and attempting to improve government systems. A good example of this is the approach used in
the project to distribute drugs through CDRs (MoH regional drug depots) rather than an NGO-led
push system. It continues to be an uphill struggle to make this pull system work, but ASSP has
additionally invested resources in upgrading and capacity building CDRs: this which was not
specified in the terms of reference, but has significant long term value.
5. The commitment of the project to identify and tackle underlying systemic issues/failures. IMA
continues to seize opportunities to harness ASSP resources to address underlying health system
inadequacies/failures. For example, responding to continued concern over government hospital
7
management issues, IMA has developed, together with the Ministry of Health, an IT system for
hospital management (which integrates the financial, pharmaceutical and HMIS data systems)
which the MoH will roll out nationally if the ASSP pilot is successful. Another example is the
way in which ASSP seeks to mitigate/provide a local solution where the MoH is unable to pay
staff salaries regularly by piloting community health endowments. These essentially work as
agricultural cooperatives, the income from which enables health facilities to generate primes
(incentives) for staff and also subsidise user fees for families in the scheme i.e. acting as a form
of mutuelle/health insurance scheme.
6. The extent to which a multi-sectoral integrated approach to achieving health results has been
taken. Whilst funds for WASH activities were expected in the project, nutrition activities were not
(beyond general health promotion). IMA identified in year one that DFID would not reach its
result targets if nutrition was not addressed as it has such a major impact on child morbidity and
mortality. ASSP used a cost-neutral budget amendment to incorporate a significant component
on nutrition (with a focus on behaviour change including home based gardening). This is
anticipated to reach 1.1 million children under five and pregnant women by the end of the project.
ASSP has also identified that household air-pollution from cooking is amongst the top three
preventable risks for reducing child mortality and plans to pilot the use of clean cook-stoves after
year one.
7. The emphasis on learning and piloting new approaches (building on experience in previous
projects). DFID has benefited from IMA’s previous experience delivering projects for other
donors, and this has been built on in the design of ASSP interventions. A good example of this is
IMA’s previous experience in WASH combined with their findings during the needs assessment
which indicated that water quality remains an issue (even with capped springs) which has led to a
“hybrid approach” to Village Assaini being piloted. Similarly, findings from initial assessments on
nutrition in ASSP revealed that embarking on standard CMAM approaches using Ready to Use
Therapeutic Food (RUTF) where commodity supply chains could not be ensured after donor
funding ends may do harm in the long run as clinic staff turn away children (as they don’t have
supplies) rather than offering basic nutrition advice and follow-up. This has informed the nutrition
package now being rolled out in ASSP which includes, at health centre level, behaviour change
and home based treatment with high energy porridge. Operations research projects have been
set up to evaluate early results from both these approaches, which are novel in DRC.
8. Value for money achieved in terms of population served and cost per capita. Compared to
expectations as set out in the business case ASSP has significantly exceeded expectations on
cost per capita and population reached. In comparison to the old Access to Healthcare
programme, ASSP was a scale up in terms of funding of 83% for service delivery, but ASSP has
delivered an increase of 247% increase in population reached. The business case anticipated
that these funds would support around 30 zones but ASSP has supported 56 in its first year. The
cost per capita for support to basic service delivery has halved, from $14 per capita per annum in
the previous project to $7. These economies were achieved by changing the model of service
delivery (working more through government delivery systems rather than in parallel). Our
estimation is that whilst this change in model has resulted in some initial reduction in utilisation
rates in previous DFID project zones, if the project achieves its final target rate of 0.6
consultations per capita per year, the overall result will be an increase of 73,180 people using
healthcare per £1m invested by DFID.
Key challenges for ASSP moving forward
In year two of implementation, the key challenges for ASSP will be to;
a. Ensure that improvements in drug and commodity supply chain are sustained. IMA should
ensure that the drug and commodity stock-out issues experienced earlier this year continue to be
addressed pro-actively. Drugs are crucial in ensuring good utilisation rates and stock-outs place
an additional burden on patients (as they have to pay for drugs externally in addition to clinic
8
user-fees during stock-outs). The initial procurements were delayed waiting for DFID to issue the
full implementation contract. Although IMA then made prompt procurements and accelerated
transportation, it took some time before stocks reached the CDRs (government regional drug
depots). The project uses government distribution systems based on a pull system where
facilities make requests against credit lines.
Both DFID and IMA under-estimated the length of time and difficulty involved in switching from a
push to a pull system. In April 2014 DFID asked IMA to produce an action plan to improve the
drug supply chain in ASSP health zones. This was completed during a joint workshop between
ASSP partners and the MoH. This plan now needs updating to track progress made and assign
clear responsibilities and timeframes for further actions. Family planning commodity issues in
particular need to be prioritised and it is important that ASSP works to improve monitoring of
stock-outs and visibility over the ability of facilities to draw down on their credit lines. IMA should
also ensure some contingency stocks e.g. of anti-malarials which rely on other projects such as
PMI. At the request of the MoH, IMA plan to move around 25% of procurement through the
national procurement agency FEDECAM in year two. This could exacerbate stock issues in the
short term, but is likely to pay long term dividends in building capacity of FEDACAM.
b. Provide system reform support and advocacy to assist the government to increase the proportion
of health facility staff receiving salaries. No single government line ministry has control over this,
so this will be extremely challenging and ASSP has only limited influence (predominantly through
improving information to allow the government to prioritise who is paid and how).
c. Improve quality and completeness of information coming through routine government systems.
Currently the preliminary results from the large baseline household survey are awaited and the
roll out of the new HMIS system is in its very early stages, so the project is lacking key
information. Once information does start coming through routine systems it will take many years
to improve the quality and completeness of data coming through and this will be a particular issue
in DRC where routine data flows and connectivity have been effectively non-existent.
In the longer term, it will be important that construction/rehabilitation work goes ahead on schedule as
infrastructure impacts on utilisation and hence results. It will also be important that support for revenue
generation in health zones started in ASSP (for example through user fees and community health
endowments) are translated into a financing model that can be sustained by government in the longer
term together with autonomous internal system management capability which can be sustained after
support is withdrawn.
Key lessons learnt
It has been more difficult to operate in health zones previously supported by DFID than the new ones.
New health zones (some with no comprehensive donor support for decades, if ever) have shown
dramatic improvements in utilisation rates and other key health indicators after just nine months of
implementation. The 20 zones supported under DFID’s previous A2H2 project have had a major
adjustment in terms of the new policies on the phased withdrawal of primes for health facility staff, the
move from an NGO-led push to a government-run pull system for drugs and the introduction of
subsidised user fees. DFID and IMA had predicted that utilisation rates would fall as a result. Utilisation
rates have now plateaued and IMA expect them to start recovering in year two. In the long run, DFID’s
calculations on VfM show that even if final utilisation rates in these zones are lower than under A2H2,
that more people will use services per £1m invested under the new project. This is outlined in more detail
in section D on VFM. More broadly, the challenges faced in the “old” health zones may indicate that
some degree of dependence was caused by the previous projects and highlights the need to consider
long term sustainability of projects. There were some key lessons learnt on the logistics of the transition
process from the previous Access to Health 2 project to ASSP. These have been documented in a
document accompanying the Year One Annual Review - see quest number 4556726. These relate in
particular to management of assets and drugs.
9
There are trade-offs in terms of utilisation and quality when trying to make existing government service
delivery systems work. It is easier and faster to deliver higher utilisation rates and improve short-term
quality of services using parallel systems: this makes them very attractive for those concerned with
demonstrating short term outputs. It will take some years of the new approach before ASSP starts
delivering services with similarly high utilisation rates/service quality to the previous projects: but the
reward for the change is the exponential increase in coverage. DFID underestimated the time taken for
the switch from a push to a pull system in the health zones supported under the previous project and the
complexity/challenge of making the existing government drug distribution system improve.
DFID delays in issuing contracts have had an on-going negative effect on value for money and results.
There was a delay in issuing the full implementation contract for ASSP. This has had knock on effects
throughout year one with the delays in procuring pharmaceuticals and commodities impacting on
utilisation rates. A key lesson is that contract delays impact on results and hence value for money:
therefore DFID country offices are right to expect high levels of service on procurement centrally, and to
be open where this has not been received.
Designing the log-frame prior to tender may be too early to be optimally effective. It would have been
better to agree log-frame outputs after the project was contracted and not during the business case prior
to tender1. ASSP has exceeded expectations on major health systems strengthening and community
level interventions and it has been difficult to compress the various components of the project into the
pre-tender log-frame. With hindsight, WHO’s six pillars of a health system would have been a more
effective framework to monitor the project, as it has developed a major health systems character, in
addition to delivering on basic service delivery and health outcomes.
There is a trade-off between specialist knowledge brought in through sub-contracting against the
complexity/overheads added. The technical sub-contracts may not have been the most effective way of
contracting the research and family planning components. See detailed discussion in section F on
performance of partnerships. In summary, it may have been more appropriate for DFID to contract the
research component directly and for the consortium lead to have scaled up in-house capacity on family
planning rather than contract it to another organisation. As an example of potential to develop in-house
capacity, the Community Empowerment and Accountability component originally planned as a subcontract to another INGO was taken over by IMA following performance issues in the design phase.
Whilst it has taken some time for IMA to do the inception work on this, there are now promising signs
that this component will produce some valuable work, and has generated much interest from the
Secretary General in the Ministry of Health.
There are benefits from engaging directly with faith based organisation system and with a lead contractor
who has longer term perspective. It has been apparent that IMA’s primary long term objective is to build
DRC’s health system rather than having an internal focus on organisational growth/objectives. They
have used ASSP as an opportunity to address systems issues which will be a barrier to the country
moving forward.
Assumptions which have not held.
A number of assumptions have not held in respect of this business case, mainly related to the
anticipated budget in outer years of the project;
a. It was assumed that DFID would contract a separate health systems strengthening project
(described as the RCI project in the business case). In October/November 2014 a decision was
taken that, given the good progress made on HSS and opportunities that were presenting within
ASSP, it would be more cost effective for DFID to invest those funds (around £2.8m GBP) in
ASSP to expand the integrated HSS work in the project (mainly on HRH, HMIS and support to
the MoH centrally). IMA has embarked on HRH and HMIS work in anticipation of this contract
1
The changes in the DFID SMART rules which give greater flexibility in the timeframe for finalising log-frames
support this finding
10
extension. To date it is unclear whether budget pressures within the office will permit these funds
to be contracted to ASSP.
b. The recent expected changes to the budget were not anticipated. The indicative RAR allocation
(i.e. for 15/16) and outer years of the project look likely to be lower than forecast in IMA’s
contract. We are anticipating that annual forecasted disbursements will be reduced for ASSP
and the project extended by one year. It is likely that this will result in early transition out of
some health zones.
c. DFID DRC contracted £4.7m to UNICEF for a two year project to support CMAM implementation
alongside the ASSP project but it is now unlikely that further funds will be identified for RUTF
(Ready to Use Food) after the UNICEF project ends, as originally expected. For this reason the
UNICEF project is currently being reconfigured – this is discussed in more detail later in the
review.
Dissemination of lessons learnt from this Annual Review will be through presentation within DFID DRC
and a planned article on Insight (DFID’s intranet). DFID DRC presented some learning on the
programme during the fragile states session in the Health Adviser CPD conference in early 2014 and a
SmartRules case study on procurement about ASSP has been published. ICAI are likely to make
recommendations in their forthcoming report on DFID’s work in Fragile States following their review of
ASSP in June 2014.
Key actions See summary recommendations on page one.
Has the log-frame been updated since the last review?
Yes. It was not possible to finalise and approve the log-frame indicators during the design phase,
because of the time required to agree which indicators to include (i.e. discuss with the MoH/other donors
which HSS activities were priorities and were feasible), time to establish baseline data from the SNIS
(National health information data) as the data required cleaning (with agreement on which data to
exclude and how) and finally, it took time to monitor demand (which was affected by new user fee tariffs
and availability of drugs) in order to predict realistic targets on indicators affected by utilisation. As a
result of this, the log-frame and milestones were not approved until May 2014. The log-frame will need
to be revised once there is agreement on which health zones will be transitioned out and when, as this
will affect milestones/results: this, in turn, depends on the conclusion of DFID’s RAR process, and any
contract amendment negotiations.
11
C: DETAILED OUTPUT SCORING
The log frame outputs for ASSP were set up as part of the business case process prior to the tender and
are relatively limited, reflecting DFID’s expectations on what could be achieved based on experience
with partners in the previous programme, which focused more, by design and nature, on health service
delivery rather than health systems strengthening. Therefore it has been challenging to condense all the
varied activities into the existing four outputs. ASSP itself manages the programme according to the six
WHO pillars of health systems strengthening. Whilst the indicators themselves, under the key outputs
have been adapted to the project, the four outputs remain.
For the purposes of this report the various components of the project are reported under the following
outputs;
Figure 2: How project activities correspond into the four ASSP outputs
No.
1
Output
Enhanced health service delivery and quality in
DFID-supported health zones
2
Increased empowerment and accountability in
health service planning and delivery in DFIDsupported health zones
Activities
 Equipping facilities and management teams
 Pharmaceutical supply chain
 Capacity building of health facility staff and
health zone management teams
 Improving family planning
 Nutrition activities
 Infrastructure construction/rehabilitation
progress



3
Improved access to health services in DFIDsupported health zones
-
4
Increased and sustainable access to safe
drinking water, improved sanitation, hygiene
education and better environmental health in
rural and peri-urban communities
-
Community empowerment and
accountability component
Set up and roll out of national Health
Management Information System
Human Resources for health work (to
address salary issues)
Standardisation of tariffs/user fees,
Community health endowments (agricultural
mutuelles)
Policy engagement on relationship between
the Government and faith based providers
WASH activities
The ORIE (Operations research and impact evaluation) component is reported under the section on
Monitoring and Evaluation. Activities in the business case contracted outside ASSP are commented on
immediately below (as these smaller components are not included in the log frame for ASSP):
Demographic Health Survey (UNICEF)
The Demographic Health Survey started implementation at the beginning of 2013, after delays during the
planning process. DFID is providing its contribution to the DHS through a Memorandum of
Understanding with UNICEF. The Minister of Plan chairs the steering committee on a quarterly basis and
this meeting is attended by other donors and ICF Macro, the technical partner for implementation of the
DHS. A technical committee also meets once a fortnight to discuss operational issues. The preliminary
12
findings of this study have now been successfully published and demonstrate significant improvement in
some key indicators, in particular infant/child mortality.
Public Financial Management (HEART PEAKS)
As part of the institutional capacity building component of this business case, technical assistance is
being provided to the DRC Ministry of Health on Public Financial Management (PFM). Good progress
has been made so far, with a technical assistant embedded in the Ministry from the end of February
2014 and a more senior consultant providing direction and oversight through a number of short visits.
Building on previous consultancy expertise this assistance has so far helped the Ministry, DFID and
other donors to better understand the real issues affecting public financial management in the health
sector. For example, analysis of budget execution data has clearly shown that the Ministry does in fact
spend the majority of the funding made available to it, contrary to the understanding of some donors, but
that the link between budgets and plans remains very weak. The technical assistant has also started to
build links between the Ministry of Health and other DFID supported programmes such as the Public
Financial Management and Accountability (PFMA) programme which is supporting central government
finance reforms which will impact on the health sector.
Output 1
Enhanced health service delivery and quality in DFID-supported health zones
Output number per LF
1
Output Score
A+
Risk:
Medium
Impact weighting (%):
40
Risk revised since last AR?
No
Impact weighting % revised
since last AR?
No
This output focuses on (i) ensuring that the full package of services is gradually rolled out in facilities (as
recommended in the government’s minimum and comprehensive packages) and (ii) that the quality of
service delivery is improved. This output relies heavily on all pillars of the health system functioning and
therefore includes all the capacity building work in the project.
At the time of writing this review, data is only available on one of the four indicators for this output
(whose performance was directly affected by the delay in issuing the implementation contract).
Therefore the assessment of this output has drawn upon wider information on progress on the relevant
activities in the work-plan (as outlined in figure 2 i.e. equipping facilities and management teams,
pharmaceutical supply chain, capacity building of health facility staff and health zone management
teams, improving family planning, nutrition and infrastructure construction/rehabilitation).
Indicator(s)
Number of facilities offering Comprehensive
Emergency Obstetric and Neonatal care (LCS,
blood transfusion etc)
Number of new acceptors of modern methods of
family planning (KPI)
Milestones
2013/2014
112
Progress (to May 2014)
107,863
88,779. This indicator was
affected by delay in procuring
commodities as a result in the
delay from DFID issuing the
implementation contract.
Data awaited from end of
project household survey
Percentage of children under five with diarrhoea
who received either ORS or RHS
NA
Percentage of Health Facilities reporting drug
stock-outs for at least one of the following five
Data will not be
available until year 2
13
Data awaited from health
facility survey in year two
Not applicable
drugs (Depo Provera, Oxytocin, SP, Zinc,
Amoxicilin 250mg)
Key Points
ASSP has managed to support health care services across a population of almost 8.6 million people in
its first year of implementation. ASSP is already demonstrating good results from improved services as
demonstrated by good performance of outcome indicators such as skilled birth attendance, vaccination
coverage, presumptive treatment of Malaria in pregnancy and couple years protection for family
planning. This is exceptional, given that roll out of a new project over such a large programme area in
this context would probably reasonably be expected to take two years and normally one would not
expect improvements in quality until after equipment, supplies and trainings have been put in place
during the first year, and clearly reflects IMA’s strong track record and leverage in DRC.
IMA has demonstrated the ability to conduct large scale procurements of supplies and equipment and
then import and distribute items efficiently2. Time taken to clear imports is considerable faster than
expected and distribution to Provinces was rapid (in part from necessity because of lack of warehouse
capacity in Kinshasa). The drug stock-out issues in year one were mainly because i) DFID failed to issue
the full implementation contract in a timely fashion, which delayed placement of the initial order and ii)
because of the policy of distributing through the government pull systems using CDRs – i.e. taking a
more long-term sustainable approach.
Capacity building of health facility staff and health zone management teams has begun and feedback
from health zone management teams has been positive. Teams now have vehicles to conduct
supervisions and operational allowances but there are opportunities to promote more supervision by
DPS and health zone teams. Training on family planning and reproductive, neonatal and maternal
health in particular were extensive with over 50 MoH “fiches techniques” being revised prior to this.
Progress on infrastructure has been slower than originally anticipated. But this has been partly because
the Ministry of Health were involved in designing and approving the construction plans as they wanted to
adopt these nationally – and have done so. Moving forward the number of new constructions and
rehabilitations has been revised down due i) the leveraging of extra resources through the inclusion of 80
health centres in ASSP health zones under the Government’s PESS project (which builds and equips
health facilities) (ii) contraction of timeframe resulting from time taken to agree designs and (iii) revisions
of cost estimates which were impacted by increased specification requested by the MoH. Section D on
VFM provides a detailed breakdown on revised targets and unit costs. Feedback from the Ministry of
Health and from the annual review field visit has highlighted that construction is both a strategic priority
for the Ministry and that lack of ownership of facilities imposes an additional cost barrier for users as
facilities are forced to raise user fees to cover rental of buildings from private landlords.
Nutrition was not included to any major extent in the terms of reference but activities were added as a
cost-neutral budget amendment after IMA identified that this was a significant issue which would hold
back delivery on infant/child mortality results. IMA have started rolling out an package of activities which
have strengthening home/community capacity on malnutrition at their core in order to promote long term
sustainability. The package includes behaviour change communication on nutrition including promotion
of household gardens and home based treatment of malnutrition using high energy porridge. This work
has drawn attention from other donors and the Ministry and is the subject of an operations research
study by Tulane University.
Family planning activities have made a good start with strong demand for longer acting methods of
contraception (as demonstrated by the Couple Years Protection target exceeding expectations in the first
year). Pathfinder have worked through the Ministry of Health to update national protocols/deliver training
to health workers and have worked more broadly on a range of reproductive health issues including
SGBV and treatment of fistulae (which exceeded expectations as this was not included in the ToR).
2
Confirmed by paragraph 12 Stefan Dercon’s Congo Note 2 report following visit in 2013 to DRC.
14
Summary of responses to issues raised in previous annual reviews
Issue raised in previous review
1. IMA need to give on-going high priority to good
coordination and effective communication
between consortium members, supported by
clear definition of roles.
Response
Communication has improved in general. Terms of
reference for the ORIE and FP components have
now been agreed. Quarterly partners’ meetings have
been strong with emphasis on holding partners to
account on delivery of work plans.
2. IMA should give high priority to articulating the
programme and its approach to stakeholders,
both directly and indirectly through its
implementing partners. At the same time, DFID
DRC should seek to maintain as high a level of
interaction with provincial authorities as possible,
in order to strengthen awareness of the
programme at that level and generate
commitment to it.
IMA and DFID have been working on a
communications strategy. DFID has had frequent
engagement with authorities in Kasai Occidental in
particular. DFID plan to travel to South Kivu in July
2014 and will meet with provincial authorities as this
province in particular has had limited engagement
with DFID.
3. IMA should develop and present an action plan An action plan has been developed in conjunction
for managing this risk (particularly in Maniema with the Ministry of Health and stakeholders.
where drug management is thought to particular
need of strengthening) in order to avoid stockouts and minimise fiduciary risk.
Recommendations
- IMA should update the action plan on pharmaceuticals supply chain. Going forward, the action plan
should include both an operational and a strategic plan with plans for starting some procurement
through FEDECAM (the national drug procurement agency).
- Develop a plan on how health zones will be transitioned through different phases of support within the
project (e.g. development, consolidation and maintenance phases).
- DFID should contract some third party monitoring using health programme funds if a cross office
contract has not been tendered by the end of 2014.
- DFID should put in place longer term advisory expertise to oversee construction (either through 10%
cadre input or as cross office- consultancy).
Output 2
Increased empowerment and accountability in health service planning and delivery in
DFID-supported health zones
Output number per LF
2
Output Score
A+
Risk:
Medium
Impact weighting (%):
20%
Risk revised since last AR?
No
Impact weighting % revised
since last AR?
No
Indicator(s)
Number of assisted health centre in which a community score card is
completed each year (The score card will include measurement of community
satisfaction and health services fees)
Number of health zones implementing IHRIS
Number and Percentage of HZs using DHIS 2 for Routine Health Information
reporting
Milestones 2013/2014
Not planned until year 2
Not planned until year 2
Not planned until year 2
This output covers community level activities in the empowerment/accountability component of ASSP,
the work on setting up and rolling out the new national health information system (DHIS-2) and the work
on Human Resources for health (which focuses on using information to improve the proportion of staff
who receive salaries).
15
Key Points
The Community empowerment and accountability component includes three main activities;
reinvigoration of CODESAs (health committees), piloting community score cards and piloting an SMS toll
free hotline for citizens to report fraud and service problems. IMA has taken time to build on learning
from the DFID-funded Tuungane programme and has conducted some initial research to inform
implementation. The Annual Review field visit found that at Mutombo Dibue health centre and
Munkamba referral health centre, there was strong commitment and ownership by the community, and
the CODESA was very dynamic, being practically involved in drug management and seeking solutions to
a range of problems experienced in running the health centre.
The work on Human Resources for Health was not originally expected within the project, but was
developed after IMA identified that the lack of government salaries of health care workers would affect
project results and sustainability long term. IMA have been working in conjunction with IntraHealth to
assist the MoH to revise norms and standards on staffing levels using the WHO WISN methodology3.
Plans to roll out an HR information system (I-HRIS) have been deliberately slowed down to ensure that
the work on validating staffing lists is done in conjunction with work planned with the Ministry of Public
Service and the World Bank. This is a very challenging area of work and its inclusion in the project has
exceeded expectations in terms of what DFID expected (which was limited to an expectation of local
advocacy to include staff on the payroll).
ASSP does not pay primes (incentives) to health facility front-line staff, as this is unsustainable. Instead,
to maintain health worker motivation, a standard tariff for user fees together with improved access to
drugs/commodities has meant that clinic revenues have improved so that staff are able to take an
increased “prime locale”. This has exceeded expectations (there had been fears that staff would be
unmotivated with the policy on no project primes). The annual review field visit found that all health
facilities in the three zones visited had experienced a significant increase in their local revenues,
especially at the hospital level than at the health centre level. For example, in llebo, the “prime locale”
for a physician working at the general referral hospital increased from 50.000 FC to 200.000 FC, the
prime locale for nurses increased from 17,000 FC to 30.000 FC and in Mweka health centre, the prime
locale for nurses rose from 2000 FC to 10,000 FC. Alongside this, Tulane University are conducting
some operations research on health worker motivation, in particular examining the case of the transition
of support in health zones under the previous project.
No specific recommendations were made in the last review on this output.
Output 3
Improved access to health services in DFID-supported health zones
Output number per LF
3
Output Score
A
Risk:
Medium
Impact weighting (%):
20%
Risk revised since last AR?
No
Impact weighting % revised
since last AR?
No
Indicator(s)
Utilisation per capita per annum
(disaggregated by sex) for curative
consultations
Number of new consultations of
pregnant women and children
under five (curative consultations)
Milestones 2013/2014
0.37
Progress
0.34
Not available
Data not available
Improving access to health services in ASSP is addressed through activities to ensure tariffs/user fees
are regulated, clinics have alternative sources of income to subsidise fees through piloting of community
3
Workload Indicators of Staffing Need
16
health endowments (agricultural mutuelles) and facilitating policy engagement between the Government
and faith based providers on what contribution the MoH makes to faith based providers.
Key Points
The starting point of access to health services is their availability. This project has extended support of
health services to 56 Zones, significantly more than the 20 of the predecessor project, or the 30
projected in the business case. The utilisation per capita progress above is in the context of the late
signing of the full implementation contract, meaning that full implementation did not start in all health
zones until the second quarter of the year. Unhindered by this, utilisation would probably have materially
exceeded the year one milestone, and is on track to exceed the eventual target.
DFID’s general policy is universal or free health care. However, as DRC is a uniquely challenging
context in terms of public financing of basic services and in view of the serious lack of prospects for
medium-term sustainability on the government side of the kind one would conventionally move towards,
DFID agreed that the use of regulated user fees (with subsidies and exemptions) is a more sustainable
locally adapted version of universal health care.
The programme has supported the setting of health-zone wide tariffs for user fees negotiated with
beneficiaries and provincial authorities. This is set out in the ASSP strategy on user fees and access for
vulnerable patients. In general, the revised tariffs are considerably lower than prior to the project. The
annual review field visit found that the prices of consultation and medical interventions have significantly
decreased: for example the fee for a caesarean section decreased from 130USD to 17USD in Ilebo
Health zone, the management of an episode of malaria in Mweka Health zone has decreased from 8000
FC to 1000 FC and the fee for major surgery has decreased from 80,000 FC to 15,000 FC in Katende
Health Zone.
Of critical importance for results and value for money is ensuring that utilisation rates are adequate and
that the poorest and most vulnerable are still able to access care. DFID assessed initially that the
benefit of the increase in coverage that the new programme has delivered through its efficiencies
outweighs the risk of decrease of uptake per user as a result of user fees. Early modelling by DFID
showed that this would not be the case below about 0.4 consultations per person per year in ASSP and
therefore it is crucial that utilisation rates continue to rise in the second year of the project.
IMA is monitoring the elasticity of demand and barriers to access on utilisation in a number of ways. IMA
reports to DFID on utilisation rate by health zone in the health zone score board of four key indicators as
part of the quarterly narrative reports, so DFID has good visibility on differences at health zone level.
Monitoring use of health care services by the poor and vulnerable is being assessed as part of the user
fee operational research study being conducted under the ORIE component by Tulane University. This
will ensure appropriate steps will be taken promptly within the project to ensure that fees are not a barrier
to access.
Progress on improvement in utilisation rates varies according to whether the health zone was supported
under the previous project or not. In new health zones utilisation rates have increased markedly. The
annual review field visit found that despite some shortages for some medicines, people are satisfied with
the availability of essential drugs in all health facilities visited and this has encouraged patients to
continue to use health services; For example, in Mweka health zone, the utilisation rate of curative
services increased from 7-8% to 13-14% during the first nine months of ASSP’s implementation. The
number of new patients rose from 40 to 170 per month. In the health centre at Mutombo Dibue, the use
of services increased from 12% to 30%. The general referral hospital of Katende has seen the number of
monthly visits increased from 60 to 350-400 since the beginning of the project.
As expected by DFID, health zones supported under the previous project experienced a fall in utilisation
rates with the change in policy under the new project on the reintroduction of user fees, phased
withdrawal of frontline staff primes and moving to a government based pull drug system which has
disrupted drug supply. Currently, utilisation rates overall stand at about 0.34 consultations per patient
17
per year with the final target for the project being 0.6 (which is intentionally lower than the 0.65-0.7 target
under the previous project).
Community health endowments. This component is piloting the use of community cooperative
agriculture which generates health facility revenue for local primes and provides subsidised care for
contributing households (effectively acting as a risk pooling mechanism). The pilot is being followed by
an operations research study. Early findings show that communities may be concealing true yields from
the fields and therefore changes have been made in terms of agreement of a minimum rather than
percentage contribution to health facilities. Some communities have adapted this to a hybrid cash-based
mutuelle as some households prefer to contribute cash rather than labour. At this stage it is too early to
predict the results of this pilot. The findings are likely to generate significant interest as few mutuelle
schemes have been successful in the poorest most rural areas of DRC.
Policy engagement on faith based provision has exceeded expectations in the degree of progress that
has been made. IMA facilitated the negotiation of the revised Memorandum of Understanding between
the major faith based institutions and the Government (which outlines commitments in terms of salaries,
tax exemptions etc.). IMA has also updated a mapping/assessment of faith based services and has
engaged on mobilising state support (for example ensuring equal support for faith based facilities
through the government’s PESS project). Moving forward the challenge will be to invigorate coordination
of the Corels (faith based organisations) and then to continue to keep discussions open on the potential
for future commissioning of services by Government through faith based providers. DFID has continued
to prompt IMA that this policy engagement is a strategic priority, which was unexpected to IMA as this is
not an area that donors have engaged on before: IMA, as a faith-based organisation with extensive
government-collaboration experience, is uniquely well-placed to deliver in this area.
Summary of responses to issues raised in previous annual reviews None
Recommendations
IMA should further develop policy engagement with the Ministry of Health on faith based providers to
strengthen coordination of corels (faith based organisations) and facilitate further discussions on the
relationship between faith based providers and government.
Output 4
Increased and sustainable access to safe drinking water, improved sanitation, hygiene
education and better environmental health in rural and peri-urban communities
Output number per LF
4
Output Score
A
Risk:
Medium
Impact weighting (%):
20%
Risk revised since last AR?
No
Impact weighting % revised
since last AR?
No
Indicator(s)
Number of villages which have completed the Village Assaini process in
targeted Health Zones;
Number and Percentage of population (rights holders) with access to safe
drinking water; (KPI)
Number and Percentage of population (rights holders) who have access to,
and use, adequate sanitation;
Milestones 2013/2014
Not planned until year 2
Not planned until year 2
Not planned until year 2
Key Points
The WASH activities were delayed by a delay in DFID approving the work-plan and approach within
ASSP. To date, DFID has mainly funded the Village Assaini approach in WASH in DRC. Whilst it has
18
demonstrated good results, it is quite an intensive, time-consuming process and is quite costly.
Currently, less than 15% of DRC has benefitted from Village Assaini and it remains unclear whether the
relevant government line ministries will have capacity to continue to roll out this approach in the absence
of external donor support in the long-run. Findings from the initial assessment within ASSP also showed
that issues of water quality were being insufficiently addressed within the Village Assaini package.
After some discussion, DFID approved IMA to roll out a mixture of WASH activities including both Village
Assaini and some “hybrid” approach sites with water quality interventions including filters and cisterns.
The hybrid approach proposed by IMA is novel and therefore Tulane University is running an operations
research study alongside to compare the cost-effectiveness of both approaches. IMA will need to
ensure that where Village Assaini is rolled out, that they adhere strictly to the VEA phase II guidance as
set out by the Ministry of Health and fully consider the community level work within the package.
The end of project milestone is for 370,400 people to access water and sanitation. This is lower than the
forecast number in our original business case (which showed for a budget of £17m DFID expected
600,000 to access water/sanitation). This is because the original estimates done by DFID were based
on Village Assaini phase I, which underestimated costs considerably. Based on a budget of £17m, the
total cost per capita in ASSP currently stands at £45 per person (compared to £30 per person for
UNICEF). This is higher because ASSP also includes activities over and above Village Assaini i.e. in
particular the construction of latrines/cisterns/filters in health facilities and provision of cisterns/social
marketing filters. When these are removed, the planned cost for VA are considerably lower (depending
on the definition of what is included in the VA package). This will need to be monitored during the life of
the project. Moving forward, there is a risk that the WASH component may need to be scaled down due
to budget pressures resulting from the RAR allocation.
Summary of responses to issues raised in previous annual reviews None
Recommendations The budget will need reviewing moving forward and further analysis on cost per
capita (excluding institutional support) should be conducted as part of that.
19
Gender
ASSP has now finalised its gender strategy which sets out clearly the objectives of the project (Quest
number 4501070). The 2013 ASSP gender review (using standard DFID DRC methodology) gave the
programme an amber rating along with recommendations for improvement (see Columns 1, 2 & 3
below). Column 4 summarises actions taken by ASSP in year one of implementation and the updated
score agreed in the gender review meeting which was conducted as part of the annual review process
on 25th June 2014.
Criterion
Observations (2013)
1. Does the
programme design
include
comprehensive
analysis of the
situation of women
and girls?
Amber: The programme design
includes a degree of analysis of the
situation of women and girls, but this
cannot be considered to be
comprehensive. It was anticipated
that this would be included in the
baseline situational analyses
performed in each of the health
zones, but it is not clear that this has
been done. There is as yet no specific
gender strategy for the programme.
2. Does the
Amber: The logframe includes
logframe include
indicators that relate specifically to
results and
women and girls, and disaggregation
indicators relating to of indicators by age is planned.
W&G, and those
However, the current logframe does
that include
not fully capture transformative
transformative
change for women and girls (although
change?
utilisation rates disaggregated by
gender will do so to an extent).
3. Does the Theory
of Change explicitly
refer to W&G
throughout?
Amber: The proposed revised TOC
presented by IMA refers explicitly to
women and girls, but does not
demonstrate how outcomes for
women and girls are expected to be
achieved, and what change will occur
as a result.
4. Does programme Amber: Implementation plans
have elements that demonstrate some awareness of
seek to address
discriminatory socio-cultural norms,
discriminatory
but this is not yet sufficiently
socio-cultural
systematic, nor is it clear yet to what
norms?
extent and how these will be
challenged through the programme.
5. Does programme
contain specific
measures on
adolescent girls?
Green: The programme contains
specific measures for providing
services to adolescents, including
sexual and reproductive health
services.
Recommendations
from 2013
A gender strategy
should be developed,
and be based upon
comprehensive
analysis of the
situation of women and
girls in the health
zones to be supported.
As the community E+A
component is further
developed, at least one
indicator that captures
the transformative
impact of this for
women and girls
should be considered
for inclusion in the
logframe.
A specific theory of
change for gender
should be developed
as part of the gender
strategy for the
programme.
The gender strategy
should include an
analysis of such norms
in the health zones
(HZ) that will be
supported and present
realistic plans for
challenging these
through the
programme.
Plans for meeting
needs of adolescents
now need to be
operationalized, and
should as much as
possible be included in
the community
empowerment and
20
Actions Taken 2013-14
Amber Green: The ASSP gender
strategy has been developed and
approved by DFID. It includes a
comprehensive situational analysis
of women and girls for the five
provinces where ASSP is
operating.
Green: 1) ASSP has included a
proxy indicator for delaying first
pregnancy by working with the
MOH to disaggregate FP data by
age group, including a dedicated
indicator within SNIS i.e. number of
acceptors < 20 years of age.
2) ASSP IPs will measure and
disaggregate data on women/girls
access to selected health services.
3) An E+A gender indicator was
created for the “Village Santé” to
track representation of women in
CAC (communities committees).
4) Pathfinder’s programme includes
education on masculinity as part of
its work on VAWG (Violence
against Women and Girls).
Green: A gender-specific Theory of
Change was developed by
Pathfinder and is included in the
ASSP project Gender Strategy.
Green: The gender strategy
includes a detailed situational
analysis of discriminatory sociocultural norms for the five
supported-provinces. The strategy
also includes behaviour change
approaches and realistic plans to
address such norms. Links to La
Pépinière have been made by
DFID.
Green: The integration of
Adolescents and youth sexual and
reproductive Health (AYSRH) in the
ASSP-supported HZs is planned for
the year 2 of the project. Pathfinder
is currently providing support to the
MOH to update/adapt the AYSRH
training materials on SRH Youth
Criterion
Recommendations
from 2013
accountability
component. The
programme should
reflect on the extent to
which provision of
SRH services to
adolescents is
constrained by the
legal framework, and
discuss with DFID
where there may be
opportunities to seek
change.
6. Does the
Green (provided this is implemented): A clear job description
programme have
During the annual review, IMA and
should be developed,
appropriate and
Pathfinder confirmed that a gender
in order to obtain the
adequate plans and lead will be recruited in the coming
appropriate scope and
resources for
weeks, and that this individual will be level of gender
gender technical
responsible for overseeing
expertise.
support?
development and implementation of a
gender strategy for the programme.
7. Do
implementing
partners
demonstrate a
strong commitment
to gender issues,
have adequate
gender capacity and
a clear gender
action plan?
Observations (2013)
Amber: The evidence for this is
currently incomplete. A gender lead
and a gender action plan are not yet
in place, but commitments to these
have been provided by the
consortium.
8. Are the
Amber: These commitments have
programme-specific been partly addressed.
commitments in the
Action Plan for
Gender?
OVERALL
AMBER
Actions Taken 2013-14
Friendly Services and Peer
education. ASSP will need to
continue to address data issues on
monitoring this and clearly define
adolescents.
Green: A technical Gender advisor
has been hired since November
2013. She supported the
development of the ASSP gender
strategy and is overseeing its
implementation. A help-desk
literature review was commissioned
by DFID on reaching adolescents
with sexual and reproductive health
services.
The level of
Amber Green: The ASSP gender
commitment to gender strategy includes a gender traffic
issues will need to be light table that will make possible to
further assessed at the track progress made by IPs on
next and subsequent implementing the gender strategy,
annual reviews.
starting in Year 2. The IPs have
been briefed on the gender strategy
including the gender traffic light
table and have integrated specific
gender activities in their Year 2
work plans.
The commitments
Amber: Program specific
need to be addressed commitments in the Action Plan for
in the gender strategy Gender are being implemented.
for ASSP.
These efforts will be strengthened
with the integration of AYSRH
activities.
GREEN
21
D: VALUE FOR MONEY & FINANCIAL PERFORMANCE
Key cost drivers and performance
DFID’s total expenditure for the financial year 2013/14 has been £27.0m with total expenditure to date
running at £35.3m out of a total budget of £184.9 million. £25.7m of the 2013/14 expenditure was
through the ASSP programme, and £1.3m was used to support the latest District Health Survey4. The
latest financial reports show that the major programme costs incurred by IMA in year one of ASSP were
for health supplies, payments to support health zone management and equipment costs as well as
transport and staff costs as shown in the table below. This represents a change from the previous annual
review as the programme has now moved into its implementation phase. IMA’s expenditure is higher
than DFID’s expenditure due to a payment made in the design phase in the previous financial year.
Budget Category
Health Supplies
Malaria Supplies
Health Equipment
Health Zone support payments
Training
Health Activities, Meetings and
Studies
Community Health Endowment
Solar Equipment
WASH
Procurement and Logistics
Construction & Rehabilitation
Staff and Communications
Management Fees
Total
Total Y1 Expenditure
by IMA
% of Y1 expenditure
by IMA
£8,347,376
£895,229
£3,179,207
£3,972,426
£1,331,745
£1,151,441
£47,307
£643,512
£1,900,140
£3,551,206
£201,054
£3,644,858
£1,790,106
£30,655,606
27%
3%
10%
13%
4%
4%
0%
2%
6%
12%
1%
12%
6%
100%
In total in ASSP about 20% is spent on indirect costs (including management fees), with another 20%
spent on health systems strengthening and the remaining 60% going towards service delivery.
VfM performance compared to the original VfM proposition in the business case
Cost per capita and population covered have exceeded expectations.
In comparison to the old Access to Healthcare programme, ASSP was a scale up in terms of funding,
with an increase of 83%, but this delivered a more than proportionate scale up in terms of the population
covered, with an increase of 247% by increasing support from 20 to 56 health zones, giving a coverage
of 8.6m people. The project exceeded expectations significantly as the business case estimated that at
around 30 zones would be supported from this envelope of funds. The cost per capita for support to
basic service delivery has halved, from 14 USD per capita per annum in the previous project to 7 USD.
DFID achieved such a large increase in the population covered by changing the model of service
delivery (working more through government delivery systems rather than in parallel) and changing
supplier through an international competitive tender. Our estimation is that whilst this change in model
has resulted in an initial reduction in utilisation rates in the 20 ‘old’ (former A2H2) zones, if the project
achieves its final target rate of 0.6 per capita per year, the overall result will be an increase of 73,180
people accessing healthcare per £1m invested by DFID.
4
Expenditure on PFM falls outside the period of this review (to March 2014).
22
The business case identified that the main factor which affects cost-effectiveness is the coverage of the
intervention i.e. uptake of the specific services. This is affected on the demand side by user fees and on
the supply side by the quality and success encouraging uptake e.g. family planning. The business case
compared different levels of donor support from 3 USD to 13 USD per capita to provide a package of
interventions including child & maternal health, nutrition, family planning, malaria and water & sanitation.
The different levels of support were compared by estimating the expected Disability Adjusted Life Years
(DALYs) averted which informed the decision to select a mid-cost managed programme model rather
than continuing with the previous high cost programme or switching to a lower cost model. Given the
above, the business case highlights that the uptake of services will be key to ensuring VfM of the
selected model.
The business case also sets out a number of measures to be used to track VfM during the programme.
Some of this will be tracked through the operational research component specifically looking at VfM.
The figure below shows the breakdown of expenditure in year 1 between direct, indirect and capital costs
and what this equates to per capita based on an estimated target population of 8.6 million. Direct costs
include all drugs, equipment, training as well as the cost of procuring and delivering these interventions.
Indirect costs cover all programme staff and office expenses and management fees. Capital costs cover
all infrastructure and equipment such as solar panels and vehicles. The figure below shows an
estimated 6 USD per capita although in reality it will be closer to 7 USD in year two given that full
implementation did not start in all health zones until the second quarter of the year. This illustrates a
significant change in comparison with the much higher per capita expenditure under the previous
programme and shows that the programme is currently spending roughly in line with the per capita level
showed to be most cost-effective in the business case.
Figure 3: Direct, indirect and capital investments per capita in year one
Y1 Expenditure
Per capita
Direct costs
£18,234,194
$4
Indirect costs
£7,067,013
$1
Capital costs
£5,354,399
$1
Total
£30,655,606
$6
The following figure shows an estimation of how infrastructure costs have changed over the inception
phase of the programme. While some unit costs have increased, as plans for what new health centres
should look like have been finalised, there has also been a reduction in the overall total due to the
government infrastructure programme targeting some ASSP health zones. The cost of constructing
health centres through ASSP is currently estimated at 93,000 USD per structure or 465 USD per square
meter. This compares favourably with construction costs seen in DFID’s police reform programme of
510-650 USD per square meter and has attracted significant interest from the DRC government.
Figure 4: Construction costs as compared to original costs
Minor Repairs HCs
Original
Construction
Targets
156
Original
Estimated Unit
Cost (£)
6,000
936,000
Revised
Construction
Targets
125
Revised
Estimated Unit
Cost (£)
6,452
Major Repairs HCs
156
12,903
2,012,903
125
19,355
2,419,355
New HCs
260
3
53,019
13,785,032
200
55,484
11,096,774
322,581
967,742
2
322,581
645,161
Rehab HGRs
104
12,903
1,341,935
52
19,355
1,006,452
Build new HGR Buildings
78
45,161
3,522,581
52
45,161
2,348,387
Repair nursing schools
52
16,129
838,710
25
16,129
403,226
Incinerators
624
780
486,452
624
780
486,452
Placenta Pits
624
323
201,290
624
323
201,290
Total
2057
24,092,645
1829
Construction
New HGRs
Original Total
(£)
806,452
19,413,548
Difference
23
Revised Total
(£)
4,679,097
Assessment of whether the programme continues to represent value for money
This programme has shown good, if slightly delayed, progress in getting up to full implementation over
the course of the first year and has demonstrated that it can manage expenditure at this scale in the
challenging context that DRC provides. Overall, we are content that the programme is on track to deliver
value for money over the course of the intervention.
The delay in hitting utilisation targets will have had an impact on the VfM of this intervention this year but
this was expected and is in large part due to the change of approach of using the state drug depots
(CDRs) rather than providing drugs directly through an NGO. This approach may take longer to get fully
functioning but will have a much more sustainable impact, well after this programme has ended.
We will need to follow utilisation rates, along with other indicators of coverage, with care to check that by
year 2 of the programme we are meeting the expectations on utilisation rates as well as delivering the
strategy set out in the business case in terms of strengthening the health system. Through the
Operations Research component contracted to Tulane University, DFID is funding an OR study to look at
outcomes for different levels of donor investment per capita using field data from the ASSP project and
other donors’ projects.
Quality of financial management
Despite large variations in quarterly invoices, DFID in fact disbursed almost exactly what was forecast in
year one of the programme. The variations against quarterly forecasts were driven by delays to the
construction component of the programme and uncertainties over when invoices for large drug and
equipment orders would arrive. Given the scale and complexity of the programme we are satisfied that
financial management systems that have been put in place are improving and will now be looking for
greater accuracy in quarterly forecasts.
Reporting requirements have been met although the arrangements for exactly what the narrative reports
should contain has been an on-going process. Financial reporting has similarly gone through a number
of iterations over the year but has now settled on a format which works well.
Auditing arrangements have now been put into place following the recommendations of the Due
Diligence Assessment with an internal audit function to start next year.
Date of last narrative & financial reports
Date of last audited annual statement
June 2014
July 2014
E: RISK
Overview of programme risk
Achieving results
The key risks to DFID achieving the planned results were set out in the Business Case:
- Insufficient commitment from the Ministry of Health (MoH) at all levels (central, provincial and zonal)
particularly financial commitment on salaries;
- Insecurity from conflict reducing access;
- Corruption and diversion of funds particularly from sub-contracted partners or operational funds
delegated for use by health management teams;
- Provincial / zonal health authorities not having sufficient capacity to manage or monitor basic service
provision to ensure adequate quality care;
- Weakness of data collection, quality and processing for monitoring and evaluation making it difficult
to judge programme effectiveness.
24
DFID and IMA have secured strong MoH commitment at the central level, and strong commitment from
Government at Provincial and Zonal level, both publicly and in practical terms. This greatly mitigates the
principal risk identified in the Business Case. The most major concern is whether progress will be made
on government salaries. Currently, it is not uncommon to find only 30% of staff in a health facility
receiving a salary. IMA’s activities on human resources for health are ambitious and even if these
activities are well implemented, the salary situation may not have improved by the end of the project
without strong government commitment for change.
Insecurity from conflict remains a risk and there have been occasions when implementing agencies have
had to temporarily reduce their support to certain zones. To ensure a consistent approach to these
situations IMA with implementing partners are developing a Business Continuity Plan (BCP). An
assessment of conflict was made during the needs assessment to ensure that they are aware of
political/ethnic tensions in the individual areas where they operate. Together with the programme risk
matrix that IMA maintains, these will form a package to assure the resilience of DFID’s investment.
There remains an opportunity for IMA and implementing partners to do more formal conflict assessments
but realistically, DFID would need to provide technical assistance for further in depth work as this is not a
core capacity of health NGOs. Communications equipment of IPs was found to be limited on the annual
review field visit and IMA should review this.
Corruption and diversion of funds by implementing partners is an on-going risk. We consider that the
planned and existing programme checks and controls (implemented as outlined in the business case)
provide adequate assurance that UK funds are not lost to fraud or corruption. Anti- bribery and
corruption policies are in all contractual documents and DFID’s zero tolerance approach to fraud is
understood by partners. IMA conducted anti -bribery and corruption training for partners to reinforce the
policy provision in their contracts. This is linked to regular financial forecasting and reporting, backed up
by internal and external audits.
DFID’s financial oversight of the project will be strengthened with the planned recruitment of an A2
finance officer for the office. DFID continues to ensure that the Consortium Lead and implementing
partners have robust policy and financial procedures. The results and recommendations of the first of
the annual external audits are being finalised. IMA has hired a Deputy Director of Operations to oversee
IMA’s procurement and logistics departments, including management and tracking of assets, which are
held mainly by partners.
Effective data collection is a challenge, both in terms of technical collection and receiving it on a regular
and timely basis. Systems are being put in place to improve processes mainly for the MoH to better
manage resources through the roll out of the new HMIS system, but this will take time. Data quality
reviews (RDQA) are planned for year two to enable MoH officials to audit quality of data received from
health zones. DFID should additionally set up some third party monitoring capacity to do spot checks
themselves.
Outstanding actions from risk assessment
A Due Diligence Assessment (DDA) of IMA identified a number of opportunities to optimise the
management of fiduciary and other forms of risk in ASSP. Audit and resilience are the two areas in the
DDA where there is still work to complete. IMA has made progress on creating its internal audit function.
The Director of Compliance and Internal Auditor positions have been filled. The internal auditor will be
based at IMA’s headquarters and will report to the board of directors. His first assignment will be in the
DRC to focus on the ASSP program. External Audits of the downstream partners have just been
completed and the report and recommendations will inform the planned full programme audit. DFID has
requested an external project audit and is in the process of agreeing terms of reference for this.
Overall risk rating: Medium
25
F: COMMERCIAL CONSIDERATIONS
Delivery against planned timeframe
There was a significant delay in DFID issuing the implementation contract for IMA which has meant that
full implementation across all zones, which was expected to commence in January 2013 did not
commence until July 2013 – implementation in the previous 20 health zones commenced in April 2013.
Agreement and approval of a number of key components of the project took longer than expected in
particular approval of WASH activities, community empowerment and accountability component,
Community Health Endowment component. This was due to the time taken to reach agreement
between DFID with IMA on approach. This was also affected to some degree by lack of clarity on
expectations from DFID and the time taken to consult/draw on expertise of cross-cutting advisers within
DFID. (See lessons learnt point on importance of space to pilot/take risks and learn within the project).
Figure 5: Timeline of key decisions for ASSP
Step
Expected
date
Approval of strategic
case
Approval of BC
Actual date
Comments
Quest
number
20/09/2011
25/05/2012
Director level approval
Approval by SoS
3214149
3531143
Preferred bidder
selected
02/07/2012
13/07/2012
OJEU ICB
Design contract
awarded
30/8/2012
18/10/2012
Only £15m contract awarded
(Staffing continuity issue in PRG)
End of design phase
Breakpoint approval
31/12/2012
28/02/2013
28/02/2013
19/03/2013
Implementation
contract awarded
01/03/2013
05/06/2013
01/04/2013
01/04/2013
5 months of design phase (longer
than predicted 3 month period)
Design phase report approval
Delay from March to June due to
change in policy requiring higher
level approval.
Implementation started in April for
existing zones, July for new
zones
End of
design
phase
14/02/2014
Drafted by DFID on behalf of IMA
4367453
21/10/13
Time taken to reach agreement
on approach (i.e. extent of use of
Village Assaini).
4202350
End of
design
phase
October
2013
Design work by PACT the original
partner was not accepted and
decision made for IMA to
implement this themselves.
End of
design
phase
10/10/2013
Pilot approved with approval for
extensions of pilot 17/4/14
Implementation phase
Delayed design
deliverable:
ORIE ToRs
Approval of WASH
component
Approval of
Community
Empowerment and
Accountability
component
Approval of CHE
(Community health
endowment
component)
End of
design
phase
26
3736131
3911194
3763721
4189995
4446681
Delayed design
deliverable:
Approval of log-frame
End design
phase Feb
2013
DFID and IMA underestimated
time required to collect and clean
SNIS (national data) for indicator
baselines
25/5/14
4457436
In terms of impact of these delays on results, since the log-frame was not finalised until May 2014, most
milestones already reflect the delays e.g. no WASH milestone was set for the first year of the project.
The RAR budget revision has meant that to reduce in year expenditure, it looks likely that DFID will
require ASSP to reduce annual forecasts on expenditure and the project will be extended by one year to
retain the original value of the contract. Reducing in year expenditure will have an impact on results as
delayed investments in construction in year two for example mean that full utilisation rates won’t be
realised until later in the project and the number of health zones supported will also be reduced meaning
that direct:indirect ratios will be less good value (as overheads will remain): in short, removing elements
of an integrated programme will result in reductions in value for money disproportionate to the funds
saved. It will be important for DFID DRC to analyse the benefits of projected alternative uses of these
funds relative to this opportunity cost.
Performance of partnerships
There are currently nine direct project partners in the consortium that is implementing ASSP. As
consortium lead, the role of IMA is to manage delivery by downstream partners, set the overarching
operational framework for the programme, to manage essential activities such as procurement and
infrastructure development and provide technical oversight/strategic direction for the project (whilst
working closely with the Ministry of Health centrally). Technical partners are responsible for providing
the programme with specific expertise in key areas, and for supporting the implementing partners to
deliver in each of those areas. The implementing partners in turn are responsible for directly providing
support to service delivery at health zone level.
IMA continues to perform very strongly as the consortium lead. In particular, it has exceeded
expectations on providing technical direction and strategic oversight to the programme. Whilst many
funds have fund management agencies which can administrate grants to implementing partners and
carry out routine monitoring of implementation, IMA has gone much further, providing strong technical
oversight, setting clear directions for IPs (based on government policy), investing resources in
understanding the blockages/issues within the health system and then proactively designing and
implementing interventions working with the Ministry to address those.
IMA has built up a very strong relationship with the Ministry which has resulted in strong Government
ownership. Officials from the Ministry have on a number of occasions referred to ASSP as “their” project
and have been very actively involved in chairing quarterly partners’ meetings, doing joint field visits and
contributing staff to work on key components e.g. HMIS roll out. Components are developed in
conjunction with the MoH before being rolled out so that the project is actively contributing to building the
government health system rather than just seeking to implement the project. For example work to
revitalise CODESAs (community health committees) began by assisting the MoH to redraft their
guidance and the construction designs for new health facilities in the project have been adopted by the
MoH as a blueprint for future government construction.
IMA have demonstrated strong management skills in overseeing implementation partners and feedback
from IPs indicates has been generally favourable, particularly now that the pattern of disbursements has
become more regular. IPs commented that they received good technical support but that communication
could be improved e.g. planning of visits from IMA/technical partners to set up new work on health
systems strengthening or nutrition for example.
Performance of implementing partners at provincial level has been fairly strong with some variation in
capacity between organisations. In particular, SANRU (a local NGO) has performed strongly and has
27
demonstrated flexibility in taking on the newer components in the project as they have been introduced
e.g. nutrition.
Oversight of the technical contracts has been more varied and probably reflects more on the programme
design than IMA’s capacity. With hindsight, it is unclear whether setting up the Operations Research
and Impact Evaluation component as a sub-contract was the most optimal way of contracting this. DFID
was reluctant to procure and manage a separate contract. It was felt that as the emphasis was on
operations/action research i.e. learning and applying findings within the project meant that there was a
benefit to the research institution being managed directly by the consortium lead. In practice, the
management and oversight of a large research contract is highly specialised (something which even
DFID centrally contract out additional technical support for) and it was unrealistic to expect a health NGO
to be able to do this. In practice, IMA has been flexible and DFID has practically assumed some of the
technical oversight of this sub-contract, whilst IMA continues to do the contract/financial management.
For example, DFID drew up terms of reference for Tulane, drafted an improvement plan and set up the
oversight committee to ensure strong governance and independence of the research produced.
It is unclear whether there are concrete advantages of subcontracting the family planning component as
a separate sub-contract to another INGO. DFID did not specify this as a requirement, only that this
component had ring-fenced funding and activities. This has conferred an additional layer of
administration and overheads when IMA could probably have developed capacity for this in-house.
Going forward it may be worth reviewing the cost-benefits of this arrangement particularly with additional
cost pressures from the RAR budget revisions.
Feedback on how DFID could be a more effective partner has highlighted a number of key issues. The
importance of setting out expectations more clearly at the outset in writing (for example on quality of
research products) was emphasised and the need for DFID to be explicit on approvals
procedures/processes that the project is required to follow (for example on break point approval and
SEQAS research approval). Implementation of a number of components in the project was significantly
delayed pending approval from DFID on the proposals submitted (see timeline of key decisions).
Maintaining momentum in decision making is vital to keep the project moving forward and there are
points at which the lead adviser needs to take a decision/set direction potentially in the absence of
consensus between cross cutting advisers in the office. Finally, the setting up of both financial and
narrative report formats/processes proved unnecessarily protracted and with hindsight, it would have
been more effective for partners to propose formats for DFID’s approval, rather than the other way
round.
Partnerships between DFID funded projects should be explored geographically to exploit synergies.
DFID should strengthen horizontal links between DFID programmes within geographical areas e.g. links
between the SGBV (Sexual and Gender Based Violence) within DFID’s SSAPR police programme and
ASSP. Potential partnerships with other donors should also be explored further (beyond the funds
already committed by Sweden). DFID should continue to engage on donor coordination/harmonisation
of support within DPS (new provincial health districts) and explore other new co-funding opportunities for
ASSP going forward.
ASSP interfaces with a number of other health projects. The project has been affected by supply chain
issues in PMI’s malaria project which provides bed-nets for health facility use for inpatients and malaria
drugs. DFID intends to follow up with USAID. Alongside ASSP, DFID has begun funding UNICEF for
two years to support CMAM (Community based Management of Acute Malnutrition) in 39 health zones
and support child health days (for vitamin A and deworming). At the moment, discussions are underway
to refocus the CMAM component of this project as two assumptions have changed. Firstly funding for
commodities of RUTF (Ready to Use Therapeutic Food) cannot be assured beyond an initial 18 months
and secondly, ASSP is now piloting home based management of malnutrition with high energy porridge
in conjunction with the ANJE (infant feeding) and home gardening behaviour change work.
DFID’s centrally funded project implemented by Liverpool Tropical School on neglected tropical diseases
has had delays moving from the assessment phase to implementation phase. IMA have maintained
28
close contact with this project and the RTI neglected tropical disease project to monitor how they
interface.
Asset monitoring and control
The programme has inherited assets from the previous Access to Health 2 project and purchased a
considerable number of assets and will continue to acquire more in the year two. The construction and
rehabilitation of health facilities will expand in the year two and will need careful documentation. The
consortium lead – IMA has recruited a director of operations who has, among other responsibilities, the
management of ASSP assets. Spot checks on assets are routinely made on field visits. IMA has yet to
share with IPs good practice guidance on keeping and maintaining programme assets on the field to
comply with DFID policy. IMA should improve asset management systems within IMA and used by
implementing partners.
G: CONDITIONALITY
Update on partnership principles
There is not a project MOU in place with the Government and this has been deferred pending a country
level agreement between DFID and the GoDRC. We have applied an assessment of the Partnership
Principles to ASSP, although DFID DRC is yet to produce a formal assessment for the DRC country
programme. We consider that the commitment of the Government to health sector development,
commitment of the Secretary General to strengthen accountability/governance within the sector,
engagement of the Ministry of Health on improving Public Financial Management and improving
transparency through roll out of new information systems (such as HR information and HMIS) is very
promising. The project works through strengthening government systems and the Ministry has taken
strong ownership over the project. We feel that this provides us a sound basis for continuing our
partnership with the Government of the DRC through ASSP.
H: MONITORING & EVALUATION
Evidence and evaluation
To date, there have been no key changes to evidence that have had implications on the programme
design or implementation. In the past year, a Theory of Change (TOC) document was completed. It has
not been directly examined in the past year’s monitoring and evaluation. The TOC, in particular the
health systems strengthening programme model will be focus on the mid-term process evaluation.
Within the overall ASSP budget, DFID has allocated £4.7m for an Operations Research and Impact
Evaluation sub-component. This subcomponent is implemented by Tulane University as a sub-contract
to IMA. An impact evaluation is planned for this programme, and the baseline study fieldwork was
conducted from March-June 2014. Data is currently being processed and preliminary results are
expected in October 2014.
Since the initial set up of the subcontract, DFID and IMA realised the complexity of managing a research
contract, which may be more than is realistically possible from an implementing agency. Therefore, since
September 2013, DFID has become more closely involved in the oversight and direction of the ORIE
component, ensuring in particular that the governance and quality assurance of the impact evaluation is
aligned with DFID evaluation policy. IMA has been receptive to DFID’s deeper involvement in the
management of the research component of the project, as it reflects recognition from all parties that
since this is a specialised function requiring appropriate technical support.
29
An Improvement Plan for the ORIE component was put in place in December 2013, stating outputs and
deliverable dates for the Impact Evaluation, Operational Research and the Governance and
Management of the ORIE. In June 2014, DFID concluded that the issues, objectives and actions in the
improvement plan were being satisfactorily addressed and expects to approve and close the
improvement plan in August 2014. In December 2013 the TOR for the ORIE component, including
requiring an ORIE plan, was drafted and submitted to SEQAS (DFID’s external quality assurance
service). This was 14 months after the inception phase started and therefore reflected some of the
discussion and work that had already been done within the project. The TOR was agreed in February
2014.
The ORIE Oversight Committee was established to guide the strategic direction of the independent
evaluation and operational research and to ensure its quality. The members consist of
representatives of DFID, Tulane, IMA, the DRC Ministry of Health and an external
research/evaluation expert. To ensure the impartiality of the evaluation quality assurance (QA), the
external consultant will not provide the formal QA services via DFID’s SEQAS contract. The first
meeting was held on June 2014.
Quality Assurance for the evaluation studies is via SEQAS, in accordance with Evaluation Department
QA procedures at entry and exit. Operational research QA is conducted by the ORIE Oversight
Committee. At entry, a three-step approval process for the OR studies has been established. This
serves to provide clarity about the research topic/theme of each study; a concept note outlining in 3-4
pages the key features of the study; a full study protocol. SEQAS QA templates are currently being
adapted for use as QA templates for OR study protocols and reports.
The ORIE Plan was drafted by Tulane University and accepted in May 2014. A total of 10 studies are
included in the ORIE component as shown below:
Figure 6: Progress on research studies in the ORIE component to date
Study
Baseline Impact
Evaluation
Mid-term Process
evaluation
Endline Impact
Evaluation
CHE
User Fee
Community
Empowerment
Health Worker
Motivation
WASH
Value For Money
Family Planning
Nutrition
(additional
funding)
Start date
Topic/
Questions
Concept
Note
Protocol
Field Work
May-14
Y
Y
Y
Y
Oct-15
Y
Y
n/a
Apr-14
Apr-14
Y
Y
Y
Y
Apr-14
Y
Aug-14
Y
Aug-14
Sep-15
Jan-15
Y
Y
tbd
Y
Report
Y
Y
Y
DFID and Tulane set the target of having all concept notes completed and approved by August 2014.
No research findings have been reported to date.
30
Monitoring progress throughout the review period
Direct feedback from stakeholders and beneficiaries
The MoH was closely engaged in the design and early implementation of the programme and has
maintained this. The steering group for the programme (with joint membership from GoDRC, donors and
IMA) was not progressed as there were concerns that the Ministerial Decree drafted by the MoH implied
that a parallel oversight body for DFID’s project would be set up. Instead, the quarterly two day partners’
workshops which are chaired by the Ministry of Health and include IPs, technical partners and
provincial/district officials have provided this oversight function.
The field visit for the annual review spoke with patients, staff, local authorities and health management
teams. Feedback from the central Ministry of Health on the progress and approach of the programme
has consistently been positive. The Secretary General commented that in general ASSP is well aligned
with government plans. He has identified that there is good cooperation between DFID and the Ministry
of Health, particularly on implementation, highlighting that senior MOH officials have participated in
previous field visits including the annual review field visit. Within health zones, concerns were raised by
local authorities and facility staff about the importance of accelerating the work on rehabilitation and or
construction of infrastructure and equipment, including access to drinking water and lighting in health
facilities. Earlier during year one of implementation DFID received feedback from a number of
stakeholders on stock-outs of drugs. This situation seems to have improved to some extent more
recently. Staff continue to raise the issue about lack of reliable government salaries and DFID/IMA
continue engagement with the Ministry of Health on this issue but do not plan to reconsider the use of
primes at health facilities. During the annual review field visit patients gave positive feedback on
improvements in health facilities (such as availability of drugs) but commented on the need for
improvements on buildings/lighting.
Monitoring activities throughout review period
DFID has regular meetings with the Ministry of Health and has attended a number of
workshops/trainings implemented by the project during the year. The figure below outlines field visits in
2013/14.
Figure 7: Field visits during year one of implementation
Date
Field visit – Province
Purpose of visit
March 2013
Kasai Occidental – Access to
Health (IRC)
Province Orientale ASSP
Maniema-ASSP
Kasai Occidental – Nutrition
programme
Equateur – ASSP with
participation of Stephan
Dercon and Head of Office
Kasai – ASSP and Support to
Malaria Control in DRC
Kasai Occidental - ASSP
Kinshasa clinics – ASSP
ORIE Component by
Evaluation adviser
Kasai Occidental
Monitoring of handover (IRC to SANRU)
July 2013
July 2013
July 2013
October 2013
October 2013
December 2013
April 2014
July 2014
4-8th Aug 2014
(planned)
South Kivu
Monitoring ASSP implementation/handover issues
Lubutu hospital transfer to ASSP
Scope ideas for new nutrition programme
Monitoring ASSP implementation in North Ubangui
Monitoring of ASSP in Tshikapa and announcement
of new malaria programme to Kasai provincial officials
Monitoring ASSP implementation in new health zones
Monitoring Tulane’s household baseline survey
training
ASSP annual review field visit to Mweke and Ilebo
Kasai Occidental
Itombwe and Minembwe
31
DFID DRC has not yet put in place a planned contract for third party monitoring or national external
audit. In late 2013 the health team devised detailed ToRs for two programme funded monitoring officers
(one technical and one financial) who would be field based. We have not pursued recruitment of these
posts following a decision to implement this capacity cross-office. In the event that third party monitoring
is not imminent by the end of the year, the health team should consider contracting this using funds from
the Access to Health care in the DRC business case.
_______________________________
32