Type of Review: Annual Review
Project Title: SARRAH
Date started:
3rd June 2013
Date review undertaken: 3-14 June 2013
Description
Outputs substantially exceeded expectation
Outputs moderately exceeded expectation
Outputs met expectation
Outputs moderately did not meet expectation
Outputs substantially did not meet expectation
Scale
A++
A+
A
B
C
Introduction and Context
What support is the UK providing?
SARRAH is a 5 year programme; it commenced in January 2010 and is due to end in December 2014.
The total committed funds are £32,025,217.00.
Original contract
Amendment 1
Amendment 2
Amendment 3
14 Jan 2010
12 Aug 2010
30 Jan 2012
22 Jan 2013
1 Jan 2010 – 12 Dec 2013
1 Jan 2010 – 13 Dec 2012
1 Jan 2010 – 13 Dec 2012
1 Jan 2010 – 12 Dec 2014
£
106,492.00
£ 13,500.000.00
£ 20.352,217.00
£ 32,025,217.00
What are the expected results?
The impact1 of this programme is to ‘meet MDG 6 targets in South Africa through delivery of the South
African National Strategic Plan on HIV, AIDS and STIs (NSP) 2007-2011 and its successor’. Progress is
expected against the following indicators:
 HIV prevalence among 15-24 year old women and girls
 HIV incidence among 15 – 49 year old women
 HIV Incidence among women (15-24)
 Death from all causes 25-29 (gender disaggregated)
 Under 5 child mortality and maternal mortality indicator ratio
The expected outcome2 is ‘improved governance of an integrated, effective response to HIV & AIDS and
health in South Africa’. Progress is expected against the following indicators:
High level plan to improve the health sector

1
2
Number of new patients (adults and children) initiated on ARVs
‘Impact’ is the DFID preferred term for the overall ‘goal’ (as used in the original logframe).
‘Outcome’ is the DFID preferred term for the overall ‘purpose’ (as used in the original logframe).
1


Proportion of TB treatment success among all TB cases
Improved donor coordination and harmonisation
Given the macro level of the Impact and the long term nature of the supported interventions, the
emphasis in this review is primarily on process indicators relating to ‘improved governance of an
integrated, effective response to HIV & AIDS and health in South Africa’.
What is the context in which UK support is provided?
The SARRAH programme was initiated at a time when the South African health system was emerging
from a period of controversial management, particularly with regard to HIV and AIDS. During President
Mbeki’s tenure the Ministry and Department of Health were adversely influenced by the ‘AIDS
denialists’ and HIV and AIDS services were severely compromised as a result. For example, despite
successful efforts to make antiretrovirals (ARV) available more readily and some being offered free of
charge, the roll-out of this life-saving treatment was painfully slow. One of the current SARRAH
partners, the Treatment Action Campaign, gained international acclaim for its successful legal
challenge which forced the government to make ARVs more widely available. The changes of
leadership which followed the recall of president Mbeki created an opportunity for donors to work on
health matters with the South African government with renewed vigour and SARRAH was designed to
make the most of this opportunity.
The level of commitment to transforming the health sector is illustrated by the steadily increasing
budget for health, which grew by 15,3% from R21,7 billion in 2010/11 to R25,7 billion in 2011/12.3
The SARRAH programme is not a single large-scale intervention, but supports national reforms,
programmes and institutions of the South African health sector through a range of strategic
interventions identified jointly by DFID, the NDOH, TAC, POCs and SANAC. This has led SARRAH to
diversify into a range of activities, each supporting different parts of the policy landscape and each
intervening at different levels in the line of management. SARRAH’s workstreams are closely aligned
with the NDOH 10 Point Plan, the policy instrument through which the Negotiated Service Delivery
Agreement is to be achieved. At the request of the NDOH, SARRAH support has shifted towards
preparatory work for NHI implementation in the last 18 months. These activities and their relationship to
the NHI building blocks are illustrated in Figures 1 and 2.
The context of a new era of opportunity which characterised the start of SARRAH remains and the
climate for improvements in the South Africa health system remains positive. Equitable access to
health care is a major policy driver for the ANC government, opposition parties and organised labour
and is therefore almost certain to remain a priority.
3
http://www.info.gov.za/aboutsa/health.htm
2
Figure 1. Summary of SARRAH’s Work Streams in support of NHI preparation
Figure 2. SARRAH’s Technical Work Streams (boxes) in support of the NHI Building Blocks
3
Section A: Detailed Output Scoring
Output 1: enable the improvement in quality of and access to HIV & AIDS and health
services in selected districts
Output 1 score and performance description: A+
SARRAH contributes to this output through the following work streams: PMTCT, OHSC, GP
contracting and TB in mines. Work on PMTCT was completed in June 2011, substantial support has
been given to the development of the OHSC over several years, and TB in mines is a relatively new
activity.
Progress against expected results:
Indicator 1: Prevention of Mother-To-Child Transmission of HIV (A+)
2013 Milestone: 2.3%
2013 Actual: This work stream was completed in June 2011. Actual MTCT rate in January 2012 was
2.7%. Current value is not available.
Indicator 2: Establish Office of Health Standards Compliance (A)
2013 Milestone:
Bill approved by National Assembly and referred to the President for signing into
Law.
2013 Actual:
Bill approved and currently with President.
-
It is a huge achievement to have successfully piloted the bill through the legislature in the past year
and this was, according to NDOH, greatly facilitated by SARRAH support. National Assembly
hearings were hosted by the Parliamentary Portfolio Committee on Health with logistical support
from SARRAH. SARRAH provided clause by clause analysis of legislation throughout this process.
SARRAH also assisted in engagement with the National Council of Provinces.
-
The level of appreciation from NDOH highlights that SARRAH objectives are entirely owned and
led by NDOH, with HLSP technical support and flexibility a key factor in successfully pushing
OHSC through. Described as ‘invaluable’ by NDOH, allowing NDOH to quickly develop ‘something
from nothing’.
Indicator 3: OHSC business case (A+)
2013 Milestone:
Final draft business case completed
2013 Actual: The business case for OHSC is complete, meaning that the building blocks are in place
for a full start up as soon as the bill is signed and promulgated.
-
Important that HLSP tried where possible to bring on national and regional TA, with positive
secondary outcome in terms of wider network of SA experts knowledgeable about OHSC. The
business case prepared by SARRAH was taken over by the Treasury’s Technical Assistance Unit
(TAU) who then used KPMG (funded by NDOH) to take the matter forward.
Indicator 4: TB in the mines (A)
2013 Milestone:
CEOs / Chamber of Mines lead development of coordinated business case for
TB and HIV.
2013 Actual:
TB in the mines concept paper submitted to DFID by SARRAH.
4
This is a very recent initiative and the intention is to engage the private sector to
drive the process. The first draft of the business case for TB in mines has just
been completed.
Recommendations:
PMTCT
-
Previous reports showed a declining trend for mother to child transmission and since the PMTCT-A
Plan has been implemented nationally (having been piloted by SARRAH in 18 districts) this trend is
likely to continue. The indicator is not tracked routinely but further tracking should be done when
routine or independent survey data become available.
OHSC
-
-
-
OHSC core standards and the reporting requirements are highly demanding and essentially
developed from the centre. It is important to invest in and develop ownership and support
monitoring and reporting systems at the local level in pilot sites to inform scale up when OHSC
goes national. Written toolkits may help, but the process may also require more hands on
mentoring by the NDOH – this may help to overcome any local resistance..
There are still areas where SARRAH support may be required, such as regulatory policy. Given the
progress and momentum at present, this final piece should be supported if funds are available.
The OHSC process in South Africa presents real learning opportunities for the region , potentially in
partnership with the International Society for Quality in Healthcare (ISQua)
There is a critical issue concerning coordination with other inspectorates and monitoring initiatives
to prevent undue respondent burden among those being evaluated. Currently health facilities are
monitored by the presidency (Department of Performance, Monitoring and Evaluation – DPME), the
Facilities Improvement Teams (FIT), mostly concentrated in NHI pilot sites, and DPSA, the latter
using its own set of standards. This is in addition to any provincial monitoring visits. An example
was given of one hospital receiving 3 unscheduled visits within a week. This is both an excessive
burden on facility staff, which could negatively affect delivery and create resistance at the provincial
level, but also duplicative – similar standards are assessed and all include improvement plans. The
appointment of provincial NHI coordinators presents an opportunity to address this issue.
OHSC could deepen engagement with the DPME on citizen-based monitoring to ensure that OHSC
is able to take stock of community health initiatives and gather a broader view on the extent to
which core standards and NDOH promises are being met.
TB in the mines:
-
No specific recommendation.
Impact Weighting (%): 10%
Revised since last Annual Review? N
Risk: Low/Medium/High
Revised since last Annual Review? N
5
Output 2: Strengthened leadership and accountability of the national response to HIV
and AIDS
Output 2 score and performance description: A+
Output 2 comprises support to SANAC, TAC, the Joint Committee on HIV and AIDS (JCH) and the
Parliamentary Portfolio Committee on Health. Previously a third parliamentary committee was being
considered, namely the Select Committee on Social Services, but this was found to be less relevant
and is not currently receiving SARRAH support.
Progress against expected results:
Indicator 1: Support to strengthen SANAC Secretariat (A)
2013 Milestone: SANAC effectively monitoring and reporting on progress of the National Strategic
Plan.
2013 Actual: An M&E Framework for the NSP was developed in the fourth quarter of 2012 but its
implementation has been slow. This is partly due to poorly constructed NSP indicators which SANAC
acknowledges and also a lack of capacity in SANAC. However, SANAC has taken responsibility for
coordinating the mandatory reporting to UNGASS, which is a positive step.
The Programme Implementation Committee has been re-constituted as the NSP Programme Review
Committee and 5 new Technical Task teams established (4 for NSP objectives and one for research).
A new Civil Society Forum has been set up to coordinate the sectors.
2013 Milestone: SANAC financial management system in place to meet requirements for SARRAH
grant
2013 Actual: Changes to SANAC are much more imbedded than during the previous AR and direction
of travel is still positive. SANAT Board of Trustees approved a 3 year budget and 13 posts have been
filled with a further 15 approved for the next financial year. PWC pre-grant due diligence assessment
was passed and the first internal audit has just been submitted to the Auditor General. A noncompetitive bid has been submitted for USAID PEPFAR support of US$ 10 million over 5 years, which
will cover about 50% of current costs. The sustainability plan is coherent and the intention is to
transition to treasury support for core costs in the medium term.
SANAT is now receiving an accountable grant from HLSP and only IT remains directly under HLSP.
This procurement process is under way and the grant should be phased out by end 2014.
2013 Milestone: SANAC begins strengthening of Provincial and District AIDS Councils
2013 Actual: Provincial Strategic Plans were developed last year and Operational Plans are being
developed. Meetings have been held with Free State, Eastern Cape and Mpumalanga Provincial AIDS
Councils. CDC funds to be used to strengthen M&E in Provinces.
Indicator 2: Establishment of Joint Committee on HIV and AIDS (JCH) and support to JCH and
Portfolio Committee on Health (A+)
2013 Milestone: Strategic plans for JCH and PCH developed.
2013 Actual: SARRAH support has allowed the committees to undertake vital strategic planning work.
Previous issue around scheduling of meetings is now resolved with both houses attending when they
are able without waiting for a mutually convenient time. This decision has resolved an impasse which
resulted in the JCH only meeting occasionally. Members are not yet proactively setting the agenda for
the JCH on HIV AIDS – this is currently done by the Secretary guided by external requests. A positive
development is that the committee has invited civil society (sex workers representatives; traditional
healers) to address them as well as addressing cross-cutting issues such as HIV and TB in prisons (a
6
human rights issue).
2013 Milestone: PCH exposed to international practice in relation to health insurance.
2013 Actual: Parliamentarians’ and secretariat’s UK study tour increased buy-in from members.
Indicator 3: TAC monitoring of NSP and effective lobbying of government and advocacy on key
issues related to HIV and health (A+)
2013 Milestone: M&E system operational and evidence suggests quarterly reviews of NSP are
informed by TAC data.
2013 Actual: M&E systems have been strengthened with a clearly developed theory of change. The
large membership base allows TAC to draw on community level data and mobilisation across the
country. High standard NSP review publications demonstrate the core mandate is being met.
2013 Milestone: 2 submissions to parliament
2013 Actual: TAC are a key player in pushing through a number of important policy reforms and
drawing attention to systemic problems in drug supply and management. These include contribution to
debate on Intellectual Property laws, advocacy for Fixed Drug Combination tablets and TB in prisons. A
positive relationship with the NDOH and the Minister of Health exists such that TAC has been
approached to investigate policy issues by the Minister and TAC serves on one of the Ministerial
Advisory Committees.
2013 Milestone: 2 TAC model districts implement advocacy campaigns on gender-based violence
2013 Actual: TAC’s focus is access to treatment and therefore their emphasis for GBV is primarily
treatment orientated. They cooperate with Sonke Gender Justice for other aspects of GBV such as
justice advocacy. TAC branches are leading on a Gender-Based Violence campaign which includes
mapping facilities where PEP is available and dialogues on various GBV topics.
2013 Milestone: Annual external DFID audit completed and financial management practices and
improvements in place.
2013 Actual: PWC audit showed a number of areas of significant improvement, however there are still
high risk exceptions that need to be addressed.
Recommendations:
SANAC:
-
Continued SARRAH support to SANAC on the development of M&E systems to bring this work up
to speed with other significant steps forward.
CEO strategy to take time over recruitment of key posts shows long term vision. This should
include focussed development of the core leadership team to ensure depth of capacity in SANAC
senior management.
Establishment of JCH on HIV and AIDS and support to JCH and PCH:
-
Currently the committee secretary appears to be setting the agenda of the Joint Committee on HIV
and AIDS – This should ideally be driven by members and the co-chairs.
SARRAH should engage with the JCH to support new DFID regional programming addressing HIV
in correctional facilities given the JCH’s existing interest and commitment to this issue.
SARRAH support the work of both TAC and the JCH, however there is little engagement at
present. This is a missed opportunity for both parties and both should explore avenues and forums
to strengthen the relationship.
7
Treatment Action Campaign monitoring of NSP:
-
-
TAC is successful in utilising links with the Minister and NDOH to advocate for access issues,
however, the relationship is more difficult at the provincial level. Collective action with government
to address pharmaceutical supply and other problems at the provincial level may ultimately be the
most effective approach, but currently TAC state that ‘the door is closed’ in the provinces. This
represents a major barrier to TAC success and TAC may need third party support to develop a
more constructive relationship with provincial government, perhaps through establishing a
provincial liaison function..
TAC showed real improvement in the last external audit, however high risk areas (operational
planning and budgeting, lack of internal audit, financial controls) still need to be addressed.
Although TAC shows commendable awareness of the need to keep focussed in their treatment
access mandate and have used a partnership approach with relation to GBV, it is important that the
work on GBV is fully supported given its great importance in South Africa.
TAC branches that work on GBV should coordinate with District Clinical Specialist Teams to ensure
local dynamics and actions are understood and enable branches to share and disseminate health
information to their members.
Impact Weighting (%): 30%
Revised since last Annual Review? N
Risk: Low/Medium/High
Revised since last Annual Review? N
Output 3: Support national interventions to improve access and equity to HIV and
health services
Output 3 score and performance description: A
The Director General spoke directly of the pivotal role of the SARRAH programme in enabling the NHI
pilot as well as drawing on HLSP technical and analytical support in developing several areas of
National Health Insurance. SARRAH TA continues to provide flexible technical assistance to NHI
workstreams, including through a joint study with WHO on NHI financing.
Progress against expected results:
Indicator 1: Progress towards the establishment of the NHI (A+)
2013 Milestone: NHI Ministerial consultative roadshows completed in all NHI Pilot Districts
2013 Actual: Continued high level of engagement and political will from the Minister of Health, through
SARRAH-funded ministerial roadshows to promote NHI in pilot sites.
2013 Milestone: NHI Conditional Grants approved and loaded in districts.
2013 Actual: R140 million has been disbursed to pilot districts through NHI conditional grants, of which
77% was spent by year end 2012/13.
2013 Milestone: Roadmap produced for strengthening management and governance of central
hospitals.
8
2013 Actual: SARRAH TA provided input on delegation of authority to Central Hospitals and ways to
strengthen governance and management.
2013 Milestone: Facility Improvement Teams established in all districts.
2013 Actual: A SARRAH-appointed FIT team coordinator (in the office of the DG) is working with FIT
teams to improve service delivery and quality of care in 10 pilot districts. She was involved in
assessments of all 40 hospitals in Limpopo. FIT teams have been established in 10 of the 11 pilot
districts – the exception is Western Cape which declined the offer of support. FIT teams include hands
on involvement of senior leadership of the NDOH (up to and including DDG level), an approach which
has galvanised staff motivation in NHI pilot sites and helps ensure that NDOH leadership is exposed to
the issues on the ground.
Indicator 2: PHC re-engineering – District Clinical Specialist Teams (A)
2013 Milestone: 50% of posts filled in all NHI pilot districts and DCST members attend orientation
training offered by NDOH.
2013 Actual: There has been considerable progress since the last annual review with 44% of posts
filled, however, as predicted, there are not sufficient numbers of specialists willing to deploy to rural
areas, even with the considerable incentives offered. Discussions are ongoing with Deans of Medical
Schools to consolidate joint appointments of District Clinical Specialists and with the HPCSA to
expedite registration of non-South African specialists. The National Coordinator assisted in developing
a proposal for a further R42 million to procure essential equipment and provide additional
compensation.
The coordinator assisted in developing an induction and orientation programme. Modules include
baseline assessment, clinical governance, leadership, mentoring and coaching, and outreach support.
Training on baseline assessment has begun.
In the remaining 41 districts 130 posts have been filled (45%) and a database of DCST members
developed.
Indicator 3: PHC re-engineering – Ward Based Outreach Teams (WBOTs) (A+)
2013 Milestone: 10% of teams constituted in the NHI pilot districts and reoriented in PHC reengineering.
2013 Actual: SARRAH appointed National Coordinator for WBOTs to support orientation and
coordination of CHW activities at provincial and district level. Posts are funded by the Conditional
Grants and a clear policy is in place. 22% of WBOTs were in place by March 2013 and 29% of the
CHWs have been re-trained in PHC re-engineering.
Indicator 4: Progress towards contracting private sector GPs in the public sector (A)
2013 Milestone: Situation analysis of NHI Pilot Districts completed.
2013 Actual: Data collection tools developed, GP Contracting Rapid Assessments and District Profile
Reports completed in all NHI Pilot Districts.
2013 Milestone: Database of interested GPs compiled.
2013 Actual: Database complied.
2013 Milestone: Draft National GP Contract with performance management framework submitted to
NDOH.
2013 Actual: Draft GP contract and SLA, performance management framework, training and induction
framework and database of GPs developed. Contract management unit using 2 full time SARRAH staff
9
set up in NDOH. GP Steering Committee established with SARRAH membership and support which
has presented contracting models to the Minister.
SARRAH supported sessions on GP contracting at the 2nd Global Symposium on Health Systems
Research (Beijing) and the Primary Health Association / Rural Doctors Association of South Africa.
Indicator 5: Timescale for application for medicine registration (B)
2013 Milestone: No further growth of evaluation backlog (stabilised)
2013 Actual: SARRAH contracted 26 evaluators during Feb/Mar 2013 to expedite the registration of
generics, thereby leveraging the 2013/14 budget buying power and increasing VfM.
The responsible NDOH manager confirmed that the backlog is still large but pointed out that the
throughput of applications during the Feb/Mar intervention had been considerably better than in the
past owing to improved systems put in place during the earlier SARRAH backlog project. Prioritisation
of ‘essential drugs’, such as ARVs and generics, is helping streamline flow but the legislation that
allows this creates its own problems. Manufacturers can demand prioritisation of a generic drug which
is designated ‘essential’ but is already over-supplied – some countries cap the number of generics.
Another issue is the submission of duplicate applications for identical products with different branding
without making this evident to the evaluators.
Another extenuating factor is that South Africa is used by pharmaceutical companies to expedite
getting drugs registered for use in the rest of Africa when there may be no intention of selling in SA.
Thus the MCC/SAHPRA is fulfilling a regional role.
The MCC has a chronic shortage of evaluators and the backlog of applications is now growing, with an
estimated delay of around 9 months before a new application would be opened. HLSP support to clear
the backlog has been effective, but without systematic changes and increase in capacity this is only a
temporary solution.
Indicator 6: SAHPRA Establishment (A)
2013 Milestone: Detailed business plan submitted to NDOH.
2013 Actual: Business plan was submitted to NDOH in July 2012 and the Crisp Report is widely seen
as comprehensive and authoritative, but there are concerns that due to the lack of skills base in South
Africa some of the recommended reforms, such as having over 600 staff, cannot be implemented
within the proposed 5 year plan. 90% of SAHPRA HR and organisational policies are finalised and the
organogram has been developed in consultation with relevant directorates.
Medical device regulations have been finalised, with industry input, and submitted to NDOH.
2013 Milestone: SAHPRA Technical Task Team (TTT) established
2013 Actual: Support given to the TTT, chaired by Prof Helen Rees (Ex-chair of MCC), to provide
oversight and assist with establishment of SAHPRA. Working groups set up to address priority areas.
Lack of progress in the establishment of SAHPRA is one of the consistently underperforming areas of
the SARRAH programme and little progress has been made since the previous annual review. This
was the only failure specifically identified by the Director General of the NDOH.
There are complex political economy factors impeding progress with SAHPRA, including public and
private sector and academia, as well as national and provincial dynamics around procurement.
Some of the building blocks for SAHPRA have moved forward with the support of SARRAH and the
Crisp Report is still considered important and relevant. The business case was identified as a best
practice example by Treasury for use by other departments.
10
The full value of the Electronic Document Management System and online tracking of applications is
yet to be realised owing to incompatibility with the Civitas and NDOH IT environment.
Recommendations:
NHI
-
NHI pilot sites are the target of both FIT and OHSC inspection visits and improvement plans.
Careful coordination is required to avoid overburdening management with too many similar
processes aimed at quick wins.
FIT teams had experienced success in addressing ‘quick wins’ around cleanliness and infection
control, but more complex and systematic problems may require greater technical support to
diagnose problems and develop solutions.
Pilot facilities should be monitored to determine the sustainability of FIT interventions after FIT
teams withdraw. Presumably this will be a function of the OHSC. Peer exchange and review at the
local level could be one potentially sustainable intervention.
DCST PHC re-engineering:
-
-
DCST’s role in overseeing clinical governance and quality should include embedding protocols for
dealing with the public health crisis of gender based violence, for example in the clinical
management of rape or in establishing appropriate referral mechanisms for domestic violence or
child sexual abuse. Currently GBV has not been a major focus and this is not included in their
indicators according to the national coordinator.
.
GP contracting:
-
No specific recommendations.
Drug registrations and SAHPRA establishment:
-
-
DFID’s regional review of the pharmaceutical sector may help to identify opportunities as SAHPRA
is being established. Although there are many difficulties, there are also a number of ideas in the
NDOH to find innovative ways to tackle problems – for example to bring in retired evaluators from
outside the region to train and mentor in SA, or dealing with duplicate dossiers. Where possible
(without SARRAH funding) the NDOH should focus support in areas which will contribute to a
more effective MCC now, in areas that will benefit SAHPRA when it is finally established – for
example in building evaluator/regulatory capacity.
SARRAH could explore mechanisms to allow the private sector funding of academic programmes
for regulatory staff that would avoid conflict of interest – for example to channel funds through a
service provider as a managing agent.
An efficient IT system would appear critical to the success of SAHPRA and this is unlikely to be
achieved without relocation to another building (which was said to be unlikely) or changes to the IT
environment (allowing another server).
Impact Weighting (%): 31%
Revised since last Annual Review? N
Risk: Low/Medium/High
Revised since last Annual Review? N
11
Output 4: Strengthen performance management & strategic planning for HIV and health
services at national and provincial level
Output 4 score and performance description: A+
Progress under this Output has involved the reorganisation of hospital CEOs, further appointments in
line with the restructuring of the NDOH head office, and establishment of the Academy for Leadership
and Management in Health Care.
Progress against expected results:
Indicator 1: Competency of public sector district and hospital managers – hiring CEOs (A+)
2013 Milestone: > 100 CEOs selected and appointed
2013 Actual: There was very strong ministerial buy-in and support for the hiring of CEOs – including
involvement in the interview process. The target of 103 CEOs was met. The majority of CEOs were rehired after having to reapply for their posts – between 10 and 20% were new recruits, so it should be
noted that for the most part the recruitment process confirmed incumbents in their posts.
Indicator 2: Organisational Development of the public health sector (A+)
2013 Milestone: New organisational structure for NDOH in place that reflects the introduction of NHI.
2013 Actual: All 6 DDG posts and CD positions have now been filled through reorganisation and
recruitment. NHI, which was initially proposed as a separate entity, has been included within the Health
Regulation & Compliance cluster under Dr Anban Pillay. The new position of Chief Operations Officer
has been filled and the Performance Management and Corporate Human Resources directorate
established. These changes are consistent with the operational plan developed by McKinsey using
SARRAH support in 2010.
Organisational development at the NDOH has been the focus, but hospital level support structures
around the CEO may be a weakness in service delivery. The point was made that a large hospital has
more staff than some smaller government departments but has a much smaller management team.
Indicator 3: Service Transformation Plans agreed and monitored through the Annual
Performance Plan (A+)
2013 Milestone: 5 of 7 Service Transformation Plans completed
2013 Actual: Nine plans have been completed and approved (two of which were submitted and
approved without SARRAH support).
Monitoring is not taking place because this activity has been inactive for some time since it was argued
that the STPs could only be taken further once the demands of NHI became apparent. However, the
SARRAH Impact Evaluation was informed by Treasury that the plans were of high quality and should
be revisited because much of what was proposed remains relevant for implementation of NHI.
Indicator 4: Development of a comprehensive HR strategy (A)
2013 Milestone: Academy for Leadership and Management in Health Care launched
2013 Actual: Technical consultants appointed as Project Director and Chair of the Advisory Committee.
The Academy was launched by the Minister in November 2012.
2013 Milestone: Academy Advisory Committee constituted.
2013 Actual: The Advisory Committee has been constituted and includes top local and international
12
experts including Prof Marian Jacobs (Chair), Previous Head of the Child Health Policy Institute and
Dean of the Faculty of Health Sciences at UCT and Lord Nigel Crisp, previous Chief Executive of the
NHS.
2013 Milestone: First cohort of CEOs receive orientation programme
2013 Actual: 88 of the 103 new or re-appointed hospital CEOs attended the orientation programme
facilitated by the Academy. The first round of CEO training and orientation, including the development
of ‘100 day plans’ to turn around hospitals took place in February 2013. The direct involvement of the
Minister, DG and a number of DDGs in the training shows a high level of political support for the
Academy and the development of CEOs and is commendable. Feedback from participants indicated
that the first course was largely successful despite some initial challenges with a mixed group of very
experienced and new CEOs. The ‘100 day plans’ are being reviewed by the Gordon Institute of
business Science (University of Pretoria).
-
-
-
The HR strategy is complete– there are also a number of positive developments in HR, including
the launch of the Academy of Leadership and Management in Health Care; the recruitment of
DCST’s and WBOTs; the development of a master plan for nursing colleges; a review of
occupation specific dispensation; and the launch of a private sector supported public health
enhancement fund to support education.
SARRAH support has been critical in developing the HR strategy and has supported piloting some
of the initiatives within the strategy, such as the leadership academy. SARRAH funding of the
HLSP technical lead on the HR strategy has now come to an end with the NDOH expected to pick
up recommendations from the strategy and evaluation and take them forward. Many of these have
not been put in place or developed yet.
The Academy for Leadership and Management is due to receive funding from the Private Sector
Social Compact, an important step in terms of sustainability. The plan is to bring the Academy
within the NDOH budget by the end of 2014.
Indicator 5: Number of asset management qualifications in AG Audit for selected provinces (B)
2013 Milestone: 1 of 4 unqualified audits in selected provinces
2013 Actual: The NDOH had an unqualified audit on assets for the first time in 2010/11, although it was
qualified again in 2011/12 but this was said to be due to moving offices at the time of the audit.
Mpumalanga dropped from 4 asset related qualifications in 2010/11 to 2 in 2011/12 and these were
resolved in a few days. Progress in the Eastern Cape is good but there are still challenges in KwaZulu
Natal.
Indicator 6: Occupational Specific Dispensation (OSD) (A+)
2013 Milestone: Gap analysis and report on OSD completed
2013 Actual: Final draft was completed by Deloitte consultants and submitted to the National Health
Council in October 2012. The report addresses what went wrong with OSD and how it can be
remedied.
Recommendations:
CEOs:
-
The Leadership Academy and the NDOH may need to review the capacity of middle management
structures in hospitals to ensure that the newly inducted CEOs have the adequate support needed
in their respective environments.
Whilst the Academy provided a good foundation and orientation for CEOs, part of this introduction
should include the development of a more structured peer network and mentoring programme, as
well as individually tailored development plans. While it is inevitable that the Academy will take time
to become fully operational it is essential that the CEOs are kept informed of progress by the
13
-
Academy leadership and not ‘left in the dark’ with the potential consequence that momentum is
lost.
Whilst the Academy aims to have technically and managerially competent CEOs in place in every
hospital, currently the bar has not been set in that there is no professional qualification and
competency framework in place for CEOs.
There is currently no professional course in hospital management in South Africa. There may be
opportunities to work with academia and the private sector to address this gap. For example, the
social compact and public health enhancement fund will support the Academy – could it also
support the development of a professional qualification and course in management?
Service Transformation Plans:
- SARRAH should consider revisiting the STPs in light of strong endorsement from Treasury.
HR Strategy and academy:
-
SARRAH funding of technical support to an HR consultant is due to end in June 2013. Now a DDG
with responsibility for HR has been appointed greater progress can be expected, however there
remain crucial gaps in HR management, for example, no electronic database, no system of
organising professions within the NDOH, and a lack of capacity in health workforce planning. The
new DDG fir HRH and central hospitals will take this agenda forward. The HR Strategy Review
currently being done by SARRAH should provide information on whether the resources are in place
to continue to move forward with the implementation of the strategy. There may be significant
opportunities to work with the private sector and link private sector CEOs with those in the public
sector, if resistance can be overcome. Additionally, private sector actors outside the healthcare
sector could potentially provide support on leadership and management, public financial
management, IT expertise and other areas – perhaps through the social compact.
Impact Weighting (%): 19%
Revised since last Annual Review? N
Risk: Low/Medium/High
Revised since last Annual Review? N
Output 5: Strengthen system to effectively monitor and evaluate national and strategic
plans for HIV & AIDS and health
Output 5 score and performance description: A+
Activities under this output entail support to the development of M&E capacity in SANAC,
improved data management in NDOH (NHIRD) and remote monitoring systems using mobile
phone technology (mHealth).
Progress against expected results:
Indicator 1: SANAC M&E systems strengthened to monitor and evaluate multisectoral
responses to HIV & AIDS in the public and private sectors (B)
2013 Milestone: Establishment of the SANAC M&E function.
2013 Actual: SANAC received a SARRAH grant of which 75% was designated for M&E support.
SANAC completed the NSP M&E framework in 2012 but this is still a work in progress as many of the
14
indicators are difficult to use because they relate to data which is not routinely collected. SANAC
management is well aware of these problems and will be working with a designated SARRAH staff
member to resolve them.
Indicator 2: UNGASS National Composite Policy Index (A)
2013 Milestone: SA reports gender-disaggregated data on all relevant UNGASS indicators through
annual NSP review and report by SANAC.
2013 Actual: SANAC coordinated the submission of the South African report of gender-disaggregated
data using UNGASS indicators to UNAIDS on time. This work was done independently of SARRAH but
reflects the increased competence of SANAC arising from SARRAH support.
Indicator 3: Monitoring and evaluation for the NDOH (A+)
2013 Milestone: National Health Information Repository and Data Warehouse (NHIRD) established.
2013 Actual: By August 2011 the DHIS data, StatsSA socio-economic data and financial data from the
NDOH, had been loaded on to the new system. The system was formally launched by the Minister of
Health in August 2011 and its potential demonstrated to stakeholders from the national and provincial
health departments as well as the media. Training and capacity building has been completed in all
provincial offices but direct access to NHIRD appears to be restricted to trained M&E personnel and a
few senior managers. However, district and provincial profiles, reports, presentations and posters for
facilities are being produced and positive reports were received from the DG and COO regarding the
usefulness of the information available.
Indicator 4: Mobile monitoring and mHealth (A+)
2013 Milestone: Mobile monitoring concept piloted in 9 districts using ART and HCT data sets.
2013 Actual: The proof of concept was done with 985 phones making this the biggest trial of its kind in
the Southern hemisphere at the time. A ‘stress test’ was completed by uploading 6 months data in 10
days. There were some teething problems with the server and this was resolved by moving to a cloud
based system using two servers simultaneously (i.e. built in redundancy).
A similar initiative, funded by the EUPHC Sector Support Grant, has been developed for collecting
information on the 6 core domains of the National Core Standards from patients attending facilities.
2013 Milestone: Plans in place to scale up to 3000 CHWs as part of WBOT
2013 Actual: 3200 CHWs will have access by 1 July 2013.
2013 Milestone: Pan African mHealth Initiative grant in place
2013 Actual: Collaboration with PAMI is being considered but the key is getting all the operators to
come on board.
The approach to date is to present the same information as is normally captured on DOH forms
allowing easy transition from paper to the electronic system. However, there appears to be a need for
change management since even though cell phone technology is widely used not all users are
sufficiently familiar with the technology to be able to use it for data entry.
Recommendations:
SANAC: It is commendable that SANAC has taken responsibility for coordinating the UNGASS
reporting for the first time. However, the M&E function of SANAC remains ineffective and although this
is being addressed it should be flagged for further support owing to its critical importance.
NHIRD: A demonstration of NHIRD was approved for the SARRAH review and evaluation team during
the interview with the DG but it was not possible to schedule this during the Annual Review. This
opportunity to see the system in operation and review the products it produces (reports, posters etc)
15
should be taken as soon as possible.
mHealth: More training is required for users to facilitate change management of the paper to phone
transition.
Careful monitoring of the impact of the mHealth initiative should be undertaken and baseline data
collected before wider roll out. Anecdotal reports indicate that districts using mobile monitoring of HCT
are experiencing fewer stock outs than elsewhere and formal evaluation should be used to confirm
such gains.
Having demonstrated the feasibility of the mobile phone data collection system the potential for wider
scale application within the region and other developing countries should be explored.
Impact Weighting (%): 10%
Revised since last Annual Review? N
Risk: Low/Medium/High
Revised since last Annual Review? N
Section B: Results and Value for Money.
1. Progress and results
1.1 Has the logframe been updated since last review? Y
Minor changes have been made to accommodate changes in strategic direction requested by
NDOH. New or improved milestones have been introduced to better track these changes.
1.2 Overall Output Score and Description:
A: Outputs met expectation.
1.3 Direct feedback from beneficiaries
For the purposes of this review the beneficiaries are considered to be the government officials working
in the National Department of Health, staff and members of South African National Aids Council, staff
of the Treatment Action Campaign and Parliamentary Committee on HIV and the Portfolio Committee
on Health.
-
Beneficiaries were, without exception, highly positive about the inputs made by the SARRAH
programme for supporting aspects of health reform;
Specifically, beneficiaries commented on high quality technical support provided when and where it
was needed;
Several beneficiaries citied SARRAH’s input as enabling them to move forward from initial ideas
that had often stalled due to lack of capacity or resources;
A number of beneficiaries citied SARRAH’s flexibility as a unique advantage which allowed the
programme to achieve more than some of the larger donors;
The continued support to both SANAC and TAC came at a time when other donors were cutting
back on funding and this has allowed both bodies to make significant advances on their respective
mandates after having successfully dealt with some very challenging issues;
16
-
The support has an exceptionally high level of ownership within NDOH (especially by the DG) who
referred to certain projects that ‘change the world or improve lives’ – she felt that SARRAH certainly
appears to be improving lives and has the potential to change the way the world goes about health
development initiatives.
1.4 Summary of overall progress
The many components of the SARRAH programme continue to make a significant contribution
to the remodelled health system in South Africa. New initiatives this year, such as the
Leadership Academy and better engagement with Parliament through the committee system,
bode well for further progress. The Leadership Academy in particular appears to be an
essential component for addressing the leadership and management capacity constraints
within the health system. It is notable that the Minister of Health and several DDGs were
personally involved in the first training course offered by the leadership Academy. Likewise,
senior managers up to and including DDGs have been ‘rolling up their sleeves’ as part of the
Facility Improvement Teams. This level of commitment to projects supported by SARRAH
would seem to confirm their value in the eyes of the beneficiaries.
The alignment of workstreams with National health Insurance, which is emphasised by the
new theory of change model (Figs 1 & 2), supports the notion that the programme continues to
support a national priority, namely universal access to healthcare. NHI is endorsed by most
parties and is gaining momentum as resources are poured into the Pilot Districts and the
initiatives developed there are already being copied by other districts (e.g. DCSTs).
1.5 Key challenges
Procurement of technical assistance and flexible support to innovation:
The flexibility and rapid responsiveness of the SARRAH programme to support catalytic or innovative
work in the NDOH was a consistent theme throughout the annual review and repeatedly by NDOH
stakeholders. They also made clear that using NDOH procurement systems to bring in short term
technical assistance or to quickly finance other interventions such as study tours, workshops or
conferences is currently difficult to do within a short timeframe. The Director General did not have an
immediate answer to the problem of responding quickly to short term technical needs once the
SARRAH programme is over, yet there will still be a long road to travel before NHI is implemented and
other reforms are fully imbedded. If financial regulations make internal procurement less flexible, the
NDOH may consider contracting a service provider to play a similar role to SARRAH in sourcing TA
and supporting short term projects. Without this there is a that some reforms may lose momentum.
Although the requirements of the Public Finance Management Act limit internal fast tracking
procurement within NDOH, SARRAH could explore innovative reform initiatives that would facilitate ongoing delivery and roll out of NHI. It appears that a well-prepared business case, such as the one done
for SAHPRA , has the potential to expedite the flow of funds.
Sustainability:
SARRAH’s success, in that it is a catalyst for so many of the reforms to health in South Africa, means
that support stretches across a wide range of NDOH departments and initiatives. If there is not
potential to establish a mechanism for flexible support within the NDOH, then SARRAH should take
stock of the most critical areas of current support and work closely with the NDOH to develop a clear
exit strategy that ensures critical reform initiatives do not stall. That may involve some reconfiguration
and prioritisation of existing support in the final 18 months of the programme.
Treatment Action Campaign (TAC) are currently reliant on SARRAH for core funding (25%) and with
other donors also reconsidering their bilateral support to South Africa there is a risk that they may not
be able to sustain their actions on the basis of membership alone. SARRAH should explore this with
TAC and support the development of a business plan to secure their core funding post 2015.
17
Despite the challenges, there are many positive developments in phasing in treasury support for key
areas such as SANAC and the Academy for Leadership and Management in Health. This should
ensure a seamless transition from areas of SARRAH support.
Senior Adviser to the National Department of Health:
According to a wide range of NDOH staff interviewed, the role of Dr Bob Fryatt (DFID seconded senior
adviser to the NDOH) has been central to the success of SARRAH to date,. This secondment will end
by the third quarter of 2013 and DFID will not be funding a replacement based in the Department.
Given the pivotal role Dr Fryatt has played in supporting the reform process, a careful assessment of
his current roles and responsibilities and critical input to different work streams is essential, with a clear
succession plan in place prior to his departure. DFID Southern Africa will also need to consider how to
maintain the close relationship with the NDOH and synergy with other DFID programmes and potential
areas of synergy and support, as well as providing the interface between the NDOH, HLSP and DFID.
This includes areas of potential UK collaboration with the NDOH to support health reform in the region,
as well as maintaining established links with the UK NHS and between respective parliamentary
committees on health.
Coordination:
A critical area that should be highlighted across the reform process is the need to coordinate within the
NDOH, with provincial health departments and with other departments such as the DPSA and DPME.
This is most apparent in relation to the inspection and monitoring functions that these various bodies
undertake. Unless these are well coordinated, front line service delivery staff may well find themselves
overwhelmed and unable to deliver. Of concern are the NHI pilot sites, which are the recipients of
many of the initiatives being piloted from the centre. The programme should remain sensitive to this
and carefully coordinate and sequence initiatives and monitoring visits to avoid staff fatigue and a
negative impact on staff morale. The appointment of provincial level NHI Coordinators presents a good
opportunity to resolve these issues.
National and provincial interface:
Many of the reforms developing with SARRAH support are led from the centre – perhaps inevitably
with reform of this scale and pace. However, as reforms become embedded, more focus on the
communication and management of change may be beneficial as NHI moves through subsequent
phases and reforms go beyond pilots. Engagement of the Minister of Health at the provincial and
district level is a positive example of this. The greater the level of consultation, the more likely changes
will be quickly accepted and embedded by district and provincial stakeholders.
GBV:
In March 2011 the NDOH Director General co-chaired an expert consultation on addressing gender
based violence as a cross cutting concern in South Africa, committing to “continuously review and
improve contributions towards an effective multi-sectoral response, and using international and national
resources to facilitate a sustained, strategic approach”. The annual review process did not reveal any
data that would suggest that there is a lack of commitment – however, it is notable that other than
Treatment Action Campaign, not a single stakeholder the review team met mentioned any initiative or
innovation to address GBV. The District Clinical Specialist Teams are well placed to address the
clinical management of rape and domestic violence, referral systems, treatment protocols and
particularly identification and response to child sexual abuse – there are a number of well-established
international protocols DSCTs can draw from without having to design new protocols from scratch. It is
not possible to conclude that DCSTs have failed to address these areas, but the AR team heard from
the national coordinator for PHC reforms that GBV had not so far featured as a priority in their work or
as an indicator of progress. SARRAH could review progress on initiatives to date in light of the Director
General’s 2011 commitments. DFID’s Social Development Adviser could provide technical support and
links to GBV prevention and response networks and the DFID funded RMCH programme may present
an opportunity to take this recommendation forward.
18
1.6 Annual Outcome Assessment
Based on available evidence SARRAH continues to strengthen the South African health system though
its many workstreams and is making a significant contribution to the roll out of NHI which seeks to
achieve universal access to healthcare. It has also begun to address some of the human resource
constraints through the Leadership Academy.
2. Costs and timescale
2.1 Is the project on-track against financial forecasts: Y
The programme is largely on track against financial forecasts. However, work on the GP contracting
work stream (which is co-funded by the EU) has faced some delays which have resulted in under
spending. However, measures have been put in place to make use of remaining funds and ensure that
the projects gets back on track.
Table 1 below shows SARRAH planned and billed expenditure as of March 2013.
Summary to date:
Total value of invoices to DFID as of 31 March 2013
£24,446,106.00
On-going milestones/Contracted
£7,248,179.00
Committed funds not contracted (TB Mines & POC)
£330,930.00
Total Contract Value to Date
£32,025,215.00
2.2 Key cost drivers
The key cost driver of the SARRAH programme is technical assistance, provided in-house by HLSP
technical leads for each work stream together with external service providers comprising private
companies, NGOs, Universities and independent consultants. As of 31 March 2013, a total of 237
contracts had been issued since the SARRAH programme began in January 2010.
In order to ensure that the rates for the technical assistance days are kept as low as possible, HLSP
has a Consultancy Services Team, based in the UK, whose role is to procure consultants at
competitive rates. A programme record of negotiation results is kept, and this shows that an average
saving of 22 % was achieved in negotiations over the past 12 months with 72% of the rates negotiated
downwards. Rates paid to consultants generally average between 40 - 50% lower than the South
African Department of Public Service and Administration’s published upper level tariffs.
The total cost of the Project Management Office (PMO) is budgeted at £6,120,612.00 which represents
19 % of the total programme budget of £32,025,215.00. However the PMO is not just administrative, it
includes Technical Leads, and Quality Assurance functions, all of which provide technical assistance to
the Partners as part of their role. The external 2012 VfM audit of SARRAH estimated that
approximately 50% of PMO costs are for technical assistance.
2.3 Is the project on-track against original timescale: Y
The SARRAH programme was tendered as a five year programme (2010 – 2014). The initial contract
was signed for the 36 month period on the understanding that a 24 month extension was possible
contingent upon funding availability, the on-going needs of SARRAH Partners, and the performance of
the service provider. The signing of Amendment 3 in January 2013 enabled a budget increase which
19
was approximately £8m over the £24m budget during the design phase of the programme. SARRAH is
on track against the five-year timescale as originally designed.
3. Evidence and Evaluation
3.1 Assess any changes in evidence and implications for the project
The overall Theory of Change which was examined in detail at the start of the SARRAH Impact
Evaluation4 remains valid. Within the current review period further efforts have been devoted to
integrating all the SARRAH components into a model explaining their contribution to NHI (See Figs. 1
& 2). While not differing significantly from the earlier TOC, the new model is very useful in showing how
all the various workstreams contribute to NHI and the interrelationship of the various outputs.
TAC has also developed a revised theory of change model for its activities and effectively linked this to
its M&E system (see Annexure 1).
3.2 Where an evaluation is planned what progress has been made?
A draft Mid-Term Evaluation report was presented to the Impact Evaluation Steering Committee in May
2013 and the report is currently being finalised. The conclusion on the overall performance of SARRAH
based on OECD-DAC evaluation criteria of Relevance, Effectiveness, Efficiency, Impact and
Sustainability was as follows.
Overall, the mid-term rating for SARRAH was 2.50, which is the average score on a scale of 1 to 4.
The nature of the scoring system implies that this score is unlikely to decrease, and much more likely
to increase as some of the longer term outcomes and impacts materialise and become observable at
the final evaluation stage. The Relevance and Efficiency scores are essentially summative (statement
on past performance) and will remain constant throughout. The Effectiveness, Impact and
Sustainability scores are of a more formative nature (designed to inform future development) and
therefore will probably increase as the initiatives start to bear fruit in a more detectable way.
The SARRAH Impact Evaluation has been granted an extension in line with that of the overall
SARRAH programme and will continue until 2015.
4. Risk
4.1 Output Risk Rating: Low/Medium/High
The programme is currently has a ‘Medium’ risk rating.
4.2 Assessment of the risk level
The risk analysis matrix for the SARRAH programme outlines key risks as well as mitigating actions.
HLSP conducts on-going assessment and updating of the risk matrix and there are currently no
additional risks identified.
Only 2 risks are identified in the matrix as high probability and high impact. They are outlined below, as
well as the mitigation strategies in bullet form:
4
Final Evaluation Strategy Report April 2012.
http://www.sarrahsouthafrica.org/ABOUTSARRAH/SARRAHEvaluation.aspx
20
i.
High profile of work being undertaken on NHI:
•
Close collaboration with the NDOH to ensure that common understanding of the
confidential nature of any information or work done under the SARRAH
programme.
•
Confidentiality agreements to be signed by members of the team working on any
highly confidential subject matter.
ii.
EU programme delays:
•
Highlighting of potential delays to the NDOH and to DFID to facilitate progress
•
Recruitment of a senior Team Leader to drive the programme forward.
•
Increased interaction with Provincial Departments and Districts to inform and
facilitate the progress.
•
Presence on the NDOH SteerComm overseeing the GP contracting work to
ensure alignment and prevent duplication.
DFID will monitor closely the risks highlighted above. It is worth noting that the strong political
commitment to this project by the NDOH will also help to ensure that possible risks to programme
delivery are addressed and mitigated to allow continued programme implementation.
4.3
Risk of funds not being used as intended
HLSP’s contract is based on a milestone matrix which clearly defines how funds are allocated and paid
out. The milestone based contract therefore ensures that invoices are only paid once DFID is satisfied
that the agreed milestones have been implemented and achieved. This therefore reduces the risk that
DFID funds are utilised for activities other than those agreed in the contract.
4.4 Climate and Environment Risk
The climate and environment risk was not directly assessed in this review, but was discussed with a
DFID Environment Adviser.
The assessment made is that the SARRAH programme does not pose any significant risk to the
climate. Its focus is on working with the NDOH to provide quality and accessible health care services to
South Africans. It therefore stands to reason that a healthier population is more adaptable to a range of
challenges and stressors such as climate change.
The service provider has put in place responsible measures to conserve the environment such as
recycling and logging their travel carbon footprint.
5. Value for Money
5.1 Performance on VfM measures
There has not been much change to the scope of delivery of the programme since 2012. Therefore,
most of the observations in the 2012 VFM report on SARRAH still hold true. Through a range of
different types of interventions SARRAH intends to support, strengthen and catalyse improvements in
the South African health system. Many of the work-stream activities will take a long time to have a
sustainable effect on the programme’s outcomes. For this reason, the Evaluation Strategy used during
the Mid-Term Evaluation adopted a process evaluation approach that focused on what has been
delivered, how and with what intermediate results and effects. It focused on evidencing the contribution
of the programme to its logframe outputs and set out to systematically evaluate SARRAH’s
effectiveness, from the perspective of the potential value of the results delivered, given the costs that
have been incurred.
Efficiency
21
The Value for Money Review completed for DFID in March 2012 concluded that ‘based on the above
creditable results the conclusion is that the SARRAH programme has to date provided a satisfactory
level of Value for Money in the production of deliverables and milestones.’ The Review concluded that
there was reasonably strong evidence that SARRAH had achieved its deliverables at the right price
(i.e. economically) and that it had successfully executed its deliverables and achieved its milestones in
the right ways (i.e. efficiently). This mid-term evaluation has found that SARRAH has developed and
delivered activities that are relevant to the health needs and priorities of the South African Government
and key stakeholders. However, it proved impossible at the mid-term stage, from a qualitative
perspective, to be certain whether the ways in which SARRAH’s activities have been delivered will
influence, enable and catalyse systematic health improvements through its key partners and
stakeholders.
Leverage
At this stage of implementation, there is limited evidence to suggest that SARRAH has leveraged a
significant amount of additional resources. This is perhaps to be expected at the mid-term stage of
SARRAH’s lifecycle. By the time of the final evaluation in 2015, it is likely that evidence of leverage by
the programme will be more prevalent as stakeholders build on the different types of structures,
systems and processes that SARRAH has developed to date. The additional resources that SARRAH
has successfully leveraged are typically in the form of human resources and commitments by donors
and the NDOH to continue funding work steams such as SAHPRA. However, the extent to which the
leveraged resources will enable the delivery of the additional benefits that are required to catalyse
significant improvements in South Africa’s health system remains unclear at this stage – for example,
additional financial commitments have been made in principle to some of the workstreams, but these
have yet to be realised pending budget allocation.
The likelihood that SARRAH will have a significant scale of effect is inconclusive. Workstreams such as
SAHPRA have the potential to have a significant scale of effect, but evidence beyond its immediate
effects have yet to emerge. It remains to be seen how well new structures, such as those introduced
through the NDOH or the NHIRD workstreams, will be engaged or used by stakeholders at the national
and regional level. Similarly, SARRAH has enabled improvements to TAC’s M&E capacity, which may
help it to improve its organisational effectiveness and attract donor funding. However, there is little
evidence of this to date.
Theory of Change
A Theory of Change evaluation requires that the assumptions underlying SARRAH’s causal linkages
should be tested. With regards to the effectiveness of the implementation of the work-streams, many of
these assumptions are neither proven nor disproven. On-going large work streams, such as SAHPRA
and SANAC, are highly dependent on political processes and the allocation of resources to enable
these activities to fulfil their objectives, which has proved challenging to date. However, although these
resources have not yet been provided, this does not necessarily mean that they will not be provided in
the future. By contrast, assumptions behind some of the completed workstreams are beginning to be
corroborated by their results. PMTCT, for example, was tested in a few districts before being scaled up,
which enabled its assumptions to be validated empirically.
There remains a risk that the larger workstreams in particular, will not deliver their intended purpose
within the remaining lifetime of SARRAH. The success of many of SARRAH’s workstreams relies on
the sustained efforts and investment by NDOH, other funders and the political drive to empower those
engaged in the activities that have been delivered. Much of this core support has yet to be provided.
For example with the NDOH restructuring, although effective reorganisation of NDOH has the potential
to influence national health service delivery, in light of the critical shortage of suitably qualified health
managers in South Africa, there remain questions as to whether the reforms can be fully implemented
in the short term. Similarly for the Quality Assurance workstream, while the international evidence for
the benefits of effective Quality Assurance systems is strong, capacity constraints are likely to limit its
implementation through 2014 and beyond.
Conclusion
The Mid-term VfM evaluation concluded that SARRAH has developed and delivered activities that are
reasonably economic and efficient. Its activities are relevant to the needs and priorities of NDOH and
its key partners and stakeholders and the assumptions that underpin SARRAH’s theory of change
currently hold true. However, the additional resources (either financial or in-kind) that SARRAH has
22
been able to evidence as leverage are currently limited. The successful achievement of the
programme’s objectives and outcomes is largely dependent on the influence and support of SARRAH’s
immediate beneficiaries and stakeholders that it has been working with to date. The mid-term
evaluation provided benchmarks that will be further developed and assessed throughout the remainder
of the SARRAH Impact Evaluation.
5.2 Commercial Improvement and Value for Money
The 2012 VfM report states that ‘the mechanics of the accounting and recording with the Milestone
Matrix appears robust and the information reflected is accurate and reliable’; this continues to hold true.
5.3 Role of project partners
HLSP (Mott McDonald) manages the implementation of the SARRAH programme. As at 31 March
2013, HLSP had issued a total of 234 contracts to a mix of private companies, NGOs and independent
consultants. HLSP adheres to the DFID guidelines on achieving value for money and they translate this
to their subcontracted partners and contracts.
5.4 Does the project still represent Value for Money : Y
Yes, the project still represents value for money. The programme’s spending and delivery of activities
is also largely on track, ensuring that DFID funds are achieving results against agreed outputs at the
agreed price.
5.5 If not, what action will you take?
Although the programme is achieving VfM, DFID will continue to monitor the programme against VfM
indicators to ensure continued spending on target and achievement of agreed results.
6. Conditionality
Not applicable
7. Conclusions and actions
Overall performance:
SARRAH continues to deliver highly valued and high quality inputs to the Government’s health
transformation programme. The support has an exceptionally high level of ownership within the NDOH
- as was evident when the Director General referred to the project having “changed the world and
changed lives” – describing the contribution SARRAH has made to improving lives in South Africa and
as a programme with the potential to change the way the world goes about health development
initiatives. The review team found that programme partners repeatedly endorsed the value of the
speed, flexibility and quality of SARRAH inputs. Most beneficiaries welcome SARRAH’s flexible
approach to catalyse change and unlock innovation. However, this model does not have an immediate
successor in that there appears to be no commensurate facility in NDOH – this should be urgently
considered to ensure the flexible approach remains available as the country progresses towards NHI in
2025.
The following are the key recommended actions for the NDOH and other stakeholders stemming from
the findings of the annual review 2012-2013.
Sustainability and transition from DFID support:
23
 Procurement of technical assistance and flexible support to innovation. If this is limited
through the Public Finance Management Act the NDoH may consider contracting a service
provider to play a similar role to SARRAH in sourcing technical assistance and supporting short
term projects.
 SARRAH partners should take stock of the most critical areas of current support and work
closely with the NDoH to develop a clear exit strategy that ensures critical reform initiatives do
not stall. That may involve some reconfiguration and prioritisation of existing support in the final
18 months of the programme.
 Treatment Action Campaign (TAC) is currently reliant on SARRAH for core funding (25%)
and there is a risk that they may not be able to sustain their actions on the basis of membership
alone. SARRAH should explore this with TAC and support the development of a business plan
to secure their core funding post 2015.
 A careful assessment of the role of (DFID seconded) Senior Adviser to the National
Department of Health and critical input to different work streams is essential, with a clear
succession plan in place prior to his departure.
 DFID SA should develop a strategy to maintain the close relationship with the NDOH and
potential areas of synergy and support, as well as providing the interface between the NDOH,
HLSP and DFID.
 DFID SA should continue to engage in areas of potential UK collaboration with the NDOH
to support health reform in the region, as well as maintaining established links with the UK NHS
and between respective parliamentary committees on health.
National Department of Health:
 Office for Health Standards Compliance (OHSC): Newly appointed provincial NHI







coordinators will need to analyse how to streamline the monitoring process to reduce
duplication and potential overburdening of facility staff and maximise impact of oversight and
monitoring.
OHSC: Should deepen engagement with the Department of Performance Monitoring and
Evaluation on Citizen-based Monitoring to ensure that OHSC takes stock of community health
initiatives and which core standards and NDOH promises are being met.
NHI and FIT Teams: Careful coordination is required to avoid overburdening management with
too many similar processes aimed at ‘quick wins’ further consideration may be required to
resolve the more complex and systematic problems i.e. the ‘slow wins.’
NHI Pilot facilities should be monitored for sustainability of FIT interventions and peer support
systems could be introduced.
District Clinical Specialist Teams (DCSTs): Gender Based Violence (GBV) and child abuse
should be a focus of DCSTs and included in their indicators. The role should include
development and oversight of protocols for GBV response.
South African Health Products Regulatory Authority (SAHPRA): Consider establishing
mechanisms to allow private sector funding of academic programmes for regulatory staff that
would avoid conflict of interest – for example through a service provider.
The Academy for Leadership should consider the second tier of hospital management once
the initial work with CEOs is complete- support for CEOs alone will not be enough without
skilled management teams in support.
A peer support network is needed for CEOs; this could draw on the private sector if
resistance can be overcome. The Academy should ensure that on-going
communication/support is maintained between courses.
HLSP technical leads:
 SAHPRA: Resource constraints notwithstanding, consider how SARRAH support could be
streamlined in areas which will contribute to a more effective Medicines Control Council now.
 PMTCT: This indicator is not currently tracked routinely - but should be reported against when
data become available (possibly through NHIRD).
 Sustainability: SARRAH should review work streams to identify the most critical/ high risk
areas in relation to sustainability, identifying where NDOH has the resources to continue to
move forward with the implementation of the strategy and where there are concerns. This
should inform a responsible and considered exit strategy for SARRAH.
24
SANAC Secretariat:
 SANAC should invest in the capacity development of the second tier of leadership to shore up
the strengthened secretariat structure for the medium term.
 The M&E function of SANAC remains ineffective but is of critical importance. Planned
improvements should be prioritised as they move forward.
Parliamentary Committees:
 As members gain experience, they should take a stronger role in setting the JCH agenda and
driving it forward.
Treatment Action Campaign:
 Deepen engagement with provincial government, parliamentarians and JCH to strengthen
relationships and take collaborative, collective action to address problems.
DFID Southern Africa:
 DFID should harness TAC experiences to include the civil society perspective in regional work
with the SARPAM programme.
 SAHPRA: A DFID funded regional review of the pharmaceutical sector may help to move the
establishment of SAHPRA forward.
 JCH & PCH: DFID should engage with the JCH to support newly launched DFID regional
programming on HIV in correctional facilities.
 DFID SA - work through the Steering Committee of the PME programme at the DPME to
develop synergy and avoid duplication with OHSC and other NDOH health monitoring and
oversight work.
 DFID’s Social Development Adviser could provide technical support and links to GBV
prevention and response networks with strategic SARRAH work streams. DFID’s RMCH
programme support to DCST coordination may provide an opportunity to develop more specific
protocols to address GBV.
mHealth:
 More training is required for users to facilitate change management of the paper to phone
transition.
 Baseline data should be collected before wider roll out to facilitate proper evaluation.
 The potential for wider scale application within the region and other developing countries should
be explored.
General considerations for the next phase of SARRAH:
The NDOH should consider developing an internal fast track procurement system to provide flexible
and quick technical assistance and project support to innovative reform initiatives and the on-going
delivery and roll out of NHI. If financial regulations make internal procurement less flexible, the NDOH
may consider contracting a service provider to play a similar role to SARRAH in sourcing TA and
supporting short term projects. With 18 months to run on the programme, HLSP should immediately
take stock of key areas of current support, identify potential areas of weakness in the absence of the
programme and work with NDOH to develop a clear exit strategy to prevent critical reform initiatives
from stalling.
Due to the urgency of health reform in South Africa, many of the reform initiatives developed with
SARRAH support have been led from the centre. Whilst it was necessary to drive reform through
quickly, it will be important to continue to manage and analyse the impact of change at a provincial
level to anticipate challenges once reforms advance beyond NHI pilot sites.
8. Review Process
25
The third Annual Review of the SARRAH programme took place between 3 and 14 June 2013 and
was carried out by Joel Harding, Governance Advisor, DFID-SA, Beaulah Muchira, Programme
Manager DFID-SA and Prof John Seager, a Freelance Public Health and Development Consultant
contracted by Coffey International Development, London.
The Reviewers were briefed by the Senior Health Adviser and Health Team Programme Manager,
DFID SA, and then by the SARRAH Programme Manager of HLSP. Technical Leads in the
implementing agency presented an overview of progress and provided further information in
subsequent discussions. Various SARRAH, TAC, NDOH and other government documents were
consulted. Representatives of the four partners (beneficiaries), namely NDOH, SANAC, Parliament
and TAC were also interviewed; there were 16 interviews in total and these were conducted in
Pretoria and Cape Town. Details of interviewees are provided in Annexure 2.
26
27