Type of Review: Annual Review
Quality Reproductive Health Medicines (QuRHM) Programme
Date started:
1 August 2011
Date review undertaken: July – October 2013
Period reviewed: 1 August 2012 – 31 July 2013 (Year 2)
Introduction and Context
What support is the UK providing?
The UK is providing up to £3.5m over 3 years, of which up to £3.2m will go to the implementing partner,
Concept Foundation via PATH. The project is implemented under the auspices of the Reproductive
Health Supplies Coalition (RHSC). The period of funding is 1 April 2011 – 31 March 2014, although
project activities started 1 August 2011. A further £100,000 per annum was allocated for independent
external review of the programme.
What are the expected results?
At impact level, the project will contribute to increased use by women of voluntary family planning
methods of an assured quality standard (as per World Health Organisation and UNFPA quality
standards).
At outcome level, it is likely that by end 2014, the project will have generated savings worth at least the
value of its inputs of around £3 million.
WHO pre-qualified combined oral contraceptive pills (COC), progesterone only contraceptive pills (POP),
emergency contraceptive pills (ECP) and injectables from generic or low-cost manufacturers, will
contribute to estimated cost savings to the international community of US$6.4 million (conservative
scenario) to US$32.4 million (optimistic scenario) with a medium scenario of $13.7 million savings by the
end of 2016. Most of the savings are for lower procurement costs for injectable contraceptives from
current cost of US$0.93 (from the international procurers database, the Reproductive Health
Interchange) for the branded injectable Depo-Provera (DMPA, depot medroxyprogesterone acetate) to
an estimated US$0.50(optimistic estimate) - 0.70 (conservative estimate). The business case is
expecting £8m out of £13 million (medium scenario) of savings from injectables alone, over two years
2015 and 2016. This level of savings may be challenging. While several generic injectables are expected
to achieve quality assured status by project end, market impact in terms of actual procurement savings
may take longer to emerge.
Greater cost savings are anticipated over time and can be applied to further procurement, hence
increasing overall supply and reducing unmet need, if assumptions in the theory of change on market
impact hold. In addition, major procurers and countries will have a wider choice of quality assured and
approved suppliers, who can manufacture quality products at lower prices. This should also lead to an
improved method mix available in country.
What are the planned outputs of the project attributable to UK support?
By mid 2014, as an attributable result of DFID funding, a total of 25 generic products (hormonal
contraceptives and maternal health medicines such as misoprostal) produced by 15 manufacturers, from
a baseline of zero at end 2010 (not including eight prequalified products from originator companies), and
seven active pharmaceutical ingredients (baseline zero) will be approved. Six major procurers (including
social marketing organizations), which are members of RHSC, will have agreed and adopted common
1
quality assurance definitions, and policies for purchasing essential reproductive health medicines. Four
developing country governments will have accepted at least in principle the importance of internationally
agreed quality criteria in their own procurement and tendering processes.
How will we determine whether the expected results have been achieved?
UNFPA manages a database, Reproductive Health Interchange, where major procurers log their
activities. The DFID Business Case expects to see a downward trend in unit prices for selected
commodities and an increase in volumes procured. Country level reports should indicate that an
expanded method mix is available to women. This is being tracked at outcome level in the log frame. In
addition, given the high risk nature of this project, DFID in partnership with the Bill and Melinda Gates
Foundation will invest in an annual review of progress, contracted to an independent consultancy, HERA
(called the Review Team in this report). A full evaluation, to be funded by the Gates Foundation, is
planned for 2015/16.
What is the context in which UK support is provided?
Over 220 million women have an unmet need for family planning. Meeting this need is one of two
priorities in DFID’s Choices for women: planned pregnancies, safe births and healthy newborns – the
UK’s RMNH Framework for Results - which aims to enable women to access contraceptive services
and improve their reproductive choices about when and how many children they have. The Framework
includes the UK goal for reaching 10 million new family planning users by 2015. This is a contribution
to UN Secretary General’s Global Strategy for Women’s and Children’s Health, and the July 2012
London Summit on Family Planning (FP2020) goal of giving 120 million more women and girls access
to lifesaving contraceptives by 2020. The DFID Human Development Department offer for 2015
outlines £20 million savings/per annum to the global reproductive health community through driving
down the prices of quality family planning methods. This project is one of several interventions aimed
at achieving that saving.
Lack of quality, affordable contraceptives is a major barrier to meeting unmet need and increased use.
To date, the quality market for hormonal contraceptives and reproductive health medicines is
dominated by originator companies and their products remain more costly (e.g. injectables, implants
and reproductive health medicines such as oxytocin and misoprostol which are used to prevent
bleeding following delivery). Although cheaper products are available from generic companies in India,
Brazil and China, they are not always quality assured by international standards1. WHO carries out
independent assessment of essential categories of product through its pre-qualification programme
(PQP). In addition, where quality assured products are in short supply, and companies may not be
ready for a full dossier review but are making progress, a commissioning agency (UNFPA for
reproductive health medicines) may request WHO to carry out an expert review panel (ERP) review,
which provides a risk assessment for major procurers to use. For reproductive health, the first ERP
was held in 2013. Some reproductive health medicines that do not have Stringent Regulatory Authority
(SRA) approval or WHO pre-qualification are approved by National Medicines Regulatory Authorities
(NMRA). Most low and middle income countries have an NMRA but concerns have been raised about
the standards that these agencies apply, and their capacity to perform the required checks of
manufacturing practice and product quality is variable.
1
Where quality assured means approval by an SRA (eg USA, Europe); pre-qualified under the WHO
Prequalification Programme for Essential Medicines (PQP); or assessed by a WHO Expert Review Panel (ERP) as
category 1 or 2.
2
Section A: Detailed Output Scoring
Output 1: Increased availability of affordable quality assured RH medicines for supply to less
developed countries
Output 1 score and performance description: A
Concept Foundation has provide the technical assistance to companies, which has enabled a
significant increase in the number of quality assured generic products to 17, from a baseline of zero,
and the output score is rated A. Output 1 is the most important of the four outputs, with a weighting of
40%.
Progress against expected results:
Evidence for achievement of the output is the range of WHO prequalified (PQ) or ERP approved
reproductive health Finished Pharmaceutical Products (FPP) and Active Pharmaceutical Ingredients
(API) that are available for purchase by international and national procurers against a baseline of zero.
Concept Foundation’s inputs include dedicated advice and technical support to companies interested in
entering the global quality assured market and assessed as being capable of reaching agreed quality
standards, for both finished products and the APIs. Concept also supports the preparation of dossiers
presented by companies to WHO ERP and PQ assessment panels, and at agreed stages during the
assessment process. Concept works very closely with both WHO’s PQP team, and UNFPA (who
commission the ERP assessments as needed from WHO). This year the Review Team was able to
review internal Concept Foundation documents, and it was able to see the correlation between
Concept’s technical assistance with generic suppliers and the eventual success of those same
suppliers in securing ERP or PQ approval.
The revised 2013 logframe lists 3 milestones for year 2, one delayed 2012 milestone and one planned
milestone for 2014.





12 additional generic products approved under ERP, resulting from 2 ERPs
4 RH APIs prequalified or ERP assessed category 1 or 2
Number of RH dossiers accepted [by WHO PQ] over 2 years of the QuRHM project, as compared
with 2008/9
Time elapsed between submission of dossiers and product PQ between 2008-2010 (2014)
Business case for manufacturers completed/disseminated
For the 1st milestone year 2 (12 additional generic products approved under ERP, resulting from 2
ERPs) the Review Team noted 3 new approvals in ERP-2013 in addition to the 12 already achieved in
the 2012 ERP. Thus in total 15 products were approved in 2 ERP procedures. Concept provided
technical assistance to companies for all these products. Sub-score: A+.
As of July 2013,10 quality assured generic products (3 PQ and 7 ERP) were available. This is 3 more
than the 7 per 31 July 2012 (2 PQ and 5 ERP).
The pipeline is well filled: Concept provided technical assistance for four more products, which then
gained WHO PQ status, and three more were granted an ERP extension by September 2013, resulting
in a total of 17 quality-assured generic RH products being available for procurement. This is a
substantial increase compared to year 1 and a significant achievement from the baseline of zero. (Note
that the quality assurance status of an ERP assessed product expires after one year, unless the
company has entered it to the PQ process or it has been given an extension by UNFPA. So by
September, 10 of the 15 ERP approved products were still valid).
With regard to the 2nd milestone year 2 (Four RH APIs prequalified): only one RH API was
prequalified per 31 July 2013 as per WHO PQ website (against a baseline of zero on 31 July 2012).
The Review Team notes that a 2nd API was prequalified in August 2013. However, 5 RH APIs (all in
receipt of technical assistance from Concept) are under assessment and likely to be approved. As
3
such, the milestone is not achieved, but good progress is being made. Sub-score: B.
The 3rd milestone year 2 (Number of RH dossiers accepted by WHO over the 2 years of the QuRHM
project vs. 2008-2010) is presented as a new Value for Money (VfM) indicator. Concept Foundation
has not yet been able to collect (or receive) baseline data from WHO. The WHO/PQ website shows
that 11 RH products (generic or originator) were under assessment in WHO PQ as per 8 Aug 2013, but
these could have been there for many years. As there is no quantitative proof in this VfM indicator, the
Review Team is unable to assess this specific milestone. (No sub-score allocated)
The 4th milestone (Time elapsed between submission of dossiers and product PQ between 20082010) is meant for assessment in year 3. The Review Team suggests that the formulation of this
indicator is improved before assessment in 2014, and that baseline data is collected. (No sub-score
allocated).
The 5th milestone (dating from year 1) (Business case for manufacturers completed and
disseminated): was not achieved by July 2013. The business or investment case summarises the case
for investment in international quality standards by generic companies, providing data on market size
and opportunity. A draft report was made available to the Review Team. Generic manufacturers (the
main target group) saw presentations of the business case during the “Generic Manufacturers Caucus
for Reproductive Health” (GEMS Caucus) meeting in Delhi (9 October 2013), which enabled useful
feedback. A separate, WHO-developed business case was also presented by BioBridge in the Delhi
meeting. Concept Foundation and WHO have agreed and are developing a joint publication which has
delayed progress. Sub-score: B.
Concept Foundation has developed a “critical pathway” tool to monitor the timely progress of each
product through the quality assurance processes. The Review Team agrees that this is a possible
monitoring tool for showing how Concept and WHO/PQP technical assistance and assessment inputs
lead to a quality assured product. However, WHO had not yet included its comments and data in the
critical pathways documents submitted to the Review Team.
Concept Foundation has responded to the recommendation from Year 1, to prioritise and reduce the
number of products supported. The total has dropped from 25 (16 product and 9 API manufacturers)
with ongoing technical support, to 23 (12 product and 11 API manufacturers), with an appropriate shift
to more APIs.
CF has successfully brought together around 10 generic manufacturers in the “Generic Manufacturers
Caucus for Reproductive Health” (GEMS Caucus), which is a notable achievement, given the previous
fragmentation of the industry. A joint WHO/UNFPA/UNICEF meeting with manufacturers was planned
for September 2013. The Review Team has noted that Concept will make a presentation.
So far, most quality assured products are combined oral or emergency contraceptive pills. However, no
injectable contraceptive, oxytocin, mifepristone, or magnesium sulphate, and only one misoprostol
product have been prequalified or ERP approved.
Recommendations:
 Concept Foundation should prioritize (in the remaining project months) technical assistance to
ensure that injectables, oxytocin, misoprostol, mifepristone, and magnesium sulphate are
prequalified, and to monitor closely the composition of the commodities submitted for ERP and PQ,
given that the quality assured list comprises mainly oral combined and emergency contraceptives.
 Concept Foundation and WHO/PQP should continue collaborating on the two business cases to
ensure synergies, to produce a joint summary product by end 2013, which includes factual and
financial information (such as forecasts per product / country / time period, information on potential
donor budgets and future order possibilities).
 Concept Foundation and WHO/PQP should complete the joint critical pathway tool for each
product, to enable progress monitoring (in time for the annual review in 2014)
 Concept Foundation and DFID should set absolute (rather than relative) numbers for milestones,
eg for assessing VfM, by early 2014.
Impact Weighting (%): 40
4
Revised since last Annual Review? Yes (logframe)
Risk: High changed to Medium
Revised since last Annual Review? Yes (the risk is considered less by the Review Team)
Output 2: Major international procurement organisations have harmonized quality
assurance definitions, policies and practices
Output 2 score and performance description: B
This output is the second most important, and scored a marginal B (nearly an A). There has been
extremely good progress towards bringing major public sector procurers to common quality standards
and policies. UNFPA as a leading procurer, together with WHO, have supported this process well,
through consensus building workshops and close liaison with USAID. However, UNFPA had not
published the procurement guidelines by Year 2 end.
Progress against expected results:
Achievement of Output 2 is measured by “improved QA policies and procedures in place at
international level based upon procurement of only PQ or SRA approved products”.
UNFPA’s quality assurance policy for reproductive health medicines (originally published March 2011,
and a baseline indicator in 2012) was updated in November 2012. DFID also published a Quality policy
for RH medicines in June 2012 (with an update in September 2012) after discussing it with country
offices and procurement organisations. Concept Foundation provided input into the finalisation of this
policy. Concept Foundation, RHSC, UNFPA and DFID had discussions in 2012 with USAID and KfW to
also revise their quality policies.
Two milestones were formulated in the revised 2013 logframe for year 2:
1. Two additional procurers to adopt and use policies based upon the UNFPA policy (or evidenced
by a transition plan)
2. UNFPA developed and published guidelines for purchasing quality products
The 1st milestone year 2 (2 additional procurers to adopt and use policies based upon the UNFPA
policy (or evidenced by a transition plan)) is considered achieved as the MSI and IPPF transition plans
were accepted by DFID (Sub-score A).
The Procurers’ meeting in New York (29 April – 1 May 2013) also contributed to a better understanding
of the challenges that procurers face in the target countries, as evidenced through the meeting minutes
and agreements by USAID in particular to work more closely with UNFPA and WHO.
2nd milestone year 2 (UNFPA develop and publish guidelines for purchasing quality products) is not
completed. There appeared to be confusion about who is responsible for developing these and what
form they should take (UNFPA or WHO). UNFPA Procurement Services Branch (PSB) have confirmed
that they will publish guidance in line with UN good practice before project closure on 31 March 2014.
As the main contracted party, Concept Foundation remains responsible for this milestone not being
achieved. (Sub-score B)
A “value for money” indicator was added for year 2 (Reduction in the number of separate technical
assessments undertaken by UNFPA compared with number undertaken in 2011). 2011 baseline data
was not available. Concept reported that no separate technical assessments were undertaken by
UNFPA in 2012 and 2013, so the frequency can be assumed to be less in 2012/13 than in 2011. The
review found that literally this indicator can be seen as achieved (Sub-score A).
Procuring from a quality assured supply source should also reduce/replace the need for annual and
5
repeated risk assessments by individual procurers, in the longer term. Regardless of whether Concept
Foundation actually spent more or less on substituting UNFPA visits to companies, fewer technical
assessment over time means that VfM will increase.
The Review Team recommends that the definition of the VfM indicator is strengthened, as it only
measures the value on one side (fewer assessments by UNFPA), and not the money (the costs of
Concept Foundation assessments could have been more costly than those of UNFPA assessments).
Also, Concept could have replaced the UNFPA assessments with more visits than UNFPA would have
undertaken.
Recommendations:


As the lead organisation, Concept is recommended to better manage/monitor the milestones
that are to be achieved by partners, and to provide a short report informing DFID of how this is
being done with UNFPA, by end of 2013/early 2014.
Concept and DFID should agree on how to improve the VfM indicator with regard to assessing
and comparing the actual costs of technical assessments pre and post project, by end
2013/early 2014.
Impact Weighting (%): 30
Revised since last Annual Review? Yes (logframe indicators)
Risk: Medium
Revised since last Annual Review? No
6
Output 3: Commitment to strengthen national procurement criteria, based upon
internationally accepted quality assurance definitions, developed by key stakeholders
in four countries (Ethiopia, Kenya, Senegal and Nepal), working through RHSC
members, WHO and UNFPA.
Output 3 score and performance description: C
Implementation of the baseline study and in-country activities is seriously delayed, with the country
workshops now planned to take place towards the end of the project. Delays are due to the poor quality
of subcontractor reports and the need for Concept to do additional data analysis, with constraints on
time (given demands on key staff in 2012/13 generated by the unplanned activities arising from the
Family Planning Summit and its follow up).
Progress against expected results:
Establishing current quality assurance practices related to procurement of reproductive health
medicines in 4 pilot countries (Ethiopia, Kenya, Nepal and Senegal) was a year 1 milestone, but the
four final assessments were still not available end Year 2. The output language was amended as
compared to 2012 logframe, in response to the recommendation from Year 1 review, to make the end
point more realistic for a three year project.
Milestone 1, years 1 and 2 (Completion of situation analysis in 4 countries to provide baseline and
documented data on market dynamics and attitudes to quality available): Activities to collect the
baseline in the countries started October 2012 with a request for proposals. The contract was awarded
to the Royal Tropical Institute in Amsterdam (KIT). By July 2013 only one draft report (Nepal) was
available to the RT, although KIT has presented some initial findings to the April procurers’ meeting.
The milestone is not achieved. (Sub-score C)
Implementation of the baseline study and in-country activities is now seriously delayed. Partially this is
due to the four pilot countries only having been selected in August 2012. Some of the delay can be
explained by the substantial and unplanned activities around the July 2012 Family Planning Summit,
which CF had been requested to undertake. The RFP for the country studies was only published in
October 2012. The contract was granted in December 2012 to KIT, with an expected delivery date of
31 March 2013. The delay in final reports appears to be due largely to delayed submission of draft
reports and weaknesses in data collection and presentation that Concept has had to address itself.
Concept has stated that the data and reports submitted by subcontractor KIT were inadequate, and
that CF had to invest considerable time to validate data and review the analysis. In one country
(Ethiopia) the data collectors experienced serious problems with official permission to collect the data.
Nevertheless, Concept Foundation remains managerially responsible.
The delay in generating the country baselines is now also affecting the work at country level. UNFPA
has been postponing national workshops as the baseline studies were not yet available, although has
started to plan the country workshops with UNFPA country offices. Given the delay in producing
country baselines, all country workshops are now planned in the last quarter before project closure
(2014Q1). The workshops risk being organised in some haste, and need involvement of the many
RHSC members at country level. DFID is also suggesting a greater involvement of UNFPA Commodity
Security Branch (Global Programme to Enhance RH Commodity Security) in the country work in order
to gain high-level policy involvement.
Substantial advocacy efforts may be needed to promote the use of quality assured products in
countries, especially if these would be more costly than already registered products. Achieving the
planned output in year 3 seems very ambitious. Even when all workshops are delivered by 31 March
2014, this will hardly allow any time for local follow-up.
7
Recommendations:
 RHSC, Concept and UNFPA are recommended to develop a workplan for each country,
including stakeholder workshops and high level policy dialogue, and increase the involvement
of RHSC members in the country activities (with UNFPA country office, PSB and CSB), by end
December 2013
 Concept Foundation is recommended to better manage, supervise and monitor its outsourced
activities, and provide a report showing how it is doing this to DFID in early January 2014
Impact Weighting (%): 15%
Revised since last Annual Review? Yes (output language)
Risk: High
Revised since last Annual Review? No
Output 4: Awareness of the QuRHM strategy and related quality issues raised among
donors and other high-level stakeholders within the RHSC membership
Output 4 score and performance description: A
There has been good progress on this output, with Concept playing a prominent role in ensuring that
quality is now one of the RHSC’s four core pillars in its revised strategy. This reflects Concept’s
contribution to generating better consensus among all interest groups in the RHSC (donors, procurers,
suppliers, providers, civil society) on the definition of quality assurance standards.
Progress against expected results:
There are 2 milestones in year 2 for this output:
1. Quality assurance issues included in RHSC working group plans and annual RHSC meeting
2. Key non-procuring donor members of RHSC agree to strengthen funding agreements related to
product procurement.
Regarding the 1st milestone year 2 (Quality assurance issues included in RHSC working group plans
and annual RHSC meeting): Concept Foundation presented QuRHM progress to the Market
Development Approaches Working Group (MDA WG) in October 2012, and a written update to the
Systems Strengthening Working Group during the annual RHSC membership meeting in October
2012. The RHSC website (www.rhsupplies.org) has three presentations by Concept Foundation on
progress of the QuRHM project (although there are no plans, progress reports or other information).
MDA WG produced survey results that 43% of members would (17% maybe, and only 9% not)
participate in group activities on QuRHM (e.g., regular calls, occasional meetings, joint activities). This
shows that the topic of quality is seen as important by MDA WG members. On the other side, a few
members commented to theReview Teamthat Concept Foundation has been running the QuRHM
project more as a separate vertical project, where they have little influence or interaction. The
Technical Advisory Committee met once by phone and once in person, and is agreeing revised terms
of reference to build RHSC involvement in the wider quality agenda, with an eye on the new draft
strategy and the need for an integrated and systematic approach to quality across the RHSC’s
activities.
It is very positive that quality will become one of four central pillars in the new RHSC strategic plan
being developed. RHSC management confirmed that a Quality Working Group is being considered in
RHSC. This will give the issue of quality a prominent discussion platform among RHSC members
8
during and outside RHSC conferences. (Sub-score A+)
Regarding the 2nd milestone year 2 (key non-procuring donor members of the RHSC agree to
strengthen funding agreements related to product procurement): In year 2 both KfW and DFID included
in their procurement policies a preference for prequalified or ERP approved RH products. USAID has
also agreed to broadly align with UNFPA policy. (Sub-score A)
Recommendations:
 Concept Foundation (with inputs from DFID) are recommended to make the milestones of
output 4 more SMART, by end January 2014
 Concept Foundation is advised to work with the RHSC to publish QuRHM project documents,
reports and annual review on its website, by end 2013
 Concept Foundation (and UNFPA, the co-chair) should continue to ensure that the Technical
Advisory Committee is engaged and its role and membership developed in line with RHSC’s
new strategy and objectives (by end March 2013)
Impact Weighting (%): 15%
Revised since last Annual Review? Yes (milestones)
Risk: Low
Revised since last Annual Review? No
Section B: Results and Value for Money
1. Progress and results
1.1 Has the logframe been updated since last review? Yes
Although the programme was supposed to last three years, activities only started 1 August 2011, due
to late approval. With an end date of 31 March 2014 this means that the project is being implemented
over 32 months only (as opposed to 36 months). The 2012 Annual review covered the period 1 August
2011 – 31 July 2012; this review covers the next 12 months, i.e. 1 August 2012 – 31 July 2013. This
means that the 3rd year will only cover a period of 8 months.
The logframe was adapted by Concept Foundation (on request of, and agreed with DFID) in response
to the 2012 Annual Review agreed recommendations, including prioritising and reducing the number of
companies provided with technical assistance and strengthening the VfM indicators. The Review Team
now feels that some of the new indicators are insufficiently defined. There are some methodological
problems with the 2013 logframe. Some indicators are not SMART or can be interpreted in different
ways. Others are relative rather than absolute indicators (while the baseline data are not available or
collected).
1.2 Overall Output Score and Description: B
The overall score in this review is a B. The Review Team found reasonable prospects for an A by
project completion next year.
In particular, the project is doing well on Output 1, with the significant progress made with regard to the
growing number of quality assured generic products, and on Output 4, with growing prominence and
consensus on quality standards and assurance across the RHSC’s diverse stakeholders.
The results for Outputs 2 and 3 were disappointing, largely driven by delays in delivery by partners and
subcontractors, which Concept should have monitored and managed more actively.
1.3 Direct feedback from beneficiaries:
9
While the ultimate beneficiaries would be women, girls and men in developing countries who should
gain better access to a wider range of family planning commodities, intermediate beneficiaries can be
identified as (i) generic manufacturers, and (ii) members of the RHSC.
In this review the Review Team approached the intermediate beneficiaries.
Manufacturers of generic medicines supported by Concept are in general very satisfied with the
technical assistance they receive. All are supporting the goals of QuRHM. Manufacturers with
successful PQ or ERP products presented themselves as the Generic Manufacturers Caucus for
Reproductive Health (GEMS) caucus group in the Delhi RHSC conference. RH companies with poorly
designed factories or not interested in pre-qualification, have already been deprioritised by CF in last
year’s review of all manufacturers. Non-supported manufacturers interviewed last year often stated that
other financial, market and strategic reasons prevented them from going for WHO prequalification for
RH medicines.
The Review Team interviewed RHSC members during and after the annual conference in Paris 1-5
October 2012 and New Delhi 7-11 October 2013. RHSC members aware of the QuRHM project were
in general quite positive about the activities of Concept Foundation, but also noted that they are not
(yet) seeing the impact, such as quality assured RH products available in target countries, or generic
alternatives for originator injectables (or implants, which are not a target for this project). They also
listed other barriers for access to quality affordable RH medicines that are currently not included in the
project design or Concept Foundation activities, such as national registration, procurement rules, and
traditional product choice, for example (and for which Concept would not be best placed to address). In
general, they were hopeful that the Concept Foundation activities would in the medium term lead to
better choice of quality products. There was no consensus whether this would have an impact on the
price level of the products.
1.4 Summary of overall progress
The overall year 2 score of the project is a B. This is due to a few indicators in the revised logframe not
being achieved or lagging seriously behind.
The project has made good progress with generic manufacturers (output 1). The number of PQ or ERP
approved products has increased from 0 (baseline) to 7 (as of July 2012), with an additional 10 (by July
2013) rising to 17 (September 2013). The pipeline is showing several promising candidates. An A+
would have been possible, if the business case work had been delivered in time.
On the other side, there are still no prequalified generic injectables, oxytocin, magnesium sulphate or
implants2. There is one misoprostol, and in emergencies (no availability of the originator product) one
generic injectable can be procured under an ERP-3 rating. The Review Team can see from internal
confidential documents that CF has been working very hard to get the injectables, oxytocine,
misoprostol and magnesium sulphate products to ERP or PQ, and the project completion milestones
for Output 1 are likely to be met.
The Review Team also noted progress in output 2 (procurers). Procurers are now more optimistic than
last year, as they have more options for procurement of quality RH medicines. However, the New York
meeting also noted substantial barriers at country level, and procurers wondered who will support the
generic manufacturers of the not yet quality approved products after the closure of QuRHM project.
Unfortunately the lack of procurement guidelines due to confusion about who is responsible for making
them (UNFPA or WHO) meant that this output scored only a B.
Output 3 (country activities) is seriously behind schedule: it is yet to start with activities in-country as
the ‘baseline’ country studies were delivered late. Even if UNFPA manages to hold workshops in all 4
pilot countries, there will be no time for follow-up. The Review Team is also concerned about the lack
of involvement of RHSC members in the country workshop planning. This output scored a C. However,
this output is not critical to achieving the outcome, because price savings in the first instance will be
2
Implants are supported by FHI360 funded by BMGF outside the QuRHM project.
10
generated by the global procurers, mainly UNFPA.
Output 4 (advocacy) is making good progress and scored an A. Quality will become the fourth pillar of
the new RHSC strategy, and RHSC plans to set up a Quality Working Group which can continue the
work started by CF under QuRHM.
Overall, the project is making progress in enabling more generic RH products to be prequalified or ERP
approved, with more products in the pipeline. The overall B score is the result of some milestones not
being achieved in outputs 2 and 3, mainly due to delays in producing country reports, the business
case (with WHO) and procurement guidelines (with UNFPA). By project completion, the project could
still achieve a score of A or A+ in outputs 1, 2 and 4. Output 3 (country work) remains a serious
challenge.
The project is also at risk of not achieving at the outcome level due to delays in approvals of
injectables, upon which the cost savings and value for money case rests. Delays in injectable
approvals are largely due to the need for bioequivalence studies, which can take several years and
which the Gates Foundation is funding separately. The absence of these study results means that the
products cannot be assessed as ERP category 1 or 2, but as category 3 (objections to procurement
unless there is no alternative). However, there are signs that more quality assured injectables will drive
lower prices. The three month injection (baseline price of $0.86) has increased to over $1 due to quality
supply shortages. One product was assessed as ERP category 3. In case procurement is necessary,
UNFPA has already negotiated a long term agreement for the product at a price well below the original
baseline.
While most of the cost savings should be generated through the large procurers (USAID, UNFPA etc),
the wider barriers in many developing countries (which are not addressed by this project) still persist
(registration, procurement regulations, consumer behaviour and preferences). The Review Team also
concluded that the three year time frame of the QuRHM project was too short for gaining full country
buyin for procuring only quality approved products.
1.5 Key challenges





Although Concept Foundation did well in outputs 1 and 4, they are still not adequately
monitoring some deliverables of subcontractor KIT or partner UNFPA in outputs 2 and 3. While
there were outside factors (work linked to the London Family Planning Summit 2012) which
meant that the country work started late, Concept Foundation as lead contractor remains
overall responsible. The Review Team repeats its 2012 recommendation that CF needs to pay
more attention to monitoring including accurate forecasting of expenditure on activities and
follow-up. Part of this is inherent to the organisation: CF did appoint new management team
members, but CF top management remains overloaded with work, and has apparently not
delegated the troublesome project areas to other staff in the organisation or outside
consultants.
The many other barriers down the pipeline (from national registration and procurement at
country level to the actual consumers with unmet need) are not yet being addressed and it is
not clear that the country level activities were the right strategy to address this longer term
change.
A “post QuRHM” strategy was discussed at the Technical Advisory Committee in October, in
the context of the new RHSC strategic plan where quality is one of four pillars. This needs to
continued among RHSC management and development partners, building on current Technical
Advisory Committee discussions.
Concept Foundation has relied on its partner UNFPA for its country activities. An opportunity is
being missed to seek assistance and involvement of RHSC members for its country activities.
Output 3 should have been reformulated to take account of (1) what other programmes and
partners are already doing at country level and to ensure that relevant linkages are built; (2) the
specific expertise in Concept Foundation and RHSC members, and (3) more realistic
timeframes.
Concept Foundation’s responsiveness and communication to partners has improved somewhat,
but is still being noted as sometimes slow and inadequate.
11


The project remains medium to high risk, but the potential outputs are worth the extra wait. The
pipeline of generic products contains some badly awaited RH medicines (especially
injectables). A more focused approach towards these products is recommended. In retrospect,
the log frame should have differentially weighted results for different product types, giving more
value to PQ or ERP success in injectables.
The project design is still focused on technical issues on the manufacturers’ side, and does not
pay enough attention to the economic challenges. The business case for generic RH
companies to invest in PQ remains weak. Concept Foundation and WHO PQP explored
different pathways to resolve this, and are now working together to maximise the value of their
two approaches. Collation of market data, needs and future order quantities is needed.
1.6 Annual Outcome Assessment
Given the pipeline of generic RH products being serviced by Concept Foundation, the Review Team is
reasonably confident that the number of prequalified or ERP approved generic RH medicines will
increase over the 17 products available in September 2013. However, the remaining time until Project
Closure (31 March 2014) might be too short to achieve all the year 3 milestones. The country based
outputs are unlikely to be achieved, by end March 2014.
It is likely that by end 2014, the project will have generated procurement savings worth at least the
value of its inputs of around £3 million. Prices for quality assured oral contraceptive products have
decreased, in accordance with the theory of change assumptions. This is a positive sign, and it is very
likely that QuRHM activities have contributed to these price reductions by enabling more quality
assured competition. These price reductions are beginning to deliver procurement savings. UNFPA
has completed its first procurement of 0.45m oral pill cycles with a company producing the pre-qualified
product at the lower price, generating a saving of $30 ,000. UNFPA is in process of tendering and
developing long term agreements with all the newly qualified manufacturers, so it is not yet possible to
assess the total cost savings related to those products.
Cost savings are also expected from injectable contraceptives. The three month injection (baseline
0.86) has increased to over $1 due to quality supply shortages. One product was assessed as ERP
category 3, for procurement only where there is no alternative. In case this is necessary, UNFPA has
negotiated a long term agreement for the product at a price well below the original baseline.
Given the ambitious time frame, the weak economic incentives (probably underestimated by the
donors) and the late development of the business cases (by WHO and Concept Foundation separately)
the overall project approach may have difficulties achieving the indicators at outcome level. Any future
approach would need to (1) address the economic realities of RH medicines’ markets; (2) be more
aware of the downstream factors; (3) involve RHSC members working in/with countries, and (4)
strengthen links with existing other programmes working on RH in those countries. Impact would also
increase if RHSC members perform activities beyond the four pilot countries and if injectables can be
advanced to PQ rapidly. It is unlikely that this project can address all these factors in its remaining time,
but it would be essential for any successor activities to address these, across all partners.
For longer term change, more attention needs to be given to barriers on the demand side (national
registration, procurement rules and systems at country level, conservative attitude of procurers and
consumers to stick with existing brands, etc). In some aspects Concept Foundation could play a role
(eg national registration), although WHO is perhaps better placed. In other areas (national advocacy,
procurement) RHSC members should play a more active role.
Recommendations



The Review Team recommends that a DFID logframe specialist review the logframe, in
particular the VfM indicators, and make recommendations to Concept in early 2014.
The project’s in country activities are delayed. Concept and UNFPA should develop a workplan
showing how these activities can be delivered and in what time scale, by end 2013. This
workplan should include involvement of the RHSC community, to address build consensus in
the countries for addressing the downstream barriers.
As it stands now, the planned evaluation (scheduled to take place before 31 March 2015) might
12
be too early to show tangible results, as market entry of quality approved products (PQ or ERP)
might take more time than envisaged. The Review Team recommend that it takes place in
2015/16.
2. Costs and timescale
2.1 Is the project on track against financial forecasts: No
The actual invoices to DFID show 89.3% of the annual budget spent in FY1 (2011/12) and 84.5% in
FY2 (2012/13). For the final FY3 a spend rate of 96.8% is predicted.
Around half of the predicted underspend (around £400,000 on the whole project) is because it was
envisaged that DFID would fund all reviews and evaluation, amounting to £100,000 per year but in fact
this component was co-funded with the Bill and Melinda Gates Foundation. The remaining underspend
in year 2 is due to the late start and slow spend on activities for Outputs 2 and 3. See 2.3
2.2 Key cost drivers
The key cost drivers are staffing, technical experts, travel, and workshop costs. No changes were
noted in these cost drivers.
2.3 Is the project on-track against original timescale: No
Outputs 1 (working with manufacturers), outputs 2 (procurers) and 4 (advocacy) are slightly behind
schedule, but could still complete all activities.
Output 3 (country work contracted out to UNFPA) is however seriously delayed due to late country
‘baseline’ reports, and unlikely to complete all activities by 31 March 2014.
3. Evidence and Evaluation
3.1 Assess any changes in evidence and implications for the project
No external changes were observed. The Theory of Change was probably too optimistic in its
assumption that countries would change behaviours and only allow prequalified or ERP approved
generic RH products into their countries. The existing non-prequalified products are already registered
by the national medicines regulatory authorities, and the procurement authorities see both groups
(prequalified and locally registered products) as “good quality” for procurement. More evidence is
needed to make the case that the existing products are or may be poor quality (which is difficult) or that
the prequalified products are less costly than the existing products (unlikely at short notice), so that
prequalified products will be procured instead of the existing products. Policy decisions to introduce
these procurement principles are possible, but will take more time.
3.2 Where an evaluation is planned what progress has been made?
The Scope of Work for HERA includes also undertaking an impact evaluation of QuRHM in 2014/15 to
be funded by the Bill and Melinda Gates Foundation. This would probably be best scheduled in
2015/16, to allow maximum time for quality assured products to reach their target groups. The terms of
13
reference will be drafted in 2014.
4. Risk
4.1 Output Risk Rating: Medium (changed from High)
4.2 Assessment of the risk level
The Review Team reduces the overall risk assessment in the new 2013 logframe from High to Medium.
This is mainly due to output 1 performing better.
The Review Team recommends changing the Risk Assessment in annex E of the Business Case:
 Risk 1 (lack of supplier willingness to accept technical assistance): the probability remains Low
 Risk 2 (perceived lack of transparency of WHO process): this was not assessed in this review
although the process is reported as clearer by the manufacturers working with Concept (but
remains Medium)
 Risk 3 (procurement agencies do not agree/abide by quality policy): the probability has been
reduced from Medium to Low, given progress made
 Risk 4 (more PQ products do not lower prices): the probability remains Medium, given the
uncertainties on market impact
4.3 Risk of funds not being used as intended
The Review Team did not find or observe any indication of improper use of funds.
See also VfM discussion below.
4.4 Climate and Environment Risk
There are no changes in the risk assessment as per the business case. The project is likely to have a
potentially substantial positive environmental impact (increased voluntary family planning use with
implications for fertility and family size) and a limited direct detrimental environmental impact. Carbon
emissions will depend on frequency of meetings and travel to chosen countries. To mitigate this
meetings are held virtually where possible (VCs and teleconferences) and “piggy-backed” on other
international meetings. Through supporting and enabling pre-qualification, the project is helping generic
manufacturers to achieve internationally approved 'good manufacturing practice', which includes safe
disposal of hazardous waste during the manufacturing process. There is also some downstream risk
concerning unsafe disposal of unused or expired contraceptives at country level by clinics and users.
DFID’s investment with UNFPA’s Global Programme for enhanced Reproductive Health Commodity
Security includes an indicator and activities with country partners for implementing guidance on correct
disposal.
5. Value for Money
5.1 Performance on VfM measures
Economy: With regard to key cost drivers, the financial reports of Concept and partner UNFPA do not
provide details of unit costs or volumes, as they collate all costs under high-level budget lines. CF has
did not report on VfM in its annual report to RHSC or in its financial reports to DFID via PATH. This
makes it difficult for the Review Team to assess VfM in project implementation.
However, UNFPA staff salaries and short term assistance fees compare well against comparator
14
benchmarks. UNFPA, in common with most of the UN development system is signed up to the
International Civil Service Commission which sets salaries at professional grades based on US civil
service comparators (with allowances depending on location)i. For locally engaged staff the basket
comparator system comes into effect, where local salaries and allowances surveys are used to
competitively benchmark national postsii. UNFPA’s international and national fee schedule for short
term technical assistance is competitively priced, with a ceiling of $750 per day for international high
level experts (similar to rates offered by the European Commission and below those paid by most
international consulting companies)iii.
Concept Foundation’s average daily rates are consistent with WHO fees for similar work and well
below commercial rates. The CF travel policy was adapted for QuRHM related travel and is in line with
DFID travel requirements (economy travel).
The issue of “pass-through” funds (DFID-PATH-CF-UNFPA-WHO) has been addressed, and a formal
contract will be in place for the next ERP between Concept and WHO for the final process under the
current grant (thus avoiding additional overhead costs).
Efficiency: In response to the recommendation of the 2012 Annual Review, Concept Foundation
included two VfM indicators in the 2013 logframe:


Output 1 – (Number of RH dossiers accepted over 2 years of the QuRHM project in comparison
with 2008/9)
Output 2 – (Reduction in the number of separate technical assessments undertaken by UNFPA
in comparison with the number in 2011)
The Review Team is not convinced that these are good VfM indicators, as per the specific comments
on these VfM indicators in the respective outputs, and recommends that further work is done to
strengthen them.
Effectiveness and overall VfM: This is largely based on anticipated cost reductions of hormonal
contraceptives when more generic products are prequalified. Reported prices for prequalified
products have fallen from the baseline, for both combined oral and emergency contraceptive pills. This
is a very positive sign, and in accordance with the theory of change assumptions.
These price reductions are beginning to deliver procurement savings. UNFPA has completed its first
procurement of 0.45m oral pill cycles with a company producing the pre-qualified product at the lower
price, generating a saving of $30,000.UNFPA is in process of tendering and developing long term
agreements with the newly qualified manufacturers. It is unlikely that these long term agreements will
have delivered the anticipated cost savings by project completion. However, if procurement develops
as expected, including for injectables, by the end 2014, the project should have generated procurement
savings worth at least the value of its inputs of around £3 million. By 2015/16, when the evaluation is
expected, it will be possible to assess the total cost savings related to those products.
Recommendations:


Concept Foundation should provide more detail on price developments of the PQ and ERP
approved RH medicines against their branded or generic competitors in its final Project
Completion Report (based on RH Interchange and its own intelligence). It should also provide
analysis towards the outcome measure of savings achieved, together with UNFPA.
Concept Foundation and partner UNFPA should strengthen VfM indicators and report on VfM in
their progress and project completion reports.
5.2 Commercial Improvement and Value for Money
The DFID accountable grant contract to RHSC/CF through PATH does not include VfM reporting
requirements, and the progress and financial reports do not provide enough details allowing the
Review Team to draw any conclusions whether VfM is being maximised.
As noted above, Concept and UNFPA have competitive rates for short term assistance and salaries.
15
The country studies were awarded after an open international bidding procedure. The Review Team
had insight in the tender analysis, and the contract was awarded in compliance of international good
procurement practices.
The contract with UNFPA was unchanged; the financial reporting by UNFPA is not detailed enough to
enable the Review Team to draw conclusions whether VfM is being achieved. However, UNFPA’s
consultancy rates for technical experts are competitive, and in line with UNFPA’s overarching policy for
consultant costs. The issue of “pass-through” funds (DFID-PATH-CF-UNFPA-WHO) has been
addressed, and a formal contract will be in place for the next ERP between Concept and WHO for the
final process under the current grant (thus avoiding additional overhead costs).
5.3 Role of project partners
Concept Foundation contracted some deliverables of outputs 2 and 3 to partner UNFPA (which
subcontracted the ERP assessments to WHO), and it contracted the country baseline studies to the
Royal Tropical Institute in Amsterdam (KIT). The Review Team requested but did not receive
Concept’s analysis of their subcontractor KIT and partner UNFPA’s performance. CF has probably
underestimated the requirements of its role in managing and monitoring the deliverables by its
subcontractor and partners.
UNFPA was contracted to prepare, and organize the New York procurers’ meetings and the ERP
processes in output 2. The 2013 New York meeting helped to get a better understanding of the
challenges that procurers face at the country level. The 2013 report was much clearer than the 2012
report.
The ERP process is well organised and can be rated as successful, as it increased the number of
quality-assured RH medicines accessible to procurers by 12 in 2012 and 3 more by 31 July 2013.
Some 2012 communication glitches have been prevented thanks to a new standard operating
procedure on communicating ERP results.
WHO PQP was contracted by UNFPA to assess the ERP dossiers after initial screening by UNFPA.
The Review Team found that WHO did this job well.
The approach by UNFPA PSB for country workshops (organising them with the local UNFPA country
office) would be enhanced by greater involvement of UNFPA CSB and RHSC members at country
level.
5.4 Does the project still represent Value for Money?
Spending less money for the same or greater value (in terms of quality assured RH medicines
procured) is a key hypothesis in the DFID business case. It is a positive sign that prices of the
prequalified generic oral pills have dropped from $0.29 in 2011 to $0.225 in 2013 (23% reduction).
Over the same period emergency contraceptive prices dropped from $0.31 to $0.26.
These price reductions are beginning to deliver procurement savings. UNFPA has completed its first
procurement of 0.45m oral pill cycles with a company producing the pre-qualified product at the lower
price, generating a saving of $30,000. UNFPA is in process of tendering and developing long term
agreements with the newly qualified manufacturers, so it is not yet possible to assess the total cost
savings related to those products.
Attribution of these price changes to the QuRHM project is problematic, given limited information about
dynamics of the global market and the portion of it which is influenced by QuRHM, but it is likely that
the project contributed.
The main savings in the Business Case are expected from reduced prices of quality approved generic
injectables: unfortunately, these products are not yet available (only one product received an ERP
category 3 rating, and so can be procured in exceptional circumstances). However, at least two
16
injectables are expected to receive a favourable rating by project end. The three month injection (with a
baseline price of 0.86) has increased to over $1 due to quality supply shortages. One generic product
was assessed as ERP category 3, for procurement only where no alternative. In case this is necessary,
UNFPA has negotiated a long term agreement for the product at a price well below the original
baseline, showing that price reductions are highly achievable. Actual price savings should be
generated and calculable by the evaluation in 2015/16.
Although the business case mentions Value for Money (VfM), the DFID contract with PATH/Concept
Foundation does not include specific VfM reporting requirements, and the new VfM indicators need
strengthening.
5.5 If not, what action will you take?
The Review Team recommends that DFID request Concept Foundation and partners to report on VfM
in the Project Completion Report. The Review Team will then independently assess these reports.
Overall VfM in terms of potential cost savings will be assessed as part of the evaluation.
6. Conditionality
6.1 Update on specific conditions
Not applicable
7. Conclusions and actions
In year 2 of QuRHM, significant progress is being made in output area 1, with 10 generic RH products
quality assured by WHO, where all received technical assistance from the project. Prices for quality
assured oral contraceptive products have decreased, and attribution to Concept’s activities is plausible
although needs more analysis.
Progress in the other outputs (mostly contracted to UNFPA, WHO and consultants) is variable, and
would benefit from more attention by Concept management, and greater involvement of the RHSC
secretariat and community.
Achieving prequalification or ERP approval is no guarantee that the quality assured products will be
quickly available and procured by countries, and available to consumers and at reduced prices. More
attention needs to be given to the downward pipeline problems in countries and the economics of the
hormonal contraceptives market. Evidence for a business case for generic RH companies to invest in
PQ is weak at present, and will need reassuring data on future needs, markets and orders.
The project’s activities are delayed at country level, but continue to be important with respect to
building commitment in the four selected countries to quality standards.
Concept Foundation’s management is still overburdened, probably by over-reliance on the Project
Manager and not enough delegation to other CF staff. CF should also strengthen the monitoring of
outsourced or partner’s programmes. Concept Foundation and RHSC members could collaborate more
intensely at country level to provide linkages with existing programmes to strengthen RH commodity
security, strengthen National Medicines Regulatory Authorities, procurement agencies etc.
The potential benefits of the project make taking the risk worthwhile. The Review Team has also
recommended that the risk be reduced from high to medium. The pipeline of generic products contains
17
some urgently awaited RH medicines (especially injectables, misoprostol, and oxytocin). A more
focused and risk-based approach towards these products would be useful.
The Technical Advisory Committee meeting in Delhi took some good initiatives for RHSC to plan for
the post-QuRHM period. The Review Team is hopeful that another donor will allow CF to ensure that
the much anticipated products in the pipeline reach prequalification status, and that the combined
efforts of RHSC members at country level and a new Quality Working Group can continue the good
work that CF has done in the QuRHM project.
DFID transparency policy requires that Annual Review reports should be made public. The 2012
Annual Review is now available at a UK government website3. On 18 April 2013 BMGF asked RHSC
on behalf of DFID and with permission of UNFPA and WHO to make the full 2012 Annual Review
report available on the RHSC website. RHSC will now also publish the 2012 annual review report on its
website.
This Year 2 review has scored the project B (ABCA). Performance has improved in most areas, and
the review team judge that the project can score an A.
Update on 2012/13 project improvement action plan and recommended 2013/14 actions
1. Ensuring realistic targets are achieved
 The design of QuRHM was highly ambitious and a number of anticipated results were not delivered
in Y1. In response to the AR recommendations, the logframe has been amended to ensure
milestones are SMARTer and at the appropriate level of ambition. Concept, with inputs from DFID,
are recommended to further SMARTen the indicators, by end 2013/early 2014.
 VfM indicators, in addition to existing purpose level indicators on commodity costs, will be added to
demonstrate efficiencies in procurement behaviour and acceleration of products through PQP/ERP
process. The review notes that indicators were added and go some way towards demonstrating
improved VfM. Again the reviewers recommend further SMARTening. Concept will work with a lead
economist in PD (James Droop) and WHO to improve these further (end 2013/early 2014).
 Concept Foundation in Y2 will prioritise TA support to suppliers with greatest potential to achieve
WHO PQP/ERP, those who supply markets in developing countries, or commodities in neglected
categories (namely injectables). This should improve programme focus and delivery. This has been
done – the reviewers note that the number of companies engaged with Concept has dropped from
32 to 23. Concept is recommended to continue to prioritise products supported in relation t o
market needs.




2. Improving programme management
Delays in programme due to contracting and lack of capacity in Y1 have been addressed through
recruitment of additional CF management staff and an organisational restructure. Additional staff
have been recruited, and financial management and reporting for example have improved. Concept
senior management are recommended to continue to build up staff capacity and delegate tasks,
and include actions taken on this in the project completion report.
Concept Foundation has initiated more regular and planned meetings between implementing
partners, to ensure better collaboration. Communications between partners have improved, and
Concept will continue to manage partner outputs more effectively, reporting progress to DFID in the
project completion report.
A joint critical pathway tool has been developed between Concept and WHO to better track
products through PQP/ERP process and help manage different roles and responsibilities of
respective organisations. The tool is improving Concept’s management of each product pathway
though the reviewers recommend that WHO also fully utilise it. Concept will review this with WHO
in early 2014, with the intention of agreeing a shared tool.
Risk management will be addressed at each programme oversight committee meeting. Risks have
been downgraded from high to medium. Major bottlenecks and risks have been addressed in Year
2: Gates is funding bio-equivalence studies for injectables, and WHO has agreed a rolling ERP for
injectables.
3 http://devtracker.dfid.gov.uk/projects/GB-1-202825/documents/
18




Interim review scheduled for summer 2013 to assess progress against recommendations and Y2
milestones, full in-depth review by March 2014 looking at progress in 4 study countries, and impact
evaluation by 2015 looking at impact on broader market dynamics. This schedule is on track. The
reviewers recommend that that the evaluation is scheduled later in 2015/16 (paid for by the Gates
Foundation).
3. Improving value for money
Economy – issue regarding pass through funds currently being resolved to deliver savings on
overheads. This has been resolved, with the final ERP contracted by Concept to WHO directly.
Concept and UNFPA will provide more information on input costs and other VfM indicators in the
project completion report.
Efficiency and Effectiveness – bottlenecks being identified and documented as part of the critical
pathway tool and mitigating actions identified, improved partnership/links with other support
(RHSC, BMGF work on DMPA). This is being achieved through the critical pathway tool, although
the indicators need improvement (see above recommendation).
Overall VfM – this needs to be better tracked through VfM indicators and market analysis. The
reviewers found that prices had indeed fallen for both combined oral, and emergency contraceptive
pills, by Year 2, which is a positive sign. The 23% reduction in price of generic combined oral
contraceptives has generated savings through new UNFPA procurement of 1m cycles. Concept
should include more contextual market and pricing data, and manufacturers’ statements, to show
the link between price drops and project inputs, in its project completion report.
8. Review Process
The TORs for this Annual Review were finalised by DFID on 4 July 2013. The review was conducted
by Wilbert Bannenberg of HERA between 9 July and 28 October 2013. The process took longer than
expected due to some delay in receipt of the revised logframe. Key staff of Concept Foundation,
RHSC, PATH, and DFID were interviewed, and RHSC members and manufacturers were interviewed
during the RHSC meeting in Delhi. A Google search was conducted on the keyword “QuRHM”. DFID,
PATH and Concept Foundation provided project documents.
The Review Team (HERA) signed a confidentiality or non-disclosure agreement with Concept for the
review, including the list of companies they are or have been supporting. This enabled the Review
Team to have a better insight into the correlation between CF’s activities and progress under
milestones 1 & 2 under Output 1. The Review Team was able to verify evidence of CF’s activities as to
which generic manufacturers they supported, how and when specific support steps were taken, and
when the manufacturer submitted dossiers to PQ or ERP. Internal QA (Marianne Schürmann, HERA)
and external QA (Cheri Grace, independent consultant) was applied to the report.
Draft conclusions and a summary report were presented to the POC on 7 November 2013 in London.
An updated report was submitted on 15 November to DFID. This review is largely based on HERA’s
report and scoring assessments, with some amendments and additions made by DFID.
There will be a full joint annual review on WHO/PQP and QuRHM and a separate Project Completion
Review of QuRHM in April-May 2014. An Evaluation is planned (likely to be re-scheduled from 2014 to
2015/16).
i
http://icsc.un.org/secretariat/sad.asp and http://www.un.org/Depts/OHRM/salaries_allowances/salary.htm
ii
iii
Manual for Salary Surveys in Duty Stations other than Headquarters and similar duty stations, UN 2013
Personnel Policies and Procedures Manual, UNFPA 2010
19