Using Multi-Criteria Decision Analysis to Assess Private Sector Agents' Preferences
and Priorities in Stocking Malaria Rapid Diagnostic Test Kits in Uganda
ARCHVES
By
MASSACHUSETTS INSTITI
OF rECHNOLOLGY
Corinne M. Carland
S.B. Chemical Engineering and Biology
Massachusetts Institute of Technology, 2013
MAY 2 6 2015
LIBRARIES
SUBMITTED TO THE ENGINEERING SYSTEMS DIVISION IN PARTIAL FULFILLMENT
OF THE REQUIREMENTS FOR THE DEGREE OF
MASTER OF SCIENCE IN TECHNOLOGY AND POLICY
AT THE
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
JUNE 2015
0 2015 Corinne M. Carland All rights reserved.
The author hereby grants MIT permission to reproduce
and to distribute publicly paper and electronic
copies of this thesis document in whole or in part
in any medium now known or hereafter created.
Signature of Author:
Signature redacted
Technology and Policy Program, Engineering Systems Division
May 8, 2015
Certified by:
Signature redacted
Jarrod Goentzel
Director, Humanitarian Response Laboratory
Thesis Supervisor
Accepted by:
________Signature redacted
Dava J. Newman
Pro ssor of Aeronautics and Astronautics
and Engineering Systems
Director, Technology and Policy Program
IT F
Using Multi-Criteria Decision Analysis to Assess Private Sector Agents'
Preferences and Priorities in Stocking Malaria Rapid Diagnostic
Test Kits in Uganda
By
Corinne Carland
Submitted to the Engineering Systems Division on 14 May 2015
in partial fulfillment of the requirements for the degree of
Master of Science in Technology and Policy
Abstract
Diagnosis of malaria is important in order to ensure early and effective treatment, to
facilitate public health surveillance, and to prevent drug resistance. Rapid diagnostic
tests (RDTs) are an important tool in resource-constrained settings, as they do not
rely on costly lab equipment and specially trained personnel. In Uganda's private
sector clinics and drug shops, which is where the majority of patients first seek care,
diagnosis of malaria is often presumptive and patients receive neither RDT nor
microscopy. Several studies have focused on the patient perspective (e.g.
willingness to pay and willingness to be tested) but much less is understood about
the supplier perspective (e.g. willingness to stock). This study aimed to understand
the preferences and priorities of agents across the malaria RDT supply chain in
Uganda on stocking the devices using multi-criteria decision analysis. This
methodology was adapted to be relevant and understandable for agents in Uganda
so that it was possible to analyze business decisions incorporating a multiplicity of
attributes such as selling price, purchase cost, sales volume, complexity of
regulations, waste management, and training available. Data surveys and semistructured interviews were collected from 28 private sector retailers (i.e.,
shopkeepers, pharmacists, clinic managers), two first line buyers, and three
distributors. Analysis of the data resulted in the construction of value functions for
all agents, the relative weights (therefore the tradeoffs) among decision criteria, and
the calculation of an overall value for the decision about whether or not to stock
RDTs for the different supply chain agents. Results indicate that the best option for
one level of the supply chain is not necessarily the best for another. A discussion
offers insights on how to align value across the supply chain, which is important for
facilitating public health interventions.
Thesis Supervisor: Jarrod Goentzel
Title: Director, Humanitarian Response Laboratory
1
Acknowledgements
This work was made possible by a grant from the U.S. Agency for International
Development (USAID), and I want to express my gratitude for their support.
Further, I am thankful for MIT's Comprehensive Initiative for Technology Evaluation
(CITE) for facilitating and supporting the research of their students.
I have benefitted from the expert guidance and help of my adviser, Dr. Jarrod
Goentzel. When I expressed a research interest, Jarrod was the person who made it
happen, proposing this research on my behalf. His confidence in me,
encouragement, and advice helped me grow immensely as a researcher and
professional. I am very appreciative that I had the opportunity to work in the
Humanitarian Response Lab for the past two years.
I am also deeply grateful for the support and advice of Dr. Gilberto Montibeller.
Without his expertise and willingness to introduce me to the field of decision
sciences, this thesis would not have been possible. I am thankful for the time he
spent on calls, providing feedback, and answering many questions.
Further, this work could not have been completed without Elizabeth Streat in
Uganda. Her and her team's advice and suggestions were invaluable in guiding
implementation and shaping our research. Additionally, this work could not have
been completed without Erin Reissman, who traveled with me to Uganda to do data
collection. Thank you for helping to shape and implement this research and also for
ensuring we also had fun during our travels.
No acknowledgement section would be complete without reference to one's family,
but cliched as it is, they have been an unending source of support and
encouragement. Especially my mother who has spent many hours listening to me
talk about research and school. Further, friends who have become family were
integral over these past two years and so I would like to thank Camila, Noam, and
Carlos.
TPP was a wonderful experience and has challenged me to grow and learn. I need to
thank Barb, Ed, Frank, and Dava for their support and advice. Further, TPP and ESD
friends and colleagues made these two years enriching and fun; I am grateful to be
surrounded by such inspiring individuals-thank you, Lauren, Stacey, Tim, Emily,
and Mark.
Finally, thank you to MIT for being my home for these past six years and shaping me
into the person I am today.
2
Contents
A b strac t ..........................................................................................................................................................
1
Acknowledgements...................................................................................................................................
2
Co nten ts . .;... .............................................................................................................................................
3
List of Tables...................................................................................................
6
L ist o f F igu res ..............................................................................................................................................
8
L ist o f A cro n y ms......................................................................................................................................
10
1 In tro d uctio n .......................................................................................................................................
11
2 Background ............................................................................................
12
2.1 Malaria............................................................................................
12
2 .1 .1 T reatm en t..........................................................................................................................
13
2 .1.2 D iagn o sis............................................................................................................................
13
2.2 Malaria Rapid Diagnostic Tests ....................................................................................
14
2.2.1 W HO Approval for Malaria Rapid Diagnostic Tests ................................... 16
2.3 Malaria Rapid Diagnostic Tests and Healthcare Systems in Uganda............17
2.3.1 Government Regulation...........................................................................................
17
2.3.2 Private Sector Retailers ..........................................................................................
17
2.3.3 Interventions to Scale up mRDT Use.................................................................
18
2.4 Pilot to Scale up use of mRDTs in Uganda...............................................................
18
2.5 Multiple Criteria Decision Analysis.............................................................................
21
2.5.1 History and Motivation of Decision Analysis .................................................
22
2.5.2 Applications of MCDA..............................................................................................
23
2.6 Aim of this Research Study.................................................................................................
23
3 Methodology...........................................................................................
24
3.1 Research Scope ..................................................................................
24
3.2 Data Collection...................................................................................
25
3.2.1
Semi-Structured Interviews with First-Line Buyers and Distributors ... 25
3.2.2 Focus Group Discussions with Retailers .........................................................
3.3 Methodology Framework................................................................................................
3.3.1 Decision Framing............................................................................................................
3.3.2 Identifying Criteria........................................................................................................
3.3.3 Eliciting value functions..........................................................................................
3.3.4 Determining Priorities (W eighting).................................................................
3.3.5 Generation of decision matrix inputs ...............................................................
3.3.6 Evaluating options.........................................................................................................
3.3.7 Sensitivity Analysis...................................................................................................
4 Inputs and Assumptions...............................................................................................................
4.1 Decision Input Matrix............................................................................................................
4.1.1 Decision Input Matrix - Retailers.......................................................................
4.1.2 Decision Input Matrix - Distributors...............................................................
4.1.3 Decision Input Matrix - First Line Buyers.......................................................
5 Results - Value Functions.........................................................................................................
5 .1 R eta ilers C riteria .....................................................................................................................
5.2 Retailers Value Functions................................................................................................
26
27
28
30
40
42
44
44
45
46
46
46
52
57
60
60
61
3
5.2.1 Quality ................................................................................................................................. 61
5.2.2 Cost ....................................................................................................................................... 63
5.2.3 Price ..................................................................................................................................... 65
5.2.4 T im e to D elivery ............................................................................................................. 67
5.2.5 Sales of O ther Products ................................................................................................ 69
5.2.6 Custom er Satisfaction ................................................................................................... 71
5.2.7 T raining .............................................................................................................................. 72
5.2.8 V olum e ................................................................................................................................ 74
5.2.9 A wareness/A dverti sing ............................................................................................... 76
5.2.10 T im e to Com plete a Sale ........................................................................................... 78
5.2.11 O pportunities ................................................................................................................ 80
5.3 D istributors ............................................................................................................................... 82
5.3.1 V olum e ................................................................................................................................ 82
5.3.2 Expiration D ate ............................................................................................................... 85
5.3.3 Profit M argin .................................................................................................................... 87
5.3.4 Cost Per Kit ........................................................................................................................ 89
5.3.5 Cost per T raining ............................................................................................................ 90
5.3.6 Effi ciency of D istribution ............................................................................................ 92
5.3.7 Cross Selling ...................................................................................................................... 95
5.4 First Line B uyers ..................................................................................................................... 97
5.4.1 Cost per D evice ................................................................................................................ 97
5.4.2 Price per Device .............................................................................................................. 99
5.4.3 Quality of D evice .......................................................................................................... 101
5.4.4 A dm inistrative T im e .................................................................................................. 103
5.4.5 Percent Profit ................................................................................................................ 104
5.4.6 R elationship w ith D onor O rganization .............................................................. 106
5.4.7 Percent Increase in Sales ......................................................................................... 108
6
R psults - Wpiprhtq -----
................... ............................................................................ 109
6.1 R etailers ...................................................................................................................................
6.2 D istributors ............................................................................................................................
6.3 First Line B uyers ..................................................................................................................
7 R esults - O verall V alue ...............................................................................................................
7.1 O verall Value - R etailers ...................................................................................................
7.2 O verall Value - D istributors ............................................................................................
7.3 O verall Value - First Line B uyers ..................................................................................
7.4 Overall Value - Comparison Between Agents .........................................................
8 Sensitivity A nalysis ......................................................................................................................
8.1 D istributors ............................................................................................................................
8.1.1 V olum e .............................................................................................................................
8.1.2 Expiration D ates ..........................................................................................................
8.1.3 Profit M argin .................................................................................................................
8.1.4 Cost per Kit .....................................................................................................................
8.1.5 Cost per T raining .........................................................................................................
8.1.6 Effi ciency of D istribution .........................................................................................
8.1.7 Cross Selling ...................................................................................................................
8.2 First Line B uyers ..................................................................................................................
110
111
112
113
113
114
116
118
120
121
121
123
124
126
127
129
130
131
4
8.2.1
Cost per Kit.....................................................................................................................
131
8.2.2
Price per Kit...................................................................................................................
133
8.2.3
Q uality ..............................................................................................................................
134
8.2.4 A dm inistrative T im e ..................................................................................................
8.2.5 Profit M argin .................................................................................................................
8.2.6 R elationship with D onor ..........................................................................................
8.2.7 Increase in Sales...........................................................................................................
9 D iscussion ........................................................................................................................................
9.1 R esult from Analysis...........................................................................................................
9.1.1 Best O ptions for A gents ............................................................................................
9.1.2 Sensitivity A nalysis.....................................................................................................
9.1.3 Im plications and Im proving O ptions..................................................................
9.1.4 Insights from Value Functions and Weighting ..................
9.1.5 Em ergent Criteria........................................................................................................
9.2
10
136
138
139
14 1
14 1
14 1
142
143
144
147
152
M ethodology ..........................................................................................................................
153
9.2.1 D ata Collection..............................................................................................................
9.2.2 D ata A nalysis.................................................................................................................
Conclusion and Future R esearch D irection ....................................................................
153
158
160
R eferences...............................................................................................................................................
162
5
List of Tables
Table
Table
Table
Table
Table
Table
Table
Table
Table
Table
Table
1. Antigens used in m RDTs ...............................................................................................
16
2. List of Criteria and Definitions for Retailers.........................................................
33
3. Criteria and m etric units ................................................................................................
34
4. Qualitative Metric Definitions for Retailers.........................................................
35
5. Criteria and ranges for retailers................................................................................
36
6. List of Criteria and Definitions for Distributors................................................... 37
7. Criteria and ranges for distributors .........................................................................
38
8. List of Criteria and Definitions for Distributors................................................... 39
9. Criteria and ranges for FLBs........................................................................................
39
10. Qualitative Metric Definitions for FLBs...............................................................
40
11. Input figures for retailers for high, base, and low estimates for options 1-4
........................................................................
...........
...... 47
Table 12. Input figures for distributors for high, base, and low estimates for options
1 -4 ........................................................................................................................................................
53
Table 13. Input figures for distributors for high, base, and low estimates for options
1 -4 ........................................................................................................................................................
58
Table 14. Criteria and ranges of criteria for retailers. ........................................................
61
Table 15. Median of retailer responses for quality ............................................................
62
Table 16. Median of retailer responses for cost per device.............................................. 64
Table 17. Median of retailer responses for price per device........................................... 66
Table 18. Removed response sets for price of mRDT .........................................................
67
Table 19. Median of retailer responses for delivery time ................................................
68
Table 20. Removed responses for Delivery Time ..............................................................
69
Table 21. Median of retailer responses for increased sales of other products ..... 70
Table 22. Median of retailer responses for customer satisfaction ................................ 72
Table 23. Median of retailer responses for training ............................................................
73
Table 24. Median of retailer responses for volume.............................................................
75
Table 25. Median of retailer responses for advertising/awareness............................ 77
Table 26. Removed responses for Advertising/Awareness ............................................. 78
Table 27. Median of retailer responses for advertising/awareness............................ 79
Table 28. Removed responses for time to complete sale ................................................
80
Table 29. Median of retailer responses for opportunities ...............................................
81
Table 30. Criteria and ranges of criteria for distributors ................................................
82
Table 31. Distributor 1 responses for volume.......................................................................
84
Table 32. Distributor 2 responses for volume.......................................................................
84
Table 33. Distributor 1 responses for expiration date ......................................................
86
Table 34. Distributor 2 responses for expiration date ......................................................
86
Table 35. Distributor 1 responses for profit margin .........................................................
88
Table 36. Distributor 2 responses for profit margin .........................................................
88
Table 37. Distributor 1 responses for cost per kit...............................................................
89
Table 38. Distributor 2 responses for cost per kit...............................................................
90
Table 39. Distributor 1 responses for cost per training....................................................
91
Table 40. Distributor 2 responses for cost per training....................................................
92
6
Table 41. Distributor 1 responses for efficiency of delivery............................................ 94
Table 42. Distributor 1 responses for efficiency of delivery............................................ 94
Table 43. Distributor 1 responses for cross selling.............................................................
96
Table 44. Distributor 2 responses for cross selling.............................................................
96
Table 45. FLB 1 responses for cost per device.......................................................................98
Table 46. FLB responses for cost per device.......................................................................
99
Table 47. FLB responses for price per device......................................................................
100
Table 48. FLB responses for price per device......................................................................
100
Table 49. FLB responses for quality of device ....................................................................
101
Table 50. FLB responses for quality of device .....................................................................
102
Table 51. FLB responses for administrative time ..............................................................
103
Table 52. FLB responses for administrative time..............................................................
104
Table 53. FLB responses for profit ...........................................................................................
105
Table 54. FLB responses for profit ...........................................................................................
106
Table 55. FLB responses for relationship with donor......................................................
107
Table 56. FLB responses for relationship with donor......................................................
108
Table 57. FLB 2 responses for percent increase in sales ....................................................
109
Table 58. Criteria and Number of Responses ..........................................................................
110
Table 59. Median and normalized weights...............................................................................
111
Table 60. W eights for Distributor 1.............................................................................................
111
Table 61. W eights for Distributor 2 .............................................................................................
112
Table 62. W eights for FLB 1............................................................................................................
112
Table 63. W eights for FLB 2 ............................................................................................................
113
Table 64. Overall Value for Retailer Options............................................................................
114
Table 65. Overall Value for Distributor 1 Options .................................................................
115
Table 66. Overall Value for Distributor 2 Options .................................................................
115
Table 67. Overall Value for Distributor 1 Options - Cost per kit is removed ............ 116
Table 68. Overall Value for Distributor 2 Options - Cost per kit is removed ............ 116
Table 69. Overall Value for FLB 1 ................................................................................................
117
Table 70. Overall Value for FLB 2..................................................................................................
117
Table 71. Overall Value for FLB 1 Options - Profit removed ............................................ 118
Table 72. Overall Value for FLB 2 Options - Profit removed ............................................ 118
Table 73.Comparison of overall value between different agents in the supply chain
(using the baseline assumptions) .......................................................................................
142
Table 74. W eights for FLB 1............................................................................................................
151
Table 75. W eights for FLB 2 ............................................................................................................
151
7
List of Figures
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
Figure
1. Diagram of the mode of action commonly used in mRDTs. ......................... 15
2 M ap of Uganda and districts .......................................................................................
19
3. Supply chain of mRDTs in Uganda in this study................................................. 20
4. Decision flow chart for agents in the supply chain........................29
5. Example given to respondents to elicit criteria................................................. 31
6. Numerical and verbal descriptions of happiness to elicit value functions. 41
7. Value function for quality of devices for retailers........................................... 62
8.. Value function for cost per device for retailers................................................ 64
9. Value function for price per device for retailers.............................................. 66
10. Value function for delivery time for retailers...................................................68
11. Value function for sales of other products for retailers............................. 70
12. Value function for customer satisfaction for retailers.................................. 72
13. Value function for training for retailers............................................................
73
14. Value function for volume of sales for retailers..............................................75
15. Value function for awareness/advertising for retailers ............................. 77
16. Value function for time to complete a sale for retailers.............................. 79
17. Value function for opportunities for retailers.................................................81
18. Value function for Distributor 1 for volume....................................................
83
19. Value function for Distributor 2 for volume....................................................
84
20. Value function for Distributor 1 for expiration date .................................... 85
21. Value function for Distributor 2 for expiration date .................................... 86
22. Value function for Distributor 1 for profit margin ........................................ 87
23. Value function for Distributor 2 for profit margin ........................................ 88
24. Value function for Distributor 1 for cost per kit..............................................89
25. Value function for Distributor 2 for cost per kit..............................................90
26. Value function for Distributor 1 for cost per training.................................. 91
27. Value function for Distributor 2 for cost per training.................................. 92
28. Value function for Distributor 1 for efficiency of distribution..................93
29. Value function for Distributor 2 for efficiency of distribution..................94
30. Value function for Distributor 1 for cross selling ........................................... 95
31. Value function for Distributor 2 for cross selling ........................................... 96
32. Value function for FLB 1 for cost per device....................................................
97
33. Value function for FLB 2 for cost per kit............................................................
98
34. Value function for FLB 1 for price per kit..........................................................
99
35. Value function for FLB 2 for price per kit............................................................
100
36. Value function for FLB 1 for quality.......................................................................
10 1
37. Value function for FLB 2 for quality.......................................................................
102
38. Value function for FLB 1 for adm inistrative tim e ............................................ 103
39. Value function for FLB 2 for adm inistrative tim e ............................................ 104
40. Value function for FLB 1 for profit .........................................................................
105
41. Value function for FLB 2 for profit .........................................................................
106
42. Value function for FLB 1 for relationship ............................................................
107
43. Value function for FLB 2 for relationship ............................................................
108
8
Figure 44. Value function for FLB 2 for percent increase in sales .................................. 109
Figure 45.Distributor versus retailer overall value for all of the high/base/low
o ptio n s 1 -4 ....................................................................................................................................
1 19
Figure 46. FLB/Distributor overall value versus Retailer overall value for all of the
high/base/low options 1-4 ....................................................................................................
120
Figure 47. Overall value versus weight for volume for distributor 1............................ 122
Figure 48. Overall value versus weight for volume for distributor 2............................ 122
Figure 49. Overall value versus weight for expiration dates for distributor 1......... 123
Figure 50. Overall value versus weight for expiration dates for distributor 2.........124
Figure 51. Overall value versus weight for profit margin for distributor 1...............125
Figure 52. Overall value versus weight for profit margin for distributor 2 ............... 125
Figure 53. Overall value versus weight for cost per kit for distributor 1.............. 126
Figure 54. Overall value versus weight for cost per kit for distributor 2.............. 127
Figure 55. Overall value versus weight for cost for training for distributor 1.........128
Figure 56. Overall value versus weight for cost for training for distributor 2.......... 128
Figure 57. Overall value versus weight for cost for efficiency of distribution for
d istrib u tor 1 . ................................................................................................................................
1 29
Figure 58. Overall value versus weight for cost for efficiency of distribution for
d istrib u to r 2 . ................................................................................................................................
1 30
Figure 59. Overall value versus weight for cross selling for distributor 1.................. 130
Figure 60. Overall value versus weight for cross selling for distributor 2..................131
Figure 61. Overall value versus weight for cost per kit for FLB 1................................... 132
Figure 62. Overall value versus weight for cost per kit for FLB 2................................... 132
Figure 63. Overall value versus weight for price per kit for FLB 1................................ 133
Figure 64. Overall value versus weight for price per kit for FLB 2. ............................... 134
Figure 65. Overall value versus weight for quality for FLB 1........................................... 135
Figure 66. Overall value versus weight for quality for FLB 2 ........................................... 135
Figure 67. Overall value versus weight for administrative time for FLB 1................. 137
Figure 68. Overall value versus weight for administrative time for FLB 2................. 137
Figure 69. Overall value versus weight for profit margin for FLB 1.............................. 138
Figure 70. Overall value versus weight for profit margin for FLB 2. The dashed line
m arks w here the options intersect.....................................................................................
139
Figure 71. Overall value versus weight for relationship with donor for FLB 1.......140
Figure 72. Overall value versus weight for relationship with donor for FLB 2........ 140
Figure 73. Overall value versus weight for increase in sales for FLB 2. The dashed
line marks w here the options intersect............................................................................
141
9
List of Acronyms
ACT
FLB
MCDA
mRDT
NGO
NDA
PDS
WHO
Artesiminin combination therapy
First line buyer
Multi criteria decision analysis
Malaria rapid diagnostic test
Non governmental organization
National Drug Authority
Panel detection score
World Health Organization
10
1
Introduction
Scaling up the use of malaria rapid diagnostic tests (mRDTs), especially in the
private sector, is an important initiative in global health. To do this, one has to take
into consideration the multiplicity of agents that are involved in the private sector
across the supply chain, including manufacturers, first line buyers, distributors, and
retailers. These agents need to work together in a concerted fashion to ensure that
products reach the end user. Each agent in the supply chain independently
considers whether or not to stock the mRDTs, what type of products to stock, and
whether or not to engage with NGOs or non-profits on intervention strategies. The
goal of this research project was to understand how each of these individuals makes
such a decision and the important and priorities of factors that are considered in the
decision. A multi criteria decision analysis (MCDA) methodology was employed to
quantify preferences and the best options for the different agents. This research was
a pilot study that tested the feasibility of this kind of rapid evaluation and the
application of MCDA in this context. This paper is divided into four main sections:
Background, Methodology, Results, and Discussion. The Background section
provides information on malaria, malaria rapid diagnostic test kits, MCDA
methodology, and the context of this study. The Methodology section explains how
MCDA was applied in this setting and the limitations and adaptations that were
necessary. The Results section summarizes the value functions, weights, overall
value for each of the supply chain agents, and a sensitivity analysis. The Discussion
section comments on the results, limitations, and suggestions for further research.
Additionally, it notes other insights that were derived over the course of data
collection and analysis.
11
2
Background
This section provides more information on malaria and rapid diagnostic tests.
Additionally, it provides context about this research study and the methodology that
was applied.
2.1
Malaria
The WHO estimates that globally 1.2 billion people are at high risk for malaria
(greater than a 1 in 1000 risk of contracting the disease in the year). In 2013 there
were an estimated 198 million cases, which caused 584,000 deaths. The greatest
malaria burden is in Africa, where 90% of the cases occur (World Health
Organisation, 2014b). Malaria is caused by a five species of parasites in the
Plasmodium genus, which are transmitted to human beings via the bite of an
infected Anopheles mosquito. One parasite, Plasmodiumfalciparum,is the most
prevalent in Africa and causes the most deaths (World Health Organisation, 2014b).
The malaria parasite first enters the human body after being bitten by the mosquito
and then travels to the liver. After an incubation period of several days in the liver,
the parasites enter red blood cells and ultimately rupture them, resulting in the
disease symptoms. These symptoms of malaria include fever, shivering, respiratory
distress, headaches, diarrhea, vomiting, and crnvlidnsnc Whilemost malaria is
considered uncomplicated, a small percentage (around 5%) of cases are severe.
Severe malaria is characterized by cerebral malaria, pulmonary edema, acute renal
failure, and anemia (Trampuz, Jereb, Muzlovic, & Prabhu, 2003). Cerebral malaria is
caused by the obstruction of small blood vessels in the brain due to build up of the
parasite (MacKintosh, Beeson, & Marsh, 2004). If left untreated, severe malaria is
deadly. Pregnant women and children are at especially high risk of mortality from
malaria (Tuteja, 2007). In Uganda, the Ministry of Health (MOH) reports that
malaria is the leading cause of morbidity and mortality in the country, accounting
for between 8-13 million cases per year, 35% of hospital admissions, and 9-14% of
hospital deaths (Yeka et al., 2012).
12
2.1.1
Treatment
The WHO recommends uncomplicated malaria caused by P.falciparumbe treated
with an artemisinin combination therapy (ACT) (World Health Organisation,
2014b). Artemisinin is currently the most effective and fast acting medicine to
combat malaria. An ACT is a combination therapy that includes artemisinin and a
companion antimalarial. Antimalarial medications with just artemisinin are called
artemisinin monotherapies; these pharmaceuticals are a threat to public health as
they are assumed to promote resistance (see subsequent sections for discussion on
resistance). Expansion of ACTs could be considered one of the most significant
concerted global health intervention efforts of the decade. It began with the Institute
of Medicine's (IOM's) convening of a panel to explore interventions to address
growing resistance to malaria medications and their subsequent central
recommendation to establish a global subsidy on artemisinin combination
treatments (Gelband, Panosian, & Arrow, 2004). As a result of this recommendation,
the Global Fund to Fight AIDS, Tuberculosis, and Malaria approved an initiative
called the Affordable Medicines Facility for malaria (AFMm) to increase access to
ACTs by subsiding purchases of the devices at the upstream levels and reselling the
medicines at a fraction of the cost (Bate & Hess, 2009). The rational is that the
subsidy will be carried down the supply chain to the retailer level where patients
will be able to access affordable ACT products. The results of the global subsidy are
monitored in periodic surveys by country. Initial results indicate that access to ACTs
in the private sector are slowly increasing (O'Connell et al., 2011).
2.1.2
Diagnosis
The WHO's official recommendation is that every suspected case of malaria should
be examined for evidence of infection by either microscopy or a malaria rapid
diagnostic test (mRDT) (World Health Organisation, 2014b). The method of
microscopy involves taking a sample of blood from a patient and examining it under
a microscope to visually determine the presence of parasites. If performed by a
trained technician with adequate laboratory resources, microscopy is the ideal and
most reliable mechanism of diagnosis. However, in resource-constrained settings,
13
the lack of time, training, and materials may render this method challenging and less
reliable. In these settings, an mRDT is the best mechanism of diagnosis.
Many studies have found that there is a lack of a rigorous diagnosis of malaria in
clinics and drug shops in Sub-Saharan Africa and presumptive treatment, based on
symptoms alone, results in an overtreatment of patients who do not have the
disease (Wilson, 2012). Over-treatment of malaria is detrimental to the patient
because it needlessly exposes the individual to antimalarials and delays the
appropriate treatment. It is further detrimental to the community because it is a
waste of resources in areas that cannot afford to do so and also costly to society
because of the global subsidy of treatments. Finally, inappropriate use of
antimalarials may contribute to antimalarial drug resistance of the parasite (Wilson,
2012), much like over prescription of antibiotics is connected with antibiotic
resistant bacteria strains. Antimalarial drug resistance is an enormous public health
concern. P. falciparum has already developed widespread resistance to older
antimalarials, including chloroquine and sulfadoxine-pyrimethamine (White, 2004).
Resistance to artemisinin, one of the most efficacious and recently developed
antimalarial, has emerged in the Greater Mekong sub-region in South East Asia
(World Health Organisation, 2011a). If this resistance hecomes widesnread. mnlaria
control efforts will be compromised, especially because no current malaria
treatment exists that provides the same levels of tolerance and efficacy as
artemisinin treatments (World Health Organisation, 2011a). Access to ACTs are one
critical aspect of suppressing antimalarial resistance and accurate and consistent
diagnosis of malaria another and is of critical import.
2.2
Malaria Rapid Diagnostic Tests
Malaria rapid diagnostic tests (mRDTs) offer a fast and accurate means of
diagnosing malaria in settings where microscopy is unavailable or unreliable. These
devices are valuable because they require no capital investment or electricity, are
easy to interpret, and require relatively little training. They work by running an
immunochromotographic assay with monoclonal antibodies directed against
14
parasite antigens. The tests require a small amount, approximately 5-15 ptL, of blood
and produce a visual result in 5-20 minutes (Wongsrichanalai, Barcus, Muth,
Sutamihardja, & Wernsdorfer, 2007). The blood travels up the lateral length of the
device and if antigens are present, they bind to the monoclonal antibodies that are
embedded in the kit (Figure 1)
ound
Lysing agent
& labled Ab
Te st e
!bound Abij
ContioO band
1.Anttbody
"o
free ubtesed
wte
bands
visible
non~ly
Nitrocellulose strp
(2 )
#s:
blood
-
flushing Agent
captured b
labelvd Ab
Blood and laeled Ab flushed Akng strip
Captured labeled
Ab Ag comple
(3)
Captured labeed Ab
ZIZIZZEZI
Labeled Ab Ag
comptei
capted by
bound Ab of
test band
LbldA
ctred by
bound Ab of
con"ro barid
Figure 1. Diagram of the mode of action commonly used in mRDTs. 1) Dye labeled and bound unlabeled
antibodies (Ab) specific for the target antigen are present at the lower end of the strip. A test line is
comprised of bound Ab and the control band is formed of either antibody specific for the labeled
antibody or the antigen. 2) Blood and a buffer solution are poured into the well at one end of the strip
and mixed wit the labeled Ab. The liquid travels up the strip. 3) If the antigen is present, the antigen
bound to labeled Ab is bound to the test line. Source: (World Health Organisation, 2006)
Currently, most mRDTs run an assay on several antigens that are either general
across parasite strains or specific to one (Table 1). WHO provides recommendations
on what types of antigens should be used in the mRDT for different areas in the
world, depending on the type of parasite most prevalent there.
15
Table 1. Antigens used in mRDTs
Antigen
Histidine rich protein 2 (HRP-2)
Parasite
P. falciparum
Plasmodium lactate dehydrogenase
(pLDH)
Can be general to all human malaria
parasites or specific
Aldolase
All human malaria parasites
Source: (World Health Organisation, 2011b)
2.2.1
WHO Approval for Malaria Rapid Diagnostic Tests
With a multiplicity of manufacturers of medical devices, it was necessary for some
standardization of devices to ensure governments and non-governmental
organizations (NGOs) could identify high quality products to procure. To that end,
in 2006 the WHO Special Programme for Research and Training in Tropical Diseases
(TNR) and the Foundation for Innovative New Diagnostics (FIND) launched an
initiative to evaluate and compare the performances of commercially available
mRDT devices (World Health Organisation, 2014a). Each year, the WHO publishes
the results of the comparative evaluation.
The criteria that the WHO considers in their evaluation include the panel detection
score (PDS), false-positive rate, invalidation instances, ease of use, thermal stability,
etc. Panel detection score is one of the key measures of performance. Panels of
malaria-positive samples at a high and low concentration are used to evaluate the
devices. The PDS is the percentage of positive results found in a batch of mRDTs
when evaluated against the malaria panel. In this way, PDS evaluates both positivity
rate and batch consistency. PDS further takes into account the mRDTs ability to
provide conclusive results by the manufacturer suggested time (World Health
Organisation, 2014a). PDS is calculated at 2000 parasites/pt and at 200 parasites/
ptL. Many devices score well at the higher concentration but clear differences
emerge at the lower concentration. Low parasite density identification is important
16
as countries reduce the prevalence of malaria and because the association of
symptoms and parasite level varies between individuals (World Health
Organisation, 2014a). Thus, the WHO's evaluation process provides a
comprehensive measurement of the relative efficacy of different mRDTs.
2.3
Malaria Rapid Diagnostic Tests and Healthcare Systems in Uganda
This study took place in the district surrounding the capital of Uganda. This section
explains the background and context surrounding the research, including
government regulation, the private sector, and diagnostic interventions.
2.3.1
Government Regulation
The Ugandan National Drug Authority (NDA) regulates pharmaceuticals and medical
devices, including mRDTs. Further, drug shops and pharmacies must be licensed by
the NDA; pharmacies can sell prescription-only drugs but drug shops can only sell
over-the-counter medication. Antimalarials were designated as over-the-counter in
2008 (ACTwatch Group, 2014). According to one report by the Ugandan Office of the
Auditor General, the length of time the NDA spends reviewing applications results in
much fewer license approvals than should be expected. Additionally, the report
claims that the NDA has not been vigilant about closing down unlicensed drug shops
(Uganda Office Of The Auditor-General, 2010).
2.3.2
Private Sector Retailers
This study focused on the private sector healthcare system because the majority of
Ugandans first seek care from private sector retailers, including private clinics,
pharmacies, and drug shops (Awor, Wamani, Bwire, Jagoe, & Peterson, 2012;
Rutebemberwa, Pariyo, Peterson, Tomson, & Kallander, 2009). Patients have a
variety of reasons for seeking care in the private sector including: the conduct,
qualifications, and experience of the health provider; provider being polite; having
'cheap' treatment; giving treatment on credit; being near; and having access to good
equipment or infrastructure (Rutebemberwa et al., 2009).
17
Unfortunately, many febrile patients are diagnosed with malaria without
confirmation by an mRDT or microscope slide. One study found that 74% of febrile
patients were given antimalarials but only 35% had a positive mRDT result
(Mbonye et al., 2013). Diagnosing malaria by clinical symptoms alone is very
challenging because many malaria symptoms are nonspecific and overlap with
many other indications. Over-diagnosis of malaria is common not only in Uganda
but across sub-Saharan Africa (Amexo, Tolhurst, Barnish, & Bates, 2004; Guerin et
al., 2002; Ndyomugyenyi, Magnussen, & Clarke, 2007).
2.3.3
Interventions to Scale up mRDT Use
Attempts to integrate malaria rapid diagnostic tests in both private and public
facilities have illuminated the challenge and complexity of this task. Many studies
have been done on considering the perspective of patients, as the end user. Mbonye
et al., (2010) found that community concern about introducing mRDTs to drug
shops included: that drug shops would over price the devices, reuse of mRDTs
would lead to infections, and that testing would reveal one's HIV status. Further,
many patients believe they have malaria, regardless of what the diagnosis says
(Chandler et al., 2011). In the paper describing their intervention to introduce
mRDTs in a district in Uganda, (A. K. Mbonye et al., 2014) report that this effort is
complex and requires the careful coordination of people, logistics, protocols, and
resources. Despite the challenges, it has been established that training and
awareness can lower false positive diagnoses in both Uganda and other sub-Saharan
African countries (D'Acremont et al., 2011; Kyabayinze et al., 2010; Williams et al.,
2008).
2.4
Pilot to Scale up use of mRDTs in Uganda
This study was performed in the context of a pilot project that was run by a nonprofit development organization. For the purposes of this paper, the development
organization will be referred to as DevOrg. DevOrg had been awarded a grant to
create a private sector market for quality-assured mRDTs in Uganda by increasing
18
access, generating demand, improving malaria case management, and strengthening
the supply chain systems in the country to ensure robust and reliable supply.
The program, scheduled to be three years, was piloted for the first year in the
Wakiso district in Uganda, the district surrounding the capital, Kampala (Figure 2).
According to preliminary reports of the 2014 Ugandan census, Wakiso has a
population of over two million, the most populous district in the country. The urban
population is about 32% and rural is 68% (Uganda Bureau of Statistics, 2014).
DevOrg has implemented the project in around 180 private sector clinics, drug
shops, and pharmacies distributed across the district.
AAPL
Figure 2 Map of Uganda and districts. Arrow indicates the district of Wakiso, where this study took place.
Source: (Wakiso District Local Government, 2009)
19
During this initial phase, DevOrg procured 1.2 million malaria rapid diagnostic tests.
The mRDTs were produced by two manufacturers that make WHO-approved
diagnostic devices. DevOrg negotiated with the manufacturers not only for the
diagnostic devices, but also for a bundled service that would be an important aspect
of this project. It was recognized that the success of scaling up mRDT sales in
Uganda would require aspects other than the device itself. For that reason, the
contract with manufacturers included provisions about waste management, medical
detailing, training to providers, specific barcoding, and marketing/advertising to
end users.
The in-country supply chain for this project consisted of four layers before the enduser: manufacturer, first-line buyer (FLB), distributor, and retailers (Figure 3). In
the system, there were essentially two parallel supply chains fueled by products
from the manufacturer. Due to the nature of the contract, retailers were divided in
half and distributors/FLBs of one product could not sell to the retailers in the supply
chain of the other manufacturer. Further, only retailers that underwent the training
program were allowed to sell the devices. Each supply chain had one manufacturer
and one FLB, but one of the FLBs employed two distributors.
Manufacturer
1
Manufacturer 2
FLB1
Distributor 3
Retailers I
FLB2
Distributor 2
Distributora
Retailers 2
Figure 3. Supply chain of mRDTs in Uganda in this study
20
Stock moves from manufacturers to retailers. The distributors were the agents who
owned and stored the stock. They purchased it from the first-line buyers and are the
entity that bears the cost of any unsold products. The distributors are also the ones
responsible for handling most aspects of the bundled service, including the
advertising, waste management, and training of providers. Each distributor was
afforded a relatively high degree of freedom in how they went about these
responsibilities, as long as they adhered to DevOrg's requirements.
This study interviewed and held focus group discussions with agents in this project.
Both first-line buyers and all three distributors were interviewed and 28 retailers
were involved in focus group discussions. The study goal was to better understand
how these agents made the decision to be a part of this pilot program, and more
broadly, how they would make a decision about whether or not to sell/stock malaria
rapid diagnostic tests. Multiple criteria decision analysis (MCDA) methodology was
used to investigate this further.
2.5
Multiple Criteria Decision Analysis
This study aimed to understand the preferences and tradeoffs that were inherent in
the decision-making process of agents in the private sector supply chain of mRDTs
in Uganda. This was a complicated endeavor for several reasons. First, there is a
plurality of stakeholders involved in the decisions, from the manufacturers to the
retailers. Each agent can make their own decision, but their respective decisions
influence and shape other agents. Further, there is a collaborative component; the
devices cannot make it to the end user without some sort of agreement between the
agents. Finally, each agent was faced with a multiplicity of criteria to be considered,
with tradeoffs between those criteria that needed to be considered before reaching
a decision. For these reasons, it was determined that multiple criteria decision
analysis (Keeney & Raiffa, 1976) would be an appropriate methodology to employ.
Multiple criteria decision analysis (MCDA) is a methodology that draws from
decision sciences, operations research, and economics. It is most appropriate for
21
complex decision problems where there are multiple objectives and no clear
optimal solution.
2.5.1
History and Motivation of Decision Analysis
It is proposed that the earliest known reference to MCDA (albeit not in so many
words) was by Benjamin Franklin, in a letter describing a method that he used to
make important decisions (Koksalan, Wallenius, & Zionts, 2011). The process
involved making a list of arguments on both sides and crossing out the arguments
that were relatively of equal importance, which was essentially a weighting process.
Key groundwork for the methodology was laid through the 20th century by
mathematicians and economists, including work on welfare economics, set theory,
and utility theory (Figueira, Greco, & Ehrogott, 2005). Then, the 1970s saw many
important contributions to theory and methods of MCDA and the 1980s an
extensive growth of application of theory in real-life situations (Koksalan et al.,
2011).
Decision analysis was described by Keeney as "a formalization of common sense for
decision problems which are too complicated for informal use of common sense" (p.
806, Keeney, 1982). The need to develop a rigorous methodology to approach
decisions was realized in the reality of decisions faced in modern society. Many
decisions are complex, involve multiple stakeholders, and require trade-offs
between conflicting objectives. For example, the classic trade off between cost and
quality. Often (not always) higher quality requires a greater cost. If one's objectives
are to minimize cost and maximize quality, what is the optimal solution?
Another example is a patient's consideration of what prostate cancer treatment to
receive (Keller & Wang, 2009). Options could include surgery, chemotherapy,
experimental drugs, or waiting. There are multiple objectives for the patient,
including maximizing chances of success, minimizing harmful side effects, and
minimizing life and career disruptions. The decision is further complicated by the
fact that another stakeholder has somewhat different (although very overlapping)
22
objectives. The doctor treating the patient may have objectives that include
minimizing costs, in addition to maximizing chances of recovery and minimizing
side effects. It is possible that there are trade-offs between the decision options. For
example, consider a treatment that has a higher chance of success in reducing tumor
size, but comes with more unpleasant side effects.
Features of complex decisions include: multiple objectives, difficulty identifying
good alternatives, intangibles, long time-horizons, many impacted groups, risk and
uncertainty, risks to life and limb, interdisciplinary substance, several decision
makers, value tradeoffs, risk attitude, and sequential nature of decisions (Keeney,
1982). These intertwined characteristics provide the motivation and impetus for
MCDA and decision science in general.
2.5.2
Applications of MCDA
MCDA has been applied in public health interventions, especially in priority setting
&
(Baltussen & Niessen, 2006; Marsh, Dolan, Kempster, & Lugon, 2013; Nutt, King,
Phillips, 2010), finance (Zopounidis & Doumpos, 2002), natural resource
&
management (Mendoza & Martins, 2006), sustainable energy (Wang, Jing, Zhang,
Zhao, 2009), and more. MCDA has particular potential in the developing world
context, where there are often multiple stakeholders and objectives, and complex
decisions to be made (Stewart, Joubert, & Janssen, 2010). This is especially the case
for NGOs and non-profits seeking to implement interventions. For example, if one
wanted to procure solar lanterns and was deciding between options, it would be
necessary to trade off between different criteria like recharge time, brightness,
battery life, durability, price, cultural appropriateness, reliability of manufacturer,
and more (Sanyal et al., 2015). It is often the case that there is no obvious "winner."
MCDA can help provide a framework for making complex decisions like this.
2.6
Aim of this Research Study
This study implemented MCDA in the context of scaling up rapid diagnostic tests for
malaria. The goal of the analysis was to better understand a supply chain agents'
23
"willingness to stock," that is, to carry the product. The previously mentioned
studies have evaluated willingness to use mRDT products, but there is limited
literature on examining this other aspect. Further, MCDA is valuable in providing
insight into the preferences and priorities of different private sector supply chain
stakeholders, which can inform and shape public health interventions.
This research study was performed on a small scale, with limited time and
resources. One researcher traveled to Uganda and interviewed 2 first line buyers, 3
distributors, and 28 retailers as a representative sample of the population of
individuals in DevOrg's project. The purpose of this research study was two-fold.
First, to gather data and derive quantitative results about the decision making
process for the agents in the supply chain when deciding to stock the devices.
Second, another important result of this research was a proof of concept and pilot
implementation of MCDA in the context of the developing world to rigorously
analyze a complex and multifaceted situation. The rest of this paper describes the
methodology and results and then offers a discussion on insights and suggestions
for further research.
3
Vlethodology
The methods used in this study were adapted from multi criteria decision analysis
(MCDA) methodology. This section describes the methodology used, adaptations,
and limitations. The data collection process will be described first and then the
specific sections of the analysis will be explained.
3.1
Research Scope
This research study was conducted on a very small scale. One researcher,
accompanied by a research assistant, spent two weeks in Uganda collecting data
that was analyzed in about a month. The purpose was to test a multi-criteria
decision analysis methodology in international development contexts where time
and data are often in short supply. It is common in situations like these to make
24
assumptions when necessary and adapt to missing information. This thesis serves to
explore a modified multi-criteria decision analysis methodology as an approach to
rapid evaluations in the developing world context. This thesis also highlights what
future work could do to refine the initial pilot.
3.2
Data Collection
Two mechanisms were used to gather data from respondents: semi-structured
interviews and focus groups with a survey component. Semi-structured interviews
were performed by the researcher on first-line buyers and distributors while the
focus groups were done with retailers.
3.2.1
Semi-Structured Interviews with First-Line Buyers and Distributors
A total of five semi-structured interviews with first-line buyers and distributors
were performed. All the individuals in the research subset were participants in
DevOrg's project to scale up mRDT use. All but one of the interviews was performed
in-person in Uganda and one was on Skype because the distributor was out of town
while the data was being collected. Follow up questions were addressed to first line
buyers through email or Skype. Interviews consisted of an introduction of the
researchers, a brief background on why this information was being collected and
why it is important, and a question of consent. After the interviewees had
consented, the initial round of questions were general, about their background,
business, and general thoughts about working on this project. After this
introductory phase, the interviewer would explain the nature of the study and that
questions had been prepared for their consideration. An important aspect of this
interview was to explain to the respondents that the questions were academic in
nature and therefore may be abstract. The interviewees were assured that it may be
confusing and they should not hesitate at all to ask for clarification. The goal of this
was to ensure that respondents were actually understanding the question and
providing accurate information that reflected their true opinions and thoughts.
Additionally, to facilitate understanding, a variety of tangible examples were
provided.
25
Throughout the interviews, it was necessary for the interviewer to act as an anchor,
returning to the set of questions, to avoid prolonged divergence of topics. While
some divergence was integral to the research, as it provided information about
questions that the researchers didn't even realize were important, it was necessary
to balance that with staying on task to gather the information necessary to
appropriately conduct the methodology.
It was critical for the interviewer to be able to synthesize and parse interviewee
responses. For example, when asked a question about what factors are important
when deciding whether to stock malaria RDTs, a respondent may reply with a long
narrative on their experience and thoughts. It is necessary for the researcher
identify criteria that can then be subsequently evaluated. This was done by the
researcher conducting this study and then articulated to the interviewees. Then, the
interviewees were asked to confirm if the summary was accurate.
3.2.2
Focus Group Discussions with Retailers
Focus group discussions were used as a means of gathering information from the
retailers. A group discussion in multi-criteria decision analysis methodology has the
benefit of allowing participants to discuss and debate criteria. Further, this study
had too short of a time frame to feasibly conduct a large number of surveys. Finally,
in a focus group discussion, it was possible to get the reasonably undivided
attention of a group of retailers for a block of time, as opposed to a survey, which
had the challenge of trying to solicit information from an individual when they are in
the middle of their work.
However, a traditional focus group was not the ideal setting because the
respondents were a mix of genders, business levels, experience, and retail types.
This diversity led to the situation where some individuals were much more vocal
than others. Even with specific encouraging, the majority of talking was performed
by 2-4 dominant men. The women in the group were much more passive. In order to
26
ensure input from all parties, the focus group included a survey component. In order
to rank their "happiness" for different attribute values and the overall weighting for
attributes, respondents were asked to fill out a piece of paper that was handed out.
The sheet of paper was not a form; it was a blank piece of paper where the
researchers explained how to fill out the form by writing up an example at the front
of the room on a large piece of paper. In the future, it is recommended that a form be
made up ahead of time to pass out to respondents and reduce confusion on what to
fill out.
The researchers led the focus group discussions but a Ugandan moderator assisted
that effort. This individual was introduced by partner organization DevOrg and was
recommended as important to the success of the discussion group. This individual
was briefed on the project to provide necessary background to effectively facilitate
the focus group discussion. One recommendation for further work would be to work
closely with a facilitator much in advance and run through the entire study with
them to ensure understanding. One value that the facilitator did provide was
clarification in the local language. Although all of the respondents spoke English
very well, sometimes it was helpful to explain in their native tongue.
3.3
Methodology Framework
The study was broken down into seven parts:
1. Decision framing
2. Identifying criteria
3. Eliciting value functions
4. Determining priorities (weighting)
5. Generation of decision matrix inputs
6. Evaluating options
7. Sensitivity analysis
27
Each of the six steps was performed for three levels of supply chain agents: first-line
buyers, distributors, and retailers. Decision framing was performed to determine
the set of options that were to be evaluated. It was done in advance of data
collection, through expert opinion in consultation with DevOrg. The options
considered were the same for all levels of the supply chain. Identifying criteria
elucidated the considerations by which a decision maker evaluated his/her options.
Value functions determined the marginal added value given an increase in
performance on a criterion. Determining priorities was performed to understand
the relative importance, or weight, of the criteria identified. A decision matrix was
generated with estimates of inputs for each criterion across the options. Evaluating
the options was then performed by applying the weighted average constructed from
the value functions and weights to integrate all factors and understand the relative
merit of the options. Sensitivity analysis was performed to understand the
robustness of the data collected. This section will first describe the data collection
methodology in terms of logistics (questionnaires, survey execution, etc.) Then, the
rest of this section will expand on how the seven aforementioned steps were
implemented.
3.3.1
Decision Framing
Decision framing was performed before interviews and focus group discussions
with stakeholders, in consultation with DevOrg representatives who could provide
expert opinions. The overarching goal of this study was to understand how agents
decide to stock and sell malaria rapid diagnostic tests (mRDTs). This also included
how the stakeholders consider intervention alternatives provided by NGOs, like
DevOrg, and also the consideration of selling WHO versus non-WHO approved
devices. The flow chart of decision options is described in Figure 4.
28
Sell?
No
Yes
Sell WHO approved?
No
+
Yes
Sell in bundled service?
No
Yes
Figure 4. Decision flow chart for agents in the supply chain. The stars represent a decision option.
A comprehensive set of decision options was put together. The options under
consideration were:
1. Sell WHO approved mRDTs outside of DevOrg's bundled service
2. Sell mRDTs through DevOrg's bundled service
3. Sell non-WHO approved mRDTs
4. Do not sell mRDTs
At the time when first-line buyers (FLBs), distributors, and retailers were signing on
to DevOrg's project, these are the exhaustive set of options available to them. Selling
non-WHO approved devices through the bundled service was not an option because
the contract required agents to only sell the devices provided through selected
manufacturers, which were all WHO approved. All three levels in the supply chain,
first-line buyers, distributors, and retailers faced this consideration. Manufacturers
only had two options: to participate or not to participate in the bundled service, as
they obviously already had the product they were making. Manufacturers were not
included in this analysis due to time constraints but further research should
consider including them in this type of analysis because many of their decisions are
29
complex and an MCDA methodology could be valuable. The rest of the study was
centered on the four decisions and the options were presented in each interview
and focus group discussion.
3.3.2
Identifying Criteria
Criteria are defined as the considerations an individual makes when deciding on an
action. In MCDA literature, the term attribute is often frequently used; both words
are used interchangeably in this paper. Ultimately, the criteria are derived from
one's fundamental objectives. Fundamental objectives are the goals one strives to
meet or achieve when making the decision. For example, in public health, the
fundamental objectives may include minimizing cost, minimizing mortality,
maximizing a health outcome, providing assistance to those most underserved, etc.
&
Criteria should be unambiguous, direct, operational, and understandable (Keeney
Gregory, 2005). Unambiguous means that a clear relationship exists between the
criteria and a consequence. Direct means that the criteria levels directly describe
the consequence of interest. Operational refers to information that can be obtained
to reasonably describe trade-offs and consequences. Understandable means that the
criteria and impact of criteria can be easily understood and communicated.
Furthermore, criteria should be non-redundant, otherwise some may receive
disproportional weight (G. Montibeller &Franco, 2007). For example, maximize
profit and maximize revenue should not be two criteria evaluated together because
profit takes revenue into account. An alternative would be to have maximize
revenue and minimize cost a$ two criteria. Criteria can be measured by three types
of metrics: quantitative direct, quantitative indirect, and qualitative (G. Montibeller
& Franco, 2007). Quantitative direct attributes directly measure the criteria in
question; examples include a company's profits or market share. Quantitative
indirect attributes serve as proxies for what one is trying to measure; for example, a
Richter scale for expressing the severity of an earthquake or pregnancy rates for
expressing unprotected sex. Qualitative attributes measure a set of levels-like high,
30
medium, and low-for criteria. One example includes creating a positive public
image. Both quantitative and qualitative attributes were used in this study.
3.3.2.1
Retailers
In this study, the respondents identified the criteria. The researchers had been
informed by literature reviews and expert advice and were prepared to offer
suggestions if respondents were reticent. This was balanced with the care to not
direct the direction of the conversation. For the focus groups, the whole group was
asked to think about what are the factors they took into consideration when
deciding whether or not to sell malaria rapid diagnostic tests and whether to enter
into this bundled service. Concrete examples that were relevant to their experiences
were given to ensure that the question was understood. For example, one scenario
described was to think about how one makes a decision about what shirt to buy.
When deciding between a few options, there are several important criteria that
come to mind (Figure 5).
Criteria
Cost
Brand
Decision: Choose the best shirt
Description
Maximize quality: What is the quality of the shirt? Does it look
like it is going to last a long time?
Minimize cost: How much does the shirt cost?
Compliment colors: I like colors that are darker because they
look better on me
Maximize flattering fit: I want a shirt that will fit well and look
good on me
Recognizable brand: I want the brand recognition
Figure 5. Example given to respondents to elicit criteria
Concrete examples like this one were given to ensure that the respondents
understood the type of question that they were being asked. Individuals raised their
hands and offered ideas. Often it was necessary for the researcher to clarify what
was said and summarize into a criterion. Always the moderator confirmed with the
original speaker to ensure that the summary was correct. After the group had
brainstormed for several minutes, the moderator offered some of the criteria that
31
had been generated before the discussions by expert opinion. Respondents
generally agreed with those criteria also.
The criteria to be included in the analysis were generated by the first focus group
discussion. It was a lengthy process, so in order to keep the attention of the group it
was important to minimize the length of the discussions. Thus, for the second focus
group, the list of criteria that had previously elicited was presented to the group.
The respondents were first asked if that list was comprehensive or if there was
anything they would like to change. The group did add two criteria: time to
complete a sale and opportunities. They removed one criterion: quality. Table 2 lists
the criteria for retailers and the definition. The researchers did remove the attribute
quality from the analysis but an improvement in the methodology would be to
include criteria that may have been dismissed by some of the group. It is possible
that there were individuals in the focus group discussion that disagreed with quality
being removed but were reluctant to speak up in the group setting. In the future,
attributes that were less important to some retailers would be reflected by the
weighting procedure.
32
Table 2. List of Criteriaand Definitions for Retailers
Criteria
Definition
Quality
The quality of the device as measured by its WHOapproval status, durability, and accuracy
Price device is sold
Price at which the device is sold to customers
Cost of device
The amount that the device costs the retailer to
purchase
Delivery time
Number of days after an order is placed that a
distributor delivers new supply
Sales of other products
Increase in the sales of other products
Customer satisfaction
How happy a customer is after a business
interaction
Training
The amount of training that the retailer receives on
how the device works, why it is important, and how
to administer it
Volume sold
Number of devices sold per month
Awareness/advertising
Amount of advertising and awareness that is
generated in consumers
Time to complete sale*
Amount of time it takes to complete a business
transaction
Opportunities*
Other opportunities, including free apparel, small
jobs, and leadership
*Criteria added by the second focus group discussion
After the criteria were established and defined, metrics were determined to
evaluate the criteria (Table 3). This included both qualitative and quantitative
metrics. The metrics directly measured the criteria defined in Table 2. Qualitative
metrics were evaluated on a high to low scale, with each interval defined. To add
more granularity, five intervals were defined on the qualitative scale: high, medium-
33
high, medium, low-medium, and low. For any attribute that had currency, Ugandan
shillings were used in discussions with respondents and then converted to U.S.
dollars (USD$) with the exchange rate of $1 USD equal to 3,000 Ugandan shillings.
Table 3. Criteria and metric units
Criteria
Metric
Quality
Qualitative, high-low
Price device is sold
Ugandan Shillings
Delivery time
Days
Sales of Other Products
Qualitative, high-low
Customer Satisfaction
Qualitative, high-low
Training
Qualitative, high-low
Volume sold
Number of kits sold per month
Awareness/ads
Qualitative, high-low
Time to complete sale
Minutes
Opportunities
Qualitative, high-low
Cost of device
Ugandan Shillings
34
The different levels were defined and explained to retailers before soliciting
responses (Table 4).
Table 4. Qualitative Metric Definitions for Retailers
High
Medium-High
Medium
Low-
Low
Medium
Quality
WHOapproved,
very durable,
very accurate
WHO-approved,
durable,
Accurate
Not WHOapproved, but
somewhat
accurate, and
somewhat
Not-WHO
approved,
not very
accurate,
not very
durable
durable
Sales increase
somewhat
Sales
increase a
little
Not-WHO
approved,
not
accurate,
not durable
Sales of other
products
Sales are
increased by a
lot
Sales are
increased by a
good amount
Customer
Satisfaction
Customers are
very happy
Customers are
somewhat
happy
Customers are
neutral
Customers
are
somewhat
unhappy
Customers
are
unhappy
Extensive,
Pretty good and
comprehensive
training
Some training,
not
comprehensive
A little
Very little
or no
training
Good
advertising,
thorough or in
multiple media
sources
Some
advertising in
one media
source
Little or
less
Very little
or no
effective
advertising
advertising
Good
opportunities in
both frequency
and value
Some
opportunities
Little or
less
valuable
opportuniti
Very little
or nothing
Sales
increase
very little
or not at all
Training
comprehensiv
e training on
all aspects
relating to
training,
missing
detail
mRDT
Awareness/
Advertising
Extensive
advertising in
multiple
media sources
(radio, TV,
etc.)
Opportunities
Many
opportunities
that are
valuable
es
Part of defining criteria included articulating the ranges, the upper and lower bound,
over which the decisions are made. This is very important because otherwise value
trade-off is distorted (G. Montibeller & Franco, 2007). For example, in the shirt case,
the price range of the shirts being considered may be from $2-$10 dollars. Thus,
35
when considering this criterion, it is important to frame the decision in terms of
these specific ranges. Ranges were elicited by the first group and then used again on
the second group (Table 5).
Table 5. Criteria and ranges for retailers
Criteria
Low Value
High Value
Quality
Low
High
Price ($USD)
0
$1.67 USD
Time to delivery (days)
7
1
Sales of other products
Low
High
Customer Satisfaction
Low
High
Training
Low
High
0
50
Low
High
0
60
Low
High
0
$1
Volume (kits sold per month)
Awareness/ads
Time to complete a sale (minutes)
Other Opportunities
Cost
3.3.2.2 Distributors
Data were collected from distributors through individual interviews. Although the
education of these individuals was greater, the type of thinking required for MCDA
is abstract and can be confusing the first time one sees it. Thus, tangible examples
were helpful in clarifying questions asked. Examples like choosing a shirt was
provided to these individuals as well. Since the level of education of these
individuals was greater, it was possible to provide a general explanation of value
and preferences. Thus these interviewees could confirm a relative increasing or
decreasing value over the range of an attribute.
As described previously, it was challenging to keep the individuals focused on the
task at hand. Many of them came into meetings with preconceived notions about
36
what the interview was about and it was necessary to explain multiple times the
goal of the research. In this case, criteria generated in advance of meetings by expert
opinion was less helpful because respondents were much more varied in the criteria
that was important to them than the retailers were. Further, it was necessary to
work around individuals' intuition about how they think through business
decisions. For example, in multiple interviews, the interviewee wanted to talk about
margin, instead of breaking it down into segmented costs, volume, and price. This
was likely because in their everyday interactions, they have a natural sense about
and talk in terms of margins. This was the case not only for distributors but also
FLBs. Furthermore, there was a challenge in unifying the responses that the various
distributors and FLBs provided. They thought in terms of different units; for
example, for volume, some individuals explained their criteria in terms of devices
sold per month, or devices sold by next year, or devices sold up until this point. If
necessary, assumptions and interpretations were made to convert the units to be
the same for responses across different agents (see Inputs, Assumptions, and
Analysis section). Distributor and FLB responses were not aggregated.
Table 6. List of Criteria and Definitions for Distributors
Criteria
Definition
Volume
Kits sold per month by the distributor
Expiration date
Months left to expiry when the kit is sold
Profit
Percent profit
Cost per kit
Cost to the distributor to purchase the kit
Cost per training
Cost for the distributor to hold one training session
Efficiency of distribution
Boxes delivered per delivery trip
Cross selling
Percent increase in sales
Table 6 lists the criteria identified as important to the distributors. Of the three
distributors interviewed, one was unable to provide responses because of a lack of
understanding and willingness to participate. Thus, that interview was recorded as
37
a non-response and data from the two other distributors was collected. Like
retailers, the researcher elicited upper and lower bounds of the criteria from the
distributor. The distributors had the same ranges for all the criteria except for two.
For cost per training, the upper bound was either $500 or $100 and for cross selling
the upper bound was either 50% or 20% (Table 7).
Table 7. Criteria and ranges for distributors
Upper Bound
Lower Bound
18,750
0
22
3
Profit margin (%)
50%
0%
Cost per kit ($USD)
$0.83
$0
Cost per training ($USD)
$2,000
$0
Efficiency of distribution (Boxes per delivery)
500 (or 100)
1
Cross selling (% increase in sales)
50% (or 20%)
0%
Criteria
Volume (kits sold per month)
Expiration date (months left to expiry at sale)
3.3.2.3
First Line Buyers
Responses from FLBs were solicited in the same manner as distributors. In the case
of missing data, the individuals were followed up with online through email and/or
Skype. The same issues that arose in distributor issues also occurred when speaking
with the FLBs, especially the difficult time differentiating preferentially independent
options (see Discussion section). The FLBs had the same criteria except one FLB had
percent increase in sales, which the first did not mention. Table 8 summarizes the
criteria and definitions used for the FLBs. Table 9 summarizes the criteria and
ranges and Table 10 explains the definitions used for the qualitative metrics.
38
Table 8. List of Criteriaand Definitions for Distributors
Criteria
Definition
Cost per kit
Cost that the FLBs pay to purchase the kits
Price per kit
Price at which the FLB sells the kit to the distributor
Quality
Quality of the kit in terms of WHO-approval status,
accuracy, and durability
Administrative time
Hours per week of administrative time that goes into
managing the sale of kits
Profit
Percent profit
Relationship with donor
Relationship with any donor organization or NGO
organization/funder
Increase in sales*
Percent increase in sales
*Only for one of the FLBs
Table 9. Criteria and ranges for FLBs
Criteria
Upper Bound
Lower Bound
Cost per kit
$0.033
1
Price per kit
$0
$1
Quality
H
L
Administrative time
1 hr.
60 hr.
Profit
0%
50% (or 100%)
H
L
0%
50%
Relationship with donor organization/funder
Increase in sales
39
Table 10. Qualitative Metric Definitions for FLBs
Quality
High
Medium-High
Medium
Low-Medium
Low
WHO-
WHO-
Not WHO-
Not-WHO
Not-WHO
approved, very
durable, very
accurate
approved,
durable,
Accurate
approved, but
somewhat
accurate, and
somewhat
approved, not
very accurate,
not very
approved,
not
accurate,
durable
not durable
No
relationship
durable
Relationship
with donor
Very strong
relationship,
collaborate
and
communicate
extensively
Strong
Somewhat
relationship,
strong
Collaborate
somewhat or
collaborate on
some projects
relationship,
may have
have
communicated
collaborated
on one
project
3.3.3
Eliciting value functions
An important part of this analysis was evaluating how an individual's marginal
added value changed given an increase in performance on a criterion. It was not
assumed that two incremental changes would result in the same net value change;
rather it is possible that value functions are non-linear. Value functions plot value as
the dependent variable and the criterion as the independent variable and the
resulting shape of the function allows one to understand how the rate of change
across the criteria compares with the rate of change in value. In doing this, one can
understand over what ranges an incremental increase or decrease is most
important.
There are two traditional means of eliciting value functions: bisection and direct
rating (G. Montibeller & Franco, 2007). Bisection works by asking participants to
identify the midpoint between two anchoring values of a criterion. Then, one can
determine whether the increase from the low to the mid value is more valued versus
the mid to highest value. Direct rating requires respondents to apply a score to
various points of the criteria. In this study, the primary method employed was direct
weighting. Respondents, both retailers and FLBs/distributors, were asked to
40
provide a numerical score for value for several points. The upper and lower bounds
on the criteria were assigned the highest and lowest (depending on whether the
objective was to maximize or minimize that particular criteria) values. The value of
certain criteria points between the boundaries was then solicited. Piecewise linear
interpolation was used to create a continuous function between the criteria and the
value.
3.3.3.1 Retailers
The bisection method was tried at first, but the retailers did not understand what
was being asked of them. Instead, the moderator asked the retailers to rank their
"happiness" on a scale of 1 (very unhappy) to 5 (very happy) for specific points of
the criteria (Figure 6). As described in the "Soliciting Responses" section, this
information was gathered at the individual level for each respondent on a piece of
paper. For example, the moderator asked the group how happy they would be if
their store sold 10 malaria rapid diagnostic tests per month. The respondents were
instructed to report a value of 1 to 5 (very unhappy to very happy). For each
criterion, two points to be evaluated were given to the retailers. When the range
was defined, the moderator indicated that the lowest range would score a 1 and the
highest would be a 5. All other values could be within that range. The median score
of all retailers per criterion was calculated and this figure was used to construct a
value function. The median was chosen to reduce the effects of outliers. A total of
four data points (the lowest and highest, and two in the middle) were plotted on a
graph of the range of the criteria versus value. Linear piecewise interpolation was
conducted to create a continuous function.
Numerical Score
5
4
3
2
1
Description
Very happy
Somewhat happy
Neutral
Somewhat unhappy
Very unhappy
Figure 6. Numerical and verbal descriptions of happiness to elicit value functions
41
3.3.3.2 Distributorsand First-Line Buyers
The direct weighting method was the primary method used to elicit values from
distributors and FLBs. The researcher first defined a 1 and a 5 on the value scale,
corresponding to the highest and lowest amount of the criteria specified in the
identifying criteria section, and explained in terms of "satisfaction," instead of
"happiness," which was more appropriate for distributors. The researcher would
ask the individual to rank their "satisfaction or happiness" on the 1-5 scale for
different levels in the criteria. Sometimes the distributor offered information for
another point and the researcher recorded this information also. Two value
functions were constructed, one for each distributor.
One complication with the interviews was that there was a variety of ways that the
respondents thought about the answers to the questions asked them. For example,
when asked about the volume of sales, some answered in terms of volume of mRDTs
sold up until this point, others in terms of the number of mRDTs sold in two years,
and still others explained volume in terms of mRDT sales per month. To compare
responses, the values were converted to the same unit. Furthermore, it was
necessary to draw a distinction between criteria that had been determined in
advance of the intpruiePAs5, bythe researchers through expert intprviepAws and
literature, and emergent criteria that the respondents brought up over the course of
the conversation. As much as possible the emergent criteria were incorporated into
the methodology, however, it was not always possible to define ranges and metrics
in the short amount of time that was allocated for interviews. In this case, the
respondents' thoughts were recorded and will be considered qualitatively (see
Discussion section).
3.3.4
Determining Priorities (Weighting)
Determining the relative importance of the attributes and the trade-offs between
them is critically important in a multi criteria decision framework. In this study,
trade-off analysis was performed as described in Montibeller & Franco (2007) using
the swing weights method. Respondents were asked to think about their most
42
important considerations when making the decision on whether or not to stock
mRDTs and whether to participate in the bundled service. Respondents (both
retailers and distributors) were asked to consider a decision option that ranked the
lowest in all of the attributes previously defined. The moderator listed a few
examples, including selling zero devices, having the highest cost, etc. Respondents
were then asked to identify the most critical attribute they would need to improve
first. This attribute was given a value of 100. Then, respondents were asked to
consider subsequent (second, third, etc.) most critical criteria to improve and assign
them a relative weight ranging from 0, not essential at all, to 100, the most essential.
As was done previously, examples were given to the respondents to illustrate the
concept. Both new examples and examples previously described were provided. It
was helpful to have a few examples that carried through the different sections of the
study. For example, after having used the shirt example to solicit criteria and elicit
value functions, it was also used to define weights. The researcher made a point
emphasize that there was no "right" answer and that someone could give a value of
90 to a criteria that someone else gave a 50 and that was perfectly fine.
An alternative method to solicit weights was considered where participants have a
set number of points and they would be free to allocate the points across the
criteria. However, the first attempt at this, in an interview with a first-line buyer,
was not productive. The individual was quite confused about the concept. When the
alternative method of assigning each criteria a score from 0-100 was proposed they
seemed much more comfortable. For consistency, this method was used for
subsequent interviews; however, later respondents did show familiarity with the
fixed-number of points method alternative. Thus, it may be worth considering this
method, and taking the time to explain it fully, in the future.
One reoccurring issue in the assigning of weights was that respondents very rarely
ranked anything below 50. One retailer had a spread of 100-98 for weights, which
provides virtually no granularity. Even though the examples provided assigned
43
weights from 10-100, very few individuals emulated that sort of spread. See
Discussion section for speculation on biases that may have prompted these results.
For the retailers, the weights were aggregated by calculating the median of the
responses. The distributor and FLB weights were not aggregated. The weights were
normalized over the sum such that the weights added to 100%.
3.3.5
Generation of decision matrix inputs
A decision matrix was constructed consisting of inputs for each of the options across
the criteria. These inputs were meant to be as reflective of reality as possible. For
example, the cost to retailers was estimated across the four decision options. Option
2 (selling mRDTs through the bundled service) was established; the cost of the
devices for the other three options then had to be estimated. The purpose of
constructing the decision matrix is then to input those figures into the value
functions and thus convert to one unit (value) for each of the criteria. Then, a
weighted sum can be calculated based on the weights. A decision matrix was
constructed for each agent in the supply chain.
Ideally, the decision maker would be involved in generating the inputs for the
decision matrix. However, due to time constraints, it was not possible to solicit this
information from the respondents. Rather, for each input, a combination of
literature sources, expert opinions from MIT faculty and DevOrg representatives,
and information from interviews was used to estimate inputs. To account for the
uncertainty in these assumptions a high and low estimate was also recorded. See
Inputs, Assumptions, and Analysis section for information behind the assumptions
for each estimate.
3.3.6
Evaluating options
An overall value was calculated for each option that was being considered. The
numbers in the decision matrix were used as inputs to the value functions. Since one
continuous value function was plotted, it was possible to calculate a corresponding
value for each discrete criterion input using the equation of the line where value is
44
the dependent variable and the criterion is the independent variable. Each section of
the function was assumed to be linear. For example, if cost was $0.55 per kit, the
equation of the line connecting the two solicited points was used to calculate the
value when cost was $0.55.
Each value was multiplied by its corresponding weight. Thus, a weighted sum can be
generated for each option. The overall value can be compared across the options;
theoretically the option with the highest value is reflective of the best choice for that
individual, taking into account preferences and priorities.
3.3.7
Sensitivity Analysis
High- and low-end estimates were also generated, in addition to the base estimate,
for each of the criteria and options in the decision matrix. These values were
inputted into the weighted sum to evaluate the robustness of the results. Especially
because there is a fairly high level of uncertainty in the input assumptions, due to
limited data from agents and conflicting information, it was important to compare a
range of assumptions.
Further, a sensitivity analysis was performed on the weights. The weights for each
attribute were varied from 0-100% and the impact on the overall value was
determined. That is, the overall value of each of the four options was plotted as a
function of the variation of weight. This analysis was done by plotting two values
and then drawing a line through the two points (Montibeller & Franco, 2007). One
value was the weight that was elicited through the data collection process, with its
corresponding calculated overall value. The second value was set as if the weight for
the attribute was 100%. The overall value, therefore, would be the criterion's value
that was determined from the value function. A straight line can be drawn through
these two points to represent the variation of weights. The analysis considers when
the options change positions, when the best choice is swapped with another option.
45
4
Inputs and Assumptions
The methodology used required certain input assumptions to be made in order to
run an analysis and examine the best options for the different agents. Especially in
the context of the developing world, there is limited and unreliable data. Thus, it
was necessary to sometimes make estimations. This section describes the numbers
used and the assumptions and rationale behind them.
4.1
Decision Input Matrix
The decision input matrix is the summary of the numerical values for each of the
options for each of the criteria identified. These figures will be inputted into the
value functions to determine the corresponding value for use in the weighted sum.
The inputs for the decision matrix were derived through a combination of expert
interviews, literature, and observation while in Uganda. Further, a high and low
estimate was also generated for analysis to examine the sensitivity of the overall
value to the uncertainty in the input values. This section provides detail about the
assumptions behind each input.
4.1.1
Decision Input Matrix - Retailers
The comprehensive list of inputs for the retailer decision matrix is summarized in
Table 11. Some criteria were not relevant for Option 4; those were marked as N/A
and explained in the corresponding section.
46
Table 11. Input figures for retailers for high, base, and low estimates for options 1-4
Quality
Option 1
Option 2
Option 3
Option 4
Price ($USD)
Option 1
Option 2
Option 3
Option 4
Delivery Time (days)
Option 1
Option 2
Option 3
Option 4
Sales of other products
Option 1
Option 2
Option 3
Option 4
Customer Satisfaction
Option 1
Option 2
Option 3
Option 4
Training
Option 1
Option 2
Option 3
Option 4
Volume (kits sold per month)
Option 1
Option 2
Option 3
Option 4
Awareness/Advertising
Option 1
Option 2
Option 3
Option 4
Cost (cost per kit, $USD)
Option 1
Option 2
Option 3
Option 4
Time to complete sale (minutes)
Option 1
Option 2
Option 3
Option 4
Opportunities
High Estimate
Base Estimate
Low Estimate
H
H
LM
N/A
MH
H
L
N/A
MH
MH
L
N/A
$1.3
$1.3
$1.2
$0
$0.75
$0.83
$0.88
$0
$0.60
$0.60
$0.50
$0
2
1
3
N/A
3
2
4
N/A
4
3
5
N/A
MH
H
M
L
M
MH
LM
L
LM
M
L
L
MH
H
M
L
M
MH
LM
L
LM
M
L
L
M
H
LM
L
LM
MH
L
L
L
M
L
L
20
40
20
0
10
20
10
0
5
10
5
0
M
H
LM
L
LM
MH
L
L
L
M
L
L
$0.61
$0.61
$0.56
$0
$0.35
$0.39
$0.41
$0
$0.28
$0.28
$0.24
$0
30
40
30
0
20
30
20
0
10
15
10
0
47
Option
Option
Option
Option
4.1.1.1
1
2
3
4
M
H
LM
L
LM
MH
L
L
L
M
L
L
Quality
When explaining quality to the respondents, it was explained that quality depends
on WHO-approval, accuracy, and durability of the device. In the base assumption,
devices in Option 2 were assigned as high because they were WHO approved and
further, vetted by DevOrg. Option 1 was assigned a value of medium-high because it
is assumed that this option would carry a variety of devices ranging from mediumhigh to high and there is no expertise to differentiate the two. Option 3 was assigned
as low because the devices would not be WHO-approved. Option 4 was eliminated
from the analysis, because if there were no devices being sold, quality is not
relevant. The high and low-end assumptions decreased Option 2 to being MH and
increased Option 3 to LM.
4.1.1.2 Price
Price was established from the ACTWatch Outlet survey (ACTwatch Group, Program
for Acessible Health, Communication & Education (PACE), & The Independent
Evaluation (IE) Team, 2012). The ACTWatch survey provides data on the average
price of mRDTs in clinics and drug stores that sell both WHO and non WHOapproved devices. A weighted average was taken of these two prices based on the
number of clinics and drug shops in the survey sample. The two pharmacies were
counted as drug shops, as ACTWatch had no data on pharmacy pricing and the one
unknown identity was removed. Option 2 was determined from interviews with
DevOrg. Option 4 was set as $0 because if the retailer is not selling the device, the
price is zero. The high estimates were increased so that they were closer to the highend of the ranges in the value functions and the low-end was decreased by $0.100.20.
48
4.1.1.3
Delivery Time
In the focus group discussions, retailers expressed delivery times ranging from 1 to
3 days, so 2 days was set as a base assumption for Option 2. Option 1 was assumed
to have a longer delivery time because there is not the established supply chain that
Option 2 has, so it was set for 3 days. Option 3 was assumed to take even further
because there are even less established networks for moving non-WHO approved
devices. Delivery time for Option 4 was eliminated from the analysis because not
carrying the devices would render delivery time irrelevant. The times were varied
by adding or subtracting one day for the high and low-end estimates.
4.1.1.4 Sale of Other Products
Sales of other products inputs were established by assumptions based on
conversations in the focus group discussions. Drug shop sellers informed the
researchers that sales of other products in their shops had increased significantly,
some said their other sales had doubled or more. This could be because these shops
that are now offering mRDTs have increased their services so customers may be
more likely to come into the store. Further, the drug stores received
advertising/marketing materials, which may have contributed to an increased
volume of customers in the store purchasing products. Thus, Option 2 was assigned
as medium high. Option 1 was assigned medium because it was assumed that selling
high quality products would result in repeat customers but the lack of advertising
would result in less sales than Option 2. Option 3 was set as low-medium because it
was assumed that selling a new product would attract some increase in sales but not
as much as options 1 and 2. Option 4 was assigned as low because not selling the
mRDTs would mean there would be no corresponding increase in other sales. High
and low-end estimates increased and decreased the estimates by one increment
except for Option 4.
4.1.1.5 Customer Satisfaction
Customer satisfaction inputs were derived from expert interviews and focus group
discussions. It was assumed that two factors of customer satisfaction include quality
49
of the product and assistance from the retailer. Option 1 was set as medium because
the retailer would be able to sell a high quality test but would be unable to actually
administer it. Option 2 was set as high because in the bundled service, the retailer
sells a high quality device and is trained on how to administer the diagnostic and
then provide advice based on the results. Option 3 was assumed to be low because a
low quality product is offered and Option 4 was also set as low because the device is
not offered at all. The high and low-end estimates increased and decreased the
estimates by one increment except for Option 4.
4.1.1.6 Training
Training inputs were determined based on the amount of, thoroughness, and
comprehensiveness of outside education that the retailer is provided with. Option 2
as a base was assigned as a value of medium-high because the distributors in the
bundled service were required to provide training. For the high-end estimate, this
was increased to high and for the low-end it was decreased to medium, representing
the range in quality of training. Option 1 was assigned as medium-low because it is
possible that a distributor providing WHO-approved diagnostics would take the
time to provide some education to retailers. Option 2 ranges were from medium to
low. Option 3 was set at low for a base and had a high-end estimate of medium-low.
Option 4 was low because the lack of the device being sold is very unlikely to result
in training for the retailer by either distributors or NGOs.
4.1.1.7 Volume Sold
Volume sold for Option 2 was assigned a base value of 20 kits per month sold, which
was based both on what the retailers said in focus group discussion and on how
many devices the distributors had sold up until this point. It was assumed that
Options 1 and 3 would have less volume because there are no corresponding
demand generation activities in those options, and thus the values were set at 10
kits sold per month. Option 4 was set at 0 kits sold per month. The high and low-end
variations in volume were determined by doubling and halving, respectively, the
base assumption.
50
4.1.1.8 Awareness/Advertising
Awareness and advertising depend on the level of outside assistance or effort that
was put into demand generation activities, like marketing and education of
consumers. Option 2 had the highest value for these inputs because the bundled
service obliged the distributors to invest in advertising and marketing. Option 1 was
assigned a base value of low medium because generalized efforts by the public
health community would increase awareness to some extent. Option 3 was assigned
as low because less advertising would be focused on promoting non WHO-approved
devices. High estimates increased the base assumptions by one increment and low
decreased by one increment.
4.1.1.9
Cost
Determining cost inputs was based on assumptions about margin. The cost of the
device for Option 2 was known from interviews on the ground and the margin was
calculated from price and cost. Then, to determine inputs for the other two options,
it was assumed that margin would be the same and from that information, cost
could be determined from the price known from the ACTWatch survey (ACTwatch
Group et al., 2012). The high and low estimates were derived from the price high
and low values.
4.1.1.10 Time to Complete a Sale
Inputs on time to complete a sale were based on focus group discussions with
retailers. In conversations, retailers relayed that they spent 30 minutes on a sale
with a customer when selling an mRDT involved because it took time not only to
convince the customer about the necessity of being tested, but also wait for the test
results, and then provide advice about what medicines to purchase. It was assumed
that Options 1 and 3 would take less time because there was no outside agency
pushing for the use in those options, and if he/she wanted to, the retailer could sell
antimalarials without a diagnostic. Option 4 was 0 because there would be no time
spent selling the diagnostics. The high estimates added 10 minutes to each input
value and the low estimates halved the base assumptions.
51
4.1.1.11 Opportunities
Assumptions about extra opportunities that would be available to the retailers were
based on focus group discussions with retailers and expert opinion. For Option 2,
the extra opportunities was inputted as medium-high because DevOrg is actively
running an intervention and so is able to provide tangible incentives like free shirts
and small jobs. Option 1 was assigned as low medium, because while there is no
active intervention in that option, it is possible that since the retailer is selling a high
quality product an NGO or the local ministry of health would be interested in
partnering or offering incentives. Options 3 and 4 were set as low for opportunities
because there are neither associated interventions nor expected support from the
global health community for selling low quality devices or not selling at all. The high
and low-end estimates increased and decreased the estimates by one increment
except for Option 4.
4.1.2
Decision Input Matrix - Distributors
The comprehensive list of inputs for the distributor decision matrix is summarized
in Table 12 and assumptions explained in the subsequent sections. Some criteria
were not relevant for Option 4; those were marked as N/A. For cross selling and
efficiency of distribution, two sets of values was inputted, one for each distributor.
52
Table 12. Input figures for distributors for high, base, and low estimates for options 1-4
Volume (kits/month)
Option 1
Option 2
Option 3
Option 4
Expiration date (months)
Option 1
High Estimate
Base Estimate
Low Estimate
80
3,200
60
0
40
1,600
30
0
20
800
15
0
12
8
6
Option 2
9
3
3
Option 3
Option 4
Profit (%)
12
N/A
8
N/A
6
N/A
Option 1
30
20
10
Option 2
10
5
1
Option 3
35
20
5
Option 4
Cost per kit ($USD)
Option 1
Option 2
Option 3
Option 4
Cost per training ($USD)
Option 1
Option 2
Option 3
Option 4
Efficiency of distribution
(boxes per delivery)
Distributor 1
Option 1
0
0
0
$0.88
$0.30
$0.95
$0
$0.78
$0.25
$0.78
$0
$0.73
$0.20
$0.65
$0
$400
$1,700
$0
$0
$100
$1,500
$0
$0
$0
$1,300
$0
$0
40
20
10
Option 2
Option 3
10
35
5
20
1
8
Option 4
Distributor 2
Option 1
Option 2
Option 3
Option 4
Cross selling (% increase in
sales)
N/A
N/A
N/A
200
20
160
0
100
5
80
0
50
1
40
0
Option 1
Option 2
50
32
25
16
12.5
8
Option 3
Option 4
27
0
18
0
6
0
53
4.1.2.1
Volume
Volume estimates for distributors were based on conversations from distributors
and the 2009 ACTWatch supply chain survey (Palafox et al., 2012). One distributor
told researchers that they are currently selling 1,600 kits per month, so this was
taken as the base assumption for Option 2 volume. The ACTWatch survey reported
that the median of sales in a week for wholesales was 10 devices. Thus, it was
estimated that sales per month was about 40 devices sold per month and this
estimate was used for a base assumption for Option 1. Option 3 was not
differentiated in the survey, and so it was assumed that there would be slightly less
volume of non WHO-approved devices sold and the volume was set at 30 for a abase
assumption. The high and low-end estimates were established by doubling and
halving, respectively the base assumptions. Option 4 was set at 0 because no devices
are being sold.
4.1.2.2
Expiration Date
The base estimation for Option 2 was set at 3 months because currently in the
project, sale volumes are too low to result in all devices being sold before the
expiration date. Thus, at this rate and if nothing changes, the bulk of the stock will
expire or be given away at its date of expiry. If sales increase quite a bit, a high-end
estimate for Option 2 was set at 9 months and the low-end stayed at 3 months
because it was the lowest option on the distributor's value functions. The expiration
dates for Options 1 and 3 were set at a base of 8 months, higher than Option 2
because it was assumed that distributors would not procure more devices than they
had historically sold and thus would run much less of a risk of stock expiring. One
distributor reported that selling a device at 6-8 months would be reasonable in
normal business operations. The high and low-end estimates were set at 12 and 6
months. Option 4 expiration dates were eliminated from the analysis because it is
not relevant.
54
4.1.2.3
Profit
Base profit estimates for Options 1 and 3 were based on the ACTwatch survey
(Palafox et al., 2012), which reported markups for distributors as 20%. Option 1
high and low-ends were set at plus or minus 10% profit. It is not clear from the
survey whether non WHO-approved devices have a higher or lower markup, so the
ranges were set wider than Option 1 at 35% and 5% margins. Option 4 was 0%
because no products are being sold.
4.1.2.4 Cost of Devices
The mRDTs are subsidized by DevOrg in Option 2. The distributors reported costs
ranging from $0.30 to $0.20 in costs, so those figures were used in the high and low
estimates and the average was taken as a base assumption. The ACTwatch 2009
supply chain survey reported that wholesale prices for mRDTs were $0.78 per kit
(Palafox et al., 2012). This value was used as the base assumption for Option 1 and
the reported range of responses were used as the high and low estimates. It was not
clear from the ACTwatch survey whether non WHO-approved devices would cost
more or less than the WHO-approved ones. Thus, the base cost was set as the same
as Option 1 ($0.78) and the upper and lower bounds were wider, from $0.95 to
$0.65. Option 4 was set at 0 because distributors are purchasing no mRDTs.
4.1.2.5 Cost of Training
Cost of training for Option 2 base assumption was set at $1,500, which was an
average of the two costs reported by distributors. The high-end and low-end
estimates were the two reported values, representing the range of actual costs by
distributors. Cost for Option 1 was set at $100 because distributors reported that
they would consider holding some sort of training for retailers to sell devices. The
high and low-end estimates for Option 1 was set at $400, if this optional training
was held, and $0, if the training was not held. Option 3 was set at $0 because it is
unlikely that a distributor or NGO would provide training on non WHO-approved
devices. Option 4 was set at $0 as there are no devices being sold.
55
4.1.2.6 Efficiencyof Distribution
Efficiency of distribution was measured in terms of boxes per delivery. Since the two
distributors had such different value functions, implying that they operated on
different levels, it was assumed that the input assumptions would be different for
the two.
Distributor 1 had a value function ranging from 1 to 100 boxes per delivery.
Distributors reported that currently, delivery was very inefficient and they were
moving low levels of stock to replenish retailers. Thus, a base assumption was set at
5 boxes per delivery, with a low of 1 and a high of 10 boxes. Option 1 was set at a
base of 20 boxes per delivery because it was assumed that distributors are not
necessarily obliged to provide replenishment if it was inefficient, as opposed to in
Option 2 where the distributors are part of an intervention and working with a
partner to ensure supply of the devices. Option 3 was set as slightly less than Option
2 because there may be less demand for lower quality devices. Option 4 was
eliminated from the analysis because no devices are being delivered.
Distributor 2 had a value function ranging from 1 to 500 boxes per delivery. Option
9 hac
simntinns Were thp same fnr the
hAM
distrihbitnrs hPcausiP thpy are hnth in
the same intervention. However, Distributor 2 reported possible higher volumes
than Distributor 1 so the high-end estimate for Distributor 2 was set at 20 boxes per
delivery. Options 1 and 3 were based on the same assumptions as articulated for
Distributor 1, just scaled up.
4.1.2.7 Cross Sales
Cross sales was measured in terms of percent increase in sales. Like efficiency of
distribution, this criterion also had different input values for the two distributors.
Distributor 1 said that his current percent increase in sales was greater than 5% and
less than 10% so 8% was set as a base assumption for Option 2. The high and low
estimates were double and half the base assumption. Option 1 was set at a base
56
assumption of 10% because it was assumed that distributors would have more time
to turn their attention to other sales, as opposed to in Option 2 where much of their
focus was on moving the mRDTs. The high and low-end estimates were set to double
and half the base assumption. Option 3 was set at slightly less than Option 1 because
it was assumed that lower quality devices would result in less cross sales.
Distributor 2 reported that his current percent increase in sales was 16%, this was
set as a base assumption for Option 2 and the high and low-end estimates were
double and half this figure. Options 1 and 3 were assigned estimates from the same
logic as Distributor 1's assumptions but scaled up to account for different scales of
operations. Option 4 for both distributors was set at 0% because no cross sales
could occur.
4.1.3
Decision Input Matrix - First Line Buyers
The comprehensive list of inputs for the first line buyer decision matrix is
summarized in Table 13 and assumptions explained in the subsequent sections.
Some criteria were not relevant for Option 4; those were marked as N/A.
57
Table 13. Input figures for distributors for high, base, and low estimates for options 1-4
Cost per kit ($USD)
Option 1
Option 2
Option 3
Option 4
Price per kit ($USD)
Option 1
Option 2
Option 3
Option 4
Quality
Option 1
Option 2
Option 3
Option 4
Administrative time
(hours/week)
Option 1
Option 2
Option 3
Option 4
Profit (%)
Option 1
Option 2
Option 3
Option 4
Relationship with donor
organization or funder
Option 1
Option 2
Option 3
Option 4
4.1.3.1
High Estimate
Base Estimate
Low Estimate
0.7
0.09
1
0
0.35
0.09
0.35
0
0.18
0.09
0.15
0
0.40
0.30
0.40
0
0.25
0.25
0.25
0
0.10
0.20
0.10
0
H
H
LM
N/A
MH
H
L
N/A
MH
MH
L
N/A
10
60
5
0
5
35
3
0
1
10
1
0
50
40
50
0
40
30
40
0
30
20
30
0
MH
H
ML
L
M
MH
L
L
ML
MH
L
L
Cost per kit
The costs were based on conversations with the FLBs. One FLB said that the
subsidized cost to them was $0.09 per kit, thus Option 2 was set at that. The FLB
also said that the unsubsidized price was $0.35 so that price was used as the base
for Options 1 and 3. The high and low-end estimates for Option 3 were assumed to
58
have a wider spread than Option 1 because there is less regulation for those
manufacturers. Option 4 was set at $0 because the diagnostic is not sold.
4.1.3.2 Price per kit
The average of the reported costs that the distributors paid was used as the price for
the kits. The base cost of the kits were set as the same, but the spread of high and
low was set higher for Options 1 and 3 than Option 2, as it is assumed there would
be much less price control. Option 4 was set at $0.
4.1.3.3
Quality
Quality was based on the same assumptions as the retailers. See the decision matrix
input assumptions for retailers
4.1.3.4 AdministrativeTime
The high and low estimates for the administrative time were the different reported
amounts from the two FLBs. The base assumption was the average of the times.
Options 1 and 3 were assumed to have much less administrative time because there
would be no coordination down the supply chain nor with an NGO organization.
However, Option 1 was set at a higher time for the high-end estimate than Option 3
because it is assumed that there is likely more administrative time to ensure
compliance and possible coordination with other organizations. Option 4 was set at
0 hours because the devices are not being sold.
4.1.3.5 Profit
Profit for Option 2 was set at 30% as a base assumption because that is what one
FLB told us they had. The high and low estimates were plus and minus 10%. Options
1 and 3 were set at 10% higher than Option 2 because it was assumed that there
was less price control and the FLBs could mark up prices as they thought
appropriate. Option 4 was set at 0%.
4.1.3.6 Relationship with Donor Organization
It was assumed that Option 2 would have the highest score in relationships because
there would be direct coordination with the organization. Option 1 would have
59
slightly lower scores, as although there may not be participation in an intervention,
the FLB is still selling WHO-approved devices and may have some interaction.
Option 3 was lower because the FLB would not be selling the approved devices.
Option 4 would be the lowest as devices are not sold at all.
5
Results - Value Functions
This section describes the value functions that were constructed for the retailers,
distributors, and first line buyers. Additionally, this section provides information on
the criteria, sample size, and comments on the shape of the value function.
5.1
Retailers Criteria
The retailers identified eleven criteria that are factors in their decision about
whether or not to stock an mRDT. A list of initial criteria was generated by expert
opinion and literature reviews and then a focus group discussion with the retailers
added and confirmed the list of criteria (see Methodology for full procedure).
Further, the ranges of the criteria were proposed based on focus group discussions.
Table 14 summarized the criteria generated and the upper and lower bounds of the
possible range.
60
Table 14. Criteria and ranges of criteria for retailers.
Criteria
Lower bound
Upper bound
Quality
Low
High
$0 USD
$1.67 USD
Time to delivery (days)
7
1
Sales of other products
Low
High
Customer Satisfaction
Low
High
Training
Low
High
0
50
Low
High
0
60
Low
High
0
$1
Price
Volume (kits sold per month)
Awareness/ads
Time to complete a sale (minutes)
Other Opportunities
Cost
5.2
Retailers Value Functions
The criteria that was included in the retailers' considerations in their decisionmaking process were: quality, price, delivery, sales of other products, customer
satisfaction, training, volume sold, awareness/advertising, time, opportunities, and
cost. A value function was constructed by having the range of criteria as the
independent variable (e.g., cost ranging from $0 to $1) and value on a scale of 1-5 as
the dependent variable. Retailers were asked about their value for two different
points and the median of all the responses was calculated and used in the value
functions. See Methodology section for how responses were solicited. This section
will exhibit and analyze the value function constructed for each of the criteria.
5.2.1
Quality
Quality referred to the quality of the mRDT that the retailer could sell. Quality took
into consideration accuracy and durability; a key factor was whether or not the
device was WHO approved (see Methodology section). Respondents were asked
61
about their preference if the quality of the device was medium-low and mediumhigh (Table 15).
Rebler Prferences for Quaity
UN
I
/
/
I
/
/
41
/
/
/
A
/
3_x
/
/
/
41
/
/
'
/
2
'
I
2
2)5y1.5
3
Qualy Low (1) trou
3.5
High (5)
4
4.5
5
Figure 7. Value function for quality of devices for retailers.
Table 15. Median of retailer responses for quality
Quality
Low
Low-medium
Medium-high
High
Value
1
1.5
5
5
The value function for quality (Figure 7) is revealed to have a sharp increase in
value followed by an increase at a decreasing rate. The value from 4 to 5 was flat.
The relative increase from low to medium-low and medium-high to high was not as
great as from medium-low to medium-high. In fact, retailers seemed to be
62
indifferent between a quality of medium-high to high. This value function implies
that retailers care about a threshold of quality, but do not value incremental
improvements in quality as much.
For quality, the sample size was 17 responses total, 11 full sets (those that provided
a response for both low-medium and medium-high), and three blanks. Two
respondents provided a value for only the medium-low question and one
respondent answered only for low-medium. The reason only 17 responses were
recorded, instead of the full 28 was that the second focus group discussion
responded that quality was not important to them. For this reason, the only data
gathered was for the first focus group discussion.
5.2.2
Cost
Cost was defined as the cost of the device to the retailer; that is, how much the
retailer has to pay to procure the device. The ranges of cost were set from $0 USD to
$1.67 USD (retailers were asked their preferences in Ugandan shillings not U.S.
dollars). The median of the responses can be found in Table 16.
63
Retailer Prefemnces for Cost per Device
I
I
V
I
I
I
I
N,
N,
4.5
4
N,
N,
N,
3.5
N
N,
3
N.
N,
2.51
2
N,
N,
0
0
0.1
0.1
02
02
Figure
0:3
03
OA
0.4
0.5
0.5
0.~
Ok
Cost per Device in SUSD
0.7
0.7
4
1
8.. Value function for cost per device for retailers.
Table 16. Median of retailer responses for cost per device
Cost per device
$0 USD
$0.50 USD
$0.67 USD
$1USD
Value
5
2
1
1
The value function for cost (Figure 8) drops sharply from $0 to $0.67, after which
retailers express a preference of 1 for any increasing cost. This implies that there is
a narrow band of ranges of cost for the device that retailers prefer.
64
The sample size for cost excluded the first focus group discussion. This is because it
is presumed that there was confusion and miscommunication about the questions.
Especially as this was the first group and one of the first questions asked, it is likely
that the methodology had not been refined enough to ensure accurate responses. Of
the 17 retailers in the first focus group, there were 4 complete responses that
indicated that a higher cost was associated with lower value, which is what one
would expect. Seven retailers indicated the opposite trend, that higher costs had a
greater value to them, 2 retailers were blank, and 4 were half-answers. It is possible
that the 7 retailers were accurately representing their values, that more expensive
items are valued more, perhaps because cost is associated with higher quality.
However, it is also very likely that the retailers were confounding price and cost,
even though the moderators did explain it. Especially since in the second focus
group discussion, where the moderators were careful to explain price and cost
separately in both English and the local language, the responses all had the initially
expected trend where higher cost is paired with lower value. For this reason, the 11
retailers in the second focus group discussion were used in the analysis and the first
discussion eliminated. Of the 11 retailers, 10 provided full responses and there was
1 blank.
5.2.3
Price
Price was defined as the price at which the retailer sells the mRDT kit to the
customer and the ranges were set from $0.33 to $0.67. Table 17 exhibits the median
of the responses for price.
65
Retaer Preferences for Price per device
5
-
4.51
4
P
I
~3b I
/
I
/
/
3
/
/
2z L/
2
/
()
1.5 L
0L
2
- OA
0h
0.8
1
12
Price per device in SUSD
1A
1.
1.8
Figure 9. Value function for price per device for retailers.
Table 17. Median of retailer responses for price per device
Price per device
$0.33 USD
$0.67 USD
$1USD
$1.67 USD
Value
1
2
4
5
The value function for price (Figure 9) demonstrates a fairly sharp increase in value
from $0.33 to $1 and then a diminishing return on value from $1 to $1.67. This
implies that retailers have a price at which they greatly want to sell the product,
likely taking into account the cost of the device and other related costs.
The sample size for the value function for price included both focus group
discussions. Both were included, despite excluding the first for looking at cost,
66
because there were only 4 respondents that indicated that a higher price had a
lower value. This implies that respondents had understood the question. Thus, the
sample size was 28 retailers asked, out of which there were 2 blanks, 2 that
answered only for $1, 1 that answered only for $0.67, and 4 that were determined a
miscommunication (for list of removed values see Table 18).
Table 18. Removed response sets for price of mRDT
Removed
Removed
Removed
Removed
respondent
respondent
respondent
respondent
1 values
2 values
3 values
4 values
$0.67
5
4
5
3
$1
1
2
2
1
5.2.4 Time to Delivery
Time to delivery was defined as the number of days after a retailer placed an order
that their supplier would provide more product. The value function was plotted for
an independent variable ranging from 1 day to 7 days. One-day replenishment was
set as the upper bound as a 5 for value and seven days was set as the lower bound as
a 1. Respondents were asked about their value for two days of delivery and 4 days of
delivery. The median of responses was recorded and used in the value function
(Table 19).
67
Re
I.riPrebrences for Desvery Tine
4
4
3
3.5
25-
2 -b
1.5
1
1
2
5
4
3
6
Time to Defivey (days)
Figure 10. Value function for delivery time for retailers.
Table 19. Median of retailer responses for delivery time
Delivery time
1 day
2 days
4days
7 days
Value
5
5
2
1
The value function for delivery time (Figure 10) exhibits an inflection point at 4 days
of delivery. Value is equal between 1 and 2 days of delivery, decreases sharply from
2 to 4 days, and decreases less sharply from 4 to 7 days. This could be because
retailers categorize delivery times in two categories-"sooner" and "later" and
anything after 4 days is considered relatively the same. This also means that a oneday decrease in delivery time from four days has a high return on value.
68
The question of delivery time was asked to all 28 retailers and 23 provided a
response. Four respondents did not provide a response at all, and 3 responded only
to the 2-day delivery time. Three responses were excluded from the analysis
because of presumed misunderstanding; they recorded higher values for longer
delivery time. Table 20
Table 20. Removed responses for Delivery Time
Removed respondent 1 values
Removed respondent 2 values
Removed respondent 3 values
5.2.5
2 day delivery
2
3
1
4 day delivery
3
4
5
Sales of Other Products
Sales of other products was evaluated in qualitative terms ranging from low (1) to
high (5). Sales of other products was defined as products that were sold because the
retailer provided mRDTs. This could be because a customer now chooses that store
or goes in to be tested and then purchases other necessary products there, or for
any other number of reasons. Low was described as almost no new sales, medium
was described as some new sales, and high was described as a lot of new sales. The
median of the responses were graphed in the value function (Table 21).
69
Fatater references for Other Sales
5
4.5
4
3.5 1.
0)
3
I
I
I
2
15
3.5
4
2
2.5
3
Increase in Other Sales Low (1) through High (5)
4.5
Figure 11. Value function for sales of other products for retailers
Table 21. Median of retailer responses for increased sales of other products
Increase of sales of other
Value
products
Low
1
Low-medium
2
Medium-high
4
High
5
The value function for sales of other products (Figure 11) is linear; value increases
equally as sales of other products increases. It is possible that the function would
70
have changed if percent sales were provided instead of qualitative values, as was
done for distributors.
Responses for sales of other products was solicited from all 28 retailers and there
were 20 full responses. Three responses were completely blank and 5 individuals
provided only an answer to medium-high. One response was eliminated from the
analysis because of presumed misunderstanding. The removed response reported
value of 1 for medium-high and 4 for low medium.
5.2.6
Customer Satisfaction
Customer satisfaction was defined as perceived satisfaction of customers after a
business interaction. Low satisfaction was described as the customer being very
unhappy, medium was neutral, and high was that the customer was very happy. The
median of responses was graphed (Table 22). The value function for customer
satisfaction was exhibited to be linear (Figure 12).
71
RetaNer Preferences for Customer Safsfactan
5
4.5
7
4
A
3.517
3
7
2.517
2
1.5
I
15
2
I
I
I
2.5
3
3.5
4
Customer Satisfaction Low (1) through High (5)
4.5
5
Figure 12. Value function for customer satisfaction for retailers
Table 22. Median of retailer responses for customer satisfaction
Customer Satisfaction
Low
Low-medium
Medium-high
High
Value
1
2
4
5
All 28 retailers were asked about customer satisfaction and 20 provided full
responses. There were 5 blanks and 3 responses that only had information provided
about medium-high satisfaction.
5.2.7 Training
Training was described as an outside agency (which could include NGOs,
distributors, etc.) providing training on how to properly administer and interpret
72
the mRDTs and then provide advice to the customer about proper next steps.
Training was described on a qualitative scale from low (1) to high (5), where low
was little or no training, medium was some training, and high was thorough and
comprehensive training and guidance. Table 23 exhibits the median of responses.
Romber Prefernces for Training
/
/
5
4.5
/
/
4
/
3.513
/
2.5 I-
2
7
1.51-_
7
z
1
T
1.5
I
2I
I
2.5
3I
3.5I
1b
2
2.5
3
3.5
Training Low (1) through High (5)
4
4.5
5
Figure 13. value function for training for retailers
Table 23. Median of retailer responses for training
Training
Low
Low-medium
Medium-high
High
Value
1
2
5
5
73
The value function for training (Figure 13) has a vaguely sigmoidal shape. The
increase in value from low training to low-medium training is not as great of an
increase as from low-medium to medium-high. Further, respondents seem to be
indifferent between medium-high and high training. This implies that retailers value
above a medium amount of training but have diminishing returns to training greater
than that.
All 28 retailers were asked about training and 20 provided a full response. There
were 5 blanks and 3 retailers that only provided a response to medium-high.
5.2.8 Volume
Volume was defined as the number of devices sold per month. The ranges were
established as 0 as the lower bound and 50 mRDTs sold per month as the upper
bound. Table 24 shows the median responses.
74
Ibtaer Prefrences for Volume
5>
-
4.5
-
4
3A
-
-
35
,5
25
-
1.5
5
10
15
25
30
20
Sales per month
35
40
45
50
Figure 14. Value function for volume of sales for retailers
Table 24. Median of retailer responses for volume
Volume
0 devices sold/month
10 devices sold/month
30 devices sold/month
50 devices sold/month
Value
1
1
4
5
The value function for volume of mRDT devices sold per month (Figure 14) is
roughly s-shaped. The fact that respondents value 10 devices sold per month as a 1,
the same as if no devices were sold implies that there is some negative consequence
to holding the devices and not selling above a threshold of them. This could occur if
retailers have to buy the devices from distributors in packages and therefore they
have some amount of sunk cost. It is also possible that having the devices sit on the
75
shelf, and take up space, is reducing their value, perhaps by taking up the space of
another product.
All 28 retailers were asked about volume and 20 complete responses were
recorded. There were 7 blanks, 1 individual who responded only to 30 devices sold
per month, and 2 who responded only to 10 devices sold per month. One response
was removed from the analysis from presumed misunderstanding. The removed
response reported 30 mRDTs sold per month as a value of 4 and 10 mRDTs sold per
month as a value of 5.
5.2.9
Awareness/Advertising
Awareness/advertising was defined as the amount of advertisements and
awareness that was generated by an outside agency (including NGOs, distributors,
etc.). The metric was qualitative ranging from low (1) to high (5), where low was
described as little or no advertising, medium was described as some advertising,
and high was a lot of and/or very effective advertising. Table 25 describes the
median responses to awareness/advertising.
76
RetNer Preferences f1r Amrns/AdvwtisIng
5
4.5
4
-
3.5
0
3
I
I
I
II
2
r
15
2
4
2.5
3
35
Awareness/Ads Low (1) through High (5)
4.5
5
Figure 15. Value function for awareness/advertising for retailers
Table 25. Median of retailer responses for advertising/awareness
Advertising/ Awareness
Low
Low-medium
Medium-high
High
Value
1
2
4
5
The value function for advertising/awareness was linear (Figure 15); the value
scaled equally with awareness and advertising.
All 28 retailers were asked about their preferences on awareness/advertising and
18 full responses were recorded. One respondent answered only for medium-high
77
and one answered only for low-medium. Two values were excluded from the
analysis from presumed miscommunication (Table 26).
Table 26. Removed responses for Advertising/Awareness
Low-medium
Medium-High
Removed respondent 1 values
4
2
Removed respondent 2 values
4
3
5.2.10 Time to Complete a Sale
Time to complete a sale was defined as the number of minutes the shopkeeper
spends with a customer spends in the store facilitating a sale. In the case of mRDTs,
this generally involved explaining the benefits to testing for malaria, affirming the
reliability and accuracy of the test, then actually administering the test, and
providing recommendations for appropriate next care (antimalarial or something
else). The times ranged from 60 minutes of time that the shopkeeper needed to be
engaged with the customer to 0 minutes, which is the interaction consisting of not
much more than the exchange of money for the product. Table 27 exhibits the
median responses for the times.
78
Retaler Prerences far Tine to Campile Sale
r
-
45
4
3.5
(A)
d0
3
2.5-
2
1.51-
1
0
.2
10
20
30
Minutes per sale
46
50
Figure 16. Value function for time to complete a sale for retailers
Table 27. Median of retailer responses for advertising/awareness
Time to complete sale
60 minutes
40
10
0
Value
1
1
4.5
5
As the time to complete a sale increases, the retailers' value decreases (Figure 16).
This makes sense because if the retailer spends an extended amount of time with
one customer he or she is unable to help others. Sometimes a sale could be lost if the
retailer is busy as a customer will leave and go to another store. The decrease in
value from 0 to 10 minutes is much less steep than from 10 minutes to 40 minutes.
This implies that retailers expect to spend a few minutes facilitating a sale but do
79
not want to spend an extended amount of time. Both 60 and 40 minutes have a value
of 1, implying that spending 40 minutes or greater on a sale is very undesirable.
Time to complete a sale was not a criterion that was identified by the first focus
group discussion. Therefore, data was only gathered from the second discussion,
which included 11 respondents. Of the 11 asked, 10 provided a full response but 2
out of 10 were removed from presumed misunderstanding. These individuals
indicated that they had a higher value for the greater amount of time spent to
complete the sale (Table 28). However, it is possible that these individuals were not
misunderstanding, and rather have reverse preferences than the rest of the cohort
for this criteria. One plausible scenario could be that these retailers were
expressing a correlation between spending longer time with a customer and greater
customer satisfaction or even more sales. However, since the chance of
miscommunication was high in the focus group discussions and because the rest of
the retailers indicated the predicted response, the values were removed from the
analysis. Further, since the high and low value levels were defined, any responses
that went against that direction were removed (see Discussion section)
Table 28. Removed responses for time to complete sale
Removed respondent 1 values
Removed respondent 2 values
10 minutes
40 minutes
1
3
3
4
5.2.11 Opportunities
Opportunities as a criterion was defined as other benefits that are associated with
selling mRDTs. For example, the retailers received a free shirt that was emblazoned
with a mosquito and the word "mRDT." Retailers also expressed that in general,
when they participate in projects or campaigns there are other giveaways, prizes,
and competitions. Further, there is opportunity to participate on a higher level, in an
organizing or mobilizing capacity, which is usually financially compensated. A
qualitative metric was used to aggregate all these aspects, and respondents were
80
asked to rank their value for opportunities ranging from low (1) to high (5). Low
opportunities was defined as very little or nothing, medium was some opportunities
provided, and high was frequent and useful opportunities. Table 29 summarizes the
median of responses.
Retailer Prebrencee br Oppa~inI~.a
5
I
I
I
I
I
4.5
4
k
3.5
0
3
/
0
2.5 I
1.5
11
-
)
2
I
1.5
2
I
I
I
2.5
3
3.5
Opportunity Low (1) through High (5)
I
4
4.5
5
Figure 17. value function for opportunities for retailers
Table 29. Median of retailer responses for opportunities
Opportunities
Low
Low-medium
Medium-high
High
Value
1
1.50
4
5
81
The value function for opportunities (Figure 17) has an inflection point at lowmedium opportunities. Value increases from low to low-medium then increases
sharply from low-medium to medium-high. There is only a slight change in slope
from medium-high to high. These results suggest that retailers see little value from
low and low-medium opportunities, but as the opportunities increase from there,
their value increases greatly.
Opportunities, like time to complete a sale, was a criterion that was not
brought up in the first discussion group. Therefore, 11 retailers were asked about
their preference and 10 provided a complete response.
5.3
Distributors
All three distributors in the supply chains studied were interviewed. Of the three,
one was unable to provide quantitative information and was classified as a nonresponse. Thus, two value functions for each criterion is presented, one for each of
the distributors that answered. The range of criteria was set as the same for both
distributors (Table 30).
Table 30. Criteria and ranges of criteria for distributors
Criteria
Volume (kits sold per month)
Expiration date (months left to expiry at sale)
Profit margin (%)
Cost per kit ($USD)
Cost per training ($USD)
Efficiency of distribution (Boxes per delivery)
Cross selling (% increase in sales)
5.3.1
Upper Bound
18,750
22
50%
$0.83
$2,000
500 (or 100)
50% (or 20%)
Lower Bound
0
3
0%
$0
$0
1
0%
Volume
The volume of sales for the distributors was recorded in units of kits sold per
month. Since distributors articulated volume in different units, it was necessary to
convert to one metric for ease of comparison of value functions (see Methodology
section). The value functions between the two distributors are similar, but
Distributor 2 (Figure 19) exhibits an inflection point while Distributor 1's (Figure
82
18) response is more linear. The two end points were established in conversations
with both distributors where the lower bound was set as no sales and the upper
bound was defined as selling all of their product in 16 months, which is a few
months before the expiration dates on the kits and before the campaign technically
ends. The distributors expressed their desire to have sold all the kits quickly both so
that they did not have stock taking up space in their warehouse and also so they did
not have to discard expired kits.
Distribmwor I Preferences
5
for Volume
4.51-
4
I-
3.5 k
3
2.51-
2
0
02
0.4
0.6
0.8
1
12
Kits sold per month
14
1.6
1.8
2
10A
Figure 18. Value function for Distributor 1 for volume
83
Table 31. Distributor 1 responses for volume
Value
1
3
5
Volume (kits sold per month)
0
9,000
18,750
Distribulor 2 Pr.$wences for Volume
I
I
I
I
I
I
I
I
~.~-*'
41
C4
7
7
Z 2~
7
it
/
7
/
/
1~
02
0.4
0.~
Oh
12
1A
?S
02
0.4
0.6
12
1
0A5
Kits sok per month
1.4
1
1 I.
2
1.8
S10'
Figure 19. Value function for Distributor 2 for volume
Table 32. Distributor 2 responses for volume
Volume (kits sold per month)
0
1,625
18,750
Value
1
2
5
84
5.3.2
Expiration Date
Expiration date was defined as the number of months left before expiry when the kit
was sold to the retailer. Ensuring that the retailers received the kit with enough
time on their end to sell the product was important. Thus, the date to expiry ranges
from 22 months if the kits were sold immediately and 3 months as the lower bound,
as the distributors informed us that after 3 months, they would not be able to sell
the product. The value functions for both distributors exhibit an inflection point at 6
months, after which value drops sharply (Figure 20 and Figure 21). This is likely
because of the aforementioned issue that the distributors are much less likely to be
able to sell the products with such a short amount of time to expiration.
mskibutr 1 Pre.rencos for Expkratn Data
D
4.51
4
j
0
3.51
y
3
0
2 5 L-
&
0
/
2
.1
2
4i
6p
8
10
p
12p
14p
16
p
18p
4
6
8
10
12
14
16
18
Months left to expiry at sale
20
22
Figure 20. Value function for Distributor 1 for expiration date
85
Table 33. Distributor
1 responses for expiration date
Months left before
expiration at time of sale
3
6
22
Value
1
2
5
mskriutmr 2 Prbfrwsces for Expiration Dahe
5
45-
4e~J
3~5
3
&
25
/
2
1,5
iI
2
4
6
a
14
12
10
Months left to expiry at sale
16
18
20
22
Figure 21. Value function for Distributor 2 for expiration date
Table 34. Distributor 2 responses for expiration date
Months left before expiration at
time of sale
3
6
22
Value
1
3
5
86
5.3.3
Profit Margin
Although profit margin was not included in the list of criteria that was
generated before the data collection process, because it is comprised of three
variables (cost, volume, and price), it was soon clear that distributors thought in
terms of profit margin and wanted to express their opinions on that. Distributors
indicated that a low bound would be 0% margin and the high bound for products
like this was 50%. The value functions for both distributors are very similar,
demonstrating an inflection point at around 25-30% (Figure 22 and Figure 23).
Value increases sharply to 25-30% profit margin and then increases at a decreasing
rate after. This agrees with what the distributors said in interviews, that having high
profit was a good thing, but they were not seeking high profits on products like
mRDTs.
Dlsrlbubr 1 Prelfmances for Proml Mgla
5
4. 5
I I
I
Q
I
4
3b
IS
3
2
1 .5
11
0
5
10
15
20
25
30
Percent proft margin
35
40
45
50
Figure 22. Value function for Distributor 1 for profit margin
87
Table 35. Distributor
1 responses for profit margin
Value
1
4
5
Profit margin (%)
01
27.5
50
DIstvbiabr 2 Prfwrec.. for Promt Marn
5
4
-
4.5F
S3
~2.5
2
1.5
I
S
5
10
15
20
25
35
30
Percent profit matgin
40
45
30
Figure 23. Value function for Distributor 2 for profit margin
Table 36. Distributor 2 responses for profit margin
Profit margin (%)
0
25
50
Value
1
4
5
88
5.3.4
Cost Per Kit
The cost per kit was the cost that the distributor pays to procure the device.
Distributor 1's value function (Figure 24) exhibits an inflection point at $0.30 while
Distributor 2's is more linear (Figure 25). This could be due to different preferences,
or from a limited number of data points. It is possible that getting more information
from Distributor 2 would reveal an inflection point.
Distdbutor 1 Preferences for Cost per Kit
I
45
I
I
I
I
I
I.
4
35b.
3
S
I
2.5
2
N.
1,.5
1
0
0.1
I
I
I
02
03
0.4
I
0.5
Cost (USD) per kit
0.6
0.7
0.8
0.9
Figure 24. value function for Distributor 1 for cost per kit
Table 37. Distributor 1 responses for cost per kit
Cost per kit ($USD)
0
0.30
0.83
Value
5
4
5
89
DIstrbutor 2 Prfornces for Cost par Kit
C
4. 4
3.
S
&
15
2-
I.
0
0.1
02
03
0.4
0.5
Cost (USD) per kit
0.6
0.7
0A. 0.
Figure 25. Value function for Distributor 2 for cost per kit
Table 38. Distributor 2 responses for cost per kit
Cost per kit ($USD)
0
0.20
0.83
5.3.5
Value
5
4
1
Cost per Training
As per DevOrg's contract, the distributors were responsible to provide training
sessions to the retailers before they could sell the mRDTs. The requirements for
each session were outlined in standard operating procedures. The cost per training
value functions for the two distributors are shaped differently. Distributor 1 (Figure
26) exhibits two inflection points at $500 and $1,700 per training. Distributor 2
(Figure 27) has a very slight inflection point at $1,350.
90
DIstributor 1 Prelrences for Cost per Training
r.
4.5 I
4
3.51
3
2.51
&
0
2
1.51
I
0
200
4DO
ODO
WO0
1000
1200
Cost (USD) per training session
1400
1600
1800
26 0
Figure 26. Value function for Distributor 1 for cost per training
Table 39. Distributor 1 responses for cost per training
Cost per training ($USD)
0
500
1,700
2,000
Value
5
3
2
1
91
Distributor 2 Prebrwces far Cost pr Training
4.5
4
c'J
3,b
0
3
0
2~5
&
0
2
1 .5
0
200
400
00
800
1000
1200
Cost (USD) per trailng session
1400
1600
1800
2030
Figure 27. Value function for Distributor 2 for cost per training
Table 40. Distributor 2 responses for cost per training
Cost per training ($USD)
0
1,350
2,000
5.3.6
Value
5
2
1
Efficiency of Distribution
Efficiency of delivery was brought up by the distributors in conversation as a factor
in their decision. A metric defined as number of boxes per delivery was established
to measure this criterion. The distributors expressed that there was a fixed cost in
fuel and salary to deliver boxes to the retailers, and delivering a small number of
products was not efficient for their business. This criterion was one of two that had
a different range for the two distributors. Distributor 1 asserted that the high-end
92
range for him was 100 boxes per delivery and for Distributor 2 it was 500 boxes per
delivery. This is likely due to the difference in overall size and extensiveness of the
two distributors. Distributor 2, for example, said that the mRDT business was less
than 1% of his portfolio. Thus, the order of magnitude his business operates is likely
much greater than Distributor 1.
Despite having different scales, the value functions (Figure 28 and Figure 29) exhibit
almost the same shape. There is an inflection point early in the range, where value
increases sharply before it and at a decreasing rate after. This suggests that
distributors seek to meet a minimum threshold of devices per delivery.
DhitrWt~r 1 Pafemeas for Efficiwcy of Dialribution
4.51
4
/-
-
0-
3.5b
3
2,5I
2
1.5
1
0-
10
20
30
40
50
60
Number of boxes per dloivary
70
80
90
100
Figure 28. Value function for Distributor 1 for efficiency of distribution
93
Table 41. Distributor 1 responses for efficiency of delivery
Boxes per delivery
1
10
100
Value
1
3
5
DI.tribu~w 2 Prferncs ~r Efficiency of Delivery
5
I
I
I
I
I
50
100
150
200
250
I
I
I
I
I
A
I
A
50
100
150
200
250
300
350
400
I
I
I
350
400
450
*-~~
-
4.5
4
36-
/
/
S3
/
/
2
1.51L
F
1:
300
Number of boxes per delivery
450
S0
Figure 29. Value function for Distributor 2 for efficiency of distribution
Table 42. Distributor 1 responses for efficiency of delivery
Boxes per delivery
1
100
500
Value
1
3
5
94
5.3.7
Cross Selling
Cross selling for distributors is analogous to the criteria "increased sales of other
products" for the retailers. However, for distributors the unit used to describe the
criteria was percent increase in sales due to carrying the mRDTs. These sales could
come from an increase in their distribution network and/or from the sale of gloves
and other materials necessary to administer the tests. The range of cross selling was
different for the two distributors. Distributor 1 expressed that the high value would
be 20% increase in sales and Distributor 2 said 50%. Again, this may have to do with
the relative scales and scopes of their respective business operations. The value
functions are almost linear. Distributor 1's value function (Figure 30) is linear with
the data points provided and Distributor 2's (Figure 31) exhibits a very slight
inflection point. This suggests that, in general, increase in cross sales scales linearly
with value.
Dilstributor 1 Prefenics for Cross Soing
4.5.
4-
0 3.5-
3
2.5
/
2
2
4
6
12
10
a
Percent increase in sales
14
16
18
20
Figure 30. Value function for Distributor 1 for cross selling
95
Table 43. Distributor 1 responses for cross selling
Value
1
Percent increase in sales (%)
0
10
20
3
5
Distriutor 2 Prfernces for Cross SeNing
5
4.
4
CM4
S2.5
2
1,5
ILI
10
15
20
25
30
Percent increase in sales
35
40
45
50
Figure 31. Value function for Distributor 2 for cross selling
Table 44. Distributor 2 responses for cross selling
Percent increase in sales (%)
0
16
50
Value
1
2
5
96
5.4
First Line Buyers
This section describes the value functions for the both of the first line buyers.
Cost per Device
5.4.1
The cost per device was defined as the cost to the FLB to purchase the mRDT from
the manufacturer. There was a difference in the shape of the value functions
between the two FLBs. FLB 1 (Figure 32) indicates minimal value at $0.333 and
higher. FLB 2 (Figure 33) has a much less dramatic decrease value corresponding
with increased cost.
FLB I Prnfernces for Cost per Device
4.51-
4
'I
-
3.5
VL
13
2
0
I
I
I
I
I
0.1
02
0.3
0.4
05
I
0.
Cost per Kit (SUSD)
~
0.7
0.8
0.
Figure 32. Value function for FLB 1 for cost per device
97
Table 45. FLB 1 responses for cost per device
Cost per device ($USD)
$0.033
$0.167
$0.333
$1
Value
5
4
1
1
FLB 2 Prtrnc.. for Cost per Device
5
I
I
I
I
I
I
I
I
4.5 1k
4
3.51N
N~
-9
3
2.5L-
2
N
N
1.5 IN
1
I0
0.1
02
03
0)6
OA
05
Cost per Kit (SUSO)
0.7
0.8
0.9
r
Figure 33. Value function for FLB 2 for cost per kit
98
Table 46. FLB 2 responses for cost per device
Value
5
3.5
1
Cost per device ($USD)
$0.033
$0.15
$1
5.4.2
Price per Device
Price per device was defined as the price at which the retailer sells the device to
their distributor. FLB 1's value function (Figure 34) reveals a very sharp increase
from a price of $0 to $0.033, then a relatively steep increase from 0.033 to $0.167,
and then a shallower increase from $0.167 to $1. FLB 2 (Figure 35) reveals a similar
pattern, with a sharp increase at first followed by a less dramatic increase.
S
I
I
FIB I preference, for Price I per Device
I
I
4.6
41
3.1
-J
LA~
----
21
-
I
1 .'
0.1
02
0.3
GA
0.~
0./
0.1
02
0.3
0.6
0.5
0.4
Price per Kit (SUSD)
0.7
0.5
a
a
0.
a
0.9
Figure 34. Value function for FLB 1 for price per kit
99
Table 47. FLB 1 responses for price per device
Price per device ($USD)
0
$0.033
$0.167
$0.333
$1
Value
5
3
3
4
5
FIB 2 Prefwences for Price per Device
5
465
4
3.51
/
N'
3
I
-
/
265
I
-
/
2
I
/
1.5
1
a
I
I
0.1
02
0.3
0.4
0.5
0.6
Price per Kit (SUSD)
0.7
0A
09
1
Figure 35. Value function for FLB 2 for price per kit
Table 48. FLB 2 responses for price per device
Price per device ($USD)
$0
$0.25
$1
Value
5
3.5
5
100
Quality of Device
5.4.3
The quality of the device was described in terms of WHO-approval, accuracy, and
durability (see Methodology section). Both FLBs (Figure 36 and Figure 37) indicate
a dramatic decrease in value when the quality of the device is low.
FLU I Pmfmnce. for audily of Device
5
I
I
I
/
I
/
45 I
4
~35 I-J
3
2.5 I-
2
'.5 j
r
#1
I
2.5
3
Qualty Low (1) ftho
1.5
Figure 36. Value function for FLB
I
3.5
High (5)
I
4
4.5
1 for quality
Table 49. FLB 1 responses for quality of device
Quality of device
High
Medium-high
Medium
Low-medium
Low
Value
5
2
2
1
1
101
FB 2 Preferences for Quality of Device
5
-77-
4.5
4
3.51
N~
9J
3
2.5
2
1.5
I
r
1.5
3
3.5
25
Qualty Low (1) through High (5)
4
4.5
5
Figure 37. Value function for FLB 2 for quality
Table 50. FLB 2 responses for quality of device
Quality of device
High
Medium-high
Medium
Low-medium
Low
Value
5
4
1
1
1
102
Administrative Time
5.4.4
Administrative time was defined as the number of hours per week that the FLB
spends managing the sale of the devices. FLB 1's value function (Figure 38) reveals a
dramatic decrease in value from 1 to 10 hours per week. FLB 2 (Figure 39) has a
similar decrease but at a rate less than FLB 1.
FLB 1 Prfrences for Hoirs of Admindxrafve Time
5
4,5
4
3.5'.
U9
%3
2b51
2
1
0
1-0
20
3
40
Hours spent per week on administration for mRDT sales
50
Figure 38. Value function for FLB 1 for administrative time
Table 51. FLB
1 responses for administrative time
Administrative Hours (hours)
1
5
10
60
Value
5
3
1
1
103
R.B 2 Preferncos for Hous of Admli*&ave Tim.
5
4,51
4
3.5
L
3
2.5
2
1.5
1
20
S10
37
40
Hours spent per week on administration for rnRDT sales
50
Figure 39. Value function for FLB 2 for administrative time
Table 52. FLB 2 responses for administrative time
Administrative Hours (hours)
1
5
10
60
5.4.5
Value
5
4
3
1
Percent Profit
Percent profit was the profit on the devices that the FLBs would receive. This was
the only criterion that had different ranges; FLB 1 indicated that their upper bound
of profit was 50% and FLB 2 indicated theirs was 100%. Both FLB's indicated a
104
sharp increase in profit initially and then an inflection point after which value
increased at a slower rate. FLB 1's value function (Figure 40) reveals a very steep
increase in value from 0% to 5-10%, and then maximal value at profits increasing
after 20%. FLB 2 (Figure 41) exhibits an inflection point at 33%.
FLO I Prebmwic" for Profit "iln
5
4.5
4
3
i
I
i
I
2.51
2
1.5
41 456-
5
10
15
20
25
30
Percent Profit Margin
35
40
45
50
Figure 40. Value function for FLB 1 for profit
Table 53. FLB 1 responses for profit
Profit margin (%)
0
5
10
20
50
Value
1
4
4
5
5
105
FLO 2 Prebmence. for Profit Margin
5
I II
I
I I
I
4.51
4
3.5 I
LhL
3
Il
2.51
2
1.5
I
Y
10
20
30
50
60
40
Percent Profit Margin
70
80
90
100
Figure 41. Value function for FLB 2 for profit
Table 54. FLB 2 responses for profit
Profit margin (%)
0
33
100
5.4.6
Value
1
3
5
Relationship with Donor Organization
Relationship with donor organization was defined as the extent to which the FLB
interacts positively with a non-profit or NGO organization, like DevOrg. The value
functions are shaped differently for the two FLBs, with FLB 1 (Figure 42) exhibiting
an inflection point at a medium relationship and FLB 2's value function (Figure 43)
being linear.
106
5
FLO I Preferences for Roabonship with Donor Organization
4.5
4
3.5
3
2.5
2
1
1.5
2
3.5
3
25
Relationship Low (1) through High (5)
Figure 42. Value function for FLB
4
45
5
1 for relationship
Table 55. FLB 1 responses for relationship with donor
Relationship with Donor
Low
Medium
High
Value
1
2
5
107
FLD 2 Preferences br RelalonuhIp with Donor (~gmu~zatIon
5
'F
'F
4.5
'F
'F
4
3.51"F
Nc
LL
3
A
2.5 I.
2
'F
1.5
'F
7
1
F
I
I
I
I
I
I
1.5
2
2.5
3
3.5
4
Relationship Low (1) throug High (5)
Figure 43. Value function for FLB 2 for relationship
Table 56. FLB 2 responses for relationship with donor
Relationship with Donor
Low
Medium
High
5.4.7
Value
1
3
5
Percent Increase in Sales
Percent increase in sales was a criterion that only came up in discussions with FLB
2. This is essentially cross sales, where by selling mRDTs, the sales of other products
that the FLB stocks will also increase. FLB 2's value function demonstrates an
108
inflection point at 30% increase in sales, before which the value increases more
sharply and after which increases less sharply.
FLB 2 Preferences IDr Precent k~creas. In Sale.
5
4,5
/
4
/
3.5
,.
/
3
2.5
2
/
Is1
I
1
a
a
a
a
5
10
15
20
a
a
a
a
a
25
30
35
40
45
Increase in sales (%)
50
Figure 44. Value function for FLB 2 for percent increase in sales
Table
57. FLB 2 responses for percent increase in sales
Percent Increase in Sales (%)
0
30
50
6
Value
1
4
5
Results - Weights
This section describes the solicited and normalized weights for the different agents.
109
6.1
Retailers
The relative preferences of retailers for each of the attributes were derived by the
swing weights procedure (see Methodology section). Each individual retailer
recorded their values and the median of the responses was calculated. No answers
were excluded from the analysis, but some retailers did not provide a response,
either at all to the weighting questions or to one of the criteria. For weighting on
quality, only the first focus group provided information because the second group
informed us that quality was not important to them (see Methodology section). For
opportunities and time to complete a sale, only the second focus group discussion
was included because those attributes did not come up for the first focus group.
Table 58 summarizes the number of responses about weight for each of the criteria.
Table 58. criteria and Number of Responses
Criteria
Quality
Price
Delivery time
Sales of other products
Customer satisfaction
Training
Volume
Awareness/Advertising
Cost
Time
Opportunities
Number of responses
16
26
25
25
25
26
26
23
26
10
10
The weights were solicited from a range of 0-100. Each individual was asked to set
at least one criteria with the highest value of 100. Then, the median of the responses
was calculated and then normalized over the sum of the weight values (Table 59). It
was noticed that there was not a wide spread in the weights, see the Discussion
section for biases that may have influenced these results.
110
Table 59. Median and normalized weights
Criteria
Quality
Price
Delivery time
Sales of other products
Customer satisfaction
Training
Volume sold
Awareness
Cost
Time to complete sale
Opportunities
6.2
Median response
75
75
80
70
90
90
62
80
70
87.5
65
Normalized weight
9.0%
9.0%
9.6%
8.4%
9.6%
10.8%
7.4%
9.6%
8.4%
10.5%
7.8%
Distributors
The relative preferences of distributors for each of the attributes were derived by
the swing weights procedure (see Methodology section). Each distributor was asked
to provide a numerical score of 0-100 for each of the criteria identified. The
responses per distributor were normalized based on the sum of the responses. As
was similar to the retailer, the spread of weights was not very large. See Discussion
section for consideration of biases and suggestions for future improvements in the
data collection.
Table 60. Weights for Distributor 1
Criteria
Volume
Expiration date
Profit margin
Cost per kit
Cost per training
Efficiency of distribution
Cross selling
Response
100
90
85
100
70
90
70
Normalized weight
16.5%
14.9%
14%
16.5%
11.6%
14.9%
11.6%
111
Table 61. Weights for Distributor 2
Criteria
Volume
Expiration date
Profit margin
Cost per kit
Cost per training
Efficiency of distribution
Cross selling
6.3
Response
100
95
80
80
90
90
80
Normalized weight
16.3%
15.4%
13.0%
14.6%
14.6%
14.6%
13.0%
First Line Buyers
The relative preferences of distributors for each of the FLBs were derived by the
swing weights procedure (see Methodology section), similarly to the distributors.
Each FLB was asked to provide a numerical score of 0-100 for each of the criteria
identified. The responses per FLB were normalized based on the sum of the
responses. Interestingly, the spread for the FLBs was much greater than for the
distributors and retailers. Further, there are some significant differences in the
weights for some of the criteria between the FLBs (see Discussion section).
Table 62. Weights for FLB 1
Criteria
Cost per device
Price per device
Quality
Administrative time
Profit
Relationship with donor
Response
30
100
90
90
20
100
Normalized Weight
6.98%
23.3%
20.9%
20.9%
4.65%
23.3%
112
Table 63. Weights for FLB 2
Criteria
Cost per device
Price per device
Quality
Administrative time
Profit
Relationship with donor
Percent increase in sales
7
Response
100
70
90
10
50
55
80
Normalized Weight
22.0%
15.4%
19.8%
2.2%
11.0%
12.1%
17.6%
Results - Overall Value
Multi-criteria decision analysis can be used to derive an overall value for each of the
decision options based on the value functions and weights. A weighted sum is set
up:
Overall Value = v1 * w1 + v2 * W2 + --- + vn *wn
Where v, is the value for criteria 1 and w, is the weight for criteria 1, etc.
Theoretically, the option with the highest overall value is the best decision choice
for that individual, taking into account their preferences and priorities. An overall
value was calculated for the agents in the mRDT supply chain and a sensitivity
analysis varying weights and input assumptions was also performed to determine
robustness of results. This section provides details about the results of the overall
value analysis and calculations.
7.1
Overall Value - Retailers
The calculated overall value was based on the equation:
Overall Value Retailers
=
+
V quality * Wquality
+ Vprice
Vother sales * Wother sales
*
Wtraining
*
Wcost
+ Vvolume
+
* Wprice
+ Vdelivery time
* Wdelivery time
Vcustomer sat. * Wcustomer sat.
* Wvolume
+
Vtraining
+ Vawareness * Wawareness + Vcost
+ Vtime sale * Wtime sale + Vopportunities * Wopportunities
113
See Results - Value Function and Weights on how the value and weights were
calculated. The overall value for the base, high, and low-end input estimates is
summarized in Table 64.
Table 64. Overall value for Retailer Options (bold values are the highest value in the subset)
Overall Value - High
Overall Value - Base
Overall Value - Low
3.45
4.19
2.47
1.57
2.74
3.78
1.85
1.57
2.27
3.13
1.69
1.57
Option 1
Option 2
Option 3
Option 4
Option 2, which is to sell the mRDTs in the bundled service ranks as the best option
across the varying input estimations.
7.2
Overall Value - Distributors
The overall value for each distributor was calculated based on their individual value
functions and weights. The equation for overall value was based on the equation:
Overall Value Distributor
~ Vvolume * Wvolume
+
Vcost kit
*
Wcost kit
+ Vexp. date
*
Wexp. date
+ Vprof it
* Wprof it
+ Vcost train. * Wcost train. + Vdistribution
* Wdistribution + Vcross sales * Wcross sales
The overall value outcomes for each of the distributors are summarized in Table 65
and Table 66. Across the estimates, Option 1 is consistently the best option for the
distributors. Option 3 is often a close second in overall value. The consistency of
results across different estimates suggests that the results are robust.
114
Table 65. Overall Value for Distributor 1 Options (bold values are the highest value in the subset)
Overall Value - High
Overall Value - Base
Overall Value - Low
3.00
2.77
2.98
1.83
2.61
2.12
2.58
1.83
2.29
1.90
2.19
1.83
Option 1
Option 2
Option 3
Option 4
Table 66. Overall Value for Distributor 2 Options (bold values are the highest in the subset)
Overall Value - High
Overall Value - Base
Overall Value - Low
3.06
2.54
2.92
1.82
2.66
1.92
2.55
1.82
2.24
1.78
2.15
1.82
Option 1
Option 2
Option 3
Option 4
Attribute metrics are supposed to be preferentially independent from one another,
otherwise one may receive a disproportional emphasis in the final calculation.
When the criteria were initially generated before the data collection in Uganda,
profit was not included; rather, costs, prices, and volumes were specifically listed.
However, in conversations, the distributors insisted that profit margin on the device
was one of their important considerations in making this decision. This implied that
the distributors thought in terms of profit percent and thus, this criterion was
included in our analysis. Cost and profit were both included in the analysis.
However, in addition, a second analysis was performed to determine the overall
value when cost was removed as a criterion. This change preserves the ordinal rank
of the first two choices, with Option 1 being the best decision, followed by Option 3,
for distributors. However, the difference in value between Options 1 and 3 becomes
much less than previously calculated. Further, the Option 2 decreases drastically in
value, as the subsidized prices of the kits were contributing to a large amount of
value for that option.
115
Table 67. Overall Value for Distributor 1 Options - Cost per kit is removed (bold values are the highest
value in the subset)
Overall Value - High
Overall Value - Base
Overall Value - Low
Option 1
2.83
2.39
2.03
Option 2
2.11
1.43
1.18
Option 3
2.81
2.37
1.86
Option 4
1.83
1.83
1.83
Table 68. Overall Value for Distributor 2 Options - Cost per kit is removed (bold values are the highest
value in the subset)
Overall Value - High
Overall Value - Base
Overall Value - Low
Option 1
2.90
2.45
2
Option 2
1.95
1.30
1.12
Option 3
2.76
2.34
1.84
Option 4
1
1
1
7.3
Overall Value - First Line Buyers
The overall value for each FLB was calculated based on their individual value
functions and weights. The equation for overall value was based on the equation:
Overall Value FLB
Vcost * Wcost
*
+
Wadmin time
+ Vprice
+ Vprof it
*
Wprice
* Wprof it
+ Vquality
* Wquality
+ Vadmin time
+ Vrelationship * Wrelationship
Vincrease sales * Wincrease sales
116
The overall value outcomes for each of the FLBs are summarized in Table 69 and
Table 70. For FLB 1, the high and base estimates reveal that Option 2 is the
preferred choice, while for the low value estimates, Option 1 is higher. For FLB 2, the
preference is consistently Option 2.
Table 69. Overall Value for FLB 1 (bold values are the highest value in the subset)
Overall Value - High
Overall Value - Base
Overall Value - Low
Option 1
3.33
2.63
3.01
Option 2
3.85
3.44
2.74
Option 3
2.02
2.40
2.71
Option 4
1.91
1.91
1.91
Table 70. Overall Value for FLB 2 (bold values are the highest value in the subset)
Overall Value - High
Overall Value - Base
Overall Value - Low
Option 1
3.66
3.10
2.49
Option 2
4.14
3.34
3.03
Option 3
2.36
2.18
1.79
Option 4
1.77
1.77
1.77
Similarly for distributors, profit is comprised of price and cost. It was included in the
analysis because the FLBs wanted to include it. However, as a comparison, the
analysis was run again removing profit and retaining price and cost. The result of
this analysis is summarized in Table 71 and Table 72. The same exact trends as with
profit included are found. That is, the best option for FLB 2 is consistently Option 2
and the best for FLB 1 is Option 2 for high and base estimates and Option 1 for lowend estimates.
117
Table 71. Overall value for FLB 1 Options - Profit removed (bold values are the highest value in the
subset)
Overall Value - High
Overall Value - Base
Overall Value - Low
Option 1
3.09
2.40
2.77
Option 2
3.62
3.20
2.50
Option 3
1.79
2.16
2.47
Option 4
1.86
1.86
1.86
Table 72. Overall Value for FLB 2 Options - Profit removed (bold values are the highest value in the
subset)
Overall Value - High
Overall Value - Base
Overall Value - Low
Option 1
3.33
2.77
2.38
Option 2
3.81
3.23
2.92
Option 3
2.04
1.85
1.68
Option 4
1.66
1.86
1.66
7.4
Overall Value - Comparison Between Agents
This section presents a scatter plot of the overall value of the different agents to
compare between them. Figure 45 compares the overall value of the retailers
versus the two distributors. Options 1 and 2 are more on the upper right side of the
graph, indicative of the fact that they have a higher value than the other two options.
At the lower left side of the graph, the two distributors have highly overlapping
values, but a clear separation emerges in the upper right quadrant. Especially for
Option 2 high-end estimates, there is a marked difference in value between the two
distributors.
118
Distributors vs. Retailer Overall value
3.5
Opt. 1 High
Opt. 3 High
3
Opt.3 Base
OpOp.
Oppt.
2.5
Opt2Hgh
I Base
1 Base
3 Low
OOpt.
Opt. 3 LowLow
2
Opt. 2 Base
Opt. 4 Base/Low/High
1.5
Opt. 2 Low
[Dist1
f1
10lDist 2
0.5
0
0
1
2
3
4
5
Retailer Overall Value
Figure 45.Distributor versus retailer overall value for all of the high/base/low options 1-4
Figure 45 compares the FLB and distributor value versus the retailer value. Since
retailers are plotted on the x-axis, clear columns emerge where the relative
difference in values for the FLBs and distributors can be examined. Similarly to the
pattern in the other graph (Figure 45), greater differences between the agents
emerge as the overall value of retailers increases. Again, Option 2 high-end
estimates reveals a large difference between FLBs and distributors; the distributors
have a much lower value than the FLBs.
119
FLB/Distributor vs Retailer Overall Value
f..!W
4.00
050
0 Fs F1I
Iriv I
W
ID:
S050
0
0.0
2
1.5
Z
.
3
ISf
4
4_
3#qalt Ov"Orf VOue
Figure 46. FLB/Distributor overall value versus Retailer overall value for all of the high/base/low
options 1-4
8 Sensitivity Analysis
It is important to understand the robustness of the results that were generated. Two
sensitivity analyses were conducted. The first was by varying the input assumptions,
having a high and low-end estimate in addition to the base numbers. This analysis
was embedded in the overall value calculations and is not included in this section.
This section specifically describes a sensitivity analysis on the weighting. The
weights for each of the criteria were varied from 0% to 100% and the overall value
was determined. As described in the methodology section, two points were plotted:
the point where the criteria's weight was the same as what was determined by the
agent. The overall value for this point, therefore, is the calculated overall value. The
second point where the weight was 100%, where the overall value was equal to the
partial value as derived from the value function. A weight sensitivity analysis was
conducted on the distributors and FLBs. Retailers were not included in the weight
sensitivity analysis because the variation between the weights for the retailer
criteria was very narrow.
120
8.1
Distributors
Weight sensitivity analysis was conducted individually for each of the distributors
for all of the criteria.
8.1.1
Volume
The weight for volume for Distributors 1 and 2 were varied from having a weight of
0% to 100% (Figure 47 and Figure 48). For Distributor 1, at the current weight of
16.5%, Option 1 is the best choice at 2.61 value (followed very closely by Option 3 at
2.58). However, as the weight of volume increases, Option 2 becomes the best
choice. At approximately 65% weight on volume, the best option switches from 1 to
2. This is because even though the distributors are not selling the volumes that they
would like in Option 2, they are still selling more than in Option 1.
Distributor 2 follows a similar trend as Distributor 1, in that at about 50% weight,
the best option switches from Option 1 to Option 2. Further, for Distributor 2, it
seems like Option 4 becomes a better option as the weight of volume becomes
almost 0%.
121
Overall Value vs. Weight - Volume - Dist. 1
,.00
4.S0
400
Opt.1and3
300
2.SO
2.00
pt. 4p
1.O
100
100
SQ
60
40
20
0
120
WeSht for Vokwi
Figure 47. Overall value versus weight for volume for distributor 1. The dashed line marks where the
options intersect.
Overall Value vs. Weight - Volume - Dist. 2
300
Opt. i
2.80
Opt. 3
260
2,40
Opt. 2
10
Opt. 4
1.40
1.20
1.00
0
20
20
0
40
50
60
0
so
90
100
Weght for Voiawe
Figure 48. Overall value versus weight for volume for distributor 2. The dashed line marks where the
options intersect.
122
8.1.2
Expiration Dates
The sensitivity analysis of the weights of expiration dates for Distributor 1 (Figure
49) and Distributor 2 (Table 50) reveals that the best choices for both distributors
do not change as the weight varies. In fact, the top choice gets even better. For
Distributor 1, Options 1 and 3 remain almost identical in value across weight, but
for Distributor 2, the values diverge at low weights.
Overall Value vs. Weight - Expiration Date - Dist. 1
5.00
4.50
4.00
.. 50
3 00
Opt 1 and 3
p.2
1,50
200
100
000
0
10
20
30
40
WOIO
0
s
or Expraefln De
70
80
90
100
Figure 49. Overall value versus weight for expiration dates for distributor 1
123
Overall Value vs. Weight- Expiration Date - Dist. 2
3 00
Opt. I
2
50
0 pt.
3
7I00
1.00
000
S
10
20
30
40
Weight
fCr
so
60
EXp*0a0" Oatft
70
g
9o
100
Figure 50. Overall value versus weight for expiration dates for distributor 2
8.1.3
Profit Margin
The sensitivity analysis on weights for profit margin for Distributor 1 (Figure 51)
and Distributor 2 (Figure 52) reveals that the order of preferences does not change
as weight changes. Like for expiration date, the relative difference between the
options increases as weight for profit margin increases.
124
-
Overall Value vs. Weight - Profit Margin
Dist. 1
A.50
too
Opt. 4
coo
)a
0
30
70
40
Weighs
SOealV le s
50
60
70
so
100
90
fm Proft Margn
egt PrftM ri t.1 D2t
Figure 51. Overall value versus weight for profit margin for distributor
.50
Overall Value vs. Weight - Profit Margon - Dist. 2
0.0
4,00
3.So
J.00
Opt. I
i
so
Opt. 3
2.00
000
Opt.
0
!0
20
30
fic
50
wpwghs Icr PrOfit Mattn
40
7D
00
90
10C
Figure 52. Overall value versus weight for profit margin for distributor 2
125
Cost per Kit
8.1.4
The weight sensitivity analysis reveals an intersection point where the best option
changes from Option 1 or 3 to Option 4. For both Distributor 1 (Figure 53) and
Distributor 2 (Figure 54) this point occurs at approximately 30% weight on cost per
kit. After this point, Option 4 becomes the preferred option. This is because the cost
of Option 4 is $0, as kits are not being purchased. Thus, value increases as the cost
goes to zero.
Overall Value vs. Weight - Cost per Kit - Dist. 1
6.00
500Opt
4
400
Opt. 2
Opt. I and 3
>
000
0
70
40
60
so
100
170
W*htW W Coo pot Kit
Figure
53. Overall value versus weight for cost per kit for distributor 1. The dashed line marks where
the options intersect.
126
Overall Value vs. Weight - Cost per Kit - Dist. 2
b.00
otni
t.
S00
1.00
opt. 1
c o
0
20
40
W04ght
so
60
o COost of1
Too
170
Figure 54. Overall value versus weight for cost per kit for distributor 2. The dashed line marks where the
options intersect.
8.1.5
Cost per Training
Similar to cost per kit, as cost for training decreases, the value increase. Thus, in the
weight sensitivity analysis, for both distributors, there is a point where the best
option goes towards Option 4, as there is no cost for training in Option 4. For
Distributor 1 (Figure 55), the value of Option 4 intersects with Option 2 at 20%
weight, after which Option 4 becomes the better option. However, for Distributor 1,
Options 1 and 3 are always better except at 100% weight for cost for training, where
Options 1, 3, and 4 are all the same. Distributor 2 (Figure 56) exhibits a change in
the best option when the weight for cost per training is 80%. After 80%, the best
option switches from Options 3 and 4 to Option 1.
127
Overall Value vs. Weight - Cost for Training
-Dist. 1
4.S0
4.00
Opt. 1 and 3
2.CIU
Opt. 2
150
Opt.4
100
050
0(X
30
?0
10
0
50
40
70
60
so
9W
100
wows fo cos for Ttrolpae
Figure 55. Overall value versus weight for cost for training for distributor 1. The dashed line marks
where the options intersect.
-
Overall Value vs. Weight - Cost for Training
Dist. 2
5,00
4.50
4.00
*3,50
Opt 3 and 4
2.so
Opt 2
Opt. 1
2.00
150
100
0.00
0
10
20
30
40
so
Wewits far Cost for
60
MV
so
90
100
Tralrng
Figure 56. Overall value versus weight for cost for training for distributor 2. The dashed line marks
where the options intersect.
128
8.1.6
Efficiency of Distribution
Sensitivity analysis for efficiency of distribution suggests that there is no change of
the best option as weights increase for both distributors (Figure 57 and Figure 58).
Distributor 2 indicates a divergence of Options 1 and 3 when weights decrease less
than 14%.
Overall Value vs. Weight - Efficiency of
Distribution - Dist. 1
50
Opt. and 3
A.00
Opt.2
ISO
1 00
0-so
000
0
70
40
60
80
100
Welgigs fo Ewemy of Domtrobuton IbMOes/d**"ry}
120
Figure 57. Overall value versus weight for cost for efficiency of distribution for distributor 1.
129
Overall Value vs. Weight - Efficiency of
Distribution - Dist. 2
ISo
Opt. 2
1.00
o0o
60
Woighttt Mr Efkfeucy of DItruo
0
(b
120
too
Be
40
70
W/dieky)
Figure 58. Overall value versus weight for cost for efficiency of distribution for distributor2.
8.1.7
Cross Selling
Sensitivity analysis for cross selling reveals no change in the best choice for either
distributor (Figure 59 and Figure 60). There is an increase in the relative vales for
each of the options as the weight increases.
-
Overall Value vs. Weight - Cross Selling
Dist.1
4.00
opt. 1
3 SO
300
1250
1.opt.
4
coo
0
10
?0
30
40
Welhgt
o
so
70
so
90
100
o CrO% SIng
Figure 59. Overall value versus weight for cross selling for distributor 1.
130
-
Overall Value vs. Weight - Cross Seliling
Dist. 2
ai oc
Opt 1
Opt. 3
Opt. 2
opt. 4
too
oo
0
10
20
30
40
Woots
so
60
70
80
90
too
for Cross so"
Figure 60. Overall value versus weight for cross selling for distributor 2
8.2
First Line Buyers
A weight sensitivity analysis was conducted individually on both FLBs for each of
the criteria.
8.2.1
Cost per Kit
A sensitivity analysis on cost per kit for the FLBs reveals that as the weight on cost
per kit increases, the best option varies. For both FLBs, when this attribute has the
greatest weight, Option 4 becomes the best choice, because for Option 4 the cost is
$0 because no kits are being purchased. For FLB 1 (Figure 61) as the weight
increases past 20%, Option 2 is still the best choice, but Option 4 becomes the
second best choice. At a weight of about 83%, Option 4 becomes the best choice for
FLB 1. For FLB 2 (Figure 62), at about 75% weight, Option 4 becomes the best
choice.
131
Overall Value vs. Weight - Cost per Kit - FLB 1
4
.5s
Opt. 2
4
pt. 4
3
005
Opt. 3
10
30
70
50
40
W.*gt
60
90
W0
70
100
fow Coo. per K"
Figure 61. Overall value versus weight for cost per kit for FLB 1. The dashed line marks where the
options intersect.
Overall Value vs. Weght- Cost per Kit - FLB 2
'I
Opt. 4
45
~~~~~Oot.
4
3
...........
OptI
1"Opt.3
a
10
20
30
40
W#4ghM
60
50
fWr C*St P"r 10
70
so
90
100
Figure 62. Overall value versus weight for cost per kit for FLB 2. The dashed line marks where the
options intersect.
132
8.2.2
Price per Kit
The price per kit sensitivity analysis finds that the best choice does not change when
the weight is varied for both FLB 1 and 2 (Figure 63 and Figure 64). At the highest
weight, Options 1, 2, and 3 converge.
Overall Value vs. Weight - Price per Kit - FLB 1
45
4
Opt 2
OptI
I
..... .......... ..................O p t. 3
2
1
Opt. 4
0.5
0
0
10
70
30
40
50
WVioM for Pr1
60
70
80
90
100
pot Kkt
Figure 63. Overall value versus weight for price per kit for FLB 1.
133
Overall Value vs. Weight - Price per Kit - FLB 2
45
4
Opt. 2
5
0.S
0~
0
10
20
10
40
so
60
70
so
90
100
WWCgt kr Price fot Kki
Figure 64. Overall value versus weight for price per kit for FLB 2.
8.2.3 Quality
The sensitivity analysis for quality reveals that for both FLBs, as the weight of
quality increases so does the value of the options, while retaining the same order
(Figure 65 and Figure 66).
134
Overall Value vs. Weight - Quality - FLB 1
S
Opt. 2
Opt. 1
Opt. 3
0
70
40
60
80
100
120
W.4ght for Qtwtity
Figure 65. Overall value versus weight for quality for FLB 1.
Overall Value vs. Weight - Quality - FLB 2
Opt. 2
........ .. .......
O p t. I
Opt. 3
0
0
20
40
60
80
100
120
W#Igtm tor Qulity
Figure 66. Overall value versus weight for quality for FLB 2
135
8.2.4
Administrative Time
As the weight on administrative time changes, the best option for both FLBs changes
dramatically. For FLB 1 (Figure 67), the order of the best options inverses, and
Option 4 becomes the best option, as there is no administrative time at all. FLB 2
(Figure 67) reveals that at 100% weight, Option 4 is also the best option, but
Options 1 and 3 are the same value at 4.
For FLB 2, Option 1 replaces Option 2 as the best choice at approximately 8% weight
(compared to the current 2.2% weight). At approximately 63%, Option 4 increases
in value to be greater than Option 1. Thus, these particular results are somewhat
sensitive to weight on administrative time. However, an increase from 2.2 to 8% is
still more than doubling, implying that while administrative time is more sensitive
than other results, it is still relatively robust.
136
Overall Value vs. Weight - Administrative Time
- FLB 1
Opt 4
s
CS
Opt. 3
4
Opt. 1
Opt. 2
0s
0
0
10
20
30
60
50
40
WIgt tot Admhbtraft#v TkW*
70
100
90
so
Figure 67. Overall value versus weight for administrative time for FLB 1. The dashed line marks where
the options intersect.
-
Overall Value vs. Weight - Administrative Time
FLB2
Opt.4
45
4
O pt1.........
1
Opt.2
A
0~O
0
to
30
40
Wogha
50
60
70
80
90
100
for Adminitratiw Thye
Figure 68. Overall value versus weight for administrative time for FLB 2. The dashed line marks where
the options intersect.
137
8.2.5
Profit Margin
For FLB 1 (Figure 69) the relative order of the best choice does not change, although
Options 1, 2, and 3 converge at a value of 5 at 100% weight. FLB 2 (Figure 70),
however, does exhibit a change in the order; at approximately 15% weight, Option 1
overtakes Option 2 as the best choice. Option 3 increases as well to converge with
Option 1 at value of 5 at 100% weight.
Overall Value vs. Weight - Profit Margin - FLB 1
45
4
Opt. 2
OpOpt.
Opt. 3
12.1,
105
0
to
70
30
40
50
60
70
80
90
100
Weight for Proft Margio
Figure 69. Overall value versus weight for profit margin for FLB 1.
138
Overall Value vs. Weight - Profit Margin
4
-
FLB 2
Opt 2
35Op1
Optp..3
Opt. 4
0
0
10
0
40
50
WOgM ow Po
60
70
80
90
100
Moron
Figure 70. Overall value versus weight for profit margin for FLB 2. The dashed line marks where the
options intersect.
8.2.6
Relationship with Donor
As the weight on relationship with donor increases, the order of the best results
does not change for the FLBs, but the relative value difference between the options
does increase (Figure 71 and Figure 72). For both FLBs, Options 3 and 4 converge at
100% weight at the lowest value of 1.
139
Overall Value vs. Weight - Relationship with
Donor - FLB 1
S
4.5
Opt 1
4
.3
3.S
3
25
2
Opt. 1
SOt.3
0
10
20
30
40
50
60
70
100
90
xO
W440K for ReatloRSNp wkh Donor
Figure 71. Overall value versus weight for relationship with donor for FLB 1.
Overall Value vs. Weight - Relationship with
Donor - FLB 2
5
45
Opt 2
4
I
Ot
Opt. 3
2
opt. 4
0.S
0
0
10
20
30
40
50
60
t0
80
90
100
WeigM for Rladanshp wth Donor
Figure 72. Overall value versus weight for relationship with donor for FLB 2.
140
8.2.7
Increase in Sales
Sensitivity analysis for increase in sales was plotted for FLB 2 only because FLB 1
did not have this as a criterion. FLB 2 exhibits a change in the best option from
Option 2 to Option 1 at a weight of about 41% (Figure 73).
-
Overall Value vs. Weight - Increase in Sales
FLB2
4.S
4
0
10
20
30
40
WOWt for
50
60
Incres" in Saw
70
80
90
100
Figure 73. Overall value versus weight for increase in sales for FLB 2. The dashed line marks where the
options intersect.
9
Discussion
The discussion section provides a commentary on the key results from the analysis
and also insights from interviews and conversations with the different agents. It
further notes limitations of the study and suggestions for improvement in further
work.
9.1
Result from Analysis
The purpose of this study was to understand the decision-making process,
preferences, and priorities of agents in the private sector supply chain for malaria
141
rapid diagnostic tests (mRDTs) in Uganda. The different agents were asked about a
decision process that included for options:
1. Sell WHO approved mRDTs outside of DevOrg's bundled service
2. Sell mRDTs through DevOrg's bundled service
3. Sell non-WHO approved mRDTs
4. Do not sell mRDTs
The analysis in this study was framed around these four options.
9.1.1
Best Options for Agents
The best options and overall value, based on value functions and weights, for the
three different agents is summarized in Table 73. The option with the highest
overall value for retailers was Option 2, selling the mRDTs through DevOrg's
bundled service. However, the best option for distributors was Option 1, which was
to sell WHO approved devices outside of the bundled service. The best option for the
FLBs was the same as the retailers, Option 2. Even varying input estimations, the
ordinal rank of the best option remains the same, suggesting robustness in the
results. The only variation in results is the low-end assumption for FLB 1, where
Option 1 becomes the best option.
Table 73.comparison of overall value between different agents in the supply chain (using the baseline
assumptions)
First Line Buyers*
Distributors*
Retailers
Option 1
2.86
2.63
2.74
Option 2
3.39
2.02
3.78
Option 3
2.29
2.56
1.85
Option 4
1.84
1.83
1.57
*The average of the two distributors and two FLBs were taken to be included in this table.
142
It is very interesting to compare the best options between the different agents in the
supply chain (Table 73). The best option for the retailers is to sell the mRDT through
the bundled service. One key factor in that may be the high value that retailers
placed on training, which was a key component in the bundle.
Distributors exhibit a marked difference between Option 2 and Option 1, which had
the highest overall value for them. This dichotomy of results aligns with the
qualitative feedback that individuals expressed during interviews and focus group
discussions. Retailers generally found a great deal of value in Option 2, while
distributors expressed reservations and concerns.
FLBs tend to prefer Option 2, but under the low-end estimates of input values, FLB 1
revealed that Option 1 was the best choice. This result could be strongly influenced
by the importance of administrative time. For Option 2, FLB 2 indicated that they
would greatly prefer a lower administrative time. The low-end estimate of times in
FLB 1's value function results in Option 1 being a 5 and Option 2 being a 1, likely
influencing the final overall value.
9.1.2
Sensitivity Analysis
A sensitivity analysis was performed on the weights and input assumptions. There
was inherent uncertainty in the input assumptions because of limited data in the
research context. To account for this uncertainty, a high and low-end estimation was
generated based on interviews, expert opinions, and literature (see Inputs and
Assumptions section). The results of this variation finds that the overall value
outcomes are very consistent. The only result that changed was for FLB 1, where for
the low-end estimates, the best option changed from Option 2 to Option 1.
A sensitivity analysis was further performed on the weights. The weights of each of
the attributes were varied from 0-100% to determine the impact on overall value.
This analysis was performed on FLBs and distributors only, because the variation of
weights of the retailers was so narrow. The sensitivity analysis finds that the results
143
are quite robust. For many attributes, varying the weights across the entire range
does not change the best option. For other attribtues, the best option does change,
but almost always changes with an increase of at least 20% points. Thus, the results
in this analysis are fairly robust.
9.1.3
Implications and Improving Options
These results have interesting policy implications. Many public health interventions
have a wide range of different independent stakeholders who all have to be brought
on board to realize positive change. This section highlights changes that could be
made or considerations that should be taken into consideration to harmonize value
across the supply chain.
9.1.3.1
Risk Sharing Across the Supply Chain
In the supply chain that was part of this research study, the distributors owned the
stock. Kits were procured through a bulk order, imported and initially owned by the
FLB, and then sold to the distributor. The distributor then stores the product in
warehouses. At this point, the distributor has paid the full cost of the product and is
the entity that stands to lose any investment made when unsold kits expire. The
FLBs are invested in supporting the distributors, but they will not face financial
consequences if product is not sold. Further, it is clear from conversations that
retailers perceive much less of a risk; they did not once mention unsold stock being
a concern or barrier to uptake of mRDTs. The distributors we spoke with all
expressed stress and great concern about not being able to sell the products and
then losing a significant amount of investment. The distributor concern about
volume is reflected in the outputs of their value functions. Option 2 for the criteria
volume for the distributors ranks less than a 2 on a scale of 1-5. Increasing the
volume of sales would significantly increase the overall value of Option 2 for
distributors, especially because volume has the greatest weight of all the attributes.
If the volume sold for Distributor 1 increases from 1,600 kits sold per month to
9,000 kits sold per month, then value increases from 1.36 to 3, and then the overall
value would increase from 2.12 to 2.39 (12.7% increase); for Distributor 2 if the
value increases from 1.98 to 3, the overall value would increase from 1.92 to 2.08
144
(8.3% increase). While this does not change the overall rank of options, it makes
Option 2 much more viable than previously.
One means of reducing the burden of unsold volume would be to distribute risk
along the supply chain. Supply chain literature has many examples of risk mitigation
strategies (Tang, 2006) that could be modified for the context of Africa. One example
that might work in this setting is to set up a contract between manufacturers, FLBs,
and distributors with a buy-back component. That is, the distributor will be at least
partially refunded for any unsold products at the end of the expiry period. This
would serve to incentivize FLBs and manufacturers to assist in driving sales and
moving product. It would be further worth considering how to engage retailers in
risk sharing. While these types of contracts are often set up with retailers in other
supply chains, it is not clear that such a scheme would work in this setting.
Individual retailers move such a small amount of stock comparatively and refunding
a large number of small drug shops for relatively little stock would be costly in
terms of both time and finances. It is worth researching and piloting other risk
sharing mechanisms with retailers. Regardless, sharing the risk between FLBs and
distributors would be greatly valued by the distributors and would likely increase
the chances that distributors are willing to continue stocking the devices and engage
in public health initiatives like this.
9.1.3.2
Demand forecasting
Demand forecasting is an important process to any business, and especially so for
manufacturers and other upper level supply chain agents, as the scale of operations
increases. Especially in the context of expiration dates, demand forecasting is critical
because stock that expires is a direct loss to the investor. Concern about expiration
dates was an important factor that was brought up by FLBs and distributors. This
was compounded by the fact that the distributors had already committed and
purchased a large number of the devices. Demand forecasting has clear benefits in
the global health space. It ensures essential products are sustainably available, it
provides manufacturers with a better picture of the market so they can develop new
145
and improved products, it allows governments and funders to effectively allocate
resources, and it reveals bottlenecks where supply doesn't meet demand, which can
shape policy and advocacy efforts (Center for Global Development, 2007). A better
understanding preferences, through studies like this one, can help improve
understanding of the market such that forecasts can be more accurate. When
demand forecasting is accurate, interventions that require an upfront investment
are less likely to result in unsold stock for purchasers. This would be another
avenue by which to improve the value of the intervention option for distributors.
9.1.3.3 Leverage Expertise
As part of the contract DevOrg set up, the distributors are responsible for almost all
of the services associated with the mRDTs, from advertising to training to waste
management. In many ways, this was a prudent framework; the distributors are
close to the market and are therefore more likely to have a better understanding of
what works best and how to accomplish the requirements of the contract. On the
other hand, the distributors expressed to us that they have a limited amount of
expertise and time to accomplish all that is required of them. Two of the distributors
brought up the idea that an NGO, either DevOrg or another in the malaria space, be
responsible for training and some demand generation. The distributors asserted
that NGOs like DevOrg have the expertise in training people about malaria and
would be better suited for one-time investments in something like training.
Investments in training costs are quite high, and based on their value functions,
distributors have about a value of 2 out of 5 (not satisfied) for current costs.
Decreasing that burden on the distributors would result in a higher overall value for
this option. For example, if the cost per training was $100, then the value for
Distributor 1 is 4.82 and Distributor 2 is 4.78. The change in overall value for
Distributor 1 for Option 2 is from 2.12 to 2.41 and for Distributor 2 it increases from
1.92 to 2.26 (for the base assumption). This represents an increase of 13.7 and
17.7%, respectively, in overall value. While this doesn't change the options overall
rankings, it does bring Option 2 to a much higher level and closer in value to the top
options.
146
The distributors further expressed that they would be content to continue to
spearhead the waste management efforts, as they already travel to the stores on a
routine basis. Thus, there would be a division of responsibilities along the
stakeholders in the project, ideally aligning with relevant expertise.
9.1.3.4 The cost to the retailer goes beyond the device
Retailers expressed that stocking and administering an mRDT includes costs that
had not been adequately accounted for by the upstream supply chain. That is, to
administer an mRDT, the retailer had to factor in the cost of gloves, bandages to
cover the finger prick, etc. The retailers said that this drives up the cost of the mRDT
and it was necessary for them to correspondingly increase the price. It is worth
considering how to better manage the extra costs associated with administering the
mRDT. Although Option 2 was already the best option for the retailers, improving
this aspect would make the option even more attractive. Further, when considering
scaling up the project, cost factors, especially in the poorer, rural areas, will become
an even greater consideration.
9.1.4
Insights from Value Functions and Weighting
The overall value was only one aspect to this study and in addition to those results,
interesting insights were derived from the elicitation of value functions and
weighting. These two results themselves are useful in understanding the
preferences of different agents in the supply chain. This section highlights some
relevant results.
9.1.4.1
Retailers value the time it takes to complete a sale and extra opportunities
Many of the criteria, or fundamental objectives, that were used in the methodology
could be determined ahead of time through expert interviews and literature. For
example, cost, volume, customer satisfaction, etc. were determined before the focus
group discussions. However, during the focus group discussions moderators always
incorporated time to include any new criteria that could come up, that hadn't
occurred to the researchers. For retailers, it was possible to incorporate the
147
emergent criteria into the decision analysis. The second focus group discussion
brought up two new and unexpected criteria: extra opportunities and time it takes
to complete a sale.
Extra opportunities was a catch-all phrase that referred to tangible and intangible
benefits of working with a project or campaign. That is, retailers informed the
researchers that when an NGO or company was trying to promote a product or
initiative, they would often provide small opportunities for the retailers. These
could include free shirts, which was part of DevOrg's bundled service, or the chance
to work in a small role helping the project and being financially compensated. Other
opportunities include prizes for selling products, promotional material, and more.
Retailers expressed that these opportunities were a factor in their decision making
process about stocking mRDTs. The median weight for the criterion opportunities
was 7.79%.
Time it takes to complete a sale refers to the amount of time that the storekeeper
spends interacting with a customer to facilitate and complete a sale. The retailers
explained that they spend around 30 minutes selling an mRDT because they first
need to explain the product to the customer, convince him/her that testing was
better for their health, draw the blood, wait for results, and then advise action based
on test results (antimalarial or other treatment). While this was occurring, another
customer may come in, see that the shopkeeper was occupied, and leave to go down
the street to another shop. Thus, the storekeeper loses a sale while being
preoccupied selling an mRDT. Retailers weighted time it takes to complete a sale as
10.5%, which was second only to training. It was quite unexpected that time to
complete the sale was such an important criterion. This is a result that public health
specialists should consider when designing interventions to increase appropriate
diagnosis of malaria.
148
9.1.4.2
FLBs are not margin driven
The FLBs expressed that achieving high margins on these products were not their
priority. Margin was not initially in the research design to ask, rather the research
broke margin into questions about volume, price, and cost. However, FLBs indicated
that they thought in terms of margin percent, instead of price and cost, and so that
information was recorded. The FLBs we spoke with indicated that one of their
objectives was to provide high quality medical devices to the market in order to
improve public health. One FLB expressed that their company would absolutely not
consider selling low quality devices because that would go against their mission and
goals. Thus, they prioritized margin much less than other factors. In fact, FLB 1
reported that margin had a weight of only 4.65% and FLB 2 had a weight on profit of
11%. Both interviewees said that they would consider continuing to import product
as long as they did not lose money (although many also expressed reservations
about the complexity, and time requirement of public health campaigns). In the
interview, one FLB expressed that a increasing the profit margin to a sustainable
level was much more important than increasing the margin above that to net
greater profits.
One important takeaway about this result is that identifying good partner
companies that are interested in a social venture would best suit NGOs and
governments seeking to conduct public health campaigns. Although companies do
need to be concerned about profit, some companies, like the FLBs here, express that
other factors-like providing a high quality product-are of equal or greater
importance. Partnering with such organizations would represent an aligning of
values between the public health entity and the private sector partners on the
ground.
9.1.4.3
The importance ofcross selling
Cross selling, or the increase of sales of other products, that occur because of selling
the mRDT was something that emerged as an important benefit to selling mRDTs.
Both distributors and retailers expressed this as a criterion in their decision about
149
stocking the devices. Retailers reported a weight of 8.4% and distributors of 11.6%
for this attribute. In conversations, two drug shop owners reported that their sales
in general had more than doubled after introducing the mRDTs. It is unclear if these
individuals are an anomaly or if those figures are common across the spectrum of
retailers. One distributor reported that an increase in his other sales was 16% and
they was not especially pleased with that amount. In interviews, distributors have
an expectation that a new initiative would result in an expansion of their
distribution network and that was an important consideration when expanding into
a new product.
One consideration could be selling the mRDTs and ACTs together as part of a
comprehensive malaria service. That is, a customer can come to a drug shop and pay
a flat fee to be tested for malaria. If the result is positive, the customer receives the
ACT antimalarial at no extra cost; if the result is negative, the customer does not get
an ACT but may get other appropriate treatment. This would facilitate cross selling,
and thus may be perceived as valuable by the retailers. It would also encourage
testing before treatment. Cross selling of ACTs with the bundled service should be
investigated further.
9.1.4.4 Retailers strongly value training
The results from the weighting analysis find that retailers most prioritize training; it
was given a normalized weight of 10.8%. In conversations, retailers expressed that
they value the opportunity to provide another service to customers. Further,
retailers appreciated that they learned why diagnosis was important and could
convey that information to customers. This would likely increase sales and could
also conceivably contribute to the prestige/reputation of the retailer. Thus, one of
the key benefits of the bundled service for the retailers is the training scheme.
9.1.4.5 FLB disagreement about priorities
There were some significant differences in priorities between the two FLBs
interviewed. Specifically cost, administrative time, and relationship with donor
differed by at least 10%. Cost for FLB 1 was quite unimportant, scoring a 30 out of
150
100, or 6.89% (Table 74), while for FLB 2 cost scored the highest at 100 and 22%
(Table 75). It is not clear why there was such a divergence; it is possible that price
and cost were being considered dependently on each other and FLB 1 scored cost as
less important if price was the most important.
FLB 1 weighted administrative time as 90 out of 100 (20.9%) while FLB 2 weighted
it as 10 out of 100 (2.2%). FLB 1 had expressed frustration at the amount of time
that the project takes up and that they had to hire two full time staff to manage the
sales of the kits. FLB 2 said that they does not mind the amount of time the project
takes, albeit they reported that they spent much less time than FLB 1. It is possible
that the onus of administrative time was shared between FLB 1 and their distributor
while for FLB 2 the distributors bore that burden. Indeed, both distributors under
FLB 1 expressed that they spent a lot of time managing the sales of the mRDTs.
Table 74. Weights for FLB 1
Criteria
Cost per device
Price per device
Quality
Administrative time
Profit
Relationship with donor
Response
30
100
90
90
20
100
Normalized Weight
6.98%
23.3%
20.9%
20.9%
4.65%
23.3%
Response
100
70
90
10
50
55
80
Normalized Weight
22.0%
15.4%
19.8%
2.2%
11.0%
12.1%
17.6%
Table 75. Weights for FLB 2
Criteria
Cost per device
Price per device
Quality
Administrative time
Profit
Relationship with donor
Percent increase in sales
151
Finally, FLB 1 recorded relationship with the donor organization or NGO as 100
(23.3%) while FLB 2 reported donor relationship as 55 (12.1%). Both FLBs affirmed
that the relationship was indeed important, however they had ultimately had a
different emphasis.
9.1.5
Emergent Criteria
Some criteria were brought up in conversations with the different supply chain
agents and were not able to be incorporated in the analysis for a variety of reasons,
including inability to define an appropriate metric in the short amount of time
available (see Methodology). However, the researcher recorded these criteria, as
they are relevant to the individual's preferences. This section provides more
information on the emergent criteria.
9.1.5.1
FLB value gaining clarityon the market
Both FLBs articulated how they appreciated the opportunity to better understand
the market, customers, and product while participating in the bundled service. They
said that by participating in this project, they have had to work more closely with
distributors and retailers, which then brings them into conversations around selling
and marketing the products. The FLBs expressed that they valued learning more
about retailers and the issues faced on the ground and would use that information
to tailor their own sales. Further, it is information that can be passed along to the
manufacturer, especially as it provides a quick feedback mechanism if there is
something wrong with the devices.
9.1.5.2 FLBs value aestheticsof kit
One FLB mentioned that a relevant attribute in their decision was the design of the
packaging of the mRDTs. They said it was important for her considerations because
it is something the retailers take into consideration when selling products. The
rationale is that retailers can put a greater markup on a product that looks better
because consumers will think of it as higher quality. Interestingly, aesthetics did not
come up in either of the focus group discussions; however, that may have been
152
because retailers were implicitly including it in the criteria of quality. Aesthetics of
kit packaging was not included in FLB considerations for a few reasons. First, the
FLB articulated that they valued this because they believed it would lead directly to
increased uptake on the retailer end; thus, the criteria being considered was actually
volume. Additionally, in the short amount of time given for conversations,
researchers were unable to define a metric that would be appropriate as a
measurement for the criteria.
9.1.5.3
Retailer Incentives
Currently, the primary incentive retailers have to sell product is that they made an
upfront purchase of 25 mRDTs. Retailers expressed to the researchers that they are
accustomed to being provided with mechanisms to incentivize selling products, like
prizes for selling the most products or bulk discounts. This preference may have
been expressed in the opportunities criterion. One distributor did say that they
provided bulk discounts on retailers if they bought five or more boxes of mRDTs.
Thus, drawing from this information, one way to aid the scale up of medical devices
like mRDTs is to incorporate these incentive mechanisms at the retailer level.
9.2
Methodology
One of the key motivations for this research study was to pilot a multi criteria
decision analysis methodology to analyze and evaluate development supply chains.
This section highlights insights from the methodology, points out improvements for
future studies, and articulates the rationale for decisions that shaped the data
collection process.
9.2.1
Data Collection
MCDA methodology was adapted to facilitate data collection in the developing world
context. This section discusses the rationale behind the adapted methods,
limitations, and suggestions for future work.
153
9.2.1.1
Choosing the studypopulation
The sample in this research study was taken from individuals participating in a
bundled service intervention run by DevOrg. DevOrg is a non-profit organization
with a mission to combat malaria and other communicable diseases and improve
maternal and child health in Africa and Asia. DevOrg had put together a pilot
initiative to scale up access to mRDTs in the private sector supply chain in Uganda.
To this end, they constructed a bundled service that included training of retailers,
marketing, waste management, and more. The contract was made with the
manufacturers of WHO-approved devices and many of the requirements (waste
management, training, etc.) were executed by the distributors. An initial amount of
mRDTs were procured and introduced to the market; at the time of this research
study, DevOrg's project was entering its second year.
This research study took place in the context of DevOrg's bundled service. There
were several reasons for this. First, at this point, mRDTs are quite uncommon in the
Ugandan drug shops and private sector clinics, so asking retailers questions about
these products would have been challenging because a significant amount of time
would have needed to be spent introducing and explaining the diagnostic. Even after
an introduction, it is unlikely that the retailer would have a good enough
understanding about the product to be able to make informed decisions about
stocking it. Second, DevOrg was a valuable partner in facilitating access to retailers,
distributors, and first line buyers in the supply chain. For short turnaround studies
like this one, rapid access to respondents is critical for the success of the evaluation.
Finally, this context, where a public health intervention is taking place, is interesting
to study. Interventions along these lines are fairly common and it is frequent that an
agent in a supply chain is making a decision that includes an option about whether
or not to partner with an NGO/government/non-profit on a project. The results
from this study are interesting because the situation is reflective of the reality of the
landscape. Further, this study can help inform and shape future public health
interventions.
154
9.2.1.2
The Importance of CulturalAdaptation
Multi criteria decision analysis is a fairly complex methodology and this type of
study has many different aspects to the data collection process, all of which are
subtle and can be easily confused. Even among individuals with an education similar
to the facilitators, there are common mistakes and biases that occur and influence
the results (Keeney, 2002; Gilberto Montibeller & von Winterfeldt, 2015). When this
methodology is introduced to the developing world context, it becomes even more
complicated because of educational and cultural backgrounds. For example, in
general and with an audience of western educated individuals, drawing and
showing the graphs to the respondents can assist in understanding about the
nuance of the questions and also confirm the shapes of the functions to ensure that
they match the respondents' intuition. However, the retailers in this study's sample
would not have been able to understand the graphs. Before collecting data in
Uganda, it was assumed that exhibiting graphs would not be appropriate and this
intuition was confirmed once on the ground in conversations with DevOrg. Instead,
the researchers adapted the methodology with the assistance of moderators in
Uganda, who provided cultural expertise. The result of this was that a "happiness"
scale was used to elicit value functions. Retailers were familiar with the type of
direct ranking and were comfortable providing answers in this format.
Examples relevant to the lives of the respondents were critical to explaining the
study. The researchers generated the examples with assistance from experts and
partners in Uganda. One situational example used that was particularly effective
was in discussing purchasing a shirt. The researcher talked through the different
considerations when purchasing a shirt (e.g., cost, quality, fit, color, etc.), the ranges
and value of the criteria, and the trade off among these considerations (see
Methodology section). In particular, using one example that continued through all
of the parts of the data collection process (soliciting criteria, eliciting value
functions, trade off analysis) was valuable for respondents' understanding. As a
155
facilitator develops expertise in this methodology in the context of the developing
world, he/she will attain a repertoire of examples that will facilitate data collection.
Another factor that likely contributed to the success of the data collection was the
presence of a moderator who interfaced between the retailers and the researchers.
This served a few purposes. First, although all of the research was conducted in
English with respondents who were at a high level of proficiency, there were
moments when translation into the local language clarified a concept. Second, the
facilitator was able to explain concepts in a fashion that would make sense to the
respondents. Finally, although the researchers made a point to be accessible and
welcoming to questions, some individuals preferred to address queries to the
mobilizer, who was a local. Again, as a researcher gains expertise as a facilitator of
this methodology the need for a mobilizer will diminish.
9.2.1.3 Direction of Value
There were several areas where issues may have arisen in eliciting value functions.
For example, as described in the Results section, several sets of responses were
eliminated from the analysis when they went opposite to the direction of value. It is
proposed that there may be two factors at play that influenced such responses. First,
the retailer may have been confused about the instructions. For example, it is highly
likely that price and cost were confused and interchanged in the focus group
discussion. Additionally, retailers may have made a mistake on their response form;
the prompts were written on a large sheet of paper at the front of the room, so it is
entirely plausible that a respondent made a transcription error.
Another possibility is that the retailers had a preference in a different direction than
had been established. For example, a respondent may have been expressing real
value when saying that a higher cost had a higher value for them. Perhaps this is
because the individual believes a higher cost represents a better product (see
subsequent section on Correlation of Attributes). If this is the case, it is possible that
the methodology used to elicit preferences would need to be adjusted and clarified
156
to the respondents. The direction of increasing value for the criteria had been set by
the researchers and then proposed to the research group. Even though the group
was asked to confirm what was proposed, it is possible that there were individuals
who did not want to speak up and express their preferences. In future studies, it is
recommended that researchers spend time establishing relative value across ranges
for the retailers and determining if they have preferences in the opposite direction
or if there is an issue in correlating two different criteria.
9.2.1.4
Correlation ofAttributes
If respondents preferred a different direction of value than established by the
researchers, then it is possible that they were associating two correlated but
preferentially independent variables. There was evidence of this issue in
conversations with FLBs so it is likely that retailers had a similar response.
Respondents had a difficult time untangling corresponding but preferentially
independent variables. An example of corresponding values could include cost and
quality; as quality increases, often cost increases subsequently. However, this is not
necessarily the case, and it is possible to conceive of valuing cost and quality
independently. If two devices are of equal quality, the one with the lower cost will
be preferred. However, in conversations, it was clear that some respondents could
not untangle this concept. Future work on this methodology in this context would be
to have concrete examples from the local context that exhibits the independence of
two variables: for example, a recognizable product that is well known to be
expensive but of poor quality.
9.2.1.5
HypotheticalDecisions
Another challenge in the data collection process was in helping the respondents
conceptualize a decision process based on a hypothetical situation. In this context,
the decision had actually already been made: all individuals in this sample were part
of DevOrg's bundled service. The researchers asked the respondents to reflect back
to when they made the decision to enter into the bundled service and consider the
options that were available to them at that time. Many individuals expressed
confusion and reluctance when first asked this and needed to be coached through
157
the thought process. Further, it is possible that their already having lived out one of
the options influenced their attributes and value functions. For example,
distributors described volume of sales in the context of having already procured
300,000-600,000 devices. Additionally, one distributor was unable at all to
understand this concept and refused to provide answers on his preferences. One
idea to combat this confusion is to position this research such that the decision is
real, not hypothetical. For example, if it were explained that DevOrg was making
plans for an initiative and wanted their input, then individuals would be in the
position of thinking about an actual decision that was being made. Future work
should place an emphasis on considering what type of situations this methodology
serves to provide clarity and insight on.
9.2.2
Data Analysis
This section comments on the data analysis methods and assumptions that were
incorporated in this study.
9.2.2.1
Weighting
Both retailers and distributors did not have a wide spread between different criteria
when assigning weight values. For retailers, there were only seven weights out of a
total of 213 (all retailers and all criteria weights) that were below 40 out of 100. The
median of the responses ranged from a low of 62 to a high of 90, and once
normalized ranged from 7.4% to 10.8%, which is a very narrow range. The
distributors ranged from 70-100 (11.6%-16.5%) for Distributor 1 and 80-100
(13%-16.3%) for Distributor 2. It is interesting to note that the FLBs had a spread
from 20 (or 10) - 100 for weights. It is possible that there was a cognitive bias
present when the data was being collected that resulted in a very narrow swing
between weights. A scaling bias may have influenced these results, where the
respondents created an artificial "floor" below which they did not assign a value.
Scaling biases can occur because of where the high and low ranges of an attribute
are set (Gilberto Montibeller & von Winterfeldt, 2015). If the ranges of a criterion
were set artificially low or high, for example cost being unreasonably high or quality
being terrible, a respondent may take that into account when assigning weights and
158
inflate the weight of a criteria that has an extreme range. For example, a retailer
would think it is important not to have a terrible quality device and so the weight on
the criterion quality would be high. However, in reality, the range of devices would
not include those of the worst quality because a retailer would not seriously
consider purchasing such a device. In evaluating the ranges of criteria that were
established in the discussions, it is possible that some of the ranges were artificially
extreme. For example, cost of a device to the retailer ranged from $0 to $1. Value
functions that emerged demonstrated that retailers' value drops to 1 (the lowest) at
about $0.70. However, many other criteria did not seem to have extreme ranges.
Thus, alternatively, for some attributes if the range was reasonable, another
possible reason for the bias was that the respondents conflated "importance" with
priority (G. Montibeller & Franco, 2007). That is, respondents articulated that all the
criteria that they had identified were important, which of course they are, because
those were the factors identified in making a decision. However, it is important to
take into consideration the range of attributes, as mentioned previously. For a
distributor, for example, if the cross selling can only reasonably increase to 20%, the
difference between the worst option and the best option is 0%-20% not 0% to some
drastic increase that is not realistic. Failure to keep this mind may result in inflated
weights for some of the criteria.
While efforts were made by the researchers to control for this bias, there were
challenges in explaining this complicated methodology in the short amount of time
available for data collection with the respondents. Further, in focus group
discussions there were many more participants than moderators and that may have
resulted in a failure to fully communicate the nuance of the instructions (see
previous section on Data Collection for thoughts on methods improvements).
9.2.2.2
Input Assumptionsand SensitivityAnalysis
After eliciting value functions and weights for the criteria, multi criteria decision
analysis can be used to derive an overall value for each of the decision options. To
159
do this, it is necessary to generate a decision matrix that summarizes the expected
outcomes of each of the options along the different criteria. Generation of these
input values was done through a combination of using data from the focus groups
and interviews, expert opinions by MIT faculty and DevOrg representatives, and
relevant literature. Ideally in MCDA methodology, the decision makers are part of
constructing the matrix of inputs. However, due to the limited amount of time the
researchers were able to spend with each informant, this was not possible.
There is inherent uncertainty in the inputs of these values. To address this, three
categories of inputs were used: a high estimate, a low estimate, and a base estimate.
The analysis was run with all three categories and the overall value ordinal rank
compared. For the retailers, regardless of whether the high, low, or base
assumptions of inputs were used, the best option was always Option 2, selling the
mRDT through the bundled service. For distributors, the best option was always
Option 1. This suggests that the results of this study are robust. Future work could
consider running a Monte Carlo simulation to further assess results.
10 Conclusion and Future Research Direction
This research study offers insight into preferences and priorities of retailers,
distributors, and first line buyers in the supply chain for malaria rapid diagnostic
test kits. A multi criteria decision analysis methodology was adapted for this context
to understand a decision process that had four possible outcomes: 1) purchase
WHO-approved mRDTs outside of a bundled service, 2) purchase WHO-approved
devices through the bundled service, 3) purchase non-WHO approved devices, and
4) do not purchase mRDTs. It was found that, in general, retailers and first line
buyers prefer Option 2 (purchase mRDTs through the bundled service) while
distributors prefer Option 1 (purchase WHO-approved mRDTs outside of the
bundled service). A sensitivity analysis on input assumptions and weights was
performed and the results appear relatively robust. The discussion comments on
160
aspects of the intervention that could be altered to harmonize the best choices
across the supply chain.
This research study was a pilot to test a rapid evaluation method and was therefore
performed on a short timeline, with only two weeks in the field to collect data. Thus,
one limitation of this research was that it was not possible to have multiple
iterations of the questionnaire that was employed. Future research would build
from this experience, and improve the data collection process. For example,
collecting relevant examples to clarify common misconceptions would be very
useful, in addition to holding focus groups with smaller numbers of individuals to
ensure understanding and communication among participants. Further, it is worth
exploring this methodology when agents are actively making a decision, instead of
after the decision, as was the case in this research study. While this decision was
interesting to understand, clarifying hypothetical situations and framing value
functions was challenging to do with participants.
Further, it was not possible to include the manufacturers in this study. Future
research should ensure to obtain their perspective, as it is an important part of the
supply chain.
Finally, it is recommended that a monte carlo simulation be performed as a
sensitivity analysis. In this study, sensitivity analyses were performed on input
parameters and weights. However, since there are so many criteria and agents, it
would be worth running simulations to better understand the robustness of the
results.
161
References
ACTwatch Group, P. (2014). ACTwatch Study Reference Document: Uganda Outlet
Survey 2013.
ACTwatch Group, Program for Acessible Health, Communication & Education
(PACE), & The Independent Evaluation (IE) Team. (2012). ACTwatch Outlet
Survey Report 2011 (Round 4).
Amexo, M., Tolhurst, R., Barnish, G., & Bates, 1. (2004). Malaria misdiagnosis: Effects
on the poor and vulnerable. Lancet, 364(9448), 1896-1898.
http://doi.org/10.1016/S0140-6736(04)17446-1
Awor, P., Wamani, H., Bwire, G., Jagoe, G., & Peterson, S. (2012). Private sector drug
shops in integrated community case management of malaria, pneumonia, and
diarrhea in children in Uganda. AmericanJournalof TropicalMedicine and
Hygiene, 87(SUPPL.5), 92-96. http://doi.org/10.4269/ajtmh.2012.11-0791
Baltussen, R., & Niessen, L. (2006). Priority setting of health interventions: the need
for multi-criteria decision analysis. Cost Effectiveness and Resource Allocation, 4,
14. http://doi.org/10.1186/1478-7547-4-14
Bate, R., & Hess, K. (2009). Affordable Medicines Facility for malaria. The Lancet
Infectious Diseases, 9(7), 396-397. http://doi.org/10.1016/S14733099(09)70158-6
Center for Global Development. (2007). A Risky Business: Saving money and
improving global health through betterdemandforecasts. Group. Retrieved from
http://www.cgdev.org/files/13784_fileRiskyBusinessFull.pdf
Chandler, C. I. R., Hall-Clifford, R., Asaph, T., Pascal, M., Clarke, S., & Mbonye, A. K.
(2011). Introducing malaria rapid diagnostic tests at registered drug shops in
Uganda: Limitations of diagnostic testing in the reality of diagnosis. Social
162
Science and Medicine, 72(6), 937-944.
http://doi.org/10.1016/j.socscimed.2011.01.009
D'Acremont, V., Kahama-Maro,
J., Swai, N.,
Mtasiwa, D., Genton, B., & Lengeler, C.
(2011). Reduction of anti-malarial consumption after rapid diagnostic tests
implementation in Dar es Salaam: a before-after and cluster randomized
controlled study. MalariaJournal,10(1), 107. http://doi.org/10.1186/14752875-10-107
Figueira, J., Greco, S., & Ehrogott, M. (2005). Multiple criteriadecision analysis:state
of the artsurveys. Boston: Kluwer Acadademic Publishers.
Gelband, H., Panosian, C., & Arrow, K. (Eds.). (2004). Saving Lives, Buying Time:
Economics of MalariaDrugs in an Age of Resistance. National Academies Press.
Guerin, P.
J., Olliaro,
J. (2002).
P., Nosten, F., Druilhe, P., Laxminarayan, R., Binka, F., ... White, N.
Malaria: Current status of control, diagnosis, treatment, and a
proposed agenda for research and development. Lancet Infectious Diseases,
2(9), 564-573. http://doi.org/10.1016/S1473-3099(02)00372-9
Keeney, R. L. (1982). Decision Analysis: An Overview. OperationsResearch, 30(5).
Keeney, R. L. (2002). Common Mistakes in Making Value Trade-Offs. Operations
Research, 50(6), 935-945. http://doi.org/10.1287/opre.50.6.935.357
Keeney, R. L., & Gregory, R. S. (2005). Selecting Attributes to Measure the
Achievement of Objectives. OperationsResearch, 53(1), 1-11.
http://doi.org/10.1287/opre.1040.0158
Keeney, R. L., & Raiffa, H. (1976). Decisionswith Multiple Objectives: Preferences and
Value Tradeoffs. New York: John Wiley & Sons.
Keller, L. R., & Wang, Y. (2009). Multiple-Objective Decision Analysis Involving
Multiple Stakeholders, 139-155. http://doi.org/10.1287/educ.1090.0066
163
Koksalan, M., Wallenius,
J., & Zionts, S.
(2011). Decision CriteriaDecision Making:
From Early History to the 21st Century. Singapore, Singapore: Mainland Press
Pte Ltd.
J., Asiimwe,
Tibenderana,
J.
C., Nakanjako, D., Nabakooza, J., Counihan, H.,
&
Kyabayinze, D.
K. (2010). Use of RDTs to improve malaria diagnosis and fever
case management at primary health care facilities in Uganda. MalariaJournal,9,
200. http://doi.org/10.1186/1475-2875-9-200
MacKintosh, C. L., Beeson,
J. G.,
& Marsh, K. (2004). Clinical features and
pathogenesis of severe malaria. Trends in Parasitology,20(12), 597-603.
http://doi.org/10.1016/j.pt.2004.09.006
Marsh, K., Dolan, P., Kempster,
J., & Lugon, M.
(2013). Prioritizing investments in
public health: A multi-criteria decision analysis. Journalof PublicHealth (United
Kingdom), 35(3), 460-466. http://doi.org/10.1093/pubmed/fds099
Mbonye, A. K., Magnussen, P., Chandler, C. I., Hansen, K. S., Lal, S., Cundill, B., ...
Clarke, S. E. (2014). Introducing rapid diagnostic tests for malaria into drug
shops in Uganda: design and implementation of a cluster randomized trial.
Trials, 15(1), 303. http://doi.org/10.1186/1745-6215-15-303
Mbonye, A. K., Ndyomugyenyi, R., Turinde, A., Magnussen, P., Clarke, S., & Chandler,
C. (2010). The feasibility of introducing rapid diagnostic tests for malaria in
drug shops in Uganda. MalariaJournal, 9(1), 367.
http://doi.org/10.1186/1475-2875-9-367
Mbonye, A., Lal, S., Cundill, B., Hansen, K. S., Clarke, S., & Magnussen, P. (2013).
Treatment of fevers prior to introducing rapid diagnostic tests for malaria in
registered drug shops in Uganda. MalarJournal, 12(1), 131.
http://doi.org/1475-2875-12-131 [pii]\nlO.1186/1475-2875-12-131
164
Mendoza, G. a., & Martins, H. (2006). Multi-criteria decision analysis in natural
resource management: A critical review of methods and new modelling
paradigms. ForestEcology and Management, 230(1-3), 1-2 2.
http://doi.org/10.1016/j.foreco.2006.03.023
Montibeller, G., & Franco, a. (2007). Decision and Risk Analysis for the evaluation of
Strategic Options. Supporting Strategy: Frameworks, Methods and Models.
Montibeller, G., & von Winterfeldt, D. (2015). Cognitive and Motivational Biases in
Decision and Risk Analysis. Risk Analysis, n/a-n/a.
http://doi.org/10.1111/risa.12360
Ndyomugyenyi, R., Magnussen, P., & Clarke, S. (2007). Diagnosis and treatment of
malaria in peripheral health facilities in Uganda: findings from an area of low
transmission in south-western Uganda. MalariaJournal,6, 39.
http://doi.org/10.1186/1475-2875-6-39
Nutt, D. J., King, L. a., & Phillips, L. D. (2010). Drug harms in the UK: A multicriteria
decision analysis. The Lancet, 376(9752), 1558-1565.
http://doi.org/10.1016/S0140-6736(10)61462-6
O'Connell, K. a, Gatakaa, H., Poyer, S., Njogu,
J., Evance, I.,
Munroe, E., . . Chavasse, D.
(2011). Got ACTs? Availability, price, market share and provider knowledge of
anti-malarial medicines in public and private sector outlets in six malariaendemic countries. MalariaJournal, 10(1), 326. http://doi.org/10.1186/14752875-10-326
Palafox, B., Patouillard, E., Tougher, S., Goodman, C., Hanson, K., Buyungo, P.,
ACTwatch Group. (2012). ACTwatch 2009 Supply Chain Survey Results,
Uganda, (January).
Rutebemberwa, E., Pariyo, G., Peterson, S., Tomson, G., & Kallander, K. (2009).
Utilization of public or private health care providers by febrile children after
165
user fee removal in Uganda. MalariaJournal, 8, 45.
http://doi.org/10.1186/1475-2875-8-45
Sanyal, B., Frey, D., Graves, S., de Weck, 0., Brine, D., Goentzel,
J.,
... Montgomery, B.
(2015). Experimentation in ProductEvaluation: The case of solar lanternsin
Uganda, Africa.
Stewart, T.
J., Joubert, A., & Janssen,
R. (2010). MCDA framework for fishing rights
allocation in South Africa. Group Decision and Negotiation, 19(3), 247-265.
http://doi.org/10.1007/s10726-009-9159-9
Tang, C. S. (2006). Perspectives in supply chain risk management. International
Journalof ProductionEconomics, 103(2), 45 1-488.
http://doi.org/10.1016/j.ijpe.2005.12.006
Trampuz, A., Jereb, M., Muzlovic, I., & Prabhu, R. M. (2003). Clinical review: Severe
malaria. CriticalCare (London, England), 7(4), 315-323.
http://doi.org/10.1186/cc2183
Tuteja, R. (2007). Malaria - An overview. FEBSJournal,274(18), 4670-4679.
http://doi.org/10.1111/j.1742-4658.2007.05997.x
Uganda Bureau of Statistics. (2014). National Population and Housing Census 2014:
Provisional Results. Retrieved from
http://unstats.un.org/unsd/demographic/sources/census/2010_PHC/Uganda
/UGA-2014-11.pdf
Uganda Office Of The Auditor-General. (2010). Value for Money Audit Report on the
Regulation of Medicines in Uganda by National Drug Authority. Retrieved from
https://afrosai-e.org.za/sites/afrosai-e.org.za/files/reports/Regulation of
medicines (2010).pdf
166
Wakiso District Local Government. (2009). Higher Local Government Statistical
Abstract. Retrieved from
http://www.ubos.org/onlinefiles/uploads/ubos/2009_HLG_
Abstract-printed/WAKISOFINAL_ 2009.pdf
Wang,
J. J., Jing, Y. Y., Zhang, C. F., &Zhao, J. H. (2009).
Review on multi-criteria
decision analysis aid in sustainable energy decision-making. Renewable and
Sustainable Energy Reviews, 13(9), 2263-2278.
http://doi.org/10.1016/j.rser.2009.06.021
White, N.
J. (2004). Review series Antimalarial drug
resistance. Trends in
Parasitology,113(8), 1084-1092. http://doi.org/10.1172/JCI200421682.1084
Williams, H. A., Causer, L., Metta, E., Malila, A., O'Reilly, T., Abdulla, S., ... Bloland, P. B.
(2008). Dispensary level pilot implementation of rapid diagnostic tests: an
evaluation of RDT acceptance and usage by providers and patients--Tanzania,
2005. MalariaJournal, 7, 239. http://doi.org/10.1186/1475-2875-7-239
Wilson, M. L. (2012). Malaria rapid diagnostic tests. ClinicalInfectious Diseases,
54(11), 1637-1641. http://doi.org/10.1093/cid/cis228
Wongsrichanalai, C., Barcus, M. J., Muth, S., Sutamihardja, A., & Wernsdorfer, W. H.
(2007). A review of malaria diagnostic tools: Microscopy and rapid diagnostic
test (RDT). American Journalof Tropical Medicine and Hygiene, 77(SUPPL. 6),
119-127. http://doi.org/10.3126/ajms.v1i2.2965
World Health Organisation. (2006). The Use of Malaria Rapid Diagnostic Tests.
Retrieved from http://www.wpro.who.int/malaria/NR/rdonlyres/44C556B80E04-4D26-BC1DD00D2743B2C0/0/TheuseofMalariaRDTs_2ndedition_2006.pdf?ua=1
World Health Organisation. (2011a). Global plan for artemisinin resistance
containment. Retrieved from
167
http://apps.who.int/iris/bitstream/10665/44482/1/9789241500838_eng.pdf
?ua=1
World Health Organisation. (2011b). Good practicesforselecting and procuring rapid
diagnostictests for malaria.
World Health Organisation. (2014a). Malaria Rapid Diagnostic Test Performance.
World Health Organisation. (2014b). World malaria report 2014.
http://doi.org/10.1007/s00108-013-3390-9
Yeka, A., Gasasira, A., Mpimbaza, A., Achan,
Rosenthal, P.
J. (2012).
J., Nankabirwa, J., Nsobya, S.,
Malaria in Uganda: challenges to control on the long
road to elimination: I. Epidemiology and current control efforts. Acta Tropica,
121(3), 184-95. http://doi.org/10.1016/j.actatropica.2011.03.004
Zopounidis, C., & Doumpos, M. (2002). Multi-criteria decision aid in financial
decision making: methodologies and literature review. Journalof MultiCriteria
DecisionAnalysis, 11(4-5), 167-186. http://doi.org/10.1002/mcda.333
168