Using Multi-Criteria Decision Analysis to Assess Private Sector Agents' Preferences and Priorities in Stocking Malaria Rapid Diagnostic Test Kits in Uganda ARCHVES By MASSACHUSETTS INSTITI OF rECHNOLOLGY Corinne M. Carland S.B. Chemical Engineering and Biology Massachusetts Institute of Technology, 2013 MAY 2 6 2015 LIBRARIES SUBMITTED TO THE ENGINEERING SYSTEMS DIVISION IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE IN TECHNOLOGY AND POLICY AT THE MASSACHUSETTS INSTITUTE OF TECHNOLOGY JUNE 2015 0 2015 Corinne M. Carland All rights reserved. The author hereby grants MIT permission to reproduce and to distribute publicly paper and electronic copies of this thesis document in whole or in part in any medium now known or hereafter created. Signature of Author: Signature redacted Technology and Policy Program, Engineering Systems Division May 8, 2015 Certified by: Signature redacted Jarrod Goentzel Director, Humanitarian Response Laboratory Thesis Supervisor Accepted by: ________Signature redacted Dava J. Newman Pro ssor of Aeronautics and Astronautics and Engineering Systems Director, Technology and Policy Program IT F Using Multi-Criteria Decision Analysis to Assess Private Sector Agents' Preferences and Priorities in Stocking Malaria Rapid Diagnostic Test Kits in Uganda By Corinne Carland Submitted to the Engineering Systems Division on 14 May 2015 in partial fulfillment of the requirements for the degree of Master of Science in Technology and Policy Abstract Diagnosis of malaria is important in order to ensure early and effective treatment, to facilitate public health surveillance, and to prevent drug resistance. Rapid diagnostic tests (RDTs) are an important tool in resource-constrained settings, as they do not rely on costly lab equipment and specially trained personnel. In Uganda's private sector clinics and drug shops, which is where the majority of patients first seek care, diagnosis of malaria is often presumptive and patients receive neither RDT nor microscopy. Several studies have focused on the patient perspective (e.g. willingness to pay and willingness to be tested) but much less is understood about the supplier perspective (e.g. willingness to stock). This study aimed to understand the preferences and priorities of agents across the malaria RDT supply chain in Uganda on stocking the devices using multi-criteria decision analysis. This methodology was adapted to be relevant and understandable for agents in Uganda so that it was possible to analyze business decisions incorporating a multiplicity of attributes such as selling price, purchase cost, sales volume, complexity of regulations, waste management, and training available. Data surveys and semistructured interviews were collected from 28 private sector retailers (i.e., shopkeepers, pharmacists, clinic managers), two first line buyers, and three distributors. Analysis of the data resulted in the construction of value functions for all agents, the relative weights (therefore the tradeoffs) among decision criteria, and the calculation of an overall value for the decision about whether or not to stock RDTs for the different supply chain agents. Results indicate that the best option for one level of the supply chain is not necessarily the best for another. A discussion offers insights on how to align value across the supply chain, which is important for facilitating public health interventions. Thesis Supervisor: Jarrod Goentzel Title: Director, Humanitarian Response Laboratory 1 Acknowledgements This work was made possible by a grant from the U.S. Agency for International Development (USAID), and I want to express my gratitude for their support. Further, I am thankful for MIT's Comprehensive Initiative for Technology Evaluation (CITE) for facilitating and supporting the research of their students. I have benefitted from the expert guidance and help of my adviser, Dr. Jarrod Goentzel. When I expressed a research interest, Jarrod was the person who made it happen, proposing this research on my behalf. His confidence in me, encouragement, and advice helped me grow immensely as a researcher and professional. I am very appreciative that I had the opportunity to work in the Humanitarian Response Lab for the past two years. I am also deeply grateful for the support and advice of Dr. Gilberto Montibeller. Without his expertise and willingness to introduce me to the field of decision sciences, this thesis would not have been possible. I am thankful for the time he spent on calls, providing feedback, and answering many questions. Further, this work could not have been completed without Elizabeth Streat in Uganda. Her and her team's advice and suggestions were invaluable in guiding implementation and shaping our research. Additionally, this work could not have been completed without Erin Reissman, who traveled with me to Uganda to do data collection. Thank you for helping to shape and implement this research and also for ensuring we also had fun during our travels. No acknowledgement section would be complete without reference to one's family, but cliched as it is, they have been an unending source of support and encouragement. Especially my mother who has spent many hours listening to me talk about research and school. Further, friends who have become family were integral over these past two years and so I would like to thank Camila, Noam, and Carlos. TPP was a wonderful experience and has challenged me to grow and learn. I need to thank Barb, Ed, Frank, and Dava for their support and advice. Further, TPP and ESD friends and colleagues made these two years enriching and fun; I am grateful to be surrounded by such inspiring individuals-thank you, Lauren, Stacey, Tim, Emily, and Mark. Finally, thank you to MIT for being my home for these past six years and shaping me into the person I am today. 2 Contents A b strac t .......................................................................................................................................................... 1 Acknowledgements................................................................................................................................... 2 Co nten ts . .;... ............................................................................................................................................. 3 List of Tables................................................................................................... 6 L ist o f F igu res .............................................................................................................................................. 8 L ist o f A cro n y ms...................................................................................................................................... 10 1 In tro d uctio n ....................................................................................................................................... 11 2 Background ............................................................................................ 12 2.1 Malaria............................................................................................ 12 2 .1 .1 T reatm en t.......................................................................................................................... 13 2 .1.2 D iagn o sis............................................................................................................................ 13 2.2 Malaria Rapid Diagnostic Tests .................................................................................... 14 2.2.1 W HO Approval for Malaria Rapid Diagnostic Tests ................................... 16 2.3 Malaria Rapid Diagnostic Tests and Healthcare Systems in Uganda............17 2.3.1 Government Regulation........................................................................................... 17 2.3.2 Private Sector Retailers .......................................................................................... 17 2.3.3 Interventions to Scale up mRDT Use................................................................. 18 2.4 Pilot to Scale up use of mRDTs in Uganda............................................................... 18 2.5 Multiple Criteria Decision Analysis............................................................................. 21 2.5.1 History and Motivation of Decision Analysis ................................................. 22 2.5.2 Applications of MCDA.............................................................................................. 23 2.6 Aim of this Research Study................................................................................................. 23 3 Methodology........................................................................................... 24 3.1 Research Scope .................................................................................. 24 3.2 Data Collection................................................................................... 25 3.2.1 Semi-Structured Interviews with First-Line Buyers and Distributors ... 25 3.2.2 Focus Group Discussions with Retailers ......................................................... 3.3 Methodology Framework................................................................................................ 3.3.1 Decision Framing............................................................................................................ 3.3.2 Identifying Criteria........................................................................................................ 3.3.3 Eliciting value functions.......................................................................................... 3.3.4 Determining Priorities (W eighting)................................................................. 3.3.5 Generation of decision matrix inputs ............................................................... 3.3.6 Evaluating options......................................................................................................... 3.3.7 Sensitivity Analysis................................................................................................... 4 Inputs and Assumptions............................................................................................................... 4.1 Decision Input Matrix............................................................................................................ 4.1.1 Decision Input Matrix - Retailers....................................................................... 4.1.2 Decision Input Matrix - Distributors............................................................... 4.1.3 Decision Input Matrix - First Line Buyers....................................................... 5 Results - Value Functions......................................................................................................... 5 .1 R eta ilers C riteria ..................................................................................................................... 5.2 Retailers Value Functions................................................................................................ 26 27 28 30 40 42 44 44 45 46 46 46 52 57 60 60 61 3 5.2.1 Quality ................................................................................................................................. 61 5.2.2 Cost ....................................................................................................................................... 63 5.2.3 Price ..................................................................................................................................... 65 5.2.4 T im e to D elivery ............................................................................................................. 67 5.2.5 Sales of O ther Products ................................................................................................ 69 5.2.6 Custom er Satisfaction ................................................................................................... 71 5.2.7 T raining .............................................................................................................................. 72 5.2.8 V olum e ................................................................................................................................ 74 5.2.9 A wareness/A dverti sing ............................................................................................... 76 5.2.10 T im e to Com plete a Sale ........................................................................................... 78 5.2.11 O pportunities ................................................................................................................ 80 5.3 D istributors ............................................................................................................................... 82 5.3.1 V olum e ................................................................................................................................ 82 5.3.2 Expiration D ate ............................................................................................................... 85 5.3.3 Profit M argin .................................................................................................................... 87 5.3.4 Cost Per Kit ........................................................................................................................ 89 5.3.5 Cost per T raining ............................................................................................................ 90 5.3.6 Effi ciency of D istribution ............................................................................................ 92 5.3.7 Cross Selling ...................................................................................................................... 95 5.4 First Line B uyers ..................................................................................................................... 97 5.4.1 Cost per D evice ................................................................................................................ 97 5.4.2 Price per Device .............................................................................................................. 99 5.4.3 Quality of D evice .......................................................................................................... 101 5.4.4 A dm inistrative T im e .................................................................................................. 103 5.4.5 Percent Profit ................................................................................................................ 104 5.4.6 R elationship w ith D onor O rganization .............................................................. 106 5.4.7 Percent Increase in Sales ......................................................................................... 108 6 R psults - Wpiprhtq ----- ................... ............................................................................ 109 6.1 R etailers ................................................................................................................................... 6.2 D istributors ............................................................................................................................ 6.3 First Line B uyers .................................................................................................................. 7 R esults - O verall V alue ............................................................................................................... 7.1 O verall Value - R etailers ................................................................................................... 7.2 O verall Value - D istributors ............................................................................................ 7.3 O verall Value - First Line B uyers .................................................................................. 7.4 Overall Value - Comparison Between Agents ......................................................... 8 Sensitivity A nalysis ...................................................................................................................... 8.1 D istributors ............................................................................................................................ 8.1.1 V olum e ............................................................................................................................. 8.1.2 Expiration D ates .......................................................................................................... 8.1.3 Profit M argin ................................................................................................................. 8.1.4 Cost per Kit ..................................................................................................................... 8.1.5 Cost per T raining ......................................................................................................... 8.1.6 Effi ciency of D istribution ......................................................................................... 8.1.7 Cross Selling ................................................................................................................... 8.2 First Line B uyers .................................................................................................................. 110 111 112 113 113 114 116 118 120 121 121 123 124 126 127 129 130 131 4 8.2.1 Cost per Kit..................................................................................................................... 131 8.2.2 Price per Kit................................................................................................................... 133 8.2.3 Q uality .............................................................................................................................. 134 8.2.4 A dm inistrative T im e .................................................................................................. 8.2.5 Profit M argin ................................................................................................................. 8.2.6 R elationship with D onor .......................................................................................... 8.2.7 Increase in Sales........................................................................................................... 9 D iscussion ........................................................................................................................................ 9.1 R esult from Analysis........................................................................................................... 9.1.1 Best O ptions for A gents ............................................................................................ 9.1.2 Sensitivity A nalysis..................................................................................................... 9.1.3 Im plications and Im proving O ptions.................................................................. 9.1.4 Insights from Value Functions and Weighting .................. 9.1.5 Em ergent Criteria........................................................................................................ 9.2 10 136 138 139 14 1 14 1 14 1 142 143 144 147 152 M ethodology .......................................................................................................................... 153 9.2.1 D ata Collection.............................................................................................................. 9.2.2 D ata A nalysis................................................................................................................. Conclusion and Future R esearch D irection .................................................................... 153 158 160 R eferences............................................................................................................................................... 162 5 List of Tables Table Table Table Table Table Table Table Table Table Table Table 1. Antigens used in m RDTs ............................................................................................... 16 2. List of Criteria and Definitions for Retailers......................................................... 33 3. Criteria and m etric units ................................................................................................ 34 4. Qualitative Metric Definitions for Retailers......................................................... 35 5. Criteria and ranges for retailers................................................................................ 36 6. List of Criteria and Definitions for Distributors................................................... 37 7. Criteria and ranges for distributors ......................................................................... 38 8. List of Criteria and Definitions for Distributors................................................... 39 9. Criteria and ranges for FLBs........................................................................................ 39 10. Qualitative Metric Definitions for FLBs............................................................... 40 11. Input figures for retailers for high, base, and low estimates for options 1-4 ........................................................................ ........... ...... 47 Table 12. Input figures for distributors for high, base, and low estimates for options 1 -4 ........................................................................................................................................................ 53 Table 13. Input figures for distributors for high, base, and low estimates for options 1 -4 ........................................................................................................................................................ 58 Table 14. Criteria and ranges of criteria for retailers. ........................................................ 61 Table 15. Median of retailer responses for quality ............................................................ 62 Table 16. Median of retailer responses for cost per device.............................................. 64 Table 17. Median of retailer responses for price per device........................................... 66 Table 18. Removed response sets for price of mRDT ......................................................... 67 Table 19. Median of retailer responses for delivery time ................................................ 68 Table 20. Removed responses for Delivery Time .............................................................. 69 Table 21. Median of retailer responses for increased sales of other products ..... 70 Table 22. Median of retailer responses for customer satisfaction ................................ 72 Table 23. Median of retailer responses for training ............................................................ 73 Table 24. Median of retailer responses for volume............................................................. 75 Table 25. Median of retailer responses for advertising/awareness............................ 77 Table 26. Removed responses for Advertising/Awareness ............................................. 78 Table 27. Median of retailer responses for advertising/awareness............................ 79 Table 28. Removed responses for time to complete sale ................................................ 80 Table 29. Median of retailer responses for opportunities ............................................... 81 Table 30. Criteria and ranges of criteria for distributors ................................................ 82 Table 31. Distributor 1 responses for volume....................................................................... 84 Table 32. Distributor 2 responses for volume....................................................................... 84 Table 33. Distributor 1 responses for expiration date ...................................................... 86 Table 34. Distributor 2 responses for expiration date ...................................................... 86 Table 35. Distributor 1 responses for profit margin ......................................................... 88 Table 36. Distributor 2 responses for profit margin ......................................................... 88 Table 37. Distributor 1 responses for cost per kit............................................................... 89 Table 38. Distributor 2 responses for cost per kit............................................................... 90 Table 39. Distributor 1 responses for cost per training.................................................... 91 Table 40. Distributor 2 responses for cost per training.................................................... 92 6 Table 41. Distributor 1 responses for efficiency of delivery............................................ 94 Table 42. Distributor 1 responses for efficiency of delivery............................................ 94 Table 43. Distributor 1 responses for cross selling............................................................. 96 Table 44. Distributor 2 responses for cross selling............................................................. 96 Table 45. FLB 1 responses for cost per device.......................................................................98 Table 46. FLB responses for cost per device....................................................................... 99 Table 47. FLB responses for price per device...................................................................... 100 Table 48. FLB responses for price per device...................................................................... 100 Table 49. FLB responses for quality of device .................................................................... 101 Table 50. FLB responses for quality of device ..................................................................... 102 Table 51. FLB responses for administrative time .............................................................. 103 Table 52. FLB responses for administrative time.............................................................. 104 Table 53. FLB responses for profit ........................................................................................... 105 Table 54. FLB responses for profit ........................................................................................... 106 Table 55. FLB responses for relationship with donor...................................................... 107 Table 56. FLB responses for relationship with donor...................................................... 108 Table 57. FLB 2 responses for percent increase in sales .................................................... 109 Table 58. Criteria and Number of Responses .......................................................................... 110 Table 59. Median and normalized weights............................................................................... 111 Table 60. W eights for Distributor 1............................................................................................. 111 Table 61. W eights for Distributor 2 ............................................................................................. 112 Table 62. W eights for FLB 1............................................................................................................ 112 Table 63. W eights for FLB 2 ............................................................................................................ 113 Table 64. Overall Value for Retailer Options............................................................................ 114 Table 65. Overall Value for Distributor 1 Options ................................................................. 115 Table 66. Overall Value for Distributor 2 Options ................................................................. 115 Table 67. Overall Value for Distributor 1 Options - Cost per kit is removed ............ 116 Table 68. Overall Value for Distributor 2 Options - Cost per kit is removed ............ 116 Table 69. Overall Value for FLB 1 ................................................................................................ 117 Table 70. Overall Value for FLB 2.................................................................................................. 117 Table 71. Overall Value for FLB 1 Options - Profit removed ............................................ 118 Table 72. Overall Value for FLB 2 Options - Profit removed ............................................ 118 Table 73.Comparison of overall value between different agents in the supply chain (using the baseline assumptions) ....................................................................................... 142 Table 74. W eights for FLB 1............................................................................................................ 151 Table 75. W eights for FLB 2 ............................................................................................................ 151 7 List of Figures Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure 1. Diagram of the mode of action commonly used in mRDTs. ......................... 15 2 M ap of Uganda and districts ....................................................................................... 19 3. Supply chain of mRDTs in Uganda in this study................................................. 20 4. Decision flow chart for agents in the supply chain........................29 5. Example given to respondents to elicit criteria................................................. 31 6. Numerical and verbal descriptions of happiness to elicit value functions. 41 7. Value function for quality of devices for retailers........................................... 62 8.. Value function for cost per device for retailers................................................ 64 9. Value function for price per device for retailers.............................................. 66 10. Value function for delivery time for retailers...................................................68 11. Value function for sales of other products for retailers............................. 70 12. Value function for customer satisfaction for retailers.................................. 72 13. Value function for training for retailers............................................................ 73 14. Value function for volume of sales for retailers..............................................75 15. Value function for awareness/advertising for retailers ............................. 77 16. Value function for time to complete a sale for retailers.............................. 79 17. Value function for opportunities for retailers.................................................81 18. Value function for Distributor 1 for volume.................................................... 83 19. Value function for Distributor 2 for volume.................................................... 84 20. Value function for Distributor 1 for expiration date .................................... 85 21. Value function for Distributor 2 for expiration date .................................... 86 22. Value function for Distributor 1 for profit margin ........................................ 87 23. Value function for Distributor 2 for profit margin ........................................ 88 24. Value function for Distributor 1 for cost per kit..............................................89 25. Value function for Distributor 2 for cost per kit..............................................90 26. Value function for Distributor 1 for cost per training.................................. 91 27. Value function for Distributor 2 for cost per training.................................. 92 28. Value function for Distributor 1 for efficiency of distribution..................93 29. Value function for Distributor 2 for efficiency of distribution..................94 30. Value function for Distributor 1 for cross selling ........................................... 95 31. Value function for Distributor 2 for cross selling ........................................... 96 32. Value function for FLB 1 for cost per device.................................................... 97 33. Value function for FLB 2 for cost per kit............................................................ 98 34. Value function for FLB 1 for price per kit.......................................................... 99 35. Value function for FLB 2 for price per kit............................................................ 100 36. Value function for FLB 1 for quality....................................................................... 10 1 37. Value function for FLB 2 for quality....................................................................... 102 38. Value function for FLB 1 for adm inistrative tim e ............................................ 103 39. Value function for FLB 2 for adm inistrative tim e ............................................ 104 40. Value function for FLB 1 for profit ......................................................................... 105 41. Value function for FLB 2 for profit ......................................................................... 106 42. Value function for FLB 1 for relationship ............................................................ 107 43. Value function for FLB 2 for relationship ............................................................ 108 8 Figure 44. Value function for FLB 2 for percent increase in sales .................................. 109 Figure 45.Distributor versus retailer overall value for all of the high/base/low o ptio n s 1 -4 .................................................................................................................................... 1 19 Figure 46. FLB/Distributor overall value versus Retailer overall value for all of the high/base/low options 1-4 .................................................................................................... 120 Figure 47. Overall value versus weight for volume for distributor 1............................ 122 Figure 48. Overall value versus weight for volume for distributor 2............................ 122 Figure 49. Overall value versus weight for expiration dates for distributor 1......... 123 Figure 50. Overall value versus weight for expiration dates for distributor 2.........124 Figure 51. Overall value versus weight for profit margin for distributor 1...............125 Figure 52. Overall value versus weight for profit margin for distributor 2 ............... 125 Figure 53. Overall value versus weight for cost per kit for distributor 1.............. 126 Figure 54. Overall value versus weight for cost per kit for distributor 2.............. 127 Figure 55. Overall value versus weight for cost for training for distributor 1.........128 Figure 56. Overall value versus weight for cost for training for distributor 2.......... 128 Figure 57. Overall value versus weight for cost for efficiency of distribution for d istrib u tor 1 . ................................................................................................................................ 1 29 Figure 58. Overall value versus weight for cost for efficiency of distribution for d istrib u to r 2 . ................................................................................................................................ 1 30 Figure 59. Overall value versus weight for cross selling for distributor 1.................. 130 Figure 60. Overall value versus weight for cross selling for distributor 2..................131 Figure 61. Overall value versus weight for cost per kit for FLB 1................................... 132 Figure 62. Overall value versus weight for cost per kit for FLB 2................................... 132 Figure 63. Overall value versus weight for price per kit for FLB 1................................ 133 Figure 64. Overall value versus weight for price per kit for FLB 2. ............................... 134 Figure 65. Overall value versus weight for quality for FLB 1........................................... 135 Figure 66. Overall value versus weight for quality for FLB 2 ........................................... 135 Figure 67. Overall value versus weight for administrative time for FLB 1................. 137 Figure 68. Overall value versus weight for administrative time for FLB 2................. 137 Figure 69. Overall value versus weight for profit margin for FLB 1.............................. 138 Figure 70. Overall value versus weight for profit margin for FLB 2. The dashed line m arks w here the options intersect..................................................................................... 139 Figure 71. Overall value versus weight for relationship with donor for FLB 1.......140 Figure 72. Overall value versus weight for relationship with donor for FLB 2........ 140 Figure 73. Overall value versus weight for increase in sales for FLB 2. The dashed line marks w here the options intersect............................................................................ 141 9 List of Acronyms ACT FLB MCDA mRDT NGO NDA PDS WHO Artesiminin combination therapy First line buyer Multi criteria decision analysis Malaria rapid diagnostic test Non governmental organization National Drug Authority Panel detection score World Health Organization 10 1 Introduction Scaling up the use of malaria rapid diagnostic tests (mRDTs), especially in the private sector, is an important initiative in global health. To do this, one has to take into consideration the multiplicity of agents that are involved in the private sector across the supply chain, including manufacturers, first line buyers, distributors, and retailers. These agents need to work together in a concerted fashion to ensure that products reach the end user. Each agent in the supply chain independently considers whether or not to stock the mRDTs, what type of products to stock, and whether or not to engage with NGOs or non-profits on intervention strategies. The goal of this research project was to understand how each of these individuals makes such a decision and the important and priorities of factors that are considered in the decision. A multi criteria decision analysis (MCDA) methodology was employed to quantify preferences and the best options for the different agents. This research was a pilot study that tested the feasibility of this kind of rapid evaluation and the application of MCDA in this context. This paper is divided into four main sections: Background, Methodology, Results, and Discussion. The Background section provides information on malaria, malaria rapid diagnostic test kits, MCDA methodology, and the context of this study. The Methodology section explains how MCDA was applied in this setting and the limitations and adaptations that were necessary. The Results section summarizes the value functions, weights, overall value for each of the supply chain agents, and a sensitivity analysis. The Discussion section comments on the results, limitations, and suggestions for further research. Additionally, it notes other insights that were derived over the course of data collection and analysis. 11 2 Background This section provides more information on malaria and rapid diagnostic tests. Additionally, it provides context about this research study and the methodology that was applied. 2.1 Malaria The WHO estimates that globally 1.2 billion people are at high risk for malaria (greater than a 1 in 1000 risk of contracting the disease in the year). In 2013 there were an estimated 198 million cases, which caused 584,000 deaths. The greatest malaria burden is in Africa, where 90% of the cases occur (World Health Organisation, 2014b). Malaria is caused by a five species of parasites in the Plasmodium genus, which are transmitted to human beings via the bite of an infected Anopheles mosquito. One parasite, Plasmodiumfalciparum,is the most prevalent in Africa and causes the most deaths (World Health Organisation, 2014b). The malaria parasite first enters the human body after being bitten by the mosquito and then travels to the liver. After an incubation period of several days in the liver, the parasites enter red blood cells and ultimately rupture them, resulting in the disease symptoms. These symptoms of malaria include fever, shivering, respiratory distress, headaches, diarrhea, vomiting, and crnvlidnsnc Whilemost malaria is considered uncomplicated, a small percentage (around 5%) of cases are severe. Severe malaria is characterized by cerebral malaria, pulmonary edema, acute renal failure, and anemia (Trampuz, Jereb, Muzlovic, & Prabhu, 2003). Cerebral malaria is caused by the obstruction of small blood vessels in the brain due to build up of the parasite (MacKintosh, Beeson, & Marsh, 2004). If left untreated, severe malaria is deadly. Pregnant women and children are at especially high risk of mortality from malaria (Tuteja, 2007). In Uganda, the Ministry of Health (MOH) reports that malaria is the leading cause of morbidity and mortality in the country, accounting for between 8-13 million cases per year, 35% of hospital admissions, and 9-14% of hospital deaths (Yeka et al., 2012). 12 2.1.1 Treatment The WHO recommends uncomplicated malaria caused by P.falciparumbe treated with an artemisinin combination therapy (ACT) (World Health Organisation, 2014b). Artemisinin is currently the most effective and fast acting medicine to combat malaria. An ACT is a combination therapy that includes artemisinin and a companion antimalarial. Antimalarial medications with just artemisinin are called artemisinin monotherapies; these pharmaceuticals are a threat to public health as they are assumed to promote resistance (see subsequent sections for discussion on resistance). Expansion of ACTs could be considered one of the most significant concerted global health intervention efforts of the decade. It began with the Institute of Medicine's (IOM's) convening of a panel to explore interventions to address growing resistance to malaria medications and their subsequent central recommendation to establish a global subsidy on artemisinin combination treatments (Gelband, Panosian, & Arrow, 2004). As a result of this recommendation, the Global Fund to Fight AIDS, Tuberculosis, and Malaria approved an initiative called the Affordable Medicines Facility for malaria (AFMm) to increase access to ACTs by subsiding purchases of the devices at the upstream levels and reselling the medicines at a fraction of the cost (Bate & Hess, 2009). The rational is that the subsidy will be carried down the supply chain to the retailer level where patients will be able to access affordable ACT products. The results of the global subsidy are monitored in periodic surveys by country. Initial results indicate that access to ACTs in the private sector are slowly increasing (O'Connell et al., 2011). 2.1.2 Diagnosis The WHO's official recommendation is that every suspected case of malaria should be examined for evidence of infection by either microscopy or a malaria rapid diagnostic test (mRDT) (World Health Organisation, 2014b). The method of microscopy involves taking a sample of blood from a patient and examining it under a microscope to visually determine the presence of parasites. If performed by a trained technician with adequate laboratory resources, microscopy is the ideal and most reliable mechanism of diagnosis. However, in resource-constrained settings, 13 the lack of time, training, and materials may render this method challenging and less reliable. In these settings, an mRDT is the best mechanism of diagnosis. Many studies have found that there is a lack of a rigorous diagnosis of malaria in clinics and drug shops in Sub-Saharan Africa and presumptive treatment, based on symptoms alone, results in an overtreatment of patients who do not have the disease (Wilson, 2012). Over-treatment of malaria is detrimental to the patient because it needlessly exposes the individual to antimalarials and delays the appropriate treatment. It is further detrimental to the community because it is a waste of resources in areas that cannot afford to do so and also costly to society because of the global subsidy of treatments. Finally, inappropriate use of antimalarials may contribute to antimalarial drug resistance of the parasite (Wilson, 2012), much like over prescription of antibiotics is connected with antibiotic resistant bacteria strains. Antimalarial drug resistance is an enormous public health concern. P. falciparum has already developed widespread resistance to older antimalarials, including chloroquine and sulfadoxine-pyrimethamine (White, 2004). Resistance to artemisinin, one of the most efficacious and recently developed antimalarial, has emerged in the Greater Mekong sub-region in South East Asia (World Health Organisation, 2011a). If this resistance hecomes widesnread. mnlaria control efforts will be compromised, especially because no current malaria treatment exists that provides the same levels of tolerance and efficacy as artemisinin treatments (World Health Organisation, 2011a). Access to ACTs are one critical aspect of suppressing antimalarial resistance and accurate and consistent diagnosis of malaria another and is of critical import. 2.2 Malaria Rapid Diagnostic Tests Malaria rapid diagnostic tests (mRDTs) offer a fast and accurate means of diagnosing malaria in settings where microscopy is unavailable or unreliable. These devices are valuable because they require no capital investment or electricity, are easy to interpret, and require relatively little training. They work by running an immunochromotographic assay with monoclonal antibodies directed against 14 parasite antigens. The tests require a small amount, approximately 5-15 ptL, of blood and produce a visual result in 5-20 minutes (Wongsrichanalai, Barcus, Muth, Sutamihardja, & Wernsdorfer, 2007). The blood travels up the lateral length of the device and if antigens are present, they bind to the monoclonal antibodies that are embedded in the kit (Figure 1) ound Lysing agent & labled Ab Te st e !bound Abij ContioO band 1.Anttbody "o free ubtesed wte bands visible non~ly Nitrocellulose strp (2 ) #s: blood - flushing Agent captured b labelvd Ab Blood and laeled Ab flushed Akng strip Captured labeled Ab Ag comple (3) Captured labeed Ab ZIZIZZEZI Labeled Ab Ag comptei capted by bound Ab of test band LbldA ctred by bound Ab of con"ro barid Figure 1. Diagram of the mode of action commonly used in mRDTs. 1) Dye labeled and bound unlabeled antibodies (Ab) specific for the target antigen are present at the lower end of the strip. A test line is comprised of bound Ab and the control band is formed of either antibody specific for the labeled antibody or the antigen. 2) Blood and a buffer solution are poured into the well at one end of the strip and mixed wit the labeled Ab. The liquid travels up the strip. 3) If the antigen is present, the antigen bound to labeled Ab is bound to the test line. Source: (World Health Organisation, 2006) Currently, most mRDTs run an assay on several antigens that are either general across parasite strains or specific to one (Table 1). WHO provides recommendations on what types of antigens should be used in the mRDT for different areas in the world, depending on the type of parasite most prevalent there. 15 Table 1. Antigens used in mRDTs Antigen Histidine rich protein 2 (HRP-2) Parasite P. falciparum Plasmodium lactate dehydrogenase (pLDH) Can be general to all human malaria parasites or specific Aldolase All human malaria parasites Source: (World Health Organisation, 2011b) 2.2.1 WHO Approval for Malaria Rapid Diagnostic Tests With a multiplicity of manufacturers of medical devices, it was necessary for some standardization of devices to ensure governments and non-governmental organizations (NGOs) could identify high quality products to procure. To that end, in 2006 the WHO Special Programme for Research and Training in Tropical Diseases (TNR) and the Foundation for Innovative New Diagnostics (FIND) launched an initiative to evaluate and compare the performances of commercially available mRDT devices (World Health Organisation, 2014a). Each year, the WHO publishes the results of the comparative evaluation. The criteria that the WHO considers in their evaluation include the panel detection score (PDS), false-positive rate, invalidation instances, ease of use, thermal stability, etc. Panel detection score is one of the key measures of performance. Panels of malaria-positive samples at a high and low concentration are used to evaluate the devices. The PDS is the percentage of positive results found in a batch of mRDTs when evaluated against the malaria panel. In this way, PDS evaluates both positivity rate and batch consistency. PDS further takes into account the mRDTs ability to provide conclusive results by the manufacturer suggested time (World Health Organisation, 2014a). PDS is calculated at 2000 parasites/pt and at 200 parasites/ ptL. Many devices score well at the higher concentration but clear differences emerge at the lower concentration. Low parasite density identification is important 16 as countries reduce the prevalence of malaria and because the association of symptoms and parasite level varies between individuals (World Health Organisation, 2014a). Thus, the WHO's evaluation process provides a comprehensive measurement of the relative efficacy of different mRDTs. 2.3 Malaria Rapid Diagnostic Tests and Healthcare Systems in Uganda This study took place in the district surrounding the capital of Uganda. This section explains the background and context surrounding the research, including government regulation, the private sector, and diagnostic interventions. 2.3.1 Government Regulation The Ugandan National Drug Authority (NDA) regulates pharmaceuticals and medical devices, including mRDTs. Further, drug shops and pharmacies must be licensed by the NDA; pharmacies can sell prescription-only drugs but drug shops can only sell over-the-counter medication. Antimalarials were designated as over-the-counter in 2008 (ACTwatch Group, 2014). According to one report by the Ugandan Office of the Auditor General, the length of time the NDA spends reviewing applications results in much fewer license approvals than should be expected. Additionally, the report claims that the NDA has not been vigilant about closing down unlicensed drug shops (Uganda Office Of The Auditor-General, 2010). 2.3.2 Private Sector Retailers This study focused on the private sector healthcare system because the majority of Ugandans first seek care from private sector retailers, including private clinics, pharmacies, and drug shops (Awor, Wamani, Bwire, Jagoe, & Peterson, 2012; Rutebemberwa, Pariyo, Peterson, Tomson, & Kallander, 2009). Patients have a variety of reasons for seeking care in the private sector including: the conduct, qualifications, and experience of the health provider; provider being polite; having 'cheap' treatment; giving treatment on credit; being near; and having access to good equipment or infrastructure (Rutebemberwa et al., 2009). 17 Unfortunately, many febrile patients are diagnosed with malaria without confirmation by an mRDT or microscope slide. One study found that 74% of febrile patients were given antimalarials but only 35% had a positive mRDT result (Mbonye et al., 2013). Diagnosing malaria by clinical symptoms alone is very challenging because many malaria symptoms are nonspecific and overlap with many other indications. Over-diagnosis of malaria is common not only in Uganda but across sub-Saharan Africa (Amexo, Tolhurst, Barnish, & Bates, 2004; Guerin et al., 2002; Ndyomugyenyi, Magnussen, & Clarke, 2007). 2.3.3 Interventions to Scale up mRDT Use Attempts to integrate malaria rapid diagnostic tests in both private and public facilities have illuminated the challenge and complexity of this task. Many studies have been done on considering the perspective of patients, as the end user. Mbonye et al., (2010) found that community concern about introducing mRDTs to drug shops included: that drug shops would over price the devices, reuse of mRDTs would lead to infections, and that testing would reveal one's HIV status. Further, many patients believe they have malaria, regardless of what the diagnosis says (Chandler et al., 2011). In the paper describing their intervention to introduce mRDTs in a district in Uganda, (A. K. Mbonye et al., 2014) report that this effort is complex and requires the careful coordination of people, logistics, protocols, and resources. Despite the challenges, it has been established that training and awareness can lower false positive diagnoses in both Uganda and other sub-Saharan African countries (D'Acremont et al., 2011; Kyabayinze et al., 2010; Williams et al., 2008). 2.4 Pilot to Scale up use of mRDTs in Uganda This study was performed in the context of a pilot project that was run by a nonprofit development organization. For the purposes of this paper, the development organization will be referred to as DevOrg. DevOrg had been awarded a grant to create a private sector market for quality-assured mRDTs in Uganda by increasing 18 access, generating demand, improving malaria case management, and strengthening the supply chain systems in the country to ensure robust and reliable supply. The program, scheduled to be three years, was piloted for the first year in the Wakiso district in Uganda, the district surrounding the capital, Kampala (Figure 2). According to preliminary reports of the 2014 Ugandan census, Wakiso has a population of over two million, the most populous district in the country. The urban population is about 32% and rural is 68% (Uganda Bureau of Statistics, 2014). DevOrg has implemented the project in around 180 private sector clinics, drug shops, and pharmacies distributed across the district. AAPL Figure 2 Map of Uganda and districts. Arrow indicates the district of Wakiso, where this study took place. Source: (Wakiso District Local Government, 2009) 19 During this initial phase, DevOrg procured 1.2 million malaria rapid diagnostic tests. The mRDTs were produced by two manufacturers that make WHO-approved diagnostic devices. DevOrg negotiated with the manufacturers not only for the diagnostic devices, but also for a bundled service that would be an important aspect of this project. It was recognized that the success of scaling up mRDT sales in Uganda would require aspects other than the device itself. For that reason, the contract with manufacturers included provisions about waste management, medical detailing, training to providers, specific barcoding, and marketing/advertising to end users. The in-country supply chain for this project consisted of four layers before the enduser: manufacturer, first-line buyer (FLB), distributor, and retailers (Figure 3). In the system, there were essentially two parallel supply chains fueled by products from the manufacturer. Due to the nature of the contract, retailers were divided in half and distributors/FLBs of one product could not sell to the retailers in the supply chain of the other manufacturer. Further, only retailers that underwent the training program were allowed to sell the devices. Each supply chain had one manufacturer and one FLB, but one of the FLBs employed two distributors. Manufacturer 1 Manufacturer 2 FLB1 Distributor 3 Retailers I FLB2 Distributor 2 Distributora Retailers 2 Figure 3. Supply chain of mRDTs in Uganda in this study 20 Stock moves from manufacturers to retailers. The distributors were the agents who owned and stored the stock. They purchased it from the first-line buyers and are the entity that bears the cost of any unsold products. The distributors are also the ones responsible for handling most aspects of the bundled service, including the advertising, waste management, and training of providers. Each distributor was afforded a relatively high degree of freedom in how they went about these responsibilities, as long as they adhered to DevOrg's requirements. This study interviewed and held focus group discussions with agents in this project. Both first-line buyers and all three distributors were interviewed and 28 retailers were involved in focus group discussions. The study goal was to better understand how these agents made the decision to be a part of this pilot program, and more broadly, how they would make a decision about whether or not to sell/stock malaria rapid diagnostic tests. Multiple criteria decision analysis (MCDA) methodology was used to investigate this further. 2.5 Multiple Criteria Decision Analysis This study aimed to understand the preferences and tradeoffs that were inherent in the decision-making process of agents in the private sector supply chain of mRDTs in Uganda. This was a complicated endeavor for several reasons. First, there is a plurality of stakeholders involved in the decisions, from the manufacturers to the retailers. Each agent can make their own decision, but their respective decisions influence and shape other agents. Further, there is a collaborative component; the devices cannot make it to the end user without some sort of agreement between the agents. Finally, each agent was faced with a multiplicity of criteria to be considered, with tradeoffs between those criteria that needed to be considered before reaching a decision. For these reasons, it was determined that multiple criteria decision analysis (Keeney & Raiffa, 1976) would be an appropriate methodology to employ. Multiple criteria decision analysis (MCDA) is a methodology that draws from decision sciences, operations research, and economics. It is most appropriate for 21 complex decision problems where there are multiple objectives and no clear optimal solution. 2.5.1 History and Motivation of Decision Analysis It is proposed that the earliest known reference to MCDA (albeit not in so many words) was by Benjamin Franklin, in a letter describing a method that he used to make important decisions (Koksalan, Wallenius, & Zionts, 2011). The process involved making a list of arguments on both sides and crossing out the arguments that were relatively of equal importance, which was essentially a weighting process. Key groundwork for the methodology was laid through the 20th century by mathematicians and economists, including work on welfare economics, set theory, and utility theory (Figueira, Greco, & Ehrogott, 2005). Then, the 1970s saw many important contributions to theory and methods of MCDA and the 1980s an extensive growth of application of theory in real-life situations (Koksalan et al., 2011). Decision analysis was described by Keeney as "a formalization of common sense for decision problems which are too complicated for informal use of common sense" (p. 806, Keeney, 1982). The need to develop a rigorous methodology to approach decisions was realized in the reality of decisions faced in modern society. Many decisions are complex, involve multiple stakeholders, and require trade-offs between conflicting objectives. For example, the classic trade off between cost and quality. Often (not always) higher quality requires a greater cost. If one's objectives are to minimize cost and maximize quality, what is the optimal solution? Another example is a patient's consideration of what prostate cancer treatment to receive (Keller & Wang, 2009). Options could include surgery, chemotherapy, experimental drugs, or waiting. There are multiple objectives for the patient, including maximizing chances of success, minimizing harmful side effects, and minimizing life and career disruptions. The decision is further complicated by the fact that another stakeholder has somewhat different (although very overlapping) 22 objectives. The doctor treating the patient may have objectives that include minimizing costs, in addition to maximizing chances of recovery and minimizing side effects. It is possible that there are trade-offs between the decision options. For example, consider a treatment that has a higher chance of success in reducing tumor size, but comes with more unpleasant side effects. Features of complex decisions include: multiple objectives, difficulty identifying good alternatives, intangibles, long time-horizons, many impacted groups, risk and uncertainty, risks to life and limb, interdisciplinary substance, several decision makers, value tradeoffs, risk attitude, and sequential nature of decisions (Keeney, 1982). These intertwined characteristics provide the motivation and impetus for MCDA and decision science in general. 2.5.2 Applications of MCDA MCDA has been applied in public health interventions, especially in priority setting & (Baltussen & Niessen, 2006; Marsh, Dolan, Kempster, & Lugon, 2013; Nutt, King, Phillips, 2010), finance (Zopounidis & Doumpos, 2002), natural resource & management (Mendoza & Martins, 2006), sustainable energy (Wang, Jing, Zhang, Zhao, 2009), and more. MCDA has particular potential in the developing world context, where there are often multiple stakeholders and objectives, and complex decisions to be made (Stewart, Joubert, & Janssen, 2010). This is especially the case for NGOs and non-profits seeking to implement interventions. For example, if one wanted to procure solar lanterns and was deciding between options, it would be necessary to trade off between different criteria like recharge time, brightness, battery life, durability, price, cultural appropriateness, reliability of manufacturer, and more (Sanyal et al., 2015). It is often the case that there is no obvious "winner." MCDA can help provide a framework for making complex decisions like this. 2.6 Aim of this Research Study This study implemented MCDA in the context of scaling up rapid diagnostic tests for malaria. The goal of the analysis was to better understand a supply chain agents' 23 "willingness to stock," that is, to carry the product. The previously mentioned studies have evaluated willingness to use mRDT products, but there is limited literature on examining this other aspect. Further, MCDA is valuable in providing insight into the preferences and priorities of different private sector supply chain stakeholders, which can inform and shape public health interventions. This research study was performed on a small scale, with limited time and resources. One researcher traveled to Uganda and interviewed 2 first line buyers, 3 distributors, and 28 retailers as a representative sample of the population of individuals in DevOrg's project. The purpose of this research study was two-fold. First, to gather data and derive quantitative results about the decision making process for the agents in the supply chain when deciding to stock the devices. Second, another important result of this research was a proof of concept and pilot implementation of MCDA in the context of the developing world to rigorously analyze a complex and multifaceted situation. The rest of this paper describes the methodology and results and then offers a discussion on insights and suggestions for further research. 3 Vlethodology The methods used in this study were adapted from multi criteria decision analysis (MCDA) methodology. This section describes the methodology used, adaptations, and limitations. The data collection process will be described first and then the specific sections of the analysis will be explained. 3.1 Research Scope This research study was conducted on a very small scale. One researcher, accompanied by a research assistant, spent two weeks in Uganda collecting data that was analyzed in about a month. The purpose was to test a multi-criteria decision analysis methodology in international development contexts where time and data are often in short supply. It is common in situations like these to make 24 assumptions when necessary and adapt to missing information. This thesis serves to explore a modified multi-criteria decision analysis methodology as an approach to rapid evaluations in the developing world context. This thesis also highlights what future work could do to refine the initial pilot. 3.2 Data Collection Two mechanisms were used to gather data from respondents: semi-structured interviews and focus groups with a survey component. Semi-structured interviews were performed by the researcher on first-line buyers and distributors while the focus groups were done with retailers. 3.2.1 Semi-Structured Interviews with First-Line Buyers and Distributors A total of five semi-structured interviews with first-line buyers and distributors were performed. All the individuals in the research subset were participants in DevOrg's project to scale up mRDT use. All but one of the interviews was performed in-person in Uganda and one was on Skype because the distributor was out of town while the data was being collected. Follow up questions were addressed to first line buyers through email or Skype. Interviews consisted of an introduction of the researchers, a brief background on why this information was being collected and why it is important, and a question of consent. After the interviewees had consented, the initial round of questions were general, about their background, business, and general thoughts about working on this project. After this introductory phase, the interviewer would explain the nature of the study and that questions had been prepared for their consideration. An important aspect of this interview was to explain to the respondents that the questions were academic in nature and therefore may be abstract. The interviewees were assured that it may be confusing and they should not hesitate at all to ask for clarification. The goal of this was to ensure that respondents were actually understanding the question and providing accurate information that reflected their true opinions and thoughts. Additionally, to facilitate understanding, a variety of tangible examples were provided. 25 Throughout the interviews, it was necessary for the interviewer to act as an anchor, returning to the set of questions, to avoid prolonged divergence of topics. While some divergence was integral to the research, as it provided information about questions that the researchers didn't even realize were important, it was necessary to balance that with staying on task to gather the information necessary to appropriately conduct the methodology. It was critical for the interviewer to be able to synthesize and parse interviewee responses. For example, when asked a question about what factors are important when deciding whether to stock malaria RDTs, a respondent may reply with a long narrative on their experience and thoughts. It is necessary for the researcher identify criteria that can then be subsequently evaluated. This was done by the researcher conducting this study and then articulated to the interviewees. Then, the interviewees were asked to confirm if the summary was accurate. 3.2.2 Focus Group Discussions with Retailers Focus group discussions were used as a means of gathering information from the retailers. A group discussion in multi-criteria decision analysis methodology has the benefit of allowing participants to discuss and debate criteria. Further, this study had too short of a time frame to feasibly conduct a large number of surveys. Finally, in a focus group discussion, it was possible to get the reasonably undivided attention of a group of retailers for a block of time, as opposed to a survey, which had the challenge of trying to solicit information from an individual when they are in the middle of their work. However, a traditional focus group was not the ideal setting because the respondents were a mix of genders, business levels, experience, and retail types. This diversity led to the situation where some individuals were much more vocal than others. Even with specific encouraging, the majority of talking was performed by 2-4 dominant men. The women in the group were much more passive. In order to 26 ensure input from all parties, the focus group included a survey component. In order to rank their "happiness" for different attribute values and the overall weighting for attributes, respondents were asked to fill out a piece of paper that was handed out. The sheet of paper was not a form; it was a blank piece of paper where the researchers explained how to fill out the form by writing up an example at the front of the room on a large piece of paper. In the future, it is recommended that a form be made up ahead of time to pass out to respondents and reduce confusion on what to fill out. The researchers led the focus group discussions but a Ugandan moderator assisted that effort. This individual was introduced by partner organization DevOrg and was recommended as important to the success of the discussion group. This individual was briefed on the project to provide necessary background to effectively facilitate the focus group discussion. One recommendation for further work would be to work closely with a facilitator much in advance and run through the entire study with them to ensure understanding. One value that the facilitator did provide was clarification in the local language. Although all of the respondents spoke English very well, sometimes it was helpful to explain in their native tongue. 3.3 Methodology Framework The study was broken down into seven parts: 1. Decision framing 2. Identifying criteria 3. Eliciting value functions 4. Determining priorities (weighting) 5. Generation of decision matrix inputs 6. Evaluating options 7. Sensitivity analysis 27 Each of the six steps was performed for three levels of supply chain agents: first-line buyers, distributors, and retailers. Decision framing was performed to determine the set of options that were to be evaluated. It was done in advance of data collection, through expert opinion in consultation with DevOrg. The options considered were the same for all levels of the supply chain. Identifying criteria elucidated the considerations by which a decision maker evaluated his/her options. Value functions determined the marginal added value given an increase in performance on a criterion. Determining priorities was performed to understand the relative importance, or weight, of the criteria identified. A decision matrix was generated with estimates of inputs for each criterion across the options. Evaluating the options was then performed by applying the weighted average constructed from the value functions and weights to integrate all factors and understand the relative merit of the options. Sensitivity analysis was performed to understand the robustness of the data collected. This section will first describe the data collection methodology in terms of logistics (questionnaires, survey execution, etc.) Then, the rest of this section will expand on how the seven aforementioned steps were implemented. 3.3.1 Decision Framing Decision framing was performed before interviews and focus group discussions with stakeholders, in consultation with DevOrg representatives who could provide expert opinions. The overarching goal of this study was to understand how agents decide to stock and sell malaria rapid diagnostic tests (mRDTs). This also included how the stakeholders consider intervention alternatives provided by NGOs, like DevOrg, and also the consideration of selling WHO versus non-WHO approved devices. The flow chart of decision options is described in Figure 4. 28 Sell? No Yes Sell WHO approved? No + Yes Sell in bundled service? No Yes Figure 4. Decision flow chart for agents in the supply chain. The stars represent a decision option. A comprehensive set of decision options was put together. The options under consideration were: 1. Sell WHO approved mRDTs outside of DevOrg's bundled service 2. Sell mRDTs through DevOrg's bundled service 3. Sell non-WHO approved mRDTs 4. Do not sell mRDTs At the time when first-line buyers (FLBs), distributors, and retailers were signing on to DevOrg's project, these are the exhaustive set of options available to them. Selling non-WHO approved devices through the bundled service was not an option because the contract required agents to only sell the devices provided through selected manufacturers, which were all WHO approved. All three levels in the supply chain, first-line buyers, distributors, and retailers faced this consideration. Manufacturers only had two options: to participate or not to participate in the bundled service, as they obviously already had the product they were making. Manufacturers were not included in this analysis due to time constraints but further research should consider including them in this type of analysis because many of their decisions are 29 complex and an MCDA methodology could be valuable. The rest of the study was centered on the four decisions and the options were presented in each interview and focus group discussion. 3.3.2 Identifying Criteria Criteria are defined as the considerations an individual makes when deciding on an action. In MCDA literature, the term attribute is often frequently used; both words are used interchangeably in this paper. Ultimately, the criteria are derived from one's fundamental objectives. Fundamental objectives are the goals one strives to meet or achieve when making the decision. For example, in public health, the fundamental objectives may include minimizing cost, minimizing mortality, maximizing a health outcome, providing assistance to those most underserved, etc. & Criteria should be unambiguous, direct, operational, and understandable (Keeney Gregory, 2005). Unambiguous means that a clear relationship exists between the criteria and a consequence. Direct means that the criteria levels directly describe the consequence of interest. Operational refers to information that can be obtained to reasonably describe trade-offs and consequences. Understandable means that the criteria and impact of criteria can be easily understood and communicated. Furthermore, criteria should be non-redundant, otherwise some may receive disproportional weight (G. Montibeller &Franco, 2007). For example, maximize profit and maximize revenue should not be two criteria evaluated together because profit takes revenue into account. An alternative would be to have maximize revenue and minimize cost a$ two criteria. Criteria can be measured by three types of metrics: quantitative direct, quantitative indirect, and qualitative (G. Montibeller & Franco, 2007). Quantitative direct attributes directly measure the criteria in question; examples include a company's profits or market share. Quantitative indirect attributes serve as proxies for what one is trying to measure; for example, a Richter scale for expressing the severity of an earthquake or pregnancy rates for expressing unprotected sex. Qualitative attributes measure a set of levels-like high, 30 medium, and low-for criteria. One example includes creating a positive public image. Both quantitative and qualitative attributes were used in this study. 3.3.2.1 Retailers In this study, the respondents identified the criteria. The researchers had been informed by literature reviews and expert advice and were prepared to offer suggestions if respondents were reticent. This was balanced with the care to not direct the direction of the conversation. For the focus groups, the whole group was asked to think about what are the factors they took into consideration when deciding whether or not to sell malaria rapid diagnostic tests and whether to enter into this bundled service. Concrete examples that were relevant to their experiences were given to ensure that the question was understood. For example, one scenario described was to think about how one makes a decision about what shirt to buy. When deciding between a few options, there are several important criteria that come to mind (Figure 5). Criteria Cost Brand Decision: Choose the best shirt Description Maximize quality: What is the quality of the shirt? Does it look like it is going to last a long time? Minimize cost: How much does the shirt cost? Compliment colors: I like colors that are darker because they look better on me Maximize flattering fit: I want a shirt that will fit well and look good on me Recognizable brand: I want the brand recognition Figure 5. Example given to respondents to elicit criteria Concrete examples like this one were given to ensure that the respondents understood the type of question that they were being asked. Individuals raised their hands and offered ideas. Often it was necessary for the researcher to clarify what was said and summarize into a criterion. Always the moderator confirmed with the original speaker to ensure that the summary was correct. After the group had brainstormed for several minutes, the moderator offered some of the criteria that 31 had been generated before the discussions by expert opinion. Respondents generally agreed with those criteria also. The criteria to be included in the analysis were generated by the first focus group discussion. It was a lengthy process, so in order to keep the attention of the group it was important to minimize the length of the discussions. Thus, for the second focus group, the list of criteria that had previously elicited was presented to the group. The respondents were first asked if that list was comprehensive or if there was anything they would like to change. The group did add two criteria: time to complete a sale and opportunities. They removed one criterion: quality. Table 2 lists the criteria for retailers and the definition. The researchers did remove the attribute quality from the analysis but an improvement in the methodology would be to include criteria that may have been dismissed by some of the group. It is possible that there were individuals in the focus group discussion that disagreed with quality being removed but were reluctant to speak up in the group setting. In the future, attributes that were less important to some retailers would be reflected by the weighting procedure. 32 Table 2. List of Criteriaand Definitions for Retailers Criteria Definition Quality The quality of the device as measured by its WHOapproval status, durability, and accuracy Price device is sold Price at which the device is sold to customers Cost of device The amount that the device costs the retailer to purchase Delivery time Number of days after an order is placed that a distributor delivers new supply Sales of other products Increase in the sales of other products Customer satisfaction How happy a customer is after a business interaction Training The amount of training that the retailer receives on how the device works, why it is important, and how to administer it Volume sold Number of devices sold per month Awareness/advertising Amount of advertising and awareness that is generated in consumers Time to complete sale* Amount of time it takes to complete a business transaction Opportunities* Other opportunities, including free apparel, small jobs, and leadership *Criteria added by the second focus group discussion After the criteria were established and defined, metrics were determined to evaluate the criteria (Table 3). This included both qualitative and quantitative metrics. The metrics directly measured the criteria defined in Table 2. Qualitative metrics were evaluated on a high to low scale, with each interval defined. To add more granularity, five intervals were defined on the qualitative scale: high, medium- 33 high, medium, low-medium, and low. For any attribute that had currency, Ugandan shillings were used in discussions with respondents and then converted to U.S. dollars (USD$) with the exchange rate of $1 USD equal to 3,000 Ugandan shillings. Table 3. Criteria and metric units Criteria Metric Quality Qualitative, high-low Price device is sold Ugandan Shillings Delivery time Days Sales of Other Products Qualitative, high-low Customer Satisfaction Qualitative, high-low Training Qualitative, high-low Volume sold Number of kits sold per month Awareness/ads Qualitative, high-low Time to complete sale Minutes Opportunities Qualitative, high-low Cost of device Ugandan Shillings 34 The different levels were defined and explained to retailers before soliciting responses (Table 4). Table 4. Qualitative Metric Definitions for Retailers High Medium-High Medium Low- Low Medium Quality WHOapproved, very durable, very accurate WHO-approved, durable, Accurate Not WHOapproved, but somewhat accurate, and somewhat Not-WHO approved, not very accurate, not very durable durable Sales increase somewhat Sales increase a little Not-WHO approved, not accurate, not durable Sales of other products Sales are increased by a lot Sales are increased by a good amount Customer Satisfaction Customers are very happy Customers are somewhat happy Customers are neutral Customers are somewhat unhappy Customers are unhappy Extensive, Pretty good and comprehensive training Some training, not comprehensive A little Very little or no training Good advertising, thorough or in multiple media sources Some advertising in one media source Little or less Very little or no effective advertising advertising Good opportunities in both frequency and value Some opportunities Little or less valuable opportuniti Very little or nothing Sales increase very little or not at all Training comprehensiv e training on all aspects relating to training, missing detail mRDT Awareness/ Advertising Extensive advertising in multiple media sources (radio, TV, etc.) Opportunities Many opportunities that are valuable es Part of defining criteria included articulating the ranges, the upper and lower bound, over which the decisions are made. This is very important because otherwise value trade-off is distorted (G. Montibeller & Franco, 2007). For example, in the shirt case, the price range of the shirts being considered may be from $2-$10 dollars. Thus, 35 when considering this criterion, it is important to frame the decision in terms of these specific ranges. Ranges were elicited by the first group and then used again on the second group (Table 5). Table 5. Criteria and ranges for retailers Criteria Low Value High Value Quality Low High Price ($USD) 0 $1.67 USD Time to delivery (days) 7 1 Sales of other products Low High Customer Satisfaction Low High Training Low High 0 50 Low High 0 60 Low High 0 $1 Volume (kits sold per month) Awareness/ads Time to complete a sale (minutes) Other Opportunities Cost 3.3.2.2 Distributors Data were collected from distributors through individual interviews. Although the education of these individuals was greater, the type of thinking required for MCDA is abstract and can be confusing the first time one sees it. Thus, tangible examples were helpful in clarifying questions asked. Examples like choosing a shirt was provided to these individuals as well. Since the level of education of these individuals was greater, it was possible to provide a general explanation of value and preferences. Thus these interviewees could confirm a relative increasing or decreasing value over the range of an attribute. As described previously, it was challenging to keep the individuals focused on the task at hand. Many of them came into meetings with preconceived notions about 36 what the interview was about and it was necessary to explain multiple times the goal of the research. In this case, criteria generated in advance of meetings by expert opinion was less helpful because respondents were much more varied in the criteria that was important to them than the retailers were. Further, it was necessary to work around individuals' intuition about how they think through business decisions. For example, in multiple interviews, the interviewee wanted to talk about margin, instead of breaking it down into segmented costs, volume, and price. This was likely because in their everyday interactions, they have a natural sense about and talk in terms of margins. This was the case not only for distributors but also FLBs. Furthermore, there was a challenge in unifying the responses that the various distributors and FLBs provided. They thought in terms of different units; for example, for volume, some individuals explained their criteria in terms of devices sold per month, or devices sold by next year, or devices sold up until this point. If necessary, assumptions and interpretations were made to convert the units to be the same for responses across different agents (see Inputs, Assumptions, and Analysis section). Distributor and FLB responses were not aggregated. Table 6. List of Criteria and Definitions for Distributors Criteria Definition Volume Kits sold per month by the distributor Expiration date Months left to expiry when the kit is sold Profit Percent profit Cost per kit Cost to the distributor to purchase the kit Cost per training Cost for the distributor to hold one training session Efficiency of distribution Boxes delivered per delivery trip Cross selling Percent increase in sales Table 6 lists the criteria identified as important to the distributors. Of the three distributors interviewed, one was unable to provide responses because of a lack of understanding and willingness to participate. Thus, that interview was recorded as 37 a non-response and data from the two other distributors was collected. Like retailers, the researcher elicited upper and lower bounds of the criteria from the distributor. The distributors had the same ranges for all the criteria except for two. For cost per training, the upper bound was either $500 or $100 and for cross selling the upper bound was either 50% or 20% (Table 7). Table 7. Criteria and ranges for distributors Upper Bound Lower Bound 18,750 0 22 3 Profit margin (%) 50% 0% Cost per kit ($USD) $0.83 $0 Cost per training ($USD) $2,000 $0 Efficiency of distribution (Boxes per delivery) 500 (or 100) 1 Cross selling (% increase in sales) 50% (or 20%) 0% Criteria Volume (kits sold per month) Expiration date (months left to expiry at sale) 3.3.2.3 First Line Buyers Responses from FLBs were solicited in the same manner as distributors. In the case of missing data, the individuals were followed up with online through email and/or Skype. The same issues that arose in distributor issues also occurred when speaking with the FLBs, especially the difficult time differentiating preferentially independent options (see Discussion section). The FLBs had the same criteria except one FLB had percent increase in sales, which the first did not mention. Table 8 summarizes the criteria and definitions used for the FLBs. Table 9 summarizes the criteria and ranges and Table 10 explains the definitions used for the qualitative metrics. 38 Table 8. List of Criteriaand Definitions for Distributors Criteria Definition Cost per kit Cost that the FLBs pay to purchase the kits Price per kit Price at which the FLB sells the kit to the distributor Quality Quality of the kit in terms of WHO-approval status, accuracy, and durability Administrative time Hours per week of administrative time that goes into managing the sale of kits Profit Percent profit Relationship with donor Relationship with any donor organization or NGO organization/funder Increase in sales* Percent increase in sales *Only for one of the FLBs Table 9. Criteria and ranges for FLBs Criteria Upper Bound Lower Bound Cost per kit $0.033 1 Price per kit $0 $1 Quality H L Administrative time 1 hr. 60 hr. Profit 0% 50% (or 100%) H L 0% 50% Relationship with donor organization/funder Increase in sales 39 Table 10. Qualitative Metric Definitions for FLBs Quality High Medium-High Medium Low-Medium Low WHO- WHO- Not WHO- Not-WHO Not-WHO approved, very durable, very accurate approved, durable, Accurate approved, but somewhat accurate, and somewhat approved, not very accurate, not very approved, not accurate, durable not durable No relationship durable Relationship with donor Very strong relationship, collaborate and communicate extensively Strong Somewhat relationship, strong Collaborate somewhat or collaborate on some projects relationship, may have have communicated collaborated on one project 3.3.3 Eliciting value functions An important part of this analysis was evaluating how an individual's marginal added value changed given an increase in performance on a criterion. It was not assumed that two incremental changes would result in the same net value change; rather it is possible that value functions are non-linear. Value functions plot value as the dependent variable and the criterion as the independent variable and the resulting shape of the function allows one to understand how the rate of change across the criteria compares with the rate of change in value. In doing this, one can understand over what ranges an incremental increase or decrease is most important. There are two traditional means of eliciting value functions: bisection and direct rating (G. Montibeller & Franco, 2007). Bisection works by asking participants to identify the midpoint between two anchoring values of a criterion. Then, one can determine whether the increase from the low to the mid value is more valued versus the mid to highest value. Direct rating requires respondents to apply a score to various points of the criteria. In this study, the primary method employed was direct weighting. Respondents, both retailers and FLBs/distributors, were asked to 40 provide a numerical score for value for several points. The upper and lower bounds on the criteria were assigned the highest and lowest (depending on whether the objective was to maximize or minimize that particular criteria) values. The value of certain criteria points between the boundaries was then solicited. Piecewise linear interpolation was used to create a continuous function between the criteria and the value. 3.3.3.1 Retailers The bisection method was tried at first, but the retailers did not understand what was being asked of them. Instead, the moderator asked the retailers to rank their "happiness" on a scale of 1 (very unhappy) to 5 (very happy) for specific points of the criteria (Figure 6). As described in the "Soliciting Responses" section, this information was gathered at the individual level for each respondent on a piece of paper. For example, the moderator asked the group how happy they would be if their store sold 10 malaria rapid diagnostic tests per month. The respondents were instructed to report a value of 1 to 5 (very unhappy to very happy). For each criterion, two points to be evaluated were given to the retailers. When the range was defined, the moderator indicated that the lowest range would score a 1 and the highest would be a 5. All other values could be within that range. The median score of all retailers per criterion was calculated and this figure was used to construct a value function. The median was chosen to reduce the effects of outliers. A total of four data points (the lowest and highest, and two in the middle) were plotted on a graph of the range of the criteria versus value. Linear piecewise interpolation was conducted to create a continuous function. Numerical Score 5 4 3 2 1 Description Very happy Somewhat happy Neutral Somewhat unhappy Very unhappy Figure 6. Numerical and verbal descriptions of happiness to elicit value functions 41 3.3.3.2 Distributorsand First-Line Buyers The direct weighting method was the primary method used to elicit values from distributors and FLBs. The researcher first defined a 1 and a 5 on the value scale, corresponding to the highest and lowest amount of the criteria specified in the identifying criteria section, and explained in terms of "satisfaction," instead of "happiness," which was more appropriate for distributors. The researcher would ask the individual to rank their "satisfaction or happiness" on the 1-5 scale for different levels in the criteria. Sometimes the distributor offered information for another point and the researcher recorded this information also. Two value functions were constructed, one for each distributor. One complication with the interviews was that there was a variety of ways that the respondents thought about the answers to the questions asked them. For example, when asked about the volume of sales, some answered in terms of volume of mRDTs sold up until this point, others in terms of the number of mRDTs sold in two years, and still others explained volume in terms of mRDT sales per month. To compare responses, the values were converted to the same unit. Furthermore, it was necessary to draw a distinction between criteria that had been determined in advance of the intpruiePAs5, bythe researchers through expert intprviepAws and literature, and emergent criteria that the respondents brought up over the course of the conversation. As much as possible the emergent criteria were incorporated into the methodology, however, it was not always possible to define ranges and metrics in the short amount of time that was allocated for interviews. In this case, the respondents' thoughts were recorded and will be considered qualitatively (see Discussion section). 3.3.4 Determining Priorities (Weighting) Determining the relative importance of the attributes and the trade-offs between them is critically important in a multi criteria decision framework. In this study, trade-off analysis was performed as described in Montibeller & Franco (2007) using the swing weights method. Respondents were asked to think about their most 42 important considerations when making the decision on whether or not to stock mRDTs and whether to participate in the bundled service. Respondents (both retailers and distributors) were asked to consider a decision option that ranked the lowest in all of the attributes previously defined. The moderator listed a few examples, including selling zero devices, having the highest cost, etc. Respondents were then asked to identify the most critical attribute they would need to improve first. This attribute was given a value of 100. Then, respondents were asked to consider subsequent (second, third, etc.) most critical criteria to improve and assign them a relative weight ranging from 0, not essential at all, to 100, the most essential. As was done previously, examples were given to the respondents to illustrate the concept. Both new examples and examples previously described were provided. It was helpful to have a few examples that carried through the different sections of the study. For example, after having used the shirt example to solicit criteria and elicit value functions, it was also used to define weights. The researcher made a point emphasize that there was no "right" answer and that someone could give a value of 90 to a criteria that someone else gave a 50 and that was perfectly fine. An alternative method to solicit weights was considered where participants have a set number of points and they would be free to allocate the points across the criteria. However, the first attempt at this, in an interview with a first-line buyer, was not productive. The individual was quite confused about the concept. When the alternative method of assigning each criteria a score from 0-100 was proposed they seemed much more comfortable. For consistency, this method was used for subsequent interviews; however, later respondents did show familiarity with the fixed-number of points method alternative. Thus, it may be worth considering this method, and taking the time to explain it fully, in the future. One reoccurring issue in the assigning of weights was that respondents very rarely ranked anything below 50. One retailer had a spread of 100-98 for weights, which provides virtually no granularity. Even though the examples provided assigned 43 weights from 10-100, very few individuals emulated that sort of spread. See Discussion section for speculation on biases that may have prompted these results. For the retailers, the weights were aggregated by calculating the median of the responses. The distributor and FLB weights were not aggregated. The weights were normalized over the sum such that the weights added to 100%. 3.3.5 Generation of decision matrix inputs A decision matrix was constructed consisting of inputs for each of the options across the criteria. These inputs were meant to be as reflective of reality as possible. For example, the cost to retailers was estimated across the four decision options. Option 2 (selling mRDTs through the bundled service) was established; the cost of the devices for the other three options then had to be estimated. The purpose of constructing the decision matrix is then to input those figures into the value functions and thus convert to one unit (value) for each of the criteria. Then, a weighted sum can be calculated based on the weights. A decision matrix was constructed for each agent in the supply chain. Ideally, the decision maker would be involved in generating the inputs for the decision matrix. However, due to time constraints, it was not possible to solicit this information from the respondents. Rather, for each input, a combination of literature sources, expert opinions from MIT faculty and DevOrg representatives, and information from interviews was used to estimate inputs. To account for the uncertainty in these assumptions a high and low estimate was also recorded. See Inputs, Assumptions, and Analysis section for information behind the assumptions for each estimate. 3.3.6 Evaluating options An overall value was calculated for each option that was being considered. The numbers in the decision matrix were used as inputs to the value functions. Since one continuous value function was plotted, it was possible to calculate a corresponding value for each discrete criterion input using the equation of the line where value is 44 the dependent variable and the criterion is the independent variable. Each section of the function was assumed to be linear. For example, if cost was $0.55 per kit, the equation of the line connecting the two solicited points was used to calculate the value when cost was $0.55. Each value was multiplied by its corresponding weight. Thus, a weighted sum can be generated for each option. The overall value can be compared across the options; theoretically the option with the highest value is reflective of the best choice for that individual, taking into account preferences and priorities. 3.3.7 Sensitivity Analysis High- and low-end estimates were also generated, in addition to the base estimate, for each of the criteria and options in the decision matrix. These values were inputted into the weighted sum to evaluate the robustness of the results. Especially because there is a fairly high level of uncertainty in the input assumptions, due to limited data from agents and conflicting information, it was important to compare a range of assumptions. Further, a sensitivity analysis was performed on the weights. The weights for each attribute were varied from 0-100% and the impact on the overall value was determined. That is, the overall value of each of the four options was plotted as a function of the variation of weight. This analysis was done by plotting two values and then drawing a line through the two points (Montibeller & Franco, 2007). One value was the weight that was elicited through the data collection process, with its corresponding calculated overall value. The second value was set as if the weight for the attribute was 100%. The overall value, therefore, would be the criterion's value that was determined from the value function. A straight line can be drawn through these two points to represent the variation of weights. The analysis considers when the options change positions, when the best choice is swapped with another option. 45 4 Inputs and Assumptions The methodology used required certain input assumptions to be made in order to run an analysis and examine the best options for the different agents. Especially in the context of the developing world, there is limited and unreliable data. Thus, it was necessary to sometimes make estimations. This section describes the numbers used and the assumptions and rationale behind them. 4.1 Decision Input Matrix The decision input matrix is the summary of the numerical values for each of the options for each of the criteria identified. These figures will be inputted into the value functions to determine the corresponding value for use in the weighted sum. The inputs for the decision matrix were derived through a combination of expert interviews, literature, and observation while in Uganda. Further, a high and low estimate was also generated for analysis to examine the sensitivity of the overall value to the uncertainty in the input values. This section provides detail about the assumptions behind each input. 4.1.1 Decision Input Matrix - Retailers The comprehensive list of inputs for the retailer decision matrix is summarized in Table 11. Some criteria were not relevant for Option 4; those were marked as N/A and explained in the corresponding section. 46 Table 11. Input figures for retailers for high, base, and low estimates for options 1-4 Quality Option 1 Option 2 Option 3 Option 4 Price ($USD) Option 1 Option 2 Option 3 Option 4 Delivery Time (days) Option 1 Option 2 Option 3 Option 4 Sales of other products Option 1 Option 2 Option 3 Option 4 Customer Satisfaction Option 1 Option 2 Option 3 Option 4 Training Option 1 Option 2 Option 3 Option 4 Volume (kits sold per month) Option 1 Option 2 Option 3 Option 4 Awareness/Advertising Option 1 Option 2 Option 3 Option 4 Cost (cost per kit, $USD) Option 1 Option 2 Option 3 Option 4 Time to complete sale (minutes) Option 1 Option 2 Option 3 Option 4 Opportunities High Estimate Base Estimate Low Estimate H H LM N/A MH H L N/A MH MH L N/A $1.3 $1.3 $1.2 $0 $0.75 $0.83 $0.88 $0 $0.60 $0.60 $0.50 $0 2 1 3 N/A 3 2 4 N/A 4 3 5 N/A MH H M L M MH LM L LM M L L MH H M L M MH LM L LM M L L M H LM L LM MH L L L M L L 20 40 20 0 10 20 10 0 5 10 5 0 M H LM L LM MH L L L M L L $0.61 $0.61 $0.56 $0 $0.35 $0.39 $0.41 $0 $0.28 $0.28 $0.24 $0 30 40 30 0 20 30 20 0 10 15 10 0 47 Option Option Option Option 4.1.1.1 1 2 3 4 M H LM L LM MH L L L M L L Quality When explaining quality to the respondents, it was explained that quality depends on WHO-approval, accuracy, and durability of the device. In the base assumption, devices in Option 2 were assigned as high because they were WHO approved and further, vetted by DevOrg. Option 1 was assigned a value of medium-high because it is assumed that this option would carry a variety of devices ranging from mediumhigh to high and there is no expertise to differentiate the two. Option 3 was assigned as low because the devices would not be WHO-approved. Option 4 was eliminated from the analysis, because if there were no devices being sold, quality is not relevant. The high and low-end assumptions decreased Option 2 to being MH and increased Option 3 to LM. 4.1.1.2 Price Price was established from the ACTWatch Outlet survey (ACTwatch Group, Program for Acessible Health, Communication & Education (PACE), & The Independent Evaluation (IE) Team, 2012). The ACTWatch survey provides data on the average price of mRDTs in clinics and drug stores that sell both WHO and non WHOapproved devices. A weighted average was taken of these two prices based on the number of clinics and drug shops in the survey sample. The two pharmacies were counted as drug shops, as ACTWatch had no data on pharmacy pricing and the one unknown identity was removed. Option 2 was determined from interviews with DevOrg. Option 4 was set as $0 because if the retailer is not selling the device, the price is zero. The high estimates were increased so that they were closer to the highend of the ranges in the value functions and the low-end was decreased by $0.100.20. 48 4.1.1.3 Delivery Time In the focus group discussions, retailers expressed delivery times ranging from 1 to 3 days, so 2 days was set as a base assumption for Option 2. Option 1 was assumed to have a longer delivery time because there is not the established supply chain that Option 2 has, so it was set for 3 days. Option 3 was assumed to take even further because there are even less established networks for moving non-WHO approved devices. Delivery time for Option 4 was eliminated from the analysis because not carrying the devices would render delivery time irrelevant. The times were varied by adding or subtracting one day for the high and low-end estimates. 4.1.1.4 Sale of Other Products Sales of other products inputs were established by assumptions based on conversations in the focus group discussions. Drug shop sellers informed the researchers that sales of other products in their shops had increased significantly, some said their other sales had doubled or more. This could be because these shops that are now offering mRDTs have increased their services so customers may be more likely to come into the store. Further, the drug stores received advertising/marketing materials, which may have contributed to an increased volume of customers in the store purchasing products. Thus, Option 2 was assigned as medium high. Option 1 was assigned medium because it was assumed that selling high quality products would result in repeat customers but the lack of advertising would result in less sales than Option 2. Option 3 was set as low-medium because it was assumed that selling a new product would attract some increase in sales but not as much as options 1 and 2. Option 4 was assigned as low because not selling the mRDTs would mean there would be no corresponding increase in other sales. High and low-end estimates increased and decreased the estimates by one increment except for Option 4. 4.1.1.5 Customer Satisfaction Customer satisfaction inputs were derived from expert interviews and focus group discussions. It was assumed that two factors of customer satisfaction include quality 49 of the product and assistance from the retailer. Option 1 was set as medium because the retailer would be able to sell a high quality test but would be unable to actually administer it. Option 2 was set as high because in the bundled service, the retailer sells a high quality device and is trained on how to administer the diagnostic and then provide advice based on the results. Option 3 was assumed to be low because a low quality product is offered and Option 4 was also set as low because the device is not offered at all. The high and low-end estimates increased and decreased the estimates by one increment except for Option 4. 4.1.1.6 Training Training inputs were determined based on the amount of, thoroughness, and comprehensiveness of outside education that the retailer is provided with. Option 2 as a base was assigned as a value of medium-high because the distributors in the bundled service were required to provide training. For the high-end estimate, this was increased to high and for the low-end it was decreased to medium, representing the range in quality of training. Option 1 was assigned as medium-low because it is possible that a distributor providing WHO-approved diagnostics would take the time to provide some education to retailers. Option 2 ranges were from medium to low. Option 3 was set at low for a base and had a high-end estimate of medium-low. Option 4 was low because the lack of the device being sold is very unlikely to result in training for the retailer by either distributors or NGOs. 4.1.1.7 Volume Sold Volume sold for Option 2 was assigned a base value of 20 kits per month sold, which was based both on what the retailers said in focus group discussion and on how many devices the distributors had sold up until this point. It was assumed that Options 1 and 3 would have less volume because there are no corresponding demand generation activities in those options, and thus the values were set at 10 kits sold per month. Option 4 was set at 0 kits sold per month. The high and low-end variations in volume were determined by doubling and halving, respectively, the base assumption. 50 4.1.1.8 Awareness/Advertising Awareness and advertising depend on the level of outside assistance or effort that was put into demand generation activities, like marketing and education of consumers. Option 2 had the highest value for these inputs because the bundled service obliged the distributors to invest in advertising and marketing. Option 1 was assigned a base value of low medium because generalized efforts by the public health community would increase awareness to some extent. Option 3 was assigned as low because less advertising would be focused on promoting non WHO-approved devices. High estimates increased the base assumptions by one increment and low decreased by one increment. 4.1.1.9 Cost Determining cost inputs was based on assumptions about margin. The cost of the device for Option 2 was known from interviews on the ground and the margin was calculated from price and cost. Then, to determine inputs for the other two options, it was assumed that margin would be the same and from that information, cost could be determined from the price known from the ACTWatch survey (ACTwatch Group et al., 2012). The high and low estimates were derived from the price high and low values. 4.1.1.10 Time to Complete a Sale Inputs on time to complete a sale were based on focus group discussions with retailers. In conversations, retailers relayed that they spent 30 minutes on a sale with a customer when selling an mRDT involved because it took time not only to convince the customer about the necessity of being tested, but also wait for the test results, and then provide advice about what medicines to purchase. It was assumed that Options 1 and 3 would take less time because there was no outside agency pushing for the use in those options, and if he/she wanted to, the retailer could sell antimalarials without a diagnostic. Option 4 was 0 because there would be no time spent selling the diagnostics. The high estimates added 10 minutes to each input value and the low estimates halved the base assumptions. 51 4.1.1.11 Opportunities Assumptions about extra opportunities that would be available to the retailers were based on focus group discussions with retailers and expert opinion. For Option 2, the extra opportunities was inputted as medium-high because DevOrg is actively running an intervention and so is able to provide tangible incentives like free shirts and small jobs. Option 1 was assigned as low medium, because while there is no active intervention in that option, it is possible that since the retailer is selling a high quality product an NGO or the local ministry of health would be interested in partnering or offering incentives. Options 3 and 4 were set as low for opportunities because there are neither associated interventions nor expected support from the global health community for selling low quality devices or not selling at all. The high and low-end estimates increased and decreased the estimates by one increment except for Option 4. 4.1.2 Decision Input Matrix - Distributors The comprehensive list of inputs for the distributor decision matrix is summarized in Table 12 and assumptions explained in the subsequent sections. Some criteria were not relevant for Option 4; those were marked as N/A. For cross selling and efficiency of distribution, two sets of values was inputted, one for each distributor. 52 Table 12. Input figures for distributors for high, base, and low estimates for options 1-4 Volume (kits/month) Option 1 Option 2 Option 3 Option 4 Expiration date (months) Option 1 High Estimate Base Estimate Low Estimate 80 3,200 60 0 40 1,600 30 0 20 800 15 0 12 8 6 Option 2 9 3 3 Option 3 Option 4 Profit (%) 12 N/A 8 N/A 6 N/A Option 1 30 20 10 Option 2 10 5 1 Option 3 35 20 5 Option 4 Cost per kit ($USD) Option 1 Option 2 Option 3 Option 4 Cost per training ($USD) Option 1 Option 2 Option 3 Option 4 Efficiency of distribution (boxes per delivery) Distributor 1 Option 1 0 0 0 $0.88 $0.30 $0.95 $0 $0.78 $0.25 $0.78 $0 $0.73 $0.20 $0.65 $0 $400 $1,700 $0 $0 $100 $1,500 $0 $0 $0 $1,300 $0 $0 40 20 10 Option 2 Option 3 10 35 5 20 1 8 Option 4 Distributor 2 Option 1 Option 2 Option 3 Option 4 Cross selling (% increase in sales) N/A N/A N/A 200 20 160 0 100 5 80 0 50 1 40 0 Option 1 Option 2 50 32 25 16 12.5 8 Option 3 Option 4 27 0 18 0 6 0 53 4.1.2.1 Volume Volume estimates for distributors were based on conversations from distributors and the 2009 ACTWatch supply chain survey (Palafox et al., 2012). One distributor told researchers that they are currently selling 1,600 kits per month, so this was taken as the base assumption for Option 2 volume. The ACTWatch survey reported that the median of sales in a week for wholesales was 10 devices. Thus, it was estimated that sales per month was about 40 devices sold per month and this estimate was used for a base assumption for Option 1. Option 3 was not differentiated in the survey, and so it was assumed that there would be slightly less volume of non WHO-approved devices sold and the volume was set at 30 for a abase assumption. The high and low-end estimates were established by doubling and halving, respectively the base assumptions. Option 4 was set at 0 because no devices are being sold. 4.1.2.2 Expiration Date The base estimation for Option 2 was set at 3 months because currently in the project, sale volumes are too low to result in all devices being sold before the expiration date. Thus, at this rate and if nothing changes, the bulk of the stock will expire or be given away at its date of expiry. If sales increase quite a bit, a high-end estimate for Option 2 was set at 9 months and the low-end stayed at 3 months because it was the lowest option on the distributor's value functions. The expiration dates for Options 1 and 3 were set at a base of 8 months, higher than Option 2 because it was assumed that distributors would not procure more devices than they had historically sold and thus would run much less of a risk of stock expiring. One distributor reported that selling a device at 6-8 months would be reasonable in normal business operations. The high and low-end estimates were set at 12 and 6 months. Option 4 expiration dates were eliminated from the analysis because it is not relevant. 54 4.1.2.3 Profit Base profit estimates for Options 1 and 3 were based on the ACTwatch survey (Palafox et al., 2012), which reported markups for distributors as 20%. Option 1 high and low-ends were set at plus or minus 10% profit. It is not clear from the survey whether non WHO-approved devices have a higher or lower markup, so the ranges were set wider than Option 1 at 35% and 5% margins. Option 4 was 0% because no products are being sold. 4.1.2.4 Cost of Devices The mRDTs are subsidized by DevOrg in Option 2. The distributors reported costs ranging from $0.30 to $0.20 in costs, so those figures were used in the high and low estimates and the average was taken as a base assumption. The ACTwatch 2009 supply chain survey reported that wholesale prices for mRDTs were $0.78 per kit (Palafox et al., 2012). This value was used as the base assumption for Option 1 and the reported range of responses were used as the high and low estimates. It was not clear from the ACTwatch survey whether non WHO-approved devices would cost more or less than the WHO-approved ones. Thus, the base cost was set as the same as Option 1 ($0.78) and the upper and lower bounds were wider, from $0.95 to $0.65. Option 4 was set at 0 because distributors are purchasing no mRDTs. 4.1.2.5 Cost of Training Cost of training for Option 2 base assumption was set at $1,500, which was an average of the two costs reported by distributors. The high-end and low-end estimates were the two reported values, representing the range of actual costs by distributors. Cost for Option 1 was set at $100 because distributors reported that they would consider holding some sort of training for retailers to sell devices. The high and low-end estimates for Option 1 was set at $400, if this optional training was held, and $0, if the training was not held. Option 3 was set at $0 because it is unlikely that a distributor or NGO would provide training on non WHO-approved devices. Option 4 was set at $0 as there are no devices being sold. 55 4.1.2.6 Efficiencyof Distribution Efficiency of distribution was measured in terms of boxes per delivery. Since the two distributors had such different value functions, implying that they operated on different levels, it was assumed that the input assumptions would be different for the two. Distributor 1 had a value function ranging from 1 to 100 boxes per delivery. Distributors reported that currently, delivery was very inefficient and they were moving low levels of stock to replenish retailers. Thus, a base assumption was set at 5 boxes per delivery, with a low of 1 and a high of 10 boxes. Option 1 was set at a base of 20 boxes per delivery because it was assumed that distributors are not necessarily obliged to provide replenishment if it was inefficient, as opposed to in Option 2 where the distributors are part of an intervention and working with a partner to ensure supply of the devices. Option 3 was set as slightly less than Option 2 because there may be less demand for lower quality devices. Option 4 was eliminated from the analysis because no devices are being delivered. Distributor 2 had a value function ranging from 1 to 500 boxes per delivery. Option 9 hac simntinns Were thp same fnr the hAM distrihbitnrs hPcausiP thpy are hnth in the same intervention. However, Distributor 2 reported possible higher volumes than Distributor 1 so the high-end estimate for Distributor 2 was set at 20 boxes per delivery. Options 1 and 3 were based on the same assumptions as articulated for Distributor 1, just scaled up. 4.1.2.7 Cross Sales Cross sales was measured in terms of percent increase in sales. Like efficiency of distribution, this criterion also had different input values for the two distributors. Distributor 1 said that his current percent increase in sales was greater than 5% and less than 10% so 8% was set as a base assumption for Option 2. The high and low estimates were double and half the base assumption. Option 1 was set at a base 56 assumption of 10% because it was assumed that distributors would have more time to turn their attention to other sales, as opposed to in Option 2 where much of their focus was on moving the mRDTs. The high and low-end estimates were set to double and half the base assumption. Option 3 was set at slightly less than Option 1 because it was assumed that lower quality devices would result in less cross sales. Distributor 2 reported that his current percent increase in sales was 16%, this was set as a base assumption for Option 2 and the high and low-end estimates were double and half this figure. Options 1 and 3 were assigned estimates from the same logic as Distributor 1's assumptions but scaled up to account for different scales of operations. Option 4 for both distributors was set at 0% because no cross sales could occur. 4.1.3 Decision Input Matrix - First Line Buyers The comprehensive list of inputs for the first line buyer decision matrix is summarized in Table 13 and assumptions explained in the subsequent sections. Some criteria were not relevant for Option 4; those were marked as N/A. 57 Table 13. Input figures for distributors for high, base, and low estimates for options 1-4 Cost per kit ($USD) Option 1 Option 2 Option 3 Option 4 Price per kit ($USD) Option 1 Option 2 Option 3 Option 4 Quality Option 1 Option 2 Option 3 Option 4 Administrative time (hours/week) Option 1 Option 2 Option 3 Option 4 Profit (%) Option 1 Option 2 Option 3 Option 4 Relationship with donor organization or funder Option 1 Option 2 Option 3 Option 4 4.1.3.1 High Estimate Base Estimate Low Estimate 0.7 0.09 1 0 0.35 0.09 0.35 0 0.18 0.09 0.15 0 0.40 0.30 0.40 0 0.25 0.25 0.25 0 0.10 0.20 0.10 0 H H LM N/A MH H L N/A MH MH L N/A 10 60 5 0 5 35 3 0 1 10 1 0 50 40 50 0 40 30 40 0 30 20 30 0 MH H ML L M MH L L ML MH L L Cost per kit The costs were based on conversations with the FLBs. One FLB said that the subsidized cost to them was $0.09 per kit, thus Option 2 was set at that. The FLB also said that the unsubsidized price was $0.35 so that price was used as the base for Options 1 and 3. The high and low-end estimates for Option 3 were assumed to 58 have a wider spread than Option 1 because there is less regulation for those manufacturers. Option 4 was set at $0 because the diagnostic is not sold. 4.1.3.2 Price per kit The average of the reported costs that the distributors paid was used as the price for the kits. The base cost of the kits were set as the same, but the spread of high and low was set higher for Options 1 and 3 than Option 2, as it is assumed there would be much less price control. Option 4 was set at $0. 4.1.3.3 Quality Quality was based on the same assumptions as the retailers. See the decision matrix input assumptions for retailers 4.1.3.4 AdministrativeTime The high and low estimates for the administrative time were the different reported amounts from the two FLBs. The base assumption was the average of the times. Options 1 and 3 were assumed to have much less administrative time because there would be no coordination down the supply chain nor with an NGO organization. However, Option 1 was set at a higher time for the high-end estimate than Option 3 because it is assumed that there is likely more administrative time to ensure compliance and possible coordination with other organizations. Option 4 was set at 0 hours because the devices are not being sold. 4.1.3.5 Profit Profit for Option 2 was set at 30% as a base assumption because that is what one FLB told us they had. The high and low estimates were plus and minus 10%. Options 1 and 3 were set at 10% higher than Option 2 because it was assumed that there was less price control and the FLBs could mark up prices as they thought appropriate. Option 4 was set at 0%. 4.1.3.6 Relationship with Donor Organization It was assumed that Option 2 would have the highest score in relationships because there would be direct coordination with the organization. Option 1 would have 59 slightly lower scores, as although there may not be participation in an intervention, the FLB is still selling WHO-approved devices and may have some interaction. Option 3 was lower because the FLB would not be selling the approved devices. Option 4 would be the lowest as devices are not sold at all. 5 Results - Value Functions This section describes the value functions that were constructed for the retailers, distributors, and first line buyers. Additionally, this section provides information on the criteria, sample size, and comments on the shape of the value function. 5.1 Retailers Criteria The retailers identified eleven criteria that are factors in their decision about whether or not to stock an mRDT. A list of initial criteria was generated by expert opinion and literature reviews and then a focus group discussion with the retailers added and confirmed the list of criteria (see Methodology for full procedure). Further, the ranges of the criteria were proposed based on focus group discussions. Table 14 summarized the criteria generated and the upper and lower bounds of the possible range. 60 Table 14. Criteria and ranges of criteria for retailers. Criteria Lower bound Upper bound Quality Low High $0 USD $1.67 USD Time to delivery (days) 7 1 Sales of other products Low High Customer Satisfaction Low High Training Low High 0 50 Low High 0 60 Low High 0 $1 Price Volume (kits sold per month) Awareness/ads Time to complete a sale (minutes) Other Opportunities Cost 5.2 Retailers Value Functions The criteria that was included in the retailers' considerations in their decisionmaking process were: quality, price, delivery, sales of other products, customer satisfaction, training, volume sold, awareness/advertising, time, opportunities, and cost. A value function was constructed by having the range of criteria as the independent variable (e.g., cost ranging from $0 to $1) and value on a scale of 1-5 as the dependent variable. Retailers were asked about their value for two different points and the median of all the responses was calculated and used in the value functions. See Methodology section for how responses were solicited. This section will exhibit and analyze the value function constructed for each of the criteria. 5.2.1 Quality Quality referred to the quality of the mRDT that the retailer could sell. Quality took into consideration accuracy and durability; a key factor was whether or not the device was WHO approved (see Methodology section). Respondents were asked 61 about their preference if the quality of the device was medium-low and mediumhigh (Table 15). Rebler Prferences for Quaity UN I / / I / / 41 / / / A / 3_x / / / 41 / / ' / 2 ' I 2 2)5y1.5 3 Qualy Low (1) trou 3.5 High (5) 4 4.5 5 Figure 7. Value function for quality of devices for retailers. Table 15. Median of retailer responses for quality Quality Low Low-medium Medium-high High Value 1 1.5 5 5 The value function for quality (Figure 7) is revealed to have a sharp increase in value followed by an increase at a decreasing rate. The value from 4 to 5 was flat. The relative increase from low to medium-low and medium-high to high was not as great as from medium-low to medium-high. In fact, retailers seemed to be 62 indifferent between a quality of medium-high to high. This value function implies that retailers care about a threshold of quality, but do not value incremental improvements in quality as much. For quality, the sample size was 17 responses total, 11 full sets (those that provided a response for both low-medium and medium-high), and three blanks. Two respondents provided a value for only the medium-low question and one respondent answered only for low-medium. The reason only 17 responses were recorded, instead of the full 28 was that the second focus group discussion responded that quality was not important to them. For this reason, the only data gathered was for the first focus group discussion. 5.2.2 Cost Cost was defined as the cost of the device to the retailer; that is, how much the retailer has to pay to procure the device. The ranges of cost were set from $0 USD to $1.67 USD (retailers were asked their preferences in Ugandan shillings not U.S. dollars). The median of the responses can be found in Table 16. 63 Retailer Prefemnces for Cost per Device I I V I I I I N, N, 4.5 4 N, N, N, 3.5 N N, 3 N. N, 2.51 2 N, N, 0 0 0.1 0.1 02 02 Figure 0:3 03 OA 0.4 0.5 0.5 0.~ Ok Cost per Device in SUSD 0.7 0.7 4 1 8.. Value function for cost per device for retailers. Table 16. Median of retailer responses for cost per device Cost per device $0 USD $0.50 USD $0.67 USD $1USD Value 5 2 1 1 The value function for cost (Figure 8) drops sharply from $0 to $0.67, after which retailers express a preference of 1 for any increasing cost. This implies that there is a narrow band of ranges of cost for the device that retailers prefer. 64 The sample size for cost excluded the first focus group discussion. This is because it is presumed that there was confusion and miscommunication about the questions. Especially as this was the first group and one of the first questions asked, it is likely that the methodology had not been refined enough to ensure accurate responses. Of the 17 retailers in the first focus group, there were 4 complete responses that indicated that a higher cost was associated with lower value, which is what one would expect. Seven retailers indicated the opposite trend, that higher costs had a greater value to them, 2 retailers were blank, and 4 were half-answers. It is possible that the 7 retailers were accurately representing their values, that more expensive items are valued more, perhaps because cost is associated with higher quality. However, it is also very likely that the retailers were confounding price and cost, even though the moderators did explain it. Especially since in the second focus group discussion, where the moderators were careful to explain price and cost separately in both English and the local language, the responses all had the initially expected trend where higher cost is paired with lower value. For this reason, the 11 retailers in the second focus group discussion were used in the analysis and the first discussion eliminated. Of the 11 retailers, 10 provided full responses and there was 1 blank. 5.2.3 Price Price was defined as the price at which the retailer sells the mRDT kit to the customer and the ranges were set from $0.33 to $0.67. Table 17 exhibits the median of the responses for price. 65 Retaer Preferences for Price per device 5 - 4.51 4 P I ~3b I / I / / 3 / / 2z L/ 2 / () 1.5 L 0L 2 - OA 0h 0.8 1 12 Price per device in SUSD 1A 1. 1.8 Figure 9. Value function for price per device for retailers. Table 17. Median of retailer responses for price per device Price per device $0.33 USD $0.67 USD $1USD $1.67 USD Value 1 2 4 5 The value function for price (Figure 9) demonstrates a fairly sharp increase in value from $0.33 to $1 and then a diminishing return on value from $1 to $1.67. This implies that retailers have a price at which they greatly want to sell the product, likely taking into account the cost of the device and other related costs. The sample size for the value function for price included both focus group discussions. Both were included, despite excluding the first for looking at cost, 66 because there were only 4 respondents that indicated that a higher price had a lower value. This implies that respondents had understood the question. Thus, the sample size was 28 retailers asked, out of which there were 2 blanks, 2 that answered only for $1, 1 that answered only for $0.67, and 4 that were determined a miscommunication (for list of removed values see Table 18). Table 18. Removed response sets for price of mRDT Removed Removed Removed Removed respondent respondent respondent respondent 1 values 2 values 3 values 4 values $0.67 5 4 5 3 $1 1 2 2 1 5.2.4 Time to Delivery Time to delivery was defined as the number of days after a retailer placed an order that their supplier would provide more product. The value function was plotted for an independent variable ranging from 1 day to 7 days. One-day replenishment was set as the upper bound as a 5 for value and seven days was set as the lower bound as a 1. Respondents were asked about their value for two days of delivery and 4 days of delivery. The median of responses was recorded and used in the value function (Table 19). 67 Re I.riPrebrences for Desvery Tine 4 4 3 3.5 25- 2 -b 1.5 1 1 2 5 4 3 6 Time to Defivey (days) Figure 10. Value function for delivery time for retailers. Table 19. Median of retailer responses for delivery time Delivery time 1 day 2 days 4days 7 days Value 5 5 2 1 The value function for delivery time (Figure 10) exhibits an inflection point at 4 days of delivery. Value is equal between 1 and 2 days of delivery, decreases sharply from 2 to 4 days, and decreases less sharply from 4 to 7 days. This could be because retailers categorize delivery times in two categories-"sooner" and "later" and anything after 4 days is considered relatively the same. This also means that a oneday decrease in delivery time from four days has a high return on value. 68 The question of delivery time was asked to all 28 retailers and 23 provided a response. Four respondents did not provide a response at all, and 3 responded only to the 2-day delivery time. Three responses were excluded from the analysis because of presumed misunderstanding; they recorded higher values for longer delivery time. Table 20 Table 20. Removed responses for Delivery Time Removed respondent 1 values Removed respondent 2 values Removed respondent 3 values 5.2.5 2 day delivery 2 3 1 4 day delivery 3 4 5 Sales of Other Products Sales of other products was evaluated in qualitative terms ranging from low (1) to high (5). Sales of other products was defined as products that were sold because the retailer provided mRDTs. This could be because a customer now chooses that store or goes in to be tested and then purchases other necessary products there, or for any other number of reasons. Low was described as almost no new sales, medium was described as some new sales, and high was described as a lot of new sales. The median of the responses were graphed in the value function (Table 21). 69 Fatater references for Other Sales 5 4.5 4 3.5 1. 0) 3 I I I 2 15 3.5 4 2 2.5 3 Increase in Other Sales Low (1) through High (5) 4.5 Figure 11. Value function for sales of other products for retailers Table 21. Median of retailer responses for increased sales of other products Increase of sales of other Value products Low 1 Low-medium 2 Medium-high 4 High 5 The value function for sales of other products (Figure 11) is linear; value increases equally as sales of other products increases. It is possible that the function would 70 have changed if percent sales were provided instead of qualitative values, as was done for distributors. Responses for sales of other products was solicited from all 28 retailers and there were 20 full responses. Three responses were completely blank and 5 individuals provided only an answer to medium-high. One response was eliminated from the analysis because of presumed misunderstanding. The removed response reported value of 1 for medium-high and 4 for low medium. 5.2.6 Customer Satisfaction Customer satisfaction was defined as perceived satisfaction of customers after a business interaction. Low satisfaction was described as the customer being very unhappy, medium was neutral, and high was that the customer was very happy. The median of responses was graphed (Table 22). The value function for customer satisfaction was exhibited to be linear (Figure 12). 71 RetaNer Preferences for Customer Safsfactan 5 4.5 7 4 A 3.517 3 7 2.517 2 1.5 I 15 2 I I I 2.5 3 3.5 4 Customer Satisfaction Low (1) through High (5) 4.5 5 Figure 12. Value function for customer satisfaction for retailers Table 22. Median of retailer responses for customer satisfaction Customer Satisfaction Low Low-medium Medium-high High Value 1 2 4 5 All 28 retailers were asked about customer satisfaction and 20 provided full responses. There were 5 blanks and 3 responses that only had information provided about medium-high satisfaction. 5.2.7 Training Training was described as an outside agency (which could include NGOs, distributors, etc.) providing training on how to properly administer and interpret 72 the mRDTs and then provide advice to the customer about proper next steps. Training was described on a qualitative scale from low (1) to high (5), where low was little or no training, medium was some training, and high was thorough and comprehensive training and guidance. Table 23 exhibits the median of responses. Romber Prefernces for Training / / 5 4.5 / / 4 / 3.513 / 2.5 I- 2 7 1.51-_ 7 z 1 T 1.5 I 2I I 2.5 3I 3.5I 1b 2 2.5 3 3.5 Training Low (1) through High (5) 4 4.5 5 Figure 13. value function for training for retailers Table 23. Median of retailer responses for training Training Low Low-medium Medium-high High Value 1 2 5 5 73 The value function for training (Figure 13) has a vaguely sigmoidal shape. The increase in value from low training to low-medium training is not as great of an increase as from low-medium to medium-high. Further, respondents seem to be indifferent between medium-high and high training. This implies that retailers value above a medium amount of training but have diminishing returns to training greater than that. All 28 retailers were asked about training and 20 provided a full response. There were 5 blanks and 3 retailers that only provided a response to medium-high. 5.2.8 Volume Volume was defined as the number of devices sold per month. The ranges were established as 0 as the lower bound and 50 mRDTs sold per month as the upper bound. Table 24 shows the median responses. 74 Ibtaer Prefrences for Volume 5> - 4.5 - 4 3A - - 35 ,5 25 - 1.5 5 10 15 25 30 20 Sales per month 35 40 45 50 Figure 14. Value function for volume of sales for retailers Table 24. Median of retailer responses for volume Volume 0 devices sold/month 10 devices sold/month 30 devices sold/month 50 devices sold/month Value 1 1 4 5 The value function for volume of mRDT devices sold per month (Figure 14) is roughly s-shaped. The fact that respondents value 10 devices sold per month as a 1, the same as if no devices were sold implies that there is some negative consequence to holding the devices and not selling above a threshold of them. This could occur if retailers have to buy the devices from distributors in packages and therefore they have some amount of sunk cost. It is also possible that having the devices sit on the 75 shelf, and take up space, is reducing their value, perhaps by taking up the space of another product. All 28 retailers were asked about volume and 20 complete responses were recorded. There were 7 blanks, 1 individual who responded only to 30 devices sold per month, and 2 who responded only to 10 devices sold per month. One response was removed from the analysis from presumed misunderstanding. The removed response reported 30 mRDTs sold per month as a value of 4 and 10 mRDTs sold per month as a value of 5. 5.2.9 Awareness/Advertising Awareness/advertising was defined as the amount of advertisements and awareness that was generated by an outside agency (including NGOs, distributors, etc.). The metric was qualitative ranging from low (1) to high (5), where low was described as little or no advertising, medium was described as some advertising, and high was a lot of and/or very effective advertising. Table 25 describes the median responses to awareness/advertising. 76 RetNer Preferences f1r Amrns/AdvwtisIng 5 4.5 4 - 3.5 0 3 I I I II 2 r 15 2 4 2.5 3 35 Awareness/Ads Low (1) through High (5) 4.5 5 Figure 15. Value function for awareness/advertising for retailers Table 25. Median of retailer responses for advertising/awareness Advertising/ Awareness Low Low-medium Medium-high High Value 1 2 4 5 The value function for advertising/awareness was linear (Figure 15); the value scaled equally with awareness and advertising. All 28 retailers were asked about their preferences on awareness/advertising and 18 full responses were recorded. One respondent answered only for medium-high 77 and one answered only for low-medium. Two values were excluded from the analysis from presumed miscommunication (Table 26). Table 26. Removed responses for Advertising/Awareness Low-medium Medium-High Removed respondent 1 values 4 2 Removed respondent 2 values 4 3 5.2.10 Time to Complete a Sale Time to complete a sale was defined as the number of minutes the shopkeeper spends with a customer spends in the store facilitating a sale. In the case of mRDTs, this generally involved explaining the benefits to testing for malaria, affirming the reliability and accuracy of the test, then actually administering the test, and providing recommendations for appropriate next care (antimalarial or something else). The times ranged from 60 minutes of time that the shopkeeper needed to be engaged with the customer to 0 minutes, which is the interaction consisting of not much more than the exchange of money for the product. Table 27 exhibits the median responses for the times. 78 Retaler Prerences far Tine to Campile Sale r - 45 4 3.5 (A) d0 3 2.5- 2 1.51- 1 0 .2 10 20 30 Minutes per sale 46 50 Figure 16. Value function for time to complete a sale for retailers Table 27. Median of retailer responses for advertising/awareness Time to complete sale 60 minutes 40 10 0 Value 1 1 4.5 5 As the time to complete a sale increases, the retailers' value decreases (Figure 16). This makes sense because if the retailer spends an extended amount of time with one customer he or she is unable to help others. Sometimes a sale could be lost if the retailer is busy as a customer will leave and go to another store. The decrease in value from 0 to 10 minutes is much less steep than from 10 minutes to 40 minutes. This implies that retailers expect to spend a few minutes facilitating a sale but do 79 not want to spend an extended amount of time. Both 60 and 40 minutes have a value of 1, implying that spending 40 minutes or greater on a sale is very undesirable. Time to complete a sale was not a criterion that was identified by the first focus group discussion. Therefore, data was only gathered from the second discussion, which included 11 respondents. Of the 11 asked, 10 provided a full response but 2 out of 10 were removed from presumed misunderstanding. These individuals indicated that they had a higher value for the greater amount of time spent to complete the sale (Table 28). However, it is possible that these individuals were not misunderstanding, and rather have reverse preferences than the rest of the cohort for this criteria. One plausible scenario could be that these retailers were expressing a correlation between spending longer time with a customer and greater customer satisfaction or even more sales. However, since the chance of miscommunication was high in the focus group discussions and because the rest of the retailers indicated the predicted response, the values were removed from the analysis. Further, since the high and low value levels were defined, any responses that went against that direction were removed (see Discussion section) Table 28. Removed responses for time to complete sale Removed respondent 1 values Removed respondent 2 values 10 minutes 40 minutes 1 3 3 4 5.2.11 Opportunities Opportunities as a criterion was defined as other benefits that are associated with selling mRDTs. For example, the retailers received a free shirt that was emblazoned with a mosquito and the word "mRDT." Retailers also expressed that in general, when they participate in projects or campaigns there are other giveaways, prizes, and competitions. Further, there is opportunity to participate on a higher level, in an organizing or mobilizing capacity, which is usually financially compensated. A qualitative metric was used to aggregate all these aspects, and respondents were 80 asked to rank their value for opportunities ranging from low (1) to high (5). Low opportunities was defined as very little or nothing, medium was some opportunities provided, and high was frequent and useful opportunities. Table 29 summarizes the median of responses. Retailer Prebrencee br Oppa~inI~.a 5 I I I I I 4.5 4 k 3.5 0 3 / 0 2.5 I 1.5 11 - ) 2 I 1.5 2 I I I 2.5 3 3.5 Opportunity Low (1) through High (5) I 4 4.5 5 Figure 17. value function for opportunities for retailers Table 29. Median of retailer responses for opportunities Opportunities Low Low-medium Medium-high High Value 1 1.50 4 5 81 The value function for opportunities (Figure 17) has an inflection point at lowmedium opportunities. Value increases from low to low-medium then increases sharply from low-medium to medium-high. There is only a slight change in slope from medium-high to high. These results suggest that retailers see little value from low and low-medium opportunities, but as the opportunities increase from there, their value increases greatly. Opportunities, like time to complete a sale, was a criterion that was not brought up in the first discussion group. Therefore, 11 retailers were asked about their preference and 10 provided a complete response. 5.3 Distributors All three distributors in the supply chains studied were interviewed. Of the three, one was unable to provide quantitative information and was classified as a nonresponse. Thus, two value functions for each criterion is presented, one for each of the distributors that answered. The range of criteria was set as the same for both distributors (Table 30). Table 30. Criteria and ranges of criteria for distributors Criteria Volume (kits sold per month) Expiration date (months left to expiry at sale) Profit margin (%) Cost per kit ($USD) Cost per training ($USD) Efficiency of distribution (Boxes per delivery) Cross selling (% increase in sales) 5.3.1 Upper Bound 18,750 22 50% $0.83 $2,000 500 (or 100) 50% (or 20%) Lower Bound 0 3 0% $0 $0 1 0% Volume The volume of sales for the distributors was recorded in units of kits sold per month. Since distributors articulated volume in different units, it was necessary to convert to one metric for ease of comparison of value functions (see Methodology section). The value functions between the two distributors are similar, but Distributor 2 (Figure 19) exhibits an inflection point while Distributor 1's (Figure 82 18) response is more linear. The two end points were established in conversations with both distributors where the lower bound was set as no sales and the upper bound was defined as selling all of their product in 16 months, which is a few months before the expiration dates on the kits and before the campaign technically ends. The distributors expressed their desire to have sold all the kits quickly both so that they did not have stock taking up space in their warehouse and also so they did not have to discard expired kits. Distribmwor I Preferences 5 for Volume 4.51- 4 I- 3.5 k 3 2.51- 2 0 02 0.4 0.6 0.8 1 12 Kits sold per month 14 1.6 1.8 2 10A Figure 18. Value function for Distributor 1 for volume 83 Table 31. Distributor 1 responses for volume Value 1 3 5 Volume (kits sold per month) 0 9,000 18,750 Distribulor 2 Pr.$wences for Volume I I I I I I I I ~.~-*' 41 C4 7 7 Z 2~ 7 it / 7 / / 1~ 02 0.4 0.~ Oh 12 1A ?S 02 0.4 0.6 12 1 0A5 Kits sok per month 1.4 1 1 I. 2 1.8 S10' Figure 19. Value function for Distributor 2 for volume Table 32. Distributor 2 responses for volume Volume (kits sold per month) 0 1,625 18,750 Value 1 2 5 84 5.3.2 Expiration Date Expiration date was defined as the number of months left before expiry when the kit was sold to the retailer. Ensuring that the retailers received the kit with enough time on their end to sell the product was important. Thus, the date to expiry ranges from 22 months if the kits were sold immediately and 3 months as the lower bound, as the distributors informed us that after 3 months, they would not be able to sell the product. The value functions for both distributors exhibit an inflection point at 6 months, after which value drops sharply (Figure 20 and Figure 21). This is likely because of the aforementioned issue that the distributors are much less likely to be able to sell the products with such a short amount of time to expiration. mskibutr 1 Pre.rencos for Expkratn Data D 4.51 4 j 0 3.51 y 3 0 2 5 L- & 0 / 2 .1 2 4i 6p 8 10 p 12p 14p 16 p 18p 4 6 8 10 12 14 16 18 Months left to expiry at sale 20 22 Figure 20. Value function for Distributor 1 for expiration date 85 Table 33. Distributor 1 responses for expiration date Months left before expiration at time of sale 3 6 22 Value 1 2 5 mskriutmr 2 Prbfrwsces for Expiration Dahe 5 45- 4e~J 3~5 3 & 25 / 2 1,5 iI 2 4 6 a 14 12 10 Months left to expiry at sale 16 18 20 22 Figure 21. Value function for Distributor 2 for expiration date Table 34. Distributor 2 responses for expiration date Months left before expiration at time of sale 3 6 22 Value 1 3 5 86 5.3.3 Profit Margin Although profit margin was not included in the list of criteria that was generated before the data collection process, because it is comprised of three variables (cost, volume, and price), it was soon clear that distributors thought in terms of profit margin and wanted to express their opinions on that. Distributors indicated that a low bound would be 0% margin and the high bound for products like this was 50%. The value functions for both distributors are very similar, demonstrating an inflection point at around 25-30% (Figure 22 and Figure 23). Value increases sharply to 25-30% profit margin and then increases at a decreasing rate after. This agrees with what the distributors said in interviews, that having high profit was a good thing, but they were not seeking high profits on products like mRDTs. Dlsrlbubr 1 Prelfmances for Proml Mgla 5 4. 5 I I I Q I 4 3b IS 3 2 1 .5 11 0 5 10 15 20 25 30 Percent proft margin 35 40 45 50 Figure 22. Value function for Distributor 1 for profit margin 87 Table 35. Distributor 1 responses for profit margin Value 1 4 5 Profit margin (%) 01 27.5 50 DIstvbiabr 2 Prfwrec.. for Promt Marn 5 4 - 4.5F S3 ~2.5 2 1.5 I S 5 10 15 20 25 35 30 Percent profit matgin 40 45 30 Figure 23. Value function for Distributor 2 for profit margin Table 36. Distributor 2 responses for profit margin Profit margin (%) 0 25 50 Value 1 4 5 88 5.3.4 Cost Per Kit The cost per kit was the cost that the distributor pays to procure the device. Distributor 1's value function (Figure 24) exhibits an inflection point at $0.30 while Distributor 2's is more linear (Figure 25). This could be due to different preferences, or from a limited number of data points. It is possible that getting more information from Distributor 2 would reveal an inflection point. Distdbutor 1 Preferences for Cost per Kit I 45 I I I I I I. 4 35b. 3 S I 2.5 2 N. 1,.5 1 0 0.1 I I I 02 03 0.4 I 0.5 Cost (USD) per kit 0.6 0.7 0.8 0.9 Figure 24. value function for Distributor 1 for cost per kit Table 37. Distributor 1 responses for cost per kit Cost per kit ($USD) 0 0.30 0.83 Value 5 4 5 89 DIstrbutor 2 Prfornces for Cost par Kit C 4. 4 3. S & 15 2- I. 0 0.1 02 03 0.4 0.5 Cost (USD) per kit 0.6 0.7 0A. 0. Figure 25. Value function for Distributor 2 for cost per kit Table 38. Distributor 2 responses for cost per kit Cost per kit ($USD) 0 0.20 0.83 5.3.5 Value 5 4 1 Cost per Training As per DevOrg's contract, the distributors were responsible to provide training sessions to the retailers before they could sell the mRDTs. The requirements for each session were outlined in standard operating procedures. The cost per training value functions for the two distributors are shaped differently. Distributor 1 (Figure 26) exhibits two inflection points at $500 and $1,700 per training. Distributor 2 (Figure 27) has a very slight inflection point at $1,350. 90 DIstributor 1 Prelrences for Cost per Training r. 4.5 I 4 3.51 3 2.51 & 0 2 1.51 I 0 200 4DO ODO WO0 1000 1200 Cost (USD) per training session 1400 1600 1800 26 0 Figure 26. Value function for Distributor 1 for cost per training Table 39. Distributor 1 responses for cost per training Cost per training ($USD) 0 500 1,700 2,000 Value 5 3 2 1 91 Distributor 2 Prebrwces far Cost pr Training 4.5 4 c'J 3,b 0 3 0 2~5 & 0 2 1 .5 0 200 400 00 800 1000 1200 Cost (USD) per trailng session 1400 1600 1800 2030 Figure 27. Value function for Distributor 2 for cost per training Table 40. Distributor 2 responses for cost per training Cost per training ($USD) 0 1,350 2,000 5.3.6 Value 5 2 1 Efficiency of Distribution Efficiency of delivery was brought up by the distributors in conversation as a factor in their decision. A metric defined as number of boxes per delivery was established to measure this criterion. The distributors expressed that there was a fixed cost in fuel and salary to deliver boxes to the retailers, and delivering a small number of products was not efficient for their business. This criterion was one of two that had a different range for the two distributors. Distributor 1 asserted that the high-end 92 range for him was 100 boxes per delivery and for Distributor 2 it was 500 boxes per delivery. This is likely due to the difference in overall size and extensiveness of the two distributors. Distributor 2, for example, said that the mRDT business was less than 1% of his portfolio. Thus, the order of magnitude his business operates is likely much greater than Distributor 1. Despite having different scales, the value functions (Figure 28 and Figure 29) exhibit almost the same shape. There is an inflection point early in the range, where value increases sharply before it and at a decreasing rate after. This suggests that distributors seek to meet a minimum threshold of devices per delivery. DhitrWt~r 1 Pafemeas for Efficiwcy of Dialribution 4.51 4 /- - 0- 3.5b 3 2,5I 2 1.5 1 0- 10 20 30 40 50 60 Number of boxes per dloivary 70 80 90 100 Figure 28. Value function for Distributor 1 for efficiency of distribution 93 Table 41. Distributor 1 responses for efficiency of delivery Boxes per delivery 1 10 100 Value 1 3 5 DI.tribu~w 2 Prferncs ~r Efficiency of Delivery 5 I I I I I 50 100 150 200 250 I I I I I A I A 50 100 150 200 250 300 350 400 I I I 350 400 450 *-~~ - 4.5 4 36- / / S3 / / 2 1.51L F 1: 300 Number of boxes per delivery 450 S0 Figure 29. Value function for Distributor 2 for efficiency of distribution Table 42. Distributor 1 responses for efficiency of delivery Boxes per delivery 1 100 500 Value 1 3 5 94 5.3.7 Cross Selling Cross selling for distributors is analogous to the criteria "increased sales of other products" for the retailers. However, for distributors the unit used to describe the criteria was percent increase in sales due to carrying the mRDTs. These sales could come from an increase in their distribution network and/or from the sale of gloves and other materials necessary to administer the tests. The range of cross selling was different for the two distributors. Distributor 1 expressed that the high value would be 20% increase in sales and Distributor 2 said 50%. Again, this may have to do with the relative scales and scopes of their respective business operations. The value functions are almost linear. Distributor 1's value function (Figure 30) is linear with the data points provided and Distributor 2's (Figure 31) exhibits a very slight inflection point. This suggests that, in general, increase in cross sales scales linearly with value. Dilstributor 1 Prefenics for Cross Soing 4.5. 4- 0 3.5- 3 2.5 / 2 2 4 6 12 10 a Percent increase in sales 14 16 18 20 Figure 30. Value function for Distributor 1 for cross selling 95 Table 43. Distributor 1 responses for cross selling Value 1 Percent increase in sales (%) 0 10 20 3 5 Distriutor 2 Prfernces for Cross SeNing 5 4. 4 CM4 S2.5 2 1,5 ILI 10 15 20 25 30 Percent increase in sales 35 40 45 50 Figure 31. Value function for Distributor 2 for cross selling Table 44. Distributor 2 responses for cross selling Percent increase in sales (%) 0 16 50 Value 1 2 5 96 5.4 First Line Buyers This section describes the value functions for the both of the first line buyers. Cost per Device 5.4.1 The cost per device was defined as the cost to the FLB to purchase the mRDT from the manufacturer. There was a difference in the shape of the value functions between the two FLBs. FLB 1 (Figure 32) indicates minimal value at $0.333 and higher. FLB 2 (Figure 33) has a much less dramatic decrease value corresponding with increased cost. FLB I Prnfernces for Cost per Device 4.51- 4 'I - 3.5 VL 13 2 0 I I I I I 0.1 02 0.3 0.4 05 I 0. Cost per Kit (SUSD) ~ 0.7 0.8 0. Figure 32. Value function for FLB 1 for cost per device 97 Table 45. FLB 1 responses for cost per device Cost per device ($USD) $0.033 $0.167 $0.333 $1 Value 5 4 1 1 FLB 2 Prtrnc.. for Cost per Device 5 I I I I I I I I 4.5 1k 4 3.51N N~ -9 3 2.5L- 2 N N 1.5 IN 1 I0 0.1 02 03 0)6 OA 05 Cost per Kit (SUSO) 0.7 0.8 0.9 r Figure 33. Value function for FLB 2 for cost per kit 98 Table 46. FLB 2 responses for cost per device Value 5 3.5 1 Cost per device ($USD) $0.033 $0.15 $1 5.4.2 Price per Device Price per device was defined as the price at which the retailer sells the device to their distributor. FLB 1's value function (Figure 34) reveals a very sharp increase from a price of $0 to $0.033, then a relatively steep increase from 0.033 to $0.167, and then a shallower increase from $0.167 to $1. FLB 2 (Figure 35) reveals a similar pattern, with a sharp increase at first followed by a less dramatic increase. S I I FIB I preference, for Price I per Device I I 4.6 41 3.1 -J LA~ ---- 21 - I 1 .' 0.1 02 0.3 GA 0.~ 0./ 0.1 02 0.3 0.6 0.5 0.4 Price per Kit (SUSD) 0.7 0.5 a a 0. a 0.9 Figure 34. Value function for FLB 1 for price per kit 99 Table 47. FLB 1 responses for price per device Price per device ($USD) 0 $0.033 $0.167 $0.333 $1 Value 5 3 3 4 5 FIB 2 Prefwences for Price per Device 5 465 4 3.51 / N' 3 I - / 265 I - / 2 I / 1.5 1 a I I 0.1 02 0.3 0.4 0.5 0.6 Price per Kit (SUSD) 0.7 0A 09 1 Figure 35. Value function for FLB 2 for price per kit Table 48. FLB 2 responses for price per device Price per device ($USD) $0 $0.25 $1 Value 5 3.5 5 100 Quality of Device 5.4.3 The quality of the device was described in terms of WHO-approval, accuracy, and durability (see Methodology section). Both FLBs (Figure 36 and Figure 37) indicate a dramatic decrease in value when the quality of the device is low. FLU I Pmfmnce. for audily of Device 5 I I I / I / 45 I 4 ~35 I-J 3 2.5 I- 2 '.5 j r #1 I 2.5 3 Qualty Low (1) ftho 1.5 Figure 36. Value function for FLB I 3.5 High (5) I 4 4.5 1 for quality Table 49. FLB 1 responses for quality of device Quality of device High Medium-high Medium Low-medium Low Value 5 2 2 1 1 101 FB 2 Preferences for Quality of Device 5 -77- 4.5 4 3.51 N~ 9J 3 2.5 2 1.5 I r 1.5 3 3.5 25 Qualty Low (1) through High (5) 4 4.5 5 Figure 37. Value function for FLB 2 for quality Table 50. FLB 2 responses for quality of device Quality of device High Medium-high Medium Low-medium Low Value 5 4 1 1 1 102 Administrative Time 5.4.4 Administrative time was defined as the number of hours per week that the FLB spends managing the sale of the devices. FLB 1's value function (Figure 38) reveals a dramatic decrease in value from 1 to 10 hours per week. FLB 2 (Figure 39) has a similar decrease but at a rate less than FLB 1. FLB 1 Prfrences for Hoirs of Admindxrafve Time 5 4,5 4 3.5'. U9 %3 2b51 2 1 0 1-0 20 3 40 Hours spent per week on administration for mRDT sales 50 Figure 38. Value function for FLB 1 for administrative time Table 51. FLB 1 responses for administrative time Administrative Hours (hours) 1 5 10 60 Value 5 3 1 1 103 R.B 2 Preferncos for Hous of Admli*&ave Tim. 5 4,51 4 3.5 L 3 2.5 2 1.5 1 20 S10 37 40 Hours spent per week on administration for rnRDT sales 50 Figure 39. Value function for FLB 2 for administrative time Table 52. FLB 2 responses for administrative time Administrative Hours (hours) 1 5 10 60 5.4.5 Value 5 4 3 1 Percent Profit Percent profit was the profit on the devices that the FLBs would receive. This was the only criterion that had different ranges; FLB 1 indicated that their upper bound of profit was 50% and FLB 2 indicated theirs was 100%. Both FLB's indicated a 104 sharp increase in profit initially and then an inflection point after which value increased at a slower rate. FLB 1's value function (Figure 40) reveals a very steep increase in value from 0% to 5-10%, and then maximal value at profits increasing after 20%. FLB 2 (Figure 41) exhibits an inflection point at 33%. FLO I Prebmwic" for Profit "iln 5 4.5 4 3 i I i I 2.51 2 1.5 41 456- 5 10 15 20 25 30 Percent Profit Margin 35 40 45 50 Figure 40. Value function for FLB 1 for profit Table 53. FLB 1 responses for profit Profit margin (%) 0 5 10 20 50 Value 1 4 4 5 5 105 FLO 2 Prebmence. for Profit Margin 5 I II I I I I 4.51 4 3.5 I LhL 3 Il 2.51 2 1.5 I Y 10 20 30 50 60 40 Percent Profit Margin 70 80 90 100 Figure 41. Value function for FLB 2 for profit Table 54. FLB 2 responses for profit Profit margin (%) 0 33 100 5.4.6 Value 1 3 5 Relationship with Donor Organization Relationship with donor organization was defined as the extent to which the FLB interacts positively with a non-profit or NGO organization, like DevOrg. The value functions are shaped differently for the two FLBs, with FLB 1 (Figure 42) exhibiting an inflection point at a medium relationship and FLB 2's value function (Figure 43) being linear. 106 5 FLO I Preferences for Roabonship with Donor Organization 4.5 4 3.5 3 2.5 2 1 1.5 2 3.5 3 25 Relationship Low (1) through High (5) Figure 42. Value function for FLB 4 45 5 1 for relationship Table 55. FLB 1 responses for relationship with donor Relationship with Donor Low Medium High Value 1 2 5 107 FLD 2 Preferences br RelalonuhIp with Donor (~gmu~zatIon 5 'F 'F 4.5 'F 'F 4 3.51"F Nc LL 3 A 2.5 I. 2 'F 1.5 'F 7 1 F I I I I I I 1.5 2 2.5 3 3.5 4 Relationship Low (1) throug High (5) Figure 43. Value function for FLB 2 for relationship Table 56. FLB 2 responses for relationship with donor Relationship with Donor Low Medium High 5.4.7 Value 1 3 5 Percent Increase in Sales Percent increase in sales was a criterion that only came up in discussions with FLB 2. This is essentially cross sales, where by selling mRDTs, the sales of other products that the FLB stocks will also increase. FLB 2's value function demonstrates an 108 inflection point at 30% increase in sales, before which the value increases more sharply and after which increases less sharply. FLB 2 Preferences IDr Precent k~creas. In Sale. 5 4,5 / 4 / 3.5 ,. / 3 2.5 2 / Is1 I 1 a a a a 5 10 15 20 a a a a a 25 30 35 40 45 Increase in sales (%) 50 Figure 44. Value function for FLB 2 for percent increase in sales Table 57. FLB 2 responses for percent increase in sales Percent Increase in Sales (%) 0 30 50 6 Value 1 4 5 Results - Weights This section describes the solicited and normalized weights for the different agents. 109 6.1 Retailers The relative preferences of retailers for each of the attributes were derived by the swing weights procedure (see Methodology section). Each individual retailer recorded their values and the median of the responses was calculated. No answers were excluded from the analysis, but some retailers did not provide a response, either at all to the weighting questions or to one of the criteria. For weighting on quality, only the first focus group provided information because the second group informed us that quality was not important to them (see Methodology section). For opportunities and time to complete a sale, only the second focus group discussion was included because those attributes did not come up for the first focus group. Table 58 summarizes the number of responses about weight for each of the criteria. Table 58. criteria and Number of Responses Criteria Quality Price Delivery time Sales of other products Customer satisfaction Training Volume Awareness/Advertising Cost Time Opportunities Number of responses 16 26 25 25 25 26 26 23 26 10 10 The weights were solicited from a range of 0-100. Each individual was asked to set at least one criteria with the highest value of 100. Then, the median of the responses was calculated and then normalized over the sum of the weight values (Table 59). It was noticed that there was not a wide spread in the weights, see the Discussion section for biases that may have influenced these results. 110 Table 59. Median and normalized weights Criteria Quality Price Delivery time Sales of other products Customer satisfaction Training Volume sold Awareness Cost Time to complete sale Opportunities 6.2 Median response 75 75 80 70 90 90 62 80 70 87.5 65 Normalized weight 9.0% 9.0% 9.6% 8.4% 9.6% 10.8% 7.4% 9.6% 8.4% 10.5% 7.8% Distributors The relative preferences of distributors for each of the attributes were derived by the swing weights procedure (see Methodology section). Each distributor was asked to provide a numerical score of 0-100 for each of the criteria identified. The responses per distributor were normalized based on the sum of the responses. As was similar to the retailer, the spread of weights was not very large. See Discussion section for consideration of biases and suggestions for future improvements in the data collection. Table 60. Weights for Distributor 1 Criteria Volume Expiration date Profit margin Cost per kit Cost per training Efficiency of distribution Cross selling Response 100 90 85 100 70 90 70 Normalized weight 16.5% 14.9% 14% 16.5% 11.6% 14.9% 11.6% 111 Table 61. Weights for Distributor 2 Criteria Volume Expiration date Profit margin Cost per kit Cost per training Efficiency of distribution Cross selling 6.3 Response 100 95 80 80 90 90 80 Normalized weight 16.3% 15.4% 13.0% 14.6% 14.6% 14.6% 13.0% First Line Buyers The relative preferences of distributors for each of the FLBs were derived by the swing weights procedure (see Methodology section), similarly to the distributors. Each FLB was asked to provide a numerical score of 0-100 for each of the criteria identified. The responses per FLB were normalized based on the sum of the responses. Interestingly, the spread for the FLBs was much greater than for the distributors and retailers. Further, there are some significant differences in the weights for some of the criteria between the FLBs (see Discussion section). Table 62. Weights for FLB 1 Criteria Cost per device Price per device Quality Administrative time Profit Relationship with donor Response 30 100 90 90 20 100 Normalized Weight 6.98% 23.3% 20.9% 20.9% 4.65% 23.3% 112 Table 63. Weights for FLB 2 Criteria Cost per device Price per device Quality Administrative time Profit Relationship with donor Percent increase in sales 7 Response 100 70 90 10 50 55 80 Normalized Weight 22.0% 15.4% 19.8% 2.2% 11.0% 12.1% 17.6% Results - Overall Value Multi-criteria decision analysis can be used to derive an overall value for each of the decision options based on the value functions and weights. A weighted sum is set up: Overall Value = v1 * w1 + v2 * W2 + --- + vn *wn Where v, is the value for criteria 1 and w, is the weight for criteria 1, etc. Theoretically, the option with the highest overall value is the best decision choice for that individual, taking into account their preferences and priorities. An overall value was calculated for the agents in the mRDT supply chain and a sensitivity analysis varying weights and input assumptions was also performed to determine robustness of results. This section provides details about the results of the overall value analysis and calculations. 7.1 Overall Value - Retailers The calculated overall value was based on the equation: Overall Value Retailers = + V quality * Wquality + Vprice Vother sales * Wother sales * Wtraining * Wcost + Vvolume + * Wprice + Vdelivery time * Wdelivery time Vcustomer sat. * Wcustomer sat. * Wvolume + Vtraining + Vawareness * Wawareness + Vcost + Vtime sale * Wtime sale + Vopportunities * Wopportunities 113 See Results - Value Function and Weights on how the value and weights were calculated. The overall value for the base, high, and low-end input estimates is summarized in Table 64. Table 64. Overall value for Retailer Options (bold values are the highest value in the subset) Overall Value - High Overall Value - Base Overall Value - Low 3.45 4.19 2.47 1.57 2.74 3.78 1.85 1.57 2.27 3.13 1.69 1.57 Option 1 Option 2 Option 3 Option 4 Option 2, which is to sell the mRDTs in the bundled service ranks as the best option across the varying input estimations. 7.2 Overall Value - Distributors The overall value for each distributor was calculated based on their individual value functions and weights. The equation for overall value was based on the equation: Overall Value Distributor ~ Vvolume * Wvolume + Vcost kit * Wcost kit + Vexp. date * Wexp. date + Vprof it * Wprof it + Vcost train. * Wcost train. + Vdistribution * Wdistribution + Vcross sales * Wcross sales The overall value outcomes for each of the distributors are summarized in Table 65 and Table 66. Across the estimates, Option 1 is consistently the best option for the distributors. Option 3 is often a close second in overall value. The consistency of results across different estimates suggests that the results are robust. 114 Table 65. Overall Value for Distributor 1 Options (bold values are the highest value in the subset) Overall Value - High Overall Value - Base Overall Value - Low 3.00 2.77 2.98 1.83 2.61 2.12 2.58 1.83 2.29 1.90 2.19 1.83 Option 1 Option 2 Option 3 Option 4 Table 66. Overall Value for Distributor 2 Options (bold values are the highest in the subset) Overall Value - High Overall Value - Base Overall Value - Low 3.06 2.54 2.92 1.82 2.66 1.92 2.55 1.82 2.24 1.78 2.15 1.82 Option 1 Option 2 Option 3 Option 4 Attribute metrics are supposed to be preferentially independent from one another, otherwise one may receive a disproportional emphasis in the final calculation. When the criteria were initially generated before the data collection in Uganda, profit was not included; rather, costs, prices, and volumes were specifically listed. However, in conversations, the distributors insisted that profit margin on the device was one of their important considerations in making this decision. This implied that the distributors thought in terms of profit percent and thus, this criterion was included in our analysis. Cost and profit were both included in the analysis. However, in addition, a second analysis was performed to determine the overall value when cost was removed as a criterion. This change preserves the ordinal rank of the first two choices, with Option 1 being the best decision, followed by Option 3, for distributors. However, the difference in value between Options 1 and 3 becomes much less than previously calculated. Further, the Option 2 decreases drastically in value, as the subsidized prices of the kits were contributing to a large amount of value for that option. 115 Table 67. Overall Value for Distributor 1 Options - Cost per kit is removed (bold values are the highest value in the subset) Overall Value - High Overall Value - Base Overall Value - Low Option 1 2.83 2.39 2.03 Option 2 2.11 1.43 1.18 Option 3 2.81 2.37 1.86 Option 4 1.83 1.83 1.83 Table 68. Overall Value for Distributor 2 Options - Cost per kit is removed (bold values are the highest value in the subset) Overall Value - High Overall Value - Base Overall Value - Low Option 1 2.90 2.45 2 Option 2 1.95 1.30 1.12 Option 3 2.76 2.34 1.84 Option 4 1 1 1 7.3 Overall Value - First Line Buyers The overall value for each FLB was calculated based on their individual value functions and weights. The equation for overall value was based on the equation: Overall Value FLB Vcost * Wcost * + Wadmin time + Vprice + Vprof it * Wprice * Wprof it + Vquality * Wquality + Vadmin time + Vrelationship * Wrelationship Vincrease sales * Wincrease sales 116 The overall value outcomes for each of the FLBs are summarized in Table 69 and Table 70. For FLB 1, the high and base estimates reveal that Option 2 is the preferred choice, while for the low value estimates, Option 1 is higher. For FLB 2, the preference is consistently Option 2. Table 69. Overall Value for FLB 1 (bold values are the highest value in the subset) Overall Value - High Overall Value - Base Overall Value - Low Option 1 3.33 2.63 3.01 Option 2 3.85 3.44 2.74 Option 3 2.02 2.40 2.71 Option 4 1.91 1.91 1.91 Table 70. Overall Value for FLB 2 (bold values are the highest value in the subset) Overall Value - High Overall Value - Base Overall Value - Low Option 1 3.66 3.10 2.49 Option 2 4.14 3.34 3.03 Option 3 2.36 2.18 1.79 Option 4 1.77 1.77 1.77 Similarly for distributors, profit is comprised of price and cost. It was included in the analysis because the FLBs wanted to include it. However, as a comparison, the analysis was run again removing profit and retaining price and cost. The result of this analysis is summarized in Table 71 and Table 72. The same exact trends as with profit included are found. That is, the best option for FLB 2 is consistently Option 2 and the best for FLB 1 is Option 2 for high and base estimates and Option 1 for lowend estimates. 117 Table 71. Overall value for FLB 1 Options - Profit removed (bold values are the highest value in the subset) Overall Value - High Overall Value - Base Overall Value - Low Option 1 3.09 2.40 2.77 Option 2 3.62 3.20 2.50 Option 3 1.79 2.16 2.47 Option 4 1.86 1.86 1.86 Table 72. Overall Value for FLB 2 Options - Profit removed (bold values are the highest value in the subset) Overall Value - High Overall Value - Base Overall Value - Low Option 1 3.33 2.77 2.38 Option 2 3.81 3.23 2.92 Option 3 2.04 1.85 1.68 Option 4 1.66 1.86 1.66 7.4 Overall Value - Comparison Between Agents This section presents a scatter plot of the overall value of the different agents to compare between them. Figure 45 compares the overall value of the retailers versus the two distributors. Options 1 and 2 are more on the upper right side of the graph, indicative of the fact that they have a higher value than the other two options. At the lower left side of the graph, the two distributors have highly overlapping values, but a clear separation emerges in the upper right quadrant. Especially for Option 2 high-end estimates, there is a marked difference in value between the two distributors. 118 Distributors vs. Retailer Overall value 3.5 Opt. 1 High Opt. 3 High 3 Opt.3 Base OpOp. Oppt. 2.5 Opt2Hgh I Base 1 Base 3 Low OOpt. Opt. 3 LowLow 2 Opt. 2 Base Opt. 4 Base/Low/High 1.5 Opt. 2 Low [Dist1 f1 10lDist 2 0.5 0 0 1 2 3 4 5 Retailer Overall Value Figure 45.Distributor versus retailer overall value for all of the high/base/low options 1-4 Figure 45 compares the FLB and distributor value versus the retailer value. Since retailers are plotted on the x-axis, clear columns emerge where the relative difference in values for the FLBs and distributors can be examined. Similarly to the pattern in the other graph (Figure 45), greater differences between the agents emerge as the overall value of retailers increases. Again, Option 2 high-end estimates reveals a large difference between FLBs and distributors; the distributors have a much lower value than the FLBs. 119 FLB/Distributor vs Retailer Overall Value f..!W 4.00 050 0 Fs F1I Iriv I W ID: S050 0 0.0 2 1.5 Z . 3 ISf 4 4_ 3#qalt Ov"Orf VOue Figure 46. FLB/Distributor overall value versus Retailer overall value for all of the high/base/low options 1-4 8 Sensitivity Analysis It is important to understand the robustness of the results that were generated. Two sensitivity analyses were conducted. The first was by varying the input assumptions, having a high and low-end estimate in addition to the base numbers. This analysis was embedded in the overall value calculations and is not included in this section. This section specifically describes a sensitivity analysis on the weighting. The weights for each of the criteria were varied from 0% to 100% and the overall value was determined. As described in the methodology section, two points were plotted: the point where the criteria's weight was the same as what was determined by the agent. The overall value for this point, therefore, is the calculated overall value. The second point where the weight was 100%, where the overall value was equal to the partial value as derived from the value function. A weight sensitivity analysis was conducted on the distributors and FLBs. Retailers were not included in the weight sensitivity analysis because the variation between the weights for the retailer criteria was very narrow. 120 8.1 Distributors Weight sensitivity analysis was conducted individually for each of the distributors for all of the criteria. 8.1.1 Volume The weight for volume for Distributors 1 and 2 were varied from having a weight of 0% to 100% (Figure 47 and Figure 48). For Distributor 1, at the current weight of 16.5%, Option 1 is the best choice at 2.61 value (followed very closely by Option 3 at 2.58). However, as the weight of volume increases, Option 2 becomes the best choice. At approximately 65% weight on volume, the best option switches from 1 to 2. This is because even though the distributors are not selling the volumes that they would like in Option 2, they are still selling more than in Option 1. Distributor 2 follows a similar trend as Distributor 1, in that at about 50% weight, the best option switches from Option 1 to Option 2. Further, for Distributor 2, it seems like Option 4 becomes a better option as the weight of volume becomes almost 0%. 121 Overall Value vs. Weight - Volume - Dist. 1 ,.00 4.S0 400 Opt.1and3 300 2.SO 2.00 pt. 4p 1.O 100 100 SQ 60 40 20 0 120 WeSht for Vokwi Figure 47. Overall value versus weight for volume for distributor 1. The dashed line marks where the options intersect. Overall Value vs. Weight - Volume - Dist. 2 300 Opt. i 2.80 Opt. 3 260 2,40 Opt. 2 10 Opt. 4 1.40 1.20 1.00 0 20 20 0 40 50 60 0 so 90 100 Weght for Voiawe Figure 48. Overall value versus weight for volume for distributor 2. The dashed line marks where the options intersect. 122 8.1.2 Expiration Dates The sensitivity analysis of the weights of expiration dates for Distributor 1 (Figure 49) and Distributor 2 (Table 50) reveals that the best choices for both distributors do not change as the weight varies. In fact, the top choice gets even better. For Distributor 1, Options 1 and 3 remain almost identical in value across weight, but for Distributor 2, the values diverge at low weights. Overall Value vs. Weight - Expiration Date - Dist. 1 5.00 4.50 4.00 .. 50 3 00 Opt 1 and 3 p.2 1,50 200 100 000 0 10 20 30 40 WOIO 0 s or Expraefln De 70 80 90 100 Figure 49. Overall value versus weight for expiration dates for distributor 1 123 Overall Value vs. Weight- Expiration Date - Dist. 2 3 00 Opt. I 2 50 0 pt. 3 7I00 1.00 000 S 10 20 30 40 Weight fCr so 60 EXp*0a0" Oatft 70 g 9o 100 Figure 50. Overall value versus weight for expiration dates for distributor 2 8.1.3 Profit Margin The sensitivity analysis on weights for profit margin for Distributor 1 (Figure 51) and Distributor 2 (Figure 52) reveals that the order of preferences does not change as weight changes. Like for expiration date, the relative difference between the options increases as weight for profit margin increases. 124 - Overall Value vs. Weight - Profit Margin Dist. 1 A.50 too Opt. 4 coo )a 0 30 70 40 Weighs SOealV le s 50 60 70 so 100 90 fm Proft Margn egt PrftM ri t.1 D2t Figure 51. Overall value versus weight for profit margin for distributor .50 Overall Value vs. Weight - Profit Margon - Dist. 2 0.0 4,00 3.So J.00 Opt. I i so Opt. 3 2.00 000 Opt. 0 !0 20 30 fic 50 wpwghs Icr PrOfit Mattn 40 7D 00 90 10C Figure 52. Overall value versus weight for profit margin for distributor 2 125 Cost per Kit 8.1.4 The weight sensitivity analysis reveals an intersection point where the best option changes from Option 1 or 3 to Option 4. For both Distributor 1 (Figure 53) and Distributor 2 (Figure 54) this point occurs at approximately 30% weight on cost per kit. After this point, Option 4 becomes the preferred option. This is because the cost of Option 4 is $0, as kits are not being purchased. Thus, value increases as the cost goes to zero. Overall Value vs. Weight - Cost per Kit - Dist. 1 6.00 500Opt 4 400 Opt. 2 Opt. I and 3 > 000 0 70 40 60 so 100 170 W*htW W Coo pot Kit Figure 53. Overall value versus weight for cost per kit for distributor 1. The dashed line marks where the options intersect. 126 Overall Value vs. Weight - Cost per Kit - Dist. 2 b.00 otni t. S00 1.00 opt. 1 c o 0 20 40 W04ght so 60 o COost of1 Too 170 Figure 54. Overall value versus weight for cost per kit for distributor 2. The dashed line marks where the options intersect. 8.1.5 Cost per Training Similar to cost per kit, as cost for training decreases, the value increase. Thus, in the weight sensitivity analysis, for both distributors, there is a point where the best option goes towards Option 4, as there is no cost for training in Option 4. For Distributor 1 (Figure 55), the value of Option 4 intersects with Option 2 at 20% weight, after which Option 4 becomes the better option. However, for Distributor 1, Options 1 and 3 are always better except at 100% weight for cost for training, where Options 1, 3, and 4 are all the same. Distributor 2 (Figure 56) exhibits a change in the best option when the weight for cost per training is 80%. After 80%, the best option switches from Options 3 and 4 to Option 1. 127 Overall Value vs. Weight - Cost for Training -Dist. 1 4.S0 4.00 Opt. 1 and 3 2.CIU Opt. 2 150 Opt.4 100 050 0(X 30 ?0 10 0 50 40 70 60 so 9W 100 wows fo cos for Ttrolpae Figure 55. Overall value versus weight for cost for training for distributor 1. The dashed line marks where the options intersect. - Overall Value vs. Weight - Cost for Training Dist. 2 5,00 4.50 4.00 *3,50 Opt 3 and 4 2.so Opt 2 Opt. 1 2.00 150 100 0.00 0 10 20 30 40 so Wewits far Cost for 60 MV so 90 100 Tralrng Figure 56. Overall value versus weight for cost for training for distributor 2. The dashed line marks where the options intersect. 128 8.1.6 Efficiency of Distribution Sensitivity analysis for efficiency of distribution suggests that there is no change of the best option as weights increase for both distributors (Figure 57 and Figure 58). Distributor 2 indicates a divergence of Options 1 and 3 when weights decrease less than 14%. Overall Value vs. Weight - Efficiency of Distribution - Dist. 1 50 Opt. and 3 A.00 Opt.2 ISO 1 00 0-so 000 0 70 40 60 80 100 Welgigs fo Ewemy of Domtrobuton IbMOes/d**"ry} 120 Figure 57. Overall value versus weight for cost for efficiency of distribution for distributor 1. 129 Overall Value vs. Weight - Efficiency of Distribution - Dist. 2 ISo Opt. 2 1.00 o0o 60 Woighttt Mr Efkfeucy of DItruo 0 (b 120 too Be 40 70 W/dieky) Figure 58. Overall value versus weight for cost for efficiency of distribution for distributor2. 8.1.7 Cross Selling Sensitivity analysis for cross selling reveals no change in the best choice for either distributor (Figure 59 and Figure 60). There is an increase in the relative vales for each of the options as the weight increases. - Overall Value vs. Weight - Cross Selling Dist.1 4.00 opt. 1 3 SO 300 1250 1.opt. 4 coo 0 10 ?0 30 40 Welhgt o so 70 so 90 100 o CrO% SIng Figure 59. Overall value versus weight for cross selling for distributor 1. 130 - Overall Value vs. Weight - Cross Seliling Dist. 2 ai oc Opt 1 Opt. 3 Opt. 2 opt. 4 too oo 0 10 20 30 40 Woots so 60 70 80 90 too for Cross so" Figure 60. Overall value versus weight for cross selling for distributor 2 8.2 First Line Buyers A weight sensitivity analysis was conducted individually on both FLBs for each of the criteria. 8.2.1 Cost per Kit A sensitivity analysis on cost per kit for the FLBs reveals that as the weight on cost per kit increases, the best option varies. For both FLBs, when this attribute has the greatest weight, Option 4 becomes the best choice, because for Option 4 the cost is $0 because no kits are being purchased. For FLB 1 (Figure 61) as the weight increases past 20%, Option 2 is still the best choice, but Option 4 becomes the second best choice. At a weight of about 83%, Option 4 becomes the best choice for FLB 1. For FLB 2 (Figure 62), at about 75% weight, Option 4 becomes the best choice. 131 Overall Value vs. Weight - Cost per Kit - FLB 1 4 .5s Opt. 2 4 pt. 4 3 005 Opt. 3 10 30 70 50 40 W.*gt 60 90 W0 70 100 fow Coo. per K" Figure 61. Overall value versus weight for cost per kit for FLB 1. The dashed line marks where the options intersect. Overall Value vs. Weght- Cost per Kit - FLB 2 'I Opt. 4 45 ~~~~~Oot. 4 3 ........... OptI 1"Opt.3 a 10 20 30 40 W#4ghM 60 50 fWr C*St P"r 10 70 so 90 100 Figure 62. Overall value versus weight for cost per kit for FLB 2. The dashed line marks where the options intersect. 132 8.2.2 Price per Kit The price per kit sensitivity analysis finds that the best choice does not change when the weight is varied for both FLB 1 and 2 (Figure 63 and Figure 64). At the highest weight, Options 1, 2, and 3 converge. Overall Value vs. Weight - Price per Kit - FLB 1 45 4 Opt 2 OptI I ..... .......... ..................O p t. 3 2 1 Opt. 4 0.5 0 0 10 70 30 40 50 WVioM for Pr1 60 70 80 90 100 pot Kkt Figure 63. Overall value versus weight for price per kit for FLB 1. 133 Overall Value vs. Weight - Price per Kit - FLB 2 45 4 Opt. 2 5 0.S 0~ 0 10 20 10 40 so 60 70 so 90 100 WWCgt kr Price fot Kki Figure 64. Overall value versus weight for price per kit for FLB 2. 8.2.3 Quality The sensitivity analysis for quality reveals that for both FLBs, as the weight of quality increases so does the value of the options, while retaining the same order (Figure 65 and Figure 66). 134 Overall Value vs. Weight - Quality - FLB 1 S Opt. 2 Opt. 1 Opt. 3 0 70 40 60 80 100 120 W.4ght for Qtwtity Figure 65. Overall value versus weight for quality for FLB 1. Overall Value vs. Weight - Quality - FLB 2 Opt. 2 ........ .. ....... O p t. I Opt. 3 0 0 20 40 60 80 100 120 W#Igtm tor Qulity Figure 66. Overall value versus weight for quality for FLB 2 135 8.2.4 Administrative Time As the weight on administrative time changes, the best option for both FLBs changes dramatically. For FLB 1 (Figure 67), the order of the best options inverses, and Option 4 becomes the best option, as there is no administrative time at all. FLB 2 (Figure 67) reveals that at 100% weight, Option 4 is also the best option, but Options 1 and 3 are the same value at 4. For FLB 2, Option 1 replaces Option 2 as the best choice at approximately 8% weight (compared to the current 2.2% weight). At approximately 63%, Option 4 increases in value to be greater than Option 1. Thus, these particular results are somewhat sensitive to weight on administrative time. However, an increase from 2.2 to 8% is still more than doubling, implying that while administrative time is more sensitive than other results, it is still relatively robust. 136 Overall Value vs. Weight - Administrative Time - FLB 1 Opt 4 s CS Opt. 3 4 Opt. 1 Opt. 2 0s 0 0 10 20 30 60 50 40 WIgt tot Admhbtraft#v TkW* 70 100 90 so Figure 67. Overall value versus weight for administrative time for FLB 1. The dashed line marks where the options intersect. - Overall Value vs. Weight - Administrative Time FLB2 Opt.4 45 4 O pt1......... 1 Opt.2 A 0~O 0 to 30 40 Wogha 50 60 70 80 90 100 for Adminitratiw Thye Figure 68. Overall value versus weight for administrative time for FLB 2. The dashed line marks where the options intersect. 137 8.2.5 Profit Margin For FLB 1 (Figure 69) the relative order of the best choice does not change, although Options 1, 2, and 3 converge at a value of 5 at 100% weight. FLB 2 (Figure 70), however, does exhibit a change in the order; at approximately 15% weight, Option 1 overtakes Option 2 as the best choice. Option 3 increases as well to converge with Option 1 at value of 5 at 100% weight. Overall Value vs. Weight - Profit Margin - FLB 1 45 4 Opt. 2 OpOpt. Opt. 3 12.1, 105 0 to 70 30 40 50 60 70 80 90 100 Weight for Proft Margio Figure 69. Overall value versus weight for profit margin for FLB 1. 138 Overall Value vs. Weight - Profit Margin 4 - FLB 2 Opt 2 35Op1 Optp..3 Opt. 4 0 0 10 0 40 50 WOgM ow Po 60 70 80 90 100 Moron Figure 70. Overall value versus weight for profit margin for FLB 2. The dashed line marks where the options intersect. 8.2.6 Relationship with Donor As the weight on relationship with donor increases, the order of the best results does not change for the FLBs, but the relative value difference between the options does increase (Figure 71 and Figure 72). For both FLBs, Options 3 and 4 converge at 100% weight at the lowest value of 1. 139 Overall Value vs. Weight - Relationship with Donor - FLB 1 S 4.5 Opt 1 4 .3 3.S 3 25 2 Opt. 1 SOt.3 0 10 20 30 40 50 60 70 100 90 xO W440K for ReatloRSNp wkh Donor Figure 71. Overall value versus weight for relationship with donor for FLB 1. Overall Value vs. Weight - Relationship with Donor - FLB 2 5 45 Opt 2 4 I Ot Opt. 3 2 opt. 4 0.S 0 0 10 20 30 40 50 60 t0 80 90 100 WeigM for Rladanshp wth Donor Figure 72. Overall value versus weight for relationship with donor for FLB 2. 140 8.2.7 Increase in Sales Sensitivity analysis for increase in sales was plotted for FLB 2 only because FLB 1 did not have this as a criterion. FLB 2 exhibits a change in the best option from Option 2 to Option 1 at a weight of about 41% (Figure 73). - Overall Value vs. Weight - Increase in Sales FLB2 4.S 4 0 10 20 30 40 WOWt for 50 60 Incres" in Saw 70 80 90 100 Figure 73. Overall value versus weight for increase in sales for FLB 2. The dashed line marks where the options intersect. 9 Discussion The discussion section provides a commentary on the key results from the analysis and also insights from interviews and conversations with the different agents. It further notes limitations of the study and suggestions for improvement in further work. 9.1 Result from Analysis The purpose of this study was to understand the decision-making process, preferences, and priorities of agents in the private sector supply chain for malaria 141 rapid diagnostic tests (mRDTs) in Uganda. The different agents were asked about a decision process that included for options: 1. Sell WHO approved mRDTs outside of DevOrg's bundled service 2. Sell mRDTs through DevOrg's bundled service 3. Sell non-WHO approved mRDTs 4. Do not sell mRDTs The analysis in this study was framed around these four options. 9.1.1 Best Options for Agents The best options and overall value, based on value functions and weights, for the three different agents is summarized in Table 73. The option with the highest overall value for retailers was Option 2, selling the mRDTs through DevOrg's bundled service. However, the best option for distributors was Option 1, which was to sell WHO approved devices outside of the bundled service. The best option for the FLBs was the same as the retailers, Option 2. Even varying input estimations, the ordinal rank of the best option remains the same, suggesting robustness in the results. The only variation in results is the low-end assumption for FLB 1, where Option 1 becomes the best option. Table 73.comparison of overall value between different agents in the supply chain (using the baseline assumptions) First Line Buyers* Distributors* Retailers Option 1 2.86 2.63 2.74 Option 2 3.39 2.02 3.78 Option 3 2.29 2.56 1.85 Option 4 1.84 1.83 1.57 *The average of the two distributors and two FLBs were taken to be included in this table. 142 It is very interesting to compare the best options between the different agents in the supply chain (Table 73). The best option for the retailers is to sell the mRDT through the bundled service. One key factor in that may be the high value that retailers placed on training, which was a key component in the bundle. Distributors exhibit a marked difference between Option 2 and Option 1, which had the highest overall value for them. This dichotomy of results aligns with the qualitative feedback that individuals expressed during interviews and focus group discussions. Retailers generally found a great deal of value in Option 2, while distributors expressed reservations and concerns. FLBs tend to prefer Option 2, but under the low-end estimates of input values, FLB 1 revealed that Option 1 was the best choice. This result could be strongly influenced by the importance of administrative time. For Option 2, FLB 2 indicated that they would greatly prefer a lower administrative time. The low-end estimate of times in FLB 1's value function results in Option 1 being a 5 and Option 2 being a 1, likely influencing the final overall value. 9.1.2 Sensitivity Analysis A sensitivity analysis was performed on the weights and input assumptions. There was inherent uncertainty in the input assumptions because of limited data in the research context. To account for this uncertainty, a high and low-end estimation was generated based on interviews, expert opinions, and literature (see Inputs and Assumptions section). The results of this variation finds that the overall value outcomes are very consistent. The only result that changed was for FLB 1, where for the low-end estimates, the best option changed from Option 2 to Option 1. A sensitivity analysis was further performed on the weights. The weights of each of the attributes were varied from 0-100% to determine the impact on overall value. This analysis was performed on FLBs and distributors only, because the variation of weights of the retailers was so narrow. The sensitivity analysis finds that the results 143 are quite robust. For many attributes, varying the weights across the entire range does not change the best option. For other attribtues, the best option does change, but almost always changes with an increase of at least 20% points. Thus, the results in this analysis are fairly robust. 9.1.3 Implications and Improving Options These results have interesting policy implications. Many public health interventions have a wide range of different independent stakeholders who all have to be brought on board to realize positive change. This section highlights changes that could be made or considerations that should be taken into consideration to harmonize value across the supply chain. 9.1.3.1 Risk Sharing Across the Supply Chain In the supply chain that was part of this research study, the distributors owned the stock. Kits were procured through a bulk order, imported and initially owned by the FLB, and then sold to the distributor. The distributor then stores the product in warehouses. At this point, the distributor has paid the full cost of the product and is the entity that stands to lose any investment made when unsold kits expire. The FLBs are invested in supporting the distributors, but they will not face financial consequences if product is not sold. Further, it is clear from conversations that retailers perceive much less of a risk; they did not once mention unsold stock being a concern or barrier to uptake of mRDTs. The distributors we spoke with all expressed stress and great concern about not being able to sell the products and then losing a significant amount of investment. The distributor concern about volume is reflected in the outputs of their value functions. Option 2 for the criteria volume for the distributors ranks less than a 2 on a scale of 1-5. Increasing the volume of sales would significantly increase the overall value of Option 2 for distributors, especially because volume has the greatest weight of all the attributes. If the volume sold for Distributor 1 increases from 1,600 kits sold per month to 9,000 kits sold per month, then value increases from 1.36 to 3, and then the overall value would increase from 2.12 to 2.39 (12.7% increase); for Distributor 2 if the value increases from 1.98 to 3, the overall value would increase from 1.92 to 2.08 144 (8.3% increase). While this does not change the overall rank of options, it makes Option 2 much more viable than previously. One means of reducing the burden of unsold volume would be to distribute risk along the supply chain. Supply chain literature has many examples of risk mitigation strategies (Tang, 2006) that could be modified for the context of Africa. One example that might work in this setting is to set up a contract between manufacturers, FLBs, and distributors with a buy-back component. That is, the distributor will be at least partially refunded for any unsold products at the end of the expiry period. This would serve to incentivize FLBs and manufacturers to assist in driving sales and moving product. It would be further worth considering how to engage retailers in risk sharing. While these types of contracts are often set up with retailers in other supply chains, it is not clear that such a scheme would work in this setting. Individual retailers move such a small amount of stock comparatively and refunding a large number of small drug shops for relatively little stock would be costly in terms of both time and finances. It is worth researching and piloting other risk sharing mechanisms with retailers. Regardless, sharing the risk between FLBs and distributors would be greatly valued by the distributors and would likely increase the chances that distributors are willing to continue stocking the devices and engage in public health initiatives like this. 9.1.3.2 Demand forecasting Demand forecasting is an important process to any business, and especially so for manufacturers and other upper level supply chain agents, as the scale of operations increases. Especially in the context of expiration dates, demand forecasting is critical because stock that expires is a direct loss to the investor. Concern about expiration dates was an important factor that was brought up by FLBs and distributors. This was compounded by the fact that the distributors had already committed and purchased a large number of the devices. Demand forecasting has clear benefits in the global health space. It ensures essential products are sustainably available, it provides manufacturers with a better picture of the market so they can develop new 145 and improved products, it allows governments and funders to effectively allocate resources, and it reveals bottlenecks where supply doesn't meet demand, which can shape policy and advocacy efforts (Center for Global Development, 2007). A better understanding preferences, through studies like this one, can help improve understanding of the market such that forecasts can be more accurate. When demand forecasting is accurate, interventions that require an upfront investment are less likely to result in unsold stock for purchasers. This would be another avenue by which to improve the value of the intervention option for distributors. 9.1.3.3 Leverage Expertise As part of the contract DevOrg set up, the distributors are responsible for almost all of the services associated with the mRDTs, from advertising to training to waste management. In many ways, this was a prudent framework; the distributors are close to the market and are therefore more likely to have a better understanding of what works best and how to accomplish the requirements of the contract. On the other hand, the distributors expressed to us that they have a limited amount of expertise and time to accomplish all that is required of them. Two of the distributors brought up the idea that an NGO, either DevOrg or another in the malaria space, be responsible for training and some demand generation. The distributors asserted that NGOs like DevOrg have the expertise in training people about malaria and would be better suited for one-time investments in something like training. Investments in training costs are quite high, and based on their value functions, distributors have about a value of 2 out of 5 (not satisfied) for current costs. Decreasing that burden on the distributors would result in a higher overall value for this option. For example, if the cost per training was $100, then the value for Distributor 1 is 4.82 and Distributor 2 is 4.78. The change in overall value for Distributor 1 for Option 2 is from 2.12 to 2.41 and for Distributor 2 it increases from 1.92 to 2.26 (for the base assumption). This represents an increase of 13.7 and 17.7%, respectively, in overall value. While this doesn't change the options overall rankings, it does bring Option 2 to a much higher level and closer in value to the top options. 146 The distributors further expressed that they would be content to continue to spearhead the waste management efforts, as they already travel to the stores on a routine basis. Thus, there would be a division of responsibilities along the stakeholders in the project, ideally aligning with relevant expertise. 9.1.3.4 The cost to the retailer goes beyond the device Retailers expressed that stocking and administering an mRDT includes costs that had not been adequately accounted for by the upstream supply chain. That is, to administer an mRDT, the retailer had to factor in the cost of gloves, bandages to cover the finger prick, etc. The retailers said that this drives up the cost of the mRDT and it was necessary for them to correspondingly increase the price. It is worth considering how to better manage the extra costs associated with administering the mRDT. Although Option 2 was already the best option for the retailers, improving this aspect would make the option even more attractive. Further, when considering scaling up the project, cost factors, especially in the poorer, rural areas, will become an even greater consideration. 9.1.4 Insights from Value Functions and Weighting The overall value was only one aspect to this study and in addition to those results, interesting insights were derived from the elicitation of value functions and weighting. These two results themselves are useful in understanding the preferences of different agents in the supply chain. This section highlights some relevant results. 9.1.4.1 Retailers value the time it takes to complete a sale and extra opportunities Many of the criteria, or fundamental objectives, that were used in the methodology could be determined ahead of time through expert interviews and literature. For example, cost, volume, customer satisfaction, etc. were determined before the focus group discussions. However, during the focus group discussions moderators always incorporated time to include any new criteria that could come up, that hadn't occurred to the researchers. For retailers, it was possible to incorporate the 147 emergent criteria into the decision analysis. The second focus group discussion brought up two new and unexpected criteria: extra opportunities and time it takes to complete a sale. Extra opportunities was a catch-all phrase that referred to tangible and intangible benefits of working with a project or campaign. That is, retailers informed the researchers that when an NGO or company was trying to promote a product or initiative, they would often provide small opportunities for the retailers. These could include free shirts, which was part of DevOrg's bundled service, or the chance to work in a small role helping the project and being financially compensated. Other opportunities include prizes for selling products, promotional material, and more. Retailers expressed that these opportunities were a factor in their decision making process about stocking mRDTs. The median weight for the criterion opportunities was 7.79%. Time it takes to complete a sale refers to the amount of time that the storekeeper spends interacting with a customer to facilitate and complete a sale. The retailers explained that they spend around 30 minutes selling an mRDT because they first need to explain the product to the customer, convince him/her that testing was better for their health, draw the blood, wait for results, and then advise action based on test results (antimalarial or other treatment). While this was occurring, another customer may come in, see that the shopkeeper was occupied, and leave to go down the street to another shop. Thus, the storekeeper loses a sale while being preoccupied selling an mRDT. Retailers weighted time it takes to complete a sale as 10.5%, which was second only to training. It was quite unexpected that time to complete the sale was such an important criterion. This is a result that public health specialists should consider when designing interventions to increase appropriate diagnosis of malaria. 148 9.1.4.2 FLBs are not margin driven The FLBs expressed that achieving high margins on these products were not their priority. Margin was not initially in the research design to ask, rather the research broke margin into questions about volume, price, and cost. However, FLBs indicated that they thought in terms of margin percent, instead of price and cost, and so that information was recorded. The FLBs we spoke with indicated that one of their objectives was to provide high quality medical devices to the market in order to improve public health. One FLB expressed that their company would absolutely not consider selling low quality devices because that would go against their mission and goals. Thus, they prioritized margin much less than other factors. In fact, FLB 1 reported that margin had a weight of only 4.65% and FLB 2 had a weight on profit of 11%. Both interviewees said that they would consider continuing to import product as long as they did not lose money (although many also expressed reservations about the complexity, and time requirement of public health campaigns). In the interview, one FLB expressed that a increasing the profit margin to a sustainable level was much more important than increasing the margin above that to net greater profits. One important takeaway about this result is that identifying good partner companies that are interested in a social venture would best suit NGOs and governments seeking to conduct public health campaigns. Although companies do need to be concerned about profit, some companies, like the FLBs here, express that other factors-like providing a high quality product-are of equal or greater importance. Partnering with such organizations would represent an aligning of values between the public health entity and the private sector partners on the ground. 9.1.4.3 The importance ofcross selling Cross selling, or the increase of sales of other products, that occur because of selling the mRDT was something that emerged as an important benefit to selling mRDTs. Both distributors and retailers expressed this as a criterion in their decision about 149 stocking the devices. Retailers reported a weight of 8.4% and distributors of 11.6% for this attribute. In conversations, two drug shop owners reported that their sales in general had more than doubled after introducing the mRDTs. It is unclear if these individuals are an anomaly or if those figures are common across the spectrum of retailers. One distributor reported that an increase in his other sales was 16% and they was not especially pleased with that amount. In interviews, distributors have an expectation that a new initiative would result in an expansion of their distribution network and that was an important consideration when expanding into a new product. One consideration could be selling the mRDTs and ACTs together as part of a comprehensive malaria service. That is, a customer can come to a drug shop and pay a flat fee to be tested for malaria. If the result is positive, the customer receives the ACT antimalarial at no extra cost; if the result is negative, the customer does not get an ACT but may get other appropriate treatment. This would facilitate cross selling, and thus may be perceived as valuable by the retailers. It would also encourage testing before treatment. Cross selling of ACTs with the bundled service should be investigated further. 9.1.4.4 Retailers strongly value training The results from the weighting analysis find that retailers most prioritize training; it was given a normalized weight of 10.8%. In conversations, retailers expressed that they value the opportunity to provide another service to customers. Further, retailers appreciated that they learned why diagnosis was important and could convey that information to customers. This would likely increase sales and could also conceivably contribute to the prestige/reputation of the retailer. Thus, one of the key benefits of the bundled service for the retailers is the training scheme. 9.1.4.5 FLB disagreement about priorities There were some significant differences in priorities between the two FLBs interviewed. Specifically cost, administrative time, and relationship with donor differed by at least 10%. Cost for FLB 1 was quite unimportant, scoring a 30 out of 150 100, or 6.89% (Table 74), while for FLB 2 cost scored the highest at 100 and 22% (Table 75). It is not clear why there was such a divergence; it is possible that price and cost were being considered dependently on each other and FLB 1 scored cost as less important if price was the most important. FLB 1 weighted administrative time as 90 out of 100 (20.9%) while FLB 2 weighted it as 10 out of 100 (2.2%). FLB 1 had expressed frustration at the amount of time that the project takes up and that they had to hire two full time staff to manage the sales of the kits. FLB 2 said that they does not mind the amount of time the project takes, albeit they reported that they spent much less time than FLB 1. It is possible that the onus of administrative time was shared between FLB 1 and their distributor while for FLB 2 the distributors bore that burden. Indeed, both distributors under FLB 1 expressed that they spent a lot of time managing the sales of the mRDTs. Table 74. Weights for FLB 1 Criteria Cost per device Price per device Quality Administrative time Profit Relationship with donor Response 30 100 90 90 20 100 Normalized Weight 6.98% 23.3% 20.9% 20.9% 4.65% 23.3% Response 100 70 90 10 50 55 80 Normalized Weight 22.0% 15.4% 19.8% 2.2% 11.0% 12.1% 17.6% Table 75. Weights for FLB 2 Criteria Cost per device Price per device Quality Administrative time Profit Relationship with donor Percent increase in sales 151 Finally, FLB 1 recorded relationship with the donor organization or NGO as 100 (23.3%) while FLB 2 reported donor relationship as 55 (12.1%). Both FLBs affirmed that the relationship was indeed important, however they had ultimately had a different emphasis. 9.1.5 Emergent Criteria Some criteria were brought up in conversations with the different supply chain agents and were not able to be incorporated in the analysis for a variety of reasons, including inability to define an appropriate metric in the short amount of time available (see Methodology). However, the researcher recorded these criteria, as they are relevant to the individual's preferences. This section provides more information on the emergent criteria. 9.1.5.1 FLB value gaining clarityon the market Both FLBs articulated how they appreciated the opportunity to better understand the market, customers, and product while participating in the bundled service. They said that by participating in this project, they have had to work more closely with distributors and retailers, which then brings them into conversations around selling and marketing the products. The FLBs expressed that they valued learning more about retailers and the issues faced on the ground and would use that information to tailor their own sales. Further, it is information that can be passed along to the manufacturer, especially as it provides a quick feedback mechanism if there is something wrong with the devices. 9.1.5.2 FLBs value aestheticsof kit One FLB mentioned that a relevant attribute in their decision was the design of the packaging of the mRDTs. They said it was important for her considerations because it is something the retailers take into consideration when selling products. The rationale is that retailers can put a greater markup on a product that looks better because consumers will think of it as higher quality. Interestingly, aesthetics did not come up in either of the focus group discussions; however, that may have been 152 because retailers were implicitly including it in the criteria of quality. Aesthetics of kit packaging was not included in FLB considerations for a few reasons. First, the FLB articulated that they valued this because they believed it would lead directly to increased uptake on the retailer end; thus, the criteria being considered was actually volume. Additionally, in the short amount of time given for conversations, researchers were unable to define a metric that would be appropriate as a measurement for the criteria. 9.1.5.3 Retailer Incentives Currently, the primary incentive retailers have to sell product is that they made an upfront purchase of 25 mRDTs. Retailers expressed to the researchers that they are accustomed to being provided with mechanisms to incentivize selling products, like prizes for selling the most products or bulk discounts. This preference may have been expressed in the opportunities criterion. One distributor did say that they provided bulk discounts on retailers if they bought five or more boxes of mRDTs. Thus, drawing from this information, one way to aid the scale up of medical devices like mRDTs is to incorporate these incentive mechanisms at the retailer level. 9.2 Methodology One of the key motivations for this research study was to pilot a multi criteria decision analysis methodology to analyze and evaluate development supply chains. This section highlights insights from the methodology, points out improvements for future studies, and articulates the rationale for decisions that shaped the data collection process. 9.2.1 Data Collection MCDA methodology was adapted to facilitate data collection in the developing world context. This section discusses the rationale behind the adapted methods, limitations, and suggestions for future work. 153 9.2.1.1 Choosing the studypopulation The sample in this research study was taken from individuals participating in a bundled service intervention run by DevOrg. DevOrg is a non-profit organization with a mission to combat malaria and other communicable diseases and improve maternal and child health in Africa and Asia. DevOrg had put together a pilot initiative to scale up access to mRDTs in the private sector supply chain in Uganda. To this end, they constructed a bundled service that included training of retailers, marketing, waste management, and more. The contract was made with the manufacturers of WHO-approved devices and many of the requirements (waste management, training, etc.) were executed by the distributors. An initial amount of mRDTs were procured and introduced to the market; at the time of this research study, DevOrg's project was entering its second year. This research study took place in the context of DevOrg's bundled service. There were several reasons for this. First, at this point, mRDTs are quite uncommon in the Ugandan drug shops and private sector clinics, so asking retailers questions about these products would have been challenging because a significant amount of time would have needed to be spent introducing and explaining the diagnostic. Even after an introduction, it is unlikely that the retailer would have a good enough understanding about the product to be able to make informed decisions about stocking it. Second, DevOrg was a valuable partner in facilitating access to retailers, distributors, and first line buyers in the supply chain. For short turnaround studies like this one, rapid access to respondents is critical for the success of the evaluation. Finally, this context, where a public health intervention is taking place, is interesting to study. Interventions along these lines are fairly common and it is frequent that an agent in a supply chain is making a decision that includes an option about whether or not to partner with an NGO/government/non-profit on a project. The results from this study are interesting because the situation is reflective of the reality of the landscape. Further, this study can help inform and shape future public health interventions. 154 9.2.1.2 The Importance of CulturalAdaptation Multi criteria decision analysis is a fairly complex methodology and this type of study has many different aspects to the data collection process, all of which are subtle and can be easily confused. Even among individuals with an education similar to the facilitators, there are common mistakes and biases that occur and influence the results (Keeney, 2002; Gilberto Montibeller & von Winterfeldt, 2015). When this methodology is introduced to the developing world context, it becomes even more complicated because of educational and cultural backgrounds. For example, in general and with an audience of western educated individuals, drawing and showing the graphs to the respondents can assist in understanding about the nuance of the questions and also confirm the shapes of the functions to ensure that they match the respondents' intuition. However, the retailers in this study's sample would not have been able to understand the graphs. Before collecting data in Uganda, it was assumed that exhibiting graphs would not be appropriate and this intuition was confirmed once on the ground in conversations with DevOrg. Instead, the researchers adapted the methodology with the assistance of moderators in Uganda, who provided cultural expertise. The result of this was that a "happiness" scale was used to elicit value functions. Retailers were familiar with the type of direct ranking and were comfortable providing answers in this format. Examples relevant to the lives of the respondents were critical to explaining the study. The researchers generated the examples with assistance from experts and partners in Uganda. One situational example used that was particularly effective was in discussing purchasing a shirt. The researcher talked through the different considerations when purchasing a shirt (e.g., cost, quality, fit, color, etc.), the ranges and value of the criteria, and the trade off among these considerations (see Methodology section). In particular, using one example that continued through all of the parts of the data collection process (soliciting criteria, eliciting value functions, trade off analysis) was valuable for respondents' understanding. As a 155 facilitator develops expertise in this methodology in the context of the developing world, he/she will attain a repertoire of examples that will facilitate data collection. Another factor that likely contributed to the success of the data collection was the presence of a moderator who interfaced between the retailers and the researchers. This served a few purposes. First, although all of the research was conducted in English with respondents who were at a high level of proficiency, there were moments when translation into the local language clarified a concept. Second, the facilitator was able to explain concepts in a fashion that would make sense to the respondents. Finally, although the researchers made a point to be accessible and welcoming to questions, some individuals preferred to address queries to the mobilizer, who was a local. Again, as a researcher gains expertise as a facilitator of this methodology the need for a mobilizer will diminish. 9.2.1.3 Direction of Value There were several areas where issues may have arisen in eliciting value functions. For example, as described in the Results section, several sets of responses were eliminated from the analysis when they went opposite to the direction of value. It is proposed that there may be two factors at play that influenced such responses. First, the retailer may have been confused about the instructions. For example, it is highly likely that price and cost were confused and interchanged in the focus group discussion. Additionally, retailers may have made a mistake on their response form; the prompts were written on a large sheet of paper at the front of the room, so it is entirely plausible that a respondent made a transcription error. Another possibility is that the retailers had a preference in a different direction than had been established. For example, a respondent may have been expressing real value when saying that a higher cost had a higher value for them. Perhaps this is because the individual believes a higher cost represents a better product (see subsequent section on Correlation of Attributes). If this is the case, it is possible that the methodology used to elicit preferences would need to be adjusted and clarified 156 to the respondents. The direction of increasing value for the criteria had been set by the researchers and then proposed to the research group. Even though the group was asked to confirm what was proposed, it is possible that there were individuals who did not want to speak up and express their preferences. In future studies, it is recommended that researchers spend time establishing relative value across ranges for the retailers and determining if they have preferences in the opposite direction or if there is an issue in correlating two different criteria. 9.2.1.4 Correlation ofAttributes If respondents preferred a different direction of value than established by the researchers, then it is possible that they were associating two correlated but preferentially independent variables. There was evidence of this issue in conversations with FLBs so it is likely that retailers had a similar response. Respondents had a difficult time untangling corresponding but preferentially independent variables. An example of corresponding values could include cost and quality; as quality increases, often cost increases subsequently. However, this is not necessarily the case, and it is possible to conceive of valuing cost and quality independently. If two devices are of equal quality, the one with the lower cost will be preferred. However, in conversations, it was clear that some respondents could not untangle this concept. Future work on this methodology in this context would be to have concrete examples from the local context that exhibits the independence of two variables: for example, a recognizable product that is well known to be expensive but of poor quality. 9.2.1.5 HypotheticalDecisions Another challenge in the data collection process was in helping the respondents conceptualize a decision process based on a hypothetical situation. In this context, the decision had actually already been made: all individuals in this sample were part of DevOrg's bundled service. The researchers asked the respondents to reflect back to when they made the decision to enter into the bundled service and consider the options that were available to them at that time. Many individuals expressed confusion and reluctance when first asked this and needed to be coached through 157 the thought process. Further, it is possible that their already having lived out one of the options influenced their attributes and value functions. For example, distributors described volume of sales in the context of having already procured 300,000-600,000 devices. Additionally, one distributor was unable at all to understand this concept and refused to provide answers on his preferences. One idea to combat this confusion is to position this research such that the decision is real, not hypothetical. For example, if it were explained that DevOrg was making plans for an initiative and wanted their input, then individuals would be in the position of thinking about an actual decision that was being made. Future work should place an emphasis on considering what type of situations this methodology serves to provide clarity and insight on. 9.2.2 Data Analysis This section comments on the data analysis methods and assumptions that were incorporated in this study. 9.2.2.1 Weighting Both retailers and distributors did not have a wide spread between different criteria when assigning weight values. For retailers, there were only seven weights out of a total of 213 (all retailers and all criteria weights) that were below 40 out of 100. The median of the responses ranged from a low of 62 to a high of 90, and once normalized ranged from 7.4% to 10.8%, which is a very narrow range. The distributors ranged from 70-100 (11.6%-16.5%) for Distributor 1 and 80-100 (13%-16.3%) for Distributor 2. It is interesting to note that the FLBs had a spread from 20 (or 10) - 100 for weights. It is possible that there was a cognitive bias present when the data was being collected that resulted in a very narrow swing between weights. A scaling bias may have influenced these results, where the respondents created an artificial "floor" below which they did not assign a value. Scaling biases can occur because of where the high and low ranges of an attribute are set (Gilberto Montibeller & von Winterfeldt, 2015). If the ranges of a criterion were set artificially low or high, for example cost being unreasonably high or quality being terrible, a respondent may take that into account when assigning weights and 158 inflate the weight of a criteria that has an extreme range. For example, a retailer would think it is important not to have a terrible quality device and so the weight on the criterion quality would be high. However, in reality, the range of devices would not include those of the worst quality because a retailer would not seriously consider purchasing such a device. In evaluating the ranges of criteria that were established in the discussions, it is possible that some of the ranges were artificially extreme. For example, cost of a device to the retailer ranged from $0 to $1. Value functions that emerged demonstrated that retailers' value drops to 1 (the lowest) at about $0.70. However, many other criteria did not seem to have extreme ranges. Thus, alternatively, for some attributes if the range was reasonable, another possible reason for the bias was that the respondents conflated "importance" with priority (G. Montibeller & Franco, 2007). That is, respondents articulated that all the criteria that they had identified were important, which of course they are, because those were the factors identified in making a decision. However, it is important to take into consideration the range of attributes, as mentioned previously. For a distributor, for example, if the cross selling can only reasonably increase to 20%, the difference between the worst option and the best option is 0%-20% not 0% to some drastic increase that is not realistic. Failure to keep this mind may result in inflated weights for some of the criteria. While efforts were made by the researchers to control for this bias, there were challenges in explaining this complicated methodology in the short amount of time available for data collection with the respondents. Further, in focus group discussions there were many more participants than moderators and that may have resulted in a failure to fully communicate the nuance of the instructions (see previous section on Data Collection for thoughts on methods improvements). 9.2.2.2 Input Assumptionsand SensitivityAnalysis After eliciting value functions and weights for the criteria, multi criteria decision analysis can be used to derive an overall value for each of the decision options. To 159 do this, it is necessary to generate a decision matrix that summarizes the expected outcomes of each of the options along the different criteria. Generation of these input values was done through a combination of using data from the focus groups and interviews, expert opinions by MIT faculty and DevOrg representatives, and relevant literature. Ideally in MCDA methodology, the decision makers are part of constructing the matrix of inputs. However, due to the limited amount of time the researchers were able to spend with each informant, this was not possible. There is inherent uncertainty in the inputs of these values. To address this, three categories of inputs were used: a high estimate, a low estimate, and a base estimate. The analysis was run with all three categories and the overall value ordinal rank compared. For the retailers, regardless of whether the high, low, or base assumptions of inputs were used, the best option was always Option 2, selling the mRDT through the bundled service. For distributors, the best option was always Option 1. This suggests that the results of this study are robust. Future work could consider running a Monte Carlo simulation to further assess results. 10 Conclusion and Future Research Direction This research study offers insight into preferences and priorities of retailers, distributors, and first line buyers in the supply chain for malaria rapid diagnostic test kits. A multi criteria decision analysis methodology was adapted for this context to understand a decision process that had four possible outcomes: 1) purchase WHO-approved mRDTs outside of a bundled service, 2) purchase WHO-approved devices through the bundled service, 3) purchase non-WHO approved devices, and 4) do not purchase mRDTs. It was found that, in general, retailers and first line buyers prefer Option 2 (purchase mRDTs through the bundled service) while distributors prefer Option 1 (purchase WHO-approved mRDTs outside of the bundled service). A sensitivity analysis on input assumptions and weights was performed and the results appear relatively robust. The discussion comments on 160 aspects of the intervention that could be altered to harmonize the best choices across the supply chain. This research study was a pilot to test a rapid evaluation method and was therefore performed on a short timeline, with only two weeks in the field to collect data. Thus, one limitation of this research was that it was not possible to have multiple iterations of the questionnaire that was employed. Future research would build from this experience, and improve the data collection process. For example, collecting relevant examples to clarify common misconceptions would be very useful, in addition to holding focus groups with smaller numbers of individuals to ensure understanding and communication among participants. Further, it is worth exploring this methodology when agents are actively making a decision, instead of after the decision, as was the case in this research study. While this decision was interesting to understand, clarifying hypothetical situations and framing value functions was challenging to do with participants. Further, it was not possible to include the manufacturers in this study. Future research should ensure to obtain their perspective, as it is an important part of the supply chain. Finally, it is recommended that a monte carlo simulation be performed as a sensitivity analysis. In this study, sensitivity analyses were performed on input parameters and weights. However, since there are so many criteria and agents, it would be worth running simulations to better understand the robustness of the results. 161 References ACTwatch Group, P. (2014). ACTwatch Study Reference Document: Uganda Outlet Survey 2013. ACTwatch Group, Program for Acessible Health, Communication & Education (PACE), & The Independent Evaluation (IE) Team. (2012). ACTwatch Outlet Survey Report 2011 (Round 4). Amexo, M., Tolhurst, R., Barnish, G., & Bates, 1. (2004). Malaria misdiagnosis: Effects on the poor and vulnerable. Lancet, 364(9448), 1896-1898. http://doi.org/10.1016/S0140-6736(04)17446-1 Awor, P., Wamani, H., Bwire, G., Jagoe, G., & Peterson, S. (2012). Private sector drug shops in integrated community case management of malaria, pneumonia, and diarrhea in children in Uganda. AmericanJournalof TropicalMedicine and Hygiene, 87(SUPPL.5), 92-96. http://doi.org/10.4269/ajtmh.2012.11-0791 Baltussen, R., & Niessen, L. (2006). Priority setting of health interventions: the need for multi-criteria decision analysis. Cost Effectiveness and Resource Allocation, 4, 14. http://doi.org/10.1186/1478-7547-4-14 Bate, R., & Hess, K. (2009). Affordable Medicines Facility for malaria. The Lancet Infectious Diseases, 9(7), 396-397. http://doi.org/10.1016/S14733099(09)70158-6 Center for Global Development. (2007). A Risky Business: Saving money and improving global health through betterdemandforecasts. Group. Retrieved from http://www.cgdev.org/files/13784_fileRiskyBusinessFull.pdf Chandler, C. I. R., Hall-Clifford, R., Asaph, T., Pascal, M., Clarke, S., & Mbonye, A. K. (2011). Introducing malaria rapid diagnostic tests at registered drug shops in Uganda: Limitations of diagnostic testing in the reality of diagnosis. Social 162 Science and Medicine, 72(6), 937-944. http://doi.org/10.1016/j.socscimed.2011.01.009 D'Acremont, V., Kahama-Maro, J., Swai, N., Mtasiwa, D., Genton, B., & Lengeler, C. (2011). Reduction of anti-malarial consumption after rapid diagnostic tests implementation in Dar es Salaam: a before-after and cluster randomized controlled study. MalariaJournal,10(1), 107. http://doi.org/10.1186/14752875-10-107 Figueira, J., Greco, S., & Ehrogott, M. (2005). Multiple criteriadecision analysis:state of the artsurveys. Boston: Kluwer Acadademic Publishers. Gelband, H., Panosian, C., & Arrow, K. (Eds.). (2004). Saving Lives, Buying Time: Economics of MalariaDrugs in an Age of Resistance. National Academies Press. Guerin, P. J., Olliaro, J. (2002). P., Nosten, F., Druilhe, P., Laxminarayan, R., Binka, F., ... White, N. Malaria: Current status of control, diagnosis, treatment, and a proposed agenda for research and development. Lancet Infectious Diseases, 2(9), 564-573. http://doi.org/10.1016/S1473-3099(02)00372-9 Keeney, R. L. (1982). Decision Analysis: An Overview. OperationsResearch, 30(5). Keeney, R. L. (2002). Common Mistakes in Making Value Trade-Offs. Operations Research, 50(6), 935-945. http://doi.org/10.1287/opre.50.6.935.357 Keeney, R. L., & Gregory, R. S. (2005). Selecting Attributes to Measure the Achievement of Objectives. OperationsResearch, 53(1), 1-11. http://doi.org/10.1287/opre.1040.0158 Keeney, R. L., & Raiffa, H. (1976). Decisionswith Multiple Objectives: Preferences and Value Tradeoffs. New York: John Wiley & Sons. Keller, L. R., & Wang, Y. (2009). Multiple-Objective Decision Analysis Involving Multiple Stakeholders, 139-155. http://doi.org/10.1287/educ.1090.0066 163 Koksalan, M., Wallenius, J., & Zionts, S. (2011). Decision CriteriaDecision Making: From Early History to the 21st Century. Singapore, Singapore: Mainland Press Pte Ltd. J., Asiimwe, Tibenderana, J. C., Nakanjako, D., Nabakooza, J., Counihan, H., & Kyabayinze, D. K. (2010). Use of RDTs to improve malaria diagnosis and fever case management at primary health care facilities in Uganda. MalariaJournal,9, 200. http://doi.org/10.1186/1475-2875-9-200 MacKintosh, C. L., Beeson, J. G., & Marsh, K. (2004). Clinical features and pathogenesis of severe malaria. Trends in Parasitology,20(12), 597-603. http://doi.org/10.1016/j.pt.2004.09.006 Marsh, K., Dolan, P., Kempster, J., & Lugon, M. (2013). Prioritizing investments in public health: A multi-criteria decision analysis. Journalof PublicHealth (United Kingdom), 35(3), 460-466. http://doi.org/10.1093/pubmed/fds099 Mbonye, A. K., Magnussen, P., Chandler, C. I., Hansen, K. S., Lal, S., Cundill, B., ... Clarke, S. E. (2014). Introducing rapid diagnostic tests for malaria into drug shops in Uganda: design and implementation of a cluster randomized trial. Trials, 15(1), 303. http://doi.org/10.1186/1745-6215-15-303 Mbonye, A. K., Ndyomugyenyi, R., Turinde, A., Magnussen, P., Clarke, S., & Chandler, C. (2010). The feasibility of introducing rapid diagnostic tests for malaria in drug shops in Uganda. MalariaJournal, 9(1), 367. http://doi.org/10.1186/1475-2875-9-367 Mbonye, A., Lal, S., Cundill, B., Hansen, K. S., Clarke, S., & Magnussen, P. (2013). Treatment of fevers prior to introducing rapid diagnostic tests for malaria in registered drug shops in Uganda. MalarJournal, 12(1), 131. http://doi.org/1475-2875-12-131 [pii]\nlO.1186/1475-2875-12-131 164 Mendoza, G. a., & Martins, H. (2006). Multi-criteria decision analysis in natural resource management: A critical review of methods and new modelling paradigms. ForestEcology and Management, 230(1-3), 1-2 2. http://doi.org/10.1016/j.foreco.2006.03.023 Montibeller, G., & Franco, a. (2007). Decision and Risk Analysis for the evaluation of Strategic Options. Supporting Strategy: Frameworks, Methods and Models. Montibeller, G., & von Winterfeldt, D. (2015). Cognitive and Motivational Biases in Decision and Risk Analysis. Risk Analysis, n/a-n/a. http://doi.org/10.1111/risa.12360 Ndyomugyenyi, R., Magnussen, P., & Clarke, S. (2007). Diagnosis and treatment of malaria in peripheral health facilities in Uganda: findings from an area of low transmission in south-western Uganda. MalariaJournal,6, 39. http://doi.org/10.1186/1475-2875-6-39 Nutt, D. J., King, L. a., & Phillips, L. D. (2010). Drug harms in the UK: A multicriteria decision analysis. The Lancet, 376(9752), 1558-1565. http://doi.org/10.1016/S0140-6736(10)61462-6 O'Connell, K. a, Gatakaa, H., Poyer, S., Njogu, J., Evance, I., Munroe, E., . . Chavasse, D. (2011). Got ACTs? Availability, price, market share and provider knowledge of anti-malarial medicines in public and private sector outlets in six malariaendemic countries. MalariaJournal, 10(1), 326. http://doi.org/10.1186/14752875-10-326 Palafox, B., Patouillard, E., Tougher, S., Goodman, C., Hanson, K., Buyungo, P., ACTwatch Group. (2012). ACTwatch 2009 Supply Chain Survey Results, Uganda, (January). Rutebemberwa, E., Pariyo, G., Peterson, S., Tomson, G., & Kallander, K. (2009). Utilization of public or private health care providers by febrile children after 165 user fee removal in Uganda. MalariaJournal, 8, 45. http://doi.org/10.1186/1475-2875-8-45 Sanyal, B., Frey, D., Graves, S., de Weck, 0., Brine, D., Goentzel, J., ... Montgomery, B. (2015). Experimentation in ProductEvaluation: The case of solar lanternsin Uganda, Africa. Stewart, T. J., Joubert, A., & Janssen, R. (2010). MCDA framework for fishing rights allocation in South Africa. Group Decision and Negotiation, 19(3), 247-265. http://doi.org/10.1007/s10726-009-9159-9 Tang, C. S. (2006). Perspectives in supply chain risk management. International Journalof ProductionEconomics, 103(2), 45 1-488. http://doi.org/10.1016/j.ijpe.2005.12.006 Trampuz, A., Jereb, M., Muzlovic, I., & Prabhu, R. M. (2003). Clinical review: Severe malaria. CriticalCare (London, England), 7(4), 315-323. http://doi.org/10.1186/cc2183 Tuteja, R. (2007). Malaria - An overview. FEBSJournal,274(18), 4670-4679. http://doi.org/10.1111/j.1742-4658.2007.05997.x Uganda Bureau of Statistics. (2014). National Population and Housing Census 2014: Provisional Results. Retrieved from http://unstats.un.org/unsd/demographic/sources/census/2010_PHC/Uganda /UGA-2014-11.pdf Uganda Office Of The Auditor-General. (2010). Value for Money Audit Report on the Regulation of Medicines in Uganda by National Drug Authority. Retrieved from https://afrosai-e.org.za/sites/afrosai-e.org.za/files/reports/Regulation of medicines (2010).pdf 166 Wakiso District Local Government. (2009). Higher Local Government Statistical Abstract. Retrieved from http://www.ubos.org/onlinefiles/uploads/ubos/2009_HLG_ Abstract-printed/WAKISOFINAL_ 2009.pdf Wang, J. J., Jing, Y. Y., Zhang, C. F., &Zhao, J. H. (2009). Review on multi-criteria decision analysis aid in sustainable energy decision-making. Renewable and Sustainable Energy Reviews, 13(9), 2263-2278. http://doi.org/10.1016/j.rser.2009.06.021 White, N. J. (2004). Review series Antimalarial drug resistance. Trends in Parasitology,113(8), 1084-1092. http://doi.org/10.1172/JCI200421682.1084 Williams, H. A., Causer, L., Metta, E., Malila, A., O'Reilly, T., Abdulla, S., ... Bloland, P. B. (2008). Dispensary level pilot implementation of rapid diagnostic tests: an evaluation of RDT acceptance and usage by providers and patients--Tanzania, 2005. MalariaJournal, 7, 239. http://doi.org/10.1186/1475-2875-7-239 Wilson, M. L. (2012). Malaria rapid diagnostic tests. ClinicalInfectious Diseases, 54(11), 1637-1641. http://doi.org/10.1093/cid/cis228 Wongsrichanalai, C., Barcus, M. J., Muth, S., Sutamihardja, A., & Wernsdorfer, W. H. (2007). A review of malaria diagnostic tools: Microscopy and rapid diagnostic test (RDT). American Journalof Tropical Medicine and Hygiene, 77(SUPPL. 6), 119-127. http://doi.org/10.3126/ajms.v1i2.2965 World Health Organisation. (2006). The Use of Malaria Rapid Diagnostic Tests. Retrieved from http://www.wpro.who.int/malaria/NR/rdonlyres/44C556B80E04-4D26-BC1DD00D2743B2C0/0/TheuseofMalariaRDTs_2ndedition_2006.pdf?ua=1 World Health Organisation. (2011a). Global plan for artemisinin resistance containment. Retrieved from 167 http://apps.who.int/iris/bitstream/10665/44482/1/9789241500838_eng.pdf ?ua=1 World Health Organisation. (2011b). Good practicesforselecting and procuring rapid diagnostictests for malaria. World Health Organisation. (2014a). Malaria Rapid Diagnostic Test Performance. World Health Organisation. (2014b). World malaria report 2014. http://doi.org/10.1007/s00108-013-3390-9 Yeka, A., Gasasira, A., Mpimbaza, A., Achan, Rosenthal, P. J. (2012). J., Nankabirwa, J., Nsobya, S., Malaria in Uganda: challenges to control on the long road to elimination: I. Epidemiology and current control efforts. Acta Tropica, 121(3), 184-95. http://doi.org/10.1016/j.actatropica.2011.03.004 Zopounidis, C., & Doumpos, M. (2002). Multi-criteria decision aid in financial decision making: methodologies and literature review. Journalof MultiCriteria DecisionAnalysis, 11(4-5), 167-186. http://doi.org/10.1002/mcda.333 168