Quality and Accountability in Health: Audit Evidence from Primary Care Providers Jishnu Das (World Bank and Centre for Policy Research) With Alaka Holla (World Bank) Karthik Muralidharan (UCSD) SITE June 2013 The Problem Strong theoretical reasons why health care should be public U(government) ≠ U(Consumer): Pendergast (2003) Patient satisfaction among narcotic addicted patients not a good measure of how good the doctor is Private sector aggregator of customer feedback Medical care arguably a credence good You don’t know what you need, but observe utility from what you get Widely believed to produce inefficiencies in the market Darby and Karni (1973): Over-treatment Wolinsky (1993): You can’t observe what you bought; treat “low”, charge “high” Gruber and Owens (1996): Caesarian sections Balfoutas and others (forthcoming): Greek taxi drivers (over provision and over charging) And yet… 77% of providers have no degree, 18% have some other degree (BAMS, BIMS, BUMS, BHMS), and only 4% have an MBBS degree (roughly equivalent to MD in the U.S.). Average village has 3.36 providers with no degree, 0.80 providers with some degree, and 0.18 providers with an MBBS degree 80% of first-contacts (primary care) in India in private sector New nationwide study: 77 percent of private providers in rural areas do not have medical training Contrast: All public providers are (supposedly) trained, the majority with an MBBS And yet… Public share increases from 20% to 35% in villages where there is a public doctor but households still visit private providers in 65% of primary care cases. Not (just) because there aren’t enough public sector providers Why? What people demand from health care providers very different from what the public sector provides Hypothesis 1: Decreases quality, increases costs Example: Demand for injections/steroids leads to lower quality for higher cost Peer and Administrative accountability from experts in regulated (and in low-income settings) health care beats customer accountability through the market Hypothesis 2: Increases cost, but at increased quality Example: Poor governance in public sector (Chaudhury and Hammer 2004, Chaudhury et. al. 2006, Das and Hammer 2007) Low effort arising from poor administrative accountability is hard to quantify But is potential one large source of losses: `Quiet Corruption’ In health care: Arrow (1963) This paper Use audit studies with patient and provider fixed-effects to assess quality in public and private sector 22 people recruited from the local community and extensively trained visits multiple providers presenting the same set of symptoms. Providers do not know that this is not a real patient Show effect of practicing in private sector on Adherence to medically required checklists Under-treatment Over-treatment Assess whether there is a price-quality relationship in private sector Three literatures it relates to Customer Accountability in private sector versus administrative accountability in public sector What’s out there and what we don’t know New empirical and theoretical literature on credence goods Dulleck and Kerschbamer (2006, 2011, 2012) Schneider (2012), Balfoutas(forthcoming) Bonroy and others (2012) Audit studies in labor markets and services Primary around issues of discrimination What are we adding This paper: What Deploy “standardized patients” (audit study) 2 Related studies People recruited from local communities and extensively trained to present with the same symptoms to multiple providers Largest such study to date (1105 interactions) Compare market care to provider in public clinic (64% not doctors) Compare same doctor in public and private practice Note: unlike audit studies of car buying or home rentals, we always observe a completed sale Problems arising from potentially off-equilibrium behavior may still remain We try out various strategies to interpret these results in the light of known issues with audits Overview of Results Significant evidence of over-treatment and under-treatment relative to medical protocols Conditions are not diagnosed or treated appropriately Many medicines are not required BUT Public clinics provide similar care to private clinics Joint effect of public sector with provider characteristics Effects vary depending on measure of quality used 72% private sector providers have no medical training The same provider in his/her private clinic provides better care than in his/her public clinic across all quality measures Customer accountability rewards better quality with higher prices No link between provider wages in public sector and measures of quality Remainder of talk Where we worked (and what does it look like) What we did What we found Ruling out (some) interpretations of the data Worry in particular about off-equilibrium behavior This paper: Where? All districts divided into 5 Socio-Cultural Regions (SCRs); one district from each SCR 20 randomly chosen villages from each district Representative sample of all types of providers in 3 districts of Madhya Pradesh (and public providers in 2 more); majority has no medical training Additional sample from (urban) Delhi MP Study: The sample 1 2 3 • In each sampled village, surveyors complete Participatory Resource Assessments (PRAs) in at least 3 different geographical locations and ask for a list of all providers they visit for primary illnesses • A unique list is compiled and a Master Code File (MCF) is filled out. A short survey is administered with each provider listed in the MCF • Then a household census is completed in which members are asked about all illness in the last one month and names and locations of providers they went to 4 • If more than 5% of households report visiting a provider in a location (village/town) outside the village, that village/town is now considered a part of the health-market for the village. These are referred as “clusters”, generally on the main highway near the village 5 • Once all clusters are identified, surveyors visit each cluster and conduct PRAs in the same manner. All providers practicing in the clusters are added to the MCF and a survey is implemented Rural India: MP 100 villages in MP, randomly selected in 5 districts—located >1000 health care providers Snapshots of the two remotest districts Standardized patients sample Sample restrictions Ruled out 2 remote districts entirely for private market Ruled out remote locations in other 3 districts Sampled All MBBS private providers All public clinics in all districts All But no more than 2 doctors per clinic private clinics of public doctors in all districts Add in untrained till we have 6 providers per sampled village Basic Sample Description Table 1: Health Market Attributes Total Inside village (1) (2) Panel A: Number of providers Total Public MBBS Public alternative qualification Public paramedical Public unqualified Private MBBS Private alternative qualification Private unqualified 11.05 (1.25) 0.50 (0.11) 0.22 (0.05) 1.58 (0.19) 1.70 (0.17) 0.42 (0.16) 1.92 (0.36) 5.40 (0.60) 3.06 (0.37) 0.05 (0.02) 0.07 (0.03) 1.13 (0.15) 0.67 (0.10) 0.00 0.00 0.23 (0.07) 1.81 (0.22) Outside village (3) 7.99 (1.29) 0.45 (0.10) 0.15 (0.04) 0.45 (0.13) 1.03 (0.15) 0.42 (0.16) 1.69 (0.37) 3.59 (0.61) (contd) Table 1: Health Market Attributes Total Inside village (1) (2) Panel B: Composition of demand Population (2001 Census of India) 3885.00 1353.74 (385.46) (103.56) Probability household visited provider in last 30 days Probability visit was to private sector Probability visit was to private sector in villages with at least 1 public doctor Probability visit was to MBBS doctor Probability visit was to unqualified doctor Number of villages Number of households Number of reported household-visits 2531.26 (378.58) 0.46 (0.00) Probability visited provider was inside/outside village Distance traveled to visited provider (km) Outside village (3) 1.66 (0.02) 0.91 (0.00) 0.79 (0.01) 0.03 (0.00) 0.76 (0.00) 100 23306 18632 0.65 (0.00) 0.40 (0.01) 0.35 (0.00) 3.92 (0.03) Standardized patients SPs 22 SPs recruited from the local community Important so that their appearance and manner conform closely to providers’ expectations Thoroughly trained to make plausible excuses to avoid invasive exams “palm” medicines if required 150+ hours of training First tried in Delhi pilot No adverse events; <1% detection rate Standardized patients Three standardized cases Unstable Agina: “Doctor, this morning I had a pain in my chest” – Ramlal, Male, 45 years old Proxy Dysentery: “Doctor, my 2 year old child has been suffering from diarrhea for 2 days” – Shankarlal, Male, 25 years old Asthma: “Doctor, last night I had a lot of difficulty in breathing” – Rajesh (Male) or Radha (Female), 25 years old Cases chosen such that Relevance to the Indian context Increasing incidence of cardiovascular and respiratory illness in India Diarrheal diseases kill approximately 200,000 children per year (Black et al. 2008) No invasive treatment required Important to minimize any potential harm to SPs Standardized patients What is measured Quality of care through adherence to required and essential checklist of questions and examinations that the provider should complete for each patient Why this may be preferable Treatment: correctness, incorrectness, use of antibiotics and steroids for cases where they are not required Diagnosis: whether given, whether correct Time spent, total questions asked, total examinations completed Relation between quality measures Figure 1a Figure 1b Consultation Time and Adherence to Checklist 0 10 20 Total Time Spent (Mins) Consultation Time and Adherence 30 Density of Time Spent .03 .01 .02 Density of Adherence .4 0 .1 0 .2 .1 .3 .2 Correct Treatment Given 0 10 20 30 Density of Time Spent 40 .5 .3 50 Adherence to Checklist and Treatment 0 20 40 60 Adherence to Checklist (Percent) Checklist Adherence and Treatment 80 Adherence Density 1. Doctors under-treat because they figured out that these were not “real patients”. But then, we should see that “correct treatment” is less likely for doctors who spend more time and complete more of the checklist, since they would be more likely to figure out that the patient is not “real”. We find exactly the opposite 2. Little evidence of signaling through medically irrelevant costly effort: more effort leads to better treatment through 90 percent of the distribution Basic Sample Description Table 1. Characteristics of Providers and Practices Audit 1 Panel A: Provider characteristics Age of Provider More than 12 years of basic education Has MBBS degree No medical training Has multiple practices Panel B: Practice characteristics Tenure in years at current location Dispense medicine Consultation fee (Rs.) Average number of patients per day Electricity Stethoscope Blood pressure cuff Thermometer Weighing Scale Handwash facility Number of providers Notes: Audit 2 Public Private (1) (2) p-value of (1)-(2) (3) 47.03 0.59 0.26 0.62 1.18 43.33 0.53 0.08 0.67 1.07 15.49 1.00 0.95 28.33 0.95 0.97 0.84 0.95 0.87 0.89 39 13.45 0.82 12.92 16.25 0.95 0.95 0.76 0.92 0.51 0.83 206 Dual p-value of (4)-(5) (6) Public Private (4) (5) 0.059 0.507 0.001 0.473 0.022 44.53 0.62 1.00 0.00 1.83 45.43 0.69 1.00 0.00 2.16 0.523 0.277 0.260 0.003 0.000 0.000 0.920 0.477 0.274 0.584 0.000 0.315 7.09 0.57 28.78 24.40 0.98 0.99 0.99 0.94 0.94 0.86 143 8.08 0.37 41.34 17.05 1.00 1.00 1.00 0.98 0.84 0.83 94 0.318 0.004 0.002 0.027 0.166 0.427 0.427 0.212 0.011 0.537 Public Private (7) (8) p-value of (7)-(8) (9) 45.43 0.69 1.00 0.00 0.000 8.08 0.37 41.34 17.05 0.96 0.99 0.99 0.95 0.93 0.83 94 1.00 1.00 1.00 0.98 0.84 0.83 94 0.087 0.328 0.328 0.435 0.074 0.915 Results Checklist adherence IRT Scores Treatment Diagnosis Prices Checklist adherence Checklist adherence Checklist adherence Full Sample Audit 1 (1) (2) (3) (4) Dependent Variable: Percentage of Checklist Items Mean 22.38 22.32 21.74 21.81 SD 16.82 16.39 16.49 16.26 Is a public provider -7.932*** -8.302*** -8.046*** -7.749*** (1.637) (1.688) (2.655) (2.664) Has MBBS 2.368 2.173 (2.706) (2.700) Has some qualification 2.205 2.144 (1.562) (1.567) Age of provider -0.023 -0.004 (0.057) (0.056) Gender of provider (1=Male) -1.097 -0.708 (3.846) (3.911) PCA of clinic infrastructure 1.101* 1.238** (0.583) (0.578) Number of patients waiting -0.196 -0.112 (0.321) (0.440) Case II (Dysentery) 19.881*** 1.167 -1.155 -1.370 (2.528) (4.725) (4.655) (4.657) Case III (Asthma) 24.476*** 6.193 3.250 3.666 (2.280) (4.829) (4.869) (4.789) Constant 26.782*** 27.064*** 24.724*** 25.076*** (4.226) (4.841) (3.951) (4.898) Market fixed effects Yes Yes Yes Yes Provider fixed effects F-stat (All SP FE = 0) 65.3 11.6 12.1 11.8 R2 0.334 0.321 0.245 0.253 Number of observations 1,129 1,013 668 640 Note: All regressions include SP fixed effects Audit 2 (5) (6) Dual (7) (8) (9) (10) 23.29 23.19 23.54 23.03 23.10 23.54 17.26 16.61 17.04 17.61 17.13 17.49 -7.226*** -7.808*** -7.483*** -7.390*** -7.726*** -7.280*** (1.973) (2.432) (2.434) (1.949) (2.419) (2.384) -0.390*** -0.301*** (0.138) (0.092) 4.314 (4.160) 0.046 -0.447 -0.620 -0.452 (1.968) (1.786) (1.788) (1.795) -0.261 -0.333 -0.290 -0.320 (0.516) (0.481) (0.776) (0.713) 58.333*** -0.596 0.445 61.149*** 0.488 2.751 (4.519) (3.897) (3.687) (5.270) (4.531) (4.412) 25.138*** -31.835*** -31.739*** 26.613*** -32.176*** -31.452*** (2.638) (2.415) (2.360) (3.272) (3.106) (2.997) 32.769*** 48.043*** 34.973*** 30.319*** 48.658*** 33.866*** (2.725) (4.823) (3.893) (3.602) (4.439) (4.574) Yes Yes Yes Yes Yes Yes 81.8 38.4 40.7 80.0 33.6 38.0 0.479 0.465 0.505 0.476 0.477 0.480 461 373 414 331 272 292 Table 3. Public-private differences in the treatment of Standardized Patients Unstable Angina Dysentery Asthma (1) (2) (3) (4) (5) (6) Dependent Variable: Percentage of Checklist Items Mean 24.690 24.687 19.033 18.762 23.796 23.911 SD (19.881) (19.229) (15.531) (15.017) (14.724) (14.528) Is a public provider -10.344* -12.717* -5.406** -4.486 -8.222*** -9.335*** (6.021) (7.545) (2.484) (2.877) (2.433) (2.896) Has MBBS 3.687 -1.771 4.481 (7.363) (4.382) (3.438) Has some qualification 1.283 3.749 -0.066 (2.912) (3.115) (2.154) Age of provider 0.077 -0.166 0.026 (0.194) (0.133) (0.133) Gender of provider (1=Male) -0.000 -3.168 2.755 (6.419) (6.788) (7.590) PCA of clinic infrastructure 2.997** -0.785 1.945* (1.416) (0.885) (1.058) Number of patients waiting 0.021 -0.178 -0.787 (0.666) (0.756) (0.632) Constant 28.010*** 22.539*** 28.807*** 38.857*** 60.972*** 23.841** (6.491) (8.652) (1.999) (5.853) (2.558) (9.488) F-stat (All SP FE = 0) 3.59 3.34 12.51 11.25 32.65 3.89 R2 0.526 0.532 0.513 0.523 0.535 0.530 Number of observations 327 294 398 358 404 361 Notes: 1) All regressions include market and SP fixed effects 2) *** Significant at 1%, ** Significant at 5%, * Significant at 10% IRT Score Table 2. Public-private differences in the treatment of Standardized Patients Dependent Variable: IRT Score (Plausible Values) Is a public provider Has MBBS Has some qualification Age of provider Gender of provider (1=Male) Constant Full Sample (1) -0.666*** (0.176) 0.289 (0.282) 0.172 (0.152) 0.003 (0.009) -0.002 (0.333) -0.722 (0.534) Yes 390 Audit 1 (2) -0.490 (0.310) 0.246 (0.265) 0.187 (0.139) 0.004 (0.008) 0.037 (0.320) -1.792 (0.560) Yes 223 Audit 2 (3) -0.768*** (0.261) Dual (4) -0.791*** (0.234) -0.044 (0.030) 0.732 (0.619) 0.359 (0.770) Yes 167 -0.051** (0.031) -1.181*** (0.373) 2.531*** (1.020) Yes 126 Market fixed effects Number of providers Notes: 1) Robust standard errors clustered at the market level are in parenthesis 2) *** Significant at 1%, ** Significant at 5%, * Significant at 10% Table 1. Treatment of Standardized Patients by Case Unstable Angina (N=327) Correct treatment Aspirin Anti-platelet agents Referred ECG ECG & Referred Antibiotic Unnecessary or harmful treatment Dysentery (N=398) Correct treatment ORS Asked to see child Antibiotic Unnecessary or harmful treatment Asthma (N=404) Correct treatment Bronchodilators Theophylline Oral Corticosteroids Antibiotic Unnecessary or harmful treatment Audit 1 Audit 2 Full Sample Public Private Public Private 0.39 0.06 0.01 0.25 0.26 0.13 0.21 0.61 0.44 0.03 0.03 0.28 0.23 0.10 0.13 0.51 0.35 0.04 0.01 0.24 0.23 0.12 0.17 0.53 0.37 0.03 0.00 0.24 0.30 0.13 0.33 0.75 0.56 0.20 0.02 0.27 0.34 0.15 0.27 0.80 0.18 0.18 0.24 0.63 0.54 0.10 0.10 0.32 0.46 0.56 0.13 0.13 0.14 0.61 0.45 0.32 0.32 0.25 0.76 0.63 0.20 0.20 0.43 0.59 0.60 0.54 0.44 0.24 0.25 0.48 0.79 0.37 0.32 0.12 0.15 0.41 0.76 0.50 0.36 0.22 0.31 0.40 0.70 0.57 0.51 0.27 0.17 0.63 0.93 0.68 0.59 0.29 0.27 0.52 0.86 Table2A. Public-Private Differences in Treatment Full Sample (1) Dependent variable: Correct Treatment Mean 0.39 SD (0.49) Is a public provider -0.029 (0.030) Has MBBS Has some qualification Number of patients waiting Age of provider Gender of provider (1=Male) Case II (Dysentery) Case III (Asthma) Constant 0.151** (0.070) 0.585*** (0.056) 0.364*** (0.082) Yes (2) Audit 1 (3) Audit 2 (4) (5) 0.39 0.39 0.34 0.45 (0.49) (0.49) (0.47) (0.50) -0.026 -0.027 0.034 -0.052 (0.042) (0.045) (0.064) (0.042) 0.178** 0.178** (0.073) (0.070) 0.075 0.075 (0.052) (0.050) -0.015** -0.027*** (0.007) (0.008) -0.002 -0.002 (0.002) (0.002) 0.041 0.071 (0.102) (0.100) -0.027 -0.347*** -0.092 -0.189*** (0.189) (0.039) (0.115) (0.062) 0.399** 0.096* 0.379*** 0.180*** (0.182) (0.053) (0.128) (0.069) 0.306* 0.487*** 0.289* 0.481*** (0.158) (0.034) (0.153) (0.059) Yes Yes Yes Yes District fixed effects Market fixed effects Provider fixed Effects R2 0.166 0.297 0.209 Number of observations 1,087 1,011 608 Notes: 1) All regressions include SP fixed effects 1) Standard errors clustered at the market level in parenthesis 0.274 605 0.126 406 Dual (6) (7) (8) 0.45 (0.50) -0.071 (0.057) 0.46 (0.50) -0.051 (0.048) 0.46 (0.50) -0.069 (0.053) 0.000 (0.011) -0.002 (0.011) -0.002 (0.477) 0.585* (0.311) 0.344 (0.222) 0.666 (0.428) -0.013 (0.014) -0.203*** -0.197** (0.077) (0.077) 0.152* 0.154* (0.087) (0.079) 0.505*** 0.528*** (0.070) (0.076) Yes Yes 0.362 406 0.115 330 Yes 0.312 330 Table2B. Public-Private Differences in Correct Treatment Coefficient on Public Reported Panel A: Unstable Angina Correct Treatment Gave/prescribed Aspirin Referred to another provider Asked/recommended EKG Both refer and EKG Panel B: Dysentery Correct Treatment ORS Asked to see child Panel C: Asthma Correct Treatment Gave/prescribed Bronchodilators Gave/prescribed Steroids Full Sample (1) (2) Audit 1 Audit 2 (3) (4) (5) Dual (6) (7) (8) -0.021 (0.072) -0.101*** (0.038) 0.022 (0.064) -0.015 (0.068) -0.011 (0.047) 0.195 (0.158) -0.033 (0.070) 0.071 (0.138) -0.024 (0.146) -0.078 (0.108) 0.155 (0.101) -0.017 (0.032) 0.043 (0.085) 0.048 (0.093) -0.022 (0.059) 0.232* (0.133) -0.034 (0.060) 0.104 (0.115) 0.008 (0.122) -0.050 (0.090) -0.195* (0.107) -0.167** (0.076) -0.006 (0.097) -0.075 (0.108) -0.009 (0.078) -0.258** (0.111) -0.202** (0.085) -0.109 (0.105) -0.079 (0.120) -0.078 (0.084) 0.053 (0.047) 0.053 (0.047) -0.007 (0.062) 0.079 (0.054) 0.079 (0.054) 0.011 (0.070) -0.054 (0.061) -0.054 (0.061) 0.197** (0.092) -0.014 (0.075) -0.014 (0.075) 0.297*** (0.096) 0.135** (0.066) 0.135** (0.066) -0.153** (0.076) 0.125 (0.086) 0.125 (0.086) -0.203* (0.108) 0.125 (0.078) 0.125 (0.078) -0.205** (0.097) 0.102 (0.068) 0.102 (0.068) -0.182** (0.089) -0.146*** (0.056) -0.082 (0.054) -0.106** (0.052) Yes -0.116 (0.072) -0.070 (0.070) -0.073 (0.054) -0.144* (0.086) -0.036 (0.086) -0.117 (0.101) Yes -0.127 (0.130) -0.008 (0.116) -0.100 (0.097) -0.140* (0.075) -0.118 (0.076) -0.088 (0.061) Yes -0.139 (0.098) -0.135 (0.099) -0.062 (0.073) -0.139 (0.088) -0.135 (0.087) -0.064 (0.065) Yes -0.147* (0.077) -0.111 (0.081) -0.095 (0.061) District fixed effects Market fixed effects Yes Yes Provider Fixed Effects Yes Other controls SP fixed effects, age, gender, qualification, number of patients waiting Notes: 1) Standard errors clustered at the market level in parenthesis Yes What exactly is happening with treatment? In audit 1, the two groups behave similarly, but there are some differences across cases MI identical in both Dysentery public sector providers 20-30% more likely to ask to see child, no difference in ORS Asthma public sector providers 12-14% less likely to give correct treatment (not statistically significant) Across all cases, public 13% less likely to give antibiotics Dual sample, with and without provider fixed effects MI: Equal likelihood of EKG/Referral but private more likely to give Aspirin Dysentery: Public 10-12% (not significant) more likely to give ORS, private 18-20% more likely to ask to see child Asthma: Public 13-15% less likely to get it correct Table3A. Public-Private Differences in Incorrect Treatment Full Sample Audit 1 Audit 2 (1) (2) (3) (4) (5) (6) Dependent variable: Unnecessary or harmful treatment Mean 0.65 0.64 0.57 0.57 0.76 0.76 SD (0.48) (0.48) (0.50) (0.50) (0.43) (0.43) Is a public provider 0.052 0.010 0.011 -0.017 0.020 0.025 (0.035) (0.040) (0.068) (0.066) (0.037) (0.051) Has MBBS 0.229*** 0.236*** (0.074) (0.073) Has some qualification -0.010 -0.011 (0.055) (0.053) Number of patients waiting 0.005 0.005 0.004 (0.006) (0.009) (0.009) Age of provider -0.001 -0.000 -0.011 (0.002) (0.002) (0.011) Gender of provider (1=Male) 0.076 0.086 0.218 (0.095) (0.096) (0.468) Case II (Dysentery) -0.083** 0.055 -0.051 0.004 -0.152*** -0.577 (0.041) (0.261) (0.055) (0.107) (0.059) (0.443) Case III (Asthma) 0.171*** 0.258 0.181*** 0.198* 0.128*** 0.305 (0.027) (0.255) (0.035) (0.113) (0.044) (0.279) Constant 0.597*** 0.322** 0.523*** 0.309** 0.756*** 0.994** (0.028) (0.149) (0.034) (0.147) (0.046) (0.416) District fixed effects Yes Yes Yes Market fixed effects Yes Yes Yes Provider fixed Effects R2 0.095 0.252 0.059 0.170 0.091 0.358 Number of observations 1,129 1,052 668 644 461 408 Notes: 1) All regressions include SP fixed effects 1) Standard errors clustered at the market level in parenthesis Dual (7) (8) 0.75 (0.43) 0.027 (0.042) 0.75 (0.43) 0.042 (0.046) 0.018 (0.014) -0.131* -0.124* (0.077) (0.067) 0.143** 0.153** (0.058) (0.061) 0.732*** 0.695*** (0.060) (0.064) Yes 0.315 331 Yes 0.300 331 Table3B. Public-Private Differences in Incorrect Treatment Coefficient on Public Reported Panel A: Unstable Angina Unnecessary or harmful treatment Gave/prescribed antibiotics Gave/prescribed steroids Panel B: Dysentery Unnecessary or harmful treatment Gave/prescribed antibiotics Panel C: Asthma Unnecessary or harmful treatment Gave/prescribed antibiotics Full Sample (1) (2) Audit 1 Audit 2 (3) (4) (5) Dual (6) (7) (8) -0.083 (0.069) 0.012 (0.054) -0.021 (0.022) -0.168 (0.148) -0.071 (0.104) -0.028 (0.044) -0.134 (0.119) -0.040 (0.055) -0.029 (0.039) -0.176 (0.128) -0.075 (0.089) -0.031 (0.038) -0.070 (0.079) 0.032 (0.094) -0.014 (0.037) -0.047 (0.094) 0.065 (0.108) -0.015 (0.042) 0.025 (0.061) 0.029 (0.062) 0.076 (0.070) -0.005 (0.073) 0.028 (0.110) -0.181* (0.099) 0.193* (0.110) -0.238** (0.119) 0.005 (0.075) 0.164** (0.073) 0.000 (0.086) 0.149 (0.097) 0.027 (0.090) 0.140 (0.087) 0.016 (0.078) 0.123 (0.075) 0.065 (0.045) 0.068 (0.062) Yes 0.026 (0.053) 0.008 (0.074) 0.021 (0.098) -0.011 (0.110) Yes -0.079 (0.092) -0.132 (0.124) 0.078* (0.047) 0.115 (0.080) Yes 0.081 (0.068) 0.110 (0.105) 0.080 (0.061) 0.109 (0.094) Yes 0.095* (0.056) 0.117 (0.081) District fixed effects Market fixed effects Yes Yes Provider Fixed Effects Yes Yes Other controls SP fixed effects, age, gender, qualifications, number of patients waiting Notes: 1) Standard errors clustered at the market level in parenthesis Diagnosis Problem: 67% interactions there is no diagnosis Noted in pilot Final survey: randomized SSPs into 2 groups 1 group turns around as they are leaving and ask the provider “Doctor, what is wrong with me?” Increases rate of diagnosis provision by 20-25 p.p. in all groups First look at likelihood of providing diagnosis Second, use randomization as instrument in selection model with binary dependent variables to deduce correct diagnosis rates Summary of Diagnosis by Public and by SP Empowered, by Sample Full Sample Yes No Panel A: By Public (Yes = Public) Gave Diagnosis 0.260 0.353 373 756 Audit 1 Yes No Audit 2 Yes No Yes No 0.190 121 0.356 547 0.294 252 0.344 209 0.270 163 0.381 168 0.652 23 0.513 195 0.507 73 0.611 72 0.270 163 0.381 168 Panel B: By SP Empowerment (Yes = Empowered) Gave Diagnosis 0.461 0.200 0.470 529 600 302 0.208 366 0.449 227 0.188 234 0.446 168 0.202 163 0.526 76 0.515 101 0.659 44 0.413 92 0.579 57 Diagnosis Correct Diagnosis Correct 0.542 96 0.523 243 0.539 267 0.575 120 0.528 142 Notes: 2) For each sample, group means and number of observations Dual Public-private differences in diagnosis Full Sample Audit 1 Audit 2 Dual (1) (2) (3) (4) (5) (6) (7) (8) (9) Dependent Variable: Diagnosis was given Mean 0.32 0.33 0.33 0.33 0.32 0.32 0.33 0.33 0.33 SD (0.47) (0.47) (0.47) (0.47) (0.47) (0.47) (0.47) (0.47) (0.47) Is a public provider -0.093*** -0.120*** -0.142*** -0.150*** -0.064 -0.095* -0.112** -0.107** -0.095* (0.029) (0.036) (0.031) (0.044) (0.047) (0.055) (0.053) (0.051) (0.053) SP was empowered 0.243*** 0.251*** 0.226*** 0.230*** 0.228*** (0.033) (0.039) (0.055) (0.057) (0.061) Case II (Dysentery) -0.283*** -0.273*** -0.305*** -0.319*** -0.340*** (0.037) (0.047) (0.057) (0.066) (0.073) Case III (Asthma) -0.077** -0.107** -0.054 -0.045 -0.072 (0.038) (0.043) (0.069) (0.074) (0.084) Number of patients waiting 0.004 0.004 0.003 -0.004 0.001 (0.006) (0.007) (0.012) (0.014) (0.019) Has MBBS -0.081 -0.075 (0.087) (0.083) Has some qualification -0.002 -0.002 (0.055) (0.053) Age of provider -0.001 -0.001 -0.002 (0.002) (0.002) (0.003) Gender of provider (1=Male) -0.073 -0.052 -0.042 (0.093) (0.092) (0.076) Constant 0.353*** 0.536*** 0.352*** 0.458*** 0.351*** 0.390*** 0.381*** 0.532*** 0.419*** (0.015) (0.106) (0.017) (0.099) (0.030) (0.063) (0.034) (0.148) (0.082) District fixed effects Yes Yes Yes Yes Market fixed effects Yes Yes Yes Yes Provider fixed effects Yes R2 0.018 0.268 0.025 0.211 0.014 0.381 0.027 0.188 0.332 Number of observations 1,129 1,052 668 644 461 461 331 302 331 Notes: Public-private differences in diagnosis, by case Unstable Angina Dysentery Asthma (1) (2) (3) (4) (5) (6) Dependent Variable: Diagnosis was given Mean 0.46 0.46 0.17 0.17 0.37 0.38 SD (0.50) (0.50) (0.37) (0.38) (0.48) (0.48) Is a public provider -0.064 -0.100 -0.086** -0.097* -0.124** -0.126 (0.058) (0.120) (0.037) (0.053) (0.051) (0.086) SP was empowered 0.104 0.228*** 0.311*** (0.096) (0.049) (0.072) Number of patients waiting -0.002 0.005 0.016 (0.010) (0.014) (0.029) Has MBBS -0.092 -0.007 -0.123 (0.177) (0.101) (0.165) Has some qualification 0.053 0.032 -0.078 (0.127) (0.081) (0.080) Age of provider -0.001 -0.003 -0.001 (0.005) (0.003) (0.004) Gender of provider (1=Male) 0.108 -0.071 -0.227 (0.202) (0.120) (0.227) Constant 0.476*** 0.415 0.194*** 0.280* 0.410*** 0.570** (0.034) (0.314) (0.024) (0.156) (0.027) (0.234) District fixed effects Yes Yes Yes Market fixed effects Yes Yes Yes R2 0.016 0.503 0.046 0.471 0.026 0.457 Number of observations 327 304 398 372 404 376 Notes: 1) Standard errors clustered at the market level in parenthesis Difference in Diagnosis by Public, by Sample Full (1) Y-variable: Diagnosis Given was Correct (marginal effects (dy/dx) reported) Is a public provider 0.014 (0.060) Case II (Dysentery) 0.058 (0.071) Case III (Asthma) 0.137** (0.056) Number of patients waiting Has MBBS Has some qualification Age of provider Gender of provider (1=Male) Audit 1 Audit 2 Dual (2) (3) (4) (5) (6) (7) (8) 0.009 (0.067) 0.080 (0.073) 0.137** (0.059) -0.000 (0.012) 0.138* (0.076) 0.049 (0.079) -0.000 (0.003) -0.064 (0.100) Yes 1,050 0.154 (0.111) 0.139 (0.092) 0.050 (0.075) 0.184 (0.116) 0.138 (0.092) 0.053 (0.074) -0.001 (0.014) 0.303** (0.145) 0.043 (0.080) 0.000 (0.003) -0.103 (0.158) Yes 644 -0.075 (0.076) -0.064 (0.108) 0.210** (0.097) -0.072 (0.073) -0.012 (0.101) 0.210* (0.112) -0.004 (0.017) -0.151* (0.090) -0.056 (0.116) 0.170 (0.112) -0.119 (0.089) -0.038 (0.109) 0.136 (0.113) -0.022 (0.021) District fixed effects Yes Number of observations 1,127 Notes: 1) Standard errors calculated using the Delta method 2) All regressions include district fixed effects Yes 668 Yes 459 -0.004 (0.004) 0.019 (0.109) Yes 406 Yes 330 -0.002 (0.004) -0.002 (0.120) Yes 301 Prices in the Private Sector Each vertical line represents a box-plot of prices charged by a provider to real patients. Providers are sorted on the x-axis by quality (measured by number of questions asked and examinations conducted) Huge variation within providers True for all MBBS, other qualified, and unqualified providers Prices, Checklists and Treatment Greater compliance with the checklist is always rewarded in higher prices Correct treatment leads to higher prices , but vanishes once we control for checklist adherence Stronger premium among MBBS providers Works across all cases, weaker for asthma Prices and Checklist Adherence .02 0 .01 40 20 0 Fees Charged 60 Density of Adherence 80 .03 Prices and Checklist Adherence 0 20 40 60 Checklist Adherence (%) 80 Fees and Checklist Adherence: All Private Density of Adherence Fees and Checklist Adherence: Only Dual Table 4. Prices Charged and Checklist Items (Full, Private Only) (1) (2) Dependent Variable: Prices in Rs. Mean 28.51 28.46 SD 26.44 26.45 Fixed effects District District Percentage of checklist items 0.434*** 0.419*** (0.072) (0.056) Dispensed medicine 20.093*** (1.691) Has MBBS 22.127*** (4.061) Has some qualification 7.614*** (2.334) Referred in Unstable Angina -8.257*** (2.060) Asked for child in Dysentery -2.446 (2.085) Time spent with SP (minutes) R2 Number of observations 0.122 755 0.292 742 (3) (4) 28.59 26.59 District 0.455*** (0.069) 20.946*** (1.707) 22.144*** (3.943) 7.488*** (2.465) -8.179*** (2.003) -0.296 (2.082) 28.44 26.22 District 0.340*** (0.082) 20.364*** (1.740) 22.368*** (3.814) 7.362*** (2.567) -8.716*** (1.958) -0.534 (1.925) 1.108*** (0.284) 0.339 705 0.309 724 (5) (6) 28.59 28.44 26.59 26.22 Market Market 0.450*** 0.346*** (0.075) (0.090) 21.154*** 20.703*** (1.925) (2.005) 25.688** 25.888** (10.453) (10.272) 3.117 3.428 (2.849) (2.865) -9.972*** -10.000*** (2.732) (2.797) 1.132 1.506 (2.044) (2.200) 0.984*** (0.338) 0.440 0.462 724 705 (7) 28.51 26.44 Provider 0.297*** (0.078) 24.301*** (2.985) 1.322*** (0.383) 0.626 755 Prices Charged and Correct Treatment (Full Sample, Private Only) (1) (2) Dependent Variable: Prices in Rs. Mean 28.07 28.07 SD (26.35) (26.40) Fixed Effects District District Correct treatment 6.903*** 8.119*** (1.913) (1.988) Dispensed medicine 20.299*** (1.759) Has MBBS 23.310*** (4.427) Has some qualification 7.818*** (2.321) Referred in Unstable Angina -8.609*** (2.270) Asked for child in Dysentery -3.048 (2.770) Percentage of checklist items Time spent with SP (minutes) R2 Number of observations 0.062 721 0.248 709 (3) (4) (5) (6) (7) (8) 28.21 28.07 28.21 28.21 28.21 28.07 (26.55) (26.35) (26.55) (26.55) (26.55) (26.35) Market Provider Market Market Market Provider 7.210*** 8.135*** 2.801 4.538** 1.930 2.890 (2.072) (2.440) (2.458) (2.246) (2.510) (2.689) 21.357*** 24.189*** 21.826*** 21.063*** 21.508*** 24.866*** (1.999) (3.062) (1.940) (2.048) (2.020) (3.159) 25.059** 23.703** 25.311** 24.196** (10.403) (10.257) (11.500) (10.971) 1.983 1.876 2.748 2.443 (2.538) (2.639) (2.513) (2.584) -9.015*** -9.525*** -7.774** -8.526*** (3.054) (2.877) (3.120) (3.035) -1.701 -0.176 -1.090 -0.097 (2.584) (2.071) (2.124) (2.152) 0.385*** 0.296*** 0.243*** (0.081) (0.092) (0.081) 1.633*** 1.159*** 1.314*** (0.333) (0.357) (0.432) 0.409 0.594 0.447 0.442 0.461 0.631 691 721 691 691 691 721 continued on next slide continued from previous slide (1) Average steroid (0-1) Average antibiotic (0-1) Number of patients waiting Age of provider Gender of provider (1=Male) Constant (2) (3) (4) (5) -2.726 3.850 (7.595) (7.695) 9.486*** 9.167*** (3.465) (3.476) 0.338 -0.029 0.314 (0.419) (0.469) (0.444) -0.130 -0.108 (0.096) (0.097) -3.220 -3.628 (4.955) (4.988) 25.425*** 9.711*** 14.777** 14.753*** 5.927 (1.852) (1.702) (6.965) (1.788) (7.985) Yes Yes Yes Yes Yes 0.062 0.248 0.409 0.594 0.447 721 709 691 721 691 District fixed effects Market fixed effects Provider Fixed Effects R2 Number of observations Note: 1) Robust standard errors clustered at the village level in parenthesis 2) *** Significant at 1%, ** Significant at 5%, * Significant at 10% (6) -0.345 (7.576) 9.012*** (3.287) 0.450 (0.433) -0.115 (0.099) -5.548 (5.048) 10.765 (7.605) (7) 4.012 (7.617) 8.905*** (3.402) 0.399 (0.439) -0.103 (0.099) -5.185 (5.162) 5.135 (8.324) Yes Yes 0.442 691 0.461 691 (8) -0.034 (0.416) 5.870** (2.605) Yes 0.631 721 Does the public sector reward quality? Public sector pay in India follows a matrix Composed of: rank, tenure, qualifications Zero effect of checklist adherence, treatment, likelihood of discussing diagnosis on wages Variation in Wages in the Public Sector By Job Position (1) (2) Dependent Variable: Log of Monthly Wages Percentage of checklist items -0.001 (0.002) Time spent with SP (minutes) -0.033** (0.013) Correct treatment Provider discussed diagnosis Is a doctor Is a nurse Age of provider Gender of provider (1=Male) Born in same district Is a dual provider Constant (3) (4) -0.056 (0.044) 0.026 (0.080) 0.908*** 0.903*** 0.935*** 0.899*** (0.168) (0.165) (0.177) (0.171) -0.104 -0.097 -0.098 -0.114 (0.170) (0.168) (0.172) (0.173) 0.017*** 0.017*** 0.017*** 0.017*** (0.006) (0.006) (0.006) (0.006) 0.032 0.034 0.039 0.032 (0.157) (0.156) (0.154) (0.156) -0.015 -0.009 -0.018 -0.016 (0.115) (0.113) (0.115) (0.114) 0.290*** 0.266** 0.280** 0.294*** (0.110) (0.108) (0.109) (0.112) 8.351*** 8.407*** 8.346*** 8.331*** (0.253) (0.251) (0.262) (0.265) Yes Yes Yes Yes 0.453 0.459 0.442 0.453 308 308 301 308 District fixed effects R2 Number of observations Notes: 1) Standard errors clustered at the market level in parenthesis 2) *** Significant at 1%, ** Significant at 5%, * Significant at 10% (5) 0.001 (0.002) -0.035** (0.016) -0.040 (0.047) 0.053 (0.088) 0.925*** (0.172) -0.085 (0.167) 0.017*** (0.006) 0.042 (0.152) -0.010 (0.109) 0.256** (0.107) 8.395*** (0.266) 0.449 301 Variation in Wages in the Public Sector By Qualification (1) (2) Dependent Variable: Log of Monthly Wages Percentage of checklist items -0.002 (0.002) Time spent with SP (minutes) -0.016 (0.012) Correct treatment (3) Has some qualification Age of provider Gender of provider (1=Male) Born in same district Is a dual provider Constant 1.283*** (0.169) 0.871*** (0.297) 0.019*** (0.006) 0.130 (0.106) 0.015 (0.080) 0.145* (0.085) 8.022*** (0.308) Yes 0.628 308 1.271*** (0.168) 0.857*** (0.290) 0.019*** (0.006) 0.130 (0.105) 0.017 (0.080) 0.137 (0.086) 8.040*** (0.315) Yes 0.629 308 (5) 0.020 (0.065) 1.275*** (0.167) 0.865*** (0.293) 0.019*** (0.006) 0.131 (0.104) 0.014 (0.081) 0.149* (0.087) 7.995*** (0.329) Yes 0.627 308 -0.000 (0.002) -0.017 (0.014) -0.038 (0.044) 0.052 (0.073) 1.303*** (0.179) 0.878*** (0.299) 0.019*** (0.006) 0.121 (0.105) 0.022 (0.081) 0.140 (0.086) 8.020*** (0.344) Yes 0.622 301 -0.047 (0.042) Provider discussed diagnosis Has MBBS (4) 1.308*** (0.174) 0.892*** (0.299) 0.019*** (0.006) 0.122 (0.107) 0.015 (0.081) 0.152* (0.086) 7.993*** (0.337) Yes 0.620 301 District fixed effects R2 Number of observations Notes: 1) Standard errors clustered at the market level in parenthesis 2) *** Significant at 1%, ** Significant at 5%, * Significant at 10% Quick back of the envelope We can provide a back of the envelope measure of costs and quality in the public and private sectors This is rough—public and private sector providers provide other services beyond primary care Cost and Checklist Completion in Public Sector Per Patient Cost in the Public Sector Facility Average doctors per facility 2.03 Monthly salary cost of doctors per month per facility Rs.64,199 Average number patients per facility per month 984 Average cost per patient Rs.101 Public Checklist Usage 25th percentile 8.33% 50th percentile 15.79% 75th percentile 99th percentile 27.27% 63.18% Cost and Checklist Completion in Private Sector Private Facility (parameters from regression) Base price (constant) Rs.2.05 Cost per percentage checklist item Rs.0.46 MBBS premium Rs.22.14 Some qualification premium Rs.7.49 Percentage Checklist Items 0% 25% 50% 75% 100% MBBS Doctor Rs.24 Rs.36 Rs.47 Rs.59 Rs.70 Some No Qualification Qualification Rs.10 Rs.2 Rs.21 Rs.14 Rs.33 Rs.25 Rs.44 Rs.37 Rs.56 Rs.48 Some further interpretation results Audit patients present the same symptoms and same script to multiple doctors in different conditions. This may be off-equilibrium behavior. 3 sets of issues “Serious” cases never go to the public sector. Therefore, if they do, it is an indication to the doctor (who is on the equilibrium path) that the patient is not serious If the same case goes, the patient presents in a different way in the public to the private sector, accounting for lower incentives to put effort The public-private difference for the same doctors may reflect incentive effects due to the presence of the private sector clinic Difficult problems: in past led to differing results between audit studies and observational data Famously, Ayres and Spiegelman (1995) versus Goldberg (1996) More recently, discrimination against African American (names): Bertrand-Mullianathan versus Fryer and Levitt What we do in addition to the audit First, observe equilibrium path behavior with real patients and check to see if the results are the same Second, try to assess patient sorting Third, try to rule out deliberately lower effort due to dual practice Some further interpretation results Is it the case that they treat “real” patients this way? Yes, we sat in their clinics for 1 day each and find identical results on things that we can measure in both (time spent, questions asked, examinations done) (link to table) Is it the case that the “regular” patient body is very different for public/private We did exit surveys with patients from all practices. Patients were not very different in illness and severity, but in private had more access to transport and had more mobile phones (72 vs. 64%) When we include (means) of the regular patient populations in the audit regressions, nothing changes It seems like people use the public clinics precisely like they use unqualified providers (link to table) Some further interpretation results Is it the case that patients “expect” something very different from public and private? If the patients know what they have, then it is likely that there will be complete separation by quality and price Cases deliberately chosen so that same symptom can reflect a minor or major condition Some further interpretation results Is it the case that public providers were “directing” patients to their private clinics? Or, would we expect very different care among public sector providers if they did not have a private clinic? None of our SPs were directed to the private clinic of the public provider. Referrals lower among dual practice People already know where the private clinic is (and sometimes this is not in the same place) Fully segmented markets Some effect of location on estimated impact in checklist and time-spent, but not on treatment and diagnosis Further, the guys with clinics in the same location are also worse in their private practice, suggesting that these guys are just selected worse We cannot tell what would happen where there are is no dual-practice We note that it is not allowed, but 80 percent of providers have them The providers who have dual practice versus the 20 percent who do not, behave identically in their public practice in treatment and have lower referral rates, but also have lower checklist completion and diagnosis rates Some further interpretation results Is it the case more educated patients would get different results from the public sector? What about urban areas? On all process measures, public providers in Delhi are worse But on treatment, they are better; could reflect higher competence since we did not observe the same doctor in both practices Time spent Public MBBS Some qualification Dysentery Patient load during visit -5.42*** (0.47) 0.80 (0.71) 2.24*** (0.60) 0.12 (0.093) 0.00015 (0.018) Percentage of IRT score checklist items -16.1*** (2.23) 6.14*** (1.17) 4.48 (2.53) 6.75*** (0.22) -0.0069 (0.054) -1.25*** (0.068) 0.34*** (0.042) 0.32*** (0.054) 0.31*** (0.017) -0.0054 (0.0047) Uttered diagnosis Correct treatment Incorrect treatment Antibiotics -0.16 (0.086) -0.14 (0.11) -0.014 (0.11) -0.34*** (0.0078) 0.00058 (0.0015) 0.29** (0.090) 0.037 (0.11) -0.019 (0.15) 0.13*** (0.016) -0.0063** (0.0025) -0.21 (0.13) 0.17 (0.14) -0.13*** (0.023) -0.28*** (0.014) 0.0020 (0.0039) -0.21 (0.15) 0.17 (0.16) -0.068* (0.034) 0.025** (0.0084) 0.0011 (0.0023) Conclusion Significant over and under treatment in both sectors Pure public versus private comparison for the same provider suggests that public sector has Lower compliance with checklist Lower correct treatment and diagnosis rates Similar rates of over-treatment Public versus private clinics is more complex Lower compliance with checklist in public sector BUT, no difference in treatment and higher correct diagnosis rates Several potential explanations (accountability, local, rules of thumb) Prices reward quality as measured through adherence to checklist and treatment in private, but not in public clinics Implications What to do with the public sector Location subsidies? Massive investments? Better administrative accountability? Reorganizing geographical locations? What to do with the private sector Better knowledge? What about equity—how to introduce subsidies