Declaration of A Commitment to Lean Research Preamble On August 1, 2014, fifty practitioners, institutional researchers and donors gathered to discuss research in the “development context,” a context1 in which researchers2 may have advantage over human subjects. The group had seen or been involved with too many studies where results were not actionable by policy makers or practitioners (i.e. were not sufficiently relevant), where researchers were unconcerned with the research experience of subjects (i.e. were not sufficiently respectful), and where samples and protocols were either too large or too small to efficiently answer the research question (i.e. were not right-sized). However, we collectively recognized that we could do better research and concluded: Research Is a Development Activity. Primary data collection constitutes an intervention in the lives of subjects. The act of primary research itself generates development outcomes, which can be beneficial or harmful, or both. Power Balances Must Be Addressed. The research process must adequately address the differences in power between the researcher and those being researched. Failure to address differences negatively impacts subjects, produces poor data, and thus undermines good decisions. Research is Often Irrelevant. Research can be meaningless to subjects, other stakeholders and decisionmakers. It often is too narrow in scope, too inaccessible to stakeholders, or too ignorant of the priorities of human subjects to be relevant. Biases in data collection and analysis skew results such that real information is misrepresented, which harms the prospect that a study could benefit stakeholders. Research is Often Not Respectful. Research is often not respectful or enjoyable for the human subjects. In some cases, the research also does not respect the subjects’ time or ideas or provide reasonable availability of the treatment if it proves successful. Research is Often Inefficient. Research protocols and activities are often too costly relative to the benefits they produce. In many cases the research does not need to be done at all, either because similar, relevant research had already been done, or because the planned research is likely of little consequence to subjects or other stakeholders. Research Can Be Better. We also concluded that research could be done better, that it was possible to make research beneficial to subjects, actionable by many stakeholders, and efficient in terms of costs. The term Lean Research was coined. It drew its inspiration from pioneers in Lean Manufacturing and Lean Management where lean has come to mean engendering maximum benefit with minimum waste. A Call to Be Leaner. Lean Research is social science research that is relevant, respectful and right-sized. It is a framework and set of principles and standards that can be used to guide field research that directly involves people, or “human subjects,” regardless of whether the research methods are quantitative, qualitative, or mixed. Its intent is to maximize outcomes for research subjects, research consumers (policymakers, suppliers of services, funders, and subjects) and researchers while minimizing 1 We likely define the context more broadly than ethical boards in the social sciences. A researcher is someone who gathers information directly from human subjects. The range of possible researchers may include university researchers, staff of governments, NGOs, and private institutes and companies. 2 1 waste. Lean Research, we agreed, must conform both to standards established in various disciplines of study or fields of practice as well as to three additional groupings of principles (the three Rs). To be Lean, research must also be: 1. Relevant. Research subjects can clearly articulate the value of the research study. Findings are understandable to a wide range of stakeholders, including subjects, practitioners, policy-makers and other decision-makers. They are accessible, actionable, and if conclusive, able to drive change and if inconclusive, deepen understanding and further inquiry. Research must be unbiased and where possible, generalizable. 2. Respectful, even delightful. It places the experience of the human subject at center of the research activity, in hopes of generating high quality data and positive development outcomes. Respectful research enables subjects to walk away from research after an informed consent process. It also values the subject’s time and ideas and offers reasonable availability of a continued treatment, if a treatment is offered. 3. Right-sized. The sample size and methods are well-suited to the research objectives. The research does not include any unnecessary questions or activities. Thorough work must be done up-front to ensure the study is needed at all and that the information could not have been gathered through secondary research or key informants. Agreement I ____________________________________________ (name) hereby agree that: 1. Research on human subjects in “development contexts” is an intervention and can cause change – beneficial or harmful – to research subjects and other stakeholders. I agree that the research process must generate positive developmental outcomes to the extent they are foreseeable. 2. Research in development contexts must not only conform to the norms and standards established by various human science disciplines and fields of practice, but it must also to additional standards that support relevant, respectful, and right-sized research. (An evolving set of principles can be found at the end of this agreement.) 3. By committing to Lean Research Principles (see following page) and by committing to their improvement, I will help generate better quality data and more efficient and effective development outcomes. I agree to support these principles, help peers support them, and contribute to the continuous improvement of standards and practices associated with them. 4. I understand that conforming to Lean Research principles may shift costs, risks and burdens from the research subject to the Principal Investigator. I am willing to take on this challenge as a researcher, donor or other facilitator of research. Name, Organization (if appropriate), Date (DO SEE FOLLOWING PAGES FOR LEAN RESEARCH PRINCIPLES) 2 Lean Research Principles Lean research consists of three principles based on a foundational assumption. From time to time, researchers may be making trade-offs between applying the standards of their disciplines (our foundational assumption) and adhering to Lean Research principles. Foundational Assumption All researchers agreeing to conform to the Lean Research principles already rigorously apply the standards of their disciplines or field of practices. We assume that those agreeing to apply the Lean Research principles, to the best of their ability, uphold the standards set by their fields of study and practice For example, if the researcher is an ethnographer or an economist, he or she follows the standards long established in those disciplines. If he or she is a humanitarian worker, he or she follows research protocols accepted in humanitarian assistance. At a minimum, If the research is an impact evaluation or trial, it is registered - for RCTs register with AEA’s social science registry and for RCTs and other methods with 3ie’s RIDIE. The research follows the highest standards of the discipline or field of practice standards for study and instrument design, sampling, data cleaning, and analysis. The research is reproducible: Data and protocols are available for reproduction by independent researchers. Research subject data are fully protected. The research is internally valid (the estimates are unbiased) and the researcher can convincingly demonstrate this. The research findings are reported transparently and responsibly. Ideally, The research is externally valid (applies in some context beyond the study context) and the researcher can demonstrate in which contexts it can be reliably generalized. The research has been reproduced (all data gathering and protocols) or verified by independent parties. The Three Lean Research Principles The Lean Research principles go above and beyond the foundational assumption. Researchers conforming to the principles are already well-versed in the rigorously application of standards in their disciplines or fields of practice. In addition to applying those standards they agree to conform to the Lean Research standards. Since research is a development intervention, the researcher from time to 3 time may be faced with choosing the standards of their disciplines with Lean Research principles. Such a choice is inevitable and the researcher agrees to make those choices as ethically as is possible. Many but not all of the principles outlined follow the Declaration of Helsinki. Principle One - Lean research Is relevant. Research subjects can clearly articulate the value of the research study. The findings are understandable to by a wide range of stakeholders, including the subjects researched. The findings are accessible, actionable, and able to drive changes in policy or practice. At a minimum, Research subjects can clearly articulate the value of the research study. The researcher is committed to understanding the highest research priorities of the communities and how their research addresses or does not address these priorities. Stakeholders are identified in advance of the research project and give inputs into how they would like to receive the findings. The results of the research are delivered to stakeholders in a timely manner (within 2 months of completing a report), so that decision makers at various levels can use the research findings effectively. De-identified data is made available as widely as possible. The research findings are accessible by a wide range of stakeholders including research subjects. The research findings are understandable by a wide range of stakeholders including research subjects. Stakeholders can make decisions based upon the findings. (Depending on the findings, a decision may be to take no action.) Regardless of the research findings, there is a process to verify that value of the research after the completion of the research. Ideally, The research represents a top priority research question of the population studied. Decision-makers agree in advance of the study to use research findings of the study to change or not change (depending on findings) a policy, product or service. The policy, product, or service change (resulting from the research) would significantly address a priority problem identified by the population studied. 4 Principle Two - Lean research is respectful, even delightful. Lean Research places the dignity and delight of the human subject at center of the experience. It enables subjects to walk away from research after an informed consent process. It also values the subject’s time and ideas and offers reasonable availability of a treatment, if a treatment is offered as part of an experiment. At a minimum, The human subject has been given all necessary information to decide whether to participate in the research. The human subject feels truly free to reject participation in the study or to drop out of a study once it has begun and suffer no negative repercussions. No interviews on human subjects exceed a length of time that is agreed to be respectful among various stakeholders. Human subjects are fairly compensated for their time and expenses. The research process provides an environment where the subject can enjoy the experience. If a treatment is introduced and if the findings determine the treatment produces benefits, both the treated and control populations have sustained access to the treatment. Enumerators are well-trained, well-treated and paid fairly in a timely way. Ideally, Research subjects co-design the research and reporting methods. Research subjects co-design and test a method for informed consent that demonstrates a subject’s ability to refuse the study without consequence. An independent research team is able to obtain feedback from subjects on their research experience (and concludes the experience was generally positive). Subjects have the opportunity to refute findings and the researchers must publish any refutations along his or her original published findings. Principle Three - Lean Research is right-sized. The sample and method are well-suited to the research objectives. The research does not include any unnecessary questions or activities. This means that thorough work should be done up-front to ensure the study is needed at all and that the information could not have been gathered through secondary research or through key informants. At a minimum, The scope of the research adequately addresses the nature of the problem (meaning the study is 5 not so narrow as to be irrelevant). The researcher can demonstrate that original research on the identified population is absolutely necessary. All other avenues of secondary or key informant information gathering have been exhausted. The research has eliminated all activities (and questions) that are not critical to the research objective. The sample is sufficiently large as to generate data proportionate to the research objective, but not larger. Ideally, Researchers can prove the study is a priority using cost-benefit or other methods to assess tradeoffs between the proposed research and other possible research. The researcher can demonstrate that the cost of the project (to subjects, donors, and the time of other stakeholders) and its benefits are optimal. 6