Lean Research Declaration Sept. 10

advertisement
Declaration of A Commitment to Lean Research
Preamble
On August 1, 2014, fifty practitioners, institutional researchers and donors gathered to discuss research
in the “development context,” a context1 in which researchers2 may have advantage over human
subjects. The group had seen or been involved with too many studies where results were not actionable
by policy makers or practitioners (i.e. were not sufficiently relevant), where researchers were
unconcerned with the research experience of subjects (i.e. were not sufficiently respectful), and where
samples and protocols were either too large or too small to efficiently answer the research question (i.e.
were not right-sized). However, we collectively recognized that we could do better research and
concluded:
Research Is a Development Activity. Primary data collection constitutes an intervention in the lives of
subjects. The act of primary research itself generates development outcomes, which can be beneficial or
harmful, or both.
Power Balances Must Be Addressed. The research process must adequately address the differences in
power between the researcher and those being researched. Failure to address differences negatively
impacts subjects, produces poor data, and thus undermines good decisions.
Research is Often Irrelevant. Research can be meaningless to subjects, other stakeholders and decisionmakers. It often is too narrow in scope, too inaccessible to stakeholders, or too ignorant of the priorities
of human subjects to be relevant. Biases in data collection and analysis skew results such that real
information is misrepresented, which harms the prospect that a study could benefit stakeholders.
Research is Often Not Respectful. Research is often not respectful or enjoyable for the human subjects.
In some cases, the research also does not respect the subjects’ time or ideas or provide reasonable
availability of the treatment if it proves successful.
Research is Often Inefficient. Research protocols and activities are often too costly relative to the
benefits they produce. In many cases the research does not need to be done at all, either because
similar, relevant research had already been done, or because the planned research is likely of little
consequence to subjects or other stakeholders.
Research Can Be Better. We also concluded that research could be done better, that it was possible to
make research beneficial to subjects, actionable by many stakeholders, and efficient in terms of costs.
The term Lean Research was coined. It drew its inspiration from pioneers in Lean Manufacturing and
Lean Management where lean has come to mean engendering maximum benefit with minimum waste.
A Call to Be Leaner. Lean Research is social science research that is relevant, respectful and right-sized.
It is a framework and set of principles and standards that can be used to guide field research that
directly involves people, or “human subjects,” regardless of whether the research methods are
quantitative, qualitative, or mixed. Its intent is to maximize outcomes for research subjects, research
consumers (policymakers, suppliers of services, funders, and subjects) and researchers while minimizing
1
We likely define the context more broadly than ethical boards in the social sciences.
A researcher is someone who gathers information directly from human subjects. The range of possible
researchers may include university researchers, staff of governments, NGOs, and private institutes and companies.
2
1
waste. Lean Research, we agreed, must conform both to standards established in various disciplines of
study or fields of practice as well as to three additional groupings of principles (the three Rs). To be
Lean, research must also be:
1. Relevant. Research subjects can clearly articulate the value of the research study. Findings are
understandable to a wide range of stakeholders, including subjects, practitioners, policy-makers
and other decision-makers. They are accessible, actionable, and if conclusive, able to drive
change and if inconclusive, deepen understanding and further inquiry. Research must be
unbiased and where possible, generalizable.
2. Respectful, even delightful. It places the experience of the human subject at center of the
research activity, in hopes of generating high quality data and positive development outcomes.
Respectful research enables subjects to walk away from research after an informed consent
process. It also values the subject’s time and ideas and offers reasonable availability of a
continued treatment, if a treatment is offered.
3. Right-sized. The sample size and methods are well-suited to the research objectives. The
research does not include any unnecessary questions or activities. Thorough work must be done
up-front to ensure the study is needed at all and that the information could not have been
gathered through secondary research or key informants.
Agreement
I ____________________________________________ (name) hereby agree that:
1. Research on human subjects in “development contexts” is an intervention and can cause change
– beneficial or harmful – to research subjects and other stakeholders. I agree that the research
process must generate positive developmental outcomes to the extent they are foreseeable.
2. Research in development contexts must not only conform to the norms and standards
established by various human science disciplines and fields of practice, but it must also to
additional standards that support relevant, respectful, and right-sized research. (An evolving set
of principles can be found at the end of this agreement.)
3. By committing to Lean Research Principles (see following page) and by committing to their
improvement, I will help generate better quality data and more efficient and effective
development outcomes. I agree to support these principles, help peers support them, and
contribute to the continuous improvement of standards and practices associated with them.
4. I understand that conforming to Lean Research principles may shift costs, risks and burdens
from the research subject to the Principal Investigator. I am willing to take on this challenge as a
researcher, donor or other facilitator of research.
Name, Organization (if appropriate), Date
(DO SEE FOLLOWING PAGES FOR LEAN RESEARCH PRINCIPLES)
2
Lean Research Principles
Lean research consists of three principles based on a foundational assumption. From time to time,
researchers may be making trade-offs between applying the standards of their disciplines (our
foundational assumption) and adhering to Lean Research principles.
Foundational Assumption
All researchers agreeing to conform to the Lean Research principles already rigorously apply the
standards of their disciplines or field of practices.
We assume that those agreeing to apply the Lean Research principles, to the best of their ability, uphold
the standards set by their fields of study and practice For example, if the researcher is an ethnographer
or an economist, he or she follows the standards long established in those disciplines. If he or she is a
humanitarian worker, he or she follows research protocols accepted in humanitarian assistance.
At a minimum,
 If the research is an impact evaluation or trial, it is registered - for RCTs register with AEA’s social
science registry and for RCTs and other methods with 3ie’s RIDIE.
The research follows the highest standards of the discipline or field of practice standards for study
and instrument design, sampling, data cleaning, and analysis.
 The research is reproducible: Data and protocols are available for reproduction by independent
researchers.
 Research subject data are fully protected.
 The research is internally valid (the estimates are unbiased) and the researcher can convincingly
demonstrate this.
 The research findings are reported transparently and responsibly.
Ideally,
 The research is externally valid (applies in some context beyond the study context) and the
researcher can demonstrate in which contexts it can be reliably generalized.
 The research has been reproduced (all data gathering and protocols) or verified by independent
parties.
The Three Lean Research Principles
The Lean Research principles go above and beyond the foundational assumption. Researchers
conforming to the principles are already well-versed in the rigorously application of standards in their
disciplines or fields of practice. In addition to applying those standards they agree to conform to the
Lean Research standards. Since research is a development intervention, the researcher from time to
3
time may be faced with choosing the standards of their disciplines with Lean Research principles. Such a
choice is inevitable and the researcher agrees to make those choices as ethically as is possible. Many but
not all of the principles outlined follow the Declaration of Helsinki.
Principle One - Lean research Is relevant.
Research subjects can clearly articulate the value of the research study. The findings are understandable
to by a wide range of stakeholders, including the subjects researched. The findings are accessible,
actionable, and able to drive changes in policy or practice.
At a minimum,
 Research subjects can clearly articulate the value of the research study.
 The researcher is committed to understanding the highest research priorities of the communities
and how their research addresses or does not address these priorities.
Stakeholders are identified in advance of the research project and give inputs into how they would
like to receive the findings.
The results of the research are delivered to stakeholders in a timely manner (within 2 months of
completing a report), so that decision makers at various levels can use the research findings
effectively.
De-identified data is made available as widely as possible.
The research findings are accessible by a wide range of stakeholders including research subjects.
The research findings are understandable by a wide range of stakeholders including research
subjects.
Stakeholders can make decisions based upon the findings. (Depending on the findings, a decision
may be to take no action.)
Regardless of the research findings, there is a process to verify that value of the research after the
completion of the research.
Ideally,
The research represents a top priority research question of the population studied.
Decision-makers agree in advance of the study to use research findings of the study to change or
not change (depending on findings) a policy, product or service.
The policy, product, or service change (resulting from the research) would significantly address a
priority problem identified by the population studied.
4
Principle Two - Lean research is respectful, even delightful.
Lean Research places the dignity and delight of the human subject at center of the experience. It
enables subjects to walk away from research after an informed consent process. It also values the
subject’s time and ideas and offers reasonable availability of a treatment, if a treatment is offered as
part of an experiment.
At a minimum,
The human subject has been given all necessary information to decide whether to participate in the
research.

The human subject feels truly free to reject participation in the study or to drop out of a study once
it has begun and suffer no negative repercussions.
No interviews on human subjects exceed a length of time that is agreed to be respectful among
various stakeholders.
Human subjects are fairly compensated for their time and expenses.
The research process provides an environment where the subject can enjoy the experience.
If a treatment is introduced and if the findings determine the treatment produces benefits, both the
treated and control populations have sustained access to the treatment.
 Enumerators are well-trained, well-treated and paid fairly in a timely way.
Ideally,

Research subjects co-design the research and reporting methods.
Research subjects co-design and test a method for informed consent that demonstrates a subject’s
ability to refuse the study without consequence.
An independent research team is able to obtain feedback from subjects on their research
experience (and concludes the experience was generally positive).
Subjects have the opportunity to refute findings and the researchers must publish any refutations
along his or her original published findings.
Principle Three - Lean Research is right-sized.
The sample and method are well-suited to the research objectives. The research does not include any
unnecessary questions or activities. This means that thorough work should be done up-front to ensure
the study is needed at all and that the information could not have been gathered through secondary
research or through key informants.
At a minimum,
 The scope of the research adequately addresses the nature of the problem (meaning the study is
5
not so narrow as to be irrelevant).
 The researcher can demonstrate that original research on the identified population is absolutely
necessary. All other avenues of secondary or key informant information gathering have been
exhausted.
The research has eliminated all activities (and questions) that are not critical to the research
objective.
The sample is sufficiently large as to generate data proportionate to the research objective, but not
larger.
Ideally,
 Researchers can prove the study is a priority using cost-benefit or other methods to assess tradeoffs between the proposed research and other possible research.
The researcher can demonstrate that the cost of the project (to subjects, donors, and the time of
other stakeholders) and its benefits are optimal.
6
Download