Assessment of Prevention, Diagnosis, and Treatment Options (APDTO)
In addition to reviewing the five PCORI Merit Review criteria, please consider how well your
assigned applications respond to APDTO specific PCORI Funding Announcement (PFA)
requirements. The following descriptions will help you understand what the APDTO program
does and does not seek to fund. By reading the PFA and using this guide, you will help PCORI
identify potential areas for improvement in applications. The offline reviewer critique template
begins on page 4.
What is APDTO interested in?
The APDTO program is seeking applications for comparative effectiveness research designed
to provide evidence-based information to inform critical decisions that face patients and
caregivers, clinicians, policy makers, and healthcare system leaders.
APDTO must fund applications that compare the effectiveness of
at least 2 viable alternatives that have established efficacy.


NOTE: “Usual care” may be a comparator if this is a
realistic choice faced by patients and other stakeholders.
If usual care is included, the application (1) must justify the
choice to use usual care, and (2) should clearly describe
the components of usual care that will be used or
measured in the research.
1
Please notify your MRO as
soon as possible if you
believe any of your
assigned applications do
not include a comparator.
Your MRO will discuss any
identified issues with
program staff and provide
you with further
instructions.
What is APDTO not interested in?
PCORI cannot fund applications that measure cost effectiveness.
 Research cannot conduct a formal cost-effectiveness analysis in
the form of dollar-cost per quality-adjusted life year.
 Research cannot measure the relative costs of care of two or
more alternative approaches as the primary criterion for
choosing the preferred alternative.
PCORI cannot fund applications that develop practice guidelines,
coverage recommendations, payment or policy recommendations.
APDTO applications should not focus on the following categories:











Study of the natural history of disease
Instrument development
Pharmacodynamics
Fundamental science or study of biological mechanisms
Creation of clinical practice guidelines or care pathways
Policy development
Developing, testing, and validating new decision aids/tools or
clinical prognostication tools
Establishing efficacy for a new clinical strategy
Pilot studies intended to inform larger efforts
Comparisons of patient characteristics rather than clinical
strategy options
Studies comparing interventions for which the primary focus is
the role of community health workers or patient navigators
2
If an application meets
any of these
conditions, please
notify your MRO as
soon as possible. Your
MRO will discuss all
identified issues with
program staff and
provide you with
further instructions.
Reminder of important required APDTO application components
 The proposed research must study the benefits and harms of
interventions and strategies delivered in real-world clinical settings
(Criterion 2)
• Will the study provide practical information that will help patients
and other stakeholders make informed decisions about their
health care and health outcomes?
 The proposed research must compare 2 or more clinical approaches
or strategies that have established efficacy (i.e., the efficiency or
effectiveness of each intervention is known) (Criterion 3)
 Does the application include information about the efficacy of
interventions to be studied?
 Projects that aim to develop new or novel interventions, which
lack evidence of efficacy or effectiveness, will be considered out of
scope.
 The proposed research must have sufficient statistical power to
detect clinically meaningful effects (Criterion 3)
• Does the application justify the proposed sample sizes?
• Explain assumptions used in all study power calculations (RQ-4)
• For observational studies, is causal inference addressed?
 The proposed research must include health outcomes that are
meaningful to the patient population (Criterion 4)
 The proposed research must adhere to all relevant PCORI
Methodology Standards (most relevant to assessment of Criterion 3).
The following methodology standards are particularly important:
 Address a true gap in evidence in their application (RQ-1)
 Identify the comparators and explain why they were selected (RQ5)
 Provide a rationale and define “usual care,” if it is proposed (RQ-5)
 Show that the outcomes chosen are relevant to patients and end
users (RQ-5)
 Show that study participants represent target populations and are
properly recruited, enrolled, and retained (PC-2)
 Describe rigorous data analysis plans (IR-1, IR-3)
 Address heterogeneity of treatment effects and state the aims of
HTE analysis (HT-1)
 Describe their plan and statistical methods for handling missing
data in their project (MD-1, MD-2)
3
Please address in
your written
critique how well
the application has
described each of
these required
components.
Assessment of Prevention, Diagnosis, and Treatment Options (APDTO) PFA
Offline Reviewer Critique Template
Please use this template only as a placeholder for your critiques. Once you have finished your critique, please
copy/paste your comments into the PCORI Online Critique Form.
Keep each criterion box to no more than 5000 characters, including spaces. Comments for each criterion should list
out strengths and weaknesses. Comments for the Human Subjects and Overall Narrative sections should be in
written in paragraph form.
Request ID:
Criterion 1: Impact of the condition on the health of individuals and populations
The proposal addresses the following questions:
 Is the condition or disease associated with a significant burden in the US population, in terms of
prevalence, mortality, morbidity, individual suffering, or loss of productivity?
 Alternatively, does the condition or disease impose a significant burden on a smaller number of
people who have a rare disease?
 Does the proposal include a particular emphasis on patients with one or more chronic
condition(s)?
Criterion Score:
Strengths:
Weaknesses:
-
Criterion 2: Potential for the study to improve health care and outcomes
The proposal has the potential to lead to meaningful improvement in the quality and efficiency of care
and to improvements in outcomes that are important to patients. It addresses the following questions:
 Does the research question address a critical gap in current knowledge as noted in systematic
reviews, guideline development efforts, or previous research prioritizations?
 Has it been identified as important by patient, caregiver, or clinician groups?
 Do wide variations in practice patterns suggest current clinical uncertainty?
 Is the research novel or innovative in its methods or approach, in the population being studied,
or in the intervention being evaluated in ways that make it likely to improve care
 Do preliminary studies indicate potential for a sizable benefit of the intervention relative to
current practice? How likely is it that positive findings could be disseminated and implemented
quickly, resulting in improvements in practice and patient outcomes?
4
Criterion Score:
Strengths:
Weaknesses:
-
Criterion 3: Technical Merit
The proposal has sufficient technical merit to ensure that the study goals will be met. It addresses the
following questions:
 Does the proposal delineate a clear conceptual framework/theory/model that anchors the
background literature and informs the design, key variables, and relationships being tested?
 Does the research plan describe rigorous methods that demonstrate adherence to PCORI’s
Methodology Standards?
 Are the comparison interventions realistic options that exist in current practice?
 Are sample size and power estimates presented that are based on realistic and careful
evaluations of the anticipated effect size?
 Is the project timeline realistic, including specific scientific and engagement milestones?
 Does the research team have the necessary expertise to conduct the project?
 Is the organizational structure and are the described resources appropriate to carry out the
project?
 Is there a diverse study population with respect to age, gender, race, ethnicity, and clinical
status, appropriate for the proposed research?
Criterion Score:
Strengths:
Weaknesses:
-
5
Criterion 4: Patient-Centeredness
The proposal demonstrates patient-centeredness at every stage of the research. It addresses the
following questions:
 Is the research focused on questions that affect outcomes of interest to patients and their
caregivers?
 Does the research address one or more of the key questions mentioned in PCORI’s definition of
patient-centered outcomes research?
Criterion Score:
Strengths:
Weaknesses:
-
Criterion 5: Patient and Stakeholder Engagement
The proposal demonstrates that people representing the population of interest and other relevant
stakeholders are engaged in ways that are appropriate and necessary in a given research context. It
addresses the following questions:
 Are patients and other stakeholders engaged in:
o Formulating research questions
o Defining essential characteristics of study participants, comparators, and outcomes
o Identifying and selecting outcomes that the population of interest notices and cares about
(e.g., survival, function, symptoms, health-related quality of life) and that inform decisionmaking relevant to the research topic
o Monitoring study conduct and progress
o Designing/suggesting plans for dissemination and implementation activities
 Are the roles and the decision-making authority of all research partners clearly stated?
 Does the proposal demonstrate the principles of reciprocal relationships, co-learning, partnership,
trust, transparency, and honesty?
Criterion Score:
6
Strengths:
Weaknesses:
-
Protection of Human Subjects:
Does the application have acceptable risks and/or adequate protections for human subjects? (Yes/No)
Comments:
Overall Narrative:
Overall Score:
7