Improving Methods for Conducting Patient-Centered Outcomes Research
(Methods)
In addition to reviewing the five PCORI Merit Review criteria, please consider how well your
assigned applications respond to Methods specific PCORI Funding Announcement (PFA)
requirements. The following descriptions will help you understand what the Methods program
does and does not seek to fund. By reading the PFA and using this guide, you will help PCORI
identify potential areas for improvement in applications. The offline reviewer critique template
begins on page 3.
What is Methods interested in?
The Methods program is seeking projects to address gaps in methodological research
relevant to conducting patient-centered outcomes research.
Methods research areas of interest include:
•
•
•
Development of methods for patient and other stakeholder
engagement in research
Development and refinement of general analytic methods
including:
o Methods related to causal inference
o Methods related to heterogeneity of treatment effect
(HTE)
o Methods related to missing data
o Methods related to adherence
Development and refinement of design-specific analytic methods
including:
o Methods related to cluster-randomized trials
o Methods related to adaptive trials
o Methods related to registries
o Methods related to diagnostic tests
o Methods related to devices
o Methods related to decision analysis and simulation
models
o Methods related to complex interventions
Special topics of interest (topics of particular scientific and/or
strategic importance):
\
•
•
Research on issues related to human subjects protections
Research on novel methods, interventions, or strategies for
increasing recruitment and retention of participants in
PCOR/CER randomized trials, observational studies, and
registries
1
Although all letters
of intent have been
screened for
programmatic fit,
please notify your
MRO if you believe
any of your assigned
applications do not
align with the
Methods program’s
research interests.
Programmatic
alignment concerns
should NOT affect
your scores or your
written critiques.
•
Development of methods to support data research networks
o Methods to improve distributed analyses in data
research networks
o Methods to obtain longitudinal and complete data in
data research networks
What is Methods NOT interested in?

PCORI cannot fund applications that measure cost effectiveness.
o Research cannot conduct a formal cost-effectiveness analysis
in the form of dollar-cost per quality-adjusted life year.
o Research cannot measure the relative costs of care of two or
more alternative approaches as the primary criterion for
choosing the preferred alternative.

Methods cannot fund applications that develop practice guidelines,
coverage recommendations, payment or policy recommendations.

Topics related to methods for the following are considered nonresponsive:
o Methods related to generating, selecting, and prioritizing
topics for research
o Conducting systematic review in patient-centered outcomes
research/comparative effectiveness research
Please notify your MRO
if an application
includes a formal costeffectiveness analysis
or proposes to develop
practice guidelines or
recommendations.
Reminder of important required application component
The proposed research must adhere to all relevant PCORI
Methodology Standards (most relevant to assessment of Criterion
3). The following methodology standards are particularly
important:








Address a true gap in evidence in their application (RQ-1)
Identify the comparators and explain why they were
selected (RQ-5)
Provide a rationale and define “usual care,” if it is
proposed (RQ-5)
Show that the outcomes chosen are relevant to patients
and end users (RQ-5)
Show that study participants represent target populations
and are properly recruited, enrolled, and retained (PC-2)
Describe rigorous data analysis plans (IR-1, IR-3)
Address heterogeneity of treatment effects and state the
aims of HTE analysis (HT-1)
Describe their plan and statistical methods for handling
missing data in their project (MD-1, MD-2)
2
Please address in your
written critique how
well the application
has described this
required component.
Improving Methods for Conducting PCOR PFA
Offline Reviewer Critique Template
Please use this template only as a placeholder for your critiques. Once you have finished your critique, please
copy/paste your comments into the PCORI Online Critique Form.
Keep each criterion box to no more than 5000 characters, including spaces. Comments for each criterion should list
out strengths and weaknesses. Comments for the Human Subjects and Overall Narrative sections should be written
in paragraph form.
Request ID:
Criterion 1: Identifies evidence gaps noted in the PCORI Methodology Report or other comparable
sources
Refers to whether the proposed study explicitly identifies gaps noted in the report or other sources.
The proposal addresses the following questions:
 Does the research question identify a critical gap in current methodological understanding as
noted in the Methodology Committee Report or in other sources? Which particular gap(s)?
*Please note that proposals that do not explicitly identify a gap noted in the Methodology Committee Report will
still be considered. However, applicants should provide strong support for their claims that their proposal does
address a current gap in PCOR methods.
Criterion Score:
Strengths:
Weaknesses:
-
Criterion 2: Potential for the study to improve PCOR methods
Refers to the potential of the proposed methodological investigation and its results to change
methodological practices in ways that improve PCOR and ultimately support the decisions made by
patients and their clinicians. The proposal addresses the following questions:
 Do existing methods weaken the validity of PCOR studies? Would improved methods increase
the validity of PCOR findings?
 How often would these methods be used, and how many PCOR studies would benefit from
these improved methods?
 Is the proposed approach feasible and likely to result in new standards or in the improvement of
existing standards?
3
Criterion Score:
Strengths:
Weaknesses:
-
Criterion 3: Technical Merit
Refers to the technical merit of the proposal. The proposal addresses the following questions:
 Is there a clear research plan with rigorous methods that demonstrates adherence to
PCORI’s Methodology Standards?
 Does the proposal delineate a clear conceptual framework/theory/model that anchors the
background literature and informs the design, key variables, and relationships being tested?
 Do the study methods reflect state-of-the-art thinking and practice in the methodological area,
so that results are likely to be accepted and heeded?
 Is the project timeline realistic, including specific scientific and engagement milestones?
 Does the research team have the necessary expertise to conduct the project? Are the
organizational structure and the described resources appropriate to carry out the project?
 Will the proposed methods help support the inclusion and study of diverse populations with
respect to age, gender, race, ethnicity, geography, or clinical status, or, alternatively, do the
methods support the inclusion of previously understudied populations in PCOR?
Criterion Score:
Strengths:
Weaknesses:
-
Criterion 4: Patient-Centeredness
The proposal demonstrates patient-centeredness at every stage of the research. It addresses the
following questions:
 Is the research focused on questions that affect outcomes of interest to patients and their
caregivers?
 Does the research address one or more of the key questions mentioned in PCORI’s definition of
patient-centered outcomes research?
4
Criterion Score:
Strengths:
Weaknesses:
-
Criterion 5: Patient and Stakeholder Engagement
The proposal demonstrates that people representing the population of interest and other relevant
stakeholders are engaged in ways that are appropriate and necessary in a given research context. It
addresses the following questions:
 Are patients and other stakeholders engaged in:
o Formulating research questions
o Defining essential characteristics of study participants, comparators, and outcomes
o Monitoring study conduct and progress
o Designing/suggesting plans for dissemination and implementation activities
 Are the roles and the decision making authority of all research partners clearly stated?
 Does the proposal demonstrate the principles of reciprocal relationships, co-learning,
partnership, trust, transparency, and honesty?
 If engagement is not applicable to the proposed research, does the application adequately
justify why it is not?
Criterion Score:
Strengths:
Weaknesses:
-
5
Protection of Human Subjects:
Does the application have acceptable risks and/or adequate protections for human subjects? (Yes/No)
Comments:
Overall Narrative:
Overall Score:
6