Supplemental Table 3. Summary of pre-meeting survey on guideline / recommendation development
Organization
American College of
Medical Genetics and
Genomics (ACMG)
Target Audience
Target audience for the
Professional Practice and
Guidelines (PP&G)
committee is any
practitioner caring for
individuals (or
populations) relevant to
the guideline.
Target audience for the
Laboratory Quality
Assurance (Lab QA)
committee is genetic
laboratories and
laboratorians.
Factors involved in
assessing the robustness
of the scientific evidence
Determined by two
ACMG committees:
PP&G committee
(handles all clinical
practice guidelines) and
the Lab QA committee
(deals with laboratory
practice guidelines).
Assessments use:
 Published evidence
 Unpublished evidence
 Professional opinions
Standardized guidance on
identification of key
elements
Standardized guidance
on composition of
members
 Convene working group
to: reviewing conflict of
interest, review of
evidence, and preparation
of the draft guideline
 Draft reviewed by the
responsible committee
(PP&G or Lab QA)
 Draft presented to the
board of directors that
provides comments but
ultimately approves the
draft to be released for
review by the entire
ACMG membership
 ACMG member
comments considered
and incorporated where
appropriate resulting in a
final guideline approved
by the Board
 Guidelines published on
ACMG website and
submitted for publication
to Genetics in Medicine
(GIM)
 Peer review by GIM
 Given the rarity of
most conditions, a
small group of
content experts lead
working groups
 Membership of PPG
and Lab QA
committees are
determined by the
Board, and chairs of
the committees are
appointed by the
ACMG president
 All committee
members and chairs
undergo COI review
before appointment
and annually
 All policies and
procedures are
defined in a
committee charter
developed by the
Governance
committee and
approved by the
Board
Management of conflict
of interests
Standardized criteria for
synthesizing the evidence
 COI performed by
committee with final
approval by the board
of directors
 In rare disease, experts
often have commercial
relationships
 Managed by making
sure workgroups have
representation from
members who may not
be content experts but
have enough
experience in the
subject to participate
and monitor for
conflicts
 Board plays hands-on
role in these instances
to monitor the conflict
 General policy that only
one individual from a
given institution can
participate (with
exceptions)
 Knowledge synthesis does
not use systematic
approaches, heavy reliance
on expert opinion
 Moving towards performing
evidence grading
 Laboratory Quality Assurance
Committee has access to
information from Quality
Control and Proficiency
Testing activities and uses
this real world information to
inform guidelines
Use of systematic reviews
Use of external review
With few exceptions
systematic reviews are not
performed or commissioned by
the ACMG. If systematic
reviews are available from
other sources these are used in
guideline development.
Have used external reviewers
on a case by case basis if the
Board is concerned about the
influence of potential conflicts
of interest. Peer review in GIM
1
American Society of
Clinical Oncology (ASCO)
Oncologists
 Quality appraisal of the
evidence is dependent
on the type of studies
being considered
 Therapeutic guidelines:
Randomized controlled
trials ( RCTs) or metaanalyses of RCTs
 Prognostic/predictive
guidelines: prospective
cohort studies or
retrospective analyses
with identifiable
control groups
Blue Cross Blue Shield
Association (BCBS TEC)
Health plans and the
public
Technologies must have
established:
 analytic validity
 clinical validity
 clinical utility
 Clinical Practice Guideline
Committee approves
proposed topics, a
protocol is generated, cochairs identified, panel
members selected,
disclosures and COI
addressed, systematic
review planned and
conducted via an
approved protocol
 Results are presented to
the panel of content and
methods experts
 Recommendations
generated and reviewed
internally and externally
as well as by reviewers at
the J Clin Oncol prior to
finalization, publication,
and posting on the ASCO
website
See TEC Assessments
(published at:
http://bcbs.com/blueresour
ces/tec/)
 Members including
chairs must not have
51% or more COI
 Balance of content
and methods experts
and at least one
patient
representative
 Described in more
detail in ASCO
Methods manual
Personal disclosure form
plus web based search
for COI
Evidence summaries are based
on systematic reviews and
sometimes meta-analyses
either published separately or
contained within supplemental
material for the specific
guideline
 Systematic reviews required
for main line guidelines
 ASCO will endorse up to date
evidence-based guidelines
based on systematic reviews
from other credible
organizations
 ASCO has adopted a
Consensus Guideline process
based on a modified Delphi
technique when high level
evidence is not available for
important clinical issues
Yes but limited to professions
inside and outside of the
Society
 19-member Medical
Advisory Panel
includes appointees
from the American
College of Physicians,
American College of
Medical Genetics,
American Academy
of Family Physicians,
American College of
Cardiology, American
Academy of
Pediatrics, and
American College of
Surgeons
Please see:
http://bcbs.com/blueres
ources/tec/
TEC Assessments and TEC
Criteria (published at:
http://bcbs.com/blueresources
/tec/)
Crucial as they provide critical
appraisal of evidence base
Systematic reviews are
externally reviewed
2
Clinical
Pharmacogenetics
Implementation
Consortium (CPIC)
See publication: PMID:
21270786
Clinicians who use
pharmacogenomic
testing
3-tier scheme used to
rate the quality of
evidence linking druggene associations
Levels of Quality:
 High: consistent
evidence from
well-designed, wellconducted studies
 Moderate: sufficient
evidence of the effects,
but evidence strength
is limited by the
number, quality, or
consistency of the
individual studies, by
the inability to
generalize to routine
practice, or by the
indirect nature of the
evidence
 Weak: insufficient
evidence to assess
effects because of the
limited number of
studies, insufficient
power of the studies,
important flaws in their
design or in the way
they were conducted,
gaps in the chain of
evidence, or lack of
information
 The key elements to
include in CPIC guidelines
are standardized and
were agreed upon by
consensus of CPIC
members
 Guidelines are peerreviewed and published in
a leading journal with
simultaneous posting to
PharmGKB with
supplemental
information/data and
updates
 For additional information
see:
http://www.pharmgkb.or
g/page/cpic
 CPIC members
review guidelines
prior to submitting
publication
 Senior author must
be a leader in the
content area
addressed by the
guideline and coauthors should have
a track record of
publication or
expertise in the
specific topic area of
the guideline
 Steering Committee
approves the
composition of the
guideline authors
and the writing plans
for each new
gene/drug CPIC
guideline
 Welcomes other
national and
international groups
to assist with the
clinical
implementation of
pharmacogenetics
(email:
feedback@pharmgkb
.org)
 CPIC members are
published on the
PharmGKB website
 Authors (and family
members) of CPIC
guidelines must fully
declare all possible
conflicts (including NIH
funding) that could be
interpreted to indicate
that authors are
“advocates” of the
enclosed
recommendations, as
well as any sources of
revenue from patents,
stock ownership, etc
 Knowledge synthesis aims to
address:
1. “What genotypes are so
severe you would really act
upon them if you knew the
patient had that diplotype
(for at least one drug)?”
2. “What drugs are so clearly
affected that you would be
wrong not to act on the
result if (a) you knew the
genotype and (b) you wanted
to prescribe that drug?”
 Authors systematically search
the literature to find
evidence relevant for making
dosing recommendations
 Any existing consensus
guidelines for the gene or
drug are also reviewed.
 Guideline will not go forward
if there is lack of consensus
 Dosing recommendations are
based on weighting the
evidence from a combination
of preclinical functional and
clinical data, as well as on
some existing disease-specific
consensus guidelines.
 Uses a slight modification of
a transparent and simple
system for just three
categories for
recommendations adopted
from the rating scale for
evidence-based
recommendations on the use
of retroviral agents
 All CPIC manuscripts are
externally peer-reviewed and
published
 Additional information at
PharmGKB is curatorreviewed
 Regular updates are peerreviewed and published
 Guidelines are re-examined
approximately every 2-3
years
 The entire project supporting
CPIC is a peer-reviewed
research project supported
by NIGMS, NIH
3
Evaluation of Genomic
Applications in Practice
and Prevention (EGAPP)
Researchers and
clinicians interested in
genomic applications
Dutch Pharmacogenetics
Working Group (DPWG)
Dutch clinicians,
pharmacists, and
physicians
 Quality of evidence
based on study design
and conduct
 Robustness of evidence
based on number of
studies and sample size
 Consistency and
precision of estimated
effects, completeness
of evaluation (
 Applicability of
evidence to the
population for whom
the answer is needed]
 Directness of evidence
(in terms of outcomes
studied and the health
impacts important to
understand)
 Ethical, legal, and social
contexts, including
current policy and
practice
For each reviewed paper
2 core parameters are
scored:
1) Level of evidence of
the gene–drug
interaction. This indicates
the quality of the
evidence found in
literature for the gene–
drug interaction, and is
scored on a five-point
scale with a range from 0
(lowest evidence) to 4
(highest evidence)
2) Clinical relevance of
the potential adverse
drug event, decreased
therapeutic response, or
Follow a template for
reporting systematic
reviews developed by
AHRQ's EPC program, by
the USPSTF and that
conforms to commonly
accepted, current guidance
on reporting systematic
reviews (i.e., PRISMA)
Please see USPSTF
Procedure manual at
www.uspreventiveserv
icestaskforce.org
Engage key informants
and expert consultants at
various aspects of the
review planning and
conduct, and specifically
invite a range of
stakeholders to conduct
peer review of the draft
report
 Follow established methods
for systematic review,
adapted to the rules
established by the decisionmaker for whom we are
reviewing the evidence
 Adhere to quality
recommendations for
reporting of findings (such as
PRISMA) relevant to our work
 Systematic reviews are key to
an EGAPP recommendation,
they articulate the series of
questions decision-makers
need to have answered in
order to consider the
pertinent evidence relevant
to an evidence-based
guideline or recommendation
Peer reviewers for the draft
report, peer reviewers for the
manuscripts (at the journal),
and will have public comment
starting soon on the draft
reports concurrent with the
USPSTF draft recommendation
statement process
 The pharmacogeneticsbased therapeutic (dose)
recommendations are
developed according to a
strict predefined format
that contains the same
components for each of
the individual gene-drug
combinations.
 In the final report for
each gene-drug
combination a concise
rationale is provided in
order to clarify how the
DPWG has arrived at its
recommendations
 Members from
multiple professions
including: clinical
pharmacists,
physicians (internal
medicine), clinical
pharmacologists,
clinical chemists,
epidemiologists, and
toxicologists
 Members are
recruited among
clinician
(researchers) with a
well recognized track
record in any of
these fields
Relevant professional
societies are consulted
before the final version
of the guideline is
released
 2 researchers perform a
systematic review of
literature
 After data collection, the
level of evidence and clinical
relevance of each article are
scored
 Results are independently
reviewed by two members of
the DPWG
 Result evaluated by the
complete DPWG
Perform in-house systematic
reviews since for most genedrug combinations no
published systematic reviews
are available
 Relevant professional
societies are consulted
before the final version of the
guideline is released
 Results are published in peer
reviewed journals
4
other clinical effect
resulting from the gene–
drug interaction. The
clinical relevance was
scored on a seven-point
scale derived from the
National Cancer
Institute's Common
Toxicity Criteria.18 A
clinical or
pharmacokinetic effect
that was not statistically
significant was classified
as AA (lowest impact),
whereas death, for
example, was classified
as F (highest impact).
Please also see full
details in:
PMID:18253145 and
PMID:21412232
5
National Comprehensive
Cancer Network (NCCN)
Oncologists or other
clinicians who interact
with cancer patients.
Guidelines are free and
available to anyone
who registers on the
NCCN website.
High level evidence
constitutes randomized
clinical trials and metaanalyses. Lower level
evidence can comprise
smaller clinical trials,
and for diseases with
less available trial data,
clinical experience.



Institutional reviews
are solicited by panel
members 8-12 weeks
prior to annual panel
meetings. External
submissions, from nonmember institution
clinicians, industry,
payers, or patients are
collated by NCCN and
submitted to the
panels prior to annual
meetings.
Institutional reviews
and external
submissions are
reviewed and
discussed. Panel votes
are taken and recorded
for substantive
changes to the
Guidelines.
Changes to
recommendations for
use of drugs and
biologics are listed in
transparency
documents (available
at nccn.org) along with
the actions taken by
the Guidelines Panel
including votes.
Guidelines Steering
Committee and
NCCN Senior Staff
recommend and
appoint panel chairs.
GL steering
committee members
nominate panel
members from their
individual
institutions.

Disclosure of all
relevant financial
interests are
required for each
member of the
NCCN Guidelines
Panel, including
chairs/co-chairs, vice
chairs, panel
members, NCCN
headquarters staff,
member of the
Guidelines Steering
Committee, and the
NCCN Board of
Directors.

The NCCN policy for
management of
conflicts of interest
mandates the
disclosure of all
financial
relationships and the
recusal from
deliberations and/or
votes of individuals
with a meaningful
conflict of interest.
Categories of evidence:
 Category 1: High-level
evidence there is uniform
NCCN consensus (85%
majority vote) that the
intervention is
appropriate.
 Category 2A: Lower level
evidence there is uniform
NCCN consensus (85%
majority vote) that the
intervention is
appropriate.
 Category 2B: Lower-level
evidence, there is NCCN
consensus (50-85%
majority vote) that the
intervention is
appropriate.
 Category 3: Any level of
evidence, but major NCCN
disagreement (at least 3
panel member from 3
institutions) that the
intervention is
appropriate.
Systematic reviews not
generally performed by the
panels; however, meta
analyses from other sources
are considered.
No formal external review.
6
National Society of
Genetic Counselors
(NSGC)
Practicing genetic
counselors, though some
guidelines are written
more broadly
 No current system to
rank evidence
 Review the supporting
literature for study
design, sample size,
rigor of analyses, and
number of studies with
concordant finding
 Multiple
methodologically
robust papers with
similar findings, or
meta-analyses, carry
more weight than
single-institution
studies, especially
those with smaller
sample sizes or
descriptive/pilot data.
 Due to limited
literature in some areas
of genetic counseling
one cannot always rely
on large trials for
supporting evidence
 Focus on making
recommendations that
are well supported by
existing literature that will
guide the specific practice
of genetic counseling
 Author groups are
evaluated for real or
perceived conflicts of
interest by requiring that
each author submit a CV
and conflict of interest
statement at the initiation
of guideline writing
 Lead author cannot have
COI, nor can a majority of
the authors.
 Evaluate any potential
bias due to authors' COI
 Authors and Practice
Guidelines
Committee are
involved in each
guideline
 Each co-author has
to be approved by
the Committee
 Lead authors cannot
have COI, nor can
the majority of a
guideline's authors
 Committee members
sign a COI disclosure
statement annually
 Those with potential
COI are not excluded
from Committee
membership, but
must recluse
themselves from
reviewing a guideline
on a topic with which
they have real or
perceived COI
 Issue concrete
timelines and
instructions to author
groups at initiation of
guideline development,
and by directing the
Practice Guidelines
Committee to focus
their review on bigpicture issues such as
whether each
recommendation in a
guideline is sufficiently
supported and whether
there's apparent bias.
 Ethics Advisory
Committee, legal
representative and
Executive Board review
and approve each
guideline
 Guidelines undergo
peer review through
the Journal of Genetic
Counseling
 Committee selects a
liaison for each
guideline
 Guideline authors bear the
responsibility for synthesizing
the evidence, typically by a
thorough literature review
 Practice Guidelines
Committee (PGC) reviews the
synthesis of evidence
compiled by the authors and
determines whether it's
sufficiently thorough and
balanced
 Do not require systematic
reviews for all guidelines,
particularly those for which
such reviews have been
published recently
 When no such reviews exist,
and when having a review in
print would aid the guideline
writing process, recommend
that guideline author groups
publish a systematic review
prior to drafting a guideline
Yes, via the Journal of Genetic
Counseling
7