PRACTICE
RECOMMENDATIONS
DEVELOPMENT MANUAL
A guidebook to assist individuals who write,
evaluate and/or use AGA Institute Practice Recommendations.
Updated September 2007
American Gastroenterological Association (AGA) Institute
Clinical Practice and Quality Management Committee
Copyright ©2007. All rights reserved.
American Gastroenterological Association (AGA) Institute, Bethesda, MD
Contact Information for Questions
All questions regarding the technical content, scope, and
objectives of the guidelines manuscripts, general policies
governing AGA Institute guidelines, and Clinical Practice
and Quality Management Committee policies should be
directed to:
Sheila Agyeman, MHA
Director of Evidence Based Medicine
AGA Institute National Office
4930 Del Ray Avenue, 6th floor
Bethesda, MD 20814
Phone: 301-272-1189 (direct)
Fax: 301-652-3890
sagyeman@gastro.org
She is also available to answer questions regarding
manuscript requirements and processing, procedures,
teleconferences and meetings, and any other concerns.
Clinicians and others who have questions or comments about
published guidelines and technical reviews should direct them
in writing to:
Chair, Clinical Practice & Quality Management Committee
AGA Institute National Office
4930 Del Ray Avenue, 6th floor
Bethesda, MD 20814
Phone: 301-654-2055
All guidelines and technical reviews can be found at
www.gastro.org/guidelines.
2
AGA Institute Practice Recommendations Development Manual
0907
CONTENTS
Part I: Introduction
Rationale for this manual ........................................................................................4
Part II: Basic Assumptions
Assumptions on which the AGA Institute Practice
Recommendations process and policies are based ................................................5
Part III: Process for Developing AGA Institute Practice Recommendations
Overview, flowchart.................................................................................................7
Types of Practice Recommendations ......................................................................9
Part IV: Instructions for Authors
Technical Review Panel: Writing the technical review ........................................11
Medical Position Panel: Writing the clinical practice
guidelines, consensus statements and measures ..............................................16
Administrative procedures ....................................................................................20
Manuscript technical specifications and procedures ............................................21
Part V: Appendix
Appendix A: Detailed process for developing AGA Practice Recommendations .....22
Appendix B: Process for reviewing existing guidelines.........................................26
Appendix C: Manuscript requirements for Gastroenterology.................................27
Appendix D: Guidelines for writing CME questions and learning objectives ....34
Appendix E: Copyright Assignment Form for Gastroenterology ..........................39
AGA Institute Practice Recommendations Development Manual
0907
3
PART I: INTRODUCTION
The American Gastroenterological Association (AGA) Institute is dedicated to
advancing the science and practice of gastroenterology. Practice recommendations are an
important tool for educating physicians and guiding their patient-care decisions. Payers
and purchasers have used practice recommendations to develop coverage and payment
policies for medical care and to establish benchmarks for “quality” or “appropriate” care
(including pay-for-performance, or P4P, standards).
The AGA Institute is committed to the ongoing development of practice
recommendations. Formal recommendations are developed in a defined area of
gastroenterology practice, based both on the scientific literature and the collective
opinions of expert physicians. In essence, by developing a practice recommendation, the
AGA Institute is doing for physicians what they would do on their own if they had the
time, expertise and support to compile, compare and evaluate numerous clinical research
studies, obtain the opinions of multiple expert consultants, and synthesize the
information into a set of policies and procedures.
With practice recommendations playing a central role in today’s value-driven
medical marketplace, it is imperative that gastroenterology establish its own position and
recommendations, rather than let others fill this need.
The first step in the AGA Institute’s practice recommendations development process
is the compilation of a technical review, which is a comprehensive background paper
based on medical and scientific evidence. Once a technical review is written, three
possible documents will emerge, based on the strength of the evidence:
1. CONSENSUS STATEMENT – a statement on an area of controversy,
identification of reasonable practice or identification of future areas of study
where less evidence is available.
2. CLINICAL PRACTICE GUIDELINE – a set of recommendations where stronger
evidence is available.
3. MEASURES – a set of performance measures for use in pay for performance or
other quality initiatives.
This manual is intended to guide authors of practice recommendations and to serve
as a reference for those who review and evaluate practice recommendations. Authors’
conformance with AGA Institute’s practice recommendation development process should
lead to the creation of recommendations that are useful to physicians and beneficial for
patients.
4
AGA Institute Practice Rcommendations Development Manual
0907
PART II: BASIC ASSUMPTIONS
The AGA Institute’s practice recommendations development process rests on the
following assumptions:
1. Purpose.
Physician education is one of the main purposes of practice recommendations.
Though often used for other purposes, such as reimbursement decisions or
hospital quality assessment criteria, practice recommendations are developed to
educate physicians and guide their patient-care decisions. Practice
recommendations should not replace professional clinical judgment based on
experience and patient-specific circumstances. The ultimate goal is to improve
patient outcomes.
2. AGA Institute’s Role.
AGA Institute is a major center for the development of gastroenterology practice
recommendations. It is critical that the professional societies that represent the
field of gastroenterology take the lead in developing and issuing such practice
recommendations consistent with available science and expert consensus opinion.
Unless explicitly noted to the contrary, AGA Institute practice recommendations
represent the official opinion of the AGA Institute, and reflect the opinions of
individuals involved in developing the position paper.
3. Quality and Cost Implications for Health Care.
Research and literature highlight the impact of variation in care on health care
quality and outcomes. Practice recommendations and their use in practice
support consistent care and can address gaps and variation in care. As some
research suggests that better quality care is less costly than poor quality care,
practice recommendations can potentially lower health care expenditures by
eliminating unnecessary or ineffective practices and procedures. However, it is
important to note that the primary focus of AGA Institute practice
recommendations is neither cost containment nor reduction of resource use.
Third-party payers often link official practice recommendations to
reimbursement and payment policies. In some cases, these policies may be the
major reason(s) for developing practice recommendations, i.e., the need to have
AGA Institute Practice Recommendations Development Manual
0907
5
“something official on paper.” However, such financial considerations or health
plan benefits alone, should not be used as the rationale for any clinical practice
recommendation.
4. Legal Implications.
Practice recommendations have been, and will continue to be, used to define
appropriate or adequate medical care in professional malpractice actions. All
clinicians will be held accountable to any relevant practice standards, which can
also provide legal protection to defendants. Thus, it is important to word
recommendations carefully, and clearly define situations in which
recommendations are applicable.
5. Due Process.
Given the potential implications of these recommendations to patients and
physicians, it is imperative that they be developed via a deliberative, open and due
process. The system adopted by AGA Institute meets these characteristics and
enables us to respond to questions of the veracity or appropriateness of any of
our recommendations.
6. Scientific Data vs. Opinion.
To the extent possible, AGA Institute practice recommendations will be based on
valid, scientific data. However, there are situations when such data are
inconclusive or absent. Therefore, some recommendations will be based partially,
or even wholly, on expressions of expert opinion. While offering such opinions is
preferable to remaining silent on an issue, it is AGA Institute policy to clearly
identify the degree to which a practice recommendation document reflects hard
facts versus consensus opinion. If practice recommendations rely on consensus
opinion, they are generally not suitable for adoption as a quality metric for P4P
standards.
7. Dissemination of practice recommendations.
Dissemination of practice recommendations to medical professionals, and to the
public, is important. Therefore, all practice recommendation documents will be
widely disseminated in AGA Institute publications and other media. Considered
public documents, they will be made available to all interested parties upon
request.
6
AGA Institute Practice Recommendations Development Manual
0907
PART III: THE PROCESS FOR DEVELOPING
AGA INSTITUTE PRACTICE RECOMMENDATIONS
Step 1
Clinical Practice & Quality
Management Committee (CPQC),
AGA Institute Council, Governing Board
and/or AGA member (s) identify topic(s)
to be developed within the coming year.
Step 2
The CPQC approves final topics,
determines scope of topic(s) and objectives
for technical review. AGA Institute
Council identifies and secures authors and
external reviewers via the appropriate
Council section.
Step 3
Technical Review Panel (TRP) is
formed consisting of the authors, member
of CPQC, and member of appropriate
Council section. TRP drafts technical
review.
Step 4
TRP determines what type of practice
recommendation to produce, based on
quality of evidence in technical review:
consensus statement, guideline or
measures (for use in P4P or other
quality-improvement efforts).
AGA Institute Practice Recommendations Development Manual
0907
7
Step 5
A Medical Position Panel (MPP)
consisting of 3-4 members of TRP,
community-based GI, primary care
physician, payer, surgeon (if applicable),
patient/patient advocate and
gastroenterologist, convene to write
the official guideline.
Step 6
Final draft of guideline sent to external
reviewers for review and comment. MPP
responds to external review comments.
Step 7
“Final” draft of guideline sent to
CPQC. CPQC members on TRP present
guideline to full committee for review.
Necessary final edits made prior to
submission to AGA Institute Governing
Board.
Step 8
Governing Board reviews and approves
guideline and corresponding technical
review. Publication in Gastroenterology.
8
AGA Institute Practice Recommendations Development Manual
0907
TYPES OF AGA INSTITUTE
PRACTICE RECOMMENDATION DOCUMENTS
The AGA Institute issues three
types of practice recommendation
documents: consensus statements,
clinical practice guidelines and
performance measures or quality
indicators for quality improvement
and/or P4P initiatives. The basis of
these three documents is a technical
review. A technical review is a
comprehensive background paper
based on evidence-based literature.
Technical Review
Consensus
Statements
Clinical
Practice
Guidelines
Measures
Technical Reviews
A technical review is a background paper from which the consensus statement,
clinical practice guideline and measures are derived. Its purpose is to provide the
user with the evidence utilized to formulate a particular recommendation. The
strength and character of that evidence is also ranked based on the U.S.
Preventative Services Task Force Ratings.
The practice recommendations will be published along with its corresponding
technical review. The primary reason for producing both documents is that they
provide flexibility in disseminating the recommendations they each contain.
Authors of technical reviews are expected to produce a paper of publishable
quality. Technical reviews will be published in Gastroenterology.
Clinical Practice Guidelines
There are numerous accepted definitions of evidence-based clinical practice
guidelines and processes for their development. Methods used to produce
clinical practice guidelines range from the very rigorous lengthy and expensive
process described in the Institute of Medicine’s (IOM) Clinical Practice
Guidelines: Directions for a New Program to the simple, but scientifically soft,
issuance of guidelines consisting of the unsupported opinions of a few experts.
The process described herein represents a workable balance between these two
AGA Institute Practice Recommendations Development Manual
0907
9
extremes. A clinical practice guideline is a concise expression of the AGA
Institute’s official opinions and clinical practice recommendations on a given
subject. The guideline is based on the relevant technical review.
Consensus Statements
Consensus statements represent the opinion of an expert panel convened by the
AGA Institute to address a set of specific questions related to a defined medical
practice issue. Typically, outside experts are invited to present their answers to
these questions based on their interpretation of the relevant evidence which the
technical review provides. The expert panel then convenes to develop its answers
to the questions based on the presentations it received. The panel produces a
report suitable for publication in an AGA Institute journal. This report
represents the opinions of the panel, not the AGA Institute.
Measures
Quality Indicators a set of metrics that are less evidence based and often process
oriented.
Performance measures are metrics that: (1) measure the degree of
accomplishment of a health objective, (2) are generally developed from evidencebased guidelines, and (3) measure health outcomes.
Composite measures are a set of measures used to better understand
information from numerous individual measures. When strong evidence and
guidelines exist, composite measures reflect the quality/outcomes of the care with
a set of measures which together reflect care of chronic/complex patient
conditions.
10
AGA Institute Practice Recommendations Development Manual
0907
PART IV: INSTRUCTIONS FOR AUTHORS
Technical Review Panel: Writing the technical review
Title Format
The papers should be titled: “AGA Institute Technical Review on [topic].”
Structure and Organization
The form of a technical review may vary somewhat depending on the subject matter. It
should, however, at least contain the following sections:
1. Introduction that includes:
— Subject of the technical review. This could be specific procedures or treatments
or management of a defined patient problem.
— Importance of the issue (e.g., patient outcomes, cost & implications).
— Objectives and contents of technical review.
— List of key questions technical review will address.
Particular emphasis should be given to issues such as:
1) when a patient will or will not need intense diagnostic or therapeutic
measures;
2) the hierarchy and sequence of recommended diagnostic or therapeutic
interventions.
2. Literature review and analysis that includes:
— The basis for the paper’s conclusions and recommendations. Should contain the
authors’ compilation and assessment of the relevant experimental, clinical and
epidemiological data (not necessarily all three) on the topic.
— Relevant quality, cost-effectiveness and cost-benefit data if available.
The following information should be included in this section:
(Note: AGA Institute staff will perform literature searches. A compilation and comparison of
this information and the study quality descriptions will be provided to authors in a tabular
display similar to what is shown in Figure 1.)
— Criteria used to include and/or exclude data from consideration (e.g.,
studies published only as abstracts were excluded; only clinical studies
AGA Institute Practice Recommendations Development Manual
0907
11
published after 1997 were included).
— Description of the electronic and manual search process, including
search terms used.
— Short descriptions of the studies cited. Description should include
such factors as study design, clinical endpoints, randomization,
number of subjects, controls employed and blinding.
— Rank the strength of the level of evidence for all references using the
U.S. Preventive Services Task Force (USPSTF) Ratings (see Figure 2).
— Quality of cited studies. Indicate their respective strengths (Wellcontrolled? Prospective? Randomized double-blinded trial? Good
statistical power?) and limitations (Short study period? Small patient
population? Questionable therapeutic endpoints?).
3. Conclusions and clinical practice recommendations based on the foregoing
analysis. Clinical recommendations should:
— Define the patient populations to which the recommendations do and
do not apply. For example, some recommendations may not be
indicated for particular types of patients by virtue of their age or sex.
— Indicate type of health care provider to which the recommendations
apply. Some recommendations may be applicable to gastroenterology
specialists but not to primary care physicians.
— Type of practice recommendation (guideline, consensus statement,
measures) that should be produced based on the quality of evidence.
— Outcomes expected from the recommendations.
— Whether or not specific recommendations lend themselves to
performance measure development. If they do, authors should
describe the performance measures and identify a way to measure
their impact or outcomes. If applicable, include suggestions on what
data or research methods could be used (e.g., chart review,
administrative data, surveys).
Recommendations should be worded so the reader is clear that they represent
published literature, the authors’ opinions or both. The latter often will be the case
where the supporting data are suggestive, not conclusive.
4. The authors’ suggestions for future research indicating:
— Areas requiring further research studies due to insufficient data, no data, poor
quality data or conflicting data. This can be included as a separate section of the
document or integrated into the review and analysis section.
5. The references cited in the body of the paper. These should be in the style used
by Gastroenterology (see Appendix C). Abstracts from meetings (published or
unpublished) should be clearly identified as such and should be used only when no
comparable studies exist. Primary reference sources are preferred over textbooks or
other secondary reference sources. Letters and case reports should be used
cautiously. Unpublished studies are not acceptable. References should be cited in
numerical order in the text by number, not author’s name or publication date.
12
AGA Institute Practice Recommendations Development Manual
0907
Fig. 1 Example of a table for comparing published studies.
AGA Institute Practice Recommendations Development Manual
0907
13
Fig. 2 U.S. Preventive Services Task Force (USPSTF) Ratings
Strength of Recommendations
The U.S. Preventive Services Task Force (USPSTF) grades its recommendations according to one of five
classifications (A, B, C, D, I) reflecting the strength of evidence and magnitude of net benefit (benefits minus harms).
A
The USPSTF strongly recommends that clinicians provide [the service] to eligible patients. The
USPSTF found good evidence that [the service] improves important health outcomes and
concludes that benefits substantially outweigh harms.
B
The USPSTF recommends that clinicians provide [this service] to eligible patients. The USPSTF
found at least fair evidence that [the service] improves important health outcomes and concludes
that benefits outweigh harms.
C
The USPSTF makes no recommendation for or against routine provision of [the service]. The USPSTF
found at least fair evidence that [the service] can improve health outcomes but concludes that the
balance of benefits and harms is too close to justify a general recommendation.
D
The USPSTF recommends against routinely providing [the service] to asymptomatic patients. The
USPSTF found at least fair evidence that [the service] is ineffective or that harms outweigh
benefits.
I
The USPSTF concludes that the evidence is insufficient to recommend for or against routinely
providing [the service]. Evidence that the [service] is effective is lacking, of poor quality, or
conflicting and the balance of benefits and harms cannot be determined.
Quality of Evidence
The U.S. Preventive Services Task Force (USPSTF) grades its recommendations according to one of five
classifications (A, B, C, D, I) reflecting the strength of evidence and magnitude of net benefit (benefits minus harms).
Good
Evidence includes consistent results from well-designed, well-conducted studies in representative
populations that directly assess effects on health outcomes.
Fair
Evidence is sufficient to determine effects on health outcomes, but the strength of the evidence
is limited by the number, quality, or consistency of the individual studies, generalizability to routine
practice, or indirect nature of the evidence on health outcomes.
Poor
Evidence is insufficient to assess the effects on health outcomes because of limited number or
power of studies, important flaws in their design or conduct, gaps in the chain of evidence, or lack
of information on important health outcomes.
Internet Citation: U.S. Preventive Services Task Force Ratings: Strength of Recommendations and Quality of
Evidence. Guide to Clinical Preventive Services, Third Edition: Periodic Updates, 2000-2003. Agency for
Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/clinic/3rduspstf/ratings.htm
14
AGA Institute Practice Recommendations Development Manual
0108
Though there is no set number of reference citations for a technical review,
generally, the list should not exceed 125. The selection and number of references is
left to the judgment of the authors. Most technical reviews will contain 75-100
references.
6. Continuing Medical Education (CME) questions and learning objectives.
Since 2006, the AGA Institute has offered CME credits related to each technical
review published. The Technical Review Panel prepares 7 questions and answers for
online CME credits. CME questions and answers are reviewed along with the
technical review. The TRP should also prepare 3 learning objectives for the paper.
See appendix D for guidance on developing CME questions and learning objectives.
Length
The length of a technical review will vary according to the subject matter, however the
typical length is 10-30 double-spaced typewritten pages.
Authorship & Publication
Technical reviews will be published in the print and/or online versions of the AGA
Institute journals as AGA Institute documents authored by Technical Review Panel
members listed in alphabetical order. Technical reviews will not be subject to the
journals’ normal manuscript review process. However, all manuscripts will be copy
edited for grammar, conciseness, and clarity, and are subject to the journal’s style and
other production requirements.
External Review
All technical reviews are sent to external reviewers. The Technical Review Panel will be
expected to consider their critiques carefully. AGA Institute staff will distribute
documents to reviewers and return comments to the Technical Review Panel. Lead
author(s) are welcome to solicit the advice and critiques of colleagues at any point in the
paper’s development. All copies of draft manuscripts should be prominently marked
“DRAFT-- Not for Distribution.”
Approval
Final Approval of technical review rests with the CPQC and Governing Board.
AGA Institute Practice Recommendations Development Manual
0907
15
Medical Position Panel: Writing the official
AGA Institute Clinical Practice Guideline
Title Format
Clinical practice guidelines should be titled as “AGA Institute Clinical Practice
Guideline: [on] [for] XYZ.”
Structure and Organization
Guidelines will have two sections: a short introduction and the main body of the paper,
which presents the AGA Institute’s recommendations:
1. The introduction will contain the following elements:
— A clear definition of the issue being addressed.
— Discussion of the importance of subject, including clinical and/or policy
considerations.
— Description of the types of patients or clinical situations to which the guidelines
apply or do not apply.
— A statement describing the provider audience to which the recommendations apply.
2. Recommendations should be formatted as follows:
— Bulleted or numbered statements or paragraphs.
— Each recommendation should be followed by a summary paragraph describing
the quality of the evidence and the global U.S. Preventive Services Task Force
(USPSTF) (see figure 2) score for each recommendation. Recommendations
must be consistent with those of the technical review.
— Recommendations relating to therapies, treatments, or procedures should be
explicitly labeled as standards, guidelines or clinical alternatives. They are
characterized as follows in AGA Institute documents:
a. Standard: A recommendation considered to be a standard of practice
applies virtually without exception to all patients or situations defined
in the guideline, is strongly supported by the clinical research literature
(epidemiological and experimental studies generally would be
considered to be weaker support), and is based almost totally on hard
facts. The words “must” and “shall” will be found in standards.
b. Clinical practice guidelines: A recommendation, or set of
recommendations, intended to assist the physician in making patient care
decisions. In most cases the recommendation should be followed, but the
decision to do so is at the option of the physician, based on the circumstances
of the individual case at hand. Support in the research literature for
guidelines ranges from suggestive to strong; hence, the guidelines will be
based, to some degree, on expert opinion. The words “should,” “usually,”
and “generally” will be used in guidelines as opposed to “must” or “shall.”
c. Clinical alternative: Frequently, two or more treatments/therapeutic
approaches, often including a “no-treatment” option, may be equally
acceptable. This may be because the research literature does not strongly
support one over the other, because the data upon which a choice could be
based are nonexistent, or because the data supporting a particular option are
16
AGA Institute Practice Recommendations Development Manual
0907
judged to be weak. In this case, recommendations should be presented as a set
of alternatives, the choice of which is left solely to the physician. Clinical
alternatives should be presented in a neutral fashion: e.g., “Data on the
comparative efficacy of A vs. B have not been reported; hence, a
recommendation cannot be made at this time.” If a preference is stated, it
should be made clear that it represents opinion or common practice only. For
example: “It is common practice to first try XYZ; however, data to support
this approach over ABC or DEF are lacking,” or “The choice of A, B, or C is
controversial. It is our belief that A is the most promising; however, further
research is needed to substantiate this opinion.” Words such as “could”
“might” and “may” will be found in statements of clinical alternatives.
3. Quality Indicator or Performance Measure:
Each guideline will include suggested quality indicators or measures as tools to
support practices in compliance with the guideline. One to five quality indicators/
measures should be provided.
Length
A guideline should not exceed four double-spaced, typewritten pages.
Authorship & Publication
All members of the Medical Position Panel are listed, but there are no official authors
of the statement as it serves as the official position of the AGA Institute. The CPQC may
change or rewrite the guideline as they deem appropriate. This emphasizes the fact that
the guideline is the opinion of the AGA Institute as a whole, not of any individual or
group of individuals. All AGA Institute guidelines will be edited for grammar and clarity
and published in the AGA Institute journal Gastroenterology.
References
Authors must cite the companion technical review as the source of the data and for further
details to what the guidelines contain. Generally, there should be no need to refer to primary
literature references as these should have been included in the technical review.
External Review
All guidelines are sent to external reviewers. The Medical Position Panel will be expected to
consider their critiques carefully. AGA Institute staff will distribute documents to reviewers
and return comments to the Medical Position Panel. Lead author(s) are welcome to solicit the
advice and critiques of colleagues at any point in the paper’s development. All copies of draft
manuscripts should be prominently marked “DRAFT-- Not for Distribution.”
Approval
Final Approval of guidelines rests with the CPQC and Governing Board.
AGA Institute Practice Recommendations Development Manual
0907
17
Writing the AGA Institute Consensus
Statement
The CPQC will select a panel of experts, which may include members of the Technical
Review Panel, to write the consensus statement. This statement will address a defined
medical practice issue for which evidence is weak by addressing a set of specific clinical
questions. Outside experts on the topic will be invited to present their answers to the
panel. The expert panel will then convene to develop the official consensus statement. A
professional medical writer, if needed, may be retained to assist the panel in writing its
report and expedite its development
Title Format
Consensus statement should be titled as: “AGA Institute Consensus Statement [on]
[for] XYZ
Review & Approval
Final approval of the consensus statement rests with the CPQC.
18
AGA Institute Practice Recommendations Development Manual
0907
Developing AGA Institute Quality/
Performance Measures
The development of measures through the AGA Institute practice recommendations
process is one mechanism by which the need for quality measures development is
identified and executed.
Title Format
Quality/Performance Measure should be titled as: “AGA Institute Quality/
Performance Measures [on] [for] XYZ”
Structure and Organization
Measures documents will have three sections: a short introduction, the measures and
proposed coding/data collection.
— Introduction includes background and evidence upon which measures based.
— Measures include defined metrics (numerators and denominators) with inclusion
and exclusion criteria.
— Proposed coding includes data elements necessary for establishing the numerator
and denominator.
External Review
The final draft of measures should be released for a 30-day public comment period.
This will meet the requirements of measure endorsement entities should the AGA
Institute decide to seek endorsement of a measure or set of measures.
AGA Institute Practice Recommendations Development Manual
0907
19
Administrative Procedures
A letter confirming deadlines and other administrative requirements will be sent to the
author(s) by staff at the AGA Institute National Office. For record keeping purposes, the
author(s) will sign and return their agreements upon receipt of the materials. Paper
deadlines are closely monitored and authors must meet them in order to adhere to strict
publication deadlines.
Expenses
AGA Institute will cover the following expenses associated with producing a technical
review and the corresponding guideline:
— Face-to-face meetings.
— Teleconferencing and long-distance phone charges.
Payment of indirect (overhead) expenses is not permitted. Expenditures for items other
than those listed above must be approved in advance by the AGA Institute National
Office.
20
AGA Institute Practice Recommendations Development Manual
0907
Manuscript Technical Specifications and
Procedures
Compliance with the following manuscript specifications will facilitate processing and
publication of technical review and clinical practice guideline manuscripts.
1. Authors of practice recommendations should provide the AGA Institute National
Office with both a double-spaced hard copy and an electronic version in Microsoft
Word (Windows 97-2003 version).
2. Each line on each page of the manuscript should be consecutively numbered along
the margin.
3. A header should be placed in the upper right corner of the document and include an
abbreviated title of the manuscript, the date and page number.
4. When practice recommendations receive the requisite final approval(s), it will be
submitted to the editor of Gastroenterology for publication. Therefore, the
publication requirements of the journal regarding reference style, margins, etc.
should be followed. A copy of the journal’s manuscript requirements is included in
the appendix of this manual. The AGA Institute National Office will coordinate this
process.
AGA Institute Practice Recommendations Development Manual
0907
21
APPENDIX - A: DETAILED STEPS OF
PRACTICE RECOMMENDATIONS
DEVELOPMENT PROCESS
The process by which AGA Institute clinical practice guidelines are developed is
shown below. The time interval from the decision to prepare a technical review to
approval by the Board of Governors of the final corresponding clinical practice guideline
will be approximately 12 months.
Step 1.
The Clinical Practice & Quality Management Committee (CPQC), AGA
Institute Council, Governing Board and/or AGA Institute member identifies
topic(s) to be developed within the coming year.
A formal “call for topics” is made to AGA members with list of criteria, in July of each
year. This call for topics is published in AGA eDigest, and/or other AGA Institute
electronic or print publications. The CPQC prioritizes and ranks the list of topics and
finalizes the list based on the following equally weighted criteria: strength of evidence;
importance in clinical practice, importance in payment and to payers; importance in P4P
(potential for development of performance measures); variation in care; and prevalence
and cost of disease.
Step 2.
The CPQC approves final topics, determines scope of topic(s) and objectives for
a technical review. AGA Institute Council Chair via the appropriate Council
section identifies and secures authors and external reviewers.
The CPQC has the final approval of topic(s), determines the scope of topic(s) and
objectives for the technical review. When a new practice recommendation is to be
developed, the CPQC will create a written description of its assessment objectives and
general content to guide prospective authors.
Step 3.
Technical Review Panel (TRP) is formed consisting of the authors, member of
CPQC, and member of appropriate Council section. TRP drafts technical review.
After authors on the TRP are secured, they will work with the Director of Evidence Based
Medicine to collate, synthesize, interpret and evaluate evidence to support writing of
technical review (and quality indicators and/or measures). The TRP will then draft the
technical review. Since 2006, the AGA has offered CME credits related to each technical
review published. The TRP prepares 7 questions and answers for online CME credits.
Those CME questions and answers are reviewed along with the technical review.
22
AGA Institute Practice Recommendations Development Manual
0907
Step 4.
TRP determines what type of practice recommendation to produce, based on
the quality of evidence in technical review: consensus statement, guideline, or
measure(s) (for use in pay for performance or other quality improvement efforts).
The TRP determines what type of statement to produce, based on the quality of the
evidence. A consensus statement is based on an area of controversy, identification of reasonable
practice or identification of future areas of study, where less evidence available. A guideline is a
statement written when stronger evidence available. The statement written may also include
measures for use in P4P or other quality-improvement efforts.
Step 5.
A Medical Position Panel (MPP) convenes to write the official practice
recommendation.
The MPP should consist of no more than 8-9 members, to include the 3-4 members of the
TRP, a community-based GI, primary care physician, a payer, a surgeon (if applicable), a
patient/patient advocate and a gastroenterologist with expertise in health services research.
The MPP holds one face-to-face meeting, and teleconferences, as needed.
The TRP members present the literature to the larger MPP Panel. The MPP will
categorize the strength of recommendations and quality of evidence using the U.S.
Preventive Services Task Force (USPSTF) Ratings to support both the AGA Institute
guideline and quality efforts. The MPP will consider the impact of the statement (or
guideline) on quality of care in the practice of gastroenterology.
The MPP will propose quality indicators or performance measures for 1) guideline
compliance if the statement is a guideline, or 2) quality improvement for other types of
statements. The format of the statement should include each recommendation followed by a
summary paragraph describing the quality of the evidence and the global USPSTF score for
each recommendation.
When the statement is published, each recommendation should be explicitly rated as to
whether or not it would be suitable for the development of P4P measures.
Step 6.
Final draft of practice recommendation and technical review sent to external
reviewers for review and comment. MPP responds to external review comments.
All Clinical Practice Guidelines and Technical Reviews are submitted in draft form to
expert reviewers. Reviewers are chosen based on their recognized expertise and
knowledge of the subject of the position paper or of particular aspects of it. Collectively,
reviewers will be representative of the clinical practice and clinical research communities
and of gastroenterology and primary care medicine.
Reviewers will be asked to consider how well the draft or sections thereof answers
questions, objectives and concerns brought up in the letter of invitation. In general, their
answers to the following will be solicited:
AGA Institute Practice Recommendations Development Manual
0907
23
1. Have the authors clearly stated the objectives of the paper?
2. Have the topic and type of patient or clinical situations to which it
applies been clearly defined?
3. Have the authors included all relevant references in their review?
Have they included any references that should be removed?
4. Have the criteria for selecting/rejecting references been given?
5. Have the cited studies and their respective results/conclusions been
accurately represented?
6. Is the paper balanced and objective? Are there any apparent biases
regarding selection of studies discussed? Have the relevant risks
(including cost) and benefits of any recommended procedure/therapy
been presented adequately?
7. Have the authors identified the extent to which their various
recommendations are based on scientific data or expert opinion?
8. If analytic measures such as meta-analysis were used, were they used
correctly?
9. Have the authors identified the outcomes that can be expected from
following the guidelines?
10.Are the indicators for various diagnostic and therapeutic interventions
clearly defined?
11.Are specific recommendations given when an intervention is
superfluous, when optional and when mandatory?
12.Is the sequence in which various measures are to be taken sound,
practical and economical?
13.Is adequate space given to alternative approaches?
14.Did the authors identify issues in need of further study or consensus
development?
15.Have the authors stated whether or not a recommendation will be
suitable for development of quality indicators or performance
measures?
Step 7.
“Final” draft of practice recommendation and technical review is sent to CPQC.
The CPQC members on TRP present papers to full committee for review.
Necessary final edits are made prior to submission to the Governing Board.
The CPQC members serving on the TRP will present the document to the full
Committee either via e-mail, teleconference or during a Committee meeting for the
CPQC’s thumbs up/down review. Prior to submission to the Governing Board, the
manuscript will be sent to a copy editor to be made as concise as possible without losing
meaning.
24
AGA Institute Practice Recommendations Development Manual
0907
Step 8.
Governing Board reviews and approves practice recommendation (and
corresponding technical review) for publication in Gastroenterology.
The approved versions of the guideline and technical review will be transmitted to the
editors of AGA’s journal for publication. The guideline and the technical review will be
published in Gastroenterology.
The AGA Institute conducts a web-based discussion board regarding newly published
guidelines and technical reviews during the month they’re published in Gastroenterology.
The discussion board helps to alert AGA members about the new paper and also provide
readers a chance to ask the authors direct questions about the paper through a secure
channel.
Staff will monitor the discussion board. However, the lead author (or designated
coauthor) will be expected to log onto the message board a few times a week for about
one month to check for and answer any questions about the guideline and technical
review. Login information will be provided to authors.
In cooperation with our journals’publisher Elsevier, AGA Institute will make guidelines
and technical reviews which have been approved by our Governing Board available to
members ahead of publication. Once a guideline has been approved by the Governing
Board it will be placed in a section of our journals’ websites (www.gastrojournal.org;
www.cghjournal.org) known as Articles in Press. It will be a PDF file of the unedited
manuscript that has been accepted for publication. The manuscript will undergo
copyediting, typesetting, and review of the resulting proof before it is published in its
final form. Once this process is complete and the paper has been reviewed by the author,
it will then replace the unedited version under Articles in Press. This final version will
then be indexed in Pubmed ahead of print. To access content housed under the Articles
in Press section, a username and password is necessary. Please note that during the
production process errors may be discovered which could affect the content, and all legal
disclaimers that apply to the journal pertain.
Reprints of AGA Clinical Practice Guidelines and Technical Reviews will be produced
and made available at the AGA National Office as well as on the AGA Web site at
www.gastro.org.
AGA Institute Practice Recommendations Development Manual
0907
25
APPENDIX - B: PROCESS FOR
REVIEW OF EXISTING GUIDELINES
Step 1.
Annually, the Clinical Practice and Quality Performance Committee (CPQC) will
screen all current AGA Institute guidelines (formerly medical position statements) and
will rank the topics to determine if they are outdated. If they are determined to be
outdated, then committee members will need to determine if they still impact the scope
of the GI practice. If not, they should be withdrawn or updated.
Step 2.
The Committee determines topics requiring revision based on (1) importance to
clinical practice, (2) change in variation of care, and (3) prevalence and cost of disease.
Staff will tally the information and present it to the committee during their September
meeting to discuss and finalize the revision schedule.
Step 3.
The Committee will determine the appropriate number of topics to be revised under
the new practice recommendation development process. The number will be determined
by the rankings of the topics based upon the criteria on the ranking form. All topics will
be reviewed annually.
Each year the CPQC will be presented with a revised list of topics to once again rate
and rank and discuss during their September committee meeting.
26
AGA Institute Practice Recommendations Development Manual
0907
APPENDIX - C: GASTROENTEROLOGY
MANUSCRIPT INSTRUCTIONS
Gastroenterology publishes clinical and basic studies of all aspects of the digestive
system, including the liver and pancreas, as well as nutrition. The types of articles
Gastroenterology publishes include original papers, review articles, case reports, and
special category manuscripts. Manuscripts must be prepared in accordance with the
“Uniform Requirements for Manuscripts Submitted to Biomedical Journals” developed
by the International Committee of Medical Journal Editors (N Engl J Med
1991;324:424-428).
Gastroenterology has a total circulation of approximately 16,000 — about 11,500 in the
United States and 4500 in other countries. Forty-five percent of subscribers are AGA
members. In the United States, about 75% of subscribers are physicians, and about 25%
of subscriptions go to residents, medical schools, and libraries. More than half of the
papers published originate overseas.
Gastroenterology is abstracted and indexed in Biological Abstracts, CABS, Chemical
Abstracts, Current Contents, Excerpta Media, Index Medicus, Nutrition Abstracts, and
Science Citation Index.
Ethical and Legal Considerations — Ethics
Gastroenterology strongly discourages the submission of more than one article dealing
with related aspects of the same study. In almost all cases, a single study is best reported
in a single paper.
The Journal editors consider research/publication misconduct to be a serious breach of
ethics and will take action as necessary to address such misconduct, which includes
submission or publication of information that:
1. Is intentionally erroneous,
2. Has been published elsewhere by a different author without
acknowledgment (plagiarism),
3. Has been published elsewhere by the same author without
acknowledgment (duplicate publication), or
4. Is subsequently published elsewhere by the same author without
acknowledgment, attribution, or permission from the AGA Institute, as
holder of the copyright, to reprint or adapt the material.
Each author who submits a manuscript must complete the Gastroenterology Copyright
Assignment, Authorship Responsibility, NIH Funding, Financial Disclosure, and
Institutional Review Board/Animal Care Committee Approval Form, thereby
affirming that:
1. None of the material in the manuscript is included in another
manuscript, has been published previously, or is currently under
consideration for publication elsewhere. This includes symposia
proceedings, transactions, books, articles published by invitation, and
preliminary publications of any kind except an abstract of less than 400
words. If there is any potential overlap with a manuscript previously
published by the authors, the related manuscripts must be included for
AGA Institute Practice Recommendations Development Manual
0907
27
editorial evaluation.
2. Only people who contributed to the intellectual content, the analysis of
data, and the writing of the manuscript are listed as authors and that all
authors take public responsibility for the research results being
reported.
3. Ethical guidelines were followed by the investigator in studies on
humans or animals and described in the paper. The approval of the
institutional review board of animal care committee must be cited in
the Methods section of the text.
Breaches in these standards may result in proscribed submission for all authors of the
concerned manuscript and, when appropriate, notification of the authors’ institutions.
All authors are fully responsible for the content of the manuscript.
The publication of abstracts is not considered duplicate publication but should be
disclosed in the cover letter accompanying the manuscript submission.
Disclosure Policy
I. Authors (Original Articles and Review Articles)
The following information must be included on the title page of submitted
manuscripts:
— A list for all authors that discloses any financial arrangement (e.g.,
consultancies, stock ownership, equity interests, patent-licensing
arrangements, research support, major honoraria, etc.) they may have
with a company whose product figures prominently in the submitted
manuscript or with a company making a competing product.
Interactions that occur from the start of the research activity in the
specific program until the time when the paper is anticipated to be
published or one year from submission date, whichever is longer, are
pertinent. In the absence of any conflict of interest, authors must make
the statement that there is no conflict to disclose.
— A statement of all funding sources supporting the work and all
institutional or corporate affiliations.
— Individuals who provided writing assistance for the manuscript and the
funding source for this assistance must be disclosed.
— Investigators must disclose potential conflicts to study participants and
must state whether they have done so.
All phase II or III trials pertaining to a commercial product (pharmaceutical or device)
require:
— A statement that the statistical analysis of the entire data sets
pertaining to efficacy (specifically primary and major secondary
efficacy endpoints) and safety (specifically, serious adverse events as
defined in federal guidelines) have been independently confirmed by a
biostatistician who is not employed by the corporate entity (the name
of the biostatistician must be provided); and
— A statement from the corresponding author that he or she had full
access to all of the data and takes full responsibility for the veracity of
the data and analysis. All authors for each manuscript will also need to
28
AGA Institute Practice Recommendations Development Manual
0907
complete and submit to the Journal’s editorial office the Authorship
Responsibility portion of the Copyright Assignment, Authorship
Responsibility, NIH Funding, Financial Disclosure, Institutional
Review Board/Animal Care Committee Approval, and Sponsorship
form upon submission of a manuscript.
Authors must describe the role of the study sponsor(s), if any, in the study design; in
the collection, analysis, and interpretation of data; in the writing of the report; and in the
decision to submit the report for publication.
II. Reviewers and Editorialists
Reviewers and editorialists will be disqualified from reviewing or from writing an
editorial if they:
— have had an ongoing collaboration, original publications or grants
with the authors within the previous two years, except in the case of
being a part of a multi-center group from a different site; or
— are from the same institution as the authors.
Reviewers and editorialists should disclose at the time of submission of their review or
editorial any financial arrangement (e.g., consultancies, stock ownership, equity interests,
patent-licensing arrangements, research support, major honoraria, etc.) they may have
with a company whose product figures prominently in the submitted manuscript or with
a company making a competing product. Interactions that occur from the start of the
research activity in the specific program until the time when the paper is anticipated to
be published or one year from submission date, whichever is longer, are pertinent.
Authorship
Each author must have participated sufficiently in the work to take public
responsibility for the content of the paper and must approve of the final version of the
manuscript. Authorship should be based on substantive contributions to each of the
following:
— conception and design of the study;
— generation, collection, assembly, analysis and/or interpretation of data;
— drafting or revision of the manuscript;
— approval of the final version of the manuscript.
National Institutes of Health (NIH) Funding
If a manuscript is accepted for publication and was supported in part, or in whole, by
the NIH, the author may request that the manuscript be automatically posted to
PubMed Central (PMC). If the author wishes to make this request, he or she should
check the appropriate box located on the Copyright Assignment Form under the NIH
Funding portion of the form. In addition, the author must provide the NIH grant
number. If an author makes this request, the manuscript will automatically be entered in
the PMC system and will be available to the public, free of charge, 12 months from the
manuscript’s publication date.
AGA Institute Practice Recommendations Development Manual
0907
29
Manuscript Preparation
Authors are asked to have their manuscript saved in Rich Text Format (.rtf) and each figure
saved as a separate electronic file, preferably named as “figure 1.tif,” “figure 2.jpg,” etc.
— Image File Formats: We support the following among dozens of file
formats: .bmp, .gif, .jpg, .pbm, .pcx, .png, .tif, .eps, and .xbm, along
with TARGA and Photoshop files. (ChemDraw, Excel, PowerPoint,
SigmaPlot, and Equation Editor formats are not supported at the
time.) Your file formats can be mixed.
— Tables: Tables should be prepared without the use of tabs; most table
editor programs can be uploaded successfully.
Arrange manuscript as follows, each component beginning on a separate page:
— 1. Title page
— 2. Abstract
— 3. Introduction
— 4. Materials and methods
— 5. Results
— 6. Discussion
— 7. References
— 8. Figures and figure legends
— 9. Tables.
Place page number and first author’s last name at top of each page.
Cite references, tables, and figures consecutively as they appear in the text.
Authors are responsible for applying for permission for both print and electronic rights
for all borrowed materials and are responsible for paying any fees related to the
applications of these permissions.
Title Page
Title:
Include animal species. Use no abbreviations. Limit: 120
characters with spaces.
Short Title:
Limit: 45 characters.
Authors:
Include first names of all authors and name and full location of
department and institution where work was performed.
Grant Support: List grant support and other assistance.
Abbreviations:
List alphabetically abbreviations not mentioned in the Style
Guide, which follows the Instructions to Authors. (Note: In
general, the use of abbreviations is discouraged.)
Correspondence: Provide name, complete address, e-mail address, telephone
number, and fax number of corresponding author.
30
AGA Institute Practice Recommendations Development Manual
0907
Abstract
Limit: 250 words. Organize according to the following headings: Background & Aims,
Methods, Results, and Conclusions.
Do not use abbreviations, footnotes, or references.
Authors of clinical research studies should submit a structured abstract of no more than
250 words organized into the following categories as applicable:
— Background & Aims: Describe the importance of the study and the
precise research objective(s) or study question(s).
— Methods: Methods should include information on the following
aspects of study design when applicable. The methods section may
employ subheadings at the discretion of the author.
— Design: describe the basic study design, e.g., randomized
controlled trial, cross sectional study, cohort study, case series,
survey, etc. Source of all non-standard reagents need to be
explicitly stated.
— Setting: specify whether the study was conducted in a primary or
tertiary care setting, in an ambulatory care clinic or hospital, in
the general community, etc.
— Participants: indicate the number of study subjects and how they
were selected, recruited, and assigned to the intervention
— Intervention: report the method of administration and duration
of the intervention.
— Results: Provide the main outcomes of the study including confidence
intervals or P values. Report the absolute values and risk differences so
that readers can determine the absolute, as well as the relative, impact
of the results.
— Conclusions: State only conclusions that are directly supported by
the evidence and the implications of the findings.
Body of Paper
Describe ethical guidelines followed (for human or animal studies); cite approval of
institutional human research review committee or animal welfare committee; describe in
detail hazardous procedures or chemicals involved, including precautions observed.
Outline statistical methods used.
When describing the results of hypothesis testing, report P values and/or confidence
intervals; avoid using phrases such as “not significant.”
Identify drugs and chemicals used by generic name (if trademarks are mentioned,
manufacturer name and city are given).
References
Cite references in order of appearance in text using superscripted Arabic numerals.
Cite personal communications and unpublished data directly in text without being
numbered.
AGA Institute Practice Recommendations Development Manual
0907
31
Conform abbreviations to those used in Index Medicus.
Conform style and punctuation to Gastroenterology requirements:
— Article (list all authors):
13. Meltzer SJ, Ahnen DJ, Battifour H, Yokokota J, Cline MJ.
Protooncogene abnormalities in colon cancers and adenomatous polyps.
Gastroenterology 1987;92:1174-1180.
— Book:
18. Day RA. How to write and publish a scientific paper. Philadelphia:
Institute for Scientific Information, 1979.
— Article in Book:
22. Costa M, Furness JB, Llewellyn-Smith IF. Histochemistry of the
enteric nervous system. In: Johnson LR, ed. Physiology of the
gastrointestinal tract. Volume 1. 2nd ed. New York: Raven, 1987:1-40.
Tables
Tables should be prepared without the use of tabs; most table editor programs can be
uploaded successfully.
Figures
Images:
Images can be clinical, pathologic (gross or microscopic),
endoscopic, or radiographic. They should be of high quality (300
dpi or greater, clear, and in good focus) and illustrate well the
diagnosis.
Photographs:
Photographs of identifiable patients must be accompanied by
written permission to publish from the patient.
Line art & graphs: Graphs, charts, and other line art will be redrawn by our Medical
Illustration Department for consistency with the overall style of
the AGA Institute journals. Please be sure that any graphs or line
art you submit is at a resolution of at least 150 dpi so that they are
readable to reviewers.
Figure legends:
Accepted figure
file formats:
Preferred figure
file formats:
32
Please do not embed or flatten the text into the image files.
Figure legends should be typed and submitted in .rtf (rich text
format). This text will be reformatted in the style of the AGA
Institute journals.
We support the following among dozens of file formats: .bmp,
.gif, .jpg, .pbm, .pcx, .png, .tif, .eps, .xbm, .psd, and .tga files.
When sending image files, please do not embed them in Word or
PowerPoint. You may submit mixed file formats (image1.jpg,
image2.tif, image3.eps, etc.).
.tiff and .psd. If you have created Photoshop image files
containing separate layers with arrows or text, please send us the
layered files (unflattened).
AGA Institute Practice Recommendations Development Manual
0907
Image file formats
not supported
at this time:
ChemDraw, CorelDraw, Canvas, FreeHand, Excel, PowerPoint,
SigmaPlot, and Equation Editor. You may export image files from
these programs as PDF, Jpeg, or other acceptable file formats.
File naming
convention:
Figures should be named consecutively such as “figure 1.tif,”
“figure 2.jpg,” etc., with the file extension appended (.tif, .jpg,
.eps, etc). Each figure should be saved as a separate electronic file.
Color Space:
Color files should be submitted in the CMYK color space.
Authors are encouraged to present color figures in a manner that
will allow the data to be interpreted by colorblind readers.
Gastroenterology suggests that authors present dual-labeled images
in green and magenta rather than in green and red. See the
website of the Jfly data depository for Drosphila researchers
(http://jfly.iam.u-tokyo.ac.jp/color/ ) for more information on
how to make figures and presentations intelligible for a
colorblind audience.
Font:
If your figures include text, an 8 to 10 point font should be used.
Acceptable fonts are “sans serif” fonts such as Arial, Helvetica,
and Myriad. Examples of unacceptable fonts (“serif” fonts) are
Times, Palatino, and Georgia. Lettering should begin with an
upper case letter, followed by lower case lettering.
Multiple panel
figures:
Do not mount multiple part figures. Please submit each panel
(image) separately. However, you may submit a multiple panel
version to suggest the order in which you would like the panels
arranged. You may also include a written, suggested layout. Each
individual panel should be of the highest possible quality (300 dpi
or higher) at actual print size.
Copyediting
Manuscripts are copyedited to make them consistent with Journal style; if a particular
section in the manuscript is not clear or requires additional information, the copy editor
will direct questions to the author. These questions, or “author queries,” will appear in
the margins of the proofs that are sent to the author.
AGA Institute Practice Recommendations Development Manual
0907
33
APPENDIX - D: AGA INSTITUTE
CME QUESTIONS AND LEARNING
OBJECTIVES WRITING GUIDELINES
Please refer to Constructing Written Test Questions for the Basic and Clinical
Sciences, Third Edition provided by the National Board of Medical Examiners (NBME)
as your guide for item writing. This guide can be reviewed at
www.nbme.org/about/itemwriting.asp.
Item Writing
All questions are to be original questions written in multiple-choice format. They
consist of the stem (question) and lead-in (instructing the examinee on how to approach
the answer, e.g., “Which of the following is the most likely diagnosis” or “Which of the
following is the most likely explanation for the findings”). Below are some quick tips
based upon information provided from NBME’s publication on item writing.
— Write using “one best answer” format; examinees select the single best
response
— Provide 5 answers, A – E (1 correct, 4 incorrect distractors); do not
include “none of the above,” “all of the above” as distractors
— Avoid negative formats (e.g., “Each is correct EXCEPT”, “Which is
the LEAST LIKELY”, “Which one of the following is NOT correct).
— Avoid True or False items
— Items must be written at the 2nd-year trainee level
— Each stem should not exceed 5 lines
— Distractors must be plausible and written consistently with other
answer choices; no “trick” answers
— Eliminate extraneous commentary about the patient, unless it impacts
diagnosis
— Eliminate reference to race or ethnicity, unless it impacts diagnosis
— Use “male” or “female” terms when referring to gender
— Avoid ambiguity. The item must be written to clearly express the
intent of the item objective. Also, avoid over using abbreviations.
— Be specific with the lead-in. Use precise terms to indicate what action
is required, e.g., “which of the following ….” or “what is the most …,”
etc. See page 40 of the Item Writing Guide.
— Avoid cueing, e.g., leading the student to the answer, e.g.,
— by including the answer within the question
— providing implausible distractors, or
— through grammatical inconsistencies in the stem and distractors.
See page 41 of the Item Writing Guide.
— Use of Case Clusters (as referenced on Page 43 of the Item Writing
34
AGA Institute Practice Recommendations Development Manual
0907
Guide). These are questions that present a scenario on which two or
more questions are asked as follow-up. Take care to construct these
items. As opposed to “stand-alone” items, case clusters are typically
“all-or-nothing” items in terms of scoring. If an examinee lacks the
skill set or knowledge required, they will likely answer all the items
incorrectly that are related to the patient care scenario. Where
possible, you may elect to rework the case cluster to determine if the
individual items can be reworded as a single question, but still meet
the author’s objective.
Item Objective
Briefly identify your goal for the item. What knowledge or skill does the item try to
assess? This will help us identify whether the item is constructed effectively so that the
examinee has the best opportunity to correctly answer the question.
Item Classification
The following information is critical in helping us ensure the reliability and validity of
the items. This information will also assist us in providing analytical feedback to
examinees and training programs.
Primary Category (according to ABIM Gastroenterology Blueprint)
— Identify the primary category or content area as: Liver, Colon,
Stomach and duodenum, Esophagus, Pancreas, Small Intestine, Biliary
tract, General
Cognitive Mechanism (2)
— Knowledge (recall of basic facts or terms)
— Application (problem resolution, critical thinking)
Rationale
— For purposes of validity, briefly justify your choice of the correct answer
Documentation
— Identify an authoritative reference for the material (e.g., a
gastroenterology textbook, the Core Curriculum, a journal article.
Chapter and page citations are necessary. This information assists in
ensuring validity (the items are appropriate to the exam and that the
answers are justified through published material).
AGA Institute Practice Rcommendations Development Manual
0907
35
WRITING A TEST ITEM
The following is an example of how to write a test question for journal CME exams.
Stem
A previously healthy 32-year-old athlete is brought to the emergency
room because of melena and a fainting episode. His blood pressure is
found to be 70mm Hg palpable. The emergency room physician calls
and asks you to immediately perform endoscopy.
Lead Line
What is the best response at this point?
Response Options A. Ask the gastroenterology nurse to meet you immediately in the
emergency room.
one correct
response and
B. Start hemodynamic support immediately and pass a nasogastric
four distracters
tube to assess for evidence of bleeding.
C. Place a nasogastric tube and if the return is clear, schedule the
patient for EGD next week.
D. Start high-dose intravenous H2 blockers and give 30cc of
antacids via the nasogastric tube.
E. Consult with the ER physician.
Rationale
(reference
numbers keyed
to chapter
reference list)
References
(will not appear
with test item)
When gastrointestinal bleeding is suspected, the most important
step is to assess the severity of the bleed.1 This can be best gauged
by the overall hemodynamic status of the patient. Nasogastric
tube aspirate can further identify patients with increased risk of
rebleeding or mortality. However, nasogastric tube aspirate is not
accurate in identifying patients who have stopped bleeding or are
actively bleeding. Hemodynamic support should take precedence
rather than diagnostic procedures. In a patient with a major
gastrointestinal hemorrhage, as identified by clinical criteria,
endoscopy should be performed promptly when the
hemodynamic status is stable.2
1. Cuellar, RE, Gavaler GS, Alexander GA et al. Gastrointestinal
hemorrhage: the value of a nasogastric aspirate. Arch Intern Med
1990, 150:1381-1412.
2. National Institutes of Health Consensus Development
conference. Therapeutic endoscopy and bleeding ulcers. JAMA
1989; 262:1369-1372.
A test item consists of a question, a set of responses, a critique, and references. The
items are referable to the corresponding article. Answers should be in a multiple-choice
question format throughout. In this format, one best answer is to be selected from a list
of five options. The options are one correct answer and four distracters. The critique
explains the correct option, and why the other options are not correct, citing any
appropriate references drawn from the article reference list.
36
AGA Institute Practice Rcommendations Development Manual
0907
The Question
Formulating an educational objective for a question is the best way to get started.
Think of an objective before writing each item. Consider what the participant will be
able to recognize, conclude, identify, etc.
After the objective of a question is clear, devise the stem of the question. This can be a
vignette, a statement of clinical background, or other information to provide a setting for
the question. The stem may convey patient history, examination findings, test results, or
other relevant data. Avoid reference to race or ethnicity unless it impacts diagnosis and
use male or female terms when referring to gender. It does not need to be long and
complex – too much information can lead to a loss of focus. Not all stems need to be cast
in the form of clinical vignettes. Instead, simple, concise statements concerning a single
area of knowledge can be effective learning vehicles.
The Lead Line
The actual question is stated in a lead line. The question must be specific, and in a
form that can be answerable without the need to look at the response options. Avoid lead
lines based solely on factual recall. Avoid vague terms such as rarely, occasionally,
frequently, etc. Also avoid negative lead lines such as, “What is least likely,” “All options
are appropriate except,” or “An outcome never seen is.” Do not include true or false
items. Here are some examples of lead lines.
— What is the most likely explanation of these findings?
— The most likely cause of this patient’s pain is:
— What is the most likely complication of this patient’s condition?
— This condition is most likely to progress to:
— Given the test results, what is the most likely additional finding?
— What is the most appropriate next step in establishing a diagnosis?
— Which laboratory study is most likely to confirm the diagnosis?
— What is the first priority in caring for this patient?
— What is the most appropriate next step?
— What is the most effective management strategy?
Response Options
Five lettered response options (A-E) follow the question stem. Only one option must
clearly be the correct answer to the lead-in question. The incorrect options, known as
distracters, must be plausible choices. Distracters may be partially correct, addressing
common forms of mismanagement, but one answer must clearly be best. Distracters should
not include “None of the above” or “All of the above”. Good response options have:
1. Plausibility. All distracters should be a plausible answer to the
question.
2. Authority. The correct options must be unambiguously superior to all
other options.
3. Homogeneity. All options should be homogeneous, consisting of the
same type of laboratory findings, diagnosis, treatment protocols, etc.
AGA Institute Practice Rcommendations Development Manual
0907
37
4. Comparability. All options should have the same grammatical structure,
use consistent language and be of similar length and complexity.
In devising options, avoid mixing and clueing. Mixing may distract participants by
raising issues not relates to the question, for example mixing testing and treatment
alternatives in one set of options. Clueing may hint at or tip off the correct option, for
example, using the correct response of one question in the next question stem.
Correct Answer and Rationale
The rationale presents the correct answer to the question and explains why each
distracter is not correct. It functions as a teaching tool, reflecting the educational
objective of the question, substantiating the correct response, and expanding on the topic
as presented in the article chapter. The rationale may be structured
— with the correct answer first, followed by discussion of the distracters;
— with discussion of the distracters first, followed by explanation of the
correct answer; or
— by following the sequence of the response options.
Sometimes a single paragraph suffices to address all the options. However, an opening
paragraph may be useful to identify the problem and set the context for discussion. Also,
the relatedness of content may dictate additional paragraphing.
References
The rationale should be supported by relevant references (approximately 3-6).
Tips for Writing Learning Objectives
Learning Objectives: You should include 3 learning objectives. These should appear
as follows:
At the end of this activity, the successful learner should:
1.Learning objective
2.Learning objective
3.Learning objective
Suggested approach to developing specific learning objectives: Connect the
statement above with a clear word or phrase which communicates the performance by
the learning. For example: assess, compare, diagnose, differentiate, distinguish, establish,
demonstrate, identify, interpret, translate, analyze, explain, integrate, formulate, evaluate.
— The following words and phrases are NOT appropriate because they
are open to many interpretations: know, understand, appreciate,
believe, value, apply scientific knowledge to, develop a knowledge of).
Finish with the specifics of what the learner will be doing when demonstrating
achievement or mastery of the objective. The entire objective is the intended outcome or
results of the instruction.
38
AGA Institute Practice Rcommendations Development Manual
0907
APPENDIX - E: COPYRIGHT ASSIGNMENT,
AUTHORSHIP RESPONSIBILITY, NIH FUNDING,
FINANCIAL DISCLOSURE, INSTITUTIONAL REVIEW
BOARD/ANIMAL CARE COMMITTEE APPROVAL,
AND SPONSORSHIP FORM
Copyright Assignment. In consideration of the American Gastroenterological Association (AGA) Institute (the
“AGA Institute”) taking action to review and credit the below-identified submission (the “Manuscript”), and for
other valuable consideration, the receipt and sufficiency of which is hereby acknowledged, the undersigned
authors and/or creators (the “Authors”), jointly and severally, hereby transfer, convey, and assign to the AGA
Institute, free and clear of any liens, licenses or encumbrances, the entire right, title, and interest in and to the
Manuscript throughout the world, including without limitation in and to any and all copyrights for the
Manuscript (including but not limited to rights to copy, publish, excerpt, collect royalties and make derivative
works) in print, electronic, Internet, broadcast, and all other forms and media now or hereafter known, and for
any and all causes of action heretofore accrued in Authors’ favor for infringement of the aforesaid copyrights, to
have and to hold the same unto the AGA Institute, its successors and assigns, for and during the existence of the
aforesaid copyrights, and all renewals and extensions thereof. At any time and from time to time hereafter, the
Authors shall upon the AGA Institute’s written request take any and all steps and execute, acknowledge and
deliver to the AGA Institute any and all further instruments and assurances necessary or expedient in order to
vest the aforesaid copyrights and causes of action more effectively in the AGA Institute. The Authors retain the
nonexclusive permission to use all or part of the Manuscript in future works of their own in a noncompeting way,
provided proper copyright credit is given to the AGA Institute. Should the AGA Institute finally determine that
it will not publish the Manuscript, the AGA Institute agrees to assign its rights therein back to the Authors.
(Note: material prepared by employees of the federal government in the course of their official duties may not
be copyrightable.)
Authorship Responsibility. I, the undersigned Author, certify that I have participated sufficiently in the
intellectual content, the analysis of data, if applicable, and the writing of the Manuscript, to take public
responsibility for it. I have reviewed the final version of the Manuscript, believe it represents valid work, and
approve it for publication. As an Author of this Manuscript, I certify that, except to the extent expressly credited
to others in the text of the Manuscript: the entire Manuscript is an original creation of the Authors; and none of
the material in the Manuscript has been published previously, is included in another manuscript, or is currently
under consideration for publication elsewhere. I also certify that this Manuscript has not been accepted for
publication elsewhere, and that I have not assigned, licensed, or otherwise transferred any right or interest in the
Manuscript to anyone. Moreover, should the AGA Institute or the editors of GASTROENTEROLOGY request
the data upon which the Manuscript is based, I shall produce it. Authors are responsible for applying for
permission for both print and electronic rights for all borrowed materials and are responsible for paying any fees
related to the applications of these permissions.
Institutional Review Board/Animal Care Committee Approval. I, the undersigned Author, certify that my
institution has approved the protocol for any investigation involving humans or animals and that all
experimentation was conducted in conformity with ethical and humane principles of research.
Sponsorship. I, the undersigned author, certify that I had full access to all of the data in this study and I take
responsibility for the integrity of the data and the accuracy of the data analysis.
Continued on next page
AGA Institute Practice Recommendations Development Manual
0907
39
Manuscript title: ___________________________________________________________________________________
Signatures: Each Author must sign and date this statement and assignment. In the case of a work made for hire, the
employer(s) must also sign. For example, for any Manuscript including any portion created in the course of employment
for another (whether as a regular employee or as an independent contractor) requires the signature of the relevant
employer(s).
____________________________________________________________
Signature
____________________________________________________________
Signature
____________________________________________ _______________
Print Name
Date
____________________________________________ _______________
Print Name
Date
____________________________________________________________
Signature
____________________________________________________________
Signature
____________________________________________ _______________
Print Name
Date
____________________________________________ _______________
Print Name
Date
Employer signature(s) as Author (required for works made for hire):
Employer signature(s) as Author (required for works made for hire):
____________________________________________________________
Employer
____________________________________________________________
Employer
____________________________________________________________
By (Signature)
____________________________________________________________
By (Signature)
____________________________________________ _______________
Print Name
Date
____________________________________________ _______________
Print Name
Date
If this Manuscript exists in the public domain because it was written as part of the official duties of the Authors as employees of the U.S. government, check this box.
National Institutes of Health (NIH) Funding. My manuscript was supported in part, or in whole, by the NIH. In
accordance with the NIH Public Access Policy, I would like to request that my manuscript, should it be accepted for
publication, be submitted to PubMed Central (PMC). I understand that my manuscript will therefore be freely accessible
by the public via PMC twelve months from the date of its publication.
I would like my manuscript automatically submitted to PMC, should it ultimately be accepted. My NIH grant number is __________________________________.
Financial Disclosure. Check the appropriate box. Sign where indicated. All Authors must sign one of the statements below.
I, the undersigned Author, certify that I have no financial
I, the undersigned Author, certify that I have included on the title
arrangements (e.g., consultancies, stock ownership, equity
interests, patent-licensing arrangements, research support, major
honoraria, etc.) with a company whose product figures
prominently in the submitted manuscript or with a company
making a competing product except as disclosed on a separate
attachment. All funding sources supporting the work are
acknowledged on the title page.
page of the manuscript any financial arrangements (e.g.,
consultancies, stock ownership, equity interests, patent-licensing
arrangements, research support, major honoraria, etc.) that I have
with a company whose product figures prominently in the submitted
manuscript or with a company making a competing product. All
funding sources supporting the work are acknowledged on the title
page.
____________________________________________________________
Signature
____________________________________________________________
Signature
____________________________________________________________
Signature
____________________________________________________________
Signature
____________________________________________________________
Signature
____________________________________________________________
Signature
____________________________________________________________
Signature
____________________________________________________________
Signature
40
AGA Institute Practice Recommendations Development Manual
0907