Evidence Analysis
Projects and Toolkits
Esther Myers, PhD, RD, FADA
Academy of Nutrition and Dietetics (AND)
Formerly called the American Dietetic Association
emyers@eatright.org
Research and Strategic Business Development Team
12/9/2012
Overview
• Introduce Academy of Nutrition and
Dietetics EAL concept
• Content in EAL ®
• Users of the EAL
•
•
•
•
Evidence Analyses process
Guideline & Toolkit Development process
Usability/feasibility testing EBNPG &Toolkit
Other products from EAL ®
2
Academy of Nutrition & Dietetics
(Formerly called American Dietetic Association)
•Largest food and nutrition professional
organization in the world
•+71,800 members
•
•
•
•
+51,000 Registered Dietitians
+1500 Dietetic Technician Registered
+17,000 students
+450 International members
•28 Dietetic Practice Groups (DPG)
•
•
Diabetes Care and Education = 6187 members
American Overseas Dietitians = 888 members
3
Academy’s Commitment
The Academy believes that research is the
foundation of the profession providing the basis
for practice, education and policy.
Castellanos, Myers, Shanklin, Research Priorities Contribute to Bright Future, JADA, 2004
4
Academy’s Evidence Analysis Library:
The best available research on nutrition topics in a
online practitioner-friendly website and includes
Evidence-Based Nutrition Practice Guidelines.
The EAL was launched in September 2004. It contains a significant amount of
analysis completed on over 42 different disease/conditions, food, nutrients and
other topic areas.
The Academy is proud of it’s rigorous process of evidence analysis which was
recognized by JCAHO as exemplary of bringing the best research to practice and
served as the template for the Nutrition Evidence Library used by the US Federal
Government for synthesizing the science to develop the 2010 US Dietary
Guidelines
5
About the Evidence Analysis Library
6
19 EvidenceBased
Nutrition
Practice
Guidelines
Year
Guideline
2005
• Disorders of Lipid Metabolism
2006
• Adult Weight Management
• Critical Illness
2007
• Pediatric Weight Management
• Oncology
2008
• Diabetes Mellitus Type 1 and 2
• Hypertension
• Heart Failure
• Chronic Obstructive Pulmonary Disease
• Gestational Diabetes
2009
• Celiac Disease
• Spinal Cord Injury
• Unintended Weight Loss
2010
• Chronic Kidney Disease
• HIV/AIDS
2011
• Vegetarian Nutrition
•Disorders of Lipid Metabolism Update
2012
• Critical Illness Update
• Food and Nutrition for Older Adults (Recommendations)
How much content is on EAL®?
Abstracted Articles/Worksheets
Number of Published Worksheets on EAL
6000
5176
5000
4450
4000
3351
3000
1008
2005
2006
2009
1356
Sep-10
Sep-11
115
Sep-12
0
2004
2004
2007
2274
2000
1000
4677
2005
2006
2007
2009 Sep-10 Sep-11 Sep-12
8
EAL ® Usage Worldwide
Users from 212 different countries
As of October 1, 2012 – 15,162,099 Page Views
Evaluating the Science
10
Universal Principles and Process
11
Application of Research
to Practice Guidelines
Two Basic Principles of
Evidence Based Medicine
(EBM) are:
• A hierarchy of strength of evidence
behind recommendations
• The judgment of the clinician when
weighing alternative management
strategies, considering patient values
and preferences, and societal values
Hierarchy of Strength of Evidence
Type and Description of Evidence
Primary Studies • Selecting studies that are both:
– Highly relevant
– Study designs minimize bias
• High strength of inference
Guyatt et al, JAMA.2000;284(10):1290-1296
Which research design?
Type of primary study preferred varies by
use:
Diagnosis
• CrossSectional
Studies
Treatment
• Randomized
Controlled
Trials
Prognosis
Etiology/Harm
• Cohort
Studies
(over time
with
outcomes)
• Cohort/Case
Control (2
groups over
time)
Steps in Evidence Based
Practice
• Identify a specific problem/area of uncertainty
• Formulate problem as question/analytic framework
• Search for and find evidence
• Evaluate the reports and grading the evidence
• Form recommendation/make decisions
• Summarize and disseminate findings
What is the Gold Standard for
Evidence Based Reviews??
16
Cochrane Reviews
Considered as gold standard by many
Protocol developed for review
• RCT (occasionally some quasi RCT)
Review includes
• Summary
• Tables of evidence
Central Database of RCTs and CCTs
• 350,000 citations searchable by
keywords
TYPICALLY NO
EPIDEMIOLOGICAL STUDIES
http://www.cochrane.org
Institute of Clinical Systems
Improvement (ICSI)
Methodology
Classes of Research Reports
Report Quality Categories
Conclusion Grades
I,II, III and Grade Not Assignable
Greer et al, Jr Comm J Qual Improv. 2000;26(12):700-712
ICSI Evidence Grading System. Available at:
http://www.icsi.org/evidence_grading_system_6/evidence_grading_system__pdf_.html.
Accessed August 2007
The Academy’s Process
19
Steps in the Evidence Analysis Process
Topic and Workgroup
Selection
Formulate Question
Gather Research
Appraise Articles
Summarize
Develop Conclusion
Statement and Grade
Guideline Development:
Develop algorithms based on
Nutrition Care Process
Draft guideline
recommendations
Internal/external review and
revise
Publish guideline on EAL®
Evidence Analysis Teams
Roles and
Responsibilities
Evidence Based Practice
Committee
Expert Work Group
Members
Project Managers
Lead Analysts
Analysts
Academy Staff
21
PICO Format
Well-built clinical questions usually contain the
following four elements:
P opulation with a specific problem
I ntervention, procedure, or approach
Comparison intervention
Outcome of interest
22
Sample
Questions
12/9/2012
23
Inclusion/Exclusion Criteria
Decisions about
•
•
•
•
•
Language
Duplicates
Abstracts
Peer-reviewed
Unpublished—Gray Literature
Within the research report
•
•
•
•
•
•
•
•
Age
Disease status
Setting
Interventions
Specific outcome measures
Sample size
Dropout rate
Study duration
24
Search Plan & Results for Each Question
•Inclusion Criteria
•Exclusion Criteria
•Databases Searched
•Search Terms
•Included Articles
•Excluded Articles
Search Plan & Results
List reason for
exclusion for
each article not
included in the
analysis; e.g.
Sample size too
small
26
Critically Appraise the Research
• Critically review the evidence- review each study
•
Worksheet developed
• Quality Criteria Checklist completed -quality rating assigned
to each study (positive, negative or neutral)
•
Based on the answers to several relevance and validity questions
Example Worksheet
•Citation / PubMed ID
•Date
•Study Design
•Class
•Rating (+/0/-)
•Research Purpose
•Inclusion Criteria
•Exclusion Criteria
•Description of Study
Protocol
•Data Collection
Summary
•Description of Actual
Data Sample
•Summary of Results
•Author Conclusion
Reviewer Comments
Classes of Research Report
PRIMARY
A RCT
B Cohort study
C Nonrandomized trial
concurrent/ historical
controls
Case control study
Population based
descriptive study
D Cross sectional, Case
series, Case report
SYTHESIZED
M Meta-analysis,
systematic review,
decision analysis, c/b
or c/e study
R Narrative review,
Consensus
statement/report
X Medical opinion
Example Quality Criteria Checklist
Questions related
to relevance
and validity
Determines if
article
Quality is
rated as:
• Positive
• Negative
• Neutral
Summary of the findings
Synthesize the studies into a narrative
evidence summary and an overview
table.
Evidence Summary - Narrative
Studies showing little or no effect of glycemic index on metabolic outcomes
Gilbertson et al, 2001 found in a positive-quality 12-month randomized controlled trial of
104 children with type 1 diabetes, that children given flexible dietary advice with an
emphasis on choosing low-GI foods consumed a diet with a mean 49% of the energy
from carbohydrate and a mean 21 g of dietary fiber, resulting in significantly lower
HbA1c levels and lower rates of hypoglycemic episodes. However, the dietary GI values
between the two groups did not differ. Gilbertson et al, 2003, in a new analysis of the
previous study, concluded that the children in the study did not have lower dietary
quality or more limited food choices than children given traditional carbohydrate
exchange advice.
In a positive-quality 8-week randomized controlled trial of subjects with type 2 diabetes,
Heilbronn et al, 2002 …….
Organized by
topic….e.g. by
direction of
findings
Summarizes
Studies showing an effect of glycemic index on metabolic outcomes
details
In a positive-quality randomized controlled trial by Frost et al, 1994,key
51 subjects
with type
2 diabetes were given either standard dietary advice based on theabout
British Diabetic
study
Association recommendations or standard dietary advice with an emphasis on low
in advice
narrative
glycemic index foods for 12 weeks. Subjects in the low glycemic index
group had
a lower calculated mean diet glycemic index intake (GI 77 vs 82), format
and a significant
correlation was detected between the glycemic index of the diet and the fall in
fructosamine (r = 0.54, p < 0.01) and fasting blood glucose (r = 0.41, p < 0.05)………
……..
Summary of Findings
. A total of fifteen studies examining the
effect of the GI on glycemic response:
• nine randomized controlled trials
• seven positive-quality
• two neutral-quality
• two cohort studies
• one positive-quality
• one neutral-quality
• two positive-quality population-based
descriptive studies
• one neutral-quality meta-analysis
• one neutral-quality narrative review.
Evidence Summary -Overview Table
Columns
customized to
question
Summarizes
key details
about study
in table
format
Conclusion Statement
Bottom Line – Answer to question based on the
science
 Determined after research analyzed
 Graded based on quality of supporting evidence
Grades I, II, III, IV, and V for Good, Fair, Limited,
Expert Opinion only, and Not Assignable.
Conclusion Grades
Grade I
Studies of strong design to answer questions
Clinically important and consistent
Adequate statistical power
Grade II
Studies of strong design to answer questions
Some uncertainty in conclusion/inconsistent results,
doubt about statistical or clinical significance or
generalizability
Separate studies of weaker design, all consistent
Conclusion Grades
(Continued)
Grade III
Strong design, substantial uncertainty/
inconsistent results/generalizability unclear
Limited studies of weaker design or small effect
size or intermediate or surrogate outcome
Grade IV
Expert opinion or consensus statement
Grade Not Assignable
Unsubstantiated by research studies
Informed medical commentators based on clinical
Conclusion statement
What is the relationship between glycemic
index and metabolic outcomes in persons
with type 1 and type 2 diabetes?
Conclusion
Fifteen short-term studies ranging from two to 12
weeks and one longer term (one year) studies
report mixed effects on A1C levels. These studies
are complicated by differing definitions of "high
GI" and "low GI" diets or quartiles, as well as
possible confounding dietary factors.
Grade II
Question, Conclusion Statement and Grade
Link to Evidence Summaries
Link to Search
Plan
Evidence Based Nutrition Practice
Guidelines
40
What are Evidence-Based Nutrition Practice
Guidelines?
Evidence Summaries &
Conclusion Statements =
what the evidence says
Guideline = course of action
for the practitioner based
on the evidence
Steps in the Evidence Analysis Process
Analysis:
Guideline Development:
Formulate Question
Develop algorithms based on
Nutrition Care Process
Gather Research
Appraise Articles
Draft guideline
recommendations
Summarize
Internal/external review and
revise
Develop Conclusion
Statement and Grade
Approval By Oversight and
Publish guideline on EAL®
Features of Guidelines
• Executive Summary of Recommendations: list
only of recommendations, no supporting evidence
• Introduction: scope, intent, methods,
benefits/harms
• Recommendations: a series of guiding
statements that propose a course of action for
practitioners
• Algorithms: step-by-step flowchart for treatment
of the specific disease/condition
Executive Summary of Recommendations
Free on EAL
Published on National
Guidelines Clearinghouse
12/9/2012
45
Executive Summary of Recommendations
Recommendation
Rating
Label (conditional or
imperative)
12/9/2012
46
Major Recommendations
12/9/2012
47
18 Major Recommendations
for Type 1 and 2 DM
Plus recommendations from
other conditions as
appropriate
Weight Management
Disorders of Lipid Metabolism
Hypertension
Recommendation Page
Recommendation Page:
-Recommendation
-Risks/Harms
-Conditions of Application
-Potential Costs
-Recommendation Narrative
-Strength Rationale
-Link to Supporting Evidence
-References
If the use of the glycemic index is proposed as a method
of meal planning, the RD should advise on the conflicting
evidence of effectiveness of this strategy. Studies comparing
high versus low GI diets report mixed effects on AIC
FAIR
CONDITIONAL
Ratings
Strength
•Strong – Clear benefits & strong evidence
•Fair – Benefits but not as strong evidence
•Weak – Benefits unclear
•Consensus – Experts recommend
•Insufficient Evidence
Applicability
•Imperative - applies to all target population
•Conditional - only applicable in certain
conditions…e.g. if xx…then y)
50
Recommendation Page
12/9/2012
51
12/9/2012
52
Guideline Algorithms
Nutrition Assessment Algorithm
Nutrition Intervention Algorithm
Review and Revision
Expert review
Internal to Academy Membership
External (MD, RN, Dentist)
Revise if necessary
Publish/disseminate
Available Resources
55
Toolkit development
56
Objectives of Toolkits
To assist RD in:
• Implementing evidence-based practice
• Implementing NCP/SL
• Promoting consistency
• Promoting quality care
• Achievement of expected outcomes
Toolkit Development
Develop toolkits to apply guidelines
Conduct 60-day usability test of toolkit
and revise
EBPC Review and Approval
Make toolkits available for purchase
Features of Evidence-Based Toolkits
Set of companion documents for application of the practice
guideline
• Disease/condition specific
• Include:
• MNT protocol for treatment of disease/condition
• Documentation forms (progress notes, summary notes)
• Outcomes monitoring sheets
• Client education resources
• Case studies
• Electronic downloadable purchase item
Useability test
60
Nutrition Care Process & Standardized Language
A means to apply the NCP in a standard way
using common language.
–
–
–
–
Nutrition Assessment
Nutrition Diagnosis
Nutrition Intervention
Monitoring and Evaluation
Evidence Analysis Library
• Started as Academy “Member benefit”
• Others wanted access
• Group subscriptions for other dietetic
associations (e.g. Australia, Israel)
• Other health professional associations
–
American Association for Diabetes Educators,
Association for Parenteral and Enteral Nutrition
• Guidelines.gov = free access very top-line
information from Guidelines
• Other companion products for sale
• Toolkits, powerpoint slides, consumer handouts,
mobile phone app
62
EAL Recommendations at your fingertips!
• NutriGuides App- NEW!!
•Provides 300+ recommendations
•Categorized by:
–Disease/condition
•Topic
•Nutrition Care Process
•Able to see the strength of the recommendation
Available for iPhones and iPads from the iTunes store
and for Android products from Google store.
NutriGuides
from the Evidence Analysis Library
For smart iPads/iPhones and Androids
Access to 300+ nutrition recommendations
Easy searching
$1.99
Are you using all of the EAL Resources?
65
Are you using all of the NCP resources?
66
Conclusion
Value of using an evidence grading methodology
Value of using all types of research in the body of
evidence based on type of question being
answered
• Epidemiological research preferred for questions
about harm, prognosis and diagnoses
• RCT preferred for treatment questions
• EAL contains ~ 39% observational types of research
Clearly demonstrate which research studies were
evaluated in tables of evidence
EAL Topics are focused on a specific question
Use existing research AND create new research
Acknowledgements
• Current and Past Evidence Based Practice
Committees (~75)
• Current and Past Research Committees
(~150)
• Evidence Workgroups (~325)
• Evidence Analysts (~210)
• Academy Staff
• Dr Alison Steiber
• Dr Paula Zeigler
• Kari Kren, Mujahed Kahn, Lisa Maloney, Grace
Whiteford
• Toni Acosta
68
Questions?
69
Thank YOU!
70
Evidence Analysis Library
•Diseases/Health Conditions
•Nutrients
•Foods
•Life Cycle & Nutrition
•Nutrition Care Process
A – Z Index
Click the +
sign to
expand a
category
72
Search Engine
Enter word or phrases in the search engine:
View
results by
category
73
74