WHO
GUIDANCE FOR THE DEVELOPMENT OF
EVIDENCE-BASED VACCINE RELATED
RECOMMENDATIONS
August 2011
Position Papers(PP) on Vaccine
Preventable Diseases: 4 sections
• Introduction
• Information on the respective disease (disease
epidemiology, the pathogen, the disease)
• Information on the available vaccines
(composition, safety, immune response, efficacy and
effectiveness, cost effectiveness, other relevant issue)
• WHO position on optimal vaccine use (based on
best available evidence, social values, preferences)
Importance of quality of evidence
• Quality of study design crucial for conclusions
• RCTs are considered gold standard; faulty
randomization or blinding may reduce the
quality of an RCT below that of a well designed
observational study.
• Quality of evidence reflects extent to which
confidence in the estimation of effect is
adequate to support a particular decision.
GRADE for assessing evidence
• Grading of Recommendations Assessment,
Development and Evaluation (GRADE):
adopted by WHO and 50 other organizations.
• WHO Vaccine Position Papers uses GRADE
method to score quality of evidence.
• SAGE also make strategic recommendations
without formal GRADE scoring but follow
GRADE approach
GRADE Process:
Systematic Literature Review
• 5 criteria (limitations in study design, inconsistency, indirectness,
imprecision, publication bias) used to downgrade quality of evidence .
• 3 criteria (magnitude of effect, dose-response gradient, ability of study to
limit biases ,control for confounding) used to upgrade quality of evidence.
• Based on this score & other factors (balance betn benefits &
risks, social values ,preferences, cost resources), recommendations
are made and scored as strong or weak.
• A strong recommendation can still occur with low or
very low quality evidence — it is the net result of how all the other
factors come into play that is important (Clear explanation would be needed)
GRADE Applicability
• GRADE tables are only applied to issues regarding the
effectiveness & safety of vaccines;generally created
for overall vaccine efficacy/effectiveness and safety
duration of protection,
schedule considerations, and use in subpopulations,
• Occasionally used for the
such as specific age or risk groups or HIV-infected populations.
Key Activities involved in SAGE Recommendations
For EBR all steps always to be conducted except 5 i.e. formal scoring
Methodology for framing Questions for
Systematic Review: PICO
1. Definition of Questions
•
•
•
•
Patient Population
Intervention of interest
Comparator
Outcomes of interest
•
•
•
•
Use of rotavirus vaccines (Definition of questions)
Population: healthy infants 2–6 mo, or HIV-children;
Intervention: 2 different vaccines, 2 or 3 doses;
Comparator: absence of vaccination, prevention (hygiene), oral rehydration;
Outcome: morbidity, hospitalization, consultations, parental work loss,
mortality, nosocomial infections
Task for working Group in the beginning
1. Definition of Questions
• Consensus on Key PICO questions for
detailed literature review
• Questions of efficacy/effectiveness, safety and
burden of the disease are key questions for
developing recommendations for vaccine use
• Formulation of the questions to be graded is of
utmost importance (validated by entire group)
An initial rating of the importance of outcomes
should precede the review of the evidence.
Issues for Consideration for Reco.
Epidemiologic Features of the Disease
1. Definition of Questions
Issues for Consideration for Reco.
Clinical characteristics
1. Definition of Questions
Issues for Consideration for Reco.
Vaccine & Immunization Characteristics
1. Definition of Questions
Issues for Consideration for Reco.
Economic considerations
1. Definition of Questions
Issues for Consideration for Reco.
Other considerations
1. Definition of Questions
• Health-system opportunities and existence
of, and interaction with, other existing
intervention and control strategies.
• Social impacts.
• Legal considerations;
• ethical considerations.
Identification of critical question to
which GRADE to be applied
2. Identification of critical Questions
• GRADE scoring needs to be applied only to
critical questions, preferably five questions.
• The formal scoring is only appropriate for
questions regarding an intervention (e.g.
vaccine use) and not for disease burden,
economic considerations or strategic
recommendations (e.g. research gaps, decision
to pursue an eradication goal, etc)
Systematic review of the literature
3. Systemic review of literature
• Data may be published or unpublished & should
be extracted using a data extraction tool
• RCTs are considered the gold standard for
assessing the effect of an intervention; for
vaccine effectiveness and safety observational
studies, including outbreak investigations,
disease, surveillance and post-market
surveillance data represent important sources
Identifying study limitations through risk
of bias in a summary table
4. Review the quality of evidence
Checklists for reviewing study quality
4. Review the quality of evidence
Scoring of the quality of evidence
5. Scoring the quality of evidence
Categorization of studies
• Studies enter GRADE system at a particular
level:all RCTs enter at level 4 & observational
studies and surveillance data enter at level 2 .
• Then downgraded (Limitation, Inconsistency,
Indirectness, Imprecision, Reporting bias) or
upgraded ( Large effect/strength of association,
populatn effect, mitigated bias or confounding)
Final scores can’t exceed 4 points or drop <1.
Quality of evidence rating
Quality of evidence rating
5. Scoring the quality of evidence
5. Scoring the quality of evidence
The GRADE tables explicitly provide the score of the
outcomes critical to the recommendation
Template of a GRADE table used to score the quality of evidence
5. Scoring the quality of evidence
5. Scoring the quality of evidence
Levels of Evidence
Am Fam Physician 2002;65:251-8.