WHO GUIDANCE FOR THE DEVELOPMENT OF EVIDENCE-BASED VACCINE RELATED RECOMMENDATIONS August 2011 Position Papers(PP) on Vaccine Preventable Diseases: 4 sections • Introduction • Information on the respective disease (disease epidemiology, the pathogen, the disease) • Information on the available vaccines (composition, safety, immune response, efficacy and effectiveness, cost effectiveness, other relevant issue) • WHO position on optimal vaccine use (based on best available evidence, social values, preferences) Importance of quality of evidence • Quality of study design crucial for conclusions • RCTs are considered gold standard; faulty randomization or blinding may reduce the quality of an RCT below that of a well designed observational study. • Quality of evidence reflects extent to which confidence in the estimation of effect is adequate to support a particular decision. GRADE for assessing evidence • Grading of Recommendations Assessment, Development and Evaluation (GRADE): adopted by WHO and 50 other organizations. • WHO Vaccine Position Papers uses GRADE method to score quality of evidence. • SAGE also make strategic recommendations without formal GRADE scoring but follow GRADE approach GRADE Process: Systematic Literature Review • 5 criteria (limitations in study design, inconsistency, indirectness, imprecision, publication bias) used to downgrade quality of evidence . • 3 criteria (magnitude of effect, dose-response gradient, ability of study to limit biases ,control for confounding) used to upgrade quality of evidence. • Based on this score & other factors (balance betn benefits & risks, social values ,preferences, cost resources), recommendations are made and scored as strong or weak. • A strong recommendation can still occur with low or very low quality evidence — it is the net result of how all the other factors come into play that is important (Clear explanation would be needed) GRADE Applicability • GRADE tables are only applied to issues regarding the effectiveness & safety of vaccines;generally created for overall vaccine efficacy/effectiveness and safety duration of protection, schedule considerations, and use in subpopulations, • Occasionally used for the such as specific age or risk groups or HIV-infected populations. Key Activities involved in SAGE Recommendations For EBR all steps always to be conducted except 5 i.e. formal scoring Methodology for framing Questions for Systematic Review: PICO 1. Definition of Questions • • • • Patient Population Intervention of interest Comparator Outcomes of interest • • • • Use of rotavirus vaccines (Definition of questions) Population: healthy infants 2–6 mo, or HIV-children; Intervention: 2 different vaccines, 2 or 3 doses; Comparator: absence of vaccination, prevention (hygiene), oral rehydration; Outcome: morbidity, hospitalization, consultations, parental work loss, mortality, nosocomial infections Task for working Group in the beginning 1. Definition of Questions • Consensus on Key PICO questions for detailed literature review • Questions of efficacy/effectiveness, safety and burden of the disease are key questions for developing recommendations for vaccine use • Formulation of the questions to be graded is of utmost importance (validated by entire group) An initial rating of the importance of outcomes should precede the review of the evidence. Issues for Consideration for Reco. Epidemiologic Features of the Disease 1. Definition of Questions Issues for Consideration for Reco. Clinical characteristics 1. Definition of Questions Issues for Consideration for Reco. Vaccine & Immunization Characteristics 1. Definition of Questions Issues for Consideration for Reco. Economic considerations 1. Definition of Questions Issues for Consideration for Reco. Other considerations 1. Definition of Questions • Health-system opportunities and existence of, and interaction with, other existing intervention and control strategies. • Social impacts. • Legal considerations; • ethical considerations. Identification of critical question to which GRADE to be applied 2. Identification of critical Questions • GRADE scoring needs to be applied only to critical questions, preferably five questions. • The formal scoring is only appropriate for questions regarding an intervention (e.g. vaccine use) and not for disease burden, economic considerations or strategic recommendations (e.g. research gaps, decision to pursue an eradication goal, etc) Systematic review of the literature 3. Systemic review of literature • Data may be published or unpublished & should be extracted using a data extraction tool • RCTs are considered the gold standard for assessing the effect of an intervention; for vaccine effectiveness and safety observational studies, including outbreak investigations, disease, surveillance and post-market surveillance data represent important sources Identifying study limitations through risk of bias in a summary table 4. Review the quality of evidence Checklists for reviewing study quality 4. Review the quality of evidence Scoring of the quality of evidence 5. Scoring the quality of evidence Categorization of studies • Studies enter GRADE system at a particular level:all RCTs enter at level 4 & observational studies and surveillance data enter at level 2 . • Then downgraded (Limitation, Inconsistency, Indirectness, Imprecision, Reporting bias) or upgraded ( Large effect/strength of association, populatn effect, mitigated bias or confounding) Final scores can’t exceed 4 points or drop <1. Quality of evidence rating Quality of evidence rating 5. Scoring the quality of evidence 5. Scoring the quality of evidence The GRADE tables explicitly provide the score of the outcomes critical to the recommendation Template of a GRADE table used to score the quality of evidence 5. Scoring the quality of evidence 5. Scoring the quality of evidence Levels of Evidence Am Fam Physician 2002;65:251-8.