Respiratory Guidelines—Which Real World?
Gary W. K. Wong1,2, Marc Miravitlles2,3, Alison Chisholm2,
Jerry Krishnan2,4
1. Department of Paediatrics and School of Public Health, Prince of Wales
Hospital, The Chinese University of Hong Kong, Hong Kong, China;
2. The Respiratory Effectiveness Group, Cambridge, United Kingdom;
3. Pneumology Department, Hospital Universitari Vall d’Hebron, CIBER de
Enfermedades Respiratorias (CIBERES), Barcelona, Spain;
4. Population Health Sciences, University of Illinois, Hospital and Health
Sciences System, Chicago, Illinois
Clinical practice guidelines: purpose
• Remaining abreast of the latest advances in medical
research and practice is a challenge for many
practicing clinicians. Need to
o Digest
o Translate and Apply
the perpetual flow of new research data
• Clinical guidance aims to: help guide clinicians to
understand, translate, and apply new evidence into
everyday practice
Clinical practice guidelines: limitations
• This purposes can be thwarted by:
o A lack of diversity and plurality of committee
members
o By limitations in evidence grading methodologies
o Limitations in the evidence base and in
randomised controlled trail (RCT) design, which
largely underpin their recommendations.
Limitations: guideline committees
• Lack of consistency in the composition of
respiratory guideline groups
• Clinical non-specialist stakeholder are often
under represented
o Experts panels are often dominated by clinicians from
university centers or tertiary hospitals who treat a
subset of (more severe) patients
o No primary care physicians are listed as authors on
GOLD’s Strategy document for the diagnosis,
management, and prevention of COPD1
1. Vestbo J, et al. Am J Respir Crit Care Med 2013;187:347–365.
Limitations: guideline committees
• Lessons can be learned from:
o Global Initiative for Asthma (GINA):2
Practitioners from primary care and specialists with
knowledge implementation and dissemination have been
recruited so GINA’s recommendations better reflect the
practical applicability of frontline daily practice around the
world.
o Spanish COPD guidelines (GesEPOC). Pharmacological
treatment of stable COPD
Included representatives from all scientific societies
involved in the care of patients with COPD within the
development group to ensure proportional representation
of primary specialists.3
2. GINA website: http://www.ginasthma.org/Committees
3. Miravitlles M, S et al. Arch Bronconeumol 2012;48:247–257.
Limitations: GRADE evidence appraisal (I)
• The best accepted methodology for developing
guidelines is the Grading of Recommendations
Assessment, Development, and Evaluation (GRADE)
system.
• GRADE has been adopted by more than 70
organizations around the world, including the American
Thoracic Society (ATS), the World Health Organization
(WHO), the Cochrane Collaboration, and the NICE (6).
• GRADE classifies the quality of evidence as very low,
low, moderate, or high based on the methodological
quality of the evidence and likelihood of outcome bias
4. Guyatt GH, et al on behalf of the GRADE Working Group. BMJ
2008;336:924–926.
Limitations: GRADE evidence appraisal (II)
• GRADE’s quality assessment starts:
o RCTs as high-quality evidence and
o Observational studies as of low-quality evidence.
• The following can reduce the quality of evidence from RCTs:
o Poorly detailed design and execution, inconsistency,
indirectness, reporting bias, and imprecision.
• Factors that can increase the quality of evidence from
observational studies include:
o robust methods of handling potential sources of bias (selection
bias, recall bias, information bias, detection bias) and a priori
study registration and planning
• GRADE then offers a quality classification for the associated
recommendation (weak or strong).
4. Guyatt GH, et al on behalf of the GRADE Working Group. BMJ
2008;336:924–926.
Limitations: GRADE evidence appraisal (III)
• GRADE offers a systematic approach to evidence
evaluation
BUT
• It is not practically possible to address all clinical
questions that may arise in everyday clinical practice or
to have answers to all the potential outcomes of interest.
• The result is a limitation in the published evidence
available to inform practicing clinicians.
o Statement issued by the American College of Physicians,
the American College of Chest Physicians, and the
European Respiratory Society on the management of
COPD offered only 7 evidence-based recommendations.5
5. Qaseem A, et al. Ann Intern Med 2011;155:179–191.
Limitations: RCTs inclusions/exclusions
• Studies have
shown that
efficacy RCTs
exclude about
95% of asthma
and 90% of COPD
routine care
populations due to
strict inclusion
criteria.6
6. Herland K, et al. Respir
Med 2005;99:11–19.
Patient RCT eligibility drop-off with sequential
application of standard inclusion criteria
COPD
Asthma
Limitations: RCT design
• In addition to strict inclusion/exclusion criteria, RCTs
typically involve:
o Close patient–physician interaction
o Rigorous intensity of follow-up (often at a level
infeasible in routine care)
• RCT design is necessary to discern a direct cause-andeffect between an intervention and an outcome but limit
the generalizability of their findings.
• Observational studies, postmarketing, and pragmatic
trials typically lack some of the internal validity of RCTs
but offer greater external validity (i.e., generalizable).
Limitations: who do RCTs miss?
• A recent study7 in >1,000 patients age ≥40 years found that
45% of patients with a physician diagnosis of COPD do
not meet the typical clinical trial criteria for defining COPD
(known risk factor, such as smoking, and a reduced
FVC/FEV1)
• The study found also found that: patients with a physician
diagnosis of COPD who were younger, female, not white,
and had diabetes or depression were more likely to be
excluded from clinical trials on the basis of the reference
standard.
• Efficacy trials in COPD often enroll a small, nonrepresentative subset of patients with physician-diagnosed
COPD.
7. Prieto-Centurion V, et al. Am J Respir Crit Care Med 2013;187:A5017.
Improving guidelines: evidence appraisal
• Sir Michael Rawlins, the former chairman of the UK’s
National Institute for Health and Clinical Excellence:7
o Hierarchies of evidence are over-simplistic and offer a
pseudoquantitative assessment of the available
evidence.
o All forms of evidence should be considered, while
taking into account the limitations and strengths of
their respective designs
o Differently designed studies should be considered as
complementary and should be used in combination by
guideline developers to help inform their judgments
and recommendations.
7. Rawlins M. De testimonio: on the evidence for decisions about
the use of therapeutic interventions. Lancet 2008;372:2152–2161.
Improving Guidelines: addressing gaps
• Greater emphasis should be placed on recognizing the
gaps in their evidence review (i.e., on the indirectness
of the evidence that efficacy trials present for most
patient populations).
• GRADE provides the opportunity to downgrade the
quality of evidence for “indirectness,” and guideline
committees should carefully review this criterion in
assigning the quality of evidence.
• The Pragmatic–Explanatory Continuum Indicator
Summary (PRECIS) wheel provides a structured
approach to assessing the extent to which studies may
provide indirect evidence
8. Thorpe KE, et al. CMAJ 2009;180:E47–E57.
Improving Guidelines: PRECIS Wheel (I)
• 9 “spokes,” each representing a
different element of the study
design (e.g., study eligibility
criteria, expertise of individuals
applying the intervention).
• Each spoke, or axis, represents
an explanatory– pragmatic (i.e.,
efficacy–effectiveness)
continuum, and aspects of a
trial are scored/positioned along
each respective axis depending
on the extent to which they
reflect the characteristics of an
explanatory (efficacy) RCT or a
pragmatic effectiveness trial
8. Thorpe KE, et al. CMAJ 2009;180:E47–E57.
Improving Guidelines: PRECIS Wheel (I)
Two asthma trials mapped on the PRECIS wheel domains – A: Price et al9), B: an efficacy
trial (Malmstrom et al10). The visually representation shows the clear gaps in the efficacy
trial design
9. Price D, et al. Health. Technol Assess 2011;15:1–132.
10. Malmstrom K, et al. Ann Intern Med 1999;130:487–495.
Integrating evidence
• Large representative observational studies:
offer the possibility of analyzing patient subgroups of
clinical interest (e.g.,pediatrics, elderly, patients with
comorbidities, smokers, etc.) to explore whether RCT
efficacy findings deviate in any of the groups of patients
that are typically excluded from RCTs.
• Pragmatic effectiveness trials:
conducted in patients with clinical disease diagnoses
(i.e., those patients the physician believes have a
condition and are treated accordingly) can further
complement data from observational studies and efficacy
RCTs.
Integrating evidence: example
• Inform asthma management decisions in current
smokers.11
o Cigarette smoking associated with worsening symptoms,
accelerated decline in lung function, and impaired
response to corticosteroids (28–30).
o Smokers are typically excluded from asthma RCTs and
guidelines provide no specific management advice
o Pragmatic trial & observational study data suggest
possible utility in:
– higher-dose inhaled corticosteroids,
– targeting the small airways of the lungs
– Targeting leukotrienes,
– Possibly a combinations of these options (27).
Conclusions: enhancing guideline utility
• To achieve their goal of helping to guide the meaningful
implementation of new research into practice, for the
benefit of all stakeholders, clinical guidelines would
benefit from:
o Wider stakeholder representation within guideline
committees (e.g., patients, primary and secondary care
clinicians, policy makers, and health insurers) while still
respecting the rules of evidence evaluation
o Keeping the ideas of “usability” and “readability” at the
forefront of the development process
o Integration of effectiveness data as well as efficacy data
into recommendations.
Conclusions: enhancing guideline utility
• Guidelines must provide a wider view of the
management of diseases, one that reflects the diversity
of decisions faced by clinicians.
• When conducting evidence reviews, guideline
committees should consider not only the robustness of
the available data but also its applicability.
• Evidence from effectiveness studies—pragmatic trials
and observational studies—should be more
consistently incorporated when assessing the risks
and benefits of different care strategies. In some
cases, (e.g., understanding the long-term sequelae of
therapy), observational studies (rather than clinical trials)
are needed.