P
G
R
C
T
Group 1
Nicole Bettes, Kelsey Brown, Greg Forlenza,
Chen Gao, Joe Ostrem, Li Ou, Shauna
Overgaard, Jincheng Zhou

From the National Institute for Clinical Excellence (NICE) Guideline Development
Methods Updated March 2005

• Developed to assist providers with care of given medical conditions.
• Often incorporated into performance measures and quality of care assessments for providers.
• Many are based on scant evidence.
• Historical examples of major harm which has occurred from guidelines not based on RCT’s.

• Ophthalmic Screening for Diabetic Retinopathy
Forlenza GP, Stewart MW. Diabetic Retinopathy in Children. Pediatr Endocrinol
Rev. 2012 Dec-2013 Jan; 10(2):2017-26

• Strengths of RCT’s:
•
•
• Well designed RCT’s are Unbiased
Well designed RCT’s are Objective
Well designed RCT’s are Consistent
• Weaknesses of “best clinical judgment:”
• Initial judgments on clinical practice are often wrong (46%) resulting in costly reversals in care.
• Reasonable judgment uses subjective criteria which is susceptible to bias and conflicts of interest.
• Lacks consistency across guidelines.
Prasad V. Why Randomized Controlled Trials are Needed to Accept New Practices. May Clin Proc. October
2013:88(10):1046-50

• Originally sold in 1950s as an OTC sedative.
• In late ‘50’s and early ‘60’s it was recommended to pregnant women by their OB/GYN’s to control morning sickness.
• Later found to cause serious birth defects and miscarriages.
• Led to serious birth defects of more than 10,000 children.
Kim JH, Scialli AR. Thalidomide: The tragedy of Birth Defects and the
Effective Treatment of Disease. Toxcol Sci 122(1), 1-6(2011).

• Prior to 1993, expert opinion on care of insulin-dependent diabetes mellitus (type 1 diabetes) was split between intense blood sugar control and “conventional” blood sugar control.
• Between 1983 and 1993 patients with T1DM were randomly assigned to one of the 2 therapies.
• The study showed that intensive therapy reduced microvascular complications by up to 76% compared to
“conventional” therapy!
• Study results drove development of new devices and medications and still serves as foundation of current diabetes care.
DCCT. The Effect of Intensive Treatment of Diabetes. NEJM 329;14: 977-986. Sep
30,1993.

• Prior to 2002, clinical practice guidelines favored the use of HRT during and after menopause.
• Millions of women all over the world used HRT.
• > 22 million prescriptions written in the U.S. in 2000.
• Women’s Health Initiative (WHI) put HRT through a placebo-controlled RCT.
• WHI study stopped early because of overwhelming evidence of risks of HRT.
• Increased risk of invasive breast cancer by 26%, of heart attacks and other coronary events by 29%, and of strokes by 41%.
Randal. The End of an Era? J Natl Cancer Inst 2002; 94:1116-1118. August 7, 2000.

• RCT’s mark the highest grade of medical evidence.
• Clinical practice guidelines provide diagnostic and treatment advice for thousands of medical providers affecting millions of people.
• Medical practice not based on high grade evidence has a long history of causing significant harm to patients.
• Incorrect treatments from faulty guidelines lead to costly reversals in care.
• Providers and patients deserve the highest level of treatment guidelines.

:
1. Generalizability of RCT
2. Feasibility of RCT
3. US Department Health and Human Services Reports
• Examples of practice guidelines developed in the absence of RCT


 Question specific
 Specific disease

 Inclusion/ Exclusion criteria

 Site specific
 Sample size

 Findings applicable to population studied
 Dose extrapolation
 Empirical recommendations
 Interracial differences
 Race specific recommendation
 Cancer therapy in Japanese vs. Caucasians
 Blood pressure management in African Americans vs. Caucasians
 Genetics variability in response
 Poor responders
 Optimal responders

 Approval process is complex, costly, heterogeneous and time-consuming
 Multicenter trials require many separate approvals, often taking >1 year to complete, costing upwards of $100,000
 In China, it takes > 1 year to obtain approvals
 Ethical issues
 Placebos - unethical to deny standard treatment, or deny a therapeutic treatment
 Unethical to impose a harmful intervention upon individuals

 Understanding the relevance and weight of evidence is an important consideration when creating evidence-based guidelines
 All levels of evidence are important and have their respective value and, therefore, should be considered
 Evidence-based practice (EBP) requires clinical decision making
1.
Sound external research evidence
2. Individual clinical expertise
3. The needs of the individual patient
 EBP is the conscientious and judicious use of current best evidence in conjunction with clinical expertise and patient values to guide clinical practice guidelines

Practice guidelines developed in the absence of RCT data as reported by the US Department of Health and Human Services
1.
Clinical practice guidelines for the American Academy of Orthopaedic
Surgeons (AAOS, 2010)
2.
Clinical practice guidelines for Practicing Chiropractors ’ Committee on Radiology Protocols (PCCRP, 2006)
3.
Clinical practice guidelines for the diagnosis and treatment of patients with encephalitis ( Infectious Diseases Society of America ; Tunkel et al. Clinical Infectious Diseases 2008)

 Clinical practice guidelines for the American Academy of
Orthopaedic Surgeons (AAOS, 2010)
– Developed using “ Best Available Evidence ”
• First considered RCT data
• However, often RCT data not available, therefore used :
– Prospective controlled trials
– Prospective comparative studies
– Retrospective comparative studies
– Case-series studies
• Acknowledge must “develop using a systematic review of current scientific and clinical information”
• Noted that, “ clinical patients may not necessarily be the same as those found in a clinical trial . Patient care and treatment (guidelines) should always be based on a clinician’s independent medical judgment, given the individual patient’s clinical circumstances.”
18

 The Eastern Association for the Surgery of Trauma
(EAST) Working Group
– Practice Management Guidelines for Trauma Surgeries
– Assessment of scientific evidence – all relevant empirical data should be evaluated for the clinical use of interventions:
• RCT
• Prospective cohort studies
• Retrospective cohort studies
• Observational studies
• Prevalence studies
• Case control Studies
• Case reports
• Case series studies
• Expert opinion
19

 Clinical practice guidelines for Practicing Chiropractors ’ Committee on Radiology Protocols (PCCRP, 2006)
– When developing clinical practice guidelines, cannot restrict evidence to RCT
• RCT for chiropractic medicine is not always relevant
• RCT narrow in methodology, which is not useful in clinical practice
• Effectiveness evidence cannot be collected by RCT as “randomization and controlled conditions play no part in everyday clinical practice.”
– Most prevalent type of research in chiropractic medicine is:
• Non-randomized clinical trials
• Case series
• Case studies
– Reasons for lack of RCT in chiropractic medicine:
• Not relevant to actual practice
• Labor intensiveness
• Cost
• Usually restricted to university research groups, which in this field there is lack of research at the university level
– Studies have shown that well-done case studies most often demonstrate findings consistent with that of RCT {Venning et al. (BMJ 1982), Benson and Hartz et al (NEJM
2000)., Rosener et al. (JMPT 2003), Concato et al. (NEJM 2000)}.
– Therefore, to develop chiropractic clinical practice guidelines that is truly evidence-based and still relevant to the chiropractic profession, all levels of evidence need to be considered .
20

 Infectious Diseases Society of America
– Developed guidelines for the diagnosis and treatment of patients with encephalitis (Tunkel et al. Clinical Infectious Diseases 2008)
– Guidelines are intended to be used by health care providers who care for patients with encephalitis; provide clinicians with evidencebased recommendations
– Determining guidelines required a systematic review and analysis of evidence that included epidemiologic, clinical and laboratory-based data
– Lack of RCT data as etiology of encephalitis unknown and no definitive treatment strategies available
– Given the lack of RCT data on the diagnosis, treatment and management of encephalitis , many recommendations were developed from:
• Case reports
• Small series reports
• Expert opinion
21

• The approval process is complex, costly, heterogeneous and timeconsuming.
– Multicenter trials require many separate approvals, take more than 1 year to complete, and cost upwards of $100,000
– In China, it takes >1 year to obtain approval
– Slows medical community therapeutic progress
• A “one-size fits all” approach is used
– Regulation of low-risk trials of well-understood drugs is similar to regulation of trials of completely new drugs, for which the risks are unknown.
• Trial participants may not represent the population as a whole and, therefore, results may not apply more generally
– Participants in RCT tend to be a distinct population that does not represent the
“real-world” population an intervention would eventually target (i.e. people who sign up for RCT usually have tried many other treatment options and are desperate for a treatment. Moreover, they are further winnowed down as researchers eliminate would-be participants with co-morbid conditions, etc.)
– RCT are typically designed with internal validity in mind (i.e. the ability to trace causal inferences to the intervention) while ignoring external validity (i.e. generalization of the results)
– Participants are carefully monitored and more adherent to treatment and followup than real-world participants
22

• Some medical questions do not permit random assignment of participants as it may be unethical to do so
• Requires clinical equipoise with an acceptable control
(standard of care treatment)
• An acceptable standard of care medical therapy or an alternative therapy may not exist
23

• 20% of clinical guidelines are cancer related (The
National Guideline Clearinghouse)
• 80% of all NCCN recommendations were category
I: high level of evidence with uniform consensus
(Poonacha et al. JCO 2013)
• There is a huge gap - need of guidelines to aid in decision making in patient care and high level evidence is not available.
24