Principles and Methods of Comparative Effectiveness Research
Spring 2015, Mondays, 2 PM – 4 PM
Education Building, Room 3225A
Class Session
Date
Topic/Assignments
The textbooks for the course are as follows:
Systematic Reviews to Support Evidence-based Medicine, 2nd edition [Paperback]
Khalid Khan (Author), Regina Kunz (Author), Jos Kleijnen (Author), Gerd Antes (Author)
Publication Date: July 29, 2011 | ISBN-10: 1853157945 | ISBN-13: 978-1853157943 | Edition: 2
A Concise Guide to Clinical Trials [Paperback]
Allan Hackshaw (Author)
Publication Date: April 27, 2009 | ISBN-10: 1405167742 | ISBN-13: 978-1405167741 | Edition: 1
(OPTIONAL)
Meta-Analysis: An Updated Collection from the Stata Journal [Paperback]
Jonathan Sterne
Book Description
Publication Date: March 18, 2009 | ISBN-10: 1597180491 | ISBN-13: 978-1597180498 | Edition: 1
Introduction and Systematic Reviews - Melnikow/Nishijima/Abbott
Assignment for Session 1:
 Lesson 16: Systematic Review and Meta-analysis – Steps 1 through 4 - Slides 1-21
Lesson 17: Pub Med Search Demohttp://guides.lib.ucdavis.edu/print_content.php?pid=492203&sid=4040877&mode=g

Reading: Khan , Systematic Reviews to Support Evidence-based Medicine, 2nd edition
[Paperback] Pgs. 1-13
NEW DATES
(1st Hour)
Introduction to CER Course (Melnikow)
March 30, 2015
(2nd Hour)
Systematic Review Project (Melnikow/Nishijima)
 Defining the Research question
 Hierarchy of Evidence
 Searching PubMed and other resources (Abbott)
Module 1: Randomized Clinical Trials (RCTs) – Kravitz/Tancredi
Assignments for Session 2:
 Readings: Hackshaw, Allan K. A Concise Guide to Clinical Trials. Chichester, UK: Wiley-Blackwell/BMJ,
2009. Chapters 1 and 5.
Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, Tunis S, Bergel E, Harvey I,
Magid DJ, Chalkidou K. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help
trial designers. CMAJ. 2009 May 12;180(10):E47-5
Writing Group for the Women's Health Initiative Investigators. Risks and benefits of estrogen plus
progestin in healthy postmenopausal women: Principal results from the women’s health initiative
randomized controlled trial. JAMA. 2002 July 12; 288(3): 321-333.

2
On-Line Lessons 2 and 3
April 6, 2015
History and Basic Elements of RCTs and Explanatory/Pragmatic Distinction
(Tancredi/Kravitz).
Assignment for Session 3:
 Readings: Hackshaw, Allan K. A Concise Guide to Clinical Trials. Chichester, UK: Wiley-Blackwell/BMJ,
2009. Chapters 2, 6, 7 and 10
Kravitz RL, Duan N, Braslow J. Evidence-based medicine, heterogeneity of treatment effects, and the
trouble with averages. Milbank Q. 2004;82(4):661-87.

3
On-Line Lessons 4, 5 and 6
April 13, 2015 Heterogeneity of Treatment Effects and Outcomes Assessment in Clinical
Trials (Kravitz/Tancredi)
COURSE HELD
AT GRANGE
Module 3: Systematic Review – Refining Research Questions – Melnikow/Nishijima
Module 2: Observational Studies – Romano
Assignment for Session 4a:




Develop study question
Complete Worksheet 1 (Formulating the Question) - email prior to session
Reading: Khan: Framing questions for a review (pgs. 9-22)
On-Line Lesson 16: slides 21-56
Assignment for Session 4b:
 On-Line Lesson 13 - Managing Dependent & Independent Variables in Observational CER Studies 76:50
(REQUIRED)

Reading: Benson, Kjell, and Arthur J. Hartz. "A Comparison of Observational Studies and Randomized,
Controlled Trials." New England Journal of Medicine 342.25 (2000): 1878-886.
Concato, John, Nirav Shah, and Ralph I. Horwitz. "Randomized, Controlled Trials, Observational Studies,
and the Hierarchy of Research Designs." New England Journal of Medicine 342.25 (2000): 1887-892.
Lawler, Debbie A., George D. Smith, K. R. Bruckdorfer, Devi Kundu, and Shah Ebrahim. "Those
Confounded Vitamins: What Can We Learn from the Differences between Observational versus
Randomised Trial Evidence? Debbie A Lawlor, George Davey Smith, K Richard Bruckdorfer, Devi Kundu,
Shah Ebrahim." www.thelancet.com. The Lancet, 22 May 2004. Web.Vol 363
Meehan, John P. "A Population-Based Comparison of the Incidence of Adverse Outcomes After
Simultaneous-Bilateral and Staged-Bilateral Total Knee Arthroplasty." The Journal of Bone and Joint
Surgery 93.23 (2011): 2203
4a (1st Hour)
April 20, 2015 Systematic Review: Refining Research Questions (Melnikow/Nishijima)
nd
4b (2 Hour)
April 20, 2015 Management of Dependent and Independent Variables in Observational
Studies: Approaches to Dealing with Confounding (Romano)
Assignment for Session 5:
 On-Line Lesson 14: Observational Studies for CER. How can they be made as good as RCTs? (53:07)
REQUIRED
 Lesson 15: Instrumental Variables and Propensity Scores (45:08) REQUIRED

OPTIONAL but Recommended
Lessons 8-12 provide background information and some very interesting examples of the
principles that we will discuss, so they can be viewed at your convenience.
o
o
o
o
o

Lesson 8: Using the UC Davis Cohort Discovery Tool (powered by i2b2) to Facilitate
Comparative Effectiveness Research (23:02)
Lesson 9: Use of Observational Data in CER Use of OSHPD Administrative Data (39:36)
Lesson 10: Computer-Aided Detection and Screening Mammography (21:07)
Lesson 11: Off-Pump vs. On-Pump Bypass Surgery (20:40)
Lesson 12: Using National HCUP Data to Evaluate CE of Pediatric Empyema Treatments (18:19)
Reading: Austin, P. C. "The Relative Ability of Different Propensity Score Methods to Balance Measured
Covariates Between Treated and Untreated Subjects in Observational Studies." Medical Decision
Making 29.6 (2009): 661-77.
Lu-Yao, G. L., P. C. Albertsen, D. F. Moore, W. Shih, Y. Lin, R. S. Dipaola, and S.-L. Yao. "Survival
Following Primary Androgen Deprivation Therapy Among Men With Localized Prostate Cancer." JAMA:
The Journal of the American Medical Association 300.2 (2008): 173-81.
Mehrotra, R., Y.-W. Chiu, K. Kalantar-Zadeh, J. Bargman, and E. Vonesh. "Similar Outcomes With
Hemodialysis and Peritoneal Dialysis in Patients With End-Stage Renal Disease." Archives of Internal
Medicine 171.2 (2011): 110-18.
5
April 27, 2015
How Can Observational Studies be Made as Good as RCTs?
(Romano)
COURSE HELD
AT GRANGE
Module 3: Systematic Reviews (continued) – Melnikow/Holmes/Nishijima
Assignment for Session 6:



6
Complete Worksheet 2 (Selection of studies to be included) and Worksheet 3 (Synthesizing the data)email prior to session
Reading: Khan: Identifying relevant literature (pgs. 23-37) and Assessing the quality of the literature
(pgs. 39-50) and Summarizing the evidence (pgs. 51-84) STATA Journal PDFs: Metan, metaregression, and funnel plots
On-Line Lesson 18: Analyzing, Interpreting, and Presenting the Results
May 4, 2015
Systematic Reviews (Holmes/Nishijima)
Selecting studies
- defining criteria
- study selection
- data abstraction
Meta-analysis (Jim Holmes)
(Note: Dr. Melnikow Unavailable)
Module 4: Health Economics and Modeling – Leigh/Nishijima
Assignment for Session 7:
 Reading: The lifetime distribution of the incremental societal costs of autism. by ML Ganz - 2007 Cited by 249 – 1.
7
JAMA. 1994 Feb 2; 271(5):375-81. Cost-effectiveness of a routine varicella vaccination program for US
children. Lieu TA1, Cochi SL, Black SB, et al Read abstract only. Routine childhood varicella vaccination.
Lebel RR. JAMA. 1994 Jun 22-29; 271(24):1906.
May 11, 2015
Health Economics – (Leigh)
“Cost of Illness: Methods and Examples”
Systematic Review: Refining Research Questions (Nishijima)
(30 minutes following session 3:30 – 4:00 PM )
Assignment for Session 8:
 Reading: Meltzer, Introduction to health economics for physicians. Health economics quintet, THE
LANCET • Vol 358 • September 22, 2001, pgs. 993-998
8
May 18, 2015
Cost Effectiveness Analysis (Melnikow)
 Introduction to cost effectiveness analysis
9
May 25, 2015
 HOLIDAY – NO SCHEDULED CLASSES
Systematic Reviews – Melnikow/Holmes/Nishijima/Other faculty
Assignment for Session 10:
 Teams come prepared to present overview of their systematic reviews
10
June 1, 2015
Systematic Review Project Presentations
Presentations of projects (10 min, 5 min discussion)
Online Lessons
http://ctsa-cermethodscourse.org/
Course Assignments and Discussion
https://mycourses.ucdmc.ucdavis.edu/portal)
Instructor Office Hours and Contact Information

Melnikow

Kravitz

Tancredi

Romano

Nishijima
Mondays, 1-2 and 4-5, 2020 Ellison; 734-1000
jamelnikow@ucdavis.edu
[INSERT OFFICE HOURS]
rlkravitz@ucdavis.edu
[INSERT OFFICE HOURS]
djtancredi@ucdavis.edu
[INSERT OFFICE HOURS]
psromano@ucdavis.edu
[INSERT OFFICE HOURS]
daniel.nishijima@ucdmc.ucdavis.edu