Statistical Issues for
Medical Devices and Diagnostics
Bethesda Marriott Hotel
Bethesda, MD
April 16 - 17, 2008
April 16, Wednesday
8:30 – 9:00
REGISTRATION AND CONTINENTAL BREAKFAST
9:00 – 9:10
Welcome
Thomas Maeder, Executive Director, MTLI, AdvaMed
Gregory Campbell, Director Division of Biostatistics, CDRH, FDA
9:10 – 9:30
Statistics in Medical Devices and Diagnostics – an Overview
Gregory Campbell, Director Division of Biostatistics, CDRH, FDA
General overview of how statistics for devices and diagnostics differs
from that in drugs and other areas. How has the field evolved over the
years, what are the current issues, and what do we hope to accomplish
here during the next two days?
9:30 – 10:50
The Use of Adaptive Design
Session planners:
Yonghong Gao, Mathematical Statistician, CDRH
Xiaolong Luo, Senior Mathematical Statistician
Cordis
A considerable amount of recent research and discussion has focused
on statistical and logistical issues associated with adaptive clinical study
designs. Presentations in this session address experiences and lessons
learned in the design and implementation of such studies – some
utilizing a frequentist adaptive approach and some a Bayesian
approach.
9:30 – 9:50
Case Study: Sample Size Re-estimation
Roseann White, Director Global Biostatistics Clinical Data Systems,
Abbott Vascular
Adaptive sample size re-estimation techniques can be useful when one
is unsure of assumptions about the true rates in a randomized or non-
randomized trial. Operational challenges arise, however, in avoiding the
introduction of bias or when enrollment time is much shorter than the
time to follow-up for the interim analysis. This session features a case
study in which many of these operational issues are creatively
addressed.
9:50 – 10:10
Case Study: The SAPPHIRE Study and Lessons Learned
David Snead, Associate Director, Johnson & Johnson
The SAPPHIRE Study used a randomized non-inferiority design with a
novel sequential interval censored survival method to compare carotid
artery stenting to endarterectomy. The study design and results will be
presented along with lessons learned.
10:10 – 10:30
A Consultant’s Perspective on Bayesian Adaptive Device Trials
Scott Berry, President, Berry Consulting
A consultant discusses aspects of Bayesian approach to device trials
that are advantageous to the sponsor, regulator, and to patients. A case
study examines an adaptive device trial using predictive probabilities to
select one of two devices and the appropriate sample size, in which
partial information for subjects is utilized with longitudinal models.
10:30 – 10:50
The FDA Perspective on Adaptive Device Trial Design
Yonghong Gao, Mathematical Statistician, CDRH
An FDA statistical reviewer shares experiences in reviewing adaptive
design trials in terms of both statistical and regulatory considerations.
Discussion points will be raised that may facilitate better communication
between industry and FDA in future trials of this type.
10:50 – 11:10
BREAK
11:10 – 12:30
The Application of Bayesian Statistics
Session planners:
Telba Irony, PhD, Chief, General and Surgical
Devices Branch, Division of Biostatistics, CDRH
Roseann White, Director Global Biostatistics
Clinical Data Systems, Abbott Vascular
In 2006, CDRH issued a draft guidance document on the use of
Bayesian statistics to help people developing or reviewing device or
diagnostic clinical trials determine whether Bayesian methods would be
effective and to apply those methods appropriately. This session
provides more detailed information and illustrative case studies on
appropriate use of the Bayesian approach. In a concluding panel
discussion, the presenters and organizers will be available to answer
questions from the audience.
11:10 – 11:15
Introductory Remarks on Bayesian Device and Diagnostics Studies
Roseann White, Director Global Biostatistics Clinical Data Systems,
Abbott Vascular
11:15 – 11:30
To Use Bayesian or Not to Use Bayesian, that is the Question
Andrew Mugglin, Research Associate Professor, University of Minnesota
With any statistical methodology one must understand not only how to
use it but also when it makes sense in terms of resources and time. This
session examines the key issues to consider before deciding to use a
Bayesian approach to the design and analysis of a study.
11:30 – 11:45
Bayesian Statistics for Therapeutic Devices
Pablo Bonangelino, PhD, Mathematical Statistician, FDA/CDRH
Xuefeng Li, PhD, Mathematical Statistician, FDA/CDRH
Bayesian adaptive designs using predictive probability and regulatory
considerations regarding these designs for therapeutic devices will be
presented. How much strength could be borrowed in a hierarchical
model for a simple case will be discussed.
11:45 – 12:00
Recent Experiences Applying Bayesian Analysis to Medical Device
Trials
A. James O’Malley, PhD, Associate Professor of Health Care Policy
Applying Bayesian analysis to medical device trials, though challenging,
can provide insights into your data that are unattainable with other
methods. Assuring that the method is appropriately applied can be
challenging, however, and this session addresses the use and
interpretation of Bayesian analyses.
12:00 – 12:10
Update on the Bayesian Guidance Document
Telba Irony, PhD, Chief, General and Surgical Devices Branch, Division
of Biostatistics, CDRH
12:10 – 12:30
Bayesian Panel Discussion
Moderator:
Roseann White
Panel Members:
Andrew Mugglin
James O’Malley
Telba Irony
12:30 – 2:00
LUNCH
2:00 – 3:20
Multiplicity
Session planners:
Shiowjen Lee, Mathematical Statistician, CDRH
Michael Lu, Edwards Lifesciences
Medical device clinical trials often consider multiple endpoints to assess
the safety and efficacy of a given treatment. In exchange for distinct
advantages, however, trials with multiple endpoints also pose
challenges, and interpretation of the results must be considered
carefully.
2:00 – 2:30
Rectifying FDA and Bayesian Views on Adjustment for Multiplicity
Gene Pennello, Mathematical Statistician, CDRH
Although frequentist properties of Bayesian designs are not routinely
evaluated in academic research, FDA can request that the Type I error
rate be assessed for proposed Bayesian trials, especially those involving
multiple testing. Justifiable under the Agency’s mandate to obtain valid
evidence of safety and effectiveness, strict control of the overall Type I
error rate can nonetheless negate the advantage of using prior
information through Bayesian analysis. How can one relax the Type I
error rate while maintaining required standards of evidence?
2:30 – 3:00
Control of Type I Error and the Correlation of Multiple Endpoints in
a Medical Device Trial
Kevin Najarian, Boston Scientific
Multiple univariate analyses can inflate the Type I error rate,
necessitating adjustments of the observed p-values. In addition to
common techniques for multiplicity adjustments, some others use
correlation information and consider the homogeneity of treatment
effects to determine the degree of adjustment necessary. An analysis of
real life simulated medical device data is considered to assess
techniques for multiplicity with added regard to the relationships among
endpoints.
3:00 – 3:20
Multiplicity Panel Discussion
Panelists:
Gene Pennello, CDRH
Shiowjen Lee, CDRH
Zengri Wang, Medtronic
Andrew Mugglin, Research Associate Professor,
University of Minnesota
3:20 – 3:40
BREAK
3:40 – 5:00
Issues with Missing Data
Session planners:
Jianxiong (George) Chu, PhD, Mathematical
Statistician, FDA/CDRH
John Evans, PhD, Senior Biostatistics Manager,
Boston Scientific
Missing data is a common problem in clinical studies, which causes the
usual statistical analysis of complete or all available data to be subject to
potential biases. However, there is no universally applicable best
method for handling missing data. In this session, speakers/panelists will
discuss theoretical considerations, technical issues and case studies
with an emphasis on the importance of sensitivity analysis to assess the
impact of missing data on statistical inference and interpretation under
different scenarios of assumptions.
3:40 – 4:00
Missing Data: An Overview
Michael Kenward, PhD, Professor of Biostatistics, London School of
Hygiene & Tropical Medicine
In this introduction and overview to the statistical problem of missing
data, the nature and cases of missingness will be considered in a trial
setting, and Rubin’s framework for missing value mechanisms will be
introduced, together with the implications for subsequent analysis. Both
ad hoc (e.g. completers, last observation carried forward) and
statistically principled (e.g. likelihood, multiple imputation) approaches to
analysis will be discussed. The inherent ambiguity of statistical analyses
for incomplete data will be stressed throughout, and the important role of
sensitivity analysis considered.
4:00 – 4:20
Case Study: Sensitivity Analysis in DES Clinical Trials from the
FDA Reviewer’s Perspective
Xu (Sherry) Yan, PhD, Mathematical Statistician, FDA/CDRH
The impact of missing data has always been a major concern in the
interpretation of clinical trial results. Procedures of statistical inference
for handling missing data, such as multiple imputation, typically require
the specification of missing data mechanisms for the implementation.
When the missing data mechanism is unknown, sensitivity analyses may
provide helpful information. This presentation addresses several
sensitivity analyses, such as worst case scenario analysis and tippingpoint analysis, that were performed for two drug-eluting stent trials.
4:20 – 4:40
Incomplete Data in Clinical Studies: Analysis & Sensitivity Analysis
Geert Molenberghs, PhD, Professor of Biostatistics, Hasselt University,
Diepenbeek, & Katholieke Universiteit, Leuven, Belgium
Incomplete data are common, as are modeling and other data analysis
tools for such data, with ever increasing complexities. Model
assumptions may directly affect inferences and substantive conclusions
without being fully verifiable from the observed data, and it is therefore
essential to recognize sensitivities caused by these assumptions before
proceeding with the analysis. In particular, one cannot formally
distinguish between MAR and MNAR mechanisms. Theoretical
considerations and practical illustrations highlight the implications for
sensitivity analysis.
4:20 – 5:00
Panel Discussion on Missing Data
Moderator:
Jason Roy, PhD, Research Investigator, Center for Health
Research, Geisinger Health System
Panelists:
Michael Kenward
Xu (Sherry) Yan
Geert Molenberghs
Recai Yucel, PhD, Assistant Professor of Biostatistics,
Department of Epidemiology & Biostatistics, School of
Public Health, SUNY-Albany
5:00
ADJOURNMENT
5:00 – 6:30
RECEPTION
April 17, Thursday
8:30 – 9:00
CONTINENTAL BREAKFAST
9:00 – 10:20
Trial Design – No Controls Available or Imperfect Controls
Session Planners:
Lilly Yue, PhD, Chief, Cardiovascular & Ophthalmic
Devices Branch, Division of Biostatistics, CDRH
Brandon Sparks, Medtronic
Randomized, well-controlled, double-blind clinical trials have been viewed
as the gold standard in the evaluation of medical products, but medical
device clinical studies often depart from this ideal paradigm for ethical or
practical reasons. In studies with imperfect or no controls, the resulting
statistical inferences may carry a lower level of scientific assurance. This
session provides an overview, illustrated with case studies, of nonrandomized or imperfectly controlled medical device clinical trials,
considering design and statistical analysis from industry and regulatory
perspectives.
9:00 – 9:20
Unique Challenges Regarding Control Groups for Neurostimulation
Therapies
Steven Broste, Director of Biostatistics & Data Management, Medtronic
Neurostimulation
Implantable neurostimulation systems are in clinical studies or
development for several new indications or anatomical regions. The
mechanism of action for these novel, invasive treatments is unclear, yet
the optimal blinded, controlled studies may not be possible, whether for
ethical reasons or because physical effects of treatment make blinding
impossible. This presentation elaborates on the challenges in such a
study, and describes attempts to deal with these issues.
9:20 – 9:40
Case Study of a Trial Design with Imperfect Control: Industry View
Bryan Randall, Senior Manager, Biostatistics & Clinical Science, Boston
Scientific
This case study features a proposed trial design with imperfect control to
establish the safety and efficacy of carotid stenting compared to carotid
surgery in high-risk patients. The essential design feature is a concurrent,
non-randomized surgery control arm that poses challenges to minimizing
bias from several sources, including operator selection, patient selection,
and outcome evaluation. How can the statistical design and analysis and
trial execution plans mitigate the impact of these potential biases?
9:40 – 10:00
Trial Design with Imperfect Controls or No Control Available: a
Regulatory Perspective
Lilly Yue, PhD, Chief, Cardiovascular & Ophthalmic Devices Branch,
Division of Biostatistics, CDRH
Ethical or practical considerations do not always permit randomized, wellcontrolled, double-blind clinical trials for the evaluation of medical
products. How does a regulatory agency approach the challenge of
imperfect or no controls?
10:00 – 10:20
Panel Discussion on No Controls Available or Imperfect Controls
Moderator:
Brandon Sparks, Medtronic
Panelists:
Steven Broste
Bryan Randall
Lilly Yue
Lei Peng, Abbott Cardiovascular
10:20 – 10:40
BREAK
10:40 – 12:00
Data Poolability
Session Planners:
Chul H. Ahn, PhD, Mathematical
Statistician, CDRH
Dennis W. King, PhD, STATKING Consulting
Poolability comprises a heterogeneous set of problems that need to be
addressed on a case by case basis, depending, among other things, on
whether the groups in question are investigational sites, geographical
regions, clinical studies (or substudies), patient populations, device
models, and so on. This session will consist of presentations by both FDA
and the industry regarding their perspective on this issue followed by the
discussion by the panelist.
10:40 – 11:00
Data Pooling in Medical Device Trials
Chul H. Ahn, PhD, Mathematical Statistician, FDA/CDRH
One often encounters data poolability problems when trying to combine
data from different groups to obtain an overall estimate of an outcome
variable. Different groups may mean different centers, studies, patient
populations, device models, and so on. This presentation addresses
some of the issues in data pooling with examples from recent clinical
studies.
11:00 – 11:20
Medical Device Case Studies in Poolability
John C. Evans, PhD, Senior Biostatistics Manager, Boston Scientific
Medical device poolability case studies will be presented from 1- and 2arm studies. Poolability of patients from different centers assumes that all
sites follow identical protocols. To determine poolability, the comparability
of baseline variables, demographic characteristics, and safety and efficacy
endpoints is assessed across sites. In single-arm studies, the effect of
study center is investigated to determine the poolability of the primary
endpoint from different institutions. For two- or more arm studies, the
effect of interaction between treatments and sites is considered to
determine the poolability of treatment differences in the primary endpoint
across sites.
11:20 – 11:40
Pooling Non-Poolable Clinical Data: So What?
Jeng Mah, PhD, Principal Biostatistician, American Medical Systems
The ability of statistics to examine underlying consistent states is based on
the assumption that random samples from each state are independently
identically distributed (IID). Acceptance of this assumption is not trivial, as
evidenced by the stringent requirements for RCTs and the difficulty of
developing complicated covariate analyses to accommodate lesser clinical
data. Homogeneity of device clinical data, even from well conducted
RCTs, may require closer scrutiny than with pharmaceuticals, because
observed effects of medical devices show a combination of patient
characteristics, device design, and implanter skill. Device studies often
prospectively balance treatment assignments within centers and
retrospectively try to verify data poolability. This study uses simulation to
evaluate the cost of neglecting center effects in terms of false positive
rate, assay sensitivity, and degree of bias in estimating treatment effects.
11:40 – 12:00
Poolability Panel Discussion
Moderator:
Dennis W. King, PhD, President & CEO, STATKING
Consulting
Panelists:
Chul H. Ahn, PhD
John C. Evans, PhD
Jeng Mah, PhD
12:00 – 1:00
LUNCH
NOTE: The two afternoon sessions are run as parallel tracks
1:00 – 5:00
TRACK A
Statistical Issues in Diagnostics and Imaging Studies
Session Planners:
Lakshmi Vishnuvajjala, CDRH
Betty Stephenson, Becton Dickinson
Vicki Petrides, Abbott Diagnostics
Pat Meyers, Abbott Diagnostics
Diagnostic studies present unique challenges in the medical device area.
For laboratory tests which are regulated by the Office of In Vitro
Diagnostics at CDRH, randomized studies are not the rule since samples
can be split in most cases to be tested by both the old and the new tests.
But certain types of bias are much more common and need to be guarded
against. Imaging studies such as mammography, ultrasound, and MRI are
greatly affected by the case mix and reader variability.
Session 1
Statistical Issues in Diagnostics and Imaging
Session Chair:
Lakshmi Vishnuvajjala, CDRH
Speakers:
Dan Sargent, Mayo Clinic (Biomarkers)
Tom Gwise, CDRH (Imaging Studies
There are many challenges that must be addressed in the development of
diagnostic and imaging products. This session will explore some of these
concerns, including co-development of biomarkers and assays, special
problems encountered in diagnostic imaging studies, and quality control
issues.
Session 2
Expert Panel on Guidance for Laboratory Tests
Session Chair:
Betty Stephenson, Becton Dickinson
Speakers:
Marina Kondratovich, CDRH, Migration Guidance,
FDA perspective)
Angel DeGuzman, Abbott Diagnostics, Migration
Guidance, industry perspective
Jan Krouwer, Krouwer Consulting, CLSI
Guidelines
Kyungsook Kim, CDRH, FDA CLIA Waiver
Guidance
There are multiple aspects of diagnostic products that need to be
evaluated by manufacturers, the FDA, and laboratories. To assist in
evaluating some of these characteristics, guidance documents have been
published through the joint efforts of interested parties. This session will
provide a forum during which these guidance documents will be
discussed.
1:00 – 5:00
TRACK B
Postmarket Statistical Issues
Session Planners:
Greg Campbell – CDRH
An Liu – Medtronic
This session will concentrate on statistical issues that arise in the postmarket arena. It will include statistical concerns arising in condition of
approval studies, as well as other post-approval studies to enable a
change in the indication for use for a marketed device. Another
postmarket issue in the session is the use of statistical methodologies in
the analysis of data from FDA’s Medical Device Reporting (MDR) system.
Session 1
General Statistical Methodologies for Post-Market Surveillance of
Medical Devices: A Regulatory Perspective (tentative)
Chang S. Lao, PhD, Senior Mathematical Statistician, Division of
Biostatistics, FDA/CDRH
Need description
Session 2
Using Seattle Angina Questionnaire (SAQ) to Evaluate Patient
Reported Outcome in a Post-marketing Setting
Vivian Mao, Clinical Research Manager, Abbot Vascular
Patient-reported quality of life outcome instrument can be used to
measure the impact of an intervention on patient health status in medical
device clinical trials. Using a valid and reliable instrument to
systematically evaluate patient’s perspective on the impact of drug eluting
stents may provide valuable information that can be lost through a
clinicians’ interpretation. This talk discusses the usage of the Seattle
Angina Questionnaire (SAQ) to evaluate patient health status in patients
receiving XIENCE™ V Everolimus Eluting Coronary Stent System
(EECSS) in a post-marketing environment and its statistical
considerations.
Session 3
Statistical Challenges in the Post-Approval Registry for Drug Eluting
Stents
Peter S. Lam, PhD, Director, Biostatistics & Clinical Sciences, Boston
Scientific
The statistical challenges in the mandatory post-approval registry for Drug
Eluting Stents (DES) to address the safety endpoints of death, myocardial
infarction, and stent thrombosis will be presented using a case example.
These endpoints were not sufficiently powered in the pre-approval trials
due to their low incidence rates. Additional challenges to attempt to
investigate the optimal duration of dual anti-platelet therapy for DES
patients will also be explored in this presentation.
Session 4
A Bayesian Design Applied to an Endovascular Post-Market Study
Yuqing Dai, PhD, Principal Statistician, Medtronic Cardiovascular
Feng Tang, PhD, Principal Statistician, Medtronic Cardiovascular
Often prior knowledge, which contains valuable information, on an existing
device is available when we conduct a new post-market study.
Appropriately using such prior information could lead to sample size
reduction, which is very desirable when the accruement of the sufficient
number of patients could be challenging due to the relatively low incidence
of a rare outcome. In this talk, we use an endovascular trial as an
example, illustrating a Bayesian design on a post-market study. Also, we
will study the relationship between the different operation characteristics,
the decision criteria and the sample size.
5:00
ADJOURNMENT
Important Notice
The information provided in this course represents the personal opinions of the instructors and
does not necessarily represent the opinions of AdvaMed staff. Companies relying on the
information do so at their own risk and assume the risk of any subsequent liability that results
from relying on the information. The information does not constitute legal advice.