What Is the Role for Analyses of
Administrative Data in Assessing
Drug Safety in Post-Market
Commitment (PMC) Studies?
Cathy W. Critchlow, PhD
Global Epidemiology, Amgen, Inc.
September 29, 2006
Outline
Why should we consider additional
approaches (e.g., analyses of administrative
data) to post-market commitment studies?
What are situations where analyses of
administrative data can be used to
supplement, or even replace, clinical postmarket commitment studies?


Study design issues
Strengths and limitations of administrative data
analyses as a component of post-market
surveillance
Post-Market Surveillance –
Continuing Assessments of Safety or Efficacy
Post-market surveillance
Post-market commitment studies
Registries or studies conducted to
‘complete’ pre-market assessments
Routine surveillance
Spontaneous reports
• Unexpected or rare AEs
Observational studies
• Rate of expected/unexpected AEs
• Relative rate of ‘hard’ endpoints
???
• Dynamic response to
emerging issues, hypotheses
Many examples
Can analyses using automated databases help meet these needs?
Need for Reevaluation of Post-Market
Assessment Strategies
Withdrawal of Cox-2 inhibitors after several years
and several million patient exposures contributes to
“perception of crisis that has compromised the
credibility of FDA and the pharmaceutical industry”*
Public’s loss of faith in the ability of industry to
deliver safe and effective drugs†


13% think pharmaceutical companies are “generally
honest and trustworthy”
60% not confident that drug companies will publicly
disclose safety data in a timely manner
*IOM Report: The Future of Drug Safety: Promoting and Protecting the Health of the Public
†Harris
Poll, 2004
Need for Reevaluation of Post-Market
Assessments Strategies (2)
Reliance on regulation alone to demonstrate longterm safety has not worked

Unmet phase 4 commitments
114 (9.6%) of 1,191 open PMCs met*

Confirmation of efficacy using hard endpoints in phase 4
commitment studies for drugs receiving “fast-track”
approval based on surrogate measures
FDA Critical Path Initiative

Use of database registries and electronic medical record
systems to compare outcomes among relevant patient
groups in post-market drug evaluations
* Federal Register 2005;70:8379-81.
Crux of the Issue…..
Post-Approval Drug Safety
Typically, 500 - 3000 patients exposed to drug in
phase 3 testing

Drug effects detectable with an incidence ~1–6 per 1000
To quantify the risk of an event with incidence of 2
per 10,000/year (precision ±1 per 10,000) with 95%
probability, need ~80,000 subjects followed for 1 year
Difficult to conduct studies this large in a timely
fashion
Opportunities
Consider additional or alternative strategies to
demonstrate long-term drug safety or efficacy

Analyses of administrative data* in post-market
surveillance
Collaboratively establish high standards for the
conduct of observational data analyses
conducted as part of post-market commitments
Demonstrate safety and effectiveness of drugs
in ‘real-world’ settings
*Claims data (commercial insurance, Medicare, Medicaid), medical record data,
national databases
Qualities of the Ideal Database
Comprehensive

Inpatient & outpatient care; ER visits; lab &
radiological tests; prescribed & OTC drugs; mental
health care; alternative therapy
Large, stable population
Unique identifiers for linkage
Regular, frequent updates
Verifiable, reliable
Capacity for chart review or patient interviews


Confounder data
Compliance
But, few databases are
ideal….(some are better than others)
Problematic situations





Illnesses that do not reliably come to medical attention
Inpatient drug exposures
Outcomes poorly defined by ICD-9 coding
When necessary confounder data cannot be obtained
Very long latency events
Need to understand the limitations of any database





Purpose for which database was created
Data quality, validity, completeness
Availability of confounder data
Patient follow-up
Access to source information
Issues to Consider in the Design of PMCs
What is the objective?

Hypothesis generation (descriptive or exploratory)
vs. confirmation?
Evaluating expected vs. unexpected events

Apriori specification of events of interest
Timing of events of interest

Short-term vs. long latency outcomes
Most PMCs are Observational…..
Issues of Study Validity
Bias


Selection bias
Information bias
Misclassification of covariates, exposure, outcome

Data validity
Confounding



By disease severity, treatment indication, comorbid conditions
Unmeasured covariates
Time-dependent confounding
Physician prescription patterns (‘channeling’)
Dosing variability according to patient responsiveness
What are appropriate comparator groups?
Other Considerations……
Urgency of need for data

Drug first in class or are other relevant data
available?
Risk vs. benefit profile
Numbers of persons to be exposed
Expected AE incidence rate
Signal detection – what constitutes a safety
signal?
Implications for study design, sample size, comparators,
interim analyses, scientific rigor required
What are situations where analyses
of administrative data can be used
to supplement, or even replace,
clinical PMC studies?
PMC Scenario 1: Single-Arm Prospective,
Observational Clinical Registry
Characterize long-term safety profile of approved
drug

Incidence of various adverse events
Drug utilization in the ‘real-world’



Special populations, e.g., children
Effect of comorbid conditions
Drug-drug interactions
PMC Scenario 1: Single-Arm Prospective,
Observational Registry
Objective: Characterize drug safety post-approval
Clinical Registry
Hypothesis generating
Pre-specified outcomes;
unexpected events
Modest sample size

Will not observe rare events
Effect measure


Absolute incidence rate
SIR (external comparator)
Virtual (Database) Registry
Hypothesis confirmation
Post-hoc analyses of
unexpected events
Large sample size

Effect measure


Prospective data collection

Difficult to assess long latency
events
Study rare events
Absolute risk in select population
Relative risk (internal
comparator)
Retrospective study design

Potential for answers sooner
PMC Scenario 1: Single-Arm Prospective,
Observational Registry (2)
Objective: Characterize drug safety post-drug approval
Clinical Registry
Data quality



Outcome adjudication
Covariate data can be obtained
Regulatory definitions of AEs
Sources of bias, confounding


Potential selection, recall or
information bias
Differential loss-to-followup with
respect to risk of outcomes?
Virtual (Database) Registry
Data quality



Validity of algorithms assessing
drug exposure, disease severity,
outcomes
Comparable ascertainment of
data from exposed and
comparator groups
Data from all medical care
providers
Sources of bias, confounding



Relevant covariate data
available?
Confounding by indication for
treatment, comorbidities
Stability of population
When/What Could Analyses of Administrative
Data Contribute to this PMC Scenario?
Objective: Characterize drug safety post-drug approval
When…





Large sample size needed to assess rare events
Events specified apriori
Objective lab-driven diagnoses
Confounder data available
Denominator needed to calculate population incidence
rates
What…..


Background incidence rates
Attributable risk of events
PMC Scenario 2: Controlled
Studies Further Assessing Efficacy
Obtain additional data regarding
meaningful clinical endpoints

Confirm estimates of efficacy of drugs
receiving “fast-track” approval based on
surrogate measures
Head-to-head comparisons

New vs. existing drug
PMC Scenario 2: Controlled Studies
Further Assessing Efficacy
Objective: Compare incidence of clinical (efficacy)
endpoints among exposed and unexposed groups
Clinical Study
Modest sample size

Expense of large study of
infrequent outcomes
Effect measure

Relative risk compared to
placebo or standard of care
Prospective data collection

Potential ethical issues in
randomized trial with placebo
control and/or long follow-up
Database Study
Large sample size

Study rare events
Effect measure


Relative risk
More easily do ‘head-to-head’
comparisons
Retrospective cohort study

Potential for answers sooner
PMC Scenario 2: Controlled Studies
Further Assessing Efficacy (2)
Objective: Compare incidence of clinical (efficacy)
endpoints among exposed and unexposed groups
Clinical Study
Data quality


Outcome adjudication
Covariate data can be
obtained
Sources of bias,
confounding


Potential selection, recall
or information bias
Differential loss-tofollowup with respect to
risk of outcome?
Database Study
Data quality

Validity of algorithms
assessing drug exposure,
disease severity, outcomes
Sources of bias,
confounding



Unmeasured confounders?
Confounding by indication
for treatment, comorbidities
Stability of population
When/What Could Analyses of Administrative
Data Contribute to this PMC Scenario?
Objective: Compare incidence of clinical (efficacy)
endpoints among exposed and unexposed groups
When…




Large sample size needed to assess rare outcomes, longlatency outcomes
Valid ascertainment of outcome
Short-term effects
Recall or interviewer bias could effect association
What…..


Timely validation of surrogate markers
Head-to-head comparison of outcome incidence
Opportunities - Revisited
Potential role for analyses of administrative
data in post-market surveillance
Collaborative establishment of regulatory
thresholds for the conduct, analysis and
interpretation of observational data analyses
conducted as part of post-market commitments
Demonstrate safety and effectiveness of drugs
in timely fashion
Thank you!