Update on
Clinical Trials Registration and
Results Reporting Requirements
Deborah A. Zarin, M.D.
ClinicalTrials.gov
February 24, 2009
1
Context:
Continuing Concerns About Lack of
Transparency of Clinical Trial Information
Recent Events: Lack of
Transparency in Clinical Research
Source: Figure 1A. Turner et al. (NEJM, 2008)
The investigation was launched following concerns…
• …although the ENHANCE trial ended in April
2006, the data had not yet been released.
• …[the sponsors] did not register the clinical
trial in a timely manner.
• …[the sponsors] attempted to change the study
endpoints, and thus the study results, prior to
the public release of the results.
ENHANCE
1
2
3
Source: Kastelein JJ, Sager PT, de Groot E, Veltri E. Am Heart J. 2005 Feb;149(2):234-9.
Primary
Primary Outcome
Outcome Measures:
Measures:
•• Change
Change in
in ultrasound-determined
ultrasound-determined average
average carotid
carotid
artery
artery intima-media
intima-media thickness
thickness (IMT)
(IMT) on
on aa per
per subject
subject
basis
basis between
between baseline
baseline and
and endpoint.
endpoint.
[[ Time
Time Frame:
Frame: 24
24 months
months ]]
Source: Silverstein FE et al. JAMA. 2000 Sep 13;284(10):1247-55.
Kaplan-Meier estimates for ulcer complications according to traditional
definition. Results are truncated after 12 months, no ulcer complications
occurred after this period. Adapted from Lu 2001.
Source: Jüni P, Rutjes AW, Dieppe PA. BMJ. 2002 Jun 1;324(7349):1287-8.
“The Neurontin Legacy – Marketing through
Misinformation and Manipulation”
• Studies “designed and commissioned specifically
to promote Neurontin use.”
• Seeding trial “to give neurologists the opportunity
to titrate to higher doses [up to twice the FDAapproved limit] when needed”
• Delayed publication of negative results to mitigate
damage to “neurontin’s marketing success”
• Summary: “the documentation of comprehensive
manipulation of research and publication related to
Neurontin is remarkable.”
10
Landefeld CS, Steinman MA. NEJM. N Engl J Med 360:103-6
11
(WSJ, Aug 2008)
PLoS Med. 2008;5(11): e217
Lee K, Bacchetti P, Sim I. PLoS Med. 2008;5(9): e191
History of ClinicalTrials.gov
• FDAMA 113 (1997): Mandates Registry
– IND trials for serious and life-threatening diseases or
conditions
• ClinicalTrials.gov Launched in February 2000
• Calls for Increased Transparency of Clinical Trials
– Maine State Law; State Attorneys General
– Journal Editors (2004)
• ClinicalTrials.gov Accommodates Other Policies
• FDAAA 801 (2007): Expands Registry and Adds
Results Database
16
Policies and Users
FDAAA
Sponsor Policy
(e.g., NIH, VA)
Maine
ICMJE
FDAMA 113
BPCA
Ottawa
Statement
WHO
ClinicalTrials.gov
Recruitment
(e.g., patients,
physicians)
Journal
Editors
Research
Funders
IRBs
Health
Policy
Makers
17
ClinicalTrials.gov Statistics
(as of 02/03/2009)
Number
Total
Type of Trial*
Observational
Interventional
– Drug & Biologic
– Surgical Procedure
– Behavioral, Gene
Transfer, Other
– Device**
International Sites (161 countries)
US only
Non-US only
US & Non-US mixed
Missing
*171 records missing Study Type information
**173 device trials – “delayed posting”
Percent
67,064
100%
10,690
57,119
42,684
8,585
16%
84%
7,997
3,862
32,772
23,109
4,064
7,119
49%
34%
6%
11%
18
ClinicalTrials.gov Statistics (cont.)
(as of 02/03/2009)
Number
Percent
18,088
21,072
28,820
67,980
27%
31%
42%
Trials by Data Provider
US Federal (including NIH)
Industry
University, Other
Total
User Statistics
Page Views per month
Unique visitors per month
40 Million
500,000
19
Search Features
• Powerful Search Engine
– Synonymy
• E.g., can find V501 studies by searching “Gardasil”
– Hierarchies
• E.g., can find Crohn’s disease by searching for IBD
– Spelling relaxation
• Interface allows for field specific searching
– E.g., can find “condition” without finding
“exclusion criterion”
Update on
Clinical Trial Registration
Public Law 110-85
Sec.801 Expanded Clinical Trial Registry
• Enacted on September 27, 2007
• Requires Trial Registration (Dec 2007)
– Phase II-IV drug and device trials for all diseases
– Data elements: ClinicalTrials.gov + ~ WHO/ICMJE
• Requires Results Reporting (Sept 2008)
– Trials of FDA-approved or cleared drugs and devices
– “Basic” Results: Baseline Characteristics, Primary &
Secondary Outcomes, Statistical Analyses
– Adverse Events (Sept 2009)
– “Expansion” of results by rulemaking (Sept 2010)
• Added enforcement provisions
26
Enforcement Provisions
• Notices of non-compliances
• Civil monetary penalties up to $10,000/day
• Withholding of NIH grant funds
Key Terms
• Applicable Clinical Trials
– Interventional trials
– Phase 2-4 drug, biologic, device
– >= one site in U.S.
– Ongoing as of 9/27/07, or later
• Responsible Party
– Sponsor, grantee
– PI if designated
• (Primary) Completion Date
Wood AJJ. Progress and deficiencies in the registration of clinical trials. NEJM. 2009
Wood AJJ. Progress and deficiencies in the registration of clinical trials. NEJM. 2009
What Information Is In the Trial
Registry?
• Basic Protocol Details
– Condition, intervention(s), design, outcome
measures, key dates
• Administrative Information
– NCT#, other IDs, Responsible Party, PI
• Recruitment and location status
• Linkages
– PubMed, FDA resources, consumer health
info
31
New Registrations
Continue to Increase
Number of New Records Since May 1, 2005
70,000
60,000
50,000
40,000
30,000
20,000
10,000
0
08
20
6/ 8
/2 0
10 1/20 8
2 0
9/ 7/20 8
1 0
8/ 3/20
1 8
7/ /200
8 8
6/ /200 8
4 0
5/ 0/20 8
3 0
3/ /20 8
24 0
2/ /20 07
20 0
1/ 6/2 07
/1 0
12 1/2 7
/1 00
1 1 7 /2
/ 7
10 /200 7
2 0
9/ 9/20 7
2 0
7/ 4/20 7
2 0
6/ 0/20 7
2 0
5/ 5/20 7
1 0
4/ 1/20
1 7
3/ 200 06
4/ 0
2/ 1/2 06
/3 0
12 6/2 06
/2 0
11 2/2 6
/2 0
10 7/20 6
1 0
9/ 3/20
1 6
8/ /200
9 6
7/ /200 6
4 0
6/ 0/20 6
3 0
4/ 6/20 6
2 0
3/ 9/20 6
1 0
2/ /20 05
15 0
1/ 1/2 5
/1 0
12 6/20 5
/ 0
11 /20 5
/2 0
10 8/20 5
2 0
8/ 4/20 5
2 0
7/ 9/20 5
1 0
6/ /20 5
15 0
5/ 0/20
05
1 5
4/ /200 5 /20
6 0 1
3/ 0/20 2/3
3 1
1/ t to
ar
st
Registration of Phase 1 and
Device Trials
• 2100 device trials registered between 9/07
and 1/09
– 175 are in “lock box”
• 162 Industry
• 13 Other
• Phase 1 trials
– 186/month in fy 2008 (73% increase from 07)
– 205/month in fy 2009* (10% increase from 08)
* First four months
Key Points: Memo from Dr. Kington,
Acting Director, to NIH Grant Awardess
• “For grants, NIH is generally not the sponsor …
and, as such, NIH would not be the responsible
party.”
• “Responsible parties who have not yet
registered their clinical trials should do so
immediately.”
• “Thank you for your attention to this important
matter and your commitment to helping enhance
the transparency of NIH funded clinical trials.”
Status of Stanford Trials
• 73 may be “applicable”
– 65 have outcome measure
– 2 have RP
– 62 have start date
– 1 has “primary completion date”
– 8 have “completion date”
• How many “results” are due?
35
Enforcement Provisions
• Notices of non-compliances
• Civil monetary penalties up to $10,000/day
• Withholding of NIH grant funds
Bottom Line
• Register prior to enrollment:
– Phase 2-4 interventional trials that include a
drug, device or biologic
– Regardless of whether or not the trial is being
used to support an FDA application
• Report results:
– Any trial described above once the drug,
device or biologic has been approved; OR
– Within one year of “primary completion date”
• Keep all information up to date!
37
Basic Results Database
38
Basic Results Database:
General Characteristics
• Results of “applicable clinical trials” of
FDA-approved/cleared medical products
• Generally, submission within 12 months of
the earlier of estimated or actual trial
completion date (of primary outcome)
• Delayed Submission of Results
– Seeking initial approval
– Seeking approval of a new use
– Extensions for “good cause”
39
Basic Results Modules
•
•
•
•
•
Participant Flow
Baseline and Demographic Characteristics
Outcome Measures
Adverse Events (summary data)
Other Information
– “Certain Agreements” Restricting Results
Disclosure
– Overall Limitations and Caveats
– Results Point of Contact
Current Status – “Basic Results”
(as of 02/06/09)
• Functional Web-based Data Entry System
• Launched in September 2008
• Ongoing system of feedback and
improvements
• 410 Results Records have been submitted
• Industry: 293 records from 72 data providers
• Other: 117 records from 80 data providers
• Anticipate greatly increased rate of
submission
41
Sample Posted Results
42
Published Participant Flow
43
Source: Kimmick GG et al. Breast J. 2006 Mar-Apr;12(2):114-22.
Arms
Milestone
Reasons Not Completed
Crossover Design
Multiple “Periods”
Published Baseline Data
Source: Richter JE et al. Am J Gastroenterol. 2001 Mar;96(3):656-65.
“Default” Required
Measures
User-Specified
Measure
Categories
Arms
50
Published Primary Outcome
51
Source: Kimmick GG et al. Breast J. 2006 Mar-Apr;12(2):114-22.
Categories
Statistical Analysis
Statistical Analysis
Published Adverse Events
57
Results Data Entry Process
Technical Issues
59
Design Requirements
• Display consists of data tables with minimal
text—must be self-explanatory
• System must accommodate range of study
designs and facilitate comparison across studies
• NLM directed to:
– Consider different methods of display based on
principles of risk communication for different
audiences
– Ensure the data are searchable in many ways
• Structured data entry required to facilitate search
and display needs
60
Design Features
• Tables are “constructed” by the data
provider
– Columns are pre-set as study arms, but can
be changed by the data provider
– Rows are measures—some are pre-set,
others are customized for each study
– Type of measure determines specific design
of “cells”
• Attempt to balance fixed structure with
flexibility
61
Principles for Using the Basic
Results Database
• Submitted data are used to develop basic
tables for the public display
• Tables must be interpretable by people not
familiar with each particular study
• Labels for rows, columns, and units of
measure must be meaningful and precise
62
Who is the Audience?
PI and Clinical Research Team (You!)
Other Medical Researchers in same field
[Study Sponsor]
Other Medical Researchers in other fields
Other Readers of the medical literature
Science Writers
Lay Public (readers of consumer health literature)
Resources to Help Data Providers
• “Helpful Hints”
– Illustrates process for entering different study
designs (parallel, crossover, diagnostic
accuracy, bioequivalence—in progress)
• Webinar
• “Common Errors”
• Individual discussions regarding particular
studies
• Presentations
Interesting Findings to Date
• Large numbers of submitted Outcome
Measures and Statistical Analyses
• Power of Defaults (e.g., “Baseline
Measures”)
– Age > 65
– Race and Ethnicity
– Region of Enrollment
• Problems with imprecise entries
72
Lessons Learned from Early
Submissions of Basic Results
• Many iterations with the QA staff are
necessary to reach minimal quality
standards and to correct serious flaws
• Data Providers must be able to
understand the study design and data
analysis
– Typically, the investigator and a
statistician will need to be involved
73
Quality Assurance Challenges
• Data tables will be the public
representation of the study—must be clear
and informative;
• NLM QA Focuses on:
– Apparent Validity (when possible)
– Meaningful Entries
– Internal consistency/logic
– Format
Common Quality Concerns
• Reporting of percentage without reporting
absolute numbers
• Improper use of terms
• Incidence
• Proportion and Ratio
• Frequency
• Reporting a change—lack of specificity
• Subtraction: minuend vs. subtrahend
• Ratio: nominator vs. denominator
• Complicated outcomes that cannot be
understood
75
Registration and Results Data
Must be Consistent
• Participant Flow Numbers and Enrollment
• Study Design and Results Tables
– Number of Arms
BEFORE Revision (Public View)
Actual enrollment (229) displayed in
the protocol section does not match
total number started in the basic
results section (220 + 211 = 431)
Summary Protocol Section:
Actual Enrollment:
Study Start Date:
Study Completion Date:
Primary Completion Date:
229
June 2006
October 2007
October 2007 (Final data collection date for primary outcome measure)
Basic Results Section:
Participant Flow: Initial Treatment
Placebo
Drug X
STARTED
220
211
COMPLETED
218
210
2
1
NOT COMPLETED
77
Table Structure Must be
Logical
BEFORE Revision (Public View)
Measured Values
Number of Participants Analyzed
Treatment Satisfaction Questionnaire
After 18 Weeks of Treatment
[units: Score]
Mean ± Standard Deviation
Drug X,
Week 10
Drug X, Change from
Week 10 to 18
88
80
81 ± 17.46
7.9 ± 12.16
Inconsistency between columns
and rows: Measure “at week 10”
and Measure “after 18 weeks of
treatment”
79
Data Must Make Sense
• Outcome Measure Name, Description,
Units and Data are Compatible
BEFORE Revision (Public View)
Measured Values
Number of Participants Analyzed
Hours Per Day of Sleep
[units: Average Hours per Day]
Mean ± Standard Deviation
Intervention X
Control
28
27
823 ± 92
864 ± 106
Inconsistency between Units of
Measure, “average hours per day,” and
Measure Data: value provided is greater
than the total number of hours in a day
81
BEFORE Revision (Public View)
Secondary Outcome Measure: Use of Community Health Resources
Measure Type
Secondary
Measure Name
Use of Community Health Resources
Measure Description Evaluation of visits to primary care
pediatrician, hospital emergency and rehospitalization
Time Frame
Up to 3 months after discharge
Safety Issue
No
Implies number
of health
resources Measured
used –
how was it
measured?
• Data are inconsistent:
percentages of what?
• Invalid entry: needs to
be numerical (cannot
include “%”)
Values
Number of Participants Analyzed
Use of Community Health Resources
[units: Number]
Early
Discharge
Standard
Discharge
90
86
4.4%
10.5%
82
BEFORE Revision (Public View)
Secondary Outcome Measure: Frequency and Magnitude of Antibody Response
Measure Type
Primary
Measure Name
Frequency and Magnitude of Antibody Response
Measure Description
Nasal secretions to Virus A/12 and B/14. Antibody
Response: Three-fold increase after immunization
Time Frame
Visit 3 (Week 15)
Safety Issue
Same unit cannot
represent measures
of “frequency” and
“magnitude”
Yes
May mean “three-fold
or greater increase”
Measured Values
Vaccine,
Low Dose
Vaccine,
High Dose
Number of Participants Analyzed
35
34
Frequency and Magnitude of Antibody Response
[units: Participants]
17
21
“Participants” is not a unit of measure
for “frequency” or “magnitude”
Best to
Best
to provide
provideboth
bothcategories
categories
for a dichotomous
for
dichotomousmeasure:
measure:
3x increase
increase
•• << 3x
3x increase
increase
•• ≥≥ 3x
83
Primary Outcome Measure: Maximum Tolerated Dose (MTD) Determination …
Measure Type
Primary
Measure Name
Maximum Tolerated Dose (MTD) Determination as
Measured by Dose Limiting Toxicity (DLT)
Measure Description
The primary variable for determination of the MTD was the
occurrence of DLT during the first treatment cycle. MTD
has been exceeded if >=2 of 6 patients experience a DLT.
Time Frame
Cycle 1
Safety Issue
Yes
Measured Values
Dose Dose Dose
A.1
A.2
B.1
Number of Participants Analyzed
Dose
B.2
Dose
C.1
Dose
C.2
6
7
3
6
7
4
Number (#) of DLT
1
3
0
2
1
0
# Patients at dose level < MTD
0
0
0
0
7
4
# Patients at dose level = MTD
6
0
3
0
0
0
# Patients at dose level > MTD
0
7
0
6
0
0
Maximum Tolerated Dose (MTD)…
[units: Participants]
Tables Must Be Informative
• Scales should include:
– Full name
– Construct or domain (e.g., pain)
– Direction of scores (Best/Worst Value)
– Other information as necessary
• Measures Have Useful Descriptions
• Avoid Abbreviations
BEFORE Revision (Public View)
Need information
about these values
(e.g., is “0” better or
worse than “2”?)
Need information about this scale
• Full Name
• Construct/domain
• Range and directionality
Baseline Measures
Investigational Drug X
GOG Performance Status
[units: Score]
Are these the only
possible scores?
0
48
1
27
2
4
Need to change to “participants” –
data represent “number of
participants” with a particular score
86
BEFORE Revision (Public View)
Needs description:
Duration of what?
Secondary Outcome Measure: Duration (Days)
Measure Type
Secondary
Measure Name
Duration (Days)
Measure Description
Extent of Exposure for All Treated Subjects
Time Frame
Duration of Study
Safety Issue
No
Needs Arm Label: What
is the intervention?
Measured Values
Open Label
Number of Participants Analyzed
Duration (Days)
[units: Days]
Mean ± Standard Deviation
403
195.5 ± 43.87
Measure Information Must be
Precise and Accurate
• Avoid misuse of terms, e.g.,
– proportion
– ratio
– incidence
• State what is being measured and how
– Do not provide results in measure description
field
BEFORE Revision (Public View)
Spell out acronym
Primary Outcome Measure: Proportion of Patients with Controlled SBP
Measure Type
Primary
Measure Name
Proportion of Patients with Controlled SBP
Measure Description
Controlled SBP defined as SBP < 130 mmHg
Time Frame
Baseline to 12 weeks
Safety Issue
No
Not a proportion
Drug X
Drug Y
Drug X + Y
Number of Participants Analyzed
351
361
384
Proportion of Patients with
Controlled SBP
[units: Participants]
186
135
287
BEFORE Revision (Public View)
Spell out acronym
Primary Outcome Measure: Change in Sitting DBP From Baseline to End of Study
Measure Type
Primary
Measure Name
Change in Sitting DBP From Baseline to End of Study
Measure Description
Change in Sitting DBP
Time Frame
Baseline to 12 weeks
Safety Issue
No
Number of Participants Analyzed
Change in Sitting DBP From
Baseline to End of Study
[units: mmHg]
Lease Squares Mean ± Standard Error
Specify calculation
details: which value was
subtracted from which?
Drug X
Drug Y
Drug X + Y
351
361
384
-8.4 ± 0.2
-6.7 ± 0.2
-11.2 ± 0.3
BEFORE Revision (Public View)
State what is
being measured,
not the purpose
Needs description:
Secondary Outcome Measure: To Compare Drug X and Drug Y
for isEfficacy
what
being
Measure Type
Secondary
Measure Name
To Compare Drug X and Drug Y for Efficacy
Measure Description
Time Frame
4 months
Safety Issue
No
measured and how?
Data in All Tables Must be
Internally Consistent and Logical
• Participants must “flow”
• “Number analyzed” must be consistent
with participant flow data
• Avoid Illogical Entries
BEFORE Revision (Public View)
Participant Flow: First Period
Placebo
Drug X
STARTED
301
299
COMPLETED
291
285
NOT COMPLETED
10
14
Number of participants STARTED in
second period of Participant Flow
needs to be the same as numbers
COMPLETED in the first period
Participant Flow: Second Period
Placebo
Drug X
STARTED
298
290
COMPLETED
288
278
NOT COMPLETED
10
12
93
BEFORE Revision (Public View)
Measured Values
Number of Participants
Hours Per Day of Sleep
[units: Average Hours per Day]
Mean ± Standard Deviation
Intervention X
Control
28
27
823 ± 92
864 ± 106
Inconsistency between Units of
Measure, “average hours per day,” and
Measure Data: value provided is greater
than the total number of hours in a day
94
Statistical Analyses
• Must be Logical
• Compatible with Data
• Informative (report informative metrics)
BEFORE Revision (Public View)
Measured Values
Early Discharge
Standard Discharge
100
100
9.3 ± 1.2
7.8 ± 2.1
Number of Participants
Parental Stress
[units: Points on a Likert Scale]
Mean ± Standard Deviation
Inconsistency between Measure Data
and Method of Estimation
• Reported Mean Difference: “9”
• By Inspection: 9.3 – 7.8 = 1.5
Statistical Analysis 1 for Parental Stress
Groups
Early Discharge vs. Standard Discharge
Method
ANOVA
P-Value
0.05
Mean Difference (Net) 9
96
BEFORE Revision (Public View)
Needs description
Secondary Outcome Measure: Time to Relapse of a Mood Episode
Measure Type
Secondary
Measure Name
Time to Relapse of a Mood Episode
Measure Description
Time Frame
24 months
Safety Issue
No
Measured Values
Number of Participants
Time to Relapse of a Mood Episode
[units: Days]
Median (Inter-Quartile Range)
Placebo
Investigational Drug X
148
153
219 (83 to NA)
NA (173 to NA)
97
Invalid entry
Where is the Quality Line?
Domains of Quality:
Quality of Entries
Not Meaningful
Meaningful
Scope of Entries
Minimal
Comprehensive
QA Staff Resources
98
Who is the Audience?
PI and Clinical Research Team (You!)
Other Medical Researchers in same field
[Study Sponsor]
Other Medical Researchers in other fields
Other Readers of the medical literature
Science Writers
Lay Public (readers of consumer health literature)
Bottom Line
• Register within 21 days of enrollment:
– Phase 2-4 interventional trials that include a drug,
device or biologic
– Regardless of whether or not the trial is being used to
support an FDA application
• Report results:
– Any trial described above within one year of “primary
completion date” OR
– once the drug, device or biologic has been approved;
• Keep all information up to date!
100
Additional Information
• Email LISTSERV and other FDAAA
information:
– http://prsinfo.clinicaltrials.gov/fdaaa.html
• Other general information:
– http://prsinfo.clinicaltrials.gov
• Questions?
– register@clinicaltrials.gov
101
Finding Results at ClinicalTrials.gov
• From Homepage
– Go to “Search for Clinical Trials”
– Select “Advanced Search”
– Select “Studies with Results” from the menu
for the Study Results field
– Select study record from results list
– Click “Study Results” tab
102
103
http://prsinfo.clinicaltrials.gov/fdaaa.html
104
105
106
107