Imaging for Treatment Assessment in Radiation Therapy
(ITART)
The first biennial meeting to focus on quantitative imaging
in radiation therapy.
What Can We Learn From
Each Other?
June 2121-22, 2010
Washington, DC
Daniel C. Sullivan, MD
Duke University Medical Center
RSNA
Accuracy of quantitative imaging
biomarker data:
• Requires collaboration from
– Physicists; Engineers; Image Processing
Scientists
– Manufacturers
– Regulators
– Biologists/Chemists
– Physicians
– Biostatisticians; Methodologists
– Patient Advocates
Acknowledgement
Some of my following comments are
influenced by:
• Coller, B. Translational Research: Forging a
New Cultural Identity.
MOUNT SINAI JOURNAL OF MEDICINE 75:478–487, 2008
All generalizations are dangerous, even
this one.
» Alexandre Dumas
Basic Science Academicians
• Motivation: Discovery of new knowledge
– Eliminate uncertainty
– Avoid coming to conclusions too rapidly;
Replicate results
– Be skeptical; challenge
• Incentives/Rewards:
–
–
–
–
Grants/contracts
Publications
Independent work; Competition
Promotions
Industry
Physicians
Motivation: Apply knowledge to improve
patients’ health (Diagnose and Cure)
–
–
–
–
Accept uncertainty
Adhere to accepted methods; Conservative
Accept high “noise levels” in clinical studies
Generalize broadly from small “n”
• Rewards/Incentives:
– Altruistic goals (helping people)
– Salary (Salary offset)
– Status/prestige/promotions
Regulators
Motivation: Deliver products (or services)
– Approach is task and goal oriented, as contrasted with the
less structured academic approach
– R&D cost
• Pharmaceutical industry – very high
• Imaging device industry – moderately high
– Time-to-market
• Pharmaceutical industry – 8 to 20 years
• Imaging device industry – months to few years
– IP “protection”
• Pharmaceutical industry – 10 to 15 years
• Imaging device industry – 2 to 4 years
• Rewards/Incentives
– Team rewards for accomplishing goals,
deliverables and product launch
– Profits
• Motivation: Fulfill legally mandated mission
– Mitigate harm
– Make binary or categorical decisions in the
absence of complete data.
– Maintain consistent interpretations of ambiguous
concepts.
• Rewards/Incentives:
– Altruistic goals
– Minimize erroneous decisions
Patients
• Motivation: Health
– Accurate diagnoses and treatment
decisions
– Minimal cost and morbidity
• Rewards/Incentives
– Ability to Live, Work and Love
RSNA Strategic Plan
• Goal 1: Shape and advance the future
of radiology.
– 1.3 Prepare professionals in radiologic
sciences for an increasingly quantitative
future.
Essentials for Collaboration
• Climate of trust
• An outcome that is valued by and
brings value to each group
• Shared ownership of the final
product or service.
• A conviction that we need each
other to be successful.
Quantitative Imaging Initiatives
• Quantitative Imaging Biomarkers
Alliance (QIBA)
• Imaging Biomarkers Roundtable
• CTSA Imaging Working Group
– Uniform Protocols for Imaging in Clinical
Trials (UPICT)
• Toward Quantitative Imaging (TQI)
– “Quantitative Imaging Reading Room” at
RSNA Annual Mtg.
http://www.rsna.org/About/upload/2010-2013_RSNA_Strategic_Plan.pdf
QIBA Background
• Began May, 2008
• Mission: Improve value and practicality
of quantitative imaging biomarkers by
reducing variability across devices,
patients, and time.
– Build “measuring devices” rather than
“imaging devices”.
QIBA Profile (2)
• Part 2 tells a vendor what they must
implement in their product to state
compliance with the Profile. (Details)
– E.g. to comply, the scanner must be able to:
» scan a Mark-324 Chest Phantom, identify the smallest
resolvable target, display the diameter of that target
» demonstrate resolving targets at least as small as
2mm diameter on the Mark-324 phantom
» scan patients according to the ACRIN NLST
acquisition protocol
– E.g. to comply, the quantification application
must be able to:
» segment a nodule (automatically or manually), derive
the volume, store it in a DICOM object
» run a user through a set of test data with known
volumes and at the end display an accuracy score
A QIBA Profile is a document with 3 parts
• Part 1 tells a user what can be accomplished
by following the Profile. (Claims)
– E.g. you will be able to detect volume
changes of greater than 10% in Stage I
lung cancer nodules which are 5mm in
diameter or greater.
QIBA Profile (3)
• Part 3 tells the user staff what they must do
for the Profile Claims to be realized. (Details)
– E.g. to comply, the site CT techs must be able
to:
» scan the patient within 10 minutes of contrast
injection
– E.g. to comply, the radiologist must be able to:
» achieve a score of 95% or better using their
segmentation application on the LIDC test set.
How we will succeed
QIBA Committee Leadership
Steering Committee
CT Quantitative Committee
Dan Sullivan, MD, Co-Chair (Duke; RSNA)
Andy Buckler, MS, Co-Chair (Buckler Biomedical LLC)
Kevin O’Donnell, PhD, Co-Chair (Toshiba)
Andrew Buckler, MS, Chair (Buckler Biomedical LLC)
P. David Mozley, MD, Co-Chair (Merck)
Lawrence Schwartz, MD, Co-Chair (Memorial Sloan-Kettering
Cancer Center)
Nicholas Petrick, PhD, Group 1A Subcommittee Chair (FDA)
Michael McNitt-Gray, PhD, Group 1B Subcommittee Chair
(UCLA)
Charles Fenimore, PhD, Group 1C Subcommittee Chair
(National Institute of Standards and Technology (NIST)
Anthony Reeves, PhD, Volcano (Cornell University)
PET/CT Quantitative Committee
Richard Frank, MD, PhD, Chair (GE Healthcare)
Richard Wahl, MD PhD, Co-Chair (Johns Hopkins)
Paul Kinahan, PhD, Co-Chair (University of
Washington)
David Clunie, MBBS, Subcommittee Co-Chair
(RadPharm)
Jeffrey Yap, PhD, Quality Control Metrics and
Covariates Rationale Technical Subcommittee Chair
(Dana Farber Cancer Institute)
Timothy Turkington, PhD, ROI Definitions Technical
Subcommittee Chair (Duke University Medical Center)
Ling X. Shao, PhD, Software Tracking Technical
Subcommittee Chair (Philips Healthcare)
COPD/Asthma Committee
Phil Judy, PhD, Chair (Brigham & Women’s)
David Lynch, MD, Co-Chair (National Jewish)
MRI Quantitative Committee
Gudrun Zahlmann, PhD, Chair (Roche)
Sandeep Gupta PhD, Co-Chair (GEHC)
Ed Jackson, PhD, Co-Chair (MD Anderson Cancer Center)
Mark Rosen, MD, PhD, Clinical Test-Retest Subcommittee
(UPenn)
Daniel Barboriak, MD, Data Simulation (Synthetic Data)
Subcommittee (Duke University Medical Center)
fMRI Committee
With basic scientists:
Facilitate funding and structured approach for
assessment of accuracy and precision.
With physicians:
Include treating physicians in developing Profile
Claims.
Provide standards for evaluating imaging
biomarkers in clinical trials.
Facilitate validated biomarkers for use in clinical
care.
With industry:
Cathy Elsinger, PhD, Co-Chair (Nordic Neurolab, Inc)
Jeffrey Petrella, MD, Co-Chair (Duke)
Joy Hirsch, PhD, Co-Chair (Columbia)
Make it actionable for engineering and
R&D, addressing both design and use
How we will succeed with industry
Result:
Widely Available, High Performance, Quantitative
Imaging
PRODUCT CREATION
PROCESS (PCP)
Customer
Requirements
Specification
(CRS)
System
Requirements
Specification
(SRS)
QIBA PROFILE
I. CLINICAL CONTEXT
II. CLAIMS
III. PROFILE DETAILS
IV. COMPLIANCE SECTION
V. ACKNOWLEDGEMENTS
Verification Plan
and Protocol
Participation and
visibility
Imaging Science, Metrology, and Biostatistics
May 2010
Buckler Biomedical LLC
19
May 2010
Buckler Biomedical LLC
20
Make it familiar to marketing and give
them a product, not just a cost
FDA Approval Vs. FDA Qualification
Familiar models:
QI
®
Approval (or Clearance) is acknowledgement
that, for the stated claim, the drug or device
has been shown to have acceptable safety
and effectiveness.
Qualification is acknowledgement that, within a
stated context of use, the measurement can
be relied upon to have a specific
interpretation in drug development and
regulatory decision-making.
BA
Familiar process:
May 2010
Buckler Biomedical LLC
21
Quantitative Imaging Test
Discovery/Development/Technical
Performance
[Private & Academic Sectors]
Quantitative Imaging
Test Drug or Device
Approval
[National regulatory
agencies, e.g., FDA
CDRH or CDER]
Clinica
l Trial
Data
How will we succeed with regulators?
Quantitative Imaging
Test Result
Qualification
[National regulatory
agencies, e.g., FDA
CDER]
Evidentiary Studies for
Coverage Decisions
[Payer organizations,
e.g., CMS]
Use in Routine
Clinical Care
Use in Drug
Development
• Provide standards
• Adopt/adapt their paradigms
• Aggregate data and perform metaanalyses
Proposed Imaging Biomarker Qualification Process
Sponsoring Collaborative
A) Declaratory information about the class of
tests drawn from test validation sources.
National Regulatory Agencies
Request
Letter
3) BQC evaluates qualification request.
4) Biomarker Qualification Management Team
(BQMT) accepts or declines the sponsor’s request to
proceed with qualification process.
5) Biomarker Qualification Review Team (BQRT)
requests briefing document from biomarker sponsor.
Briefing
Document
6) BQ Project Manger schedules face-to-face meeting
between the sponsor and the BQRT.
7) BQRT evaluates the briefing document and prepares
for the Biomarker Qualification face-to-face meeting.
8) BQRT and Sponsor BQDS Meeting.
F) Clinical Efficacy Groundwork to qualify
biomarker as a surrogate endpoint in "real world"
imaging conditions
G) Draft advice guidance on incorporation of
imaging biomarker into clinical trials.
CONSULTATION AND ADVICE PROCESS
2) Biomarker Sponsor submits to BQC a written
request for qualification of an exploratory biomarker.
B) Phantom and other controlled condition
support material for “stand-alone” assessment
and required initial and ongoing quality control
specifics.
C) Implement and refine protocols for the
intended use
D) Process map detailing steps contemplated to
support qualification of the biomarker
E) Clinical Performance Groundwork to
characterize sensitivity and specificity for readers
using the imaging test when interpreted as a
biomarker under limited conditions.
1) Informal discussion of a potential biomarker sponsor
with the Biomarker Qualification Coordinator (BQC).
Imaging Biomarkers
Roundtable
Your Logo
Here
9) BQRT identifies and requests additional data from
sponsor.
Full Data
Package
10) BQRT receives full data package and review period
begins
FNIH
11) BQRT writes draft biomarker qualification review.
13) BQ Project Manager schedules the BQ review for
presentation at a CDER Regulatory Briefing.
14) CDER Regulatory Briefing presentation and
discussion is held.
REVIEW PROCESS
12) BQC routes the draft biomarker qualification
reviews to all Offices
15) CDER Office Directors make decisions to accept or
reject the BQRT recommendations.
H) Promote use of the imaging biomarker through
education.
Signoff
Letter
16) BQC drafts letter for sign-off by the Director of
CDER communicating to the sponsor the results of the
biomarker qualification.
UPICT Template Headings
UPICT Imaging Protocol Template
•Executive Summary
•Context of the Imaging Protocol within the Clinical Trial
•Site Selection, Qualification and Training
•Subject Scheduling
•Subject Preparation
•Imaging-related Substance Preparation and Administration
•Individual Subject Imaging-related Quality Control
•Imaging Procedure
•Image Post-processing
•Image Analysis
•Image Interpretation
•Archival and Distribution of Data
•Quality Control
•Imaging-associated Risks and Risk Management
APPENDICES
UPICT Template
UPICT Template Imaging Protocol
•Executive Summary
•Context of the Imaging Protocol within the Clinical Trial
•Site Selection, Qualification and Training
•Subject Scheduling
•Subject Preparation
•Imaging-related Substance Preparation and Administration
•Individual Subject Imaging-related Quality Control
•Imaging Procedure
•Image Post-processing
•Image Analysis
•Image Interpretation
•Archival and Distribution of Data
•Quality Control
•Imaging-associated Risks and Risk Management
APPENDICES
Summary
What can we learn from each other?
• Understand each other’s needs (what
key missing component will help
accomplish their job?)
• Satisfy each other’s
motivations/agendas
• Help each other achieve their
rewards/incentives