Some Ideas for CTAC
better, faster, cheaper
David M. Dilts, PhD MBA CPA CMA
Managing Partner, Dilts+Partners, LLC
Professor, Healthcare Management, Oregon Health & Science University
Portland, Oregon, USA
dilts@diltspartners.com
1
When…
• you hear “Australian clinical trials” what is the
first thing that comes to mind?
2
Australia is making a great
deal of progress
• The Clinical Trials Action Group (CTAG)
• 2013 McKeon Report
• The Federal Budget Measures
– Implement the CTAG recommendations
– Streamline ethics review
• The CTAC
• The Clinical Trials Jurisdictional Working Group
(CTJWG)
3
These Groups have come up with some
great ideas
• Streamlining Ethics Review
• “Drop-dead” dates (i.e., have it done in 30-60
days)
• Interoperability of systems
• Australian-wide standards, including
standardized cost models
• Identifying strengths for running clinical
trials in Australia
4
Special but not unique..
• Some of the forces you are facing
– Increasing competition
– Increased complexity in clinical trial design
• More exclusion/inclusion criteria, more secondary endpoints, more correlatives
– “complex, time consuming, & costly approvals process for ethics and
governance”
– Insufficient and/or slow accruals to trails
– Diverse, heterogeneous and incompatible
• Information systems
• Clinical trial cost models
• Clinical trial practice
• So,
– What you need to be more efficient & more effective
– Both locally and internationally
5
Things Right and the Right Things
• Efficiency
– Doing things right
• Effectiveness
– Doing the right things
6
Three dimensions of comparison
• Better, faster, cheaper
• Cautions:
– Trying to do all three is confusing and nearly impossible in
most situations, so you need to focus
– So, key is to set priorities on which 1 (or at most 2)
– Right off the bat, the developed world will not be
cheaper
• Unless CROs start thinking about “Total Cost of
Ownership”, which is not likely
– Deal breaker: not using data to know, to measure
performance and to tell people
• Finally, will doing this achieve the desired objective?
7
An Integrated Approach
Mission
&
Strategy
Consequential
Impact
Operational
Issues
Resource
Allocation
Synchronicity
8
Mission & Strategy
Mission
&
Strategy
Consequential
Impact
Operational
Issues
Resource
Allocation
Synchronicity
9
Mission
When someone says “Australian wines?”
“Australia is one of the world powers of wine. The wine
industry of Australia is perhaps the most technologically
advanced, forward-thinking on earth, and the success of
Australian wines around the world is the envy of wine
producers in many on the countries.”
Wine for Dummies, 5th edition
10
So…
1. In what ways is Australia uniquely qualified to
answer the worlds call with respect to clinical
research?
2. On what dimension(s) do you want Australia to
be recognized and envied by other countries
with respect to clinical research?
Mission
One possible dimension: Variance Reduction
Blue – SRC time
Red - PI holding time
Green – Ethics time
High Variance = High Uncertainty = High Dissatisfaction
Maister The Psychology of Waiting Lines
Mission
So…
• When I asked folks in NSW and Victoria earlier
this year what made their trials unique
– Most common comment: the diversity of the
population
• But….
– However, one really interesting comment:
• Australia is a leading indicator (compared to the rest of
the world) for one condition
• Another researcher said….
14
Mission
Another interesting dimension for Australia
What
happened?
Australia is # 3 for men and # 7 for women in latest stats
15
There are a host of different
possible dimensions
Ones for Quality
Product
Service
Professional Quality
•Aesthetics
•Conformance
•Durability
•Features
•Perceived Quality
•Performance
•Reliability
•Robustness
•Serviceability
•Access
•Assurance
•Communication
•Competence
•Convenience
•Courtesy
•Credibility
•Empathy
•Personalization
•Responsiveness
•Security
•Tangibles
•Auxiliary Information
•Location
•Personalization
•Professional interaction
•Professionalism
•Reputation
•Staff interactions/
competence
Overarching:
1. Economics 2. Reliability
Mission
Key to success
• Once the critical dimension(s) have been
selected:
– Communicate, communicate, communicate
– Verify and assure goal alignment
– Manage the process using project
management skills
Quality /
Speed
100%
75%
Qualifier
50%
Knowledge /
Reputation
25%
0%
Value /
Incentives
Access /
Adaptability
Capabilities /
Technologies
17
Operational Issues
Mission
&
Strategy
Consequential
Impact
Operational
Issues
Resource
Allocation
Synchronicity
18
Operational
Time for Opening a
Phase III Cooperative Group Trial
Median: 784 to 808 days*
Range: 435-1604 days
Median: 116 to 252 days*
Range: 21-836 days
Total Median Time from idea to opening~920 days (2.5 years)
Range: 456 – 2440 days (1.25 - 6.7 yrs)
* Depending upon site, based on the Phase III trials studied
Dilts DM et al, “Chutes and Ladders”, Clin Can Res, 2010 16(22): 5361-69
19
Operational
Critical Findings
• No matter how much the PIs complained, Ethics review
was never the rate limiting process
– Budget and contract reviews were much longer
– Note: be careful of floating-bottlenecks
• No one knew the scope of the problem using data
– But everyone had an opinion
– i.e., Eminence- not evidence-based decision making
• The variance was totally unexpected
• Everyone wanted to share data & systems
– So long as it was their system that was selected
– Interoperability is a hard problem
20
Operational
Results using evidence (1)
Accruals Per Trial1
Six Major NCI Comprehensive Center Centers
Accrual Per
Trial2
CCC 1 CCC 2
CCC 3 CCC 4
CCC 5
CCC 6
Total
Time Period
1/2001-7/2005
1/2000-9/2006 1/2000-12/2005 1/2000-4/2007 1/2002-12/2008 1/2000-3/2009
N
148
323
104
323
393
0
20.9%
26.9%
26.9%
34.4%
22.1%
35.1% 29.0%3
1 to 4
32.4%
31.0%
26.9%
31.3%
29.8%
38.1% 32.6%
5 or more
46.6%
42.1%
46.2%
34.4%
48.1%
26.8% 38.4%
496
1,787
1Excludes
pediatric studies; Therapeutic Studies Only
per trial at time of closure
3Over 500 of nearly 1800 trials result in zero accruals
2Accruals
21
Operational
Results Using Evidence (2)
Phase III ECOG Studies Closed to Accrual (n=15*): Ratio of
Actual Accruals vs. Expected Accrual
1.2
Ratio of Accruals (Actual / Expected)
1
0.8
0.6
0.4
0.2
0
a
b
c
d
e
f
g
h
studies
i
j **
k
l **
m
n
o
•All phase III studies activated and closed to accrual between 1/2000 – 7/2006
•Color Code:
• red : studies taking greater than the median time to open
• blue: studies taking less than the median time to open
• gray: studies closed due to reasons other than poor accrual
Dilts DM et al, “Chutes and Ladders”, Clin Can Res, 2010 16(22): 5361-69
22
Resource Allocation
Mission
&
Strategy
Consequential
Impact
Operational
Issues
Resource
Allocation
Synchronicity
23
Using a standard, “plain” trial, the
differences in start-up costs
Comparison Budget of Four Branches by Start-up Cost Categories
CHM
CRM
$6,000
$400 $2,200 $750 $1,850
$3,500
PHO
$2,900
$400 $2,200 $750 $1,500
$8,000
$400 $2,200 $750 $1,500
Prostate
MIN
$19,304
$3,500
$2,200 $750
MAX
$2,200 $750
~$22,500 difference in start-up costs
$19,304
Site Specific Startup
Cancer Institute Admin Start Up Fee
$2,900
CRRC Review Fee
Initial IRB Review
$400 $2,200 $750 $1,850
Pharmacy Setup
Closeout
24
Resource Allocation
Using a standard, “plain” trial, the
differences in per-pt cost
Comparison Budget of Four Branches by Patient Cost Categories
CHM $70
$2,880
CRM$66.50
PHO $106
Prostate $140
MIN $70
MAX $140
$1,905
$4,563
$75
$500
$2,857.50
$3,000
$3,000
$2,857.50
$3,000
$4,563
$220
$3,037.50
$3,090
$1,408
$500
$3,016
$4,062.50
$3,081.25
$4,050
$816
$1,200 $682.50
$2,125
$2,834
$75
$3,087
$150
$275
~$11,000 difference per pt
$816 $1,200$150
$3,081.25
$3,281.25
$4,062.50
$3,037.50
Study Drug (MDV3100) dispense
12-Lead ECG (triplicate)
Blood Draws & Procedures
Survival status/phone contact
PI Fee
Research RN
Study Coordinator/Regulatory/Data Management Fee
Finance Administration
$3,281.25
$682.50
25
Resource Allocation
Make “League Standings” Public
Different “Branches”
•
•
Percent of Clinic FTE
Line denotes average FTE
26
Resource Allocation
Another “League Standing” Method
Analysis of inputs versus outputs
Total NIH R01 Funding $K vs. CTSA
Annual Support ($k, 2010)
9000
$500,000
8000
$450,000
7000
$400,000
6000
$350,000
Output: R01 $k
Output: Number of Publications (2010 in PubMed)
Total PubMed Pubs vs.
CTSA Annual Support ($k, 2010)
5000
4000
3000
$300,000
$250,000
$200,000
$150,000
2000
$100,000
1000
$50,000
0
$-
$5,000
$10,000
$15,000
$20,000
Input: Annual CTSA $k/yr
$25,000
$$-
$5,000
Note: these are only example inputs/outputs
Key: what are your critical inputs & outcomes?
DEA Paper forthcoming
$10,000 $15,000 $20,000
Input: Annual CTSA $k/yr
$25,000
27
Consequential Impact
Mission
&
Strategy
Consequential
Impact
Operational
Issues
Resource
Allocation
Synchronicity
The most difficult of all the tasks to do
28
Consequential Impact
Evaluation of a Portfolio on Strategic Dimensions
(n=161)
Blue = anticipated (desired) Red = actual
29
Synchronicity
Mission
&
Strategy
Consequential
Impact
Operational
Issues
Resource
Allocation
Synchronicity
30
Synchronicity
What happens if we double a
cooperative group’s budget?
Answer: nearly nothing
…But they wouldn’t turn the money down
31
Synchronicity
High Level Process Flow for
Phase III Studies
32
Synchronicity
Simulation Results of Working Together
/ st. err
* Simulation period defined over a period of 5 years (1825 Calendar Days)
* Note: Axes on the Timing Distribution Graphs are different
33
Synchronicity
Synchronizing Data Capture
Screening
Diagnostics
Treatment
Planning
Treatment
Follow-up
• Process for flow is fairly consistent across sites
BUT
• Data elements are captured and utilized inconsistently across
sites at multiple points in the workflow
duplication  missing  responsibility  source/location
Dilts, DM; Cheng, SK; et al. “Developing the Next-Generation Cancer Care System Around the Patient and the Provider:
Analysis of Workflow and Addressing Pain Points” (Under Review)
34
Synchronicity
Synchronicity is where powerful metrics
can be developed
• Total Time reduction:
–
–
–
–
To open a trial
To complete a trial
When the next trial opening will be available
Time to: publication, regulatory approval, etc.
• Total Quality Improvement
– Measurable quality improvements among all sites
– Demonstrated better quality than other countries
• Variance Reduction
– Intra- and inter-institutional
35
An Integrated Approach
Mission
&
Strategy
Consequential
Impact
Operational
Issues
Resource
Allocation
Synchronicity
36
What happens using this framework?
The Portfolio became more aligned with the Vision & Mission,
resulting in:
• 52% reduction in number of studies opened
• and a 4% increase in new patients enrolled on therapeutic
trials
Patients on a more aligned and prioritized set of trials
• 20% increase in studies with complete enrollment*
• 7% decrease in studies with non-enrollment**
With metrics for transparency & embed accountability, and
• 54% reduction in median development time
* Complete enrollment = Within 75% of accrual ceiling
** Non-enrollment = below 25% of accrual ceiling
To make it work…
• Know what you will focus on:
– Efficiency and Effectiveness
• Faster or Better and achieving the desired objectives?
• Managing the process well
– Use professional project managers (and conflict coaches)
• Always use data (preferably “their” data)
• And….
38
Finally…
• In the future, when someone says “Australian
Clinical Trials” what is the first thing you want
them to think of?
40
Thank you
41
Why Transformation Efforts Fail
Kotter, 1995
1.
2.
3.
4.
5.
6.
Not establishing a great enough sense of urgency
Not creating a powerful enough guiding coalition
Lacking a vision (or a clear vision)
Undercommunicating the vision by a factor of 10
Not removing obstacles to the new vision
Not systematically planning for, and creating, short-term
wins
7. Declaring victory too soon
8. Not anchoring changes in the corporation’s culture
Resource Allocation
General Thought
Where should you spend the money:
– On specific kinds of trials, or
– On infrastructure to support lots of trials?
Key question: what best achieves your mission?
44