Welcome
Follow us and stay connected
through social media:
@mySCRS and #SiteSolutionsSummit
Site Solutions Summit Networkers Group
Disclaimer
• Check with your professional consultants
(legal, financial, HR, etc.) to obtain specific
information pertaining to your company.
• This information is not meant to direct you on
how to address any specific issues.
• Antitrust–compliance policy adherence.
http://myscrs.org/24-revision-v1/policies/antitrust-compliance-policy/
Beach Rules
• Picnic lunch on the beach
• Sponsor & CRO representative at each table
• Table 45 changed to Eli Lilly
• Map in your program – pg 8 & 9
• Roundtables
• 1:45 – 2:30
• 2:45 – 3:30
• 3:45 – 4:30
• Receptions & Eagle Award Gala
• Meet up for breakfast 8 – 9:20am
Goggle was originally named?
•
•
•
•
Find It
Sum Up
Back Rub
Dig It
Successful people are always looking
for opportunities to help others.
Unsuccessful people are always
asking, “What’s in it for me?”
- Brian Tracy
What We All Know
• Too Long
• Too Expensive
• Too Unpredictable
What We All Want
Time
Predictability
Cost
Quality
What We All Need
Quality Data
Reliability
Speed
Enrollment
Cycle Time for Multisite Clinical Trials in
the United States
CTTI – Diana Abbott, Robert Califf, Briggs Morrison, Jean Bolte, Swati Chakraborty
•
•
•
•
•
Retrospective study
19 organizations submitted data
5,396 studies were submitted
10,673 line of data submitted
About ½ of the data unusable
– despite some steep intervention
Conclusion
• We are not collecting standardized measures
of cycle times … to support evidence-base
health care, standard measures should be
developed…
11
Where We All Need To Go - Together
• Establish Global Industry Standards
– Sites and the Rest of You Too
• Determine What’s Important
– Compliance & Business Operations
– Relationship of the metrics
• Then we’ll talk about all the rest…
– i.e. accreditation, etc.
Site Terminology & Metrics
Business Foundation
• Human Resource
• Facility & Equipment
• Business Development &
Marketing
• Quality
• Finances
• Commitment
Site Terminology & Metrics
Opportunity
• First learn of the study –
study decision for your site
– Feasibility
– Pre-site Visit
Site Terminology & Metrics
Pre-Study
• Site Selection – Study Activation
–
–
–
–
–
–
–
Budget & Contract
Regulatory Submission
Recruitment Plan
Training
PI Meeting
Source Document Development
Equipment/Supplies/IP
Management
Site Terminology & Metrics
Enrollment
• Site Activation-Study Enrollment Ends
–
–
–
–
–
–
Screening
Consenting
Randomization
Retention
Data Management
Control of
• Supplies/inventory
• IP
• Regulatory Management
Site Terminology & Metrics
Follow-Up
• Enrollment Ends – LPLV
–
–
–
–
–
Retention
Data management
Control of inventory
Control of IP
Regulatory management
Site Terminology & Metrics
Close-Out
• LPLV - Study Close Our Visit
– Data Management
– Control/Return of
Inventory/IP
– Regulatory close out
– Achieved
5 Standard Site Terms
1.
2.
3.
4.
5.
Opportunity
Pre-Study
Enrolling
Follow-Up
Close-Out
Summit Survey Demographics
Academic Sites
Hospital Sites
Private Practices
Sand-alone research centers
8%
29%
39%
25%
______________________________________________________________________________________
Investigators
Site Manager/Director/VP
Coordinators
39%
26%
19%
Geographic Distribution
•
•
•
•
•
North America 54%
Europe
30%
South America 8%
Asia-Pacific
3%
Other
5%
Summit Survey Results
354 sites
• 29% SCRS members
• 20% Summit attendees, but not SCRS members
Site in business
Overall
SCRS Member
Non-Member
5 yrs. or les
17%
16%
19%
6-10 yrs.
27%
22%
29%
11 + yrs.
56%
62%
52%
Statically, SCRS members have sites that have been in business for
more years than non-SCRS members (χ²: 7.89; p=0.05)
Data Referenced Beyond The
Summit Survey
• Provided un-blinded
Business Foundation – Step 1
• Human Resource
• Facility & Equipment
• Business Development &
Marketing
• Quality
• Finances
• Commitment
Interviewing Practice
Always check references
2012 78%
2013 60%
Site Solutions Summit Survey 2012 n = 142 & Site Solutions Summit 2013 n=146
Now That They’re Hired
2012
2013
Have a formal orientation
program
62%
51%
Do not always follow the
orientation program
22%
73%
Site Solutions Summit Survey 2012 n = 142 & Site Solutions Summit 2013 n = 146
Certification of Site Staff
Sponsor/CRO
Sites that
require SC
certification
Sites that
require SC
certification
2012
2012
2013
Sites that
require
Investigator
certification
2013
Would prefer to
place studies at
sites with
certified SCs
63%
29%
37%
37%
Would prefer to
place studies at
sites with
certified
Investigators
65%
---
---
---
54% - 2011
Site Solutions Summit Sponsor/CRO Survey 2011 n = 35, 2012 n=151 & Site
Solutions Summit Site Survey 2011 n = 141 & Site Solutions Summit 2013 n = 129
Sites That Have a QA, Auditing Process
• 74%
QA ongoing studies
37% Always
29% Often
23% Sometimes
11% Rarely
1% Never
Site Solutions Summit Survey 2012 n = 115
Best Describes Your QA System
•
•
•
•
•
37%
24%
14%
6%
4%
Staff review each others work
One designated person
Work closely with the CRAs
Want to, but can’t fine the money
Hire an outside firm
Site Solutions Summit Survey 2012 n = 115
Percent of Sites You Work With That
You Think Are Audit Ready?
• <25%
32%
• 26-50%
32%
• 51-75%
29%
• 75%+
Sponsor/CRO Site Solutions Summit Survey 2012, n=149
7%
Opportunity – Step 2
• First learn of the study –
study decision for your site
– Feasibility
– Pre-site Visit
EMR Realties
• 51% Have an EMR system
• 78% Use it during the feasibility process
to identify potential subjects
– 96% SCRS Members use it during feasibility
• 44% Flag identified potential records
–
–
–
–
27% don’t have that capabilities
22% do not flag charts
8% are unsure if they do
67% SCRS members flag charts
Site Solutions Summit 2013 n=181
Feasibility Investment
Overall
2 yrs. or less
3-5 yrs.
6-10 yrs.
11+ yrs.
<30 minute
10%
0
13%
17%
7%
30-60
Minutes
31%
33%
39%
31%
32%
1-2 hours
20%
17%
22%
15%
20%
2-3 hours
19%
8%
13%
21%
19%
3+ hours
19%
42%
13%
15%
19%
Site Solutions Summit 2013 n =181
Conversion Rate
2013 80% of studies pursued were awarded
2012 70% of studies pursued were awarded
Site Solutions Summit Survey 2012 n=119, 2013 n = 181
Study Load
Studies Initiate
2012
Overall
SCRS Members
Non-Members
<4
31%
19%
37%
5 – 10
40%
36%
41%
11- 20
17%
28%
12%
21+
12%
17%
10%
Statistically, a greater number of SCRS member sites were
likely to be involved in higher study loads (>11) than nonSCRS members (χ²: 11.98, p<0.05)
Site Solutions Summit 2013 n = 181
Pre-Study – Step 2
• Site Selection – Study Activation
–
–
–
–
–
–
–
Budget & Contract
Regulatory Submission
Recruitment Plan
Training
PI Meeting
Source Document Development
Equipment/Supplies/IP
Management
Cycle Times (days)
All
SCRS Members
Non Members
IRB contact to site
activation*
109
83
223
Pre-site visit to site
activation
75
76
74
Study activation to
FPFV
32
29
35
Study activation to
LPLV
317
335
303
*Unclear if the IRB contact is submission or approval, those that reported the name of the IRB mostly listed central IRBs
2013 SCRS Survey
Summary: IRB, Site Activation, Visits
• Statistically, SCRS members were more likely
to be quicker from IRB contact to site
activation than non-SCRS members (r=0.65,
p<0.05)
• Statistically, there was no difference in the
cycle times between SCRS and non-SCRS
members for other aspects of site action and
patient visits. (r=0.12, p>0.05)
Let’s Talk About It
Multi Tasking Budget, Contract & Reg
Docs Simultaneously
• Yes
• Sometime
• No
Site Solutions Summit 2013 n =108
61%
31%
8%
On average how many working days
does it take a site to return a budget &
contract? – Sponsors & CROs Only
•
•
•
•
•
0-5 days
6-10 days
11-15 days
16-21 days
21+ days
Budget & Contract Cycle Time
Working
Days
Overall
SCRS Member
2 yrs. or less
exp
11+ yrs. exp
0-5
41%
44%
25%
42%
6-10
25%
31%
37%
27%
11-15
21%
19%
16-20
5%
0
21+
8%
1%
Site Solutions Summit 2013 n = 108
Implication of Cross Indemnification
•
•
•
•
No extra time required
More than 2 weeks
More than a month
Deal breaker, won’t sign
– SCRS Members
– Non member
Site Solutions Summit 2013 n=108
47%
32%
9%
12%
19%
8%
Accepts First Study Budget Offer
Overall
SCRS
Member
Non
Member
< 5 yrs.
11+ yrs.
Never
23%
25%
20%
14%
28%
Less than
50%
30%
36%
29%
33%
30%
More than
50%
43%
39%
41%
45%
39%
Always
7%
0%
10%
29%
3%
Site Solutions Summit 2013 n=106
What Do You Think?
Best Negotiation Tips
1. Provide additional justification
of costs
2. Provide additional justification
of staff costs
3. Escalate to a more senior staff person
4. Have the PI involved & escalate the issue
5. Provide past performance metrics
6. Walk away (or indicate strongly you will)
How Are Your Negotiation Skills?
Negotiate
above initial
offer
Overall
SCRS Member
2 yrs. or less
exp
11+ yrs. exp
0%
6%
0
0
5%
1-10%
22%
22%
13%
25%
11-15%
32%
36%
25%
28%
16-20%
30%
22%
25%
34%
21%+
9%
20%
37%
8%
Site Solutions Summit n=141
Successful Negotiation Actions
• 51% Provide additional justification of costs
• 17% Provide additional justification of staff costs
• 13% Have the PI involved & escalate the issue
• 11% Provide past performance metrics
Site Solutions Summit 2013 n = 181
Non-Refundable Payment,
Why?
1. Lack of monthly payment
and this is a way to front
load cash requirement
2. To recuperate start up costs
3. Seldom request it because we can never get it
Now The Truth
Non-Refundable Payment
64%
To recuperate start up costs
18%
Lack of monthly payment and this is way to front load
cash requirement
8%
Seldom request, we can never get it
SCRS Members
5%
Non members
11%
Site Solutions Summit n=108
First Payment
• 57% non-refundable/administrative fee
• 24% have “refundable” advances
CFS Financial n = 11,472 studies
Non-Refundable Start Up Received
•
•
•
•
•
•
2006
2007
2008
2009
2011
2013
• SCRS Member
• Non-Member
$2,367
$1,381
$2,550
$2,740
$3,205
$2,927
$3,323
$2,927
Site Solutions Summit Survey 2007, 2008, 2009 & 2010, n=1,260 studies,
2011 =800 studies, 2013 Summit Survey n= 1,181 studies
Average Cost per Visit
across 10 different therapeutic categories
• $809 (includes vaccines)
• $880 (excludes vaccines)
Both exclude screen failures and discontinues
CFS Financial n = 11,427
Invoicing Practices
Of interest – the sites in business the most years, with
the highest GR and doing the most studies, invoice the
least.
Frequency
Always
Sometimes
Never
Often
Rarely
11%
19%
23%
25%
22%
2012 Site Solutions Summit Survey n= 125 sites
Site Solutions Summit 2012 n=125
26% all payments
processed are
invoiceable items & not
driven by EDC
~5 different invoiceable
line items on average per
CTA
~ $1,168 per study
2013 CFS Clinical n = 11,427 studies
Enrollment – Step 3
• Site Activation-Study Enrollment Ends
–
–
–
–
–
–
Screening
Consenting
Randomization
Retention
Data Management
Control of
• Supplies/inventory
• IP
• Regulatory Management
Enrollment Cycle Times
All
SCRS
Members
Non-Members
Average # of subjects
enrolled (consented)
39
50
30
% of times
randomized goal met
7%
6%
8%
Number of days to
meet randomized goal
from study activation
192 days
180 days
201 days
% of times
randomization goal
exceeded
25%
23%
26%
% randomization goal
was exceed by
69%
72%
67%
2013 SCRS Survey n= ~2,000 studies
Summary: Enrollment Cycle
• SCRS members enrolled more patients (Mean=50.24,
SD=12.46) than non-SCRS members (Mean=30.48, SD=14.34).
This difference was statistically significant (p=0.024)
• SCRS members utilized a lower number of days to meet
randomized goal from study activation (Mean=180.20,
SD=140.36) than non-SCRS members (Mean=201.09,
SD=203.49). This difference was statistically significant
(p=0.048)
• There was no difference between SCRS and non-SCRS
members over %times randomized goals met, exceeded and
% exceeded by
Agree Neutral Disagree
1.
Site pay greater attention to enrolling
studies that have monthly payment terms.
2.
Slow payment terms & withholding
money earned places our site in
financial stress.
3.
When the contract states the sites is paid on a ratio of screen
failure to randomize subjects and the site has not had great
success in randomization g a number of consented subject they
will stop screening.
Is It Time To Look at Lackluster Enrollment
from a Different Perspective?
Agree
Neutral
Disagree
Site pay greater attention to enrolling
studies that have monthly payment terms.
51%
34%
22%
Slow payment terms & withholding
money earned places our site in financial
stress.
71%
31%
6%
When the contract states the sites is paid
on a ratio of screen failure to randomize
subjects and the site has not had great
success in randomizing a number of
consented subject they will stop
screening
30%
46%
31%
Site Solutions Summit 2013 n =108
Data Entry
Overall
SCRS Members
Non-Members
Day of the visit
29%
26%
31%
Within 3 days of
visit
60%
70%
55%
4-10 day of the visit
8%
4%
10%
No policy, when we
have time
3%
0
4%
Site Solutions Summit 2013 n=146
Average # of Days From Visit to
Data Entry
• 45 Days
CFS Financial n=11,427
Timeliness Improves as Study Progresses
Median Time to eCRF Entry As Study Progresses
9
8
Cycle Time (Days)
7
6
5
4
3
2
1
0
0-5 Visits
6-10 Visits
11-15 Visits
16-20 Visits
Subject Visits Conducted
Source: Medidata Insights Metrics Warehouse 2013-SEP and Medidata analysis
20+ Visits
“Correlation Does Not
Imply Causation”
But it might make us think!
eCRF Entry Timeliness => Higher Enrollment …
Total Patients Enrolled
Mean eCRF Entry Cycle Time vs. Total Enrolled Patients
Mean eCRF Entry Cycle Time (Days)
Source: Medidata Insights Metrics Warehouse 2013-SEP and Medidata analysis
# 1 Pain Point for Any
Business CASH Management
Payment Terms
Monthly
Every Other Month
Quarterly
2012 Site Solutions Summit Survey 2012 n=840 studies
2012
25%
9%
46%
2013
39%
1%
37%
SCRS Supports CISCRP
The Study for Information & Study on Clinical
Research Participation
•
•
•
•
Educate the general public about clinical research
“Aware for All” Days
Hero Campaign
Pamphlets
SCRS Believes in CISCRP’s Mission
We have donated over
40K in 4 years
SCRS will continue to match 100% of all donations
Box at registration desk
Contributions are tax deducible donation
Cash, credit cards, checks
Follow-Up – Step 4
• Enrollment Ends – LPLV
–
–
–
–
–
Retention
Data management
Control of inventory
Control of IP
Regulatory management
Subject Retention Metrics
All
SCRS Members
Non-Members
Screen Failures
37%
34%
39%
Completion
42%
43%
41%
Average dropped
rate
18%
17%
18%
There was no significant difference between SCRS and non-SCRS members for
screen failures, completion and dropped rates (p>0.05)
2013 SCRS Survey
eCRF Data Correction Rate
9.0%
8.0%
7.0%
6.0%
5.0%
4.0%
3.0%
2.0%
1.0%
0.0%
Source: Medidata Insights Metrics Warehouse 2013-SEP and Medidata analysis
Response Time to Sponsor (DM) Queries (Days)
Phase 2 & 3 Studies
8
7
6
5
4
3
2
1
0
Source: Medidata Insights Metrics Warehouse 2013-SEP and Medidata analysis
Close Out – Step 5
• LPLV - Study Close Our Visit
– Data Management
– Control/Return of
Inventory/IP
– Regulatory close out
– Achieved
Withholding Trends
•
•
•
•
2006
2007
2008
2009
• 2013
• 2013
12%
13%
13%
11%
48% of CTA have holdbacks
12%
Site Solutions Summit 2007, 2008, 2009 & 2010, n=126, * CFS Financial n= 11,427 studies
Profit Margins
25
Percent
20
15
10
5
0
0-5%
Site Solutions Summit 2013 n=141
6-10%
16-20%
21-25%
A Request to Industry
56% A post study site report card would be
extremely valuable
Receiving a post study site report card would..
28% Definitely change the conduct of our work
67% Probably change the conduct of our work
Site Solutions Summit Survey 2013
“The way to get started is to
quit talking and begin doing.”
Walt Disney