I do not object to people looking at their
watches when I'm speaking.
But I strongly object when they start
shaking them to make sure they are still going.
William Norman Birkett
April 14, 2009
Sponsor-CRO Relationships:
Managing Risk
Bradley Merrill Thompson
Topics
1.
2.
3.
4.
5.
6.
7.
3
ABCs of CROs
Planning
RFP and Due Diligence
Contracting
Execution of Work
Evaluation
Take Aways
I feel like Zsa Zsa Gabor’s fifth husband.
I know what I'm supposed to do
but I don't know if I can make it interesting.
Al Gore
It’s good to know the basics
politicalhumor.about.com
4
SMOs Defined
–
What is a Site Management Organization
(SMO)?
o A person that is retained as an independent
contractor of the principal investigator or
research site to provide administrative support
of the conduct of the study, such as
recruitment of the subjects, collection and
preparation of the study data and reports for
submission to the sponsor.
o A CRO may perform SMO functions
o Definition based on agency relationships
6
CROs v SMOs
Sponsor
Sponsor’s
Agent (CRO)
Researcher’s
Agent (SMO)
Research Site/
Investigator
7
Topics
1.
2.
3.
4.
5.
6.
7.
ABCs of CROs
Planning
RFP and Due Diligence
Contracting
Execution of Work
Evaluation
Take Aways
I've often wondered how some people
in positions of this kind . . .
manage without having had any acting experience.
Ronald Reagan
9
Potential Risks of Outsourcing to CROs
– Risks generally associated with reduced control
of the clinical trial process by the Sponsor
– Risks include:
o Delays in completion of studies
o Lost or poor data
o Regulatory infractions produce indirect consequences
 FDA regulations/GCPs
 HIPAA
 Fraud and Abuse
o Private litigation exposure
12
Weighing risk can be tricky
www.txrollergirls.com
13
Delegation of Authority to CRO
– Sponsors may delegate responsibility for
regulatory obligations to a CRO
– However, device sponsor remains liable
– This is different from the drug world, where
the drug regulations contemplate that
sponsors may transfer regulatory
obligations to CROs such that they assume
regulatory liability
– FDA may be coming out with guidance in
this area
14
Delegation of Authority to CRO
– CRO may have important role in meeting regulatory
responsibilities, even though obligation is not
“transferred” to CRO
– Example: Adverse Event Reporting
o Sponsor retains responsibility to report to
investigators and FDA, and the regulatory
liability for failure to do so
o CRO may still be responsible for:
Reporting AEs to Sponsor
Drafting AE descriptions
Providing summary reports of AEs
Following up with sites to ensure complete AE
information is obtained
 Safety database maintenance and/or reconciliation




- These responsibilities should be described in the
Work Order
15
Traditional Pattern of Outsourcing Integration
Strategic Alliance
Partnership
Preferred Supplier
into strategic
planning
Transactional
 Responsiveness
to requests
 Incorporates client
 Establish core team
 Builds collaborative
mechanisms
 Develops
understanding of
client’s customers
16
Shares common
goal/objectives

Engages in joint
strategic planning

Forms
relationship-based
structure

Makes openended
agreements

New business
forms
 Assigns dedicated
relationship
management
 Makes directed
investments
 Builds collaborative
mechanisms
Source: August 2004, Goldman Sachs Research

Checks and Balances
Primary Vendor
– Clinical research
operations
 Protocol Development
 Site/Investigator Recruitment
and Selection
 Personnel Training
 Clinical Trial Management
 Data Analysis
 Medical Writing
 Processing and Preparing of
Regulatory Filings and liaison
with Regulatory bodies
– QA and QC
17
Secondary Vendors
– Monitoring
– Systems auditing
– Trial auditing
– Contract negotiations
Topics
1.
2.
3.
4.
5.
6.
7.
18
ABCs of CROs
Planning
RFP and Due Diligence
Contracting
Execution of Work
Evaluation
Take Aways.
Politics gives guys so much power that
they tend to behave badly around women.
And I hope I never get into that.
Bill Clinton
CRO Selection
– Business and legal risks will be minimized when the
Sponsor-CRO relationship is based on open
communication and trust
– Imperative that Sponsor adopt a CRO selection
process that assesses:
o Expertise: Does the CRO have the expertise to
undertake duties Sponsor desires?
o Compatibility: Can the CRO fulfill its duties in
manner that is compatible with the Sponsor (i.e., is
there a good fit)?
o Capacity: Does the CRO have the ability to commit
the manpower and resources needed in the
Sponsor’s timeframe?
o Price: Does the CRO provide services at same price
as competitors? If not, what added value is being
offered?
19
CRO Selection
– Sponsor usually conducts an RFP process tailored
to the specific trial at hand
– Sponsor’s due diligence should include:
o
o
o
o
o
Web research
Public directories
Careful analysis of Responses to RFP
Interviews with operational leaders
Site visits
– Information gleaned by Sponsor during the CRO
selection process should be communicated to
counsel to ensure that CRO’s verbal claims are
reflected in the CRO contract
20
QA Questions to Ask
– How does the CRO measure quality?
– What CRO processes are in place to
continually monitor quality?
– What is the size of the CRO quality
assurance (QA) department?
– What internal QA procedures are in
place? For example, does QA audit
studies independent of sponsor-CRO
contract, as part of internal CRO quality
control?
Winther
21
Quality is important
www.spelrite.com
22
Topics
1.
2.
3.
4.
ABC of CROs
Planning
RFP and Due Diligence
Contracting
a. Risks to manage
b. Metrics
c. 3rd Party Agreement
d. Change Orders
e. Staffing Issues
5. Execution of Work
6. Evaluation
7. Take Aways
23
A lot of this is obvious
24
Common Project Risks
– Inefficiency and quality issues from staff
changeover or inexperience
– Delays in site initiation
– Delays in patient recruitment
– Difficulties with management of clinical trial
supplies
– Issues with vendors
– Delays with the processing of CRFs
– Utilization of resources at a level that exceeds the
budget
– Delays in obtaining regulatory and IRB approvals
25
Factors Most Often Causing Study Delays in the U.S.
75%
50%
38%
31%
26%
25%
24%
24%
0%
Contract/budget
negotiation &
approval
Patient recruitment
& enrollment
Protocol
amendment &
refinement
Legal review &
approval
Review & approval
of consent form
Source: Thomson CenterWatch 2007 Survey of 522 Investigative Sites in the U.S.
26
Expect the Unexpected
27
Topics
1.
2.
3.
4.
ABCs of CROs
Planning
RFP and Due Diligence
Contracting
a. Risks to manage
b. Metrics
c. 3rd Party Agreement
d. Change Orders
e. Staffing Issues
My mother never saw the
5. Execution of Work
irony in calling me a son-of6. Evaluation
a-bitch.
7. Takeaways
Jack Nicholson
28
CRO and Sponsor Relationships
– Transparency of performance builds
trust
o Assures alignment on study quality
o Provides tools for proactive and focused
management of issues early so they
don’t become bigger problems
o Allows for shared consensus on steps for
resolution
29
Metrics in a CRO Relationship
Metric Creation
Key Performance
Indicators
Critical
Success
Factors
Metric Use
Specific
Performance
Indicators
Feedback
Senior
Committee
Discussion
Feedback &
Adjustment
Junior
Committee
CRO Engagement
Measurement Analysis
Planning
30
RFP and Due
Diligence
Contracting
Execution
of work
Evaluation
Discussion
Make metric development collaborative
www.bp3.blogger.com
31
Key Metrics for Early Identification of Project Risk
Clinical Operations
• Start-up metrics
• Enrollment metrics
• CRF backlog
Data Management
• Data entry
• Data review
• Data quality
Varawalla
32
Example Metric Definition Table
KPI
Related
Metrics
Achievement
of key
project
timelines
Final protocol
to first
patient
recruited
Visit data to
lab data
available in
database
Cuddigan & King
33
Data
Source
Target*
Number of
days from final
protocol
approval by
sponsor to first
patient
enrolled at first
site
Clinical trial
managemen
t system
(CTMS)
60 days
(mean); 90
days
(oncology)
Average
number of
days from
patient visit to
lab data
available in
CDMS
Clinical data
managemen
t system
(CDMS)
5 days
Definition
*All days specified are elapsed time
Example Metric Definition Table, contd.
KPI
Related
Metrics
Definition
Target*
Last
patient
last visit
(LPLV) to
database
lock
Number of days
from LPLV at site
to database
locked
CDMS
5 days
Database
lock to
final
study
report
Number of days
between
database lock
and final approval
study report
CTMS
21 days
*All days specified are elapsed time
34
Data
Source
Cuddigan & King
Example Metric Definition Table, contd.
KPI
Achievement
of target
quality
standards
Related
Metrics
Database
error rate
Number of
database errors
found on
database
audit/total
number of data
fields audited
Percent of
nonperforming
centers
Percent of sites
enrolled that
recruits 1 subject
within one month
of initiation visit
Cuddigan & King
35
Definition
Data Source
Data manager
CTMS
Example Metric Definition Table, contd.
KPI
Compliance
with agreed
contract value
Related
Metrics
Definition
Data Source
Final
actual
contract
cost as
percent of
initial
contract
value
Total project fees
after final invoice
paid, compared
with initial contract
value (includes
pass-through
costs)
Finance
Number
of change
orders
Number of change
orders approved
after initial
contract sign-off
Finance
Cuddigan & King
36
Metrics in a CRO Relationship
Metric Creation
Key Performance
Indicators
Critical
Success
Factors
Metric Use
Specific
Performance
Indicators
Feedback
Senior
Committee
Discussion
Junior
Committee
Discussion
Feedback &
Adjustment
CRO Engagement
Measurement
Planning
37
RFP and Due
Diligence
Contracting
Execution
of work
Analysis
Evaluation
Utility in Managing Sites
Compare sites on the basis of:
– Subjects (overall enrollment, enrollment
per month, screen fail rate, dropout rate)
– Start up (contract completion time,
document completion time)
– Data (query rate, query completion time)
– Money (overall cost per patient,
advertising cost per patient)
Winter related injuries occur more often in the winter.
-newswoman for WHIZ-TV, Zanesville, Ohio
38
How Do We Use Metrics?
– Just as you would internally, for
performance management
– Dispute Resolution (later)
– Contractually in a few cases
o Contractual management process and
dispute resolution
o Contractual rewards
o Contractual penalties
39
Topics
1.
2.
3.
4.
40
ABC of CROs
Planning
RFP and Due Diligence
Contracting
a. Risks to manage
b. Metrics
c. 3rd Party Agreement
d. Change Orders
e. Staffing Issues
Attorney: Now doctor, isn’t it
true that when a person
5. Execution of Work
dies in his sleep, he
doesn’t know about it until
6. Evaluation
the next morning?
-testimony from court
7. Takeaways
records
DSMB,
MRC, etc
Third party
vendors (e.g.,
labs)
Services
Agreements
SPONSOR
CTA
CTA
CRO*
CTAs
*Contracts on behalf
INSTITUTION/
CLINICAL SITE
INVESTIGATOR
of sponsor (donated by
dotted line)
Blue = CTA
1572
Informed
Consent
Green = Regulatory
“agreement”
**May or may not be a written
agreement - depends on
whether IRB is commercial,
local, centralized, etc
Orange = Service
Agreement
IRB**
FDA
SUBJECT
Ability to Bind Sponsor to CTAs
– Sponsor may want CRO to handle CTAs
because of:
o Speed
o Cost
o Strength of CRO’s relationships with research
sites/investigators
– However, sponsors may have problems with
CROs protecting sponsors’ interests
o CRO sometimes want to get contract in place quickly
and not expend resources in lengthy negotiations
o CRO may not have adequate legal expertise
42
Ability to Bind Sponsor to CTAs
– Contractual controls: Range of Possibilities
o CTA template
o CRO authority to negotiate
 Only extends to certain provisions
 Sponsor provides approved fall back
language and negotiation
guidance/instruction
 CRO must submit all changes to
Sponsor for review and approval
o Notification of changes and amendments
43
Topics
1.
2.
3.
4.
ABC of CROs
Planning
RFP and Due Diligence
Contracting
a.
b.
c.
d.
e.
5.
6.
7.
44
Risks to manage
Metrics
3rd Party Agreement
Change Orders
Staffing Issues
Execution of Work
Evaluation
Takeaways
Attorney: You say the stairs went
down to the basement?
Witness: Yes.
Attorney: And these stairs, did they
go up also?
-testimony from court records
Change Orders
– Statement of Work sets forth services to be
provided by CRO
o Services beyond or different from SoW are generally
subject to a “change order”
– Points to consider
o Require signed change orders before implementation
that might affect the budget–
 Require CRO to notify you if your instructions would
have the effect of changing scope
 Absent such notification, preclude cost recovery
o CRO must prepare the change order and describe in
detail changes to scope, timing, costs, personnel, etc
o CRO agrees to not unreasonably refuse change order
 Remember, you’ll always get a better price if you
address issues upfront, before need for change order
45
Topics
1.
2.
3.
4.
ABC of CROs
Planning
RFP and Due Diligence
Contracting
a. Risks to manage
b. Metrics
c. 3rd Party Agreement
d. Change Orders
e. Staffing Issues
Democracy used to be a
good thing,
5. Execution of Work
but now it has gotten into the
wrong hands.
6. Evaluation
-Senator Jesse Helms
7. Takeaways
46
Personnel
– Experience & qualification of CRO
personnel varies widely
o
o
o
o
o
Educational background
Therapeutic area experience
Monitoring experience
Training
Accreditation (e.g. ACRP, SoCRA)
– Study personnel change frequently
o Advancement
o Turnover
o Re-assignment
47
Managing Personnel
― Specify minimum qualifications.
― Change control. Retain the right to approve
at least key personnel, including additional
or replacement personnel
― Costs of change. Consider specifying
training requirements for the study team,
including any replacement personnel
o Cost of study-specific training typically included in
the budget
o Training of replacement personnel may generally
be allocated to the CRO
– Reduce impact. Institutionalize
communication, documents and systems to
reduce dependence on outstanding people
48
Topics
1.
2.
3.
ABC of CROs
Planning
RFP and Due Diligence
4. Contracting
5. Execution of Work
a.
b.
c.
d.
Communications
Monitoring performance
Red Flags
Dispute resolution
6. Evaluation
7. Takeaways
I have always wanted to be somebody.
I guess I should have been more specific.
-Lily Tomlin
49
Sometimes the obvious stuff escapes us
Farm1.static.flicker.com
50
Managing Communications
– Poor communication is often cited as a
prime reason for frosty relations
between CRO and Sponsors
– You need a comprehensive plan:
o People
 CRO contract should contain an express liaison
provision establishing a point of contact for each
party
 List of all key operations, billing and legal
personnel
o Structure
 Provide for basic joint committee structure
o Frequency
o Form (e.g. meetings, conference calls,
written and other content issues)
51
Topics
1.
2.
3.
4.
5.
ABC of CROs
Planning
RFP and Due Diligence
Contracting
Execution of Work
a.
b.
c.
d.
Communications
Monitoring performance
Red Flags
Dispute resolution
6. Evaluation
7. Takeaways
52
Monitoring Performance
– Monitoring reports should be
o Submitted by CRO timely
o Reviewed by sponsor timely
o Resolved timely, including evaluation and
escalation, if necessary, of reported protocol
violations and other non-compliance
– Contract language should address this
o Commitments
o Metrics
53
CRO Progress Reports
– In the agreement, require written progress
reports at regular intervals
– Budget for frequent status calls between
sponsor and CRO
– Topics
o
o
o
o
o
54
Contract status
IRB approval status
Site initiation/monitoring/close-out visits
Enrollment
Issues/concerns of sponsor or CRO (e.g. CFR
completion or protocol noncompliance concerns
that may require clarification to study team
and/or sites, AE trends, other problems with
sites)
Topics
1.
2.
3.
4.
5.
ABC of CROs
Planning
RFP and Due Diligence
Contracting
Execution of Work
a. Communications
b. Monitoring performance
c. Red Flags
d. Dispute resolution
6. Evaluation
7. Takeaways
55
Watch for Red Flags
–
–
–
–
–
–
–
–
–
–
Selection of inexperienced investigators by the
CRO
Questions from study site directed to sponsor
Inadequate or untimely monitoring reports
Enrollment of patients who don’t fit criteria
Higher screening-to-enrollment ratio at one site
than others
Frequent rescheduling of meetings and reports by
CRO
Delays in cleaning up CRFs
Delays in adverse event reporting
Changes in CRO personnel
Unscheduled request for payment by the CRO
Vogel
56
Compliance isn’t always easy
www.mustreadblogs.blogspot.com
57
Scope creep
– If you have fee for service or unitized
costs--
o Limit. Use limits on unitized costs (e.g. not to
exceed without approval)
o Monitor. Be proactive in monitoring “scope
creep”
 Be cognizant of
– Your instructions as the sponsor
– CRO activities; example: Repeated monitoring visits
» Site uncooperative? If so, WHY?
» CRO personnel inexperienced, untrained?
» If/when will you bear the costs?
o Communicate. If unsure, ASK – better than
being surprised later
– Ties back to change control
58
Topics
1. ABC of CROs
2. Planning
3. RFP and Due Diligence
4. Contracting
5. Execution of Work
a.
b.
c.
d.
Communications
Monitoring performance
Red Flags
Dispute resolution
6. Evaluation
7. Takeaways
59
Make sure materials are clear
– Make sure materials are
clear (contract, statement
of work, budget, payment
schedule, etc)
– Good dispute resolution
process that allows
o Discussion among joint
committees and
o Business escalation
www.robertstevensonfiles.com
60
Payment Terms
–
CRO should remain cash neutral: not a bank
–
Negotiate payment for performance, including
milestones and deliverables
o Clearly define “delivery” and “completion”
o Remember, “milestones” link back to work order –
what is the plan for arriving at the milestone?
–
Make delivery of data independent of payment
o Sponsor timeframes can move forward even with a
payment dispute
o CRO will want limitations on liability, such as limiting
damages to contract price, not actual damages due to
delay
o If CRO does withhold data to secure payment, the CRO
opens itself up to actual damages—the limitation is lost
61
Make lemonade out of lemons
Mchenrycountyblog.com
62
Topics
1. ABC of CROs
2. Planning
3. RFP and Due Diligence
4.
5.
6.
7.
Contracting
Execution of Work
Evaluation
Takeaways
Lawyer: "You don't know what it was, and
you didn't know what it looked like,
but can you describe it?"
63
Post Study Learning
– Review the performance of:
o The CRO
o Other third parties
o The sponsor
– What unexpected problems
were there, and how can they
be better managed in the
future?
– How can operations be
improved, including
communication?
– Channel that learning back to
the contracting and
operations units
64
Capitalinsight.com
Topics
1. ABC of CROs
2. Planning
3. RFP and Due Diligence
4.
5.
6.
7.
Contracting
Execution of Work
Evaluation
Takeaways
Counsel: Are you qualified to give a urine sample?
Witness: Yes, I have been since early childhood.
65
Takeaways
–
Establish realistic objectives and expectations
o Understand your internal capabilities and advise the
other party accordingly
–
–
Budget early, and well
Deliberately -o Establish clear lines of communications between
parties and
o Identify liaisons
–
Use contract templates with caution
o Do templates contain party’s wish list or
minimum requirements?
o Review/update templates periodically, particularly
MSAs
66
Different Cultures, Different Rules
www.break.com
67
Questions?
Two lawyers were out hunting when they came upon a pair of
tracks. They stopped and examined the tracks closely.
The first lawyer announced, "Those are deer tracks. It's deer
season, so we should follow the tracks and find our prey."
The second lawyer responded, "Those are clearly elk tracks, and elk
are out of season. If we follow your advice, we'll waste the day."
Each attorney believed himself to be the superior woodsman, and
they both bitterly stuck to their guns.
They were still arguing when the train hit them.