How to Analyze and Implement
a Research Protocol
Phyllis Carelllo, CCRC
Bridget Adams, MSHS, CCRA
Susan Peterson, RN
Oregon Health & Sciences University
NIH CA P30 069533, NCRR UL1 024120
What is a protocol?
 A detailed plan that sets forth the study:
– Objectives
– Design
– Methodology
Why do we need a Protocol?
 Why do we need one?
– Scientific validity
– Subject safety
– Replicate the science if necessary
– Regulatory requirements
Federal Regulations
 45 CFR 46
– Applies to research involving human subjects
conducted, supported or regulated by any federal
department or agency
– IRB’s are required to review and approve research
– IRBs have the authority to suspend or terminate
approval of research that puts subjects at risk or is not
conducted in accordance with IRB requirements
 21 CFR 50, and 312 or 812
– Applies to FDA regulated studies
Who Writes the Protocol?
 Sponsor
– Commercial manufacturer
● Pharmaceutical or Biotech company
● Medical Device company
 Investigators and study team
– Investigator initiated
– OHSU Investigator or investigator at another
institution
Industry Sponsored Protocols
 Protocol generally part of larger investigational plan to get
the drug/device approved by FDA
– May be large multi-site trials
– Changing protocol is difficult
– Investigator usually has no input in to study design
 Centralized research activities
– Data Analysis
– FDA reporting
– May use a Central Lab
 Funding
– Sponsor usually pays for all research activities
– Contract between sponsor and OHSU
Investigator-Initiated
 Independently conceived and developed by scientists - the
primary means by which biomedical research is conducted
 Range from a small pilot study to large multi-site clinical trials
 May study a disease process or an intervention
 Investigator responsible for all research activities
– Reporting to funding and oversight entities
– Data Analysis
– Publications
 Funding
–
–
–
–
–
Company (pharmaceutical, biotech, medical device)
DHHS (NIH, DOD, DOE, VA, etc.)
Foundations
Departmental
OCTRI Awards
Who Evaluates the Protocol?
 IRB : Subject Protection
 Funding agency (NIH, Foundation): $
 Federal Oversight (FDA) if applicable:
Protect Public Health
 Investigator: Feasibility, Scientific Interest
 Study Coordinator: Operational
Implementation
IRB Reviewers





Is this scientifically valid?
Is it safe?
Is the risk/benefit ratio appropriate?
Is this ethical?
Is it compliant with the regulations?
Funding/Oversight Agencies




Is it scientific valid?
Is it feasible?
What it the experience of the Investigator(s)?
What are the facilities and resources
available to conduct the study?
Investigator
 Industry/Multicenter Studies: PI reads
protocol to find out if they are interested in
participating
– Academic Interest?
– Is it scientifically valid?
– Does the PI/prospective Co-investigators have
the patients in their clinic?
– Does the PI have time to conduct the study?
Coordinator
 Learn the protocol requirements
– What do I need to do?
 May use protocol to develop checklists and data
collection forms
 Will the proposed budget cover the costs?
 Are there unfamiliar procedures?
– Specialized expertise
– Training
 Time
– How long is study enrollment open?
– Does the PI have time?
– Are study personnel over-committed?
Coordinator, cont.
 Provide feedback on the feasibility of the study (feasibility
checklist
http://www.octri.org/octri/public/index.aspx?pageid=126&sit
eid=1&MenuSelectedIndex=7)
– Is there adequate space?
● Clinic space
● Storage space
● Coordinator space
– Doe we have the subject population?
● In clinic
● Competing studies
● Advertising
– Equipment
● Is equipment available for use?
● Do we have to buy equipment?
● Will equipment be provided by the sponsor?
Protocol Review
High
Low
Scientific Reviewers:
NIH/ Foundation
IRB Members
Co-Investigators
Statistician
Auditors
Study Coordinators
Laboratory Staff
“Protocol” Documents
High
Grants Contain:
Protocols Contain:
Operations Manual:
High Level Scientific
Concepts
May describe several
studies
Generally have little detail
Generally not revised
Revisions require funding
source approval
High Level Scientific
Concepts
Generally describe one
study
Moderate Level of detail
May be revised with IRB
approval
Little or no scientific
information
High level of detail for
completing tasks
May be revised frequently
without IRB approval (must
be consistent with the IRB
approved protocol)
Low
Scientific Reviewers:
NIH/ Foundation
IRB Members
Co-Investigators
Statistician
Auditors
Study Coordinators
Laboratory Staff
Study Phase
Considerations
Clinical Trials
 Any research project that prospectively assigns human
subjects to intervention and comparison groups to study the
cause-and-effect relationship between a medical
intervention and a health outcome.
 Intervention can be:
–
–
–
–
–
Drug
Device
Behavioral
Educational
Other
 Control can be:
– Placebo
– Standard of Care
– Nothing
Study Trial Phases
Phase I
 Population:
– If healthy volunteers – where will you recruit?
– If ill population – what is standard of care?
 Visit Duration:
– What study personnel need to be available and for how long
at each study visit?
– Where do you have the space to conduct a long visit?
 Design
– Open label
 Aim
– Safety
Study Phases cont’d
Phase II
 Population
– Criteria is generally strict – do we have the population?
– Do we have conflicting studies?
 Duration
– Studies generally aren’t long, so when does enrollment end?
– If subjects have to go off their meds for a period of time, will you
expect a high drop out rate?
 Design
– Randomized - Who will randomize subjects?
– Double blind studies - Who is going to be available to unblind if
necessary?
 Aim
– Continued Safety Evaluations
– Initial Efficacy Data
Study Phases Cont.
Phase III
 Population
– Generally large # of subjects, do we have the capacity to see
them?
– Representative Population (broader inclusion/exclusion criteria), do
we have these subjects in our clinic?
 Duration
– Parallels expected treatment use (several days to years) – do we
have the capacity to follow the subjects for a long time?
 Design
– Generally double bind
 Aims
– Establish safety and efficacy in the target population
Phase IV
 Population
– Population prescribed the drug – who will pay for the drug?
 Duration
– Long Term for chronic diseases – do we have the capacity to see the
subjects for years?
 Design
– Generally open label
– Procedures generally same as clinical care – what is research and what is
Standard of care? Who pays?
 Aim
– Economic comparisons
– Quality of life
– Adherence and compliance
Study Design
Blinding
 Single Blind – subject doesn’t know what
treatment group they have been assigned
 Double Blind – subject and investigator don’t
know treatment assignments
Randomization
 Selection of treatment/control group by chance
 Purpose – Minimize Bias
 Several methods
– Fixed Allocation Randomization
● Simple (flipping a coin)
● Blocked
● Stratified
– Adaptive randomization
● Baseline
● Response randomization
Parallel Design
 Subjects are assigned to study arms/groups
and stay in that group for the duration of
their study participation
Randomization
Group 1
Group 2
Group 3
Group 4
Cross-over Design
 One group is assigned to treatment and one
to a control group, at some point in the study
they switch assignments
Randomization
Intervention
Placebo
Group 1
Group 2
Open Label Extension Study
 Investigators and subjects know what
intervention they receive
Group 1
Observational Study
 No Intervention
 Draw inferences about the effect of a treatment on
subjects, where the assignment of subjects into a
treated group versus a control group is outside the
control of the investigator
– Used when it may be unethical or impractical to conduct
a randomized trial
– Used for studying public health effects
 May involve clinical procedures
 May be long term
Break!
 Back in 10
Analyze, Analyze, Analyze
If we knew what it was we
were doing, it would not be
called research, would it?
Albert Einstein
Analyzing a Protocol






Who?
What?
Where?
When?
How?
Why?
Protocol Elements




Title
Introduction, background, rational, literature review
Purpose(s), Objective(s)
Research Methods/ Study design
●
●
●
●
●
●
●
Endpoints
Stopping Rules
Duration
Drug/ Device/ Interventions schedule, modifications, precautions
Study procedures
Subjects (#, inclusion/exclusion, withdrawal criteria)
Safety Assessments
 Risks
 Statistical Analysis
Protocol Elements, cont.
Title page
● Protocol title
● Sponsor
● Spaces for sign-off by investigator (industry
sponsor)
● Date and version
● Study Identifier (e-IRB#, OCTRI#, IND#, Sponsor
protocol #), as applicable
Protocol Elements, cont.
 Introduction with background, rationale, and
Literature Review
– Provide basis for the research
– Summary/Introduction
● Brief description of what is currently known about
the drug and/or disease condition to be studied
● Background information (journal articles)
● Justification for dose
● Route of administration
Protocol Elements, cont.
 Objectives/Specific Aims
– Characterize population
– Scope
– Duration
– Outcomes
– Experimental measures
Protocol Elements cont.
 Study Population
– Total number of subjects (including screen failures/subject
withdrawals)
– Inclusion/Exclusion criteria
● Age restrictions
● Diagnostic methods (HGB > 12) to determine eligibility
● Pre-existing conditions
– Disallowed concomitant medications and/or treatment
– Inclusion of vulnerable populations (if applicable)
– Individual subject withdrawal criteria
– Allowances for temporarily stopping drug/intervention
– Dose Modifications
– Non-compliance
Protocol Elements cont.
 Research Design and Methods
– Endpoints
● Identifiable change that shows the intervention did
what it was supposed to do
– Primary Endpoint: measures the specific clinical
effect the intervention is preventing/treating (i.e.
survival, resolution of desease)
– Surrogate endpoints: measures changes in
symptom/biological indicator for the success of
the intervention (i.e. T-cell count, radiographic
imaging, etc.)
Protocol Elements cont.
 Research Design and Methods, cont
– Endpoints, cont
● Should be Measurable
● Measures should be as objective as possible to
avoid bias
● Should be feasible
Example: Endpoints
 Study to determine efficacy of new antibiotic
for pneumonia (n=300)
– Not objective: patient report of “cure”
– More objective: clinical cure evaluation by
physician
– Most Objective: chest x-ray
Protocol Elements cont.
 Methodology and study schedule
–
–
–
–
–
–
–
–
–
Description of recruitment and enrollment procedures
Screening and allocation of subjects to study groups
Subject numbering system to be employed
Preparation of study drug for use
Schedule for administration of test article to subject
Contraindications
Schedule of visits and assessments
Measures to maintain blinding/ unblinding procedures
Required documentation
Protocol Elements cont.
 Study measures and assessments
– Physical examinations
– Measurements of vital signs
– Assessment procedures
– Clinical Laboratory
– Handling of samples
– Special safety requirements
● Detailed
Protocol Elements cont.
 Research Design and Methods cont.
– Description of study procedures/interventions
● Research procedures described in detail so they
can be performed consistently
● Common medical procedures may have no detail to
allow for appropriate variation among clinicians (if
scientifically acceptable)
● Procedures for carrying out physical exams, vital
signs, handling samples may be very detailed if it is
essential to the data collection
Identification of Unspecified Procedures
 Procedures required for safe conduct of a
study aren’t always listed in the protocol
 Example: the protocol states a Dexa Scan
will be completed at Visit 3
 What is a potential unspecified procedure for
a Dexa Scan?
Identification of Labs
 Example: Protocol states 20 mls of blood
– Why?
– What tubes to you need to draw?
– What is the minimum amount of blood needed?
– Does the sample need to be on ice?
– Will the sample be processed at OHSU or at a
central lab?
– Does it need to be shipped that day?
Identification of Standard of Care
PI or MD investigator makes determination
Not always obvious in the protocol
Not always obvious to the MD
Standard of care varies from institution to
institution
 Be consistent –




– don’t bill insurance for one subject and not
another without a documented reason.
Protocol Elements cont.
 Drug/Device
– Drug preparation, administration, storage
– Dose modifications
– Device Implantation Procedures
– Required concomitant medications, if applicable
– Drug/Device accountability, retention, final
disposition
Protocol Elements cont.
– Risks
● Known Risks
● Protection from risks
● Risk/Benefit Analysis
– Safety Assessments
● Identify procedures for safety monitoring
● Adverse event recognition, documentation, and
reporting requirements
– May not be specifically stated in the protocol
● Special precautions for specific side effects, if required
Protocol Elements Cont
 Adverse Experiences
–
–
–
–
–
Recognition and management
Documentation
Criteria for grading severity
Criteria for grading relatedness
Follow-up for subjects
● After an adverse event - usually follow until event is resolved
● UPs that occur within 30 days of last study intervention/subject
discontinuation must be reported to the OHSU IRB
Protocol Elements Cont
 Research Design and Methods (cont.)
– Stopping rules/criteria
● Based on results of interim analysis
– Study will be stopped if a certain pre-determined
proportion of adverse events occur
– Study will be stopped if endpoints aren’t being
met
– Study may be stopped if a clear benefit is
shown
– May be determined by sponsor, DSMB, or PI
depending on the study
Protocol Elements Cont’d
 Research Design and Methods (cont.)
– Duration (individual subjects and study)
● Make sure it is long enough to collect data and
accommodate enrollment
● Should be no longer than required to collect enough
data to evaluate study endpoints
● Balance Scientific Need with Subject Burden
Protocol Elements Cont’d
 Data analyses
– Variables to be analyzed
– Statistical approaches to employed
– Geared toward statisticians
– Selection of subjects to be analyzed
● all enrolled subjects (“intent to treat”)
● all eligible subjects (e.g., 80% compliant)
Other Important Study Documents
 OHSU IRB Initial Review Questionnaire
(IRQ)
 Investigator Drug/Device Brochure
 Laboratory/Operations Manual
 Standard Operating Procedures (SOPs)
 Grant/Contract
Initial Review Questionnaire (IRQ)
 Approved by the IRB
 Some information that may not be in the
protocol
– Approved research staff
– Funding information
– # of subjects at OHSU
– Recruiting and consenting procedures
Investigator Drug Brochure/ Device
Manual






Reviewed by the IRB
The IB contains information on the drug
Study results to date
All known adverse events and risks
Information for the investigator on dosing
Device Manual will include
implant/compatibility/programming
requirements
Lab/Operations Manual
 Do not require IRB approval
 Maybe known by other names
 Include minute detail on:
– Lab processing
– Case Report Form Completion
– Patient flow during a visit (e.g. call ahead to see if X-ray
is backed up before you head down there with a
patient)
– Calling and paying for a taxi
– Many, many other details
SOPs
 Do not require IRB approval
 Unit, not protocol, specific
 Define the who, what, where, when and how of research
activities not outlined in your protocol
– Obtaining/verifying informed consent
– Drug accountability
 Not required of study sites per regulations (required for
sponsors) but good to have:
– Assure study conduct/data quality
– Looked upon favorably by sponsors and regulators
 If you have SOPs you will be held to them in an audit
Grant/Contract
 Grants are reviewed by RGC and the IRB
 Contracts are reviewed and negotiated by the
Clinical Trials Office, not reviewed by the IRB
 Contracts must be signed by the appropriate
OHSU institutional signatory (not the PI or
Department) before the study can begin
Contract
 Industry sponsored trials
–
–
–
–
–
–
# of subjects you can enroll
# of screen failures
Additional procedures
Payment terms
Invoicing (or not)
Term of study
 Contracts must be amended if:
–
–
–
–
PI changes
Budget Changes
Want to enroll more subjects
The term of the contract is exceeded
Break!
 Back in 10
Protocol Implementation
 Ready, Set, Go!
Protocol Implementation
 Not quite yet …
 There are a few more
hoops…
Training
 Research Education Requirements
– Responsible Conduct of Research for all
– Responsible Conduct of Research Involving
Human Subjects
– Other RCR as applicable
Training, cont.
 Verify study staff have training on Core
Competencies
– (see OHSU Clinical Research Coordinator Required
training Checklist)
– Basic Life Support (BLS – initial and every two years)
– Phlebotomy (contact Steve Osgood for training)
– Point of care testing (contact Jane Culbertson)
● ECG, BP, Vital signs, HCG urine, Urine dipstick
 EPIC access/training as required
Training
 Staff protocol training
– Make sure study staff read and understand the
protocol
– Have study staff acknowledge their roles in the
study
– Make sure study staff know how to identify and
report deviations
– Document that training was completed (sign-in
sheet template - see Inservice Documentation Log )
Training, cont.
 Research Pharmacy Services (RPS) study
initiation visit
– PI/ study contact to set up appoint with
pharmacy staff per pharmacy policy (see
http://www.ohsu.edu/pharmacy/rps/policies.htm)
Training, cont.
 OHSU Inservice(s)
 If beside nurses will be involved in the study
– Complete and submit Clinical Research Study Nursing
Summary (link available in the electronic IRQ)
– The PI/designee is responsible to provide special training to
some or all of bedside nursing staff. Training fees may apply.
 Clinical and Translational Research Institute
– Inservices required for all studies requesting CTRC nursing
services
– Complete CTRC nursing services request form
– PI/study staff will present protocol and clarify any study
specific procedures to be conducted at the CTRC
Other considerations
 Required Health screening/immunizations
– TB test
– Hepatitis B immunizations
– Ask Employee Health if you have questions
503.494.5271
Work-flow planning
 Where are prospective subjects to report?
 How are subjects routed during the study visit?
 Where are study working materials stored?
– Template order forms (lab, radiology, RPS)
– MD orders
– Blank consent documents
 How will you communicate changes in
procedures/protocols etc to study staff?
Scheduling considerations
 Reconcile protocol requirements with practical constraints
– Fasting?
– Duration of study visit?
– Study Personnel availability?
– Are you dispensing research medication(s)?
– Inpatient studies (who is available to discharge, who is
going to see the subject each day of admission)
– Lab considerations?
– MD orders to
● CTRC -48 hours in advance
● Pharmacy
Scheduling Considerations cont
– Does the subject have/need a medical record?
– You need a medical number if the subjects are going to
have any clinical services at OHSU (e.g. labs, x-rays,
MRI)
– Medical Record Number Request form at
http://ozone.ohsu.edu/healthsystem/HIS/medrectrx/req
mrnumhim.pdf
Scheduling Example
 Lumbar puncture
– Who: MD available
– What: LP Tray – are separate needles required?
Does the MD have a preference?
– When: Time of day? Are subjects NPO? How
long is the observation period?
– Where: Private room, Bed or gurney?
– Why: Endpoint data?
Common Scheduling Problems
 Not scheduling enough time
 Not scheduling in the appropriate clinical
area
 Not scheduling with the appropriate
personnel (MD available)
 Not informing everyone that needs to know
 Not formally getting subjects on the
clinic/CTRC/or other schedule
EPIC and Research
 Consent forms and research documentation must be in
Epic if subjects are having clinical services at OHSU per
the OHSU Research Documentation in Medical Records
Policy (coming soon) and the Informed Consent Forms for
Research in Medical Records, Clin 02.15
http://ozone.ohsu.edu/healthsystem/POLICYMANUALS/Cli
n02MedRec/Clin02-15.pdf
 Research Flags – identify a patient as a research subject
and provide study- specific information to all personnel
involved in their care
 Setting research flags in the Integrated Health Record
http://www.ohsu.edu/research/crp/docs/researchFYI.doc
Epic and Research Records
 To request the ability to set research flags complete the
Research Flag Access Request form at
http://www.ohsu.edu/research/crp/docs/research
coordinator.xls
Scheduling Cont.
 Research Admission - request form available at
http://ozone.ohsu.edu/research/rates/researchadmi
ssion.pdf)
 Scheduling at the CTRC instructions available at
http://www.octri.org/octri/public/index.aspx?pageid
=183&menuid=51&siteid=1&MenuSelectedIndex=1
Equipment and Supplies
 Do you have everything you need?
 Who is responsible for providing supplies
 Industry sponsored trials usually provide lab kits(but not
always)
 Who is responsible for ordering?
 Who is responsible for tracking?
Equipment and Supplies Cont.
 Use of Research Devices and Equipment
– http://ozone.ohsu.edu/HealthSystems/Adm10En
viron/Adm10-04-10.html
– Must be inspected by Clinical Technology
Services if the device/equipment was not
purchased by OHSU
 Computers
– If the sponsor provides you with a computer for
the study (not purchased through OHSU) it
must be inspected by ITG
Administrative Set-up
 Prior to enrollment
– OGA #/ FAID
– Industrial Accounts/ Research Rates (see Research
Study Rate Request, Account(s) Setup, and Billing of
Clinical Services, Adm 05.17 )
– Send account #s to Research pharmacy
Administrative Set-up
 Clinical Trials Registration
– Guidance Regarding the International Committee of Medical
Journal Editors (ICJME) Requirement for Clinical Trial Registration
– OHSU does not have an institutional account # , so make sure to
register for an individual account #
– Takes up to two weeks
– Need to submit IRB approval memo and keep study status up to
date (minimum updates every 6 months)
– Not Required if:
● Study doesn’t meet definition of clinical trial
● Industry-sponsored multicenter (Sponsor registers)
● Multicenter trials where OHSU is not the coordinating center
(Coordinating center registers)
Administrative Set-up
 Investigational Devices
– Prior to enrolling Medicare beneficiaries you
Must submit to OHSU’s Medicare Fiscal
Intermediary in advance for review and approval
– http://ozone.ohsu.edu/HealthSystems/Adm05Bil
ling/Adm05-18.html
Administrative Set -up
 Off campus activities
– If OHSU employees will be working where they
are directly contacting human subjects in
facilities that are not owned or leased by OHSU
you need to complete the Off Campus Clinical
Activities Authorization Request
Incomplete Protocol Analysis and
Implementation
 Leads to protocol non-compliance
 Leads to poor decision making
– Subject Safety
– Study Conclusions
 Leads to protocols that go nowhere
 Leads to budget deficits
– More involved in the study than you thought
 Regulatory Headaches
Common Protocol Deviations
 Non-adherence to inclusion/exclusion criteria
 Failure to comply with dispensing and dosing
requirements
 Incorrect storage of study medications
 Use of prohibited concomitant medications
 Visits outside of study windows
 Protocol requirements not followed
– tests not done
– incorrect tests
– safety labs not done
Common Protocol Deviations cont.
Failure to follow subject termination plan
Failure to report adverse events
Over-enrolling study
Failure to follow statistical plan
Failure to amend protocol and obtain IRB
approval prior to implementing changes
 Failure to document and report deviations





How to Avoid Deviations cont.
 Follow OHSU Research and Hospital
Policies:
– IRB Policies:
http://www.ohsu.edu/xd/about/services/integrity/
policies/
– OHSU Research Policies:
http://www.octri.org/octri/public/index.aspx?pag
eid=123&siteid=1&menuid=1&siteid=1&MenuSe
lectedIndex=6
How to Avoid Deviations cont.
 PI needs to supervise the conduct of the
study
 Delegate study tasks to qualified individuals
(e.g. physical exams delegated to MD, FNP,
PA)
Conducting the Protocol for Compliance
 Once the study has IRB approval the study
team is expected to adhere to the protocol
without deviations – unless it is for the safety
of the subject
Research Protocol
Conducting the Protocol cont.
 Protocol Modifications
– An amendment/modification can be submitted for IRB
approval to address new information/difficulties
– Should analyze new procedures prior to implementation
– Strive to minimize modifications so that data remains
“poolable”
● Don’t want to compare apples and oranges
– Maintain consistency within and between the protocol,
consent, and procedure manuals
Conducting Protocol cont.
 Modifications
– Re-evaluate before submitting modification to
the IRB, when appropriate, but always before
implementation
● Budget
● Location/Scheduling
● Training
● Feasibility
Conducting the Protocol cont.
 Document, Document, Document
– If it wasn’t documented it wasn’t done
– If the procedure documentation is
questionable so is the data
– Document all special circumstances that
might affect the interpretation of the
results
Research
Records
References
 Drugs Dispensed to Research Subjects,
Clin 05.11
http://ozone.ohsu.edu/healthsystem/POLI
CYMANUALS/Clin05Meds/Clin05-11.html