Clinical Evaluation:
what it is and
how should it be done
Dr Pete Wall
Isca Healthcare Research
CE2012 Birmingham, 20th September 2012
© Isca Healthcare Research, 2012
Context
 medical devices are distinguished from drugs by their
mechanism of action
 devices operate via physical or mechanical means

not dependent on metabolism to achieve their primary mode of
action and effect
 this fundamental difference has important implications for clinical
trials involving medical devices, their validation and subsequent
introduction into the market place
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
What is a clinical investigation ?
 any systematic investigation of the safety and/or performance of
a device in or on human beings, undertaken to generate clinical
data intended to be used for the assessment of conformity of the
device with the requirements of the directive
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Why are clinical evaluations required ?
 confirm, validate or supplement data from bench and/or animal
testing
 support novel design, new technology and/or new indications for
use
 clinical portion of any medical device development plan should
always be considered in parallel with marketing goals
 provide data



for marketing authorisation
to obtain coverage and reimbursement in the targeted markets
support competitive claims
 can be split into three categories, each fulfilling a specific functional
role:



pilot
pivotal
post-marketing
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Objectives of a clinical investigation
 verification, under normal conditions of use that device is
designed, manufactured and packaged in such a way that it
functions as described and achieve the performance intended
as specified by the manufacturer
 verification that device achieves the intended benefits to the
patient as specified by the manufacturer
 determination of any undesirable side-effects, under normal
conditions of use, and assessment of whether they constitute
acceptable risks when weighed against the intended device
benefits
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Fundamental differences - device and drug trials.…1
 proof of efficacy is not a fundamental requirement for medical
device registration - demonstration of safety and performance is


the majority of devices in clinical use have no evidence for their efficacy
those with demonstrated efficacy are usually more recent introductions to the
market or devices that have been comparatively tested in post-marketing
clinical trials
 evidence for safety and performance of medical devices is not
restricted to carefully defined and well controlled clinical studies,
but can come from a variety of sources e.g.:




partially controlled studies
studies and objective trials without matched control
well documented case histories conducted by qualified experts
reports of significant human experience with a marketed device
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Fundamental differences - device and drug trials.…2
 the standard for regulatory and marketing approval for medical
devices differs from that of pharmaceuticals


drugs frequently require multiple trials and replication of clinical findings
devices frequently require a single pivotal study
 the normal user of a medical device is a healthcare professional
and therefore clinical outcomes measured in any study of a
medical device are a function of the user as well as the
interaction between device and patient



a device always performs better in the hands of an experienced expert
it is frequently impossible to blind users to the treatment intervention
training in the use of any medical device used in a clinical study is a
fundamental component of any investigation of performance and the
eventual marketing of the device post-registration
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Fundamental differences - device and drug trials.…3
 the design of well controlled prospective trials for medical
devices presents unique challenges that differ widely from those
faced in trials involving pharmaceutical compounds



e.g. clinical outcomes in medical device trials are influenced not only by the
product under investigation and the patient, but also by the skill and discretion
of the user, who is typically a healthcare professional, but may in some
situations be the patient
the influence of the medical device user is a variable unique to medical
device trials and is frequently the cause of the greatest degree of variability in
the clinical outcome.
being aware of, and controlling for, the user influence on device performance
is a critical consideration when designing a device clinical trial
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Regulatory framework: EU MDDs
 specify the ERs which must be met prior to any device being sold
or used clinically



devices must be designed so that they do not compromise the clinical
condition or safety of patients, safety and health of users
devices must achieve the performance intended by the manufacturer
any risks which may be associated with their use constitute acceptable risks
when weighed against the benefits
 introduce controls covering safety, performance, specification,
design, manufacture, labelling and packaging
 specify requirements for pre-clinical assessment of clinical study
investigation submissions
 specify action(s) to be taken following any device related
adverse event
 specify a framework to ensure devices conform to the ERs
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Essential Requirements
 ER are mandatory and can be broken down into two groups:


General requirements for safety and performance that apply to all
devices
Specific technical requirements with regard to design and
manufacturing that may or may not apply depending on the nature
of the device; e.g. requirement for electrical safety would not apply
to a urinary catheter
 only products complying with ER may be placed on the market
and used and ER must be applied as a function of the hazards
inherent to a given product
 the need for clinical data in the CE marking process arises from
general requirements for demonstrating safety and performance
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Devices without the CE Mark
 do not conform to the relevant ER in the Directives
 have not been demonstrated to be fit for intended purpose
and therefore
 CANNOT be sold or marketed
 CANNOT be used to treat patients

unless they are part of a clinical investigation approved by the
national Competent Authority and labelled accordingly
or

have been approved by the national Competent Authority for
humanitarian use on a named patient basis
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Device trials….1
 pilot studies



a.k.a. feasibility studies; usually confined to one clinical site, i.e. single centred,
and involve a limited number of patients designed to accomplish a number of
objectives with a defined clinical testing programme
not usually designed as hypothesis testing studies, but to generate data in
support of the design of a hypothesis testing trial
provide first opportunity to evaluate the role of the user in device performance
under clinical conditions and gather information on design considerations
 pivotal studies



pivotal studies provide objective evidence of the effectiveness of a medical
device via single or multiple clinical outcomes
frequently the only clinical trial that many medical device manufacturers will
undertake
a single well designed and implemented pivotal study can provide all the
necessary data needed for device registration
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Device trials….2
 post-marketing studies (two groups)


mandated post approval studies (usually confined to Class II (US)
devices as a mandatory condition of PMA approval)
post-market surveillance studies – these are undertaken by a device
manufacturer for a number of reasons:



comparative studies with alternative or competing devices
effectiveness claims
expanded label claims
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Exceptional Use
 approval can be sought to supply non-complying devices on
humanitarian grounds
 guidance has been prepared by MHRA for clinicians and
manufacturers to help clarify how, and under what
circumstances they may make applications to the MHRA for the
use of a non CE-marked device for an individual named patient
in the interest of the protection of their health
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
UK clinical investigations....1
 to CE mark any device, compliance with ERs has to be
demonstrated
 this will usually require provision of clinical data, via
a critical evaluation of relevant scientific and clinical literature
covering safety performance, design characteristics and intended
purpose to demonstrate (i) equivalence of the new device to the
device data used and (ii) the data demonstrate ER compliance
OR
 a critical evaluation of the results of all clinical investigations made
with the new device
OR
 a critical evaluation of the combined data from both of the above
sources

© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
UK clinical investigations….2
 a clinical trial WILL be required unless safety and performance
can be adequately demonstrated by other means
 the design of the trial must be such that:


the performance of the device, as intended by the manufacturer,
can be verified
any undesirable side effects that occur under normal conditions of
use can be identified and that an assessment can be made as to
what, if any risks these side effects have when weighed against the
intended performance of the device
 user handling/preference trials should only be undertaken on CE
marked devices UNLESS they form part of a safety assessment for
CE marking
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Stages of clinical evaluation (MEDDEV.2.7.1 Rev.3)
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Practicalities: pre-study requirements
 definition of research question(s)
 development of the protocol and associated study materials





development and production of SAP
qualification / selection of Investigator(s) and study site(s)
budget; study contract; insurance
approvals for the study (as required):







e.g. IC, PIS, IVB, CRFs
Competent Authority (ies) – MHRA (UK): all documentation
REC: CIP, PIS; IC; IVB; Questionnaires; Advertising
NHS R&D: CIP; PIS; IC; contract
preparation of TMFs (Sponsor and Investigator(s))
recruitment plan
monitoring plan
data management

DMP: database design, test & validation, data entry etc.,
 Investigator Meeting (incl. device training)
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Clinical investigation plan (CIP)aka the protocol….1

contents:











introduction
identification: i.e. title; reference number; version number; date;
summary of revision history to include all amendments; page numbers
with reference and revision details on all pages, registration number(s)
name and address of study sponsor
details of Investigator(s) and Site(s)
synopsis
details of investigational device
justification for design of study
risks and benefits of investigational device and study
objectives and hypotheses of study
study design, including: type of study; endpoints; comparators;
subjects; recruitment; duration of involvement; procedures;
monitoring plan
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Clinical investigation plan (CIP) aka the protocol….2

continued:












statistical considerations
data management
amendments to protocol
deviations from protocol
device accountability
statement(s) of compliance (i.e. Declaration of Helsinki, ISO, GCP and
regional/national requirements, insurance)
consent process
adverse events and safety reporting
reporting
early termination
publication policy
bibliography
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Model Clinical Investigation Agreement
 a model Clinical Investigation Agreement (mCIA) which is
designed to be used without modification for companysponsored commercial research involving medical devices in
patients in hospitals throughout the NHS was launched in
November 2008
 versions of the agreement for use throughout the UK and
guidance notes are all available on the UK Clinical Research
Collaboration (UKCRC) website http://www.ukcrc.org/
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
The application process
 made via the Integrated Research Application System (IRAS)
www.myresearchproject.org.uk
 IRAS is a UK-wide system that streamlines the process for applying
for permissions and approvals to conduct health and social care
research, including clinical investigations of medical devices

CA, REC and NHS R&D
 PCA1 and PCA2 forms
 Sterilisation Pro-Forma
 print and sign before making submission to MHRA
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Fees
REC
no fee for review (NRES), but some hospitals may charge
NHS R&D
fees vary
MHRA
Class I, IIa, or IIb other than implantable or long-term invasive
 £3,020 (£2,120)
Class IIb implantable or long-term invasive, Class III, and active
implantable
 £4,240 (£2,770)
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
MHRA submission documentation….1
 all documentation must be in English – if any part of supporting
information is in another language a translation must be
provided

plus an original copy in original language
 signed statement confirming compliance with ERs other than




those being evaluated in trial
signed statement indicating whether or not device contains as
an integral part a substance or human blood derivative
signed statement indicating whether or not device utilises tissues
of animal origin
sterilisation validation (as appropriate)
signed copies of PCA1 and PCA2 forms
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
MHRA submission documentation….2
 General Information

date; contact details; submission status; other Member States
involvement; NB approvals and certifications
 Device details

device name; model - name and number; manufacturer; design
drawings; circuit diagrams; test certificates; biocompatibility; materials
data sheets; sterilisation methodology and validation; pre-clinical
data; risk analysis; ER checklist; Standards listing; IFUs; photographs;
software (plus separate risk analysis); classification; labelling; User
Manual; packaging
 Other


CIP (signed); PIS; IC; CRFs; IVB; adverse event form; insurance
certificate; clinical literature review ; Investigator CVs
REC and local NHS R&D approvals
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
MHRA submission documentation….3
 1x hardcopy of the full submission and 8x rewritable CD-ROM’s
are required



printed and collated with all pages in their correct numbered
sequence, including reprints, diagrams, tables and other data
the method of reproduction used must allow for legible presentation
of the text and any relevant drawings with their captions
CD-ROMS - documents must be identical to those in printed copy and
arranged on the CD in such a way that they can be easily identified
by title alone (it is recommended that each CD includes a document
index and all documents are appropriately named)
 the documentation should be clearly labelled ‘Documentation
Only' and sent by recorded delivery
 trial notifications will only be accepted by the MHRA once the
signed forms, necessary supporting documentation and the
appropriate fee have been received
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Ethics Committees
 the UK Competent Authority does not accept approvals from
independent ethics committees
 manufacturers should seek the opinion of a National Research
Ethics Service (NRES) (or equivalent services in Scotland, Wales
and Northern Ireland) appointed ethics committee in all cases
unless they can demonstrate a reason why an NRES (or
equivalent) appointed committees would not assess their clinical
investigation
 in such cases the manufacturer will need to demonstrate that
any independent ethics committee appointed was constituted
in line with NRES guidelines
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
REC submission documentation
 REC submission letter
 REC checklist
 IRAS submitted REC form
 IRAS exported REC form
 CIP (signed)
 PIS
 IC
 IVB
 IFU
 GP letter
 insurance certificate
 Chief Investigator CV and GCP certification
 ER compliance letter
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
NHS R&D submission documentation














NHS R&D submission letter & NHS data protection number
copies of MHRA submission letter and PCA1, PCA2 forms
mCIA – applicable national version incl. budget
NHS SSI form generated via IRAS
NHS R&D form generated via IRAS
CIP (signed)
PIS
IC
IVB
IFU
GP letter
insurance certificate
Chief Investigator CV and GCP certification
ER compliance letter
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Review process
 notice of receipt of application
 MHRA reference Number
 starting date for 60 day clock
 at least 2 reviewers (14d)
 further information may be requested
 notification via email and letter
 clock will not stop
 decision
 “Objection“ - study cannot proceed; MHRA notifies other EU CAs and
Commission (re-submission may be made once reason for objection has
been addressed)
 “No Objection” – study may start when ALL approvals (MHRA, REC, local
NHS R&D) in place
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Managing the trial
 CRO or in-house
 in-use training is essential for device trials
 GCP and ICH requirements
 Helsinki Declaration
 regular site monitoring

protocol compliance; accuracy and integrity of trial data; source document
verification; rights and wellbeing of participants; study supplies reconciliation;
AE and SAE reporting and follow-up
 data management
 data query resolution
 Investigator and site responsibilities post-trial
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Practicalities: study management
 study initiation
 first patient in
 monitoring visits and reports, including safety reporting
 last patient out (completion of last patient, incl. all follow up)
 data management



data entry from CRFs and data query resolution
database cleaning→database lock→analysis (according to SAP)
statistical analysis report, review and sign-off
 study report preparation, review and sign-off
 site(s) close out
 notification to REC and MHRA

safety report (12m); study completion; copy of final study report
 archiving of all study documentation(15y minimum)
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Amendments to trial
 any and all proposed changes to an approved trial, e.g. device,
Investigator, site, safety information, MUST be notified in writing to
MHRA (non-CE) and REC and not implemented until written
approval is received from ALL authorities
 requests for review and approval of amendments must include:


MHRA trial reference number; proposed change(s) and their rationale; signed
statement from Manufacturer confirming that the proposed changes do not
predictable increase risk to patient, user or third parties
confirmation that amendment has been submitted to REC and NHS R&D
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Adverse Events
Manufacturers are required to report fully to the MHRA all adverse
events occurring in the UK:
a serious adverse event shall mean any adverse event, related to the device
intended for clinical investigation, to a comparator or the investigation
procedure, consisting in an untoward medical occurrence, unintended
disease or injury or any untoward clinical signs, including abnormal
laboratory findings, in subjects to a clinical investigation, users or other
persons, that
•
•
led to death
or led to a serious deterioration in health that either
• resulted in a life-threatening illness or injury
• resulted in a permanent impairment of a body structure or a body function, or
• required in-patient hospitalisation or prolongation of existing hospitalisation, or
• resulted in medical or surgical intervention to prevent life-threatening illness or
injury or permanent impairment to a body structure or function,
• or led to foetal distress, foetal death or a congenital abnormality or birth defect
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Post-marketing clinical follow up (PMCF)….1
 PCMF


continuous process to collect and evaluate clinical data, gathered
systematically in the post-market phase, concerning the assessment
of long term safety and performance of a device
undertaken in line with a PCMF plan


if PCMF not deemed necessary or appropriate, documented justification is
required
PCMF plan should contain documented methods and procedures to
proactively collect clinical data from actual use in human of a CE
marked device within the intended purpose




confirm safety and performance throughout expected lifetime
monitor identified side effects and contra-indications
identify and analyse emergent risks using factual evidence
assure continued acceptability of benefit/risk ratio
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Post-marketing clinical follow up (PMCF)….2
 PCMF plan



detail methods and procedures e.g. clinical data, complaints, user
feedback, literature screening
rationale for appropriateness of methods and procedures used
reference relevant parts of previously undertaken clinical
investigations
 Manufacturer




analyse findings, document results in evaluation reports to be
incorporated into Technical File
update clinical literature review
update risk management
implement corrective actions as appropriate
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
MHRA Guidance Documents
Further information may be found in the following:
 Guidance Note 1 - Guidance Notes for manufacturers on clinical
investigations to be carried out in the UK
 Guidance Note 3 - Information for clinical investigators
 Guidance Note 5 - Guidance on biocompatibility assessment
 Guidance Note 17 - Guidance Notes for manufacturers on
statistical considerations for clinical investigations of medical
devices
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Tips and Advice: 1
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Tips and Advice: 2
 ~25% of submissions to MHRA are rejected because of
inadequate documentation and/or poorly designed trials
 common reasons for rejection are:








poor or absent study end-points
no/inadequate risk analysis
flawed study design
inadequate pre-clinical testing or assessment
inadequate toxicological data
no sterilisation validation
inadequate electrical safety testing
risks outweigh benefits
 read the guidance documents that are available
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Tips and Advice: 3
 just reading the regulations (or having them on your bookshelf)
does not make you an expert
 for anything other than the most simple of devices, you will
almost certainly need help to produce the required
documentation to the necessary standard and ensure you have
all the information needed to demonstrate compliance with all
the ERs except those that are the subject of the clinical trial
 unless you have in-house expertise (very unlikely) it is strongly
recommended that you talk to, and engage, an expert with
proven experience and a track record in the conduct of
medical device clinical trials and MHRA submissions
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Tips and Advice: 4
 engage with the MHRA prior to making your submission
 respond promptly to MHRA requests for further information
 cite the unique MHRA study reference number on ALL
communications with MHRA
 notify the MHRA of any problems
 ensure that Adverse Events are reported within the appropriate
timeframes and to the appropriate authorities
 seek help and advice (from appropriately qualified and
experienced sources) as and when necessary
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
In vitro Diagnostics
 there is no clinical investigation system for in vitro diagnostic
medical devices
 performance evaluations of in vitro diagnostic devices that are
performed outside of the manufacturer’s premises should be
notified to the MHRA in accordance with the Medical Devices
Regulations 2002: Section 44
 see MHRA Guidance Document 18
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012
Contact details
Dr Pete Wall
Isca Healthcare Research
Hill House
Belmont Hill
Caerleon
NP18 1JX
Tel: 01633 423641 / 07813 799193
Email: pw@iscahcr.co.uk
© Isca Healthcare Research, 2012
CE2012 Birmingham, 20th September 2012