Readability Testing of Package
Information Leaflets and Bridging
for the Generic Industry
By: Kurt Kinsey
Pharma-EU s.r.o.
Malostranske Namesti 23
Prague 1
118 00
Czech Republic
Telephone: 00 420 739 355 429
kurt@pharma-eu.com
Overview:
• Introduction to readability testing
• What is readability testing?
• Readability Testing and Timing for MRP, DCP, CP and
National Procedures
– MRPs
– DCPs
– Central Procedures
– National Procedures
– Renewals and variations
• Experience and Feedback with Readability Testing in the
Generic Industry
• Ensuring success with Bridging
Introduction to Readability Testing
–
–
–
–
–
–
All Health Agencies across Europe required User Testing
since October, 2005 for, CP, MRP and DCP.
Implementation was staggered across Europe, but all 27
members now require it for MRP and DCP applications.
Completion of the testing has not been required for most
national procedures to date (UK is the exception).
The UK required all PILs for nationally approved
products to be tested or bridged by July 2008.
A few member states are now requiring all PILs to be
tested as well for their existing nationally authorised
products. FR and DE in particular.
In summary, readability testing has not been required for
most national procedures and variations (exceptions
being the UK).
Why User Test ?
• The leaflet must reflect the product licence (the Summary of
Product Characteristics (SPC) of the product to which it refers.
Differences between the SPCs for the same medicine available
from different MA holders led to inconsistent information in
the PIL, resulting in frequent complaints from patients.
• Many leaflets were lengthy due to the complexity of the SPC,
and were poorly laid out. Patients quickly lost interest in the
document, failing to read or understand information crucial to
the safe use of the medicine.
Why User Test ?
• Readability testing qualifies that the medical information
contained in the leaflet is usable i.e. understood by potential
users of the medication.
Why Bridge?
• To show these requirements are met while not being forced to
perform repeated readability testing of PILs with similar text
and key safety messages, for closely similar or related
products with standardized styles and formats.
What is User Testing ?
How many people here have experience with user testing? Do I
need to refer to the methodology?
Readability Testing Methodology
“Diagnostic user testing was pioneered in Australia in the early
1990s and was recommended in guidelines on Readability in
Europe by the European Commission in 1999. It is a
performance based, flexible development tool which identifies
barriers to people’s ability to understand and use the
information presented, and indicates problem areas which
should be rectified”
“It is particularly useful as part of a leaflet development
process”
Medicinal products have benefited by user testing in the result of
high quality, user friendly and uniformly formatted PILs.
Before Testing
Before Testing:
• PILs should first be ordered and prepared in the QRD format.
• PILs should be written and reviewed to ensure they are
prepared for readability testing.
• Using nationally mandated texts can be problematic.
• Mock Ups should be created to EMEA readability guidelines.
Questionnaire Development
• Key issues for safe use of the medication should be identified.
• A systematic approach and development process should be
used and reported to explain how key issues for safe use were
addressed and how the questions are proportionate to the PIL
in question.
Questionnaire
• A questionnaire is developed that contains 12-20 questions
specific to safety issues of the medicine in question.
– EMEA guidance on assessing readability testing states that
12-15 questions should be used. This is not always
practical, as more complex medicines may require further
testing to adequately test the key issues for safe use.
– i.e. linezolid, nevirapine
How will the testing be performed?
• The individual tests are conducted with one to one, face to face
structured interviews.
• The interviewer should be experienced and have a medical or
pharmaceutical background.
– This is important to satisfy any enquiries or questioning of how correct
answers are assessed.
• PIL writers should be involved in the User Testing as they will
gain valuable experience.
How many participants need to be
tested?
Minimal of 3 rounds of testing on 23-25 participants
• Preliminary Testing
– Minimally 3-5 participants are tested to determine if there are
any major revisions needed to the PIL.
• 1st Round of Testing
– Minimally 10 participants are tested to achieve a passing result
– Minor revisions may be performed after this as needed.
• 2nd Round of Testing
– Minimally an additional 10 participants are tested.
• Additional rounds of testing may be required if:
– A passing result is not achieved
– The PIL is revised during the registration procedure.
» The Health Authorities will advise as how to proceed.
» A bridging report may suffice and save time and
€€€€€€€€.
What Aspects Need to be Assessed?
The 3 main criteria that should be assessed:
– Traceability of the information.
– Comprehensibility of the information.
– Applicability of the information.
Who should participate?
Care should be used to when recruiting participants to ensure
inclusion and exclusion criteria are met.
• Certain member states have differing requirements.
• We have had success recruiting participants who represent the
most likely demographic samples to experience difficulty with
PILs.
Reporting the Testing
• Detailed reports are needed for compliance.
• Different competent authorities require certain specific
information.
– Ensure your reporting contains information as needed
conforming to the most rigorous assessing authorities (it
may be useful for future bridging)
– Qualitative data should be reported
Readability Testing and Timing
• MRP
• DCP
• Centralised Procedures
• National Procedures
• Variations and Renewals
Timings for the PIL readability
testing
• Testing takes at least 3 weeks for 2 rounds to be performed in
an ideal environment.
– We have performed complete tests in 1 week for urgent situations.
• Most testing houses require at least 6 weeks.
• More time may be needed if the reference PIL is problematic
to user test. Leave time for text and art work revisions.
• A very detailed report should be submitted to adequately
address revisions made to the PIL if necessary.
MRP
 Require a readability test or bridging study for validation.
• Early Preparative Work for MRP PIL Readability Testing
– The PIL should be reformatted to QRD and reviewed to ensure
positive readability testing results.
– For Generics, comprehensive SmPC comparisons should be
complete at least 3-6 months prior to dossier completion and the
core SmPC & PIL compiled.
– Copying the originators or nationally approved text can lead to
problems during readability testing
– Discussions with the Competent Authority regarding the PIL
should be addressed for the sake of readability
– Preliminary testing is usually required to support revisions if
needed.
– User testing should begin at least 3 months before the
completion of the dossier to allow time for preparation of the
submission and PIL revisions if needed.
DCP
• Require a readability test, but it can be performed during the
clock stop.
• Hungary has been the only noted exception to this rule.
DCP
•
•
•
•
•
•
•
The PIL should be reviewed by an experienced person with readability testing
prior to beginning the procedure.
– Copying the originators text can lead to problems during readability testing
Testing should not be required for validation of the dossier (Hungary is a noted
exception)
The Preliminary Assessment Report issued at Day 70 (including comments on
SPC, PIL and labelling) …. “will include a comment on whether ‘user
consultation’ of the PIL has been performed or is foreseen, or the justification
for its absence is acceptable”.
The RMS will have the main responsibility for the assessment of the PIL.
Proposed core SmPC and PIL discussed with RMS and tentative timings.
Readability testing may be performed during the clock stop once the the SmPC
and PIL are agreed.
In situations where readability testing is performed prior to the procedure, the
MA Holder should be prepared to perform additional testing after Day 70
assessments as the PIL and SmPC may have extensive revisions.
Centralised Procedure
Centralised procedures have not consistently required readability
testing when the reference product has been tested and approved
by the innovator. Centrally approved products should have a
readability tested reference PIL available.
EMEA Guidance for the CP
Articles 59(3) and 61(1) of Directive 2001/83/EC, as amended, require
that the package leaflet reflects the results of consultations with target
patient groups (‘user consultation’) to ensure that it is legible, clear
and easy to use and that the results of assessments carried out in
cooperation with target patient groups are provided to the competent
authority.
• A user consultation is always required in the following situations:
• First authorization of a medicinal product with a new active
substance,
• Medicinal products which have undergone a change in legal status,
• Medicinal products with a new presentation,
• Medicinal products with particular critical safety issues.
EMEA Guidance for the CP
• However, reference to already approved package leaflets may
be acceptable where appropriate, based on a sound justification
by the applicant. Examples of when this may be considered
acceptable as well as the considerations to be taken into
account when choosing the types of ‘reference’ package
leaflets are detailed in the“Guidance concerning consultations
with target patient groups for the package leaflet”.”
• This topic should be discussed during the pre submission
scientific advice.
EMEA Guidance for the CP
• Originators have not been required to perform testing prior to
the procedure.
• Testing has been performed post approval.
• In the case of generics it is not 100% clear if you can use the
reference products PIL.
The key safety messages in the PILs for generic applications
should be identical to the CP reference products. However,
the generic applicant does not have rights of access to the
innovator’s PIL readability test data so will generally have to
test their PIL to prove compliance – style and formats will
differ and can contribute significantly to test outcome.
However, bridging is in theory possible.
National Procedures
• Required by the UK consistently.
• Performed before procedure. (Germany will allow it to be
performed during a clock stop)
• Other member states have staggered implementation.
• Compliance looming in France and Germany.
Variations and Renewals
• Reformating existing PILs to QRD
• New PIL readability tests where approvals pre-dated
mandatory PIL testing
• Bridging reports may be possible in other cases
• Readability testing implementation across EU has been
staggered.
• Wait for health agencies to require.
• Variation & Renewal Applications
• Testing has not been required to date across the EU on a
consistent basis
Experience and Feedback with Readability
Testing in the Generic Industry
• Feedback from the competent authorities for UK National
requirements, CP, DCP and MRP readability testing has been
minimal for testing performed by us and submitted by partner
companies.
• The MHRA has required all PILs for UK PILs to be tested
– Generics required to test or bridge all PILs
– Work sharing initiatives marginally successful
– Bridging results not capitalised on 100%
– Countless PILs tested by originators and generics for the same
APIs
– MHRA sometimes now require a separate PIL other than a
MRP or DCP approved PIL for products in their nation.
• Acceptance has been 100% for Pharma-EU
Experience
• The PILs themselves have been assessed and there is generally
always some revision required during the procedures.
• When the PIL (previously readability tested by the MA holder)
has been assessed and revisions have been required, additional
rounds of readability testing are rarely ordered. Bridging
suffices or in most instances, the revised PIL is approved.
(DCP can be an exception to this rule as it is common to have
“major revisions” needed to the PIL that are considered to
compromise the readability)
• For generic MRP and DCP procedures, the reference PIL has
now commonly already been readability tested by the
innovator. Generics commonly copy this PIL, so additional
testing has become more of a compliance issue, rather than an
exercise in PIL development.
Copying the Originators PIL
– Certain originator companies are performing poor quality PIL
development and testing, so the reference PIL can have
difficulty passing readability tests. This occurs commonly in
our experience.
– The competent authorities are reluctant to approve revisions
when a PIL has previously been tested and approved.
– The testing becomes simply an issue of compliance. We have
tested numerous PILs for the same APIs over the past few
years.
– From a testing house for generics our position is:
• Achieve a passing result to support the originators
findings.
• Use qualitative data to provide constructive feedback for
revising the PIL and leave the decisions to the competent
authority.
Work Sharing and Bridging
Experience to Date
• Work sharing initiatives amongst smaller companies such as
the BGMA scheme for the UK have saved our partner
companies hundreds of thousands of Pounds/Euros in
unnecessary testing.
• Bridging has been accepted as a common means for
complying.
• Bridging has literally saved our partner companies over a
million Euro in unnecessary testing.
• Cooperate with testing houses that have work sharing
initiatives. Don’t let your company be cornered into
performing millions of Euros of unnecessary readability
testing, when we already know the results!
Ensuring success with Bridging
• Bridging is defined as using the readability testing results of
the PIL for one medication to support the readability of the
PIL for a similar or the same medication.
• Bridging is commonly used for generics.
– There is no known way of gaining access to originators
readability testing results for use in bridging to date without
their direct consent.
Bridging
• An approved quality PIL and Readability Test are needed to
reference for bridging.
• It is a cost effective and pragmatic way to comply with the
burden of readability testing.
• We’ve achieved success with bridging for the UK Patient
Information Quality initiative and saved budgets over 1
million Euro on unnecessary readability testing.
• Bridging is also used for MRPs and DCPs when applicable.
• It is more useful and direct for developing uniform, patient
friendly PILs at this stage.
• It is our clients 1st choice before complete readability testing
now.
Bridging Guidance
EMEA “Guidance concerning consultations with target
patient groups for the package leaflet”
“The evidence from tests on similar package leaflets may be use
where appropriate. Examples of when this may be considered
acceptable based on a sound justification by the
applicant/marketing authorisation holder are:
• extensions for the same route of administration e.g.
intravenous/intramuscular or oropharyngeal/laryngopharyngea,
• same safety issues identified,
• same class of medicinal product.”
Bridging CMD Guidance
The CMD further defines 5 criteria which should be met to justify
a bridging report
• Same drug class, Line extensions for the same route of
administration
• Complexity of the message and language used
• Same safety issues identified
• Same patient population
• Same format of the PIL
What to expect with Bridging
• The worst result you can achieve with a properly prepared and
presented bridging study:
– A focus test (takes about one week to perform and costs
2,000 euro normally) followed by approval
• The best result you can achieve with a bridging study:
– Approval
Bridging Strategy
Bridging can be most effectively performed when there is a
strategy in place:
• Bridging on an Ad Hoc basis can be problematic if the PILs do
not conform to a “House style” format.
– Additional focus testing can be required
• When performing readability testing, care should be used to
ensure the most complex PIL of a line extension, or therapeutic
group is tested first and consideration should be used for which
one will be used by the most broad patient base.
– It is ideal and most effective to test the PIL considered most
complex in its class
– This helps support the readability of similar PILs with less or
slightly differing information.
Bridging Strategy Example
If you have a group of PILs for Product X with differing
pharmaceutical forms, but the same API, which would be the
most complex? (Generally Speaking):
• Infusion for Injection
• Tablets
• Capsules
• Prolonged release tablets
• Powder For Oral Suspension
Preparing PILs for Bridging
• PIL writing should be taught within the company to the
responsible people to ensure uniformity
– There are different styles which are acceptable, but one
school of thought should be subscribed to within a company.
• In originator companies, R&D writes PILs. In the generics, it is
usually a RO.
• Glossaries and templates should be used for PIL writing
• When preparing Module 1 for MRP or DCP, the PIL should be
presented in the company house style, not an exact copy of the
reference PIL.
– It is difficult and sometimes impossible to bridge different
writing styles to a certain company house styles.
– When you copy an originator’s PIL, chances are we will not
be allowed access to Readability Testing Reports performed
on PILs in their house style.
Effective Bridging
Bridging is most effective when the daughter
PIL is written and prepared for Bridging to the
Parent PIL or PILs!
• This shows assessors care was used not to compromise
readability
• It definitely shows the pragmatic approach of bridging and
leads to success
Successful Bridging
In order for bridging to be successful both the “parent PIL or
PILs” and “daughter” PIL should meet most of the criteria:
• Indication (identical or similar).
• Pharmaceutical form (it is most common to bridge different
forms and strengths).
• Headings and Subheadings (should be identical).
• Placement of information (ensure messages are in same location
on the PIL).
• Key messages for safe use (similar).
• Writing style should be similar.
– Same wording should be used wherever possible.
– Layouts, headings and descriptions should follow the same
format.
Successful Bridging
• Formatting of sections should be identical or closely similar.
• Complexity of the message should be similar.
– Meaning a PIL with 2 listed contraindications should not be
used to try and support a PIL with 10 contraindications.
• The artwork should be similar .
– Same dimensions of template.
– Same use of graphics, fonts and print size.
• It has become common to bridge PILs to 2 previously tested PILs
to ensure all criteria is met. Multiple “Parent” PILs can be used.
Number 1 Point for Bridging
The number 1 point for ensuring a successful bridge is well
prepared PILs and readability tests! The reference PIL should be
written to the highest quality and supported with a Readability
Test. The PIL being bridged should also be written to the highest
quality incorporating the text of the Parent PIL or PILs where
ever possible.
Bridging Reports for Different
Nations
• Yes, it is possible.
• MEB has previously allowed for UK (MHRA) approved tests
to be submitted with bridging studies for assessment.
• Success depends upon assessors in AFSSAPS and BfArM
appraisal of the situation.
• National licenses have quite a few differences when compared,
so additional testing may be required.
• Major originator companies are trying to use the UK testing
results for the French market.
Budgeting for Readability
Testing
For nations other than the UK, where readability
testing will become a major burden, work
sharing initiatives and well thought out bridging
strategies will save the Generics Industry
Millions!
Thank you for your attention!
If there are any other questions, please do not hesitate to contact me:
Kurt Kinsey
Pharma-EU s.r.o.
Malostranske Namesti 23
Prague 1
118 00
Czech Republic
Telephone: 00 420 739 355 429
kurt@pharma-eu.com
References:
• European Commission's Draft Guidance: Revised September 2006
“Guideline on the Readability of the Label and Package Information
Leaflet of Medicinal Products for Human Use”
• European Commission's Guidance: May 2006 “Guidance concerning
consultation with target patient groups for the package leaflet”
• EFPIA Document: March 2002 General Recommendations for Readability
User Testing of Package Leaflets for Medicinal Products of Human Use
Submitted and Approved under the European Centralized Procedure
• CMD(H): October 2007: Consultation with Target Patient Groups meeting the requirements of article 59(3) without the need for a full test recommendations for bridging