The Importance of Validated Rapid Methods for
International Trade
Robert Koeritzer
Sr. Technical Manager
3M Food Safety
© 3M 2009. All Rights Reserved.
Past-President, AOAC International
Agenda
Method validation
– Complex global web
– Comparison of various schemes
– Moving towards harmonization
Trends in method validation.
2
Analytical Science in Trade/Commerce
Codex Technical Committees
Import Testing
Emerging Chemical
Contaminates
3
Quality Systems
Emerging Pathogens
European standards
EN ISO 17025
Manufacturer
QC/QA
4
Horizontal Methods
Preparation of
test samples
Alternative methods
Vertical
EN ISO 8261 / IDF 122
diluents
EN ISO 6887 1-2-3-4
Accreditation
Reference methods
International
European national
Routine
Routine
ISO NF ISO
National
National
or EN NF EN,…
NF,DIN
DIN,…
,… or EN ISO
NF EN ISO
NF,
EN ISO 6887-1
EN
ENISO
ISO16140
16140
Validation
Validation
ISO 11133 : Guidelines on QA and performance testing of
culture media
ISO 7218 : General rules for microbial examination
European Recognition Committees
(Alternative Method)
Country
France
Germany
UK
Nordic
Europe
Standards
Bodies
AFNOR
DIN
BSI
NMKL
CEN
Recognition
Systems
AFNOR
None,
Technical
report
EMMAS NordVal
Since 2004, Standardisation of the alternative method
performance assessment with EN ISO 16140 is possible
MicroVal
European requirements for alternative methods
Requirements:
“The use of alternative analytical methods is acceptable, when the
methods are validated against the reference method set in Annex and
certified by a third party in accordance with the protocol set in EN/ISO
standard 16140 or other internationally accepted similar protocols.
If the food business operator wishes to use analytical methods other
than those validated and certified as described above, the methods
shall at least be well documented and scientifically validated.”
Consequences:
6
AFNOR validated methods will be accepted everywhere in Europe
AOAC validated methods will be accepted if they have been validated
against an EN ISO method
Three major global validation
AFNOR (ISO 16140 protocol)
MicroVal (ISO 16140 protocol)
AOAC International (AOAC protocol – Harmonized with ISO 16140)
7
Validation Methods Comparison
Overall
AOAC R-I, PTM
AOAC Intl OMA
ISO 16140:2003(E)
Proprietary
Proprietary & Noncommercial
Proprietary &
Non-commercial
Reference
Methods
AOACI, FDA, USDA, ISO
AOACI, FDA, USDA
ISO, CEN
Time to
Validation
6-12 months
12-18 months
(minimum)
12 months
(minimum)
Methods
Reviewed
Required yearly
N/A
Required every 4 yrs
Type of
Method
8
Part 1: Methods Comparison
[Pre-collaborative]
AOAC R-I, PTM
AOAC Intl OMA
ISO 16140:2003(E)
Food Categories
Claim Dependent
6 (for all or most foods)
5 (for all or most foods)
Food Types
Claim Dependent
20 representative food types
3 per category
3 (H, M, L) + uninoculated
3 (H, M, L) + uninoculated
5 levels
if necessary
if necessary
5 per level
5 per level
2 per level (preferably 5-10)
Required in 1 food type
Required in 1 food type
NA
Required
Suggested
NA
Level of Analyte
Quantitative
# of Samples
Competitive micro
flora
Ruggedness
9
Part 2: Methods Comparison
Inter-laboratory study (Collaborative)
AOAC R-I, PTM
AOAC Intl OMA
ISO 16140:2003(E)
Participating Labs
NA
Minimum of 8
Minimum of 8
Food Categories
NA
6 (for all or most foods)
1 Food Proposed 4/2010
1
Food Types
NA
1 per category
1
Level of Analyte
NA
3 contamination levels, plus
uninoculated
3 contamination levels, plus
uninoculated
if necessary
if necessary
2 per level
2 per level
# of Samples
10
NA
AFNOR
The principles of the certification procedure
The method validation study is carried out in 2 phases:
[technical protocol based on EN ISO 16140]
Phase 1: (preliminary study)
Characterize the method and assess it’s effectiveness by
comparing it to the effectiveness of the standardized reference
method.
Comparison carried out at the expert laboratory.
Diversity of samples and experimental conditions (sufficient number of
naturally contaminated samples) reflect real conditions in the field.
11
AFNOR
The principles of the certification procedure
Phase 2: (inter-laboratory study)
Compare the efficacy of the method to be validated and the
reference method, in several laboratories (8-10 minimum).
Under defined conditions of reproducibility and repeatability.
AFNOR Difference
– Emphasis on sample diversity (natural contamination)
– Confirmation of positive results
– Evaluation of the quality assurance procedure at the production site
12
MicroVal Offers:
13
An open platform for all European Certification Bodies.
Aiming at a harmonized interpretation of the EN ISO 16140.
Working towards a harmonized international market for all test kits.
Is headed by one European platform with all European stakeholders involved.
A neutral MicroVal secretariat not part of a certification organization.
14
15
AOAC International Validation/Standardization
16
Volunteer Consensus Standards
Process of developing standards is open and transparent, with written procedures
covering each step of the process, such as you find in the Official Methods
Program.
System is known as voluntary because participation is on a voluntary basis and
because compliance with the standards is voluntary (unless adopted or referred to
by a government authority, like Official Methods)
17
Initial Method Validation
“Pre-collaborative
Performance parameters
–
–
–
–
–
Precision
Accuracy
Ruggedness
Linearity
Limit of detection
Comparison to other methods
Acceptable method performance (Expert Lab study)
Meets Claims in package insert
18
Method Collaborative Study
8-10 laboratories minimum
– Minimum of 8 labs with acceptable data (chemistry and quantitative microbiology)
– Minimum of 10 labs (qualitative microbiology)
5 samples or test materials (chemistry)
– Materials must cover the range of assay applicability
Blind duplicates
Standard RM or Control
19
AOAC “Communities” model
Issue/Need
Training & Conferences
Collaborative
Study Validation
Engagement
Industry/Regulatory/Academia
Single Lab
Validation
Community
Formed
Expert Review
Panel Formed
20
AOAC Communities
Marine & Freshwater Toxins
– Gathering Stakeholders
(Industry and regulatory)
– International Workshops
– Prioritize methods needed
– Develop methods using “best science”
– Validate methods
– Set the industry standards
21
AOAC Communities
Dietary Supplements
– Gathering Stakeholders
(Industry and regulatory)
– Prioritize methods needed
– Develop methods using “best science”
– Validate methods
– Set the industry standards
22
Trends in Method Validation: Business Perspective
Cost
Reference Lab Government/
Regulatory
Diagnostic
Manufacturer
Food Processor
Time to Results
Routine
Diagnostic
Use
Accuracy
Inclusivity
Exclusivity
Validation Costs: People/Resources, Time, Euros
23
Diagnostic Manufacturer Validation Costs and Time
Validation Costs per Method
$70,000
$60,000
$50,000
$40,000
$30,000
$20,000
$10,000
$-
Study/Lab Expenses
Application Fees
AOAC-PTM AOACOMA
AFNOR
Time to Complete Validation (per Method)
Acronyms:
AOAC-PTM: Performance Tested Method
AOAC-OMA: Official Method of Analysis
40
35
30
25
Weeks 20
15
10
5
0
Protocol, Analysis,
Submit Results
Complete Study
AOAC PTM Initial
Approval
AOAC-PTM AOAC-OMA
24
AFNOR
Food Processors: Wait and Re-Validate?
Post Method Approval Validation
– Food Matrix and Diluent Validation
– Local Flora Evaluation
– Local Reference Method Validation
For EACH country!
Protection from Litigation
OR: Internal Validation (AOAC or AFNOR protocol)
–
–
–
–
25
Third-Party approvals are taking too long
Internal Study: cost can be US$ 200,000
External (reference lab) Study – usually very limited in scope
Participation in Proficiency Testing Programs
Governments & Regulatory Agencies Validations
Regulatory Balancing Act
– International approvals acceptable
for international commerce
– Are validated proprietary methods
acceptable for testing local foods?
• If “yes”, what validations are
acceptable?
Better test
Accuracy
Government Challenges
– Scarce Resources
– Who pays for the validation study?
– Proprietary Method Validation
Processes
– Many proprietary methods are
seeking governmental validation
26
Consumer
Safety
Time and
Resources
Governments & Regulatory Agencies Validations (con’t)
Collaborative Model: AOAC Official Method of Analysis
Food
Processor
Government
Agency
AOAC
Collaborative
Diagnostic
Manufacturer
Regulatory Agency
Method Adoption
– Are Third Party validations diminishing in importance?
– Fewer Food Processors are volunteering for collaborative studies
AOAC Performance Tested Method
– 3rd Party Validation
– Lower Cost; Shorter Time
– AOAC PTM approvals are more compatible with rapidly changing technology
27
Validation Trends
Cost and Time Requirements Increasing
– Increasing number of products requiring validation
– Increasing numbers of regulatory agencies
– Products marketed to more and more countries (import
regulations)
Convergence
– Harmonization of methods
• AOAC and ISO 16140 closely aligned
– “Fit for Purpose” Validations
• AOAC Performance Tested Method (AOAC RI)
• Screening versus Actionable
28
29
30
Back Up Slides
31
BPMM Methodology
Review scientific literature.
Review current International Standards.
Consult other (topic) experts.
Review ISO, AFNOR, and AOAC validation protocols.
Discuss/Debate via teleconference and email.
Make consensus recommendations.
32
Key Issues To Address
Does the intended purpose of a method necessitate different levels
of sampling and testing.
Consider factors influencing confidence (uncertainty) of data.
Method Extension to new matrices and/or target organisms.
33
Key Issues To Address
Validation in absence of a reference method, confirmation procedure, or superior
performance of new method
Scientific and statistical bases for:
– Determining the limit of detection
– Developing performance standards for methods
– How to validate performance to meet a performance standard?
What are effective means for determining and articulating the uncertainty
associated with microbiological methods?
34
BPMM
Recommendations
Statistics Work Group
Use International Standards
• ISO 16140 (validation of proprietary methods)
• ISO 5725 (precision & trueness)
• ISO 11843 (capability of detection)
• ISO 19036 (measurement uncertainty)
Use robust statistics
Define Measurement Uncertainty
• Use LOD50 for Qualitative methods
• Use ISO 19036 for Quantitative methods
35
BPMM
Recommendations
Matrix Extension Work Group
Standardize recommended species/strains
(include in ISO 16140 revision)
Use categorization of food groups based key attributes – Charts define when
additional validation is needed (include in ISO 16140 revision)
– Focused on testing for inhibitory or other unique technical considerations that might
prevent application of the method
36
BPMM
Recommendations
Sampling Work Group
Follow ISO standards. (statistical process control)
Identify and discuss sources of variation as part of method
validation.
37
BPMM
Recommendations
Detection Limits Work Group
Use a detailed protocol for preparation of inoculum.
(include in ISO 16140 revision)
Confirm method results where alternative method is more sensitive
then reference method. (include in ISO 16140 revision)
38