HIV Drug Resistance Training
Module 11:
Quality Control and
Quality Assurance
1
Topics
Quality Assurance vs. Quality Control
Quality Assurance Before, During, and After
Testing
 Quality Control
 EQA


2
Objectives
Identify requirements for QC/QA.
Identify types of quality controls that must be in
place and the rationale for each.
 Given individual lab needs, develop a plan for
setting up an internal QC program.
 Identify types of quality assurance programs and
their purpose.
 Given individual lab needs, develop a plan for
setting up an external QA program.


3
quality assurance vs. quality
control
What is the difference between QA and QC?
4
Terms


Quality Control
Quality Assurance
 Quality Assessment
5
Definitions
Term
Definition
Quality
Control
The measures that must be included during each
assay to verify that the test is working properly.
Quality
Assurance
The overall program that ensures that the final
results reported by the laboratory are accurate
Quality
Assessment
A means to determine the quality of the results
generated by the laboratory. It is usually an
external evaluation of the laboratory's performance.
Quality assessment is a challenge to the
effectiveness of the QA and QC programs.
6
Quality Assurance vs. Quality Control
Quality Assurance
Activities to ensure process
are adequate for a system
Definition to achieve its objectives
Establish standard
procedures for sample
collection
 Define criteria for
acceptable samples

Examples
Quality Control
Activities to evaluate a
product or work result
Analyze known QC
sample to determine if a
test is valid
 Decide if a sample is
acceptable for testing

7
Quality Control is Part of Quality Assurance
Quality
Assurance
Quality
Control
"The aim of quality control is simply to ensure that the results generated
by the test are correct. However, quality assurance is concerned with
much more: that the right test is carried out on the right specimen, and
that the right result and right interpretation is delivered to the right
person at the right time.”
8
QA or QC?










HIV-negative plasma control
SOP writing
Personnel records
Phylogenetic analysis
DBS condition inspection
Instrument calibration
Proficiency testing
New operator training
Assay validation
Lot release testing
9
The Quality Assurance Cycle
Sample
collection
Personnel
competency
testing
Reporting
Record keeping
and data
management
• Data and Lab
Management
• Safety
• Customer Service
QA and QC
Genotype
testing
Sample receipt
and
accessioning
10
Discussion

What is the difference between QA and QC?
11
quality assurance before, during,
and after testing
What are some examples of errors that occur before, during, and
after testing, and what can we do about them?
12
The Quality Assurance Cycle
Sample
collection
Personnel
competency
testing
Reporting
Record keeping
and data
management
• Data and Lab
Management
• Safety
• Customer Service
QA and QC
Genotype
testing
Sample receipt
and
accessioning
13
Pretesting
Sample
collection
Personnel
competency
testing
Reporting
Record keeping
and data
management
• Data and Lab
Management
• Safety
• Customer Service
QA and QC
Genotype
testing
Sample receipt
and
accessioning
14
Preventing and Detecting Errors
Before Testing










Lab personnel training and certification
Sample criteria set and used
Specimen labeling, collection and transport
Specimen storage and processing
Check storage and room temperature
Lab set up
Reagent inventory and expiration dates
SOPs for all related procedures
Preventive maintenance and equipment checks
External Quality Assessment (proficiency testing)
15
During Testing
Sample
collection
Personnel
competency
testing
Reporting
Record keeping
and data
management
• Data and Lab
Management
• Safety
• Customer Service
QA and QC
Genotype
testing
Sample receipt
and
accessioning
16
Preventing and Detecting Errors
During Testing






Conducting test according to SOPs
Quality Control (QC) (in-process controls)
Safety precautions
Prepare reagents
Correctly interpreting test results
Time management and planning
17
Post-testing
Sample
collection
Personnel
competency
testing
Reporting
Record keeping
and data
management
• Data and Lab
Management
• Safety
• Customer Service
QA and QC
Genotype
testing
Sample receipt
and
accessioning
18
Preventing and Detecting Errors
After Testing




Re-check patient/client identifier
Write legibly
Clean up and dispose of contaminated waste
Complete quality control review
– Positive and negative control results
– Contamination check (phylogeny)
– Other QA checking (e.g. SQUAT)



Transcribing, reporting and interpreting results
Critical review of results and interpretation
Record Maintenance
19
SQUAT: Sequence Quality Assessment Tool
QA tool developed for WHO by Dr. Rami Kantor
and colleagues, Brown University
 Open-source and freely available
 To be used by genotyping labs before sending
sequences to national HIVDR working groups for
surveillance and monitoring activities

– Labs should perform initial investigations and take
corrective actions if required

Sequences can be assessed by WHO for external
quality assurance as part of ongoing accreditation
process
20
SQUAT Functions
Sequence alignment
HIV PR and/or RT sequence identification
Mutation identification and generation of export
file for uploading to database
 Flagging of problematic sequences



–
–
–
–
–
–
ambiguous bases (mixtures)
stop codons
insertions/deletions
frameshifts
genetic distance anomalies
atypical and ambiguous amino acids
21
Quality Assurance is Everywhere!
Sample
collection
Personnel
competency
testing
Reporting
Record keeping
and data
management
• Data and Lab
Management
• Safety
• Customer Service
QA and QC
Genotype
testing
Sample receipt
and
accessioning
22
Find It, Then Fix It
Problem
Corrective
Action
Use a problem-solving team.
Investigate root causes.
Develop and implement appropriate corrective
actions.
 Examine effectiveness.
 Record all actions and findings.
23



Discussion
Why do errors occur?
What are some common errors that might occur
with HIVDR Genotyping?
 Where is QA applied in a HIVDR testing site?
 What are some steps to take before, during, and
after testing to assure the quality of results?
 Describe the impact that errors will have on
public health.


24
Quality Control
What do we need to do to set up a quality control program for our
genotyping?
25
What is quality control (QC)?
Monitoring the test quality itself
Ensuring that the assay is working appropriately
and accurate test results are reported with
confidence.
 Prevention
 Detection
 Correction


26
QC: Two Levels/Two Types

Levels
– Testing of samples with known results to verify the
procedure is working properly
– Interpreting the presence or absence of controls.

Types
– Internal quality control
– External quality control
27
Internal QC

Internal QC samples with known reactivity are
included with the following HIVDR test kits.
– TruGene
– ViroSeq
For in-house assays, prepare and qualify your
own internal QC samples
 Test the IQC samples as you would test patient
specimens.
 Include positive and negative controls in every
run.
 Know what to do in case of unexpected results

28
Genotyping Assay Procedures
Patient virus
RNA Extraction
cDNA synthesis (RT)
Positive and
negative controls
cDNA amplification (PCR)
Sequencing reactions
Positive control
Electrophoresis
Data Analysis (base calling)
Report (sequence, mutations, interpretation)
29
Guidelines for HIVDR Genotyping Internal QC
Policies

Negative controls
– HIV-negative plasma, at extraction step
– Water, at RT step and at PCR step
• If any negative control shows a detectable PCR product, all
samples must be discarded
– Investigate to locate source of contamination

Positive controls
– HIV-positive plasma, at extraction step
– HIV DNA at PCR step (low copy number)
• If positive control does not produce a PCR product, investigate
to determine cause; handling of products from test samples
may depend on outcome
– HIV DNA at sequencing step

SOPs required
30
External QC

External QC samples do not come with the test
kit and are provided by an external source
– Regional reference laboratory
– Commercial supplier
– Another laboratory that has prepared validated quality
control in-house samples

Test the EQC samples as you would test patient
specimens.
31
Internal or External Control Failure


If QCs fail, repeat the run.
If problems or errors occur, then corrective
actions should be taken before results
distribution.
Problem
Corrective
Action
32
Planning Your Internal QC Program



Incoming samples
Incoming reagents and kits
Methodology or instrument comparison
33
Incoming Samples

Upon arrival, all incoming sample shipments
should be evaluated for proper collection,
transport and storage
34
Incoming Reagents and Kits
Evaluate all incoming reagents and kits upon
arrival.
 Perform lot-to-lot comparisons.

– Assay previously tested patient samples or controls on
both the previous lot and the new lot and compare the
results.
– When a new lot has been evaluated and ‘passed,’ label
container “Ready to Use.”
35
Lot-to-lot Comparison for Critical Reagents

Lot-to-lot comparison for critical reagents:
– For commercial kits:
• Perform parallel run with new vs. existing kits
• Define acceptance criteria for reagent lot to lot
comparison
– For home-brew assays:
• Perform parallel run with new vs. existing lots
• New lot of primers, RT-PCR enzymes, sequencing
reagents

Source of specimens
– Positive controls
– Previously tested samples
– Old proficiency panels
36
Critical Reagent Quality Control
Extraction buffers
Enzymes (RT, DNA
polymerase)
RNA Purification
RT-PCR
Primers
Reaction buffers
Sequencing reactions
Capillary matrix polymer
Electrophoresis
Results
37
Methodology or Instrument Comparison
For tests that have multiple instruments available
in the laboratory for use, compare instruments.
 Results should be comparable.
 Compare and document results routinely
(depending on lab throughput).

=
?
38
Small Group Discussion
1.
2.
3.
4.
5.
What do you need to do to set up a QC program
for genotyping? What components should be
included?
If a RT-PCR negative control becomes positive,
what should you do?
How can you perform lot-to-lot
comparisons in your lab ?
What is critical to include in a QC SOP?
Once you are done with a lot-to-lot comparison
and obtained satisfactory results, what should
you do for the next step?
39
EQA
What is EQA and why is it important?
Which EQA Methods should we employ?
• Proficiency Testing
• On-Site Evaluation
• Re-testing
40
A Systems Approach to Laboratory Quality
Organization
Personnel
Equipment
Stock
Management
Quality
Control
Data
Management
SOP,
Documents &
Records
Occurrence
Management
Assessment
Process
Improvement
Specimen
Management
Safety &
Waste
Management
41
External Quality Assessment (EQA):
Definition and Purpose






Objective assessment of a test site’s operations
and performance by an external agency or
personnel
Allows comparison of performance and results
among different test labs
Provides early warning for systematic problems
associated with kits or operations
Provides objective evidence of testing quality
Indicates areas that need improvement
Identifies training needs
42
Management Responsibilities








Determine policies for EQA (WHO, WHAT, WHEN,
HOW)
Assign responsibility
Establish & maintain a system and schedule
Receive EQA results and support corrective action
measures
Manage corrective action efforts
Monitor and maintain records
Investigate deficiencies
Communicate outcomes
43
Testing Personnel’s Responsibilities:
Overview




Participate in the EQA program
Take corrective actions
Maintain EQA records
Communicate outcomes to supervisors
44
EQA Methods
On-site
Evaluation
Retesting
Proficiency
Testing
45
What is Proficiency Testing?
Panels of specimens are sent
to multiple test labs by
reference laboratory/EQA
provider
 Test labs perform tests and
report results
 Results indicate quality of
personnel performance and
test lab operations
 Results are often compared
across several testing labs

Proficiency
Testing
46
Proficiency Panels for HIVDR EQA







VQA (NIH-supported, Rush University, Chicago1)
TAQAS (TREAT Asia2)
ENVA/QCMD3
ANRS France
DigitalPT/Accutest
Acrometrix
Others?
1. Huang et al. J Clin Microbiol.43(8):3963-70 Aug 2005
2. Land et al. Journal of Virological Methods 159(2): 185-193; Aug 2009
3. Pandit et al. J Clin Virol. 43(4): 401-6; Dec 2008
47
What is On-site Evaluation?

On-site
Evaluation
Periodic site visits to perform
systematic assessment of lab
practices
– Focuses on how the lab
monitors its operations and
ensures testing quality
– Provides information for
internal process improvement
48
What is On-site Evaluation? – Cont’d



On-site
Evaluation


Also referred to as audits,
assessments, or supervisory
visits
Learn “where we are”
Part of every lab quality
system
Measures gaps or deficiency
Collect information for:
– Planning & implementation
– Monitoring
– Continuous improvement
49
What is Re-testing?
The process by which a
random selection of
specimens are collected from
the routine workload at the
test lab and sent to the
reference laboratory for
validation
 Used to detect errors

Retesting
50
EQA Should Lead to
Corrective Actions
EQA
Identify
problems
Take
corrective
action
51
Sample of Corrective Action Form
52
Small Group Discussion
What are the methods in a EQA program?
What type of EQA is most appropriate in our
own situation?
3. What is the potential of internal bias in selfevaluation of quality?
1.
2.
53
How To Implement EQA
Proficiency
testing
Retesting
On-site
evaluation
54
On-Site Evaluation Process
Preevaluation
preparation
Entrance
interview
Information
gathering
Outcome
assessment
Exit
conference
Reporting
55
On-Site Evaluation Checklist
56
Tester Responsibilities:
Ensuring a Productive Site Visit

Before Visit:
–
–
–
–

Record keeping is essential. Get organized
Confirm date of visit
Review written policies and procedures
Conduct internal assessment in preparation of site visit
During Visit
– Participate in visits – cooperate
– Ask questions of site assessors

After Visit
– Take corrective actions, where necessary

Remember – visits are instructive, not punitive
57
On-site Evaluation:
 Pre-Evaluation Preparation

Assign Responsibility
– Laboratory management
– Quality Manager


Use Integrated team approach
Determine who will conduct on-site evaluations
– Select auditors with necessary skills:
• Attention to detail
• Ability to communicate effectively
• Diplomacy
– Provide appropriate training

Schedule site visits
58
On-site Evaluation:
 Pre-Evaluation Preparation

Determine site visit frequency:
– Established sites - twice per year/once per year


For efficiency, cluster evaluations geographically
Schedule in advance
– Announced or unannounced visits
59
On-site Evaluation:
 Entrance Interview
The entrance interview sets the tone for the
entire visit
 Provide overview of process in terms of what will
be done

–
–
–
–
–
–
Review of facility
Record review
Observation
Interview with testing staff
Use of proficiency panel
Exit interview
60
On-site Evaluation:
 Information Gathering


Observe physical layout of the lab
Evaluate testing operations
–
–
–
–
–


Specimen collection
Observation of test performance
Quality control
Inventory of kits
Record-keeping
Use quality systems checklist
Conduct in an instructional, not punitive, manner.
61
On-site Evaluation:
 Outcome Assessment
YES
Evidence of
implementation
& maintenance of
quality system?
• Conclude visit
• No deficiencies
NO
• Ensure visit is
sufficient to
objectively document
evidence
• Expand visit or
observation
• Reference
deficiencies
62
On-Site Evaluation:
 Exit Conference
Review findings with supervisory and testing staff
Make positive statements first – acknowledge
staff cooperation and support
 Address negative findings – allow test site to
discuss findings and provide additional
information
 Provide instructions and timeframe for submitting
plan for correcting problems


63
On-site Evaluation:
 Reporting

Include information:
–
–
–
–
–

Lab Name & Location
Date of Visit
Assessment Team Members
Major Findings
Recommendations for corrective actions
Submit completed checklist and report to relevant
authorities
64
Minimal Package for QC/QA
WHO-accredited DR labs should be enrolled in at
least one EQA program
 Perform self-assessment at least once a year
 Internal/external controls must be included in
each assay run and the results recorded.

– Trending over time can be a very useful quality
indicator

Clearly define valid testing run criteria in related
SOPs
65
Discussion


What is EQA and why is it important?
Which EQA Methods should we employ?
– Proficiency Testing
– On-Site Evaluation
– Re-testing
66
Reflection

What are our responsibilities in preventing,
detecting, and correcting errors before, during,
and after testing?
67
Summary
Identify requirements for QC/QA.
Identify types of quality controls that must be in
place and the rationale for each.
 Given individual lab needs, develop a plan for
setting up an internal QC program.
 Identify types of quality assurance programs and
their purpose.
 Given individual lab needs, develop a plan for
setting up an external QA program.


68