New_York_State_Guidelines

New York State Guidelines







CLINICAL APPLICATION OF NEXT GENERATION SEQUENCING AND IMPLICATION OF

NY STATE GUIDELINES

Hannah Murfet (BSc, PCQI), Product Quality Manager, Horizon Discovery

This article aims to provide an overview of the clinical application of NGS and the implication of the recently released NY State Guidelines. A summary of the validation requirements and use of reference materials is below:

Validation Requirements:

Test for Accuracy: Sequence a well-characterized reference sample to determine error rate across all target areas.

Test for Precision: This must be test on patient samples. Prior to final validation non-patient defined mixtures of cell line DNAs (not plasmids) can be used to will help with initial optimisation and understanding of precision for the detection of variants at the same codon.

Understand Analytical Sensitivity: Establish the sensitivity of variant calling for each type of variant. This can initially be established with defined mixtures of cell line DNAs (not plasmids) followed by 3 – 5 patient samples.







Reference Standard/Control Use:

A positive/sensitivity control should be included in each NGS run.

This control should be a low positive DNA sample containing multiple known variants(mutations) at challenging sensitivities to test the workflow and platform

A defined rotation schedule should be employed if not all variants can be covered in a single control sample.

The clinical applications of Next-Generation Sequencing (NGS) technologies are continuing to advance.

Sanger sequencing is still generally considered the gold standard but this comes with low throughput and high cost. However advances in this Next Generation Sequencing are reducing the cost of sequencing, whilst facilitating new mechanisms for disease prediction.

Next Generation sequencing is becoming an appealing option with much higher throughput and potential to change the face of genetic medicine (American College of Medical Genetics and Genomics, 2013).

However care must be taken to ensure high levels of control are maintained, this can be achieved through validation and reporting controls where specific NGS requirements are expected, in addition to those mandated by clinical accreditation and certification programmes.

t: +44 (0)1223 655580 f: +44 (0)1223 655581 e: info@horizondiscovery.com w: www.horizondiscovery.com

Horizon Discovery Ltd., Building 7100, Cambridge Research Park, Waterbeach, Cambridge, CB25 9TL, United Kingdom

UK Registration No: 05363294 UK VAT No: GB 189 1993 44

Clinical Context of NGS

Meeting Requirements

Figure 1: Overview of the Clinical Context of NGS.

Figure 2: Overview of Clinical NGS Requirement areas.

In the clinical context quality assurance consists of the maintenance of a desired level of quality for laboratory services. Systematic implementation of a quality management system helps maintain consistency and quality of services offered, whilst allowing the development of efficiency and definition of authority.

Quality assurance is essential to ensure efficient and reproducible operations and document controls form one essential part of quality assurance. While the requirements specifically identify Standard Operating

Procedures (SOPs) as a requirement, supporting documentation will help provide context.

Typically quality management systems take a three tier hierarchy. At the highest level the policies define the organisation’s strategy and focus; here you will find organisational goals, expectations and commitments. Procedures define and document instructions on performing business/quality management or technical activities. In the case of New York State Department of Health guidelines, there is clear focus on the requirement for SOPs.




Figure 3: Document hierarchy for management system documents.

SOPs can be broken into two levels. The first level represents the flow of information and useful for demonstrating the sequence of event and associated responsibilities or authorities. The first level procedures are best kept at a relatively high level and work well as flow diagrams supported by supplementing text such as the example below.

Figure 4: Level one process example including steps, assigned responsibility/authority and sub steps. A level

one process may reference more specific and detailed level two processes.

An overview of the typical testing sequence is shown below; this may be incorporated into one or a few level one processes depending on the complexity of the clinical laboratory’s operations.

Figure 5: Testing sequence overview.




Level two processes are best documented as clear ‘how to guides’, detailing all the responsibilities, materials and procedures necessary to complete the activity. For laboratory focussed activities, these are best used as inputs to and outputs from validation studies to establish consistent and clear protocols that support good training and operation of the laboratory.

Records are an essential practice and should include which test instruments were used in each test and documentation of all reagent lot numbers. Any deviations from standard procedures should be recorded, including any corrective measures to report and resolve any deviations. Templates may be generated to ensure consistency in output records for both testing and reporting.

In addition to documented processes, implementation of predetermined checkpoints or key performance indicators should be included to permit the monitoring of quality assurance over time. Once established these may act as a trigger for assay drift, operator variability or equipment issues.

Controls for reports and data must be implemented. In the US compliance to the HIPAA Act (Health

Insurance Portability and Accountability Act) must be implemented to ensure traceability and protection of patient data. Many authorities mandate record retention periods, including CLIA who mandate that records and test reports must be stored for at least 2 years. With strong implementation of quality assurance systems clinical laboratories can assure reliability and continuity of results and may look to further certification such as the implementation of ISO 15189, especially in countries where there are no formal accreditation schemes.

Validation for clinical laboratories involves the assessment of various parameters including those used for protocols, tests, materials and platforms. The aim of validation is to provide confidence to meet critical requirements. The following data section (section 3) should be considered when determining the scope of validation required.

Figure 6: Validation and assurance levels.

An overview of the specific validation requirements for Next Generation Sequencing is given below, this may be used as a basic checklist of coverage, supplemental to more general accreditation or certification requirements e.g. those required by CLIA or ISO 15189.




Figure 7: Overview of NGS Specific Validation Requirements.

Given the huge amount of data that may be generated through Next Generation Sequencing, accurate and efficient systems are essential. Data is generally established through validation and then monitored through the establishment of predetermined checkpoints or key performance indicators to ensure consistency and accuracy of service provision.

Accuracy

 Validation including a minimum of 50 patient samples with representation for material type (FFPE) and variants across target areas and confirmed by an independent reference method. Minimum of 10 positive samples for each type of variant.



A recomended approach is to sequence a well characterised reference sample to determine specificity.

 If vigorous validation of reported variants has not been completed in the original studies, ongoing confirmation by independent reference methods must be performed until at least 10 reference points have been independently validated.



A disclaimer must be used where incidental finding of unknown significance are included where there is no established confirmatory assay. The disclaimer must clearly state that the variant has not been verified.

Robustness



Robustness is the likelihood of assay success; adequate quality control measures must be in place to determine success of techniques such as extraction, library preparation or sequencing.




Precision



Precision is related to within run control.



For each type of variant a minimum of 3 positive samples containing variants near the stated sensitivity of the assay must be analysed in triplicate in the same run using different barcodes.



Renewable reference samples can be used to determine the analytical validity of the test. These can establish baseline data to which future modifications can be compared.

Repeatability and Reproducibility



Repeatability and reproducibility is related to between run controls to determine ability to return identical results under identical (repeatability) or changed (reproducibility) conditions.



For each type of variant a minimum of 3 positive samples containing variants near the stated sensitivity of the assay must be analysed in three separate runs using different barcodes on different days by two different technologists where possible.



If multiplexing samples with distinct barcodes, it must be verified that there is no cross talk and that all target areas and variants are reproducible independent of which patient/barcode combination is used.



It is useful to consider instrument-instrument variability as well as interoperator variability. Parameters for expected reproducibility should be established, typically around 95-98%.

Analytical Sensitivity and Specificity



Sensitivity and specificity refers to positive and negative percent respectively, in agreement of results when compared to gold standard.



Interrogate all types of variants in three target areas with consistently poor coverage and three with consistently good coverage. This can be established with defined mixtures of cell line DNA and not plasmids, but needs to be verified with 3 – 5 patient samples.



The limit of detection should be established.



Confidence intervals for variant types must be determined.

Table 1: Overview of NGS Specific Data Requirements.

A minimum data set is expected to establish key performance characteristics:-

Figure 8: NGS minimum data set.




In contrast to quality assurance where the infrastructure for quality is established to maintain the right service, quality control addresses testing and sampling to confirm outputs against requirements. Quality control takes place across various areas from reagents used, software and in assay controls:

Overview of Quality Controls



Quality control of reagent lots is best implemented at the point of goods in inspection. A clear label should be placed on the reagent under inspection then testing can be performed to validate/confirm analytical sensitivity.



Quality control of software updates can be handled through a version control and impact assessment process.

All re-validation must be clearly documented and demonstrate consistency in analytical sensitivity.



Sample identity is essential especially if samples are pooled. Proficiency testing protocols must be established to allow for execution as required by clinical accreditation bodies (such as CLIA).

 Quality control stops may be added to laboratory process before the sequencing run, to the run itself and at the end before data analysis.



Use of control materials such as positive and negative controls. An overview of these is presented in the next figure.

Table 2: Overview of quality controls.

Figure 9: Control Requirements.

Several QC requirements may be required and documentation of these requirements is best detailed in the relevant SOP to ensure they are documented and available at point of use. The quality control measures applied can vary depending of chosen methods and sequencing instrument but should include methods to identify sample preparation failures and failed sequencing runs.

Both accreditation programmes such as CLIA and standards certifications such as ISO 15189 contain specific requirements around the generation, approval, issue and re-issue of reports. The most essential requirements related to Next Generation Sequencing have been listed below.

Overview of Quality Controls



The laboratory director is responsible for designing the advantages and limitations of their test offerings so that healthcare providers can make an informed decision.



Turnaround times for reports should be prepared to ensure there are clear requirements for NGS test prioritisation, these turnaround times should be clinically appropriate.

 All detected somatic variants, identifying each variant’s significance.






Incidental findings including clinical relevance.



Identify limitations of the assay, including for which target areas the assay lacked sufficient coverage to confidently determine mutational status.



Include information related to compare the level of Exome vs. Genome sequencing to an available disease specific panel test.



Two references were used to generate this list: (American College of Medical Genetics and Genomics, 2013)

(New York State Department of Health, 2014).

Table 3: Overview of NGS report requirements.

Conclusions

The overall picture for Next Generation Sequencing is evolving with clearer requirements for everything from quality assurance to reporting requirements. The key with Next Generation Sequencing, as with any clinical technology is to establish validity and clearly document any assumptions or limitations of the platforms and assays involved, especially when reporting to the requesting clinician. While there is not complete understanding the clinical implications of some variants, there are clear prospects emerging for this technology to improve the standard of care and further advance the development and clinical support for companion diagnostics.

Horizon Diagnostics

Horizon Diagnostics is proud to support clinical laboratories with the provision of sustainable reference materials for research use; applications include the validation of equipment and consumables offline from patient testing. This report has been provided for information only.

Bibliograpy



American College of Medical Genetics and Genomics. (2013, September). ACMG clinical laboratory standards for next-generation sequencing.



Horizon Discovery. (n.d.). ISO 15189: A Standard of Yin and Yang.



New York State Department of Health. (2014, January). "Next Generation" Sequencing (NGS) guidelines for somatic genetic variant detection.




New_York_State_Guidelines

New York State Guidelines

CLINICAL APPLICATION OF NEXT GENERATION SEQUENCING AND IMPLICATION OF

NY STATE GUIDELINES

Related documents

Products

Support

New_York_State_Guidelines

New York State Guidelines

CLINICAL APPLICATION OF NEXT GENERATION SEQUENCING AND IMPLICATION OF

NY STATE GUIDELINES

Related documents

Add this document to collection(s)

Add this document to saved

Suggest us how to improve StudyLib