Four-Year Laboratory Performance of the First College of American Pathologists In Silico Next-Generation Sequencing Bioinformatics Proficiency Testing Surveys.

CONTEXT.­: In 2016, the College of American Pathologists (CAP) launched the first next-generation sequencing (NGS) in silico bioinformatics proficiency testing survey to evaluate the performance of clinical laboratory bioinformatics pipelines for the detection of oncology-associated variants at varying allele fractions. This survey focused on 2 commonly used oncology panels, the Illumina TruSeq Amplicon Cancer Panel and the Thermo Fisher Ion AmpliSeq Cancer Hotspot v2 Panel. OBJECTIVE.­: To review the analytical performance of laboratories participating in the CAP NGS bioinformatics (NGSB) surveys, comprising NGSB1 for Illumina users and NGSB2 for Thermo Fisher Ion Torrent users, between 2016 and 2019. DESIGN.­: Responses from 78 laboratories were analyzed for accuracy and associated performance characteristics. RESULTS.­: The analytical sensitivity was 90.0% (1901 of 2112) for laboratories using the Illumina platform and 94.8% (2153 of 2272) for Thermo Fisher Ion Torrent users. Variant type and variant allele fraction were significantly associated with performance. False-negative results were seen mostly for multi-nucleotide variants and variants engineered at variant allele fractions of less than 25%. Analytical specificity for all participating laboratories was 99.8% (9303 of 9320). There was no statistically significant association between deletion-insertion length and detection rate. CONCLUSIONS.­: These results demonstrated high analytical sensitivity and specificity, supporting the feasibility and utility of using in silico mutagenized NGS data sets as a supplemental challenge to CAP surveys for oncology-associated variants based on physical samples. This program demonstrates the opportunity and challenges that can guide future surveys inclusive of customized in silico programs.

Next-Generation Sequencing Somatic and Germline Assay Troubleshooting Guide Derived From Proficiency Testing Data.

CONTEXT.­: Next-generation sequencing-based assays are increasingly used in clinical molecular laboratories to detect somatic variants in solid tumors and hematologic malignancies and to detect constitutional variants. Proficiency testing data are potential sources of information about challenges in performing these assays. OBJECTIVE.­: To examine the most common sources of unacceptable results from the College of American Pathologists Next-Generation Sequencing Bioinformatics, Hematological Malignancies, Solid Tumor, and Germline surveys and provide recommendations on how to avoid these pitfalls and improve performance. DESIGN.­: The College of American Pathologists next-generation sequencing somatic and germline proficiency testing survey results from 2016 to 2019 were analyzed to identify the most common causes of unacceptable results. RESULTS.­: On somatic and germline proficiency testing surveys, 95.9% (18 815/19 623) and 97.8% (33 890/34 641) of all variants were correctly identified, respectively. The most common causes of unacceptable results related to sequencing were false-negative errors in genomic regions that were difficult to sequence because of high GC content. False-positive errors occurred in the context of homopolymers and pseudogenes. Recurrent errors in variant annotation were seen for dinucleotide and duplication variants and included unacceptable transcript selection and outdated variant nomenclature. A small percentage of preanalytic or postanalytic errors were attributed to specimen swaps and transcription errors. CONCLUSIONS.­: Laboratories demonstrate overall excellent performance for detecting variants in both somatic and germline proficiency testing surveys. Proficiency testing survey results highlight infrequent, but recurrent, analytic and nonanalytic challenges in performing next- generation sequencing-based assays and point to remedies to help laboratories improve performance.

Genotype and Phenotype Concordance for Pharmacogenetic Tests Through Proficiency Survey Testing.

CONTEXT.­: As pharmacogenetic testing is incorporated into routine care, it is essential for laboratories to provide accurate and consistent results. Certified laboratories must successfully complete proficiency testing. OBJECTIVES.­: To understand and examine trends in participation and performance of laboratories participating in the College of American Pathologists pharmacogenetic proficiency testing surveys. DESIGN.­: Results from College of American Pathologists pharmacogenetic proficiency testing challenges from 2012 through 2017 were reviewed for concordance with expected genotype and phenotype for each sample (intended responses). RESULTS.­: Laboratories correctly reported results for 96.7% to 100% of samples with no variants. Excluding CYP2D6, laboratories correctly detected and reported variant alleles for each gene (93.7%-99.2% correct). CYP2D6 showed lower concordance, with 83.1% of laboratories reporting the intended genotype across all samples; however, in many cases, the laboratories that did not report a variant allele did not test for that allele. Among laboratories reporting the intended genotype, most successfully reported the intended phenotype (85.9%-99.0%). CONCLUSIONS.­: Although laboratories are generally performing well, there is room for additional improvement, particularly for challenging genes, such as CYP2D6. Efforts in the field of pharmacogenomics to recommend alleles that should be included in clinical tests, identify reference materials, and standardize translation from genotype to phenotype may address some of the remaining variability in results.

Marked Variability in Reported Minimal Residual Disease Lower Level of Detection of 4 Hematolymphoid Neoplasms: A Survey of Participants in the College of American Pathologists Flow Cytometry Proficiency Testing Program.

CONTEXT: Flow cytometry is often applied to minimal residual disease (MRD) testing in hematolymphoid neoplasia. Because flow-based MRD tests are developed in the laboratory, testing methodologies and lower levels of detection (LODs) are laboratory dependent. OBJECTIVES: To broadly survey flow cytometry laboratories about MRD testing in laboratories, if performed, including indications and reported LODs. DESIGN: Voluntary supplemental questions were sent to the 549 laboratories participating in the College of American Pathologists (CAP) FL3-A Survey (Flow Cytometry-Immunophenotypic Characterization of Leukemia/Lymphoma) in the spring of 2014. RESULTS: A total of 500 laboratories (91%) responded to the supplemental questions as part of the FL3-A Survey by April 2014; of those 500 laboratories, 167 (33%) currently perform MRD for lymphoblastic leukemia, 118 (24%) for myeloid leukemia, 99 (20%) for chronic lymphocytic leukemia, and 91 (18%) for plasma cell myeloma. Other indications include non-Hodgkin lymphoma, hairy cell leukemia, neuroblastoma, and myelodysplastic syndrome. Most responding laboratories that perform MRD for lymphoblastic leukemia reported an LOD of 0.01%. For myeloid leukemia, chronic lymphocytic leukemia, and plasma cell myeloma, most laboratories indicated an LOD of 0.1%. Less than 3% (15 of 500) of laboratories reported LODs of 0.001% for one or more MRD assays performed. CONCLUSIONS: There is major heterogeneity in the reported LODs of MRD testing performed by laboratories subscribing to the CAP FL3-A Survey. To address that heterogeneity, changes to the Flow Cytometry Checklist for the CAP Laboratory Accreditation Program are suggested that will include new requirements that each laboratory (1) document how an MRD assay's LOD is measured, and (2) include the LOD or lower limit of enumeration for flow-based MRD assays in the final diagnostic report.