Benchmarking medical laboratory performance on a global scale.

Background and aims: Laboratory performance as a relative concept needs repetitive benchmarking for continuous improvement of laboratory procedures and medical processes. Benchmarking as such establishes reference levels as a basis for improvements efforts for healthcare institutions along the diagnosis cycle, with the patient at its center. But while this concept seems to be generally acknowledged in laboratory medicine, a lack of practical implementation hinders progress at a global level. The aim of this study was to examine the utility of a specific combination of indicators and survey-based data collection approach, and to establish a global benchmarking dataset of laboratory performance for decision makers in healthcare institutions. Methods: The survey consisted of 44 items relating to laboratory operations in general and three subscales identified in previous studies. A global sample of laboratories was approached by trained professionals. Results were analyzed with standard descriptive statistics and exploratory factor analysis. Dimensional reduction of specific items was performed using confirmatory factor analysis, resulting in individual laboratory scores for the three subscales of "Operational performance," "Integrated clinical care performance," and "Financial sustainability" for the high-level concept of laboratory performance. Results and conclusions: In total, 920 laboratories from 55 countries across the globe participated in the survey, of which 401 were government hospital laboratories, 296 private hospital laboratories, and 223 commercial laboratories. Relevant results include the need for digitalization and automation along the diagnosis cycle. Formal quality management systems (ISO 9001, ISO 15189 etc.) need to be adapted more broadly to increase patient safety. Monitoring of key performance indicators (KPIs) relating to healthcare performance was generally low (in the range of 10-30% of laboratories overall), and as a particularly salient result, only 19% of laboratories monitored KPIs relating to speeding up diagnosis and treatment. Altogether, this benchmark elucidates current practice and has the potential to guide improvement efforts and standardization in quality & safety for patients and employees alike as well as sustainability of healthcare systems around the globe.

External quality assessments for SARS-CoV-2 genome detection in Austria : A comparison of the first postpandemic round with results from the pandemic era.

BACKGROUND: External quality assessment (EQA) schemes provide objective feedback to participating laboratories about the performance of their analytical systems and information about overall regional analytical performance. The EQAs are particularly important during pandemics as they also assess the reliability of individual test results and show opportunities to improve test strategies. With the end of the COVID-19 pandemic, the testing frequency significantly decreased in Austria. Here, we analyzed whether this decrease had an effect on participation and/or performance in SARS-CoV­2 virus detection EQAs, as compared to the pandemic era. MATERIAL AND METHODS: Identical samples were sent to all participating laboratories, and the EQA provider evaluated the agreement of the reported results with defined targets. The EQA was operated under two schemes with identical samples and therefore we analyzed it as a single EQA round. The performance of testing was reported as true positive ratios, comparing the post-pandemic data to previous rounds. Furthermore, subgroups of participants were analyzed stratified by laboratory type (medical or nonmedical) and the test system format (fully automated or requiring manual steps). RESULTS: While the frequency of false negative results per sample did not change during the 3 years of the pandemic (5.7%, 95% confidence interval [CI] 3.1-8.4%), an average per sample false negative ratio of 4.3% was observed in the first post-pandemic EQA (0%, 1.8%, and 11% for the 3 positive samples included in the test panel, n = 109 test results per sample). In this first post-pandemic EQA medical laboratories (average 0.4% false negative across 3 samples, n = 90) and automated test systems (average 1.2% false negative, n = 261) had lower false negative ratios than nonmedical laboratories (22.8%, n = 19) and manual test systems (16.7%, n = 22). These lower average ratios were due to a low concentration sample, where nonmedical laboratories reported 36.8% and manual test systems 54.5% true positive results. CONCLUSION: Overall ratios of true positive results were below the mean of all results during the pandemic but were similar to the first round of the pandemic. A lower post-pandemic true positive ratio was associated with specific laboratory types and assay formats, particularly for samples with low concentration. The EQAs will continue to monitor the laboratory performance to ensure the same quality of epidemiological data after the pandemic, even if vigilance has decreased.

The European Organisation of External Quality Assurance Providers in Laboratory Medicine (EQALM) Statement: guidelines for publishing about interlaboratory comparison studies (PubILC).

Data and results from interlaboratory comparison (ILC) studies, external quality assessment (EQA) and proficiency testing (PT) activities are important and valuable contributions both to the further development of all disciplines of medical laboratory diagnostics, and to the evaluation and comparison of in vitro diagnostic assays. So far, however, there are no recommendations as to which essential items should be addressed in publications on interlaboratory comparisons. The European Organization of External Quality Assurance Providers in Laboratory Medicine (EQALM) recognized the need for such recommendations, and these were developed by a group of experts. The result of this endeavor is the EQALM Statement on items recommended to be addressed in publications on interlaboratory comparison activities (PubILC), in conjunction with a user-friendly checklist. Once adopted by authors and journals, the EQALM Statement will ensure essential information and/or study-related facts are included within publications on EQA/PT activities.

Virus sequencing performance during the SARS-CoV-2 pandemic: a retrospective analysis of data from multiple rounds of external quality assessment in Austria.

Introduction: A notable feature of the 2019 coronavirus disease (COVID-19) pandemic was the widespread use of whole genome sequencing (WGS) to monitor severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infections. Countries around the world relied on sequencing and other forms of variant detection to perform contact tracing and monitor changes in the virus genome, in the hopes that epidemic waves caused by variants would be detected and managed earlier. As sequencing was encouraged and rewarded by the government in Austria, but represented a new technicque for many laboratories, we designed an external quality assessment (EQA) scheme to monitor the accuracy of WGS and assist laboratories in validating their methods. Methods: We implemented SARS-CoV-2 WGS EQAs in Austria and report the results from 7 participants over 5 rounds from February 2021 until June 2023. The participants received sample material, sequenced genomes with routine methods, and provided the sequences as well as information about mutations and lineages. Participants were evaluated on the completeness and accuracy of the submitted sequence and the ability to analyze and interpret sequencing data. Results: The results indicate that performance was excellent with few exceptions, and these exceptions showed improvement over time. We extend our findings to infer that most publicly available sequences are accurate within ≤1 nucleotide, somewhat randomly distributed through the genome. Conclusion: WGS continues to be used for SARS-CoV-2 surveillance, and will likely be instrumental in future outbreak scenarios. We identified hurdles in building next-generation sequencing capacity in diagnostic laboratories. EQAs will help individual laboratories maintain high quality next-generation sequencing output, and strengthen variant monitoring and molecular epidemiology efforts.

Performance of SARS-CoV-2 nucleic acid amplification testing in Austria as measured by external quality assessment schemes during 3 years of the COVID-19 pandemic: an observational retrospective study.

BACKGROUND: The aim of external quality assessment (EQA) schemes is to evaluate the analytical performance of laboratories and test systems in a near-to-real-life setting. This monitoring service provides feedback to participant laboratories and serves as a control measure for the epidemiological assessment of the regional incidence of a pathogen, particularly during epidemics. Using data from EQA schemes implemented as a result of the intensive effort to monitor SARS-CoV-2 infections in Austria, we aimed to identify factors that explained the variation in laboratory performance for SARS-CoV-2 detection over the course of the COVID-19 pandemic. METHODS: For this observational study, we retrospectively analysed 6308 reverse transcriptase quantitative PCR (RT-qPCR) test results reported by 191 laboratories on 71 samples during 14 rounds of three SARS-CoV-2 pathogen detection EQA schemes in Austria between May 18, 2020, and Feb 20, 2023. We calculated the overall rates of false and true-negative, false and true-positive, and inconclusive results. We then assessed laboratory performance by estimating the sensitivity by testing whether significant variation in the odds of obtaining a true-positive result could be explained by virus concentration, laboratory type, or assay format. We also assessed whether laboratory performance changed over time. FINDINGS: 4371 (93·7%) of 4663 qPCR test results were true-positive, 241 (5·2%) were false-negative, and 51 (1·1%) were inconclusive. The mean per-sample sensitivity was 99·7% in samples with high virus concentrations (1383 [99·4%] true-positive, three [0·2%] false-negative, and five [0·4%] inconclusive results for 1391 tests in which the sample cycle threshold was ≤32), whereas detection rates were lower in samples with low virus concentrations (mean per-sample sensitivity 92·5%; 2988 [91·3%] true-positive, 238 [7·3%] false-negative, and 46 [1·4%] inconclusive results for 3272 tests in which the cycle threshold was >32). Of the 1645 results expected to be negative, 1561 (94·9%) were correctly reported as negative, 10 (0·6%) were incorrectly reported as positive, and 74 (4·5%) were reported as inconclusive. Notably, the overall performance of the tests did not change significantly over time. The odds of reporting a correct result were 2·94 (95% CI 1·75-4·96) times higher for a medical laboratory than for a non-medical laboratory, and 4·60 (2·91-7·41) times greater for automated test systems than for manual test systems. Automated test systems within medical laboratories had the highest sensitivity when compared with systems requiring manual intervention in both medical and non-medical laboratories. INTERPRETATION: High rates of false-negativity in all PCR analyses evaluated in comprehensive, multiple, and repeated EQA schemes outline a clear path for improvement in the future. The performance of some laboratories (eg, non-medical laboratories or those using non-automated test systems) should receive additional scrutiny-for example, by requiring additional EQA schemes for certification or accreditation-if the aggregated data from EQA rounds suggest lower sensitivity than that recorded by others. This strategy will provide assurances that epidemiological data as a whole are reliable when testing on such a large scale. Although performance did not improve over time, we cannot exclude extenuating circumstances-such as shortages and weakened supply chains-that could have prevented laboratories from seeking alternative methods to improve performance. FUNDING: None.

Classification of "Near-patient" and "Point-of-Care" SARS-CoV-2 Nucleic Acid Amplification Test Systems and a first approach to evaluate their analytical independence of operator activities.

BACKGROUND: European legislation defines as "near-patient testing" (NPT) what is popularly and in other legislations specified as "point-of-care testing" (POCT). Systems intended for NPT/POCT use must be characterized by independence from operator activities during the analytic procedure. However, tools for evaluating this are lacking. We hypothesized that the variability of measurement results obtained from identical samples with a larger number of identical devices by different operators, expressed as the method-specific reproducibility of measurement results reported in External Quality Assessment (EQA) schemes, is an indicator for this characteristic. MATERIALS AND METHODS: Legal frameworks in the EU, the USA and Australia were evaluated about their requirements for NPT/POCT. EQA reproducibility of seven SARS-CoV-2-NAAT systems, all but one designated as "POCT", was calculated from variabilities in Ct values obtained from the respective device types in three different EQA schemes for virus genome detection. RESULTS: A matrix for characterizing test systems based on their technical complexity and the required operator competence was derived from requirements of the European In Vitro Diagnostic Regulation (IVDR) 2017/746. Good EQA reproducibility of the measurement results of the test systems investigated implies that different users in different locations have no recognizable influence on their measurement results. CONCLUSION: The fundamental suitability of test systems for NPT/POCT use according to IVDR can be easily verified using the evaluation matrix presented. EQA reproducibility is a specific characteristic indicating independence from operator activities of NPT/POCT assays. EQA reproducibility of other systems than those investigated here remains to be determined.

Design of external quality assessment schemes and definition of the roles of their providers in future epidemics.

During an epidemic, individual test results form the basis of epidemiological indicators such as case numbers or incidence. Therefore, the accuracy of measures derived from these indicators depends on the reliability of individual results. In the COVID-19 pandemic, monitoring and evaluating the performance of the unprecedented number of testing facilities in operation, and novel testing systems in use, was urgently needed. External quality assessment (EQA) schemes are unique sources of data reporting on testing performance, and their providers are recognised contacts and support for test facilities (for technical-analytical topics) and health authorities (for planning the monitoring of infection diagnostics). To identify information provided by SARS-CoV-2 genome detection EQA schemes that is relevant for public health microbiology, we reviewed the current literature published in PubMed between January, 2020, and July, 2022. We derived recommendations for EQA providers and their schemes for best practices to monitor pathogen-detection performance in future epidemics. We also showed laboratories, test facilities, and health authorities the information and benefits they can derive from EQA data, and from the non-EQA services of their providers.

Three rounds of a national external quality assessment reveal a link between disharmonic anti-SARS-CoV-2 antibody quantifications and the infection stage.

OBJECTIVES: The WHO's standardized measuring unit, "binding antibody units per milliliter (BAU/mL)," should allow the harmonization of quantitative results by different commercial Anti-SARS-CoV-2 immunoassays. However, multiple studies demonstrate inter-assay discrepancies. The antigenic changes of the Omicron variant affect the performance of Spike-specific immunoassays. This study evaluated the variation of quantitative Anti-SARS-CoV-2-Spike antibody measurements among 46, 50, and 44 laboratories in three rounds of a national external quality assessment (EQA) prior to and after the emergence of the Omicron variant in a diagnostic near-to-real-life setting. METHODS: We analyzed results reported by the EQA participant laboratories from single and sequential samples from SARS-CoV-2 convalescent, acutely infected, and vaccinated individuals, including samples obtained after primary and breakthrough infections with the Omicron variant. RESULTS: The three immunoassays most commonly used by the participants displayed a low intra-assay and inter-laboratory variation with excellent reproducibility using identical samples sent to the participants in duplicates. In contrast, the inter-assay variation was very high with all samples. Notably, the ratios of BAU/mL levels quantified by different immunoassays were not equal among all samples but differed between vaccination, past, and acute infection, including primary infection with the Omicron variant. The antibody kinetics measured in vaccinated individuals strongly depended on the applied immunoassay. CONCLUSIONS: Measured BAU/mL levels are only inter-changeable among different laboratories when the same assay was used for their assessment. Highly variable ratios of BAU/mL quantifications among different immunoassays and infection stages argue against the usage of universal inter-assay conversion factors.

Converting to an international unit system improves harmonization of results for SARS-CoV-2 quantification: Results from multiple external quality assessments.

BACKGROUND: The detection of SARS-CoV-2 vRNA in clinical samples has relied almost exclusively on RT-qPCR as the gold standard test. Published results from various external quality assessments ("ring trials") worldwide have shown that there is still a large variability in results reported for the same samples. As reference standards of SARS-CoV-2 RNA are available, we tested whether using standard curves to convert Ct values into copies/mL (cp/mL) improved harmonization. METHODS: Nine laboratories using 23 test systems (15 of which were unique) prepared standard dilution curves to convert Ct values of 13 SARS-CoV-2 positive samples to cp/mL (hereafter IU/mL). The samples were provided in three rounds of a virus genome detection external quality assessment (EQA) scheme. We tested the precision and accuracy of results reported in IU/mL, and attempted to identify the sources of variability. RESULTS: Reporting results as IU/mL improved the precision of the estimated concentrations of all samples compared to reporting Ct values, although some inaccuracy remained. Variance analysis showed that nearly all variability in data was explained by individual test systems within individual laboratories. When controlling for this effect, there was no significant difference between all other factors tested (test systems, EQA rounds, sample material). CONCLUSIONS: Converting results to copies/mL improved precision across laboratory test systems. However, it seems the results are still very specific to test systems within laboratories. Further efforts could be made to improve accuracy and achieve full harmonization across diagnostic laboratories.

Results of a SARS-CoV-2 virus genome detection external quality assessment round focusing on sensitivity of assays and pooling of samples.

OBJECTIVES: Results of earlier external quality assessment (EQA) rounds suggested remarkable differences in the sensitivity of SARS-CoV PCR assays. Although the test systems are intended to detect SARS-CoV-2 in individual samples, screening is often applied to sample pools to increase efficiency and decrease costs. However, it is unknown to what extent these tests actually meet the manufacturer's specifications for sensitivity and how they perform when testing sample pools. METHODS: The sensitivity of assays in routine use was evaluated with a panel of positive samples in a round of a SARS-CoV-2 virus genome detection EQA scheme. The panel consisted of samples at or near the lower limit of detection ("weakly positive"). Laboratories that routinely test sample pools were asked to also analyze the pooled EQA samples according to their usual pool size and dilution method. RESULTS: All participants could detect a highly positive patient-derived sample (>106 copies/mL). Most (96%) of the test systems could detect at least 1,000 copies/mL, meeting the minimum acceptable benchmark, and many (94%) detected the vRNA in a sample with lower concentration (500 copies/mL). The false negative ratio increased to 16 and 26% for samples with 100 and 50 copies/mL, respectively. CONCLUSIONS: The performance of most assays met or exceeded their specification on sensitivity. If assays are to be used to analyze sample pools, the sensitivity of the assay and the number of pooled samples must be balanced.

Benchmarking medical laboratory performance: survey validation and results for Europe, Middle East, and Africa.

OBJECTIVES: Medical laboratory performance is a relative concept, as are quality and safety in medicine. Therefore, repetitive benchmarking appears to be essential for sustainable improvement in health care. The general idea in this approach is to establish a reference level, upon which improvement may be strived for and quantified. While the laboratory community traditionally is highly aware of the need for laboratory performance and public scrutiny is more intense than ever due to the SARS-CoV-2 pandemic, few initiatives span the globe. The aim of this study was to establish a good practice approach towards benchmarking on a high abstraction level for three key dimensions of medical laboratory performance, generate a tentative snapshot of the current state of the art in the region of Europe, Middle East, and Africa (EMEA), and thus set the stage for global follow-up studies. METHODS: The questionnaire used and previously published in this initiative consisted of 50 items, roughly half relating to laboratory operations in general with the other half addressing more specific topics. An international sample of laboratories from EMEA was approached to elicit high fidelity responses with the help of trained professionals. Individual item results were analyzed using standard descriptive statistics. Dimensional reduction of specific items was performed using exploratory factor analysis and assessed with confirmatory factor analysis, resulting in individual laboratory scores for the three subscales of "Operational performance", "Integrated clinical care performance", and "Financial sustainability". RESULTS: Altogether, 773 laboratories participated in the survey, of which 484 were government hospital laboratories, 129 private hospital laboratories, 146 commercial laboratories, and 14 were other types of laboratories (e.g. research laboratories). Respondents indicated the need for digitalization (e.g. use of IT for order management, auto-validation), automation (e.g. pre-analytics, automated sample transportation), and establishment of formal quality management systems (e.g. ISO 15189, ISO 9001) as well as sustainably embedding them in the fabric of laboratory operations. Considerable room for growth also exists for services provided to physicians, such as "Diagnostic pathways guidance", "Proactive consultation on complex cases", and "Real time decision support" which were provided by less than two thirds of laboratories. Concordantly, the most important kind of turn-around time (TAT) for clinicians, sample-to-result TAT, was monitored by only 40% of respondents. CONCLUSIONS: Altogether, the need for stronger integration of laboratories into the clinical care process became apparent and should be a main trajectory of future laboratory management. Factor analysis confirmed the theoretical constructs of the questionnaire design phase, resulting in a reasonably valid tool for further benchmarking activities on the three aimed-for key dimensions.

External quality assessment providers' services appear to more impact the immunohaematology performance of laboratories than national regulatory and economic conditions.

OBJECTIVES: Medical laboratories may, at their own discretion, exceed but not undercut regulatory quality requirements. Available economic resources, however, may drive or hinder eagerness to exceed minimum requirements. Depending on the respective scopes of regulatory and economic framework conditions, differing levels of quality efforts to safeguard laboratory performance can be anticipated. However, this has not yet been investigated. METHODS: Immunohaematology external quality assessment (EQA) results collected by 26 EQA providers from their participant laboratories in 73 countries from 2004 to 2019 were evaluated. Error rates were aggregated in groups according to the respective national regulatory and economic framework conditions, to whether or not expert advice was provided in case of incorrect results, and the frequency of EQA samples. RESULTS: These representative data indicate no association between national regulatory (mandatory participation in EQA, monitoring of performance of individual laboratories by authorities, financial consequences of incorrect results) and economic (level of national income, share of national health expenditure) conditions to the quality performance of medical laboratories in immunohaematology. However, EQA providers' support for laboratories in the event of incorrect results appear to be associated with lower error rates, but a high EQA sample frequency with higher error rates. CONCLUSIONS: Further research into the impact of introducing or changing services of EQA providers is needed to confirm the results found in this first of its kind study.

Inadequate design of mutation detection panels prevents interpretation of variants of concern: results of an external quality assessment for SARS-CoV-2 variant detection.

OBJECTIVES: Mutation-specific PCR assays have quickly found their way into laboratory diagnostics due to their capacity to be a fast, easy to implement and high-throughput method for the detection of known SARS-CoV-2 variants of concern (VoCs). However, little is known about the performance of such assays in routine laboratory analysis. METHODS: The results reported in a recent round of an external quality assessment (EQA) scheme for SARS-CoV-2 mutation-specific PCR were retrospectively analyzed. For the determination of individual variant-specific sequences as well as for the interpretation results for certain virus variants, correct, incorrect, and unreported results were evaluated, and their possible causes were investigated. RESULTS: A total of 34 laboratories participated in this study. For five samples containing the VoC Alpha + E484K, Beta, Gamma, Delta, or B.1.1.318 (as a variant of interest), 848 results for SARS-2-CoV mutation detection were reported, 824 (97.2%, range per sample 88-100%) of which were correct. Melting curve assays gave 99% correct results, real-time RT-qPCR 94%, microarray-based assays 100%, and MALDI-TOF MS 96%. A total of 122/167 (73%) reported results for SARS-CoV-2 variant determination were correct. Of the 45 inconclusive or incorrect results, 33 (73%) were due to inadequate selection of targets that did not allow identification of contemporary VoC, 11 (24%) were due to incorrect results, and one (3%) was due to correct results of mutation-specific PCR. CONCLUSIONS: Careful and up-to-date selection of the targets used in mutation-specific PCR is essential for successful detection of current SARS-CoV-2 variants.

Benchmarking diagnostic laboratory performance: Survey results for Germany, Austria, and Switzerland.

BACKGROUND AND AIMS: The need for patient safety through consistent diagnostic performance has increasingly been brought into focus during the last two decades. Around the globe operational efficiency of diagnostic laboratories plays a key role in satisfying this need, which has impressively been shown during the recent months of the SARS-CoV­2 pandemic. On a global level, however, there has been a lack to collate and benchmark data for diagnostic laboratories. The goals of this study were to design and pilot a questionnaire addressing key aspects of diagnostic laboratory management. METHODS: The questionnaire was designed using an iterative process and taking into consideration information that could be extracted from the literature, author experience and feedback from informal focus groups of laboratory professionals. The resulting tool consisted of 50 items, either relating to general information or more specifically addressing the topics of "operational performance", "integrated clinical care performance", and "financial sustainability". A limited number of laboratories were surveyed to be able to further improve the newly developed tool and motivate the global laboratory community to participate in further benchmarking activity. RESULTS AND CONCLUSION: Altogether, 65 laboratories participated in the survey, 42 were hospital laboratories and 23 were commercial laboratories. Potential for further improvement and standardization became apparent across the board, e.g. use of IT for order management, auto-validation, or turn-around time (TAT) monitoring. Notably, a gap was identified regarding services provided to physicians, in particular "reflexive test suggestions", "proactive consultation on complex cases", and "diagnostic pathways guidance", which were only provided by about two thirds of laboratories. Concordantly, within-laboratory TAT (Lab TAT) was monitored by about 80% of respondents, while sample-to-result TAT, which is arguably the TAT most relevant to clinicians, was only monitored by 32% of respondents. Altogether, the need for stronger integration of the laboratory into the clinical care process became apparent and should be a main trajectory of future laboratory management.

RT-PCR based SARS-CoV-2 variant screening assays require careful quality control.

BACKGROUND: Distinctive genotypes of SARS-CoV-2 have emerged that are or may be associated with increased transmission, pathogenicity, and/or antibody escape. In many countries, clinical and diagnostic laboratories are under mandate to identify and report these so-called variants of concern (VOC). OBJECTIVES: We used an external quality assessment scheme to determine the scope, accuracy, and reliability of laboratories using various molecular diagnostic assays to identify current VOC (03 March 2021). STUDY DESIGN: Participant laboratories were sent the same five patient-derived samples and were asked to provide their variant detection methods, variant detection results and interpretation of results. RESULTS: Twenty-five laboratories reported a range of RT-qPCR-based assays to identify specific variations in the SARS-CoV-2 spike protein that are characteristic of three VOC lineages. Laboratories that detected VOC-associated nucleotide mutations at four specific sites had the highest ratio of correct classification. Low template copy number and additional variation in target regions resulted in loss of confidence and accuracy in sample classification. CONCLUSIONS: Melting-curve-based assays to identify genomic variants are less time-consuming and require less bioinformatic analysis compared to partial or whole genome sequencing. However, our results suggest that correct classification of a given genotype/lineage (e.g., a VOC) relies on the ability to detect more than one variant site, adequate template in the sample (i.e., relatively high viral load/copy number) and results may be unclear in certain samples with additional genetic variations. These initial results suggest that some diagnostic laboratories may require additional training to interpret and report complex genetic information about a dynamic emerging virus.

The versatility of external quality assessment for the surveillance of laboratory and in vitro diagnostic performance: SARS-CoV-2 viral genome detection in Austria.

OBJECTIVES: External quality assessment (EQA) schemes provide information on individual and general analytical performance of participating laboratories and test systems. The aim of this study was to investigate the use and performance of SARS-CoV-2 virus genome detection systems in Austrian laboratories and their preparedness to face challenges associated with the pandemic. METHODS: Seven samples were selected to evaluate performance and estimate variability of reported results. Notably, a dilution series was included in the panel as a measure of reproducibility and sensitivity. Several performance criteria were evaluated for individual participants as well as in the cohort of all participants. RESULTS: A total of 109 laboratories participated and used 134 platforms, including 67 different combinations of extraction and PCR platforms and corresponding reagents. There were no false positives and 10 (1.2%) false negative results, including nine in the weakly positive sample (Ct ∼35.9, ∼640 copies/mL). Twenty (22%) laboratories reported results of mutation detection. Twenty-five (19%) test systems included amplification of human RNA as evidence of proper sampling. The overall linearity of Ct values from individual test systems for the dilution series was good, but inter-assay variability was high. Both operator-related and systematic failures appear to have caused incorrect results. CONCLUSIONS: Beyond providing certification for participating laboratories, EQA provides the opportunity for participants to evaluate their performance against others so that they may improve operating procedures and test systems. Well-selected EQA samples offer additional inferences to be made about assay sensitivity and reproducibility, which have practical applications.