Variability of cycle threshold values in an external quality assessment scheme for detection of the SARS-CoV-2 virus genome by RT-PCR.

OBJECTIVES: The qualitative results of SARS-CoV-2 specific real-time reverse transcription (RT) PCR are used for initial diagnosis and follow-up of Covid-19 patients and asymptomatic virus carriers. However, clinical decision-making and health management policies often are based additionally on cycle threshold (Ct) values (i.e., quantitative results) to guide patient care, segregation and discharge management of individuals testing positive. Therefore, an analysis of inter-protocol variability is needed to assess the comparability of the quantitative results. METHODS: Ct values reported in a SARS-CoV-2 virus genome detection external quality assessment challenge were analyzed. Three positive and two negative samples were distributed to participating test laboratories. Qualitative results (positive/negative) and quantitative results (Ct values) were assessed. RESULTS: A total of 66 laboratories participated, contributing results from 101 distinct test systems and reporting Ct values for a total of 92 different protocols. In all three positive samples, the means of the Ct values for the E-, N-, S-, RdRp-, and ORF1ab-genes varied by less than two cycles. However, 7.7% of reported results deviated by more than ±4.0 (maximum 18.0) cycles from the respective individual means. These larger deviations appear to be systematic errors. CONCLUSIONS: In an attempt to use PCR diagnostics beyond the identification of infected individuals, laboratories are frequently requested to report Ct values along with a qualitative result. This study highlights the limitations of interpreting Ct values from the various SARS-CoV genome detection protocols and suggests that standardization is necessary in the reporting of Ct values with respect to the target gene.

The versatility of external quality assessment for the surveillance of laboratory and in vitro diagnostic performance: SARS-CoV-2 viral genome detection in Austria.

OBJECTIVES: External quality assessment (EQA) schemes provide information on individual and general analytical performance of participating laboratories and test systems. The aim of this study was to investigate the use and performance of SARS-CoV-2 virus genome detection systems in Austrian laboratories and their preparedness to face challenges associated with the pandemic. METHODS: Seven samples were selected to evaluate performance and estimate variability of reported results. Notably, a dilution series was included in the panel as a measure of reproducibility and sensitivity. Several performance criteria were evaluated for individual participants as well as in the cohort of all participants. RESULTS: A total of 109 laboratories participated and used 134 platforms, including 67 different combinations of extraction and PCR platforms and corresponding reagents. There were no false positives and 10 (1.2%) false negative results, including nine in the weakly positive sample (Ct ∼35.9, ∼640 copies/mL). Twenty (22%) laboratories reported results of mutation detection. Twenty-five (19%) test systems included amplification of human RNA as evidence of proper sampling. The overall linearity of Ct values from individual test systems for the dilution series was good, but inter-assay variability was high. Both operator-related and systematic failures appear to have caused incorrect results. CONCLUSIONS: Beyond providing certification for participating laboratories, EQA provides the opportunity for participants to evaluate their performance against others so that they may improve operating procedures and test systems. Well-selected EQA samples offer additional inferences to be made about assay sensitivity and reproducibility, which have practical applications.

Converting to an international unit system improves harmonization of results for SARS-CoV-2 quantification: Results from multiple external quality assessments.

BACKGROUND: The detection of SARS-CoV-2 vRNA in clinical samples has relied almost exclusively on RT-qPCR as the gold standard test. Published results from various external quality assessments ("ring trials") worldwide have shown that there is still a large variability in results reported for the same samples. As reference standards of SARS-CoV-2 RNA are available, we tested whether using standard curves to convert Ct values into copies/mL (cp/mL) improved harmonization. METHODS: Nine laboratories using 23 test systems (15 of which were unique) prepared standard dilution curves to convert Ct values of 13 SARS-CoV-2 positive samples to cp/mL (hereafter IU/mL). The samples were provided in three rounds of a virus genome detection external quality assessment (EQA) scheme. We tested the precision and accuracy of results reported in IU/mL, and attempted to identify the sources of variability. RESULTS: Reporting results as IU/mL improved the precision of the estimated concentrations of all samples compared to reporting Ct values, although some inaccuracy remained. Variance analysis showed that nearly all variability in data was explained by individual test systems within individual laboratories. When controlling for this effect, there was no significant difference between all other factors tested (test systems, EQA rounds, sample material). CONCLUSIONS: Converting results to copies/mL improved precision across laboratory test systems. However, it seems the results are still very specific to test systems within laboratories. Further efforts could be made to improve accuracy and achieve full harmonization across diagnostic laboratories.