Accuracy of quantitative viral secondary standards: a re-examination.

Interlaboratory agreement of viral load assays depends on the accuracy and uniformity of quantitative calibrators. Previous work demonstrated poor agreement of secondary cytomegalovirus (CMV) standards with nominal values. This study re-evaluated this issue among commercially produced secondary standards for both BK virus (BKV) and CMV, using digital polymerase chain reaction (dPCR) to compare the materials from three different manufacturers. Overall, standards showed an improved agreement compared to prior work, against nominal values in both log10 copies/mL and log10 international unit (IU)/mL, with bias from manufacturer-assigned nominal values of 0.0-0.9 log10 units (either copies or IU)/mL. Standards normalized to IU and those values assigned by dPCR rather than by real-time PCR (qPCR) showed better agreement with nominal values. The latter reinforces prior conclusions regarding the utility of using such methods for quantitative value assignment in reference materials. Quantitative standards have improved over the last several years, and the remaining bias from nominal values might be further reduced by universal implementation of dPCR methods for value assignment, normalized to IU. IMPORTANCE: Interlaboratory agreement of viral load assays depends on accuracy and uniformity of quantitative calibrators. Previous work, published in JCM several years ago, demonstrated poor agreement of secondary cytomegalovirus (CMV) standards with nominal values. This study re-evaluated this issue among commercially produced secondary standards for both BK virus (BKV) and CMV, using digital polymerase chain reaction (dPCR) to compare the materials from three different manufacturers. Overall, standards showed an improved agreement compared to prior work, against nominal values, indicating a substantial improvement in the production of accurate secondary viral standards, while supporting the need for further work in this area and for the broad adaption of international unit (IU) as a reporting standard for quantitative viral load results.

Matrix Matters: Assessment of Commutability among BK Virus Assays and Standards.

Quantitative testing of BK virus (BKPyV) nucleic acid has become the standard of care in transplant patients. While the relationship between interassay harmonization and commutability has been well characterized for other transplant-related viruses, it has been less well studied for BKPyV, particularly regarding differences in commutability between matrices. Here, interassay agreement was evaluated among six real-time nucleic acid amplification tests (NAATs) and one digital PCR (dPCR) BKPyV assay. Differences in the commutability of three quantitative standards was examined across all assays using a variety of statistical approaches. Panels, including 40 samples each of plasma and urine samples previously positive for BKPyV, together with one previously negative plasma sample and four previously negative urine samples, were tested using all assays, with each real-time NAAT utilizing its usual quantitative calibrators. Serial dilutions of WHO, National Institute for Standards and Technology (NIST), and commercially produced (Exact/Bio-Rad) reference materials were also run by each assay as unknowns. The agreement of the clinical sample values was assessed as a group and in a pairwise manner. The commutability was estimated using both relativistic and quantitative means. The quantitative agreement across assays in the urine samples was within a single log10 unit across all assays, while the results from the plasma samples varied by 2 to 3 log10 IU/mL. The commutability showed a similar disparity between the matrices. Recalibration using international standards diminished the resulting discrepancies in some but not all cases. Differences in the sample matrix can affect the commutability and interassay agreement of quantitative BKPyV assays. Differences in commutability between matrices may largely be due to factors other than those such as amplicon size, previously described as important in the case of cytomegalovirus. Continued efforts to standardize viral load measurements must address multiple sources of variability and account for differences in assay systems, quantitative standards, and sample matrices.

Persistent Challenges of Interassay Variability in Transplant Viral Load Testing.

While quantification of viruses that cause important infections in transplant recipients has been the standard of care for years, important challenges related to standardization remain. The issues are wide ranging, and until they are adequately addressed, the full impact of viral load testing regarding clinical management decisions will not be realized. This review focuses on a broad array of problems, including the lack of available FDA-approved/cleared tests, limited uptake of international standards, accurate quantification of secondary standards, specific assay characteristics, and commutability. Though some of these topics are nuanced, taken together they greatly influence the clinical utility of testing. For example, it has not been possible to define thresholds that predict the risk of developing disease and determine significant changes in serial viral load values for a given patient. Moreover, the utility of international guidelines may be limited due to the lack of a standardized assay. By summarizing the issues, the hope is that commercial companies, regulatory agencies, and professional societies can come together to advance the field and solve these problems.

Impact of Fragmentation on Commutability of Epstein-Barr Virus and Cytomegalovirus Quantitative Standards.

Despite the adaptation of international standards, quantitative viral load testing of transplant-associated viruses continues to be limited by interlaboratory disagreement. Studies have suggested that this disagreement and the poor commutability of standards may, in some cases, be linked to amplicon size and the fragmentation of circulating viral DNA. We evaluated target fragmentation as a cause of noncommutability and pretest fragmentation of quantitative standards as a potential means of increasing commutability and interassay agreement. Forty-two cytomegalovirus (CMV)-positive and 41 Epstein-Barr virus (EBV)-positive plasma samples, together with two different quantitative standards for each virus, were tested as unknowns using 10 different quantitative PCR assays at 5 different laboratories. Standards were tested both intact and after intentional fragmentation by ultrasonication. Quantitative agreement between methods was assessed, together with commutability, using multiple statistical approaches. Most assays yielded results within 0.5 log10 IU/ml of the mean for CMV, while for EBV a greater variability of up to 1.5 log10 IU/ml of the mean was shown. Commutability showed marked improvement following fragmentation of both CMV standards but not after fragmentation of the EBV standards. These findings confirm the impact of amplicon size and target fragmentation on commutability for CMV and suggest that for some (but not all) viruses, interlaboratory harmonization can be improved through the use of fragmented quantitative standards.

A Comprehensive Statistical Framework for Determination of Commutability, Accuracy, and Agreement in Clinical DNAemia Assays.

Despite advances in the standardization of quantitative DNAemia tests and efforts to better understand and characterize the performance of reference materials in different assays, it remains unclear how the commutability performance of reference materials is related to intra- and interassay agreement. Building upon previous work, we describe a comprehensive framework to determine the relationship of commutability with assay accuracy and agreement. The use of this framework is illustrated using previously generated data regarding the performance of four quantitative Epstein-Bar virus (EBV) PCR assays with the WHO and ABI standards as examples. The use of these statistical tools can link the performance characteristics of one or more assays with predetermined clinical decision limits and may help improve the development, validation, and clinical utility of such DNAemia tests.

Progress in Quantitative Viral Load Testing: Variability and Impact of the WHO Quantitative International Standards.

It has been hoped that the recent availability of WHO quantitative standards would improve interlaboratory agreement for viral load testing; however, insufficient data are available to evaluate whether this has been the case. Results from 554 laboratories participating in proficiency testing surveys for quantitative PCR assays of cytomegalovirus (CMV), Epstein-Barr virus (EBV), BK virus (BKV), adenovirus (ADV), and human herpesvirus 6 (HHV6) were evaluated to determine overall result variability and then were stratified by assay manufacturer. The impact of calibration to international units/ml (CMV and EBV) on variability was also determined. Viral loads showed a high degree of interlaboratory variability for all tested viruses, with interquartile ranges as high as 1.46 log10 copies/ml and the overall range for a given sample up to 5.66 log10 copies/ml. Some improvement in result variability was seen when international units were adopted. This was particularly the case for EBV viral load results. Variability in viral load results remains a challenge across all viruses tested here; introduction of international quantitative standards may help reduce variability and does so more or less markedly for certain viruses.

Comparative Performance of Reagents and Platforms for Quantitation of Cytomegalovirus DNA by Digital PCR.

A potential benefit of digital PCR is a reduction in result variability across assays and platforms. Three sets of PCR reagents were tested on two digital PCR systems (Bio-Rad and RainDance), using three different sets of PCR reagents for quantitation of cytomegalovirus (CMV). Both commercial quantitative viral standards and 16 patient samples (n = 16) were tested. Quantitative accuracy (compared to nominal values) and variability were determined based on viral standard testing results. Quantitative correlation and variability were assessed with pairwise comparisons across all reagent-platform combinations for clinical plasma sample results. The three reagent sets, when used to assay quantitative standards on the Bio-Rad system, all showed a high degree of accuracy, low variability, and close agreement with one another. When used on the RainDance system, one of the three reagent sets appeared to have a much better correlation to nominal values than did the other two. Quantitative results for patient samples showed good correlation in most pairwise comparisons, with some showing poorer correlations when testing samples with low viral loads. Digital PCR is a robust method for measuring CMV viral load. Some degree of result variation may be seen, depending on platform and reagents used; this variation appears to be greater in samples with low viral load values.

Quantitative Assessment of Commutability for Clinical Viral Load Testing Using a Digital PCR-Based Reference Standard.

Given recent advances in the development of quantitative standards, particularly WHO international standards, efforts to better understand the commutability of reference materials have been made. Existing approaches in evaluating commutability include prediction intervals and correspondence analysis; however, the results obtained from existing approaches may be ambiguous. We have developed a "deviation-from-ideal" (DFI) approach to evaluate commutability of standards and applied it to the assessment of Epstein-Bar virus (EBV) load testing in four quantitative PCR assays, treating digital PCR as a reference assay. We then discuss advantages and limitations of the DFI approach as well as experimental design to best evaluate the commutability of an assay in practice.

Commutability of the First World Health Organization International Standard for Human Cytomegalovirus.

Quantitative detection of cytomegalovirus (CMV) DNA has become a standard part of care for many groups of immunocompromised patients; recent development of the first WHO international standard for human CMV DNA has raised hopes of reducing interlaboratory variability of results. Commutability of reference material has been shown to be necessary if such material is to reduce variability among laboratories. Here we evaluated the commutability of the WHO standard using 10 different real-time quantitative CMV PCR assays run by eight different laboratories. Test panels, including aliquots of 50 patient samples (40 positive samples and 10 negative samples) and lyophilized CMV standard, were run, with each testing center using its own quantitative calibrators, reagents, and nucleic acid extraction methods. Commutability was assessed both on a pairwise basis and over the entire group of assays, using linear regression and correspondence analyses. Commutability of the WHO material differed among the tests that were evaluated, and these differences appeared to vary depending on the method of statistical analysis used and the cohort of assays included in the analysis. Depending on the methodology used, the WHO material showed poor or absent commutability with up to 50% of assays. Determination of commutability may require a multifaceted approach; the lack of commutability seen when using the WHO standard with several of the assays here suggests that further work is needed to bring us toward true consensus.

Comparative evaluation of three commercial quantitative cytomegalovirus standards by use of digital and real-time PCR.

The recent development of the 1st WHO International Standard for human cytomegalovirus (CMV) and the introduction of commercially produced secondary standards have raised hopes of improved agreement among laboratories performing quantitative PCR for CMV. However, data to evaluate the trueness and uniformity of secondary standards and the consistency of results achieved when these materials are run on various assays are lacking. Three concentrations of each of the three commercially prepared secondary CMV standards were tested in quadruplicate by three real-time and two digital PCR methods. The mean results were compared in a pairwise fashion with nominal values provided by each manufacturer. The agreement of results among all methods for each sample and for like concentrations of each standard was also assessed. The relationship between the nominal values of standards and the measured values varied, depending upon the assay used and the manufacturer of the standards, with the degree of bias ranging from +0.6 to -1.0 log10 IU/ml. The mean digital PCR result differed significantly among the secondary standards, as did the results of the real-time PCRs, particularly when plotted against nominal log10 IU values. Commercially available quantitative secondary CMV standards produce variable results when tested by different real-time and digital PCR assays, with various magnitudes of bias compared to nominal values. These findings suggest that the use of such materials may not achieve the intended uniformity among laboratories measuring CMV viral load, as envisioned by adaptation of the WHO standard.

Multicenter evaluation of a commercial cytomegalovirus quantitative standard: effects of commutability on interlaboratory concordance.

Commutability of quantitative reference materials has proven important for reliable and accurate results in clinical chemistry. As international reference standards and commercially produced calibration material have become available to address the variability of viral load assays, the degree to which such materials are commutable and the effect of commutability on assay concordance have been questioned. To investigate this, 60 archived clinical plasma samples, which previously tested positive for cytomegalovirus (CMV), were retested by five different laboratories, each using a different quantitative CMV PCR assay. Results from each laboratory were calibrated both with lab-specific quantitative CMV standards ("lab standards") and with common, commercially available standards ("CMV panel"). Pairwise analyses among laboratories were performed using mean results from each clinical sample, calibrated first with lab standards and then with the CMV panel. Commutability of the CMV panel was determined based on difference plots for each laboratory pair showing plotted values of standards that were within the 95% prediction intervals for the clinical specimens. Commutability was demonstrated for 6 of 10 laboratory pairs using the CMV panel. In half of these pairs, use of the CMV panel improved quantitative agreement compared to use of lab standards. Two of four laboratory pairs for which the CMV panel was noncommutable showed reduced quantitative agreement when that panel was used as a common calibrator. Commutability of calibration material varies across different quantitative PCR methods. Use of a common, commutable quantitative standard can improve agreement across different assays; use of a noncommutable calibrator can reduce agreement among laboratories.

Comparison of droplet digital PCR to real-time PCR for quantitative detection of cytomegalovirus.

Quantitative real-time PCR (QRT-PCR) has been widely implemented for clinical viral load testing, but a lack of standardization and relatively poor precision have hindered its usefulness. Digital PCR offers highly precise, direct quantification without requiring a calibration curve. Performance characteristics of real-time PCR were compared to those of droplet digital PCR (ddPCR) for cytomegalovirus (CMV) load testing. Tenfold serial dilutions of the World Health Organization (WHO) and the National Institute of Standards and Technology (NIST) CMV quantitative standards were tested, together with the AcroMetrix CMV tc panel (Life Technologies, Carlsbad, CA) and 50 human plasma specimens. Each method was evaluated using all three standards for quantitative linearity, lower limit of detection (LOD), and accuracy. Quantitative correlation, mean viral load, and variability were compared. Real-time PCR showed somewhat higher sensitivity than ddPCR (LODs, 3 log(10) versus 4 log(10) copies/ml and IU/ml for NIST and WHO standards, respectively). Both methods showed a high degree of linearity and quantitative correlation for standards (R(2) ≥ 0.98 in each of 6 regression models) and clinical samples (R(2) = 0.93) across their detectable ranges. For higher concentrations, ddPCR showed less variability than QRT-PCR for the WHO standards and AcroMetrix standards (P < 0.05). QRT-PCR showed less variability and greater sensitivity than did ddPCR in clinical samples. Both digital and real-time PCR provide accurate CMV load data over a wide linear dynamic range. Digital PCR may provide an opportunity to reduce the quantitative variability currently seen using real-time PCR, but methods need to be further optimized to match the sensitivity of real-time PCR.

Effect of HSV-2 suppressive therapy on genital tract HIV-1 RNA shedding among women on HAART: a pilot randomized controlled trial.

BACKGROUND: The role of suppressive HSV therapy in women coinfected with HSV-2 and HIV-1 taking highly active antiretroviral therapy (HAART) is unclear. METHODS: 60 women with HIV-1/HSV-2 coinfection on HAART with plasma HIV-1 viral load (PVL) ≤75 copies/mL were randomized to receive acyclovir (N = 30) or no acyclovir (N = 30). PVL, genital tract (GT) HIV-1, and GT HSV were measured every 4 weeks for one year. RESULTS: Detection of GT HIV-1 was not significantly different in the two arms (OR 1.23, P = 0.67), although this pilot study was underpowered to detect this difference. When PVL was undetectable, the odds of detecting GT HIV were 0.4 times smaller in the acyclovir arm than in the control arm, though this was not statistically significant (P = 0.07). The odds of detecting GT HSV DNA in women receiving acyclovir were significantly lower than in women in the control group, OR 0.38, P < 0.05. CONCLUSIONS: Chronic suppressive therapy with acyclovir in HIV-1/HSV-2-positive women on HAART significantly reduces asymptomatic GT HSV shedding, though not GT HIV shedding or PVL. PVL was strongly associated with GT HIV shedding, reinforcing the importance of HAART in decreasing HIV sexual transmission.

Factors contributing to variability of quantitative viral PCR results in proficiency testing samples: a multivariate analysis.

While viral load testing has gained widespread acceptance, a primary limitation remains the variability of results, particularly between different laboratories. While some work has demonstrated the importance of standardized quantitative control material in reducing this variability, little has been done to explore other important factors in the molecular amplification process. Results of 185 laboratories enrolled in the College of American Pathologists (CAP) 2009 viral load proficiency testing (PT) survey (VLS) were examined. This included 165 labs (89.2%) testing for cytomegalovirus (CMV), 99 (53.5%) for Epstein-Barr virus (EBV), and 64 (34.6%) for BK virus (BKV). At the time of PT, laboratories were asked a series of questions to characterize their testing methods. The responses to these questions were correlated to mean viral load (MVL) and result variability (RV). Contribution of individual factors to RV was estimated through analysis of variance (ANOVA) modeling and the use of backward selection of factors to fit those models. Selection of the quantitative calibrator, commercially prepared primers and probes, and amplification target gene were found most prominently associated with changes in MVL or RV for one or more of the viruses studied. Commercially prepared primers and probes and amplification target gene made the largest contribution to overall variability. Factors contributing to MVL and RV differed among viruses, as did relative contribution of each factor to overall variability. The marked variability seen in clinical quantitative viral load results is associated with multiple aspects of molecular testing design and performance. The reduction of such variability will require a multifaceted approach to improve the accuracy, reliability, and clinical utility of these important tests.

Interlaboratory comparison of cytomegalovirus viral load assays.

To assess interlaboratory variability in qualitative and quantitative cytomegalovirus (CMV) viral load (VL) testing, we distributed a panel of samples to 33 laboratories in the USA, Canada and Europe who performed testing using commercial reagents (n = 17) or laboratory-developed assays (n = 18). The panel included two negatives, seven samples constructed from purified CMV nucleocapsids in plasma (2.0-6.0 log(10) copies/mL) and three clinical plasma samples. Interlaboratory variation was observed in both actual (range, 2.0-4.0 log(10) copies/mL) and self-reported lower limits of detection (range, 1.0-4.0 log(10) copies/mL). Variation observed in reported results for individual samples ranged from 2.0 log(10) (minimum) to 4.3 log(10) (maximum)(.) Variation was greatest at low VLs. Assuming +/- 0.5 log(10) relative to the expected result represents an acceptable result, 57.6% of results fell within this range. Use of commercially available reagents and procedures was associated with less variability compared with laboratory-developed assays. Interlaboratory variability on replicate samples was significantly greater than intralaboratory variability (p < 0.0001). The significant interlaboratory variability in CMV VL observed may be impacting patient care and limiting interinstitutional comparisons. The creation of an international reference standard for CMV VL assay calibration would be an important step in quality improvement of this laboratory tool.

Interlaboratory comparison of epstein-barr virus viral load assays.

To assess interlaboratory variability in qualitative and quantitative Epstein-Barr virus (EBV) viral load (VL) testing, we distributed a panel of samples to 28 laboratories in the USA, Canada and Europe who performed testing using commercially available reagents (n = 12) or laboratory-developed assays (n = 18). The panel included two negatives, seven constructed samples using Namalwa and Molt-3 cell lines diluted in plasma (1.30-5.30 log(10) copies/mL) and three clinical plasma samples. Significant interlaboratory variation was observed for both actual (range 1.30-4.30 log(10) copies/mL) and self-reported (range, 1.70-3.30 log(10) copies/mL) lower limits of detection. The variation observed in reported results on individual samples ranged from 2.28 log(10) (minimum) to 4.14 log(10) (maximum). Variation was independent of dynamic range and use of commercial versus laboratory-developed assays. Overall, only 47.0% of all results fell within acceptable standards of variation: defined as the expected result +/- 0.50 log(10). Interlaboratory variability on replicate samples was significantly greater than intralaboratory variability (p < 0.0001). Kinetics of change in VL appears more relevant than absolute values and clinicians should understand the uncertainty associated with absolute VL values at their institutions. The creation of an international reference standard for EBV VL assay calibration would be an initial important step in quality improvement of this laboratory tool.

Multilaboratory comparison of hepatitis C virus viral load assays.

We report a multilaboratory evaluation of hepatitis C virus (HCV) viral load assays to determine their linear range, reproducibility, subtype detection, and agreement. A panel of HCV RNA samples ranging in nominal concentration from 1.0 to 7.0 log10 IU/ml was constructed by diluting a clinical specimen (genotype 1b). Replicates of the panel were tested in multiple laboratories using the Abbott TaqMan analyte-specific reagent (Abbott reverse transcription-PCR [RT-PCR]), Roche TaqMan RUO (Roche RT-PCR), Roche Amplicor Monitor HCV 2.0 (Roche Monitor), and Bayer VERSANT HCV RNA 3.0 (Bayer bDNA) assays. Bayer bDNA-negative specimens were tested reflexively using the Bayer VERSANT HCV RNA qualitative assay (Bayer TMA). Abbott RT-PCR and Roche RT-PCR detected all 28 replicates with a concentration of 1.0 log10 IU/ml and were linear to 7.0 log10 IU/ml. Roche Monitor and Bayer bDNA detected 27 out of 28 and 13 out of 28 replicates, respectively, of 3.0 log10 IU/ml. Bayer TMA detected all seven replicates with 1.0 log10 IU/ml. Bayer bDNA was the most reproducible of the four assays. The mean viral load values for panel members in the linear ranges of the assays were within 0.5 log10 for the different tests. Eighty-nine clinical specimens of various genotypes (1 through 4) were tested in the Bayer bDNA, Abbott RT-PCR, and Roche RT-PCR assays. For Abbott RT-PCR, mean viral load values were 0.61 to 0.96 log10 greater than the values for Bayer bDNA assay for samples with genotype 1, 2, or 3 samples and 0.08 log10 greater for genotype 4 specimens. The Roche RT-PCR assay gave mean viral load values that were 0.28 to 0.82 log10 greater than those obtained with the Bayer bDNA assay for genotype 1, 2, and 3 samples. However, for genotype 4 samples the mean viral load value obtained with the Roche RT-PCR assay was, on average, 0.15 log10 lower than that of the Bayer bDNA. Based on these data, we conclude that the sensitivity and linear range of the Abbott and Roche RT-PCR assays enable them to be used for HCV diagnostics and therapeutic monitoring. However, the differences in the viral load values obtained with the different assays underscore the importance of using one assay when monitoring response to therapy.

Real-time PCR improves detection of Trichomonas vaginalis infection compared with culture using self-collected vaginal swabs.

OBJECTIVE: To compare a real-time polymerase chain reaction (PCR) assay with broth culture for the detection of Trichomonas vaginalis using self-collected vaginal swabs. METHODS: Self-collected vaginal swabs were obtained from adolescent and young adult African-American women participating in HIV-1 prevention programs. T. vaginalis culture was performed using the InPouch TV System. Samples for the real-time PCR assay were collected using the BDProbeTec ET Culturette Direct Dry Swab system and tested in a laboratory-developed assay which targeted a repeated sequence of the genome. Discrepant samples that were culture negative and positive in the real-time PCR assay were tested in a confirmatory PCR which targeted a different region of the T. vaginalis genome, the18S ribosomal DNA gene. RESULTS: Of the 524 specimens tested by both culture and real-time PCR, 36 were culture positive and 54 were positive in the real-time PCR assay; 16 of the 18 discrepant specimens were also positive in the confirmatory PCR assay. Using a modified gold standard of positive by culture or positive in both PCR assays, the sensitivity of the real-time PCR assay was 100% and the specificity was 99.6%, whereas culture had a sensitivity of 69.2% and a specificity of 100%. CONCLUSIONS: The real-time PCR assay was sensitive and specific for the detection of T. vaginalis DNA from self-collected vaginal swab specimens. The ability to use the BDProbeTec dry swab system for the real-time PCR testing allowed for the detection of Chlamydia trachomatis, Neisseria gonorrhoeae, and T. vaginalis from a single specimen.