Bias due to composite reference standards in diagnostic accuracy studies.

Composite reference standards (CRSs) have been advocated in diagnostic accuracy studies in the absence of a perfect reference standard. The rationale is that combining results of multiple imperfect tests leads to a more accurate reference than any one test in isolation. Focusing on a CRS that classifies subjects as disease positive if at least one component test is positive, we derive algebraic expressions for sensitivity and specificity of this CRS, sensitivity and specificity of a new (index) test compared with this CRS, as well as the CRS-based prevalence. We use as a motivating example the problem of evaluating a new test for Chlamydia trachomatis, an asymptomatic disease for which no gold-standard test exists. As the number of component tests increases, sensitivity of this CRS increases at the expense specificity, unless all tests have perfect specificity. Therefore, such a CRS can lead to significantly biased accuracy estimates of the index test. The bias depends on disease prevalence and accuracy of the CRS. Further, conditional dependence between the CRS and index test can lead to over-estimation of index test accuracy estimates. This commonly-used CRS combines results from multiple imperfect tests in a way that ignores information and therefore is not guaranteed to improve over a single imperfect reference unless each component test has perfect specificity, and the CRS is conditionally independent of the index test. When these conditions are not met, as in the case of C. trachomatis testing, more realistic statistical models should be researched instead of relying on such CRSs.

The Association Between Vitamin D Deficiency and Diabetes in Adult African Americans and Whites: An NHANES Study.

OBJECTIVE: The primary objective of this cross-sectional study is to investigate the association between vitamin D deficiency (VDD) and diabetes and see if this association is the same for adult (age ≥ 20) African Americans (AAs) and Whites. The secondary objective is to examine the distribution of the 25-hydroxyvitamin D test among AAs and Whites and to evaluate the appropriateness of using the same cut-off point for both groups to diagnose VDD. METHODS: Our analysis is based on the 2011-2014 National Health and Nutrition Examination Surveys (NHANES). We used two common propensity score adjustment methods to analyze the data-propensity score matching (PSM) and the inverse probability of treatment weighting (IPTW). RESULTS: The prevalence of diabetes for AAs and Whites was 12.27% (95% CI, 10.47-14.07%) and 7.24% (95% CI, 6.35-8.13%), respectively. The prevalence of VDD for AAs and Whites was 65.29% (95% CI, 62.01-68.58%) and 19.49% (95% CI, 16.53-22.45%), respectively. Under PSM, the odds ratios for the diabetes-VDD association for AAs and Whites were 0.94 (95% CI, 0.70-1.27) and 2.16 (95% CI, 1.49-3.13), respectively. Under IPTW, the VDD-diabetes odds ratios for AAs and Whites were 0.83 (95% CI, 0.64-1.10) and 2.35 (95% CI, 1.67-3.30), respectively. Our results further demonstrate that the 25-hydroxyvitamin D measurements are significantly different for AAs and Whites across the general population, as well as the vitamin D-sufficient and vitamin D-deficient populations. CONCLUSION: The prevalence of VDD and diabetes was higher for AAs compared to Whites. However, VDD was associated with increased diabetes risk for Whites but not for AAs. Though more research is needed to explain why this is the case, a reason for this may be that the 25-hydroxyvitamin D test or its associated cut-off point for defining VDD may not accurately reflect the vitamin D status among AAs.

Evaluation of screening tests for detecting Chlamydia trachomatis: bias associated with the patient-infected-status algorithm.

In recent years, the evaluation of nucleic acid amplification tests (NAATs) for detecting Chlamydia trachomatis and Neisseria gonorrhea is based on a methodology called the patient-infected-status algorithm (PISA). In the simplest version of PISA, 4 test-specimen combinations (comparator tests) are used to define the gold standard. If a person shows a positive result by any 2 or more of these 4 comparator tests, the person is classified as infected; otherwise, the person is considered to be uninfected. A new test is then compared with this diagnostic algorithm. PISA-based sensitivity and specificity estimates of nucleic acid amplification tests have been published in the medical and microbiologic literature and have been included in FDA-approved package inserts of NAATs for detecting C. trachomatis. Using simulations, we compare 2 versions of the patient-infected-status algorithm with latent-class models and an imperfect gold standard. We show that the PISA can produce highly biased test-performance parameter estimates. In a series of simulated scenarios, none of the 95% confidence intervals for PISA-based estimates of sensitivity and prevalence contained the true values. In addition, the PISA-based estimates of sensitivity and specificity change markedly as the true prevalence changes. We recommend that PISA should not be used for estimating the sensitivity and specificity of tests.

WITHDRAWN: Problems associated with reproducibility, validity and confirmation of nucleic acid amplification tests for detecting Chlamydia trachomatis.

This article has been withdrawn.

Modeling conditional dependence between diagnostic tests: a multiple latent variable model.

Applications of latent class analysis in diagnostic test studies have assumed that all tests are measuring a common binary latent variable, the true disease status. In this article we describe a new approach that recognizes that tests based on different biological phenomena measure different latent variables, which in turn measure the latent true disease status. This allows for adjustment of conditional dependence between tests within disease categories. The model further allows for the inclusion of measured covariates and unmeasured random effects affecting test performance within latent classes. We describe a Bayesian approach for model estimation and describe a new posterior predictive check for evaluating candidate models. The methods are motivated and illustrated by results from a study of diagnostic tests for Chlamydia trachomatis.

Evaluation of nucleic acid amplification tests in the absence of a perfect gold-standard test: a review of the statistical and epidemiologic issues.

During the past 10 years, medical diagnostic testing for sexually transmitted infections (STIs) has changed markedly as a result of the rapid expansion and marketing of nucleic acid amplification tests (NAATs). Among such new DNA/RNA-amplification techniques are the polymerase chain reaction (PCR), the ligase chain reaction (LCR), and the transcription-mediated amplification (TMA) tests. Regrettably, the test evaluation process undergone by these tests has not always been rigorous or scientifically sound. Here, we review the controversy surrounding the statistical evaluation of these NAATs. We also review some of the traditional and recent statistical methods developed to estimate test sensitivity and specificity parameters in the absence of reliable gold-standard tests. In particular, we review the traditional latent class modeling approach that requires the assumption of independence between diagnostic tests conditional on the true disease status, and the more recent procedures that relax the conditional independence assumption. Finally, we apply some of these statistical modeling techniques to real data to estimate the sensitivity and specificity of a NAAT for Chlamydia trachomatis. On the basis of the latent class modeling approach with a pessimistic prior for culture sensitivity, the NAAT specificity estimate was 97.6% and, on the basis of an optimistic prior, the specificity was 95.3%. Similarly, the sensitivity estimates ranged from 88.1% to 89.6%.