Joint inference about the AUC and Youden index for paired biomarkers.

It is common to compare biomarkers' diagnostic or prognostic performance using some summary ROC measures such as the area under the ROC curve (AUC) or the Youden index. We propose to compare two paired biomarkers using both the AUC and the Youden index since the two indices describe different aspects of the ROC curve. This comparison can be made by estimating the joint confidence region (an elliptical area) of the differences of the paired AUCs and the Youden indices. Furthermore, for deciding if one marker is better than the other in terms of both the AUC and the Youden index (J), we can test H0:AUCa≤AUCb or Ja≤Jb against Ha:AUCa>AUCb and Ja>Jb using the paired differences. The construction of such a joint hypothesis is an example of the multivariate order-restricted hypotheses. For such a hypothesis, we propose and compare three testing procedures: (1) the intersection-union test ( IUT ); (2) the conditional test; and (3) the joint test. The performance of the proposed inference methods was evaluated and compared through simulations. The simulation results demonstrate that the proposed joint confidence region maintains the desired confidence level, and all three tests maintain the type I error under the null. Furthermore, among the three proposed testing methods, the conditional test is the preferred approach with markedly larger power consistently than the other two competing methods.

Joint confidence region estimation on predictive values.

For evaluating diagnostic accuracy of inherently continuous diagnostic tests/biomarkers, sensitivity and specificity are well-known measures both of which depend on a diagnostic cut-off, which is usually estimated. Sensitivity (specificity) is the conditional probability of testing positive (negative) given the true disease status. However, a more relevant question is "what is the probability of having (not having) a disease if a test is positive (negative)?". Such post-test probabilities are denoted as positive predictive value (PPV) and negative predictive value (NPV). The PPV and NPV at the same estimated cut-off are correlated, hence it is desirable to make the joint inference on PPV and NPV to account for such correlation. Existing inference methods for PPV and NPV focus on the individual confidence intervals and they were developed under binomial distribution assuming binary instead of continuous test results. Several approaches are proposed to estimate the joint confidence region as well as the individual confidence intervals of PPV and NPV. Simulation results indicate the proposed approaches perform well with satisfactory coverage probabilities for normal and non-normal data and, additionally, outperform existing methods with improved coverage as well as narrower confidence intervals for PPV and NPV. The Alzheimer's Disease Neuroimaging Initiative (ADNI) data set is used to illustrate the proposed approaches and compare them with the existing methods.

Joint hypothesis testing of the area under the receiver operating characteristic curve and the Youden index.

In the receiver operating characteristic (ROC) analysis, the area under the ROC curve (AUC) serves as an overall measure of diagnostic accuracy. Another popular ROC index is the Youden index (J), which corresponds to the maximum sum of sensitivity and specificity minus one. Since the AUC and J describe different aspects of diagnostic performance, we propose to test if a biomarker beats the pre-specified targeting values of AUC0 and J0 simultaneously with H0 : AUC ≤ AUC0 or J ≤ J0 against Ha : AUC > AUC0 and J > J0 . This is a multivariate order restrictive hypothesis with a non-convex space in Ha , and traditional likelihood ratio-based tests cannot apply. The intersection-union test (IUT) and the joint test are proposed for such test. While the IUT test independently tests for the AUC and the Youden index, the joint test is constructed based on the joint confidence region. Findings from the simulation suggest both tests yield similar power estimates. We also illustrated the tests using a real data example and the results of both tests are consistent. In conclusion, testing jointly on AUC and J gives more reliable results than using a single index, and the IUT is easy to apply and have similar power as the joint test.

Methods and reagent-lot comparisons by regression analysis: Sample size considerations.

BACKGROUND: Parametric regression analysis is widely used in methods comparisons and more recently in checking the concordance of test results following receipt of new reagent lots. The greater frequency of reagent-lot evaluations increases pressure to detect bias with smallest possible sample sizes (i.e. smallest consumption of time and resources). This study revisits bias detection using the joint slope, intercept confidence region as an alternative to slope and intercept confidence intervals. METHODS: Four cases were considered representing constant errors, proportional errors (constant CV) and two more complicated error patterns typical of immunoassays. Maximum:minimum range ratios varied from 2:1 to 2000:1. After setting a maximum tolerable difference a series of slope, intercept combinations, each of which predicted the critical difference, were systematically evaluated in simulations which determined the minimum sample size required to detect the difference, firstly using slope, intercept confidence intervals and secondly using the joint slope, intercept confidence region. RESULTS: At small to moderate range ratios, bias detection by joint confidence region required greatly reduced sample sizes to the extent that it should encourage reagent-lot evaluations or, alternatively, transform those already routinely performed into considerably less costly exercises. CONCLUSIONS: While some software is available to calculate joint confidence regions in real-life analyses, shifting this testing method into the mainstream will require a greater number of software developers incorporating the necessary code into their regression programs. The computer program used to conduct this study is freely available and can be used to model any laboratory test.

Comparison of Spectroscopic Techniques for Determining the Peroxide Value of 19 Classes of Naturally Aged, Plant-Based Edible Oils.

The peroxide value of edible oils is a measure of the degree of oxidation, which directly relates to the freshness of the oil sample. Several studies previously reported in the literature have paired various spectroscopic techniques with multivariate analyses to rapidly determine peroxide values using field portable and process instrumentation; those efforts presented "best-case scenarios" with oils from narrowly defined training and test sets. The purpose of this paper is to evaluate the use of near- and mid-infrared absorption and Raman scattering spectroscopies on oil samples from different oil classes, including seasonal and vendor variations, to determine which measurement technique or combination thereof is best for predicting peroxide values. Following peroxide value assays of each oil class using an established titration-based method, global and global-subset calibration models were constructed from spectroscopic data collected on the 19 oil classes used in this study. Spectra from each optical technique were used to create partial least squares regression calibration models to predict the peroxide value of unknown oil samples. A global peroxide value model based on near-infrared (8 mm optical path length) oil measurements produced the lowest RMSEP (4.9), followed by 24 mm optical path length near infrared (5.1), Raman (6.9) and 50 µm optical path length mid-infrared (7.3). However, it was determined that the Raman RMSEP resulted from chance correlations. Global peroxide value models based on low-level fusion of the NIR (8 and 24 mm optical path length) data and all infrared data produced the same RMSEP of 5.1. Global subset models, based on any of the spectroscopies and olive oil training sets from any class (pure, extra light, extra virgin), all failed to extrapolate to the non-olive oils. However, the near-infrared global subset model built on extra virgin olive oil could extrapolate to test samples from other olive oil classes. This work demonstrates the difficulty of developing a truly global method for determining peroxide value of oils.

EXPRESS: Comparison of Spectroscopic Techniques for Determining the Peroxide Value of 19 Classes of Naturally Aged, Plant-Based Edible Oils.

The peroxide value (PV) of edible oils is a measure of the degree of oxidation, which directly relates to the freshness of the oil sample. Several studies previously reported in the literature have paired various spectroscopic techniques with multivariate analyses to rapidly determine PVs using field portable and process instrumentation; those efforts presented âbest-caseâ scenarios with oils from narrowly defined training and test sets. The purpose of this paper is to evaluate the use of near- and mid-infrared absorption and Raman scattering spectroscopies on oil samples from different oil classes, including seasonal and vendor variations, to determine which measurement technique, or combination thereof, is best for predicting PVs. Following PV assays of each oil class using an established titration-based method, global and global-subset calibration models were constructed from spectroscopic data collected on the 19 oil classes used in this study. Spectra from each optical technique were used to create partial least squares regression (PLSR) calibration models to predict the PV of unknown oil samples. A global PV model based on near-infrared (8 mm optical path length â OPL) oil measurements produced the lowest RMSEP (4.9), followed by 24 mm OPL near infrared (5.1), Raman (6.9) and 50 λm OPL mid-infrared (7.3). However, it was determined that the Raman RMSEP resulted from chance correlations. Global PV models based on low-level fusion of the NIR (8 and 24 mm OPL) data and all infrared data produced the same RMSEP of 5.1. Global subset models, based on any of the spectroscopies and olive oil training sets from any class (pure, extra light, extra virgin), all failed to extrapolate to the non-olive oils. However, the near-infrared global subset model built on extra virgin olive oil could extrapolate to test samples from other olive oil classes.