Evaluating joint confidence region of hypervolume under ROC manifold and generalized Youden index.

In biomarker evaluation/diagnostic studies, the hypervolume under the receiver operating characteristic manifold ( HUM K $$ {\mathrm{HUM}}_K $$ ) and the generalized Youden index ( J K $$ {J}_K $$ ) are the most popular measures for assessing classification accuracy under multiple classes. While HUM K $$ {\mathrm{HUM}}_K $$ is frequently used to evaluate the overall accuracy, J K $$ {J}_K $$ provides direct measure of accuracy at the optimal cut-points. Simultaneous evaluation of HUM K $$ {\mathrm{HUM}}_K $$ and J K $$ {J}_K $$ provides a comprehensive picture about the classification accuracy of the biomarker/diagnostic test under consideration. This article studies both parametric and non-parametric approaches for estimating the confidence region of HUM K $$ {\mathrm{HUM}}_K $$ and J K $$ {J}_K $$ for a single biomarker. The performances of the proposed methods are investigated by an extensive simulation study and are applied to a real data set from the Alzheimer's Disease Neuroimaging Initiative.

Joint inference about the AUC and Youden index for paired biomarkers.

It is common to compare biomarkers' diagnostic or prognostic performance using some summary ROC measures such as the area under the ROC curve (AUC) or the Youden index. We propose to compare two paired biomarkers using both the AUC and the Youden index since the two indices describe different aspects of the ROC curve. This comparison can be made by estimating the joint confidence region (an elliptical area) of the differences of the paired AUCs and the Youden indices. Furthermore, for deciding if one marker is better than the other in terms of both the AUC and the Youden index (J), we can test H0:AUCa≤AUCb or Ja≤Jb against Ha:AUCa>AUCb and Ja>Jb using the paired differences. The construction of such a joint hypothesis is an example of the multivariate order-restricted hypotheses. For such a hypothesis, we propose and compare three testing procedures: (1) the intersection-union test ( IUT ); (2) the conditional test; and (3) the joint test. The performance of the proposed inference methods was evaluated and compared through simulations. The simulation results demonstrate that the proposed joint confidence region maintains the desired confidence level, and all three tests maintain the type I error under the null. Furthermore, among the three proposed testing methods, the conditional test is the preferred approach with markedly larger power consistently than the other two competing methods.

Simplified Model-Dependent and Model-Independent Approaches for Dissolution Profile Comparison for Oral Products: Regulatory Perspective for Generic Product Development.

Dissolution profile comparison among different formulations plays a critical role during new drug as well as generic product development. In the generic product development, dissolution profile comparison is a mandate for biowaivers (BCS-based, for lower strengths and IVIVC-based biowaivers) and also from quality control perspective. Even though traditionally similarity factor or f2 is used as a metric for dissolution profile comparison, it comes with multiple limitations and requirements (e.g., number of time points and variability). To overcome this, regulatory agencies suggested model-independent (e.g., MSD) and model-dependent (e.g., zero order, Weibull) dissolution profile comparison methods. Although most of regulatory guidance documents mention about such approaches, their usage in reality is limited probably due to lack of clear, detailed, and step-wise procedure. In this context, the present article describes simplistic yet detailed procedures of dissolution profile comparison with case studies covering generic product development scenario's from a regulatory perspective. Detailed review of regulatory guidances from various agencies was made along with examples of such approaches in regulatory submissions. Data from three formulations-Formulations A, B, and C-were utilized to perform dissolution profile comparison using MSD, zero-order, and Weibull release profile-based comparisons. Dissolution profile comparisons were made using all of these three approaches complying with regulatory requirements. These examples demonstrated value and utility of these approaches and the simplified and detailed procedure explained in this manuscript can be adapted for generic product applications.

Empirical likelihood-based inferences for median medical cost regression models with censored data.

Recent studies show that medical cost data can be heavily censored and highly skewed, which leads to have more complex cost data analysis. In this paper, we propose influence function and empirical likelihood (EL)-based methods to construct confidence regions for regression parameters in median cost regression models with censored data. We further propose confidence intervals for the median cost with given covariates using the proposed EL-based confidence regions. Simulation studies are conducted to compare the proposed EL-based confidence regions with the existing normal approximation-based confidence regions in terms of coverage probabilities. The new EL-based methods are observed to have better finite sample performances than existing methods particularly when the censoring proportion is high. The new methods are also illustrated through a real data example.

Joint confidence region estimation on predictive values.

For evaluating diagnostic accuracy of inherently continuous diagnostic tests/biomarkers, sensitivity and specificity are well-known measures both of which depend on a diagnostic cut-off, which is usually estimated. Sensitivity (specificity) is the conditional probability of testing positive (negative) given the true disease status. However, a more relevant question is "what is the probability of having (not having) a disease if a test is positive (negative)?". Such post-test probabilities are denoted as positive predictive value (PPV) and negative predictive value (NPV). The PPV and NPV at the same estimated cut-off are correlated, hence it is desirable to make the joint inference on PPV and NPV to account for such correlation. Existing inference methods for PPV and NPV focus on the individual confidence intervals and they were developed under binomial distribution assuming binary instead of continuous test results. Several approaches are proposed to estimate the joint confidence region as well as the individual confidence intervals of PPV and NPV. Simulation results indicate the proposed approaches perform well with satisfactory coverage probabilities for normal and non-normal data and, additionally, outperform existing methods with improved coverage as well as narrower confidence intervals for PPV and NPV. The Alzheimer's Disease Neuroimaging Initiative (ADNI) data set is used to illustrate the proposed approaches and compare them with the existing methods.

Joint hypothesis testing of the area under the receiver operating characteristic curve and the Youden index.

In the receiver operating characteristic (ROC) analysis, the area under the ROC curve (AUC) serves as an overall measure of diagnostic accuracy. Another popular ROC index is the Youden index (J), which corresponds to the maximum sum of sensitivity and specificity minus one. Since the AUC and J describe different aspects of diagnostic performance, we propose to test if a biomarker beats the pre-specified targeting values of AUC0 and J0 simultaneously with H0 : AUC ≤ AUC0 or J ≤ J0 against Ha : AUC > AUC0 and J > J0 . This is a multivariate order restrictive hypothesis with a non-convex space in Ha , and traditional likelihood ratio-based tests cannot apply. The intersection-union test (IUT) and the joint test are proposed for such test. While the IUT test independently tests for the AUC and the Youden index, the joint test is constructed based on the joint confidence region. Findings from the simulation suggest both tests yield similar power estimates. We also illustrated the tests using a real data example and the results of both tests are consistent. In conclusion, testing jointly on AUC and J gives more reliable results than using a single index, and the IUT is easy to apply and have similar power as the joint test.

Simple confidence interval and region formulas for comparing diagnostic likelihood ratios under a paired design.

A population-based paired design is often used for comparing the diagnostic likelihood ratios of two binary diagnostic tests. However, a case-control paired design, which involves the application of both diagnostic tests to two independent samples, is a good alternative study design especially when the disease is rare. Existing methods for comparing two diagnostic likelihood ratios have been mainly focused on the population-based paired design with little attention paid to the case-control paired design. In this paper, we derive a confidence interval formula for the relative diagnostic likelihood ratio (the ratio of two diagnostic likelihood ratios), which can be used for the comparison of two positive or negative diagnostic likelihood ratios separately. We also derive a confidence region formula for the two relative positive and negative diagnostic likelihood ratios, which allows simultaneous comparison of two positive and negative diagnostic likelihood ratios. The proposed confidence interval and region formulas are simple to compute and can be used for both population-based paired design and case-control paired designs. Simulation studies are used to assess the finite sample performance of the confidence interval and region formulas. The proposed methods are applied to a real data set on coronary artery disease and two diagnostic tests.

On the use of comparison regions in visualizing stochastic uncertainty in some two-parameter estimation problems.

When considering simultaneous inference for two parameters, it is very common to visualize stochastic uncertainty by plotting two-dimensional confidence regions. This allows us to test post hoc null hypotheses about a single point in a simple manner. However, in some applications the interest is not in rejecting hypotheses on single points, but in demonstrating evidence for the two parameters to be in a convex subset of the parameter space. The specific convex subset to be considered may vary from one post hoc analysis to another. Then it is of interest to have a visualization allowing to perform corresponding analyses. We suggest comparison regions as a simple tool for this task.

Constrained empirical-likelihood confidence regions in nonignorable covariate-missing data problems.

Missing covariates in regression analysis are a pervasive problem in medical, social, and economic researches. We study empirical-likelihood confidence regions for unconstrained and constrained regression parameters in a nonignorable covariate-missing data problem. For an assumed conditional mean regression model, we assume that some covariates are fully observed but other covariates are missing for some subjects. By exploitation of a probability model of missingness and a working conditional score model from a semiparametric perspective, we build a system of unbiased estimating equations, where the number of equations exceeds the number of unknown parameters. Based on the proposed estimating equations, we introduce unconstrained and constrained empirical-likelihood ratio statistics to construct empirical-likelihood confidence regions for the underlying regression parameters without and with constraints. We establish the asymptotic distributions of the proposed empirical-likelihood ratio statistics. Simulation results show that the proposed empirical-likelihood methods have a better finite-sample performance than other competitors in terms of coverage probability and interval length. Finally, we apply the proposed empirical-likelihood methods to the analysis of a data set from the US National Health and Nutrition Examination Survey.

Confidence regions for treatment effects in subgroups in biomarker stratified designs.

Subgroup analysis has important applications in the analysis of controlled clinical trials. Sometimes the result of the overall group fails to demonstrate that the new treatment is better than the control therapy, but for a subgroup of patients, the treatment benefit may exist; or sometimes, the new treatment is better for the overall group but not for a subgroup. Hence we are interested in constructing a simultaneous confidence interval for the difference of the treatment effects in a subgroup and the overall group. Subgroups are usually formed on the basis of a predictive biomarker such as age, sex, or some genetic marker. While, for example, age can be detected precisely, it is often only possible to detect the biomarker status with a certain probability. Because patients detected with a positive or negative biomarker may not be truly biomarker positive or negative, responses in the subgroups depend on the treatment therapy as well as on the sensitivity and specificity of the assay used in detecting the biomarkers. In this work, we show how (approximate) simultaneous confidence intervals and confidence ellipsoid for the treatment effects in subgroups can be found for biomarker stratified clinical trials using a normal framework with normally distributed or binary data. We show that these intervals maintain the nominal confidence level via simulations.

Using pilot data to size a two-arm randomized trial to find a nearly optimal personalized treatment strategy.

A personalized treatment strategy formalizes evidence-based treatment selection by mapping patient information to a recommended treatment. Personalized treatment strategies can produce better patient outcomes while reducing cost and treatment burden. Thus, among clinical and intervention scientists, there is a growing interest in conducting randomized clinical trials when one of the primary aims is estimation of a personalized treatment strategy. However, at present, there are no appropriate sample size formulae to assist in the design of such a trial. Furthermore, because the sampling distribution of the estimated outcome under an estimated optimal treatment strategy can be highly sensitive to small perturbations in the underlying generative model, sample size calculations based on standard (uncorrected) asymptotic approximations or computer simulations may not be reliable. We offer a simple and robust method for powering a single stage, two-armed randomized clinical trial when the primary aim is estimating the optimal single stage personalized treatment strategy. The proposed method is based on inverting a plugin projection confidence interval and is thereby regular and robust to small perturbations of the underlying generative model. The proposed method requires elicitation of two clinically meaningful parameters from clinical scientists and uses data from a small pilot study to estimate nuisance parameters, which are not easily elicited. The method performs well in simulated experiments and is illustrated using data from a pilot study of time to conception and fertility awareness.

Joint confidence region estimation for area under ROC curve and Youden index.

In the field of diagnostic studies, the area under the ROC curve (AUC) serves as an overall measure of a biomarker/diagnostic test's accuracy. Youden index, defined as the overall correct classification rate minus one at the optimal cut-off point, is another popular index. For continuous biomarkers of binary disease status, although researchers mainly evaluate the diagnostic accuracy using AUC, for the purpose of making diagnosis, Youden index provides an important and direct measure of the diagnostic accuracy at the optimal threshold and hence should be taken into consideration in addition to AUC. Furthermore, AUC and Youden index are generally correlated. In this paper, we initiate the idea of evaluating diagnostic accuracy based on AUC and Youden index simultaneously. As the first step toward this direction, this paper only focuses on the confidence region estimation of AUC and Youden index for a single marker. We present both parametric and non-parametric approaches for estimating joint confidence region of AUC and Youden index. We carry out extensive simulation study to evaluate the performance of the proposed methods. In the end, we apply the proposed methods to a real data set.

A random set approach to confidence regions with applications to the effective dose with combinations of agents.

The effective dose (ED) is the pharmaceutical dosage required to produce a therapeutic response in a fixed proportion of the patients. When only one drug is considered, the problem is a univariate one and has been well-studied. However, in the multidimensional setting, that is, in the presence of combinations of agents, estimation of the ED becomes more difficult. This study is focused on the plug-in logistic regression estimator of the multidimensional ED. We discuss consistency of such estimators and focus on the problem of simultaneous confidence regions. We develop a bootstrap algorithm to estimate confidence regions for the multidimensional ED. Through simulation, we show that the proposed method gives 95% confidence regions, which have better empirical coverage than the previous method for moderate to large sample sizes. The novel approach is illustrated on a cytotoxicity study on the effect of two toxins in the leukemia cell line HL-60 and a decompression sickness study of the effects of the duration and depth of the dive.

Constructing networks for comparison of collagen types.

Collagens are structural proteins that are predominantly found in the extracellular matrix of multicellular animals, where they are mainly responsible for the stability and structural integrity of various tissues. All collagens contain polypeptide strands (α-chains). There are several types of collagens, some of which differ significantly in form, function, and tissue specificity. Because of their importance in clinical research, they are grouped into subdivisions, the so-called collagen families, and their sequences are often analysed. However, problems arise with highly homologous sequence segments. To increase the accuracy of collagen classification and prediction of their functions, the structure of these collagens and their expression in different tissues could result in a better focus on sequence segments of interest. Here, we analyse collagen families with different levels of conservation. As a result, clusters with high interconnectivity can be found, such as the fibrillar collagens, the COL4 network-forming collagens, and the COL9 FACITs. Furthermore, a large cluster between network-forming, FACIT, and COL28a1 α-chains is formed with COL6a3 as a major hub node. The formation of clusters also signifies, why it is important to always analyse the α-chains and why structural changes can have a wide range of effects on the body.

On the Construction of Confidence Regions for Uncertain Planar Displacements.

This paper studies the statistical concept of confidence region for a set of uncertain planar displacements with a certain level of confidence or probabilities. Three different representations of planar displacements are compared in this context and it is shown that the most commonly used representation based on the coordinates of the moving frame is the least effective. The other two methods, namely the exponential coordinates and planar quaternions, are equally effective in capturing the group structure of SE(2). However, the former relies on the exponential map to parameterize an element of SE(2), while the latter uses a quadratic map, which is often more advantageous computationally. This paper focus on the use of planar quaternions to develop a method for computing the confidence region for a given set of uncertain planar displacements. Principal component analysis (PCA) is another tool used in our study to capture the dominant direction of movements. To demonstrate the effectiveness of our approach, we compare it to an existing method called rotational and translational confidence limit (RTCL). Our examples show that the planar quaternion formulation leads to a swept volume that is more compact and more effective than the RTCL method, especially in cases when off-axis rotation is present.

Ultrastructural calibration model for proficiency testing.

Proficiency testing (PT) determines the performance of individual laboratories for specific tests or measurements and it is used to monitor the reliability of laboratories measurements. PT plays a highly valuable role as it provides objective evidence of the competence of the participant laboratories. In this paper, we propose a multivariate calibration model to assess equivalence among laboratories measurements in PT. Our method allows to deal with multivariate data, where the item under test is measured at different levels. Although intuitive, the proposed model is nonergodic, which means that the asymptotic Fisher information matrix is random. As a consequence, a detailed asymptotic analysis was carried out to establish the strategy for comparing the results of the participating laboratories. To illustrate, we apply our method to analyze the data from the Brazilian engine test group, PT program, where the power of an engine was measured by eight laboratories at several levels of rotation.

Empirical likelihood for spatial dynamic panel data models.

Spatial dynamic panel data (SDPD) models have received great attention in economics in recent 10 years. Existing approaches for the estimation and test of SDPD models are quasi-maximum likelihood (QML) approach and generalized method of moments (GMM). In this article, we introduce the empirical likelihood (EL) method to the statistical inference for SDPD models. The EL ratio statistics are constructed for the parameters of spatial dynamic panel data models. It is shown that the limiting distributions of the empirical likelihood ratio statistics are chi-squared distributions, which are used to construct confidence regions for the parameters of the models. Simulation results show that the EL based confidence regions outperform the normal approximation based confidence regions.

Comparison of Spectroscopic Techniques for Determining the Peroxide Value of 19 Classes of Naturally Aged, Plant-Based Edible Oils.

The peroxide value of edible oils is a measure of the degree of oxidation, which directly relates to the freshness of the oil sample. Several studies previously reported in the literature have paired various spectroscopic techniques with multivariate analyses to rapidly determine peroxide values using field portable and process instrumentation; those efforts presented "best-case scenarios" with oils from narrowly defined training and test sets. The purpose of this paper is to evaluate the use of near- and mid-infrared absorption and Raman scattering spectroscopies on oil samples from different oil classes, including seasonal and vendor variations, to determine which measurement technique or combination thereof is best for predicting peroxide values. Following peroxide value assays of each oil class using an established titration-based method, global and global-subset calibration models were constructed from spectroscopic data collected on the 19 oil classes used in this study. Spectra from each optical technique were used to create partial least squares regression calibration models to predict the peroxide value of unknown oil samples. A global peroxide value model based on near-infrared (8 mm optical path length) oil measurements produced the lowest RMSEP (4.9), followed by 24 mm optical path length near infrared (5.1), Raman (6.9) and 50 µm optical path length mid-infrared (7.3). However, it was determined that the Raman RMSEP resulted from chance correlations. Global peroxide value models based on low-level fusion of the NIR (8 and 24 mm optical path length) data and all infrared data produced the same RMSEP of 5.1. Global subset models, based on any of the spectroscopies and olive oil training sets from any class (pure, extra light, extra virgin), all failed to extrapolate to the non-olive oils. However, the near-infrared global subset model built on extra virgin olive oil could extrapolate to test samples from other olive oil classes. This work demonstrates the difficulty of developing a truly global method for determining peroxide value of oils.

EXPRESS: Comparison of Spectroscopic Techniques for Determining the Peroxide Value of 19 Classes of Naturally Aged, Plant-Based Edible Oils.

The peroxide value (PV) of edible oils is a measure of the degree of oxidation, which directly relates to the freshness of the oil sample. Several studies previously reported in the literature have paired various spectroscopic techniques with multivariate analyses to rapidly determine PVs using field portable and process instrumentation; those efforts presented âbest-caseâ scenarios with oils from narrowly defined training and test sets. The purpose of this paper is to evaluate the use of near- and mid-infrared absorption and Raman scattering spectroscopies on oil samples from different oil classes, including seasonal and vendor variations, to determine which measurement technique, or combination thereof, is best for predicting PVs. Following PV assays of each oil class using an established titration-based method, global and global-subset calibration models were constructed from spectroscopic data collected on the 19 oil classes used in this study. Spectra from each optical technique were used to create partial least squares regression (PLSR) calibration models to predict the PV of unknown oil samples. A global PV model based on near-infrared (8 mm optical path length â OPL) oil measurements produced the lowest RMSEP (4.9), followed by 24 mm OPL near infrared (5.1), Raman (6.9) and 50 λm OPL mid-infrared (7.3). However, it was determined that the Raman RMSEP resulted from chance correlations. Global PV models based on low-level fusion of the NIR (8 and 24 mm OPL) data and all infrared data produced the same RMSEP of 5.1. Global subset models, based on any of the spectroscopies and olive oil training sets from any class (pure, extra light, extra virgin), all failed to extrapolate to the non-olive oils. However, the near-infrared global subset model built on extra virgin olive oil could extrapolate to test samples from other olive oil classes.

Real-time prediction of influenza outbreaks in Belgium.

Seasonal influenza is a worldwide public health concern. Forecasting its dynamics can improve the management of public health regulations, resources and infrastructure, and eventually reduce mortality and the costs induced by influenza-related absenteism. In Belgium, a network of Sentinel General Practitioners (SGPs) is in place for the early detection of the seasonal influenza epidemic. This surveillance network reports the weekly incidence of influenza-like illness (ILI) cases, which makes it possible to detect the epidemic onset, as well as other characteristics of the epidemic season. In this paper, we present an approach for predicting the weekly ILI incidence in real-time by resorting to a dynamically calibrated compartmental model, which furthermore takes into account the dynamics of other influenza seasons. In order to validate the proposed approach, we used data collected by the Belgian SGPs for the influenza seasons 2010-2016. In spite of the great variability among different epidemic seasons, providing weekly predictions makes it possible to capture variations in the ILI incidence. The confidence region becomes more representative of the epidemic behavior as ILI data from more seasons become available. Since the SIR model is then calibrated dynamically every week, the predicted ILI curve gets rapidly tuned to the dynamics of the ongoing season. The results show that the proposed method can be used to characterize the overall behavior of an epidemic.