Bayesian modeling and inference for diagnostic accuracy and probability of disease based on multiple diagnostic biomarkers with and without a perfect reference standard.

The area under the receiver operating characteristic (ROC) curve (AUC) is used as a performance metric for quantitative tests. Although multiple biomarkers may be available for diagnostic or screening purposes, diagnostic accuracy is often assessed individually rather than in combination. In this paper, we consider the interesting problem of combining multiple biomarkers for use in a single diagnostic criterion with the goal of improving the diagnostic accuracy above that of an individual biomarker. The diagnostic criterion created from multiple biomarkers is based on the predictive probability of disease, conditional on given multiple biomarker outcomes. If the computed predictive probability exceeds a specified cutoff, the corresponding subject is allocated as 'diseased'. This defines a standard diagnostic criterion that has its own ROC curve, namely, the combined ROC (cROC). The AUC metric for cROC, namely, the combined AUC (cAUC), is used to compare the predictive criterion based on multiple biomarkers to one based on fewer biomarkers. A multivariate random-effects model is proposed for modeling multiple normally distributed dependent scores. Bayesian methods for estimating ROC curves and corresponding (marginal) AUCs are developed when a perfect reference standard is not available. In addition, cAUCs are computed to compare the accuracy of different combinations of biomarkers for diagnosis. The methods are evaluated using simulations and are applied to data for Johne's disease (paratuberculosis) in cattle.

Flexible regression models for ROC and risk analysis, with or without a gold standard.

A novel semiparametric regression model is developed for evaluating the covariate-specific accuracy of a continuous medical test or biomarker. Ideally, studies designed to estimate or compare medical test accuracy will use a separate, flawless gold-standard procedure to determine the true disease status of sampled individuals. We treat this as a special case of the more complicated and increasingly common scenario in which disease status is unknown because a gold-standard procedure does not exist or is too costly or invasive for widespread use. To compensate for missing data on disease status, covariate information is used to discriminate between diseased and healthy units. We thus model the probability of disease as a function of 'disease covariates'. In addition, we model test/biomarker outcome data to depend on 'test covariates', which provides researchers the opportunity to quantify the impact of covariates on the accuracy of a medical test. We further model the distributions of test outcomes using flexible semiparametric classes. An important new theoretical result demonstrating model identifiability under mild conditions is presented. The modeling framework can be used to obtain inferences about covariate-specific test accuracy and the probability of disease based on subject-specific disease and test covariate information. The value of the model is illustrated using multiple simulation studies and data on the age-adjusted ability of soluble epidermal growth factor receptor - a ubiquitous serum protein - to serve as a biomarker of lung cancer in men. SAS code for fitting the model is provided. Copyright © 2015 John Wiley & Sons, Ltd.

Limited utility of current MRI criteria for distinguishing multiple sclerosis from common mimickers: primary and secondary CNS vasculitis, lupus and Sjogren's syndrome.

BACKGROUND: Magnetic resonance imaging (MRI) criteria play an important role in making an earlier diagnosis of multiple sclerosis (MS) in patients presenting with clinically isolated syndrome. OBJECTIVE: The objective of this paper is to determine whether MRI criteria may be used to distinguish MS from primary and secondary central nervous system (CNS) vasculitis, lupus, and Sjogren's syndrome. METHODS: MRI criteria were applied retrospectively to images for patients with clinically definite MS (CDMS), primary CNS vasculitis, secondary CNS vasculitis, and autoimmune disorders including systemic lupus erythematosus (SLE) and Sjogren's syndrome. Classical statistics and Bayesian analyses were performed. RESULTS: Overall modified Barkhof's MRI criteria were statistically significant in distinguishing CDMS (60%) from SLE/Sjogren's syndrome (17%, p = 0.0173) but not in distinguishing CDMS from primary CNS vasculitis (50%, p = 0.7376) or secondary CNS vasculitis (58%, p = 1.0000). Four of the five other MRI criteria tested were demonstrated to be superior to modified Barkhof's criteria in predicting MS: nine or more T2 lesions (a component of Barkhof's criteria), one or more ovoid periventricular T2 lesions, one or more perpendicular periventricular T2 lesions, and one or more T2 lesions larger than 6 mm. CONCLUSIONS: MRI criteria, including the modified Barkhof's criteria, were unsuccessful in distinguishing MS from primary CNS vasculitis or secondary CNS vasculitis and mildly successful in distinguishing MS from SLE/Sjogren's syndrome.

Bayesian nonparametric variable selection as an exploratory tool for discovering differentially expressed genes.

High-throughput scientific studies involving no clear a priori hypothesis are common. For example, a large-scale genomic study of a disease may examine thousands of genes without hypothesizing that any specific gene is responsible for the disease. In these studies, the objective is to explore a large number of possible factors (e.g., genes) in order to identify a small number that will be considered in follow-up studies that tend to be more thorough and on smaller scales. A simple, hierarchical, linear regression model with random coefficients is assumed for case-control data that correspond to each gene. The specific model used will be seen to be related to a standard Bayesian variable selection model. Relatively large regression coefficients correspond to potential differences in responses for cases versus controls and thus to genes that might 'matter'. For large-scale studies, and using a Dirichlet process mixture model for the regression coefficients, we are able to find clusters of regression effects of genes with increasing potential effect or 'relevance', in relation to the outcome of interest. One cluster will always correspond to genes whose coefficients are in a neighborhood that is relatively close to zero and will be deemed least relevant. Other clusters will correspond to increasing magnitudes of the random/latent regression coefficients. Using simulated data, we demonstrate that our approach could be quite effective in finding relevant genes compared with several alternative methods. We apply our model to two large-scale studies. The first study involves transcriptome analysis of infection by human cytomegalovirus. The second study's objective is to identify differentially expressed genes between two types of leukemia.

Modelling variation in test sensitivity for monitoring leptospirosis in beef cattle.

When Bayesian latent class analysis is used for diagnostic test data in the absence of a gold standard test, it is common to assume that any unknown test sensitivities and specificities are constant across different populations. Indeed this assumption is often necessary for model identifiability. However there are a number of practical situations, depending on the type of test and the nature of the disease, where this assumption may not be true. We present a case study of using a microscopic agglutination test to diagnose leptospiroris infection in beef cattle, which strongly suggests that sensitivity in particular varies among herds. We develop and fit an alternative model in which sensitivity is related to within-herd prevalence, and discuss the statistical and epidemiological implications.

Sample size calculations for ROC studies: parametric robustness and Bayesian nonparametrics.

Methods for sample size calculations in ROC studies often assume independent normal distributions for test scores among the diseased and nondiseased populations. We consider sample size requirements under the default two-group normal model when the data distribution for the diseased population is either skewed or multimodal. For these two common scenarios we investigate the potential for robustness of calculated sample sizes under the mis-specified normal model and we compare to sample sizes calculated under a more flexible nonparametric Dirichlet process mixture model. We also highlight the utility of flexible models for ROC data analysis and their importance to study design. When nonstandard distributional shapes are anticipated, our Bayesian nonparametric approach allows investigators to determine a sample size based on the use of more appropriate distributional assumptions than are generally applied. The method also provides researchers a tool to conduct a sensitivity analysis to sample size calculations that are based on a two-group normal model. We extend the proposed approach to comparative studies involving two continuous tests. Our simulation-based procedure is implemented using the WinBUGS and R software packages and example code is made available.

Patterns of Sexual Activity and the Development of Sexual Pain Across the Menopausal Transition.

OBJECTIVE: To examine whether patterns of sexual intercourse frequency and demographic, menopausal status, genitourinary, health, and psychosocial factors are associated with developing sexual pain across the menopausal transition. METHODS: These were longitudinal analyses of questionnaire data from the multicenter, multiracial and ethnic prospective cohort SWAN (Study of Women's Health Across the Nation) (1995-2008). We used multivariable discrete-time proportional hazards models to examine whether incident sexual pain was associated with preceding long-term (up to 10 visits) or short-term (two and three visits) sexual intercourse frequency patterns or other factors (eg, menopause status, genitourinary symptoms, lifestyle factors, and mental health). RESULTS: Of the 2,247 women with no sexual pain at baseline, 1,087 (48.4%) developed sexual pain at least "sometimes" up to 10 follow-up visits over 13 years. We found no consistent association between prior patterns of sexual intercourse frequency and development of sexual pain. For example, neither decreases in intercourse frequency from baseline (adjusted hazard ratio [aHR] 0.93, 95% CI 0.73-1.19) nor decreases in frequency over three prior visits (aHR 1.00, 95% CI 0.72-1.41) were associated with incident pain. Reasons for interruptions in intercourse activity at the prior visit, including lack of interest (aHR 1.64, 95% CI 0.74-3.65) and relationship issues (aHR 0.36, 95% CI 0.04-2.88), were not associated with developing pain. Being postmenopausal using hormone therapy (aHR 3.16, 95% CI 1.46-6.85), and reported vaginal dryness (aHR 3.73, 95% CI 2.88-4.83) were most strongly associated with incident sexual pain. CONCLUSION: Long-term and short-term declines in sexual intercourse frequency across the menopausal transition were not associated with increased hazard of developing pain with intercourse. This empirical evidence does not support the common belief that a reduction in women's sexual frequency is responsible for their symptoms of sexual pain.

Estimating equations for biomarker based exposure estimation under non-steady-state conditions.

Unrealistic steady-state assumptions are often used to estimate toxicant exposure rates from biomarkers. A biomarker may instead be modeled as a weighted sum of historical time-varying exposures. Estimating equations are derived for a zero-inflated gamma distribution for daily exposures with a known exposure frequency. Simulation studies suggest that the estimating equations can provide accurate estimates of exposure magnitude at any reasonable sample size, and reasonable estimates of the exposure variance at larger sample sizes.

Diagnostic performance tests for suspected scaphoid fractures differ with conventional and latent class analysis.

BACKGROUND: Evaluation of the diagnostic performance characteristics of radiographic tests for diagnosing a true fracture among suspected scaphoid fractures is hindered by the lack of a consensus reference standard. Latent class analysis is a statistical method that takes advantage of unobserved, or latent, classes in the data that can be used to determine diagnostic performance characteristics when there is no consensus reference (gold) standard. PURPOSES: We therefore compared the diagnostic performance characteristics of MRI, CT, bone scintigraphy, and physical examination to identify true fractures among suspected scaphoid fractures. PATIENTS AND METHODS: We used data from two studies, one that prospectively studied 34 patients who had MRI and CT of the wrist, and a second that studied 78 patients who had MRI, bone scintigraphy, and structured physical examination. We compared the diagnostic performance characteristics calculated by latent class analysis with those calculated using formulas based on a reference standard. RESULTS: In the first cohort, the calculated sensitivity and specificity with latent class analysis were different than those with traditional reference standard-based calculations for the CT in the scaphoid planes (sensitivity, 0.78 versus 0.67; specificity, 1.0 versus 0.96) and the MRI (sensitivity, 0.80 versus 0.67; specificity, 0.93 versus 0.89). In the second cohort, the greatest differences were in the sensitivity of MRI (0.84 versus 0.75) and the sensitivities of physical examination maneuvers (range, 0.63-0.73 versus 1.0). CONCLUSIONS: The diagnostic performance characteristics calculated using latent class analysis may differ from those calculated according to formulas based on a reference standard. We believe latent class analysis merits further study as an option for assessing diagnostic performance characteristics for orthopaedic conditions when there is no consensus reference standard. LEVEL OF EVIDENCE: Level II, prognostic study. See the Guidelines for Authors for a complete description of levels of evidence.

Predictive comparison of joint longitudinal-survival modeling: a case study illustrating competing approaches.

The joint modeling of longitudinal and survival data has received extraordinary attention in the statistics literature recently, with models and methods becoming increasingly more complex. Most of these approaches pair a proportional hazards survival with longitudinal trajectory modeling through parametric or nonparametric specifications. In this paper we closely examine one data set previously analyzed using a two parameter parametric model for Mediterranean fruit fly (medfly) egg-laying trajectories paired with accelerated failure time and proportional hazards survival models. We consider parametric and nonparametric versions of these two models, as well as a proportional odds rate model paired with a wide variety of longitudinal trajectory assumptions reflecting the types of analyses seen in the literature. In addition to developing novel nonparametric Bayesian methods for joint models, we emphasize the importance of model selection from among joint and non joint models. The default in the literature is to omit at the outset non joint models from consideration. For the medfly data, a predictive diagnostic criterion suggests that both the choice of survival model and longitudinal assumptions can grossly affect model adequacy and prediction. Specifically for these data, the simple joint model used in by Tseng et al. (Biometrika 92:587-603, 2005) and models with much more flexibility in their longitudinal components are predictively outperformed by simpler analyses. This case study underscores the need for data analysts to compare on the basis of predictive performance different joint models and to include non joint models in the pool of candidates under consideration.

Analysis of German BSE Surveillance Data: Estimation of the Prevalence of Confirmed Cases versus the Number of Infected, but Non-Detected, Cattle to Assess Confidence in Freedom from Infection.

Quantitative risk assessments for Bovine spongiform encephalopathy (BSE) necessitate estimates for key parameters such as the prevalence of infection, the probability of absence of infection in defined birth cohorts, and the numbers of BSE-infected, but non-detected cattle entering the food chain. We estimated three key parameters with adjustment for misclassification using the German BSE surveillance data using a Gompertz model for latent (i.e., unobserved) age-dependent detection probabilities and a Poisson response model for the number of BSE cases for birth cohorts 1999 to 2015. The models were combined in a Bayesian framework. We estimated the median true BSE prevalence between 3.74 and 0.216 cases per 100,000 animals for the birth cohorts 1990 to 2001 and observed a peak for the 1996 birth cohort with a point estimate of 16.41 cases per 100,000 cattle. For birth cohorts ranging from 2002 to 2013, the estimated median prevalence was below one case per 100,000 heads. The calculated confidence in freedom from disease (design prevalence 1 in 100,000) was above 99.5% for the birth cohorts 2002 to 2006. In conclusion, BSE surveillance in the healthy slaughtered cattle chain was extremely sensitive at the time, when BSE repeatedly occurred in Germany (2000-2009), because the entry of BSE-infected cattle into the food chain could virtually be prevented by the extensive surveillance program during these years and until 2015 (estimated non-detected cases/100.000 [95% credible interval] in 2000, 2009, and 2015 are 0.64 [0.5,0.8], 0.05 [0.01,0.14], and 0.19 [0.05,0.61], respectively).

Identifiability of models for multiple diagnostic testing in the absence of a gold standard.

We discuss the issue of identifiability of models for multiple dichotomous diagnostic tests in the absence of a gold standard (GS) test. Data arise as multinomial or product-multinomial counts depending upon the number of populations sampled. Models are generally posited in terms of population prevalences, test sensitivities and specificities, and test dependence terms. It is commonly believed that if the degrees of freedom in the data meet or exceed the number of parameters in a fitted model then the model is identifiable. Goodman (1974, Biometrika 61, 215-231) established that this was not the case a long time ago. We discuss currently available models for multiple tests and argue in favor of an extension of a model that was developed by Dendukuri and Joseph (2001, Biometrics 57, 158-167). Subsequently, we further develop Goodman's technique, and make geometric arguments to give further insight into the nature of models that lack identifiability. We present illustrations using simulated and real data.

Bayesian estimation of the receiver operating characteristic curve for a diagnostic test with a limit of detection in the absence of a gold standard.

The receiver operating characteristic (ROC) curve is commonly used for evaluating the discriminatory ability of a biomarker. Measurements for a diagnostic test may be subject to an analytic limit of detection leading to immeasurable or unreportable test results. Ignoring the scores that are beyond the limit of detection of a test leads to a biased assessment of its discriminatory ability, as reflected by indices such as the associated area under the curve (AUC). We propose a Bayesian approach for the estimation of the ROC curve and its AUC for a test with a limit of detection in the absence of gold standard based on assumptions of normally and gamma-distributed data. The methods are evaluated in simulation studies, and a truncated gamma model with a point mass is used to evaluate quantitative real-time polymerase chain reaction data for bovine Johne's disease (paratuberculosis). Simulations indicated that estimates of diagnostic accuracy and AUC were good even for relatively small sample sizes (n=200). Exceptions were when there was a high per cent of unquantifiable results (60 per cent) or when AUC was < or =0.6, which indicated a marked overlap between the outcomes in infected and non-infected populations.

Evaluation of diagnostic tests for Johne's disease (Mycobacterium avium subspecies paratuberculosis) in New Zealand dairy cows.

To quantify the sensitivity and specificity of a serum enzyme-linked immunosorbent assay (ELISA) and fecal culture (FC) tests and to estimate the prevalence of Johne's disease (JD) in New Zealand dairy herds using Bayesian methods, 4 New Zealand dairy herds were tested simultaneously by ELISA and FC 5 times over 3 lactations. Test results were dichotomized. A Bayesian regression model was developed that considered test sensitivity as a function of the covariates parity, lactation stage, and prevalence of JD, which is expected to vary between herds. It was applied to a cross-sectional subset of the data and the full, repeated measures data set. Results were compared with frequentist pseudo gold standard results of the full data. Using the regression model, sensitivity of the ELISA was higher in older animals, but the sensitivity of the FC test showed no trend across age groups. Both FC and ELISA sensitivity were lower in late lactation. Estimated prevalence was lower and FC sensitivity higher when analyzing the complete data. The regression model enabled a more accurate diagnosis of JD to be made because it incorporated cow-specific information in the diagnosis, such as age and lactation stage. The model also enabled the incorporation of previous test results for an individual when diagnosing disease. The trends in results from the regression model support the current understanding of the disease process. The advantage of repeated testing of individuals in the assessment of test performance is discussed in the current study.

Bayesian nonparametric nonproportional hazards survival modeling.

We develop a dependent Dirichlet process model for survival analysis data. A major feature of the proposed approach is that there is no necessity for resulting survival curve estimates to satisfy the ubiquitous proportional hazards assumption. An illustration based on a cancer clinical trial is given, where survival probabilities for times early in the study are estimated to be lower for those on a high-dose treatment regimen than for those on the low dose treatment, while the reverse is true for later times, possibly due to the toxic effect of the high dose for those who are not as healthy at the beginning of the study.

DIFFERENTIAL IMPACT OF WEST NILE VIRUS ON CALIFORNIA BIRDS.

The strain of West Nile virus (WNV) currently epidemic in North America contains a genetic mutation elevating its virulence in birds, especially species in the family Corvidae. Although dead American Crows (Corvus brachyrhynchos) have been the hallmark of the epidemic, the overall impact of WNV on North America's avifauna remains poorly understood and has not been addressed thoroughly in California. Here, we evaluate variation by species in the effect of WNV on California birds from 2004 to 2007 by using (1) seroprevalence in free-ranging birds, (2) percentage of carcasses of each species reported by the public that tested positive for WNV, (3) mortality determined from experimental infections, and (4) population declines detected by trend analysis of Breeding Bird Survey (BBS) data. Using Bayesian linear models, we extrapolate trends in BBS data from 1980-2003 (pre-WNV) to 2004-2007 (post-WNV). We attribute significant declines from expected abundance trends in areas supporting epiornitics to WNV transmission. We combine risk assessed from each of the four data sets to generate an overall score describing WNV risk by species. The susceptibility of California avifauna to WNV varies widely, with overall risk scores ranging from low for the refractory Rock Pigeon (Columba livia) through high for the susceptible American Crow. Other species at high risk include, in descending order, the House Finch (Carpodacus mexicanus), Black-crowned Night-Heron (Nycticorax nycticorax), Western Scrub-Jay (Aphelocoma californica), and Yellow-billed Magpie (Pica nuttalli). Our analyses emphasize the importance of multiple data sources in assessing the effect of an invading pathogen.

Gold standards are out and Bayes is in: Implementing the cure for imperfect reference tests in diagnostic accuracy studies.

Bayesian mixture models, often termed latent class models, allow users to estimate the diagnostic accuracy of tests and true prevalence in one or more populations when the positive and/or negative reference standards are imperfect. Moreover, they allow the data analyst to show the superiority of a novel test over an old test, even if this old test is the (imperfect) reference standard. We use published data on Toxoplasmosis in pigs to explore the effects of numbers of tests, numbers of populations, and dependence structure among tests to ensure model (local) identifiability. We discuss and make recommendations about use of priors, sensitivity analysis, model identifiability and study design options, and strongly argue for the use of Bayesian mixture models as a logical and coherent approach for estimating the diagnostic accuracy of two or more tests.

Comparing Objective and Subjective Bayes Factors for the Two-Sample Comparison: The Classification Theorem in Action.

Many Bayes factors have been proposed for comparing population means in two-sample (independent samples) studies. Recently, Wang and Liu (2015) presented an "objective" Bayes factor (BF) as an alternative to a "subjective" one presented by Gönen et al. (2005). Their report was evidently intended to show the superiority of their BF based on "undesirable behavior" of the latter. A wonderful aspect of Bayesian models is that they provide an opportunity to "lay all cards on the table." What distinguishes the various BFs in the two-sample problem is the choice of priors (cards) for the model parameters. This article discusses desiderata of BFs that have been proposed, and proposes a new criterion to compare BFs, no matter whether subjectively or objectively determined: A BF may be preferred if it correctly classifies the data as coming from the correct model most often. The criterion is based on a famous result in classification theory to minimize the total probability of misclassification. This criterion is objective, easily verified by simulation, shows clearly the effects (positive or negative) of assuming particular priors, provides new insights into the appropriateness of BFs in general, and provides a new answer to the question, "Which BF is best?"