Comparing methods for statistical inference with model uncertainty.

Probability models are used for many statistical tasks, notably parameter estimation, interval estimation, inference about model parameters, point prediction, and interval prediction. Thus, choosing a statistical model and accounting for uncertainty about this choice are important parts of the scientific process. Here we focus on one such choice, that of variables to include in a linear regression model. Many methods have been proposed, including Bayesian and penalized likelihood methods, and it is unclear which one to use. We compared 21 of the most popular methods by carrying out an extensive set of simulation studies based closely on real datasets that span a range of situations encountered in practical data analysis. Three adaptive Bayesian model averaging (BMA) methods performed best across all statistical tasks. These used adaptive versions of Zellner's g-prior for the parameters, where the prior variance parameter g is a function of sample size or is estimated from the data. We found that for BMA methods implemented with Markov chain Monte Carlo, 10,000 iterations were enough. Computationally, we found two of the three best methods (BMA with g=√n and empirical Bayes-local) to be competitive with the least absolute shrinkage and selection operator (LASSO), which is often preferred as a variable selection technique because of its computational efficiency. BMA performed better than Bayesian model selection (in which just one model is selected).

Semi-supervised mixture multi-source exchangeability model for leveraging real-world data in clinical trials.

The traditional trial paradigm is often criticized as being slow, inefficient, and costly. Statistical approaches that leverage external trial data have emerged to make trials more efficient by augmenting the sample size. However, these approaches assume that external data are from previously conducted trials, leaving a rich source of untapped real-world data (RWD) that cannot yet be effectively leveraged. We propose a semi-supervised mixture (SS-MIX) multisource exchangeability model (MEM); a flexible, two-step Bayesian approach for incorporating RWD into randomized controlled trial analyses. The first step is a SS-MIX model on a modified propensity score and the second step is a MEM. The first step targets a representative subgroup of individuals from the trial population and the second step avoids borrowing when there are substantial differences in outcomes among the trial sample and the representative observational sample. When comparing the proposed approach to competing borrowing approaches in a simulation study, we find that our approach borrows efficiently when the trial and RWD are consistent, while mitigating bias when the trial and external data differ on either measured or unmeasured covariates. We illustrate the proposed approach with an application to a randomized controlled trial investigating intravenous hyperimmune immunoglobulin in hospitalized patients with influenza, while leveraging data from an external observational study to supplement a subgroup analysis by influenza subtype.

A flexible Bayesian framework for individualized inference via adaptive borrowing.

The explosion in high-resolution data capture technologies in health has increased interest in making inferences about individual-level parameters. While technology may provide substantial data on a single individual, how best to use multisource population data to improve individualized inference remains an open research question. One possible approach, the multisource exchangeability model (MEM), is a Bayesian method for integrating data from supplementary sources into the analysis of a primary source. MEM was originally developed to improve inference for a single study by asymmetrically borrowing information from a set of similar previous studies and was further developed to apply a more computationally intensive symmetric borrowing in the context of basket trial; however, even for asymmetric borrowing, its computational burden grows exponentially with the number of supplementary sources, making it unsuitable for applications where hundreds or thousands of supplementary sources (i.e., individuals) could contribute to inference on a given individual. In this article, we propose the data-driven MEM (dMEM), a two-stage approach that includes both source selection and clustering to enable the inclusion of an arbitrary number of sources to contribute to individualized inference in a computationally tractable and data-efficient way. We illustrate the application of dMEM to individual-level human behavior and mental well-being data collected via smartphones, where our approach increases individual-level estimation precision by 84% compared with a standard no-borrowing method and outperforms recently proposed competing methods in 80% of individuals.

Bayesian joint models for multi-regional clinical trials.

In recent years, multi-regional clinical trials (MRCTs) have increased in popularity in the pharmaceutical industry due to their ability to accelerate the global drug development process. To address potential challenges with MRCTs, the International Council for Harmonisation released the E17 guidance document which suggests the use of statistical methods that utilize information borrowing across regions if regional sample sizes are small. We develop an approach that allows for information borrowing via Bayesian model averaging in the context of a joint analysis of survival and longitudinal data from MRCTs. In this novel application of joint models to MRCTs, we use Laplace's method to integrate over subject-specific random effects and to approximate posterior distributions for region-specific treatment effects on the time-to-event outcome. Through simulation studies, we demonstrate that the joint modeling approach can result in an increased rejection rate when testing the global treatment effect compared with methods that analyze survival data alone. We then apply the proposed approach to data from a cardiovascular outcomes MRCT.

Bayesian multiregional clinical trials using model averaging.

Multiregional clinical trials (MRCTs) provide the benefit of more rapidly introducing drugs to the global market; however, small regional sample sizes can lead to poor estimation quality of region-specific effects when using current statistical methods. With the publication of the International Conference for Harmonisation E17 guideline in 2017, the MRCT design is recognized as a viable strategy that can be accepted by regional regulatory authorities, necessitating new statistical methods that improve the quality of region-specific inference. In this article, we develop a novel methodology for estimating region-specific and global treatment effects for MRCTs using Bayesian model averaging. This approach can be used for trials that compare two treatment groups with respect to a continuous outcome, and it allows for the incorporation of patient characteristics through the inclusion of covariates. We propose an approach that uses posterior model probabilities to quantify evidence in favor of consistency of treatment effects across all regions, and this metric can be used by regulatory authorities for drug approval. We show through simulations that the proposed modeling approach results in lower MSE than a fixed-effects linear regression model and better control of type I error rates than a Bayesian hierarchical model.

Rethinking 13C-metabolic flux analysis - The Bayesian way of flux inference.

Metabolic reaction rates (fluxes) play a crucial role in comprehending cellular phenotypes and are essential in areas such as metabolic engineering, biotechnology, and biomedical research. The state-of-the-art technique for estimating fluxes is metabolic flux analysis using isotopic labelling (13C-MFA), which uses a dataset-model combination to determine the fluxes. Bayesian statistical methods are gaining popularity in the field of life sciences, but the use of 13C-MFA is still dominated by conventional best-fit approaches. The slow take-up of Bayesian approaches is, at least partly, due to the unfamiliarity of Bayesian methods to metabolic engineering researchers. To address this unfamiliarity, we here outline similarities and differences between the two approaches and highlight particular advantages of the Bayesian way of flux analysis. With a real-life example, re-analysing a moderately informative labelling dataset of E. coli, we identify situations in which Bayesian methods are advantageous and more informative, pointing to potential pitfalls of current 13C-MFA evaluation approaches. We propose the use of Bayesian model averaging (BMA) for flux inference as a means of overcoming the problem of model uncertainty through its tendency to assign low probabilities to both, models that are unsupported by data, and models that are overly complex. In this capacity, BMA resembles a tempered Ockham's razor. With the tempered razor as a guide, BMA-based 13C-MFA alleviates the problem of model selection uncertainty and is thereby capable of becoming a game changer for metabolic engineering by uncovering new insights and inspiring novel approaches.

Flexible evaluation of surrogate markers with Bayesian model averaging.

When long-term follow up is required for a primary endpoint in a randomized clinical trial, a valid surrogate marker can help to estimate the treatment effect and accelerate the decision process. Several model-based methods have been developed to evaluate the proportion of the treatment effect that is explained by the treatment effect on the surrogate marker. More recently, a nonparametric approach has been proposed allowing for more flexibility by avoiding the restrictive parametric model assumptions required in the model-based methods. While the model-based approaches suffer from potential mis-specification of the models, the nonparametric method fails to give desirable estimates when the sample size is small, or when the range of the data does not follow certain conditions. In this paper, we propose a Bayesian model averaging approach to estimate the proportion of treatment effect explained by the surrogate marker. Our procedure offers a compromise between the model-based approach and the nonparametric approach by introducing model flexibility via averaging over several candidate models and maintains the strength of parametric models with respect to inference. We compare our approach with previous model-based methods and the nonparametric method. Simulation studies demonstrate the advantage of our method when surrogate supports are inconsistent and sample sizes are small. We illustrate our method using data from the Diabetes Prevention Program study to examine hemoglobin A1c as a surrogate marker for fasting glucose.

Optimal model averaging for partially linear models with missing response variables and error-prone covariates.

We consider the problem of optimal model averaging for partially linear models when the responses are missing at random and some covariates are measured with error. A novel weight choice criterion based on the Mallows-type criterion is proposed for the weight vector to be used in the model averaging. The resulting model averaging estimator for the partially linear models is shown to be asymptotically optimal under some regularity conditions in terms of achieving the smallest possible squared loss. In addition, the existence of a local minimizing weight vector and its convergence rate to the risk-based optimal weight vector are established. Simulation studies suggest that the proposed model averaging method generally outperforms existing methods. As an illustration, the proposed method is applied to analyze an HIV-CD4 dataset.

Model-based bioequivalence approach for sparse pharmacokinetic bioequivalence studies: Model selection or model averaging?

Conventional pharmacokinetic (PK) bioequivalence (BE) studies aim to compare the rate and extent of drug absorption from a test (T) and reference (R) product using non-compartmental analysis (NCA) and the two one-sided test (TOST). Recently published regulatory guidance recommends alternative model-based (MB) approaches for BE assessment when NCA is challenging, as for long-acting injectables and products which require sparse PK sampling. However, our previous research on MB-TOST approaches showed that model misspecification can lead to inflated type I error. The objective of this research was to compare the performance of model selection (MS) on R product arm data and model averaging (MA) from a pool of candidate structural PK models in MBBE studies with sparse sampling. Our simulation study was inspired by a real case BE study using a two-way crossover design. PK data were simulated using three structural models under the null hypothesis and one model under the alternative hypothesis. MB-TOST was applied either using each of the five candidate models or following MS and MA with or without the simulated model in the pool. Assuming T and R have the same PK model, our simulation shows that following MS and MA, MB-TOST controls type I error rates at or below 0.05 and attains similar or even higher power than when using the simulated model. Thus, we propose to use MS prior to MB-TOST for BE studies with sparse PK sampling and to consider MA when candidate models have similar Akaike information criterion.

Predicting the multivariate zero-inflated counts: A novel model averaging method under Pearson loss.

Excessive zeros in multivariate count data are often observed in scenarios of biomedicine and public health. To provide a better analysis on this type of data, we first develop a marginalized multivariate zero-inflated Poisson (MZIP) regression model to directly interpret the overall exposure effects on marginal means. Then, we define a multiple Pearson residual for our newly developed MZIP regression model by simultaneously taking heterogeneity and correlation into consideration. Furthermore, a new model averaging prediction method is introduced based on the multiple Pearson residual, and the asymptotical optimality of this model averaging prediction is proved. Simulations and two empirical applications in medicine are used to illustrate the effectiveness of the proposed method.

Toxicity of rare earth elements (REEs) to marine organisms: Using species sensitivity distributions to establish water quality guidelines for protecting marine life.

A lack of chronic rare earth element (REE) toxicity data for marine organisms has impeded the establishment of numerical REE water quality benchmarks (e.g., guidelines) to protect marine life and assess ecological risk. This study determined the chronic no (significant) effect concentrations (N(S)ECs) and median-effect concentrations (EC50s) of eight key REEs (yttrium (Y), lanthanum (La), cerium (Ce), praseodymium (Pr), neodymium (Nd), gadolinium (Gd), dysprosium (Dy) and lutetium (Lu)) for 30 coastal marine organisms (encompassing 22 phyla and five trophic levels from temperate and tropical habitats). Organisms with calcifying life stages were most vulnerable to REEs, which competitively inhibit calcium uptake. The most sensitive organism was a sea urchin, with N(S)ECs ranging from 0.64 µg/L for Y to 1.9 µg/L for La and Pr, and EC50s ranging from 4.3 µg/L for Y to 14.4 µg/L for Pr. Conversely, the least sensitive organism was a cyanobacterium, with N(S)ECs ranging from 21 µg/L for Y to 73 µg/L for Pr, and EC50s ranging from 153 µg/L for Y to 535 µg/L for La. Median sensitivity varied 215-fold across all organisms. The two-fold difference in median toxicity (µmol/L EC50) among REEs (Y∼Gd > Lu∼Nd∼Dy∼Ce > La∼Pr) was attributed to offset differences in binding affinity (log K) to cell surface receptors and the percentage of free metal ion (REE3+) in the test waters. The toxicity (EC50) of the remaining REEs (samarium, europium, terbium, holmium, thulium and ytterbium) was predicted using a combination of physicochemical data and measured EC50s for the eight tested REEs, with good agreement between predicted and measured EC50s for selected organisms. Numerical REE water quality guidelines to protect marine life were established using species sensitivity distributions (e.g., for 95 % species protection, values ranged from 1.1 µg/L for Y to 3.0 µg/L for La, Pr or Lu).

The impact of lipidome on breast cancer: a Mendelian randomization study.

OBJECTIVE: This study aims to investigate the association between specific lipidomes and the risk of breast cancer (BC) using the Two-Sample Mendelian Randomization (TSMR) approach and Bayesian Model Averaging Mendelian Randomization (BMA-MR) method. METHOD: The study analyzed data from large-scale GWAS datasets of 179 lipidomes to assess the relationship between lipidomes and BC risk across different molecular subtypes. TSMR was employed to explore causal relationships, while the BMA-MR method was carried out to validate the results. The study assessed heterogeneity and horizontal pleiotropy through Cochran's Q, MR-Egger intercept tests, and MR-PRESSO. Moreover, a leave-one-out sensitivity analysis was performed to evaluate the impact of individual single nucleotide polymorphisms on the MR study. RESULTS: By examining 179 lipidome traits as exposures and BC as the outcome, the study revealed significant causal effects of glycerophospholipids, sphingolipids, and glycerolipids on BC risk. Specifically, for estrogen receptor-positive BC (ER+ BC), phosphatidylcholine (P < 0.05) and phosphatidylinositol (OR: 0.916-0.966, P < 0.05) within glycerophospholipids play significant roles, along with the importance of glycerolipids (diacylglycerol (OR = 0.923, P < 0.001) and triacylglycerol, OR: 0.894-0.960, P < 0.05)). However, the study did not observe a noteworthy impact of sphingolipids on ER+BC. In the case of estrogen receptor-negative BC (ER- BC), not only glycerophospholipids, sphingolipids (OR = 1.085, P = 0.008), and glycerolipids (OR = 0.909, P = 0.002) exerted an influence, but the protective effect of sterols (OR: 1.034-1.056, P < 0.05) was also discovered. The prominence of glycerolipids was minimal in ER-BC. Phosphatidylethanolamine (OR: 1.091-1.119, P < 0.05) was an important causal effect in ER-BC. CONCLUSIONS: The findings reveal that phosphatidylinositol and triglycerides levels decreased the risk of BC, indicating a potential protective role of these lipid molecules. Moreover, the study elucidates BC's intricate lipid metabolic pathways, highlighting diverse lipidome structural variations that may have varying effects in different molecular subtypes.

Bayesian model averaging of longitudinal dose-response models.

Selecting a safe and clinically beneficial dose can be difficult in drug development. Dose justification often relies on dose-response modeling where parametric assumptions are made in advance which may not adequately fit the data. This is especially problematic in longitudinal dose-response models, where additional parametric assumptions must be made. This paper proposes a class of longitudinal dose-response models to be used in the Bayesian model averaging paradigm which improve trial operating characteristics while maintaining flexibility a priori. A new longitudinal model for non-monotonic longitudinal profiles is proposed. The benefits and trade-offs of the proposed approach are demonstrated through a case study and simulation.

Testing Conditional Independence in Psychometric Networks: An Analysis of Three Bayesian Methods.

Network psychometrics uses graphical models to assess the network structure of psychological variables. An important task in their analysis is determining which variables are unrelated in the network, i.e., are independent given the rest of the network variables. This conditional independence structure is a gateway to understanding the causal structure underlying psychological processes. Thus, it is crucial to have an appropriate method for evaluating conditional independence and dependence hypotheses. Bayesian approaches to testing such hypotheses allow researchers to differentiate between absence of evidence and evidence of absence of connections (edges) between pairs of variables in a network. Three Bayesian approaches to assessing conditional independence have been proposed in the network psychometrics literature. We believe that their theoretical foundations are not widely known, and therefore we provide a conceptual review of the proposed methods and highlight their strengths and limitations through a simulation study. We also illustrate the methods using an empirical example with data on Dark Triad Personality. Finally, we provide recommendations on how to choose the optimal method and discuss the current gaps in the literature on this important topic.

Bayesian Averaging Evaluation Method of Accelerated Degradation Testing Considering Model Uncertainty Based on Relative Entropy.

To evaluate the lifetime and reliability of long-life, high-reliability products under limited resources, accelerated degradation testing (ADT) technology has been widely applied. Furthermore, the Bayesian evaluation method for ADT can comprehensively utilize historical information and overcome the limitations caused by small sample sizes, garnering significant attention from scholars. However, the traditional ADT Bayesian evaluation method has inherent shortcomings and limitations. Due to the constraints of small samples and an incomplete understanding of degradation mechanisms or accelerated mechanisms, the selected evaluation model may be inaccurate, leading to potentially inaccurate evaluation results. Therefore, describing and quantifying the impact of model uncertainty on evaluation results is a challenging issue that urgently needs resolution in the theoretical research of ADT Bayesian methods. This article addresses the issue of model uncertainty in the ADT Bayesian evaluation process. It analyzes the modeling process of ADT Bayesian and proposes a new model averaging evaluation method for ADT Bayesian based on relative entropy, which, to a certain extent, can resolve the issue of evaluation inaccuracy caused by model selection uncertainty. This study holds certain theoretical and engineering application value for conducting ADT Bayesian evaluation under model uncertainty.

Clusters of cognitive performance predict long-term cognitive impairment in elderly patients with subjective memory complaints and healthy controls.

INTRODUCTION: Patients with subjective memory complaints (SMC) may include subgroups with different neuropsychological profiles and risks of cognitive impairment. METHODS: Cluster analysis was performed on two datasets (n: 630 and 734) comprising demographic and neuropsychological data from SMC and healthy controls (HC). Survival analyses were conducted on clusters. Bayesian model averaging assessed the predictive utility of clusters and other biomarkers. RESULTS: Two clusters with higher and lower than average cognitive performance were detected in SMC and HC. Assignment to the lower performance cluster increased the risk of cognitive impairment in both datasets (hazard ratios: 1.78 and 2.96; Plog-rank: 0.04 and <0.001) and was associated with lower hippocampal volumes and higher tau/amyloid beta 42 ratios in cerebrospinal fluid. The effect of SMC was small and confounded by mood. DISCUSSION: This study provides evidence of the presence of cognitive clusters that hold biological significance and predictive value for cognitive decline in SMC and HC. HIGHLIGHTS: Patients with subjective memory complaints include two cognitive clusters. Assignment to the lower performance cluster increases risk of cognitive impairment. This cluster shows a pattern of biomarkers consistent with incipient Alzheimer's disease pathology. The same cognitive cluster structure is found in healthy controls. The effect of memory complaints on risk of cognitive decline is small and confounded.

Uncertainty-based saltwater intrusion prediction using integrated Bayesian machine learning modeling (IBMLM) in a deep aquifer.

Data-driven machine learning approaches are promising to substitute physically based groundwater numerical models and capture input-output relationships for reducing computational burden. But the performance and reliability are strongly influenced by different sources of uncertainty. Conventional researches generally rely on a stand-alone machine learning surrogate approach and fail to account for errors in model outputs resulting from structural deficiencies. To overcome this issue, this study proposes a flexible integrated Bayesian machine learning modeling (IBMLM) method to explicitly quantify uncertainties originating from structures and parameters of machine learning surrogate models. An Expectation-Maximization (EM) algorithm is combined with Bayesian model averaging (BMA) to find out maximum likelihood and construct posterior predictive distribution. Three machine learning approaches representing different model complexity are incorporated in the framework, including artificial neural network (ANN), support vector machine (SVM) and random forest (RF). The proposed IBMLM method is demonstrated in a field-scale real-world "1500-foot" sand aquifer, Baton Rouge, USA, where overexploitation caused serious saltwater intrusion (SWI) issues. This study adds to the understanding of how chloride concentration transport responds to multi-dimensional extraction-injection remediation strategies in a sophisticated saltwater intrusion model. Results show that most IBMLM exhibit r values above 0.98 and NSE values above 0.93, both slightly higher than individual machine learning, confirming that the IBMLM is well established to provide better model predictions than individual machine learning models, while maintaining the advantage of high computing efficiency. The IBMLM is found useful to predict saltwater intrusion without running the physically based numerical simulation model. We conclude that an explicit consideration of machine learning model structure uncertainty along with parameters improves accuracy and reliability of predictions, and also corrects uncertainty bounds. The applicability of the IBMLM framework can be extended in regions where a physical hydrogeologic model is difficult to build due to lack of subsurface information.

Model averaging for right censored data with measurement error.

This paper studies a novel model averaging estimation issue for linear regression models when the responses are right censored and the covariates are measured with error. A novel weighted Mallows-type criterion is proposed for the considered issue by introducing multiple candidate models. The weight vector for model averaging is selected by minimizing the proposed criterion. Under some regularity conditions, the asymptotic optimality of the selected weight vector is established in terms of its ability to achieve the lowest squared loss asymptotically. Simulation results show that the proposed method is superior to the other existing related methods. A real data example is provided to supplement the actual performance.

Estimation of urinary cadmium benchmark dose thresholds for preschool children in a cadmium-polluted area based on Bayesian model averaging.

Urinary cadmium (U-Cd) values are indicators for determining chronic cadmium toxicity, and previous studies have calculated U-Cd indicators using renal injury biomarkers. However, most of these studies have been conducted in adult populations, and there is a lack of research on U-Cd thresholds in preschool children. We aimed to apply benchmark dose (BMD) analysis to estimate the U-Cd threshold level associated with renal impairment in preschool children in the cadmium-polluted area. 518 preschool children aged 3-5 years were selected by systematic sampling (275 boys, 243 girls). Urinary cadmium and three biomarkers of early renal injury (urinary N-acetyl-ß-D-glucosaminidase, UNAG; urinary ß2-microglobulin, Uß2-MG; urinary retinol-binding protein, URBP) were determined. Bayesian model averaging estimated the BMD and lower confidence interval limit (BMDL) of U-Cd. The medians U-Cd levels in both boys and girls exceeded the recommended national standard threshold (5 µg/g cr) and U-Cd levels were higher in girls than in boys. Urinary N-acetyl-ß-D-glucosaminidase (UNAG) was the most sensitive biomarker of renal effects in preschool children. The overall BMDL5 (BMDL at a benchmark response value of 5) was 2.76 µg/g cr. In the gender analysis, the BMDL5 values were 1.92 µg/g cr for boys and 4.12 µg/g cr for girls. This study shows that the U-Cd threshold (BMDL5) is lower than the national standard (5 µg/g cr) and boys' BMDL5 was lower than the limit set by the European Parliament and Council in 2019 (2 µg/g cr), which provides a reference point for making U-Cd thresholds for preschool children.

A tutorial on Bayesian model-averaged meta-analysis in JASP.

Researchers conduct meta-analyses in order to synthesize information across different studies. Compared to standard meta-analytic methods, Bayesian model-averaged meta-analysis offers several practical advantages including the ability to quantify evidence in favor of the absence of an effect, the ability to monitor evidence as individual studies accumulate indefinitely, and the ability to draw inferences based on multiple models simultaneously. This tutorial introduces the concepts and logic underlying Bayesian model-averaged meta-analysis and illustrates its application using the open-source software JASP. As a running example, we perform a Bayesian meta-analysis on language development in children. We show how to conduct a Bayesian model-averaged meta-analysis and how to interpret the results.