Innovative statistical approaches: the use of neural networks reduces the sample size in the splenectomy-MCAO mouse model.

AIM: To compare the effectiveness of artificial neural network (ANN) and traditional statistical analysis on identical data sets within the splenectomy-middle carotid artery occlusion (MCAO) mouse model. METHODS: Mice were divided into the splenectomized (SPLX) and sham-operated (SPLX-sham) group. A splenectomy was conducted 14 days before middle carotid artery occlusion (MCAO). Magnetic resonance imaging (MRI), bioluminescent imaging, neurological scoring (NS), and histological analysis, were conducted at two, four, seven, and 28 days after MCAO. Frequentist statistical analyses and ANN analysis employing a multi-layer perceptron architecture were performed to assess the probability of discriminating between SPLX and SPLX-sham mice. RESULTS: Repeated measures ANOVA showed no significant differences in body weight (F (5, 45)=0.696, P=0.629), NS (F (2.024, 18.218)=1.032, P=0.377) and brain infarct size on MRI between the SPLX and SPLX-sham groups post-MCAO (F (2, 24)=0.267, P=0.768). ANN analysis was employed to predict SPLX and SPL-sham classes. The highest accuracy in predicting SPLX class was observed when the model was trained on a data set containing all variables (0.7736±0.0234). For SPL-sham class, the highest accuracy was achieved when it was trained on a data set excluding the variable combination MR contralateral/animal mass/NS (0.9284±0.0366). CONCLUSION: This study validated the neuroprotective impact of splenectomy in an MCAO model using ANN for data analysis with a reduced animal sample size, demonstrating the potential for leveraging advanced statistical methods to minimize sample sizes in experimental biomedical research.

Replication of null results: Absence of evidence or evidence of absence?

In several large-scale replication projects, statistically non-significant results in both the original and the replication study have been interpreted as a 'replication success.' Here, we discuss the logical problems with this approach: Non-significance in both studies does not ensure that the studies provide evidence for the absence of an effect and 'replication success' can virtually always be achieved if the sample sizes are small enough. In addition, the relevant error rates are not controlled. We show how methods, such as equivalence testing and Bayes factors, can be used to adequately quantify the evidence for the absence of an effect and how they can be applied in the replication setting. Using data from the Reproducibility Project: Cancer Biology, the Experimental Philosophy Replicability Project, and the Reproducibility Project: Psychology we illustrate that many original and replication studies with 'null results' are in fact inconclusive. We conclude that it is important to also replicate studies with statistically non-significant results, but that they should be designed, analyzed, and interpreted appropriately.

Upstrapping to determine futility: predicting future outcomes nonparametrically from past data.

BACKGROUND: Clinical trials often involve some form of interim monitoring to determine futility before planned trial completion. While many options for interim monitoring exist (e.g., alpha-spending, conditional power), nonparametric based interim monitoring methods are also needed to account for more complex trial designs and analyses. The upstrap is one recently proposed nonparametric method that may be applied for interim monitoring. METHODS: Upstrapping is motivated by the case resampling bootstrap and involves repeatedly sampling with replacement from the interim data to simulate thousands of fully enrolled trials. The p-value is calculated for each upstrapped trial and the proportion of upstrapped trials for which the p-value criteria are met is compared with a pre-specified decision threshold. To evaluate the potential utility for upstrapping as a form of interim futility monitoring, we conducted a simulation study considering different sample sizes with several different proposed calibration strategies for the upstrap. We first compared trial rejection rates across a selection of threshold combinations to validate the upstrapping method. Then, we applied upstrapping methods to simulated clinical trial data, directly comparing their performance with more traditional alpha-spending and conditional power interim monitoring methods for futility. RESULTS: The method validation demonstrated that upstrapping is much more likely to find evidence of futility in the null scenario than the alternative across a variety of simulations settings. Our three proposed approaches for calibration of the upstrap had different strengths depending on the stopping rules used. Compared to O'Brien-Fleming group sequential methods, upstrapped approaches had type I error rates that differed by at most 1.7% and expected sample size was 2-22% lower in the null scenario, while in the alternative scenario power fluctuated between 15.7% lower and 0.2% higher and expected sample size was 0-15% lower. CONCLUSIONS: In this proof-of-concept simulation study, we evaluated the potential for upstrapping as a resampling-based method for futility monitoring in clinical trials. The trade-offs in expected sample size, power, and type I error rate control indicate that the upstrap can be calibrated to implement futility monitoring with varying degrees of aggressiveness and that performance similarities can be identified relative to considered alpha-spending and conditional power futility monitoring methods.

Using Bayesian statistics in confirmatory clinical trials in the regulatory setting: a tutorial review.

Bayesian statistics plays a pivotal role in advancing medical science by enabling healthcare companies, regulators, and stakeholders to assess the safety and efficacy of new treatments, interventions, and medical procedures. The Bayesian framework offers a unique advantage over the classical framework, especially when incorporating prior information into a new trial with quality external data, such as historical data or another source of co-data. In recent years, there has been a significant increase in regulatory submissions using Bayesian statistics due to its flexibility and ability to provide valuable insights for decision-making, addressing the modern complexity of clinical trials where frequentist trials are inadequate. For regulatory submissions, companies often need to consider the frequentist operating characteristics of the Bayesian analysis strategy, regardless of the design complexity. In particular, the focus is on the frequentist type I error rate and power for all realistic alternatives. This tutorial review aims to provide a comprehensive overview of the use of Bayesian statistics in sample size determination, control of type I error rate, multiplicity adjustments, external data borrowing, etc., in the regulatory environment of clinical trials. Fundamental concepts of Bayesian sample size determination and illustrative examples are provided to serve as a valuable resource for researchers, clinicians, and statisticians seeking to develop more complex and innovative designs.

Bayesian estimation of the measurement of interactions in epidemiological studies.

Background: Interaction identification is important in epidemiological studies and can be detected by including a product term in the model. However, as Rothman noted, a product term in exponential models may be regarded as multiplicative rather than additive to better reflect biological interactions. Currently, the additive interaction is largely measured by the relative excess risk due to interaction (RERI), the attributable proportion due to interaction (AP), and the synergy index (S), and confidence intervals are developed via frequentist approaches. However, few studies have focused on the same issue from a Bayesian perspective. The present study aims to provide a Bayesian view of the estimation and credible intervals of the additive interaction measures. Methods: Bayesian logistic regression was employed, and estimates and credible intervals were calculated from posterior samples of the RERI, AP and S. Since Bayesian inference depends only on posterior samples, it is very easy to apply this method to preventive factors. The validity of the proposed method was verified by comparing the Bayesian method with the delta and bootstrap approaches in simulation studies with example data. Results: In all the simulation studies, the Bayesian estimates were very close to the corresponding true values. Due to the skewness of the interaction measures, compared with the confidence intervals of the delta method, the credible intervals of the Bayesian approach were more balanced and matched the nominal 95% level. Compared with the bootstrap method, the Bayesian method appeared to be a competitive alternative and fared better when small sample sizes were used. Conclusions: The proposed Bayesian method is a competitive alternative to other methods. This approach can assist epidemiologists in detecting additive-scale interactions.

Well-spread samples with dynamic sample sizes.

A spatial sampling design determines where sample locations are placed in a study area so that population parameters can be estimated with relatively high precision. If the response variable has spatial trends, spatially balanced or well-spread designs give precise results for commonly used estimators. This article proposes a new method that draws well-spread samples over arbitrary auxiliary spaces and can be used for master sampling applications. All we require is a measure of the distance between population units. Numerical results show that the method generates well-spread samples and compares favorably with existing designs. We provide an example application using several auxiliary variables to estimate total aboveground biomass over a large study area in Eastern Amazonia, Brazil. Multipurpose surveys are also considered, where the totals of aboveground biomass, primary production, and clay content (3 responses) are estimated from a single well-spread sample over the auxiliary space.

The effective sample size in Bayesian information criterion for level-specific fixed and random-effect selection in a two-level nested model.

Popular statistical software provides the Bayesian information criterion (BIC) for multi-level models or linear mixed models. However, it has been observed that the combination of statistical literature and software documentation has led to discrepancies in the formulas of the BIC and uncertainties as to the proper use of the BIC in selecting a multi-level model with respect to level-specific fixed and random effects. These discrepancies and uncertainties result from different specifications of sample size in the BIC's penalty term for multi-level models. In this study, we derive the BIC's penalty term for level-specific fixed- and random-effect selection in a two-level nested design. In this new version of BIC, called BIC E 1 , this penalty term is decomposed into two parts if the random-effect variance-covariance matrix has full rank: (a) a term with the log of average sample size per cluster and (b) the total number of parameters times the log of the total number of clusters. Furthermore, we derive the new version of BIC, called BIC E 2 , in the presence of redundant random effects. We show that the derived formulae, BIC E 1 and BIC E 2 , adhere to empirical values via numerical demonstration and that BIC E ( E indicating either E 1 or E 2 ) is the best global selection criterion, as it performs at least as well as BIC with the total sample size and BIC with the number of clusters across various multi-level conditions through a simulation study. In addition, the use of BIC E 1 is illustrated with a textbook example dataset.

GRADE guidance 37: rating imprecision in a body of evidence on test accuracy.

OBJECTIVES: To provide guidance on rating imprecision in a body of evidence assessing the accuracy of a single test. This guide will clarify when Grading of Recommendations Assessment, Development and Evaluation (GRADE) users should consider rating down the certainty of evidence by one or more levels for imprecision in test accuracy. STUDY DESIGN AND SETTING: A project group within the GRADE working group conducted iterative discussions and presentations at GRADE working group meetings to produce this guidance. RESULTS: Before rating the certainty of evidence, GRADE users should define the target of their certainty rating. GRADE recommends setting judgment thresholds defining what they consider a very accurate, accurate, inaccurate, and very inaccurate test. These thresholds should be set after considering consequences of testing and effects on people-important outcomes. GRADE's primary criterion for judging imprecision in test accuracy evidence is considering confidence intervals (i.e., CI approach) of absolute test accuracy results (true and false, positive, and negative results in a cohort of people). Based on the CI approach, when a CI appreciably crosses the predefined judgment threshold(s), one should consider rating down certainty of evidence by one or more levels, depending on the number of thresholds crossed. When the CI does not cross judgment threshold(s), GRADE suggests considering the sample size for an adequately powered test accuracy review (optimal or review information size [optimal information size (OIS)/review information size (RIS)]) in rating imprecision. If the combined sample size of the included studies in the review is smaller than the required OIS/RIS, one should consider rating down by one or more levels for imprecision. CONCLUSION: This paper extends previous GRADE guidance for rating imprecision in single test accuracy systematic reviews and guidelines, with a focus on the circumstances in which one should consider rating down one or more levels for imprecision.

Penalized weighted smoothed quantile regression for high-dimensional longitudinal data.

Quantile regression, known as a robust alternative to linear regression, has been widely used in statistical modeling and inference. In this paper, we propose a penalized weighted convolution-type smoothed method for variable selection and robust parameter estimation of the quantile regression with high dimensional longitudinal data. The proposed method utilizes a twice-differentiable and smoothed loss function instead of the check function in quantile regression without penalty, and can select the important covariates consistently using the efficient gradient-based iterative algorithms when the dimension of covariates is larger than the sample size. Moreover, the proposed method can circumvent the influence of outliers in the response variable and/or the covariates. To incorporate the correlation within each subject and enhance the accuracy of the parameter estimation, a two-step weighted estimation method is also established. Furthermore, we prove the oracle properties of the proposed method under some regularity conditions. Finally, the performance of the proposed method is demonstrated by simulation studies and two real examples.

On the distribution of the power function for the scale parameter of exponential families.

The expected value of the standard power function of a test, computed with respect to a design prior distribution, is often used to evaluate the probability of success of an experiment. However, looking only at the expected value might be reductive. Instead, the whole probability distribution of the power function induced by the design prior can be exploited. In this article we consider one-sided testing for the scale parameter of exponential families and we derive general unifying expressions for cumulative distribution and density functions of the random power. Sample size determination criteria based on alternative summaries of these functions are discussed. The study sheds light on the relevance of the choice of the design prior in order to construct a successful experiment.

Craniocervical instability in patients with Ehlers­Danlos syndromes: outcomes analysis following occipito­cervical fusion of published cases.

This critique provides a critical analysis of the outcomes following occipito-cervical fusion in patients with Ehlers-Danlos syndromes (EDS) and craniocervical instability. The study examines the efficacy of the surgical intervention and evaluates its impact on patient outcomes. While the article offers valuable insights into the management of EDS-related craniocervical instability, several limitations and areas for improvement are identified, including sample size constraints, the absence of a control group, and the need for long-term follow-up data. Future research efforts should focus on addressing these concerns to optimize treatment outcomes for individuals with EDS.

Factors Associated With Premature Termination of Hyperacute Stroke Trials: A Review.

BACKGROUND: We performed a review of acute stroke trials to determine features associated with premature termination of trial enrollment, defined by the authors as not meeting preplanned sample size. METHODS AND RESULTS: MEDLINE was searched for randomized clinical stroke trials published in 9 major clinical journals between 2013 and 2022. We included randomized clinical trials that were phase 2 or 3 with a preplanned sample size ≥100 and a time-to-treatment within 24 hours of onset for transient ischemic attack, ischemic stroke, or intracerebral hemorrhage. Data were abstracted on trial features including trial design, inclusion criteria, imaging, location and number of sites, masking, treatment complexity, control group (standard therapy, placebo), industry involvement, and preplanned stopping rules (futility and efficacy). Least absolute shrinkage and selection operator regression was used to select the most important factors associated with premature termination; then, a multivariable logistic regression was fit including only the least absolute shrinkage and selection operator selected variables. Of 1475 studies assessed, 98 trials met eligibility criteria. Forty-five (46%) trials were prematurely terminated, of which 27% were stopped for benefit/efficacy, 20% for lack of money/slow enrollment, 18% for futility, 16% for newly available evidence, 17% for other reasons, and 4% due to harm. Complex trials (adjusted odds ratio [aOR], 2.76 [95% CI, 1.13-7.49]), presence of a futility rule (aOR, 4.43 [95% CI, 1.62-17.91]), and exclusion of prestroke dependency (none/slight disability only; aOR, 2.19 [95% CI, 0.84-6.72] versus dependency allowed) were identified as the strongest predictors. CONCLUSIONS: Nearly half of acute stroke trials were terminated prematurely. Broadening inclusion criteria and simplifying trial design may decrease the likelihood of unplanned termination, whereas planned futility analyses may appropriately terminate trials early, saving money and resources.

Bayesian functional analysis for untargeted metabolomics data with matching uncertainty and small sample sizes.

Untargeted metabolomics based on liquid chromatography-mass spectrometry technology is quickly gaining widespread application, given its ability to depict the global metabolic pattern in biological samples. However, the data are noisy and plagued by the lack of clear identity of data features measured from samples. Multiple potential matchings exist between data features and known metabolites, while the truth can only be one-to-one matches. Some existing methods attempt to reduce the matching uncertainty, but are far from being able to remove the uncertainty for most features. The existence of the uncertainty causes major difficulty in downstream functional analysis. To address these issues, we develop a novel approach for Bayesian Analysis of Untargeted Metabolomics data (BAUM) to integrate previously separate tasks into a single framework, including matching uncertainty inference, metabolite selection and functional analysis. By incorporating the knowledge graph between variables and using relatively simple assumptions, BAUM can analyze datasets with small sample sizes. By allowing different confidence levels of feature-metabolite matching, the method is applicable to datasets in which feature identities are partially known. Simulation studies demonstrate that, compared with other existing methods, BAUM achieves better accuracy in selecting important metabolites that tend to be functionally consistent and assigning confidence scores to feature-metabolite matches. We analyze a COVID-19 metabolomics dataset and a mouse brain metabolomics dataset using BAUM. Even with a very small sample size of 16 mice per group, BAUM is robust and stable. It finds pathways that conform to existing knowledge, as well as novel pathways that are biologically plausible.

Uncertainty limits the use of power analysis.

The calculation of statistical power has been taken up as a simple yet informative tool to assist in designing an experiment, particularly in justifying sample size. A difficulty with using power for this purpose is that the classical power formula does not incorporate sources of uncertainty (e.g., sampling variability) that can impact the computed power value, leading to a false sense of precision and confidence in design choices. We use simulations to demonstrate the consequences of adding two common sources of uncertainty to the calculation of power. Sampling variability in the estimated effect size (Cohen's d) can introduce a large amount of uncertainty (e.g., sometimes producing rather flat distributions) in power and sample-size determination. The addition of random fluctuations in the population effect size can cause values of its estimates to take on a sign opposite the population value, making calculated power values meaningless. These results suggest that calculated power values or use of such values to justify sample size add little to planning a study. As a result, researchers should put little confidence in power-based choices when planning future studies. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Bioequivalence trials for the approval of generic drugs in Saudi Arabia: a descriptive analysis of design aspects.

BACKGROUND: This retrospective analysis aimed to comprehensively review the design and regulatory aspects of bioequivalence trials submitted to the Saudi Food and Drug Authority (SFDA) since 2017. METHODS: This was a retrospective, comprehensive analysis study. The Data extracted from the SFDA bioequivalence assessment reports were analyzed for reviewing the overall design and regulatory aspects of the successful bioequivalence trials, exploring the impact of the coefficient of variation of within-subject variability (CVw) on some design aspects, and providing an in-depth assessment of bioequivalence trial submissions that were deemed insufficient in demonstrating bioequivalence. RESULTS: A total of 590 bioequivalence trials were included of which 521 demonstrated bioequivalence (440 single active pharmaceutical ingredients [APIs] and 81 fixed combinations). Most of the successful trials were for cardiovascular drugs (84 out of 521 [16.1%]), and the 2 × 2 crossover design was used in 455 (87.3%) trials. The sample size tended to increase with the increase in the CVw in trials of single APIs. Biopharmaceutics Classification System Class II and IV drugs accounted for the majority of highly variable drugs (58 out of 82 [70.7%]) in the study. Most of the 51 rejected trials were rejected due to concerns related to the study center (n = 21 [41.2%]). CONCLUSION: This comprehensive analysis provides valuable insights into the regulatory and design aspects of bioequivalence trials and can inform future research and assist in identifying opportunities for improvement in conducting bioequivalence trials in Saudi Arabia.

Realism and robustness require increased sample size when studying both sexes.

A recent article claimed that researchers need not increase the overall sample size for a study that includes both sexes. This Formal Comment points out that that study assumed two sexes to have the same variance, and explains why this is a unrealistic assumption.

phylaGAN: data augmentation through conditional GANs and autoencoders for improving disease prediction accuracy using microbiome data.

MOTIVATION: Research is improving our understanding of how the microbiome interacts with the human body and its impact on human health. Existing machine learning methods have shown great potential in discriminating healthy from diseased microbiome states. However, Machine Learning based prediction using microbiome data has challenges such as, small sample size, imbalance between cases and controls and high cost of collecting large number of samples. To address these challenges, we propose a deep learning framework phylaGAN to augment the existing datasets with generated microbiome data using a combination of conditional generative adversarial network (C-GAN) and autoencoder. Conditional generative adversarial networks train two models against each other to compute larger simulated datasets that are representative of the original dataset. Autoencoder maps the original and the generated samples onto a common subspace to make the prediction more accurate. RESULTS: Extensive evaluation and predictive analysis was conducted on two datasets, T2D study and Cirrhosis study showing an improvement in mean AUC using data augmentation by 11% and 5% respectively. External validation on a cohort classifying between obese and lean subjects, with a smaller sample size provided an improvement in mean AUC close to 32% when augmented through phylaGAN as compared to using the original cohort. Our findings not only indicate that the generative adversarial networks can create samples that mimic the original data across various diversity metrics, but also highlight the potential of enhancing disease prediction through machine learning models trained on synthetic data. AVAILABILITY AND IMPLEMENTATION: https://github.com/divya031090/phylaGAN.