Meta-análisis: Intervalos de confianza e Intervalos de predicción / Meta-analysis: Confidence intervals and Prediction intervals

En los informes meta-analíticos se suelen reportar varios tipos de intervalos, hecho que ha generado cierta confusión a la hora de interpretarlos. Los intervalos de confianza reflejan la incertidumbre relacionada con un número, el tamaño del efecto medio paramétrico. Los intervalos de predicción reflejan el tamaño paramétrico probable en cualquier estudio de la misma clase que los incluidos en un meta-análisis. Su interpretación y aplicaciones son diferentes. En este artículo explicamos su diferente naturaleza y cómo se pueden utilizar para responder preguntas específicas. Se incluyen ejemplos numéricos, así como su cálculo con el paquete metafor en R.(AU)

Several types of intervals are usually employed in meta-analysis, a fact that has generated some confusion when interpreting them. Confidence intervals reflect the uncertainty related to a single number, the parametric mean effect size. Prediction intervals reflect the probable parametric effect size in any study of the same class as those included in a meta-analysis. Its interpretation and applications are different. In this article we explain in de-tail their different nature and how they can be used to answer specific ques-tions. Numerical examples are included, as well as their computation with the metafor Rpackage.(AU)

A guide to interpreting systematic reviews and meta-analyses in neurosurgery and surgery.

INTRODUCTION: Systematic reviews (SRs) and meta-analyses (MAs) are methods of data analysis used to synthesize information presented in multiple publications on the same topic. A thorough understanding of the steps involved in conducting this type of research and approaches to data analysis is critical for appropriate understanding, interpretation, and application of the findings of these reviews. METHODS: We reviewed reference texts in clinical neuroepidemiology, neurostatistics and research methods and other previously related articles on meta-analyses (MAs) in surgery. Based on existing theories and models and our cumulative years of expertise in conducting MAs, we have synthesized and presented a detailed pragmatic approach to interpreting MAs in Neurosurgery. RESULTS: Herein we have briefly defined SRs sand MAs and related terminologies, succinctly outlined the essential steps to conduct and critically appraise SRs and MAs. A practical approach to interpreting MAs for neurosurgeons is described in details. Based on summary outcome measures, we have used hypothetical examples to illustrate the Interpretation of the three commonest types of MAs in neurosurgery: MAs of Binary Outcome Measures (Pairwise MAs), MAs of proportions and MAs of Continuous Variables. Furthermore, we have elucidated on the concepts of heterogeneity, modeling, certainty, and bias essential for the robust and transparent interpretation of MAs. The basics for the Interpretation of Forest plots, the preferred graphical display of data in MAs are summarized. Additionally, a condensation of the assessment of the overall quality of methodology and reporting of MA and the applicability of evidence to patient care is presented. CONCLUSION: There is a paucity of pragmatic guides to appraise MAs for surgeons who are non-statisticians. This article serves as a detailed guide for the interpretation of systematic reviews and meta-analyses with examples of applications for clinical neurosurgeons.

Sampling weighting strategies in causal mediation analysis.

BACKGROUND: Causal mediation analysis plays a crucial role in examining causal effects and causal mechanisms. Yet, limited work has taken into consideration the use of sampling weights in causal mediation analysis. In this study, we compared different strategies of incorporating sampling weights into causal mediation analysis. METHODS: We conducted a simulation study to assess 4 different sampling weighting strategies-1) not using sampling weights, 2) incorporating sampling weights into mediation "cross-world" weights, 3) using sampling weights when estimating the outcome model, and 4) using sampling weights in both stages. We generated 8 simulated population scenarios comprising an exposure (A), an outcome (Y), a mediator (M), and six covariates (C), all of which were binary. The data were generated so that the true model of A given C and the true model of A given M and C were both logit models. We crossed these 8 population scenarios with 4 different sampling methods to obtain 32 total simulation conditions. For each simulation condition, we assessed the performance of 4 sampling weighting strategies when calculating sample-based estimates of the total, direct, and indirect effects. We also applied the four sampling weighting strategies to a case study using data from the National Survey on Drug Use and Health (NSDUH). RESULTS: Using sampling weights in both stages (mediation weight estimation and outcome models) had the lowest bias under most simulation conditions examined. Using sampling weights in only one stage led to greater bias for multiple simulation conditions. DISCUSSION: Using sampling weights in both stages is an effective approach to reduce bias in causal mediation analyses under a variety of conditions regarding the structure of the population data and sampling methods.

Survival analysis for AdVerse events with VarYing follow-up times (SAVVY): summary of findings and assessment of existing guidelines.

BACKGROUND: The SAVVY project aims to improve the analyses of adverse events (AEs) in clinical trials through the use of survival techniques appropriately dealing with varying follow-up times and competing events (CEs). This paper summarizes key features and conclusions from the various SAVVY papers. METHODS: Summarizing several papers reporting theoretical investigations using simulations and an empirical study including randomized clinical trials from several sponsor organizations, biases from ignoring varying follow-up times or CEs are investigated. The bias of commonly used estimators of the absolute (incidence proportion and one minus Kaplan-Meier) and relative (risk and hazard ratio) AE risk is quantified. Furthermore, we provide a cursory assessment of how pertinent guidelines for the analysis of safety data deal with the features of varying follow-up time and CEs. RESULTS: SAVVY finds that for both, avoiding bias and categorization of evidence with respect to treatment effect on AE risk into categories, the choice of the estimator is key and more important than features of the underlying data such as percentage of censoring, CEs, amount of follow-up, or value of the gold-standard. CONCLUSIONS: The choice of the estimator of the cumulative AE probability and the definition of CEs are crucial. Whenever varying follow-up times and/or CEs are present in the assessment of AEs, SAVVY recommends using the Aalen-Johansen estimator (AJE) with an appropriate definition of CEs to quantify AE risk. There is an urgent need to improve pertinent clinical trial reporting guidelines for reporting AEs so that incidence proportions or one minus Kaplan-Meier estimators are finally replaced by the AJE with appropriate definition of CEs.

Statistical analysis plan for the TRANSLATE (TRANSrectal biopsy versus Local Anaesthetic Transperineal biopsy Evaluation of potentially clinically significant prostate cancer) multicentre randomised controlled trial.

BACKGROUND: The TRANSLATE (TRANSrectal biopsy versus Local Anaesthetic Transperineal biopsy Evaluation) trial assesses the clinical and cost-effectiveness of two biopsy procedures in terms of detection of clinically significant prostate cancer (PCa). This article describes the statistical analysis plan (SAP) for the TRANSLATE randomised controlled trial (RCT). METHODS/DESIGN: TRANSLATE is a parallel, superiority, multicentre RCT. Biopsy-naïve men aged ≥ 18 years requiring a prostate biopsy for suspicion of possible PCa are randomised (computer-generated 1:1 allocation ratio) to one of two biopsy procedures: transrectal (TRUS) or local anaesthetic transperineal (LATP) biopsy. The primary outcome is the difference in detection rates of clinically significant PCa (defined as Gleason Grade Group ≥ 2, i.e. any Gleason pattern ≥ 4 disease) between the two biopsy procedures. Secondary outcome measures are th eProBE questionnaire (Perception Part and General Symptoms) and International Index of Erectile Function (IIEF, Domain A) scores, International Prostate Symptom Score (IPSS) values, EQ-5D-5L scores, resource use, infection rates, complications, and serious adverse events. We describe in detail the sample size calculation, statistical models used for the analysis, handling of missing data, and planned sensitivity and subgroup analyses. This SAP was pre-specified, written and submitted without prior knowledge of the trial results. DISCUSSION: Publication of the TRANSLATE trial SAP aims to increase the transparency of the data analysis and reduce the risk of outcome reporting bias. Any deviations from the current SAP will be described and justified in the final study report and results publication. TRIAL REGISTRATION: International Standard Randomised Controlled Trial Number ISRCTN98159689, registered on 28 January 2021 and registered on the ClinicalTrials.gov (NCT05179694) trials registry.

StatiCAL: an interactive tool for statistical analysis of biomedical data and scientific valorization.

BACKGROUND: In the realm of biomedical research, the growing volume, diversity and quantity of data has escalated the demand for statistical analysis as it is indispensable for synthesizing, interpreting, and publishing data. Hence the need for accessible analysis tools drastically increased. StatiCAL emerges as a user-friendly solution, enabling researchers to conduct basic analyses without necessitating extensive programming expertise. RESULTS: StatiCAL includes divers functionalities: data management, visualization on variables and statistical analysis. Data management functionalities allow the user to freely add or remove variables, to select sub-population and to visualise selected data to better perform the analysis. With this tool, users can freely perform statistical analysis such as descriptive, graphical, univariate, and multivariate analysis. All of this can be performed without the need to learn R coding as the software is a graphical user interface where all the action can be performed by clicking a button. CONCLUSIONS: StatiCAL represents a valuable contribution to the field of biomedical research. By being open-access and by providing an intuitive interface with robust features, StatiCAL allow researchers to gain autonomy in conducting their projects.

Alternatives to the P value: connotations of significance.

The statistical significance of a clinical trial analysis result is determined by a mathematical calculation and probability based on null hypothesis significance testing. However, statistical significance does not always align with meaningful clinical effects; thus, assigning clinical relevance to statistical significance is unreasonable. A statistical result incorporating a clinically meaningful difference is a better approach to present statistical significance. Thus, the minimal clinically important difference (MCID), which requires integrating minimum clinically relevant changes from the early stages of research design, has been introduced. As a follow-up to the previous statistical round article on P values, confidence intervals, and effect sizes, in this article, we present hands-on examples of MCID and various effect sizes and discuss the terms statistical significance and clinical relevance, including cautions regarding their use.

The impact of different imputation methods on estimates and model performance: an example using a risk prediction model for premature mortality.

OBJECTIVE: To compare how different imputation methods affect the estimates and performance of a prediction model for premature mortality. STUDY DESIGN AND SETTING: Sex-specific Weibull accelerated failure time survival models were run on four separate datasets using complete case, mode, single and multiple imputation to impute missing values. Six performance measures were compared to access predictive accuracy (Nagelkerke R2, integrated brier score), discrimination (Harrell's c-index, discrimination slope) and calibration (calibration in the large, calibration slope). RESULTS: The highest proportion of missingness for a single variable was 10.86% for the female model and 8.24% for the male model. Comparing the performance measures for complete case, mode, single and multiple imputation: the Nagelkerke R2 values for the female model was 0.1084, 0.1116, 0.1120 and 0.111-0.1120 with the male model exhibited similar variation of 0.1050, 0.1078, 0.1078 and 0.1078-0.1081. Harrell's c-index also demonstrated small variation with values of 0.8666, 0.8719, 0.8719 and 0.8711-0.8719 for the female model and 0.8549, 0.8548, 0.8550 and 0.8550-0.8553 for the male model. CONCLUSION: In the scenarios examined in this study, mode imputation performed well when using a population health survey compared to single and multiple imputation when predictive performance measures is the main model goal. To generate unbiased hazard ratios, multiple imputation methods were superior. This study shows the need to consider the best imputation approach for a predictive model development given the conditions of missing data and the goals of the analysis.

Accounting for center-level effects in multicenter randomized controlled trials.

Investigators often conduct randomized controlled trials (RCTs) at multiple centers/sites when determining the effect of a treatment or an intervention. Diversifying recruitment across multiple institutions allows investigators to make recruitment go faster within a shorter timeframe and allows generalizing the study results across diverse populations. Despite having a common study protocol across multiple centers, the eligible participants may be heterogeneous, site policies and practices may vary, and the investigators' experience, training, and expertise may also vary across sites. These factors may contribute to the heterogeneity in effect estimates across centers. As a result, we usually observe some degree of heterogeneity in effect estimates across centers, despite all centers following the same study protocol. During the analysis of such a trial, investigators typically ignore center effects, but some have suggested considering centers as fixed or random effects in the model. It is not clear how considering the effects of centers, either as fixed or random effects, impacts the test of the primary hypothesis. In this article, we first review the practice of accounting for center effects in the analyses of published RCTs and illustrate the extent of heterogeneity observed in a few preexisting multicenter RCTs. To determine the impact of heterogeneity on the test of a primary hypothesis of an RCT, we considered continuous and binary outcomes and the corresponding appropriate model, namely, a simple linear regression model for a continuous outcome and a logistic regression model for the binary outcome. For each model type, we considered three methods: (a) ignore the center effect, (b) account for centers as fixed effects, or (c) account for centers as random effects. Based on simulation studies of these models, we then examine whether considering the center as a fixed or random effect in the model helps to preserve or reduce the type I and type II error rates during the analysis phase of an RCT. Finally, we outline the threshold at which center-level effects are negligible and thus negligible and provide recommendations on when it may be necessary to account for center effects during the analyses of multicenter randomized controlled trials.

[How to interpret subgroup analyses in cardiovascular trials?] / Come leggere le analisi per sottogruppi nei trial cardiovascolari?

Clinical trials provide the best evidence on the effect of a treatment, but they evaluate this effect on the total population of the study as if the effect of the randomized treatment was identical in all possible subgroups of patients (young, elderly, male, female, etc.). Subgroup analyses are an important tool to evaluate the presence of any diversity of the treatment effect concerning specific patient characteristics, if there are practical questions about who to treat and when, or if there are doubts about the benefit/risk profile of a therapy in a specific subpopulation. Subgroup analyses should be defined a priori, biologically plausible, and limited to few clinically important questions. Subgroup analyses have greater relevance in the context of studies that have demonstrated an overall significant difference between treatments. In the case of neutral or negative studies, any significant analyses between subgroups should be considered as essentially exploratory. Post-hoc subgroup analyses should be treated with great caution and considered more credible as the results are consistent with other studies. If significant heterogeneity is expected in specific subgroups of patients when planning a trial, they should have sufficient statistical power to detect the difference in the effect. In this review, we propose a critical approach for interpreting subgroup analyses in cardiovascular trials.

On exact randomization-based covariate-adjusted confidence intervals.

Randomization-based inference using the Fisher randomization test allows for the computation of Fisher-exact P-values, making it an attractive option for the analysis of small, randomized experiments with non-normal outcomes. Two common test statistics used to perform Fisher randomization tests are the difference-in-means between the treatment and control groups and the covariate-adjusted version of the difference-in-means using analysis of covariance. Modern computing allows for fast computation of the Fisher-exact P-value, but confidence intervals have typically been obtained by inverting the Fisher randomization test over a range of possible effect sizes. The test inversion procedure is computationally expensive, limiting the usage of randomization-based inference in applied work. A recent paper by Zhu and Liu developed a closed form expression for the randomization-based confidence interval using the difference-in-means statistic. We develop an important extension of Zhu and Liu to obtain a closed form expression for the randomization-based covariate-adjusted confidence interval and give practitioners a sufficiency condition that can be checked using observed data and that guarantees that these confidence intervals have correct coverage. Simulations show that our procedure generates randomization-based covariate-adjusted confidence intervals that are robust to non-normality and that can be calculated in nearly the same time as it takes to calculate the Fisher-exact P-value, thus removing the computational barrier to performing randomization-based inference when adjusting for covariates. We also demonstrate our method on a re-analysis of phase I clinical trial data.

Incorporating nonparametric methods for estimating causal excursion effects in mobile health with zero-inflated count outcomes.

In mobile health, tailoring interventions for real-time delivery is of paramount importance. Micro-randomized trials have emerged as the "gold-standard" methodology for developing such interventions. Analyzing data from these trials provides insights into the efficacy of interventions and the potential moderation by specific covariates. The "causal excursion effect," a novel class of causal estimand, addresses these inquiries. Yet, existing research mainly focuses on continuous or binary data, leaving count data largely unexplored. The current work is motivated by the Drink Less micro-randomized trial from the UK, which focuses on a zero-inflated proximal outcome, i.e., the number of screen views in the subsequent hour following the intervention decision point. To be specific, we revisit the concept of causal excursion effect, specifically for zero-inflated count outcomes, and introduce novel estimation approaches that incorporate nonparametric techniques. Bidirectional asymptotics are established for the proposed estimators. Simulation studies are conducted to evaluate the performance of the proposed methods. As an illustration, we also implement these methods to the Drink Less trial data.

Using Bayesian statistics in confirmatory clinical trials in the regulatory setting: a tutorial review.

Bayesian statistics plays a pivotal role in advancing medical science by enabling healthcare companies, regulators, and stakeholders to assess the safety and efficacy of new treatments, interventions, and medical procedures. The Bayesian framework offers a unique advantage over the classical framework, especially when incorporating prior information into a new trial with quality external data, such as historical data or another source of co-data. In recent years, there has been a significant increase in regulatory submissions using Bayesian statistics due to its flexibility and ability to provide valuable insights for decision-making, addressing the modern complexity of clinical trials where frequentist trials are inadequate. For regulatory submissions, companies often need to consider the frequentist operating characteristics of the Bayesian analysis strategy, regardless of the design complexity. In particular, the focus is on the frequentist type I error rate and power for all realistic alternatives. This tutorial review aims to provide a comprehensive overview of the use of Bayesian statistics in sample size determination, control of type I error rate, multiplicity adjustments, external data borrowing, etc., in the regulatory environment of clinical trials. Fundamental concepts of Bayesian sample size determination and illustrative examples are provided to serve as a valuable resource for researchers, clinicians, and statisticians seeking to develop more complex and innovative designs.

Statistical analysis plan for the replacing protein via enteral nutrition in a stepwise approach in critically ill patients (REPLENISH) randomized clinical trial.

BACKGROUND: The optimal amount and timing of protein intake in critically ill patients are unknown. REPLENISH (Replacing Protein via Enteral Nutrition in a Stepwise Approach in Critically Ill Patients) trial evaluates whether supplemental enteral protein added to standard enteral nutrition to achieve a high amount of enteral protein given from ICU day five until ICU discharge or ICU day 90 as compared to no supplemental enteral protein to achieve a moderate amount of enteral protein would reduce all-cause 90-day mortality in adult critically ill mechanically ventilated patients. METHODS: In this multicenter randomized trial, critically ill patients will be randomized to receive supplemental enteral protein (1.2 g/kg/day) added to standard enteral nutrition to achieve a high amount of enteral protein (range of 2-2.4 g/kg/day) or no supplemental enteral protein to achieve a moderate amount of enteral protein (0.8-1.2 g/kg/day). The primary outcome is 90-day all-cause mortality; other outcomes include functional and health-related quality-of-life assessments at 90 days. The study sample size of 2502 patients will have 80% power to detect a 5% absolute risk reduction in 90-day mortality from 30 to 25%. Consistent with international guidelines, this statistical analysis plan specifies the methods for evaluating primary and secondary outcomes and subgroups. Applying this statistical analysis plan to the REPLENISH trial will facilitate unbiased analyses of clinical data. CONCLUSION: Ethics approval was obtained from the institutional review board, Ministry of National Guard Health Affairs, Riyadh, Saudi Arabia (RC19/414/R). Approvals were also obtained from the institutional review boards of each participating institution. Our findings will be disseminated in an international peer-reviewed journal and presented at relevant conferences and meetings. TRIAL REGISTRATION: ClinicalTrials.gov, NCT04475666 . Registered on July 17, 2020.

Improved clinical data imputation via classical and quantum determinantal point processes.

Imputing data is a critical issue for machine learning practitioners, including in the life sciences domain, where missing clinical data is a typical situation and the reliability of the imputation is of great importance. Currently, there is no canonical approach for imputation of clinical data and widely used algorithms introduce variance in the downstream classification. Here we propose novel imputation methods based on determinantal point processes (DPP) that enhance popular techniques such as the multivariate imputation by chained equations and MissForest. Their advantages are twofold: improving the quality of the imputed data demonstrated by increased accuracy of the downstream classification and providing deterministic and reliable imputations that remove the variance from the classification results. We experimentally demonstrate the advantages of our methods by performing extensive imputations on synthetic and real clinical data. We also perform quantum hardware experiments by applying the quantum circuits for DPP sampling since such quantum algorithms provide a computational advantage with respect to classical ones. We demonstrate competitive results with up to 10 qubits for small-scale imputation tasks on a state-of-the-art IBM quantum processor. Our classical and quantum methods improve the effectiveness and robustness of clinical data prediction modeling by providing better and more reliable data imputations. These improvements can add significant value in settings demanding high precision, such as in pharmaceutical drug trials where our approach can provide higher confidence in the predictions made.

Efficient data integration under prior probability shift.

Conventional supervised learning usually operates under the premise that data are collected from the same underlying population. However, challenges may arise when integrating new data from different populations, resulting in a phenomenon known as dataset shift. This paper focuses on prior probability shift, where the distribution of the outcome varies across datasets but the conditional distribution of features given the outcome remains the same. To tackle the challenges posed by such shift, we propose an estimation algorithm that can efficiently combine information from multiple sources. Unlike existing methods that are restricted to discrete outcomes, the proposed approach accommodates both discrete and continuous outcomes. It also handles high-dimensional covariate vectors through variable selection using an adaptive least absolute shrinkage and selection operator penalty, producing efficient estimates that possess the oracle property. Moreover, a novel semiparametric likelihood ratio test is proposed to check the validity of prior probability shift assumptions by embedding the null conditional density function into Neyman's smooth alternatives (Neyman, 1937) and testing study-specific parameters. We demonstrate the effectiveness of our proposed method through extensive simulations and a real data example. The proposed methods serve as a useful addition to the repertoire of tools for dealing dataset shifts.

Sequential covariate-adjusted randomization via hierarchically minimizing Mahalanobis distance and marginal imbalance.

In comparative studies, covariate balance and sequential allocation schemes have attracted growing academic interest. Although many theoretically justified adaptive randomization methods achieve the covariate balance, they often allocate patients in pairs or groups. To better meet the practical requirements where the clinicians cannot wait for other participants to assign the current patient for some economic or ethical reasons, we propose a method that randomizes patients individually and sequentially. The proposed method conceptually separates the covariate imbalance, measured by the newly proposed modified Mahalanobis distance, and the marginal imbalance, that is the sample size difference between the 2 groups, and it minimizes them with an explicit priority order. Compared with the existing sequential randomization methods, the proposed method achieves the best possible covariate balance while maintaining the marginal balance directly, offering us more control of the randomization process. We demonstrate the superior performance of the proposed method through a wide range of simulation studies and real data analysis, and also establish theoretical guarantees for the proposed method in terms of both the convergence of the imbalance measure and the subsequent treatment effect estimation.

The left atrial appendage closure by surgery 2 trial: statistical analysis plan for a randomized multicenter trial exploring if the closure of the left atrial appendage during open-heart surgery reduces stroke irrespective of patients' stroke risk and preoperative atrial fibrillation status.

BACKGROUND: Surgical left atrial appendage (LAA) closure concomitant to open-heart surgery prevents thromboembolism in high-risk patients. Nevertheless, high-level evidence does not exist for LAA closure performed in patients with any CHA2DS2-VASc score and preoperative atrial fibrillation or flutter (AF) status-the current trial attempts to provide such evidence. METHODS: The study is designed as a randomized, open-label, blinded outcome assessor, multicenter trial of adult patients undergoing first-time elective open-heart surgery. Patients with and without AF and any CHA2DS2-VASc score will be enrolled. The primary exclusion criteria are planned LAA closure, planned AF ablation, or ongoing endocarditis. Before randomization, a three-step stratification process will sort patients by site, surgery type, and preoperative or expected oral anticoagulation treatment. Patients will undergo balanced randomization (1:1) to LAA closure on top of the planned cardiac surgery or standard care. Block sizes vary from 8 to 16. Neurologists blinded to randomization will adjudicate the primary outcome of stroke, including transient ischemic attack (TIA). The secondary outcomes include a composite outcome of stroke, including TIA, and silent cerebral infarcts, an outcome of ischemic stroke, including TIA, and a composite outcome of stroke and all-cause mortality. LAA closure is expected to provide a 60% relative risk reduction. In total, 1500 patients will be randomized and followed for 2 years. DISCUSSION: The trial is expected to help form future guidelines within surgical LAA closure. This statistical analysis plan ensures transparency of analyses and limits potential reporting biases. TRIAL REGISTRATION: Clinicaltrials.gov, NCT03724318. Registered 26 October 2018, https://clinicaltrials.gov/study/NCT03724318 . PROTOCOL VERSION: https://doi.org/10.1016/j.ahj.2023.06.003 .