Application of causal inference methods in individual-participant data meta-analyses in medicine: addressing data handling and reporting gaps with new proposed reporting guidelines.

Observational data provide invaluable real-world information in medicine, but certain methodological considerations are required to derive causal estimates. In this systematic review, we evaluated the methodology and reporting quality of individual-level patient data meta-analyses (IPD-MAs) conducted with non-randomized exposures, published in 2009, 2014, and 2019 that sought to estimate a causal relationship in medicine. We screened over 16,000 titles and abstracts, reviewed 45 full-text articles out of the 167 deemed potentially eligible, and included 29 into the analysis. Unfortunately, we found that causal methodologies were rarely implemented, and reporting was generally poor across studies. Specifically, only three of the 29 articles used quasi-experimental methods, and no study used G-methods to adjust for time-varying confounding. To address these issues, we propose stronger collaborations between physicians and methodologists to ensure that causal methodologies are properly implemented in IPD-MAs. In addition, we put forward a suggested checklist of reporting guidelines for IPD-MAs that utilize causal methods. This checklist could improve reporting thereby potentially enhancing the quality and trustworthiness of IPD-MAs, which can be considered one of the most valuable sources of evidence for health policy.

The Bayes factor, HDI-ROPE, and frequentist equivalence tests can all be reverse engineered-Almost exactly-From one another: Reply to Linde et al. (2021).

Following an extensive simulation study comparing the operating characteristics of three different procedures used for establishing equivalence (the frequentist "TOST," the Bayesian "HDI-ROPE," and the Bayes factor interval null procedure), Linde et al. (2021) conclude with the recommendation that "researchers rely more on the Bayes factor interval null approach for quantifying evidence for equivalence" (p. 1). We redo the simulation study of Linde et al. (2021) in its entirety but with the different procedures calibrated to have the same predetermined maximum Type I error rate. Our results suggest that, when calibrated in this way, the Bayes factor, HDI-ROPE, and frequentist equivalence tests all have similar-almost exactly-Type II error rates. In general any advocating for frequentist testing as better or worse than Bayesian testing in terms of empirical findings seems dubious at best. If one decides on which underlying principle to subscribe to in tackling a given problem, then the method follows naturally. Bearing in mind that each procedure can be reverse-engineered from the others (at least approximately), trying to use empirical performance to argue for 1 approach over another seems like tilting at windmills. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

A Bayesian approach to estimating COVID-19 incidence and infection fatality rates.

Naive estimates of incidence and infection fatality rates (IFR) of coronavirus disease 2019 suffer from a variety of biases, many of which relate to preferential testing. This has motivated epidemiologists from around the globe to conduct serosurveys that measure the immunity of individuals by testing for the presence of SARS-CoV-2 antibodies in the blood. These quantitative measures (titer values) are then used as a proxy for previous or current infection. However, statistical methods that use this data to its full potential have yet to be developed. Previous researchers have discretized these continuous values, discarding potentially useful information. In this article, we demonstrate how multivariate mixture models can be used in combination with post-stratification to estimate cumulative incidence and IFR in an approximate Bayesian framework without discretization. In doing so, we account for uncertainty from both the estimated number of infections and incomplete deaths data to provide estimates of IFR. This method is demonstrated using data from the Action to Beat Coronavirus erosurvey in Canada.

Equivalence testing for linear regression.

We introduce equivalence testing procedures for linear regression analyses. Such tests can be very useful for confirming the lack of a meaningful association between a continuous outcome and a continuous or binary predictor. Specifically, we propose an equivalence test for unstandardized regression coefficients and an equivalence test for semipartial correlation coefficients. We review how to define valid hypotheses, how to calculate p values, and how these tests compare to an alternative Bayesian approach with applications to examples in the literature. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Unanchored Population-Adjusted Indirect Comparison Methods for Time-to-Event Outcomes Using Inverse Odds Weighting, Regression Adjustment, and Doubly Robust Methods With Either Individual Patient or Aggregate Data.

OBJECTIVES: Several methods for unanchored population-adjusted indirect comparisons (PAICs) are available. Exploring alternative adjustment methods, depending on the available individual patient data (IPD) and the aggregate data (AD) in the external study, may help minimize bias in unanchored indirect comparisons. However, methods for time-to-event outcomes are not well understood. This study provides an overview and comparison of methods using a case study to increase familiarity. A recent method is applied to marginalize conditional hazard ratios, which allows for the comparisons of methods, and a doubly robust method is proposed. METHODS: The following PAIC methods were compared through a case study in third-line small cell lung cancer, comparing nivolumab with standard of care based on a single-arm phase II trial (CheckMate 032) and real-world study (Flatiron) in terms of overall survival: IPD-IPD analyses using inverse odds weighting, regression adjustment, and a doubly robust method; IPD-AD analyses using matching-adjusted indirect comparison, simulated treatment comparison, and a doubly robust method. RESULTS: Nivolumab extended survival versus standard of care with hazard ratios ranging from 0.63 (95% CI 0.44-0.90) in naive comparisons (identical estimates for IPD-IPD and IPD-AD analyses) to 0.69 (95% CI 0.44-0.98) in the IPD-IPD analyses using regression adjustment. Regression-based and doubly robust estimates yielded slightly wider confidence intervals versus the propensity score-based analyses. CONCLUSIONS: The proposed doubly robust approach for time-to-event outcomes may help to minimize bias due to model misspecification. However, all methods for unanchored PAIC rely on the strong assumption that all prognostic covariates have been included.

Application of Causal Inference Methods to Pooled Longitudinal Non- Randomized Studies: A Methodological Systematic Review.

Observational data provide invaluable real-world information in medicine, but certain methodological considerations are required to derive causal estimates. In this systematic review, we evaluated the methodology and reporting quality of individual-level patient data meta-analyses (IPD-MAs) published in 2009, 2014, and 2019 that sought to estimate a causal relationship in medicine. We screened over 16,000 titles and abstracts, reviewed 45 full-text articles out of the 167 deemed potentially eligible, and included 29 into the analysis. Unfortunately, we found that causal methodologies were rarely implemented, and reporting was generally poor across studies. Specifically, only three of the 29 articles used quasi-experimental methods, and no study used G-methods to adjust for time-varying confounding. To address these issues, we propose stronger collaborations between physicians and methodologists to ensure that causal methodologies are properly implemented in IPD-MAs. In addition, we put forward a suggested checklist of reporting guidelines for IPD-MAs that utilize causal methods. This checklist could improve reporting thereby potentially enhancing the quality and trustworthiness of IPD-MAs, which can be considered one of the most valuable sources of evidence for health policy.

A Comparison of Alternative Network Meta-Analysis Methods in the Presence of Nonproportional Hazards: A Case Study in First-Line Advanced or Metastatic Renal Cell Carcinoma.

OBJECTIVES: Network meta-analysis (NMA) of time-to-event outcomes based on constant hazard ratios can result in biased findings when the proportional hazards (PHs) assumption does not hold in a subset of trials. We aimed to summarize the published non-PH NMA methods for time-to-event outcomes, demonstrate their application, and compare their results. METHODS: The following non-PH NMA methods were compared through an illustrative case study in oncology of 4 randomized controlled trials in terms of progression-free survival and overall survival: (1) 1-step or (2) 2-step multivariate NMAs based on traditional survival distributions or fractional polynomials, (3) NMAs with restricted cubic splines for baseline hazard, and (4) restricted mean survival NMA. RESULTS: For progression-free survival, the PH assumption did not hold across trials and non-PH NMA methods better reflected the relative treatment effects over time. The most flexible models (fractional polynomials and restricted cubic splines) fit better to the data than the other approaches. Estimated hazard ratios obtained with different non-PH NMA methods were similar at 5 years of follow-up but differed thereafter in the extrapolations. Although there was no strong evidence of PH violation for overall survival, non-PH NMA methods captured this uncertainty in the relative treatment effects over time. CONCLUSIONS: When the PH assumption is questionable in a subset of the randomized controlled trials, we recommend assessing alternative non-PH NMA methods to estimate relative treatment effects for time-to-event outcomes. We propose a transparent and explicit stepwise model selection process considering model fit, external constraints, and clinical validity. Given inherent uncertainty, sensitivity analyses are suggested.

Limitations introduced by a low participation rate of SARS-CoV-2 seroprevalence data.

BACKGROUND: There has been a large influx of COVID-19 seroprevalence studies, but comparability between the seroprevalence estimates has been an issue because of heterogeneities in testing platforms and study methodology. One potential source of heterogeneity is the response or participation rate. METHODS: We conducted a review of participation rates (PR) in SARS-CoV-2 seroprevalence studies collected by SeroTracker and examined their effect on the validity of study conclusions. PR was calculated as the count of participants for whom the investigators had collected a valid sample, divided by the number of people invited to participate in the study. A multivariable beta generalized linear model with logit link was fitted to determine if the PR of international household and community-based seroprevalence studies was associated with the factors of interest, from 1 December 2019 to 10 March 2021. RESULTS: We identified 90 papers based on screening and were able to calculate the PR for 35 out of 90 papers (39%), with a median PR of 70% and an interquartile range of 40.92; 61% of the studies did not report PR. CONCLUSIONS: Many SARS-CoV-2 seroprevalence studies do not report PR. It is unclear what the median PR rate would be had a larger portion not had limitations in reporting. Low participation rates indicate limited representativeness of results. Non-probabilistic sampling frames were associated with higher participation rates but may be less representative. Standardized definitions of participation rate and data reporting necessary for the PR calculations are essential for understanding the representativeness of seroprevalence estimates in the population of interest.

Adjusting for misclassification of an exposure in an individual participant data meta-analysis.

A common problem in the analysis of multiple data sources, including individual participant data meta-analysis (IPD-MA), is the misclassification of binary variables. Misclassification may lead to biased estimators of model parameters, even when the misclassification is entirely random. We aimed to develop statistical methods that facilitate unbiased estimation of adjusted and unadjusted exposure-outcome associations and between-study heterogeneity in IPD-MA, where the extent and nature of exposure misclassification may vary across studies. We present Bayesian methods that allow misclassification of binary exposure variables to depend on study- and participant-level characteristics. In an example of the differential diagnosis of dengue using two variables, where the gold standard measurement for the exposure variable was unavailable for some studies which only measured a surrogate prone to misclassification, our methods yielded more accurate estimates than analyses naive with regard to misclassification or based on gold standard measurements alone. In a simulation study, the evaluated misclassification model yielded valid estimates of the exposure-outcome association, and was more accurate than analyses restricted to gold standard measurements. Our proposed framework can appropriately account for the presence of binary exposure misclassification in IPD-MA. It requires that some studies supply IPD for the surrogate and gold standard exposure, and allows misclassification to follow a random effects distribution across studies conditional on observed covariates (and outcome). The proposed methods are most beneficial when few large studies that measured the gold standard are available, and when misclassification is frequent.

Systematic Review Reveals Lack of Causal Methodology Applied to Pooled Longitudinal Observational Infectious Disease Studies.

OBJECTIVES: Among ID studies seeking to make causal inferences and pooling individual-level longitudinal data from multiple infectious disease cohorts, we sought to assess what methods are being used, how those methods are being reported, and whether these factors have changed over time. STUDY DESIGN AND SETTING: Systematic review of longitudinal observational infectious disease studies pooling individual-level patient data from 2+ studies published in English in 2009, 2014, or 2019. This systematic review protocol is registered with PROSPERO (CRD42020204104). RESULTS: Our search yielded 1,462 unique articles. Of these, 16 were included in the final review. Our analysis showed a lack of causal inference methods and of clear reporting on methods and the required assumptions. CONCLUSION: There are many approaches to causal inference which may help facilitate accurate inference in the presence of unmeasured and time-varying confounding. In observational ID studies leveraging pooled, longitudinal IPD, the absence of these causal inference methods and gaps in the reporting of key methodological considerations suggests there is ample opportunity to enhance the rigor and reporting of research in this field. Interdisciplinary collaborations between substantive and methodological experts would strengthen future work.

Current trends in the application of causal inference methods to pooled longitudinal non-randomised data: a protocol for a methodological systematic review.

INTRODUCTION: Causal methods have been adopted and adapted across health disciplines, particularly for the analysis of single studies. However, the sample sizes necessary to best inform decision-making are often not attainable with single studies, making pooled individual-level data analysis invaluable for public health efforts. Researchers commonly implement causal methods prevailing in their home disciplines, and how these are selected, evaluated, implemented and reported may vary widely. To our knowledge, no article has yet evaluated trends in the implementation and reporting of causal methods in studies leveraging individual-level data pooled from several studies. We undertake this review to uncover patterns in the implementation and reporting of causal methods used across disciplines in research focused on health outcomes. We will investigate variations in methods to infer causality used across disciplines, time and geography and identify gaps in reporting of methods to inform the development of reporting standards and the conversation required to effect change. METHODS AND ANALYSIS: We will search four databases (EBSCO, Embase, PubMed, Web of Science) using a search strategy developed with librarians from three universities (Heidelberg University, Harvard University, and University of California, San Francisco). The search strategy includes terms such as 'pool*', 'harmoniz*', 'cohort*', 'observational', variations on 'individual-level data'. Four reviewers will independently screen articles using Covidence and extract data from included articles. The extracted data will be analysed descriptively in tables and graphically to reveal the pattern in methods implementation and reporting. This protocol has been registered with PROSPERO (CRD42020143148). ETHICS AND DISSEMINATION: No ethical approval was required as only publicly available data were used. The results will be submitted as a manuscript to a peer-reviewed journal, disseminated in conferences if relevant, and published as part of doctoral dissertations in Global Health at the Heidelberg University Hospital.

Measurement error in meta-analysis (MEMA)-A Bayesian framework for continuous outcome data subject to non-differential measurement error.

Ideally, a meta-analysis will summarize data from several unbiased studies. Here we look into the less than ideal situation in which contributing studies may be compromised by non-differential measurement error in the exposure variable. Specifically, we consider a meta-analysis for the association between a continuous outcome variable and one or more continuous exposure variables, where the associations may be quantified as regression coefficients of a linear regression model. A flexible Bayesian framework is developed which allows one to obtain appropriate point and interval estimates with varying degrees of prior knowledge about the magnitude of the measurement error. We also demonstrate how, if individual-participant data (IPD) are available, the Bayesian meta-analysis model can adjust for multiple participant-level covariates, these being measured with or without measurement error.

Current trends in the application of causal inference methods to pooled longitudinal observational infectious disease studies-A protocol for a methodological systematic review.

INTRODUCTION: Pooling (or combining) and analysing observational, longitudinal data at the individual level facilitates inference through increased sample sizes, allowing for joint estimation of study- and individual-level exposure variables, and better enabling the assessment of rare exposures and diseases. Empirical studies leveraging such methods when randomization is unethical or impractical have grown in the health sciences in recent years. The adoption of so-called "causal" methods to account for both/either measured and/or unmeasured confounders is an important addition to the methodological toolkit for understanding the distribution, progression, and consequences of infectious diseases (IDs) and interventions on IDs. In the face of the Covid-19 pandemic and in the absence of systematic randomization of exposures or interventions, the value of these methods is even more apparent. Yet to our knowledge, no studies have assessed how causal methods involving pooling individual-level, observational, longitudinal data are being applied in ID-related research. In this systematic review, we assess how these methods are used and reported in ID-related research over the last 10 years. Findings will facilitate evaluation of trends of causal methods for ID research and lead to concrete recommendations for how to apply these methods where gaps in methodological rigor are identified. METHODS AND ANALYSIS: We will apply MeSH and text terms to identify relevant studies from EBSCO (Academic Search Complete, Business Source Premier, CINAHL, EconLit with Full Text, PsychINFO), EMBASE, PubMed, and Web of Science. Eligible studies are those that apply causal methods to account for confounding when assessing the effects of an intervention or exposure on an ID-related outcome using pooled, individual-level data from 2 or more longitudinal, observational studies. Titles, abstracts, and full-text articles, will be independently screened by two reviewers using Covidence software. Discrepancies will be resolved by a third reviewer. This systematic review protocol has been registered with PROSPERO (CRD42020204104).

Can we disregard the whole model? Omnibus non-inferiority testing for R2 in multi-variable linear regression and η̂2 in ANOVA.

Determining a lack of association between an outcome variable and a number of different explanatory variables is frequently necessary in order to disregard a proposed model (i.e., to confirm the lack of a meaningful association between an outcome and predictors). Despite this, the literature rarely offers information about, or technical recommendations concerning, the appropriate statistical methodology to be used to accomplish this task. This paper introduces non-inferiority tests for ANOVA and linear regression analyses, which correspond to the standard widely used F test for η̂2 and R2 , respectively. A simulation study is conducted to examine the Type I error rates and statistical power of the tests, and a comparison is made with an alternative Bayesian testing approach. The results indicate that the proposed non-inferiority test is a potentially useful tool for 'testing the null'.

Conditional equivalence testing: An alternative remedy for publication bias.

We introduce a publication policy that incorporates "conditional equivalence testing" (CET), a two-stage testing scheme in which standard NHST is followed conditionally by testing for equivalence. The idea of CET is carefully considered as it has the potential to address recent concerns about reproducibility and the limited publication of null results. In this paper we detail the implementation of CET, investigate similarities with a Bayesian testing scheme, and outline the basis for how a scientific journal could proceed to reduce publication bias while remaining relevant.

Comparison of Glycated Hemoglobin Results Based on At-Home and In-Lab Dried Blood Spot Sampling to Routine Venous Blood Sampling In-Lab in Adult Patients With Type 1 or Type 2 Diabetes.

OBJECTIVES: Regular measurement of glycated hemoglobin (A1C) is logistically demanding. Home blotter-paper collection offers an alternative. This study tested the viability of at-home blotter-paper A1C measurement. METHODS: Objective: compare accuracy of A1C levels collected on blotter paper at home (home-blotter) and blotter-paper collection in laboratory (lab-blotter) with venous A1C (routine measurement). Agreement was assessed by Pearson correlation, Lin concordance correlation coefficient (CCC), positive and negative predictive values (PPVs, NPVs) and Bland-Altman plots and associated statistics. RESULTS: Home-blotter, lab-blotter and venous A1C correlated strongly (0.93, 0.93). Home- and lab-blotter results were upwardly biased (0.387%, 0.1%). Bias increased with time. Bias correction provided agreement for both blotters (CCC >0.9); blotters correctly identifying levels above 7% (53 mmol/mol) were 100% for corrected home-blotters and 87% (95% confidence interval) for corrected lab-blotters. NPVs (% blotters correctly identifying levels of 7% or lower [53 mmol/mol]) were 100% for corrected home-blotters and 83% for corrected lab-blotters. After correction, >92% of corrected blotters had errors of 8% or less. Of our subjects, 88.5% found home sampling preferable to routine laboratory sampling. CONCLUSIONS: Home-blotter collection is an alternative to routine collection.

Sleep assessments for a mild traumatic brain injury trial in a military population.

Baseline sleep characteristics were explored for 71 U.S. military service members with mild traumatic brain injury (mTBI) enrolled in a post-concussive syndrome clinical trial. The Pittsburgh Sleep Quality Index (PSQI), sleep diary, several disorder-specific questionnaires, actigraphy and polysomnographic nap were collected. Almost all (97%) reported ongoing sleep problems. The mean global PSQI score was 13.5 (SD=3.8) and 87% met insomnia criteria. Sleep maintenance efficiency was 79.1% for PSQI, 82.7% for sleep diary and 90.5% for actigraphy; total sleep time was 288, 302 and 400 minutes, respectively. There was no correlation between actigraphy and subjective questionnaires. Overall, 70% met hypersomnia conditions, 70% were at high risk for obstructive sleep apnea (OSA), 32% were symptomatic for restless legs syndrome, and 6% reported cataplexy. Nearly half (44%) reported coexisting insomnia, hypersomnia and high OSA risk. Participants with post-traumatic stress disorder (PTSD) had higher PSQI scores and increased OSA risk. Older participants and those with higher aggression, anxiety or depression also had increased OSA risk. The results confirm poor sleep quality in mTBI with insomnia, hypersomnia, and OSA risk higher than previously reported, and imply sleep disorders in mTBI may be underdiagnosed or exacerbated by comorbid PTSD.

Impact of patient use of an online patient portal on diabetes outcomes.

OBJECTIVE: To assess the effect of patient use of an online patient portal on diabetes outcomes. METHODS: Patients included were those with diabetes who were newly referred to a Vancouver-based tertiary care diabetologist between April 2008 and October 2012. Each patient was assessed by the diabetologist, received initial diabetes education and was referred, as necessary, for further education and self-management training. All patients who provided an e-mail address at registration were invited to open an online patient portal account. The portal provided access to diabetes education material, personal laboratory values and a messaging system allowing communication with the diabetologist and staff. Patients who logged in 1 or more times were defined as portal users (n=50); patients who never logged in to the portal were defined as non-users (n=107). A1C was measured at 2 time points: at baseline (i.e. initial, in-clinic visit) and at last follow up (visit no less than 6 months and no more than 2 years after the initial visit). Because usership is self-selected, propensity score matching was used to create comparable user/non-user groups based on available baseline covariates. RESULTS: Compared to non-users, a higher proportion of users achieved A1C ≤7% at follow up (56% vs. 32%) (p=0.031). CONCLUSION: Accessing an online patient portal is associated with improved glycemic control.

An evaluation of the effectiveness of a commercial mechanical trap to reduce abundance of adult nuisance mosquito populations.

In this study, we explore the potential of a commercially available mechanical mosquito control device, the Liberty Plus Mosquito Magnet (hereafter referred to as Mosquito Magnet), to reduce the abundance of adult nuisance mosquito populations in public recreational areas. Mosquitoes were trapped on 2 replicate sites close to a campground at Brae Island Regional Park near Fort Langley, British Columbia, Canada. Each site comprised a treatment (Mosquito Magnets used) and control subsection (Mosquito Magnets not used). Mosquito numbers were assessed before and after the treatment period in both subsections at each site with Centers for Disease Control and Prevention (CDC) black light traps. Although nearly 200,000 mosquitoes from 14 different species were collected over 366 trap-nights from May 31 to July 31, 2008, the majority of those identified were Aedes sticticus (68%) and Ae. vexans (22%)-2 of the most notorious nuisance mosquito species in British Columbia. The number of mosquitoes captured by CDC black light traps increased overall during the study period due to natural seasonal variation. Nevertheless, a significant treatment effect (P = 0.0389) was associated with an average decrease of about 32% in the average number of adult mosquitoes collected per day. These results strongly suggest that Mosquito Magnets can reduce the abundance of nuisance mosquitoes, potentially reducing the biting pressure on the public, and providing another tool in mosquito control operations.