ABSTRACT
Double-zero-event studies (DZS) pose a challenge for accurately estimating the overall treatment effect in meta-analysis. Current approaches, such as continuity correction or omission of DZS, are commonly employed, yet these ad hoc methods can yield biased conclusions. Although the standard bivariate generalized linear mixed model can accommodate DZS, it fails to address the potential systemic differences between DZS and other studies. In this paper, we propose a zero-inflated bivariate generalized linear mixed model (ZIBGLMM) to tackle this issue. This two-component finite mixture model includes zero-inflation for a subpopulation with negligible or extremely low risk. We develop both frequentist and Bayesian versions of ZIBGLMM and examine its performance in estimating risk ratios (RRs) against the bivariate generalized linear mixed model and conventional two-stage meta-analysis that excludes DZS. Through extensive simulation studies and real-world meta-analysis case studies, we demonstrate that ZIBGLMM outperforms the bivariate generalized linear mixed model and conventional two-stage metaanalysis that excludes DZS in estimating the true effect size with substantially less bias and comparable coverage probability.
ABSTRACT
Network meta-analysis, also known as mixed treatments comparison meta-analysis or multiple treatments meta-analysis, extends conventional pairwise meta-analysis by simultaneously synthesizing multiple interventions in a single integrated analysis. Despite the growing popularity of network metaanalysis within comparative effectiveness research, it comes with potential challenges. For example, within-study correlations among treatment comparisons are rarely reported in the published literature. Yet, these correlations are pivotal for valid statistical inference. As demonstrated in earlier studies, ignoring these correlations can inflate mean squared errors of the resulting point estimates and lead to inaccurate standard error estimates. This paper introduces a composite likelihood-based approach that ensures accurate statistical inference without requiring knowledge of the within-study correlations. The proposed method is computationally robust and efficient, with substantially reduced computational time compared to the state-of-the-science methods implemented in R packages. The proposed method was evaluated through extensive simulations and applied to two important applications including a network meta-analysis comparing interventions for primary open-angle glaucoma, and another comparing treatments for chronic prostatitis and chronic pelvic pain syndrome.
ABSTRACT
Systematic reviews and meta-analyses are essential tools in contemporary evidence-based medicine, synthesizing evidence from various sources to better inform clinical decision-making. However, the conclusions from different meta-analyses on the same topic can be discrepant, which has raised concerns about their reliability. One reason is that the result of a meta-analysis is sensitive to factors such as study inclusion/exclusion criteria and model assumptions. The arm-based meta-analysis model is growing in importance due to its advantage of including single-arm studies and historical controls with estimation efficiency and its flexibility in drawing conclusions with both marginal and conditional effect measures. Despite its benefits, the inference may heavily depend on the heterogeneity parameters that reflect design and model assumptions. This article aims to evaluate the robustness of meta-analyses using the arm-based model within a Bayesian framework. Specifically, we develop a tipping point analysis of the between-arm correlation parameter to assess the robustness of meta-analysis results. Additionally, we introduce some visualization tools to intuitively display its impact on meta-analysis results. We demonstrate the application of these tools in three real-world meta-analyses, one of which includes single-arm studies.
Subject(s)
Bayes Theorem , Evidence-Based Medicine , Meta-Analysis as Topic , Humans , Evidence-Based Medicine/methods , Evidence-Based Medicine/standards , Evidence-Based Medicine/statistics & numerical data , Reproducibility of Results , Systematic Reviews as Topic/methods , Models, Statistical , AlgorithmsABSTRACT
Network meta-analysis (NMA) is a statistical procedure to simultaneously compare multiple interventions. Despite the added complexity of performing an NMA compared with the traditional pairwise meta-analysis, under proper assumptions the NMA can lead to more efficient estimates on the comparisons of interventions by combining and contrasting the direct and indirect evidence into a form of evidence that can be used to underpin treatment guidelines. Two broad classes of NMA methods are commonly used in practice: the contrast-based (CB-NMA) and the arm-based (AB-NMA) models. While CB-NMA only focuses on the relative effects by assuming fixed intercepts, the AB-NMA offers greater flexibility on the estimands, including both the absolute and relative effects by assuming random intercepts. A major criticism of the AB-NMA, on which we aim to elaborate in this paper, is that it does not retain randomization within trials, which may introduce bias in the estimated relative effects in some scenarios. This criticism was drawn under the implicit assumption that a given relative effect is transportable, in which case the data generating mechanism favors the inference based on CB-NMA, which models the relative effect. In this article, we aim to review, summarize, and elaborate on the underlying assumptions, similarities and differences, and also the advantages and disadvantages, between CB-NMA and AB-NMA methods. As indirect treatment comparison is susceptible to risk of bias no matter which approach is taken, it is important to consider both approaches in practice as complementary sensitivity analyses and to provide the totality of evidence from the data.
ABSTRACT
OBJECTIVES: To apply a hierarchical model (HM) that addresses measurement error in regression of the treatment effect on the control group event rate (CR). We compare HM to weighted linear regression (WLR) which is subject to measurement error and mathematical coupling. STUDY DESIGN AND SETTING: We reviewed published HMs that address measurement error and implemented a Bayesian version in open-source code to facilitate adoption by meta-analysts. We compared WLR and HM across a very large convenience sample of meta-analyses published in the Cochrane Database of Systematic Reviews. RESULTS: We applied both approaches (WLR and an HM that addresses measurement error) to 3193 meta-analyses that included 33,071 studies (average 10.28 studies per meta-analysis). A statistically significant slope suggesting an association between the treatment effect and CR was demonstrated with both approaches in 568 (17.19%) meta-analyses, with neither approach in 2036 (63.77%) meta-analyses, only with WLS in 229 (7.17%) and only with HM in 360 (11.28%) meta-analyses. The majority of slopes was negative (WLR 85%, HM 83%). In the majority of cases, HM had wider confidence intervals (72.53%) and slopes farther from the null (64.77%). CONCLUSION: Approximately 28% of meta-analyses demonstrate a significant association between the treatment effect and CR when HM is used to address measurement error, which can suggest frequent lack of portability of the relative effect across baseline risks. User-friendly open-source code is provided to meta-analysts interested in exploring this association.
Subject(s)
Bayes Theorem , Humans , Meta-Analysis as Topic , Models, Statistical , Control Groups , Linear Models , Treatment OutcomeABSTRACT
BACKGROUND: Empirical evidence suggests that lack of blinding may be associated with biased estimates of treatment benefit in randomized controlled trials, but the influence on medication-related harms is not well-recognized. We aimed to investigate the association between blinding and clinical trial estimates of medication-related harms. METHODS: We searched PubMed from January 1, 2015, till January 1, 2020, for systematic reviews with meta-analyses of medication-related harms. Eligible meta-analyses must have contained trials both with and without blinding. Potential covariates that may confound effect estimates were addressed by restricting trials within the comparison or by hierarchical analysis of harmonized groups of meta-analyses (therefore harmonizing drug type, control, dosage, and registration status) across eligible meta-analyses. The weighted hierarchical linear regression was then used to estimate the differences in harm estimates (odds ratio, OR) between trials that lacked blinding and those that were blinded. The results were reported as the ratio of OR (ROR) with its 95% confidence interval (CI). RESULTS: We identified 629 meta-analyses of harms with 10,069 trials. We estimated a weighted average ROR of 0.68 (95% CI: 0.53 to 0.88, P < 0.01) among 82 trials in 20 meta-analyses where blinding of participants was lacking. With regard to lack of blinding of healthcare providers or outcomes assessors, the RORs were 0.68 (95% CI: 0.53 to 0.87, P < 0.01 from 81 trials in 22 meta-analyses) and 1.00 (95% CI: 0.94 to 1.07, P = 0.94 from 858 trials among 155 meta-analyses) respectively. Sensitivity analyses indicate that these findings are applicable to both objective and subjective outcomes. CONCLUSIONS: Lack of blinding of participants and health care providers in randomized controlled trials may underestimate medication-related harms. Adequate blinding in randomized trials, when feasible, may help safeguard against potential bias in estimating the effects of harms.
Subject(s)
Health Personnel , Humans , Retrospective Studies , Randomized Controlled Trials as Topic , Systematic Reviews as Topic , Linear ModelsABSTRACT
Population-adjusted indirect comparison (PAIC) is an increasingly used technique for estimating the comparative effectiveness of different treatments for the health technology assessments when head-to-head trials are unavailable. Three commonly used PAIC methods include matching-adjusted indirect comparison (MAIC), simulated treatment comparison (STC), and multilevel network meta-regression (ML-NMR). MAIC enables researchers to achieve balanced covariate distribution across two independent trials when individual participant data are only available in one trial. In this article, we provide a comprehensive review of the MAIC methods, including their theoretical derivation, implicit assumptions, and connection to calibration estimation in survey sampling. We discuss the nuances between anchored and unanchored MAIC, as well as their required assumptions. Furthermore, we implement various MAIC methods in a user-friendly R Shiny application Shiny-MAIC. To our knowledge, it is the first Shiny application that implements various MAIC methods. The Shiny-MAIC application offers choice between anchored or unanchored MAIC, choice among different types of covariates and outcomes, and two variance estimators including bootstrap and robust standard errors. An example with simulated data is provided to demonstrate the utility of the Shiny-MAIC application, enabling a user-friendly approach conducting MAIC for healthcare decision-making. The Shiny-MAIC is freely available through the link: https://ziren.shinyapps.io/Shiny_MAIC/.
Subject(s)
Algorithms , Comparative Effectiveness Research , Computer Simulation , Humans , Technology Assessment, Biomedical , Models, Statistical , Research Design , Software , Calibration , Regression Analysis , Data Interpretation, Statistical , Network Meta-Analysis , Cost-Benefit AnalysisABSTRACT
A reference interval represents the normative range for measurements from a healthy population. It plays an important role in laboratory testing, as well as in differentiating healthy from diseased patients. The reference interval based on a single study might not be applicable to a broader population. Meta-analysis can provide a more generalizable reference interval based on the combined population by synthesizing results from multiple studies. However, the assumptions of normally distributed underlying study-specific means and equal within-study variances, which are commonly used in existing methods, are strong and may not hold in practice. We propose a Bayesian nonparametric model with more flexible assumptions to extend random effects meta-analysis for estimating reference intervals. We illustrate through simulation studies and two real data examples the performance of our proposed approach when the assumptions of normally distributed study means and equal within-study variances do not hold.
Subject(s)
Health Status , Humans , Bayes Theorem , Computer Simulation , Sample SizeABSTRACT
For patients with triple-class exposed/refractory multiple myeloma (TCE/RMM), where effective treatments options are limited, B-cell maturation antigen and CD3-directed bispecific antibodies offer a promising new approach. Teclistamab gained conditional approval in Europe and accelerated Food and Drug Administration (FDA) approval based on the MajesTEC-1 trial (NCT03145181). Elranatamab, approved by the FDA demonstrated its safety and efficacy in the MagnetisMM-3 trial (NCT04649359). Given the absence of head-to-head trials, an unanchored matching-adjusted indirect comparison (MAIC) was conducted to assess their relative efficacy. Key baseline characteristics were adjusted to be comparable between the two trials. In the MAIC, elranatamab demonstrated significantly better objective response rate and progression-free survival (PFS) than teclistamab, and numerically better complete response, duration of response, and overall survival (OS). These results suggest that elranatamab is an efficacious option for treating patients with TCE/R MM.
Subject(s)
Multiple Myeloma , Humans , Multiple Myeloma/drug therapy , Multiple Myeloma/mortality , Male , Female , Middle Aged , Aged , Antibodies, Bispecific/therapeutic use , Antibodies, Bispecific/adverse effects , Drug Resistance, Neoplasm , Treatment Outcome , B-Cell Maturation Antigen/antagonists & inhibitors , B-Cell Maturation Antigen/immunology , Adult , Antibodies, Monoclonal, Humanized/therapeutic use , Antibodies, Monoclonal, Humanized/adverse effects , Aged, 80 and overABSTRACT
PURPOSE: Although maternal depression is associated with adverse outcomes in women and children, its relationship with lower urinary tract symptoms (LUTS) in offspring is less well-characterized. We examined the association between prenatal and postpartum maternal depression and LUTS in primary school-age daughters. DESIGN: Observational cohort study. SUBJECTS AND SETTING: The sample comprised 7148 mother-daughter dyads from the Avon Longitudinal Study of Parents and Children. METHOD: Mothers completed questionnaires about depressive symptoms at 18 and 32 weeks' gestation and 21 months postpartum and their children's LUTS (urinary urgency, nocturia, and daytime and nighttime wetting) at 6, 7, and 9 years of age. Multivariable logistic regression models were used to estimate the association between maternal depression and LUTS in daughters. RESULTS: Compared to daughters of mothers without depression, those born to mothers with prenatal and postpartum depression had higher odds of LUTS, including urinary urgency (adjusted odds ratio [aOR] range = 1.99-2.50) and nocturia (aOR range = 1.67-1.97) at 6, 7, and 9 years of age. Additionally, daughters born to mothers with prenatal and postpartum depression had higher odds of daytime wetting (aOR range = 1.81-1.99) and nighttime wetting (aOR range = 1.63-1.95) at 6 and 7 years of age. Less consistent associations were observed for depression limited to the prenatal or postpartum periods only. CONCLUSIONS: Exposure to maternal depression in the prenatal and postpartum periods was associated with an increased likelihood of LUTS in daughters. This association may be an important opportunity for childhood LUTS prevention. Prevention strategies should reflect an understanding of potential biological and environmental mechanisms through which maternal depression may influence childhood LUTS.
Subject(s)
Depression, Postpartum , Lower Urinary Tract Symptoms , Nocturia , Pregnancy , Child , Female , Humans , Cohort Studies , Depression, Postpartum/complications , Depression, Postpartum/epidemiology , Longitudinal Studies , Depression/complications , Depression/epidemiology , Nuclear Family , Nocturia/complications , Nocturia/epidemiology , Lower Urinary Tract Symptoms/complications , Lower Urinary Tract Symptoms/epidemiology , SchoolsABSTRACT
In a recent systematic review, Bastos et al. (Ann Intern Med. 2021;174(4):501-510) compared the sensitivities of saliva sampling and nasopharyngeal swabs in the detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection by assuming a composite reference standard defined as positive if either test is positive and negative if both tests are negative (double negative). Even under a perfect specificity assumption, this approach ignores the double-negative results and risks overestimating the sensitivities due to residual misclassification. In this article, we first illustrate the impact of double-negative results in the estimation of the sensitivities in a single study, and then propose a 2-step latent class meta-analysis method for reevaluating both sensitivities using the same published data set as that used in Bastos et al. by properly including the observed double-negative results. We also conduct extensive simulation studies to compare the performance of the proposed method with Bastos et al.'s method for varied levels of prevalence and between-study heterogeneity. The results demonstrate that the sensitivities are overestimated noticeably using Bastos et al.'s method, and the proposed method provides a more accurate evaluation with nearly no bias and close-to-nominal coverage probability. In conclusion, double-negative results can significantly impact the estimated sensitivities when a gold standard is absent, and thus they should be properly incorporated.
Subject(s)
COVID-19 , Humans , COVID-19/diagnosis , SARS-CoV-2 , Negative Results , Saliva , NasopharynxABSTRACT
Meta-analysis is commonly used to combine results from multiple clinical trials, but traditional meta-analysis methods do not refer explicitly to a population of individuals to whom the results apply and it is not clear how to use their results to assess a treatment's effect for a population of interest. We describe recently-introduced causally interpretable meta-analysis methods and apply their treatment effect estimators to two individual-participant data sets. These estimators transport estimated treatment effects from studies in the meta-analysis to a specified target population using the individuals' potentially effect-modifying covariates. We consider different regression and weighting methods within this approach and compare the results to traditional aggregated-data meta-analysis methods. In our applications, certain versions of the causally interpretable methods performed somewhat better than the traditional methods, but the latter generally did well. The causally interpretable methods offer the most promise when covariates modify treatment effects and our results suggest that traditional methods work well when there is little effect heterogeneity. The causally interpretable approach gives meta-analysis an appealing theoretical framework by relating an estimator directly to a specific population and lays a solid foundation for future developments.
Subject(s)
Meta-Analysis as Topic , Research Design , HumansABSTRACT
Drug repurposing refers to the process of discovering new therapeutic uses for existing medicines. Compared to traditional drug discovery, drug repurposing is attractive for its speed, cost, and reduced risk of failure. However, existing approaches for drug repurposing involve complex, computationally-intensive analytical methods that are not widely used in practice. Instead, repurposing decisions are often based on subjective judgments from limited empirical evidence. In this article, we develop a novel Bayesian network meta-analysis (NMA) framework that can predict the efficacy of an approved treatment in a new indication and thereby identify candidate treatments for repurposing. We obtain predictions using two main steps: first, we use standard NMA modeling to estimate average relative effects from a network comprised of treatments studied in both indications in addition to one treatment studied in only one indication. Then, we model the correlation between relative effects using various strategies that differ in how they model treatments across indications and within the same drug class. We evaluate the predictive performance of each model using a simulation study and find that the model minimizing root mean squared error of the posterior median for the candidate treatment depends on the amount of available data, the level of correlation between indications, and whether treatment effects differ, on average, by drug class. We conclude by discussing an illustrative example in psoriasis and psoriatic arthritis and find that the candidate treatment has a high probability of success in a future trial.
Subject(s)
Psoriasis , Humans , Network Meta-Analysis , Bayes Theorem , Psoriasis/drug therapyABSTRACT
INTRODUCTION: For patients with triple-class exposed/refractory multiple myeloma (TCE/R MM), prognosis is poor and effective treatment options are limited. Elranatamab is a novel B-cell maturation antigen (BCMA)- and CD3-directed bispecific antibody which was approved by the US Food and Drug Administration in August 2023 and demonstrated safety and efficacy in patients with TCE/R MM in the phase 2, single-arm MagnetisMM-3 trial (NCT04649359). To compare the effectiveness of elranatamab vs physician's choice of treatment (PCT) in the absence of head-to-head comparative data, a matching-adjusted indirect comparison (MAIC) was conducted. METHODS: Individual patient data from MagnetisMM-3 (Cohort A [BCMA-naïve] N = 123, 14.7 months of follow-up) were reweighted to match published summary data from two real-world studies of PCT in patients with TCE/R MM (LocoMMotion and MAMMOTH) using a propensity score-type logistic regression. Unanchored MAIC analyses were conducted according to National Institute for Health and Care Excellence (NICE) Decision Support Unit (DSU) 18 guidance. RESULTS: Compared with PCT in LocoMMotion, elranatamab was associated with a significantly higher objective response rate (ORR rate difference: 37.52; 95% CI 26.20-48.83; odds ratio: 4.85; 95% CI 2.85-8.23) and complete or stringent complete response rate (≥CR rate difference: 42.29; 95% CI 31.84-52.74; odds ratio: 184.01; 95% CI 24.66-1372.86), longer progression-free survival (PFS HR 0.32; 95% CI 0.20-0.49), and overall survival (OS HR 0.62; 95% CI 0.40-0.94). Compared with PCT in MAMMOTH, elranatamab was associated with significantly higher ORR (rate difference: 28.14; 95% CI 16.77-39.52; odds ratio: 3.24; 95% CI 1.98-5.32) and ≥ CR (rate difference: 26.22; 95% CI 16.40-36.05; odds ratio: 5.48; 95% CI 2.88-10.44), as well as longer PFS (HR 0.25; 95% CI 0.17-0.37) and OS (HR 0.49; 95% CI 0.33-0.71). Sensitivity analysis results were consistent with the base case. CONCLUSION: In the MAIC, elranatamab was consistently associated with improved rates and depth of response and significantly longer PFS and OS versus PCT in LocoMMotion and MAMMOTH.
Subject(s)
Mammoths , Multiple Myeloma , Physicians , Humans , Animals , Multiple Myeloma/drug therapy , B-Cell Maturation Antigen/therapeutic use , Treatment OutcomeABSTRACT
The appropriate use of regulatory agilities has the potential to accelerate regulatory review, utilize resources more efficiently and deliver medicines and vaccines more rapidly, all without compromising quality, safety and efficacy. This was clearly demonstrated during the COVID-19 pandemic where regulators and industry rapidly adapted to ensure continued supply of existing critical medicines and review and approve new innovative medicines. In this retrospective study, we analyze the impact of regulatory agilities on the review and approval of Pfizer/BioNTech's BNT162b2 mRNA COVID-19 Vaccine globally using regulatory approval data from 73 country/regional approvals. We report on the critical role of reliance and provide evidence that demonstrates reliance approaches and certain regulatory agilities reduced review times for the COVID-19 vaccine. These findings support the case for more widespread implementation of regulatory agilities and demonstrate the important role of such approaches to improve public health outcomes.
ABSTRACT
When evaluating a diagnostic test, it is common that a gold standard may not be available. One example is the diagnosis of SARS-CoV-2 infection using saliva sampling or nasopharyngeal swabs. Without a gold standard, a pragmatic approach is to postulate a "reference standard," defined as positive if either test is positive, or negative if both are negative. However, this pragmatic approach may overestimate sensitivities because subjects infected with SARS-CoV-2 may still have double-negative test results even when both tests exhibit perfect specificity. To address this limitation, we propose a Bayesian hierarchical model for simultaneously estimating sensitivity, specificity, and disease prevalence in the absence of a gold standard. The proposed model allows adjusting for study-level covariates. We evaluate the model performance using an example based on a recently published meta-analysis on the diagnosis of SARS-CoV-2 infection and extensive simulations. Compared with the pragmatic reference standard approach, we demonstrate that the proposed Bayesian method provides a more accurate evaluation of prevalence, specificity, and sensitivity in a meta-analytic framework.
Subject(s)
COVID-19 , Humans , COVID-19/diagnosis , COVID-19/epidemiology , SARS-CoV-2 , Bayes Theorem , Sensitivity and Specificity , Diagnostic Tests, Routine/methods , COVID-19 TestingABSTRACT
BACKGROUND: Sensitivity and specificity are characteristics of a diagnostic test and are not expected to change as the prevalence of the target condition changes. We sought to evaluate the association between prevalence and changes in sensitivity and specificity. METHODS: We retrieved data from meta-analyses of diagnostic test accuracy published in the Cochrane Database of Systematic Reviews (2003-2020). We used mixed-effects random-intercept linear regression models to evaluate the association between prevalence and logit-transformed sensitivity and specificity. The model evaluated all meta-analyses as nested within each systematic review. RESULTS: We analyzed 6909 diagnostic test accuracy studies from 552 meta-analyses that were included in 92 systematic reviews. For sensitivity, compared with the lowest quartile of prevalence, the second, third and fourth quartiles were associated with significantly higher odds of identifying a true positive case (odds ratio [OR] 1.17, 95% confidence interval [CI] 1.09-1.26; OR 1.32, 95% CI 1.23-1.41; OR 1.47, 95% CI 1.37-1.58; respectively). For specificity, compared with the lowest quartile of prevalence, the second, third and fourth quartiles were associated with significantly lower odds of identifying a true negative case (OR 0.74, 95% CI 0.69-0.80; OR 0.65, 95% CI 0.60-0.70; OR 0.47, 95% CI 0.44-0.51; respectively). Pooled regression coefficients from bivariate models conducted within each meta-analysis showed that prevalence was positively associated with sensitivity and negatively associated with specificity. Findings were consistent across subgroups. INTERPRETATION: In this large sample of diagnostic studies, higher prevalence was associated with higher estimated sensitivity and lower estimated specificity. Clinicians should consider the implications of disease prevalence and spectrum when interpreting the results from studies of diagnostic test accuracy.