On the Q statistic with constant weights in meta-analysis of binary outcomes.

BACKGROUND: Cochran's Q statistic is routinely used for testing heterogeneity in meta-analysis. Its expected value (under an incorrect null distribution) is part of several popular estimators of the between-study variance, [Formula: see text]. Those applications generally do not account for use of the studies' estimated variances in the inverse-variance weights that define Q (more explicitly, [Formula: see text]). Importantly, those weights make approximating the distribution of [Formula: see text] rather complicated. METHODS: As an alternative, we are investigating a Q statistic, [Formula: see text], whose constant weights use only the studies' arm-level sample sizes. For log-odds-ratio (LOR), log-relative-risk (LRR), and risk difference (RD) as the measures of effect, we study, by simulation, approximations to distributions of [Formula: see text] and [Formula: see text], as the basis for tests of heterogeneity. RESULTS: The results show that: for LOR and LRR, a two-moment gamma approximation to the distribution of [Formula: see text] works well for small sample sizes, and an approximation based on an algorithm of Farebrother is recommended for larger sample sizes. For RD, the Farebrother approximation works very well, even for small sample sizes. For [Formula: see text], the standard chi-square approximation provides levels that are much too low for LOR and LRR and too high for RD. The Kulinskaya et al. (Res Synth Methods 2:254-70, 2011) approximation for RD and the Kulinskaya and Dollinger (BMC Med Res Methodol 15:49, 2015) approximation for LOR work well for [Formula: see text] but have some convergence issues for very small sample sizes combined with small probabilities. CONCLUSIONS: The performance of the standard [Formula: see text] approximation is inadequate for all three binary effect measures. Instead, we recommend a test of heterogeneity based on [Formula: see text] and provide practical guidelines for choosing an appropriate test at the .05 level for all three effect measures.

The win odds: statistical inference and regression.

Generalized pairwise comparisons and win statistics (i.e., win ratio, win odds and net benefit) are advantageous in analyzing and interpreting a composite of multiple outcomes in clinical trials. An important limitation of these statistics is their inability to adjust for covariates other than by stratified analysis. Because the win ratio does not account for ties, the win odds, a modification that includes ties, has attracted attention. We review and combine information on the win odds to articulate the statistical inferences for the win odds. We also show alternative variance estimators based on the exact permutation and bootstrap as well as statistical inference via the probabilistic index. Finally, we extend multiple-covariate regression probabilistic index models to the win odds with a univariate outcome. As an illustration we apply the regression models to the data in the CHARM trial.

The stratified win statistics (win ratio, win odds, and net benefit).

The win odds and the net benefit are related directly to each other and indirectly, through ties, to the win ratio. These three win statistics test the same null hypothesis of equal win probabilities between two groups. They provide similar p-values and powers, because the Z-values of their statistical tests are approximately equal. Thus, they can complement one another to show the strength of a treatment effect. In this article, we show that the estimated variances of the win statistics are also directly related regardless of ties or indirectly related through ties. Since its introduction in 2018, the stratified win ratio has been applied in designs and analyses of clinical trials, including Phase III and Phase IV studies. This article generalizes the stratified method to the win odds and the net benefit. As a result, the relations of the three win statistics and the approximate equivalence of their statistical tests also hold for the stratified win statistics.

Win statistics (win ratio, win odds, and net benefit) can complement one another to show the strength of the treatment effect on time-to-event outcomes.

Conventional analyses of a composite of multiple time-to-event outcomes use the time to the first event. However, the first event may not be the most important outcome. To address this limitation, generalized pairwise comparisons and win statistics (win ratio, win odds, and net benefit) have become popular and have been applied to clinical trial practice. However, win ratio, win odds, and net benefit have typically been used separately. In this article, we examine the use of these three win statistics jointly for time-to-event outcomes. First, we explain the relation of point estimates and variances among the three win statistics, and the relation between the net benefit and the Mann-Whitney U statistic. Then we explain that the three win statistics are based on the same win proportions, and they test the same null hypothesis of equal win probabilities in two groups. We show theoretically that the Z-values of the corresponding statistical tests are approximately equal; therefore, the three win statistics provide very similar p-values and statistical powers. Finally, using simulation studies and data from a clinical trial, we demonstrate that, when there is no (or little) censoring, the three win statistics can complement one another to show the strength of the treatment effect. However, when the amount of censoring is not small, and without adjustment for censoring, the win odds and the net benefit may have an advantage for interpreting the treatment effect; with adjustment (e.g., IPCW adjustment) for censoring, the three win statistics can complement one another to show the strength of the treatment effect. For calculations we use the R package WINS, available on the CRAN (Comprehensive R Archive Network).

Adjusting win statistics for dependent censoring.

For composite outcomes whose components can be prioritized on clinical importance, the win ratio, the net benefit and the win odds apply that order in comparing patients pairwise to produce wins and subsequently win proportions. Because these three statistics are derived using the same win proportions and they test the same hypothesis of equal win probabilities in the two treatment groups, we refer to them as win statistics. These methods, particularly the win ratio and the net benefit, have received increasing attention in methodological research and in design and analysis of clinical trials. For time-to-event outcomes, however, censoring may introduce bias. Previous work has shown that inverse-probability-of-censoring weighting (IPCW) can correct the win ratio for bias from independent censoring. The present article uses the IPCW approach to adjust win statistics for dependent censoring that can be predicted by baseline covariates and/or time-dependent covariates (producing the CovIPCW-adjusted win statistics). Theoretically and with examples and simulations, we show that the CovIPCW-adjusted win statistics are unbiased estimators of treatment effect in the presence of dependent censoring.

Estimation in meta-analyses of mean difference and standardized mean difference.

Methods for random-effects meta-analysis require an estimate of the between-study variance, τ2 . The performance of estimators of τ2 (measured by bias and coverage) affects their usefulness in assessing heterogeneity of study-level effects and also the performance of related estimators of the overall effect. However, as we show, the performance of the methods varies widely among effect measures. For the effect measures mean difference (MD) and standardized MD (SMD), we use improved effect-measure-specific approximations to the expected value of Q for both MD and SMD to introduce two new methods of point estimation of τ2 for MD (Welch-type and corrected DerSimonian-Laird) and one WT interval method. We also introduce one point estimator and one interval estimator for τ2 in SMD. Extensive simulations compare our methods with four point estimators of τ2 (the popular methods of DerSimonian-Laird, restricted maximum likelihood, and Mandel and Paule, and the less-familiar method of Jackson) and four interval estimators for τ2 (profile likelihood, Q-profile, Biggerstaff and Jackson, and Jackson). We also study related point and interval estimators of the overall effect, including an estimator whose weights use only study-level sample sizes. We provide measure-specific recommendations from our comprehensive simulation study and discuss an example.

Estimation in meta-analyses of response ratios.

BACKGROUND: For outcomes that studies report as the means in the treatment and control groups, some medical applications and nearly half of meta-analyses in ecology express the effect as the ratio of means (RoM), also called the response ratio (RR), analyzed in the logarithmic scale as the log-response-ratio, LRR. METHODS: In random-effects meta-analysis of LRR, with normal and lognormal data, we studied the performance of estimators of the between-study variance, τ2, (measured by bias and coverage) in assessing heterogeneity of study-level effects, and also the performance of related estimators of the overall effect in the log scale, λ. We obtained additional empirical evidence from two examples. RESULTS: The results of our extensive simulations showed several challenges in using LRR as an effect measure. Point estimators of τ2 had considerable bias or were unreliable, and interval estimators of τ2 seldom had the intended 95% coverage for small to moderate-sized samples (n<40). Results for estimating λ differed between lognormal and normal data. CONCLUSIONS: For lognormal data, we can recommend only SSW, a weighted average in which a study's weight is proportional to its effective sample size, (when n≥40) and its companion interval (when n≥10). Normal data posed greater challenges. When the means were far enough from 0 (more than one standard deviation, 4 in our simulations), SSW was practically unbiased, and its companion interval was the only option.

The inverse-probability-of-censoring weighting (IPCW) adjusted win ratio statistic: an unbiased estimator in the presence of independent censoring.

The win ratio method has received much attention in methodological research, ad hoc analyses, and designs of prospective studies. As the primary analysis, it supported the approval of tafamidis for the treatment of cardiomyopathy to reduce cardiovascular mortality and cardiovascular-related hospitalization. However, its dependence on censoring is a potential shortcoming. In this article, we propose the inverse-probability-of-censoring weighting (IPCW) adjusted win ratio statistic (i.e., the IPCW-adjusted win ratio statistic) to overcome censoring issues. We consider independent censoring, common censoring across endpoints, and right censoring. We develop an asymptotic variance estimator for the logarithm of the IPCW-adjusted win ratio statistic and evaluate it via simulation. Our simulation studies show that, as the amount of censoring increases, the unadjusted win proportions may decrease greatly. Consequently, the bias of the unadjusted win ratio estimate may increase greatly, producing either an overestimate or an underestimate. We demonstrate theoretically and through simulation that the IPCW-adjusted win ratio statistic gives an unbiased estimate of treatment effect.

The win ratio: Impact of censoring and follow-up time and use with nonproportional hazards.

The win ratio has been studied methodologically and applied in data analysis and in designing clinical trials. Researchers have pointed out that the results depend on follow-up time and censoring time, which are sometimes used interchangeably. In this article, we distinguish between follow-up time and censoring time, show theoretically the impact of censoring on the win ratio, and illustrate the impact of follow-up time. We then point out that, if the treatment has long-term benefit from a more important but less frequent endpoint (eg, death), the win ratio can show that benefit by following patients longer, avoiding masking by more frequent but less important outcomes, which occurs in conventional time-to-first-event analyses. For the situation of nonproportional hazards, we demonstrate that the win ratio can be a good alternative to methods such as landmark survival rate, restricted mean survival time, and weighted log-rank tests.

Meta-Analysis of Odds Ratios: Current Good Practices.

BACKGROUND: Many systematic reviews of randomized clinical trials lead to meta-analyses of odds ratios (ORs). The customary methods of estimating an overall OR involve weighted averages of the individual trials' estimates of the logarithm of the OR. That approach, however, has several shortcomings, arising from assumptions and approximations, that render the results unreliable. Although the problems have been documented in the literature for many years, the conventional methods persist in software and applications. A well-developed alternative approach avoids the approximations by working directly with the numbers of subjects and events in the arms of the individual trials. OBJECTIVE: We aim to raise awareness of methods that avoid the conventional approximations, can be applied with widely available software, and produce more-reliable results. METHODS: We summarize the fixed-effect and random-effects approaches to meta-analysis; describe conventional, approximate methods and alternative methods; apply the methods in a meta-analysis of 19 randomized trials of endoscopic sclerotherapy in patients with cirrhosis and esophagogastric varices; and compare the results. We demonstrate the use of SAS, Stata, and R software for the analysis. RESULTS: In the example, point estimates and confidence intervals for the overall log-odds-ratio differ between the conventional and alternative methods, in ways that can affect inferences. Programming is straightforward in the 3 software packages; an appendix, Suppemental Digital Content 1 (http://links.lww.com/MLR/B335) gives the details. CONCLUSIONS: The modest additional programming required should not be an obstacle to adoption of the alternative methods. Because their results are unreliable, use of the conventional methods for meta-analysis of ORs should be discontinued.

Misunderstandings about Q and 'Cochran's Q test' in meta-analysis.

Many meta-analyses report using 'Cochran's Q test' to assess heterogeneity of effect-size estimates from the individual studies. Some authors cite work by W. G. Cochran, without realizing that Cochran deliberately did not use Q itself to test for heterogeneity. Further, when heterogeneity is absent, the actual null distribution of Q is not the chi-squared distribution assumed for 'Cochran's Q test'. This paper reviews work by Cochran related to Q. It then discusses derivations of the asymptotic approximation for the null distribution of Q, as well as work that has derived finite-sample moments and corresponding approximations for the cases of specific measures of effect size. Those results complicate implementation and interpretation of the popular heterogeneity index I(2) . Also, it turns out that the test-based confidence intervals used with I(2) are based on a fallacious approach. Software that outputs Q and I(2) should use the appropriate reference value of Q for the particular measure of effect size and the current meta-analysis. Q is a key element of the popular DerSimonian-Laird procedure for random-effects meta-analysis, but the assumptions of that procedure and related procedures do not reflect the actual behavior of Q and may introduce bias. The DerSimonian-Laird procedure should be regarded as unreliable.

An empirical comparison of univariate and multivariate meta-analyses for categorical outcomes.

Treatment effects for multiple outcomes can be meta-analyzed separately or jointly, but no systematic empirical comparison of the two approaches exists. From the Cochrane Library of Systematic Reviews, we identified 45 reviews, including 1473 trials and 258,675 patients, that contained two or three univariate meta-analyses of categorical outcomes for the same interventions that could also be analyzed jointly. Eligible were meta-analyses with at least seven trials reporting all outcomes for which the cross-classification tables were exactly recoverable (e.g., outcomes were mutually exclusive, or one was a subset of the other). This ensured known correlation structures. Outcomes in 40 reviews had an is-subset-of relationship, and those in 5 were mutually exclusive. We analyzed these data with univariate and multivariate models based on discrete and approximate likelihoods. Discrete models were fit in the Bayesian framework using slightly informative priors. The summary effects for each outcome were similar with univariate and multivariate meta-analyses (both using the approximate and discrete likelihoods); however, the multivariate model with the discrete likelihood gave smaller between-study variance estimates, and narrower predictive intervals for new studies. When differences in the summary treatment effects were examined, the multivariate models gave similar summary estimates but considerably longer (shorter) uncertainty intervals because of positive (negative) correlation between outcome treatment effects. It is unclear whether any of the examined reviews would change their overall conclusions based on multivariate versus univariate meta-analyses, because extra-analytical and context-specific considerations contribute to conclusions and, secondarily, because numerical differences were often modest.

Factors influencing the development of antibiotic associated diarrhea in ED patients discharged home: risk of administering IV antibiotics.

OBJECTIVE: Antibiotic-associated diarrhea (AAD) and Clostridium difficile infection (CDI) are well-known outcomes from antibiotic administration. Because emergency department (ED) visits frequently result in antibiotic use, we evaluated the frequency of AAD/CDI in adults treated and discharged home with new prescriptions for antibiotics to identify risk factors for acquiring AAD/CDI. METHODS: This prospective multicenter cohort study enrolled adult patients who received antibiotics in the ED and were discharged with a new prescription for antibiotics. Antibiotic-associated diarrhea was defined as 3 or more loose stools for 2 days or more within 30 days of starting the antibiotic. C difficile infection was defined by the detection of toxin A or B within this same period. We used multivariate logistic regression to assess predictors of developing AAD. RESULTS: We enrolled and followed 247 patients; 45 (18%) developed AAD, and 2 (1%) developed CDI. Patients who received intravenous (IV) antibiotics in the ED were more likely to develop AAD/CDI than patients who did not: 25.7% (95% confidence interval [CI], 17.4-34.0) vs 12.3% (95% CI, 6.8-17.9). Intravenous antibiotics had adjusted odds ratio of 2.73 (95% CI, 1.38-5.43), and Hispanic ethnicity had adjusted odds ratio of 3.04 (95% CI, 1.40-6.58). Both patients with CDI had received IV doses of broad-spectrum antibiotics. CONCLUSION: Intravenous antibiotic therapy administered to ED patients before discharge was associated with higher rates of AAD and with 2 cases of CDI. Care should be taken when deciding to use broad-spectrum IV antibiotics to treat ED patients before discharge home.

Efficient network meta-analysis: a confidence distribution approach.

Network meta-analysis synthesizes several studies of multiple treatment comparisons to simultaneously provide inference for all treatments in the network. It can often strengthen inference on pairwise comparisons by borrowing evidence from other comparisons in the network. Current network meta-analysis approaches are derived from either conventional pairwise meta-analysis or hierarchical Bayesian methods. This paper introduces a new approach for network meta-analysis by combining confidence distributions (CDs). Instead of combining point estimators from individual studies in the conventional approach, the new approach combines CDs which contain richer information than point estimators and thus achieves greater efficiency in its inference. The proposed CD approach can e ciently integrate all studies in the network and provide inference for all treatments even when individual studies contain only comparisons of subsets of the treatments. Through numerical studies with real and simulated data sets, the proposed approach is shown to outperform or at least equal the traditional pairwise meta-analysis and a commonly used Bayesian hierarchical model. Although the Bayesian approach may yield comparable results with a suitably chosen prior, it is highly sensitive to the choice of priors (especially the prior of the between-trial covariance structure), which is often subjective. The CD approach is a general frequentist approach and is prior-free. Moreover, it can always provide a proper inference for all the treatment effects regardless of the between-trial covariance structure.

Evaluating the Clinical Utility of Epstein-Barr Virus Antibodies as Biomarkers in Multiple Sclerosis: A Systematic Review.

BACKGROUND: EBV is a necessary but not sufficient factor in the pathophysiology of multiple sclerosis (MS). EBV antibodies to the nuclear antigen (EBNA1) and viral capsid antigen (VCA) rise rapidly prior to MS disease manifestations, and their absence has clinical utility with a high negative predictive value. It remains unclear whether EBV levels act as prognostic, monitoring, or pharmacodynamic/response biomarkers. Substantial literature on this topic exists but has not been systematically reviewed. We hypothesized that EBV levels against EBNA1 and VCA are potential prognostic and monitoring biomarkers in MS, and that patient population, MS clinical phenotype, and EBV assay method may play important roles in explaining variation among study outcomes. METHODS: We systematically searched PubMed and EMBASE from inception to April 1, 2022. After removal of duplicates, records were screened by abstract. Remaining full-text articles were reviewed. Clinical and MRI data were extracted from full-text articles for comparison and synthesis. RESULTS: Searches yielded 696 unique results; 285 were reviewed in full, and 36 met criteria for data extraction. Heterogeneity in sample population, clinical outcome measures, assay methods and statistical analyses precluded a meta-analysis. EBV levels were not consistently associated with clinical disease markers including conversion from CIS to RRMS, neurological disability, or disease phenotype. Studies using repeated-measures design suggest that EBNA1 levels may temporarily reflect inflammatory disease activity as assessed by gadolinium-enhancing Magnetic Resonance Imaging (MRI) lesions. Limited data also suggest a decrease in EBV levels following initiation of certain disease-modifying therapies. CONCLUSION: Heterogeneous methodology limited generalization and meta-analysis. EBV antibody levels are unlikely to represent prognostic biomarkers in MS. The areas of highest ongoing promise relate to diagnostic exclusion and pharmacodynamic/disease response. Use of EBV antibodies as biomarkers in clinical practice remains additionally limited by lack of methodological precision, reliability, and validation.

Estimation of heterogeneity variance based on a generalized Q statistic in meta-analysis of log-odds-ratio.

For estimation of heterogeneity variance τ 2 in meta-analysis of log-odds-ratio, we derive new mean- and median-unbiased point estimators and new interval estimators based on a generalized Q statistic, Q F , in which the weights depend on only the studies' effective sample sizes. We compare them with familiar estimators based on the inverse-variance-weights version of Q , Q IV . In an extensive simulation, we studied the bias (including median bias) of the point estimators and the coverage (including left and right coverage error) of the confidence intervals. Most estimators add 0.5 to each cell of the 2 × 2 table when one cell contains a zero count; we include a version that always adds 0.5 . The results show that: two of the new point estimators and two of the familiar point estimators are almost unbiased when the total sample size n ≥ 250 and the probability in the Control arm ( p iC ) is 0.1, and when n ≥ 100 and p iC is 0.2 or 0.5; for 0.1 ≤ τ 2 ≤ 1 , all estimators have negative bias for small to medium sample sizes, but for larger sample sizes some of the new median-unbiased estimators are almost median-unbiased; choices of interval estimators depend on values of parameters, but one of the new estimators is reasonable when p iC = 0.1 and another, when p iC = 0.2 or p iC = 0.5 ; and lack of balance between left and right coverage errors for small n and/or p iC implies that the available approximations for the distributions of Q IV and Q F are accurate only for larger sample sizes.

Associations Between Natural Language Processing-Enriched Social Determinants of Health and Suicide Death Among US Veterans.

Importance: Social determinants of health (SDOHs) are known to be associated with increased risk of suicidal behaviors, but few studies use SDOHs from unstructured electronic health record notes. Objective: To investigate associations between veterans' death by suicide and recent SDOHs, identified using structured and unstructured data. Design, Setting, and Participants: This nested case-control study included veterans who received care under the US Veterans Health Administration from October 1, 2010, to September 30, 2015. A natural language processing (NLP) system was developed to extract SDOHs from unstructured clinical notes. Structured data yielded 6 SDOHs (ie, social or familial problems, employment or financial problems, housing instability, legal problems, violence, and nonspecific psychosocial needs), NLP on unstructured data yielded 8 SDOHs (social isolation, job or financial insecurity, housing instability, legal problems, barriers to care, violence, transition of care, and food insecurity), and combining them yielded 9 SDOHs. Data were analyzed in May 2022. Exposures: Occurrence of SDOHs over a maximum span of 2 years compared with no occurrence of SDOH. Main Outcomes and Measures: Cases of suicide death were matched with 4 controls on birth year, cohort entry date, sex, and duration of follow-up. Suicide was ascertained by National Death Index, and patients were followed up for up to 2 years after cohort entry with a study end date of September 30, 2015. Adjusted odds ratios (aORs) and 95% CIs were estimated using conditional logistic regression. Results: Of 6 122 785 veterans, 8821 committed suicide during 23 725 382 person-years of follow-up (incidence rate 37.18 per 100 000 person-years). These 8821 veterans were matched with 35 284 control participants. The cohort was mostly male (42 540 [96.45%]) and White (34 930 [79.20%]), with 6227 (14.12%) Black veterans. The mean (SD) age was 58.64 (17.41) years. Across the 5 common SDOHs, NLP-extracted SDOH, on average, retained 49.92% of structured SDOHs and covered 80.03% of all SDOH occurrences. SDOHs, obtained by structured data and/or NLP, were significantly associated with increased risk of suicide. The 3 SDOHs with the largest effect sizes were legal problems (aOR, 2.66; 95% CI, 2.46-2.89), violence (aOR, 2.12; 95% CI, 1.98-2.27), and nonspecific psychosocial needs (aOR, 2.07; 95% CI, 1.92-2.23), when obtained by combining structured data and NLP. Conclusions and Relevance: In this study, NLP-extracted SDOHs, with and without structured SDOHs, were associated with increased risk of suicide among veterans, suggesting the potential utility of NLP in public health studies.