Evaluating agreement between evidence from randomised controlled trials and cohort studies in nutrition: a meta-research replication study.

This meta-research study aims to evaluate the agreement of effect estimates between bodies of evidence (BoE) from RCTs and cohort studies included in the same nutrition evidence synthesis, to identify factors associated with disagreement, and to replicate the findings of a previous study. We searched Medline, Epistemonikos and the Cochrane Database of Systematic Reviews for nutrition systematic reviews that included both RCTs and cohort studies for the same patient-relevant outcome or intermediate-disease marker. We rated similarity of PI/ECO (population, intervention/exposure, comparison, outcome) between BoE from RCTs and cohort studies. Agreement of effect estimates across BoE was analysed by pooling ratio of risk ratios (RRR) for binary outcomes and difference of standardised mean differences (DSMD) for continuous outcomes. We performed subgroup and sensitivity analyses to explore determinants associated with disagreements. We included 82 BoE-pairs from 51 systematic reviews. For binary outcomes, the RRR was 1.04 (95% confidence interval (CI) 0.99 to 1.10, I2 = 59%, τ2 = 0.02, prediction interval (PI) 0.77 to 1.41). For continuous outcomes, the pooled DSMD was - 0.09 (95% CI - 0.26 to 0.09, PI - 0.55 to 0.38). Subgroup analyses yielded that differences in type of intake/exposure were drivers towards disagreement. We replicated the findings of a previous study, where on average RCTs and cohort studies had similar effect estimates. Disagreement and wide prediction intervals were mainly driven by PI/ECO-dissimilarities. More research is needed to explore other potentially influencing factors (e.g. risk of bias) on the disagreement between effect estimates of both BoE.Trial registration: CRD42021278908.

An empirical evaluation of the impact scenario of pooling bodies of evidence from randomized controlled trials and cohort studies in medical research.

BACKGROUND: Randomized controlled trials (RCTs) and cohort studies are the most common study design types used to assess treatment effects of medical interventions. We aimed to hypothetically pool bodies of evidence (BoE) from RCTs with matched BoE from cohort studies included in the same systematic review. METHODS: BoE derived from systematic reviews of RCTs and cohort studies published in the 13 medical journals with the highest impact factor were considered. We re-analyzed effect estimates of the included systematic reviews by pooling BoE from RCTs with BoE from cohort studies using random and common effects models. We evaluated statistical heterogeneity, 95% prediction intervals, weight of BoE from RCTs to the pooled estimate, and whether integration of BoE from cohort studies modified the conclusion from BoE of RCTs. RESULTS: Overall, 118 BoE-pairs based on 653 RCTs and 804 cohort studies were pooled. By pooling BoE from RCTs and cohort studies with a random effects model, for 61 (51.7%) out of 118 BoE-pairs, the 95% confidence interval (CI) excludes no effect. By pooling BoE from RCTs and cohort studies, the median I2 was 48%, and the median contributed percentage weight of RCTs to the pooled estimates was 40%. The direction of effect between BoE from RCTs and pooled effect estimates was mainly concordant (79.7%). The integration of BoE from cohort studies modified the conclusion (by examining the 95% CI) from BoE of RCTs in 32 (27%) of the 118 BoE-pairs, but the direction of effect was mainly concordant (88%). CONCLUSIONS: Our findings provide insights for the potential impact of pooling both BoE in systematic reviews. In medical research, it is often important to rely on both evidence of RCTs and cohort studies to get a whole picture of an investigated intervention-disease association. A decision for or against pooling different study designs should also always take into account, for example, PI/ECO similarity, risk of bias, coherence of effect estimates, and also the trustworthiness of the evidence. Overall, there is a need for more research on the influence of those issues on potential pooling.

Evaluating agreement between bodies of evidence from randomized controlled trials and cohort studies in medical research: a meta-epidemiological study.

BACKGROUND: Randomized controlled trials (RCTs) and cohort studies are the most common study design types used to assess the treatment effects of medical interventions. To evaluate the agreement of effect estimates between bodies of evidence (BoE) from randomized controlled trials (RCTs) and cohort studies and to identify factors associated with disagreement. METHODS: Systematic reviews were published in the 13 medical journals with the highest impact factor identified through a MEDLINE search. BoE-pairs from RCTs and cohort studies with the same medical research question were included. We rated the similarity of PI/ECO (Population, Intervention/Exposure, Comparison, Outcome) between BoE from RCTs and cohort studies. The agreement of effect estimates across BoE was analyzed by pooling ratio of ratios (RoR) for binary outcomes and difference of mean differences for continuous outcomes. We performed subgroup analyses to explore factors associated with disagreements. RESULTS: One hundred twenty-nine BoE pairs from 64 systematic reviews were included. PI/ECO-similarity degree was moderate: two BoE pairs were rated as "more or less identical"; 90 were rated as "similar but not identical" and 37 as only "broadly similar". For binary outcomes, the pooled RoR was 1.04 (95% CI 0.97-1.11) with considerable statistical heterogeneity. For continuous outcomes, differences were small. In subgroup analyses, degree of PI/ECO-similarity, type of intervention, and type of outcome, the pooled RoR indicated that on average, differences between both BoE were small. Subgroup analysis by degree of PI/ECO-similarity revealed high statistical heterogeneity and wide prediction intervals across PI/ECO-dissimilar BoE pairs. CONCLUSIONS: On average, the pooled effect estimates between RCTs and cohort studies did not differ. Statistical heterogeneity and wide prediction intervals were mainly driven by PI/ECO-dissimilarities (i.e., clinical heterogeneity) and cohort studies. The potential influence of risk of bias and certainty of the evidence on differences of effect estimates between RCTs and cohort studies needs to be explored in upcoming meta-epidemiological studies.

The use of the GRADE dose-response gradient domain in nutrition evidence syntheses varies considerably.

OBJECTIVES: This study aimed to identify and describe the use of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) dose-response gradient domain to upgrade the certainty of evidence (CoE) in nutrition systematic reviews (SRs). STUDY DESIGN AND SETTING: We searched for SRs of observational studies of nutrition topics that used GRADE and upgraded the CoE of at least one outcome for a dose-response gradient or reported reasons for not upgrading. RESULTS: Within eligible SRs (21/281), 123 of 371 outcomes were upgraded for a dose-response gradient. For 118 outcomes, the authors conducted linear dose-response analyses, and for 106 outcomes, the authors conducted nonlinear dose-response analyses. From these, 107 outcomes showed a statistically significant (P < 0.05) association in the linear dose-response model, and for 28 outcomes, the test for nonlinearity was statistically significant. The CoE for 0.8% of all outcomes was rated as high, 47.2% as moderate, 43.9% as low, and 8.1% as very low. Fifty-five percent of outcomes that were upgraded for a dose-response gradient were already downgraded for at least one domain. This is contrary to GRADE guidance. CONCLUSION: The approach for rating up the CoE for dose-response relationship is not consistent in nutrition reviews, likely because of a lack of clear guidance for when and how to do it. Therefore, more comprehensive GRADE guidance is necessary to enhance the correct use and comparability of dose-response upgrading.

Approach bias modification training in bulimia nervosa and binge-eating disorder: A pilot randomized controlled trial.

OBJECTIVE: Bulimia nervosa (BN) and binge-eating disorder (BED) are associated with poorly controlled approach behavior toward food resulting in binge eating. Approach bias modification (ABM) may reduce these automatic action tendencies (i.e., approach bias) toward food and may thus decrease binge eating and related symptoms. METHOD: A total of 56 patients with BN/BED participated in this double-blind, randomized controlled trial (RCT) comparing real and sham ABM. The real ABM condition adopted an implicit learning paradigm in which participants were trained to show avoidance behavior in response to food cues. Participants in the sham condition used a similar task but were not trained to avoid food cues. Both conditions comprised 10 training sessions within 4 weeks. RESULTS: Participants in both groups experienced significant reductions in binge eating, eating disorder symptoms, trait food craving, and food cue reactivity. Real ABM tended to result in greater reductions in eating disorder symptoms than sham ABM. Food intake, approach bias, and attention bias toward food did not change. DISCUSSION: This is the first RCT on ABM in eating disorders. The findings provide limited support for the efficacy of ABM in BN/BED and pose questions regarding its active ingredients and its usefulness as a stand-alone treatment for eating disorders.

Food cue-induced craving in individuals with bulimia nervosa and binge-eating disorder.

Individuals with bulimia nervosa (BN) or binge-eating disorder (BED) experience more frequent and intense food cravings than individuals without binge eating. However, it is currently unclear whether they also show larger food cue-induced increases in craving (i.e., food cue reactivity) than those without binge eating, as suggested by conditioning theories of binge eating. A group of individuals with BN or BED (binge-eating group, n = 27) and a group of individuals with low trait food craving scores and without binge eating (control group, n = 19) reported their current food craving before and after a food cue exposure. Although food craving intensity significantly increased in both groups, this increase was significantly stronger in the binge-eating group than in the control group. This result is in line with conditioning models of binge eating that propose that food cues are conditioned stimuli that elicit a conditioned response (e.g., food craving) and that this association is stronger in individuals with binge eating. As food craving increased in individuals with low trait food craving scores as well-although to a lesser extent-previous null results might be explained by methodological considerations such as not screening control participants for trait food craving.