Sensitivity and Specificity of Using GPT-3.5 Turbo Models for Title and Abstract Screening in Systematic Reviews and Meta-analyses.

BACKGROUND: Systematic reviews are performed manually despite the exponential growth of scientific literature. OBJECTIVE: To investigate the sensitivity and specificity of GPT-3.5 Turbo, from OpenAI, as a single reviewer, for title and abstract screening in systematic reviews. DESIGN: Diagnostic test accuracy study. SETTING: Unannotated bibliographic databases from 5 systematic reviews representing 22 665 citations. PARTICIPANTS: None. MEASUREMENTS: A generic prompt framework to instruct GPT to perform title and abstract screening was designed. The output of the model was compared with decisions from authors under 2 rules. The first rule balanced sensitivity and specificity, for example, to act as a second reviewer. The second rule optimized sensitivity, for example, to reduce the number of citations to be manually screened. RESULTS: Under the balanced rule, sensitivities ranged from 81.1% to 96.5% and specificities ranged from 25.8% to 80.4%. Across all reviews, GPT identified 7 of 708 citations (1%) missed by humans that should have been included after full-text screening at the cost of 10 279 of 22 665 false-positive recommendations (45.3%) that would require reconciliation during the screening process. Under the sensitive rule, sensitivities ranged from 94.6% to 99.8% and specificities ranged from 2.2% to 46.6%. Limiting manual screening to citations not ruled out by GPT could reduce the number of citations to screen from 127 of 6334 (2%) to 1851 of 4077 (45.4%), at the cost of missing from 0 to 1 of 26 citations (3.8%) at the full-text level. LIMITATIONS: Time needed to fine-tune prompt. Retrospective nature of the study, convenient sample of 5 systematic reviews, and GPT performance sensitive to prompt development and time. CONCLUSION: The GPT-3.5 Turbo model may be used as a second reviewer for title and abstract screening, at the cost of additional work to reconcile added false positives. It also showed potential to reduce the number of citations before screening by humans, at the cost of missing some citations at the full-text level. PRIMARY FUNDING SOURCE: None.

Substitution of Dairy Products and Risk of Death and Cardiometabolic Diseases: A Systematic Review and Meta-Analysis of Prospective Studies.

Substitution models in epidemiologic studies specifying both substitute and substituted food in relation to disease risk may be useful to inform dietary guidelines. A systematic review of prospective observational studies was performed to quantify the risks of all-cause mortality, cardiovascular disease, and type 2 diabetes (T2D) associated with the substitution of dairy products with other foods and between different dairy products. We systematically searched MEDLINE, Embase, and Web of Science until 28th June, 2023. We calculated summary relative risks (SRRs) and 95% confidence intervals (95% CI) in random-effects meta-analyses. We assessed the risk of bias with the Risk Of Bias In Non-randomized Studies - of Exposure (ROBINS-E) tool and certainty of evidence (CoE) using the Grading of Recommendations Assessment, Development, and Evaluations (GRADE) approach. Fifteen studies (with 34 publications) were included. There was moderate CoE that the substitution of low-fat dairy with red meat was associated with a higher risk of mortality, coronary artery disease, and T2D [SRR (95% CI): 1.11 (1.06, 1.16), 1.13 (1.08, 1.18), and 1.20 (1.16, 1.25)]. A higher risk of mortality and T2D was also observed when substituting low-fat dairy with processed meat [SRR (95% CI): 1.19 (1.11, 1.28) and 1.41 (1.33, 1.49); moderate CoE]. A lower mortality risk was associated with the substitution of dairy and yogurt with whole grains [SRR (95% CI): 0.89 (0.84, 0.93) and 0.91 (0.85, 0.97)], and butter with olive oil [SRR (95% CI): 0.94 (0.92, 0.97); all moderate CoE]. Mainly no associations were observed when substituting dairy products against each other on disease and mortality risk. Our findings indicate associations between substituting dairy with red or processed meat and higher disease risk, whereas its substitution with whole grains was associated with a lower risk. However, there is little robust evidence that substituting whole-fat with low-fat dairy is associated with disease risk. (CRD42022303198).

Evaluating agreement between evidence from randomised controlled trials and cohort studies in nutrition: a meta-research replication study.

This meta-research study aims to evaluate the agreement of effect estimates between bodies of evidence (BoE) from RCTs and cohort studies included in the same nutrition evidence synthesis, to identify factors associated with disagreement, and to replicate the findings of a previous study. We searched Medline, Epistemonikos and the Cochrane Database of Systematic Reviews for nutrition systematic reviews that included both RCTs and cohort studies for the same patient-relevant outcome or intermediate-disease marker. We rated similarity of PI/ECO (population, intervention/exposure, comparison, outcome) between BoE from RCTs and cohort studies. Agreement of effect estimates across BoE was analysed by pooling ratio of risk ratios (RRR) for binary outcomes and difference of standardised mean differences (DSMD) for continuous outcomes. We performed subgroup and sensitivity analyses to explore determinants associated with disagreements. We included 82 BoE-pairs from 51 systematic reviews. For binary outcomes, the RRR was 1.04 (95% confidence interval (CI) 0.99 to 1.10, I2 = 59%, τ2 = 0.02, prediction interval (PI) 0.77 to 1.41). For continuous outcomes, the pooled DSMD was - 0.09 (95% CI - 0.26 to 0.09, PI - 0.55 to 0.38). Subgroup analyses yielded that differences in type of intake/exposure were drivers towards disagreement. We replicated the findings of a previous study, where on average RCTs and cohort studies had similar effect estimates. Disagreement and wide prediction intervals were mainly driven by PI/ECO-dissimilarities. More research is needed to explore other potentially influencing factors (e.g. risk of bias) on the disagreement between effect estimates of both BoE.Trial registration: CRD42021278908.

Clinical and methodological implications for research elements in systematic reviews on COVID-19 treatment were often unstructured and under-reported: a metaresearch study.

OBJECTIVES: Numerous systematic reviews (SRs) have been published in the first months of the COVID-19 pandemic and clinical trials were designed rapidly highlighting the importance of informative implications for research (IfRs) sections in SRs. IfR is one item of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 checklist and the Cochrane Handbook suggests considering population, intervention, control, outcome (PICO) and Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) domains when developing IfR. We aimed (1) to assess whether SRs on COVID-19 treatments included any IfR statements and, for SRs with an IfR statement, (2) to examine which elements informed the IfR statement. STUDY DESIGN AND SETTING: We conducted a metaresearch study based on SRs on COVID-19 treatment identified in the Living OVerview of the Evidence COVID-19 database in May 2021 as part of another research project (CRD42021240423). We defined an IfR statement as at least one sentence that contained at least one bit of information that could be informative for planning future research. We extracted any IfR statements anywhere in the SRs on predefined IfR variables, in particular PICO elements, study design, and concepts underlying GRADE domains. Three authors extracted data independently after piloting the data extraction form. We resolved discrepancies in weekly discussions to ensure a high-quality data extraction. RESULTS: We included 326 SRs, of which 284 SRs (87.1%) stated IfR. Of these 284 SRs, 201 (70.8%) reported using Preferred Reporting Items for Systematic Reviews and Meta-Analyses and 66 (23.2%) using GRADE. IfR statements (n = 284) addressing PICO were unstructured and commonly reported 'population' (n = 195, 68.7%), 'intervention' (n = 242, 85.2%), and 'outcome' (n = 127, 44.7%) but not 'control' (n = 29, 10.2%). Concepts underlying GRADE domains were infrequently reported in IfR statements of SRs (n = 284): 'risk of bias' (n = 14, 4.9%), 'imprecision' (n = 8, 2.8%), 'inconsistency' (n = 7, 2.5%), 'publication bias' (n = 3, 1.1%), and 'indirectness' (n = 1, 0.4%). Additional IfR elements mentioned in IfR were 'better reporting' of future studies (n = 17, 6.0%) and 'standardization of procedures in clinical trials' (n = 12, 4.2%). CONCLUSION: Almost 90% of SRs on COVID-19 treatments reported IfR. IfR statements addressing PICO were unstructured across SRs and concepts underlying GRADE were rarely reported to inform IfR. Further work is needed to assess generalizability beyond COVID-19 and to define more precisely which IfR elements should be considered, and how they should be reported in SRs of interventions. Until then, considering PICO elements and concepts underlying GRADE to derive IfR seems to be a sensible starting point.

Substitution of animal-based with plant-based foods on cardiometabolic health and all-cause mortality: a systematic review and meta-analysis of prospective studies.

BACKGROUND: There is growing evidence that substituting animal-based with plant-based foods is associated with a lower risk of cardiovascular diseases (CVD), type 2 diabetes (T2D), and all-cause mortality. Our aim was to summarize and evaluate the evidence for the substitution of any animal-based foods with plant-based foods on cardiometabolic health and all-cause mortality in a systematic review and meta-analysis. METHODS: We systematically searched MEDLINE, Embase, and Web of Science to March 2023 for prospective studies investigating the substitution of animal-based with plant-based foods on CVD, T2D, and all-cause mortality. We calculated summary hazard ratios (SHRs) and 95% confidence intervals (95% CI) using random-effects meta-analyses. We assessed the certainty of evidence (CoE) using the GRADE approach. RESULTS: In total, 37 publications based on 24 cohorts were included. There was moderate CoE for a lower risk of CVD when substituting processed meat with nuts [SHR (95% CI): 0.73 (0.59, 0.91), n = 8 cohorts], legumes [0.77 (0.68, 0.87), n = 8], and whole grains [0.64 (0.54, 0.75), n = 7], as well as eggs with nuts [0.83 (0.78, 0.89), n = 8] and butter with olive oil [0.96 (0.95, 0.98), n = 3]. Furthermore, we found moderate CoE for an inverse association with T2D incidence when substituting red meat with whole grains/cereals [0.90 (0.84, 0.96), n = 6] and red meat or processed meat with nuts [0.92 (0.90, 0.94), n = 6 or 0.78 (0.69, 0.88), n = 6], as well as for replacing poultry with whole grains [0.87 (0.83, 0.90), n = 2] and eggs with nuts or whole grains [0.82 (0.79, 0.86), n = 2 or 0.79 (0.76, 0.83), n = 2]. Moreover, replacing red meat for nuts [0.93 (0.91, 0.95), n = 9] and whole grains [0.96 (0.95, 0.98), n = 3], processed meat with nuts [0.79 (0.71, 0.88), n = 9] and legumes [0.91 (0.85, 0.98), n = 9], dairy with nuts [0.94 (0.91, 0.97), n = 3], and eggs with nuts [0.85 (0.82, 0.89), n = 8] and legumes [0.90 (0.89, 0.91), n = 7] was associated with a reduced risk of all-cause mortality. CONCLUSIONS: Our findings indicate that a shift from animal-based (e.g., red and processed meat, eggs, dairy, poultry, butter) to plant-based (e.g., nuts, legumes, whole grains, olive oil) foods is beneficially associated with cardiometabolic health and all-cause mortality.

In Cochrane nutrition reviews assessment of dietary adherence varied considerably.

OBJECTIVES: Our aim was to investigate if and how Cochrane nutrition reviews assess dietary adherence to a specific dietary regimen. STUDY DESIGN AND SETTING: Cochrane nutrition reviews fulfilling the following criteria were included: systematic review of randomized controlled trials including adults and investigating the effect of caloric restriction, dietary pattern, foods, nutrients, supplements, or other nutrition-related-interventions. Extensive data extraction and descriptive statistics were conducted. RESULTS: Overall, 226 Cochrane reviews were included. Most reviews mentioned dietary adherence in the main text (n = 174), predominantly in the Methods and Results. Dietary adherence was assessed in 76 reviews and defined in 19. It was included in the risk of bias (RoB) assessment in 20 reviews with nine using a newly created RoB domain for dietary adherence, and considered as outcome in 37 reviews. Seventy-five reviews addressed degree of adherence and five treatment effects considering the degree of adherence. CONCLUSION: Dietary adherence was reported in a heterogeneous manner in Cochrane nutrition reviews. Due to its high importance, we suggest that systematic reviews report the assessment and degree of dietary adherence measured in primary studies. Dietary adherence can further be examined as outcome, evaluated within the RoB (deviations from intended interventions) and included in sensitivity analyses.

Effects of Dairy Intake on Markers of Cardiometabolic Health in Adults: A Systematic Review with Network Meta-Analysis.

The health effects of dairy products are still a matter of scientific debate owing to inconsistent findings across trials. Therefore, this systematic review and network meta-analysis (NMA) aimed to compare the effects of different dairy products on markers of cardiometabolic health. A systematic search was conducted in 3 electronic databases [MEDLINE, Cochrane Central Register of Controlled Trials (CENTRAL), and Web of Science; search date: 23 September 2022]. This study included randomized controlled trials (RCTs) with a ≥12-wk intervention comparing any 2 of the eligible interventions [e.g., high dairy (≥3 servings/d or equal amount in grams per day), full-fat dairy, low-fat dairy, naturally fermented milk products, and low dairy/control (0-2 servings/d or usual diet)]. A pairwise meta-analysis and NMA using random-effects model was performed in the frequentist framework for 10 outcomes [body weight, BMI, fat mass, waist circumference, low-density lipoprotein cholesterol, high-density lipoprotein (HDL) cholesterol, triglycerides, fasting glucose, glycated hemoglobin, and systolic blood pressure]. Continuous outcome data were pooled using mean differences (MDs) and dairy interventions ranked using the surface under the cumulative ranking curve. Nineteen RCTs with 1427 participants were included. High-dairy intake (irrespective of fat content) showed no detrimental effects on anthropometric outcomes, blood lipids, and blood pressure. Both low-fat and full-fat dairy improved systolic blood pressure (MD: -5.22 to -7.60 mm Hg; low certainty) but, concomitantly, may impair glycemic control (fasting glucose-MD: 0.31-0.43 mmol/L; glycated hemoglobin-MD: 0.37%-0.47%). Full-fat dairy may increase HDL cholesterol compared with a control diet (MD: 0.26 mmol/L; 95% CI: 0.03, 0.49 mmol/L). Yogurt improved waist circumference (MD: -3.47 cm; 95% CI: -6.92, -0.02 cm; low certainty), triglycerides (MD: -0.38 mmol/L; 95% CI: -0.73, -0.03 mmol/L; low certainty), and HDL cholesterol (MD: 0.19 mmol/L; 95% CI: 0.00, 0.38 mmol/L) compared with milk. In conclusion, our findings indicate that there is little robust evidence that a higher dairy intake has detrimental effects on markers of cardiometabolic health. This review was registered at PROSPERO as CRD42022303198.

Self-management in adults after solid-organ transplantation: a scoping review protocol.

INTRODUCTION: After solid-organ transplantation (SOTx), recipients must adhere to a lifelong medical regimen, change their lifestyle and cope with physiological and psychosocial challenges. This requires active participation in their care and self-management abilities. The concept of self-management after SOTx has only been described regarding specific organs and focused on adherence to medical treatment. A comprehensive conceptualisation of self-management entailing all solid organs and beyond medical aspects does not exist. This might lead to unmet self-management support needs of SOTx recipients and hinder a more holistic and integrative approach in self-management support. Therefore, a better understanding of the concept of self-management after SOTx is needed to facilitate a comprehensive evidence base for healthcare providers and researchers. The purpose of this scoping review is to explore existing evidence on self-management in adults after SOTx. METHODS AND ANALYSIS: To identify relevant evidence, six electronic databases and three study registers will be searched, supplemented by handsearches, reference checking and expert recommendations. Screening and selection of available evidence will be carried out in a two-step process by two independent reviewers. International evidence published in English or German reporting on adults after heart, lung, liver, pancreas, kidney or small bowel transplantation will be considered. To meet inclusion criteria, articles have to focus on either: self-management, self-management support or recipients' or healthcare providers' perspectives of challenges and needs potentially addressable by self-management. Data extraction will be performed by two reviewers independently using a structured form. Data will be analysed descriptively and using content analysis procedures. Findings will be summarised narratively and presented in tabular format. ETHICS AND DISSEMINATION: The consultation and approval of an ethics committee is not required for this scoping review. Findings of the scoping review will be published in a peer-reviewed open-access journal and presented at conferences.

Using GRADE Evidence to Decision frameworks to support the process of health policy-making: an example application regarding taxation of sugar-sweetened beverages.

BACKGROUND: Grading of Recommendations, Assessment, Development and Evaluation (GRADE) Evidence to Decision (EtD) frameworks are well-known tools that enable guideline panels to structure the process of developing recommendations and making decisions in healthcare and public health. To date, they have not regularly been used for health policy-making. This article aims to illustrate the application of the GRADE EtD frameworks in the process of nutrition-related policy-making for a European country. METHODS: Based on methodological guidance by the GRADE Working Group and the findings of our recently published scoping review, we illustrate the process of moving from evidence to recommendations, by applying the EtD frameworks to a fictitious example. Sugar-sweetened beverage (SSB) taxation based on energy density was chosen as an example application. RESULTS: A fictitious guideline panel was convened by a national nutrition association to develop a population-level recommendation on SSB taxation aiming to reduce the burden of overweight and obesity. Exemplary evidence was summarized for each EtD criterion and conclusions were drawn based on all judgements made in relation to each criterion. As a result of the high priority to reduce the burden of obesity and because of the moderate desirable effects on health outcomes, but considering scarce or varying research evidence for other EtD criteria, the panel made a conditional recommendation for SSB taxation. Decision-makers may opt for conducting a pilot study prior to implementing the policy on a national level. CONCLUSIONS: GRADE EtD frameworks can be used by guideline panels to make the process of developing recommendations in the field of health policy more systematic, transparent and comprehensible.

An empirical evaluation of the impact scenario of pooling bodies of evidence from randomized controlled trials and cohort studies in medical research.

BACKGROUND: Randomized controlled trials (RCTs) and cohort studies are the most common study design types used to assess treatment effects of medical interventions. We aimed to hypothetically pool bodies of evidence (BoE) from RCTs with matched BoE from cohort studies included in the same systematic review. METHODS: BoE derived from systematic reviews of RCTs and cohort studies published in the 13 medical journals with the highest impact factor were considered. We re-analyzed effect estimates of the included systematic reviews by pooling BoE from RCTs with BoE from cohort studies using random and common effects models. We evaluated statistical heterogeneity, 95% prediction intervals, weight of BoE from RCTs to the pooled estimate, and whether integration of BoE from cohort studies modified the conclusion from BoE of RCTs. RESULTS: Overall, 118 BoE-pairs based on 653 RCTs and 804 cohort studies were pooled. By pooling BoE from RCTs and cohort studies with a random effects model, for 61 (51.7%) out of 118 BoE-pairs, the 95% confidence interval (CI) excludes no effect. By pooling BoE from RCTs and cohort studies, the median I2 was 48%, and the median contributed percentage weight of RCTs to the pooled estimates was 40%. The direction of effect between BoE from RCTs and pooled effect estimates was mainly concordant (79.7%). The integration of BoE from cohort studies modified the conclusion (by examining the 95% CI) from BoE of RCTs in 32 (27%) of the 118 BoE-pairs, but the direction of effect was mainly concordant (88%). CONCLUSIONS: Our findings provide insights for the potential impact of pooling both BoE in systematic reviews. In medical research, it is often important to rely on both evidence of RCTs and cohort studies to get a whole picture of an investigated intervention-disease association. A decision for or against pooling different study designs should also always take into account, for example, PI/ECO similarity, risk of bias, coherence of effect estimates, and also the trustworthiness of the evidence. Overall, there is a need for more research on the influence of those issues on potential pooling.

Empirical evidence of study design biases in nutrition randomised controlled trials: a meta-epidemiological study.

BACKGROUND: Instruments to critically appraise randomised controlled trials (RCTs) are based on evidence from meta-epidemiological studies. We aim to conduct a meta-epidemiological study on the average bias associated with reported methodological trial characteristics such as random sequence generation, allocation concealment, blinding, incomplete outcome data, selective reporting, and compliance of RCTs in nutrition research. METHODS: We searched the Cochrane Database of Systematic Reviews, for systematic reviews of RCTs, published between 01 January 2010 and 31 December 2019. We combined the estimates of the average bias (e.g. ratio of risk ratios [RRR] or differences in standardised mean differences) in meta-analyses using the random-effects model. Subgroup analyses were conducted to investigate the potential differences among the RCTs with low versus high/unclear risk of bias with respect to the different types of interventions (e.g. micronutrients, fatty acids, dietary approach), outcomes (e.g. mortality, pregnancy outcomes), and type of outcome (objective, subjective). Heterogeneity was assessed through I2 and τ2, and prediction intervals were calculated. RESULTS: We included 27 Cochrane nutrition reviews with 77 meta-analyses (n = 927 RCTs). The available evidence suggests that intervention effect estimates may not be exaggerated in RCTs with high/unclear risk of bias (versus low) judgement for sequence generation (RRR 0.97, 95% CI 0.93 to 1.02; I2 = 28%; τ2 = 0.002), allocation concealment (RRR 1.00, 95% CI 0.96 to 1.04; I2 = 27%; τ2 = 0.001), blinding of participants and personnel (RRR 0.95, 95% CI 0.91 to 1.00; I2 = 23%; τ2 = 0), selective reporting (RRR 0.97, 95% CI 0.92 to 1.02; I2 = 24%; τ2 = 0), and compliance (RRR 0.95, 95% CI 0.89 to 1.02; I2 = 0%; τ2 = 0). Intervention effect estimates seemed to be exaggerated in RCTs with a high/unclear risk of bias judgement for blinding of outcome assessment (RRR 0.81, 95% CI 0.70 to 0.94; I2 = 26%; τ2 = 0.03), which was predominately driven by subjective outcomes, and incomplete outcome data (RRR 0.92, 95% CI 0.88 to 0.97; I2 = 22%; τ2 = 0.001). For continuous outcomes, no differences were observed, except for selective reporting. CONCLUSIONS: On average, most characteristics of nutrition RCTs may not exaggerate intervention effect estimates, but the average bias appears to be greatest in trials of subjective outcomes. Replication of this study is suggested in this field to keep this conclusion updated.

Three out of four published systematic reviews on COVID-19 treatments were not registered and one-third of those registered were published: a meta-research study.

OBJECTIVES: The aim of this study is to describe (1) registered and (2) published systematic reviews (SRs) on COVID-19 treatments, and to analyze (3) the proportion of publications among registered SRs and (4) the proportion of registrations among published SRs. STUDY DESIGN AND SETTING: This meta-research study (CRD42021240423) is part of CEOsys (http://www.covid-evidenz.de/). Two reviewers identified protocols in PROSPERO (registered January 2020 to September 2020) and SRs published as preprint or peer-reviewed article in L·OVE (Living OVerview of the Evidence) COVID-19 (by May 2021). SRs of all types assessing COVID-19 treatments in humans were included. RESULTS: We included 239 PROSPERO protocols and 346 SRs published in L·OVE. In both samples, the affiliation of the corresponding author with an Asian institution, standard SR as review type, and meta-analysis as synthesis method were the most frequent characteristics. Living SRs made up ≤10%. A total of 71 of 239 (29.7%) PROSPERO protocols were published as SR by February 2022, that is, after at least 17 months of follow-up (25 of 71 as preprints, 35.2%). In L·OVE, 261 of 346 (75.4%) SRs published by May 2021 were not registered in PROSPERO. CONCLUSION: Overall, one-third PROSPERO protocols were published and three-fourth published SRs were not registered. We strongly encourage authors to register and publish their SRs promptly to reduce research waste and to allocate resources efficiently during the pandemic and beyond.

Willingness to participate in, support or carry out scientific studies for benefit assessment of available medical interventions: A stakeholder survey.

BACKGROUND: Post-entry studies are a key element in managed entry agreements and aim at generating evidence about the additional benefit of new medical interventions before reimbursement decisions are made. This study evaluates the willingness of different stakeholder groups to engage post-entry in studies for benefit assessment and to assess differences in their willingness by study type, i.e. randomised controlled trial or observational study. METHODS: We conducted a cross-sectional, web-based survey with a self-administrated questionnaire in German language. We disseminated invitations to patients, patient representatives, healthcare providers, trialists & scientists and representatives of the medical private sector, using a snowball system, public contact details of German associations and organisations, and social media. We analysed quantitative data descriptively and qualitative data inductively. RESULTS: Data of 154 respondents were available for analysis. The majority (>85%) was willing to engage in the studies in general, and regarding different study types. Scientists reported a higher willingness to conduct and support RCTs (p = 0.01) as compared to observational studies. Representatives of the private sector were mainly willing to support, but not to carry out post-entry studies. Stakeholders frequently mentioned that potential personal benefit and altruistic motives were relevant for their decision to engage in studies. Practical inconveniences, poor integration into daily life, high demand for time and personnel, and lack of resources were commonly mentioned barriers. DISCUSSION AND CONCLUSION: Stakeholders clearly reported to be willing to engage in post-entry studies for benefit assessment. Self-reported willingness to participate in and support for studies seems higher than practical recruitment rates. The survey might be subject to survey error and self-enhancement of participants. Inquiring about the willingness of hypothetical studies might have caused participants to report higher willingness. Motives for and against participation may be possible starting points for approaches to overcome recruitment difficulties and facilitate successful study conduct.

Unintended consequences of measures implemented in the school setting to contain the COVID-19 pandemic: a scoping review.

BACKGROUND: With the emergence of SARS-CoV-2 in late 2019, governments worldwide implemented a multitude of non-pharmaceutical interventions in order to control the spread of the virus. Most countries have implemented measures within the school setting in order to reopen schools or keep them open whilst aiming to contain the spread of SARS-CoV-2. For informed decision-making on implementation, adaptation, or suspension of such measures, it is not only crucial to evaluate their effectiveness with regard to SARS-CoV-2 transmission, but also to assess their unintended consequences. OBJECTIVES: To comprehensively identify and map the evidence on the unintended health and societal consequences of school-based measures to prevent and control the spread of SARS-CoV-2. We aimed to generate a descriptive overview of the range of unintended (beneficial or harmful) consequences reported as well as the study designs that were employed to assess these outcomes. This review was designed to complement an existing Cochrane Review on the effectiveness of these measures by synthesising evidence on the implications of the broader system-level implications of school measures beyond their effects on SARS-CoV-2 transmission. SEARCH METHODS: We searched the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, Embase, four non-health databases, and two COVID-19 reference collections on 26 March 2021, together with reference checking, citation searching, and Google searches. SELECTION CRITERIA: We included quantitative (including mathematical modelling), qualitative, and mixed-methods studies of any design that provided evidence on any unintended consequences of measures implemented in the school setting to contain the SARS-CoV-2 pandemic. Studies had to report on at least one unintended consequence, whether beneficial or harmful, of one or more relevant measures, as conceptualised in a logic model.  DATA COLLECTION AND ANALYSIS: We screened the titles/abstracts and subsequently full texts in duplicate, with any discrepancies between review authors resolved through discussion. One review author extracted data for all included studies, with a second review author reviewing the data extraction for accuracy. The evidence was summarised narratively and graphically across four prespecified intervention categories and six prespecified categories of unintended consequences; findings were described as deriving from quantitative, qualitative, or mixed-method studies. MAIN RESULTS: Eighteen studies met our inclusion criteria. Of these, 13 used quantitative methods (3 experimental/quasi-experimental; 5 observational; 5 modelling); four used qualitative methods; and one used mixed methods. Studies looked at effects in different population groups, mainly in children and teachers. The identified interventions were assigned to four broad categories: 14 studies assessed measures to make contacts safer; four studies looked at measures to reduce contacts; six studies assessed surveillance and response measures; and one study examined multiple measures combined. Studies addressed a wide range of unintended consequences, most of them considered harmful. Eleven studies investigated educational consequences. Seven studies reported on psychosocial outcomes. Three studies each provided information on physical health and health behaviour outcomes beyond COVID-19 and environmental consequences. Two studies reported on socio-economic consequences, and no studies reported on equity and equality consequences. AUTHORS' CONCLUSIONS: We identified a heterogeneous evidence base on unintended consequences of measures implemented in the school setting to prevent and control the spread of SARS-CoV-2, and summarised the available study data narratively and graphically. Primary research better focused on specific measures and various unintended outcomes is needed to fill knowledge gaps and give a broader picture of the diverse unintended consequences of school-based measures before a more thorough evidence synthesis is warranted. The most notable lack of evidence we found was regarding psychosocial, equity, and equality outcomes. We also found a lack of research on interventions that aim to reduce the opportunity for contacts. Additionally, study investigators should provide sufficient data on contextual factors and demographics in order to ensure analyses of such are feasible, thus assisting stakeholders in making appropriate, informed decisions for their specific circumstances.

Characteristics of registered and published systematic reviews focusing on the prevention of COVID-19: a meta-research study.

OBJECTIVE: We investigated characteristics of systematic reviews (SRs) assessing measures to prevent COVID-19 by (1) identifying SR registrations in Prospective Register of Systematic Reviews (PROSPERO), (2) identifying published SRs in COVID-19 Living Overview of the Evidence (L-OVE) and (3) estimating the proportion of PROSPERO registrations published as full SR between 8 and 16 months after registration. STUDY DESIGN: This meta-research study is part of the German CEOsys project, www.covid-evidenz.de. We searched PROSPERO entries registered between 1 January 2020 and 31 August 2020, and we searched COVID-19 L-OVE for published SRs (search date: 5 May 2021) focusing on measures to prevent COVID-19 and SARS-CoV-2 transmission. The two samples were screened for eligibility and key characteristics were extracted and summarised. RESULTS: Of 612 PROSPERO registrations, 47 focused on prevention and were included. The preventive measures included public health interventions (20), followed by personal protective equipment (10), vaccinations (9) and others (8). In total, 13 of 47 (28%) PROSPERO registrations had been published as full SR (as preprint only (6), as peer-reviewed article only (6), as preprint and peer-reviewed article (1)). Median time between PROSPERO registration and publication was 5 months for peer-reviewed SRs and 2 months for preprints.Of the 2182 entries identified in COVID-19 L-OVE, 51 published SRs focused on prevention and were included. Similar to the PROSPERO sample, most published SRs focused on public health interventions (21). The number of included primary studies ranged between 0 and 64 (median: 7). Nine published SRs did not include any studies because of a lack of primary studies. CONCLUSION: Considering the urgent information needs of policymakers and the public, our findings reveal the high-speed publication of preprints and lack of primary studies in the beginning of the COVID-19 crisis. Further meta-research on COVID-19 SRs is important to improve research efficiency among researchers across the world. PROSPERO REGISTRATION NUMBER: CRD42021240423.

Pooling of cohort studies and RCTs affects GRADE certainty of evidence in nutrition research.

BACKGROUND: There is a little empirical evidence of the impact of pooling randomized controlled trials (RCTs) and cohort studies (CSs) on the certainty-of-evidence. To evaluate the hypothetical-scenario of pooling bodies-of-evidence from RCTs with matched bodies-of-evidence from CSs on the certainty-of-evidence. METHODS: We extracted GRADE ratings of bodies-of-evidence from RCTs in Cochrane reviews, and rated the certainty-of-evidence from matched bodies-of-evidence from CSs. We then evaluated the impact of pooling both bodies-of-evidence on the overall certainty-of-evidence, and on individual GRADE domains. RESULTS: Fourty-two pooled bodies-of-evidence were rated, ranging from very-low (bodies-of-evidenceRCTs: 9.5%; bodies-of-evidenceCSs: 40.5%; pooled-bodies-of-evidence: 0%) to low (bodies-of-evidenceRCTs: 38.1%; bodies-of-evidenceCSs: 45.2%; pooled-bodies-of-evidence: 19.1%), moderate (bodies-of-evidenceRCTs: 33.4%; bodies-of-evidenceCSs: 14.3%; pooled-bodies-of-evidence: 57.1%), and high (bodies-of-evidenceRCTs: 19%; bodies-of-evidenceCSs: 0%; pooled-bodies-of-evidence: 23.8%). Certainty-of-evidence was downgraded mostly for imprecision and risk of bias for bodies-of-evidence from RCTs, and for risk of bias and inconsistency for bodies-of-evidence from CSs. Pooling both bodies-of-evidence mitigates rating down for imprecision compared to bodies-of-evidence from RCTs and inconsistency compared to bodies-of-evidence from CSs. CONCLUSION: Our hypothetical study suggests that pooling both bodies-of-evidence would reduce the amount of very-low and low certainty-of-evidence ratings, but how to integrate RCTs and CSs and whether or not to pool these bodies-of-evidence requires proper guidance before systematic review authors or guideline developers should consider this approach.

Effects of nutritional intervention strategies in the primary prevention of overweight and obesity in school settings: systematic review and network meta-analysis.

Objective: To examine the effects of different nutritional intervention strategies in the school setting on anthropometric and quality of diet outcomes by comparing and ranking outcomes in a network meta-analysis. Design: Systematic review and network meta-analysis. Data sources: PubMed, Cochrane Central Register of Controlled Trials (CENTRAL), Web of Science, Education Resources Information Centre (ERIC), PsycInfo, CAB Abstracts, Campbell Library, Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre) BiblioMap, Australian Education Index, Joanna Briggs Institute Evidence-Based Practice (JBI EBP) database, Practice-based Evidence in Nutrition (PEN) database, ClinicalTrials.gov, Current Controlled Trials, and World Health Organization International Clinical Trials Registry Platform. Eligibility criteria for selecting studies: A systematic literature search was performed from inception to 2 May 2022. Cluster randomised controlled trials meeting these study criteria were included: generally healthy school students aged 4-18 years; intervention with ≥1 nutritional components in a school setting; and studies that assessed anthropometric measures (eg, body mass index, body fat) or measures related to the quality of diet (eg, intake of fruit and vegetables), or both. Random effects pairwise meta-analyses and network meta-analyses were performed with a frequentist approach. P scores, a frequentist analogue to surface under the cumulative ranking curve, ranging from 0 to 1 (indicating worst and best ranked interventions, respectively) were calculated. Risk of bias was assessed with Cochrane's RoB 2 tool. The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) framework was used to rate the certainty of evidence. Results: 51 cluster randomised controlled trials involving 75 954 participants and seven intervention nodes were included. Inconsistency could not be assessed (except for intake of fruit and vegetables) because the network meta-analyses were based mainly on star shaped networks with no direct evidence for specific pairs of nutritional interventions. Overall, little or no evidence was found to support a difference in body mass index, body weight, body fat, or waist circumference and moderate improvements in intake of fruit and vegetables with nutritional interventions in a school setting. Low to moderate certainty of evidence further suggested that multicomponent nutritional interventions likely reduced the prevalence (odds ratio 0.66, 95% confidence interval 0.55 to 0.80) and incidence (0.67, 0.47 to 0.96) of overweight compared with a control group. Based on low certainty of evidence, nutrition education and multicomponent interventions may be more effective than a control group (ie, usual practice) for increasing intake of fruit and vegetables. Multicomponent nutritional interventions were ranked the most effective for reducing body mass index (P score 0.76) and intake of fat (0.82). Nutrition education was ranked as best for body mass index z score (0.99), intake of fruit and vegetables (0.82), intake of fruit (0.92), and intake of vegetables (0.88). Conclusions: The findings suggest that nutritional interventions in school settings may improve anthropometric and quality of diet measures, potentially contributing to the prevention of overweight and obesity in childhood and adolescence. The findings should be interpreted with caution because the certainty of evidence was often rated as low. The results of the network meta-analysis could be used by policy makers in developing and implementing effective, evidence based nutritional intervention strategies in the school setting. Systematic review registration: PROSPERO CRD42020220451.

Evaluating Concordance of Bodies of Evidence from Randomized Controlled Trials, Dietary Intake, and Biomarkers of Intake in Cohort Studies: A Meta-Epidemiological Study.

We aimed to identify and compare empirical data to determine the concordance of diet-disease effect estimates of bodies of evidence (BoE) from randomized controlled trials (RCTs), dietary intake, and biomarkers of dietary intake in cohort studies (CSs). The Cochrane Database of Systematic Reviews and MEDLINE were searched for systematic reviews (SRs) of RCTs and SRs of CSs that investigated both dietary intake and biomarkers of intake published between 1 January 2010 and 31 December 2019. For matched diet-disease associations, the concordance between results from the 3 different BoE was analyzed using 2 definitions: qualitative (e.g., 95% CI within a predefined range) and quantitative (test hypothesis on the z score). Moreover, the differences in the results coming from BoERCTs, BoECSs dietary intake, and BoECSs biomarkers were synthesized to get a pooled ratio of risk ratio (RRR) across all eligible diet-disease associations, so as to compare the 3 BoE. Overall, 49 diet-disease associations derived from 41 SRs were identified and included in the analysis. Twenty-four percent, 10%, and 39% of the diet-disease associations were qualitatively concordant comparing BoERCTs with BoECSs dietary intake, BoERCTs with BoECSs biomarkers, and comparing both BoE from CSs, respectively; 88%, 69%, and 90% of the diet-disease associations were quantitatively concordant comparing BoERCTs with BoECSs dietary intake, BoERCTs with BoECSs biomarkers, and comparing both BoE from CSs, respectively. The pooled RRRs comparing effects from BoERCTs with effects from BoECSs dietary intake were 1.09 (95% CI: 1.06, 1.13) and 1.18 (95% CI: 1.10, 1.25) compared with BoECSs biomarkers. Comparing both BoE from CSs, the difference in the results was also small (RRR: 0.92; 95% CI: 0.88, 0.96). Our findings suggest that BoE from RCTs and CSs are often quantitatively concordant. Prospective SRs in nutrition research should include, whenever possible, BoE from RCTs and CSs on dietary intake and biomarkers of intake to provide the whole picture for an investigated diet-disease association.

Subsequent full publication of qualitative studies presented at United Kingdom Royal College of Nursing Research Conference 2015 and 2016: A follow-up study.

A considerable proportion of quantitative research remains unpublished once completed. Little research has documented non-dissemination and dissemination bias in qualitative research. This study aimed to generate evidence on the extent of non-dissemination in qualitative research. We followed a cohort of qualitative studies presented as conference abstracts to ascertain their subsequent publication status. We searched for subsequent full publication in MEDLINE, in the Cumulative Index to Nursing & Allied Health Literature and in Google Scholar. We matched abstracts to subsequent publications according to authors, method of data collection and phenomenon of interest. Fisher's exact test was calculated to examine associations between study characteristics and publication. Factors potentially associated with time to publication were evaluated with Cox regression analysis. For 91 of 270 included abstracts (33.70%; 95% CI 28.09%-39.68%), no full publication was identified. Factors that were found to be associated with subsequent full publication were oral presentation (OR 4.62; 95% CI 2.43-8.94) and university affiliation (OR 1.96; 95% CI 1.05-3.66). Compared to oral presentations, studies presented as posters took longer time to reach full publication (hazard ratio 0.35, 95% CI 0.21-0.58). This study shows that it was not possible to retrieve a full publication for over one-third of abstracts. Our findings suggest that where this non-dissemination is systematic, it may lead to distortions of the qualitative evidence-base for decision-making through dissemination bias. Our findings are congruent with those of other studies. Further research might investigate non-dissemination of qualitative studies in other disciplines to consolidate our findings.

Non-pharmacological measures implemented in the setting of long-term care facilities to prevent SARS-CoV-2 infections and their consequences: a rapid review.

BACKGROUND: Starting in late 2019, COVID-19, caused by the novel coronavirus SARS-CoV-2, spread around the world. Long-term care facilities are at particularly high risk of outbreaks, and the burden of morbidity and mortality is very high among residents living in these facilities. OBJECTIVES: To assess the effects of non-pharmacological measures implemented in long-term care facilities to prevent or reduce the transmission of SARS-CoV-2 infection among residents, staff, and visitors. SEARCH METHODS: On 22 January 2021, we searched the Cochrane COVID-19 Study Register, WHO COVID-19 Global literature on coronavirus disease, Web of Science, and CINAHL. We also conducted backward citation searches of existing reviews. SELECTION CRITERIA: We considered experimental, quasi-experimental, observational and modelling studies that assessed the effects of the measures implemented in long-term care facilities to protect residents and staff against SARS-CoV-2 infection. Primary outcomes were infections, hospitalisations and deaths due to COVID-19, contaminations of and outbreaks in long-term care facilities, and adverse health effects. DATA COLLECTION AND ANALYSIS: Two review authors independently screened titles, abstracts and full texts. One review author performed data extractions, risk of bias assessments and quality appraisals, and at least one other author checked their accuracy. Risk of bias and quality assessments were conducted using the ROBINS-I tool for cohort and interrupted-time-series studies, the Joanna Briggs Institute (JBI) checklist for case-control studies, and a bespoke tool for modelling studies. We synthesised findings narratively, focusing on the direction of effect. One review author assessed certainty of evidence with GRADE, with the author team critically discussing the ratings. MAIN RESULTS: We included 11 observational studies and 11 modelling studies in the analysis. All studies were conducted in high-income countries. Most studies compared outcomes in long-term care facilities that implemented the measures with predicted or observed control scenarios without the measure (but often with baseline infection control measures also in place). Several modelling studies assessed additional comparator scenarios, such as comparing higher with lower rates of testing. There were serious concerns regarding risk of bias in almost all observational studies and major or critical concerns regarding the quality of many modelling studies. Most observational studies did not adequately control for confounding. Many modelling studies used inappropriate assumptions about the structure and input parameters of the models, and failed to adequately assess uncertainty. Overall, we identified five intervention domains, each including a number of specific measures. Entry regulation measures (4 observational studies; 4 modelling studies) Self-confinement of staff with residents may reduce the number of infections, probability of facility contamination, and number of deaths. Quarantine for new admissions may reduce the number of infections. Testing of new admissions and intensified testing of residents and of staff after holidays may reduce the number of infections, but the evidence is very uncertain. The evidence is very uncertain regarding whether restricting admissions of new residents reduces the number of infections, but the measure may reduce the probability of facility contamination. Visiting restrictions may reduce the number of infections and deaths. Furthermore, it may increase the probability of facility contamination, but the evidence is very uncertain. It is very uncertain how visiting restrictions may adversely affect the mental health of residents. Contact-regulating and transmission-reducing measures (6 observational studies; 2 modelling studies) Barrier nursing may increase the number of infections and the probability of outbreaks, but the evidence is very uncertain. Multicomponent cleaning and environmental hygiene measures may reduce the number of infections, but the evidence is very uncertain. It is unclear how contact reduction measures affect the probability of outbreaks. These measures may reduce the number of infections, but the evidence is very uncertain. Personal hygiene measures may reduce the probability of outbreaks, but the evidence is very uncertain.  Mask and personal protective equipment usage may reduce the number of infections, the probability of outbreaks, and the number of deaths, but the evidence is very uncertain. Cohorting residents and staff may reduce the number of infections, although evidence is very uncertain. Multicomponent contact -regulating and transmission -reducing measures may reduce the probability of outbreaks, but the evidence is very uncertain. Surveillance measures (2 observational studies; 6 modelling studies) Routine testing of residents and staff independent of symptoms may reduce the number of infections. It may reduce the probability of outbreaks, but the evidence is very uncertain. Evidence from one observational study suggests that the measure may reduce, while the evidence from one modelling study suggests that it probably reduces hospitalisations. The measure may reduce the number of deaths among residents, but the evidence on deaths among staff is unclear.  Symptom-based surveillance testing may reduce the number of infections and the probability of outbreaks, but the evidence is very uncertain. Outbreak control measures (4 observational studies; 3 modelling studies) Separating infected and non-infected residents or staff caring for them may reduce the number of infections. The measure may reduce the probability of outbreaks and may reduce the number of deaths, but the evidence for the latter is very uncertain. Isolation of cases may reduce the number of infections and the probability of outbreaks, but the evidence is very uncertain. Multicomponent measures (2 observational studies; 1 modelling study) A combination of multiple infection-control measures, including various combinations of the above categories, may reduce the number of infections and may reduce the number of deaths, but the evidence for the latter is very uncertain. AUTHORS' CONCLUSIONS: This review provides a comprehensive framework and synthesis of a range of non-pharmacological measures implemented in long-term care facilities. These may prevent SARS-CoV-2 infections and their consequences. However, the certainty of evidence is predominantly low to very low, due to the limited availability of evidence and the design and quality of available studies. Therefore, true effects may be substantially different from those reported here. Overall, more studies producing stronger evidence on the effects of non-pharmacological measures are needed, especially in low- and middle-income countries and on possible unintended consequences of these measures. Future research should explore the reasons behind the paucity of evidence to guide pandemic research priority setting in the future.