RESUMO
PurposeTo assess the trustworthiness and impact of preprint trial reports during the COVID-19 pandemic. Data sourcesWHO COVID-19 database and the L-OVE COVID-19 platform by the Epistemonikos Foundation (up to August 3rd, 2021) DesignWe compare the characteristics of COVID-19 trials with and without preprints, estimate time to publication of COVID-19 preprint reports, describe discrepancies in key methods and results between preprint and published trial reports, report the number of retracted preprints and publications, and assess whether including versus excluding preprint reports affects meta-analytic estimates and the certainty of evidence. For the effects of eight therapies on mortality and mechanical ventilation, we performed meta-analyses including preprints and excluding preprints at 1 month, 3 months, and 6 months after the first trial addressing the therapy became available either as a preprint or publication (120 meta-analyses in total). ResultsWe included 356 trials, 101 of which are only available as preprints, 181 as journal publications, and 74 as preprints first and subsequently published in journals. Half of all preprints remain unpublished at six months and a third at one year. There were few important differences in key methods and results between trial preprints and their subsequent published reports. We identified four retracted trials, three of which were published in peer-reviewed journals. With two exceptions (2/60; 3.3%), point estimates were consistent between meta-analyses including versus excluding preprints as to whether they indicated benefit, no appreciable effect, or harm. There were nine comparisons (9/60; 15%) for which the rating of the certainty of evidence differed when preprints were included versus excluded, for four of these comparisons the certainty of evidence including preprints was higher and for five of these comparisons the certainty of evidence including preprints was lower. LimitationsThe generalizability of our results is limited to COVID-19. Preprints that are subsequently published in journals may be the most rigorous and may not represent all trial preprints. ConclusionWe found no compelling evidence that preprints provide less trustworthy results than published papers. We show that preprints remain the only source of findings of many trials for several months, a length of time that is unacceptable in a health emergency. We show that including preprints may affect the results of meta-analyses and the certainty of evidence. We encourage evidence users to consider data from preprints in contexts in which decisions are being made rapidly and evidence is being produced faster than can be peer-reviewed. O_TEXTBOXSummary Box 1O_ST_ABSWhat is already known on this topicC_ST_ABSO_LIClinicians and decision-makers need rapidly available and credible data addressing the comparative effectiveness of treatments and prophylaxis for COVID-19. C_LIO_LIInvestigators have adopted preprint servers, which allow the rapid dissemination of research findings before publication in peer-reviewed journals. C_LI What this study addsO_LIWe found no compelling evidence that preprints provide less trustworthy results than published papers. C_LIO_LIWe show that including preprints may affect the results of meta-analyses and the certainty of evidence and we encourage evidence users to consider data from preprints in contexts in which decisions are being made rapidly and evidence is being produced faster than can be peer-reviewed. C_LI C_TEXTBOX
RESUMO
ObjectiveTo compare the effects of interleukin-6 (IL-6) receptor blockers, with or without corticosteroids, on mortality in patients with COVID-19. DesignSystematic review and network meta-analysis Data sourcesWHO COVID-19 database, a comprehensive multilingual source of global covid-19 literature, and two prospective meta-analyses Study selectionTrials in which people with suspected, probable, or confirmed COVID-19 were randomized to IL-6 receptor blockers (with or without corticosteroids), corticosteroids, placebo, or standard care. ResultsWe assessed the risk of bias of included trials using a modification of the Cochrane risk of bias tool. We performed a Bayesian fixed effect network meta-analysis and assessed the certainty of evidence using the GRADE approach. We identified 45 eligible trials (20,650 patients), 36 (19,350 patients) of which could be included in the network meta-analysis. 27 of 36 trials were rated at high risk of bias, primarily due to lack of blinding. Tocilizumab (20 more per 1000, 15 fewer to 59 more; low certainty) and sarilumab (11 more per 1000, 38 fewer to 55 more; low certainty) alone may not reduce the risk of death. Tocilizumab, in combination with corticosteroids, probably reduces the risk of death compared to corticosteroids alone (35 fewer per 1000, 52 fewer to 18 more; moderate certainty) and sarilumab, in combination with corticosteroids, may reduce the risk of death compared to corticosteroids alone (43 fewer, 73 fewer to 12 more; low certainty). Tocilizumab and sarilumab, both in combination with corticosteroids, may have similar effects (8 more per 1000, 20 fewer to 35 more; low certainty). ConclusionIL-6 receptor blockers, when added to standard care that includes corticosteroids, in patients with severe or critical COVID-19, probably reduce mortality. Tocilizumab and sarilumab may have similar effectiveness. Systematic review registrationNA What is already known on this topic?O_LIIL-6 receptor blockers have immunosuppressive effects that may be important in COVID-19 patients with immune system dysfunction and inflammation C_LIO_LICorticosteroids reduce the risk of death in patients with severe or critical COVID-19 C_LI What this study addsO_LIOur systematic review and network meta-analysis provides a comprehensive review of the evidence addressing the effects of IL-6 receptor blockers, alone or in combination with corticosteroids, in COVID-19 C_LIO_LIIL-6 receptor blockers when added to a standard care that includes corticosteroids, in patients with severe or critical COVID-19, probably reduce mortality. C_LIO_LITocilizumab and sarilumab in combination with corticosteroids may have similar effectiveness for reducing mortality. C_LI
RESUMO
BackgroundThe coronavirus disease (Covid-19) pandemic has produced a large number of clinical trial reports with unprecedented rapidity, raising concerns about methodological quality and potential for research waste. ObjectivesTo describe the characteristics of randomized clinical trials (RCTs) investigating prophylaxis or treatment of Covid-19 infection and examine the effect of trial characteristics on whether the study reported a statistically significant effect on the primary outcome(s). Study DesignMeta-epidemiological study of Covid-19 treatment and prophylaxis RCTs. Eligibility criteriaEnglish-language RCTs (peer-reviewed or preprint) that evaluated pharmacologic agents or blood products compared to standard care, placebo, or an active comparator among participants with suspected or confirmed Covid-19 or at risk for Covid-19. We excluded trials of vaccines or traditional herbal medicines. Information sourcesWe searched 25 databases in the US Centre for Disease Control Downloadable Database from January 1 to October 21, 2020. Trial appraisal and synthesis methodsWe extracted trial characteristics including number of centres, funding sources (industry versus non-industry), and sample size. We assessed risk of bias (RoB) using the modified Cochrane RoB 2.0 Tool. We used descriptive statistics to summarize trial characteristics and logistic regression to evaluate the association between RoB due to the randomization process, centre status (single vs. multicentre), funding source, and sample size, and statistically significant effect in the primary outcome. ResultsWe included 91 RCTs (46,802 participants) evaluating Covid-19 therapeutic drugs (n = 76), blood products (n = 9) or prophylactic drugs (n = 6). Of these, 40 (44%) were single-centre, 23 (25.3%) enrolled < 50 patients, and 28 (30.8%) received industry funding. RoB varied across trials, with high or probably high overall RoB in 75 (82.4%) trials, most frequently due to deviations from the intended protocol (including blinding) and randomization processes. Thirty-eight trials (41.8%) found a statistically significant effect in the primary outcome. RoB due randomization (odds ratio [OR] 3.77, 95% confidence interval [CI], 1.47 to 9.72) and single centre trials (OR 3.15, 95% CI, 1.25 to 7.97) were associated with higher likelihood of finding a statistically significant effect. ConclusionsThere was high variability in RoB amongst Covid-19 trials. RoB attributed to the randomization process and single centre status were associated with a three-fold increase in the odds of finding a statistically significant effect. Researchers, funders, and knowledge users should remain cognizant of the impact of study characteristics, including RoB, on trial results when designing, conducting, and appraising Covid-19 trials.
RESUMO
IntroductionIn an attempt to improve outcomes for patients with coronavirus disease 19 (COVID-19), several drugs, such as remdesivir, hydroxychloroquine (with or without azithromycin), and lopinavir/ritonavir, have been evaluated for treatment. While much attention focuses on potential benefits of these drugs, this must be weighed against their adverse effects. MethodsWe searched 32 databases in multiple languages from 1 December 2019 to 27 October 2020. We included randomized trials if they compared any of the drugs of interest to placebo or standard care, or against each other. A related world health organization (WHO) guideline panel selected the interventions to address and identified possible adverse effects that might be important to patients. Pairs of reviewers independently extracted data and assessed risk of bias. We analyzed data using a fixed-effects pairwise meta-analysis and assessed the certainty of evidence using the GRADE approach. ResultsWe included 16 randomized trials which enrolled 8226 patients. Compared to standard care or placebo, low certainty evidence suggests that remdesivir may not have an important effect on acute kidney injury (risk difference [RD] 8 fewer per 1000, 95% confidence interval (CI): 27 fewer to 21 more) or cognitive dysfunction/delirium (RD 3 more per 1000, 95% CI: 12 fewer to 19 more). Low certainty evidence suggests that hydroxychloroquine may increase the risk of serious cardiac toxicity (RD 10 more per 1000, 95% CI: 0 more to 30 more) and cognitive dysfunction/delirium (RD 33 more per 1000, 95% CI: 18 fewer to 84 more), whereas moderate certainty evidence suggests hydroxychloroquine probably increases the risk of diarrhoea (RD 106 more per 1000, 95% CI: 48 more to 175 more) and nausea and/or vomiting (RD 62 more per 1000, 95% CI: 23 more to 110 more) compared to standard care or placebo. Low certainty evidence suggests lopinavir/ritonavir may increase the risk of diarrhoea (RD 168 more per 1000, 95% CI: 58 more to 330 more) and nausea and/or vomiting (RD 160 more per 1000, 95% CI: 100 more to 210 more) compared to standard care or placebo. ConclusionHydroxychloroquine probably increases the risk of diarrhoea and nausea and/or vomiting and may increase the risk of cardiac toxicity and cognitive dysfunction/delirium. Remdesivir may have no effect on risk of acute kidney injury or cognitive dysfunction/delirium. Lopinavir/ritonavir may increase the risk of diarrhoea and nausea and/or vomiting. These findings provide important information to support the development of evidence-based management strategies for patients with COVID-19.
