Global monitoring of the impact of the COVID-19 pandemic through online surveys sampled from the Facebook user base.

Simultaneously tracking the global impact of COVID-19 is challenging because of regional variation in resources and reporting. Leveraging self-reported survey outcomes via an existing international social media network has the potential to provide standardized data streams to support monitoring and decision-making worldwide, in real time, and with limited local resources. The University of Maryland Global COVID-19 Trends and Impact Survey (UMD-CTIS), in partnership with Facebook, has invited daily cross-sectional samples from the social media platform's active users to participate in the survey since its launch on April 23, 2020. We analyzed UMD-CTIS survey data through December 20, 2020, from 31,142,582 responses representing 114 countries/territories weighted for nonresponse and adjusted to basic demographics. We show consistent respondent demographics over time for many countries/territories. Machine Learning models trained on national and pooled global data verified known symptom indicators. COVID-like illness (CLI) signals were correlated with government benchmark data. Importantly, the best benchmarked UMD-CTIS signal uses a single survey item whereby respondents report on CLI in their local community. In regions with strained health infrastructure but active social media users, we show it is possible to define COVID-19 impact trajectories using a remote platform independent of local government resources. This syndromic surveillance public health tool is the largest global health survey to date and, with brief participant engagement, can provide meaningful, timely insights into the global COVID-19 pandemic at a local scale.

Reporting Quality of Journal Abstracts for Surgical Randomized Controlled Trials Before and After the Implementation of the CONSORT Extension for Abstracts.

BACKGROUND: Adequate reporting is crucial in full-text publications but even more so in abstracts because they are the most frequently read part of a publication. In 2008, an extension for abstracts of the Consolidated Standards of Reporting Trials (CONSORT-A) statement was published, defining which items should be reported in abstracts of randomized controlled trials (RCTs). Therefore, we compared the adherence of RCT abstracts to CONSORT-A before and after the publication of CONSORT-A. METHODS: RCTs published in the five surgical journals with the highest impact factor were identified through PubMed for 2005-2007 and 2014-2016. Adherence to 15 CONSORT-A items and two additional items for abstracts of non-pharmacological trials was assessed in duplicate. We compared the overall adherence to CONSORT-A between the two time periods using an unpaired t test and explored adherence to specific items. RESULTS: A total of 192 and 164 surgical RCT abstracts were assessed (2005-2007 and 2014-2016, respectively). In the pre-CONSORT-A phase, the mean score of adequately reported items was 6.14 (95% confidence interval [CI] 5.90-6.38) and 8.11 in the post-CONSORT-A phase (95% CI 7.83-8.39; mean difference 1.97, 95% CI 1.60-2.34; p < 0.0001). The comparison of individual items indicated a significant improvement in 9 of the 15 items. The three least reported items in the post-CONSORT-A phase were randomization (2.4%), blinding (13.4%), and funding (0.0%). Specific items for non-pharmacological trials were rarely reported (approximately 10%). CONCLUSION: The reporting in abstracts of surgical RCTs has improved after the implementation of CONSORT-A. More importantly, there is still ample room for improvement.

Reporting transparency and completeness in trials: Paper 3 - trials conducted using administrative databases do not adequately report elements related to use of databases.

OBJECTIVE: We evaluated reporting completeness and transparency in randomized controlled trials (RCTs) conducted using administrative data based on 2021 CONSORT Extension for Trials Conducted Using Cohorts and Routinely Collected Data (CONSORT-ROUTINE) criteria. STUDY DESIGN AND SETTING: MEDLINE and the Cochrane Methodology Register were searched (2011 and 2018). Eligible RCTs used administrative databases for identifying eligible participants or collecting outcomes. We evaluated reporting based on CONSORT-ROUTINE, which modified eight items from CONSORT 2010 and added five new items. RESULTS: Of 33 included trials (76% used administrative databases for outcomes, 3% for identifying participants, 21% both), most were conducted in the United States (55%), Canada (18%), or the United Kingdom (12%). Of eight items modified in the extension; six were adequately reported in a majority (>50%) of trials. For the CONSORT-ROUTINE modification portion of those items, three items were reported adequately in >50% of trials, two in <50%, two only applied to some trials, and one only had wording modifications and was not evaluated. For five new items, four that address use of routine data in trials were reported inadequately in most trials. CONCLUSION: How administrative data are used in trials is often sub-optimally reported. CONSORT-ROUTINE uptake may improve reporting.

Reporting transparency and completeness in trials: Paper 4 - reporting of randomised controlled trials conducted using routinely collected electronic records - room for improvement.

OBJECTIVE: To describe characteristics of randomized controlled trials (RCTs) conducted using electronic health records (EHRs), including completeness and transparency of reporting assessed against the 2021 CONSORT Extension for RCTs Conducted Using Cohorts and Routinely Collected Data (CONSORT-ROUTINE) criteria. STUDY DESIGN: MEDLINE and Cochrane Methodology Register were searched for a sample of RCTs published from 2011-2018. Completeness of reporting was assessed in a random sample using a pre-defined coding form. RESULTS: Of the 183 RCT publications identified, 122 (67%) used EHRs to identify eligible participants, 139 (76%) used the EHR as part of the intervention and 137 (75%) to ascertain outcomes. When 60 publications were evaluated against the CONSORT 2010 item and the corresponding extension for the 8 modified items, four items were 'adequately reported' for most trials. Five new reporting items were identified for the CONSORT-ROUTINE extension; when evaluated, one was 'adequately reported', three were reported 'inadequately or not at all', the other 'partially'. There were, however, some encouraging signs with adequate and partial reporting of many important items, including descriptions of trial design, the consent process, outcome ascertainment and interpretation. CONCLUSION: Aspects of RCTs using EHRs are sub-optimally reported. Uptake of the CONSORT-ROUTINE Extension may improve reporting.

Reporting transparency and completeness in Trials: Paper 2 - reporting of randomised trials using registries was often inadequate and hindered the interpretation of results.

OBJECTIVE: Registries are important data sources for randomized controlled trials (RCTs), but reporting of how they are used may be inadequate. The objective was to describe the current adequacy of reporting of RCTs using registries. STUDY DESIGN AND SETTING: We used a database of trials using registries from a scoping review supporting the development of the 2021 CONSORT extension for Trials Conducted Using Cohorts and Routinely Collected Data (CONSORT-ROUTINE). Reporting completeness of 13 CONSORT-ROUTINE items was assessed. RESULTS: We assessed reports of 47 RCTs that used a registry, published between 2011 and 2018. Of the 13 CONSORT-ROUTINE items, 6 were adequately reported in at least half of reports (2 in at least 80%). The 7 other items were related to routinely collected data source eligibility (32% adequate), data linkage (8% adequate), validation and completeness of data used for outcome assessment (8% adequate), validation and completeness of data used for participant recruitment (0% adequate), participant flow (9% adequate), registry funding (6% adequate) and interpretation of results in consideration of registry use (25% adequate). CONCLUSION: Reporting of trials using registries was often poor, particularly details on data linkage and quality. Better reporting is needed for appropriate interpretation of the results of these trials.

Treatment effects in randomised trials using routinely collected data for outcome assessment versus traditional trials: meta-research study.

OBJECTIVE: To compare effect estimates of randomised clinical trials that use routinely collected data (RCD-RCT) for outcome ascertainment with traditional trials not using routinely collected data. DESIGN: Meta-research study. DATA SOURCE: Studies included in the same meta-analysis in a Cochrane review. ELIGIBILITY CRITERIA FOR STUDY SELECTION: Randomised clinical trials using any type of routinely collected data for outcome ascertainment, including from registries, electronic health records, and administrative databases, that were included in a meta-analysis of a Cochrane review on any clinical question and any health outcome together with traditional trials not using routinely collected data for outcome measurement. REVIEW METHODS: Effect estimates from trials using or not using routinely collected data were summarised in random effects meta-analyses. Agreement of (summary) treatment effect estimates from trials using routinely collected data and those not using such data was expressed as the ratio of odds ratios. Subgroup analyses explored effects in trials based on different types of routinely collected data. Two investigators independently assessed the quality of each data source. RESULTS: 84 RCD-RCTs and 463 traditional trials on 22 clinical questions were included. Trials using routinely collected data for outcome ascertainment showed 20% less favourable treatment effect estimates than traditional trials (ratio of odds ratios 0.80, 95% confidence interval 0.70 to 0.91, I2=14%). Results were similar across various types of outcomes (mortality outcomes: 0.92, 0.74 to 1.15, I2=12%; non-mortality outcomes: 0.71, 0.60 to 0.84, I2=8%), data sources (electronic health records: 0.81, 0.59 to 1.11, I2=28%; registries: 0.86, 0.75 to 0.99, I2=20%; administrative data: 0.84, 0.72 to 0.99, I2=0%), and data quality (high data quality: 0.82, 0.72 to 0.93, I2=0%). CONCLUSIONS: Randomised clinical trials using routinely collected data for outcome ascertainment show smaller treatment benefits than traditional trials not using routinely collected data. These differences could have implications for healthcare decision making and the application of real world evidence.

Antibiotic prescription monitoring and feedback in primary care in Switzerland: Design and rationale of a nationwide pragmatic randomized controlled trial.

INTRODUCTION: Antibiotic consumption is highest in primary care, and antibiotic overuse furthers antimicrobial resistance. In our recently published pilot-RCT, we used monthly aggregated claims data to provide personalized antibiotic prescription feedback to general practitioners (GPs). The pilot-RCT has shown that personalized prescription feedback is a feasible and promising low-cost intervention to reduce antibiotic prescribing. Here, we describe the rationale and design of the follow-up RCT with 3426 GPs in Switzerland. We now have access to pseudonymized patient-level data from routinely collected health insurance data of the three largest health insurers in Switzerland. METHODS AND ANALYSIS: 1713 GPs randomized to the intervention group received once evidence-based treatment guidelines at the beginning, including region-specific antibiotic resistance information from the community and personalized feedback of their antibiotic prescribing, followed by quarterly personalized prescription feedback for two years. The first and the last mailings were sent out in December 2017 and September 2019, respectively. The 1713 GPs randomized to the control group were not notified about the study and they received no guidelines and no prescription feedback. The personalized prescription feedbacks and the analyses of the primary and secondary outcomes are entirely based on pseudonymized patient-level data from routinely collected health insurance data. The primary outcome is prescribed antibiotics per 100 patient consultations during the second year of intervention. The secondary outcomes include antibiotic use during the entire two-year trial period, use of broad-spectrum antibiotics, hospitalization rates (all-cause and infection-related), and antibiotic use in different age groups. If the feedback intervention proves to be efficacious, the intervention could be continued systemwide. ETHICS AND DISSEMINATION: The trial is publicly funded by the Swiss National Science Foundation (SNSF, grant number 407240_167066). The trial was approved by the ethics committee "Ethikkommission Nordwest-und Zentralschweiz" (EKNZ Project-ID 2017-00888). Results will be disseminated in peer-reviewed journals and international conferences.

Nonrandomized studies using causal-modeling may give different answers than RCTs: a meta-epidemiological study.

OBJECTIVES: To evaluate how estimated treatment effects agree between nonrandomized studies using causal modeling with marginal structural models (MSM-studies) and randomized trials (RCTs). STUDY DESIGN: Meta-epidemiological study. SETTING: MSM-studies providing effect estimates on any healthcare outcome of any treatment were eligible. We systematically sought RCTs on the same clinical question and compared the direction of treatment effects, effect sizes, and confidence intervals. RESULTS: The main analysis included 19 MSM-studies (1,039,570 patients) and 141 RCTs (120,669 patients). MSM-studies indicated effect estimates in the opposite direction from RCTs for eight clinical questions (42%), and their 95% CI (confidence interval) did not include the RCT estimate in nine clinical questions (47%). The effect estimates deviated 1.58-fold between the study designs (median absolute deviation OR [odds ratio] 1.58; IQR [interquartile range] 1.37 to 2.16). Overall, we found no systematic disagreement regarding benefit or harm but confidence intervals were wide (summary ratio of odds ratios [sROR] 1.04; 95% CI 0.88 to 1.23). The subset of MSM-studies focusing on healthcare decision-making tended to overestimate experimental treatment benefits (sROR 1.44; 95% CI 0.99 to 2.09). CONCLUSION: Nonrandomized studies using causal modeling with MSM may give different answers than RCTs. Caution is still required when nonrandomized "real world" evidence is used for healthcare decisions.

Current use and costs of electronic health records for clinical trial research: a descriptive study.

BACKGROUND: Electronic health records (EHRs) may support randomized controlled trials (RCTs). We aimed to describe the current use and costs of EHRs in RCTs, with a focus on recruitment and outcome assessment. METHODS: This descriptive study was based on a PubMed search of RCTs published since 2000 that evaluated any medical intervention with the use of EHRs. Cost information was obtained from RCT investigators who used EHR infrastructures for recruitment or outcome measurement but did not explore EHR technology itself. RESULTS: We identified 189 RCTs, most of which (153 [81.0%]) were carried out in North America and were published recently (median year 2012 [interquartile range 2009-2014]). Seventeen RCTs (9.0%) involving a median of 732 (interquartile range 73-2513) patients explored interventions not related to EHRs, including quality improvement, screening programs, and collaborative care and disease management interventions. In these trials, EHRs were used for recruitment (14 [82%]) and outcome measurement (15 [88%]). Overall, in most of the trials (158 [83.6%]), the outcome (including many of the most patient-relevant clinical outcomes, from unscheduled hospital admission to death) was measured with the use of EHRs. The per-patient cost in the 17 EHR-supported trials varied from US$44 to US$2000, and total RCT costs from US$67 750 to US$5 026 000. In the remaining 172 RCTs (91.0%), EHRs were used as a modality of intervention. INTERPRETATION: Randomized controlled trials are frequently and increasingly conducted with the use of EHRs, but mainly as part of the intervention. In some trials, EHRs were used successfully to support recruitment and outcome assessment. Costs may be reduced once the data infrastructure is established.

Contrasting evidence to reimbursement reality for off-label use (OLU) of drug treatments in cancer care: rationale and design of the CEIT-OLU project.

Background: Off-label use (OLU) of a drug reflects a perceived unmet medical need, which is common in oncology. Cancer drugs are often highly expensive and their reimbursement is a challenge for many healthcare systems. OLU is frequently regulated by reimbursement restrictions. For evidence-based healthcare, treatment ought to be reimbursed if there is sufficient clinical evidence for treatment benefit independently of patient factors not related to the treatment indication. However, little is known about the reality of OLU reimbursement and its association with the underlying clinical evidence. Here, we aim to investigate the relationship of reimbursement decisions with the underlying clinical evidence. Methods/ design: We will extract patient characteristics and details on treatment and reimbursement of cancer drugs from over 3000 patients treated in three Swiss hospitals. We will systematically search for clinical trial evidence on benefits associated with OLU in the most common indications. We will describe the prevalence of OLU in Switzerland and its reimbursement in cancer care, and use multivariable logistic regression techniques to investigate the association of approval/rejection of a reimbursement requests to the evidence on treatment effects and to further factors, including type of drug, molecular predictive markers and the health insurer. Discussion: Our study will provide a systematic overview and assessment of OLU and its reimbursement reality in Switzerland. We may provide a better understanding of the access to cancer care that is regulated by health insurers and we hope to identify factors that determine the level of evidence-based cancer care in a highly diverse western healthcare system.

Routinely collected data for randomized trials: promises, barriers, and implications.

BACKGROUND: Routinely collected health data (RCD) are increasingly used for randomized controlled trials (RCTs). This can provide three major benefits: increasing value through better feasibility (reducing costs, time, and resources), expanding the research agenda (performing trials for research questions otherwise not amenable to trials), and offering novel design and data collection options (e.g., point-of-care trials and other designs directly embedded in routine care). However, numerous hurdles and barriers must be considered pertaining to regulatory, ethical, and data aspects, as well as the costs of setting up the RCD infrastructure. Methodological considerations may be different from those in traditional RCTs: RCD are often collected by individuals not involved in the study and who are therefore blinded to the allocation of trial participants. Another consideration is that RCD trials may lead to greater misclassification biases or dilution effects, although these may be offset by randomization and larger sample sizes. Finally, valuable insights into external validity may be provided when using RCD because it allows pragmatic trials to be performed. METHODS: We provide an overview of the promises, challenges, and potential barriers, methodological implications, and research needs regarding RCD for RCTs. RESULTS: RCD have substantial potential for improving the conduct and reducing the costs of RCTs, but a multidisciplinary approach is essential to address emerging practical barriers and methodological implications. CONCLUSIONS: Future research should be directed toward such issues and specifically focus on data quality validation, alternative research designs and how they affect outcome assessment, and aspects of reporting and transparency.

Protocol for the development of a CONSORT extension for RCTs using cohorts and routinely collected health data.

BACKGROUND: Randomized controlled trials (RCTs) are often complex and expensive to perform. Less than one third achieve planned recruitment targets, follow-up can be labor-intensive, and many have limited real-world generalizability. Designs for RCTs conducted using cohorts and routinely collected health data, including registries, electronic health records, and administrative databases, have been proposed to address these challenges and are being rapidly adopted. These designs, however, are relatively recent innovations, and published RCT reports often do not describe important aspects of their methodology in a standardized way. Our objective is to extend the Consolidated Standards of Reporting Trials (CONSORT) statement with a consensus-driven reporting guideline for RCTs using cohorts and routinely collected health data. METHODS: The development of this CONSORT extension will consist of five phases. Phase 1 (completed) consisted of the project launch, including fundraising, the establishment of a research team, and development of a conceptual framework. In phase 2, a systematic review will be performed to identify publications (1) that describe methods or reporting considerations for RCTs conducted using cohorts and routinely collected health data or (2) that are protocols or report results from such RCTs. An initial "long list" of possible modifications to CONSORT checklist items and possible new items for the reporting guideline will be generated based on the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) and The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statements. Additional possible modifications and new items will be identified based on the results of the systematic review. Phase 3 will consist of a three-round Delphi exercise with methods and content experts to evaluate the "long list" and generate a "short list" of key items. In phase 4, these items will serve as the basis for an in-person consensus meeting to finalize a core set of items to be included in the reporting guideline and checklist. Phase 5 will involve drafting the checklist and elaboration-explanation documents, and dissemination and implementation of the guideline. DISCUSSION: Development of this CONSORT extension will contribute to more transparent reporting of RCTs conducted using cohorts and routinely collected health data.