The United States COVID-19 Forecast Hub dataset.

Academic researchers, government agencies, industry groups, and individuals have produced forecasts at an unprecedented scale during the COVID-19 pandemic. To leverage these forecasts, the United States Centers for Disease Control and Prevention (CDC) partnered with an academic research lab at the University of Massachusetts Amherst to create the US COVID-19 Forecast Hub. Launched in April 2020, the Forecast Hub is a dataset with point and probabilistic forecasts of incident cases, incident hospitalizations, incident deaths, and cumulative deaths due to COVID-19 at county, state, and national, levels in the United States. Included forecasts represent a variety of modeling approaches, data sources, and assumptions regarding the spread of COVID-19. The goal of this dataset is to establish a standardized and comparable set of short-term forecasts from modeling teams. These data can be used to develop ensemble models, communicate forecasts to the public, create visualizations, compare models, and inform policies regarding COVID-19 mitigation. These open-source data are available via download from GitHub, through an online API, and through R packages.

The Zoltar forecast archive, a tool to standardize and store interdisciplinary prediction research.

Forecasting has emerged as an important component of informed, data-driven decision-making in a wide array of fields. We introduce a new data model for probabilistic predictions that encompasses a wide range of forecasting settings. This framework clearly defines the constituent parts of a probabilistic forecast and proposes one approach for representing these data elements. The data model is implemented in Zoltar, a new software application that stores forecasts using the data model and provides standardized API access to the data. In one real-time case study, an instance of the Zoltar web application was used to store, provide access to, and evaluate real-time forecast data on the order of 108 rows, provided by over 40 international research teams from academia and industry making forecasts of the COVID-19 outbreak in the US. Tools and data infrastructure for probabilistic forecasts, such as those introduced here, will play an increasingly important role in ensuring that future forecasting research adheres to a strict set of rigorous and reproducible standards.

Automatic extraction of quantitative data from ClinicalTrials.gov to conduct meta-analyses.

OBJECTIVES: Systematic reviews and meta-analyses are labor-intensive and time-consuming. Automated extraction of quantitative data from primary studies can accelerate this process. ClinicalTrials.gov, launched in 2000, is the world's largest trial repository of results data from clinical trials; it has been used as a source instead of journal articles. We have developed a Web application called EXACT (EXtracting Accurate efficacy and safety information from ClinicalTrials.gov) that allows users without advanced programming skills to automatically extract data from ClinicalTrials.gov in analysis-ready format. We have also used the automatically extracted data to examine the reproducibility of meta-analyses in three published systematic reviews. STUDY DESIGN AND SETTING: We developed a Python-based software application (EXACT) that automatically extracts data required for meta-analysis from the ClinicalTrials.gov database in a spreadsheet format. We confirmed the accuracy of the extracted data and then used those data to repeat meta-analyses in three published systematic reviews. To ensure that we used the same statistical methods and outcomes as the published systematic reviews, we repeated the meta-analyses using data manually extracted from the relevant journal articles. For the outcomes whose results we were able to reproduce using those journal article data, we examined the usability of ClinicalTrials.gov data. RESULTS: EXACT extracted data at ClincalTrials.gov with 100% accuracy, and it required 60% less time than the usual practice of manually extracting data from journal articles. We found that 87% of the data elements extracted using EXACT matched those extracted manually from the journal articles. We were able to reproduce 24 of 28 outcomes using the journal article data. Of these 24 outcomes, we were able to reproduce 83.3% of the published estimates using data at ClinicalTrials.gov. CONCLUSION: EXACT (http://bio-nlp.org/EXACT) automatically and accurately extracted data elements from ClinicalTrials.gov and thus reduced time in data extraction. The ClinicalTrials.gov data reproduced most meta-analysis results in our study, but this conclusion needs further validation.