OpenSAFELY: A platform for analysing electronic health records designed for reproducible research.

Electronic health records (EHRs) and other administrative health data are increasingly used in research to generate evidence on the effectiveness, safety, and utilisation of medical products and services, and to inform public health guidance and policy. Reproducibility is a fundamental step for research credibility and promotes trust in evidence generated from EHRs. At present, ensuring research using EHRs is reproducible can be challenging for researchers. Research software platforms can provide technical solutions to enhance the reproducibility of research conducted using EHRs. In response to the COVID-19 pandemic, we developed the secure, transparent, analytic open-source software platform OpenSAFELY designed with reproducible research in mind. OpenSAFELY mitigates common barriers to reproducible research by: standardising key workflows around data preparation; removing barriers to code-sharing in secure analysis environments; enforcing public sharing of programming code and codelists; ensuring the same computational environment is used everywhere; integrating new and existing tools that encourage and enable the use of reproducible working practices; and providing an audit trail for all code that is run against the real data to increase transparency. This paper describes OpenSAFELY's reproducibility-by-design approach in detail.

Barriers and best practices to improving clinical trials transparency at UK public research institutions: A qualitative interview study.

OBJECTIVES: Since 2017, the UK government has made concerted efforts to ensure the dissemination of clinical trials conducted at public research institutions. This study aims to understand how stakeholders within these institutions responded to these pressures and modified internal policies and processes while identifying best practices and barriers to improved transparency practice. METHODS: Research governance and trial management staff from UK public research institutions (i.e., Universities and NHS Trusts) in England, Scotland and Wales participated in semi-structured interviews. Interviews were analysed using thematic analysis, aided by the framework method. RESULTS: Between November 2020 and July 2021, 14 individual participants were recruited from 11 different institutions. They worked in research governance, administration, and management. Almost universally, new policies and procedures have been established to ensure investigators are aware of, and supported in, fulfilling their transparency commitments, however challenges remain. Trials of medicinal products, as the most closely regulated research, consequently received the most attention. National professional networks aid in sharing knowledge and best practice within this community. CONCLUSIONS: Investment in the institutional governance of transparency is essential to achieving optimal transparency practices. Universities and hospitals share responsibility for ensuring research is performed and reported to regulatory standards. Facing political pressure, public research institutions in the UK have made efforts to improve their transparency practice which can provide key insights for similar efforts elsewhere.

Availability of results of clinical trials registered on EU Clinical Trials Register: cross sectional audit study.

Objective: To identify the availability of results for trials registered on the European Union Clinical Trials Register (EUCTR) compared with other dissemination routes to understand its value as a results repository. Design: Cross sectional audit study. Setting: EUCTR protocols and results sections, data extracted 1-3 December 2020. Population: Random sample of 500 trials registered on EUCTR with a completion date of more than two years from the beginning of searches (ie, 1 December 2018). Main outcome measures: Proportion of trials with results across the examined dissemination routes (EUCTR, ClinicalTrials.gov, ISRCTN registry, and journal publications), and for each dissemination route individually. Prespecified secondary outcomes were number and proportion of unique results, and the timing of results, for each dissemination route. Results: In the sample of 500 trials, availability of results on EUCTR (53.2%, 95% confidence interval 48.8% to 57.6%) was similar to the peer reviewed literature (58.6%, 54.3% to 62.9%) and exceeded the proportion of results available on other registries with matched records. Among the 383 trials with any results, 55 (14.4%, 10.9% to 17.9%) were only available on EUCTR. Also, after the launch of the EUCTR results database, median time to results was fastest on EUCTR (1142 days, 95% confidence interval 812 to 1492), comparable with journal publications (1226 days, 1074 to 1551), and exceeding ClinicalTrials.gov (3321 days, 1653 to undefined). For 117 trials (23.4%, 19.7% to 27.1%), however, results were published elsewhere but not submitted to the EUCTR registry, and no results were located in any dissemination route for 117 trials (23.4%, 19.7% to 27.1). Conclusions: EUCTR should be considered in results searches for systematic reviews and can help researchers and the public to access the results of clinical trials, unavailable elsewhere, in a timely way. Reporting requirements, such as the EU's, can help in avoiding research waste by ensuring results are reported. The registry's true value, however, is unrealised because of inadequate compliance with EU guidelines, and problems with data quality that complicate the routine use of the registry. As the EU transitions to a new registry, continuing to emphasise the importance of EUCTR and the provision of timely and complete data is critical. For the future, EUCTR will still hold important information from the past two decades of clinical research in Europe. With increased efforts from sponsors and regulators, the registry can continue to grow as a source of results of clinical trials, many of which might be unavailable from other dissemination routes.

Dissemination of Registered COVID-19 Clinical Trials (DIRECCT): a cross-sectional study.

BACKGROUND: The results of clinical trials should be completely and rapidly reported during public health emergencies such as COVID-19. This study aimed to examine when, and where, the results of COVID-19 clinical trials were disseminated throughout the first 18 months of the pandemic. METHODS: Clinical trials for COVID-19 treatment or prevention were identified from the WHO ICTRP database. All interventional trials with a registered completion date ≤ 30 June 2021 were included. Trial results, published as preprints, journal articles, or registry results, were located using automated and manual techniques across PubMed, Google Scholar, Google, EuropePMC, CORD-19, the Cochrane COVID-19 Study Register, and clinical trial registries. Our main analysis reports the rate of dissemination overall and per route, and the time from registered completion to results using Kaplan-Meier methods, with additional subgroup and sensitivity analyses reported. RESULTS: Overall, 1643 trials with completion dates ranging from 46 to 561 days prior to the start of results searches were included. The cumulative probability of reporting was 12.5% at 3 months from completion, 21.6% at 6 months, and 32.8% at 12 months. Trial results were most commonly disseminated in journals (n = 278 trials, 69.2%); preprints were available for 194 trials (48.3%), 86 (44.3%) of which converted to a full journal article. Trials completed earlier in the pandemic were reported more rapidly than those later in the pandemic, and those involving ivermectin were more rapidly reported than other common interventions. Results were robust to various sensitivity analyses except when considering only trials in a "completed" status on the registry, which substantially increased reporting rates. Poor trial registry data on completion status and dates limits the precision of estimates. CONCLUSIONS: COVID-19 trials saw marginal increases in reporting rates compared to standard practice; most registered trials failed to meet even the 12-month non-pandemic standard. Preprints were common, complementing journal publication; however, registries were underutilized for rapid reporting. Maintaining registry data enables accurate representation of clinical research; failing to do so undermines these registries' use for public accountability and analysis. Addressing rapid reporting and registry data quality must be emphasized at global, national, and institutional levels.

Preventable deaths involving opioids in England and Wales, 2013-2022: a systematic case series of coroners' reports.

BACKGROUND: Opioid deaths have increased in England and Wales. Coroners' Prevention of Future Deaths reports (PFDs) provide important insights that may enable safer use and avert harms, yet reports implicating opioids have not been synthesized. We aimed to identify opioid-related PFDs and explore coroners' concerns to prevent future deaths. METHODS: In this systematic case series, we screened 3897 coronial PFDs dated between 01 July 2013 and 23 February 2022, obtained by web scraping the UK's Courts and Tribunals Judiciary website. PFDs were included when an opioid was implicated in the death. Included PFDs were descriptively analysed, and content analysis was used to assess concerns reported by coroners. RESULTS: Opioids were involved in 219 deaths reported in PFDs (5·6% of PFDs), equating to 4418 years of life lost (median 33 years/person). Morphine (29%), methadone (23%) and diamorphine (16%) were the most common implicated opioids. Coroners most frequently raised concerns regarding systems and protocols (52%) or safety issues (15%). These concerns were most often addressed to National Health Service (NHS) organizations (51%), but response rates were low overall (47%). CONCLUSIONS: Opioids could be used more safely if coroners' concerns in PFDs were addressed by national organizations such as NHS bodies, government agencies and policymakers, as well as individual prescribing clinicians.

Completeness and consistency of primary outcome reporting in COVID-19 publications in the early pandemic phase: a descriptive study.

BACKGROUND: The COVID-19 pandemic saw a steep increase in the number of rapidly published scientific studies, especially early in the pandemic. Some have suggested COVID-19 trial reporting is of lower quality than typical reports, but there is limited evidence for this in terms of primary outcome reporting. The objective of this study was to assess the prevalence of completely defined primary outcomes reported in registry entries, preprints, and journal articles, and to assess consistent primary outcome reporting between these sources. METHODS: This is a descriptive study of a cohort of registered interventional clinical trials for the treatment and prevention of COVID-19, drawn from the DIssemination of REgistered COVID-19 Clinical Trials (DIRECCT) study dataset. The main outcomes are: 1) Prevalence of complete primary outcome reporting; 2) Prevalence of consistent primary outcome reporting between registry entry and preprint as well as registry entry and journal article pairs. RESULTS: We analyzed 87 trials with 116 corresponding publications (87 registry entries, 53 preprints and 63 journal articles). All primary outcomes were completely defined in 47/87 (54%) registry entries, 31/53 (58%) preprints and 44/63 (70%) journal articles. All primary outcomes were consistently reported in 13/53 (25%) registry-preprint pairs and 27/63 (43%) registry-journal article pairs. No primary outcome was specified in 13/53 (25%) preprints and 8/63 (13%) journal articles. In this sample, complete primary outcome reporting occurred more frequently in trials with vs. without involvement of pharmaceutical companies (76% vs. 45%), and in RCTs vs. other study designs (68% vs. 49%). The same pattern was observed for consistent primary outcome reporting (with vs. without pharma: 56% vs. 12%, RCT vs. other: 43% vs. 22%). CONCLUSIONS: In COVID-19 trials in the early phase of the pandemic, all primary outcomes were completely defined in 54%, 58%, and 70% of registry entries, preprints and journal articles, respectively. Only 25% of preprints and 43% of journal articles reported primary outcomes consistent with registry entries.

Trends, variation, and clinical characteristics of recipients of antiviral drugs and neutralising monoclonal antibodies for covid-19 in community settings: retrospective, descriptive cohort study of 23.4 million people in OpenSAFELY.

Objective: To ascertain patient eligibility status and describe coverage of antiviral drugs and neutralising monoclonal antibodies (nMAB) as treatment for covid-19 in community settings in England. Design: Retrospective, descriptive cohort study, approved by NHS England. Setting: Routine clinical data from 23.4 million people linked to data on covid-19 infection and treatment, within the OpenSAFELY-TPP database. Participants: Outpatients with covid-19 at high risk of severe outcomes. Interventions: Nirmatrelvir/ritonavir (paxlovid), sotrovimab, molnupiravir, casirivimab/imdevimab, or remdesivir, used in the community by covid-19 medicine delivery units. Results: 93 870 outpatients with covid-19 were identified between 11 December 2021 and 28 April 2022 to be at high risk of severe outcomes and therefore potentially eligible for antiviral or nMAB treatment (or both). Of these patients, 19 040 (20%) received treatment (sotrovimab, 9660 (51%); molnupiravir, 4620 (24%); paxlovid, 4680 (25%); casirivimab/imdevimab, 50 (<1%); and remdesivir, 30 (<1%)). The proportion of patients treated increased from 9% (190/2220) in the first week of treatment availability to 29% (460/1600) in the latest week. The proportion treated varied by high risk group, being lowest in those with liver disease (16%; 95% confidence interval 15% to 17%); by treatment type, with sotrovimab favoured over molnupiravir and paxlovid in all but three high risk groups (Down's syndrome (35%; 30% to 39%), rare neurological conditions (45%; 43% to 47%), and immune deficiencies (48%; 47% to 50%)); by age, ranging from ≥80 years (13%; 12% to 14%) to 50-59 years (23%; 22% to 23%); by ethnic group, ranging from black (11%; 10% to 12%) to white (21%; 21% to 21%); by NHS region, ranging from 13% (12% to 14%) in Yorkshire and the Humber to 25% (24% to 25%) in the East of England); and by deprivation level, ranging from 15% (14% to 15%) in the most deprived areas to 23% (23% to 24%) in the least deprived areas. Groups that also had lower coverage included unvaccinated patients (7%; 6% to 9%), those with dementia (6%; 5% to 7%), and care home residents (6%; 6% to 7%). Conclusions: Using the OpenSAFELY platform, we were able to identify patients with covid-19 at high risk of severe outcomes who were potentially eligible to receive treatment and assess the coverage of these new treatments among these patients. In the context of a rapid deployment of a new service, the NHS analytical code used to determine eligibility could have been over-inclusive and some of the eligibility criteria not fully captured in healthcare data. However targeted activity might be needed to resolve apparent lower treatment coverage observed among certain groups, in particular (at present): different NHS regions, ethnic groups, people aged ≥80 years, those living in socioeconomically deprived areas, and care home residents.

Registration of health and medical research.

Registration of health and medical research is an effective way of improving the transparency and credibility of evidence. Registration involves pre-specifying the research objectives, design, methods and analytic plan on a publicly accessible repository before conducting the study. Registration can reduce bias and improve the transparency and credibility of research findings. Registration is mandated for clinical trials, but it is also relevant to systematic reviews, observational and preclinical experimental research. This paper describes how researchers can register their research and outlines possible barriers and challenges in doing so. Widespread adoption of research registration can reduce research waste and improve evidence-informed clinical and policy decision making.

E-cigarette manufacturers' compliance with clinical trial reporting expectations: a case series of registered trials by Juul Labs.

BACKGROUND: Electronic cigarettes (e-cigarettes) are a frequently debated topic in public health. It is essential that clinical trials examining e-cigarettes are fully and accurately reported, especially given long-standing concerns about tobacco industry research. We assess the reporting of clinical trials sponsored by Juul Labs, the largest e-cigarette company in the USA, against accepted reporting standards. METHODS: We searched ClinicalTrials.gov for all trials sponsored by Juul Labs and determined those with registry data consistent with coverage by the Food and Drug Administration (FDA) Amendments Act 2007 (FDAAA). For trials with a primary completion date more than 1 year earlier, we searched ClinicalTrials.gov, the academic literature and a Juul-funded research database (JLI Science) for results. For located results, we compared reported outcomes with registered outcomes in line with Consolidated Standards of Reporting Trials (CONSORT) reporting guidelines. RESULTS: We located five registered trials sponsored by Juul Labs that appeared covered by the FDAAA 2007 in the public data. All five trials did not have results available on ClinicalTrials.gov. We found one publication and four poster presentations reporting results for four of the five covered trials outside of ClinicalTrials.gov. Of 61 specified outcomes, 28 were CONSORT compliant. Specific outcome reporting issues are detailed. DISCUSSION: Our findings raise substantial concerns regarding these trials. Clinicians, public health professionals, and the public cannot make informed choices about the benefits or hazards of e-cigarettes if the results of clinical trials are not completely and transparently reported. Clarification and potential enforcement of reporting laws may be required.

Sharing study materials in health and medical research.

Making study materials available allows for a more comprehensive understanding of the scientific literature. Sharing can take many forms and include a wide variety of outputs including code and data. Biomedical research can benefit from increased transparency but faces unique challenges for sharing, for instance, confidentiality concerns around participants' medical data. Both general and specialised repositories exist to aid in sharing most study materials. Sharing may also require skills and resources to ensure that it is done safely and effectively. Educating researchers on how to best share their materials, and properly rewarding these practices, requires action from a variety of stakeholders including journals, funders and research institutions.

Comparative effectiveness of sotrovimab and molnupiravir for prevention of severe covid-19 outcomes in patients in the community: observational cohort study with the OpenSAFELY platform.

OBJECTIVE: To compare the effectiveness of sotrovimab (a neutralising monoclonal antibody) with molnupiravir (an antiviral) in preventing severe outcomes of covid-19 in adult patients infected with SARS-CoV-2 in the community and at high risk of severe outcomes from covid-19. DESIGN: Observational cohort study with the OpenSAFELY platform. SETTING: With the approval of NHS England, a real world cohort study was conducted with the OpenSAFELY-TPP platform (a secure, transparent, open source software platform for analysis of NHS electronic health records), and patient level electronic health record data were obtained from 24 million people registered with a general practice in England that uses TPP software. The primary care data were securely linked with data on SARS-CoV-2 infection and treatments, hospital admission, and death, over a period when both drug treatments were frequently prescribed in community settings. PARTICIPANTS: Adult patients with covid-19 in the community at high risk of severe outcomes from covid-19, treated with sotrovimab or molnupiravir from 16 December 2021. INTERVENTIONS: Sotrovimab or molnupiravir given in the community by covid-19 medicine delivery units. MAIN OUTCOME MEASURES: Admission to hospital with covid-19 (ie, with covid-19 as the primary diagnosis) or death from covid-19 (ie, with covid-19 as the underlying or contributing cause of death) within 28 days of the start of treatment. RESULTS: Between 16 December 2021 and 10 February 2022, 3331 and 2689 patients were treated with sotrovimab and molnupiravir, respectively, with no substantial differences in baseline characteristics. Mean age of all 6020 patients was 52 (standard deviation 16) years; 59% were women, 89% were white, and 88% had received three or more covid-19 vaccinations. Within 28 days of the start of treatment, 87 (1.4%) patients were admitted to hospital or died of infection from SARS-CoV-2 (32 treated with sotrovimab and 55 with molnupiravir). Cox proportional hazards models stratified by area showed that after adjusting for demographic information, high risk cohort categories, vaccination status, calendar time, body mass index, and other comorbidities, treatment with sotrovimab was associated with a substantially lower risk than treatment with molnupiravir (hazard ratio 0.54, 95% confidence interval 0.33 to 0.88, P=0.01). Consistent results were found from propensity score weighted Cox models (0.50, 0.31 to 0.81, P=0.005) and when restricted to people who were fully vaccinated (0.53, 0.31 to 0.90, P=0.02). No substantial effect modifications by other characteristics were detected (all P values for interaction >0.10). The findings were similar in an exploratory analysis of patients treated between 16 February and 1 May 2022 when omicron BA.2 was the predominant variant in England. CONCLUSIONS: In routine care of adult patients in England with covid-19 in the community, at high risk of severe outcomes from covid-19, those who received sotrovimab were at lower risk of severe outcomes of covid-19 than those treated with molnupiravir.

Issues with reporting and interpretation of Khan et al. 2021.

A recent publication in BMC Infectious Diseases concerning the use of convalescent plasma for the treatment of COVID-19 had a number of major issues. This correspondence details specific instances of unclear reporting as well as major omissions when discussing the context of the trial. These render the study's findings and conclusions misleading.

Improving medical research in the United Kingdom.

Poor quality medical research causes serious harms by misleading healthcare professionals and policymakers, decreasing trust in science and medicine, and wasting public funds. Here we outline underlying problems including insufficient transparency, dysfunctional incentives, and reporting biases. We make the following recommendations to address these problems: Journals and funders should ensure authors fulfil their obligation to share detailed study protocols, analytical code, and (as far as possible) research data. Funders and journals should incentivise uptake of registered reports and establish funding pathways which integrate evaluation of funding proposals with initial peer review of registered reports. A mandatory national register of interests for all those who are involved in medical research in the UK should be established, with an expectation that individuals maintain the accuracy of their declarations and regularly update them. Funders and institutions should stop using metrics such as citations and journal's impact factor to assess research and researchers and instead evaluate based on quality, reproducibility, and societal value. Employers and non-academic training programmes for health professionals (clinicians hired for patient care, not to do research) should not select based on number of research publications. Promotions based on publication should be restricted to those hired to do research.

Trends and variation in data quality and availability on the European Union Clinical Trials Register: A cross-sectional study.

BACKGROUND/AIMS: The European Union Clinical Trials Register is a public facing portal containing information on trials of medicinal products conducted under the purview of the European Union regulatory system. As of September 2021, the registry holds information on over 40,000 trials. Given its distinct regulatory purpose, and results reporting requirements, the European Union Clinical Trials Register should be a valuable open-source hub for trial information. Past work examining the European Union Clinical Trials Register has suggested that data quality on the registry may be lacking. We therefore set out to examine the quality and availability of trial data on the registry with a focus on areas that fall under the authority of regulators in each European Union/European Economic Area country. METHODS: Using data scraped from the full European Union Clinical Trials Register public dataset, we examined the extent of issues with three areas of trial data availability linked to European Union regulations. We examined whether there is evidence for missing registration of protocols in the public database, whether information on the completion of clinical trials is being made available and how often the results of trials are posted to the registry. We assessed each area overall, and examined variation between national regulators and over time. RESULTS: Major issues with the availability of expected protocols and information on trial completion were focused in a few countries. Overall, when comparing enrolment countries from tabular results to available registrations, 26,932 of 31,118 (86.5%) expected protocols were available and 22 of 30 (73%) countries had over 90% of expected protocols available. The majority of missing protocols, totalling 2764 (66%), were from just three countries: France, Norway and Poland. Evidence for this issue is further supported by data on trends in new registrations by country over time. Low availability of data on trial completion is also most pronounced in a minority of countries, like Spain and the Netherlands, with consistent trends for missingness over time. Finally, overall results availability is substantially worse among the 23,623 trials with a single registered European Union protocol (n = 6259, 26.5%) compared to 13,897 of those taking place in multiple countries (n = 8423, 60.6%). Reporting for single-protocol trials was consistently low across both time and location. CONCLUSION: Deficiencies in the public availability of trial protocols, trial completion information and summary results complicate the utility of the European Union Clinical Trials Register for research, transparency and accountability efforts. Users of the registry would benefit from a more complete and accurate accounting of the European research environment via the official European Union registry. We recommend regulators at the national and pan-national level undertake routine audits of approved trials to ensure national-level issues are proactively and transparently identified, documented and addressed.

Comparison of methods for predicting COVID-19-related death in the general population using the OpenSAFELY platform.

BACKGROUND: Obtaining accurate estimates of the risk of COVID-19-related death in the general population is challenging in the context of changing levels of circulating infection. METHODS: We propose a modelling approach to predict 28-day COVID-19-related death which explicitly accounts for COVID-19 infection prevalence using a series of sub-studies from new landmark times incorporating time-updating proxy measures of COVID-19 infection prevalence. This was compared with an approach ignoring infection prevalence. The target population was adults registered at a general practice in England in March 2020. The outcome was 28-day COVID-19-related death. Predictors included demographic characteristics and comorbidities. Three proxies of local infection prevalence were used: model-based estimates, rate of COVID-19-related attendances in emergency care, and rate of suspected COVID-19 cases in primary care. We used data within the TPP SystmOne electronic health record system linked to Office for National Statistics mortality data, using the OpenSAFELY platform, working on behalf of NHS England. Prediction models were developed in case-cohort samples with a 100-day follow-up. Validation was undertaken in 28-day cohorts from the target population. We considered predictive performance (discrimination and calibration) in geographical and temporal subsets of data not used in developing the risk prediction models. Simple models were contrasted to models including a full range of predictors. RESULTS: Prediction models were developed on 11,972,947 individuals, of whom 7999 experienced COVID-19-related death. All models discriminated well between individuals who did and did not experience the outcome, including simple models adjusting only for basic demographics and number of comorbidities: C-statistics 0.92-0.94. However, absolute risk estimates were substantially miscalibrated when infection prevalence was not explicitly modelled. CONCLUSIONS: Our proposed models allow absolute risk estimation in the context of changing infection prevalence but predictive performance is sensitive to the proxy for infection prevalence. Simple models can provide excellent discrimination and may simplify implementation of risk prediction tools.