ABSTRACT
OBJECTIVES: To provide an overview of analytical methods in scientific literature for comparing uncontrolled medicine trials to external controls from individual patient-level real-world data (IPD-RWD). In addition, to compare these methods with recommendations made in guidelines from European regulatory and health technology assessment (HTA) organizations and with their evaluations described in assessment reports. METHODS: A systematic literature review (until March 1st 2023) in PubMed and Connected Papers was performed to identify analytical methods for comparing uncontrolled trials with external controls from IPD-RWD. These methods were compared descriptively to methods recommended in method guidelines and encountered in assessment reports of the European Medicines Agency (2015-2020) and four European HTA organizations (2015-2023). RESULTS: Thirty-four identified scientific articles described analytical methods for comparing uncontrolled trial data to IPD-RWD-based external controls. The various methods covered controlling for confounding and/or dependent censoring, correction for missing data; and analytical comparative modelling methods. Seven guidelines also focused on research design, RWD quality and transparency aspects, and four of those recommended analytical methods for comparisons with IPD-RWD. The methods discussed in regulatory (n=15) and HTA (n=35) assessment reports were often based on aggregate data and lacked transparency due to the few details provided. CONCLUSION: Literature and guidelines suggest a methodological approach to comparing uncontrolled trials with external controls from IPD-RWD similar to target trial emulation, using state-of-the-art methods. External controls supporting regulatory and HTA decision-making were rarely in line with this approach. Twelve recommendations are proposed to improve the quality and acceptability of these methods.
ABSTRACT
PURPOSE: Real-world evidence (RWE) is increasingly used for medical regulatory decisions, yet concerns persist regarding its reproducibility and hence validity. This study addresses reproducibility challenges associated with diversity across real-world data sources (RWDS) repurposed for secondary use in pharmacoepidemiologic studies. Our aims were to identify, describe and characterize practices, recommendations and tools for collecting and reporting diversity across RWDSs, and explore how leveraging diversity could improve the quality of evidence. METHODS: In a preliminary phase, keywords for a literature search and selection tool were designed using a set of documents considered to be key by the coauthors. Next, a systematic search was conducted up to December 2021. The resulting documents were screened based on titles and abstracts, then based on full texts using the selection tool. Selected documents were reviewed to extract information on topics related to collecting and reporting RWDS diversity. A content analysis of the topics identified explicit and latent themes. RESULTS: Across the 91 selected documents, 12 topics were identified: 9 dimensions used to describe RWDS (organization accessing the data source, data originator, prompt, inclusion of population, content, data dictionary, time span, healthcare system and culture, and data quality), tools to summarize such dimensions, challenges, and opportunities arising from diversity. Thirty-six themes were identified within the dimensions. Opportunities arising from data diversity included multiple imputation and standardization. CONCLUSIONS: The dimensions identified across a large number of publications lay the foundation for formal guidance on reporting diversity of data sources to facilitate interpretation and enhance replicability and validity of RWE.