Increase transparency and reproducibility of real-world evidence in rare diseases through disease-specific Federated Data Networks.

PURPOSE: In rare diseases, real-world evidence (RWE) generation is often restricted due to small patient numbers and global geographic distribution. A federated data network (FDN) approach brings together multiple data sources harmonized for collaboration to increase the power of observational research. In this paper, we review how to increase reproducibility and transparency of RWE studies in rare diseases through disease-specific FDNs. METHOD: To be successful, a multiple stakeholder scientific FDN collaboration requires a strong governance model in place. In such a model, each database owner remains in full control regarding the use of and access to patient-level data and is responsible for data privacy, ethical, and legal compliance. Provided that all this is well documented and good database descriptions are in place, such a governance model results in increased transparency, while reproducibility is achieved through data curation and harmonization, and distributed analytical methods. RESULTS: Leveraging the OHDSI community set of methods and tools, two rare disease-specific FDNs are discussed in more detail. For multiple myeloma, HONEUR-the Haematology Outcomes Network in Europe-has built a strong community among the data partners dedicated to scientific exchange and research. To advance scientific knowledge in pulmonary hypertension (PH) an FDN, called PHederation, was established to form a partnership of research institutions with PH databases coming from diverse origins.

Clinical evaluation of code-based algorithms to identify patients with pulmonary arterial hypertension in healthcare databases.

Pulmonary arterial hypertension (PAH) is a rare subgroup of pulmonary hypertension (PH). Claims and administrative databases can be particularly important for research in rare diseases; however, there is a lack of validated algorithms to identify PAH patients using administrative codes. We aimed to measure the accuracy of code-based PAH algorithms against the true clinical diagnosis by right heart catheterization (RHC). This study evaluated algorithms in patients who were recorded in two linkable data assets: the Stanford Healthcare administrative electronic health record database and the Stanford Vera Moulton Wall Center clinical PH database (which records each patient's RHC diagnosis). We assessed the sensitivity and specificity achieved by 16 algorithms (six published). In total, 720 PH patients with linked data available were included and 558 (78%) of these were PAH patients. Algorithms consisting solely of a P(A)H-specific diagnostic code classed all or almost all PH patients as PAH (sensitivity >97%, specificity <12%) while multicomponent algorithms with well-defined temporal sequences of procedure, diagnosis and treatment codes achieved a better balance of sensitivity and specificity. Specificity increased and sensitivity decreased with increasing algorithm complexity. The best-performing algorithms, in terms of fewest misclassified patients, included multiple components (e.g., PH diagnosis, PAH treatment, continuous enrollment for ≥6 months before and ≥12 months following index date) and achieved sensitivities and specificities of around 95% and 38%, respectively. Our findings help researchers tailor their choice and design of code-based PAH algorithms to their research question and demonstrate the importance of including well-defined temporal components in the algorithms.

Electronic health records (EHRs) in clinical research and platform trials: Application of the innovative EHR-based methods developed by EU-PEARL.

OBJECTIVE: Electronic Health Record (EHR) systems are digital platforms in clinical practice used to collect patients' clinical information related to their health status and represents a useful storage of real-world data. EHRs have a potential role in research studies, in particular, in platform trials. Platform trials are innovative trial designs including multiple trial arms (conducted simultaneously and/or sequentially) on different treatments under a single master protocol. However, the use of EHRs in research comes with important challenges such as incompleteness of records and the need to translate trial eligibility criteria into interoperable queries. In this paper, we aim to review and to describe our proposed innovative methods to tackle some of the most important challenges identified. This work is part of the Innovative Medicines Initiative (IMI) EU Patient-cEntric clinicAl tRial pLatforms (EU-PEARL) project's work package 3 (WP3), whose objective is to deliver tools and guidance for EHR-based protocol feasibility assessment, clinical site selection, and patient pre-screening in platform trials, investing in the building of a data-driven clinical network framework that can execute these complex innovative designs for which feasibility assessments are critically important. METHODS: ISO standards and relevant references informed a readiness survey, producing 354 criteria with corresponding questions selected and harmonised through a 7-round scoring process (0-1) in stakeholder meetings, with 85% of consensus being the threshold of acceptance for a criterium/question. ATLAS cohort definition and Cohort Diagnostics were mainly used to create the trial feasibility eligibility (I/E) criteria as executable interoperable queries. RESULTS: The WP3/EU-PEARL group developed a readiness survey (eSurvey) for an efficient selection of clinical sites with suitable EHRs, consisting of yes-or-no questions, and a set-up of interoperable proxy queries using physicians' defined trial criteria. Both actions facilitate recruiting trial participants and alignment between study costs/timelines and data-driven recruitment potential. CONCLUSION: The eSurvey will help create an archive of clinical sites with mature EHR systems suitable to participate in clinical trials/platform trials, and the interoperable proxy queries of trial eligibility criteria will help identify the number of potential participants. Ultimately, these tools will contribute to the production of EHR-based protocol design.

Time to diagnosis of pulmonary hypertension and diagnostic burden: A retrospective analysis of nationwide US healthcare data.

The main aim of this analysis was to investigate time from symptom onset (chronic unexplained dyspnoea [CUD]) to diagnosis of Group 1 pulmonary hypertension (PH)-pulmonary arterial hypertension (PAH)-and to characterize healthcare resource utilization leading up to diagnosis using a nationwide US claims and an electronic health record (EHR) database from Optum©. Eligible patients were ≥18 years old at first CUD diagnosis (index event) and had a PAH diagnosis on or after index date. Based on administrative codes, PAH was defined as right heart catheterization (RHC), ≥ 2 PAH diagnoses (1 within a year of RHC), and ≥1 post-RHC prescription for PAH treatment. All values are median (1st quartile-3rd quartile) unless otherwise stated. Of 854,722 patients with CUD in the claims database, 582 (0.1%) had PAH. Time from CUD to PAH diagnosis was 2.26 (0.73-4.22) years. PAH patients experienced 3 (2-4) transthoracic echocardiograms (TTEs), 6 (3-12) specialist visits, and 2 (1-4) hospitalizations during the diagnostic interval. Almost one-third of patients (29%) waited 10 months or more to have a TTE. Findings from the EHR database were broadly similar. Resource utilization during the diagnostic interval was also analyzed in an overall PH cohort: findings were generally similar to the PAH cohort (2 [1-3] TTEs, 4 [2-9] specialist visits and 2 [1-4] hospitalizations). These data indicate a delay in the diagnostic pathway for PAH, and illustrate the burden associated with PAH diagnosis.

A machine learning approach to identifying patients with pulmonary hypertension using real-world electronic health records.

BACKGROUND: This study aimed to develop a machine learning (ML) model to identify patients who are likely to have pulmonary hypertension (PH), using a large patient-level US-based electronic health record (EHR) database. METHODS: A gradient boosting model, XGBoost, was developed using data from Optum's US-based de-identified EHR dataset (2007-2019). PH and disease control adult patients were identified using diagnostic, treatment and procedure codes and were randomly split into the training (90%) or test set (10%). Model features included patient demographics, physician visits, diagnoses, procedures, prescriptions, and laboratory test results. SHapley Additive exPlanations values were used to determine feature importance. RESULTS: We identified 11,279,478 control and 115,822 PH patients (mean age, respectively: 62 and 68 years, both 53% female). The final model used 165 features, with the most important predictive features including diagnosis of heart failure, shortness of breath and atrial fibrillation. The model predicted PH with an area under the receiver operating characteristic curve (AUROC) of 0.92. AUROC remained above 0.80 for the prediction of PH up to and beyond 18 months before diagnosis. Among the PH patients, we also identified 955 pulmonary arterial hypertension (PAH) and 1432 chronic thromboembolic pulmonary hypertension (CTEPH) patients, and the range of AUROCs obtained for these cohorts was 0.79-0.90 and 0.87-0.96, respectively. CONCLUSIONS: This model to detect PH based on patients' EHR records is viable and performs well in subgroups of PAH and CTEPH patients. This approach has the potential to improve patient outcomes by reducing diagnostic delay in PH.

Standardizing registry data to the OMOP Common Data Model: experience from three pulmonary hypertension databases.

BACKGROUND: The Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) can be used to transform observational health data to a common format. CDM transformation allows for analysis across disparate databases for the generation of new, real-word evidence, which is especially important in rare disease where data are limited. Pulmonary hypertension (PH) is a progressive, life-threatening disease, with rare subgroups such as pulmonary arterial hypertension (PAH), for which generating real-world evidence is challenging. Our objective is to document the process and outcomes of transforming registry data in PH to the OMOP CDM, and highlight challenges and our potential solutions. METHODS: Three observational studies were transformed from the Clinical Data Interchange Standards Consortium study data tabulation model (SDTM) to OMOP CDM format. OPUS was a prospective, multi-centre registry (2014-2020) and OrPHeUS was a retrospective, multi-centre chart review (2013-2017); both enrolled patients newly treated with macitentan in the US. EXPOSURE is a prospective, multi-centre cohort study (2017-ongoing) of patients newly treated with selexipag or any PAH-specific therapy in Europe and Canada. OMOP CDM version 5.3.1 with recent OMOP CDM vocabulary was used. Imputation rules were defined and applied for missing dates to avoid exclusion of data. Custom target concepts were introduced when existing concepts did not provide sufficient granularity. RESULTS: Of the 6622 patients in the three registry studies, records were mapped for 6457. Custom target concepts were introduced for PAH subgroups (by combining SNOMED concepts or creating custom concepts) and World Health Organization functional class. Per the OMOP CDM convention, records about the absence of an event, or the lack of information, were not mapped. Excluding these non-event records, 4% (OPUS), 2% (OrPHeUS) and 1% (EXPOSURE) of records were not mapped. CONCLUSIONS: SDTM data from three registries were transformed to the OMOP CDM with limited exclusion of data and deviation from the SDTM database content. Future researchers can apply our strategy and methods in different disease areas, with tailoring as necessary. Mapping registry data to the OMOP CDM facilitates more efficient collaborations between researchers and establishment of federated data networks, which is an unmet need in rare diseases.

Evaluation of code-based algorithms to identify pulmonary arterial hypertension and chronic thromboembolic pulmonary hypertension patients in large administrative databases.

Large administrative healthcare (including insurance claims) databases are used for various retrospective real-world evidence studies. However, in pulmonary arterial hypertension and chronic thromboembolic pulmonary hypertension, identifying patients retrospectively based on administrative codes remains challenging, as it relies on code combinations (algorithms) and the accuracy for patient identification of most of them is unknown. This study aimed to assess the performance of various algorithms in correctly identifying patients with pulmonary arterial hypertension or chronic thromboembolic pulmonary hypertension in administrative databases. A systematic literature review was performed to find publications detailing code-based algorithms used to identify pulmonary arterial hypertension and chronic thromboembolic pulmonary hypertension patients. PheValuator, a diagnostic predictive modelling tool, was applied to three US claims databases, yielding models that estimated the probability of a patient having the disease. These models were used to evaluate the performance characteristics of selected pulmonary arterial hypertension and chronic thromboembolic pulmonary hypertension algorithms. With increasing algorithm complexity, average positive predictive value increased (pulmonary arterial hypertension: 13.4-66.0%; chronic thromboembolic pulmonary hypertension: 10.3-75.1%) and average sensitivity decreased (pulmonary arterial hypertension: 61.5-2.7%; chronic thromboembolic pulmonary hypertension: 20.7-0.2%). Specificities and negative predictive values were high (≥97.5%) for all algorithms. Several of the algorithms performed well overall when considering all of these four performance parameters, and all algorithms performed with similar accuracy across the three claims databases studied, even though most were designed for patient identification in a specific database. Therefore, it is the objective of a study that will determine which algorithm may be most suitable; one- or two-component algorithms are most inclusive and three- or four-component algorithms identify most precise pulmonary arterial hypertension or chronic thromboembolic pulmonary hypertension populations, respectively.

Prediction of Real-World Drug Effectiveness Prelaunch: Case Study in Rheumatoid Arthritis.

BACKGROUND: Decision makers often need to assess the real-world effectiveness of new drugs prelaunch, when phase II/III randomized controlled trials (RCTs) but no other data are available. OBJECTIVE: To develop a method to predict drug effectiveness prelaunch and to apply it in a case study in rheumatoid arthritis (RA). METHODS: The approach 1) identifies a market-approved treatment ( S) currently used in a target population similar to that of the new drug ( N); 2) quantifies the impact of treatment, prognostic factors, and effect modifiers on clinical outcome; 3) determines the characteristics of patients likely to receive N in routine care; and 4) predicts treatment outcome in simulated patients with these characteristics. Sources of evidence include expert opinion, RCTs, and observational studies. The framework relies on generalized linear models. RESULTS: The case study assessed the effectiveness of tocilizumab (TCZ), a biologic disease-modifying antirheumatic drug (DMARD), combined with conventional DMARDs, compared to conventional DMARDs alone. Rituximab (RTX) combined with conventional DMARDs was identified as treatment S. Individual participant data from 2 RCTs and 2 national registries were analyzed. The model predicted the 6-month changes in the Disease Activity Score 28 (DAS28) accurately: the mean change was -2.101 (standard deviation [SD] = 1.494) in the simulated patients receiving TCZ and conventional DMARDs compared to -1.873 (SD = 1.220) in retrospectively assessed observational data. It was -0.792 (SD = 1.499) in registry patients treated with conventional DMARDs. CONCLUSION: The approach performed well in the RA case study, but further work is required to better define its strengths and limitations.