A Framework for Systematic Assessment of Clinical Trial Population Representativeness Using Electronic Health Records Data.

BACKGROUND: Clinical trials are the gold standard for generating robust medical evidence, but clinical trial results often raise generalizability concerns, which can be attributed to the lack of population representativeness. The electronic health records (EHRs) data are useful for estimating the population representativeness of clinical trial study population. OBJECTIVES: This research aims to estimate the population representativeness of clinical trials systematically using EHR data during the early design stage. METHODS: We present an end-to-end analytical framework for transforming free-text clinical trial eligibility criteria into executable database queries conformant with the Observational Medical Outcomes Partnership Common Data Model and for systematically quantifying the population representativeness for each clinical trial. RESULTS: We calculated the population representativeness of 782 novel coronavirus disease 2019 (COVID-19) trials and 3,827 type 2 diabetes mellitus (T2DM) trials in the United States respectively using this framework. With the use of overly restrictive eligibility criteria, 85.7% of the COVID-19 trials and 30.1% of T2DM trials had poor population representativeness. CONCLUSION: This research demonstrates the potential of using the EHR data to assess the clinical trials population representativeness, providing data-driven metrics to inform the selection and optimization of eligibility criteria.

Clinical comparison between trial participants and potentially eligible patients using electronic health record data: A generalizability assessment method.

OBJECTIVE: To present a generalizability assessment method that compares baseline clinical characteristics of trial participants (TP) to potentially eligible (PE) patients as presented in their electronic health record (EHR) data while controlling for clinical setting and recruitment period. METHODS: For each clinical trial, a clinical event was defined to identify patients of interest using available EHR data from one clinical setting during the trial's recruitment timeframe. The trial's eligibility criteria were then applied and patients were separated into two mutually exclusive groups: (1) TP, which were patients that participated in the trial per trial enrollment data; (2) PE, the remaining patients. The primary outcome was standardized differences in clinical characteristics between TP and PE per trial. A standardized difference was considered prominent if its absolute value was greater than or equal to 0.1. The secondary outcome was the difference in mean propensity scores (PS) between TP and PE per trial, in which the PS represented prediction for a patient to be in the trial. Three diverse trials were selected for illustration: one focused on hepatitis C virus (HCV) patients receiving a liver transplantation; one focused on leukemia patients and lymphoma patients; and one focused on appendicitis patients. RESULTS: For the HCV trial, 43 TP and 83 PE were found, with 61 characteristics evaluated. Prominent differences were found among 69% of characteristics, with a mean PS difference of 0.13. For the leukemia/lymphoma trial, 23 TP and 23 PE were found, with 39 characteristics evaluated. Prominent differences were found among 82% of characteristics, with a mean PS difference of 0.76. For the appendicitis trial, 123 TP and 242 PE were found, with 52 characteristics evaluated. Prominent differences were found among 52% of characteristics, with a mean PS difference of 0.15. CONCLUSIONS: Differences in clinical characteristics were observed between TP and PE among all three trials. In two of the three trials, not all of the differences necessarily compromised trial generalizability and subsets of PE could be considered similar to their corresponding TP. In the remaining trial, lack of generalizability appeared present, but may be a result of other factors such as small sample size or site recruitment strategy. These inconsistent findings suggest eligibility criteria alone are sometimes insufficient in defining a target group to generalize to. With caveats in limited scalability, EHR data quality, and lack of patient perspective on trial participation, this generalizability assessment method that incorporates control for temporality and clinical setting promise to better pinpoint clinical patterns and trial considerations.

A Data Element-Function Conceptual Model for Data Quality Checks.

INTRODUCTION: In aggregate, existing data quality (DQ) checks are currently represented in heterogeneous formats, making it difficult to compare, categorize, and index checks. This study contributes a data element-function conceptual model to facilitate the categorization and indexing of DQ checks and explores the feasibility of leveraging natural language processing (NLP) for scalable acquisition of knowledge of common data elements and functions from DQ checks narratives. METHODS: The model defines a "data element", the primary focus of the check, and a "function", the qualitative or quantitative measure over a data element. We applied NLP techniques to extract both from 172 checks for Observational Health Data Sciences and Informatics (OHDSI) and 3,434 checks for Kaiser Permanente's Center for Effectiveness and Safety Research (CESR). RESULTS: The model was able to classify all checks. A total of 751 unique data elements and 24 unique functions were extracted. The top five frequent data element-function pairings for OHDSI were Person-Count (55 checks), Insurance-Distribution (17), Medication-Count (16), Condition-Count (14), and Observations-Count (13); for CESR, they were Medication-Variable Type (175), Medication-Missing (172), Medication-Existence (152), Medication-Count (127), and Socioeconomic Factors-Variable Type (114). CONCLUSIONS: This study shows the efficacy of the data element-function conceptual model for classifying DQ checks, demonstrates early promise of NLP-assisted knowledge acquisition, and reveals the great heterogeneity in the focus in DQ checks, confirming variation in intrinsic checks and use-case specific "fitness-for-use" checks.

Comparative effectiveness of generic and brand-name medication use: A database study of US health insurance claims.

BACKGROUND: To the extent that outcomes are mediated through negative perceptions of generics (the nocebo effect), observational studies comparing brand-name and generic drugs are susceptible to bias favoring the brand-name drugs. We used authorized generic (AG) products, which are identical in composition and appearance to brand-name products but are marketed as generics, as a control group to address this bias in an evaluation aiming to compare the effectiveness of generic versus brand medications. METHODS AND FINDINGS: For commercial health insurance enrollees from the US, administrative claims data were derived from 2 databases: (1) Optum Clinformatics Data Mart (years: 2004-2013) and (2) Truven MarketScan (years: 2003-2015). For a total of 8 drug products, the following groups were compared using a cohort study design: (1) patients switching from brand-name products to AGs versus generics, and patients initiating treatment with AGs versus generics, where AG use proxied brand-name use, addressing negative perception bias, and (2) patients initiating generic versus brand-name products (bias-prone direct comparison) and patients initiating AG versus brand-name products (negative control). Using Cox proportional hazards regression after 1:1 propensity-score matching, we compared a composite cardiovascular endpoint (for amlodipine, amlodipine-benazepril, and quinapril), non-vertebral fracture (for alendronate and calcitonin), psychiatric hospitalization rate (for sertraline and escitalopram), and insulin initiation (for glipizide) between the groups. Inverse variance meta-analytic methods were used to pool adjusted hazard ratios (HRs) for each comparison between the 2 databases. Across 8 products, 2,264,774 matched pairs of patients were included in the comparisons of AGs versus generics. A majority (12 out of 16) of the clinical endpoint estimates showed similar outcomes between AGs and generics. Among the other 4 estimates that did have significantly different outcomes, 3 suggested improved outcomes with generics and 1 favored AGs (patients switching from amlodipine brand-name: HR [95% CI] 0.92 [0.88-0.97]). The comparison between generic and brand-name initiators involved 1,313,161 matched pairs, and no differences in outcomes were noted for alendronate, calcitonin, glipizide, or quinapril. We observed a lower risk of the composite cardiovascular endpoint with generics versus brand-name products for amlodipine and amlodipine-benazepril (HR [95% CI]: 0.91 [0.84-0.99] and 0.84 [0.76-0.94], respectively). For escitalopram and sertraline, we observed higher rates of psychiatric hospitalizations with generics (HR [95% CI]: 1.05 [1.01-1.10] and 1.07 [1.01-1.14], respectively). The negative control comparisons also indicated potentially higher rates of similar magnitude with AG compared to brand-name initiation for escitalopram and sertraline (HR [95% CI]: 1.06 [0.98-1.13] and 1.11 [1.05-1.18], respectively), suggesting that the differences observed between brand and generic users in these outcomes are likely explained by either residual confounding or generic perception bias. Limitations of this study include potential residual confounding due to the unavailability of certain clinical parameters in administrative claims data and the inability to evaluate surrogate outcomes, such as immediate changes in blood pressure, upon switching from brand products to generics. CONCLUSIONS: In this study, we observed that use of generics was associated with comparable clinical outcomes to use of brand-name products. These results could help in promoting educational interventions aimed at increasing patient and provider confidence in the ability of generic medicines to manage chronic diseases.

Differences in rates of switchbacks after switching from branded to authorized generic and branded to generic drug products: cohort study.

OBJECTIVES: To compare rates of switchbacks to branded drug products for patients switched from branded to authorized generic drug products, which have the same active ingredients, appearance, and excipients as the branded product, with patients switched from branded to generic drug products, which have the same active ingredients as the branded product but may differ in appearance and excipients. DESIGN: Observational cohort study. SETTING: Private (a large commercial health plan) and public (Medicaid) insurance programs in the US. PARTICIPANTS: Beneficiaries of a large US commercial health insurer between 2004 and 2013 (primary cohort) and Medicaid beneficiaries between 2000 and 2010 (replication cohort). MAIN OUTCOME MEASURES: Patients taking branded products for one of the study drugs (alendronate tablets, amlodipine tablets, amlodipine-benazepril capsules, calcitonin salmon nasal spray, escitalopram tablets, glipizide extended release tablets, quinapril tablets, and sertraline tablets) were identified when they switched to an authorized generic or a generic drug product after the date of market entry of generic drug products. These patients were followed for switchbacks to the branded drug product in the year after their switch to an authorized generic or a generic drug product. Cox proportional hazard models were used to estimate hazard ratios and 95% confidence intervals after adjusting for demographics, including age, sex, and calendar year. Inverse variance meta-analysis was used to pool adjusted hazard ratios across all drug products. RESULTS: A total of 94 909 patients switched from branded to authorized generic drug products and 116 017 patients switched from branded to generic drug products and contributed to the switchback analysis. Unadjusted incidence rates of switchback varied across drug products, ranging from a low of 3.8 per 100 person years (for alendronate tablets) to a high of 17.8 per 100 person years (for amlodipine-benazepril capsules), with an overall rate of 8.2 per 100 person years across all drug products. Adjusted switchback rates were consistently lower for patients who switched from branded to authorized generic drug products compared with branded to generic drug products in the primary cohort (pooled hazard ratio 0.72, 95% confidence interval 0.64 to 0.81). Similar results (0.75, 0.62 to 0.91) were observed in the replication cohort. CONCLUSION: Switching from branded to authorized generic drug products was associated with lower switchback rates compared with switching from branded to generic drug products.

The relative benefits of claims and electronic health record data for predicting medication adherence trajectory.

BACKGROUND: Healthcare providers are increasingly encouraged to improve their patients' adherence to chronic disease medications. Prediction of adherence can identify patients in need of intervention, but most prediction efforts have focused on claims data, which may be unavailable to providers. Electronic health records (EHR) are readily available and may provide richer information with which to predict adherence than is currently available through claims. METHODS: In a linked database of complete Medicare Advantage claims and comprehensive EHR from a multi-specialty outpatient practice, we identified patients who filled a prescription for a statin, antihypertensive, or oral antidiabetic during 2011 to 2012. We followed patients to identify subsequent medication filling patterns and used group-based trajectory models to assign patients to adherence trajectories. We then identified potential predictors from both claims and EHR data and fit a series of models to evaluate the accuracy of each data source in predicting medication adherence. RESULTS: Claims were highly predictive of patients in the worst adherence trajectory (C=0.78), but EHR data also provided good predictions (C=0.72). Among claims predictors, presence of a prior gap in filling of at least 6 days was by far the most influential predictor. In contrast, good predictions from EHR data required complex models with many variables. CONCLUSION: EHR data can provide good predictions of adherence trajectory and therefore may be useful for providers seeking to deploy resource-intensive interventions. However, prior adherence information derived from claims is most predictive, and can supplement EHR data when it is available.

Assessment of coronary vascular function with cardiac PET in relation to serum uric acid.

BACKGROUND: Elevated serum uric acid (SUA) levels have been independently associated with cardiovascular disease. Stress myocardial perfusion positron emission tomography (PET) allows for measurement of absolute myocardial blood flow (MBF) and quantification of global left ventricular coronary flow reserve (CFR). A CFR <2.0 is considered impaired coronary vascular function, and it is associated with increased cardiovascular risk. We evaluated the relationship between SUA and PET-measured markers of coronary vascular function. METHODS: We studied adults undergoing a stress myocardial perfusion PET on clinical grounds (1/2006-3/2014) who also had ≥1 SUA measurement within 180 days from the PET date. Multivariable linear regression estimated the association between SUA and PET-derived MBF and CFR. We also stratified analyses by diabetes status. RESULTS: We included 382 patients with mean (SD) age of 68.4 (12.4) years and mean (SD) SUA level of 7.2 (2.6) mg/dl. 36% were female and 29% had gout. Median [IQR] CFR was reduced at 1.6 [1.2, 2.0] and median [IQR] stress MBF was 1.5 [1.1, 2.1] ml/min/g. In the adjusted analysis, SUA was inversely associated with stress MBF (ß = -0.14, p = 0.01) but not with CFR. Among patients without diabetes (n = 215), SUA had a negative association with CFR (ß = -0.15, p = 0.02) and stress MBF (ß = -0.19, p = 0.01) adjusting for age, sex, extent of myocardial scar and ischemia, serum creatinine and gout. In diabetic patients (n = 167), SUA was not associated with either CFR or MBF. CONCLUSIONS: In this cross-sectional study, higher SUA is modestly associated with worse CFR and stress MBF among patients without diabetes.

Comparison of Benefit-Risk Assessment Methods for Prospective Monitoring of Newly Marketed Drugs: A Simulation Study.

OBJECTIVES: To compare benefit-risk assessment (BRA) methods for determining whether and when sufficient evidence exists to indicate that one drug is favorable over another in prospective monitoring. METHODS: We simulated prospective monitoring of a new drug (A) versus an alternative drug (B) with respect to two beneficial and three harmful outcomes. We generated data for 1000 iterations of six scenarios and applied four BRA metrics: number needed to treat and number needed to harm (NNT|NNH), incremental net benefit (INB) with maximum acceptable risk, INB with relative-value-adjusted life-years, and INB with quality-adjusted life-years. We determined the proportion of iterations in which the 99% confidence interval for each metric included and excluded the null and we calculated mean time to alerting. RESULTS: With no true difference in any outcome between drugs A and B, the proportion of iterations including the null was lowest for INB with relative-value-adjusted life-years (64%) and highest for INB with quality-adjusted life-years (76%). When drug A was more effective and the drugs were equally safe, all metrics indicated net favorability of A in more than 70% of the iterations. When drug A was safer than drug B, NNT|NNH had the highest proportion of iterations indicating net favorability of drug A (65%). Mean time to alerting was similar among methods across the six scenarios. CONCLUSIONS: BRA metrics can be useful for identifying net favorability when applied to prospective monitoring of a new drug versus an alternative drug. INB-based approaches similarly outperform unweighted NNT|NNH approaches. Time to alerting was similar across approaches.

Prospective Benefit-Risk Monitoring of New Drugs for Rapid Assessment of Net Favorability in Electronic Health Care Data.

BACKGROUND: Benefit-risk assessment (BRA) methods can combine measures of benefits and risks into a single value. OBJECTIVES: To examine BRA metrics for prospective monitoring of new drugs in electronic health care data. METHODS: Using two electronic health care databases, we emulated prospective monitoring of three drugs (rofecoxib vs. nonselective nonsteroidal anti-inflammatory drugs, prasugrel vs. clopidogrel, and denosumab vs. bisphosphonates) using a sequential propensity score-matched cohort design. We applied four BRA metrics: number needed to treat and number needed to harm; incremental net benefit (INB) with maximum acceptable risk; INB with relative-value-adjusted life-years; and INB with quality-adjusted life-years (QALYs). We determined whether and when the bootstrapped 99% confidence interval (CI) for each metric excluded zero, indicating net favorability of one drug over the other. RESULTS: For rofecoxib, all four metrics yielded a negative value, suggesting net favorability of nonselective nonsteroidal anti-inflammatory drugs over rofecoxib, and the 99% CI for all but the number needed to treat and number needed to harm excluded the null during follow-up. For prasugrel, only the 99% CI for INB-QALY excluded the null, but trends in values over time were similar across the four metrics, suggesting overall net favorability of prasugrel versus clopidogrel. The 99% CI for INB-relative-value-adjusted life-years and INB-QALY excluded the null in the denosumab example, suggesting net favorability of denosumab over bisphosphonates. CONCLUSIONS: Prospective benefit-risk monitoring can be used to determine net favorability of a new drug in electronic health care data. In three examples, existing BRA metrics produced qualitatively similar results but differed with respect to alert generation.