RESUMEN
Importance: The Sentinel System is a key component of the US Food and Drug Administration (FDA) postmarketing safety surveillance commitment and uses clinical health care data to conduct analyses to inform drug labeling and safety communications, FDA advisory committee meetings, and other regulatory decisions. However, observational data are frequently deemed insufficient for reliable evaluation of safety concerns owing to limitations in underlying data or methodology. Advances in large language models (LLMs) provide new opportunities to address some of these limitations. However, careful consideration is necessary for how and where LLMs can be effectively deployed for these purposes. Observations: LLMs may provide new avenues to support signal-identification activities to identify novel adverse event signals from narrative text of electronic health records. These algorithms may be used to support epidemiologic investigations examining the causal relationship between exposure to a medical product and an adverse event through development of probabilistic phenotyping of health outcomes of interest and extraction of information related to important confounding factors. LLMs may perform like traditional natural language processing tools by annotating text with controlled vocabularies with additional tailored training activities. LLMs offer opportunities for enhancing information extraction from adverse event reports, medical literature, and other biomedical knowledge sources. There are several challenges that must be considered when leveraging LLMs for postmarket surveillance. Prompt engineering is needed to ensure that LLM-extracted associations are accurate and specific. LLMs require extensive infrastructure to use, which many health care systems lack, and this can impact diversity, equity, and inclusion, and result in obscuring significant adverse event patterns in some populations. LLMs are known to generate nonfactual statements, which could lead to false positive signals and downstream evaluation activities by the FDA and other entities, incurring substantial cost. Conclusions and Relevance: LLMs represent a novel paradigm that may facilitate generation of information to support medical product postmarket surveillance activities that have not been possible. However, additional work is required to ensure LLMs can be used in a fair and equitable manner, minimize false positive findings, and support the necessary rigor of signal detection needed for regulatory activities.
Asunto(s)
Procesamiento de Lenguaje Natural , Vigilancia de Productos Comercializados , United States Food and Drug Administration , Vigilancia de Productos Comercializados/métodos , Humanos , Estados Unidos , Registros Electrónicos de SaludRESUMEN
Background: Current guidelines recommend simultaneous initiation of multidrug guideline-directed medical therapy classes for heart failure with reduced ejection fraction. Objectives: The purpose of this study was to evaluate county-level variation in use of triple guideline-directed medical therapy, defined as simultaneous prescription fills for beta-blockers, renin-angiotensin system inhibitors or angiotensin receptor neprilysin inhibitors, and mineralocorticoid receptor antagonists, in heart failure with reduced ejection fraction. Methods: We conducted a cohort study using Medicare Fee-for-Service claims data (parts A, B, and D between 2013 and 2019). Features of counties including area-level indicators of poverty, employment, and educational attainment and aggregated patient-level sociodemographic and medical history variables were compared by quintiles of triple therapy use. A multilevel logistic regression model was constructed to estimate the contextual effect of clustering by counties, which was expressed as a median OR. Results: 304,857 patients from 2,600 counties (83% of all U.S. counties) were included. The median for triple therapy use was 14.3% (IQR: 10.3%-18.8%) across included counties with a wide variation (range: 0%-54.5%). Compared to counties in the highest use quintile, counties in lowest triple therapy use quintile had worse area-level indicators of socioeconomic status (% unemployment 6.8% vs 6.2%). Counties in lowest quintile had higher proportion of Black patients (13.3% vs 5.7% in highest quintile) and patients with low-income subsidy (29.3% vs 25.8% in highest quintile). The median OR was 1.30 (95% CI: 1.28-1.33). Conclusions: We observed variation in triple therapy use across counties in the United States with suboptimal local use patterns correlating with indicators of socioeconomic disadvantage.
RESUMEN
Electronic health record (EHR) data are seen as an important source for Pharmacoepidemiology studies. In the US healthcare system, EHR systems often only identify fragments of patients' health information across the care continuum, including primary care, specialist care, hospitalizations, and pharmacy dispensing. This leads to unobservable information in longitudinal evaluations of medication effects causing unmeasured confounding, misclassification, and truncated follow-up times. A remedy is to link EHR data with longitudinal claims data which record all encounters during a defined enrollment period across all care settings. We evaluate EHR and claims data sources in three aspects relevant to etiologic studies of medical products: data continuity, data granularity, and data chronology. Reflecting on the strengths and limitations of EHR and insurance claims data, it becomes obvious that they complement each other. The combination of both will improve the validity of etiologic studies and expand the range of questions that can be answered. As the research community transitions towards a future state with access to large-scale combined EHR+claims data, we outline analytic templates to improve the validity and broaden the scope of pharmacoepidemiology studies in the current environment where EHR data are available only for a subset of patients with claims data.
RESUMEN
BACKGROUND: Unmeasured confounding is often raised as a source of potential bias during the design of non-randomized studies but quantifying such concerns is challenging. METHODS: We developed a simulation-based approach to assess the potential impact of unmeasured confounding during the study design stage. The approach involved generation of hypothetical individual-level cohorts using realistic parameters including a binary treatment (prevalence 25%), a time-to-event outcome (incidence 5%), 13 measured covariates, a binary unmeasured confounder (u1, 10%), and a binary measured 'proxy' variable (p1) correlated with u1. Strength of unmeasured confounding and correlations between u1 and p1 were varied in simulation scenarios. Treatment effects were estimated with, a) no adjustment, b) adjustment for measured confounders (Level 1), c) adjustment for measured confounders and their proxy (Level 2). We computed absolute standardized mean differences in u1 and p1 and relative bias with each level of adjustment. RESULTS: Across all scenarios, Level 2 adjustment led to improvement in balance of u1, but this improvement was highly dependent on the correlation between u1 and p1. Level 2 adjustments also had lower relative bias than Level 1 adjustments (in strong u1 scenarios: relative bias of 9.2%, 12.2%, 13.5% at correlations 0.7, 0.5, and 0.3, respectively versus 16.4%, 15.8%, 15.0% for Level 1, respectively). CONCLUSION: An approach using simulated individual-level data was useful to explicitly convey the potential for bias due to unmeasured confounding while designing non-randomized studies and can be helpful in informing design choices.
RESUMEN
BACKGROUND: Machine learning-based approaches that seek to accomplish individualized treatment effect prediction have gained traction; however, some salient challenges lack wider recognition. METHODS: We describe key methodologic considerations for individualized treatment effect prediction models using data from the Treatment of Preserved Cardiac Function Heart Failure with an Aldosterone Antagonist Trial for spironolactone in heart failure with preserved ejection fraction. The causal survival forest algorithm was used for model development. Calibration and discrimination were evaluated using a bootstrapping-based internal validation procedure. Observed benefits were described for predicted benefit quartiles and quartiles of a known effect modifier: ejection fraction. A negative control analysis with noncardiovascular death as the outcome was implemented to detect confounding. RESULTS: Among 3445 participants, 671 events occurred over a median of 3.3 years of follow-up. In internal validation, a higher average observed benefit was noted among patients in the highest quartile of predicted benefit. The median (interquartile range) of the observed restricted mean survival time difference at 3.3 years at the highest quartile of model-predicted benefit was 62 days (32 to 83) and was 47 days (26 to 67) at the lowest quartile of ejection fraction. Body-mass index had higher contribution to prediction of benefit relative to other included measures (33.7% vs. glomerular filtration rate [27.3%], ejection fraction [15.1%], and younger age [12.8%]) No benefit was observed for noncardiovascular death at higher model-predicted benefit quartiles, although benefit for noncardiovascular death was observed at lower quartiles. CONCLUSIONS: Carefully applied and validated predictive models hold promise in identifying heterogeneous treatment effects and are useful for hypothesis generation regarding the role of phenotypic characteristics in modifying the benefit of experimental interventions in clinical trials. (Funded by the National Heart, Lung, and Blood Institute; ClinicalTrials.gov number, NCT00094302.).
Asunto(s)
Insuficiencia Cardíaca , Aprendizaje Automático , Antagonistas de Receptores de Mineralocorticoides , Espironolactona , Anciano , Femenino , Humanos , Masculino , Persona de Mediana Edad , Algoritmos , Insuficiencia Cardíaca/tratamiento farmacológico , Insuficiencia Cardíaca/mortalidad , Insuficiencia Cardíaca/fisiopatología , Antagonistas de Receptores de Mineralocorticoides/uso terapéutico , Medicina de Precisión/métodos , Espironolactona/uso terapéutico , Volumen Sistólico/efectos de los fármacos , Resultado del TratamientoRESUMEN
Objective: Partially observed confounder data pose challenges to the statistical analysis of electronic health records (EHR) and systematic assessments of potentially underlying missingness mechanisms are lacking. We aimed to provide a principled approach to empirically characterize missing data processes and investigate performance of analytic methods. Methods: Three empirical sub-cohorts of diabetic SGLT2 or DPP4-inhibitor initiators with complete information on HbA1c, BMI and smoking as confounders of interest (COI) formed the basis of data simulation under a plasmode framework. A true null treatment effect, including the COI in the outcome generation model, and four missingness mechanisms for the COI were simulated: completely at random (MCAR), at random (MAR), and two not at random (MNAR) mechanisms, where missingness was dependent on an unmeasured confounder and on the value of the COI itself. We evaluated the ability of three groups of diagnostics to differentiate between mechanisms: 1)-differences in characteristics between patients with or without the observed COI (using averaged standardized mean differences [ASMD]), 2)-predictive ability of the missingness indicator based on observed covariates, and 3)-association of the missingness indicator with the outcome. We then compared analytic methods including "complete case", inverse probability weighting, single and multiple imputation in their ability to recover true treatment effects. Results: The diagnostics successfully identified characteristic patterns of simulated missingness mechanisms. For MAR, but not MCAR, the patient characteristics showed substantial differences (median ASMD 0.20 vs 0.05) and consequently, discrimination of the prediction models for missingness was also higher (0.59 vs 0.50). For MNAR, but not MAR or MCAR, missingness was significantly associated with the outcome even in models adjusting for other observed covariates. Comparing analytic methods, multiple imputation using a random forest algorithm resulted in the lowest root-mean-squared-error. Conclusion: Principled diagnostics provided reliable insights into missingness mechanisms. When assumptions allow, multiple imputation with nonparametric models could help reduce bias.
RESUMEN
Lasso regression is widely used for large-scale propensity score (PS) estimation in healthcare database studies. In these settings, previous work has shown that undersmoothing (overfitting) Lasso PS models can improve confounding control, but it can also cause problems of non-overlap in covariate distributions. It remains unclear how to select the degree of undersmoothing when fitting large-scale Lasso PS models to improve confounding control while avoiding issues that can result from reduced covariate overlap. Here, we used simulations to evaluate the performance of using collaborative-controlled targeted learning to data-adaptively select the degree of undersmoothing when fitting large-scale PS models within both singly and doubly robust frameworks to reduce bias in causal estimators. Simulations showed that collaborative learning can data-adaptively select the degree of undersmoothing to reduce bias in estimated treatment effects. Results further showed that when fitting undersmoothed Lasso PS-models, the use of cross-fitting was important for avoiding non-overlap in covariate distributions and reducing bias in causal estimates.
RESUMEN
Objectives: Partially observed confounder data pose a major challenge in statistical analyses aimed to inform causal inference using electronic health records (EHRs). While analytic approaches such as imputation are available, assumptions on underlying missingness patterns and mechanisms must be verified. We aimed to develop a toolkit to streamline missing data diagnostics to guide choice of analytic approaches based on meeting necessary assumptions. Materials and methods: We developed the smdi (structural missing data investigations) R package based on results of a previous simulation study which considered structural assumptions of common missing data mechanisms in EHR. Results: smdi enables users to run principled missing data investigations on partially observed confounders and implement functions to visualize, describe, and infer potential missingness patterns and mechanisms based on observed data. Conclusions: The smdi R package is freely available on CRAN and can provide valuable insights into underlying missingness patterns and mechanisms and thereby help improve the robustness of real-world evidence studies.
RESUMEN
OBJECTIVE: To determine whether annual changes in prices for clinician-administered drugs are associated with changes in patient out-of-pocket costs. DATA SOURCES AND STUDY SETTING: National commercial claims database, 2009 to 2018. STUDY DESIGN: In a serial, cross-sectional study, we calculated the annual percent change in manufacturer list prices and net prices after rebates. We used two-part generalized linear models to assess the relationship between annual changes in price with (1) the percentage of individuals incurring any out-of-pocket costs and (2) the percent change in median non-zero out-of-pocket costs. DATA COLLECTION/EXTRACTION METHODS: We created annual cohorts of privately insured individuals who used one of 52 brand-name clinician-administered drugs. PRINCIPAL FINDINGS: List prices increased 4.4%/yr (interquartile range [IQR], 1.1% to 6.0%) and net prices 3.3%/yr (IQR, 0.3% to 5.5%). The median percentage of patients with any out-of-pocket costs increased from 38% in 2009 to 48% in 2018, and median non-zero annual out-of-pocket costs increased by 9.6%/yr (IQR, 4.1% to 15.4%). There was no association between changes in prices and out-of-pocket costs for individual drugs. CONCLUSIONS: From 2009 to 2018, prices and out-of-pocket costs for brand-name clinician-administered drugs increased, but these were not directly related for individual drugs. This may be due to changes to insurance benefit design and private insurer drug reimbursement rates.
RESUMEN
RATIONALE & OBJECTIVE: Head-to-head data comparing the effectiveness and safety of oral anticoagulants in patients with atrial fibrillation (AF) and advanced chronic kidney disease (CKD) are lacking. We compared the safety and effectiveness of warfarin or rivaroxaban versus apixaban in patients with AF and non-dialysis-dependent CKD stage 4/5. STUDY DESIGN: Propensity score-matched cohort study. SETTING & PARTICIPANTS: 2 nationwide US claims databases, Medicare and Optum's deidentified Clinformatics Data Mart Database, were searched for the interval from January 1, 2013, through March 31, 2022, for patients with nonvalvular AF and CKD stage 4/5 who initiated warfarin versus apixaban (matched cohort, n=12,488) and rivaroxaban versus apixaban (matched cohort, n = 5,720). EXPOSURES: Warfarin, rivaroxaban, or apixaban. OUTCOMES: Primary outcomes included major bleeding and ischemic stroke. Secondary outcomes included all-cause mortality, major gastrointestinal bleeding, and intracranial bleeding. ANALYTICAL APPROACH: Cox regression was used to estimate HRs, and 1:1 propensity-score matching was used to adjust for 80 potential confounders. RESULTS: Compared with apixaban, warfarin initiation was associated with a higher rate of major bleeding (HR, 1.85; 95% CI, 1.59-2.15), including major gastrointestinal bleeding (1.86; 1.53-2.25) and intracranial bleeding (2.15; 1.42-3.25). Compared with apixaban, rivaroxaban was also associated with a higher rate of major bleeding (1.69; 1.33-2.15). All-cause mortality was similar for warfarin (1.08; 0.98-1.18) and rivaroxaban (0.94; 0.81-1.10) versus apixaban. Furthermore, no statistically significant differences for ischemic stroke were observed for warfarin (1.14; 0.83-1.57) or rivaroxaban (0.71; 0.40-1.24) versus apixaban, but the CIs were wide. Similar results were observed for warfarin versus apixaban in the positive control cohort of patients with CKD stage 3, consistent with randomized trial findings. LIMITATIONS: Few ischemic stroke events, potential residual confounding. CONCLUSIONS: In patients with AF and advanced CKD, rivaroxaban and warfarin were associated with higher rates of major bleeding compared with apixaban, suggesting a superior safety profile for apixaban in this high-risk population. PLAIN-LANGUAGE SUMMARY: Different anticoagulants have been shown to reduce the risk of stroke in patients with atrial fibrillation, such as warfarin and direct oral anticoagulants like apixaban and rivaroxaban. Unfortunately, the large-scale randomized trials that compared direct anticoagulants versus warfarin excluded patients with advanced chronic kidney disease. Therefore, the comparative safety and effectiveness of warfarin, apixaban, and rivaroxaban are uncertain in this population. In this study, we used administrative claims data from the United States to answer this question. We found that warfarin and rivaroxaban were associated with increased risks of major bleeding compared with apixaban. There were few stroke events, with no major differences among the 3 drugs in the risk of stroke. In conclusion, this study suggests that apixaban has a better safety profile than warfarin and rivaroxaban.
Asunto(s)
Fibrilación Atrial , Accidente Cerebrovascular Isquémico , Pirazoles , Insuficiencia Renal Crónica , Accidente Cerebrovascular , Humanos , Anciano , Estados Unidos/epidemiología , Warfarina/efectos adversos , Rivaroxabán/efectos adversos , Fibrilación Atrial/tratamiento farmacológico , Fibrilación Atrial/epidemiología , Estudios de Cohortes , Estudios Retrospectivos , Medicare , Anticoagulantes/efectos adversos , Piridonas/efectos adversos , Accidente Cerebrovascular/epidemiología , Accidente Cerebrovascular/etiología , Accidente Cerebrovascular/prevención & control , Hemorragia Gastrointestinal/inducido químicamente , Hemorragia Gastrointestinal/complicaciones , Insuficiencia Renal Crónica/complicaciones , Insuficiencia Renal Crónica/epidemiología , Insuficiencia Renal Crónica/inducido químicamenteRESUMEN
BACKGROUND: We aimed to determine whether integrating concepts from the notes from the electronic health record (EHR) data using natural language processing (NLP) could improve the identification of gout flares. METHODS: Using Medicare claims linked with EHR, we selected gout patients who initiated the urate-lowering therapy (ULT). Patients' 12-month baseline period and on-treatment follow-up were segmented into 1-month units. We retrieved EHR notes for months with gout diagnosis codes and processed notes for NLP concepts. We selected a random sample of 500 patients and reviewed each of their notes for the presence of a physician-documented gout flare. Months containing at least 1 note mentioning gout flares were considered months with events. We used 60% of patients to train predictive models with LASSO. We evaluated the models by the area under the curve (AUC) in the validation data and examined positive/negative predictive values (P/NPV). RESULTS: We extracted and labeled 839 months of follow-up (280 with gout flares). The claims-only model selected 20 variables (AUC = 0.69). The NLP concept-only model selected 15 (AUC = 0.69). The combined model selected 32 claims variables and 13 NLP concepts (AUC = 0.73). The claims-only model had a PPV of 0.64 [0.50, 0.77] and an NPV of 0.71 [0.65, 0.76], whereas the combined model had a PPV of 0.76 [0.61, 0.88] and an NPV of 0.71 [0.65, 0.76]. CONCLUSION: Adding NLP concept variables to claims variables resulted in a small improvement in the identification of gout flares. Our data-driven claims-only model and our combined claims/NLP-concept model outperformed existing rule-based claims algorithms reliant on medication use, diagnosis, and procedure codes.
Asunto(s)
Gota , Anciano , Humanos , Estados Unidos/epidemiología , Gota/diagnóstico , Gota/epidemiología , Procesamiento de Lenguaje Natural , Registros Electrónicos de Salud , Medicare , Brote de los Síntomas , AlgoritmosRESUMEN
Post marketing safety surveillance depends in part on the ability to detect concerning clinical events at scale. Spontaneous reporting might be an effective component of safety surveillance, but it requires awareness and understanding among healthcare professionals to achieve its potential. Reliance on readily available structured data such as diagnostic codes risk under-coding and imprecision. Clinical textual data might bridge these gaps, and natural language processing (NLP) has been shown to aid in scalable phenotyping across healthcare records in multiple clinical domains. In this study, we developed and validated a novel incident phenotyping approach using unstructured clinical textual data agnostic to Electronic Health Record (EHR) and note type. It's based on a published, validated approach (PheRe) used to ascertain social determinants of health and suicidality across entire healthcare records. To demonstrate generalizability, we validated this approach on two separate phenotypes that share common challenges with respect to accurate ascertainment: 1) suicide attempt; 2) sleep-related behaviors. With samples of 89,428 records and 35,863 records for suicide attempt and sleep-related behaviors, respectively, we conducted silver standard (diagnostic coding) and gold standard (manual chart review) validation. We showed Area Under the Precision-Recall Curve of â¼ 0.77 (95% CI 0.75-0.78) for suicide attempt and AUPR â¼ 0.31 (95% CI 0.28-0.34) for sleep-related behaviors. We also evaluated performance by coded race and demonstrated differences in performance by race were dissimilar across phenotypes and require algorithmovigilance and debiasing prior to implementation.
RESUMEN
OBJECTIVE: The objective of this study was to compare the clinical effectiveness of biologic disease-modifying antirheumatic drugs (bDMARDs) or Janus kinase inhibitors (JAKi) among seropositive versus seronegative patients with rheumatoid arthritis (RA) in a real-world setting. METHODS: We used Optum's deidentified Clinformatics Data Mart Database (January 1, 2004, to March 31, 2021) linked with outpatient laboratory test results. The study population was adult patients with RA who initiated a bDMARD or JAKi. The index date was the dispensing of the first-ever study drug. At least 1-year continuous enrollment before and after the index date was required. Disenrollment due to death after the index date was allowed. Serostatus was defined using laboratory test results or the International Classification of Diseases, 10th Revision code M05x or M06.0x any time prior to the index date. Treatment effectiveness was measured based on a claims-based composite endpoint at 1-year post index, including nonoccurrence of any of the following: addition of conventional synthetic DMARDs, addition of or switching to new bDMARDs/JAKi, initiation of glucocorticoids, increased glucocorticoid dose, or death. Log-binomial regression models were constructed to estimate the risk ratio (RR) with 95% confidence interval (CI) comparing seropositive patients with seronegative patients, adjusting for more than 60 baseline covariates. RESULTS: We identified a total of 7813 seropositive patients and 4202 seronegative patients. The mean (±SD) age was 56.7 (±14.0) years; 77.9% were female. The risk of 1-year treatment effectiveness was 70.2% among seropositive patients and 69.8% among seronegative patients. The adjusted RR (95% CI) was 1.00 (0.98-1.02). CONCLUSION: In this real-world cohort study, seropositive and seronegative patients with RA had similar 1-year treatment effectiveness after initiating a bDMARD/JAKi.
RESUMEN
AIMS: The effectiveness of sodium-glucose cotransporter 2 inhibitors (SGLT2i) in patients with heart failure (HF) in routine clinical practice is not extensively studied. This study aimed to evaluate the comparative effectiveness of SGLT2i vs. sitagliptin in older adults with HF and type 2 diabetes and to investigate whether there were any differences between agents within the SGLT2i class or for reduced and preserved ejection fraction. METHODS AND RESULTS: Using Medicare claims data (April 2013 to December 2019), 16 253 SGLT2i initiators vs. 43 352 initiators of sitagliptin aged ≥65 years with type 2 diabetes and HF were included. The primary outcome was a composite of all-cause mortality, hospitalization for HF or urgent visit requiring intravenous diuretics; secondary outcomes included its individual components. Propensity score fine stratification weighted Cox regression was used to adjust for 100 pre-exposure characteristics. Mean age was 74 years; 49.8% were women. Initiation of SGLT2i vs. sitagliptin was associated with a lower risk of the primary composite outcome [adjusted hazard ratio (HR) 0.72; 95% confidence interval 0.67-0.77]. The adjusted HRs were 0.70 (0.63-0.78) for all-cause mortality, 0.64 (0.58-0.70) for hospitalization for HF, and 0.77 (0.69-0.86) for urgent visit requiring intravenous diuretics. Similar associations with the primary composite outcome were observed for all three agents within the SGLT2i class, for reduced and preserved ejection fraction, and subgroups based on demographics, comorbidities, and other HF treatments. Bias-calibrated HRs for the primary endpoint using negative and positive control outcomes ranged between 0.81 and 0.89, suggesting that the observed benefit could not be fully explained by residual confounding. CONCLUSION: In routine US clinical practice, SGLT2i demonstrated robust clinical effectiveness in older adults with HF and type 2 diabetes compared with sitagliptin, with no evidence of heterogeneity across the SGLT2i class or across ejection fraction.
Asunto(s)
Diabetes Mellitus Tipo 2 , Insuficiencia Cardíaca , Fosfato de Sitagliptina , Inhibidores del Cotransportador de Sodio-Glucosa 2 , Humanos , Inhibidores del Cotransportador de Sodio-Glucosa 2/uso terapéutico , Insuficiencia Cardíaca/complicaciones , Insuficiencia Cardíaca/tratamiento farmacológico , Insuficiencia Cardíaca/mortalidad , Fosfato de Sitagliptina/uso terapéutico , Estudios de Cohortes , Anciano , Masculino , Femenino , Diabetes Mellitus Tipo 2/complicaciones , Diabetes Mellitus Tipo 2/tratamiento farmacológico , Canagliflozina/uso terapéutico , Compuestos de Bencidrilo/uso terapéutico , Glucósidos/uso terapéutico , Insuficiencia Cardíaca Diastólica/epidemiología , Hospitalización , Medicare , Resultado del TratamientoRESUMEN
INTRODUCTION: Pharmacovigilance programs protect patient health and safety by identifying adverse event signals through postmarketing surveillance of claims data and spontaneous reports. Electronic health records (EHRs) provide new opportunities to address limitations of traditional approaches and promote discovery-oriented pharmacovigilance. METHODS: To evaluate the current state of EHR-based medication safety signal identification, we conducted a scoping literature review of studies aimed at identifying safety signals from routinely collected patient-level EHR data. We extracted information on study design, EHR data elements utilized, analytic methods employed, drugs and outcomes evaluated, and key statistical and data analysis choices. RESULTS: We identified 81 eligible studies. Disproportionality methods were the predominant analytic approach, followed by data mining and regression. Variability in study design makes direct comparisons difficult. Studies varied widely in terms of data, confounding adjustment, and statistical considerations. CONCLUSION: Despite broad interest in utilizing EHRs for safety signal identification, current efforts fail to leverage the full breadth and depth of available data or to rigorously control for confounding. The development of best practices and application of common data models would promote the expansion of EHR-based pharmacovigilance.
Asunto(s)
Sistemas de Registro de Reacción Adversa a Medicamentos , Registros Electrónicos de Salud , Humanos , Farmacovigilancia , Minería de DatosAsunto(s)
Insuficiencia Cardíaca , Humanos , Estados Unidos/epidemiología , Anciano , Insuficiencia Cardíaca/tratamiento farmacológico , Insuficiencia Cardíaca/diagnóstico , Volumen Sistólico , Valsartán , Aminobutiratos/uso terapéutico , Compuestos de Bifenilo , Combinación de Medicamentos , Antagonistas de Receptores de Angiotensina/uso terapéutico , Tetrazoles/uso terapéuticoRESUMEN
The impact of electronic health record (EHR) discontinuity (i.e., receiving care outside of a given EHR system) on EHR-based risk prediction is unknown. We aimed to assess the impact of EHR-continuity on the performance of clinical risk scores. The study cohort consisted of patients aged ≥ 65 years with ≥ 1 EHR encounter in the 2 networks in Massachusetts (MA; 2007/1/1-2017/12/31, internal training and validation dataset), and one network in North Carolina (NC; 2007/1/1-2016/12/31, external validation dataset) that were linked with Medicare claims data. Risk scores were calculated using EHR data alone vs. linked EHR-claims data (not subject to misclassification due to EHR-discontinuity): (i) combined comorbidity score (CCS), (ii) claim-based frailty score (CFI), (iii) CHAD2 DS2 -VASc, and (iv) Hypertension, Abnormal renal/liver function, Stroke, Bleeding, Labile, Elderly, and Drugs (HAS-BLED). We assessed the performance of CCS and CFI predicting death, CHAD2 DS2 -VASc predicting ischemic stroke, and HAS-BLED predicting bleeding by area under receiver operating characteristic curve (AUROC), stratified by quartiles of predicted EHR-continuity (Q1-4). There were 319,740 patients in the MA systems and 125,380 in the NC system. In the external validation dataset, AUROC for EHR-based CCS predicting 1-year risk of death was 0.583 in Q1 (lowest) EHR-continuity group, which increased to 0.739 in Q4 (highest) EHR-continuity group. The corresponding improvement in AUROC was 0.539 to 0.647 for CFI, 0.556 to 0.637 for CHAD2 DS2 -VASc, and 0.517 to 0.556 for HAS-BLED. The AUROC in Q4 EHR-continuity group based on EHR alone approximates that based on EHR-claims data. The prediction performance of four clinical risk scores was substantially worse in patients with lower vs. high EHR-continuity.
Asunto(s)
Fibrilación Atrial , Accidente Cerebrovascular , Humanos , Anciano , Estados Unidos , Registros Electrónicos de Salud , Medición de Riesgo , Medicare , Factores de Riesgo , HemorragiaRESUMEN
Importance: Nonrandomized studies using insurance claims databases can be analyzed to produce real-world evidence on the effectiveness of medical products. Given the lack of baseline randomization and measurement issues, concerns exist about whether such studies produce unbiased treatment effect estimates. Objective: To emulate the design of 30 completed and 2 ongoing randomized clinical trials (RCTs) of medications with database studies using observational analogues of the RCT design parameters (population, intervention, comparator, outcome, time [PICOT]) and to quantify agreement in RCT-database study pairs. Design, Setting, and Participants: New-user cohort studies with propensity score matching using 3 US claims databases (Optum Clinformatics, MarketScan, and Medicare). Inclusion-exclusion criteria for each database study were prespecified to emulate the corresponding RCT. RCTs were explicitly selected based on feasibility, including power, key confounders, and end points more likely to be emulated with real-world data. All 32 protocols were registered on ClinicalTrials.gov before conducting analyses. Emulations were conducted from 2017 through 2022. Exposures: Therapies for multiple clinical conditions were included. Main Outcomes and Measures: Database study emulations focused on the primary outcome of the corresponding RCT. Findings of database studies were compared with RCTs using predefined metrics, including Pearson correlation coefficients and binary metrics based on statistical significance agreement, estimate agreement, and standardized difference. Results: In these highly selected RCTs, the overall observed agreement between the RCT and the database emulation results was a Pearson correlation of 0.82 (95% CI, 0.64-0.91), with 75% meeting statistical significance, 66% estimate agreement, and 75% standardized difference agreement. In a post hoc analysis limited to 16 RCTs with closer emulation of trial design and measurements, concordance was higher (Pearson r, 0.93; 95% CI, 0.79-0.97; 94% meeting statistical significance, 88% estimate agreement, 88% standardized difference agreement). Weaker concordance occurred among 16 RCTs for which close emulation of certain design elements that define the research question (PICOT) with data from insurance claims was not possible (Pearson r, 0.53; 95% CI, 0.00-0.83; 56% meeting statistical significance, 50% estimate agreement, 69% standardized difference agreement). Conclusions and Relevance: Real-world evidence studies can reach similar conclusions as RCTs when design and measurements can be closely emulated, but this may be difficult to achieve. Concordance in results varied depending on the agreement metric. Emulation differences, chance, and residual confounding can contribute to divergence in results and are difficult to disentangle.