RESUMO
We sought to determine whether machine learning and natural language processing (NLP) applied to electronic medical records could improve performance of automated health-care claims-based algorithms to identify anaphylaxis events using data on 516 patients with outpatient, emergency department, or inpatient anaphylaxis diagnosis codes during 2015-2019 in 2 integrated health-care institutions in the Northwest United States. We used one site's manually reviewed gold-standard outcomes data for model development and the other's for external validation based on cross-validated area under the receiver operating characteristic curve (AUC), positive predictive value (PPV), and sensitivity. In the development site 154 (64%) of 239 potential events met adjudication criteria for anaphylaxis compared with 180 (65%) of 277 in the validation site. Logistic regression models using only structured claims data achieved a cross-validated AUC of 0.58 (95% CI: 0.54, 0.63). Machine learning improved cross-validated AUC to 0.62 (0.58, 0.66); incorporating NLP-derived covariates further increased cross-validated AUCs to 0.70 (0.66, 0.75) in development and 0.67 (0.63, 0.71) in external validation data. A classification threshold with cross-validated PPV of 79% and cross-validated sensitivity of 66% in development data had cross-validated PPV of 78% and cross-validated sensitivity of 56% in external data. Machine learning and NLP-derived data improved identification of validated anaphylaxis events.
Assuntos
Anafilaxia , Processamento de Linguagem Natural , Humanos , Anafilaxia/diagnóstico , Anafilaxia/epidemiologia , Aprendizado de Máquina , Algoritmos , Serviço Hospitalar de Emergência , Registros Eletrônicos de SaúdeRESUMO
INTRODUCTION: Transfusion-related acute lung injury (TRALI), an adverse event occurring during or within 6 hours of transfusion, is a leading cause of transfusion-associated fatalities reported to the US Food and Drug Administration. There is limited information on the validity of diagnosis codes for TRALI recorded in inpatient electronic medical records (EMRs). STUDY DESIGNS AND METHODS: We conducted a validation study to establish the positive predictive value (PPV) of TRALI International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes recorded within a large hospital system between 2013 and 2015. A physician with critical care expertise confirmed the TRALI diagnosis. As TRALI is likely underdiagnosed, we used the specific code (518.7), and codes for respiratory failure (518.82) in combination with transfusion reaction (999.80, 999.89, E934.7). RESULTS: Among almost four million inpatient stays, we identified 208 potential TRALI cases with ICD-9-CM codes and reviewed 195 medical records; 68 (35%) met clinical definitions for TRALI (26 [38%] definitive, 15 [22%] possible, 27 [40%] delayed). Overall, the PPV for all inpatient TRALI diagnoses was 35% (95% confidence interval (CI), 28-42). The PPV for the TRALI-specific code was 44% (95% CI, 35-54). CONCLUSION: We observed low PPVs (<50%) for TRALI ICD-9-CM diagnosis codes as validated by medical charts, which may relate to inconsistent code use, incomplete medical records, or other factors. Future studies using TRALI diagnosis codes in EMR databases may consider confirming diagnoses with medical records, assessing TRALI ICD, Tenth Revision, Clinical Modification codes, or exploring alternative ways for of accurately identifying TRALI in EMR databases. KEY POINTS: In 169 hospitals, we identified 208 potential TRALI cases, reviewed 195 charts, and confirmed 68 (35%) cases met TRALI clinical definitions. As many potential TRALI cases identified with diagnosis codes did not meet clinical definitions, medical record confirmation may be prudent.
Assuntos
Transfusão de Sangue , Insuficiência Respiratória/complicações , Reação Transfusional/complicações , Lesão Pulmonar Aguda Relacionada à Transfusão/diagnóstico , Adolescente , Adulto , Idoso , Idoso de 80 Anos ou mais , Transfusão de Sangue/mortalidade , Transfusão de Sangue/estatística & dados numéricos , Criança , Pré-Escolar , Bases de Dados Factuais , Registros Eletrônicos de Saúde/estatística & dados numéricos , Feminino , Hospitalização , Hospitais , Humanos , Lactente , Pacientes Internados , Classificação Internacional de Doenças , Masculino , Pessoa de Meia-Idade , Projetos Piloto , Valor Preditivo dos Testes , Respiração Artificial , Lesão Pulmonar Aguda Relacionada à Transfusão/mortalidade , Estados Unidos , United States Food and Drug AdministrationRESUMO
PURPOSE: Identifying hospitalizations for serious infections among patients dispensed biologic therapies within healthcare databases is important for post-marketing surveillance of these drugs. We determined the positive predictive value (PPV) of an ICD-10-CM-based diagnostic coding algorithm to identify hospitalization for serious infection among patients dispensed biologic therapy within the FDA's Sentinel Distributed Database. METHODS: We identified health plan members who met the following algorithm criteria: (1) hospital ICD-10-CM discharge diagnosis of serious infection between July 1, 2016 and August 31, 2018; (2) either outpatient/emergency department infection diagnosis or outpatient antimicrobial treatment within 7 days prior to hospitalization; (3) inflammatory bowel disease, psoriasis, or rheumatological diagnosis within 1 year prior to hospitalization, and (4) were dispensed outpatient biologic therapy within 90 days prior to admission. Medical records were reviewed by infectious disease clinicians to adjudicate hospitalizations for serious infection. The PPV (95% confidence interval [CI]) for confirmed events was determined after further weighting by the prevalence of the type of serious infection in the database. RESULTS: Among 223 selected health plan members who met the algorithm, 209 (93.7% [95% CI, 90.1%-96.9%]) were confirmed to have a hospitalization for serious infection. After weighting by the prevalence of the type of serious infection, the PPV of the ICD-10-CM algorithm identifying a hospitalization for serious infection was 80.2% (95% CI, 75.3%-84.7%). CONCLUSIONS: The ICD-10-CM-based algorithm for hospitalization for serious infection among patients dispensed biologic therapies within the Sentinel Distributed Database had 80% PPV for confirmed events and could be considered for use within pharmacoepidemiologic studies.
Assuntos
Hospitalização , Classificação Internacional de Doenças , Terapia Biológica , Bases de Dados Factuais , Humanos , FarmacoepidemiologiaRESUMO
Use of disease risk score (DRS)-based confounding adjustment when estimating treatment effects on multiple outcomes is not well studied. We designed an empirical cohort study to compare dabigatran initiators and warfarin initiators with respect to risks of ischemic stroke and major bleeding in 12 sequential monitoring periods (90 days each), using data from the Truven Marketscan database (Truven Health Analytics, Ann Arbor, Michigan). We implemented 2 approaches to combine DRS for multiple outcomes: 1) 1:1 matching on prognostic propensity scores (PPS), created using DRS for bleeding and stroke as independent variables in a propensity score (PS) model; and 2) simultaneous 1:1 matching on DRS for bleeding and stroke using Mahalanobis distance (M-distance), and compared their performance with that of traditional PS matching. M-distance matching appeared to produce more stable results in the early marketing period than both PPS and traditional PS matching; hazard ratios from unadjusted analysis, traditional PS matching, PPS matching, and M-distance matching after 4 periods were 0.72 (95% confidence interval (CI): 0.51, 1.03), 0.61 (95% CI: 0.31, 1.09), 0.55 (95% CI: 0.33, 0.91), and 0.78 (95% CI: 0.45, 1.34), respectively, for stroke and 0.65 (95% CI: 0.53, 0.80), 0.78 (95% CI: 0.60, 1.01), 0.75 (95% CI: 0.59, 0.96), and 0.78 (95% CI: 0.64, 0.95), respectively, for bleeding. In later periods, estimates were similar for traditional PS matching and M-distance matching but suggested potential residual confounding with PPS matching. These results suggest that M-distance matching may be a valid approach for extension of DRS-based confounding adjustments for multiple outcomes of interest.
Assuntos
Fatores de Confusão Epidemiológicos , Projetos de Pesquisa Epidemiológica , Medição de Risco/métodos , Anticoagulantes/administração & dosagem , Simulação por Computador , Dabigatrana/administração & dosagem , Interpretação Estatística de Dados , Hemorragia/induzido quimicamente , Humanos , Pontuação de Propensão , Acidente Vascular Cerebral/prevenção & controle , Varfarina/administração & dosagemRESUMO
INTRODUCTION: The psychological burden of type 1 diabetes mellitus (T1DM) is considerable. The condition affects the daily lives of adults living with T1DM (ALWT1DM) in many ways. International guidelines highlight the importance of providing psychological support to ALWT1DM to improve health outcomes and well-being. METHODS: We conducted a systematic literature review of randomised controlled trials (RCTs) to identify the evidence on the impact of psychological interventions on glycaemic control and psychological outcomes in ALWT1DM. Literature searches of Medline, Embase, Cochrane Central Register of Controlled Trials, PsycInfo, and the grey literature were performed to identify relevant RCTs, published in English, from 2001 onward. Fourteen RCTs of ten psychological interventions in ALWT1DM were eligible and included in the qualitative synthesis. The studies varied considerably in terms of duration, target population, endpoints, and efficacy. RESULTS: Overall, psychological interventions did not perform significantly better than control treatments in improving glycaemic control, although selected patient groups did report benefits from some psychological intervention types, such as cognitive behavioural therapy. Although most of the psychological interventions produced small, nonsignificant improvements in self-reported patient functioning, some treatments were effective in reducing diabetes distress and improving mental health, even if no impact on glycaemic control was observed. DISCUSSION: Current guidelines for the treatment of T1DM recommend access to psychological services; however, there is a paucity of high-quality evidence from clinical trials on the effectiveness or preferred structure of psychological support. There is a clear need for more rigorous, large-scale, international research to address the efficacy of psychological interventions in ALWT1DM.
RESUMO
Many people with Type 2 diabetes (T2D) who could benefit from digital health technologies (DHTs) are either not using DHTs or do use them, but not for long enough to reach their behavioral or metabolic goals. We aimed to identify subgroups within DHT adopters and non-adopters and describe their unique profiles to better understand the type of tailored support needed to promote effective and sustained DHT use across a diverse T2D population. We conducted latent class analysis of a sample of adults with T2D who responded to an internet survey between December 2021 and March 2022. We describe the clinical and psychological characteristics of DHT adopters and non-adopters, and their attitudes toward DHTs. A total of 633 individuals were characterized as either DHT "Adopters" (n = 376 reporting any use of DHT) or "Non-Adopters" (n = 257 reporting never using any DHT). Within Adopters, three subgroups were identified: 21% (79/376) were "Self-managing Adopters," who reported high health activation and self-efficacy for diabetes management, 42% (158/376) were "Activated Adopters with dropout risk," and 37% (139/376) were "Non-Activated Adopters with dropout risk." The latter two subgroups reported barriers to using DHTs and lower rates of intended future use. Within Non-Adopters, two subgroups were identified: 31% (79/257) were "Activated Non-Adopters," and 69% (178/257) were "Non-Adopters with barriers," and were similarly distinguished by health activation and barriers to using DHTs. Beyond demographic characteristics, psychological, and clinical factors may help identify different subgroups of Adopters and Non-Adopters.
In this study, we characterized subgroups of adopters and non-adopters of digital health technologies (DHTs) for managing Type 2 diabetes, such as apps to track nutrition, continuous glucose monitors, and activity monitors like Fitbit. Self-efficacy for diabetes management, health activation, and perceived barriers to use DHT emerged as characteristics that distinguished subgroups. Notably, subgroups of adopters differed in their interest to use these technologies in the next 3 months; groups with low levels of self-efficacy and health activation were least interested in using them and thus at risk of discontinuing use. The ability to identify these subgroups can inform strategies tailored to each subgroup that motivate adoption of DHTs and promote long-term engagement.
Assuntos
Diabetes Mellitus Tipo 2 , Análise de Classes Latentes , Humanos , Diabetes Mellitus Tipo 2/psicologia , Diabetes Mellitus Tipo 2/terapia , Masculino , Feminino , Pessoa de Meia-Idade , Adulto , Idoso , Comportamentos Relacionados com a Saúde , Tecnologia Digital , Inquéritos e Questionários , Tecnologia Biomédica , Saúde DigitalRESUMO
BACKGROUND: Collaboration between people with type 2 diabetes (T2DM) and their health care teams is important for optimal control of the disease and outcomes. Digital technologies could potentially tie together several health care-related devices and platforms into connected ecosystems (CES), but attitudes about CES are unknown. OBJECTIVE: We surveyed convenience samples of patients and physicians to better understand which patient characteristics are associated with higher likelihoods of (1) participating in a potential CES program, as self-reported by patients with T2DM and (2) clinical benefit from participation in a potential CES program, as reported by physicians. METHODS: Adults self-reporting a diagnosis of T2DM and current insulin use (n=197), and 33 physicians whose practices included ≥20% of such patients, were enrolled in the United States, France, and Germany. We surveyed both groups about the likelihood of patient participation in a CES. We then examined the associations between patients' clinical and sociodemographic characteristics and this likelihood. We also described characteristics of patients likely to clinically benefit from CES use, according to physicians. RESULTS: Compared with patients in Germany and France, US patients were younger (mean age 45.3 [SD 11.9] years vs 61.9 [SD 9.2] and 65.8 [SD 9.4] years, respectively), more often female, more highly educated, and more often working full-time. In all, 51 (44.7%) US patients, 16 (36.4%) German patients, and 18 (46.3%) French patients indicated strong interest in a CES program, and 115 (78.7%) reported currently using ≥1 connected device or app. However, physicians believed that only 11.3%-19.2% of their patients were using connected devices or apps to manage their disease. Physicians also reported infrequently recommending or prescribing connected devices to their patients, although ≥80% (n=28) of them thought that a CES could help support their patients in managing their disease. The factors most predictive of patient likelihood of participating in a CES program were cost, inclusion of medication reminders, and linking blood glucose levels to behaviors such as eating and exercise. In all countries, the most common patient expectations for a CES program were that it could help them eat more healthfully, increase their physical activity, increase their understanding of how blood glucose relates to behavior such as exercise and eating, and reduce stress. Physicians thought that newly diagnosed patients, sicker patients-those who had been hospitalized for diabetes, were currently using insulin, or who had any comorbid condition-and patients who were nonadherent to treatment were most likely to benefit from CES use. CONCLUSIONS: In this study, there was a high degree of interest in the future use of CES, although additional education is needed among both patients with T2DM and their physicians to achieve the full potential of such systems to improve self-management and clinical care for the disease.
RESUMO
OBJECTIVE: Claims-based algorithms are used in the Food and Drug Administration Sentinel Active Risk Identification and Analysis System to identify occurrences of health outcomes of interest (HOIs) for medical product safety assessment. This project aimed to apply machine learning classification techniques to demonstrate the feasibility of developing a claims-based algorithm to predict an HOI in structured electronic health record (EHR) data. MATERIALS AND METHODS: We used the 2015-2019 IBM MarketScan Explorys Claims-EMR Data Set, linking administrative claims and EHR data at the patient level. We focused on a single HOI, rhabdomyolysis, defined by EHR laboratory test results. Using claims-based predictors, we applied machine learning techniques to predict the HOI: logistic regression, LASSO (least absolute shrinkage and selection operator), random forests, support vector machines, artificial neural nets, and an ensemble method (Super Learner). RESULTS: The study cohort included 32 956 patients and 39 499 encounters. Model performance (positive predictive value [PPV], sensitivity, specificity, area under the receiver-operating characteristic curve) varied considerably across techniques. The area under the receiver-operating characteristic curve exceeded 0.80 in most model variations. DISCUSSION: For the main Food and Drug Administration use case of assessing risk of rhabdomyolysis after drug use, a model with a high PPV is typically preferred. The Super Learner ensemble model without adjustment for class imbalance achieved a PPV of 75.6%, substantially better than a previously used human expert-developed model (PPV = 44.0%). CONCLUSIONS: It is feasible to use machine learning methods to predict an EHR-derived HOI with claims-based predictors. Modeling strategies can be adapted for intended uses, including surveillance, identification of cases for chart review, and outcomes research.
Assuntos
Registros Eletrônicos de Saúde , Aprendizado de Máquina , Eletrônica , Humanos , Avaliação de Resultados em Cuidados de Saúde , Projetos PilotoRESUMO
INTRODUCTION: While medical chart review remains the gold standard to validate health conditions or events identified in administrative claims and electronic health record databases, it is time consuming, expensive and can involve subjective decisions. AIM: The aim of this study was to describe the landscape of technology-enhanced approaches that could be used to facilitate medical chart review within and across distributed data networks. METHOD: We conducted a semi-structured survey regarding processes for medical chart review with organizations that either routinely do medical chart review or use technologies that could facilitate chart review. RESULTS: Fifteen out of 17 interviewed organizations used optical character recognition (OCR) or natural language processing (NLP) in their chart review process. None used handwriting recognition software. While these organizations found OCR and NLP to be useful for expediting extraction of useful information from medical charts, they also mentioned several challenges. Quality of medical scans can be variable, interfering with the accuracy of OCR. Additionally, linguistic complexity in medical notes and heterogeneity in reporting templates used by different healthcare systems can reduce the transportability of NLP-based algorithms to diverse healthcare settings. CONCLUSION: New technologies including OCR and NLP are currently in use by various organizations involved in medical chart review. While technology-enhanced approaches could scale up capacity to validate key variables and make information about important clinical variables from medical records more generally available for research purposes, they often require considerable customization when employed in a distributed data environment with multiple, diverse healthcare settings.