RESUMEN
BACKGROUND: Evidence-based practice in community-acquired pneumonia often assumes an accurate initial diagnosis. OBJECTIVE: To examine the evolution of pneumonia diagnoses among patients hospitalized from the emergency department (ED). DESIGN: Retrospective nationwide cohort. SETTING: 118 U.S. Veterans Affairs medical centers. PATIENTS: Aged 18 years or older and hospitalized from the ED between 1 January 2015 and 31 January 2022. MEASUREMENTS: Discordances between initial pneumonia diagnosis, discharge diagnosis, and radiographic diagnosis identified by natural language processing of clinician text, diagnostic coding, and antimicrobial treatment. Expressions of uncertainty in clinical notes, patient illness severity, treatments, and outcomes were compared. RESULTS: Among 2 383 899 hospitalizations, 13.3% received an initial or discharge diagnosis and treatment of pneumonia: 9.1% received an initial diagnosis and 10.0% received a discharge diagnosis. Discordances between initial and discharge occurred in 57%. Among patients discharged with a pneumonia diagnosis and positive initial chest image, 33% lacked an initial diagnosis. Among patients diagnosed initially, 36% lacked a discharge diagnosis and 21% lacked positive initial chest imaging. Uncertainty was frequently expressed in clinical notes (58% in ED; 48% at discharge); 27% received diuretics, 36% received corticosteroids, and 10% received antibiotics, corticosteroids, and diuretics within 24 hours. Patients with discordant diagnoses had greater uncertainty and received more additional treatments, but only patients lacking an initial pneumonia diagnosis had higher 30-day mortality than concordant patients (14.4% [95% CI, 14.1% to 14.7%] vs. 10.6% [CI, 10.4% to 10.7%]). Patients with diagnostic discordance were more likely to present to high-complexity facilities with high ED patient load and inpatient census. LIMITATION: Retrospective analysis; did not examine causal relationships. CONCLUSION: More than half of all patients hospitalized and treated for pneumonia had discordant diagnoses from initial presentation to discharge. Treatments for other diagnoses and expressions of uncertainty were common. These findings highlight the need to recognize diagnostic uncertainty and treatment ambiguity in research and practice of pneumonia-related care. PRIMARY FUNDING SOURCE: The Gordon and Betty Moore Foundation.
RESUMEN
Diagnostic error, a cause of substantial morbidity and mortality, is largely discovered and evaluated through self-report and manual review, which is costly and not suitable to real-time intervention. Opportunities exist to leverage electronic health record data for automated detection of potential misdiagnosis, executed at scale and generalized across diseases. We propose a novel automated approach to identifying diagnostic divergence considering both diagnosis and risk of mortality. Our objective was to identify cases of emergency department infectious disease misdiagnoses by measuring the deviation between predicted diagnosis and documented diagnosis, weighted by mortality. Two machine learning models were trained for prediction of infectious disease and mortality using the first 24h of data. Charts were manually reviewed by clinicians to determine whether there could have been a more correct or timely diagnosis. The proposed approach was validated against manual reviews and compared using the Spearman rank correlation. We analyzed 6.5 million ED visits and over 700 million associated clinical features from over one hundred emergency departments. The testing set performances of the infectious disease (Macro F1 = 86.7, AUROC 90.6 to 94.7) and mortality model (Macro F1 = 97.6, AUROC 89.1 to 89.1) were in expected ranges. Human reviews and the proposed automated metric demonstrated positive correlations ranging from 0.231 to 0.358. The proposed approach for diagnostic deviation shows promise as a potential tool for clinicians to find diagnostic errors. Given the vast number of clinical features used in this analysis, further improvements likely need to either take greater account of data structure (what occurs before when) or involve natural language processing. Further work is needed to explain the potential reasons for divergence and to refine and validate the approach for implementation in real-world settings.
RESUMEN
The Department of Veterans Affairs (VA) aims to reduce homelessness among veterans through programs such as Supportive Services for Veteran Families (SSVF). An important component of SSVF is temporary financial assistance. Previous research has demonstrated the effectiveness of temporary financial assistance in reducing short-term housing instability, but studies have not examined its long-term effect on housing outcomes. Using data from the VA's electronic health record system, we analyzed the effect of temporary financial assistance on veterans' housing instability for three years after entry into SSVF. We extracted housing outcomes from clinical notes, using natural language processing, and compared the probability of unstable housing among veterans who did and did not receive temporary financial assistance. We found that temporary financial assistance rapidly reduced the probability of unstable housing, but the effect attenuated after forty-five days. Our findings suggest that to maintain long-term housing stability for veterans who have exited SSVF, additional interventions may be needed.
Asunto(s)
Personas con Mala Vivienda , Veteranos , Estados Unidos , Humanos , Vivienda , United States Department of Veterans Affairs , ProbabilidadRESUMEN
Written clinical language embodies and reflects the clinician's mental models of disease. Prior to the COVID-19 pandemic, pneumonia was shifting away from concern for healthcare-associated pneumonia and toward recognition of heterogeneity of pathogens and host response. How these models are reflected in clinical language or whether they were impacted by the pandemic has not been studied. We aimed to assess changes in the language used to describe pneumonia following the COVID-19 pandemic.
Asunto(s)
COVID-19 , Neumonía , Humanos , COVID-19/diagnóstico , Pandemias , Neumonía/diagnóstico , Lingüística , Lenguaje , Prueba de COVID-19RESUMEN
Introduction: Measuring long-term housing outcomes is important for evaluating the impacts of services for individuals with homeless experience. However, assessing long-term housing status using traditional methods is challenging. The Veterans Affairs (VA) Electronic Health Record (EHR) provides detailed data for a large population of patients with homeless experiences and contains several indicators of housing instability, including structured data elements (e.g., diagnosis codes) and free-text clinical narratives. However, the validity of each of these data elements for measuring housing stability over time is not well-studied. Methods: We compared VA EHR indicators of housing instability, including information extracted from clinical notes using natural language processing (NLP), with patient-reported housing outcomes in a cohort of homeless-experienced Veterans. Results: NLP achieved higher sensitivity and specificity than standard diagnosis codes for detecting episodes of unstable housing. Other structured data elements in the VA EHR showed promising performance, particularly when combined with NLP. Discussion: Evaluation efforts and research studies assessing longitudinal housing outcomes should incorporate multiple data sources of documentation to achieve optimal performance.
RESUMEN
OBJECTIVES: Age is important for prognosis in community-onset pneumonia, but how it influences admission decisions in the emergency department (ED) is not well characterized. Using clinical data from the electronic health record in a national cohort, we examined pneumonia hospitalization patterns, variation, and relationships with mortality among older versus younger Veterans. METHODS: In a retrospective cohort of patients ≥ 18 years presenting to EDs with a diagnosis of pneumonia at 118 VA Medical Centers January 1, 2006, to December 31, 2016, we compared observed, predicted, and residual hospitalization risk for Veterans < 70, 70-79, and ≥ 80 years of age using generalized estimating equations and machine learning models with 71 patient factors. We examined facility variation in residual hospitalization across facilities and explored whether facility differences in hospitalization risk correlated with differences in 30-day mortality. RESULTS: Among 297,498 encounters, 165,003 (55%) were for Veterans < 70 years, 61,076 (21%) 70-80, and 71,419 (24%) ≥ 80. Hospitalization rates were 52%, 67%, and 76%, respectively. After other patient factors were adjusting for, age 70-79 had an odds ratio (OR) of 1.39 (95% confidence interval [CI] 1.34-1.44) and ≥ 80 had an OR of 2.1 (95% CI 2.0-2.2) compared to age < 70. There was substantial variation in hospitalization across facilities among Veterans < 70 (<35% hospitalization at the lowest decile of facilities vs. > 66% at the highest decile) that was similar but with higher risk for patients 70-79 years (54% vs. 82%) and ≥ 80 years (59% vs. 85%) and remained after accounting for patient factors, with no consistently positive or negative associations with facility-level 30-day mortality. CONCLUSIONS: Older Veterans with community-onset pneumonia experience high risk of hospitalization, with widespread facility variation that has no clear relationship to short-term mortality.
Asunto(s)
Neumonía , Veteranos , Humanos , Estados Unidos/epidemiología , Anciano , Estudios Retrospectivos , Hospitalización , Hospitales , Neumonía/terapiaRESUMEN
Homelessness prevention and rapid rehousing (RRH) programs are increasingly important components of the homeless assistance system in the United States. Yet, there are key gaps in knowledge about the dynamics of the utilization of these programs, with scant attention paid to examining the duration of homelessness prevention and RRH service episodes or to patterns of repeated use of these programs over time. To address these gaps, we use data from the U.S. Department of Veterans Affairs' (VA) Supportive Services for Veteran Families (SSVF) program-the largest program in the country providing homelessness prevention and RRH services-to assess the relationship between individual and program-level factors and exits to stable housing, length of service episodes, and patterns of repeated service use over time. We analyze data for a primary cohort of 570,798 of Veterans who received SSVF services during Fiscal Years (FY) 2012-2021, and for separate cohorts of Veterans who received SSVF prevention and RRH services, respectively, during FY 2016-2021. We find that participants' income, indicators of their health status, their use of other VA homeless programs, and rurality are consistent predictors of our outcomes. These findings have implications for how to allocate homelessness prevention and RRH resources in the most efficient manner to help households maintain or obtain stable housing.
Asunto(s)
Personas con Mala Vivienda , Veteranos , Humanos , Estados Unidos , Vivienda , Evaluación de Programas y Proyectos de Salud , RentaRESUMEN
The Electronic Health Record (EHR) contains information about social determinants of health (SDoH) such as homelessness. Much of this information is contained in clinical notes and can be extracted using natural language processing (NLP). This data can provide valuable information for researchers and policymakers studying long-term housing outcomes for individuals with a history of homelessness. However, studying homelessness longitudinally in the EHR is challenging due to irregular observation times. In this work, we applied an NLP system to extract housing status for a cohort of patients in the US Department of Veterans Affairs (VA) over a three-year period. We then applied inverse intensity weighting to adjust for the irregularity of observations, which was used generalized estimating equations to estimate the probability of unstable housing each day after entering a VA housing assistance program. Our methods generate unique insights into the long-term outcomes of individuals with a history of homelessness and demonstrate the potential for using EHR data for research and policymaking.
Asunto(s)
Registros Electrónicos de Salud , Personas con Mala Vivienda , Humanos , Procesamiento de Lenguaje Natural , Vivienda , Determinantes Sociales de la SaludRESUMEN
OBJECTIVE: Analyze responses to a national request for information (RFI) to uncover gaps in policy, practice, and understanding of veteran suicide to inform federal research strategy. DATA SOURCE: An RFI with 21 open-ended questions generated from Presidential Executive Order #1386, administered nationally from July 3 to August 5, 2019. STUDY DESIGN: Semi-structured, open-ended responses analyzed using a collaborative qualitative and text-mining data process. DATA EXTRACTION METHODS: We aligned traditional qualitative methods with natural language processing (NLP) text-mining techniques to analyze 9040 open-ended question responses from 722 respondents to provide results within 3 months. Narrative inquiry and the medical explanatory model guided the data extraction and analytic process. RESULTS: Five major themes were identified: risk factors, risk assessment, prevention and intervention, barriers to care, and data/research. Individuals and organizations mentioned different concepts within the same themes. In responses about risk factors, individuals frequently mentioned generic terms like "illness" while organizations mentioned specific terms like "traumatic brain injury." Organizations and individuals described unique barriers to care and emphasized ways to integrate data and research to improve points of care. Organizations often identified lack of funding as barriers while individuals often identified key moments for prevention such as military transitions and ensuring care providers have military cultural understanding. CONCLUSIONS: This study provides an example of a rapid, adaptive analysis of a large body of qualitative, public response data about veteran suicide to support a federal strategy for an important public health topic. Combining qualitative and text-mining methods allowed a representation of voices and perspectives including the lived experiences of individuals who described stories of military transition, treatments that worked or did not, and the perspective of organizations treating veterans for suicide. The results supported the development of a national strategy to reduce suicide risks for veterans as well as civilians.
Asunto(s)
Personal Militar , Prevención del Suicidio , Veteranos , HumanosRESUMEN
Objective: To evaluate the feasibility, accuracy, and interoperability of a natural language processing (NLP) system that extracts diagnostic assertions of pneumonia in different clinical notes and institutions. Materials and Methods: A rule-based NLP system was designed to identify assertions of pneumonia in 3 types of clinical notes from electronic health records (EHRs): emergency department notes, radiology reports, and discharge summaries. The lexicon and classification logic were tailored for each note type. The system was first developed and evaluated using annotated notes from the Department of Veterans Affairs (VA). Interoperability was assessed using data from the University of Utah (UU). Results: The NLP system was comprised of 782 rules and achieved moderate-to-high performance in all 3 note types in VA (precision/recall/f1: emergency = 88.1/86.0/87.1; radiology = 71.4/96.2/82.0; discharge = 88.3/93.0/90.1). When applied to UU data, performance was maintained in emergency and radiology but decreased in discharge summaries (emergency = 84.7/94.3/89.3; radiology = 79.7/100.0/87.9; discharge = 65.5/92.7/76.8). Customization with 34 additional rules increased performance for all note types (emergency = 89.3/94.3/91.7; radiology = 87.0/100.0/93.1; discharge = 75.0/95.1/83.4). Conclusion: NLP can be used to accurately identify the diagnosis of pneumonia across different clinical settings and institutions. A limited amount of customization to account for differences in lexicon, clinical definition of pneumonia, and EHR structure can achieve high accuracy without substantial modification.
RESUMEN
Housing stability is an important determinant of health. The US Department of Veterans Affairs (VA) administers several programs to assist Veterans experiencing unstable housing. Measuring long-term housing stability of Veterans who receive assistance from VA is difficult due to a lack of standardized structured documentation in the Electronic Health Record (EHR). However, the text of clinical notes often contains detailed information about Veterans' housing situations that may be extracted using natural language processing (NLP). We present a novel NLP-based measurement of Veteran housing stability: Relative Housing Stability in Electronic Documentation (ReHouSED). We first develop and evaluate a system for classifying documents containing information about Veterans' housing situations. Next, we aggregate information from multiple documents to derive a patient-level measurement of housing stability. Finally, we demonstrate this method's ability to differentiate between Veterans who are stably and unstably housed. Thus, ReHouSED provides an important methodological framework for the study of long-term housing stability among Veterans receiving housing assistance.
Asunto(s)
Personas con Mala Vivienda , Veteranos , Documentación , Electrónica , Vivienda , Humanos , Procesamiento de Lenguaje Natural , Estados Unidos , United States Department of Veterans AffairsRESUMEN
BACKGROUND: Patient travel history can be crucial in evaluating evolving infectious disease events. Such information can be challenging to acquire in electronic health records, as it is often available only in unstructured text. OBJECTIVE: This study aims to assess the feasibility of annotating and automatically extracting travel history mentions from unstructured clinical documents in the Department of Veterans Affairs across disparate health care facilities and among millions of patients. Information about travel exposure augments existing surveillance applications for increased preparedness in responding quickly to public health threats. METHODS: Clinical documents related to arboviral disease were annotated following selection using a semiautomated bootstrapping process. Using annotated instances as training data, models were developed to extract from unstructured clinical text any mention of affirmed travel locations outside of the continental United States. Automated text processing models were evaluated, involving machine learning and neural language models for extraction accuracy. RESULTS: Among 4584 annotated instances, 2659 (58%) contained an affirmed mention of travel history, while 347 (7.6%) were negated. Interannotator agreement resulted in a document-level Cohen kappa of 0.776. Automated text processing accuracy (F1 85.6, 95% CI 82.5-87.9) and computational burden were acceptable such that the system can provide a rapid screen for public health events. CONCLUSIONS: Automated extraction of patient travel history from clinical documents is feasible for enhanced passive surveillance public health systems. Without such a system, it would usually be necessary to manually review charts to identify recent travel or lack of travel, use an electronic health record that enforces travel history documentation, or ignore this potential source of information altogether. The development of this tool was initially motivated by emergent arboviral diseases. More recently, this system was used in the early phases of response to COVID-19 in the United States, although its utility was limited to a relatively brief window due to the rapid domestic spread of the virus. Such systems may aid future efforts to prevent and contain the spread of infectious diseases.
Asunto(s)
Enfermedades Transmisibles Emergentes/diagnóstico , Registros Electrónicos de Salud , Almacenamiento y Recuperación de la Información/métodos , Vigilancia en Salud Pública/métodos , Viaje/estadística & datos numéricos , Algoritmos , COVID-19/epidemiología , Enfermedades Transmisibles Emergentes/epidemiología , Estudios de Factibilidad , Femenino , Humanos , Aprendizaje Automático , Masculino , Persona de Mediana Edad , Procesamiento de Lenguaje Natural , Reproducibilidad de los Resultados , Estados Unidos/epidemiologíaRESUMEN
Despite impressive success of machine learning algorithms in clinical natural language processing (cNLP), rule-based approaches still have a prominent role. In this paper, we introduce medspaCy, an extensible, open-source cNLP library based on spaCy framework that allows flexible integration of rule-based and machine learning-based algorithms adapted to clinical text. MedspaCy includes a variety of components that meet common cNLP needs such as context analysis and mapping to standard terminologies. By utilizing spaCy's clear and easy-to-use conventions, medspaCy enables development of custom pipelines that integrate easily with other spaCy-based modules. Our toolkit includes several core components and facilitates rapid development of pipelines for clinical text.
Asunto(s)
Algoritmos , Procesamiento de Lenguaje Natural , Humanos , Aprendizaje AutomáticoRESUMEN
INTRODUCTION: Identifying occurrences of medication side effects and adverse drug events (ADEs) is an important and challenging task because they are frequently only mentioned in clinical narrative and are not formally reported. METHODS: We developed a natural language processing (NLP) system that aims to identify mentions of symptoms and drugs in clinical notes and label the relationship between the mentions as indications or ADEs. The system leverages an existing word embeddings model with induced word clusters for dimensionality reduction. It employs a conditional random field (CRF) model for named entity recognition (NER) and a random forest model for relation extraction (RE). RESULTS: Final performance of each model was evaluated separately and then combined on a manually annotated evaluation set. The micro-averaged F1 score was 80.9% for NER, 88.1% for RE, and 61.2% for the integrated systems. Outputs from our systems were submitted to the NLP Challenges for Detecting Medication and Adverse Drug Events from Electronic Health Records (MADE 1.0) competition (Yu et al. in http://bio-nlp.org/index.php/projects/39-nlp-challenges , 2018). System performance was evaluated in three tasks (NER, RE, and complete system) with multiple teams submitting output from their systems for each task. Our RE system placed first in Task 2 of the challenge and our integrated system achieved third place in Task 3. CONCLUSION: Adding to the growing number of publications that utilize NLP to detect occurrences of ADEs, our study illustrates the benefits of employing innovative feature engineering.
Asunto(s)
Sistemas de Registro de Reacción Adversa a Medicamentos/tendencias , Efectos Colaterales y Reacciones Adversas Relacionados con Medicamentos/epidemiología , Registros Electrónicos de Salud/tendencias , Procesamiento de Lenguaje Natural , Sistemas de Registro de Reacción Adversa a Medicamentos/normas , Efectos Colaterales y Reacciones Adversas Relacionados con Medicamentos/diagnóstico , Registros Electrónicos de Salud/normas , HumanosRESUMEN
Free-text reports in electronic health records (EHRs) contain medically significant information - signs, symptoms, findings, diagnoses - recorded by clinicians during patient encounters. These reports contain rich clinical information which can be leveraged for surveillance of disease and occurrence of adverse events. In order to gain meaningful knowledge from these text reports to support surveillance efforts, information must first be converted into a structured, computable format. Traditional methods rely on manual review of charts, which can be costly and inefficient. Natural language processing (NLP) methods offer an efficient, alternative approach to extracting the information and can achieve a similar level of accuracy. We developed an NLP system to automatically identify mentions of surgical site infections in radiology reports and classify reports containing evidence of surgical site infections leveraging these mentions. We evaluated our system using a reference standard of reports annotated by domain experts, administrative data generated for each patient encounter, and a machine learning-based approach.