ABSTRACT
Objectives: Clinical note section identification helps locate relevant information and could be beneficial for downstream tasks such as named entity recognition. However, the traditional supervised methods suffer from transferability issues. This study proposes a new framework for using large language models (LLMs) for section identification to overcome the limitations. Materials and Methods: We framed section identification as question-answering and provided the section definitions in free-text. We evaluated multiple LLMs off-the-shelf without any training. We also fine-tune our LLMs to investigate how the size and the specificity of the fine-tuning dataset impacts model performance. Results: GPT4 achieved the highest F1 score of 0.77. The best open-source model (Tulu2-70b) achieved 0.64 and is on par with GPT3.5 (ChatGPT). GPT4 is also found to obtain F1 scores greater than 0.9 for 9 out of the 27 (33%) section types and greater than 0.8 for 15 out of 27 (56%) section types. For our fine-tuned models, we found they plateaued with an increasing size of the general domain dataset. We also found that adding a reasonable amount of section identification examples is beneficial. Discussion: These results indicate that GPT4 is nearly production-ready for section identification, and seemingly contains both knowledge of note structure and the ability to follow complex instructions, and the best current open-source LLM is catching up. Conclusion: Our study shows that LLMs are promising for generalizable clinical note section identification. They have the potential to be further improved by adding section identification examples to the fine-tuning dataset.
ABSTRACT
BACKGROUND: Mechanical thrombectomy (MT) is the standard of care for acute ischemic stroke (AIS) patients with large vessel occlusion (LVO). The SOFAST study collected clinical evidence on the safety and efficacy of the 6 French SOFIA Flow Plus aspiration catheter (SOFIA 6F) when used as first-line treatment. METHODS: This was a prospective, multicenter investigation to assess the safety and efficacy of SOFIA 6F used for first-line aspiration. Anterior circulation LVO stroke patients were enrolled. The primary endpoint was the final modified Thrombolysis in Cerebral Infarction (mTICI)≥2b rate. Secondary endpoints included first-pass and first-line mTICI≥2b rates, times from arteriotomy to clot contact and mTICI≥2b, and 90-day modified Rankin Scale (mRS)≤2. First-line and final mTICI scores were adjudicated by an independent imaging core lab. Safety events were assessed by an independent clinical events adjudicator. RESULTS: A total of 108 patients were enrolled across 12 centers from July 2020 to June 2022. Median age was 67 years, median National Institutes of Health Stroke Scale (NIHSS) was 15.5, and 56.5% of patients received intravenous thrombolytics. At the end of the procedure, 97.2%, 85.2%, and 55.6% of patients achieved mTICI≥2b, ≥2c, and 3, respectively. With SOFIA 6F first-line aspiration, 87.0%, 79.6%, and 52.8% achieved mTICI≥2b, ≥2c, and 3, respectively. After the first pass, 75.0%, 70.4%, and 50.9% achieved mTICI≥2b, ≥2c, and 3, respectively. Median times from arteriotomy to clot contact and successful revascularization were 12 and 17 min, respectively. At 90 days, 66.7% of patients achieved mRS≤2. CONCLUSIONS: First-line aspiration with SOFIA 6F is safe and effective with high revascularization rates and short procedure times.
ABSTRACT
Short-interfering RNA (siRNA) has gained significant interest for treatment of neurological diseases by providing the capacity to achieve sustained inhibition of nearly any gene target. Yet, achieving efficacious drug delivery throughout deep brain structures of the CNS remains a considerable hurdle. We herein describe a lipid-siRNA conjugate that, following delivery into the cerebrospinal fluid (CSF), is transported effectively through perivascular spaces, enabling broad dispersion within CSF compartments and through the CNS parenchyma. We provide a detailed examination of the temporal kinetics of gene silencing, highlighting potent knockdown for up to five months from a single injection without detectable toxicity. Single-cell RNA sequencing further demonstrates gene silencing activity across diverse cell populations in the parenchyma and at brain borders, which may provide new avenues for neurological disease-modifying therapies.
ABSTRACT
OBJECTIVE: To address challenges in large-scale electronic health record (EHR) data exchange, we sought to develop, deploy, and test an open source, cloud-hosted app "listener" that accesses standardized data across the SMART/HL7 Bulk FHIR Access application programming interface (API). METHODS: We advance a model for scalable, federated, data sharing and learning. Cumulus software is designed to address key technology and policy desiderata including local utility, control, and administrative simplicity as well as privacy preservation during robust data sharing, and artificial intelligence (AI) for processing unstructured text. RESULTS: Cumulus relies on containerized, cloud-hosted software, installed within a healthcare organization's security envelope. Cumulus accesses EHR data via the Bulk FHIR interface and streamlines automated processing and sharing. The modular design enables use of the latest AI and natural language processing tools and supports provider autonomy and administrative simplicity. In an initial test, Cumulus was deployed across 5 healthcare systems each partnered with public health. Cumulus output is patient counts which were aggregated into a table stratifying variables of interest to enable population health studies. All code is available open source. A policy stipulating that only aggregate data leave the institution greatly facilitated data sharing agreements. DISCUSSION AND CONCLUSION: Cumulus addresses barriers to data sharing based on (1) federally required support for standard APIs, (2) increasing use of cloud computing, and (3) advances in AI. There is potential for scalability to support learning across myriad network configurations and use cases.
Subject(s)
Artificial Intelligence , Electronic Health Records , Humans , Software , Cloud Computing , Health Information Interoperability , Information DisseminationABSTRACT
Objectives: This study combines two innovative mouse models in a major gene discovery project to assess the influence of host genetics on asbestos related disease (ARD). Conventional genetics studies provided evidence that some susceptibility to mesothelioma is genetic. However, the identification of host modifier genes, the roles they may play, and whether they contribute to disease susceptibility remain unknown. Here we report a study designed to rapidly identify genes associated with mesothelioma susceptibility by combining the Collaborative Cross (CC) resource with the well-characterised MexTAg mesothelioma mouse model. Methods: The CC is a powerful mouse resource that harnesses over 90% of common genetic variation in the mouse species, allowing rapid identification of genes mediating complex traits. MexTAg mice rapidly, uniformly, and predictably develop mesothelioma, but only after asbestos exposure. To assess the influence of host genetics on ARD, we crossed 72 genetically distinct CC mouse strains with MexTAg mice and exposed the resulting CC-MexTAg (CCMT) progeny to asbestos and monitored them for traits including overall survival, the time to ARD onset (latency), the time between ARD onset and euthanasia (disease progression) and ascites volume. We identified phenotype-specific modifier genes associated with these traits and we validated the role of human orthologues in asbestos-induced carcinogenesis using human mesothelioma datasets. Results: We generated 72 genetically distinct CCMT strains and exposed their progeny (2,562 in total) to asbestos. Reflecting the genetic diversity of the CC, there was considerable variation in overall survival and disease latency. Surprisingly, however, there was no variation in disease progression, demonstrating that host genetic factors do have a significant influence during disease latency but have a limited role once disease is established. Quantitative trait loci (QTL) affecting ARD survival/latency were identified on chromosomes 6, 12 and X. Of the 97-protein coding candidate modifier genes that spanned these QTL, eight genes (CPED1, ORS1, NDUFA1, HS1BP3, IL13RA1, LSM8, TES and TSPAN12) were found to significantly affect outcome in both CCMT and human mesothelioma datasets. Conclusion: Host genetic factors affect susceptibility to development of asbestos associated disease. However, following mesothelioma establishment, genetic variation in molecular or immunological mechanisms did not affect disease progression. Identification of multiple candidate modifier genes and their human homologues with known associations in other advanced stage or metastatic cancers highlights the complexity of ARD and may provide a pathway to identify novel therapeutic targets.
ABSTRACT
Platform trials are generally regarded as an innovative approach to address clinical valuation of early stage candidates, regardless of modality as the evidence evolves. As a type of randomized clinical trial (RCT) design construct in which multiple interventions are evaluated concurrently against a common control group allowing new interventions to be added and the control group to be updated throughout the trial, they provide a dynamic and efficient mechanism to compare and potentially discriminate new treatment candidates. Their recent use in the evaluation of new therapies for COVID-19 has spurred new interest in the approach. The paucity of platform trials is less influenced by the novelty and operational requirements as opposed to concerns regarding the sharing of intellectual property (IP) and the lack of infrastructure to operationalize the conduct in the context of IP and data sharing. We provide a mechanism how this can be accomplished through the use of a digital research environment (DRE) providing a safe and secure platform for clinical researchers, quantitative and physician scientists to analyze and develop tools (e.g., models) on sensitive data with the confidence that the data and models developed are protected. A DRE, in this context, expands on the concept of a trusted research environment (TRE) by providing remote access to data alongside tools for analysis in a securely controlled workspace, while allowing data and tools to be findable, accessible, interoperable, and reusable (FAIR), version-controlled, and dynamically grow in size or quality as a result of each treatment evaluated in the trial.
Subject(s)
COVID-19 , Humans , Information Dissemination/methods , SARS-CoV-2 , Randomized Controlled Trials as Topic/methods , Research Design , Intellectual PropertyABSTRACT
Inflammatory central nervous system (CNS) diseases, such as multiple sclerosis (MS) and myelin oligodendrocyte glycoprotein (MOG) antibody-associated disease (MOGAD), are characterized by humoral immune abnormalities. Anti-MOG antibodies are not specific to MOGAD, with their presence described in MS. Autoantibodies may also be present and play a role in various neurodegenerative diseases. Amyotrophic lateral sclerosis (ALS) is a fatal neurodegenerative disease driven by motor neuron dysfunction. While immune involvement in ALS has been recognized, the presence of antibodies targeting CNS myelin antigens has not been established. We aimed to establish a live cell-based assay for quantification of serum anti-MOG IgG1 in patients with CNS diseases, including MS and ALS. In total, 771 serum samples from the John L. Trotter MS Center and the Northeast ALS Consortium were examined using a live cell-based assay for detection of anti-MOG IgG1. Samples from three cohorts were tested in blinded fashion: healthy control (HC) subjects, patients with clinically diagnosed MOGAD, and an experimental group of ALS and MS patients. All samples from established MOGAD cases were positive for anti-MOG antibodies, while all HC samples were negative. Anti-MOG IgG1 was detected in 65 of 658 samples (9.9%) from MS subjects and 4 of 108 (3.7%) samples from ALS subjects. The presence of serum anti-MOG IgG1 in MS and ALS patients raises questions about the contribution of these antibodies to disease pathophysiology as well as accuracy of diagnostic approaches for CNS inflammatory diseases.
Subject(s)
Amyotrophic Lateral Sclerosis , Autoantibodies , Immunoglobulin G , Myelin-Oligodendrocyte Glycoprotein , Myelin-Oligodendrocyte Glycoprotein/immunology , Humans , Autoantibodies/blood , Female , Male , Middle Aged , Amyotrophic Lateral Sclerosis/blood , Amyotrophic Lateral Sclerosis/immunology , Amyotrophic Lateral Sclerosis/diagnosis , Immunoglobulin G/blood , Neurodegenerative Diseases/immunology , Neurodegenerative Diseases/blood , Neurodegenerative Diseases/diagnosis , Aged , Neuroinflammatory Diseases/immunology , Neuroinflammatory Diseases/blood , Adult , Multiple Sclerosis/immunology , Multiple Sclerosis/blood , AnimalsABSTRACT
BACKGROUND: Real-time surveillance of emerging infectious diseases necessitates a dynamically evolving, computable case definition, which frequently incorporates symptom-related criteria. For symptom detection, both population health monitoring platforms and research initiatives primarily depend on structured data extracted from electronic health records. OBJECTIVE: This study sought to validate and test an artificial intelligence (AI)-based natural language processing (NLP) pipeline for detecting COVID-19 symptoms from physician notes in pediatric patients. We specifically study patients presenting to the emergency department (ED) who can be sentinel cases in an outbreak. METHODS: Subjects in this retrospective cohort study are patients who are 21 years of age and younger, who presented to a pediatric ED at a large academic children's hospital between March 1, 2020, and May 31, 2022. The ED notes for all patients were processed with an NLP pipeline tuned to detect the mention of 11 COVID-19 symptoms based on Centers for Disease Control and Prevention (CDC) criteria. For a gold standard, 3 subject matter experts labeled 226 ED notes and had strong agreement (F1-score=0.986; positive predictive value [PPV]=0.972; and sensitivity=1.0). F1-score, PPV, and sensitivity were used to compare the performance of both NLP and the International Classification of Diseases, 10th Revision (ICD-10) coding to the gold standard chart review. As a formative use case, variations in symptom patterns were measured across SARS-CoV-2 variant eras. RESULTS: There were 85,678 ED encounters during the study period, including 4% (n=3420) with patients with COVID-19. NLP was more accurate at identifying encounters with patients that had any of the COVID-19 symptoms (F1-score=0.796) than ICD-10 codes (F1-score =0.451). NLP accuracy was higher for positive symptoms (sensitivity=0.930) than ICD-10 (sensitivity=0.300). However, ICD-10 accuracy was higher for negative symptoms (specificity=0.994) than NLP (specificity=0.917). Congestion or runny nose showed the highest accuracy difference (NLP: F1-score=0.828 and ICD-10: F1-score=0.042). For encounters with patients with COVID-19, prevalence estimates of each NLP symptom differed across variant eras. Patients with COVID-19 were more likely to have each NLP symptom detected than patients without this disease. Effect sizes (odds ratios) varied across pandemic eras. CONCLUSIONS: This study establishes the value of AI-based NLP as a highly effective tool for real-time COVID-19 symptom detection in pediatric patients, outperforming traditional ICD-10 methods. It also reveals the evolving nature of symptom prevalence across different virus variants, underscoring the need for dynamic, technology-driven approaches in infectious disease surveillance.
Subject(s)
Biosurveillance , COVID-19 , Physicians , SARS-CoV-2 , United States , Humans , Child , Artificial Intelligence , Retrospective Studies , COVID-19/diagnosis , COVID-19/epidemiologyABSTRACT
BACKGROUND: Albumin continues to be used routinely by cardiac anaesthesiologists perioperatively despite lack of evidence for improved outcomes. The Multicenter Perioperative Outcomes Group (MPOG) data ranked our institution as one of the highest intraoperative albumin users during cardiac surgery. Therefore, we designed a quality improvement project (QIP) to introduce a bundle of interventions to reduce intraoperative albumin use in cardiac surgical patients. METHODS: Our institutional MPOG data were used to analyse the FLUID-01-C measure that provides the number of adult cardiac surgery cases where albumin was administered intraoperatively by anaesthesiologists from 1 July 2019 to 30 June 2022. The QIP involved introduction of the following interventions: (1) education about appropriate albumin use and indications (January 2021), (2) email communications reinforced with OR teaching (March 2021), (3) removal of albumin from the standard pharmacy intraoperative medication trays (April 2021), (4) grand rounds presentation discussing the QIP and highlighting the interventions (May 2021) and (5) quarterly provider feedback (starting July 2021). Multivariable segmented regression models were used to assess the changes from preintervention to postintervention time period in albumin utilisation, and its total monthly cost. RESULTS: Among the 5767 cardiac surgery cases that met inclusion criteria over the 3-year study period, 16% of patients received albumin intraoperatively. The total number of cases that passed the metric (albumin administration was avoided), gradually increased as our interventions went into effect. Intraoperative albumin utilisation (beta=-101.1, 95% CI -145 to -56.7) and total monthly cost of albumin (beta=-7678, 95% CI -10712 to -4640) demonstrated significant decrease after starting the interventions. CONCLUSIONS: At a single academic cardiac surgery programme, implementation of a bundle of simple and low-cost interventions as part of a coordinated QIP were effective in significantly decreasing intraoperative use of albumin, which translated into considerable costs savings.
Subject(s)
Albumins , Cardiac Surgical Procedures , Quality Improvement , Humans , Cardiac Surgical Procedures/methods , Cardiac Surgical Procedures/statistics & numerical data , Albumins/therapeutic use , Female , Male , Intraoperative Care/methods , Intraoperative Care/statistics & numerical data , Intraoperative Care/standards , Middle Aged , AgedABSTRACT
As patients continue to live longer from diseases that predispose them to right ventricular (RV) dysfunction or failure, many more patients will require surgery for acute or chronic health issues. Because RV dysfunction results in significant perioperative morbidity if not adequately assessed or managed, understanding appropriate assessment and treatments is important in preventing subsequent morbidity and mortality in the perioperative period. In light of the epidemiology of right heart disease, a working knowledge of right heart anatomy and physiology and an understanding of the implications of right-sided heart function for perioperative care are essential for perioperative practitioners. However, a significant knowledge gap exists concerning this topic. This manuscript is one part of a collection of papers from the PeriOperative Quality Initiative (POQI) IX Conference focusing on "Current Perspectives on the Right Heart in the Perioperative Period." This review aims to provide perioperative clinicians with an essential understanding of right heart physiology by answering five key questions on this topic and providing an explanation of seven fundamental concepts concerning right heart physiology.
ABSTRACT
Objective: The application of Natural Language Processing (NLP) in the clinical domain is important due to the rich unstructured information in clinical documents, which often remains inaccessible in structured data. When applying NLP methods to a certain domain, the role of benchmark datasets is crucial as benchmark datasets not only guide the selection of best-performing models but also enable the assessment of the reliability of the generated outputs. Despite the recent availability of language models (LMs) capable of longer context, benchmark datasets targeting long clinical document classification tasks are absent. Materials and Methods: To address this issue, we propose LCD benchmark, a benchmark for the task of predicting 30-day out-of-hospital mortality using discharge notes of MIMIC-IV and statewide death data. We evaluated this benchmark dataset using baseline models, from bag-of-words and CNN to instruction-tuned large language models. Additionally, we provide a comprehensive analysis of the model outputs, including manual review and visualization of model weights, to offer insights into their predictive capabilities and limitations. Results and Discussion: Baseline models showed 28.9% for best-performing supervised models and 32.2% for GPT-4 in F1-metrics. Notes in our dataset have a median word count of 1687. Our analysis of the model outputs showed that our dataset is challenging for both models and human experts, but the models can find meaningful signals from the text. Conclusion: We expect our LCD benchmark to be a resource for the development of advanced supervised models, or prompting methods, tailored for clinical text. The benchmark dataset is available at https://github.com/Machine-Learning-for-Medical-Language/long-clinical-doc.
ABSTRACT
OBJECTIVE: The timely stratification of trauma injury severity can enhance the quality of trauma care but it requires intense manual annotation from certified trauma coders. The objective of this study is to develop machine learning models for the stratification of trauma injury severity across various body regions using clinical text and structured electronic health records (EHRs) data. MATERIALS AND METHODS: Our study utilized clinical documents and structured EHR variables linked with the trauma registry data to create 2 machine learning models with different approaches to representing text. The first one fuses concept unique identifiers (CUIs) extracted from free text with structured EHR variables, while the second one integrates free text with structured EHR variables. Temporal validation was undertaken to ensure the models' temporal generalizability. Additionally, analyses to assess the variable importance were conducted. RESULTS: Both models demonstrated impressive performance in categorizing leg injuries, achieving high accuracy with macro-F1 scores of over 0.8. Additionally, they showed considerable accuracy, with macro-F1 scores exceeding or near 0.7, in assessing injuries in the areas of the chest and head. We showed in our variable importance analysis that the most important features in the model have strong face validity in determining clinically relevant trauma injuries. DISCUSSION: The CUI-based model achieves comparable performance, if not higher, compared to the free-text-based model, with reduced complexity. Furthermore, integrating structured EHR data improves performance, particularly when the text modalities are insufficiently indicative. CONCLUSIONS: Our multi-modal, multiclass models can provide accurate stratification of trauma injury severity and clinically relevant interpretations.
Subject(s)
Electronic Health Records , Machine Learning , Wounds and Injuries , Humans , Wounds and Injuries/classification , Injury Severity Score , Registries , Trauma Severity Indices , Natural Language ProcessingABSTRACT
BACKGROUND: The optimal recanalization goal and number of endovascular thrombectomy (EVT) passes for elderly patients with large vessel occlusion strokes is unclear. METHODS: Consecutive patients 80 years or older undergoing EVT were identified from 2016 to 2022 at a single center. Clinical information, procedural details, and modified treatment in cerebral ischemia (mTICI) scores were collected. Primary outcome was modified Rankin scale (mRS) at 90 days. Bivariate and multivariable analyses were conducted to assess associations between mTICI scores, EVT passes, and 90-day outcomes. RESULTS: One hundred twenty-six patients were identified. At 90 days, mTICI 2b recanalization resulted in high rates of poor outcomes (8.7% functional independence and 60.9% mortality) not significantly different from mTICI 0, 1 or 2a (median mRS 6 vs. 6, P = 0.61). Complete recanalization (mTICI 2c or 3) led to significantly better mRS outcomes at 90 days compared to mTICI 2b (median mRS 4 vs. 6, adjusted P = 0.038), with 26.8% functional independence and 37.8% mortality. In multivariable analysis, complete recanalization was significantly associated with better 90-day outcomes than mTICI 2b or lower recanalization (odds ratio 4.24 [95% Confidence interval 1.46-12.3]; P = 0.002), while the number of passes was not independently associated with worse outcomes (P = 0.98). CONCLUSIONS: For octogenarians, mTICI 2b recanalization yields limited clinical benefit and results in poor 90-day outcomes. In contrast, complete recanalization is independently associated with significantly better outcomes. Thus, once the decision is made to pursue EVT in the elderly, mTICI 2c or better recanalization should be the angiographic goal. Providers should not withhold thrombectomy passes based on age alone.