Validation of an Electronic Health Record-Based Machine Learning Model Compared to Clinical Risk Scores for Gastrointestinal Bleeding.

BACKGROUND & AIMS: Guidelines recommend use of risk stratification scores for patients presenting with gastrointestinal bleeding (GIB) to identify very-low-risk patients eligible for discharge from emergency departments. Machine learning models may outperform existing scores and can be integrated within the electronic health record (EHR) to provide real-time risk assessment without manual data entry. We present the first EHR-based machine learning model for GIB. METHODS: The training cohort comprised 2,546 patients and internal validation of 850 patients presenting with overt GIB (hematemesis, melena, hematochezia) to emergency departments of 2 hospitals from 2014-2019. External validation was performed on 926 patients presenting to a different hospital with the same EHR from 2014-2019. The primary outcome was a composite of red-blood-cell transfusion, hemostatic intervention (endoscopic, interventional radiologic, or surgical), and 30-day all-cause mortality. We used structured data fields in the EHR available within 4 hours of presentation and compared performance of machine learning models to current guideline-recommended risk scores, Glasgow-Blatchford Score (GBS) and Oakland Score. Primary analysis was area under the receiver-operating-characteristic curve (AUC). Secondary analysis was specificity at 99% sensitivity to assess proportion of patients correctly identified as very-low-risk. RESULTS: The machine learning model outperformed the GBS (AUC=0.92 vs. 0.89;p<0.001) and Oakland score (AUC=0.92 vs. 0.89;p<0.001). At the very-low-risk threshold of 99% sensitivity, the machine learning model identified more very-low-risk patients: 37.9% vs. 18.5% for GBS and 11.7% for Oakland score (p<0.001 for both comparisons). CONCLUSIONS: An EHR-based machine learning model performs better than currently recommended clinical risk scores and identifies more very-low-risk patients eligible for discharge from the emergency department.

Achieving Value by Risk Stratification With Machine Learning Model or Clinical Risk Score in Acute Upper Gastrointestinal Bleeding: A Cost Minimization Analysis.

INTRODUCTION: We estimate the economic impact of applying risk assessment tools to identify very low-risk patients with upper gastrointestinal bleeding who can be safely discharged from the emergency department using a cost minimization analysis. METHODS: We compare triage strategies (Glasgow-Blatchford score = 0/0-1 or validated machine learning model) with usual care using a Markov chain model from a US health care payer perspective. RESULTS: Over 5 years, the Glasgow-Blatchford score triage strategy produced national cumulative savings over usual care of more than $2.7 billion and the machine learning strategy of more than $3.4 billion. DISCUSSION: Implementing risk assessment models for upper gastrointestinal bleeding reduces costs, thereby increasing value.

Optimizing large language models in digestive disease: strategies and challenges to improve clinical outcomes.

Large Language Models (LLMs) are transformer-based neural networks with billions of parameters trained on very large text corpora from diverse sources. LLMs have the potential to improve healthcare due to their capability to parse complex concepts and generate context-based responses. The interest in LLMs has not spared digestive disease academics, who have mainly investigated foundational LLM accuracy, which ranges from 25% to 90% and is influenced by the lack of standardized rules to report methodologies and results for LLM-oriented research. In addition, a critical issue is the absence of a universally accepted definition of accuracy, varying from binary to scalar interpretations, often tied to grader expertise without reference to clinical guidelines. We address strategies and challenges to increase accuracy. In particular, LLMs can be infused with domain knowledge using Retrieval Augmented Generation (RAG) or Supervised Fine-Tuning (SFT) with reinforcement learning from human feedback (RLHF). RAG faces challenges with in-context window limits and accurate information retrieval from the provided context. SFT, a deeper adaptation method, is computationally demanding and requires specialized knowledge. LLMs may increase patient quality of care across the field of digestive diseases, where physicians are often engaged in screening, treatment and surveillance for a broad range of pathologies for which in-context learning or SFT with RLHF could improve clinical decision-making and patient outcomes. However, despite their potential, the safe deployment of LLMs in healthcare still needs to overcome hurdles in accuracy, suggesting a need for strategies that integrate human feedback with advanced model training.

Validation of a Machine Learning Model That Outperforms Clinical Risk Scoring Systems for Upper Gastrointestinal Bleeding.

BACKGROUND & AIMS: Scoring systems are suboptimal for determining risk in patients with upper gastrointestinal bleeding (UGIB); these might be improved by a machine learning model. We used machine learning to develop a model to calculate the risk of hospital-based intervention or death in patients with UGIB and compared its performance with other scoring systems. METHODS: We analyzed data collected from consecutive unselected patients with UGIB from medical centers in 4 countries (the United States, Scotland, England, and Denmark; n = 1958) from March 2014 through March 2015. We used the data to derive and internally validate a gradient-boosting machine learning model to identify patients who met a composite endpoint of hospital-based intervention (transfusion or hemostatic intervention) or death within 30 days. We compared the performance of the machine learning prediction model with validated pre-endoscopic clinical risk scoring systems (the Glasgow-Blatchford score [GBS], admission Rockall score, and AIMS65). We externally validated the machine learning model using data from 2 Asia-Pacific sites (Singapore and New Zealand; n = 399). Performance was measured by area under receiver operating characteristic curve (AUC) analysis. RESULTS: The machine learning model identified patients who met the composite endpoint with an AUC of 0.91 in the internal validation set; the clinical scoring systems identified patients who met the composite endpoint with AUC values of 0.88 for the GBS (P = .001), 0.73 for Rockall score (P < .001), and 0.78 for AIMS65 score (P < .001). In the external validation cohort, the machine learning model identified patients who met the composite endpoint with an AUC of 0.90, the GBS with an AUC of 0.87 (P = .004), the Rockall score with an AUC of 0.66 (P < .001), and the AIMS65 with an AUC of 0.64 (P < .001). At cutoff scores at which the machine learning model and GBS identified patients who met the composite endpoint with 100% sensitivity, the specificity values were 26% with the machine learning model versus 12% with GBS (P < .001). CONCLUSIONS: We developed a machine learning model that identifies patients with UGIB who met a composite endpoint of hospital-based intervention or death within 30 days with a greater AUC and higher levels of specificity, at 100% sensitivity, than validated clinical risk scoring systems. This model could increase identification of low-risk patients who can be safely discharged from the emergency department for outpatient management.

Advancing care for acute gastrointestinal bleeding using artificial intelligence.

The future of gastrointestinal bleeding will include the integration of machine learning algorithms to enhance clinician risk assessment and decision making. Machine learning algorithms have shown promise in outperforming existing clinical risk scores for both upper and lower gastrointestinal bleeding but have not been validated in any prospective clinical trials. The adoption of electronic health records provides an exciting opportunity to deploy risk prediction tools in real time and also to expand the data available to train predictive models. Machine learning algorithms can be used to identify patients with acute gastrointestinal bleeding using data extracted from the electronic health record. This can lead to an automated process to find patients with symptoms of acute gastrointestinal bleeding so that risk prediction tools can be then triggered to consistently provide decision support to the physician. Neural network models can be used to provide continuous risk predictions for patients who are at higher risk, which can be used to guide triage of patients to appropriate levels of care. Finally, the future will likely include neural network-based analysis of endoscopic stigmata of bleeding to help guide best practices for hemostasis during the endoscopic procedure. Machine learning will enhance the delivery of care at every level for patients with acute gastrointestinal bleeding through identifying very low risk patients for outpatient management, triaging high risk patients for higher levels of care, and guiding optimal intervention during endoscopy.

Challenges of developing artificial intelligence-assisted tools for clinical medicine.

Machine learning, a subset of artificial intelligence (AI), is a set of computational tools that can be used to enhance provision of clinical care in all areas of medicine. Gastroenterology and hepatology utilize multiple sources of information, including visual findings on endoscopy, radiologic imaging, manometric testing, genomes, proteomes, and metabolomes. However, clinical care is complex and requires a thoughtful approach to best deploy AI tools to improve quality of care and bring value to patients and providers. On the operational level, AI-assisted clinical management should consider logistic challenges in care delivery, data management, and algorithmic stewardship. There is still much work to be done on a broader societal level in developing ethical, regulatory, and reimbursement frameworks. A multidisciplinary approach and awareness of AI tools will create a vibrant ecosystem for using AI-assisted tools to guide and enhance clinical practice. From optically enhanced endoscopy to clinical decision support for risk stratification, AI tools will potentially transform our practice by leveraging massive amounts of data to personalize care to the right patient, in the right amount, at the right time.

Medical and surgical complications of inflammatory bowel disease in the elderly: a systematic review.

BACKGROUND/AIMS: The complications of therapy, hospitalization, and surgery related to inflammatory bowel disease (IBD) in the elderly are not well described. While multiple reviews have described the management and complications of elderly patients with IBD, none have been performed in a systematic fashion. METHODS: We performed a systematic review using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines to evaluate the association between elderly patients with IBD and complications from therapy, hospitalizations, and surgery. Eligible studies were identified via structured keyword searches in PubMed and manual literature searches. RESULTS: A total of 5,644 publications were identified. Of these, fourteen studies met inclusion criteria, encompassing 963 elderly IBD patients (113 Crohn's disease and 850 ulcerative colitis patients), over 37,000 hospitalizations of elderly IBD patients and over 4,500 controls. Consistent associations were observed between increased age and higher nocturnal stool frequency post-ileal pouch anal anastomosis. Only two studies met inclusion criteria for medication-related complications, one observed an increased mortality and infection risk among elderly patients treated with tumor necrosis factor antagonists and the other observed increased hospital-related complications among elderly patients treated with steroids. CONCLUSIONS: Elderly patients with IBD are at an increased risk of hospital- and therapy-related complications. We found a paucity of high-quality studies evaluating outcomes in elderly patients with IBD. Further studies of elderly patients with IBD are needed to further evaluate the effect of age on medical and surgical complications.

Review article: Upper gastrointestinal bleeding - review of current evidence and implications for management.

BACKGROUND: Acute upper gastrointestinal bleeding (UGIB) is a common emergency requiring hospital-based care. Advances in care across pre-endoscopic, endoscopic and post-endoscopic phases have led to improvements in clinical outcomes. AIMS: To provide a detailed, evidence-based update on major aspects of care across pre-endoscopic, endoscopic and post-endoscopic phases. METHODS: We performed a structured bibliographic database search for each topic. If a recent high-quality meta-analysis was not available, we performed a meta-analysis with random effects methods and odds ratios with 95% confidence intervals. RESULTS: Pre-endoscopic management of UGIB includes risk stratification, a restrictive red blood cell transfusion policy unless the patient has cardiovascular disease, and pharmacologic therapy with erythromycin and a proton pump inhibitor. Patients with cirrhosis should be treated with prophylactic antibiotics and vasoactive medications. Tranexamic acid should not be used. Endoscopic management of UGIB depends on the aetiology. For peptic ulcer disease (PUD) with high-risk stigmata, endoscopic therapy, including over-the-scope clips (OTSCs) and TC-325 powder spray, should be performed. For variceal bleeding, treatment should be customised by severity and anatomic location. Post-endoscopic management includes early enteral feeding for all UGIB patients. For high-risk PUD, PPI should be continued for 72 h, and rebleeding should initially be evaluated with a repeat endoscopy. For variceal bleeding, high-risk patients or those with further bleeding, a transjugular intrahepatic portosystemic shunt can be considered. CONCLUSIONS: Management of acute UGIB should include treatment plans for pre-endoscopic, endoscopic and post-endoscopic phases of care, and customise treatment decisions based on aetiology and severity of bleeding.

Optimization of hepatological clinical guidelines interpretation by large language models: a retrieval augmented generation-based framework.

Large language models (LLMs) can potentially transform healthcare, particularly in providing the right information to the right provider at the right time in the hospital workflow. This study investigates the integration of LLMs into healthcare, specifically focusing on improving clinical decision support systems (CDSSs) through accurate interpretation of medical guidelines for chronic Hepatitis C Virus infection management. Utilizing OpenAI's GPT-4 Turbo model, we developed a customized LLM framework that incorporates retrieval augmented generation (RAG) and prompt engineering. Our framework involved guideline conversion into the best-structured format that can be efficiently processed by LLMs to provide the most accurate output. An ablation study was conducted to evaluate the impact of different formatting and learning strategies on the LLM's answer generation accuracy. The baseline GPT-4 Turbo model's performance was compared against five experimental setups with increasing levels of complexity: inclusion of in-context guidelines, guideline reformatting, and implementation of few-shot learning. Our primary outcome was the qualitative assessment of accuracy based on expert review, while secondary outcomes included the quantitative measurement of similarity of LLM-generated responses to expert-provided answers using text-similarity scores. The results showed a significant improvement in accuracy from 43 to 99% (p < 0.001), when guidelines were provided as context in a coherent corpus of text and non-text sources were converted into text. In addition, few-shot learning did not seem to improve overall accuracy. The study highlights that structured guideline reformatting and advanced prompt engineering (data quality vs. data quantity) can enhance the efficacy of LLM integrations to CDSSs for guideline delivery.

Predicting response to non-selective beta-blockers with liver-spleen stiffness and heart rate in patients with liver cirrhosis and high-risk varices.

INTRODUCTION: Non-selective beta-blockers (NSBB) are used for primary prophylaxis in patients with liver cirrhosis and high-risk varices (HRVs). Assessing therapeutic response is challenging due to the invasive nature of hepatic venous pressure gradient (HVPG) measurement. This study aims to define a noninvasive machine-learning based approach to determine response to NSBB in patients with liver cirrhosis and HRVs. METHODS: We conducted a prospective study on a cohort of cirrhotic patients with documented HRVs receiving NSBB treatment. Patients were followed-up with clinical and elastography appointments at 3, 6, and 12 months after NSBB treatment initiation. NSBB response was defined as stationary or downstaging variceal grading at the 12-month esophagogastroduodenoscopy (EGD). In contrast, non-response was defined as upstaging variceal grading at the 12-month EGD or at least one variceal hemorrhage episode during the 12-month follow-up. We chose cut-off values for univariate and multivariate model with 100% specificity. RESULTS: According to least absolute shrinkage and selection operator (LASSO) regression, spleen stiffness (SS) and liver stiffness (LS) percentual decrease, along with changes in heart rate (HR) at 3 months were the most significant predictors of NSBB response. A decrease > 11.5% in SS, > 16.8% in LS, and > 25.3% in HR was associated with better prediction of clinical response to NSBB. SS percentual decrease showed the highest accuracy (86.4%) with high sensitivity (78.8%) when compared to LS and HR. The multivariate model incorporating SS, LS, and HR showed the highest discrimination and calibration metrics (AUROC = 0.96), with the optimal cut-off of 0.90 (sensitivity 94.2%, specificity 100%, PPV 95.7%, NPV 100%, accuracy 97.5%).

Harnessing the power of synthetic data in healthcare: innovation, application, and privacy.

Data-driven decision-making in modern healthcare underpins innovation and predictive analytics in public health and clinical research. Synthetic data has shown promise in finance and economics to improve risk assessment, portfolio optimization, and algorithmic trading. However, higher stakes, potential liabilities, and healthcare practitioner distrust make clinical use of synthetic data difficult. This paper explores the potential benefits and limitations of synthetic data in the healthcare analytics context. We begin with real-world healthcare applications of synthetic data that informs government policy, enhance data privacy, and augment datasets for predictive analytics. We then preview future applications of synthetic data in the emergent field of digital twin technology. We explore the issues of data quality and data bias in synthetic data, which can limit applicability across different applications in the clinical context, and privacy concerns stemming from data misuse and risk of re-identification. Finally, we evaluate the role of regulatory agencies in promoting transparency and accountability and propose strategies for risk mitigation such as Differential Privacy (DP) and a dataset chain of custody to maintain data integrity, traceability, and accountability. Synthetic data can improve healthcare, but measures to protect patient well-being and maintain ethical standards are key to promote responsible use.

Trends in Upper Gastrointestinal Bleeding in Patients on Primary Prevention Aspirin: A Nationwide Emergency Department Sample Analysis, 2016-2020.

BACKGROUND: Recent guidelines do not recommend routine use of aspirin for primary cardiovascular prevention (ppASA) and suggest avoidance of ppASA in older individuals due to bleeding risk. However, ppASA is frequently taken without an appropriate indication. Estimates of the incidence of upper gastrointestinal bleeding due to ppASA in the United States are lacking. In this study, we provide national estimates of upper gastrointestinal bleeding incidence, characteristics, and costs in ppASA users from 2016-2020. METHODS: Primary cardiovascular prevention users (patients on long-term aspirin therapy without cardiovascular disease) presenting with upper gastrointestinal bleeding were identified in the Nationwide Emergency Department Sample using International Statistical Classification of Diseases and Related Health Problems, 10th revision codes. Trends in upper gastrointestinal bleeding incidence, etiology, severity, associated Medicare reimbursements, and the impact of ppASA on bleeding outcomes were assessed with regression models. RESULTS: From 2016-2020, adjusted incidence of upper gastrointestinal bleeding increased 29.2% among ppASA users, with larger increases for older patients (increase of 41.6% for age 65-74 years and 36.0% for age ≥75 years). The most common etiology among ppASA users was ulcer disease but increases in bleeding incidence due to angiodysplasias were observed. The proportion of hospitalizations with major complications or comorbidities increased 41.5%, and Medicare reimbursements increased 67.6%. Among patients without cardiovascular disease, ppASA was associated with increased odds of hospital admission, red blood cell transfusion, and endoscopic intervention as compared to no ppASA use. CONCLUSIONS: Considering recent guideline recommendations, the rising incidence, severity, and costs associated with upper gastrointestinal bleeding among patients on ppASA highlights the importance of careful assessment for appropriate ppASA use.