Optimization of hepatological clinical guidelines interpretation by large language models: a retrieval augmented generation-based framework.

Large language models (LLMs) can potentially transform healthcare, particularly in providing the right information to the right provider at the right time in the hospital workflow. This study investigates the integration of LLMs into healthcare, specifically focusing on improving clinical decision support systems (CDSSs) through accurate interpretation of medical guidelines for chronic Hepatitis C Virus infection management. Utilizing OpenAI's GPT-4 Turbo model, we developed a customized LLM framework that incorporates retrieval augmented generation (RAG) and prompt engineering. Our framework involved guideline conversion into the best-structured format that can be efficiently processed by LLMs to provide the most accurate output. An ablation study was conducted to evaluate the impact of different formatting and learning strategies on the LLM's answer generation accuracy. The baseline GPT-4 Turbo model's performance was compared against five experimental setups with increasing levels of complexity: inclusion of in-context guidelines, guideline reformatting, and implementation of few-shot learning. Our primary outcome was the qualitative assessment of accuracy based on expert review, while secondary outcomes included the quantitative measurement of similarity of LLM-generated responses to expert-provided answers using text-similarity scores. The results showed a significant improvement in accuracy from 43 to 99% (p < 0.001), when guidelines were provided as context in a coherent corpus of text and non-text sources were converted into text. In addition, few-shot learning did not seem to improve overall accuracy. The study highlights that structured guideline reformatting and advanced prompt engineering (data quality vs. data quantity) can enhance the efficacy of LLM integrations to CDSSs for guideline delivery.

Predicting response to non-selective beta-blockers with liver-spleen stiffness and heart rate in patients with liver cirrhosis and high-risk varices.

INTRODUCTION: Non-selective beta-blockers (NSBB) are used for primary prophylaxis in patients with liver cirrhosis and high-risk varices (HRVs). Assessing therapeutic response is challenging due to the invasive nature of hepatic venous pressure gradient (HVPG) measurement. This study aims to define a noninvasive machine-learning based approach to determine response to NSBB in patients with liver cirrhosis and HRVs. METHODS: We conducted a prospective study on a cohort of cirrhotic patients with documented HRVs receiving NSBB treatment. Patients were followed-up with clinical and elastography appointments at 3, 6, and 12 months after NSBB treatment initiation. NSBB response was defined as stationary or downstaging variceal grading at the 12-month esophagogastroduodenoscopy (EGD). In contrast, non-response was defined as upstaging variceal grading at the 12-month EGD or at least one variceal hemorrhage episode during the 12-month follow-up. We chose cut-off values for univariate and multivariate model with 100% specificity. RESULTS: According to least absolute shrinkage and selection operator (LASSO) regression, spleen stiffness (SS) and liver stiffness (LS) percentual decrease, along with changes in heart rate (HR) at 3 months were the most significant predictors of NSBB response. A decrease > 11.5% in SS, > 16.8% in LS, and > 25.3% in HR was associated with better prediction of clinical response to NSBB. SS percentual decrease showed the highest accuracy (86.4%) with high sensitivity (78.8%) when compared to LS and HR. The multivariate model incorporating SS, LS, and HR showed the highest discrimination and calibration metrics (AUROC = 0.96), with the optimal cut-off of 0.90 (sensitivity 94.2%, specificity 100%, PPV 95.7%, NPV 100%, accuracy 97.5%).

Review article: Upper gastrointestinal bleeding - review of current evidence and implications for management.

BACKGROUND: Acute upper gastrointestinal bleeding (UGIB) is a common emergency requiring hospital-based care. Advances in care across pre-endoscopic, endoscopic and post-endoscopic phases have led to improvements in clinical outcomes. AIMS: To provide a detailed, evidence-based update on major aspects of care across pre-endoscopic, endoscopic and post-endoscopic phases. METHODS: We performed a structured bibliographic database search for each topic. If a recent high-quality meta-analysis was not available, we performed a meta-analysis with random effects methods and odds ratios with 95% confidence intervals. RESULTS: Pre-endoscopic management of UGIB includes risk stratification, a restrictive red blood cell transfusion policy unless the patient has cardiovascular disease, and pharmacologic therapy with erythromycin and a proton pump inhibitor. Patients with cirrhosis should be treated with prophylactic antibiotics and vasoactive medications. Tranexamic acid should not be used. Endoscopic management of UGIB depends on the aetiology. For peptic ulcer disease (PUD) with high-risk stigmata, endoscopic therapy, including over-the-scope clips (OTSCs) and TC-325 powder spray, should be performed. For variceal bleeding, treatment should be customised by severity and anatomic location. Post-endoscopic management includes early enteral feeding for all UGIB patients. For high-risk PUD, PPI should be continued for 72 h, and rebleeding should initially be evaluated with a repeat endoscopy. For variceal bleeding, high-risk patients or those with further bleeding, a transjugular intrahepatic portosystemic shunt can be considered. CONCLUSIONS: Management of acute UGIB should include treatment plans for pre-endoscopic, endoscopic and post-endoscopic phases of care, and customise treatment decisions based on aetiology and severity of bleeding.

Achieving Value by Risk Stratification With Machine Learning Model or Clinical Risk Score in Acute Upper Gastrointestinal Bleeding: A Cost Minimization Analysis.

INTRODUCTION: We estimate the economic impact of applying risk assessment tools to identify very low-risk patients with upper gastrointestinal bleeding who can be safely discharged from the emergency department using a cost minimization analysis. METHODS: We compare triage strategies (Glasgow-Blatchford score = 0/0-1 or validated machine learning model) with usual care using a Markov chain model from a US health care payer perspective. RESULTS: Over 5 years, the Glasgow-Blatchford score triage strategy produced national cumulative savings over usual care of more than $2.7 billion and the machine learning strategy of more than $3.4 billion. DISCUSSION: Implementing risk assessment models for upper gastrointestinal bleeding reduces costs, thereby increasing value.

Harnessing the power of synthetic data in healthcare: innovation, application, and privacy.

Data-driven decision-making in modern healthcare underpins innovation and predictive analytics in public health and clinical research. Synthetic data has shown promise in finance and economics to improve risk assessment, portfolio optimization, and algorithmic trading. However, higher stakes, potential liabilities, and healthcare practitioner distrust make clinical use of synthetic data difficult. This paper explores the potential benefits and limitations of synthetic data in the healthcare analytics context. We begin with real-world healthcare applications of synthetic data that informs government policy, enhance data privacy, and augment datasets for predictive analytics. We then preview future applications of synthetic data in the emergent field of digital twin technology. We explore the issues of data quality and data bias in synthetic data, which can limit applicability across different applications in the clinical context, and privacy concerns stemming from data misuse and risk of re-identification. Finally, we evaluate the role of regulatory agencies in promoting transparency and accountability and propose strategies for risk mitigation such as Differential Privacy (DP) and a dataset chain of custody to maintain data integrity, traceability, and accountability. Synthetic data can improve healthcare, but measures to protect patient well-being and maintain ethical standards are key to promote responsible use.

Trends in Upper Gastrointestinal Bleeding in Patients on Primary Prevention Aspirin: A Nationwide Emergency Department Sample Analysis, 2016-2020.

BACKGROUND: Recent guidelines do not recommend routine use of aspirin for primary cardiovascular prevention (ppASA) and suggest avoidance of ppASA in older individuals due to bleeding risk. However, ppASA is frequently taken without an appropriate indication. Estimates of the incidence of upper gastrointestinal bleeding due to ppASA in the United States are lacking. In this study, we provide national estimates of upper gastrointestinal bleeding incidence, characteristics, and costs in ppASA users from 2016-2020. METHODS: Primary cardiovascular prevention users (patients on long-term aspirin therapy without cardiovascular disease) presenting with upper gastrointestinal bleeding were identified in the Nationwide Emergency Department Sample using International Statistical Classification of Diseases and Related Health Problems, 10th revision codes. Trends in upper gastrointestinal bleeding incidence, etiology, severity, associated Medicare reimbursements, and the impact of ppASA on bleeding outcomes were assessed with regression models. RESULTS: From 2016-2020, adjusted incidence of upper gastrointestinal bleeding increased 29.2% among ppASA users, with larger increases for older patients (increase of 41.6% for age 65-74 years and 36.0% for age ≥75 years). The most common etiology among ppASA users was ulcer disease but increases in bleeding incidence due to angiodysplasias were observed. The proportion of hospitalizations with major complications or comorbidities increased 41.5%, and Medicare reimbursements increased 67.6%. Among patients without cardiovascular disease, ppASA was associated with increased odds of hospital admission, red blood cell transfusion, and endoscopic intervention as compared to no ppASA use. CONCLUSIONS: Considering recent guideline recommendations, the rising incidence, severity, and costs associated with upper gastrointestinal bleeding among patients on ppASA highlights the importance of careful assessment for appropriate ppASA use.

Trends in characteristics, management, and outcomes of patients presenting with gastrointestinal bleeding to emergency departments in the United States from 2006 to 2019.

BACKGROUND: Recent epidemiologic studies of trends in gastrointestinal bleeding (GIB) provided results through 2014 or earlier and assessed only hospitalised patients, excluding patients presenting to emergency departments (EDs) who are not hospitalised. AIMS: To provide the first U.S. nationwide epidemiological evaluation of all patients presenting to EDs with GIB METHODS: We used the Nationwide Emergency Department Sample for 2006-2019 to calculate yearly projected incidence of patients presenting to EDs with primary diagnoses of GIB, categorised by location and aetiology. Outcomes were assessed with multivariable analyses. RESULTS: The age/sex-adjusted incidence for GIB increased from 378.4 to 397.5/100,000 population from 2006 to 2019. Upper gastrointestinal bleeding (UGIB) incidence decreased from 2006 to 2014 (112.3-94.4/100,000) before increasing to 116.2/100,000 by 2019. Lower gastrointestinal bleeding (LGIB) incidence increased from 2006 to 2015 (146.0 to 161.0/100,000) before declining to 150.2/100,000 by 2019. The proportion of cases with one or more comorbidities increased from 27.4% to 35.9% from 2006 to 2019. Multivariable analyses comparing 2019 to 2006 showed increases in ED discharges (odds ratio [OR] = 1.45; 95% confidence interval [CI] = 1.43-1.48) and decreases in red blood cell (RBC) transfusions (OR = 0.62; 0.61-0.63), endoscopies (OR = 0.60; 0.59-0.61), death (OR = 0.51; 0.48-0.54) and length of stay (relative ratio [RR] = 0.81; 0.80-0.82). Inpatient cost decreased from 2012 to 2019 (RR = 0.92; 0.91-0.93). CONCLUSIONS: The incidence of GIB in the U.S. is increasing. UGIB incidence has been increasing since 2014 while LGIB incidence has been decreasing since 2015. Despite a more comorbid population in 2019, case fatality rate, length of stay and costs have decreased. More patients are discharged from the ED and the rate of RBC transfusions has decreased, possibly reflecting changing clinical practice in response to updated guidelines.

Randomized Clinical Trials of Machine Learning Interventions in Health Care: A Systematic Review.

Importance: Despite the potential of machine learning to improve multiple aspects of patient care, barriers to clinical adoption remain. Randomized clinical trials (RCTs) are often a prerequisite to large-scale clinical adoption of an intervention, and important questions remain regarding how machine learning interventions are being incorporated into clinical trials in health care. Objective: To systematically examine the design, reporting standards, risk of bias, and inclusivity of RCTs for medical machine learning interventions. Evidence Review: In this systematic review, the Cochrane Library, Google Scholar, Ovid Embase, Ovid MEDLINE, PubMed, Scopus, and Web of Science Core Collection online databases were searched and citation chasing was done to find relevant articles published from the inception of each database to October 15, 2021. Search terms for machine learning, clinical decision-making, and RCTs were used. Exclusion criteria included implementation of a non-RCT design, absence of original data, and evaluation of nonclinical interventions. Data were extracted from published articles. Trial characteristics, including primary intervention, demographics, adherence to the CONSORT-AI reporting guideline, and Cochrane risk of bias were analyzed. Findings: Literature search yielded 19 737 articles, of which 41 RCTs involved a median of 294 participants (range, 17-2488 participants). A total of 16 RCTS (39%) were published in 2021, 21 (51%) were conducted at single sites, and 15 (37%) involved endoscopy. No trials adhered to all CONSORT-AI standards. Common reasons for nonadherence were not assessing poor-quality or unavailable input data (38 trials [93%]), not analyzing performance errors (38 [93%]), and not including a statement regarding code or algorithm availability (37 [90%]). Overall risk of bias was high in 7 trials (17%). Of 11 trials (27%) that reported race and ethnicity data, the median proportion of participants from underrepresented minority groups was 21% (range, 0%-51%). Conclusions and Relevance: This systematic review found that despite the large number of medical machine learning-based algorithms in development, few RCTs for these technologies have been conducted. Among published RCTs, there was high variability in adherence to reporting standards and risk of bias and a lack of participants from underrepresented minority groups. These findings merit attention and should be considered in future RCT design and reporting.

Randomized Controlled Trials of Artificial Intelligence in Clinical Practice: Systematic Review.

BACKGROUND: The number of artificial intelligence (AI) studies in medicine has exponentially increased recently. However, there is no clear quantification of the clinical benefits of implementing AI-assisted tools in patient care. OBJECTIVE: This study aims to systematically review all published randomized controlled trials (RCTs) of AI-assisted tools to characterize their performance in clinical practice. METHODS: CINAHL, Cochrane Central, Embase, MEDLINE, and PubMed were searched to identify relevant RCTs published up to July 2021 and comparing the performance of AI-assisted tools with conventional clinical management without AI assistance. We evaluated the primary end points of each study to determine their clinical relevance. This systematic review was conducted following the updated PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 guidelines. RESULTS: Among the 11,839 articles retrieved, only 39 (0.33%) RCTs were included. These RCTs were conducted in an approximately equal distribution from North America, Europe, and Asia. AI-assisted tools were implemented in 13 different clinical specialties. Most RCTs were published in the field of gastroenterology, with 15 studies on AI-assisted endoscopy. Most RCTs studied biosignal-based AI-assisted tools, and a minority of RCTs studied AI-assisted tools drawn from clinical data. In 77% (30/39) of the RCTs, AI-assisted interventions outperformed usual clinical care, and clinically relevant outcomes improved with AI-assisted intervention in 70% (21/30) of the studies. Small sample size and single-center design limited the generalizability of these studies. CONCLUSIONS: There is growing evidence supporting the implementation of AI-assisted tools in daily clinical practice; however, the number of available RCTs is limited and heterogeneous. More RCTs of AI-assisted tools integrated into clinical practice are needed to advance the role of AI in medicine. TRIAL REGISTRATION: PROSPERO CRD42021286539; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=286539.

Multiscale PHATE identifies multimodal signatures of COVID-19.

As the biomedical community produces datasets that are increasingly complex and high dimensional, there is a need for more sophisticated computational tools to extract biological insights. We present Multiscale PHATE, a method that sweeps through all levels of data granularity to learn abstracted biological features directly predictive of disease outcome. Built on a coarse-graining process called diffusion condensation, Multiscale PHATE learns a data topology that can be analyzed at coarse resolutions for high-level summarizations of data and at fine resolutions for detailed representations of subsets. We apply Multiscale PHATE to a coronavirus disease 2019 (COVID-19) dataset with 54 million cells from 168 hospitalized patients and find that patients who die show CD16hiCD66blo neutrophil and IFN-γ+ granzyme B+ Th17 cell responses. We also show that population groupings from Multiscale PHATE directly fed into a classifier predict disease outcome more accurately than naive featurizations of the data. Multiscale PHATE is broadly generalizable to different data types, including flow cytometry, single-cell RNA sequencing (scRNA-seq), single-cell sequencing assay for transposase-accessible chromatin (scATAC-seq), and clinical variables.

EMBEDDING SIGNALS ON GRAPHS WITH UNBALANCED DIFFUSION EARTH MOVER'S DISTANCE.

In modern relational machine learning it is common to encounter large graphs that arise via interactions or similarities between observations in many domains. Further, in many cases the target entities for analysis are actually signals on such graphs. We propose to compare and organize such datasets of graph signals by using an earth mover's distance (EMD) with a geodesic cost over the underlying graph. Typically, EMD is computed by optimizing over the cost of transporting one probability distribution to another over an underlying metric space. However, this is inefficient when computing the EMD between many signals. Here, we propose an unbalanced graph EMD that efficiently embeds the unbalanced EMD on an underlying graph into an L 1 space, whose metric we call unbalanced diffusion earth mover's distance (UDEMD). Next, we show how this gives distances between graph signals that are robust to noise. Finally, we apply this to organizing patients based on clinical notes, embedding cells modeled as signals on a gene graph, and organizing genes modeled as signals over a large cell graph. In each case, we show that UDEMD-based embeddings find accurate distances that are highly efficient compared to other methods.

Generating hard-to-obtain information from easy-to-obtain information: Applications in drug discovery and clinical inference.

Often when biological entities are measured in multiple ways, there are distinct categories of information: some information is easy-to-obtain information (EI) and can be gathered on virtually every subject of interest, while other information is hard-to-obtain information (HI) and can only be gathered on some. We propose building a model to make probabilistic predictions of HI using EI. Our feature mapping GAN (FMGAN), based on the conditional GAN framework, uses an embedding network to process conditions as part of the conditional GAN training to create manifold structure when it is not readily present in the conditions. We experiment on generating RNA sequencing of cell lines perturbed with a drug conditioned on the drug's chemical structure and generating FACS data from clinical monitoring variables on a cohort of COVID-19 patients, effectively describing their immune response in great detail.

Neural network predicts need for red blood cell transfusion for patients with acute gastrointestinal bleeding admitted to the intensive care unit.

Acute gastrointestinal bleeding is the most common gastrointestinal cause for hospitalization. For high-risk patients requiring intensive care unit stay, predicting transfusion needs during the first 24 h using dynamic risk assessment may improve resuscitation with red blood cell transfusion in admitted patients with severe acute gastrointestinal bleeding. A patient cohort admitted for acute gastrointestinal bleeding (N = 2,524) was identified from the Medical Information Mart for Intensive Care III (MIMIC-III) critical care database and separated into training (N = 2,032) and internal validation (N = 492) sets. The external validation patient cohort was identified from the eICU collaborative database of patients admitted for acute gastrointestinal bleeding presenting to large urban hospitals (N = 1,526). 62 demographic, clinical, and laboratory test features were consolidated into 4-h time intervals over the first 24 h from admission. The outcome measure was the transfusion of red blood cells during each 4-h time interval. A long short-term memory (LSTM) model, a type of Recurrent Neural Network, was compared to a regression-based models on time-updated data. The LSTM model performed better than discrete time regression-based models for both internal validation (AUROC 0.81 vs 0.75 vs 0.75; P < 0.001) and external validation (AUROC 0.65 vs 0.56 vs 0.56; P < 0.001). A LSTM model can be used to predict the need for transfusion of packed red blood cells over the first 24 h from admission to help personalize the care of high-risk patients with acute gastrointestinal bleeding.