Accuracy and consistency of online large language model-based artificial intelligence chat platforms in answering patients' questions about heart failure.

BACKGROUND: Heart failure (HF) is a prevalent condition associated with significant morbidity. Patients may have questions that they feel embarrassed to ask or will face delays awaiting responses from their healthcare providers which may impact their health behavior. We aimed to investigate the potential of large language model (LLM) based artificial intelligence (AI) chat platforms in complementing the delivery of patient-centered care. METHODS: Using online patient forums and physician experience, we created 30 questions related to diagnosis, management and prognosis of HF. The questions were posed to two LLM-based AI chat platforms (OpenAI's ChatGPT-3.5 and Google's Bard). Each set of answers was evaluated by two HF experts, independently and blinded to each other, for accuracy (adequacy of content) and consistency of content. RESULTS: ChatGPT provided mostly appropriate answers (27/30, 90%) and showed a high degree of consistency (93%). Bard provided a similar content in its answers and thus was evaluated only for adequacy (23/30, 77%). The two HF experts' grades were concordant in 83% and 67% of the questions for ChatGPT and Bard, respectively. CONCLUSION: LLM-based AI chat platforms demonstrate potential in improving HF education and empowering patients, however, these platforms currently suffer from issues related to factual errors and difficulty with more contemporary recommendations. This inaccurate information may pose serious and life-threatening implications for patients that should be considered and addressed in future research.

Performance of risk models to predict mortality risk for patients with heart failure: evaluation in an integrated health system.

BACKGROUND: Referral of patients with heart failure (HF) who are at high mortality risk for specialist evaluation is recommended. Yet, most tools for identifying such patients are difficult to implement in electronic health record (EHR) systems. OBJECTIVE: To assess the performance and ease of implementation of Machine learning Assessment of RisK and EaRly mortality in Heart Failure (MARKER-HF), a machine-learning model that uses structured data that is readily available in the EHR, and compare it with two commonly used risk scores: the Seattle Heart Failure Model (SHFM) and Meta-Analysis Global Group in Chronic (MAGGIC) Heart Failure Risk Score. DESIGN: Retrospective, cohort study. PARTICIPANTS: Data from 6764 adults with HF were abstracted from EHRs at a large integrated health system from 1/1/10 to 12/31/19. MAIN MEASURES: One-year survival from time of first cardiology or primary care visit was estimated using MARKER-HF, SHFM, and MAGGIC. Discrimination was measured by the area under the receiver operating curve (AUC). Calibration was assessed graphically. KEY RESULTS: Compared to MARKER-HF, both SHFM and MAGGIC required a considerably larger amount of data engineering and imputation to generate risk score estimates. MARKER-HF, SHFM, and MAGGIC exhibited similar discriminations with AUCs of 0.70 (0.69-0.73), 0.71 (0.69-0.72), and 0.71 (95% CI 0.70-0.73), respectively. All three scores showed good calibration across the full risk spectrum. CONCLUSIONS: These findings suggest that MARKER-HF, which uses readily available clinical and lab measurements in the EHR and required less imputation and data engineering than SHFM and MAGGIC, is an easier tool to identify high-risk patients in ambulatory clinics who could benefit from referral to a HF specialist.

Deep Learning for Cardiovascular Imaging: A Review.

Importance: Artificial intelligence (AI), driven by advances in deep learning (DL), has the potential to reshape the field of cardiovascular imaging (CVI). While DL for CVI is still in its infancy, research is accelerating to aid in the acquisition, processing, and/or interpretation of CVI across various modalities, with several commercial products already in clinical use. It is imperative that cardiovascular imagers are familiar with DL systems, including a basic understanding of how they work, their relative strengths compared with other automated systems, and possible pitfalls in their implementation. The goal of this article is to review the methodology and application of DL to CVI in a simple, digestible fashion toward demystifying this emerging technology. Observations: At its core, DL is simply the application of a series of tunable mathematical operations that translate input data into a desired output. Based on artificial neural networks that are inspired by the human nervous system, there are several types of DL architectures suited to different tasks; convolutional neural networks are particularly adept at extracting valuable information from CVI data. We survey some of the notable applications of DL to tasks across the spectrum of CVI modalities. We also discuss challenges in the development and implementation of DL systems, including avoiding overfitting, preventing systematic bias, improving explainability, and fostering a human-machine partnership. Finally, we conclude with a vision of the future of DL for CVI. Conclusions and Relevance: Deep learning has the potential to meaningfully affect the field of CVI. Rather than a threat, DL could be seen as a partner to cardiovascular imagers in reducing technical burden and improving efficiency and quality of care. High-quality prospective evidence is still needed to demonstrate how the benefits of DL CVI systems may outweigh the risks.

A comparative study of pretrained language models for long clinical text.

OBJECTIVE: Clinical knowledge-enriched transformer models (eg, ClinicalBERT) have state-of-the-art results on clinical natural language processing (NLP) tasks. One of the core limitations of these transformer models is the substantial memory consumption due to their full self-attention mechanism, which leads to the performance degradation in long clinical texts. To overcome this, we propose to leverage long-sequence transformer models (eg, Longformer and BigBird), which extend the maximum input sequence length from 512 to 4096, to enhance the ability to model long-term dependencies in long clinical texts. MATERIALS AND METHODS: Inspired by the success of long-sequence transformer models and the fact that clinical notes are mostly long, we introduce 2 domain-enriched language models, Clinical-Longformer and Clinical-BigBird, which are pretrained on a large-scale clinical corpus. We evaluate both language models using 10 baseline tasks including named entity recognition, question answering, natural language inference, and document classification tasks. RESULTS: The results demonstrate that Clinical-Longformer and Clinical-BigBird consistently and significantly outperform ClinicalBERT and other short-sequence transformers in all 10 downstream tasks and achieve new state-of-the-art results. DISCUSSION: Our pretrained language models provide the bedrock for clinical NLP using long texts. We have made our source code available at https://github.com/luoyuanlab/Clinical-Longformer, and the pretrained models available for public download at: https://huggingface.co/yikuan8/Clinical-Longformer. CONCLUSION: This study demonstrates that clinical knowledge-enriched long-sequence transformers are able to learn long-term dependencies in long clinical text. Our methods can also inspire the development of other domain-enriched long-sequence transformers.

A Simple Score to Identify Increased Risk of Transthyretin Amyloid Cardiomyopathy in Heart Failure With Preserved Ejection Fraction.

Importance: Transthyretin amyloid cardiomyopathy (ATTR-CM) is a form of heart failure (HF) with preserved ejection fraction (HFpEF). Technetium Tc 99m pyrophosphate scintigraphy (PYP) enables ATTR-CM diagnosis. It is unclear which patients with HFpEF have sufficient risk of ATTR-CM to warrant PYP. Objective: To derive and validate a simple ATTR-CM score to predict increased risk of ATTR-CM in patients with HFpEF. Design, Setting, and Participants: Retrospective cohort study of 666 patients with HF (ejection fraction ≥ 40%) and suspected ATTR-CM referred for PYP at Mayo Clinic, Rochester, Minnesota, from May 10, 2013, through August 31, 2020. These data were analyzed September 2020 through December 2020. A logistic regression model predictive of ATTR-CM was derived and converted to a point-based ATTR-CM risk score. The score was further validated in a community ATTR-CM epidemiology study of older patients with HFpEF with increased left ventricular wall thickness ([WT] ≥ 12 mm) and in an external (Northwestern University, Chicago, Illinois) HFpEF cohort referred for PYP. Race was self-reported by the participants. In all cohorts, both case patients and control patients were definitively ascertained by PYP scanning and specialist evaluation. Main Outcomes and Measures: Performance of the derived ATTR-CM score in all cohorts (referral validation, community validation, and external validation) and prevalence of a high-risk ATTR-CM score in 4 multinational HFpEF clinical trials. Results: Participant cohorts included were referral derivation (n = 416; 13 participants [3%] were Black and 380 participants [94%] were White; ATTR-CM prevalence = 45%), referral validation (n = 250; 12 participants [5%]were Black and 228 participants [93%] were White; ATTR-CM prevalence = 48% ), community validation (n = 286; 5 participants [2%] were Black and 275 participants [96%] were White; ATTR-CM prevalence = 6% ), and external validation (n = 66; 23 participants [37%] were Black and 36 participants [58%] were White; ATTR-CM prevalence = 39%). Score variables included age, male sex, hypertension diagnosis, relative WT more than 0.57, posterior WT of 12 mm or more, and ejection fraction less than 60% (score range -1 to 10). Discrimination (area under the receiver operating characteristic curve [AUC] 0.89; 95% CI, 0.86-0.92; P < .001) and calibration (Hosmer-Lemeshow; χ2 = 4.6; P = .46) were strong. Discrimination (AUC ≥ 0.84; P < .001 for all) and calibration (Hosmer-Lemeshow χ2 = 2.8; P = .84; Hosmer-Lemeshow χ2 = 4.4; P = .35; Hosmer-Lemeshow χ2 = 2.5; P = .78 in referral, community, and external validation cohorts, respectively) were maintained in all validation cohorts. Precision-recall curves and predictive value vs prevalence plots indicated clinically useful classification performance for a score of 6 or more (positive predictive value ≥25%) in clinically relevant ATTR-CM prevalence (≥10% of patients with HFpEF) scenarios. In the HFpEF clinical trials, 11% to 35% of male and 0% to 6% of female patients had a high-risk (≥6) ATTR-CM score. Conclusions and Relevance: A simple 6 variable clinical score may be used to guide use of PYP and increase recognition of ATTR-CM among patients with HFpEF in the community. Further validation in larger and more diverse populations is needed.

Hyponatremia Is a Powerful Predictor of Poor Prognosis in Left Ventricular Assist Device Patients.

Serum sodium is an established prognostic marker in heart failure (HF) patients and is associated with an increased risk of morbidity and mortality. We sought to study the prognostic value of serum sodium in left ventricular assist device (LVAD) patients and whether hyponatremia reflects worsening HF or an alternative mechanism. We identified HF patients that underwent LVAD implantation between 2008 and 2019. Hyponatremia was defined as Na ≤134 mEq/L at 3 months after implantation. We assessed for differences in hyponatremia before and after LVAD implantation. We also evaluated the association of hyponatremia with all-cause mortality and recurrent HF hospitalizations. There were 342 eligible LVAD patients with a sodium value at 3 months. Among them, there was a significant improvement in serum sodium after LVAD implantation compared to preoperatively (137.2 vs. 134.7 mEq/L, P < 0.0001). Patients with and without hyponatremia had no significant differences in echocardiographic and hemodynamic measurements. In a multivariate analysis, hyponatremia was associated with a markedly increased risk of all-cause mortality (HR 3.69, 95% CI, 1.93-7.05, P < 0.001) when accounting for age, gender, co-morbidities, use of loop diuretics, and B-type natriuretic peptide levels. Hyponatremia was also significantly associated with recurrent HF hospitalizations (HR 2.11, 95% CI, 1.02-4.37, P = 0.04). Hyponatremia in LVAD patients is associated with significantly higher risk of all-cause mortality and recurrent HF hospitalizations. Hyponatremia may be a marker of ongoing neurohormonal activation that is more sensitive than other lab values, echocardiography parameters, and hemodynamic measurements.

Advances in Machine Learning Approaches to Heart Failure with Preserved Ejection Fraction.

Heart failure with preserved ejection fraction (HFpEF) represents a prototypical cardiovascular condition in which machine learning may improve targeted therapies and mechanistic understanding of pathogenesis. Machine learning, which involves algorithms that learn from data, has the potential to guide precision medicine approaches for complex clinical syndromes such as HFpEF. It is therefore important to understand the potential utility and common pitfalls of machine learning so that it can be applied and interpreted appropriately. Although machine learning holds considerable promise for HFpEF, it is subject to several potential pitfalls, which are important factors to consider when interpreting machine learning studies.

Cases from a busy nuclear cardiology laboratory: Potential pitfalls in the interpretation of cardiac scintigraphy for ATTR cardiac amyloidosis.

Cardiac scintigraphy has emerged as a key diagnostic test for transthyretin cardiac amyloidosis (ATTR-CA). However, there are potential limitations and pitfalls in the interpretation of cardiac scintigraphy for ATTR-CA that are worth noting. We present here a series of three cases which illustrate some of these important principles.

DeepCOVID-XR: An Artificial Intelligence Algorithm to Detect COVID-19 on Chest Radiographs Trained and Tested on a Large U.S. Clinical Data Set.

Background There are characteristic findings of coronavirus disease 2019 (COVID-19) on chest images. An artificial intelligence (AI) algorithm to detect COVID-19 on chest radiographs might be useful for triage or infection control within a hospital setting, but prior reports have been limited by small data sets, poor data quality, or both. Purpose To present DeepCOVID-XR, a deep learning AI algorithm to detect COVID-19 on chest radiographs, that was trained and tested on a large clinical data set. Materials and Methods DeepCOVID-XR is an ensemble of convolutional neural networks developed to detect COVID-19 on frontal chest radiographs, with reverse-transcription polymerase chain reaction test results as the reference standard. The algorithm was trained and validated on 14 788 images (4253 positive for COVID-19) from sites across the Northwestern Memorial Health Care System from February 2020 to April 2020 and was then tested on 2214 images (1192 positive for COVID-19) from a single hold-out institution. Performance of the algorithm was compared with interpretations from five experienced thoracic radiologists on 300 random test images using the McNemar test for sensitivity and specificity and the DeLong test for the area under the receiver operating characteristic curve (AUC). Results A total of 5853 patients (mean age, 58 years ± 19 [standard deviation]; 3101 women) were evaluated across data sets. For the entire test set, accuracy of DeepCOVID-XR was 83%, with an AUC of 0.90. For 300 random test images (134 positive for COVID-19), accuracy of DeepCOVID-XR was 82%, compared with that of individual radiologists (range, 76%-81%) and the consensus of all five radiologists (81%). DeepCOVID-XR had a significantly higher sensitivity (71%) than one radiologist (60%, P < .001) and significantly higher specificity (92%) than two radiologists (75%, P < .001; 84%, P = .009). AUC of DeepCOVID-XR was 0.88 compared with the consensus AUC of 0.85 (P = .13 for comparison). With consensus interpretation as the reference standard, the AUC of DeepCOVID-XR was 0.95 (95% CI: 0.92, 0.98). Conclusion DeepCOVID-XR, an artificial intelligence algorithm, detected coronavirus disease 2019 on chest radiographs with a performance similar to that of experienced thoracic radiologists in consensus. © RSNA, 2020 Supplemental material is available for this article. See also the editorial by van Ginneken in this issue.

Predicting High-Risk Patients and High-Risk Outcomes in Heart Failure.

Identifying patients with heart failure at high risk for poor outcomes is important for patient care, resource allocation, and process improvement. Although numerous risk models exist to predict mortality, hospitalization, and patient-reported health status, they are infrequently used for several reasons, including modest performance, lack of evidence to support routine clinical use, and barriers to implementation. Artificial intelligence has the potential to enhance the performance of risk prediction models, but has its own limitations and remains unproved.

When to tell and test for genetic carrier status: perspectives of adolescents and young adults from fragile X families.

We report here on our findings from adolescent and young adult females (ages 14-25) with a family history of fragile X syndrome regarding their perceptions of the optimal ages for (1) learning fragile X is inherited, (2) learning one could be a carrier for fragile X, and (3) offering carrier testing for fragile X. Three groups were enrolled: those who knew they were carriers or noncarriers and those who knew only they were at-risk to be a carrier. Only 2 of the 53 participants felt that offering carrier testing should be delayed until the age of 18 years. Participants who knew only that they were at-risk to be a carrier provided older optimal ages for offering carrier testing than those who knew their actual carrier status. Participants did not express regret or negative emotions about the timing of the disclosure of genetic risk information regarding their own experiences. Participants' reasoning behind reported ages for informing about genetic risk and offering carrier testing varied depending on what type of information was being disclosed, which carrier status group the participant belonged to, and the preferred age for learning the information. Study findings suggest that decisions regarding the timing to inform about genetic risk and offer testing should be tailored to the individual needs of the child and his/her family.