Systematic analysis of ChatGPT, Google search and Llama 2 for clinical decision support tasks.

It is likely that individuals are turning to Large Language Models (LLMs) to seek health advice, much like searching for diagnoses on Google. We evaluate clinical accuracy of GPT-3·5 and GPT-4 for suggesting initial diagnosis, examination steps and treatment of 110 medical cases across diverse clinical disciplines. Moreover, two model configurations of the Llama 2 open source LLMs are assessed in a sub-study. For benchmarking the diagnostic task, we conduct a naïve Google search for comparison. Overall, GPT-4 performed best with superior performances over GPT-3·5 considering diagnosis and examination and superior performance over Google for diagnosis. Except for treatment, better performance on frequent vs rare diseases is evident for all three approaches. The sub-study indicates slightly lower performances for Llama models. In conclusion, the commercial LLMs show growing potential for medical question answering in two successive major releases. However, some weaknesses underscore the need for robust and regulated AI models in health care. Open source LLMs can be a viable option to address specific needs regarding data privacy and transparency of training.

Machine Learning in the Parkinson's disease smartwatch (PADS) dataset.

The utilisation of smart devices, such as smartwatches and smartphones, in the field of movement disorders research has gained significant attention. However, the absence of a comprehensive dataset with movement data and clinical annotations, encompassing a wide range of movement disorders including Parkinson's disease (PD) and its differential diagnoses (DD), presents a significant gap. The availability of such a dataset is crucial for the development of reliable machine learning (ML) models on smart devices, enabling the detection of diseases and monitoring of treatment efficacy in a home-based setting. We conducted a three-year cross-sectional study at a large tertiary care hospital. A multi-modal smartphone app integrated electronic questionnaires and smartwatch measures during an interactive assessment designed by neurologists to provoke subtle changes in movement pathologies. We captured over 5000 clinical assessment steps from 504 participants, including PD, DD, and healthy controls (HC). After age-matching, an integrative ML approach combining classical signal processing and advanced deep learning techniques was implemented and cross-validated. The models achieved an average balanced accuracy of 91.16% in the classification PD vs. HC, while PD vs. DD scored 72.42%. The numbers suggest promising performance while distinguishing similar disorders remains challenging. The extensive annotations, including details on demographics, medical history, symptoms, and movement steps, provide a comprehensive database to ML techniques and encourage further investigations into phenotypical biomarkers related to movement disorders.

The Necessity of Multiple Data Sources for ECG-Based Machine Learning Models.

Even though the interest in machine learning studies is growing significantly, especially in medicine, the imbalance between study results and clinical relevance is more pronounced than ever. The reasons for this include data quality and interoperability issues. Hence, we aimed at examining site- and study-specific differences in publicly available standard electrocardiogram (ECG) datasets, which in theory should be interoperable by consistent 12-lead definition, sampling rate, and measurement duration. The focus lies upon the question of whether even slight study peculiarities can affect the stability of trained machine learning models. To this end, the performances of modern network architectures as well as unsupervised pattern detection algorithms are investigated across different datasets. Overall, this is intended to examine the generalization of machine learning results of single-site ECG studies.

Classification of Parkinson's Disease from Voice - Analysis of Data Selection Bias.

A growing number of studies have been researching biomarkers of Parkinson's disease (PD) using mobile technology. Many have shown high accuracy in PD classification using machine learning (ML) and voice records from the mPower study, a large database of PD patients and healthy controls. Since the dataset has unbalanced class, gender and age distribution, it is important to consider appropriate sampling when assessing classification scores. We analyse biases, such as identity confounding and implicit learning of non-disease-specific characteristics and present a sampling strategy to highlight and prevent these problems.

Post Hoc Sample Size Estimation for Deep Learning Architectures for ECG-Classification.

Deep Learning architectures for time series require a large number of training samples, however traditional sample size estimation for sufficient model performance is not applicable for machine learning, especially in the field of electrocardiograms (ECGs). This paper outlines a sample size estimation strategy for binary classification problems on ECGs using different deep learning architectures and the large publicly available PTB-XL dataset, which includes 21801 ECG samples. This work evaluates binary classification tasks for Myocardial Infarction (MI), Conduction Disturbance (CD), ST/T Change (STTC), and Sex. All estimations are benchmarked across different architectures, including XResNet, Inception-, XceptionTime and a fully convolutional network (FCN). The results indicate trends for required sample sizes for given tasks and architectures, which can be used as orientation for future ECG studies or feasibility aspects.

DeepTSE: A Time-Sensitive Deep Embedding of ICU Data for Patient Modeling and Missing Data Imputation.

Missing data is a common problem in the intensive care unit as a variety of factors contribute to incomplete data collection in this clinical setting. This missing data has a significant impact on the accuracy and validity of statistical analyses and prognostic models. Several imputation methods can be used to estimate the missing values based on the available data. Although simple imputations with mean or median generate reasonable results in terms of mean absolute error, they do not account for the currentness of the data. Furthermore, heterogeneous time span of data records adds to this complexity, especially in high-frequency intensive care unit datasets. Therefore, we present DeepTSE, a deep model that is able to cope with both, missing data and heterogeneous time spans. We achieved promising results on the MIMIC-IV dataset that can compete with and even outperform established imputation methods.

Predictors for major in-hospital complications after catheter ablation of ventricular arrhythmias: validation and modification of the Risk in Ventricular Ablation (RIVA) Score.

OBJECTIVE AND BACKGROUND: Catheter-based treatment of patients with ventricular arrhythmias (VA) reduces VA and mortality in selected patients. With regard to potential risks of catheter ablation, a benefit-risk assessment should be carried out. This can be performed with risk scores such as the recently published "Risk in Ventricular Ablation (RIVA) Score". We sought to validate this score and to test for possible additional predictors in a large database of VT ablations. METHODS AND RESULTS: We analyzed 1964 catheter ablations for VA in patients with (1069; 54.4%) and without (893, 45.6%) structural heart disease (SHD) and observed an overall major adverse event rate of 4.0% with an in-hospital mortality of 1.3% with significantly less complications occurring in patients without structural heart disease (6.5% vs. 1.1%; p ≤ 0.01). The RIVA Score demonstrated to be a valid predictive tool for major in-hospital complications (OR 1.18; 95% CI 1.12, 1.25; p ≤ 0.001). NYHA Class ≥ III (OR 2.5; 95% CI 1.5, 4.2; p < 0.001) and age (OR 1.04; 95% CI 1.02, 1.07; p ≤ 0.001) proved to be additional predictive parameters. Hence, a modified RIVA Score (mRIVA) model was analyzed with a subset of established predictors (SHD, eGFR, epicardial puncture) as well as new predictive parameters (age, NYHA Class ≥ III), that achieved a higher predictive value for major complications compared with the model based on all RIVA variables. CONCLUSION: Adding age and functional heart failure status (NYHA class) as simple clinical parameters to the recently published RIVA Score increases the predictive value for ablation-associated complications in a large VT ablations registry.

Detection of Patients with Congenital and Often Concealed Long-QT Syndrome by Novel Deep Learning Models.

INTRODUCTION: The long-QT syndrome (LQTS) is the most common ion channelopathy, typically presenting with a prolonged QT interval and clinical symptoms such as syncope or sudden cardiac death. Patients may present with a concealed phenotype making the diagnosis challenging. Correctly diagnosing at-risk patients is pivotal to starting early preventive treatment. OBJECTIVE: Identification of congenital and often concealed LQTS by utilizing novel deep learning network architectures, which are specifically designed for multichannel time series and therefore particularly suitable for ECG data. DESIGN AND RESULTS: A retrospective artificial intelligence (AI)-based analysis was performed using a 12-lead ECG of genetically confirmed LQTS (n = 124), including 41 patients with a concealed LQTS (33%), and validated against a control cohort (n = 161 of patients) without known LQTS or without QT-prolonging drug treatment but any other cardiovascular disease. The performance of a fully convolutional network (FCN) used in prior studies was compared with a different, novel convolutional neural network model (XceptionTime). We found that the XceptionTime model was able to achieve a higher balanced accuracy score (91.8%) than the associated FCN metric (83.6%), indicating improved prediction possibilities of novel AI architectures. The predictive accuracy prevailed independently of age and QTc parameters. CONCLUSIONS: In this study, the XceptionTime model outperformed the FCN model for LQTS patients with even better results than in prior studies. Even when a patient cohort with cardiovascular comorbidities is used. AI-based ECG analysis is a promising step for correct LQTS patient identification, especially if common diagnostic measures might be misleading.

Utilizing a Non-Motor Symptoms Questionnaire and Machine Learning to Differentiate Movement Disorders.

Parkinson's disease (PD) is a common neurodegenerative disorder that severely impacts quality of life as the condition progresses. Early diagnosis and treatment is important to reduce burden and costs. Here, we evaluate the diagnostic potential of the Non-Motor symptoms (NMS) questionnaire by the International Parkinson and Movement Disorder Society based on patient-completed answers from a large single-center prospective study. In this study data from 489 study participants consisting of a PD group, a healthy control (HC) group and patients with differential diagnosis (DD) have been recorded with a smartphone-based system. Evaluation of the study data has shown a significant difference in NMS between the representative groups. Cross-validation of Machine Learning based classification achieves balanced accuracy scores of 88.7% in PD vs. HC, 72.1% in PD vs. DD and 82.6% when discriminating between all movement disorders (PD + DD) and the HC group. The results indicate potentially high feature importance of a simple self-administered questionnaire that could support early diagnosis.

Supporting AI-Explainability by Analyzing Feature Subsets in a Machine Learning Model.

Machine learning algorithms become increasingly prevalent in the field of medicine, as they offer the ability to recognize patterns in complex medical data. Especially in this sensitive area, the active usage of a mostly black box is a controversial topic. We aim to highlight how an aggregated and systematic feature analysis of such models can be beneficial in the medical context. For this reason, we introduce a grouped version of the permutation importance analysis for evaluating the influence of entire feature subsets in a machine learning model. In this way, expert-defined subgroups can be evaluated in the decision-making process. Based on these results, new hypotheses can be formulated and examined.

Prediction of Acute Kidney Injury in the Intensive Care Unit: Preliminary Findings in a European Open Access Database.

Acute kidney injury (AKI) is a common complication in critically ill patients and is associated with long-term complications and an increased mortality. This work presents preliminary findings from the first freely available European intensive care database released by Amsterdam UMC. A machine learning (ML) model was developed to predict AKI in the intensive care unit 12 hours before the actual event. Main features of the model included medications and hemodynamic parameters. Our models perform with an accuracy of 81.8% on moderate to severe AKI and 79.8% on all AKI patients. Those results can compete with models reported in the literature and introduce an ML model for AKI based on European patient data.

Machine learning in the detection and management of atrial fibrillation.

Machine learning has immense novel but also disruptive potential for medicine. Numerous applications have already been suggested and evaluated concerning cardiovascular diseases. One important aspect is the detection and management of potentially thrombogenic arrhythmias such as atrial fibrillation. While atrial fibrillation is the most common arrhythmia with a lifetime risk of one in three persons and an increased risk of thromboembolic complications such as stroke, many atrial fibrillation episodes are asymptomatic and a first diagnosis is oftentimes only reached after an embolic event. Therefore, screening for atrial fibrillation represents an important part of clinical practice. Novel technologies such as machine learning have the potential to substantially improve patient care and clinical outcomes. Additionally, machine learning applications may aid cardiologists in the management of patients with already diagnosed atrial fibrillation, for example, by identifying patients at a high risk of recurrence after catheter ablation. We summarize the current state of evidence concerning machine learning and, in particular, artificial neural networks in the detection and management of atrial fibrillation and describe possible future areas of development as well as pitfalls. Typical data flow in machine learning applications for atrial fibrillation detection.

PHOTONAI-A Python API for rapid machine learning model development.

PHOTONAI is a high-level Python API designed to simplify and accelerate machine learning model development. It functions as a unifying framework allowing the user to easily access and combine algorithms from different toolboxes into custom algorithm sequences. It is especially designed to support the iterative model development process and automates the repetitive training, hyperparameter optimization and evaluation tasks. Importantly, the workflow ensures unbiased performance estimates while still allowing the user to fully customize the machine learning analysis. PHOTONAI extends existing solutions with a novel pipeline implementation supporting more complex data streams, feature combinations, and algorithm selection. Metrics and results can be conveniently visualized using the PHOTONAI Explorer and predictive models are shareable in a standardized format for further external validation or application. A growing add-on ecosystem allows researchers to offer data modality specific algorithms to the community and enhance machine learning in the areas of the life sciences. Its practical utility is demonstrated on an exemplary medical machine learning problem, achieving a state-of-the-art solution in few lines of code. Source code is publicly available on Github, while examples and documentation can be found at www.photon-ai.com.