CliqueFluxNet: Unveiling EHR Insights with Stochastic Edge Fluxing and Maximal Clique Utilisation Using Graph Neural Networks.

Electronic Health Records (EHRs) play a crucial role in shaping predictive are models, yet they encounter challenges such as significant data gaps and class imbalances. Traditional Graph Neural Network (GNN) approaches have limitations in fully leveraging neighbourhood data or demanding intensive computational requirements for regularisation. To address this challenge, we introduce CliqueFluxNet, a novel framework that innovatively constructs a patient similarity graph to maximise cliques, thereby highlighting strong inter-patient connections. At the heart of CliqueFluxNet lies its stochastic edge fluxing strategy - a dynamic process involving random edge addition and removal during training. This strategy aims to enhance the model's generalisability and mitigate overfitting. Our empirical analysis, conducted on MIMIC-III and eICU datasets, focuses on the tasks of mortality and readmission prediction. It demonstrates significant progress in representation learning, particularly in scenarios with limited data availability. Qualitative assessments further underscore CliqueFluxNet's effectiveness in extracting meaningful EHR representations, solidifying its potential for advancing GNN applications in healthcare analytics.

Continuous patient state attention model for addressing irregularity in electronic health records.

BACKGROUND: Irregular time series (ITS) are common in healthcare as patient data is recorded in an electronic health record (EHR) system as per clinical guidelines/requirements but not for research and depends on a patient's health status. Due to irregularity, it is challenging to develop machine learning techniques to uncover vast intelligence hidden in EHR big data, without losing performance on downstream patient outcome prediction tasks. METHODS: In this paper, we propose Perceiver, a cross-attention-based transformer variant that is computationally efficient and can handle long sequences of time series in healthcare. We further develop continuous patient state attention models, using Perceiver and transformer to deal with ITS in EHR. The continuous patient state models utilise neural ordinary differential equations to learn patient health dynamics, i.e., patient health trajectory from observed irregular time steps, which enables them to sample patient state at any time. RESULTS: The proposed models' performance on in-hospital mortality prediction task on PhysioNet-2012 challenge and MIMIC-III datasets is examined. Perceiver model either outperforms or performs at par with baselines, and reduces computations by about nine times when compared to the transformer model, with no significant loss of performance. Experiments to examine irregularity in healthcare reveal that continuous patient state models outperform baselines. Moreover, the predictive uncertainty of the model is used to refer extremely uncertain cases to clinicians, which enhances the model's performance. Code is publicly available and verified at https://codeocean.com/capsule/4587224 . CONCLUSIONS: Perceiver presents a computationally efficient potential alternative for processing long sequences of time series in healthcare, and the continuous patient state attention models outperform the traditional and advanced techniques to handle irregularity in the time series. Moreover, the predictive uncertainty of the model helps in the development of transparent and trustworthy systems, which can be utilised as per the availability of clinicians.

Data encoding for healthcare data democratization and information leakage prevention.

The lack of data democratization and information leakage from trained models hinder the development and acceptance of robust deep learning-based healthcare solutions. This paper argues that irreversible data encoding can provide an effective solution to achieve data democratization without violating the privacy constraints imposed on healthcare data and clinical models. An ideal encoding framework transforms the data into a new space where it is imperceptible to a manual or computational inspection. However, encoded data should preserve the semantics of the original data such that deep learning models can be trained effectively. This paper hypothesizes the characteristics of the desired encoding framework and then exploits random projections and random quantum encoding to realize this framework for dense and longitudinal or time-series data. Experimental evaluation highlights that models trained on encoded time-series data effectively uphold the information bottleneck principle and hence, exhibit lesser information leakage from trained models.

Use of Temporally Validated Machine Learning Models To Predict Outcomes of Percutaneous Nephrolithotomy Using Data from the British Association of Urological Surgeons Percutaneous Nephrolithotomy Audit.

BACKGROUND AND OBJECTIVE: Machine learning (ML) is a subset of artificial intelligence that uses data to build algorithms to predict specific outcomes. Few ML studies have examined percutaneous nephrolithotomy (PCNL) outcomes. Our objective was to build, streamline, temporally validate, and use ML models for prediction of PCNL outcomes (intensive care admission, postoperative infection, transfusion, adjuvant treatment, postoperative complications, visceral injury, and stone-free status at follow-up) using a comprehensive national database (British Association of Urological Surgeons PCNL). METHODS: This was an ML study using data from a prospective national database. Extreme gradient boosting (XGB), deep neural network (DNN), and logistic regression (LR) models were built for each outcome of interest using complete cases only, imputed, and oversampled and imputed/oversampled data sets. All validation was performed with complete cases only. Temporal validation was performed with 2019 data only. A second round used a composite of the most important 11 variables in each model to build the final model for inclusion in the shiny application. We report statistics for prognostic accuracy. KEY FINDINGS AND LIMITATIONS: The database contains 12 810 patients. The final variables included were age, Charlson comorbidity index, preoperative haemoglobin, Guy's stone score, stone location, size of outer sheath, preoperative midstream urine result, primary puncture site, preoperative dimercapto-succinic acid scan, stone size, and image guidance (https://endourology.shinyapps.io/PCNL_Demographics/). The areas under the receiver operating characteristic curve was >0.6 in all cases. CONCLUSIONS AND CLINICAL IMPLICATIONS: This is the largest ML study on PCNL outcomes to date. The models are temporally valid and therefore can be implemented in clinical practice for patient-specific risk profiling. Further work will be conducted to externally validate the models. PATIENT SUMMARY: We applied artificial intelligence to data for patients who underwent a keyhole surgery to remove kidney stones and developed a model to predict outcomes for this procedure. Doctors could use this tool to advise patients about their risk of complications and the outcomes they can expect after this surgery.

A scalable federated learning solution for secondary care using low-cost microcomputing: privacy-preserving development and evaluation of a COVID-19 screening test in UK hospitals.

BACKGROUND: Multicentre training could reduce biases in medical artificial intelligence (AI); however, ethical, legal, and technical considerations can constrain the ability of hospitals to share data. Federated learning enables institutions to participate in algorithm development while retaining custody of their data but uptake in hospitals has been limited, possibly as deployment requires specialist software and technical expertise at each site. We previously developed an artificial intelligence-driven screening test for COVID-19 in emergency departments, known as CURIAL-Lab, which uses vital signs and blood tests that are routinely available within 1 h of a patient's arrival. Here we aimed to federate our COVID-19 screening test by developing an easy-to-use embedded system-which we introduce as full-stack federated learning-to train and evaluate machine learning models across four UK hospital groups without centralising patient data. METHODS: We supplied a Raspberry Pi 4 Model B preloaded with our federated learning software pipeline to four National Health Service (NHS) hospital groups in the UK: Oxford University Hospitals NHS Foundation Trust (OUH; through the locally linked research University, University of Oxford), University Hospitals Birmingham NHS Foundation Trust (UHB), Bedfordshire Hospitals NHS Foundation Trust (BH), and Portsmouth Hospitals University NHS Trust (PUH). OUH, PUH, and UHB participated in federated training, training a deep neural network and logistic regressor over 150 rounds to form and calibrate a global model to predict COVID-19 status, using clinical data from patients admitted before the pandemic (COVID-19-negative) and testing positive for COVID-19 during the first wave of the pandemic. We conducted a federated evaluation of the global model for admissions during the second wave of the pandemic at OUH, PUH, and externally at BH. For OUH and PUH, we additionally performed local fine-tuning of the global model using the sites' individual training data, forming a site-tuned model, and evaluated the resultant model for admissions during the second wave of the pandemic. This study included data collected between Dec 1, 2018, and March 1, 2021; the exact date ranges used varied by site. The primary outcome was overall model performance, measured as the area under the receiver operating characteristic curve (AUROC). Removable micro secure digital (microSD) storage was destroyed on study completion. FINDINGS: Clinical data from 130 941 patients (1772 COVID-19-positive), routinely collected across three hospital groups (OUH, PUH, and UHB), were included in federated training. The evaluation step included data from 32 986 patients (3549 COVID-19-positive) attending OUH, PUH, or BH during the second wave of the pandemic. Federated training of a global deep neural network classifier improved upon performance of models trained locally in terms of AUROC by a mean of 27·6% (SD 2·2): AUROC increased from 0·574 (95% CI 0·560-0·589) at OUH and 0·622 (0·608-0·637) at PUH using the locally trained models to 0·872 (0·862-0·882) at OUH and 0·876 (0·865-0·886) at PUH using the federated global model. Performance improvement was smaller for a logistic regression model, with a mean increase in AUROC of 13·9% (0·5%). During federated external evaluation at BH, AUROC for the global deep neural network model was 0·917 (0·893-0·942), with 89·7% sensitivity (83·6-93·6) and 76·6% specificity (73·9-79·1). Site-specific tuning of the global model did not significantly improve performance (change in AUROC <0·01). INTERPRETATION: We developed an embedded system for federated learning, using microcomputing to optimise for ease of deployment. We deployed full-stack federated learning across four UK hospital groups to develop a COVID-19 screening test without centralising patient data. Federation improved model performance, and the resultant global models were generalisable. Full-stack federated learning could enable hospitals to contribute to AI development at low cost and without specialist technical expertise at each site. FUNDING: The Wellcome Trust, University of Oxford Medical and Life Sciences Translational Fund.

Self-Aware SGD: Reliable Incremental Adaptation Framework For Clinical AI Models.

Healthcare is dynamic as demographics, diseases, and therapeutics constantly evolve. This dynamic nature induces inevitable distribution shifts in populations targeted by clinical AI models, often rendering them ineffective. Incremental learning provides an effective method of adapting deployed clinical models to accommodate these contemporary distribution shifts. However, since incremental learning involves modifying a deployed or in-use model, it can be considered unreliable as any adverse modification due to maliciously compromised or incorrectly labelled data can make the model unsuitable for the targeted application. This paper introduces self-aware stochastic gradient descent (SGD), an incremental deep learning algorithm that utilises a contextual bandit-like sanity check to only allow reliable modifications to a model. The contextual bandit analyses incremental gradient updates to isolate and filter unreliable gradients. This behaviour allows self-aware SGD to balance incremental training and integrity of a deployed model. Experimental evaluations on the Oxford University Hospital datasets highlight that self-aware SGD can provide reliable incremental updates for overcoming distribution shifts in challenging conditions induced by label noise.

Prognostic model for atrial fibrillation after cardiac surgery: a UK cohort study.

OBJECTIVE: To develop a validated clinical prognostic model to determine the risk of atrial fibrillation after cardiac surgery as part of the PARADISE project (NIHR131227). METHODS: Prospective cohort study with linked electronic health records from a cohort of 5.6 million people in the United Kingdom Clinical Practice Research Datalink from 1998 to 2016. For model development, we considered a priori candidate predictors including demographics, medical history, medications, and clinical biomarkers. We evaluated associations between covariates and the AF incidence at the end of follow-up using logistic regression with the least absolute shrinkage and selection operator. The model was validated internally with the bootstrap method; subsequent performance was examined by discrimination quantified with the c-statistic and calibration assessed by calibration plots. The study follows TRIPOD guidelines. RESULTS: Between 1998 and 2016, 33,464 patients received cardiac surgery among the 5,601,803 eligible individuals. The final model included 13-predictors at baseline: age, year of index surgery, elevated CHA2DS2-VASc score, congestive heart failure, hypertension, acute coronary syndromes, mitral valve disease, ventricular tachycardia, valve surgery, receiving two combined procedures (e.g., valve replacement + coronary artery bypass grafting), or three combined procedures in the index procedure, statin use, and ethnicity other than white or black (statins and ethnicity were protective). This model had an optimism-corrected C-statistic of 0.68 both for the derivation and validation cohort. Calibration was good. CONCLUSIONS: We developed a model to identify a group of individuals at high risk of AF and adverse outcomes who could benefit from long-term arrhythmia monitoring, risk factor management, rhythm control and/or thromboprophylaxis.

Incremental Trainable Parameter Selection in Deep Neural Networks.

This article explores the utilization of the effective degree-of-freedom (DoF) of a deep learning model to regularize its stochastic gradient descent (SGD)-based training. The effective DoF of a deep learning model is defined only by a subset of its total parameters. This subset is highly responsive or sensitive toward the training loss, and its cardinality can be used to govern the effective DoF of a model during training. To this aim, the incremental trainable parameter selection (ITPS) algorithm is introduced in this article. The proposed ITPS algorithm acts as a wrapper over SGD and incrementally selects the parameters for updation that exhibit the maximum sensitivity toward the training loss. Hence, it gradually increases the DoF of the model during training. In ideal cases, the proposed algorithm arrives at a model configuration (i.e., DoF) optimum for the task at hand. This whole process results in a regularization-like behavior induced by a gradual increment of the DoF. Since the selection and updation of parameters is a function of the training loss, the proposed algorithm can be seen as a task and data-dependent regularization mechanism. This article exhibits the general utility of ITPS by evaluating it on various prominent neural network architectures such as CNNs, transformers, recurrent neural networks (RNNs), and multilayer perceptrons. These models are trained for image classification and healthcare tasks using the publicly available CIFAR-10, SLT-10, and MIMIC-III datasets.

Dynamic Neural Graphs Based Federated Reptile for Semi-Supervised Multi-Tasking in Healthcare Applications.

AI healthcare applications rely on sensitive electronic healthcare records (EHRs) that are scarcely labelled and are often distributed across a network of the symbiont institutions. It is challenging to train the effective machine learning models on such data. In this work, we propose dynamic neural graphs based federated learning framework to address these challenges. The proposed framework extends Reptile, a model agnostic meta-learning (MAML) algorithm, to a federated setting. However, unlike the existing MAML algorithms, this paper proposes a dynamic variant of neural graph learning (NGL) to incorporate unlabelled examples in the supervised training setup. Dynamic NGL computes a meta-learning update by performing supervised learning on a labelled training example while performing metric learning on its labelled or unlabelled neighbourhood. This neighbourhood of a labelled example is established dynamically using local graphs built over the batches of training examples. Each local graph is constructed by comparing the similarity between embedding generated by the current state of the model. The introduction of metric learning on the neighbourhood makes this framework semi-supervised in nature. The experimental results on the publicly available MIMIC-III dataset highlight the effectiveness of the proposed framework for both single and multi-task settings under data decentralisation constraints and limited supervision.

Predicting Glaucoma before Onset Using Deep Learning.

PURPOSE: To assess the accuracy of deep learning models to predict glaucoma development from fundus photographs several years before disease onset. DESIGN: Algorithm development for predicting glaucoma using data from a prospective longitudinal study. PARTICIPANTS: A total of 66 721 fundus photographs from 3272 eyes of 1636 subjects who participated in the Ocular Hypertension Treatment Study (OHTS) were included. MAIN OUTCOME MEASURES: Accuracy and area under the curve (AUC). METHODS: Fundus photographs and visual fields were carefully examined by 2 independent readers from the optic disc and visual field reading centers of the OHTS. When an abnormality was detected by the readers, the subject was recalled for retesting to confirm the abnormality and for further confirmation by an end point committee. By using 66 721 fundus photographs, deep learning models were trained and validated using 85% of the fundus photographs and further retested (validated) on the remaining (held-out) 15% of the fundus photographs. RESULTS: The AUC of the deep learning model in predicting glaucoma development 4 to 7 years before disease onset was 0.77 (95% confidence interval [CI], 0.75-0.79). The accuracy of the model in predicting glaucoma development approximately 1 to 3 years before disease onset was 0.88 (95% CI, 0.86-0.91). The accuracy of the model in detecting glaucoma after onset was 0.95 (95% CI, 0.94-0.96). CONCLUSIONS: Deep learning models can predict glaucoma development before disease onset with reasonable accuracy. Eyes with visual field abnormality but not glaucomatous optic neuropathy had a higher tendency to be missed by deep learning algorithms.

Convex Representations Using Deep Archetypal Analysis for Predicting Glaucoma.

Goal: The purpose of this study was to identify clinically relevant patterns of glaucomatous vision loss through convex representation to predict glaucoma several years prior to disease onset. Methods: We developed a deep archetypal analysis to identify patterns of glaucomatous vision loss, and then projected visual fields over the identified patterns. Projections provided a representation that was more accurate in detecting glaucomatous vision loss, thus, more appropriate for recognizing preclinical signs of glaucoma prior to disease development. To overcome the class imbalance in prediction, we implemented a class-balanced bagging with neural networks. Results: Using original visual field as features of the class-balanced bagging classification provided an area under the receiver-operating characteristic curve (AUC) of 0.55 for predicting glaucoma approximately four years prior to disease development. Using convex representation of the visual fields as input features provided an AUC of 0.61 while using deep convex representation as input features improved the AUC to 0.71. Relevance vector machine (RVM) achieved an AUC of 0.64. Conclusion: Deep archetypal analysis representation of visual functional features with balanced bagging classification could serve as an automated tool for predicting glaucoma. Significance: Glaucoma is the second leading cause of worldwide blindness. Most people with glaucoma have no early symptoms or pain, delaying diagnosis in many patients until they reach late irreversible vision loss stages. In fact, about 50% of people with glaucoma are unaware they have the disease. Deep archetypal analysis models may impact clinical practice in effectively identifying at-risk glaucoma patients well prior to disease development.

Deep metric learning for bioacoustic classification: Overcoming training data scarcity using dynamic triplet loss.

Bioacoustic classification often suffers from the lack of labeled data. This hinders the effective utilization of state-of-the-art deep learning models in bioacoustics. To overcome this problem, the authors propose a deep metric learning-based framework that provides effective classification, even when only a small number of per-class training examples are available. The proposed framework utilizes a multiscale convolutional neural network and the proposed dynamic variant of the triplet loss to learn a transformation space where intra-class separation is minimized and inter-class separation is maximized by a dynamically increasing margin. The process of learning this transformation is known as deep metric learning. The triplet loss analyzes three examples (referred to as a triplet) at a time to perform deep metric learning. The number of possible triplets increases cubically with the dataset size, making triplet loss more suitable than the cross-entropy loss in data-scarce conditions. Experiments on three different publicly available datasets show that the proposed framework performs better than existing bioacoustic classification methods. Experimental results also demonstrate the superiority of dynamic triplet loss over cross-entropy loss in data-scarce conditions. Furthermore, unlike existing bioacoustic classification methods, the proposed framework has been extended to provide open-set classification.

Local compressed convex spectral embedding for bird species identification.

This paper proposes a multi-layer alternating sparse-dense framework for bird species identification. The framework takes audio recordings of bird vocalizations and produces compressed convex spectral embeddings (CCSE). Temporal and frequency modulations in bird vocalizations are ensnared by concatenating frames of the spectrogram, resulting in a high dimensional and highly sparse super-frame-based representation. Random projections are then used to compress these super-frames. Class-specific archetypal analysis is employed on the compressed super-frames for acoustic modeling, obtaining the convex-sparse CCSE representation. This representation efficiently captures species-specific discriminative information. However, many bird species exhibit high intra-species variations in their vocalizations, making it hard to appropriately model the whole repertoire of vocalizations using only one dictionary of archetypes. To overcome this, each class is clustered using Gaussian mixture models (GMM), and for each cluster, one dictionary of archetypes is learned. To calculate CCSE for any compressed super-frame, one dictionary from each class is chosen using the responsibilities of individual GMM components. The CCSE obtained using this GMM-archetypal analysis framework is referred to as local CCSE. Experimental results corroborate that local CCSE either outperforms or exhibits comparable performances to existing methods including support vector machine powered by dynamic kernels and deep neural networks.