Comprehensive Integration of Single-Cell Data.

Single-cell transcriptomics has transformed our ability to characterize cell states, but deep biological understanding requires more than a taxonomic listing of clusters. As new methods arise to measure distinct cellular modalities, a key analytical challenge is to integrate these datasets to better understand cellular identity and function. Here, we develop a strategy to "anchor" diverse datasets together, enabling us to integrate single-cell measurements not only across scRNA-seq technologies, but also across different modalities. After demonstrating improvement over existing methods for integrating scRNA-seq data, we anchor scRNA-seq experiments with scATAC-seq to explore chromatin differences in closely related interneuron subsets and project protein expression measurements onto a bone marrow atlas to characterize lymphocyte populations. Lastly, we harmonize in situ gene expression and scRNA-seq datasets, allowing transcriptome-wide imputation of spatial gene expression patterns. Our work presents a strategy for the assembly of harmonized references and transfer of information across datasets.

Single-cell profiling of transcriptome and histone modifications with EpiDamID.

Recent advances in single-cell sequencing technologies have enabled simultaneous measurement of multiple cellular modalities, but the combined detection of histone post-translational modifications and transcription at single-cell resolution has remained limited. Here, we introduce EpiDamID, an experimental approach to target a diverse set of chromatin types by leveraging the binding specificities of single-chain variable fragment antibodies, engineered chromatin reader domains, and endogenous chromatin-binding proteins. Using these, we render the DamID technology compatible with the genome-wide identification of histone post-translational modifications. Importantly, this includes the possibility to jointly measure chromatin marks and transcription at the single-cell level. We use EpiDamID to profile single-cell Polycomb occupancy in mouse embryoid bodies and provide evidence for hierarchical gene regulatory networks. In addition, we map H3K9me3 in early zebrafish embryogenesis, and detect striking heterochromatic regions specific to notochord. Overall, EpiDamID is a new addition to a vast toolbox to study chromatin states during dynamic cellular processes.

Multi-modal features-based human-herpesvirus protein-protein interaction prediction by using LightGBM.

The identification of human-herpesvirus protein-protein interactions (PPIs) is an essential and important entry point to understand the mechanisms of viral infection, especially in malignant tumor patients with common herpesvirus infection. While natural language processing (NLP)-based embedding techniques have emerged as powerful approaches, the application of multi-modal embedding feature fusion to predict human-herpesvirus PPIs is still limited. Here, we established a multi-modal embedding feature fusion-based LightGBM method to predict human-herpesvirus PPIs. In particular, we applied document and graph embedding approaches to represent sequence, network and function modal features of human and herpesviral proteins. Training our LightGBM models through our compiled non-rigorous and rigorous benchmarking datasets, we obtained significantly better performance compared to individual-modal features. Furthermore, our model outperformed traditional feature encodings-based machine learning methods and state-of-the-art deep learning-based methods using various benchmarking datasets. In a transfer learning step, we show that our model that was trained on human-herpesvirus PPI dataset without cytomegalovirus data can reliably predict human-cytomegalovirus PPIs, indicating that our method can comprehensively capture multi-modal fusion features of protein interactions across various herpesvirus subtypes. The implementation of our method is available at https://github.com/XiaodiYangpku/MultimodalPPI/.

Genetics of reticular pseudodrusen in age-related macular degeneration.

Reticular pseudodrusen (RPD) are subretinal deposits that, when observed with age-related macular degeneration (AMD), form a distinct phenotype, often associated with late-stage disease. To date, RPD genetic risk associations overlap six well-established AMD-risk regions. Determining RPD-specific underlying genetic causes by using adequate imaging methods should improve our understanding of the pathophysiology of RPD.

Compound-SNE: Comparative alignment of t-SNEs for multiple single-cell omics data visualisation.

SUMMARY: One of the first steps in single-cell omics data analysis is visualization, which allows researchers to see how well-separated cell-types are from each other. When visualizing multiple datasets at once, data integration/batch correction methods are used to merge the datasets. While needed for downstream analyses, these methods modify features space (e.g. gene expression)/PCA space in order to mix cell-types between batches as well as possible. This obscures sample-specific features and breaks down local embedding structures that can be seen when a sample is embedded alone. Therefore, in order to improve in visual comparisons between large numbers of samples (e.g., multiple patients, omic modalities, different time points), we introduce Compound-SNE, which performs what we term a soft alignment of samples in embedding space. We show that Compound-SNE is able to align cell-types in embedding space across samples, while preserving local embedding structures from when samples are embedded independently. AVAILABILITY AND IMPLEMENTATION: Python code for Compound-SNE is available for download at https://github.com/HaghverdiLab/Compound-SNE. SUPPLEMENTARY INFORMATION: Available online. Provides algorithmic details and additional tests.

Explaining slow seizure propagation with white matter tractography.

Epileptic seizures recorded with stereoelectroencephalography (SEEG) can take a fraction of a second or several seconds to propagate from one region to another. What explains such propagation patterns? We combine tractography and SEEG to determine the relationship between seizure propagation and the white matter architecture and to describe seizure propagation mechanisms. Patient-specific spatiotemporal seizure propagation maps were combined with tractography from diffusion imaging of matched subjects from the Human Connectome Project. The onset of seizure activity was marked on a channel-by-channel basis by two board-certified neurologists for all channels involved in the seizure. We measured the tract connectivity (number of tracts) between regions-of-interest pairs among the seizure onset zone, regions of seizure spread, and non-involved regions. We also investigated how tract-connected the seizure onset zone is to regions of early seizure spread compared to regions of late spread. Comparisons were made after correcting for differences in distance. Sixty-nine seizures were marked across 26 patients with drug-resistant epilepsy; 11 were seizure free after surgery (Engel IA) and 15 were not (Engel IB-IV). The seizure onset zone was more tract connected to regions of seizure spread than to non-involved regions (p<0.0001); however, regions of seizure spread were not differentially tract-connected to other regions of seizure spread compared to non-involved regions. In seizure free patients only, regions of seizure spread were more tract connected to the seizure onset zone than to other regions of spread (p<0.0001). Over the temporal evolution of a seizure, the seizure onset zone was significantly more tract connected to regions of early spread compared to regions of late spread in seizure free patients only (p<0.0001). By integrating information on structure, we demonstrate that seizure propagation is likely mediated by white matter tracts. The pattern of connectivity between seizure onset zone, regions of spread and non-involved regions demonstrates that the onset zone may be largely responsible for seizures propagating throughout the brain, rather than seizures propagating to intermediate points, from which further propagation takes place. Our findings also suggest that seizure propagation over seconds may be the result of a continuous bombardment of action potentials from the seizure onset zone to regions of spread. In non-seizure free patients, the paucity of tracts from the presumed seizure onset zone to regions of spread suggests that the onset zone was missed. Fully understanding the structure-propagation relationship may eventually provide insight into selecting the correct targets for epilepsy surgery.

HMMF: a hybrid multi-modal fusion framework for predicting drug side effect frequencies.

BACKGROUND: The identification of drug side effects plays a critical role in drug repositioning and drug screening. While clinical experiments yield accurate and reliable information about drug-related side effects, they are costly and time-consuming. Computational models have emerged as a promising alternative to predict the frequency of drug-side effects. However, earlier research has primarily centered on extracting and utilizing representations of drugs, like molecular structure or interaction graphs, often neglecting the inherent biomedical semantics of drugs and side effects. RESULTS: To address the previously mentioned issue, we introduce a hybrid multi-modal fusion framework (HMMF) for predicting drug side effect frequencies. Considering the wealth of biological and chemical semantic information related to drugs and side effects, incorporating multi-modal information offers additional, complementary semantics. HMMF utilizes various encoders to understand molecular structures, biomedical textual representations, and attribute similarities of both drugs and side effects. It then models drug-side effect interactions using both coarse and fine-grained fusion strategies, effectively integrating these multi-modal features. CONCLUSIONS: HMMF exhibits the ability to successfully detect previously unrecognized potential side effects, demonstrating superior performance over existing state-of-the-art methods across various evaluation metrics, including root mean squared error and area under receiver operating characteristic curve, and shows remarkable performance in cold-start scenarios.

Multi-modal chemical information reconstruction from images and texts for exploring the near-drug space.

Identification of new chemical compounds with desired structural diversity and biological properties plays an essential role in drug discovery, yet the construction of such a potential space with elements of 'near-drug' properties is still a challenging task. In this work, we proposed a multimodal chemical information reconstruction system to automatically process, extract and align heterogeneous information from the text descriptions and structural images of chemical patents. Our key innovation lies in a heterogeneous data generator that produces cross-modality training data in the form of text descriptions and Markush structure images, from which a two-branch model with image- and text-processing units can then learn to both recognize heterogeneous chemical entities and simultaneously capture their correspondence. In particular, we have collected chemical structures from ChEMBL database and chemical patents from the European Patent Office and the US Patent and Trademark Office using keywords 'A61P, compound, structure' in the years from 2010 to 2020, and generated heterogeneous chemical information datasets with 210K structural images and 7818 annotated text snippets. Based on the reconstructed results and substituent replacement rules, structural libraries of a huge number of near-drug compounds can be generated automatically. In quantitative evaluations, our model can correctly reconstruct 97% of the molecular images into structured format and achieve an F1-score around 97-98% in the recognition of chemical entities, which demonstrated the effectiveness of our model in automatic information extraction from chemical patents, and hopefully transforming them to a user-friendly, structured molecular database enriching the near-drug space to realize the intelligent retrieval technology of chemical knowledge.

Multi-modal optimization to identify personalized biomarkers for disease prediction of individual patients with cancer.

Finding personalized biomarkers for disease prediction of patients with cancer remains a massive challenge in precision medicine. Most methods focus on one subnetwork or module as a network biomarker; however, this ignores the early warning capabilities of other modules with different configurations of biomarkers (i.e. multi-modal personalized biomarkers). Identifying such modules would not only predict disease but also provide effective therapeutic drug target information for individual patients. To solve this problem, we developed a novel model (denoted multi-modal personalized dynamic network biomarkers (MMPDNB)) based on a multi-modal optimization mechanism and personalized dynamic network biomarker (PDNB) theory, which can provide multiple modules of personalized biomarkers and unveil their multi-modal properties. Using the genomics data of patients with breast or lung cancer from The Cancer Genome Atlas database, we validated the effectiveness of the MMPDNB model. The experimental results showed that compared with other advanced methods, MMPDNB can more effectively predict the critical state with the highest early warning signal score during cancer development. Furthermore, MMPDNB more significantly identified PDNBs containing driver and biomarker genes specific to cancer tissues. More importantly, we validated the biological significance of multi-modal PDNBs, which could provide effective drug targets of individual patients as well as markers for predicting early warning signals of the critical disease state. In conclusion, multi-modal optimization is an effective method to identify PDNBs and offers a new perspective for understanding tumor heterogeneity in cancer precision medicine.

CNN-based multi-modal radiomics analysis of pseudo-CT utilization in MRI-only brain stereotactic radiotherapy: a feasibility study.

BACKGROUND: Pseudo-computed tomography (pCT) quality is a crucial issue in magnetic resonance image (MRI)-only brain stereotactic radiotherapy (SRT), so this study systematically evaluated it from the multi-modal radiomics perspective. METHODS: 34 cases (< 30 cm³) were retrospectively included (2021.9-2022.10). For each case, both CT and MRI scans were performed at simulation, and pCT was generated by a convolutional neural network (CNN) from planning MRI. Conformal arc or volumetric modulated arc technique was used to optimize the dose distribution. The SRT dose was compared between pCT and planning CT with dose volume histogram (DVH) metrics and gamma index. Wilcoxon test and Spearman analysis were used to identify key factors associated with dose deviations. Additionally, original image features were extracted for radiomic analysis. Tumor control probability (TCP) and normal tissue complication probability (NTCP) were employed for efficacy evaluation. RESULTS: There was no significant difference between pCT and planning CT except for radiomics. The mean value of Hounsfield unit of the planning CT was slightly higher than that of pCT. The Gadolinium-based agents in planning MRI could increase DVH metrics deviation slightly. The median local gamma passing rates (1%/1 mm) between planning CTs and pCTs (non-contrast) was 92.6% (range 63.5-99.6%). Also, differences were observed in more than 85% of original radiomic features. The mean absolute deviation in TCP was 0.03%, and the NTCP difference was below 0.02%, except for the normal brain, which had a 0.16% difference. In addition, the number of SRT fractions and lesions, and lesion morphology could influence dose deviation. CONCLUSIONS: This is the first multi-modal radiomics analysis of CNN-based pCT from planning MRI for SRT of small brain lesions, covering dosiomics and radiomics. The findings suggest the potential of pCT in SRT plan design and efficacy prediction, but caution needs to be taken for radiomic analysis.

Gray matter atrophy is constrained by normal structural brain network architecture in depression.

BACKGROUND: There is growing evidence that gray matter atrophy is constrained by normal brain network (or connectome) architecture in neuropsychiatric disorders. However, whether this finding holds true in individuals with depression remains unknown. In this study, we aimed to investigate the association between gray matter atrophy and normal connectome architecture at individual level in depression. METHODS: In this study, 297 patients with depression and 256 healthy controls (HCs) from two independent Chinese dataset were included: a discovery dataset (105 never-treated first-episode patients and matched 130 HCs) and a replication dataset (106 patients and matched 126 HCs). For each patient, individualized regional atrophy was assessed using normative model and brain regions whose structural connectome profiles in HCs most resembled the atrophy patterns were identified as putative epicenters using a backfoward stepwise regression analysis. RESULTS: In general, the structural connectome architecture of the identified disease epicenters significantly explained 44% (±16%) variance of gray matter atrophy. While patients with depression demonstrated tremendous interindividual variations in the number and distribution of disease epicenters, several disease epicenters with higher participation coefficient than randomly selected regions, including the hippocampus, thalamus, and medial frontal gyrus were significantly shared by depression. Other brain regions with strong structural connections to the disease epicenters exhibited greater vulnerability. In addition, the association between connectome and gray matter atrophy uncovered two distinct subgroups with different ages of onset. CONCLUSIONS: These results suggest that gray matter atrophy is constrained by structural brain connectome and elucidate the possible pathological progression in depression.

Implementation Requires Evaluation of Adoption: Lessons From a Multimodal Pain Regimen Order Set.

INTRODUCTION: Multimodal pain regimen (MMPR) protocols are the standard of care per the 2020 Trauma Quality Improvement Program guidelines. MMPR implementation methodology in trauma services has not been reported. The primary objective of this study was to evaluate the adoption of an MMPR order set at a level 1 trauma center and to describe its implementation. We hypothesized that order set utilization would be about 50%, and barriers to adoption would be related to personal biases. METHODS: This was a mixed-methods study at a level 1 trauma center. We retrospectively evaluated MMPR utilization from July 1, 2021 to February 28, 2022. Agile implementation was the method used to implement a clinical decision support tool for the MMPR: a flow chart order set in the electronic medical record. This methodology utilizes short experiment sprints during which data are collected to guide the next iterations. During this process quantitative as well as qualitative data were collected. This included end user testing of the order set and a survey distributed to surgical residents about the order set. Manual thematic network analysis was employed to identify basic and organizing themes from the survey responses. RESULTS: A total of 587 trauma patients were admitted during the study period and 95 patients (16.2%) had MMPR ordered through the order set. The survey response rate was 19% (13/68). We identified ease of use, desire for options, inadequate education, and assumption of personal expertise as the four basic themes from the survey. These basic themes were further analyzed to two organizing themes: heuristics and overconfidence bias. CONCLUSIONS: The MMPR order set was easy to use but had low adoption at our center in the first 8 months of implementation. Agile implementation methodology provided an ideal framework to identify reasons for low adoption and guide the next sprint to address personal biases, improve heuristics, and provide effective education and dissemination. Evaluation of utilization and qualitative analysis are key components to ensuring clinical decision support tool adoption.

Elevated plasma neurofilament light was associated with multi-modal neuroimaging features in Alzheimer's disease signature regions and predicted future tau deposition.

BACKGROUND: Neurofilament Light (NfL) is a biomarker for early neurodegeneration in Alzheimer's disease (AD). This study aims to examine the association between plasma NfL and multi-modal neuroimaging features across the AD spectrum and whether NfL predicts future tau deposition. METHODS: The present study recruited 517 participants comprising Aß negative cognitively normal (CN-) participants (n = 135), Aß positive cognitively normal (CN +) participants (n = 64), individuals with amnestic mild cognitive impairment (aMCI) (n = 212), and those diagnosed with AD dementia (n = 106). All the participants underwent multi-modal neuroimaging examinations. Cross-sectional and longitudinal associations between plasma NfL and multi-modal neuro-imaging features were evaluated using partial correlation analysis and linear mixed effects models. We also used linear regression analysis to investigate the association of baseline plasma NfL with future PET tau load. Mediation analysis was used to explore whether the effect of NfL on cognition was mediated by these imaging biomarkers. RESULTS: The results showed that baseline NfL levels and the rate of change were associated with Aß deposition, brain atrophy, brain connectome, glucose metabolism, and brain perfusion in AD signature regions (P<0.05). In both Aß positive CN and MCI participants, baseline NfL showed a significant predictive value of elevating tau burden in the left medial orbitofrontal cortex and para-hippocampus (ß = 0.336, P = 0.032; ß = 0.313, P = 0.047). Lastly, the multi-modal neuroimaging features mediated the association between plasma NfL and cognitive performance. CONCLUSIONS: The study supports the association between plasma NfL and multi-modal neuroimaging features in AD-vulnerable regions and its predictive value for future tau deposition.

Long-term abacus training gains in children are predicted by medial temporal lobe anatomy and circuitry.

Abacus-based mental calculation (AMC) is a widely used educational tool for enhancing math learning, offering an accessible and cost-effective method for classroom implementation. Despite its universal appeal, the neurocognitive mechanisms that drive the efficacy of AMC training remain poorly understood. Notably, although abacus training relies heavily on the rapid recall of number positions and sequences, the role of memory systems in driving long-term AMC learning remains unknown. Here, we sought to address this gap by investigating the role of the medial temporal lobe (MTL) memory system in predicting long-term AMC training gains in second-grade children, who were longitudinally assessed up to fifth grade. Leveraging multimodal neuroimaging data, we tested the hypothesis that MTL systems, known for their involvement in associative memory, are instrumental in facilitating AMC-induced improvements in math skills. We found that gray matter volume in bilateral MTL, along with functional connectivity between the MTL and frontal and ventral temporal-occipital cortices, significantly predicted learning gains. Intriguingly, greater gray matter volume but weaker connectivity of the posterior parietal cortex predicted better learning outcomes, offering a more nuanced view of brain systems at play in AMC training. Our findings not only underscore the critical role of the MTL memory system in AMC training but also illuminate the neurobiological factors contributing to individual differences in cognitive skill acquisition. A video abstract of this article can be viewed at https://youtu.be/StVooNRc7T8. RESEARCH HIGHLIGHTS: We investigated the role of medial temporal lobe (MTL) memory system in driving children's math learning following abacus-based mental calculation (AMC) training. AMC training improved math skills in elementary school children across their second and fifth grade. MTL structural integrity and functional connectivity with prefrontal and ventral temporal-occipital cortices predicted long-term AMC training-related gains.

Multimodal contrastive representation learning for drug-target binding affinity prediction.

In the biomedical field, the efficacy of most drugs is demonstrated by their interactions with targets, meanwhile, accurate prediction of the strength of drug-target binding is extremely important for drug development efforts. Traditional bioassay-based drug-target binding affinity (DTA) prediction methods cannot meet the needs of drug R&D in the era of big data. Recent years we have witnessed significant success on deep learning-based models for drug-target binding affinity prediction task. However, these models only considered a single modality of drug and target information, and some valuable information was not fully utilized. In fact, the information of different modalities of drug and target can complement each other, and more valuable information can be obtained by fusing the information of different modalities. In this paper, we introduce a multimodal information fusion model for DTA prediction that is called FMDTA, which fully considers drug/target information in both string and graph modalities and balances the feature representations of different modalities by a contrastive learning approach. In addition, we exploited the alignment information of drug atoms and target residues to capture the positional information of string patterns, which can extract more useful feature information in SMILES and target sequences. Experimental results on two benchmark datasets show that FMDTA outperforms the state-of-the-art model, demonstrating the feasibility and excellent feature capture capability of FMDTA. The code of FMDTA and the data are available at: https://github.com/bestdoubleLin/FMDTA.

Cholinergic system correlates of postural control changes in Parkinson's disease freezers.

Postural instability and freezing of gait are the most debilitating dopamine-refractory motor impairments in advanced stages of Parkinson's disease because of increased risk of falls and poorer quality of life. Recent findings suggest an inability to efficaciously utilize vestibular information during static posturography among people with Parkinson's disease who exhibit freezing of gait, with associated changes in cholinergic system integrity as assessed by vesicular acetylcholine transporter PET. There is a lack of adequate understanding of how postural control varies as a function of available sensory information in patients with Parkinson's disease with freezing of gait. The goal of this cross-sectional study was to examine cerebral cholinergic system changes that associate with inter-sensory postural control processing features as assessed by dynamic computerized posturography and acetylcholinesterase PET. Seventy-five participants with Parkinson's disease, 16 of whom exhibited freezing of gait, underwent computerized posturography on the NeuroCom© Equitest sensory organization test platform, striatal dopamine, and acetylcholinesterase PET scanning. Findings demonstrated that patients with Parkinson's disease with freezing of gait have greater difficulty maintaining balance in the absence of reliable proprioceptive cues as compared to those without freezing of gait [ß = 0.28 (0.021, 0.54), P = 0.034], an effect that was independent of disease severity [ß = 0.16 (0.062, 0.26), P < 0.01] and age [ß = 0.092 (-0.005, 0.19), P = 0.062]. Exploratory voxel-based analysis revealed an association between postural control and right hemispheric cholinergic network related to visual-vestibular integration and self-motion perception. High anti-cholinergic burden predicted postural control impairment in a manner dependent on right hemispheric cortical cholinergic integrity [ß = 0.34 (0.065, 0.61), P < 0.01]. Our findings advance the perspective that cortical cholinergic system might play a role in supporting postural control after nigro-striatal dopaminergic losses in Parkinson's disease. Failure of cortex-dependent visual-vestibular integration may impair detection of postural instability in absence of reliable proprioceptive cues. Better understanding of how the cholinergic system plays a role in this process may augur novel treatments and therapeutic interventions to ameliorate debilitating symptoms in patients with advanced Parkinson's disease.

Cerebral physiologic insult burden in acute traumatic neural injury: a Canadian High Resolution-TBI (CAHR-TBI) descriptive analysis.

BACKGROUND: Over the recent decades, continuous multi-modal monitoring of cerebral physiology has gained increasing interest for its potential to help minimize secondary brain injury following moderate-to-severe acute traumatic neural injury (also termed traumatic brain injury; TBI). Despite this heightened interest, there has yet to be a comprehensive evaluation of the effects of derangements in multimodal cerebral physiology on global cerebral physiologic insult burden. In this study, we offer a multi-center descriptive analysis of the associations between deranged cerebral physiology and cerebral physiologic insult burden. METHODS: Using data from the Canadian High-Resolution TBI (CAHR-TBI) Research Collaborative, a total of 369 complete patient datasets were acquired for the purposes of this study. For various cerebral physiologic metrics, patients were trichotomized into low, intermediate, and high cohorts based on mean values. Jonckheere-Terpstra testing was then used to assess for directional relationships between these cerebral physiologic metrics and various measures of cerebral physiologic insult burden. Contour plots were then created to illustrate the impact of preserved vs impaired cerebrovascular reactivity on these relationships. RESULTS: It was found that elevated intracranial pressure (ICP) was associated with more time spent with cerebral perfusion pressure (CPP) < 60 mmHg and more time with impaired cerebrovascular reactivity. Low CPP was associated with more time spent with ICP > 20 or 22 mmHg and more time spent with impaired cerebrovascular reactivity. Elevated cerebrovascular reactivity indices were associated with more time spent with CPP < 60 mmHg as well as ICP > 20 or 22 mmHg. Low brain tissue oxygenation (PbtO2) only demonstrated a significant association with more time spent with CPP < 60 mmHg. Low regional oxygen saturation (rSO2) failed to produce a statistically significant association with any particular measure of cerebral physiologic insult burden. CONCLUSIONS: Mean ICP, CPP and, cerebrovascular reactivity values demonstrate statistically significant associations with global cerebral physiologic insult burden; however, it is uncertain whether measures of oxygen delivery provide any significant insight into such insult burden.

Forecasting acute kidney injury and resource utilization in ICU patients using longitudinal, multimodal models.

BACKGROUND: Advances in artificial intelligence (AI) have realized the potential of revolutionizing healthcare, such as predicting disease progression via longitudinal inspection of Electronic Health Records (EHRs) and lab tests from patients admitted to Intensive Care Units (ICU). Although substantial literature exists addressing broad subjects, including the prediction of mortality, length-of-stay, and readmission, studies focusing on forecasting Acute Kidney Injury (AKI), specifically dialysis anticipation like Continuous Renal Replacement Therapy (CRRT) are scarce. The technicality of how to implement AI remains elusive. OBJECTIVE: This study aims to elucidate the important factors and methods that are required to develop effective predictive models of AKI and CRRT for patients admitted to ICU, using EHRs in the Medical Information Mart for Intensive Care (MIMIC) database. METHODS: We conducted a comprehensive comparative analysis of established predictive models, considering both time-series measurements and clinical notes from MIMIC-IV databases. Subsequently, we proposed a novel multi-modal model which integrates embeddings of top-performing unimodal models, including Long Short-Term Memory (LSTM) and BioMedBERT, and leverages both unstructured clinical notes and structured time series measurements derived from EHRs to enable the early prediction of AKI and CRRT. RESULTS: Our multimodal model achieved a lead time of at least 12 h ahead of clinical manifestation, with an Area Under the Receiver Operating Characteristic Curve (AUROC) of 0.888 for AKI and 0.997 for CRRT, as well as an Area Under the Precision Recall Curve (AUPRC) of 0.727 for AKI and 0.840 for CRRT, respectively, which significantly outperformed the baseline models. Additionally, we performed a SHapley Additive exPlanation (SHAP) analysis using the expected gradients algorithm, which highlighted important, previously underappreciated predictive features for AKI and CRRT. CONCLUSION: Our study revealed the importance and the technicality of applying longitudinal, multimodal modeling to improve early prediction of AKI and CRRT, offering insights for timely interventions. The performance and interpretability of our model indicate its potential for further assessment towards clinical applications, to ultimately optimize AKI management and enhance patient outcomes.

M2AI-CVD: Multi-modal AI approach cardiovascular risk prediction system using fundus images.

Cardiovascular diseases (CVD) represent a significant global health challenge, often remaining undetected until severe cardiac events, such as heart attacks or strokes, occur. In regions like Qatar, research focused on non-invasive CVD identification methods, such as retinal imaging and dual-energy X-ray absorptiometry (DXA), is limited. This study presents a groundbreaking system known as Multi-Modal Artificial Intelligence for Cardiovascular Disease (M2AI-CVD), designed to provide highly accurate predictions of CVD. The M2AI-CVD framework employs a four-fold methodology: First, it rigorously evaluates image quality and processes lower-quality images for further analysis. Subsequently, it uses the Entropy-based Fuzzy C Means (EnFCM) algorithm for precise image segmentation. The Multi-Modal Boltzmann Machine (MMBM) is then employed to extract relevant features from various data modalities, while the Genetic Algorithm (GA) selects the most informative features. Finally, a ZFNet Convolutional Neural Network (ZFNetCNN) classifies images, effectively distinguishing between CVD and Non-CVD cases. The research's culmination, tested across five distinct datasets, yields outstanding results, with an accuracy of 95.89%, sensitivity of 96.89%, and specificity of 98.7%. This multi-modal AI approach offers a promising solution for the accurate and early detection of cardiovascular diseases, significantly improving the prospects of timely intervention and improved patient outcomes in the realm of cardiovascular health.

Directed retreat and navigational mechanisms in trail following Formica obscuripes.

Ant species exhibit behavioural commonalities when solving navigational challenges for successful orientation and to reach goal locations. These behaviours rely on a shared toolbox of navigational strategies that guide individuals under an array of motivational contexts. The mechanisms that support these behaviours, however, are tuned to each species' habitat and ecology with some exhibiting unique navigational behaviours. This leads to clear differences in how ant navigators rely on this shared toolbox to reach goals. Species with hybrid foraging structures, which navigate partially upon a pheromone-marked column, express distinct differences in their toolbox, compared to solitary foragers. Here, we explore the navigational abilities of the Western Thatching ant (Formica obscuripes), a hybrid foraging species whose navigational mechanisms have not been studied. We characterise their reliance on both the visual panorama and a path integrator for orientation, with the pheromone's presence acting as a non-directional reassurance cue, promoting continued orientation based on other strategies. This species also displays backtracking behaviour, which occurs with a combination of unfamiliar terrestrial cues and the absence of the pheromone, thus operating based upon a combination of the individual mechanisms observed in solitarily and socially foraging species. We also characterise a new form of goalless orientation in these ants, an initial retreating behaviour that is modulated by the forager's path integration system. The behaviour directs disturbed inbound foragers back along their outbound path for a short distance before recovering and reorienting back to the nest.