The evolving landscape of salivary gland tumors.

Salivary gland cancers are a rare, histologically diverse group of tumors. They range from indolent to aggressive and can cause significant morbidity and mortality. Surgical resection remains the mainstay of treatment, but radiation and systemic therapy are also critical parts of the care paradigm. Given the rarity and heterogeneity of these cancers, they are best managed in a multidisciplinary program. In this review, the authors highlight standards of care as well as exciting new research for salivary gland cancers that will strive for better patient outcomes.

Complementary multi-modality molecular self-supervised learning via non-overlapping masking for property prediction.

Self-supervised learning plays an important role in molecular representation learning because labeled molecular data are usually limited in many tasks, such as chemical property prediction and virtual screening. However, most existing molecular pre-training methods focus on one modality of molecular data, and the complementary information of two important modalities, SMILES and graph, is not fully explored. In this study, we propose an effective multi-modality self-supervised learning framework for molecular SMILES and graph. Specifically, SMILES data and graph data are first tokenized so that they can be processed by a unified Transformer-based backbone network, which is trained by a masked reconstruction strategy. In addition, we introduce a specialized non-overlapping masking strategy to encourage fine-grained interaction between these two modalities. Experimental results show that our framework achieves state-of-the-art performance in a series of molecular property prediction tasks, and a detailed ablation study demonstrates efficacy of the multi-modality framework and the masking strategy.

SynergyX: a multi-modality mutual attention network for interpretable drug synergy prediction.

Discovering effective anti-tumor drug combinations is crucial for advancing cancer therapy. Taking full account of intricate biological interactions is highly important in accurately predicting drug synergy. However, the extremely limited prior knowledge poses great challenges in developing current computational methods. To address this, we introduce SynergyX, a multi-modality mutual attention network to improve anti-tumor drug synergy prediction. It dynamically captures cross-modal interactions, allowing for the modeling of complex biological networks and drug interactions. A convolution-augmented attention structure is adopted to integrate multi-omic data in this framework effectively. Compared with other state-of-the-art models, SynergyX demonstrates superior predictive accuracy in both the General Test and Blind Test and cross-dataset validation. By exhaustively screening combinations of approved drugs, SynergyX reveals its ability to identify promising drug combination candidates for potential lung cancer treatment. Another notable advantage lies in its multidimensional interpretability. Taking Sorafenib and Vorinostat as an example, SynergyX serves as a powerful tool for uncovering drug-gene interactions and deciphering cell selectivity mechanisms. In summary, SynergyX provides an illuminating and interpretable framework, poised to catalyze the expedition of drug synergy discovery and deepen our comprehension of rational combination therapy.

RNA trafficking and subcellular localization-a review of mechanisms, experimental and predictive methodologies.

RNA localization is essential for regulating spatial translation, where RNAs are trafficked to their target locations via various biological mechanisms. In this review, we discuss RNA localization in the context of molecular mechanisms, experimental techniques and machine learning-based prediction tools. Three main types of molecular mechanisms that control the localization of RNA to distinct cellular compartments are reviewed, including directed transport, protection from mRNA degradation, as well as diffusion and local entrapment. Advances in experimental methods, both image and sequence based, provide substantial data resources, which allow for the design of powerful machine learning models to predict RNA localizations. We review the publicly available predictive tools to serve as a guide for users and inspire developers to build more effective prediction models. Finally, we provide an overview of multimodal learning, which may provide a new avenue for the prediction of RNA localization.

Adaptive risk-aware sharable and individual subspace learning for cancer survival analysis with multi-modality data.

Biomedical multi-modality data (also named multi-omics data) refer to data that span different types and derive from multiple sources in clinical practices (e.g. gene sequences, proteomics and histopathological images), which can provide comprehensive perspectives for cancers and generally improve the performance of survival models. However, the performance improvement of multi-modality survival models may be hindered by two key issues as follows: (1) how to learn and fuse modality-sharable and modality-individual representations from multi-modality data; (2) how to explore the potential risk-aware characteristics in each risk subgroup, which is beneficial to risk stratification and prognosis evaluation. Additionally, learning-based survival models generally refer to numerous hyper-parameters, which requires time-consuming parameter setting and might result in a suboptimal solution. In this paper, we propose an adaptive risk-aware sharable and individual subspace learning method for cancer survival analysis. The proposed method jointly learns sharable and individual subspaces from multi-modality data, whereas two auxiliary terms (i.e. intra-modality complementarity and inter-modality incoherence) are developed to preserve the complementary and distinctive properties of each modality. Moreover, it equips with a grouping co-expression constraint for obtaining risk-aware representation and preserving local consistency. Furthermore, an adaptive-weighted strategy is employed to efficiently estimate crucial parameters during the training stage. Experimental results on three public datasets demonstrate the superiority of our proposed model.

Identifying spatial domain by adapting transcriptomics with histology through contrastive learning.

Recent advances in spatial transcriptomics have enabled measurements of gene expression at cell/spot resolution meanwhile retaining both the spatial information and the histology images of the tissues. Accurately identifying the spatial domains of spots is a vital step for various downstream tasks in spatial transcriptomics analysis. To remove noises in gene expression, several methods have been developed to combine histopathological images for data analysis of spatial transcriptomics. However, these methods either use the image only for the spatial relations for spots, or individually learn the embeddings of the gene expression and image without fully coupling the information. Here, we propose a novel method ConGI to accurately exploit spatial domains by adapting gene expression with histopathological images through contrastive learning. Specifically, we designed three contrastive loss functions within and between two modalities (the gene expression and image data) to learn the common representations. The learned representations are then used to cluster the spatial domains on both tumor and normal spatial transcriptomics datasets. ConGI was shown to outperform existing methods for the spatial domain identification. In addition, the learned representations have also been shown powerful for various downstream tasks, including trajectory inference, clustering, and visualization.

Multi-modality attribute learning-based method for drug-protein interaction prediction based on deep neural network.

Identification of active candidate compounds for target proteins, also called drug-protein interaction (DPI) prediction, is an essential but time-consuming and expensive step, which leads to fostering the development of drug discovery. In recent years, deep network-based learning methods were frequently proposed in DPIs due to their powerful capability of feature representation. However, the performance of existing DPI methods is still limited by insufficiently labeled pharmacological data and neglected intermolecular information. Therefore, overcoming these difficulties to perfect the performance of DPIs is an urgent challenge for researchers. In this article, we designed an innovative 'multi-modality attributes' learning-based framework for DPIs with molecular transformer and graph convolutional networks, termed, multi-modality attributes (MMA)-DPI. Specifically, intermolecular sub-structural information and chemical semantic representations were extracted through an augmented transformer module from biomedical data. A tri-layer graph convolutional neural network module was applied to associate the neighbor topology information and learn the condensed dimensional features by aggregating a heterogeneous network that contains multiple biological representations of drugs, proteins, diseases and side effects. Then, the learned representations were taken as the input of a fully connected neural network module to further integrate them in molecular and topological space. Finally, the attribute representations were fused with adaptive learning weights to calculate the interaction score for the DPIs tasks. MMA-DPI was evaluated in different experimental conditions and the results demonstrate that the proposed method achieved higher performance than existing state-of-the-art frameworks.

Orthogonal multimodality integration and clustering in single-cell data.

Multimodal integration combines information from different sources or modalities to gain a more comprehensive understanding of a phenomenon. The challenges in multi-omics data analysis lie in the complexity, high dimensionality, and heterogeneity of the data, which demands sophisticated computational tools and visualization methods for proper interpretation and visualization of multi-omics data. In this paper, we propose a novel method, termed Orthogonal Multimodality Integration and Clustering (OMIC), for analyzing CITE-seq. Our approach enables researchers to integrate multiple sources of information while accounting for the dependence among them. We demonstrate the effectiveness of our approach using CITE-seq data sets for cell clustering. Our results show that our approach outperforms existing methods in terms of accuracy, computational efficiency, and interpretability. We conclude that our proposed OMIC method provides a powerful tool for multimodal data analysis that greatly improves the feasibility and reliability of integrated data.

Ocular surface information seen from the somatosensory thalamus and cortex.

Ocular Surface (OS) somatosensory innervation detects external stimuli producing perceptions, such as pain or dryness, the most relevant symptoms in many OS pathologies. Nevertheless, little is known about the central nervous system circuits involved in these perceptions, and how they integrate multimodal inputs in general. Here, we aim to describe the thalamic and cortical activity in response to OS stimulation of different modalities. Electrophysiological extracellular recordings in anaesthetized rats were used to record neural activity, while saline drops at different temperatures were applied to stimulate the OS. Neurons were recorded in the ophthalmic branch of the trigeminal ganglion (TG, 49 units), the thalamic VPM-POm nuclei representing the face (Th, 69 units) and the primary somatosensory cortex (S1, 101 units). The precise locations for Th and S1 neurons receiving OS information are reported here for the first time. Interestingly, all recorded nuclei encode modality both at the single neuron and population levels, with noxious stimulation producing a qualitatively different activity profile from other modalities. Moreover, neurons responding to new combinations of stimulus modalities not present in the peripheral TG subsequently appear in Th and S1, being organized in space through the formation of clusters. Besides, neurons that present higher multimodality display higher spontaneous activity. These results constitute the first anatomical and functional characterization of the thalamocortical representation of the OS. Furthermore, they provide insight into how information from different modalities gets integrated from the peripheral nervous system into the complex cortical networks of the brain. KEY POINTS: Anatomical location of thalamic and cortical ocular surface representation. Thalamic and cortical neuronal responses to multimodal stimulation of the ocular surface. Increasing functional complexity along trigeminal neuroaxis. Proposal of a new perspective on how peripheral activity shapes central nervous system function.

Maternal Outcomes Among Pregnant Women With Congenital Heart Disease-Associated Pulmonary Hypertension.

BACKGROUND: Studies focused on pregnant women with congenital heart disease (CHD)-associated pulmonary hypertension (PH) are scarce and limited by small sample sizes and single-center design. This study sought to describe the pregnancy outcomes in women with CHD with and without PH. METHODS: Outcomes for pregnant women with CHD were evaluated retrospectively from 1993 to 2016 and prospectively from 2017 to 2019 from 7 tertiary hospitals. PH was diagnosed on the basis of echocardiogram or catheterization. The incidence of maternal death, cardiac complications, and obstetric and offspring complications was compared for women with CHD and no PH, mild, and moderate-to-severe PH. RESULTS: A total of 2220 pregnant women with CHD had completed pregnancies. PH associated with CHD was identified in 729 women, including 398 with mild PH (right ventricle to right atrium gradient 30-50 mm Hg) and 331 with moderate-to-severe PH (right ventricle to right atrium gradient >50 mm Hg). Maternal mortality occurred in 1 (0.1%), 0, and 19 (5.7%) women with CHD and no, mild, or moderate-to-severe PH, respectively. Of the 729 patients with PH, 619 (85%) had CHD-associated pulmonary arterial hypertension, and 110 (15%) had other forms of PH. Overall, patients with mild PH had better maternal outcomes than those with moderate-to-severe PH, including the incidence of maternal mortality or heart failure (7.8% versus 39.6%; P<0.001), other cardiac complications (9.0% versus 32.3%; P<0.001), and obstetric complications (5.3% versus 15.7%; P<0.001). Brain natriuretic peptide >100 ng/L (odds ratio, 1.9 [95% CI, 1.0-3.4], P=0.04) and New York Heart Association class III to IV (odds ratio, 2.9 [95% CI, 1.6-5.3], P<0.001) were independently associated with adverse maternal cardiac events in pregnancy with PH, whereas follow-up with a multidisciplinary team (odds ratio, 0.4 [95% CI, 0.2-0.6], P<0.001) and strict antenatal supervision (odds ratio, 0.5 [95% CI, 0.3-0.7], P=0.001) were protective. CONCLUSIONS: Women with CHD-associated mild PH appear to have better outcomes compared with women with CHD-associated moderate-to-severe PH, and with event rates similar for most outcomes with women with CHD and no PH. Multimodality risk assessment, including PH severity, brain natriuretic peptide level, and New York Heart Association class, may be useful in risk stratification in pregnancy with PH. Follow-up with a multidisciplinary team and strict antenatal supervision during pregnancy may also help to mitigate the risk of adverse maternal cardiac events.

Survivorship program including long-term toxicities and quality-of-life development over 10 years in a randomized trial in operable stage III non-small-cell lung cancer (ESPATUE).

Over 40% stage-III non-small-cell lung cancer (NSCLC) patients (pts) experience 5-year survival following multimodality treatment. Nevertheless, little is known about relevant late toxicities and quality-of-life (QoL) in the further long-term follow-up. Therefore, we invited pts from our randomized phase-III trial (Eberhardt et al., Journal of Clinical Oncology 2015) after 10 years from diagnosis to participate within a structured survivorship program (SSP) including follow-up imaging, laboratory parameters, cardio-pulmonary investigations, long-term toxicity evaluations and QoL questionnaires. Of 246 pts initially accrued, 161 were considered potentially resectable following the induction therapy and were randomized (80 to arm A: definitive chemoradiation; 81 to arm B: definitive surgery; 85 not randomized for different reasons; group C). 31 from 37 pts still alive after 10 years agreed to the SSP (13 in A; 12 in B; 6 in C). Clinically relevant long-term toxicities (grade 3 and 4) were rarely observed with no signal favoring any of the randomization arms. Furthermore, available data from the global QoL analysis did not show a signal favoring any definitive locoregional approach (Mean QoL in SSP A pts: 56.41/100, B pts: 64.39/100) and no late decline in comparison to baseline and early 1-year follow-up. This is the first comprehensive SSP of very late survival follow-up reported in stage-III NSCLC treated within a randomized multimodality trial and it may serve as important baseline information for physicians and pts deciding for a locoregional treatment option.

Combining Nuclear Medicine With Other Modalities: Future Prospect for Multimodality Imaging.

This meeting report summarizes a consultants meeting that was held at International Atomic Energy Agency Headquarters, Vienna, in July 2022 to provide an update on the development of multimodality imaging by combining nuclear medicine imaging agents with other nonradioactive molecular probes and/or biomedical imaging techniques.

Deep learning radiomics under multimodality explore association between muscle/fat and metastasis and survival in breast cancer patients.

Sarcopenia is correlated with poor clinical outcomes in breast cancer (BC) patients. However, there is no precise quantitative study on the correlation between body composition changes and BC metastasis and survival. The present study proposed a deep learning radiomics (DLR) approach to investigate the effects of muscle and fat on distant metastasis and death outcomes in BC patients. Image feature extraction was performed on 4th thoracic vertebra (T4) and 11th thoracic vertebra (T11) on computed tomography (CT) image levels by DLR, and image features were combined with clinical information to predict distant metastasis in BC patients. Clinical information combined with DLR significantly predicted distant metastasis in BC patients. In the test cohort, the area under the curve of model performance on clinical information combined with DLR was 0.960 (95% CI: 0.942-0.979, P < 0.001). The patients with distant metastases had a lower pectoral muscle index in T4 (PMI/T4) than in patients without metastases. PMI/T4 and visceral fat tissue area in T11 (VFA/T11) were independent prognostic factors for the overall survival in BC patients. The pectoralis muscle area in T4 (PMA/T4) and PMI/T4 is an independent prognostic factor for distant metastasis-free survival in BC patients. The current study further confirmed that muscle/fat of T4 and T11 levels have a significant effect on the distant metastasis of BC. Appending the network features of T4 and T11 to the model significantly enhances the prediction performance of distant metastasis of BC, providing a valuable biomarker for the early treatment of BC patients.

Multi-modality artificial intelligence in digital pathology.

In common medical procedures, the time-consuming and expensive nature of obtaining test results plagues doctors and patients. Digital pathology research allows using computational technologies to manage data, presenting an opportunity to improve the efficiency of diagnosis and treatment. Artificial intelligence (AI) has a great advantage in the data analytics phase. Extensive research has shown that AI algorithms can produce more up-to-date and standardized conclusions for whole slide images. In conjunction with the development of high-throughput sequencing technologies, algorithms can integrate and analyze data from multiple modalities to explore the correspondence between morphological features and gene expression. This review investigates using the most popular image data, hematoxylin-eosin stained tissue slide images, to find a strategic solution for the imbalance of healthcare resources. The article focuses on the role that the development of deep learning technology has in assisting doctors' work and discusses the opportunities and challenges of AI.

ICSDA: a multi-modal deep learning model to predict breast cancer recurrence and metastasis risk by integrating pathological, clinical and gene expression data.

Breast cancer patients often have recurrence and metastasis after surgery. Predicting the risk of recurrence and metastasis for a breast cancer patient is essential for the development of precision treatment. In this study, we proposed a novel multi-modal deep learning prediction model by integrating hematoxylin & eosin (H&E)-stained histopathological images, clinical information and gene expression data. Specifically, we segmented tumor regions in H&E into image blocks (256 × 256 pixels) and encoded each image block into a 1D feature vector using a deep neural network. Then, the attention module scored each area of the H&E-stained images and combined image features with clinical and gene expression data to predict the risk of recurrence and metastasis for each patient. To test the model, we downloaded all 196 breast cancer samples from the Cancer Genome Atlas with clinical, gene expression and H&E information simultaneously available. The samples were then divided into the training and testing sets with a ratio of 7: 3, in which the distributions of the samples were kept between the two datasets by hierarchical sampling. The multi-modal model achieved an area-under-the-curve value of 0.75 on the testing set better than those based solely on H&E image, sequencing data and clinical data, respectively. This study might have clinical significance in identifying high-risk breast cancer patients, who may benefit from postoperative adjuvant treatment.

Visualization, benchmarking and characterization of nested single-cell heterogeneity as dynamic forest mixtures.

A major topic of debate in developmental biology centers on whether development is continuous, discontinuous, or a mixture of both. Pseudo-time trajectory models, optimal for visualizing cellular progression, model cell transitions as continuous state manifolds and do not explicitly model real-time, complex, heterogeneous systems and are challenging for benchmarking with temporal models. We present a data-driven framework that addresses these limitations with temporal single-cell data collected at discrete time points as inputs and a mixture of dependent minimum spanning trees (MSTs) as outputs, denoted as dynamic spanning forest mixtures (DSFMix). DSFMix uses decision-tree models to select genes that account for variations in multimodality, skewness and time. The genes are subsequently used to build the forest using tree agglomerative hierarchical clustering and dynamic branch cutting. We first motivate the use of forest-based algorithms compared to single-tree approaches for visualizing and characterizing developmental processes. We next benchmark DSFMix to pseudo-time and temporal approaches in terms of feature selection, time correlation, and network similarity. Finally, we demonstrate how DSFMix can be used to visualize, compare and characterize complex relationships during biological processes such as epithelial-mesenchymal transition, spermatogenesis, stem cell pluripotency, early transcriptional response from hormones and immune response to coronavirus disease. Our results indicate that the expression of genes during normal development exhibits a high proportion of non-uniformly distributed profiles that are mostly right-skewed and multimodal; the latter being a characteristic of major steady states during development. Our study also identifies and validates gene signatures driving complex dynamic processes during somatic or germline differentiation.

ALDPI: adaptively learning importance of multi-scale topologies and multi-modality similarities for drug-protein interaction prediction.

MOTIVATION: Effective computational methods to predict drug-protein interactions (DPIs) are vital for drug discovery in reducing the time and cost of drug development. Recent DPI prediction methods mainly exploit graph data composed of multiple kinds of connections among drugs and proteins. Each node in the graph usually has topological structures with multiple scales formed by its first-order neighbors and multi-order neighbors. However, most of the previous methods do not consider the topological structures of multi-order neighbors. In addition, deep integration of the multi-modality similarities of drugs and proteins is also a challenging task. RESULTS: We propose a model called ALDPI to adaptively learn the multi-scale topologies and multi-modality similarities with various significance levels. We first construct a drug-protein heterogeneous graph, which is composed of the interactions and the similarities with multiple modalities among drugs and proteins. An adaptive graph learning module is then designed to learn important kinds of connections in heterogeneous graph and generate new topology graphs. A module based on graph convolutional autoencoders is established to learn multiple representations, which imply the node attributes and multiple-scale topologies composed of one-order and multi-order neighbors, respectively. We also design an attention mechanism at neighbor topology level to distinguish the importance of these representations. Finally, since each similarity modality has its specific features, we construct a multi-layer convolutional neural network-based module to learn and fuse multi-modality features to obtain the attribute representation of each drug-protein node pair. Comprehensive experimental results show ALDPI's superior performance over six state-of-the-art methods. The results of recall rates of top-ranked candidates and case studies on five drugs further demonstrate the ability of ALDPI to discover potential drug-related protein candidates. CONTACT: zhang@hlju.edu.cn.

Imaging Findings in Cardiovascular Involvements of IgG4 Related Disease: A Systematic Review Study.

BACKGROUND: Immunoglobulin G4-related disease (IgG4-RD) is a fibroinflammatory condition characterized by IgG4-positive plasma cell infiltration that can affect multiple organs, including the cardiovascular system. The diagnosis of IgG4-RD relies on a combination of clinical, serological, radiological, and pathological findings. However, due to the varied and insidious clinical presentations, normal IgG4 levels in a significant percentage of patients, and frequent multi-organ involvement, imaging plays a crucial role in the diagnosis of IgG4-RD. The aim of study is to comprehensively examine the imaging findings in IgG4-related cardiovascular disease for accurate diagnosis and appropriate treatment. METHODS: A systematic search was conducted across electronic databases, PubMed, Scopus, and Web of Sciences, until 1 September 2023, following PRISMA guidelines by searching major databases for studies reporting detailed cardiovascular imaging findings in IgG4-RD. RESULTS: The search yielded 68 studies (60 case reports, 5 case series, 2 cross-sectional, 1 case-control) with 120 cases of cardiovascular IgG4-RD. Most of the cases were male, averaging 62.8 years. The common initial symptoms were dyspnea and chest pain. The most common imaging finding was vasculopathy, including vessel wall thickening, periarteritits, periaortitis, aortitis, stenosis, ectasia, aneurysm formation, intramural hemorrhage, fistula formation, and dissection, followed by pericardial involvement and mediastinal masses. Case series and cross-sectional studies also showed vasculopathy being the most common finding on various imaging modalities, including angiography and PET/CT, highlighting the complex pathology of IgG4-RD. CONCLUSION: This study evaluated current IgG4-RD articles, revealing a higher prevalence in men and vasculopathy as the most common cardiovascular complication.

ASO Practice Guidelines Series: Management of Resectable, Borderline Resectable, and Locally Advanced Pancreas Cancer.

Pancreatic adenocarcinoma is an aggressive disease marked by high rates of both local and distant failure. In the minority of patients with potentially resectable disease, multimodal treatment paradigms have allowed for prolonged survival in an increasingly larger pool of well-selected patients. Therefore, it is critical for surgical oncologists to be abreast of current guideline recommendations for both surgical management and multimodal therapy for pancreas cancer. We discuss these guidelines, as well as the underlying data supporting these positions, to offer surgical oncologists a framework for managing patients with pancreatic adenocarcinoma.

18F-FDG PET/CT Radiomics-Based Multimodality Fusion Model for Preoperative Individualized Noninvasive Prediction of Peritoneal Metastasis in Advanced Gastric Cancer.

PURPOSE: This study was designed to develop and validate a machine learning-based, multimodality fusion (MMF) model using 18F-fluorodeoxyglucose (FDG) PET/CT radiomics and kernelled support tensor machine (KSTM), integrated with clinical factors and nuclear medicine experts' diagnoses to individually predict peritoneal metastasis (PM) in advanced gastric cancer (AGC). METHODS: A total of 167 patients receiving preoperative PET/CT and subsequent surgery were included between November 2006 and September 2020 and were divided into a training and testing cohort. The PM status was confirmed via laparoscopic exploration and postoperative pathology. The PET/CT signatures were constructed by classic radiomic, handcrafted-feature-based model and KSTM self-learning-based model. The clinical nomogram was constructed by independent risk factors for PM. Lastly, the PET/CT signatures, clinical nomogram, and experts' diagnoses were fused using evidential reasoning to establish the MMF model. RESULTS: The MMF model showed excellent performance in both cohorts (area under the curve [AUC] 94.16% and 90.84% in training and testing), and demonstrated better prediction accuracy than clinical nomogram or experts' diagnoses (net reclassification improvement p < 0.05). The MMF model also had satisfactory generalization ability, even in mucinous adenocarcinoma and signet ring cell carcinoma which have poor uptake of 18F-FDG (AUC 97.98% and 89.71% in training and testing). CONCLUSIONS: The 18F-FDG PET/CT radiomics-based MMF model may have significant clinical implications in predicting PM in AGC, revealing that it is necessary to combine the information from different modalities for comprehensive prediction of PM.