Mutually enhanced multi-view information learning for segmentation of lung tumor in CT images.

Objective.The accurate automatic segmentation of tumors from computed tomography (CT) volumes facilitates early diagnosis and treatment of patients. A significant challenge in tumor segmentation is the integration of the spatial correlations among multiple parts of a CT volume and the context relationship across multiple channels.Approach.We proposed a mutually enhanced multi-view information model (MEMI) to propagate and fuse the spatial correlations and the context relationship and then apply it to lung tumor CT segmentation. First, a feature map was obtained from segmentation backbone encoder, which contained many image region nodes. An attention mechanism from the region node perspective was presented to determine the impact of all the other nodes on a specific node and enhance the node attribute embedding. A gated convolution-based strategy was also designed to integrate the enhanced attributes and the original node features. Second, transformer across multiple channels was constructed to integrate the channel context relationship. Finally, since the encoded node attributes from the gated convolution view and those from the channel transformer view were complementary, an interaction attention mechanism was proposed to propagate the mutual information among the multiple views.Main results.The segmentation performance was evaluated on both public lung tumor dataset and private dataset collected from a hospital. The experimental results demonstrated that MEMI was superior to other compared segmentation methods. Ablation studies showed the contributions of node correlation learning, channel context relationship learning, and mutual information interaction across multiple views to the improved segmentation performance. Utilizing MEMI on multiple segmentation backbones also demonstrated MEMI's generalization ability.Significance.Our model improved the lung tumor segmentation performance by learning the correlations among multiple region nodes, integrating the channel context relationship, and mutual information enhancement from multiple views.

Topological structure and global features enhanced graph reasoning model for non-small cell lung cancer segmentation from CT.

Objective.Accurate and automated segmentation of lung tumors from computed tomography (CT) images is critical yet challenging. Lung tumors are of various sizes and locations and have indistinct boundaries adjacent to other normal tissues.Approach.We propose a new segmentation model that can integrate the topological structure and global features of image region nodes to address the challenges. Firstly, we construct a weighted graph with image region nodes. The graph topology reflects the complex spatial relationships among these nodes, and each node has its specific attributes. Secondly, we propose a node-wise topological feature learning module based on a new graph convolutional autoencoder (GCA). Meanwhile, a node information supplementation (GNIS) module is established by integrating specific features of each node extracted by a convolutional neural network (CNN) into each encoding layer of GCA. Afterwards, we construct a global feature extraction model based on multi-layer perceptron (MLP) to encode the features learnt from all the image region nodes which are crucial complementary information for tumor segmentation.Main results.Ablation study results over the public lung tumor segmentation dataset demonstrate the contributions of our major technical innovations. Compared with other segmentation methods, our new model improves the segmentation performance and has generalization ability on different 3D image segmentation backbones. Our model achieved Dice of 0.7827, IoU of 0.6981, and HD of 32.1743 mm on the public dataset 2018 Medical Segmentation Decathlon challenge, and Dice of 0.7004, IoU of 0.5704 and HD of 64.4661 mm on lung tumor dataset from Shandong Cancer Hospital.Significance. The novel model improves automated lung tumor segmentation performance especially the challenging and complex cases using topological structure and global features of image region nodes. It is of great potential to apply the model to other CT segmentation tasks.

Semantic Meta-Path Enhanced Global and Local Topology Learning for lncRNA-Disease Association Prediction.

Since abnormal expression of long non-coding RNAs (lncRNAs) is associated with various human diseases, identifying disease-related lncRNAs helps reveal the pathogenesis of diseases. Existing methods for lncRNA-disease association prediction mainly focus on multi-sourced data related to lncRNAs and diseases. The rich semantic information of meta-paths, composed of multiple kinds of connections between lncRNA and disease nodes, is neglected. We propose a new prediction method, MGLDA, to encode and integrate the semantics of multiple meta-paths, the global topology of heterogeneous graph, and pairwise attributes of lncRNA and disease nodes. First, a tri-layer heterogeneous graph is constructed to associate multi-sourced data across the lncRNA, disease, and miRNA nodes. Afterwards, we establish multiple meta-paths connecting the lncRNA and disease nodes to derive and denote various semantics. Each meta-path contains its specific semantics formulated by an embedding strategy, and each embedding covers local topology formed by the diverse semantic connections among the lncRNA, disease, and miRNA nodes. We construct multiple graph convolutional autoencoders (GCA) with topology-level attention to learn global and multiple local topologies from the tri-layer graph and each meta-path, respectively. The topology-level attention mechanism can learn the importance of various global and local topologies for adaptive pairwise topology fusion. Finally, a convolutional autoencoder learns the attribute representations of lncRNA-disease pairs, which integrates the learnt detailed and representative pairwise features. Experimental results show that MGLDA outperforms other state-of-the-art prediction methods in comparison and retrieves more real lncRNA-disease associations in the top-ranked candidates. The ablation study also demonstrates the important contributions of the local and global topology learning, and pairwise attribute learning. Case studies on three diseases further demonstrate MGLDA's ability to identify potential disease-related lncRNAs.

Graph based multi-scale neighboring topology deep learning for kidney and tumor segmentation.

Objective.Effective learning and modelling of spatial and semantic relations between image regions in various ranges are critical yet challenging in image segmentation tasks.Approach.We propose a novel deep graph reasoning model to learn from multi-order neighborhood topologies for volumetric image segmentation. A graph is first constructed with nodes representing image regions and graph topology to derive spatial dependencies and semantic connections across image regions. We propose a new node attribute embedding mechanism to formulate topological attributes for each image region node by performing multi-order random walks (RW) on the graph and updating neighboring topologies at different neighborhood ranges. Afterwards, multi-scale graph convolutional autoencoders are developed to extract deep multi-scale topological representations of nodes and propagate learnt knowledge along graph edges during the convolutional and optimization process. We also propose a scale-level attention module to learn the adaptive weights of topological representations at multiple scales for enhanced fusion. Finally, the enhanced topological representation and knowledge from graph reasoning are integrated with content features before feeding into the segmentation decoder.Main results.The evaluation results over public kidney and tumor CT segmentation dataset show that our model outperforms other state-of-the-art segmentation methods. Ablation studies and experiments using different convolutional neural networks backbones show the contributions of major technical innovations and generalization ability.Significance.We propose for the first time an RW-driven MCG with scale-level attention to extract semantic connections and spatial dependencies between a diverse range of regions for accurate kidney and tumor segmentation in CT volumes.

Convolutional bi-directional learning and spatial enhanced attentions for lung tumor segmentation.

BACKGROUND AND OBJECTIVE: Accurate lung tumor segmentation from computed tomography (CT) is complex due to variations in tumor sizes, shapes, patterns and growing locations. Learning semantic and spatial relations between different feature channels, image regions and positions is critical yet challenging. METHODS: We propose a new segmentation method, PRCS, by learning and integrating multi-channel contextual relations, and spatial and position dependencies across image regions. Firstly, to extract contextual relationships between different deep image feature tensor channels, we propose a new convolutional bi-directional gated recurrent unit based module for forward and backward learning. Secondly, a novel cross-channel region-level attention mechanism is proposed to discriminate the contributions of different local regions and associated features in the global learning process. Finally, spatial and position dependencies are formulated by a new position-enhanced self-attention mechanism. The new attention can measure the diverse contributions of other positions to a target position and obtain an enhanced adaptive feature vector for the target position. RESULTS: Our model outperformed seven state-of-the-art segmentation methods on both public and in-house lung tumor datasets in terms of spatial overlapping and shape similarity. Ablation study results proved the effectiveness of three technical innovations and generalization ability on different 3D CNN segmentation backbones. CONCLUSION: The proposed model enhanced the learning and propagation of contextual, spatial and position relations in 3D volumes, improving lung tumours' segmentation performance with large variations and indistinct boundaries. PRCS provides an effective automated approach to support precision diagnosis and treatment planning of lung cancer.

Domain adaptation based self-correction model for COVID-19 infection segmentation in CT images.

The capability of generalization to unseen domains is crucial for deep learning models when considering real-world scenarios. However, current available medical image datasets, such as those for COVID-19 CT images, have large variations of infections and domain shift problems. To address this issue, we propose a prior knowledge driven domain adaptation and a dual-domain enhanced self-correction learning scheme. Based on the novel learning scheme, a domain adaptation based self-correction model (DASC-Net) is proposed for COVID-19 infection segmentation on CT images. DASC-Net consists of a novel attention and feature domain enhanced domain adaptation model (AFD-DA) to solve the domain shifts and a self-correction learning process to refine segmentation results. The innovations in AFD-DA include an image-level activation feature extractor with attention to lung abnormalities and a multi-level discrimination module for hierarchical feature domain alignment. The proposed self-correction learning process adaptively aggregates the learned model and corresponding pseudo labels for the propagation of aligned source and target domain information to alleviate the overfitting to noises caused by pseudo labels. Extensive experiments over three publicly available COVID-19 CT datasets demonstrate that DASC-Net consistently outperforms state-of-the-art segmentation, domain shift, and coronavirus infection segmentation methods. Ablation analysis further shows the effectiveness of the major components in our model. The DASC-Net enriches the theory of domain adaptation and self-correction learning in medical imaging and can be generalized to multi-site COVID-19 infection segmentation on CT images for clinical deployment.

RA-UNet: A Hybrid Deep Attention-Aware Network to Extract Liver and Tumor in CT Scans.

Automatic extraction of liver and tumor from CT volumes is a challenging task due to their heterogeneous and diffusive shapes. Recently, 2D deep convolutional neural networks have become popular in medical image segmentation tasks because of the utilization of large labeled datasets to learn hierarchical features. However, few studies investigate 3D networks for liver tumor segmentation. In this paper, we propose a 3D hybrid residual attention-aware segmentation method, i.e., RA-UNet, to precisely extract the liver region and segment tumors from the liver. The proposed network has a basic architecture as U-Net which extracts contextual information combining low-level feature maps with high-level ones. Attention residual modules are integrated so that the attention-aware features change adaptively. This is the first work that an attention residual mechanism is used to segment tumors from 3D medical volumetric images. We evaluated our framework on the public MICCAI 2017 Liver Tumor Segmentation dataset and tested the generalization on the 3DIRCADb dataset. The experiments show that our architecture obtains competitive results.