Hierarchical multimodal fusion framework based on noisy label learning and attention mechanism for cancer classification with pathology and genomic features.

Classification of subtype and grade is imperative in the clinical diagnosis and prognosis of cancer. Many deep learning-based studies related to cancer classification are based on pathology and genomics. However, most of them are late fusion-based and require full supervision in pathology image analysis. To address these problems, we present an integrated framework for cancer classification with pathology and genomics data. This framework consists of two major parts, a weakly supervised model for extracting patch features from whole slide images (WSIs), and a hierarchical multimodal fusion model. The weakly supervised model can make full use of WSI labels, and mitigate the effects of label noises by the self-training strategy. The generic multimodal fusion model is capable of capturing deep interaction information through multi-level attention mechanisms and controlling the expressiveness of each modal representation. We validate our approach on glioma and lung cancer datasets from The Cancer Genome Atlas (TCGA). The results demonstrate that the proposed method achieves superior performance compared to state-of-the-art methods, with the competitive AUC of 0.872 and 0.977 on these two datasets respectively. This paper establishes insight on how to build deep networks on multimodal biomedical data and proposes a more general framework for pathology image analysis without pixel-level annotation.

Customized Total Variation Algorithm for Metal Artifact Reduction in Computed Tomography.

Metal artifact reduction (MAR) is a challenge for commercial CT systems. The metal objects of high density adversely affect the measurement process and bring difficulties to image reconstruction. Compressed sensing (CS) reconstruction algorithms have been successfully applied in MAR. Ideally, the desired anatomical information can be restored from incomplete projection data. However, in most practical cases, these conventional CS algorithms may instead introduce severe secondary artifacts due to improper prior information. In this paper, we propose a customized total variation (CTV) method to reduce the metal artifacts based on the specific pattern of the artifacts. The gradient operator within the TV norm is redefined according to the distribution of both the metal objects and tissues for each MAR case. We also provide a weighting strategy to further protect the fine details. Experimental results show that the CTV method achieves better performances than those of the conventional methods.