MHD-Net: Memory-Aware Hetero-Modal Distillation Network for Thymic Epithelial Tumor Typing With Missing Pathology Modality.

Fusing multi-modal radiology and pathology data with complementary information can improve the accuracy of tumor typing. However, collecting pathology data is difficult since it is high-cost and sometimes only obtainable after the surgery, which limits the application of multi-modal methods in diagnosis. To address this problem, we propose comprehensively learning multi-modal radiology-pathology data in training, and only using uni-modal radiology data in testing. Concretely, a Memory-aware Hetero-modal Distillation Network (MHD-Net) is proposed, which can distill well-learned multi-modal knowledge with the assistance of memory from the teacher to the student. In the teacher, to tackle the challenge in hetero-modal feature fusion, we propose a novel spatial-differentiated hetero-modal fusion module (SHFM) that models spatial-specific tumor information correlations across modalities. As only radiology data is accessible to the student, we store pathology features in the proposed contrast-boosted typing memory module (CTMM) that achieves type-wise memory updating and stage-wise contrastive memory boosting to ensure the effectiveness and generalization of memory items. In the student, to improve the cross-modal distillation, we propose a multi-stage memory-aware distillation (MMD) scheme that reads memory-aware pathology features from CTMM to remedy missing modal-specific information. Furthermore, we construct a Radiology-Pathology Thymic Epithelial Tumor (RPTET) dataset containing paired CT and WSI images with annotations. Experiments on the RPTET and CPTAC-LUAD datasets demonstrate that MHD-Net significantly improves tumor typing and outperforms existing multi-modal methods on missing modality situations.

HSA-net with a novel CAD pipeline boosts both clinical brain tumor MR image classification and segmentation.

Brain tumors are among the most prevalent neoplasms in current medical studies. Accurately distinguishing and classifying brain tumor types accurately is crucial for patient treatment and survival in clinical practice. However, existing computer-aided diagnostic pipelines are inadequate for practical medical use due to tumor complexity. In this study, we curated a multi-centre brain tumor dataset that includes various clinical brain tumor data types, including segmentation and classification annotations, surpassing previous efforts. To enhance brain tumor segmentation accuracy, we propose a new segmentation method: HSA-Net. This method utilizes the Shared Weight Dilated Convolution module (SWDC) and Hybrid Dense Dilated Convolution module (HDense) to capture multi-scale information while minimizing parameter count. The Effective Multi-Dimensional Attention (EMA) and Important Feature Attention (IFA) modules effectively aggregate task-related information. We introduce a novel clinical brain tumor computer-aided diagnosis pipeline (CAD) that combines HSA-Net with pipeline modification. This approach not only improves segmentation accuracy but also utilizes the segmentation mask as an additional channel feature to enhance brain tumor classification results. Our experimental evaluation of 3327 real clinical data demonstrates the effectiveness of the proposed method, achieving an average Dice coefficient of 86.85 % for segmentation and a classification accuracy of 95.35 %. We also validated the effectiveness of our proposed method using the publicly available BraTS dataset.