MGIML: Cancer Grading With Incomplete Radiology-Pathology Data via Memory Learning and Gradient Homogenization.

Taking advantage of multi-modal radiology-pathology data with complementary clinical information for cancer grading is helpful for doctors to improve diagnosis efficiency and accuracy. However, radiology and pathology data have distinct acquisition difficulties and costs, which leads to incomplete-modality data being common in applications. In this work, we propose a Memory- and Gradient-guided Incomplete Modal-modal Learning (MGIML) framework for cancer grading with incomplete radiology-pathology data. Firstly, to remedy missing-modality information, we propose a Memory-driven Hetero-modality Complement (MH-Complete) scheme, which constructs modal-specific memory banks constrained by a coarse-grained memory boosting (CMB) loss to record generic radiology and pathology feature patterns, and develops a cross-modal memory reading strategy enhanced by a fine-grained memory consistency (FMC) loss to take missing-modality information from well-stored memories. Secondly, as gradient conflicts exist between missing-modality situations, we propose a Rotation-driven Gradient Homogenization (RG-Homogenize) scheme, which estimates instance-specific rotation matrices to smoothly change the feature-level gradient directions, and computes confidence-guided homogenization weights to dynamically balance gradient magnitudes. By simultaneously mitigating gradient direction and magnitude conflicts, this scheme well avoids the negative transfer and optimization imbalance problems. Extensive experiments on CPTAC-UCEC and CPTAC-PDA datasets show that the proposed MGIML framework performs favorably against state-of-the-art multi-modal methods on missing-modality situations.

FedDM: Federated Weakly Supervised Segmentation via Annotation Calibration and Gradient De-Conflicting.

Weakly supervised segmentation (WSS) aims to exploit weak forms of annotations to achieve the segmentation training, thereby reducing the burden on annotation. However, existing methods rely on large-scale centralized datasets, which are difficult to construct due to privacy concerns on medical data. Federated learning (FL) provides a cross-site training paradigm and shows great potential to address this problem. In this work, we represent the first effort to formulate federated weakly supervised segmentation (FedWSS) and propose a novel Federated Drift Mitigation (FedDM) framework to learn segmentation models across multiple sites without sharing their raw data. FedDM is devoted to solving two main challenges (i.e., local drift on client-side optimization and global drift on server-side aggregation) caused by weak supervision signals in FL setting via Collaborative Annotation Calibration (CAC) and Hierarchical Gradient De-conflicting (HGD). To mitigate the local drift, CAC customizes a distal peer and a proximal peer for each client via a Monte Carlo sampling strategy, and then employs inter-client knowledge agreement and disagreement to recognize clean labels and correct noisy labels, respectively. Moreover, in order to alleviate the global drift, HGD online builds a client hierarchy under the guidance of history gradient of the global model in each communication round. Through de-conflicting clients under the same parent nodes from bottom layers to top layers, HGD achieves robust gradient aggregation at the server side. Furthermore, we theoretically analyze FedDM and conduct extensive experiments on public datasets. The experimental results demonstrate the superior performance of our method compared with state-of-the-art approaches. The source code is available at https://github.com/CityU-AIM-Group/FedDM.