MSCT-UNET: multi-scale contrastive transformer within U-shaped network for medical image segmentation.

Objective.Automatic mutli-organ segmentation from anotomical images is essential in disease diagnosis and treatment planning. The U-shaped neural network with encoder-decoder has achieved great success in various segmentation tasks. However, a pure convolutional neural network (CNN) is not suitable for modeling long-range relations due to limited receptive fields, and a pure transformer is not good at capturing pixel-level features.Approach.We propose a new hybrid network named MSCT-UNET which fuses CNN features with transformer features at multi-scale and introduces multi-task contrastive learning to improve the segmentation performance. Specifically, the multi-scale low-level features extracted from CNN are further encoded through several transformers to build hierarchical global contexts. Then the cross fusion block fuses the low-level and high-level features in different directions. The deep-fused features are flowed back to the CNN and transformer branch for the next scale fusion. We introduce multi-task contrastive learning including a self-supervised global contrast learning and a supervised local contrast learning into MSCT-UNET. We also make the decoder stronger by using a transformer to better restore the segmentation map.Results.Evaluation results on ACDC, Synapase and BraTS datasets demonstrate the improved performance over other methods compared. Ablation study results prove the effectiveness of our major innovations.Significance.The hybrid encoder of MSCT-UNET can capture multi-scale long-range dependencies and fine-grained detail features at the same time. The cross fusion block can fuse these features deeply. The multi-task contrastive learning of MSCT-UNET can strengthen the representation ability of the encoder and jointly optimize the networks. The source code is publicly available at:https://github.com/msctunet/MSCT_UNET.git.

Multi-task contrastive learning for semi-supervised medical image segmentation with multi-scale uncertainty estimation.

Objective. Automated medical image segmentation is vital for the prevention and treatment of disease. However, medical data commonly exhibit class imbalance in practical applications, which may lead to unclear boundaries of specific classes and make it difficult to effectively segment certain tail classes in the results of semi-supervised medical image segmentation.Approach. We propose a novel multi-task contrastive learning framework for semi-supervised medical image segmentation with multi-scale uncertainty estimation. Specifically, the framework includes a student-teacher model. We introduce global image-level contrastive learning in the encoder to address the class imbalance and local pixel-level contrastive learning in the decoder to achieve intra-class aggregation and inter-class separation. Furthermore, we propose a multi-scale uncertainty-aware consistency loss to reduce noise caused by pseudo-label bias.Main results. Experiments on three public datasets ACDC, LA and LiTs show that our method achieves higher segmentation performance compared with state-of-the-art semi-supervised segmentation methods.Significance. The multi-task contrastive learning in our method facilitates the negative impact of class imbalance and achieves better classification results. The multi-scale uncertainty estimation encourages consistent predictions for the same input under different perturbations, motivating the teacher model to generate high-quality pseudo-labels. Code is available athttps://github.com/msctransu/MCSSMU.git.