Segmentation model of soft tissue sarcoma based on self-supervised learning.

Introduction: Soft tissue sarcomas, similar in incidence to cervical and esophageal cancers, arise from various soft tissues like smooth muscle, fat, and fibrous tissue. Effective segmentation of sarcomas in imaging is crucial for accurate diagnosis. Methods: This study collected multi-modal MRI images from 45 patients with thigh soft tissue sarcoma, totaling 8,640 images. These images were annotated by clinicians to delineate the sarcoma regions, creating a comprehensive dataset. We developed a novel segmentation model based on the UNet framework, enhanced with residual networks and attention mechanisms for improved modality-specific information extraction. Additionally, self-supervised learning strategies were employed to optimize feature extraction capabilities of the encoders. Results: The new model demonstrated superior segmentation performance when using multi-modal MRI images compared to single-modal inputs. The effectiveness of the model in utilizing the created dataset was validated through various experimental setups, confirming the enhanced ability to characterize tumor regions across different modalities. Discussion: The integration of multi-modal MRI images and advanced machine learning techniques in our model significantly improves the segmentation of soft tissue sarcomas in thigh imaging. This advancement aids clinicians in better diagnosing and understanding the patient's condition, leveraging the strengths of different imaging modalities. Further studies could explore the application of these techniques to other types of soft tissue sarcomas and additional anatomical sites.

Transformers for colorectal cancer segmentation in CT imaging.

PURPOSE: Most recently transformer models became the state of the art in various medical image segmentation tasks and challenges, outperforming most of the conventional deep learning approaches. Picking up on that trend, this study aims at applying various transformer models to the highly challenging task of colorectal cancer (CRC) segmentation in CT imaging and assessing how they hold up to the current state-of-the-art convolutional neural network (CNN), the nnUnet. Furthermore, we wanted to investigate the impact of the network size on the resulting accuracies, since transformer models tend to be significantly larger than conventional network architectures. METHODS: For this purpose, six different transformer models, with specific architectural advancements and network sizes were implemented alongside the aforementioned nnUnet and were applied to the CRC segmentation task of the medical segmentation decathlon. RESULTS: The best results were achieved with the Swin-UNETR, D-Former, and VT-Unet, each transformer models, with a Dice similarity coefficient (DSC) of 0.60, 0.59 and 0.59, respectively. Therefore, the current state-of-the-art CNN, the nnUnet could be outperformed by transformer architectures regarding this task. Furthermore, a comparison with the inter-observer variability (IOV) of approx. 0.64 DSC indicates almost expert-level accuracy. The comparatively low IOV emphasizes the complexity and challenge of CRC segmentation, as well as indicating limitations regarding the achievable segmentation accuracy. CONCLUSION: As a result of this study, transformer models underline their current upward trend in producing state-of-the-art results also for the challenging task of CRC segmentation. However, with ever smaller advances in total accuracies, as demonstrated in this study by the on par performances of multiple network variants, other advantages like efficiency, low computation demands, or ease of adaption to new tasks become more and more relevant.

Balancing High-performance and Lightweight: HL-UNet for 3D Cardiac Medical Image Segmentation.

RATIONALE AND OBJECTIVES: Cardiac magnetic resonance imaging is a crucial tool for analyzing, diagnosing, and formulating treatment plans for cardiovascular diseases. Currently, there is very little research focused on balancing cardiac segmentation performance with lightweight methods. Despite the existence of numerous efficient image segmentation algorithms, they primarily rely on complex and computationally intensive network models, making it challenging to implement them on resource-constrained medical devices. Furthermore, simplified models designed to meet the requirements of device lightweighting may have limitations in comprehending and utilizing both global and local information for cardiac segmentation. MATERIALS AND METHODS: We propose a novel 3D high-performance lightweight medical image segmentation network, HL-UNet, for application in cardiac image segmentation. Specifically, in HL-UNet, we propose a novel residual-enhanced Adaptive attention (REAA) module that combines residual-enhanced connectivity with an adaptive attention mechanism to efficiently capture key features of input images and optimize their representation capabilities, and integrates the Visual Mamba (VSS) module to enhance the performance of HL-UNet. RESULTS: Compared to large-scale models such as TransUNet, HL-UNet increased the Dice of the right ventricular cavity (RV), left ventricular myocardia (MYO), and left ventricular cavity (LV), the key indicators of cardiac image segmentation, by 1.61%, 5.03% and 0.19%, respectively. At the same time, the Params and FLOPs of the model decreased by 41.3 M and 31.05 G, respectively. Furthermore, compared to lightweight models such as the MISSFormer, the HL-UNet improves the Dice of RV, MYO, and LV by 4.11%, 3.82%, and 4.33%, respectively, when the number of parameters and computational complexity are close to or even lower. CONCLUSION: The proposed HL-UNet model captures local details and edge information in images while being lightweight. Experimental results show that compared with large-scale models, HL-UNet significantly reduces the number of parameters and computational complexity while maintaining performance, thereby increasing frames per second (FPS). Compared to lightweight models, HL-UNet shows substantial improvements across various key metrics, with parameter count and computational complexity approaching or even lower.

WET-UNet: Wavelet integrated efficient transformer networks for nasopharyngeal carcinoma tumor segmentation.

Nasopharyngeal carcinoma is a malignant tumor that occurs in the epithelium and mucosal glands of the nasopharynx, and its pathological type is mostly poorly differentiated squamous cell carcinoma. Since the nasopharynx is located deep in the head and neck, early diagnosis and timely treatment are critical to patient survival. However, nasopharyngeal carcinoma tumors are small in size and vary widely in shape, and it is also a challenge for experienced doctors to delineate tumor contours. In addition, due to the special location of nasopharyngeal carcinoma, complex treatments such as radiotherapy or surgical resection are often required, so accurate pathological diagnosis is also very important for the selection of treatment options. However, the current deep learning segmentation model faces the problems of inaccurate segmentation and unstable segmentation process, which are mainly limited by the accuracy of data sets, fuzzy boundaries, and complex lines. In order to solve these two challenges, this article proposes a hybrid model WET-UNet based on the UNet network as a powerful alternative for nasopharyngeal cancer image segmentation. On the one hand, wavelet transform is integrated into UNet to enhance the lesion boundary information by using low-frequency components to adjust the encoder at low frequencies and optimize the subsequent computational process of the Transformer to improve the accuracy and robustness of image segmentation. On the other hand, the attention mechanism retains the most valuable pixels in the image for us, captures the remote dependencies, and enables the network to learn more representative features to improve the recognition ability of the model. Comparative experiments show that our network structure outperforms other models for nasopharyngeal cancer image segmentation, and we demonstrate the effectiveness of adding two modules to help tumor segmentation. The total data set of this article is 5000, and the ratio of training and verification is 8:2. In the experiment, accuracy = 85.2% and precision = 84.9% can show that our proposed model has good performance in nasopharyngeal cancer image segmentation.

Learnable weight initialization for volumetric medical image segmentation.

Hybrid volumetric medical image segmentation models, combining the advantages of local convolution and global attention, have recently received considerable attention. While mainly focusing on architectural modifications, most existing hybrid approaches still use conventional data-independent weight initialization schemes which restrict their performance due to ignoring the inherent volumetric nature of the medical data. To address this issue, we propose a learnable weight initialization approach that utilizes the available medical training data to effectively learn the contextual and structural cues via the proposed self-supervised objectives. Our approach is easy to integrate into any hybrid model and requires no external training data. Experiments on multi-organ and lung cancer segmentation tasks demonstrate the effectiveness of our approach, leading to state-of-the-art segmentation performance. Our proposed data-dependent initialization approach performs favorably as compared to the Swin-UNETR model pretrained using large-scale datasets on multi-organ segmentation task. Our source code and models are available at: https://github.com/ShahinaKK/LWI-VMS.

An automatic parathyroid recognition and segmentation model based on deep learning of near-infrared autofluorescence imaging.

INTRODUCTION: Near-infrared autofluorescence imaging (NIFI) can be used to identify parathyroid gland (PG) during surgery. The purpose of the study is to establish a new model, help surgeons better identify, and protect PGs. METHODS: Five hundred and twenty three NIFI images were selected. The PGs were recorded by NIFI and marked with artificial intelligence (AI) model. The recognition rate for PGs was calculated. Analyze the differences between surgeons of different years of experience and AI recognition, and evaluate the diagnostic and therapeutic efficacy of AI model. RESULTS: Our model achieved 83.5% precision and 57.8% recall in the internal validation set. The visual recognition rate of AI model was 85.2% and 82.4% on internal and external sets. The PG recognition rate of AI model is higher than that of junior surgeons (p < 0.05). CONCLUSIONS: This AI model will help surgeons identify PGs, and develop their learning ability and self-confidence.

Modified DeeplabV3+ with multi-level context attention mechanism for colonoscopy polyp segmentation.

The development of automated methods for analyzing medical images of colon cancer is one of the main research fields. A colonoscopy is a medical treatment that enables a doctor to look for any abnormalities like polyps, cancer, or inflammatory tissue inside the colon and rectum. It falls under the category of gastrointestinal illnesses, and it claims the lives of almost two million people worldwide. Video endoscopy is an advanced medical imaging approach to diagnose gastrointestinal disorders such as inflammatory bowel, ulcerative colitis, esophagitis, and polyps. Medical video endoscopy generates several images, which must be reviewed by specialists. The difficulty of manual diagnosis has sparked research towards computer-aided techniques that can quickly and reliably diagnose all generated images. The proposed methodology establishes a framework for diagnosing coloscopy diseases. Endoscopists can lower the risk of polyps turning into cancer during colonoscopies by using more accurate computer-assisted polyp detection and segmentation. With the aim of creating a model that can automatically distinguish polyps from images, we presented a modified DeeplabV3+ model in this study to carry out segmentation tasks successfully and efficiently. The framework's encoder uses a pre-trained dilated convolutional residual network for optimal feature map resolution. The robustness of the modified model is tested against state-of-the-art segmentation approaches. In this work, we employed two publicly available datasets, CVC-Clinic DB and Kvasir-SEG, and obtained Dice similarity coefficients of 0.97 and 0.95, respectively. The results show that the improved DeeplabV3+ model improves segmentation efficiency and effectiveness in both software and hardware with only minor changes.

Advancing Barrett's Esophagus Segmentation: A Deep-Learning Ensemble Approach with Data Augmentation and Model Collaboration.

This approach provides a thorough investigation of Barrett's esophagus segmentation using deep-learning methods. This study explores various U-Net model variants with different backbone architectures, focusing on how the choice of backbone influences segmentation accuracy. By employing rigorous data augmentation techniques and ensemble strategies, the goal is to achieve precise and robust segmentation results. Key findings include the superiority of DenseNet backbones, the importance of tailored data augmentation, and the adaptability of training U-Net models from scratch. Ensemble methods are shown to enhance segmentation accuracy, and a grid search is used to fine-tune ensemble weights. A comprehensive comparison with the popular Deeplabv3+ architecture emphasizes the role of dataset characteristics. Insights into training saturation help optimize resource utilization, and efficient ensembles consistently achieve high mean intersection over union (IoU) scores, approaching 0.94. This research marks a significant advancement in Barrett's esophagus segmentation.

Pixel Diffuser: Practical Interactive Medical Image Segmentation without Ground Truth.

Medical image segmentation is essential for doctors to diagnose diseases and manage patient status. While deep learning has demonstrated potential in addressing segmentation challenges within the medical domain, obtaining a substantial amount of data with accurate ground truth for training high-performance segmentation models is both time-consuming and demands careful attention. While interactive segmentation methods can reduce the costs of acquiring segmentation labels for training supervised models, they often still necessitate considerable amounts of ground truth data. Moreover, achieving precise segmentation during the refinement phase results in increased interactions. In this work, we propose an interactive medical segmentation method called PixelDiffuser that requires no medical segmentation ground truth data and only a few clicks to obtain high-quality segmentation using a VGG19-based autoencoder. As the name suggests, PixelDiffuser starts with a small area upon the initial click and gradually detects the target segmentation region. Specifically, we segment the image by creating a distortion in the image and repeating it during the process of encoding and decoding the image through an autoencoder. Consequently, PixelDiffuser enables the user to click a part of the organ they wish to segment, allowing the segmented region to expand to nearby areas with pixel values similar to the chosen organ. To evaluate the performance of PixelDiffuser, we employed the dice score, based on the number of clicks, to compare the ground truth image with the inferred segment. For validation of our method's performance, we leveraged the BTCV dataset, containing CT images of various organs, and the CHAOS dataset, which encompasses both CT and MRI images of the liver, kidneys and spleen. Our proposed model is an efficient and effective tool for medical image segmentation, achieving competitive performance compared to previous work in less than five clicks and with very low memory consumption without additional training.

Automated evaluation of cardiac contractile dynamics and aging prediction using machine learning in a Drosophila model.

The Drosophila model has proven tremendously powerful for understanding pathophysiological bases of several human disorders including aging and cardiovascular disease. Relevant high-speed imaging and high-throughput lab assays generate large volumes of high-resolution videos, necessitating next-generation methods for rapid analysis. We present a platform for deep learning-assisted segmentation applied to optical microscopy of Drosophila hearts and the first to quantify cardiac physiological parameters during aging. An experimental test dataset is used to validate a Drosophila aging model. We then use two novel methods to predict fly aging: deep-learning video classification and machine-learning classification via cardiac parameters. Both models suggest excellent performance, with an accuracy of 83.3% (AUC 0.90) and 77.1% (AUC 0.85), respectively. Furthermore, we report beat-level dynamics for predicting the prevalence of cardiac arrhythmia. The presented approaches can expedite future cardiac assays for modeling human diseases in Drosophila and can be extended to numerous animal/human cardiac assays under multiple conditions.

Medical Image Segmentation with Learning Semantic and Global Contextual Representation.

Automatic medical image segmentation is an essential step toward accurate diseases diagnosis and designing a follow-up treatment. This assistive method facilitates the cancer detection process and provides a benchmark to highlight the affected area. The U-Net model has become the standard design choice. Although the symmetrical structure of the U-Net model enables this network to encode rich semantic representation, the intrinsic locality of the CNN layers limits this network's capability in modeling long-range contextual dependency. On the other hand, sequence to sequence Transformer models with a multi-head attention mechanism can enable them to effectively model global contextual dependency. However, the lack of low-level information stemming from the Transformer architecture limits its performance for capturing local representation. In this paper, we propose a two parallel encoder model, where in the first path the CNN module captures the local semantic representation whereas the second path deploys a Transformer module to extract the long-range contextual representation. Next, by adaptively fusing these two feature maps, we encode both representations into a single representative tensor to be further processed by the decoder block. An experimental study demonstrates that our design can provide rich and generic representation features which are highly efficient for a fine-grained semantic segmentation task.

TransCUNet: UNet cross fused transformer for medical image segmentation.

Accurate segmentation of medical images is crucial for clinical diagnosis and evaluation. However, medical images have complex shapes, the structures of different objects are very different, and most medical datasets are small in scale, making it difficult to train effectively. These problems increase the difficulty of automatic segmentation. To further improve the segmentation performance of the model, we propose a multi-branch network model, called TransCUNet, for segmenting medical images of different modalities. The model contains three structures: cross residual fusion block (CRFB), pyramidal pooling module (PPM) and gated axial-attention, which achieve effective extraction of high-level and low-level features of images, while showing high robustness to different size segmentation objects and different scale datasets. In our experiments, we use four datasets to train, validate and test the models. The experimental results show that TransCUNet has better segmentation performance compared to the current mainstream segmentation methods, and the model has a smaller size and number of parameters, which has great potential for clinical applications.

Transformer with progressive sampling for medical cellular image segmentation.

The convolutional neural network, as the backbone network for medical image segmentation, has shown good performance in the past years. However, its drawbacks cannot be ignored, namely, convolutional neural networks focus on local regions and are difficult to model global contextual information. For this reason, transformer, which is used for text processing, was introduced into the field of medical segmentation, and thanks to its expertise in modelling global relationships, the accuracy of medical segmentation was further improved. However, the transformer-based network structure requires a certain training set size to achieve satisfactory segmentation results, and most medical segmentation datasets are small in size. Therefore, in this paper we introduce a gated position-sensitive axial attention mechanism in the self-attention module, so that the transformer-based network structure can also be adapted to the case of small datasets. The common operation of the visual transformer introduced to visual processing when dealing with segmentation tasks is to divide the input image into equal patches of the same size and then perform visual processing on each patch, but this simple division may lead to the destruction of the structure of the original image, and there may be large unimportant regions in the divided grid, causing attention to stay on the uninteresting regions, affecting the segmentation performance. Therefore, in this paper, we add iterative sampling to update the sampling positions, so that the attention stays on the region to be segmented, reducing the interference of irrelevant regions and further improving the segmentation performance. In addition, we introduce the strip convolution module (SCM) and pyramid pooling module (PPM) to capture the global contextual information. The proposed network is evaluated on several datasets and shows some improvement in segmentation accuracy compared to networks of recent years.

Tumor type detection in brain MR images of the deep model developed using hypercolumn technique, attention modules, and residual blocks.

Brain cancer is a disease caused by the growth of abnormal aggressive cells in the brain outside of normal cells. Symptoms and diagnosis of brain cancer cases are producing more accurate results day by day in parallel with the development of technological opportunities. In this study, a deep learning model called BrainMRNet which is developed for mass detection in open-source brain magnetic resonance images was used. The BrainMRNet model includes three processing steps: attention modules, the hypercolumn technique, and residual blocks. To demonstrate the accuracy of the proposed model, three types of tumor data leading to brain cancer were examined in this study: glioma, meningioma, and pituitary. In addition, a segmentation method was proposed, which additionally determines in which lobe area of the brain the two classes of tumors that cause brain cancer are more concentrated. The classification accuracy rates were performed in the study; it was 98.18% in glioma tumor, 96.73% in meningioma tumor, and 98.18% in pituitary tumor. At the end of the experiment, using the subset of glioma and meningioma tumor images, it was determined which at brain lobe the tumor region was seen, and 100% success was achieved in the analysis of this determination. In this study, a hybrid deep learning model is presented to determine the detection of the brain tumor. In addition, open-source software was proposed, which statistically found in which lobe region of the human brain the brain tumor occurred. The methods applied and tested in the experiments have shown promising results with a high level of accuracy, precision, and specificity. These results demonstrate the availability of the proposed approach in clinical settings to support the medical decision regarding brain tumor detection.

Two methods for modifed Doo-Sabin modeling of nonsmooth surfaces-applied to right ventricle modeling.

Purpose: In recent years, there has been increased clinical interest in the right ventricle (RV) of the heart. RV dysfunction is an important prognostic marker for several cardiac diseases. Accurate modeling of the RV shape is important for estimating the performance. We have created computationally effective models that allow for accurate estimation of the RV shape. Approach: Previous approaches to cardiac shape modeling, including modeling the RV geometry, has used Doo-Sabin surfaces. Doo-Sabin surfaces allow effective computation and adapt to smooth, organic surfaces. However, they struggle with modeling sharp corners or ridges without many control nodes. We modified the Doo-Sabin surface to allow for sharpness using weighting of vertices and edges instead. This was done in two different ways. For validation, we compared the standard Doo-Sabin versus the sharp Doo-Sabin models in modeling the RV shape of 16 cardiac ultrasound images, against a ground truth manually drawn by a cardiologist. A Kalman filter fitted the models to the ultrasound images, and the difference between the volume of the model and the ground truth was measured. Results: The two modified Doo-Sabin models both outperformed the standard Doo-Sabin model in modeling the RV. On average, the regular Doo-Sabin had an 8-ml error in volume, whereas the sharp models had 7- and 6-ml error, respectively. Conclusions: Compared with the standard Doo-Sabin, the modified Doo-Sabin models can adapt to a larger variety of surfaces while still being compact models. They were more accurate on modeling the RV shape and could have uses elsewhere.