A transformer-guided cross-modality adaptive feature fusion framework for esophageal gross tumor volume segmentation.

BACKGROUND AND OBJECTIVE: Accurate segmentation of esophageal gross tumor volume (GTV) indirectly enhances the efficacy of radiotherapy for patients with esophagus cancer. In this domain, learning-based methods have been employed to fuse cross-modality positron emission tomography (PET) and computed tomography (CT) images, aiming to improve segmentation accuracy. This fusion is essential as it combines functional metabolic information from PET with anatomical information from CT, providing complementary information. While the existing three-dimensional (3D) segmentation method has achieved state-of-the-art (SOTA) performance, it typically relies on pure-convolution architectures, limiting its ability to capture long-range spatial dependencies due to convolution's confinement to a local receptive field. To address this limitation and further enhance esophageal GTV segmentation performance, this work proposes a transformer-guided cross-modality adaptive feature fusion network, referred to as TransAttPSNN, which is based on cross-modality PET/CT scans. METHODS: Specifically, we establish an attention progressive semantically-nested network (AttPSNN) by incorporating the convolutional attention mechanism into the progressive semantically-nested network (PSNN). Subsequently, we devise a plug-and-play transformer-guided cross-modality adaptive feature fusion model, which is inserted between the multi-scale feature counterparts of a two-stream AttPSNN backbone (one for the PET modality flow and another for the CT modality flow), resulting in the proposed TransAttPSNN architecture. RESULTS: Through extensive four-fold cross-validation experiments on the clinical PET/CT cohort. The proposed approach acquires a Dice similarity coefficient (DSC) of 0.76 ± 0.13, a Hausdorff distance (HD) of 9.38 ± 8.76 mm, and a Mean surface distance (MSD) of 1.13 ± 0.94 mm, outperforming the SOTA competing methods. The qualitative results show a satisfying consistency with the lesion areas. CONCLUSIONS: The devised transformer-guided cross-modality adaptive feature fusion module integrates the strengths of PET and CT, effectively enhancing the segmentation performance of esophageal GTV. The proposed TransAttPSNN has further advanced the research of esophageal GTV segmentation.

ULM-MbCNRT: In vivo Ultrafast Ultrasound Localization Microscopy by Combining Multi-branch CNN and Recursive Transformer.

Ultrasound localization microscopy (ULM) overcomes the acoustic diffraction limit by localizing tiny microbubbles (MBs), thus enabling the microvascular to be rendered at sub-wavelength resolution. Nevertheless, to obtain such superior spatial resolution, it is necessary to spend tens of seconds gathering numerous ultrasound (US) frames to accumulate MB events required, resulting in ULM imaging still suffering from trade-offs between imaging quality, data acquisition time and data processing speed. In this paper, we present a new deep learning (DL) framework combining multi-branch CNN and recursive Transformer, termed as ULM-MbCNRT, that is capable of reconstructing a super-resolution image directly from a temporal mean low-resolution image generated by averaging much fewer raw US frames, i.e., implement an ultrafast ULM imaging. To evaluate the performance of ULM-MbCNRT, a series of numerical simulations and in vivo experiments are carried out. Numerical simulation results indicate that ULM-MbCNRT achieves high-quality ULM imaging with ~10-fold reduction in data acquisition time and ~130-fold reduction in computation time compared to the previous DL method (e.g., the modified sub-pixel convolutional neural network, ULM-mSPCN). For the in vivo experiments, when comparing to the ULM-mSPCN, ULM-MbCNRT allows ~37-fold reduction in data acquisition time (~0.8 s) and ~2134-fold reduction in computation time (~0.87 s) without sacrificing spatial resolution. It implies that ultrafast ULM imaging holds promise for observing rapid biological activity in vivo, potentially improving the diagnosis and monitoring of clinical conditions.

Condition control training-based ConVMLP-ResU-Net for semantic segmentation of esophageal cancer in 18F-FDG PET/CT images.

The precise delineation of esophageal gross tumor volume (GTV) on medical images can promote the radiotherapy effect of esophagus cancer. This work is intended to explore effective learning-based methods to tackle the challenging auto-segmentation problem of esophageal GTV. By employing the progressive hierarchical reasoning mechanism (PHRM), we devised a simple yet effective two-stage deep framework, ConVMLP-ResU-Net. Thereinto, the front-end ConVMLP integrates convolution (ConV) and multi-layer perceptrons (MLP) to capture localized and long-range spatial information, thus making ConVMLP excel in the location and coarse shape prediction of esophageal GTV. According to the PHRM, the front-end ConVMLP should have a strong generalization ability to ensure that the back-end ResU-Net has correct and valid reasoning. Therefore, a condition control training algorithm was proposed to control the training process of ConVMLP for a robust front end. Afterward, the back-end ResU-Net benefits from the yielded mask by ConVMLP to conduct a finer expansive segmentation to output the final result. Extensive experiments were carried out on a clinical cohort, which included 1138 pairs of 18F-FDG positron emission tomography/computed tomography (PET/CT) images. We report the Dice similarity coefficient, Hausdorff distance, and Mean surface distance as 0.82 ± 0.13, 4.31 ± 7.91 mm, and 1.42 ± 3.69 mm, respectively. The predicted contours visually have good agreements with the ground truths. The devised ConVMLP is apt at locating the esophageal GTV with correct initial shape prediction and hence facilitates the finer segmentation of the back-end ResU-Net. Both the qualitative and quantitative results validate the effectiveness of the proposed method.

Automatic segmentation of esophageal gross tumor volume in 18F-FDG PET/CT images via GloD-LoATUNet.

BACKGROUND AND OBJECTIVE: For esophageal squamous cell carcinoma, radiotherapy is one of the primary treatments. During the planning before radiotherapy, the intractable task is to precisely delineate the esophageal gross tumor volume (GTV) on medical images. In current clinical practice, the manual delineation suffers from high intra- and inter-rater variability, while also exhausting the oncologists on a treadmill. There is an urgent demand for effective computer-aided automatic segmentation methods. To this end, we designed a novel deep network, dubbed as GloD-LoATUNet. METHODS: GloD-LoATUNet follows the effective U-shape structure. On the contractile path, the global deformable dense attention transformer (GloDAT), local attention transformer (LoAT), and convolution blocks are integrated to model long-range dependencies and localized information. On the center bridge and the expanding path, convolution blocks are adopted to upsample the extracted representations for pixel-wise semantic prediction. Between the peer-to-peer counterparts, enhanced skip connections are built to compensate for the lost spatial information and dependencies. By exploiting complementary strengths of the GloDAT, LoAT, and convolution, GloD-LoATUNet has remarkable representation learning capabilities, performing well in the prediction of the small and variable esophageal GTV. RESULTS: The proposed approach was validated in the clinical positron emission tomography/computed tomography (PET/CT) cohort. For 4 different data partitions, we report the Dice similarity coefficient (DSC), Hausdorff distance (HD), and Mean surface distance (MSD) as: 0.83±0.13, 4.88±9.16 mm, and 1.40±4.11 mm; 0.84±0.12, 6.89±12.04 mm, and 1.18±3.02 mm; 0.84±0.13, 3.89±7.64 mm, and 1.28±3.68 mm; 0.86±0.09, 3.71±4.79 mm, and 0.90±0.37 mm; respectively. The predicted contours present a desirable consistency with the ground truth. CONCLUSIONS: The inspiring results confirm the accuracy and generalizability of the proposed model, demonstrating the potential for automatic segmentation of esophageal GTV in clinical practice.

CM-SegNet: A deep learning-based automatic segmentation approach for medical images by combining convolution and multilayer perceptron.

Accurate segmentation of lesions in medical images is of great significance for clinical diagnosis and evaluation. The low contrast between lesions and surrounding tissues increases the difficulty of automatic segmentation, while the efficiency of manual segmentation is low. In order to increase the generalization performance of segmentation model, we proposed a deep learning-based automatic segmentation model called CM-SegNet for segmenting medical images of different modalities. It was designed according to the multiscale input and encoding-decoding thoughts, and composed of multilayer perceptron and convolution modules. This model achieved communication of different channels and different spatial locations of each patch, and considers the edge related feature information between adjacent patches. Thus, it could fully extract global and local image information for the segmentation task. Meanwhile, this model met the effective segmentation of different structural lesion regions in different slices of three-dimensional medical images. In this experiment, the proposed CM-SegNet was trained, validated, and tested using six medical image datasets of different modalities and 5-fold cross validation method. The results showed that the CM-SegNet model had better segmentation performance and shorter training time for different medical images than the previous methods, suggesting it is faster and more accurate in automatic segmentation and has great potential application in clinic.

Feasibility of a Generative Adversarial Network for Artifact Removal in Experimental Photoacoustic Imaging.

Photoacoustic tomography (PAT) reconstruction is an expeditiously growing interest among biomedical researchers because of its possible transition from laboratory to clinical pre-eminence. Nonetheless, the PAT inverse problem is yet to achieve an optimal solution in rapid and precise reconstruction under practical constraints. Precisely, the sparse sampling problem and random noise are the main impediments to attaining accuracy but in support of rapid PAT reconstruction. The limitations are associated with acquiring undersampled artifacts that deteriorate the optimality of the reconstruction task. Therefore, the former achievements of fast image formation limit the modality for clinical settings. Delving into the problem, here we explore a deep learning-based generative adversarial network (GAN) to improve the image quality by denoising and removing these artifacts. The specially designed attributes and unique manner of optimizing the problem, such as incorporating the data set limitations and providing stable training performance, constitute the main motivation behind the employment of GAN. Moreover, exploitation of the U-net variant as a generator network offers robust performance in terms of quality and computational cost, which is further validated with the detailed quantitative and qualitative analysis. The quantitatively evaluated structured similarity indexing method = 0.980 ± 0.043 and peak signal-to-noise ratio = 31 ± 0.002 dB state that the proposed solution provides the high-resolution image at the output, even training with a low-quality data set.

Gross Tumor Volume Definition and Comparative Assessment for Esophageal Squamous Cell Carcinoma From 3D 18F-FDG PET/CT by Deep Learning-Based Method.

Background: The accurate definition of gross tumor volume (GTV) of esophageal squamous cell carcinoma (ESCC) can promote precise irradiation field determination, and further achieve the radiotherapy curative effect. This retrospective study is intended to assess the applicability of leveraging deep learning-based method to automatically define the GTV from 3D 18F-FDG PET/CT images of patients diagnosed with ESCC. Methods: We perform experiments on a clinical cohort with 164 18F-FDG PET/CT scans. The state-of-the-art esophageal GTV segmentation deep neural net is first employed to delineate the lesion area on PET/CT images. Afterwards, we propose a novel equivalent truncated elliptical cone integral method (ETECIM) to estimate the GTV value. Indexes of Dice similarity coefficient (DSC), Hausdorff distance (HD), and mean surface distance (MSD) are used to evaluate the segmentation performance. Conformity index (CI), degree of inclusion (DI), and motion vector (MV) are used to assess the differences between predicted and ground truth tumors. Statistical differences in the GTV, DI, and position are also determined. Results: We perform 4-fold cross-validation for evaluation, reporting the values of DSC, HD, and MSD as 0.72 ± 0.02, 11.87 ± 4.20 mm, and 2.43 ± 0.60 mm (mean ± standard deviation), respectively. Pearson correlations (R2) achieve 0.8434, 0.8004, 0.9239, and 0.7119 for each fold cross-validation, and there is no significant difference (t = 1.193, p = 0.235) between the predicted and ground truth GTVs. For DI, a significant difference is found (t = -2.263, p = 0.009). For position assessment, there is no significant difference (left-right in x direction: t = 0.102, p = 0.919, anterior-posterior in y direction: t = 0.221, p = 0.826, and cranial-caudal in z direction: t = 0.569, p = 0.570) between the predicted and ground truth GTVs. The median of CI is 0.63, and the gotten MV is small. Conclusions: The predicted tumors correspond well with the manual ground truth. The proposed GTV estimation approach ETECIM is more precise than the most commonly used voxel volume summation method. The ground truth GTVs can be solved out due to the good linear correlation with the predicted results. Deep learning-based method shows its promising in GTV definition and clinical radiotherapy application.

G-DipC: An Improved Feature Representation Method for Short Sequences to Predict the Type of Cargo in Cell-Penetrating Peptides.

Cell-penetrating peptides (CPPs) are functional short peptides with high carrying capacity. CPP sequences with targeting functions for the highly efficient delivery of drugs to target cells. In this paper, which is focused on the prediction of the cargo category of CPPs, a biocomputational model is constructed to efficiently distinguish the category of cargo carried by CPPs as macromolecular carriers among the seven known deliverable cargo categories. Based on dipeptide composition (DipC), an improved feature representation method, general dipeptide composition (G-DipC) is proposed for short peptide sequences and can effectively increase the abundance of features represented. Then linear discriminant analysis (LDA) is applied to mine some important low-dimensional features of G-DipC and a predictive model is built with the XGBoost algorithm. Experimental results with five-fold cross validation show that G-DipC improves accuracy by 25 and 5 percent compared with amino acid composition (AAC) and DipC, respectively. G-DipC is even found to be better than tripeptide composition (TipC). Thus, the proposed model provides a novel resource for the study of cell-penetrating peptides, and the improved dipeptide composition G-DipC can be widely adapted to determine the feature representation of other biological sequences.

Protein subnuclear localization based on a new effective representation and intelligent kernel linear discriminant analysis by dichotomous greedy genetic algorithm.

A wide variety of methods have been proposed in protein subnuclear localization to improve the prediction accuracy. However, one important trend of these means is to treat fusion representation by fusing multiple feature representations, of which, the fusion process takes a lot of time. In view of this, this paper novelly proposed a method by combining a new single feature representation and a new algorithm to obtain good recognition rate. Specifically, based on the position-specific scoring matrix (PSSM), we proposed a new expression, correlation position-specific scoring matrix (CoPSSM) as the protein feature representation. Based on the classic nonlinear dimension reduction algorithm, kernel linear discriminant analysis (KLDA), we added a new discriminant criterion and proposed a dichotomous greedy genetic algorithm (DGGA) to intelligently select its kernel bandwidth parameter. Two public datasets with Jackknife test and KNN classifier were used for the numerical experiments. The results showed that the overall success rate (OSR) with single representation CoPSSM is larger than that with many relevant representations. The OSR of the proposed method can reach as high as 87.444% and 90.3361% for these two datasets, respectively, outperforming many current methods. To show the generalization of the proposed algorithm, two extra standard datasets of protein subcellular were chosen to conduct the expending experiment, and the prediction accuracy by Jackknife test and Independent test is still considerable.