Evaluation of Breast Cancer Tumor-Infiltrating Lymphocytes on Ultrasound Images Based on a Novel Multi-Cascade Residual U-Shaped Network.

OBJECTIVE: Breast cancer has become the leading cancer of the 21st century. Tumor-infiltrating lymphocytes (TILs) have emerged as effective biomarkers for predicting treatment response and prognosis in breast cancer. The work described here was aimed at designing a novel deep learning network to assess the levels of TILs in breast ultrasound images. METHODS: We propose the Multi-Cascade Residual U-Shaped Network (MCRUNet), which incorporates a gray feature enhancement (GFE) module for image reconstruction and normalization to achieve data synergy. Additionally, multiple residual U-shaped (RSU) modules are cascaded as the backbone network to maximize the fusion of global and local features, with a focus on the tumor's location and surrounding regions. The development of MCRUNet is based on data from two hospitals and uses a publicly available ultrasound data set for transfer learning. RESULTS: MCRUNet exhibits excellent performance in assessing TILs levels, achieving an area under the receiver operating characteristic curve of 0.8931, an accuracy of 85.71%, a sensitivity of 83.33%, a specificity of 88.64% and an F1 score of 86.54% in the test group. It outperforms six state-of-the-art networks in terms of performance. CONCLUSION: The MCRUNet network based on breast ultrasound images of breast cancer patients holds promise for non-invasively predicting TILs levels and aiding personalized treatment decisions.

Deep Learning with Transformer or Convolutional Neural Network in the Assessment of Tumor-Infiltrating Lymphocytes (TILs) in Breast Cancer Based on US Images: A Dual-Center Retrospective Study.

This study aimed to explore the feasibility of using a deep-learning (DL) approach to predict TIL levels in breast cancer (BC) from ultrasound (US) images. A total of 494 breast cancer patients with pathologically confirmed invasive BC from two hospitals were retrospectively enrolled. Of these, 396 patients from hospital 1 were divided into the training cohort (n = 298) and internal validation (IV) cohort (n = 98). Patients from hospital 2 (n = 98) were in the external validation (EV) cohort. TIL levels were confirmed by pathological results. Five different DL models were trained for predicting TIL levels in BC using US images from the training cohort and validated on the IV and EV cohorts. The overall best-performing DL model, the attention-based DenseNet121, achieved an AUC of 0.873, an accuracy of 79.5%, a sensitivity of 90.7%, a specificity of 65.9%, and an F1 score of 0.830 in the EV cohort. In addition, the stratified analysis showed that the DL models had good discrimination performance of TIL levels in each of the molecular subgroups. The DL models based on US images of BC patients hold promise for non-invasively predicting TIL levels and helping with individualized treatment decision-making.

ATFE-Net: Axial Transformer and Feature Enhancement-based CNN for ultrasound breast mass segmentation.

Breast mass is one of the main clinical symptoms of breast cancer. Recently, many CNN-based methods for breast mass segmentation have been proposed. However, these methods have difficulties in capturing long-range dependencies, causing poor segmentation of large-scale breast masses. In this paper, we propose an axial Transformer and feature enhancement-based CNN (ATFE-Net) for ultrasound breast mass segmentation. Specially, an axial Transformer (Axial-Trans) module and a Transformer-based feature enhancement (Trans-FE) module are proposed to capture long-range dependencies. Axial-Trans module only calculates self-attention in width and height directions of input feature maps, which reduces the complexity of self-attention significantly from O(n2) to O(n). In addition, Trans-FE module can enhance feature representation by capturing dependencies between different feature layers, since deeper feature layers have richer semantic information and shallower feature layers have more detailed information. The experimental results show that our ATFE-Net achieved better performance than several state-of-the-art methods on two publicly available breast ultrasound datasets, with Dice coefficient of 82.46% for BUSI and 86.78% for UDIAT, respectively.

Learning multi-frequency features in convolutional network for mammography classification.

Breast cancer is a common life-threatening disease among women. Computer-aided methods can provide second opinion or decision support for early diagnosis in mammography images. However, the whole images classification is highly challenging due to small sizes of lesion and slow contrast between lesions and fibro-glandular tissue. In this paper, inspired by conventional machine learning methods, we present a Multi Frequency Attention Network (MFA-Net) to highlight the salient features. The network decomposes the features into low spatial frequency components and high spatial frequency components, and then recalibrates discriminating features based on two-dimensional Discrete Cosine Transform in two different frequency parts separately. Low spatial frequency features help determine if there is a tumor while high spatial frequency features help focus more on the margin of the tumor. Our studies empirically show that compared to traditional convolutional neural network (CNN), the proposed method mitigates the influence of the margin of pectoral muscle and breast in mammography, which brings significant improvement. For malignant and benign classification, by using transfer learning, the proposed MFA-Net achieves the AUC index 91.71% on the INbreast dataset.

FS-UNet: Mass segmentation in mammograms using an encoder-decoder architecture with feature strengthening.

Breast mass segmentation in mammograms is still a challenging and clinically valuable task. In this paper, we propose an effective and lightweight segmentation model based on convolutional neural networks to automatically segment breast masses in whole mammograms. Specifically, we first developed feature strengthening modules to enhance relevant information about masses and other tissues and improve the representation power of low-resolution feature layers with high-resolution feature maps. Second, we applied a parallel dilated convolution module to capture the features of different scales of masses and fully extract information about the edges and internal texture of the masses. Third, a mutual information loss function was employed to optimise the accuracy of the prediction results by maximising the mutual information between the prediction results and the ground truth. Finally, the proposed model was evaluated on both available INbreast and CBIS-DDSM datasets, and the experimental results indicated that our method achieved excellent segmentation performance in terms of dice coefficient, intersection over union, and sensitivity metrics.

DCANet: Dual contextual affinity network for mass segmentation in whole mammograms.

PURPOSE: Breast mass segmentation in mammograms remains a crucial yet challenging topic in computer-aided diagnosis systems. Existing algorithms mainly used mass-centered patches to achieve mass segmentation, which is time-consuming and unstable in clinical diagnosis. Therefore, we aim to directly perform fully automated mass segmentation in whole mammograms with deep learning solutions. METHODS: In this work, we propose a novel dual contextual affinity network (a.k.a., DCANet) for mass segmentation in whole mammograms. Based on the encoder-decoder structure, two lightweight yet effective contextual affinity modules including the global-guided affinity module (GAM) and the local-guided affinity module (LAM) are proposed. The former aggregates the features integrated by all positions and captures long-range contextual dependencies, aiming to enhance the feature representations of homogeneous regions. The latter emphasizes semantic information around each position and exploits contextual affinity based on the local field-of-view, aiming to improve the indistinction among heterogeneous regions. RESULTS: The proposed DCANet is greatly demonstrated on two public mammographic databases including the DDSM and the INbreast, achieving the Dice similarity coefficient (DSC) of 85.95% and 84.65%, respectively. Both segmentation performance and computational efficiency outperform the current state-of-the-art methods. CONCLUSION: According to extensive qualitative and quantitative analyses, we believe that the proposed fully automated approach has sufficient robustness to provide fast and accurate diagnoses for possible clinical breast mass segmentation.

An automatic segmentation method of a parameter-adaptive PCNN for medical images.

PURPOSE: Since pre-processing and initial segmentation steps in medical images directly affect the final segmentation results of the regions of interesting, an automatic segmentation method of a parameter-adaptive pulse-coupled neural network is proposed to integrate the above-mentioned two segmentation steps into one. This method has a low computational complexity for different kinds of medical images and has a high segmentation precision. METHODS: The method comprises four steps. Firstly, an optimal histogram threshold is used to determine the parameter [Formula: see text] for different kinds of images. Secondly, we acquire the parameter [Formula: see text] according to a simplified pulse-coupled neural network (SPCNN). Thirdly, we redefine the parameter V of the SPCNN model by sub-intensity distribution range of firing pixels. Fourthly, we add an offset [Formula: see text] to improve initial segmentation precision. RESULTS: Compared with the state-of-the-art algorithms, the new method achieves a comparable performance by the experimental results from ultrasound images of the gallbladder and gallstones, magnetic resonance images of the left ventricle, and mammogram images of the left and the right breast, presenting the overall metric UM of 0.9845, CM of 0.8142, TM of 0.0726. CONCLUSION: The algorithm has a great potential to achieve the pre-processing and initial segmentation steps in various medical images. This is a premise for assisting physicians to detect and diagnose clinical cases.

Automatic gallbladder and gallstone regions segmentation in ultrasound image.

PURPOSE: As gallbladder diseases including gallstone and cholecystitis are mainly diagnosed by using ultra-sonographic examinations, we propose a novel method to segment the gallbladder and gallstones in ultrasound images. METHODS: The method is divided into five steps. Firstly, a modified Otsu algorithm is combined with the anisotropic diffusion to reduce speckle noise and enhance image contrast. The Otsu algorithm separates distinctly the weak edge regions from the central region of the gallbladder. Secondly, a global morphology filtering algorithm is adopted for acquiring the fine gallbladder region. Thirdly, a parameter-adaptive pulse-coupled neural network (PA-PCNN) is employed to obtain the high-intensity regions including gallstones. Fourthly, a modified region-growing algorithm is used to eliminate physicians' labeled regions and avoid over-segmentation of gallstones. It also has good self-adaptability within the growth cycle in light of the specified growing and terminating conditions. Fifthly, the smoothing contours of the detected gallbladder and gallstones are obtained by the locally weighted regression smoothing (LOESS). RESULTS: We test the proposed method on the clinical data from Gansu Provincial Hospital of China and obtain encouraging results. For the gallbladder and gallstones, average similarity percent of contours (EVA) containing metrics dice's similarity , overlap fraction and overlap value is 86.01 and 79.81%, respectively; position error is 1.7675 and 0.5414 mm, respectively; runtime is 4.2211 and 0.6603 s, respectively. Our method then achieves competitive performance compared with the state-of-the-art methods. CONCLUSIONS: The proposed method is potential to assist physicians for diagnosing the gallbladder disease rapidly and effectively.

An SPCNN-GVF-based approach for the automatic segmentation of left ventricle in cardiac cine MR images.

PURPOSE: Accurate segmentation of left ventricle (LV) is essential for the cardiac function analysis. However, it is labor intensive and time consuming for radiologists to delineate LV boundary manually. In this paper, we present a novel self-correcting framework for the fully automatic LV segmentation. METHODS: Firstly, a time-domain method is designed to extract a rectangular region of interest around the heart. Then, the simplified pulse-coupled neural network (SPCNN) is employed to locate the LV cavity. Different from the existing approaches, SPCNN can realize the self-correcting segmentation due to its parameter controllability. Subsequently, the post-processing based on the maximum gradient searching is proposed to obtain the accurate endocardium. Finally, a new external force based on the shape similarity is defined and integrated into the gradient vector flow (GVF) snake with the balloon force to segment the epicardium. RESULTS: We obtain encouraging segmentation results tested on the database provided by MICCAI 2009. The average percentage of good contours is 92.26 %, the average perpendicular distance is 2.38 mm, and the overlapping dice metric is 0.89. Besides, the experiment results show good correlations between the automatic segmentation and the manual delineation (for the LV ejection fraction and the LV myocardial mass, the correlation coefficients R are 0.9683 and 0.9278, respectively). CONCLUSION: We propose an effective and fast method combing the SPCNN and the improved GVF for the automatic segmentation of LV.

A new method of detecting micro-calcification clusters in mammograms using contourlet transform and non-linking simplified PCNN.

BACKGROUND AND OBJECTIVES: Mammography analysis is an effective technology for early detection of breast cancer. Micro-calcification clusters (MCs) are a vital indicator of breast cancer, so detection of MCs plays an important role in computer aided detection (CAD) system, this paper proposes a new hybrid method to improve MCs detection rate in mammograms. METHODS: The proposed method comprises three main steps: firstly, remove label and pectoral muscle adopting the largest connected region marking and region growing method, and enhance MCs using the combination of double top-hat transform and grayscale-adjustment function; secondly, remove noise and other interference information, and retain the significant information by modifying the contourlet coefficients using nonlinear function; thirdly, we use the non-linking simplified pulse-coupled neural network to detect MCs. RESULTS: In our work, we choose 118 mammograms including 38 mammograms with micro-calcification clusters and 80 mammograms without micro-calcification to demonstrate our algorithm separately from two open and common database including the MIAS and JSMIT; and we achieve the higher specificity of 94.7%, sensitivity of 96.3%, AUC of 97.0%, accuracy of 95.8%, MCC of 90.4%, MCC-PS of 61.3% and CEI of 53.5%, these promising results clearly demonstrate that the proposed approach outperforms the current state-of-the-art algorithms. In addition, this method is verified on the 20 mammograms from the People's Hospital of Gansu Province, the detection results reveal that our method can accurately detect the calcifications in clinical application. CONCLUSIONS: This proposed method is simple and fast, furthermore it can achieve high detection rate, it could be considered used in CAD systems to assist the physicians for breast cancer diagnosis in the future.

An Efficient Approach for Automated Mass Segmentation and Classification in Mammograms.

Breast cancer is becoming a leading death of women all over the world; clinical experiments demonstrate that early detection and accurate diagnosis can increase the potential of treatment. In order to improve the breast cancer diagnosis precision, this paper presents a novel automated segmentation and classification method for mammograms. We conduct the experiment on both DDSM database and MIAS database, firstly extract the region of interests (ROIs) with chain codes and using the rough set (RS) method to enhance the ROIs, secondly segment the mass region from the location ROIs with an improved vector field convolution (VFC) snake and following extract features from the mass region and its surroundings, and then establish features database with 32 dimensions; finally, these features are used as input to several classification techniques. In our work, the random forest is used and compared with support vector machine (SVM), genetic algorithm support vector machine (GA-SVM), particle swarm optimization support vector machine (PSO-SVM), and decision tree. The effectiveness of our method is evaluated by a comprehensive and objective evaluation system; also, Matthew's correlation coefficient (MCC) indicator is used. Among the state-of-the-art classifiers, our method achieves the best performance with best accuracy of 97.73%, and the MCC value reaches 0.8668 and 0.8652 in unique DDSM database and both two databases, respectively. Experimental results prove that the proposed method outperforms the other methods; it could consider applying in CAD systems to assist the physicians for breast cancer diagnosis.