Chest x-ray diagnosis via spatial-channel high-order attention representation learning.

Objective. Chest x-ray image representation and learning is an important problem in computer-aided diagnostic area. Existing methods usually adopt CNN or Transformers for feature representation learning and focus on learning effective representations for chest x-ray images. Although good performance can be obtained, however, these works are still limited mainly due to the ignorance of mining the correlations of channels and pay little attention on the local context-aware feature representation of chest x-ray image.Approach. To address these problems, in this paper, we propose a novel spatial-channel high-order attention model (SCHA) for chest x-ray image representation and diagnosis. The proposed network architecture mainly contains three modules, i.e. CEBN, SHAM and CHAM. To be specific, firstly, we introduce a context-enhanced backbone network by employing multi-head self-attention to extract initial features for the input chest x-ray images. Then, we develop a novel SCHA which contains both spatial and channel high-order attention learning branches. For the spatial branch, we develop a novel local biased self-attention mechanism which can capture both local and long-range global dependences of positions to learn rich context-aware representation. For the channel branch, we employ Brownian Distance Covariance to encode the correlation information of channels and regard it as the image representation. Finally, the two learning branches are integrated together for the final multi-label diagnosis classification and prediction.Main results. Experiments on the commonly used datasets including ChestX-ray14 and CheXpert demonstrate that our proposed SCHA approach can obtain better performance when comparing many related approaches.Significance. This study obtains a more discriminative method for chest x-ray classification and provides a technique for computer-aided diagnosis.

A sonogram radiomics model for differentiating granulomatous lobular mastitis from invasive breast cancer: a multicenter study.

PURPOSE: To construct a nomogram based on sonogram features and radiomics features to differentiate granulomatous lobular mastitis (GLM) from invasive breast cancer (IBC). MATERIALS AND METHODS: A retrospective collection of 213 GLMs and 472 IBCs from three centers was divided into a training set, an internal validation set, and an external validation set. A radiomics model was built based on radiomics features, and the RAD score of the lesion was calculated. The sonogram radiomics model was constructed using ultrasound features and RAD scores. Finally, the diagnostic efficacy of the three sonographers with different levels of experience before and after combining the RAD score was assessed in the external validation set. RESULTS: The RAD score, lesion diameter, orientation, echogenicity, and tubular extension showed significant differences in GLM and IBC (p < 0.05). The sonogram radiomics model based on these factors achieved optimal performance, and its area under the curve (AUC) was 0.907, 0.872, and 0.888 in the training, internal, and external validation sets, respectively. The AUCs before and after combining the RAD scores were 0.714, 0.750, and 0.830 and 0.834, 0.853, and 0.878, respectively, for sonographers with different levels of experience. The diagnostic efficacy was comparable for all sonographers when combined with the RAD score (p > 0.05). CONCLUSION: Radiomics features effectively enhance the ability of sonographers to discriminate between GLM and IBC and reduce interobserver variation. The nomogram combining ultrasound features and radiomics features show promising diagnostic efficacy and can be used to identify GLM and IBC in a noninvasive approach.

Contrast-Enhanced Ultrasound with Deep Learning with Attention Mechanisms for Predicting Microvascular Invasion in Single Hepatocellular Carcinoma.

RATIONALE AND OBJECTIVES: Prediction of microvascular invasion (MVI) status of hepatocellular carcinoma (HCC) holds clinical significance for decision-making regarding the treatment strategy and evaluation of patient prognosis. We developed a deep learning (DL) model based on contrast-enhanced ultrasound (CEUS) to predict MVI of HCC. MATERIALS AND METHODS: We retrospectively analyzed the data for single primary HCCs that were evaluated with CEUS 1 week before surgical resection from December 2014 to February 2022. The study population was divided into training (n = 198) and test (n = 54) cohorts. In this study, three DL models (Resnet50, Resnet50+BAM, Resnet50+SE) were trained using the training cohort and tested in the test cohort. Tumor characteristics were also evaluated by radiologists, and multivariate regression analysis was performed to determine independent indicators for the development of predictive nomogram models. The performance of the three DL models was compared to that of the MVI prediction model based on radiologist evaluations. RESULTS: The best-performing model, ResNet50+SE model achieved the ROC of 0.856, accuracy of 77.2, specificity of 93.9%, and sensitivity of 52.4% in the test group. The MVI prediction model based on a combination of three independent predictors showed a C-index of 0.729, accuracy of 69.4, specificity of 73.8%, and sensitivity of 62%. CONCLUSION: The DL algorithm can accurately predict MVI of HCC on the basis of CEUS images, to help identify high-risk patients for the assist treatment.

Weakly Alignment-Free RGBT Salient Object Detection With Deep Correlation Network.

RGBT Salient Object Detection (SOD) focuses on common salient regions of a pair of visible and thermal infrared images. Existing methods perform on the well-aligned RGBT image pairs, but the captured image pairs are always unaligned and aligning them requires much labor cost. To handle this problem, we propose a novel deep correlation network (DCNet), which explores the correlations across RGB and thermal modalities, for weakly alignment-free RGBT SOD. In particular, DCNet includes a modality alignment module based on the spatial affine transformation, the feature-wise affine transformation and the dynamic convolution to model the strong correlation of two modalities. Moreover, we propose a novel bi-directional decoder model, which combines the coarse-to-fine and fine-to-coarse processes for better feature enhancement. In particular, we design a modality correlation ConvLSTM by adding the first two components of modality alignment module and a global context reinforcement module into ConvLSTM, which is used to decode hierarchical features in both top-down and button-up manners. Extensive experiments on three public benchmark datasets show the remarkable performance of our method against state-of-the-art methods.

M5L: Multi-Modal Multi-Margin Metric Learning for RGBT Tracking.

Classifying hard samples in the course of RGBT tracking is a quite challenging problem. Existing methods only focus on enlarging the boundary between positive and negative samples, but ignore the relations of multilevel hard samples, which are crucial for the robustness of hard sample classification. To handle this problem, we propose a novel Multi-Modal Multi-Margin Metric Learning framework named M5L for RGBT tracking. In particular, we divided all samples into four parts including normal positive, normal negative, hard positive and hard negative ones, and aim to leverage their relations to improve the robustness of feature embeddings, e.g., normal positive samples are closer to the ground truth than hard positive ones. To this end, we design a multi-modal multi-margin structural loss to preserve the relations of multilevel hard samples in the training stage. In addition, we introduce an attention-based fusion module to achieve quality-aware integration of different source data. Extensive experiments on large-scale datasets testify that our framework clearly improves the tracking performance and performs favorably the state-of-the-art RGBT trackers.

Multi-Interactive Dual-Decoder for RGB-Thermal Salient Object Detection.

RGB-thermal salient object detection (SOD) aims to segment the common prominent regions of visible image and corresponding thermal infrared image that we call it RGBT SOD. Existing methods don't fully explore and exploit the potentials of complementarity of different modalities and multi-type cues of image contents, which play a vital role in achieving accurate results. In this paper, we propose a multi-interactive dual-decoder to mine and model the multi-type interactions for accurate RGBT SOD. In specific, we first encode two modalities into multi-level multi-modal feature representations. Then, we design a novel dual-decoder to conduct the interactions of multi-level features, two modalities and global contexts. With these interactions, our method works well in diversely challenging scenarios even in the presence of invalid modality. Finally, we carry out extensive experiments on public RGBT and RGBD SOD datasets, and the results show that the proposed method achieves the outstanding performance against state-of-the-art algorithms. The source code has been released at: https://github.com/lz118/Multi-interactive-Dual-decoder.