Pyramid Pixel Context Adaption Network for Medical Image Classification With Supervised Contrastive Learning.

Spatial attention (SA) mechanism has been widely incorporated into deep neural networks (DNNs), significantly lifting the performance in computer vision tasks via long-range dependency modeling. However, it may perform poorly in medical image analysis. Unfortunately, the existing efforts are often unaware that long-range dependency modeling has limitations in highlighting subtle lesion regions. To overcome this limitation, we propose a practical yet lightweight architectural unit, pyramid pixel context adaption (PPCA) module, which exploits multiscale pixel context information to recalibrate pixel position in a pixel-independent manner dynamically. PPCA first applies a well-designed cross-channel pyramid pooling (CCPP) to aggregate multiscale pixel context information, then eliminates the inconsistency among them by the well-designed pixel normalization (PN), and finally estimates per pixel attention weight via a pixel context integration. By embedding PPCA into a DNN with negligible overhead, the PPCA network (PPCANet) is developed for medical image classification. In addition, we introduce supervised contrastive learning to enhance feature representation by exploiting the potential of label information via supervised contrastive loss (CL). The extensive experiments on six medical image datasets show that the PPCANet outperforms state-of-the-art (SOTA) attention-based networks and recent DNNs. We also provide visual analysis and ablation study to explain the behavior of PPCANet in the decision-making process.

Multi-style spatial attention module for cortical cataract classification in AS-OCT image with supervised contrastive learning.

BACKGROUND AND OBJECTIVE: Precise cortical cataract (CC) classification plays a significant role in early cataract intervention and surgery. Anterior segment optical coherence tomography (AS-OCT) images have shown excellent potential in cataract diagnosis. However, due to the complex opacity distributions of CC, automatic AS-OCT-based CC classification has been rarely studied. In this paper, we aim to explore the opacity distribution characteristics of CC as clinical priori to enhance the representational capability of deep convolutional neural networks (CNNs) in CC classification tasks. METHODS: We propose a novel architectural unit, Multi-style Spatial Attention module (MSSA), which recalibrates intermediate feature maps by exploiting diverse clinical contexts. MSSA first extracts the clinical style context features with Group-wise Style Pooling (GSP), then refines the clinical style context features with Local Transform (LT), and finally executes group-wise feature map recalibration via Style Feature Recalibration (SFR). MSSA can be easily integrated into modern CNNs with negligible overhead. RESULTS: The extensive experiments on a CASIA2 AS-OCT dataset and two public ophthalmic datasets demonstrate the superiority of MSSA over state-of-the-art attention methods. The visualization analysis and ablation study are conducted to improve the explainability of MSSA in the decision-making process. CONCLUSIONS: Our proposed MSSANet utilized the opacity distribution characteristics of CC to enhance the representational power and explainability of deep convolutional neural network (CNN) and improve the CC classification performance. Our proposed method has the potential in the early clinical CC diagnosis.

Attention to region: Region-based integration-and-recalibration networks for nuclear cataract classification using AS-OCT images.

Nuclear cataract (NC) is a leading eye disease for blindness and vision impairment globally. Accurate and objective NC grading/classification is essential for clinically early intervention and cataract surgery planning. Anterior segment optical coherence tomography (AS-OCT) images are capable of capturing the nucleus region clearly and measuring the opacity of NC quantitatively. Recently, clinical research has suggested that the opacity correlation and repeatability between NC severity levels and the average nucleus density on AS-OCT images is high with the interclass and intraclass analysis. Moreover, clinical research has suggested that opacity distribution is uneven on the nucleus region, indicating that the opacities from different nucleus regions may play different roles in NC diagnosis. Motivated by the clinical priors, this paper proposes a simple yet effective region-based integration-and-recalibration attention (RIR), which integrates multiple feature map region representations and recalibrates the weights of each region via softmax attention adaptively. This region recalibration strategy enables the network to focus on high contribution region representations and suppress less useful ones. We combine the RIR block with the residual block to form a Residual-RIR module, and then a sequence of Residual-RIR modules are stacked to a deep network named region-based integration-and-recalibration network (RIR-Net), to predict NC severity levels automatically. The experiments on a clinical AS-OCT image dataset and two OCT datasets demonstrate that our method outperforms strong baselines and previous state-of-the-art methods. Furthermore, attention weight visualization analysis and ablation studies verify the capability of our RIR-Net for adjusting the relative importance of different regions in feature maps dynamically, agreeing with the clinical research.

Mixed pyramid attention network for nuclear cataract classification based on anterior segment OCT images.

Nuclear cataract (NC) is a leading ocular disease globally for blindness and vision impairment. NC patients can improve their vision through cataract surgery or slow the opacity development with early intervention. Anterior segment optical coherence tomography (AS-OCT) image is an emerging ophthalmic image type, which can clearly observe the whole lens structure. Recently, clinicians have been increasingly studying the correlation between NC severity levels and clinical features from the nucleus region on AS-OCT images, and the results suggested the correlation is strong. However, automatic NC classification research based on AS-OCT images has rarely been studied. This paper presents a novel mixed pyramid attention network (MPANet) to classify NC severity levels on AS-OCT images automatically. In the MPANet, we design a novel mixed pyramid attention (MPA) block, which first applies the group convolution method to enhance the feature representation difference of feature maps and then construct a mixed pyramid pooling structure to extract local-global feature representations and different feature representation types simultaneously. We conduct extensive experiments on a clinical AS-OCT image dataset and a public OCT dataset to evaluate the effectiveness of our method. The results demonstrate that our method achieves competitive classification performance through comparisons to state-of-the-art methods and previous works. Moreover, this paper also uses the class activation mapping (CAM) technique to improve our method's interpretability of classification results.

Adaptive feature squeeze network for nuclear cataract classification in AS-OCT image.

Nuclear cataract (NC) is an age-related cataract disease. Cataract surgery is an effective method to improve the vision and life quality of NC patients. Anterior segment optical coherence tomography (AS-OCT) images are noninvasive, reproductive, and easy-measured, which can capture opacity clearly on the lens nucleus region. However, automatic AS-OCT-based NC classification research has not been extensively studied. This paper proposes a novel convolutional neural network (CNN) framework named Adaptive Feature Squeeze Network (AFSNet) to classify NC severity levels automatically. In the AFSNet, we construct an adaptive feature squeeze module to dynamically squeeze local feature representations and update the relative importance of global feature representations, which is comprised of a squeeze block and a global adaptive pooling operation. We conduct comprehensive experiments on a clinical AS-OCT image dataset and a public OCT images dataset, and results demonstrate our method's effectiveness and superiority over strong baselines and previous state-of-the-art methods. Furthermore, this paper also demonstrates that CNNs achieve better NC classification results on the nucleus region than the lens region. We also adopt the class activation mapping (CAM) technique to localize the discriminative regions that CNN models learned, which enhances the interpretability of classification results.